Antivirus Systems: Detection Methods and Architectures

Abstract

Antivirus systems have evolved from static pattern matchers into complex algorithmic ecosystems that encapsulate the broader logic of modern cybersecurity. This review deconstructs their internal architecture, tracing the transition from deterministic string-matching automata to probabilistic, behavioral, and cloud-assisted paradigms. Foundational modules such as scanners, heuristic analyzers, behavioral monitors, and sandbox environments operate as interconnected computational strata, forming adaptive feedback loops that mirror principles of distributed intelligence. Signature-based methods, such as Aho-Corasick, Boyer-Moore, and Wu-Manber, remain core to real-time filtering, while probabilistic reasoning through Bayesian inference, Markov modeling, and Hidden Markov Models extends detection to polymorphic and metamorphic threats. Behavioral analysis, empowered by Support Vector Machines, deep neural architectures, and temporal models, enables semantic inference over system-call graphs and runtime telemetry. Moreover, cloud-assisted frameworks integrate federated learning and global reputation graphs, which transform detection into a collective intelligence process.

1. Introduction

Malware activity continues to expand in both scale and sophistication, driven by automated toolchains, polymorphic code generation, and adversarial adaptation [1,2,3,4,5,6,7,8,9,10]. Recent large-scale threat intelligence reports document sustained year-over-year growth in malware samples, attack campaigns, and financially motivated intrusions, with global economic losses from cyberattacks reaching hundreds of billions of dollars annually [11,12,13,14]. Industry analyses further indicate that modern malware increasingly targets cloud infrastructures, software supply chains, and heterogeneous endpoint ecosystems, thereby amplifying systemic risk across interconnected digital environments [15,16,17]. These trends confirm that antivirus systems remain a foundational component of contemporary cybersecurity, not as static defensive tools, but as adaptive algorithmic infrastructures operating under continuous adversarial pressure [1]. The architecture of antivirus systems reflects the algorithmic evolution of cybersecurity itself, which transitioned from deterministic pattern scanning to dynamic, learning-based defense mechanisms [1,2]. Early antivirus software operated under static paradigms, which relied exclusively on deterministic signature matching that mirrored the algorithmic simplicity of lexical pattern search engines [1,2]. As malware complexity increased through polymorphism, packing, and obfuscation, static detection became insufficient to maintain detection coverage, since exact byte-level signatures were invalidated by code mutation and packing techniques [1,4]. This limitation motivated the adoption of probabilistic and heuristic models capable of generalization beyond known signatures [1,3,5]. Such a shift marked the emergence of antivirus systems as algorithmic ecosystems. These ecosystems consist of multi-layered detection architectures built from deterministic and probabilistic modules. They fuse distinct computational disciplines, such as automata theory, statistical inference, and machine learning, within a unified defense framework [1,5]. In modern times, antivirus engines embody multi-layered integration, where deterministic modules (e.g., Aho–Corasick pattern matchers) coexist with stochastic classifiers, dynamic behavior profilers, and sandboxed execution environments [1,6]. These components communicate through structured data channels, hash indices, control flow graphs, and feature vectors. Together, they form an architecture that mirrors distributed intelligence in larger cyber defense infrastructures [7,8]. The study of antivirus structures thus extends beyond software engineering. These systems constitute algorithmic microcosms of adaptive security systems. They derive resilience from redundancy, parallelism, and feedback-driven updates [1,5,6]. Thus, this review seeks to delineate the algorithmic strata of antivirus architectures through a discussion of their internal mechanisms according to computational function, operational hierarchy, and capacity for adaptation [9]. The alignment of these strata (i.e., signature-based, heuristic, behavioral, and cloud-assisted) with broader algorithmic paradigms demonstrates how antivirus architectures evolved into feedback-controlled detection systems that balance precision and generalization under adversarial pressure [10].

Methodological Scope of the Review

This review adopts a structured narrative methodology. Its aim is to synthesize the internal architecture and algorithmic evolution of antivirus systems rather than to perform a quantitative meta-analysis. Source selection was guided by architectural relevance, algorithmic impact, and influence on subsequent detection paradigms, with emphasis on foundational contributions and representative modern extensions. The literature corpus was constructed through targeted searches in major scientific databases, including IEEE Xplore, ACM Digital Library, SpringerLink, ScienceDirect, and arXiv. We also used authoritative technical reports when architectural practices were not sufficiently documented in academic venues. Works were included if they introduced, formalized, or systematized core detection mechanisms. These mechanisms included signature-based, heuristic, behavioral, and cloud-assisted detection. Purely operational reports and non-technical descriptions were excluded. The temporal scope spans early signature-based systems to contemporary cloud and federated architectures. The analysis proceeds through a comparative architectural synthesis. Each contribution is assigned to functional layers and linked to major shifts in antivirus detection methods over time. This structure ensures transparency and conceptual reproducibility throughout the review. No fixed benchmark protocol is defined. Any future empirical assessment of the architectural components discussed here would require representative static malware corpora, dynamic execution trace datasets, and telemetry datasets. The search process covered works published between 2000 and 2025, with earlier sources used where necessary for historical context. Example search strings included “antivirus architecture”, “malware detection review”, “signature based malware detection”, “heuristic malware detection”, “behavior based malware detection”, “sandbox malware analysis”, and “cloud assisted malware detection”. Studies were included when they addressed a detection layer, a core algorithm, or an architectural component relevant to antivirus systems. Studies were excluded when they were purely operational reports, short news items, non-technical summaries, or domain specific works with no clear architectural relevance. The initial search yielded about 230 records. After title and abstract review, about 120 records remained. After full-text assessment, 77 references were retained. Studies were assigned to layers according to their dominant functional role within the antivirus pipeline.

Unlike method-centered surveys, the present review adopts an architecture-first perspective and examines antivirus systems as layered, feedback-driven detection architectures. The contribution of this review is threefold. First, it systematizes the evolution of antivirus systems through an architectural perspective. This perspective unifies deterministic, heuristic, behavioral, and cloud-assisted paradigms. Secondly, it provides a comparative synthesis of detection layers, highlighting their functional roles, limitations, and interactions under adversarial pressure. Third, it identifies emerging architectural trends and unresolved challenges that shape future antivirus research. The remainder of the manuscript is organized as follows: Section 2 outlines the foundational architecture of antivirus systems; Section 3, Section 4 and Section 5 analyze signature-based, heuristic, behavioral, and learning-based detection paradigms; Section 6 and Section 7 examine cloud-assisted and hardware-accelerated architectures; Section 8 discusses emerging directions; and Section 9 concludes with a synthesized outlook.

2. Foundational Architecture of Antivirus Systems

The internal architecture of antivirus systems consists of a hierarchy of computational modules (see Supplementary Materials). Each module has a distinct algorithmic role, and together they approximate an adaptive decision engine (Figure 1) [1,18,19]. Antivirus homeostasis arises from the interaction of layered modules rather than from any single detector in isolation (Figure 1). The scanner acts as the ingestion and traversal module. It forwards binary streams toward deterministic signature filters. Archive unpacking and preselection mechanisms reduce unnecessary pattern evaluations. Bloom filter-based prefilters discard non-candidate substrings before automaton traversal, thereby limiting computational overhead [20]. Above this layer, heuristic and behavioral modules operate on progressively abstract representations, feeding their outputs into a centralized decision logic that determines remediation actions. Continuous feedback between monitoring components and the update engine closes the adaptation loop, enabling real-time synchronization with global threat intelligence [6,21]. At the foundation lies the scanner, a deterministic parser. It traverses memory and storage segments through hash indexing, block-level mapping, and recursive unpacking of executable structures [1,22].

This component acts as the I/O interface for the pattern matching layer. The pattern matching layer directs binary data toward the signature database. Compressed finite automata or trie structures in that database encode known byte sequences and instruction patterns for deterministic recognition [23]. The signature database represents the curated repository of known malicious patterns and the primary scalability bottleneck of the antivirus engine. Modern implementations utilize Aho-Corasick automatons or Wu-Manber filters, optimized through Bloom hashing and block compression, to ensure sublinear lookup times across millions of signatures [24,25]. Updates are incremental and delta-compressed, minimizing network load and reducing synchronization latency across distributed clients [26].

2.1. Signature Corpus Maintenance

Operational deployment requires continuous curation of the signature corpus [1,22]. Vendors maintain an active malware bank whose entries show recent field prevalence, as indicated by endpoint telemetry and laboratory intake [1,26]. Signatures and families with no field presence over a defined horizon (e.g., 6–12 months) move from the active bank to an archive used for reproducible regression tests [1]. The active bank draws on recent field samples, and signature patterns are regenerated when a family returns under new packing or obfuscation [1]. For heuristic and behavioral modules, periodic threshold recalibration on recent clean software, plus false alert audits, prevents silent degradation as benign software and attacker tactics evolve. This maintenance cycle keeps the signature database aligned with current threats while preserving historical coverage for verification [1,22,26].

2.2. Score Threshold Maintenance

Drift control uses feedback from field alert logs collected by antivirus clients and from later corrections after manual review of flagged files (i.e., cases initially flagged as suspicious but later confirmed as clean) [1,22]. Drift affects score-based detection, not exact signature matches [1]. Heuristic and behavioral modules output a suspicion score [18,19]. ML (machine learning)-based behavioral detectors output a number for each file or process, where larger values indicate higher suspicion. An alert occurs only when this number exceeds a chosen score limit. The score limit is set by testing the detector on a verified clean set of files and applications that represent what users run in the current period. The score limit is raised or lowered until the number of false alerts on this clean set stays under a stated limit, for example, at most 1 false alert per 10,000 clean files, or at most a fixed number of alerts per day across clients. Scores close to the score limit do not trigger an immediate verdict; they trigger a second check, such as sandbox execution or cloud reputation, to reduce false alerts. The same score limit rule applies to heuristic scoring modules: the antivirus engine raises an alert only when the score exceeds a score limit set in the configuration (vendor default or administrator setting) [1,22]. Antivirus client logs record which files trigger alerts [1]. Some of these files are later confirmed as clean after review, for example, via user reports and analyst confirmation [1,22]. If the number of false alerts in the current update window is higher than in the prior update window, the score limit is increased so that an alert requires a higher threshold [1]. The increase is not arbitrary: the score limit is raised until the false alert rate measured on confirmed clean files from the current window (for example, the last 30–90 days) drops back below the stated limit [1]. The adjusted score limit then stays fixed for the next update window [1]. False alerts are measured on a separate clean set collected later and not used during the reset step, for example, the next 30–90 days. This check verifies that the false alert rate is back below the stated limit.

2.3. Cross-Layer Decision Pipeline

Each signature entry acts as a decision point within the recognition structure [27]. The next step depends on two things: the byte pattern itself and its position within the file or data stream (i.e., contextual offsets) [27]. Above the deterministic layer, the heuristic analyzer introduces probabilistic reasoning [28]. It assigns suspicion scores through static code metrics such as opcode frequency, import table irregularity, and entropy deviation from benign binaries [28]. These metrics feed into heuristic trees or Bayesian classifiers [29]. The result is an estimate of malicious intent and detection of previously unseen polymorphic variants [29]. The behavioral monitor extends this logic temporally because it captures API (Application Programming Interface) calls, registry writes, and network I/O patterns during execution to reconstruct dynamic behavioral signatures in a multidimensional feature space [30,31]. Such behavioral fingerprints are processed through graph-based classifiers or Hidden Markov Models to identify anomalous control flows consistent with malicious persistence [29]. Moreover, the sandbox functions as a high-fidelity emulation layer. It isolates execution and captures instruction traces, stack manipulations, and inter-process communications [32]. It provides ground-truth behavioral data that refines heuristic thresholds and trains machine learning modules embedded within the monitor. Meanwhile, the update engine sustains the evolutionary feedback loop of the antivirus ecosystem, which allows the synchronization of local detection models with cloud-assisted telemetry to ensure global coherence in response to emerging threats [33]. Information flow between these modules is both hierarchical and cyclical [18,34]. Deterministic scanners feed heuristic analyzers. Behavioral monitors supply feature updates to the database. Sandboxes validate probabilistic predictions. The update engine then integrates global intelligence back into local signatures [18,34]. In enterprise settings, false positive management also depends on policy controls, not only on score adjustment. Common measures include allow lists for trusted software, exceptions for known vendors, and exclusions tied to approved paths, hashes, or administrative tools (e.g., PowerShell, backup agents, or remote management utilities). These controls matter because a strict alert rule can block normal business activity, such as software deployment, maintenance, or internal automation. For this reason, large organizations often prefer false positives that can be predicted and managed, rather than maximum detector sensitivity, since analyst time, user interruption, and service disruption all carry direct costs. In operational terms, these layers do not act as independent judges, but as staged filters whose outputs are fused within a common decision path [18,19,34]. Exact signature matches (e.g., a direct byte pattern or hash match) usually carry the highest confidence and can trigger immediate action, whereas heuristic and behavioral scores contribute weighted suspicion rather than final verdicts on their own (i.e., a score increase rather than an automatic quarantine decision) [18,19]. When signals conflict, the engine can defer the case to a higher cost layer, such as sandbox analysis or cloud reputation, instead of issuing an immediate alert (e.g., a suspicious static score followed by sandbox execution or a cloud-side reputation check) [26,32,33]. Alert triage further reduces analyst overload by reserving manual review for uncertain, high-risk, or recurrent cases (e.g., repeated alerts on the same file family or alerts on privileged system processes), while routine low-confidence events are suppressed, grouped, or resolved automatically through prior reputation and rule-based policy (e.g., allow lists, trusted publisher status, or duplicate alert collapse) [21,26,34]. This interdependence transforms the antivirus engine into a feedback-driven detection architecture, where redundancy and adaptation maintain detection integrity (Figure 1) [18].

The stratified architecture of antivirus systems did not emerge as a design preference, but as a response to fundamental computational constraints. Early monolithic scanners remained effective for the threat landscape of their time, but they became harder to maintain as signature databases grew and input throughput increased [35]. Part of this pressure came from cumulative signature growth, especially when older malware families remained in the active database rather than moved to archive status. The problem was amplified by the slower pace of software and hardware change in earlier periods, which often gave malware families a longer operational lifetime and extended the period during which old signatures remained relevant. Nevertheless, deterministic scanners provide predictable latency, but they lose part of their effectiveness under polymorphism [28,29]. Segmented hexadecimal signatures can still detect many polymorphic malware samples, even when exact whole pattern matches fail. This architectural decomposition reflects a systemic trade-off between throughput, adaptability, and interpretability, where no single layer can independently satisfy real-time constraints, detection coverage, and robustness against evasion.

3. Signature-Based Detection

Signature-based detection remains the algorithmic backbone of antivirus systems. It rests on the principle that deterministic recognition of known byte patterns enables low-latency filtering of malicious code at scale (

Table 1. Signature search methods. Aho-Corasick has linear scan cost, medium build cost, and high memory use, and it suits multi-pattern exact matches. Boyer-Moore has sublinear average scan cost, low build cost, and low memory use, and it suits long literal searches. Wu-Manber has sublinear average scan cost, medium build cost, and medium memory use, and it suits large signature sets. Bloom filter prefilters have a linear scan plus a verify step, low build cost, and low memory use, and they suit fast rejection of non-candidates before an exact match. The chunk hash index has a near linear scan cost, a medium build cost, and medium memory use, and it suits partial block matches.

Method	Scan	Build	Memory	Best Use
Aho-Corasick	linear	medium	high	many patterns
Boyer-Moore	sublinear (avg)	low	low	long literals
Wu-Manber	sublinear (avg)	medium	medium	large sets
Bloom filter prefilter	linear + verify	low	low	fast reject
Chunk hash index	near linear	medium	medium	partial blocks

linear = O(n), sublinear (avg) = typical < O(n), near linear = ~O(n), verify = secondary exact check.

) [3]. The Aho-Corasick algorithm, conceived as a linear-time automaton for multi-pattern search, became the canonical core of scanning engines due to its ability to traverse text or binary streams with constant amortized complexity independent of the number of signatures (Table 1) [2,23].

Its finite-state machine representation, optimized through transition compression and failure-link caching, allows antivirus scanners to identify thousands of signatures in a single pass while maintaining cache coherence and predictable memory access [36]. Complementary implementations such as the Boyer-Moore algorithm use backward pattern analysis and skip heuristics to reduce unnecessary comparisons [37]. This works especially well for long, distinctive substrings characteristic of executable headers and polymorphic stubs (i.e., core code that decrypts malware) (Figure 2 and Table 1) [37]. Signature matching progresses from strict exact recognition toward broader but less interpretable forms of detection as matching constraints relax (Figure 2). For instance, the upper region-match layer applies strict cryptographic or checksum-based signatures, which yield high confidence for non-polymorphic malware but fail under code mutation. The intermediate chunk-match layer tolerates localized variation through hexadecimal or block-level signatures, thereby extending coverage to mildly obfuscated binaries. The lowest frequency match layer abandons fixed patterns altogether. It relies on statistical or heuristic signatures that capture distributional properties of code. As a result, it can generalize across polymorphic and metamorphic families. This hierarchy formalizes the transition from exact recognition to probabilistic inference within signature engines. Wu-Manber extends this logic through block hashing and shift tables [24,38]. It achieves sublinear average time complexity by collapsing frequent byte sequences into hashed keys. This approach underpins modern deep packet inspection and inline antivirus gateways [24,38]. In order to counteract the explosion of signature sets, modern antivirus systems employ hybrid indexing architectures that integrate deterministic tries with probabilistic filters [20,39].

Bloom filters are compact probabilistic membership structures. They prefilter candidate substrings through multiple hash functions. The result is a sharp reduction in automaton traversal frequency with controllable false positive rates (Figure 1) [20]. Hash chaining and locality-sensitive hashing further accelerate pattern verification in large-scale signature databases distributed across cloud-assisted backends [40]. Compression of trie-based signature trees through shared prefix encoding and vectorized traversal reduces memory overhead. It also improves SIMD (Single Instruction, Multiple Data) parallelism during scanning. This allows throughput suitable for kernel-level or gateway deployments [41]. Real-time scanning imposes strict performance constraints. Antivirus engines must therefore balance detection accuracy against latency under concurrent I/O operations. Hardware acceleration through GPU-based parallel pattern matching and FPGA (Field Programmable Gate Array) pipelines has become a central optimization strategy. It supports multi-gigabit per second inspection of data streams without loss of detection coverage [42]. Multithreaded scheduling can work together with lock-free buffer management (i.e., buffer access without thread blocking from explicit locks) in multiprocessor environments. This combination helps the detection engine to preserve scalability even when signature density is high. In such conditions, scan cost can remain close to linear with respect to input size [43]. In effect, signature-based detection persists not as a legacy subsystem but as a mathematically mature filter that defines the first algorithmic boundary between benign and malicious computation (Figure 2) [1]. Nevertheless, the gradual relaxation of signature constraints illustrates a fundamental trade-off between precision and resilience. Exact signatures achieve near-zero false positives but are brittle under polymorphism and code packing. Chunk-based and frequency-based signatures increase coverage at the cost of higher false-positive rates and reduced interpretability. The expansion of signature databases further introduces memory pressure and cache inefficiency, which directly motivated the integration of probabilistic prefilters and hardware acceleration. Thus, signature-based detection evolved not due to algorithmic obsolescence, but because adversarial pressure shifted the optimization frontier from exactness toward scalability and robustness.

4. Heuristic and Statistical Analysis

Heuristic detection emerged as the algorithmic bridge between deterministic signature matching and adaptive behavioral analysis. It introduced probabilistic reasoning into the traditionally discrete decision logic of antivirus engines [3]. Two forms of heuristic detection should be distinguished. Rule-based heuristics rely on manually engineered conditions and thresholds, such as entropy limits, suspicious import patterns (e.g., unusual use of VirtualAlloc, WriteProcessMemory, or CreateRemoteThread), malformed headers, or unusual instruction sequences selected by analysts from prior malware knowledge [18,28]. Statistical and machine learning heuristics, by contrast, derive their decision rules from data, either through estimated feature distributions, fitted model parameters, or learned classification boundaries built from labeled benign and malicious samples (e.g., classifiers trained on opcode frequencies, n-gram profiles, API call statistics, or section-level metadata) [19,29,44]. In practice, antivirus systems often combine both forms: expert-defined rules provide transparent early filters (e.g., a score increase for packed sections or for rare import combinations), while data-driven models extend coverage beyond fixed expert criteria (e.g., Bayesian scoring, support vector machines, or learned sequence models). This distinction is important because manually engineered thresholds reflect analyst judgment, whereas learned thresholds and decision boundaries depend on the composition, age, and representativeness of the training data. Unlike signature systems that rely on byte-exact correspondences, heuristic analyzers operate through feature abstraction, extracting statistical and structural attributes such as instruction entropy, import-table anomalies, and API frequency distributions to infer malicious intent [5,18]. Each attribute is weighted through heuristic scoring functions, which map deviations from normative baselines into suspicion indices calibrated by empirical frequency data drawn from benign and malicious corpora [6,21]. Code emulation serves as the execution substrate of this layer. It lets antivirus systems interpret machine instructions inside a controlled virtual CPU (Central Processing Unit). This exposes latent behavior that packing or encryption can hide [16,32]. Such emulation also faces anti-analysis evasion. Many samples test for virtual machine artifacts and analysis hooks via hardware identifiers, device drivers, timing checks, or absent user activity. If such checks succeed, the sample can delay execution or follow a benign path during analysis, which reduces behavior visibility and can bias score-based decisions. This motivates multi-environment analysis, artifact reduction, and trigger stimulation as standard countermeasures in dynamic pipelines [22,32]. Emulated instruction traces yield dynamic feature sets, such as memory writes to executable regions, self-modifying code segments, and recursive decryption loops, high-entropy indicators of polymorphic or metamorphic transformations (Figure 2) [8,32]. Such dynamic indicators feed probabilistic classifiers [26,43]. These classifiers compute posterior infection likelihoods from conditional dependencies across observed features [26,43]. Naïve Bayes classifiers, despite their simplicity, remain effective for heuristic detection due to their independence assumption, which permits rapid computation of posterior probabilities across large feature spaces without matrix inversion or parameter-estimation overhead [26]. More expressive models, such as discrete-time Markov chains, capture sequential dependencies within opcode transitions or system call sequences [28,44]. This allows quantification of behavioral predictability and identification of stochastic deviations consistent with obfuscation or mutation [28,44]. Hidden Markov Models further extend this framework. They infer latent behavioral states and support classification even when observable features remain partially masked by runtime polymorphism [14,45]. Detection of polymorphism and metamorphism thus reduces to recognition of statistical inconsistencies between expected and observed code grammars [13]. Polymorphic malware alters surface-level representations through changes in encryption keys, NOP padding, and instruction reordering. At the same time, it preserves semantic invariants that can still be detected through opcode n-gram entropy analysis and invariant subgraph extraction (i.e., stable graph fragments that remain present across variants) [29,46]. Metamorphic strains, by contrast, restructure their entire syntactic form. This requires statistical normalization techniques such as opcode canonicalization (i.e., conversion to a common opcode form) and control flow graph alignment in order to recover invariant behavioral signatures [7,27,30]. Heuristic detection introduces probabilistic reasoning to overcome the rigidity of signatures, but this flexibility comes at the cost of uncertainty and threshold sensitivity. Statistical models depend heavily on feature selection and calibration, exposing them to concept drift and adversarial manipulation. Thus, while Bayesian and Markovian models capture structural regularities, they struggle with long-range dependencies and high-dimensional feature interactions. Consequently, heuristic systems improve generalization but trade deterministic guarantees for probabilistic confidence, requiring continuous retraining and careful threshold management to control false-positive escalation [1].

5. Behavioral and Anomaly-Based Detection

Behavioral detection transcends the static limits of syntactic inspection by analyzing the semantics of program execution, namely how software behaves rather than how it appears [7]. The central construct of this paradigm is system call monitoring, in which sequences of kernel or API invocations form temporal signatures that reflect the functional behavior of code [31]. Antivirus engines instrument execution through API hooks and kernel-level interception to capture low-level interactions with the operating system, file system, and network stack. These actions reconstruct behavioral graphs that describe causal relations between events [32]. These event graphs become algorithmic fingerprints whose topology distinguishes legitimate software from malicious actors seeking persistence or privilege escalation [33]. Anomaly-based detectors interpret such behavioral traces within statistical or machine learning frameworks. They first define a model of normal behavior from benign samples. They then treat deviations from that model as potential intrusions [34]. Early classifiers relied on distance metrics and clustering of feature vectors derived from system-call frequencies and arguments, while later models adopted discriminative learning paradigms capable of nonlinear decision boundaries [35]. Support Vector Machines project behavioral features into high-dimensional spaces and construct hyperplanes that maximize separation between benign and malicious activity [26]. In contrast, a decision tree splits the data step by step by asking which feature best separates benign files from malicious ones at each branch [36]. A Random Forest applies the same idea many times through many separate trees, then combines their votes into one final decision. The result is a set of explicit “if then” rules that a human analyst can read. These rules can capture stable behavior patterns (i.e., repeated actions such as file writes, registry changes, process injection, or network contact that remain present across malware variants). [36]. Deep learning architectures introduced hierarchical feature abstraction, removed the need for manual feature design, and enabled models to learn behavioral semantics directly from raw event sequences [37]. Convolutional Neural Networks (CNNs) capture spatial correlations within feature matrices derived from system-activity logs, effectively recognizing structural regularities of malicious workflows [38]. Autoencoders perform unsupervised reconstruction of normal behavior, marking high reconstruction error as an anomaly indicator, a concept that underpins many self-learning antivirus modules [39]. These architectures enable detection of obfuscated malware families whose surface-level traces have diverged beyond the recognition capacity of traditional models. Temporal and sequence-aware methods extend this paradigm by explicitly modeling the order and timing of events. Hidden Markov Models represent program execution as a stochastic process of hidden behavioral states and estimate transition probabilities that encode typical sequences of operations [14]. Recurrent Neural Networks (RNNs), in particular Long Short-Term Memory variants, preserve temporal dependencies across extended execution windows. Such models capture complex persistence strategies and delayed payload activation [41,47,48]. Delayed activation is a direct evasion tactic against short dynamic runs. The sample postpones malicious actions via long sleeps, slow loops, or explicit triggers such as time, user input, or a remote command. As a result, a limited analysis budget can end before the payload appears, which yields incomplete traces and underestimates risk. Practical pipelines address this via sleep skipping, trigger forcing, and longer or staged runs for suspicious cases. When integrated within cloud-assisted detection ecosystems, these models continuously retrain on telemetry streams, evolving their decision boundaries to counter adversarial drift in malware behavior [42,49]. Thus, behavioral and anomaly-based detection embodies the transition of antivirus logic from fixed pattern recognition toward adaptive inference, transforming static protection engines into continuously learning cyber-organisms [1]. The integration of machine learning into antivirus engines introduces critical trade-offs between model complexity and endpoint feasibility. Deep architectures improve representational power but impose increased computational load, memory footprint, and energy consumption, directly affecting latency and user experience on constrained endpoints [50]. Moreover, adversarial robustness and generalization constitute conflicting objectives: models hardened against adversarial perturbations often lose sensitivity to novel malware variants, while highly generalizable models exhibit fragile decision boundaries exploitable through evasion attacks [51]. Furthermore, despite their expressive power, machine learning-based detectors introduce new attack surfaces. Thus, behavioral classifiers remain vulnerable to adversarial evasion [52]. Main routes include mimicry attacks, poisoning of training data, and perturbation of runtime telemetry [52]. Mimicry targets behavior-based scoring and forces the trace to resemble normal software. The sample can add benign-like actions, reorder operations, or pad activity so that event statistics and sequences match the baseline. This can lower the suspicion score while malicious capability remains. The risk is highest when the detector relies on simple event counts or short windows (i.e., how many file writes, registry edits, or network connections occur within a brief interval) rather than on task semantics and long-range dependencies [51]. In practice, antivirus systems often place a deterministic filter (i.e., an exact rule or signature-based check) before the machine learning stage. Clear cases can then be resolved early (e.g., an exact signature match may trigger immediate action), while only ambiguous files are passed to the classifier. Protection also depends on confidence calibration. In this setting, the model score is not treated as a final yes or no decision. A score near the alert limit is treated as uncertain and can trigger a second check instead of an immediate verdict. A further safeguard compares the outputs of more than one detector. If one model flags the file but another remains uncertain, the case is treated as unresolved and sent to a higher validation stage. Thus, rather than accept one model output as final, the engine can defer the case to sandbox or cloud validation. Moreover, deep architectures also suffer from opacity and limited explainability, which complicates forensic validation and error analysis [53].

6. Cloud-Assisted and Collective Intelligence

The move of antivirus intelligence from isolated endpoints to cloud-centric infrastructures transformed detection into a distributed algorithmic process [43,54]. That process now depends on collective telemetry and federated learning [43,54]. In this paradigm, local agents act as sensors that extract feature vectors, cryptographic hashes, and behavioral metadata from inspected binaries. These feature vectors are then transmitted as anonymized signatures to centralized analytic backends for global model refinement (Figure 3) [17,42]. Cloud-assisted detection extends local verdicts into collective inference through aggregated reputation and telemetry signals (Figure 3). Individual clients contribute reputation signals derived from local classification confidence, behavioral observations, or historical trust scores. These signals are aggregated through weighted voting mechanisms that determine global maliciousness assessments for files, IP addresses, or URLs. The same consensus logic extends from file-level reputation to network entities, and this enables one common decision process across heterogeneous data sources (i.e., files, IPs, URLs, and telemetry records). Such probabilistic aggregation limits noise from individual endpoints, which accelerates the zero-day recognition through collective evidence. This architecture transforms every protected node into a contributor to a large-scale inference network, where aggregated statistics amplify the capacity for early threat recognition [1].

Cloud-assisted detection introduces a fundamental trade-off between analytical power and privacy exposure. While centralized aggregation enables rapid zero-day recognition and collective inference, it requires continuous telemetry transfer that raises concerns regarding data minimization, user profiling, and regulatory compliance [55]. Practical antivirus systems therefore operate under constrained data-sharing regimes, balancing detection efficacy against latency, bandwidth consumption, and privacy guarantees.

Federated signature intelligence enables antivirus vendors to construct continuously evolving reputation graphs [52,54]. These graphs link file hashes, publisher certificates, and behavioral artifacts observed across heterogeneous user populations [52,54]. These graphs form the backbone of cloud-assisted reputation systems such as Microsoft Defender Cloud and Google Safe Browsing, where the trustworthiness of an object is inferred through its relational context rather than intrinsic content [44,56]. The exchange of hash data, when combined with telemetry feedback loops, allows near-instantaneous propagation of new threat identifiers across millions of endpoints and drastically reduces the mean time to detection for zero-day malware [9]. The analytical core of these systems integrates ensemble classifiers trained on petabyte-scale telemetry, employing gradient-boosted decision trees, graph embeddings, and deep learning architectures that continuously reweight features based on global frequency shifts [10,57]. Thus, feedback reinforcement transforms collective intelligence into an adaptive ecosystem, where each detection outcome from a client subtly adjusts the global threat model through probabilistic consensus mechanisms (Figure 3) [52]. Despite its efficiency, the cloud-assisted paradigm introduces intrinsic privacy and latency dilemmas. Data centralization increases the risk that attackers recover sensitive information from shared model outputs or telemetry. An attacker may, for example, infer whether a specific sample, user, or organization contributed data to the system. For this reason, antivirus systems may use privacy protection methods that hide or limit exposed information during model updates. Examples include differential privacy, which adds controlled random noise to mask origin while preserving overall pattern. Other methods include secure aggregation, which combines client updates without exposure of each one separately, and homomorphic encryption, which allows computation on encrypted data [51,58].

Also, latency constraints challenge real-time protection, as network dependence delays classification feedback; this is mitigated through hybrid caching, local pre-filtering, and edge-inference models that operate autonomously when disconnected [52,59]. This architecture fuses distributed intelligence with localized autonomy, pushing toward a self-optimizing continuum in which antivirus systems function as nodes in a global, semi-autonomous defense organism [44]. Thus, the solutions toward cloud-assisted detection introduce structural dependencies that fundamentally alter the threat model. Network latency can delay response during fast-moving attacks, while bandwidth consumption imposes scalability constraints in high-throughput environments. Centralized telemetry aggregation raises privacy concerns, and this includes inference leakage and regulatory compliance risks. Moreover, dependence on continuous Internet connectivity reduces resilience in offline or degraded-network scenarios. An additional crucial risk is that such federated reputation can also amplify error propagation. If an early verdict is wrong, the same false classification can spread across multiple systems through shared reputation signals, reinforce false positives, and unfairly penalize legitimate software. The risk becomes more serious when downstream systems reuse inherited reputation labels with limited independent verification. Such cascades can also affect legitimate security software, especially when early false labels propagate without later independent review.

7. Architecture Integration and Optimization

Modern antivirus solutions converge into hybrid architectures that integrate deterministic, probabilistic, and adaptive algorithms into unified detection pipelines capable of operating under adversarial uncertainty [1]. The fusion of signature-based recognition with heuristic inference and behavioral analytics establishes a multi-layered decision hierarchy in which each module contributes complementary evidence toward the classification of executable behavior [3,7]. Signatures provide high-confidence detection with negligible false positives, while heuristic and behavioral engines compensate for generalization and resistance to obfuscation, collectively forming an ensemble where inference propagates across algorithmic strata rather than existing in isolation [1,7,8]. At the architectural level, these pipelines operate as directed acyclic graphs of computation (i.e., processing paths that move forward from one stage to the next without circular return) [10,60]. One set of nodes extracts features. Another set performs pattern matching. A third set performs classification. These nodes exchange data through shared memory buffers and low-latency queues. The result is deterministic scheduling and bounded latency, that is, a known maximum delay for each decision, even under load [10,60]. At a higher operational scale, these integrated pipelines interface with broader antivirus infrastructures composed of client agents, web servers, and laboratory systems responsible for continuous update and signature propagation (Figure 4 and

Table 2. Cross-layer detection summary. Signature matching uses bytes and hashes and outputs an exact match; it is fast and precise, but brittle under mutation, with packing and polymorphism as common evasion. Heuristic scoring uses static features and outputs a suspicion score; it extends coverage beyond exact patterns, but it suffers drift and false alerts, with feature shift as common evasion. Behavioral scoring uses event and call traces and outputs a suspicion score; it provides runtime evidence, but it has higher cost and trace gaps, with mimicry and delay as common evasion. Sandbox analysis uses an instrumented run and outputs a trace and verdict; it provides deep visibility, but it is limited by time budget, with anti-analysis as a common evasion. Cloud reputation uses telemetry and graphs and outputs a reputation value; it adds global context, but it faces latency and privacy limits, with fast rotation and low prevalence as common evasion.

Layer	Main signal	Output	Strength	Weakness	Evasion
Signature matching	bytes, hashes	match	fast, precise	brittle	packing, polymorphism
Heuristic scoring	static features	score	broader coverage	drift, false alerts	feature shift
Behavioral scoring	events, calls	score	runtime evidence	cost, gaps	mimicry, delay
Sandbox analysis	instrumented run	trace + verdict	deep visibility	time budget	anti analysis
Cloud reputation	telemetry, graphs	reputation	global context	latency, privacy	fast rotation

). Update propagation, laboratory validation, and client feedback form a closed operational loop rather than a one-way delivery chain (Figure 4).

Endpoint clients perform continuous local scanning and submit suspicious artifacts or statistical summaries to a central web service (

Table 3. Representative antivirus system architectures by input, output, and primary role.

Architecture	Input	Output	Role
Endpoint antivirus	files, events, context	alerts, scores, telemetry	local protection
Gateway scanning	traffic, attachments, downloads	block or allow verdicts, alerts	stream inspection
Cloud reputation	hashes, metadata, client telemetry	reputation, verdicts, updates	global consensus

). Telemetry typically includes file hashes, alert labels, local score values, selected behavioral traces, and basic execution metadata such as process lineage, path, time, and host context. These records are aggregated at the central web service and forwarded, where necessary, to laboratory systems for deeper analysis and validation. Validation combines automated triage, duplicate removal, sandbox confirmation for selected cases, and analyst review for uncertain or high-impact samples. Once validated, updates are packaged as signature additions, heuristic rule changes, reputation updates, or model parameter revisions and are then distributed to endpoint clients through the regular update channel. If a new update produces false alerts or instability, the affected rules, signatures, or model revisions can be withdrawn and replaced by the last stable version. In other words, the web layer aggregates telemetry and forwards selected samples to laboratory environments, where automated analysis and signature extraction occur. Updated detection artifacts are redistributed to all clients, which creates a closed feedback loop between detection, analysis, and deployment. This architecture establishes the structural foundation upon which modern cloud-assisted and federated antivirus systems are built.

GPU and FPGA acceleration increasingly dominate the optimization frontier, which supports parallel pattern traversal and vectorized hashing for real-time inspection of multi-gigabit data streams [24,42]. GPUs execute thousands of concurrent threads that evaluate pattern matches and anomaly scores at the same time. The result is an order of magnitude improvement in throughput without loss of detection precision [61]. The FPGA pipelines, by contrast, implement deterministic automata such as Aho-Corasick or Wu-Manber directly in hardware logic, reducing control flow overhead and ensuring predictable timing for inline network deployments [23,24,62].

Cross-layer communication allows modules to adapt dynamically: heuristic analyzers recalibrate thresholds based on behavioral feedback, while cloud intelligence updates propagate feature-weight adjustments directly into local scanning heuristics [17,42]. This bidirectional data flow creates a feedback-controlled system where local misclassifications influence global retraining, effectively closing the loop between detection, learning, and adaptation [52]. Incremental model updates are disseminated through compressed deltas or quantized weight matrices. This approach reduces transmission cost and preserves synchronization between endpoint agents and central analytic servers (Figure 4) [51,63].

Such integration transforms antivirus engines from static filtering systems into continuously learning cybernetic frameworks [10]. In these frameworks, modular components evolve through feedback, redundancy, and computational acceleration [10,52]. The convergence toward hybrid antivirus architectures reflects the impossibility of optimization for all detection objectives within a single paradigm. Deterministic engines guarantee speed, probabilistic models ensure adaptability, and cloud intelligence enables global context, but each introduces irreducible limitations [1]. Architectural integration thus represents a systems-level compromise, where redundancy and cross-layer feedback compensate for the weaknesses of individual modules. Evaluation in antivirus research must extend beyond raw detection accuracy. Common measures include false positive rate, false negative rate, time to detect, and alert stability under repeated updates. Results also depend on dataset composition, since class imbalance, sampling bias, and outdated malware corpora can distort reported performance. Concept drift further degrades validity when models are tested on data that do not reflect current software and current attacker behavior. In real deployment, evaluation must also account for latency, CPU load, memory use, and, on constrained devices, energy cost, since a detector that is accurate in isolation may still be impractical under endpoint conditions. Trade offs also differ across deployment contexts. Endpoint protection must operate under tight limits on CPU use, memory footprint, I/O overhead, and user-visible latency. Gateway scanning can tolerate higher throughput-oriented compute costs, but it must sustain continuous stream inspection at line rate. Cloud-side analysis offers broader compute capacity and global context, but it introduces higher bandwidth demands, network delay, and privacy risk due to telemetry transfer and central aggregation. These measurable constraints help explain why modern antivirus systems distribute detection tasks across endpoint, gateway, and cloud layers rather than concentrate them in a single location (Table 3).

7.1. Empirical Evidence on Polymorphism and Cross Layer Trade Offs

Recent empirical work reports measurable loss of detection under polymorphic change and supports layer combination across signature rules, client telemetry, and commercial engines [64,65,66,67,68,69,70,71,72]. Large-scale Android measurement reports broad use of obfuscation in malicious apps and shows that benign apps also use obfuscation, which limits detectors that assume fixed obfuscator markers [73]. Studies that use API call records from sandbox analysis report improved detection with longer observation time, since delayed payload actions can appear late [74]. Graph representations that combine call context (i.e., which calls occur and how they connect) with library relations (i.e., which libraries a program uses and how they link) report strong accuracy and capture long-range dependencies that short observation windows miss [75].

7.2. Automaton Representation Under Scale Constraints

Polymorphism and obfuscation increase signature diversity and inflate rule sets, which shifts the bottleneck from matching theory to automaton representation and memory locality. Memory locality describes whether successive automaton steps read data from nearby memory addresses, so most reads come from the CPU cache rather than from RAM (Random Access Memory). Large rule sets rarely use a full dense DFA (Deterministic Finite Automaton), because state count and transition tables grow too fast. Practical scan engines use compressed transitions, sparse encodings, and split or layered automata that preserve deterministic traversal while reducing memory pressure. Hybrid pipelines keep a deterministic core for known patterns, then route uncertain cases to higher layers, a structure that matches modern antivirus scan designs and the Aho–Corasick deployment lineages reviewed in [76,77]. In practice, engines often mix NFA (Nondeterministic Finite Automaton) style fragments for complex signature rules with DFA fast paths for frequent literals. They combine literal prefilters with selective automaton steps. Such engines first reject most input via literal filters, then run a compact automaton only on the remaining candidates, often with compressed transitions and SIMD support, as in Hyperscan.

7.3. Adversarial Adaptation and Evasion Across Layers

Adversarial pressure affects each detection layer through distinct evasion routes. The signature layer is most exposed to packing, code obfuscation, polymorphism, and metamorphism (i.e., packers, encrypted payload wrappers, byte-level mutation, and instruction-level rewriting), since these methods alter surface form and reduce the durability of exact byte patterns and fixed structural markers [4,27,30,72]. The heuristic layer remains vulnerable to feature manipulation, entropy masking, and controlled structural variation (e.g., padding, section reshaping, import table changes, or benign feature injection) that preserve malicious function while they shift static indicators away from learned or engineered thresholds [18,28,29]. The behavioral and sandbox layers face delayed activation, trigger-based execution, anti-virtual machine checks, and mimicry (e.g., long sleep calls, time or user action triggers, hardware and driver checks, or insertion of benign-like file and registry activity), all of which reduce trace visibility or force malicious activity to resemble normal software behavior during limited observation windows [32,48,50,52,74]. Living off the land techniques create a further challenge, because they rely on trusted system tools, scripts, and administrative utilities (e.g., PowerShell, cmd.exe, WMI, rundll32, or scheduled tasks) rather than on clearly distinct malware binaries, which weakens both signature recognition and behavior-based separation [18,50]. The cloud and reputation layers are less exposed to local code mutation but remain sensitive to rapid infrastructure change, sparse prevalence, and the quality of telemetry used for consensus [26,33,34,56]. These patterns show that adversarial adaptation does not target one detector in isolation; it probes the boundaries between layers and exploits the assumptions on which each layer depends [4,18,19].

8. Thoughts for the Future

Antivirus architectures are converging toward a post-signature paradigm in which deterministic recognition is progressively supplanted by continuous inference grounded in behavioral semantics and zero-trust logic [1,45,64]. The main technical drivers of this shift include behavior-centered inference, adversarially aware learning, explainability, and autonomous recovery mechanisms. In these systems, no entity or process is implicitly trusted; each computational action is validated through provenance, execution context, and dynamic behavioral verification [7,47,65]. This inversion of trust transforms the antivirus engine from a reactive scanner into a proactive inference-driven detection system that captures intent rather than merely detecting static artifacts [52,66]. Behavior-first frameworks redirect analysis from syntactic identification toward semantic comprehension of program behavior. Their objective is to align detection with causal inference and reinforcement-based learning principles, where decision policies evolve through feedback rather than fixed rule sets [1,47,67]. These models anticipate malicious intent through analysis of operational context and control-flow evolution, detecting pre-activation indicators that remain invisible to traditional heuristics or signature abstractions [41,68]. These developments also expose open research gaps, especially in robust learning under adversarial pressure, stable adaptation under drift, and reliable inference across changing operational contexts. The growing interplay between defensive and adversarial machine learning defines the next critical frontier [48,49,51]. Malware developers increasingly exploit vulnerabilities in neural classifiers through evasion perturbations, gradient masking, and poisoning of training data [48,49,51]. Defensive adaptation, therefore, relies on resilient training regimes, adversarial retraining strategies, and ensemble-based consensus mechanisms that stabilize classifier behavior under synthetic manipulation and distributional shift [49,50,69]. Such ensembles reduce the attack surface of any single model and dilute the impact of adversarial bias. Practical constraints remain equally important, since future systems must satisfy auditability, controllability, privacy protection, and deployment efficiency under real operational limits. Nevertheless, robust detection increasingly depends on transparency and interpretability rather than black-box optimization alone. Explainable AI techniques can clarify the internal decision logic of autonomous classifiers [50,70]. Examples include Shapley value attribution, which estimates the contribution of each feature to the final decision. Other examples include symbolic abstraction, which converts complex model behavior into simpler rule-like form, and causal saliency analysis, which identifies the input elements that most strongly drive the verdict. All these methods also support auditability, forensic trust, and regulatory compliance because they make the basis of the decision easier for human analysts and auditors to review and explain [50,53,70]. Interpretability further enables rapid model correction under concept drift, preventing silent degradation of detection quality. In parallel, autonomous antivirus ecosystems integrate self-healing capabilities, continuous feedback loops that identify, isolate, and remediate compromised modules through redundancy, predictive rollback, and resilience-oriented orchestration [52,71]. The future antivirus thus ceases to be a static product and becomes a distributed adaptive organism: a system that perceives, reasons, and restores itself within adversarial digital ecologies, operating under perpetual uncertainty and active opposition [1].

Position Relative to Existing Surveys

Several recent surveys have already reviewed malware detection from method-based perspectives. These include a systematic review of malware detection and classification by Berrios et al. [78], a review of deep learning methods by Song [79], a systematic survey of artificial intelligence use in malware detection by Gaber et al. [80], a survey of graph representation learning for malware detection by Bilot et al. [81], and a broad survey of malware detection techniques by Joshi et al. [82]. These works provide useful syntheses of method families, datasets, and performance trends, but they usually organize the field by analytical technique or model class rather than by the internal structure of antivirus systems. In contrast, the present review adopts an architecture-first perspective. It examines how signature, heuristic, behavioral, sandbox, and cloud-assisted modules act as interacting layers within the antivirus pipeline. This perspective adds a system-level account of how input signals pass across layers, how local outputs support higher-level decisions, and how update logic, deployment constraints, and feedback paths shape the full detection process. In this sense, the contribution of the present article is not another catalog of malware detection methods but an architectural synthesis of antivirus systems as layered decision structures.

9. Conclusions

Antivirus systems have evolved into layered algorithmic ecosystems whose effectiveness arises from the controlled interaction of heterogeneous detection paradigms rather than from any single dominant technique. Deterministic signature engines provide verifiability and predictable filtering; heuristic and behavioral layers enable contextual generalization under code mutation, while machine learning components introduce probabilistic inference in regimes of uncertainty. Cloud-assisted aggregation further externalizes decision authority, transforming individual endpoints into contributors within a distributed inference fabric. Viewed collectively, modern antivirus architectures resemble composite decision systems in which resilience emerges from architectural stratification rather than algorithmic novelty alone. A central insight emerging from this review is the growing tension between learning-based detection and adversarial manipulation. While deep learning models extend detection beyond handcrafted rules, their susceptibility to adversarial perturbations and transfer attacks limits their reliability as standalone defenses. In response, contemporary systems increasingly reintroduce determinism around learning components through prefilters, confidence thresholds, consensus mechanisms, and reputation-based validation. This implicit re-determinization indicates a broader architectural trend. In practice, antivirus systems do not leave final decisions entirely to unconstrained adaptive models. They place limits around learning-based components through fixed rules, score thresholds, reputation checks, and secondary validation. As a result, controllability and robustness take priority over purely data-driven optimization. The historical evolution of antivirus systems thus follows a recurring pattern driven by adversarial escalation and computational constraint. Each transition, signature, heuristic, behavior, and cloud intelligence resolved specific bottlenecks while simultaneously exposing new attack surfaces. To date, antivirus architectures have not converged toward a universal solution. Instead, they distribute detection across signature, heuristic, behavioral, sandbox, and cloud layers because no single layer can simultaneously provide speed, generalization, interpretability, and robustness under continuous adversarial pressure. Future research is therefore likely to focus less on deeper models and more on architectural guarantees, including adversarially robust hybrid designs, confidence-calibrated inference pipelines, explainable decision mechanisms, and formally constrained interactions between deterministic and probabilistic layers. Such directions point toward antivirus systems that are not just adaptive, but also transparent, controllable, and resilient under adversarial pressure.