WinAPIReplay: Safely Re-Executing Win32 and NT-Native Malware API-Call Logs to Measure Behavioral Reproducibility and Generate Labeled Endpoint Telemetry

Abstract

Behavioral malware analysis relies on dynamic-analysis logs—sequences of Windows API calls recorded by sandboxes such as CAPEv2—assumed to represent the malware’s effects faithfully. To the best of our knowledge, this assumption has never been tested by re-executing the recorded calls. We propose WinAPIReplay, which re-executes each recorded Win32 and NT-native call as a real operating-system call, without the malware binary, using a unified handle map for cross-layer handle chains and a three-tier sandbox confining every side effect to disposable places. Across 500 WinMET samples from five families, 74.04% of modeled Layer-1 behavior-domain calls are reproduced (95% bootstrap CI [71.02, 76.75]); the Layer-1 domain covers 57.98% of all recorded calls, and a conservative rate excluding substituted calls is 71.15%. An ablation attributes this causally to the per-category executors—handle-validity falls from 94.9% to 10.2% without them—and no side effect escapes the sandbox on the channels the tool models and monitors. A four-way taxonomy assigns most of the residual to intrinsic, environment-dependent behavior; re-execution is near-deterministic (98.90% stable). Under Sysmon, it safely generates family-labeled file/registry telemetry for the reproduced subset (46,150 events) from static logs alone—while its result record recovers the malware’s process arguments and network destinations—and a classifier over it reaches 92.0% leave-one-out accuracy on 100 samples, statistically indistinguishable from an API-category baseline. WinAPIReplay thus provides, to the best of our knowledge, the first quantitative measurement of dynamic-log reproducibility within a demonstrated safety envelope, and a safe route to labeled, environment-consistent endpoint telemetry.

1. Introduction

Windows malware is routinely studied through dynamic analysis: a sample is detonated inside an instrumented sandbox, and the sequence of API calls it makes—together with arguments, return values, and object handles—is recorded as a structured log. Public corpora of such logs now reach tens of thousands of samples, and a large body of work uses them as features for malware family classification and detection. All of this work rests on an implicit premise: that a recorded API-call sequence is a meaningful representation of the effects the malware produced—the files it created, the registry values it wrote, the connections it opened. The premise is intuitive, but to the best of our knowledge it has never been examined empirically for malware. As we discuss in Section 2, recorded API calls have been re-issued before but only to test operating-system components; no prior study takes a malware log and runs it, call by call, to observe what the recorded behavior does when carried out again.

This paper addresses that question directly, asking whether executing the file-related calls in a log creates the intended files, whether the registry calls reach the intended keys, whether a sequence that interleaves NT-native and Win32 calls reproduces as a chain—given that a handle minted by one layer is later consumed by the other—and what fraction of a real malware log reproduces, together with where and why reproduction fails. Answering this matters for two reasons. First, it supplies missing empirical footing for behavioral analysis: dynamic-analysis logs are widely trusted as behavior specifications, yet how much of a recorded log reproduces when carried out again had not been measured. Establishing this opens a third path between dynamic analysis, which is faithful but dangerous, and static analysis, which is safe but behavior blind. Second, and most concretely, the labeled endpoint telemetry that malware-classification and behavioral-detection research needs—for instance, Sysmon file and registry events, in live form, tagged by family—can today be obtained only by detonating real malware under a monitored host, which is dangerous, environment-dependent, and hard to reproduce. A method that regenerates such telemetry from a static log, without ever running the binary, would remove that barrier, turning existing log corpora into a safe source of labeled training data. Concretely, an analyst who holds a sample’s recorded API-call log but cannot—or must not—detonate the sample can use WinAPIReplay to re-execute that log inside an isolated virtual machine and obtain the file- and registry-events that the re-execution produces—captured by Sysmon and tagged with the sample’s family—turning a static log into labeled training data for malware-family classification, without the binary ever running. The same re-execution simultaneously yields the reproducibility measurement itself—a call-by-call account of which recorded operations carry over to a fresh environment and which do not.

Re-executing a recorded log is not trivial, which is why the question has remained open. Logged handle values are specific to the capture environment and invalid elsewhere; NT-native and Win32 calls interleave and must share one handle namespace; the re-execution must not damage or infect the host; and logged arguments—strings, structures, pointers—must be reconstructed into real call arguments. In this paper, we propose WinAPIReplay, which resolves these obstacles and, in doing so, turns a passive log into a controlled, repeatable re-execution.

Applying WinAPIReplay to 500 malware logs from five families, we find that 74.04% of modeled Layer-1 behavior-domain calls reproduce—the Layer-1 domain covering 57.98% of all recorded calls—that this reproduction is produced by the tool’s per-category executors rather than by a generic fallback, that the residual is dominated by intrinsic, environment-dependent behavior rather than by fixable tool gaps, that every side effect stays contained across all samples on the channels the tool models and monitors, and that the re-execution safely generates family-labeled endpoint telemetry from which malware families can be classified. This paper makes four contributions, none of which, to the best of our knowledge, appears in prior work.

• C1—A behavior-reproduction tool (WinAPIReplay). A tool that re-executes both Win32 and NT-native API calls from a static log as real operating-system calls, using a unified handle map for cross-layer chains and a three-tier sandbox that contains all file, registry, and network side effects.
• C2—The first quantitative measurement of dynamic-log reproducibility. Using WinAPIReplay, we measure how much of a real malware log reproduces in a fresh Windows environment, across 500 samples and five families.
• C3—A four-way failure taxonomy. A method that separates fundamental, environment-dependent failure (true failure) from engineering-solvable barriers such as unresolved NT-to-Win32 handle dependencies, and quantifies each.
• C4—Safe generation of labeled endpoint telemetry. A demonstration that running the re-execution under Sysmon synthesizes family-labeled file and registry events (Event ID 11/12/13/23) from static logs alone, without executing any malware binary—while the tool’s result record additionally recovers the original process arguments and network destinations that process suppression and network redirection remove from the corresponding Sysmon events—and that the generated file/registry telemetry is discriminative.

We organize the study around five research questions, which the experiments in Section 5 address in turn.

• RQ1. How much does the Layer-1 executor design improve behavior reproduction compared with a Layer-2-only (generic-dispatch) configuration?
• RQ2. To what extent can WinAPIReplay reproduce Layer-1 API calls—as measured by the Layer-1 Behavior Reproduction Rate (L1 BRR)—and how does the rate vary across malware families and API categories?
• RQ3. What are the principal causes of reproduction failure, and what fraction represents intrinsic, irreducible failure (true failure)?
• RQ4. Is side-effect isolation complete across all samples, and how stable is the output—to what extent does an identical input under an identical initial state yield the same outcome sequence, and what accounts for the variation that remains?
• RQ5. Can WinAPIReplay with Sysmon safely generate family-labeled endpoint telemetry, and is the generated data useful for machine-learning-based classification?

These questions map onto the four contributions: RQ1 and RQ4 establish that the tool (C1) reproduces behavior through its executor design and does so safely and stably; RQ2 delivers the reproducibility measurement (C2); RQ3 provides the failure taxonomy (C3); and RQ5 demonstrates safe telemetry generation and its utility (C4). Execution cost is reported alongside RQ4 as a measure of practicality.

The remainder of the paper is organized as follows. Section 2 situates the work against prior recording, replay, emulation, and classification systems—as well as the one prior mechanism that re-issues recorded API calls and empirical studies of how malware behavior varies across environments—and states precisely what is new here. Section 3 describes the design and implementation of WinAPIReplay—the two-layer executor model, the cross-layer handle map, and the three-tier sandbox. Section 4 defines the corpus, the experimental environment and design, the reproduction metrics and their biases, the failure taxonomy, and the statistical methods. Section 5 reports the eight experiments that answer the five research questions. Section 6 discusses what the measurement means, why re-execution, rather than static transformation, is required to generate faithful telemetry, what this implies for detection-model training, the safety and stability that make the approach usable, and how the work stands in relation to prior art. Section 7 sets out the limitations and threats to validity, and Section 8 concludes.

2. Related Work

This section positions WinAPIReplay against prior work. It helps to fix three actions at the outset, because the distinction between them is what separates our contribution from existing systems. A tool records behavior when it runs a program and logs what it does; it replays a binary when it re-runs a program from a low-level recording of a past execution; and it observes behavior when it analyzes already-collected logs without running anything. Our aim is none of these: we re-execute an already-recorded API-call log—without the original program—and measure how faithfully the recorded operations reproduce when carried out again.

2.1. Background: The Two-Layer Windows API and Object Handles

Three concepts recur below and in the rest of the paper. First, Windows exposes its services through two layers of programming interface. Application code normally calls the documented Win32 layer (for example CreateFileW to open a file or RegSetValueExW to write a registry value). Each Win32 function is implemented internally by calling the lower, largely undocumented NT-native layer (for example NtCreateFile, NtSetValueKey, and their equivalently named Zw* entry points). Malware frequently calls the NT-native layer directly, bypassing Win32, both to access functionality Win32 does not expose and to evade security tools that monitor only the Win32 layer.

Second, when a program opens a file, a registry key, or another operating-system object, the system returns a handle: an opaque numeric token that the program passes to later calls to identify that object (for example, the handle returned by NtCreateFile is later passed to WriteFile). A handle is valid only within the process and session that obtained it; the same numeric value has no meaning in a different execution. Reproducing a recorded sequence therefore requires translating each recorded handle to a live one, and because a handle opened by an NT-native call may be used by a later Win32 call (and vice versa), both layers must share a single translation table—what we call a unified handle namespace.

Third, running a malware sample so that it executes and produces its effects is called detonation. Dynamic analysis detonates a sample inside an instrumented, isolated environment (a sandbox) and records the API calls it makes. Our work never detonates the sample: it takes an already-recorded log and re-executes the recorded calls individually. When we speak of the tool’s own sandbox, we mean something narrower: not an isolating environment but redirection applied to the arguments of each modeled call so that the call acts on a disposable file, registry, or network target instead of the one the malware named.

2.2. Sandboxes and Frameworks That Record API-Call Logs

The tools closest to us in subject matter, but opposite in direction, are those that produce API-call logs by detonating malware. Cuckoo Sandbox [1] is an influential open-source dynamic-analysis system that runs a malware binary in a virtual machine and records the API calls it makes; although its original version is no longer actively maintained, it established the design that later systems build on. CAPEv2 [2] is one such successor: it extends Cuckoo with payload extraction and additional instrumentation, and it is the sandbox used to build our evaluation dataset. TTAnalyze [3], an early and influential analyzer, is directly relevant to our two-layer framing: it runs a Windows binary in an emulated environment and monitors both the program’s Win32 API calls and its NT-native system calls, precisely because malware uses the native layer to evade tools that hook only Win32. The MALVADA framework [4] parses CAPEv2 reports into structured execution traces and was used to build the WinMET dataset  [5,6] on which we evaluate; each WinMET trace is a report listing, for every process, the ordered Win32 and NT-native calls with their parameters, return values, and accessed resources.

The dataset is also described in its own data article [6]. The same group has since released MalGraphIQ [7], which reads these traces and matches them against a hierarchical catalog of API-call patterns derived from MITRE’s Malware Behavior Catalog, producing visual summaries of what a sample did; it was demonstrated on twelve WinMET families, one hundred traces each. It reads the trace, as we do, but reads it as a record to be interpreted rather than as a sequence to be carried out again—which is precisely the distinction this section draws. All of these tools record: they run the binary and capture what it does. None re-executes a recorded log in a fresh environment, and none measures whether the recorded calls reproduce. They are complementary to our work rather than competing with it: they generate the very logs that WinAPIReplay consumes as input.

2.3. Instruction-Level Record-and-Replay

A mature line of systems records an execution at a very low level—individual processor instructions or device inputs—and then replays that recording to reconstruct the original run exactly. ReVirt [8] logs a virtual machine’s execution beneath the operating system and replays it instruction by instruction to support intrusion analysis. V2E [9] records a malware execution using hardware virtualization—which is harder for malware to detect than a software emulator—and then replays that recording in software so that heavyweight analysis can be applied; it was evaluated on samples specifically designed to resist software emulators. Malrec [10] uses whole-system record-and-replay, built on the PANDA analysis platform, to capture a large corpus of complete malware executions for later inspection.

Because the word “replay” is common to this line of work and to ours, we state the distinction explicitly. These systems take the malware binary plus a low-level recording (processor and device activity, typically gigabytes per run) and reconstruct one specific past execution faithfully. WinAPIReplay takes only a high-level list of API calls, needs no binary and no low-level recording, and re-issues those calls as fresh operating-system calls in a new environment. The objectives differ accordingly: instruction-level replay asks whether a particular past execution can be reconstructed exactly, whereas we ask whether the operations recorded in a log reproduce when carried out again in a clean environment, and how many of them succeed. The two uses of “replay” thus differ in input (an API-call log versus a binary and a low-level trace), in granularity (one API call versus one processor instruction), and in purpose (measuring reproduction versus reconstructing a specific run).

2.4. API-Emulation Frameworks

Closest to WinAPIReplay in mechanism—though not in input or purpose—is a family of binary-emulation frameworks that model the Windows environment in software and run a malware binary against that model rather than on a real machine. Binee [11] builds a near-complete Windows process model, including a mock file system and loaded libraries, and emulates the kernel so that it can log every API call a binary makes with its parameters; comparable frameworks include Mandiant’s Speakeasy [12] and the Qiling framework [13]. Vouvoutsis et al. [14] evaluate this class of tools for malware classification and show that emulating a binary and recording its API calls can classify malware accurately at lower cost than a full sandbox. These frameworks are directly relevant to us because they, too, must supply substitutes for file, registry, and network operations so that execution can proceed—the same problem our three-tier sandbox solves.

The difference is nonetheless fundamental, along two axes. First, in input: an emulation framework takes the malware binary and drives execution from it, whereas WinAPIReplay takes only a recorded API-call log and needs no binary. Second, in what actually runs: an emulation framework models each API in software and returns simulated results, so no real operating-system call is made; WinAPIReplay instead issues each recorded call to the real operating system—which is exactly why real file and registry effects (and hence genuine Sysmon telemetry) are produced, and why side-effect containment becomes necessary. In short, emulation frameworks imitate API behavior from a binary, whereas we reproduce API behavior from a log by executing it for real under containment.

2.5. Re-Issuing Recorded API Calls: Prior Art in Software Testing

The single prior mechanism that, like ours, re-issues recorded API calls rather than replaying instructions comes from software testing. The now-expired Microsoft patent US 7,380,235 [15] (filed 2003, granted 2008, patent term ended March 2025) describes an “application program interface call replay tool” that takes recorded API-call records and re-issues each call in a controlled environment to test and verify operating-system components for compatibility, security, and regression—explicitly contrasting itself with debuggers that step through a program’s binary code. It therefore shares with WinAPIReplay the core idea of re-executing recorded API calls instead of instructions, and it even reconstructs some call context (for example, resolving the memory locations of call parameters through lookup tables).

Its purpose, input, and technical scope nonetheless differ fundamentally from ours. Its input is a set of API calls chosen for compatibility testing, not a trace recorded from real malware, and its replay environment is moreover populated with resources extracted from the original executable so that—unlike ours—it is not binary-free; its goal is to test operating-system components, not to analyze malware; and it provides none of the four capabilities that define our contribution: it does not contain side effects within a disposable sandbox, does not reconstruct handle chains that cross the NT-native/Win32 boundary, does not measure how faithfully the calls reproduce, and does not generate endpoint telemetry. Accordingly, we do not claim novelty for the bare idea of re-issuing recorded API calls; our novelty lies in the four security- and measurement-oriented contributions stated in Section 2.8, none of which this patent provides.

2.6. Environmental Variation in Malware Behavior

A separate body of work motivates why the reproducibility of a log must be measured empirically rather than assumed. Avllazagaj et al. [16] carried out the first large-scale quantitative study of behavioral variability, using 7.6 million execution traces collected from 5.4 million real computers, and showed that an individual sample’s observed behavior changes substantially from one execution to another because of differences in the environment, deliberate evasion, and the passage of time. Kaya et al. [17] demonstrated a direct practical consequence: machine learning detectors trained on sandbox traces achieve a high true-positive rate (the fraction of malicious samples correctly flagged) on further sandbox traces—around 95%—but that rate falls to about 17% when the same detectors are evaluated on traces captured from real endpoint computers.

These studies observe that recorded behavior varies and that models trained on sandbox data transfer poorly to real hosts; they analyze collected traces but do not take a recorded log and actively re-execute it, call by call, to measure how much of it reproduces in a new environment and why the rest does not. WinAPIReplay supplies precisely that missing active measurement. Moreover, Kaya et al. note that training detectors on real endpoint traces is itself difficult because such labeled telemetry is costly to collect; producing endpoint-style telemetry safely from static logs, as the telemetry-generation capability introduced in Section 2.7 does, directly addresses that difficulty.

2.7. Classifiers That Use Recorded API Traces as Features

Recorded API-call traces are widely used as features for malware classification and detection [18], where a trace is treated as a static feature vector without asking whether the recorded calls would reproduce if carried out again. Our aim is not to propose another classifier but to provide a safe source of labeled data: because our re-execution issues real file and registry operations, running it while the Windows monitoring tool System Monitor (Sysmon) [19] is active causes Sysmon to record genuine file and registry events for the operations that reproduce, yielding family-labeled telemetry from a static log without detonating any malware. The random-forest classifier in Section 5.7 is used only to confirm that this telemetry is discriminative—a means of validation, not a contribution in itself.

2.8. Research Gap and Novelty

Table 1. Positioning of WinAPIReplay relative to prior lines of work. A dash (—) marks a column that does not apply because the line does not re-execute a log; “n/r” marks containment that isolates only the original binary during capture, a different purpose from containing the side effects of re-execution.

Line of Work	Input	Re-Executes	Cross-Layer	Contains Re-Exec.	Generates
		a Log?	Handle Chain	Side Effects	Telemetry?
Recording tools [1,2,3,4,5,6,7]	binary	no (records)	—	n/r (capture only)	no (records only)
Instruction replay [8,9,10]	binary + trace	no (replays binary)	—	n/r (capture only)	no
API-emulation frameworks [11,12,13]	binary	no (emulates APIs)	—	n/r (modeled)	no
API-testing patent [15]	API-call log + binary resources	yes (for testing)	no	no	no
Variability studies [16,17]	traces	no (observes)	—	—	no
Trace classifiers [18]	traces	no (features)	—	—	no
WinAPIReplay (this work)	API-call log	yes (for security)	yes	yes (three-tier)	yes

summarizes the positioning. Recording tools (Cuckoo, CAPEv2, TTAnalyze, and MALVADA/WinMET) produce API-call logs but never re-execute them; instruction-level systems (ReVirt, V2E, and Malrec) reconstruct a past execution from the binary and a low-level trace; and API-emulation frameworks (Binee and related tools) run a binary against a software model of the operating system, simulating each API rather than issuing real calls. The testing-oriented patent US 7,380,235 re-issues recorded API calls, but without side-effect containment, cross-layer handle reconstruction, fidelity measurement, or telemetry generation; empirical studies (Avllazagaj et al. and Kaya et al.) observe that behavior varies without actively re-executing logs; and feature-based classifiers (MAMBA+) consume recorded traces without re-executing them.

To delimit the gap, we surveyed the literature and found no study that takes a recorded log of NT-native and Win32 API calls, re-executes it as real operating-system calls while containing all side effects, and quantifies how faithfully the calls reproduce and why they fail. To the best of our knowledge, WinAPIReplay is therefore the first system to combine all four of the following: (i) binary-free re-execution of malware API-call logs across both the NT-native and Win32 layers; (ii) quantitative measurement of per-call reproduction fidelity; (iii) systematic classification of the causes of reproduction failure; and (iv) generation of family-labeled endpoint telemetry—file and registry Sysmon events, with process and network intent recovered—without detonating the malware. We state this as a claim “to the best of our knowledge” rather than as an absolute first, because the non-existence of prior work cannot be proved exhaustively and we cannot rule out unpublished proprietary tools.

3. Binary-Free Re-Execution of API-Call Logs: Executor Design

This section describes WinAPIReplay as a system, independently of any experiment. The tool takes one input—a recorded API-call log—and re-executes each recorded call as a real operating-system call, without the malware binary, inside a contained environment.

3.1. Input: The Recorded API-Call Log

The input is a JSON log conforming to a fixed schema. It consists of metadata (the recorded process’s bit-width—whether it ran as a 32- or 64-bit process—and how the log was collected) and an ordered list of events. Each event records one API call and carries:

• A sequence number fixing its position in the recorded order;
• The identifiers of the process and thread that made the call;
• The API name and the list of argument values;
• Any handles that the call takes as input or produces as output, in the sense defined in Section 2.1;
• The originally recorded return value and status.

The log is a pre-existing input: WinAPIReplay consumes it and never reads the malware binary.

3.2. Architecture: A Two-Layer Execution Model

The central design principle separates genuine behavior reproduction from mere execution. Issuing a recorded call with its arguments zeroed or guessed—so that the call runs but does nothing meaningful—does not reproduce behavior. Behavior is reproduced only when each recorded argument is faithfully reconstructed into a real, correctly typed argument: the recorded path becomes a real path, the recorded handle becomes a live handle to the same type of object, and the recorded structure is rebuilt in memory. WinAPIReplay therefore reconstructs arguments precisely for the calls it models—handled by its first, signature-defined layer (Layer 1, described next)—and falls back to a best-effort generic mechanism only for calls it does not model (Layer 2), counting only the former as reproduced behavior. This distinction is what the evaluation in Section 5 measures, and it is enforced throughout the design.

WinAPIReplay dispatches each call through one of two layers. Layer 1 handles calls described by a signature database: a table that specifies, for each supported API, its category, the type of every argument, its return type, and whether it produces or invalidates a handle. Each call in the database is executed by a per-category executor that uses this signature to reconstruct arguments exactly, redirect side effects, and update the tool’s run-time state.

Layer 2 is a generic dispatcher that handles any call absent from the database: it locates the target function dynamically—by searching the dynamic-link libraries (DLLs, the shared code modules that provide Windows API functions) loaded in the process for a function exported under the recorded name—and invokes it. Because Layer 2 has no signature and therefore cannot prepare typed arguments, such a call may dereference an invalid pointer and trigger an access violation (a fault raised when code touches an invalid memory address); the dispatcher therefore invokes the function inside a protective wrapper that catches such a fault so that it does not crash the whole re-execution. A call that faults in this way yields no usable return value, and is recorded as skipped rather than classified by outcome; only a Layer 2 call that returns normally is recorded as a success (Section 3.8). For the same reason, Layer 2 does not attempt handle translation: pointer-width integer arguments (which may be handles) are passed as zero rather than resolved through the handle map since blindly resolving a non-handle integer that happens to match a registered value would corrupt the call. Only string arguments are reconstructed as real buffers. Consequently, a Layer 2 call runs but is not expected to reproduce behavior that depends on a live handle—which is consistent with Layer 2 outcomes not being credited as reproduced behavior. A single dispatch table maps each API name to its executor, so the layer decision is made in constant time. Layer 1 is the domain over which behavior reproduction is measured; Layer 2 exists so that an unmodeled call does not abort the run, but its outcomes are not credited as reproduced behavior.

The two layers share one design consequence that motivates much of what follows. A recorded log contains concrete values—handles, memory addresses, and process identifiers—that were valid only in the environment where the log was captured. Re-executing the log therefore requires reconstructing these values for the new environment, which is the purpose of the maps and argument-preparation logic described next.

For each recorded event, the tool follows a fixed sequence of steps. It first checks the call against the safety block-list of Section 3.7 and skips it if listed; otherwise it looks up the API name in the dispatch table to choose Layer 1 or Layer 2. For a Layer 1 call, it prepares the arguments from the signature as described in Section 3.4, invokes the real call, and classifies the outcome by the rules of Section 3.8; it then updates the three maps from the result, and records one result entry for the call. Figure 1 shows how these components fit together, and the subsections that follow describe each step in turn.

3.3. Run-Time-State Reconstruction: Handle, Pointer, and Process-Identifier Maps

Handles, memory addresses, and process identifiers are run-time values that a static log cannot carry across environments. Three lookup tables translate recorded values into live ones. The handle map records, for each handle in the log, the live handle actually obtained when the producing call is re-executed, and substitutes the live handle wherever the recorded handle is later used. A handle enters the map by either of two routes: when a call returns the handle directly as its result (for example a Win32 file-open call), or when a call delivers the handle through an output argument (for example an NT-native file-open call, or a registry-open call that writes the opened key into a caller-supplied location). A third route covers the case where neither has happened. Before execution begins, the tool indexes the log by the handle value each event produced; if a later call then presents a handle the map does not hold, the tool looks up the call that produced it and re-creates the object in the redirect targets—a file, a registry key, or a handle to the tool’s own process—so that the chain can continue. Without it, every call downstream of an unresolved handle would be recorded as approximate. Malware commonly calls the NT-native layer directly rather than through the Win32 wrapper, so a handle produced by an NT-native call may be consumed by a later Win32 call, and vice versa. This single map therefore spans both API layers, and it is how the tool reproduces the cross-layer handle chains that the research problem identified as a core obstacle. Figure 2 shows a concrete example: an NT-native NtCreateFile in the log records a handle, and re-executing it yields a different, live handle in the new environment, which the map stores against the recorded value; when a later Win32 WriteFile is re-executed with the recorded handle, the map substitutes the live handle so that the write reaches the file the earlier call opened. Without this cross-layer substitution the recorded handle would be meaningless and the write would fail. Six well-known pseudo-handles—fixed handle values that the operating system reserves rather than allocating them per process, namely the five predefined registry roots (such as HKEY_CURRENT_USER) and the reserved value that marks an invalid handle (the invalid-handle marker)—are never entered in the map and never resolved through it. They are, however, normalized: a log captured from a 32-bit process records these roots in a truncated form, which the tool converts to the sign-extended form a 64-bit environment expects before the argument is passed on. When the call that closes a handle is re-executed, the corresponding entry is invalidated. The pointer map performs the analogous translation for memory addresses: it records addresses returned by memory-allocation calls, and also addresses delivered through pointer-valued output arguments so that a later call operating on that memory receives a valid address. The process-identifier map links a process created during re-execution to its recorded identifier so that, for example, a later call that opens a process by identifier targets the process actually created.

All three maps are shared across execution, and the tool offers two execution modes: it can group the recorded events by thread and re-execute each thread concurrently, or it can re-execute all events in a single thread following their recorded sequence order. The recorded order is meaningful because a trace stores every thread’s calls in one time-ordered list; single-threaded re-execution follows that order directly and, by serializing access to the shared maps, avoids handle collisions and network-address races that concurrent re-execution can introduce. Which mode the experiments use is stated in Section 4.

3.4. Type-Aware Argument Reconstruction

For each Layer 1 call, an argument preparer turns the recorded arguments into real ones according to the call’s signature. Each argument is classified into one of fourteen types and handled accordingly. The main cases are these:

• A handle argument is resolved through the handle map, and a process-identifier argument through the process-identifier map;
• A handle-array argument, as used by calls that wait on several objects at once, is rebuilt by resolving each recorded handle in turn;
• A string argument is rebuilt as a real character buffer, and a direct scalar is passed as recorded;
• An optional pointer that is normally null is passed as null;
• An input-data buffer is allocated at the recorded size and zero-filled, and an output buffer at the size the call expects;
• A size-reporting argument, together with the paired output buffer it governs, is allocated at a fixed default capacity of 64 KiB, which suffices for these calls in practice;
• A socket-address argument is supplied from the address the network layer resolved, as described in Section 3.7.

Two string sub-types drive side-effect containment: a file-path argument of a file call is redirected into the file sandbox, and a registry sub-key path is redirected into the registry sandbox of Section 3.7; redirecting a registry sub-key also rewrites the preceding root-handle argument to point into the sandbox so that the whole hierarchical key operation stays internally consistent. Registry value-name arguments are deliberately not redirected, since the key handle they operate on has already been redirected. Because the same preparation machinery serves both NT-native and Win32 calls, an interleaved cross-layer sequence is reconstructed as one coherent chain. Every argument is checked for the null case first so that a pointer recorded as an address meaningless in the new environment is safely replaced by null rather than causing a type mismatch.

After a Layer 1 call returns, the tool updates the three maps from the call’s signature: a handle- or socket-returning call registers its live result in the handle map; a memory-allocating call registers its result in the pointer map; a call that outputs a handle, a pointer, or a created process through an output argument registers each in the corresponding map; and a call that closes a handle invalidates the corresponding entry.

3.5. Per-Category Executor Behavior

The per-category executors differ in the non-trivial steps each must take so that a recorded call reproduces its intended effect. We describe the representative cases; each reflects a concrete requirement of the corresponding Windows call.

The process executor must initialize, before each process-creation call, the size field of the start-up-information structure that the call expects; the underlying CreateProcess call requires that field to be set, so the executor sets it to the structure’s true size. A recorded OpenProcess call is retargeted through the process-identifier map to the process actually created earlier in the run. Thread-creation calls are re-executed with a placeholder start routine that returns immediately, so a thread is created but performs none of the recorded work: the thread’s existence is reproduced, its recorded code is not (a deliberate fidelity limit). The synchronization executor overrides the recorded wait time on wait calls with a configurable timeout (five seconds by default), because a recorded infinite wait would otherwise hang the entire re-execution. Where the waited-on handle resolves, the wait is performed against the live object and the timeout guarantees liveness; where it does not resolve, no wait is performed and the call reports success, so the call is counted as reproduced although no wait took place. Section 4.5 lists this among the measurement biases affecting the rate. The DLL executor deliberately does not redirect library paths into the sandbox so that a recorded library load resolves against the real system libraries it names. The cryptography executor, when a recorded key-container acquisition fails because the container does not exist in the fresh environment, retries while requesting the creation of a new container, and, if that also fails, falls back to acquiring a provider context with no named container at all so that later calls in the chain still receive a usable provider handle. The service executor continues past calls that fail for lack of administrative privilege (creating or starting a Windows service requires it), recording the failure without stopping the run. The hook and token executors record that an operation was attempted without carrying out its dangerous effect: a hook-installation call returns a non-null sentinel in place of a hook handle without installing any real hook, and a token-privilege adjustment returns success without altering any privilege, so each call is reported as succeeding while no active malicious behavior is put in place. These bounded fidelity limits are consolidated in Section 3.10.

3.6. Implemented API Set and Call-Volume Coverage

The signature database defines 302 API entries. Of these, 237 are assigned to twelve behavior categories—file, registry, Winsock (low-level socket) networking, process, DLL loading, synchronization, service, WinINet (HTTP/HTTPS) networking, cryptography, shell, security-token, and hook operations—and are the domain over which reproduction is measured; the remaining 65 are utility calls (for example handle-closing and system-information calls) that are left uncategorized and treated as Layer 2 for measurement purposes. The categorized calls are handled by eighteen executor classes, exposing 303 API strings that correspond to 257 logically distinct calls, and the implementation is validated by 471 unit and integration tests, including tests of cross-layer handle chains.

The implemented set was chosen by call frequency, and the choice is justified by a coverage analysis of the evaluation corpus. Across all 500 samples and 2,333,242 recorded calls, only 221 distinct API names appear, and the implemented executor set accounts for 87.94% of the total call volume (2,051,822 calls); the 25 most frequent APIs alone account for 74.8%. Of the 221 distinct names, 119 are implemented; the remaining 102 account for the other 12.06% of the call volume and are handled by Layer 2. Most of them are individually rare—70 of the 102 occur fewer than 1000 times in the whole corpus—though the two largest, GetProcessHeap (51,519 calls) and NtQuerySystemInformation (31,977 calls), are considerably more frequent than that. Concentrating implementation effort on the high-frequency, security-relevant calls thus covers close to seven-eighths of all recorded activity. This call-volume coverage (87.94%) is higher than the Layer-1 domain coverage reported later (57.98% of executed events, Section 5.2), because some implemented system-information and component-object calls are categorized as utility and so counted outside the Layer-1 behavior domain.

3.7. Side-Effect Containment: Three-Tier Sandbox and Dangerous-Call Blocking

Because WinAPIReplay issues real calls, containing their effects is a primary design concern rather than an afterthought. Three redirections confine every observable effect to disposable locations while still letting the operation genuinely occur—which is what makes later telemetry generation possible.

The file tier rewrites every file-system path so that all file operations take place beneath a single disposable sandbox directory (C:\Sandbox in our experiments), preserving the relative structure of the original path so that directory relationships are retained; local absolute paths, network share paths (of the form \\server\share), and relative paths are each mapped into a dedicated area under that root. The registry tier redirects every top-level registry root—including the machine-wide configuration root and the file-type/component registration root that malware often targets—into a dedicated sub-tree under the current user’s software key (HKCU\Software\WinAPIReplaySandbox, where HKCU abbreviates HKEY_CURRENT_USER), and removes that sub-tree when the run ends; redirecting the machine-wide root into the user’s own area also sidesteps the need for administrative privileges. Before execution begins, the tool scans the log for the keys it will touch and creates them so that a recorded open of a key the malware had itself created earlier does not fail merely because the run starts from an empty sub-tree. The network tier redirects address resolution and socket endpoints to the loopback address (127.0.0.1, which always routes back to the same host and never reaches the network) and answers them with a benign local responder so that no packet leaves the host; the responder is “always ready”—on connection it offers a fixed 64 KiB block of data and then echoes whatever it receives—so that both client-initiated and server-initiated exchanges complete. The tool holds one Winsock reference of its own for the whole run so that recorded start-up and shutdown calls—which are themselves executed—can never tear down the session that later socket operations depend on. Data-carrying send-and-receive operations are satisfied with placeholder buffers of the recorded length, as are the input buffers of file and cryptographic calls elsewhere in the tool—zero-filled, except that the hashing call uses a constant non-zero byte. Registry values are an exception in the other direction: the log records the value data, so it is written as recorded. Section 3.10 and Section 4.5 treat this fidelity limitation.

Independently of these redirections, eight process- and thread-termination calls (for example TerminateProcess and NtTerminateProcess) are placed on a block-list and skipped before the layer decision is even made so that neither layer can execute them. This matters because, were such a call dispatched to Layer 2 and invoked, it would terminate the re-execution’s own process and truncate every remaining event; blocking it up front keeps the run intact at the cost of recording that single call as skipped. A second, disjoint block-list holds six calls that have no safe Layer-1 executor and would be hazardous if dispatched to the generic path with the log’s recorded arguments: an extended multi-object wait (NtWaitForMultipleObjectsEx), a COM object-instantiation variant (CoCreateInstanceEx), two low-level process-creation variants (NtCreateProcessEx, ZwCreateProcessEx), and two synchronization-object open variants (ZwOpenSemaphore, ZwOpenTimer). These are checked before the layer decision and, by design invariant, never overlap any executor’s supported set, so each such call is recorded as skipped and excluded from the reproduction rate. Process creation is contained by substitution rather than by blocking: the native and shell creation variants always report the success the caller expects without spawning anything, and for CreateProcess and WinExec an option—enabled in every run of Section 5—does the same so that no child process is ever created while the intended command line and creation flags are still preserved as process events (Section 3.9). The net intent is that real file, registry, and network effects occur but only inside designated, reversible locations, and never in a form that can terminate the run or escape the host.

A third form of containment applies to individual calls whose real execution would leave a persistent or system-wide side effect that no redirection can confine. Rather than being redirected or skipped, these calls are executed as safe substitutes that return the success value the caller expects while performing none of the dangerous action. A hook-installation call (SetWindowsHookEx) returns a non-null sentinel without installing any system-wide hook, which would otherwise burden every desktop thread on the machine. A privilege-adjustment call (AdjustTokenPrivileges) returns success without altering the process token, whose privileges the tool must keep intact for its own sandboxed operations. For a non-system library path supplied by the malware, a library-load call is issued with a data-file-only flag (LOAD_LIBRARY_AS_DATAFILE) so that the named library is mapped for inspection without executing its initialization routine (DllMain). A service-creation call returns a handle to an existing benign service instead of writing a new entry into the service-control database, which is persisted in the registry outside the sandbox. A debugger-attach call (DbgUiConnectToDbg), which malware uses as an anti-analysis probe and which would attach the tool’s own process to the kernel debug subsystem, returns success without connecting. In each case the call is reported as succeeding, so control flow proceeds, while its dangerous effect is never realized.

3.8. Outcome Determination

After each re-executed call, WinAPIReplay classifies its outcome, because the evaluation in Section 5 rests on this classification. The outcome is one of four values. A call is skipped when it is never invoked—a blocked termination call, a call on the generic-path block-list of Section 3.7, a call made during a dry run (a diagnostic mode in which the tool prepares each call but does not invoke it), or a Layer-2 call whose target cannot be resolved. An invoked call is judged success or failed by comparing its return value against the failure condition implied by its return type: a call returning a status code succeeds when the code indicates success, a call returning a handle succeeds when the handle is neither zero nor the invalid-handle marker, a call returning a pointer succeeds when the pointer is non-null, and a Boolean-returning call succeeds when it returns non-zero. For a minority of calls the executor reports the status the caller expects without inspecting the outcome of the underlying operation, so this test does not apply; Section 4.5 quantifies this case and bounds its effect on the measured rate. A call dispatched by the generic path carries no signature and therefore no return type to test; it is recorded as a success and, lying outside the Layer-1 behavior domain, is excluded from the reproduction rate of Section 4.5.

A failed call is then re-examined and, if its failure is attributable to the re-execution environment rather than to the tool, promoted to a fourth outcome, approximate, which is recorded separately and not counted as a reproduction failure.

Three conditions trigger this promotion. The first is an unmapped handle: the call takes a handle whose recorded value never entered the handle map—typically because the object was opened before log capture began, so no producing call exists to re-execute—and therefore fails with an invalid-handle error that no tool could avoid from this log. The second and third conditions arise only when the respective sandboxes are active. Under the network sandbox, a socket operation—a connect, send, receive, or teardown call—may fail because the simulated endpoint does not hold the state or data the real peer would have provided; under the registry sandbox, a registry query, open, or delete call may fail because the isolated registry sub-tree does not hold the value the original environment would have had. All three are properties of re-executing a log in a clean environment, not defects of reconstruction, which is why the failure taxonomy of Section 5.3 separates them from genuine (true) failures. Figure 3 summarizes the whole classification.

3.9. Outputs: Path Corrections and Recorded Process Intent

Beyond re-executing the log, WinAPIReplay emits a structured result record that later stages consume, and two features of this record bridge re-execution to the telemetry application of Section 5.6. First, whenever the sandbox rewrites a file or registry path, the tool records the rewrite as a transform: a mapping from the sandbox path back to the original path the malware named. A downstream analyzer uses these transforms to relabel the telemetry that the monitoring driver emits so that events name the malware’s intended targets rather than the sandbox locations; the same mechanism records, for network calls, the original destination the malware attempted (before redirection to loopback). Second, because process creation can be suppressed for safety, the tool separately records, for the two high-level creation entry points (CreateProcessW and CreateProcessA), the intended arguments—the application, command line, and working directory—as process events, preserving process-level intent without ever spawning a child. Neither feature runs anything additional: they record what the reproduced behavior would have named or launched, which is exactly what makes the generated telemetry both safe and faithfully labeled.

3.10. Intrinsic Design Limitations

Several limitations follow directly from the tool’s design and bound what any re-execution of a log can achieve; their empirical consequences are revisited in Section 7. Because input data buffers are reconstructed as zero-filled (the recorded payloads are not part of the log), a write or send call succeeds but writes zeroed content rather than the original bytes; likewise, received network data is the responder’s benign filler rather than a real server’s reply. Calls that operate across process boundaries—reading or writing another process’s memory, or allocating memory in another process—cannot be faithfully reproduced, because the target address space differs from the one recorded; the tool resolves the target process handle but cannot restore the foreign address, and treats such calls on a best-effort basis. In addition, Component Object Model (COM) calls—the inter-component interface used by, for example, scripting and office automation—are reproduced only partially: an object-creation call such as CoCreateInstance invokes the real COM API when the recorded class and interface identifiers can be parsed, but returns a class-not-registered result when they cannot, and one unsafe variant (CoCreateInstanceEx) is block-listed (Section 3.7). Windows Management Instrumentation (WMI, the system-management interface) has no dedicated executor and falls to the generic Layer 2 path. Finally, a handle that was opened before log capture began has no producing call to re-execute and therefore cannot be mapped. What a dependent call then does varies by executor. Some return a failure, and—as Section 3.8 described—such failures are re-examined and classified as approximate rather than as reproduction failures because they reflect a structural gap in the input log rather than a defect of the tool. Others return the success value the caller expects so that execution continues; these are counted as successes and never reach that re-examination. Because only failures are recorded as handle dependent, this second group is not counted separately; Section 4.5 bounds the effect of such reporting on the measured rate. These limits are intrinsic to reproducing behavior from an API-call log alone, and the evaluation is interpreted in light of them.

4. Materials and Methods

4.1. Dataset, Sample Selection, and Run Preparation

We evaluate on the WinMET dataset [5,6], a public collection of Windows malware execution traces recorded with the CAPEv2 sandbox. From it we select five malware families—AgentTesla, Amadey, Berbew, Dacic, and Redline. They were chosen to span not only malware types but the API-usage profiles that a re-execution tool must handle: Redline exercises file and cryptographic calls, AgentTesla interleaves NT-native and Win32 calls, Amadey is a comparatively simple downloader, and Berbew is a backdoor with a complex call pattern. The five thus cover the axes along which reproduction was expected to vary; they are a purposive rather than a random selection, and Section 7 states what this means for generality. Samples are assigned to a family by the dataset’s consensus family label—derived by the AVClass tool [20], which aggregates the differing labels of many anti-virus engines into a single agreed family name (for Dacic the sandbox’s own detection label is used instead, because “Dacic” is a CAPEv2-specific label for which AVClass has no corresponding entry). We additionally exclude traces with fewer than 50 recorded calls and duplicate SHA-256 hashes, both within and across families; seven cross-family duplicates are removed and replaced. For each family, we take the first 100 samples that satisfy these criteria in the order in which the dataset’s own label-to-report index—the same index that supplies the family labels—lists that family, giving 500 samples and 2,333,242 recorded API-call events for the main experiments.

Table 2. The evaluation corpus: five families, 100 samples each, with the total number of recorded API-call events per family.

Family	Malware Type	Samples	Recorded Events
AgentTesla	information-stealer	100	842,647
Amadey	downloader	100	1,215,839
Berbew	backdoor	100	22,369
Dacic	code-injection malware	100	202,678
Redline	information-stealer	100	49,709
Total		500	2,333,242

gives the per-family counts.

The ablation and telemetry experiments, which do not need all 500 samples, use the first 20 samples per family (100 samples in total); the determinism check, which re-runs each sample three times, uses a smaller subset of four samples per family (20 samples in total). Because selection follows a fixed listing order and is independent of any measured outcome, the corpus is fixed before any result is computed, which rules out outcome-dependent selection bias. It does not, however, make the corpus representative: taking the first eligible samples in that order rather than sampling at random leaves open an ordering bias, and we acknowledge it as such. That order is the dataset’s own listing order, and it is not aligned with how the dataset is stored: the five volumes partition the collection by SHA-256, so a sample’s volume follows from its hash rather than from when, from where, or under what sandbox configuration it was collected. Two checks bound what the listing order implies for the results. Within each family, selection position is essentially uncorrelated with volume (Pearson $\left|r\right|\le 0.09$) and shows no significant association with the per-sample reproduction rate (r from $-0.09$ to $+0.18$; $p>0.05$ in all five families). The five volumes also yield similar per-sample means (63.0% to 66.7%, a spread of 3.7 percentage points), with every family drawn from all five. The corpus is reproducible but not a random sample of its families.

Each sample’s trace is converted from the CAPEv2 report format into the tool’s input schema by a converter that gathers the call sequence from the processes at the root of the report’s recorded process tree—those whose parent process is not itself recorded—and orders it by timestamp, preserves the recorded call order, expands repeated calls into individual events, and normalizes the argument and return-value encodings—return values into a uniform hexadecimal form and statuses into a success/error pair. The converter also absorbs the field-name variations that occur across WinMET reports, such as the alternative spellings of the return-value, status, and thread-identifier fields, and records alongside the events the metadata the tool needs: the target process name, its bit-width, the time of capture, and the operating system on which the trace was taken. The tool then consumes the converted log and never reads the original malware binary. Unless otherwise noted, every run uses the same configuration—file, registry, and network sandboxing all enabled, single-threaded execution in recorded order, process creation suppressed, and a five-second per-call timeout—so that all experiments share one execution regime. One sample of the 500 is run with a one-second timeout; no call in that sample returned a timeout status, so the shorter limit never took effect. Before each sample is re-executed, the file and registry sandbox locations are cleared (the sandbox directory and the sandbox registry sub-tree are deleted) so that every run begins from an identical initial state.

4.2. Experimental Environment

Re-execution runs on an isolated Windows 10 virtual machine under Hyper-V (Microsoft’s hardware virtualization platform), driven from a Windows 11 host over PowerShell Direct (a control channel that reaches a virtual machine from its host without a network connection); no API call executes on the host.

The host carries an Intel Core i7-10750H processor at 2.60 GHz, 64 GB of memory, and solid-state storage, and the virtual machine is allocated six virtual processors and 4 GB of memory. Figure 4 shows this arrangement: the host holds the dataset and the analysis tools and only converts logs and analyzes results, whereas every recorded call is re-executed—and every real side effect confined—inside the virtual machine. The tool binary and the conversion and evaluation scripts were frozen for the entire campaign and their integrity verified (by matching file sizes and modification timestamps) so that every sample is processed by identical code. The safety measurements reported in Section 5.4 are taken on the virtual machine by comparing file-system, registry, and network state before and after each run.

4.3. Experimental Design and Research-Question Mapping

The evaluation comprises eight experiments that together answer the five research questions of Section 1.

Table 3. The eight experiments, their scale, principal metric, and the research question each addresses. “Same 500-sample run” indicates data collected together from one execution campaign.

Experiment	Samples	Principal Metric	RQ
Layer-1 ablation	100 (5×20)	handle-validity rate (full vs. baseline)	RQ1
Reproduction measurement	500 (5×100)	L1 BRR (by family, by category)	RQ2
Failure taxonomy	from 500-run data	True Failure Rate	RQ3
Side-effect safety	same 500-sample run	safety violations (target: zero)	RQ4
Execution cost	same 500-sample run	time per event, correlation with size	RQ4
Determinism (near-)	20 (4 per family)	event-level stability across runs	RQ4
Telemetry generation	100 (5×20)	Sysmon file/registry events (Event IDs 11/12/13/23), process/network intent	RQ5
Family classification	100 (5×20)	leave-one-out accuracy	RQ5

lists each experiment, the sample count it uses, its principal metric, and the research question it addresses.

The eight experiments do not require eight separate campaigns. Three of them—the reproduction measurement, the safety check, and the cost analysis—are collected from a single 500-sample run because safety and cost are properties of the very executions whose reproduction is being measured; taking all three from one run also means that they describe identical work rather than three similar but distinct campaigns. The failure taxonomy adds no runs at all: it re-examines the outcomes already recorded in that same run. The ablation and the telemetry generation each draw on the 100-sample subset, and the family classification then trains on whatever telemetry those runs produce, again without re-executing anything. Only the determinism check works on a smaller set still, the 20 samples of four per family. The sample counts differ because the questions differ in what they need: a reproduction rate is a proportion over calls and gains precision from every sample added, whereas an ablation compares two configurations on the same samples and a determinism check compares repetitions of one sample, so both answer at a scale where the cost of re-executing the whole corpus several times would buy little. Each of these sets keeps the families in equal proportion so that no single family dominates a measurement that is later reported per family. Three of the eight serve RQ4 alone, because a re-execution that is unsafe, unaffordable, or irreproducible would undermine the measurement no matter how faithful it was: safety, cost, and determinism each test a different way in which that could happen. Results are reported in Section 5 in an explanatory arc rather than in numerical experiment order, and each results subsection names the research question it answers. The four Sysmon Event IDs named in Table 3 identify the kinds of event the driver records: ID 11 marks the creation of a file, ID 12 the creation of a registry key or value, ID 13 a write to a registry value, and ID 23 the deletion of a file.

4.4. Experimental Procedures

Each experiment is carried out as follows.

Reproduction measurement. Every one of the 500 samples is re-executed once under the common configuration (Section 4.1); the per-call outcomes are recorded and aggregated into the metrics of Section 4.5. The safety and cost measurements below are taken during these same runs so that all three describe identical executions.

Layer-1 ablation. To isolate the contribution of the signature-defined executors, each of the 100 ablation samples is re-executed twice: once with the full tool, and once in a baseline configuration that disables Layer 1 (the –no-l1-executor option) so that every call falls to the Layer-2 generic dispatcher. The two runs are compared on the same samples using the handle-validity and return-match rates defined in Section 4.5; no reproduction rate is computed for the baseline since with Layer 1 removed, the notion of a Layer-1 domain does not apply.

Side-effect safety. After each run, three counts of effects outside the sandbox are recorded: files present outside the sandbox directory (enumerated under the user-profile tree, excluding the sandbox and experiment directories, and restricted to files whose last-write time falls within a window covering the sample’s timed re-execution and its dry run plus a thirty-second margin); registry entries outside the sandbox sub-tree (obtained by comparing a recursive listing of the user’s software key taken before and after the run); and established network connections whose remote address is not the loopback address. These three locations are the ones an escaped effect would surface in: because the re-execution requires no administrative privilege, as Section 3.7 explains, and redirects every file, registry, and network operation into a user-writable sandbox, an effect that escaped redirection would appear in the user-profile file tree, the user’s registry software key, or as a non-loopback network connection—the locations these three counts inspect. Whether an observed effect is induced by the re-execution or is pre-existing background activity of the operating system is determined when the results are analyzed in Section 5.4 so that only re-execution-induced effects are ultimately reported as violations.

Execution cost. Wall-clock time is recorded for each sample’s run and for a dry run of the same sample (Section 4.5), and time is related to the number of events per sample.

Determinism. Each of the 20 determinism samples is re-executed three times; because the sandbox locations are cleared before each run (Section 4.1), every run starts from the same initial state, and the three outcome sequences are compared event by event.

Telemetry generation. Each of the 100 telemetry samples is re-executed on the virtual machine while the Sysmon driver (v15.21, schema 4.90) is active under a fixed configuration that admits only events whose emitting image is the re-execution tool itself so that concurrent operating-system activity cannot enter the telemetry. Because the driver writes to the channel asynchronously, a run that finishes quickly can be exported before its events are on disk; the procedure therefore waits, after each sample, until the process-creation event that the tool itself emits at startup appears in the channel, and records a warning if it does not, before the log is exported. The file and registry events Sysmon emits are then collected, and their sandbox paths are relabeled to the malware’s intended targets using the tool’s transforms. The same result record additionally captures the process-creation arguments and network destinations recovered during the run (Section 3.9).

Family classification. From the generated telemetry we form several feature sets and train a random-forest classifier (200 trees, fixed random seed) to predict a sample’s family, evaluating each feature set by leave-one-out cross-validation (Section 4.7). The feature sets are:

• The per-category API rates already computed for the reproduction measurement of Section 5.2 (five features), used as the baseline;
• Sysmon event counts (four features);
• Those counts augmented with their normalized ratios and log-total (seven features, the Sysmon-summary set);
• A bag-of-words encoding of the 60 most frequent registry paths;
• A bag-of-words encoding of the 10 most frequent file paths;
• Combinations of these.

The primary feature set combines the Sysmon-summary features with the registry-path encoding (67 features in total) and is compared against the API-category-rate baseline; adding the file-path encoding on top of it is also evaluated but it does not improve accuracy, so file paths are not part of the primary set.

4.5. Reproduction Metrics

The primary metric is the Layer-1 Behavior Reproduction Rate (L1 BRR), the fraction of behavior-bearing (Layer-1) calls that re-execute successfully:

$$\mathrm{L}1\mathrm{BRR}={\displaystyle \frac{\mathrm{L}1\mathrm{success}}{\mathrm{L}1\mathrm{total}}},\mathrm{L}1\mathrm{total}=(\mathrm{success}+\mathrm{failed}+\mathrm{approx})-\mathrm{layer}2\text{\_}\mathrm{executed},$$

where success, failed, and approx are three of the four call outcomes defined in Section 3.8, and layer2_executed removes the calls counted outside the Layer-1 behavior domain—both those handled by the Layer-2 generic dispatcher and the utility calls treated as Layer-2 for measurement (Section 3.6)—so that L1 BRR measures only the modeled behavior domain.

Two further outcome groups are excluded from L1 total by construction. Calls that appear in the log but have no re-execution record (missing events) are reported separately as a completeness figure, so they neither inflate nor deflate the rate; and skipped calls—blocked terminations, generic-path block-list calls, dry-run calls, and unresolved Layer-2 calls (Section 3.8)—are deliberately not invoked and so count as neither successes nor reproduction failures. An approximate call, by contrast, is retained in L1 total: although Section 3.8 does not label it a reproduction failure, it is excluded from the numerator and therefore lowers L1 BRR, a deliberately conservative choice whose solvable part is restored by the Projected L1 BRR of Section 4.6.

Table 4. Metrics used in the evaluation. Primary metrics carry the reported claims; auxiliary metrics provide context only.

Metric	Definition	Used in	Role
L1 BRR	L1 successes ÷ L1-domain calls	reproduction	primary
Conservative L1 BRR	L1 successes, excluding substituted calls (Section 5.2) ÷ L1-domain calls	reproduction	primary
L1 Coverage Rate	L1-domain calls ÷ all calls	reproduction	primary
True Failure Rate	intrinsic failures ÷ L1-domain calls	failure analysis	primary
Handle-validity rate (hvr)	valid live handles ÷ handle-returning calls	ablation	primary
Return-match rate (rmr)	matching return values ÷ calls	ablation	auxiliary
Attack success rate (ASR)	all successful calls ÷ all calls	reproduction	auxiliary
Behavior-match rate (BMR)	calls whose outcome matches the record ÷ all calls	reproduction	auxiliary

defines all eight quantities used in the evaluation, the experiment each serves, and whether it is a primary measure (carrying a reported claim) or an auxiliary one (context only). Two conventions are load-bearing. First, L1 BRR is always reported together with the L1 Coverage Rate so that a reproduction rate is never read without the share of behavior it applies to. Second, the two whole-run success rates—the attack success rate (ASR) and the behavior-match rate (BMR)—are auxiliary only, never headline results: ASR counts every successful call, including utility calls outside the behavior domain, and BMR credits a call whenever a recorded failure is reproduced as a failure so that a tool doing nothing would score highly on it.

Four known biases make L1 BRR an upper-bound estimate of behavior reproduction;

Table 5. Measurement biases affecting L1 BRR. All are upward or neutral, so the reported rate is an upper-bound estimate of behavior reproduction.

Bias Source	Direction	Est. Magnitude	Design Reason
A registry-value query for which the log supplies no usable handle value returns zero (a “phantom” success)	upward	small (few APIs)	safe fallback when no handle is available
A file open request that opens an existing file or creates it if absent (an “open-if-exists” disposition) never meets a “not found” failure, because the sandbox tree is always fresh	upward	moderate	operations run against a fresh sandbox
Zero-filled input buffers reproduce the call but not its written contents	neutral to upward	small (content, not count)	reproducing written contents is out of scope
Calls whose outcome is reported without checking the underlying result	upward	bounded (≤+6.64 points)	the executor returns the status the caller expects
Combined effect of the first three: $+1$ to $+3$ points; the fourth is bounded, not sized.

lists them with their direction and estimated magnitude. All four push the measured rate upward or leave it unchanged. For the first three we estimate a combined effect of roughly one to three percentage points. The fourth we can bound but not size: for 89,877 Layer-1 calls, the executor returns the status that the caller expects without inspecting the outcome of the underlying operation—a memory release whose result is discarded, a process-handle open that falls back to the tool’s own process, and a wait that is satisfied without waiting when the handle cannot be resolved. Some of these calls do perform the recorded operation and some do not, and the record does not distinguish them; excluding all of them would lower L1 BRR by 6.64 percentage points, which is therefore an upper bound on this bias. The substituted calls described in Section 5.2 differ in exactly this respect: there the record shows the operation is not carried out, so they can be excluded outright rather than bounded. We interpret L1 BRR as an upper bound rather than as a point estimate.

4.6. Failure Taxonomy

Each Layer-1 call that does not succeed is assigned to exactly one of four causes. Handle-dependent failures arise when an NT-to-Win32 handle chain cannot be resolved; environment-difference failures arise from benign divergence between the capture and re-execution environments; data-gap failures arise when the log lacks information the call needs—in the implemented taxonomy, a multi-object wait whose recorded handle list is absent from the log; and true failures are the residual—calls that cannot reproduce for intrinsic reasons. These four causes refine the outcome scheme of Section 3.8: an approximate outcome caused by an unmapped handle is a handle-dependent failure, and one caused by a network- or registry-sandbox miss is an environment-difference failure; one caused by a missing recorded handle list is a data-gap failure; and a failed call not promoted to approximate is a true failure. Of these four, handle-dependent and data-gap failures are solvable by engineering the tool (better handle recovery and richer logs); environment-difference failures are solvable only by matching the capture environment, not by improving the tool; and true failure is fundamental. Because any call whose approximate-outcome note does not match a recognized solvable pattern is conservatively assigned to true failure, the True Failure Rate is biased slightly upward—it is an over-estimate of the genuinely irreducible fraction, not an under-estimate.

From this decomposition we also report a Projected L1 BRR, defined as (L1 success + handle-dependent + data-gap) ÷ L1 total: the rate that would be attained if the two barriers most clearly solvable by better handle recovery and richer logs—handle-dependent and data-gap failures—were eliminated. It serves as an engineering upper bound. Environment-difference failures are not added back, as they reflect genuine differences between the capture and re-execution environments rather than a tool limitation.

4.7. Statistical Methods

We report a 95% Wilson confidence interval for the two proportions on which a comparison rests—the Layer-1 BRR, in aggregate and per family (Section 5.2), and the leave-one-out classification accuracy (Section 5.7)—and report the remaining proportions as point estimates, since they describe the one corpus rather than support an inference from it. A Wilson interval is an interval, well behaved for proportions near zero or one and for small samples, within which the true proportion lies with 95% confidence. A Wilson interval on L1 BRR, however, treats the individual calls as independent observations, whereas calls are clustered within samples and samples within families; it therefore states how precisely the rate is known over the calls in this corpus, not how far it would move under a different draw of samples. For L1 BRR we therefore also report a 95% bootstrap interval obtained by resampling whole samples with replacement, stratified by family, over 10,000 replicates. The two answer different questions and we give both rather than replacing one with the other: the Wilson interval bounds sampling error at the call level, the bootstrap interval bounds it at the sample level.

For the family-classification experiment of Section 5.7 we compare the primary feature set against a baseline feature set. Accuracy is estimated by leave-one-out cross-validation: with n samples, the classifier is trained on $n-1$ samples and tested on the one held out, repeated so that every sample is tested exactly once, and the accuracy is the fraction of held-out samples classified correctly. Where the execution cost of Section 5.4 is related to the size of a log, we report two coefficients side by side: Pearson’s r, which measures how close the relation is to a straight line, and Spearman’s $\rho$, which measures only whether one quantity rises with the other, irrespective of the shape of the relation. To test whether the two feature sets differ, we apply McNemar’s test—the standard test for comparing two classifiers on the same samples, which examines only the samples they classify differently—using the continuity-corrected form of the statistic, which slightly reduces the test’s tendency to over-report significance on small samples. Because leave-one-out folds share $n-2$ of their training samples and are therefore not statistically independent, McNemar’s test is mildly anti-conservative here (its p-value is optimistic—more likely to indicate a difference than the data strictly warrant); we therefore treat the p-value as directional evidence rather than as a strict significance threshold, following published cautions on cross-validation-based testing [21,22].

The classifier uses features derived from the file and registry paths in the generated telemetry, encoded as bag-of-words vectors (each distinct path token becomes one feature, whose value is its count). Such text-derived features risk information leakage—a test sample influencing its own features—if the set of tokens (the vocabulary) is chosen using the test sample. To prevent this, the path vocabulary is built exclusively from each fold’s training samples so that a held-out sample’s paths never influence which features exist.

4.8. Use of Generative AI

Claude Code (version 2.1.268, Anthropic, San Francisco, CA, USA) was used to draft and revise the source code of WinAPIReplay, of the collector, and of the conversion and evaluation scripts, to write their unit tests, and to run the campaigns of Section 4.4 from the host machine. Claude Opus 5 (Anthropic) was used to draft and revise the text of this manuscript and to generate the confusion matrices of Section 5.7. No tool was given authority over a result: every figure, table and number reported here was produced by the released code from the recorded logs, and the authors checked each against that output.

5. Results

5.1. Necessity of the Layer-1 Executors: An Ablation Study (RQ1)

To test whether the per-category executors are responsible for reproduction, the first of the eight experiments of Section 4.3 compares the full tool against a Layer-2-only configuration in which the executors are disabled and every call falls to the generic dispatcher. The comparison is a paired ablation over 79 sample pairs: of the 100 ablation samples, the 20 AgentTesla baseline runs all terminated with an unhandled access violation and produced no comparable pair, and one Dacic sample likewise produced none, leaving 79.

Table 6. Ablation: full tool versus a Layer-2-only baseline (79 paired samples). Rates are computed over the handle-returning calls (handle-validity) and over all calls (return-match) of each configuration.

Metric	Full Tool	Layer-2-Only Baseline
Handle-validity rate	94.9% (945/996)	10.2% (82/804)
Return-match rate	69.8% (99,566/142,639)	0.3% (445/130,907)

shows that the executors are decisive. The handle-validity rate falls from 94.9% with the full tool to 10.2% without the executors, and the return-match rate from 69.8% to 0.3%; without the signature-defined executors almost no handle chain survives and almost no return value matches. The reproduction reported below is therefore produced by the executor design, not by the generic dispatch fallback. The AgentTesla collapse sharpens the point: without the executors that family does not merely reproduce less, it cannot be re-executed to the end at all. (We do not report an L1 BRR for the baseline since with the executors removed the notion of a Layer-1 domain is not meaningful).

5.2. Layer-1 Behavior Reproduction Rate, by Family and Category (RQ2)

Over the 500 samples, the aggregate L1 BRR is 74.04%, computed over 1,352,778 Layer-1 calls of which 1,001,634 reproduce. Resampling whole samples with replacement, stratified by family, gives a 95% bootstrap interval of [71.02, 76.75]; the Wilson interval over calls is [73.97, 74.12]. The bootstrap interval is some thirty-eight times wider, and it is the one that reflects how far the figure would move under a different draw of samples. The Layer-1 domain covers 57.98% of all calls, and the per-sample mean L1 BRR is 65.20% (SD 14.04). The two differ because the aggregate rate pools every Layer-1 call so that the samples carrying the most calls dominate it, whereas the per-sample mean weights every sample alike. The pooling is uneven: within Dacic the three largest samples supply 46.0% of that family’s Layer-1 calls, and within Redline 29.9%, so a handful of long traces carries much of the aggregate. Across the 500 samples the per-sample rate ranges from 23.29% to 97.71%, with quartiles at 59.57%, 62.36% and 76.92%—a spread the aggregate figure alone does not convey. We therefore report the aggregate rate as the headline figure, since it states what fraction of recorded behavior reproduces, and the per-sample distribution alongside it as the rate a single sample attains. Where a single number is quoted below, it is this aggregate rate; the conservative rate defined next is its lower counterpart under a stricter reading of what counts as reproduced, and the two are stated together wherever the result is summarized. Missing entries—logged calls with no re-execution record—amount to 26,208 calls (1.1% of all recorded calls) and are reported separately as a completeness figure rather than folded into the rate.

Not every call counted as reproduced results in the operation the malware recorded. Where an operation cannot be carried out safely, or where the log does not carry the data it needs, the tool answers the call with a substitute and reports the return value the caller expects (Section 3.5, Section 3.7 and Section 3.10).

Table 7. Calls counted as reproduced whose recorded operation was not carried out, by reason.

Reason the Operation Was Not Carried out	Calls	Share of L1 Successes (%)
Process and shell creation suppressed	3382	0.34
Thread created with a placeholder start routine	4002	0.40
Cross-process memory access not performed	10,890	1.09
Hook installation answered with a sentinel	91	0.01
Service start reported without being attempted	114	0.01
Thread-attribute update not performed	40	0.00
Data-carrying calls issued with substitute buffers	15,014	1.50
Opens of device paths and named pipes short-circuited	5644	0.56
Total	39,177	3.91

separates these calls from the rest. They number 39,177, or 3.91% of the 1,001,634 successes. Excluding all of them yields a conservative L1 BRR of 71.15% (95% bootstrap CI [68.15, 73.88]; Wilson [71.07, 71.22]), against 74.04% when they are counted. We report both: the first states what fraction of recorded behavior was carried out as recorded, the second what fraction of recorded calls the tool was able to answer.

Table 8. Layer-1 reproduction by family (500 samples). Two intervals are given for L1 BRR: a 95% Wilson interval over calls and a 95% bootstrap interval resampled over samples. ASR and BMR are reported as auxiliary quantities.

Family	L1 Cov. (%)	L1 BRR (%)	Wilson CI	Bootstrap CI	ASR (%)	BMR (%)
AgentTesla	63.51	70.46	[70.34, 70.58]	[65.84, 75.44]	76.00	76.30
Amadey	54.55	77.80	[77.70, 77.90]	[73.95, 80.25]	85.31	76.16
Berbew	21.73	60.06	[58.68, 61.43]	[59.79, 60.35]	89.53	89.98
Dacic	58.56	73.81	[73.55, 74.05]	[57.57, 84.26]	82.76	85.12
Redline	62.09	58.66	[58.11, 59.21]	[51.64, 63.99]	70.33	75.84
Overall	57.98	74.04	[73.97, 74.12]	[71.02, 76.75]	81.44	77.11

shows that reproduction varies by family.

Amadey reproduces best (77.80%) and Redline worst (58.66%); Berbew combines the lowest Layer-1 coverage (21.73%) with a moderate reproduction rate, reflecting a call mix dominated by utility calls that fall outside the Layer-1 domain. Figure 5 shows the family breakdown: Amadey alone exceeds the overall rate, and Berbew’s Wilson interval is the widest, since its samples contribute only 4860 Layer-1 calls against 663,222 for Amadey. These family figures are descriptive: we report each with the two intervals of Table 8 but do not draw inferential family-to-family comparisons, which are out of scope. Reproduction also varies sharply by API category: process operations reproduce at 95.9% (549,627 of 573,218), Winsock networking at 89.2% (58 of 65), file operations at 80.5% (98,882 of 122,841), registry operations at 58.2% (269,193 of 462,839), and DLL operations at only 15.9% (18,451 of 115,820). Figure 6 shows these five categories, which together account for 94.2% of all Layer-1 calls; of the remainder, synchronization reproduces at 76.1% over 52,494 calls and cryptography at 99.95% over 25,187, while the three other categories that occur at all contribute fewer than 130 calls each. The DLL category sits some 42 percentage points below the next lowest. The file and registry rates matter most for the telemetry application, and both are substantial. One caveat qualifies the process figure: under the safety configuration of Section 3.7, process- and shell-creation calls are answered with a synthesized success and no child process is spawned. They number 3382, the first entry of Table 7, and are counted as reproduced because the caller receives the return value it originally recorded; the intent they carry is preserved separately as the process events of Section 3.9 rather than as a running child.

5.3. The Residual Failures: A Four-Way Taxonomy (RQ3)

Table 9. Failure taxonomy over 1,352,778 Layer-1 calls. Bold marks the projected rate that would follow if the solvable causes were removed.

Outcome	Count	Share of L1 (%)
Reproduced (success)	1,001,634	74.04
Handle-dependent failure	2268	0.17
Environment difference	7	0.00
Data gap	0	0.00
True failure	348,869	25.79
Projected L1 BRR (solvable removed)		74.21

decomposes the residual, applying the taxonomy of Section 4.6 to the 1,352,778 Layer-1 calls. True failure—intrinsic non-reproducibility not removable by better handle recovery or richer logs—accounts for 348,869 calls (25.79%). The engineering-solvable barriers are small by comparison: handle-dependent failures number 2268 (0.17%), environment differences 7, and data gaps 0. Removing the solvable barriers would raise reproduction only to a Projected L1 BRR of 74.21%.

The practical reading is that the 74% figure is close to what this approach can reach on this dataset without information beyond the log itself: the shortfall is dominated by behavior that an API-call log alone cannot reconstitute, not by fixable gaps in the tool.

One qualification bounds this decomposition. The environment-difference test of Section 3.8 is keyed on the Win32 registry entry points, whereas the corpus is dominated by their native counterparts: of the 193,646 registry calls that fail under re-execution, none reaches that test, and all are therefore counted as true failures. Because these calls read a sandbox sub-tree that is empty by construction, some share of them are environment differences rather than intrinsic non-reproducibility. The 121,832 whose original call succeeded bound that share from above, at 9.01% of Layer-1 calls; the true-failure figure is accordingly an upper bound and the solvable-barrier figures lower bounds. Reclassifying all 121,832 as environment differences—the most favorable reading—moves the True Failure Rate from 25.79% to 16.78% and the environment-difference share from 0.00% to 9.01%; the Projected L1 BRR is unchanged at 74.21%, because the projection adds back only handle-dependent and data-gap failures (Section 4.6). The true-failure fraction therefore lies between 16.78% and 25.79%. The Layer-1 BRR is unaffected either way since failed and approximate calls enter the rate identically. The decomposition is stated on the same basis as the headline L1 BRR. Its four causes classify the Layer-1 calls that did not succeed, so the substituted calls of Table 7 are not among them; they sit in the reproduced row. Restating the decomposition on the conservative basis would require a category of its own for those 39,177 calls, which we do not attempt here. What the projection bounds—the gain from removing the solvable barriers—is unchanged either way: 0.17 percentage points, since the conservative reading leaves the solvable-barrier counts unchanged.

5.4. Completeness of Side-Effect Isolation (RQ4)

Safety. 

Table 10. Side-effect safety over 500 samples. Re-execution operations are intercepted and redirected into the sandbox; re-execution-induced escapes are effects that reached outside it; background activity is non-re-execution operating-system activity observed on the virtual machine during the campaign.

Domain	Intercepted & Redirected	Re-Execution-Induced Escapes	Background (Non-Re-Execution)
File	122,841	0	23 file operations
Registry	462,839	0	0
Network	65	0	645 non-loopback connections

gives the counts: across all 500 samples, the number of re-execution-induced side effects that escaped the sandbox was zero. The tool intercepts and redirects 122,841 file operations, 462,839 registry operations, and 65 Winsock network operations, with no registry activity of any kind observed outside the sandbox sub-tree. The file and network activity observed on the virtual machine during the campaign (23 file operations and 645 non-loopback connection records) is attributable to background system activity rather than to the re-execution: the file modifications occur in six samples, while the non-loopback connections—traffic from pre-existing Windows operating-system processes—are observed across 241 of the 500 samples. Side-effect isolation is therefore complete on the file, registry and network channels for every sample in the study.

Cost. The mean wall-clock time per sample is 5406 ms and the median 994 ms (a dry run averages 441 ms), corresponding to roughly 1.17 ms per re-executed call in aggregate. Cost is not a function of log length. Runtime rises with call count in rank order (Spearman $\rho =0.79$) but not in proportion to it (Pearson $r=0.325$ on the untransformed values), and Figure 7 shows why: on logarithmic axes the samples separate into one cluster per family rather than falling along a single line. AgentTesla and Amadey record almost the same number of events at the median (6964 and 6143) yet differ twenty-one-fold in time (22,206 ms against 1056 ms), because a call of the kind AgentTesla makes costs seventeen times as much: 2.49 ms per event at the median against 0.14 ms. A second structure is visible as horizontal bands near five and ten seconds, where 54 samples—42 of them Redline—sit at multiples of the five-second per-call timeout of Section 3.5; for these samples the time is set by a single waiting call rather than by the length of the log. At roughly one second per sample in the median, the approach is practical for corpus-scale application.

5.5. Near-Determinism of Re-Execution (RQ4)

We ask whether an identical input under an identical initial state yields an identical outcome sequence. Executing 20 samples (four per family) three times each, most samples are not byte-identical across all three runs—live handle values differ from run to run—but the output is near-deterministic: 98.90% of individual call outcomes are stable across runs (685 of 62,507 events vary), per-sample agreement ranges from 95.09% to 100%, with a mean of 98.51%, and the aggregate reproduction metric varies by at most 0.41 percentage points across runs (80.17%/80.51%/80.10%). Critically, the residual variation is almost entirely localized to handle-returning calls: 684 of the 685 varying events involve handle values, which the operating system assigns differently on each run, and only a single varying event lies outside that class. Fourteen of the twenty samples meet a near-determinism criterion (at least 98% event agreement, residual confined to handle calls, and per-sample reproduction-metric swing below 0.5 percentage points); three (all from the Berbew set) are fully byte-identical. The re-execution is thus not strictly deterministic—because live handle values inevitably differ—but it is stable at the level of reproduced behavior, which is what downstream measurement and telemetry generation depend on.

5.6. Automatic Generation of Endpoint Telemetry (RQ5)

Running the re-execution for 100 samples under a live Sysmon driver produces 46,150 file/registry telemetry events without executing any malware binary: 4938 file-create events (Event ID 11), 40,192 registry key/value-create events (12), 212 registry-value-set events (13), and 808 file-delete events (23). One hundred of these events (0.22%) record the tool writing its own result file rather than reproduced malware behavior; the count is reported as the driver emitted it, without post-hoc filtering. File or registry telemetry was produced for all 100 samples.

Table 11. Endpoint telemetry generated per family (100 samples). The Event-ID columns and Total are Sysmon file/registry events (IDs 11/12/13/23); Proc. (process-creation events) and Net. (network destinations) are recovered in the tool’s result record, where all 220 process events reconstructed successfully.

Family	Ev. 11	Ev. 12	Ev. 13	Ev. 23	Total	Proc.	Net.
AgentTesla	2950	25,689	54	3	28,696	29	1
Amadey	931	9361	79	312	10,683	5	0
Berbew	123	800	60	0	983	0	0
Dacic	601	3384	1	16	4002	179	0
Redline	333	958	18	477	1786	7	0
Total	4938	40,192	212	808	46,150	220	1

gives the volume by family, which varies from 28,696 events for AgentTesla to 983 for Berbew. Registry key and value creation (Event ID 12) dominates every family, and Berbew alone produced no file-delete events at all, which is consistent with its behavior as a backdoor rather than a file-modifying payload. Because operations execute under sandbox redirection, 4015 events (8.7%) carry sandbox paths that the transforms table correct back to the malware’s intended targets, yielding telemetry labeled with the paths the malware named rather than the sandbox paths. The result is family-labeled, path-correct file/registry telemetry generated safely from static logs.

The same run also records, in the tool’s result record, the malware’s original process-creation and network intent, which the safety measures strip from the Sysmon stream. It captures 220 process-creation events—the intended application, command line, and working directory, every one successfully reconstructed—across 50 of the 100 samples, the code-injection family Dacic contributing the most (179; Table 11), and, through the same transforms mechanism, the original network destinations for the samples that contacted the network (a single destination in this corpus, whose network activity is sparse: the Winsock category comprises just 65 operations, Section 5.2). This intent is preserved separately because process creation is suppressed and network traffic is redirected to a loopback responder, so the corresponding Sysmon events—Event ID 1 (process creation) and Event ID 3 (network connection)—would otherwise carry the tool’s own image and the loopback address rather than the malware’s. Merging this recovered intent with the file and registry events yields family-labeled endpoint telemetry that also reflects the malware’s intended process and network behavior; the classification experiment below uses only the file/registry telemetry.

5.7. Family Classification from the Generated Telemetry (RQ5)

To test whether the generated telemetry is discriminative, we train a random-forest family classifier on features derived from it and evaluate by leave-one-out cross-validation over all 100 samples; every sample produced telemetry in this run, so the five families contribute 20 samples each and the per-family figures below rest on equal numbers of samples. The primary feature set—combining the Sysmon-summary features with a registry-path bag-of-words encoding whose vocabulary is built inside each fold—reaches 92.0% leave-one-out accuracy (F1 92.0%; 95% Wilson CI [85.0, 95.9]), against 94.0% (F1 94.0%) for a baseline of per-category API rates;

Table 12. Family-classification accuracy by feature set (leave-one-out CV, 100 samples). Path-derived sets use per-fold vocabularies. Bold marks the primary feature set, against which the others are compared. The primary set is compared to the API-category-rate baseline by continuity-corrected McNemar.

Feature Set	Dim.	LOO acc. (%)	F1 (%)	95% CI
API-category rates (baseline)	5	94.0	94.0	[87.5, 97.2]
Sysmon event counts	4	93.0	93.0	[86.3, 96.6]
Sysmon summary	7	92.0	91.9	[85.0, 95.9]
File paths	10	78.0	77.2	[68.9, 85.0]
Registry paths	60	91.0	90.9	[83.8, 95.2]
Sysmon summary + registry (primary)	67	92.0	92.0	[85.0, 95.9]
Sysmon summary + registry + file	77	91.0	91.0	[83.8, 95.2]

gives the accuracy for every feature set. Adding a file-path encoding does not improve on this and is omitted from the primary set. A continuity-corrected McNemar test comparing the primary set to the baseline gives ${\chi }^2=0.167$, $p=0.683$, with two samples classified correctly only by the primary set and four only by the baseline: the two feature sets are statistically indistinguishable at this sample size, and with 100 samples the test remains underpowered. We therefore report the path-aware telemetry as carrying family-level information that we cannot distinguish from that of the API-category rates it is generated from, while offering richer, human-interpretable features, rather than claiming an advantage in either direction.

A sensitivity check separates two factors that could account for these figures: the telemetry itself and the sample set. An earlier collection procedure yields telemetry for only 79 of the 100 samples, so we evaluate the re-measured telemetry on that same 79-sample subset as well as on all one hundred. The API-category baseline uses no telemetry, so it isolates the effect of the sample set: it scores 91.1% on both the old and the re-measured 79-sample runs and 94.0% on all one hundred. Against that control, the registry-path feature set falls from 96.2% to 94.9% when only the telemetry changes, and from 94.9% to 91.0% when the sample set is enlarged—a 3.9-point effect attributable to the sample set alone. The 79-sample subset is therefore favorable to that feature set, and the balanced set of one hundred samples, twenty per family, is the basis for the figures above.

Figure 8 shows the confusion matrices for the three feature sets that bracket the result—the primary set, the primary set extended with file paths, and the baseline—which the figure abbreviates as BC_T, BCE_T and D. These matrices come from five-fold cross-validation rather than from the leave-one-out procedure used above, and at this sample size the two schemes do not agree: five-fold gives 93.0%, 92.0% and 92.0% for the three sets, reversing the ordering that leave-one-out produces. That the ordering is not stable under a change of resampling scheme is itself consistent with the McNemar result, and we read the matrices as showing where the errors fall rather than as a second estimate of accuracy. Per family, five-fold F1 for the primary set is 100% (Berbew), 95.0% (AgentTesla), 91.9% (Amadey), 90.5% (Redline), and 87.8% (Dacic). Berbew is classified without error by all three sets, and no sample of any family is ever assigned to it. The primary set misplaces seven samples and the baseline eight; four of these are the same samples, so relative to the baseline the primary set removes four errors and introduces three. Adding file paths does not simply extend the primary set’s errors: it resolves one of them and introduces two others, an Amadey and a Redline sample both drawn toward Dacic. The generated telemetry is therefore useful for classification, which closes the loop from static log to safely generated, discriminative endpoint data.

6. Discussion

6.1. Interpreting the 74% Reproduction Rate

The headline finding is not that a quarter of behavior is irreproducible but that the reproducibility of dynamic-analysis logs is now measured rather than assumed, and that the measured value is both substantial and well understood. Three results together make 74.04% interpretable. The ablation of Section 5.1 shows the figure is produced by the executor design, not by chance or by generic fallback. The failure taxonomy of Section 5.3 shows the shortfall is dominated by intrinsic, environment-dependent behavior (25.79% true failure) rather than by solvable engineering gaps (a Projected L1 BRR of only 74.21%). The clearest case is the DLL category, which reproduces at only 15.9%: a recorded procedure-address lookup (LdrGetProcedureAddressForCaller, among the most frequent calls in AgentTesla, Dacic, and Redline) resolves its recorded module handle against the libraries actually present in the re-execution environment, and reports a procedure-not-found status whenever the module or procedure it names cannot be located there. Library references are deliberately not redirected into the sandbox, as Section 3.5 describes, so this is a dependence on the original environment that tool engineering alone cannot remove.

Two disclosures also bound the figure from above: the measurement biases of Table 5, and the 39,177 calls of Table 7 that were answered without the recorded operation being carried out. Excluding the latter gives 71.15%. The gap between 71.15% and 74.04% is not sampling error but a difference of definition—whether a call answered by a substitute counts as reproduced behavior—and we report both figures rather than choose between them. The contribution is thus a quantitative map—how much reproduces, where it varies by family and category, and why the residual fails—rather than a claim of complete reproduction.

6.2. Re-Execution Versus Static Transformation for Telemetry (C4)

A natural objection to the telemetry application is that one could statically transform an API-call log into Sysmon-formatted records without executing anything. We argue that re-execution earns its cost precisely where static transformation is hardest. Sysmon telemetry is a function not only of the observed activity but also of the collection environment and the deployed Sysmon configuration. Consequently, statically transforming an API-call log into Sysmon-formatted records that are consistent with a given environment and configuration would require re-implementing (i) the configuration-dependent filtering, field selection, and rule-tagging performed by Sysmon, and (ii) the environment’s runtime-only fields (for example the process GUID, precise timestamps, and emitting image) together with the kernel’s state-dependent event-firing semantics—whether a given NtCreateFile raises a file-create event depends on success, disposition flags, and pre-existence. Such a static model is laborious and must be re-derived whenever the environment or the Sysmon configuration changes. WinAPIReplay instead inherits this dependency directly: running the re-execution inside the target virtual machine under a specific Sysmon configuration lets the real Sysmon driver emit telemetry consistent with that environment and configuration automatically, with no hand-crafted model, and, because the tool itself never reads that configuration, re-running under a changed one adapts the output without additional engineering.

Two caveats bound this claim, and they connect back to the measurement. First, operations execute under three-tier sandbox redirection, and their paths are corrected post hoc via the transforms table, so the telemetry reflects the virtual-machine execution environment and configuration rather than the original victim’s pre-existing on-disk and registry state. This is the same mechanism that produces one of the L1 BRR biases we disclosed in Section 4.5: because operations run against a fresh sandbox tree, an open-if-exists file disposition never encounters the “already exists” condition it might have met on the victim, which both slightly inflates reproduction success and shapes which file-create events Sysmon emits. Second, only successfully reproduced file/registry operations yield events; operations that fail to re-execute produce none. The 100 samples together produced 46,150 events across Event IDs 11/12/13/23. We therefore claim environment- and configuration-consistent telemetry for the reproduced subset of behavior, not an identical reconstruction of what detonating the original binary would produce. The trade-off is explicit: re-execution costs more than static transformation and requires a running virtual machine with Sysmon, in exchange for automatic environment/configuration fidelity over the reproduced subset.

6.3. Implications for Detection-Model Training

The classification result (Section 5.7) indicates that the generated telemetry is discriminative enough to support family-level analysis, and the path-aware features that drive it are human-interpretable. Because the telemetry is produced without detonating malware, the approach offers a safe route to enlarging labeled Sysmon corpora for training family classifiers and other machine-learning detectors, under a chosen Sysmon configuration and in a repeatable manner. This complements, rather than replaces, live-detonation telemetry: it trades some fidelity (the reproduced subset, the sandbox-environment caveat) for safety, reproducibility, and configuration control.

6.4. Safety and Stability of Re-Execution

Two supporting results underpin the measurement and telemetry contributions: the re-execution is safe, and it is stable. Both bear on whether the approach can be trusted and reused.

Safety is what makes the method usable at all. Across the 500 samples, the tool carried out and redirected 122,841 file, 462,839 registry, and 65 Winsock operations, and no side effect escaped the sandbox (Section 5.4). This validates the three-tier containment as a design rather than a hope: each effect channel the tool issues is redirected before it reaches the host, so real side effects occur only inside disposable locations, which is exactly what lets Sysmon observe genuine telemetry (Section 5.6) without exposing the host. The result is nonetheless bounded by what the sandbox models: the zero-escape figure establishes complete containment of the effect channels the tool redirects over this corpus, not a guarantee for an input that reaches the host through some channel the tool does not model—a scope we detail in Section 7.

Stability determines whether the measurement and the generated telemetry are reproducible. The re-execution is not strictly deterministic—live handle values are assigned afresh on each run—but it is near-deterministic where it matters: 98.90% of call outcomes are stable across repeated runs, and 684 of the 685 varying events are handle values rather than reproduced behavior (Section 5.5). Because the aggregate reproduction metric moves by at most 0.41 percentage points across runs, the 74% figure and the family-labeled telemetry are repeatable quantities rather than artifacts of a single run—a property that downstream training pipelines require. The cost of obtaining both is modest: a median of roughly one second per sample. It is not the longest logs that dominate the total but the most expensive family—the fifty slowest samples account for 54.1% of the campaign’s running time and forty-nine of them are AgentTesla, whereas the fifty longest logs account for only 16.9%—so a corpus whose composition is known can be costed from its family mix rather than from its size.

6.5. Relation to Prior Art

The two bracketing lines of work remain complementary. Recording frameworks such as MALVADA/WinMET [4,5,6] supply the logs we consume; instruction-level replay systems [8,9,10] reconstruct executions from binaries, which we never touch. The testing-oriented API-replay of US 7,380,235 [15] shares the idea of reissuing recorded calls, and reconstructs some call context of its own, but it works from resources extracted from the original binary and provides no containment, no fidelity measurement, no handle chains spanning the NT-native and Win32 layers, and no telemetry generation. Positioned among these, WinAPIReplay contributes the missing capability of measuring and exploiting the reproducibility of API-call logs themselves.

7. Limitations and Threats to Validity

Fidelity ceiling. Reproduction is bounded by intrinsic, environment-dependent behavior: 25.79% of Layer-1 calls are true failures, so the tool measures and generates a reproduced subset, not the whole of a sample’s behavior. Telemetry generation inherits this ceiling.

Dataset-specific handle gaps. A structural feature of the WinMET traces is that some object handles are created before capture begins and thus never appear as a producing call; such handles cannot be mapped, so calls that depend on them cannot reproduce. This is a property of the input corpus, not of the executor design; the failures it directly causes are captured by the handle-dependent category (recovered in the Projected L1 BRR), but the gap bounds what any re-execution of these logs can achieve.

Near-determinism, not determinism. Because the operating system assigns live handle values afresh on each run, output is generally not byte-identical across runs—the exceptions being samples that produce no live handle values; we demonstrate near-determinism (98.90% event stability, residual confined to handle calls, aggregate metrics within 0.41 percentage points) rather than strict determinism.

Telemetry scope. The faithful, malware-labeled Sysmon telemetry is limited to file and registry events (Event ID 11/12/13/23), whose sandbox paths the transforms correct back to the malware’s intended targets. The corresponding process-creation (Event ID 1) and network-connection (Event ID 3) events instead carry the tool’s own image and a loopback destination, because process creation is suppressed and traffic is redirected for safety; the malware’s intended process arguments and network destinations are preserved in the tool’s result record, so process-name- and C2-address-based detection must consume that record rather than the Sysmon stream directly.

Statistical power. The classification comparison uses 100 samples, and its McNemar test is both underpowered and, because leave-one-out folds are not independent, mildly anti-conservative; we therefore report the path-aware telemetry as statistically indistinguishable from the API-category-rate baseline, not as an improvement on it.

Environment scope. Side-effect operations execute against a sandbox on a single virtual-machine configuration, so both the reproduction measurement and the generated telemetry reflect that environment rather than the original victim’s state, and the disclosed measurement biases, together with the 39,177 calls answered without the recorded operation being carried out (Table 7), make L1 BRR an upper-bound estimate of behavior reproduction. Containment is likewise scoped to the effect channels the tool models—file, registry, and network operations, together with the process- and thread-termination calls on the block-list—so an effect reaching the host through an unmodeled channel would fall outside its guarantee, even though none did across the 500 samples. It is worth stating which channels those are. Named kernel objects are the one case in which this corpus exercises an unsandboxed channel: the 6205 recorded mutant and event calls are re-executed with their original names, so the objects are created in the system-wide namespace rather than under a sandbox root, and although the operating system releases them when the re-execution process exits, they are visible to other processes while it runs. The remaining channels are either neutralized or absent. Process, thread, and shell launches (7444 calls) are suppressed by the no-spawn policy and are listed in Table 7; the 125 service-control calls never reach the service database, because service start reports success without an attempt and no service is ever created; opens of device paths and named pipes are short-circuited, and registry operations are rewritten into the sandbox sub-tree, both of which the zero-escape measurement covers; of the 69 Winsock calls, the 20 that would open or address a connection are redirected to a loopback responder and the rest are library initialization and socket closure; and 3764 calls are refused outright, these being the two block-listed termination routines that the corpus actually invokes. COM activation is attempted against the real subsystem but never succeeded here (0 of the 38,549 calls actually executed); the attempt itself, however, initializes the COM subsystem, and the telemetry run recorded sixteen key-creation events under a WMI configuration key in eight samples—a subsystem side effect that the before-and-after comparison of Section 5.4 does not register as a persistent change, but that a monitor watching the process does see. Device-control requests—the only route by which this corpus could reach symbolic links or junctions—are dispatched generically with zeroed arguments (3949 calls). Named pipes, scheduled tasks, symbolic-link creation, and alternate data streams do not occur in the corpus at all, so the zero-escape result says nothing about them.

Generality. The evaluation covers five families and one dataset; the specific rates should not be extrapolated to other families, corpora, or Windows versions without further study. The tool also presupposes that API-call logs are already available—it consumes logs, it does not acquire them.

8. Conclusions

Dynamic-analysis pipelines routinely treat a recorded API-call log as a faithful specification of a sample’s behavior, yet how much of that behavior reproduces had not been measured. In this paper, we have proposed WinAPIReplay, which re-executes pre-obtained Win32 and NT-native API-call logs as real operating-system calls—without ever running the malware binary—using a cross-layer handle map that carries live handles between the two API layers and a three-tier sandbox that redirects file, registry, and network effects; across 500 samples every side effect stayed contained, with zero escapes on the channels the tool models and monitors—Section 7 states which channels those are, and which this corpus does not exercise at all. Using this tool, we have provided what is, to the best of our knowledge, the first quantitative measurement of dynamic-log reproducibility: a Layer-1 Behavior Reproduction Rate of 74.04%—that is, 74.04% of modeled Layer-1 behavior-domain calls reproduced, the Layer-1 domain covering 57.98% of all recorded calls, with a 95% bootstrap interval over samples of [71.02, 76.75]. The measurement biases we disclose are upward or neutral, so this is an upper-bound estimate (Table 5). Counting only calls whose recorded operation was actually carried out lowers the figure to 71.15%, and we report both. An ablation attributes this figure causally to the signature executors rather than to a generic fallback; a four-way failure taxonomy shows the residual (25.79% true failures) is dominated by intrinsic, environment-dependent behavior—for instance, procedure-address lookups that fail when the module or procedure they name is absent from the re-execution environment—rather than by fixable tool gaps, and the re-execution is near-deterministic, with variation confined to freshly assigned handle values, and inexpensive enough to apply at corpus scale.

Beyond measurement, we have shown that running the re-execution under Sysmon safely generates family-labeled file- and registry-telemetry from static logs alone—inheriting the deployed environment and Sysmon configuration rather than re-implementing them, and recovering the malware’s process and network intent in the tool’s result record—and that this telemetry, for the reproduced subset of behavior, is statistically indistinguishable from an API-category baseline in discriminating malware families (92.0% leave-one-out accuracy). WinAPIReplay is therefore both a measurement—a map of how far current API-call logs can be trusted as behavior specifications, most of whose shortfall is intrinsic rather than fixable—and a capability: a safe, configuration-controllable route to family-labeled, environment-consistent endpoint telemetry that complements, rather than replaces, live detonation. Its reach is bounded by the reproduced subset, a single sandbox environment, and five families on one corpus; future work includes recovering pre-capture handles—whose attainable gain the failure taxonomy bounds to the Projected L1 BRR of 74.21% since the residual is dominated by intrinsic failure—extending the evaluation across more families, datasets, and Windows versions, and studying how configuration-controlled generated telemetry transfers to detection tasks.