LLM-Assisted Porting of Security-Critical C Libraries to Idiomatic Rust: A Multi-Model Empirical Study

Abstract

Memory-safety vulnerabilities remain the dominant class of security defects in C/C++ software underpinning Internet infrastructure. Rust offers a structural solution through its ownership system, yet migrating existing codebases remains costly. This paper defines a structured methodology for LLM-assisted porting of security-critical C libraries to idiomatic Rust and applies it to cJSON (∼3200 LOC, 14 CVEs). A manual expert porting serves as the baseline; five LLMs (Claude Opus 4.6, Gemini 3 Pro, GPT-5.4, Kimi K2.7-Code, and Qwen3.5-27B) produce independent portings in agentic mode. Verification uses an end-to-end pipeline: CVE-specific tests, coverage-guided and differential fuzzing (>1.7 billion executions), Miri analysis, and comparative benchmarking. All six portings eliminate all in-scope CVE classes by construction, with zero unsafe blocks and zero memory-safety crashes. In this case study, structural safety holds consistently across all five evaluated models and across all five Kimi repetitions, whereas code quality varies widely (0–9 residual bugs). Because only Kimi was repeated ($N=5$), its variance bounds run-to-run noise at the 95% confidence level, against which some but not all between-model differences are distinguishable from chance; a single porting attempt costs approximately $3 in API usage. Differential fuzzing reveals complementary bugs in the manual and LLM portings, supporting a hybrid workflow, and we translate these findings into concrete practical guidance for teams planning a similar migration. These results are scoped to one compact, single-threaded C library. The entire codebase and evaluation pipeline are publicly released.

1. Introduction

Memory-safety vulnerabilities, including buffer overflows, use-after-free errors, double frees, and data races, constitute the most prevalent class of security defects in software written in C and C++. Data published by the Microsoft Security Response Center indicate that approximately 70% of security vulnerabilities patched annually in Microsoft products are attributable to memory-safety issues [1]. An analogous analysis by the Chromium project confirmed the same proportion among high-severity security bugs in Google Chrome [2]. Because C and C++ power a vast portion of the software that underpins today’s Internet, from JSON parsers and TLS libraries to DNS resolvers and embedded firmware, the security implications extend well beyond individual applications to the reliability of networked systems at large.

The persistence of these vulnerability classes has prompted formal institutional action. CISA, NSA, and partner agencies published a joint report in 2023 recommending the adoption of memory-safe languages [3]. In February 2024, the White House elevated the issue to the executive level [4], and in June 2025, CISA and NSA identified memory-safe language adoption as the most comprehensive mitigation against memory vulnerability classes [5]. The strategic relevance of C-to-Rust translation is further confirmed by DARPA’s TRACTOR program (Translating All C to Rust), launched in 2024, which aims to develop AI-driven tools for automated conversion of legacy C codebases to idiomatic Rust [6].

Rust [7,8] provides a structural solution: its ownership and borrowing system, verified statically by the compiler, eliminates, at compile time, the conditions that produce use-after-free, double-free, and data races. The RustBelt project [9] formally verified that safe Rust code cannot exhibit undefined behavior. Adoption in security-critical contexts has accelerated: the Linux kernel integrated Rust as a second development language [10], Google reported that the introduction of Rust in Android reduced memory-safety vulnerabilities from 76% in 2019 to under 20% in 2025 [11,12,13] (Figure 1), and the Binder driver rewrite achieved performance within ±2% of C [14]. Notably, the first CVE assigned to Rust code in the Linux kernel (CVE-2025-68260) was a race condition in explicitly unsafe code, confirming that Rust eliminates traditional memory-safety classes but not concurrency bugs in unsafe regions.

Migrating existing C codebases to idiomatic Rust remains costly. Automated transpilers such as c2rust [15] produce non-idiomatic, predominantly unsafe code with formally demonstrated limits [16,17]. Manual porting requires deep expertise and considerable effort. Large Language Models (LLMs), built on the Transformer architecture [18], have demonstrated promising code-generation capabilities [19], and recent industrial applications have shown their potential for legacy-code migration, from COBOL modernization [20] to the Bun runtime’s Zig-to-Rust rewrite [21]. However, their effectiveness for the porting of real-world, security-critical C libraries, with documented vulnerabilities, to idiomatic Rust has not been systematically evaluated through multi-model comparison with an expert baseline and end-to-end security verification.

This paper addresses this gap with four contributions: (1) a structured methodology with a fully released, reusable verification pipeline; (2) a quantitative evaluation on cJSON (14 CVEs, Common Vulnerabilities and Exposures identifiers, the standard public catalog used to track disclosed software vulnerabilities), comparing a manual expert baseline against five LLM-assisted portings; (3) an intra-model variance study ($N=5$ on Kimi K2.7-Code) that yields an empirical statistical bound on run-to-run noise, used to assess which single-run cross-model differences are distinguishable from chance; and (4) concrete practical guidance for a team planning a similar migration (Section 5.7), including an analysis of self-correction triggers across all five models (Section 5.2) showing that self-generated test-suite quality, rather than model identity or generation time, is the strongest predictor of which semantic bugs get caught before release, going beyond a single “best model” recommendation to a verification-centered workflow.

We structure the empirical investigation around four research questions, to which we return explicitly in the Results and Discussion sections.

• RQ1 (Structural safety). Do LLM-assisted Rust portings of a security-critical C library eliminate the library’s documented memory-safety vulnerability classes by construction, and does this hold consistently across models that differ in scale, architecture, and deployment mode? This question is addressed by the per-CVE verification and fuzzing results in Section 4 and revisited in Section 5.
• RQ2 (Code quality and performance). How do the correctness, idiomaticity, and runtime performance of LLM-generated portings compare to one another and to a manual expert baseline, and what factors are associated with the observed differences? This question is addressed in Section 4 and Section 5, including the self-correction analysis and the manual-versus-LLM complementarity study.
• RQ3 (Stability). How much of the observed variation in code quality and performance across models reflects genuine model-level differences, as opposed to stochastic variation inherent to a single agentic generation attempt? This question motivates the Kimi $N=5$ intra-model variance study and the statistical treatment described in Section 3.5 and reported in Section 4.
• RQ4 (Practice). Given the above, what verification workflow and practical guidance follow for a team planning an LLM-assisted C-to-Rust migration? This question is addressed directly in Section 5.7.

We emphasize, at the outset, that this is a single-library, single-family case study; Section 3 details this scope choice, and Section 5 discusses its implications for generalizability alongside the other limitations raised during review.

The remainder of this paper is organized as follows. Section 2 reviews prior work on memory-safe languages and automated and LLM-assisted C-to-Rust translation and positions this study relative to it. Section 3 describes the target library, the dual manual-versus-LLM methodology, the experimental setup for the five models, the four-axis verification pipeline, the statistical treatment applied to the repeated-run data, and the study’s threats to validity. Section 4 reports the empirical findings for each of the four verification axes, the cross-model comparison, and the intra-model variance study, organized around RQ1 through RQ3. Section 5 interprets these findings and discusses their practical implications (RQ4), the challenges of scaling the methodology to industrial deployment, and the broader implications for Internet infrastructure security. Section 6 summarizes the findings that are directly supported by the experiments, distinguishes them from our interpretations and from the study’s limitations, and outlines future work.

2. Related Work

This section situates the present study within three bodies of prior work: the case for memory-safe languages in general and Rust in particular, including the institutional and industrial momentum behind that shift; automated and LLM-assisted approaches to C-to-Rust translation specifically, from early, non-idiomatic transpilers to the rapidly advancing 2025–2026 generation of LLM-based tools; and, finally, a positioning of the present study relative to this literature, arguing that it trades breadth for depth: rather than a large benchmark of many targets evaluated once, it applies an unusually deep verification methodology, namely per-CVE security analysis, differential fuzzing, Miri, benchmarking, and repeated-run variance measurement, to a single, well-understood target.

2.1. Memory Safety, Rust, and Institutional Momentum

The characterization of memory-safety vulnerabilities as an “eternal war in memory” [22] remains apt. Mitigations at the OS and compiler level, such as Address Space Layout Randomization (ASLR, which randomizes where a program’s code and data are placed in memory to make exploits harder to construct), stack canaries (sentinel values placed before a function’s return address to detect stack-buffer overflows before they can hijack control flow), and Control-Flow Integrity (CFI, which restricts a program’s execution to a set of legitimate control-flow paths determined ahead of time), reduce exploitability but do not prevent defect creation: the underlying memory-safety bug is still present in the code; it is only harder to weaponize. Runtime analysis tools such as AddressSanitizer [23] detect memory bugs dynamically by instrumenting memory accesses at compile time and checking them against a shadow map of valid regions; in safe Rust, they are unnecessary because the compiler prevents these bug classes statically, before the program ever runs. Rust eliminates these vulnerability classes structurally through its ownership system [8], with formal guarantees [9]. An exhaustive study of 186 Rust CVEs confirmed that all memory-safety vulnerabilities require the presence of unsafe code [24]. Li et al. [25] analyzed 240 Linux kernel CVEs (2020–2024), finding that over 80% would be addressable by Rust.

2.2. Automated and LLM-Assisted C-to-Rust Translation

The c2rust transpiler [15] produces compilable but non-idiomatic, predominantly unsafe code. Laertes [16], Crown [26], and C2SaferRust [27] attempt post-transpilation refinement but face formal aliasing limits [17]. Yang et al. [28] combined LLM generation with formal verification via WebAssembly reference models (VERT), improving verified translations from 1% to 42%.

Pan et al. [29] studied bugs introduced by LLMs during code translation (2.1–47.3% success). Eniser et al. [30] found that simple compiler-feedback loops outperform complex agentic strategies. Li et al. [31] conducted the first user study on C-to-Rust translation; 20 of 31 participants using LLM assistance reported the generated code was error-prone.

In 2025–2026, the field advanced rapidly. CRUST-Bench [32] introduced the first C-to-safe-Rust benchmark (100 repositories; best one-shot of 22%, rising to 48% with repair). SafeTrans [33] achieved 80% correct translations with memory-safety vulnerability elimination. SACTOR [34] used FFI-based equivalence verification (93% on TransCoder-IR). SmartC2Rust [35] achieved a 100% test pass rate on 21 C programs. EvoC2Rust [36] introduced skeleton-guided project-level translation (100% compilation). Rustine [37] translated repositories up to 13,200 LOC at $0.48/project. Tadesse et al. [38] demonstrated that improvements in one quality dimension systematically cause regressions in others. Figure 2 summarizes these reported success rates.

2.3. Positioning of This Work

This study distinguishes itself through depth rather than scale: a single real-world target (cJSON, ∼3200 LOC) is ported by five LLMs and one human expert, with per-CVE security verification, differential fuzzing, Miri analysis, benchmarking, and intra-model variance measurement ($N=5$).

Table 1. Comparison of approaches to C-to-Rust translation. In bold the results related to our research.

Approach	Correct.	Idiom.	Scale	Security	Multi-Model	Open
c2rust [15]	High	Low	High	No	N/A	Yes
VERT [28]	Verified	Med.	Low	No	No	Yes
SafeTrans [33]	80%	High	Med.	Implicit	No	No
SACTOR [34]	84–93%	High	Med.	FFI-based	Yes	Yes
SmartC2Rust [35]	100%	High	Med.	Implicit	No	Yes
Rustine [37]	87%	High	Project	No	No	Yes
This work	Verified	High	Med.	Explicit	Yes (5 + N = 5)	Yes

summarizes the comparison.

3. Materials and Methods

This section describes the empirical methodology in six parts: the target library, cJSON, and the justification for its selection, including the consequence of that choice for what the results can and cannot be taken to show; the dual manual-versus-LLM methodology that structures the whole study; the five evaluated language models, their interfaces, and the experimental infrastructure; the four-axis verification pipeline of functional correctness, security, idiomaticity, and performance, applied uniformly to all six portings, together with an explanation of why each procedure provides appropriate evidence for the claims made in Section 4 and Section 5; the statistical treatment applied to the repeated-run data; and, finally, a systematic discussion of the study’s threats to validity.

3.1. Target Library: cJSON

cJSON [39] is an ultra-lightweight JSON parser (conforming to RFC 8259 [40]) written in ANSI C, consisting of a single source file (cJSON.c, ∼3200 LOC). It was selected because: (i) it is widely deployed (over 10,000 GitHub dependents), powering JSON processing in IoT devices, embedded systems, and Web APIs; (ii) it has 14 documented CVEs (Common Vulnerabilities and Exposures, 2016–2025) covering all major memory-safety classes; (iii) its size fits within current LLM context windows, meaning every model could read and reason about the entire source file in a single pass rather than needing to work from a partial or summarized view; and (iv) one CVE (CVE-2016-10749) was discovered by one of the authors [41].

Table 2. Classification of cJSON’s 14 CVEs and Rust prevention mechanisms.

Vulnerability Class	Count	Example CVE	Rust Prevention
Buffer overflow/OOB write	4	2016-4303	Bounds checking on Vec/slice
Buffer over-read	3	2016-10749	&str has known length; no raw pointers
Use-after-free	1	2018-1000217	Ownership: single owner, no aliasing
Double free	1	2018-1000216	Ownership: no explicit free()
NULL pointer dereference	4	2019-1010239	Option<T>; exhaustive pattern matching
Memory leak	1	2018-1000215	RAII: automatic Drop

classifies the CVEs.

The porting scope covers the core module (parser, printer, tree manipulation, and comparison). Excluded are cJSON_Utils (one CVE), the global state (eliminated by design), and custom allocator hooks. With these exclusions, 11 of 14 CVEs fall within scope; three portings (Claude, Codex, and Kimi) also implement cJSON_Minify, covering 2 additional CVEs.

cJSON was deliberately chosen as a compact, single-file, single-threaded, dependency-free library: this keeps the entire source within every evaluated model’s context window in a single pass and keeps the manual and automated verification pipelines tractable within the resources available for this study. This choice has a direct and important consequence for how the results should be read: because every model saw the whole file at once, none of the five LLMs had to solve the harder problem of porting code it could only partially observe, for example, by summarizing, chunking, or making cross-file assumptions about a codebase too large to fit in context. For a library an order of magnitude larger or split across many files with cross-module dependencies, models would need a fundamentally different multi-pass or repository-level strategy (Section 5.6 discusses concrete approaches from the literature), and the speed, cost, and code-quality figures reported here should not be assumed to hold in that regime. The corollary, discussed further in Section 5 is that the reported findings characterize LLM-assisted porting of a library of this profile and should not be read as a claim about larger, multi-module, concurrent, or cryptographic codebases; Section 5 outlines the concrete steps required to extend the methodology to such targets.

3.2. Dual-Approach Methodology

The evaluation adopts a dual approach (Figure 3): a manual expert porting as baseline and five independent LLM-assisted portings. The two pipelines share the same target architecture (algebraic enum type, zero unsafe, and error handling via Result) and converge on a four-axis evaluation. Because every porting starts from this same brief, we can compare how far each one drifted from it in practice: we define a divergence, reported later in Section 4, as a design or implementation choice in a given porting that departs from this shared target architecture, for example, choosing Option instead of Result for error handling, or restructuring the suggested module layout; divergences were identified through manual comparative code review across the six portings rather than through an automated diff. Divergence counts are therefore a measure of how much a porting’s author, whether human or model, deviated from the common starting brief, not a measure of correctness or quality on their own; Section 4 and Section 5 discuss what the pattern of divergences across models does and does not indicate.

3.3. LLM Models and Experimental Setup

Five LLMs were employed (

Table 3. Experimental setup.

Model	Interface	Context	Deploy	Date	Runs
Claude Opus 4.6	Claude Code v2.1	200 K	Cloud	Feb 2026	1
Gemini 3 Pro Preview	Gemini CLI v0.29.5	1 M	Cloud	Feb 2026	1
Qwen3.5-27B	OpenCode + Ollama	64 K	Local	Mar 2026	1
GPT-5.4 (Codex)	Codex CLI v0.120.0	n/a	Cloud	Apr 2026	1
Kimi K2.7-Code	OpenCode + Moonshot	n/a	Cloud	Jun 2026	5

). All received an identical structured prompt inspired by chain-of-thought prompting [42], specifying the task, target architecture, constraints (zero unsafe, comprehensive testing), and autonomous operation (an abridged version is reproduced in Appendix A, Listing A1). Physical isolation was enforced through separate Git worktrees. Each model operated in an agentic loop, generating code, then compiling it; running its own tests; running Clippy; and revising its output in response to whatever it observed, without further human input, until it judged the task-completion constraints in the prompt to be satisfied. We define a self-correction, reported later in Section 4 and discussed in Section 5.2, as one such observe-and-revise cycle, categorized by what triggered it (a compilation error, a failing test, a Clippy warning, or a semantic issue the model identified through its own reasoning rather than a tool), and counted from the per-model DEVLOG session logs recording each model’s agentic session. The complete prompts issued to every model, together with the full interaction transcripts, prompt-refinement notes, and per-model DEVLOG session logs, are released in the public repository (see the Data Availability Statement) to support exact replication of the experimental conditions.

All six portings were produced and evaluated on the same machine, a MacBook with an M4 Max chip and 36 GB of memory, using Rust 1.93.1 (stable) and 1.95.0 (nightly, required only for the tools below). Two specialized tools underpin the verification pipeline described next. Miri [43] is an interpreter for Rust’s mid-level intermediate representation that executes a program symbolically and flags undefined behavior, such as out-of-bounds memory access or violations of Rust’s aliasing rules, that would otherwise only manifest as a crash under specific, hard-to-reproduce conditions; because it does not rely on the triggering of a crash to find a problem, it can catch classes of bugs that ordinary testing misses. Fuzzing was performed with cargo-fuzz 0.13.1 [44], a Rust wrapper around the libFuzzer [45] engine that repeatedly generates and mutates random inputs to a target function, searching for inputs that trigger a crash or a Miri-detected violation; criterion.rs 0.5.1 was used for statistically robust performance benchmarking.

3.4. Verification Pipeline

Every porting, manual and LLM-generated alike, was passed through the same four-axis pipeline so that all six could be compared on equal footing. The four axes, functional correctness, security, idiomaticity, and performance, were chosen because they correspond to logically distinct failure modes that a single check cannot jointly rule out: code can pass its own tests yet still contain an unpatched CVE class (motivating the security axis as distinct from functional correctness); code can be memory-safe yet unidiomatic and hard to maintain (motivating idiomaticity as distinct from security); and code can be correct, safe, and idiomatic yet commercially unviable if it regresses performance by an order of magnitude relative to the C original (motivating the performance axis). Correctness and security, together, provide the evidence for RQ1; all four axes jointly provide the evidence for RQ2.

Functional correctness. Each porting was checked against two test suites: the tests the model (or, for the manual baseline, the author) generated during development and a shared suite of regression tests built from the 13 in-scope CVEs described above. The pass criterion was strict: 100% of both suites had to pass, with no partial credit. We required both suites, rather than either alone, because they provide different and complementary evidence. The model’s own suite reflects what that model considered worth testing and therefore indicates whether the model understood its own implementation; relying on it exclusively, however, would let a model that wrote few or weak tests appear to “pass” without ever exercising the historically vulnerable code paths. The shared CVE suite closes that gap by holding every porting to the same, author-independent floor of regression coverage for the specific vulnerabilities cJSON has actually had in the past, so a pass on this suite is evidence that a porting does not silently reintroduce a previously disclosed bug, independent of how thorough its own author’s testing happened to be.

Security. For each of the 13 in-scope CVEs, we first verified, by structural inspection, that the Rust code eliminates the vulnerability class by construction, meaning that the language’s own guarantees, not an ad hoc runtime check the porting’s author happened to add, make the bug class unrepresentable (for example, that a buffer-overflow CVE is prevented by Rust’s compiler-enforced bounds checking on Vec and slice indexing rather than by a manually inserted length check that could, itself, contain an off-by-one error). This structural judgment for each CVE is recorded in Table 2 and

Table 4. Per-CVE elimination verification. ✓ = eliminated; N/P = function not ported. The table has 12 rows covering 13 distinct CVE identifiers: the row labeled “2019-11834/35” reports two CVEs (both addressed by the same bounds-checking mechanism in cJSON_Minify) together, since they share an identical outcome across all six portings.

CVE-ID	Class	CVSS	Rust Mechanism	Man.	Cl.	Gem.	Qw.	Cdx.	Kimi
2016-4303	Heap overflow	9.8	Hex valid. in parse_hex4	✓	✓	✓	✓	✓	✓
2016-10749	Buffer over-read	9.8	Bounds check, no raw ptr	✓	✓	✓	✓	✓	✓
2018-1000215	Memory leak	7.5	RAII: String/Vec, Drop	✓	✓	✓	✓	✓	✓
2018-1000216	Double free	8.8	Ownership: no free()	✓	✓	✓	✓	✓	✓
2018-1000217	Use-after-free	9.8	&str + to_string()	✓	✓	✓	✓	✓	✓
2019-1010239	NULL deref	7.5	Option<&T>, no null	✓	✓	✓	✓	✓	✓
2019-11834/35	OOB in Minify	9.8	Bounds on bytes.len()	N/P	✓	N/P	N/P	✓	✓
2023-26819	Stack overflow	2.9	No fixed buffer; parse::<f64>()	✓	✓	✓	✓	✓	✓
2023-50471	NULL deref	7.5	Exhaustive match on enum	✓	✓	✓	✓	✓	✓
2023-50472	NULL deref	7.5	Non-object → false	✓	✓	✓	✓	✓	✓
2023-53154	Heap over-read	5.5	&str has known length	✓	✓	✓	✓	✓	✓
2024-31755	NULL deref	7.6	&str cannot be NULL	✓	✓	✓	✓	✓	✓

. We paired this static, per-CVE reasoning with three independent dynamic verification methods on the basis of the view that no single method is individually sufficient: a static argument can miss an interaction the reviewer did not anticipate, and a single fuzzing campaign can miss a bug an unlucky random search never happens to trigger, but the three methods below have largely independent blind spots, so a bug that survives all three is unlikely to be a mere gap in one verification modality. First, we confirmed, by inspection, that no porting contains an unsafe block anywhere in its source, the only mechanism through which Rust allows a programmer to bypass its safety guarantees; this is a binary, unambiguous check; a single unsafe block would have failed it, regardless of whether that block happened to be safe in practice. Second, we applied coverage-guided fuzzing, running each porting against cargo-fuzz for 3600 s per target, using code-coverage feedback to steer input generation toward unexplored code paths; the pass criterion was zero crashes and zero sanitizer-detected memory violations across the campaign. Third, we applied differential fuzzing, which feeds the same randomly generated input to two implementations at once and flags any case where their outputs diverge; we used this both to compare each porting against the original C implementation, with results reported later in Section 4, and, for the manual-versus-Claude case study in Section 5.3, to compare two independently produced Rust portings directly against each other, which surfaces semantic disagreements that neither implementation’s own test suite or C-referenced fuzzing run happened to trigger. Finally, every porting was checked with Miri (defined above), whose pass criterion was zero reported violations; because Miri detects undefined behavior by symbolic execution rather than by waiting for a crash, it can catch violations that are latent (correct on every input the fuzzer happened to generate but still technically undefined) and that the other three methods, all of which depend on either static reasoning or on a crash actually being triggered, could not.

Idiomaticity. Code quality and Rust idiom adherence were assessed with Clippy, Rust’s standard linter, which flags stylistic and idiomatic issues beyond what the compiler itself checks (for example, unnecessary cloning of data or a manual loop where an iterator method would be more idiomatic); the criterion we report is the count of warnings on the final, released version of each porting, with zero being the baseline all six portings ultimately reached. Clippy’s automated checks were supplemented by qualitative pattern review; a manual, comparative reading of how each porting solved the same design problem (Section 4 illustrates this with the divergent error-handling patterns different portings chose for object-key lookup); and simple structural metrics such as lines of code, used as a coarse proxy for solution complexity and verbosity rather than as a quality measure in its own right. We include this axis because the passing of tests and elimination of CVEs are necessary but not sufficient conditions for code a development team would actually want to maintain: idiomatic Rust is easier for a Rust-literate team to review, extend, and reason about than a literal, non-idiomatic translation of the original C control flow, even when the two are behaviorally equivalent, which is directly relevant to RQ2 and to the practical guidance in Section 5.7.

Performance. Each porting was benchmarked with criterion.rs (100 samples per measurement, reporting a 95% confidence interval, i.e., the range within which the true mean runtime is expected to fall with 95% probability under repeated measurement) against the original C implementation, using 10 representative JSON test files chosen to vary in size (from tens of bytes to several kilobytes) and structure (flat objects, nested objects, and numeric arrays) so that the comparison is not dominated by a single file’s idiosyncrasies. We report the geometric rather than arithmetic mean of the per-file ratios, which is the standard aggregation for ratio data, since it weights a $2\times$ slowdown on one file and a $2\times$ speedup on another as canceling out, whereas an arithmetic mean would not. We include this axis and report it separately from correctness and security because a migration that is safe and correct but an order of magnitude slower than the original would be difficult to justify commercially, regardless of its safety benefits; Section 4 shows that this was a real, not merely hypothetical, risk in our data.

3.5. Statistical Treatment

The experimental design imposes a hard constraint on inferential statistics: four of the five models were run once ($N=1$), so no paired significance test (e.g., a paired t-test or Wilcoxon signed-rank test) can be computed across models without conflating between-model differences with within-model stochastic variation. We therefore restrict formal inferential statistics to the one setting where repeated measurement exists, the Kimi K2.7-Code $N=5$ study, and use the resulting variance estimates as an empirical, model-derived bound on the run-to-run noise that a single-run comparison cannot separate from genuine between-model differences. For the Kimi repetitions, we report the mean, sample standard deviation, and 95% confidence interval (Student’s t-distribution, 4 degrees of freedom) for LOC, generated tests, residual bugs, and the parse and print performance ratios (Section 4.5). We report this openly as a limitation rather than a resolved concern: it bounds but does not eliminate the risk that some of the between-model differences reported in Section 4 reflect sampling noise rather than genuine model-level effects; Section 5 returns to this point.

3.6. Threats to Validity

We organize the study’s limitations around the four validity categories conventionally used in empirical software engineering research, construct, internal, external, and conclusion validity, since each captures a logically distinct way the reported findings could be misleading.

Construct validity concerns whether our operationalization of “structural memory safety” actually measures what we claim it measures. We operationalize it as the conjunction of zero unsafe blocks (verified by inspection) with a clean record across coverage-guided fuzzing, differential fuzzing, and Miri (Section 3.4); this is well grounded in prior work, which found that every one of 186 real-world Rust memory-safety CVEs required the presence of unsafe code [24], so the absence of unsafe is a strong, literature-supported proxy for the absence of this vulnerability class rather than a construct we invented for this study. A more specific concern is circularity: the prompt itself required zero unsafe instances (Section 3.3), so compliance with that instruction alone would be an unsurprising, almost tautological result, since a model can trivially satisfy “use no unsafe” by refusing to use it. The non-trivial empirical result and the one we intend RQ1 to capture is not that the models complied with the instruction but that the resulting unsafe-free code was also functionally correct on the shared CVE suite and survived over 1.7 billion fuzzing executions and Miri analysis without incident; an assistant could, in principle, have satisfied the letter of the prompt with broken or incomplete functionality, and none did.

Internal validity concerns whether some factor other than genuine C-to-Rust reasoning could explain the observed outcomes, the most salient candidate being training-data contamination. cJSON has been public since 2011, so all five models may have encountered its source, and possibly fragments of existing third-party Rust ports of similar small JSON libraries, during training; a model could, in principle, reproduce memorized code rather than reasoning about the port from first principles, which would inflate the apparent correctness and safety without reflecting general C-to-Rust translation capability. We cannot rule this out directly, since we have no way to inspect the models’ training data. The strongest evidence we can offer against wholesale memorization is architectural: the six portings diverge substantially from one another (782–2216 LOC, 8 to 18+ structural divergences from a shared reference architecture; see Section 4) and make different, sometimes incompatible design choices, for example, the error-handling divergence documented in Section 4 (typed Result in the manual porting versus Option or bool in every LLM porting); this pattern is inconsistent with five models converging on a single memorized reference implementation, though it does not rule out contamination at the level of smaller, reusable idioms and patterns, which remains an open confound we cannot fully exclude.

External validity concerns the extent to which these findings generalize beyond the specific setting studied. The study relies on a single case study, i.e., cJSON, a compact, single-file, single-threaded, dependency-free library selected, in part, because it fits entirely within every evaluated model’s context window (Section 3); results may not transfer to larger, multi-module, concurrent, or cryptographic codebases such as OpenSSL, mbedTLS, SQLite, libpng, or libexpat, which differ from cJSON not only in scale but in threading model, API surface, and the context-management strategies a model would need to port them at all. We view this as the study’s most consequential limitation, and Section 5 returns to it directly when discussing industrial deployment and generalizability.

Conclusion validity concerns whether the statistical and comparative claims we draw are properly supported by the data. Four of the five LLM portings are based on a single run ($N=1$); only the Kimi K2.7-Code study includes repetition ($N=5$, Section 3.5), which we use to derive an empirical bound on run-to-run stochastic noise rather than to draw a general conclusion about Kimi specifically. This asymmetry means that the cross-model comparisons reported in Section 4 are properly read as descriptive rather than statistically significant: with $N=1$ per model for four of the five models, no paired significance test can be validly computed across them, and we report none. Where the Kimi-derived noise envelope is narrower than an observed between-model gap, we treat that gap as more likely to reflect a genuine model-level difference than sampling noise (Section 4); where the two are comparable in magnitude, we treat the comparison as inconclusive, pending repeated runs of the other four models, which we identify as necessary future work (Section 6) rather than a gap this study resolves.

4. Results

This section reports the empirical findings, organized around the three research questions introduced in Section 1. The first two subsections below, Functional Correctness and Security and Idiomaticity, together, address RQ1, whether structural memory safety is achieved consistently across models. Performance and Multi-Model Comparison address RQ2, how code quality and runtime performance compare across models and against the manual baseline. The Intra-Model Variance Study addresses RQ3, how much of the observed between-model spread reflects genuine differences rather than stochastic noise. We interpret these findings jointly and in relation to prior work and practical implications in Section 5.

4.1. Functional Correctness and Security

All six portings achieve 100% pass rates on their respective test suites and on the shared CVE test suite, which covers all 13 of the 14 documented cJSON CVEs that fall within the porting scope defined in Section 3 (11 core-module CVEs tested against all six portings, plus two further CVEs specific to cJSON_Minify, tested only against the three portings that implement it; the one CVE outside this scope is the excluded cJSON_Utils vulnerability; Section 3). Table 4 presents the per-CVE verification: all vulnerability classes are eliminated by construction, without the models knowing about any CVE. The mechanisms are intrinsic to Rust (bounds checking, ownership, and Option<T>).

Fuzzing confirms these structural guarantees: zero memory-safety crashes across over 1.7 billion executions collectively and zero undefined behavior under Miri on the five portings whose test suites were large enough to give Miri meaningful code coverage (Miri checks the specific code paths a test suite exercises, so a result on Gemini’s comparatively small six-test suite would reflect too little coverage to be informative and is excluded from this claim on those grounds, not because a violation was found; Kimi was clean across all five variance runs).

4.2. Idiomaticity

All six portings present zero Clippy warnings in their final versions, so automated linting alone does not distinguish them; the more informative evidence is qualitative. Listing 1 illustrates a pattern that recurs throughout the codebase: for object-key lookup, the manual porting returns a typed Result, using Rust’s error type to distinguish a missing key from a type mismatch, while all five LLM portings instead return Option or bool, mirroring the null-or-sentinel-value convention of the original C API. This is not an isolated stylistic accident; it is the single largest, most consistent pattern difference we observed between the manual and LLM portings, and it went the same direction in all five independent LLM portings despite the five models otherwise diverging substantially in architecture, LOC, and structure, as the multi-model comparison later in this section shows.

We read this pattern as evidence that the models’ default translation strategy stayed closer to the original C library’s own API surface than the human expert’s did: cJSON’s original functions signal absence or failure with NULL or a boolean return, and Option/bool are the most direct structural analog of that convention in Rust, whereas Result requires the porting’s author to design a new error taxonomy that has no direct counterpart in the C original; this is a genuinely idiomatic redesign rather than a structural translation. Both choices are memory-safe, and both passed every test in our pipeline, so this is a code-quality and API-design observation rather than a correctness or security finding; a caller cannot distinguish “key not found” from “wrong type” through the LLM portings’ Option-returning interface the way they can through the manual porting’s Result-returning one, which is the kind of downstream usability cost a team adopting LLM-generated Rust should expect to review and, if desired, correct.

Listing 1. Error-handling comparison for object-key lookup.

1   2 // Manual porting : typed Result   3 pub fn get_object_item (& self , key : &str ) -> Result <& JsonValue > {   4      if let JsonValue :: Object ( obj) = self {   5            for (k, v) in obj {   6                 if k == key { return Ok(v); }   7           }   8           Err ( Error :: KeyNotFound { key : key . to_string () })   9       } else { 10           Err ( Error :: TypeError { expected : " object ", 11                                            found : self . type_name () }) 12       } 13  } 14 15  // Claude porting : Option (C- faithful ) 16  pub fn get_item (& self , key : & str ) -> Option <& JsonValue > { 17       match self { 18            JsonValue :: Object ( items ) => items . iter () 19                 . find (|(k, _)| k. eq_ignore_ascii_case ( key)) 20                 . map (|(_, v)| v), 21            _ => None , 22       } 23  }

4.3. Performance

Figure 4 presents per-file parsing times. To keep the per-file plot legible, it shows the original C implementation alongside four representative portings (Manual, Claude, Gemini, and Qwen); Codex and Kimi are omitted from this specific breakdown, since, with six series across ten test files, the chart becomes difficult to read, but both are included, along with all other portings, in the aggregate geometric-mean comparison in

Table 5. Performance ratios relative to C (geometric mean). Claude post-fix; Kimi canonical run. Values < 1.0× = faster than C.

Operation	Manual	Claude	Gemini	Qwen	Codex	Kimi
Parse	0.98×	0.99×	1.25×	1.29×	0.90×	0.94×
Print formatted	0.95×	0.67×	0.83×	1.50×	0.74×	0.76×
Print compact	0.94×	0.67×	0.82×	1.18×	0.66×	0.75×

, which covers the complete set of six.

Performance-bug case study. The initial Claude porting contained an $O\left(n^2\right)$ pattern in string parsing: per-character std::str::from_utf8() calls on the entire remaining input. A 12-line fix reduced the geometric mean from 2.12× to 0.99×, exemplifying the recommended hybrid workflow.

Interpretation. Three patterns in Table 5 and Figure 4 are worth drawing out explicitly, beyond the raw ratios. First, the direction of the effect is not uniform across models and not even uniform within a single slower model: on parsing specifically, Manual, Claude, Codex, and Kimi all land within a few percentage points of parity with C (0.90×–0.99×), while Gemini (1.25×) and Qwen (1.29×) are both 25–29% slower. On the two printing operations, however, Gemini is competitive with or faster than C (0.82×–0.83×, in the same range as Manual), so its slowdown is specific to parsing rather than general; Qwen, by contrast, is slower than C on all three operations (1.18×–1.50×), the only model of the six for which that is true. This is a materially different and more specific finding than “LLM-ported Rust code is somewhat slower than C”: three of the five LLMs matched or beat C on every operation we measured; one (Gemini) has an isolated parsing-specific regression; and only one (Qwen) is a broad, across-the-board underperformer. Collapsing these into a single geometric mean across all six, as 

Table 6. Comprehensive multi-model comparison, including the manual baseline. Kimi: canonical run. “n/a” denotes a metric not meaningfully defined for the manual baseline (see text).

Dimension	Manual	Claude	Gemini	Qwen	Codex	Kimi
Generation time	∼8 h	∼10 min	∼25 min	∼2 h	∼15 min	∼17 min
LOC	n/a	2216	782	1345	1546	1399
Tests generated	n/a	95	6	53	20	26
Divergences (final)	n/a	8	18+	12+	16	13
Self-corrections	n/a	5	0	4	3	5
Residual bugs	n/a †	0	9	8	3	3
Parse/C	0.98×	0.99×	1.25×	1.29×	0.90×	0.94×
Print fmt/C	0.95×	0.67×	0.83×	1.50×	0.74×	0.76×
CVEs eliminated	11/11	13/13	11/11	11/11	13/13	13/13
unsafe blocks	0	0	0	0	0	0
Diff. fuzz vs. C	n/a †	0/235 M	3 (imm.)	2 (imm.)	2 (imm.)	imm.

^† The “Residual bugs” and “Diff. fuzz vs. C” columns report each LLM porting’s bug count and outcome of differential fuzzing against the original C implementation, using the standard pipeline of Section 3.4, uniformly applied to all five LLM portings. “imm.” (immediately) indicates that a divergence was found within the first few seconds of the campaign, so the run was not extended further; Claude’s campaign, by contrast, found no divergence and was extended to 235 million executions specifically to stress-test that clean result, which is why only Claude carries an execution count. This is a different and not directly comparable exercise from the manual-versus-Claude cross-comparison of Section 5.3, which differentially fuzzes the manual and Claude portings directly against each other rather than against C and which found two bugs exclusive to the manual porting, one exclusive to Claude, and one shared by both; that finding does not contradict Claude’s “0” here, since it comes from a different, more targeted comparison applied only to this one pair, conducted specifically because both portings looked unusually clean under the standard pipeline.

does for compactness, obscures a distinction that matters for anyone deciding which operations of a candidate porting to scrutinize first. Second, the effect is not uniform across test files either, at least among the four portings plotted in Figure 4 (Codex and Kimi are not part of this specific per-file breakdown (Section 4.3), so we do not extend this file-level claim to them): on test9, a small file dominated by numeric arrays, each of Manual, Claude, Gemini, and Qwen is faster than C, by a factor ranging from roughly 1.4× (Gemini) to nearly 3× (Manual and Claude), consistent with Rust’s Vec performing a small number of amortized reallocations for a growing array where the original C code calls malloc once per node; on test4, the largest file in the suite, all four are slower than C, including the manual baseline, plausibly because the bounds checks Rust inserts on every buffer access, while individually cheap, compound over a much larger number of accesses on a large input in a way they do not on a small one and because incremental String/Vec growth involves reallocation-and-copy steps that a single well-sized malloc in the C original avoids. We flag this as a plausible mechanism rather than a confirmed one, since we did not separately profile test4. Third, the two slower models, Gemini and Qwen, are also the two models with the weakest showing on other axes: Gemini generated the fewest tests (six), performed zero self-corrections, and produced the most residual bugs (nine, Table 6), consistent with a comparatively minimal, less iterated first-pass implementation. Qwen, despite taking, by far, the longest to generate (∼2 h, on local consumer hardware) and generating the second most tests of any model (53, behind only Claude’s 95 and nearly triple Codex’s 20), still showed both the weakest performance and the second highest bug count (eight), which suggests that generation time and self-testing volume alone do not guarantee performance-aware implementation choices and that model capability or capacity at this parameter scale plausibly plays an independent role. Whether a 25–29% slowdown is practically significant depends on the deployment context: for a JSON parser embedded in a low-throughput configuration file reader, it is unlikely to matter; for a JSON parser on the hot path of a high-throughput Web API or IoT message pipeline, the kind of deployment context motivating this study (Section 1), a consistent quarter-to-third slowdown is the kind of regression a team would reasonably want to catch and address before shipping, which is precisely the role we envision for the performance axis of the verification pipeline in practice (Section 5.7).

4.4. Multi-Model Comparison

Table 6 consolidates all evaluation dimensions, including the manual baseline where a given metric is meaningfully defined for it. Four of the metrics below, generation time in the LLM-agentic sense, lines of code, the number of self-generated tests, and the number of self-corrections during generation, describe properties of the autonomous LLM generation process itself and do not have a directly comparable equivalent for a human-authored baseline that followed ordinary software-engineering practice rather than an agentic generate-and-self-correct loop; these cells are marked “n/a” for Manual rather than populated with an approximate or inferred figure. Where a genuinely comparable number exists elsewhere in the paper, we report it here instead: Manual’s performance ratios are carried over from Table 5, its CVE coverage from Table 4, and its unsafe-block count from the paper-wide invariant established in Section 4. Because four of the five LLMs contribute a single run, the between-model differences below are descriptive; Section 3.5 and Section 4.5 quantify, via the Kimi repetitions, how much of this spread stochastic variation alone can plausibly account for.

Interpretation. Three patterns in Table 6 stand out beyond the individual cell values. First, Claude and Gemini sit at opposite ends of a coherent profile rather than differing on just one metric: Claude produced the most self-generated tests (95), the most self-corrections (5), and the largest implementation (2216 LOC) and ended with zero residual bugs, while Gemini produced the fewest tests (6), zero self-corrections, one of the smallest implementations (782 LOC), and the most residual bugs (9); this is consistent with the relationship between test-suite investment and self-correction discussed further in Section 5 and suggests the two are not independent metrics but different facets of how thoroughly a given run engaged with the task. Second and less intuitively, Gemini also shows the highest count of divergences from the shared target architecture (18+) despite writing the least code, meaning its smaller implementation is not a more literal or more architecturally compliant one; a smaller diff against the C original is not the same as a smaller diff against the requested Rust design, and the two should not be conflated. Third, Codex and Kimi cluster together on most dimensions (3 residual bugs each, both implementing cJSON_Minify for full 13/13 CVE coverage, with comparable divergence counts), which is a useful reminder that “the five LLMs” are not a uniform population; a two-cluster pattern (Claude alone, Codex and Kimi together, or Gemini and Qwen together, the latter of which are also the two slower performers, as per Section 4.3) is at least as visible in this data as a single ranking from best to worst. We emphasize, per Section 3.5 and consistent with the conclusion-validity discussion in Section 3.6, that with $N=1$ for four of these five models, these are patterns observed in the specific runs we collected, not validated general properties of the models; the Kimi variance study below is our one direct check on how much a single run can be expected to move.

4.5. Intra-Model Variance Study (Kimi, $N=5$)

The Kimi K2.7-Code porting was repeated five times independently (∼$15 total).

Table 7. Intra-model variance: Kimi K2.7-Code, $N=5$. Run 1 (bold) = canonical. Ratios are geometric means vs. C.

Run	LOC	Tests	CVE	Unsafe	Miri	Parse	Print	Bugs
1	1399	26	14/14	0	0	0.94×	0.76×	3
2	1366	41	14/14	0	0	0.95×	0.91×	3
3	1431	32	14/14	0	0	0.79×	0.86×	3
4	1544	31	14/14	0	0	0.94×	0.86×	4
5	1318	27	14/14	0	0	0.87×	0.90×	2
Range	1318–1544	26–41	14/14	0	0	0.79–0.95×	0.76–0.91×	2–4
Mean ± 95% CI	1412 ± 106	31.4 ± 7.4	14/14 (all)	0 (all)	0 (all)	0.90 ± 0.09	0.86 ± 0.07	3.0 ± 0.9

shows the results.

The five repetitions separate sharply into what is invariant and what is not. Invariant across all five: zero unsafe blocks, a 14/14 pass rate on the CVE suite, zero Clippy warnings, and zero Miri-detected undefined behavior; not one of the five independent runs deviated from full structural safety, which is the strongest single piece of evidence in this study that structural safety is not a fragile, lucky outcome of one particular generation but a stable property of prompting a model to target safe Rust. What is not invariant is everything downstream of that: LOC varies by about 9%, the number of self-generated tests ranges from 26 to 41, residual bugs range from 2 to 4, and the parse-performance ratio ranges from 0.79× to 0.95×. One further finding does not fit neatly into either category and deserves its own mention: two of the five runs panic on multi-byte (non-ASCII) input, while the other three handle it correctly. A panic is not a memory-safety violation; Rust converts what would be undefined behavior in C into a controlled, safe process abort, so this finding does not contradict the structural-safety invariant above, but it is still a denial-of-service in a production parser that must handle arbitrary input, and it illustrates concretely why “memory-safe” and “production-ready” are not synonyms, a distinction we return to in Section 5.7. Taken together, what these five runs support is a specific, bounded claim: for this model, on this library, structural safety held five times out of five, while code quality, robustness, and performance did not; we do not extrapolate this to a claim that all five evaluated models would show the same invariant-versus-variable split under repetition, since only Kimi was actually repeated.

The 95% confidence intervals in Table 7 (Student’s t, 4 degrees of freedom) are computed from the five Kimi runs and give an empirical sense of scale for the run-to-run noise inherent in a single agentic porting attempt: a spread of roughly ±9 percentage points on the parse ratio and ±1 bug is attainable from stochastic variation in one model alone. Several of the between-model gaps in Table 6, for example, the parse-ratio gap between Codex (0.90×) and Qwen (1.29×), exceed this envelope and are therefore unlikely to be explained by sampling noise alone; smaller gaps, such as the one between Claude (0.99×) and Kimi (0.94×) on the same metric, fall within it and should be interpreted cautiously pending repeated runs of the other four models.

5. Discussion

This section interprets the results reported in Section 4 in relation to the four research questions, prior work, the study’s limitations, and its generalizability rather than restating the numbers. The first two subsections revisit RQ1 and RQ2, the separation between structural safety and code quality and the role of self-generated test-suite quality in explaining which semantic bugs a model catches on its own, respectively. The following two subsections examine what the manual-versus-LLM comparison adds beyond the per-model results, first through the complementary-bugs finding, then through a more realistic accounting of human effort than raw generation time provides. The next three subsections turn to RQ4, practical consequences: the monetary and computational cost of adopting this workflow, the challenges that stand between this case study and industrial deployment, and the concrete guidance we draw from the study for a team planning a similar migration. The section closes by relating the findings to the broader question of Internet infrastructure security that motivated the study.

5.1. Structural Safety vs. Code Quality

The most robust finding of this study and our answer to RQ1 is a separation between structural safety and code quality. All five LLMs, despite differing substantially in architecture, parameter count, and deployment mode, from commercial frontier systems to a 27B open-weight model running locally on consumer hardware, produced Rust code that eliminated all in-scope CVE classes by construction once explicitly instructed to avoid unsafe. The Kimi variance study provides the strongest single piece of evidence for treating this as a stable property rather than a one-off outcome: structural safety held across all five independent repetitions, while every other metric we measured, lines of code, test count, residual bugs, and performance, varied (Section 4.5). This is consistent with and extends the finding of Xu et al. [24] that every one of 186 real-world Rust memory-safety CVEs required the presence of unsafe code: our result shows that five different LLMs, operating autonomously and without CVE-specific knowledge, reliably produce code that satisfies that necessary condition when asked to.

We are nonetheless careful not to overstate this finding for two reasons directly tied to the construct- and conclusion-validity discussion in Section 3.6. First, it concerns one relatively small, single-file, single-threaded C library; we have not shown that the same separation holds for a large, multi-module, or concurrent codebase, where a model would need to reason about safety properties, aliasing and data races among them, that never arose in this study (Section 5.6). Second, four of the five LLM portings are based on a single run; the Kimi repetitions establish that structural safety is stable for that one model, and we treat it as suggestive rather than independently confirmed, and that the same stability holds for Claude, Gemini, Qwen, and Codex. What the experiments do establish is narrower and, we think, still useful: that the Rust type system, together with an explicit zero-unsafe constraint, eliminates the specific memory-safety vulnerability classes documented in cJSON’s CVE history, consistently across five architecturally diverse models and across repeated runs of at least one of them in a case study of this size and profile. We avoid stating the broader, unrestricted claim that structural safety is a language-level property independent of model or scale, since that claim would require evidence from targets and models beyond what we tested.

Code quality is the axis on which this uniformity breaks down. Residual bugs ranged from 0 to 9, and performance ranged from 0.90× to 1.29× the original C implementation on parsing (Table 6); unlike structural safety, code quality did not survive as an invariant, even within the one model we repeated five times (Section 4.5). Read together with RQ1, this suggests a specific, falsifiable division of labor between the language and the model: the elimination of the classic memory-safety vulnerability classes appears to be substantially a property of Rust’s compiler and type system, robust to which model is doing the porting, while correctness beyond the tested vulnerability classes, idiomaticity, and performance remains property of the individual generation attempt, sensitive to both which model is used and run-to-run stochastic variation within a single model. Section 5.2 examines one candidate explanation for part of that remaining variation.

5.2. Self-Correction and Test-Suite Quality

The five models span the full spectrum of self-correction behavior during generation: Claude performed five self-corrections, including one semantic fix, while Kimi performed five (compilation, test, and Clippy-driven), Qwen performed four (compilation-driven only), Codex performed three (compilation-driven only), and Gemini performed none. Cross-referencing this against the residual-bug counts in Table 6 suggests a specific mechanism rather than a mere correlation: every semantic bug left latent by a model other than Claude shares a common trait: none of them emerged from that model’s own generated test suite. In other words, models did not fail to notice bugs they had a chance to catch; they failed to write tests capable of exposing the bugs in the first place. Compilation errors and Clippy warnings are, by construction, always visible to the tool that produces them, so every model in our study that made such an error had the opportunity to self-correct it and mostly did; a semantic bug, by contrast, is only visible to a model if its own test suite happens to exercise the specific input or code path where the model’s reasoning went wrong, which is precisely what the shared CVE suite is designed to guarantee for the vulnerabilities we already knew about (Section 3.4) but cannot guarantee for a novel semantic error a model introduces on its own.

This has a direct practical implication, one we build on in Section 5.7: test-suite quality is not merely a proxy for how thorough a model was; it is plausibly a causal lever a practitioner can pull. If self-generated test-suite quality is the main determinant of which semantic bugs get caught before a human ever reviews the code, then supplying a model with a required minimum test suite or a reference suite analogous to our shared CVE regression tests is a comparatively low-cost intervention that this study’s own methodology suggests should improve outcomes, independent of which underlying model is used. We note this as a plausible, testable implication of the pattern we observed rather than as a separately validated finding, since we did not run a controlled comparison of prompting with versus without a required test suite; doing so is a natural extension of this study’s design.

5.3. Complementarity of Manual and LLM Portings

Differential fuzzing between the manual and Claude portings, run directly against each other rather than against the original C implementation (unlike the per-model, C-referenced differential fuzzing summarized in Table 6), revealed that each porting contained bugs absent from the other: the manual porting had two exclusive bugs, the initial Claude porting had one exclusive bug (a numeric overflow that produced an incorrect infinity value instead of the expected error), and one bug (also an overflow-to-infinity case) was shared by both. This exercise, together with the performance issue described in Section 4.3, is what the “Claude post-fix” labeling used throughout Table 5 and Table 6 refers to: once this cross-comparison surfaced Claude’s exclusive bug, the authors patched it, which is why Claude’s residual-bug count in Table 6 reads 0 rather than 1. The two manual-porting bugs, by contrast, were identified through this same cross-comparison but were not folded back into a corrected release of the manual baseline, since our goal was to characterize the LLM portings against a fixed reference point rather than to keep re-patching the baseline every time a new comparison method surfaced an issue in it. This finding, taken together, supports a hybrid workflow: an LLM generates a first draft, a human refines it, and differential fuzzing between independently produced implementations cross-verifies both, catching bugs that neither a single implementation’s own test suite nor its C-referenced fuzzing campaign happened to trigger. In relation to RQ2, this result qualifies the code-quality comparison in Section 4: even the LLM porting with the strongest standard-pipeline results (Section 4) was not bug-free until this additional cross-comparison step, which is consistent with prior findings that LLM-introduced translation bugs can be subtle and unevenly distributed across verification methods [29], and argues against treating any single verification pass, however thorough, as a final word on a porting’s correctness.

5.4. Realistic Time Accounting

The apparent 50× acceleration suggested by comparing raw generation time to the manual baseline (10 min versus 8 h, Table 6) is, on its own, a misleading efficiency claim because it accounts only for the model’s own output and ignores everything a team would still need to do before treating that output as finished. Section 4.3 and Section 5.3 both show concrete instances of required post-generation work: a differential-fuzzing pass that surfaced a semantic bug invisible to standard verification and a manual fix for an $O\left(n^2\right)$ performance regression that the model’s own review did not catch. Accounting for this kind of review, which we estimate at approximately 5 h based on the scope of changes involved in the case study above, brings the total hybrid-workflow time (generation plus review) to roughly 60–65% of the manual baseline’s 8 h, an approximately 35–40% time saving rather than the 50× figure the raw generation numbers alone would suggest. We consider this the more honest number to report and the more useful one for a team estimating the cost of adopting this workflow, precisely because it is not the number that makes the strongest marketing claim.

The source of the remaining advantage is qualitative as much as quantitative: review of LLM-generated code starts from a working baseline of code that already compiles, already passes its own tests and the shared CVE suite, and already has zero unsafe blocks, so the reviewer’s task shifts from writing and debugging to auditing and refining. This is a different and, in our experience, less effortful activity than porting from scratch, even when the wall-clock time saved is more modest than the headline generation-time comparison implies. We return to the practical consequence of this distinction, i.e., that a team should budget for verification effort rather than for generation time when planning an LLM-assisted migration, in Section 5.7.

5.5. Computational and Monetary Cost

Generation time (Table 6) is only a partial proxy for the practical cost of adopting this workflow, and the reviewer is correct that our original discussion under-specified this dimension. Three cost regimes are represented in this study. Qwen3.5-27B was run locally via Ollama on consumer-grade hardware (Section 3.3): it incurs no per-token monetary cost but requires a machine capable of hosting a 27B-parameter model and is, at ∼2 h, the slowest of the five. Claude, Gemini, and Codex were accessed through commercial cloud APIs billed per token; for these three, only the Kimi-style repeated-run protocol yields a directly comparable monetary figure in this study: the five Kimi K2.7-Code repetitions, together, cost approximately $15 in API usage (Section 4.5), i.e., about $3 per porting attempt at this library’s size (∼3200 LOC of C). We did not meter and record equivalent per-run token costs for the single Claude, Gemini, Codex, and Qwen sessions, so we do not report point cost estimates for them here rather than risk quoting figures we cannot substantiate; a systematic per-token cost accounting across all five models, using consistent metering, is identified as necessary follow-up work and will be included, together with raw token counts, in the public repository. Even absent a full cost table, the Kimi figure indicates that, for a library of this size, the monetary cost of an individual agentic porting attempt is small relative to the ∼8-h senior-engineer-time cost of the manual baseline; whether this ratio holds for larger codebases, where context-window limits force multi-pass or multi-session strategies with correspondingly higher token consumption, remains open and is a natural extension of the industrial-deployment discussion in Section 5.6.

5.6. Industrial Deployment Challenges

Beyond the controlled, single-file setting evaluated here, moving this workflow into industrial practice raises challenges that this case study is not designed to resolve but that are worth naming explicitly. Scale and context limits. Codebases beyond a few thousand lines or split across many files with cross-module dependencies exceed what can be handed to a model in one pass; this motivates skeleton-guided or repository-level strategies such as those of Rustine [37] and EvoC2Rust [36] rather than the single-pass prompt used here. Build systems and FFI boundaries. Real deployments rarely port a library in isolation: the ported Rust code must interoperate with existing C callers through an FFI boundary, integrate with the surrounding build system, and preserve ABI-level guarantees that a from-scratch idiomatic redesign like the one adopted here does not need to satisfy. Concurrency. None of the five models or the manual baseline had to reason about a shared mutable state across threads; concurrent and multi-threaded C code introduces data-race classes that Rust’s type system also addresses structurally, but doing so requires different prompting and verification strategies than the ones used in this study. Review and sign-off process. An industrial adoption path needs a human-review and acceptance protocol for agent-generated safety-critical code, of which the differential-fuzzing cross-verification step used here (Section 5.3) is one component, not a complete answer. Long-term maintenance. This study evaluates a single porting event; it says nothing about how agent-assisted Rust code is maintained, re-verified, and extended over subsequent development cycles. We view each of these as a concrete, separately tractable extension of the present methodology rather than as a reason to defer LLM-assisted migration entirely.

5.7. Practical Guidance for Practitioners

A natural question for a team planning to port C code to Rust is what, concretely, our results imply for how they should operate. Reading Section 4 in isolation might suggest a one-line answer: use Claude, since it produced the fewest residual bugs and the best print-formatting performance in this comparison. We think that answer is too simple for a reason the study itself demonstrates: the Kimi intra-model variance study (Section 4.5) shows that a single run of even one fixed model can swing by several percentage points in performance and by a couple of bugs in either direction, purely from run-to-run stochastic variation, and Claude, Gemini, Qwen, and Codex were each evaluated with exactly one run. A team that tries Claude once and happens to land on an unusually weak run or tries a weaker-scoring model once and happens to land on an unusually strong run could easily draw the opposite conclusion from ours. With that caveat firmly in mind, we offer five more actionable recommendations that follow directly from our findings.

Expect structural safety regardless of model choice but not code quality. All five models, spanning commercial frontier systems down to a 27B open-weight model running on a laptop, produced Rust code with zero unsafe blocks and zero memory-safety crashes across billions of fuzzing executions when explicitly instructed to avoid unsafe (Section 4). This is the one part of our results a team can treat as a reliable, close-to-model-independent property of porting to Rust with an LLM: a straightforward, explicit prompt requiring a safe-Rust target is enough to obtain the elimination of the classic memory-safety vulnerability classes (buffer overflow, use-after-free, double free, and NULL dereference; see Table 2) by construction. Code quality, in contrast, is not free and did vary substantially between models (0 to 9 residual bugs, Table 6); budget human-review effort accordingly, and do not assume that a clean-looking, compiling, safe-Rust output is also a correct or well-performing one.

Supply or demand a comprehensive test suite up front. The strongest predictor of whether a model caught its own semantic bugs during generation was the quality of its own test suite (Section 5.2): every semantic bug left behind by a model other than Claude had gone completely untested by that model’s own tests. A practical, low-cost intervention is therefore to supply a reference test suite or a minimum required test coverage as part of the porting task itself rather than leaving test generation entirely to the model’s discretion; our own shared 13-CVE regression suite (Section 3) is one example a team could adapt.

Treat the LLM’s output as a first draft, not a deliverable. The complementary-bugs finding of Section 5.3 and the $O\left(n^2\right)$ performance regression described in Section 4.3 both show that even the best-scoring porting in this study needed a human-driven fix after generation. A practical workflow is the hybrid one this study argues for throughout: generate with an LLM; review and benchmark the result; and, where feasible, differentially fuzz it against either the original C implementation or an independently produced second porting (human or LLM) to surface the kind of semantic divergence that neither implementation’s own test suite happened to trigger.

Match the target’s scale and profile to the methodology, not just the library’s popularity. Our results describe a compact (∼3200 LOC), single-file, single-threaded, dependency-free target that fits entirely within a single LLM context window (Section 3). A team porting a codebase of this profile can reasonably expect similar dynamics; a team porting a larger, multi-file, concurrent, or cryptographic codebase should instead plan for the repository-level and concurrency-aware strategies discussed in Section 5.6, budget for FFI and build-system integration work that this study’s from-scratch redesign did not need to do, and treat our specific numbers (time, cost, and bug rates) as illustrative rather than directly transferable.

Budget for verification, not generation. At this library’s scale, LLM generation itself is fast (minutes to a couple of hours; Table 6) and cheap (on the order of $3 per attempt; Section 5.5) relative to the manual baseline’s ∼8 engineer hours. The practical bottleneck our results point to is therefore not model access but the verification pipeline around it, fuzzing, Miri analysis, differential cross-checks, and human review, which is where a team should plan to invest most of its time and infrastructure when adopting this kind of workflow.

5.8. Implications for Internet Infrastructure Security

Returning to the motivation set out in Section 1, these findings bear directly on the security of Internet infrastructure, subject to the scope limits already discussed in relation to RQ1 (Section 5) and construct and external validity (Section 3.6). JSON parsers such as cJSON are embedded in Web APIs, IoT firmware, cloud services, and mobile applications. The structural elimination of memory-safety vulnerability classes through migration to Rust, accelerated by LLM assistance, offers a scalable path to hardening of the software supply chain. The demonstrated feasibility of producing functionally correct, memory-safe Rust code from a real-world C library in minutes rather than days suggests that LLM-assisted migration could significantly lower the barrier to adopting memory-safe languages in the Internet ecosystem, provided that human review and cross-verification remain integral to the workflow and provided the scale, FFI, concurrency, and process challenges outlined in Section 5.6 are addressed for the target codebase.

6. Conclusions

This paper has defined and applied a structured methodology for LLM-assisted porting of cJSON (∼3200 LOC, 14 CVEs) to idiomatic Rust, with the first intra-model variance measurement ($N=5$) and an accompanying statistical framework for interpreting single-run cross-model comparisons. We summarize the paper’s findings below, distinguishing what the experiments directly support from our interpretation of those results before turning to the study’s limitations and to future work.

Findings directly supported by the experiments. Across all six portings, manual and LLM-generated alike, and across all five independent repetitions of the Kimi study, the resulting Rust code eliminated every in-scope CVE class, contained zero unsafe blocks, and survived over 1.7 billion fuzzing executions and Miri analysis without a memory-safety crash or a detected undefined-behavior violation (RQ1, Section 4). Code quality, by contrast, did not hold constant: residual bugs ranged from 0 to 9, and parsing performance ranged from 0.90× to 1.29× the original C implementation across models, and even the one model repeated five times showed variation in residual bugs (2 to 4), tests generated (26 to 41), and performance (0.79× to 0.95×) (RQ2 and RQ3; Section 4 and Section 4.5). Every semantic bug a model left in its final porting had gone untested by that model’s own test suite, and differential fuzzing between two independently produced, standard-pipeline-clean portings (manual and Claude) surfaced bugs that neither implementation’s own verification had caught (Section 5). The Kimi repetitions bound the scale of run-to-run stochastic noise at roughly ±9 percentage points in parsing performance and ±1 residual bug; several between-model gaps in Table 6 exceed this envelope and are therefore unlikely to be sampling noise alone, while smaller gaps remain inconclusive pending repeated runs of the other four models. A single porting attempt cost approximately $3 in API usage in the one case we metered, and accounting for the human review and cross-verification that our own results show is still necessary narrows the apparent 50× generation-time speedup to a net 35–40% time saving relative to the manual baseline.

Our interpretation of these findings. We read the safety-versus-quality separation as evidence for a division of labor between the language and the model: elimination of the classic memory-safety vulnerability classes appears to be substantially a property of Rust’s compiler and type system, robust to which of five architecturally diverse models performed the port, while correctness beyond the tested vulnerabilities, idiomaticity, and performance remain properties of the individual generation attempt, sensitive to model choice and stochastic variation within a single model (Section 5). We do not extend this to an unrestricted, model- and scale-independent claim; Section 5 and the limitations below explain why. Given the $N=1$ caveat that applies to four of the five models, we also do not read these results as supporting a single “best model” recommendation; our practical guidance (Section 5.7) is instead a verification-centered workflow: expect structural safety from any of the five models once unsafe is explicitly disallowed but not code quality, demand or supply a comprehensive test suite up front, treat model output as a first draft requiring human review and differential fuzzing, and match the target codebase’s scale and profile to the methodology before assuming these figures transfer to a larger or concurrent target. The verification pipeline itself, being model-independent and publicly released, is the part of this methodology we intend to be reusable regardless of how the underlying models change.

Limitations. The study’s threats to validity are discussed systematically in Section 3.6. In summary, the findings above come from a single, compact, single-file, single-threaded library, so extension of external validity to larger or multi-module, concurrent, or cryptographic codebases is untested. Possible training-data contamination, given cJSON’s public availability since 2011, cannot be fully excluded as an internal-validity confound, though the substantial architectural divergence across the six portings argues against wholesale memorization. The zero-unsafe requirement in the prompt makes compliance with that specific instruction unsurprising by construction, so we treat the non-trivial result as the resulting code’s functional correctness and safety under fuzzing, not the mere absence of unsafe. Finally, the predominantly single-run design limits the conclusion validity of the cross-model comparisons, which we report descriptively rather than as statistically significant differences.

Future Work

The most direct extension is to test whether the structural-safety-versus-code-quality separation observed here generalizes to larger, multi-module, and industrial-scale C projects spanning different domains, including cryptographic libraries such as OpenSSL and mbedTLS, database engines such as SQLite, and other widely deployed parsers such as libpng and libexpat. A second priority is to close this study’s main statistical limitation by collecting repeated runs ($N>1$) for all evaluated models, not only Kimi, which would enable formal paired statistical testing, for example, paired t-tests or Wilcoxon signed-rank tests with associated confidence intervals, of the differences currently reported only descriptively in Table 6. Further extensions include incorporating additional and more recent commercial and open-weight coding models as they become available; performing Rust-versus-C differential fuzzing via FFI; reporting a systematic, consistently metered computational and monetary cost comparison across all evaluated models (Section 5.5); validating the practical guidance of Section 5.7 on a second, differently profiled codebase; and employing specialized vulnerability-discovery models [46], though independent evaluations suggest a non-negligible false-positive rate [47]. A longitudinal study would track how these findings evolve as both the underlying models and the surrounding tooling improve.