Next Article in Journal
Optimizing Parking Efficiency Using Parking Duration Prediction: A Case Study of a Shopping Mall Parking Facility
Previous Article in Journal
Influence of Injection-Induced Secondary Fault Slip on the Stability of an Adjacent Critically Stressed Fault
Previous Article in Special Issue
WaveletMask: Wavelet-Domain Mask-Guided Degradation Detection for Old-Film Restoration
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Coordinating Drag-Based Structure Editing and Reference Style Transfer in Diffusion Models for Anime Images

Shanghai Film Academy, Shanghai University, Shanghai 200072, China
*
Author to whom correspondence should be addressed.
Appl. Sci. 2026, 16(13), 6703; https://doi.org/10.3390/app16136703
Submission received: 10 May 2026 / Revised: 28 May 2026 / Accepted: 2 July 2026 / Published: 4 July 2026
(This article belongs to the Special Issue Latest Research on Computer Vision and Image Processing, 2nd Edition)

Abstract

Reference guided anime editing is challenging when the target requires both rendering style transfer and local structural change. Existing diffusion stylization methods that do not require training usually assume a fixed content layout, while drag-based editors deform local structures without enforcing a separate style reference. Directly combining them is unstable: reference attention can disrupt handle tracking during dragging, whereas stylization after dragging can weaken the edited structure. This paper proposes AnchorHandoff, a temporally coordinated diffusion framework for joint drag and style editing. Drag optimization is performed with style injection disabled, followed by a short interval without style injection that lets the edited structure stabilize. A predicted clean sample from this state after dragging is then used as an anchor: content queries are refreshed from the anchor, and reference style keys and values are replayed on the edited layout. Soft correspondences from intermediate attention features guide style injection toward compatible regions without parsers or segmentation labels. On a curated anime benchmark, controlled comparisons, ablations, and a blind study with 36 participants show that AnchorHandoff reduces residual tracking error and feature structure distortion while maintaining comparable distribution level style alignment. The method remains limited under very large structural changes, but the results highlight temporal handoff as an important factor in joint anime structure and style editing.

1. Introduction

Diffusion models have become a common foundation for image generation and image editing [1,2,3,4,5]. They are now used for text to image generation, inpainting, local editing, stylization, and reference-guided generation [6,7,8]. In reference-guided editing, the reference image gives a visual target for color, texture, line style, and overall appearance, which is useful when the desired result is hard to describe precisely with text prompts alone.
Anime images make this setting more delicate. Compared with natural images, anime style depends more strongly on deliberate visual design. The appearance of an anime character is shaped not only by color, texture, and rendering style, but also by character design, line organization, contour abstraction, and local structural proportion. Prior work on artistic, portrait, and cartoon stylization also shows that stylization quality depends on preserving perceptually meaningful structure while changing rendering statistics [9,10,11,12,13,14]. Eye shape, facial proportion, hair outline, and simplified facial components can determine whether an image reads as a particular anime style. For this reason, anime style editing is not simply a problem of repainting the source image with the colors or textures of a reference image.
Figure 1 summarizes the joint drag and style editing task.
Existing reference image editing methods can bring an output closer to a reference in color, texture, rendering style, and overall visual impression [15,16,17,18,19]. However, in anime images, many style differences also appear as local structural differences. If the method only absorbs the overall appearance of the reference image, the result may look stylized while the character design of the source image is largely unchanged. Drag editing gives a direct way to adjust such local structures [20,21]. With specified control points, a drag method can move or reshape regions such as eyes, face contours, and hair boundaries. Drag editing, however, normally keeps the source appearance. It can move a structure, but it does not decide how that edited structure should be rendered in a separate reference style.
The natural idea is to combine the two operations: use drag editing for the local structure and reference style transfer for the final appearance. In practice this simple composition is unstable. Drag editing relies on intermediate diffusion features for point tracking and local deformation, while style injection changes these features and can make the drag process less reliable. Postponing stylization does not fully solve the problem. The dragged structure may appear in the intermediate trajectory, but the later style transfer step can soften it or pull it back toward the original layout.
This failure suggests that the transition between dragging and stylization should be treated as part of the method. The proposed framework, referred to as AnchorHandoff, separates drag optimization from style injection and resumes reference replay from a post-drag anchor state. In this way, the style stage operates on the edited layout rather than on queries associated with the structure before dragging.
The local transfer of style information also needs to be constrained. Direct reference injection does not explicitly know which content regions correspond to which reference regions. In anime images, this can lead to color mismatch or style leakage. To reduce this problem, this paper uses intermediate attention features of the diffusion model to estimate soft local correspondences between the content image and the reference image. These correspondences guide style information toward more compatible local regions.
The main contributions are as follows:
  • The paper studies joint anime drag and style editing, where the output must satisfy both specified local deformation and reference-guided anime style transfer.
  • It introduces a temporal anchor handoff strategy that separates drag optimization from style replay and resumes reference injection from a post drag clean sample anchor.
  • It proposes anchor conditioned correspondence replay, which refreshes content queries from the edited anchor and routes reference style information toward compatible local regions without external parsing networks or semantic labels. The framework is evaluated on a curated anime benchmark through controlled quantitative comparisons, component ablations, and a blind forced choice study with 36 participants.

2. Related Work

2.1. Reference Style Transfer in Diffusion Models

Classical style transfer methods such as Gatys et al. [22] and AdaIN [23] represent appearance through feature statistics. Surveys of neural and artistic style transfer further summarize this line as a progression from global feature statistics to more controllable and structure aware stylization [10,24,25]. They are useful background for style representation, but they do not provide the attention interface used in this paper. AdaIN is therefore used only as a lightweight initialization tool for the replay latent after the anchor.
The closer reference point is training free diffusion stylization. StyleID [17] is especially relevant because it transfers style by replacing self attention keys and values with those from a reference image. Cross Image Attention [18] and Style Aligned [26] also use attention as an image to image appearance transfer interface. Diffusion in Style [15], StyleDiffusion [16], FreeMix [27], multi source stylization [28], training free latent diffusion style transfer [29], and IP Adapter [19] cover related reference-guided or image conditioned stylization settings.
These methods are strong when the content layout is treated as fixed. The joint task here breaks that assumption. After a drag edit, the content queries associated with the original image can become a poor description of the desired layout. A style pass may then improve color and line quality but reuse the wrong structural state. This is why the replay stage in this paper is not applied directly after dragging; it is started from a clean sample anchor and refreshed queries.

2.2. Drag-Based Image Editing with Diffusion Models

Drag-based editing provides the point control needed by the proposed task. DragGAN [20] manipulates local image structures through handle and target point pairs on a generative image manifold, and DragDiffusion [21] adapts this idea to diffusion-based real image editing by optimizing latent representations during denoising. Later methods improve handle stability, deformation propagation, and interaction efficiency. FreeDrag [30], DragNoise [31], and StableDrag [32] focus on feature propagation, stability, and scheduling, GoodDrag [33] summarizes practical design choices for diffusion drag editing, and DragonDiffusion [34] introduces stronger correspondence or generation priors.
The common target of these methods is still a geometric edit of the source image. This is exactly the part this paper needs, but not the whole task. In anime editing, moving an eye corner, a hair boundary, or a face contour is only useful if the edited region is then rendered coherently in the reference style. A drag only method can provide the geometry, but it does not decide how the style branch should inherit that geometry.

2.3. Attention-Based Style Injection and Local Correspondence

Attention layers are often used as an internal control surface for diffusion editing. Cross attention relates text tokens to image regions, while self attention is more closely tied to image structure, appearance propagation, and consistency across regions. This usage follows the broader role of attention and transformer architectures in vision models [35,36]. Plug and Play Diffusion Features [37], MasaCtrl [38], StyleID [17], and dual attention control [39] all show that internal attention features can be edited or reused during denoising.
The difficulty in this paper is more specific than attention control in general. For style transfer, keeping content queries and injecting style keys/values is attractive because the query side preserves layout while the key/value side provides reference appearance. Similar concerns appear in recent work on controllable diffusion models, where the conditioning signal must improve appearance without destroying the intended content state [39,40]. After dragging, however, the old content queries may no longer describe the edited layout. This motivates refreshing queries from the clean sample anchor before replay.
The second difficulty is local part compatibility. Plain key/value replacement does not check whether a content token and a reference token correspond to similar regions. In anime images this error is easy to see: hair color can leak into the face, eye texture can be borrowed from unrelated regions, and facial contours can receive inconsistent line style. The correspondence used here is kept as a soft attention signal rather than a labeled part map. It does not assign categories such as hair, face, or eye; it only biases replay toward reference tokens that better match the content features after dragging.

2.4. Diffusion Trajectories and Stage Coordination

Diffusion editing also depends strongly on the denoising trajectory. Methods based on DDIM inversion [41] and Null-text Inversion [42] show that real image editing is not only determined by the final conditioning signal. The intermediate latent states also matter, because they decide how much of the source image is preserved and how much room remains for the edit.
For drag-based editing, this trajectory issue is central. A drag operation cannot be inserted at an arbitrary denoising step. At very noisy stages, the local structure is not stable enough for handle tracking. At late stages, the image is already too determined, and latent optimization has little room to move the structure. The drag step is therefore a scheduling problem rather than only an optimization problem.
Reference style transfer makes the scheduling problem harder. If style attention is active during drag optimization, the internal features used for point tracking can change in a way that makes the handle motion less reliable. If stylization is applied immediately after the drag update, the newly edited structure may not yet be stable.

3. Method

Given an anime content image I c , a style reference image I s , and a set of specified control point pairs { ( h i , t i ) } , the goal is to generate an edited result that preserves the content identity of the source image while achieving the specified local geometric edit and maintaining coherence with the target reference style. Unlike style transfer on a fixed layout, this task allows the reference style to imply local structural adaptation. The editing process must handle the interaction between geometric deformation and style injection.

3.1. Overview

Stable Diffusion [2] is used as the base model. Figure 2 shows the full pipeline.
The pipeline is easiest to describe by following the latent that will become the final image. A content LoRA improves reconstruction during dragging, DDIM inversion provides the starting latent, and two auxiliary passes cache the content and style attention tensors. Drag optimization is performed without style injection. After a short recovery interval, a predicted clean sample is taken as the anchor. Style replay then resumes from this anchor using refreshed content queries together with style keys and values from the reference image.
The full procedure is formalized in Algorithm 1.
Algorithm 1: AnchorHandoff for temporally coordinated anime drag and style editing
Input: Content image I c , style reference I s , control pairs { ( h i , t i ) } i = 1 N , and mask M
Output: Edited result I g
Fit a lightweight LoRA [43] adapter to I c for the drag stage;
Invert I c into x T by DDIM inversion [41]; Applsci 16 06703 i001
Unload LoRA, refresh Q c from the anchor, and initialize replay with AdaIN [23]; Applsci 16 06703 i002

3.2. Structure Editing by Drag Optimization

Diffusion drag editing methods usually optimize a fixed inverted latent and then perform a separate denoising process. DragDiffusion [21] follows this strategy and can achieve effective point-based deformation. The separation, however, weakens the temporal coupling between geometric control and the evolving denoising trajectory. Here, drag optimization is embedded directly into the denoising loop, so the deformation objective acts on intermediate spatial features at a selected noise level.
Drag effectiveness varies substantially across the diffusion trajectory. In early high noise stages, the latent remains noisy and feature maps do not yet carry stable spatial structure, making reliable handle point tracking difficult. Late low noise stages have the opposite problem: image details are largely fixed, and the latent becomes less responsive to optimization. The useful drag window is therefore narrow. The latent must be structured enough to provide trackable local features, but still flexible enough to support deformation. Although the implementation allows a configurable drag interval, the reported configuration performs drag optimization at a single selected reverse diffusion step t drag .
Dragging at multiple steps provides more control opportunities but substantially increases optimization cost, since each activated step requires sufficient inner iterations to produce effective deformation. Different denoising stages also expose different semantic scales: early stages affect coarse layout, middle stages are more suitable for local structure, and late stages mainly modify texture and detail. Repeatedly applying the same point objective across these stages may make the optimization target inconsistent and weaken local feature correspondence. The final setting uses a single selected drag step and treats the surrounding denoising schedule as part of the timing design.
As shown in Figure 3, at the selected drag step t drag , intermediate U-Net features F t are first extracted with style injection disabled, giving relatively clean spatial features for point tracking. The handle points are then updated through feature matching between consecutive timesteps. The current latent is then optimized with gradient-based updates using the following objective:
L = i w i ( 2 r + 1 ) 2 F t [ Ω h i ] F t [ Ω h i + δ ^ i ] 1 + λ ( x ^ cur x ^ ref ) ( 1 M ) 1 ,
where Ω p is the radius-r patch around position p , and δ ^ i points from the i-th handle point toward its target. The first term pulls the selected local feature patch along the desired motion direction. The second term compares the denoised latent from the optimized state, x ^ cur , with the corresponding unoptimized reference, x ^ ref , outside the editable mask M , so that regions unrelated to the drag remain close to the original trajectory. The scalar λ sets the strength of this outside-mask preservation.
The implementation also supports distance aware drag strength control for variants in which different control points have different displacement magnitudes. For the i-th control point, its Euclidean displacement is d i = t i h i 2 , and the corresponding drag weight is defined as
w i = clip d i median j { d j } α , w min , w max ,
where the exponent α controls how strongly the weights respond to displacement length, and w min and w max keep the range bounded. This gives longer motions more optimization pressure while preventing them from overwhelming nearby facial landmarks. When this switch is disabled, w i = 1 for every handle point. The reported main setting enables this adaptive point weighting.

3.3. Style Attention Injection and Temporal Coordination

Following StyleID [17], style transfer is implemented through self-attention operations in the U-Net decoder. The content and style caches are intentionally asymmetric. The content branch keeps queries, which carry layout-related structure; the style branch keeps keys and values, which carry reference appearance. After anchor handoff, the replay branch refreshes the content queries from the anchor result, so the style operation follows the dragged layout. Cached tensors are injected in the selected decoder self-attention layers.
Query injection. At each replay step, the live query is blended with the cached query refreshed from the anchor and scaled by a temperature factor:
Q out = η γ Q c cached + ( 1 γ ) Q c ,
The two queries play different roles. Q c cached comes from the content cache refreshed from the anchor, while Q c is the live query at the current replay step. The mixing coefficient γ keeps the replay close to the anchor layout when it is large; the temperature factor η adjusts the sharpness of the resulting attention. Keeping a live query component is useful because the replay branch still has to adapt as denoising proceeds.
Key/value replacement. The self-attention keys and values are taken from the style reference branch. A scalar s kv controls the strength of this style signal:
Attn ( Q out , K s , V s ) = softmax Q out ( s kv K s ) d k + B ( s kv V s ) ,
Here K s and V s are the cached style keys and values, d k is the key dimension, and s kv is the global style strength. The bias term B is left as zero for plain key/value replacement and becomes the correspondence bias in Section 3.6.
Coordination during dragging. Drag optimization is activated only at the selected reverse step t drag . Before this step, the trajectory follows the reference query/key/value schedule used in the final pipeline; in practice, this makes recognizable structure appear earlier and lets the drag step move forward in the reverse trajectory. During the drag step, both the inner optimization loop and the subsequent denoising update disable style injection so that the tracking features used by the drag loss are not distorted by style attention replacement.

3.4. Clean Sample Anchor Handoff

At the specified anchor timestep t anchor , after the short interval without style injection, one U-Net forward pass is run with style injection disabled and the scheduler predicted clean sample x ^ 0 is extracted from the current noisy latent x t anchor . This predicted clean sample is used as the structural anchor for the following replay stage.

3.5. Style Replay from the Anchor

Replay remains short so that stylization strengthens reference appearance without drifting away from the edited structure. Content queries are refreshed from the anchor result, cached style keys and values are reused after the edited structure has stabilized, and soft local correspondences from intermediate diffusion attention features modulate the style signal.
Refreshing content queries from the anchor result. Before the replay stage begins, the LoRA adapter is unloaded, and the content query cache is recomputed from the anchor result.
Replay initialization from the anchor. The same anchor result initializes the short replay branch. This carries the dragged geometry into stylization directly, instead of forcing replay to recover it from the layout before editing. Starting from the anchor-based initialization, AdaIN [23] aligns the replay latent statistics with those of the style reference latent:
z init = α s AdaIN ( z a , z s ) + ( 1 α s ) z a .
where AdaIN ( z c , z s ) denotes the adaptive instance normalization operator:
AdaIN ( z c , z s ) = σ ( z s ) z c μ ( z c ) σ ( z c ) + μ ( z s ) .
The AdaIN statistics are computed spatially in latent space. z a is the replay latent based on the anchor, z s is the style reference latent collected earlier, and α s determines how much of this statistical style initialization is applied before replay starts. Thus, α s = 0.75 means that the replay initialization uses a 75% AdaIN transformed latent and a 25% clean anchor latent.
Style replay with refreshed content queries. During replay, reference-guided self-attention uses content queries refreshed from the anchor result and style keys/values cached from the reference branch. As a result, the style signal acts on the stabilized dragged layout. The branch is intentionally short: it has enough steps to strengthen reference appearance, but fewer chances to drift away from the clean sample anchor. The semantic guidance described next controls where the cached style signal is injected.

3.6. Semantic Correspondence Modulation

Direct key/value replacement injects style information, but it does not check whether local parts are compatible. Content tokens near the eyes may attend strongly to hair or background style tokens, producing leakage or local artifacts. Intermediate attention features provide a useful cue here: visually related regions often form soft feature neighborhoods, and tokens near eyes, hair boundaries, facial contours, or background areas tend to match related regions more strongly than unrelated ones. During replay, semantic correspondence modulation uses this implicit local regularity without semantic segmentation, class labels, parsing networks, or manually annotated part masks. As illustrated in Figure 4, this modulation estimates soft local correspondences from intermediate attention features and uses them to route reference style information toward compatible content regions.
The module estimates a soft matching signal between content and style tokens from refreshed content queries and cached style keys. The signal enters replay attention as a logit bias, a gain for each content token, and a gain for each style token. Here, “semantic” refers to local regularities that emerge in intermediate diffusion features, not to explicit labels.
Semantic similarity bias. At a designated reference layer l ref , a soft similarity matrix is computed between cached content queries and style keys:
S = s b τ Q ^ c ( l ref ) K ^ s ( l ref ) .
The query and key tokens are 2 -normalized along the feature dimension before the similarity is computed. The temperature τ controls how peaked the matching scores become, and s b sets the bias strength. The resulting matrix is used as a soft correspondence map between the dragged content layout and the style reference. This correspondence is not a part label: it only biases tokens toward reference regions with similar intermediate features.
When constructing the soft correspondence bias, each content token keeps its nearest k style tokens. Entries outside this set are not removed by a hard attention mask; instead, they receive zero additional bias. The filtered bias matrix is then added to the attention logits in Equation (4) as the correspondence bias term B .
Content confidence gain. For each content token, matching confidence is measured against the retained style tokens. Specifically, softmax probabilities are first computed from S over the nearest k candidates, and a confidence score is then obtained from the margin between the top two probabilities and the distribution entropy. The gain for the i-th content token is defined as
g c ( i ) = 1 + s c · c ( i ) .
Here c ( i ) is the confidence score for the i-th content token, and s c is the base gain scale. The same confidence value is then sent to two separate controls, one for query magnitude and one for the attention bias:
Q out ( i ) 1 + s q ( g c ( i ) 1 ) Q out ( i ) , B ( i , : ) 1 + s bias ( g c ( i ) 1 ) B ( i , : ) .
This enhances the influence of content tokens with clear correspondence matches, while keeping query strength and bias strength independently controllable.
Style support gain. The column side of the correspondence map is also useful. Some style tokens are repeatedly selected by many content tokens, while others receive little support. Let a ¯ ( j ) be the normalized aggregated attention weight received by the j-th style token, averaged over all content positions and clipped to [ g min , g max ] . The gain for the j-th style token is defined as
g s ( j ) = 1 + s c a ¯ ( j ) 1 .
The same base scale s c is used here. After clipping, the gain is applied only to the style values through an independent value scale s v :
V s ( j ) ( 1 + s v ( g s ( j ) 1 ) ) V s ( j ) .
Style tokens that receive high aggregated attention from content tokens are upweighted, while those receiving lower attention are downweighted. The style keys K s are not rescaled, as modifying the key distribution can destabilize attention patterns.

4. Experiments

This section evaluates AnchorHandoff on the joint anime drag and style editing task. The evaluation covers geometric accuracy, reference style alignment, preservation outside the editable region, component behavior, computational cost, qualitative comparisons, and blind human preference.

4.1. Experimental Settings

The evaluation benchmark contains 120 manually curated anime editing cases. Each case includes a content image to be structurally edited, a paired style reference for appearance guidance, annotated handle–target points, and a binary editing mask. The benchmark is centered on faces by design, because anime identity and reference style are strongly tied to eye shape, facial proportion, mouth and nose placement, facial contour, and local line abstraction. The point pairs are therefore placed on meaningful facial landmarks instead of generic image locations.
Table 1 summarizes the composition of the benchmark with 120 cases. The table reports annotation statistics where applicable and summarizes qualitative coverage for reference rendering, character design, and framing conditions. Representative task examples are shown in Figure 1. Diagnostic examples for drag and anchor timestep selection are shown in Figure 5 and Figure 6, and additional qualitative benchmark examples are presented later in Figure 7 These examples show the corresponding range of facial edits, paired reference appearances, and gaps between the content image and style reference. The benchmark focuses on cases where local facial geometry editing and reference-guided stylization interact in anime portraits.
Diffusion-based editing methods use Stable Diffusion v1.5 [2] where applicable. Baseline implementations follow their public protocols when available. Drag-based comparisons use the same content and reference cases, resolution, handle–target points, and masks, while style-only comparisons use the same content and reference image pairs and resolution; the same masks are used for mask-based metric computation. For AnchorHandoff, the lightweight content LoRA is fitted for the drag stage and unloaded before style replay, and the replay stage itself introduces no additional training. All experiments use fixed method configurations on the same local NVIDIA GeForce RTX 4090 48 GB GPU (NVIDIA Corporation, Santa Clara, CA, USA). Alternative schedules are tested in the ablation study, and the main settings of our method are listed in Table 2.

4.2. Baselines and Experimental Design

Comparisons are organized around the two main factors in the task: the drag branch and the style branch. Since the joint task is not a standard setting in existing reference stylization or point-based editing papers, we construct baselines from representative methods for the two subproblems. Drag only and style only settings isolate each branch, while the joint setting compares the sequential DragDiffusion + StyleID baseline with the AnchorHandoff replay formulation. Branch replacement variants are additionally used in the human preference study to separate structural and style effects.
Geometry only baselines. These baselines execute point-based dragging without subsequent style transfer. DragDiffusion is included as a representative diffusion dragging baseline, and FreeDrag [30], DragNoise [31], GoodDrag [33], and DragonDiffusion [34] are included as additional drag editing baselines. Ours (drag only) denotes the drag branch of the framework without replay after the handoff.
Style only baselines. The style only setting applies reference style transfer directly to the input image without drag optimization. StyleID is used as the representative training free diffusion stylization baseline, and Ours (style only) denotes the replay branch without the preceding drag update. We additionally include representative style transfer baselines, including AdaIN [23], STROTSS [44], StyA2K [45], StyTr2 [46], AdaAttN [47], and AesPA-Net [48], to broaden the style-only comparison. This setting measures style alignment to the reference and identity preservation when the content layout is fixed.
Joint baselines. The main quantitative comparison uses DragDiffusion + StyleID as the primary sequential baseline and also evaluates StyleID after alternative drag editing front ends. Ours is the final integrated method. Intermediate replay and scheduling variants are analyzed separately in the ablation study. For the human study, Ours (drag only) + StyleID and DragDiffusion + Ours (style only) are included as branch replacement variants, so that structural and style preferences can be judged separately.

4.3. Evaluation Metrics

We use the following metrics to evaluate geometry accuracy, content preservation, style alignment, feature structure consistency, and human preference. Unless otherwise specified, quantitative tables report mean and standard deviation values under the corresponding protocol.
Geometry accuracy. Local control is evaluated with a tracking-based Mean Distance (MD), defined as the average Euclidean residual between each tracked handle point in the edited result and its designated target point under the same point tracking protocol for all compared methods. The edited handle locations are obtained from the point tracking process used by the drag optimization procedure and are measured in pixel coordinates at the evaluation resolution. Lower MD indicates that more of the requested point motion survives the subsequent stylization stage. Succ.@20 reports the fraction of cases whose residual is below 20 pixels. Since anime drag and style editing often involves large and ambiguous local deformations, MD is interpreted together with the style metrics.
Content preservation. To quantify how well the edited result preserves source regions outside the edited area, a Mask out identity fidelity score is used:
Mask out IF = 1 LPIPS mask out ,
where LPIPS is computed only outside the convex hull region between the edited image and the source image after standard normalization [49]. Higher Mask out IF indicates stronger preservation of the source appearance in areas away from the edit. The same masks are used for style-only comparisons to keep the preservation diagnostic consistent across protocols. We use Mask out IF as a local preservation diagnostic rather than a complete perceptual quality measure.
Style alignment and feature structure diagnostics. ArtFID (Artistic Fréchet Inception Distance) is reported to measure distribution level alignment with the target artistic appearance, where lower values indicate better style matching. FID-based and perceptual features are also common references for distributional and semantic evaluation in generative image studies [50], while stylization specific evaluation remains known to require multiple complementary criteria [51]. To complement this style statistic, we also report a content feature self similarity distance (CFSD) on top of a frozen ImageNet pretrained VGG19 encoder. CFSD is not a direct style reference distance. Instead, it measures whether the edited image preserves the source image’s pairwise patch feature affinity structure after stylization. This diagnostic is useful in the joint setting because an apparently stronger style transfer may still damage the local structure that the drag stage is intended to preserve. In the implementation, ϕ is extracted from the torchvision ImageNet pretrained VGG19 feature stack at layer index 11, after ImageNet mean and standard deviation normalization. This gives conv3 block patch features with channel dimension d = 256 . Let ϕ ( I ) R N × d denote the flattened VGG patch features extracted from image I . For an image I , we form a row stochastic patch self affinity matrix:
A i j ( I ) = exp ϕ ( I ) i ϕ ( I ) j / τ m j exp ϕ ( I ) i ϕ ( I ) j / τ m ,
where τ m = d 2 follows the implementation’s feature scaling, which divides each feature vector by d before the dot product. Given an edited result I g and the source content image I c , CFSD is measured as the average KL divergence between the content and generated self affinity distributions:
CFSD = 1 N i = 1 N KL A i , : ( I c ) A i , : ( I g ) .
Lower CFSD indicates that the generated image better preserves the content image’s VGG patch self similarity structure.
Human evaluation. Automatic metrics are complemented by blind forced choice preferences for structural correctness, style faithfulness, and overall quality, because the perceptual success of anime editing depends on local shape, line rendering, color, and character appearance jointly.

4.4. Quantitative Comparison

Using the matched protocol, the quantitative comparison first isolates the drag and style branches and then evaluates their joint combinations. Table 3 reports the drag-only comparison before style replay is introduced, Table 4 reports the expanded style-only comparison, and the joint comparison measures the final task performance.
Table 3 isolates geometry editing before joint replay is introduced. The drag-only results do not show a single dominant method across all metrics: DragDiffusion gives the lowest tracking-based MD, whereas Ours (drag only) gives the highest Succ.@20. Preservation metrics favor different baselines, with DragonDiffusion preserving outside mask regions best and DragNoise obtaining the lowest CFSD. Ours (drag only) is reported as the isolated geometry branch of the proposed pipeline before style replay is activated.
Table 4 reports the style-only comparison. Ours (style only) improves Mask out IF and CFSD relative to StyleID, while STROTSS and StyA2K obtain the lowest ArtFID values and StyA2K obtains the best CFSD score. Representative style transfer baselines provide additional reference points for fixed layout stylization.
Table 5 reports the main comparison under the joint drag and style editing protocol.
Ours reduces tracking-based geometric residuals and feature structure distortion over the sequential DragDiffusion + StyleID baseline while keeping the same ArtFID. Replacing the drag stage with stronger drag-only baselines does not consistently improve the joint balance after StyleID stylization, and style injection during the drag step worsens the tracking-based residual and increases CFSD. The largest difference appears in CFSD, indicating lower VGG feature structure distortion in the final rendered result.

4.5. Computational Cost

Runtime and memory are measured under the same local evaluation protocol at 512 × 512 resolution. Table 6 reports the average runtime per case and peak GPU memory for practical joint editing pipelines. The staged baselines run a drag editing stage followed by a separate StyleID stylization stage, whereas AnchorHandoff performs drag optimization, anchor extraction, and replay within one coordinated editing trajectory.
Under this matched local protocol, the integrated execution avoids an additional full stylization pass and yields a lower practical runtime and memory footprint than the staged pipelines.

4.6. Ablation Study

The ablation study examines replay strength, temporal handoff, and semantic replay. Each ablation block uses a fixed controlled protocol, so values should be compared within the same block.
Table 7 reports the sensitivity to stronger replay gains.
Table 8 evaluates the effect of handoff timing and style injection during dragging.
Table 9 evaluates the semantic replay components with a fixed drag anchor.
Table 7 shows that correspondence replay is not a pure strength knob. Stronger query side or all token gains do not consistently improve all metrics, and the final setting gives the most balanced result in this block.
Table 8 shows the largest degradation for immediate handoff and style injection during dragging, indicating that temporal coordination is important.
Table 9 shows that semantic replay mainly affects CFSD with a fixed drag anchor, while geometry-related metrics remain comparable.

4.7. Human Evaluation

A blind human evaluation is conducted with 36 participants on 12 cases sampled from the curated benchmark, giving 432 judgments for each question. Participants are students with animation, digital media, computer science, electronic information, or film-related backgrounds, and all had prior exposure to animation or visual media production. Before the evaluation, participants are informed that the study is an anonymous perceptual comparison, no sensitive personal data are collected, and they may withdraw before submitting responses. Each case presents the content image with geometric control annotations, the style reference, and anonymous results at a consistent display size on the same display device. Method names are hidden, and the display order is randomized independently for each case and participant. Incomplete responses are excluded before aggregation.
The protocol asks three questions aligned with the task. Participants judge whether the result better preserves the requested drag geometry, whether it better matches the style reference, and which final result is preferred overall. The structure question compares DragDiffusion + Ours (style only) and Ours; the style question compares Ours (drag only) + StyleID and Ours, where StyleID is applied to the intermediate result produced by our drag branch; and the overall question compares four selected joint variants. For the two pairwise questions, statistical significance is assessed with a two sided binomial test against equal preference. For the four way overall question, the preference for Ours is tested against the 25% chance level. Wilson 95% confidence intervals are reported for preference rates, and Fleiss’ κ is reported as a diagnostic for agreement among raters.
Table 10 summarizes the blind human preference results.
All three preferences are statistically significant ( p < 0.001 ). In the four way overall comparison, the remaining selections are 93 for DragDiffusion + StyleID, 63 for Ours (drag only) + StyleID, and 36 for DragDiffusion + Ours (style only). These preferences complement the automatic metrics: ArtFID differences are relatively small in the joint table, while the human study indicates a perceptual advantage for our method in preserving the requested structure and maintaining reference style.

4.8. Qualitative Results

Figure 7 provides qualitative comparisons sampled from the curated benchmark, focusing on cases where local shape and reference style interact visibly.
The sequential baselines move toward the reference appearance after StyleID stylization, but the style stage can soften or undo part of the dragged geometry. Our method is most visibly different around facial contours, eye regions, and local rendering details, where anime style transfer often couples appearance with small structural changes. Very large or ambiguous local deformations remain difficult.
Beyond the curated benchmark examples in Figure 7, Figure 8 provides additional qualitative examples with broader stylized character appearances. These examples illustrate adjacent stylized portrait and character conditions and are not included in the quantitative benchmark.

5. Discussion

The experiments support the central premise of the framework: in joint anime drag and style editing, geometry control and reference style transfer should not be treated as two independent post processing operations. The isolated drag and style results in Table 3 and Table 4 show that the two branches can be analyzed separately, but the joint results in Table 5 show that the main difficulty appears when the two controls interact in the same denoising trajectory. The sequential DragDiffusion + StyleID baseline transfers reference appearance, but it also tends to soften or partially undo the requested local deformation. By contrast, replay from the clean sample anchor lowers the tracking-based residual and reduces VGG patch structure distortion.
The temporal handoff block in Table 8 clarifies why the handoff is useful. Immediate replay after dragging performs poorly because the optimized latent is still close to a transient state: the target motion is present, but the denoising trajectory has not yet stabilized around it. Keeping style injection active during the drag step is also harmful, because reference attention modifies the features used for handle tracking. The short interval without style injection therefore has a functional role rather than serving only as a scheduling detail. It gives the denoising process time to consolidate the dragged structure before the style branch is reactivated. This interpretation is also consistent with the qualitative examples in Figure 7, where our method is most visibly different around eyes, facial contours, and local line abstractions.
The relation to prior attention-based stylization is also important. StyleID [17] demonstrates that replacing self attention keys and values can transfer reference appearance without training, while Style Aligned [26] and Cross Image Attention [18] show that shared or cross image attention can serve as an effective appearance transfer interface. The present setting differs because the content layout is intentionally changed by drag controls before stylization is completed. If the replay branch uses queries associated with the pre drag layout, the style operation can follow the wrong spatial organization. The anchor handoff addresses this mismatch by refreshing the content queries from the post drag clean sample, while the semantic correspondence modulation biases reference tokens toward locally compatible content regions.
The ablations suggest that correspondence replay is not simply a matter of increasing injection strength. The replay strength results in Table 7 show that stronger query side or all token gains do not monotonically improve all metrics. Increasing the gain can preserve some correspondence style benefit, but it can also worsen ArtFID or reduce preservation outside the editable region. Similarly, Table 9 shows that freezing the drag anchor already captures much of the geometric benefit, while the semantic replay components mainly reduce CFSD, suggesting lower feature structure distortion during stylization. Overall, the ablations indicate that the final performance depends on balancing replay strength, correspondence modulation, and handoff timing rather than on any single stronger injection setting.

Operating Boundaries

The current implementation has clear operating boundaries. Large requested structural changes are the hardest cases, especially when they interact with a large gap between the content image and style reference. In these cases, the model may need to synthesize missing internal details instead of only moving existing ones. The schedule is manually chosen, and both the anchor timestep and the replay length affect the balance between structure retention and stylization strength. The drag weighting is deterministic and does not estimate local tracking confidence, so weak texture, flat shading, or ambiguous correspondence can still cause unstable handle motion. Disabling style injection during the active drag step protects tracking features, but it also keeps style from participating in geometric evolution itself. Representative examples of these boundary cases are shown in Figure 9.
An analysis by displacement scale provides a more concrete view of this boundary. When cases are grouped by the maximum requested point displacement, the mean residual distance increases from 91.57 pixels for cases with maximum displacement no greater than 20 pixels to 111.73 pixels for the 20–40 pixel group and 114.52 pixels for cases above 40 pixels. Succ.@20 does not decrease monotonically across these groups, indicating that deformation difficulty is not determined by displacement magnitude alone. Local part type, texture strength, mask extent, and correspondence ambiguity also affect tracking stability. In practice, the pipeline is most reliable for moderate local deformations, such as eye corner shifts, contour adjustments, or hair boundary nudges, while edits that require synthesizing new internal structure, such as enlarging a narrow eye, remain challenging.
These boundaries indicate several directions for future work. A confidence aware drag objective could reduce the influence of unreliable handle matches in flat or weakly textured anime regions. A learned or adaptive handoff policy could replace the manually selected drag and anchor timesteps, especially for cases with very different deformation magnitudes. Finally, allowing limited style information to participate in geometry evolution without corrupting tracking features may help with edits where the reference style implies new internal structure rather than a deformation of existing contours.

6. Conclusions

Anime reference style editing becomes harder when the character layout is allowed to move, because the style branch can no longer rely on the original content layout. The useful handoff state is not the immediate drag result, but a predicted clean sample after a short interval without style injection. Replay from this anchor keeps more of the requested geometry while still allowing reference appearance to enter through attention features. Large requested structural changes remain difficult, especially when the edit requires new internal details rather than moving visible structure.

Author Contributions

Conceptualization, Y.D. and W.Y.; methodology, Y.D. and W.Y.; software, W.Y.; validation, Y.D. and W.Y.; formal analysis, Y.D., W.Y. and Y.G.; investigation, Y.D. and W.Y.; resources, Y.D. and F.C.; data curation, W.Y. and Y.G.; writing—original draft preparation, Y.D. and W.Y.; writing—review and editing, Y.D., W.Y., Y.G. and F.C.; visualization, W.Y.; supervision, Y.D. and F.C.; project administration, Y.D. and F.C. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Ethical review and approval were waived for this study because the reported human evaluation protocol involves anonymous, non-interventional perceptual assessment of generated images only and does not collect sensitive personal data or biomedical information.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the human evaluation before response submission.

Data Availability Statement

The evaluation protocol, annotation format, metric definitions, and experimental configuration are available at https://github.com/KERUkun-kun/anime-drag-style-editing (accessed on 1 July 2026). The curated benchmark annotations, including handle points, masks, and style reference pairings, are included in the repository.

Acknowledgments

The authors thank the project collaborators for discussions on the editing pipeline, annotation workflow, experiment organization, and manuscript preparation.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Ho, J.; Jain, A.; Abbeel, P. Denoising Diffusion Probabilistic Models. In Proceedings of the Advances in Neural Information Processing Systems, Virtual, 6–12 December 2020; Volume 33, pp. 6840–6851. [Google Scholar] [CrossRef]
  2. Rombach, R.; Blattmann, A.; Lorenz, D.; Esser, P.; Ommer, B. High-Resolution Image Synthesis with Latent Diffusion Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 19–24 June 2022; pp. 10674–10685. [Google Scholar] [CrossRef]
  3. Song, W.; Ma, W.; Zhang, M.; Zhang, Y.; Zhao, X. Lightweight Diffusion Models: A Survey. Artif. Intell. Rev. 2024, 57, 161. [Google Scholar] [CrossRef]
  4. Croitoru, F.A.; Hondru, V.; Ionescu, R.T.; Shah, M. Diffusion Models in Vision: A Survey. IEEE Trans. Pattern Anal. Mach. Intell. 2023, 45, 10850–10869. [Google Scholar] [CrossRef] [PubMed]
  5. Chen, H.; Xiang, Q.; Hu, J.; Ye, M.; Yu, C.; Cheng, H.; Zhang, L. Comprehensive Exploration of Diffusion Models in Image Generation: A Survey. Artif. Intell. Rev. 2025, 58, 99. [Google Scholar] [CrossRef]
  6. Zhang, L.; Rao, A.; Agrawala, M. Adding Conditional Control to Text-to-Image Diffusion Models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France, 2–6 October 2023; pp. 3813–3824. [Google Scholar] [CrossRef]
  7. Watanabe, Y.; Togo, R.; Maeda, K.; Ogawa, T.; Haseyama, M. Text-Guided Image Editing Based on Post Score for Gaining Attention on Social Media. Sensors 2024, 24, 921. [Google Scholar] [CrossRef] [PubMed]
  8. Yu, X.; Gu, X.; Hu, X.; Sun, J. ENGDM: Enhanced Non-Isotropic Gaussian Diffusion Model for Progressive Image Editing. Sensors 2025, 25, 2970. [Google Scholar] [CrossRef] [PubMed]
  9. Ruder, M.; Dosovitskiy, A.; Brox, T. Artistic Style Transfer for Videos and Spherical Images. Int. J. Comput. Vis. 2018, 126, 1199–1219. [Google Scholar] [CrossRef]
  10. Jing, Y.; Yang, Y.; Feng, Z.; Ye, J.; Yu, Y.; Song, M. Neural Style Transfer: A Review. IEEE Trans. Vis. Comput. Graph. 2020, 26, 3365–3385. [Google Scholar] [CrossRef] [PubMed]
  11. Selim, A.; Elgharib, M.; Doyle, L. Painting Style Transfer for Head Portraits Using Convolutional Neural Networks. ACM Trans. Graph. 2016, 35, 129. [Google Scholar] [CrossRef]
  12. Zhao, H.; Zheng, J.; Wang, Y.; Yuan, X.; Li, Y. Portrait Style Transfer Using Deep Convolutional Neural Networks and Facial Segmentation. Comput. Electr. Eng. 2020, 85, 106655. [Google Scholar] [CrossRef]
  13. Wang, L. Cartoon-Style Image Rendering Transfer Based on Neural Networks. Comput. Intell. Neurosci. 2022, 2022, 2958338. [Google Scholar] [CrossRef] [PubMed]
  14. Asperti, A.; Colasuonno, G.; Guerra, A. Portrait Reification with Generative Diffusion Models. Appl. Sci. 2023, 13, 6487. [Google Scholar] [CrossRef]
  15. Everaert, M.N.; Bocchio, M.; Arpa, S.; Süsstrunk, S.; Achanta, R. Diffusion in Style. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France, 2–6 October 2023; pp. 2251–2261. [Google Scholar] [CrossRef]
  16. Wang, Z.; Zhao, L.; Xing, W. StyleDiffusion: Controllable Disentangled Style Transfer via Diffusion Models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France, 2–6 October 2023; pp. 7643–7655. [Google Scholar] [CrossRef]
  17. Chung, J.; Hyun, S.; Heo, J.P. Style Injection in Diffusion: A Training-Free Approach for Adapting Large-Scale Diffusion Models for Style Transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 17–21 June 2024; pp. 8795–8805. [Google Scholar] [CrossRef]
  18. Alaluf, Y.; Garibi, D.; Patashnik, O.; Averbuch-Elor, H.; Cohen-Or, D. Cross-Image Attention for Zero-Shot Appearance Transfer. In Proceedings of the ACM SIGGRAPH 2024 Conference Papers; Association for Computing Machinery: New York, NY, USA, 2024. [Google Scholar] [CrossRef]
  19. Ye, H.; Zhang, J.; Liu, S.; Han, X.; Yang, W. IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models. arXiv 2023, arXiv:2308.06721. [Google Scholar] [CrossRef]
  20. Pan, X.; Tewari, A.; Leimkühler, T.; Liu, L.; Meka, A.; Theobalt, C. Drag Your GAN: Interactive Point-Based Manipulation on the Generative Image Manifold. In Proceedings of the ACM SIGGRAPH 2023 Conference Proceedings; Association for Computing Machinery: New York, NY, USA, 2023. [Google Scholar] [CrossRef]
  21. Shi, Y.; Xue, C.; Liew, J.H.; Pan, J.; Yan, H.; Zhang, W.; Tan, V.Y.F.; Bai, S. DragDiffusion: Harnessing Diffusion Models for Interactive Point-Based Image Editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 17–21 June 2024; pp. 8839–8849. [Google Scholar] [CrossRef]
  22. Gatys, L.A.; Ecker, A.S.; Bethge, M. Image Style Transfer Using Convolutional Neural Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 2414–2423. [Google Scholar] [CrossRef]
  23. Huang, X.; Belongie, S. Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017; pp. 1501–1510. [Google Scholar] [CrossRef]
  24. Cai, Q.; Ma, M.; Wang, C.; Li, H. Image Neural Style Transfer: A Review. Comput. Electr. Eng. 2023, 108, 108723. [Google Scholar] [CrossRef]
  25. Xu, Y.; Xia, M.; Hu, K.; Zhou, S.; Weng, L. Style Transfer Review: Traditional Machine Learning to Deep Learning. Information 2025, 16, 157. [Google Scholar] [CrossRef]
  26. Hertz, A.; Voynov, A.; Fruchter, S.; Cohen-Or, D. Style Aligned Image Generation via Shared Attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 17–21 June 2024; pp. 4775–4785. [Google Scholar] [CrossRef]
  27. Kang, M.; Choi, Y.S. FreeMix: Personalized Structure and Appearance Control Without Finetuning. Appl. Sci. 2025, 15, 9889. [Google Scholar] [CrossRef]
  28. Yu, C.; Han, C.; Zhang, C. Multi-Source Training-Free Controllable Style Transfer via Diffusion Models. Symmetry 2025, 17, 290. [Google Scholar] [CrossRef]
  29. Xiang, Z.; Wan, X.; Xu, L.; Yu, X.; Mao, Y. A Training-Free Latent Diffusion Style Transfer Method. Information 2024, 15, 588. [Google Scholar] [CrossRef]
  30. Ling, P.; Chen, L.; Zhang, P.; Chen, H.; Jin, Y.; Zheng, J. FreeDrag: Feature Dragging for Reliable Point-Based Image Editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 17–21 June 2024; pp. 6860–6870. [Google Scholar] [CrossRef]
  31. Liu, H.; Xu, C.; Yang, Y.; Zeng, L.; He, S. Drag Your Noise: Interactive Point-Based Editing via Diffusion Semantic Propagation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 17–21 June 2024; pp. 6743–6752. [Google Scholar] [CrossRef]
  32. Cui, Y.; Zhao, X.; Zhang, G.; Cao, S.; Ma, K.; Wang, L. StableDrag: Stable Dragging for Point-Based Image Editing. In Proceedings of the Computer Vision—ECCV 2024; Springer: Cham, Switzerland, 2024; pp. 340–356. [Google Scholar] [CrossRef]
  33. Zhang, Z.; Liu, H.; Chen, J.; Xu, X. GoodDrag: Towards Good Practices for Drag Editing with Diffusion Models. In Proceedings of the Thirteenth International Conference on Learning Representations, Singapore, 24–28 April 2025. [Google Scholar] [CrossRef]
  34. Mou, C.; Wang, X.; Song, J.; Shan, Y.; Zhang, J. DragonDiffusion: Enabling Drag-Style Manipulation on Diffusion Models. In Proceedings of the Twelfth International Conference on Learning Representations, Vienna, Austria, 7–11 May 2024. [Google Scholar] [CrossRef]
  35. Guo, M.H.; Xu, T.X.; Liu, J.J.; Liu, Z.N.; Jiang, P.T.; Mu, T.J.; Zhang, S.H.; Martin, R.R.; Cheng, M.M.; Hu, S.M. Attention Mechanisms in Computer Vision: A Survey. Comput. Vis. Media 2022, 8, 331–368. [Google Scholar] [CrossRef]
  36. Khan, S.; Naseer, M.; Hayat, M.; Zamir, S.W.; Khan, F.S.; Shah, M. Transformers in Vision: A Survey. ACM Comput. Surv. 2022, 54, 200. [Google Scholar] [CrossRef]
  37. Tumanyan, N.; Geyer, M.; Bagon, S.; Dekel, T. Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, 18–22 June 2023; pp. 1921–1930. [Google Scholar] [CrossRef]
  38. Cao, M.; Wang, X.; Qi, Z.; Shan, Y.; Qie, X.; Zheng, Y. MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France, 2–6 October 2023; pp. 22503–22513. [Google Scholar] [CrossRef]
  39. Pan, Z.; Kuang, Y.; Lan, J.; Zhang, L. High-Precision Image Editing via Dual Attention Control in Diffusion Models Without Fine-Tuning. Appl. Sci. 2025, 15, 1079. [Google Scholar] [CrossRef]
  40. Jiang, R.; Zheng, G.; Li, T.; Yang, T.; Wang, J.; Li, X. A Survey of Multimodal Controllable Diffusion Models. J. Comput. Sci. Technol. 2024, 39, 509–541. [Google Scholar] [CrossRef]
  41. Song, J.; Meng, C.; Ermon, S. Denoising Diffusion Implicit Models. In Proceedings of the International Conference on Learning Representations, Virtual, 3–7 May 2021. [Google Scholar] [CrossRef]
  42. Mokady, R.; Hertz, A.; Aberman, K.; Pritch, Y.; Cohen-Or, D. Null-Text Inversion for Editing Real Images Using Guided Diffusion Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, 18–22 June 2023; pp. 6038–6047. [Google Scholar] [CrossRef]
  43. Hu, E.J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; Chen, W. LoRA: Low-Rank Adaptation of Large Language Models. In Proceedings of the International Conference on Learning Representations, Virtual, 25–29 April 2022. [Google Scholar] [CrossRef]
  44. Kolkin, N.; Salavon, J.; Shakhnarovich, G. Style Transfer by Relaxed Optimal Transport and Self-Similarity. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 16–20 June 2019; pp. 10051–10060. [Google Scholar] [CrossRef]
  45. Zhu, M.; He, X.; Wang, N.; Wang, X.; Gao, X. All-to-Key Attention for Arbitrary Style Transfer. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France, 2–6 October 2023; pp. 23109–23119. [Google Scholar] [CrossRef]
  46. Deng, Y.; Tang, F.; Dong, W.; Ma, C.; Pan, X.; Wang, L.; Xu, C. StyTr2: Image Style Transfer With Transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 19–24 June 2022; pp. 11326–11336. [Google Scholar] [CrossRef]
  47. Liu, S.; Lin, T.; He, D.; Li, F.; Wang, M.; Li, X.; Sun, Z.; Li, Q.; Ding, E. AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfer. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Virtual, 10–17 October 2021; pp. 6649–6658. [Google Scholar] [CrossRef]
  48. Hong, K.; Jeon, S.; Lee, J.; Ahn, N.; Kim, K.; Lee, P.; Kim, D.; Uh, Y.; Byun, H. AesPA-Net: Aesthetic Pattern-Aware Style Transfer Networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France, 2–6 October 2023; pp. 22758–22767. [Google Scholar] [CrossRef]
  49. Zhang, R.; Isola, P.; Efros, A.A.; Shechtman, E.; Wang, O. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–22 June 2018; pp. 586–595. [Google Scholar] [CrossRef]
  50. Heusel, M.; Ramsauer, H.; Unterthiner, T.; Nessler, B.; Hochreiter, S. GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium. In Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA, 4–9 December 2017; Volume 30, pp. 6626–6637. [Google Scholar] [CrossRef]
  51. Ioannou, E.; Maddock, S. Evaluation in Neural Style Transfer: A Review. Comput. Graph. Forum 2024, 43, e15165. [Google Scholar] [CrossRef]
Figure 1. Illustration of the joint drag and style editing task. Each triplet shows the content image with drag controls, the style reference, and the final result produced by our method. Red and blue points indicate the source and target positions of the drag controls, respectively.
Figure 1. Illustration of the joint drag and style editing task. Each triplet shows the content image with drag controls, the style reference, and the final result produced by our method. Red and blue points indicate the source and target positions of the drag controls, respectively.
Applsci 16 06703 g001
Figure 2. Illustration of the proposed AnchorHandoff pipeline for anime drag and style editing. Stage A performs structure editing by drag optimization, referencing Q/K/V guidance before dragging encourages early structure emergence, and style injection is disabled during the drag step. Stage B extracts a clean sample anchor from the stabilized trajectory. Stage C starts style replay from this anchor, using refreshed content queries, cached style keys and values, and soft correspondence between content and style tokens to guide style injection toward compatible regions. Red and blue points denote the source and target positions of the drag controls, respectively.
Figure 2. Illustration of the proposed AnchorHandoff pipeline for anime drag and style editing. Stage A performs structure editing by drag optimization, referencing Q/K/V guidance before dragging encourages early structure emergence, and style injection is disabled during the drag step. Stage B extracts a clean sample anchor from the stabilized trajectory. Stage C starts style replay from this anchor, using refreshed content queries, cached style keys and values, and soft correspondence between content and style tokens to guide style injection toward compatible regions. Red and blue points denote the source and target positions of the drag controls, respectively.
Applsci 16 06703 g002
Figure 3. Detailed illustration of Stage A, structure editing by drag optimization. Reference Q/K/V guidance is applied before t drag , while style injection is disabled during drag optimization because style attention distorts the spatial features used for handle point tracking. The current latent is updated using U-Net tracking features, feature matching for point tracking, adaptive drag weighting, and a drag loss with motion supervision and content preservation terms. The latent then continues for a short interval before clean sample anchor handoff.
Figure 3. Detailed illustration of Stage A, structure editing by drag optimization. Reference Q/K/V guidance is applied before t drag , while style injection is disabled during drag optimization because style attention distorts the spatial features used for handle point tracking. The current latent is updated using U-Net tracking features, feature matching for point tracking, adaptive drag weighting, and a drag loss with motion supervision and content preservation terms. The latent then continues for a short interval before clean sample anchor handoff.
Applsci 16 06703 g003
Figure 4. Detailed illustration of semantic correspondence modulation in the replay stage. Content queries refreshed from the anchor and cached style keys estimate an implicit soft correspondence without external segmentation or part labels. The resulting signal is injected into replay attention through the nearest k style tokens, an attention logit bias, a confidence gain for content tokens, and a support gain for style tokens. Style values are therefore replayed more strongly through compatible local regions while the anchor layout is preserved.
Figure 4. Detailed illustration of semantic correspondence modulation in the replay stage. Content queries refreshed from the anchor and cached style keys estimate an implicit soft correspondence without external segmentation or part labels. The resulting signal is injected into replay attention through the nearest k style tokens, an attention logit bias, a confidence gain for content tokens, and a support gain for style tokens. Style values are therefore replayed more strongly through compatible local regions while the anchor layout is preserved.
Applsci 16 06703 g004
Figure 5. Diagnostic visualization for drag step selection under unified query/key/value guidance. The red boxes mark the selected drag step. Step 9 provides an effective operating point because recognizable facial structure has emerged while the trajectory still retains sufficient latent flexibility for local geometric adjustment; earlier steps are too noisy for reliable handle tracking, and later steps are too rigid.
Figure 5. Diagnostic visualization for drag step selection under unified query/key/value guidance. The red boxes mark the selected drag step. Step 9 provides an effective operating point because recognizable facial structure has emerged while the trajectory still retains sufficient latent flexibility for local geometric adjustment; earlier steps are too noisy for reliable handle tracking, and later steps are too rigid.
Applsci 16 06703 g005
Figure 6. Diagnostic visualization for anchor step selection on the same trajectory. The red boxes mark the selected anchor step. Step 40 provides a stable replay state because the dragged structure has consolidated without the excessive texture saturation and rigidity observed in later candidates such as step 45 or 50.
Figure 6. Diagnostic visualization for anchor step selection on the same trajectory. The red boxes mark the selected anchor step. Step 40 provides a stable replay state because the dragged structure has consolidated without the excessive texture saturation and rigidity observed in later candidates such as step 45 or 50.
Applsci 16 06703 g006
Figure 7. Qualitative comparison on representative joint drag and style cases. Each row shows the content image with drag controls, the target style reference, StyleID applied after different drag editors, and our joint result. The comparison highlights how sequential drag then style baselines can weaken local geometry or produce less coherent facial structure after stylization, while AnchorHandoff better preserves the requested edits and reference-guided rendering.
Figure 7. Qualitative comparison on representative joint drag and style cases. Each row shows the content image with drag controls, the target style reference, StyleID applied after different drag editors, and our joint result. The comparison highlights how sequential drag then style baselines can weaken local geometry or produce less coherent facial structure after stylization, while AnchorHandoff better preserves the requested edits and reference-guided rendering.
Applsci 16 06703 g007
Figure 8. Additional qualitative examples. Each triplet shows the content image with drag controls, the style reference, and the AnchorHandoff result. The examples include anime, painterly portrait, line art, and stylized character cases.
Figure 8. Additional qualitative examples. Each triplet shows the content image with drag controls, the style reference, and the AnchorHandoff result. The examples include anime, painterly portrait, line art, and stylized character cases.
Applsci 16 06703 g008
Figure 9. Representative boundary cases. The first row shows a highly decorative portrait paired with a large style gap, where internal facial details around the nose and mouth become oversmoothed. The second row shows an occluded side view case, where foreground structure and ambiguous correspondence lead to less stable local deformation. The third row shows a line art and chibi like structure gap, where dragged regions can lose or distort local facial details.
Figure 9. Representative boundary cases. The first row shows a highly decorative portrait paired with a large style gap, where internal facial details around the nose and mouth become oversmoothed. The second row shows an occluded side view case, where foreground structure and ambiguous correspondence lead to less stable local deformation. The third row shows a line art and chibi like structure gap, where dragged regions can lose or distort local facial details.
Applsci 16 06703 g009
Table 1. Composition of the curated anime editing benchmark with 120 cases.
Table 1. Composition of the curated anime editing benchmark with 120 cases.
AspectBenchmark Composition
Case definitionEach case contains a content image, a paired style reference, handle–target point annotations, and a binary editing mask.
Point profileFace-centered landmark controls: full-face profiles account for about 94% of the cases, while eyes–nose–mouth profiles account for about 6%.
Drag scaleSmall (≤20 px): about 28%; medium (20–40 px): about 50%; large (>40 px): about 22%, measured by the largest handle–target displacement in each case.
Edited regionThe controls are centered on faces. Most cases involve eye or eyebrow landmarks together with nearby facial structure, while the remaining cases emphasize facial contour, nose, or mouth adjustment.
Reference renderingThe paired style references include cel shaded anime, retro anime illustration, monochrome manga or line dominant comic style, painterly or oil painting like illustration, semi painterly rendering, and high detail decorative illustration examples.
Character design and proportionThe cases include examples with large eye anime proportions, more near human facial proportions, chibi or cartoon like proportions, and distinctive design elements such as complex hair, accessories, glasses, hats, or decorative facial details.
Framing and backgroundThe cases mainly cover closeup and bust anime portraits with plain, moderate, or decorative backgrounds.
Table 2. Main hyperparameters of our method.
Table 2. Main hyperparameters of our method.
ModuleHyperparameterValue
Main trajectoryDDIM steps50
Drag optimization t drag 9
Drag optimizationInner iterations80
Drag optimizationPatch radius r/search radius r p 1/3
Drag lossOutside-mask weight λ 0.1
Drag weightingAdaptive point weightson
Drag weighting α / w min / w max 0.5/0.7/1.5
Anchor handoff t anchor 40
Anchor initializationAdaIN strength α s 0.75
Style replayReplay DDIM steps20
Style replayAttention layers7–11
Query mixing γ / η 0.75/1.5
Style injection s kv 1.0
Correspondence replayReference layer l ref /top-k6/16
Table 3. Drag-only comparison with representative drag editing baselines.
Table 3. Drag-only comparison with representative drag editing baselines.
MethodMD ↓Succ.@20 ↑Mask Out IF ↑CFSD ↓
DragDiffusion 124.7 ± 53.5 0.135 ± 0.083 0.95 ± 0.02 0.64 ± 0.08
FreeDrag 129.9 ± 47.7 0.124 ± 0.112 0.96 ± 0.03 0.57 ± 0.09
DragNoise 128.6 ± 44.6 0.143 ± 0.151 0.96 ± 0.03 0.49 ± 0.05
GoodDrag 134.9 ± 47.3 0.122 ± 0.132 0.97 ± 0.09 0.68 ± 0.10
DragonDiffusion 124.8 ± 41.4 0.140 ± 0.170 0.98 ± 0.05 0.61 ± 0.06
Ours (drag only) 127.9 ± 58.7 0.154 ± 0.119 0.86 ± 0.08 0.57 ± 0.07
Note: Upward arrows indicate higher values are better, downward arrows indicate lower values are better, and bold values mark the best result in each column.
Table 4. Style-only comparison with representative style transfer baselines.
Table 4. Style-only comparison with representative style transfer baselines.
MethodMask Out IF ↑ArtFID ↓CFSD ↓
StyleID 0.73 ± 0.11 33.0 ± 3.1 0.69 ± 0.09
AdaIN 0.68 ± 0.10 31.2 ± 3.4 1.08 ± 0.19
STROTSS 0.59 ± 0.11 30.1 ± 3.0 1.57 ± 0.12
StyA2K 0.70 ± 0.11 30.1 ± 2.9 0.55 ± 0.15
StyTr2 0.62 ± 0.09 30.2 ± 3.3 1.60 ± 0.19
AdaAttN 0.67 ± 0.10 33.6 ± 3.2 1.59 ± 0.17
AesPA-Net 0.66 ± 0.07 39.0 ± 3.8 1.37 ± 0.16
Ours (style only) 0.75 ± 0.09 32.9 ± 3.1 0.60 ± 0.09
Note: Upward arrows indicate higher values are better, downward arrows indicate lower values are better, and bold values mark the best result in each column.
Table 5. Main joint drag and style editing comparison under tracking-based geometry, preservation, style, and feature structure diagnostics.
Table 5. Main joint drag and style editing comparison under tracking-based geometry, preservation, style, and feature structure diagnostics.
MethodMD ↓Succ.@20 ↑Mask Out IF ↑ArtFID ↓CFSD ↓
DragDiffusion + StyleID 116.6 ± 24.4 0.157 ± 0.127 0.78 ± 0.07 20.7 ± 2.4 0.79 ± 0.10
FreeDrag + StyleID 127.4 ± 42.4 0.209 ± 0.132 0.76 ± 0.05 23.0 ± 2.8 0.79 ± 0.09
DragNoise + StyleID 127.5 ± 41.9 0.196 ± 0.119 0.76 ± 0.05 23.5 ± 2.9 0.76 ± 0.08
GoodDrag + StyleID 134.2 ± 44.4 0.196 ± 0.122 0.77 ± 0.05 23.1 ± 2.8 0.91 ± 0.10
DragonDiffusion + StyleID 132.6 ± 46.0 0.172 ± 0.130 0.76 ± 0.05 23.5 ± 2.9 0.85 ± 0.08
Style during drag 126.9 ± 31.3 0.115 ± 0.092 0.80 ± 0.04 22.6 ± 2.9 0.90 ± 0.12
Ours 107.5 ± 25.2 0.217 ± 0.152 0.79 ± 0.04 20.7 ± 2.4 0.51 ± 0.07
Note: Upward arrows indicate higher values are better, downward arrows indicate lower values are better, and bold values mark the best result in each column.
Table 6. Computational cost under the matched local evaluation protocol.
Table 6. Computational cost under the matched local evaluation protocol.
PipelineResolutionRuntime/Case ↓Peak VRAM ↓
DragDiffusion + StyleID 512 × 512 213.6 s44.7 GB
FreeDrag + StyleID 512 × 512 226.4 s19.8 GB
DragonDiffusion + StyleID 512 × 512 116.6 s19.8 GB
GoodDrag + StyleID 512 × 512 205.8 s20.5 GB
DragNoise + StyleID 512 × 512 417.4 s19.8 GB
Ours 512 × 512 72.8 s17.8 GB
Note: Downward arrows indicate lower values are better.
Table 7. Replay-strength sensitivity, showing that stronger replay gains are not uniformly better.
Table 7. Replay-strength sensitivity, showing that stronger replay gains are not uniformly better.
VariantMD ↓Succ.@20 ↑Mask out IF ↑ArtFID ↓CFSD ↓
Ours 107.5 ± 25.2 0.217 ± 0.152 0.79 ± 0.04 20.7 ± 2.4 0.51 ± 0.07
Q gain strong 110.1 ± 26.0 0.217 ± 0.149 0.79 ± 0.05 22.0 ± 2.6 0.53 ± 0.08
All strong 113.4 ± 27.8 0.244 ± 0.157 0.77 ± 0.05 24.4 ± 2.9 0.57 ± 0.09
Note: Upward arrows indicate higher values are better, and downward arrows indicate lower values are better.
Table 8. Temporal coordination ablation, where immediate handoff and style injection during dragging are weaker.
Table 8. Temporal coordination ablation, where immediate handoff and style injection during dragging are weaker.
VariantMD ↓Succ.@20 ↑Mask Out IF ↑ArtFID ↓CFSD ↓
Ours 107.5 ± 25.2 0.217 ± 0.152 0.79 ± 0.04 20.7 ± 2.4 0.51 ± 0.07
Immediate handoff 126.9 ± 31.4 0.205 ± 0.122 0.70 ± 0.09 32.0 ± 3.2 1.35 ± 0.12
Earlier handoff, step 30 105.2 ± 25.9 0.250 ± 0.113 0.80 ± 0.05 22.4 ± 2.7 0.76 ± 0.09
Style during drag 126.9 ± 31.3 0.115 ± 0.092 0.80 ± 0.04 22.6 ± 2.9 0.90 ± 0.12
Note: Upward arrows indicate higher values are better, and downward arrows indicate lower values are better.
Table 9. Semantic replay ablation with a fixed drag anchor, showing that semantic replay mainly improves CFSD.
Table 9. Semantic replay ablation with a fixed drag anchor, showing that semantic replay mainly improves CFSD.
VariantMD ↓Succ.@20 ↑Mask Out IF ↑ArtFID ↓CFSD ↓
Ours 98.5 ± 22.1 0.278 ± 0.123 0.80 ± 0.04 20.9 ± 2.2 0.68 ± 0.08
w/o semantic 96.3 ± 22.7 0.286 ± 0.127 0.80 ± 0.05 20.2 ± 2.1 0.78 ± 0.09
w/o explicit sim. bias 97.0 ± 22.5 0.278 ± 0.123 0.80 ± 0.05 21.0 ± 2.2 0.69 ± 0.08
w/o gain 96.4 ± 22.7 0.282 ± 0.125 0.80 ± 0.05 20.3 ± 2.1 0.77 ± 0.09
Note: Upward arrows indicate higher values are better, and downward arrows indicate lower values are better.
Table 10. Blind human preference results over 36 participants and 12 sampled benchmark cases.
Table 10. Blind human preference results over 36 participants and 12 sampled benchmark cases.
QuestionComparisonOurs VotesPreference95% CI κ
Structure correctnessOurs vs. DragDiffusion + Ours (style only)366/43284.7%[81.0, 87.8]0.61
Style faithfulnessOurs vs. Ours (drag only) + StyleID287/43266.4%[61.9, 70.7]0.53
Overall qualityOurs selected in four way comparison240/43255.6%[50.8, 60.2]0.45
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Ding, Y.; Yu, W.; Geng, Y.; Cai, F. Coordinating Drag-Based Structure Editing and Reference Style Transfer in Diffusion Models for Anime Images. Appl. Sci. 2026, 16, 6703. https://doi.org/10.3390/app16136703

AMA Style

Ding Y, Yu W, Geng Y, Cai F. Coordinating Drag-Based Structure Editing and Reference Style Transfer in Diffusion Models for Anime Images. Applied Sciences. 2026; 16(13):6703. https://doi.org/10.3390/app16136703

Chicago/Turabian Style

Ding, Youdong, Wenjing Yu, Yafan Geng, and Feifan Cai. 2026. "Coordinating Drag-Based Structure Editing and Reference Style Transfer in Diffusion Models for Anime Images" Applied Sciences 16, no. 13: 6703. https://doi.org/10.3390/app16136703

APA Style

Ding, Y., Yu, W., Geng, Y., & Cai, F. (2026). Coordinating Drag-Based Structure Editing and Reference Style Transfer in Diffusion Models for Anime Images. Applied Sciences, 16(13), 6703. https://doi.org/10.3390/app16136703

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop