MSHEdit: Enhanced Text-Driven Image Editing via Advanced Diffusion Model Architecture

Abstract

To address limitations in structural preservation and detail fidelity in existing text-driven image editing methods, we propose MSHEdit—a novel editing framework built upon a pre-trained diffusion model. MSHEdit is designed to achieve high semantic alignment during image editing without the need for additional training or fine-tuning. The framework integrates two key components: the High-Order Stable Diffusion Sampler (HOS-DEIS) and the Multi-Scale Window Residual Bridge Attention Module (MS-WRBA). HOS-DEIS enhances sampling precision and detail recovery by employing high-order integration and dynamic error compensation, while MS-WRBA improves editing region localization and edge blending through multi-scale window partitioning and dual-path normalization. Extensive experiments on public datasets including DreamBench-v2 and DreamBench++ demonstrate that compared to recent mainstream models, MSHEdit reduces structural distance by 2% and background LPIPS by 1.2%. These results demonstrate its ability to achieve natural transitions between edited regions and backgrounds in complex scenes while effectively mitigating object edge blurring. MSHEdit exhibits excellent structural preservation, semantic consistency, and detail restoration, providing an efficient and generalizable solution for high-quality text-driven image editing.

1. Introduction

Text-to-Image Editing (T2IE) is a text-driven image manipulation technique that leverages generative AI to transform natural language descriptions into visual content through direct text-to-image synthesis. It has been widely adopted in fields such as film production and advertising design, significantly reducing the cost and complexity of visual storytelling and creative development. Unlike conventional generative models that create images from scratch, T2IE focuses on modifying existing images by interpreting textual instructions—ranging from minor visual adjustments to complete transformations of core visual elements.

In recent years, diffusion models, inspired by principles of nonequilibrium thermodynamics [1], have emerged as a leading approach in text-driven image editing. By simulating noise diffusion and reverse denoising processes in physical systems, these models are capable of reconstructing complex image content from random noise distributions [1,2,3,4]. As a result, diffusion-based image generation methods have garnered widespread attention. Notable examples include Imagen [5] and Stable Diffusion [6], which exhibit strong capabilities in producing intricate objects and diverse scenes, particularly excelling in tasks such as object replacement and pose transformation. The general development history of the diffusion model is shown in

Table 1. The development history of diffusion models in the field of image editing. The development process of the diffusion model in this table is divided into three stages, and the core processes and representative methods of each period are attached.

Time Period	Stage Category	Key Characteristics	Representative Methods
Before 2020	Early Era of Diffusion Models	Nonequilibrium thermodynamics-inspired generative theory proposed; image editing not yet addressed.	Deep Unsupervised Learning using Nonequilibrium Thermodynamics [1]
2020–2021	Development of Diffusion Probabilistic Models	DDPM and DDIM introduced with efficient sampling, laying the technical groundwork for later editing.	DDPM [2], DDIM [4]
2022–present	Prosperous Era of Stable Diffusion-based Editing	Latent diffusion cuts computational cost; explosive growth of text-/instruction-/multi-condition editing frameworks; standardization underway.	Stable Diffusion [6], InstructPix2Pix [7], DreamBooth [8], MagicBrush, Imagic [9], ICEdit [10]

.

Recent research predominantly adopts Stable Diffusion [6] as the core framework for text-to-image diffusion models. Broadly, existing approaches can be classified into three categories: training-based methods [7,11], test-time fine-tuning techniques [9,12], and model-free strategies [13,14]. Among them, model-free methods eliminate the need for training or fine-tuning, enabling faster deployment, real-time editing, and the integration of multi-modal inputs. Building upon this paradigm, we propose an image editing method that functions entirely without training or fine-tuning. In certain diffusion-based editing applications, mechanisms such as cross-attention are employed to ensure semantic alignment between the generated (or edited) images and the input prompts [13].

Although diffusion model-based image editing methods have made notable progress in generation quality and semantic controllability [15,16,17], considerable limitations persist in structure preservation, detail reconstruction, and precise editing region control.

On one hand, most existing approaches rely on cross-attention mechanisms to achieve semantic alignment between text and image; however, the limited resolution of attention maps impedes fine-grained control over local structures, often resulting in blurred boundaries in edited regions and unnatural fusion between content and background.

On the other hand, mainstream reconstruction methods typically employ inversion strategies such as DDIM [4] or DEIS [18], yet when editing conditions are introduced, the accumulation of numerical errors frequently leads to texture drifting or loss of detail. This issue is especially prominent in style transfer tasks, in which inconsistent style expression or structural distortions may occur. Moreover, current methods often lack efficient spatial localization capability for the editing target in complex scenarios, making it challenging to simultaneously ensure both semantic accuracy and structural fidelity.

To overcome these limitations, we introduce MSHEdit, a diffusion-based image editing framework that requires neither training nor fine-tuning. Users are only required to provide an input image and a concise text prompt describing the desired modification. The system ensures both structural integrity and stylistic coherence while generating results that align with the semantic intent of the input text. Leveraging pre-trained text-to-image diffusion models, MSHEdit significantly reduces computational overhead compared to conventional image editing pipelines, while maintaining high semantic and visual fidelity. Some outstanding experimental results are shown in Figure 1.

The core of MSHEdit lies in style editing and semantic editing, where the style editing adopts the High-Order Stable Diffusion Sampler (HOS-DEIS) module. Currently, most image editing methods using diffusion models employ the DDIM inversion process, but this approach suffers from low sampling efficiency and sampling distortion when capturing overall style and texture. Therefore, we adopt the more advanced DEIS [18] as the foundation for our model’s inversion process and propose the HOS-DEIS inversion process, which better achieves style consistency in generated images.

The semantic editing of MSHEdit utilizes the Multi-Scale Windowed Residual Bridge Attention (MS-WRBA) module, addressing precise determination of editing regions and smooth edge fusion. Since existing methods have issues with inaccurate localization of fine structural areas, we employ cross-scale residual window cross-attention. This module adopts multi-scale window partitioning and a dual-path normalization architecture, dividing input features into multiple local windows and independently calculating attention weights within each window. The model can accurately capture detailed changes in local regions, significantly improving editing area localization accuracy. Moreover, edge areas often exhibit drastic pixel value variations that easily trigger gradient instability. This normalization architecture combined with the residual link mechanism standardizes feature distributions, effectively alleviating gradient explosion and disappearance phenomena in edge regions. It enhances feature expression stability and consistency while reducing edge artifacts and segmentation artifacts.

In summary, our technical contributions are as follows:

(1)

To enhance fidelity in style transfer, we introduce an advanced HOS-DEIS module. Improvements to the DEIS index integrator—specifically through refined higher-order coefficient calculations and integrals—enable high-order stable solutions in the diffusion model’s inverse process. The incorporation of a logarithmic space coefficient remapping mechanism and a dynamic error-compensated integrator further boosts sampling efficiency and image detail reproduction, significantly reducing ambiguous style expression and overfitting typically encountered during style transfer.

(2)

For fine-grained structural editing, we propose a novel cross-scale residual window cross-attention module (MS-WRBA). By leveraging Pre-LayerNorm, dual-path multi-layer normalization, windowed attention computation, and cross-directional feature interaction, our approach substantially decreases the computational complexity of attention matrices while improving the model’s multi-scale contextual understanding. This results in more accurate localization of edited regions and improved edge fusion, yielding seamless integration between edited areas and background.

(3)

By integrating (1) and (2), we introduce MSHEdit, a training-free image editing framework based on a pre-trained text-conditioned diffusion model. The framework synergistically integrates the High-Order Stable Diffusion Sampler (HOS-DEIS) and the multi-scale window residual bridging attention mechanism (MS-WRBA), enabling high-quality and semantically consistent image editing without the need for additional training or fine-tuning. By incorporating high-order exponential integration and dynamic error compensation, the framework significantly improves numerical stability and sampling accuracy during the reverse diffusion process. Furthermore, the MS-WRBA module leverages localized windowed attention modeling and cross-scale feature fusion to enhance the precision of editing region localization and the naturalness of boundary transitions.

We validate the effectiveness of MSHEdit across a range of image editing tasks, including object color replacement, element substitution, shape transformation, and holistic style modification. Experimental results demonstrate that the method not only accurately locates editing targets and maintains semantic consistency in complex scenarios but also outperforms existing methods across multiple mainstream evaluation metrics, showcasing broad practical application potential. On a single RTX 4090, MSHEdit processes a 512 × 512 image in about 2.6 FPS, indicating near-real-time potential after future TensorRT or pruning optimization. Additionally, comprehensive ablation studies further elucidate the contributions of individual components and help clarify current limitations.

2. Related Work

2.1. Three Improvement Directions of the Diffusion Model Under Image Editing

Most current image editing models are developed from the Stable Diffusion [6] diffusion model, which is primarily used for text-to-image generation. This paper therefore adopts the Stable Diffusion [6] model to implement image editing tasks. The editing model first employs “concept inversion” technology to map the concepts in the input image into a representation space controllable by the diffusion model. It then combines this conceptual representation space with the user’s input text description as conditional input to modify and edit the image into a new version that incorporates both the user-specified concepts and aligns with textual semantics. Therefore, improving the diffusion model to achieve superior editing results can be categorized into three main approaches.

The first category comprises training-based approaches. DiffusionCLIP [11], functioning as a weakly supervised model, utilizes CLIP’s text embedding to guide the diffusion model in image editing. Through the DDIM, it converts images into latent noise and fine-tunes the diffusion model during the inversion diffusion process. InstructPix2Pix [7], an instruction-driven fully supervised training model, represents the first method to learn image editing that follows human instructions, training the model through generating original-image-to-edited-image pairs.

The second approach involves fine-tuning during testing. As a hybrid model, Imagic [9] first converts target text into text embeddings, then optimizes these embeddings to reconstruct input images while fine-tuning the diffusion model. Null-Text Inversion [19], functioning as an embedding fine-tuning model, addresses reconstruction failures in DDIM inversion by adjusting empty text embeddings. This reduces the distance between sampling trajectories and inversion trajectories, thereby enhancing reconstruction performance. StyleDiffusion [12] introduces a mapping network that aligns input image features with text prompt embeddings in the embedding space, generating corresponding prompt embeddings [20,21].

While training- and inference-time fine-tuning approaches can be effective for specific tasks, they remain fundamentally limited by their dependence on data-driven parameter updates. Training-based methods require large annotated datasets, incur high costs, and lack flexibility for new editing types, as they demand retraining or adaptation. Inference-time optimization increases computational overhead and may yield unstable results due to inconsistent objectives. Moreover, both approaches often lack explicit modeling of editing regions, making it difficult to achieve precise local control without compromising structural consistency. As a result, such dependence on training restricts both the practicality and scalability of these methods in open-domain image editing scenarios. Consequently, most researchers have shifted their focus to the third approach, and this paper also adopts this method to build the model [22].

The third category comprises methods independent of training and fine-tuning. Prompt-to-Prompt [13] and Pix2Pix-zero [14], as two attention modification models, identify the role of cross-attention layers in spatial relationships between image layouts and prompts, autonomously discovering editing directions within text embedding space. Blended Diffusion [23] integrates text-conditioned content into target images through function substitution and multi-level feature blending, while introducing attention masks to prioritize specific word impressions within designated areas. MagicQuill enables users to directly express editing intent with simple brushes, allowing MLLM to automatically comprehend prompts and diffusion models to precisely execute edits. These methods achieve rapid deployment and real-time editing capabilities without requiring fine-tuning or training, while integrating multi-modal inputs.

2.2. Inversion Process of the Diffusion Model

The denoising process in diffusion models has been a core focus of recent research, with significant advancements, including methods like DDPM [2], DDIM [4], DEIS [18], and PNDM [24], which have contributed to accelerated sampling and improved generation quality. DDPM, the foundational approach in diffusion models, generates high-quality samples by gradually adding noise and learning inverse processes. DDIM enhances sampling through deterministic sampling and adjustable noise parameters. PNDM reduces required sampling steps while maintaining high-quality results by employing non-Markov forward noise processes and optimized backward denoising. DEIS employs an exponential integrator numerical method to efficiently discretize the diffusion model’s backward denoising process, enabling high-quality sample generation with fewer computational steps. Building on DEIS, this paper further optimizes diffusion model sampling efficiency by solving diffusion equations using the exponential integrator. DEIS approaches diffusion models as differential equation problems on manifolds, proposing a pseudo-numeric method that achieves high-quality image generation within just 50 steps [2,4,18,24].

Although inversion methods such as DDIM, DEIS, and PNDM have improved sampling efficiency to varying degrees, their practical application in image editing reveals an inherent trade-off between reconstruction fidelity and editing controllability. Specifically, applying editing conditions with DDIM and DDPM often introduces irreversible texture drift and structural distortions in unedited regions. In contrast, DEIS and PNDM compress the number of sampling steps via higher-order numerical integration, enhancing efficiency but exacerbating truncation errors. As a result, high-frequency details are irreversibly lost during inversion and resampling, manifesting as edge artifacts or inconsistent style representations, particularly in style transfer or local replacement tasks.

HOS-DEIS and MS-WRBA address error accumulation and semantic drift from the perspectives of numerical stability and local semantic control. HOS-DEIS suppresses numerical errors during long inversion using high-order integration and dynamic error compensation. MS-WRBA achieves precise editing region localization and smooth boundary transitions via localized windowed attention and cross-scale feature fusion. Together, they significantly improve structural consistency and semantic accuracy in image editing without requiring training or fine-tuning.

2.3. Attention Mechanism in Semantic Editing

The controllability of diffusion models in image editing tasks largely depends on their built-in attention mechanisms’ precise regulation of text–image semantic alignment and spatial structure preservation. In the U-Net architecture of Stable Diffusion [6], the attention maps encoded by cross-attention and self-attention layers serve dual functions [25]: bridging language prompts to visual regions while maintaining global geometric consistency within self-attention, thereby providing interpretable semantic–spatial prior knowledge for subsequent editing operations. Therefore, the application of attention mechanisms can be broadly categorized into three types: Attention Map Replacement (AMR) modifies original attention maps during editing paths to preserve unedited regions [13,14]; Attention Feature Replacement (AFR) replaces Key/Value features in self-attention layers with reference image structural information [26,27]; Attention + Inversion (A+I) optimizes attention-related parameters (e.g., Key/Value or empty text embedding) during inversion phases to resolve reconstruction failures [19,28]. The introduction of attention maps directs pixel updates to specific regions associated with modified text, enabling precise and localized image modifications without masking techniques [29].

However, these strategies still face several practical challenges. First, while attention graphs and features demonstrate high interpretability, their nonlinear mapping to high-dimensional latent spaces lacks a comprehensive theoretical framework, leading to semantic drift and geometric distortion under extreme editing magnitudes [30,31]. Second, existing methods predominantly rely on fixed semantic spaces from pre-trained text encoders, making it difficult to dynamically adapt to fine-grained or composite instructions, thereby limiting editing flexibility in complex scenarios. Additionally, although local attention constraints enable localized operations, sub-pixel-level mask boundary alignment errors may accumulate into global artifacts, particularly evident in object edges and high-frequency texture regions.

3. Methods

In conventional diffusion-based approaches, the Denoising Diffusion Implicit Model (DDIM) [4] achieves image generation through progressive noise injection and reverse denoising. However, DDIM’s inversion process exhibits discrepancies between noise estimation and sampling, where accumulated errors degrade image quality. Additionally, its multiple denoising steps may lead to over-denoising and detail loss [32], while DEIS suffers from artifacts like edge detail distortion when handling outliers. To address these challenges, this study proposes HOS-DEIS as the core generative framework. As an efficient high-order solver, HOS-DEIS enables rapid and accurate implementation of DDIM’s inverse process. While inheriting DDIM’s deterministic sampling characteristics in diffusion denoising, HOS-DEIS significantly enhances generative efficiency through optimized algorithms.

In MSHEdit, HOS-DEIS (Section 3.1) is adopted as the primary diffusion guidance method. We optimize the numerical solver used during the reverse diffusion process to enable precise grid sampling and address issues such as boundary instability and shadow artifacts. Furthermore, MS-WRBA (Section 3.2) is introduced to enhance the blending performance of high-resolution generation with respect to spatial positioning and editing fidelity along the boundary regions. As illustrated in Figure 2, the model first takes the input image $x_0$ and extracts features using the CLIP image encoder, obtaining ${E_{clip}(x}_0)$.

Simultaneously, the text prompt is encoded via the CLIP text encoder to obtain $E_{clip}\left(P\right)$. These encoded features are then fused through the CLIP cross-attention module to produce the condition embedding ${ϵ}_{\theta }(x_t,t)$. During the diffusion process, noisy image $X_{noise}$ at time step $t$ is iteratively denoised, incorporating this condition embedding. Using HOS-DEIS, we generate an intermediate image $x_t$. This intermediate result is subsequently refined by MS-WRBA, which performs multi-scale window partitioning and attention-based normalization to further aggregate conditional information. Finally, the denoising module $p_{\theta }\left(\left.x_t\right|x_{t-1}\right)$ eliminates residual noise, producing the final edited output image $x_0$.

3.1. DEIS and HOS-DEIS Inversion Process

Style editing tasks are typically categorized into “explicit editing” and “implicit editing”. Explicit editing allows users to specify target styles through explicit text or reference images, with the model directly applying style migration according to instructions. In contrast, implicit editing emphasizes the model’s automatic capture and flexible sampling of style features. Rather than relying on precise stylistic constraints, it achieves diverse style expressions within the style space through a sampling mechanism [31].

HOS-DEIS employs advanced integrators and dynamic error compensation mechanisms to achieve precise restoration of image details and textures during diffusion inverse process sampling. This approach ensures accurate reproduction of stylistic features in explicit style editing while enhancing diversity and controllability of style sampling in implicit style editing. Through logarithmic space coefficient remapping and multi-order differential correction, HOS-DEIS effectively prevents loss or confusion of stylistic features during sampling, ensuring clearer expression of style characteristics. Furthermore, HOS-DEIS adopts a sampling method that requires no training or fine-tuning, directly applying to pre-trained diffusion models without separate training or adjustments for each style or content.

3.1.1. DEIS (Diffusion Exponential Integrator Sampler)

Reverse Diffusion Process: In diffusion models, the reverse diffusion process is the core step where the model reconstructs the target image from noisy data step by step [14,20]. The most critical mathematical relationships are

$${\mathrm{x}}_{\mathrm{t}}=\sqrt{{\overline{\mathsf{\alpha }}}_{\mathrm{t}}}{\mathrm{x}}_0+\sqrt{1-{\overline{\mathsf{\alpha }}}_{\mathrm{t}}}{\mathsf{ϵ}}_{\mathrm{t}}$$

(1)

$${\widehat{\mathrm{x}}}_0={\displaystyle \frac{{\mathrm{x}}_{\mathrm{t}}-\sqrt{1-{\overline{\mathsf{\alpha }}}_{\mathrm{t}}+\mathsf{ϵ}}{\mathsf{ϵ}}_{\mathsf{\theta }}({\mathrm{x}}_{\mathrm{t}},\mathrm{t})}{\sqrt{{\overline{\mathsf{\alpha }}}_{\mathrm{t}}}}}$$

(2)

Here, ${\alpha }_t=\prod _{s=1}^t(1-{\beta }_s)$ represents the cumulative product, ${\widehat{x}}_0$ is the predicted image, and ${\beta }_s$ is the signal loss rate at each time step. This process simulates a gradual denoising dynamic, relying on conditional probability distributions for step-by-step sampling generation (Figure 3). Specifically, assuming that at time step $t$, the input is the noisy intermediate state $x_t$, the model aims to generate the conditional probability distribution of its previous state $x_{t-1}$, which can be expressed as a Gaussian distribution:

$${\mathrm{p}}_{\mathsf{\theta }}({\mathrm{x}}_{\mathrm{t}-1}\mid {\mathrm{x}}_{\mathrm{t}})=\mathcal{N}({\mathrm{x}}_{\mathrm{t}-1};{\mathsf{\mu }}_{\mathsf{\theta }}({\mathrm{x}}_{\mathrm{t}},\mathrm{t}),{\mathsf{\Sigma }}_{\mathsf{\theta }}({\mathrm{x}}_{\mathrm{t}},\mathrm{t}\left)\right)$$

(3)

Here, ${\mu }_{\theta }(x_t,t)$ and ${\mathsf{\Sigma }}_{\theta }(x_t,t)$ represent the mean and covariance matrix of the Gaussian distribution, respectively, both of which are learned by a parameterized U-Net Diffusion Model [33]. The mean ${\mu }_{\theta }(x_t,t)$, which is equal to ${ϵ}_{\theta }(x_t,t)$, determines the center of the sampling, while the covariance matrix ${\mathsf{\Sigma }}_{\theta }(x_t,t)$ controls the uncertainty of the sampling.

DEIS Exponential Integrator Method: DEIS (Diffusion Exponential Integrator Sampling) accelerates reverse diffusion by using exponential integrators. Unlike traditional methods needing many time steps, DEIS reduces sampling steps while preserving high image editing quality. In the DEIS method, the updated formula for the image from time step $t$ to $t-1$ is

$${\mathrm{x}}_{\mathrm{t}-1}={\mathrm{x}}_{\mathrm{t}}+\sum _{\mathrm{i}=1}^{\mathrm{s}}{\mathrm{b}}_{\mathrm{i}}(\mathrm{t}){\mathsf{ϵ}}_{\mathsf{\theta }}({\mathrm{x}}_{\mathrm{t}},\mathrm{t})$$

(4)

Here, $s$ represents the order of the integrator, $b_i(t)$ are time-dependent weights on the time step, and ${ϵ}_{\theta }(x_t,t)$ is the noise predicted by the model, reflecting the noise component contained in $x_0$. DEIS simulates reverse diffusion dynamics by accumulating terms of different orders and can be precisely described using continuous-time dynamical systems theory.

$${\mathrm{x}}_{\mathrm{t}-1}={\mathrm{x}}_{\mathrm{t}}+{\int }_{\mathrm{t}-\mathsf{\Delta }\mathrm{t}}^{\mathrm{t}}\mathsf{\Psi }(\mathrm{t},\mathsf{\tau })\mathrm{F}(\mathsf{\tau })\mathrm{d}\mathsf{\tau }$$

(5)

$$\mathrm{F}(\mathsf{\tau })=-{\displaystyle \frac{1}{2}}\mathrm{G}(\mathsf{\tau }){\mathrm{G}}^{\mathrm{T}}(\mathsf{\tau }){\mathsf{\Sigma }}^{-1}(\mathsf{\tau }){\mathsf{ϵ}}_{\mathsf{\theta }}({\mathrm{x}}_{\mathsf{\tau }},\mathsf{\tau })$$

(6)

Here, $\mathsf{\Psi }(t,\tau )$ is the transition matrix, describing the transition characteristics of the system state from time $\tau$ to $t$, and $G\left(\tau \right)$ is the diffusion coefficient. This integral form shows DEIS’s numerical solution for the inverse stochastic differential equation. By precisely controlling noise propagation along the integral path, it improves numerical stability and sampling efficiency.

3.1.2. HOS-DEIS (High-Order Stable DEIS Solver)

Before introducing the HOS-DEIS method, we first outline its structural improvements over traditional DEIS. Figure 4 shows a flowchart comparing the integral process from continuous ODEs to discrete time steps, with DEIS and HOS-DEIS side by side.

Mitigating Numerical Instability in DEIS: In traditional DEIS, when $\sqrt{{\overline{a}}_t}$ nears zero during the final editing stages, predicted images show abnormal brightness and noise at edges and details. Adding a small $ϵ$ prevents division by zero but can cause large values, leading to precision loss and instability. To fix this, the formula is modified as follows:

$${\widehat{\mathrm{x}}}_0={\displaystyle \frac{{\mathrm{x}}_{\mathrm{t}}-\sqrt{1-{\overline{\mathsf{\alpha }}}_{\mathrm{t}}+\mathsf{ϵ}}{\mathsf{ϵ}}_{\mathsf{\theta }}({\mathrm{x}}_{\mathrm{t}},\mathrm{t})}{\sqrt{{\overline{\mathsf{\alpha }}}_{\mathrm{t}}+\mathsf{ϵ}}}}$$

(7)

This approach reduces numerical instability, making noise prediction easier, prevents artifacts and distortions, and ensures consistent editing across image regions.

Exponential Integrator Optimization: Based on the integral definition, we propose an inversion integral compensation term ${\delta }_t$ for DEIS, which corrects errors from discrete time steps. This term is weighted by a parameter that dynamically adjusts error accumulation during sampling, improving integral approximation accuracy:

$${\mathsf{\delta }}_{\mathrm{t}}=\mathsf{\eta }\sqrt{{\displaystyle \frac{1-{\mathsf{\alpha }}_{\mathrm{t}-1}}{1-{\mathsf{\alpha }}_{\mathrm{t}}}}\left(1-{\displaystyle \frac{{\mathsf{\alpha }}_{\mathrm{t}}}{{\mathsf{\alpha }}_{\mathrm{t}-1}+\mathsf{ϵ}}}\right)}$$

(8)

After introducing the compensation term ${\delta }_t$, the DEIS exponential integrator is modified to provide a new expression that retains noise modeling and reduces discretization errors, improving numerical stability and sampling quality.

$${\mathrm{x}}_{\mathrm{t}-1}={\mathrm{x}}_{\mathrm{t}}+{\int }_{\mathrm{t}-\mathsf{\Delta }\mathrm{t}}^{\mathrm{t}}\mathsf{\Psi }(\mathrm{t},\mathsf{\tau })\mathrm{F}(\mathsf{\tau })\mathrm{d}\mathsf{\tau }+{\mathsf{\delta }}_{\mathrm{t}}\cdot {\mathrm{z}}_{\mathrm{t}}$$

(9)

Large time steps in integration can cause error accumulation and reduce accuracy. To address this, we dynamically adjust an error compensation term based on the difference between predicted and true noise, improving numerical integration precision:

$$\mathsf{\Delta }\mathrm{x}={\mathrm{x}}_{\mathrm{t}-1}-{\mathrm{x}}_{\mathrm{t}}{\int }_{\mathrm{t}-\mathsf{\Delta }\mathrm{t}}^{\mathrm{t}}\mathsf{\Psi }(\mathrm{t},\mathsf{\tau })\mathrm{F}(\mathsf{\tau })\mathrm{d}\mathsf{\tau }+{\mathsf{\delta }}_{\mathrm{t}}\cdot {\mathrm{z}}_{\mathrm{t}}$$

(10)

Finally, the error in the sampling step is represented by $\mathsf{\Delta }x$, and the error is quantified using the integral expression. This method corrects sampling errors and supports further error control and model optimization, improving overall stability and precision.

Logarithmic Domain Coefficient Remapping: During the diffusion process, the transition from time step $t$ to $t-1$ requires a scaling coefficient:

$${\mathrm{c}}_1=\sqrt{{\displaystyle \frac{{\overline{\mathsf{\alpha }}}_{\mathrm{t}-1}}{{\overline{\mathsf{\alpha }}}_{\mathrm{t}}}}}$$

(11)

The coefficient $c_1$ adjusts signal strength during denoising by compensating for attenuation caused by noise addition. For example, $c_1>1$ indicates amplification to restore the original signal.

When ${\overline{\alpha }}_t$ nears $0$ and ${\overline{\alpha }}_{t-1}$ is slightly larger but still small, division can cause underflow, leading to large output fluctuations. Multiple inversion-editing steps may also distort image semantics. To prevent this, a logarithmic transform and stability term $ϵ$ is introduced:

$$\mathrm{l}\mathrm{o}\mathrm{g}{\mathrm{c}}_1={\displaystyle \frac{1}{2}}\mathrm{l}\mathrm{o}\mathrm{g}\left({\displaystyle \frac{{\overline{\mathsf{\alpha }}}_{\mathrm{t}-1}+\mathsf{ϵ}}{{\overline{\mathsf{\alpha }}}_{\mathrm{t}}+\mathsf{ϵ}}}\right)$$

(12)

This enhances signal structure during diffusion and denoising, improving signal strength precision and preserving spatial structure and semantics during multi-step sampling. It reduces structural distortion and detail loss from noise or step discretization.

Multi-Order Correction Mechanism: In the numerical implementation of the original DEIS integrator, the local truncation error is typically $E_{local}=O\left(h^2\right)$, where the time step $h=t-(t-1)=1$. Although the single-step error is small, in the global multi-step integration process, the error accumulates with the number of sampling steps $N$, resulting in a global error $E_{global}=N\cdot O\left(h^2\right)=N\cdot O\left(1\right)$. Furthermore, according to the error accumulation formula, as the number of sampling steps increases, the accumulated error is approximately

$$\sum _{\mathrm{t}=1}^{\mathrm{T}}{\mathrm{E}}_{\mathrm{t}}\approx \mathrm{T}\cdot \mathrm{O}\left({\mathrm{h}}^2\right)=\mathrm{T}\cdot \mathrm{O}(1)$$

(13)

This means that in a longer DEIS sampling interval, for example, when $T=30$, the error may reach $30\times 0.01=0.3$, which is significant for image tasks with pixel values in the range of $[-1, 1]$. Therefore, based on the standard state of the original DEIS, in the first-order integral framework ${ϵ}_{\theta }^{\left(1\right)}={ϵ}_{\theta }(x_t,t)$, we further propose second-order and third-order differences to correct the insufficient precision of DEIS in first-order integral calculations. The second-order and third-order differences are as follows:

$${\mathsf{ϵ}}_{\mathsf{\theta }}^{\left(2\right)}={\mathrm{c}}_2({\mathrm{x}}_{\mathrm{t}},\mathrm{t})-{\mathrm{c}}_2({\mathrm{x}}_{\mathrm{t}+1},\mathrm{t}+1)$$

(14)

$${\mathsf{ϵ}}_{\mathsf{\theta }}^{\left(3\right)}={\mathrm{c}}_3({\mathrm{x}}_{\mathrm{t}},\mathrm{t})-2{\mathrm{c}}_3({\mathrm{x}}_{\mathrm{t}+1},\mathrm{t}+1)+{\mathrm{c}}_3({\mathrm{x}}_{\mathrm{t}+2},\mathrm{t}+2)$$

(15)

Here, $c_2=e^{-\mathsf{\Delta }t}\sqrt{{\displaystyle \frac{1-{\overline{\alpha }}_{t-1}}{1-{\overline{\alpha }}_t}}}$ and $c_3=e^{-2\mathsf{\Delta }t}{\left({\displaystyle \frac{1-{\overline{\alpha }}_{t-1}}{1-{\overline{\alpha }}_t}}\right)}^{3/4}$. Time decay factors are added to control the weight of historical information. The exponent $3/4$ in $c_3$ is an empirical constant balancing precision and stability. The multi-order correction mechanism greatly improves the integrator’s numerical precision and stability during high-step sampling, especially for pixel-sensitive editing tasks. It reduces error accumulation, preserving structural consistency and detail fidelity. The following

Table 2. This table compares local error, global error, and typical step counts across integration methods. Higher-order integrators generally achieve greater accuracy with fewer steps. Step count ranges are empirical and can be adjusted based on application needs.

Order of Integration	Local Error	Global Error	Step Requirements
Level 1 (DDIM)	$O\left(h^2\right)$	$O\left({Nh}^2\right)$	50–100 steps
Level 2 (DEIS)	$O\left(h^3\right)$	$O\left({Nh}^3\right)$	20–30 steps
Level 3 (HOS-DEIS)	$O\left(h^4\right)$	$O\left({Nh}^4\right)$	10–15 steps

quantifies this error reduction:

After second-order correction with high-order integration, the integral error drops by 50–70%. Third-order correction further improves detail fidelity in complex images, reducing the required sampling steps. During style conversion, this prevents banding or blurring in smooth gradients and complex texture areas caused by numerical errors.

3.2. MS-WRBA (Multi-Scale Windowed Residual Bridge Attention)

Recently, a considerable amount of research has adopted large-scale diffusion models [5,27,34], employing various denoising methods. Following the original noise addition and removal processes in diffusion models, enhancing and optimizing the attention mechanisms of diffusion models themselves has become a new research direction. Our approach differs from traditional cross-attention methods [35,36] by utilizing the Multi-Scale Windowed Residual Bridge Attention (MS-WRBA) module.

This module uses multi-scale window partitioning and dual-path normalization. It divides input features into local windows to compute attention independently, capturing fine details while integrating global context. Normalization and residual connections stabilize gradients at edges, reducing artifacts. Local features are then recombined into global features, enabling natural blending and smooth transitions in edited regions.

In our method (Figure 5), to optimize the module’s performance, we partition the input tensor using square local windows while maintaining the original sequence length. Specifically, we divide $X\in {\mathbb{R}}^{B\times H\times W\times C}$ into windows of size $w$, where $H$ and $W$ represent the width and height of the input tensor, and $C$ is the number of channels. The partitioning results in multiple non-overlapping local windows of size $w\times w$, preserving the original sequence length. The partitioned tensor is represented as

$${\mathrm{X}}_{\mathrm{window}}=\mathrm{P}\mathrm{a}\mathrm{r}\mathrm{t}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}(\mathrm{X},\mathrm{w})$$

(16)

Here, $w$ is the window size, $X\in {\mathbb{R}}^{B\times H\times w^2\times C}$, and $N$ is the number of windows, with $N={\displaystyle \frac{H\times W}{w^2}}$.

Within each window, we compute the $Q$, $K$, and $V$ matrices to establish relationships among local features:

$$\mathrm{Q}={\mathrm{W}}_{\mathrm{q}}{\mathrm{X}}_{\mathrm{window}}, \mathrm{K}={\mathrm{W}}_{\mathrm{k}}{\mathrm{X}}_{\mathrm{window}}, \mathrm{V}={\mathrm{W}}_{\mathrm{v}}{\mathrm{X}}_{\mathrm{window}}$$

(17)

Here, $W_q$, $W_k$, and $W_v$ are linear transformation matrices for query, key, and value, respectively. They map input features to different semantic spaces, ensuring that the matrix computation process expresses semantic information in a structured manner within each window. Before computing cross-attention matrices, we apply LayerNorm normalization to the input features to enhance the model’s generalization capability. This yields the fine-scale and coarse-scale feature windows $\tilde{X}$ and $\tilde{Y}$. Subsequently, we apply linear transformations to obtain the query, key, and value matrices:

$$\mathrm{Q}=\tilde{\mathrm{X}}{\mathrm{W}}_{\mathrm{Q}}$$

(18)

$${\mathrm{K}}_{\mathrm{X}}=\tilde{\mathrm{X}}{\mathrm{W}}_{\mathrm{K}\mathrm{X}},{\mathrm{K}}_{\mathrm{Y}}=\tilde{\mathrm{Y}}{\mathrm{W}}_{\mathrm{K}\mathrm{Y}}$$

(19)

$${\mathrm{V}}_{\mathrm{X}}=\tilde{\mathrm{X}}{\mathrm{W}}_{\mathrm{V}\mathrm{X}},{\mathrm{V}}_{\mathrm{Y}}=\tilde{\mathrm{Y}}{\mathrm{W}}_{\mathrm{V}\mathrm{Y}}$$

(20)

Subsequently, we employ the cross-attention method, incorporating an attention mask to restrict the scope of attention and prevent information interaction between windows:

$$\mathrm{A}\mathrm{t}\mathrm{t}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\left(\mathrm{Q},{\mathrm{K}}_{\mathrm{X}},{\mathrm{V}}_{\mathrm{X}}\right)=\mathrm{S}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}\left({\displaystyle \frac{\mathrm{Q}{\mathrm{K}}_{\mathrm{X}}^{\mathrm{T}}}{\sqrt{{\mathrm{p}}_{\mathrm{k}}}}}+{\mathrm{M}}_{\mathrm{X}}\right){\mathrm{V}}_{\mathrm{X}}$$

(21)

$$\mathrm{A}\mathrm{t}\mathrm{t}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\left(\mathrm{Q},{\mathrm{K}}_{\mathrm{Y}},{\mathrm{V}}_{\mathrm{Y}}\right)=\mathrm{S}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}\left({\displaystyle \frac{\mathrm{Q}{\mathrm{K}}_{\mathrm{Y}}^{\mathrm{T}}}{\sqrt{{\mathrm{p}}_{\mathrm{k}}}}}+{\mathrm{M}}_{\mathrm{Y}}\right){\mathrm{V}}_{\mathrm{Y}}$$

(22)

Here, $p_k$ is the dimension of the key vector, used to scale dot-product attention to stabilize gradient propagation and avoid numerical instability. $M$ is the attention mask, ensuring that attention calculations are confined within the current window, preventing cross-window information interaction and reducing the computational intensity of global cross-attention. In this way, the model efficiently captures long-range dependencies among features within local windows. The two attention results are fused, processed through a convolutional layer, and then combined with the normalized input features via a residual connection to obtain the window features.

After completing the attention computation within the windows, to restore the original sequence structure and integrate local and global information, we process the window features through a convolutional layer and combine them with the normalized input features via a residual connection to obtain the window features. We then perform a reverse operation to recombine the processed window features into a global feature representation. This process is as follows:

$${\mathrm{X}}_{\mathrm{window}}=\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}\mathrm{s}\left(\left[\mathrm{A}\mathrm{t}\mathrm{t}\mathrm{n}\left(\mathrm{Q},{\mathrm{K}}_{\mathrm{X}},{\mathrm{V}}_{\mathrm{X}}\right),\mathrm{A}\mathrm{t}\mathrm{t}\mathrm{n}\left(\mathrm{Q},{\mathrm{K}}_{\mathrm{Y}},{\mathrm{V}}_{\mathrm{Y}}\right)\right]\right)+\tilde{\mathrm{X}}$$

(23)

$${\mathrm{X}}_{\mathrm{out}}=\mathrm{R}\mathrm{e}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{s}\mathrm{e}\mathrm{W}\mathrm{i}\mathrm{n}\mathrm{d}\mathrm{o}\mathrm{w}\mathrm{s}\left({\mathrm{X}}_{\mathrm{window}},\mathrm{L}\right)$$

(24)

Here, $L$ is the original sequence length, and $X_{\mathrm{out}}\in {\mathbb{R}}^{B\times L\times C}$ is the final output, with dimensions consistent with the input sequence. The window reversal operation reassembles the local window features into a global sequence, enabling the model to leverage both local features and global context information.

Thus, MS-WRBA, by introducing multi-scale window partitioning and dual-path normalization mechanisms, significantly enhances the model’s perception of local regions, achieving fine-grained control over edited areas. This module independently computes attention weights within each local window, combining cross-scale feature interactions to precisely capture detail changes in edited objects while fully integrating global context information. This enables high-precision object editing in complex scenes. Additionally, MS-WRBA employs residual connections and normalization operations to significantly mitigate edge blurring of edited objects and ensures the consistency and natural transition of the background structure in the edited image through effective fusion of features within and outside the windows.

4. Experiments

Our method can adjust the overall structure of the image and edit the real image. The following content is the detailed results of our experiments based on Stable Diffusion v1.4 [6].

4.1. Experimental Settings

4.1.1. Task Description

To comprehensively evaluate the proposed method, we conducted experiments across multiple creative and diverse image editing tasks. These included the following: (1) Clothing color replacement: Changing a character’s clothing color from one hue to another. (2) Object substitution: Replacing a bicycle with a motorcycle while preserving the original semantic relationship between the subject and vehicle. (3) Object enhancement: Adding sunglasses to a cat’s eyes. (4) Style transformation: Two distinct experimental cases involved converting photos into Van Gogh-style paintings and Chinese urban landscape paintings, respectively.

4.1.2. Evaluation Indicators

To comprehensively evaluate the effectiveness of image editing results, we adopted three metrics: CLIP accuracy (CLIPAcc) [37], structural distance (Structure Dist) [38], and Background-Learned Perceptual Image Patch Similarity (BG LPIPS). The latter is specifically introduced to ensure background retention during editing. This metric helps detect the LPIPS value between the original image and the edited image’s background region, which is why BG LPIPS was incorporated into our evaluation framework.

CLIPAcc is used to measure the similarity between edited images and target text descriptions. Specifically, CLIPAcc evaluates consistency by calculating the cosine similarity between the CLIP embedding vector of the edited image and that of the target text. The formula is expressed as

$$\mathrm{CLIPAcc}={\displaystyle \frac{\mathrm{CLIP}\left({\mathrm{I}}_{\mathrm{edited}}\right)\cdot \mathrm{CLIP}\left({\mathrm{T}}_{\mathrm{target}}\right)}{\parallel \mathrm{CLIP}\left({\mathrm{I}}_{\mathrm{edited}}\right)\parallel \cdot \parallel \mathrm{CLIP}\left({\mathrm{T}}_{\mathrm{target}}\right)\parallel }}$$

(25)

Among them, $I_{\mathrm{edited}}$ represents the edited image and $T_{\mathrm{target}}$ represents the target text description.

Structure Dist is used to measure the structural similarity between the edited image and the original image. We evaluate the structural difference by calculating the Euclidean distance between the structural feature maps of the edited image and the original image. The formula is expressed as

$$\mathrm{Structure}\mathrm{Dist}={\displaystyle \frac{1}{\mathrm{N}}}\sum _{\mathrm{i}=1}^{\mathrm{N}}\parallel \mathrm{S}{\left({\mathrm{I}}_{\mathrm{edited}}\right)}_{\mathrm{i}}-\mathrm{S}{\left({\mathrm{I}}_{\mathrm{original}}\right)}_{\mathrm{i}}{\parallel }^2$$

(26)

Among them, $S\left(I\right)$ represents the structure feature map of the image $I$, and $N$ represents the number of elements in the feature map.

BG LPIPS is used to measure the background region difference between the edited image and the original image. We use LPIPS (Learned Perceptual Image Patch Similarity) to measure the perceptual difference in the background region. The formula is expressed as

$$\mathrm{BG}\mathrm{LPIPS}={\displaystyle \frac{1}{M}}\sum _{j=1}^M\parallel \varphi \left(B{\left(I_{\mathrm{edited}}\right)}_j\right)-\varphi \left(B{\left(I_{\mathrm{original}}\right)}_j\right){\parallel }^2$$

(27)

Among them, $B\left(I\right)$ represents the background region of the image $I$, $\varphi$ represents the feature extraction function of the LPIPS network, and $M$ represents the number of blocks in the background region.

4.1.3. Experimental Environment

This study conducted experiments on a single RTX 4090 GPU (ASUS, Taiwan, China) equipped with 24 GB of video memory, using the Python 3.9 programming language and implementing two algorithms—MS-WRBA and HOS-DEIS—based on the PyTorch 2.3.0 deep learning framework. During image reconstruction, the diffusion model was loaded via the Hugging Face platform, utilizing its pre-trained weights and network architecture for initial configuration. To enhance reconstruction efficiency, we adopted the deterministic HOS-DEIS method, which iteratively maps input images into latent space through stepwise processing. In the editing phase, text prompt embeddings were adjusted in conjunction with the MS-WRBA mechanism to achieve precise image modifications. The default settings for HOS-DEIS during reconstruction included 50 steps and a solver order of 2, while the MS-WRBA employed eight square-shaped windows with encoding dimensions of 2 × 77 × 1024. This configuration yields an average latency of 0.38 s per 512 × 512 image, equivalent to 2.6 FPS.

4.1.4. Experimental Dataset

In this study, to comprehensively evaluate the performance of the proposed image editing method, we selected multiple representative and diverse open datasets that cover rich image content and varied editing tasks. These datasets not only exhibit significant differences in image types, editing operation categories, and data scale, but also provide a solid evaluation foundation for assessing the model’s generalization capabilities and editing outcomes.

Our experimental samples were drawn from three datasets: DreamBooth [8], DreamBooth-v2 [39], and DreamBench++ [40]. The original DreamBooth dataset contains 3000 images, while its v2 extension expands to 750 prompts, generating 3000 additional images. The advanced version further increases the number of prompts to 1350, covering 150 pairs of reference images. In our experiments, we randomly sampled 300 images from DreamBooth-v2 as experimental samples, as shown in

Table 3. Experimental data introduction. This table summarizes the quantities and proportions of various data types selected from the DreamBench-v2 dataset. “Object” refers to images with static, inanimate main subjects; “Living subjects” denotes images featuring different animals or humans as the primary focus; “Styles” indicates landscape images used primarily for style transfer, where the main content is not prominent.

Data Type	Number of Images	Number of Prompts	Proportion
Objects	150	150	50%
Living subjects	90	90	30%
Styles	60	60	20%
Total	300	300	100%

below, while using DreamBench++ as the prompt-image dataset for samples [8,39,40]. These datasets provide comprehensive image resources that enhance the representativeness and stability of experimental results. They achieve full coverage in terms of image content, editing task types, and sample size, laying a solid foundation for evaluating the robustness and generalizability of the proposed method.

4.2. Qualitative Results

The qualitative results in Figure 6 demonstrate the effectiveness of the proposed method across various image editing tasks, including clothing color replacement, bicycle-to-motorcycle conversion, cat sunglasses application, and image style transformation. In the style editing task, the integration of HOS-DEIS with dynamic error compensation enables the model to meticulously reconstruct multi-layered stylistic elements such as color, brushwork, and texture within the style image. Comparative images reveal that traditional methods often suffer from ambiguous style expression or detail loss during style transfer, whereas the proposed method achieves natural fusion and diverse expression of styles while preserving the original image structure. Particularly in scenarios with ambiguous implicit style definitions, HOS-DEIS significantly enhances the clarity and diversity of style representation. More experimental results on processing complex textures, high-resolution images, and other image editing tasks can be found in Appendix A.1. Some cases of editing failure can be found in Appendix A.2.

Furthermore, in fine-tuning tasks such as object addition, the MS-WRBA module significantly enhances the model’s perception of local regions through multi-scale window partitioning and residual bridging attention mechanisms.

Figure 7 presents experimental results showcasing various image editing tasks, including cake replacement, editing of cars and their backgrounds, and dog posture changes. The cake editing task involves substituting the original cake with different objects. For cars, emphasis is placed on modifying background elements, such as the street behind the vehicle, while maintaining semantic coherence. MSHEdit demonstrates a strong understanding of prompts like “blossom street,” “flooded street,” and “snowy street,” preserving overall structure and fine details like metallic textures. Dog editing focuses on posture changes, such as standing, while retaining structural integrity, texture, and seamless blending. Throughout, MSHEdit maintains breed and color consistency, achieving semantic fidelity between original and edited images.

To address potential background noise, we computed BG-LPIPS on the masked non-edited regions in Figure 6. The resulting value (0.044) is an order of magnitude smaller than the edit-region change (0.31), indicating that residual shadows do not introduce significant background disturbance.

These results fully validate the effectiveness of collaborative optimization between HOS-DEIS and MS-WRBA.

4.3. Quantitative Results

4.3.1. Performance Comparison

Table 4. Comparison of image editing methods using diffusion models across different tasks. The first two tasks (clothing color replacement, bicycle → motorcycle) were evaluated using CLIP Acc, BG LPIPS, and StructureDist metrics, which assess editing level, background retention, and structural changes in images, respectively. The other two tasks (cat wearing sunglasses, image style replacement) utilized only CLIP Acc and StructureDist, as background reconstruction was irrelevant to these tasks. Our method achieved the highest CLIP classification accuracy while preserving input image details, demonstrated by low LPIPS scores for background retention and minimal structural distance representation. The data in bold in the table is the best data for the experimental project.

Method	Change the Color of the Clothes			Bicycle → Motorcycle			The Cat Wore Sunglasses		Image Style Replacement
	CLIP Acc ↑	BG LPIPS ↓	Structure Dist ↓	CLIP Acc ↑	BG LPIPS ↓	Structure Dist ↓	CLIP Acc ↑	Structure Dist ↓	CLIP Acc ↑	Structure Dist ↓
Prompt-to-Prompt	18.4%	0.342	0.095	66.0%	0.327	0.085	34.0%	0.082	30.8%	0.079
Pix2Pix	92.2%	0.261	0.082	77.2%	0.273	0.087	69.6%	0.081	52.4%	0.082
MagicQuill	94.0%	0.0667	0.057	87.9%	0.269	0.069	71.2%	0.028	74.3%	0.063
ICEdit	96.0%	0.045	0.048	93.5%	0.243	0.065	74.9%	0.027	77.5%	0.054
Ours (MSHEdit)	86.4%	0.044	0.044	92.8%	0.241	0.063	74.6%	0.025	77.6%	0.052

shows the performance comparison of different methods on various indicators. The proposed method performs well in CLIPAcc, Structure Dist, and BG LPIPS.

4.3.2. Discussion of Experimental Results

In this section, we compare our method with some previous image editing methods based on diffusion models. As can be seen from Table 4, our method (HOS-DEIS + MS-WRBA) shows relatively excellent comprehensive performance in multiple image editing tasks.

As shown in Table 4 and Figure 7, while Prompt-to-Prompt effectively preserves the original image structure, its limited editing capacity makes it difficult to achieve extensive semantic modifications. Pix2Pix shows improved semantic consistency but still exhibits some blurriness during complex structural transformations. MagicQuill relies on manual user annotation for editing areas, and though it achieves higher CLIP scores, its limitations in precise localization and automation remain. In contrast, our proposed method not only strikes a better balance between semantic alignment and structural preservation but also significantly enhances both sampling efficiency and editing quality.

Compared with the recent mainstream and outstanding model ICEdit [10], MSHEdit exhibits comparable performance in experiments, primarily due to its innovative architecture. The HOS-DEIS module enhances sampling accuracy and detail restoration through high-order integration and dynamic error compensation, particularly improving structure preservation and edge blending. The MS-WRBA module boosts editing region localization precision via multi-scale window partitioning and normalization, resulting in more natural and refined edits. Additionally, MSHEdit’s training- and fine-tuning-free design enables quick adaptation to diverse editing tasks, showing strong generalization. Although slightly lower than ICEdit [10] in CLIP accuracy for some tasks, MSHEdit’s advantages in structural retention and background consistency highlight its potential and practicality in image editing.

In quantitative evaluations of mainstream image editing tasks, the proposed method (HOS-DEIS + MS-WRBA) demonstrates superior performance across core metrics including CLIP accuracy, structural distance, and background LPIPS. HOS-DEIS significantly enhances sampling precision and detail reproduction through advanced integration mechanisms and dynamic error compensation, delivering edited images that outperform traditional sampling methods in both structural integrity and detail richness. Simultaneously, the multi-scale window residual bridging attention module in MS-WRBA effectively improves model localization capabilities for editing regions and edge fusion effects, ensuring natural transitions between edited objects and backgrounds in complex scenarios. The synergistic integration of HOS-DEIS and MS-WRBA enables the model to achieve high-quality, efficient text-driven image editing without requiring additional training or fine-tuning, while maintaining editorial accuracy, structural fidelity, and stylistic expression capabilities.

4.4. Ablation Experiment

Table 5. Ablation experiment. We conducted ablation studies under identical experimental conditions, adding one method at a time to evaluate their effects. The configurations included the following: no implementation of our proposed method, using only the MS-WRBA module, using only the HOS-DEIS module, and the final model we proposed (HOS-DEIS + MS-WRBA). The data in bold in the table is the best data for the experimental project.

Method	Change the Color of the Clothes			Bicycle → Motorcycle			The Cat Wore Sunglasses		Image Style Replacement
	CLIP Acc ↑	BG LPIPS ↓	Structure Dist ↓	CLIP Acc ↑	BG LPIPS ↓	Structure Dist ↓	CLIP Acc ↑	Structure Dist ↓	CLIP Acc ↑	Structure Dist ↓
BL + DDIM	85.3%	0.183	0.123	72.0%	0.279	0.087	37.6%	0.085	32.4%	0.082
MS-WRBA + DDIM	90.1%	0.162	0.078	84.4%	0.191	0.072	62.4%	0.081	35.2%	0.081
BL + HOS-DEIS	86.0%	0.281	0.089	72.6%	0.276	0.094	38.0%	0.087	80.2%	0.064
HOS-DEIS + MS-WRBA	86.4%	0.044	0.044	92.8%	0.241	0.063	74.6%	0.025	77.6%	0.052

shows the performance comparison of different cross-attention methods on various indicators.

In this section, we present the components of each experimental group in the ablation study and demonstrate their outcomes. As shown in Table 5, there are four distinct ablation experiment configurations. The experiments evaluated the following four model combinations: (A) Baseline method (using only DDIM inversion); (B) incorporating only the MS-WRBA module; (C) incorporating only the HOS-DEIS module; (D) integrating both HOS-DEIS and MS-WRBA (full methodology).

Experimental results indicate that while baseline method A achieves basic image editing capabilities, it demonstrates mediocre performance in structural preservation and background consistency, with suboptimal localization of editing regions. The introduction of MS-WRBA (B) alone significantly enhances the model’s focus capability and semantic consistency within editing areas, enabling better identification and processing of target objects. However, there remains room for improvement in structural and background restoration. When HOS-DEIS (C) was introduced independently, sampling quality and structural preservation capabilities were notably enhanced, resulting in more natural and detailed editing outcomes. Nevertheless, the absence of attention guidance mechanisms led to a slight decrease in semantic alignment accuracy under certain task conditions.

When HOS-DEIS and MS-WRBA work synergistically (D), the model achieves optimal performance across all mainstream metrics. Specifically, CLIP accuracy, structural distance, and background LPIPS scores all reach peak levels, demonstrating that their combined approach not only enhances localization precision and detail reproduction in editing regions but also maintains semantic consistency and structural fidelity.

Comparative experiments conclusively also prove, in style transfer tasks, that HOS-DEIS contributes significantly by enhancing sampling accuracy through higher-order integration and dynamic error compensation, resulting in more natural and clear textures. The proportion of HOS-DEIS contributing to this task is around 65%. It improves CLIP accuracy by approximately 10% and reduces structural distance by around 15%. Conversely, MS-WRBA reduces background LPIPS by about 20%, mainly through precise localization and natural boundary transitions in edited regions.

And in object replacement tasks, the proportion of MS_WRBA contributing to this task is around 60%. MS-WRBA plays a larger role, with its multi-scale window partitioning and residual bridging attention effectively capturing style details, especially complex textures, reducing structural distance by about 10%. Meanwhile, HOS-DEIS improves CLIP accuracy by approximately 15%, notably restoring fine details and maintaining style consistency in style transfer.

4.5. Subjective Evaluation

To complement the quantitative metrics, a lightweight user study was conducted. Ten non-expert participants performed a two-alternative forced choice (2AFC) test on 20 image pairs generated by the proposed MSHEdit and the strongest baseline (ICEdit). Participants selected (i) the image that better followed the textual instruction and (ii) the one with a more natural structure. MSHEdit was preferred in 84.7% (170/200) of instruction-following trials and 82.0% (164/200) of naturalness trials, both significantly above chance (binomial test, p < 0.001, Cohen’s g = 0.33). The inter-rater agreement was 0.78 (ICC), indicating reliable preference. This mini evaluation confirms that the objective gains reported in Table 4 are perceived by human users.

4.6. Experimental Conclusions

Through systematic experimental evaluations, the proposed method demonstrates significant comprehensive advantages in diverse image editing tasks. First, the HOS-DEIS High-Order Stable Diffusion Sampler effectively reduces sampling steps while enhancing sampling accuracy and detail reproduction capabilities, thereby significantly improving editing efficiency and image quality. Second, the MS-WRBA multi-scale window residual bridging attention mechanism strengthens the model’s precise localization of editing regions and edge fusion capabilities, resulting in editing outcomes that outperform mainstream methods in structural preservation, semantic consistency, and stylistic expression.

Quantitative experiments demonstrate that the proposed model outperforms existing approaches in key metrics, including CLIP accuracy, structural distance, and background LPIPS, achieving balanced performance in semantic editing alignment, structural fidelity, and detail restoration. Ablation experiments further validate the independent contributions and synergistic effects of HOS-DEIS and MS-WRBA: While introducing either module alone enhances performance, their combined application delivers optimal overall performance.

5. Conclusions

In this work, we propose MSHEdit, a text-conditioned pre-trained diffusion model for image editing, with key innovations in the HOS-DEIS and MS-WRBA modules. HOS-DEIS enhances numerical stability and sampling accuracy during the reverse diffusion process by introducing higher-order integrators and dynamic error compensation mechanisms. Specifically, it employs logarithmic coefficient remapping to prevent detail loss due to numerical instability at small step sizes and dynamically compensates integration errors to achieve finer texture and color restoration in complex style transfer tasks. The MS-WRBA module improves editing precision and boundary blending through multi-scale window partitioning and residual bridging attention. By computing attention weights within local windows and integrating cross-scale feature interactions, this module effectively captures fine-grained changes in edited regions while mitigating gradient explosion and vanishing at edges through normalization, ensuring consistent and natural background transitions. As demonstrated by the experiment, this combined approach enables MSHEdit to achieve high-quality, efficient text-driven image editing without requiring training or fine-tuning, balancing semantic alignment, structural fidelity, and detail preservation. With a latency of 0.38 s per 512 × 512 image on an RTX 4090, the framework already approaches interactive speeds and can be further accelerated.

However, our model still has several limitations. First, when processing images with complex textures and fine structures, the model may fail to perfectly preserve or generate all details. Second, despite adopting a self-training and self-tuning design, the model’s generalization ability and editing performance might be somewhat compromised when handling entirely novel and complex datasets. Additionally, processing high-resolution images still requires substantial computational resources, potentially failing to meet real-time requirements in high-demand applications. Finally, there remains room for improvement in the model’s editing accuracy during extreme complex scenarios, particularly when dealing with intricate interactions between multiple objects.

In the meantime, HOS-DEIS inadequately accounts for chromatic channels in structural signal optimization, resulting in local color shifts. To address this, a color correction module based on high-order color space mapping and local color adjustment will be introduced to refine color distribution and ensure color accuracy and consistency. Moreover, to enhance MS-WRBA performance on complex textures and fine structures, we plan to incorporate multi-scale feature fusion to improve local detail capture and explore adaptive attention mechanisms that dynamically adjust attention weights according to task complexity, enabling more precise editing. Finally, to accelerate processing speed for high-resolution images and real-time applications, MSHEdit’s computational architecture will be optimized via pruning and quantization to reduce model parameters, alongside parallel computing strategies leveraging multi-GPU and distributed resources to speed up inference. These improvements aim to extend MSHEdit’s applicability, delivering higher quality and more efficient text-driven image editing and advancing the state of the art in this field.