Frequency-Domain Vision Transformers: Architectures, Applications, and Open Challenges

Abstract

Vision Transformers (ViTs) have achieved strong performance in computer vision but suffer from limited inductive bias, high data requirements, and reduced sensitivity to high-frequency visual details. To address these limitations, Frequency-Domain ViTs (FD-ViTs) incorporate spectral representations—such as Fourier, wavelet, and discrete cosine transforms—into the Transformer pipeline to improve feature expressiveness and robustness. This survey provides a systematic review of FD-ViT architectures and introduces a unified taxonomy based on spectral transformation type, integration level, and computational characteristics. We summarize empirical findings across image classification, image restoration, and domain-specific applications, including medical imaging and remote sensing, highlighting consistent performance patterns and task-dependent trade-offs. Our analysis shows that frequency-domain integration yields modest, context-dependent gains in large-scale classification, while offering more consistent advantages in frequency-sensitive tasks such as image restoration and noise-robust visual analysis. We further discuss key open challenges, including spectral aliasing, phase information loss, evaluation inconsistency, and deployment efficiency, and outline emerging directions toward dynamic spectral operators, multimodal integration, and hardware-aware designs. To the best of our knowledge, this work constitutes the first systematic survey that consolidates the growing body of research on FD-ViT, providing a structured conceptual and methodological reference for future studies on spectral representations in Transformer-based visual learning.

1. Introduction

The field of computer vision has undergone a radical transformation in the last decade under the influence of deep learning. Methods used in modeling visual representations have evolved from Convolutional Neural Network (CNN)-based architectures to Vision Transformer (ViT) architectures, and finally to hybrid approaches that combine these two paradigms. CNN models (AlexNet [1], ResNet [2], and EfficientNet [3], etc.) have formed a revolutionary foundation for learning deep visual representations. These models have been able to exhibit high generalization success even under limited data conditions thanks to their strong “inductive bias” features such as local filtering, weight sharing, and translational equivariance. However, the local nature of CNNs naturally limits their ability to capture long-range relationships; therefore, CNNs may be insufficient for modeling large-scale spatial contexts.

To overcome this limitation, ViT [4], developed by dividing visual data into fixed-size patches, learns long-range dependencies between these patches using a self-attention mechanism. Thanks to its ability to directly model the global context, the ViT architecture has achieved State-of-the-Art (SOTA) performance in many tasks such as image classification, object detection, and segmentation. However, the ViT structure also has some fundamental limitations. High computational cost, weak inductive bias, and lack of translation robustness are the most significant disadvantages of this architecture [5]. In particular, the quadratic complexity of the self-attention mechanism significantly increases the computational load of ViT in large visual datasets. In addition, the lack of structural biases inherent in CNNs reinforces ViT’s dependence on a large amount of labeled data and limits its generalization performance in small datasets.

These limitations have led researchers to develop hybrid solutions combining CNN and Transformer architectures. Models such as DeiT [6], MobileViT [7], EfficientFormer [8], and EdgeViT [9] have achieved more balanced results in terms of both performance and speed by combining the local information extraction power of CNNs with the global context modeling ability of Transformers. However, these approaches still only work on the spatial domain and do not directly evaluate the structural features of the image in the frequency domain. Since visual data inherently contain both spatial and spectral components, processing only the spatial domain can limit information density and reduce the representational power of the model.

To address this deficiency, studies on integrating frequency-domain representations into deep-learning architectures have increased in recent years. This approach aims to represent image information in the frequency domain in a more compact and meaningful way by incorporating classical signal processing methods, such as the Fourier and Wavelet transforms, into modern Transformer architectures. This new family of models, called Frequency-Domain Vision Transformers (FD-ViTs), combines both the local frequency sensitivity of CNNs and the global attention mechanism of Transformers. In this way, the model combines both low-frequency structural information and high-frequency details in the same representation space; this combination improves generalization performance, especially in small datasets, and strengthens noise resistance. Prominent studies in this field include SpectFormer [10], FFTFormer [11], Wave-ViT [12], etc.

However, there is no comprehensive review (survey) study in the literature that systematically addresses these Transformer-based approaches using frequency domain representations [13,14,15]. The vast majority of existing studies only classify spatial or hybrid (CNN–ViT) models; they exclude the new generation Transformer architectures operating in the frequency domain. Therefore, this study aims to fill this gap and present research on FD-ViTs from a holistic perspective.

The main objective of this article is to classify existing FD-ViT approaches in terms of theoretical framework, architectural structure, and computational efficiency, to discuss the advantages and limitations of these methods, and to establish a conceptual framework for future research. Accordingly, the study aims not only to summarize the existing literature but also to guide advancements in this field by discussing the new perspective that spectral representations bring to deep visual learning.

1.1. Why Frequency Matters in Vision Transformers?

When it comes to structural analysis of images, the importance of both spatial and frequency components has long been known. In particular, in the context of image processing and visual modeling, Fourier, Wavelet, and DCT (Discrete Cosine Transform) transforms allow signals to be expressed in the frequency domain, enabling the separation of low- and high-frequency components. While such spectral analysis tools have been used for many years with classical methods, they also provide an important framework for understanding the intrinsic behavior of deep-learning models. For example, Fourier-based analyses are an effective tool for revealing which frequency components models give more weight to; in some studies, the theoretical basis of this approach to interpreting the specialized behavior of ViT in the frequency domain has been examined. In particular, it has been shown that the self-attention mechanism treats the input signals like a low-pass filter and therefore cannot adequately preserve high-frequency details, and it has been reported that this situation oversmoothes the feature maps of the model as it deepens [16].

Studies on the frequency characteristics of CNN-based models reveal that these models exhibit a strong bias toward low-frequency components across layers, and this bias is part of their generalization success. On the other hand, Transformer-based architectures, unlike classical CNNs, do not directly exhibit a bias in the frequency domain; their self-attention mechanism tends to treat all spatial components equally, which can lead to the model focusing relatively less on high-frequency components. This can negatively impact the model’s generalization ability, particularly in small datasets or tasks requiring great detail [17].

These theoretical observations provide motivation for improving the performance of Transformers in terms of spectral features. Efforts to develop structures that directly process in the frequency domain or incorporate frequency components into the model’s learning process, particularly in self-attention-based models like ViTs, are driven by this motivation. In this context, it is argued that architectures containing spectral layers and Fourier-based processing allow for the modeling of frequency information along with spatial information, enabling the model to better capture both high and low-frequency details. For example, architectures like SpectFormer [10] aim to enrich visual representations by combining frequency domain components with self-attention. In addition, multi-scale frequency analysis tools such as wavelet transforms have been integrated into Transformer architectures. In wavelet-based ViT approaches, wavelet decomposition separates high- and low-frequency components in a compact and lossless manner, enabling the model to learn both global and local frequency characteristics. Such integrations demonstrate that spectral analysis methods have the potential to improve both computational efficiency and local–global frequency sensitivity in Transformer models [12].

In conclusion, making attention-based models like ViT sensitive to the frequency domain in addition to the spatial domain can improve both the spatial and spectral information representation of the models. Accordingly, spectral analysis provides a powerful framework for understanding and improving the learning behavior and generalization ability of Transformers from a deeper frequency perspective.

1.2. Objectives and Contributions of This Survey

This study aims to provide a focused and systematic survey of ViT architectures that explicitly integrate frequency-domain representations as a core component of their design. While existing survey papers on ViT and hybrid CNN–Transformer models have discussed spectral bias, frequency-aware learning behavior, or frequency analysis primarily as supporting theoretical perspectives, they do not offer a dedicated, architecture-centric review of methods that operate directly in the frequency domain. In particular, prior surveys typically treat frequency-related concepts as auxiliary analysis tools or application-specific techniques, rather than as a unifying architectural principle for Transformer-based visual learning.

Within this framework, the present work positions itself as one of the first surveys to systematically consolidate and organize FD-ViTs under a unified methodological framework. The scope of this survey is intentionally limited to Transformer-based architectures that incorporate explicit frequency-domain transformations—such as Fourier, Wavelet, and Discrete Cosine Transforms—within the model pipeline, including tokenization, attention mechanisms, or intermediate feature processing stages. Approaches that merely analyze frequency behavior without architectural integration, or that rely solely on convolutional frequency processing without Transformer-based token mixing, are considered outside the primary scope.

Accordingly, the main objective of this survey is to synthesize existing FD-ViT studies through a structured taxonomy that highlights architectural integration strategies, representational trade-offs, and computational characteristics, thereby revealing common design patterns, empirical limitations, and open research directions in frequency-domain Transformer-based visual learning. The main contributions of this article can be summarized as follows:

• This survey introduces a structured taxonomy of FD-ViTs by categorizing existing FD-ViT models according to their spectral transformation types and integration levels, thereby reducing conceptual fragmentation in the literature.
• It provides a comparative synthesis of Fourier-, wavelet-, and DCT-based approaches, highlighting their respective impacts on performance characteristics, computational efficiency, and representational behavior across different vision tasks.
• The survey systematically analyzes frequency-aware attention and token-mixing mechanisms, with particular emphasis on their implications for computational complexity and information density in Transformer architectures.
• By aggregating and organizing experimental evidence from diverse application domains, this work identifies recurring empirical patterns and evaluation gaps in current FD-ViT studies.
• Finally, the survey summarizes key open research challenges, including phase information handling, spectral aliasing, and generalization behavior, and positions them within a coherent framework to inform future investigations.

Overall, this work provides a consolidated reference that clarifies the role of frequency-domain representations in ViT, bridging architectural design choices with empirical observations across tasks and application domains.

2. Background and Theoretical Foundations

In this survey, the terms frequency-domain, spectral, and frequency-aware are used with distinct but related meanings. The term frequency-domain refers specifically to representations obtained through explicit mathematical transformations such as Fourier, wavelet, or discrete cosine transforms. The term spectral is used in a broader sense to describe representations, operators, or analyses involving frequency components of visual signals. In contrast, frequency-aware refers to models or mechanisms that incorporate sensitivity to frequency characteristics implicitly or explicitly, without necessarily operating directly in the frequency domain. For consistency, this survey uses frequency-domain integration when referring to architectures that explicitly apply spectral transformations within the Transformer pipeline.

Frequency domain analysis is a fundamental approach that has long been used to understand the intrinsic structure of visual data. An image consists of energy-differentiated frequency components; lower frequencies represent the overall structure and form, while higher frequencies represent fine details and textural information. This separation allows for a more organized and compressed expression of visual information. Transforms such as Fourier, Wavelet, and DCT enable the separation of these components, providing an important theoretical foundation for both modeling and feature extraction [18,19]. DCT [20] stands out particularly for its ability to concentrate energy density in low-frequency components. This transformation suppresses the effect of high-frequency components by expressing the signal on a cosine basis, converting information into a more compact form. DCT’s success in tasks such as image compression, feature extraction, and noise reduction stems from its efficient capture of the inherent frequency organization of visual information. This feature provides a valuable prerequisite for making the learning process in deep models more stable and data-efficient.

In modern approaches, these classical transformations are no longer considered merely as preprocessing tools, but as part of the model’s learning process. Data represented in the frequency domain provides a more abstract level of information by filtering out unnecessary spatial details. This strengthens both the noise-resistant nature and the generalization capacity of the model [17]. In this respect, frequency analysis provides structural regularization to the learning process while also increasing the interpretability of the obtained representations. Within this theoretical framework, FD-ViTs models integrate frequency transformations directly into the model’s pipeline. This allows the energy distribution of the signal to be considered, and the learning process to be carried out in both the spatial and spectral domains. As a result, frequency information becomes not just an additional feature for modern Transformer architectures, but a fundamental component of learning.

2.1. Vision Transformers: A Brief Overview

The ViT [4] architecture has, for the first time, demonstrated the potential of the attention mechanism in visual modeling on large-scale image data. Introduced by Dosovitskiy et al., the ViT differs from classical convolutional architectures by partitioning the input image into fixed-size patch blocks and processing these patches as a sequential tokenized representation, analogous to word tokens in natural language processing. As illustrated in Figure 1, each patch is mapped into a patch embedding space through a linear projection, and a learnable [CLS] token, used for classification tasks, is appended to the token sequence. Subsequently, positional encoding is added to the token sequence to preserve spatial information.

The Transformer Encoder layers, which form the core of the ViT architecture, consist of multi-head self-attention (MHSA) and position-independent feed-forward (MLP) sub-layers, as shown in Figure 1; this structure is further supported by residual connections and layer normalization (LayerNorm). Encoder layers are stacked multiple times (N layers), allowing the model to learn relationships between all patches in the image in a global context. While this global relationship modeling capability eliminates the limited receptive field constraint in CNNs, it leads to O(N²) computational complexity depending on the number of tokens and creates a high computational cost, especially in high-resolution images [21,22].

Among the representations produced at the output of the Transformer encoder, only the output vector corresponding to the [CLS] token is used for classification, and this representation is passed to the feed-forward–based classification head (MLP head) illustrated in Figure 1. The sequential stacking of encoder layers enables the model to learn image relationships at multiple levels of abstraction, and the final output is generated based on this representation.

Although the ViT architecture offers high flexibility and strong representational capacity in global context modeling, it lacks inductive biases such as local filtering and weight sharing found in convolutional networks. While these biases in CNNs provide built-in regularity and translational stability to the model, ViT learns these relationships entirely from the data. This increases ViT’s dependence on large-scale labeled datasets. Indeed, while the original ViT model exhibited high performance when trained on massive datasets like JFT-300M, significant accuracy losses were observed when trained directly on more limited datasets (e.g., ImageNet-1K) [4,22]. In this context, the fact that the ViT architecture presented in Figure 1 operates solely on spatial representations emerges as one of the primary factors limiting the model’s data efficiency. Since the pure attention mechanism does not explicitly model local structural information, such relationships must be learned indirectly during training, which in turn increases the tendency toward overfitting, particularly when trained on small datasets [23]. More efficient variants, such as DeiT [6] or Swin Transformer [24], have proposed approaches like knowledge distillation and local attention windows to mitigate these limitations. However, these methods still fundamentally operate on spatial representations, and the model’s sensitivity to frequency-domain information remains limited.

In summary, while the ViT architecture illustrated in Figure 1 provides a powerful framework for modeling global context, it exhibits certain limitations in terms of data efficiency and sensitivity to local structural information. As discussed in the next section, these limitations make the integration of frequency-based transformations (Fourier, Wavelet, and DCT) into the ViT framework a strong motivation from both theoretical and practical perspectives.

2.2. Frequency-Domain Fundamentals

Frequency domain representation is a fundamental concept that has been used for many years to understand the structural content of image signals. While images may consist of densely correlated pixels in the spatial domain, the same information can be represented more compactly in the frequency domain. Frequency transformations decompose the image signal into its energy-based components, revealing which structural information is dominant at different scales. This perspective offers critical advantages in terms of information density, noise robustness, and explainability in deep-learning models [18,19].

Classical signal processing methods such as the Fourier, Wavelet, and DCT transforms stand out with their different frequency resolutions and localization characteristics. In modern deep models, these transforms are used not only for feature extraction but also as a regularizing part of the learning process [16,17]. The following subsections briefly present the theoretical framework of frequency domain analysis through these three fundamental transformations.

2.2.1. Fourier Transform

The Fourier Transform (FT) is a fundamental mathematical tool that reveals the energy spectrum of a signal by decomposing it into sinusoidal components at different frequencies. In the context of image processing, the two-dimensional FT describes the overall structure of an image by analyzing the frequency correspondences of spatial variations [18]. Low-frequency components represent wide-range brightness variations, while high-frequency components represent edges, textures, and details. Therefore, the FT is the most commonly used representation for information decomposition based on the energy density of visual data.

In natural images, energy distribution is typically concentrated in low-frequency components, a phenomenon attributed to the 1/f-type spectral profile of signals [25]. This property has also been used to explain the behavior known as “frequency bias” in deep-learning models, whereby neural networks tend to learn low-frequency components more readily than high-frequency details [26]. This observation implies that models employing self-attention mechanisms, such as ViTs, can effectively capture low-frequency structural information, while often remaining insufficient in modeling high-frequency details.

Fourier representations are effective in capturing large-scale structural relationships thanks to their global analytical power. However, this transformation loses local location information; that is, it does not show where a particular frequency component is located in the image. This limitation is attempted to be compensated for, especially in ViT structures, by mechanisms such as Fourier positional encoding and spectral attention [27]. These methods make it possible to develop models that can represent both global and local structures by combining amplitude and phase information in the frequency domain with attention maps.

In addition, Fourier analysis provides a theoretical framework for understanding the internal behavior of deep networks. Convertini et al. [28] demonstrated that DFT (Discrete Fourier Transform)-based spectral features provide better generalization than classical CNN representations in distinguishing deepfake images. In recent years, the direct integration of the Fourier transform into the attention module in frequency-based Transformer models has given rise to a new paradigm called frequency-aware self-attention. In this approach, each attention head is weighted not only according to spatial similarities but also according to the energy distribution in the frequency domain. Thus, the model becomes sensitive to high-frequency details while preserving low-frequency structural information [29]. Such hybrid methods combine both the global modeling power of ViT and the advantage of compact representation of the frequency domain.

In summary, FT is not only a preprocessing tool in frequency-based visual learning but also a model component that provides spectral awareness. Thanks to both its energy-based representational power and low computational complexity, it is predicted to be used as a fundamental building block of Fourier analysis in architectures such as FD-ViT.

2.2.2. Wavelet Transform

Wavelet Transform (WT) is a multi-resolution transform that can simultaneously resolve both the spatial and spectral components of a signal. Unlike the global nature of FT, WT separates the signal into local regions, incorporating both the overall low-frequency structure and the detailed high-frequency information within the same representation [19]. This feature is of great importance, especially for modeling the local-global energy balance in natural images. Mathematically, WT projects the signal into translated and scaled versions at different scales and locations using a mother wavelet function ψ(t). This allows capturing the variable frequency components in the signal. Thus, wavelet analysis overcomes the limitation of FT by providing both time (or space) and frequency resolution [30]. With this advantage, WT can extract both global and local information through multi-scale energy separation. The low-frequency components of the images carry general shape and color variations, while the high-frequency components represent edges, textures, and details [31]. This feature has made WT the preferred tool for tasks such as image restoration, noise reduction, and super-resolution [32,33].

In the deep-learning literature, the integration of WT has been achieved through wavelet decomposition in CNN architectures. For example, the Multi-Level Wavelet CNN (MWCNN) model by Liu, Zhang, Zhang, Lin, and Zuo [32] decomposes feature maps into multi-level frequency components by applying the Discrete Wavelet Transform (DWT) in each convolution layer. In this way, both large-scale structural information and local details can be learned in the same representation. Similarly, Wavelet-SRNet [33] has ensured the preservation of high-frequency details by combining WT-based subbands with CNN layers in super-resolution tasks. There are also advances in representing wavelet basis functions in learnable wavelets instead of fixed ones. Khan and Yener [34] achieved adaptive frequency sensitivity by optimizing wavelet filter widths during neural network training. This approach reduces the spectral bias of deep networks and improves their noise stability.

In conclusion, WT represents not only a transformation tool for deep visual models but also a powerful building block for frequency-based multiscale learning. The potential to integrate the local filtering nature of CNNs with the global modeling capabilities of Transformers has made WT a central component in FD-ViTs research.

2.2.3. Discrete Cosine Transform

DCT is a frequency transform that concentrates most of the energy in low-frequency components by representing a signal solely through cosine basis functions. Unlike FT, it uses only real coefficients, thereby reducing computational costs and increasing the energy density of the representation. This feature of DCT allows the original structure to be largely preserved even while neglecting phase information in the signals. Therefore, DCT has long been preferred in tasks requiring compact, energy-based representation in the frequency domain [20,35].

The most common application of DCT is seen in the JPEG compression standard. The image is divided into 8 × 8 blocks, and two-dimensional DCT is applied to each block; while low-frequency coefficients are preserved, high-frequency components are brought close to zero, significantly reducing the amount of data [36]. This process is based on the principle that the human visual system is less sensitive to high frequencies and provides a powerful feature called energy compaction. In this respect, DCT is an effective tool not only in image compression but also in tasks such as noise suppression and feature extraction [37].

Today, DCT has evolved from being merely a classical signal processing tool to being used as a frequency-based representation mechanism in deep-learning models. Particularly in tasks requiring frequency sensitivity, the DCT transform is integrated into CNN and Transformer-based models with the aim of directly processing spectral information. For example, Li et al. [38], in their study called Discrete Cosin Transformer (DCFormer), increased both processing efficiency and learning capacity by modeling images in the DCT space. Similarly, Zeng et al. [39] showed that structural differences in multitemporal remote sensing images can be detected more effectively with a DCT-based attention mechanism.

DCT offers advantages not only in frequency domain information representation but also in computational efficiency. Compared to FT, it works with real coefficients and requires less computation because it consists only of cosine terms. This feature allows DCT to be used in embedded systems or resource-constrained devices [35]. Furthermore, DCT has the potential to direct the information flow of attention mechanisms on a frequency-based approach by analyzing the energy density between image blocks. In this respect, it is one of the fundamental building blocks that can be used in the FD-ViTs family. In conclusion, DCT is a powerful tool that provides energy compaction, frequency sensitivity, and information efficiency in both classical signal processing and modern deep-learning approaches. In FD-ViT research, DCT-based representations have been shown to improve explainability in Transformer architectures and support more stable model learning under low-data conditions.

2.3. CNNs vs. ViTs in the Spectral Domain

When the architectural differences between CNNs and ViTs are examined at the spectral level, the fundamental distinction between the learning behaviors of these two approaches becomes clearer. The energy distribution of the low and high-frequency components of image signals plays a direct and decisive role in generalization capacity, noise stability, and data efficiency, depending on how a model represents these components [16]. Therefore, comparing the frequency-domain characteristics of the models is a fundamental necessity to understand the underlying reasons for architectural differences.

CNNs exhibit a layer-dependent behavior in the frequency domain that differs from that of ViTs, owing to their local convolutional filters. Although neural networks generally demonstrate a spectral bias toward learning low-frequency functions more rapidly [26], the convolutional filters in the early layers of CNNs can operate with higher sensitivity to high-frequency components such as edges, textures, and local details. As depth increases, these representations progressively evolve toward lower-frequency and more abstract structural information [23]. This characteristic enables CNNs to effectively capture local texture and edge details, while simultaneously making them relatively more sensitive to high-frequency noise.

ViT architectures, in this regard, exhibit a distinct spectral profile. The multi-head self-attention (MSA) mechanism that forms the core of ViTs tends to emphasize low-frequency components in the frequency domain. As ViT layers become deeper, the repeated application of this effect leads to a progressive attenuation of high-frequency signals and causes representations to homogenize around low-frequency components. This phenomenon is considered one of the fundamental frequency-domain causes underlying issues such as oversmoothing, rank collapse, and patch uniformity, which are known to limit the performance of deep ViT models in the literature [16].

Spectral differences are important not only in terms of accuracy, but also with respect to robustness and behavior under distribution shifts. Paul and Chen [40] report that ViTs can exhibit greater robustness than CNNs to general corruptions, suggesting that this behavior may be associated with the low-frequency–dominated representations learned by ViTs. In contrast, Kim et al. [41] analyze the adversarial behavior of ViTs and show that these models can be more vulnerable than CNNs to phase-based attacks that specifically target low-frequency regions. When examining the architectural origins of these behavioral differences, three key factors stand out. First, the local receptive fields in CNNs are effective at capturing high-frequency details; by contrast, the patch-based tokenization employed by ViTs may lead to an early attenuation of local structures and, consequently, high-frequency information [42]. Second, the MSA mechanism, in its default form, tends to suppress high-frequency signals, which causes spectral sensitivity to be governed less by the architecture itself and more by the learning process or auxiliary modules. Third, the spectral diversification capacity of CNNs and the spectral aggregation (smoothing) tendency of ViTs provide a complementary relationship that is particularly beneficial for hybrid architectures [23].

Accordingly, the spectral differences between CNNs and ViTs are instructive for the design of new model architectures. The complementary relationship between the ability of CNNs to capture high-frequency components and the strength of ViTs in modeling low-frequency global context constitutes the core motivation for frequency-domain–based hybrid approaches. Accordingly, integrating techniques such as AttnScale—which aims to make the self-attention mechanism more spectrally balanced by approximating the attention layer to an all-pass filter—or FeatScale—which reweights high-frequency components—into ViT architectures seeks to mitigate this structural imbalance [16].

In conclusion, when the CNN–ViT comparison is considered at the spectral level, it is seen that the difference stems not only from architectural preferences but also from the dynamics of the learning process in the frequency domain. While CNNs produce more structurally oriented and stable representations in the frequency domain, ViTs tend to produce more flexible but spectrally unstable representations. This observation positions FD-ViT approaches not as optional enhancements but as a structural necessity for addressing the fundamental limitations of ViT architectures.

2.4. Anti-Aliasing in Patch Tokenization

Patch embedding in ViT architectures can be interpreted, from a signal-processing perspective, as a strided sampling operation that may introduce aliasing artifacts when high-frequency components exceed the Nyquist limit. In particular, non-overlapping patch extraction can disrupt spatial continuity and propagate discontinuity-induced artifacts into attention maps. This observation highlights the importance of band-limited representations prior to tokenization. Different FD-ViT architectures address aliasing through strategies that depend on the underlying spectral transformation.

Wavelet-based architectures provide the most structurally grounded approach to aliasing mitigation. Models such as Wave-ViT [12] and SwinWave-SR [43] replace conventional pooling with invertible downsampling mechanisms. These approaches decompose images into multi-resolution sub-bands (LL, LH, HL, HH) before spatial resolution reduction, effectively isolating low- and high-frequency components. In some architectures, such as WaveFusion-style designs, filter banks are constructed to satisfy aliasing-cancelation conditions, making wavelet tokenization naturally consistent with Nyquist–Shannon sampling principles [44].

Fourier-based FD-ViT architectures typically mitigate aliasing indirectly through spectral filtering rather than through explicit sampling-theoretic constraints. Models such as GFNet [45] and SpectFormer [10] apply learnable global spectral filters during token-mixing stages. These filters can suppress unstable high-frequency components introduced by discontinuities, thereby reducing aliasing artifacts. However, unlike wavelet-based approaches, this strategy functions primarily as spectral regularization rather than guaranteeing band-limited sampling.

DCT-based approaches, including DCT-ViT [46], DctViT [47], and related architectures, manage aliasing risk through energy-compaction mechanisms. By pruning or selectively retaining DCT coefficients before token compression, these models suppress high-frequency components that may contribute to aliasing. However, this strategy introduces a fundamental trade-off between aliasing control and preservation of fine spatial texture, as aggressive high-frequency suppression may degrade detailed image structures.

3. Taxonomy of FD-ViT Approaches

As discussed in the previous section, the spectral differences between CNN and ViT architectures indicate that ViTs require explicit structural guidance in the frequency domain. Emerging from this need, FD-ViT approaches aim to mitigate spectral imbalances by integrating different frequency transformations into ViT architectures. However, existing studies in the literature address these approaches in a fragmented manner, primarily categorized by the type of frequency transformation employed, and therefore lack a unified framework. In this section, existing FD-ViT studies are examined under a systematic classification based on the type of frequency transformation used. Accordingly, the literature is organized into three main categories: Fourier-based, Wavelet-based, and DCT-based approaches. This classification not only provides a transformation-oriented taxonomy but also enables a comparative evaluation of how each frequency representation influences ViT architectures in terms of representational capacity, computational cost, and learning dynamics. In this way, the contributions of frequency-domain representations to ViTs can be analyzed from a clearer and more coherent perspective.

Fourier, Wavelet, and DCT-based methods offer distinct characteristics in terms of frequency resolution, spatial localization, and energy compaction. The integration of these transformations into ViT architectures significantly influences the model’s spectral sensitivity and learning behavior. However, even studies employing the same frequency transformation can exhibit markedly different spectral characteristics, depending on how the transformation interacts with the underlying architecture. In this regard, not only the type of frequency transformation but also how it is utilized within the ViT architecture becomes a determining factor. While some studies apply frequency transformations at the input stage as a preprocessing or alternative tokenization strategy, others integrate these representations directly into the attention mechanism or intermediate layers. These differing usage patterns emerge as a key factor distinguishing the spectral behaviors of methods belonging to the same transformation family.

Although Fourier, wavelet, and DCT-based FD-ViT architectures demonstrate promising performance across various tasks, their advantages are strongly task-dependent. Fourier-based models tend to improve global representation learning and computational scalability, but may lack local texture sensitivity without hybrid spatial modules. Wavelet-based approaches provide more stable performance in restoration and medical imaging tasks due to their multi-scale decomposition capability, yet often introduce additional architectural complexity. DCT-based models offer substantial efficiency gains through energy compaction and frequency pruning, but aggressive suppression of high-frequency components can negatively affect texture-sensitive tasks such as segmentation or fine-grained recognition. These observations suggest that frequency integration is not universally beneficial, and its effectiveness depends on how spectral representations interact with token-mixing mechanisms and task-specific frequency requirements.

Nevertheless, the primary focus of this section is to systematically examine the contributions of frequency transformations to ViT architectures based on the type of transformation employed. Accordingly, the FD-ViT literature is organized into three main categories: Fourier-based, Wavelet-based, and DCT-based approaches. In each subsection, the fundamental properties of the corresponding frequency transformation are summarized, and its interaction with ViT architectures is evaluated in terms of representational capacity, computational cost, and learning dynamics. In addition, recent approaches that render frequency transformations learnable are discussed within the scope of the corresponding transformation family. While such methods allow the spectral bias in the frequency domain to be shaped in a more flexible and adaptive manner, they also introduce new challenges, including optimization instability and overfitting.

3.1. Fourier-Based Vision Transformers

Fourier-based ViT (Fourier-ViT) architectures represent a significant area of research, aiming to compensate for the low-pass filter tendency of Transformer models by representing visual data in the spectral domain. Oversmoothing and rank collapse problems caused by standard self-attention mechanisms in deep layers limit the model’s capacity to represent high-frequency details. Fourier-based approaches overcome these spectral limitations, enabling the simultaneous and efficient modeling of both low-frequency structural information and high-frequency fine details [48].

One of the pioneering works in this field, Global Filter Network (GFNet) [45], redefined the concept of the global receptive field in the spectral plane by replacing the self-attention layer with a 2D discrete Fourier transform and learnable global filters. GFNet achieved competitive results on ImageNet with a lower number of parameters compared to pure CNNs and demonstrated that the Fourier transform could serve as an effective alternative for ViT-like global context modeling. This model provided a direct theoretical foundation for subsequent ViT-based spectral approaches such as SpectFormer, AFNO, and SVT.

Adaptive Fourier Neural Operator (AFNO) [48] was an operator-learning–based approach that modeled the token-mixing process as a continuous global convolution. AFNO adapted standard Fourier Neural Operators (FNOs) to visual datasets through techniques such as block-diagonal weighting structures and soft-threshold–based attenuation of frequency modes. This architecture reduced computational complexity O(N²) to $O\left(NlogN\right)$ and demonstrated that spectral operators could be directly employed as Transformer building blocks, particularly for high-resolution data, while preserving information density.

SpectFormer [10] introduced a hybrid architecture by sequentially combining Fourier-based spectral layers with multi-head attention blocks. In this model, spectral layers were positioned in the early stages to capture high-frequency features such as edges and line structures, while attention mechanisms placed in deeper layers were used to model token interactions and long-range dependencies. SpectFormer demonstrated improved generalization capability through spectral–attention interaction, reporting approximately a 2% accuracy improvement on ImageNet-1K compared to strong baselines such as DeiT and Less attention vIsion Transformer (LIT) [49].

In the field of medical imaging, Fourier ViT [50] developed a multi-branch architecture that integrated spatial- and frequency-domain features for histopathological image classification. Its FFT-based filtering mechanism was able to distinguish fine-grained details in low-contrast tissue regions with higher sensitivity compared to CNN-based models. Another notable study demonstrating the effectiveness of Fourier transforms in multi-modal systems was presented by Bin et al. [51]. This model employed a global attention mechanism in the Fourier domain to effectively fuse data from visible (VI) and infrared (IR) modalities, achieving significant performance gains over conventional methods in complex detection tasks such as ethanol leakage detection.

In image restoration and denoising tasks, Fourier-based approaches have stood out due to their computational efficiency and ability to disentangle signal components. FFTFormer [11] proposed a U-shaped CNN–Transformer hybrid architecture incorporating Fourier modules to remove noise in both the spatial and frequency domains. This model demonstrated that spectral awareness enhances noise robustness by achieving PSNR improvements of up to 2.12 dB, particularly in low-light image enhancement tasks. Similarly, the SFHformer [52] architecture, developed for image restoration, integrated an FFT mechanism directly into the token-mixing module, enabling efficient global modeling with $O\left(NlogN\right)$ complexity. SFHformer outperformed existing SOTA models in tasks such as deblurring and deraining, confirming that frequency-domain priors can effectively optimize restoration quality.

In ViT architectures, generalized spectral approaches that go beyond the classical Fourier transform demonstrated strong performance, particularly on complex data types. In this context, the Fractional Fourier Image Transformer (FrIT) [53] employed the Fractional Fourier Transform (FrFT) to perform feature extraction across multiple fractional domains for the classification of multimodal remote sensing data, including hyperspectral imagery (HSI) and LiDAR. Unlike conventional FFT-based methods, FrIT leveraged variable transform angles to establish a dynamic balance between the spatial and frequency domains, thereby enhancing cross-modality generalization capability.

The QHSA-ViT [54] architecture, which combines spectral awareness with the quantum computing paradigm, used a Quantum Discrete Fourier Transform (QDFT)-based frequency domain feature representation module. This model achieved a hierarchical self-attention fusion in tasks such as traffic sign recognition by taking advantage of quantum superposition and entanglement properties, and provided an average accuracy increase of 9.01% compared to base models.

Focusing on super-resolution problems in Earth System Models (ESMs), ViFOR [55] integrated a ViT architecture with Fourier-based Implicit Neural Representation (INR) networks. By replacing standard MLP layers with the FOREN structure incorporating Fourier-based activation functions, ViFOR achieved PSNR gains of up to 4.18 dB on surface temperature datasets, particularly in the reconstruction of high-frequency details.

Finally, the Fourier Convolution ViT (FCViT) [56], developed for medical image analysis, presented a three-branch architecture that combined convolutional information with fractal characteristics through the Fractional-Order Fourier Transform. FCViT reported strong generalization performance in glaucoma diagnosis from fundus images by jointly processing global features in the Fourier domain and local texture variations through convolutional layers.

When the studies in the literature are evaluated with a holistic approach, it has been determined that Fourier-ViT architectures offer three fundamental contributions to visual representation learning, which can be summarized as follows:

• Spectral Balance and Information Preservation: The low-pass filtering characteristic exhibited by standard Transformer models in deep layers, which leads to the oversmoothing problem, is mitigated through Fourier-based global filtering mechanisms. This approach enables high-frequency components, such as edge and texture details, to remain represented even in deeper layers.
• Computational Efficiency: The quadratic complexity (O(N²)) exhibited by self-attention mechanisms with respect to the number of patches is reduced to a log-linear complexity ($O\left(NlogN\right)$) through the use of FFT operators, providing a significant advantage in processing high-resolution data.
• Generalization and Robustness: Manipulating spectral bias in the Fourier domain and incorporating high-frequency components into the learning process enhance model robustness to out-of-distribution data and strengthen generalization capability.

Nevertheless, the inherent lack of local contextual inductive bias in Fourier representations, their susceptibility to noise infiltration, and the absence of convolutional priors remain active research challenges. To address these limitations, recent studies have combined Fourier transforms with multi-scale spectral analysis methods based on wavelets and scattering transforms. In particular, approaches such as the Dual-Tree Complex Wavelet Transform (DTCWT) provide higher invertibility and directional sensitivity, playing a critical role in preserving local details and enhancing noise robustness [57].

3.2. Wavelet-Based Vision Transformers

Wavelet-based ViT (Wavelet-ViT) architectures integrate the multi-scale nature of frequency-based visual representation with deep learning to overcome the spatial limitations and high computational costs of Transformer models’ self-attention mechanisms. While traditional Fourier-based models can only capture global frequency components, wavelet transforms have the ability to model both local texture details and global topological information at multiple scales. This structural advantage enhanced the high-frequency sensitivity of ViTs, thereby minimizing information loss, particularly around object boundaries and fine textures. One of the pioneering works in this area, Wave-ViT [12], employed an invertible DWT module to perform lossless subsampling on the keys and values, addressing the irreversible information loss caused by conventional patch tokenization. The methodological novelty of the model lay in allocating low-frequency components (LL) to structural information representation, while assigning high-frequency components (LH, HL, HH) to detail learning, and in strengthening local context through an expanded receptive field enabled by the inverse discrete wavelet transform (IDWT). Following a similar theoretical perspective, the Scattering Vision Transformer (SVT) [57] replaced the classical DWT with the DTCWT to achieve higher directional selectivity and invertibility, and introduced Einstein multiplication to reduce computational complexity in high-frequency gating operations.

In image restoration and super-resolution tasks, wavelet-based Transformers demonstrate remarkable success in balancing computational efficiency and detail preservation. The WaveletFormerNet [58] architecture introduced WaveletFormer and IWaveletFormer blocks to solve the problem of dense and non-homogeneous dehazing in the real world. This model captured multi-frequency information with a lightweight mechanism using parallel convolution within Transformer blocks and preserved image resolution through the feature aggregation module (FAM). SwinWave-SR [43] proposed for underwater image enhancement, integrated DWT and IDWT processes into the Swin Transformer architecture to achieve lossless subsampling. The model further employed the Lottery Ticket Hypothesis (LTH) to identify lightweight subnetworks that reduced training cost by 70.44%, thereby enabling practical deployment on resource-constrained underwater robotic systems. Developed with a focus on computational efficiency and image restoration, the Efficient Wavelet Transformer (EWT) [59] leveraged the invertible nature of the wavelet transform as a Frequency-domain Conversion Pipeline (FCP) to compress image resolution without losing critical features. This architecture was built upon a Dual-stream Feature Extraction Block (DFEB) that combined the local feature extraction capability of CNNs with the global modeling strength of Transformers, achieving an 80% increase in processing speed while reducing GPU memory usage by 60%. Designed for real-world denoising tasks, DnSwin [60] introduced low-frequency window-based self-attention (LF-WSA) and high-frequency shifted window-based self-attention (HF-SWSA) mechanisms through a Wavelet Sliding-Window Transformer (WSWT) architecture, enabling the modeling of inter-frequency dependencies via compact and coherent representations. Similarly, WIGformer [61], proposed for low-light image enhancement, employed a Wavelet Feature Enhancement Convolution (WFEConv) module that performed hierarchical wavelet decomposition, together with an Illumination-Guided Enhanced Multihead Self-Attention (IGEMSA) mechanism that leveraged illumination features as guidance. This design suppressed noise in dark regions while preserving texture integrity.

In biomedical image analysis, wavelet integration plays a critical role in capturing heterogeneous tissue features in the frequency domain. He et al. [62] incorporated the DWT directly into the network input and decomposed the image into four frequency bands using Haar filters. When combined with the Cross-Covariance Attention (XCA) mechanism, this approach enabled effective feature extraction from noisy ultrasound images and achieved a high AUC value of 0.984. On the other hand, to address the aliasing problem encountered in histopathological image classification, Ding et al. [63] integrated the wavelet transform into the system as a Wavelet Position Embedding (WPE) module prior to the self-attention mechanism. This architecture improved classification accuracy by smoothing discontinuous feature information through spectral analysis, while reducing computational cost through the use of external attention. In medical image segmentation analysis, WaveCoformer [64] presented a hybrid architecture that processed spatial and spectral features in parallel. The methodology of the model was based on a Spectral Feature Convolution (SpectraConv) module, which handled low-frequency structural information in the spectral domain using dilated convolutions, while processing high-frequency texture details with depthwise separable convolutions; these two representations were dynamically fused through a Cross-Context Attention (CCA) mechanism. DWNAT-Net [65], which achieved an accuracy of 99.66% on breast cancer histopathology images, enriched the Neighborhood Attention Transformer (NAT) architecture with DWT blocks, constraining self-attention computation to the local neighborhood of each token and thereby enabling more efficient capture of discriminative features in the frequency domain. WaveFormer [66], developed for hyperspectral image classification, addressed the aliasing phenomenon and information loss caused by conventional average pooling operations by employing DWT-based invertible subsampling during self-attention learning, thereby preserving the spectral integrity of the data.

In the domains of multi-scale object representation and multimodal fusion, WaveFusion [44] enabled more flexible feature extraction by employing adaptive wavelet filters learned from image content instead of fixed wavelet filters. The architecture optimized spectral-level alignment between infrared and visible images by utilizing self-attention for intra-modality consistency and a negative softmax–based cross-attention mechanism for inter-modality complementarity through a dual-aggregation attention fusion module (DAFM). SiWformer [67], which performed scale-invariant object classification, was able to effectively represent both small and large objects within a single architecture through an MSFE module that decomposed the image into multiple frequency bands. In remote sensing super-resolution tasks, the WTT architecture [68] positioned high-frequency components specifically as keys within the self-attention mechanism, thereby preserving low-frequency information while enabling more effective learning of high-frequency details such as edges and textures. Finally, the wavelet-based Image Tokenizer proposed by Google DeepMind [69] represented high-resolution images using block-sparse projections by replacing patch-wise convolutions with wavelet transforms, thereby making models more robust and faster while using parameter sets that were reduced by a factor of four.

3.3. DCT-Based Vision Transformers

DCT-based ViT architectures have recently emerged as a strong alternative to Fourier and Wavelet-based methods, attracting particular attention in terms of energy compaction, computational efficiency, and hardware compatibility. Unlike the Fourier transform, which works with complex numbers, DCT is a structure that does not include phase components and expresses the signal only in terms of real-valued cosine terms. This feature allows Transformer-based models to learn information representations in a more compact and noise-resistant way by concentrating a large portion of the signal energy in a small number of low-frequency coefficients. Furthermore, DCT-based approaches are computationally more efficient than Fourier transforms, especially in resource-constrained environments such as mobile and edge devices, because they do not contain imaginary parts. DCT-ViT-derived models developed in recent years have begun to utilize these advantages in the frequency domain more effectively to achieve high performance on low-quality or compressed images [70].

Among the pioneering works in this area, the Discrete Cosine TransFormer (DCFormer) model proposed by Li, Zhang, Yuan, Lu, and Zhu [38] successfully learned semantic representations directly from DCT-based frequency-domain representations. The authors demonstrated that the mathematical formulation of the inverse discrete cosine transform (IDCT) exhibits strong similarities to the self-attention mechanism, thereby showing that Transformer blocks can effectively simulate this process. Similarly, approaches such as DCT-Former [71] applied the DCT to the query, key, and value matrices within each attention block, performing attention computations in the frequency domain and reducing complexity from a quadratic to an approximately linear scale. By strategically discarding high-frequency components (e.g., noise) and concentrating energy in low-frequency bands, these models achieved up to a 25% reduction in memory usage compared to conventional ViT models, while attaining comparable accuracy levels with significantly lower computational costs (FLOPs).

In parallel with these developments, the DctViT [47] architecture introduced a novel perspective on spatial tokenization and resolution downsampling strategies used in conventional ViT models. Instead of convolutional layers, the model employed a dedicated block termed DCT-Attention Down-sample (DAD), which combined the Discrete Cosine Transform with channel-wise self-attention. This structure reorganized the image signal by decomposing it into frequency components and learned global dependencies through a Multi-head Channel-wise Self-attention (MHCSA) unit, while reducing computational cost. Experimental results on the ImageNet-1K dataset showed that the DctViT-L model achieved a Top-1 accuracy of 84.8%, outperforming models of similar complexity, such as CMT and Next-ViT. Furthermore, when used as a backbone for object detection with RetinaNet, DctViT produced a 2.5% higher mAP score compared to CMT-S.

DCT-based frequency analysis also played a critical role in eliminating redundant information from models. The DCT-ViT [46] study proposed a frequency pruning strategy that removed high-frequency “noise” components, motivated by the tendency of the human visual system to focus on low-frequency information. Inspired by the energy compaction principle used in JPEG compression standards, this approach encouraged ViT models to prioritize shape information over texture details. Experiments conducted on the DeiT-Small model showed that this method reduced computational cost (FLOPs) by 25% while achieving a 0.18% increase in accuracy. Similarly, on the DeiT-Tiny model, the approach reduced computational load by 47% without incurring any performance degradation, thereby demonstrating substantial efficiency gains.

Another significant contribution to the field was the Discrete Cosine Transform–Based Decorrelated Attention (DCTDA) model [72]. This model proposed a DCT-coefficient–based initialization strategy to address instabilities arising from the random initialization of self-attention weights in Transformer architectures. By initializing the weight matrices (Q, K, V) using DCT basis vectors with full-band coverage, this approach enabled the model to learn the entire frequency spectrum in a more balanced manner from the outset. In addition, by compressing the weight matrices in the DCT domain based on the principle that high-frequency components correspond to noise, the study successfully reduced the number of trainable parameters in the Swin-T model by 22.1% while preserving accuracy. Such decorrelated attention mechanisms reduced model sensitivity to noise, particularly in data-limited settings, and facilitated the learning of more generalizable visual representations.

Leveraging the spectral advantages of the DCT in multi-channel visual analysis tasks, DCTransformer [73] was developed for HSI classification by combining DCT-based spectral feature extraction with a channel attention mechanism (CAM). The architecture consisted of a spatial feature extractor (DFE) that learned detailed contextual information and a spectral feature extractor (BFE) that captured broad frequency information. By dynamically fusing low-frequency structural information and high-frequency texture components through a Dynamic Fusion Mechanism (DFM), the model achieved an average accuracy (AA) of 94.89% on the University of Houston (UH) dataset.

The use of DCT in image enhancement and domain-specific applications went beyond the limitations of standard methods. The DCT-driven Enhancement Transformer (DEFormer) [74], developed for low-light image restoration, incorporated a learnable frequency branch (LFB) that provided frequency information as an auxiliary cue to the network. The model’s curvature-based frequency enhancement (CFE) module adaptively focused on frequency bands containing rich textures, while the Cross-Domain Fusion (CDF) unit minimized discrepancies between the RGB and frequency domains, resulting in superior performance in terms of PSNR and SSIM metrics.

In the field of agricultural informatics, a novel Transformer architecture proposed by Ince et al. [75], which employed the Dynamic Range Enhanced Discrete Cosine Transform (DRE-DCT) method, produced groundbreaking results for the detection of bean leaf diseases. By incorporating frequency weighting into both the forward and inverse transform equations, this approach revealed high-frequency components and fine details that are imperceptible to the human eye. The authors matched the DCT block size (16 × 16) with the Transformer patch size and achieved a classification accuracy of 99.56% on the iBean dataset, representing an improvement of approximately 4% over models relying solely on Transformers. Such hybrid approaches combine the energy compaction capability of the DCT with the global context modeling strength of Transformers, delivering noise-robust and highly accurate solutions even in data-scarce scenarios.

From a self-supervised learning perspective, de Sousa Moraes, dos Santos, Busson, Duarte, and Colcher [70] adapted the SimCLR framework to work on quantized DCT coefficients derived from JPEG compression. The researchers demonstrated that DCT-based contrast learning can learn robust features even in low-quality or highly compressed images. In particular, it was observed that ViT-based embeddings, when combined with spatial or frequency domain inputs, mitigate compression-induced performance loss and improve the transferability of visual representations.

In the field of medical imaging, the high resolution of histopathological images posed a significant challenge for ViT models due to their quadratic computational complexity. To address this issue, the DCT-HistoTransformer [76] architecture integrated DCT-Attention and MobileConv units through parallel processing pathways. By filtering high-frequency components in the input image to reduce input dimensionality, the model achieved a high classification accuracy of 96.00% in breast cancer diagnosis with low computational cost.

Similarly, in data-scarce scenarios such as rare animal species classification, Kang and Zhang [77] proposed a hybrid architecture that combined an adaptive DCT preprocessing module with ViT-B16 and ResNet50 backbones. In this approach, an adaptive frequency partitioning mechanism that learned low-, mid-, and high-frequency boundaries through backpropagation was employed, enabling superior performance compared to standard CNNs and fixed-band DCT methods reported in the literature, even under extreme data scarcity.

Finally, in the digitization of cultural heritage documents, the automatic assessment of image quality played a critical role. In the study conducted by Sivan et al. [78], DCT coefficients were combined with a ViT architecture to classify degradation levels in palm leaf manuscripts. By integrating the ability of the DCT to efficiently represent degradation variations in the frequency domain with the capacity of ViTs to capture spatial context, the model achieved a test F1 score of 90%, outperforming purely CNN-based approaches.

All these studies confirm that DCT-based Transformer architectures not only provide computational efficiency but also offer a more robust structure compared to traditional spatial methods in learning visual semantics in noisy, compressed, or limited datasets. DCT’s exclusion of phase information and its operation with only real-valued coefficients significantly reduces the computational load compared to the Fourier transform using complex numbers. This feature enables real-time performance, especially in environments with limited hardware resources such as mobile and edge devices, because DCT minimizes data transmission overhead and memory usage by concentrating most of the signal energy in a small number of coefficients. However, the non-local nature of DCT (representing the entire block as a whole) can lead to loss of fine texture details and decreased spatial sensitivity, especially when high compression or pruning is applied. In the literature, it has been reported that focusing DCT coefficients on low frequencies is successful in recognizing object shapes, but the lack of high-frequency “texture” information can negatively affect performance in high-precision tasks such as instance segmentation [38]. Therefore, current research is moving towards multi-domain approaches where DCT is combined with Wavelet or Fourier transforms in a hybrid way.

4. Comparative Analysis and Benchmarking

FD-ViT architectures have surpassed the limitations of conventional models in terms of both performance and computational efficiency by processing visual data with spectral depth beyond spatial pixels. In this section, a comprehensive comparison of Fourier, Wavelet, and DCT-based methods with SOTA models in the literature is presented under criteria including accuracy, restoration quality, domain-specific performance, and hardware efficiency.

4.1. Overall Recognition Performance on ImageNet-1K

ImageNet-1K is not only a classification dataset for ViT-based architectures; it is also a de facto standard for testing generalization capabilities and architectural biases. The performance of FD-ViT models on ImageNet-1K is critical to understanding how frequency domain integration affects ‘shape bias’. The results summarized in

Table 1. Performance Comparison on ImageNet-1K Dataset.

Model	Method	Params (M)	FLOPs (G)	Top-1 Acc (%)	Key Innovation	Ref.
Swin-B	Spatial	88.0	15.4	83.5	Shifted Windows	[57]
SVT-H-L	Wavelet (DTCWT)	54.0	12.7	85.7	Invertible Scattering	[57]
SpectFormer-H-L	Fourier (FFT)	54.7	12.7	85.7	Initial Spectral Layers	[10]
Wave-ViT-L	Wavelet (DWT)	57.5	14.8	85.5	Lossless Down-sampling	[12]
DctViT-L	DCT	35.8	6.0	84.8	DAD and MHCSA Blocks	[47]
DCFormer-NA-B	DCT (Zigzag)	90.0	13.7	84.3	Semantics from Frequency	[38]
GFNet-H-B	Fourier (FFT)	54.0	8.6	82.9	2D Discrete Fourier	[45]
DCT-ViT-S	DCT (Pruning)	20.8	-	79.1	Token Pruning in Frequency	[46]
AFNO-ViT-S	Fourier (FNO)	16.0	15.3	80.9	Adaptive Operators	[48]

confirm that FD-ViT models provide consistent gains under certain conditions compared to classical ViT and CNN–ViT hybrids.

It is important to note that the results summarized in Table 1 are collected from independent studies conducted under different training protocols, including variations in training epochs, data augmentation strategies, distillation settings, model scaling, and optimization configurations. Therefore, the reported values should not be interpreted as strictly controlled benchmark comparisons, but rather as indicative performance trends across different FD-ViT design strategies. In several DeiT-based comparisons reported in the literature, FD-ViT architectures are evaluated under comparable training pipelines, providing more reliable evidence of performance improvements associated with frequency-domain integration.

In particular, Fourier-based architectures such as SpectFormer and GFNet derivatives have reported higher Top-1 accuracy values than DeiT variants with comparable parameter budgets in their respective experimental settings [45]. These results suggest that frequency-based representations can facilitate the learning of both low-frequency structural information and high-frequency details in a more decorrelated manner during training, potentially contributing to improved convergence behavior and data efficiency [72]. However, these gains are not uniform across studies and appear to depend strongly on how and where frequency information is integrated into the architecture (e.g., input stage, subsampling, or attention mechanism), rather than on the use of a specific transformation alone.

Fourier-based FD-ViT architectures have been reported to achieve relatively consistent performance improvements in several studies, particularly when spectral filtering modules are integrated into early or intermediate layers of the network [10]. One possible explanation discussed in the literature is that the low-pass filtering tendency observed in deep ViT representations—sometimes associated with attention collapse—may be partially mitigated by spectral operators that help preserve higher-frequency components during feature propagation [17]. As a result, edge and texture information relevant to object recognition may be retained more effectively in deeper layers. By contrast, approaches that apply frequency transformations only as a preprocessing step at the input stage, or without interaction with the attention mechanism, often report more modest gains on datasets such as ImageNet-1K. These observations suggest that the effectiveness of frequency-domain integration depends less on the transformation itself and more on how spectral representations interact with token-mixing and attention mechanisms within the architecture.

Wavelet-based ViT models, while relying on more complex mathematical structures than Fourier-based approaches, have been reported to exhibit relatively stable performance behavior in several ImageNet-1K configurations. Architectures such as Wave-ViT, particularly in small and medium-sized model variants, may contribute to more stable training dynamics by reducing information loss associated with traditional patch tokenization and pooling operations through reversible spectral transformations. However, these representational advantages often come with increased architectural complexity and additional computational overhead. In large-scale ImageNet settings, performance differences between wavelet-based and Fourier-based models (e.g., SpectFormer-H-L 85.7% vs. Wave-ViT-L 85.5%) appear relatively small, suggesting that the benefits of local–frequency decomposition do not always translate into clear accuracy gains in datasets dominated by global semantic structure. Instead, the strengths of wavelet-based representations tend to become more apparent in tasks where locality and texture preservation are critical, such as image restoration and enhancement.

DCT-based ViT approaches generally occupy an efficiency-oriented position in ImageNet-1K evaluations, distinguishing themselves from many Fourier- and wavelet-based methods in terms of computational cost. Rather than targeting absolute peak accuracy, these models are typically designed to reduce computational complexity while maintaining comparable accuracy levels or achieving modest improvements. DCT-based attention and token-downsampling mechanisms often discard portions of high-frequency information based on energy-compaction principles inspired by classical signal processing and human visual perception, leading to reductions in FLOPs and memory usage with limited impact on Top-1 accuracy. For example, the DCT-ViT model reported a 25% reduction in computational cost in a DeiT-Small–based configuration while achieving a small accuracy gain of 0.18% [46]. Such results suggest that DCT-based architectures can serve as efficiency-oriented alternatives within the FD-ViT design space, particularly in scenarios where computational resources are constrained, such as edge or mobile deployment settings.

When the overall results are considered from a broader perspective, the empirical findings suggest that performance gains observed on ImageNet-1K are influenced more by the level of architectural integration than by the specific type of frequency transformation employed (Fourier, Wavelet, or DCT). FD-ViT models that incorporate frequency-domain information within attention mechanisms or hybrid token-mixing strategies (e.g., SpectFormer or SVT) have reported more consistent improvements than approaches that use spectral representations only as a preprocessing step at the input stage [10]. These observations indicate that the effectiveness of frequency-domain integration depends less on the presence of spectral information alone and more on how it interacts with token-mixing processes within Transformer architectures. In this regard, spectral decomposition can help separate global structural components from local high-frequency details, potentially supporting the learning of more discriminative feature representations in certain settings.

Empirical findings on ImageNet-1K suggest that FD-ViT approaches can provide measurable performance improvements under certain conditions, while also highlighting scenarios in which their benefits become less pronounced. Several studies indicate that classical ViT architectures may exhibit reduced sensitivity to high-frequency visual components—particularly edge and fine-texture information—as representations become progressively smoother in deeper layers. Frequency-domain integrations are therefore often introduced to improve the spectral stability of ViT representations by mitigating this tendency. However, in large-scale training settings that involve extensive datasets and strong data-augmentation strategies such as RandAugment, spatial-domain ViT models can implicitly learn frequency-related characteristics during optimization. Under such conditions, the additional advantage provided by explicit frequency-domain integration may become more limited or configuration-dependent.

Therefore, the performance gains observed on ImageNet-1K should be interpreted not as evidence of universal superiority of FD-ViT approaches, but rather as architectural improvements that target spectral issues associated with self-attention representations in deeper Transformer layers. Experimental findings across the literature suggest that FD-ViT models typically report Top-1 accuracy gains in the range of approximately 0.5% to 2.0%, although these improvements depend strongly on model scale, integration level, and training configuration. From this perspective, the contribution of frequency-domain integration appears to lie less in absolute accuracy gains and more in improving the stability of token-mixing processes within Transformer blocks and promoting more spectrally balanced representation learning. Several studies further suggest that such spectral regularization may help mitigate phenomena such as over-smoothing and rank collapse during training [16].

4.2. Image Restoration and Enhancement Performance

Image restoration and enhancement tasks constitute the problem class where the unique advantages of FD-ViT approaches are most clearly demonstrated. Tasks such as denoising, deblurring, low-light enhancement, dehazing, and super-resolution are inherently critically dependent on high-frequency components such as object edges, textures, and fine structural details. The literature has shown that the self-attention mechanism in classical ViT architectures behaves like a natural low-pass filter as depth increases, leading to over-smoothing and loss of high-frequency detail [16]. The experimental results presented in

Table 2. Comparison of Image Restoration Tasks (PSNR in dB).

Model	Task	Dataset	PSNR (dB)	SSIM	Specific Gain	Ref.
SFHformer	Dehazing	SOTS-Indoor	43.03	0.9966	30% FLOPs vs. TaylorFormer	[52]
SFHformer	Deraining	Rain200L	41.85	0.9908	20% FLOPs vs. DRSformer	[52]
SwinWave-SR	Super-Res.	USR-248	32.46	0.9252	Invertible Down-sampling	[43]
WIGformer	Low-light	LOL-V1	26.12	0.871	Illumination Guidance	[61]
FFTFormer	Low-light	FiveK	24.64	0.876	Noise-aware Fusion	[11]
DEFormer	Low-light	LOL	23.73	0.821	Learnable Frequency Branch	[74]
EWT	Denoising	Kodak24	31.73	-	80% Faster Inference	[59]

confirm that FD-ViT models systematically overcome this spectral limitation by decomposing the noise in the frequency domain.

Fourier-based restoration models (e.g., SFHformer, FFTFormer, and DEFormer) have demonstrated consistent gains, particularly in removing complex noise patterns and under low-light conditions. The primary advantage of these models lies in their ability to efficiently model global frequency components and to separate noise from structural information through frequency-based “signatures.” Fast Fourier Transform (FFT)–based modules reduce the quadratic complexity of self-attention to $O\left(NlogN\right)$ while selectively reweighting high-frequency signals using learnable spectral filters rather than suppressing them outright. As a result of this strategy, models such as FFTFormer were reported to improve PSNR by 2.12 dB on the MIT-Adobe FiveK dataset, while SFHformer achieved a high PSNR of 43.03 dB in dehazing tasks [11,52]. In the restoration literature, gains of this magnitude are regarded not as marginal improvements but as structural achievements at the architectural level.

The comparative results presented in Table 2 are compiled from multiple studies with task-specific training configurations and evaluation protocols. As a result, the reported values should be interpreted as qualitative indicators of architectural tendencies rather than direct performance rankings. In particular, differences in dataset characteristics, optimization strategies, and model scaling may influence the reported results. The table is therefore intended to highlight application-level trends in FD-ViT design rather than to provide strictly controlled experimental comparisons.

In contrast, wavelet-based FD-ViT approaches exhibit a more balanced and detail-preserving performance than Fourier-based models in scenarios that require local texture sensitivity and time–frequency localization. Architectures such as SwinWave-SR, EWT, and Wave-ViT, in particular, completely prevent high-frequency information loss during downsampling by employing invertible wavelet transforms. This technical advantage is reflected not so much in absolute PSNR values, but rather in the relative gains achieved by wavelet-based models over their task-specific baseline methods within their respective domains of expertise.

DCT-based approaches play a more limited yet strategic role in the restoration context. Rather than aiming to maximize absolute restoration quality, these models are typically designed to balance computational efficiency with acceptable visual quality. DCT-based attention or frequency pruning mechanisms suppress high-frequency noise components at early stages, enabling competitive PSNR/SSIM performance with significantly lighter models. This characteristic makes DCT-based ViT models a practical alternative, particularly for real-time or resource-constrained restoration scenarios [46].

Overall, image restoration and enhancement tasks constitute the strongest and most compelling application domain for FD-ViT approaches. Unlike the gains observed on classification-oriented datasets such as ImageNet-1K, frequency-based integrations in restoration tasks deliver meaningful and consistent improvements in both quantitative metrics and visual quality. The literature confirms that degradations such as low illumination, haze, or noise admit much more compact and disentangleable representations in the frequency domain. This indicates that frequency-domain information is not merely an auxiliary architectural feature, but a fundamental representational requirement for this class of problems. Accordingly, FD-ViT approaches should be positioned not as simple architectural variations, but as a more suitable solution paradigm that directly targets structural deficiencies of conventional ViT and CNN-based methods, such as low-pass filtering behavior and the loss of high-frequency details [52].

4.3. Field-Specific Application Analysis

In specialized fields requiring high accuracy and noise resistance, such as medical imaging, precision agriculture, and remote sensing, FD-ViT models surpass traditional spatial methods. This success is based on the ability of frequency information to sharpen object edges, better define fractal structures (vascular networks, etc.), and separate spectral signatures from noise.

Table 3. Domain-Specific Application Benchmarking.

Model	Domain	Dataset	Metric	Result	Ref.
DRE-DCT	Agriculture	iBean	Accuracy	99.56%	[75]
DWNAT-Net	Medical (Hist.)	BreakHis	Accuracy	99.66%	[65]
WaveFormer	Remote Sensing	UH (HSI)	OA (%)	96.54%	[66]
FCViT	Medical (Eye)	Fundus	Accuracy	93.52%	[56]
DCT-Histo	Medical (Canc.)	BreaKHis	Accuracy	96.00%	[76]
FrIT	Remote Sensing	Houston	OA (%)	92.60%	[53]
SiWformer	Scene Recognition	VOC 2012	Accuracy	89.90%	[67]

presents a comparative analysis of the performance of FD-ViT architectures in these specific fields.

The empirical results presented in Table 3 clearly demonstrate how frequency-domain information enhances object recognition sharpness. In particular, the DRE-DCT model developed for agricultural applications achieved a remarkable accuracy of 99.56% in leaf disease classification by exactly matching the DCT block size with the ViT patch size (16 × 16). In the medical imaging domain, DWNAT-Net combined the Neighborhood Attention (NA) mechanism with the DWT and achieved a near-perfect performance of 99.66% accuracy in cancer diagnosis.

In the medical imaging domain, FD-ViT models play a critical role in overcoming challenges such as aliasing and data scarcity that are commonly encountered by classical ViT architectures, particularly in histopathological and ultrasound-based diagnosis. For example, the DWNAT-Net architecture integrated the NA mechanism with the DWT, achieving a near-perfect accuracy of 99.66% on breast cancer histopathology images. Lightweight models such as DCT-HistoTransformer, on the other hand, reduce computational cost by strategically filtering high-frequency components and are able to reach State-of-the-Art performance even with limited datasets. In ophthalmic disease diagnosis, the FCViT model analyzed fractal retinal vascular networks using the Fractional Fourier Transform and achieved an accuracy of 93.52% in glaucoma detection. Such models particularly benefit from the high-frequency sensitivity provided by Fourier and Wavelet transforms, which is essential for accurately delineating tissue boundaries and lesion edges.

In remote sensing and hyperspectral imaging tasks, FD-ViT models stand out due to their ability to extract multi-scale spatial–spectral features. The WaveFormer architecture established a new benchmark by extracting wavelet-based multi-scale features from hyperspectral data cubes and achieving an accuracy of 96.54% on the University of Houston dataset. The FrIT model, which employed the Fractional Fourier Transform, processed LiDAR and hyperspectral data across multiple fractional domains and outperformed ViT-based baselines by approximately 5% in the classification of complex terrain structures.

SiWformer, developed for scene recognition tasks, learns a scale-invariant representation by separating images into different frequency bands and systematically improves object classification accuracy in the PASCAL VOC 2012 dataset. This hybridization enables meaningful alignment of heterogeneous data from different sensors and prevents information loss.

When examining the common denominator of success specific to these fields, it is seen that FD-ViT approaches provide a significant advantage, especially in scenarios where high-frequency structural information directly influences the classification decision. In medical imaging, precision agriculture, and remote sensing applications, decision limits are often defined not by color or spherical semantic cues but by subtle edge discontinuities, texture density variations, and fractal-like geometric patterns. While the low-pass spectral tendency of classical ViT architectures leads to the attenuation of such details in deep layers, frequency domain integration clearly preserves this information, increasing the model’s discriminative power. Furthermore, these application areas are often characterized by limited and unstable data regimes. In this regard, frequency-based transformations not only provide an additional representation channel to the model but also act as an implicit regularizer. Through spectral decomposition, noise-sensitive high-frequency components are suppressed in a controlled manner, while structurally meaningful frequency bands are emphasized. This mechanism explains why FD-ViT models exhibit stable learning even on small datasets and offer higher generalization reliability compared to classical ViT architectures.

4.4. Overall Comparison Matrix of FD-ViT Architectures (All Models)

Prominent modern FD-ViT models reported in the literature are consolidated in

Table 4. A comprehensive comparison of FD-ViT models.

Model	Method	Domain	Primary Result	Params (M)	FLOPs (G)
SVT-H-L [57]	Wavelet	Image Recognition	85.7% (Top-1)	54.0	12.7
SpectFormer-H-L [10]	Fourier	Image Recognition	85.7% (Top-1)	54.7	12.7
Wave-ViT-L [12]	Wavelet	Image Recognition	85.5% (Top-1)	57.5	14.8
DctViT-L [47]	DCT	Image Recognition	84.8% (Top-1)	35.8	6.0
DCFormer-NA-B [38]	DCT	Image Recognition	84.3% (Top-1)	90.0	13.7
GFNet-H-B [45]	Fourier	Image Recognition	82.9% (Top-1)	54.0	8.6
DCT-ViT-S [46]	DCT	Image Recognition	79.1% (Top-1)	20.8	-
DCTDA [72]	DCT	Image Recognition	81.47% (Top-1)	25.4	4.18
AFNO-ViT-S [48]	Fourier	Image Recognition	80.9% (Top-1)	16.0	15.3
SFHformer [52]	Fourier	Restoration	43.03 (PSNR)	3.87	26.59
FFTFormer [11]	Fourier	Low-light	24.64 (PSNR)	-	-
WIGformer [61]	Wavelet	Low-light	26.12 (PSNR)	1.96	11.73
WaveletFormerNet [58]	Wavelet	Dehazing	21.68 (PSNR)	2.26	4.08
EWT [59]	Wavelet	Denoising	31.73 (PSNR)	5.18	-
SwinWave-SR [43]	Wavelet	Underwater SR	32.46 (PSNR)	3.28	135.0
DWNAT-Net [65]	Wavelet	Medical (Hist.)	99.66% (Acc)	20.0	2.7
DRE-DCT [75]	DCT	Agriculture	99.56% (Acc)	-	-
WaveFormer [66]	Wavelet	Hyperspectral	96.54% (OA)	-	-
DCT-Histo [76]	DCT	Medical (Canc.)	96.00% (Acc)	-	-
FCViT [56]	FrFT	Medical (Eye)	93.52% (Acc)	117.5	-
QHSA-ViT [54]	Quantum DFT	Traffic Sign	97.15% (Acc)	-	-
FrIT [53]	Fractional Fourier	Remote Sensing	92.60% (OA)	-	-
Fourier-ViT-B [50],	Fourier	Medical (Hist.)	87.06% (AUC)	111.7	22.5
DCT-Former (Vanilla) [71]	DCT	Memory Efficiency	59.7% (Acc)	-	-
DnSwin [60]	Wavelet	Real Denoising	40.29(PSNR)	23.24	-

with respect to methodology, application domain, performance outcomes, parameter counts, and FLOPs. As discussed in the previous sections across different tasks and application domains, FD-ViT approaches exhibit substantial architectural diversity. This diversity necessitates comparisons under common criteria that go beyond isolated performance tables. To this end, Table 4 brings together all FD-ViT architectures considered in this study according to the employed frequency transformation, level of frequency integration, computational complexity, and target task types. Rather than ranking models by absolute performance values, the presented comparison aims to summarize the general trends observed in the literature.

Across the table, it can be observed that the types of frequency transformations employed are associated with distinct representational characteristics across models. Fourier-based approaches are predominantly represented by architectures that process global frequency components and offer relatively low computational complexity, whereas wavelet-based models form a group dominated by structures relying on multi-scale frequency decomposition. DCT-based approaches, in contrast, are positioned as more compact and computationally efficient models by concentrating energy in low-frequency components. This distribution indicates that the choice of transformation type is closely coupled with architectural design preferences.

Another prominent observation highlighted in Table 4 is that model behavior varies sharply depending on the level at which frequency information is integrated into the architecture. Architectures that employ frequency representations solely at the input level as a preprocessing step or within a simple tokenization stage (e.g., DCT-Histo or basic ViT–DCT combinations) are typically specialized for specific tasks and exhibit a limited scope of effectiveness. In contrast, models that integrate frequency information directly into self-attention computations (e.g., WTT, SVT) or into token-mixing operations within intermediate layers (e.g., SFHformer, FFTFormer) report superior performance across a much broader range of tasks, from image restoration to medical segmentation. This observation indicates that the primary driving force behind the architectural diversity observed in the literature lies in how deeply spectral information is embedded into the dynamic learning process of Transformers.

Table 4 further reveals that FD-ViT architectures do not exhibit a homogeneous structure in terms of computational complexity; rather, they span a strategic spectrum. Some models (e.g., AFNO, DCT-Former, FNet [79]) aim for high efficiency by reducing the quadratic complexity of classical self-attention mechanisms (O(N²)) to quasi−linear levels $\left(O\right(NlogN\left)\right)$ through Fourier and DCT-based token-mixing mechanisms. On the other hand, certain models (e.g., SVT or WaveFormer) employ more complex frequency decomposition techniques (such as DTCWT), which moderately increase computational cost but, in return, preserve fine-grained texture and edge details, achieving SOTA accuracy. These differences demonstrate that FD-ViT approaches can successfully accommodate both efficiency-oriented designs for edge devices and high accuracy-oriented designs for precision critical analyses.

Table 4 indicates that there is no single dominant architectural paradigm in the FD-ViT literature; instead, models adopt distinct design choices according to the type of frequency transformation employed (Fourier, Wavelet, or DCT), the depth of integration, and the target tasks. For instance, DCT-based models stand out in efficiency through energy compaction and pruning, Fourier-based models specialize in global context modeling, and Wavelet-based models excel in lossless downsampling and local detail preservation. This comparative summary presents the architectural richness of the literature within a systematic framework and establishes a common reference point for the more detailed discussions of spectral capabilities addressed in the following section.

From a design perspective, the empirical findings across FD-ViT architectures suggest that the effectiveness of spectral integration depends strongly on task characteristics and computational constraints. Fourier-based architectures tend to provide consistent benefits in tasks requiring global context modeling and frequency-aware noise separation, such as image restoration, denoising, and multimodal fusion. However, in large-scale classification settings where strong data augmentation and large datasets are available, their advantage over spatial ViTs becomes less pronounced, as classical ViT models can implicitly learn frequency characteristics during training.

Wavelet-based FD-ViT models offer stable performance in texture-sensitive and multi-scale tasks due to their invertible decomposition and local–frequency representation capability. Nevertheless, their practical deployment may be limited by increased architectural complexity, additional transform operations, and memory overhead associated with multi-branch spectral processing, which can affect scalability in large models.

DCT-based approaches primarily provide efficiency-oriented improvements through energy compaction and frequency pruning mechanisms. These models are particularly advantageous in resource-constrained environments and edge deployment scenarios. However, aggressive suppression of high-frequency components may reduce sensitivity to fine texture information, which can negatively affect tasks such as segmentation, fine-grained recognition, or instance-level analysis. This highlights a fundamental trade-off between computational efficiency and representational richness in DCT-based FD-ViT architectures.

Overall, empirical results indicate that FD-ViT improvements are typically modest and configuration-dependent rather than universally consistent. In large-scale classification settings, aggressive data augmentation and large training datasets reduce the relative advantage of frequency-domain integration. In contrast, tasks that are inherently sensitive to high-frequency components—such as restoration, denoising, and medical imaging—benefit more consistently from spectral modeling. This suggests that FD-ViT architectures should be viewed as task-adaptive architectural extensions rather than general replacements for spatial-domain ViT.

5. Multi-Dimensional Taxonomy of FD-ViTs

Although the FD-ViT literature is commonly categorized according to the type of spectral transformation employed (Fourier, Wavelet, or DCT), the architectural diversity of existing models extends beyond this single-dimensional classification. A closer examination of prior studies shows that the impact of frequency-domain information on model performance varies substantially depending on where spectral representations are integrated within the network, whether the spectral operators are fixed or learnable, and whether the design primarily targets global or local frequency modeling. For this reason, the FD-ViT literature can be more effectively characterized by introducing three additional structural dimensions that complement the transformation-based taxonomy.

5.1. Integration Level

The stage at which frequency-domain information is incorporated into the architecture is one of the most critical factors influencing both computational efficiency and representational capacity.

• Input/Tokenization Level: In this approach, spectral transformations are applied before the data enters Transformer blocks, typically as a preprocessing or embedding mechanism. The goal is to reduce spatial redundancy or introduce an early spectral inductive bias into the model. For example, the Wavelet-Based Image Tokenizer generates sparse representations by tokenizing images as wavelet coefficients instead of pixel patches. Similarly, DCT-HistoTransformer applies a DCT transformation at the input stage to suppress high-frequency components and reduce computational cost.
• Token Mixing/Attention Replacement: Models in this category replace the standard self-attention mechanism to overcome its quadratic complexity (O(N²)) by using spectral operators for token interaction. For instance, GFNet replaces self-attention with FFT-based global filtering using learnable spectral filters, reducing complexity to approximately $O\left(NlogN\right)$. AFNO and FNet follow a similar strategy by performing token mixing directly in the frequency domain.
• Hybrid/Attention Modulation: In hybrid approaches, frequency-domain information is used to enrich or guide the attention mechanism rather than replace it. SpectFormer captures both periodic and local features by combining spectral blocks with attention blocks in a sequential manner. Wave-ViT and WTT incorporate wavelet transformations into key–value projections, encouraging the attention mechanism to better preserve and utilize high-frequency details.

5.2. Nature of Spectral Operator

Spectral filtering strategies in FD-ViT architectures can be broadly categorized according to their level of adaptability in the frequency domain.

• Fixed Operators: In this category, transformation coefficients or filtering rules are predefined and remain unchanged during training. For example, FNet employs a parameter-free Fourier transform, where learning occurs only in the subsequent MLP layers. This approach offers high computational efficiency, but its ability to adapt to dataset-specific frequency characteristics is inherently limited.
• Learnable/Adaptive Operators: In contrast, learnable spectral operators optimize filter weights (e.g., amplitude and phase) or transformation parameters during training. SpectFormer uses a Spectral Gating Network to learn which frequency components should be preserved or suppressed. SFHformer employs frequency dynamic convolution (FDC) to adapt spectral filters according to input content. Similarly, FrIT introduces a learnable parameter for the fractional Fourier transform (FrFT) angle (α), allowing the model to identify an optimal spectral representation plane during training.

5.3. Modeling Scope

The mathematical properties of the spectral transformation used in FD-ViT architectures influence the effective receptive-field characteristics of the model.

• Global Context: Fourier- and DCT-based models (e.g., GFNet and DCFormer) naturally exhibit a global receptive field because they process the entire image through a unified spectral representation. This property is effective for capturing long-range dependencies, although it may be less sensitive to localized irregularities or fine-grained anomalies.
• Multi-scale/Local Context: Wavelet-based models (e.g., SVT and WaveFormer) leverage time–frequency localization to model both local texture details and broader structural information in a multi-scale manner. This capability allows them to preserve fine spatial information while simultaneously capturing hierarchical contextual features.

Table 5. Multi-dimensional Taxonomy of FD-ViTs.

Model	Transform Type	Integration Level	Operator Nature	Modeling Scope	Key Mechanism
SpectFormer	Fourier	Hybrid (Sequential Blocks)	Learnable (Gating)	Global	Spectral Gating Network
GFNet	Fourier	Token Mixer Replacement	Learnable (Global Filter)	Global	Learnable Freq. Filters
AFNO	Fourier	Token Mixer Replacement	Learnable (Adaptive)	Global	Adaptive Operator Mixing
SFHformer	Fourier	Hybrid (In-Mixer)	Learnable (Dynamic)	Global	Freq. Dynamic Conv.
FFTFormer	Fourier	Hybrid (Encoder–Decoder)	Learnable	Global	Spatial-Freq. Fusion
ViFOR	Fourier	Activation/MLP	Learnable	Global	Fourier-based INR
QHSA-ViT	Quantum Fourier	Hybrid (Attention Fusion)	Fixed (QDFT)	Global	Quantum DFT + Attention
FrIT	Fractional Fourier	Token Mixer Replacement	Learnable (α angle)	Global/Flexible	Multi-domain Feature Ext.
FCViT	Fractional Fourier	Hybrid (Parallel Branch)	Learnable	Global/Fractal	Three-branch Architecture
Wave-ViT	Wavelet	Attention Modulation	Fixed (DWT)	Multi-scale	Invertible Down-sampling
SVT	Wavelet	Token Mixer Replacement	Learnable (Einstein)	Multi-scale	Scattering + Einstein Mul.
WTT	Wavelet	Attention (Key/Value)	Fixed (DWT)	Multi-scale	Freq-guided Attention
WaveFormer	Wavelet	Token Mixer Embedding	Fixed (DWT)	Multi-scale	Embedded Wavelet Trans.
SwinWave-SR	Wavelet	Hybrid (Within Block)	Fixed (DWT)	Multi-scale	Lossless Down-sampling
WIGformer	Wavelet	Hybrid (Enhancement)	Learnable	Multi-scale	Illumination Guidance
DCFormer	DCT	Token Mixer Replacement	Learnable	Global	DCT-based Semantic Learning
DctViT	DCT	Hybrid (Down-sampling)	Learnable	Global	DCT-Attention Downsample
DCT-Histo	DCT	Input/Preprocessing	Fixed (Pruning)	Global	Low-pass Filtering
DCT-ViT	DCT	Token Pruning	Fixed (Zigzag)	Global	High-Freq. Token Pruning
DCTDA	DCT	Attention Initialization	Fixed (Basis)	Global	Decorrelated Attention Init.

classifies the major FD-ViT architectures reviewed in this survey according to the proposed multi-dimensional taxonomy. Rather than grouping models solely by transformation type, the matrix highlights how FD-ViT designs diverge in terms of architectural integration strategy, spectral operator adaptability, and modeling scope. This perspective emphasizes the design philosophy behind each architecture, providing a more structured view of the FD-ViT design space. As shown in Table 5, a large portion of the literature focuses on introducing innovations at the token-mixing stage to reduce the computational burden of Transformer architectures. In particular, several Fourier- and DCT-based models replace or approximate self-attention using spectral token-mixing operators, often targeting global context modeling and computational efficiency. In contrast, wavelet-based architectures typically exploit multi-scale spectral decomposition to preserve fine spatial details, making them particularly suitable for tasks that require texture fidelity and structural reconstruction, such as image restoration and enhancement. The taxonomy also reveals a broader trend toward learnable spectral operators in more recent FD-ViT designs. Architectures such as SpectFormer and SVT employ adaptive spectral filtering mechanisms that allow models to capture dataset-specific frequency characteristics, moving beyond fixed transformations toward data-driven spectral representation learning. This shift reflects an increasing emphasis on integrating spectral representations more tightly with Transformer token-mixing dynamics.

6. Challenges and Open Problems

Although FD-ViT architectures have achieved notable success across various tasks and domains by approaching visual data from a spectral perspective, the current literature indicates that neither the theoretical limits nor the practical applicability of these approaches has yet been fully clarified. While the reviewed studies reveal the potential of frequency-domain integration, they also point to a set of unresolved fundamental challenges related to architectural design, generalization behavior, computational efficiency, and evaluation protocols. One of the key challenges in the FD-ViT literature lies in the insufficient disentanglement of which components of frequency representations directly contribute to task performance. Many studies treat frequency-domain integration as a holistic improvement, without systematically separating the roles of magnitude and phase information. In particular, the partial or complete neglect of phase information in Fourier- and DCT-based approaches can lead to inadequate representation of high-frequency structural details in certain tasks. This highlights the need for a more fine-grained and controlled analysis of the informational content carried by frequency representations. Another major open problem concerns spectral information loss and aliasing effects arising from the interaction between frequency transformations and patch-based tokenization or downsampling processes. When standard pooling or strided convolution operations are applied in a manner inconsistent with sampling theory, high-frequency components may be irreversibly distorted. Although wavelet-based invertible transformations aim to address this issue, their repeated use across deep architectures can introduce new challenges, such as channel proliferation, numerical instability, or hardware-dependent degradation. Moreover, a unified evaluation framework for measuring and reporting aliasing effects in a comparative manner has yet to be established [63].

The spectral behavior exhibited by FD-ViT models as network depth increases has also emerged as an important research direction. It is well known that ViT architectures tend to exhibit a low-pass filtering characteristic across successive self-attention layers, progressively attenuating high-frequency signals and giving rise to phenomena such as patch uniformity and rank collapse. Although existing FD-ViT approaches attempt to mitigate this effect through spectral scaling or hybrid architectures, a parameter-free, task-agnostic spectral balancing mechanism that can effectively prevent oversmoothing in deep networks has yet to be established [16]. From a practical deployment perspective, it is observed that although frequency transformations have low theoretical computational complexity, this advantage does not always translate into real-world inference speed. While modern GPU architectures and acceleration libraries are highly optimized for spatial convolutions and dense matrix operations, an equally mature hardware and software ecosystem for frequency-domain operations such as FFT, DCT, and DWT has not yet emerged. As a result, some FD-ViT architectures that appear lightweight in theory may exhibit higher-than-expected latency or energy consumption when executed on standard hardware platforms [46].

Although several FD-ViT architectures report reduced theoretical complexity (e.g., replacing O(N²) attention with $O(N log N)$ spectral token mixing), these reductions do not always translate into proportional inference speed improvements in practice. Modern GPU and accelerator architectures are highly optimized for dense matrix multiplications and convolution-like operations, whereas FFT and wavelet-based operators may introduce additional memory-access overhead and kernel-launch costs. As a result, theoretical FLOPs reduction and actual runtime latency may diverge depending on implementation details, hardware backend, and batch size. This gap highlights the importance of hardware-aware spectral operator design and benchmarking FD-ViT architectures using runtime metrics such as throughput and latency in addition to FLOPs.

Data efficiency and domain adaptation also remain unresolved issues in the FD-ViT literature. The lack of local inductive biases in Transformer-based models can limit generalization performance, particularly on small and imbalanced datasets. Although frequency-domain integrations partially alleviate this issue, achieving stable and consistent performance in low-data regimes remains a challenging problem. Moreover, in many studies, the boundaries of frequency bands (low, mid, high) are determined heuristically, which prevents frequency decomposition from adapting effectively to the underlying data distribution.

The behavior of FD-ViT approaches in multimodal and cross-domain scenarios has not yet been sufficiently understood. In hybrid models that combine spatial and frequency representations, there remains a theoretical gap regarding how these heterogeneous information sources should be effectively aligned and how they should interact at the semantic level. While existing studies largely rely on simple fusion strategies, the lack of dynamic and learnable frequency operators constitutes a significant open problem for future research in this area [44].

Beyond identifying these limitations, several research directions emerge for improving FD-ViT architectures. For example, the challenge of phase information loss suggests the need for architectures capable of jointly modeling amplitude and phase components, potentially through complex-valued attention mechanisms or phase-aware spectral tokenization strategies. Similarly, spectral aliasing remains an underexplored issue in FD-ViT design. Future work may investigate adaptive sampling strategies, learnable frequency-domain downsampling operators, or hybrid spatial–spectral reconstruction losses to reduce aliasing artifacts during tokenization and feature compression.

While many FD-ViT architectures primarily operate on magnitude representations or implicitly discard phase information during spectral filtering, several recent approaches attempt to preserve or utilize phase-aware representations. For example, architectures employing learnable spectral filters (e.g., SpectFormer and AFNO-style operators) implicitly model both amplitude and phase components through complex-valued frequency-domain transformations. Similarly, fractional Fourier–based models such as FrIT introduce learnable spectral parameters that allow the model to operate in intermediate frequency domains where both magnitude and phase relationships can be preserved more effectively.

In addition, hybrid spectral–spatial architectures that combine spectral operators with attention mechanisms partially mitigate phase-information loss by reintroducing spatial alignment through token-mixing layers. These approaches suggest that the magnitude–phase trade-off in FD-ViT design is often addressed indirectly through adaptive spectral filtering and hybrid architectural integration rather than through explicit phase modeling. Nevertheless, explicit phase-aware Transformer architectures remain relatively underexplored, representing an important future research direction for FD-ViTs.

From a systems perspective, efficiency and deployment constraints highlight the importance of hardware-aware spectral operators and lightweight frequency-domain modules. Research on structured spectral pruning, low-rank spectral filtering, and accelerator-friendly FFT approximations may help bridge the gap between theoretical efficiency gains and real-time inference performance.

Finally, the lack of standardized evaluation protocols for FD-ViT architectures remains a key challenge. Differences in training pipelines, dataset scales, and augmentation strategies complicate cross-paper comparison. Establishing task-specific evaluation benchmarks for frequency-aware Transformer models could significantly improve reproducibility and comparability in the field.

7. Emerging Trends and Future Directions

In recent years, the FD-ViT literature has undergone a significant transformation aimed at overcoming representational and scalability limitations encountered in the spatial domain. An examination of existing studies indicates that FD-ViT approaches are increasingly positioned not merely as variations of existing architectures, but as components of more general and flexible visual representation paradigms. In this context, a number of prominent research trends are emerging that are shaping the future of the field.

The first major trend is the stronger integration of FD-ViT architectures with multimodal and volumetric data. Recent works demonstrate that spectral representations are no longer confined to 2D visual inputs, but can be effectively aligned with text, infrared, LiDAR, and 3D volumetric data. Particularly in medical imaging, processing 3D CT or MRI volumes in the frequency domain not only reduces computational cost but also enables more consistent representation learning when combined with textual reports. This trend opens the possibility for FD-ViTs to be incorporated into vision–language and foundation model frameworks, becoming components of instruction-tuned and context-aware systems. Similarly, in multimodal scenarios, such as visible–infrared fusion, spectral decomposition has been observed to enhance inter-modality complementarity.

Recent developments in generative modeling, including diffusion models and masked image modeling (MIM), suggest new opportunities for FD-ViTs. Although most FD-ViT research has focused on discriminative tasks such as classification, restoration, and segmentation, several studies in related areas indicate that frequency-domain representations can improve training stability and convergence behavior in generative pipelines by separating global structure from high-frequency detail components.

In masked image modeling frameworks, frequency-aware tokenization and spectral reconstruction objectives have been explored as alternatives to purely spatial masking strategies, potentially improving reconstruction fidelity for texture-rich regions. Similarly, diffusion-based image generation models have begun incorporating spectral filtering and frequency-guided denoising steps to stabilize iterative generation and reduce high-frequency noise artifacts.

While explicit FD-ViT architectures designed specifically for generative tasks remain limited, these early connections suggest that frequency-domain token mixing and multi-scale spectral representations may provide useful inductive biases for generative Transformer models, particularly in heavy-reconstruction learning paradigms. Exploring FD-ViT design principles in diffusion and MIM frameworks, therefore, represents a promising direction for future research.

A second important trend is the shift from fixed-frequency transformations to data-driven, dynamic spectral modules. Most existing FD-ViT architectures rely on predefined Fourier, DCT, or wavelet transforms. However, the sensitivity of different tasks and data distributions to specific frequency bands varies substantially. Consequently, adaptive mechanisms that learn frequency band boundaries and spectral weights directly from data are gaining increasing importance. Dynamic frequency convolutions and adaptive spectral partitioning approaches allow frequency representations to evolve jointly with the learning process, rather than remaining static preprocessing steps.

Another notable direction in FD-ViT research concerns designs focused on computational efficiency and sustainability. Although frequency-domain operations are theoretically advantageous, hardware compatibility and energy consumption remain practical constraints. In this regard, Einstein-multiplication–based mixing mechanisms and scattering-based approaches [57] contribute to the development of lighter models by reducing parameter counts and quadratic complexity. By lowering memory usage and energy consumption, such methods make FD-ViTs more suitable for edge devices and resource-constrained environments, aligning the field with the principles of “Green AI.”

A more forward-looking research direction involves extending frequency-based learning beyond classical computational paradigms. Quantum-based frequency analysis approaches (e.g., QHSA-ViT [54]) suggest that high-dimensional spectral representations can be processed more efficiently in Hilbert space. Techniques such as the Quantum Discrete Fourier Transform may enable parallel spectral processing in complex and dynamic scenes, opening a new research horizon for the scalability of FD-ViT architectures. Although still at an early stage, this direction offers noteworthy potential for the long-term evolution of frequency-based models.

Finally, future FD-ViT research is expected to increasingly focus on robustness and universal representation learning. It is becoming increasingly evident that frequency-domain representations provide a natural filtering effect against high-frequency noise and adversarial perturbations. Wavelet-based tokenization and spectral decomposition mechanisms enable models to behave more robustly in out-of-distribution scenarios and data-scarce regimes. This capability positions FD-ViTs not merely as architectures optimized for specific tasks, but as promising candidates for developing generalizable visual representations across diverse data regimes.

Overall, emerging trends analysis indicates that FD-ViT approaches are in a maturing phase, and the research focus is shifting towards more fundamental, holistic, and application-oriented problems. Future studies that treat frequency information not as a secondary feature, but as a fundamental building block of Transformer-based visual learning, will significantly increase both the theoretical depth and practical impact of this field.

8. Conclusions

This survey has examined FD-ViT architectures from a comprehensive and systematic perspective, providing a holistic view of their role in visual understanding, enhancement, and classification tasks. The reviewed literature demonstrates that structural limitations of classical ViT architectures—such as quadratic computational complexity, limited local inductive bias, and spectral information loss—can be addressed to varying degrees through Fourier-, wavelet-, and DCT-based frequency integrations. Accordingly, it becomes evident that frequency information should be treated not merely as a preprocessing step, but as a fundamental representational component that directly influences the learning dynamics of Transformer blocks.

The proposed taxonomy and comparative analyses reveal that FD-ViT approaches do not follow a single architectural paradigm; rather, they are shaped by distinct design choices depending on the employed spectral transformation, the level of integration, and the target task. Fourier-based architectures stand out in global context modeling and computational efficiency, particularly for large-scale structural restoration tasks, while wavelet-based approaches deliver more consistent performance in domains where texture sensitivity is critical—such as medical imaging and super-resolution—owing to their local–frequency decomposition and invertible downsampling capabilities. DCT-based models, on the other hand, provide a more balanced trade-off between computational cost and performance through energy compaction and frequency pruning mechanisms, making them a suitable alternative for resource-constrained and sustainable applications.

Results reported in domain-specific applications further indicate that the noise robustness and spectral signature–capturing capabilities of FD-ViT architectures offer pronounced advantages in expertise-driven fields such as medical diagnosis, precision agriculture, and remote sensing. These successes highlight the ability of the frequency domain to more effectively represent discriminative information components that often remain implicit in visual data, suggesting that frequency-based approaches may constitute a structural necessity for certain classes of problems.

Nevertheless, significant challenges remain that hinder the widespread adoption of FD-ViT approaches in practical applications. In particular, the fact that modern hardware accelerators are not yet optimized for frequency-domain operations to the same extent as for spatial convolutions and dense matrix multiplications prevents theoretical computational advantages from being fully realized in real-time inference scenarios. This underscores the importance of future research on hardware–software co-design, standardized evaluation protocols, and scalable architectural strategies.

In conclusion, this survey presents the FD-ViT literature not merely as a compilation of existing studies but as a reference framework that more clearly defines the role of frequency-based representation learning within ViT architectures. The integrated use of spectral depth and spatial attention enables a more balanced modeling of both global structures and fine-grained details in visual data, pointing toward a meaningful paradigm shift in the future of Transformer-based visual learning. The analyses and syntheses provided herein aim to facilitate more informed, comparable, and application-oriented progress in FD-ViT research.