Optimization Design of Microwave Filters Based on Deep Learning and Metaheuristic Algorithms

Abstract

To address the efficiency bottlenecks of traditional full-wave simulation methods in the high-performance design and rapid optimization of microwave filters, this study proposes an efficient design method based on an improved surrogate model and a hybrid optimization algorithm. A one-dimensional dense convolutional autoencoder (1D-DenseCAE) model is constructed to enhance the model’s ability to extract key features and improve convergence speed. Additionally, the Ivy–Hiking optimization algorithm (IHOA) is introduced, combining the advantages of global search and local fine-tuning. Experiments demonstrate that this method achieves approximately a 25% improvement in convergence speed over the standard one-dimensional convolutional autoencoder (1D-CAE) in cavity filter design, and enables efficient optimization in complex structures such as interdigital filters and seventh-order cross-coupled cavity filters, meeting design requirements of return loss below −20 dB and in-band ripple under 0.5 dB. This method provides an effective technical pathway for the intelligent design of microwave filters.

1. Introduction

Microwave filters represent a pivotal component within contemporary wireless communication systems, exerting a direct influence on the systems’ signal selectivity and their capacity to withstand interference. Electromagnetic (EM)-based yield optimization is a challenging task in microwave design due to its need for extensive EM simulations, which are typically time-consuming [1]. Employing EM-based fine models within the traditional Monte Carlo (MC) framework incurs prohibitive computational costs, owing to the extensive EM simulations required to achieve significant yield improvement [2]. Developing efficient yield-driven optimization techniques for microwave systems with high computational costs has been the objective of substantial research efforts throughout the last decade [3,4,5,6,7].

Significant progress has been made in the research and application of surrogate models in the field of microwave. In 2020, Zhang et al. proposed a neuro-TF-based surrogate model [1]. In terms of optimization methods in the microwave field, Jin et al. proposed an advanced cognition-driven electromagnetic optimization approach based on a transfer-function feature-based surrogate model [8]. Zhao et al. put forward a homotopy optimization approach using an artificial neural network (ANN)-based surrogate model [9]. Compared with conventional ones, these methods considerably improve computational efficiency. Subsequently, a hybrid surrogate model-assisted evolutionary algorithm for filter optimization (H-SMEAFO) method [10], an AI-assisted surrogate modeling and optimization (SAO) method [11], a pattern search optimization-based surrogate modeling space definition (PSOMSD) method [12], and an offline surrogate model-assisted evolutionary algorithm [13] were proposed. These have effectively cut electromagnetic simulation costs and shortened the design cycle.

In recent years, studies in this field have introduced methods like neural networks and spatial mapping into surrogate models. For example, a microwave microfluidic sensor optimization and design approach based on the particle-ant colony optimization algorithm (PACO) and wolf colony algorithm (WCA) has been proposed [14]. By optimizing the microfluidic channel routing, the sensor sensitivity was enhanced by approximately 78.6%. An improved biogeography-based optimization algorithm with multiple adaptive mutation operators (BBOIMAM) [15] was proposed to strengthen local search abilities. Recent innovations have integrated response feature extraction technology with the particle swarm optimization (PSO) algorithm [16], resulting in a significant reduction in the required number of training data points. The Ivy algorithm (IVYA) [17] exhibits remarkable global exploration and diversity maintenance capabilities, offering extensive solution-space coverage during optimization. However, it is relatively less effective in fine-tuning near solutions. The Hiking optimization algorithm (HOA) [18] excels at leveraging “gradient” information for local solution fine-tuning, achieving high-precision convergence. However, when used alone, it may encounter premature convergence or insufficient global exploration in complex, high-dimensional problems. In order to address the aforementioned challenges, a hybrid approach combining IVYA and HOA can be adopted. In the early search phase, the IVYA mechanism is dominant, utilizing the diffusion growth strategy of ivy to extensively explore the solution space and rapidly identify several regions in close proximity to the global optimum. In the subsequent search phase, the local search mechanism of HOA is introduced. The utilization of gradient information, in conjunction with the direction provided by a lead hiker, serves to further explore the regions of potential solution identified in the initial stages of the process. This approach enables the refinement of filter parameters, thereby ensuring the optimal performance of the system. The present paper puts forward a collaborative innovation framework based on a “surrogate model optimization algorithm”:

• The proposed 1D-DenseCAE surrogate model incorporates cross-layer feature reuse through DenseNet [19] (densely connected convolutional network) connectivity, effectively mitigating the gradient-vanishing problem. Furthermore, the embedded efficient channel attention (ECA-Net) mechanism dynamically weights key frequency-band features, enhancing the accuracy of S-parameter prediction.
• The IHOA integrates IVYA’s population-collaborative diffusion mechanism with HOA’s terrain-adaptive step-length strategy, establishing a closed-loop optimization framework that balances global exploration and local exploitation.
• The 1D-DenseCAE surrogate model substitutes electromagnetic simulation for fitness evaluation in the optimization process. When integrated with IHOA, this combined approach enables efficient exploration of the parameter space.

Experimental results demonstrate 25% faster convergence with the 1D-DenseCAE model compared to conventional 1D-CAE for cavity filter optimization. Notably, the IHOA integrated with 1D-DenseCAE achieves significant performance in diverse complex microwave filters, satisfying stringent specifications, including passband return loss below −20 dB and in-band ripple under 0.5 dB. This work provides a robust theoretical foundation and an efficient computational framework for high-performance microwave filter optimization.

2. Construction of the 1D-DenseCAE Surrogate Model

Manufacturing errors can significantly degrade filter performance. Thus, obtaining a filter design robust to such errors is crucial. However, the main challenge in filter yield optimization is efficiency. To address this, surrogate models have been proposed [20,21,22,23]. Surrogate models are computationally cheap approximation models mapping the filter design parameters to the filter responses or features extracted from filter responses [1]. As the most advanced technique for feature learning with convolution kernels in an unsupervised way, the convolutional autoencoder (CAE) has been recently developed to tackle the issue of feature extraction for classification tasks [24]. However, standard CAE has limitations: it depends heavily on random initialization and data statistics and lacks explicit control over the physical significance of filters, shape adaptability, feature diversity, and sensitivity to subtle features. There is also a risk that optimization goals may not fully align with final application objectives. To address these issues, we improve upon the standard CAE and introduce the 1D-DenseCAE surrogate model. The network structure comprises two core components: an encoder and a decoder. The encoder transforms input S-parameter responses (including real and imaginary parts) into corresponding geometric design parameters. The decoder performs the reverse mapping, reconstructing the S-parameters from geometric parameters. The key innovation lies in integrating a dense feature reuse mechanism and channel attention guidance, significantly enhancing prediction accuracy and generalization ability. The model architecture is illustrated in Figure 1.

2.1. Encoder: Multi-Scale Feature Extraction

The encoder compresses the input S-parameters (dimension [400, 2]) into a 256-dimensional latent vector through hierarchical feature extraction. The first layer is a 1D convolutional layer with 16 filters of size 3 and stride 1, producing a feature map of [400, 16]. Three cascaded DenseBlocks perform deep abstraction; each contains four convolutional layers (with 3 × 3 kernels), utilizing dense connections to maximize feature reuse and gradient propagation. An efficient channel attention (ECA) module (with a 1D convolutional kernel of size 3) follows each convolutional layer, adaptively weighting key channels. Transition layers, incorporating downsampling with stride 2, connect the blocks and progressively compress the spatial dimensions ([400, 64] → [200, 32] → [100, 44] → [50, 44]), while expanding the receptive field. Finally, the feature maps are flattened and passed through a fully connected layer to generate the latent representation.

2.2. Decoder: High-Fidelity Response Reconstruction

The decoder maps the 256-dimensional latent vector back to the original S-parameter space of [400, 2]. The first fully connected layer (256 nodes, Mish activation [25]) decodes the latent features. The second fully connected layer (1024 nodes, Mish activation) expands the feature dimensionality. The third fully connected layer (800 nodes, Mish activation) outputs a vector reshaped into a [400, 2] matrix, reconstructing the S-parameter response.

2.3. Loss Function

During the training of the 1D-DenseCAE model, the loss function is based on reconstruction error, aiming to evaluate the model’s ability to reconstruct the input data. The mathematical expression is as follows:

$$L_{\mathit{rec}}={\displaystyle \frac{1}{N}}\sum _{i=1}^N|T_{\mathit{real},i}-T_{\mathit{pred},i}|$$

(1)

In this context, $T_{\mathit{real},i}$ denotes the actual value input to the encoder, and $T_{\mathit{pred},i}$ represents the reconstructed value output by the decoder.

The specific technical details of the constructed 1D-DenseCAE model are depicted in Figure 2.

Table 1. Network parameter settings of the 1D-DenseCAE model.

Layer		Size/Nodes	Stride	Output Shape	Activation
Input Layer		[400, 2]	-	[400, 2]	-
Encoder
Initial Conv Layer		3 × 16	1	[400, 16]	-
Dense Block 1	Conv1	3 × 12	1	[400, 28]	ReLU
	Conv2	3 × 12	1	[400, 40]	ReLU
	Conv3	3 × 12	1	[400, 52]	ReLU
	Conv4	3 × 12	1	[400, 64]	ReLU
Transition Layer 1		1 × 24	2	[200, 32]	ReLU
Dense Block 2	Conv1	3 × 12	1	[200, 32]	ReLU
	Conv2	3 × 12	1	[200, 44]	ReLU
	Conv3	3 × 12	1	[200, 56]	ReLU
	Conv4	3 × 12	1	[200, 68]	ReLU
Transition Layer 2		1 × 48	2	[100, 40]	ReLU
Dense Block 3	Conv1	3 × 12	1	[100, 52]	ReLU
	Conv2	3 × 12	1	[100, 64]	ReLU
	Conv3	3 × 12	1	[100, 76]	ReLU
	Conv4	3 × 12	1	[100, 88]	ReLU
Transition Layer 3		1 × 48	2	[50, 44]	ReLU
Flatten Layer		2200	-	[2200]	-
Decoder
Fully Connected Layer 1		256 Nodes	-	[256]	Mish
Fully Connected Layer 2		1024 Nodes	-	[1024]	Mish
Fully Connected Layer 3		800 Nodes	-	[800]	Mish
Reshape Layer		-	-	[400, 2]	-

presents a comprehensive overview of the main parameter configurations for the 1D-DenseCAE model, including layer names, the scale of nodes or convolutional kernels, stride, output shape, and the activation functions utilized.

3. Design of IHOA

To address local optima entrapment and slow convergence in microwave filter optimization, this study proposes the IHOA. The IHOA implementation process is as follows. During initialization, the population is generated using max–min Latin hypercube sampling (LHS) [26] to ensure uniform coverage of the solution space. In the global search phase, the IVYA is employed to simulate vine spreading behavior [18]. Individuals sense the direction of the global optimum and dynamically adjust their positions for extensive exploration. For local optimization, the best-performing individuals are selected, and the HOA is applied. Utilizing Tobler’s hiking function [27] to dynamically adjust step sizes, position updates incorporate global guidance and random perturbations. Boundary handling ensures solution validity. Throughout iterations, global spreading and local exploitation alternate. Elite individuals are retained until the maximum iterations or a convergence threshold is met, and the global optimum is ultimately output.

3.1. Mathematical Models of Algorithm Components

3.1.1. Mathematical Model of the IVYA Global Search Module

The IVYA module drives population diffusion via a vine-growth-based model, exploring the global optimum through fitness guidance and position updates. Its formula for calculating the global optimum $X_{\mathrm{best}}$ is as follows:

$$X_{\mathrm{best}}=\arg {\min }_{X_i\in \mathit{Pop}}f\left(X_i\right)$$

(2)

Here, $f\left(X_i\right)$ represents the fitness of individual $X_i$, and $\mathit{Pop}$ denotes the current population. The position update mechanism enables individuals to adjust their growth direction according to the global optimum. The update formula is as follows:

$$X_i^{\mathit{new}}=X_i+\alpha \cdot \left(X_{\mathit{best}}-X_i\right)$$

(3)

Here, $X_i$ is the current position of individual $i$, and $\alpha$ is the factor controlling the guidance strength of individuals towards the global optimum.

The module mimics vine phototropism [28]. Within the maximum iterations, it updates individuals’ growth rates and positions, calculates new fitness values, and decides on accepting new positions. After each iteration, it updates the global optimal individual and fitness, finally outputting the global optimum. Through population collaboration and adaptive search, IVYA efficiently optimizes in complex spaces.

3.1.2. The HOA Local Optimization Module

The HOA module performs a sophisticated search on the premium solution regions that have been generated by the IVYA module. The system performs exhaustive local solution mining through velocity modelling and position updates. The velocity modeling adopts the Tobler hiking function:

$$V_{i,t}=6\cdot \exp \left(-3.5\cdot \left|\tan \left({\theta }_{i,t}\right)+0.05\right|\right)$$

(4)

Here, $V_{i,t}$ is the velocity of individual $i$ at iteration $t$, and ${\theta }_{i,t}\in \left[0^{\circ },{50}^{\circ }\right]$ is the random slope angle used to simulate the steepness of terrain. After obtaining the walking velocity, each individual performs a guided-offset update based on the position information of the current global best individual $X_{\mathit{best}}$. The position update formula is as follows:

$$X_{i,t+1}=X_{i,t}+V_{i,t}+{\gamma }_{i,t}\cdot \left(X_{\mathit{best}}-{\alpha }_{i,t}\cdot X_{i,t}\right)$$

(5)

In the above formula, $X_{i,t}$ is the current position of individual $i$; ${\alpha }_{i,t}$ is the sweeping factor, controlling the offset intensity of individuals from the optimal direction, with ${\alpha }_{i,t}\in \left[\mathrm{1,2}\right]$; ${\gamma }_{i,t}$ is the perturbation coefficient, indicating the adjustment amplitude of the individual update direction, and ${\gamma }_{i,t}\in \left[\mathrm{0,1}\right]$. By dynamically adjusting velocity and position, HOA conducts a fine-grained local search to avoid falling into a local optimum.

3.2. A Case Study of Filter Design Based on Surrogate Model and IHOA

Figure 3 illustrates the design process driven by the 1D-DenseCAE model and IHOA. The 1D-DenseCAE surrogate model, trained on EM simulation data by integrating DenseNet and ECA-Net, accurately captures the mapping between filter geometry and S-parameters. The trained model is integrated into the IHOA framework, which combines IVYA and HOA. These algorithms work together to iteratively optimize filter design parameters until preset performance criteria are met, significantly boosting design efficiency without sacrificing accuracy.

IHOA first initializes the population and related parameters and then enters the main loop. After performing a global and local search, the algorithm screens the population based on the fitness results and dynamically adjusts the step size to improve convergence and search accuracy. Finally, when the termination conditions are met, the global optimal solution and its corresponding filter performance are output. The algorithm implementation process is detailed in Algorithm 1.

Algorithm 1: IHOA (data processing automation and output reading)

Input: Population size $Npop$, dimension D, search bounds $I_{max}$ and $I_{min}$, max iterations MaxIter Output: Global best individual $I_{best}$ and its fitness $f_{best}$ 1. Initialize the population using LHS 2. Evaluate the fitness of all individuals 3. Identify current global best $I_{best}$ and $f_{best}$ 4. For each iteration until the maximum iteration limit is reached:  // IVYA’s global search 5. Generate a new candidate solution by updating position based on global diffusion (Use the IVYA strategy to simulate growth and expansion) 6. If the new fitness is better, Then update the current individual with this new solution 7. Rank individuals based on fitness and select the top-K with the highest fitness for local refinement 8. For each of the selected K individuals:  // HOA’s local optimization 9. Randomly generate a terrain slope angle between 0° and 50° 10. Compute walking velocity based on slope using Tobler’s hiking function 11. Calculate a global guidance component to follow the current global best 12. Update the individual’s position based on walking direction and velocity 13. If the new solution improves fitness, Then replace the original individual 14. Return the best solution ($I_{best}$) and its corresponding fitness ($f_{best}$)

4. Experimental Results and Discussion

4.1. Validation of the 1D-DenseCAE Model

This section validates the 1D-DenseCAE model using a cavity filter model (Figure 4). Data were collected via Python (3.13.5)-HFSS co-simulation. First, the geometric parameters $H={\left[h_1,h_2,h_3,w_1,w_2\right]}^T$ were determined and randomly perturbed within a reasonable range. These parameters were input into HFSS to obtain $S_{11}$ and $S_{21}$ data. This process continued until 350 data points were collected.

After data collection, the 1D-DenseCAE model was trained using these sample data, and the loss during training was calculated using Equation (1).

As shown in Figure 5, the loss decreases rapidly initially. The 1D-DenseCAE model’s loss stabilizes after around 6000 training iterations, approximately 25% faster than the 8000 iterations required by the traditional 1D-CAE model. This indicates that 1D-DenseCAE can more quickly identify an effective parameter space and adapt to the training data. Figure 6 and Figure 7 illustrate the training processes. The stabilized loss value of 1D-DenseCAE is significantly lower than that of 1D-CAE. Its fitting curve closely overlaps the simulation data, while the traditional model shows slight deviations even after 8000 iterations.

4.2. Verification of the Surrogate Model and IHOA

For verification, we selected two types of passband microwave filters as research samples. The first was an interdigital filter with a center frequency of $f_0=10\mathrm{G}\mathrm{H}\mathrm{z}$, a passband from 8 to $12\mathrm{GHz}$, and a requirement for $S_{11}$ below −20 dB within the passband, with in-band ripple under 0.5 dB. The second was a more complex seventh-order cross-coupled cavity filter with a center frequency of $f_0=1.94\mathrm{G}\mathrm{H}\mathrm{z}$, a passband from 1.92 to $1.96\mathrm{GHz}$, and a requirement for $S_{11}$ below −20 dB within the passband, also with in-band ripple under 0.5 dB. These filters help evaluate the applicability and performance of optimization methods across different topological structures.

We designed a comprehensive data-driven optimization workflow, whose core lies in decoupling traditional time-consuming electromagnetic simulation tasks from intelligent agent models, thereby significantly enhancing optimization efficiency. First, a parameterizable 3D filter model was established in HFSS. Then, using the Python (3.13.5) automation interface (COM API) provided by HFSS, we controlled HFSS to perform batch simulations of different parameter combinations. For each set of geometric parameters, we extracted the corresponding S-parameters within their frequency range. After each simulation was completed, the system saved the geometric parameters (input) and the corresponding S-parameters (output) as a set of data samples, accumulating a total of 300 datasets. The data were stored in .csv format, forming the original training dataset for subsequent model learning. After the data preparation was complete, a proxy model was constructed and trained based on the 1D-DenseCAE structure. This model takes the geometric parameters of the filter as input and outputs the corresponding S-parameter spectrum. In the optimization phase, first, based on the S-parameters output by the proxy model, a fitness function was constructed that comprehensively considered design constraints such as center frequency, insertion loss, and in-band fluctuation. The function is as follows:

$$f\left(x\right)={m\mathit{ax}}_{f\in [f_1,f_2]}\left(\left|S_{11}\left(f,x\right)\right|-\left(-20\mathrm{dB}\right)\right)+\mathsf{\lambda }\cdot \sum _{i=1}^n{\left(x_i-x_0\right)}^2$$

(6)

Then, the IHOA was used to perform global search and local optimization of the filter structure parameters. After several iterations, the current optimal design was re-entered into HFSS for real simulation, and the results were added to the training set for incremental learning of the proxy model. This ensured the generalization ability of the proxy model while continuously improving its prediction accuracy in key design areas.

4.2.1. Verification of the Optimization Design of the Interdigital Filter Model

As shown in Figure 8, the interdigital filter model has geometric parameters $H={\left[h_1,h_2,h_3,h_4,h_5\right]}^T$.

By adopting a surrogate model to replace repetitive simulations and combining it with a global–local collaborative optimization strategy, efficient searching was achieved. The S11 values of the filter were measured to be below −20 dB in the frequency range of 8 GHz to 12 GHz, thereby meeting the design target requirements. The optimized performance was significantly improved, as shown in Figure 9, and further verified by

Table 2. Values of the geometric design parameters for each stage.

	Stage 1	Stage 2	Stage 3	Stage 4	Stage 5	Stage 6
h1	0.329	0.418	0.378	0.259	0.288	0.274
h2	1.674	1.593	1.733	1.631	1.653	1.642
h3	1.838	1.677	1.718	1.812	1.834	1.805
h4	1.678	1.698	1.671	1.734	1.753	1.779
h5	1.628	1.583	1.794	1.822	1.797	1.806

.

4.2.2. Verification of the Optimization Design of a Seventh-Order Cross-Coupled Cavity Filter

Figure 10 presents a complex seventh-order cross-coupled cavity filter. Its adjustable size parameters were $H={\left[h_1,h_2,h_3,h_4,h_5,h_6,h_7,h_{12},h_{23},h_{34},h_{45},h_{56},h_{67},h_{13},h_{46}\right]}^T$. Despite its complexity, the surrogate model could efficiently predict S-parameters using limited samples. Figure 11 shows the S-parameter responses obtained at different optimization stages using the IHOA.

Table 3. Values of the geometric design parameters for each stage.

	Stage 1	Stage 2	Stage 3	Stage 4	Stage 5	Stage 6
h1	3.75	3.924	4.113	4.074	3.926	3.967
h2	3.846	4.179	4.188	3.989	4.125	4.146
h3	3.826	4.173	3.885	4.249	4.176	4.118
h4	3.751	3.747	4.258	3.922	4.066	4.05
h5	3.945	3.99	3.712	4.137	4.052	3.997
h6	4.025	3.894	3.776	4.033	4.259	3.839
h7	3.966	4.214	3.789	3.753	3.944	4.059
h12	6.831	7	6.716	6.951	6.86	6.969
h23	7.257	7.029	7.049	7.084	7.104	7.289
h34	7.255	6.945	7.133	7.029	6.989	7.021
h45	6.794	6.873	6.728	7.205	7.141	7.148
h56	7.093	7.1	7.297	7.028	7.033	6.97
h67	6.924	7.269	7.077	7.159	7.189	7.276
h13	5.13	4.876	4.702	5.05	5.032	4.948
h46	5.175	4.736	4.792	4.937	5.101	5.166

lists the geometric design parameter values at each stage.

In optimizing the seventh-order cross-coupled cavity filter, the IHOA combined with the 1D-DenseCAE surrogate model performed remarkably. Over six iterations, key parameters like h₁ converged from 3.75 mm to 3.967 mm (a fluctuation of ±5.7%), and h₄₆ was fine-tuned from 5.175 mm to 5.166 mm (a change of <0.2%), highlighting the algorithm’s coordination of complex parameters. After optimization, the filter achieved better than −20 dB return loss ($S_{11}$) within the passband from 1.92 to 1.96 GHz, indicating good impedance matching. A sharp transmission zero was observed at approximately 1.91 GHz, where the $S_{21}$ dropped to around −100 dB, effectively blocking signal transmission at that frequency. This deep null is attributed to the cross-coupling between non-adjacent resonators near the input side (between h₁ and h₃), which forms a signal cancellation path through destructive interference. The resulting improvement in out-of-band selectivity confirms the effectiveness of the cross-coupled topology in enhancing interference suppression. For high-dimensional, strongly coupled microwave filter structures, the IHOA efficiently achieves design goals using a limited number of samples.

5. Conclusions

This study proposes a solution driven by the 1D-DenseCAE surrogate model and IHOA. The 1D-DenseCAE enhances feature reuse via DenseNet, improving S-parameter prediction accuracy and training efficiency. It converges 25% faster compared to traditional 1D-CAE. In optimization, the IHOA shows high global exploration efficiency and meets strict design requirements (passband return loss below −20 dB, in-band ripple under 0.5 dB) in complex structures like interdigital filters and seventh-order cross-coupled cavity filters. This solution offers a reliable foundation for designing high-performance microwave filters.