A Nonlinear Optimization Design Algorithm for Nearly Linear-Phase 2D IIR Digital Filters

Abstract

In this paper, a new optimization method for the design of nearly linear-phase two-dimensional infinite impulse (2D IIR) digital filters with a separable denominator is proposed. A design framework for 2D IIR digital filters is formulated as a nonlinear constrained optimization problem where the group delay deviation in the passband is minimized under prescribed soft magnitude constraints and hard stability requirements. To achieve this goal, sub-level sets of the group delay deviations are utilized to generate a sequence of filters, from which the one with the best performance is selected. The quality of the obtained filter is evaluated using three quality factors, namely, the passband magnitude quality factor $Q_h$ and the group delay deviation quality factor $Q_{\tau }$, while the third one is a new quality factor $Q_s$ that assesses the performance in the stopband relative to the minimum filter gain in the passband. The proposed framework is implemented using the interior-point (IP) method in a MATLAB environment, and the experimental results show that filters designed using the proposed method have good magnitude response and low group delay deviation. The performance of the resulting filters is compared with the results of other methods.

1. Introduction

Two-dimensional (2D) digital filters have found many signal processing applications, such as image processing, video signal filtering, beamforming systems, and remote sensing. Digital filters are broadly classified into two main categories, namely, finite impulse response (FIR) filters and infinite impulse response (IIR) filters [1]. An FIR filter is one whose impulse response is of finite duration. The output of such a filter is calculated solely from the current and previous input values. This type of filter is hence said to be non-recursive. On the other hand, an IIR filter is one whose impulse response (theoretically) continues forever in time. They are also termed as recursive filters. In addition to the current and previous input values, the current output of such a filter depends on the previous output values.

FIR filters are generally easier to implement, as they are non-recursive, always stable (by definition), and can be designed to have a linear phase characteristic. They are, however, not well-suited for very sharp (narrow transition band) frequency responses. IIR filters, on the other hand, are much more difficult to design due to their non-linear phase and their stability constraints. However, they can effectively accommodate very narrow transition band frequency responses, which makes them suitable for a broad range of applications. Moreover, selective IIR filters are much more efficient than FIR filters, as the latter require higher orders (i.e., more multiplications) [1,2,3,4,5,6,7,8,9,10]. The efficiency of IIR filters stems from their ability to satisfy the required filter specifications with a much lower filter order, thereby reducing the computational requirements. In addition, IIR filters can have a much smaller group delay compared with FIR filters [1,11,12,13]. Due to these advantages, IIR filters find extensive applications in the domains of de-noising of digital images, biomedical imaging and digital mammography, X-ray image enhancement, and seismic data processing.

Since many signal processing applications are very sensitive to phase non-linearity, the design of IIR filters needs to take into consideration not only the amplitude requirements but also the phase constraints. Therefore, for most of these applications, the designed IIR filters should have a nearly linear-phase response. Moreover, hard constraints on the location of the poles of IIR filters must be enforced to ensure the stability of the obtained filters [14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29]. These and other design requirements usually lead to highly nonlinear constrained optimization problems that require highly sophisticated optimization methods [1,12,29,30,31,32]. Research on 2D IIR filters has produced many constrained and non-constrained optimization problems, depending on the researchers’ design objectives. Moreover, each researcher has designed specialized algorithms and techniques to handle their optimization problems. Most of these methods can be classified into linear and nonlinear techniques. In linear design techniques, the nonlinear optimization problem is approximated by a sequence of linear optimization problems [1,11], which are then solved using linear programming. Starting from an initial filter that satisfies most of the design specifications, the algorithm linearizes the nonlinear optimization around this initial filter to produce a linear optimization problem. Solving this problem leads to an improved filter, around which the linearization is carried out. This iterative process is repeated until an acceptable filter is achieved. Although the linear methods can be very fast, and the designer might be able to find an initial filter satisfying most of the design specifications, there is no guarantee that the will converge or produce accurate solutions. As such, nonlinear techniques are preferred. Some researchers have attempted to develop design methods based on modern optimization algorithms.

In [29], the Sequential Partial Optimization algorithm was proposed to convert the minmax design problem into a sequence of smaller sub-problems, each updating only a single second-order denominator factor.

In [33], a modern optimization methodology known as semi-definite programming (SDP) is utilized. This SDP method includes minmax and weighted least-squares designs of FIR filters as well as minmax formulations of stable separable-denominator IIR filters. In [33] describes how SDP has the potential to serve as a unified optimization engine for designing a wide range of 2D digital filters.

The Quasi-Newton method has been applied to design 2D IIR digital filters [7], but it may result in unstable filters. Neural networks were used in [34] to design 2D IIR filters, which can guarantee the stability of the designed filters with low computational time. Some researchers have employed the genetic algorithm (GA) to design 2D IIR digital filters [32,35,36,37,38]. Others have proposed hybrid methods that combine different evolutionary algorithms, such as biogeography-based optimization and particle swarm optimization in [39], to tackle the nonlinear 2D IIR optimization design.

In this paper, a robust nonlinear optimization method for the design of nearly linear-phase 2D IIR digital filters with separable-denominators is proposed. This design problem is formulated as a nonlinear constrained optimization problem where the group delay deviation is minimized under prescribed soft magnitude constraints and hard stability requirements. We also rely on sub-level sets of the group delay deviations to obtain a sequence of filters, from which we choose the most optimal one. The performance of the obtained 2D IIR filters is evaluated using three quality factors. These first two quality factors are the usual passband magnitude quality factor $Q_h$ and the group delay deviation quality factor $Q_{\tau }$, while the third one is a new quality factor $Q_s$ that assesses the performance of the filter within the stopband relative to the minimum filter gain in the passbands, and that is because amplitude response anomalies in transition regions are undesirable [40]. The obtained results show that these quality factors are all necessary to judge the optimality of the filter. The proposed framework is implemented using the interior-point (IP) method in a MATLAB environment, and the results are very promising.

The remainder of the paper is organized as follows. The constrained optimization problem for 2D IIR digital filters is formulated in Section 2. Section 3 is for the stability requirements for the 2D IIR digital filters with a separable denominator. Section 4 presents the constrained optimization design problem for a 2D IIR digital filters. Section 5 is dedicated to the quality of the design, and Section 6 to the experimental results. Section 7 concludes with a brief summary and future work directions.

2. Formulation of the Design Problem

Much attention has been paid to the optimal design of 2D filters with separable denominators [18,29,30,33,41,42,43]. The stability of such 2D filters can be easily guaranteed by ensuring the roots of both denominators are within the unit bi-disc independently. As such, the internal stability triangles for 1D second-order factors are used as stability constraints [44]. In addition, separable denominators reduce the complexity of the 2D filter and the induced optimization problem and also result in fewer coefficients [29,45]. Although the phase of the optimized 2D IIR filter is only nearly linear, one can see that the separable denominators 2D IIR filter offers good selectivity and computation efficiency and reduced system delay without compromising the required amplitude specifications when compared with the corresponding other 2D filters [45]. As such, we will assume in this paper that the filters of interest have separable denominators. The desired 2D IIR filter can then be represented by the following transfer function [46,47,48]:

$$\begin{array}{cc}\hfill H(a,b,z_1,z_2)& ={\displaystyle \frac{N(z_1,z_2,b)}{D\left(z_1,z_2,a\right)}}\hfill \\ \hfill \\ \hfill & =H_0{\displaystyle \frac{N\left(z_1,z_2,b\right)}{D_1\left(z_1,a_1\right)D_2\left(z_2,a_2\right)}},\hfill \end{array}$$

(1)

where b is a vector of the form $[b_{00}{b}_{01}\dots b_{N_1{N}_2}]$, and $N(z_1,z_2,b)$ is a multivariate polynomial of the form:

$$\begin{array}{c}\hfill N\left(z_1,z_2,b\right)=\sum _{j=0}^{N_2}\sum _{i=0}^{N_1}{b}_{ij}{z}_1^{-i}{z}_2^{-j},\end{array}$$

(2)

$D_1(z_1,a_1)$ and $D_2(z_2,a_2)$ are polynomials in $z_1$ and $z_2$, respectively, of the form

$$\begin{array}{cc}\hfill D_1(z_1,a_1)& =\prod _{n=1}^{J_1}(1+a_{1n,1}{z}_1^{-1}+a_{1n,2}{z}_1^{-2}),\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill D_2(z_2,a_2)& =\prod _{n=1}^{J_2}(1+a_{2n,1}{z}_2^{-1}+a_{2n,2}{z}_2^{-2}),\hfill \end{array}$$

(4)

where $J_1=N_1/2$ and $J_2=N_2/2$ and $N_1$ and $N_2$ are even.

The vectors $a_1$ and $a_2$ are given by:

$$\begin{array}{cc}\hfill a_1& =[a_{11,1}{a}_{11,2}{a}_{12,1}{a}_{12,2}\dots a_{1J_1,1}{a}_{1J_1,2}], \mathrm{and}\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill a_2& =[a_{21,1}{a}_{21,2}{a}_{22,1}{a}_{22,2}\dots a_{2J_2,1}{a}_{2J_2,2}].\hfill \end{array}$$

(6)

We then construct a vector c by concatenating the vectors b, $a_1$, and $a_2$ as follows [44]:

$$c={\left[b{a}_1{a}_2{H}_0\right]}^T.$$

Using the same notation symbol, the transfer function can be written as follows:

$$\begin{array}{cc}\hfill H(c,z_1,z_2)& ={\displaystyle \frac{N(z_1,z_2,b)}{D\left(z_1,z_2,a\right)}}.\hfill \end{array}$$

(7)

Now, the main design problem is to determine the coefficients in the numerator and denominator of Equation (7) in such a way that the obtained transfer function $H(z_1,z_2)$ satisfies all the desired specifications as explained in the following sections.

2.1. Passband Group Delay Deviations

Let $\varphi (c,w_1,w_2)$ and $\psi (c,w_1,w_2)$ be the phase angles of $N(e^{j{w}_1},e^{j{w}_2},b)$ and $D(e^{j{w}_1},e^{j{w}_2},a)$, respectively. The group delay of the transfer function $H(c,z_1,z_2)$ is given as a partial derivative with respect to $w_1$ and $w_2$ as follows:

$$\begin{array}{cc}\hfill {\tau }_1(c,w_1,w_2)& =\frac{\partial \varphi (c,w_1,w_2)}{\partial w_1}-\frac{\partial \psi (c,w_1,w_2)}{\partial w_1}, \mathrm{and}\hfill \\ \hfill {\tau }_2(c,w_1,w_2)& =\frac{\partial \varphi (c,w_1,w_2)}{\partial w_2}-\frac{\partial \psi (c,w_1,w_2)}{\partial w_2}.\hfill \end{array}$$

Letting ${\tau }_{10}$ and ${\tau }_{20}$ be the intended average passband group delays of the initial filter in each dimension over a domain ${\Omega }_{pb}$, the deviation can be computed as follows:

$$\begin{array}{cc}\hfill e{{}_g}_i(x,w_1,w_2)=& {\tau }_i(x,w_1,w_2)-{\tau }_{i0},\hfill \\ & \mathrm{for} i=1,2,\forall (w_1,w_2)\in {\Omega }_{pb},\hfill \end{array}$$

(8)

where $x={\left[c^T{\tau }_{10}{\tau }_{20}\right]}^T$, and ${\Omega }_{pb}$ is the passband region [44].

At the start of the optimization, ${\tau }_{10}$ and ${\tau }_{20}$ can be assigned values equal to the average group delays of the initial filter. As the optimization progresses, their values can be adjusted along with the coefficients of the filter so as to minimize the group delay deviation. The end result will be that ${\tau }_{10}$ and ${\tau }_{20}$ will track the average value of the group delays in each dimension. As such, we have added them to the vector of variables to be optimized.

2.2. Passband Amplitude Error

For the passband error, if we consider $H_d(w_1,w_2)$ to be the desired frequency response of the filter and that $c_k$ is the value of vector c at the start of the $k_{th}$ iteration, a passband error function at frequencies $w_1$ and $w_2$ can be defined as

$$e_h(x,w_1,w_2)\approx |H(c{}_k,w_1,w_2)|-|H_d(w_1,w_2)|.$$

Assuming that the desired amplitude response is unity in the passband, therefore, the passband error function becomes:

$$e_h(x,w_1,w_2)\approx \left|H(c{}_k,w_1,w_2)\right|-1,\forall (w_1,w_2)\in {\Omega }_{pb}.$$

(9)

The expression above can be used as is in the case of the nonlinear approach.

For the transition and stopband regions, the magnitude response is subject to prescribed levels of transition-band and stopband gains, as explained in Section 4.

3. Filter Stability

One of the major problems underlying the design task is satisfying the stability criterion for the 2D IIR filter transfer function. For the stability constraints of the 2D IIR filter, since the filter transfer function is separable in the denominator, the stability analysis is similar to the 1D case used in [11,44,49], where all the poles of $D_1(z_1,a_1)$ and $D_2(z_2,a_2)$ have to be located inside the unit circle U of the z-plane for both $z_1$ and $z_2$.

4. The Constrained Optimization Problem

To solve the design problem, the group delay deviation is minimized under hard and soft constraints. Hard constraints are the requirements that the filter must satisfy. These requirements must include at least the stability constraints of the filter. On the other hand, soft constraints are the requirements that the filter should try to satisfy. These can, for example, include the passband ripple constraint, transition-band gain. and stopband attenuation.

The general constrained optimization problem at hand can be stated as follows:

$$\begin{array}{cc}\hfill \underset{x}{min}& \underset{w_1,w_2\in {\Omega }_{pb}}{max}\left\{\right|e_{g1}(x,w_1,w_2)|,|e_{g2}(x,w_1,w_2)\left|\right\}\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill \mathrm{subject}\mathrm{to}:& \hfill \\ \hfill & \mathrm{passband}\mathrm{error}\le {\Gamma }_{pb},\hfill \\ \hfill & \mathrm{transition}\text{-}\mathrm{band}\mathrm{gain}\le {\Gamma }_{tb},\hfill \\ \hfill & \mathrm{stopband}\mathrm{gain}\le {\Gamma }_{sb},\hfill \\ \hfill & \mathrm{stability}\mathrm{margin}\ge {\epsilon }_s,\hfill \end{array}$$

(11)

where ${\Gamma }_{pb},{\Gamma }_{tb}$, and ${\Gamma }_{sb}$ are the maximum prescribed levels for the passband error, transition-band gain, and stopband gain, respectively. The stability margin, ${\epsilon }_s$, is defined as ${\epsilon }_s=1-r_{max}^{\left(p\right)}$ where $r_{max}^{\left(p\right)}$ is the maximum pole radius allowed [11]. The above general optimization problem for the nearly linear-phase 2D IIR digital filters can now be expressed as follows:

$$\begin{array}{cc}\hfill \underset{x}{min}\underset{w_1,w_2\in {\Omega }_{pb}}{max}& \left\{\right|{\tau }_1(x,w_1,w_2)-{\tau }_{10}|,|{\tau }_2(x,w_1,w_2)-{\tau }_{20}\left|\right\}\hfill \end{array}$$

(12)

$$\begin{array}{cc}\hfill \mathrm{subject}\mathrm{to}:& \hfill \\ \hfill & |e_h(x,w_1,w_2)|\le {\Gamma }_{pb},\forall (w_1,w_2)\in {\Omega }_{pb},\hfill \\ \hfill & |H(x,w_1,w_2)|\le {\Gamma }_{tb},\forall (w_1,w_2)\in {\Omega }_{tb},\hfill \\ \hfill & |H(x,w_1,w_2)|\le {\Gamma }_{sb},\forall (w_1,w_2)\in {\Omega }_{sb},\hfill \\ \hfill & |{\tau }_1-{\tau }_{10}|\le {\Gamma }_{g_1},\forall (w_1,w_2)\in {\Omega }_{pb},\hfill \\ \hfill & |{\tau }_2-{\tau }_{20}|\le {\Gamma }_{g_2},\forall (w_1,w_2)\in {\Omega }_{pb},\hfill \\ \hfill & {\mathbf{B}}_{1}x\le 1-\gamma ,\hfill \\ \hfill & {\mathbf{B}}_{2}x\le 1-\gamma ,\hfill \end{array}$$

(13)

where ${\Gamma }_{g_1}$ and ${\Gamma }_{g_2}$ are the maximum errors between the group delays of the actual and the initial filters. $\gamma =1-{(1-{\epsilon }_s)}^2$, ${\mathbf{B}}_{\mathbf{1}}$ and ${\mathbf{B}}_{\mathbf{2}}$ are the stability matrices for $D_1(z_1,a_1)$ and $D_2(z_2,a_2)$, respectively [11,44]. The 2D IIR filter design problem can be readily solved using interior-point method. This is because the interior-point method is efficient from a computational point of view and also highly efficient in practice [50,51]. To make our implementation easier, we used ${\Gamma }_g={\Gamma }_{g_1}={\Gamma }_{g_2}$. By varying ${\Gamma }_g$, and therefore ${\Gamma }_{g_1}$ and ${\Gamma }_{g_2}$, we obtain different filters from which we choose the best filter by looking at the quality factors discussed in the next section. The proposed framework is implemented using the interior-point (IP) method in a MATLAB environment and the flowchart of the optimization process is shown in Figure 1.

5. Quality of the Design

Design efficiency and the performance of the proposed method are evaluated in terms of the quality of the group delay characteristic $Q_{\tau }$, the quality of the magnitude response in the passband $Q_h$, and a new quality factor $Q_s$ for the magnitude attenuation in the stopband. We introduced this new factor, $Q_s$, to measure the quality of the magnitude in the stopband. The quality of the group delay characteristic of the 2D filter will be measured by using the normalized maximum variation of the filter group delay over the passband as a percentage as follows:

$$Q_{\tau }=max\left\{{\displaystyle \frac{{\tau }_{1max}-{\tau }_{1min}}{{\tau }_{1max}+{\tau }_{1min}}},{\displaystyle \frac{{\tau }_{2max}-{\tau }_{2min}}{{\tau }_{2max}+{\tau }_{2min}}}\right\}\times 100\%,$$

(14)

To measure the quality of the magnitude in the passband, we use:

$$Q_h={\displaystyle \frac{H_{max}-H_{min}}{H_{max}+H_{min}}}\times 100\%,$$

(15)

where $H_{max}$ and $H_{min}$ are the maximum and minimum magnitudes in the passband, respectively. We observe that when we design the filter, we improve the passband ripple and the group delays, but sometimes the stopband attenuation is not good enough. Based on that observation, we introduced a new quality factor for the stopband attenuation, which is measured in terms of the minimum variation of the passband ripple and the maximum variation of the stopband attenuation. To measure the quality of the magnitude response in the stopband, we use:

$$Q_s={\displaystyle \frac{H_{max}^s}{H_{min}-H_{max}^s}},$$

(16)

where $H_{max}^s$ is the maximum magnitude in the stopband region, and $H_{min}$ is the minimum magnitude in the passband. This quality factor, $Q_s$, is valid for $H_{min}>H_{max}^s$, i.e., negative values are excluded. Note that these quality factors are inverse measures, and therefore a good filter is one where these quality factors are small.

6. Experimental Results

This section is dedicated to implementing Equations (12) and (13) to optimize the design of 2D IIR filters with nearly linear phase. Our proposed approach will be evaluated against other methods described in the literature. Specifically, we will compare the designs obtained using the proposed method to the ones presented in [8]. Some of these examples are discussed in [52,53] as well. The desired frequency response will be defined in each example, and the group delays ${\tau }_1$ and ${\tau }_2$ along the ${\omega }_1$ and ${\omega }_2$ directions are the same, i.e., ${\tau }_1={\tau }_2=\tau$. In each of the designed examples, the starting values for ${\tau }_{10}$ and ${\tau }_{20}$ are set as the mean values of the passband group delays of the initial filter, and the stability margin ${\epsilon }_s$ is set to 0.02, unless otherwise mentioned in the design example. The MATLAB environment is used to solve the constrained optimization problems. The MATLAB Symbolic Math Toolbox and the interior-point method (IP) were used to implement the constrained optimization problem.

6.1. 2D Highpass Filter

In this example, it is required to design a 2D highpass filter with the ideal filter specifications shown in Figure 2, where ${\theta }_i=w_{i}T$. In addition to the given design specifications, we are including the transition band in our design, which aligns with the filter obtained in [8]. To assess the performance of our filter and the one proposed by [8], we have calculated the maximum transfer function magnitude in the transition band $H_{max}^t$ for both filters, as presented in

Table 1. Comparison results for highpass filter.

	Proposed Method	Method in [8]
Orders	(4,4)	(4,4)
$H_{min}$	0.437	0.262
$H_{max}$	1.338	1.066
$Q_h$	50.76	60.54
$H_{max}^t$	0.664	0.849
$H_{max}^s$	0.389	0.426
$Q_s$	8.104	$-2.598$
${\tau }_{min}$	0.962	0.009
${\tau }_{max}$	4.397	11.614
$Q_{\tau }$	64.09	99.84

.

In this example, the max. iterations = 140, $ϵ=1\times {10}^{-7}$, ${\epsilon }_s=0.1$, ${\Gamma }_{pb}=0.01$, ${\Gamma }_{tb}=1.5$, ${\Gamma }_{sb}=0.1$, $H_0=1$. We implement the proposed algorithm for various values of ${\Gamma }_g$ starting from $0.1$ to $4.0$. Using the quality factors defined in Section 5, the filter with ${\Gamma }_g=1.65$ is the most optimal filter among the filters obtained. Figure 3 shows the magnitude response of the designed filter using the proposed method, and the filter presented in [8]. Figure 4 and Figure 5 show the group delays ${\tau }_1$ and ${\tau }_2$ of the obtained 2D IIR highpass filter and the filter in [8], respectively. Table 1 compares the results of the highpass filter designed by the proposed algorithm with the highpass filter presented in [8].

The quality measure of the magnitude response in the passband, defined by Equation (15), of the obtained 2D IIR highpass filter is $Q_h=50.76$, where $H_{min}=0.437$ and $H_{max}=1.338$. In the same manner, the quality measure of the magnitude response in the passband of the filter presented in [8] is $Q_h=60.54$, where $H_{min}=0.262$ and $H_{max}=1.066$. The quality of the magnitude response in the stopband defined by Equation (16) is $Q_s=8.104$ for the filter obtained by the proposed method and for the filter presented in [8], $Q_s$ is negative, indicating too low stopband attenuation compared with the minimum gain in the passband. The quality of the group delay characteristic, defined by Equation (14), of the highpass filter designed by the proposed method is better than the one designed in [8]. It can be seen that $Q_{\tau }=64.09$, where ${\tau }_{min}=0.962$ and ${\tau }_{max}=4.397$ for the filter obtained, and $Q_{\tau }=99.84$, whereas ${\tau }_{min}=0.009$ and ${\tau }_{max}=11.614$ for the filter designed in [8].

6.2. 2D Lowpass Filter

The proposed method is used to design a 2D IIR lowpass filter with the ideal design specifications shown in Figure 6. The filter was presented in [8], and its order was selected as (4,4). The transition band constraint is included in this example, and $H_{max}^t$ is computed for both filters, where the results are shown in

Table 2. Comparison results for lowpass filter.

	Proposed Method	Method in [8]
Orders	(4,4)	(4,4)
$H_{min}$	0.843	0.642
$H_{max}$	1.120	1.069
$Q_h$	14.25	25.26
$H_{max}^t$	1.0079	1.020
$H_{max}^s$	0.403	0.446
$Q_s$	0.916	2.275
${\tau }_{min}$	2.217	1.738
${\tau }_{max}$	6.864	54.091
$Q_{\tau }$	47.650	93.774

.

This 2D lowpass filter is designed by the proposed method using the following values: Max. iterations = 140, $ϵ=1\times {10}^{-7}$, ${\epsilon }_s=0.1$, ${\Gamma }_{pb}=0.0001$, ${\Gamma }_{tb}=1$, ${\Gamma }_{sb}=0.01$, $H_0=1$, ${\tau }_1={\tau }_2=2$. The algorithm is implemented for different values of ${\Gamma }_g$, starting from 0.1 to 4. The filter with ${\Gamma }_g=2.35$ has the best quality in this example.

Figure 7 shows the impulse and magnitude responses of the filter obtained by using the proposed method and the filter presented in [8]. Figure 8 and Figure 9 show the group delays, ${\tau }_1$ and ${\tau }_2$, of the obtained 2D lowpass filter for ${\Gamma }_g=2.35$. The group delays, ${\tau }_1$ and ${\tau }_2$, of the 2D lowpass filter presented in [8] are shown in Figure 10 and Figure 11, respectively. Table 2 compares the results of the filter designed by the proposed method and the filter obtained in [8]. The quality measure of the magnitude response in the passband, defined by Equation (15) of the 2D lowpass filter obtained by the proposed method is $Q_h=14.25$, where $H_{min}=0.843$ and $H_{max}=1.120$, and for the filter presented in [8] is $Q_h=25.26$, where $H_{min}=0.642$ and $H_{max}=1.069$. The quality of the magnitude response in the stopband defined by Equation (16) is $Q_s=0.916$ for the filter designed by the proposed method, and $Q_s=2.275$ for the filter presented in [8]. The quality of the group delay characteristic, defined by Equation (14), for the 2D lowpass filter obtained by the proposed method is $Q_{\tau }=47.650$, where ${\tau }_{min}=2.217$ and ${\tau }_{max}=6.864$. The same quality measure for the 2D lowpass filter presented in [8] is found to be $Q_{\tau }=93.774$, where ${\tau }_{min}=1.738$ and ${\tau }_{max}=54.091$. We can see that the filter designed with the proposed method is better than the one designed in [8], especially with respect to the passband phase linearity.

6.3. 2D Bandpass Filter

In this example, a 2D IIR bandpass filter with the design specifications shown in Figure 12 is presented. It is also required that the order of this filter remain low, specifically (4,4) as in [8]. In addition to these design specifications and to assess the performance of our filter and the one proposed in [8], we have included the transition band constraint in our design. As a result, we calculated the maximum transfer function magnitude $H_{max}^t$ for both filters, as presented in

Table 3. Comparison results for bandpass filter.

	Proposed Method	Method in [8]
Orders	(4,4)	(4,4)
$H_{min}$	0.809	0.633
$H_{max}$	1.191	1.167
$Q_h$	19.100	29.667
$H_{max}^t$	0.855	0.798
$H_{max}^s$	0.324	0.268
$Q_s$	0.668	0.734
${\tau }_{min}$	1.822	1.828
${\tau }_{max}$	5.073	6.737
$Q_{\tau }$	47.150	57.315

. The algorithm is implemented with various values of ${\Gamma }_g$, starting from 0.1 to 4, and the resulting filter with ${\Gamma }_g=1.6$ has the best quality.

The magnitude response of the obtained 2D IIR bandpass filter and the filter designed in [8] are shown in Figure 13. The group delays ${\tau }_1$ and ${\tau }_2$ of the designed 2D bandpass filter are illustrated in Figure 14. Figure 15 shows the group delays of the same filter in the first quadrant. The group delays of the 2D bandpass filter presented in [8] are illustrated in Figure 16. The first quadrant plot of the group delays ${\tau }_1$ and ${\tau }_2$ of the same filter are shown in Figure 17. Table 3 compares the results of the bandpass filter designed by the proposed algorithm and the bandpass filter obtained in [8]. The quality measure of the magnitude response in the passband, defined by Equation (15), of the 2D IIR bandpass filter obtained by the proposed method is $Q_h=19.09$, where $H_{min}=0.809$ and $H_{max}=1.191$. In the same manner, the quality measure of the magnitude in the passband region of the 2D IIR bandpass filter presented in [8] is $Q_h=29.667$, where $H_{min}=0.633$ and $H_{max}=1.167$. The quality of the magnitude response in the stopband defined by Equation (16) is $Q_s=0.668$ for the filter obtained by the proposed method, and $Q_s=0.734$ for the filter presented in [8]. The quality of the group delay characteristic, defined by Equation (14), of the 2D IIR bandpass filter obtained by the proposed method is $Q_{\tau }=47.150$, where ${\tau }_{min}=1.822$ and ${\tau }_{max}=5.073$. The same measure is applied to the 2D IIR bandpass filter designed in [8], which gives $Q_{\tau }=57.315$, where ${\tau }_{min}=1.828$ and ${\tau }_{max}=6.737$.

In this example, the filter obtained with the proposed algorithm has a better passband quality factor, $Q_h$, stopband quality factor, $Q_s$, and group delay quality factor, $Q_{\tau }$, than the filter presented in [8].

7. Conclusions

A new framework for a nonlinear optimization design method for nearly linear-phase 2D IIR digital filters with separable denominators is presented. The method is used to solve the design problem, expressed as a nonlinear constrained optimization problem. In our implementation, we distinguish two kinds of constraints: soft and hard constraints. Meeting the soft constraints is not compulsory and, therefore, not decisive in rejecting the filter. The hard constraints, however, must be satisfied to accept the filter. The designed filters are evaluated with three quality factors, namely passband amplitude and group delay quality factors, and a new quality factor for the stopband magnitude response. compared with other 2D IIR filters designed with other methods, the results show that the framework produces very good results for different filter specifications.

Evaluating the resulted filters, this method improves the linearity of the phase in the passband with magnitude improvements, generally without significant deterioration in the magnitude response. The method is also flexible enough to be used after any other 2D IIR design algorithm by taking the resulting filter as the initial filter and running this method to obtain possible further improvements.

In future work, we intend to improve our nonlinear optimization method by directly optimizing the quality factors instead of the group delay deviations only.