Performance of Hammerstein Spline Adaptive Filtering Based on Fair Cost Function for Denoising Electrocardiogram Signals

Abstract

This paper proposes a simplified adaptive filtering approach using a Hammerstein function and the spline interpolation based on a Fair cost function for denoising electrocardiogram (ECG) signals. The use of linear filters in real-world applications has many limitations. Adaptive nonlinear filtering is a key development in tackling the challenge of discovering the specific characteristics of biomimetic systems for each person in order to eliminate unwanted signals. A biomimetic system refers to a system that mimics certain biological processes or characteristics of the human body, in this case, the individual features of a person’s cardiac signals (ECG). Here, the adaptive nonlinear filter is designed to cope with ECG variations and remove unwanted noise more effectively. The objective of this paper is to explore an individual biomedical filter based on adaptive nonlinear filtering for denoising the corrupted ECG signal. The Hammerstein spline adaptive filter (HSAF) architecture consists of two structural blocks: a nonlinear block connected to a linear one. In order to make a smooth convergence, the Fair cost function is introduced for convergence enhancement. The affine projection algorithm (APA) based on the Fair cost function is used to denoise the contaminated ECG signals, and also provides fast convergence. The MIT-BIH 12-lead database is used as the source of ECG biomedical signals contaminated by random noises modelled by Cauchy distribution. Experimental results show that the estimation error of the proposed HSAF–APA–Fair algorithm, based on the Fair cost function, can be reduced when compared with the conventional least mean square-based algorithm for denoising ECG signals.

1. Introduction

Electrocardiogram (ECG) signals have attracted significant interest in both medical and engineering fields [1]. ECG signals are a kind of biological signal corrupted by artefacts including patient movement and/or electrical interference. ECG signals consist of cardiac information that is essential to the diagnosis of heart diseases. With the advancement of technology, mathematical techniques can be adapted and applied to solve problems in nonlinear systems. Based on adaptive linear filtering, the finite impulse response (FIR) filter has been the conventional approach to eliminating corrupted ECG signals, including electrode motion artefacts and muscle artefacts [2].

Adaptive linear filtering techniques have been utilised, but have limitations for practical use since biomedical signals in real-world applications are generally dominated by nonlinear characteristics and nonlinear adaptive filtering. Thus, nonlinear adaptive filtering is required in order to solve the real problems of biological signals. Therefore, adaptive nonlinear filtering is a key approach to the challenge of discovering the specific characteristics of biomimetic systems for each person in order to eliminate unwanted signals. It will be shown that there are many research projects that present the application of nonlinear adaptive filtering in the field of signal processing, such as spline adaptive filtering (SAF) and Hammerstein spline adaptive filtering (HSAF) [3,4].

As stated in [3], a block-oriented model is one of the most basic nonlinear filtering techniques represented by the Hammerstein model, sometimes indicated as a nonlinear–linear model, which can be applied for biological nonlinear system identification. According to the adaptive approach, adaptive nonlinear system identification based on SAF has been achieved using a Hammerstein model comprising spline interpolation [4], which can determine the characteristic of biomedical signals in practice. As stated in the M-estimator algorithm, the adaptation of location M-estimators has also been undertaken [5,6]. As mentioned in the spline interpolation function [7], the curve of the spline basis matrix is a representation of a local interpolating approach and also uses continuous function to calculate the local parameter approximation from biomedical input signals. This approach is well-suited to capturing biological mechanisms and implementation in the field of biomedical system identification.

With assistance from the spline interpolation function and adaptive filtering, HSAF has been introduced to suppress both known and unknown disturbance sources. A common type of spline interpolation method used in many applications is the cubic spline, which can produces a smooth, natural-looking curve by applying low-degree polynomials to small data segments [8,9]. In [8], a cubic spline algorithm is used for fitting a flight path and smoothing the processing of the path planning of unmanned aerial vehicles using a bio-inspired spider–wasp algorithm. In [9], cubic spline interpolation was used for the collaborative path enhancement of multi-autonomous underwater vehicles’ collision avoidance. In [10], an improved HSAF was designed to extract the ocular noise from electroencephalogram signals in healthcare specialities. Meanwhile, HSAFs are nonlinear adaptive filters used in engineering research, such as the nonlinear system identification [11] against the impulsive noise environment [5], nonlinear acoustic echo-cancellation [12], and full-duplex communication at 2.4 GHz [13], and so on.

HSAF [4] comprises two main connected parts. The first part is nonlinear adaptive filtering, with the adaptive control point coefficients implemented in the form of a lookup table (LUT) that uses the coefficients of control point vectors defined by the spline interpolation function. The second part is adaptive linear FIR filtering with an optimisation scheme. Later research extended the design by adding a nonlinear filter structure to the original HSAF structure, which is called the Nonlinear–Linear–Nonlinear (NLN) model [3]. The experimental results show that it has a good ability to solve problems for simulation situations using real working conditions. The NLN structure is called a cascade Hammerstein spline adaptive filtering (CHSAF) structure. A previous study extended the design by adding a nonlinear filter structure to the original NLN-HSAF structure [3]. The experimental results show that it performed effectively in solving problems in simulation situations using real working conditions. The CHSAF model comprises the cascade of a memoryless nonlinear Hammerstein function, a linear Wiener filter, and a second memoryless nonlinear function, which can be considered as the cascade of a nonlinear Hammerstein system connected to a Wiener system and nonlinear system [3].

To reduce computational complexity while processing, the Fair cost function [14] has been modified for monitoring the estimated error and improving the robustness compared with the quadratic cost function in [14]. The affine projection algorithm (APA) is a stochastic algorithm that has fast convergence efficiency [15]. APA combines the advantages of normalised least mean square (NLMS) and recursive least square (RLS) algorithms, which have lower complexity and achieve faster convergence between RLS and NLMS methods in the adaptive filtering [16]. In [17], the authors improved this APA with robustness against the impulse noise. In [18], the improved APA performed with rapid convergence under non-Gaussian colour noise for sparse system identification. For fast convergence and robustness, a variable step-size technique is an important method that can support the achievement of coefficient optimisation and the inclusion of unconstrained criteria against noisy inputs and impulsive environments [19].

Therefore, HSAF applies the concepts derived from signal processing to biological signals. The goal is to apply mathematics to solve the interference present in biomedical information. The advantages of proposed HSAF-based scheme are its adaptive approach, versatile solutions for nonlinear systems, and ability to perform specific characteristic functions of biomimetic systems to defend against corrupted signals. This paper focuses on a nonlinear adaptive filter design using a HSAF structure with the help of APA based on the Fair cost function for achieving a fast mean square error convergence rate. We propose utilising this HSAF-based model, which has achieved notable success in many engineering fields, to optimise the Fair cost function for the more efficient denoising of ECG signals compared to conventional linear filtering based on a least mean square approach. The objective of this work is to explore the use of a biomedical filter based on HSAF for denoising contaminated ECG signals.

2. Hammerstein Spline Adaptive Filter

The architecture of the Hammerstein spline adaptive filter (HSAF) consists of two structural blocks: a nonlinear Hammerstein structure connected to a linear adaptive filtering, as shown in Figure 1. The nonlinear block, implemented as an adaptive lookup table (LUT), uses a spline function for interpolation with the first coefficient ${\mathbf{q}}_{i,n}$. The linear structure block is an adaptive linear finite impulse response (FIR) type of linear structure for the coefficient ${\mathbf{w}}_n$.

Figure 1. Proposed HSAF–APA–Fair architecture.

From Figure 1, the input signal ${\mathbf{x}}_n$ is the input vector of the adaptive LUT block, while the output signal ${\mathbf{s}}_n$ of the LUT block becomes the input of the linear adaptive FIR filter block

$$\begin{array}{cc}\hfill s_n& ={\mathbf{u}}_n^T\mathbf{C}{\mathbf{q}}_{i,n},\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill {\mathbf{s}}_n& ={\left[s_n{s}_{n-1}\dots s_{n-N+1}\right]}^T,\hfill \end{array}$$

(2)

$$\begin{array}{cc}\hfill i& =⌊{\displaystyle \frac{x_n}{\Delta x}}⌋+{\displaystyle \frac{Q-1}{2}},\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill u_n& ={\displaystyle \frac{x_n}{\Delta x}}-⌊{\displaystyle \frac{x_n}{\Delta x}}⌋,\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill {\mathbf{u}}_n& ={\left[u_n^3{u}_n^2{u}_{n}1\right]}^T,\hfill \end{array}$$

(5)

where $\Delta x$ is the coefficient between two consecutive control point coefficients and Q is the number of coefficients used. $\lfloor ·\rfloor$ is the floor operator and C is a spline basis matrix [4], with ${\mathbf{u}}_n$ being the coordinate between two points with the coefficient vector of the nonlinear filter.

Hence, the error $e_n$ at the output can be expressed by

$$\begin{array}{cc}\hfill e_n& =d_n-{\mathbf{w}}_n^T{\mathbf{s}}_n,\hfill \end{array}$$

(6)

where ${\mathbf{s}}_n$ is evaluated by the spline interpolation in (2).

Following [4], based on the minimised cost function $\mathbf{J}(\mathbf{q},\mathbf{w})$, we have

$$\begin{array}{c}\hfill \mathbf{J}(\mathbf{q},\mathbf{w})={\displaystyle \frac{1}{2}}\underset{\mathbf{q},\mathbf{w}}{min}{e}_n^2,\end{array}$$

(7)

where $e_n$ is given in (6).

As stated in the Hammerstein function, the least mean square (LMS) method is applied on adaptive weight coefficients ${\mathbf{q}}_{i,n}$

$$\begin{array}{cc}\hfill {\mathbf{q}}_{i,n+1}& ={\mathbf{q}}_{i,n}-{\mu }_q{\displaystyle \frac{\partial \mathbf{J}(\mathbf{q},\mathbf{w})}{\partial {\mathbf{q}}_{i,n}}},\hfill \end{array}$$

(8)

where ${\displaystyle \frac{\partial \mathbf{J}(\mathbf{q},\mathbf{w})}{\partial {\mathbf{q}}_{i,n}}}$ is given by

$$\begin{array}{c}\hfill {\displaystyle \frac{\partial \mathbf{J}(\mathbf{q},\mathbf{w})}{\partial {\mathbf{q}}_{i,n}}}={\displaystyle \frac{1}{2}}{\displaystyle \frac{\partial {(d_n-{\mathbf{w}}_n^T{\mathbf{s}}_n)}^2}{\partial {\mathbf{q}}_{i,n}}}=-{\mathbf{u}}_n^T\mathbf{C}{\mathbf{w}}_n{e}_n,\end{array}$$

(9)

where the Catmull–Rom (CR) spline basis matrix $\mathbf{C}$ that represents a local interpolating scheme [4] is

$$\begin{array}{c}\hfill \mathbf{C}={\displaystyle \frac{1}{2}}\left[\begin{array}{cccc}\hfill -1& \hfill 3& \hfill -3& \hfill 1\\ \hfill 2& \hfill -5& \hfill 4& \hfill -1\\ \hfill -1& \hfill 0& \hfill 1& \hfill 0\\ \hfill 0& \hfill 2& \hfill 0& \hfill 0\end{array}\right].\end{array}$$

(10)

Substituting (9) into (8), we have

$$\begin{array}{cc}\hfill {\mathbf{q}}_{i,n+1}& ={\mathbf{q}}_{i,n}+{\mu }_q{\mathbf{u}}_n^T\mathbf{C}{\mathbf{w}}_n.\hfill \end{array}$$

(11)

In a similar fashion, the adaptive linear coefficient ${\mathbf{w}}_n$ is defined by

$$\begin{array}{cc}\hfill {\mathbf{w}}_{n+1}& ={\mathbf{w}}_n+{\mu }_w{\displaystyle \frac{\partial \mathbf{J}(\mathbf{q},\mathbf{w})}{\partial {\mathbf{w}}_n}},\hfill \end{array}$$

(12)

where ${\displaystyle \frac{\partial \mathbf{J}(\mathbf{q},\mathbf{w})}{\partial \mathbf{w}\left(k\right)}}$ is expressed by

$$\begin{array}{c}\hfill {\displaystyle \frac{\partial \mathbf{J}(\mathbf{q},\mathbf{w})}{\partial {\mathbf{w}}_n}}={\displaystyle \frac{1}{2}}{\displaystyle \frac{\partial {(d_n-{\mathbf{w}}_n^T{\mathbf{s}}_n)}^2}{\partial {\mathbf{w}}_n}}=-{\mathbf{s}}_n{e}_n,\end{array}$$

(13)

By substituting (13) into (12), we arrive at

$$\begin{array}{cc}\hfill \therefore {\mathbf{w}}_{n+1}& ={\mathbf{w}}_n+{\mu }_w{\mathbf{s}}_n{e}_n,\hfill \end{array}$$

(14)

where ${\mu }_q$ and ${\mu }_w$ denote the learning rates for ${\mathbf{q}}_{i,n+1}$ and ${\mathbf{w}}_n$, respectively.

3. Proposed HSAF with Affine Projection Algorithm Based on Fair Cost Function

This section introduces an adaptive nonlinear HSAF with an affine projection algorithm (APA) based on the Fair cost function (APA–Fair), which is designed to provide a specific characteristic in biomimetic systems. Initially, the Fair cost function is briefly introduced. The HSAF–APA–Fair algorithm is then developed in detail based on the HSAF model.

Following [14], the minimised Fair cost function stands on the approximate coefficient ${\tilde{\mathbf{w}}}_n$ at symbol k, as follows:

$$\begin{array}{cc}\hfill {\tilde{\mathbf{J}}}_n& ={\alpha }^2\left\{{\displaystyle \frac{|{\tilde{e}}_n|}{\alpha }}-log\left(1+{\displaystyle \frac{|{\tilde{e}}_n|}{\alpha }}\right)\right\},\hfill \end{array}$$

(15)

where $\alpha >0$ is a threshold value.

The approximate a priori error ${\tilde{e}}_n$ at symbol n is related to the desired symbol $d_n$

$$\begin{array}{c}\hfill {\tilde{e}}_n=d_n-y_n=d_n-{\tilde{\mathbf{w}}}_{n-1}^T{\mathbf{s}}_n,\end{array}$$

(16)

where $y_n$ is the output signal, ${\tilde{\mathbf{w}}}_n$ is the estimated weight vector, and ${\mathbf{s}}_n$ is given in (2).

To minimise the criterion based on the Fair cost function in (15), we assume that the adaptive tap-weight vector ${\tilde{\mathbf{q}}}_{i,n}$ and ${\tilde{\mathbf{w}}}_n$ based on APA can be expressed as

$$\begin{array}{cc}\hfill {\tilde{\mathbf{q}}}_{i,n}& ={\tilde{\mathbf{q}}}_{i,n-1}+{\tilde{\mu }}_{q_n}{\displaystyle \frac{{\nabla }_{\mathbf{q}}{J}_n}{({\mathbf{X}}_n{\mathbf{X}}_n^T+\varrho \mathbf{I})}},\hfill \end{array}$$

(17)

$$\begin{array}{cc}\hfill {\tilde{\mathbf{w}}}_n& ={\tilde{\mathbf{w}}}_{n-1}+{\tilde{\mu }}_{w_n}{\displaystyle \frac{{\nabla }_{\mathbf{w}}{J}_n}{{\mathbf{S}}_n{\mathbf{S}}_n^T}},\hfill \end{array}$$

(18)

where ${\tilde{\mu }}_{q_n}$ and ${\tilde{\mu }}_{w_n}$ are adaptive step-size for ${\tilde{\mathbf{q}}}_{i,n}$ and ${\tilde{\mathbf{w}}}_n$, respectively. ${\nabla }_{\mathbf{w}}{J}_n$ and ${\nabla }_{\mathbf{q}}{J}_n$ denote the gradient matrices. ${\mathbf{X}}_n$ and ${\mathbf{S}}_n$ are given by

$$\begin{array}{cc}\hfill {\mathbf{X}}_n& =\left[{\mathbf{x}}_n{\mathbf{x}}_{n-1}\dots {\mathbf{x}}_{n-N+1}\right],\hfill \end{array}$$

(19)

$$\begin{array}{cc}\hfill {\mathbf{S}}_n& =\left[{\mathbf{s}}_n{\mathbf{s}}_{n-1}\dots {\mathbf{s}}_{n-N+1}\right].\hfill \end{array}$$

(20)

Then, the gradient ${\nabla }_{\mathbf{q}}{\tilde{\mathbf{J}}}_n$ results from differentiating the Fair cost function in (15) with respect to ${\tilde{\mathbf{q}}}_{i,n}$, which is given by

$$\begin{array}{cc}\hfill \therefore {\nabla }_{\mathbf{q}}{J}_n& ={\displaystyle \frac{\partial }{\partial {\tilde{\mathbf{q}}}_{i,n}}}\left\{{\alpha }^2\left({\displaystyle \frac{|{\tilde{e}}_n|}{\alpha }}-log(1+{\displaystyle \frac{|{\tilde{e}}_n|}{\alpha }})\right)\right\}\hfill \\ \hfill & ={\alpha }^2\left({\displaystyle \frac{{\mathbf{u}}_n^T\mathbf{C}{\tilde{\mathbf{w}}}_n}{\alpha }}-{\displaystyle \frac{{\mathbf{u}}_n^T\mathbf{C}{\tilde{\mathbf{w}}}_n}{\alpha (1+\frac{|{\tilde{e}}_n|}{\alpha })}}\right)={\displaystyle \frac{\alpha {\mathbf{u}}_n^T\mathbf{C}{\tilde{\mathbf{w}}}_n{\tilde{e}}_n}{\alpha +|{\tilde{e}}_n|}}.\hfill \end{array}$$

(21)

Moreover, the gradient ${\nabla }_{\mathbf{w}}{J}_n$ is obtained by differentiating the Fair cost function in (15) with respect to ${\tilde{\mathbf{w}}}_n$, which can be derived by

$$\begin{array}{cc}\hfill \therefore {\nabla }_{\mathbf{w}}{J}_n& ={\displaystyle \frac{\partial }{\partial {\tilde{\mathbf{w}}}_n}}\left\{{\alpha }^2\left({\displaystyle \frac{|{\tilde{e}}_n|}{\alpha }}-log(1+{\displaystyle \frac{|{\tilde{e}}_n|}{\alpha }})\right)\right\}\hfill \\ \hfill & ={\alpha }^2\left({\displaystyle \frac{{\mathbf{s}}_n}{\alpha }}-{\displaystyle \frac{{\mathbf{s}}_n}{\alpha (1+\frac{|{\tilde{e}}_n|}{\alpha })}}\right)={\displaystyle \frac{\alpha {\mathbf{s}}_n{\tilde{e}}_n}{\alpha +|{\tilde{e}}_n|}}.\hfill \end{array}$$

(22)

By replacing (21) into (17), the adaptive control points ${\tilde{\mathbf{q}}}_{i,n}$ can be expressed as

$$\begin{array}{cc}\hfill {\tilde{\mathbf{q}}}_{i,n}& ={\tilde{\mathbf{q}}}_{i,n-1}+{\displaystyle \frac{{\tilde{\mu }}_{q_n}}{{\mathbf{X}}_n{\mathbf{X}}_n^T}}\left\{{\displaystyle \frac{\alpha {\mathbf{u}}_n^T\mathbf{C}{\tilde{\mathbf{w}}}_n{\tilde{e}}_n}{\alpha + |{\tilde{e}}_n|}}\right\},\hfill \end{array}$$

(23)

where ${\tilde{e}}_n$ is given in (16). Note that the input of the nonlinear block ${\mathbf{X}}_n$ is related to the adaptive control points ${\tilde{\mathbf{q}}}_{i,n}$.

By substituting (22) into (18), the adaptive coefficient ${\tilde{\mathbf{w}}}_n$ can be obtained by

$$\begin{array}{cc}\hfill {\tilde{\mathbf{w}}}_n& ={\tilde{\mathbf{w}}}_{n-1}+{\displaystyle \frac{{\tilde{\mu }}_{w_n}}{{\mathbf{S}}_n{\mathbf{S}}_n^T}}\left\{{\displaystyle \frac{\alpha {\mathbf{s}}_n{\tilde{e}}_n}{\alpha + |{\tilde{e}}_n|}}\right\},\hfill \end{array}$$

(24)

where ${\mathbf{s}}_n$ is given in (2). It is noticed that the input of the linear block ${\mathbf{S}}_n$ is related to the adaptive coefficient ${\tilde{\mathbf{w}}}_n$.

Following [20], adaptive step sizes ${\tilde{\mu }}_{q_n}$ and ${\tilde{\mu }}_{w_n}$ for ${\tilde{\mathbf{q}}}_{i,n}$ and ${\tilde{\mathbf{w}}}_n$ can be expressed with the help of a gradient vector, as follows:

$$\begin{array}{cc}\hfill {\tilde{\mu }}_{q_n}& ={\beta }_q{\tilde{\mu }}_{q_{n-1}}+{\rho }_q{\mathbf{g}}_{q_{n-1}}^T{\mathbf{g}}_{q_n}{\zeta }_n^2,\hfill \end{array}$$

(25)

$$\begin{array}{cc}\hfill {\tilde{\mu }}_{w_n}& ={\beta }_w{\tilde{\mu }}_{w_{n-1}}+{\rho }_w{\mathbf{g}}_{w_{n-1}}^T{\mathbf{g}}_{w_n}{\zeta }_n^2,\hfill \end{array}$$

(26)

$$\begin{array}{cc}\hfill {\zeta }_n& =\sigma {\zeta }_{n-1}+(1-\sigma ){\tilde{e}}_n^2,\hfill \end{array}$$

(27)

where ${\beta }_q$, ${\beta }_w$, and ${\rho }_q$, ${\rho }_w$ are the smooth parameters for smooth convergence.

So, the gradient vectors ${\mathbf{g}}_{q_n}$ and ${\mathbf{g}}_{w_n}$, based on gradient weighted average algorithm [20], are given by

$$\begin{array}{cc}\hfill {\mathbf{g}}_{q_n}& =\psi {\mathbf{g}}_{q_{n-1}}+(1-\psi ){\mathbf{u}}_n^T\mathbf{C}{\tilde{\mathbf{w}}}_n{\tilde{e}}_n,\hfill \end{array}$$

(28)

$$\begin{array}{cc}\hfill {\mathbf{g}}_{w_n}& =\psi {\mathbf{g}}_{w_{n-1}}+(1-\psi ){\mathbf{s}}_n{\tilde{e}}_n,\hfill \end{array}$$

(29)

where $\psi <1$ and ${\tilde{e}}_n$ is defined in (16).

A summary of the proposed HSAF-APA based on Fair cost function (HSAF-APA-Fair) algorithm with adaptive step-size algorithm is introduced in Algorithm 1.

Algorithm 1 Summary of proposed HSAF–APA–Fair algorithm with adaptive step-size approach

Initial value: $\tilde{\mathbf{w}}\left(0\right)={[10\cdots 0]}^T$ ${\tilde{\mathbf{q}}}_{i,n}\left(0\right)={[10\cdots 0]}^T$ and $\varrho =1\times {10}^{-7}$ Fixed parameter: $0<\alpha <1$ For $n=1,2,\dots ,N$ To calculate ${\mathbf{s}}_n$ as $$\begin{array}{cc}\hfill s_n& ={\mathbf{u}}_{n,1}^T\mathrm{C}{\tilde{\mathbf{q}}}_{i,n},\hfill \\ \hfill {\mathbf{s}}_n& ={\left[s_n{s}_{n-1}\dots s_{n-N+1}\right]}^T\hfill \end{array}$$ To compute ${\mathbf{u}}_n$ and i as $$\begin{array}{cc}\hfill {\mathbf{u}}_n& ={\left[u_n^3{u}_n^2{u}_{n}1\right]}^T\hfill \\ \hfill u_n& ={\displaystyle \frac{x_n}{\Delta x}}-⌊{\displaystyle \frac{x_n}{\Delta x}}⌋\hfill \\ \hfill i& =⌊{\displaystyle \frac{x_n}{\Delta x}}⌋+{\displaystyle \frac{Q-1}{2}}\hfill \end{array}$$ To compute ${\tilde{e}}_n$ as $$\begin{array}{c}\hfill {\tilde{e}}_n=d_n-{\tilde{\mathbf{w}}}_{n-1}^T{\mathbf{s}}_n\end{array}$$ To determine ${\tilde{\mathbf{q}}}_{i,n}$ as: $$\begin{array}{cc}\hfill {\tilde{\mathbf{q}}}_{i,n}& ={\tilde{\mathbf{q}}}_{i,n-1}+{\displaystyle \frac{{\tilde{\mu }}_{q_n}}{{\mathbf{X}}_n{\mathbf{X}}_n^T}}\left\{{\displaystyle \frac{\alpha {\mathbf{u}}_n^T\mathbf{C}{\tilde{\mathbf{w}}}_n{\tilde{e}}_n}{\alpha + |{\tilde{e}}_n|}}\right\}\hfill \\ \hfill {\mathbf{X}}_n& =\left[{\mathbf{x}}_n{\mathbf{x}}_{n-1}\dots {\mathbf{x}}_{n-N+1}\right]\hfill \end{array}$$ To obtain ${\tilde{\mu }}_{q_n}$ as $$\begin{array}{cc}\hfill {\tilde{\mu }}_{q_n}& ={\beta }_q{\tilde{\mu }}_{q_{n-1}}+{\rho }_q{\mathbf{g}}_{q_{n-1}}^T{\mathbf{g}}_{q_n}{\zeta }_n^2\hfill \\ \hfill {\mathbf{g}}_{q_n}& =\psi {\mathbf{g}}_{q_{n-1}}+(1-\psi ){\mathbf{u}}_n^T\mathbf{C}{\tilde{\mathbf{w}}}_n{\tilde{e}}_n\hfill \\ \hfill {\zeta }_n& =\sigma {\zeta }_{n-1}+(1-\sigma ){\tilde{e}}_n^2\hfill \end{array}$$ To calculate ${\tilde{\mathbf{w}}}_n$ $$\begin{array}{cc}\hfill {\tilde{\mathbf{w}}}_n& ={\tilde{\mathbf{w}}}_{n-1}+{\displaystyle \frac{{\tilde{\mu }}_{w_n}}{{\mathbf{S}}_n{\mathbf{S}}_n^T}}\left\{{\displaystyle \frac{\alpha {\mathbf{s}}_n{\tilde{e}}_n}{\alpha + |{\tilde{e}}_n|}}\right\}\hfill \\ \hfill {\mathbf{S}}_n& =\left[{\mathbf{s}}_n{\mathbf{s}}_{n-1}\dots {\mathbf{s}}_{n-N+1}\right]\hfill \end{array}$$ To obtain ${\tilde{\mu }}_{w_n}$ as $$\begin{array}{cc}\hfill {\tilde{\mu }}_{w_n}& ={\beta }_w{\tilde{\mu }}_{w_{n-1}}+{\rho }_w{\mathbf{g}}_{w_{n-1}}^T{\mathbf{g}}_{w_n}{\zeta }_n^2\hfill \\ \hfill {\mathbf{g}}_{w_n}& =\psi {\mathbf{g}}_{w_{n-1}}+(1-\psi ){\mathbf{s}}_n{\tilde{e}}_n\hfill \end{array}$$ end

4. Performance Analysis of Learning Rate

An analysis of the convergence properties of the proposed step-size parameters is conducted in virtue of a posteriori estimate error and Taylor series expansion in order to ensure the steady-state condition.

We assume a posteriori error ${\tilde{e}}_{p_n}$ by

$$\begin{array}{c}\hfill {\tilde{e}}_{p_n}=d_n-{\tilde{\mathbf{w}}}_{n+1}^T{\mathbf{s}}_n={\tilde{e}}_{{\mathbf{min}}_n}-\delta {\tilde{\mathbf{w}}}_n^T{\mathbf{s}}_n.\end{array}$$

(30)

where the difference in previous and present ${\delta }_{{\tilde{\mathbf{w}}}_n}$ is given by ${\tilde{\mathbf{w}}}_n$, and can be expressed by

$$\begin{array}{c}\hfill \delta {\tilde{\mathbf{w}}}_n={\tilde{\mathbf{w}}}_{n+1}-{\tilde{\mathbf{w}}}_n={\displaystyle \frac{{\tilde{\mu }}_{w_n}}{{\mathbf{S}}_n{\mathbf{S}}_n^T}}\left\{{\displaystyle \frac{\alpha {\mathbf{s}}_n{\tilde{e}}_n}{\alpha + |{\tilde{e}}_n|}}\right\}.\end{array}$$

(31)

By substituting (31) into (30), a posteriori estimate error can be obtained by

$$\begin{array}{c}\hfill {\tilde{e}}_{p_n}={\tilde{e}}_{{\mathbf{min}}_n}-{\displaystyle \frac{{\tilde{\mu }}_{w_n}}{{\mathbf{S}}_n{\mathbf{S}}_n^T}}\left\{{\displaystyle \frac{\alpha {\mathbf{s}}_n{\mathbf{s}}_n^T{\tilde{e}}_n}{\alpha + |{\tilde{e}}_n|}}\right\}.\end{array}$$

(32)

We assume that $|{\tilde{e}}_{p_n}|$≃$|{\tilde{e}}_{{\mathbf{min}}_n}|$ at steady state. In order to demonstrate the range of ${\tilde{\mu }}_{{\tilde{\mathbf{w}}}_n}$, we take the norm of both sides of (32), which can be given by

$$\begin{array}{c}\hfill \therefore 0<{\tilde{\mu }}_{w_n}<{\displaystyle \frac{\left({\mathbf{S}}_n{\mathbf{S}}_n^T\right)·(\alpha +|{\tilde{e}}_n\left|\right)}{\alpha {\mathbf{s}}_n{\mathbf{s}}_n^T}}.\end{array}$$

(33)

By means of Taylor series expansion, a posteriori estimate error with respect to the ${\tilde{\mathbf{q}}}_{i,n}$ can be given by

$$\begin{array}{c}\hfill {\tilde{e}\prime }_{p_{n+1}}={\tilde{e}\prime }_{p_n}+{\displaystyle \frac{\partial {\tilde{e}\prime }_{p_n}}{\partial {\tilde{\mathbf{q}}}_{i,n}}}·\delta {\tilde{\mathbf{q}}}_{i,n},\end{array}$$

(34)

where the difference in previous and present $\delta {\tilde{\mathbf{q}}}_{i,n}$ can be expressed as

$$\begin{array}{c}\hfill \delta {\tilde{\mathbf{q}}}_{i,n}={\tilde{\mathbf{q}}}_{i,n+1}-{\tilde{\mathbf{q}}}_{i,n}={\displaystyle \frac{{\tilde{\mu }}_{q_n}}{{\mathbf{X}}_n{\mathbf{X}}_n^T}}\left\{{\displaystyle \frac{\alpha \left({\mathbf{u}}_n^T\mathbf{C}{\tilde{\mathbf{w}}}_n\right){\tilde{e}}_n}{\alpha + |{\tilde{e}}_n|}}\right\}.\end{array}$$

(35)

We demonstrate the a posteriori estimate ${\tilde{e}\prime }_{p_n}$ by

$$\begin{array}{c}\hfill {\tilde{e}\prime }_{p_n}=d_n-{\tilde{\mathbf{w}}}_{n+1}^T{\mathbf{u}}_n\mathbf{C}{\tilde{\mathbf{q}}}_{i,n},\end{array}$$

(36)

and to take the derivative on ${\tilde{e}\prime }_{p_n}$, with respect to ${\tilde{\mathbf{q}}}_{i,n}$, by

$$\begin{array}{c}\hfill {\displaystyle \frac{\partial {\tilde{e}\prime }_{p_n}}{\partial {\tilde{\mathbf{q}}}_{i,n}}}={\displaystyle \frac{\partial }{\partial {\tilde{\mathbf{q}}}_{i,n}}}\left\{d_n-{\tilde{\mathbf{w}}}_{n+1}^T{\mathbf{u}}_n\mathbf{C}{\tilde{\mathbf{q}}}_{i,n}\right\}=-{\tilde{\mathbf{w}}}_{n+1}^T\mathbf{C}{\mathbf{u}}_n.\end{array}$$

(37)

By substituting (35) and (37) into (34), the a posteriori estimate error ${\tilde{e}\prime }_{p_n}$ can be expressed as

$$\begin{array}{c}\hfill {\tilde{e}\prime }_{p_{n+1}}={\tilde{e}\prime }_{p_n}-{\displaystyle \frac{\alpha {\tilde{\mu }}_{q_n}\left({\mathbf{u}}_n^T\mathbf{C}{\tilde{\mathbf{w}}}_n\right)·{\left({\mathbf{u}}_n^T\mathbf{C}{\tilde{\mathbf{w}}}_n\right)}^T{\tilde{e}}_n}{\left({\mathbf{X}}_n{\mathbf{X}}_n^T\right)·(\alpha +|{\tilde{e}}_n\left|\right)}}.\end{array}$$

(38)

In a similar way, we obtain the range of ${\tilde{\mu }}_{q_n}$ by taking the norm on both sides of (38) as

$$\begin{array}{c}\hfill \therefore 0<{\tilde{\mu }}_{q_n}<{\displaystyle \frac{\left({\mathbf{X}}_n{\mathbf{X}}_n^T\right)·(\alpha +|{\tilde{e}}_n\left|\right)}{\alpha \left({\mathbf{u}}_n^T\mathbf{C}{\tilde{\mathbf{w}}}_n\right)·{\left({\mathbf{u}}_n^T\mathbf{C}{\tilde{\mathbf{w}}}_n\right)}^T}},\end{array}$$

(39)

where we assume that all constants are positive and $|{\tilde{e}\prime }_{p_n}|$≃$|{\tilde{e}\prime }_{n+1}|$ at steady state.

Note that if the learning rates of ${\tilde{\mu }}_{w_n}$ and ${\tilde{\mu }}_{q_n}$ are designated as mentioned in (33) and (39), then the proposed method can be confirmed to converge at its own optimum value. These bounds in (33) and (39) are helpful to certify the convergence of the proposed approach.

5. Simulation Results

The efficiency of the proposed HSAF–APA–Fair algorithm was investigated using mean squared error (MSE) to assess the performance of the model and its applicability for ECG signal denoising.

5.1. Experimental Results of Proposed HSAF–APA–Fair Model

Random processes were simulated in the computer experiments. The performance of the proposed HSAF–APA–Fair algorithm was evaluated by comparing it to the HSAF-LMS [4] across 100 Monte Carlo trials using 7000 samples.

The input signal ($x_n$) is generated by

$$\begin{array}{c}\hfill x_n=\vartheta ·x_{n-1}+{\eta }_n=\vartheta ·x_{n-1}+\sqrt{1-{\vartheta }^2}{\xi }_n,\end{array}$$

(40)

where ${\eta }_n$ is the random noise, ${\xi }_n$ is a zero-mean white Gaussian noise with unitary variance, and $\vartheta$ is set to [0.01, 0.99].

In the system identification, an unknown Wiener system comprising a linear component is identified as [3]:

$$\begin{array}{c}\hfill {\mathbf{w}}_0=[0.6,-0.4,0.25,-0.15,0.1].\end{array}$$

Following [5], a nonlinear memoryless target function is implemented by a 23-point length LUT ${\mathbf{q}}_0$, which is interpolated by a uniform third-degree spline with an interval sampling $\Delta x=0.2$ as

$$\begin{array}{cc}\hfill {\mathbf{q}}_0& =\{-2.2,-2,-1.8,\dots ,-1.0,-0.8,-0.91,0.42,\hfill \\ & -0.01,-0.1,0.1,-0.15,0.58,1.2,1.0,1.2,\dots ,2.0,2.2\}\hfill \end{array}$$

The initial parameters of HSAF–APA–Fair model are $\alpha =0.01$; ${\mu }_w=3.75\times {10}^{-2}$; ${\mu }_q=3.55\times {10}^{-2}$; ${\rho }_q=2.95\times {10}^{-3}$; ${\rho }_w=2.75\times {10}^{-3}$; $\psi =0.975$; and a signal to noise ratio of $SNR=25$ dB. The length of the tap (M) is 7 for all algorithms.

The comparison of model performance between the proposed HSAF–APA–Fair method and other nonlinear models [21,22] is proved. Volterra series [21] and adaptive Hammerstein nonlinear system model by Jeraj and Mathews [22] are the representations of other nonlinear models. Volterra series [21] is a simple and powerful approach for analysing a systematic and characteristic model for a nonlinear system. Jeraj and Mathews [22] evaluated the performance of an adaptive nonlinear system by using a Hammerstein model consisting of static nonlinearity with adaptive linear filtering.

The adapted nonlinearity of the proposed HSAF–APA–Fair model compared with the traditional HSAF-LMS [4], Volterra [21] and Jeraj & Mathews [22] is shown in Figure 2. It can be clearly seen that the averaged results of adapted nonlinearity for Volterra [21] and Jeraj and Mathews [22] do not overlap with the nonlinear memoryless target ${\mathbf{q}}_0$ when the traditional HSAF-LMS scheme is non-adjustable while updating at 7000 samples over 100 trials. Note that the result of the proposed HSAF–APA–Fair obtained by a spline interpolation function employed in adaptive control points ${\tilde{\mathbf{q}}}_{i,n}$ overlies the target in order to correspond with other nonlinear algorithms.

Figure 2. Target and adapted nonlinearities of HSAF-APA-Fair compared with HSAF-LMS [4], Volterra [21] and Jeraj & Mathews [22] with $\vartheta =0.15$.

We determine the mean square error $\mathbb{MSE}\left(n\right)$ that is given in dB by

$$\begin{array}{c}\hfill \mathbb{MSE}\left(n\right)=10log\left[e_n^2\right]=10log{\left[d_n-{\mathbf{w}}_{n-1}^T{\mathbf{s}}_n\right]}^2.\end{array}$$

(41)

Figure 3 shows the performance of different approaches in terms of the MSE calculated by (41), where MSE curves were obtained from a test of 7000 samples with 100 trials. The comparison of MSE with $\vartheta =0.15$ is provided and shows the MSE curves of the proposed HSAF–APA–Fair model and HSAF-LMS [4] model with $\vartheta =0.15$ based on (40). It is clear that the MSE curves of the proposed HSAF–APA–Fair algorithm provides slower convergence compared to the traditional HSAF-LMS algorithm at steady state, while reaching the value of noise level.

Figure 3. MSE curves of proposed HSAF-APA-Fair and HSAF-LMS [4], Volterra [21] and Jeraj & Mathews [22] where $\vartheta =0.15$.

In addition, the learning rates of ${\mu }_{{\tilde{\mathbf{q}}}_n}$ and ${\mu }_{{\tilde{\mathbf{w}}}_n}$ are the crucial hyperparameters for controlling the coefficients ${\tilde{\mathbf{w}}}_n$ and ${\tilde{\mathbf{q}}}_{i,n}$ of the proposed HSAF–APA–Fair model with respect to the estimated error during updating. Figure 4 shows the learning rate of ${\mu }_{q_n}$ of adaptive control points ${\mathbf{q}}_{i,n}$, with the different initial values of ${\mu }_q\left(0\right)$. Figure 5 shows the learning rate of ${\mu }_{w_n}$ of the adaptive coefficient ${\mathbf{w}}_n$ with the different initial values of ${\mu }_q\left(0\right)$. The proposed learning rate models of step-size parameters ${\mu }_{q_n}$ and ${\mu }_{w_n}$ begin to display slight fluctuations in learning while processing after several hundreds of samples. However, both adaptive step-size models still maintain a reasonable convergence rate of MSE performance. It is noticed that the learning rate of both ${\mu }_{q_n}$ and ${\mu }_{w_n}$ can converge effectively to a steady-state environment, starting with the 1000-fold change in different initial values of each step-size.

Figure 4. Learning rate trajectories of step-size ${\mu }_{q_n}$ of adaptive control points ${\mathbf{q}}_{i,n}$ at different initial values of ${\mu }_q\left(0\right)$ and ${\mu }_w\left(0\right)$.

Figure 5. Learning rate trajectories of step-size ${\mu }_{w_n}$ of adaptive coefficient ${\mathbf{w}}_n$ at different initial values of ${\mu }_q\left(0\right)$ and ${\mu }_w\left(0\right)$.

Regarding the relationship between the MSE curves in Figure 3 and the learning rate of step-size parameters in Figure 4 and Figure 5, if the learning rate is too high at starting point, it ensures that the MSE curve of the proposed model converges quickly to find the optimal value. At steady state, we found that the learning rates adaptively were of lower values when updating the model.

5.2. Denoising ECG Signal

For this experiment, a dataset of ECG signals was provided by the MIT-BIH database [23,24,25]. The ECG input signal ($x_{{ECG}_n}$) was generated in a random process across 100 Monte Carlo trials using 7000 samples as follows:

$$\begin{array}{cc}\hfill x_{{ECG}_n}& =\vartheta ·x_{{ECG}_{n-1}}+\sqrt{1-{\vartheta }^2}{\xi }_n.\end{array}$$

(42)

Based on statistical inference [26], the Cauchy distribution function (${\eta }_{{\mathrm{Cauchy}}_n}$) is a representation of the random noise that appeared in the biomedical signal:

$$\begin{array}{cc}\hfill {\eta }_{{\mathrm{Cauchy}}_n}& ={\displaystyle \frac{\lambda }{\pi ·(x_{{ECG}_n}^2+\lambda )}},\hfill \end{array}$$

(43)

where $\lambda$ is a scale parameter at 0.01.

Following [26], the corrupted ECG input signal ($d_{{\eta }_n}$) was contaminated by Gaussian Cauchy distribution:

$$\begin{array}{c}\hfill d_{{\eta }_n}=(1-\lambda )·d_{{\eta }_{n-1}}+\lambda ·{\eta }_{{\mathrm{Cauchy}}_n}.\end{array}$$

(44)

Figure 6 shows the comparison between original ECG, the ECG corrupted by Gaussian Cauchy noise, and the ECG signal denoised by proposed HSAF–APA–Fair model. It can be observed that the corrupted ECG signal in Figure 6b was disturbed by Cauchy random noise. The denoised ECG signal in Figure 6c is very similar to the original ECG shown in Figure 6a after filtering using the proposed HSAF–APA–Fair method in order to eliminate the Cauchy undesired noise.

Figure 6. Denoised ECG signal corrupted by Gaussian Cauchy distribution function: (a) ECG signal; (b) ECG corrupted by Cauchy noise; and (c) denoised ECG by proposed HSAF–APA–Fair model.

Figure 7 shows the performance of proposed HSAF–APA–Fair algorithm with the ECG input corrupted in Figure 6b in terms of MSE, where the MSE curves were obtained from a test of 7000 samples with 100 trials. The comparison of MSE with $\vartheta =0.15,0.90$ is provided, and shows the the MSE curves of the proposed HSAF–APA–Fair approach can converge at the steady state.

Figure 7. MSE curves of proposed HSAF–APA–Fair algorithm with the ECG input corrupted in Figure 6b at different $\vartheta =0.15,0.90$.

6. Discussion

The proposed HSAF–APA–Fair model achieves faster convergence and greater robustness to random processes compared to the traditional HSAF-LMS model. The proposed HSAF–APA–Fair model with adaptive learning rates outperforms the conventional HSAF-LMS and other nonlinear models. The shape of the adapted nonlinearity function of the proposed model can be smoothly transformed during processing. The performance of the proposed model can be optimised by step-size parameters, which were comprehensively implemented. The tuning process using an adaptive step-size algorithm aimed to enhance the convergence properties of the proposed model. This suggests that the proposed model adapts effectively after learning from a Gaussian random process. Its ability to optimise the corrupted ECG signal was tested against corruption with Cauchy noise. Despite convergence, the proposed model continues to demonstrate stable performance, capturing the underlying biomedical patterns in ECG signal and nonlinear system identification, making the proposed model a suitable option for real-world application.

7. Conclusions

This paper evaluates the effectiveness of the proposed HSAF–APA–Fair model, which is based on the Fair cost function for a general case of nonlinear system identification. The HSAF-based architecture incorporates a memoryless function that is adjusted during the learning process, with spline control points being automatically regulated through a gradient-based method from the APA approach. By leveraging the gradient vector for adaptive learning rate models, the step-size tuning parameters are proved. The bounds of the step-size parameters are verified with the assistance of Taylor series expansion. A systematic analysis of the learning rate of the step-size parameters was conducted to prove that the model can converge effectively to steady-state environment starting with the 1000-fold change in different initial values of each step-size parameter. Several experiments on the Gaussian random process and real data of corrupted ECG signals demonstrated the performance of the proposed HSAF–APA–Fair algorithm with regard to MSE evaluation. Simulation results show that the proposed HSAF–APA–Fair algorithm provides greater robustness and outperforms the conventional HSAF-LMS algorithm and other nonlinear systems in the tasks of system identification and ECG signal denoising.