# Block-Adaptive Rényi Entropy-Based Denoising for Non-Stationary Signals

^{1}

^{2}

^{3}

^{4}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Materials and Methods

#### 2.1. Background Theory

#### 2.2. A LRE-Based Criterion for Useful Class Selection

- 1.
- The first class ${\rho}_{1}(n,m)$, consisting of the smallest coefficients obtained from expression Exp. (8), associated to the LRE function ${H}_{1}\left(p\right)$, is discarded as noise.
- 2.
- Starting from the second class, ${\rho}_{2}(n,m)$, all consecutive classes for which for at least one instant p ${H}_{k}\left(p\right)\ge {H}_{1}\left(p\right)$ holds are classified as noise, and thus discarded.
- 3.
- Considering that in the previous steps a total of r classes have been discarded, for the remaining classes, ${\rho}_{k}\left(p\right),r<k\le K$ we introduce a threshold i as follows. We first introduce a closeness value ${d}_{k}$ on the remaining classes by$${d}_{k}=min\left\{\right|{H}_{k}\left(p\right)-{H}_{k+1}\left(p\right)|,\phantom{\rule{4.pt}{0ex}}\mathrm{across}\phantom{\rule{4.pt}{0ex}}\mathrm{all}\phantom{\rule{4.pt}{0ex}}p\}.$$We now define i as the least index of ${d}_{i}$ such that ${d}_{k}<{d}_{i}$ for $k<i$ and ${d}_{k}\ge {d}_{i}$ for $k>i$.
- 4.
- The remaining classes, with indexes $k=i,...,K$, are added up to obtain the useful information content of the signal, i.e., the signal components.

#### 2.3. A Short-Term LRE Approach for Variable Noise Intensity Conditions

- 1.
- Initially t is set to one.
- 2.
- The first class ${\rho}_{1,t}(n,f)$, consisting of the smallest coefficients obtained from Exp. (15) and associated with the LRE function ${H}_{1,t}\left(p\right)$, is discarded as noise.
- 3.
- Starting from the second class, ${\rho}_{2,t}(n,f)$, all consecutive classes for which for at least one instant p holds ${H}_{k,t}\left(p\right)\ge {H}_{1,t}\left(p\right)$ are discarded as noise.
- 4.
- Assuming that in steps 2 and 3 a total of ${r}_{t}$ classes has been discarded, for the remaining classes ${\rho}_{k,t}\left(p\right)$, ${r}_{t}<k\le K$ we introduce a threshold ${i}_{t}$ as follows. We first introduce a closeness value ${d}_{k,t}$ on the remaining classes by$$\begin{array}{c}\hfill {d}_{k,t}=min\left\{\right|{H}_{k,t}\left(p\right)-{H}_{k+1,t}\left(p\right)|,\phantom{\rule{4.pt}{0ex}}\mathrm{across}\phantom{\rule{4.pt}{0ex}}\mathrm{all}\phantom{\rule{4.pt}{0ex}}p\}.\end{array}$$We now define ${i}_{t}$ as the least index of ${d}_{{i}_{t},t}$ such that ${d}_{k,t}<{d}_{i,t}$ for $k<{i}_{t}$ and ${d}_{k,t}\ge {d}_{i,t}$ for $k>{i}_{t}$.
- 5.
- The classes with indices $k={i}_{t},\dots ,K$ are summed together to obtain the TFD building block with the useful information content of the signal at block t,$$\begin{array}{c}\hfill {\mathrm{UI}}_{t}(n,m)=\sum _{k={i}_{t}}^{K}{\rho}_{k,t}(n,m).\end{array}$$
- 6.
- t is incremented by one and the procedure is repeated from step 2, until $t=N/\Delta t$.
- 7.
- The total useful information of the signal is obtained by summing the extracted information over all the building blocks as$$\begin{array}{c}\hfill \mathrm{UI}=\sum _{t=1}^{N/\Delta t}{\mathrm{UI}}_{t}.\end{array}$$

## 3. Results

#### 3.1. Real Data

#### 3.2. Simulation Results

## 4. Discussion

## 5. Conclusions

## Author Contributions

## Funding

## Data Availability Statement

## Conflicts of Interest

## Abbreviations

Abbreviations | |

AWGN | Additive white Gaussian noise |

TFD | Time-frequency distribution |

DFT | Discrete Fourier transform |

TF | Time-frequency |

LRE | Local Rényi entropy |

SNR | Signal-to-noise ratio |

Nomenclature | |

${A}_{l}$ | instantaneous amplitude of the l-th component |

$\mathbf{C}$ | set of observations derived from a TFD |

${\mathbf{C}}_{\mathbf{t}}$ | set of observations derived from one TFD building block |

${C}_{I}$ | subset of TFD’s observations derived from useful information |

${C}_{k}$ | k-th set of partitioned TFD’s observations |

${C}_{k,t}$ | k-th set of partitioned observations from one TFD building block |

${d}_{k}$ | closeness value between LRE estimates ${H}_{k}$ and ${H}_{k+1}$ |

${d}_{k}$ | closeness value between block-wise LRE estimates ${H}_{k,t}$ and ${H}_{k+1,t}$ |

f | continuous frequency |

${f}_{l}\left(t\right)$ | instantaneous frequency of the l-th component |

${f}_{s}$ | sampling frequency |

G | time-lag kernel |

${H}_{k}$ | Rényi entropy of the k-th TFD class |

${H}_{k,t}$ | Rényi entropy of the k-th class, and t-th building block |

i | smallest index of useful classes ${\rho}_{k}$ |

${i}_{t}$ | smallest index of building-block useful classes ${\rho}_{k,t}$ |

L | number of signal components |

K | number of TFD classes |

l | discrete lag |

m | discrete frequency |

M | number of frequency bins |

n | discrete time |

${n}_{0}$ | parameter defining the amplitude modulation time-shift |

N | number of time samples |

p | specific time instant |

t | continuous time |

T | parameter defining the signal component amplitude modulation variance |

$UI$ | extracted TFD useful information |

$U{I}_{t}$ | useful information extracted per one TFD building-block |

w | non-constant positive noise amplitude modulation |

${x}_{l}$ | l-th signal component |

x | noise-free signal |

y | noisy signal |

$\alpha $ | Rényi entropy order |

$\beta $ | parameter defining the amplitude noise modulation variance |

$\Delta m$ | frequency resolution |

$\Delta n$ | duration of LRE estimation interval |

$\Delta t$ | duration of one TFD building block |

$\nu $ | additive white noise |

$\rho $ | quadratic TFD |

${\rho}_{I}$ | TFD of extracted useful information |

${\rho}_{k}$ | k-th TFD class |

${\rho}_{k,t}$ | k-th class of TFD building block |

${\rho}_{t}$ | TFD building block |

${\rho}_{y}$ | TFD of a noisy signal |

$\tau $ | continuous lag |

${\Phi}_{l}$ | instantaneous phase of the l-th component |

## References

- Boashash, B. Time Frequency Signal Analysis and Processing: A Comprehensive Reference; Elsevier: Amsterdam, The Netherlands, 2016. [Google Scholar]
- Linh-Trung, N.; Belouchrani, A.; Abed-Meraim, K.; Boashash, B. Separating More Sources Than Sensors Using Time-Frequency Distributions. EURASIP J. Adv. Signal Process.
**2005**, 17, 2828–2847. [Google Scholar] [CrossRef] [Green Version] - Lerga, J.; Saulig, N.; Lerga, R.; Štajduhar, I. TFD Thresholding in Estimating the Number of EEG Components and the Dominant IF Using the Short-Term Rényi Entropy. In Proceedings of the 10th International Symposium on Image and Signal Processing and Analysis, Ljubljana, Slovenia, 18–20 September 2017; pp. 80–85. [Google Scholar]
- Stanković, S.; Stanković, L.J. An architecture for the realization of a system for time-frequency signal analysis. IEEE Trans. Circuits Syst.
**1997**, 7, 600–604. [Google Scholar] [CrossRef] - Saulig, N.; Pustelnik, N.; Borgnat, P.; Flandrin, P.; Sucic, V. Instantaneous counting of components in nonstationary signals. In Proceedings of the European Signal Processing Conference (EUSIPCO), Marrakesh, Marocco, 9–13 September 2013; pp. 1–5. [Google Scholar]
- Lerga, J.; Saulig, N.; Mozetič, V. Algorithm Based on the Short-Term Rényi Entropy and IF Estimation for Noisy EEG Signals Analysis. Comput. Biol. Med.
**2017**, 80, 1–13. [Google Scholar] [CrossRef] [PubMed] - Arık, S.Ö.; Jun, H.; Diamos, G. Fast Spectrogram Inversion Using Multi-Head Convolutional Neural Networks. IEEE Signal Process. Lett.
**2019**, 26, 94–98. [Google Scholar] [CrossRef] [Green Version] - Saulig, N.; Lerga, J.; Milanović, Ž. Extraction of Useful Information Content from Noisy Signals Based on Structural Affinity of Clustered TFDs’ Coefficients. IEEE Trans. Signal Process
**2019**, 12, 3154–3167. [Google Scholar] [CrossRef] - Yu, G.; Mallat, S.; Bacry, E. Audio denoising by time-frequency block thresholding. IEEE Trans. Signal Process
**2008**, 56, 1830–1839. [Google Scholar] [CrossRef] - Hassanpour, H.; Mesbah, M.; Boashash, B. Time-frequency feature extraction of newborn EEG seizure using SVD-based techniques. EURASIP J. Adv. Signal Process
**2004**, 56, 2544–2554. [Google Scholar] [CrossRef] [Green Version] - Leprette, B.; Martin, N. Extraction of pertinent subsets from time-frequency representations for detection and recognition purposes. Signal Process.
**2002**, 82, 229–238. [Google Scholar] [CrossRef] [Green Version] - Steinberg, R.; Martin, N. Segmentation of a speech spectrogram using mathematical morphology. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Las Vegas, NV, USA, 30 March 2008; pp. 1637–1640. [Google Scholar]
- Millioz, F.; Martin, N. Time-frequency segmentation for engine speed monitoring. In Proceedings of the Thirteenth International Congress on Sound and Vibration, ICSV13, Vienna, Austria, 2–6 July 2006. [Google Scholar]
- Meyer, C.; Spiertz, M. Audio segmentation using different time-frequency representations. In Proceedings of the 12th International Student Conference on Electrical Engineering, Prague, Czech Republic, 15 May 2008; pp. 1–5. [Google Scholar]
- Tantibundhit, C.; Pernkopf, F.; Kubin, G. Audio segmentation using different time-frequency representations. IEEE Trans. Audio Speech Lang. Process.
**2010**, 18, 1417–1428. [Google Scholar] [CrossRef] - Meignen, s.; Pham, D.H. Retrieval of the Modes of Multicomponent Signals From Downsampled Short-Time Fourier Transform. IEEE Trans. Signal Process.
**2010**, 66, 6204–6215. [Google Scholar] [CrossRef] - Laurent, N.; Meignen, S. A Novel Ridge Detector for Nonstationary Multicomponent Signals: Development and Application to Robust Mode Retrieval. IEEE Trans. Signal Process.
**2021**, 69, 3325–3336. [Google Scholar] [CrossRef] - Barkat, B.; Abed-Meraim, K. Algorithms for blind components separation and extraction from the time-frequency distribution of their mixture. EURASIP J. Adv. Signal Process.
**2004**, 2004, 978487. [Google Scholar] [CrossRef] [Green Version] - Bruni, V.; Tartaglione, M.; Vitulano, D. Radon spectrogram-based approach for automatic IFs separation. EURASIP J. Adv. Signal Process.
**2020**, 13, 1–21. [Google Scholar] [CrossRef] [Green Version] - Saulig, N.; Milanović, Ž.; Ioana, C. A local entropy-based algorithm for information content extraction from time-frequency distributions of noisy signals. Digit. Signal Process.
**2017**, 70, 155–165. [Google Scholar] [CrossRef] - Vranković, A.; Lerga, J.; Saulig, N. A novel approach to extracting useful information from noisy TFDs using 2D local entropy measures. EURASIP J. Adv. Signal Process.
**2020**, 2020, 18. [Google Scholar] [CrossRef] - Frank, I.E.; Todeschini, R. The Data Analysis Handbook; Elsevier: New York, NY, USA, 1994. [Google Scholar]
- Plandrin, P.; Borgnat, P. Time-frequency energy distributions meet compressed sensing. IEEE Trans. Signal Process.
**2010**, 58, 2974–2982. [Google Scholar] [CrossRef] - Saulig, N.; Milanović, Ž.; Lerga, J.; Griparić, K. On the Selection of the Proper Number of Classes in TFD Segmentation for Extraction of Useful Information Content from Noisy Signals. In Proceedings of the 3rd International Conference on Smart and Sustainable Technologies Splitech, Split, Croatia, 26–29 June 2018; pp. 1–5. [Google Scholar]

**Figure 1.**TFD of a noisy signal (

**a**), LRE estimates for $K=5$ classes (1. class blue, 2. class red, 3. class orange, 4. class purple, and 5. class green) (

**b**), and useful information extracted by the LRE criterion (

**c**).

**Figure 2.**Noisy signal TFD (

**a**), LRE estimates over different building blocks for $K=5$ classes (1. class blue, 2. class red, 3. class orange, 4. class purple, and 5. class green) (

**b**), and useful information extracted by the proposed block-adaptive LRE criterion (

**c**).

**Figure 3.**Noisy TFD of a flute sound signal (

**a**), noisy TFD of a bird song signal (

**a**), extracted signal components of the flute sound by the non-adaptive LRE algorithm (

**c**), extracted signal components of the bird song signal by the non-adaptive LRE algorithm (

**d**), extracted components of the flute sound signal by the proposed block-adaptive LRE algorithm (

**e**), and extracted components of the bird song signal by the proposed block-adaptive LRE algorithm (

**f**).

**Figure 4.**Noise-free TFD (

**a**,

**b**), noisy TFD (

**c**,

**d**), extracted signal components by the non-adaptive LRE algorithm (

**e**,

**f**), extracted useful information by the ICI method (

**g**,

**h**), and extracted useful information by the proposed block-adaptive LRE algorithm (

**i**,

**j**). Signals are referred to as Sig 1 (left column) and Sig 2 (right column).

**Table 1.**Parameters of the test signals from Figure 4.

$\begin{array}{c}y\left(n\right)={\sum}_{l=1}^{L}{x}_{l}\left(n\right)+w\left(n\right)\nu \left(n\right),\phantom{\rule{0.166667em}{0ex}}{x}_{l}\left(n\right)={A}_{l}\left(n\right){e}^{j{\Phi}_{l}\left(n\right)}\\ L=2,w\left(n\right)={e}^{-2{\beta}^{2}\frac{{(n-N/2)}^{2}}{{(N-1)}^{2}}},\phantom{\rule{0.166667em}{0ex}}1\le n\le N,\phantom{\rule{0.166667em}{0ex}}N=500\end{array}$ | |||||

$\beta $ | ${A}_{1}\left(n\right)$ | ${A}_{2}\left(n\right)$ | ${\Phi}_{1}\left(n\right)$ | ${\Phi}_{2}\left(n\right)$ | |

Sig 1 | 1 | 1 | $\begin{array}{c}{e}^{-\pi \frac{{(n-{n}_{0})}^{2}}{{T}^{2}}}\\ {n}_{0}=250,\\ T=490\end{array}$ | $2\pi (195\times {10}^{-6}{n}^{2}+249\times {10}^{-3}n-76)$ | $2\pi (283\times {10}^{-6}{n}^{2}+9\times {10}^{-3}n-21)$ |

Sig 2 | $3.5$ | 1 | $\begin{array}{c}2{e}^{-\pi \frac{{(n-{n}_{0})}^{2}}{{T}^{2}}},\\ {n}_{0}=250,\\ T=290\end{array}$ | $0.07Nsin(\frac{2\pi (n-35)}{0.7N}+1)+0.4\pi n-111$ | $2\pi ({10}^{-6}{n}^{3}-8\times {10}^{-4}{n}^{2}+0.4n)$ |

**Table 2.**Performance comparison for signals in Figure 4. Values are averaged from 1000 simulations of the signal with different noise realizations.

(a) Error Rate (%) | (b) False Negative (%) | ||||||||
---|---|---|---|---|---|---|---|---|---|

ER, Sig 1 | FN, Sig 1 | ||||||||

SNR | −3 | 0 | 3 | 6 | −3 | 0 | 3 | 6 | |

Proposed method | Proposed method | ||||||||

$N/5$ | 13.55 | 11.86 | 10.28 | 9.86 | 12.24 | 11.11 | 9.25 | 9.54 | |

$\Delta t=$ | $N/7$ | 13.62 | 12.05 | 10.20 | 9.60 | 12.45 | 11.36 | 9.33 | 9.24 |

$N/9$ | 13.94 | 11.81 | 10.42 | 9.51 | 12.94 | 11.13 | 9.61 | 9.42 | |

Non-adaptive LRE method | Non-adaptive LRE method | ||||||||

16.48 | 14.6 | 12.16 | 11.05 | 15.31 | 13.17 | 11.34 | 10.65 | ||

ICI method | ICI | ||||||||

14.55 | 12.92 | 10.88 | 10.51 | 12.79 | 11.29 | 9.93 | 9.12 | ||

ER, Sig 2 | FN, Sig 2 | ||||||||

SNR | −3 | 0 | 3 | 6 | −3 | 0 | 3 | 6 | |

Proposed method | Proposed method | ||||||||

$N/5$ | 11.08 | 9.98 | 9.08 | 8.21 | 9.89 | 8.75 | 8.10 | 7.35 | |

$\Delta t=$ | $N/7$ | 11.05 | 9.91 | 9.12 | 8.65 | 9.61 | 9.03 | 8.27 | 7.80 |

$N/9$ | 11.14 | 9.72 | 9.38 | 9.01 | 10.00 | 8.76 | 8.31 | 7.90 | |

Non-adaptive LRE method | Non-adaptive LRE method | ||||||||

14.95 | 13.42 | 12.52 | 13.30 | 14.57 | 13.42 | 12.11 | 13.30 | ||

ICI method | ICI | ||||||||

14.65 | 12.95 | 11.26 | 9.74 | 10.28 | 10.53 | 10.08 | 7.47 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Saulig, N.; Lerga, J.; Miličić, S.; Tomasović, Ž.
Block-Adaptive Rényi Entropy-Based Denoising for Non-Stationary Signals. *Sensors* **2022**, *22*, 8251.
https://doi.org/10.3390/s22218251

**AMA Style**

Saulig N, Lerga J, Miličić S, Tomasović Ž.
Block-Adaptive Rényi Entropy-Based Denoising for Non-Stationary Signals. *Sensors*. 2022; 22(21):8251.
https://doi.org/10.3390/s22218251

**Chicago/Turabian Style**

Saulig, Nicoletta, Jonatan Lerga, Siniša Miličić, and Željka Tomasović.
2022. "Block-Adaptive Rényi Entropy-Based Denoising for Non-Stationary Signals" *Sensors* 22, no. 21: 8251.
https://doi.org/10.3390/s22218251