To evaluate the performance of the proposed measure, a series of experiments were conducted using both synthetic and real-world signals. The synthetic signals allowed for controlled assessment of the measure’s sensitivity to various signal parameters and noise levels, while the real-world signal provided insight into its applicability in practical scenarios.
3.1. Selection of the Signal Examples
The performance of the proposed measure was evaluated using three synthetic signals. The first signal,
with
samples, is defined in (
17). The second signal,
with
samples, consists of four time-limited LFM components with different amplitudes embedded in AWGN with
dB:
Here, the rectangular function is defined as
, where
,
, and
denote the starting time, ending time, and duration of the signal component, respectively. The introduction of time-limited components and AWGN simulates more realistic signal conditions and allows for assessing the measure’s robustness to noise and interference. The third signal,
with
samples, comprises two quadratic frequency-modulated (QFM) components:
In addition to the synthetic signals, a real-world gravitational wave signal (this research has made use of data, software, and/or web tools obtained from the LIGO Open Science Center (
https://losc.ligo.org (accessed on 17 February 2025)), a service of LIGO Laboratory and the LIGO Scientific Collaboration. LIGO is funded by the U.S. National Science Foundation), [
28,
41,
42], denoted as
, was included in the evaluation. To prepare the signal for analysis, established pre-processing techniques were employed, including downsampling the original signal by a factor of 14, resulting in
samples, corresponding to a duration of 0.25 to 0.45 s and a frequency range of
Hz. This downsampling procedure is consistent with prior work and serves to reduce computational complexity without significantly compromising the essential signal characteristics [
28].
Figure 4 illustrates the WVDs of the considered signals
,
,
, and
. It is evident from these figures that the WVDs are characterized by significant cross-term interference (and noise for
), due to the lack of any filtering. These artifacts underscore the need for effective cross-term suppression techniques, such as the CS-based approach employed in this study.
3.4. Results: Synthetic and Real-World Signals
The reconstructed TFDs for the experimental tests were generated using the TwIST algorithm with distinct
values to represent different reconstruction outcomes. Case 1 utilized a high
parameter, resulting in significant auto-term loss. Case 2 employed
values that led to unsuccessful reconstructions dominated by interference. Case 3 TFDs were generated with
values between those of Cases 1 and 2, achieving a balance of improved auto-term preservation and reduced interference.
Figure 7 illustrates these reconstructed TFDs for all considered signals across Cases 1–3, while
Table 2 presents a comparative analysis of the proposed measure against existing measures in evaluating the reconstructed TFD performance.
The results, based on the
-norm, indicate that the Case 3 reconstructed TFDs consistently provide a closer approximation to the ideal TFDs than those in Cases 1 and 2 across all signal examples. In line with prior research [
27], the global measures
and
R tend to favor overly sparse TFDs, incorrectly identifying Case 1 as optimal due to their inherent disregard for crucial auto-term information. For signals
and
, the LRE-based metric effectively penalizes the TFDs in Cases 1 and 2, assigning them higher error values and correctly identifying the reconstructed TFD in Case 3 as the best performing. However, this trend does not hold for signal
, where
erroneously identifies the overly sparse reconstructed TFD in Case 1 as optimal. It is important to note that
values were not computed for the WVD due to two primary reasons: first, LRE is typically applied to QTFDs where interference is mitigated; and second, the appropriate reference TFD for the WVD would also be a WVD, which is inherently incomparable to the reference TFDs used for the reconstructed TFDs.
In contrast, the proposed measure components offer more nuanced insights into TFD performance. Across all signal examples, indicates that the WVD and the reconstructed TFDs in Case 2 retain all auto-terms, while the reconstructed TFDs in Case 3 exhibit superior performance compared to those in Case 1. Furthermore, effectively identifies the TFDs with the highest auto-term resolution, with the reconstructed TFDs in Case 3 surpassing those in Case 2, which suffer from poor resolution. As for , it accurately identifies the overly sparse TFDs as free from both interference and noise artifacts, a characteristic not shared by the WVD. It also confirms that the reconstructed TFDs in Case 3 demonstrate a lower degree of unsuppressed interference compared to those in Case 2. Consequently, when the proposed measure components are equally averaged, the reconstructed TFDs in Case 3 are consistently identified as the best performing for all three signal examples.
These observations concerning the measures’ performance in evaluating reconstructed TFDs extend to the real-world gravitational signal
.
Figure 8 illustrates the Case 1–3 reconstructed TFDs for this signal, while
Table 3 presents the corresponding performance results. Consistent with the trends observed for the synthetic signals, the
and
R measures favor overly sparse TFDs, and the measure
incorrectly identifies the corrupted reconstructed TFD in Case 2 as the best performing. The proposed measure component
highlights the WVD as having full retention of auto-terms. However, in contrast to the synthetic signals,
reveals that the reconstructed TFDs in Case 2 are missing several auto-term samples. This discrepancy arises from the limitations of the rectangular CS-AF area, which fails to adapt its vertices appropriately for higher-order FM components present in signal
[
28]. The
component correctly identifies the overly sparse reconstructed TFDs in Case 1 as exhibiting the best auto-term resolution and minimal interference. In summary, the averaged proposed measure components accurately demonstrate that the reconstructed TFDs in Case 3 offer the best overall performance among the tested cases.
3.5. Meta-Heuristic Optimization Comparison
Figure 9 presents a visual comparison of optimized reconstructed TFDs obtained using MOPSO with two distinct sets of objective functions: the proposed
measures and the existing
measures.
Table 4 and
Table 5 provide a corresponding numerical comparison.
The results indicate that optimization with the proposed measures consistently yields reconstructed TFDs with well-preserved auto-terms, high resolution, and suppressed interference. Furthermore, the proposed measures led to a consistent improvement in optimized reconstructed TFDs for all considered signals compared to the results obtained using . Specifically, the -norm, a measure of the deviation from an ideal TFD, was improved by 3.68% to 65.10%, demonstrating the superior ability of to generate reconstructed TFDs that closely approximate the ideal.
It is important to note that for all considered signals, the proposed
measures resulted in better preservation of auto-terms, as evidenced by consistently lower
values. While the
objectives sometimes yielded slightly higher auto-term resolution (higher
), this improvement often came at the cost of losing essential parts of the auto-terms (see
Figure 9a,c). For the noisy signals
and
, the
objectives resulted in lower
values, indicative of less effective interference suppression (see
Figure 9b,d), as confirmed by generally worse
values, except in the case of
.
Finally, the time required to achieve optimized reconstructed TFDs using the proposed objectives was significantly reduced, ranging from 44.17% to 60.63%, compared to the time required for the existing objectives. This reduction in computational cost highlights the efficiency of the proposed measure for meta-heuristic optimization.
3.6. Noise Sensitivity Analysis
To further evaluate the robustness of meta-heuristic optimization using the proposed objective functions, synthetic signal examples were embedded in AWGN at SNR levels ranging from 0 dB to 9 dB. The results, summarized in
Table 6, demonstrate a consistent improvement when using the proposed
measures over the existing
measures across all tested SNR levels.
Specifically, the -norm, a measure of the deviation from the ideal TFD, was improved by 16.26% at 0 dB SNR, 61.06% at 3 dB SNR, 57.67% at 6 dB SNR, and 51.79% at 9 dB SNR. This indicates that the proposed objective functions lead to more accurate TFD reconstructions, even in the presence of significant noise.
Furthermore, the results suggest that the performance degradation due to decreasing AWGN SNR levels is less pronounced when optimizing with the proposed objectives compared to the objectives. This highlights the superior noise resilience of the proposed approach.
3.7. Interpretation of the Results
As shown in [
27], global measures
and
R tend to favor overly sparse TFDs (Case 1) for all signal examples, limiting their applicability when auto-term preservation is a priority. While the LRE-based metric successfully penalizes the reconstructed TFDs in Cases 1 and 2 with higher error values for signals
and
, its performance is inconsistent for signals
and
. This stems from the LRE limitation when analyzing intersecting components, as in signal
, where a drop in the estimated local number of components leads to favoring overly sparse TFDs. In the case of signal
,
Figure 10 illustrates that the hyperbolic behavior of the signal’s auto-term makes it unsuitable for LRE localization. Specifically, both LRE estimations exhibit an artificial increase when the component changes its slope with respect to the time or frequency axis, causing
to favor the corrupted reconstructed TFD in Case 2.
Such misclassifications can confuse end-users when selecting the best-performing reconstructed TFD. Furthermore, even when the LRE-based measure provides a correct assessment, it lacks the interpretability needed to guide the adjustment of the parameter. Instead, users must analyze the relationship between the and curves to infer the presence of over-sparsity or interference. However, this analysis can be ambiguous, especially when the number of local components increases post-reconstruction, resulting in a higher curve compared to . This discrepancy can stem from low-resolution auto-terms or the reconstruction of cross-terms and noise, yet the precise cause remains elusive.
Conversely, the proposed measure components provide clearer insights than the LRE metric. For instance, a low
combined with a high
suggests well-preserved auto-terms with reduced resolution. A significantly high
, indicates interference in the reconstructed TFDs, suggesting that
is set too low (as in Case 2 with
). Conversely, a high
with very low
and
points to an overly sparse reconstructed TFD with inconsistent auto-terms, implying that
is set too high (as in Case 1). In both scenarios, the proposed measures offer guidance on whether
should be increased or decreased, providing a distinct advantage over the existing LRE-based metric used in [
27]. The capacity to decouple
,
, and
is a particular advantage since that enables an unambiguous interpretation of the result.
Quantitatively, a of 0.0056 for Case 3’s reconstructed TFD for signal signifies a very high resolution of auto-terms, averaging just two samples per estimated IF. Also, a of 0.4149 for Case 2’s reconstructed TFD for signal signifies that 41.49% of the TFD (of size ) is corrupted by interference. These quantitative indicators directly enhance the interpretability of the proposed measure.
In addition to the increased interpretability, the findings demonstrate that the proposed measure components, when averaged equally, successfully highlight the best-performing reconstructed TFD. These findings and conclusions drawn from the synthetic signals also extend to the real-world gravitational signal. The various presented signal examples with multiple LFM and QFM components, with varying amplitudes and intersecting cases, demonstrate the robustness and suitability of the proposed measure for a wide range of signals.
When implementing the proposed measure components in meta-heuristic optimization, the results shown in
Table 4 and
Table 5 illustrate that reconstructed TFDs optimized using the proposed measure’s components exhibit well-preserved auto-terms and suppressed interference. Even though optimizing with the LRE-based measure can lead to reconstructed TFDs with preserved auto-terms and less interference, the usage of the proposed measures further improves optimization performance. To complement these results,
Table 6 indicates a meta-heuristic optimization improvement in noisy environments, where reconstructed TFDs optimized using the proposed measure’s components proved to be better than those obtained using the LRE-based measure across all considered SNR levels. This highlights the proposed measure over the existing LRE-based measure in noisy environments, where the limitations of the LRE are pronounced.
Even though the CS-based TFD reconstruction method is typically considered for offline analysis, the results illustrate that the proposed measure is beneficial for optimizing TFD, as the optimization time is significantly reduced by using the proposed measures as objective functions. The reasons for this are twofold. Firstly, the proposed measures are not as conflicted as and . Secondly, can yield the same increased value for both overly sparse TFDs (high ) and TFDs with interference (low ), which confuses and slows the convergence of meta-heuristics. While memory efficiency for storing information is not always a primary concern in this field, it is important to consider the memory requirement associated with different approaches. Storing values alongside the corresponding measures provides a comprehensive understanding of the impact of on TFD performance. Analyzing the existing measures in a similar way would necessitate substantially greater memory resources, as it would involve storing not only but also the vectors from both the original and reconstructed TFDs, and in many cases, the full reconstructed TFDs. The memory would increase significantly, potentially reaching hundreds of megabytes.
The inclusion of CAM in the proposed measure enables the automatic selection of IF and GD estimates, circumventing the need for manual selection of time and frequency localization. Therefore, users are not required to manually identify and separate the components best suited for localization via time or frequency slices, which increases the overall convenience of the method. However, it should be noted that, by its very construction, the inclusion of CAM and a mandatory matrix with IF and/or GD estimates indicates that the proposed measure requires more inputs than the LRE.
The selection of the LRE method should be discussed. This study uses the original LRE approach, which was established in LRE-based measures in [
27,
28]. This original approach was robust and applicable to a different set of signals. An iterative LRE approach proposed in [
44] may be used in special cases of signals comprising multiple components with significant differences in amplitudes. However, the iterative LRE implies several other limitations, such as incorrect component removal for intersecting components and significantly higher sensitivity to noise, making this approach applicable only for special cases. A study in [
45] circumvented some limitations of the original and iterative LRE approach by using convolutional neural networks. This involves neural networks trained using a diverse dataset of signals to predict the local number of components from an input QTFD. This study did not utilize that approach for two reasons. Firstly, the network is trained on QTFDs, meaning that it can predict local numbers of components for a starting TFD, but not for a reconstructed TFD. This would require retraining the network on this specific task. Secondly, estimates provided by the network are very volatile, and research on applying some smoothing filters is still required before usage.
The optimization of and parameters relies on numerical simulations in sparse TFD reconstructions. Furthermore, for overall evaluation, the measured components have been averaged for overall interpretation, meaning that all three components have equal importance. Users are encouraged to retest parameters and and pronounce the importance of some measured components if they find them beneficial in specific TFD or advanced methods.
The enhanced assessment of reconstructed TFDs using the proposed measure has significant potential in this field. The improvement is demonstrated through the interpretability of the proposed measure when compared to the existing methods. This opens up many research directions in machine-related applications.