The Module Gradient Descent Algorithm via L2 Regularization for Wavelet Neural Networks

Mohamed, Khidir Shaib; Suliman, Ibrahim. M. A.; Alhalangy, Abdalilah; Adam, Alawia; Suhail, Muntasir; Ibrahim, Habeeb; Mohamed, Mona A.; Saad, Sofian A. A.; Mohammed, Yousif Shoaib

doi:10.3390/axioms14120899

Open AccessArticle

The Module Gradient Descent Algorithm via L₂ Regularization for Wavelet Neural Networks

by

Khidir Shaib Mohamed

^1,*

,

Ibrahim. M. A. Suliman

²,

Abdalilah Alhalangy

³

,

Alawia Adam

¹,

Muntasir Suhail

¹,

Habeeb Ibrahim

¹,

Mona A. Mohamed

¹,

Sofian A. A. Saad

¹ and

Yousif Shoaib Mohammed

⁴

¹

Department of Mathematics, College of Sciences, Qassim University, Buraydah 51452, Saudi Arabia

²

General Department of College of Technical Engineering, Bright Star University, Al-Brega P.O. Box 858, Libya

³

Department of Computer Engineering, College of Computer, Qassim University, Buraydah 51452, Saudi Arabia

⁴

Department of Physics, College of Science, Qassim University, Buraydah 51452, Saudi Arabia

^*

Author to whom correspondence should be addressed.

Axioms 2025, 14(12), 899; https://doi.org/10.3390/axioms14120899

Submission received: 20 October 2025 / Revised: 27 November 2025 / Accepted: 29 November 2025 / Published: 4 December 2025

Download

Browse Figures

Versions Notes

Abstract

Although wavelet neural networks (WNNs) combine the expressive capability of neural models with multiscale localization, there are currently few theoretical guarantees for their training. We investigate the weight decay (

L_{2}

regularization) optimization dynamics of gradient descent (GD) for WNNs. Using explicit rates controlled by the spectrum of the regularized Gram matrix, we first demonstrate global linear convergence to the unique ridge solution for the feature regime when wavelet atoms are fixed and only the linear head is trained. Second, for fully trainable WNNs, we demonstrate linear rates in regions satisfying a Polyak–Łojasiewicz (PL) inequality and establish convergence of GD to stationary locations under standard smoothness and boundedness of wavelet parameters; weight decay enlarges these regions by suppressing flat directions. Third, we characterize the implicit bias in the over-parameterized neural tangent kernel (NTK) regime: GD converges to the minimum reproducing kernel Hilbert space (RKHS) norm interpolant associated with the WNN kernel with

L_{2}

. In addition to an assessment process on synthetic regression, denoising, and ablations across λ and stepsize, we supplement the theory with useful recommendations on initialization, stepsize schedules, and regularization scales. Together, our findings give a principled prescription for dependable training that has broad applicability to signal processing applications and shed light on when and why

L_{2}

-regularized GD is stable and quick for WNNs.

Keywords: wavelet neural networks; Mexican hat wavelet; gradient descent algorithm; L₂ regularization; numerical results

Share and Cite

MDPI and ACS Style

Mohamed, K.S.; Suliman, I.M.A.; Alhalangy, A.; Adam, A.; Suhail, M.; Ibrahim, H.; Mohamed, M.A.; Saad, S.A.A.; Mohammed, Y.S. The Module Gradient Descent Algorithm via L₂ Regularization for Wavelet Neural Networks. Axioms 2025, 14, 899. https://doi.org/10.3390/axioms14120899

AMA Style

Mohamed KS, Suliman IMA, Alhalangy A, Adam A, Suhail M, Ibrahim H, Mohamed MA, Saad SAA, Mohammed YS. The Module Gradient Descent Algorithm via L₂ Regularization for Wavelet Neural Networks. Axioms. 2025; 14(12):899. https://doi.org/10.3390/axioms14120899

Chicago/Turabian Style

Mohamed, Khidir Shaib, Ibrahim. M. A. Suliman, Abdalilah Alhalangy, Alawia Adam, Muntasir Suhail, Habeeb Ibrahim, Mona A. Mohamed, Sofian A. A. Saad, and Yousif Shoaib Mohammed. 2025. "The Module Gradient Descent Algorithm via L₂ Regularization for Wavelet Neural Networks" Axioms 14, no. 12: 899. https://doi.org/10.3390/axioms14120899

APA Style

Mohamed, K. S., Suliman, I. M. A., Alhalangy, A., Adam, A., Suhail, M., Ibrahim, H., Mohamed, M. A., Saad, S. A. A., & Mohammed, Y. S. (2025). The Module Gradient Descent Algorithm via L₂ Regularization for Wavelet Neural Networks. Axioms, 14(12), 899. https://doi.org/10.3390/axioms14120899

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Menu

The Module Gradient Descent Algorithm via L₂ Regularization for Wavelet Neural Networks

Abstract

Share and Cite

Article Metrics

Article Access Statistics

Further Information

Guidelines

MDPI Initiatives

Follow MDPI