Skip to Content
You are currently on the new version of our website. Access the old version .
SymmetrySymmetry
  • Article
  • Open Access

4 October 2025

19 Pages

Restoring Spectral Symmetry in Gradients: A Normalization Approach for Efficient Neural Network Training

,
,
,
,
and
1
College of Physics and Information Engineering, Quanzhou Normal University, Quanzhou 362000, China
2
College of Artificial Intelligence, Hebei Oriental University, Langfang 065000, China
*
Author to whom correspondence should be addressed.
This article belongs to the Section A: Computer Science

Abstract

Neural network training often suffers from spectral asymmetry, where gradient energy is disproportionately allocated to high-frequency components, leading to suboptimal convergence and reduced efficiency. This paper introduces Gradient Spectral Normalization (GSN), a novel optimization technique designed to restore spectral symmetry by dynamically reshaping gradient distributions in the frequency domain. GSN transforms gradients using FFT, applies layer-specific energy redistribution to enforce a symmetric balance between low- and high-frequency components, and reconstructs the gradients for parameter updates. By tailoring normalization schedules for attention and MLP layers, GSN enhances inference performance and improves model accuracy with minimal overhead. Our approach leverages the principle of symmetry to create more stable and efficient neural systems, offering a practical solution for resource-constrained environments. This frequency-domain paradigm, grounded in symmetry restoration, opens new directions for neural network optimization with broad implications for large-scale AI systems.

Article Metrics

Citations

Article Access Statistics

Multiple requests from the same IP address are counted as one view.