DARN: Distributed Adaptive Regularized Optimization with Consensus for Non-Convex Non-Smooth Composite Problems

This paper proposes a Distributed Adaptive Regularization Algorithm (DARN) for solving composite non-convex and non-smooth optimization problems in multi-agent systems. The algorithm employs a three-phase iterative framework to achieve efficient collaborative optimization: (1) a local regularized optimization step, which utilizes proximal mappings to enforce strong convexity of weakly convex objectives and ensure subproblem well-posedness; (2) a consensus update based on doubly stochastic matrices, guaranteeing asymptotic convergence of agent states to a global consensus point; and (3) an innovative adaptive regularization mechanism that dynamically adjusts regularization strength using local function value variations to balance stability and convergence speed. Theoretical analysis demonstrates that the algorithm maintains strict monotonic descent under non-convex and non-smooth conditions by constructing a mixed time-scale Lyapunov function, achieving a sublinear convergence rate. Notably, we prove that the projection-based update rule for regularization parameters preserves lower-bound constraints, while spectral decay properties of consensus errors and perturbations from local updates are globally governed by the Lyapunov function. Numerical experiments validate the algorithm’s superiority in sparse principal component analysis and robust matrix completion tasks, showing a 6.6% improvement in convergence speed and a 51.7% reduction in consensus error compared to fixed-regularization methods. This work provides theoretical guarantees and an efficient framework for distributed non-convex optimization in heterogeneous networks.