Many adaptive gradient methods have been successfully applied to train deep neural networks, such as Adagrad, Adadelta, RMSprop and Adam. These methods perform local optimization with an element-wise scaling learning rate based on past gradients. Alt...
Handwritten character recognition is currently getting the attention of researchers because of possible applications in assisting technology for blind and visually impaired users, human–robot interaction, automatic data entry for business documents,...
The paper solves the problem of constructing step adjustment algorithms for a gradient method based on the principle of the steepest descent. The expansion of the step adjustment principle, its formalization and parameterization led the researchers t...
Adaptive gradient descent methods such as Adam, RMSprop, and AdaGrad achieve great success in training deep learning models. These methods adaptively change the learning rates, resulting in a faster convergence speed. Recent studies have shown their...
The role of unmanned aerial vehicles (UAVs) in time-sensitive missions such as low-altitude reconnaissance and disaster rescue has gained increasing significance. To address the challenge of visual localisation for UAVs operating in complex terrains...
Optimization methods, namely, gradient optimization methods, are a key part of neural network training. In this paper, we propose a new gradient optimization method using exponential decay and the adaptive learning rate using a discrete second-order...
In this paper, we transform the problem of solving the Sylvester matrix equation into an optimization problem through the Kronecker product primarily. We utilize the adaptive accelerated proximal gradient and Newton accelerated proximal gradient meth...
In this paper, we propose a stochastic primal-dual adaptive method based on an inexact augmented Lagrangian function to solve non-convex programs, referred to as the SPDAM. Different from existing methods, SPDAM incorporates adaptive step size and mo...
Motivated by sparse principal component analysis (PCA), this paper proposes ATE-ManPG, a safeguarded transport-extrapolated proximal gradient method for minimizing the sum of a smooth function and a convex, possibly nonsmooth function over the Stiefe...
The most important advantage of conjugate gradient methods (CGs) is that these methods have low memory requirements and convergence speed. This paper contains two main parts that deal with two application problems, as follows. In the first part, thre...
The State of Charge (SOC) estimation is a significant issue for safe performance and the lifespan of Lithium-ion (Li-ion) batteries. In this paper, a Robust Adaptive Online Long Short-Term Memory (RoLSTM) method is proposed to extract SOC estimation...
After decades of research, X-band marine radars have been broadly used for wind measurement. For retrieving the wind direction based on the wind-induced streaks, a lot of effort has been expended on three celebrated approaches—the local gradien...
The goal of this article is to train deep neural networks that accelerate useful adaptive learning rate optimization algorithms such as AdaGrad, RMSProp, Adam, and AMSGrad. To reach this goal, we devise an iterative algorithm combining the existing a...
Wave propagation modeling in heterogeneous media requires numerical methods that can simultaneously handle complex geometries, artificial boundary reflections, and spatially varying resolution demands. In this study, we present a Discontinuous Galerk...
In Long Range Wide Area Network (LoRaWAN), the data rate of the devices can be adjusted to optimize the throughput by changing the spreading factor. However, the adaptive data rate has to be carefully utilized because the collision probability, which...
Low-light image enhancement is an important problem in image processing. In this paper, we propose a novel adaptive variational model based on fractional-order gradients for low-light image enhancement. Specifically, the low-light part of the low-lig...
SwarmPNM combines an adaptive second-moment estimate and positive–negative gradient momentum with attraction towards the lowest-loss weights stored along a single training trajectory. The attraction is inspired by particle swarm optimization, b...
Unmanned underwater vehicles (UUVs) face significant challenges in achieving safe and efficient autonomous navigation in complex marine environments due to uncertain perception, dynamic obstacles, and nonlinear coupled motion control. This study prop...
Fuzzy clustering is a form of unsupervised learning that assigns the elements of a dataset into multiple clusters with varying degrees of membership rather than assigning them to a single cluster. The classical Fuzzy C-Means algorithm operates as an...
Stochastic gradient descent is the method of choice for solving large-scale optimization problems in machine learning. However, the question of how to effectively select the step-sizes in stochastic gradient descent methods is challenging, and can gr...
As a new machine learning technique, federated learning has received more attention in recent years, which enables decentralized model training across data silos or edge intelligent devices in the Internet of Things without exchanging local raw data....
In this study, we focus on solving the nonlinear fractional optimization problem in which the numerator is smooth convex and the denominator is smooth concave. To achieve this goal, we develop an algorithm called the adaptive projection gradient meth...
Gradient descent method is an essential algorithm for learning of neural networks. Among diverse variations of gradient descent method that have been developed for accelerating learning speed, the natural gradient learning is based on the theory of i...
Deep neural networks (DNNs) have famously been applied in various ordinary duties. However, DNNs are sensitive to adversarial attacks which, by adding imperceptible perturbation samples to an original image, can easily alter the output. In state-of-t...
The offset-tracking method (OTM) utilizing SAR image intensity can detect large deformations, which makes up for the inability of interferometric synthetic aperture radar (InSAR) technology in large mining deformation monitoring, and has been widely...
The distributed policy gradient (PG) method has become a mainstream technique for finding optimal policies in multi-agent reinforcement learning. However, global convergence guarantees, which are the most basic measures of theoretical convergence per...
The main objective of this article is to propose a new method that would extend Popov’s extragradient method by changing two natural projections with two convex optimization problems. We also show the weak convergence of our designed method by...
Reliable individual identification is essential for precision livestock farming. To reduce the performance disparity among coat-color groups in open-set sheep face recognition, this study proposes a color-adaptive recognition method. A coat color cla...
The safety and robustness of convolutional neural networks (CNNs) have raised increasing concerns, especially in safety-critical areas, such as medical applications. Although CNNs are efficient in image classification, their predictions are often sen...
In modern warfare, frequency-hopping communication serves as the primary method for battlefield information transmission, with its significance continuously growing. Fighting for the control of electromagnetic power on the battlefield has become an i...
The stochastic variance-reduced gradient (SVRG) theory is particularly well-suited for addressing gradient variance in deep neural network (DNN) training; however, its direct application to DNN training is hindered by adaptation challenges. To tackle...
In constrained waters, multi-USV cooperative encirclement of highly maneuverable targets is strongly affected by partial observability as well as obstacle and boundary constraints, posing substantial challenges to stable cooperative control. Existing...
In this article, we propose a modified self-adaptive conjugate gradient algorithm for handling nonlinear monotone equations with the constraints being convex. Under some nice conditions, the global convergence of the method was established. Numerical...
Coal fires that are induced by natural spontaneous combustion or result from human activities occurring on the surface and in underground coal seams destroy coal resources and cause serious environmental degradation. Thermal infrared image data, whic...
The self-adaptive gradient-based thresholding (SAGBT) method is a simple non-interactive coal fire detection approach involving segmentation and a threshold identification algorithm that adapts to the spatial distribution of thermal features over a l...
Foot-mounted Inertial Pedestrian-Positioning Systems (FIPPSs) based on Micro Inertial Measurement Units (MIMUs), have recently attracted widespread attention with the rapid development of MIMUs. The can be used in challenging environments such as fir...
In wavefront sensorless adaptive optics (WFS-less AO) systems, stochastic parallel gradient descent (SPGD) is the primary optimization method for correcting wavefront distortions. However, as the intensity of atmospheric turbulence interference incre...
Hybrid conjugate gradient methods are considered as an efficient family of conjugate gradient (CG) methods used to solve unconstrained optimization problems. In this paper, on account of the outstanding performance of the PRP (Polak–Ribiè...
A new gradient-like methodology has been developed for aeroengine diagnosis, determining the engine health condition, which is defined by the engine degradation from an undegraded state and uses measurements at various sensors distributed along the e...
This paper proposes a Distributed Adaptive Regularization Algorithm (DARN) for solving composite non-convex and non-smooth optimization problems in multi-agent systems. The algorithm employs a three-phase iterative framework to achieve efficient coll...
This paper presents a frequency adaptive grid voltage sensorless control scheme of a grid-connected inductive–capacitive–inductive (LCL)-filtered inverter, which is based on an adaptive current controller and a grid voltage observer. The...
For a high-power slab solid-state laser, obtaining high output power and high output beam quality are the most important indicators. Adaptive optics systems can significantly improve beam qualities by compensating for the phase distortions of the las...
Adam (Adaptive Moment Estimation) is a well-known algorithm for the first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments. As shown by computational experiments, with an increase...
In adaptive control, error models use system output error and adaptive laws to update controller parameters for control or identification tasks. Fractional-order calculus, involving non-integer-order derivatives and integrals, is increasingly importa...
The transfer learning method is used to extend our existing model to more difficult scenarios, thereby accelerating the training process and improving learning performance. The conditional adversarial domain adaptation method proposed in 2018 is a pa...
Although many adaptive techniques for active vibration reduction have been designed to achieve optimal performance in practical applications, few are related to reinforcement learning (RL). To explore the best performance of the active vibration redu...
In this article, we propose a new modified extragradient-like method to solve pseudomonotone equilibrium problems in real Hilbert space with a Lipschitz-type condition on a bifunction. This method uses a variable stepsize formula that is updated at e...
Despite their high prediction accuracy, deep learning-based soft sensor (DLSS) models face challenges related to adversarial robustness against malicious adversarial attacks, which hinder their widespread deployment and safe application. Although adv...
This paper proposes a new adaptive algorithm for the second-order blind signal separation (BSS) problem with convolutive mixtures by utilising a combination of an accelerated gradient and a conjugate gradient method. For each iteration of the adaptiv...
We study the generalization properties of stochastic optimization methods under adaptive data sampling schemes, focusing on the setting of pairwise learning, which is central to tasks like ranking, metric learning, and AUC maximization. Unlike pointw...
of 38