Adaptive gradient descent methods such as Adam, RMSprop, and AdaGrad achieve great success in training deep learning models. These methods adaptively change the learning rates, resulting in a faster convergence speed. Recent studies have shown their...
Many adaptive gradient methods have been successfully applied to train deep neural networks, such as Adagrad, Adadelta, RMSprop and Adam. These methods perform local optimization with an element-wise scaling learning rate based on past gradients. Alt...
The most important advantage of conjugate gradient methods (CGs) is that these methods have low memory requirements and convergence speed. This paper contains two main parts that deal with two application problems, as follows. In the first part, thre...
Fuzzy clustering is a form of unsupervised learning that assigns the elements of a dataset into multiple clusters with varying degrees of membership rather than assigning them to a single cluster. The classical Fuzzy C-Means algorithm operates as an...
Stochastic gradient descent is the method of choice for solving large-scale optimization problems in machine learning. However, the question of how to effectively select the step-sizes in stochastic gradient descent methods is challenging, and can gr...
In wavefront sensorless adaptive optics (WFS-less AO) systems, stochastic parallel gradient descent (SPGD) is the primary optimization method for correcting wavefront distortions. However, as the intensity of atmospheric turbulence interference incre...
Optimization methods, namely, gradient optimization methods, are a key part of neural network training. In this paper, we propose a new gradient optimization method using exponential decay and the adaptive learning rate using a discrete second-order...
Foot-mounted Inertial Pedestrian-Positioning Systems (FIPPSs) based on Micro Inertial Measurement Units (MIMUs), have recently attracted widespread attention with the rapid development of MIMUs. The can be used in challenging environments such as fir...
Adam (Adaptive Moment Estimation) is a well-known algorithm for the first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments. As shown by computational experiments, with an increase...
This paper proposes a new adaptive algorithm for the second-order blind signal separation (BSS) problem with convolutive mixtures by utilising a combination of an accelerated gradient and a conjugate gradient method. For each iteration of the adaptiv...
In this paper, we present a rapid gradient descent method for solving low-rank matrix recovery problems. Our method extends the conventional gradient descent framework by exploiting the problem’s unique features to develop an innovative fast gr...
The paper solves the problem of constructing step adjustment algorithms for a gradient method based on the principle of the steepest descent. The expansion of the step adjustment principle, its formalization and parameterization led the researchers t...
For a high-power slab solid-state laser, obtaining high output power and high output beam quality are the most important indicators. Adaptive optics systems can significantly improve beam qualities by compensating for the phase distortions of the las...
Designing waveforms with a Constant Modulus Constraint (CMC) to achieve desirable Slow-Time Ambiguity Function (STAF) characteristics is significantly important in radar technology. The problem is NP-hard, due to its non-convex quartic objective func...
Gradient descent method is an essential algorithm for learning of neural networks. Among diverse variations of gradient descent method that have been developed for accelerating learning speed, the natural gradient learning is based on the theory of i...
We are proposing an adaptation of the gradient descent method to optimize the position and orientation of sensors for the sensor placement problem. The novelty of the proposed method lies in the combination of gradient descent optimization with a rea...
To address the issues of DC-side voltage fluctuation and three-phase current distortion in rectifier systems under pulsed load conditions, this paper proposes a control strategy that integrates Model Predictive Control (MPC) with a Luenberger observe...
Optimization algorithms have now played an important role in many fields, and the issue of how to design high-efficiency algorithms has gained increasing attention, for which it has been shown that advanced control theories could be helpful. In this...
We study the generalization properties of stochastic optimization methods under adaptive data sampling schemes, focusing on the setting of pairwise learning, which is central to tasks like ranking, metric learning, and AUC maximization. Unlike pointw...
During the interaction process of a manipulator executing a grasping task, to ensure no damage to the object, accurate force and position control of the manipulator’s end-effector must be concurrently implemented. To address the computationally...
Deep unfolding networks (DUNs) have attracted growing attention in compressed sensing (CS) due to their good interpretability and high performance. However, many DUNs often improve the reconstruction effect at the price of a large number of parameter...
Magnetic and inertial sensors have been widely used to estimate the orientation of human segments due to their low cost, compact size and light weight. However, the accuracy of the estimated orientation is easily affected by external factors, especia...
Winter jujube harvesting is traditionally labor-intensive, yet declining labor availability and rising costs necessitate robotic automation to maintain agricultural competitiveness. Path planning for robotic arms in orchards faces challenges due to t...
In this paper, we develop momentum-based adaptive update laws for parameter identification and control to improve parameter estimation error convergence and control system performance for uncertain dynamical systems. Specifically, we introduce three...
Deep learning has achieved many successes in different fields but can sometimes encounter an overfitting problem when there are insufficient amounts of labeled samples. In solving the problem of learning with limited training data, meta-learning is p...
Efficient biogas production significantly impacts greenhouse gas (GHG) emissions and carbon sequestration by reducing emissions and enhancing carbon storage. Nonetheless, the consistency and optimization of biogas production are hindered by fluctuati...
Digital Elevation Models (DEMs) are commonly used for environment, engineering, and architecture-related studies. One of the most important factors for the accuracy of DEM generation is the process of spatial interpolation, which is used for estimati...
The efficient utilization of structural information in High-Range Resolution Profiles (HRRPs) is of great significance for improving recognition performance. This paper proposes a size estimation method based on L1-norm variable fractional-order grad...
In this paper, we present an algorithm for the problem of multi-channel blind deconvolution which can adapt to un-known sources with both sub-Gaussian and super-Gaussian probability density distributions using a generalized gaussian source model.
We...
We propose a novel dynamic gradient descent (DGD) framework integrated with reinforcement learning (RL) for AI-enhanced indoor environmental simulation, addressing the limitations of static optimization in dynamic settings. The proposed method combin...
The ISP (Internet Service Provider) industry relies heavily on internet traffic forecasting (ITF) for long-term business strategy planning and proactive network management. Effective ITF frameworks are necessary to manage these networks and prevent n...
Task offloading in satellite networks, which involves distributing computational tasks among heterogeneous satellite nodes, is crucial for optimizing resource utilization and minimizing system latency. However, existing approaches such as static offl...
This paper introduces a broad class of Mirror Descent (MD) and Generalized Exponentiated Gradient (GEG) algorithms derived from trace-form entropies defined via deformed logarithms. Leveraging these generalized entropies yields MD and GEG algorithms...
Focusing on the inverse synthetic aperture radar (ISAR) imaging of maneuvering targets, this paper presents a new imaging method which works well when the target’s maneuvering is not too severe. After translational motion compensation, we descr...
In pellet production, the uniformity of material distribution directly affects the subsequent roasting effect and the quality of finished products. Aiming at the problems of uneven distribution in traditional shuttle distribution systems, such as mat...
Quantum state tomography (QST) is one of the key steps in determining the state of the quantum system, which is essential for understanding and controlling it. With statistical data from measurements and Positive Operator-Valued Measures (POVMs), the...
In this paper, a discrete second order linear equation with the Krasnosel’skii-Pokrovskii (KP) operator is used to describe the piezoelectric actuated stage. The weights of the KP operators are identified by the gradient descent algorithm. To s...
In recent years, deep neural networks (DNN) have been widely used in many fields. Lots of effort has been put into training due to their numerous parameters in a deep network. Some complex optimizers with many hyperparameters have been utilized to ac...
Deep neural networks (DNNs) have achieved great success in the last decades. DNN is optimized using the stochastic gradient descent (SGD) with learning rate annealing that overtakes the adaptive methods in many tasks. However, there is no common choi...
This paper presents a frequency adaptive grid voltage sensorless control scheme of a grid-connected inductive–capacitive–inductive (LCL)-filtered inverter, which is based on an adaptive current controller and a grid voltage observer. The...
For inertial platforms with unknown model parameters and internal information, traditional model-free controllers fail to resist external vibrations solely based on the platform gyroscope, deteriorating the performance of inertial platforms. Therefor...
Quantum machine learning stands out as one of the most promising applications of quantum computing, widely believed to possess potential quantum advantages. In the era of noisy intermediate-scale quantum, the scale and quality of quantum computers ar...
Backpropagation (BP)-based gradient descent is the general approach to train a neural network with a multilayer perceptron. However, BP is inherently slow in learning, and it sometimes traps at local minima, mainly due to a constant learning rate. Th...
As a new machine learning technique, federated learning has received more attention in recent years, which enables decentralized model training across data silos or edge intelligent devices in the Internet of Things without exchanging local raw data....
Learning is a fundamental property of intelligent systems, observed across biological organisms and engineered systems. While modern intelligent systems typically rely on gradient descent for learning, the need for exact gradients and complex informa...
First, this paper recalls a recently introduced method of adaptive monitoring of dynamical systems and presents the most recent extension with a multiscale-enhanced approach. Then, it is shown that this concept of real-time data monitoring establishe...
An adaptive sliding mode control (ASMC) based on improved linear extended state observer (LESO) is proposed for nonlinear systems with unknown and uncertain dynamics. An improved LESO is designed to estimate total disturbance of the uncertain nonline...
Fault alarm time lag is one of the difficulties in fault diagnosis of wind turbine generators (WTGs), and the existing methods are insufficient to achieve accurate and rapid fault diagnosis of WTGs, and the operation and maintenance costs of WTGs are...
Adaptive digital back-propagation (A-DBP) is a promising candidate for mitigating Kerr nonlinearity due to its ability to estimate the optimal nonlinear scaling factor adaptively. However, the adaptive process relying on the gradient-dependent algori...
The traditional High-Resolution Range Profile (HRRP) target recognition method has difficulty automatically extracting target deep features, and has low recognition accuracy under low training samples. To solve these problems, a ship recognition meth...
of 4