Search Results (96)

In this paper, the problem of pitching attitude finite-horizon optimization for aircraft is posed with system uncertainties, external disturbances, and input constraints. First, a neural network (NN) and a nonlinear disturbance observer (NDO) are employed to estimate the value of system uncertainties and external disturbances. Taking input constraints into account, an auxiliary system is designed to compensate for the constrained input. Subsequently, the backstepping control containing NN and NDO is used to ensure the stability of systems and suppress the adverse effects caused by the system uncertainties and external disturbances. In order to avoid the derivation operation in the process of backstepping, a dynamic surface control (DSC) technique is utilized. Simultaneously, the estimations of the NN and NDO are applied to derive the backstepping control law. For the purpose of achieving finite-horizon optimization for pitching attitude control, an adaptive method termed adaptive dynamic programming (ADP) with a single NN-termed critic is applied to obtain the optimal control. Time-varying feature functions are applied to construct the critic NN in order to approximate the value function in the Hamilton–Jacobi–Bellman (HJB) equation. Furthermore, a supplementary term is added to the weight update law to minimize the terminal constraint. Lyapunov stability theory is used to prove that the signals in the control system are uniformly ultimately bounded (UUB). Finally, simulation results illustrate the effectiveness of the proposed finite-horizon optimal attitude control method. Full article

This paper investigates a multiple unmanned aerial vehicle (UAV) enabled network for supporting emergency communication services, where each drone acts as a base station (also called the drone small cell (DSC)). The novelty of this paper is that a mean field game (MFG)-based strategy is conceived for jointly controlling the three-dimensional (3D) locations of these drones to guarantee the distortion requirement of lossy communications, while considering the inter-cell interference and the flight energy consumption of drones. More explicitly, we derive the Hamilton–Jacobi–Bellman (HJB) and Fokker–Planck–Kolmogorov (FPK) equations, and propose an algorithm where both the Lax–Friedrichs scheme and the Lagrange relaxation are invoked for solving the HJB and FPK equations with 3D control vectors and state vectors. The numerical results show that the proposed algorithm can achieve a higher access rate with a similar flight energy consumption. Full article

The shipping industry, a cornerstone of global trade, faces emissions reduction challenges amid tightening environmental policies. Blockchain technology, leveraging distributed symmetric architectures, enhances supply chain transparency by reducing information asymmetry, yet its dynamic interplay with emissions strategies remains underexplored. This study employs symmetry-driven differential game theory to model four blockchain scenarios in port-shipping networks: no blockchain (N), port-led (PB), shipping company-led (CB), and a joint platform (FB). By solving Hamilton–Jacobi–Bellman equations, we derive optimal emissions reduction efforts, green investments, and blockchain strategies under symmetric and asymmetric decision-making frameworks. Results show blockchain adoption improves emissions reduction and service quality under cost thresholds, with port-led systems maximizing low-cost profits and shipping firms gaining asymmetrically in high-freight contexts. Joint platforms achieve symmetry in profit distribution through fee-trust synergy, enabling win–win outcomes. Integrating graph-theoretic principles, we have designed dynamic state equations for emissions and service levels, segmenting shippers by low-carbon preferences. This work bridges dynamic emissions strategies with blockchain’s network symmetry, fostering economic–environmental synergies to advance sustainable maritime supply chains. Full article

This article investigates the inverse optimal fault-tolerant formation-containment control problem for a group of unmanned helicopters, where the leaders form a desired formation pattern under the guidance of a virtual leader while the followers move toward the convex hull established by leaders. To facilitate control design and stability analysis, each helicopter’s dynamics are separated into an outer-loop (position) and an inner-loop (attitude) subsystem by exploiting their multi-time-scale characteristics. Next, the serial-parallel estimation model, designed to account for prediction error, is developed. On this foundation, the composite updating law for network weights is derived. Using these intelligent approximations, a fault estimation observer is constructed. The estimated fault information is further incorporated into the inverse optimal fault-tolerant control framework that avoids tackling either the Hamilton–Jacobi–Bellman or Hamilton–Jacobi–Issacs equation. Finally, simulation results are presented to demonstrate the superior control performance and accuracy of the proposed method. Full article

This paper is devoted to the study of stochastic optimal control of averaged stochastic differential delay equations (SDDEs) with semi-Markov switchings and their applications in economics. By using the Dynkin formula and solution of the Dirichlet–Poisson problem, the Hamilton–Jacobi–Bellman (HJB) equation and the inverse HJB equation are derived. Applications are given to a new Ramsey stochastic models in economics, namely the averaged Ramsey diffusion model with semi-Markov switchings. A numerical example is presented as well. Full article

This paper investigates Mean Field Game methods to solve missile interception strategies in three-dimensional space, with a focus on analyzing the pursuit–evasion problem in many-to-many scenarios. By extending traditional missile interception models, an efficient solution is proposed to avoid dimensional explosion and communication burdens, particularly for large-scale, multi-missile systems. The paper presents a system of stochastic differential equations with control constraints, describing the motion dynamics between the missile (pursuer) and the target (evader), and defines the associated cost function, considering proximity group distributions with other missiles and targets. Next, Hamilton–Jacobi–Bellman equations for the pursuers and evaders are derived, and the uniqueness of the distributional solution is proved. Furthermore, using the $ϵ$-Nash equilibrium framework, it is demonstrated that, under the MFG model, participants can deviate from the optimal strategy within a certain tolerance, while still minimizing the cost. Finally, the paper summarizes the derivation process of the optimal strategy and proves that, under reasonable assumptions, the system can achieve a uniquely stable equilibrium, ensuring the stability of the strategies and distributions of both the pursuers and evaders. The research provides a scalable solution to high-risk, multi-agent control problems, with significant practical applications, particularly in fields such as missile defense systems. Full article

Given the increasing global emphasis on sustainable energy usage and the rising energy demands of cellular wireless networks, this work seeks an optimal short-term, continuous-time power-procurement schedule to minimize operating expenditure and the carbon footprint of cellular wireless networks equipped with energy-storage capacity, and hybrid energy systems comprising uncertain renewable energy sources. Despite the stochastic nature of wireless fading channels, the network operator must ensure a certain quality-of-service (QoS) constraint with high probability. This probabilistic constraint prevents using the dynamic programming principle to solve the stochastic optimal control problem. This work introduces a novel time-continuous Lagrangian relaxation approach tailored for real-time, near-optimal energy procurement in cellular networks, overcoming tractability problems associated with the probabilistic QoS constraint. The numerical solution procedure includes an efficient upwind finite-difference solver for the Hamilton–Jacobi–Bellman equation corresponding to the relaxed problem, and an effective combination of the limited memory bundle method (LMBM) for handling nonsmooth optimization and the stochastic subgradient method (SSM) to navigate the stochasticity of the dual problem. Numerical results, based on the German power system and daily cellular traffic data, demonstrate the computational efficiency of the proposed numerical approach, providing a near-optimal policy in a practical timeframe. Full article

In recent years, high-frequency trading has become increasingly popular in financial markets, making the dynamic modeling of the limit book and the optimization of market maker strategies become key topics. However, existing studies often lacked detailed descriptions of order books and failed to fully characterize the optimal decisions of market makers in complex market environments, especially in China’s A-share market. Based on Markov queue theory, this paper proposes the dynamic model of the limit order and the optimal strategy of the market maker. The model uses a state transition probability matrix to refine the market diffusion state, order generation, and trading process and incorporates indicators such as optimal quote deviation and restricted order trading probability. Then, the optimal control model is constructed and the reference strategy is derived using the Hamilton–Jacobi–Bellman (HJB) equation. Then, the key parameters are estimated using the high-frequency data of Ping An Bank for a single trading day. In the empirical aspect, the six-month high-frequency trading data of 114 representative stocks in different market states such as the bull market and bear market in China’s A-share market were selected for strategy verification. The results showed that the proposed strategy had robust returns and stable profits in the bull market and that frequent capture of market fluctuations in the bear market can earn relatively high returns while maintaining 50% of the order coverage rate and 66% of the stable order winning rate. Our study used Markov queuing theory to describe the state and price dynamics of the limit order book in detail and used optimization methods to construct and solve the optimal market maker strategy. The empirical aspect broadens the empirical scope of market maker strategies in the Chinese market and studies the stability and effectiveness of market makers in different market states. Full article

Optimal control theory aims to find an optimal protocol to steer a system between assigned boundary conditions while minimizing a given cost functional in finite time. Equations arising from these types of problems are often non-linear and difficult to solve numerically. In this article, we describe numerical methods of integration for two partial differential equations that commonly arise in optimal control theory: the Fokker–Planck equation driven by a mechanical potential for which we use the Girsanov theorem; and the Hamilton–Jacobi–Bellman, or dynamic programming, equation for which we find the gradient of its solution using the Bismut–Elworthy–Li formula. The computation of the gradient is necessary to specify the optimal protocol. Finally, we give an example application of the numerical techniques to solving an optimal control problem without spacial discretization using machine learning. Full article

This paper is dedicated to studying the optimal investment proportions of three types of assets with symmetry, namely, risky assets, risk-free assets, and wealth management products, when the stochastic expenditure process follows a jump-diffusion model. The stochastic expenditure process is treated as an exogenous cash flow and is assumed to follow a stochastic differential process with jumps. Under the Cox–Ingersoll–Ross interest rate term structure, it is presumed that the prices of multiple risky assets evolve according to a multi-dimensional geometric Brownian motion. By employing stochastic control theory, the Hamilton–Jacobi–Bellman (HJB) equation for the household portfolio problem is formulated. Considering various risk-preference functions, particularly the Hyperbolic Absolute Risk Aversion (HARA) function, and given the algebraic form of the objective function through the terminal-value maximization condition, an explicit solution for the optimal investment strategy is derived. The findings indicate that when household investment behavior is characterized by random expenditures and symmetry, as the risk-free interest rate rises, the optimal proportion of investment in wealth-management products also increases, whereas the proportion of investment in risky assets continually declines. As the expected future expenditure increases, households will decrease their acquisition of risky assets, and the proportion of risky-asset purchases is sensitive to changes in the expectation of unexpected expenditures. Full article

In the digital economy era, information symmetry, transparency, and traceability in food supply chains have increasingly garnered consumer attention. To motivate supply chain members to engage in product traceability, this paper examines the competitive and cooperative dynamics among participants in the food supply chain over continuous time. By developing a differential game model involving manufacturers and retailers with three decision-making modes, we solve the model using the Hamilton–Jacobi–Bellman (HJB) equation and perform a simulation analysis to assess the impact of different modes on overall supply chain profits. Additionally, we analyze how various parameters affect the profits of manufacturers and retailers. The key findings of this study indicate that centralized decision-making enhances the overall benefits of the food supply chain. Among the three decision-making models, the cost-sharing model proves to be the optimal approach, as it leads to a Pareto improvement in the profits of both manufacturers and retailers. These conclusions provide valuable insights for supply chain members seeking to optimize product traceability and enhance supply chain efficiency, as well as for government authorities involved in traceable supply chain governance. Full article

We present a finite difference method working on sparse grids to solve higher dimensional heterogeneous agent models. If one wants to solve the arising Hamilton–Jacobi–Bellman equation on a standard full grid, one faces the problem that the number of grid points grows exponentially with the number of dimensions. Discretizations on sparse grids only involve $\mathcal{O}\left(N{(logN)}^{d-1}\right)$ degrees of freedom in comparison to the $\mathcal{O}\left(N^d\right)$ degrees of freedom of conventional methods, where N denotes the number of grid points in one coordinate direction and d is the dimension of the problem. While one can show convergence for the used finite difference method on full grids by using the theory introduced by Barles and Souganidis, we explain why one cannot simply use their results for sparse grids. Our numerical studies show that our method converges to the full grid solution for a two-dimensional model. We analyze the convergence behavior for higher dimensional models and experiment with different sparse grid adaptivity types. Full article

In this paper, we address the utility maximization problem of an infinitely lived agent who has the option to increase their income. The agent can increase their income at any time, but doing so incurs a wealth cost proportional to the amount of the increase. To prevent the agent from infinitely increasing their income and borrowing against future income, we additionally consider a non-negative wealth constraint that prohibits borrowing based on future income. This utility maximization problem is a mixture of stochastic control, where the agent chooses consumption and investment, and singular control, where the agent chooses a non-decreasing income process. To solve this non-trivial and challenging problem, we derive the Hamilton–Jacobi–Bellman (HJB) equation with a gradient constraint using the dynamic programming principle (DPP). Then, using the guess-and-verify method and a linearization technique, we obtain a closed-form solution to the HJB equation and, based on this, find the optimal strategy. Full article

This paper presents a novel approach to modeling the repertoire of the immune system and its adaptation in response to the evolutionary dynamics of pathogens associated with their genetic variability. It is based on application of a dynamic programming-based framework to model the antigen-driven immune repertoire synthesis. The processes of formation of new receptor specificity of lymphocytes (the growth of their affinity during maturation) are described by an ordinary differential equation (ODE) with a piecewise-constant right-hand side. Optimal control synthesis is based on the solution of the Hamilton–Jacobi–Bellman equation implementing the dynamic programming approach for controlling Gaussian random processes generated by a stochastic differential equation (SDE) with the noise in the form of the Wiener process. The proposed description of the clonal repertoire of the immune system allows us to introduce an integral characteristic of the immune repertoire completeness or the integrative fitness of the whole immune system. The quantitative index for characterizing the immune system fitness is analytically derived using the Feynman–Kac–Kolmogorov equation. Full article

This paper is devoted to new propagation and regularity results for a class of first-order Hamilton–Jacobi–Bellman-type problems in a separable infinite-dimensional Hilbert space. Specifically, the related Cauchy problem is investigated by employing the Faedo–Galerkin approximation method. Under some structural assumptions, the main result is obtained by using the probabilistic representation formula of the solution in order to define the weak continuity assumptions, by assuming the existence of a symmetric positive definite Hilbert–Schmidt operator and by employing modulus continuity arguments. Full article