Survey of Neurodynamic Methods for Control and Computation in Multi-Agent Systems

Abstract

Neurodynamics is recognized as a powerful tool for addressing various problems in engineering, control, and intelligent systems. Over the past decade, neurodynamics-based methods and models have been rapidly developed, particularly in emerging areas such as neural computation and multi-agent systems. In this paper, we provide a brief survey of neurodynamics applied to computation and multi-agent systems. Specifically, we highlight key models and approaches related to time-varying computation, as well as cooperative and competitive behaviors in multi-agent systems. Furthermore, we discuss current challenges, potential opportunities, and promising future directions in this evolving field.

1. Introduction

Neurodynamics is a powerful tool for handling various problems, which is inspired by the working principles of human brains. The early work on neurodynamics can be traced back to the 1980s when Hopfield proposed a famous neural network, which was later called the Hopfield neural network. Sometimes, neurodynamics is also referred to as recurrent neural networks when we are concerned about its architecture of neuron connections compared to the traditional forward neural networks that are widely used in pattern recognition [1,2,3,4]. Neurodynamics is also referred to as dynamic neural networks when we are concerned with its dynamic features, as neurodynamic models or methods are characterized by differential equations. Neurodynamics has a strong connection with cybernetics or control theory since feedback plays an important role in neurodynamic methods or models, which ensures that such methods or models can have a theoretically guaranteed performance, such as globally asymptotic convergence. Neurodynamics, together with forward neural networks [5,6,7], fuzzy systems [8,9], evolutionary computation [10,11,12,13,14,15,16], and meta-heuristic algorithms [17,18,19] are the major components of computational intelligence. Recently, some researchers tried to use neurodynamics to boost the performance of forward neural networks [20,21]. This field now plays a significant role in bridging biological inspirations and engineering implementations, especially in contexts where traditional static neural architectures fall short in solving dynamic, time-sensitive, or optimization-intensive problems.

Neurodynamics has been applied to the location of underwater acoustic sensor networks [22], resource allocation for end–edge collaboration in the internet of vehicles [23], portfolio analysis in high-frequency trading [24], path planning [25,26,27], and motion control of redundant manipulators [28,29,30,31,32,33]. In particular, the theory and application of neurodynamics for computation have been extensively investigated in the past decade. The basic idea of neurodynamics for handling computation problems is by developing neurodynamic models whose equilibrium include the solution of the considered problems, and a proper design can ensure that the equilibrium is globally asymptotically stable. With suitable activation functions acting as nonlinear mapping of feedback information, some neurodynamic models can have finite-time stability by which some state variables can converge to the solutions of the computation problems within finite time [34,35,36,37,38]. An interesting part of neurodynamics for computation is its extendability to various applications stated above by converting the problems in the applications into computation problems like linear equations, matrix inversion problems, or optimization problems. Furthermore, compared with traditional numerical solvers, neurodynamic models possess strong capabilities in terms of parallel implementation, robustness against noise, and real-time convergence, which have made them particularly attractive in embedded control and autonomous decision-making scenarios. Recent works have further explored fixed-time and finite-time neural solvers [39], neural ODE frameworks [40], and biologically plausible dynamics that aim to generalize neurodynamics to more complex nonlinear and time-varying systems. Meanwhile, neurodynamics has also been applied to multi-agent systems. In a multi-agent system, there are multiple agents with sensing, computation, communication, and actuation capabilities, where some fundamental problems for such systems include their distributed optimization and control. In terms of distributed optimization, agents in a multi-agent system try to solve an optimization problem by local interaction with neighboring agents who have communication links with them [41]. In terms of distributed control, a fundamental problem in multi-agent systems is the consensus problem which requires that certain state variables of the agents can achieve common values with local negotiations [42].

The integration of neurodynamics with multi-agent systems offers a promising framework to model decentralized intelligence and emergent collective behaviors. This synergy is particularly important for large-scale systems such as smart grids, autonomous vehicle fleets, and sensor swarms, where coordination must be achieved with limited global information and under stringent real-time constraints. Neurodynamic models enable each agent to evolve its internal state based on local observations and interactions, thereby achieving system-level objectives through local computation. Despite the rapid progress and increasing interest in this area, a comprehensive survey that systematically reviews the intersection of neurodynamics and both computational theory and multi-agent systems is still lacking. Previous reviews have largely focused on static neural networks or specific application domains, leaving a gap in unified understanding across neurodynamic models, convergence theory, application scenarios, and emerging challenges.

Thus, in this paper, we aim at providing a brief survey of this topic by introducing the related methods and models, based on which we also outline some interesting topics worth further investigation. The main contributions of this survey are listed as follows.

(1)

Methods and models of neurodynamics for computation and multi-agent systems proposed in the past decade as well as their related engineering applications are analyzed and compared.

(2)

Challenges and opportunities in the area of neurodynamics for computation and multi-agent systems are discussed.

The rest of this paper is organized as follows. Some background knowledge for neurodynamics and multi-agent systems is given in Section 2. The advances of neurodynamics for computation and multi-agent systems in the past decade are reviewed in Section 3 and Section 4, respectively. Section 5 identifies research opportunities and challenges in the considered area. Then, Section 6 concludes this paper.

2. Preliminary

In this section, we provide background knowledge that can be useful for newcomers in the area of neurodynamics and multi-agent systems. This includes a review of the mathematical formulation of neurodynamic systems, graph-theoretic tools for describing inter-agent connectivity, and basic concepts underpinning stability and convergence analysis.

2.1. Notations for Neurodynamics

Let $\mathbb{R}$ denote the set of real numbers and ${\mathbb{R}}^n$ the vector field of n-dimensional vectors whose elements are real numbers. Neurodynamics is characterized by the dynamics of the neurons. Let $\mathbf{x}={[{\mathbf{x}}_1,{\mathbf{x}}_2,\cdots ,{\mathbf{x}}_n]}^{\mathrm{T}}\in {\mathbb{R}}^n$ denote the state vector of the neurons in a dynamic neural network with $x_i$ being the state variable of the i-th neuron, the dynamics of the neurons is represented by the following differential equation:

$$\epsilon \dot{\mathbf{x}}\left(t\right)=f(t,\mathbf{x}\left(t\right)),$$

(1)

where $\epsilon >0\in \mathbb{R}$ is a gain parameter scaling the state evolution speed of the neurons, t is a virtual time variable, $\dot{\mathbf{x}}=\mathrm{d}\mathbf{x}/\mathrm{d}t$ is the derivative of the state vector with respect to time t, and $f(·):{\mathbb{R}}^{n+1}\to {\mathbb{R}}^n$ is the mapping of evolution of the neuron states. For different neurodynamic models, the difference lies in the number of neurons and the definition of the mapping $f(·)$. For the convenience of presentation, in this paper, the time variable t is omitted whenever there is no confusion. As seen in (1) of the general neurodynamic model, the evolution of state vector $\mathbf{x}$ of the neurons is a result of its previous values, which is called feedback in control theory or cybernetics. It is the existence of this feedback information that makes neurodynamics essentially differential from traditional forward neural networks. Typically, equilibrium points of (1) are designed to correspond to solutions of specific computational problems such as optimization or equation solving. In the area of neurodynamics, stability of the neurodynamic models is essential. The stability analysis is often based on the Lypuanov stability theory.

2.2. Algebraic Graph Theory

When dealing with the coordination or competition of multi-agent systems, it is necessary to model the communication topology of the agents and the algebraic graph theory is a tool to achieve this goal. Depending on the characteristics of the communication topologies, different types of graphs may be used. In this modeling process, an agent in a multi-agent system is viewed as a node in a graph, and there is an edge between two nodes if there is a communication channel between the corresponding two agents. Consider an undirected graph $\mathbb{G}=\{\mathbb{V},\mathbb{E}\}$ consisting of N nodes, where $\mathbb{V}=\{1,2,\cdots ,N\}$ is the set of nodes and $\mathbb{E}$ is the set of edges where each edge $(i,j)$ means that there is a bidirectional communication link between agent i and agent j. The weight of the edge $(i,j)$ is denoted by $w_{ij}$, which is identical to $a_{ji}$ for an undirected graph. Usually, $w_{ij}>0$ if the corresponding edge exits, and $a_{ij}=0$ if the corresponding edge does not exist while $w_{ii}=0$, since self-loop is not allowed. The weights $w_{ij}$ form a matrix W called the weight matrix. Based on the definition of the weight matrix, a degree matrix can be defined by a diagonal matrix D with its diagonal elements represented by $d_{ii}={\sum }_{j=1}^N{w}_{ij}$. There is a matrix widely used in graph analysis, which is called the Laplacian matrix and denoted by $L=D-W$. The set of neighbors of an agent i is denoted by $\mathbb{N}\left(i\right)=\{j\in \mathbb{V}|a_{ij}>0\}$. The undirected graph $\mathbb{G}$ is connected if there is a path between any two nodes, which is equivalent to the positiveness of the algebraic connectivity of the graph. The algebraic connectivity is defined by the second smallest eigenvalues of the Laplacian matrix [43].

3. Neurodynamics for Computation

Basically, there are three types of neurodynamics that are widely encountered, i.e., gradient neurodynamics, zeroing neurodynamics, and projection neurodynamics.

3.1. Gradient Neurodynamics

The gradient neurodynamics is constructed based on gradient descent, whose early version is the Hopfield neural network. Gradient neurodynamics is widely considered a powerful tool for handling static computation problems whose theoretical solutions do not change over time t. However, its application scope is traditionally limited to static systems, and the extension to time-varying problems remains nontrivial and often problem-specific.

The design of gradient neurodynamics to solve computation problems contains three steps. Firstly, an energy function should be designed. The energy function is a nonnegative function with respect to the unknown variable of the computation problem, and its value is zero only if the unknown variable converges to the theoretical solution. Secondly, the gradient descent method is used to obtain the evolution rule of the unknown variable such that the energy function vanishes to zero over time. Finally, activation functions are designed to accelerate the convergence of the gradient neurodynamic model. While these steps are systematic and mathematically elegant, they tend to rely on heuristic choices of energy and activation functions, which may not generalize well across problem types. As an example to illustrate the design of gradient neurodynamic models, consider the linear equation as follows:

$$A\mathbf{x}=\mathbf{b},$$

(2)

where $A\in {\mathbb{R}}^{m\times n}$ and $\mathbf{b}\in {\mathbb{R}}^m$ are the known coefficient matrix and vector, respectively, and $\mathbf{x}\in {\mathbb{R}}^n$ is the unknown vector to be found. An energy function for the linear equation is selected as

$$\mathcal{E}={\displaystyle \frac{1}{2}}{\parallel A\mathbf{x}-\mathbf{b}\parallel }_2^2,$$

(3)

where ${\parallel ·\parallel }_2$ denotes the 2-norm of a vector. The gradient of the energy function is

$${\displaystyle \frac{\partial \mathcal{E}}{\partial \mathbf{x}}}=A^{\mathbf{T}}(A\mathbf{x}-\mathbf{b}).$$

(4)

Starting from a randomly generated initial state $\mathbf{x}\left(0\right)$, to make $\mathbf{x}$ move to the theoretical solution, the following evolution rule is adopted:

$$\epsilon \dot{\mathbf{x}}=-A^{\mathbf{T}}(A\mathbf{x}-\mathbf{b}),$$

(5)

where $\epsilon >0\in \mathbb{R}$ is the gain parameter. The gradient neurodynamic model activated by nonlinear activation functions is further developed as follows [44]:

$$\epsilon \dot{\mathbf{x}}=-A^{\mathbf{T}}\varphi (A\mathbf{x}-\mathbf{b}),$$

(6)

where $\varphi (·)$ is the activation function, i.e., an element-wise nonlinear mapping, which is monotonically increasing and odd, for which $\varphi \left(\mathbf{x}\right)={[\varphi \left(x_1\right),\varphi \left(x_2\right),\cdots ,\varphi \left(x_n\right)]}^{\mathrm{T}}$.

Existing research has revealed that with selected nonlinear activation functions, the convergence speed of neurodynamic models can be accelerated. For linear Equation (2), without using nonlinear activation functions, exponential convergence can be achieved for which the convergence time is infinite and we have

$$\underset{t\to +\infty }{lim}\mathbf{x}\left(t\right)={\mathbf{x}}^{\ast },$$

(7)

where ${\mathbf{x}}^{\ast }$ denotes the theoretical solution of linear Equation (2). Exponential convergence implies asymptotic behavior with an infinite horizon, which may not be acceptable in real-time or embedded computing scenarios. However, with properly chosen activation functions, finite-time convergence can be achieved, for which

$$\underset{t\to t_f}{lim}\mathbf{x}\left(t\right)={\mathbf{x}}^{\ast },$$

(8)

where $t_f>0$ is a finite value. Evidently, finite-time convergence is more favorable than exponential convergence since the convergence time is finite. There is a particular case for which the convergence time $t_f$ is independent of the initial state $\mathbf{x}\left(0\right)$ and this case is referred to as fixed-time convergence. In the past two decades, much effort has been devoted to the development of activation functions to accelerate the convergence speed of neurodynamic models. A comparison of activation functions is shown in

Table 1. Comparison of activation functions in neurodynamics.

Literature	Activation Function
[34]	${\displaystyle \varphi (x)=\left\{\begin{array}{c}{x}^p,\mathrm{if}\left|x\right|>1,\hfill \\ \frac{1-exp(-\epsilon x)}{1+exp(-\epsilon x)}·\frac{1+exp(-\epsilon )}{1-exp(-\epsilon )}\mathrm{if}\left|x\right|\le 1\hfill \end{array}\right.}$
[45]	$\varphi (x)=\left(\right|x{|}^{\alpha }+\left|x{|}^{1/\alpha }\right)\mathrm{sign}(x)$
[46]	$\varphi (x)=\left({\rho }_1\right|x{|}^{\alpha }+{\rho }_2\left|x{|}^{\nu }\right)\mathrm{sign}(x)$
[44]	$\varphi (x)=({\rho }_1{\left|x\right|}^{\alpha }+{\rho }_2{\left|x\right|}^{\nu })\mathrm{sign}(x)+{\rho }_{3}x+{\rho }_4\mathrm{sign}(x)$
[47]	${\displaystyle \varphi (x)=\frac{\eta }{c}{exp\left(\right|x|}^c{\left)\right|x|}^{1-c}\mathrm{sign}(x)+\nu \mathrm{sign}(x)}$
[44]	$\varphi (x)=\left\{\begin{array}{c}{\rho }_1{\left|x\right|}^{\alpha }\mathrm{sign}\left(x\right)+{\rho }_{3}x,\mathrm{if}\left|x\right|\le 1\hfill \\ {\rho }_2{\left|x\right|}^{\nu }\mathrm{sign}\left(x\right)+{\rho }_{3}x,\mathrm{otherwise}.\hfill \end{array}\right.$
[44]	$\varphi (x)=({\rho }_1{\left|x\right|}^{\alpha }+{\rho }_2{\left|x\right|}^{\nu })\mathrm{sign}(x)+{\rho }_{3}x$
[48]	$\varphi (x)={\sum }_{k=1}^3{x}^{2k-1}$
[49]	$\varphi (x)={\displaystyle \frac{\eta }{c}}{exp\left(\right|x|}^c{\left)\right|x|}^{1-c}sign(x)+\nu sign(x)$
[50]	$\left(c_1{\sum }_{i=1}^m{\left|e\right|}^{{\nu }_i}+c_2{\sum }_{i=1}^m{\left|e\right|}^{{\kappa }_i}\right)sgn(e)+c_{3}e+c_{4}sinh(e)$

. Yet, the increasing complexity of activation functions often introduces additional computational burdens, and may compromise numerical stability or hardware implementability.

Based on the above framework, gradient neurodynamic models have been developed to handle various static computation problems. Conventionally, it is known that gradient neurodynamic models following the above design steps cannot accurately solve time-varying computation problems whose theoretical solutions vary over time no matter what activation functions are adopted. Hopefully, recent work indicates that by using norm correction, gradient neurodynamic models can indeed solve time-varying computation problems accurately and robustly with finite-time convergence. Consider the following time-varying linear equation:

$$A\left(t\right)\mathbf{x}\left(t\right)=\mathbf{b}\left(t\right),$$

(9)

where $A\left(t\right)\in {\mathbb{R}}^{m\times n}$ and $\mathbf{b}\left(t\right)\in {\mathbb{R}}^m$ are the known time-varying smooth coefficient matrix and vector, respectively, and $\mathbf{x}\left(t\right)\in {\mathbb{R}}^n$ is the unknown vector to be found. Since the parameters vary over time, the theoretical solution of the equation is a function of time t. A finite-time convergent gradient neurodynamic model based on norm correction is given as follows [51]:

$${\epsilon \parallel A\left(t\right)\mathbf{x}\left(t\right)-\mathbf{b}\left(t\right)\parallel }_2\dot{\mathbf{x}}=-A^{\mathbf{T}}\left(t\right)(A\left(t\right)\mathbf{x}\left(t\right)-\mathbf{b}\left(t\right)).$$

(10)

It has been proved in [51] that the state vector $\mathbf{x}\left(t\right)$ of the gradient neurodynamic model (10) converges to the theoretical solution of the time-varying linear Equation (9) in finite time even in the presence of bounded additive noise if the parameter $\epsilon$ is large enough. The norm correction method is quite interesting as it provide another way to improve the performance of neurodynamic models. Compared with the complicated nonlinear activation functions shown in Table 1, the norm correction method makes it feasible to design finite-time convergence neurodynamic models with a much lower computational burden. Currently, gradient neurodynamic models based on norm correction have also been developed for time-varying matrix inversion [52], time-varying Moore–Penrose inversion [53], and time-varying Lyapunov equations [54]. The combination of norm correction with the Daivdenko method for time-varying nonlinear equations was also reported [55].

The gradient neurodynamic model based on norm correction relies on the choice of the parameter $\epsilon$ based on some prior knowledge about the time-varying problem, but they do not need accurate feedback of real-time derivatives of the parameters in the time-varying problems, such as the derivatives of $A\left(t\right)$ and $\mathbf{b}\left(t\right)$ with respect to time t in the time-varying linear Equation (9). To some extent, the norm correction provides a mechanism that is a bit similar to sliding mode control, where a sufficient gain can compensate for bounded unknown disturbances.

3.2. Zeroing Neurodynamics

Zeroing neurodynamics is currently the mainstream method for solving time-varying computation problems whose solutions vary over time, and lots of zeroing neurodynamic models have been developed for solving a variety of time-varying computation problems since the first zeroing neurodynamic model was developed in 2002 [56]. The core idea of zeroing neurodynamic methods is to convert a computation problem into a zero-finding problem, where preferred dynamics of a selected error function is developed to ensure that the error function converges to zero. The first step for developing zeroing neurodynamic models is to convert the computation problem into an equation problem, which can be a linear equation or nonlinear equation. The second step is the selection of the error function for the equation. The third step is to design an error dynamics for the error function to make it converge. Then, similar to gradient neurodynamics, nonlinear activation functions can be added to the derived zeroing neurodynamics to further accelerate convergence. Although this approach is systematic and widely applicable, it often relies heavily on the careful hand-design of error dynamics and activation functions, which may lack generalizability and require heuristic tuning in different application scenarios. Consider time-varying linear Equation (9) as an example. To develop a zeroing neurodynamic models for solving (9), we can select an error function as follows:

$$\mathbf{e}=A\left(t\right)\mathbf{x}\left(t\right)-\mathbf{b}\left(t\right).$$

(11)

Then, the following error dynamics with nonlinear activation functions can be used to ensure the convergence of $\mathbf{e}\left(t\right)$:

$$\dot{\mathbf{e}}=-\lambda \varphi \left(\mathbf{e}\right),$$

(12)

where $\lambda >0\in \mathbb{R}$ is the gain parameter. Substituting (11) into (12), the following zeroing neurodynamic model is obtained for solving time-varying linear Equation (9) [57,58,59,60]:

$$A\left(t\right)\dot{\mathbf{x}}=-\lambda \varphi (A\left(t\right)\mathbf{x}\left(t\right)-\mathbf{b}\left(t\right))-\dot{A}\left(t\right)\mathbf{x}\left(t\right).$$

(13)

In view of the above steps, different error dynamics, error functions, or activation functions give rise to different zeroing neurodynamic models for solving a time-varying computation problem. In the past decade, lots of error has been devoted to the design of error dynamics, error functions, and activation functions for boosting the performance of zeroing neurodynamic models. As (13), zeroing neurodynamic models are represented by implicit dynamics, for which there is a mass matrix in the different equation, such as $A\left(t\right)$ for $\dot{\mathbf{x}}$. It should be noted that this kind of dynamics can be realized in analog circuits by following the Kirchhoff circuit laws. However, in the implementation of such models in digital computers, additional effort is needed, yielding models with a much complex structure. For example, for the zeroing neurodynamic model (13), when $A\left(t\right)$ is a rectangle matrix with $m\ne n$, the following model is used for the implementation of (13) in digital computers:

$$\dot{\mathbf{x}}=-\lambda {\left(A^{\mathrm{T}}\left(t\right)A\left(t\right)\right)}^{-1}{A}^{\mathrm{T}}\left(t\right)\varphi (A\left(t\right)\mathbf{x}\left(t\right)-\mathbf{b}\left(t\right))-\dot{A}\left(t\right)\mathbf{x}\left(t\right),$$

(14)

which works for the case where $A\left(t\right)$ is full rank.

A time-varying nonlinear equation is given as follows:

$$f(t,\mathbf{x}(t\left)\right)=0,$$

(15)

where $f(·):{\mathbb{R}}^n\to {\mathbb{R}}^n$ is a smooth nonlinear function, and $\mathbf{x}\left(t\right)\in {\mathbb{R}}^n$ is an unknown vector. Based on the idea of zeroing neurodynamics, the following error function can be selected:

$$\mathbf{e}\left(t\right)=f(t,\mathbf{x}).$$

(16)

Then, by using (12), a zeroing neurodynamic model for solving time-varying nonlinear Equation (15) is derived as follows [61,62]:

$$J(t,x)\dot{\mathbf{x}}=-\lambda \varphi \left(f(t,\mathbf{x})\right)-{\displaystyle \frac{\partial f(t,\mathbf{x})}{\partial t}},$$

(17)

where $J(t,\mathbf{x})=\partial f(t,\mathbf{x})/\partial \mathbf{x}$. To deal with disturbance during the implementation of zeroing neurodynamic models for time-varying nonlinear equations, Li et al. [63] further proposed the following error dynamics:

$$\dot{\mathbf{e}}=-\alpha \mathsf{\Phi }\left(\mathbf{e}\right)-\beta {\int }_0^t\mathsf{\Psi }\left(\mathbf{e}\left(\tau \right)\right)\mathrm{d}\tau ,$$

(18)

where $\mathsf{\Phi }(·)$ and $\mathsf{\Psi }(·)$ are element-wise activation functions like $\varphi (·)$ while $\alpha >0\in \mathbb{R}$ and $\beta >0\in \mathbb{R}$ are gain parameters. Based on the error dynamics (18), the following zeroing neurodynamic model is developed for the time-varying nonlinear equation above:

$$J(t,x)\dot{\mathbf{x}}=-\alpha \mathsf{\Phi }\left(f(t,\mathbf{x})\right)-{\displaystyle \frac{\partial f(t,\mathbf{x})}{\partial t}}-\beta {\int }_0^t\mathsf{\Phi }\left(f(\tau ,\mathbf{x}\left(\tau \right))\right)\mathrm{d}\tau .$$

(19)

To remove the requirement of complicated nonlinear activation functions for finite-time convergence and the integral operation for improving robustness, Dai et al. [64] proposed the following error dynamics to develop zeroing neurodynamic models for solving the time-varying nonlinear equation above:

$${\parallel \mathbf{e}\parallel }_2\dot{\mathbf{e}}=-\lambda \mathbf{e},$$

(20)

by which the corresponding zeroing neurodynamic model is given as

$${\parallel f(t,\mathbf{x})\parallel }_{2}J(t,x)\dot{\mathbf{x}}=-\lambda \varphi \left(f(t,\mathbf{x})\right)-{\parallel f(t,\mathbf{x})\parallel }_2{\displaystyle \frac{\partial f(t,\mathbf{x})}{\partial t}}.$$

(21)

It has been proved in [64] that the zeroing neurodynamic model (21) has finite-time convergence for solving time-varying nonlinear equations even if there are bounded disturbances. Although this approach achieves finite-time convergence under bounded disturbances, it remains sensitive to vanishing error norms near convergence, potentially resulting in numerical stiffness or discontinuities in digital implementation.

By following the idea of zeroing neurodynamics for addressing linear equations and nonlinear equations, zeroing neurodynamics has also been applied to time-varying matrix inversion [65,66,67,68,69,70,71], time-varying Moore–Penrose inversion [72,73], and time-varying matrix equations [74,75,76,77,78,79,80,81,82]. A general time-varying matrix equation is given as

$$A\left(t\right)\mathbf{x}B\left(t\right)=C\left(t\right),$$

(22)

with $A\left(t\right)$, $B\left(t\right)$, and $C\left(t\right)$ denoting the smooth dynamic coefficient matrices, for which the error function can be defined as

$$E\left(t\right)=A\left(t\right)X\left(t\right)B\left(t\right)-C\left(t\right)$$

(23)

and the error dynamics can be designed as

$$\dot{E}=-\lambda \varphi \left(E\right).$$

(24)

Then, the following zeroing neurodynamic model can be obtained for the above matrix equation:

$$\begin{array}{cc}\hfill A\left(t\right)\dot{X}\left(t\right)B\left(t\right)=& -\lambda \varphi (A\left(t\right)X\left(t\right)B\left(t\right)-C\left(t\right))-\dot{A}\left(t\right)X\left(t\right)B\left(t\right)\hfill \\ \hfill & -A\left(t\right)X\left(t\right)\dot{B}\left(t\right).\hfill \end{array}$$

(25)

Let $\overrightarrow{A}$ denote the vectorization of matrix A which stacks its columns into a column vector with $\overrightarrow{A}={[A_{1:}^{\mathrm{T}},A_{2:}^{\mathrm{T}},\cdots ,A_{n:}^{\mathrm{T}}]}^{\mathrm{T}}$. Let ⊗ denote the Kronecker product. Then, the zeroing neurodynamic model (25) can be rewritten as

$$\begin{array}{cc}\hfill (B^{\mathrm{T}}(t)\otimes A(t))\dot{\overrightarrow{X}}(t)=& -\lambda \varphi ((B^{\mathrm{T}}(t)\otimes A(t))\overrightarrow{X}(t)-\overrightarrow{C}(t))\hfill \\ \hfill & -(B^{\mathrm{T}}(t)\otimes \dot{A}(t))\overrightarrow{X}(t)\hfill \\ \hfill & -({\dot{B}}^{\mathrm{T}}(t)\otimes A(t))X(t).\hfill \end{array}$$

(26)

which can be used for the implementation of model (25) in digital computers. Despite their generality, such formulations often suffer from scalability issues when applied to large-scale matrix systems, and Kronecker-based representations introduce high memory and computational costs.

Dai et al. [83] showed that zeroing neurodynamics can also be adopted to handle time-varying linear inequality as follows:

$$A\left(t\right)\mathbf{x}\le \mathbf{b}\left(t\right),$$

(27)

where operator “≤” works in an element-wise manner, by converting the time-varying linear inequality into the following equation:

$$max\{0,A\mathbf{x}(t)-\mathbf{b}(t\left)\right\}=0.$$

(28)

Zeroing neurodynamics has also been used to handle the following unconstrained time-varying optimization problem:

$$\underset{\mathbf{x}\in {\mathbb{R}}^n}{min}h(t,\mathbf{x})$$

(29)

where $h\left(\mathbf{x}\right)\in \mathbb{R}$ is a convex smooth function with respect to $\mathbf{x}\in {\mathbb{R}}^n$ and time variable $t\in \mathbb{R}$. By the convex optimization theory, the above minimization problem can be converted into the following problem:

$${\displaystyle \frac{\partial h(t,\mathbf{x})}{\partial \mathbf{x}}}=0,$$

(30)

which is a time-varying nonlinear equation, which can then be solved by using zeroing neurodynamics [84,85]. Such an idea is also extended to equality-constrained time-varying quadratic minimization and even quadratic programs with inequality constraints [86,87,88,89,90]. By combining zeroing neural dynamics with evolutionary computation techniques, nonconvex time-varying optimization problems can also be solved [91,92,93,94]. The efficiency of zeroing neurodynamics for computation problems makes it a promising alternative for handling motion control problems in robotics, especially the redundancy resolution problem of manipulators [95,96,97,98,99]. In robotics, the inverse kinematics problem requires finding the joint space configuration for a given end-effector configuration, and the mapping between the joint vector and the end-effector pose vector is a nonlinear function. Due to the redundancy feature, for which the number of joints of a serial manipulator is larger than the degrees of freedom needed to complete a given task, there are infinite solutions to the nonlinear equation for the inverse kinematics, which makes it possible to optimize the motion of the manipulators. By using the zeroing neurodynamics, the constraint of motions in the joint angle level can be converted into the constraints on the joint velocity level, which is an affine constraint.

3.3. Projection Neurodynamics

Being widely deployed to solve constrained optimization problems, projection neurodynamics is characterized by the usage of projection operators. Given a closed set $\mathsf{\Omega }\subset {\mathbb{R}}^n$, the projection of a vector $\mathbf{x}\in {\mathbb{R}}^n$ on $\mathsf{\Omega }$ is defined as

$$P_{\mathsf{\Omega }}(\mathbf{x})={\mathrm{argmin}}_{\mathbf{y}\in \mathsf{\Omega }}{\parallel \mathbf{x}-\mathbf{y}\parallel }_2.$$

(31)

In the following, we review the classical applications and principles of projection neurodynamics.

The minimum-velocity-norm redundancy resolution problem of robot manipulators can be formulated as the following optimization problem:

$$\begin{array}{c}\underset{\mathbf{u}\in {\mathbb{R}}^n}{min}{\displaystyle \frac{1}{2}}{\mathbf{u}}^{\mathrm{T}}\mathbf{u}\hfill \\ \hfill \mathrm{subject} \mathrm{to} {\dot{\mathbf{r}}}_{\mathrm{d}}=J\left(\theta \right)\mathbf{u}-\nu ({\mathbf{r}}_{\mathrm{d}}\left(t\right)-f\left(\theta \right)),\hfill \\ \hfill {\eta }^{-}\le \mathbf{u}\le {\eta }^{+},\hfill \end{array}$$

(32)

where $J\left(\theta \right)$ is the kinematic Jacobian, $\mathbf{u}=\dot{\theta }$ is the joint velocity vector of the manipulator while $\theta$ is the joint angle vector, ${\mathbf{r}}_{\mathrm{d}}$ is the reference trajectory for the end-effector, ${\eta }^{-}$ is the lower bound vector of the joint velocity while ${\eta }^{+}$ is the upper bound of the joint velocity vector, $f\left(\theta \right)$ is the mapping from the joint angle space to the workspace, and $\nu >0\in \mathbb{R}$ is a parameter for velocity feedback. Let $\mathsf{\Omega }=\left\{\mathbf{x}\right|{\eta }^{-}\le \mathbf{x}\le {\eta }^{+}\}$. A classical projection neurodynamic model for the redundancy resolution problem (32) is given as follows [100,101]:

$$\begin{array}{cc}\hfill \dot{\mathbf{u}}& =\epsilon (-\mathbf{u}+P_{\mathsf{\Omega }}\left(J^{\mathrm{T}}\left(\theta \right)\lambda \right)),\hfill \\ \hfill \dot{\lambda }& =\epsilon ({\dot{\mathbf{r}}}_{\mathrm{d}}-J\left(\theta \right)\mathbf{u}+\nu ({\mathbf{r}}_{\mathrm{d}}\left(t\right)-f\left(\theta \right))),\hfill \end{array}$$

(33)

with $\epsilon >0\in \mathbb{R}$ denoting the design parameter. The projection neurodynamic model can also deal with the case that constraints on the varying rate of the joint velocity are imposed. For example, in [100], an additional constraint is added into the redundancy resolution problem (32), i.e.,

$${\parallel \dot{\mathbf{u}}\parallel }_2\le C,$$

(34)

for which a modified projection neurodynamic model was developed for deal with the redundancy resolution in the velocity level while considering joint acceleration constraints:

$$\begin{array}{cc}\hfill \rho \dot{\mathbf{u}}& =\epsilon (-\mathbf{u}+P_{\mathsf{\Omega }}\left(J^{\mathrm{T}}\left(\theta \right)\lambda \right)),\hfill \\ \hfill \rho \dot{\lambda }& =\epsilon ({\dot{\mathbf{r}}}_{\mathrm{d}}-J\left(\theta \right)\mathbf{u}+\nu ({\mathbf{r}}_{\mathrm{d}}\left(t\right)-f\left(\theta \right)),\hfill \end{array}$$

(35)

where the adaptive parameter $\rho$ is defined as

$$\rho ={\displaystyle \frac{\epsilon (-\mathbf{u}+P_{\mathsf{\Omega }}\left(J^{\mathrm{T}}\left(\theta \right)\lambda \right))}{\parallel P_{\mathbb{C}}\left(\epsilon (-\mathbf{u}+P_{\mathsf{\Omega }}\left(J^{\mathrm{T}}\left(\theta \right)\lambda \right))\right){\parallel }_2}}$$

(36)

with $\mathbb{C}=\{\mathbf{x}\in {\mathbb{R}}^n|{\parallel \mathbf{x}\parallel }_2\le C\}$. By the definition of the projection rule, the calculation formula of $P_{\mathbb{C}}(\mathbf{x})$ is given as follows:

$$P_{\mathbb{C}}\left(\mathbf{x}\right)=\left\{\begin{array}{c}\mathbf{x},\mathrm{if}{\parallel \mathbf{x}\parallel }_2\le C\hfill \\ C{\displaystyle \frac{\mathbf{x}}{{\parallel \mathbf{x}\parallel }_2}},\mathrm{otherwise}.\hfill \end{array}\right.$$

(37)

By formulating the redundancy resolution or kinematic control problem of redundant manipulators as constrained optimizations, extensive effort has been put into the design of neurodynamics-based control laws [102]. However, while projection neurodynamics provides a systematic approach to incorporating constraints via projection operators, it often relies on the availability of an accurate and computationally tractable projection operator, which may become computationally intensive in high-dimensional or nonconvex constraint sets. Moreover, the convergence behavior of the dynamic system can be sensitive to the choice of the projection set geometry and the regularization parameters.

One of the advantages of projection neurodynamics for handling such a problem is the capability to deal with kinematic uncertainty, for which the Jacobian matrix $J\left(\theta \right)$ is unknown. A general kinematic control problem of redundant manipulators with unknown D-H parameters is formulated as follows [103]:

$$\begin{array}{c}\underset{\mathbf{u}\in {\mathbb{R}}^n}{min}{\displaystyle \frac{1}{2}}{\mathbf{u}}^{\mathrm{T}}A\mathbf{u}+{\mathbf{b}}^{\mathrm{T}}\mathbf{u}\hfill \\ \hfill \mathrm{subject} \mathrm{to} {\dot{\mathbf{r}}}_{\mathrm{d}}=J\left(\theta \right)\mathbf{u}-\nu ({\mathbf{r}}_{\mathrm{d}}\left(t\right)-f\left(\theta \right)),\hfill \\ \hfill J\left(\theta \right)=W\phi \left(\theta \right),\hfill \\ \hfill {\eta }^{-}\le \mathbf{u}\le {\eta }^{+},\hfill \end{array}$$

(38)

where $A\in {\mathbb{R}}^{n\times n}$ is a symmetric and positive definite matrix and $b\in {\mathbb{R}}^n$ is a vector, W is an unknown constant matrix while $\phi \left(\theta \right)$ is a known vector whose elements are functions of $\theta$. Zhang et al. [103] proposed the following adaptive projection neurodynamic model to solve the problem (38):

$$\begin{array}{cc}\hfill \dot{\check{\mathbf{u}}}& =\epsilon (-\check{\mathbf{u}}+P_{\mathsf{\Omega }}(-\check{\mathbf{u}}-A\check{\mathbf{u}}-\mathbf{b}+{\phi }^{\mathrm{T}}\left(\theta \right){\widehat{W}}^{\mathrm{T}}\lambda )),\hfill \\ \hfill \dot{\lambda }& =\epsilon ({\dot{\mathbf{r}}}_{\mathrm{d}}-\widehat{W}\phi \left(\theta \right)\check{\mathbf{u}}+\nu ({\mathbf{r}}_{\mathrm{d}}\left(t\right)-f\left(\theta \right))),\hfill \\ \hfill \dot{\widehat{W}}& =-\epsilon (\widehat{W}\phi \left(\theta \right)\check{\mathbf{u}}-\dot{\mathbf{r}}){\mathbf{u}}^{\mathrm{T}}{\phi }^{\mathrm{T}}\left(\theta \right),\hfill \\ \hfill \check{\mathbf{u}}& =\mathbf{u}+\rho ,\hfill \end{array}$$

(39)

where $\rho$ is bounded independent and identically distributed random noise of zero mean and $\mathbf{r}$ is the end-effector pose of the manipulator. In the projection neurodynamic model (39), $\widehat{W}$ is an estimate of W to construct the actual Jacobian matrix of the forward kinematics. It has been proved in [103] that model (39) can ensure parameter convergence of $\widehat{W}$ to the theoretical parameter W while finding the optimal solution to (38) with user given accuracy. Projection neurodynamics was also applied to kinematic teleportation of serial manipulators under deception attacks [104,105]. Despite its practical flexibility, the reliance on gradient-type dynamics and the linear feedback structure of projection neurodynamics models limits their ability to handle global nonlinearities or nonconvexities in the objective and constraints. This makes it challenging to extend such models to more complex multi-task coordination scenarios or discontinuous control tasks without substantial reformulation.

Projection neurodynamics has also been considered for solving visual servoing problems of manipulators like image-based visual servoing, where the motion control of manipulators is based on image sensing results [106,107]. In [106,107], the image-based visual servoing problem of serial manipulators is formulated as

$$\begin{array}{c}\underset{\mathbf{u}\in {\mathbb{R}}^n}{min}{\displaystyle \frac{1}{2}}{\mathbf{u}}^{\mathrm{T}}\mathbf{u}\hfill \\ \hfill \mathrm{subject} \mathrm{to} -\gamma (\mathbf{s}-{\mathbf{s}}_{\mathrm{d}})=J_{\mathrm{img}}(\mathbf{s},z)J_{\mathrm{r}}\left(\theta \right)\mathbf{u},\hfill \\ \hfill {\eta }^{-}\le \mathbf{u}\le {\eta }^{+},\hfill \end{array}$$

(40)

where $\mathbf{s}\in {\mathbb{R}}^2$ and ${\mathbf{s}}_{\mathrm{d}}\in {\mathbb{R}}^2$ denote the actual and desired feature coordinates in the image plane, respectively; $J_{\mathrm{img}}(\mathbf{s},z)$ is the image Jacobian matrix with z denoting the image depth and $J_{\mathrm{r}}\left(\theta \right)$ is the Jacobian matrix of forward kinematics of the manipulator. Traditionally, when dealing with visual seroving problems, there are two steps, i.e., finding the mapping from the feature point to the end-effector pose and obtaining the joint configuration from the desired end-effector pose. Problem (40) directly combines the steps into a single step by directly solving an optimization problem, and includes the joint constraints of manipulators, which is more desirable for practical visual servoing. A projection neurodynamic model for handling (40) is given as

$$\begin{array}{cc}\hfill \dot{\mathbf{u}}& \hfill =\epsilon (-\mathbf{u}+P_{\mathsf{\Omega }}\left(J_{\mathrm{r}}^{\mathrm{T}}\left(\theta \right)J_{\mathrm{img}}(\mathbf{s},z)\lambda \right)),\hfill \\ \hfill \dot{\lambda }& \hfill =\epsilon (-J_{\mathrm{img}}(\mathbf{s},z)J_{\mathrm{r}}\left(\theta \right)\mathbf{u}-\nu (\mathbf{s}-{\mathbf{s}}_{\mathrm{d}})).\hfill \end{array}$$

(41)

The above model requires that the depth information $\mathbf{z}$ is available. For the case where the depth information is not available, projection neurodynamic models can also be designed to solve the image-based visual servoing problem (40) [108]. Nevertheless, the effectiveness of projection neurodynamics in visual servoing tasks largely depends on accurate depth information and image Jacobians, which are difficult to acquire robustly under occlusions, varying lighting conditions, or dynamic environments. These dependencies raise robustness concerns and indicate a need for integrating more adaptive or learning-based perception models into the framework.

Projection neurodynamics is also useful for designing near-optimal control laws for nonlinear systems [109,110,111,112,113,114,115]. For example, consider the following underactuated nonlinear system:

$$\begin{array}{cc}\hfill \dot{\mathbf{x}}& \hfill =f\left(\mathbf{x}\right)+g\left(\mathbf{x}\right)\mathbf{u},\hfill \\ \hfill \mathbf{y}& \hfill =h\left(\mathbf{x}\right),\hfill \end{array}$$

(42)

with $\mathbf{x}\in {\mathbb{R}}^n$, $\mathbf{u}\in {\mathbb{R}}^r$, and $\mathbf{y}\in {\mathbb{R}}^n$ being its state, input, and output vectors, respetively. A finite-horizon optimal control problem of the above system is described as

$$\begin{array}{c}\underset{\mathbf{u}\in {\mathbb{R}}^r}{min}J(t)\hfill \\ \hfill \mathrm{subject} \mathrm{to} \dot{\mathbf{x}}=f\left(\mathbf{x}\right)+g\left(\mathbf{x}\right)\mathbf{u},\hfill \\ \hfill \mathbf{y}=h\left(\mathbf{x}\right),\hfill \\ \hfill {\mathbf{u}}^{-}\le \mathbf{u}\le {\mathbf{u}}^{+},\hfill \end{array}$$

(43)

where ${\mathbf{u}}^{-}$ and ${\mathbf{u}}^{+}$ are the lower and upper bounds of the input vector, respectively, and the performance index $J\left(t\right)$ is defined as

$$J\left(t\right)={\int }_0^T{(\mathbf{y}(t+\tau )-{\mathbf{y}}_{\mathrm{d}}(t+\tau ))}^{\mathrm{T}}Q(\mathbf{y}(t+\tau )-{\mathbf{y}}_{\mathrm{d}}(t+\tau )\mathrm{d}\tau ,$$

(44)

where $T>0\in \mathbb{R}$ represents the length of horizon, ${\mathbf{y}}_{\mathrm{d}}\left(t\right)$ represents the desired output at time instant t, and $Q\in {\mathbb{R}}^{r\times r}$ is a positive definite symmetric matrix. In [109,110,111,112,113,114,115], it has been shown that the above optimal control problem can be converted into a quadratic programming problem by making approximations of $\mathbf{y}(t+\tau )$ and ${\mathbf{y}}_{\mathrm{d}}(t+\tau )$ by using Taylor expansion [116]. Then, a projection neurodynamic model was adopted for solving the problem:

$$\lambda \dot{\mathbf{u}}=-\mathbf{u}+P_{\mathsf{\Omega }}(\mathbf{u}-2\mathsf{\Xi }\mathbf{u}-\mathbf{p}),$$

(45)

where $\mathsf{\Xi }=\kappa {(L_g{L}_f^{\rho -1}h(\mathbf{x}))}^{\mathrm{T}}Q{L}_g{L}_f^{\rho -1}h(\mathbf{x})$, $\mathsf{\Omega }=\left\{\mathbf{u}\right|{\mathbf{u}}^{-}\le \mathbf{u}\le {\mathbf{u}}^{+}\}$, $\mathbf{p}=-2{(L_g{L}_f^{\rho -1}h(\mathbf{x}))}^{\mathrm{T}}$$Q^{\mathrm{T}}E{\mathbf{v}}^T$, $E=Y_{\mathrm{d}}-Y$, and the definitions of Y, $Y_{\mathrm{d}}$, and $\kappa$ can be found in [109]. The above design can also be extended to cope with model uncertainty of the nonlinear systems as well as external disturbances such as periodic disturbances [109,112,113,114,115]. In terms of parameter uncertainty, auxiliary systems can be designed to reconstruct the dynamics of the nonlinear system, by which the design of the projection neural dynamic model can be conducted based on the auxiliary system. Yet, most projection-based neurodynamic schemes rely on continuous-time formulations, and their discrete-time implementations often lack rigorous convergence analysis. This gap between theory and implementation deserves more attention, particularly in real-time embedded robotic systems where sampling effects and model uncertainties can significantly affect performance. The properties of classical neurodynamics mentioned in this paper are compared in

Table 2. Comparison of different neurodynamics.

Literature	Problem	Model Type	Model Features
[44]	linear matrix equations	gradient neurodynamics	finite-time convergence
[51]	linear matrix equations	gradient neurodynamics	noise-tolerance, finite-time convergence
[53]	Moore–Penrose inverses	gradient neurodynamics	finite-time convergence
[54]	Lyapunov	gradient neurodynamics	noise-tolerance, finite-time convergence
[75,76]	Lyapunov	zeroing neurodynamics	finite-time convergence
[74]	Lyapunov	zeroing neurodynamics	noise-tolerance, finite-time convergence
[82]	Sylvester	zeroing neurodynamics	finite-time convergence, noise-tolerance
[81]	Sylvester	zeroing neurodynamics	time-varying parameter
[84,85]	nonlinear optimization	zeroing neurodynamics	noise-tolerance, limited-time convergence
[86,87]	quadratic programming	zeroing neurodynamics	noise-tolerance, finite-time convergence
[91,92]	nonconvex optimization	zeroing neurodynamics	noise-tolerance, finite-time convergence
[80]	Stein	zeroing neurodynamics	predefined-time convergence
[79]	matrix square root	zeroing neurodynamics	predefined-time convergence, noise-tolerance
[77,78]	linear matrix equations	zeroing neurodynamics	finite-time convergence
[100,101]	redundancy resolution	projection neurodynamics	global convergence
[103]	kinematic control	projection neurodynamics	adaptive parameter convergence
[108]	image-based visual servoing	projection neurodynamics	global asymptotic convergence
[110,111,112]	near-optimal control	projection neurodynamics	exponential stability

.

4. Neurodynamics for Multi-Agent Systems

In the past two decades, there have been a lot of papers on multi-agent systems, covering distributed computation and control. It is interesting that the past decade has witnessed the development of neurodynamics-based methods for multi-agent systems.

4.1. Neurodynamics for Distributed Competition

One of the attractive areas of neurodynamic applications is the k-winners-take-all (k-WTA), which selects the k largest inputs among n inputs ($n>k>0$) where each agent owns one input. The k-winners-take-all is the extension of winner-take-all which is the special case of k-WTA with $k=1$. Both k-WTA and WTA can be used to model competition behaviors in social and biological networks [117].

Suppose that there is a network of n agents whose communication graph is described by a graph Laplacian matrix L. For each agent i, it has a decision input $x_i$ and a decision output $z_i$. Let $x_k^{\ast }$ denotes the k-th largest input among all the n inputs of the agents. For k-WTA, the relationship between each input and output is defined as follows [118]:

$$z_i=\left\{\begin{array}{c}1,\mathrm{if}{x}_i\ge x_k^{\ast },\hfill \\ 0,\mathrm{otherwise}.\hfill \end{array}\right.$$

(46)

Agents with output $z_i=1$ are called the winners, while those with $z_i=0$ are called the losers. Although the binary output of the k-WTA model is appealing for its simplicity, such a hard selection mechanism may not be robust to small fluctuations or noise in the input signals, potentially leading to instability in dynamic or uncertain environments. The k-WTA can be formulated as the following quadratic program:

$$\begin{array}{c}\underset{\mathbf{z}\in {\mathbb{R}}^n}{min}{\displaystyle \frac{\alpha }{2}}{\mathbf{z}}^{\mathrm{T}}\mathbf{z}-{\mathbf{x}}^{\mathrm{T}}\mathbf{z}\hfill \\ \hfill \mathrm{subject} \mathrm{to} {\mathbf{1}}^{\mathrm{T}}\mathbf{z}=k,\hfill \\ \hfill \mathbf{0}\le \mathbf{z}\le \mathbf{1},\hfill \end{array}$$

(47)

where $\alpha >0\in \mathbb{R}$ is a parameter, $\mathbf{0}\in {\mathbb{R}}^n$ is a zero vector with all elements being 0, and $\mathbf{1}\in {\mathbb{R}}^n$ is a 1 vector with all elements being 1. It was proved in [119] that the solution to the quadratic program (47) satisfies the k-WTA operation if $\alpha$ satisfies

$$0<\alpha \le |x_k^{\ast }-x_{k+1}^{\ast }|,$$

(48)

where $x_{k+1}^{\ast }$ denotes the $(k+1)$-th largest elements among the inputs $x_i$.

Based on the above quadratic program, a classical projection neurodynamic model for k-WTA operation is given as follows [120]:

$$\begin{array}{c}\hfill \dot{\chi }=\gamma (-{\mathbf{1}}^{\mathrm{T}}\mathbf{z}+k),\hfill \\ \hfill \mathbf{z}=P_{\mathsf{\Omega }}\left(\mathbf{1}\chi +{\displaystyle \frac{\mathbf{x}}{\alpha }}\right),\hfill \end{array}$$

(49)

where $\mathsf{\Omega }=\{\mathbf{0}\le \mathbf{z}\le \mathbf{1}\}$ and $\gamma >0\in \mathbb{R}$, $\mathbf{x}={[x_1,x_2,\cdots ,x_n]}^{\mathrm{T}}$, and $\mathbf{z}={[z_1,z_2,\cdots ,z_n]}^{\mathrm{T}}$. The neurodynamics-based k-WTA model (49) has global asymptotic convergence. With (49), for each agent i, its dynamics for computing $z_i$ is given as

$$\begin{array}{cc}\hfill \dot{\chi }& =\gamma (-\sum _{i=1}^n{z}_i+k),\hfill \\ \hfill z_i& =g\left(\chi +{\displaystyle \frac{x_i}{\alpha }}\right),\hfill \end{array}$$

(50)

where $g\left(x\right)=P_{\overline{\mathsf{\Omega }}}\left(x\right)$ with $\overline{\mathsf{\Omega }}=\{x\in {\mathbb{R}}^n|0\le x\le 1\}$. Evidently, for each agent i, to update the decision output $z_i$, it needs to know the updated value of $\chi$ whose computation relies on the decision output of all the agents in the network. Thus, the neurodynamics-based k-WTA model (49) is centralized. This raises concerns about scalability, fault tolerance, and single-point-of-failure vulnerabilities, which are often overlooked in theoretical designs but are crucial for deployment in autonomous and resilient multi-agent systems. A centralized model has several disadvantages. Firstly, the model is not suitable for large-scale networks, which would pose a great requirement on the communication and computation capacity of the agent dealing with the computation of $\chi$. Secondly, centralized models are vulnerable to failure of central nodes. Thus, distributed k-WTA models are worth investigating.

The first distributed k-WTA model is based on projection neurodynamics, whose design is based on the centralized k-WTA model (49) and consensus filters, and the obtained projection neurodynamic model is described as follows [121]:

$$\begin{array}{cc}\hfill \epsilon \dot{\mathbf{u}}& =C_0\left(-\mathbf{y}-\mathbf{z}+{\displaystyle \frac{k\mathbf{1}}{n}}\right)-C_{0}L\mathbf{u},\hfill \\ \hfill \epsilon \dot{\mathbf{y}}& =-L(\mathbf{y}+\mathbf{z}),\hfill \\ \hfill \mathbf{z}& =P_{\mathsf{\Omega }}\left(\mathbf{x}+{\displaystyle \frac{\mathbf{x}}{\alpha }}\right),\hfill \end{array}$$

(51)

where $\mathbf{u}={[u_1,u_2,\cdots ,u_n]}^{\mathrm{T}}$ and $\mathbf{y}={[y_1,y_2,\cdots ,y_n]}^{\mathrm{T}}$ are auxiliary vectors; $C_0>0\in \mathbb{R}$ and $\epsilon >0\in \mathbb{R}$ are design parameters. Based on (51), for each agent in the network, there are two auxiliary variables, i.e., $u_i$ and $y_i$, and the dynamics of each agent’s states is

$$\begin{array}{cc}\hfill \epsilon {\dot{u}}_i& =C_0\left(-y_i-z_i+{\displaystyle \frac{k}{n}}\right)-C_0\sum _{j\in \mathbb{N}\left(i\right)}{w}_{ij}(u_i-u_j),\hfill \\ \hfill \epsilon {\dot{y}}_i& =-\sum _{j\in \mathbb{N}\left(i\right)}{w}_{ij}(y_i+z_i-y_j-z_j),\hfill \\ \hfill z_i& =P_{\overline{\mathsf{\Omega }}}\left(u_i+{\displaystyle \frac{x_i}{\alpha }}\right).\hfill \end{array}$$

(52)

Evidently, in view of (52), the k-WTA model (51) is distributed, since updating each agent’s state variables only needs its own state information and information of neighboring agents. As proved in [121], if $\mathbf{y}\left(0\right)=\mathbf{0}$, ${\lambda }_{2\min }\left(L\right)\ge 7.5C_0$, and the communication graph of the network is undirected connected, where ${\lambda }_{2\min }$ denotes the algebraic connectivity of the Laplacian matrix, then $\mathbf{z}$ of the distributed k-WTA model (51) based on projection neurodynamics globally asymptotically converges to the solution of the k-WTA problem. The requirement for the initialization of $\mathbf{y}$ and the setting of parameter $C_0$ could limit the deployment of the model in engineering applications. While the distributed implementation alleviates some scalability issues, the model’s reliance on multiple auxiliary variables and tightly tuned parameters (such as $\mathbf{y}\left(0\right)=\mathbf{0}$ and ${\lambda }_{2\min }\left(L\right)\ge 7.5C_0$) could hinder practical deployment in dynamically changing networks where such initialization and global spectral properties are difficult to guarantee.

To address the limitations of the neurodynamic model (51) for the k-WTA operation, by observing that $\mathbf{y}\left(0\right)=\mathbf{0}$ and its value on the steady-state is also $\mathbf{0}$, for undirected connected communication topologies modeled by Laplacian matrix L, the following neurodynamic model was proposed [122]:

$$\begin{array}{cc}\hfill \epsilon \dot{\mathbf{u}}& =-\mathbf{z}+{\displaystyle \frac{k\mathbf{1}}{n}}-L\mathbf{u},\hfill \\ \hfill \mathbf{z}& =P_{\mathsf{\Omega }}\left(\mathbf{x}+{\displaystyle \frac{\mathbf{x}}{\alpha }}\right),\hfill \end{array}$$

(53)

for which the dynamics of each agent is given as

$$\begin{array}{cc}\hfill \epsilon {\dot{u}}_i& =-z_i+{\displaystyle \frac{k}{n}}-\sum _{j\in \mathbb{N}\left(i\right)}{w}_{ij}(u_i-u_j),\hfill \\ \hfill z_i& =P_{\overline{\mathsf{\Omega }}}\left(u_i+{\displaystyle \frac{x_i}{\alpha }}\right).\hfill \end{array}$$

(54)

Evidently, compared to neurodynamic model (51), neurodynamic model (53) has a simple structure, making its computational efficiency more favorable. It has been proved in [122] that, if $u_k^{\ast }-u_{k+1}^{\ast }>2\alpha (\sqrt{n}{\lambda }_{2\min }^{-1}\left(L\right)+1)$ with $\alpha >0$, then output $\mathbf{z}$ of neurodynamic model (53) globally asymptotically converges to the solution of the k-WTA problem. An interesting property of k-WTA model (53) is its inherent privacy protection capability. If the competition input $x_i$ is private data that agents do not want to share with neighboring agents, then neurodynamic model (51) may not be desirable. Given that $z_i\in [0,1]$, by the definition of the projection operator, we have $x_i=\alpha (z_i-u_i)$, which means that obtaining the value of $z_i$ indicates obtaining the value of $x_i$. For neurodynamic model (51), agents need to share $z_i$ with neighboring agents, while such a requirement is not needed for neurodynamic model (53).

The above two distributed k-WTA models are essentially based on a centralized k-WTA model by performing distributed estimation of centralized terms. Such a design intrinsically yields a two-step convergence process for finding the k-WTA solution; that is, convergence of estimators for centralized terms followed by convergence of the equivalent centralized k-WTA model, making the overall convergence speed slow. In [123], it is shown that the k-WTA problem can be formulated into the following form:

$$\begin{array}{c}\underset{\mathbf{z}\in {\mathbb{R}}^n}{min}{\displaystyle \frac{\alpha }{2}}{\mathbf{z}}^{\mathrm{T}}\mathbf{z}-{\mathbf{x}}^{\mathrm{T}}\mathbf{z}\hfill \\ \hfill \mathrm{subject} \mathrm{to} \mathbf{z}={\displaystyle \frac{k\mathbf{1}}{n}}+L\mathbf{y},\hfill \\ \hfill \mathbf{0}\le \mathbf{z}\le \mathbf{1},\hfill \end{array}$$

(55)

where the Laplacian matrix L of the undirected connected communication graph is embedded, and $\mathbf{y}$ is an unknown vector to be found. The problem description (55) allows direct design of distributed k-WTA models based on projection neurodynamics, and the following neurodynamic model was designed [123]:

$$\begin{array}{cc}\hfill \epsilon \dot{\mathbf{z}}& =-\mathbf{z}+P_{\mathsf{\Omega }}({\displaystyle \frac{\mathbf{x}-\lambda }{\alpha }}),\hfill \\ \hfill \epsilon \dot{\mathbf{y}}& ={\displaystyle \frac{L\lambda }{\alpha }},\hfill \\ \hfill \epsilon \dot{\lambda }& ={\displaystyle \frac{\mathbf{x}}{\alpha }}-{\displaystyle \frac{k\mathbf{1}}{n\alpha }}-{\displaystyle \frac{L\mathbf{y}}{\alpha }}-{\displaystyle \frac{L\lambda }{\alpha }},\hfill \end{array}$$

(56)

by which the state evolution of each agent is described by

$$\begin{array}{cc}\hfill \epsilon {\dot{z}}_i& =-z_i+P_{\overline{\mathsf{\Omega }}}({\displaystyle \frac{x_i-{\lambda }_i}{\alpha }}),\hfill \\ \hfill \epsilon {\dot{y}}_i& ={\displaystyle \frac{{\sum }_{j\in \mathbb{N}\left(i\right)}{w}_{ij}({\lambda }_i-{\lambda }_j)}{\alpha }},\hfill \\ \hfill \epsilon {\lambda }_i& ={\displaystyle \frac{x_i}{\alpha }}-{\displaystyle \frac{k}{n\alpha }}-{\displaystyle \frac{{\sum }_{j\in \mathbb{N}\left(i\right)}{w}_{ij}(y_i-y_j)}{\alpha }}-{\displaystyle \frac{{\sum }_{j\in \mathbb{N}\left(i\right)}{w}_{ij}({\lambda }_i-{\lambda }_j)}{\alpha }},\hfill \end{array}$$

(57)

indicating that k-WTA model (56) based on projection neurodynamics is also distributed. One of the advantages of the design for k-WTA model (56) based on the novel formulation of the k-WTA problem with embedded graph Laplacian matrices is the ability to cope with various local constraints on decision outputs $z_i$. In practice, agents generally have computation, actuation, communication, and sensing abilities. However, in some exceptional cases, such as system faults, agents may lose actuation ability, making them not suitable for performing tasks, but they are still able to propagate information. One of the potential applications of distributed k-WTA models is distributed task allocations, and k-WTA model (56) can be easily extended to deal with local constraints, which is illustrated in [123] in detail.

In [41], the performance of a distributed k-WTA model on various types of communication graphs is tested using extensive computer simulations. It is shown that the distributed k-WTA model works well on time-varying undirected connected graphs and joint connected undirected communication graphs. In [41], an application of the distributed k-WTA model based on projection neurodynamics is illustrated to assign tasks to mobile robots. The dynamics of the mobile robots are described as follows:

$${\dot{\mathbf{p}}}_i={\mathbf{v}}_i$$

(58)

with $i=1,2,\cdots ,n$; ${\mathbf{p}}_i={[p_{ix},p_{iy}]}^{\mathrm{T}}$ and ${\mathbf{v}}_i={[p_{vx},p_{vy}]}^{\mathrm{T}}$ represent the position and velocity input of each robot. There is a moving target whose real-time position is denoted by ${\mathbf{p}}_t={[p_{tx},p_{ty}]}^{\mathrm{T}}$. The communication graph of the mobile robots is undirected and connected with the Laplacian matrix being L. The objective is to capture the moving target by the mobile robots using a competition mechanism, for which at each time, only k mobile robots track the moving target, and the assignment of mobile robots to execute the tracking task is based on the distances from the mobile robots to the moving target. To this end, the competition input of every agent i is defined as

$$u_i=-\parallel {\mathbf{p}}_i-{\mathbf{p}}_t{\parallel }_2,$$

(59)

and the velocity input of each mobile robot is given as

$${\mathbf{v}}_i=-z_{i}c\varphi ({\mathbf{p}}_i-{\mathbf{p}}_t)+z_i{\dot{\mathbf{p}}}_t,$$

(60)

with $c>0\in \mathbb{R}$ being a design parameter, $z_i\in [0,1]$ the task assignment signal generated by a k-WTA model, and $\varphi$ an activation function defined as

$$\begin{array}{c}\hfill \varphi \left(y\right)=\left\{\begin{array}{c}{\left|y\right|}^r+{\left|y\right|}^{1/r},\mathrm{if}y>0,\hfill \\ 0,\mathrm{if}y=0,\hfill \\ -{\left|y\right|}^r-{\left|y\right|}^{1/r},\mathrm{if}y<0,\hfill \end{array}\right.\end{array}$$

(61)

where $r\in (0,1)$ is a function parameter.

Overall, the neurodynamics-based k-WTA models represent a promising direction for distributed decision-making. However, current models are primarily validated in deterministic and idealized settings, with limited investigation into their robustness under model uncertainties, communication delays, or asynchronous update factors that are intrinsic to most real-world multi-agent systems. Bridging this theory–application gap is a critical avenue for future research.

4.2. Neurodynamics for Distributed Coordination

The coordination of multi-agent systems is a powerful tool to enlarge the ability of agents to complete complicated tasks that may not be finished by a single agent, which is useful in various fields like robots and embedded systems [124,125,126,127,128,129,130,131]. By coordination, it means agents cooperatively work to complete a given task. In the scenario of distributed coordination, agents are located in different positions and they complete tasks cooperatively via leveraging their sensing, communication, actuation, and communication abilities [132]. However, current coordination models often assume ideal communication links and perfectly synchronized updates, which may not hold in practical scenarios involving time delays, packet loss, or asynchronous events. These assumptions limit the robustness and real-time applicability of some proposed algorithms.

Consider a multi-agent system with n agents whose communication topology is represented by an undirected connected graph and the Laplacian matrix is denoted by L. The agent dynamics is given as

$$\begin{array}{cc}\hfill {\dot{x}}_i& =v_i,\hfill \\ \hfill {\dot{v}}_i& =u_i,\hfill \end{array}$$

(62)

where $x_i$, $v_i$, and $u_i$ denote the position, velocity, and control input of the i-th agent. The objective is to design distributed control inputs $u_i$ to reach state consensus with $x_1=x_2=\cdots =x_n$. Agents are with limited actuation capabilities, modeled by $u_i^{-}\le u_i\le u_i^{+}$, where $u_i^{-}$ and $u_i^{+}$ are the lower and upper bound of the input of the i-th agent. The agent dynamics is written in a compact form as follows:

$$\begin{array}{cc}\hfill \dot{\mathbf{x}}& =\mathbf{v},\hfill \\ \hfill \dot{\mathbf{v}}& =\mathbf{u},\hfill \end{array}$$

(63)

where $\mathbf{x}={[x_1,x_2,\cdots ,x_n]}^{\mathrm{T}}\in {\mathbb{R}}^n$, $\mathbf{v}={[v_1,v_2,\cdots ,v_n]}^{\mathrm{T}}\in {\mathbb{R}}^n$, and $\mathbf{u}={[u_1,u_2,\cdots ,u_n]}^{\mathrm{T}}\in {\mathbb{R}}^n$. The input constraints of the agents are also written in a compact form as $\mathbf{u}\in \mathsf{\Omega }$, where $\mathsf{\Omega }=\{\mathbf{u}\in {\mathbb{R}}^n|{\mathbf{u}}^{-}\le \mathbf{u}\le {\mathbf{u}}^{+}\}$, with ${\mathbf{u}}^{-}={[u_1^{-},u_2^{-},\cdots ,u_n^{-}]}^{\mathrm{T}}\in {\mathbb{R}}^n$ and ${\mathbf{u}}^{+}={[u_1^{+},u_2^{+},\cdots ,u_n^{+}]}^{\mathrm{T}}\in {\mathbb{R}}^n$ denoting the lower bound and upper bound of the control inputs. In [42], a projection neurodynamics-based method is proposed to address the consensus problem using an optimal control framework, where three performance indices are considered and the corresponding distributed consensus protocols are designed and tested, which are shown in

Table 3. Comparison of performance index and consensus protocols in [42] using projection neurodynamics.

Performance Index	Consensus Protocol
$J\left(t\right)={\int }_0^T{\mathbf{x}}^{\mathrm{T}}(t+\tau )L\mathbf{x}(t+\tau )\mathrm{d}\tau$	$\epsilon \dot{\mathbf{u}}=-\mathbf{u}+P_{\mathsf{\Omega }}\left(\mathbf{u}-\frac{T^5}{10}L\mathbf{u}-\frac{T^3}{3}L\mathbf{x}-\frac{T^4}{4}L\mathbf{v}\right)$
$J\left(t\right)={\int }_0^T{\mathbf{x}}^{\mathrm{T}}(t+\tau )L\mathbf{x}(t+\tau )\mathrm{d}\tau +\frac{1}{2}{\mathbf{u}}^{\mathrm{T}}\left(t\right)\mathbf{u}\left(t\right)$	$\epsilon \dot{\mathbf{u}}=-\mathbf{u}+P_{\mathsf{\Omega }}\left(-\frac{T^5}{10}L\mathbf{u}-\frac{T^3}{3}L\mathbf{x}-\frac{T^4}{4}L\mathbf{v}\right)$
$J\left(t\right)={\int }_0^T{\mathbf{x}}^{\mathrm{T}}(t+\tau )L\mathbf{x}(t+\tau )+{\mathbf{v}}^{\mathrm{T}}(t+\tau )\mathbf{v}(t+\tau )\mathrm{d}\tau +\frac{1}{2}{\mathbf{u}}^{\mathrm{T}}\left(t\right)\mathbf{u}\left(t\right)$	$\epsilon \dot{\mathbf{u}}=-\mathbf{u}+P_{\mathsf{\Omega }}\left(-\frac{T^5}{10}L\mathbf{u}-\frac{T^3}{3}L\mathbf{x}-\frac{T^4}{4}L\mathbf{v}-\frac{2T^3}{3}\mathbf{u}-T^2\mathbf{v}\right)$

. Using different performance indices, different purposes can be achieved. For example, the performance index $J\left(t\right)={\int }_0^T{\mathbf{x}}^{\mathrm{T}}(t+\tau )L\mathbf{x}(t+\tau )\mathrm{d}\tau +\frac{1}{2}{\mathbf{u}}^{\mathrm{T}}\left(t\right)\mathbf{u}\left(t\right)$ ensures that the velocity of the agents is zero when consensus is reached, which means that the position of agents will eventually be static. The consensus approach proposed in [42] can be readily extended to multi-dimensional multi-agent systems, where the position state of each agent is a vector instead of a scalar, by following the method in [133]. It was latter shown in [134] that the method proposed in [42] can also be extended to the case where privacy protection of agents’ initial position is necessary, by leveraging homomorphic encryption techniques like the Paillier cryptosystem. These methods typically rely on convexity assumptions and predefined communication topologies. In realistic environments with dynamic graphs or nonconvex performance surfaces, the stability and convergence properties of the projection-based neurodynamics may not be preserved without significant modifications.

A multiobjective optimization problem is described as

$$\begin{array}{c}\hfill \min \mathbf{f}(\mathbf{x})={[f_1\left(\mathbf{x}\right),f_2\left(\mathbf{x}\right),\cdots ,f_k\left(\mathbf{x}\right)]}^{\mathrm{T}}\hfill \\ \hfill \mathrm{s}.\mathrm{t}.\mathbf{x}\in \Gamma \hfill \end{array}$$

(64)

where $\mathbf{x}\in {\mathbb{R}}^n$ is the decision variable, and $\mathsf{\Omega }=\{\mathbf{x}\in {\mathbb{R}}^n|A\mathbf{x}\le \mathbf{b},g\left(\mathbf{x}\right)\le 0,\mathbf{x}\in \mathsf{\Omega }\}$ is called feasible region; $\mathbf{f}\left(\mathbf{x}\right)\in {\mathbb{R}}^k$ is a vector of objective functions with $f_i\left(\mathbf{x}\right)\in \mathbb{R}$ being the k-th objective function; $\mathsf{\Omega }$ is a closed convex set. The problem is assumed to be convex, for which all the objective functions are convex and the feasible region is also convex. In [135], the multiobjective optimization problem (64) is converted to

$$\begin{array}{c}\hfill \min \sum _{i=1}^k{w}_i{f}_i\left({\mathbf{x}}_i\right)\hfill \\ \hfill \mathrm{s}.\mathrm{t}.\mathbf{L}\tilde{\mathbf{x}}=0,\hfill \\ \hfill {\mathbf{x}}_i\in \Gamma \hfill \end{array}$$

(65)

with $w_i>0\in \mathbb{R}$, where $\tilde{\mathbf{x}}=\mathrm{col}\{{\mathbf{x}}_1,{\mathbf{x}}_2,\cdots ,{\mathbf{x}}_k\}\in {\mathbb{R}}^{nk}$, $\mathbf{L}=L\otimes I_n$ with $I_n$ representing an n-by-n identity matrix, and L is the Laplacian matrix of the undirected connected communication graph. Problem (65) is called a consensus optimization problem, where each agent has a state vector ${\mathbf{x}}_i$ and a local function $f_i\left({\mathbf{x}}_i\right)$, and the agents collectively work to reach a common value on ${\mathbf{x}}^{\ast }$ that minimizes the cost function while satisfying all the constraints. A collective projection neurodynamic model was proposed in [135] to solve the above problem and the dynamics of the i-th agent is given by

$$\begin{array}{cc}\hfill {\dot{\mathbf{x}}}_i& =-{\mathbf{x}}_i+P_{\mathsf{\Omega }}({\mathbf{x}}_i-w_i{\displaystyle \frac{\partial f_i({\mathbf{x}}_i)}{{\mathbf{x}}_i}}-{\displaystyle \frac{\partial g({\mathbf{x}}_i)}{\partial {\mathbf{x}}_i}}{\mathbf{y}}_i-{\mathbf{A}}^{\mathrm{T}}{\mathbf{z}}_i)\hfill \\ \hfill & +\sum _{j\in \mathbb{N}(i)}{w}_{ij}({\mathbf{x}}_j+{\eta }_j-{\mathbf{x}}_i-{\eta }_i)\hfill \\ \hfill {\dot{\mathbf{y}}}_i& =-{\mathbf{y}}_i+{({\mathbf{y}}_i+g({\mathbf{x}}_i))}^{+},\hfill \\ \hfill {\dot{\mathbf{z}}}_i& =A{\mathbf{x}}_i-\mathbf{b},\hfill \\ \hfill {\dot{\eta }}_i& =\sum _{j\in \mathbb{N}(i)}{w}_{ij}({\mathbf{x}}_i-{\mathbf{x}}_j),\hfill \end{array}$$

(66)

where ${\left(\mathbf{x}\right)}^{+}=max\{\mathbf{x},\mathbf{0}\}$. It was proved in [135] that neurodynamic model (66) is globally asymptotically stable and can find an accurate solution to the above consensus optimization problem. Based on consensus optimization, projection neurodynamics is also applied to other distributed optimization problems [136,137,138]. It is worth mentioning that the scalability of such neurodynamic models to large-scale systems remains an open challenge. The computational and communication complexity grows with network size, and the real-time feasibility under strict latency and resource constraints is yet to be thoroughly validated. Projection neurodynamics was also applied to distributed maneuvering of autonomous surface vehicles in [139] in the present of unknown kinetics and state constraint. Nonetheless, most existing works still rely on relatively idealized assumptions about the model knowledge and network connectivity. Future studies should rigorously investigate the robustness of neurodynamic methods under practical uncertainties, model mismatches, and adversarial communication environments.

4.3. Scalability and Practical Constraints

The issue of scalability is a key consideration in the practical deployment of neurodynamics-based models for multi-agent systems. In this regard, centralized k-WTA model (49) poses evident limitations. As the number of agents increases, the requirement for a central unit to compute the global variable $\chi$ becomes increasingly prohibitive in terms of communication and computation, thereby making the model unsuitable for large-scale systems. Moreover, such centralized designs are vulnerable to single points of failure and cannot tolerate the dynamic nature of real-world networks where agents may join or leave.

To mitigate these limitations, distributed models such as (51) and (53) have been developed. These models only require local communication among neighboring agents, which significantly improves scalability and robustness. Nonetheless, practical deployment still faces several constraints. First, the model in (51) requires careful parameter tuning, particularly the choice of $C_0$ and the initialization $\mathbf{y}\left(0\right)=\mathbf{0}$. These conditions, while theoretically reasonable, may not always be guaranteed in real-world implementations, especially in asynchronous or delayed communication environments. The improved model (53) addresses some of these issues by eliminating auxiliary variables and reducing parameter sensitivity, leading to a simpler and more scalable architecture. However, this model still relies on continuous-time dynamics and assumes ideal communication among neighbors, which may not hold in practical digital systems with quantization, communication delay, or packet loss.

Furthermore, another practical constraint lies in the need for agents to exchange intermediate variables (e.g., $u_i$ or $z_i$). While the original input $x_i$ may be private and not directly shared, these derived variables could still leak sensitive information through reverse engineering. Thus, privacy-preserving mechanisms or noise-injection strategies may be needed for privacy-critical applications. Finally, in large-scale settings, issues such as computation burden per agent, convergence speed, and robustness to network changes (e.g., time-varying topologies or link failures) must also be addressed.

5. Challenges and Opportunities

In this section, based on the above review on the development of neurodynamics for computation and multi-agent systems in the past decade, the challenges and opportunities in the field of neurodynamics are discussed.

5.1. Gradient Neurodynamics for Time-Varying Computation Problems

Recently, it has been reported that gradient neurodynamics based on norm correction can be adopted to solve some time-varying problems with robust finite-time convergence, covering time-varying linear equations, time-varying matrix inversion, time-varying Moore–Penrose inversion, and time-varying Lyapunov equations [51,52,53,54]. There are lots of time-varying problems that have been extensively investigated by using the zeroing neurodynamic method with accurate time-derivative feedback, which is not required in the gradient neurodynamic method, making gradient neurodynamics more favorable for engineering applications where accurately measuring derivatives could be expensive or infeasible. The performance of such gradient neurodynamics to other time-varying computation problems is worth investigating. Meanwhile, the discretization techniques for norm-based gradient neurodynamics is still not explored. Due to the existence of the norm operation, the development of efficient discrete-time models for norm-based gradient neurodynamic model is a challenging problem.

5.2. Zeroing Neurodynamics for Distributed Control and Optimization

While zeroing neurodynamics are widely recognized as a powerful tool for handling time-varying problems, the existing zeroing neurodynamic models are centralized, and are not applicable to the scenario of distributed control and optimization. Considering the matured techniques for ensuring finite-time convergence and robustness to external disturbances, the extension of zeroing neurodynamics to distributed control and optimization is significant to enlarge the impact of zeroing neurodynamics. However, such an extension is challenging due to the inherent feature of implicit dynamics of conventional design of zeroing neurodynamic models. Thanks to the recent work of [140], it becomes feasible to design explicit zeroing neurodynamic model to solve time-varying nonlinear equations by choosing a norm-type error function. However, the method in [140] still cannot be directly extended to distributed computation scenarios due to the existence of global error norm. New design techniques are still needed for the zeroing neurodynamics to address distributed computation and control problems.

5.3. Projection Neurodynamics for Distributed Coordination

Currently, most existing work on distributed coordination of multi-agent systems is based on the assumption that the communication graphs are undirected connected. Considering that the actual communication may not be bidirectional and communication networks of physical agents may change with time owing to physical factors or cyber attacks, it would be useful to investigate the application of projection neurodynamics to the distributed coordination of multi-agent systems over a directed communication topology and even joint connected communication topology [141,142]. In addition, the effects of cyber attacks and the requirement of privacy protection can be considered. In terms of privacy protection, it remains a challenging problem to develop projection neurodynamics-based computationally efficient privacy-preserving distributed coordination methods with formal privacy analysis like differential privacy.

5.4. Applications of Neurodynamics

The applications of neurodynamics can be further exploited. For example, in anomaly detection [143,144,145], existing methods ignore the inherent time-varying features of anomaly detection, where anomaly samples can change over time. It would be interesting to incorporate neurodynamics with existing forward neural networks to improve the efficiency and accuracy of detection, and such an idea also applies to the applications of neurodynamics to internet of things [146,147], social network analysis [148], bioinformatics [149], and games [150,151].

5.5. Practical Deployment Considerations

Despite the promising theoretical performance of neurodynamics in solving optimization and control problems, their practical deployment in real-world systems remains an active area of exploration. Real-time constraints, hardware limitations, and noise sensitivity pose significant challenges for directly applying these models in embedded or edge devices. For instance, while fixed-time and finite-time neurodynamics offer desirable convergence properties, they may require higher computational precision or exhibit numerical stiffness in discretized implementations. Moreover, integrating neurodynamic solvers into larger control architectures (e.g., in autonomous vehicles or industrial robotics) necessitates careful co-design with perception and communication modules. As such, research efforts are increasingly focusing on lightweight, robust, and interpretable neural dynamics that can be deployed under resource-constrained or safety-critical conditions. This highlights the gap between theoretical advances and system-level implementation, and points to important directions for future work.

Some recent works have begun bridging this gap. For example, in [152], neurodynamics-inspired controllers have been tested in real-time path tracking of unmanned aerial vehicles and energy-efficient multi-robot coordination. These deployments often require careful tuning of system parameters and model structure to balance accuracy, convergence speed, and computational load. While still limited in scope, such case studies provide initial evidence of the applicability of neurodynamics in embedded systems and industrial settings.

5.6. Future Work

Despite the remarkable advancements in the application of neurodynamics to computation and multi-agent systems, several critical directions remain insufficiently explored and offer rich potential for future investigation.

(1)

Resilience and robustness under complex agent interactions: Real-world multi-agent systems are often subject to communication delays, asynchronous updates, adversarial behaviors, and heterogeneous dynamics. There is a pressing need to develop robust neurodynamic models that can maintain stability and coordination performance under such adversarial or uncertain settings. Investigating the interplay between robustness, convergence rate, and computational efficiency in neurodynamics remains an open research challenge.

(2)

High-dimensional and distributed optimization: As multi-agent systems increasingly engage in distributed machine learning, federated optimization, and cooperative decision-making, extending neurodynamic models to high-dimensional, nonconvex, and communication-efficient optimization tasks becomes imperative. This calls for scalable neurodynamic architectures that leverage sparsity, structure-exploitation, and decentralization principles.

(3)

Cross-disciplinary integration with deep learning and neuroscience: Inspired by biological systems, the integration of neurodynamics with deep neural architectures or spiking neural networks can bring new perspectives to adaptive control and collective intelligence. On the one hand, embedding neurodynamics into neural learning modules could improve interpretability and dynamical adaptability. On the other hand, introducing biologically plausible mechanisms such as plasticity, inhibition, and homeostasis into neurodynamic control may enrich the expressiveness and learning capacity of multi-agent coordination models.

(4)

Real-world deployment and standardization: A critical step toward broader adoption lies in validating neurodynamic methods in realistic benchmark environments, such as robot swarms, UAV formations, distributed sensor networks, and autonomous vehicle systems. Moreover, establishing standardized evaluation metrics, public codebases, and simulation platforms for neurodynamic algorithms will facilitate reproducibility and practical deployment, helping bridge the gap between theoretical advances and engineering applications.

6. Conclusions

In this paper, we have provided a brief review of neurodynamics for computation and multi-agent systems in the past decade. Gradient neurodynamics, zeroing neurodynamics, and projection neurodynamics have been covered in this survey with respect to their application in computation problems, where the basic design ideas of the corresponding neurodynamic models have been illustrated. The applications of neurodynamics in multi-agent systems have also been discussed, including distributed competition and distributed coordination. Based on the survey on related literature of the past decade, research challenges and opportunities of neurodynamics for computation and multi-agent systems have also been identified.