Solutions of Optimization Problems on Hadamard Manifolds with Lipschitz Functions

Abstract

The aims of this paper are twofold. First, it is shown, for the first time, which types of nonsmooth functions are characterized by all vector critical points as being efficient or weakly efficient solutions of vector optimization problems in constrained and unconstrained scenarios on Hadamard manifolds. This implies the need to extend different concepts, such as the Karush–Kuhn–Tucker vector critical points and generalized invexity functions, to Hadamard manifolds. The relationships between these quantities are clarified through a great number of explanatory examples. Second, we present an economic application proving that Nash’s critical and equilibrium points coincide in the case of invex payoff functions. This is done on Hadamard manifolds, a particular case of noncompact Riemannian symmetric spaces.

1. Introduction

Firstly, our area of interest is the Hadamard manifolds. This paper is concerned with the pursuit of solutions of optimization problems defined on Hadamard manifolds through critical points, where the objective function may be nonsmooth. Optimal conditions are obtained under weaker assumptions than those already existing in the literature.

The idea of convex sets in a linear space is based upon the possibility of connecting any two points of the space using line segments. In nonlinear spaces such as Hadamard manifolds, linear segments are replaced by geodesic arcs. The idea behind this is the same as the one that inspired the 19th century geometricians who created non-Euclidean geometry.

The use of Hadamard manifolds has the following advantages:

(a)

Nonconvex constrained problems in ${\mathbb{R}}^n$ are transformed into convex ones in the Hadamard manifolds (see [1]).

(b)

Moreover, for example, the set $X=\left\{\right(cost,sint):t\in [\pi /4,3\pi /4\left]\right\}$ is not convex in the usual sense with $X\subset {\mathbb{R}}^2$, but X is a geodesic convex on the Poincaré upper-plane model $({\mathbb{H}}^2,g_{\mathbb{H}})$, as it is the image of a geodesic segment (see [2]).

Secondly, in this paper, we consider the concept of invexity because of the great computational advantages it offers. The optimality conditions that invexity involves are essential in obtaining optimal points through the search for critical points with practical numerical methods. The invexity concept, introduced by Hanson [3], is an extension of differentiable convexity. A scalar function is invex if and only if every critical point is a global minimum solution.

From the mind of Ben-Israel and Mond [4] the pseudoinvex functions emerged and although in the scalar case these functions coincide with the invex ones in the vector case they are different (see Ruiz-Garzón et al. ([5], Example 3.2)).

Thirdly, the nonsmooth optimization formulation is found to have several clear advantages over its smooth counterpart, the main one being that it produces exact solutions to optimization problems while smoothing variants only produce approximate solutions (see Li et al. [6]). The importance of generalizing optimization methods to locally Lipschitz functions lies in their applications. For example, in controlled thermonuclear fusion research [7], engineering [8], stereo vision processing [9], and machine learning or computer vision [10,11]. In the field of medicine, symmetric Riemannian manifolds have been used in the analysis of medical images of tumor growth, as shown by Fletcher et al. [12]. The space of diffusion tensors required in these cases is a curved manifold named as a Riemannian symmetric space. In Bejenaru and Udriste [13], the authors extended multivariate optimal control techniques to Riemannian optimization problems in order to derive a Hamiltonian approach.

Finally, for this paper, special mention should be made of studies on Nash–Stampacchia equilibria. Kristály [2,14] studied the existence and relationship of Nash’s critical and equilibrium points using strategy sets based on geodesic convex subsets of Hadamard manifolds and convex payoff functions, taking advantage of the geometrical features of these spaces. Equilibrium theory plays a very important role within the game theory created by von Neumann and Morgenstern [15] in 1944 and the development of the “Prisoner’s Dilemma" by Tucker and Nash in 1950 [16].

The state of the art is as follows. The initial idea for this article came from a paper written by Kristály [2] in which he relates Nash’s critical points and equilibrium points under conditions of convexity.

Hosseini and Pouryayevali [17] presented a subdifferential calculus for locally Lipschitz functions to prove Lebourg’s mean value theorem in Riemannian manifolds. Later, the same authors [18] obtained necessary optimality conditions for an optimization problem on complete Riemannian manifolds, but they did not obtain characterizations. Kiliçman and Saleh [19] presented a Karush–Kuhn–Tucker sufficient optimality condition as well as a new Hermite–Hadamard-type integral inequality using differentiable sub-b-s-preinvex functions.

Other authors, such as Papa Quiroz and Oliveira [20], have used the concept of subdifferentials on Hadamard manifolds to prove the global convergence of their method of solving optimization problems to the critical point of a function.

Bento and Cruz [21] developed a subgradient-type method for solving non-smooth vectorial optimization problems. Their method converges to a Pareto optimal point through a vector critical point on a manifold with nonnegative sectional curvature.

In 2012, Colao et al. [1] proved the existence of a Nash equilibrium point on Hadamard manifolds under the condition of convexity of the payoff functions.

Chen et al. [22] discussed how to obtain efficient solutions involving generalized invex functions and Karush–Kuhn–Tucker (KKT) sufficient conditions on Riemannian manifolds.

In 2014, Boumal et al. [23] authored a Matlab toolbox for optimization on manifolds (www.manopt.org). An extension of optimization methods for solving minimization problems on Hadamard manifolds when the objective function is Lipschitz was proposed by Grohs and Hosseini [24].

In 2016, Gutiérrez et al. [25] provided a characterization of pseudoinvexity through the vector critical point and found efficient solutions to multiobjective optimization problems using Lipschitz functions on linear spaces. Two years later, Ruiz-Garzón et al. [26] extended these properties on Riemannian manifolds in the smooth case. In 2019, Ruiz-Garzón et al. [27] showed the existence of KKT optimality conditions for weakly efficient Pareto solutions for vector equilibrium problems, with particular focus on the Nash equilibrium problem, but only in the differential case.

Contributions. The aim of our work is to characterize the types of nonsmooth functions for which the critical points are solutions to constrained and unconstrained optimization problems on Hadamard manifolds and to extend the results obtained by Gutiérrez et al. [25] and Ruiz-Garzón et al. [26] on linear spaces.

For this aim, in Section 2, we introduce a number of different generalized invexity concepts (pseudoinvexity and strong pseudoinvexity, respectively) and consider the so-called generalized Jacobian, a natural subdifferential associated with a locally Lipschitz function. We illustrate these new definitions of functions with examples on Hadamard manifolds.

In Section 3, the concept of pseudoinvexity allows us to determine efficient and weakly efficient Pareto solutions of an unconstrained vector optimization problem through an adequate nonsmooth vector critical point concept. As a particular case, we show that, in the scalar case and on Hadamard manifolds, the invexity and pseudoinvexity concepts coincide.

In Section 4, the vector critical point and pseudoinvexity concepts are extended from unconstrained to constrained vector optimization problems. We analyze the necessary characteristics of the objective and constraint functions of a vector optimization problem so that the KKT vector critical point is an efficient and weakly efficient solution on Hadamard manifolds in the nonsmooth case.

In Section 5, we prove the equivalence between Nash critical and equilibrium points with invex payoff functions. Finally, Section 6 presents the conclusions to this study.

2. Preliminaries

Let M be a Riemannian manifold endowed with a Riemannian metric $g_x$ on a tangent space $T_{x}M$. The corresponding norm is denoted by ${\parallel .\parallel }_x$ and the length of a piecewise $C^1$ curve $\alpha :[a,b]\to M$ is defined by

$$L\left(\alpha \right)={\int }_a^b{\parallel {\alpha }^{\prime }\left(t\right)\parallel }_{\alpha \left(t\right)}dt$$

Let d be the distance that induces the original topology on M, defined as

$$d(x,y)=\inf \left\{L\left(\alpha \right)\right|\alpha \mathrm{is}\mathrm{a}\mathrm{piecewise}{C}^1\mathrm{curve}\mathrm{joining}x\mathrm{and}y,\forall x,y\in M\}$$

It is known that any path $\alpha$ joining x and y in M such that $L\left(\alpha \right)=d(x,y)$ is a geodesic, and is called a minimal geodesic. If M is complete, then any points in M can be joined by a minimal geodesic.

The derivatives of the curves at a point x on the manifold lie in a vector space $T_{x}M$. We denote by $T_{x}M$ the n-dimensional tangent space of M at x, and denote by $TM={\bigcup }_{x\in M}{T}_{x}M$ the tangent bundle of M. Let $\overline{T}M$ be an open neighborhood of M such that $\exp :\overline{T}M\to M$ is defined as ${\exp }_x\left(v\right)={\alpha }_v(1,x)$ for every $v\in \overline{T}M$, where ${\alpha }_v$ is the geodesic starting at x with velocity v (i.e., $\alpha \left(0\right)=x,{\alpha }^{\prime }\left(0\right)=v$) [22]. It is easy to see that ${\exp }_x\left(tv\right)={\alpha }_v(t,x)$.

Let $\eta :M\times M\to TM$ be a map defined on the product manifold such that

$$\eta (x,y)\in T_y\left(M\right),\forall x,y\in M.$$

Of all the classes of Riemannian manifolds, this work is dedicated to the Hadamard manifolds.

Definition 1.

Recall that a simply connected complete Riemannian manifold of nonpositive sectional curvature is called a Hadamard manifold.

Let M be a Hadamard manifold. Then, ${\exp }_x:T_{x}M\to M$ is a diffeomorphism, and for any two points $x,y\in M$, there exists a unique minimal geodesic ${\alpha }_{x,y}={\exp }_x\left(t{\exp }_x^{-1}y\right)$ for all $t\in [0,1]$ joining x to y.

We now define a generalization of the concept of convex sets and convex functions in $R^n$:

Definition 2.

[28] A subset X of M is said to be a geodesic convex if, for any two points $x,y\in X$, the geodesic α of M has endpoints x and y belonging to X; that is, if $\alpha :[0,1]\to M$ such that $\alpha \left(0\right)=x$ and $\alpha \left(1\right)=y$, then $\alpha \left(t\right)\in X$ for all $t\in [0,1]$. Furthermore, on a Hadamard manifold, X is a geodesic convex if and only if ${exp}_y\left(t{exp}_y^{-1}x\right)\in X$.

Definition 3.

[28] Let M be a Hadamard manifold and $X\subseteq M$ be a geodesic convex. A function $\theta :X\to \mathbb{R}$ is said to be convex if, for every $x,y\in X$,

$$\theta \left({\alpha }_{x,y}\left(t\right)\right)\le tf\left(x\right)+(1-t)f\left(y\right),\forall t\in [0,1]$$

where $\alpha \left(t\right)={exp}_y\left(t{exp}_y^{-1}x\right)$ for every $t\in [0,1]$.

Let us now recall the following concepts in the nonsmooth case.

Definition 4.

A real-valued function θ defined on a Hadamard manifold M is said to satisfy a Lipschitz condition of rank k on a given subset X of M if $\left|\theta \right(x)-\theta (y\left)\right|\le kd(x,y)$ for every $x,y\in X$.

A function θ is said to be Lipschitz near $x\in M$ if it satisfies the Lipschitz condition of some rank on an open neighborhood of x.

A function θ is said to be locally Lipschitz on M if θ is Lipschitz near x for every $x\in M$.

Example 1.

The space of symmetric $n\times n$ positive-definite matrices $S(n,\mathbb{R})$ endowed with the Frobenius metric defined by $<U,V>=tr(U,V)$ is an example of Hadamard manifold. If ${\lambda }_1,\dots ,{\lambda }_n$ denote the n real eigenvalues of $A\in S(n,\mathbb{R})$ then ${\lambda }_k:S(n,\mathbb{R})\to \mathbb{R}$ is a locally Lipschitz function.

With Lipschitz functions, generalized gradients or subdifferentials replace the classical derivative.

Definition 5.

[24] Suppose $\theta :M\to \mathbb{R}$ is a locally Lipschitz function on a Hadamard manifold M. Given another point $y\in M$, consider ${\alpha }_{y,v}\left(t\right)={exp}^{-1}\left(tw\right)$ to be a geodesic passing through y with derivative w. Then, the Clarke generalized directional derivative of θ at $x\in M$ in the direction $v\in T_{x}M$, denoted by ${\theta }^0(x,v)$, is defined as

$${\theta }^0(x,v)=\underset{y\to x,t\downarrow 0}{lim\; sup}\frac{\theta \left({\alpha }_{y,v}\left(t\right)\right)-\theta \left(y\right)}{t}$$

Definition 6.

We define the subdifferential of θ at x, denoted by $\partial \theta \left(x\right)$, as the subset of $T_{x}M$ with the support function given by ${\theta }^0(x;.)$, i.e., for every $v\in T_{x}M$,

$${\theta }^0(x,v)=sup\{<A,v>:A\in \partial \theta \left(x\right)\}$$

It can be proved that the generalized Jacobian is

$$\partial \theta \left(x\right)=conv\{\underset{i\to \infty }{lim}\mathrm{grad}\theta \left(x_i\right):\left\{x_i\right\}\subseteq X,x_i\to x\}$$

where X is a dense subset of M on which $\theta$ is differentiable and $conv(·)$ denotes the convex hull.

We briefly examine some particular cases.

(a)

When $\theta$ is a locally Lipschitz convex function, we have ${\theta }^0(x;v)={\theta }^{\prime }(x;v)$ for all $x\in M$. For a convex function $\theta :M\to \mathbb{R}$, the directional derivative of $\theta$ at the point $x\in M$ in the direction $v\in T_{y}M$ is defined by

$${\theta }^{\prime }(x,v)=\underset{t\to 0^{+}}{lim}\frac{\theta \left({\exp }_x\left(tv\right)\right)-\theta \left(x\right)}{t}$$

and the subdifferential of $\theta$ at x is

$$\partial \theta \left(x\right)=\{A\in T_{x}M|{\theta }^{\prime }(x;v)\ge <A,v>,\forall v\in T_{x}M\}$$

(b)

If $\theta$ is differentiable at $x\in M$, we define the gradient of $\theta$ as the unique vector $\mathrm{grad}\theta \left(x\right)\in T_{x}M$ that satisfies

$$d{\theta }_x\left(v\right)=<grad\theta \left(x\right),v>\forall v\in T_{x}M$$

However, for the vector function $f=(f_1,\cdots ,f_p):M\to {\mathbb{R}}^p$, the generalized Jacobian $\partial f\left(x\right)$ is contained and, in general, is different from the Cartesian product of Clarke subdifferentials of the components of f.

We denote by ${\mathbb{R}}_{+}^p$ the nonnegative orthant of ${\mathbb{R}}^p$, and the order in ${\mathbb{R}}^p$ is defined in the usual way: $y,z\in {\mathbb{R}}^p$, $y\le z\iff z-y\in {\mathbb{R}}_{+}^p$ where $\mathrm{int}{\mathbb{R}}_{+}^p$ denotes the interior of ${\mathbb{R}}_{+}^p$ in ${\mathbb{R}}^p$ and $-\mathrm{int}{\mathbb{R}}_{+}^p$ the opposite of the interior (see [29]).

The notions of generalized invexity introduced by Osuna-Gómez et al. [30] for differentiable functions, and later by Gutiérrez et al. [31] for locally Lipschitz functions using the generalized Jacobian in a finite-dimensional context, can be extended to Hadamard manifolds as follows.

Definition 7.

Let M be a Hadamard manifold, X be an open geodesic convex subset of M, $\eta :M\times M\to TM$ be a not necessarily differentiable function, and $f:X\subseteq M\to {\mathbb{R}}^p$ be a locally Lipschitz function. The function f is said to be:

(a)

Invex (IX) at $\overline{x}$ with respect to η on X if, $\forall x\in X\subseteq M$, there exist some $\eta (x,\overline{x})\in T_{\overline{x}}M$, $A\in \partial f\left(\overline{x}\right)$ such that

$$f\left(x\right)-f\left(\overline{x}\right)-A\eta (x,\overline{x})\in {\mathbb{R}}_{+}^p.$$

(b)

Pseudoinvex (PIX) at $\overline{x}$ with respect to η on X if, $\forall x\in X\subseteq M$, there exist some $\eta (x,\overline{x})\in T_{\overline{x}}M$, $A\in \partial f\left(\overline{x}\right)$ such that

$$f\left(x\right)-f\left(\overline{x}\right)\in -\mathrm{int}{\mathbb{R}}_{+}^p\Rightarrow A\eta (x,\overline{x})\in -\mathrm{int}{\mathbb{R}}_{+}^p.$$

(c)

Strong pseudoinvex (SGPIX) at $\overline{x}$ with respect to η on X if, $\forall x\in X\subseteq M$, there exist some $\eta (x,\overline{x})\in T_{\overline{x}}M$, $A\in \partial f\left(\overline{x}\right)$ such that

$$f\left(x\right)-f\left(\overline{x}\right)\in -{\mathbb{R}}_{+}^p\backslash \left\{0\right\}\Rightarrow A\eta (x,\overline{x})\in -\mathrm{int}{\mathbb{R}}_{+}^p.$$

The function f is said to be invex (resp. pseudoinvex, strong pseudoinvex) with respect to η on X if, for every $x\in X$, f is invex (resp. pseudoinvex, strong pseudoinvex) at x with respect to η on X.

The following examples illustrate the above definitions and relations on Hadamard manifolds.

Example 2.

Let $\mathsf{\Omega }=\{p=(p_1,p_2)\in {\mathbb{R}}^2:p_2>0\}$ be a set and let G be a $2\times 2$ matrix defined by $G\left(p\right)=(g_{ij}\left(p\right))$ with

$$g_{11}\left(p\right)=g_{22}\left(p\right)=\frac{1}{p_2^2},g_{12}\left(p\right)=g_{21}\left(p\right)=0$$

Endowing Ω with the Riemannian metric $\ll u,v\gg =<G\left(p\right)u,v>$, we obtain a complete Riemannian manifold ${\mathbb{H}}^2$, namely, the upper half-plane model of hyperbolic space.

Let $f\left(p\right)=(f_1,f_2)(p_1,p_2):\mathsf{\Omega }\to {\mathbb{R}}^2$ be a function with $f_1(p_1,p_2)=p_1+5p_2$ and

$$f_2(p_1,p_2)=\left\{\begin{array}{cc}-3p_1+3\hfill & if{p}_1<1\hfill \\ 0\hfill & if1\le p_1\le 2\hfill \\ p_1-2\hfill & if{p}_1>2\hfill \end{array}\right.$$

The function f is invex on Ω because its components are linear functions, i.e., convex and concave functions simultaneously.

Example 3.

Let $f\left(p\right)=(f_1,f_2)(p_1,p_2):\mathsf{\Omega }\to {\mathbb{R}}^2$ be a function with $f\left(p\right)=\left(\right|p_1+p_2|,-|p_1+p_2\left|\right)$. The function f is strong pseudoinvex with respect to any η because $f\left(p\right)-f\left(\overline{p}\right)\in -{\mathbb{R}}_{+}^p\backslash \left\{0\right\}$ is not satisfied for $p\in \mathsf{\Omega }$.

Example 4.

Let Ω be the upper half-plane model of hyperbolic space with the Riemannian metric $\ll u,v\gg =<G\left(p\right)u,v>$, let G be a $2\times 2$ matrix defined by $G\left(p\right)=\left(g_{ij}\left(p\right)\right)$ with

$$g_{11}\left(p\right)=g_{22}\left(p\right)=\frac{1}{p_2^2},g_{12}\left(p\right)=g_{21}\left(p\right)=0$$

and $A=G{\left(p\right)}^{-1}\nabla \theta \left(p\right)\in \partial f\left(p\right)$. Let $f\left(p\right)=(f_1,f_2)(p_1,p_2):\mathsf{\Omega }\to {\mathbb{R}}^2$ be a function defined as $f_1(p_1,p_2)=p_1$ and

$$f_2(p_1,p_2)=\left\{\begin{array}{cc}-p_1,\hfill & if{p}_1<0\hfill \\ 0,\hfill & if0\le p_1\le 1\hfill \\ 1-p_1,\hfill & if{p}_1>1\hfill \end{array}\right.$$

We are going to prove that f is a pseudoinvex function but not strong pseudoinvex or invex.

We have that the following:

(a)

If $p_1<0$ or $p_1>1$, then

$$f^0(p,v)=\langle \left(\begin{array}{cc}{p}_2^2& 0\\ 0& p_2^2\end{array}\right)\left(\begin{array}{cc}1& 0\\ -1& 0\end{array}\right),\left(\begin{array}{c}{v}_1\\ v_2\end{array}\right)\rangle =\left(\begin{array}{cc}{p}_2^2& 0\\ -p_2^2& 0\end{array}\right)\left(\begin{array}{c}{v}_1\\ v_2\end{array}\right)=\left(\begin{array}{c}{p}_2^2{v}_1\\ -p_2^2{v}_1\end{array}\right)$$

(b)

If $p_1=0$ or $p_1=1$ and $-1\le a\le 0$, then

$$f^0(p,v)=\langle \left(\begin{array}{cc}{p}_2^2& 0\\ 0& p_2^2\end{array}\right)\left(\begin{array}{cc}1& 0\\ a& 0\end{array}\right),\left(\begin{array}{c}{v}_1\\ v_2\end{array}\right)\rangle =\left(\begin{array}{cc}{p}_2^2& 0\\ p_2^{2}a& 0\end{array}\right)\left(\begin{array}{c}{v}_1\\ v_2\end{array}\right)=\left(\begin{array}{c}{p}_2^2{v}_1\\ p_2^{2}a{v}_1\end{array}\right)$$

(c)

If $0<p_1<1$, then

$$f^0(p,v)=\langle \left(\begin{array}{cc}{p}_2^2& 0\\ 0& p_2^2\end{array}\right)\left(\begin{array}{cc}1& 0\\ 0& 0\end{array}\right),\left(\begin{array}{c}{v}_1\\ v_2\end{array}\right)\rangle =\left(\begin{array}{cc}{p}_2^2& 0\\ 0& 0\end{array}\right)\left(\begin{array}{c}{v}_1\\ v_2\end{array}\right)=\left(\begin{array}{c}{p}_2^2{v}_1\\ 0\end{array}\right)$$

In summary,

$$\partial f\left(p\right)=\left\{\begin{array}{cc}\left(\begin{array}{cc}{p}_2^2& 0\\ -p_2^2& 0\end{array}\right)\hfill & if{p}_1<0or{p}_1>1\hfill \\ \left(\begin{array}{cc}{p}_2^2& 0\\ p_2^{2}a& 0\end{array}\right)\hfill & if{p}_1=0or{p}_1=1with-1\le a\le 0\hfill \\ \left(\begin{array}{cc}{p}_2^2& 0\\ 0& 0\end{array}\right)\hfill & if0<p_1<1.\hfill \end{array}\right.$$

The function f is pseudoinvex with respect to every $\eta (p,\overline{p})=3p-\overline{p}=(v_1,v_2)$ on Ω because $f\left(p\right)-f\left(\overline{p}\right)\in -\mathrm{int}{\mathbb{R}}_{+}^2$ implies that f should be nondecreasing, but $f_2$ is nonincreasing and this previous condition is not satisfied.

However, f is not strong pseudoinvex on Ω with respect to any $\eta (p,\overline{p})=(v_1,v_2)$ because we can choose $p=(0,1)$, $\overline{p}=(1,1)$, and

$$A=\left(\begin{array}{cc}{p}_2^2& 0\\ p_2^{2}a& 0\end{array}\right)\in \partial f(1,1)$$

and then

$$f\left(p\right)-f\left(\overline{p}\right)=f(0,1)-f(1,1)=(0,0)-(1,0)=(-1,0)\in -{\mathbb{R}}_{+}^2\backslash \left\{(0,0)\right\},$$

but there exists no $\eta (p,\overline{p})=(v_1,v_2)\in T_{\overline{p}}\mathsf{\Omega }$ such that $A\eta (p,\overline{p})=(p_2^2{v}_1,p_2^{2}a{v}_1)\in -\mathrm{int}{\mathbb{R}}_{+}^2$ with

$-1\le a\le 0$.

In the same manner, f is not invex on Ω because if we choose $p=(0,1)$ and $\overline{p}=(1,1)$, there exists no $\eta (p,\overline{p})=(v_1,v_2)\in T_{\overline{p}}\mathsf{\Omega }$ such that

$$f\left(p\right)-f\left(\overline{p}\right)-A\eta (p,\overline{p})=(0,0)-(1,0)-(p_2^2{v}_1,p_2^{2}a{v}_1)\in {\mathbb{R}}_{+}^2\forall a\in [-1,0].$$

(1)

Expression (1) implies that $-1\ge v_1$ and $0\ge a{v}_1$, but for $a=-1$, there is a contradiction between them.

In summary, it is well known that invexity and strong pseudoinvexity imply pseudoinvexity (see [31]), but we have found that pseudoinvexity does not imply either invexity or strong pseudoinvexity.

$$IX\Rightarrow PIX\Leftarrow SGPIX$$

We now have all the tools required to discuss critical points and solutions of vector optimization problems in the next section.

3. Relations between Solutions of Vector Optimization Problems and Vector Critical Points on Hadamard Manifolds

The objective of this section is to check whether nonsmooth optimality conditions obtained in linear spaces can be extended to Hadamard manifolds.

In Ruiz-Garzón et al. [26], we studied the role of invexity in the scalar case on Riemannian manifolds for the differential scenario, but not that of pseudoinvexity. In this section, we study the role of pseudoinvexity in both the scalar and vector cases on the Hadamard manifolds in unconstrained VOPs when the functions are nondifferentiable. We examine when vectorial critical points coincide with efficient and weakly efficient points.

In this section, we consider the unconstrained multiobjective programming problem (VOP) defined as:

$$\begin{array}{ll}\left(\mathrm{VOP}\right)& minf\left(x\right)\\ & x\in X\subseteq M\end{array}$$

where $f=(f_1,\cdots f_p):X\subseteq M\to {\mathbb{R}}^p$, with $f_i:X\subseteq M\to \mathbb{R}$ for all $i:1,\dots ,p$, locally Lipschitz functions on the open set $X\subseteq M$, and M assumed to be a Riemannian manifold.

Let us recall two classic concepts of vectorial optimization:

Definition 8.

A feasible point $\overline{x}$ is said to be:

(a)

An efficient solution for (VOP) if there does not exist another feasible point x such that

$$f\left(x\right)-f\left(\overline{x}\right)\in -{\mathbb{R}}_{+}^p\backslash \left\{0\right\}.$$

(b)

A weakly efficient solution for (VOP) if there does not exist another feasible point x such that

$$f\left(x\right)-f\left(\overline{x}\right)\in -\mathrm{int}{\mathbb{R}}_{+}^p.$$

We now study some relations between solutions of (VOP) and vector critical points. We will start by defining the concept of the vector critical point:

Definition 9.

Let M be a Hadamard manifold, X be an open geodesic convex subset of M, and $f:X\subseteq M\to {\mathbb{R}}^p$ be a locally Lipschitz function. A feasible point $\overline{x}\in X$ is said to be a vector critical point (VCP) with respect to η if there exist some $x\in X\subseteq M$ with $\eta (x,\overline{x})\in T_{\overline{x}}M$ not identically zero and $\lambda \in {\mathbb{R}}_{+}^p\backslash \left\{0\right\}$ such that ${\lambda }^{T}A\eta (x,\overline{x})=0$ for some $A\in \partial f\left(\overline{x}\right)$.

The importance of VCPs in obtaining weakly efficient points (efficient points) can be illustrated through a characterization of pseudoinvexity (resp. strong pseudoinvexity).

Theorem 1.

Let M be a Hadamard manifold, X be an open geodesic convex subset of M, and $f:X\subseteq M\to {\mathbb{R}}^p$ be a locally Lipschitz function. Every VCP with respect to η is a weakly efficient solution of (VOP) if and only if the function f is PIX with respect to the same η on X.

Proof. 

Firstly, we prove that f is pseudoinvex with respect to $\eta$.

(a)

We consider two points $x,\overline{x}\in X$ and assume that $f\left(x\right)-f\left(\overline{x}\right)\in -\mathrm{int}{\mathbb{R}}_{+}^p$. Then, $\overline{x}$ is not a weakly efficient solution of (VOP). By the hypothesis, we derive that $\overline{x}$ is not a VCP with respect to $\eta$, i.e., there do not exist some $x\in X\subseteq M$ with $\eta (x,\overline{x})\in T_{\overline{x}}M$ not identically zero and $\lambda \in {\mathbb{R}}_{+}^p\backslash \left\{0\right\}$ such that ${\lambda }^{T}A\eta (x,\overline{x})=0$ for some $A\in \partial f\left(\overline{x}\right)$. It follows from ([29], Theorem 5.1) that $A\eta (x,\overline{x})\in -\mathrm{int}{\mathbb{R}}_{+}^p$ and f is PIX.

(b)

For any points $x,\overline{x}\in X$ such that $f\left(x\right)-f\left(\overline{x}\right)\notin -\mathrm{int}{\mathbb{R}}_{+}^p$, we define $\eta (x,\overline{x})=0$, and therefore f is PIX with respect to $\eta$ on X.

We now prove the sufficient condition. We assume by hypothesis that f is PIX with respect to $\eta$ and that $\overline{x}$ is a VCP with respect to the same $\eta$. Thus,

$${\lambda }^{T}A\eta (x,\overline{x})=0$$

(2)

for some $x\in X\subseteq M$ with $\eta (x,\overline{x})\in T_{\overline{x}}M$, $\lambda \in {\mathbb{R}}_{+}^p\backslash \left\{0\right\}$, and $A\in \partial f\left(\overline{x}\right)$.

We need to prove that $\overline{x}$ is a weakly efficient point. By reductio ad absurdum, suppose that $\overline{x}$ is not a weakly efficient solution of (VOP). Then, there exists a point $x\in X$ such that $f\left(x\right)-f\left(\overline{x}\right)\in -\mathrm{int}{\mathbb{R}}_{+}^p$. Using the fact that f is PIX at $\overline{x}$ with respect to $\eta$ on X, we have $A\eta (x,\overline{x})\in -\mathrm{int}{\mathbb{R}}_{+}^p$, and so ${\lambda }^{T}A\eta (x,\overline{x})<0$, which contradicts (2). □

In the same way, we can prove the following corollary.

Corollary 1.

Let M be a Hadamard manifold, X be an open geodesic convex subset of M, and $f:X\subseteq M\to {\mathbb{R}}^p$ be a locally Lipschitz function. Every VCP with respect to η is an efficient solution of (VOP) if and only if the function f is strong pseudoinvex (SGPIX) with respect to η on X.

Let us underline that Theorem 1 and Corollary 1 show that pseudoinvexity (resp. strong pseudoinvexity) is a minimal requirement for the property that every VCP is a weakly efficient (resp. efficient) solution of problem (VOP) on a Hadamard manifold in the nonsmooth case.

In summary, we have that

$$\left[\mathrm{VCP}\iff \mathrm{WEff}\left(\mathrm{VOP}\right)\right]\iff \mathrm{PIX}$$

$$\left[\mathrm{VCP}\iff \mathrm{Eff}\left(\mathrm{VOP}\right)\right]\iff \mathrm{SGPIX}$$

Theorem 1 extends Theorem 2.2 of Osuna et al. [30] and Theorem 5 of Gutiérrez et al. [25] from linear spaces to Hadamard manifolds.

Next, an example is given to demonstrate the applicability of the previous results.

Example 5.

Consider the unconstrained vector optimization problem:

$$\begin{array}{ll}\left(VOP\right)& minf\left(p\right)=(f_1,f_2)(p_1,p_2)\\ & subjecttop\in \mathsf{\Omega }\end{array}$$

Consider the function f of Example 4. It was proved that f is pseudoinvex with respect to $\eta (p,\overline{p})=3p-\overline{p}$ on $\mathsf{\Omega }=\{p=(p_1,p_2)\in {\mathbb{R}}^2:p_2>0\}$.

It is easy to choose some $x\in X\subseteq M$ with $\eta (x,\overline{x})\in T_{\overline{x}}M$ not identically zero and $\lambda \in {\mathbb{R}}_{+}^p\backslash \left\{0\right\}$ such that ${\lambda }^{T}A\eta (x,\overline{x})=0$ for some $A\in \partial f\left(\overline{x}\right)$, and therefore $VCP=\mathsf{\Omega }$. By applying Theorem 1, we conclude that $\mathrm{WEff}\left(VOP\right)=\mathsf{\Omega }$.

For scalar functions, we can go one step further.

Corollary 2.

Assume that $\theta :\mathsf{\Omega }\to \mathbb{R}$ is locally Lipschitz and $X\subset \mathsf{\Omega }$ is open. Then, the following statements are equivalent:

(a)

θ is invex (IX) with respect to η on X.

(b)

Every critical point (CP) of θ with respect to η on X is a global minimum of θ on X.

(c)

θ is PIX with respect to η on X.

Proof. 

$\left(a\right)\Rightarrow \left(b\right)$ If $\theta$ is IX at $\overline{x}$, then $\forall x\in X\subseteq M$ there exist some $\eta (x,\overline{x})\in T_{\overline{x}}M$, $A\in \partial \theta \left(\overline{x}\right)$ such that

$$\theta \left(x\right)-\theta \left(\overline{x}\right)-A\eta (x,\overline{x})\ge 0$$

(3)

If $\overline{x}$ is a VCP, then there exists some $\lambda \ge 0$ such that

$${\lambda }^{T}A\eta (x,\overline{x})=0$$

(4)

for some $A\in \partial \theta \left(\overline{x}\right)$. From (3) and (4), this implies that

$$\theta \left(x\right)-\theta \left(\overline{x}\right)\ge 0$$

and thus, $\overline{x}$ is a global minimum.

$\left(b\right)\Rightarrow \left(a\right)$ We will prove that, $\forall x\in X\subseteq M$, there exist some $\eta (x,\overline{x})\in T_{\overline{x}}M$, $A\in \partial \theta \left(\overline{x}\right)$ such that

$$\theta \left(x\right)-\theta \left(\overline{x}\right)-A\eta (x,\overline{x})\ge 0$$

• Firstly, if

  $$\theta \left(x\right)-\theta \left(\overline{x}\right)<0$$

  (5)

  then there exist some $\eta (x,\overline{x})\in T_{\overline{x}}M$, $A\in \partial \theta \left(\overline{x}\right)$ such that $A\eta (x,\overline{x})<0$. This is because, if $A\eta (x,\overline{x})\ge 0$, then $\overline{x}$ will be a VCP and $\overline{x}$ is a global minimum, i.e.,

  $$\theta \left(x\right)-\theta \left(\overline{x}\right)\ge 0,\forall x\in X$$

  which contradicts (5).
  Therefore, $A\eta (x,\overline{x})<0$. Then, as ${\theta }^0(x,·)$ is positively homogeneous, it follows that $\theta \left(x\right)-\theta \left(\overline{x}\right)\ge A\eta (x,\overline{x})$, and thus $\theta$ is IX with respect to $\eta (x,\overline{x})=tv$, where t is an arbitrary positive real number.
• Secondly, if

  $$\theta \left(x\right)-\theta \left(\overline{x}\right)\ge 0$$

  then $\theta$ is IX with respect to $\eta (x,\overline{x})=0$.

$\left(b\right)\iff \left(c\right)$ The result is given by Theorem 1. □

In summary,

$$\left[\mathrm{CP}\iff \mathrm{Minimum}\right]\iff \mathrm{IX}\iff \mathrm{PIX}$$

Corollary 2 provides us with a necessary and sufficient invexity condition for locally Lipschitz functions on Hadamard manifolds. It extends a result given by Gutiérrez et al. [25] for Euclidean spaces. In Ruiz-Garzón et al. [26], only invexity was characterized on Riemannian manifolds; now, we have shown that invexity and pseudoinvexity coincide. They describe a wider class of differentiable and locally Lipschitz functions in which the critical points are global minima in unconstrained problems on Hadamard manifolds.

The question that now arises is whether, in the case of the constrained vector optimization problem, solutions and vector critical points also coincide when applying pseudoinvexity assumptions.

4. Relations between Solutions of the Constrained VOP and KKT VCPs on Hadamard Manifolds

The objective of this section is to extend the results obtained in the previous section for the unconstrained case to the constrained case. We want to determine the conditions under which KKT VCPs and efficient and weakly efficient points coincide.

We consider the constrained multiobjective programming problem (CVOP) defined as:

$$\begin{array}{cc}\hfill \left(\mathrm{CVOP}\right)& minf\left(x\right)\hfill \\ & g\left(x\right)\underline{\le }0\hfill \\ & x\in X\subseteq M\hfill \end{array}$$

where $f=(f_1,\cdots f_p):X\subseteq M\to {\mathbb{R}}^p$, with $f_i:X\subseteq M\to \mathbb{R}$ for all $i:1,\dots ,p$, $g=(g_1,\cdots ,g_m):X\subseteq M\to {\mathbb{R}}^m$ are locally Lipschitz functions on the open set $X\subseteq M$, and M is a Riemannian manifold.

As for the unconstrained case, we are going to use KKT VCPs, which are defined as follows.

Definition 10.

A feasible point $\overline{x}$ for (CVOP) is said to be a KKT VCP with respect to η if there exist some $x\in X\subseteq M$ with $\eta (x,\overline{x})\in T_{\overline{x}}M$, $\lambda \in {\mathbb{R}}^p$, $\mu \in {\mathbb{R}}^m$, $A\in \partial f\left(\overline{x}\right)$, $Bj\in \partial g_j\left(\overline{x}\right),j\in I\left(\overline{x}\right)$ such that

$${\lambda }^{T}A\eta (x,\overline{x})+{\mu }_{I\left(\overline{x}\right)}^T{B}_{I\left(\overline{x}\right)}\eta (x,\overline{x})=0$$

(6)

$${\mu }^{T}g\left(\overline{x}\right)=0$$

(7)

$$\mu \underline{\ge }0$$

(8)

$$\lambda \ge 0$$

(9)

where$I\left(\overline{x}\right)=\{j=1,\cdots ,m:g_j\left(\overline{x}\right)=0\}$.

A new type of invex function that involves the objective and constraint functions is needed to study the efficient solutions for (CVOP) using KKT VCPs.

Definition 11.

Problem (CVOP) is said to be KKT-pseudoinvex (KKT-PIX) at $\overline{x}$ with respect to $\eta :M\times M\to TM$ if, $\forall x\in X\subseteq M$, there exist some $\eta (x,\overline{x})\in T_{\overline{x}}M$, $A\in \partial f\left(\overline{x}\right)$, $Bj\in \partial g_j\left(\overline{x}\right),j\in I\left(\overline{x}\right)$ such that

$$f\left(x\right)-f\left(\overline{x}\right)\in -\mathrm{int}{\mathbb{R}}_{+}^p\Rightarrow \left\{\begin{array}{c}A\eta (x,\overline{x})\in -\mathrm{int}{\mathbb{R}}_{+}^p\hfill \\ B_{I\left(\overline{x}\right)}\eta (x,\overline{x})\in -{\mathbb{R}}_{+}^p\backslash \left\{0\right\}\hfill \end{array}\right.$$

Definition 12.

Problem (CVOP) is said to be strong KKT-pseudoinvex (SG-KKT-PIX) with respect to $\eta :M\times M\to TM$ if, $\forall x\in X\subseteq M$, there exist some $\eta (x,\overline{x})\in T_{\overline{x}}M$, $A\in \partial f\left(\overline{x}\right)$, $Bj\in \partial g_j\left(\overline{x}\right),j\in I\left(\overline{x}\right)$ such that

$$f\left(x\right)-f\left(\overline{x}\right)\in -{\mathbb{R}}_{+}^p\backslash \left\{0\right\}\Rightarrow \left\{\begin{array}{c}A\eta (x,\overline{x})\in -\mathrm{int}{\mathbb{R}}_{+}^p\hfill \\ B_{I\left(\overline{x}\right)}\eta (x,\overline{x})\in -{\mathbb{R}}_{+}^p\backslash \left\{0\right\}\hfill \end{array}\right.$$

Remark 1.

Obviously, if there are no constraints, these definitions coincide with those given in the preliminaries and are an extension to Hadamard manifolds of those given by Osuna et al. [30,32] and Gutiérrez et al. [31].

The following theorem shows us the importance and usefulness of (CVOP) being SG-KKT-PIX in locating the efficient points through the KKT-VCP points.

Theorem 2.

Every KKT-VCP with respect to η is an efficient solution of (CVOP) if and only if (CVOP) is SG-KKT-PIX with respect to the same η.

Proof. 

We prove that (CVOP) is SG-KKT-PIX with respect to $\eta$ at $\overline{x}$. Let us suppose that there exists some $x\in X\subseteq M$ such that

$$f\left(x\right)-f\left(\overline{x}\right)\in -{\mathbb{R}}_{+}^p\backslash \left\{0\right\},$$

(10)

because otherwise (CVOP) would be SG-KKT-PIX with respect to $\eta$, and the result would be proved. From (10), we have that $\overline{x}$ is not an efficient solution, and using the initial hypothesis, $\overline{x}$ is not a KKT-VCP, i.e., then there exist some $A\in \partial f\left(\overline{x}\right)$, $B_j\in \partial g_j\left(\overline{x}\right),j\in I\left(\overline{x}\right)$ where

$${\lambda }^{T}Av+{\mu }_{I\left(\overline{x}\right)}^T{B}_{I\left(\overline{x}\right)}v=0$$

has no solution $\overline{\lambda }\ge 0,{\overline{\mu }}_{I\left(\overline{x}\right)}\underline{\ge }0$. Therefore, by Motzkin’s Alternative theorem [33], the system

$$\left.\begin{array}{c}Av<0\hfill \\ B_{I\left(\overline{x}\right)}v\underline{\le }0,\hfill & j\in I\left(\overline{x}\right)\hfill \end{array}\right\}$$

has the solution $v=\eta (x,\overline{x})\in T_{\overline{x}}M$. In consequence, (CVOP) is SG-KKT-PIX.

Let us now prove the reciprocal condition. Let $\overline{x}$ be a KKT-VCP with respect to $\eta$ and (CVOP) be SG-KKT-PIX with respect to the same $\eta$. We have to prove that $\overline{x}$ is an efficient solution for (CVOP). By reductio ad absurdum, consider a feasible point x such that

$$f\left(x\right)-f\left(\overline{x}\right)\in -{\mathbb{R}}_{+}^p\backslash \left\{0\right\}$$

By hypothesis, (CVOP) is SG-KKT-PIX with respect to $\eta$ at $\overline{x}$ if, $\forall x\in X\subseteq M$, there exist some $\eta (x,\overline{x})\in T_{\overline{x}}M$, $A\in \partial f\left(\overline{x}\right)$, $Bj\in \partial g_j\left(\overline{x}\right),j\in I\left(\overline{x}\right)$ such that

$$\left.\begin{array}{c}A\eta (x,\overline{x})\in -\mathrm{int}{\mathbb{R}}_{+}^p\hfill \\ B_{I\left(\overline{x}\right)}\eta (x,\overline{x})\in -{\mathbb{R}}_{+}^p\backslash \left\{0\right\}\hfill \end{array}\right\}$$

(11)

As $\overline{x}$ is a KKT-VCP, then $\exists (\overline{\lambda },{\overline{\mu }}_{I\left(\overline{x}\right)})\ge 0,\overline{\lambda }\ne 0$ and $v=\eta (x,\overline{x})\in T_{\overline{x}}M$ not identically zero such that there exist $A\in \partial f\left(\overline{x}\right)$, $B_j\in \partial g_j\left(\overline{x}\right),j\in I\left(\overline{x}\right)$ for which

$${\overline{\lambda }}^{T}A\eta (x,\overline{x})+{\overline{\mu }}_{I\left(\overline{x}\right)}^T{B}_{I\left(\overline{x}\right)}\eta (x,\overline{x})=0$$

(12)

However, as $\overline{\lambda }\ge 0$, ${\overline{\mu }}_{I\left(\overline{x}\right)}\underline{\ge }0$ and from (11), it follows that

$${\overline{\lambda }}^{T}A\eta (x,\overline{x})+{\overline{\mu }}_{I\left(\overline{x}\right)}^T{B}_{I\left(\overline{x}\right)}\eta (x,\overline{x})<0$$

which contradicts (12). Therefore, $\overline{x}$ is an efficient solution for (CVOP). □

Arguing in the same form, we can prove the following corollary.

Corollary 3.

Every KKT-VCP is a weakly efficient solution of (CVOP) if and only if (CVOP) is KKT-PIX with respect to η.

In summary, we have that:

$$\left[\mathrm{KKT}-\mathrm{VCP}\iff \mathrm{WEff}\left(\mathrm{VOP}\right)\right]\iff \mathrm{KKT}-\mathrm{PIX}$$

$$\left[\mathrm{KKT}-\mathrm{VCP}\iff \mathrm{Eff}\left(\mathrm{VOP}\right)\right]\iff \mathrm{SG}-\mathrm{KKT}-\mathrm{PIX}$$

These results extend Theorem 3.7 and Corollary 3.8 obtained by Ruiz et al. [26] on Hadamard manifolds from the differentiable case to the nondifferentiable case, and extend Theorem 3.7 obtained by Osuna et al. [32] or Theorem 2.3 obtained by Osuna et al. [30] in finite-dimensional Euclidean spaces.

We illustrate the above results with an example.

Example 6.

Consider the following constrained vector optimization problem:

$$\begin{array}{lll}\left(CVOP\right)& minf\left(p\right)& =(f_1,f_2)(p_1,p_2)\\ & subjectto& g_1\left(p\right)=2p_1-1\ge 0\\ & & g_2\left(p\right)=p_2-1/2\ge 0\\ & & p\in \mathsf{\Omega }\end{array}$$

where$f_1(p_1,p_2)=p_1$and$f_2(p_1,p_2)=\left\{\begin{array}{cc}-p_1,\hfill & if{p}_1<0\hfill \\ 0,\hfill & if0\le p_1\le 1\hfill \\ 1-p_1,\hfill & if{p}_1>1\hfill \end{array}\right.$

Let Ω be the upper half-plane model of hyperbolic space and use the Riemannian metric. We will prove that $\overline{p}=(1/2,1/2)$ is a weakly efficient solution for (CVOP). There exists $\eta (p,\overline{p})=3p-\overline{p}=(1,0)$ such that

$$A\eta (p,\overline{p})=\langle \left(\begin{array}{cc}{p}_2^2& 0\\ 0& p_2^2\end{array}\right)\left(\begin{array}{cc}1& 0\\ 0& 0\end{array}\right),\left(\begin{array}{c}1\\ 0\end{array}\right)\rangle =\left(\begin{array}{cc}{p}_2^2& 0\\ 0& 0\end{array}\right)\left(\begin{array}{c}1\\ 0\end{array}\right)=\left(\begin{array}{c}{p}_2^2\\ 0\end{array}\right)$$

$$B_{I\left(\overline{p}\right)}\eta (p,\overline{p})=\langle \left(\begin{array}{cc}{p}_2^2& 0\\ 0& p_2^2\end{array}\right)\left(\begin{array}{cc}2& 0\\ 0& 1\end{array}\right),\left(\begin{array}{c}1\\ 0\end{array}\right)\rangle =\left(\begin{array}{cc}2p_2^2& 0\\ 0& p_2^2\end{array}\right)\left(\begin{array}{c}1\\ 0\end{array}\right)=\left(\begin{array}{c}2p_2^2\\ 0\end{array}\right)$$

and we can choose $\lambda =\mu =(0,1)$ such that

$${\lambda }^{T}A\eta (p,\overline{p})+{\mu }_{I\left(\overline{p}\right)}^T{B}_{I\left(\overline{p}\right)}\eta (p,\overline{p})=0$$

$${\mu }^{T}g\left(\overline{p}\right)=0$$

$$\mu \underline{\ge }0$$

$$\lambda \ge 0$$

hold. Thus, $\overline{p}=(1/2,1/2)$ is a KKT-VCP and (CVOP) is KKT-PIX with respect to the same $\eta =3p-\overline{p}$. By Corollary 3, $\overline{p}$ is a weakly efficient solution.

5. Application: Relations between Nash Equilibrium Points and Nash Critical Points

In this section, we relate Nash’s equilibrium and critical points. A Nash strategy requires n players, each optimizing his own criterion given that all other criteria are fixed by the rest of the players. When no player can further improve his criterion, then a change of strategy by one player does not cause the other players to change their strategies. In this case, the system has reached a state called Nash equilibrium. When the equilibrium is achieved, none of the players has an incentive to unilaterally deviate from this point. In general, there may be one or more Nash equilibrium points.

The following concepts were described by Kristály [2].

Definition 13.

Let $K_1,\dots ,K_n$ be the nonempty sets of strategies of the players and $f_i:K=K_1\times ,\dots ,\times K_n\to \mathbb{R}$ be the payoff functions. A point $p=(p_1,\dots ,p_n)\in K$ is a Nash equilibrium point (NEP) for ($f,K)$ if

$$f_i(p;q_i)\ge f_i\left(p\right)\forall q_i\in K,i=1,\dots ,n$$

where $(p;q_i)=(p_1,\dots ,q_i,\dots ,p_n)$.

Definition 14.

Let $M_i$ be complete finite-dimensional Riemannian manifolds, $K_i\subset M_i$ be nonempty, geodesic convex sets, and $f_i:(K;D_i)\to \mathbb{R}$ be functions such that $D_i\ni q_i\to f_i(p;q_i)$ is locally Lipschitz for every $p\in K$, where $(K;D_i)=K_1\times \dots ,\times D_i\times \dots K_n$, with $D_i$ open and geodesic convex and $K_i\subseteq D_i\subseteq M_i$ with $i=1,2\dots ,n$. A point $p\in K$ is a Nash critical point (NCP) for $(f,K)$ if

$$f_i^0(p,{exp}_{p_i}^{-1}\left(q_i\right))\ge 0,\forall q_i\in K,i=1,\dots ,n$$

We can relate Nash’s critical points and equilibrium points in the following theorem, the proof of which contains steps similar to that used for Proposition 1.2 of Kristály [2]:

Theorem 3.

Any NEP for $(f,K)$ is an NCP. If $q_i\to f_i(p;q_i)$ is invex with respect to $\eta (p_i,q_i)={exp}_{p_i}^{-1}\left(q_i\right)$ for every $p\in K,i=1,\dots ,n$, the converse also holds.

Proof. 

Let $p\in K$ be an NEP for $(f,K)$∀ fixed $q_i\in K,i=1,\dots ,n$. Then,

$$f_i\left(p,{\exp }_{p_i}\left(t{\exp }_{p_i}^{-1}\left(q_i\right)\right)\right)-f_i\left(p\right)\ge 0,\forall t\in [0,1]$$

(13)

Additionally,

$$\underset{q_i\to p_i}{lim}{f}_i(p;q_i)=f_i\left(p\right)$$

(14)

and for every $t\in [0,1]$ we have

$$\underset{q_i\to p_i}{lim}{\alpha }_{q_i,{\exp }_{p_i}^{-1}\left(q_i\right)}^i\left(t\right)={\exp }_{p_i}\left(t{\exp }_{p_i}^{-1}\left(q_i\right)\right)$$

(15)

Note that

$$f_i^0(p,{\exp }_{p_i}^{-1}\left(q_i\right))=\underset{q_i\to p_i,t\downarrow 0^{+}}{lim\; sup}\frac{f_i\left(p;{\alpha }_{q_i,{\exp }_{p_i}^{-1}\left(q_i\right)}^i\right)\left(t\right)-f_i(p;q_i)}{t}$$

(16)

Therefore, from (14)–(16), it follows that

$$f_i^0(p,{\exp }_{p_i}^{-1}\left(q_i\right))\ge 0$$

(17)

Thus, $p\in K$ is an NCP for $(f,K)$.

We will prove the sufficient condition. Suppose that $p\in K$ is an NCP for $(f,K)$. We have

$$0\le f_i^0(p,{\exp }_{p_i}^{-1}\left(q_i\right))=\underset{t\to 0^{+}}{lim}\frac{f_i\left(p;{\exp }_{p_i}\left(t{\exp }_{p_i}^{-1}\left(q_i\right)\right)\right)-f_i\left(p\right)}{t}$$

(18)

Based on the invexity of $f_i$, (17) implies that

$$0\le f_i\left(p;{\exp }_{p_i}\left({\exp }_{p_i}^{-1}\left(q_i\right)\right)\right)-f_i\left(p\right)=f_i(p;q_i)-f_i\left(p\right)$$

(19)

Thus, p is an NEP for $(f,K)$. □

We have proven that the relationship between Nash’s critical and equilibrium points is obtained for invex payoff functions, extending the results obtained for convex payoff functions given by Kristály [2].

In summary, in invexity environments, we have that:

$$\mathrm{NCP}\iff \mathrm{NEP}$$

Let us illustrate this property with an example.

Example 7.

Let $K_1=K_2=[-2,2]$ and consider a two-player game with payoff functions defined as:

$$f_1(x,y)=x^2-3x+\left|x\right|y$$

$$f_2(x,y)=\frac{y^2}{2}-xy$$

We are going to prove that the point $(\overline{x},\overline{y})=(1,1)$ is an NEP and an NCP simultaneously.

We have that $f_1(·,y)$ is a locally Lipschitz function on $\mathbb{R}$ for every $y\in K_2$ and $f_2(x,·)$ is $C^1$ function on $\mathbb{R}$ for every $y\in K_1$.

One hand, we can calculate the subdifferential:

$$\partial f_1(x,y)=\left\{\begin{array}{cc}2x-3+y,\hfill & ifx>0\hfill \\ 2x-3-y,\hfill & ifx<0\hfill \\ 2x-3+a,-1\le a\le 1,\hfill & ifx=0\hfill \end{array}\right.$$

$$\partial f_2(x,y)=y-x$$

The NCPs are the solutions $(\overline{x},\overline{y})\in K$ of the system:

$$\left\{\begin{array}{cc}{f}_1^0((\overline{x},\overline{y}),{\exp }_{\overline{x}}^{-1}\left(q\right))=<\partial f_1(\overline{x},\overline{y}),(q-\overline{x})>\ge 0\hfill & \forall q\in K_1\hfill \\ f_2^0((\overline{x},\overline{y}),{\exp }_{\overline{y}}^{-1}\left(q\right))=(\overline{y}-\overline{x})(q-\overline{y})\ge 0,\hfill & \forall q\in K_2\hfill \end{array}\right.$$

On the other hand, one way to get the NEP is through the rational reaction sets. For two players, let $R_i$ be the rational reaction set for player i. For example,

$$R_1=\{(\overline{x},y)\in K_1\times K_{2}suchthat{f}_1(\overline{x},y)\le f_1(x,y)\}$$

$$R_2=\{(x,\overline{y})\in K_1\times K_{2}suchthat{f}_2(x,\overline{y})\le f_2(x,y)\}$$

We can calculate the partial derivative:

$$\frac{\partial f_1(x,y)}{\partial x}=\left\{\begin{array}{cc}2x-3+y,\hfill & ifx>0\hfill \\ 2x-3-y,\hfill & ifx<0\hfill \end{array}\right.$$

$$\frac{\partial f_2(x,y)}{\partial y}=y-x$$

The NEP is obtained from the intersection of the two rational reaction sets:

$$R_1=\{x|\frac{\partial f_1(x,y)}{\partial x}=0\}$$

$$R_2=\{y|\frac{\partial f_2(x,y)}{\partial y}=0\}$$

Obviously, $K_1=K_2\subset M=\mathbb{R}$ are convex. Additionally, $f_1(·,y)$ is a convex function and threfore an invex function on $K_1$ for every $y\in K_2$ and $f_2(x,·)$ is invex on $K_2\subset \mathbb{R}$ for every $x\in K_1$. In our case, this solution is the point $(\overline{x},\overline{y})=(1,1)$, which is both an NEP and an NCP.

6. Conclusions

This paper has shown, for the first time, which types of functions are characterized by all VCPs being efficient or weakly efficient solutions of vector optimization problems with and without constraints on Hadamard manifolds. We have extended the results given by Gutiérrez et al. [25] and Ruiz-Garzón et al. [26] from linear spaces to nonlinear spaces and in the more general case of nonsmooth functions. We have introduced a great number of explanatory examples, and have presented an economics application showing that Nash’s critical and equilibrium points coincide in the case of invex payoff functions.

The results presented in this paper lead to the following conclusions:

• There is a need to extend the different concepts of invexity to Hadamard manifolds and clarify the relationships between them.
• It is important to use an adequate definition of VCPs or KKT-VCPs.
• There are applications of invexity in the search for equilibrium points, which are so desirable in economics.

In our opinion, in the future, we should search for algorithms or software that reflect the theoretical results achieved here, and identify further applications in the fields of physics and economics.