Towards Nonlinearity: The p-Regularity Theory

Abstract

We present recent advances in the analysis of nonlinear problems involving singular (degenerate) operators. The results are obtained within the framework of p-regularity theory, which has been successfully developed over the past four decades. We illustrate the theory with applications to degenerate problems in various areas of mathematics, including optimization and differential equations. In particular, we address the problem of describing the tangent cone to the solution set of nonlinear equations in singular cases. The structure of p-factor operators is used to propose optimality conditions and to construct novel numerical methods for solving degenerate nonlinear equations and optimization problems. The numerical methods presented in this paper represent the first approaches targeting solutions to degenerate problems such as the Van der Pol differential equation, boundary-value problems with small parameters, and partial differential equations where Poincaré’s method of small parameters fails. Additionally, these methods may be extended to nonlinear degenerate dynamical systems and other related problems.

1. Introduction

Many fundamental results in nonlinear analysis and classical numerical methods in Banach spaces X and Y rely on the regularity of the mapping $F:X\to Y$ at a given point $\overline{x}\in X$. The regularity of a Fréchet differentiable mapping F is commonly understood as the surjectivity of its Fréchet derivative $F^{\prime }$. However, a growing number of applications in areas such as partial differential equations, control theory, and optimization require the development of special approaches to deal with nonregular problems.

We present the theory of p-regularity, which originated in the 1980s with the aim of providing constructive tools for the analysis of nonregular problems. To date, the theory of p-regularity has found successful applications in various contexts and different areas of mathematics, as discussed in numerous papers. This paper highlights the most distinguished applications of the theory of p-regularity, with the goal of reviewing important results and indicating potential and promising directions for its future development and applications.

The theory of p-regularity, also known as higher-order regularity theory, offers a framework for studying nonlinear problems in situations where regularity assumptions are not satisfied. It focuses on utilizing higher-order derivatives to analyze and understand the behavior of mappings having first-order derivatives that are not onto or lack regularity.

Definition 1

(cf. Definition 1.16 of [1]). Let $F:X\to Y$ be a continuously differentiable mapping from an open set $U\subset X$ of a Banach space X into a Banach space Y. A vector $\overline{x}\in U$ is called a regular point of F if $F^{\prime }\left(\overline{x}\right)$ maps X onto the entire space Y, expressed as $Im{F}^{\prime }\left(\overline{x}\right)=Y.$ If $Im{F}^{\prime }\left(\overline{x}\right)\ne Y$, we refer to $\overline{x}$ as a singular (nonregular, irregular, degenerate) point of F.

1.1. Recollection of the Fundamental Results in the Regular Case

Regularity is a common assumption in many fundamental results of real and functional analysis, such as the inverse function theorem and the Implicit Function Theorem (IFT). In this section, we revisit some of these results. The theorems presented in this section have their roots in the following classical result.

Theorem 1 

(Banach open mapping principle [2], see also [3]). Let X and Y be Banach spaces. For any linear and bounded single-valued mapping $A:X\to Y$, the following properties are equivalent:

 (a) 

A is surjective.

 (b) 

A is open.

 (c) 

There is a constant $\kappa >0$, such that for all $y\in Y$ there exists $x\in X$ with $Ax=y$ and $\parallel x\parallel \le \kappa \parallel y\parallel$.

The Banach open mapping principle can be extended to nonlinear mappings in various ways. One such result is stated in the next theorem. Let $B_X(x,t)$ denote the open ball in X with center at x and radius t.

Theorem 2 

(Graves’ Theorem [4]). Let X and Y be Banach spaces, and let $F:X\to Y$ be a continuous function with $F\left(0\right)=0$. Let A be a linear operator from X onto Y, and let $\kappa >0$ be the corresponding constant from Theorem 1. Suppose that there exists a constant $\delta >0$ with $\delta <{\kappa }^{-1}$ and $\epsilon >0$, such that

$$|F\left(x_1\right)-F\left(x_2\right)-A(x_1-x_2)|\le \delta |x_1-x_2|$$

(1)

for all $x_1,x_2\in B_X(0,\epsilon )$. Then, the equation $y=F\left(x\right)$ has a solution $x\in B_X(0,\epsilon )$ whenever $\left|y\right|\le {\kappa }^{-1}-\delta$.

Note that the assumption of differentiability of F at 0 is not made, as the concept of a strictly differentiable function was introduced several years after the publication of Graves’ work [4]. Instead, the surjectivity of the operator A is used. The proof of Theorem 2, along with its reformulation in terms of a strictly differentiable function F and related discussions, can be found in [5]. For historical remarks, refer to [6].

Both the inverse function theorem and the Implicit Function Theorem can be deduced from Theorem 3 below; see Theorem 1.20 in Section 1.2 of [1] for details. We should also mention that certain variants of the inverse function theorem can be derived directly from the Implicit Function Theorem; see Section 4.2 for details.

To state Theorem 3, let us recall the definition of the Banach constant, denoted by $C\left(A\right)$, for a bounded linear operator A between Banach spaces X and Y, as given in [1]:

$$C\left(A\right)=sup\{r\ge 0\mid B_Y(0,r)\subset A\left(B_X(0,1)\right)\}=inf\{\parallel y\parallel \mid y\notin A\left(B_X(0,1)\right)\}.$$

(2)

Theorem 3 

(Lyusternik–Graves Theorem, [1]). Let X and Y be Banach spaces. Suppose that $F:X\to Y$ is strictly differentiable and regular at $\overline{x}\in X$. Then, for any positive $r<C\left(F^{\prime }\left(\overline{x}\right)\right)$, there exists an $\epsilon >0$ such that

$$B_Y(F\left(x\right),rt)\subset F\left(B_X(x,t)\right),$$

whenever $\parallel x-\overline{x}\parallel <\epsilon$ and $0\le t<\epsilon$.

For a thorough analysis of numerous consequences of Theorem 3, we refer the reader to the paper written by Dmitruk, Milyutin, and Osmolovskii [7], where the theorem is called the “generalized Lyusternik theorem”. In the monographs by Dontchev [3], Ioffe [1], and Dontchev and Rockafellar [8], Theorem 3 is also called the “Lyusternik-Graves theorem”. From a more general point of view, the theorem is treated by Dontchev and Frankowska in [9,10].

One of the consequences of Theorem 3 is the description of tangent vectors to the level set of a continuously differentiable mapping F at regular points (see Section 4.1 below).

1.2. Generalizations

Since the early 1970s, due to theoretical interests and an increasing number of involved economic and industrial applications, a vast body of literature has been devoted to relaxing the surjectivity assumption of the derivative in the fundamental results, some of which are given above, while maintaining as much of their conclusions as possible. It is beyond the scope of this paper to provide an exhaustive survey of the existing generalizations of the theorems stated above. For the purpose of this paper, we can distinguish between generalizations exploiting higher-order derivatives (see, e.g., Frankowska [11]) and generalizations that attempt to relax the surjectivity assumption of the derivative without referring to higher-order derivatives (see, e.g., Ekeland [12], Hamilton [13], Bednarczuk, Leśniewski, and Rutkowski [14]). The theory of p-regularity belongs to the first group of generalizations where higher-order derivatives are involved.

In this manuscript, we present the main concepts and results of the p-regularity theory, which has been developing successfully for the last forty years. One of the main goals of the theory of p-regularity is to replace the operator of the first derivative, which is not surjective, by a special mapping that is onto. Nonlinear mappings analyzed within the framework of p-regularity theory are those for which the derivatives up to the order $p-1$ are not surjective at a given point $\overline{x}$, where $p\ge 2$. The main concept of the p-regularity theory is the construction of the p-factor-operator, which is surjective at the point $\overline{x}$ (see Definition 4). The special definition and the property of surjectivity of the p-factor operator lead to generalizations of the fundamental results of analysis, including IFT and some classical numerical methods. The p-factor operator is defined in such a constructive way that it efficiently replaces the nonsurjective first derivative in a variety of situations. The structure of the p-factor-operator is used as a basis for analyzing nonregular problems and for constructing numerical methods for solving degenerate nonlinear equations and optimization problems. We discuss these generalizations in this paper.

There are many publications that focus on the case of $p=2$ and use a 2-factor operator in a variety of applications. In this work, we consider a more general case of $p\ge 2$ and do not make some additional assumptions introduced and required in the publications of other authors.

In the framework of metric spaces, the concept related to the problems discussed in the present paper and attempting to generalize the classical results given above is the concept of metric regularity; see, for example, [1,3]. For a function f acting between Banach spaces X and Y and being strictly differentiable at the given point $\overline{x}$, Corollary 5.3 in [3] complements Theorem 5.1 in [3]. It concludes that metric regularity at $\overline{x}$ for $f\left(\overline{x}\right)$ is equivalent to surjectivity of the Fréchet derivative $Df\left(\overline{x}\right)$ of f at $\overline{x}$. In the case when $X=Y={\mathbb{R}}^n$, this is the same as the nonsingularity of the Jacobian matrix $\nabla f\left(\overline{x}\right)$.

The theory of p-regularity and the apparatus of p-factor operators make it possible to create new methods in computational mathematics to solve nonlinear problems of mathematical physics, such as the Van der Pol differential equation, boundary-value problems with a small parameter, Partial Differential Equations (PDEs) where Poincaré’s method of small parameters fails, nonlinear degenerate dynamical systems, and others. This is associated with the proposed fundamentally new design (after Newton) of a numerical method for solving essentially (degenerate) problems, which is described in this paper. Moreover, the proposed approach will allow us to construct a new type of difference scheme in computational mathematics for solving problems of nonlinear mathematical physics that are stable and converge quickly to a solution.

This pertains to the numerical solution of nonlinear equations such as the nonlinear heat equation, Burgers equation, Korteweg–de Vries equation, Navier–Stokes equation, etc. It also enables us to reorganize Numerical Analysis in a novel way, with solutions obtained being related to the mentioned problems. All of this is applicable to the emerging prospects for developing new technologies and designs in Computational Mathematics for solving problems and models related to Artificial Intelligence, optimization problems, dynamical systems, optimal control problems, etc. New opportunities are emerging for modeling and researching neural networks and creating new architectures for supercomputing.

It is important to highlight the development of fundamentally new and innovative methods in computational mathematics. The resulting schemes were far from any previous designs, emerging from years of research into the structure of degeneracy—specifically, the structure of degenerate mappings and solution sets of degenerate systems. Analyzing these structures requires methods that differ significantly from those used in the analysis of linear problems, leading to entirely new forms of mathematical objects, such as the p-factor-operator: $F^{\left(p\right)}\left(\overline{x}\right){\left[h\right]}^{p-1},$ where $h\in X$. In general form, this operator is formally introduced in Definition 4.

In degenerate problems, where the first derivative operator is not surjective, the p-factor-operator serves as its replacement. At the same time, this research has uncovered a previously hidden nonlinear world with unexpectedly rich diversity (cf. Theorem 4). The range of possible new methods is remarkably broad, yet it follows a structured framework that remains stable under small changes—similar to regularization techniques. These methods can also be adjusted based on the specific problem being solved.

Moreover, recent studies revealed that the so-called ill-posed problems and essentially nonlinear ones are locally equivalent. This finding suggests that many important problems, such as inverse problems, can be solved using the p-factor method or p-factor regularization. This represents a new and promising direction in both theoretical mathematics and practical applications.

1.3. Aims and Scope

The main focus of this work is on analyzing and solving nonlinear equations of the form

$$F\left(x\right)=0,$$

(3)

and optimization problems of the form

$$minf\left(x\right)\mathrm{subject}\mathrm{to}F\left(x\right)=0,$$

(4)

where $f:X\to \mathbb{R}$ and $F:X\to Y$ are sufficiently smooth mappings, and X and Y are Banach spaces. Many interesting applied nonlinear problems can be written in one of these forms.

Nonlinear mappings F and problems of the form (3) and (4) can be divided into two classes, called regular (or nonsingular) and singular (or degenerate). The classification depends on the mapping F, which is either regular (that is, $F^{\prime }\left(\overline{x}\right):X\to Y$ is onto for a given $\overline{x}\in X$) or singular (that is, if $F^{\prime }(\overline{x}$) is not onto). Roughly speaking, regular mappings are those for which the Implicit Function Theorem arguments can be applied, and singular problems are those for which they cannot, at least, be directly applied.

The purpose of this paper is to give an overview of methods and tools of the p-regularity theory, and to show how they can be applied to analyze and develop methods for solving singular (irregular, degenerate) nonlinear equations and equality-constrained optimization problems. The development of the theory of p-regularity started in approximately 1983–1984, with the concept of p-regularity introduced by Tret’yakov in [15,16].

One of the main results of the theory of p-regularity is a detailed description of the structure of the zero set $\{x\in X,F(x)=0\}$ of a nonregular nonlinear mapping $F:X\to Y$. It is interesting to note that there have been several examples in the history of mathematics when fundamental results were obtained independently in the same general time period. One such example related to the theory of p-regularity concerns theorems about the structure of the zero sets of an irregular mapping satisfying a special higher-order regularity condition. The result that we are referring to was simultaneously obtained by Buchner, Marsden and Schecter [17] and Tretyakov [16]. The approaches proposed in [17] and in [16] are the same. The difference is in motivation and the context for the main result in both papers. In [17], the structure of the zero set around a point where the derivative is not surjective was studied in the context of bifurcation theory. Theorem 1.3 in [17] is referred to as a blowing-up result. In Fink and Rheinboldt [18], it was noted that Theorem 1.3 in [17] was a powerful generalization of the Morse Lemma, and some interesting counterexamples for a naive approach to the Morse Lemma were found. The same theorem derived by Tretyakov [16] is one of the main results for the p-regularity theory. The result led to various theoretical developments and applications of the theory to nonregular (or degenerate) problems in many areas of mathematics. We should note that the results and constructions introduced by Marsden and Tret’yakov are the same in the completely degenerate case.

This paper is organized as follows. We discuss essential nonlinearity and singular mappings in Section 2. We then recall the main concepts and definitions of the p-regularity theory in Section 3. We discuss some classical results of analysis and methods for solving nonlinear problems via the p-regularity theory in Section 4. In each subsection, we focus on singular problems that illustrate that the classical results are not necessarily satisfied in the nonregular case. We present generalizations of the same classical results, which were derived during the last forty years using the constructions and definitions of the p-regularity theory.

In this manuscript, we consider a variety of applications. We start Section 4 with the Lyusternik theorem in Section 4.1. The Lyusternik theorem plays an important role in the description of the solution sets of nonlinear equations and feasible sets of optimization problems in the regular case. However, the classical Lyusternik theorem might not hold if mapping F is singular at a given point $\overline{x}$. The first generalization of the classical Lyusternik theorem for p-regular mappings was derived and proved simultaneously in [15,17]. It can be applied to describe the zero set of a p-regular mapping. Representation Theorem and Morse Lemma are also presented in Section 4.1. We continue with the consideration of the Implicit Function Theorem in Section 4.2. Numerous books and papers, such as [13,19], discuss the classical Implicit Function Theorem. However, the classical version of the theorem is not applicable when a mapping $F:X\times Y\to Z$ is not regular, meaning that $F_y^{\prime }(\overline{x},\overline{y}):Y\to Z$ is not onto for some $(\overline{x},\overline{y})$, where the index y denotes the partial derivative with respect to the variable y (for a more detailed explanation of the notation, see the “General Notation” below). We present a generalization of the Implicit Function Theorem for nonregular mappings in Section 4.2. In Section 4.3, we cover the p-factor Newton’s method for solving nonlinear Equation (3) and finding critical points of an unconstrained optimization problem. Optimality conditions for equality-constrained optimization problems and Lagrange multiplier theorems for the regular and degenerate cases are considered in Section 4.4. The modified Lagrange function method for 2-regular problems is covered in Section 4.5. Singular problems of the calculus of variations and optimality conditions for p-regular problems of the calculus of variations are considered in Section 4.6. The existence of solutions to nonlinear equations in regular and degenerate cases is covered in Section 4.7. The second-order nonlinear ordinary differential equations with boundary conditions are presented in Section 4.8. Newton interpolation polynomials and the p-factor interpolation method are considered in Section 4.9. We make some concluding remarks in Section 5.

General Notation

Let $\mathcal{L}(X,Y)$ be the space of all continuous linear operators from X to Y, and for a given linear operator $\Lambda \in \mathcal{L}(X,Y)$, let us denote its kernel and image by $Ker\Lambda =\{x\in X\mid \Lambda x=0\}$ and $Im\Lambda =\{y\in Y\mid y=\Lambda xforsomex\in X\}$, respectively. Also, ${\Lambda }^{*}:Y^{*}\to X^{*}$ denotes the adjoint of $\Lambda$, where $X^{*}$ and $Y^{*}$ denote the dual spaces of X and Y, respectively.

Let p be a natural number and let $\mathcal{A}:X\times X\times \dots \times X(\mathrm{with} p \mathrm{copies} \mathrm{of} X)\to Y$ be a continuous symmetric p-multilinear mapping. The p-form associated with $\mathcal{A}$ is the map $\mathcal{A}{\left[x\right]}^p:X\to Y$ defined by

$$\mathcal{A}{\left[x\right]}^p=\mathcal{A}(x,x,\dots ,x),$$

for $x\in X$, where all instances of x in the expression $\mathcal{A}(x,x,\dots ,x)$ are the same. Alternatively, we may simply view $\mathcal{A}{[·]}^p$ as a homogeneous polynomial $Q:X\to Y$ of degree p with $Q\left(\alpha x\right)={\alpha }^{p}Q\left(x\right)$. Therefore, the space of continuous homogeneous polynomials $Q:X\to Y$ of degree p is denoted by ${\mathcal{Q}}^p(X,Y)$.

If $F:X\to Y$ is of class $C^2$, then its derivative $F^{\prime }$ at a point $\overline{x}\in X$ is a continuous linear operator $F^{\prime }\left(\overline{x}\right)\in \mathcal{L}(X,Y)$. The second derivative $F^{\prime \prime }\left(\overline{x}\right)$ is a bilinear operator on $X\times X$ and can be viewed as a mapping from X to $\mathcal{L}(X,Y)$. See ([20], Chapter VIII) for further details.

If $F:X\to Y$ is of class $C^p$, we denote by $F^{\left(p\right)}\left(\overline{x}\right):X^p\to Y$ the pth-order derivative of F at a given point $\overline{x}$. This is a symmetric p-multilinear map from $X^p$ to Y. The associated p-form, also called the pth–order mapping, is defined as

$$F^{\left(p\right)}\left(\overline{x}\right){\left[h\right]}^p=F^{\left(p\right)}\left(\overline{x}\right)(h,h,\dots ,h).$$

In particular, for $p=2$, we have

$$F^{\prime \prime }\left(\overline{x}\right){\left[h\right]}^2=F^{\prime \prime }\left(\overline{x}\right)(h,h).$$

Furthermore, for a given p-multilinear map, we introduce the following key notation for the p-kernel of the pth-order mapping:

$${Ker}^p{F}^{\left(p\right)}\left(\overline{x}\right)=\{h\in X|F^{\left(p\right)}\left(\overline{x}\right){\left[h\right]}^p=0\}.$$

Here, h represents elements of X that are repeatedly applied in the multilinear mapping. This set is also referred to as the locus of $F^{\left(p\right)}\left(\overline{x}\right)$.

When $F:X\times Y\to Z$ is a continuously differentiable mapping, we use the notation $F_y^{\prime }(\overline{x},\overline{y})$ to denote the partial derivative of F with respect to y. Specifically, for $(\overline{x},\overline{y})\in X\times Y$, the operator $F_y^{\prime }(\overline{x},\overline{y})$ is defined as the Fréchet derivative of F with respect to y, satisfying:

$$\underset{h\to 0}{lim}{\displaystyle \frac{F(\overline{x},\overline{y}+h)-F(\overline{x},\overline{y})-F_y^{\prime }(\overline{x},\overline{y})h}{\parallel h\parallel }}=0,$$

where $h\in Y$ and $F_y^{\prime }(\overline{x},\overline{y}):Y\to Z$ is a continuous linear operator.

2. Essential Nonlinearity and Singular Mappings

Let $X,Y$ be Banach spaces. Let $F:X\to Y$ be a mapping in ${\mathcal{C}}^1\left(W\right)$, where W is a neighborhood of a point $\overline{x}\in X$. According to Definition 1, a mapping F is called regular at $\overline{x}$, if

$$Im{F}^{\prime }\left(\overline{x}\right)=Y.$$

(5)

The following lemma on the local representation of a regular mapping holds.

Lemma 1 

(Lemma 1, Section 1.3.3 of [21]). Let X and Y be Banach spaces, let W be a neighborhood of a point $\overline{x}\in X$, and let $F:X\to Y$ be of class ${\mathcal{C}}^1\left(W\right)$. If F is regular at $\overline{x}$, then there exist a neighborhood U of 0, a neighborhood $V\subset W$, and a diffeomorphism $\phi :U\to V$, such that:

 1. 

$\phi \left(0\right)=\overline{x}$,

 2. 

$F\left(\phi \left(x\right)\right)=F\left(\overline{x}\right)+F^{\prime }\left(\overline{x}\right)x$ for all $x\in U$,

 3. 

${\phi }^{\prime }\left(0\right)=I_X$ (the identity mapping on X).

Lemma 1 states that the diffeomorphism $\phi$ transforms F locally into an affine mapping:

$$F\left(\phi \left(x\right)\right)=F\left(\overline{x}\right)+F^{\prime }\left(\overline{x}\right)x\mathrm{for}\mathrm{all}x\in U.$$

(6)

In other words, $\phi$ provides a local reparametrization under which F takes an affine form in U. This result is also known as the local “trivialization theorem” (Theorem 1.26 of [1]).

If the regularity condition (5) is not satisfied, then, in general, F cannot be locally linearized because such a diffeomorphism $\phi$ does not exist.

There exist many mappings that do not admit local linearization. The concept of essentially nonlinear mappings, introduced in [22], provides a formal framework for describing such cases.

Definition 2. 

Let V be a neighborhood of a point $\overline{x}$ in X, and let $U\subset X$ be a neighborhood of 0. A mapping $F:V\to Y$, where $F\in {\mathcal{C}}^2\left(V\right)$, is said to be essentially nonlinear at $\overline{x}$ if there exists a perturbation of the form

$$\tilde{F}(\overline{x}+x)=F(\overline{x}+x)+\omega \left(x\right),where\parallel \omega \left(x\right)\parallel =o(\parallel x\parallel ),$$

such that there does not exist any nondegenerate transformation $\phi :U\to V$, $\phi \in {\mathcal{C}}^1\left(U\right)$, satisfying $\phi \left(0\right)=\overline{x}$, ${\phi }^{\prime }\left(0\right)=I_X$, and such that Equation (6) holds with φ and $\tilde{F}$.

We say that $\parallel \omega \left(x\right)\parallel =o(\parallel x\parallel )$ as $x\to 0$ if $\underset{x\to 0}{lim}{\displaystyle \frac{\omega \left(x\right)}{\parallel x\parallel }}=0.$ For example, if $\omega :X\to \mathbb{R}$, then $\omega \left(x\right)={\parallel x\parallel }^2$ is $o(\parallel x\parallel )$.

Definition 3. 

We say that the mapping $F:X\to Y$ is singular (or degenerate) at $\overline{x}\in X$ if it fails to be regular; that is, if its derivative is not onto:

$$Im{F}^{\prime }\left(\overline{x}\right)\ne Y.$$

(7)

The following Theorem 4, which establishes the relationship between the two notions of essential nonlinearity and singularity, was derived as Theorem 2.3 in [22]. We provide its proof here to complete our development.

Theorem 4. 

Suppose V is a neighborhood of a given point $\overline{x}\in X,$ and $X,Y$ are Banach spaces. Suppose $F:V\to Y$ is in $C^2$. If $F\left(\overline{x}\right)=0$, then F is essentially nonlinear at the point $\overline{x}$ if and only if F is singular at $\overline{x}$.

Proof. 

Suppose F is singular at the point $\overline{x}$ and $F\left(\overline{x}\right)=0$. Since $Im{F}^{\prime }\left(\overline{x}\right)\ne Y$, there exists a nonzero element $\xi \in Y$, such that

$$\xi \notin Im{F}^{\prime }\left(\overline{x}\right).$$

(8)

Thus, $\xi \in Y\setminus Im{F}^{\prime }\left(\overline{x}\right)$. Since $F^{\prime }\left(\overline{x}\right)$ is linear, we can assume $\parallel \xi \parallel =1$.

Assume on the contrary that F is not essentially nonlinear at $\overline{x}$. Define the mapping $\tilde{F}:X\to Y$ by

$$\tilde{F}(\overline{x}+x)=F\left(\overline{x}\right)+F^{\prime }\left(\overline{x}\right)x+\xi {\parallel x\parallel }^2,$$

(9)

where ${\xi \parallel x\parallel }^2\notin Im{F}^{\prime }\left(\overline{x}\right).$

By Definition 2, with $\tilde{F}$ defined in (9), there exist a neighborhood $U\subset X$ of 0 and a nondegenerate transformation $\phi :U\to V$, $\phi \in {\mathcal{C}}^1\left(U\right)$, such that $\phi \left(0\right)=\overline{x}$, ${\phi }^{\prime }\left(0\right)=I_X$, and (6) holds with $\phi$ and $\tilde{F}$:

$$\tilde{F}\left(\phi \left(x\right)\right)=\tilde{F}\left(\overline{x}\right)+{\tilde{F}}^{\prime }\left(\overline{x}\right)x=F\left(\overline{x}\right)+F^{\prime }\left(\overline{x}\right)x$$

(10)

for all $x\in U$.

Since $F\left(\overline{x}\right)=0$ and $F^{\prime }\left(\overline{x}\right)x\in Im{F}^{\prime }\left(\overline{x}\right)$, it follows from (10) that

$$\tilde{F}\left(\phi \left(x\right)\right)\in Im{F}^{\prime }\left(\overline{x}\right).$$

(11)

However, using $F\left(\overline{x}\right)=0$, $\phi \left(0\right)=\overline{x}$, and ${\phi }^{\prime }\left(0\right)=I_X$, we obtain

$$\begin{array}{cc}\tilde{F}\left(\phi \left(x\right)\right)\hfill & =F(\overline{x}+(\phi \left(x\right)-\overline{x}))\hfill \\ \hfill & =F\left(\overline{x}\right)+F^{\prime }\left(\overline{x}\right)(\phi \left(x\right)-\overline{x})+\xi {\parallel \phi \left(x\right)-\overline{x}\parallel }^2\hfill \\ \hfill & =F^{\prime }\left(\overline{x}\right)(\phi \left(x\right)-\overline{x})+\xi {\parallel \phi \left(0\right)+{\phi }^{\prime }\left(0\right)x+{\omega }_1\left(x\right)-\overline{x}\parallel }^2\hfill \\ \hfill & =F^{\prime }\left(\overline{x}\right)(\phi \left(x\right)-\overline{x})+\xi {\parallel x+{\omega }_1\left(x\right)\parallel }^2,\hfill \end{array}$$

(12)

where $\parallel {\omega }_1\left(x\right)\parallel =o(\parallel x\parallel )$. Thus, for small x,

$$\xi \parallel x+{\omega }_1{\left(x\right)\parallel }^2\ne 0.$$

Taking into account that ${\xi \parallel x\parallel }^2\notin Im{F}^{\prime }\left(\overline{x}\right)$ for any $x\in V$, along with Equation (12) and the fact that $F^{\prime }\left(\overline{x}\right)(\phi \left(x\right)-\overline{x})\in Im{F}^{\prime }\left(\overline{x}\right)$, we conclude that

$$\tilde{F}\left(\phi \left(x\right)\right)\notin Im{F}^{\prime }\left(\overline{x}\right).$$

(13)

This contradicts (11), and therefore F is essentially nonlinear at $\overline{x}$.

To prove the converse, suppose that F is essentially nonlinear at $\overline{x}$ but not singular; that is, suppose F is regular at this point.

By the persistence of the regularity condition, for any perturbation

$$\tilde{F}(\overline{x}+x)=F(\overline{x}+x)+\omega \left(x\right),$$

where $\parallel \omega \left(x\right)\parallel =o(\parallel x\parallel )$, the map $\tilde{F}(\overline{x}+x)$ remains regular at $\overline{x}$, and $F^{\prime }\left(\overline{x}\right)={\tilde{F}}^{\prime }\left(\overline{x}\right)$. Hence, by Lemma 1, $\tilde{F}(\overline{x}+x)$ can be written as

$$\tilde{F}\left(\phi \left(x\right)\right)=\tilde{F}\left(\overline{x}\right)+{\tilde{F}}^{\prime }\left(\overline{x}\right)x,$$

(14)

where $\phi \left(0\right)=\overline{x}$ and ${\phi }^{\prime }\left(0\right)=I_X$. This contradicts the definition of the essential nonlinearity of the mapping F. □

Under additional splitting assumptions, which are not made here, the representation (14) would be a standard consequence of the IFT, as in, for example, ([23], §2.5).

3. Elements of $\mathbf{p}$-Regularity Theory

For the purpose of describing essentially nonlinear problems, a concept of p-regularity was introduced by Tret’yakov [15,16,24] using the notion of a p-factor operator. In this section, we introduce the main definitions of the p-regularity theory, as presented, for example, in [21,22,24].

Let X and Y be Banach spaces. Suppose that $F:X\to Y$ is a $C^p$-class mapping that is singular (nonregular) at a given point $\overline{x}\in X$. We construct the p-factor operator under the assumption that the space Y can be decomposed into the (topological) direct sum

$$Y=Y_1\oplus \dots \oplus Y_p,$$

(15)

where $Y_1=\mathrm{cl}(Im{F}^{\prime }\left(\overline{x}\right)),$ is the closure of the image of the first derivative of F evaluated at $\overline{x}$. To define the remaining spaces, let $S_1=Y$ and let $S_2\subset Y$ be a closed complementary subspace to $Y_1$, that is, $Y=Y_1\oplus S_2$ if ${\mathcal{S}}_2$ exists. Next, let $P_{S_2}:Y\to S_2$ be the projection operator onto $S_2$ along $Y_1$. Define $Y_2$ as the closed linear span of the projection of the quadratic map image:

$$Y_2=\mathrm{cl}(\mathrm{span}Im{P}_{S_2}{F}^{\left(2\right)}\left(\overline{x}\right){\left[h\right]}^2).$$

More generally, define $Y_i$ inductively as follows:

$$Y_i=\mathrm{cl}(\mathrm{span}Im{P}_{S_i}{F}^{\left(i\right)}\left(\overline{x}\right){\left[h\right]}^i)\subseteq S_i,i=2,\dots ,p-1,p>2,$$

where $S_i$ is a choice of a closed complementary subspace for $Y_1\oplus \dots \oplus Y_{i-1}$ with respect to Y, and $P_{S_i}:Y\to S_i$ is the projection operator onto $S_i$ along $Y_1\oplus \dots \oplus Y_{i-1}$ for $i=2,\dots ,p$. Finally, let $Y_p=S_p.$ The order p is the minimum number for which (15) holds. In particular, for $p=2$, we have $Y=S_1=Y_1\oplus S_2$. When Y is a Hilbert space, there exists a complementary subspace to $Y_1$, namely the orthogonal subspace $Y_2=Y_1^{\perp }$.

Remark 1. 

The subspaces $Y_i$ in assumption (15) can be replaced, in further considerations, by subspaces constructed using the so-called factorization procedure. Specifically, we define

$$Y_1=\mathrm{cl}(Im{F}^{\prime }\left(\overline{x}\right)),$$

as before. However, instead of $Y_2$, we use the space $Y/Y_1$, called the quotient (or factor) space. Note that the quotient space is itself a Banach space (see, e.g., [25]). Moreover, if decomposition (15) holds, then $Y_2$ is isomorphic to $Y/Y_1$. For simplicity of presentation, we continue to use assumption (15).

Define the following mappings (see Tret’yakov [24]):

$$f_i:X\to Y_i,f_i\left(x\right)=P_{Y_i}F\left(x\right),i=1,\dots ,p,$$

(16)

where $P_{Y_i}:Y\to Y_i$ is the projection operator onto $Y_i$ along $Y_1\oplus \dots \oplus Y_{i-1}\oplus Y_{i+1}\oplus \dots \oplus Y_p$ with respect to Y for $i=1,\dots ,p$. Recall that $P_{Y_i}$ is the projection onto $Y_i$ along (or parallel to) $W_i=Y_1\oplus \dots \oplus Y_{i-1}\oplus Y_{i+1}\oplus \dots \oplus Y_p$ if $Ker{P}_{Y_i}=W_i$.

In our notation, $f_i^{\left(k\right)}\left(\overline{x}\right)$ denotes the k-th derivative of $f_i$ at $\overline{x}$. By the construction of the subspaces $Y_i$, we have

$$f_i^{\left(k\right)}\left(\overline{x}\right)=P_{Y_i}{F}^{\left(k\right)}\left(\overline{x}\right)=0,i=1,\dots ,p,k=1,\dots ,i-1.$$

(17)

We define a mapping F as completely degenerate up to order p if

$$F^{\left(k\right)}\left(\overline{x}\right)=0fork=1,\dots ,p-1.$$

(18)

Remark 2. 

If the mapping F is completely degenerate up to order p, then (17) implies that each mapping $f_i$, defined in (16), is also completely degenerate at $\overline{x}$ up to order $i-1$ for $i=1,\dots ,p$. That is,

$$f_i^{\left(k\right)}\left(\overline{x}\right)=0,k=1,\dots ,i-1,i=1,\dots ,p.$$

With all the notation established above, we are now ready to define the p-factor operator.

Definition 4. 

For a fixed vector $h\in X,$$h\ne 0$, and mappings $f_i$, defined in (16), the linear operator ${\Psi }_p\left(h\right)\in \mathcal{L}(X,Y_1\oplus \dots \oplus Y_p)$,

$${\Psi }_p\left(h\right)x=f_1^{\prime }\left(\overline{x}\right)x+f_2^{\prime \prime }\left(\overline{x}\right)\left[h\right]x+\dots +f_p^{\left(p\right)}\left(\overline{x}\right){\left[h\right]}^{p-1}x,x\in X,$$

(19)

is called the p-factor operator. Alternatively, the following equivalent form can be used:

$${\Psi }_p\left(h\right)x=f_1^{\prime }\left(\overline{x}\right)x+{\displaystyle \frac{1}{2!}}{f}_2^{\prime \prime }\left(\overline{x}\right)\left[h\right]x+\dots +{\displaystyle \frac{1}{p!}}{f}_p^{\left(p\right)}\left(\overline{x}\right){\left[h\right]}^{p-1}x.$$

Note that when F is regular at $\overline{x}$, meaning $\mathrm{Im}F\left(\overline{x}\right)=Y$, we have $Y_1=Y$. In this case, the p-factor operator reduces to the operator of the first derivative: ${\Psi }_1\left(h\right)x=F^{\prime }\left(\overline{x}\right)x$ for any $x\in X$.

For $p=2$, the p-factor-operator (19) takes the form

$${\Psi }_2\left(h\right)x=f_1^{\prime }\left(\overline{x}\right)x+f_2^{\prime \prime }\left(\overline{x}\right)\left[h\right]x,x\in X,$$

(20)

or, equivalently, ${\Psi }_2\left(h\right)x=f_1^{\prime }\left(\overline{x}\right)x+{\displaystyle \frac{1}{2}}{f}_2^{\prime \prime }\left(\overline{x}\right)\left[h\right]x$ for $x\in X,$ where $h\in X$ and $h\ne 0$. In view of (17), the construction of the operator ${\Psi }_2\left(h\right)$ (and ${\Psi }_p\left(h\right)$ in general) is closely tied to the decomposition of the image space (15). The idea is to use higher-order derivatives of F up to order p to obtain (15).

In particular, for $p=2$ and ${\Psi }_2\left(h\right)$ given by (20), we seek those $h\in X$ that ensure the equality $Im{f}_2^{\prime \prime }\left(\overline{x}\right)\left[h\right]\left(X\right)=S_2$, where $S_2$ is the complementary space to $Y_1$.

If a mapping F is completely degenerate up to order p, meaning that (18) holds, and $Im{F}^{\left(p\right)}\left(\overline{x}\right){\left[h\right]}^{p-1}=Y$, then the p-factor operator simplifies to ${\Psi }_p\left(h\right)x=F^{\left(p\right)}\left(\overline{x}\right){\left[h\right]}^{p-1}x$.

Recall that a bounded linear operator $T:X\to Y$ between Banach spaces X and Y is called Fredholm if the kernel of T has finite dimension and the image of T is a closed subspace of finite codimension in Y (see, for example ([26], Chapter 4) and [27]).

Hence, in the case of a Fredholm operator $F^{\prime }\left(\overline{x}\right)$, the subspace $Y_1=Im{F}^{\prime }\left(\overline{x}\right)$ has a complementary finite-dimensional subspace $Z_2$ such that $Y=Y_1\oplus Z_2$.

With the p-factor operator ${\Psi }_p\left(h\right)$ defined in (19), we are now ready to state a few definitions of various types of p-regularity for a $C^p$-class mapping $F:X\to Y$.

Definition 5. 

We say that the mapping $F:X\to Y$ is p-regular at a given point $\overline{x}$ along an element $h\in X$ if

$$\mathrm{Im}{\Psi }_p\left(h\right)=Y.$$

Remark 3. 

The condition of p-regularity of the mapping F at the point $\overline{x}$ along $h\in X$ is equivalent to the following condition:

$$Im{f}_p^{\left(p\right)}\left(\overline{x}\right){\left[h\right]}^{p-1}\left(Ker{\Psi }_{p-1}\left(h\right)\right)=Y_p,$$

(21)

where ${\Psi }_{p-1}\left(h\right)=f_1^{\prime }\left(\overline{x}\right)+f_2^{\prime \prime }\left(\overline{x}\right)\left[h\right]+\dots +f_{p-1}^{(p-1)}\left(\overline{x}\right){\left[h\right]}^{p-2}.$ In particular, when $p=2$, we have ${\Psi }_1\left(h\right)=f_1^{\prime }\left(\overline{x}\right)$, and condition (21) reduces to $Im{f}_2^{\prime \prime }\left(\overline{x}\right)\left[h\right]\left(Ker{f}_1^{\prime }\left(\overline{x}\right)\right)=Y_2$, which follows from elementary algebra.

We also define the k-kernel of the kth-order mapping $f_k^{\left(k\right)}\left(\overline{x}\right)$ as follows:

$${Ker}^k{f}_k^{\left(k\right)}\left(\overline{x}\right)=\{h\in X\mid f_k^{\left(k\right)}\left(\overline{x}\right){\left[h\right]}^k=0\}.$$

(22)

Definition 6. 

We say the mapping F is p-regular at $\overline{x}$ if it is p-regular along any h from the set

$$H_p\left(\overline{x}\right)=\left\{h\in X\mid h\in \bigcap _{i=1}^p{Ker}^i{f}_i^{\left(i\right)}\left(\overline{x}\right)\right\}\setminus \left\{0\right\},$$

(23)

where the i-kernel of the ith-order mapping $f_i^{\left(i\right)}\left(\overline{x}\right)$ is defined in (22).

For a linear surjective operator ${\Psi }_p\left(h\right):X\to Y$ between Banach spaces, we denote its right inverse by ${\left\{{\Psi }_p\left(h\right)\right\}}^{-1}$ (see [28]). Therefore, ${\left\{{\Psi }_p\left(h\right)\right\}}^{-1}:Y\to 2^X$ and we have

$${\left\{{\Psi }_p\left(h\right)\right\}}^{-1}\left(y\right)=\left\{x\in X\mid {\Psi }_p\left(h\right)x=y\right\}.$$

(24)

We define the norm of ${\left\{{\Psi }_p\left(h\right)\right\}}^{-1}$ by

$$\parallel {\left\{{\Psi }_p\left(h\right)\right\}}^{-1}\parallel =\underset{\parallel y\parallel =1}{sup}inf\{\parallel x\parallel \mid x\in {\left\{{\Psi }_p\left(h\right)\right\}}^{-1}\left(y\right)\}.$$

(25)

We say that ${\left\{{\Psi }_p\left(h\right)\right\}}^{-1}$ is bounded if $\parallel {\left\{{\Psi }_p\left(h\right)\right\}}^{-1}\parallel <\infty .$

Definition 7. 

A mapping $F\in {\mathcal{C}}^p$ is called strongly p-regular at a point $\overline{x}$ if there exists $\alpha >0$ such that

$$\underset{h\in H_p^{\alpha }\left(\overline{x}\right)}{sup}∥{\left\{{\Psi }_p\left(h\right)\right\}}^{-1}∥<\infty ,$$

where ${\left\{{\Psi }_p\left(h\right)\right\}}^{-1}$ is the right inverse operator of ${\Psi }_p\left(h\right)$ and

$$H_p^{\alpha }\left(\overline{x}\right)=\left\{h\in X|\parallel f_i^{\left(i\right)}\left(\overline{x}\right){\left[h\right]}^i\parallel \le \alpha \mathrm{for}\mathrm{all}i=1,\dots ,p,\parallel h\parallel =1\right\}.$$

The following examples illustrate the construction of the p-factor operator for the cases $p=2$ and $p=3$.

Example 1. 

Consider the mapping $F:{\mathbb{R}}^2\to {\mathbb{R}}^2$ defined by

$$F\left(x\right)=\left(\begin{array}{c}{x}_1+x_2\\ x_1{x}_2\end{array}\right).$$

Let $\overline{x}={(0,0)}^T$. Then, the Jacobian $F^{\prime }\left(\overline{x}\right)=\left(\begin{array}{cc}1& 1\\ 0& 0\end{array}\right)$ is singular (degenerate) at $\overline{x}.$ Hence, $Im{F}^{\prime }\left(\overline{x}\right)=span\left\{(1,0)\right\}\ne {\mathbb{R}}^2$. Let $Y_1=span\left\{(1,0)\right\}$ and $Y_2=span\left\{(0,1)\right\}.$ To construct the 2-factor operator, we use the projection matrices

$$P_{Y_1}=\left(\begin{array}{cc}1& 0\\ 0& 0\end{array}\right)and{P}_{Y_2}=\left(\begin{array}{cc}0& 0\\ 0& 1\end{array}\right).$$

According to Equation (16), the mappings $f_1:{\mathbb{R}}^2\to Y_1$ and $f_2:{\mathbb{R}}^2\to Y_2$ have the form

$$f_1\left(x\right)=\left(\begin{array}{c}{x}_1+x_2\\ 0\end{array}\right)and{f}_2\left(x\right)=\left(\begin{array}{c}0\\ x_1{x}_2\end{array}\right).$$

Then

$$f_1^{\prime }\left(x\right)=\left(\begin{array}{cc}1& 1\\ 0& 0\end{array}\right),f_2^{\prime }\left(x\right)=\left(\begin{array}{cc}0& 0\\ x_2& x_1\end{array}\right)$$

and

$$f_2^{\prime \prime }\left(x\right)h=\left(\begin{array}{cc}0& 0\\ h_2& h_1\end{array}\right).$$

Hence, for $h={(h_1,h_2)}^T\in {\mathbb{R}}^2$, the 2-factor operator is defined by

$${\Psi }_2\left(h\right)=f_1^{\prime }\left(\overline{x}\right)+f_2^{\prime \prime }\left(\overline{x}\right)h=\left(\begin{array}{cc}1& 1\\ h_2& h_1\end{array}\right).$$

It can be verified that the 2-factor operator is surjective whenever $h_1\ne h_2$.

In this example, we have

$${Ker}^1{f}_1^{\prime }\left(\overline{x}\right)=span\left\{\left(1,-1\right)\right\}and{Ker}^2{f}_2^{\prime \prime }\left(\overline{x}\right)=span\left\{\left(1,0\right)\right\}\cup span\left\{\left(0,1\right)\right\}.$$

This result implies that $H_2\left(\overline{x}\right)=\varnothing$. Hence, according to Definition 5, the mapping F is 2-regular at $\overline{x}$ along any $h\in X$ with $h_1\ne h_2$, but it is not 2-regular at $\overline{x}$. As we observe, it may happen that F is 2-regular along some $h\in X$ but $H_2\left(\overline{x}\right)=\varnothing$. Therefore, a given mapping F may fail to be 2-regular with respect to all $h\in X$, $h\ne 0$.

Example 2. 

Case $p=3$. Consider mapping $F:{\mathbb{R}}^2\to {\mathbb{R}}^3$ defined by

$$F\left(x\right)=\left(\begin{array}{c}{x}_1+x_2\\ x_1{x}_2^2\\ x_1^3\end{array}\right).$$

With $\overline{x}={(0,0)}^T=0$, we obtain

$$F^{\prime }\left(x\right)=\left(\begin{array}{cc}1& 1\\ x_2^2& 2x_1{x}_2\\ 3x_1^2& 0\end{array}\right)and{F}^{\prime }\left(0\right)=\left(\begin{array}{cc}1& 1\\ 0& 0\\ 0& 0\end{array}\right).$$

Then, with $h={(h_1,h_2)}^T$,

$$F^{\prime }\left(x\right)h=\left(\begin{array}{c}{h}_1+h_2\\ x_2^2{h}_1+2x_1{x}_2{h}_2\\ 3x_1^2{h}_1\end{array}\right),$$

$$F^{\prime \prime }\left(x\right)\left[h\right]h=F^{\prime \prime }\left(x\right){\left[h\right]}^2=\left(\begin{array}{c}0\\ 4x_2{h}_1{h}_2+2x_1{h}_2^2\\ 6x_1{h}_1^2\end{array}\right),F^{\prime \prime \prime }\left(x\right){\left[h\right]}^2=\left(\begin{array}{cc}0& 0\\ 2h_2^2& 4h_1{h}_2\\ 6h_1^2& 0\end{array}\right).$$

In this example,

$$Y_1=\mathrm{Im}{F}^{\prime }\left(0\right)=\mathrm{span}\left\{(1,0,0)\right\},Y_2=(0,0,0),Y_3=\mathrm{span}\{(0,1,0),(0,0,1)\}.$$

To construct the 3-factor operator, we use the projection matrices

$$P_{Y_1}=\left(\begin{array}{ccc}1& 0& 0\\ 0& 0& 0\\ 0& 0& 0\end{array}\right)and{P}_{Y_3}=\left(\begin{array}{ccc}0& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right).$$

Then, using Equation (16), we define $f_1$ and $f_3$ as follows:

$$f_1\left(x\right)=P_{Y_1}F\left(x\right)=\left(\begin{array}{c}{x}_1+x_2\\ 0\\ 0\end{array}\right)and{f}_3\left(x\right)=P_{Y_3}F\left(x\right)=\left(\begin{array}{c}0\\ x_1{x}_2^2\\ x_1^3\end{array}\right).$$

By the definition of the 3-factor-operator, we obtain

$$\begin{array}{cc}\hfill {\Psi }_3\left(h\right)& =f_1^{\prime }\left(\overline{x}\right)+f_3^{\prime \prime \prime }\left(\overline{x}\right){\left[h\right]}^2,\hfill \\ \hfill & =\left(\begin{array}{cc}1& 1\\ 0& 0\\ 0& 0\end{array}\right)+\left(\begin{array}{cc}0& 0\\ 2h_2^2& 4h_1{h}_2\\ 6h_1^2& 0\end{array}\right)=\left(\begin{array}{cc}1& 1\\ 2h_2^2& 4h_1{h}_2\\ 6h_1^2& 0.\end{array}\right).\hfill \end{array}$$

For $h=(h_1,0)$, the 3-factor operator takes the form

$${\Psi }_3\left(h\right)=\left(\begin{array}{cc}1& 1\\ 0& 0\\ 6h_1^2& 0\end{array}\right)$$

and $clIm{\Psi }_3\left(h\right)=span\{(1,0,0),(1,0,1)\}$.

For $h=(0,h_2)$, the 3-factor operator takes the form

$${\Psi }_3\left(h\right)=\left(\begin{array}{cc}1& 1\\ 2h_2^2& 0\\ 0& 0\end{array}\right)$$

and $clIm{\Psi }_3\left(h\right)=span\{(1,1,0),(1,0,0)\}$.

Now, using (22), we determine the elements $h=(h_1,h_2)$ in the kernels by solving the following equations:

$$f_1^{\prime }\left(\overline{x}\right)h=\left(\begin{array}{c}{h}_1+h_2\\ 0\\ 0\end{array}\right)=\left(\begin{array}{c}0\\ 0\\ 0\end{array}\right),f_3^{\prime \prime \prime }\left(\overline{x}\right){\left[h\right]}^3=\left(\begin{array}{c}0\\ 6h_1{h}_2^2\\ 6h_1^3\end{array}\right)=\left(\begin{array}{c}0\\ 0\\ 0\end{array}\right).$$

Thus, we obtain

$$Ker{f}_1^{\prime }\left(0\right)=\{h=(h_1,h_2)|h_1+h_2=0\}=span\left\{(1,-1)\right\},$$

and

$$Ker{f}_3^{\prime \prime \prime }\left(0\right)=\{h=(h_1,h_2)|6h_1{h}_2^2=0,6h_1^3=0\}=span\left\{(0,1)\right\}.$$

Finally, one can verify that

$$Ker{f}_2^{\prime \prime }(0,0)={\mathbb{R}}^2.$$

4. Singular Problems and Classical Results via the $\mathit{p}$-Regularity Theory

4.1. Lyusternik Theorem and Description of Solution Sets

The Lyusternik theorem plays an important role in describing solution sets of nonlinear equations and feasible sets of optimization problems in the regular case. This theorem has practical applications across various fields. It is particularly important in the study of optimization and variational problems. By characterizing the tangent cone, the theorem provides valuable information about critical points and the behavior of solutions in their vicinity. In control theory, the Lyusternik theorem can be used to analyze the stability and controllability of nonlinear control systems. By examining the tangent cone, one can gain insights into system behavior near critical points and determine the conditions necessary for stability and controllability. The Lyusternik theorem is also useful in the development and analysis of optimization algorithms, such as gradient-based methods. By characterizing the tangent cone, the theorem helps in designing efficient algorithms and understanding their convergence properties.

These are just a few examples of the practical applications of the Lyusternik theorem. Its insights into the tangent cone are valuable in many areas, including optimization, control theory, partial differential equations, and geometry, providing a deeper understanding of the behavior of solutions and critical points in a variety of mathematical problems.

Consider a nonlinear mapping $F:U\to Y$, where U is a neighborhood of a point $\overline{x}\in X$. We are interested in the description of the set $M\left(\overline{x}\right)$:

$$M\left(\overline{x}\right)=\left\{x\in U\mid F\left(x\right)=F\left(\overline{x}\right)\right\}.$$

(26)

This notation highlights the fact that we will focus our attention on $\overline{x}.$ It is useful to recall the following definition of tangent vectors and tangent cones (see, for instance, [29]).

Definition 8. 

Let M be a subset of a Banach space X. A vector $h\in X$ is said to be tangent to the set M at a point $\overline{x}\in M$ if there exist $\epsilon >0$ and a mapping $r\left(t\right)$, $r:[0,\epsilon ]\to X$, such that

$$\overline{x}+th+r\left(t\right)\in M\forall t\in [0,\epsilon ],$$

and

$$\underset{t\to 0}{\lim }{\displaystyle \frac{\parallel r\left(t\right)\parallel }{t}}=0.$$

The set of vectors tangent to M at the point $\overline{x}$ is called the tangent cone to the set M at $\overline{x}$, and is denoted by $T_{1}M\left(\overline{x}\right)$.

4.1.1. Lyusternik Theorem in the Regular Case

In the regular case, the Lyusternik theorem (see [30]) can be formulated as follows.

Theorem 5 

(Lyusternik theorem). Let X and Y be Banach spaces, and let U be a neighborhood of $\overline{x}$ in X. Assume that $F:U\to Y$ is Fréchet differentiable on U, and that its derivative $F^{\prime }:U\to \mathcal{L}(X,Y)$ is continuous at $\overline{x}$. Suppose further that F is regular at $\overline{x}$.

Then, the tangent cone to the set $M\left(\overline{x}\right)$ defined in (26) coincides with the kernel of the operator $F^{\prime }\left(\overline{x}\right)$:

$$T_{1}M\left(\overline{x}\right)=Ker{F}^{\prime }\left(\overline{x}\right).$$

(27)

If F is singular at $\overline{x}$, then in some problems we may have $T_{1}M\left(\overline{x}\right)\ne Ker{F}^{\prime }\left(\overline{x}\right)$, as illustrated in the following example.

Example 3. 

Let $X={\mathbb{R}}^2$, and let $x=(x_1,x_2)\in {\mathbb{R}}^2$. Define the mapping $F:{\mathbb{R}}^2\to \mathbb{R}$ by

$$F\left(x\right)=x_1^2-x_2^2+{o(\parallel x\parallel }^2).$$

Then, the derivative of F is given by $F^{\prime }(x_1,x_2)=[2x_1,-2x_2].$ Evaluating at $\overline{x}=(0,0)$, we obtain $F(0,0)=0$, $F^{\prime }(0,0)=(0,0)$, and $Ker{F}^{\prime }(0,0)={\mathbb{R}}^2$. Calculating

$$T_{1}M(0,0)=\mathrm{span}\left\{(1,1)\right\}\cup \mathrm{span}\left\{(1,-1)\right\},$$

we conclude that $T_{1}M(0,0)\ne Ker{F}^{\prime }(0,0)$.

Example 4. 

Let $F:C\left(\right[0,1\left]\right)\to C\left(\right[0,1\left]\right)$ be defined as $F\left(x\right(t\left)\right)=x\left(t\right).$ Then, the set

$$M=\left\{x\right(t)\in C([0,1])\mid F(x\left(t\right))=F(0)=0\}=\left\{0\right\}$$

consists only of the zero function. The derivative of F, given by $F^{\prime }\left(x\left(t\right)\right)=I$, is surjective, where I is the identity operator on $C\left(\right[0,1\left]\right)$. Moreover, we have $Ker{F}^{\prime }\left(0\right)=\left\{0\right\}=T_{1}M\left(0\right).$

Example 5. 

Let $M=\left\{x\left(t\right)\in C\left([0,1]\right)\mid \underset{0}{\overset{1}{\int }}sinx\left(t\right)dt={\displaystyle \frac{2}{\pi }}\right\}$ and define $\overline{x}\left(t\right)=\pi t$. To calculate $T_{1}M\left(\overline{x}\right)$, it is enough to apply Lyusternik’s theorem with $F:C\left(\right[0,1\left]\right)\to \mathbb{R}$ given by $F\left(x\left(t\right)\right)=\underset{0}{\overset{1}{\int }}sinx\left(t\right)dt.$

Using the trigonometric addition formulas, we obtain

$${\displaystyle \frac{F(x+h)-F\left(x\right)}{\parallel h\parallel }}={\int }_0^{1}sinx\left(t\right){\displaystyle \frac{cosh\left(t\right)-1}{\parallel h\left(t\right)\parallel }}dt+{\int }_0^{1}cosx\left(t\right){\displaystyle \frac{sinh\left(t\right)}{\parallel h\left(t\right)\parallel }}dt.$$

The first term on the right-hand side approaches 0 as $\parallel h\left(t\right)\parallel \to 0$. In the second term, we use the fact that ${\displaystyle \frac{sinh\left(t\right)}{\parallel h\left(t\right)\parallel }}$ approaches 1 as $\parallel h\left(t\right)\parallel \to 0.$

Therefore, the derivative of F at $\overline{x}$ is given by

$$F^{\prime }\left(\overline{x}\left(t\right)\right)\left(x\left(t\right)\right)={\int }_0^{1}x\left(t\right)cos\overline{x}\left(t\right)dt,$$

which is surjective onto $\mathbb{R}$. By Theorem 5, we conclude that

$$T_{1}M\left(\overline{x}\right)=Ker{F}^{\prime }\left(\overline{x}\right)=\{x\left(t\right)\in C\left([0,1]\right)\mid {\int }_0^{1}x\left(t\right)cos\overline{x}\left(t\right)dt=0\}.$$

The problem of describing solution sets in more general settings (e.g., nonlinear systems of inequalities) is approached qualitatively using metric regularity [31,32] and geometric derivability [33].

4.1.2. A Generalization of the Lyusternik Theorem

Consider the problem of describing the set $M\left(\overline{x}\right)$ in the nonregular case. As demonstrated in Example 3, the classical Lyusternik theorem 5 may not hold when F is singular at $\overline{x}$, so that $T_{1}M\left(\overline{x}\right)\ne Ker{F}^{\prime }\left(\overline{x}\right)$.

The first generalization of the classical Lyusternik theorem for p-regular mappings was independently derived and proved in [15,17]; see also [21]. This generalization can be used to describe the zero set of a p-regular mapping.

Theorem 6 

(Generalized Lyusternik theorem, [15]). Let X and Y be Banach spaces, and let U be a neighborhood of a point $\overline{x}\in X$. Assume that $F:U\to Y$ is a p–times continuously Fréchet differentiable mapping on U. Assume also that F is p-regular at $\overline{x}$.

Then,

$$T_{1}M\left(\overline{x}\right)=H_p\left(\overline{x}\right),$$

where the set $H_p\left(\overline{x}\right)$ is defined in (23).

The problem of describing the tangent cone to the solution set $M\left(\overline{x}\right)$ of a nonlinear Equation (3) with a singular mapping F has also been studied in other papers (see, for example, [16,34]).

Example 6. 

To illustrate the statement of Theorem 6, define mapping $F:{\mathbb{R}}^3\to {\mathbb{R}}^2$ by

$$F\left(x\right)=\left(\begin{array}{c}{x}_1^2-x_2^2+x_3^2\\ x_1^2-x_2^2+x_3^2+x_2{x}_3\end{array}\right).$$

(28)

Consider $\overline{x}={(0,0,0)}^T.$ A straightforward computation shows that $F^{\prime }\left(\overline{x}\right)=0$, and

$$F^{\prime \prime }\left(\overline{x}\right)=\left(\begin{array}{c}\left(\begin{array}{ccc}2& 0& 0\\ 0& -2& 0\\ 0& 0& 2\end{array}\right)\\ \\ \left(\begin{array}{ccc}2& 0& 0\\ 0& -2& 1\\ 0& 1& 2\end{array}\right)\end{array}\right).$$

Also,

$${Ker}^2{F}^{\prime \prime }\left(\overline{x}\right)=span\left(\left[\begin{array}{c}1\\ -1\\ 0\end{array}\right]\right)\bigcup span\left(\left[\begin{array}{c}1\\ 1\\ 0\end{array}\right]\right).$$

Let $h={(1,1,0)}^T$ (or $h={(1,-1,0)}^T$), then $Im{F}^{\prime \prime }\left(\overline{x}\right)h={\mathbb{R}}^2.$ Hence, the mapping $F\left(x\right)$ is 2-regular at $\overline{x}=0$. Then the statement of Theorem 6 in this example reduces to

$$T_{1}M\left(\overline{x}\right)=H_2\left(\overline{x}\right)={Ker}^2{F}^{\prime \prime }\left(\overline{x}\right)$$

or

$$T_{1}M\left(\overline{x}\right)={Ker}^2{F}^{\prime \prime }\left(\overline{x}\right)=span\left(\left[\begin{array}{c}1\\ -1\\ 0\end{array}\right]\right)\bigcup span\left(\left[\begin{array}{c}1\\ 1\\ 0\end{array}\right]\right).$$

The next theorem presents another version of Theorem 6, which was formulated in [24] (see also [17,21] for additional results along these lines). To state the result, we denote by $\mathrm{dist}(x,M)$, the distance function from a point $x\in X$ to a set M:

$$\mathrm{dist}(x,M)=\underset{y\in M}{inf}\parallel x-y\parallel ,x\in X.$$

Theorem 7. 

Let X and Y be Banach spaces, and let U be a neighborhood of a point $\overline{x}\in X$. Assume that $F:X\to Y$ is a p-times continuously Fréchet differentiable mapping in U. Assume also that F is strongly p-regular at $\overline{x}$. Then, there exist a neighborhood $U^{\prime }\subseteq U$ of $\overline{x}$, a mapping $\xi \to x\left(\xi \right):U^{\prime }\to X$, and constants ${\delta }_1>0$ and ${\delta }_2>0$, such that for all $\xi \in U^{\prime }$ the following holds:

$$F(\xi +x\left(\xi \right))=F\left(\overline{x}\right),$$

$$\mathrm{dist}(\xi ,M\left(\overline{x}\right))\le \parallel x\left(\xi \right)\parallel \le {\delta }_1\sum _{i=1}^p{\displaystyle \frac{\parallel f_i\left(\xi \right)-f_i\left(\overline{x}\right)\parallel }{\parallel \xi -\overline{x}{\parallel }^{i-1}}},$$

(29)

where $f_i$ are given by (16), and

$$\mathrm{dist}(\xi ,M\left(\overline{x}\right))\le \parallel x\left(\xi \right)\parallel \le {\delta }_2\sum _{i=1}^p{\parallel f_i\left(\xi \right)-f_i\left(\overline{x}\right)\parallel }^{1/i}.$$

For the proof, see [21].

4.1.3. Representation Theorem

The Representation Theorem is used in nonlinear analysis and is relevant to the study of the local behavior and representation of a mapping F around a special point $\overline{x}$. It also guarantees the existence of certain auxiliary mappings that have desirable properties and relate to the given mapping F and its local representation in some neighborhood of $\overline{x}$.

The Representation Theorem can be used, for example, in the study of optimization problems, particularly in constrained optimization. It helps in establishing the existence of critical points and characterizing their properties, which is essential for finding optimal solutions. The theorem is also relevant to variational methods, partial differential equations, and other areas of mathematical analysis. Moreover, it is useful in various numerical methods and computational techniques for approximating solutions of equations. Its versatility and utility stem from its ability to provide insights into the local behavior and representations of mappings near critical points, with wide-ranging applications in mathematical analysis and optimization. Its versatility and usefulness stem from its ability to provide insights into the local behavior and representations of mappings around critical points, which has wide-ranging applications in mathematical analysis and optimization.

To simplify the presentation of the next result, we state it for the case of the completely degenerate mapping F, defined in (18). Recall that in this case, $Im{F}^{\left(p\right)}\left(\overline{x}\right){\left[h\right]}^{p-1}=Y$, and the p-factor operator can be simplified to ${\Psi }_p\left(h\right)x={\displaystyle \frac{1}{p!}}{F}^{\left(p\right)}\left(\overline{x}\right){\left[h\right]}^{p-1}x$.

Theorem 8 

([22]). Let X and Y be Banach spaces, and let V be a neighborhood of $\overline{x}$ in X. Suppose that $F:V\to Y$ is of class ${\mathcal{C}}^{p+1}$, and that $F^{\left(i\right)}\left(\overline{x}\right)=0$ for $i=1,\dots ,p-1$. Also assume the existence of a constant $C>0$, such that

$$\underset{\parallel h\parallel =1}{sup}∥{\left\{F^{\left(p\right)}\left(\overline{x}\right){\left[h\right]}^{p-1}\right\}}^{-1}∥\le C.$$

Then, there exist a neighborhood U of 0 in X, a neighborhood V of $\overline{x}$ in X, and mappings $\phi :U\to X$ and $\psi :V\to X$, such that φ and ψ are Fréchet-differentiable at 0 and $\overline{x}$, respectively, and the following hold:

 1. 

$\phi \left(0\right)=\overline{x},$$\psi \left(\overline{x}\right)=0$;

 2. 

$F\left(\phi \left(x\right)\right)=F\left(\overline{x}\right)+{\displaystyle \frac{1}{p!}}{F}^{\left(p\right)}\left(\overline{x}\right){\left[x\right]}^p$ for all $x\in U$;

 3. 

$F\left(x\right)=F\left(\overline{x}\right)+{\displaystyle \frac{1}{p!}}{F}^{\left(p\right)}\left(\overline{x}\right){\left[\psi \left(x\right)\right]}^p$ for all $x\in V$;

 4. 

${\phi }^{\prime }\left(0\right)={\psi }^{\prime }\left(\overline{x}\right)=I_X$.

All assumptions of Theorem 8 are satisfied, for example, by the mapping

$$F(x_1,x_2)=x_1^p-x_2^p+x_1^{p+1}+x_2^{p+1},$$

where $p\ge 2$, $p\in \mathbb{N}$. See also [35] for additional work on the representation theorem.

4.1.4. Morse Lemma

The Morse Lemma is another fundamental result in analysis that relates the behavior of a smooth function near a nondegenerate critical point $\overline{x}$ to the local structure of its level sets. The Morse Lemma has several important applications in various areas of mathematics.

The Morse Lemma is used in differential geometry to analyze the behavior of geodesics and study the geometry of manifolds. By considering a function that measures the length or energy of curves on a manifold, the Morse Lemma allows us to understand the critical points of this function and their geometric implications. It provides insights into the existence, stability, and bifurcations of geodesics on a manifold.

The Morse Lemma has important applications in optimization and control theory, where it is used to analyze the behavior of objective functions and control systems near critical points. It helps characterize the local behavior of optimal solutions and understand stability properties. The Lemma can be employed to find critical points, perform sensitivity analysis, and study bifurcations in optimization problems and dynamical systems.

The Morse Lemma is also utilized in singularity theory, which focuses on the properties and classification of singular points or critical points of differentiable mappings. It provides a framework for understanding the local behavior of singularities and the ways in which their structure may change under small perturbations. The Lemma plays a key role in the classification and analysis of singular points and their stability.

The most interesting formulation of the Morse Lemma in the finite-dimensional case is given in the following lemma.

Lemma 2 

(Morse Lemma). Let $\overline{x}\in {\mathbb{R}}^n$, and let $f:{\mathbb{R}}^n\to \mathbb{R}$ be a function of class ${\mathcal{C}}^3\left({\mathbb{R}}^n\right)$, such that $f^{\prime }\left(\overline{x}\right)=0$ and the Hessian $f^{\prime \prime }\left(\overline{x}\right)$ is not degenerate. Then, in a neighborhood V of $\overline{x}$, there exist a curvelinear coordinate system $(y_1,\dots ,y_n)$ and an integer number $k\in \{0,\dots ,n\}$, such that

$$f\left(x\right)=f\left(\overline{x}\right)+\sum _{i=1}^k{y}_i^2-\sum _{i=k+1}^n{y}_i^2$$

for all $x\in V$.

Proof. 

Without loss of generality, we can assume that Hessian matrix $f^{\prime \prime }\left(\overline{x}\right)$ is diagonal:

$$f^{\prime \prime }\left(\overline{x}\right)=\left(\begin{array}{ccccc}1& 0& \dots & 0& 0\\ 0& 1& \dots & 0& 0\\ ⋮& ⋮& \ddots & ⋮& ⋮\\ 0& 0& \dots & -1& 0\\ 0& 0& \dots & 0& -1\end{array}\right),$$

where for some number $k^{\prime }$ between 0 and n, the first $k^{\prime }$ columns have 1 on the main diagonal, and the other columns have $-1$. Otherwise, changing the basis, we can transform the Hessian to be a diagonal matrix.

Then, in this case,

$$f\left(x\right)=f\left(\overline{x}\right)+\sum _{i=1}^{k^{\prime }}{(x_i-x_i^{*})}^2-\sum _{i=k^{\prime }+1}^n{(x_i-x_i^{*})}^2+o(\parallel x-\overline{x}{\parallel }^3).$$

Note that if the assumptions of the Morse Lemma hold, then the assumptions of the representation Theorem 8 are satisfied with $p=2$ and $F=f$. Hence, there exists a mapping $\psi \left(x\right):U\to \mathbb{R}$, such that

$$f\left(\psi \left(x\right)\right)=f\left(\overline{x}\right)+{\displaystyle \frac{1}{2}}{f}^{\prime \prime }\left(\overline{x}\right){\left[x\right]}^2,$$

where $\parallel \psi \left(x\right)-(x-\overline{x})\parallel =o(\parallel x-\overline{x}\parallel )$ and ${\psi }^{\prime }\left(\overline{x}\right)=I_X$. It follows that $k=k^{\prime }$. Note that if $k=k^{\prime }$, then $\psi \left(x\right)=x-\overline{x}+o(x-\overline{x})$, and if $k\ne k^{\prime }$, then we obtain a contradiction. Now, we can apply the statement of the representation Theorem 8 to the mapping $y=\psi \left(x\right)$ to get the statement of Morse Lemma 2. □

See additional work on Morse Lemma in [36].

4.2. Implicit Function Theorem

In this section, we consider the equation $F(x,y)=0$, where $F:X\times Y\to Z$ and X, Y, and Z are Banach spaces. Let $(\overline{x},\overline{y})$ be a given point in $X\times Y$ that satisfies $F(\overline{x},\overline{y})=0$. We are interested in the existence of a mapping $\phi \left(x\right)$ defined in a neighborhood $U\left(\overline{x}\right)$, such that $\phi \left(x\right):U\left(\overline{x}\right)\to Y$ is a solution of the equation $F(x,y)=0$ near the given point $(\overline{x},\overline{y})$. This mapping should satisfy the following conditions:

$$F(x,\phi \left(x\right))=0and\overline{y}=\phi \left(\overline{x}\right).$$

(30)

4.2.1. Implicit Function Theorem in the Regular Case

In the case when $F:X\times Y\to Z$ is a continuously differentiable mapping, we denote its (Fréchet) derivative with respect to y at a point $(\overline{x},\overline{y})\in X\times Y$ by $F_y^{\prime }(\overline{x},\overline{y}):Y\to Z$.

In the case when F is regular at a point $(\overline{x},\overline{y})$, meaning $F_y^{\prime }(\overline{x},\overline{y})$ is onto, the classical Implicit Function Theorem (IFT) guarantees the existence of a smooth mapping $\phi \left(x\right)$ defined in a neighborhood $U\left(\overline{x}\right)$, such that (30) holds and $\parallel \phi \left(x\right)-\overline{y}\parallel \le C\parallel F(x,\overline{y})\parallel$ for all x in $U\left(\overline{x}\right)$, where $C>0.$ There are numerous books and papers devoted to the IFT, including [13,19]. Various formulations of the standard IFT exist, and Theorem 9 presents one such statement.

Theorem 9 

(Implicit Function Theorem). Let X and Y be Banach Spaces. Assume that $F:X\times Y\to Z$ is continuously Fréchet differentiable at $(\overline{x},\overline{y})\in X\times Y$, $F(\overline{x},\overline{y})=0$, and that $Im{F}_y^{\prime }(\overline{x},\overline{y})=Z.$ Then, there exist constants $C,C_1>0$, a sufficiently small $\delta >0$, and a function $\phi :B(\overline{x},\delta )\to Y$ such that, for $x\in B(\overline{x},\delta )$, the following holds:

$$\phi \left(\overline{x}\right)=\overline{y},F(x,\phi \left(x\right))=0,\parallel \phi \left(x\right)-\overline{y}\parallel \le C_1\parallel F(x,\overline{y})\parallel \le C\parallel x-\overline{x}\parallel .$$

(31)

The situation changes when the mapping F is degenerate (nonregular) at $(\overline{x},\overline{y})$; that is, when $F_y^{\prime }(\overline{x},\overline{y})$ is not onto. In this case, the classical IFT cannot be applied to guarantee the (local) existence of a solution $y=\phi \left(x\right)$. The importance of examining this situation arises from the need to solve various nonlinear problems, many of which, as shown in [22], are singular (degenerate).

4.2.2. Implicit Function Theorem in the Degenerate Case

In this section, we focus on the case when mapping $F:X\times Y\to Z$ is not regular; that is, when $F_y^{\prime }(\overline{x},\overline{y})$ is not onto.

As an example, consider mapping $F:\mathbb{R}\times \mathbb{R}\to \mathbb{R}$, $F(x,y)=x-y^p$, where $p=2k+1$ with some $k\in \mathbb{N}$. If $(\overline{x},\overline{y})=(0,0)$, then $F(\overline{x},\overline{y})=0$ and $F_y^{\prime }(\overline{x},\overline{y})=0$, so the mapping F is not surjective. The classical IFT is not applicable in this case. However, there exists mapping $\phi \left(x\right)=x^{1/p}$, such that $F(x,x^{1/p})=x-{\left(x^{1/p}\right)}^p=0$. Moreover, $\parallel \phi \left(x\right)-\overline{y}\parallel =\parallel F(x,\overline{y}){\parallel }^{1/p}$, and, by (31), the following inequality holds with $C=1>0$:

$$\parallel \phi \left(x\right)-\overline{y}\parallel \le C\parallel F(x,\overline{y}){\parallel }^{1/p}.$$

Thus, while the conditions of a standard implicit function are not satisfied in the example, the statement similar to (31) holds. The example serves as a motivation and illustration for the p-order IFT. To our knowledge, the first generalization of the IFT for nonregular mappings was formulated in [24]. Generalizations of the IFT for 2-regular mappings were obtained in [21,36]. We will present a few versions of the IFT for p-regular mappings in this section.

To simplify the presentation, we begin with Theorem 10, which is stated in Euclidean spaces. A slight modification of this theorem was derived in [21]. To formulate the theorem, we first need to define the operator ${\Psi }_p\left(h\right)$ related to the mapping $F:X\times Y\to Z$. To do so, and similarly to the mappings introduced in Section 3, we define the following mappings (see [24]):

$$f_i(x,y):X\times Y\to Z_i,f_i(x,y)=P_{Z_i}F(x,y),i=1,\dots ,p,$$

(32)

where $P_{Z_i}:Z\to Z_i$ is the projection operator onto $Z_i$ along $Z_1\oplus \dots \oplus Z_{i-1}\oplus Z_{i+1}\oplus \dots \oplus Z_p$ with respect to Z for $i=1,\dots ,p$. The definition of $Z_i$ is similar to the definition of the subspaces $Y_i$ in Section 3.

Now we are ready to present the definition of the linear operator ${\Psi }_p\left(h\right):Y\to Z_1\times \dots \times Z_p$, which is similar to the operator ${\Psi }_p\left(h\right)$ defined in (19). Since the construction of the p factor-operators are similar, we retain the same notation to keep the presentation clear and consistent. For a fixed vector $h\in Y,$$h\ne 0$, and mappings $f_i$ defined in (16), the linear operator ${\Psi }_p\left(h\right)\in \mathcal{L}(Y,Z_1\oplus \dots \oplus Z_p)$ is given by

$${\Psi }_p\left(h\right)y={f_1^{\prime }}_y(\overline{x},\overline{y})y+{f_2^{\prime \prime }}_{yy}(\overline{x},\overline{y})\left[h\right]y+\dots +{f_p^{\left(p\right)}}_{y\dots y}(\overline{x},\overline{y}){\left[h\right]}^{p-1}y,y\in Y,$$

(33)

where $y\dots y$ indicates that all derivatives are taken with respect to the same variable y, which belongs to Y.

Before stating Theorem 10, we introduce some additional notation that will be used:

• In the expression ${f_i^{\left(r\right)}}_{\underset{q}{\underbrace{x\dots x}}\underset{r-q}{\underbrace{y\dots y}}}(\overline{x},\overline{y})$, r represents the total order of differentiation, where differentiation is performed q times with respect to x and $r-q$ times with respect to y.
• While the notation ${\left[h\right]}^{r-1}$ appears in the definition (33) of the linear operator ${\Psi }_p\left(h\right)$, the expression ${f_i^{\left(r\right)}}_{\underset{q}{\underbrace{x\dots x}}\underset{r-q}{\underbrace{y\dots y}}}(\overline{x},\overline{y})=0$ signifies that all components of the derivative are equal to zero.
• The subscript notation $x\dots x$ (q-times) indicates partial differentiation with respect to the first variable x performed q times.
• For $r=0$, the notation $f_i^{\left(0\right)}(\overline{x},\overline{y})$ represents the function value $f_i(\overline{x},\overline{y})$ itself.

Theorem 10 

(Implicit Function Theorem [22]). Suppose that X, Y, and Z are Euclidean spaces, and let W be a neighborhood of a point $(\overline{x},\overline{y})$ in $X\times Y$. Assume that $F:W\to Z$ is of class ${\mathcal{C}}^2$. Suppose $F(\overline{x},\overline{y})=0$ and there exists a neighborhood $U\left(\overline{x}\right)$ in X, such that the following conditions hold:

• The Singularity Condition:

  $${f_i^{\left(r\right)}}_{\underset{q}{\underbrace{x\dots x}}\underset{r-q}{\underbrace{y\dots y}}}(\overline{x},\overline{y})=0,r=0,\dots ,i-1,q=0,\dots ,r-1,i=1,\dots ,p;$$

  $${f_i^{\left(i\right)}}_{\underset{q}{\underbrace{x\dots x}}\underset{i-q}{\underbrace{y\dots y}}}(\overline{x},\overline{y})=0,q=1,\dots ,i-1,i=1,\dots ,p.$$
• The pth Order Regularity Condition at the Point $(\overline{x},\overline{y})$:
  The operator ${\Psi }_p\left(h\right)$ defined in (33) satisfies

  $${\Psi }_p\left(h\right)Y=Z$$

  for all $h\in {\left\{{\left({\Psi }_p\left(h\right)\right)}_y\right\}}^{-1}(-F(x,\overline{y}))$ and all $x\in U\left(\overline{x}\right)$, such that $F(x,\overline{y})\ne 0$.
• The Banach Condition:
  There exists a constant $c>0$ such that, for any $z\in Z$ with $\parallel z\parallel =1$, the following holds:

  $${\Psi }_p\left(h\right)y=z,\parallel y\parallel \le c.$$
• The Elliptic Condition with respect to x:
  There exists a constant $m>0$ such that

  $$\parallel f_i(x,\overline{y})\parallel \ge m\parallel x-\overline{x}\parallel$$

  for all $x\in U\left(\overline{x}\right)$ and for all $i=1,\dots ,p$.

If conditions 1 to 4 are satisfied, then for any $\epsilon >0$, there exist $\delta >0$ and $K>0$ such that $B(\overline{x},\delta )\subset U\left(\overline{x}\right)$, and there is a map $\phi :B(\overline{x},\delta )\to B(\overline{y},\epsilon )$ satisfying:

 (a) 

$\phi \left(\overline{x}\right)=\overline{y}$;

 (b) 

$F(x,\phi (x\left)\right)=0$ for all $x\in B(\overline{x},\delta )$;

 (c) 

$\parallel \phi \left(x\right)-\overline{y}\parallel \le K\sum _{i=1}^p{∥f_i(x,\overline{y})∥}^{1/i}$ for all $x\in B(\overline{x},\delta )$.

The alternative version of the IFT for nonregular mappings, presented as Theorem 11, was proved in [37]. Before stating the theorem, we introduce the following definition (Definition 2.3 in [38]).

Definition 9. 

The mapping $F:X\times Y\to Z$ is called uniformly p-regular over a set M in Y if

$$\underset{h\in M}{sup}\parallel {\left\{{\Psi }_p\left(\overline{h}\right)\right\}}^{-1}\parallel <\infty ,\overline{h}={\displaystyle \frac{h}{\parallel h\parallel }},h\ne 0,$$

where

$$\parallel {\left\{{\Psi }_p\left(\overline{h}\right)\right\}}^{-1}\parallel =\underset{\parallel z\parallel =1}{sup}inf\{\parallel y\parallel \left|{\Psi }_p\left(h\right)\left[y\right]=z\right.\}.$$

Additionally, we define the mapping ${\Phi }_p:Y\to Z_1\oplus \dots \oplus Z_p$ by

$${\Phi }_p=\left(f_{1y}^{\prime }(\overline{x},\overline{y}),{\displaystyle \frac{1}{2}}{f}_{2yy}^{\prime \prime }(\overline{x},\overline{y}),\dots ,{\displaystyle \frac{1}{p!}}{f}_{py\dots y}^{\left(p\right)}(\overline{x},\overline{y})\right),$$

where

$${\Phi }_p{\left[y\right]}^p=\left(f_{1y}^{\prime }(\overline{x},\overline{y})\left[y\right],{\displaystyle \frac{1}{2}}{f}_{2yy}^{\prime \prime }(\overline{x},\overline{y}){\left[y\right]}^2,\dots ,{\displaystyle \frac{1}{p!}}{f}_{py\dots y}^{\left(p\right)}(\overline{x},\overline{y}){\left[y\right]}^p\right).$$

Under the assumption that $Z_1\oplus \dots \oplus Z_p=Z$, we also introduce the corresponding inverse multivalued operator ${\Phi }_p^{-1}$:

$${\Phi }_p^{-1}\left(z\right)=\left\{\eta \in Y\left|\left(f_{1y}^{\prime }(\overline{x},\overline{y})\left[\eta \right],\dots ,{\displaystyle \frac{1}{p!}}{f}_{py\dots y}^{\left(p\right)}(\overline{x},\overline{y}){\left[\eta \right]}^p\right)\right.=(z_1,z_2,\dots ,z_p)\right\},$$

where $z_i\in Z_i$, $i=1,\dots ,p$.

Theorem 11 

(The pth-order IFT). Let X, Y and Z be Banach spaces, and let $U\left(\overline{x}\right)$ and $U\left(\overline{y}\right)$ be sufficiently small neighborhoods of $\overline{x}\in X$ and $\overline{y}\in Y$, respectively. Suppose that $F\in C^{p+1}(X\times Y)$ and $F(\overline{x},\overline{y})=0$. Assume that the mappings $f_i(x,y)$, $i=1,\dots ,p$, introduced in Equation (32), satisfy the following conditions:

(1)

The Singularity Condition:

$$\begin{array}{c}{f}_{i\underset{q}{\underbrace{\phantom{(}x\dots x}}\underset{r-q}{\underbrace{\phantom{(}y\dots y}}}^{\left(r\right)}(\overline{x},\overline{y})=0,r=1,\dots ,i-1,q=0,\dots ,r-1,i=1,\dots ,p,\hfill \\ f_{i\underset{q}{\underbrace{\phantom{(}x\dots x}}\underset{i-q}{\underbrace{\phantom{(}y\dots y}}}^{\left(i\right)}(\overline{x},\overline{y})=0,q=1,\dots ,i-1,i=1,\dots ,p.\hfill \end{array}$$

(2)

The p-Factor Approximation Condition:

There exists a sufficiently small $\epsilon >0$ such that, for all $y_1,y_2\in (U\left(\overline{y}\right)\setminus \left\{\overline{y}\right\})$, the following holds:

$$\begin{array}{cc}\hfill & ∥f_i(x,\overline{y}+y_1)-f_i(x,\overline{y}+y_2)-{\displaystyle \frac{1}{i!}}{f}_{iy\dots y}^{\left(i\right)}(\overline{x},\overline{y}){\left[y_1\right]}^i+{\displaystyle \frac{1}{i!}}{f}_{iy\dots y}^{\left(i\right)}(\overline{x},\overline{y}){\left[y_2\right]}^i∥\hfill \\ \hfill & \le \epsilon \left(\parallel y_1{\parallel }^{i-1}+{\parallel y_2\parallel }^{i-1}\right)\parallel y_1-y_2\parallel ,i=1,\dots ,p.\hfill \end{array}$$

(3)

The Banach Condition:

There exists a nonempty open set $\Gamma \left(\overline{x}\right)\subset U\left(\overline{x}\right)$ in X such that for any sufficiently small γ, the intersection of the set $\Gamma \left(\overline{x}\right)$ with the ball $B(\overline{x},\gamma )$ is not empty and $\Gamma \left(\overline{x}\right)\cap B(\overline{x},\gamma )\ne \left\{\overline{x}\right\}$. Moreover, for $x\in \Gamma \left(\overline{x}\right)$, there exist $h\left(x\right):X\to Y$ and a constant c such that $0<c_1<\infty$ and

$${\Phi }_p{\left[h\left(x\right)\right]}^p=-F(x,\overline{y}),\parallel h\left(x\right)\parallel \le c_1\sum _{r=1}^p{\parallel f_r(x,\overline{y})\parallel }^{1/r},$$

(34)

(4)

The Uniform p-Regularity Condition:

The mapping $F(x,y)$ is uniformly p-regular over the set ${\Phi }_p^{-1}(-F(x,\overline{y})).$

If conditions 1 to 4 are satisfied, then there exists a constant $k>0$, a sufficiently small $\delta \le \gamma$, and a mapping $\phi :\Gamma \left(\overline{x}\right)\cap B(\overline{x},\delta )\to U\left(\overline{y}\right)$ such that the following hold for $x\in \Gamma \left(\overline{x}\right)\cap B(\overline{x},\delta )$:

$$\phi \left(\overline{x}\right)=\overline{y};$$

$$F(x,\phi (x\left)\right)=0,$$

$$\parallel \phi \left(x\right)-\overline{y}\parallel \le k\sum _{r=1}^p{\parallel f_r(x,\overline{y})\parallel }^{1/r}.$$

There are generalizations of IFT for nonregular mappings derived by other authors. Some examples include a generalization of the IFT and its application to a parametric linear time-optimal control problem presented in [39], generalized IFT applied to ordinary differential equations in [40], and IFT for 2-regular mappings in [41,42].

4.3. Newton’s Method

4.3.1. Classical Newton’s Method for Nonlinear Equations and Unconstrained Optimization Problems

Consider the problem of solving the nonlinear Equation (3), where $F:X\to Y$ is sufficiently smooth, so that $F\in {\mathcal{C}}^{p+1}\left(X\right)$ for some $p\in \mathbb{N}.$ Let $\overline{x}$ be a solution of (3), that is, $F\left(\overline{x}\right)=0$. Assume that mapping F is singular at the point $\overline{x}$.

In the finite dimensional case, when $F\left(x\right)={(F_1\left(x\right),\dots ,F_n\left(x\right))}^T$, $X={\mathbb{R}}^n$, and $Y={\mathbb{R}}^n$, the singularity of F at $\overline{x}$ means that the Jacobian $F^{\prime }\left(\overline{x}\right)$ of F is singular at $\overline{x}$, as in the following example.

Example 7 

([43]). Consider function $F:{\mathbb{R}}^2\to {\mathbb{R}}^2$ from Example 1, defined by

$$F\left(x\right)=\left(\begin{array}{c}{x}_1+x_2\\ x_1{x}_2\end{array}\right),$$

where $\overline{x}={(0,0)}^T$ is a solution to Equation (3) and $F^{\prime }\left(\overline{x}\right)=\left(\begin{array}{cc}1& 1\\ 0& 0\end{array}\right)$ is singular (degenerate) at the point $\overline{x}.$

Consider a sufficiently small $\epsilon >0$ and some initial point $x^0\in B(0,\epsilon )$. The classical Newton method is defined by

$$x^{k+1}=x^k-{\left(F^{\prime }\left(x^k\right)\right)}^{-1}F\left(x^k\right),k=0,1,\dots .$$

(35)

If $x^k=(x_1,x_2)$ in this example, we obtain

$$F^{\prime }\left(x^k\right)=\left(\begin{array}{cc}1& 1\\ x_2& x_1\end{array}\right),{\left(F^{\prime }\left(x^k\right)\right)}^{-1}={\displaystyle \frac{1}{x_1-x_2}}\left(\begin{array}{cc}{x}_1& -1\\ -x_2& 1\end{array}\right).$$

Then

$${\left(F^{\prime }\left(x^k\right)\right)}^{-1}F\left(x^k\right)={\displaystyle \frac{1}{x_1-x_2}}\left(\begin{array}{c}{x}_1^2\\ -x_2^2\end{array}\right),$$

$$x^{k+1}=x^k-{\left(F^{\prime }\left(x^k\right)\right)}^{-1}F\left(x^k\right)={\displaystyle \frac{1}{x_1-x_2}}\left(\begin{array}{c}-x_1{x}_2\\ x_1{x}_2\end{array}\right).$$

If $x_1=x_2$, then ${\left(F^{\prime }\left(x^k\right)\right)}^{-1}$ does not exist and, hence, method (35) is not applicable. Even in the case when ${\left(F^{\prime }\left(x^k\right)\right)}^{-1}$ exists, method (35) might be diverging. As an example, consider point $x^k={(t+t^3,t)}^T$, where t is sufficiently small. Then

$$x^{k+1}={\displaystyle \frac{1}{t^3}}\left(\begin{array}{c}-t^2-t^4\\ t^2+t^4\end{array}\right)={\left(-{\displaystyle \frac{1}{t}}-t,{\displaystyle \frac{1}{t}}+t\right)}^T$$

and, for a sufficiently small values of t, $\parallel x^{k+1}-\overline{x}\parallel =\parallel x^{k+1}-0\parallel \approx {\displaystyle \frac{1}{t}}\to \infty$ when $t\to 0^{+}.$ For instance, if $t={10}^{-5}$, then $\parallel x^{k+1}-0\parallel \approx {10}^5$ and the method (35) is diverging.

For the overview of the existing approaches to Newton-like methods for singular operators, see, e.g., [44].

Now we consider Newton’s method for finding critical points of an unconstrained optimization problem:

$$\underset{x\in {\mathbb{R}}^2}{min}f\left(x\right),$$

(36)

where $f:{\mathbb{R}}^2\to \mathbb{R}$. The classical Newton’s method applied to problem (36) has the form

$$x^{k+1}=x^k-{\left(f^{\prime \prime }\left(x^k\right)\right)}^{-1}{f}^{\prime }\left(x^k\right).$$

(37)

As an example, consider minimization of function f given by $f\left(x\right)=x_1^2+x_1^2{x}_2+x_2^4$ (see [43]). One of the critical points of the function f is $\overline{x}={(0,0)}^T$. Let $x^0={(x_1^0,x_2^0)}^T$ where $x_1^0=x_2^0\sqrt{6(1+x_2^0)}$. Then

$$f^{\prime \prime }\left(x^0\right)=\left(\begin{array}{cc}2+2x_2^0& 2x_2^0\sqrt{6(1+x_2^0)}\\ 2x_2^0\sqrt{6(1+x_2^0)}& 12{\left(x_2^0\right)}^2\end{array}\right)$$

and $det{f}^{\prime \prime }\left(x^0\right)=0.$ Hence, ${\left(f^{\prime \prime }\left(x^0\right)\right)}^{-1}$ does not exist, so Newton’s method (37) is not applicable.

4.3.2. The p-Factor Newton’s Method

In this section, we describe a method for solving nonlinear Equation (3), where $F:{\mathbb{R}}^n\to {\mathbb{R}}^n$ and the matrix $F^{\prime }\left(\overline{x}\right)$ is singular at the solution point $\overline{x}$ (see [43]). The proposed method is based on the construction of the p-factor operator.

There are various publications describing the p-factor-method for solving degenerate nonlinear systems and nonregular optimization problems. Some examples are given in [43,45,46].

Let $h\in {\mathbb{R}}^n$. Similarly to the definitions in Section 3, now we define $Y_1$ by $Y_1=\mathrm{Im}{F}^{\prime }\left(\overline{x}\right),$ and define the projection ${\overline{P}}_1=P_{Y_1^{\perp }}$ as the projection of Y onto the orthogonal complementary subspace $Y_1^{\perp }$ of $Y_1$ in Y. Similarly, we can define $Y_2$ as

$$Y_2=\mathrm{Im}\left(F^{\prime }\left(\overline{x}\right)+{\overline{P}}_1{F}^{\prime \prime }\left(\overline{x}\right)h\right),\mathrm{and}{\overline{P}}_2=P_{Y_2^{\perp }}.$$

Continuing in the same way for each $k=2,\dots ,p-1,$ we obtain ${\overline{P}}_{k+1}=P_{Y_{k+1}^{\perp }}$ and

$$Y_{k+1}=Im\left(F^{\prime }\left(\overline{x}\right)+\sum _{i=1}^k{\overline{P}}_i{F}^{\prime \prime }\left(\overline{x}\right)h+\sum _{\begin{array}{c}{i}_2>i_1\\ i_1,i_2\in \{1,2,3\}\end{array}}{\overline{P}}_{i_2}{\overline{P}}_{i_1}{F}^{\left(3\right)}\left(\overline{x}\right){\left[h\right]}^2+\dots \right.$$

$$\left.+\sum _{\begin{array}{c}{i}_k>\dots >i_1\\ i_1,\dots ,i_k\in \{1,\dots ,k\}\end{array}}{\overline{P}}_{i_k}\dots {\overline{P}}_{i_1}{F}^{\left(k\right)}\left(\overline{x}\right){\left[h\right]}^{(k-1)}\right).$$

Let h be a fixed vector such that $\parallel h\parallel =1$ and mapping F is p-regular at the solution $\overline{x}$ along vector h. Let matrices $P_i,i=1,\dots ,p-1$, be defined as follows:

$$P_1=\sum _{i=1}^{p-1}{\overline{P}}_i,P_2=\sum _{\begin{array}{c}{i}_2>i_1\\ i_1,i_2\in \{1,\dots ,p-1\}\end{array}}{\overline{P}}_{i_2}{\overline{P}}_{i_1},P_{k+1}=\sum _{\begin{array}{c}{i}_k>\dots >i_1\\ i_1,\dots ,i_k\in \{1,\dots ,p-1\}\end{array}}{\overline{P}}_{i_k}\dots {\overline{P}}_{i_1}$$

for all $k=2,\dots ,p-1.$

We assume that $\overline{x}$ is a solution of $F\left(x\right)=0$. Now, instead of $F\left(\overline{x}\right)=0$, consider

$$F\left(\overline{x}\right)+P_1{F}^{\prime }\left(\overline{x}\right)h+\dots +P_{p-1}{F}^{(p-1)}\left(\overline{x}\right){\left[h\right]}^{p-1}=0.$$

The assumption of p-regularity of the mapping F at the solution $\overline{x}$ along the vector h implies that the p-factor matrix given by

$$F^{\prime }\left(\overline{x}\right)+P_1{F}^{\prime \prime }\left(\overline{x}\right)h+\dots +P_{p-1}{F}^{\left(p\right)}\left(\overline{x}\right){\left[h\right]}^{p-1}$$

(38)

is not singular. Hence, ${\overline{P}}_p=0$ and $Y_p={\mathbb{R}}^n.$

Then, the p-factor Newton method can be defined as

$$x^{k+1}=x^k-{\left(F^{\prime }\left(x^k\right)+P_1{F}^{\prime \prime }\left(x^k\right)h+\dots +P_{p-1}{F}^{\left(p\right)}\left(x^k\right){\left[h\right]}^{p-1}\right)}^{-1}$$

(39)

$$\times \left(F\left(x^k\right)+P_1{F}^{\prime }\left(x^k\right)h+\dots +P_{p-1}{F}^{(p-1)}\left(x^k\right){\left[h\right]}^{p-1}\right).$$

The following theorem provides conditions that ensure the quadratic convergence of the p-factor Newton method (39).

Theorem 12 

([43]). Let $F\in {\mathcal{C}}^p\left({\mathbb{R}}^n\right)$, and let $\overline{x}$ be a solution of $F\left(x\right)=0$. Assume that there exists a vector $h\in {\mathbb{R}}^n$, $\parallel h\parallel =1$, such that the p-factor matrix defined in Equation (38) is not singular. Then, for any $x^0\in U_{\epsilon }\left(\overline{x}\right)$ (with $\epsilon >0$ sufficiently small) and for the sequence $\left\{x^k\right\}$ generated by the method in Equation (39), the following inequality holds for some constant $c>0$:

$$\parallel x^{k+1}-\overline{x}\parallel \le c\parallel x^k-\overline{x}{\parallel }^2,k=0,1,\dots .$$

(40)

In the case of $p=2$, the p-factor Newton method (39) reduces to the following:

$$x^{k+1}=x^k-{\left(F^{\prime }\left(x^k\right)+P_1{F}^{\prime \prime }\left(x^k\right)h\right)}^{-1}(F\left(x^k\right)+P_1{F}^{\prime }\left(x^k\right)h)$$

(41)

where $P_1$ is the orthogonal projection onto $\mathrm{Im}{\left(F^{\prime }\left(\overline{x}\right)\right)}^{\perp }$, and the vector h$(\parallel h\parallel =1)$ is chosen such that the 2-factor matrix

$$F^{\prime }\left(\overline{x}\right)+P_1{F}^{\prime \prime }\left(\overline{x}\right)h$$

(42)

is not singular. This condition is equivalent to F being 2-regular at $\overline{x}$ along h. In this case, the equation

$$F\left(\overline{x}\right)+P_1{F}^{\prime }\left(\overline{x}\right)h=0$$

is satisfied at $\overline{x}$. Note that (42) implies that $\overline{x}$ is a locally unique solution of (3).

The 2-factor Newton method presented here can be applied to solve the equation in Example 7. Specifically, instead of using the iterative procedure (35), the 2-factor Newton method given by (41) should be used.

Example 8 

([43]). Consider the following problem

$$\underset{x\in {\mathbb{R}}^2}{min}f\left(x\right),$$

where $f:{\mathbb{R}}^2\to \mathbb{R}$ is defined by $f\left(x\right)=x_1^2+x_1^2{x}_2+x_2^4$. If $F\left(x\right)=f^{\prime }\left(x\right)$, then $F\left(x\right)=\left(\begin{array}{c}2x_1+2x_1{x}_2\\ x_1^2+4x_2^3\end{array}\right)$, and for $\overline{x}={(0,0)}^T$, we have $F(0,0)={(0,0)}^T.$ It can be shown that F is 3-regular at $(0,0)$ along $h={(1,1)}^T$.

Namely, according to the previous definitions, in this example,

$${\overline{P}}_1=\left(\begin{array}{cc}0& 0\\ 0& 1\end{array}\right),{\overline{P}}_2={\displaystyle \frac{1}{2}}\left(\begin{array}{cc}1& -1\\ -1& 1\end{array}\right),$$

$$P_1={\overline{P}}_1+{\overline{P}}_2={\displaystyle \frac{1}{2}}\left(\genfrac{}{}{0ex}{}{1}{-1}\genfrac{}{}{0ex}{}{-1}{3}\right),P_2={\overline{P}}_2{\overline{P}}_1={\displaystyle \frac{1}{2}}\left(\genfrac{}{}{0ex}{}{0}{0}\genfrac{}{}{0ex}{}{-1}{1}\right).$$

Then, the following matrix is nonsingular:

$$F^{\prime }\left(0\right)+P_1{F}^{\prime \prime }\left(0\right)h+P_2{F}^{\left(3\right)}\left(0\right){\left[h\right]}^2=f^{\prime \prime }\left(0\right)+P_1{f}^{\left(3\right)}\left(0\right)h+P_2{f}^{\left(4\right)}\left(0\right){\left[h\right]}^2=\left(\genfrac{}{}{0ex}{}{2}{2}\genfrac{}{}{0ex}{}{-11}{11}\right).$$

Consider the 3-factor method:

$$\begin{array}{c}{x}^{k+1}=x^k-\hfill \\ {\left(f^{\prime \prime }\left(0\right)+P_1{f}^{\left(3\right)}\left(0\right)\left[h\right]+P_2{f}^{\left(4\right)}\left(0\right){\left[h\right]}^2\right)}^{-1}\left(f^{\prime }\left(x^k\right)+P_1{f}^{\prime \prime }\left(x^k\right)\left[h\right]+P_2{f}^{\left(3\right)}\left(x^k\right){\left[h\right]}^2\right).\hfill \end{array}$$

Let $x^k={(x_1,x_2)}^T.$ Then

$$\begin{array}{cc}\hfill \parallel x^{k+1}-0\parallel & =∥x^k-{\left(\begin{array}{cc}2& -11\\ 2& 11\end{array}\right)}^{-1}\left(\begin{array}{c}2x_1-11x_2+2x_1{x}_2-6x_2^2\\ 2x_1+11x_2+x_1^2+18x_2^2+4x_2^3\end{array}\right)∥=\hfill \\ \hfill & ={\displaystyle \frac{1}{44}}∥\left(\begin{array}{c}11x_1^2+132x_2^2+22x_1{x}_2+44x_2^3\\ 2x_1^2+48x_2^2-4x_1{x}_2+8x_2^3\end{array}\right)∥\le 10{\parallel x^k-0\parallel }^2.\hfill \end{array}$$

4.4. Optimality Conditions for Equality-Constrained Optimization Problems

In this section, we consider optimization problem (4):

$$minf\left(x\right)\mathrm{subject}\mathrm{to}F\left(x\right)=0,$$

where $f:X\to \mathbb{R}$ is a sufficiently smooth function and $F:X\to Y$ is a sufficiently smooth mapping from a Banach space X to a Banach space Y.

4.4.1. Optimality Conditions: Lagrange Multiplier Theorem

There is an extensive body of literature discussing optimality conditions for regular constrained optimization problems, which are problems that satisfy certain constraint qualifications. One notable reference on this topic is Chapter 3 of the book [47].

The classical optimality conditions state that if $\overline{x}$ is a regular solution of Problem (4), then there exists a Lagrange multiplier in the form of a constant vector $\overline{\lambda }\in Y^{*}$, such that

$$f^{\prime }\left(\overline{x}\right)=F^{\prime }{\left(\overline{x}\right)}^{*}\overline{\lambda },$$

(43)

where $F^{\prime }{\left(\overline{x}\right)}^{*}:Y^{*}\to X^{*}$ denotes the adjoint of $F^{\prime }\left(\overline{x}\right)$, and $X^{*}$ and $Y^{*}$ denote the dual spaces of X and Y, respectively.

The situation changes in the degenerate case when the derivative $F\prime \left(\overline{x}\right)$ is not surjective. In such cases, the classical optimality conditions in the form of Equation (43) do not hold, as illustrated in the following example.

Example 9. 

Consider the problem

$$\begin{array}{cc}\underset{x\in {\mathbb{R}}^3}{\mathrm{minimize}}\hfill & x_2^2+x_3\hfill \\ \mathrm{subject}\mathrm{to}\hfill & \left(\begin{array}{c}{x}_1^2-x_2^2+x_3^2\\ x_1^2-x_2^2+x_3^2+x_2{x}_3\end{array}\right)=\left(\genfrac{}{}{0ex}{}{0}{0}\right).\hfill \end{array}$$

(44)

Note that mapping $F\left(x\right)=\left(\begin{array}{c}{x}_1^2-x_2^2+x_3^2\\ x_1^2-x_2^2+x_3^2+x_2{x}_3\end{array}\right)$ was introduced in (28).

In this example, if $\overline{x}={(0,0,0)}^T$, then

$$f^{\prime }\left(\overline{x}\right)={(0,0,1)}^T,and{F}^{\prime }\left(\overline{x}\right)=\left(\genfrac{}{}{0ex}{}{0}{0}\genfrac{}{}{0ex}{}{0}{0}\genfrac{}{}{0ex}{}{0}{0}\right).$$

Hence $f^{\prime }\left(\overline{x}\right)\ne F^{\prime }{\left(\overline{x}\right)}^T\overline{\lambda }$ and Equation (43) does not hold.

4.4.2. Optimality Conditions for p-Regular Optimization Problems

In this section, we will focus on the case when the equality constraints defined by mapping $F\left(x\right)$ are not regular at a solution $\overline{x}$ of the problem (4). We define the p-factor-Lagrange function ${\mathcal{L}}_p(x,\lambda \left(h\right),h):X\times (Y_1^{*}\times \dots \times Y_p^{*})\times X\to \mathbb{R}$ as

$${\mathcal{L}}_p(x,\lambda \left(h\right),h)=f\left(x\right)+\sum _{i=1}^p⟨{\lambda }_i\left(h\right),f_i^{(i-1)}\left(x\right){\left[h\right]}^{i-1}⟩,$$

(45)

where $x,h\in X$, ${\lambda }_i\left(h\right)\in Y_i^{*}$ for $i=1,\dots ,p$, and the mappings $f_i\left(x\right)$ are defined in (16). Note that the p-factor-Lagrange function is a generalization of the classical Lagrange function and reduces to it in the regular case.

The development of optimality conditions for nonregular problems has become an active area of research (see [16,48,49,50,51] and references therein).

To state the sufficient conditions in Theorem 13, we also introduce an alternative version of the p-factor-Lagrange function, ${\overline{\mathcal{L}}}_p(x,\lambda \left(h\right),h:X\times (Y_1^{*}\times \dots \times Y_p^{*})\times X\to \mathbb{R},$ which is defined as follows:

$${\overline{\mathcal{L}}}_p(x,\lambda \left(h\right),h)=f\left(x\right)+\sum _{i=1}^p{\displaystyle \frac{2}{i(i+1)}}⟨{\lambda }_i\left(h\right),f_i^{(i-1)}\left(x\right){\left[h\right]}^{i-1}⟩.$$

(46)

To state optimality conditions for p-regular optimization problems, we use the definition of strong p-regularity at $\overline{x}$ given in Definition 7. We also use the set $H_p\left(\overline{x}\right)$ defined in Equation (23), and the operator ${\Psi }_p\left(h\right)$ defined in Equation (19).

Theorem 13 

([16], necessary and sufficient conditions for optimality). Assume that X and Y are Banach spaces, U is a neighborhood of a point $\overline{x}$ in X, $f:U\to \mathbb{R}$ is a twice continuously Fréchet differentiable function in U, and $F:U\to Y$ is a $(p+1)$-times Fréchet differentiable mapping in U.

Necessary conditions for optimality.

Assume that for an element $h\in H_p\left(\overline{x}\right)$, the set $Im{\Psi }_p\left(h\right)$ is closed in $Y_1\oplus \dots \oplus Y_p$. Suppose that F is p-regular at the point $\overline{x}$ along the vector $h\in H_p\left(\overline{x}\right)$. If $\overline{x}$ is a local minimizer of problem (4), then there exist multipliers $\overline{\lambda }\left(h\right)=({\overline{\lambda }}_1\left(h\right),\dots ,{\overline{\lambda }}_p\left(h\right))\in (Y_1^{*}\times \dots \times Y_p^{*})$ such that the partial derivative of the function ${\overline{\mathcal{L}}}_p$ with respect to x, denoted by ${\left({\mathcal{L}}_p^{\prime }\right)}_x$, satisfies

$${\left({\mathcal{L}}_p^{\prime }\right)}_x(\overline{x},\overline{\lambda }\left(h\right),h)=0.$$

(47)

Sufficient conditions for optimality.

Assume that the set $\mathrm{Im}{\Psi }_p\left(h\right)$ is closed in $Y_1\oplus \dots \oplus Y_p$ for every $h\in H_p\left(\overline{x}\right)$, and that $\mathrm{Im}{\Psi }_p\left(h\right)=Y_1\oplus \dots \oplus Y_p.$ Assume also that the mapping F is strongly p-regular at $\overline{x}.$ Suppose that there exist a constant $\alpha >0$ and a multiplier $\overline{\lambda }\left(h\right)$ such that Equation (47) is satisfied, and that the second-order partial derivative of the function ${\overline{\mathcal{L}}}_p$ (defined in (46)) with respect to x, denoted by ${\left({\overline{\mathcal{L}}}_p\right)}_{xx}$, satisfies

$${\left({\overline{\mathcal{L}}}_p\right)}_{xx}(\overline{x},\overline{\lambda }\left(h\right),h){\left[h\right]}^2\ge \alpha {\parallel h\parallel }^2$$

(48)

for every $h\in H_p\left(\overline{x}\right).$ Then $\overline{x}$ is a strict local minimizer of the problem Equation (4).

 Example 10. 

In this example, we continue with the analysis of problem (44) from Section 4.4.1. It can be verified that the point $\overline{x}=({\overline{x}}_1,{\overline{x}}_2,{\overline{x}}_3)={(0,0,0)}^T$ is a local minimizer of Equation (44). In Example 6, we showed that the mapping $F\left(x\right)=\left(\begin{array}{c}{x}_1^2-x_2^2+x_3^2\\ x_1^2-x_2^2+x_3^2+x_2{x}_3\end{array}\right)$ is 2-regular at $\overline{x}$ along the vector $h={(1,1,0)}^T$.

In this example, the 2-factor Lagrange function ${\mathcal{L}}_2$, defined in (45) for $p=2$, is given by

$${\mathcal{L}}_2(x,\lambda \left(h\right),h)=f\left(x\right)+⟨{\lambda }_1\left(h\right),f_1\left(x\right)⟩+⟨{\lambda }_2\left(h\right),f_2^{\prime }\left(x\right)\left[h\right]⟩,$$

where $\lambda \left(h\right)=({\lambda }_1\left(h\right),{\lambda }_2\left(h\right))$, ${\lambda }_1\left(h\right)={(0,0)}^T$, and ${\lambda }_2\left(h\right)={(\alpha ,\beta )}^T.$ Substituting the given expressions, we obtain the following form:

$${\mathcal{L}}_2(x,\lambda \left(h\right),h)=x_2^2+x_3+\alpha (x_1-x_2)+\beta (x_1-x_2+{\displaystyle \frac{1}{2}}{x}_3).$$

Solving the equation

$${\mathcal{L}}_{2x}^{\prime }(\overline{x},\lambda \left(h\right),h)=0,$$

we obtain the following system:

$$\begin{array}{ccc}\hfill \alpha +\beta & =& 0\hfill \\ \hfill 2{\overline{x}}_2-\alpha -\beta & =& 0\hfill \\ \hfill 1+{\displaystyle \frac{1}{2}}\beta & =& 0.\hfill \end{array}$$

Substituting ${\overline{x}}_2=0$, we obtain $\beta =-2$ and $\alpha =2$. Hence, the function ${\overline{\mathcal{L}}}_2(x,\lambda \left(h\right),h),$ defined in (46), takes the following form in this example:

$${\overline{\mathcal{L}}}_2(x,\lambda \left(h\right),h)=x_2^2+x_3+{\displaystyle \frac{2}{3}}(x_1-x_2)-{\displaystyle \frac{2}{3}}(x_1-x_2+{\displaystyle \frac{1}{2}}{x}_3)=x_2^2+{\displaystyle \frac{2}{3}}{x}_3.$$

Recall that the set $H_2\left(\overline{x}\right)$ was determined in Example 6 as

$$H_2\left(\overline{x}\right)={Ker}^2{F}^{\prime \prime }\left(\overline{x}\right)=span\left(\left[\begin{array}{c}1\\ -1\\ 0\end{array}\right]\right)\bigcup span\left(\left[\begin{array}{c}1\\ 1\\ 0\end{array}\right]\right).$$

Then, the second derivative of $\overline{\mathcal{L}}$, defined in (46) for $p=2$, satisfies

$${\left({\overline{\mathcal{L}}}_2^{\prime \prime }\right)}_{xx}(\overline{x},\lambda \left(h\right),h){\left[h\right]}^2=2\ge \alpha {\parallel h\parallel }^2$$

for some $\alpha >0,$ and for every $h\in H_2\left(\overline{x}\right)$. Hence, the sufficient conditions in Theorem 13 are satisfied, and we conclude that $\overline{x}$ is a strict local minimizer of problem Equation (44).

4.5. Modified Lagrangian Function Method

4.5.1. The Problem

Consider the following constrained optimization problem:

$$minf\left(x\right),\mathrm{subject}\mathrm{to}{g}_i\left(x\right)\le 0,i=1,\dots ,m,$$

(49)

where $f:{\mathbb{R}}^n\to \mathbb{R}$ is an objective function, and $g_i:{\mathbb{R}}^n\to \mathbb{R}$ are constraint functions. The goal is to find a vector $(\overline{x}\in {\mathbb{R}}^n$ such that $f\left(x\right)$ is minimized while satisfying all constraints. To solve this, we introduce the modified Lagrangian function $L_E(x,\lambda ):{\mathbb{R}}^n\times {\mathbb{R}}^m\to \mathbb{R}$, which incorporates both the objective function and the constraints (see, e.g., [45,52,53]):

$$L_E(x,\lambda )=f\left(x\right)+{\displaystyle \frac{1}{2}}\sum _{i=1}^m{\lambda }_i^2{g}_i\left(x\right),$$

(50)

where $\lambda =({\lambda }_1,\dots ,{\lambda }_m)$. This modified Lagrangian function transforms the nonlinear optimization problem into a system of nonlinear equations.

Define the mapping $G:{\mathbb{R}}^n\times {\mathbb{R}}^m\to {\mathbb{R}}^{n+m}$ by

$$G(x,\lambda )=\left(\begin{array}{c}\nabla f\left(x\right)+{\displaystyle \frac{1}{2}}{\displaystyle \sum _{i=1}^m}{\lambda }_i^2\nabla g_i\left(x\right)\\ D\left(\lambda \right)g\left(x\right)\end{array}\right),$$

(51)

where $D\left(\lambda \right)=\mathrm{diag}\left\{{\lambda }_i\right\},$$i=1,\dots ,m$, and $\lambda \in {\mathbb{R}}^m$.

Consider the equation

$$G(x,\lambda )=0.$$

(52)

Let $g^{\prime }\left(x\right)$ be the Jacobian matrix of the mapping $g\left(x\right)$. Then, the Jacobian matrix $G^{\prime }(x,\lambda )$ of the mapping $G(x,\lambda )$ is given by

$$G^{\prime }(x,\lambda )=\left(\begin{array}{cc}{\nabla }^{2}f\left(x\right)+{\displaystyle \frac{1}{2}}{\displaystyle \sum _{i=1}^m}{\lambda }_i^2{g}_i^{\prime \prime }\left(x\right)& {\left(g^{\prime }\left(x\right)\right)}^{T}D\left(\lambda \right)\\ D\left(\lambda \right){\left(g^{\prime }\left(x\right)\right)}^T& D\left(g\right(x\left)\right)\end{array}\right).$$

(53)

Define the set $I\left(\overline{x}\right)=\{j=1,\dots ,m\mid g_j\left(\overline{x}\right)=0\}$ consisting of active constraints, and the set

$$I_0\left(\overline{x}\right)=\{j=1,\dots ,m\mid {\overline{\lambda }}_j=0,g_j\left(\overline{x}\right)=0\}\subset I\left(\overline{x}\right),$$

consisting of weakly active constraints, and the set $I_{+}\left(\overline{x}\right)=I\left(\overline{x}\right)\setminus I_0\left(\overline{x}\right)$, consisting of strongly active constraints.

Recall that the Strict Complementary Condition (SCQ) means that, for each index $j=1,\dots ,m$, one and only one of $g_j\left(\overline{x}\right)$ and ${\overline{\lambda }}_j$ is equal to zero. If $(\overline{x},\overline{\lambda })$ is a solution of Problem (52), and for some index j, both $g_i\left(\overline{x}\right)=0$ and ${\overline{\lambda }}_i=0$, then the set $I_0\left(\overline{x}\right)$ is nonempty, and the SCQ fails. Consequently, $G^{\prime }(\overline{x},\overline{\lambda })$ is a degenerate matrix. Example 11 illustrates this situation.

Example 11 

([45]). Consider the problem

$$\underset{x\in {\mathbb{R}}^n}{min}(x_1^2+x_2^2+4x_1{x}_2)\mathit{subject}\mathit{to}{x}_1\ge 0,x_2\ge 0.$$

(54)

A direct argument confirms that $\overline{x}={(0,0)}^T$ is a solution of Problem (54) with the corresponding Lagrange multiplier $\overline{\lambda }={(0,0)}^T$.

The modified Lagrange function in this case is

$$L_E(x,\lambda )=x_1^2+x_2^2+4x_1{x}_2-{\displaystyle \frac{1}{2}}{\lambda }_1^2{x}_1-{\displaystyle \frac{1}{2}}{\lambda }_2^2{x}_2.$$

The mapping G is defined by

$$G(x,\lambda )=\left(\begin{array}{c}2x_1+4x_2-{\displaystyle \frac{1}{2}}{\lambda }_1^2\\ 2x_2+4x_1-{\displaystyle \frac{1}{2}}{\lambda }_2^2\\ -{\lambda }_1{x}_1\\ -{\lambda }_2{x}_2\end{array}\right),$$

and, therefore, the Jacobian matrix $G^{\prime }(\overline{x},\overline{\lambda })$ defined in (53) is singular.

4.5.2. Modified Lagrange Function Method for 2-Regular Problems

In this section, we consider the constrained optimization problem (49) with the modified Lagrangian function $L_E(x,\lambda )$ defined in (50). We focus on the nonregular case when the Jacobian matrix $G^{\prime }(\overline{x},\overline{\lambda })$ defined in (53) is singular at the solution $(\overline{x},\overline{\lambda })$ of (52).

We will show that the mapping $G(x,\lambda )$ defined in (51) is 2-regular at $(\overline{x},\overline{\lambda })$.

Without loss of generality, assume that $I_0\left(\overline{x}\right)=\{1,\dots ,s\}$, so that ${\overline{\lambda }}_j=0$ and $g_j\left(\overline{x}\right)=0$ for all $j=1,\dots ,s$. Additionally, we assume that $I_{+}\left(\overline{x}\right)=\{s+1,s+2,\dots ,p\}$. Introduce the notation $l=m-p$. Then, the rows of matrix $G^{\prime }(\overline{x},\overline{\lambda })$ with the numbers from the $(n+1)$th to the $(n+s)$th contain only zeros. Define the vector $h\in {\mathbb{R}}^{n+m}$ as follows

$$h^T=\left(0_n^T,1_s^T,0_{m-s}^T\right),$$

(55)

where $1_s^T$ is an s-dimensional all-one row vector.

Let the mapping $\Phi :{\mathbb{R}}^n\times {\mathbb{R}}^m$ be given by

$$\Phi (x,\lambda )=G(x,\lambda )+G^{\prime }(x,\lambda )h,$$

(56)

where h is defined in (55).

The following result is well known.

Lemma 3 

([45]). Let an $n\times n$ matrix V and an $n\times p$ matrix Q satisfy the properties:

 1. 

Q has linearly independent columns, and

 2. 

$x^{T}Vx>0$ for all $x\in Ker{Q}^T\setminus \left\{0\right\}$.

Assume also that $D_N$ is a full-rank diagonal $l\times l$ matrix. Then, the matrix $\overline{A}$ defined by

$$\overline{A}=\left(\begin{array}{ccc}V& Q& 0\\ Q^T& 0& 0\\ 0& 0& D_N\end{array}\right)$$

(57)

is a nonsingular matrix.

The Linear Independence Constraint Qualification (LICQ) holds for the optimization problem (49) if the gradients of active constraints are linearly independent.

The second-order sufficient optimality condition holds if there exists $\alpha >0$ such that

$$z^T{\nabla }_{xx}^2{L}_E(\overline{x},\overline{\lambda })z\ge \alpha {\parallel z\parallel }^2$$

(58)

for all $z\in {\mathbb{R}}^n$ that satisfy the conditions

$${(\nabla g_j\left(\overline{x}\right))}^{T}z\le 0\forall j\in I\left(\overline{x}\right).$$

Lemma 4 

([45]). Let $f,g_i\in {\mathcal{C}}^3\left({\mathbb{R}}^n\right)$, for $i=1,\dots ,m$. Assume that the LICQ (Linear Independence Constraint Qualification) and the second-order sufficient optimality conditions are satisfied at the solution $(\overline{x},\overline{\lambda })$ of (52), and that Φ is a mapping given by Equation (56). Then, the 2-factor operator

$${\Phi }^{\prime }(x,\lambda )=G^{\prime }(x,\lambda )+G^{\prime \prime }(x,\lambda )h$$

is nonsingular at the point $(\overline{x},\overline{\lambda })$.

The proof of Lemma 4 can be derived from Lemma 3.

Indeed, if $D\left(\lambda \right)$ is a diagonal matrix with ${\lambda }_j$ as the j-th diagonal entry,

$$V={\nabla }_{xx}^2{L}_E(\overline{x},\overline{\lambda }),D_N=D\left(g_N\left(\overline{x}\right)\right),g_N\left(x\right)={\left(g_{p+1}\left(x\right),\dots ,g_m\left(x\right)\right)}^T,$$

and

$$Q=\left[\nabla g_1\left(\overline{x}\right),\dots ,\nabla g_s\left(\overline{x}\right),{\overline{\lambda }}_{s+1}\nabla g_{s+1}\left(\overline{x}\right),\dots ,{\overline{\lambda }}_p\nabla g_p\left(\overline{x}\right)\right],$$

then ${\Phi }^{\prime }(\overline{x},\overline{\lambda })=\overline{A}$, where matrix $\overline{A}$ is defined in (57). Lemma 4 implies that the 2-factor Newton method is given by

$$w^{k+1}=w^k-{\left(G^{\prime }\left(w^k\right)+G^{\prime \prime }\left(w^k\right)h\right)}^{-1}\left(G\left(w^k\right)+G^{\prime }\left(w^k\right)h\right),k=0,1,\dots ,$$

(59)

and it can be applied to solve the system (52), where G is defined in (51). As a result, we have the following theorem.

Theorem 14 

([45]). Let $\overline{x}$ be a solution to (49) and $f,g_i\left(x\right)\in {\mathcal{C}}^3\left({\mathbb{R}}^n\right)$, for $i=1,\dots ,m$. Assume that the LICQ and the second-order sufficient optimality conditions (58) are satisfied at the point $\overline{x}$. Then, there exists a sufficiently small open ball $B(\overline{w},\epsilon )$, where $\overline{w}=(\overline{x},\overline{\lambda })$, such that the estimate

$$\parallel w^{k+1}-\overline{w}\parallel \le \beta \parallel w^k-\overline{w}{\parallel }^2,$$

holds for the method (59), where $w^0\in B(\overline{w},\epsilon )$ and $\beta >0$ is a constant independent of k.

In addition, there are other publications where a modified Lagrange function is used in various contexts, such as [54,55]. Higher-order analysis of optimality conditions has been performed in [56].

4.6. Calculus of Variations

The methods of the calculus of variations are widely used to solve many problems in physics and classical mechanics. However, since the classical approach cannot be directly applied to many of these problems, there is a need to extend or reformulate classical theorems to accommodate irregular cases. Over the years, various types of irregular problems in the calculus of variations have been extensively studied in both mathematics and its applications (see, e.g., [1,3,32,57,58,59,60,61,62]).

4.6.1. Singular Problems of Calculus of Variations

In this section, we consider the following Lagrange problem, which involves finding a curve $x=x\left(t\right)$, such that (see [63]):

$$J_0\left(x\right)={\int }_{t_1}^{t_2}f(t,x,\dot{x})dt\to min$$

(60)

subject to the subsidiary conditions:

$$\Gamma \left(x\right)=0,q(x\left(t_1\right),x\left(t_2\right))=0,$$

(61)

where

$${\displaystyle \Gamma \left(x\right)\left(t\right)=\mathcal{G}(t,x\left(t\right),\dot{x}\left(t\right))=0forallt\in [t_1,t_2],\dot{x}=\frac{dx}{dt},}$$

$$X=C_n^1\left([t_1,t_2]\right),Y=C_m\left([t_1,t_2]\right),x\left(t\right)\in X,\Gamma \in {\mathcal{C}}^{p+1}(X,Y),$$

$$\mathcal{G}:\mathbb{R}\times {\mathbb{R}}^n\times {\mathbb{R}}^n\to {\mathbb{R}}^m,\mathcal{G}(t,x\left(t\right),\dot{x}\left(t\right))=({\mathcal{G}}_1(t,x\left(t\right),\dot{x}\left(t\right)),\dots ,{\mathcal{G}}_m(t,x\left(t\right),\dot{x}\left(t\right)),$$

and

$$q:{\mathbb{R}}^n\times {\mathbb{R}}^n\to {\mathbb{R}}^k,f:\mathbb{R}\times {\mathbb{R}}^n\times {\mathbb{R}}^n\to \mathbb{R}.$$

We assume that all mappings and their partial derivatives are continuous with respect to $t,$$x,$ and $\dot{x}$. We denote by $\overline{x}\left(t\right)$ a solution to Problem (60) and (61). While each of x, $\dot{x}$, and each component of x is a function of t, (e.g., $x=x\left(t\right)$, and $\dot{x}=\dot{x}\left(t\right)$), we do not write this explicitly in order to avoid over complicated notation.

In the regular case, when $Im{\Gamma }^{\prime }\left(\overline{x}\right)=Y,$ the Euler–Lagrange necessary conditions are satisfied and take the form (see, e.g., [60,64]):

$$f_x+\lambda \left(t\right){\mathcal{G}}_x-{\displaystyle \frac{d}{dt}}(f_{\dot{x}}+\lambda \left(t\right){\mathcal{G}}_{\dot{x}})=0\mathrm{for}\mathrm{all}t\in [t_1,t_2].$$

(62)

Let $\lambda \left(t\right)={({\lambda }_1\left(t\right),\dots ,{\lambda }_m\left(t\right))}^T.$ Then

$$\lambda \left(t\right)\mathcal{G}={\lambda }_1\left(t\right){\mathcal{G}}_1+\dots +{\lambda }_m\left(t\right){\mathcal{G}}_{m}and\lambda \left(t\right){\mathcal{G}}_x={\lambda }_1\left(t\right){\mathcal{G}}_{1x}+\dots +{\lambda }_m\left(t\right){\mathcal{G}}_{mx}.$$

In the singular case, when $Im{\Gamma }^{\prime }\left(\overline{x}\right)\ne Y,$ we can only guarantee that the following equation is satisfied:

$${\lambda }_0{f}_x+\lambda \left(t\right){\mathcal{G}}_x-{\displaystyle \frac{d}{dt}}({\lambda }_0{f}_{\dot{x}}+\lambda \left(t\right){\mathcal{G}}_{\dot{x}})=0,$$

(63)

where ${\lambda }_0^2+{\parallel \lambda \left(t\right)\parallel }^2=1.$ In this case, ${\lambda }_0$ might be equal to 0, which results in no constructive conditions for the description or finding $\overline{x}\left(t\right)$.

Example 12 

([63]). Consider the following problem of finding a curve $x\left(t\right)=(x_1\left(t\right),\dots ,x_5\left(t\right))$ such that

$$J_0\left(x\right)={\int }_0^{2\pi }(x_1^2+x_2^2+x_3^2+x_4^2+x_5^2)\mathrm{d}t\to min$$

(64)

subject to

$$\Gamma \left(x\right)=\left(\begin{array}{c}\dot{x_1}-x_2+x_3^2{x}_1+x_4^2{x}_2-x_5^2(x_1+x_2)\\ \dot{x_2}+x_1+x_3^2{x}_2-x_4^2{x}_1-x_5^2(x_2-x_1)\end{array}\right)=0,$$

(65)

$$x_i\left(0\right)=x_i\left(2\pi \right),i=1,\dots ,5,$$

where $\Gamma :C_5^2\left([0,2\pi ]\right)\to C_2\left([0,2\pi ]\right),$ ${\displaystyle {\dot{x}}_1=\frac{d{x}_1}{dt}}$, and ${\displaystyle {\dot{x}}_2=\frac{d{x}_2}{dt}.}$

Here,

$$f\left(x\right)=x_1^2+x_2^2+x_3^2+x_4^2+x_5^2,and{q}_i(x\left(0\right),x\left(2\pi \right))=x_i\left(0\right)-x_i\left(2\pi \right),i=1,\dots ,5.$$

The solution of Problem (64) and (65) is $\overline{x}\left(t\right)=0$ and ${\Gamma }^{\prime }\left(0\right)$ is singular. Indeed, using the differentiation rules in functional spaces, we obtain

$${\Gamma }^{\prime }\left(0\right)=\left(\begin{array}{ccccc}{\dot{(·)}}_1& -{(·)}_2& 0& 0& 0\\ {(·)}_1& {\dot{(·)}}_2& 0& 0& 0\end{array}\right)and{\Gamma }^{\prime }\left(0\right)x=\left(\begin{array}{c}{\dot{x}}_1-x_2\\ x_1+{\dot{x}}_2\end{array}\right).$$

Introducing the notation $z\left(t\right)=x_1\left(t\right)$ and using the methods of differential equations, one can show that the mapping $\Gamma \left(z\left(t\right)\right)=z^{\prime \prime }\left(t\right)+z\left(t\right),$ with boundary conditions $z\left(0\right)=z\left(2\pi \right),$ is not surjective. Indeed, for $y\in \mathcal{C}[0,2\pi ],$ satisfying

$${\int }_0^{2\pi }sin\tau y\left(\tau \right)\mathrm{d}\tau \ne 0or{\int }_0^{2\pi }cos\tau y\left(\tau \right)\mathrm{d}\tau \ne 0,$$

the equation $z^{\prime \prime }\left(t\right)+z\left(t\right)=y\left(t\right)$ does not have a solution.

With $z=z\left(t\right)$, the corresponding Euler–Lagrange equations in this case are as follows:

$$\begin{array}{ccc}\hfill 2{\lambda }_{0}z+{\lambda }_2-{\dot{\lambda }}_1+{\lambda }_1{x}_3^2+{\lambda }_2{x}_5^2-{\lambda }_2{x}_4^2& =& 0,\hfill \\ \hfill 2{\lambda }_0{x}_2-{\lambda }_1-{\dot{\lambda }}_2+{\lambda }_1{x}_4^2+{\lambda }_2{x}_3^2-{\lambda }_1{x}_5^2-{\lambda }_2{x}_5^2& =& 0,\hfill \\ \hfill 2{\lambda }_0{x}_3+2{\lambda }_{1}z{x}_3+2{\lambda }_2{x}_2{x}_3& =& 0,\hfill \\ \hfill 2{\lambda }_0{x}_4+2{\lambda }_1{x}_2{x}_4-{\lambda }_{2}z{x}_4& =& 0,\hfill \\ \hfill 2{\lambda }_0{x}_5-2{\lambda }_1{x}_{5}z-2{\lambda }_1{x}_2{x}_5-2{\lambda }_2{x}_2{x}_5+2{\lambda }_{2}z{x}_5& =& 0,\hfill \\ \hfill {\lambda }_i\left(0\right)={\lambda }_i\left(2\pi \right),i=1,2.& \end{array}$$

Unfortunately, we cannot guarantee that ${\lambda }_0\ne 0$. For ${\lambda }_0=0$, we obtain a series of spurious solutions to the problem (64) and (65):

$$z\left(t\right)=asint,x_2=acost,x_3=x_4=x_5=0,{\lambda }_1=bsint,{\lambda }_2=bcost,a,b\in \mathbb{R}.$$

(66)

The derivation of the solutions (66) is based on standard techniques, so we are omitting the technical details from the paper.

4.6.2. Optimality Conditions for p-Regular Problems of Calculus of Variations

To formulate optimality conditions for the problem (60) and (61) in the singular case, we define the p-factor Euler–Lagrange function by

$$E\left(x\right)=f\left(x\right)+\lambda \left(t\right){\Gamma }^{(p-1)}\left(x\right){\left[h\right]}^{p-1},$$

where

$${\Gamma }^{(p-1)}\left(x\right){\left[h\right]}^{p-1}=g_1\left(x\right)+g_2^{\prime }\left(x\right)\left[h\right]+\dots +g_p^{(p-1)}\left(x\right){\left[h\right]}^{p-1},$$

$$\lambda \left(t\right){\Gamma }^{(p-1)}\left(x\right){\left[h\right]}^{p-1}=⟨\lambda \left(t\right),\left(g_1\left(x\right)+g_2^{\prime }\left(x\right)\left[h\right]+\dots +g_p^{(p-1)}\left(x\right){\left[h\right]}^{p-1}\right)⟩,$$

$$\lambda \left(t\right)={({\lambda }_1\left(t\right),\dots ,{\lambda }_m\left(t\right))}^T,h=h\left(t\right)\in X.$$

Functions $g_i\left(x\right),$$i=1,\dots ,p$, are determined for the mapping $\Gamma \left(x\right)$ in a way that is similar to how functions $f_i\left(x\right),$$i=1,\dots ,p,$ are defined for the mapping $F\left(x\right),$ in Equation (16). Namely,

$$g_k\left(x\right)=P_{Y_k}\Gamma \left(x\right),k=1,\dots ,p.$$

Let

$$g_k^{(k-1)}\left(x\right){\left[h\right]}^{k-1}=\sum _{i+j=k-1}{C}_{k-1}^i{g_k^{(k-1)}}_{x^i{\left(\dot{x}\right)}^j}\left(x\right){\left[h\right]}^i{\left[h^{\prime }\right]}^j,k=1,\dots ,p,$$

where ${g_k^{(k-1)}}_{x^i{\left(\dot{x}\right)}^j}\left(x\right)={g_k^{(k-1)}}_{\underset{i}{\underbrace{x\dots x}}\underset{j}{\underbrace{\dot{x}\dots \dot{x}}}}\left(x\right).$

Definition 10. 

Let $X=C_n^1\left([t_1,t_2]\right)$ and $Y=C_m\left([t_1,t_2]\right)$. We say that problem (60) and (61) is p-regular at $\overline{x}\left(t\right)\in X$ along some vector $h\left(t\right)\in X$, $h\left(t\right)\in \bigcap _{k=1}^p{Ker}^k{g}_k^{\left(k\right)}\left(\overline{x}\left(t\right)\right),$$\parallel h\left(t\right)\parallel \ne 0$, if

$$Im\left(g_1^{\prime }\left(\overline{x}\left(t\right)\right)+\dots +g_p^{\left(p\right)}\left(\overline{x}\left(t\right)\right){\left[h\left(t\right)\right]}^{p-1}\right)=Y.$$

Theorem 15 

([63]). Assume that the problem (60) and (61) is p-regular at its solution $\overline{x}\left(t\right)\in X$ along $h=h\left(t\right)\in X$, $h\in \bigcap _{k=1}^p{Ker}^k{g}_k^{\left(k\right)}\left(\overline{x}\left(t\right)\right).$ Then, there exists a multiplier $\widehat{\lambda }\left(t\right)={({\widehat{\lambda }}_1\left(t\right),\dots ,{\widehat{\lambda }}_m\left(t\right))}^T$ such that the following p-factor Euler–Lagrange equation holds:

$$\begin{array}{c}{E}_x\left(\overline{x}\left(t\right)\right)-{\displaystyle \frac{d}{dt}}{E}_{\dot{x}}\left(\overline{x}\left(t\right)\right)=f_x\left(\overline{x}\left(t\right)\right)+{⟨\widehat{\lambda }\left(t\right),\sum _{k=1}^p\sum _{i+j=k-1}{C}_{k-1}^i{g}_{x^i{\left(\dot{x}\right)}^j}^{(k-1)}\left(\overline{x}\left(t\right)\right){\left[h\right]}^i{\left(\dot{h}\right)}^j⟩}_x\\ -{\displaystyle \frac{d}{dt}}\left[f_{\dot{x}}\left(\overline{x}\left(t\right)\right)+{⟨\widehat{\lambda }\left(t\right),\sum _{k=1}^p\sum _{i+j=k-1}{C}_{k-1}^i{g}_{x^i{\left(\dot{x}\right)}^j}^{(k-1)}\left(\overline{x}\left(t\right)\right){\left[h\right]}^i{\left(\dot{h}\right)}^j⟩}_{\dot{x}}\right]=0,\\ {\lambda }_i\left(0\right)={\lambda }_i\left(2\pi \right),i=1,2.\end{array}$$

(67)

The proof of Theorem 15 is similar to the one for the singular isoperimetric problem in [65].

We now go back to Example 12 for further consideration. The mapping $\Gamma$ is 2-regular at $\overline{x}\left(t\right)={(asint,acost,0,0,0)}^T$ along $h\left(t\right)={(asint,acost,1,1,1)}^T$. This means that in this problem $p=2$.

Consider the following equation

$$f_x\left(x\right)+{({\Gamma }^{\prime }\left(x\right)+P_{Y_2}{\Gamma }^{\prime \prime }\left(x\right)h)}^{*}\lambda \left(t\right)=0.$$

The equation is equivalent to the system of Euler–Lagrange equations

$$\left\{\begin{array}{c}2x_1-{\dot{\lambda }}_1+{\lambda }_2=0\hfill \\ 2x_2-{\dot{\lambda }}_2-{\lambda }_1=0\hfill \\ 2x_3+2{\lambda }_{1}asint+2{\lambda }_{2}acost=0\hfill \\ 2x_4+2{\lambda }_{1}acost-2{\lambda }_{2}asint=0\hfill \\ 2x_5+2{\lambda }_{1}a(cost-sint)+2{\lambda }_{2}a(sint-cost)=0.\hfill \\ {\lambda }_i\left(0\right)={\lambda }_i\left(2\pi \right),i=1,2.\hfill \end{array}\right.$$

(68)

One can verify that the following “false solutions” of (64) and (65),

$$x_1=asint,x_2=acost,x_3=x_4=x_5=0,$$

do not satisfy the system (68) if $a\ne 0.$ This implies that

$$x_1=asint,x_2=acost,x_3=x_4=x_5$$

are not solutions to the two-factor Euler–Lagrange Equation (67) from Theorem 15. Therefore, the only solution to Example 12 is $\overline{x}\left(t\right)={(0,0,0,0,0)}^T.$ Indeed, the two-factor Euler–Lagrange equation in this case has the following form:

$$\left\{\begin{array}{c}-{\dot{\lambda }}_1+{\lambda }_2=0\hfill \\ -{\dot{\lambda }}_2-{\lambda }_1=0\hfill \\ 2{\lambda }_{1}asint+2{\lambda }_{2}acost=0\hfill \\ 2{\lambda }_{1}acost-2{\lambda }_{2}asint=0\hfill \\ 2{\lambda }_{1}a(cost-sint)+2{\lambda }_{2}a(sint-cost)=0.\hfill \\ {\lambda }_i\left(0\right)={\lambda }_i\left(\pi \right),i=1,2.\hfill \end{array}\right.$$

This system has the solution $\overline{x}\left(t\right)={(0,0,0,0,0)}^T$ and ${\overline{\lambda }}_i\left(t\right)=0$, $i=1,2.$

4.7. Existence of Solutions to Nonlinear Equations

This section addresses the existence of a solution to an equation of the form (3), $F\left(x\right)=0$, in the neighbourhood of a chosen point $\overline{x}$. A very general setting is considered, where the function F maps from a Banach space X to a Banach space Y, and the assumptions pertain to the properties of its derivatives in the neighborhood under consideration. This is one of the classical problems of nonlinear analysis, with many important applications, especially in the theory of differential equations (cf. [66,67,68]).

One well-known method for addressing this problem is Newton’s method (see [69]). The solution is obtained as the limit of a recursively defined sequence of approximations. This method is applied in the proof of the first theorem in this section. In particular, the existence of the inverse operator to the derivative of the function at a chosen point is assumed.

The next theorem presented is more general and uses the p-factor construction of the operator for functions of class ${\mathcal{C}}^{p+1}$. A certain limitation of this construction is the assumption of the existence of continuous projections onto subspaces of Y corresponding to successive orders of the derivatives of the function F.

4.7.1. Existence of Solutions to Nonlinear Equations in the Regular Case

Let X and Y be Banach spaces. Consider a mapping $F:X\to Y$ and a problem of existence of a point $\overline{x}$ such that $F\left(\overline{x}\right)=0.$ We know that equation $F\left(x\right)=0$ is solvable and has a solution $\overline{x}$ when the operator $F^{\prime }\left(x_0\right)$ is surjective [27,70]. A modified version of the following theorem was given in [70].

Theorem 16. 

Let X and Y be Banach spaces, and let $x_0\in X$, and let $0<\epsilon <1.$ Assume $F\in {\mathcal{C}}^2\left(B(x_0,\epsilon )\right)$ and $∥F\left(x_0\right)∥=\eta$ for some constant $\eta >0$. Suppose that the derivative $F^{\prime }\left(x_0\right)$ is invertible and there exist constants $\delta >0$ and $C>0$ such that $∥{\left(F^{\prime }\left(x_0\right)\right)}^{-1}∥=\delta ,$$\underset{x\in B(x_0,\epsilon )}{sup}∥F^{\prime \prime }\left(x\right)∥=C<+\infty$. If, moreover, the following conditions are satisfied:

 1. 

$\delta \eta \le {\displaystyle \frac{\epsilon }{2}},$

 2. 

$\delta C\epsilon \le {\displaystyle \frac{1}{4}},$

 3. 

$C\epsilon \le {\displaystyle \frac{1}{2}}$,

then the equation $F\left(x\right)=0$ has a solution $\overline{x}\in B(x_0,\epsilon ).$

If the first derivative of F at $x_0$ is not surjective, then Theorem 16 cannot be applied. Consider, for example, a mapping $F:\mathbb{R}\to \mathbb{R}$ defined by

$$F\left(x\right)={\displaystyle \frac{1}{7!}}{x}^7+x^5+{\displaystyle \frac{1}{{10}^3}}.$$

Note that if $x_0=0$, the assumptions of Theorem 16 are not satisfied, but the equation $F\left(x\right)=0$ still has a solution $\overline{x}\approx -0.251188$.

4.7.2. Existence of Solutions to Nonlinear Equations in the Singular Case

In this section, we continue considering the problem introduced in Section 4.7.1. Specifically, let X and Y be Banach spaces, and let $F:X\to Y$. Assume that $F\left(x_0\right)\ne 0$ for some $x_0$. We are interested in the existence of a solution $\overline{x}$ to the equation $F\left(x\right)=0$ in some open ball $B(x_0,\epsilon )$ of $x_0$ such that $F\left(\overline{x}\right)=0$. Most of the work in solving this problem focuses on Newton’s method or its modifications, under the assumption that $F^{\prime }\left(x_0\right)$ is regular (see, e.g., [71]).

Now, consider the degenerate case where $F^{\prime }\left(x_0\right)$ is not regular. The focus here is on finding a small constant $\epsilon >0$ such that the neighborhood $B(x_0,\epsilon )$ contains a solution $\overline{x}$ to the equation $F\left(x\right)=0$. We introduce the following notation and assumptions for some $p\ge 2$:

$$\delta =∥F\left(x_0\right)∥,$$

(69)

$$\eta =∥{\left\{{\Psi }_p\left(h\right)\right\}}^{-1}∥<\infty ,h\in \bigcap _{k=1}^p{Ker}^k{f}_k^{\left(k\right)}\left(x_0\right),\parallel h\parallel =1,$$

(70)

$$c=\underset{k=1,\dots ,p}{max}\underset{x\in B(x_0,\epsilon )}{sup}∥f_k^{(k+1)}\left(x\right)∥,d=4\underset{k=1,\dots ,p}{max}{\displaystyle \frac{1}{(k-1)!}}∥f_k^{\left(k\right)}\left(x_0\right)∥,$$

$$\alpha =min\left\{{\displaystyle \frac{3}{4^{p+2}\eta }},\underset{k=1,\dots ,p}{min}{\displaystyle \frac{∥f_k^{\left(k\right)}\left(x_0\right)∥}{(k-1)!}}\right\}.$$

(71)

The following theorem was proved in [72].

Theorem 17. 

Let X and Y be Banach spaces, and let $F:X\to Y$ be of class ${\mathcal{C}}^{p+1}\left(X\right).$ Assume that there exists $h\in \bigcap _{k=1}^p{Ker}^k{f}_k^{\left(k\right)}\left(x_0\right),$ with $\parallel h\parallel =1$, such that F is a p-regular mapping at $x_0\in X$ along $h.$

Assume also that there exists $\omega ,$$0<\omega <{\displaystyle \frac{1}{2}}\nu ,$ where $\nu \in (0,1),$ such that the following inequalities hold:

 1. 

$\eta \delta \le \alpha {\displaystyle \frac{{\omega }^p}{2pd}},$

 2. 

${\displaystyle \frac{4^{p+2}}{3}}c\omega \eta \le {\displaystyle \frac{1}{2}}.$

Then the equation $F\left(x\right)=0$ has a solution $\overline{x}=x_0+\omega h+\overline{x}\left(\omega \right)\in B_{\nu }\left(x_0\right)$, where $\overline{x}\left(\omega \right)$ is a fixed point such that $\parallel \overline{x}\left(\omega \right)\parallel \le {\displaystyle \frac{1}{2}}\omega .$

Recall that if our focus is on finding a radius $\epsilon >0$ such that the open ball $B(x_0,\epsilon )$ contains a solution $\overline{x}$ of $F\left(x\right)=0$, then Theorem 17 implies that $\epsilon =\omega +\overline{x}\left(\omega \right)$. For example, we can take $\epsilon ={\displaystyle \frac{3}{2}}\omega .$

As an example of singular nonlinear equation, we consider the problem of existence of local nontrivial solutions of the Boundary Value Problem (BVP) for the ordinary differential equation

$$y^{\prime \prime }\left(t\right)+y\left(t\right)+g\left(y\left(t\right)\right)=x\left(t\right)$$

(72)

with the boundary conditions

$$y\left(0\right)=y\left(\pi \right)=0,$$

(73)

which is degenerate at $\overline{y}\left(t\right)=0$. Here, $y\left(t\right)\in {\mathcal{C}}^2\left([0,\pi ]\right)$ and $g,x$ are given functions such that

$$x\in \mathcal{C}[0,\pi ],g\in {\mathcal{C}}^{p+1}\left([0,\pi ]\right),g\left(0\right)=g^{\prime }\left(0\right)=0.$$

Remark 4. 

Recall that the operator ${\Psi }_p$ is defined in (19). The surjectivity of the operator ${\Psi }_p\left(\omega h\right)$ for any $\omega \ne 0$ implies the p-regularity condition of the mapping F at the point $x_0$ (by the definition). It is also equivalent to the following inequality with a vector h such that $\parallel h\parallel =1$:

$$\parallel {\left\{{\Psi }_p\left(\omega h\right)\right\}}^{-1}y\parallel \le \left(1+{\displaystyle \frac{1}{\omega }}+{\displaystyle \frac{1}{w^2}}+\dots +{\displaystyle \frac{1}{{\omega }^{p-1}}}\right).$$

4.8. Differential Equations

4.8.1. Nonlinear Boundary-Value Problem

The nonlinear BVP analyzed in this section has the form

$$y^{\prime \prime }\left(t\right)+y\left(t\right)+g\left(y\left(t\right)\right)=x\left(t\right)$$

(74)

with boundary conditions

$$y\left(0\right)=y\left(\pi \right)=0,$$

(75)

where $y\left(t\right)\in C^2[0,\pi ]$, $x\left(t\right)\in C[0,\pi ]$, and g is a $C^3$ function from $\mathbb{R}$ to $\mathbb{R}$, satisfying

$$g\left(0\right)=g^{\prime }\left(0\right)=0,x\left(0\right)=x\left(\pi \right)=0.$$

(76)

We are interested in the problem of the existence of a solution $y\left(t\right)$ to the BVP (74) and (75) for given functions $x\left(t\right)$ and $g\left(t\right)$.

Introduce the notation

$$F(x,y)=y^{\prime \prime }+y+g\left(y\right)-x,$$

(77)

and regard F as a mapping $F:X\times Y\to Z,$ where

$$X=\{x\in C[0,\pi ]\mid x\left(0\right)=x\left(\pi \right)=0\},Y=\{y\in C^2[0,\pi ]\mid y\left(0\right)=y\left(\pi \right)=0\},$$

and $Z=C[0,\pi ].$ We can rewrite Equation (74) as

$$F(x,y)=0.$$

(78)

The assumptions (75) and (76) imply that $(0,0)$ is a solution of (78): $F(0,0)=0$. Without loss of generality, we restrict our attention to a neighborhood $U\times V\subset X\times Y$ of the point $(0,0)$. The problem of existence of a solution $y\left(t\right)$ to the BVP (74) and (75) for a given function $x\left(t\right)\in U$ is equivalent to the problem of existence of an implicit function $\phi \left(x\right):U\to Y$, such that $y=\phi \left(x\right)$ and

$$F(x,y)=y^{\prime \prime }+y+g\left(y\right)-x=0,y\left(0\right)=y\left(\pi \right)=0.$$

(79)

If $F(0,0)=0$ and the mapping F is regular at $(0,0)$—that is, if the partial derivative of F with respect to y, denoted $F_y^{\prime }(0,0)$, is a surjective linear operator—then the classical IFT 9 guarantees the existence of a smooth mapping $\phi$ defined on a neighborhood of $\overline{x}=0$ such that $F(x,\phi (x\left)\right)=0$ and $\phi \left(0\right)=0$. In this case, the operator $F_y^{\prime }(0,0)$ is given by

$$F_y^{\prime }(0,0)y=y^{\prime \prime }+y+g^{\prime }\left(0\right)=y^{\prime \prime }+y,$$

(80)

since $g^{\prime }\left(0\right)=0$.

However, the situation changes in the nonregular case. Consider, for example, the BVP

$$y^{\prime \prime }\left(t\right)+y\left(t\right)=sint,y\left(0\right)=y\left(\pi \right)=0,$$

which has no solution. To see this, multiply both sides of the equation by $sint$ and integrate from 0 to $\pi$. The left-hand side, after integration by parts, evaluates to zero, while the right-hand side is nonzero. In this example, the operator $F_y^{\prime }(0,0)$ is not surjective, and, therefore, the classical Implicit Function Theorem does not apply to guarantee the existence of a solution to Equation (78).

4.8.2. Nonlinear Boundary-Value Problem in the Nonregular Case

We consider the boundary-value problem (74) and (75) in the nonregular case, using the definitions and notation introduced in Section 4.8.1. Our analysis is restricted to a neighborhood of the point $(\overline{x}\left(t\right),\overline{y}\left(t\right))=(0,0)$, $t\in [0,\pi ].$

As shown in Section 4.8.1, the operator $F_y^{\prime }(0,0)$ is not surjective. In this case, we apply the pth-order Implicit Function Theorem 11 with $p=2$ to derive conditions for the existence of an implicit function $y=\phi \left(x\right)$, and, consequently, for the existence of a solution to the BVP (74) and (75).

To apply Theorem 11, we first introduce some auxiliary spaces and functions for the mapping $F(x,y)$, in accordance with Section 4.2.2.

By the definition of the operator $F_y^{\prime }(0,0)$ in (80), its image is the set of all $z\left(t\right)\in Z$, such that there exists $y\in Y$ satisfying

$$y^{″}+y=z\left(t\right).$$

(81)

The general solution of (81) has the form:

$$y\left(t\right)=C_1\cos t+C_2\sin t-\sin t{\int }_0^t\cos \tau z\left(\tau \right)d\tau +\cos t{\int }_0^t\sin \tau z\left(\tau \right)d\tau ,C_1,C_2\in \mathbb{R}.$$

Substituting the boundary conditions $y\left(0\right)=y\left(\pi \right)=0$ yields $C_1=0$ and

$${\int }_0^{\pi }\sin \tau z\left(\tau \right)d\tau =0.$$

Hence,

$$Z_1=\mathrm{Im}{F}_y^{\prime }(0,0)=\left\{z\left(t\right)\in Z\left|\underset{0}{\overset{\pi }{\int }}\sin \tau z\left(\tau \right)d\tau =0\right\}\right.,$$

(82)

and, as expected, $Z_1\ne Z$. The kernel of $F_y^{\prime }(0,0)$ is defined by the boundary value problem

$$y^{″}+y=0,y\left(0\right)=y\left(\pi \right)=0,$$

whose solution is $y\left(t\right)=C\sin t$, with $C\in \mathbb{R}$. Therefore, $\mathrm{Ker}\left(F_y^{\prime }(0,0)\right)=\mathrm{span}\left(\sin t\right)$.

Let $Z_2$ be a closed complementary subspace to $Z_1$. Then, $Z_2=\mathrm{span}\left(\sin t\right),$ and the projection operator $P_{Z_2}$ is defined as

$$P_{Z_2}z\left(t\right)={\displaystyle \frac{2}{\pi }}\sin t\underset{0}{\overset{\pi }{\int }}\sin \left(\tau \right)z\left(\tau \right)d\tau ,z\left(t\right)\in Z.$$

(83)

Next, define the mappings $f_1(x,y)$ and $f_2(x,y)$ by

$$f_1(x,y)=F(x,y),f_2(x,y)=P_{Z_2}F(x,y).$$

(84)

For $p=2$, the 2-factor-operator has the form:

$${\Psi }_2\left(h\right)y\left(t\right)={\left(y\right(t\left)\right)}^{″}+\left(y\left(t\right)\right)+P_{Z_2}{g}^{″}\left(0\right)\left[h\right],$$

(85)

where $h=h\left(x\right(t\left)\right)$ is a function.

Example 13. 

Consider the following nonlinear BVP:

$$y^{″}\left(t\right)+y\left(t\right)+y^2\left(t\right)=v\sin t,y\left(0\right)=y\left(\pi \right)=0,$$

(86)

where $g\left(y\right)=y^2$, $x\left(t\right)=v\sin t$, $F(x,y)=y^{\prime \prime }+y+y^2-v\sin t$, v is a constant, and $F:X\times Y\to Z$, with X, Y and Z defined above.

We now verify that all conditions of the pth-order Implicit Function Theorem 11 are satisfied for the mapping $F(x,y)$ with a sufficiently small $v>0$ and $p=2$. Note that $\overline{y}\left(t\right)=0$ is a solution of the homogeneous BVP corresponding to (86), so that $F(\overline{x},\overline{y})=0$.

For $p=2$, Condition 1 of Theorem 11 holds for F due to the structure of the mapping $g\left(y\right)$, as well as $f_1(x,y)$ and $f_2(x,y)$ introduced in (84).

Condition 2 (the 2-factor-approximation) depends only on the properties of the mapping $g\left(y\right)=y^2$ and reduces to the existence of a sufficiently small $\epsilon >0$ and a neighborhood $U\left(\overline{y}\right)$ of $\overline{y}$ such that for all $y_1,y_2\in U\left(\overline{y}\right)$,

$$∥P_{Z_1}(y_1^2-y_2^2)∥\le ∥y_1^2-y_2^2∥\le \epsilon \parallel y_1-y_2\parallel ,$$

and

$$∥P_{Z_2}\left(y_1^2-y_2^2-y_1^2+y_2^2\right)∥\le \epsilon \left(\parallel y_1\parallel +\parallel y_2\parallel \right)\parallel y_1-y_2\parallel .$$

Both inequalities hold, and hence Condition 2 is satisfied.

Condition 3 is equivalent to the existence of a neighborhood $U\left(\overline{x}\right)$ such that for some $x\in U\left(\overline{x}\right)$, there exists a function $h=h\left(x\right(t\left)\right)$ and $c_1>0$ such that

$$h^{″}+h+{\displaystyle \frac{2}{\pi }}\sin t\underset{0}{\overset{\pi }{\int }}\sin \left(\tau \right)h^2\left(\tau \right)d\tau =v\sin t.$$

(87)

Problem (87) has an explicit solution

$$h\left(t\right)=\sqrt{{\displaystyle \frac{3\pi v}{8}}}\sin t,$$

(88)

which exists only for $v>0$. Then, Condition 3 reduces to verifying that there exists a constant $c_1>0$ such that

$$∥\sqrt{{\displaystyle \frac{3\pi v}{8}}}\sin t∥\le c_1{\parallel v\sin t\parallel }^{1/2}.$$

This inequality is equivalent to

$$\sqrt{{\displaystyle \frac{3\pi }{8}}}\le c_1,$$

which is satisfied, for instance, by taking $c_1=\sqrt{{\displaystyle \frac{3\pi }{8}}}$.

To verify Condition 4, we observe that with $x=v\sin t$ and h given by (88), the set ${\Phi }_2^{-1}(-F(x,\overline{y}))$ consists of a single element $\left\{h\right\}.$ The operator ${\Psi }_2(\overline{h})$, defined in (85), with $\overline{h}\left(t\right)=\sin t,$ takes the form:

$${\Psi }_2\left(\overline{h}\right)y=y^{″}+y+{\displaystyle \frac{2}{\pi }}\sin t{\int }_0^{\pi }\left(2y\right){\sin }^2\tau d\tau ,$$

which is surjective, and therefore Condition 4 is satisfied.

Having verified all four conditions of Theorem 11, we conclude that there exists a solution $y\left(t\right)$ to the BVP (86), satisfying

$$\parallel y\left(t\right)\parallel \le {c\parallel v\sin t\parallel }^{1/2}\le c{v}^{1/2},c>0.$$

4.9. Interpolation by Polynomials

In this section, we consider one of the newest applications of the p-regularity theory. There are many books on numerical analysis and numerical methods where the topics of interpolation and polynomial approximation are described in detail (see, for example, [73,74]).

4.9.1. Newton Interpolation Polynomial

Let f be ${\mathcal{C}}^{p+1}\left([a,b]\right)$ and consider the equation

$$f\left(x\right)=0,$$

where $x\in [a,b]$. For some $\Delta x>0$, define the points $x_i$, $i=0,\dots ,n$, as follows:

$$x_0=a,x_1=x_0+\Delta x,x_2=x_1+\Delta x,\dots ,x_n=b.$$

Let

$$y_i=f\left(x_i\right),i=0,1,\dots ,n.$$

The problem of interpolation is to find a polynomial ${\mathcal{P}}_n\left(x\right)$ of degree at most n such that ${\mathcal{P}}_n\left(x_i\right)=y_i$, $i=0,\dots ,n$, and that gives a good approximation of the function $f\left(x\right)$.

Let $\epsilon =\Delta x$ be sufficiently small and assume that $|f\left(x\right)-{\mathcal{P}}_n\left(x\right)|\le C_1\epsilon$, where $C_1\ge 0$ is a constant. Assume that the equation $f\left(x\right)=0$ has a solution $\overline{x}\in (a,b)$, and the equation ${\mathcal{P}}_n\left(x\right)=0$ has a solution $\tilde{x}\in (a,b)$. Our goal is to use the interpolation polynomial ${\mathcal{P}}_n\left(x\right)$ and its solution $\tilde{x}$ to obtain the ${\epsilon }^2$-accuracy of the solution $\overline{x}$, in the sense that

$$|\overline{x}-\tilde{x}|\le C{\epsilon }^2,$$

(89)

where $C\ge 0$ is a constant. In the regular case, this can be obtained by using, for example, the Newton interpolation polynomial ${\mathcal{P}}_n\left(x\right)$ with $\Delta x=\epsilon .$

Recall that the Newton interpolation polynomial of degree n, related to the data points

$$(x_0,y_0),(x_1,y_1),\dots ,(x_n,y_n),$$

is defined by

$$\begin{array}{ll}{\mathcal{P}}_n\left(x\right)=& {\alpha }_0+{\alpha }_1(x-x_0)+{\alpha }_2(x-x_0)(x-x_1)+\hfill \\ & \dots +{\alpha }_n(x-x_0)(x-x_1)\dots (x-x_{n-1})\hfill \\ & ={\displaystyle \sum _{k=0}^n}{\alpha }_k{\omega }_k\left(x\right),\hfill \end{array}$$

where

$$\begin{array}{c}{\omega }_0\left(x\right)=1,\hfill \\ {\omega }_i\left(x\right)=(x-x_0)(x-x_1)\dots (x-x_{i-1})=\prod _{j=0}^{i-1}(x-x_j),i=1,\dots ,n.\hfill \end{array}$$

(90)

The coefficients ${\alpha }_k$ are called divided differences and are defined using the following relations:

$${\alpha }_k=[y_0,\dots ,y_k],k=0,1,\dots ,n,$$

(91)

where

$$\begin{array}{ccc}\hfill \left[y_k\right]& & =y_k,k=0,\dots ,n,\hfill \\ \hfill [y_k,\dots ,y_{k+j}]& & ={\displaystyle \frac{[y_{k+1},\dots ,y_{k+j}]-[y_k,\dots ,y_{k+j-1}]}{x_{k+j}-x_k}}],\hfill \\ & & k=0,\dots ,n-j,j=1,\dots ,k.\hfill \end{array}$$

In the following example, we consider a nonlinear function $f\left(x\right)$, which is not regular at a solution of the equation $f\left(x\right)=0$, and investigate whether a solution of the equation ${\mathcal{P}}_n\left(x\right)=0$ provides the desired accuracy (89) for the solution $\overline{x}$ of $f\left(x\right)=0$, assuming that $|f\left(x\right)-{\mathcal{P}}_n\left(x\right)|\le C_1\epsilon$ holds for a sufficiently small $\epsilon$.

Example 14. 

Consider the function $f\left(x\right)=x^3$. The solution of the equation $f\left(x\right)=0$ is $\overline{x}=0$. The function $f\left(x\right)$ is singular at $\overline{x}=0$ up to the second order because $f^{\left(i\right)}\left(0\right)=0$ for $i=1,2.$ The goal in this example is to investigate whether the estimate (89) is satisfied when using the interpolation polynomial ${\mathcal{P}}_1\left(x\right)$ and a solution of ${\mathcal{P}}_1\left(x\right)=0$ to approximate the solution of $f\left(x\right)=0$. Using the equations given above with $n=1$, we obtain

$${\mathcal{P}}_1\left(x\right)={\alpha }_0+{\alpha }_1(x-x_0),$$

where the coefficients ${\alpha }_0$ and ${\alpha }_1$ are determined by using Equation (91).

Let $\epsilon =\Delta x$ be sufficiently small and consider the segment $[a,b]=[-{\displaystyle \frac{1}{3}}\epsilon ,{\displaystyle \frac{2}{3}}\epsilon ]$. The interpolation points are $x_0=a=-{\displaystyle \frac{1}{3}}\epsilon$ and $x_1=b={\displaystyle \frac{2}{3}}\epsilon$. Calculating the coefficients, we obtain

$${\alpha }_0=f\left(x_0\right)=-{\displaystyle \frac{{\epsilon }^3}{27}}and{\alpha }_1={\displaystyle \frac{f\left(x_1\right)-f\left(x_0\right)}{x_1-x_0}}={\displaystyle \frac{{\epsilon }^2}{3}}.$$

Hence, the interpolation polynomial has the form

$$W_1\left(x\right)=-{\displaystyle \frac{{\epsilon }^3}{27}}+{\displaystyle \frac{{\epsilon }^2}{3}}(x+{\displaystyle \frac{1}{3}}\epsilon )=-{\displaystyle \frac{{\epsilon }^3}{27}}+{\displaystyle \frac{{ϵ}^2}{3}}x+{\displaystyle \frac{{\epsilon }^3}{9}}={\displaystyle \frac{2{\epsilon }^3}{27}}+{\displaystyle \frac{{ϵ}^2}{3}}x.$$

Moreover,

$$|{\mathcal{P}}_1\left(x\right)-f\left(x\right)|\approx {\epsilon }^3\le C_2\epsilon ,C_2\ge 0,$$

for a sufficiently small ε.

The solution of the equation ${\mathcal{P}}_1\left(x\right)=0$ is

$$\tilde{x}=-{\displaystyle \frac{2}{9}}\epsilon ,$$

which is not satisfactory from the approximation accuracy point of view, since

$$|\tilde{x}-\overline{x}|=\left|-{\displaystyle \frac{2}{9}}\epsilon -0\right|\approx \epsilon >{\epsilon }^2$$

and the desired accuracy (89) is not obtained.

Thus, in the degenerate case, contrary to the regular case, while we have the required accuracy of the approximation for the function $f\left(x\right)=x^3$, the accuracy of the solution is only of the order ε, and not ${\epsilon }^2$.

4.9.2. The p-Factor Interpolation Method

In this section, we demonstrate that the desired accuracy (89) for the solution of the equation $f\left(x\right)=0$ in the degenerate case can be achieved by using the p-factor interpolation polynomial, rather than the classical Newton interpolation polynomial, to obtain an approximate solution of $f\left(x\right)=0$.

Let $f:\mathbb{R}\to \mathbb{R}$ be a ${\mathcal{C}}^{p+1}$ function that is singular at a point $\overline{x}$.

For some $p>1$, we associate f with its corresponding p-factor function $\overline{f}$, defined as

$$\overline{f}\left(x\right)=f\left(x\right)+f^{\prime }\left(x\right)h+\dots +f^{(p-1)}\left(x\right){\left[h\right]}^{p-1},$$

where $h\in \mathbb{R}$, $h\ne 0$. Similarly to the Newton interpolation method, we construct the p-factor interpolation polynomial $\overline{{\mathcal{P}}_n}\left(x\right)$ using the function $\overline{f}$ as follows:

$$\overline{{\mathcal{P}}_n}\left(x\right)=\sum _{k=0}^n{\overline{\alpha }}_k{\omega }_k\left(x\right),$$

where the functions ${\omega }_k\left(x\right)$ are defined in the same way as in (90), and the coefficients ${\overline{\alpha }}_k$, for $k=0,1,\dots ,n$, are given by

$$\begin{array}{c}{\overline{\alpha }}_0=\left[{\overline{y}}_0\right]=\overline{f}\left(x_0\right), \mathrm{and} \left[{\overline{y}}_i\right]=\overline{f}\left(x_i\right), \mathrm{for} i=1,\dots ,n,\hfill \\ {\overline{\alpha }}_1=[{\overline{y}}_0,{\overline{y}}_1]={\displaystyle \frac{\left[{\overline{y}}_1\right]-\left[{\overline{y}}_0\right]}{x_1-x_0}},\hfill \\ ⋮\hfill \\ {\overline{\alpha }}_n=[{\overline{y}}_0,\dots ,{\overline{y}}_n]={\displaystyle \frac{[{\overline{y}}_1,\dots ,{\overline{y}}_n]-[{\overline{y}}_0,\dots ,{\overline{y}}_{n-1}]}{x_n-x_0}}.\hfill \end{array}$$

Theorem 18. 

Let the equation $f\left(x\right)=0$ has a solution $\overline{x}\in (a,b)$. Assume that $f\in {\mathcal{C}}^p\left([a,b]\right)$ is p-regular along $h\ne 0$ at the point $\overline{x}$. Suppose that $\overline{{\mathcal{P}}_n}\left(x\right)$ is the Newton interpolation polynomial for the associated function $\overline{f}$, constructed with a sufficiently small interpolation step $\epsilon =\Delta >0$.

Then, the equation $\overline{{\mathcal{P}}_n}\left(x\right)=0$ has a solution $\widehat{x}\in (a,b)$ such that

$$|\widehat{x}-\overline{x}| \le c{\epsilon }^2,$$

where $c>0$ is an independent constant.

We omit the proof, as it is similar to the proof of convergence of the classical iterative Newton method.

As in the previous sections, we say that a function $f\in {\mathcal{C}}^p\left([a,b]\right)$ is p-regular along $h\ne 0$ at the point $\overline{x}\in (a,b)$ if there is a natural number $p\ge 2$ such that

$$f^{\left(i\right)}\left(\overline{x}\right)=0,i=1,\dots ,p-1,and{f}^{\left(p\right)}\left(\overline{x}\right)\ne 0.$$

Note that if $p=1$, the definition of a p-regular function f reduces to the standard definition of a regular function, and the p-factor interpolation polynomial $\overline{{\mathcal{P}}_n}\left(x\right)$ coincides with the classical Newton interpolation polynomial ${\mathcal{P}}_n\left(x\right)$.

Example 15. 

We will apply the p-factor interpolation method to the function from Example 14. Define the function $\overline{f}$ for $p=2$ and $h=1$ as

$$\overline{f}\left(x\right)=f\left(x\right)+f^{\prime }\left(x\right)h+f^{″}\left(x\right){\left[h\right]}^2=x^3+3x^2+6x,$$

and consider the p-factor interpolation polynomial $\overline{{\mathcal{P}}_1}\left(x\right)$. Using the same interval as in Example 14, the interpolation points are $x_0=a=-{\displaystyle \frac{1}{3}}\epsilon$ and $x_1=b={\displaystyle \frac{2}{3}}\epsilon$. The coefficients are given by

$${\overline{\alpha }}_0=\overline{f}\left(x_0\right)=-{\displaystyle \frac{1}{27}}{\epsilon }^3+{\displaystyle \frac{1}{3}}{\epsilon }^2-2\epsilon$$

and

$${\overline{\alpha }}_1={\displaystyle \frac{\overline{f}\left(x_1\right)-\overline{f}\left(x_0\right)}{x_1-x_0}}={\displaystyle \frac{1}{3}}{\epsilon }^2+\epsilon +6.$$

Thus, the p-factor interpolation polynomial is

$$\begin{array}{ccc}\hfill \overline{{\mathcal{P}}_1}\left(x\right)& =& {\overline{\alpha }}_0+{\overline{\alpha }}_1(x-x_0)\hfill \\ & =& -{\displaystyle \frac{1}{27}}{\epsilon }^3+{\displaystyle \frac{1}{3}}{\epsilon }^2-2\epsilon +\left({\displaystyle \frac{1}{3}}{\epsilon }^2+\epsilon +6\right)\left(x+{\displaystyle \frac{1}{3}}\epsilon \right)\hfill \\ & =& \left({\displaystyle \frac{1}{3}}{\epsilon }^2+\epsilon +6\right)x+{\displaystyle \frac{2}{27}}{\epsilon }^3+{\displaystyle \frac{2}{3}}{\epsilon }^2.\hfill \end{array}$$

Hence, for a sufficiently small ε, we have

$$\left|\overline{{\mathcal{P}}_1}\left(x\right)-f\left(x\right)\right|\le C_3\epsilon ,C_3\ge 0.$$

Solving the equation ${\overline{W}}_1\left(x\right)=0$, we obtain

$$\widehat{x}=-{\displaystyle \frac{{\displaystyle \frac{2}{27}}{\epsilon }^3+{\displaystyle \frac{2}{3}}{\epsilon }^2}{{\displaystyle \frac{1}{3}}{\epsilon }^2+\epsilon +6}}=-{\displaystyle \frac{{\displaystyle \frac{2}{9}}{\epsilon }^3+2{\epsilon }^2}{{\epsilon }^2+3\epsilon +18}}.$$

Therefore,

$$|\widehat{x}-\overline{x}|=\left|-{\displaystyle \frac{{\displaystyle \frac{2}{9}}{\epsilon }^3+2{\epsilon }^2}{{\epsilon }^2+3\epsilon +18}}-0\right|<{\displaystyle \frac{3{\epsilon }^2}{18}}={\displaystyle \frac{{\epsilon }^2}{6}},$$

an we thus obtain the desired ${\epsilon }^2$-accuracy stated in estimate (89) for the solution of the equation $f\left(x\right)=0$.

Let us now compare the use of the classical polynomial ${\mathcal{P}}_1\left(x\right)$ with the p-factor interpolation polynomial $\overline{{\mathcal{P}}_1}\left(x\right)$ in approximating the solution $\overline{x}$ of the equation $f\left(x\right)=0$, for the function f from Example 14. As mentioned earlier, the polynomial ${\mathcal{P}}_1\left(x\right)$ is a good approximation for the function $f\left(x\right)$ in the sense that

$$|{\mathcal{P}}_1\left(x\right)-f\left(x\right)|\approx {\epsilon }^3\le C_2\epsilon ,C_2\ge 0.$$

However, the solution $\tilde{x}$ of ${\mathcal{P}}_1\left(x\right)=0$ does not yield the desired accuracy for $\overline{x}$, since

$$\left|\tilde{x}-\overline{x}\right|\approx \epsilon \ge C_4\epsilon ,C_4\ge 0,$$

and the target accuracy of order ${\epsilon }^2$ is not achieved.

In contrast, the p-factor interpolation polynomial $\overline{{\mathcal{P}}_1}\left(x\right)$ approximates $f\left(x\right)$ with order $ϵ$:

$$\left|\overline{{\mathcal{P}}_1}\left(x\right)-f\left(x\right)\right|\approx \epsilon .$$

Thus, using the p-factor interpolation polynomial $\overline{{\mathcal{P}}_1}\left(x\right)$, we achieve the desired accuracy for the solution $\overline{x}$ of $f\left(x\right)=0$. Specifically, as shown above, the solution $\widehat{x}$ of $\overline{{\mathcal{P}}_1}\left(x\right)=0$ satisfies estimate (89):

$$|\widehat{x}-\overline{x}|\le {\displaystyle \frac{1}{6}}{\epsilon }^2.$$

This level of accuracy could not be achieved using the classical interpolation polynomial ${\mathcal{P}}_1\left(x\right)$.

5. Conclusions

In this paper, we described various applications of the theory of p-regularity, including the generalization of the Lyusternik and Implicit Function theorems, the Newton method, optimality conditions for equality and inequality constraints, calculus of variations, and the solvability of nonlinear equations.

We should note that we did not cover all areas where the results of the theory can be applied. In addition, there are other areas of mathematics where the theory of p-regularity (or p-factor-analysis) has not yet been applied. For example, we did not provide examples of applying the theory to the analysis of the existence of solutions for singular nonlinear partial differential equations, such as the Burger’s nonlinear equation, the Laplace nonlinear differential equation, and others. We also did not cover results related to the existence of solutions depending on a parameter for the Van der Pol differential equation, the Duffing equation, and others. Other results not covered in this paper include examples of applying the theory of p-regularity for the analysis of nonlinear dynamical systems, optimality conditions for optimal control problems in the nonregular (degenerate) case. Based on the theory of p-regularity, we can develop the theory of so-called p-convexity, which can be effective for the analysis of nonlinear problems. Additional information can be found in other studies by the authors.