Multiplicity of Solutions for the Noncooperative Kirchhoff-Type Variable Exponent Elliptic System with Nonlinear Boundary Conditions

Abstract

Considering the solutions of a class of noncooperative Kirchhoff-type $p\left(x\right)$-Laplacian elliptic systems with nonlinear boundary conditions, we derive a sequence of solutions utilizing both the variational method and limit index theory under certain underlying assumptions. The novelty of this study is that we verify the ${\left(PS\right)}_c^{*}$ condition using another method, diverging from the approaches cited in the previous literature.

1. Introduction

In this paper, we delve into an exploration of the existence and multiplicity of solutions to the Kirchhoff $p\left(x\right)$–Laplacian elliptic system:

$$\left\{\begin{array}{cccc}\hfill & M\left({\int }_{\mathsf{\Omega }}\frac{{|\nabla u|}^{p\left(x\right)}+{\left|u\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right)({\Delta }_{p\left(x\right)}u-{\left|u\right|}^{p\left(x\right)-2}u)=F_u(x,u,v),\hfill & \hfill & \mathrm{in}\mathsf{\Omega },\hfill \\ \hfill & M\left({\int }_{\mathsf{\Omega }}\frac{{|\nabla v|}^{p\left(x\right)}+{\left|v\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right)(-{\Delta }_{p\left(x\right)}v+{\left|v\right|}^{p\left(x\right)-2}v)=F_v(x,u,v),\hfill & \hfill & \mathrm{in}\mathsf{\Omega },\hfill \\ \hfill & {|\nabla u|}^{p\left(x\right)-2}\frac{\partial u}{\partial \nu }={\left|u\right|}^{p_{*}\left(x\right)-2}{u,|\nabla v|}^{p\left(x\right)-2}\frac{\partial v}{\partial \nu }={\left|v\right|}^{p_{*}\left(x\right)-2}v,\hfill & \hfill & \mathrm{on}\partial \mathsf{\Omega },\hfill \end{array}\right.$$

(1)

where $\mathsf{\Omega }\subset {\mathbb{R}}^N$ is a smooth bounded domain, $p\left(x\right)$ is Lipschitz-continuous and radially symmetric on $\overline{\mathsf{\Omega }}$ and fulfills $1<p^{-}<p\left(x\right)<p^{+}<N$ with $p^{+}=\underset{x\in \overline{\mathsf{\Omega }}}{sup}p\left(x\right),p^{-}=\underset{x\in \overline{\mathsf{\Omega }}}{min}p\left(x\right)$, $p_{*}\left(x\right)=\frac{(N-1)p\left(x\right)}{N-p\left(x\right)}$, ${\Delta }_{p\left(x\right)}u:={div\left(\right|\nabla u|}^{p\left(x\right)-2}\nabla u)$ is a $p\left(x\right)$-Laplacian operator, $F=F(x,u,v)$, $F_u=\frac{\partial F}{\partial u},F_v=\frac{\partial F}{\partial v}$, and $\frac{\partial }{\partial \nu }$ is the outer normal derivative.

Assuming that $M:{\mathbb{R}}_0^{+}:=[0,+\infty )\to {\mathbb{R}}^{+}:=(0,+\infty )$ is a continuous Kirchhoff function, which fulfills the following conditions:

($M_1$)

If $m_0>0$ exists, then

$$M\left(t\right)⩾m_0,\forall t\in {\mathbb{R}}_0^{+};$$

($M_2$)

There exists $\theta \in [p^{-},\frac{p_{*}^{-}}{p^{+}})$ such that $\theta \widehat{M}\left(t\right):=\theta {\int }_0^{t}M\left(\tau \right)d\tau ⩾M\left(t\right)t⩾p^{-}\widehat{M}\left(t\right)$ for any $t\in {\mathbb{R}}_0^{+}$.

The nonlinearity F satisfies the following:

($F_1$)

$F\in C^1(\overline{\mathsf{\Omega }}\times {\mathbb{R}}^2,{\mathbb{R}}^{+})$, $F(x,s,t)=F(x,-s,-t)$ and $F(x,s,t)=F\left(\right|x|,s,t)$ for every $(x,s,t)\in \mathsf{\Omega }\times {\mathbb{R}}^2$;

($F_2$)

$s{F}_s(x,s,t)⩾0$ for every $(x,s,t)\in \mathsf{\Omega }\times {\mathbb{R}}^2$;

($F_3$)

There exist $C_1,C_2>0,\theta p^{+}<r\left(x\right)<p^{*}\left(x\right)$, where $p^{*}\left(x\right)=Np\left(x\right)/(N-p\left(x\right))$ such that

$$\left|F(x,s,t)\right|⩽C_1\left({\left|s\right|}^{r\left(x\right)}+{\left|t\right|}^{r\left(x\right)}\right)+C_2;$$

($F_4$)

$0<F(x,s,t)⩽\frac{1}{p^{-}}s{F}_s(x,s,t)+\frac{1}{p_{*}^{-}}t{F}_t(x,s,t)$, for every $(x,s,t)\in \mathsf{\Omega }\times {\mathbb{R}}^2$;

($F_5$)

there exist $L,m_1,C_3,C_4>0$ (where $L,m_1,C_3,C_4$ will be determined later) and ${\xi <\left|\mathsf{\Omega }\right|}^{-1}\{(\frac{1}{\theta p^{+}}-\frac{1}{p_{*}^{-}})m_1-\frac{C_3}{{\left(p^{-}\right)}^{\theta }}-C_4\}$ such that, for every $(x,s,t)\in \mathsf{\Omega }\times {\mathbb{R}}^2,F(x,s,t)⩾L{\left|t\right|}^{\theta p^{+}}-\xi$.

A typical example for M is given in $M\left(t\right)=m_0+b_1{t}^{\theta -1}$ with $\theta >p^{-}$, $m_0\in {\mathbb{R}}^{+}$ and $b_1>0$, and an example for F is $F(x,s,t)=C_1\left({\left|s\right|}^{p^{-}}+{\left|t\right|}^{p^{-}}\right)+C_2$, where $C_1,C_2>0$.

We now present our significant conclusion.

Theorem 1.

Assuming $F(x,u,v)$ meets conditions $\left(F_1\right)$–$\left(F_5\right)$ and $M\left(t\right)$ satisfies conditions $\left(M_1\right)$–$\left(M_2\right)$, then there exists $k_0>1$ such that the problem (1) possesses at least $k_0-1$ pairs of nontrivial weak solutions.

Exploring the realm of differential equations that feature variable exponents, alongside the challenges posed by variational issues, proves to be a captivating area of interest. This field finds its roots in nonlinear elastic theory and electrorheological fluids, among other examples. Over the preceding few years, the study of variable exponential problems has received increased interest, particularly the nonlinear problem with variable exponentials, which not only extends beyond the traditional constant exponential problem but also reflects the physical phenomenon of “point-by-point anisotropy”. This type of problem is broadly applicable to mathematics and physics, where it is used to model elastomechanical or electrorheological fluids (alternatively known as “smart fluids”).

Variable exponent Lebesgue spaces were first proposed in 1931 by the Polish mathematician Orlicz [1], who considered the variable exponent space $L^{p\left(x\right)}\left([a,b]\right)$ on a line on which he proved that Hölder inequality still holds, but he did not pursue this further. In 1961, Tsenov [2] presented the following problem: how to find the minimum value of

$${\int }_a^b{|u\left(x\right)-v\left(x\right)|}^{p\left(x\right)}dx.$$

Based on this problem, Sharapudinov [3] proved that the space $L^{p\left(x\right)}\left([a,b]\right)$ is reflexive under the condition that the variable exponential function $p\left(t\right)$ satisfies $1<p^{-}⩽p^{+}<+\infty$. After that, Zhikov [4] studied Lavrentiev’s phenomenon (that is, the lower certainty of an integral functional on a Sobolev space is strictly smaller than its lower certainty on a smooth function space) of variational problems with variable exponents against the background of problems in nonlinear elasticity, proposing the famous Zhikov conjecture. This also reflects an essential difference between the variable exponential problem and the constant exponential problem. In fact, for the constant exponential case, Lavrentiev’s phenomenon does not occur at all. In the early 1990s, Czech mathematicians Kováčik and Rákosnik [5] made a major breakthrough in the theory of variable exponential spaces, establishing the fundamental theory of Lebesgue and Sobolev spaces in ${\mathbb{R}}^N$. Fan and Alkhutov continued Zhikov’s work later, around 1995, enriching the fundamental theory of Lebesgue and Sobolev spaces [6,7,8,9].

Around the year 2000, rapid advancements in various fields caused the variational exponential space to undergo a systematic and intense phase of research, and scholars were aware of the inextricative links between variational problems of variational exponents and some models in electronic rheological fluids. Among them, ref. [10] is a monograph on the application background of the variable index problem in electronic rheological fluids, which is considered a milestone in the in-depth development of the research on variable exponent problems.

There has been an abundance of interest in variable exponent problems that involve nonstandard $p\left(x\right)$ growth conditions, and much progress has been achieved. For the problem with a $p\left(x\right)$-Laplacian operator, we usually study the definite solution problem, initial value problem, initial boundary value problem, boundary value problem, free boundary value problem, eigenvalue problem, and regularity problem. This kind of problem can be used to describe the dynamic phenomena of circuit variable fluid and elastic mechanics. Systems with a $p\left(x\right)$-Laplacian operator reflect the physical phenomenon of “point-by-point anisotropy”. Traditional theories and techniques like Sobolev space theory are not suitable, so variable exponential space theory is widely used. Under the condition of $p\left(x\right)$ growth, the established basic theories of generalized Lebesgue space $L^{p\left(x\right)}$ and Sobolev space $W^{k,p\left(x\right)}$ provide sufficient theoretical basis for the study of the above problems.

The Kirchhoff equation studied in this paper is a typical example of an elliptic partial differential equation. In 1883, German physicist Kirchhoff proposed the following model [11] when studying the problem of string length change caused by the vibration of elastic strings

$$\rho \frac{{\partial }^{2}u}{\partial t^2}=\left(\frac{P_0}{h}+\frac{E}{2L}{\int }_0^L{\left|\frac{\partial u}{\partial x}\right|}^{2}dx\right)\frac{{\partial }^{2}u}{\partial x^2}.$$

This model studies the free vibration of an elastic string. The coefficients on the right side of the model contain global integral terms, and the coefficients depend on the average kinetic energy$\frac{1}{2L}{\int }_0^L{\left|\frac{\partial u}{\partial x}\right|}^{2}dx$. As a result, the Kirchhoff equation is no longer a point-by-point identity, so the Kirchhoff equation is also called a class of non-local problems. This kind of problem comes from the phenomena produced via non-local mechanics, non-local quantum mechanics, etc., and it has a wide range of practical applications. As an important method to study Kirchhoff-type problems, the variational method has been applied by many scholars. Its basic idea is to transform a large number of variational problems into critical point problems or extreme value problems of a corresponding function under certain conditions. The non-local variational problem has a wide range of practical applications when it is limited due to various boundary value conditions. It not only promotes the study and calculation of nonlinear partial differential equations but also has a certain reference value for nonlinear problems in the fields of imaging, electromagnetism, optics, quantum mechanics, climate, and so on.

Over the past few years, there has been an increased focus on investigating noncooperative elliptic systems. In 2009, Lin and Li [12] studied the noncooperative elliptic system

$$\left\{\begin{array}{cc}▵u={\left|u\right|}^{2^{*}-2}u+F_u(x,u,v),\hfill & \mathrm{in}\mathsf{\Omega },\hfill \\ -▵v={\left|v\right|}^{2^{*}-2}v+F_v(x,u,v),\hfill & \mathrm{in}\mathsf{\Omega },\hfill \\ {\left.u\right|}_{\partial \mathsf{\Omega }}=0,{\left.v\right|}_{\partial \mathsf{\Omega }}=0,\hfill \end{array}\right.$$

where $\mathsf{\Omega }\subset {\mathbb{R}}^N$. They overcame the difficulty with the embedding $H_0^1\left(\mathsf{\Omega }\right)\hookrightarrow L^{2_{*}}\left(\mathsf{\Omega }\right)$ not being compact. By making assumptions about the nonlinear part, they identified the existence of solutions.

The next year, Fang and Zhang [13] extended the above results to $(p,q)$-Laplacian operators. By employing the same method as above, the multiplicity results for the solutions were obtained.

In 2012, utilizing the concentration–compactness principle, Liang and Zhang [14] conducted an in-depth investigation into the noncooperative p-Laplace elliptic system.

$$\left\{\begin{array}{cc}{\Delta }_{p}u-{\left|u\right|}^{p-2}u=F_u(x,u,v),\hfill & \mathrm{in}\mathsf{\Omega },\hfill \\ -{\Delta }_{p}v+{\left|v\right|}^{p-2}v=F_v(x,u,v),\hfill & \mathrm{in}\mathsf{\Omega },\hfill \\ {|\nabla u|}^{p-2}\frac{\partial u}{\partial \nu }={\left|u\right|}^{p^{*}-2}{u,|\nabla v|}^{p-2}\frac{\partial v}{\partial \nu }={\left|v\right|}^{p^{*}-2}v,\hfill & \mathrm{on}\partial \mathsf{\Omega },\hfill \end{array}\right.$$

where $1<p<N,N⩾3$. Also, using the same methods, they obtained a sequence of solutions.

In 2020, similar results were also obtained by N. T. Chung [15] for the Kirchhoff-type system with a p-biharmonic operator.

Furthermore, with the help of [16], Liang [17] carried out further research in the field of variable exponential space and obtained multiple solutions for the problem below:

$$\left\{\begin{array}{cc}{\Delta }_{p\left(x\right)}u-{\left|u\right|}^{p\left(x\right)-2}u={\left|u\right|}^{p^{*}\left(x\right)-2}u+F_u(x,u,v),\hfill & \mathrm{in}\mathsf{\Omega },\hfill \\ -{\Delta }_{p\left(x\right)}v+{\left|v\right|}^{p\left(x\right)-2}v={\left|v\right|}^{p^{*}\left(x\right)-2}v+F_v(x,u,v),\hfill & \mathrm{in}\mathsf{\Omega },\hfill \\ u=0,v=0,\hfill & \mathrm{on}\partial \mathsf{\Omega },\hfill \end{array}\right.$$

where $\mathsf{\Omega }\subseteq {\mathbb{R}}^N(N⩾3)$ is a smooth-bounded, radially symmetric domain, while $0\notin \overline{\mathsf{\Omega }}$.

Afterwards, in 2013, Liang [18] extended the above system to ${\mathbb{R}}^N$. In 2017, Liang and Zhang [19] investigated a class of noncooperative Schrödinger–Kirchhoff-type systems with critical nonlinearities in ${\mathbb{R}}^N$.

Motivated by the references mentioned above, we consider a similar problem concerning the $p\left(x\right)$-Laplacian operator with nonlinear boundary conditions involving the Kirchhoff function. The novelty of this paper is as follows: in all the aforementioned papers, limit index theory [16] was applied, but the ${\left(PS\right)}_c^{*}$ condition, which is described in Definition 2, should be considered. However, in the papers of Chung [15,20], Chems Eddine [21], Liang and Shi [17], Liang and Zhang [14,18,19], Li and Song [22], Sun and Bai et al. [23], and Song and Shi [24], with the concentration–compactness principle [25], the boundness of the ${\left(PS\right)}_c^{*}$ sequence $\left\{(u_{n_k},v_{n_k})\right\}$ was determined by applying

$$\begin{array}{cc}\hfill & c+o_k\left(1\right){∥u_{n_k}∥}_E⩾J_{n_k}\left(u_{n_k},0\right)-\frac{1}{p^{**}}〈J_{n_k}^{\prime }\left(u_{n_k},v_{n_k}\right),\left(u_{n_k},0\right)〉,\hfill \\ \hfill & c+o\left(1\right){∥v_{n_k}∥}_p⩾J_{n_k}\left(0,v_{n_k}\right)-\frac{1}{\tau }〈J_{n_k}^{\prime }\left(u_{n_k},v_{n_k}\right),\left(0,v_{n_k}\right)〉,\hfill \end{array}$$

(2)

and the strong convergence of $\left\{(u_{n_k},v_{n_k})\right\}$ was achieved by discussing

$$\begin{array}{cc}\hfill & 〈-\mathrm{d}{J}_{n_k}\left(u_{n_k}-u,v_{n_k}\right),\left(u_{n_k}-u,0\right)〉\to 0\mathrm{as}n\to \infty ,\hfill \\ \hfill & 〈d{J}_{n_k}\left(0,v_{n_k}-v_0\right),\left(0,v_{n_k}-v_0\right)〉\to 0\mathrm{as}n\to \infty ,\hfill \\ \hfill & o_k\left(1\right)=〈-J_{n_k}^{\prime }\left(u_{n_k},v_{n_k}-v\right),\left(0,v_{n_k}-v\right)〉.\hfill \end{array}$$

(3)

In this paper, applying the suitable assumptions concerning F, we do not use the concentration–compactness principle to confirm the ${\left(PS\right)}_c^{*}$ condition. In fact, we provide another way without (2) and (3), which is solved in Section 5; then, the solutions for problem (1) are obtained.

The structure of this paper is outlined below: Section 2 revisits essential preliminaries and key technical lemmas. Section 3 lays out pertinent definitions and propositions associated with limit index theory. The construction of the index is delineated in Section 4. In Section 5, we complete the proof of the ${\left(PS\right)}_c^{*}$ condition. Finally, the paper culminates with a thorough proof of Theorem 1 in the concluding section.

2. Preliminaries and Some Technical Lemmas

We review some basic definitions of the variable exponent Lebesgue–Sobolev space $L^{p(·)}\left(\mathsf{\Omega }\right)$ and $W^{1,p(·)}\left(\mathsf{\Omega }\right)$ [26]. Let

$$C_{+}\left(\overline{\mathsf{\Omega }}\right)=\left\{h\in C\left(\mathsf{\Omega }\right):\underset{x\in \overline{\mathsf{\Omega }}}{min}h\left(x\right)>1\right\}.$$

For $p\in C_{+}\left(\overline{\mathsf{\Omega }}\right)$,

$$L^{p\left(x\right)}\left(\mathsf{\Omega }\right)=\left\{u:u\mathrm{is}\mathrm{a}\mathrm{measurable},\mathrm{real}\text{-}\mathrm{valued}\mathrm{function}\mathrm{such}\mathrm{that}{\int }_{\mathsf{\Omega }}{\left|u\left(x\right)\right|}^{p\left(x\right)}dx<\infty \right\},$$

which is equipped with the norm via

$${∥u∥}_{L^{p\left(x\right)}\left(\mathsf{\Omega }\right)}={∥u∥}_{p\left(x\right)}=inf\left\{\mu >0:{\int }_{\mathsf{\Omega }}{\left|\frac{u\left(x\right)}{\mu }\right|}^{p\left(x\right)}dx⩽1\right\}.$$

The variable exponent Lebesgue space $W^{1,p\left(x\right)}\left(\mathsf{\Omega }\right)$ is defined as follows

$$W^{1,p\left(x\right)}\left(\mathsf{\Omega }\right):=\left\{u\in L^{p\left(x\right)}\left(\mathsf{\Omega }\right)\mid |\nabla u|\in L^{p\left(x\right)}\left(\mathsf{\Omega }\right)\right\},$$

and it can be equipped with the norm

$$∥u∥={∥u∥}_{p\left(x\right)}+{∥\nabla u∥}_{p\left(x\right)},\forall u\in W^{1,p\left(x\right)}\left(\mathsf{\Omega }\right).$$

The equivalent norm for $W^{1,p\left(x\right)}\left(\mathsf{\Omega }\right)$ is used in this paper

$${∥u∥}_{1,p\left(x\right)}=inf\left\{\mu >0:{\int }_{\mathsf{\Omega }}\left({\left|\frac{\nabla u}{\mu }\right|}^{p\left(x\right)}+{\left|\frac{u}{\mu }\right|}^{p\left(x\right)}\right)dx⩽1\right\}.$$

In the following discussion, we refer to the boundary measure of $\partial \mathsf{\Omega }$ with $dS$. We define the variable exponent Lebesgue space $L^{z\left(x\right)}(\partial \mathsf{\Omega })$ with

$$L^{z\left(x\right)}(\partial \mathsf{\Omega })=\left\{u:\partial \mathsf{\Omega }\to \mathbb{R}\mathrm{is}\mathrm{measurable}\mathrm{and}{\int }_{\partial \mathsf{\Omega }}{\left|u\left(x\right)\right|}^{z\left(x\right)}dS<\infty \right\},$$

for any $z\in C(\partial \mathsf{\Omega })$ with $z\left(x\right)⩾1$. The corresponding Luxemburg norm is determined with

$${\parallel u\parallel }_{r\left(x\right),\partial \mathsf{\Omega }}=inf\left\{\lambda >0:{\int }_{\partial \mathsf{\Omega }}{\left|\frac{u}{\lambda }\right|}^{r\left(x\right)}dS⩽1\right\}.$$

The embedding results in the corresponding space are given below.

Proposition 1 

([26,27]). Let $\mathsf{\Omega }\subseteq {\mathbb{R}}^N$ be an open-bounded domain with a Lipschitz boundary. Then,

$\left(i\right)$

if $p,z\in C\left(\overline{\mathsf{\Omega }}\right)$ is such that $1<p\left(x\right)<N$ and $1⩽z\left(x\right)<p^{*}\left(x\right)$ on $\overline{\mathsf{\Omega }}$, there exists a continuous and compact embedding, $W^{1,p\left(x\right)}\left(\mathsf{\Omega }\right)\hookrightarrow L^{z\left(x\right)}\left(\mathsf{\Omega }\right)$;

$\left(ii\right)$

if $p\in C\left(\overline{\mathsf{\Omega }}\right)$ is such that $1<p\left(x\right)<N$ on $\overline{\mathsf{\Omega }}$, then there is a continuous boundary trace embedding, $W^{1,p\left(x\right)}\left(\mathsf{\Omega }\right)\hookrightarrow L^{p_{*}\left(x\right)}(\partial \mathsf{\Omega })$; and

$\left(iii\right)$

for each $h\in C(\partial \mathsf{\Omega })$ with $1⩽h\left(x\right)<p_{*}\left(x\right)$ on $\partial \mathsf{\Omega }$, there is a compact boundary trace embedding, $W^{1,p\left(x\right)}\left(\mathsf{\Omega }\right)\hookrightarrow$$L^{h\left(x\right)}(\partial \mathsf{\Omega })$.

Remark 1.

We define the following:

$$S=\underset{u\in W^{1,p\left(x\right)}\left(\mathsf{\Omega }\right)\setminus \left\{0\right\}}{inf}\frac{{\parallel u\parallel }_{1,p\left(x\right)}}{{\parallel u\parallel }_{p_{*}\left(x\right),\partial \mathsf{\Omega }}}.$$

(4)

Proposition 2 

([28]). Let $I\left(u\right)={\int }_{\mathsf{\Omega }}{\left(\right|\nabla u|}^{p\left(x\right)}+{\left|u\right|}^{p\left(x\right)})dx$. If $u,u_n\in W^{1,p\left(x\right)}\left(\mathsf{\Omega }\right)$; then, the relationships listed below are valid:

$\left(i\right)$

${∥u∥}_{1,p(·)}<1(=1;>1)\iff I\left(u\right)<1(=1;>1)$;

$\left(ii\right)$

${∥u∥}_{1,p(·)}>1\Rightarrow {∥u∥}_{1,p(·)}^{p^{-}}⩽I\left(u\right)⩽{∥u∥}_{1,p(·)}^{p^{+}}$;

$\left(iii\right)$

${∥u∥}_{1,p(·)}<1\Rightarrow {∥u∥}_{1,p(·)}^{p^{+}}⩽I\left(u\right)⩽{∥u∥}_{1,p(·)}^{p^{-}}$;

$\left(iv\right)$

${∥u_n-u∥}_{1,p(·)}\to 0\iff I(u_n-u)\to 0$.

In 2001, Fan and shen [26] et al., proved the following Hölder inequality.

Proposition 3 

([26]). If $\frac{1}{p\left(x\right)}+\frac{1}{p^{\prime }\left(x\right)}=1$, then for each $u\in L^{p\left(x\right)}\left(\mathsf{\Omega }\right)$ and $v\in L^{p^{\prime }\left(x\right)}\left(\mathsf{\Omega }\right)$, the ensuing inequality can be established:

$$\left|{\int }_{\mathsf{\Omega }}uvdx\right|⩽\left(\frac{1}{p^{-}}+\frac{1}{{\left(p^{-}\right)}^{\prime }}\right){\parallel u\parallel }_{p\left(x\right)}{\parallel v\parallel }_{p^{\prime }\left(x\right)}.$$

Remark 2.

Similar to Propositions 2 and 3, the above inequalities are also true for ${\int }_{\partial \mathsf{\Omega }}{\left|u\left(x\right)\right|}^{z\left(x\right)}dS$.

Proposition 4 

([29]). Assume $1⩽p\left(x\right),r\left(x\right)<\infty ,f\in C(\mathsf{\Omega }\times {\mathbb{R}}^2)$ and

$$f(x,s,t)⩽c_1\left({\left|s\right|}^{\frac{p\left(x\right)}{r\left(x\right)}}+{\left|t\right|}^{\frac{p\left(x\right)}{r\left(x\right)}}\right).$$

Then, for every  $(u,v)\in {\left(L^{p\left(x\right)}\left(\mathsf{\Omega }\right)\right)}^2,f(·,u,v)\in L^{r\left(x\right)}\left(\mathsf{\Omega }\right)$and the operator

$$T_1:{\left(L^{p\left(x\right)}\left(\mathsf{\Omega }\right)\right)}^2\to L^{r\left(x\right)}\left(\mathsf{\Omega }\right):(u,v)\mapsto f(x,u,v)$$

is continuous.

3. Limit Index Theory

To solve the problem, we have to recall limit index theory [16]. Set Z is a G-Banach space; for detailed descriptions of both space Z and topological group G, refer to [30]. To understand the definition of index i, we direct our attention to reference [31]. The definitions and propositions introduced below play an important role in this paper, which are related to the index.

Definition 1 

([30]). An index is considered to conform to the d-dimension property when a positive integer d exists, ensuring that

$$i(V^{dk}\cap S_1)=k,$$

for all $dk$-dimensional subspaces $V^{dk}\in \Sigma$ such that $V^{dk}\cap FixG=\left\{0\right\}$, where $S_1$ is the unit sphere in Z.

Suppose that U and V are closed subspaces of Z, both of which are invariant under the action of G. Now, consider that

$$Z=U\oplus V,$$

where V is infinite dimensional and

$$V=\overline{\bigcup _{j=1}^{\infty }{V}_j},$$

where $V_j$ is a $d{n}_j$-dimensional G-invariant subspace of V, $j=1,2,\cdots$, and $V_1\subset V_2\subset \cdots V_n\subset \cdots$. Set

$$Z_j=U\oplus V_j,$$

and $\forall A\in \Sigma$, and set

$$A_j=A\oplus Z_j.$$

Proposition 5 

([16]). If $A,B\in \Sigma$, $i^{\infty }$ meets the following conditions:

$\left(i\right)$

$A=\varnothing \Rightarrow i^{\infty }=-\infty$;

$\left(ii\right)$

$A\subset B\Rightarrow i^{\infty }\left(A\right)⩽i^{\infty }\left(B\right)$;

$\left(iii\right)$

$i^{\infty }(A\bigcup B)⩽i^{\infty }\left(A\right)+i^{\infty }\left(B\right)$;

$\left(iv\right)$

$i^{\infty }(S_p\bigcap V)=0$ if $V\bigcap FixG=\left\{0\right\}$, where $S_p=\{z\in Z:∥z∥=\rho \}$;

$\left(v\right)$

If  $Y_0=\widetilde{Y_0}$  are  G-invariant closed subspaces of V, where $V=Y_0⨁\widetilde{Y_0}$ and $\widetilde{Y_0}\subset V_{j_0}$ for some $j_0$, with $dim\widetilde{Y_0}=dm$, then the $i^{\infty }(S_p\bigcap Y_0)⩾-m$.

Definition 2 

([16]). A functional $J\in C^1(Z,R)$ is said to satisfy the ${\left(PS\right)}_c^{*}$ condition if every sequence $\left\{u_{n_k}\right\}$ satisfying

$$u_{n_k}\in Z_{n_k},J\left(u_{n_k}\right)\to c,d{J}_{n_k}\left(u_{n_k}\right)\to 0,\mathrm{as}k\to \infty ,$$

(5)

possesses a subsequence that converges in Z to a critical point of J, where $Z_{n_k}$ is the $n_k$-dimension subspace of Z, $J_{n_k}={\left.J\right|}_{Z_{n_k}}$.

Theorem 2 

([16]). Suppose that

($A_1$)

$J\in C^1(Z,R)$  is G-invariant.

($A_2$)

If U and V are G-invariant, closed subspaces, then V is infinite-dimensional, where $Z=U\oplus V$.

($A_3$)

If there is a sequence of G-invariant, finite-dimensional subspaces $V_1\subset V_2\subset \cdots \subset V_j\subset \cdots ,dim{V}_j=d{n}_j$, then $V=\overline{{\cup }_{j=1}^{\infty }{V}_j}$.

($A_4$)

An index theory, i on Z, exists that satisfies the property of the d-dimension.

($A_5$)

If $Y_0,\widetilde{Y_0},Y_1$ of V are G-invariant subspaces, then $V=$$Y_0\oplus \widetilde{Y_0},Y_1,\widetilde{Y_0}\subset V_{j_0}$ for some $j_0$ and $dim\widetilde{Y_0}=dm<dk=dim{Y}_1$.

($A_6$)

If there exist α and $\beta ,\alpha <\beta$, then J fulfills ${\left(PS\right)}_c^{*},\forall c\in [\alpha ,\beta ]$.

($A_7$)

$$\left\{\begin{array}{c}\left(1\right)\mathrm{either}\mathrm{Fix}G\subset U\oplus Y_1,\mathrm{or}\mathrm{Fix}G\cap V=\left\{0\right\},\hfill \\ \left(2\right)\mathrm{there}\mathrm{is}\rho >0\mathrm{such}\mathrm{that}\forall u\in Y_0\cap S_{\rho },J\left(u\right)⩾\alpha ,\hfill \\ \left(3\right)\forall z\in U\oplus Y_1,J\left(z\right)⩽\beta ,\hfill \end{array}\right.$$

If the limit index that corresponds to i is $i^{\infty }$, then the numbers

$$c_j=\underset{i^{\infty }\left(A\right)⩾j}{inf}\underset{z\in A}{sup}J\left(u\right),-k+1⩽j⩽-m,$$

are critical values of J, and $\alpha ⩽c_{-k+1}⩽\cdots ⩽c_{-m}⩽\beta$. Furthermore, while $c=c_l=\cdots =c_{l+r},r⩾0$, $i\left(K_c\right)⩾r+1$, where $K_c=\{z\in Z:dJ\left(z\right)=$$0,J\left(z\right)=c\}$.

4. Construction of the Index

The definiton of an energy function related to problem (1) is as follows:

$$\begin{array}{cc}\hfill \tilde{J}(u,v)=& -\widehat{M}\left({\int }_{\mathsf{\Omega }}\frac{{|\nabla u|}^{p\left(x\right)}+{\left|u\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right)+\widehat{M}\left({\int }_{\mathsf{\Omega }}\frac{{|\nabla v|}^{p\left(x\right)}+{\left|v\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right)\hfill \\ \hfill & -{\int }_{\partial \mathsf{\Omega }}\frac{1}{p_{*}\left(x\right)}{\left|u\right|}^{p_{*}\left(x\right)}dS-{\int }_{\partial \mathsf{\Omega }}\frac{1}{p_{*}\left(x\right)}{\left|v\right|}^{p_{*}\left(x\right)}dS-{\int }_{\mathsf{\Omega }}F(x,u,v)dx,\hfill \end{array}$$

(6)

for $(u,v)\in W^{1,p\left(x\right)}\left(\mathsf{\Omega }\right)\times W^{1,p\left(x\right)}\left(\mathsf{\Omega }\right)$.

$$\begin{array}{cc}\hfill & 〈d\tilde{J}(u,v),(\widehat{u},\widehat{v})〉\hfill \\ \hfill & =-M\left({\int }_{\mathsf{\Omega }}\frac{{|\nabla u|}^{p\left(x\right)}+{\left|u\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right){\int }_{\mathsf{\Omega }}{\left(\right|\nabla u|}^{p\left(x\right)-2}\nabla u\nabla \widehat{u}+{\left|u\right|}^{p\left(x\right)-2}u\widehat{u})dx\hfill \\ \hfill & +M\left({\int }_{\mathsf{\Omega }}\frac{{|\nabla v|}^{p\left(x\right)}+{\left|v\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right){\int }_{\mathsf{\Omega }}{\left(\right|\nabla v|}^{p\left(x\right)-2}\nabla v\nabla \widehat{v}+{\left|v\right|}^{p\left(x\right)-2}v\widehat{v})dx\hfill \\ \hfill & -{\int }_{\partial \mathsf{\Omega }}{\left|u\right|}^{p_{*}\left(x\right)-2}u\widehat{u}dS-{\int }_{\partial \mathsf{\Omega }}{\left|v\right|}^{p_{*}\left(x\right)-2}v\widehat{v}dS\hfill \\ \hfill & -{\int }_{\mathsf{\Omega }}{F}_u(x,u,v)\widehat{u}dx-{\int }_{\mathsf{\Omega }}{F}_v(x,u,v)\widehat{v}dx,\hfill \end{array}$$

(7)

for every $(\widehat{u},\widehat{v})\in W^{1,p\left(x\right)}\left(\mathsf{\Omega }\right)\times W^{1,p\left(x\right)}\left(\mathsf{\Omega }\right)$.

Now, take $G_1$ to be the group of orthogonal linear transformations in ${\mathbb{R}}^N$, where $G_1=O\left(N\right)$.

$$E_{G_1}:=W_{O\left(N\right)}^{1,p\left(x\right)}\left(\mathsf{\Omega }\right)=\left\{u\in W^{1,p\left(x\right)}\left(\mathsf{\Omega }\right):gu\left(x\right)=u\left(g^{-1}x\right)=u\left(x\right),g\in O\left(N\right)\right\}.$$

Denote $X=E_{G_1}\times E_{G_1}$. The condition $\left(F_1\right)$ indicates that $\tilde{J}$ is $O\left(N\right)$-invariant. According to [32], we may deduce that $(u,v)$ is a critical point of $\tilde{J}$ precisely when it is a critical point for $J=\tilde{J}{|}_X$. Therefore, demonstrating the existence of critical points of J within X is sufficient.

In accordance with [33], there exists a Schauder basis, ${\left\{e_n^{\prime }\right\}}_{n=1}^{\infty }$, for $W^{1,p\left(x\right)}\left(\mathsf{\Omega }\right)$. Let $e_n={\int }_{O\left(N\right)}{e}_n^{\prime }\left(g\left(x\right)\right)d{\mu }_g$, selecting one in identical elements where necessary. It is obvious that ${\left\{e_n\right\}}_{n=1}^{\infty }$ is a Schauder basis for $E_{G_1}$, since $E_{G_1}$ is reflexive, and

$$e_n^{*}\left(e_m\right)={\delta }_{n,m}=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}n=m,\hfill \\ 0,\hfill & \mathrm{if}n\ne m,\hfill \end{array}\right.$$

forms a basis for $E_{G_1}^{*}$. Set

$$E_{G_1}^{\left(n\right)}=span\left\{e_1,\dots ,e_n\right\},E_{G_1}^{{\left(n\right)}^{\perp }}=\overline{span\left\{e_{n+1},\dots \right\}}$$

and

$$E_{G_1}^{*\left(n\right)}=span\left\{e_1^{*},\dots ,e_n^{*}\right\}.$$

Let $P_n:E_{G_1}\to E_{G_1}^{\left(n\right)}$ be the projector corresponding to decomposition $E_{G_1}=E_{G_1}^{\left(n\right)}\oplus E_{G_1}^{{\left(n\right)}^{\perp }}$ and let $P_n^{*}:E_{G_1}^{*}\to E_{G_1}^{*\left(n\right)}$ be the projector corresponding to the decomposition, and $E_{G_1}^{*}=E_{G_1}^{*\left(n\right)}\oplus E_{G_1}^{*{\left(n\right)}^{\perp }}$. Then, $P_{n}u\to u,P_n^{*}{v}^{*}\to v^{*}$ for any $u\in E_{G_1},v^{*}\in E_{G_1}^{*}$ as $n\to \infty$ and $〈P_n^{*}{v}^{*},u〉=〈v^{*},P_{n}u〉$.

Now, Denote $X_n=E_{G_1}\times E_{G_1}^{\left(n\right)}$. Through setting $\tau (u,v)=(-u,-v)$, we then define a group action, $G_2=\{1,\tau \}\cong {\mathbb{Z}}_2$; thus, fix $G=\left\{0\right\}\times \left\{0\right\}$. Define the following:

$$\Sigma :=\{A\subset X\setminus \{0\}:A\mathrm{is}\mathrm{closed}\mathrm{in}X\mathrm{and}(u,v)\in A\Rightarrow (-u,-v)\in A\}.$$

Define an index $\gamma$ on $\Sigma$ with

$$\gamma \left(A\right)=\left\{\begin{array}{c}min\left\{N\in \mathbb{Z}:\exists h\in C(A,{\mathbb{R}}^N\setminus \left\{0\right\})\mathrm{such}\mathrm{that}h(-u,-v)=h(u,v)\right\},\hfill \\ 0,\mathrm{if}A=\varnothing ,\hfill \\ +\infty ,\mathrm{if}\mathrm{such}anh\mathrm{does}\mathrm{not}\mathrm{exist}.\hfill \end{array}\right.$$

After that, we derive the statement from reference [34]: We ascertain that $\gamma$ is an index that aligns with the attributes outlined in Definition 5.9 in reference [31]. Furthermore, $\gamma$ meets the conditions of a one-dimension property. By applying Definition 2.4 in [16], we derive a limit index, ${\gamma }^{\infty }$, in relation to $\left(X_n\right)$ from $\gamma$.

5. Local Palais–Smale Condition

Lemma 1.

Suppose that conditions $\left(M_1\right)-\left(M_2\right)$,$\left(F_1\right)-\left(F_3\right)$ hold; $\left\{\left(u_{n_k},v_{n_k}\right)\right\}$ is a ${\left(PS\right)}_c^{*}$ sequence that satisfies (5), and then $\left\{\left(u_{n_k},v_{n_k}\right)\right\}$ is bounded in X.

Proof. 

Let ${∥u_{n_k}∥}_{1,p\left(x\right)}>1$ and ${∥v_{n_k}∥}_{1,p\left(x\right)}>1$ for any integer n. Using conditions $\left(M_1\right),\left(F_2\right)$ and Proposition 2, we have

$$\begin{array}{cc}\hfill & o\left(1\right){∥u_{n_k}∥}_{1,p\left(x\right)}\hfill \\ \hfill & ⩾-〈\mathrm{d}{J}_{n_k}\left(u_{n_k},v_{n_k}\right),\left(u_{n_k},0\right)〉\hfill \\ \hfill & =M\left({\int }_{\mathsf{\Omega }}\frac{|\nabla u_{n_k}{|}^{p\left(x\right)}+{\left|u_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right){\int }_{\mathsf{\Omega }}({\left|\nabla u_{n_k}\right|}^{p\left(x\right)}+{\left|u_{n_k}\right|}^{p\left(x\right)})\mathrm{d}x+{\int }_{\partial \mathsf{\Omega }}{\left|u_{n_k}\right|}^{p_{*}\left(x\right)}\mathrm{d}S\hfill \\ \hfill & +{\int }_{\mathsf{\Omega }}{F}_u\left(x,u_{n_k},v_{n_k}\right)u_{n_k}\mathrm{d}x\hfill \\ \hfill & ⩾m_0{\int }_{\mathsf{\Omega }}({\left|\nabla u_{n_k}\right|}^{p\left(x\right)}+{\left|u_{n_k}\right|}^{p\left(x\right)})\mathrm{d}x+{\int }_{\partial \mathsf{\Omega }}{\left|u_{n_k}\right|}^{p_{*}\left(x\right)}\mathrm{d}S\hfill \\ \hfill & ⩾m_0{∥u_{n_k}∥}_{1,p\left(x\right)}^{p^{-}}+{\int }_{\partial \mathsf{\Omega }}{\left|u_{n_k}\right|}^{p_{*}\left(x\right)}\mathrm{d}S,\hfill \end{array}$$

since $p^{-}>1$, we can infer that ${∥u_{n_k}∥}_{1,p\left(x\right)}$ is bounded. Based on $\left(M_1\right),\left(M_2\right)$, and $\left(F_3\right)$, we have

$$\begin{array}{cc}\hfill & c+{∥u_{n_k}∥}_{1,p\left(x\right)}+{∥v_{n_k}∥}_{1,p\left(x\right)}\hfill \\ \hfill & ⩾J\left(u_{n_k},v_{n_k}\right)-\langle \mathrm{d}{J}_{n_k}\left(u_{n_k},v_{n_k}\right),(\frac{1}{p^{-}}{u}_{n_k},\frac{1}{p_{*}^{-}}{v}_{n_k})\rangle \hfill \\ \hfill & =-\widehat{M}\left({\int }_{\mathsf{\Omega }}\frac{|\nabla u_{n_k}{|}^{p\left(x\right)}+{\left|u_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right)+\widehat{M}\left({\int }_{\mathsf{\Omega }}\frac{|\nabla v_{n_k}{|}^{p\left(x\right)}+{\left|v_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right)\hfill \\ \hfill & -{\int }_{\partial \mathsf{\Omega }}\frac{1}{p_{*}\left(x\right)}|u_{n_k}{|}^{p_{*}\left(x\right)}dS-{\int }_{\partial \mathsf{\Omega }}\frac{1}{p_{*}\left(x\right)}{\left|v_{n_k}\right|}^{p_{*}\left(x\right)}dS-{\int }_{\mathsf{\Omega }}F(x,u_{n_k},v_{n_k})dx\hfill \\ \hfill & +\frac{1}{p^{-}}M\left({\int }_{\mathsf{\Omega }}\frac{|\nabla u_{n_k}{|}^{p\left(x\right)}+{\left|u_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right){\int }_{\mathsf{\Omega }}\left(\right|\nabla u_{n_k}{|}^{p\left(x\right)}+|u_{n_k}{|}^{p\left(x\right)})dx\hfill \\ \hfill & -\frac{1}{p_{*}^{-}}M\left({\int }_{\mathsf{\Omega }}\frac{|\nabla v_{n_k}{|}^{p\left(x\right)}+{\left|v_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right){\int }_{\mathsf{\Omega }}\left(\right|\nabla v_{n_k}{|}^{p\left(x\right)}+|v_{n_k}{|}^{p\left(x\right)})dx\hfill \\ \hfill & +\frac{1}{p^{-}}{\int }_{\partial \mathsf{\Omega }}|u_{n_k}{|}^{p_{*}\left(x\right)}dS+\frac{1}{p_{*}^{-}}{\int }_{\partial \mathsf{\Omega }}{\left|v_{n_k}\right|}^{p_{*}\left(x\right)}dS\hfill \\ \hfill & +\frac{1}{p^{-}}{\int }_{\mathsf{\Omega }}{F}_u(x,u_{n_k},v_{n_k})u_{n_k}dx+\frac{1}{p_{*}^{-}}{\int }_{\mathsf{\Omega }}{F}_v(x,u_{n_k},v_{n_k})v_{n_k}dx.\hfill \end{array}$$

Since ${∥u_{n_k}∥}_{1,p\left(x\right)}$ is bounded,

$$\widehat{M}\left({\int }_{\mathsf{\Omega }}\frac{|\nabla u_{n_k}{|}^{p\left(x\right)}+{\left|u_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right),$$

$$M\left({\int }_{\mathsf{\Omega }}\frac{|\nabla u_{n_k}{|}^{p\left(x\right)}+{|u_{n_k}|}^{p\left(x\right)}}{p\left(x\right)}dx\right){\int }_{\mathsf{\Omega }}\left(\right|\nabla u_{n_k}{|}^{p\left(x\right)}+|u_{n_k}{|}^{p\left(x\right)})dx$$

are also bounded. Thus, there exists $C>0$, and we can obtain

$$\begin{array}{cc}\hfill & \frac{1}{p^{-}}M\left({\int }_{\mathsf{\Omega }}\frac{|\nabla u_{n_k}{|}^{p\left(x\right)}+{\left|u_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right){\int }_{\mathsf{\Omega }}\left(\right|\nabla u_{n_k}{|}^{p\left(x\right)}+|u_{n_k}{|}^{p\left(x\right)})dx\hfill \\ \hfill & -\widehat{M}\left({\int }_{\mathsf{\Omega }}\frac{|\nabla u_{n_k}{|}^{p\left(x\right)}+{\left|u_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right)⩾-C.\hfill \end{array}$$

Therefore,

$$\begin{array}{cc}\hfill & c+{∥u_{n_k}∥}_{1,p\left(x\right)}+{∥v_{n_k}∥}_{1,p\left(x\right)}\hfill \\ \hfill & ⩾\frac{1}{\theta }M\left({\int }_{\mathsf{\Omega }}\frac{|\nabla v_{n_k}{|}^{p\left(x\right)}+{\left|v_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right){\int }_{\mathsf{\Omega }}\frac{|\nabla v_{n_k}{|}^{p\left(x\right)}+{\left|v_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\hfill \\ \hfill & -\frac{1}{p_{*}^{-}}M\left({\int }_{\mathsf{\Omega }}\frac{|\nabla v_{n_k}{|}^{p\left(x\right)}+{\left|v_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right){\int }_{\mathsf{\Omega }}\left(\right|\nabla v_{n_k}{|}^{p\left(x\right)}+|v_{n_k}{|}^{p\left(x\right)})dx\hfill \\ \hfill & -\frac{1}{p_{*}^{-}}{\int }_{\partial \mathsf{\Omega }}|u_{n_k}{|}^{p_{*}\left(x\right)}dS+\frac{1}{p^{-}}{\int }_{\partial \mathsf{\Omega }}|u_{n_k}{|}^{p_{*}\left(x\right)}dS-\frac{1}{p_{*}^{-}}{\int }_{\partial \mathsf{\Omega }}{\left|v_{n_k}\right|}^{p_{*}\left(x\right)}dS\hfill \\ \hfill & +\frac{1}{p_{*}^{-}}{\int }_{\partial \mathsf{\Omega }}{\left|v_{n_k}\right|}^{p_{*}\left(x\right)}dS-{\int }_{\mathsf{\Omega }}F(x,u_{n_k},v_{n_k})dx+{\int }_{\mathsf{\Omega }}F(x,u_{n_k},v_{n_k})dx-C\hfill \\ \hfill & ⩾(\frac{1}{\theta p^{+}}-\frac{1}{p_{*}^{-}})m_0{∥v_{n_k}∥}_{1,p\left(x\right)}^{p^{-}}-C.\hfill \end{array}$$

This implies that $\left\{v_{n_k}\right\}$ is bounded in $E_{G_1}$ since $p^{-}>1$. Thus, $\left\{\left(u_{n_k},v_{n_k}\right)\right\}$ is bounded in X. □

Due to the boundedness of $\left\{u_{n_k}\right\}$ and $\left\{v_{n_k}\right\}$ in $E_{G_1}$, up to a subsequence,

$$\begin{array}{cc}\hfill & u_{n_k}\rightharpoonup u,v_{n_k}\rightharpoonup v\mathrm{in}{E}_{G_1},\hfill \\ \hfill & u_{n_k}\to u\mathrm{a}.\mathrm{e}.\mathrm{on}\mathsf{\Omega },\hfill \\ \hfill & v_{n_k}\to v\mathrm{a}.\mathrm{e}.\mathrm{on}\mathsf{\Omega }.\hfill \end{array}$$

(8)

In addition, we can presume that

$$\begin{array}{cc}\hfill & {\int }_{\mathsf{\Omega }}\frac{|\nabla u_{n_k}{|}^{p\left(x\right)}+{\left|u_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\to t_1⩾0,\mathrm{as}k\to \infty ,\hfill \\ \hfill & {\int }_{\mathsf{\Omega }}\frac{|\nabla v_{n_k}{|}^{p\left(x\right)}+{\left|v_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\to t_2⩾0,\mathrm{as}k\to \infty .\hfill \end{array}$$

Accordingly, we obtain the next lemma:

Lemma 2.

Assume that $\left(F_1\right)-\left(F_3\right)$ hold; then, for every $(\widehat{u},\widehat{v})\in X$, we can get

(1) 

$$\begin{array}{cc}\hfill & {\int }_{\mathsf{\Omega }}|\nabla u_{n_k}{|}^{p\left(x\right)-2}\nabla u_{n_k}\nabla \widehat{u}dx\to {\int }_{\mathsf{\Omega }}{|\nabla u|}^{p\left(x\right)-2}\nabla u\nabla \widehat{u}dx,\hfill \\ \hfill & {\int }_{\mathsf{\Omega }}|\nabla v_{n_k}{|}^{p\left(x\right)-2}\nabla v_{n_k}\nabla \widehat{v}dx\to {\int }_{\mathsf{\Omega }}{|\nabla v|}^{p\left(x\right)-2}\nabla v\nabla \widehat{v}dx.\hfill \end{array}$$

(9)

(2) 

$$\begin{array}{c}\hfill {\int }_{\mathsf{\Omega }}{F}_u\left(x,u_{n_k},v_{n_k}\right)\widehat{u}dx\to {\int }_{\mathsf{\Omega }}{F}_u(x,u,v)\widehat{u}dx,\\ \hfill {\int }_{\mathsf{\Omega }}{F}_v\left(x,u_{n_k},v_{n_k}\right)\widehat{v}dx\to {\int }_{\mathsf{\Omega }}{F}_v(x,u,v)\widehat{v}dx.\end{array}$$

(10)

(3) 

$$\begin{array}{cc}\hfill & -M\left(t_1\right){\int }_{\mathsf{\Omega }}{\left(\right|\nabla u|}^{p\left(x\right)-2}\nabla u\nabla \widehat{u}+{\left|u\right|}^{p\left(x\right)-2}u\widehat{u})dx+M\left(t_2\right){\int }_{\mathsf{\Omega }}{\left(\right|\nabla v|}^{p\left(x\right)-2}\nabla v\nabla \widehat{v}+{\left|v\right|}^{p\left(x\right)-2}v\widehat{v})dx\hfill \\ \hfill & -{\int }_{\partial \mathsf{\Omega }}{\left|u\right|}^{p_{*}\left(x\right)-2}u\widehat{u}dS-{\int }_{\partial \mathsf{\Omega }}{\left|v\right|}^{p_{*}\left(x\right)-2}v\widehat{v}dS-{\int }_{\mathsf{\Omega }}{F}_u(x,u,v)\widehat{u}dx-{\int }_{\mathsf{\Omega }}{F}_v(x,u,v)\widehat{v}dx=0.\hfill \end{array}$$

(11)

Proof. 

(1) To verify (9), we recognize renowned inequalities

$$\left\{\begin{array}{cccc}\hfill & 〈{\left|x\right|}^{p\left(x\right)-2}x-{\left|y\right|}^{p\left(x\right)-2}y,x-y〉⩾C{|x-y|}^{p\left(x\right)},\hfill & \hfill & \mathrm{if}p\left(x\right)⩾2,\hfill \\ \hfill & {\left(\left|x\right|+\left|y\right|\right)}^{2-p\left(x\right)}〈{\left|x\right|}^{p\left(x\right)-2}x-{\left|y\right|}^{p\left(x\right)-2}y,x-y〉⩾C{|x-y|}^2,\hfill & \hfill & \mathrm{if}1<p\left(x\right)<2.\hfill \end{array}\right.$$

for a constant $C>0$. Define

$$\begin{array}{cc}\hfill & P_1\left(x\right)=〈|\nabla u_{n_k}{|}^{p\left(x\right)-2}\nabla u_{n_k}-{|\nabla u|}^{p\left(x\right)-2}\nabla u,\nabla u_{n_k}-\nabla u〉\left(x\right)⩾0,\hfill \\ \hfill & P_2\left(x\right)=〈|\nabla v_{n_k}{|}^{p\left(x\right)-2}\nabla v_{n_k}-{|\nabla v|}^{p\left(x\right)-2}\nabla v,\nabla v_{n_k}-\nabla v〉\left(x\right)⩾0.\hfill \end{array}$$

Let $\psi$ be a $C^{\infty }$ function such that $0⩽\psi ⩽1$; then, for every $R>0$,

$$\psi \equiv 1\mathrm{in}{B}_R\left(0\right)and\psi \equiv 0\mathrm{in}\mathsf{\Omega }\setminus B_{2R}\left(0\right).$$

Observing that $〈dJ(u_{n_k},v_{n_k}),((u_{n_k}-u)\psi ,0)〉\to 0$, we have

$$\begin{array}{cc}\hfill & 〈dJ(u_{n_k},v_{n_k}),((u_{n_k}-u)\psi ,0)〉\hfill \\ \hfill & =-M\left({\int }_{\mathsf{\Omega }}\frac{|\nabla u_{n_k}{|}^{p\left(x\right)}+{\left|u_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right){\int }_{\mathsf{\Omega }}({|\nabla u_{n_k}|}^{p\left(x\right)-2}\nabla u_{n_k}\nabla (u_{n_k}-u)\psi \hfill \\ \hfill & +|\nabla u_{n_k}{|}^{p\left(x\right)-2}\nabla u_{n_k}(u_{n_k}-u)\nabla \psi +{\left|u_{n_k}\right|}^{p\left(x\right)-2}{u}_{n_k}(u_{n_k}-u)\psi )dx\hfill \\ \hfill & -{\int }_{\partial \mathsf{\Omega }}{\left|u_{n_k}\right|}^{p_{*}\left(x\right)-2}{u}_{n_k}(u_{n_k}-u)\psi dS-{\int }_{\mathsf{\Omega }}{F}_u(x,u_{n_k},v_{n_k})(u_{n_k}-u)\psi dx\hfill \\ \hfill & =-M\left({\int }_{\mathsf{\Omega }}\frac{|\nabla u_{n_k}{|}^{p\left(x\right)}+{\left|u_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right){\int }_{\mathsf{\Omega }}(P_1\left(x\right)\psi +{|\nabla u|}^{p\left(x\right)-2}\nabla u\nabla (u_{n_k}-u)\psi \hfill \\ \hfill & +|\nabla u_{n_k}{|}^{p\left(x\right)-2}\nabla u_{n_k}(u_{n_k}-u)\nabla \psi +{\left|u_{n_k}\right|}^{p\left(x\right)-2}{u}_{n_k}(u_{n_k}-u)\psi )dx\hfill \\ \hfill & -{\int }_{\partial \mathsf{\Omega }}{\left|u_{n_k}\right|}^{p_{*}\left(x\right)-2}{u}_{n_k}(u_{n_k}-u)\psi dS-{\int }_{\mathsf{\Omega }}{F}_u(x,u_{n_k},v_{n_k})(u_{n_k}-u)\psi dx\to 0.\hfill \end{array}$$

From the Hölder inequality, the boundedness of $u_{n_k}$ in $E_{G_1}$, and Remark 1, we derive the following:

$$\begin{array}{cc}\hfill & {\int }_{\mathsf{\Omega }}{|\nabla u_{n_k}|}^{p\left(x\right)-2}\nabla u_{n_k}(u_{n_k}-u)\nabla \psi dx\hfill \\ \hfill & ⩽{∥{\left|\nabla u_{n_k}\right|}^{p\left(x\right)-1}∥}_{L^{p^{\prime }\left(x\right)}}{∥u_{n_k}-u∥}_{L^{p\left(x\right)}}{∥\nabla \psi ∥}_{\infty }\hfill \\ \hfill & ⩽C^{\prime }{∥u_{n_k}-u∥}_{L^{p\left(x\right)}}\to 0.\hfill \end{array}$$

Similarly, we can also get

$$\begin{array}{cc}\hfill & {\int }_{\mathsf{\Omega }}{\left|u_{n_k}\right|}^{p\left(x\right)-2}{u}_{n_k}(u_{n_k}-u)\psi dx\to 0,\hfill \\ \hfill & {\int }_{\partial \mathsf{\Omega }}{\left|u_{n_k}\right|}^{p_{*}\left(x\right)-2}{u}_{n_k}(u_{n_k}-u)\psi dS\to 0.\hfill \end{array}$$

From Proposition 2, the assumptions of continuity for F, and $\left(F_3\right)$, we have the following:

$$\begin{array}{cc}\hfill & {\int }_{\mathsf{\Omega }}{F}_u(x,u_{n_k},v_{n_k})(u_{n_k}-u)\psi dx\hfill \\ \hfill & ⩽{\int }_{\mathsf{\Omega }}{F}_u(x,u_{n_k},v_{n_k})(u_{n_k}-u)dx\hfill \\ \hfill & ⩽{\int }_{\mathsf{\Omega }}{C}_1\left(|u_{n_k}{|}^{r\left(x\right)}+{\left|v_{n_k}\right|}^{r\left(x\right)}\right)\left(u_{n_k}-u\right)dx+C_2{\int }_{\mathsf{\Omega }}\left(u_{n_k}-u\right)dx\hfill \\ \hfill & ⩽C_1{\int }_{\mathsf{\Omega }}|u_{n_k}{|}^{r\left(x\right)}\left(u_{n_k}-u\right)dx+C_1{\int }_{\mathsf{\Omega }}{\left|v_{n_k}\right|}^{r\left(x\right)}\left(u_{n_k}-u\right)dx\hfill \\ \hfill & +C_2{\int }_{\mathsf{\Omega }}\left(u_{n_k}-u\right)dx\hfill \\ \hfill & ⩽C_1{∥|u_{n_k}{|}^{r\left(x\right)}∥}_{L^{p^{\prime }\left(x\right)}}{∥u_{n_k}-u∥}_{L^{p\left(x\right)}}+C_1{∥|v_{n_k}{|}^{r\left(x\right)}∥}_{L^{p^{\prime }\left(x\right)}}{∥u_{n_k}-u∥}_{L^{p\left(x\right)}}\hfill \\ \hfill & +C_2{\int }_{\mathsf{\Omega }}\left(u_{n_k}-u\right)dx\to 0.\hfill \end{array}$$

In addition, since $u_{n_k}\rightharpoonup u\mathrm{in}{E}_{G_1}$,

$${\int }_{\mathsf{\Omega }}{|\nabla u|}^{p\left(x\right)-2}\nabla u\nabla (u_{n_k}-u)\psi dx\to 0.$$

From $\left(M_1\right)$, we can obtain

$${\int }_{\mathsf{\Omega }}{P}_1\left(x\right)\psi dx\to 0.$$

Then,

$$0⩽{\int }_{B_R\left(0\right)}{P}_1\left(x\right)dx⩽{\int }_{\mathsf{\Omega }}{P}_1\left(x\right)\psi dx\to 0.$$

Hence, we can get

$${\int }_{B_R\left(0\right)}\left(|\nabla u_{n_k}{|}^{p\left(x\right)-2}\nabla u_{n_k}-{|\nabla u|}^{p\left(x\right)-2}\nabla u\right)\left(\nabla u_{n_k}-\nabla u\right)dx\to 0.$$

If $p\left(x\right)⩾2$, we can obtain

$${\int }_{B_R\left(0\right)}{|\nabla u_{n_k}-\nabla u|}^{p\left(x\right)}dx⩽C{\int }_{B_R\left(0\right)}\left(|\nabla u_{n_k}{|}^{p\left(x\right)-2}\nabla u_{n_k}-{|\nabla u|}^{p\left(x\right)-2}\nabla u\right)\left(\nabla u_{n_k}-\nabla u\right)dx\to 0.$$

If $1<p\left(x\right)<2$, from Proposition 3, we have

$${\int }_{B_R\left(0\right)}{|\nabla u_{n_k}-\nabla u|}^{p\left(x\right)}dx⩽C{∥g_n∥}_{L^{\frac{2}{p\left(x\right)}}\left(B_R\left(0\right)\right)}{∥h_n∥}_{L^{\frac{2}{2-p\left(x\right)}}\left(B_R\left(0\right)\right)},$$

where

$$\begin{array}{cc}\hfill & g_n\left(x\right)=\frac{|\nabla u_{n_k}{\left(x\right)-\nabla u\left(x\right)|}^{p\left(x\right)}}{\left(\right|\nabla u_{n_k}{\left(x\right)|+|\nabla u\left(x\right)\left|\right)}^{\frac{p\left(x\right)(2-p(x\left)\right)}{2}}},\hfill \\ \hfill & h_n\left(x\right)={|\nabla u_{n_k}\left(x\right)+\nabla u\left(x\right)|}^{\frac{p\left(x\right)(2-p(x\left)\right)}{2}},\hfill \end{array}$$

and $C>0$. By computing directly, we note that $\left\{{∥h_n∥}_{L^{\frac{2}{2-p\left(x\right)}}\left(B_R\left(0\right)\right)}\right\}$ is a bounded sequence, and

$${\int }_{B_R\left(0\right)}{\left|g_n\right|}^{\frac{2}{p\left(x\right)}}dx⩽C{\int }_{B_R\left(0\right)}{P}_1\left(x\right)dx.$$

Thus,

$$\underset{n\to \infty }{lim}{\int }_{B_R\left(0\right)}{|\nabla u_{n_k}-\nabla u|}^{p\left(x\right)}dx=0.$$

Therefore, $\nabla u_{n_k}\to \nabla u$ in ${\left(L^{p\left(x\right)}\left(B_R\left(0\right)\right)\right)}^N$. Hence, up to a subsequence, $\nabla u_{n_k}\to \nabla u$$a.e.$ in $B_R\left(0\right)$. Since R is arbitrary, up to a subsequence, we have $\nabla u_{n_k}\to \nabla u$$a.e.$ in $\mathsf{\Omega }$. Because $\left(|\nabla u_{n_k}{|}^{p\left(x\right)-2}\nabla u_{n_k}\right)$ is bounded in ${\left(L^{\frac{p\left(x\right)}{p\left(x\right)-1}}\left(\mathsf{\Omega }\right)\right)}^N$, up to a subsequence, $|\nabla u_{n_k}{|}^{p\left(x\right)-2}\nabla u_{n_k}\rightharpoonup {|\nabla u|}^{p\left(x\right)-2}\nabla u$ in ${\left(L^{\frac{p\left(x\right)}{p\left(x\right)-1}}\left(\mathsf{\Omega }\right)\right)}^N$. Similarly, we can deduce that $|\nabla v_{n_k}{|}^{p\left(x\right)-2}\nabla v_{n_k}\rightharpoonup {|\nabla v|}^{p\left(x\right)-2}\nabla v$ in ${\left(L^{\frac{p\left(x\right)}{p\left(x\right)-1}}\left(\mathsf{\Omega }\right)\right)}^N$. Thus, (9) holds.

(2) From (8), we can get

$$(u_{n_k},v_{n_k})\to (u,v)\mathrm{in}\left(L^{p\left(x\right)}\left(\mathsf{\Omega }\right)\cap L^{m_1\left(x\right)}\left(\mathsf{\Omega }\right)\right)\times \left(L^{p\left(x\right)}\left(\mathsf{\Omega }\right)\cap L^{m_2\left(x\right)}\left(\mathsf{\Omega }\right)\right).$$

From Hölder inequality,

$$\begin{array}{cc}\hfill & {\int }_{\mathsf{\Omega }}\left|F_u\left(x,u_{n_k},v_{n_k}\right)\widehat{u}-F_u(x,u,v)\widehat{u}\right|\mathrm{d}x⩽2{\left|F_u\left(x,u_{n_k},v_{n_k}\right)-F_u(x,u,v)\right|}_{m_1^{\prime }}{\left|\widehat{u}\right|}_{m_1},\hfill \\ \hfill & {\int }_{\mathsf{\Omega }}\left|F_v\left(x,u_{n_k},v_{n_k}\right)\widehat{u}-F_v(x,u,v)\widehat{u}\right|\mathrm{d}x⩽2{\left|F_v\left(x,u_{n_k},v_{n_k}\right)-F_v(x,u,v)\right|}_{m_2^{\prime }}{\left|\widehat{v}\right|}_{m_2},\hfill \end{array}$$

where $\frac{1}{m_1^{\prime }\left(x\right)}+\frac{1}{m_1\left(x\right)}=1,\frac{1}{m_2^{\prime }\left(x\right)}+\frac{1}{m_2\left(x\right)}=1,m_1\left(x\right),m_2\left(x\right)<p^{*}\left(x\right)$.

From $\left(F_3\right)$ and Proposition 4, we have

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}{F}_u\left(x,u_{n_k},v_{n_k}\right)=F_u\left(x,u,v\right),\\ \hfill \underset{n\to \infty }{lim}{F}_v\left(x,u_{n_k},v_{n_k}\right)=F_v\left(x,u,v\right).\end{array}$$

Then, (10) holds. Using $\left(F_3\right)$, we can also obtain

$$\begin{array}{cc}\hfill & {\int }_{\mathsf{\Omega }}{F}_u(x,u_{n_k},v_{n_k})u_{n_k}dx\to {\int }_{\mathsf{\Omega }}{F}_u(x,u,v)udx,\mathrm{as}n\to \infty ,\hfill \\ \hfill & {\int }_{\mathsf{\Omega }}{F}_v(x,u_{n_k},v_{n_k})v_{n_k}dx\to {\int }_{\mathsf{\Omega }}{F}_v(x,u,v)vdx,\mathrm{as}n\to \infty .\hfill \end{array}$$

(12)

(3) Since $u_{n_k}\rightharpoonup u,v_{n_k}\rightharpoonup v\mathrm{in}{E}_{G_1}$, we also have

$$\begin{array}{cc}\hfill & {\int }_{\mathsf{\Omega }}|u_{n_k}{|}^{p\left(x\right)-2}{u}_{n_k}\widehat{u}dx\to {\int }_{\mathsf{\Omega }}{\left|u\right|}^{p\left(x\right)-2}u\widehat{u}dx,\hfill \\ \hfill & {\int }_{\mathsf{\Omega }}|v_{n_k}{|}^{p\left(x\right)-2}{v}_{n_k}\widehat{v}dx\to {\int }_{\mathsf{\Omega }}{\left|v\right|}^{p\left(x\right)-2}v\widehat{v}dx,\hfill \\ \hfill & {\int }_{\partial \mathsf{\Omega }}|u_{n_k}{|}^{p_{*}\left(x\right)-2}{u}_{n_k}\widehat{u}dS\to {\int }_{\partial \mathsf{\Omega }}{\left|u\right|}^{p_{*}\left(x\right)-2}u\widehat{u}dS,\hfill \\ \hfill & {\int }_{\partial \mathsf{\Omega }}|v_{n_k}{|}^{p_{*}\left(x\right)-2}{v}_{n_k}\widehat{v}dS\to {\int }_{\partial \mathsf{\Omega }}{\left|v\right|}^{p_{*}\left(x\right)-2}v\widehat{v}dS.\hfill \end{array}$$

Observing the continuity of $M\left(t\right)$, we can get

$$\begin{array}{cc}\hfill & M\left({\int }_{\mathsf{\Omega }}\frac{|\nabla u_{n_k}{|}^{p\left(x\right)}+{\left|u_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right)\to M\left(t_1\right)⩾m_0>0,\mathrm{as}k\to \infty ,\hfill \\ \hfill & M\left({\int }_{\mathsf{\Omega }}\frac{|\nabla v_{n_k}{|}^{p\left(x\right)}+{\left|v_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right)\to M\left(t_2\right)⩾m_0>0,\mathrm{as}k\to \infty .\hfill \end{array}$$

From (7), it is evident that

$$\begin{array}{cc}\hfill & -M\left({\int }_{\mathsf{\Omega }}\frac{|\nabla u_{n_k}{|}^{p\left(x\right)}+{\left|u_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right){\int }_{\mathsf{\Omega }}\left(\right|\nabla u_{n_k}{|}^{p\left(x\right)-2}\nabla u_{n_k}\nabla \widehat{u}+|u_{n_k}{|}^{p\left(x\right)-2}{u}_{n_k}\widehat{u})dx\hfill \\ \hfill & +M\left({\int }_{\mathsf{\Omega }}\frac{|\nabla v_{n_k}{|}^{p\left(x\right)}+{\left|v_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right){\int }_{\mathsf{\Omega }}\left(\right|\nabla v_{n_k}{|}^{p\left(x\right)-2}\nabla v_{n_k}\nabla \widehat{v}+|v_{n_k}{|}^{p\left(x\right)-2}{v}_{n_k}\widehat{v})dx\hfill \\ \hfill & -{\int }_{\partial \mathsf{\Omega }}|u_{n_k}{|}^{p_{*}\left(x\right)-2}{u}_{n_k}\widehat{u}dS-{\int }_{\partial \mathsf{\Omega }}{\left|v_{n_k}\right|}^{p_{*}\left(x\right)-2}{v}_{n_k}\widehat{v}dS\hfill \\ \hfill & -{\int }_{\mathsf{\Omega }}{F}_u(x,u_{n_k},v_{n_k})\widehat{u}dx-{\int }_{\mathsf{\Omega }}{F}_v(x,u_{n_k},v_{n_k})\widehat{v}dx\to 0,\hfill \end{array}$$

and then (11) holds.

Set $(\widehat{u},\widehat{v})=(u,0)$ in (11); then, the following equation holds:

$$\begin{array}{cc}\hfill & M\left(t_1\right){\int }_{\mathsf{\Omega }}{\left(\right|\nabla u|}^{p\left(x\right)}+{\left|u\right|}^{p\left(x\right)})dx+{\int }_{\partial \mathsf{\Omega }}{\left|u\right|}^{p_{*}\left(x\right)}dS\hfill \\ \hfill & +{\int }_{\mathsf{\Omega }}{F}_u(x,u,v)udx=0.\hfill \end{array}$$

(13)

Similarly,

$$\begin{array}{cc}\hfill & M\left(t_2\right){\int }_{\mathsf{\Omega }}{\left(\right|\nabla v|}^{p\left(x\right)}+{\left|v\right|}^{p\left(x\right)})dx-{\int }_{\partial \mathsf{\Omega }}{\left|v\right|}^{p_{*}\left(x\right)}dS\hfill \\ \hfill & -{\int }_{\mathsf{\Omega }}{F}_v(x,u,v)vdx=0.\hfill \end{array}$$

(14)

□

Lemma 3.

Suppose that $\left\{\left(u_{n_k},v_{n_k}\right)\right\}$ is a ${\left(PS\right)}_c^{*}$ sequence; if

$$c\in \left(-\infty ,\left(\frac{1}{\theta p^{+}}-\frac{1}{p_{*}^{-}}\right)m_1\right),$$

where $m_1=min\left\{m_0^{\frac{p_{*}^{+}}{p_{*}^{+}-p^{-}}}{S}^{\frac{p^{-}{p}_{*}^{+}}{p_{*}^{+}-p^{-}}},m_0^{\frac{p_{*}^{+}}{p_{*}^{+}-p^{+}}}{S}^{\frac{p^{+}{p}_{*}^{+}}{p_{*}^{+}-p^{+}}},m_0^{\frac{p_{*}^{-}}{p_{*}^{-}-p^{-}}}{S}^{\frac{p^{-}{p}_{*}^{-}}{p_{*}^{-}-p^{-}}},m_0^{\frac{p_{*}^{-}}{p_{*}^{-}-p^{+}}}{S}^{\frac{p^{+}{p}_{*}^{-}}{p_{*}^{-}-p^{+}}}\right\}$, then $u_{n_k}\to u$, $v_{n_k}\to v$ in X.

Proof. 

From (7)

$$\begin{array}{cc}\hfill 〈\mathrm{d}{J}_{n_k}\left(u_{n_k},v_{n_k}\right),\left(u_{n_k},0\right)〉=& -M\left({\int }_{\mathsf{\Omega }}\frac{|\nabla u_{n_k}{|}^{p\left(x\right)}+{\left|u_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right){\int }_{\mathsf{\Omega }}\left(\right|\nabla u_{n_k}{|}^{p\left(x\right)}+|u_{n_k}{|}^{p\left(x\right)})dx\hfill \\ \hfill & -{\int }_{\partial \mathsf{\Omega }}{\left|u_{n_k}\right|}^{p_{*}\left(x\right)}dS-{\int }_{\mathsf{\Omega }}{F}_u(x,u_{n_k},v_{n_k})u_{n_k}dx\to 0.\hfill \end{array}$$

(15)

Thus, according to the Brézis–Lieb lemma [35], let ${\omega }_{n_k}=u_{n_k}-u$; (15) can be changed to

$$\begin{array}{cc}\hfill & -M\left({\int }_{\mathsf{\Omega }}\frac{|\nabla u_{n_k}{|}^{p\left(x\right)}+{\left|u_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right){\int }_{\mathsf{\Omega }}\left(\right|\nabla {\omega }_{n_k}{|}^{p\left(x\right)}+|{\omega }_{n_k}{|}^{p\left(x\right)})dx\hfill \\ \hfill & -M\left({\int }_{\mathsf{\Omega }}\frac{|\nabla u_{n_k}{|}^{p\left(x\right)}+{\left|u_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right){\int }_{\mathsf{\Omega }}{\left(\right|\nabla u|}^{p\left(x\right)}+{\left|u\right|}^{p\left(x\right)})dx\hfill \\ \hfill & -{\int }_{\partial \mathsf{\Omega }}|{\omega }_{n_k}{|}^{p_{*}\left(x\right)}dS-{\int }_{\partial \mathsf{\Omega }}{\left|u\right|}^{p_{*}\left(x\right)}dS-{\int }_{\mathsf{\Omega }}{F}_u(x,u_{n_k},v_{n_k})u_{n_k}dx\to 0.\hfill \end{array}$$

(16)

It follows from (12), (13), and (16) that

$$M\left({\int }_{\mathsf{\Omega }}\frac{|\nabla u_{n_k}{|}^{p\left(x\right)}+{\left|u_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right){\int }_{\mathsf{\Omega }}\left(\right|\nabla {\omega }_{n_k}{|}^{p\left(x\right)}+|{\omega }_{n_k}{|}^{p\left(x\right)})dx+{\int }_{\partial \mathsf{\Omega }}{\left|{\omega }_{n_k}\right|}^{p_{*}\left(x\right)}dS\to 0,$$

which yields

$$M\left({\int }_{\mathsf{\Omega }}\frac{|\nabla u_{n_k}{|}^{p\left(x\right)}+{\left|u_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right){\int }_{\mathsf{\Omega }}\left(\right|\nabla {\omega }_{n_k}{|}^{p\left(x\right)}+|{\omega }_{n_k}{|}^{p\left(x\right)})dx\to 0,$$

so $u_{n_k}\to u\mathrm{in}{E}_{G_1}$.

In addition,

$$\begin{array}{cc}\hfill 〈\mathrm{d}{J}_{n_k}\left(u_{n_k},v_{n_k}\right),\left(0,v_{n_k}\right)〉=& M\left({\int }_{\mathsf{\Omega }}\frac{|\nabla v_{n_k}{|}^{p\left(x\right)}+{\left|v_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right){\int }_{\mathsf{\Omega }}\left(\right|\nabla v_{n_k}{|}^{p\left(x\right)}+|v_{n_k}{|}^{p\left(x\right)})dx\hfill \\ \hfill & -{\int }_{\partial \mathsf{\Omega }}{\left|v_{n_k}\right|}^{p_{*}\left(x\right)}dS-{\int }_{\mathsf{\Omega }}{F}_v(x,u_{n_k},v_{n_k})v_{n_k}dx\to 0.\hfill \end{array}$$

(17)

Let ${\zeta }_{n_k}=v_{n_k}-v$, and (17) can be changed to

$$\begin{array}{cc}\hfill & M\left({\int }_{\mathsf{\Omega }}\frac{|\nabla v_{n_k}{|}^{p\left(x\right)}+{\left|v_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right){\int }_{\mathsf{\Omega }}\left(\right|\nabla {\zeta }_{n_k}{|}^{p\left(x\right)}+|{\zeta }_{n_k}{|}^{p\left(x\right)})dx\hfill \\ \hfill & +M\left({\int }_{\mathsf{\Omega }}\frac{|\nabla v_{n_k}{|}^{p\left(x\right)}+{\left|v_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right){\int }_{\mathsf{\Omega }}{\left(\right|\nabla v|}^{p\left(x\right)}+{\left|v\right|}^{p\left(x\right)})dx\hfill \\ \hfill & -{\int }_{\partial \mathsf{\Omega }}|{\zeta }_{n_k}{|}^{p_{*}\left(x\right)}dS-{\int }_{\partial \mathsf{\Omega }}{\left|v\right|}^{p_{*}\left(x\right)}dS-{\int }_{\mathsf{\Omega }}{F}_v(x,u_{n_k},v_{n_k})v_{n_k}dx\to 0.\hfill \end{array}$$

(18)

It follows from (12), (14), and (18) that

$$M\left({\int }_{\mathsf{\Omega }}\frac{|\nabla v_{n_k}{|}^{p\left(x\right)}+{\left|v_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right){\int }_{\mathsf{\Omega }}\left(\right|\nabla {\zeta }_{n_k}{|}^{p\left(x\right)}+|{\zeta }_{n_k}{|}^{p\left(x\right)})dx-{\int }_{\partial \mathsf{\Omega }}{\left|{\zeta }_{n_k}\right|}^{p_{*}\left(x\right)}dS\to 0.$$

(19)

From (19), we may assume

$$\begin{array}{cc}\hfill & \underset{n\to \infty }{lim}M\left({\int }_{\mathsf{\Omega }}\frac{{\left|\nabla v_{n_k}\right|}^{p\left(x\right)}+{\left|v_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right){\int }_{\mathsf{\Omega }}{\left(|\nabla {\zeta }_{n_k}\right|}^{p\left(x\right)}+{\left|{\zeta }_{n_k}\right|}^{p\left(x\right)})dx\hfill \\ \hfill & =M\left(t_2\right)\underset{n\to \infty }{lim}{\int }_{\mathsf{\Omega }}{\left(|\nabla {\zeta }_{n_k}\right|}^{p\left(x\right)}+{\left|{\zeta }_{n_k}\right|}^{p\left(x\right)})dx=m,\hfill \\ \hfill & \underset{n\to \infty }{lim}{\int }_{\partial \mathsf{\Omega }}{\left|{\zeta }_{n_k}\right|}^{p_{*}\left(x\right)}dS=m.\hfill \end{array}$$

If $m=0$, then $v_{n_k}\to v$ in $E_{G_1}$, and the proof is done. If not, we claim the following:

$$m⩾m_1=min\left\{m_0^{\frac{p_{*}^{+}}{p_{*}^{+}-p^{-}}}{S}^{\frac{p^{-}{p}_{*}^{+}}{p_{*}^{+}-p^{-}}},m_0^{\frac{p_{*}^{+}}{p_{*}^{+}-p^{+}}}{S}^{\frac{p^{+}{p}_{*}^{+}}{p_{*}^{+}-p^{+}}},m_0^{\frac{p_{*}^{-}}{p_{*}^{-}-p^{-}}}{S}^{\frac{p^{-}{p}_{*}^{-}}{p_{*}^{-}-p^{-}}},m_0^{\frac{p_{*}^{-}}{p_{*}^{-}-p^{+}}}{S}^{\frac{p^{+}{p}_{*}^{-}}{p_{*}^{-}-p^{+}}}\right\}.$$

In fact, from Remark 1, Proposition 2, and Remark 2, we have

(i)

if ${∥{\zeta }_{n_k}∥}_{1,p\left(x\right)}>1,{∥{\zeta }_{n_k}∥}_{p_{*}\left(x\right),\partial \mathsf{\Omega }}>1$.

$$\begin{array}{cc}\hfill m& =M\left(t_2\right)\underset{n\to \infty }{lim}{\int }_{\mathsf{\Omega }}{\left(|\nabla {\zeta }_{n_k}\right|}^{p\left(x\right)}+{\left|{\zeta }_{n_k}\right|}^{p\left(x\right)})dx\hfill \\ \hfill & ⩾m_0\underset{n\to \infty }{lim}{S}^{p^{-}}{∥{\zeta }_{n_k}∥}_{p_{*}\left(x\right),\partial \mathsf{\Omega }}^{p^{-}}\hfill \\ \hfill & ⩾m_0{S}^{p^{-}}{m}^{\frac{p^{-}}{p_{*}^{+}}},\hfill \end{array}$$

then $m⩾m_0^{\frac{p_{*}^{+}}{p_{*}^{+}-p^{-}}}{S}^{\frac{p^{-}{p}_{*}^{+}}{p_{*}^{+}-p^{-}}}$.

(ii)

if ${∥{\zeta }_{n_k}∥}_{1,p\left(x\right)}<1,{∥{\zeta }_{n_k}∥}_{p_{*}\left(x\right),\partial \mathsf{\Omega }}>1$.

$$\begin{array}{cc}\hfill m& =M\left(t_2\right)\underset{n\to \infty }{lim}{\int }_{\mathsf{\Omega }}{\left(|\nabla {\zeta }_{n_k}\right|}^{p\left(x\right)}+{\left|{\zeta }_{n_k}\right|}^{p\left(x\right)})dx\hfill \\ \hfill & ⩾m_0\underset{n\to \infty }{lim}{S}^{p^{+}}{∥{\zeta }_{n_k}∥}_{p_{*}\left(x\right),\partial \mathsf{\Omega }}^{p^{+}}\hfill \\ \hfill & ⩾m_0{S}^{p^{+}}{m}^{\frac{p^{+}}{p_{*}^{+}}},\hfill \end{array}$$

then $m⩾m_0^{\frac{p_{*}^{+}}{p_{*}^{+}-p^{+}}}{S}^{\frac{p^{+}{p}_{*}^{+}}{p_{*}^{+}-p^{+}}}$.

(iii)

if ${∥{\zeta }_{n_k}∥}_{1,p\left(x\right)}>1,{∥{\zeta }_{n_k}∥}_{p_{*}\left(x\right),\partial \mathsf{\Omega }}<1$.

$$\begin{array}{cc}\hfill m& =M\left(t_2\right)\underset{n\to \infty }{lim}{\int }_{\mathsf{\Omega }}{\left(|\nabla {\zeta }_{n_k}\right|}^{p\left(x\right)}+{\left|{\zeta }_{n_k}\right|}^{p\left(x\right)})dx\hfill \\ \hfill & ⩾m_0\underset{n\to \infty }{lim}{S}^{p^{-}}{∥{\zeta }_{n_k}∥}_{p_{*}\left(x\right),\partial \mathsf{\Omega }}^{p^{-}}\hfill \\ \hfill & ⩾m_0{S}^{p^{-}}{m}^{\frac{p^{-}}{p_{*}^{-}}},\hfill \end{array}$$

then $m⩾m_0^{\frac{p_{*}^{-}}{p_{*}^{-}-p^{-}}}{S}^{\frac{p^{-}{p}_{*}^{-}}{p_{*}^{-}-p^{-}}}$.

(iv)

if ${∥{\zeta }_{n_k}∥}_{1,p\left(x\right)}<1,{∥{\zeta }_{n_k}∥}_{p_{*}\left(x\right),\partial \mathsf{\Omega }}<1$.

$$\begin{array}{cc}\hfill m& =M\left(t_2\right)\underset{n\to \infty }{lim}{\int }_{\mathsf{\Omega }}{\left(|\nabla {\zeta }_{n_k}\right|}^{p\left(x\right)}+{\left|{\zeta }_{n_k}\right|}^{p\left(x\right)})dx\hfill \\ \hfill & ⩾m_0\underset{n\to \infty }{lim}{S}^{p^{+}}{∥{\zeta }_{n_k}∥}_{p_{*}\left(x\right),\partial \mathsf{\Omega }}^{p^{+}}\hfill \\ \hfill & ⩾m_0{S}^{p^{+}}{m}^{\frac{p^{+}}{p_{*}^{-}}},\hfill \end{array}$$

then $m⩾m_0^{\frac{p_{*}^{-}}{p_{*}^{-}-p^{+}}}{S}^{\frac{p^{+}{p}_{*}^{-}}{p_{*}^{-}-p^{+}}}$.

Note that

$$m_1=min\left\{m_0^{\frac{p_{*}^{+}}{p_{*}^{+}-p^{-}}}{S}^{\frac{p^{-}{p}_{*}^{+}}{p_{*}^{+}-p^{-}}},m_0^{\frac{p_{*}^{+}}{p_{*}^{+}-p^{+}}}{S}^{\frac{p^{+}{p}_{*}^{+}}{p_{*}^{+}-p^{+}}},m_0^{\frac{p_{*}^{-}}{p_{*}^{-}-p^{-}}}{S}^{\frac{p^{-}{p}_{*}^{-}}{p_{*}^{-}-p^{-}}},m_0^{\frac{p_{*}^{-}}{p_{*}^{-}-p^{+}}}{S}^{\frac{p^{+}{p}_{*}^{-}}{p_{*}^{-}-p^{+}}}\right\};$$

then, $m⩾m_1$.

According to $\left(F_4\right)$ and $\left(M_2\right)$, we obtain

$$\begin{array}{cc}\hfill c=& \underset{n\to \infty }{lim}[J\left(u_{n_k},v_{n_k}\right)-\langle \mathrm{d}{J}_{n_k}\left(u_{n_k},v_{n_k}\right),(\frac{1}{p^{-}}{u}_{n_k},\frac{1}{p_{*}^{-}}{v}_{n_k})\rangle ]\hfill \\ \hfill & ⩾\underset{n\to \infty }{lim}[-\widehat{M}\left({\int }_{\mathsf{\Omega }}\frac{|\nabla u_{n_k}{|}^{p\left(x\right)}+{\left|u_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right)\hfill \\ \hfill & +\frac{1}{p^{-}}M\left({\int }_{\mathsf{\Omega }}\frac{|\nabla u_{n_k}{|}^{p\left(x\right)}+{\left|u_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right){\int }_{\mathsf{\Omega }}\left(\right|\nabla u_{n_k}{|}^{p\left(x\right)}+|u_{n_k}{|}^{p\left(x\right)})dx\hfill \\ \hfill & +\frac{1}{\theta }M\left({\int }_{\mathsf{\Omega }}\frac{|\nabla v_{n_k}{|}^{p\left(x\right)}+{\left|v_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right){\int }_{\mathsf{\Omega }}\frac{|\nabla v_{n_k}{|}^{p\left(x\right)}+{\left|v_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\hfill \\ \hfill & -\frac{1}{p_{*}^{-}}M\left({\int }_{\mathsf{\Omega }}\frac{|\nabla v_{n_k}{|}^{p\left(x\right)}+{\left|v_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right){\int }_{\mathsf{\Omega }}\left(\right|\nabla v_{n_k}{|}^{p\left(x\right)}+|v_{n_k}{|}^{p\left(x\right)})dx\hfill \\ \hfill & -\frac{1}{p_{*}^{-}}{\int }_{\partial \mathsf{\Omega }}|u_{n_k}{|}^{p_{*}\left(x\right)}dS+\frac{1}{p^{-}}{\int }_{\partial \mathsf{\Omega }}|u_{n_k}{|}^{p_{*}\left(x\right)}dS-\frac{1}{p_{*}^{-}}{\int }_{\partial \mathsf{\Omega }}{\left|v_{n_k}\right|}^{p_{*}\left(x\right)}dS\hfill \\ \hfill & +\frac{1}{p_{*}^{-}}{\int }_{\partial \mathsf{\Omega }}{\left|v_{n_k}\right|}^{p_{*}\left(x\right)}dS-{\int }_{\mathsf{\Omega }}F(x,u_{n_k},v_{n_k})dx+\frac{1}{p^{-}}{\int }_{\mathsf{\Omega }}{F}_u(x,u_{n_k},v_{n_k})u_{n_k}dx\hfill \\ & +\frac{1}{p_{*}^{-}}{F}_v(x,u_{n_k},v_{n_k})v_{n_k}dx]\hfill \\ \hfill & ⩾\underset{n\to \infty }{lim}[(\frac{1}{\theta p^{+}}-\frac{1}{p_{*}^{-}})M\left({\int }_{\mathsf{\Omega }}\frac{|\nabla v_{n_k}{|}^{p\left(x\right)}+{\left|v_{n_k}\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right){\int }_{\mathsf{\Omega }}\left(\right|\nabla {\zeta }_{n_k}{|}^{p\left(x\right)}+|{\zeta }_{n_k}{|}^{p\left(x\right)})dx\hfill \\ \hfill & +{\int }_{\mathsf{\Omega }}{\left(\right|\nabla v|}^{p\left(x\right)}+{\left|v\right|}^{p\left(x\right)}\left)dx\right]\hfill \\ \hfill & ⩾(\frac{1}{\theta p^{+}}-\frac{1}{p_{*}^{-}})m\hfill \\ \hfill & ⩾(\frac{1}{\theta p^{+}}-\frac{1}{p_{*}^{-}})m_1.\hfill \end{array}$$

This is a contradiction. Consequently, we have finished proving Lemma 3. □

6. Proof of Theorem 1

Next, we begin the proof of Theorem 2.

Denote

$$\begin{array}{cccc}\hfill X& =U\oplus V,\hfill & \hfill U& =E_{G_1}\times \left\{0\right\},V=\left\{0\right\}\times E_{G_1},\hfill \\ \hfill Y_0& =\left\{0\right\}\times E_{G_1}^{m^{\perp }},\hfill & \hfill Y_1& =\left\{0\right\}\times E_{G_1}^{\left(k\right)},\hfill \end{array}$$

where m and k are to be determined. Obviously, $\left(A_1\right),\left(A_2\right),$ and $\left(A_4\right)$ in Theorem 2 are fulfilled. Let $V_j=E_{G_1}^{\left(j\right)}=span\left\{e_1,e_2,\cdots ,e_j\right\}$; then, $\left(A_3\right)$ holds. Since $1=\dim {\tilde{Y}}_0<k_0<\dim Y_1$, $\left(A_5\right)$ is true. Now, we verify (2), (3) of $\left(A_7\right)$.

(i) From condition $\left(M_2\right)$, it is guaranteed that there are constants $C_3,C_4>0$, ensuring that

$$\widehat{M}\left(t\right)⩽C_3{t}^{\theta }+C_4,\forall t⩾0.$$

(20)

For every $(u,v)\in U⨁Y_1$, from (20), $\left(F_5\right)$, we have

$$\begin{array}{cc}\hfill J(u,v)& =-\widehat{M}\left({\int }_{\mathsf{\Omega }}\frac{{|\nabla u|}^{p\left(x\right)}+{\left|u\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right)+\widehat{M}\left({\int }_{\mathsf{\Omega }}\frac{{|\nabla v|}^{p\left(x\right)}+{\left|v\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right)\hfill \\ \hfill & -{\int }_{\partial \mathsf{\Omega }}\frac{1}{p_{*}\left(x\right)}{\left|u\right|}^{p_{*}\left(x\right)}dS-{\int }_{\partial \mathsf{\Omega }}\frac{1}{p_{*}\left(x\right)}{\left|v\right|}^{p_{*}\left(x\right)}dS-{\int }_{\mathsf{\Omega }}F(x,u,v)dx\hfill \\ \hfill & ⩽C_3{\left({\int }_{\mathsf{\Omega }}\frac{{|\nabla v|}^{p\left(x\right)}+{\left|v\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right)}^{\theta }+C_4-{\int }_{\mathsf{\Omega }}{\left(L\right|v|}^{\theta p^{+}}-\xi )dx\hfill \\ \hfill & ⩽\frac{C_3}{{\left(p^{-}\right)}^{\theta }}{\left({\int }_{\mathsf{\Omega }}{|\nabla v|}^{p\left(x\right)}+{\left|v\right|}^{p\left(x\right)}dx\right)}^{\theta }-{\int }_{\mathsf{\Omega }}{\left(L\right|v|}^{\theta p^{+}}-\xi )dx+C_4.\hfill \end{array}$$

If ${∥v∥}_{1,p\left(x\right)}<1$, then

$$\begin{array}{cc}\hfill J(u,v)& ⩽\frac{C_3}{{\left(p^{-}\right)}^{\theta }}{∥v∥}_{1,p\left(x\right)}^{\theta p^{-}}-{L\left|v\right|}_{p\left(x\right)}^{\theta p^{+}}+\xi \left|\mathsf{\Omega }\right|+C_4\hfill \\ \hfill & ⩽\frac{C_3}{{\left(p^{-}\right)}^{\theta }}+\xi \left|\mathsf{\Omega }\right|+C_4.\hfill \end{array}$$

If ${∥v∥}_{1,p\left(x\right)}>1$, it follows from the equivalence of all norms on the finite-dimensional space $Y_1$; then, a constant $C_5>0$ can be found such that ${∥v∥}_{1,p\left(x\right)}⩽C_5{\left|v\right|}_{p\left(x\right)}$. From $\left(F_5\right)$, we derive

$$\begin{array}{cc}\hfill J(u,v)& ⩽\frac{C_3}{{\left(p^{-}\right)}^{\theta }}{∥v∥}_{1,p\left(x\right)}^{\theta p^{+}}-{L\left|v\right|}_{p\left(x\right)}^{\theta p^{+}}+\xi \left|\mathsf{\Omega }\right|+C_4\hfill \\ \hfill & ⩽\frac{C_3{C}_5}{{\left(p^{-}\right)}^{\theta }}{\left|v\right|}_{p\left(x\right)}^{\theta p^{+}}-{L\left|v\right|}_{p\left(x\right)}^{\theta p^{+}}+\xi \left|\mathsf{\Omega }\right|+C_4\hfill \\ \hfill & =(C_6-L){\left|v\right|}_{p\left(x\right)}^{\theta p^{+}}+\xi \left|\mathsf{\Omega }\right|+C_4,\hfill \end{array}$$

where $C_6=\frac{C_3{C}_5}{{\left(p^{-}\right)}^{\theta }}$. By taking $L⩾C_6$, we obtain $J(u,v)⩽\xi \left|\mathsf{\Omega }\right|+C_4$.

Let $\beta =\frac{C_3}{{\left(p^{-}\right)}^{\theta }}+\xi \left|\mathsf{\Omega }\right|+C_4$, so we obtain (3) in $\left(A_7\right)$.

(ii) If $(0,v)\in Y_0\bigcap B_{\rho }\left(0\right)$ (where $\rho$ is to be determined), then using $\left(M_1\right),\left(M_2\right),\left(F_2\right),\left(F_3\right)$ and Proposition 1, we have

$$\begin{array}{cc}\hfill J(0,v)& =\widehat{M}\left({\int }_{\mathsf{\Omega }}\frac{{|\nabla v|}^{p\left(x\right)}+{\left|v\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right)-{\int }_{\partial \mathsf{\Omega }}\frac{1}{p_{*}\left(x\right)}{\left|v\right|}^{p_{*}\left(x\right)}dS-{\int }_{\mathsf{\Omega }}F(x,0,v)dx\hfill \\ \hfill & ⩾\frac{1}{\theta }M\left({\int }_{\mathsf{\Omega }}\frac{{|\nabla v|}^{p\left(x\right)}+{\left|v\right|}^{p\left(x\right)}}{p\left(x\right)}dx\right){\int }_{\mathsf{\Omega }}\frac{{|\nabla v|}^{p\left(x\right)}+{\left|v\right|}^{p\left(x\right)}}{p\left(x\right)}dx-\frac{1}{p_{*}^{-}}{\int }_{\partial \mathsf{\Omega }}{\left|v\right|}^{p_{*}\left(x\right)}dS\hfill \\ \hfill & -{\int }_{\mathsf{\Omega }}\left(C_1{\left|v\right|}^{r\left(x\right)}+C_2\right)dx\hfill \\ \hfill & ⩾\frac{m_0}{\theta p^{+}}min\left\{{∥v∥}_{1,p\left(x\right)}^{p^{-}},{∥v∥}_{1,p\left(x\right)}^{p^{+}}\right\}-\frac{C_7}{p_{*}^{-}}max\left\{{∥v∥}_{1,p\left(x\right)}^{p_{*}^{-}},{∥v∥}_{1,p\left(x\right)}^{p_{*}^{+}}\right\}\hfill \\ \hfill & -C_{8}max\left\{{∥v∥}_{1,p\left(x\right)}^{r^{-}},{∥v∥}_{1,p\left(x\right)}^{r^{+}}\right\}-C_2\left|\mathsf{\Omega }\right|.\hfill \end{array}$$

Let $t={∥v∥}_{1,p\left(x\right)}$, and analyze the function $h:(0,+\infty )\to \mathbb{R}$, which is provided via

$$h\left(t\right)=\frac{m_0}{\theta p^{+}}min\left\{t^{p^{-}},t^{p^{+}}\right\}-\frac{C_7}{p_{*}^{-}}max\left\{t^{p_{*}^{-}},t^{p_{*}^{+}}\right\}-C_{8}max\left\{t^{r^{-}},t^{r^{+}}\right\}-C_2\left|\mathsf{\Omega }\right|,$$

and since $p^{-}<p^{+}<p_{*}^{-}<p_{*}^{+}$, we have $h\left(t\right)\to -\infty$, as $t\to +\infty$. Choose $\alpha <\beta$; then, if $t_0>0$ exists, $J(0,v)⩾h\left(t_0\right)=\alpha <\beta$ for ${∥v∥}_{1,p\left(x\right)}=t_0=\rho$ holds. That is (2) of $\left(A_7\right)$.

On the basis of Lemma 3, $J(u,v)$ meets the ${\left(PS\right)}_c^{*}$ condition for any $c\in [\alpha ,\beta ]$, which means that $\left(A_6\right)$ in Theorem 2 holds. Thus, in accordance with Theorem 2,

$$c_j=\underset{i^{\infty }\left(A\right)⩾j}{inf}\underset{(u,v)\in A}{sup}J(u,v),-k_0+1⩽j⩽-1$$

are critical values of $J,\alpha ⩽c_{-k_0+1}⩽\cdots ⩽c_{-1}⩽\beta$; then, J has at least $k_0-1$ pairs of critical points.

7. Conclusions

In this paper, we have mainly dealt with a class of noncooperative Kirchhoff-type variable exponent elliptic systems with nonlinear boundary conditions. Using the variational method, the solutions to the problem (1) correspond to the critical points of the functional J. Combining the ${\left(PS\right)}_c^{*}$ condition without the concentration compactness principle, we used limit index theory for the functional J and got at least $k_0-1$ pairs of critical points; that is, a multiplicity of solutions for problem (1) can be obtained.

Nevertheless, there are still many challenging problems to be addressed. For instance, we can try to add the nonlinear terms with parameters to the elliptic system. Furthermore, problem (1) can be extended to fractional elliptic systems. These problems will be further investigated in our future work.