Nontrivial Solutions for a Class of Quasilinear Schrödinger Systems

Abstract

In this thesis, we research quasilinear Schrödinger system as follows in which $3<N\in \mathbb{R}$, $2<p<N$, $2<q<N$, $V_1(x),V_2(x)$ are continuous functions, $k,\iota$ are parameters with $k,\iota >0$, and nonlinear terms $f,h\in C({\mathbb{R}}^N\times {\mathbb{R}}^2,\mathbb{R})$. We find a nontrivial solution $(u,v)$ for all $\iota >{\iota }_1(k)$ by means of the mountain-pass theorem and change of variable theorem. Our main novelty of the thesis is that we extend $\Delta$ to ${\Delta }_p$ and ${\Delta }_q$ to find the existence of a nontrivial solution.

1. Introduction

We concerned the following quasilinear Schrödinger system for this paper

$$\left\{\begin{array}{cc}-{\Delta }_{p}u+V_1{(x)|u|}^{p-2}u+\frac{k}{2}[{\Delta }_p{|u|}^2]u=\iota f(x,u,v),\hfill & x\in {\mathbb{R}}^N,\\ -{\Delta }_{q}v+V_2{(x)|v|}^{q-2}v+\frac{k}{2}[{\Delta }_q{|v|}^2]v=\iota h(x,u,v),\hfill & x\in {\mathbb{R}}^N,\end{array}\right.$$

(1)

in which $3<N\in \mathbb{R}$, $2<p<N$, $2<q<N$, $V_1(x),V_2(x)$ are continuous positive functions, k is a sufficiently large positive parameter, $\iota$ is a positive parameter, and $f,h\in C({\mathbb{R}}^N\times {\mathbb{R}}^2,\mathbb{R})$.

For a quasilinear Schrödinger system (1), by the symmetric mountain-pass theorem, ref. [1] found infinite solutions, for given nonlinear terms $f,h$. When $k=-2$, ref. [2] proved that it had nontrivial solutions.

The above quasilinear Schrödinger system for $p=q=2$ is inspired by the quasilinear Schrödinger equation as below

$$iϵ\partial z=-ϵ\Delta z+V(x)z-{k(|z|}^2{)z-lϵ\Delta h(|z|}^2)h^{\prime }{(|z|}^2)z,\mathrm{for}x\in {\mathbb{R}}^N,$$

(2)

in which $V(x)$ is fixed potential, l is a constant, and k and h are real functions. In [3,4,5], Equation (2) is used to study several physical phenomenon with different h.

For $h(t)=t,l(t)=\mu t^{\frac{p-1}{2}}$ and $k>0$, let $z(s,x)=exp(-iFs)u(x)$, an equivalent elliptic equation with variational structure is obtained

$$-ϵ\Delta u+E(x)u-{ϵk(\Delta (|u|}^2))u={\mu |u|}^{p-1}u,u>0x\in {\mathbb{R}}^N,N>2,$$

(3)

in which $E(x)=V(x)-F$ is also the potential function. There is a lot of research for problems similar to problem (3). Ref. [6] studied a problem that had multiple solutions by dual-approach techniques and variational methods when $k>0$ is small enough. Ref. [7] used a minimization argument established on the ground states of soliton solutions. The symmetric critical principle and the mountain-pass theorem were used for finding solutions in [8]. In [9], for a type of quasilinear Schrödinger equation like (3), the author used the method developed by [10,11] to study ground state solutions. In addition, refs. [12,13,14] have also conducted research on equations of this type.

There is a large amount of research on system (1) for $p=q=2$. In [15], Pohožaev manifold and Moser iteration were used for obtaining a ground state solution. By a suitable Nehari–Pohožaev-type constraint set and analyzing relational minimization issues, Wang and Huang found the ground state solutions for the same class system in [16]. In the Orlicz space, the concentration compactness principle and Nehari manifold method were used for finding a ground state solution in [17]. Ref. [18] used the monotonicity trick and the Moser iteration to obtain the result of positive solutions. In [19], Chen and Zhang found ground state solutions through minimization principle. By applying innovative application of variable transformation and the mountain-pass theorem, ref. [20] proved that quasilinear Schrödinger systems have a nontrivial solution.

Many papers mention replacing $\Delta$ with ${\Delta }_p$ to study the properties of the equation or system after changes, such as [1,13,21]. In fact, $\Delta$ is a special case of ${\Delta }_p$, that is ${\Delta }_p=\Delta$ if $p=2$. What we are interested in is the nontrivial solution to system (1) when $k>0$ is large enough.

Throughout this paper, we need some assumptions. Firstly, we make $V_1(x)$, $V_2(x)\in C({\mathbb{R}}^N$, $\mathbb{R})$ and encounter the ensuing properties.

$({\mathcal{V}}_1)$

$\tilde{V}:=min\{{inf}_{x\in {\mathbb{R}}^N}{V}_1(x),{inf}_{x\in {\mathbb{R}}^N}{V}_2(x)\}$ and $0<\tilde{V}$;

$({\mathcal{V}}_2)$

∃$V_0>0$, ∀$V\ge V_0$, $m\left(\{x\in {\mathbb{R}}^N:V_i(X)\le V\}\right)$ is bounded, where $i=1,2$, m is defined as the Lebesgue measure in ${\mathbb{R}}^N$.

Meanwhile, assume that the terms $f,h$ conform to the properties as follows:

$(f_1)$

$\frac{f(x,s,t)}{|(s,t)|}\to 0$, $\frac{h(x,s,t)}{|(s,t)|}\to 0$, as $(s,t)\to (0,0)$;

$(f_2)$

∃$C_0>0$, which makes $⟨\nabla \zeta (x,s,t),(s,t)⟩\le C_0\left({|(s,t)|}^{(p-1,q-1)}+{|(s,t)|}^{(l_p-1,l_q-1)}\right)$, $\forall s,t\in \mathbb{R}$, $p<l_p<p^{*}$, $q<l_q<q^{*}$, where ${|(s,t)|}^{(l_p,l_q)}=s^{l_p}+t^{l_q},p^{*}=\frac{Np}{N-p},q^{*}=\frac{Nq}{N-q}$;

$(f_3)$

∃$\theta >0$ satisfying $0<\theta \zeta (x,s,t)\le (s,t)\nabla \zeta (x,s,t)$, and $uf(x,u,v)\ge 0,vh(x,u,v)\ge 0$, in which $\nabla \zeta (x,s,t)=\left(f(x,s,t),h(x,s,t)\right)$.

The paper’s core result is given below.

Theorem 1.

For given $k>0$, there is ${\iota }_1(k)>0$ for all $\iota >{\iota }_1(k)$, when $({\mathcal{V}}_1),({\mathcal{V}}_2)$, and $(h_1)-(h_3)$ are true, in that system (1) has a nontrivial solution $(u,v)\in H$ and ${max}_{x\in {\mathbb{R}}^N}|(u(x),v(x))|\le \left({(\frac{1}{2^{p-3}k})}^{\frac{1}{p}},{(\frac{1}{2^{q-3}k})}^{\frac{1}{q}}\right)$.

Let me introduce the basic framework of this paper. Preparation work was completed in Section 2. In Section 3, we consider issues related to the solution of the modified system. We acquire the solution for the first system (1) by use of the Morse iteration technique in Section 4. Section 5 makes a conclusion.

In this article, we use C to denote dissimilar positive constants, and $B_{R}0$ stands for a ball with its radius $R>0$ and center at the origin. The operation $(x_1,x_2)(y_1,y_2)=x_1{y}_1+x_2{y}_2$, and the operation ${(x_1,x_2)}^{(y_1,y_2)}=x_1^{y_1}+x_2^{y_2}$.

2. Preliminary Work

The corresponding Euler–Lagrange functional for (1) is as follows:

$$\begin{array}{cc}\hfill J_k(u,v)& =\frac{1}{p}{\int }_{{\mathbb{R}}^N}(1-k{2}^{p-2}{|u|}^p{)|\nabla u|}^{p}dx+\frac{1}{p}{\int }_{{\mathbb{R}}^N}{V}_1(x){|u|}^{p}dx\hfill \\ & +\frac{1}{q}{\int }_{{\mathbb{R}}^N}(1-k{2}^{q-2}{|v|}^q{)|\nabla v|}^{q}dx+\frac{1}{q}{\int }_{{\mathbb{R}}^N}{V}_2(x){|v|}^{q}dx-\iota {\int }_{{\mathbb{R}}^N}\zeta (x,u,v)dx.\hfill \end{array}$$

The functional $J_k$ has quasilinear terms, and it is difficult to consider the critical points in the Sobolev spaces.

We stipulate that $D=D_1\times D_2$ and

$$∥(u,v)∥=∥u∥+∥v∥,$$

in which

$$D_1=\{u\in W^{1,p}({\mathbb{R}}^N):{\int }_{{\mathbb{R}}^N}{V}_1(x){|u|}^{p}dx<+\infty \}$$

given the norm

$$∥u∥={\left({\int }_{{\mathbb{R}}^N}{(|\nabla u|}^p+V_1{(x)|u|}^p)dx\right)}^{\frac{1}{p}}$$

and

$$D_2=\{v\in W^{1,q}({\mathbb{R}}^N):{\int }_{{\mathbb{R}}^N}{V}_2(x){|v|}^{q}dx<+\infty \}$$

given the norm

$$∥v∥={\left({\int }_{{\mathbb{R}}^N}{(|\nabla v|}^q+V_2{(x)|v|}^q)dx\right)}^{\frac{1}{q}},$$

$W^{1,p}({\mathbb{R}}^N)$ and $W^{1,q}({\mathbb{R}}^N)$ are the Sobolev space.

To make $(u,v)$ a solution for (1), if ∀$\phi ,\psi \in C_0^{\infty }({\mathbb{R}}^N)$, $(u,v)\in H$ satisfies

$$\begin{array}{cc}\hfill & {\int }_{{\mathbb{R}}^N}\left((1-2^{(p-2)}{k|u|}^p{)|\nabla u|}^{p-1}\nabla \phi -2^{(p-2)}{k|\nabla u|}^p{|u|}^{p-1}\phi \right)dx+{\int }_{{\mathbb{R}}^N}{V}_1(x){|u|}^{p-2}u\phi dx\hfill \\ & +{\int }_{{\mathbb{R}}^N}\left((1-2^{(q-2)}{k|v|}^q{)|\nabla v|}^{q-1}\nabla \psi -2^{(q-2)}{k|\nabla v|}^q{|v|}^{q-1}\psi \right)dx+{\int }_{{\mathbb{R}}^N}{V}_2(x){|v|}^{q-2}v\psi dx\hfill \\ & =\iota {\int }_{{\mathbb{R}}^N}\left(f(x,u,v)\phi +h(x,u,v)\psi \right)dx.\hfill \end{array}$$

(4)

Let $1-2^{p-2}k{|u|}^p>0$, we define the functions as follows:

$$y_u(u)=\left\{\begin{array}{c}{y}_u(-u),t<0,\hfill \\ {(1-2^{(p-2)}k{|u|}^p)}^{\frac{1}{p}},0\le u<{(\frac{1}{2^{(p-3)}k})}^{\frac{1}{p}},\hfill \\ \frac{1}{2^{(\frac{1}{p}+p-2)}k{u}^p}+2^{(-\frac{1}{p}-1)},u\ge {(\frac{1}{2^{(p-3)}k})}^{\frac{1}{p}},\hfill \end{array}\right.$$

and

$$y_v(v)=\left\{\begin{array}{c}{y}_v(-v),t<0,\hfill \\ {(1-2^{(q-2)}k{|v|}^q)}^{\frac{1}{q}},0\le v<{(\frac{1}{2^{(q-3)}k})}^{\frac{1}{q}},\hfill \\ \frac{1}{2^{(\frac{1}{q}+q-2)}k{v}^q}+2^{(-\frac{1}{q}-1)},v\ge {(\frac{1}{2^{(q-3)}k})}^{\frac{1}{q}}.\hfill \end{array}\right.$$

Then, $y_i(i)\in C^1(\mathbb{R},(2^{(-\frac{1}{q}-1)},1]),i=u,v$, $y_i$ is even and a convex function.

Affected by [22], we handle the following modified quasilinear Schrödinger system,

$$\left\{\begin{array}{cc}-div(y_u^p{(u)|\nabla u|}^{p-1})+y_u^{p-1}(u)y_u^{\prime }{(u)|\nabla u|}^p+V_1(x){|u|}^{p-1}=\iota f(x,u,v),\hfill & x\in {\mathbb{R}}^N,\\ -div(y_v^q{(v)|\nabla v|}^{q-1})+y_v^{q-1}(v)y_v^{\prime }{(v)|\nabla v|}^q+V_2(x){|v|}^{q-1}=\iota h(x,u,v),\hfill & x\in {\mathbb{R}}^N.\end{array}\right.$$

(5)

Clearly, ∀$\phi ,\psi \in C_0^{\infty }({\mathbb{R}}^N)$ and $(u,v)\in H$, $(u,v)$ is a weak solution for (5), if it holds

$$\begin{array}{cc}\hfill & {\int }_{{\mathbb{R}}^N}\left(y_u^p{(u)|\nabla u|}^{p-1}\nabla \phi +y_u^{p-1}(u)y_u^{\prime }{(u)|\nabla u|}^p\phi +V_1(x){|u|}^{p-1}\phi \right)dx\hfill \\ & +{\int }_{{\mathbb{R}}^N}\left(y_v^q{(v)|\nabla v|}^{q-1}\nabla \psi +y_v^{q-1}(v)y_v^{\prime }{(v)|\nabla v|}^q\psi +V_2(x){|v|}^{q-1}\psi \right)dx\hfill \\ & =\iota {\int }_{{\mathbb{R}}^N}\left(f(x,u,v)\phi +h(x,u,v)\psi \right)dx.\hfill \end{array}$$

(6)

Obviously, if ${∥(u,v)∥}_{\infty }\le \left({(\frac{1}{2^{(p-3)}k})}^{\frac{1}{p}},{(\frac{1}{2^{(q-3)}k})}^{\frac{1}{q}}\right)$ and $(u,v)$ is a solution for (5), so this particular solution $(u,v)$ also satisfies system (1). Utilize the change of variable as follows:

$$z=Y(u)={\int }_0^u{y}_u(t)dt,w=Y(w)={\int }_0^w{y}_v(t)dt;$$

then, the issue (5) can be simplified as:

$$\left\{\begin{array}{cc}-{\Delta }_{p}z+\frac{V_1(x){(Y^{-1}(z))}^{p-1}}{y_u(Y^{-1}(z))}=\iota \frac{f(x,Y^{-1}(z),Y^{-1}(w))}{Y^{-1}(z)},\hfill & x\in {\mathbb{R}}^N,\\ -{\Delta }_{q}w+\frac{V_2(x){(Y^{-1}(w))}^{q-1}}{y_v(Y^{-1}(w))}=\iota \frac{h(x,Y^{-1}(z),Y^{-1}(w))}{Y^{-1}(w)},\hfill & x\in {\mathbb{R}}^N,\end{array}\right.$$

(7)

among them $Y^{-1}$ and Y are inverse functions of each other, respectively. The corresponding function about (7) is

$$\begin{array}{cc}\hfill I_k(z,w)& =\frac{1}{p}{\int }_{{\mathbb{R}}^N}{|\nabla z|}^{p}dx+\frac{1}{p}{\int }_{{\mathbb{R}}^N}{V}_1(x){|Y^{-1}(z)|}^{p}dx\hfill \\ & +\frac{1}{q}{\int }_{{\mathbb{R}}^N}{|\nabla w|}^{q}dx+\frac{1}{q}{\int }_{{\mathbb{R}}^N}{V}_2(x){|Y^{-1}(w)|}^{q}dx-\iota {\int }_{{\mathbb{R}}^N}\zeta (x,Y^{-1}(z),Y^{-1}(w))dx.\hfill \end{array}$$

(8)

Obviously, $I_k$ has a good definition in D, and we can obtain the following lemma that are similar to [23].

Lemma 1.

The functions $y_u(u),y_v(v),Y(u),Y(v),Y^{-1}(z),Y^{-1}(w)$ satisfy these conditions as follows:

(i)

$Y(i)$ and its inverse function $Y^{-1}(z),Y^{-1}(w)$ are odd, where $i=u,v$;

(ii)

$-1\le \frac{u}{y_u(u)}{y}_u^{\prime }(u)\le 0$, $-1\le \frac{v}{y_v(v)}{y}_v^{\prime }(v)\le 0$ for all $u,v\in \mathbb{R}$;

(iii)

$|z|\le |Y^{-1}(z)|\le 2^{(-\frac{1}{p}-1)}|z|$, $|w|\le |Y^{-1}(w)|\le 2^{(-\frac{1}{q}-1)}|w|$ for every $z,w\in \mathbb{R}$;

(iv)

${lim}_{i\to 0}\frac{Y^{-1}(i)}{i}=1$, ${lim}_{z\to \infty }\frac{Y^{-1}(z)}{z}=2^{(-\frac{1}{p}-1)}$, ${lim}_{w\to \infty }\frac{Y^{-1}(w)}{w}=2^{(-\frac{1}{q}-1)}$, where $i=z,w$;

(v)

$y_u(Y^{-1}(z))\le \frac{z}{Y^{-1}(z)}$, $y_v(Y^{-1}(w))\le \frac{w}{Y^{-1}(w)}$ for all $z,w\in \mathbb{R}$.

Proof. 

Clearly, $(i)$ is established. The definition of $y_u$ and $y_v$ include

$$\underset{z\to 0}{lim}\frac{Y^{-1}(z)}{z}=\underset{z\to 0}{lim}\frac{1}{y_u(Y^{-1}(z))}=\frac{1}{y(0)}=1,$$

$$\underset{w\to 0}{lim}\frac{Y^{-1}(w)}{w}=\underset{w\to 0}{lim}\frac{1}{y_v(Y^{-1}(w))}=\frac{1}{y(0)}=1,$$

$$\underset{z\to \infty }{lim}\frac{Y^{-1}(z)}{z}=\underset{i\to \infty }{lim}\frac{1}{y_u(Y^{-1}(z))}=2^{(-\frac{1}{p}-1)},$$

$$\underset{w\to \infty }{lim}\frac{Y^{-1}(w)}{w}=\underset{i\to \infty }{lim}\frac{1}{y_v(Y^{-1}(w))}=2^{(-\frac{1}{q}-1)}.$$

Thus, $(iv)$ is proven. Since $y_u,y_v$ are decreasing in $|u|,|v|$, we obtain

$$Y(u)\ge u{y}_u(u)\ge 0,u\ge 0\mathrm{and}Y(u)\le u{y}_u(u)<0,u<0,$$

$$Y(v)\ge v{y}_v(v)\ge 0,v\ge 0\mathrm{and}Y(v)\le v{y}_v(v)<0,v<0,$$

and $(v)$ has also been proven. From

$$\frac{d}{dz}\left[\frac{Y^{-1}(z)}{z}\right]=\frac{z-Y^{-1}(z)y_u(Y^{-1}(z))}{y_u(Y^{-1}(z))z^2}\left\{\begin{array}{c}\ge 0,z\ge 0,\hfill \\ <0,z<0,\hfill \end{array}\right.$$

$$\frac{d}{dw}\left[\frac{Y^{-1}(w)}{w}\right]=\frac{w-Y^{-1}(w)y_v(Y^{-1}(w))}{y_v(Y^{-1}(w))w^2}\left\{\begin{array}{c}\ge 0,w\ge 0,\hfill \\ <0,w<0,\hfill \end{array}\right.$$

and $(iv)$, we obtain $(iii)$. Next, we prove $(ii)$. We consider $u,v\ge 0$. $\frac{u}{y_u(u)}{y}_u^{\prime }(u)\le 0$ and $\frac{v}{y_v(v)}{y}_v^{\prime }(v)\le 0$ are clear. For $0\le u<{(\frac{1}{2^{(p-3)}k})}^{\frac{1}{p}}$ and $0\le v<{(\frac{1}{2^{(q-3)}k})}^{\frac{1}{q}}$, we have

$$\frac{u}{y_u(u)}{y}_u^{\prime }(u)=\frac{-2^{(p-2)}k}{u^{-p}-2^{(p-2)}k}\ge -1,$$

and

$$\frac{v}{y_v(v)}{y}_v^{\prime }(v)=\frac{-2^{(q-2)}k}{v^{-q}-2^{(q-2)}k}\ge -1.$$

For $u\ge {(\frac{1}{2^{(p-3)}k})}^{\frac{1}{p}}$ and $v\ge {(\frac{1}{2^{(q-3)}k})}^{\frac{1}{q}}$, we have

$$\frac{u}{y_u(u)}{y}_u^{\prime }(u)=\frac{-2^{(-\frac{1}{p}-p+2)}{k}^{-1}p}{2^{(2-p-\frac{1}{p})}{k}^{-1}+2^{(-\frac{1}{p}-1)}{u}^p}\ge -1,$$

and

$$\frac{v}{y_v(v)}{y}_v^{\prime }(v)=\frac{-2^{(-\frac{1}{q}-q+2)}{k}^{-1}q}{2^{(2-q-\frac{1}{q})}{k}^{-1}+2^{(-\frac{1}{q}-1)}{v}^q}\ge -1.$$

When $u,v<0$, the proof method is similar to this. □

Lemma 2.

Let $({\mathcal{V}}_1),({\mathcal{V}}_2)$ and $(f_1)-(f_3)$ be true. To make $(u,v)=(Y^{-1}(z),Y^{-1}(w))$ a solution for (5), it is required that $(z,w)\in D$ is a critical point of $J_k$.

Proof. 

Because $(z,w)\in D$ is a critical point of $I_k$, ∀$(\phi ,\psi )\in D$, we have

$$\begin{array}{cc}\hfill & {\int }_{{\mathbb{R}}^N}{|\nabla z|}^{p-2}\nabla z\nabla \phi dx+{\int }_{{\mathbb{R}}^N}{V}_1(x)\frac{|Y^{-1}{(z)|}^{p-1}}{y_u(Y^{-1}(z))}\phi dx\hfill \\ & +{\int }_{{\mathbb{R}}^N}{|\nabla w|}^{q-2}\nabla w\nabla \psi dx+{\int }_{{\mathbb{R}}^N}{V}_2(x)\frac{|Y^{-1}{(w)|}^{q-1}}{y_v(Y^{-1}(w))}\psi dx\hfill \\ & =\iota {\int }_{{\mathbb{R}}^N}\left(\frac{f(x,Y^{-1}(z),Y^{-1}(w))}{y_u(Y^{-1}(z))}\phi +\frac{h(x,Y^{-1}(z),Y^{-1}(w))}{y_v(Y^{-1}(w))}\psi \right)dx.\hfill \end{array}$$

(9)

By Lemma 1, we know $(u,v):=(Y^{-1}(z),Y^{-1}(w))\in D$. Arbitrary to ${\phi }_0,{\psi }_0\in C_0^{\infty }({\mathbb{R}}^N)$, let $(\phi ,\psi ):=(y_u(u){\phi }_0,y_v(v){\psi }_0)\in D$ in (9) simplify to

$$\begin{array}{cc}\hfill & {\int }_{{\mathbb{R}}^N}{|\nabla z|}^{p-2}\nabla z(y_u^{\prime }(u){\phi }_0\nabla u+y_u(u)\nabla {\phi }_0)dx+{\int }_{{\mathbb{R}}^N}{V}_1(x)\frac{{|u|}^{p-1}}{y_u(u)}{g}_u(u){\phi }_{0}dx\hfill \\ & +{\int }_{{\mathbb{R}}^N}{|\nabla w|}^{q-2}\nabla w(y_v^{\prime }(v){\psi }_0\nabla v+y_v(v)\nabla {\psi }_0)dx+{\int }_{{\mathbb{R}}^N}{V}_2(x)\frac{{|v|}^{q-1}}{y_v(v)}{y}_v(v){\psi }_{0}dx\hfill \\ & =\iota {\int }_{{\mathbb{R}}^N}\left(\frac{f(x,u,v)}{y_u(u)}{y}_u(u){\phi }_0+\frac{h(x,u,v)}{y_v(v)}{y}_v(v){\psi }_0\right)dx.\hfill \end{array}$$

Applying the fact that $z=Y(u)$, $w=Y(v)$, $\nabla z=y_u(u)\nabla u$ and $\nabla w=y_v(v)\nabla v$, after calculation, obtaining (6), thus, $(u,v)$ is a weak solution of (5). □

Lemma 3.

Make $({\mathcal{V}}_1),({\mathcal{V}}_2)$ real. In $D_1,D_2$, $\{z_n\},\{w_n\}$ are bounded, then, there is $z\in D_1\cap L^{r_1}$ and $w\in D_2\cap L^{r_2}$, up to a subsequence, $z_n\to z$ in $L^{r_1}$, $r_1\in [p,p^{*})$, $w_n\to w$ in $L^{r_2}$, $r_2\in [q,q^{*})$.

Proof. 

The proof process is as shown in reference [1]. □

3. The Solution of the Modified System

Now, we study the modified system (5) and find its solution.

Lemma 4.

If $(f_1)-(f_3)$ are accurate, in that way

$(i)$

there are $\rho ,\pi >0$ makes $I_k(z,w)\ge \pi$ valid for every $(z,w)$ with $∥(z,w)∥=\rho$;

$(ii)$

the existence of $(z,w)\in D\setminus \{(0,0)\}$ makes $I_k(z,w)\le 0$ vaild.

Proof. 

$(i)$ By $(f_2)$, ∀$ϵ>0$, ∃$C_0>0$ settle for

$$\begin{array}{c}\hfill ⟨\nabla \zeta (x,s,t),(s,t)⟩\le {ϵ|(s,t)|}^{(p-1,q-1)}+C_0{|(s,t)|}^{(l_p-1,l_q-1)},\end{array}$$

(10)

where $p<l_p<p^{*}$ and $q<l_q<q^{*}$. Then,

$$\begin{array}{c}\hfill \zeta (x,s,t)\le ϵ(\frac{1}{p},\frac{1}{q})|(s,t){|}^{(p,q)}+C(\frac{1}{l_p},\frac{1}{l_q}){|(s,t)|}^{(l_p,l_q)}.\end{array}$$

(11)

Let $ϵ=min\{\frac{2^{\frac{1}{p}}{V}_1(x)}{\iota },\frac{2^{\frac{1}{q}}{V}_2(x)}{\iota }\}$, by Sobolev inequality, the Lemma 1$(iii)$ and (11), assuming that $\frac{1}{2p}{∥z∥}^p>\frac{1}{2q}{∥w∥}^q,\frac{2^{(-\frac{l_p}{p}-l_p)}{C}^{l_p}}{l_p}{∥z∥}^{l_p}>\frac{2^{(-\frac{l_q}{q}-l_q)}{C}^{l_q}}{l_q}{∥w∥}^{l_q}$, we have

$$\begin{array}{cc}\hfill I_k(z,w)& \ge \frac{1}{p}{\int }_{{\mathbb{R}}^N}{|\nabla z|}^{p}dx+\frac{1}{p}{\int }_{{\mathbb{R}}^N}{V}_1(x){|z|}^{p}dx\hfill \\ & +\frac{1}{q}{\int }_{{\mathbb{R}}^N}{|\nabla w|}^{q}dx+\frac{1}{q}{\int }_{{\mathbb{R}}^N}{V}_2(x){|w|}^{q}dx\hfill \\ & -\iota {\int }_{{\mathbb{R}}^N}\left(ϵ(\frac{1}{p},\frac{1}{q})|(Y^{-1}(z),Y^{-1}(w)){|}^{(p,q)}+C(\frac{1}{l_p},\frac{1}{l_q}){|(Y^{-1}(z),Y^{-1}(w))|}^{(l_p,l_q)}\right)dx\hfill \\ & \ge \frac{1}{p}{\int }_{{\mathbb{R}}^N}{|\nabla z|}^{p}dx+\frac{1}{p}{\int }_{{\mathbb{R}}^N}{V}_1(x){|z|}^{p}dx\hfill \\ & +\frac{1}{q}{\int }_{{\mathbb{R}}^N}{|\nabla w|}^{q}dx+\frac{1}{q}{\int }_{{\mathbb{R}}^N}{V}_2(x){|w|}^{q}dx\hfill \\ & -\iota {\int }_{{\mathbb{R}}^N}\left(\frac{ϵ}{p}{2}^{(-1-p)}{|z|}^p+\frac{ϵ}{q}{2}^{(-1-q)}{|w|}^q+\frac{C}{l_p}{2}^{(-\frac{l_p}{p}-l_p)}{|z|}^{l_p}+\frac{C}{l_q}{2}^{(-\frac{l_q}{q}-l_q)}{|w|}^{l_q}\right)dx.\hfill \end{array}$$

For $\frac{1}{2p}{∥z∥}^p+\frac{1}{2q}{∥w∥}^q$, when $∥z∥\ge 1,∥w∥\ge 1,p>q$,

$$\begin{array}{c}\hfill \frac{1}{2p}{∥z∥}^p+\frac{1}{2q}{∥w∥}^q\ge \frac{1}{2p}\frac{1}{2^q}{∥(z,w)∥}^q,\end{array}$$

when $∥z∥\ge 1,∥w∥\ge 1,p<q$,

$$\begin{array}{c}\hfill \frac{1}{2p}{∥z∥}^p+\frac{1}{2q}{∥w∥}^q\ge \frac{1}{2q}\frac{1}{2^p}{∥(z,w)∥}^p,\end{array}$$

when $∥z∥\ge 1,∥w∥<1,p>q$,

$$\begin{array}{c}\hfill \frac{1}{2p}{∥z∥}^p+\frac{1}{2q}{∥w∥}^q\ge \frac{1}{2p}\frac{1}{2^q}{∥(z,w)∥}^q,\end{array}$$

when $∥z∥\ge 1,∥w∥<1,2<p<q$,

$$\begin{array}{c}\hfill \frac{1}{2p}{∥z∥}^p+\frac{1}{2q}{∥w∥}^q\ge \frac{1}{2p}{∥z∥}^p\ge \frac{1}{2p},\end{array}$$

when $∥z∥<1,∥w∥\ge 1,p>q$,

$$\begin{array}{c}\hfill \frac{1}{2p}{∥z∥}^p+\frac{1}{2q}{∥w∥}^q\ge \frac{1}{2q}{∥w∥}^q\ge \frac{1}{2q},\end{array}$$

when $∥z∥<1,∥w∥\ge 1,p<q$,

$$\begin{array}{c}\hfill \frac{1}{2p}{∥z∥}^p+\frac{1}{2q}{∥w∥}^q\ge \frac{1}{2q}\frac{1}{2^q}{∥(z,w)∥}^q,\end{array}$$

when $∥z∥<1,∥w∥<1,p>q$,

$$\begin{array}{c}\hfill \frac{1}{2p}{∥z∥}^p+\frac{1}{2q}{∥w∥}^q\ge \frac{1}{2p}\frac{1}{2^q}{∥(z,w)∥}^q,\end{array}$$

when $∥z∥<1,∥w∥<1,p<q$,

$$\begin{array}{c}\hfill \frac{1}{2p}{∥z∥}^p+\frac{1}{2q}{∥w∥}^q\ge \frac{1}{2q}\frac{1}{2^p}{∥(z,w)∥}^p.\end{array}$$

For ${∥z∥}^{l_p}+{∥w∥}^{l_q}$, when $∥z∥\ge 1,∥w∥\ge 1,l_p>l_q$,

$$\begin{array}{c}\hfill {∥z∥}^{l_p}+{∥w∥}^{l_q}\le {∥(z,w)∥}^{l_p},\end{array}$$

when $∥z∥\ge 1,∥w∥\ge 1,l_p<l_q$,

$$\begin{array}{c}\hfill {∥z∥}^{l_p}+{∥w∥}^{l_q}\le {∥(z,w)∥}^{l_q},\end{array}$$

when $∥z∥\ge 1,∥w∥<1,l_p>l_q>q$,

$$\begin{array}{c}\hfill {∥z∥}^{l_p}+{∥w∥}^{l_q}\le {∥z∥}^{l_p}+∥w∥\le {∥(z,w)∥}^{l_p}+∥(z,w)∥,\end{array}$$

when $∥z∥\ge 1,∥w∥<1,l_p<l_q$,

$$\begin{array}{c}\hfill {∥z∥}^{l_p}+{∥w∥}^{l_q}\le {∥(z,w)∥}^{l_q},\end{array}$$

when $∥z∥<1,∥w∥\ge 1,p<l_p<l_q$,

$$\begin{array}{c}\hfill {∥z∥}^{l_p}+{∥w∥}^{l_q}\le ∥z∥+{∥w∥}^{l_q}\le {∥(z,w)∥}^{l_q}+∥(z,w)∥,\end{array}$$

when $∥z∥<1,∥w∥\ge 1,l_p>l_q$,

$$\begin{array}{c}\hfill {∥z∥}^{l_p}+{∥w∥}^{l_q}\le {∥(z,w)∥}^{l_p},\end{array}$$

when $∥z∥<1,∥w∥<1,l_p<l_q$,

$$\begin{array}{c}\hfill {∥z∥}^{l_p}+{∥w∥}^{l_q}\le {∥(z,w)∥}^{l_p},\end{array}$$

when $∥z∥<1,∥w∥<1,l_p>l_q$,

$$\begin{array}{c}\hfill {∥z∥}^{l_p}+{∥w∥}^{l_q}\le {∥(z,w)∥}^{l_q}.\end{array}$$

Hence,

$$\begin{array}{cc}\hfill I_k(z,w)& \ge min\{\frac{1}{2p}\frac{1}{2^q}{∥(z,w)∥}^q,\frac{1}{2p}\frac{1}{2^p}{∥(z,w)∥}^p,\frac{1}{2q}\frac{1}{2^p}{∥(z,w)∥}^p,\frac{1}{2q}\frac{1}{2^q}{∥(z,w)∥}^q,\frac{1}{2p},\frac{1}{2q}\}\hfill \\ & -max\{{∥(z,w)∥}^{l_p}{l}_s+∥(z,w)∥l_s,{∥(z,w)∥}^{l_q}{l}_s+∥(z,w)∥l_s\},\hfill \end{array}$$

where $l_s:=max\{\iota \frac{C}{l_p}{2}^{(-\frac{l_p}{p}-l_p)}{C}^{l_p},\iota \frac{C}{l_q}{2}^{(-\frac{l_q}{q}-l_q)}{C}^{l_q}\}$. Take $∥(z,w)∥=\rho$ small enough to satisfy

$$I_{k,\iota }(z,w)\ge \pi :=min\{\frac{1}{2p}\frac{1}{2^q}{\rho }^q,\frac{1}{2p}\frac{1}{2^p}{\rho }^p,\frac{1}{2q}\frac{1}{2^p}{\rho }^p,\frac{1}{2q}\frac{1}{2^q}{\rho }^q,\frac{1}{2p},\frac{1}{2q}\}-max\{{\rho }^{l_p}{l}_s+\rho l_s,{\rho }^{l_q}{l}_s+\rho l_s\}.$$

$(ii)$ Choose $({\tau }_1,{\tau }_2)\in D$ with ${\tau }_1,{\tau }_2>0$, from Lemma 1$(iii)$, we obtain

$$|{\tau }_1{|}^p\le \frac{|Y^{-1}(t{\tau }_1){|}^p}{t^p}\le 2^{(-p-1)}|{\tau }_1{|}^p,|{\tau }_2{|}^q\le \frac{|Y^{-1}(t{\tau }_2){|}^q}{t^q}\le 2^{(-q-1)}{|{\tau }_2|}^q.$$

By $(f_3)$, we know ${lim}_{|(s_1,s_2)|\to +\infty }\frac{\zeta (x,s_1,s_2)}{|(s_1,s_2){|}^p}=+\infty$. Therefore, for $p>q$, we have

$$\begin{array}{cc}\hfill \frac{I_k(t{\tau }_1,t{\tau }_2)}{t^p}& \le \frac{1}{p}{\int }_{{\mathbb{R}}^N}|\nabla {\tau }_1{|}^{p}dx+\frac{2^{(-p-1)}}{p}{\int }_{{\mathbb{R}}^N}{V}_1(x){|{\tau }_1|}^{p}dx\hfill \\ & +\frac{1}{q}{\int }_{{\mathbb{R}}^N}|\nabla {\tau }_2{|}^{q}dx+\frac{2^{(-q-1)}}{q}{\int }_{{\mathbb{R}}^N}{V}_2(x){|{\tau }_2|}^{q}dx\hfill \\ & -\iota {\int }_{{\mathbb{R}}^N}\frac{\zeta (x,Y^{-1}(t{\tau }_1),Y^{-1}(t{\tau }_2))}{{(Y^{-1}(t{\tau }_1),Y^{-1}(t{\tau }_2))}^p}\frac{{(Y^{-1}(t{\tau }_1),Y^{-1}(t{\tau }_2))}^p}{t^p}dx\to -\infty ,\mathrm{as}t\to +\infty .\hfill \end{array}$$

(12)

In the same way, for $p<q$, as $t\to +\infty$, we have

$$\begin{array}{c}\hfill \frac{I_k(t{\tau }_1,t{\tau }_2)}{t^q}\to -\infty .\end{array}$$

Hence, ∃$t_0>0$ large enough, such that $(z,w)=(t_0{\tau }_1,t_0{\tau }_2)$ with $I_k(z,w)\le 0$. □

To sum up, the ${(PS)}_c$ sequence exists and is denoted as $(z_n,w_n)\subset D$, therefore, as $n\to \infty$, we obtain

$$\begin{array}{c}\hfill I_k(z_n,w_n)\to c,I_k^{\prime }(z_n,w_n)\to 0\end{array}$$

(13)

and

$$c=\underset{\gamma \in \Gamma }{inf}\underset{t\in [0,1]}{sup}{I}_k(z_t,w_t),$$

$$\Gamma =\{(z_t,w_t)\in C([0,1]\times [0,1],D):(z_0,w_0)=(0,0),(z_1,w_1)\ne (0,0),I_k(z_1,w_1)<0\}.$$

Lemma 5.

If $(f_3)$ are accurate, in that way for all ${(PS)}_c$ sequence $(z_n,w_n)$ is bounded in D.

Proof. 

For $p<q$, combining (13) and Lemma 1$(ii),(iii)$ with $(f_3)$, there is

$$\begin{array}{cc}\hfill c& +1+o_n(1)∥(z_n,w_n)∥\hfill \\ & \ge I_k(z_n,w_n)-\frac{1}{\theta }⟨I_k^{\prime }(z_n,w_n),(Y^{-1}(z_n)y_u(Y^{-1}(z_n)),Y^{-1}(w_n)y_v(Y^{-1}(w_n)))⟩\hfill \\ & =(\frac{1}{p}-\frac{1}{\theta }){\int }_{{\mathbb{R}}^N}|\nabla z_n{|}^{p}dx-\frac{1}{\theta }{\int }_{{\mathbb{R}}^N}{|\nabla z_n|}^p\frac{Y^{-1}(z_n)}{y_u(Y^{-1}(z_n))}{y}_u^{\prime }(Y^{-1}(z_n))dx\hfill \\ & +(\frac{1}{p}-\frac{1}{\theta }){\int }_{{\mathbb{R}}^N}{V}_1(x){|Y^{-1}(z_n)|}^{p}dx\hfill \\ & +(\frac{1}{q}-\frac{1}{\theta }){\int }_{{\mathbb{R}}^N}|\nabla w_n{|}^{q}dx-\frac{1}{\theta }{\int }_{{\mathbb{R}}^N}{|\nabla w_n|}^q\frac{Y^{-1}(w_n)}{y_v(Y^{-1}(w_n))}{y}_v^{\prime }(Y^{-1}(w_n))dx\hfill \\ & +(\frac{1}{q}-\frac{1}{\theta }){\int }_{{\mathbb{R}}^N}{V}_2(x){|Y^{-1}(w_n)|}^{q}dx\hfill \\ & +{\int }_{{\mathbb{R}}^N}(\frac{1}{\theta }⟨\nabla \zeta (x,Y^{-1}(z_n),Y^{-1}(w_n)),(Y^{-1}(z_n)y_u^{\prime }(Y^{-1}(z_n)),Y^{-1}(w_n)y_v^{\prime }(Y^{-1}(w_n)))⟩\hfill \\ & -\zeta (x,Y^{-1}(z_n),Y^{-1}(w_n)))dx\hfill \\ & \ge (\frac{1}{p}-\frac{1}{\theta }){\int }_{{\mathbb{R}}^N}\left(|\nabla z_n{|}^q+V_1(x){|z_n|}^p\right)dx+(\frac{1}{q}-\frac{1}{\theta }){\int }_{{\mathbb{R}}^N}\left(|\nabla w_n{|}^q+V_2(x){|w_n|}^q\right)dx\hfill \\ & =(\frac{1}{p}-\frac{1}{\theta })({∥z_n∥}^p+{∥w_n∥}^q),\hfill \end{array}$$

(14)

when $∥z_n∥\ge 1,∥w_n∥\ge 1$,

$$\begin{array}{c}\hfill (\frac{1}{p}-\frac{1}{\theta })({∥z_n∥}^p+{∥w_n∥}^q)\ge (\frac{1}{p}-\frac{1}{\theta })({∥z_n∥}^p+{∥w_n∥}^p)\ge \frac{1}{2^p}{∥(z_n,w_n)∥}^p,\end{array}$$

when $∥z_n∥\ge 1,∥w_n∥<1$,

$$\begin{array}{c}\hfill (\frac{1}{p}-\frac{1}{\theta })({∥z_n∥}^p+{∥w_n∥}^q)\ge (\frac{1}{p}-\frac{1}{\theta })∥z_n∥,\end{array}$$

when $∥z_n∥<1,∥w_n∥\ge 1$,

$$\begin{array}{c}\hfill (\frac{1}{p}-\frac{1}{\theta })({∥z_n∥}^p+{∥w_n∥}^q)\ge (\frac{1}{p}-\frac{1}{\theta })∥w_n∥,\end{array}$$

when $∥z_n∥<1,∥w_n∥<1$,

$$\begin{array}{c}\hfill (\frac{1}{p}-\frac{1}{\theta })({∥z_n∥}^p+{∥w_n∥}^q)\ge (\frac{1}{p}-\frac{1}{\theta })({∥z_n∥}^q+{∥w_n∥}^q)\ge \frac{1}{2^q}{∥(z_n,w_n)∥}^q.\end{array}$$

Overall, for $p<q$, $(z_n,w_n)\subset D$ is bounded; similarly, for $p>q$, $(z_n,w_n)\subset D$ is also bounded. □

Since, ${(PS)}_c$ sequence $(z_n,w_n)\subset D$ is bounded, there is $(z,w)$, and $(z_n,w_n)$ have a subsequence recorded as $(z_n,w_n)$ meet

$$\begin{array}{cc}\hfill & (z_n,w_n)\rightharpoonup (z,w)\mathrm{in}D,\hfill \\ & (z_n(x),w_n(x))\rightharpoonup (z(x),w(x))\mathrm{a}.\mathrm{e}.\mathrm{in}{\mathbb{R}}^N\times {\mathbb{R}}^N,\hfill \\ & (z_n,w_n)\to (z,w)\mathrm{in}{L}^r,L^r=L^{r_1}\times L^{r_2}.\hfill \end{array}$$

(15)

$I_k$ of (8) also is defined as

$$\begin{array}{cc}\hfill I_k(z_n,w_n)& =\frac{1}{p}{\int }_{{\mathbb{R}}^N}|\nabla z_n{|}^{p}dx+\frac{1}{p}{\int }_{{\mathbb{R}}^N}{V}_1(x){|z_n|}^{p}dx\hfill \\ & +\frac{1}{q}{\int }_{{\mathbb{R}}^N}|\nabla w_n{|}^{q}dx+\frac{1}{q}{\int }_{{\mathbb{R}}^N}{V}_2(x){|w_n|}^{q}dx-\iota {\int }_{{\mathbb{R}}^N}\eta (x,z_n,w_n)dx,\hfill \end{array}$$

(16)

and

$$\begin{array}{cc}\hfill \eta (x,z_n,w_n)& =\frac{1}{p}{V}_1(x)\left(|z_n{|}^p-{|Y^{-1}(z_n)|}^p\right)+\frac{1}{q}{V}_2(x)\left(|w_n{|}^q-{|Y^{-1}(w_n)|}^q\right)\hfill \\ & +\iota \zeta \left(x,Y^{-1}(z_n),Y^{-1}(w_n)\right),\hfill \end{array}$$

in the same

$$\begin{array}{cc}\hfill ⟨\nabla \eta (x,z_n,w_n),(z_n,w_n)⟩& =V_1(x)(|z_n{|}^p-\frac{|Y^{-1}(z_n){|}^{p-1}}{y_u(Y^{-1}(z_n))})+\iota \frac{f(x,Y^{-1}(z_n),Y^{-1}(w_n))}{y_u(Y^{-1}(z_n))}{z}_n\hfill \\ & +V_2(x)(|w_n{|}^q-\frac{|Y^{-1}(w_n){|}^{q-1}}{y_v(Y^{-1}(w_n))})+\iota \frac{h(x,Y^{-1}(w_n),Y^{-1}(z_n))}{y_v(y^{-1}(w_n))}{w}_n.\hfill \end{array}$$

Lemma 6.

If $(f_1),(f_2),({\mathcal{V}}_1)$, and $({\mathcal{V}}_2)$ are accurate, $(z_n,w_n)$ is a ${(PS)}_c$ sequence, and $(z_n,w_n)\rightharpoonup (z,w)$ in D, as $n\to \infty$, in that way

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}{\int }_{{\mathbb{R}}^N}⟨\nabla \eta (x,z_n,w_n),(z_n,w_n)⟩dx={\int }_{{\mathbb{R}}^N}⟨\nabla \eta (x,z,w),(z,w)⟩dx,\end{array}$$

(17)

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}{\int }_{{\mathbb{R}}^N}⟨\nabla \eta (x,z_n,w_n),(z,w)⟩dx={\int }_{{\mathbb{R}}^N}⟨\nabla \eta (x,z,w),(z,w)⟩dx.\end{array}$$

(18)

Proof. 

From Lemma 3, since $z_n\to z$ in $L^{r_1}$, $w_n\to w$ in $L^{r_2}$, $r_1\in [p,p^{*})$, $r_2\in [q,q^{*})$, for $\forall \epsilon >0$, there is $R_1>0$ satisfied

$$\begin{array}{c}\hfill {\int }_{B_{R_1}^c}|z_n{|}^{p}dx\le \epsilon ,{\int }_{B_{R_1}^c}{|z|}^{p}dx\le \epsilon ,\\ \hfill {\int }_{B_{R_1}^c}|w_n{|}^{q}dx\le \epsilon ,{\int }_{B_{R_1}^c}{|w|}^{q}dx\le \epsilon .\end{array}$$

(19)

Then,

$$\begin{array}{c}\hfill {\int }_{B_{R_1}^c}{V}_1(x)|z_n{|}^{p}dx\le C\epsilon ,{\int }_{B_{R_1}^c}{V}_1(x){|z|}^{p}dx\le C\epsilon ,\\ \hfill {\int }_{B_{R_1}^c}{V}_2(x)|w_n{|}^{q}dx\le C\epsilon ,{\int }_{B_{R_1}^c}{V}_2(x){|w|}^{q}dx\le C\epsilon .\end{array}$$

(20)

It is from (15) that

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}{\int }_{B_{R_1}}{V}_1(x)|z_n{|}^{p}dx={\int }_{B_{R_1}}{V}_1(x){|z|}^{p}dx,\\ \hfill \underset{n\to \infty }{lim}{\int }_{B_{R_1}}{V}_2(x)|w_n{|}^{q}dx={\int }_{B_{R_1}}{V}_2(x){|w|}^{q}dx.\end{array}$$

(21)

By (20) and (21), we obtain

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}{\int }_{{\mathbb{R}}^N}{V}_1(x)|z_n{|}^{p}dx={\int }_{{\mathbb{R}}^N}{V}_1(x){|z|}^{p}dx,\\ \hfill \underset{n\to \infty }{lim}{\int }_{{\mathbb{R}}^N}{V}_2(x)|w_n{|}^{q}dx={\int }_{{\mathbb{R}}^N}{V}_2(x){|w|}^{q}dx.\end{array}$$

(22)

Deriving from Lemma 1 $(ii)$ and $(iii)$ that

$$\frac{|Y^{-1}(z_n){|}^{p-1}}{y_u(Y^{-1}(z_n))}{z}_n\le 2^{-\frac{(p-1)(p+1)}{p}}|z_n{|}^{p-1},\frac{|Y^{-1}(w_n){|}^{q-1}}{y_v(Y^{-1}(w_n))}{w}_n\le 2^{-\frac{(q-1)(q+1)}{q}}{|w_n|}^{q-1},$$

it follows from (22), (20) and (21) that

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}{\int }_{{\mathbb{R}}^N}{V}_1(x)\frac{|Y^{-1}(z_n){|}^{p-1}}{y_u(Y^{-1}(z_n))}{z}_{n}dx={\int }_{{\mathbb{R}}^N}{V}_1(x)\frac{|Y^{-1}{(z)|}^{p-1}}{y_u(Y^{-1}(z))}zdx,\\ \hfill \underset{n\to \infty }{lim}{\int }_{{\mathbb{R}}^N}{V}_2(x)\frac{|Y^{-1}(w_n){|}^{q-1}}{y_v(Y^{-1}(w_n))}{w}_{n}dx={\int }_{{\mathbb{R}}^N}{V}_2(x)\frac{|Y^{-1}{(w)|}^{q-1}}{y_v(Y^{-1}(w))}wdx.\end{array}$$

(23)

By (10), Lemma 1 $(iii)$ and Hölder inequality,

$$\begin{array}{cc}\hfill & |\frac{f(x,Y^{-1}(z_n),Y^{-1}(w_n))}{y_u(Y^{-1}(z_n))}{z}_n+\frac{h(x,Y^{-1}(w_n),Y^{-1}(z_n))}{y_v(Y^{-1}(w_n))}{w}_n|\hfill \\ & =|⟨\nabla \zeta (x,z_n,w_n),(z_n,w_n)⟩|\hfill \\ & \le 2^{(-\frac{(p+1)(p-1)}{p})}ϵ|z_n{|}^p+2^{(-\frac{(q+1)(q-1)}{q})}ϵ|w_n{|}^q+2^{(-\frac{(p+1)(l_p-1)}{p})}C|z_n{|}^{l_p}+2^{(-\frac{(q+1)(l_q-1)}{q})}C{|w_n|}^{l_q}.\hfill \end{array}$$

By (19), we obtain

$$\begin{array}{c}\hfill {\int }_{B_{R_1}^c}|z_n{|}^{l_p}dx\le C\epsilon ,{\int }_{B_{R_1}^c}{|z|}^{l_p}dx\le C\epsilon ,\\ \hfill {\int }_{B_{R_1}^c}|w_n{|}^{l_q}dx\le C\epsilon ,{\int }_{B_{R_1}^c}{|w|}^{l_q}dx\le C\epsilon .\end{array}$$

(24)

Thus,

$$\begin{array}{cc}\hfill & {\int }_{B_{R_1}^c}|⟨\nabla \zeta (x,Y^{-1}(z_n),Y^{-1}(w_n)),(z_n,w_n)⟩|dx\hfill \\ & \le 2^{(-\frac{(p+1)(p-1)}{p})}ϵ{\int }_{B_{R_1}^c}|z_n{|}^{p}dx+2^{(-\frac{(q+1)(q-1)}{q})}ϵ{\int }_{B_{R_1}^c}{|w_n|}^{q}dx\hfill \\ & +2^{(-\frac{(p+1)(l_p-1)}{p})}C{\int }_{B_{R_1}^c}|z_n{|}^{l_p}dx+2^{(-\frac{(q+1)(l_q-1)}{q})}C{\int }_{B_{R_1}^c}{|w_n|}^{l_q}dx\hfill \\ & =(2^{(-\frac{(p+1)(p-1)}{p})}ϵ+2^{(-\frac{(q+1)(q-1)}{q})}ϵ+2^{(-\frac{(p+1)(l_p-1)}{p})}C+2^{(-\frac{(q+1)(l_q-1)}{q})}C)\epsilon .\hfill \end{array}$$

(25)

By (15),

$$\begin{array}{cc}\hfill & \underset{n\to \infty }{lim}{\int }_{B_{R_1}}|⟨\nabla \zeta (x,Y^{-1}(z_n),Y^{-1}(w_n)),(z_n,w_n)⟩|dx\hfill \\ & ={\int }_{B_{R_1}}|⟨\nabla \zeta (x,Y^{-1}(z),Y^{-1}(w)),(z,w)⟩|dx.\hfill \end{array}$$

(26)

By (25) and (26),

$$\begin{array}{cc}\hfill & \underset{n\to \infty }{lim}{\int }_{{\mathbb{R}}^N}⟨\nabla \zeta (x,Y^{-1}(z_n),Y^{-1}(w_n)),(z_n,w_n)⟩dx\hfill \\ & =\underset{n\to \infty }{lim}{\int }_{{\mathbb{R}}^N}\left(\frac{f(x,Y^{-1}(z_n),Y^{-1}(w_n))}{y_u(Y^{-1}(z_n))}{z}_n+\frac{h(x,Y^{-1}(w_n),Y^{-1}(z_n))}{y_v(Y^{-1}(w_n))}{w}_n\right)dx\hfill \\ & ={\int }_{{\mathbb{R}}^N}\left(\frac{f(x,Y^{-1}(z),Y^{-1}(w))}{y_u(Y^{-1}(z))}z+\frac{h(x,Y^{-1}(w),Y^{-1}(z))}{y_v(Y^{-1}(w))}w\right)dx.\hfill \end{array}$$

(27)

Combining (22), (23) and (27), it is easy to obtain (17). Similarly, (18) can also be obtained. □

Lemma 7.

If $(f_1),(f_2),({\mathcal{V}}_1)$ and $({\mathcal{V}}_2)$ are satisfied, then, in D, any ${(PS)}_c$ sequence $(z_n,w_n)$ received in (13) exhibits a robust subsequence of convergence.

Proof. 

It follows from Lemma 5 that $(z_n,w_n)$ is bounded in D and its subsequences $(z_n,w_n)$, as $n\to \infty$ satisfy $(z_n,w_n)\rightharpoonup (z,w)\in D$ and $⟨I_k^{\prime }(z_n,w_n),(z_n,w_n)⟩=o_n(1)$, adding Lemma 6 to reveal

$$\begin{array}{cc}\hfill & \underset{n\to \infty }{lim}({∥z_n∥}^p+{∥w_n∥}^q)={\int }_{{\mathbb{R}}^N}⟨\nabla \eta (x,z,w),(z,w)⟩dx.\hfill \end{array}$$

From $⟨I_k^{\prime }(z_n,w_n),(z,w)⟩=o_n(1)$,

$$\begin{array}{cc}\hfill & \underset{n\to \infty }{lim}({\int }_{{\mathbb{R}}^N}|\nabla z_n{|}^{p-1}\nabla zdx+{\int }_{{\mathbb{R}}^N}{V}_1(x)|z_n{|}^{p-1}zdx+{\int }_{{\mathbb{R}}^N}|\nabla w_n{|}^{q-1}\nabla wdx+{\int }_{{\mathbb{R}}^N}{V}_2(x)|w_n{|}^{q-1}wdx)\hfill \\ & ={\int }_{{\mathbb{R}}^N}\nabla \eta (x,z,w)dx+o_n(1),\hfill \end{array}$$

equivalent to

$$\underset{n\to \infty }{lim}({∥z_n∥}^p+{∥w_n∥}^q)={∥z∥}^p+{∥w∥}^q.$$

Therefore, $(z_n,w_n)\to (z,w)$ in D. □

From Lemmas 4–7, similar to [20], Theorem 2 can be concluded.

Theorem 2.

$(z,w)$ is nontrivial solution of the problem (7), if $({\mathcal{V}}_1),({\mathcal{V}}_2)$ and $(f_1)-(f_3)$ is true.

4. Proof of Main Result

Now, we try to prove that the solution $(u,v)=(Y^{-1}(z),Y^{-1}(w))$ is solution of (1).

Lemma 8.

$(z,w)$ is a nontrivial critical point of $I_k$, the critical value is c, in that, $\exists K\in \mathbb{R}$ with $K>0$ unrelated to ι make

$$\begin{array}{c}\hfill {∥z∥}^p+{∥w∥}^q\le Kc.\end{array}$$

(28)

Proof. 

Add (14) from Lemma 1 $(ii)$, $(iii)$ and $(f_3)$, there is

$$\begin{array}{cc}\hfill \theta c& =\theta I_k(z,w)-⟨I_k^{\prime }(z,w),(Y^{-1}(z)y_u(Y^{-1}(z)),Y^{-1}(w)y_v(Y^{-1}(w)))⟩\hfill \\ & \le (\frac{1}{p}-\frac{1}{\theta })({∥z∥}^p+{∥w∥}^q).\hfill \end{array}$$

Hence,

$${∥z∥}^p+{∥w∥}^q\le \frac{\theta pc}{\theta -p}=Kc.$$

The proof is completed. □

Lemma 9.

$(z,w)$ is the critical point of the function $I_k(z,w)$, in that, $\exists C\in \mathbb{R}$ with $C>0$, and C is unrelated to ι makes

$$\begin{array}{c}\hfill {∥z∥}_{\infty }\le C{\iota }^{\frac{1}{p^{*}-l_p}}{∥z∥}_{p^{*}},{∥w∥}_{\infty }\le C{\iota }^{\frac{1}{q^{*}-l_q}}{∥w∥}_{q^{*}}.\end{array}$$

(29)

Proof. 

For every $n_0\in \mathbb{N}$, taking $\beta >1$ be the given constant, let

$$A_{n_0}=\{x\in {\mathbb{R}}^N{:|z|}^{\beta -1}\le n_0{,|w|}^{\beta -1}\le n_0\},B_{n_0}={\mathbb{R}}^N\setminus A_{n_0},$$

$$(u_{n_0},v_{n_0})=\left\{\begin{array}{c}{(z|z|}^{p(\beta -1)}{,w|w|}^{q(\beta -1)}),x\in A_{n_0},\hfill \\ {n_0}^2(z,w),x\in B_{n_0}\hfill \end{array}\right.$$

as well as

$$(z_{n_0},w_{n_0})=\left\{\begin{array}{c}{(z|z|}^{\beta -1}{,w|w|}^{\beta -1}),x\in A_{n_0},\hfill \\ ({n_0}^{\frac{2}{p}},{n_0}^{\frac{2}{q}})(z,w),x\in B_{n_0}.\hfill \end{array}\right.$$

Apparently, $(u_{n_0},v_{n_0}),(z_{n_0},w_{n_0})\in D$. For $(z,w)$ is a nontrivial solution of (7), in that

$$\begin{array}{cc}\hfill & {\int }_{{\mathbb{R}}^N}\left({|\nabla z|}^{p-2}\nabla z\nabla u_{n_0}+V_1(x)\frac{|Y^{-1}{(z)|}^{p-1}}{y_u(Y^{-1}(z))}{u}_{n_0}\right)dx\hfill \\ & +{\int }_{{\mathbb{R}}^N}\left({|\nabla w|}^{q-2}\nabla w\nabla v_{n_0}+V_2(x)\frac{|Y^{-1}{(w)|}^{q-1}}{y_v(Y^{-1}(w))}{v}_{n_0}\right)dx\hfill \\ & =\iota {\int }_{{\mathbb{R}}^N}\left(\frac{f(x,Y^{-1}(z),Y^{-1}(w))}{y_u(Y^{-1}(z))}{u}_{n_0}+\frac{h(x,Y^{-1}(z),Y^{-1}(w))}{y_v(Y^{-1}(w))}{v}_{n_0}\right)dx.\hfill \end{array}$$

Furthermore,

$$\begin{array}{cc}\hfill & {\int }_{{\mathbb{R}}^N}{|\nabla z|}^{p-1}\nabla u_{n_0}dx+{\int }_{{\mathbb{R}}^N}{|\nabla w|}^{q-1}\nabla v_{n_0}dx\hfill \\ \hfill =& (p\beta -p+1){\int }_{A_{n_0}}{|\nabla z|}^p{|z|}^{p(\beta -1)}dx+(q\beta -q+1){\int }_{A_{n_0}}{|\nabla w|}^q{|w|}^{q(\beta -1)}dx\hfill \\ & +{n_0}^2{\int }_{B_{n_0}}{|\nabla z|}^{p}dx+{n_0}^2{\int }_{B_{n_0}}{|\nabla w|}^{q}dx,\hfill \end{array}$$

(30)

$$\begin{array}{cc}\hfill & {\int }_{{\mathbb{R}}^N}|\nabla z_{n_0}{|}^{p}dx+{\int }_{{\mathbb{R}}^N}{|\nabla w_{n_0}|}^{q}dx\hfill \\ \hfill =& {\beta }^p{\int }_{A_{n_0}}{|\nabla z|}^p{|z|}^{p(\beta -1)}dx+{\beta }^q{\int }_{A_{n_0}}{|\nabla w|}^q{|w|}^{q(\beta -1)}dx+{n_0}^2{\int }_{B_{n_0}}{|\nabla z|}^{p}dx+{n_0}^2{\int }_{B_{n_0}}{|\nabla w|}^{q}dx.\hfill \end{array}$$

(31)

Therefore, we have

$$\begin{array}{cc}\hfill & {\int }_{{\mathbb{R}}^N}{|\nabla z|}^{p-1}\nabla u_{n_0}dx+{\int }_{{\mathbb{R}}^N}{|\nabla w|}^{q-1}\nabla v_{n_0}dx-{n_0}^2{\int }_{B_{n_0}}{|\nabla z|}^{p}dx-{n_0}^2{\int }_{B_{n_0}}{|\nabla w|}^{q}dx\hfill \\ \hfill \ge & {\int }_{A_{n_0}}{|\nabla z|}^p{|z|}^{p(\beta -1)}dx+{\int }_{A_{n_0}}{|\nabla w|}^q{|w|}^{q(\beta -1)}dx.\hfill \end{array}$$

(32)

From (31) and (32), let ${\beta }^c=max\{{\beta }^p,{\beta }^q\}$, we obtain

$$\begin{array}{cc}\hfill & {\int }_{{\mathbb{R}}^N}|\nabla z_{n_0}{|}^{p}dx+{\int }_{{\mathbb{R}}^N}{|\nabla w_{n_0}|}^{q}dx\hfill \\ \hfill \le & {\beta }^c{\int }_{{\mathbb{R}}^N}{|\nabla z|}^{p-1}\nabla u_{n_0}dx+{\beta }^c{\int }_{{\mathbb{R}}^N}{|\nabla w|}^{q-1}\nabla v_{n_0}dx\hfill \\ & -{n_0}^2{\beta }^c{\int }_{B_{n_0}}{|\nabla z|}^{p}dx-{n_0}^2{\beta }^c{\int }_{B_{n_0}}{|\nabla w|}^{q}dx+{n_0}^2{\int }_{B_{n_0}}{|\nabla z|}^{p}dx+{n_0}^2{\int }_{B_{n_0}}{|\nabla w|}^{q}dx\hfill \\ \hfill \le & {\beta }^c{\int }_{{\mathbb{R}}^N}{|\nabla z|}^{p-1}\nabla u_{n_0}dx+{\beta }^c{\int }_{{\mathbb{R}}^N}{|\nabla w|}^{q-1}\nabla v_{n_0}dx.\hfill \end{array}$$

(33)

It follows from (33) and $\beta >1$ that

$$\begin{array}{cc}\hfill & {\int }_{{\mathbb{R}}^N}|\nabla z_{n_0}{|}^{p}dx+{\int }_{{\mathbb{R}}^N}{|\nabla w_{n_0}|}^{q}dx+{\beta }^c{\int }_{{\mathbb{R}}^N}\left(V_1(x)\frac{|Y^{-1}{(z)|}^{p-1}}{y_u(Y^{-1}(z))}{u}_{n_0}+V_2(x)\frac{|Y^{-1}{(w)|}^{q-1}}{y_v(Y^{-1}(w))}{v}_{n_0}\right)dx\hfill \\ \hfill \le & {\beta }^c{\int }_{{\mathbb{R}}^N}{|\nabla z|}^{p-1}\nabla u_{n_0}dx+{\beta }^c{\int }_{{\mathbb{R}}^N}{|\nabla w|}^{q-1}\nabla v_{n_0}dx\hfill \\ & +{\beta }^c{\int }_{{\mathbb{R}}^N}\left(V_1(x)\frac{|Y^{-1}{(z)|}^{p-1}}{y_u(Y^{-1}(z))}{u}_{n_0}+V_2(x)\frac{|Y^{-1}{(w)|}^{q-1}}{y_v(Y^{-1}(w))}{v}_{n_0}\right)dx\hfill \\ \hfill =& {\beta }^c\iota {\int }_{{\mathbb{R}}^N}\left(\frac{f(x,Y^{-1}(z),Y^{-1}(w))}{y_u(Y^{-1}(z))}{u}_{n_0}+\frac{h(x,Y^{-1}(z),Y^{-1}(w))}{y_v(Y^{-1}(w))}{v}_{n_0}\right)dx\hfill \\ \hfill \le & {\beta }^c\iota {\int }_{{\mathbb{R}}^N}ϵ\left(\frac{{(Y^{-1}(z))}^{p-1}}{y_u(Y^{-1}(z))}{u}_{n_0}+\frac{{(Y^{-1}(w))}^{q-1}}{y_v(Y^{-1}(w))}{v}_{n_0}\right)dx\hfill \\ & +{\beta }^c\iota {\int }_{{\mathbb{R}}^N}C\left(\frac{{(Y^{-1}(z))}^{l_p-1}}{y_u(Y^{-1}(z))}{u}_{n_0}+\frac{{(Y^{-1}(w))}^{l_q-1}}{y_v(Y^{-1}(w))}{v}_{n_0}\right)dx\hfill \\ \hfill \le & {\beta }^c{\int }_{{\mathbb{R}}^N}\left(V_1(x)\frac{{(Y^{-1}(z))}^{p-1}}{y_u(Y^{-1}(z))}{u}_{n_0}+V_2(x)\frac{{(Y^{-1}(w))}^{q-1}}{y_v(Y^{-1}(w))}{v}_{n_0}\right)dx\hfill \\ & +{\beta }^c\iota {\int }_{{\mathbb{R}}^N}C\left(\frac{{(Y^{-1}(z))}^{l_p-1}}{y_u(Y^{-1}(z))}{u}_{n_0}+\frac{{(Y^{-1}(w))}^{l_q-1}}{y_v(Y^{-1}(w))}{v}_{n_0}\right)dx,\hfill \end{array}$$

(34)

where $0<ϵ<min\{\frac{V_1(x)}{\iota },\frac{V_1(x)}{\iota }\}$. By Lemma $(iii)$ and the fact of $z^{p-1}{u}_{n_0}=z_{n_0}^p$ and $w^{q-1}{v}_{n_0}=w_{n_0}^q$, we can obtain

$$\begin{array}{cc}\hfill & {\int }_{{\mathbb{R}}^N}|\nabla z_{n_0}{|}^{p}dx+{\int }_{{\mathbb{R}}^N}{|\nabla w_{n_0}|}^{q}dx\hfill \\ \hfill \le & {\beta }^c\iota C{\int }_{{\mathbb{R}}^N}\left(\frac{{(Y^{-1}(z))}^{l_p-1}}{y_u(Y^{-1}(z))}{u}_{n_0}+\frac{{(Y^{-1}(w))}^{l_q-1}}{y_v(Y^{-1}(w))}{v}_{n_0}\right)dx\hfill \\ \hfill \le & {\beta }^c\iota C{\int }_{{\mathbb{R}}^N}\left(2^{(-\frac{l_p(p+1)}{p})}{|z|}^{l_p-p}{z}_{n_0}^p+2^{(-\frac{l_q(q+1)}{q})}{|w|}^{l_q-q}{w}_{n_0}^q\right)dx\hfill \\ \hfill \le & {\beta }^c\iota C{\int }_{{\mathbb{R}}^N}\left({|z|}^{l_p-p}{z}_{n_0}^p+{|w|}^{l_q-q}{w}_{n_0}^q\right)dx.\hfill \end{array}$$

(35)

If $J(a)+J(b)\le L(a)+L(b)$, we have $J(a)\le L(a),J(b)\le L(b)$. It follows from (35) that

$$\begin{array}{c}\hfill {\int }_{{\mathbb{R}}^N}|\nabla z_{n_0}{|}^{p}dx\le {\beta }^c\iota C{\int }_{{\mathbb{R}}^N}{|z|}^{l_p-p}{z}_{n_0}^{p}dx,\end{array}$$

(36)

$$\begin{array}{c}\hfill {\int }_{{\mathbb{R}}^N}|\nabla w_{n_0}{|}^{q}dx\le {\beta }^c\iota C{\int }_{{\mathbb{R}}^N}{|w|}^{l_q-q}{w}_{n_0}^{q}dx.\end{array}$$

(37)

From the Sobolev inequality, when $S>0$, we have

$$({\int }_{A_{n_0}}|\nabla z_{n_0}{|}^{p^{*}}{dx)}^{\frac{N-p}{N}}\le S{\int }_{{\mathbb{R}}^N}{|\nabla z_{n_0}|}^{p}dx,$$

combining (36) and Hölder inequality, we can obtain

$$({\int }_{A_{n_0}}|\nabla z_{n_0}{{|}^{p^{*}}dx)}^{\frac{N-p}{N}}\le {\beta }^c\iota SC{∥z∥}_{p_2}^{l_p-p}{∥z∥}_{p_1}^p,$$

where $\frac{p}{p_1}+\frac{l_p-p}{p^{*}}=1$. Note that $|z_{n_0}{|=|z|}^{\beta }$ in $A_{n_0}$ and $|z_{n_0}{|\le |z|}^{\beta }$, thus

$$({\int }_{A_{n_0}}|\nabla z_{n_0}{{|}^{p^{*}\beta }dx)}^{\frac{N-p}{N}}\le {\beta }^c\iota SC{∥z∥}_{p_2}^{l_p-p}{∥z∥}_{p_1\beta }^{p\beta }.$$

Action $n_0\to \infty$ on the above equation, with

$$\begin{array}{c}\hfill {∥z∥}_{\beta p^{*}}\le {\beta }^{\frac{c}{p\beta }}{\iota }^{\frac{1}{p\beta }}{S}^{\frac{1}{p\beta }}{C}^{\frac{1}{p\beta }}{∥z∥}_{p_2}^{(l_p-p)\frac{1}{p\beta }}{∥z∥}_{p_1\beta }.\end{array}$$

(38)

Denoting $\sigma =\frac{p^{*}}{p_1}$ and let $\beta =\sigma$ in (38), we can obtain

$$\begin{array}{c}\hfill {∥z∥}_{\sigma p^{*}}\le {\sigma }^{\frac{c}{p\sigma }}{\iota }^{\frac{1}{p\sigma }}{S}^{\frac{1}{p\sigma }}{C}^{\frac{1}{p\sigma }}{∥z∥}_{p_2}^{(l_p-p)\frac{1}{p\sigma }}{∥z∥}_{p^{*}}.\end{array}$$

(39)

Taking $\beta ={\sigma }^2$, we see that

$$\begin{array}{c}\hfill {∥z∥}_{{\sigma }^2{p}^{*}}\le {\sigma }^{\frac{2c}{p{\sigma }^2}}{\iota }^{\frac{1}{p{\sigma }^2}}{S}^{\frac{1}{p{\sigma }^2}}{C}^{\frac{1}{p{\sigma }^2}}{∥z∥}_{p_2}^{(l_p-p)\frac{1}{p{\sigma }^2}}{∥z∥}_{p^{*}\sigma }.\end{array}$$

(40)

From (39) and (40), we have

$$\begin{array}{c}\hfill {∥z∥}_{{\sigma }^2{p}^{*}}\le {\sigma }^{\frac{c}{p}(\frac{1}{\sigma }+\frac{2}{{\sigma }^2})}{\iota }^{\frac{1}{p}(\frac{1}{\sigma }+\frac{1}{{\sigma }^2})}{S}^{\frac{1}{p}(\frac{1}{\sigma }+\frac{1}{{\sigma }^2})}{C}^{\frac{1}{p}(\frac{1}{\sigma }+\frac{1}{{\sigma }^2})}{∥z∥}_{p_2}^{(l_p-p)\frac{1}{p}(\frac{1}{\sigma }+\frac{1}{{\sigma }^2})}{∥z∥}_{p^{*}}.\end{array}$$

For (38), continuing this approch by taking $\beta ={\sigma }^j(j=1,2,\dots )$, then

$$\begin{array}{c}\hfill {∥z∥}_{{\sigma }^i{p}^{*}}\le {\sigma }^{\frac{c}{p}{\sum }_{j=1}^i\frac{i}{{\sigma }^j}}{({\iota }^{\frac{1}{p}}{S}^{\frac{1}{p}}{C}^{\frac{1}{p}}{∥z∥}_{p_2}^{(l_p-p)\frac{1}{p}})}^{{\sum }_{j=1}^i\frac{1}{{\sigma }^j}}{∥z∥}_{p^{*}}.\end{array}$$

Setting $i\to +\infty$ and using the Sobolev inequality, then

$$\begin{array}{cc}\hfill {∥z∥}_{\infty }& \le {\sigma }^{\frac{c}{p{(\sigma -1)}^2}}{({\iota }^{\frac{1}{p}}{S}^{\frac{1}{p}}{C}^{\frac{1}{p}}C)}^{\frac{1}{(\sigma -1)}}{∥z∥}_{p^{*}}\hfill \\ & =C{\iota }^{\frac{1}{p(\sigma -1)}}{∥z∥}_{p^{*}}\hfill \\ & =C{\iota }^{\frac{1}{p^{*}-l_p}}{∥z∥}_{p^{*}},\hfill \end{array}$$

(41)

in which C is not related to $\iota$. Similarly, we have

$$\begin{array}{c}\hfill {∥w∥}_{\infty }\le C{\iota }^{\frac{1}{q^{*}-l_q}}{∥w∥}_{q^{*}},\end{array}$$

where C is not related to $\iota$. □

Proof of Theorem 1.

Let $\chi >0$ and

$$T=\{x\in {\mathbb{R}}^N:{\varphi }_1(x)\ge \chi \}\cap \{x\in {\mathbb{R}}^N:{\varphi }_2(x)\ge \chi \}$$

be a nonempty set. From $(f_2)$ and $(f_3)$, for $x\in T$, there is $C>0$, which makes

$$\begin{array}{c}\hfill \zeta (x,s,t)\ge {C|(s,t)|}^{(l_p,l_q)}.\end{array}$$

(42)

Supposing that $(z,w)$ be a critical point of $I_k$ with the critical value c. By Theorem 2 and (42), we obtain

$$\begin{array}{cc}\hfill c\le & \underset{t>0}{max}{I}_k(t{\varphi }_1,t{\varphi }_2)\hfill \\ \hfill \le & \underset{t>0}{max}(\frac{t^p}{p}{\int }_{{\mathbb{R}}^N}|\nabla {\varphi }_1{|}^{p}dx+\frac{t^p}{p}{\int }_{{\mathbb{R}}^N}{2}^{(-1-p)}{V}_1(x){|{\varphi }_1|}^{p}dx\hfill \\ & +\frac{t^q}{q}{\int }_{{\mathbb{R}}^N}|\nabla {\varphi }_2{|}^{q}dx+\frac{t^q}{q}{\int }_{{\mathbb{R}}^N}{2}^{(-1-q)}{V}_2(x){|{\varphi }_2|}^{q}dx\hfill \\ & -\iota t^{l_p}C{2}^{-\frac{(p+1)l_p}{p}}{\int }_{{\mathbb{R}}^N}|{\varphi }_1{|}^{l_p}dx-\iota t^{l_q}C{2}^{-\frac{(q+1)l_q}{q}}{\int }_{{\mathbb{R}}^N}{|{\varphi }_2|}^{l_q}dx)\hfill \\ \hfill \le & C{\iota }^{-\frac{p}{l_p-p}}.\hfill \end{array}$$

From (28), (29) and the continuous embedding $D_1\hookrightarrow L^{r_1},D_2\hookrightarrow L^{r_2}$, we obtain

$$\begin{array}{cc}\hfill {∥z∥}_{\infty }& \le C{\iota }^{\frac{1}{p^{*}-l_p}}{∥z∥}_{p^{*}}\le C{\iota }^{\frac{1}{p^{*}-l_p}}∥z∥\le C{\iota }^{\frac{1}{p^{*}-l_p}}{(Kc)}^{\frac{1}{p}}\hfill \\ & \le C{\iota }^{\frac{1}{p^{*}-l_p}}{(K{\iota }^{-\frac{p}{l_p-p}})}^{\frac{1}{p}}\le C_1{\iota }^{\frac{2l_p-p^{*}-p}{(p^{*}-l_p)(l_p-p)}},\hfill \end{array}$$

(43)

where $C_1$ is a constant. Since $p<l_p<p^{*}$, for given $k>0$, there is ${\iota }_1(k)={(2^{3}k{C}_1^p)}^{\frac{(p^{*}-l_p)(l_p-p)}{p(p^{*}-2l_p+p)}}$, which makes for each $\iota >{\iota }_1(k)$, it satisfies

$$\begin{array}{c}\hfill {∥u∥}_{\infty }={∥Y^{-1}(z)∥}_{\infty }\le 2^{-\frac{1}{p}-1}({∥z∥}_{\infty }\le 2^{-\frac{1}{p}-1}{C}_1{\iota }^{\frac{2l_p-p^{*}-p}{(p^{*}-l_p)(l_p-p)}}\le {(\frac{1}{2^{p-3}k})}^{\frac{1}{p}}.\end{array}$$

Similarly, we may obtain ${∥v∥}_{\infty }\le {(\frac{1}{2^{q-3}k})}^{\frac{1}{q}}.$ Hence, the system (1) has a nontrivial solution $(u,v)=(Y^{-1}(z),Y^{-1}(w))$. □

5. Conclusions

We study the related problem of the quasilinear Schrödinger system containing the operator ${\Delta }_p$ and ${\Delta }_q$. By using the variable transformation to process quasilinear terms, combined with the mountain-pass theorem, we received a nontrivial solution of the system. It is worth considering whether the variable exponent has an impact on the above conclusion, and trying to extend p and q to $p(x)$ and $q(x)$ is also a meaningful issue.