A Class of Subcritical and Critical Schrödinger–Kirchhoff Equations with Variable Exponents

Abstract

In the present paper, we discuss a Schrödinger–Kirchhoff equation involving the $p\left(x\right)$-Laplacian in the entire space ${\mathbb{R}}^N$. The primary focus of this article is on subcritical and critical nonlinearities. We deduce the existence of solutions by employing the mountain pass theorem in two distinct scenarios. Firstly, we discuss the equation when the potential function satisfies a weaker condition in the subcritical case. Secondly, we address the lack of compactness in the critical case without utilizing the concentration compactness principle.

1. Introduction

Over the past few decades, Kirchhoff problems have emerged in various models of physical and biological systems. Specifically, Kirchhoff, in [1], proposed the following model:

$$\rho {\displaystyle \frac{{\partial }^2\upsilon }{\partial t^2}}-\left({\displaystyle \frac{p_0}{\lambda }}+{\displaystyle \frac{e}{2L}}{\int }_0^L{\left|{\displaystyle \frac{\partial \upsilon }{\partial x}}\right|}^{2}dx\right){\displaystyle \frac{{\partial }^2\upsilon }{\partial x^2}}=0,$$

where the constants $\rho$, $\lambda$, $p_0$, e, and L represent some specific physical meanings. Kirchhoff model is a generalization of the classical d’Alembert wave equation. The classical Schrödinger equation is one of the most fundamental equations in quantum mechanics, which profoundly reveals the laws of motion of microscopic particles in space and time. Recently, the focus of attention has been on the study of the following Schrödinger equation:

$$-\Delta \upsilon +V\left(x\right)\upsilon =h(x,\upsilon ),x\in {\mathbb{R}}^N,$$

where V represents the potential function and $h(x,\upsilon )$ satisfies some suitable assumptions as detailed in [2,3]. The Schrödinger–Kirchhoff equation not only has wide applications in physics but also appears in many models of biological systems. Since its introduction, various forms of the Schrödinger–Kirchhoff equation have been formulated, and the existence of solutions was extensively studied mainly using critical point theory [4,5,6,7,8,9,10,11,12].

The $p(·)$-Laplacian is a generalization of the p-Laplacian, which itself is a generalization of the Laplacian. It not only has many important applications for electrorheological fluids and image processing but also presents mathematical challenges, such as inhomogeneity. In [13], Cammaroto and Vilasi investigated the following Schrödinger–Kirchhoff equation involving the $p\left(x\right)$-Laplacian:

$$k\left({\int }_{{\mathbb{R}}^N}{\displaystyle \frac{{|\nabla \upsilon |}^{p\left(x\right)}+V\left(x\right){\left|\upsilon \right|}^{p\left(x\right)}}{p\left(x\right)}}dx\right)(-{\Delta }_{p\left(x\right)}\upsilon +{V\left(x\right)\left|\upsilon \right|}^{p\left(x\right)-2}\upsilon )=\phi (\lambda ,\mu ,x,\upsilon ),x\in {\mathbb{R}}^N,$$

where $\phi (\lambda ,\mu ,x,\upsilon )=\lambda f(x,\upsilon )+\mu g(x,\upsilon )$, $\lambda ,\mu$ are real parameters. The multiplicity results are obtained based on the variational approach and the range of the parameters. Xie and Chen [14] derived the existence of multiple solutions for a Schrödinger $p\left(x\right)$-Laplacian equation utilizing Morse theory and minimax methods. For additional details and recent works, refer to [15,16,17,18,19,20,21,22].

Currently, the critical exponent problem is a focal area of ongoing research, attracting considerable attention from scholars. It is worth stressing that critical nonlinearity will bring the difficulty of lack of compactness to elliptic equations. In [23], Fu and Zhang first proved that the concentration compactness principle (CCP) applies in the case of the $p\left(x\right)$-Laplacian. Building upon these pivotal results, Zhang and Qin [24] focused their attention on the following critical Choquard–Kirchhoff problem with variable exponents:

$$\left\{\begin{array}{c}\mathcal{M}\left(H\right(x,\left|\upsilon \right|,|\nabla \upsilon |\left)\right)\left[-{\Delta }_{p\left(x\right)}\upsilon +{\left|\upsilon \right|}^{p\left(x\right)-2}\upsilon \right]=\lambda \left[{\int }_{{\mathbb{R}}^N}{\displaystyle \frac{F\left(\upsilon \right(y\left)\right)}{{|x-y|}^{\alpha (x,y)}}}dy\right]f\left(\upsilon \right)+{\left|\upsilon \right|}^{p^{\ast }-2}\upsilon ,\hfill \\ \upsilon \in W^{1,p\left(x\right)}\left({\mathbb{R}}^N\right),\hfill \end{array}\right.$$

where $\mathcal{M}$ is a Kirchhoff function, F is the primitive of f, and $p^{\ast }=Np\left(x\right)/(N-p\left(x\right))$ is the critical Sobolev exponent. In addition, the existence results for Schrödinger $p\left(x\right)$-Laplacian equations with a concave–convex term and critical growth were investigated in [25,26], respectively, by using the CCP in the context of weighted variable-exponent Sobolev spaces. For additional insights into critical exponent problems, please refer to [27,28,29,30,31,32,33,34,35,36,37]. Notably, Fan [36] derived some innovative results regarding the fractional Choquard–Kirchhoff equation

$$\left\{\begin{array}{c}M\left({\int }_{{\mathbb{R}}^N}{\int }_{{\mathbb{R}}^N}{\displaystyle \frac{{|\upsilon \left(x\right)-\upsilon \left(y\right)|}^2}{{|x-y|}^{N+2s}}}dxdy\right){(-\Delta )}^s\upsilon =\lambda {\int }_{\Omega }{\displaystyle \frac{{\left|\upsilon \right|}^p}{{|x-y|}^{\mu }}}{\left|\upsilon \right|}^{p-2}\upsilon +{\left|\upsilon \right|}^{q-2}\upsilon ,x\in \Omega ,\hfill \\ \upsilon =0,x\in {\mathbb{R}}^N\setminus \Omega ,\hfill \end{array}\right.$$

where $M\left(t\right)=a+b{t}^{\theta -1}$, with $\theta \in (1,2_s^{\ast }/2)$ and $2\theta <q\le 2_s^{\ast }=2N/(N-2s)$, and where $a,b>0$ are constants. In the critical case, the author studied the mountain pass level and established the existence of a solution by comparing the zeros of certain functions, rather than relying on the CCP.

To our knowledge, however, the findings pertaining to Schrödinger–Kirchhoff equations involving the $p\left(x\right)$-Laplacian in the entire space ${\mathbb{R}}^N$ remain relatively scarce. Inspired by the aforementioned literature, this paper delves into the following problem:

$$M\left(\Phi \left(\upsilon \right)\right)\left[-{\Delta }_{p\left(x\right)}\upsilon +\lambda V\left(x\right){\left|\upsilon \right|}^{p\left(x\right)-2}\upsilon \right]=h(x,\upsilon ),x\in {\mathbb{R}}^N,\left(H_{\lambda }\right)$$

where

$$\begin{array}{c}\hfill \Phi \left(\upsilon \right)={\int }_{{\mathbb{R}}^N}{\displaystyle \frac{{|\nabla \upsilon |}^{p\left(x\right)}+\lambda V\left(x\right){\left|\upsilon \right|}^{p\left(x\right)}}{p\left(x\right)}}dx\end{array}$$

and $p\in C\left({\mathbb{R}}^N\right)$ with $p\left(x\right)<N$ for each $x\in {\mathbb{R}}^N$. M is a Kirchhoff function, $V\in C({\mathbb{R}}^N,{\mathbb{R}}^{+})$ is a potential function, $\lambda >0$ is a parameter, and $h\in C({\mathbb{R}}^N\times \mathbb{R},\mathbb{R})$ is a Carathéodory function. The operator $-{\Delta }_{p\left(x\right)}$ represents the $p\left(x\right)$-Laplacian, which is defined by

$$-{\Delta }_{p\left(x\right)}\upsilon \left(x\right):=-{div\left(\right|\nabla \upsilon |}^{p\left(x\right)-2}\nabla \upsilon ).$$

First, we will introduce some notations. For any function $y\in C\left({\mathbb{R}}^N\right)$, we denote

$$y^{-}:=\underset{x\in {\mathbb{R}}^N}{inf}y\left(x\right),y^{+}:=\underset{x\in {\mathbb{R}}^N}{sup}y\left(x\right),$$

and

$$C_{+}\left(R^N\right):=\left\{y\in C\left({\mathbb{R}}^N\right):1<y^{-}\le y\left(x\right)\le y^{+}<\infty \right\}.$$

Throughout this article, $p^{\ast }\left(x\right)={\displaystyle \frac{Np\left(x\right)}{N-p\left(x\right)}}$ denotes the critical exponent. Next, we consider the Kirchhoff function M that satisfies the following condition:

(M1)

$M\in C\left({\mathbb{R}}_0^{+}\right)$, and there exists a constant $m>0$ such that ${inf}_{t\ge 0}M\left(t\right)\ge m$.

(M2)

There exists $\zeta \in (1,{\left(p^{\ast }\right)}^{-}/p^{+})$ satisfying $M\left(t\right)t\le \zeta \widehat{M}\left(t\right)$ for each $t\ge 0$, where $\widehat{M}\left(t\right)={\int }_0^{t}M\left(ϵ\right)dϵ$.

A classic example of a Kirchhoff function is given by $M\left(t\right)=m+b{t}^{\zeta }$, where $b\ge 0$ and $t\ge 0$. In order to overcome the lack of compactness when studying an elliptic equation in ${\mathbb{R}}^N$, various methods can be employed, such as the weighting method [16], the radially symmetric method [24] and the method where the potential is coercive [22]. In this work, we assume the following conditions on the potential function V:

(V1)

$V\in C({\mathbb{R}}^N,{\mathbb{R}}^{+})$. Moreover, there exists $a>0$ such that $\mathcal{A}:=\{x\in {\mathbb{R}}^N:V\left(x\right)\le a\}\ne \varnothing$ and $meas\{x\in {\mathbb{R}}^N:V\left(x\right)\le a\}<+\infty$.

(V2)

$V\left(x\right)$ satisfies ${inf}_{x\in {\mathbb{R}}^N}V\left(x\right)=V_0>0$, and for every $b>0$, $meas\{x\in {\mathbb{R}}^N:V\left(x\right)\le b\}<+\infty$.

It is important to note that condition (V1) is less stringent than (V2), and consequently, it fails to ensure the compact embedding of the Sobolev space into the Lebesgue space. This limitation precludes the application of standard critical point theory. Inspired by Ding [6] and Guo [8], who dealt with the Schrödinger equation, we will discuss the problem $\left(H_{\lambda }\right)$ for the subcritical case with condition (V1) and study the problem $\left(H_{\lambda }\right)$ for the critical case with condition (V2). Our primary analytical tool will be the mountain pass theorem. Compared to [6,8], the variable exponent we deal with is more complex. In addition, we address the lack of compactness in the critical case without using the CCP, which is different from the work of [23,24]. Now, we present the following hypotheses on $h(x,\upsilon )$.

(H1)

There exist $d_1,d_2>0$ and $q\left(x\right)\in C_{+}\left({\mathbb{R}}^N\right)$ such that

$$\left|h(x,\upsilon )\right|\le d_1{\left|\upsilon \right|}^{p\left(x\right)-1}+d_2{\left|\upsilon \right|}^{q\left(x\right)-1},(x,\upsilon )\in {\mathbb{R}}^N\times \mathbb{R},$$

where $2\le p^{-}\le p\left(x\right)\le p^{+}<q^{-}\le q\left(x\right)\le q^{+}<{\left(p^{\ast }\right)}^{-}\le p^{\ast }\left(x\right)$ and ${\displaystyle \frac{q\left(x\right)-1}{p\left(x\right)-1}}<{\displaystyle \frac{N}{N-p\left(x\right)}}$.

(H2)

$h(x,\upsilon )=o\left(\right|\upsilon {|}^{p^{+}-2}\upsilon )$ as $\left|\upsilon \right|\to 0$ uniformly in $x\in {\mathbb{R}}^N$.

(H3)

${lim}_{\left|\upsilon \right|\to \infty }{\displaystyle \frac{H(x,\upsilon )}{{\left|\upsilon \right|}^{\zeta p^{+}}}}=\infty$ uniformly in $x\in {\mathbb{R}}^N$.

(H4)

There exists $\xi >0$ such that

$$\zeta {\left(p^{+}\right)}^{2}H(x,\upsilon )-h(x,\upsilon )\upsilon \le \xi {\left|\upsilon \right|}^{p\left(x\right)},\mathrm{for}\mathrm{any}(x,\upsilon )\in {\mathbb{R}}^N\times \mathbb{R},$$

where $H(x,\upsilon )={\int }_0^{\upsilon }h(x,s)ds$.

(H5)

$h(x,\upsilon )={\mu \left|\upsilon \right|}^{r\left(x\right)-2}\upsilon +{\left|\upsilon \right|}^{p^{\ast }\left(x\right)-2}\upsilon$, where $1<p^{-}\le p^{+}<\zeta p^{+}<r^{-}\le r^{+}<(p*{)}^{-}$.

Our main results are stated as follows.

Theorem 1.

Suppose that conditions (M1)–(M2), (V1), and (H1)–(H4) hold. Then, equations $\left(H_{\lambda }\right)$ possess a weak solution for sufficiently large $\lambda >0$.

Theorem 2.

Suppose that conditions (M1)–(M2), (V2), and (H5) hold. Then, there exists ${\mu }^{\ast }>0$ such that for any $\mu \ge {\mu }^{\ast }$, the equations $\left(H_{\lambda }\right)$ possess at least one weak solution.

The remainder of this article is organized as follows. In Section 2, we provide some necessary definitions, basic lemmas, and important theorems. In Section 3 and Section 4, we present the proofs of Theorem 1 and Theorem 2, respectively.

2. Preliminaries

In this section, we review some known results of corresponding function spaces and propose a functional framework related to problems $\left(H_{\lambda }\right)$.

Let $\mathcal{M}\left({\mathbb{R}}^N\right)$ be the space of all measurable functions. Subsequently, for each $\theta \in C_{+}\left({\mathbb{R}}^N\right)$, we define the Lebesgue space with a variable exponent as

$$L^{\theta \left(x\right)}\left({\mathbb{R}}^N\right):=\left\{\upsilon \in \mathcal{M}\left({\mathbb{R}}^N\right),{\int }_{{\mathbb{R}}^N}{\left|\upsilon \left(x\right)\right|}^{\theta \left(x\right)}dx<\infty \right\},$$

which is a reflexive and separable Banach space (see [15]) with the Luxemburg norm

$${\left|\upsilon \right|}_{\theta \left(x\right)}={\left|\upsilon \right|}_{L^{\theta \left(x\right)}\left({\mathbb{R}}^N\right)}:=\inf \left\{\chi >0:{\int }_{{\mathbb{R}}^N}{\left|{\displaystyle \frac{\upsilon \left(x\right)}{\chi }}\right|}^{\theta \left(x\right)}dx\le 1\right\}.$$

Lemma 1

([15]). Suppose that the modular $\varrho \left(\upsilon \right)={\int }_{{\mathbb{R}}^N}{\left|\upsilon \right|}^{\theta \left(x\right)}dx$, $\upsilon \in L^{\theta \left(x\right)}\left({\mathbb{R}}^N\right)$. Then, we have the following:

(i) $\chi ={\left|\upsilon \right|}_{\theta \left(x\right)}$ if and only if $\varrho \left({\displaystyle \frac{\upsilon }{\chi }}\right)=1$;

(ii) ${\left|\upsilon \right|}_{\theta \left(x\right)}>1\Rightarrow {\left|\upsilon \right|}_{\theta \left(x\right)}^{{\theta }^{-}}\le \varrho \left(\upsilon \right)\le {\left|\upsilon \right|}_{\theta \left(x\right)}^{{\theta }^{+}}$;

(iii) ${\left|\upsilon \right|}_{\theta \left(x\right)}<1\Rightarrow {\left|\upsilon \right|}_{\theta \left(x\right)}^{{\theta }^{+}}\le \varrho \left(\upsilon \right)\le {\left|\upsilon \right|}_{\theta \left(x\right)}^{{\theta }^{-}};$

(iv) ${\left|\upsilon \right|}_{\theta \left(x\right)}<1(=1;>1)\iff \varrho \left(\upsilon \right)<1(=1;>1)$;

(v) ${\left|\upsilon \right|}_{\theta \left(x\right)}^{{\theta }^{-}}-1\le \varrho \left(\upsilon \right)\le {\left|\upsilon \right|}_{\theta \left(x\right)}^{{\theta }^{+}}+1.$

Lemma 2

([38]). The space $(L^{{\theta }^{\prime }\left(x\right)}\left({\mathbb{R}}^N\right),|\upsilon {|}_{{\theta }^{\prime }\left(x\right)})$ is conjugate space of space $(L^{\theta \left(x\right)}\left({\mathbb{R}}^N\right),|\upsilon {|}_{\theta \left(x\right)})$, where ${\theta }^{\prime }\left(x\right)$ is the conjugate function of $\theta \left(x\right)$. Let

$${\displaystyle \frac{1}{{\theta }^{\prime }\left(x\right)}}+{\displaystyle \frac{1}{\theta \left(x\right)}}=1,x\in {\mathbb{R}}^N.$$

Then, Hölder’s inequality holds, that is,

$$\left|{\int }_{{\mathbb{R}}^N}\upsilon udx\right|\le \left({\displaystyle \frac{1}{{\left({\theta }^{\prime }\right)}^{-}}}+{\displaystyle \frac{1}{{\theta }^{-}}}\right){\left|\upsilon \right|}_{\theta \left(x\right)}{\left|u\right|}_{{\theta }^{\prime }\left(x\right)}\le {2\left|\upsilon \right|}_{\theta \left(x\right)}{\left|u\right|}_{{\theta }^{\prime }\left(x\right)},$$

for any $\upsilon \in L^{\theta \left(x\right)}\left({\mathbb{R}}^N\right),u\in L^{{\theta }^{\prime }\left(x\right)}\left({\mathbb{R}}^N\right)$.

Remark 1.

Assume that the function $q_1\left(x\right),q_2\left(x\right)\in C_{+}\left({\mathbb{R}}^N\right)$ and $q_1\left(x\right)<q_2\left(x\right)$ for any $x\in {\mathbb{R}}^N$. Then, there exists a continuous embedding $L^{q_2\left(x\right)}\left({\mathbb{R}}^N\right)\hookrightarrow L^{q_1\left(x\right)}\left({\mathbb{R}}^N\right)$. Furthermore, the embedding is compact.

The variable-exponent Sobolev space is given by

$$W=W^{1,p\left(x\right)}\left({\mathbb{R}}^N\right)=\left\{\upsilon \in L^{p\left(x\right)}\left({\mathbb{R}}^N\right),|\nabla \upsilon |\in L^{p\left(x\right)}\left({\mathbb{R}}^N\right)\right\},$$

endowed with the norm

$${\left|\upsilon \right|}_W:={|\nabla \upsilon |}_{p\left(x\right)}+{\left|\upsilon \right|}_{p\left(x\right)}.$$

For the potential term V, we define the linear subspace as

$$E=\left\{\upsilon :\upsilon \in W,{\varrho }_E\left(\upsilon \right)<+\infty \right\}$$

with respect to the norm

$$\parallel \upsilon \parallel :=\inf \left\{\chi >0:{\varrho }_E\left({\displaystyle \frac{\upsilon }{\chi }}\right)\le 1\right\},$$

where

$${\varrho }_E\left(\upsilon \right)={\int }_{{\mathbb{R}}^N}{|\nabla \upsilon \left(x\right)|}^{p\left(x\right)}dx+{\int }_{{\mathbb{R}}^N}\lambda V\left(x\right){\left|\upsilon \left(x\right)\right|}^{p\left(x\right)}dx.$$

Both $(W,|·{|}_W)$ and $(E,\parallel ·\parallel )$ are separable and reflexive Banach spaces (see [38]). From Proposition 2.1 in [17], we have the following connection between the modular ${\varrho }_E(·)$ and norm $\parallel ·\parallel$.

Lemma 3.

Suppose that ${\upsilon }_n,\upsilon \in E$. Then, we derive the following:

(i) $\chi =\parallel \upsilon \parallel$ if and only if ${\varrho }_E\left({\displaystyle \frac{\upsilon }{\chi }}\right)=1$;

(ii) $\parallel \upsilon \parallel >1\Rightarrow {\parallel \upsilon \parallel }^{p^{-}}\le {\varrho }_E\left(\upsilon \right)\le {\parallel \upsilon \parallel }^{p^{+}}$;

(iii) $\parallel \upsilon \parallel <1\Rightarrow {\parallel \upsilon \parallel }^{p^{+}}\le {\varrho }_E\left(\upsilon \right)\le {\parallel \upsilon \parallel }^{p^{-}}$;

(iv) $\parallel \upsilon \parallel <1(=1;>1)\iff {\varrho }_E\left(\upsilon \right)<1(=1;>1)$;

(v) ${lim}_{n\to \infty }\parallel {\upsilon }_n-\upsilon \parallel =0\iff {lim}_{n\to \infty }{\varrho }_E({\upsilon }_n-\upsilon )=0.$

By analogy to the proof of Lemma 2.6 in [17], we present the following embedding results.

Theorem 3.

Let $\theta \in C_{+}\left({\mathbb{R}}^N\right)$ and $p\in C_{+}\left({\mathbb{R}}^N\right)$ with $p\left(x\right)<N$ for each $x\in {\mathbb{R}}^N$. Assuming that condition (V1) holds and that $p\left(x\right)\le \theta \left(x\right)\le p^{\ast }\left(x\right)$ for any $x\in {\mathbb{R}}^N$, the embedding $E\hookrightarrow L^{\theta \left(x\right)}\left({\mathbb{R}}^N\right)$ is continuous. That is, there are constants $C_{\theta }$ such that

$${\left|\upsilon \right|}_{\theta \left(x\right)}\le C_{\theta }\parallel \upsilon \parallel ,\upsilon \in E.$$

Moreover, if condition (V2) holds, E is continuously embedded in $L^{\theta \left(x\right)}\left({\mathbb{R}}^N\right)$ for each $\theta \left(x\right)\in [p\left(x\right),p^{\ast }\left(x\right)]$ and compactly embedded in $L^{\theta \left(x\right)}\left({\mathbb{R}}^N\right)$ for each $\theta \left(x\right)\in [p\left(x\right),p^{\ast }\left(x\right))$. Consequently, we define

$$S:=\underset{\upsilon \in E\setminus \left\{0\right\}}{inf}{\displaystyle \frac{\parallel \upsilon \parallel }{{\left|\upsilon \right|}_{L^{p^{\ast }\left(x\right)}\left({\mathbb{R}}^N\right)}}}>0.$$

(1)

Next, we give some lemmas and corresponding variational forms related to the problems $\left(H_{\lambda }\right)$.

Definition 1.

A function $\upsilon \in E$ is said to be a weak solution of problem $\left(H_{\lambda }\right)$ if

$$\begin{array}{c}\hfill M\left(\Phi \left(\upsilon \right)\right)⟨{\Phi }^{\prime }\left(\upsilon \right),\psi ⟩={\int }_{{\mathbb{R}}^N}h(x,\upsilon \left(x\right))\psi \left(x\right)dx,\end{array}$$

(2)

for each $\psi \in E$, where

$$\begin{array}{c}\hfill ⟨{\Phi }^{\prime }\left(\upsilon \right),\psi ⟩={\int }_{{\mathbb{R}}^N}{|\nabla \upsilon |}^{p\left(x\right)-2}\nabla \upsilon \nabla \psi dx+{\int }_{{\mathbb{R}}^N}\lambda V\left(x\right){\left|\upsilon \right|}^{p\left(x\right)-2}\upsilon \psi dx.\end{array}$$

The functional $J:E\to \mathbb{R}$ associated with equation $\left(H_{\lambda }\right)$ is defined as

$$\begin{array}{c}\hfill J\left(\upsilon \right)=\widehat{M}\left(\Phi \left(\upsilon \right)\right)-{\int }_{{\mathbb{R}}^N}H(x,\upsilon \left(x\right))dx\end{array}$$

(3)

for each $\upsilon \in E$. Under our assumptions, the functional $J:E\to \mathbb{R}$ is of $C^1$ (see [14]). Then,

$$\begin{array}{c}\hfill ⟨J^{\prime }\left(\upsilon \right),\psi ⟩=M\left(\Phi \left(\upsilon \right)\right)⟨{\Phi }^{\prime }\left(\upsilon \right),\psi ⟩-{\int }_{{\mathbb{R}}^N}h(x,\upsilon \left(x\right))\psi \left(x\right)dx,\upsilon ,\psi \in E.\end{array}$$

(4)

Lemma 4

([39]). Let E be a real Banach space and $J\in C^1(E,R)$ with $J\left(0\right)=0$. Suppose that J satisfies the Palais–Smale $\left(PS\right)$ condition and that the following statements are true:

(i) 

there are constants $\beta ,\delta >0$ such that ${J|}_{\partial B_{\beta }}\ge \delta$;

(ii) 

there is a $\varphi \in E$ with $\parallel \varphi \parallel >\beta$ such that $J\left(\varphi \right)<0$.

Then, J possesses a critical value $c>\delta$, where

$$c=\underset{\alpha \in \Lambda }{inf}\underset{0\le t\le 1}{max}J\left(\alpha \left(t\right)\right),$$

and

$$\Lambda =\{\alpha \in C([0,1],E):\alpha (0)=0,\alpha (1)=\varphi \}.$$

Lemma 5.

Assume that $\Omega \subset {\mathbb{R}}^N$ is an open set, and for some $d>0$, $1\le k\left(x\right)<l\left(x\right)<\infty$, the function $h\in C(\Omega \times {\mathbb{R}}^N,\mathbb{R})$ satisfies the inequality $\left|h(x,\upsilon )\right|\le d\left(\right|\upsilon {|}^{k\left(x\right)}+\left|\upsilon {|}^{l\left(x\right)}\right)$. Let $l\left(x\right)\le p\left(x\right)<\infty$, $k\left(x\right)\le q\left(x\right)<\infty$, $q\left(x\right)>1$. Suppose $\left\{{\upsilon }_n\right\}$ is a bounded sequence in $L^{p\left(x\right)}(\Omega )\cap L^{q\left(x\right)}(\Omega )$, such that ${\upsilon }_n\to \upsilon$ a.e. in Ω and in $L^{p\left(x\right)}(\Omega \cap B_R)\cap L^{q\left(x\right)}(\Omega \cap B_R)$ for each $R>0$. Then, there exists a sequence $\left\{{\omega }_n\right\}$ such that

$${\omega }_n\to \upsilon \mathrm{in}{L}^{p\left(x\right)}(\Omega )\cap L^{q\left(x\right)}(\Omega )$$

and

$$h(x,{\upsilon }_n)-h(x,{\upsilon }_n-{\omega }_n)-h(x,\upsilon )\to 0\mathrm{in}{L}^{{\displaystyle \frac{p\left(x\right)}{l\left(x\right)}}}(\Omega )+L^{{\displaystyle \frac{q\left(x\right)}{k\left(x\right)}}}(\Omega ),$$

where $B_R=\{x\in {\mathbb{R}}^N:\left|x\right|\le R\}$, the space $L^{p\left(x\right)}(\Omega )\cap L^{q\left(x\right)}(\Omega )$ has the norm ${\left|\upsilon \right|}_{p\left(x\right)\wedge q\left(x\right)}:={\left|\upsilon \right|}_{p\left(x\right)}+{\left|\upsilon \right|}_{q\left(x\right)}$, and the space $L^{p\left(x\right)}(\Omega )+L^{q\left(x\right)}(\Omega )$ has the norm ${\left|\upsilon \right|}_{p\left(x\right)\vee q\left(x\right)}:=inf\left\{\right|{\upsilon }_1{|}_{p\left(x\right)}+|{\upsilon }_2{|}_{q\left(x\right)}:{\upsilon }_1\in L^{p\left(x\right)}(\Omega ),{\upsilon }_2\in L^{q\left(x\right)}(\Omega ),\upsilon ={\upsilon }_1+{\upsilon }_2\}$.

Proof. 

Let $\phi \in C(\mathbb{R},[0,1\left]\right)$ be a function that satisfies $\phi \left(t\right)=0$ for $\left|t\right|\ge 2$ and $\phi \left(t\right)=1$ for $\left|t\right|\le 1$. Set

$$h_1(x,\upsilon ):=\phi \left(\right|\upsilon \left|\right)h(x,\upsilon ),h_2(x,\upsilon ):=(1-\phi (\left|\upsilon \right|\left)\right)h(x,\upsilon ).$$

Hence, following from ([39], Theorem A.4), we obtain

$$h_1(x,\upsilon )\le d_1{\left|\upsilon \right|}^{k\left(x\right)},h_2(x,\upsilon )\le d_2{\left|\upsilon \right|}^{l\left(x\right)}.$$

Now we prove that $h_1(x,{\upsilon }_n)-h_1(x,{\upsilon }_n-{\omega }_n)-h_1(x,\upsilon )\to 0$ in $L^{{\displaystyle \frac{q\left(x\right)}{k\left(x\right)}}}(\Omega )$. Note that the sequence $\left\{{\upsilon }_n\right\}$ is a bounded sequence in $L^{q\left(x\right)}(\Omega )$, and ${\upsilon }_n\to \upsilon$ a.e. in $\Omega$ as well as in $L^{q\left(x\right)}(\Omega \cap B_R)$. Consequently, for every $j\ge 1$ and almost all n, we have

$${\int }_{\Omega \cap B_j}(|{\upsilon }_n{|}^{q\left(x\right)}-{\left|\upsilon \right|}^{q\left(x\right)})dx\le {\displaystyle \frac{1}{j}}.$$

(5)

For each $\epsilon$, we choose an appropriate $R>0$ according to Lemma 5 such that

$${\int }_{\Omega \setminus B_R}{\left|\upsilon \right|}^{q\left(x\right)}dx\le \epsilon .$$

(6)

Thus, there is a subsequence $\left\{{\upsilon }_{n_j}\right\}$ of $\left\{{\upsilon }_n\right\}$ and a sequence $\left\{R_{n_j}\right\}$ with $R_{n_j}\to \infty$ such that

$$\begin{array}{cc}\hfill {\int }_{\Omega \cap B_{R_{n_j}}\setminus B_R}{\left|{\upsilon }_{n_j}\right|}^{q\left(x\right)}dx=& {\int }_{\Omega \cap B_{R_{n_j}}}(|{\upsilon }_{n_j}{|}^{q\left(x\right)}-{\left|\upsilon \right|}^{q\left(x\right)})dx+{\int }_{\Omega \cap B_{R_{n_j}}\setminus B_R}{\left|\upsilon \right|}^{q\left(x\right)}dx\hfill \\ \hfill & +{\int }_{\Omega \cap B_R}{\left(\right|\upsilon |}^{q\left(x\right)}-|{\upsilon }_{n_j}{|}^{q\left(x\right)})dx\hfill \\ \hfill \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\to & 0,\hfill \end{array}$$

(7)

as $j\to \infty$ whenever $R_{n_j}=j>R$.

Assume that ${\omega }_{n_j}\left(x\right)=\vartheta \left(2\right|x|/R_{n_j})\upsilon \left(x\right)$, and consider a smooth function $\vartheta \in C^{\infty }(\mathbb{R},[0,1])$ such that $\vartheta \left(t\right)=0$ whenever $\left|t\right|\ge 2$ and $\vartheta \left(t\right)=1$ whenever $\left|t\right|\le 1$. Obviously, ${\omega }_{n_j}\left(x\right)\to \upsilon$ in $L^{q\left(x\right)}(\Omega )$. Since the Nemytskii operator is continuous, we have

$$h_1(x,{\upsilon }_{n_j})-h_1(x,{\upsilon }_{n_j}-{\omega }_{n_j})-h_1(x,\upsilon )\to 0\mathrm{in}{L}^{{\displaystyle \frac{q\left(x\right)}{k\left(x\right)}}}(\Omega \cap B_R),$$

(8)

and

$$\begin{array}{cc}\hfill & {\left|h_1(x,{\upsilon }_{n_j})-h_1(x,{\upsilon }_{n_j}-{\omega }_{n_j})-h_1(x,\upsilon )\right|}_{L^{{\displaystyle \frac{q\left(x\right)}{k\left(x\right)}}}(\Omega \setminus B_R)}\hfill \\ \hfill \le & {\left|h_1(x,{\upsilon }_{n_j})-h_1(x,{\upsilon }_{n_j}-{\omega }_{n_j})-h_1(x,{\omega }_{n_j})\right|}_{L^{{\displaystyle \frac{q\left(x\right)}{k\left(x\right)}}}(\Omega \setminus B_R)}\hfill \\ \hfill & +{\left|h_1(x,{\omega }_{n_j})-h_1(x,\upsilon )\right|}_{L^{{\displaystyle \frac{q\left(x\right)}{k\left(x\right)}}}(\Omega \setminus B_R)}.\hfill \end{array}$$

(9)

By ${\omega }_{n_j}\left(x\right)\to \upsilon$ in $L^{q\left(x\right)}(\Omega )$ and the continuity of the Nemytskii operator, we obtain ${\left|h_1(x,{\omega }_{n_j})-h_1(x,\upsilon )\right|}_{L^{{\displaystyle \frac{q\left(x\right)}{k\left(x\right)}}}(\Omega \setminus B_R)}\to 0$. Since $|{\omega }_{n_j}|\le |\upsilon |$, it follows from (6) and (7) that

$$\begin{array}{cc}\hfill & {\left|h_1(x,{\upsilon }_{n_j})-h_1(x,{\upsilon }_{n_j}-{\omega }_{n_j})-h_1(x,{\omega }_{n_j})\right|}^{{\displaystyle \frac{q\left(x\right)}{k\left(x\right)}}}\hfill \\ \hfill \le & d_1^{{\displaystyle \frac{q^{+}}{k^{-}}}}{\left(|{\upsilon }_{n_j}{|}^{k\left(x\right)}+|{\upsilon }_{n_j}-{\omega }_{n_j}{|}^{k\left(x\right)}+{\left|{\omega }_{n_j}\right|}^{k\left(x\right)}\right)}^{{\displaystyle \frac{q\left(x\right)}{k\left(x\right)}}}\hfill \\ \hfill \le & d_3\left(|{\upsilon }_{n_j}{|}^{q\left(x\right)}+{\left|\upsilon \right|}^{q\left(x\right)}\right)\to 0\hfill \end{array}$$

(10)

for $x\in \Omega \setminus B_{R_{n_j}}$. By (8)–(10), we obtain

$$h_1(x,{\upsilon }_n)-h_1(x,{\upsilon }_n-{\omega }_n)-h_1(x,\upsilon )\to 0\mathrm{in}{L}^{{\displaystyle \frac{q\left(x\right)}{k\left(x\right)}}}(\Omega ).$$

By analogy to the proof for $h_1$, by choosing the same subsequence $\left\{{\upsilon }_{n_j}\right\}$ for both $h_1$ and $h_2$, we deduce that

$$h_2(x,{\upsilon }_n)-h_2(x,{\upsilon }_n-{\omega }_n)-h_2(x,\upsilon )\to 0\mathrm{in}{L}^{{\displaystyle \frac{p\left(x\right)}{l\left(x\right)}}}(\Omega ).$$

Since

$$\begin{array}{cc}\hfill & {\left|h(x,{\upsilon }_n)-h(x,{\upsilon }_n-{\omega }_n)-h(x,\upsilon )\right|}_{L^{{\displaystyle \frac{q\left(x\right)}{k\left(x\right)}}}(\Omega )\vee L^{{\displaystyle \frac{p\left(x\right)}{l\left(x\right)}}}(\Omega )}\hfill \\ \hfill \le & {\left|h_1(x,{\upsilon }_n)-h_1(x,{\upsilon }_n-{\omega }_n)-h_1(x,{\omega }_n)\right|}_{L^{{\displaystyle \frac{q\left(x\right)}{k\left(x\right)}}}(\Omega )}\hfill \\ \hfill & +{\left|h_2(x,{\upsilon }_n)-h_2(x,{\upsilon }_n-{\omega }_n)-h_2(x,{\omega }_n)\right|}_{L^{{\displaystyle \frac{p\left(x\right)}{l\left(x\right)}}}(\Omega )},\hfill \end{array}$$

it follows that

$$h(x,{\upsilon }_n)-h(x,{\upsilon }_n-{\omega }_n)-h(x,\upsilon )\to 0\mathrm{in}{L}^{{\displaystyle \frac{q\left(x\right)}{k\left(x\right)}}}(\Omega )+L^{{\displaystyle \frac{p\left(x\right)}{l\left(x\right)}}}(\Omega ).$$

This completes the proof. □

3. Proof of Theorem 1

To complete the proof of Theorem 1, we need the following results.

Definition 2.

Let E be a Banach space and $J\in C^1(E,\mathbb{R})$. If any Palais–Smale (PS) sequence ${\left\{{\upsilon }_n\right\}}_{n\in \mathbb{N}}\subset E$, namely,

$$J\left({\upsilon }_n\right)\to c,J^{\prime }\left({\upsilon }_n\right)\to 0,\mathrm{as}n\to \infty ,$$

(11)

has a convergent subsequence in E, we say that J satisfies the $\left(PS\right)$ condition at the level $c\in \mathbb{R}$ (${\left(PS\right)}_c$ condition).

Lemma 6.

If the conditions (M1)–(M2), (V1), and (H1)–(H4) are satisfied, then J fulfills the ${\left(PS\right)}_c$ condition for sufficiently large $\lambda >0$.

Proof. 

Let ${\left\{{\upsilon }_n\right\}}_{n\in \mathbb{N}}\subset E$ be a ${\left(PS\right)}_c$ sequence of J. Then

$$|J\left({\upsilon }_n\right)|\le c,J^{\prime }\left({\upsilon }_n\right)\to 0\mathrm{as}n\to \infty$$

for some constant $c>0$, which implies that

$$\begin{array}{c}\hfill -c\le \widehat{M}\left(\Phi \left({\upsilon }_n\right)\right)-{\int }_{{\mathbb{R}}^N}H(x,{\upsilon }_n)dx\le c.\end{array}$$

(12)

(i) First, we prove, by contradiction, that ${\left\{{\upsilon }_n\right\}}_{n\in \mathbb{N}}$ is bounded in E. Assume that

$$\parallel {\upsilon }_n\parallel \to \infty ,\mathrm{as}n\to \infty .$$

(13)

Let ${\kappa }_n={\displaystyle \frac{{\upsilon }_n}{\parallel {\upsilon }_n\parallel }}$. Then, ${\left\{{\kappa }_n\right\}}_{n\in \mathbb{N}}\subset E$ and $\parallel {\kappa }_n\parallel =1$. By Theorem 3, there exists a subsequence ${\left\{{\kappa }_n\right\}}_{n\in \mathbb{N}}$ and $\kappa \in E$ such that

$${\kappa }_n\rightharpoonup \kappa \mathrm{weakly}\mathrm{in}E,{\kappa }_n\to \kappa \mathrm{strongly}\mathrm{in}{L}_{loc}^{\theta \left(x\right)}\left({\mathbb{R}}^N\right),{\kappa }_n\to \kappa \mathrm{a}.\mathrm{e}.\mathrm{in}{\mathbb{R}}^N,$$

(14)

for $\theta \in C_{+}\left({\mathbb{R}}^N\right)$, $\theta \left(x\right)\in [p\left(x\right),p^{\ast }\left(x\right))$.

Let ${\Omega }_0:=\{x\in {\mathbb{R}}^N:\left|\kappa \left(x\right)\right|>0\}$. Thus, we have $|{\upsilon }_n\left(x\right)|\to +\infty$ for all $x\in {\Omega }_0$. Therefore, by hypothesis (H3), for any $x\in {\Omega }_0$ and sufficiently large n, we obtain

$$\underset{n\to \infty }{lim\; sup}{\displaystyle \frac{H(x,{\upsilon }_n)}{\parallel {\upsilon }_n{\parallel }^{\zeta p^{+}}}}=\underset{n\to \infty }{lim\; sup}{\displaystyle \frac{H(x,{\upsilon }_n){\left|{\kappa }_n\right|}^{\zeta p^{+}}}{|{\upsilon }_n{|}^{\zeta p^{+}}}}=\infty .$$

By Fatou’s lemma, we obtain

$$\underset{n\to \infty }{lim\; sup}{\int }_{{\mathbb{R}}^N}{\displaystyle \frac{H(x,{\upsilon }_n)}{\parallel {\upsilon }_n{\parallel }^{\zeta p^{+}}}}dx=\underset{n\to \infty }{lim\; sup}{\int }_{{\mathbb{R}}^N}{\displaystyle \frac{H(x,{\upsilon }_n){\left|{\kappa }_n\right|}^{\zeta p^{+}}}{|{\upsilon }_n{|}^{\zeta p^{+}}}}dx=\infty .$$

(15)

From (12), we derive

$$\begin{array}{cc}\hfill {\int }_{{\mathbb{R}}^N}{\displaystyle \frac{H(x,{\upsilon }_n)}{\parallel {\upsilon }_n{\parallel }^{\zeta p^{+}}}}dx\le & {\displaystyle \frac{1}{\parallel {\upsilon }_n{\parallel }^{\zeta p^{+}}}}\widehat{M}\left(\Phi \left({\upsilon }_n\right)\right)+{\displaystyle \frac{c}{\parallel {\upsilon }_n{\parallel }^{\zeta p^{+}}}}.\hfill \end{array}$$

It follows from (M2) that

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim\; sup}{\int }_{{\mathbb{R}}^N}{\displaystyle \frac{H(x,{\upsilon }_n)}{\parallel {\upsilon }_n{\parallel }^{\zeta p^{+}}}}dx\le {\displaystyle \frac{\widehat{M}\left(1\right)}{{\left(p^{-}\right)}^{\zeta }}},\end{array}$$

(16)

which contradicts (15). Therefore, $\kappa \left(x\right)=0$ a.e. in ${\mathbb{R}}^N$. Thus, by (V1) and Lemma 1, we have

$$\begin{array}{cc}\hfill {\int }_{{\mathbb{R}}^N}{\left|{\kappa }_n\right|}^{p\left(x\right)}dx=& {\int }_{V\left(x\right)\ge a}|{\kappa }_n{|}^{p\left(x\right)}dx+{\int }_{V\left(x\right)<a}{\left|{\kappa }_n\right|}^{p\left(x\right)}dx\hfill \\ \hfill \le & {\displaystyle \frac{1}{\lambda a}}{\varrho }_E\left({\kappa }_n\right)+o\left(1\right)\to {\displaystyle \frac{1}{\lambda a}}\hfill \end{array}$$

(17)

as $n\to \infty$. According to (H4), we can obtain

$$\zeta {\left(p^{+}\right)}^{2}H(x,\upsilon )-h(x,\upsilon )\upsilon \le \xi {\left|\upsilon \right|}^{p\left(x\right)},\mathrm{for}\mathrm{any}(x,\upsilon )\in {\mathbb{R}}^N\times \mathbb{R}.$$

Consequently, we obtain

$$\begin{array}{cc}\hfill & o_n\left(1\right)={\displaystyle \frac{1}{\parallel {\upsilon }_n{\parallel }^{p\left(x\right)}}}\left[\zeta {\left(p^{+}\right)}^{2}J\left({\upsilon }_n\right)-⟨J^{\prime }\left({\upsilon }_n\right),{\upsilon }_n⟩\right]\hfill \\ \hfill =& {\displaystyle \frac{1}{\parallel {\upsilon }_n{\parallel }^{p\left(x\right)}}}\left[\zeta {\left(p^{+}\right)}^2\widehat{M}\left(\Phi \left({\upsilon }_n\right)\right)-M\left(\Phi \left({\upsilon }_n\right)\right)⟨{\Phi }^{\prime }\left({\upsilon }_n\right),{\upsilon }_n⟩\right]\hfill \\ \hfill & +{\displaystyle \frac{1}{\parallel {\upsilon }_n{\parallel }^{p\left(x\right)}}}{\int }_{{\mathbb{R}}^N}\left(h(x,{\upsilon }_n){\upsilon }_n-\zeta {\left(p^{+}\right)}^{2}H(x,{\upsilon }_n)\right)dx\hfill \\ \hfill \ge & {\displaystyle \frac{1}{\parallel {\upsilon }_n{\parallel }^{p\left(x\right)}}}\left[{\left(p^{+}\right)}^{2}M\left(\Phi \left({\upsilon }_n\right)\right)\Phi \left({\upsilon }_n\right)-M\left(\Phi \left({\upsilon }_n\right)\right){\varrho }_E\left({\upsilon }_n\right)-\xi {\int }_{{\mathbb{R}}^N}{\left|\upsilon \right|}^{p\left(x\right)}dx\right]\hfill \\ \hfill \ge & {\displaystyle \frac{1}{\parallel {\upsilon }_n{\parallel }^{p\left(x\right)}}}\left[m(p^{+}-1){\varrho }_E\left({\upsilon }_n\right)-\xi {\int }_{{\mathbb{R}}^N}{\left|\upsilon \right|}^{p\left(x\right)}dx\right]\hfill \\ \hfill \ge & m(p^{+}-1)-{\displaystyle \frac{\xi }{\lambda a}},\hfill \end{array}$$

(18)

for $\lambda$ large enough that the term $m(p^{+}-1)-{\displaystyle \frac{\xi }{\lambda a}}>0$, which contradicts (18). Hence, $\left\{{\upsilon }_n\right\}$ is bounded in E. □

(ii) Next, we prove that there exists $\upsilon \in E$ such that ${\upsilon }_n\to \upsilon$ in E. Since $\left\{{\upsilon }_n\right\}$ is bounded in E, we can extract a subsequence, denoted by $\left\{{\upsilon }_n\right\}$ again, that satisfies

$${\upsilon }_n\rightharpoonup \upsilon \mathrm{in}E,{\upsilon }_n\to \upsilon \mathrm{in}{L}_{loc}^{\theta \left(x\right)}\left({\mathbb{R}}^N\right)\mathrm{for}p\left(x\right)\le \theta \left(x\right)<p^{\ast }\left(x\right),{\upsilon }_n\to \upsilon \mathrm{a}.\mathrm{e}.\mathrm{in}{\mathbb{R}}^N.$$

(19)

Taking ${\omega }_n\left(x\right)=\vartheta \left(2\right|x|/R_n)\upsilon \left(x\right)$, it follows from Lemma 5 that ${\omega }_n\to \upsilon$ in $L^{p\left(x\right)}\left({\mathbb{R}}^N\right)\cap L^{q\left(x\right)}\left({\mathbb{R}}^N\right)$, where $R_n$ is a constant sequence with $R_n\to \infty$ as $n\to \infty$. Moreover,

$$\begin{array}{cc}\hfill {\varrho }_E({\omega }_n-\upsilon )=& {\int }_{{\mathbb{R}}^N}|\nabla (\vartheta \left(2\right|x|/R_n{)\upsilon \left(x\right)-\upsilon \left(x\right))|}^{p\left(x\right)}dx\hfill \\ \hfill & +{\int }_{{\mathbb{R}}^N}\left|\vartheta \right(2\left|x\right|/R_n{)-1|}^{p\left(x\right)}\lambda V\left(x\right){\left|\upsilon \left(x\right)\right|}^{p\left(x\right)}dx\hfill \\ \hfill \le & 2^{p^{+}-1}{\int }_{{\mathbb{R}}^N}\left|\right(\vartheta \left(2\right|x|/R_n-{1\left)\right|}^{p\left(x\right)}{|\nabla \upsilon |}^{p\left(x\right)}dx\hfill \\ \hfill & +2^{p^{+}-1}{\int }_{{\mathbb{R}}^N}(2|x|/R_n{)}^{p\left(x\right)}|{\vartheta }^{\prime }\left(2\right|x|/R_n{\left)\right|}^{p\left(x\right)}{\left|\upsilon \right|}^{p\left(x\right)}dx\hfill \\ \hfill & +{\int }_{{\mathbb{R}}^N}\left|\vartheta \right(2\left|x\right|/R_n{)-1|}^{p\left(x\right)}\lambda V\left(x\right){\left|\upsilon \left(x\right)\right|}^{p\left(x\right)}dx.\hfill \end{array}$$

Since $\upsilon \in E$, there is a $\varsigma =\varsigma \left(ϵ\right)$ such that

$${\int }_{{\mathbb{R}}^N\setminus B_{\varsigma }}{|\nabla \upsilon |}^{p\left(x\right)}dx\le ϵ\mathrm{and}{\int }_{{\mathbb{R}}^N\setminus B_{\varsigma }}\lambda V\left(x\right){\left|\upsilon \right|}^{p\left(x\right)}dx\le ϵ,$$

for each $ϵ>0$, which implies that

$$\begin{array}{cc}\hfill {\varrho }_E({\omega }_n-\upsilon )\le & 2^{p^{+}-1}{\int }_{B_{\varsigma }}\left|\right(\vartheta \left(2\right|x|/R_n-{1\left)\right|}^{p\left(x\right)}{|\nabla \upsilon |}^{p\left(x\right)}dx\hfill \\ \hfill & +2^{p^{+}-1}{\int }_{{\mathbb{R}}^N}(2|x|/R_n{)}^{p\left(x\right)}|{\vartheta }^{\prime }\left(2\right|x|/R_n{\left)\right|}^{p\left(x\right)}{\left|\upsilon \right|}^{p\left(x\right)}dx\hfill \\ \hfill & +{\int }_{B_{\varsigma }}\left|\vartheta \right(2\left|x\right|/R_n{)-1|}^{p\left(x\right)}\lambda V\left(x\right){\left|\upsilon \left(x\right)\right|}^{p\left(x\right)}dx+c_0ϵ.\hfill \end{array}$$

Thus, ${\omega }_n\to \upsilon$ in E, as $n\to \infty$. Applying Lemma 5 again, if we choose $k\left(x\right)=q\left(x\right)-1$, $l\left(x\right)=p\left(x\right)-1$, then

$$h(x,{\upsilon }_n)-h(x,{\upsilon }_n-{\omega }_n)-h(x,\upsilon )\to 0\mathrm{in}{L}^{{\displaystyle \frac{p\left(x\right)}{p\left(x\right)-1}}}\left({\mathbb{R}}^N\right)+L^{{\displaystyle \frac{q\left(x\right)}{q\left(x\right)-1}}}\left({\mathbb{R}}^N\right).$$

It follows that

$$\begin{array}{cc}\hfill & {\int }_{{\mathbb{R}}^N}|h(x,{\upsilon }_n)-h(x,\upsilon )-h(x,{\upsilon }_n-{\omega }_n)\left|\right|{\upsilon }_n-\upsilon |dx\hfill \\ \hfill \le & |h(x,{\upsilon }_n)-h(x,\upsilon )-h(x,{\upsilon }_n-{\omega }_n){|}_{p^{\prime }\left(x\right)\vee q^{\prime }\left(x\right)}{|{\upsilon }_n-\upsilon |}_{p\left(x\right)\wedge q\left(x\right)}\to 0\hfill \end{array}$$

(20)

and

$$\begin{array}{c}\hfill {\int }_{{\mathbb{R}}^N}\left|h\right(x,{\upsilon }_n-{\omega }_n\left|\right|({\omega }_n-\upsilon )|dx\le |h(x,{\upsilon }_n-{\omega }_n){|}_{p^{\prime }\left(x\right)\vee q^{\prime }\left(x\right)}{|{\omega }_n-\upsilon |}_{p\left(x\right)\wedge q\left(x\right)}\to 0\end{array}$$

as $n\to \infty$. Therefore,

$$\begin{array}{cc}\hfill & \left|{\int }_{{\mathbb{R}}^N}h(x,{\upsilon }_n-{\omega }_n)({\upsilon }_n-\upsilon )dx\right|\hfill \\ \hfill \le & {\int }_{{\mathbb{R}}^N}|h(x,{\upsilon }_n-{\omega }_n)\left|\right|{\upsilon }_n-{\omega }_n|dx+{\int }_{{\mathbb{R}}^N}|h(x,{\upsilon }_n-{\omega }_n)\left|\right|{\omega }_n-\upsilon |dx\hfill \\ \hfill \le & {\int }_{{\mathbb{R}}^N}|h(x,{\upsilon }_n-{\omega }_n)\left|\right|{\upsilon }_n-{\omega }_n|dx+o\left(1\right).\hfill \end{array}$$

(21)

Let ${\xi }_n={\upsilon }_n-{\omega }_n$. Then, by ${\xi }_n\rightharpoonup 0$ in E and (V1), we have

$${\int }_{{\mathbb{R}}^N}|{\xi }_n{|}^{p\left(x\right)}dx\le {\displaystyle \frac{1}{\lambda a}}{\varrho }_E\left({\xi }_n\right)+o\left(1\right)\le {\displaystyle \frac{1}{\lambda a}}{\parallel {\xi }_n\parallel }^{p^{-}}+o\left(1\right)$$

(22)

as $n\to \infty$. From (H1), (22), and Lemma 2, we obtain

$$\begin{array}{cc}\hfill {\int }_{{\mathbb{R}}^N}{\left|{\xi }_n\right|}^{q\left(x\right)}dx\le & {\left[{\int }_{{\mathbb{R}}^N}{\left|{\xi }_n\right|}^{p\left(x\right)}dx\right]}^{{\displaystyle \frac{1}{p\left(x\right)}}}{\left[{\int }_{{\mathbb{R}}^N}{\left|{\xi }_n\right|}^{{\displaystyle \frac{p\left(x\right)\left(q\right(x)-1)}{p\left(x\right)-1}}}dx\right]}^{{\displaystyle \frac{p\left(x\right)-1}{p\left(x\right)}}}\hfill \\ \hfill \le & {\displaystyle \frac{d_3}{{\left(\lambda a\right)}^{1/p^{+}}}}{\parallel {\xi }_n\parallel }^{q^{-}}+o\left(1\right).\hfill \end{array}$$

(23)

Consequently, by (H1) and (21)–(23), we have

$$\begin{array}{cc}\hfill & \left|{\int }_{{\mathbb{R}}^N}h(x,{\upsilon }_n-{\omega }_n)({\upsilon }_n-\upsilon )dx\right|\hfill \\ \hfill \le & d_1{\int }_{{\mathbb{R}}^N}|{\xi }_n{|}^{p\left(x\right)}dx+d_2{\int }_{{\mathbb{R}}^N}{\left|{\xi }_n\right|}^{q\left(x\right)}dx+o\left(1\right)\hfill \\ \hfill \le & {\displaystyle \frac{d_1}{\lambda a}}\parallel {\xi }_n{\parallel }^{p^{-}}+{\displaystyle \frac{d_3}{{\left(\lambda a\right)}^{1/p^{+}}}}{\parallel {\xi }_n\parallel }^{q^{-}}+o\left(1\right).\hfill \end{array}$$

(24)

Since $J^{\prime }\left({\upsilon }_n\right)\to 0$ and ${\upsilon }_n\rightharpoonup \upsilon$ in E, we conclude that

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}⟨J^{\prime }\left({\upsilon }_n\right)-J^{\prime }\left(\upsilon \right),{\upsilon }_n-\upsilon ⟩=0.\end{array}$$

(25)

Fix $(x,y)\in {\mathbb{R}}^N\times {\mathbb{R}}^N$. According to the Simon inequality [8], we can obtain

$$(|{\eta }_1{|}^{\tau -2}{\eta }_1-|{\eta }_2{|}^{\tau -2}{\eta }_2)({\eta }_1-{\eta }_2)>{\displaystyle \frac{1}{2^{\tau }}}{|{\eta }_1-{\eta }_2|}^{\tau },\tau \ge 2$$

(26)

and

$$\begin{array}{cc}\hfill & ⟨J^{\prime }\left({\upsilon }_n\right)-J^{\prime }\left(\upsilon \right),{\upsilon }_n-\upsilon ⟩\hfill \\ \hfill =& M\left(\Phi \left({\upsilon }_n\right)\right)⟨{\Phi }^{\prime }\left({\upsilon }_n\right),{\upsilon }_n-\upsilon ⟩-M\left(\Phi \left(\upsilon \right)\right)⟨{\Phi }^{\prime }\left(\upsilon \right),{\upsilon }_n-\upsilon ⟩-{\int }_{{\mathbb{R}}^N}(h(x,{\upsilon }_n)-h(x,\upsilon ))({\upsilon }_n-\upsilon )dx\hfill \\ \hfill \ge & {\displaystyle \frac{m}{2^{p^{-}}}}\left[{\int }_{{\mathbb{R}}^N}|\nabla {\upsilon }_n{\left(x\right)-\nabla \upsilon \left(x\right)|}^{p\left(x\right)}dx+{\int }_{{\mathbb{R}}^N}\lambda V\left(x\right){|{\upsilon }_n\left(x\right)-\upsilon \left(x\right)|}^{p\left(x\right)}dx\right]\hfill \\ \hfill & -{\int }_{{\mathbb{R}}^N}(h(x,{\upsilon }_n)-h(x,\upsilon ))({\upsilon }_n-\upsilon )dx.\hfill \end{array}$$

(27)

By (20), (24), (25), and (27), we have

$$\begin{array}{cc}\hfill 0\ge & \underset{n\to \infty }{lim\; inf}\left({\displaystyle \frac{m}{2^{p^{-}}}}{\varrho }_E({\upsilon }_n-\upsilon )-{\int }_{{\mathbb{R}}^N}(h(x,{\upsilon }_n)-h(x,\upsilon ))({\upsilon }_n-\upsilon )dx\right)\hfill \\ \hfill \ge & {\displaystyle \frac{m}{2^{p^{-}}}}\underset{n\to \infty }{lim}{\varrho }_E({\upsilon }_n-\upsilon )-\underset{n\to \infty }{lim\; sup}\left[{\displaystyle \frac{d_1}{\lambda a}}\parallel {\xi }_n{\parallel }^{p^{-}}+{\displaystyle \frac{d_3}{{\left(\lambda a\right)}^{1/p^{+}}}}{\parallel {\xi }_n\parallel }^{q^{-}}\right].\hfill \end{array}$$

(28)

This means that

$$\underset{n\to \infty }{lim}{\varrho }_E({\upsilon }_n-\upsilon )=0$$

for large $\lambda >0$. If not, since $\left\{{\xi }_n\right\}$ is bounded in E and $\lambda$ is large enough that the last term in (28) is positive, this leads to a contradiction. Thus, we conclude that ${\upsilon }_n\to \upsilon$ in E.

Proof of Theorem 1.

From (H2), for each $\epsilon >0$, there is an $\eta ={\eta }_{\epsilon }>0$ such that

$$\left|h(x,\upsilon )\right|\le p^{+}{\epsilon \left|\upsilon \right|}^{p^{+}-1},\mathrm{for}\mathrm{each}x\in {\mathbb{R}}^N\mathrm{and}\left|\upsilon \right|\le \eta .$$

(29)

By (H1), for all $x\in {\mathbb{R}}^N$ and $\left|\upsilon \right|\ge \eta$, we obtain

$$\left|h(x,\upsilon )\right|\le \left(d_2+d_1{\eta }^{{\displaystyle \frac{p\left(x\right)-1}{q\left(x\right)-1}}}\right){\left|\upsilon \right|}^{q\left(x\right)-1}.$$

(30)

Combining with (29) and (30), we can find a $C_{\epsilon }>0$ satisfying

$$\left|H(x,\upsilon )\right|\le {\epsilon \left|\upsilon \right|}^{p^{+}}+C_{\epsilon }{\left|\upsilon \right|}^{q\left(x\right)},\mathrm{for}\mathrm{each}(x,\upsilon )\in {\mathbb{R}}^N\times \mathbb{R}.$$

(31)

Using Theorem 3, (31), and Hölder’s inequality, we obtain

$$\begin{array}{cc}\hfill J\left(\upsilon \right)=& \widehat{M}\left(\Phi \left(\upsilon \right)\right)-{\int }_{{\mathbb{R}}^N}H(x,\upsilon \left(x\right))dx\hfill \\ \hfill \ge & {\displaystyle \frac{m}{\zeta p^{+}}}{\varrho }_E\left(\upsilon \right)-\epsilon {\int }_{{\mathbb{R}}^N}{\left|\upsilon \right|}^{p^{+}}dx-C_{\epsilon }{\int }_{{\mathbb{R}}^N}{\left|\upsilon \right|}^{q\left(x\right)}dx\hfill \\ \hfill \ge & {\displaystyle \frac{m}{\zeta p^{+}}}{\parallel \upsilon \parallel }^{p^{+}}-\epsilon C_{p^{+}}^{p^{+}}{\parallel \upsilon \parallel }^{p^{+}}-C_{\epsilon }{C}_{q^{-}}^{q^{-}}{\parallel \upsilon \parallel }^{q^{-}}.\hfill \end{array}$$

Taking $\epsilon ={\displaystyle \frac{m}{2\zeta p^{+}{C}_{p+}^{p+}}}$, we have

$$J\left(\upsilon \right)\ge {\displaystyle \frac{m}{2\zeta p^{+}}}{\parallel \upsilon \parallel }^{p^{+}}-C_{\epsilon }{C}_{q^{-}}^{q^{-}}{\parallel \upsilon \parallel }^{q^{-}}.$$

Since $q^{-}>p^{+}>1$, there exists a small $\beta >0$ such that $I\left(\upsilon \right)\ge \delta >0$ for $\parallel \upsilon \parallel =\beta$.

By (H1) and (H3), for any $\sigma >0$, there is a positive constant $C_{\sigma }$ that satisfies

$$\left|H(x,\upsilon )\right|\ge {\sigma \left|\upsilon \right|}^{\zeta p^{+}}-C_{\sigma }{\left|\upsilon \right|}^{p\left(x\right)}.$$

Then,

$$\begin{array}{cc}\hfill J\left(t\upsilon \right)=& \widehat{M}\left(\Phi \left(t\upsilon \right)\right)-{\int }_{{\mathbb{R}}^N}H(x,t\upsilon \left(x\right))dx\hfill \\ \hfill \le & \widehat{M}\left(1\right){(\Phi \left(t\upsilon \right))}^{\zeta }-\sigma {\int }_{{\mathbb{R}}^N}{\left|t\upsilon \right|}^{\zeta p^{+}}dx+C_{\sigma }{\int }_{{\mathbb{R}}^N}{\left|t\upsilon \right|}^{p\left(x\right)}dx\hfill \\ \hfill =& {\displaystyle \frac{\widehat{M}\left(1\right)}{{\left(p^{-}\right)}^{\zeta }}}{t}^{\zeta p^{+}}{\parallel \upsilon \parallel }^{\zeta p^{+}}-\sigma t^{\zeta p^{+}}{\int }_{{\mathbb{R}}^N}{\left|\upsilon \right|}^{\zeta p^{+}}dx+C_{\sigma }{t}^{p^{+}}{\int }_{{\mathbb{R}}^N}{\left|\upsilon \right|}^{p^{+}}dx\hfill \end{array}$$

(32)

as $t\to \infty$. If $\sigma$ is large enough that

$$\begin{array}{c}\hfill {\displaystyle \frac{\widehat{M}\left(1\right)}{{\left(p^{-}\right)}^{\zeta }}}{\parallel \upsilon \parallel }^{\zeta p^{+}}<\sigma {\int }_{{\mathbb{R}}^N}{\left|\upsilon \right|}^{\zeta p^{+}}dx,\end{array}$$

it follows from (32) and $p^{+}<\zeta p^{+}$ that

$$\begin{array}{c}\hfill J\left(t\upsilon \right)\to -\infty \end{array}$$

as $t\to \infty$. Therefore, there exists $\varphi \in E$ with $\parallel \varphi \parallel >\beta$ such that $J\left(\varphi \right)<0$. Additionally, $J\left(0\right)=0$, and by Lemma 6, J satisfies the ${\left(PS\right)}_c$ condition. This completes the proof. □

4. Proof of Theorem 2

In this section, we turn to the critical case. By analogy to (3) and (4), we define the functional $I:E\to \mathbb{R}$ associated with problems $\left(H_{\lambda }\right)$ with

$$\begin{array}{c}\hfill I\left(\upsilon \right)=\widehat{M}\left(\Phi \left(\upsilon \right)\right)-{\displaystyle \frac{\mu }{r\left(x\right)}}{\int }_{{\mathbb{R}}^N}{\left|\upsilon \right|}^{r\left(x\right)}dx-{\displaystyle \frac{1}{p^{\ast }\left(x\right)}}{\int }_{{\mathbb{R}}^N}{\left|\upsilon \right|}^{p^{\ast }\left(x\right)}dx\end{array}$$

(33)

and

$$\begin{array}{c}\hfill ⟨I^{\prime }\left(\upsilon \right),\psi ⟩=M\left(\Phi \left(\upsilon \right)\right)⟨{\Phi }^{\prime }\left(\upsilon \right),\psi ⟩-\mu {\int }_{{\mathbb{R}}^N}{\left|\upsilon \right|}^{r\left(x\right)-2}\upsilon \psi dx-{\int }_{{\mathbb{R}}^N}{\left|\upsilon \right|}^{p^{\ast }\left(x\right)-2}\upsilon \psi dx\end{array}$$

(34)

for each $\upsilon ,\psi \in E$.

Lemma 7.

Assume that $\left\{{\upsilon }_n\right\}\subset E$ is a ${\left(PS\right)}_c$ sequence for I. Then, there exists a subsequence (denoted by ${\left\{{\upsilon }_n\right\}}_{n\in \mathbb{N}}$ again) and $\upsilon \in E$ such that ${\upsilon }_n\rightharpoonup \upsilon$ in E. Furthermore, υ is a solution of $\left(H_{\lambda }\right)$, meaning that $⟨I^{\prime }\left(\upsilon \right),\upsilon ⟩=0$.

Proof. 

From (11), we can find a sequence $\left\{{\upsilon }_n\right\}$ and some $c\in \mathbb{R}$ satisfying

$$I\left({\upsilon }_n\right)\to c,I^{\prime }\left({\upsilon }_n\right)\to 0,n\to \infty .$$

(35)

Combining it with (33) and (34), we obtain

$$\begin{array}{cc}\hfill c+o\left(1\right)=& I\left({\upsilon }_n\right)-{\displaystyle \frac{1}{r^{-}}}⟨I^{\prime }\left({\upsilon }_n\right),{\upsilon }_n⟩\hfill \\ \hfill \ge & \widehat{M}\left(\Phi \left(\upsilon \right)\right)-{\displaystyle \frac{1}{r^{-}}}M\left(\Phi \left(\upsilon \right)\right)\left(\Phi \left(\upsilon \right)\right)+\left({\displaystyle \frac{1}{r^{-}}}-{\displaystyle \frac{1}{p^{\ast }\left(x\right)}}\right){\int }_{{\mathbb{R}}^N}{\left|\upsilon \right|}^{p^{\ast }\left(x\right)}dx\hfill \\ \hfill \ge & \left({\displaystyle \frac{1}{\zeta }}-{\displaystyle \frac{1}{r^{-}}}\right)M\left(\Phi \left(\upsilon \right)\right)\Phi \left(\upsilon \right)\hfill \\ \hfill \ge & \left({\displaystyle \frac{m}{\zeta p^{+}}}-{\displaystyle \frac{m}{r^{-}{p}^{+}}}\right)min\{\parallel {\upsilon }_n{\parallel }^{p^{+}},\parallel {\upsilon }_n{\parallel }^{p^{-}}\}.\hfill \end{array}$$

Therefore, we conclude that $\left\{{\upsilon }_n\right\}$ is bounded. Consequently, we can extract a subsequence in E (denoted again by ${\left\{{\upsilon }_n\right\}}_{n\in \mathbb{N}}$ again) and $\upsilon \in E$ that satisfies the following:

$${\upsilon }_n\rightharpoonup \upsilon \mathrm{weakly}\mathrm{in}E\mathrm{and}{L}^{p^{\ast }\left(x\right)}\left({\mathbb{R}}^N\right);$$

$${\upsilon }_n\to \upsilon \mathrm{strongly}\mathrm{in}{L}^{\theta \left(x\right)}\left({\mathbb{R}}^N\right)(p\left(x\right)\le \theta \left(x\right)<p^{\ast }\left(x\right));$$

$${\upsilon }_n\to \upsilon a.e.\mathrm{in}{\mathbb{R}}^N.$$

(36)

As $n\to \infty$, we have ${\int }_{{\mathbb{R}}^N}|{\upsilon }_n{|}^{r\left(x\right)}dx\to {\int }_{{\mathbb{R}}^N}{\left|\upsilon \right|}^{r\left(x\right)}dx$. Since $\left\{{\upsilon }_n\right\}$ converges weakly to $\upsilon$ in $L^{p^{\ast }\left(x\right)}\left({\mathbb{R}}^N\right)$, $|{\upsilon }_n{|}^{p^{\ast }\left(x\right)-2}{\upsilon }_n$ converges weakly to ${\left|\upsilon \right|}^{p^{\ast }\left(x\right)-2}\upsilon$ in $L^{p^{\ast }\left(x\right)/p^{\ast }\left(x\right)-1}\left({\mathbb{R}}^N\right)$. Thus,

$${\int }_{{\mathbb{R}}^N}|{\upsilon }_n{|}^{p^{\ast }\left(x\right)-2}{\upsilon }_n\psi dx\to {\int }_{{\mathbb{R}}^N}{\left|\upsilon \right|}^{p^{\ast }\left(x\right)-2}\upsilon \psi dx,\psi \in L^{p^{\ast }\left(x\right)}\left({\mathbb{R}}^N\right).$$

(37)

Note that $\left\{{\upsilon }_n\right\}$ is bounded and ${\upsilon }_n\rightharpoonup \upsilon$ in E; accordingly, we have

$$M\left(\Phi \left({\upsilon }_n\right)\right)⟨{\Phi }^{\prime }\left({\upsilon }_n\right),\psi ⟩\to M\left(\Phi \left(\upsilon \right)\right)⟨{\Phi }^{\prime }\left(\upsilon \right),\psi ⟩,\psi \in E.$$

Therefore, we deduce that

$$\begin{array}{cc}\hfill ⟨I^{\prime }\left({\upsilon }_n\right),\psi ⟩=& M\left(\Phi \left({\upsilon }_n\right)\right)⟨{\Phi }^{\prime }\left({\upsilon }_n\right),\psi ⟩-\mu {\int }_{{\mathbb{R}}^N}{\left|{\upsilon }_n\right|}^{r\left(x\right)-2}{\upsilon }_n\psi dx\hfill \\ \hfill & -{\int }_{{\mathbb{R}}^N}{\left|{\upsilon }_n\right|}^{p^{\ast }\left(x\right)-2}{\upsilon }_n\psi dx\to ⟨I^{\prime }\left(\upsilon \right),\psi ⟩.\hfill \end{array}$$

Thus, $⟨I^{\prime }\left(\upsilon \right),\psi ⟩=0$, $\upsilon$ is a solution of $\left(H_{\lambda }\right)$, and $⟨I^{\prime }\left(\upsilon \right),\upsilon ⟩=0$. □

Lemma 8.

Let $c\in \mathbb{R}$. Assume that $\left\{{\upsilon }_n\right\}\subset E$ is a ${\left(PS\right)}_c$ sequence for I and ${\upsilon }_n\rightharpoonup \upsilon$ with

$$c<\left({\displaystyle \frac{m}{\zeta p^{+}}}-{\displaystyle \frac{m}{{\left(p^{\ast }\right)}^{-}}}\right){\left(m{S}^{{\left(p^{\ast }\right)}^{-}}\right)}^{{\displaystyle \frac{p^{+}}{{\left(p^{\ast }\right)}^{-}-p^{+}}}}=c_{\ast }.$$

Then, ${\upsilon }_n\to \upsilon .$

Proof. 

Note that $\left\{{\upsilon }_n\right\}$ is a ${\left(PS\right)}_c$ sequence, and therefore, (35) holds. From Lemma 7, we can find a subsequence in E (denoted by ${\left\{{\upsilon }_n\right\}}_{n\in \mathbb{N}}$ again) and $\upsilon \in E$ satisfying ${\upsilon }_n\rightharpoonup \upsilon$ and $⟨I^{\prime }\left(\upsilon \right),\upsilon ⟩=0$.

Assume that ${\varpi }_n={\upsilon }_n-\upsilon$. Then, ${\varpi }_n\rightharpoonup 0$. Using the Brézis–Lieb-type lemma for variable exponents in [35], we obtain

$$\begin{array}{c}\hfill {\varrho }_E\left({\upsilon }_n\right)={\varrho }_E\left({\varpi }_n\right)+{\varrho }_E\left(\upsilon \right)\end{array}$$

and

$$\begin{array}{c}\hfill {\int }_{{\mathbb{R}}^N}|{\upsilon }_n{|}^{p^{\ast }\left(x\right)}dx={\int }_{{\mathbb{R}}^N}|{\varpi }_n{|}^{p^{\ast }\left(x\right)}dx+{\int }_{{\mathbb{R}}^N}{\left|\upsilon \right|}^{p^{\ast }\left(x\right)}dx.\end{array}$$

From $⟨I^{\prime }\left(\upsilon \right),\upsilon ⟩=0$ and ${\int }_{{\mathbb{R}}^N}|{\upsilon }_n{|}^{r\left(x\right)}dx\to {\int }_{{\mathbb{R}}^N}{\left|\upsilon \right|}^{r\left(x\right)}dx$, we obtain

$$\begin{array}{c}\hfill ⟨I^{\prime }\left({\upsilon }_n\right),{\upsilon }_n⟩\ge m{\varrho }_E\left({\varpi }_n\right)-{\int }_{{\mathbb{R}}^N}{\left|{\varpi }_n\right|}^{p^{\ast }\left(x\right)}dx+o\left(1\right).\end{array}$$

(38)

According to (1), we have

$$\begin{array}{cc}\hfill ⟨I^{\prime }\left({\upsilon }_n\right),{\upsilon }_n⟩\ge & m{\varrho }_E\left({\varpi }_n\right)-{\int }_{{\mathbb{R}}^N}{\left|{\varpi }_n\right|}^{p^{\ast }\left(x\right)}dx+o\left(1\right)\hfill \\ \hfill \ge & m\parallel {\varpi }_n{\parallel }^{p^{+}}-S^{-{\left(p^{\ast }\right)}^{-}}{\parallel {\varpi }_n\parallel }^{{\left(p^{\ast }\right)}^{-}}+o\left(1\right).\hfill \end{array}$$

(39)

Let ${lim}_{n\to \infty }\parallel {\varpi }_n\parallel =l$. Then, $m{l}^{p^{+}}<S^{-{\left(p^{\ast }\right)}^{-}}{l}^{{\left(p^{\ast }\right)}^{-}}$ as $n\to \infty$. It follows that either $l=0$ or

$$l\ge {\left(m{S}^{{\left(p^{\ast }\right)}^{-}}\right)}^{{\displaystyle \frac{1}{{\left(p^{\ast }\right)}^{-}-p^{+}}}}=l_0.$$

If $l\ge l_0$, we obtain

$$\begin{array}{cc}\hfill I\left({\upsilon }_n\right)-{\displaystyle \frac{1}{{\left(p^{\ast }\right)}^{-}}}⟨I^{\prime }\left({\upsilon }_n\right),{\upsilon }_n⟩\ge & I\left(\upsilon \right)+m\left({\displaystyle \frac{1}{\zeta p^{+}}}-{\displaystyle \frac{1}{{\left(p^{\ast }\right)}^{-}}}\right){\parallel {\varpi }_n\parallel }^{p^{+}}+o\left(1\right).\hfill \end{array}$$

As $n\to \infty$, we obtain

$$I\left(\upsilon \right)\le c-m\left({\displaystyle \frac{1}{\zeta p^{+}}}-{\displaystyle \frac{1}{{\left(p^{\ast }\right)}^{-}}}\right)l^{p^{+}}\le c-c_{\ast }<0,$$

which means that $I\left(\upsilon \right)<0$.

Since $⟨I^{\prime }\left(\upsilon \right),\upsilon ⟩=0$, we have

$$\begin{array}{cc}\hfill I\left(\upsilon \right)=& I\left(\upsilon \right)-\frac{1}{r^{-}}⟨I^{\prime }\left(\upsilon \right),\upsilon ⟩\hfill \\ \hfill \ge & \widehat{M}\left(\Phi \left(\upsilon \right)\right)-\frac{1}{r^{-}}M\left(\Phi \left(\upsilon \right)\right)\left(\Phi \left(\upsilon \right)\right)+\left(\frac{1}{r^{-}}-\frac{1}{p^{\ast }\left(x\right)}\right){\int }_{{\mathbb{R}}^N}{\left|\upsilon \right|}^{p^{\ast }\left(x\right)}dx\hfill \\ \hfill \ge & m(\frac{1}{\zeta p^{+}}-\frac{1}{r^{-}{p}^{+}}){\varrho }_E(v)>0.\hfill \end{array}$$

which is a contradiction. Thus, $l=0$ and ${\upsilon }_n\to \upsilon$. The proof is completed. □

Lemma 9.

The functional I possesses the geometric properties designated (i) and (ii) in Lemma 4.

Proof. 

(i) From (M1)–(M2), Lemma 3, and Theorem 3, we have

$$\begin{array}{cc}\hfill I\left(\upsilon \right)=& \widehat{M}\left(\Phi \left(\upsilon \right)\right)-{\displaystyle \frac{\mu }{r\left(x\right)}}{\int }_{{\mathbb{R}}^N}{\left|\upsilon \right|}^{r\left(x\right)}dx-{\displaystyle \frac{1}{p^{\ast }\left(x\right)}}{\int }_{{\mathbb{R}}^N}{\left|\upsilon \right|}^{p^{\ast }\left(x\right)}dx\hfill \\ \hfill \ge & {\displaystyle \frac{m}{\zeta p^{+}}}{\varrho }_E\left(\upsilon \right)-{\displaystyle \frac{\mu C_{r^{-}}^{r^{-}}}{r^{-}}}{\parallel \upsilon \parallel }^{r^{-}}-{\displaystyle \frac{S^{-{\left(p^{\ast }\right)}^{-}}}{{\left(p^{\ast }\right)}^{-}}}{\parallel \upsilon \parallel }^{{\left(p^{\ast }\right)}^{-}}\hfill \\ \hfill \ge & {\displaystyle \frac{m}{\zeta p^{+}}}{\parallel \upsilon \parallel }^{p^{+}}-{\displaystyle \frac{\mu C_{r^{-}}^{r^{-}}}{r^{-}}}{\parallel \upsilon \parallel }^{r^{-}}-{\displaystyle \frac{S^{-{\left(p^{\ast }\right)}^{-}}}{{\left(p^{\ast }\right)}^{-}}}{\parallel \upsilon \parallel }^{{\left(p^{\ast }\right)}^{-}}.\hfill \end{array}$$

(40)

Note that when $p^{+}<r^{-}<{\left(p^{\ast }\right)}^{-}$, the first conclusion of Lemma 4 holds provided that we choose $\beta$ to be sufficiently small.

(ii) Fix ${\upsilon }_0\in E$ with ${\upsilon }_0\ne 0$. We deduce from (M2) that

$$\begin{array}{cc}\hfill I\left(t{\upsilon }_0\right)=& \widehat{M}\left(\Phi \left(t{\upsilon }_0\right)\right)-{\displaystyle \frac{\mu }{r\left(x\right)}}{\int }_{{\mathbb{R}}^N}|t{\upsilon }_0{|}^{r\left(x\right)}dx-{\displaystyle \frac{1}{p^{\ast }\left(x\right)}}{\int }_{{\mathbb{R}}^N}{|t{\upsilon }_0|}^{p^{\ast }\left(x\right)}dx\hfill \\ \hfill \le & \widehat{M}\left(1\right){(\Phi \left(t{\upsilon }_0\right))}^{\zeta }-{\displaystyle \frac{t^{{\left(p^{\ast }\right)}^{-}}}{{\left(p^{\ast }\right)}^{+}}}{\int }_{{\mathbb{R}}^N}{|{\upsilon }_0|}^{p^{\ast }\left(x\right)}dx\hfill \\ \hfill =& {\displaystyle \frac{\widehat{M}\left(1\right)}{{\left(p^{-}\right)}^{\zeta }}}{t}^{\zeta p^{+}}\parallel {\upsilon }_0{\parallel }^{\zeta p^{+}}-{\displaystyle \frac{t^{{\left(p^{\ast }\right)}^{-}}}{{\left(p^{\ast }\right)}^{+}}}{\int }_{{\mathbb{R}}^N}{\left|{\upsilon }_0\right|}^{p^{\ast }\left(x\right)}dx.\hfill \end{array}$$

(41)

Note that when $\zeta p^{+}<{\left(p^{\ast }\right)}^{-}$, we have $I\left(t{\upsilon }_0\right)\to -\infty$. Consequently, I satisfies property (ii). The lemma is proved. □

Lemma 10.

Suppose that (M1)–(M2), (V2), and (H5) hold. Then, we can find a constant ${\mu }^{\ast }>0$ such that

$$c=\underset{\alpha \in \Lambda }{inf}\underset{0\le t\le 1}{max}I\left(\alpha \left(t\right)\right)<\left({\displaystyle \frac{m}{\zeta p^{+}}}-{\displaystyle \frac{m}{{\left(p^{\ast }\right)}^{-}}}\right){\left(m{S}^{{\left(p^{\ast }\right)}^{-}}\right)}^{{\displaystyle \frac{p^{+}}{{\left(p^{\ast }\right)}^{-}-p^{+}}}}$$

for each $\mu \ge {\mu }^{\ast }$.

Proof. 

Let $u_0\in E$ such that $\parallel u_0\parallel =1$. Obviously, when

$$\begin{array}{c}\hfill I\left(0\right)=0,\underset{t\to \infty }{lim}I\left(t{u}_0\right)=-\infty ,\end{array}$$

there exists $t_{\mu }>0$ satisfying ${max}_{t\ge 0}I\left(t{u}_0\right)=I\left(t_{\mu }{u}_0\right)$. Therefore, $⟨I\left(t_{\mu }{u}_0\right),t_{\mu }{u}_0⟩=0$, that is,

$$\begin{array}{c}\hfill M\left(\Phi \left(t_{\mu }{u}_0\right)\right){\varrho }_E\left(t_{\mu }{u}_0\right)=\mu {\int }_{{\mathbb{R}}^N}|t_{\mu }{u}_0{|}^{r\left(x\right)}dx+{\int }_{{\mathbb{R}}^N}{\left|t_{\mu }{u}_0\right|}^{p^{\ast }\left(x\right)}dx.\end{array}$$

(42)

According to (42) and (M2), we have

$$\begin{array}{cc}\hfill \zeta p^{+}\widehat{M}\left(\Phi \left(t_{\mu }{u}_0\right)\right)\ge & M\left(\Phi \left(t_{\mu }{u}_0\right)\right){\varrho }_E\left(t_{\mu }{u}_0\right)\hfill \\ \hfill =& \mu {\int }_{{\mathbb{R}}^N}|t_{\mu }{u}_0{|}^{r\left(x\right)}dx+{\int }_{{\mathbb{R}}^N}{\left|t_{\mu }{u}_0\right|}^{p^{\ast }\left(x\right)}dx\hfill \\ \hfill \ge & {\int }_{{\mathbb{R}}^N}{t}_{\mu }^{p^{\ast }\left(x\right)}{\left|u_0\right|}^{p^{\ast }\left(x\right)}dx.\hfill \end{array}$$

Without loss of generality, we assume that $t_{\mu }\ge 1$ for each $\mu >0$. Using (M2) once more, we obtain

$$\begin{array}{cc}\hfill t_{\mu }^{{\left(p^{\ast }\right)}^{-}}{\int }_{{\mathbb{R}}^N}{\left|u_0\right|}^{p^{\ast }\left(x\right)}dx\le & \zeta p^{+}\widehat{M}\left(\Phi \left(t_{\mu }{u}_0\right)\right)\le \zeta p^{+}\widehat{M}\left(1\right){\left(\Phi \left(t_{\mu }{u}_0\right)\right)}^{\zeta }\hfill \\ \hfill \le & {\displaystyle \frac{\zeta p^{+}\widehat{M}\left(1\right)}{{\left(p^{-}\right)}^{\zeta }}}{t}_{\mu }^{\zeta p^{+}}.\hfill \end{array}$$

Thus, $\left\{t_{\mu }\right\}$ is bounded. Next, we prove that $t_{\mu }\to 0$ as $\mu \to \infty$. Assume, for the sake of contradiction, that there is a constant $t_0>0$ and a sequence ${\mu }_j\to \infty$ as $j\to \infty$ satisfying

$$t_{{\mu }_j}\to t_0,\mathrm{as}j\to \infty .$$

By the Lebesgue Dominated Convergence Theorem, we have

$${\int }_{{\mathbb{R}}^N}|t_{{\mu }_j}{u}_0{|}^{r\left(x\right)}dx\to {\int }_{{\mathbb{R}}^N}{\left|t_0{u}_0\right|}^{r\left(x\right)}dx,$$

for $j\to \infty$. Thus,

$$\begin{array}{c}\hfill \underset{j\to \infty }{lim}{\mu }_j{\int }_{{\mathbb{R}}^N}{\left|t_{{\mu }_j}{u}_0\right|}^{r\left(x\right)}dx\to \infty .\end{array}$$

Combining this with (42), we conclude that

$$\begin{array}{c}\hfill M\left(\Phi \left(t_0{u}_0\right)\right){\varrho }_E\left(t_0{u}_0\right)=\infty .\end{array}$$

This is impossible. Thus, $t_{\mu }\to 0$ as $\mu \to \infty$. Consequently, we obtain

$$\begin{array}{c}\hfill \underset{\mu \to \infty }{lim}\mu {\int }_{{\mathbb{R}}^N}{\left|t_{\mu }{u}_0\right|}^{r\left(x\right)}dx\to 0.\end{array}$$

Therefore, we can easily derive that

$$\begin{array}{c}\hfill \underset{\mu \to \infty }{lim}\underset{t\ge 0}{max}I\left(t{u}_0\right)=\underset{\mu \to \infty }{lim}I\left(t_{\mu }{u}_0\right).\end{array}$$

This implies that there exists ${\mu }_{\ast }$ such that, for each $\mu \ge {\mu }_{\ast }$,

$$\begin{array}{c}\hfill \underset{t\ge 0}{max}I\left(t{u}_0\right)<\left({\displaystyle \frac{m}{\zeta p^{+}}}-{\displaystyle \frac{m}{{\left(p^{\ast }\right)}^{-}}}\right){\left(m{S}^{{\left(p^{\ast }\right)}^{-}}\right)}^{{\displaystyle \frac{p^{+}}{{\left(p^{\ast }\right)}^{-}-p^{+}}}}.\end{array}$$

Taking $\varphi =T_0{u}_0$ to verify $I\left(\varphi \right)<0$, we have

$$\begin{array}{c}\hfill c\le \underset{t\in [0,1]}{max}I\left(\alpha \left(t\right)\right),\alpha \left(t\right)=t{T}_0{u}_0.\end{array}$$

Therefore,

$$\begin{array}{c}\hfill c\le \underset{t\in [0,1]}{max}I\left(\alpha \left(t\right)\right)<\left({\displaystyle \frac{m}{\zeta p^{+}}}-{\displaystyle \frac{m}{{\left(p^{\ast }\right)}^{-}}}\right){\left(m{S}^{{\left(p^{\ast }\right)}^{-}}\right)}^{{\displaystyle \frac{p^{+}}{{\left(p^{\ast }\right)}^{-}-p^{+}}}}\end{array}$$

for each $\mu \ge {\mu }_{\ast }$. The proof is now completed. □

Proof of Theorem 2.

According to Lemma 9, the functional I satisfies properties (i) and (ii) of the mountain pass theorem (see Lemma 4). By Lemmas 7, 8, and 10, we can deduce that Theorem 2 holds. □