Existence and Symmetry of Solutions for a Class of Fractional Schrödinger–Poisson Systems

Abstract

In this paper, we investigate a class of Schrödinger–Poisson systems with critical growth. By the principle of concentration compactness and variational methods, we prove that the system has radially symmetric solutions, which improve the related results on this topic.

1. Introduction

In recent years, fractional equations or systems have been studied extensively by researchers due to their various applications in various fields, such as obstacle problems, electrical circuits, quantum mechanics, and phase transitions; see [1,2,3,4,5,6,7] and their references. It is particularly important to mention that Laskin in [7] established the following time-dependent Schrödinger equation involving a fractional Laplacian when he expanded the Feynman path integral, from Brownian-like to Lévy-like quantum mechanical paths

$$i\frac{\partial \psi }{\partial t}={(-\Delta )}^{\alpha }\psi +(V\left(x\right)+\kappa )\psi -g(x,t), (x,t)\in {\mathbb{R}}^3\times \mathbb{R}.$$

(1)

where $\kappa >0$ is a constant and ${(-\Delta )}^{\alpha }={\mathcal{F}}^{-1}{\left(\right|\xi |}^{2\alpha }\mathcal{F}u)$ is the fractional Laplacian of order $\alpha$, and $\mathcal{F}$ denotes the usual Fourier transform in ${\mathbb{R}}^3$.

The fractional Schrödinger equation is a fundamental equation in the fractional quantum mechanics when investigating the quantum particles on stochastic fields modeled, and it has been getting a lot of attention from researchers; see [8,9,10,11,12,13] and their references. For example, Li et al. in [13] studied the following form of fractional Schrödinger equations

$${(-\Delta )}^{\alpha }u+V\left(x\right)u=\lambda K\left(x\right)f\left(u\right)+{\left|u\right|}^{2_{\alpha }^{*}-2}u,$$

(2)

where $\alpha \in (0,1)$, $2_{\alpha }^{*}=\frac{2N}{N-2\alpha }$ is the fractional critical Sobolev exponent, $\lambda >0$ is a parameter, $V\left(x\right)$ is potential function with $\underset{\left|x\right|\to +\infty }{lim}V\left(x\right)=0$, i.e., $V\left(x\right)$ is vanishing at infinity. Using the variational method, they obtained the existence of a positive solution for (2).

When dealing with the quantum particle in three-dimensional space interacting with an unknown electromagnetic field, Benci and Fortunato in [14] first introduced the following classical Schrödinger–Poisson system

$$\left\{\begin{array}{cc}-\Delta u+V\left(x\right)u+\varphi u=f(x,u),\hfill & x\in {\mathbb{R}}^3,\hfill \\ -\Delta \varphi =u^2,\hfill & x\in {\mathbb{R}}^3,\hfill \end{array}\right.$$

(3)

The authors obtained a sequence solution of (3) by variational methods. Such a system (3), also called the Schrödinger–Maxwell system, arises in an interesting physical context. According to a classical model, the interaction of a charge particle with an electromagnetic field can be described by coupling the nonlinear Schrödinger–Poisson system. Recently, system (3) has been widely investigated because it has a strong physical meaning; see [15,16,17] and their references. For example, Azzollini and Pomponio in [15] considered system (3) by variational methods; they established the existence of a ground state solution when potential $V\left(x\right)$ is a positive constant or non-constant.

Fractional Schrödinger–Poisson systems have received lots of attention in recent years, and many of the works have studied the existence of solutions of it; see [18,19,20,21,22] and their references. As far as we know, few studies have considered the existence of solutions for the fractional Schrödinger–Poisson system with critical growth. Gu et al. in [19] only studied the existence of a positive solution by variational methods, and there are no relevant articles that consider the existence of radially symmetric solutions of the fractional Schrödinger–Poisson system with critical growth. We tried to deal with this problem and obtained novel existence results by using new analytical methods, which are different from the related conclusions on this topic.

Motivated by above results, in this paper, we mainly study the following fractional Schrödinger–Poisson system

$$\left\{\begin{array}{cc}{(-\Delta )}^{\alpha }u+V\left(x\right)u+\varphi u=K\left(x\right)f\left(u\right)+\lambda {\left|u\right|}^{2_{\alpha }^{*}-2}u,\hfill & x\in {\mathbb{R}}^3,\hfill \\ {(-\Delta )}^t\varphi =u^2,\hfill & x\in {\mathbb{R}}^3,\hfill \end{array}\right.$$

(4)

where $0<\alpha \le t<1$, $4\alpha +2t>3$, $2_{\alpha }^{*}=\frac{6}{3-2\alpha }$. We assume $V\left(x\right)$ and $K\left(x\right)$ are radially symmetric, i.e., $V\left(x\right)=V\left(\right|x\left|\right)$ and $K\left(x\right)=K\left(\right|x\left|\right)$ for any $x\in {\mathbb{R}}^3$, and satisfies the following assumptions

(V1)

$V\in C^1({\mathbb{R}}^3,\mathbb{R})$, $\nabla V\left(x\right)·x\le 0$ for any $x\in {\mathbb{R}}^3$;

(V2)

Positive constants $V_1$ and $V_1$ exist such that $V_1\le V\left(x\right)\le V_2$ for any $x\in {\mathbb{R}}^3$;

(K1)

$K\in C^1({\mathbb{R}}^3,\mathbb{R})$ and there exists a constant $K_0$ such that $0\le \nabla K\left(x\right)·x\le K_0$ for any $x\in {\mathbb{R}}^3$;

(K2)

Positive constants $K_1$ and $K_1$ exist such that $K_1\le K\left(x\right)\le K_2$ for any $x\in {\mathbb{R}}^3$;

We make the following assumptions for the function f.

(f1)

${lim}_{t\to 0}\frac{f\left(t\right)}{t}=0$;

(f2)

There exits $C>0$ such that $\left|f\left(t\right)\right|\le C(1+|t{|}^{p-1}) \mathrm{for} \mathrm{all} t\in \mathbb{R}$, where $p\in (2,2_{\alpha }^{*})$.

(f3)

There exists $\theta >4$ such that $0<\theta F\left(t\right)\le tf\left(t\right)$ for all $t\in \mathbb{R}$, where $F\left(t\right)={\int }_0^{t}f\left(\eta \right)d\eta$.

(f4)

$f\left(t\right)=f(-t)$ for all $t\in \mathbb{R}$.

Now we state the main results of this paper.

Theorem 1.

Assume that $V\left(x\right),K\left(x\right)$ fulfills (V1), (V2), (K1), (K2) and f satisfies (f1)–(f3). There exists ${\lambda }_1>0$ such that for any $0<\lambda <{\lambda }_1$, problem (4) has a nontrivial radially symmetric solution.

Theorem 2.

Assume that $V\left(x\right),K\left(x\right)$ fulfills (V1), (V2), (K1), (K2) and f satisfies (f1)–(f4). Then, problem (4) has infinitely many radially symmetric solutions.

This paper is organized as follows. In Section 2, we introduce the preliminaries of fractional Sobolev space and some lemmas. In Section 3, we give the proofs of our main result.

2. Preliminaries

In this section, we present a short review of fractional Sobolev spaces and some lemmas, which we used to prove our main result. The complete introduction of fractional Sobolev spaces can be found in [23]. For the properties of the function ${\varphi }_u^s$, see [18]. Throughout this paper, we denote ${\parallel ·\parallel }_p$ as the usual norm of $L^p\left({\mathbb{R}}^3\right)$, $C_i(i=1,2,\cdots )$ or C denotes the positive constants.

For $\alpha \in (0,1)$, the fractional Sobolev space $H^{\alpha }\left({\mathbb{R}}^3\right)$ is defined by

$$H^{\alpha }\left({\mathbb{R}}^3\right):=\left\{u\in L^2\left({\mathbb{R}}^3\right):{\int }_{{\mathbb{R}}^3}{(1+|\xi |}^{2\alpha }\left)\right|\mathcal{F}u\left(\xi \right){|}^{2}d\xi <+\infty \right\}.$$

and the norm is

$${\parallel u\parallel }_{H^{\alpha }}:={\left({\int }_{{\mathbb{R}}^3}{\left|u\left(\xi \right)\right|}^{2}d\xi +{\int }_{{\mathbb{R}}^3}{\left|\xi \right|}^{2\alpha }{\left|\mathcal{F}u\left(\xi \right)\right|}^{2}d\xi \right)}^{\frac{1}{2}}.$$

$H^{\alpha }\left({\mathbb{R}}^3\right)$ is a Hilbert space with the inner product for any $u,v\in H^{\alpha }\left({\mathbb{R}}^3\right)$

$$\langle u,v\rangle :={\int }_{{\mathbb{R}}^3}{(-\Delta )}^{\frac{\alpha }{2}}u{(-\Delta )}^{\frac{\alpha }{2}}vdx+{\int }_{{\mathbb{R}}^3}uvdx,$$

and norm

$$\parallel u\parallel :={\left({\int }_{{\mathbb{R}}^3}{\left|{(-\Delta )}^{\frac{\alpha }{2}}u\right|}^{2}dx+{\int }_{{\mathbb{R}}^3}{u}^{2}dx\right)}^{\frac{1}{2}}.$$

For the second equation in problem (4), it has an unique solution ${\varphi }_u^t$ (see [18]). Substituting ${\varphi }_u^t$ in (4), we have the following fractional Schrödinger equation

$$\begin{array}{c}\hfill {(-\Delta )}^{\alpha }u+V\left(x\right)u+{\varphi }_u^{t}u=K\left(x\right)f\left(u\right)+\lambda {\left|u\right|}^{2_{\alpha }^{*}-2}u,x\in {\mathbb{R}}^3,\end{array}$$

(5)

whose solutions can be obtained by seeking critical points of the energy functional $I_{\lambda }:H^{\alpha }\left({\mathbb{R}}^3\right)\to \mathbb{R}$

$$\begin{array}{cc}\hfill I_{\lambda }\left(u\right):=& \frac{1}{2}{\int }_{{\mathbb{R}}^3}\left({|(-\Delta )}^{\frac{\alpha }{2}}{u|}^2+V\left(x\right)u^2\right)dx+\frac{1}{4}{\int }_{{\mathbb{R}}^3}{\varphi }_u^t{u}^{2}dx\hfill \\ \hfill & -{\int }_{{\mathbb{R}}^3}K\left(x\right)F\left(u\right)dx-\frac{\lambda }{2_{\alpha }^{*}}{\int }_{{\mathbb{R}}^3}{\left|u\right|}^{2_{\alpha }^{*}}dx.\hfill \end{array}$$

which is well defined in $H^{\alpha }\left({\mathbb{R}}^3\right)$ and $I\in C^1(H^{\alpha }\left({\mathbb{R}}^3\right),\mathbb{R})$, and for all $u,v\in H^{\alpha }\left({\mathbb{R}}^3\right)$,

$$\begin{array}{cc}\hfill \langle I_{\lambda }^{\prime }\left(u\right),v\rangle :=& {\int }_{{\mathbb{R}}^3}{(-\Delta )}^{\frac{\alpha }{2}}u{(-\Delta )}^{\frac{\alpha }{2}}vdx+{\int }_{{\mathbb{R}}^3}V\left(x\right)uvdx+{\int }_{{\mathbb{R}}^3}{\varphi }_u^{t}uvdx\hfill \\ \hfill & -{\int }_{{\mathbb{R}}^3}K\left(x\right)f\left(u\right)vdx-\lambda {\int }_{{\mathbb{R}}^3}{\left|u\right|}^{2_{\alpha }^{*}-2}uvdx.\hfill \end{array}$$

Definition 1

([24]). Let X be a Banach space, $\phi \in C^1(X,{\mathbb{R}}^1))$. A sequence $\left\{u_n\right\}$ in X is a (PS)-sequence if $\left|\phi \right(u_n\left)\right|\le c$ uniformly in n, while $\parallel {\phi }^{\prime }\left(u_n\right)\parallel \to 0$ as $n\to \infty$.

Lemma 1

([23]). Let $\alpha \in (0,1)$ and $N\ge 1$ satisfies $N>2\alpha$. Then there exists $C=C(N,\alpha )>0$ such that

$${\parallel u\parallel }_{L^{2_{\alpha }^{*}}\left({\mathbb{R}}^3\right)}\le C{\parallel u\parallel }_{H^{\alpha }\left({\mathbb{R}}^3\right)}$$

for every $u\in H^{\alpha }\left({\mathbb{R}}^3\right)$, where $2_{\alpha }^{*}=\frac{2N}{N-2\alpha }$ is the fractional critical exponent. Moreover, the embedding $H^{\alpha }\left({\mathbb{R}}^3\right)\hookrightarrow L^{\gamma }\left({\mathbb{R}}^3\right)$ is continuous for each $\gamma \in [2,2_{\alpha }^{*}]$ and is locally compact for$\gamma \in [2,2_{\alpha }^{*})$.

In the following, the symmetric mountain pass lemma is presented and used to prove our main result.

Lemma 2

([25]). Let X be an infinite-dimensional Banach space with $X=M⨁N$, where M is a finite-dimensional subspace of X. Assume that $I\in C^1(X,\mathbb{R})$ is a functional that satisfies the (PS)-condition and the following properties:

(A1)

I is even for all $u\in X$ and$I\left(0\right)=0$.

(A2)

There exist two constant $\varrho ,\sigma >0$ such that$I\left(u\right)\ge \sigma$.

(A3)

There exists constant $\xi =\xi \left(\Omega \right)>0$ such that $0\ge I\left(u\right)$ for any $u\in X\setminus B_{\xi }\left(\Omega \right)$, where Ω is an arbitrary finite-dimensional subspace of X and$B_{\xi }\left(\Omega \right):=\{u\in \Omega :\parallel u\parallel \le \xi \}$.

Then the functional I has an unbounded sequence of critical values.

In the following, we prove that the functional $I_{\lambda }$ satisfies the mountain pass geometry.

Lemma 3.

Suppose $V\left(x\right),K\left(x\right)$ satisfies (V1), (V2), (K1), (K2) and f satisfies (f1)–(f3), then the functional $I_{\lambda }$ satisfies

(i)

There exists $\beta ,\rho >0$ such that $I_{\lambda }\left(u\right)\ge \beta$ for$\parallel u\parallel =\rho$.

(ii)

There exists $e\in H^{\alpha }\left({\mathbb{R}}^3\right)$ with $\parallel e\parallel >\rho$ such that$I_{\lambda }\left(e\right)<0$.

Proof. 

(i) If $V\left(X\right),K\left(X\right)$ satisfies (V1), (V2), (K1) and (K2), by (f1) and (f2), for $\epsilon >0$, there exists $C_{\epsilon }>0$ such that

$$\left|f\left(t\right)\right|\le \epsilon \left|t\right|+C_{\epsilon }{\left|t\right|}^{p-1},$$

(6)

and

$$\left|F\left(t\right)\right|\le \frac{\epsilon }{2}{\left|t\right|}^2+\frac{C_{\epsilon }}{p}{\left|t\right|}^p.$$

(7)

Therefore, by Lemma 1, (7) and Hölder’s inequality, we have

$$\begin{array}{cc}\hfill I_{\lambda }\left(u\right)=& \frac{1}{2}{\parallel u\parallel }^2+\frac{1}{4}{\int }_{{\mathbb{R}}^3}{\varphi }_u^t{u}^{2}dx-{\int }_{{\mathbb{R}}^3}K\left(x\right)F\left(u\right)dx-\frac{\lambda }{2_{\alpha }^{*}}{\int }_{{\mathbb{R}}^3}{\left|u\right|}^{2_{\alpha }^{*}}dx\hfill \\ \hfill \ge & \frac{1}{2}{\parallel u\parallel }^2-{\int }_{{\mathbb{R}}^3}K\left(x\right)F\left(u\right)dx-\frac{\lambda }{2_{\alpha }^{*}}{\int }_{{\mathbb{R}}^3}{\left|u\right|}^{2_{\alpha }^{*}}dx\hfill \\ \hfill \ge & \frac{1}{2}{\parallel u\parallel }^2-{\int }_{{\mathbb{R}}^3}K\left(x\right)\left[\frac{\epsilon }{2}{\left|t\right|}^2+\frac{C_{\epsilon }}{p}{\left|t\right|}^p\right]dx-\frac{\lambda }{2_{\alpha }^{*}}{\int }_{{\mathbb{R}}^3}{\left|u\right|}^{2_{\alpha }^{*}}dx\hfill \\ \hfill =& \frac{1}{2}{\parallel u\parallel }^2-\frac{\epsilon }{2}{\int }_{{\mathbb{R}}^3}{K\left(x\right)\left|t\right|}^{2}dx-\frac{C_{\epsilon }}{p}{\int }_{{\mathbb{R}}^3}{K\left(x\right)\left|t\right|}^{p}dx-\frac{\lambda C_{2_{\alpha }^{*}}}{2_{\alpha }^{*}}{\parallel u\parallel }^{2_{\alpha }^{*}}\hfill \\ \hfill \ge & \frac{1-\epsilon d_2}{2}{\parallel u\parallel }^2-\frac{C_{\epsilon }{d}_p}{p}{\parallel u\parallel }^p-\frac{\lambda C_{2_{\alpha }^{*}}}{2_{\alpha }^{*}}{\parallel u\parallel }^{2_{\alpha }^{*}},\hfill \end{array}$$

which implies that there exist two positive constants $\beta ,\rho >0$ such that $I_{\lambda }\left(u\right)\ge \beta$ for $\parallel u\parallel =\rho$.

(ii) If $V\left(X\right),K\left(X\right)$ satisfies (V1), (V2), (K1) and (K2), from (f3), $C_1$ exists such that

$$\begin{array}{c}\hfill F\left(u\right)\ge C_1{\left|u\right|}^{\theta }\end{array}$$

(8)

for $\left|u\right|>M_1$, where $M_1\to +\infty$. By the properties of the function ${\varphi }_u^s$ (see Lemma 2.3-(2) in [18]) and (8), for $t>0$ and any $u\in H^{\alpha }\left({\mathbb{R}}^3\right)$, we have

$$\begin{array}{cc}\hfill I_{\lambda }\left(tu\right)=& \frac{t^2}{2}{\parallel u\parallel }^2+\frac{t^4}{4}{\int }_{{\mathbb{R}}^3}{\varphi }_u^t{u}^{2}dx-{\int }_{{\mathbb{R}}^3}K\left(x\right)F\left(tu\right)dx-\frac{\lambda t^{2_{\alpha }^{*}}}{2_{\alpha }^{*}}{\int }_{{\mathbb{R}}^3}{\left|u\right|}^{2_{\alpha }^{*}}dx\hfill \\ \hfill \le & \frac{t^2}{2}{\parallel u\parallel }^2+\frac{C_2{t}^4}{4}{\parallel u\parallel }^4-C_1{t}^{\theta }{\parallel u\parallel }^{\theta }-\frac{\lambda C_{2_{\alpha }^{*}}}{2_{\alpha }^{*}}{\parallel u\parallel }^{2_{\alpha }^{*}}.\hfill \end{array}$$

Therefore, $I_{\lambda }\left(tu\right)\to -\infty$ as $t\to +\infty$, which means that there exists a constant $M_2>0$ such that $t_0>M_2$ such that $I_{\lambda }\left(t_{0}u\right)\le 0$. Hence, we can choose $e:=t_{0}u\in H^{\alpha }\left({\mathbb{R}}^3\right)$ with $\parallel e\parallel >\rho$ such that $I_{\lambda }\left(e\right)<0$. □

Lemma 4.

Assume that (f1)–(f4) re satisfied. Then, any (PS)-sequence for $I_{\lambda }$ is bounded.

Proof. 

Let $\left\{u_n\right\}\subset$ be a (PS)-sequence. By (f3), for $\theta >4$, we have

$$\begin{array}{cc}\hfill 1+\parallel u_n\parallel \ge & I_{\lambda }\left(u_n\right)-\frac{1}{\theta }\langle I_{\lambda }^{\prime }\left(u_n\right),u_n\rangle \hfill \\ \hfill =& \left(\frac{1}{2}-\frac{1}{\theta }\right){\parallel u_n\parallel }^2+\left(\frac{1}{4}-\frac{1}{\theta }\right){\int }_{{\mathbb{R}}^3}{\varphi }_{u_n}^t{u_n}^{2}dx\hfill \\ \hfill & +{\int }_{{\mathbb{R}}^3}K\left(x\right)\left(\frac{1}{\theta }f\left(u_n\right)u_n-F\left(u_n\right)\right)dx+\lambda \left(\frac{1}{\theta }-\frac{1}{2_{\alpha }^{*}}\right){\int }_{{\mathbb{R}}^3}{\left|u_n\right|}^{2_{\alpha }^{*}}dx\hfill \\ \hfill \ge & \left(\frac{1}{2}-\frac{1}{\theta }\right)\parallel u_n{\parallel }^2+\lambda \left(\frac{1}{\theta }-\frac{1}{2_{\alpha }^{*}}\right)C_{2_{\alpha }^{*}}{\parallel u_n\parallel }^{2_{\alpha }^{*}},\hfill \end{array}$$

which implies that $\left\{u_n\right\}$ is bounded in $H^{\alpha }\left({\mathbb{R}}^3\right)$. □

Lemma 5

([26]). Let $\left\{u_n\right\}\subset {\dot{H}}^{\alpha }\left({\mathbb{R}}^3\right)$ be a sequence and $u_n\rightharpoonup u$ weakly as $n\to \infty$ and such that ${|(-\Delta )}^{\frac{\alpha }{2}}{u}_n{|}^2\rightharpoonup \mu$ and $|u_n{|}^{2_{\alpha }^{*}}\rightharpoonup \nu$ weakly-* in $\mathcal{M}\left({\mathbb{R}}^3\right)$. Then, either $u_n\to u$ in $L_{loc}^{2_{\alpha }^{*}}\left({\mathbb{R}}^3\right)$ or there exists a (at most countable) set of distinct points ${\left\{x_j\right\}}_{j\in J}\subset \overline{\Omega }$ and positive numbers ${\left\{{\nu }_j\right\}}_{j\in J}$ such that $\nu ={\left|u\right|}^{2_{\alpha }^{*}}+{\sum }_{j\in J}{\nu }_j{\delta }_{x_j}$, where $\Omega \subseteq {\mathbb{R}}^3$ is an open subset. If, Ω is bounded, then there exists a positive measure $\tilde{\mu }\in \mathcal{M}\left({\mathbb{R}}^3\right)$ with supp$\tilde{\mu }\subset \overline{\Omega }$ and positive numbers ${\left\{{\mu }_j\right\}}_{j\in J}$ such that ${\mu =|(-\Delta )}^{\frac{\alpha }{2}}{u}_n{|}^2+\tilde{\mu }+{\sum }_{j\in J}{\mu }_j{\delta }_{x_j}$.

Lemma 6

([27]). Define

$${\mu }_{\infty }=\underset{R\to \infty }{lim}\underset{n\to \infty }{lim\; sup}{\int }_{\{x\in {\mathbb{R}}^3:|x|>R\}}{(-\Delta )}^{\frac{\alpha }{2}}{u}_n{|}^{2}dx,$$

$${\nu }_{\infty }=\underset{R\to \infty }{lim}\underset{n\to \infty }{lim\; sup}{\int }_{\{x\in {\mathbb{R}}^3:|x|>R\}}{\left|u_n\right|}^{2_{\alpha }^{*}}dx.$$

Then the quantities ${\mu }_{\infty }$ and ${\nu }_{\infty }$ are well defined and satisfy

$$\underset{n\to \infty }{lim\; sup}{\int }_{{\mathbb{R}}^3}{\left|{(-\Delta )}^{\frac{\alpha }{2}}{u}_n\right|}^{2}dx={\int }_{{\mathbb{R}}^3}d\mu +{\mu }_{\infty },$$

$$\underset{n\to \infty }{lim\; sup}{\int }_{{\mathbb{R}}^3}{\left|u_n\right|}^{2_{\alpha }^{*}}dx={\int }_{{\mathbb{R}}^3}d\nu +{\nu }_{\infty }.$$

Lemma 7

([27]). Let $\left\{u_n\right\}\subset {\dot{H}}^{\alpha }\left({\mathbb{R}}^3\right)$ such that $u_n\rightharpoonup u$ weakly-* in ${\dot{H}}^{\alpha }\left({\mathbb{R}}^3\right)$, ${|(-\Delta )}^{\frac{\alpha }{2}}{u}_n{|}^2\rightharpoonup \mu$ and $|u_n{|}^{2_{\alpha }^{*}}\rightharpoonup \nu$ weakly-* in $\mathcal{M}\left({\mathbb{R}}^3\right)$ as $n\to \infty$. Then, ${\nu }_i\le {\left(S_{\alpha }^{-1}\mu \left\{x_i\right\}\right)}^{\frac{2_{\alpha }^{*}}{2}}$ for $i\in J$ and ${\nu }_{\infty }\le {\left(S_{\alpha }^{-1}{\mu }_{\infty }\right)}^{\frac{2_{\alpha }^{*}}{2}}$, where $x_i,{\nu }_i$ are from Lemma 5 and ${\mu }_{\infty },{\nu }_{\infty }$ are given in Lemma 6, $S_{\alpha }$ is the best Sobolev constant of ${\dot{H}}^{\alpha }\left({\mathbb{R}}^3\right)\hookrightarrow L^{2_{\alpha }^{*}}\left({\mathbb{R}}^3\right)$, i.e.,

$$S_{\alpha }=\underset{u\in {\dot{H}}^{\alpha }\left({\mathbb{R}}^3\right)}{inf}\frac{{\int }_{{\mathbb{R}}^3}{\left|{(-\Delta )}^{\frac{\alpha }{2}}u\right|}^{2}dx}{{\parallel u\parallel }_{L^{2_{\alpha }^{*}}\left({\mathbb{R}}^3\right)}^2}.$$

(9)

In the following lemma, we show that the functional $I_{\lambda }$ satisfies the (PS)-condition.

Lemma 8.

There exists ${\lambda }_{*}>0$, in which each bounded $\left(PS\right)$ sequence for $I_{\lambda }$ contains a convergent subsequence for any$\lambda \in (0,{\lambda }_{*})$.

Proof. 

Let $\left\{u_n\right\}\subset H^{\alpha }\left({\mathbb{R}}^3\right)$ be a bounded (PS) sequence, i.e.,

$$I_{\lambda }\left(u_n\right)\le C_3 \mathrm{and} I_{\lambda }^{\prime }\left(u_n\right)\to 0 \mathrm{in} E, \mathrm{as} n\to \infty .$$

Passing to a subsequence, still denoted by $\left\{u_n\right\}$. Set $u_n\rightharpoonup u_0$ weakly in E. According to Lemma 1, we have $u_n\to u_0$ in $L^p\left({\mathbb{R}}^3\right)$ and $u_n\to u_0$ a.e. in ${\mathbb{R}}^3$ as $n\to \infty$. Therefore, by Prokhorov’s Theorem [28], there exists $\mu ,\nu \in \mathcal{M}\left({\mathbb{R}}^3\right)$ such that

$${|(-\Delta )}^{\frac{\alpha }{2}}{u}_n{|}^2\to \mu \mathrm{and} {\left|u_n\right|}^{2_{\alpha }^{*}}\to \nu \mathrm{weakly}-* \mathrm{in} \mathcal{M}\left({\mathbb{R}}^3\right) \mathrm{as} n\to \infty .$$

By Lemma 5, we have $u_n\to u_0$ in $L_{loc}^{2_{\alpha }^{*}}\left({\mathbb{R}}^3\right)$ or $\nu =|u_0{|}^{2_{\alpha }^{*}}+{\sum }_{j\in J}{\nu }_j{\delta }_{x_j}$ as $n\to \infty$.

For any $\varphi \in H^{\alpha }\left({\mathbb{R}}^3\right)$, we have

$$\begin{array}{cc}\hfill & \langle I_{\lambda }^{\prime }\left(u_n\right),\varphi \rangle -\langle I_{\lambda }^{\prime }\left(u_0\right),\varphi \rangle \hfill \\ \hfill =& {\int }_{{\mathbb{R}}^3}{(-\Delta )}^{\frac{\alpha }{2}}(u_n-u_0){(-\Delta )}^{\frac{\alpha }{2}}\varphi dx+{\int }_{{\mathbb{R}}^3}V\left(x\right)(u_n-u_0)\varphi dx\hfill \\ \hfill & +\frac{1}{4}{\int }_{{\mathbb{R}}^3}{\varphi }_{u_n}^t{\left(u_n\right)}^{2}dx-\frac{1}{4}{\int }_{{\mathbb{R}}^3}{\varphi }_{u_0}^t{\left(u_0\right)}^{2}dx-{\int }_{{\mathbb{R}}^3}K\left(x\right)(f\left(u_n\right)-f\left(u_0\right)\varphi dx\hfill \\ \hfill & -\lambda {\int }_{{\mathbb{R}}^3}\left(|u_n{|}^{2_{\alpha }^{*}-2}{u}_n-{\left|u_0\right|}^{2_{\alpha }^{*}-2}{u}_0\right)\varphi dx.\hfill \end{array}$$

Since $u_n\rightharpoonup u_0$ weakly in $H^{\alpha }\left({\mathbb{R}}^3\right)$, then

$${\int }_{{\mathbb{R}}^3}{(-\Delta )}^{\frac{\alpha }{2}}(u_n-u_0){(-\Delta )}^{\frac{\alpha }{2}}\varphi dx+{\int }_{{\mathbb{R}}^3}V\left(x\right)(u_n-u_0)\varphi dx\to 0 \mathrm{as} n\to \infty .$$

Since

$$\left\{\right|u_n{|}^{2_{\alpha }^{*}-2}{u}_n-|u_0{{|}^{2_{\alpha }^{*}-2}{u}_0\}}_n \mathrm{is} \mathrm{bounded} \mathrm{in} L^{\frac{2_{\alpha }^{*}}{2_{\alpha }^{*}-1}}\left({\mathbb{R}}^3\right)$$

and

$$|u_n{|}^{2_{\alpha }^{*}-2}{u}_n-{\left|u_0\right|}^{2_{\alpha }^{*}-2}{u}_0\to 0 \mathrm{a}.\mathrm{e}. \mathrm{in} {\mathbb{R}}^3 \mathrm{as} n\to \infty ,$$

then

$$|u_n{|}^{2_{\alpha }^{*}-2}{u}_n-{\left|u_0\right|}^{2_{\alpha }^{*}-2}{u}_0\to 0 \mathrm{weakly} \mathrm{in} L^{\frac{2_{\alpha }^{*}}{2_{\alpha }^{*}-1}}\left({\mathbb{R}}^3\right) \mathrm{as} n\to \infty ,$$

which implies that

$${\int }_{{\mathbb{R}}^3}\left(\right|u_n{|}^{2_{\alpha }^{*}-2}{u}_n-|u_0{|}^{2_{\alpha }^{*}-2}{u}_0)\varphi dx\to 0 \mathrm{as} n\to \infty .$$

By the properties of the function ${\varphi }_u^s$ (see Lemma 2.3-(6) in [18])

$${\int }_{{\mathbb{R}}^3}{\varphi }_{u_n}^t{\left(u_n\right)}^{2}dx-{\int }_{{\mathbb{R}}^3}{\varphi }_{u_0}^t{\left(u_0\right)}^{2}dx\to 0 \mathrm{as} n\to \infty .$$

In the following, we prove that

$${\int }_{{\mathbb{R}}^3}K\left(x\right)(f\left(u_n\right)-f\left(u_0\right))\varphi dx\to 0 \mathrm{as} n\to \infty .$$

As $\langle I_{\lambda }^{\prime }\left(u_n\right),\varphi \rangle \to 0$, then $\langle I_{\lambda }^{\prime }\left(u_0\right),\varphi \rangle =0$, i.e., $I_{\lambda }^{\prime }\left(u_0\right)=0$. Thus,

$$\begin{array}{cc}\hfill & {\int }_{{\mathbb{R}}^3}{\left|{(-\Delta )}^{\frac{\alpha }{2}}{u}_0\right|}^{2}dx+{\int }_{{\mathbb{R}}^3}V\left(x\right)u_0^{2}dx+\frac{1}{4}{\int }_{{\mathbb{R}}^3}{\varphi }_{u_0}^t{\left(u_0\right)}^{2}dx\hfill \\ \hfill & ={\int }_{{\mathbb{R}}^3}K\left(x\right)f\left(u_0\right)u_{0}dx+\frac{\lambda }{2_{\alpha }^{*}}{\int }_{{\mathbb{R}}^3}{\left|u_0\right|}^{2_{\alpha }^{*}}dx.\hfill \end{array}$$

By (6) and the Young inequality, one has

$$\begin{array}{cc}\hfill & \left|[f\left(u_n\right)-f\left(u_0\right)]\varphi \right|\hfill \\ \hfill \le & |u_n\left|\right|\varphi |+C_4|u_n{|}^{p-1}\left|\varphi \right|+|u_0\left|\right|\varphi |+C_5|u_0{|}^{p-1}\left|\varphi \right|\hfill \\ \hfill \le & |u_n-u_0\left|\right|\varphi |+2|u_0\left|\right|\varphi |+C_4|u_n-u_0{|}^{p-1}\left|\varphi \right|+C_5|u_n-u_0{|}^{p-1}\left|\varphi \right|\hfill \\ \hfill \le & \epsilon |u_n-u_0{|}^2+C_{\epsilon }{\left|\varphi \right|}^2+2|u_0\left|\right|\varphi |+\epsilon |u_n-u_0{|}^p+C_4{C}_{\epsilon }{\left|\varphi \right|}^p+C_5|u_0{|}^{p-1}\left|\varphi \right|.\hfill \end{array}$$

Set

$$G_{\epsilon ,n}\left(x\right)=max\left\{\right|K\left(x\right)(f\left(u_n\right)-f\left(u_0\right))\varphi ]|-\epsilon |u_n-u_0{|}^2-\epsilon |u_n-u_0{|}^p,0\}.$$

Then

$$0\le G_{\epsilon ,n}\left(x\right)\le C_{\epsilon }{\left|\varphi \right|}^2+2|u_0\left|\right|\varphi |+C_4{C}_{\epsilon }{\left|\varphi \right|}^p+C_5|u_0{|}^{p-1}\left|\varphi \right|\in L^1\left({\mathbb{R}}^3\right),$$

and $G_{\epsilon ,n}\left(x\right)\to 0$ a.e. on ${\mathbb{R}}^3$. By the Lebesgue dominated convergence theorem, we have

$${\int }_{{\mathbb{R}}^3}{G}_{\epsilon ,n}\left(x\right)dx\to 0 \mathrm{as} n\to \infty .$$

Therefore,

$$\begin{array}{cc}\hfill & \underset{n\to \infty }{lim\; sup}\left|{\int }_{{\mathbb{R}}^3}K\left(x\right)(f\left(u_n\right)-f\left(u_0\right))\varphi dx\right|\hfill \\ \hfill \le & \underset{n\to \infty }{lim\; sup}{\int }_{{\mathbb{R}}^3}{G}_{\epsilon ,n}\left(x\right)dx+\epsilon \underset{n\to \infty }{lim\; sup}{\int }_{{\mathbb{R}}^3}|u_n-u_0{|}^{2}dx+\epsilon \underset{n\to \infty }{lim\; sup}{\int }_{{\mathbb{R}}^3}{|u_n-u_0|}^{p}dx\hfill \\ \hfill \le & C_3\epsilon .\hfill \end{array}$$

By the arbitrariness of $\epsilon$, we have

$${\int }_{{\mathbb{R}}^3}K\left(x\right)(f\left(u_n\right)-f\left(u_0\right))\varphi dx\to 0.$$

Next we will verify that $u_n\to u_0$ in $L^{2_{\alpha }^{*}}\left({\mathbb{R}}^3\right)$. We show that there exists ${\lambda }_{*}>0$ such that ${\nu }_i=0$ and ${\nu }_{\infty }=0$ for $0<\lambda <{\lambda }_{*}$ and $i\in J$. We argue by contradiction. Assume that $i_0\in J$ exists such that ${\nu }_{i_0}>0$ or ${\nu }_{\infty }>0$, by Lemma 7, we have

$${\nu }_{i_0}\le {\left(S_{\alpha }^{-1}\mu \left(x_{i_0}\right)\right)}^{\frac{2_{\alpha }^{*}}{2}}.$$

(10)

Let $\phi \in C_0^{\infty }\left({\mathbb{R}}^3\right)$ satisfy $\phi \in (0,1)$, $\phi =1$ in $B(0,1)$ and $\phi =0$ in ${\mathbb{R}}^3\setminus B(0,2)$. For any $\epsilon >0$, we define ${\phi }_{\epsilon }\left(x\right)=\phi \left(\frac{x-x_{i_0}}{\epsilon }\right)$, where $i_0\in J$. Using (9), one has

$${\int }_{{\mathbb{R}}^3}{\left|u_n{\phi }_{\epsilon }\right|}^{2_{\alpha }^{*}}dx\le {\left(S_{\alpha }^{-1}{\int }_{{\mathbb{R}}^3}{\int }_{{\mathbb{R}}^3}\frac{|u_n\left(x\right){\phi }_{\epsilon }\left(x\right)-u_n\left(y\right){\phi }_{\epsilon }{\left(y\right)|}^2}{{|x-y|}^{N+2\alpha }}dxdy\right)}^{\frac{2_{\alpha }^{*}}{2}},$$

which means that

$${\int }_{{\mathbb{R}}^3}{\left|u_n{\phi }_{\epsilon }\right|}^{2_{\alpha }^{*}}dx\to {\int }_{{\mathbb{R}}^3}{\phi }_{\epsilon }^{2_{\alpha }^{*}}d\nu \mathrm{as} n\to \infty ,$$

(11)

and

$${\int }_{{\mathbb{R}}^3}{\phi }_{\epsilon }^{2_{\alpha }^{*}}d\nu \to \nu \left(\left\{x_{i_0}\right\}\right)={\nu }_{i_0}, \mathrm{as} \epsilon \to 0.$$

(12)

By (f3), we obtain

$$\begin{array}{cc}\hfill & \theta I_{\lambda }\left(u_n\right)-\langle I_{\lambda }^{\prime }\left(u_n\right),u_n\rangle \hfill \\ \hfill =& \frac{\theta }{2}\parallel u_n{\parallel }^2+\frac{\theta }{4}{\int }_{{\mathbb{R}}^3}{\varphi }_{u_n}^t{\left(u_n\right)}^{2}dx-\theta {\int }_{{\mathbb{R}}^3}K\left(x\right)F\left(u_n\right)dx-\frac{\lambda \theta }{2_{\alpha }^{*}}{\int }_{{\mathbb{R}}^3}{\left|u_n\right|}^{2_{\alpha }^{*}}dx\hfill \\ \hfill & -\parallel u_n{\parallel }^2-\frac{1}{4}{\int }_{{\mathbb{R}}^3}{\varphi }_{u_n}^t{\left(u_n\right)}^{2}dx+{\int }_{{\mathbb{R}}^3}K\left(x\right)f\left(u_n\right)u_{n}dx+\lambda {\int }_{{\mathbb{R}}^3}{\left|u_n\right|}^{2_{\alpha }^{*}}dx\hfill \\ \hfill =& \left(\frac{\theta -2}{2}\right){\parallel u_n\parallel }^2+\left(\frac{\theta -1}{4}\right){\int }_{{\mathbb{R}}^3}{\varphi }_{u_n}^t{\left(u_n\right)}^{2}dx+{\int }_{{\mathbb{R}}^3}K\left(x\right)[f\left(u_n\right)u_n-\theta F\left(u_n\right)]dx\hfill \\ \hfill & +\lambda \left(1-\frac{\theta }{2_{\alpha }^{*}}\right){\int }_{{\mathbb{R}}^3}{\left|u_n\right|}^{2_{\alpha }^{*}}dx\hfill \\ \hfill \ge & \lambda \left(1-\frac{\theta }{2_{\alpha }^{*}}\right){\int }_{{\mathbb{R}}^3}{\left|u_n\right|}^{2_{\alpha }^{*}}dx\hfill \\ \hfill \ge & \lambda \left(\frac{2_{\alpha }^{*}-\theta }{2_{\alpha }^{*}}\right){\int }_{{\mathbb{R}}^3}{\left|u_n{\phi }_{\epsilon }\right|}^{2_{\alpha }^{*}}dx.\hfill \end{array}$$

Let $n\to \infty$, then we have $\theta C_3\ge \lambda \left(\frac{2_{\alpha }^{*}-\theta }{2_{\alpha }^{*}}\right){\int }_{{\mathbb{R}}^3}{\phi }_{\epsilon }^{2_{\alpha }^{*}}dx$. By (11) and (12), we know that

$$2C_3\ge \lambda \left(\frac{2_{\alpha }^{*}-\theta }{2_{\alpha }^{*}}\right){\nu }_{i_0}.$$

(13)

Since $\left\{u_n\right\}$ is bounded in $H^{\alpha }\left({\mathbb{R}}^3\right)$, we know that $\left\{u_n{\phi }_{\epsilon }\right\}$ is also bounded in $H^{\alpha }\left({\mathbb{R}}^3\right)$. Thus

$$\langle I_{\lambda }^{\prime }\left(u_n\right),u_n{\phi }_{\epsilon }\rangle \to 0 \mathrm{as} n\to \infty .$$

which means that

$$\begin{array}{cc}\hfill & {\int }_{{\mathbb{R}}^3}{(-\Delta )}^{\frac{\alpha }{2}}{u}_n·{(-\Delta )}^{\frac{\alpha }{2}}\left(u_n{\phi }_{\epsilon }\right)dx\hfill \\ \hfill =& {\int }_{{\mathbb{R}}^3}\left[K\left(x\right)f\left(u_n\right)u_n+\lambda |u_n{|}^{2_{\alpha }^{*}}-V\left(x\right){\left|u_n\right|}^2\right]{\phi }_{\epsilon }dx+o\left(1\right)\hfill \end{array}$$

(14)

By (6), we have

$$tf\left(t\right)\le {\epsilon \left|t\right|}^2+C_{\epsilon }{\left|t\right|}^p.$$

Then

$$\begin{array}{cc}\hfill & {\int }_{{\mathbb{R}}^3}\left[K\left(x\right)f\left(u_n\right)u_n+\lambda |u_n{|}^{2_{\alpha }^{*}}-V\left(x\right){\left|u_n\right|}^2\right]{\phi }_{\epsilon }dx\hfill \\ \hfill \le & {\int }_{{\mathbb{R}}^3}\left[\epsilon K\left(x\right)|u_n{|}^2+C_{\epsilon }K\left(x\right)|u_n{|}^{2_{\alpha }^{*}}+\lambda {\left|u_n\right|}^{2_{\alpha }^{*}}\right]{\phi }_{\epsilon }dx.\hfill \end{array}$$

Since

$${\int }_{{\mathbb{R}}^3}K\left(x\right)|u_n{|}^2{\phi }_{\epsilon }dx={\int }_{B(x_i,2\epsilon )}K\left(x\right)|u_n{|}^2{\phi }_{\epsilon }dx\to {\int }_{B(x_i,2\epsilon )}K\left(x\right){\left|u_0\right|}^2{\phi }_{\epsilon }dx \mathrm{as} n\to \infty ,$$

and

$${\int }_{B(x_i,2\epsilon )}K\left(x\right){\left|u_0\right|}^2{\phi }_{\epsilon }dx\to 0 \mathrm{as} \epsilon \to 0,$$

then

$$\begin{array}{cc}\hfill & \underset{\epsilon \to 0}{lim\; sup}\underset{n\to \infty }{lim\; sup}\left[K\left(x\right)f\left(u_n\right)u_n+\lambda |u_n{|}^{2_{\alpha }^{*}}-V\left(x\right){\left|u_n\right|}^2\right]{\phi }_{\epsilon }dx\hfill \\ \hfill \le & (C_{\epsilon }+\lambda )\underset{\epsilon \to 0}{lim\; sup}\underset{n\to \infty }{lim\; sup}{\int }_{{\mathbb{R}}^3}{\left|u_n\right|}^{2_{\alpha }^{*}}{\phi }_{\epsilon }dx\hfill \\ \hfill \le & (C_{\epsilon }+\lambda )\underset{\epsilon \to 0}{lim\; sup}{\int }_{{\mathbb{R}}^3}{\phi }_{\epsilon }^{2_{\alpha }^{*}}dx\hfill \\ \hfill =& (C_{\epsilon }+\lambda ){\nu }_{i_0}.\hfill \end{array}$$

Choose $\lambda$ large enough to satisfy $\lambda >C_{\epsilon }$, then

$$\underset{\epsilon \to 0}{lim\; sup}\underset{n\to \infty }{lim\; sup}\left[K\left(x\right)f\left(u_n\right)u_n+\lambda |u_n{|}^{2_{\alpha }^{*}}-V\left(x\right){\left|u_n\right|}^2\right]{\phi }_{\epsilon }dx\le 2\lambda {\nu }_{i_0}.$$

(15)

By the proposition 3.6 in [23], we have

$${\int }_{{\mathbb{R}}^3}{\int }_{{\mathbb{R}}^3}\frac{{|u\left(x\right)-u\left(y\right)|}^2}{{|x-y|}^{N+2\alpha }}dxdy={\int }_{{\mathbb{R}}^3}{\left|{(-\Delta )}^{\frac{\alpha }{2}}u\left(x\right)\right|}^{2}dx.$$

(16)

Therefore, for any $v\in H^{\alpha }\left({\mathbb{R}}^3\right)$, we have

$${\int }_{{\mathbb{R}}^3}{\int }_{{\mathbb{R}}^3}\frac{\left(u\right(x)-u(y\left)\right)\left(v\right(x)-v(y\left)\right)}{{|x-y|}^{N+2\alpha }}dxdy={\int }_{{\mathbb{R}}^3}{(-\Delta )}^{\frac{\alpha }{2}}u\left(x\right)·{(-\Delta )}^{\frac{\alpha }{2}}v\left(x\right)dx.$$

(17)

By (17), we have

$$\begin{array}{cc}\hfill & {\int }_{{\mathbb{R}}^3}{(-\Delta )}^{\frac{\alpha }{2}}{u}_n·{(-\Delta )}^{\frac{\alpha }{2}}\left(u_n{\phi }_{\epsilon }\right)dx\hfill \\ \hfill =& {\int }_{{\mathbb{R}}^3}{\int }_{{\mathbb{R}}^3}\frac{(u_n\left(x\right)-u_n\left(y\right))(u_n\left(x\right){\phi }_{\epsilon }\left(x\right)-u_n\left(y\right){\phi }_{\epsilon }\left(y\right))}{{|x-y|}^{N+2\alpha }}dxdy\hfill \\ \hfill =& {\int }_{{\mathbb{R}}^3}{\int }_{{\mathbb{R}}^3}\frac{{(u_n\left(x\right)-u_n\left(y\right))}^2{\phi }_{\epsilon }\left(y\right)}{{|x-y|}^{N+2\alpha }}dxdy\hfill \\ \hfill & +{\int }_{{\mathbb{R}}^3}{\int }_{{\mathbb{R}}^3}\frac{(u_n\left(x\right)-u_n\left(y\right))({\phi }_{\epsilon }\left(x\right)-{\phi }_{\epsilon }\left(y\right))u_n\left(x\right)}{{|x-y|}^{N+2\alpha }}dxdy.\hfill \end{array}$$

It is easy to know that

$${\int }_{{\mathbb{R}}^3}{\int }_{{\mathbb{R}}^3}\frac{{(u_n\left(x\right)-u_n\left(y\right))}^2{\phi }_{\epsilon }\left(y\right)}{{|x-y|}^{N+2\alpha }}dxdy\to {\int }_{{\mathbb{R}}^3}{\phi }_{\epsilon }d\mu \mathrm{as} n\to \infty ,$$

and

$${\int }_{{\mathbb{R}}^3}{\phi }_{\epsilon }d\mu \to \mu \left(\left\{x_{i_0}\right\}\right) \mathrm{as} \epsilon \to 0.$$

By the Hölder inequality, one has

$$\begin{array}{cc}\hfill & {\int }_{{\mathbb{R}}^3}{\int }_{{\mathbb{R}}^3}\frac{(u_n\left(x\right)-u_n\left(y\right))({\phi }_{\epsilon }\left(x\right)-{\phi }_{\epsilon }\left(y\right))u_n\left(x\right)}{{|x-y|}^{N+2\alpha }}dxdy\hfill \\ \hfill \le & {\int }_{{\mathbb{R}}^3}{\int }_{{\mathbb{R}}^3}\frac{|u_n\left(x\right)-u_n\left(y\right)\left|\right|{\phi }_{\epsilon }\left(x\right)-{\phi }_{\epsilon }\left(y\right)\left|\right|u_n\left(x\right)|}{{|x-y|}^{N+2\alpha }}dxdy\hfill \\ \hfill \le & C{\left({\int }_{{\mathbb{R}}^3}{\int }_{{\mathbb{R}}^3}\frac{u_n^2\left(x\right){|{\phi }_{\epsilon }\left(x\right)-{\phi }_{\epsilon }\left(y\right)|}^2}{{|x-y|}^{N+2\alpha }}dxdy\right)}^{\frac{1}{2}}.\hfill \end{array}$$

By the following equality (see (3.7) in [27])

$$\underset{\epsilon \to 0}{lim\; sup}\underset{n\to \infty }{lim\; sup}{\int }_{{\mathbb{R}}^3}{\int }_{{\mathbb{R}}^3}\frac{u_n^2\left(x\right){|{\phi }_{\epsilon }\left(x\right)-{\phi }_{\epsilon }\left(y\right)|}^2}{{|x-y|}^{N+2\alpha }}dxdy=0,$$

(14) and (15), we obtain that

$$\mu \left(\left\{x_{i_0}\right\}\right)\le 2\lambda {\nu }_{i_0} \mathrm{for} \mathrm{any} i_0\in J.$$

From (10) and (13), one has

$$2C_3\ge \lambda \left(\frac{2_{\alpha }^{*}-\theta }{2_{\alpha }^{*}}\right){\left(\frac{{\lambda }^{-1}{S}_{\alpha }}{2}\right)}^{\frac{2_{\alpha }^{*}}{2_{\alpha }^{*}-2}}=\left(\frac{2_{\alpha }^{*}-\theta }{2_{\alpha }^{*}}\right){\left(\frac{S_{\alpha }}{2}\right)}^{\frac{N}{2\alpha }}{\lambda }^{\frac{2\alpha -N}{2\alpha }},$$

which implies

$$\lambda \ge \left(\frac{2_{\alpha }^{*}-\theta }{2C_3{2}_{\alpha }^{*}}\right){\left(\frac{S_{\alpha }}{2}\right)}^{\frac{2_{\alpha }^{*}}{2}}:={\lambda }_{*},$$

which contradicts $0<\lambda <{\lambda }_{*}$. Therefore, for any $i\in J,{\nu }_i=0$ and ${\nu }_{\infty }=0$. By Lemma 6, we know

$$\underset{n\to \infty }{lim\; sup}{\int }_{{\mathbb{R}}^3}|u_n{|}^{2_{\alpha }^{*}}dx={\int }_{{\mathbb{R}}^3}{\left|u_0\right|}^{2_{\alpha }^{*}}dx.$$

Since $|u_n-u_0{|}^{2_{\alpha }^{*}}\le 2^{2_{\alpha }^{*}}\left(\right|u_n{|}^{2_{\alpha }^{*}}+|u_0{|}^{2_{\alpha }^{*}})$, by Fatou’s Lemma, we obtain

$$\begin{array}{cc}\hfill {\int }_{{\mathbb{R}}^3}{2}^{2_{\alpha }^{*}+1}{\left|u_0\right|}^{2_{\alpha }^{*}}dx=& {\int }_{{\mathbb{R}}^3}\underset{n\to \infty }{lim\; inf}\left(2^{2_{\alpha }^{*}}|u_n{|}^{2_{\alpha }^{*}}+2^{2_{\alpha }^{*}}|u_0{|}^{2_{\alpha }^{*}}-{|u_n-u_0|}^{2_{\alpha }^{*}}\right)dx\hfill \\ \hfill \le & \underset{n\to \infty }{lim\; inf}{\int }_{{\mathbb{R}}^3}\left(2^{2_{\alpha }^{*}}|u_n{|}^{2_{\alpha }^{*}}+2^{2_{\alpha }^{*}}|u_0{|}^{2_{\alpha }^{*}}-{|u_n-u_0|}^{2_{\alpha }^{*}}\right)dx\hfill \\ \hfill & +2^{2_{\alpha }^{*}+1}{\int }_{{\mathbb{R}}^3}|u_0{|}^{2_{\alpha }^{*}}dx-\underset{n\to \infty }{lim\; sup}{\int }_{{\mathbb{R}}^3}{|u_n-u_0|}^{2_{\alpha }^{*}}dx,\hfill \end{array}$$

which implies that

$$\underset{n\to \infty }{lim\; sup}{\int }_{{\mathbb{R}}^3}{|u_n-u_0|}^{2_{\alpha }^{*}}dx=0,$$

then $u_n\to u_0$ in $L^{2_{\alpha }^{*}}\left({\mathbb{R}}^3\right)$ as $n\to \infty$.

Note that $I_{\lambda }^{\prime }\left(u_n\right)\to 0$, then

$$\begin{array}{cc}\hfill & \underset{n\to \infty }{lim\; sup}{\int }_{{\mathbb{R}}^3}{\left|{(-\Delta )}^{\frac{\alpha }{2}}{u}_n\right|}^{2}dx\hfill \\ \hfill =& \underset{n\to \infty }{lim\; sup}({\int }_{{\mathbb{R}}^3}K\left(x\right)f\left(u_n\right)u_{n}dx+\lambda {\int }_{{\mathbb{R}}^3}{\left|u_n\right|}^{2_{\alpha }^{*}}dx-{\int }_{{\mathbb{R}}^3}V\left(x\right)u_n^{2}dx\hfill \\ \hfill & -\frac{1}{4}{\int }_{{\mathbb{R}}^3}{\varphi }_{u_n}^t{\left(u_n\right)}^{2}dx)\hfill \\ \hfill \le & {\int }_{{\mathbb{R}}^3}K\left(x\right)f\left(u_0\right)u_{0}dx+\lambda {\int }_{{\mathbb{R}}^3}{\left|u_0\right|}^{2_{\alpha }^{*}}dx-{\int }_{{\mathbb{R}}^3}V\left(x\right)u_0^{2}dx-\frac{1}{4}{\int }_{{\mathbb{R}}^3}{\varphi }_{u_0}^t{\left(u_0\right)}^{2}dx\hfill \\ \hfill \le & \underset{n\to \infty }{lim\; inf}{\int }_{{\mathbb{R}}^3}{\left|{(-\Delta )}^{\frac{\alpha }{2}}{u}_n\right|}^{2}dx,\hfill \end{array}$$

which means that

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}{\int }_{{\mathbb{R}}^3}{|(-\Delta )}^{\frac{\alpha }{2}}{u}_n{|}^{2}dx={\int }_{{\mathbb{R}}^3}{\left|{(-\Delta )}^{\frac{\alpha }{2}}{u}_0\right|}^{2}dx.\end{array}$$

(18)

Thus,

$$\begin{array}{cc}\hfill & \underset{n\to \infty }{lim\; sup}{\int }_{{\mathbb{R}}^3}V\left(x\right)u_n^{2}dx\hfill \\ \hfill =& \underset{n\to \infty }{lim\; sup}({\int }_{{\mathbb{R}}^3}K\left(x\right)f\left(u_n\right)u_{n}dx+\lambda {\int }_{{\mathbb{R}}^3}|u_n{|}^{2_{\alpha }^{*}}dx-{\int }_{{\mathbb{R}}^3}{\left|{(-\Delta )}^{\frac{\alpha }{2}}{u}_n\right|}^{2}dx\hfill \\ \hfill & -\frac{1}{4}{\int }_{{\mathbb{R}}^3}{\varphi }_{u_n}^t{\left(u_n\right)}^{2}dx)\hfill \\ \hfill =& {\int }_{{\mathbb{R}}^3}K\left(x\right)f\left(u_0\right)u_{0}dx+\lambda {\int }_{{\mathbb{R}}^3}|u_0{|}^{2_{\alpha }^{*}}dx-{\int }_{{\mathbb{R}}^3}{\left|{(-\Delta )}^{\frac{\alpha }{2}}{u}_0\right|}^{2}dx-\frac{1}{4}{\int }_{{\mathbb{R}}^3}{\varphi }_{u_0}^t{\left(u_0\right)}^{2}dx\hfill \\ \hfill =& {\int }_{{\mathbb{R}}^3}V\left(x\right)u_0^{2}dx.\hfill \end{array}$$

Since $u_n\rightharpoonup u_0$ weakly in $H^{\alpha }\left({\mathbb{R}}^3\right)$, then by (18), we have

$$\underset{n\to \infty }{lim}{\int }_{{\mathbb{R}}^3}{\left|{(-\Delta )}^{\frac{\alpha }{2}}(u_n-u_0)\right|}^{2}dx+{\int }_{{\mathbb{R}}^3}V\left(x\right){(u_n-u_0)}^{2}dx=0,$$

which implies that $u_n\to u_0$ in $H^{\alpha }\left({\mathbb{R}}^3\right)$. □

Lemma 9.

Let Ω be every finite subspace of $H^{\alpha }\left({\mathbb{R}}^3\right)$. Then, for any $u\in \Omega$, it holds

$$I_{\lambda }\left(u\right)\to -\infty \mathrm{as} \parallel u\parallel \to \infty .$$

Proof. 

Arguing by contradiction, there exists $C_6>0$ such that $I\left(u_n\right)\ge -C_6$ for any $n\in \mathbb{N}$. Let $v_n:=\frac{u_n}{\parallel u_n\parallel }$, then $\parallel v_n\parallel =1$. Passing a subsequence, still denote by $v_n$, we assume that $v_n\rightharpoonup v$ in $H^{\alpha }\left({\mathbb{R}}^3\right)$, by Lemma 1, we have $v_n\to v$ in $L^r\left({\mathbb{R}}^3\right)$ and $v_n\to v$ a.e. on ${\mathbb{R}}^3$.

By condition (f3), we know $F\left(u_n\right)>0$ for every $u_n\in H^{\alpha }\left({\mathbb{R}}^3\right)$, then $F\left(u_n\right)\ge -C_7{u}_n^2$. Define ${\Omega }_n(0,r_1):=\{x\in {\mathbb{R}}^3:0<|u_n|<r_1\}$, where $r_1>0$ is a constant. For $u_n\in E$, we have

$$\begin{array}{cc}\hfill \frac{1}{\parallel u_n{\parallel }^4}{\int }_{{\Omega }_n(0,r_1)}K\left(x\right)F\left(u\right)dx& \ge -\frac{C_7}{\parallel u_n{\parallel }^4}{\int }_{{\Omega }_n(0,r_1)}K\left(x\right)u^{2}dx\hfill \\ \hfill & \ge -\frac{C_7{\parallel u_n\parallel }^2}{\parallel u_n{\parallel }^4}.\hfill \end{array}$$

(19)

Using the properties of the function ${\varphi }_u^s$ (Lemma 2.3-(2) in [18]), (19) and Fatou’s Lemma, we have

$$\begin{array}{cc}\hfill 0=& \underset{n\to \infty }{lim}\frac{I_{\lambda }\left(u_n\right)}{\parallel u_n{\parallel }^4}\hfill \\ \hfill =& \underset{n\to \infty }{lim}\left[\frac{1}{4\parallel u_n{\parallel }^4}{\int }_{{\mathbb{R}}^3}{\varphi }_u^t{u}^{2}dx-\frac{1}{\parallel u_n{\parallel }^4}{\int }_{{\mathbb{R}}^3}K\left(x\right)F\left(u\right)dx-\frac{\lambda }{2_{\alpha }^{*}{\parallel u_n\parallel }^4}{\int }_{{\mathbb{R}}^3}{\left|u\right|}^{2_{\alpha }^{*}}dx\right]\hfill \\ \hfill \le & C_8+\underset{n\to \infty }{lim}\left[-\frac{1}{\parallel u_n{\parallel }^4}{\int }_{{\Omega }_n(0,r_1)}K\left(x\right)F\left(u\right)dx-\frac{1}{\parallel u_n{\parallel }^4}{\int }_{{\Omega }_n(r_1,\infty )}K\left(x\right)F\left(u\right)dx\right]\hfill \\ \hfill \le & C_8+\underset{n\to \infty }{lim\; sup}\left[\frac{\parallel v_n{\parallel }_{L_K^2}^2}{\parallel u_n{\parallel }^2}-{\int }_{{\Omega }_n(r_1,\infty )}\frac{K\left(x\right)F\left(u\right)}{u_n^4}{v}_n^{4}dx\right]\hfill \\ \hfill \le & C_8-\underset{n\to \infty }{lim\; inf}\left[{\int }_{{\Omega }_n(r_1,\infty )}\frac{K\left(x\right)F\left(u\right)}{u_n^4}{v}_n^{4}dx\right]\hfill \\ \hfill \le & C_8-{\int }_{{\mathbb{R}}^3}\underset{n\to \infty }{lim\; inf}\frac{K\left(x\right)F\left(u\right)}{u_n^4}\left[{\chi }_{{\Omega }_n(r_1,\infty )}\right]v_n^{4}dx=-\infty ,\hfill \end{array}$$

where ${\chi }_{{\Omega }_n(r_1,\infty )}$ is the characteristic function of ${\Omega }_n(r_1,\infty )$. This is a contradiction and for any $u\in \Omega$, we obtain $I\left(u\right)\to -\infty \mathrm{as} \parallel u\parallel \to \infty .$ □

Proof of Theorem 1

Let ${\lambda }_1:=min\{{\lambda }_0,{\lambda }_{*}\}$, where ${\lambda }_0,{\lambda }_{*}$ are given in Lemma 3 and Lemma 8. By mountain pass theorem and Lemma 3, there exists a (PS) sequence $\left\{u_n\right\}\subset E$ for $I_{\lambda }$ in $H^{\alpha }\left({\mathbb{R}}^3\right)$ for any $0<\lambda <{\lambda }_1$. According to Lemma 8, there exists a subsequence of $\left\{u_n\right\}\subset H^{\alpha }\left({\mathbb{R}}^3\right)$, still denoted by $\left\{u_n\right\}$, and $\left\{u_{**}\right\}\subset E$ such that $u_n\to u_{**}$ in $H^{\alpha }\left({\mathbb{R}}^3\right)$ as $n\to 0$. Moreover, $I_{\lambda }^{\prime }\left(u_{**}\right)=0$ and $I_{\lambda }\left(u_{**}\right)\ge \beta >0$. Hence, $u_{**}$ is a nontrivial radially symmetric solution of problem (4). □

Proof of Theorem 2

From the definition of I, we have $I_{\lambda }\left(0\right)=0$. By the condition (f4), we easily know $I_{\lambda }$ is an even functional, i.e., $I_{\lambda }(-u)=I_{\lambda }\left(u\right)$. Hence, (A1) of Lemma 2 holds. According to Lemma 9, there exists $\xi =\xi \left(\Omega \right)>0$, for any $u\in \Omega$ with $\parallel u\parallel \ge \xi$, there is $I_{\lambda }\left(u\right)\le 0$. Therefore, (A3) of Lemma 2 is satisfied. By Lemma 3-(i), we know that (A2) of Lemma 2 is satisfied. Consequently, by Lemma 2, problem (4) has infinitely nontrivial radially symmetric solutions. □

3. Conclusions

In this paper, we study a class of fractional Schrödinger–Poisson systems with critical growth (4). Problem (4) comes from the interaction of a charge particle with an electromagnetic field in three-dimensional space. By using the principle of concentration compactness, we established a compactness result about functional $I_{\lambda }$. Applying the mountain pass theorem, a nontrivial radially symmetric solution was obtained. Infinitely nontrivial radially symmetric solutions are presented by the symmetric mountain pass lemma. Therefore, our results improve the related conclusions on this topic.