Normalized Ground States for Mixed Fractional Schrödinger Equations with Combined Local and Nonlocal Nonlinearities

Abstract

This paper studies the existence, regularity, and properties of normalized ground state solutions for the mixed fractional Schrödinger equations. For subcritical cases, we establish the boundedness and Sobolev regularity of solutions, derive Pohozaev identities, and prove the existence of radial, decreasing ground states, while showing nonexistence in the $L^2$-critical case. For $L^2$-supercritical exponents, we identify parameter regimes where ground states exist, characterized by a negative Lagrange multiplier. The analysis combines variational methods, scaling techniques, and the careful study of fibering maps to address challenges posed by competing nonlinearities and nonlocal interactions.

1. Introduction

In this paper, we investigate the existence and regularity of normalized ground state solutions for a class of mixed fractional Schrödinger equations with combined nonlinearities. Specifically, we consider the problem

$${(-\mathsf{\Delta })}^{s_1}u+{(-\mathsf{\Delta })}^{s_2}u=\lambda u+\nu (I_{\mu }{\ast |u|}^{1+\frac{\mu }{N}}{)|u|}^{\frac{\mu }{N}-1}u+\gamma {|u|}^{p-2}u,\mathrm{in}{\mathbb{R}}^N,$$

(1)

where $0<s_1<s_2<1,$ $N⩾3$, $\mu \in (0,N)$, $\gamma ,\nu >0$, $2<p<2_{s_1}^{\ast }=\frac{2N}{N-2s_1}$, and $I_{\mu }\left(x\right)={|x|}^{\mu -N}$. ${(-\mathsf{\Delta })}^s$ is the fractional Laplacian operator defined as

$${(-\mathsf{\Delta })}^{s}u\left(x\right)=C_{N,s}P.V.{\int }_{{\mathbb{R}}^N}\frac{u\left(x\right)-u\left(y\right)}{{|x-y|}^{N+2s}}dy,x\in {\mathbb{R}}^N,$$

where $C_{N,s}$ is a suitable normalization constant and $P.V.$ is a commonly used abbreviation for the principal value sense.

Equation (1) has emerged as a fundamental mathematical model with widespread applications in various scientific fields. Its significance stems from its ability to describe complex phenomena driven by the interplay of multiple nonlocal diffusion processes. These mixed fractional operators arise naturally in systems that combine different Lévy processes, from classical Brownian motion to long-range stochastic interactions, and have been widely used to model populations with heterogeneous diffusion mechanisms.

Due to these important applications, research on the elliptic problem of mixed fractional Laplacians is receiving increasing attention. Chergui–Gou–Hajaiej [1] studied the existence and dynamics of normalized solutions to the following equation with mixed fractional Laplacians

$${(-\mathsf{\Delta })}^{s_1}u+{(-\mathsf{\Delta })}^{s_2}u=\mu u+{|u|}^{p-2}u,x\in {\mathbb{R}}^N,$$

with $N\ge 1,p\ge 2+\frac{4s_1}{N}$. The threshold exponent $\tilde{p}\left(s\right)=2+\frac{4s}{N}$ is the $L^2$-critical exponent or $L^2$-critical exponent. Chergui [2] studied the existence of normalized solutions for equation with Hartree type nonlinearity. Additional advances, including the analysis of ground state solutions under the prescribed $L^2$-norm constraints, have been developed in [3,4], among others.

The study of normalized solutions (i.e., solutions with prescribed $L^2$-norm) for nonlinear Schrödinger-type equations has seen significant advances in the past decade, driven by both theoretical questions and applications to Bose–Einstein condensation and nonlinear optics. A critical challenge in this field lies in handling nonhomogeneous nonlinearities or competing interactions, where the interplay between different terms can lead to rich solution structures. Below, we highlight key contributions relevant to our work.

The seminal work of Bellazzini, Jeanjean, and Luo [5] investigated the existence and instability of standing waves for Schrödinger–Poisson equations with prescribed $L^2$-norm constraints.

This direction was further developed by Jeanjean, Luo, and Wang [6], who established a framework for proving the existence of multiple normalized solutions in quasi-linear Schrödinger equations. By combining mountain pass techniques with Pohozaev constraints, they demonstrated that certain energy functionals admit two critical points under $L^2$-constraints. Their methods have inspired subsequent studies on systems with nonlocal terms, including the Schrödinger–Poisson case. For more results on the ground state solutions for the nonlinear fractional Schrödinger equation with prescribed mass, we refer to [7,8,9,10,11,12,13,14] and the references therein.

The analysis of equations with combined nonlinearities was advanced by Soave [15], who systematically studied normalized ground states for the nonlinear Schrödinger equation with mixed power type terms:

$$-\mathsf{\Delta }u+\lambda u={\mu |u|}^{q-2}u+{|u|}^{p-2}u,$$

(2)

where $\mu \in \mathbb{R},2<q\le p<2^{\ast }$. By introducing a two-parameter variational approach, Soave characterized the existence regimes for ground states and uncovered threshold phenomena related to the $L^2$-critical exponent $\tilde{p}=2+4/N$. Notably, for $q=2+4/N$ ($L^2$-critical) and $p>q$, he proved the existence of a second solution with higher energy, complementing earlier results on purely subcritical or supercritical cases. In [16], Sovae extended (2) to the Sobolev critical case. The research was further extended to planar systems by Cingolani and Jeanjean [17], who addressed special challenges in two dimensions and developed refined compactness methods.

In [18], Yang considered the following equation:

$${(-\mathsf{\Delta })}^{s}u=\lambda u+\mu (I_{\alpha }{\ast |u|}^p{)|u|}^{p-2}u+{|u|}^{q-2}u,\mathrm{in}{\mathbb{R}}^N,$$

(3)

where $N⩾3,s\in (0,1),\alpha \in (0,N)$, $q\in (2+\frac{4s}{N},2_s^{\ast }]$, and $p\in [1+\frac{2s+\alpha }{N},\frac{N+\alpha }{N-2s})$. By applying a refined version of the minmax principle, he successfully established the existence of a critical point solution to Equation (3) when the relevant parameters satisfied certain structural conditions.

Most existing results (e.g., [6,15,16,18]) address classical Laplacians or single-order fractional operators. The case of ${(-\mathsf{\Delta })}^{s_1}+{(-\mathsf{\Delta })}^{s_2}$ ($s_1\ne s_2$) is largely unexplored. The combined effects of Choquard terms and power-type nonlinearities under $L^2$ constraints require new analytical tools, particularly when p approaches critical exponents.

Our first main result, Theorem 1, establishes key regularity properties of solutions, including $L^{\infty }$ boundedness, higher Sobolev regularity, and Pohozaev-type identities. These identities play a crucial role in analyzing the behavior of solutions and deriving necessary conditions for their existence.

Theorem 1.

Let $N⩾3$, $\frac{1}{2}<s_1+\frac{1}{2}<s_2<1$, $2<p<2_{s_2}^{\ast }$, $\nu ,\gamma >0$, $\mu \in (max\left\{1,N-4s_2\right\},N-2s_2)$, and $(u,\lambda )\in H^{s_2}\left({\mathbb{R}}^N\right)\times {\mathbb{R}}^{-}$ be a couple of solution for Equation (1). Then, we have the following results:

 (i) 

$u\in L^{\infty }\left({\mathbb{R}}^N\right)$.

 (ii) 

$u\in H^2\left({\mathbb{R}}^N\right)$.

 (iii) 

The following Pohozaev identities hold:

$$\begin{array}{cc}\hfill & \frac{N-2s_1}{2}{\parallel u\parallel }_{D^{s_1,2}\left({\mathbb{R}}^N\right)}^2+\frac{N-2s_2}{2}{\parallel u\parallel }_{D^{s_2,2}\left({\mathbb{R}}^N\right)}^2\hfill \\ \hfill =& \frac{\lambda N}{2}{\int }_{{\mathbb{R}}^N}{|u|}^2\mathrm{d}x+\frac{N\nu }{2}{\int }_{{\mathbb{R}}^N}(I_{\mu }{\ast |u|}^{1+\frac{\mu }{N}}{)|u|}^{1+\frac{\mu }{N}}\mathrm{d}x+\frac{N\gamma }{p}{\int }_{{\mathbb{R}}^N}{|u|}^p\mathrm{d}x,\hfill \end{array}$$

and

$$\begin{array}{c}\hfill s_1{\parallel u\parallel }_{D^{s_1,2}\left({\mathbb{R}}^N\right)}^2+s_2{\parallel u\parallel }_{D^{s_2,2}\left({\mathbb{R}}^N\right)}^2=\frac{\gamma N(p-2)}{2p}{\int }_{{\mathbb{R}}^N}{|u|}^p\mathrm{d}x.\end{array}$$

It is well known that the normalized solutions for Equation (1) are critical points of the energy functional

$$\begin{array}{cc}\hfill J_{\nu ,\gamma }\left(u\right):=& \frac{1}{2}{\int }_{{\mathbb{R}}^N}{|(-\mathsf{\Delta })}^{\frac{s_1}{2}}{u|}^2\mathrm{d}x+\frac{1}{2}{\int }_{{\mathbb{R}}^N}{|{(-\mathsf{\Delta })}^{\frac{s_2}{2}}u|}^2\mathrm{d}x\hfill \\ \hfill & -\frac{N\nu }{2(N+\mu )}{\int }_{{\mathbb{R}}^N}(I_{\mu }{\ast |u|}^{1+\frac{\mu }{N}}{)|u|}^{1+\frac{\mu }{N}}\mathrm{d}x-\frac{\gamma }{p}{\int }_{{\mathbb{R}}^N}{|u|}^p\mathrm{d}x,\hfill \end{array}$$

restricted to the (prescribed $L^2$-norm) constraint

$$\begin{array}{c}\hfill {\mathcal{M}}_c:=\left\{u\in H^s\left({\mathbb{R}}^N\right):{\int }_{{\mathbb{R}}^N}{|u|}^2\mathrm{d}x=c^2\right\}.\end{array}$$

We define

$$m_{\nu ,\gamma }\left(c\right):=\underset{u\in {\mathcal{M}}_c}{inf}{J}_{\nu ,\gamma }\left(u\right).$$

(4)

In Theorem 2, we prove the existence of a normalized ground state solution $\overline{u}$, which is radially symmetric and decreasing in $|x|$. Moreover, we provide an explicit upper bound for the associated Lagrange multiplier $\overline{\lambda }$, demonstrating its negativity.

Define

$$S_{\mu }=inf\left\{{\int }_{{\mathbb{R}}^N}{|u|}^2\mathrm{d}x:u\in L^2\left({\mathbb{R}}^N\right),{\int }_{{\mathbb{R}}^N}(I_{\mu }{\ast |u|}^{1+\frac{\mu }{N}}{)|u|}^{1+\frac{\mu }{N}}\mathrm{d}x=1\right\}>0.$$

Theorem 2.

Let $N⩾3$, $0<s_1<s_2<1,$ $2<p<\tilde{p}\left(s_2\right)$, $\mu \in (0,N)$, and $\nu ,\gamma >0$. Then,

$$m_{\nu ,\gamma }\left(c\right)<-\frac{N\nu }{2(N+\mu )}{S}_{\mu }^{-(1+\frac{\mu }{N})}{c}^{2(1+\frac{\mu }{N})}$$

and $m_{\nu ,\gamma }\left(c\right)$ is attained at a function $\overline{u}\in {\mathcal{M}}_c$ with the following properties:

 (i) 

$\overline{u}$ is radially symmetric and decreasing in $|x|$.

 (ii) 

$\overline{u}$ is the solution of (1) and the corresponding Lagrange multiplier

$$\overline{\lambda }<-\frac{N\nu }{N+\mu }{S}_{\mu }^{-(1+\mu /N)}{c}^{\frac{2\mu }{N}}.$$

Furthermore, $\overline{u}$ is a normalized ground state solution of (1).

Remark 1.

In the process of proving Theorem 2, we must face two fundamental difficulties as follows:

 (1) 

The competing effects between the local and nonlocal nonlinearities create new obstacles in the energy estimates and require a delicate analysis of the interaction terms.

 (2) 

The interaction between different fractional orders creates competing regularity requirements that complicate the analysis of critical points, particularly when combined with the nonlocal Hartree nonlinearity.

Remark 2.

For radially symmetric solutions, we can refer to [19].

For $u\in {\mathcal{M}}_c$ and $t\in \mathbb{R}$, let

$$(t★u)\left(x\right):=e^{\frac{N}{2}t}u\left(e^{t}x\right),$$

for a.e. $x\in \mathbb{R}$. This yields that $(t★u)\left(x\right)\in {\mathcal{M}}_c$.

We introduce the fibering map

$$\begin{array}{cc}\hfill {\mathsf{\Phi }}_u^{\nu ,\gamma }\left(t\right):=& J_{\nu ,\gamma }(t★u)=\frac{e^{2s_{1}t}}{2}{\parallel u\parallel }_{D^{s_1,2}\left({\mathbb{R}}^N\right)}^2+\frac{e^{2s_{2}t}}{2}{\parallel u\parallel }_{D^{s_2,2}\left({\mathbb{R}}^N\right)}^2\hfill \\ \hfill & -\frac{N\nu }{2(N+\mu )}{\int }_{{\mathbb{R}}^N}(I_{\mu }{\ast |u|}^{1+\frac{\mu }{N}}{)|u|}^{1+\frac{\mu }{N}}\mathrm{d}x-\frac{\gamma e^{\frac{N(p-2)}{2}t}}{p}{\parallel u\parallel }_{L^p\left({\mathbb{R}}^N\right)}^p.\hfill \end{array}$$

(5)

Firstly, we consider the case $p=\tilde{p}\left(s_2\right)$. For every $u\in {\mathcal{M}}_c$, by Lemma 2, we obtain that

$$\begin{array}{cc}\hfill {\left({\mathsf{\Phi }}_u^{\nu ,\gamma }\right)}^{\prime }\left(t\right)=& s_1{e}^{2s_{1}t}{\parallel u\parallel }_{D^{s_1,2}\left({\mathbb{R}}^N\right)}^2+s_2{e}^{2s_{2}t}{\parallel u\parallel }_{D^{s_2,2}\left({\mathbb{R}}^N\right)}^2-\frac{N\gamma (\tilde{p}\left(s_2\right)-2)e^{2s_{2}t}}{2\tilde{p}\left(s_2\right)}{\parallel u\parallel }_{L^p\left({\mathbb{R}}^N\right)}^p\hfill \\ \hfill & ⩾s_1{e}^{2s_{1}t}{\parallel u\parallel }_{D^{s_1,2}\left({\mathbb{R}}^N\right)}^2+s_2{e}^{2s_{2}t}\left(1-\frac{N\gamma c^{\frac{4s_2}{N}}}{N+2s_2}{C}_{N,s,p}\right){\parallel u\parallel }_{D^{s_2,2}\left({\mathbb{R}}^N\right)}^2.\hfill \end{array}$$

If

$$0<\gamma <\frac{N+2s_2}{N{c}^{\frac{4s_2}{N}}{C}_{N,s,p}},$$

(6)

we derive that ${\left({\mathsf{\Phi }}_u^{\nu ,\gamma }\right)}^{\prime }\left(t\right)>0$ for all $t>0$. This implies that ${\mathsf{\Phi }}_u^{\nu ,\gamma }\left(t\right)$ is strictly increasing, and we present the following non existence result.

Theorem 3.

Let $N⩾3$, $0<s_1<s_2<1,$ $\mu \in (0,N)$, $p=\tilde{p}\left(s_2\right),$ and (6) hold. Then, the functional $J_{\nu ,\gamma }\left(u\right)$ has no critical point on ${\mathcal{M}}_c$.

In what follows, we focus on the case $\tilde{p}\left(s_2\right)<p<2_{s_1}^{\ast }$. We use the notation

$${\mathcal{P}}_{c,\gamma }:=\{u\in {\mathcal{M}}_c:G_{\gamma }\left(u\right)=0\},$$

where

$$G_{\gamma }\left(u\right):=s_1{\parallel u\parallel }_{D^{s_1,2}\left({\mathbb{R}}^N\right)}^2+s_2{\parallel u\parallel }_{D^{s_2,2}\left({\mathbb{R}}^N\right)}^2-\frac{\gamma N(p-2)}{2p}{\parallel u\parallel }_{L^p\left({\mathbb{R}}^N\right)}^p.$$

It is obvious that all critical points of $J_{\nu ,\gamma }{|}_{{\mathcal{M}}_c}$ stay in ${\mathcal{P}}_{c,\gamma }$ according to the Pohozaev identity. From a similar discussion to ([20], Lemmas 2.12 and 2.13), we deduce that ${\mathcal{P}}_{c,\gamma }$ is a natural constraint.

Proposition 1.

Let $N⩾3$, $0<s_1<s_2<1,$ $\tilde{p}\left(s_2\right)<p<2_{s_1}^{\ast }$. Then, ${\mathcal{P}}_{c,\gamma }$ is a smooth manifold with codimension 2 in $H^{s_2}\left({\mathbb{R}}^N\right)$ and 1 in ${\mathcal{M}}_c$. Furthermore, if $u\in {\mathcal{P}}_{c,\gamma }$ is a critical point of $J_{\nu ,\gamma }{|}_{{\mathcal{P}}_{c,\gamma }}$, then u is a critical point of $J_{\nu ,\gamma }{|}_{{\mathcal{M}}_c}$.

We will show that $J_{\nu ,\gamma }{|}_{{\mathcal{P}}_{c,\gamma }}$ is bounded from below. The structure of ${\mathcal{P}}_{c,\gamma }$ is strongly influenced by the monotonicity and convexity properties of ${\mathsf{\Phi }}_u^{\nu ,\gamma }$. Through simple calculations, we see that

$$\begin{array}{cc}\hfill {\left({\mathsf{\Phi }}_u^{\nu ,\gamma }\right)}^{\prime }\left(t\right)=& s_1{e}^{2s_{1}t}{\parallel u\parallel }_{D^{s_1,2}\left({\mathbb{R}}^N\right)}^2+s_2{e}^{2s_{2}t}{\parallel u\parallel }_{D^{s_2,2}\left({\mathbb{R}}^N\right)}^2\hfill \\ \hfill & -\frac{\gamma N(p-2)e^{\frac{N(p-2)}{2}t}}{2p}{\parallel u\parallel }_{L^p\left({\mathbb{R}}^N\right)}^p=G_{\gamma }(t★u),\hfill \end{array}$$

(7)

which yields that $t\in \mathbb{R}$ is a critical point of ${\mathsf{\Phi }}_u^{\nu ,\gamma }$ if and only if $(t★u)\in {\mathcal{P}}_{c,\gamma }$. Moreover, ${\left({\mathsf{\Phi }}_u^{\nu ,\gamma }\right)}^{\prime }\left(0\right)=G_{\gamma }\left(u\right)$. Let us consider decomposing ${\mathcal{P}}_{c,\gamma }$ into disjoint union sets

$${\mathcal{P}}_{c,\gamma }={\mathcal{P}}_{c,\gamma }^{+}\cup {\mathcal{P}}_{c,\gamma }^0\cup {\mathcal{P}}_{c,\gamma }^{-},$$

where

$$\begin{array}{c}\hfill {\mathcal{P}}_{c,\gamma }^{+}=\left\{u\in {\mathcal{P}}_{c,\gamma }:{\left({\mathsf{\Phi }}_u^{\nu ,\gamma }\right)}^{″}\left(0\right)>0\right\}=\left\{u\in {\mathcal{M}}_c:{\left({\mathsf{\Phi }}_u^{\nu ,\gamma }\right)}^{\prime }\left(0\right)=0,{\left({\mathsf{\Phi }}_u^{\nu ,\gamma }\right)}^{″}\left(0\right)>0\right\},\\ \hfill {\mathcal{P}}_{c,\gamma }^0=\left\{u\in {\mathcal{P}}_{c,\gamma }:{\left({\mathsf{\Phi }}_u^{\nu ,\gamma }\right)}^{″}\left(0\right)=0\right\}=\left\{u\in {\mathcal{M}}_c:{\left({\mathsf{\Phi }}_u^{\nu ,\gamma }\right)}^{\prime }\left(0\right)=0,{\left({\mathsf{\Phi }}_u^{\nu ,\gamma }\right)}^{″}\left(0\right)=0\right\},\\ \hfill {\mathcal{P}}_{c,\gamma }^{-}=\left\{u\in {\mathcal{P}}_{c,\gamma }:{\left({\mathsf{\Phi }}_u^{\nu ,\gamma }\right)}^{″}\left(0\right)<0\right\}=\left\{u\in {\mathcal{M}}_c:{\left({\mathsf{\Phi }}_u^{\nu ,\gamma }\right)}^{\prime }\left(0\right)=0,{\left({\mathsf{\Phi }}_u^{\nu ,\gamma }\right)}^{″}\left(0\right)<0\right\}.\end{array}$$

Therefore, for $u\in {\mathcal{P}}_{c,\gamma }$, we derive that

$$\begin{array}{cc}\hfill {\left({\mathsf{\Phi }}_u^{\nu ,\gamma }\right)}^{″}\left(0\right)=& 2s_1^2{\parallel u\parallel }_{D^{s_1,2}\left({\mathbb{R}}^N\right)}^2+2s_2^2{\parallel u\parallel }_{D^{s_2,2}\left({\mathbb{R}}^N\right)}^2-\frac{N^2\gamma {(p-2)}^2}{4p}{\parallel u\parallel }_{L^p\left({\mathbb{R}}^N\right)}^p\hfill \\ \hfill ⩽& \frac{s_{2}N\gamma (p-2)}{p}{\parallel u\parallel }_{L^p\left({\mathbb{R}}^N\right)}^p-\frac{N^2\gamma {(p-2)}^2}{4p}{\parallel u\parallel }_{L^p\left({\mathbb{R}}^N\right)}^p\hfill \\ \hfill =& \frac{N\gamma (p-2)}{p}\left(s_2-\frac{N(p-2)}{4}\right){\parallel u\parallel }_{L^p\left({\mathbb{R}}^N\right)}^p.\hfill \end{array}$$

According to $\tilde{p}\left(s_2\right)<p<2_{s_1}^{\ast }$, we have

$${\left({\mathsf{\Phi }}_u^{\nu ,\gamma }\right)}^{″}\left(0\right)<0,\mathrm{for}\mathrm{all}u\in {\mathcal{P}}_{c,\gamma },$$

(8)

and

$${\mathcal{P}}_{c,\gamma }^{+}={\mathcal{P}}_{c,\gamma }^0=\varnothing .$$

Using the Pohozaev set ${\mathcal{P}}_{c,\gamma }$, based on the above discussion, we can obtain the next result.

Theorem 4.

Let $N⩾3$, $0<s_1<s_2<1,$ $\mu \in (0,N)$, $\tilde{p}\left(s_2\right)<p<2_{s_1}^{\ast }$, and $\nu >0$ and assume that $c,\gamma ,>0$ satisfy

$$\begin{array}{cc}\hfill & {\gamma }^{\frac{4s_1}{N(p-2)-4s_1}}{c}^{\frac{4s_1+2(p-2)N}{N(p-2)-4s_1}+\frac{2(N+\mu )}{N}}\hfill \\ \hfill <& \frac{[N(p-2)-4s_1](N+\mu )}{N^2(p-2)\nu }{\left(\frac{2s_{1}p}{N(p-2)C_{N,s_1,p}}\right)}^{\frac{4s_1}{N(p-2)-4s_1}}{S}_{\mu }^{1+\frac{\mu }{N}}.\hfill \end{array}$$

(9)

Then, there exists a constant ${\gamma }_0>0$ such that for any $\gamma >{\gamma }_0$, (1) admits a radial ground state solution $\overline{u}$ and the corresponding Lagrange multiplier $\overline{\lambda }<0$.

In this paper, we use the following notations:

• $L^p\left({\mathbb{R}}^N\right),1\le p\le +\infty$ denotes a Lebesgue space; the norm in $L^p\left({\mathbb{R}}^N\right)$ is denoted by ${\parallel ·\parallel }_{L^p\left({\mathbb{R}}^N\right)}$.
• $C,C_i$ denote (possibly different) any positive constant.

The rest of this paper is organized as follows. In Section 2, we present some preliminary results. We obtain the regularity of solutions to Equation (1) in Section 3 and prove Theorem 2 in Section 4. Section 5 is devoted to the proof of Theorem 4.

2. Preliminaries

In this section, we begin by summarizing key established results on fractional Sobolev spaces. For $0<s<1$, the space $D^{s,2}\left({\mathbb{R}}^N\right)$ is defined to be the completion of $C_0^{\infty }\left({\mathbb{R}}^N\right)$ with the Gagliaardo seminorm

$${\parallel u\parallel }_{D^{s,2}\left({\mathbb{R}}^N\right)}^2={\int \int }_{{\mathbb{R}}^{2N}}\frac{{|u\left(x\right)-u\left(y\right)|}^2}{{|x-y|}^{N+2s}}\mathrm{d}x\mathrm{d}y,$$

and the fractional Sobolev space is defined as

$$H^s\left({\mathbb{R}}^N\right):=\{u\in L^2\left({\mathbb{R}}^N\right):{\int \int }_{{\mathbb{R}}^{2N}}\frac{{|u\left(x\right)-u\left(y\right)|}^2}{{|x-y|}^{N+2s}}\mathrm{d}x\mathrm{d}y<+\infty \},$$

endowed with the natural norm

$${\parallel u\parallel }_{H^s\left({\mathbb{R}}^N\right)}^2={\int }_{{\mathbb{R}}^N}{|u|}^2\mathrm{d}x+{\int \int }_{{\mathbb{R}}^{2N}}\frac{{|u\left(x\right)-u\left(y\right)|}^2}{{|x-y|}^{N+2s}}\mathrm{d}x\mathrm{d}y.$$

For the convenience of readers, we introduce some preliminary results to prove our main theorems.

Firstly, we review the following compactness result, which can be found in [21].

Lemma 1

(See [21]). Let $N>2s$ and $0<s<1$. Then, there exists a constant $S=S(N,s)$ such that

$$S=\underset{u\in H^s\left({\mathbb{R}}^N\right)\setminus \left\{0\right\}}{inf}\frac{{\parallel (-\mathsf{\Delta })}^{\frac{s}{2}}{u\parallel }_{L^2\left({\mathbb{R}}^N\right)}^2}{{\parallel u\parallel }_{L^{2_s^{\ast }}\left({\mathbb{R}}^N\right)}^2}.$$

Furthermore, $H^s\left({\mathbb{R}}^N\right)$ is continuously embedded into $L^q\left({\mathbb{R}}^N\right)$ for all $2⩽q⩽2_s^{\ast }$ and compactly embedded into $L_{loc}^q\left({\mathbb{R}}^N\right)$ for all $2⩽q<2_s^{\ast }$.

Before describing more details, let us introduce the following fractional Gagliardo–Nirenberg–Sobolev inequality in [19].

Lemma 2.

For $u\in H^s\left({\mathbb{R}}^N\right)$ and $p\in \left(2,\frac{2N}{N-2s}\right)$. Then, there exists a constant $C_{N,s,p}=S^{-\frac{N(p-2)}{4s}}>0$ such that

$${\int }_{{\mathbb{R}}^N}{|u|}^p\mathrm{d}x⩽C_{N,s,p}{\left({\int }_{{\mathbb{R}}^N}{|{(-\mathsf{\Delta })}^{\frac{s}{2}}u|}^2\mathrm{d}x\right)}^{\frac{N(p-2)}{4s}}{\left({\int }_{{\mathbb{R}}^N}{|u|}^2\mathrm{d}x\right)}^{\frac{2sp-(p-2)N}{4s}}.$$

(10)

Proof. 

To interpolate between $L^2$ and $L^{2_s^{\ast }}$, express $\frac{1}{p}$ as

$$\frac{1}{p}=\frac{\theta }{2}+\frac{1-\theta }{2_s^{\ast }}=\frac{\theta }{2}+\frac{(1-\theta )(N-2s)}{2N},\theta \in (0,1),$$

where $2_s^{\ast }=\frac{2N}{N-2s}$. Then, we derive that

$$\frac{2N}{p}=N\theta +(1-\theta )(N-2s)=N\theta +N-2s-N\theta +2s\theta =N-2s+2s\theta ,$$

which yields that

$$\theta =\frac{\frac{2N}{p}-N+2s}{2s}=\frac{2N-Np+2sp}{2sp}=\frac{2N-(N-2s)p}{2ps}.$$

Then, $1-\theta =\frac{N(p-2)}{2ps}$ follows from $\frac{2N-(N-2s)p}{2ps}+\frac{N(p-2)}{2ps}=1$. Hölder’s inequality for $\frac{1}{p}=\frac{\theta }{2}+\frac{1-\theta }{2_s^{\ast }}$ gives

$$\begin{array}{cc}\hfill {\parallel u\parallel }_{L^p}={\left({\int }_{{\mathbb{R}}^N}{|u|}^{p}dx\right)}^{1/p}\le & {\left({\int }_{{\mathbb{R}}^N}{|u|}^{2·\frac{p\theta }{2}}dx\right)}^{\frac{1}{2}·\frac{2}{p\theta }}{\left({\int }_{{\mathbb{R}}^N}{|u|}^{2_s^{\ast }·\frac{p(1-\theta )}{2_s^{\ast }}}dx\right)}^{\frac{1}{2_s^{\ast }}·\frac{2_s^{\ast }}{p(1-\theta )}}\hfill \\ \hfill =& {\parallel u\parallel }_{L^2}^{\theta }{\parallel u\parallel }_{L^{2_s^{\ast }}}^{1-\theta },\hfill \end{array}$$

which implies that

$${\parallel u\parallel }_{L^p\left({\mathbb{R}}^N\right)}⩽{\parallel u\parallel }_{L^2\left({\mathbb{R}}^N\right)}^{\frac{2N-(N-2s)p}{2ps}}{\parallel u\parallel }_{L^{2_s^{\ast }}\left({\mathbb{R}}^N\right)}^{\frac{N(p-2)}{2ps}}.$$

Applying Lemma 1, we deduce that

$${\int }_{{\mathbb{R}}^N}{|u|}^p\mathrm{d}x⩽{\left({\int }_{{\mathbb{R}}^N}{|u|}^2\mathrm{d}x\right)}^{\frac{2sp-(p-2)N}{4s}}{\left(S^{-1}{\int }_{{\mathbb{R}}^N}{|{(-\mathsf{\Delta })}^{\frac{s}{2}}u|}^2\mathrm{d}x\right)}^{\frac{N(p-2)}{4s}}.$$

□

We also require the following Hardy–Littlewood–Sobolev inequality.

Lemma 3

([22]). Let $N\ge 2,\mu \in (0,N),r,t>1$ with $\begin{array}{c}\hfill \frac{1}{r}+\frac{1}{t}=1+\frac{\mu }{N}.\end{array}$ For any $f\in L^r\left({\mathbb{R}}^N\right)$ and $g\in L^t\left({\mathbb{R}}^N\right)$, one has

$$\begin{array}{c}\hfill \left|{\int \int }_{{\mathbb{R}}^{2N}}\frac{f\left(x\right)g\left(y\right)}{{|x-y|}^{N-\mu }}\mathrm{d}x\mathrm{d}y\right|\le {C(N,\mu ,r)\parallel f\parallel }_{L^r\left({\mathbb{R}}^N\right)}{\parallel g\parallel }_{L^t\left({\mathbb{R}}^N\right)}.\end{array}$$

Lemma 4

([22]). Let $1\le \beta \le \infty ,g\in L^{{\gamma }_1}\left({\mathbb{R}}^N\right)$ and $h\in L^{{\gamma }_2}\left({\mathbb{R}}^N\right)$. Then, there exists a constant $C>0$ such that

$${\parallel g\ast h\parallel }_{L^{\beta }\left({\mathbb{R}}^N\right)}\le {C\parallel g\parallel }_{L^{{\gamma }_1}\left({\mathbb{R}}^N\right)}{\parallel h\parallel }_{L^{{\gamma }_2}\left({\mathbb{R}}^N\right)},$$

where

$$\frac{1}{{\gamma }_1}+\frac{1}{{\gamma }_2}=1+\frac{1}{\beta }.$$

It follows from Lemma 4 that for any $v\in L^q\left({\mathbb{R}}^N\right),q\in (1,\frac{N}{\mu })$, $I_{\mu }\ast v\in L^{\frac{Nq}{N-\mu q}}\left({\mathbb{R}}^N\right)$, and

$$\parallel I_{\mu }{\ast v\parallel }_{L^{\frac{Nq}{N-\mu q}}\left({\mathbb{R}}^N\right)}\le C(\mu ,N,q){\parallel v\parallel }_{L^q\left({\mathbb{R}}^N\right)}.$$

(11)

From Lemma 3, for any $u\in H^s\left({\mathbb{R}}^N\right)$, if ${|u|}^p\in L^r\left({\mathbb{R}}^N\right)$ with $\frac{1}{r}=\frac{1}{2}(1+\frac{\mu }{N})$, thus

$${\int \int }_{{\mathbb{R}}^{2N}}\frac{{|u\left(x\right)|}^p{|u\left(y\right)|}^p}{{|x-y|}^{N-\mu }}\mathrm{d}x\mathrm{d}y$$

is well defined. Together with Lemma 4, we obtain that

$$1+\frac{\mu }{N}⩽p⩽\frac{N+\mu }{N-2s}.$$

$1+\frac{\mu }{N},\frac{N+\mu }{N-2s}$ are the Hardy–Littlewood–Sobolev lower and upper critical exponent, respectively. Particularly, for any $u\in H^s\left({\mathbb{R}}^N\right)$,

$${\int }_{{\mathbb{R}}^N}(I_{\mu }{\ast |u|}^{1+\frac{\mu }{N}}{)|u|}^{1+\frac{\mu }{N}}\mathrm{d}x⩽S_{\mu }^{-(1+\frac{\mu }{N})}{\left({\int }_{{\mathbb{R}}^N}{|u|}^2\mathrm{d}x\right)}^{\frac{N+\mu }{N}},$$

(12)

where

$$S_{\mu }=inf\left\{{\int }_{{\mathbb{R}}^N}{|u|}^2\mathrm{d}x:u\in L^2\left({\mathbb{R}}^N\right),{\int }_{{\mathbb{R}}^N}(I_{\mu }{\ast |u|}^{1+\frac{\mu }{N}}{)|u|}^{1+\frac{\mu }{N}}\mathrm{d}x=1\right\}>0.$$

(13)

It follows from [22] (Theorem 4.3) that $S_{\mu }$ is attained by

$$u\left(x\right)=U_{ϵ}\left(x\right):=C{\left(\frac{ϵ}{{ϵ}^2+{|x-y|}^2}\right)}^{\frac{N}{2}},$$

(14)

for some $C\in \mathbb{R},ϵ>0$ and $y\in {\mathbb{R}}^N$.

The next two lemmas are useful in proving the splitting property of the energy functional.

Lemma 5

([23], Lemma 2.4). Let $N\in \mathbb{N}$, $\mu \in (0,N)$, $q\in [1,\frac{2N}{N+\mu })$, and $\left\{u_n\right\}$ be a bounded sequence in $L^{\frac{2Nq}{N+\mu }}\left({\mathbb{R}}^N\right)$. If $u_n\to u$ a.e. on ${\mathbb{R}}^N$ as $n\to \infty$, then

$$\underset{n\to \infty }{lim}\left[{\int }_{{\mathbb{R}}^N}(I_{\mu }\ast |u_n{|}^q)|u_n{|}^q\mathrm{d}x-{\int }_{{\mathbb{R}}^N}(I_{\mu }\ast |u_n{-u|}^q)|u_n{-u|}^q\mathrm{d}x\right]={\int }_{{\mathbb{R}}^N}(I_{\mu }{\ast |u|}^q{)|u|}^q\mathrm{d}x.$$

The next result is a splitting property of the nonlocal energy functional for fractional Choquard equation in ${\mathbb{R}}^N$ with purely power.

Lemma 6

([24], Lemma 2.7). Let $q\in [1+\frac{\mu }{N},\frac{N+\mu }{N-2s}]$ and $r\in [2,2_s^{\ast }]$, and $\left\{u_n\right\}\subset H^s\left({\mathbb{R}}^N\right)$ such that $u_n\rightharpoonup u$ in $H^s\left({\mathbb{R}}^N\right)$. Then, for all $v\in H^s\left({\mathbb{R}}^N\right)$,

$${\int }_{{\mathbb{R}}^N}(I_{\mu }\ast |u_n{|}^q)|u_n{|}^{q-2}{u}_{n}v\mathrm{d}x\to {\int }_{{\mathbb{R}}^N}(I_{\mu }{\ast |u|}^q{)|u|}^{q-2}uv\mathrm{d}x,$$

and

$${\int }_{{\mathbb{R}}^N}|u_n{|}^{r-2}{u}_{n}v\mathrm{d}x\to {\int }_{{\mathbb{R}}^N}{|u|}^{r-2}uv\mathrm{d}x$$

as $n\to \infty$.

3. The Nonlocal Brézis-Kato’s Type Regularity Estimate

In this section, we now study the regularity of solutions to Equation (1). We shall restrict $\lambda <0$, and verify it in the proof of Theorem 2. We present a preliminary result, which is crucial for the subsequent proof.

Lemma 7.

Let $N⩾3$ and $u\in H^{s_2}\left({\mathbb{R}}^N\right)$ be a solution of

$${(-\mathsf{\Delta })}^{s_1}u+{(-\mathsf{\Delta })}^{s_2}u=g(x,u),in{\mathbb{R}}^N,$$

(15)

where $g:{\mathbb{R}}^N\times \mathbb{R}\to \mathbb{R}$ is a Carathéodory function such that

$$|g(x,t)|⩽C(|t|+|t{|}^{p-1}),$$

(16)

for a.e. $x\in {\mathbb{R}}^N$ and for all $t\in \mathbb{R}$, for some $p\in [2,2_{s_2}^{\ast }]$ and $C>0$. Then, $u\in L^q\left({\mathbb{R}}^N\right)$ for $q\in \left[2,\infty \right]$.

Proof. 

Let $u\in H^{s_2}\left({\mathbb{R}}^N\right)$ be a solution to Equation (15). For each $T>2$, we define

$$\begin{array}{c}\hfill u_T\left(x\right)=\left\{\begin{array}{cc}T,\hfill & \mathrm{if}u\left(x\right)>T,\hfill \\ u\left(x\right),\hfill & \mathrm{if}-T⩽u\left(x\right)⩽T,\hfill \\ -T,\hfill & \mathrm{if}u\left(x\right)<-T.\hfill \end{array}\right.\end{array}$$

Then, ${\tilde{u}}_T=u{u}_T^{2(b-1)}\in H^{s_2}\left({\mathbb{R}}^N\right)$ for $b>1$.

Hence, we derive that

$$\begin{array}{cc}\hfill & {\int }_{{\mathbb{R}}^N}{\int }_{{\mathbb{R}}^N}\frac{(u\left(x\right)-u\left(y\right))({\tilde{u}}_T\left(x\right)-{\tilde{u}}_T\left(y\right))}{{|x-y|}^{N+2s_1}}\mathrm{d}x\mathrm{d}y\hfill \\ \hfill & +{\int }_{{\mathbb{R}}^N}{\int }_{{\mathbb{R}}^N}\frac{(u\left(x\right)-u\left(y\right))({\tilde{u}}_T\left(x\right)-{\tilde{u}}_T\left(y\right))}{{|x-y|}^{N+2s_2}}\mathrm{d}x\mathrm{d}y\hfill \\ \hfill =& {\int }_{{\mathbb{R}}^N}g(x,u){\tilde{u}}_T\mathrm{d}x.\hfill \end{array}$$

(17)

Define

$$\begin{array}{c}\hfill \mathsf{\Lambda }:=\frac{{|t|}^2}{2}\mathrm{and}Y\left(t\right):={\int }_0^t{\left({\tilde{u}}_T^{\prime }\left(\tau \right)\right)}^{\frac{1}{2}}\mathrm{d}\tau ,\end{array}$$

for $t>0$. Since ${\tilde{u}}_T$ is an increasing function, we have that

$$\begin{array}{c}\hfill (\alpha -\beta )({\tilde{u}}_T\left(\alpha \right)-{\tilde{u}}_T\left(\beta \right))⩾0\mathrm{for} \mathrm{any}\alpha ,\beta ⩾0.\end{array}$$

Using Jensen’s inequality, it is sufficient to show that

$$\begin{array}{c}\hfill {\mathsf{\Lambda }}^{\prime }(\alpha -\beta )({\tilde{u}}_T\left(\alpha \right)-{\tilde{u}}_T\left(\beta \right))⩾{|Y\left(\alpha \right)-Y\left(\beta \right)|}^2\mathrm{for} \mathrm{any}\alpha ,\beta ⩾0,\end{array}$$

which yields that

$$\begin{array}{c}\hfill {|Y\left(u\left(x\right)\right)-Y\left(u\left(y\right)\right)|}^2⩽(u\left(x\right)-u\left(y\right))(\left(u{u}_T^{2(b-1)}\right)\left(x\right)-\left(u{u}_T^{2(b-1)}\right)\left(y\right)).\end{array}$$

Noting that $Y\left(u\right)⩾\frac{1}{b}u{u}_T^{b-1}$, by Lemma 1, we obtain

$$\begin{array}{cc}\hfill & {\int }_{{\mathbb{R}}^N}{\int }_{{\mathbb{R}}^N}\frac{(u\left(x\right)-u\left(y\right))({\tilde{u}}_T\left(x\right)-{\tilde{u}}_T\left(y\right))}{{|x-y|}^{N+2s_1}}\mathrm{d}x\mathrm{d}y\hfill \\ \hfill & +{\int }_{{\mathbb{R}}^N}{\int }_{{\mathbb{R}}^N}\frac{(u\left(x\right)-u\left(y\right))({\tilde{u}}_T\left(x\right)-{\tilde{u}}_T\left(y\right))}{{|x-y|}^{N+2s_2}}\mathrm{d}x\mathrm{d}y\hfill \\ \hfill ⩾& {\parallel Y\left(u\right)\parallel }_{D^{s_1,2}\left({\mathbb{R}}^N\right)}^2+{\parallel Y\left(u\right)\parallel }_{D^{s_2,2}\left({\mathbb{R}}^N\right)}^2\hfill \\ \hfill ⩾& {C\parallel Y\left(u\right)\parallel }_{L^{2_{s_2}^{\ast }}\left({\mathbb{R}}^N\right)}^2⩾\frac{C}{b^2}{\parallel u{u}_T^{b-1}\parallel }_{L^{2_{s_2}^{\ast }}\left({\mathbb{R}}^N\right)}^2.\hfill \end{array}$$

(18)

By virtue of (16)–(18), we have

$$\begin{array}{c}\hfill \parallel u{u}_T^{b-1}{\parallel }_{L^{2_{s_2}^{\ast }}\left({\mathbb{R}}^N\right)}^2⩽C{\lambda }^2\left[{\int }_{{\mathbb{R}}^N}{|u|}^2|u_T{|}^{2(b-1)}\mathrm{d}x+{\int }_{{\mathbb{R}}^N}{|u|}^p{|u_T|}^{2(b-1)}\mathrm{d}x\right].\end{array}$$

(19)

We notice that there exists $C>0$ and a function $H⩾0$ such that

$${|u|}^2|u_T{|}^{2(b-1)}+{|u|}^p|u_T{|}^{2(b-1)}⩽{(C+H)|u|}^2{|u_T|}^{2(b-1)},$$

(20)

where $H\in L^{\frac{N}{2s_2}}\left({\mathbb{R}}^N\right)$ and is independent of $T,b$. Indeed, we note that

$${|u|}^2|u_T{|}^{2(b-1)}+{|u|}^p|u_T{|}^{2(b-1)}={|u|}^2|u_T{|}^{2(b-1)}+{|u|}^{p-2}{u}^2{|u_T|}^{2(b-1)}.$$

Furthermore,

$${|u|}^{p-2}⩽1+H,$$

for some $H\in L^{\frac{N}{2s_2}}\left({\mathbb{R}}^N\right)$. In fact,

$${|u|}^{p-2}={\chi }_{\{0⩽|u|⩽1\}}{|u|}^{p-2}+{\chi }_{\{|u|>1\}}{|u|}^{p-2}⩽1+{\chi }_{\{|u|>1\}}{|u|}^{p-2},$$

and if $(p-2)\frac{N}{2s_2}<2$, thus,

$${\int }_{{\mathbb{R}}^N}{\chi }_{\{|u|>1\}}{|u|}^{(p-2)\frac{N}{2s_2}}\mathrm{d}x⩽{\int }_{{\mathbb{R}}^N}{\chi }_{\{|u|>1\}}{|u|}^2\mathrm{d}x⩽{\int }_{{\mathbb{R}}^N}{|u|}^2\mathrm{d}x<+\infty ,$$

if $(p-2)\frac{N}{2s_2}⩾2$, we conclude that $(p-2)\frac{N}{2s_2}\in [2,2_{s_2}^{\ast }]$. Combining (19) and (20), we derive that

$$\parallel u{u}_T^{b-1}{\parallel }_{L^{2_{s_2}^{\ast }}\left({\mathbb{R}}^N\right)}^2⩽C{b}^2\left[{\int }_{{\mathbb{R}}^N}{(C+H)|u|}^2{|u_T|}^{2(b-1)}\mathrm{d}x\right].$$

Taking $T\to \infty$, we obtain that

$$\parallel u^b{\parallel }_{L^{2_{s_2}^{\ast }}\left({\mathbb{R}}^N\right)}^2⩽C{b}^2\left[{\int }_{{\mathbb{R}}^N}{|u|}^{2b}\mathrm{d}x+{\int }_{{\mathbb{R}}^N}H{|u|}^{2b}\mathrm{d}x\right].$$

(21)

Choose $D>0$ and let $B_1=\{H⩽D\}$ and $B_2=\{H>D\}$. Therefore,

$${\int }_{{\mathbb{R}}^N}{H|u|}^{2b}\mathrm{d}x⩽D\parallel u^b{\parallel }_{L^2\left({\mathbb{R}}^N\right)}^2+\epsilon \left(D\right){\parallel u^b\parallel }_{L^{2_{s_2}^{\ast }}\left({\mathbb{R}}^N\right)}^2,$$

(22)

where

$$\epsilon \left(D\right)={\left({\int }_{B_2}{H}^{N/2s_2}\mathrm{d}x\right)}^{\frac{2s_2}{N}}\to 0,asD\to +\infty .$$

Taking into account of (21) and (22), we obtain that

$$\parallel u^b{\parallel }_{L^{2_{s_2}^{\ast }}\left({\mathbb{R}}^N\right)}^2⩽b^2(C+D)\parallel u^b{\parallel }_{L^2\left({\mathbb{R}}^N\right)}^2+b^2\epsilon \left(D\right){\parallel u^b\parallel }_{L^{2_{s_2}^{\ast }}\left({\mathbb{R}}^N\right)}^2.$$

(23)

Letting $D>0$ enough such that $b^2\epsilon \left(D\right)<\frac{1}{2}$, and applying (23), we conclude that

$$\parallel u^b{\parallel }_{L^{2_{s_2}^{\ast }}\left({\mathbb{R}}^N\right)}^2⩽2b^2(C+D){\parallel u^b\parallel }_{L^2\left({\mathbb{R}}^N\right)}^2.$$

(24)

Since $u\in L^{2_{s_2}^{\ast }}\left({\mathbb{R}}^N\right)$, taking $b_1=\frac{N}{N-2s_2}$ in (24), we deduce that $u\in L^{\frac{2N^2}{{(N-2s_2)}^2}}\left({\mathbb{R}}^N\right)$. Using (24) again, after k iterations, we obtain that $u\in L^{\frac{2N^k}{{(N-2s_2)}^k}}\left({\mathbb{R}}^N\right)$, that is, $u\in L^q\left({\mathbb{R}}^N\right)$ for $q\in \left[2,\infty \right)$.

Next, we show that $u\in L^{\infty }\left({\mathbb{R}}^N\right)$. From $u\in L^q\left({\mathbb{R}}^N\right)$ for $q\in \left[2,\infty \right)$, we observe that $H\in L^{\frac{N}{s_2}}\left({\mathbb{R}}^N\right)$. By Hölder’s inequality and Young;s inequality, we conclude that for all $\vartheta >0$,

$$\begin{array}{cc}\hfill {\int }_{{\mathbb{R}}^N}H{|u|}^{2b}\mathrm{d}x⩽& {\parallel H\parallel }_{L^{N/s_2}\left({\mathbb{R}}^N\right)}\parallel u^b{\parallel }_{L^2\left({\mathbb{R}}^N\right)}{\parallel u^b\parallel }_{L^{2_{s_2}^{\ast }}\left({\mathbb{R}}^N\right)}\hfill \\ \hfill ⩽& {\parallel H\parallel }_{L^{N/s_2}\left({\mathbb{R}}^N\right)}\left(\vartheta \parallel u^b{\parallel }_{L^2\left({\mathbb{R}}^N\right)}^2+\frac{1}{\vartheta }{\parallel u^b\parallel }_{L^{2_{s_2}^{\ast }}\left({\mathbb{R}}^N\right)}^2\right).\hfill \end{array}$$

Together with (21), we conclude that

$$\parallel u^b{\parallel }_{L^{2_{s_2}^{\ast }}\left({\mathbb{R}}^N\right)}^2⩽b^2{(C+\parallel H\parallel }_{L^{N/s_2}\left({\mathbb{R}}^N\right)}\vartheta )\parallel u^b{\parallel }_{L^2\left({\mathbb{R}}^N\right)}^2+\frac{C{b}^2{\parallel H\parallel }_{L^{N/s_2}\left({\mathbb{R}}^N\right)}}{\vartheta }{\parallel u^b\parallel }_{L^{2_{s_2}^{\ast }}\left({\mathbb{R}}^N\right)}^2.$$

Taking $\vartheta >0$ such that

$$\frac{C{b}^2{\parallel H\parallel }_{L^{N/s_2}\left({\mathbb{R}}^N\right)}}{\vartheta }=\frac{1}{2},$$

we dereive that

$$\parallel u^b{\parallel }_{L^{2_{s_2}^{\ast }}\left({\mathbb{R}}^N\right)}^2⩽2b^2{(C+\parallel H\parallel }_{L^{N/s_2}\left({\mathbb{R}}^N\right)}\vartheta )\parallel u^b{\parallel }_{L^2\left({\mathbb{R}}^N\right)}^2=K_b{\parallel u^b\parallel }_{L^2\left({\mathbb{R}}^N\right)}^2.$$

Note that there exists some $K_0>0$ independent of b such that

$$K_b⩽C{b}^4⩽K_0^2{e}^{2\sqrt{b}},$$

which yields that

$${\parallel u\parallel }_{L^{2_{s_2}^{\ast }b}\left({\mathbb{R}}^N\right)}⩽K_0^{\frac{1}{b}}{e}^{\frac{1}{\sqrt{b}}}{\parallel u\parallel }_{L^{2b}\left({\mathbb{R}}^N\right)}.$$

Iterating this relation and taking

$$b_0=1,b_{n+1}=\frac{2_{s_2}^{\ast }}{2}{b}_n,$$

we obtain that

$${\parallel u\parallel }_{L^{2_{s_2}^{\ast }{b}_n}\left({\mathbb{R}}^N\right)}⩽K_0^{{\sum }_{i=0}^n\frac{1}{b_i}}{e}^{{\sum }_{i=0}^n\frac{1}{\sqrt{b_i}}}{\parallel u\parallel }_{L^2\left({\mathbb{R}}^N\right)}.$$

Due to $b_n={\left(\frac{N}{N-2s_2}\right)}^n$, we observe that

$$\sum _{i=0}^n\frac{1}{b_i}<\infty ,\sum _{i=0}^n\frac{1}{\sqrt{b_i}}<\infty ,$$

which implies that

$${\parallel u\parallel }_{L^{\infty }\left({\mathbb{R}}^N\right)}=\underset{n\to \infty }{lim}{\parallel u\parallel }_{L^{2_{s_2}^{\ast }{b}_n}\left({\mathbb{R}}^N\right)}<\infty .$$

□

We study the $L^2$ estimate of ${(-\mathsf{\Delta })}^{s_2}u$.

Lemma 8.

Let $N⩾3$, $\frac{1}{2}<s_1+\frac{1}{2}<s_2<1$ and $u\in H^{s_2}\left({\mathbb{R}}^N\right)\cap L^{\infty }\left({\mathbb{R}}^N\right)$. Then, there exists $C>0$ (independent of u) such that

$$\begin{array}{c}\hfill {\parallel (-\mathsf{\Delta })}^{s_1}{u\parallel }_{L^2\left({\mathbb{R}}^N\right)}⩽C{\parallel u\parallel }_{H^{s_2}\left({\mathbb{R}}^N\right)}.\end{array}$$

Proof. 

We split the term ${(-\mathsf{\Delta })}^{s_1}u$ into two terms as follows:

$$\begin{array}{cc}\hfill {(-\mathsf{\Delta })}^{s_1}u={\int }_{\{|x-y|⩽1\}}\frac{u\left(x\right)-u\left(y\right)}{{|x-y|}^{N+2s_1}}\mathrm{d}y+{\int }_{\{|x-y|>1\}}\frac{u\left(x\right)-u\left(y\right)}{{|x-y|}^{N+2s_1}}\mathrm{d}y:=& K_1+K_2,\hfill \end{array}$$

and then

$$\begin{array}{c}\hfill {\parallel (-\mathsf{\Delta })}^{s_1}{u\parallel }_{L^2\left({\mathbb{R}}^N\right)}^2={\int }_{{\mathbb{R}}^N}{|K_1+K_2|}^2\mathrm{d}x⩽2\left({\int }_{{\mathbb{R}}^N}|K_1{|}^2\mathrm{d}x+{\int }_{{\mathbb{R}}^N}{|K_2|}^2\mathrm{d}x\right).\end{array}$$

From $\frac{1}{2}<s_1+\frac{1}{2}<s_2<1$, one has

$$\begin{array}{c}\hfill 2s_1+1<2s_2<s_2+1,\end{array}$$

which gives $s_2>2s_1$. Then, taking $z=x-y$ and applying Hölder’s inequality, we have

$$\begin{array}{cc}\hfill \parallel K_1{\parallel }_{L^2\left({\mathbb{R}}^N\right)}⩽& {\left[{\int }_{{\mathbb{R}}^N}{\left({\int }_{|z|⩽1}\frac{|u(x)-u(x-z)|}{{|z|}^{N+2s_1}}\mathrm{d}z\right)}^2\mathrm{d}x\right]}^{\frac{1}{2}}\hfill \\ \hfill =& {\left[{\int }_{{\mathbb{R}}^N}{\left({\int }_{|z|⩽1}\frac{|u(x)-u(x-z)|}{{|z|}^{\frac{N}{2}+s_2}}\frac{1}{{|z|}^{\frac{N}{2}+2s_1-s_2}}\mathrm{d}z\right)}^2\mathrm{d}x\right]}^{\frac{1}{2}}\hfill \\ \hfill ⩽& {\left[{\int }_{{\mathbb{R}}^N}\left({\int }_{|z|⩽1}\frac{{|u\left(x\right)-u(x-z)|}^2}{{|z|}^{N+2s_2}}\mathrm{d}z\right)\left({\int }_{|z|⩽1}\frac{1}{{|z|}^{N+4s_1-2s_2}}\mathrm{d}z\right)\mathrm{d}x\right]}^{\frac{1}{2}}\hfill \\ \hfill =& {\left[{\int }_{{\mathbb{R}}^N}{\int }_{|z|⩽1}\frac{{|u\left(x\right)-u(x-z)|}^2}{{|z|}^{N+2s_2}}\mathrm{d}z\mathrm{d}x\right]}^{\frac{1}{2}}{\left[{\int }_{|z|⩽1}\frac{1}{{|z|}^{N+4s_1-2s_2}}\mathrm{d}z\right]}^{\frac{1}{2}}\hfill \\ \hfill ⩽& {\parallel u\parallel }_{D^{s_2,2}\left({\mathbb{R}}^N\right)}{\left[{\int }_{|z|⩽1}\frac{1}{{|z|}^{N+4s_1-2s_2}}\mathrm{d}z\right]}^{\frac{1}{2}}\hfill \\ \hfill ⩽& {C\parallel u\parallel }_{H^{s_2}\left({\mathbb{R}}^N\right)}{\left[{\int }_0^1\frac{{\rho }^{N-1}}{{\rho }^{N+4s_1-2s_2}}\mathrm{d}\rho \right]}^{\frac{1}{2}}\hfill \\ \hfill =& {C\parallel u\parallel }_{H^{s_2}\left({\mathbb{R}}^N\right)}{\left[{\int }_0^1\frac{1}{{\rho }^{1+4s_1-2s_2}}\mathrm{d}\rho \right]}^{\frac{1}{2}}\hfill \\ \hfill ⩽& {C\parallel u\parallel }_{H^{s_2}\left({\mathbb{R}}^N\right)},\hfill \end{array}$$

where $(4s_1-2s_2)+1<1$. Then,

$$\begin{array}{cc}\hfill \parallel K_2{\parallel }_{L^2\left({\mathbb{R}}^N\right)}⩽& {\left[{\int }_{{\mathbb{R}}^N}{\left({\int }_{|z|>1}\frac{|u(x)-u(x-z)|}{{|z|}^{N+2s_1}}\mathrm{d}z\right)}^2\mathrm{d}x\right]}^{\frac{1}{2}}\hfill \\ \hfill ⩽& {\left[{\int }_{{\mathbb{R}}^N}{\left({\int }_{|z|>1}\frac{|u(x)|+|u(x-z)|}{{|z|}^{N+2s_1}}\mathrm{d}z\right)}^2\mathrm{d}x\right]}^{\frac{1}{2}}\hfill \\ \hfill ⩽& C\left[{\int }_{{\mathbb{R}}^N}{|u\left(x\right)|}^2{\left({\int }_{|z|>1}\frac{1}{{|z|}^{N+2s_1}}\mathrm{d}z\right)}^2\mathrm{d}x\right.\hfill \\ \hfill & +{\left.{\int }_{{\mathbb{R}}^N}{|u(x-z)|}^2{\left({\int }_{|z|>1}\frac{1}{{|z|}^{N+2s_1}}\mathrm{d}z\right)}^2\mathrm{d}x\right]}^{\frac{1}{2}}\hfill \\ \hfill ⩽& {C\parallel u\parallel }_{L^2\left({\mathbb{R}}^N\right)}{\int }_1^{\infty }\frac{{\rho }^{N-1}}{{\rho }^{N+2s_1}}\mathrm{d}\rho \hfill \\ \hfill =& {C\parallel u\parallel }_{L^2\left({\mathbb{R}}^N\right)}{\int }_1^{\infty }\frac{1}{{\rho }^{2s_1+1}}\mathrm{d}\rho ⩽C{\parallel u\parallel }_{H^{s_2}\left({\mathbb{R}}^N\right)},\hfill \end{array}$$

where $2s_1+1>1$. □

Lemma 9.

Let $N⩾3$, $\frac{1}{2}<s_1+\frac{1}{2}<s_2<1$ and $u\in H^{2s_2}\left({\mathbb{R}}^N\right)\cap L^{\infty }\left({\mathbb{R}}^N\right)$. Then, there exists $C>0$ (independent of u) such that

$$\begin{array}{c}\hfill {\parallel (-\mathsf{\Delta })}^{s_1+\frac{1}{2}}{u\parallel }_{L^2\left({\mathbb{R}}^N\right)}⩽C{\parallel u\parallel }_{H^{2s_2}\left({\mathbb{R}}^N\right)}.\end{array}$$

Proof. 

Recalling that $\frac{1}{2}<s_1+\frac{1}{2}<s_2<1$, one has

$$\begin{array}{cc}\hfill & {\parallel (-\mathsf{\Delta })}^{s_1+\frac{1}{2}}{u\parallel }_{L^2\left({\mathbb{R}}^N\right)}\hfill \\ \hfill =& {\int }_{{\mathbb{R}}^N}{|\xi |}^{4s_1+2}{|\widehat{u}\left(\xi \right)|}^2\mathrm{d}\xi \hfill \\ \hfill ⩽& {\left({\int }_{{\mathbb{R}}^N}{|\xi |}^{2(2s_1+1)\frac{2s_2}{2s_1+1}}{|\widehat{u}\left(\xi \right)|}^{\frac{2s_1+1}{s_2}\frac{2s_2}{2s_1+1}}\mathrm{d}\xi \right)}^{\frac{2s_1+1}{2s_2}}{\left({\int }_{{\mathbb{R}}^N}{|\widehat{u}\left(\xi \right)|}^{\frac{2s_2-2s_1-1}{s_2}\frac{2s_2}{2s_2-2s_1-1}}\mathrm{d}\xi \right)}^{\frac{2s_2-2s_1-1}{2s_2}}\hfill \\ \hfill =& {\left({\int }_{{\mathbb{R}}^N}{|\xi |}^{4s_2}{|\widehat{u}\left(\xi \right)|}^2\mathrm{d}\xi \right)}^{\frac{2s_1+1}{2s_2}}{\left({\int }_{{\mathbb{R}}^N}{|\widehat{u}\left(\xi \right)|}^2\mathrm{d}\xi \right)}^{\frac{2s_2-2s_1-1}{2s_2}}\hfill \\ \hfill ⩽& {C\parallel u\parallel }_{H^{2s_2}\left({\mathbb{R}}^N\right)}^2.\hfill \end{array}$$

□

We can write Equation (1) in the next form

$$\begin{array}{c}\hfill u={[{(-\mathsf{\Delta })}^{s_2}-\lambda ]}^{-1}\left[\nu (I_{\mu }{\ast |u|}^{1+\frac{\mu }{N}}{)|u|}^{\frac{\mu }{N}-1}u+\gamma {|u|}^{p-2}u-{(-\mathsf{\Delta })}^{s_1}u\right].\end{array}$$

On the other hand, using Fourier representation, we have

$$\begin{array}{c}\hfill u=h\ast \left[\nu (I_{\mu }{\ast |u|}^{1+\frac{\mu }{N}}{)|u|}^{\frac{\mu }{N}-1}u+\gamma {|u|}^{p-2}u-{(-\mathsf{\Delta })}^{s_1}u\right],\end{array}$$

where h is the Green’s function, which is a kernel with the Fourier representation

$$\begin{array}{c}\hfill \mathcal{F}h\left(\xi \right)=\frac{1}{{|\xi |}^{2s_2}-\lambda }.\end{array}$$

Lemma 10.

Let $N⩾3$, $0<s_1<s_2<1$ and $\lambda <0$. Then,

$$\begin{array}{c}\hfill \frac{1}{{(-\mathsf{\Delta })}^{s_2}-\lambda }\end{array}$$

and

$$\begin{array}{c}\hfill \frac{{(-\mathsf{\Delta })}^{s_2}}{{(-\mathsf{\Delta })}^{s_2}-\lambda }\end{array}$$

are bounded multipliers $L^2\left({\mathbb{R}}^N\right)\to L^2\left({\mathbb{R}}^N\right)$.

Proof. 

Define $l\left(\xi \right):={|\xi |}^{2s_2}-\lambda$. Since $\lambda <0$, we see that $l\left(\xi \right)>0$. Therefore,

$$\begin{array}{c}\hfill 0⩽\frac{1}{{|\xi |}^{2s_2}-\lambda }⩽\frac{1}{min\{1,-\lambda \}(|\xi {|}^{2s_2}+1)}⩽C,\end{array}$$

and

$$\begin{array}{c}\hfill 0⩽\frac{{|\xi |}^{2s_2}}{{|\xi |}^{2s_2}-\lambda }⩽\frac{{|\xi |}^{2s_2}}{min\{1,-\lambda \}(|\xi {|}^{2s_2}+1)}⩽C,\end{array}$$

where $C>0$ is independent of $\xi$. □

By Lemmas 7 and 8, we can find the next regularity result.

Lemma 11.

Let $N⩾3$, $\frac{1}{2}<s_1+\frac{1}{2}<s_2<1$, $p\in (2,2_{s_2}^{\ast })$, $\mu \in (max\left\{1,N-4s_2\right\},N)$, and $(u,\lambda )\in H^{s_2}\left({\mathbb{R}}^N\right)\times {\mathbb{R}}^{-}$ be solutions for Equation (1). Then, $u\in H^2\left({\mathbb{R}}^N\right)\cap L^{\infty }\left({\mathbb{R}}^N\right)$.

Proof. 

By Lemma 7, we derive that $u\in H^{s_2}\left({\mathbb{R}}^N\right)\cap L^{\infty }\left({\mathbb{R}}^N\right)$. Lemma 8 and (11) imply that $I_{\mu }\ast {|u|}^{1+\frac{\mu }{N}}\in L^{\infty }\left({\mathbb{R}}^N\right)$. Combining Lemmas 8 and 10, we deduce that

$$\begin{array}{cc}\hfill {\parallel u\parallel }_{D^{2s_2,2}\left({\mathbb{R}}^N\right)}^2=& {\parallel (-\mathsf{\Delta })}^{s_2}{u\parallel }_{L^2\left({\mathbb{R}}^N\right)}^2\hfill \\ \hfill =& {∥\frac{{(-\mathsf{\Delta })}^{s_2}}{{(-\mathsf{\Delta })}^{s_2}-\lambda }\left(\nu (I_{\mu }{\ast |u|}^{1+\frac{\mu }{N}}{)|u|}^{\frac{\mu }{N}-1}u+\gamma {|u|}^{p-2}u-{(-\mathsf{\Delta })}^{s_1}u\right)∥}_{L^2\left({\mathbb{R}}^N\right)}^2\hfill \\ \hfill ⩽& C{∥\nu (I_{\mu }{\ast |u|}^{1+\frac{\mu }{N}}{)|u|}^{\frac{\mu }{N}-1}u+\gamma {|u|}^{p-2}u-{(-\mathsf{\Delta })}^{s_2}u∥}_{L^2\left({\mathbb{R}}^N\right)}^2\hfill \\ \hfill ⩽& C\left[{\parallel u\parallel }_{L^{\infty }\left({\mathbb{R}}^N\right)}^{2(\frac{\mu }{N}-2)}+{2\parallel u\parallel }_{L^{\infty }\left({\mathbb{R}}^N\right)}^{(\frac{\mu }{N}-2)+(p-2)}+{\parallel u\parallel }_{L^{\infty }\left({\mathbb{R}}^N\right)}^{2(p-2)}\right]{\int }_{{\mathbb{R}}^N}{|u|}^2\mathrm{d}x\hfill \\ \hfill & +C\left[{\parallel u\parallel }_{L^{\infty }\left({\mathbb{R}}^N\right)}^{2(\frac{\mu }{N}-1)}+{2\parallel u\parallel }_{L^{\infty }\left({\mathbb{R}}^N\right)}^{(p-1)+(\frac{\mu }{N}-1)}+{\parallel u\parallel }_{L^{\infty }\left({\mathbb{R}}^N\right)}^{2(p-1)}\right]{\parallel {(-\mathsf{\Delta })}^{s_2}u\parallel }_{L^2\left({\mathbb{R}}^N\right)}^2\hfill \\ \hfill ⩽& C,\hfill \end{array}$$

then $u\in D^{2s_2,2}\left({\mathbb{R}}^N\right)$. According to $u\in L^2\left({\mathbb{R}}^N\right)$, we derive that $u\in H^{2s_2}\left({\mathbb{R}}^N\right)\subset H^1\left({\mathbb{R}}^N\right)$ for $\frac{1}{2}⩽s_2<1$.

Set

$$\begin{array}{c}\hfill f=\nu (I_{\mu }{\ast |u|}^{1+\frac{\mu }{N}}{)|u|}^{\frac{\mu }{N}-1}u+\gamma {|u|}^{p-2}u-{(-\mathsf{\Delta })}^{s_1}u.\end{array}$$

Since $u\in H^{2s_1}\left({\mathbb{R}}^N\right)\cap L^{\infty }\left({\mathbb{R}}^N\right)$, and ${|\nabla (|u|}^{p-2}u)|⩽{(p-1)|u|}^{p-2}|\nabla u|$ a.e. in ${\mathbb{R}}^N$, we conclude that

$$\begin{array}{cc}\hfill |\nabla f|⩽& \nu \left|\nabla \left[\right(I_{\mu }{\ast |u|}^{1+\frac{\mu }{N}}{)|u|}^{\frac{\mu }{N}-1}u]\right|+{\gamma |\nabla (|u|}^{p-2}{u)|+|(-\mathsf{\Delta })}^{s_1+\frac{1}{2}}u|\hfill \\ \hfill =& \nu \left|\nabla \left({\int }_{{\mathbb{R}}^N}\frac{{|u\left(y\right)|}^{1+\frac{\mu }{N}}}{{|x-y|}^{N-\mu }}\mathrm{d}y{|u\left(x\right)|}^{\frac{\mu }{N}-1}u\right)\right|+{\gamma |\nabla (|u|}^{p-2}{u)|+|(-\mathsf{\Delta })}^{s_1+\frac{1}{2}}u|\hfill \\ \hfill ⩽& \nu \left|{\int }_{{\mathbb{R}}^N}\frac{{|u\left(y\right)|}^{1+\frac{\mu }{N}}}{{|x-y|}^{N-\mu }}{\mathrm{d}y\nabla (|u\left(x\right)|}^{\frac{\mu }{N}-1}u\left(x\right))\right|\hfill \\ \hfill & +\nu \left|\nabla \left({\int }_{{\mathbb{R}}^N}\frac{{|u\left(y\right)|}^{\frac{\mu }{N}+1}}{{|x-y|}^{N-\mu }}\mathrm{d}y\right){|u\left(x\right)|}^{\frac{\mu }{N}-1}u\left(x\right)\right|\hfill \\ \hfill & +{\gamma |\nabla (|u|}^{p-2}{u)|+|(-\mathsf{\Delta })}^{s_1+\frac{1}{2}}u|\hfill \\ \hfill ⩽& {C|u|}^{\frac{\mu }{N}-1}|\nabla u|+\left|(\mu -N){\int }_{{\mathbb{R}}^N}\frac{x-y}{{|x-y|}^{N-\mu +2}}{|u\left(y\right)|}^{\frac{\mu }{N}+1}\mathrm{d}y{|u\left(x\right)|}^{\frac{\mu }{N}-1}u\left(x\right)\right|\hfill \\ \hfill & +{C|u|}^{p-2}{|\nabla u|+|(-\mathsf{\Delta })}^{s_1+\frac{1}{2}}u|\hfill \\ \hfill ⩽& {C|u|}^{\frac{\mu }{N}-1}|\nabla u|\hfill \\ \hfill & +C{\int }_{{\mathbb{R}}^N}\frac{{|u\left(y\right)|}^p}{{|x-y|}^{N-\mu +1}}{\mathrm{d}y|u\left(x\right)|}^{\frac{\mu }{N}}+{C|u|}^{p-2}{|\nabla u|+|(-\mathsf{\Delta })}^{s_1+\frac{1}{2}}u|\hfill \\ \hfill =& {C|u|}^{\frac{\mu }{N}-1}|\nabla u|+C(I_{\mu -1}{\ast |u|}^{\frac{\mu }{N}+1}{)|u\left(x\right)|}^{\frac{\mu }{N}}+{C|u|}^{p-2}{|\nabla u|+|(-\mathsf{\Delta })}^{s_1+\frac{1}{2}}u|.\hfill \end{array}$$

(25)

Since $\mu \in (1,N)$, Lemma 8 and (11), we know $I_{\mu -1}\ast {|u|}^{\frac{\mu }{N}+1}\in L^{\infty }\left({\mathbb{R}}^N\right)$. By (25), Lemma 9, and Lemma 7, we conclude that

$$\begin{array}{cc}\hfill {\parallel \nabla f\parallel }_{L^2\left({\mathbb{R}}^N\right)}^2⩽& C\left({\parallel u\parallel }_{L^{\infty }\left({\mathbb{R}}^N\right)}^{2(\frac{\mu }{N}-1)}+{2\parallel u\parallel }_{L^{\infty }\left({\mathbb{R}}^N\right)}^{(p-2)+(\frac{\mu }{N}-1)}+{\parallel u\parallel }_{L^{\infty }\left({\mathbb{R}}^N\right)}^{2(p-2)}\right){\parallel \nabla u\parallel }_{L^2\left({\mathbb{R}}^N\right)}^2\hfill \\ \hfill & {+C\parallel (-\mathsf{\Delta })}^{s_1+\frac{1}{2}}{u\parallel }_{L^2\left({\mathbb{R}}^N\right)}⩽C.\hfill \end{array}$$

This implies that $f\in H^1\left({\mathbb{R}}^N\right)\cap L^{\infty }\left({\mathbb{R}}^N\right)$. Using the same procedures as above, we deduce that

$$\begin{array}{cc}\hfill {\parallel u\parallel }_{D^{2s_2+1,2}\left({\mathbb{R}}^N\right)}^2=& {\parallel (-\mathsf{\Delta })}^{s_2+\frac{1}{2}}{u\parallel }_{L^2\left({\mathbb{R}}^N\right)}^2\hfill \\ \hfill =& {∥\frac{{(-\mathsf{\Delta })}^{s_2}}{{(-\mathsf{\Delta })}^{s_2}-\lambda }{(-\mathsf{\Delta })}^{\frac{1}{2}}f∥}_{L^2\left({\mathbb{R}}^N\right)}^2\hfill \\ \hfill ⩽& {C\parallel (-\mathsf{\Delta })}^{\frac{1}{2}}{f\parallel }_{L^2\left({\mathbb{R}}^N\right)}^2\hfill \\ \hfill =& {C\parallel f\parallel }_{D^{1,2}\left({\mathbb{R}}^N\right)}^2⩽C,\hfill \end{array}$$

which yields $u\in H^{2s_2+1}\left({\mathbb{R}}^N\right)\subset H^2\left({\mathbb{R}}^N\right)$. □

Proof of Theorem 1.

Rewrite (1) as

$${(-\mathsf{\Delta })}^{s_1}u+{(-\mathsf{\Delta })}^{s_2}u=g(x,u):=\lambda u+\nu (I_{\mu }{\ast |u|}^{1+\frac{\mu }{N}}{)|u|}^{\frac{\mu }{N}-1}u+\gamma {|u|}^{p-2}u.$$

Note that $u\in H^{s_2}\left({\mathbb{R}}^N\right)\hookrightarrow L^q\left({\mathbb{R}}^N\right)$ for all $2⩽q⩽2_{s_2}^{\ast }$. From ([11], Theorem 1.1) and Lemma 7, we dereive that $u\in L^{\infty }\left({\mathbb{R}}^N\right)$. By using Lemma 11, similar to ([1], Lemma 2.2), we can obtain the Pohozaev identity

$$\begin{array}{cc}\hfill & \frac{N-2s_1}{2}{\parallel u\parallel }_{D^{s_1,2}\left({\mathbb{R}}^N\right)}^2+\frac{N-2s_2}{2}{\parallel u\parallel }_{D^{s_2,2}\left({\mathbb{R}}^N\right)}^2\hfill \\ \hfill =& \frac{\lambda N}{2}{\int }_{{\mathbb{R}}^N}{|u|}^2\mathrm{d}x+\frac{N\nu }{2}{\int }_{{\mathbb{R}}^N}(I_{\mu }{\ast |u|}^{1+\frac{\mu }{N}}{)|u|}^{1+\frac{\mu }{N}}\mathrm{d}x+\frac{N\gamma }{p}{\int }_{{\mathbb{R}}^N}{|u|}^p\mathrm{d}x.\hfill \end{array}$$

(26)

By employing the Nehari identity, we obtain

$$\begin{array}{cc}\hfill & {\parallel u\parallel }_{D^{s_1,2}\left({\mathbb{R}}^N\right)}^2+{\parallel u\parallel }_{D^{s_2,2}\left({\mathbb{R}}^N\right)}^2\hfill \\ \hfill =& \lambda {\int }_{{\mathbb{R}}^N}{|u|}^2\mathrm{d}x+\nu {\int }_{{\mathbb{R}}^N}(I_{\mu }{\ast |u|}^{1+\frac{\mu }{N}}{)|u|}^{1+\frac{\mu }{N}}\mathrm{d}x+\gamma {\int }_{{\mathbb{R}}^N}{|u|}^p\mathrm{d}x.\hfill \end{array}$$

(27)

Combining (26) and (27), we have

$$\begin{array}{c}\hfill s_1{\parallel u\parallel }_{D^{s_1,2}\left({\mathbb{R}}^N\right)}^2+s_2{\parallel u\parallel }_{D^{s_2,2}\left({\mathbb{R}}^N\right)}^2=\frac{\gamma N(p-2)}{2p}{\int }_{{\mathbb{R}}^N}{|u|}^p\mathrm{d}x.\end{array}$$

□

4. Proof of Theorem 2

Having established the basic properties of solutions in Theorem 1, we now turn to the existence of normalized ground states.

Lemma 12.

Let $N⩾3$ and $2<p<\tilde{p}\left(s_2\right)$. Then, the functional $J_{\nu ,\gamma }$ is bounded from below and is coercive on ${\mathcal{M}}_c$.

Proof. 

It follows from (10) and (12) that for each $u\in {\mathcal{M}}_c$,

$$\begin{array}{cc}\hfill J_{\nu ,\gamma }\left(u\right)⩾& \frac{1}{2}{\parallel u\parallel }_{D^{s_1,2}\left({\mathbb{R}}^N\right)}^2+\frac{1}{2}{\parallel u\parallel }_{D^{s_2,2}\left({\mathbb{R}}^N\right)}^2-\frac{N\nu }{2(N+\mu )}{S}_{\mu }^{-(1+\frac{\mu }{N})}{c}^{2(1+\frac{\mu }{N})}\hfill \\ \hfill -& \frac{\gamma }{p}{C}_{N,s,p}{\parallel u\parallel }_{D^{s_2,2}\left({\mathbb{R}}^N\right)}^{\frac{N(p-2)}{2s_2}}{c}^{\frac{2s_{2}p-N(p-2)}{2s_2}}.\hfill \end{array}$$

(28)

Due to $2<p<\tilde{p}\left(s_2\right)$, we observe that $0<\frac{N(p-2)}{2s_2}<2$, which yields that $J_{\nu ,\gamma }$ is bounded from below and coercive on ${\mathcal{M}}_c$. □

Lemma 13.

Let $N⩾3$ and $2<p<\tilde{p}\left(s_2\right)$. Then,

$$m_{\nu ,\gamma }\left(c\right)<-\frac{N\nu }{2(N+\mu )}{S}_{\mu }^{-(1+\frac{\mu }{N})}{c}^{2(1+\frac{\mu }{N})}<0,$$

where $m_{\nu ,\gamma }\left(c\right)$ is defined in (4).

Proof. 

From (14), one sees that

$${\int }_{{\mathbb{R}}^N}(I_{\mu }\ast |U_{ϵ}{|}^{1+\frac{\mu }{N}})|U_{ϵ}{|}^{1+\frac{\mu }{N}}\mathrm{d}x=S_{\mu }^{-(1+\frac{\mu }{N})}{\left({\int }_{{\mathbb{R}}^N}{|U_{ϵ}|}^2\mathrm{d}x\right)}^{\frac{N+\mu }{N}}.$$

According to the above equality, we set

$$\varphi :=\frac{U_{ϵ}c}{\parallel U_{ϵ}{\parallel }_{L^2\left({\mathbb{R}}^N\right)}}\mathrm{and}(t★\varphi )\left(x\right):=e^{\frac{Nt}{2}}\varphi \left(e^{t}x\right),\mathrm{for}x\in {\mathbb{R}}^N.$$

Clearly, $\varphi \in {\mathcal{M}}_c$ and $(t★\varphi )\in {\mathcal{M}}_c$. We compute that

$$\begin{array}{cc}\hfill J_{\nu ,\gamma }(t★\varphi )=& \frac{e^{2s_{1}t}}{2}{\parallel \varphi \parallel }_{D^{s_1,2}\left({\mathbb{R}}^N\right)}^2+\frac{e^{2s_{2}t}}{2}{\parallel \varphi \parallel }_{D^{s_2,2}\left({\mathbb{R}}^N\right)}^2\hfill \\ \hfill & -\frac{N\nu }{2(N+\mu )}{\int }_{{\mathbb{R}}^N}(I_{\mu }{\ast |\varphi |}^{1+\frac{\mu }{N}}{)|\varphi |}^{1+\frac{\mu }{N}}\mathrm{d}x-\frac{\gamma e^{\frac{N(p-2)}{2}t}}{p}{\parallel \varphi \parallel }_{L^p\left({\mathbb{R}}^N\right)}^p\hfill \\ \hfill =& \frac{e^{2s_{1}t}}{2}{\parallel \varphi \parallel }_{D^{s_1,2}\left({\mathbb{R}}^N\right)}^2+\frac{e^{2s_{2}t}}{2}{\parallel \varphi \parallel }_{D^{s_2,2}\left({\mathbb{R}}^N\right)}^2-\frac{N\nu }{2(N+\mu )}{S}_{\mu }^{-(1+\frac{\mu }{N})}{c}^{2(1+\frac{\mu }{N})}\hfill \\ \hfill & -\frac{\gamma e^{\frac{N(p-2)}{2}t}}{p}{\parallel \varphi \parallel }_{L^p\left({\mathbb{R}}^N\right)}^p.\hfill \end{array}$$

Therefore, due to $2<p<\tilde{p}\left(s_2\right)$, there exists $t_0\ll -1$ such that

$$m_{\nu ,\gamma }\left(c\right)<-\frac{N\nu }{2(N+\mu )}{S}_{\mu }^{-(1+\frac{\mu }{N})}{c}^{2(1+\frac{\mu }{N})}.$$

□

Lemma 14.

Let $c_1,c_2>0$ be such that $c_1^2+c_2^2=c^2.$ Then,

$$m_{\nu ,\gamma }\left(c\right)<m_{\nu ,\gamma }\left(c_1\right)+m_{\nu ,\gamma }\left(c_2\right).$$

Proof. 

Let $a>0,\tau >1$ and $\left\{u_n\right\}\subset {\mathcal{M}}_a$ be a minimizing sequence of $m_{\nu ,\gamma }\left(a\right)$. Then, we obtain that

$$\begin{array}{cc}\hfill m_{\nu ,\gamma }\left(\tau a\right)⩽& J_{\nu ,\gamma }\left(\tau u_n\right)\hfill \\ \hfill =& \frac{{\tau }^2}{2}\parallel u_n{\parallel }_{D^{s_1,2}\left({\mathbb{R}}^N\right)}^2+\frac{{\tau }^2}{2}\parallel u_n{\parallel }_{D^{s_2,2}\left({\mathbb{R}}^N\right)}^2-\frac{N\nu {\tau }^{2(1+\frac{\mu }{N})}}{2(N+\mu )}{\int }_{{\mathbb{R}}^N}(I_{\mu }\ast |u_n{|}^{1+\frac{\mu }{N}})|u_n{|}^{1+\frac{\mu }{N}}\mathrm{d}x\hfill \\ \hfill & -\frac{\gamma {\tau }^p}{p}{\parallel u_n\parallel }_{L^p\left({\mathbb{R}}^N\right)}^p<{\tau }^2{J}_{\nu ,\gamma }\left(u_n\right),\hfill \end{array}$$

noting that $\tau >1$ and $p>2$. This implies that $m_{\nu ,\gamma }\left(\tau a\right)⩽{\tau }^2{m}_{\nu ,\gamma }\left(a\right)$, and the equality holds if and only if ${\int }_{{\mathbb{R}}^N}(I_{\mu }\ast |u_n{|}^{1+\frac{\mu }{N}})|u_n{|}^{1+\frac{\mu }{N}}\mathrm{d}x+{\parallel u_n\parallel }_{L^p\left({\mathbb{R}}^N\right)}^p\to 0$ as $n\to \infty$. But, this is just not impossible. If not, from Lemma 13, we conclude that

$$0>m_{\nu ,\gamma }\left(a\right)=\underset{n\to \infty }{lim}{J}_{\nu ,\gamma }\left(u_n\right)⩾\underset{n\to \infty }{lim\; inf}\frac{1}{2}\parallel u_n{\parallel }_{D^{s_1,2}\left({\mathbb{R}}^N\right)}^2+\frac{1}{2}{\parallel u_n\parallel }_{D^{s_2,2}\left({\mathbb{R}}^N\right)}^2⩾0.$$

Hence, the strict inequality $m_{\nu ,\gamma }\left(\tau a\right)<{\tau }^2{m}_{\nu ,\gamma }\left(a\right)$ holds. Set $\tau a=c$ and $\frac{1}{\sqrt{{\tau }^2-1}}>1$. Thus, we conclude that

$$\begin{array}{cc}\hfill m_{\nu ,\gamma }\left(c\right)& <{\tau }^2{m}_{\nu ,\gamma }\left(\frac{c}{\tau }\right)=m_{\nu ,\gamma }\left(\frac{c}{\tau }\right)+({\tau }^2-1)m_{\nu ,\gamma }\left(\frac{1}{\sqrt{{\tau }^2-1}}·\frac{\sqrt{{\tau }^2-1}}{\tau }c\right)\hfill \\ \hfill & <m_{\nu ,\gamma }\left(\frac{c}{\tau }\right)+m_{\nu ,\gamma }\left(\frac{\sqrt{{\tau }^2-1}}{\tau }c\right),\hfill \end{array}$$

which ends the proof. □

Lemma 15.

Let $N⩾3$ and $2<p<\tilde{p}\left(s_2\right)$ and $\left\{u_n\right\}\subset H^{s_2}\left({\mathbb{R}}^N\right)$ be a sequence such that

$$J_{\nu ,\gamma }\left(u_n\right)\to m_{\nu ,\gamma }\left(c\right),{\parallel u_n\parallel }_{L^2\left({\mathbb{R}}^N\right)}=c_n\to c.$$

Then, there exists a subsequence, still denoted by $\left\{u_n\right\}$, a sequence of points $\left\{z_n\right\}\subset {\mathbb{R}}^N$ and $\overline{u}\in {\mathcal{M}}_c$ such that $u_n(·+z_n)\to \overline{u}$ strongly in $H^{s_2}\left({\mathbb{R}}^N\right)$.

Proof. 

It follows easily from (28) and $2<p<\tilde{p}\left(s_2\right)$ that $\left\{u_n\right\}$ is bounded in $H^{s_2}\left({\mathbb{R}}^N\right)$. Thus, there exists a subsequence of $\left\{u_n\right\}$ (still denoted by $\left\{u_n\right\}$) and $\overline{u}\in H^{s_2}\left({\mathbb{R}}^N\right)$ such that

$$u_n\rightharpoonup \overline{u},\mathrm{in}{H}^{s_2}\left({\mathbb{R}}^N\right),u_n\to \overline{u}\mathrm{in}{L}_{loc}^2\left({\mathbb{R}}^N\right).$$

We claim that $\overline{u}\ne 0$. In fact, if $\overline{u}=0$, then $u_n\rightharpoonup 0$ in $H^{s_2}\left({\mathbb{R}}^N\right)$. From Lemma 6, we derive that

$${\int }_{{\mathbb{R}}^N}(I_{\mu }\ast |u_n{|}^{1+\frac{\mu }{N}})|u_n{|}^{1+\frac{\mu }{N}}\mathrm{d}x=o_n\left(1\right),{\int }_{{\mathbb{R}}^N}{|u_n|}^p\mathrm{d}x=o_n\left(1\right).$$

Therefore,

$$m_{\nu ,\gamma }\left(c\right)=J_{\nu ,\gamma }\left(u_n\right)+o_n\left(1\right)=\frac{1}{2}\parallel u_n{\parallel }_{D^{s_1,2}\left({\mathbb{R}}^N\right)}^2+\frac{1}{2}{\parallel u_n\parallel }_{D^{s_2,2}\left({\mathbb{R}}^N\right)}^2⩾0,$$

which contradicts Lemma 13. Then, we have showed $\overline{u}\ne 0$. There exists a sequence of $\left\{z_n\right\}$ such that ${\overline{u}}_n:=u_n(·+z_n)$ converges weakly to $\overline{u}\ne 0$ in $H^{s_2}\left({\mathbb{R}}^N\right)$. From Brezis–Lieb Lemma for the nonlocal term of the functional ([23], Lemma 2.4) and ([25], Lemma 5.1), we obtain that

$$c^2=\parallel u_n{\parallel }_{L^2\left({\mathbb{R}}^N\right)}^2=\parallel u_n-\overline{u}{\parallel }_{L^2\left({\mathbb{R}}^N\right)}^2+{\parallel \overline{u}\parallel }_{L^2\left({\mathbb{R}}^N\right)}^2+o_n\left(1\right),$$

(29)

$$J_{\nu ,\gamma }(u_n)=J_{\nu ,\gamma }(u_n-\overline{u})+J_{\nu ,\gamma }(\overline{u})+o_n(1).$$

(30)

If $\parallel \overline{u}{\parallel }_{L^2({\mathbb{R}}^N)}^2<c^2$, we set $\tau :=\frac{c}{\parallel \overline{u}{\parallel }_{L^2({\mathbb{R}}^N)}},$ then $\tau >1,\tau \overline{u}\in {\mathcal{M}}_c$ and

$$\begin{array}{cc}\hfill J_{\nu ,\gamma }(\tau \overline{u})=& \frac{{\tau }^2}{2}\parallel \overline{u}{\parallel }_{D^{s_1,2}({\mathbb{R}}^N)}^2+\frac{{\tau }^2}{2}\parallel \overline{u}{\parallel }_{D^{s_2,2}({\mathbb{R}}^N)}^2-\frac{N\nu {\tau }^{2(1+\frac{\mu }{N})}}{2(N+\mu )}{\int }_{{\mathbb{R}}^N}(I_{\mu }\ast |\overline{u}{|}^{1+\frac{\mu }{N}})|\overline{u}{|}^{1+\frac{\mu }{N}}\mathrm{d}x\hfill \\ \hfill & -\frac{\gamma {\tau }^p}{p}{\int }_{{\mathbb{R}}^N}{|\overline{u}|}^p\mathrm{d}x,\hfill \end{array}$$

which yields that

$$\begin{array}{cc}\hfill & J_{\nu ,\gamma }(\overline{u})\hfill \\ \hfill =& \frac{1}{{\tau }^2}{J}_{\nu ,\gamma }(\tau \overline{u})+\frac{N\nu ({\tau }^{\frac{2\mu }{N}}-1)}{2(N+\mu )}{\int }_{{\mathbb{R}}^N}(I_{\mu }\ast |\overline{u}{|}^{1+\frac{\mu }{N}})|\overline{u}{|}^{1+\frac{\mu }{N}}\mathrm{d}x+\frac{\gamma ({\tau }^{p-2}-1)}{p}{\int }_{{\mathbb{R}}^N}{|\overline{u}|}^p\mathrm{d}x.\hfill \end{array}$$

(31)

Similarly, taking ${\tau }_n:=\frac{c}{\parallel u_n-\overline{u}{\parallel }_{L^2({\mathbb{R}}^N)}}⩾1,{\tau }_n(u_n-\overline{u})\in {\mathcal{M}}_c$, we derive that

$$\begin{array}{cc}\hfill J_{\nu ,\gamma }(u_n-\overline{u})=& \frac{1}{{\tau }_n^2}{J}_{\nu ,\gamma }({\tau }_n(u_n-\overline{u}))+\frac{\gamma ({\tau }_n^{p-2}-1)}{p}{\int }_{{\mathbb{R}}^N}{|u_n-\overline{u}|}^p\mathrm{d}x\hfill \\ \hfill & +\frac{N\nu ({\tau }_n^{\frac{2\mu }{N}}-1)}{2(N+\mu )}{\int }_{{\mathbb{R}}^N}(I_{\mu }\ast |u_n-\overline{u}{|}^{1+\frac{\mu }{N}})|u_n-\overline{u}{|}^{1+\frac{\mu }{N}}\mathrm{d}x\hfill \\ \hfill ⩾& \frac{1}{{\tau }_n^2}{J}_{\nu ,\gamma }({\tau }_n(u_n-\overline{u})).\hfill \end{array}$$

(32)

Combining (29)–(32), by (12), we conclude that

$$\begin{array}{cc}\hfill m_{\nu ,\gamma }(c)=& J_{\nu ,\gamma }(u_n)+o_n(1)=J_{\nu ,\gamma }(\overline{u})+J_{\nu ,\gamma }(u_n-\overline{u})+o_n(1)\hfill \\ \hfill ⩾& \frac{1}{{\tau }^2}{J}_{\nu ,\gamma }(\tau \overline{u})+\frac{1}{{\tau }_n^2}J({\tau }_n(u_n-\overline{u}))+\frac{N\nu ({\tau }^{\frac{2\mu }{N}}-1)}{2(N+\mu )}{\int }_{{\mathbb{R}}^N}(I_{\mu }\ast |\overline{u}{|}^{1+\frac{\mu }{N}})|\overline{u}{|}^{1+\frac{\mu }{N}}\mathrm{d}x\hfill \\ \hfill & +\frac{\gamma ({\tau }^{p-2}-1)}{p}{\int }_{{\mathbb{R}}^N}{|\overline{u}|}^p\mathrm{d}x+o_n(1)\hfill \\ \hfill >& \frac{1}{{\tau }^2}{m}_{\nu ,\gamma }(c)+\frac{1}{{\tau }_n^2}{m}_{\nu ,\gamma }(c)=m_{\nu ,\gamma }(c),\hfill \end{array}$$

which is a contradiction. Hence, $\parallel \overline{u}{\parallel }_{L^2({\mathbb{R}}^N)}^2=c^2$. This yields that ${\overline{u}}_n:=u_n(·+z_n)$ converges strongly to $\overline{u}$ in $L^2({\mathbb{R}}^N)$. Therefore, ([23], Lemma 2.4) indicates that

$${\int }_{{\mathbb{R}}^N}(I_{\mu }\ast |{\overline{u}}_n{|}^{1+\frac{\mu }{N}})|{\overline{u}}_n{|}^{1+\frac{\mu }{N}}\mathrm{d}x={\int }_{{\mathbb{R}}^N}(I_{\mu }\ast |\overline{u}{|}^{1+\frac{\mu }{N}})|\overline{u}{|}^{1+\frac{\mu }{N}}\mathrm{d}x+o_n(1).$$

(33)

Recalling interpolation inequality and the fractional Sobolev embedding theorem, we deduce that

$$\parallel {\overline{u}}_n-\overline{u}{\parallel }_{L^p({\mathbb{R}}^N)}⩽\parallel {\overline{u}}_n-\overline{u}{\parallel }_{L^2({\mathbb{R}}^N)}^{\theta }\parallel {\overline{u}}_n-\overline{u}{\parallel }_{L^{2_{s_2}^{\ast }}({\mathbb{R}}^N)}^{1-\theta }⩽C{\parallel {\overline{u}}_n-\overline{u}\parallel }_{L^2({\mathbb{R}}^N)}^{\theta }\to 0,$$

(34)

as $n\to \infty$, where $p\in (2,2_{s_2}^{\ast })$ and $\frac{1}{p}=\frac{\theta }{2}+\frac{1-\theta }{2_{s_2}^{\ast }}$. Applying (33) and (34) and the weakly lower semicontinuity of the norm, we infer that

$$m_{\nu ,\gamma }(c)⩽J_{\nu ,\gamma }(\overline{u})⩽\underset{n\to \infty }{lim\; inf}{J}_{\nu ,\gamma }({\overline{u}}_n)=\underset{n\to \infty }{lim\; inf}{J}_{\nu ,\gamma }(u_n)=m_{\nu ,\gamma }(c),$$

which implies that $\parallel {\overline{u}}_n{\parallel }_{D^{s_i,2}\left({\mathbb{R}}^N\right)}\to {\parallel \overline{u}\parallel }_{D^{s_i,2}\left({\mathbb{R}}^N\right)}$ as $n\to \infty$ for $i=1,2$. □

Proof of Theorem 2.

From Lemma 15, there is a minimizer $\overline{u}$ for $J_{\nu ,\gamma }$ on ${\mathcal{M}}_c$. Let $|\overline{u}{|}^{\ast }$ denote the symmetric decreasing rearrangement of $\overline{u}$ [22]. Obviously,

$$\parallel |\overline{u}{|\parallel }_{L^2\left({\mathbb{R}}^N\right)}=\parallel |\overline{u}{|}^{\ast }{\parallel }_{L^2\left({\mathbb{R}}^N\right)},\parallel |\overline{u}{|\parallel }_{L^p\left({\mathbb{R}}^N\right)}=\parallel |\overline{u}{{|}^{\ast }\parallel }_{L^p\left({\mathbb{R}}^N\right)}.$$

(35)

Moreover, by the fractional Polya–Szegö inequality [26] and (A.11) in [27], we observe that

$$\parallel |\overline{u}{|}^{\ast }{\parallel }_{D^{s_i,2}\left({\mathbb{R}}^N\right)}^2⩽\parallel |\overline{u}{|\parallel }_{D^{s_i,2}\left({\mathbb{R}}^N\right)}^2⩽{\parallel \overline{u}\parallel }_{D^{s_i,2}\left({\mathbb{R}}^N\right)}^2,i=1,2.$$

(36)

Meanwhile, the Riesz’s rearrangement inequality ([22], Theorem 3.4) indicates that

$${\int }_{{\mathbb{R}}^N}(I_{\mu }\ast |\overline{u}{|}^{1+\frac{\mu }{N}})|\overline{u}{|}^{1+\frac{\mu }{N}}\mathrm{d}x⩽{\int }_{{\mathbb{R}}^N}(I_{\mu }\ast (|\overline{u}{|}^{\ast }{)}^{1+\frac{\mu }{N}})(|\overline{u}{{|}^{\ast })}^{1+\frac{\mu }{N}}\mathrm{d}x.$$

(37)

Gathering (35)–(37), we obtain that $|\overline{u}{|}^{\ast }\in {\mathcal{M}}_c$ and $J_{\nu ,\gamma }(|\overline{u}{|}^{\ast })⩽J_{\nu ,\gamma }(|\overline{u}|)=m_{\nu ,\gamma }(c)$. Then, $m_{\nu ,\gamma }(c)$ is attained by $|\overline{u}{|}^{\ast }$, which is radially symmetric decreasing. For simplity, we still denote it by $\overline{u}$. There eixsts a Lagrange multiplier $\overline{\lambda }$ corresponding to $\overline{u}$ such that

$$\begin{array}{cc}\hfill \overline{\lambda }{c}^2=& \parallel \overline{u}{\parallel }_{D^{s_1,2}({\mathbb{R}}^N)}^2+\parallel \overline{u}{\parallel }_{D^{s_2,2}({\mathbb{R}}^N)}^2-\nu {\int }_{{\mathbb{R}}^N}(I_{\mu }\ast |\overline{u}{|}^{1+\frac{\mu }{N}})|\overline{u}{|}^{1+\frac{\mu }{N}}\mathrm{d}x-\gamma {\parallel \overline{u}\parallel }_{L^p({\mathbb{R}}^N)}^p\hfill \\ \hfill =& 2m_{\nu ,\gamma }(c)-\frac{\mu \nu }{N+\mu }{\int }_{{\mathbb{R}}^N}(I_{\mu }\ast |\overline{u}{|}^{1+\frac{\mu }{N}})|\overline{u}{|}^{1+\frac{\mu }{N}}\mathrm{d}x-\frac{(p-2)\gamma }{p}{\parallel \overline{u}\parallel }_{L^p({\mathbb{R}}^N)}^p\hfill \\ \hfill ⩽& 2m_{\nu ,\gamma }(c)<-\frac{N\nu }{(N+\mu )}{S}_{\mu }^{-(1+\frac{\mu }{N})}{c}^{2(1+\frac{\mu }{N})}<0,\hfill \end{array}$$

recalling Lemma 13, which yields that

$$\overline{\lambda }<-\frac{N\nu }{(N+\mu )}{S}_{\mu }^{-(1+\frac{\mu }{N})}{c}^{\frac{2\mu }{N}}<0.$$

The proof is completed. □

5. Proof of Theorem 4

In this section, we shall prove Theorem 4.

Lemma 16.

Let $N⩾3$, $\tilde{p}(s_2)<p<2_{s_1}^{\ast }$. For each $u\in {\mathcal{M}}_c$, ${\mathsf{\Phi }}_u^{\nu ,\gamma }$ admits a unique critical point $t_u\in \mathbb{R}$ such that

$$J_{\nu ,\gamma }(t_u★u)=\underset{t\in \mathbb{R}}{max}{J}_{\nu ,\gamma }(t★u),(t_u★u)\in {\mathcal{P}}_{c,\gamma }.$$

(38)

Particularly, the map $u\in {\mathcal{M}}_c\mapsto t_u\in \mathbb{R}$ is of class $C^1$.

Proof. 

For $u\in {\mathcal{M}}_c$, we see that

$$\begin{array}{cc}\hfill {({\mathsf{\Phi }}_u^{\nu ,\gamma })}^{\prime }(t)=& s_1{e}^{2s_{1}t}({\parallel u\parallel }_{D^{s_1,2}({\mathbb{R}}^N)}^2+\frac{s_2}{s_1}{e}^{2(s_2-s_1)t}{\parallel u\parallel }_{D^{s_2,2}({\mathbb{R}}^N)}^2\hfill \\ \hfill & \left.-\frac{N\gamma (p-2)}{2p}{e}^{(\frac{N(p-2)}{2}-2s_1)t}{\parallel u\parallel }_{L^p({\mathbb{R}}^N)}^p\right).\hfill \end{array}$$

(39)

Thanks to $s_2>s_1$ and $\tilde{p}(s_2)<p$, we can derive that ${({\mathsf{\Phi }}_u^{\nu ,\gamma })}^{\prime }(t)\to 0^{+}$ as $t\to -\infty$ and ${({\mathsf{\Phi }}_u^{\nu ,\gamma })}^{\prime }(t)\to -\infty$ as $t\to +\infty$. Furthermore, from (8), we conclude that ${({\mathsf{\Phi }}_u^{\nu ,\gamma })}^{\prime }(t)$ has a unique zero point $t_u$, which is the unique maximum point of ${\mathsf{\Phi }}_u^{\nu ,\gamma }(t)$. Together with (5) and (7), (38) holds.

We denote by $\mathsf{\Psi }:\mathbb{R}\times {\mathcal{M}}_c\mapsto \mathbb{R}$ the function $\mathsf{\Psi }(t,u)={({\mathsf{\Phi }}_u^{\nu ,\gamma })}^{\prime }(t)$. Applying the implicit functon theorem to the $C^1$ function $\mathsf{\Psi }$, we can complete the proof. □

Setting

$$\begin{array}{cc}\hfill K_{c,\gamma }:=& \frac{N(p-2)-4s_1}{2N(p-2)}{\left(\frac{2s_{1}p}{\gamma N(p-2)}{C}_{N,s,p}^{-1}{c}^{-\frac{2s_{1}p+(p-2)N}{2s_1}}\right)}^{\frac{4s_1}{N(p-2)-4s_1}}\hfill \\ \hfill & -\frac{N\nu }{2(N+\mu )}{S}_{\mu }^{-(1+\frac{\mu }{N})}{c}^{\frac{2(N+\mu )}{N}},\hfill \end{array}$$

we see that $K_{c,\gamma }>0$, according to (9).

Lemma 17.

Let $N⩾3$, $\tilde{p}\left(s_2\right)<p<2_{s_1}^{\ast }$ and (9) hold. Then, $J_{\nu ,\gamma }$ is coercive on ${\mathcal{P}}_{c,\gamma }$ and

$$\sigma \left(c\right):=\underset{u\in {\mathcal{P}}_{c,\gamma }}{inf}{J}_{\nu ,\gamma }\left(u\right)>K_{c,\gamma }>0.$$

Proof. 

For $u\in {\mathcal{P}}_{c,\gamma }$, by $\tilde{p}\left(s_2\right)<p<2_{s_1}^{\ast }$ and (10), we conclude that

$$\begin{array}{cc}\hfill & {\parallel u\parallel }_{D^{s_1,2}\left({\mathbb{R}}^N\right)}^2⩽{\parallel u\parallel }_{D^{s_1,2}\left({\mathbb{R}}^N\right)}^2+{\parallel u\parallel }_{D^{s_2,2}\left({\mathbb{R}}^N\right)}^2⩽\frac{\gamma N(p-2)}{2s_{1}p}{\parallel u\parallel }_{L^p\left({\mathbb{R}}^N\right)}^p\hfill \\ \hfill ⩽& \frac{\gamma N(p-2)}{2s_{1}p}{C}_{N,s_1,p}{\left({\int }_{{\mathbb{R}}^N}{|{(-\mathsf{\Delta })}^{\frac{s_1}{2}}u|}^2\mathrm{d}x\right)}^{\frac{N(p-2)}{4s_1}}{\left({\int }_{{\mathbb{R}}^N}{|u|}^2\mathrm{d}x\right)}^{\frac{2s_{1}p+(p-2)N}{4s_1}},\hfill \end{array}$$

which yields that

$${\parallel u\parallel }_{D^{s_1,2}\left({\mathbb{R}}^N\right)}^2⩾{\left(\frac{2s_{1}p}{\gamma N(p-2)}{C}_{N,s_1,p}^{-1}{c}^{-\frac{2s_{1}p+(p-2)N}{2s_1}}\right)}^{\frac{4s_1}{N(p-2)-4s_1}}.$$

(40)

Thus, for each $u\in {\mathcal{P}}_{c,\gamma }$, from (9), (12), and (40), we observe that

$$\begin{array}{cc}\hfill J_{\nu ,\gamma }\left(u\right)=& \frac{1}{2}{\int }_{{\mathbb{R}}^N}{|(-\mathsf{\Delta })}^{\frac{s_1}{2}}{u|}^2\mathrm{d}x+\frac{1}{2}{\int }_{{\mathbb{R}}^N}{|{(-\mathsf{\Delta })}^{\frac{s_2}{2}}u|}^2\mathrm{d}x\hfill \\ \hfill & -\frac{N\nu }{2(N+\mu )}{\int }_{{\mathbb{R}}^N}(I_{\mu }{\ast |u|}^{1+\frac{\mu }{N}}{)|u|}^{1+\frac{\mu }{N}}\mathrm{d}x-\frac{\gamma }{p}{\int }_{{\mathbb{R}}^N}{|u|}^p\mathrm{d}x\hfill \\ \hfill ⩾& \frac{N(p-2)-4s_1}{2N(p-2)}{\int }_{{\mathbb{R}}^N}{|{(-\mathsf{\Delta })}^{\frac{s_1}{2}}u|}^2\mathrm{d}x-\frac{N\nu }{2(N+\mu )}{S}_{\mu }^{-(1+\frac{\mu }{N})}{c}^{\frac{2(N+\mu )}{N}}\hfill \\ \hfill ⩾& \frac{N(p-2)-4s_1}{2N(p-2)}{\left(\frac{2s_{1}p}{\gamma N(p-2)}{C}_{N,s,p}^{-1}{c}^{-\frac{2s_{1}p+(p-2)N}{2s_1}}\right)}^{\frac{4s_1}{N(p-2)-4s_1}}\hfill \\ \hfill & -\frac{N\nu }{2(N+\mu )}{S}_{\mu }^{-(1+\frac{\mu }{N})}{c}^{\frac{2(N+\mu )}{N}}\hfill \\ \hfill :=& K_{c,\gamma }>0.\hfill \end{array}$$

□

Define

$$\begin{array}{cc}\hfill H_{rad}^{s_2}\left({\mathbb{R}}^N\right):=& \{u\in H^{s_2}\left({\mathbb{R}}^N\right):u\mathrm{is}\mathrm{radially}\mathrm{symmetric}\},\hfill \\ \hfill {\mathcal{M}}_c^{rad}:=& {\mathcal{M}}_c\cap H_{rad}^{s_2}\left({\mathbb{R}}^N\right),\hfill \\ \hfill {\mathcal{P}}_{c,\gamma }^{rad}:=& {\mathcal{P}}_{c,\gamma }\cap H_{rad}^{s_2}\left({\mathbb{R}}^N\right).\hfill \end{array}$$

Lemma 18.

Let (9) hold. Then,

$$\underset{u\in {\mathcal{P}}_{c,\gamma }}{inf}{J}_{\nu ,\gamma }\left(u\right)=\underset{u\in {\mathcal{P}}_{c,\gamma }^{rad}}{inf}{J}_{\nu ,\gamma }\left(u\right).$$

Proof. 

From ${\mathcal{P}}_{c,\gamma }^{rad}\subset {\mathcal{P}}_{c,\gamma }$, we see that

$$0<K_{c,\gamma }⩽\underset{u\in {\mathcal{P}}_{c,\gamma }}{inf}{J}_{\nu ,\gamma }\left(u\right)⩽\underset{u\in {\mathcal{P}}_{c,\gamma }^{rad}}{inf}{J}_{\nu ,\gamma }\left(u\right).$$

Therefore, it is only necessary to prove that

$$\underset{u\in {\mathcal{P}}_{c,\gamma }}{inf}{J}_{\nu ,\gamma }\left(u\right)⩾\underset{u\in {\mathcal{P}}_{c,\gamma }^{rad}}{inf}{J}_{\nu ,\gamma }\left(u\right).$$

(41)

For this purpose, let ${|u|}^{\ast }$ denote the symmetric decreasing rearrangement of $|u|$. According to (35)–(37), we derive that ${|u|}^{\ast }\in {\mathcal{M}}_c^{rad}$ and

$${\mathsf{\Phi }}_{{|u|}^{\ast }}^{\nu ,\gamma }\left(t\right)=J_{\nu ,\gamma }{(t★|u|}^{\ast })⩽J_{\nu ,\gamma }(t★u)={\mathsf{\Phi }}_u^{\nu ,\gamma }\left(t\right).$$

It follows from (35), (36) and (39) that $-\infty <t_{{|u|}^{\ast }}⩽t_u$. Together with Lemma 16, we conclude that

$${\mathsf{\Phi }}_u^{\nu ,\gamma }\left(t_u\right)⩾{\mathsf{\Phi }}_u^{\nu ,\gamma }\left(t_{{|u|}^{\ast }}\right)⩾{\mathsf{\Phi }}_{{|u|}^{\ast }}^{\nu ,\gamma }\left(t_{{|u|}^{\ast }}\right).$$

According to $u\in {\mathcal{P}}_{c,\gamma }$, we obtain that $t_u=0$ and then

$$J_{\nu ,\gamma }\left(u\right)={\mathsf{\Phi }}_u^{\nu ,\gamma }\left(0\right)⩾{\mathsf{\Phi }}_{{|u|}^{\ast }}^{\nu ,\gamma }\left(t_{{|u|}^{\ast }}\right)=J_{\nu ,\gamma }(t_{{|u|}^{\ast }}{★|u|}^{\ast }).$$

Since $t_{{|u|}^{\ast }}★{|u|}^{\ast }\in {\mathcal{P}}_{c,\gamma }^{rad}$, we infer that

$$\underset{u\in {\mathcal{P}}_{c,\gamma }}{inf}{J}_{\nu ,\gamma }\left(u\right)⩾\underset{u\in {\mathcal{P}}_{c,\gamma }}{inf}{J}_{\nu ,\gamma }(t_{{|u|}^{\ast }}{★|u|}^{\ast })⩾\underset{u\in {\mathcal{P}}_{c,\gamma }^{rad}}{inf}{J}_{\nu ,\gamma }(u),$$

which yields that (41) holds. □

Lemma 19.

For $u\in {\mathcal{M}}_c^{rad}$ and $t\in \mathbb{R}$, the map

$$T_u{\mathcal{M}}_c^{rad}\to T_{t_u★u}{\mathcal{M}}_c^{rad},\psi \mapsto t_u★\psi$$

is a linear isomorphism with inverse $\varphi \mapsto (-t)★\varphi$, where $T_u{\mathcal{M}}_c^{rad}$ is the tangent space to ${\mathcal{M}}_c^{rad}$ in u.

Proof. 

The proof is standard, see ([14], Lemma 5.5). □

Next, we consider the functional ${\overline{J}}_{\nu ,\gamma }:{\mathcal{M}}_c^{rad}\mapsto \mathbb{R}$ defined by

$${\overline{J}}_{\nu ,\gamma }=J_{\nu ,\gamma }(t_u★u).$$

It follows from Lemma 16 that ${\overline{J}}_{\nu ,\gamma }$ is of class $C^1$. Similar to ([17], Lemma 3.15), we derive the following result.

Lemma 20.

It holds that

$${\overline{J}}_{\nu ,\gamma }^{\prime }\left(u\right)\left[\psi \right]={\overline{J}}_{\nu ,\gamma }^{\prime }(t_u★u)[t_u★\psi ],$$

for each $u\in {\mathcal{M}}_c^{rad}$ and $\psi \in T_u{\mathcal{M}}_c^{rad}$.

Once again, analogue to ([17], Lemma 3.16), we obtain the existence of Palais–Smale sequences to a general homotopy-stable family, according to Lemmas 19 and 20.

Lemma 21.

Let $\mathcal{H}$ be a homotopy-stable family of compact subsets of ${\mathcal{M}}_c^{rad}$ with a closed boundary $\mathcal{D}$ and define

$${\kappa }_{\mathcal{H}}:=\underset{B\in \mathcal{H}}{inf}\underset{u\in B}{max}{\overline{J}}_{\nu ,\gamma }\left(u\right).$$

Assume that $\mathcal{D}$ is contained in a connected component of ${\mathcal{P}}_{c,\gamma }^{rad}$ and

$$max\{sup{\overline{J}}_{\nu ,\gamma }\left(\mathcal{D}\right),0\}<{\kappa }_{\mathcal{H}}<\infty .$$

Then, there exists a Palais–Smale sequence $\left\{u_n\right\}\subset {\mathcal{P}}_{c,\gamma }^{rad}$ of ${\overline{J}}_{\nu ,\gamma }$ restricted to ${\mathcal{M}}_c^{rad}$ at level ${\kappa }_{\mathcal{H}}$.

Applying Lemma 21, we shall present the existence of a Palais–Smale sequence $\left\{u_n\right\}\subset {\mathcal{P}}_{c,\gamma }^{rad}$ of ${\overline{J}}_{\nu ,\gamma }$ restricted to ${\mathcal{M}}_c^{rad}$ at level $\sigma \left(c\right)>0$.

Lemma 22.

Let $N⩾3$, $\tilde{p}\left(s_2\right)<p<2_{s_1}^{\ast }$ and $c,\nu ,\gamma >0$ and (9) hold. Then, there exists a Palais–Smale sequence $\left\{u_n\right\}\subset {\mathcal{P}}_{c,\gamma }^{rad}$ for $J_{\nu ,\gamma }{|}_{{\mathcal{M}}_c}$ at level $\sigma \left(c\right)>0$.

Proof. 

Let $\overline{\mathcal{H}}$ be a family of all singletons belonging to ${\mathcal{M}}_c^{rad}$. Clearly, the boundary $\mathcal{D}$ is empty. Thus, it is a homotopy-stable family of compact subset of ${\mathcal{M}}_c^{rad}$ without a boundary, due to ([28], Definition 3.1). Taking into account of Lemma 18, we derive that

$$\begin{array}{c}\hfill {\kappa }_{\overline{\mathcal{H}}}=\underset{B\in \overline{\mathcal{H}}}{inf}\underset{u\in B}{max}{\overline{J}}_{\nu ,\gamma }\left(u\right)=\underset{u\in {\mathcal{M}}_c^{rad}}{inf}{\overline{J}}_{\nu ,\gamma }\left(u\right)=\underset{u\in {\mathcal{P}}_c^{rad}}{inf}{J}_{\nu ,\gamma }\left(u\right)=\underset{u\in {\mathcal{P}}_c}{inf}{J}_{\nu ,\gamma }\left(u\right)=\sigma \left(c\right).\end{array}$$

Therefore, using Lemma 21, we end the proof. □

Next, we discuss the convergence of special Palais–Smale sequences that satisfy appropriate additional conditions, following the idea first proposed by Jeanjean in [29].

Lemma 23.

Let $N⩾3$, $\tilde{p}\left(s_2\right)<p<2_{s_1}^{\ast }$ and $\left\{u_n\right\}\subset {\mathcal{P}}_{c,\gamma }^{rad}$ be a Palais–Smale sequence for $J_{\nu ,\gamma }{|}_{{\mathcal{M}}_c}$ at level $\sigma \left(c\right)>0$. If $\left\{u_n\right\}$ is bounded in $H_{rad}^{s_2}\left({\mathbb{R}}^N\right)$, then there eixsts ${\gamma }_0>0$ such that for each $\gamma >{\gamma }_0$, up to a subsequence, $u_n\to \overline{u}$ strongly in $H_{rad}^{s_2}\left({\mathbb{R}}^N\right)$.

Proof. 

The proof is divided into five main steps.

Step 1. Since $\left\{u_n\right\}\subset {\mathcal{P}}_{c,\gamma }^{rad}$ is bounded and the embedding $H_{rad}^{s_2}\left({\mathbb{R}}^N\right)\hookrightarrow L^q\left({\mathbb{R}}^N\right)$ is compact for $q\in (2,2_{s_1}^{\ast })$, there exists $\overline{u}\in H_{rad}^{s_2}\left({\mathbb{R}}^N\right)$ such that

$$u_n\rightharpoonup \overline{u}\mathrm{in}{H}_{rad}^{s_2}\left({\mathbb{R}}^N\right),u_n\to \overline{u}\mathrm{in}{L}^q\left({\mathbb{R}}^N\right),\mathrm{for}q\in (2,2_{s_1}^{\ast })\mathrm{and}\mathrm{a}.\mathrm{e}.\mathrm{in}{\mathbb{R}}^N.$$

(42)

Moreover, there exists a sequence $\left\{{\lambda }_n\right\}\subset \mathbb{R}$ such that for any $v\in H_{rad}^{s_2}\left({\mathbb{R}}^N\right)$,

$$\begin{array}{cc}\hfill & {\int }_{{\mathbb{R}}^N}{(-\mathsf{\Delta })}^{\frac{s_1}{2}}{u}_n{(-\mathsf{\Delta })}^{\frac{s_1}{2}}v\mathrm{d}x+{\int }_{{\mathbb{R}}^N}{(-\mathsf{\Delta })}^{\frac{s_2}{2}}{u}_n{(-\mathsf{\Delta })}^{\frac{s_2}{2}}v\mathrm{d}x-{\lambda }_n{\int }_{{\mathbb{R}}^N}{u}_{n}v\mathrm{d}x\hfill \\ \hfill & -\nu {\int }_{{\mathbb{R}}^N}(I_{\mu }\ast |u_n{|}^{1+\frac{\mu }{N}})|u_n{|}^{\frac{\mu }{N}-1}{u}_{n}v\mathrm{d}x-\gamma {\int }_{{\mathbb{R}}^N}|u_n{|}^{p-2}{u}_{n}v\mathrm{d}x=o_n\left(1\right)\parallel v\parallel .\hfill \end{array}$$

(43)

Taking $v=u_n$ in (43), we observe that

$$\begin{array}{cc}\hfill -{\lambda }_n{c}^2=& \gamma \parallel u_n{\parallel }_{L^p\left({\mathbb{R}}^N\right)}^p+\nu {\int }_{{\mathbb{R}}^N}(I_{\mu }\ast |u_n{|}^{1+\frac{\mu }{N}})|u_n{|}^{\frac{\mu }{N}+1}\mathrm{d}x\hfill \\ \hfill & -\parallel u_n{\parallel }_{D^{s_1,2}\left({\mathbb{R}}^N\right)}^2-{\parallel u_n\parallel }_{D^{s_2,2}\left({\mathbb{R}}^N\right)}^2+o_n\left(1\right),\hfill \end{array}$$

which yields that ${\lambda }_n$ is bounded. Then, up to a subsequence, there exists $\overline{\lambda }\in \mathbb{R}$ such that ${\lambda }_n\to \overline{\lambda }$ as $n\to \infty$.

Step 2. $\overline{\lambda }<0$. From $\tilde{p}\left(s_2\right)<p<2_{s_1}^{\ast }$ and $\left\{u_n\right\}\subset {\mathcal{P}}_{c,\gamma }^{rad}$, we infer that

$$\begin{array}{cc}\hfill -{\lambda }_n{c}^2=& \left(\frac{2p{s}_1-N(p-2)}{N(p-2)}\right)\parallel u_n{\parallel }_{D^{s_1,2}\left({\mathbb{R}}^N\right)}^2+\left(\frac{2p{s}_2-N(p-2)}{N(p-2)}\right){\parallel u_n\parallel }_{D^{s_2,2}\left({\mathbb{R}}^N\right)}^2\hfill \\ \hfill & +\nu {\int }_{{\mathbb{R}}^N}(I_{\mu }\ast |u_n{|}^{1+\frac{\mu }{N}})|u_n{|}^{\frac{\mu }{N}+1}\mathrm{d}x⩾0,\hfill \end{array}$$

(44)

which leads to $\overline{\lambda }⩽0$. We will show that $\overline{\lambda }\ne 0$; if not, due to (44) and $G_{\gamma }\left(u_n\right)=o_n\left(1\right)$, we can see that $\sigma \left(c\right)+o_n\left(1\right)=J_{\nu ,\gamma }\left(u_n\right)=o_n\left(1\right)$, which contradicts Lemma 17. Thus, $\overline{\lambda }<0.$

Step 3. $\overline{u}\ne 0$. Assume by contradiction that $\overline{u}=0$. Hence, $\parallel u_n{\parallel }_{D^{s_i,2}\left({\mathbb{R}}^N\right)}^2={\parallel u_n\parallel }_{L^p\left({\mathbb{R}}^N\right)}^p=o_n\left(1\right),i=1,2$. Together with (44) and $\overline{\lambda }<0$, we deduce that

$$0<-\overline{\lambda }{c}^2=\underset{n\to \infty }{lim}-{\lambda }_n{c}^2=\underset{n\to \infty }{lim}\nu {\int }_{{\mathbb{R}}^N}(I_{\mu }\ast |u_n{|}^{1+\frac{\mu }{N}})|u_n{|}^{\frac{\mu }{N}+1}\mathrm{d}x.$$

On the other hand,

$$0<\sigma \left(c\right)=\underset{n\to \infty }{lim}{J}_{\nu ,\gamma }\left(u_n\right)=-\frac{N\nu }{2(N+\mu )}{\int }_{{\mathbb{R}}^N}(I_{\mu }\ast |u_n{|}^{1+\frac{\mu }{N}})|u_n{|}^{1+\frac{\mu }{N}}\mathrm{d}x<0,$$

which yields a contradiction. Therefore, $\overline{u}\ne 0$.

Step 4. The upper bound of $\sigma \left(c\right)-\frac{\overline{\lambda }}{2}{c}^2$. By (13) and (14), we obtain that

$${\int }_{{\mathbb{R}}^N}(I_{\mu }\ast |U_{ϵ}{|}^{1+\frac{\mu }{N}})|U_{ϵ}{|}^{1+\frac{\mu }{N}}\mathrm{d}x=S_{\mu }^{-(1+\frac{\mu }{N})}{\left({\int }_{{\mathbb{R}}^N}{|U_{ϵ}|}^2\mathrm{d}x\right)}^{\frac{N+\mu }{N}}.$$

Set

$$\phi :=\frac{c{U}_{ϵ}}{\parallel U_{ϵ}{\parallel }_{L^2\left({\mathbb{R}}^N\right)}},\mathrm{and}(t★\phi )\left(x\right):=e^{\frac{N}{2}t}\phi \left(e^{t}x\right),$$

for a.e. $x\in {\mathbb{R}}^N$. Clearly, $\phi \in {\mathcal{M}}_c$ and $(t★\phi )\left(x\right)\in {\mathcal{M}}_c$. From Lemma 16, there exists a unique $t_{\phi }\in \mathbb{R}$ such that

$$J_{\nu ,\gamma }(t_{\phi }★\phi )=\underset{t\in \mathbb{R}}{max}{J}_{\nu ,\gamma }(t★\phi ),t_{\phi }★\phi \in {\mathcal{P}}_{c,\gamma }.$$

Lemma 18 yields that

$$\sigma \left(c\right)⩽J_{\nu ,\gamma }(t_{\phi }★\phi ).$$

Furthermore, by direct calculations, it can be concluded that

$$\begin{array}{cc}\hfill \sigma \left(c\right)⩽& J_{\nu ,\gamma }(t_{\phi }★\phi )\hfill \\ \hfill =& \frac{e^{2s_1{t}_{\phi }}}{2}{\parallel \phi \parallel }_{D^{s_1,2}\left({\mathbb{R}}^N\right)}^2+\frac{e^{2s_2{t}_{\phi }}}{2}{\parallel \phi \parallel }_{D^{s_2,2}\left({\mathbb{R}}^N\right)}^2-\frac{N\nu }{2(N+\mu )}{\int }_{{\mathbb{R}}^N}(I_{\mu }{\ast |\phi |}^{1+\frac{\mu }{N}}{)|\phi |}^{1+\frac{\mu }{N}}\mathrm{d}x\hfill \\ \hfill & -\frac{\gamma e^{\frac{N(p-2)}{2}{t}_{\phi }}}{p}{\parallel \phi \parallel }_{L^p\left({\mathbb{R}}^N\right)}^p\hfill \\ \hfill =& \frac{e^{2s_1{t}_{\phi }}}{2}{\parallel \phi \parallel }_{D^{s_1,2}\left({\mathbb{R}}^N\right)}^2+\frac{e^{2s_2{t}_{\phi }}}{2}{\parallel \phi \parallel }_{D^{s_2,2}\left({\mathbb{R}}^N\right)}^2-\frac{\gamma e^{\frac{N(p-2)}{2}{t}_{\phi }}}{p}{\parallel \phi \parallel }_{L^p\left({\mathbb{R}}^N\right)}^p\hfill \\ \hfill & -\frac{N\nu }{2(N+\mu )}{S}_{\mu }^{-(1+\frac{\mu }{N})}{c}^{2(1+\frac{\mu }{N})}.\hfill \end{array}$$

Therefore, taking

$${\gamma }_0:=\left(\frac{e^{2s_1{t}_{\phi }}}{2}{\parallel \phi \parallel }_{D^{s_1,2}\left({\mathbb{R}}^N\right)}^2+\frac{e^{2s_2{t}_{\phi }}}{2}{\parallel \phi \parallel }_{D^{s_2,2}\left({\mathbb{R}}^N\right)}^2\right)\frac{p{e}^{-\frac{N(p-2)}{2}{t}_{\phi }}}{{\parallel \phi \parallel }_{L^p\left({\mathbb{R}}^N\right)}^p},$$

we observe that for any $\gamma >{\gamma }_0$,

$$\sigma \left(c\right)<-\frac{N\nu }{2(N+\mu )}{S}_{\mu }^{-(1+\frac{\mu }{N})}{c}^{2(1+\frac{\mu }{N})}.$$

(45)

Therefore, applying (45), we conclude that

$$\sigma \left(c\right)-\frac{\overline{\lambda }}{2}{c}^2<-\frac{N\nu }{2(N+\mu )}{S}_{\mu }^{-(1+\frac{\mu }{N})}{c}^{2(1+\frac{\mu }{N})}-\frac{\overline{\lambda }}{2}{c}^2.$$

Now, we define a function $g:{\mathbb{R}}^{+}\to \mathbb{R}$

$$g\left(c\right):=-\frac{N\nu }{2(N+\mu )}{S}_{\mu }^{-(1+\frac{\mu }{N})}{c}^{2(1+\frac{\mu }{N})}-\frac{\overline{\lambda }}{2}{c}^2.$$

Obviously, there exists a unique critical point

$$c_0={\left(-\overline{\lambda }{\nu }^{-1}{S}_{\mu }^{1+\frac{\mu }{N}}\right)}^{\frac{N}{2\mu }},$$

and

$$g\left(c_0\right)=\frac{\mu }{2(N+\mu )}{\nu }^{-\frac{N}{\mu }}{\left(-\overline{\lambda }{S}_{\mu }^{1+\frac{\mu }{N}}\right)}^{\frac{N+\mu }{\mu }}$$

is the maximum of g. Hence, it holds that

$$\sigma \left(c\right)-\frac{\overline{\lambda }}{2}{c}^2<\frac{\mu }{2(N+\mu )}{\nu }^{-\frac{N}{\mu }}{\left(-\overline{\lambda }{S}_{\mu }\right)}^{\frac{N+\mu }{\mu }}.$$

(46)

Step 5. $u_n\to \overline{u}$ in $H_{rad}^{s_2}\left({\mathbb{R}}^N\right)$. Since $u_n\rightharpoonup \overline{u}$ in $H_{rad}^{s_2}\left({\mathbb{R}}^N\right)$, from (43) and Lemma 6, we conclude that $\overline{u}$ is a weak solution of

$${(-\mathsf{\Delta })}^{s_1}\overline{u}+{(-\mathsf{\Delta })}^{s_2}\overline{u}=\overline{\lambda }\overline{u}+\nu (I_{\mu }\ast |\overline{u}{|}^{1+\frac{\mu }{N}})|\overline{u}{|}^{\frac{\mu }{N}-1}+\gamma {|\overline{u}|}^{p-2}u,\mathrm{in}{\mathbb{R}}^N.$$

(47)

Then, we obtain

$$G_{\gamma }\left(\overline{u}\right)=s_1\parallel \overline{u}{\parallel }_{D^{s_{1}2}\left({\mathbb{R}}^N\right)}^2+s_2\parallel \overline{u}{\parallel }_{D^{s_2,2}\left({\mathbb{R}}^N\right)}^2-\frac{\gamma N(p-2)}{2p}{\parallel \overline{u}\parallel }_{L^p\left({\mathbb{R}}^N\right)}^p=0.$$

Set $v_n=u_n-\overline{u}$, then $v_n\rightharpoonup 0$ in $H_{rad}^{s_2}\left({\mathbb{R}}^N\right)$. Hence,

$$\parallel u_n{\parallel }_{D^{s_i,2}\left({\mathbb{R}}^N\right)}^2=\parallel \overline{u}{\parallel }_{D^{s_i,2}\left({\mathbb{R}}^N\right)}^2+{\parallel v_n\parallel }_{D^{s_i,2}\left({\mathbb{R}}^N\right)}^2,i=1,2.$$

From (42) and Lemma 5, we observe that

$$\begin{array}{cc}\hfill {\int }_{{\mathbb{R}}^N}(I_{\mu }\ast |u_n{|}^{1+\frac{\mu }{N}})|u_n{|}^{1+\frac{\mu }{N}}\mathrm{d}x=& {\int }_{{\mathbb{R}}^N}(I_{\mu }\ast |v_n{|}^{1+\frac{\mu }{N}})|v_n{|}^{1+\frac{\mu }{N}}\mathrm{d}x\hfill \\ \hfill & +{\int }_{{\mathbb{R}}^N}(I_{\mu }\ast |\overline{u}{|}^{1+\frac{\mu }{N}})|\overline{u}{|}^{1+\frac{\mu }{N}}\mathrm{d}x+o_n\left(1\right),\hfill \end{array}$$

(48)

and

$${\int }_{{\mathbb{R}}^N}|u_n{|}^p\mathrm{d}x={\int }_{{\mathbb{R}}^N}{|\overline{u}|}^p\mathrm{d}x+o_n\left(1\right).$$

(49)

Therefore, applying $G_{\gamma }\left(u_n\right)=0$ and $G_{\gamma }\left(\overline{u}\right)=0$, we obtain that

$$\parallel u_n{\parallel }_{D^{s_i,2}\left({\mathbb{R}}^N\right)}^2=\parallel \overline{u}{\parallel }_{D^{s_i,2}\left({\mathbb{R}}^N\right)}^2,{\parallel v_n\parallel }_{D^{s_i,2}\left({\mathbb{R}}^N\right)}^2=o_n\left(1\right),i=1,2.$$

(50)

On the other hand, from (47), we obtain that for any $v\in H_{rad}^{s_2}\left({\mathbb{R}}^N\right)$,

$$J_{\nu ,\gamma }^{\prime }\left(\overline{u}\right)v-\lambda {\int }_{{\mathbb{R}}^N}\overline{u}v\mathrm{d}x=0.$$

(51)

Taking $v=u_n-\overline{u}$ in (43) and (51), we conclude that

$$\begin{array}{cc}\hfill \parallel v_n{\parallel }_{D^{s_1,2}\left({\mathbb{R}}^N\right)}^2+{\parallel v_n\parallel }_{D^{s_2,2}\left({\mathbb{R}}^N\right)}^2=& \overline{\lambda }\parallel v_n{\parallel }_{L^2\left({\mathbb{R}}^N\right)}^2+\nu {\int }_{{\mathbb{R}}^N}(I_{\mu }\ast |v_n{|}^{1+\frac{\mu }{N}})|v_n{|}^{1+\frac{\mu }{N}}\mathrm{d}x\hfill \\ \hfill & +\gamma \parallel v_n{\parallel }_{L^p\left({\mathbb{R}}^N\right)}^p+o_n\left(1\right).\hfill \end{array}$$

Applying (49) and (50), we induce that

$$\Lambda :=-\overline{\lambda }\parallel v_n{\parallel }_{L^2\left({\mathbb{R}}^N\right)}^2=\nu {\int }_{{\mathbb{R}}^N}(I_{\mu }\ast |v_n{|}^{1+\frac{\mu }{N}})|v_n{|}^{1+\frac{\mu }{N}}\mathrm{d}x.$$

(52)

Recalling (12), we see that

$$\Lambda =0\mathrm{or}\mathsf{\Lambda }⩾{\nu }^{-\frac{N}{\mu }}{(-\overline{\lambda }{S}_{\mu })}^{\frac{N+\mu }{\mu }}.$$

If $\Lambda =0$, then $u_n\to \overline{u}$ in $H_{rad}^{s_2}\left({\mathbb{R}}^N\right)$, and we end the proof. If $\Lambda ⩾{\nu }^{-\frac{N}{\mu }}{(-\overline{\lambda }{S}_{\mu })}^{\frac{N+\mu }{\mu }}$, by (48)–(50), we can observe that

$$J_{\nu ,\gamma }\left(\overline{u}\right)>\underset{n\to \infty }{lim}{J}_{\nu ,\gamma }\left(u_n\right)=\sigma \left(c\right)>0.$$

Together with (48)–(50) and (52), recalling that $\overline{\lambda }<0$, we have that

$$\begin{array}{cc}\hfill \sigma \left(c\right)-\frac{\overline{\lambda }}{2}{c}^2=& \sigma \left(c\right)-\frac{\overline{\lambda }}{2}\underset{n\to \infty }{lim}{\parallel u_n\parallel }_{L^2\left({\mathbb{R}}^N\right)}^2\hfill \\ \hfill ⩾& \sigma \left(c\right)-\frac{\overline{\lambda }}{2}\underset{n\to \infty }{lim}{\parallel v_n\parallel }_{L^2\left({\mathbb{R}}^N\right)}^2\hfill \\ \hfill =& J_{\nu ,\gamma }\left(\overline{u}\right)+\underset{n\to \infty }{lim}\left(J_{\nu ,\gamma }\left(v_n\right)-\frac{\overline{\lambda }}{2}{\parallel v_n\parallel }_{L^2\left({\mathbb{R}}^N\right)}^2\right)\hfill \\ \hfill =& J_{\nu ,\gamma }\left(\overline{u}\right)+\underset{n\to \infty }{lim}\left(-\frac{N\nu }{2(N+\mu )}{\int }_{{\mathbb{R}}^N}(I_{\mu }\ast |v_n{|}^{1+\frac{\mu }{N}})|v_n{|}^{1+\frac{\mu }{N}}\mathrm{d}x-\frac{\overline{\lambda }}{2}{\parallel v_n\parallel }_{L^2\left({\mathbb{R}}^N\right)}^2\right)\hfill \\ \hfill =& J_{\nu ,\gamma }\left(\overline{u}\right)+\frac{\mu }{2(N+\mu )}\underset{n\to \infty }{lim}\nu {\int }_{{\mathbb{R}}^N}(I_{\mu }\ast |v_n{|}^{1+\frac{\mu }{N}})|v_n{|}^{1+\frac{\mu }{N}}\mathrm{d}x\hfill \\ \hfill ⩾& J_{\nu ,\gamma }\left(\overline{u}\right)+\frac{\mu }{2(N+\mu )}{\nu }^{-\frac{N}{\mu }}{(-\overline{\lambda }{S}_{\mu })}^{\frac{N+\mu }{\mu }}\hfill \\ \hfill >& \frac{\mu }{2(N+\mu )}{\nu }^{-\frac{N}{\mu }}{(-\overline{\lambda }{S}_{\mu })}^{\frac{N+\mu }{\mu }},\hfill \end{array}$$

which contradicts (46). Then, we complete the proof. □

Proof of Theorem 4.

By Lemma 22, $\left\{u_n\right\}\subset {\mathcal{P}}_{c,\gamma }^{rad}$ is a Palais–Smale sequence for $J_{\nu ,\gamma }{|}_{{\mathcal{M}}_c}$ at level $\sigma \left(c\right)>0$. Due to Lemma 17, we obtain that $\left\{u_n\right\}$ is bounded in $H_{rad}^{s_2}\left({\mathbb{R}}^N\right)$. Then, from Lemma 23, there eixsts ${\gamma }_0>0$ such that for each $\gamma >{\gamma }_0$, up to a subsequence, $u_n\to \overline{u}$ strongly in $H_{rad}^{s_2}\left({\mathbb{R}}^N\right)$. Lemma 18 indicates that $\overline{u}$ is a radial minimizer of $J_{\nu ,\gamma }$ on ${\mathcal{P}}_{c,\gamma }$, and it is a solution of (1) with $\overline{\lambda }<0$. Taking into account Lemma 16, we derive that $\overline{u}$ is a ground state solution of $J_{\nu ,\gamma }$ on ${\mathcal{M}}_c$. The proof is completed. □

6. Conclusions

This paper has studied the existence and regularity of normalized ground state solutions for a mixed-order fractional Schrödinger equation involving combined local and nonlocal nonlinearities. The main results establish the key regularity properties of the solution, derive essential Pohozaev identities, and determine precise parameter regimes under which normalized solutions exist.

For $L^2$-subcritical nonlinear interactions, we obtain the attainment of energy minimizers and characterize their geometric properties, showing radial symmetry and monotonicity while also obtaining sharp bounds on the associated Lagrange multiplier. The analysis reveals how competing effects between different fractional orders and nonlinear terms introduce delicate analytical challenges, particularly in maintaining coercivity and compactness under critical scaling conditions.

Furthermore, we identify a $L^2$-critical exponent threshold beyond which no constrained critical points exist, demonstrating the structural limitations imposed by the interplay of nonlocal diffusion and Hartree-type interactions. In the $L^2$-supercritical case, a refined small-$L^2$-constraint guarantees the existence of ground states, provided the local nonlinearity dominates in a controlled manner.

These results extend the understanding of constrained variational problems with mixed nonlinearities, offering new insights into the role of mixed-order fractional operators and nonlocal nonlinearities in the existence of solitary waves. Future directions may include studying multi-peak solutions, or systems with competing fractional orders in more general domains.