Ground State Solutions for a Non-Local Type Problem in Fractional Orlicz Sobolev Spaces

Abstract

In this paper, we study the following non-local problem in fractional Orlicz–Sobolev spaces: ${(-{\Delta }_{\Phi })}^{s}u+V\left(x\right)a(|u|)u=f(x,u), x\in {\mathbb{R}}^N$, where ${(-{\Delta }_{\Phi })}^s(s\in (0,1))$ denotes the non-local and maybe non-homogeneous operator, the so-called fractional $\Phi$-Laplacian. Without assuming the Ambrosetti–Rabinowitz type and the Nehari type conditions on the non-linearity f, we obtain the existence of ground state solutions for the above problem with periodic potential function $V\left(x\right)$. The proof is based on a variant version of the mountain pass theorem and a Lions’ type result in fractional Orlicz–Sobolev spaces.

1. Introduction and Main Results

In recent decades, much attention has been devoted to the study of the non-linear Schrödinger equations involving non-local operators. These types of operators can be used to model many phenomena in the natural sciences, such as fluid dynamics, quantum mechanics, phase transitions, finance, and so on, see [1,2,3,4] and the references therein. Due to the important work of Fernández Bonder and Salort [5], a new generalized fractional $\Phi$-Laplacian operator has caused great interest among scholars in recent years, since it allows to model non-local problems involving a non-power behavior, see [6,7,8,9,10,11,12,13] and the references therein.

In this paper, we are interested in studying the following non-local problem involving fractional $\Phi$-Laplacian:

$$\begin{array}{c}\hfill {(-{\Delta }_{\Phi })}^{s}u+V\left(x\right)a(|u|)u=f(x,u), x\in {\mathbb{R}}^N,\end{array}$$

(1)

where $s\in (0,1),N\in \mathbb{N}$, the function $a:[0,+\infty )\to \mathbb{R}$ is such that $\varphi :\mathbb{R}\to \mathbb{R}$ defined by:

$$\begin{array}{c}\hfill \varphi \left(t\right)=\left\{\begin{array}{c}\begin{array}{cc}a(|t|)t\hfill & \mathrm{for}t\ne 0,\hfill \\ 0\hfill & \mathrm{for}t=0,\hfill \end{array}\hfill \end{array}\right.\end{array}$$

(2)

is an increasing homeomorphism from $\mathbb{R}$ onto $\mathbb{R}$, and $\Phi :[0,+\infty )\to [0,+\infty )$ defined by:

$$\Phi \left(t\right)={\int }_0^t\varphi \left(\tau \right)d\tau$$

is an N-function (see Section 2 for details), which together with the potential V and the non-linearity f satisfy the following basic assumptions:

$\left({\varphi }_1\right)$$1<l≔\underset{t>0}{\inf }\frac{t\varphi \left(t\right)}{\Phi \left(t\right)}\le \underset{t>0}{\sup }\frac{t\varphi \left(t\right)}{\Phi \left(t\right)}=:m<\min \{\frac{N}{s},l^{*}\}$ where $l^{*}≔\frac{Nl}{N-sl};$

$\left(V\right)$$V\in C({\mathbb{R}}^N,{\mathbb{R}}_{+})$ is 1-periodic in $x_1,\cdots ,x_N$ (called 1-periodic in x for short), and so, there exist two constants ${\alpha }_1,{\alpha }_2>0$ such that ${\alpha }_1\le V\left(x\right)\le {\alpha }_2 \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} x\in {\mathbb{R}}^N;$

$\left(f_1\right)$$f\in C({\mathbb{R}}^N\times \mathbb{R})$ is 1-periodic in x satisfying:

$$\underset{|t|\to 0}{\lim }\frac{f(x,t)}{\varphi (|t|)}=0 \mathrm{a}\mathrm{n}\mathrm{d} \underset{|t|\to \infty }{\lim }\frac{f(x,t)}{{\Phi }_{*}^{\prime }(|t|)}=0, \mathrm{u}\mathrm{n}\mathrm{i}\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{m}\mathrm{l}\mathrm{y} \mathrm{i}\mathrm{n} x\in {\mathbb{R}}^N,$$

where ${\Phi }_{*}$ denotes the Sobolev conjugate function of $\Phi$ (see Section 2 for details).

For $s\in (0,1)$, the so-called fractional $\Phi$-Laplacian operator is defined as:

$$\begin{array}{c}\hfill {(-{\Delta }_{\Phi })}^{s}u\left(x\right)≔P.V.{\int }_{{\mathbb{R}}^N}a(|D_{s}u|)\frac{D_{s}u}{{|x-y|}^{N+s}}dy, \mathrm{w}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{e} D_{s}u≔\frac{u\left(x\right)-u\left(y\right)}{{|x-y|}^s}\end{array}$$

(3)

and $P.V.$ denotes the principal value of the integral. Notice that if $\Phi \left(t\right)={|t|}^p(p>1)$, then the fractional $\Phi$-Laplacian operator reduces to the following fractional p-Laplacian operator:

$${(-{\Delta }_p)}^{s}u\left(x\right)≔P.V.{\int }_{{\mathbb{R}}^N}\frac{{|u\left(x\right)-u\left(y\right)|}^{p-2}(u\left(x\right)-u\left(y\right))}{{|x-y|}^{N+ps}}dy.$$

To study this class of non-local problem involving fractional p-Laplacian, the variational method has become one of the important tools over the past several decades, see [14,15,16,17,18,19,20] and the references therein. In many studies on p-superlinear elliptic problems, to ensure the boundedness of the Palais–Smale sequence or Cerami sequence of the energy functional, the following (AR) type condition for the non-linearity f due to Ambrosetti–Rabinowitz [21] was always assumed:

For (AR), there exists a constant $\mu >p$ such that:

$$0<\mu F(x,t)\le tf(x,t), \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} t\ne 0,$$

where the following is true: $F(x,t)=\underset{0}{\overset{t}{\int }}f(x,\tau )d\tau .$

In fact, (AR) implies that there exist two positive constants $c_1,c_2$ such that:

$$F(x,t)\ge c_1{|t|}^{\mu }-c_2, \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} (x,t)\in {\mathbb{R}}^N\times \mathbb{R},$$

which is obviously stronger than the following p-superlinear growth condition:

$\left(F_1\right)$$\underset{|t|\to \infty }{\lim }\frac{F(x,t)}{{|t|}^p}=+\infty$, uniformly in $x\in {\mathbb{R}}^N.$

$\left(F_1\right)$ was first introduced by Liu and Wang in [22] for the case $p=2$ and has since been commonly used in recent papers. With the development of the variational theory and application, certain new restrictive conditions have been established in order to weaken (AR). However, the majority of these conditions are just complementary to (AR). For example, one can replace (AR) with $\left(F_1\right)$ and the following Nehari type condition:

(Ne) $\frac{f(x,t)}{{|t|}^{p-1}}$ is (strictly) increasing in t for all $x\in {\mathbb{R}}^N$.

For the case $p=2$, Li, Wang and Zeng proved the existence of ground state by Nehari method in [23]. Besides, for the case $p=2$, Ding and Szulkin in [24] replaced (AR) with $\left(F_1\right)$ and the following condition:

$\left(F_2\right)$$\mathcal{F}(x,t)>0$ for all $t\ne 0$, and ${|f(x,t)|}^{\sigma }\le c_3\mathcal{F}(x,t){|t|}^{\sigma }$ for some $c_3>0,\sigma >\max \{1,\frac{N}{2}\}$ and all $(x,t)$ with $|t|$ larger enough, where $\mathcal{F}(x,t)=tf(x,t)-2F(x,t)$.

They demonstrated that $\left(F_1\right)$ and $\left(F_2\right)$ are valid when the non-linearity f satisfies both (AR) and a subcritical growth condition that $|f(x,t)|\le c_4(|t|+{|t|}^{q-1})$ for some $c_4>0,q\in (2,2^{*})$ and all $(x,t)\in {\mathbb{R}}^N\times \mathbb{R}$, where $2^{*}=\frac{2N}{N-2}$ if $N\ge 3$ and $2^{*}=\infty$ if $N=1$ or $N=2$. In [25,26], some conditions similar to $\left(F_2\right)$ were introduced for the case $p>1$. Moreover, in [27], Tang introduced the following new and weaker super-quadratic condition:

$\left(F_3\right)$ there exists a ${\theta }_0\in (0,1)$ such that:

$$\frac{1-{\theta }^2}{2}tf(x,t)\ge {\int }_{\theta t}^{t}f(x,\tau )d\tau =F(x,t)-F(x,\theta t), \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} \theta \in [0,{\theta }_0],(x,t)\in {\mathbb{R}}^N\times \mathbb{R}.$$

Tang proved that $\left(F_3\right)$ is weaker than both (AR) and (Ne) and also different from $\left(F_2\right)$. It is worth noting that $\left(F_3\right)$ has been extended for the case $p>1$ in [28].

To the best of our knowledge, some conditions mentioned above have been successfully generalized to the non-local problem involving fractional $\Phi$-Laplacian. In [29], for Equation (1) with potential $V\left(x\right)\equiv 1$, by applying the mountain pass theorem, Sabri, Ounaies, and Elfalah proved the existence of a non-trivial solution when the autonomous non-linearity $f\left(u\right)$ satisfies an (AR) type condition. On the whole space ${\mathbb{R}}^N$, to overcome the difficulty due to the lack of compactness of the Sobolev embedding, the authors reconstructed the compactness by choosing a radially symmetric function subspace as the working space. In [13], for Equation (1) with unbounded or bounded potentials V, by applying the Nehari manifold method, Silva, Carvalho, de Albuquerque, and Bahrouni proved the existence of ground state solutions when the non-linearity f satisfies the following both (AR) and (Ne) type conditions:

For ${\left(\mathrm{AR}\right)}^{*}$, there exists $\theta >m$ such that $\theta F(x,t)\le tf(x,t)$, for $(x,t)\in {\mathbb{R}}^N\times \mathbb{R};$

For ${\left(\mathrm{Ne}\right)}^{*}$, the map $t\to \frac{f(x,t)}{{|t|}^{m-1}}$ is strictly increasing for $t>0$ and strictly decreasing for $t<0$.

To be precise, for the case when V is unbounded, the authors reconstructed the compactness by assuming that V is coercive and then choosing a subspace depending on V as the working space. For the case when V is bounded, to overcome the difficulty due to the lack of compactness and obtain a non-trivial solution, the authors assumed that V and f are 1-periodic in x and introduced an important Lions’ type result for fractional Orlicz–Sobolev spaces (see Theorem 1.6 in [13]). Since the ground state solution is obtained as a minimizer of the energy functional on the Nehari manifold $\mathcal{N}$, it is crucial to require that f is of class $C^1$. Otherwise $\mathcal{N}$ may not be a $C^1$-manifold and it is not clear that the minimizer on the Nehari manifold $\mathcal{N}$ is a critical point of the energy functional.

Motivated by [13], in this paper, we still study the existence of ground state for Equation (1) under the assumption that V and f are 1-periodic in x. We manage to extend the above p-superlinear growth conditions $\left(F_2\right)$ and $\left(F_3\right)$ to the non-local problem involving fractional $\Phi$-Laplacian. Instead of applying the Nehari manifold method, we firstly prove that Equation (1) has a non-trivial solution by using a variant mountain pass theorem (see Theorem 3 in [30]). Subsequently, we prove the existence of ground state by using the Lions’ type result for fractional Orlicz–Sobolev spaces and some techniques of Jeanjean and Tanaka (see Theorem 4.5 in [31]).

Next, we present our main results as follows.

Theorem 1. 

Assume that $\left({\varphi }_1\right)$, $\left(V\right)$, $\left(f_1\right)$ and the following conditions hold:

$\left({\varphi }_2\right)$$\underset{t\to 0}{\mathrm{lim\; sup}}\frac{{|t|}^l}{\Phi (|t|)}<+\infty ;$

$\left(f_2\right)$$\underset{|t|\to \infty }{\lim }\frac{F(x,t)}{\Phi (|t|)}=+\infty ,$ uniformly in $x\in {\mathbb{R}}^N;$

$\left(f_3\right)$$\widehat{F}(x,t)>0$ for all $t\ne 0$, and ${|F(x,t)|}^k\le c\widehat{F}(x,t){|t|}^{lk}$ for some $c>0,$$k>\frac{N}{sl}$ and all $(x,t)$ with $|t|$ larger enough, where $\widehat{F}(x,t)=tf(x,t)-mF(x,t).$

Then, Equation (1) has at least one ground state solution.

Theorem 2. 

Assume that $\left({\varphi }_1\right)$, $\left(V\right)$, $\left(f_1\right)$ and the following conditions hold:

$\left(f_4\right)$$F(x,t)\ge 0$ for all $(x,t)\in {\mathbb{R}}^N\times \mathbb{R},$ and $\underset{|t|\to \infty }{\lim }\frac{F(x,t)}{{|t|}^m}=+\infty ,$ uniformly in $x\in {\mathbb{R}}^N;$

$\left(f_5\right)$ there exists a ${\theta }_0\in (0,1)$ such that:

$$\frac{1-{\theta }^l}{m}tf(x,t)\ge {\int }_{\theta t}^{t}f(x,\tau )d\tau =F(x,t)-F(x,\theta t),for all \theta \in [0,{\theta }_0],(x,t)\in {\mathbb{R}}^N\times \mathbb{R}.$$

Then, Equation (1) has at least one ground state solution.

Remark 1. 

To some extent, Theorem 2 improves the result of Theorem 1.8 in [13]. In fact, our results do not require the smoothness condition that functions f and a are of class $C^1$. Moreover, it is obvious that $\left({\phi }_4\right)$ in [13] implies $\left({\varphi }_1\right)$ and $\left(f_0\right)$ in [13] implies our subcritical growth condition given by $\left(f_1\right)$. Furthermore, when $\Phi \left(t\right)={|t|}^2$, $\left(f_5\right)$ is weaker than both (AR) type condition $\left(f_4\right)$ and (Ne) type condition $\left(f_4\right)$ in [13] (see [27]).

Remark 2. 

Theorem 2 extends and improves the result of Theorem 1.1 in [32]. In fact, when $\Phi \left(t\right)={|t|}^2$, our subcritical growth condition given by $\left(f_1\right)$ reduces to:

$$\begin{array}{c}\hfill \underset{|t|\to \infty }{\lim }\frac{f(x,t)}{{|t|}^{2^{*}-1}}=0, uniformly in x\in {\mathbb{R}}^N,\end{array}$$

(4)

which is weaker than $\left(A_2\right)$ in [32]. For example, it is easy to check that function $f\left(t\right)=\frac{{|t|}^{2^{*}-2}t}{log(e+|t|)}$ satisfies (4) but does not satisfy $\left(A_2\right)$ in [32]. Moreover, it is obvious that Theorem 1 is different from Theorem 1.2 in [32] even when the fractional Φ-Laplacian Equation (1) reduces to the fractional Schrödinger equation.

The rest of this paper is organized as follows. In Section 2, we recall some definitions and basic properties on the Orlicz and fractional Orlicz–Sobolev spaces. In Section 3, we complete the proofs of the main results. In Section 4, we present some examples about the function $\varphi$ defined by (2) and non-linearity f to illustrate our results.

2. Preliminaries

In this section, we make a brief introduction about Orlicz and fractional Orlicz–Sobolev spaces. For more details, we refer the reader to [5,33,34] and references therein.

To begin with, we recall the notion of N-function. Let $\varphi :[0,+\infty )\to [0,+\infty )$ be a right continuous and monotone increasing function that satisfies the following conditions:

(1)

$\varphi \left(0\right)=0$;

(2)

$\underset{t\to +\infty }{lim}\varphi \left(t\right)=+\infty$;

(3)

$\varphi \left(t\right)>0$ whenever $t>0$.

Then, the function defined on $[0,+\infty )$ by $\Phi \left(t\right)={\int }_0^t\varphi \left(\tau \right)d\tau$ is called an N-function. It is obvious that $\Phi \left(0\right)=0$ and $\Phi$ is strictly increasing and convex in $[0,+\infty )$.

An N-function $\Phi$ is said to satisfy the ${\Delta }_2$-condition if there exists a constant $K>0$ such that $\Phi \left(2t\right)\le K\Phi \left(t\right)$ for all $t\ge 0$. $\Phi$ satisfies the ${\Delta }_2$-condition if and only if for any given $c\ge 1$, there exists a constant $K_c>0$ such that $\Phi \left(ct\right)\le K_c\Phi \left(t\right)$ for all $t\ge 0$.

Given two N-functions A and B, B is said to dominate A globally if there exists a constant $K>0$ such that $A\left(t\right)\le B\left(Kt\right)$ for all $t\ge 0$. Furthermore, B is said to be essentially stronger than A, denoted by $A\prec \prec B$, if for each $c>0$ it holds that:

$$\underset{t\to +\infty }{\lim }\frac{A\left(ct\right)}{B\left(t\right)}=0.$$

For the N-function introduced above, the complement of $\Phi$ is defined by:

$$\tilde{\Phi }\left(t\right)=\underset{\rho \ge 0}{\max }\{t\rho -\Phi \left(\rho \right)\}, \mathrm{f}\mathrm{o}\mathrm{r} t\ge 0.$$

Then, it holds that Young’s inequality:

$$\begin{array}{c}\hfill \rho t\le \Phi \left(\rho \right)+\tilde{\Phi }\left(t\right), \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} \rho ,t\ge 0,\end{array}$$

(5)

and the inequality (see Lemma A.2 in [35]):

$$\begin{array}{c}\hfill \tilde{\Phi }\left(\varphi \left(t\right)\right)\le \Phi \left(2t\right), \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} t\ge 0.\end{array}$$

(6)

Now, we recall the Orlicz space $L^{\Phi }\left({\mathbb{R}}^N\right)$ associated with $\Phi$. When $\Phi$ satisfies the ${\Delta }_2$-condition, the Orlicz space $L^{\Phi }\left({\mathbb{R}}^N\right)$ is the vectorial space of the measurable functions $u:{\mathbb{R}}^N\to \mathbb{R}$ satisfying:

$${\int }_{{\mathbb{R}}^N}\Phi (|u|)dx<+\infty .$$

The space $L^{\Phi }\left({\mathbb{R}}^N\right)$ is a Banach space endowed with the Luxemburg norm:

$${\parallel u\parallel }_{\Phi }={\parallel u\parallel }_{L^{\Phi }\left({\mathbb{R}}^N\right)}≔\inf \left\{\lambda >0:{\int }_{{\mathbb{R}}^N}\Phi \left(\frac{|u|}{\lambda }\right)dx\le 1\right\}.$$

Particularly, when $\Phi \left(t\right)={|t|}^p(p>1)$, the corresponding Orlicz space $L^{\Phi }\left({\mathbb{R}}^N\right)$ reduces to the classical Lebesgue space $L^p\left({\mathbb{R}}^N\right)$ endowed with the norm:

$${\parallel u\parallel }_p=L^p\left({\mathbb{R}}^N\right)≔{\left({\int }_{{\mathbb{R}}^N}{|u\left(x\right)|}^{p}dx\right)}^{\frac{1}{p}}.$$

The fact that $\Phi$ satisfies ${\Delta }_2$-condition implies that:

$$\begin{array}{c}\hfill u_n\to u \mathrm{i}\mathrm{n} L^{\Phi }(\Omega )⟺{\int }_{\Omega }\Phi (|u_n-u|)dx\to 0,\end{array}$$

(7)

where $\Omega$ is an open set of ${\mathbb{R}}^N$. Moreover, by the Young’s inequality (5), the following generalized version of Hölder’s inequality holds (see [33,34]):

$$\begin{array}{c}\hfill \left|{\int }_{{\mathbb{R}}^N}uvdx\right|\le {2\parallel u\parallel }_{\Phi }{\parallel v\parallel }_{\tilde{\Phi }}, \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} u\in L^{\Phi }\left({\mathbb{R}}^N\right),v\in L^{\tilde{\Phi }}\left({\mathbb{R}}^N\right).\end{array}$$

Given an N-function $\Phi$ and a fractional parameter $0<s<1$, we recall the fractional Orlicz–Sobolev space $W^{s,\Phi }\left({\mathbb{R}}^N\right)$ defined as:

$$W^{s,\Phi }\left({\mathbb{R}}^N\right)≔\left\{u\in L^{\Phi }\left({\mathbb{R}}^N\right):\underset{{\mathbb{R}}^{2N}}{\int \int }\Phi (|D_{s}u|)d\mu <+\infty \right\},$$

where $D_{s}u$ is defined by (3) and $d\mu (x,y)≔\frac{dxdy}{{|x-y|}^N}$. The space $W^{s,\Phi }\left({\mathbb{R}}^N\right)$ is a Banach space endowed with the following norm:

$${\parallel u\parallel }_{s,\Phi }={\parallel u\parallel }_{W^{s,\Phi }\left({\mathbb{R}}^N\right)}≔{\parallel u\parallel }_{\Phi }+{\left[u\right]}_{s,\Phi },$$

where the so-called $(s,\Phi )$-Gagliardo semi-norm is defined as:

$${\left[u\right]}_{s,\Phi }≔\inf \left\{\lambda >0:\underset{{\mathbb{R}}^{2N}}{\int \int }\Phi \left(\frac{|D_{s}u|}{\lambda }\right)d\mu \le 1\right\}.$$

The following lemmas will be useful in the following.

Lemma 1. 

(see [33,35]) Assume that Φ is an N-function. Then, the following conditions are equivalent:

(1) 

$$\begin{array}{c}\hfill 1<l=\underset{t>0}{\inf }\frac{t\varphi \left(t\right)}{\Phi \left(t\right)}\le \underset{t>0}{\sup }\frac{t\varphi \left(t\right)}{\Phi \left(t\right)}=m<+\infty ;\end{array}$$

(8)

(2) 

Let ${\zeta }_1\left(t\right)=\min \{t^l,t^m\}$, ${\zeta }_2\left(t\right)=\max \{t^l,t^m\}$, for $t\ge 0$. Then, Φ satisfies:

$${\zeta }_1\left(t\right)\Phi \left(\rho \right)\le \Phi \left(\rho t\right)\le {\zeta }_2\left(t\right)\Phi \left(\rho \right), for all\rho ,t\ge 0;$$

(3) 

Φ satisfies the ${\Delta }_2$-condition.

Lemma 2. 

(see [11,35]) Assume that Φ is an N-function and (8) holds. Then, Φ satisfies:

(1) 

$${\zeta }_1{(\parallel u\parallel }_{\Phi })\le {\int }_{{\mathbb{R}}^N}\Phi (|u|)dx\le {\zeta }_2{(\parallel u\parallel }_{\Phi }), \mathit{for} \mathit{all} u\in L^{\Phi }\left({\mathbb{R}}^N\right);$$

(2) 

$${\zeta }_1\left({\left[u\right]}_{s,\Phi }\right)\le {\int \int }_{{\mathbb{R}}^{2N}}\Phi (|D_{s}u|)d\mu \le {\zeta }_2\left({\left[u\right]}_{s,\Phi }\right), \mathit{for} \mathit{all} u\in W^{s,\Phi }\left({\mathbb{R}}^N\right).$$

Lemma 3. 

(see [35]) Assume that Φ is an N-function and (8) holds with $l>1$. Let $\tilde{\Phi }$ be the complement of Φ and ${\zeta }_3\left(t\right)=\min \{t^{\tilde{l}},t^{\tilde{m}}\}$, ${\zeta }_4\left(t\right)=\max \{t^{\tilde{l}},t^{\tilde{m}}\}$, for $t\ge 0$, where $\tilde{l}≔\frac{l}{l-1}$ and $\tilde{m}≔\frac{m}{m-1}$. Then, $\tilde{\Phi }$ satisfies:

(1) 

$$\tilde{m}=\underset{t>0}{\inf }\frac{t\tilde{\Phi }\prime \left(t\right)}{\tilde{\Phi }\left(t\right)}\le \underset{t>0}{\sup }\frac{t\tilde{\Phi }\prime \left(t\right)}{\tilde{\Phi }\left(t\right)}=\tilde{l};$$

(2) 

$${\zeta }_3\left(t\right)\tilde{\Phi }\left(\rho \right)\le \tilde{\Phi }\left(\rho t\right)\le {\zeta }_4\left(t\right)\tilde{\Phi }\left(\rho \right), \mathit{for} \mathit{all} \rho ,t\ge 0;$$

(3) 

$${\zeta }_3{(\parallel u\parallel }_{\tilde{\Phi }})\le {\int }_{{\mathbb{R}}^N}\tilde{\Phi }(|u|)dx\le {\zeta }_4{(\parallel u\parallel }_{\tilde{\Phi }}), \mathit{for} \mathit{all} u\in L^{\tilde{\Phi }}\left({\mathbb{R}}^N\right).$$

Remark 3. 

By Lemmas 1 and 3, $\left({\varphi }_1\right)$ implies that Φ and $\tilde{\Phi }$ are two N-functions satisfying the ${\Delta }_2$-condition. The fact that Φ and $\tilde{\Phi }$ satisfy the ${\Delta }_2$-condition implies that $L^{\Phi }\left({\mathbb{R}}^N\right)$ and $W^{s,\Phi }\left({\mathbb{R}}^N\right)$ are separable and reflexive Banach spaces. Moreover, $C_c^{\infty }\left({\mathbb{R}}^N\right)$ is dense in $W^{s,\Phi }\left({\mathbb{R}}^N\right)$ (see [5,33,34]).

Next, we recall the Sobolev conjugate function of $\Phi$, which is denoted by ${\Phi }_{*}$. Suppose that:

$$\begin{array}{c}\hfill {\int }_0^1\frac{{\Phi }^{-1}\left(\tau \right)}{{\tau }^{\frac{N+s}{N}}}d\tau <+\infty \mathrm{and} {\int }_1^{+\infty }\frac{{\Phi }^{-1}\left(\tau \right)}{{\tau }^{\frac{N+s}{N}}}d\tau =+\infty .\end{array}$$

(9)

Then, ${\Phi }_{*}$ is defined by:

$${\Phi }_{*}^{-1}\left(t\right)={\int }_0^t\frac{{\Phi }^{-1}\left(\tau \right)}{{\tau }^{\frac{N+s}{N}}}d\tau , \mathit{for} t\ge 0.$$

Lemma 4. 

(see [6,36]) Assume that Φ is an N-function and (8) holds with $l,m\in (1,\frac{N}{s})$. Then, (9) holds. Let ${\zeta }_5\left(t\right)=\min \{t^{l^{*}},t^{m^{*}}\}$, ${\zeta }_6\left(t\right)=\max \{t^{l^{*}},t^{m^{*}}\}$, for $t\ge 0$, where $l^{*}≔\frac{Nl}{N-sl}$, $m^{*}≔\frac{Nm}{N-sm}$. Then, ${\Phi }_{*}$ satisfies:

(1) 

$$l^{*}=\underset{t>0}{\inf }\frac{t{\Phi }_{*}^{\prime }\left(t\right)}{{\Phi }_{*}\left(t\right)}\le \underset{t>0}{\sup }\frac{t{\Phi }_{*}^{\prime }\left(t\right)}{{\Phi }_{*}\left(t\right)}=m^{*};$$

(2) 

$${\zeta }_5\left(t\right){\Phi }_{*}\left(\rho \right)\le {\Phi }_{*}\left(\rho t\right)\le {\zeta }_6\left(t\right){\Phi }_{*}\left(\rho \right), \mathit{for} \mathit{all} \rho ,t\ge 0;$$

(3) 

$${\zeta }_5{(\parallel u\parallel }_{{\Phi }_{*}})\le {\int }_{{\mathbb{R}}^N}{\Phi }_{*}(|u|)dx\le {\zeta }_6{(\parallel u\parallel }_{{\Phi }_{*}}), \mathit{for} \mathit{all} u\in L^{{\Phi }_{*}}\left({\mathbb{R}}^N\right).$$

The conjugate function ${\Phi }_{*}$ plays a crucial role in the following embedding results, which will be used frequently in our proofs.

Lemma 5. 

(see [13,33,36]) Assume that Φ is an N-function and (8) holds with $l,m\in (1,\frac{N}{s})$. Then, the following embedding results hold:

(1) 

the embedding $W^{s,\Phi }\left({\mathbb{R}}^N\right)\hookrightarrow L^{{\Phi }_{*}}\left({\mathbb{R}}^N\right)$ is continuous;

(2) 

the embedding $W^{s,\Phi }\left({\mathbb{R}}^N\right)\hookrightarrow L^{\Phi }\left({\mathbb{R}}^N\right)$ is continuous;

(3) 

the embedding $W^{s,\Phi }\left({\mathbb{R}}^N\right)\hookrightarrow L^{\Psi }\left({\mathbb{R}}^N\right)$ is continuous if Φ dominates Ψ globally;

(4) 

the embedding $W^{s,\Phi }\left({\mathbb{R}}^N\right)\hookrightarrow L^{\Psi }\left({\mathbb{R}}^N\right)$ is continuous if Ψ satisfies the ${\Delta }_2$-condition, $\Psi \prec \prec {\Phi }_{*}$ and

$$\underset{t\to 0^{+}}{\lim }\frac{\Psi \left(t\right)}{\Phi \left(t\right)}=0;$$

(5) 

when ${\mathbb{R}}^N$ is replaced by a $C^{0,1}$ bounded open subset D of ${\mathbb{R}}^N$, then the embedding $W^{s,\Phi }\left(D\right)\hookrightarrow L^{\Psi }\left(D\right)$ is compact if $\Psi \prec \prec {\Phi }_{*}$. Explicitly, when $m<l^{*}$, the embedding $W^{s,\Phi }\left(B_r\right)\hookrightarrow L^{\Phi }\left(B_r\right)$ is compact, where the following is true: $B_r≔\{x\in {\mathbb{R}}^N:|x|<r\}$ for $r>0$.

Notation: Throughout this paper, $C_d$ is used to denote a positive constant which depends only on the constant or function d.

3. Proofs

In fractional Orlicz–Sobolev space $W^{s,\Phi }\left({\mathbb{R}}^N\right)$, denoted by W for simplicity, the energy functional I associated with Equation (1) is defined by:

$$\begin{array}{c}\hfill I\left(u\right)≔{\int \int }_{{\mathbb{R}}^{2N}}\Phi (|D_{s}u|)d\mu +{\int }_{{\mathbb{R}}^N}V\left(x\right)\Phi (|u|)dx-{\int }_{{\mathbb{R}}^N}F(x,u)dx.\end{array}$$

(10)

It follows $\left(f_1\right)$ that for any given constant $\epsilon >0$, there exists a constant $C_{\epsilon }>0$ such that:

$$\begin{array}{c}\hfill |f(x,t)|\le \epsilon \varphi (|t|)+C_{\epsilon }{\Phi }_{*}^{\prime }(|t|) \mathrm{a}\mathrm{n}\mathrm{d} |F(x,t)|\le \epsilon \Phi (|t|)+C_{\epsilon }{\Phi }_{*}(|t|), \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} (x,t)\in {\mathbb{R}}^N\times \mathbb{R}.\end{array}$$

(11)

Thus, by using standard arguments as [8], we have that $I\in C^1(W,\mathbb{R})$ and its derivative is given by:

$$\begin{array}{c}\hfill ⟨I^{\prime }\left(u\right),v⟩={\int \int }_{{\mathbb{R}}^{2N}}a(|D_{s}u|)D_{s}u{D}_{s}vd\mu +{\int }_{{\mathbb{R}}^N}V\left(x\right)a(|u|)uvdx-{\int }_{{\mathbb{R}}^N}f(x,u)vdx, \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} u,v\in W.\end{array}$$

(12)

Thus, the critical points of I are weak solutions of Equation (1).

Define $I_i(i=1,2):W\to \mathbb{R}$ by:

$$\begin{array}{c}\hfill I_1\left(u\right)={\int \int }_{{\mathbb{R}}^{2N}}\Phi (|D_{s}u|)d\mu +{\int }_{{\mathbb{R}}^N}V\left(x\right)\Phi (|u|)dx\end{array}$$

(13)

and:

$$\begin{array}{c}\hfill I_2\left(u\right)={\int }_{{\mathbb{R}}^N}F(x,u)dx.\end{array}$$

(14)

Then:

$$I\left(u\right)=I_1\left(u\right)-I_2\left(u\right), \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} u,v\in W$$

and:

$$\begin{array}{ccc}\hfill ⟨I_1^{\prime }\left(u\right),v⟩& =& {\int \int }_{{\mathbb{R}}^{2N}}a(|D_{s}u|)D_{s}u{D}_{s}vd\mu +{\int }_{{\mathbb{R}}^N}V\left(x\right)a(|u|)uvdx, \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} u,v\in W,\hfill \end{array}$$

(15)

$$\begin{array}{ccc}\hfill ⟨I_2^{\prime }\left(u\right),v⟩& =& {\int }_{{\mathbb{R}}^N}f(x,u)vdx, \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} u,v\in W.\hfill \end{array}$$

(16)

Lemma 6. 

Assume that $\left({\varphi }_1\right)$, $\left(V\right)$ and $\left(f_1\right)$ hold. Then, there exist two constants $\rho ,\eta >0$ such that $I\left(u\right)\ge \eta$ for all $u\in W$ with ${\parallel u\parallel }_{s,\Phi }=\rho$.

Proof. 

When ${\parallel u\parallel }_{s,\Phi }={\parallel u\parallel }_{\Phi }+{\left[u\right]}_{s,\Phi }\le 1$, by (10), $\left(V\right)$, (11) with taking $\epsilon <{\alpha }_1$, Lemma 2, $\left(3\right)$ in Lemma 4 and $\left(1\right)$ in Lemma 5, we have:

$$\begin{array}{ccc}\hfill I\left(u\right)& \ge & {\int \int }_{{\mathbb{R}}^{2N}}\Phi (|D_{s}u|)d\mu +{\alpha }_1{\int }_{{\mathbb{R}}^N}\Phi (|u|)dx-{\int }_{{\mathbb{R}}^N}|F(x,u)|dx\hfill \\ & \ge & {\int \int }_{{\mathbb{R}}^{2N}}\Phi (|D_{s}u|)d\mu +({\alpha }_1-\epsilon ){\int }_{{\mathbb{R}}^N}\Phi (|u|)dx-C_{\epsilon }{\int }_{{\mathbb{R}}^N}{\Phi }_{*}(|u|)dx\hfill \\ & \ge & {\left[u\right]}_{s,\Phi }^m+({\alpha }_1-\epsilon ){\parallel u\parallel }_{\Phi }^m-C_{\epsilon }{\max \{\parallel u\parallel }_{{\Phi }_{*}}^{l^{*}}{,\parallel u\parallel }_{{\Phi }_{*}}^{m^{*}}\}\hfill \\ & \ge & \min \{1,{\alpha }_1-\epsilon \}{C}_m{\parallel u\parallel }_{s,\Phi }^m-C_{\epsilon }{C}_{{\Phi }_{*}}^{l^{*}}{\parallel u\parallel }_{s,\Phi }^{l^{*}}-C_{\epsilon }{C}_{{\Phi }_{*}}^{m^{*}}{\parallel u\parallel }_{s,\Phi }^{m^{*}}.\hfill \end{array}$$

Taking into account that $m<l^{*}\le m^{*}$, it follows from the aforementioned inequality that there exist sufficiently small positive constants $\rho$ and $\eta$ such that $I\left(u\right)\ge \eta$ for all $u\in W$ with ${\parallel u\parallel }_{s,\Phi }=\rho$.  □

Lemma 7. 

Assume that $\left({\varphi }_1\right)$, $\left(V\right)$, $\left(f_1\right)$ and $\left(f_2\right)$ (or $\left(f_4\right)$) hold. Then, there exists a $u_0\in W$ such that $I\left(t{u}_0\right)\to -\infty$ as $t\to +\infty$.

Proof. 

For any given constant $M>{\alpha }_2$, by $\left(f_1\right)$ and $\left(f_2\right)$ (or combine $\left(f_4\right)$ with $\left(2\right)$ in Lemma 1), there exists a constant $C_M>0$ such that:

$$\begin{array}{c}\hfill F(x,t)\ge M\Phi (|t|)-C_M, \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l}(x,t)\in {\mathbb{R}}^N\times \mathbb{R}.\end{array}$$

(17)

Now, choose $u_0\in C_c^{\infty }\left(B_r\right)\setminus \left\{\mathbf{0}\right\}$ with $0\le u_0\left(x\right)\le 1$. Then $u_0\in W,$ and by (10), $\left(V\right)$, (17), $\left(2\right)$ in Lemma 1 and the fact $F(x,0)=0$ for all $x\in {\mathbb{R}}^N$, when $t>0$ we have:

$$\begin{array}{ccc}\hfill I\left(t{u}_0\right)& =& {\int \int }_{{\mathbb{R}}^{2N}}\Phi (|D_s\left(t{u}_0\right)|)d\mu +{\int }_{{\mathbb{R}}^N}V\left(x\right)\Phi (|t{u}_0|)dx-{\int }_{{\mathbb{R}}^N}F(x,t{u}_0)dx\hfill \\ & =& {\int \int }_{{\mathbb{R}}^{2N}}\Phi (t|D_s{u}_0|)d\mu +{\int }_{{\mathbb{R}}^N}V\left(x\right)\Phi (t|u_0|)dx-{\int }_{B_r}F(x,t{u}_0)dx\hfill \\ & \le & \Phi \left(t\right){\int \int }_{{\mathbb{R}}^{2N}}\max \{|D_s{u}_0{|}^l,|D_s{u}_0{|}^m\}d\mu +{\alpha }_2{\int }_{{\mathbb{R}}^N}\Phi (t|u_0|)dx-M{\int }_{B_r}\Phi (t|u_0|)+C_M|B_r|\hfill \\ & \le & \Phi \left(t\right){\int \int }_{{\mathbb{R}}^{2N}}(|D_s{u}_0{|}^l+|D_s{u}_0{|}^m)d\mu -(M-{\alpha }_2)\Phi \left(t\right){\int }_{{\mathbb{R}}^N}\min \{|u_0{|}^l,|u_0{|}^m\}dx+C_M|B_r|\hfill \\ & =& \Phi \left(t\right)\left[{\int \int }_{{\mathbb{R}}^{2N}}(|D_s{u}_0{|}^l+|D_s{u}_0{|}^m)d\mu -(M-{\alpha }_2){\parallel u_0\parallel }_m^m\right]+C_M|B_r|.\hfill \end{array}$$

Note that $\underset{t\to +\infty }{lim}\Phi \left(t\right)=+\infty$. We can choose $M>\frac{1}{\parallel u_0{\parallel }_m^m}\{\underset{{\mathbb{R}}^{2N}}{\int \int }(|D_s{u}_0{|}^l+|D_s{u}_0{|}^m\left)d\mu \right\}+{\alpha }_2$ such that $I\left(t{u}_0\right)\to -\infty$ as $t\to +\infty$. What needs to be pointed out is that here we used the fact that $u_0\in W^{s,\Psi }\left({\mathbb{R}}^N\right)$, where $\Psi \left(t\right)={|t|}^l+{|t|}^m,t\ge 0.$ So, ${\int \int }_{{\mathbb{R}}^{2N}}(|D_s{u}_0{|}^l+|D_s{u}_0{|}^m)d\mu <+\infty$.  □

Lemmas 6 and 7 and the fact that $I\left(\mathbf{0}\right)=0$ show that the energy functional I has a mountain pass geometry; that is, setting:

$$\Gamma =\{\gamma \in C([0,1],W):\gamma \left(0\right)=\mathbf{0},\parallel \gamma \left(1\right){\parallel }_{s,\Phi }>\rho \mathrm{a}\mathrm{n}\mathrm{d} I\left(\gamma \left(1\right)\right)\le 0\},$$

we have $\Gamma \ne \varnothing$. Then, by using the variant version of the mountain pass theorem (see Theorem 3 in [30]), we deduce that I possesses a ${\left(C\right)}_c$-sequence $\left\{u_n\right\}$ with the level $c\ge \eta >0$ given by:

$$\begin{array}{c}\hfill c=\underset{\gamma \in \Gamma }{\inf }\underset{t\in [0,1]}{\max }I\left(\gamma \left(t\right)\right).\end{array}$$

(18)

We recall that ${\left(C\right)}_c$-sequence $\left\{u_n\right\}$ of I in W means

$$\begin{array}{c}\hfill I\left(u_n\right)\to c \mathrm{a}\mathrm{n}\mathrm{d} (1+\parallel u_n{\parallel }_{s,\Phi })\parallel I^{\prime }\left(u_n\right){\parallel }_{W^{*}}\to 0, \mathrm{a}\mathrm{s} n\to \infty .\end{array}$$

(19)

To prove the boundedness of the ${\left(C\right)}_c$-sequence $\left\{u_n\right\}$ of I in W, we will use the Lions’ type result for fractional Orlicz–Sobolev spaces (see Theorem 1.6 in [13]). We note that the claim $u_n\rightharpoonup 0$ in X of Theorem 1.6 in [13] is not necessary. With the same proof as Theorem 1.6 in [13], we can get the following result.

Lemma 8. 

(Lions’ type result for fractional Orlicz–Sobolev spaces). Suppose that the function ϕ defined by (2) satisfies $\left({\varphi }_1\right)$ and:

$$\underset{t\to 0^{+}}{\lim }\frac{\Psi \left(t\right)}{\Phi \left(t\right)}=0.$$

Let $\left\{u_n\right\}$ be a bounded sequence in $W^{s,\Phi }\left({\mathbb{R}}^N\right)$ in such a way that:

$$\underset{n\to \infty }{\lim }\underset{y\in {\mathbb{R}}^N}{\sup }{\int }_{B_r\left(y\right)}\Phi (|u_n|)dx=0,$$

for some $r>0$. Then, $u_n\to \mathbf{0}$ in $L^{\Psi }\left({\mathbb{R}}^N\right)$, where Ψ is an N-function such that $\Psi \prec \prec {\Phi }_{*}$.

Lemma 9. 

Assume that $\left({\varphi }_1\right)$, $\left({\varphi }_2\right)$, $\left(V\right)$ and $\left(f_1\right)$-$\left(f_3\right)$ hold. Then, any ${\left(C\right)}_c$-sequence of I in W is bounded for all $c\ge 0$.

Proof. 

Let $\left\{u_n\right\}$ be a ${\left(C\right)}_c$-sequence of I in W for $c\ge 0$. By (19), we have:

$$\begin{array}{c}\hfill I\left(u_n\right)\to c \mathrm{a}\mathrm{n}\mathrm{d}\left|⟨I^{\prime }\left(u_n\right),\frac{1}{m}{u}_n⟩\right|\to 0,\mathrm{a}\mathrm{s} n\to \infty .\end{array}$$

(20)

Then, by (10), (12), $\left({\varphi }_1\right)$, and $\left(V\right)$, for n large, we have:

$$\begin{array}{ccc}\hfill c+1& \ge & I\left(u_n\right)-⟨I^{\prime }\left(u_n\right),\frac{1}{m}{u}_n⟩\hfill \\ & =& {\int \int }_{{\mathbb{R}}^{2N}}\left(\Phi (|D_s{u}_n|)-\frac{1}{m}a(|D_s{u}_n|)|D_s{u}_n{|}^2\right)d\mu \hfill \\ & & +{\int }_{{\mathbb{R}}^N}V\left(x\right)\left(\Phi (|u_n|)-\frac{1}{m}a(|u_n|)u_n^2\right)dx\hfill \\ & & +{\int }_{{\mathbb{R}}^N}\left(\frac{1}{m}{u}_{n}f(x,u_n)-F(x,u_n)\right)dx\hfill \\ & \ge & \frac{1}{m}{\int }_{{\mathbb{R}}^N}\widehat{F}(x,u_n)dx.\hfill \end{array}$$

(21)

To prove the boundedness of $\left\{u_n\right\}$, arguing by contradiction, we suppose that there exists a subsequence of $\left\{u_n\right\}$, still denoted by $\left\{u_n\right\}$, such that $\parallel u_n{\parallel }_{s,\Phi }\to \infty$, as $n\to \infty$. Let ${\tilde{u}}_n=\frac{u_n}{\parallel u_n{\parallel }_{s,\Phi }}$. Then $\parallel {\tilde{u}}_n{\parallel }_{s,\Phi }=1$.

Firstly, we claim that:

$$\begin{array}{c}\hfill {\lambda }_1≔\underset{n\to \infty }{\lim }\underset{y\in {\mathbb{R}}^N}{\sup }{\int }_{B_2\left(y\right)}\Phi (|{\tilde{u}}_n|)dx=0.\end{array}$$

(22)

Indeed, if ${\lambda }_1\ne 0$, there exist a constant $\delta >0$, a subsequence of $\left\{{\tilde{u}}_n\right\}$, still denoted by $\left\{{\tilde{u}}_n\right\}$, and a sequence $\left\{z_n\right\}\in {\mathbb{Z}}^N$ such that:

$$\begin{array}{c}\hfill {\int }_{B_2\left(z_n\right)}\Phi (|{\tilde{u}}_n|)dx>\delta , \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} n\in \mathbb{N}.\end{array}$$

(23)

Let ${\overline{u}}_n={\tilde{u}}_n(·+z_n)$. Then $\parallel {\overline{u}}_n{\parallel }_{s,\Phi }={\parallel {\tilde{u}}_n\parallel }_{s,\Phi }=1$, that is, $\left\{{\overline{u}}_n\right\}$ is bounded in W. Passing to a subsequence of $\left\{{\overline{u}}_n\right\}$, still denoted by $\left\{{\overline{u}}_n\right\}$, by Remark 3 and $\left(5\right)$ in Lemma 5, we can assume that there exists a $\overline{u}\in W$ such that:

$$\begin{array}{c}\hfill {\overline{u}}_n\rightharpoonup \overline{u} \mathrm{i}\mathrm{n} W, {\overline{u}}_n\to \overline{u} \mathrm{i}\mathrm{n} L^{\Phi }\left(B_2\right) \mathrm{a}\mathrm{n}\mathrm{d} {\overline{u}}_n\left(x\right)\to \overline{u}\left(x\right)\mathrm{a}.\mathrm{e}. \mathrm{i}\mathrm{n} B_2.\end{array}$$

(24)

Note that:

$$\begin{array}{c}\hfill {\int }_{B_2}\Phi (|{\overline{u}}_n|)dx={\int }_{B_2\left(z_n\right)}\Phi (|{\tilde{u}}_n|)dx.\end{array}$$

Then, by (23), (24), and (7), we obtain that $\overline{u}\ne \mathbf{0}$ in $L^{\Phi }\left(B_2\right)$, that is, $[\overline{u}\ne 0]≔\{x\in B_2:\overline{u}\left(x\right)\ne 0\}$ has non-zero Lebesgue measure. Let $u_n^{*}=u_n(·+z_n)$. Then $\parallel u_n^{*}{\parallel }_{s,\Phi }={\parallel u_n\parallel }_{s,\Phi }$, and it follows from the fact that V and f are 1-periodic in x that:

$$\begin{array}{c}\hfill I\left(u_n^{*}\right)=I\left(u_n\right) \mathrm{a}\mathrm{n}\mathrm{d} \parallel I^{\prime }\left(u_n^{*}\right){\parallel }_{W^{*}}={\parallel I^{\prime }\left(u_n\right)\parallel }_{W^{*}}, \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} n\in \mathbb{N},\end{array}$$

which imply that $\left\{u_n^{*}\right\}$ is also a ${\left(C\right)}_c$-sequence of I. Then, by (21), for n large, we have:

$$\begin{array}{c}\hfill {\int }_{{\mathbb{R}}^N}\widehat{F}(x,u_n^{*})dx\le m(c+1).\end{array}$$

(25)

However, by $\left(2\right)$ in Lemma 1, $\left(f_2\right)$ and $\left(f_3\right)$ imply:

$$\begin{array}{c}\hfill \underset{|t|\to \infty }{\lim }\widehat{F}(x,t)=+\infty , \mathrm{u}\mathrm{n}\mathrm{i}\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{m}\mathrm{l}\mathrm{y} \mathrm{i}\mathrm{n} x\in {\mathbb{R}}^N.\end{array}$$

(26)

Moreover, by (24), ${\overline{u}}_n={\tilde{u}}_n(·+z_n)=\frac{u_n(·+z_n)}{\parallel u_n{\parallel }_{s,\Phi }}=\frac{u_n^{*}}{\parallel u_n{\parallel }_{s,\Phi }}$ implies:

$$\begin{array}{c}\hfill |u_n^{*}\left(x\right)|=|{\overline{u}}_n\left(x\right)|\parallel u_n{\parallel }_{s,\Phi }\to \infty , \mathrm{a}.\mathrm{e}. x\in [\overline{u}\ne 0].\end{array}$$

(27)

Then, it follows from $\left(f_3\right)$, (26), (27) and Fatou’s Lemma that:

$$\begin{array}{c}\hfill {\int }_{{\mathbb{R}}^N}\widehat{F}(x,u_n^{*})dx\ge {\int }_{[\overline{u}\ne 0]}\widehat{F}(x,u_n^{*})dx\to +\infty , \mathrm{as} n\to \infty ,\end{array}$$

which contradicts (25). Therefore, ${\lambda }_1=0$, and thus, (22) holds.

Next, for given $p\in (l,l^{*})$ and $c>0$, by $\left({\varphi }_1\right)$, $\left({\varphi }_2\right)$ and $2)$ in Lemma 4, we have:

$$\begin{array}{c}\hfill \underset{t\to 0^{+}}{\lim }\frac{t^p}{\Phi \left(t\right)}=0 \mathrm{and} \underset{t\to +\infty }{\lim }\frac{{\left(ct\right)}^p}{{\Phi }_{*}\left(t\right)}\le \underset{t\to +\infty }{\lim }\frac{c^p{t}^p}{{\Phi }_{*}\left(1\right)\min \{t^{l^{*}},t^{m^{*}}\}}=0.\end{array}$$

(28)

Then, by Lemma 8, (22) and (28) imply that:

$$\begin{array}{c}\hfill {\tilde{u}}_n\to \mathbf{0} \mathrm{in} L^p\left({\mathbb{R}}^N\right), \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} p\in (l,l^{*}).\end{array}$$

(29)

In addition, let $\Psi ={|t|}^l,t\ge 0.$ Combining $\left({\varphi }_1\right)$ and $\left({\varphi }_2\right)$ with Lemma 1, we can easily check that $\Phi$ dominates $\Psi$ globally. Then, it follows from $3)$ in Lemma 5 that the embedding $W\hookrightarrow L^l\left({\mathbb{R}}^N\right)$ is continuous, which implies that there exists a constant $M_1>0$ such that:

$$\begin{array}{c}\hfill \parallel {\tilde{u}}_n{\parallel }_l^l\le M_1, \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} n\in \mathbb{N}.\end{array}$$

(30)

Finally, to get a contradiction, we will divide both sides of formula $I\left(u_n\right)=I_1\left(u_n\right)-I_2\left(u_n\right)$ by $\parallel u_n{\parallel }_{s,{\Phi }_1}^l$ and let $n\to \infty$. On the ond hand, by (20), it is clear that:

$$\begin{array}{c}\hfill \frac{I\left(u_n\right)}{\parallel u_n{\parallel }_{s,\Phi }^l}\to 0, \mathrm{as} n\to \infty .\end{array}$$

(31)

On the other hand, by (13), $\left(V\right)$ and Lemma 2, we have:

$$\begin{array}{ccc}\hfill \frac{I_1\left(u_n\right)}{\parallel u_n{\parallel }_{s,\Phi }^l}& =& \frac{1}{\parallel u_n{\parallel }_{s,\Phi }^l}\{\underset{{\mathbb{R}}^{2N}}{\int \int }\Phi (|D_s{u}_n|)d\mu +\underset{{\mathbb{R}}^N}{\int }V\left(x\right)\Phi (|u_n|)dx\}\hfill \\ & \ge & \frac{\min \{{\left[u_n\right]}_{s,\Phi }^l,{\left[u_n\right]}_{s,\Phi }^m\}+{\alpha }_1\min \{\parallel u_n{\parallel }_{\Phi }^l,\parallel u_n{\parallel }_{\Phi }^m\}}{\parallel u_n{\parallel }_{s,\Phi }^l}\hfill \\ & \ge & \frac{{\left[u_n\right]}_{s,\Phi }^l+{\alpha }_1{\parallel u_n\parallel }_{\Phi }^l-1-{\alpha }_1}{\parallel u_n{\parallel }_{s,\Phi }^l}\hfill \\ & \ge & \frac{\min \{1,{\alpha }_1\}{C}_l({\left[u_n\right]}_{s,\Phi }+\parallel u_n{{\parallel }_{\Phi })}^l-1-{\alpha }_1}{\parallel u_n{\parallel }_{s,\Phi }^l}\to \min \{1,{\alpha }_1\}{C}_l,\mathrm{as} n\to \infty .\hfill \end{array}$$

(32)

Moreover, by $\left(2\right)$ in Lemma 1, $\left(f_1\right)$ implies that:

$$\begin{array}{c}\hfill \underset{|t|\to 0}{\lim }\frac{F(x,t)}{{|t|}^l}=0, \mathrm{u}\mathrm{n}\mathrm{i}\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{m}\mathrm{l}\mathrm{y} \mathrm{i}\mathrm{n} x\in {\mathbb{R}}^N.\end{array}$$

Then, for any given constant $\epsilon >0$, there exists a constant $R_{\epsilon }>0$ such that:

$$\begin{array}{c}\hfill \frac{|F(x,t)|}{{|t|}^l}\le \epsilon , \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} x\in {\mathbb{R}}^N,|t|\le R_{\epsilon }.\end{array}$$

(33)

For the above $R_{\epsilon }>0$, by $\left(f_1\right)$ and $\left(f_3\right)$, there exists a constant $C_R>0$ such that:

$$\begin{array}{c}\hfill {\left(\frac{|F(x,t)|}{{|t|}^l}\right)}^k\le C_R\widehat{F}(x,t), \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} x\in {\mathbb{R}}^N,|t|>R_{\epsilon }.\end{array}$$

(34)

Let:

$$X_n=\{x\in {\mathbb{R}}^N:|u_n\left(x\right)|\le R_{\epsilon }\} \mathrm{and} Y_n=\{x\in {\mathbb{R}}^N:|u_n\left(x\right)|>R_{\epsilon }\}.$$

Then:

$$\begin{array}{c}\hfill \frac{|I_2\left(u_n\right)|}{\parallel u_n{\parallel }_{s,\Phi }^l}\le {\int }_{X_n}\frac{|F(x,u_n)|}{\parallel u_n{\parallel }_{s,\Phi }^l}dx+{\int }_{Y_n}\frac{|F(x,u_n)|}{\parallel u_n{\parallel }_{s,\Phi }^l}dx.\end{array}$$

(35)

By (33) and (30), we have:

$$\begin{array}{c}\hfill {\int }_{X_n}\frac{|F(x,u_n)|}{\parallel u_n{\parallel }_{s,\Phi }^l}dx={\int }_{X_n}\frac{|F(x,u_n)|}{|u_n{|}^l}|{\tilde{u}}_n{|}^{l}dx\le \epsilon {\parallel {\tilde{u}}_n\parallel }_l^l\le \epsilon M_1.\end{array}$$

(36)

The claim $k>\frac{N}{sl}$ given by $\left(f_3\right)$ implies that $\frac{lk}{k-1}\in (l,l^{*})$. Hence, by Hölder’s inequality, (34), (21), (29), and the fact that $\widehat{F}(x,t)\ge 0$, we have:

$$\begin{array}{ccc}\hfill {\int }_{Y_n}\frac{|F(x,u_n)|}{\parallel u_n{\parallel }_{s,\Phi }^l}dx& =& {\int }_{Y_n}\frac{|F(x,u_n)|}{|u_n{|}^l}{|{\tilde{u}}_n|}^{l}dx\hfill \\ & \le & {\left({\int }_{Y_n}{\left(\frac{|F(x,u_n)|}{|u_n{|}^l}\right)}^{k}dx\right)}^{\frac{1}{k}}{\left({\int }_{Y_n}{|{\tilde{u}}_n|}^{\frac{lk}{k-1}}dx\right)}^{\frac{k-1}{k}}\hfill \\ & \le & {\left({\int }_{Y_n}{C}_R\widehat{F}(x,u_n)dx\right)}^{\frac{1}{k}}{\parallel {\tilde{u}}_n\parallel }_{\frac{lk}{k-1}}^l\hfill \\ & \le & {\left[C_{R}m(c+1)\right]}^{\frac{1}{k}}{\parallel {\tilde{u}}_n\parallel }_{\frac{lk}{k-1}}^l\to 0, \mathrm{as} n\to \infty .\hfill \end{array}$$

(37)

Since $\epsilon$ is arbitrary, it follows from (35), (36), and (37) that:

$$\begin{array}{c}\hfill \frac{I_2\left(u_n\right)}{\parallel u_n{\parallel }_{s,\Phi }^l}\to 0, \mathrm{as} n\to \infty .\end{array}$$

(38)

By dividing both sides of formula $I\left(u_n\right)=I_1\left(u_n\right)-I_2\left(u_n\right)$ by $\parallel u_n{\parallel }_{s,{\Phi }_1}^l$ and letting $n\to \infty$, we get a contradiction via (31), (32), and (38). Therefore, the ${\left(C\right)}_c$-sequence $\left\{u_n\right\}$ is bounded.  □

Lemma 10. 

Assume that $\left({\varphi }_1\right)$, $\left(V\right)$, $\left(f_1\right)$, $\left(f_4\right)$ and $\left(f_5\right)$ are satisfied. Then, for $u\in W$, it holds that:

$$I\left(u\right)\ge I\left(tu\right)+\frac{1-t^l}{m}⟨I^{\prime }\left(u\right),u⟩, for all t\in [0,{\theta }_0],$$

where ${\theta }_0$ is given in $\left(f_5\right)$.

Proof. 

When $u\in W$, $0\le t\le 1$, by (10), (12), and Lemma 1, we have:

$$\begin{array}{ccc}& & I\left(u\right)-I\left(tu\right)-\frac{1-t^l}{m}⟨I^{\prime }\left(u\right),u⟩\hfill \\ & =& {\int \int }_{{\mathbb{R}}^{2N}}\Phi (|D_{s}u|)d\mu +{\int }_{{\mathbb{R}}^N}V\left(x\right)\Phi (|u|)dx-{\int }_{{\mathbb{R}}^N}F(x,u)dx\hfill \\ & & -{\int \int }_{{\mathbb{R}}^{2N}}\Phi (|D_{s}tu|)d\mu -{\int }_{{\mathbb{R}}^N}V\left(x\right)\Phi (|tu|)dx+{\int }_{{\mathbb{R}}^N}F(x,tu)dx\hfill \\ & & -\frac{1-t^l}{m}{\int \int }_{{\mathbb{R}}^{2N}}a(|D_{s}u|)|D_s{u|}^{2}d\mu -\frac{1-t^l}{m}{\int }_{{\mathbb{R}}^N}V\left(x\right)a(|u|)u^{2}dx+\frac{1-t^l}{m}{\int }_{{\mathbb{R}}^N}uf(x,u)dx\hfill \\ & \ge & {\int \int }_{{\mathbb{R}}^{2N}}\Phi (|D_{s}u|)d\mu -\max \{t^l,t^m\}{\int \int }_{{\mathbb{R}}^{2N}}\Phi (|D_{s}u|)d\mu -(1-t^l){\int \int }_{{\mathbb{R}}^{2N}}\Phi (|D_{s}u|)d\mu \hfill \\ & & +{\int }_{{\mathbb{R}}^N}V\left(x\right)\Phi (|u|)dx-\max \{t^l,t^m\}{\int }_{{\mathbb{R}}^N}V\left(x\right)\Phi (|u|)dx-(1-t^l){\int }_{{\mathbb{R}}^N}V\left(x\right)\Phi (|u|)dx\hfill \\ & & +{\int }_{{\mathbb{R}}^N}\left[\frac{1-t^l}{m}uf(x,u)-F(x,u)+F(x,tu)\right]dx\hfill \\ & =& {\int }_{{\mathbb{R}}^N}\left[\frac{1-t^l}{m}uf(x,u)-{\int }_{tu}^{u}f(x,\tau )d\tau \right]dx.\hfill \end{array}$$

Then, it follows from $\left(f_5\right)$ that:

$$I\left(u\right)\ge I\left(tu\right)+\frac{1-t^l}{m}⟨I^{\prime }\left(u\right),u⟩, \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} t\in [0,{\theta }_0],$$

for some ${\theta }_0\in (0,1)$.  □

Lemma 11. 

Assume that $\left({\varphi }_1\right)$, $\left(V\right)$, $\left(f_1\right)$, $\left(f_4\right)$ and $\left(f_5\right)$ hold. Then any ${\left(C\right)}_c$-sequence of I in W is bounded for all $c\ge 0$.

Proof. 

Let $\left\{u_n\right\}$ be a ${\left(C\right)}_c$-sequence of I in W for $c\ge 0$. By (19), we have:

$$\begin{array}{c}\hfill I\left(u_n\right)\to c\mathrm{and}\left|⟨I^{\prime }\left(u_n\right),u_n⟩\right|\to 0,\mathrm{as} n\to \infty .\end{array}$$

(39)

To prove the boundedness of $\left\{u_n\right\}$, arguing by contradiction, we suppose that there exists a subsequence of $\left\{u_n\right\}$, still denoted by $\left\{u_n\right\}$, such that $\parallel u_n{\parallel }_{s,\Phi }\to \infty$, as $n\to \infty$. Let ${\tilde{u}}_n=\frac{u_n}{\parallel u_n{\parallel }_{s,\Phi }}$. Then $\parallel {\tilde{u}}_n{\parallel }_{s,\Phi }=1$.

Firstly, we claim that:

$$\begin{array}{c}\hfill {\lambda }_2≔\underset{n\to \infty }{\lim }\underset{y\in {\mathbb{R}}^N}{\sup }{\int }_{B_2\left(y\right)}\Phi (|{\tilde{u}}_n|)dx=0.\end{array}$$

(40)

Indeed, if ${\lambda }_2\ne 0$, there exist a constant $\delta >0$, a subsequence of $\left\{{\tilde{u}}_n\right\}$, still denoted by $\left\{{\tilde{u}}_n\right\}$, and a sequence $\left\{z_n\right\}\in {\mathbb{Z}}^N$ such that:

$$\begin{array}{c}\hfill \underset{B_2\left(z_n\right)}{\int }\Phi (|{\tilde{u}}_n|)dx>\delta , \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} n\in \mathbb{N}.\end{array}$$

(41)

Let ${\overline{u}}_n={\tilde{u}}_n(·+z_n)$. Then $\parallel {\overline{u}}_n{\parallel }_{s,\Phi }={\parallel {\tilde{u}}_n\parallel }_{s,\Phi }=1$, that is, $\left\{{\overline{u}}_n\right\}$ is bounded in W. Passing to a subsequence of $\left\{{\overline{u}}_n\right\}$, still denoted by $\left\{{\overline{u}}_n\right\}$, by Remark 3 and $\left(5\right)$ in Lemma 5, we can assume that there exists a $\overline{u}\in W$ such that:

$$\begin{array}{c}\hfill {\overline{u}}_n\rightharpoonup \overline{u} \mathrm{in} W, {\overline{u}}_n\to \overline{u} \mathrm{in} L^{\Phi }\left(B_2\right) \mathrm{and} {\overline{u}}_n\left(x\right)\to \overline{u}\left(x\right)\mathrm{a}.\mathrm{e}. \mathrm{in} B_2.\end{array}$$

(42)

Note that:

$$\begin{array}{c}\hfill {\int }_{B_2}\Phi (|{\overline{u}}_n|)dx={\int }_{B_2\left(z_n\right)}\Phi (|{\tilde{u}}_n|)dx.\end{array}$$

Then, by (41), (42), and (7), we obtain that $\overline{u}\ne \mathbf{0}$ in $L^{\Phi }\left(B_2\right)$, that is, $[\overline{u}\ne 0]≔\{x\in B_2:\overline{u}\left(x\right)\ne 0\}$ has non-zero Lebesgue measure. Let $u_n^{*}=u_n(·+z_n)$. Then $\parallel u_n^{*}{\parallel }_{s,\Phi }={\parallel u_n\parallel }_{s,\Phi }$, and:

$$\begin{array}{c}\hfill |u_n^{*}\left(x\right)|=|{\overline{u}}_n\left(x\right)|\parallel u_n{\parallel }_{s,\Phi }\to \infty , \mathrm{a}.\mathrm{e}. x\in [\overline{u}\ne 0].\end{array}$$

(43)

Then, it follows from (14), $\left(f_4\right)$, (43) and Fatou’s Lemma that:

$$\begin{array}{ccc}\hfill \frac{I_2\left(u_n\right)}{\parallel u_n{\parallel }_{s,\Phi }^m}& =& {\int }_{{\mathbb{R}}^N}\frac{F(x,u_n)}{\parallel u_n{\parallel }_{s,\Phi }^m}dx\hfill \\ & =& {\int }_{{\mathbb{R}}^N}\frac{F(x+z_n,u_n^{*})}{|u_n^{*}{|}^m}{|{\overline{u}}_n|}^{m}dx\hfill \\ & \ge & {\int }_{[\overline{u}\ne 0]}\frac{F(x+z_n,u_n^{*})}{|u_n^{*}{|}^m}{|{\overline{u}}_n|}^{m}dx\to +\infty , \mathrm{as} n\to \infty .\hfill \end{array}$$

(44)

Moreover, it follows from (13), $\left(V\right)$, and Lemma 2 that:

$$\begin{array}{ccc}\hfill \underset{n\to \infty }{\mathrm{lim\; sup}}\frac{I_1\left(u_n\right)}{\parallel u_n{\parallel }_{s,\Phi }^m}& =& \underset{n\to \infty }{\mathrm{lim\; sup}}\frac{1}{\parallel u_n{\parallel }_{s,\Phi }^m}\{\underset{{\mathbb{R}}^{2N}}{\int \int }\Phi (|D_s{u}_n|)d\mu +\underset{{\mathbb{R}}^N}{\int }V\left(x\right)\Phi (|u_n|)dx\}\hfill \\ & \le & \underset{n\to \infty }{\mathrm{lim\; sup}}\frac{\max \{{\left[u_n\right]}_{s,\Phi }^l,{\left[u_n\right]}_{s,\Phi }^m\}+{\alpha }_2\max \{\parallel u_n{\parallel }_{\Phi }^l,\parallel u_n{\parallel }_{\Phi }^m\}}{\parallel u_n{\parallel }_{s,\Phi }^m}\hfill \\ & \le & 1+{\alpha }_2.\hfill \end{array}$$

(45)

By dividing both sides of formula $I\left(u_n\right)=I_1\left(u_n\right)-I_2\left(u_n\right)$ by $\parallel u_n{\parallel }_{s,{\Phi }_1}^m$ and letting $n\to \infty$, we get a contradiction via (39), (44), and (45). Therefore, ${\lambda }_2=0$ and thus (40) holds. Then, by using the Lions’ type result for fractional Orlicz–Sobolev spaces, with the similar discussion as in Lemma 9, we have:

$$\begin{array}{c}\hfill {\tilde{u}}_n\to \mathbf{0} \mathrm{in} L^p\left({\mathbb{R}}^N\right), \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} p\in (m,l^{*}).\end{array}$$

(46)

Besides, it follows from $\left(1\right)$ in Lemma 2, $\left(3\right)$ in Lemma 4, $\left(1\right)$–$\left(2\right)$ in Lemma 5 and the fact $\parallel {\tilde{u}}_n{\parallel }_{s,\Phi }=1$ that there exists a constant $M_2>0$ such that:

$$\begin{array}{ccc}\hfill & & {\int }_{{\mathbb{R}}^N}\left(\Phi (|{\tilde{u}}_n|)+{\Phi }_{*}(|{\tilde{u}}_n|)\right)dx\hfill \\ & \le & \max \left\{\parallel {\tilde{u}}_n{\parallel }_{\Phi }^l,{\parallel {\tilde{u}}_n\parallel }_{\Phi }^m\right\}+\max \left\{\parallel {\tilde{u}}_n{\parallel }_{{\Phi }_{*}}^{l^{*}},{\parallel {\tilde{u}}_n\parallel }_{{\Phi }_{*}}^{m^{*}}\right\}\hfill \\ & \le & M_2, \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} n\in \mathbb{N}.\hfill \end{array}$$

(47)

Next, for any given $R>1$, let $t_n=\frac{R}{\parallel u_n{\parallel }_{s,\Phi }}$. Since $\parallel u_n{\parallel }_{s,\Phi }\to \infty$ as $n\to \infty$, it follows that $t_n\in (0,{\theta }_0)$ for n large enough. Thus, by (39) and Lemma 10, we have:

$$\begin{array}{ccc}\hfill c+o_n\left(1\right)& =& I\left(u_n\right)\hfill \\ & \ge & I\left(t_n{u}_n\right)+\frac{1-t_n^l}{m}⟨I^{\prime }\left(u_n\right),u_n⟩\hfill \\ & =& I\left(\frac{R}{\parallel u_n{\parallel }_{s,\Phi }}{u}_n\right)+o_n\left(1\right)\hfill \\ & =& I\left(R{\tilde{u}}_n\right)+o_n\left(1\right)\hfill \\ & =& I_1\left(R{\tilde{u}}_n\right)-I_2\left(R{\tilde{u}}_n\right)+o_n\left(1\right).\hfill \end{array}$$

(48)

For the above R and any given $\epsilon >0$, by $\left(f_1\right)$, the continuity of F and the fact that $\Phi$ and ${\Phi }_{*}$ satisfy the ${\Delta }_2$-condition, there exist constants $C_{\epsilon }>0$ and $p\in (m,l^{*})$ such that:

$$\begin{array}{c}\hfill |F(x,Rt)|\le \epsilon (\Phi (|t|)+{\Phi }_{*}(|t|))+C_{\epsilon }{|t|}^p, \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} (x,t)\in {\mathbb{R}}^N\times \mathbb{R}.\end{array}$$

(49)

Then, by (14), (46), (47), and (49), we have:

$$\begin{array}{ccc}\hfill |I_2\left(R{\tilde{u}}_n\right)|& \le & {\int }_{{\mathbb{R}}^N}|F(x,R{\tilde{u}}_n)|dx\hfill \\ & \le & \epsilon {\int }_{{\mathbb{R}}^N}(\Phi (|{\tilde{u}}_n|)+{\Phi }_{*}(|{\tilde{u}}_n|))dx+C_{\epsilon }{\int }_{{\mathbb{R}}^N}{|{\tilde{u}}_n|}^{p}dx\hfill \\ & \le & \epsilon M_2+o_n\left(1\right).\hfill \end{array}$$

(50)

Since $\epsilon >0$ is arbitrary, (50) implies that:

$$\begin{array}{c}\hfill I_2\left(R{\tilde{u}}_n\right)=o_n\left(1\right).\end{array}$$

(51)

Moreover, for the above $R>1$, by (13), Lemma 1 and the fact $\parallel {\tilde{u}}_n{\parallel }_{s,\Phi }={\parallel {\tilde{u}}_n\parallel }_{\Phi }+{\left[{\tilde{u}}_n\right]}_{s,\Phi }=1$, we have:

$$\begin{array}{ccc}\hfill I_1\left(R{\tilde{u}}_n\right)& =& {\int \int }_{{\mathbb{R}}^{2N}}\Phi (|D_s\left(R{\tilde{u}}_n\right)|)d\mu +{\int }_{{\mathbb{R}}^N}V\left(x\right)\Phi (|R{\tilde{u}}_n|)dx\hfill \\ & \ge & \min \{R^l,R^m\}\left(\min \{{\left[{\tilde{u}}_n\right]}_{s,\Phi }^l,{\left[{\tilde{u}}_n\right]}_{s,\Phi }^m\}+{\alpha }_1\min \{\parallel {\tilde{u}}_n{\parallel }_{\Phi }^l,\parallel {\tilde{u}}_n{\parallel }_{\Phi }^m\}\right)\hfill \\ & =& R^l\left({\left[{\tilde{u}}_n\right]}_{s,\Phi }^m+{\alpha }_1{\parallel {\tilde{u}}_n\parallel }_{\Phi }^m\right)\hfill \\ & \ge & \min \{1,{\alpha }_1\}{R}^l\left({\left[{\tilde{u}}_n\right]}_{s,\Phi }^m+{\parallel {\tilde{u}}_n\parallel }_{\Phi }^m\right)\hfill \\ & \ge & \min \{1,{\alpha }_1\}{R}^l{C}_m,\hfill \end{array}$$

(52)

where $C_m≔{\inf }_{|u|+|v|=1}{\{|u|}^m+{|v|}^m\}>0$. Then, by the arbitrariness of R, combining (51) and (52) with (48), we get a contradiction. Therefore, the ${\left(C\right)}_c$-sequence $\left\{u_n\right\}$ is bounded.  □

Lemma 12. 

Assume that $\left({\varphi }_1\right)$, $\left(V\right)$, and $\left(f_1\right)$ hold. Then $I^{\prime }:W\to W^{*}$ is weakly sequentially continuous. Namely, if $u_n\rightharpoonup u$ in W, then $I^{\prime }\left(u_n\right)\rightharpoonup I^{\prime }\left(u\right)$ in the dual space $W^{*}$ of W.

Proof. 

Since W is reflexive, it is enough to prove $I^{\prime }\left(u_n\right)\stackrel{w^{*}}{\rightharpoonup }{I}^{\prime }\left(u\right)$ in $W^{*}$. Namely, to prove:

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }⟨I^{\prime }\left(u_n\right),v⟩=⟨I^{\prime }\left(u\right),v⟩, \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} v\in W.\end{array}$$

(53)

Firstly, we prove that $I^{\prime }$ is bounded on each bounded subset of W. Indeed, by (12), $\left(V\right)$, (5), (11), (6), Lemma 2, $\left(3\right)$ in Lemma 4, $\left(1\right)$ in Lemma 5, and the fact that $\Phi$, $\tilde{\Phi }$ and ${\Phi }_{*}$ satisfy the ${\Delta }_2$-condition, we have:

$$\begin{array}{ccc}\hfill \parallel I^{\prime }{\left(u\right)\parallel }_{W^{*}}& =& \underset{v\in {W,\parallel v\parallel }_{s,\Phi }=1}{\sup }|⟨I^{\prime }\left(u\right),v⟩|\hfill \\ & \le & \underset{v\in {W,\parallel v\parallel }_{s,\Phi }=1}{\sup }\left({\int \int }_{{\mathbb{R}}^{2N}}a(|D_{s}u|)|D_{s}u||D_{s}v|d\mu +{\int }_{{\mathbb{R}}^N}V\left(x\right)a(|u|)|u||v|dx\right.\hfill \\ & & \left.+{\int }_{{\mathbb{R}}^N}|f(x,u)||v|dx\right)\hfill \\ & \le & \underset{v\in {W,\parallel v\parallel }_{s,\Phi }=1}{\sup }\left({\int \int }_{{\mathbb{R}}^{2N}}\tilde{\Phi }\left(a\right(|D_{s}u|)|D_{s}u|)d\mu +{\int \int }_{{\mathbb{R}}^{2N}}\Phi (|D_{s}v|)d\mu \right.\hfill \\ & & +({\alpha }_2+\epsilon ){\int }_{{\mathbb{R}}^N}\tilde{\Phi }\left(a\right(|u|)|u|)dx+({\alpha }_2+\epsilon ){\int }_{{\mathbb{R}}^N}\Phi (|v|)dx\hfill \\ & & \left.+C_{\epsilon }{\int }_{{\mathbb{R}}^N}{\tilde{\Phi }}_{*}({\Phi }_{*}^{\prime }(|u|))dx+C_{\epsilon }{\int }_{{\mathbb{R}}^N}{\Phi }_{*}(|v|)dx\right)\hfill \\ & \le & \left({\int \int }_{{\mathbb{R}}^{2N}}\Phi (2|D_{s}u|)d\mu +({\alpha }_2+\epsilon ){\int }_{{\mathbb{R}}^N}\Phi (2|u|)dx+C_{\epsilon }{\int }_{{\mathbb{R}}^N}{\Phi }_{*}(2|u|)dx\right)\hfill \\ & & +\underset{v\in {W,\parallel v\parallel }_{s,\Phi }=1}{\sup }\left(\max \{{\left[v\right]}_{s,\Phi }^l,{\left[v\right]}_{s,\Phi }^m\}+({\alpha }_2+\epsilon ){\max \{\parallel v\parallel }_{\Phi }^l{,\parallel v\parallel }_{\Phi }^m\}\right.\hfill \\ & & \left.+C_{\epsilon }{\max \{\parallel v\parallel }_{{\Phi }_{*}}^{l^{*}}{,\parallel v\parallel }_{{\Phi }_{*}}^{m^{*}}\}\right)\hfill \\ & \le & K_2\left({\int \int }_{{\mathbb{R}}^{2N}}\Phi (|D_{s}u|)d\mu +({\alpha }_2+\epsilon ){\int }_{{\mathbb{R}}^N}\Phi (|u|)dx+C_{\epsilon }{\int }_{{\mathbb{R}}^N}{\Phi }_{*}(|u|)dx\right)\hfill \\ & & +1+{\alpha }_2+\epsilon +C_{\epsilon }{C}_{{\Phi }_{*}}\hfill \\ & \le & K_2\left((1+{\alpha }_2+\epsilon ){\parallel u\parallel }_{s,\Phi }^m+C_{\epsilon }{C}_{{\Phi }_{*}}{\parallel u\parallel }_{s,\Phi }^{m^{*}}\right)+(K_2+1)(1+{\alpha }_2+\epsilon +C_{\epsilon }{C}_{{\Phi }_{*}}),\hfill \end{array}$$

which implies that $I^{\prime }$ is bounded on each bounded subset of W. Moreover, $C_c^{\infty }\left({\mathbb{R}}^N\right)$ is dense in W. Then, to prove (53) we only need to prove:

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }⟨I^{\prime }\left(u_n\right),w⟩=⟨I^{\prime }\left(u\right),w⟩, \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} w\in C_c^{\infty }\left({\mathbb{R}}^N\right).\end{array}$$

(54)

To get (54), arguing by contradiction, we suppose that there exist constant $\delta >0$, $w_0\in C_c^{\infty }\left({\mathbb{R}}^N\right)$ with $\mathrm{supp}\left\{w_0\right\}\subset B_r$ for some $r>0$, and a subsequence of $\left\{u_n\right\}$, still denoted by $\left\{u_n\right\}$, such that:

$$\begin{array}{c}\hfill |⟨I^{\prime }\left(u_n\right),w_0⟩-⟨I^{\prime }\left(u\right),w_0⟩|\ge \delta ,\mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} n\in {\mathbb{R}}^N.\end{array}$$

(55)

Since $u_n\rightharpoonup u$ in W, by $\left(5\right)$ in Lemma 5, there exists a subsequence of $\left\{u_n\right\}$, still denoted by $\left\{u_n\right\}$, such that

$$\begin{array}{c}\hfill u_n\to u \mathrm{in} L_{loc}^{\Phi }\left({\mathbb{R}}^N\right), u_n\left(x\right)\to u\left(x\right)\mathrm{a}.\mathrm{e}. \mathrm{in} {\mathbb{R}}^N \mathrm{and} D_s{u}_n\to D_{s}u \mathrm{a}.\mathrm{e}. \mathrm{in} {\mathbb{R}}^{2N}.\end{array}$$

Next, we claim that

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }{\int }_{{\mathbb{R}}^N}f(x,u_n)w_{0}dx={\int }_{{\mathbb{R}}^N}f(x,u)w_{0}dx.\end{array}$$

(56)

Indeed, it follows $\left(f_1\right)$ that for any given constant $\epsilon >0$, there exists a constant $C_{\epsilon }>0$ such that:

$$\begin{array}{c}\hfill |f(x,t)|\le C_{\epsilon }+\epsilon {\Phi }_{*}^{\prime }(|t|), \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} (x,t)\in {\mathbb{R}}^N\times \mathbb{R}.\end{array}$$

Then, by using standard arguments, we can obtain that the sequence $\left\{f(x,u_n)\right\}$ is bounded in $L^{{\tilde{\Phi }}_{*}}\left(B_r\right)$. Moreover, $f(x,u_n)\to f(x,u)$ a.e. in $B_r$. Then, by applying Lemma 2.1 in [37], we get $f(x,u_n)\rightharpoonup f(x,u)$ in $L^{{\tilde{\Phi }}_{*}}\left(B_r\right)$, and thus (56) holds because $w_0\in L^{{\Phi }_{*}}\left(B_r\right)$.

Similarly, we can get:

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }{\int \int }_{{\mathbb{R}}^{2N}}a(|D_s{u}_n|)D_s{u}_n{D}_s{w}_{0}d\mu ={\int \int }_{{\mathbb{R}}^{2N}}a(|D_{s}u|)D_{s}u{D}_s{w}_{0}d\mu \end{array}$$

(57)

and:

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }{\int }_{{\mathbb{R}}^N}V\left(x\right)a(|u_n|)u_n{w}_{0}dx={\int }_{{\mathbb{R}}^N}V\left(x\right)a(|u|)u{w}_{0}dx,\end{array}$$

(58)

which is based on the fact that the sequence $\left\{a\right(|D_s{u}_n|)D_s{u}_n\}$ is bounded in $L^{\tilde{\Phi }}({\mathbb{R}}^{2N},d\mu )$, $a(|D_s{u}_n|)D_s{u}_n\to a(|D_{s}u|)D_{s}u$ a.e. in ${\mathbb{R}}^{2N}$, $D_s{w}_0\in L^{\Phi }({\mathbb{R}}^{2N},d\mu )$, and the sequence $\left\{V\left(x\right)a\right(|u_n|)u_n\}$ is bounded in $L^{\tilde{\Phi }}\left({\mathbb{R}}^N\right)$, $V\left(x\right)a(|u_n|)u_n\to V\left(x\right)a(|u|)u$ a.e. in ${\mathbb{R}}^N$, $w_0\in L^{\Phi }\left({\mathbb{R}}^N\right)$, respectively.

Therefore, combining (56)–(58) with (12), we can conclude that:

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }|⟨I^{\prime }\left(u_n\right),w_0⟩-⟨I^{\prime }\left(u\right),w_0⟩|=0,\end{array}$$

which contradicts (55). Thus, (54) holds and the proof is completed.  □

Lemma 13. 

Equation (1) has at least a non-trivial solution under the assumptions of Theorem 1 and Theorem 2, respectively.

Proof. 

Let $\left\{u_n\right\}$ be the ${\left(C\right)}_c$-sequence of I in W for the level $c>0$ given in (18). Lemmas 9 and 11 show that the sequence $\left\{u_n\right\}$ is bounded in W under the assumptions of Theorem 1 and Theorem 2, respectively.

First, we claim that:

$$\begin{array}{c}\hfill {\lambda }_3≔\underset{n\to \infty }{\lim }\underset{y\in {\mathbb{R}}^N}{\sup }{\int }_{B_2\left(y\right)}\Phi (|u_n|)dx>0.\end{array}$$

(59)

Indeed, if ${\lambda }_3=0$, by using the Lions’ type result for fractional Orlicz–Sobolev spaces again, we have:

$$\begin{array}{c}\hfill u_n\to \mathbf{0} \mathrm{in} L^p\left({\mathbb{R}}^N\right), \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} p\in (m,l^{*}).\end{array}$$

(60)

Given $p\in (m,l^{*})$, by $\left(f_1\right)$, $\left({\varphi }_1\right)$ and the definition $F(x,t)={\int }_0^{t}f(x,\tau )d\tau$, for any given constant $\epsilon >0$, there exists a constant $C_{\epsilon }>0$ such that:

$$\begin{array}{c}\hfill |F(x,t)|\le \epsilon (\Phi (|t|)+{\Phi }_{*}(|t|))+C_{\epsilon }{|t|}^p, \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} (x,t)\in {\mathbb{R}}^N\times \mathbb{R}\end{array}$$

(61)

and:

$$\begin{array}{c}\hfill |tf(x,t)|\le \epsilon (\Phi (|t|)+{\Phi }_{*}(|t|))+C_{\epsilon }{|t|}^p, \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} (x,t)\in {\mathbb{R}}^N\times \mathbb{R}.\end{array}$$

(62)

Then, it follows from (60)–(62), $\left(1\right)$ in Lemma 2, $\left(3\right)$ in Lemma 4 and $\left(1\right)$ in Lemma 5, the boundedness of $\left\{u_n\right\}$, and the arbitrariness of $\epsilon$ that:

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }{\int }_{{\mathbb{R}}^N}F(x,u_n)dx=\underset{n\to \infty }{\lim }{\int }_{{\mathbb{R}}^N}{u}_{n}f(x,u_n)dx=0.\end{array}$$

(63)

Hence, by (10), (12), (19), $\left({\varphi }_1\right)$, $\left(V\right)$, and (63), we have:

$$\begin{array}{ccc}\hfill c& =& \underset{n\to \infty }{\lim }\left\{I\left(u_n\right)-⟨I^{\prime }\left(u_n\right),\frac{1}{l}{u}_n⟩\right\}\hfill \\ & =& \underset{n\to \infty }{\lim }\left\{{\int \int }_{{\mathbb{R}}^{2N}}\left(\Phi (|D_s{u}_n|)-\frac{1}{l}a(|D_s{u}_n|)|D_s{u}_n{|}^2\right)d\mu \right.\hfill \\ & & +{\int }_{{\mathbb{R}}^N}V\left(x\right)\left(\Phi (|u_n|)-\frac{1}{l}a(|u_n|)u_n^2\right)dx\hfill \\ & & \left.+{\int }_{{\mathbb{R}}^N}\left(\frac{1}{l}{u}_{n}f(x,u_n)-F(x,u_n)\right)dx\right\}\hfill \\ & \le & \underset{n\to \infty }{\lim }\left\{{\int }_{{\mathbb{R}}^N}\left(\frac{1}{l}{u}_{n}f(x,u_n)-F(x,u_n)\right)dx\right\}=0,\hfill \end{array}$$

which contradicts $c>0$. Therefore, ${\lambda }_3>0$, and thus, (59) holds.

Then, it follows from (59) that there exist a constant $\delta >0$, a subsequence of $\left\{u_n\right\}$, still denoted by $\left\{u_n\right\}$, and a sequence $\left\{z_n\right\}\subset {\mathbb{Z}}^N$ such that:

$$\begin{array}{c}\hfill {\int }_{B_2\left(z_n\right)}\Phi (|u_n|)dx={\int }_{B_2}(\Phi (|u_n^{*}|)dx>\delta , \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} n\in \mathbb{N},\end{array}$$

(64)

where $u_n^{*}≔u_n(·+z_n)$. Since V and F are 1-periodic in x, $\left\{u_n^{*}\right\}$ is also a ${\left(C\right)}_c$-sequence of I. Then, passing to a subsequence of $\left\{u_n^{*}\right\}$, still denoted by $\left\{u_n^{*}\right\}$, we can assume that there exists a $u^{*}\in W$ such that:

$$\begin{array}{c}\hfill u_n^{*}\rightharpoonup u^{*} \mathrm{in} W \mathrm{and} u_n^{*}\to u^{*} \mathrm{in} L^{\Phi }\left(B_2\right).\end{array}$$

(65)

Thus, by (64), (65), and (7), we obtain that $u^{*}\ne \mathbf{0}$. Moreover, it follows from Lemma 12 and (19) that:

$$\begin{array}{c}\hfill \parallel I^{\prime }\left(u^{*}\right){\parallel }_{W^{*}}\le \underset{n\to \infty }{\mathrm{lim\; inf}}{\parallel I^{\prime }\left(u_n^{*}\right)\parallel }_{W^{*}}=0,\end{array}$$

which implies $I^{\prime }\left(u^{*}\right)=\mathbf{0}$, that is, $u^{*}$ is a non-trivial solution of Equation (1).  □

Lemma 14. 

Assume that $\left({\varphi }_1\right)$, $\left(V\right)$ and $\left(f_1\right)$ hold. Then:

$$⟨I^{\prime }\left(u\right),u⟩=⟨I_1^{\prime }\left(u\right),u⟩-o\left(⟨I_1^{\prime }\left(u\right),u⟩\right) \mathit{as} {\parallel u\parallel }_{s,\Phi }\to 0.$$

Proof. 

By using the continuity of $I_i^{\prime }(i=1,2)$ defined by (15) and (16), we can easily verify that $⟨I_i^{\prime }\left(u\right),u⟩=o\left(1\right)(i=1,2)$ as ${\parallel u\parallel }_{s,\Phi }\to 0$. Then, it is sufficient to prove $⟨I_2^{\prime }\left(u\right),u⟩=o\left(⟨I_1^{\prime }\left(u\right),u⟩\right)$ as ${\parallel u\parallel }_{s,\Phi }\to 0$ because $⟨I^{\prime }\left(u\right),u⟩=⟨I_1^{\prime }\left(u\right),u⟩-⟨I_2^{\prime }\left(u\right),u⟩$.

For any given constant $\epsilon >0$, it follows $\left(f_1\right)$, $\left({\varphi }_1\right)$ and (5) that there exists a constant $C_{\epsilon }>0$ such that:

$$\begin{array}{c}\hfill |tf(x,t)|\le \epsilon \Phi (|t|)+C_{\epsilon }{\Phi }_{*}(|t|), \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} (x,t)\in {\mathbb{R}}^N\times \mathbb{R}.\end{array}$$

(66)

Then, by (16) and (66), we have:

$$\begin{array}{ccc}\hfill |⟨I_2^{\prime }\left(u\right),u⟩|& \le & {\int }_{{\mathbb{R}}^N}|uf(x,u)|dx\hfill \\ & \le & \epsilon {\int }_{{\mathbb{R}}^N}\Phi (|u|)dx+C_{\epsilon }{\int }_{{\mathbb{R}}^N}{\Phi }_{*}(|u|)dx.\hfill \end{array}$$

(67)

Moreover, by (15), $\left({\varphi }_1\right)$, and $\left(V\right)$, we have:

$$\begin{array}{ccc}\hfill ⟨I_1^{\prime }\left(u\right),u⟩& =& {\int \int }_{{\mathbb{R}}^{2N}}a(|D_{s}u|)|D_s{u|}^{2}d\mu +{\int }_{{\mathbb{R}}^N}V\left(x\right)a(|u|)u^{2}dx\hfill \\ & \ge & l{\int \int }_{{\mathbb{R}}^{2N}}\Phi (|D_{s}u|)d\mu +{\alpha }_{1}l{\int }_{{\mathbb{R}}^N}\Phi (|u|)dx.\hfill \end{array}$$

(68)

Then, (67), (68), Lemma 2, $\left(3\right)$ in Lemma 4, $\left(1\right)$ in Lemma 5, and the fact that $1<m<l^{*}$ imply that:

$$\begin{array}{ccc}\hfill \underset{{\parallel u\parallel }_{s,\Phi }\to 0}{\lim }\frac{|⟨I_2^{\prime }\left(u\right),u⟩|}{⟨I_1^{\prime }\left(u\right),u⟩}& \le & \underset{{\parallel u\parallel }_{s,\Phi }\to 0}{\lim }\frac{\epsilon {\int }_{{\mathbb{R}}^N}\Phi (|u|)dx+C_{\epsilon }{\int }_{{\mathbb{R}}^N}{\Phi }_{*}(|u|)dx}{l{\int \int }_{{\mathbb{R}}^{2N}}\Phi (|D_{s}u|)d\mu +{\alpha }_{1}l{\int }_{{\mathbb{R}}^N}\Phi (|u|)dx}\hfill \\ & \le & \frac{\epsilon }{{\alpha }_{1}l}+\underset{{\parallel u\parallel }_{s,\Phi }\to 0}{\lim }\frac{C_{\epsilon }{\int }_{{\mathbb{R}}^N}{\Phi }_{*}(|u|)dx}{\min \{1,{\alpha }_1\}l\left({\int \int }_{{\mathbb{R}}^{2N}}\Phi (|D_{s}u|)d\mu +{\int }_{{\mathbb{R}}^N}\Phi (|u|)dx\right)}\hfill \\ & \le & \frac{\epsilon }{{\alpha }_{1}l}+\underset{{\parallel u\parallel }_{s,\Phi }\to 0}{\lim }\frac{C_{\epsilon }\max \{C_{{\Phi }_{*}}^{l^{*}},C_{{\Phi }_{*}}^{m^{*}}\}{\parallel u\parallel }_{s,\Phi }^{l^{*}}}{\min \{1,{\alpha }_1\}l{C}_m{\parallel u\parallel }_{s,\Phi }^m}\hfill \\ & =& \frac{\epsilon }{{\alpha }_{1}l}.\hfill \end{array}$$

Since $\epsilon$ is arbitrary, we conclude that $|⟨I_2^{\prime }\left(u\right),u⟩|=o\left(⟨I_1^{\prime }\left(u\right),u⟩\right)$ as ${\parallel u\parallel }_{s,\Phi }\to 0$, which implies that $⟨I_2^{\prime }\left(u\right),u⟩=o\left(⟨I_1^{\prime }\left(u\right),u⟩\right)$ as ${\parallel u\parallel }_{s,\Phi }\to 0$.  □

Proof of Theorems 1 and 2. 

Lemma 13 shows that Equation (1) has at least a non-trivial solution under the assumptions of Theorem 1 and Theorem 2, respectively. Next, we prove Equation (1) has a ground state solution. Let:

$$\mathcal{N}≔\{u\in W\setminus \left\{\mathbf{0}\right\}:I^{\prime }\left(u\right)=\mathbf{0}\} \mathrm{a}\mathrm{n}\mathrm{d} d≔\underset{u\in \mathcal{N}}{\inf }\left\{I\left(u\right)\right\}.$$

First, we claim that $d\ge 0$. Indeed, for any given non-trivial critical point $u\in \mathcal{N}$, by (10), (12), $\left({\varphi }_1\right)$, $\left(V\right)$ and $\left(f_3\right)$ (or $\left(f_5\right)$), we have:

$$\begin{array}{ccc}\hfill I\left(u\right)& =& I\left(u\right)-⟨I^{\prime }\left(u\right),\frac{1}{m}u⟩\hfill \\ & =& {\int \int }_{{\mathbb{R}}^{2N}}\left(\Phi (|D_{s}u|)-\frac{1}{m}a(|D_{s}u|)|D_s{u|}^2\right)d\mu \hfill \\ & & +{\int }_{{\mathbb{R}}^N}V\left(x\right)\left(\Phi (|u|)-\frac{1}{m}a(|u|)u^2\right)dx\hfill \\ & & +{\int }_{{\mathbb{R}}^N}\left(\frac{1}{m}uf(x,u)-F(x,u)\right)dx\hfill \\ & \ge & \frac{1}{m}{\int }_{{\mathbb{R}}^N}\widehat{F}(x,u)dx\ge 0.\hfill \end{array}$$

Since the non-trivial critical point u of I is arbitrary, we conclude $d\ge 0$. Choose a sequence $\left\{u_n\right\}\subset \mathcal{N}$ such that $I\left(u_n\right)\to d$ as $n\to \infty$. Then, it is obvious that $\left\{u_n\right\}$ is a ${\left(C\right)}_d$-sequence of I for the level d. Lemmas 9 and 11 show that $\left\{u_n\right\}$ is bounded in W. Moreover, combining Lemma 14 with the fact that $\left\{u_n\right\}\subset \mathcal{N}$, we can conclude that there exists a constant $M_3>0$ such that:

$$\begin{array}{c}\hfill \parallel u_n{\parallel }_{s,\Phi }\ge M_3, \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} n\in \mathbb{N}.\end{array}$$

(69)

Now, we claim that:

$$\begin{array}{c}\hfill {\lambda }_4≔\underset{n\to \infty }{\lim }\underset{y\in {\mathbb{R}}^N}{\sup }{\int }_{B_2\left(y\right)}\Phi (|u_n|)dx>0.\end{array}$$

(70)

Indeed, if ${\lambda }_4=0$, similar to (63), we can get:

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }{\int }_{{\mathbb{R}}^N}{u}_{n}f(x,u_n)dx=0.\end{array}$$

(71)

Then, by (12), $\left({\varphi }_1\right)$, $\left(V\right)$, and (71), we have:

$$\begin{array}{ccc}\hfill 0& =& \underset{n\to \infty }{\lim }\left\{⟨I^{\prime }\left(u_n\right),u_n⟩+{\int }_{{\mathbb{R}}^N}{u}_{n}f(x,u_n)dx\right\}\hfill \\ & =& \underset{n\to \infty }{\lim }\left\{{\int \int }_{{\mathbb{R}}^{2N}}a(|D_s{u}_n|)|D_s{u}_n{|}^{2}d\mu +{\int }_{{\mathbb{R}}^N}V\left(x\right)a(|u_n|)u_n^{2}dx\right\}\hfill \\ & \ge & \underset{n\to \infty }{\lim }\left\{l{\int \int }_{{\mathbb{R}}^{2N}}\Phi (|D_s{u}_n|)d\mu +{\alpha }_{1}l{\int }_{{\mathbb{R}}^N}\Phi (|u_n|)dx\right\}\hfill \\ & \ge & 0,\hfill \end{array}$$

which together with Lemma 2 implies that $\parallel u_n{\parallel }_{s,\Phi }={\parallel u_n\parallel }_{\Phi }+{\left[u_n\right]}_{s,\Phi }\to 0$ as $n\to \infty$, which contradicts (69). Therefore, ${\lambda }_4>0$, and thus, (70) holds.

Next, with similar arguments as those in Lemma 13, let $u_n^{*}≔u_n(·+z_n)$. Then, $\left\{u_n^{*}\right\}$ is also a ${\left(C\right)}_d$-sequence of I. Moreover, there exist a subsequence of $\left\{u_n^{*}\right\}$, still denoted by $\left\{u_n^{*}\right\}$, and a $u^{*}\in W$ such that $u_n^{*}\rightharpoonup u^{*}$ in W with $u^{*}\ne \mathbf{0}$ and $I^{\prime }\left(u^{*}\right)=\mathbf{0}$. This shows that $u^{*}\in \mathcal{N}$, and thus, $I\left(u^{*}\right)\ge d$.

On the other hand, by (10), (12), $\left({\varphi }_1\right)$, $\left(V\right)$, $\left(f_3\right)$ (or $\left(f_5\right)$), and Fatou’s Lemma, we have:

$$\begin{array}{ccc}\hfill I\left(u^{*}\right)& =& I\left(u^{*}\right)-⟨I^{\prime }\left(u^{*}\right),\frac{1}{m}{u}^{*}⟩\hfill \\ & =& {\int \int }_{{\mathbb{R}}^{2N}}\left(\Phi (|D_s{u}^{*}|)-\frac{1}{m}a(|D_s{u}^{*}|)|D_s{u}^{*}{|}^2\right)d\mu \hfill \\ & & +{\int }_{{\mathbb{R}}^N}V\left(x\right)\left(\Phi (|u^{*}|)-\frac{1}{m}a(|u^{*}|){|u^{*}|}^2\right)dx\hfill \\ & & +{\int }_{{\mathbb{R}}^N}\left(\frac{1}{m}{u}^{*}f(x,u^{*})-F(x,u^{*})\right)dx\hfill \\ & \le & \underset{n\to \infty }{\mathrm{lim\; inf}}\left\{I\left(u_n^{*}\right)-⟨I^{\prime }\left(u_n^{*}\right),\frac{1}{m}{u}_n^{*}⟩\right\}\hfill \\ & =& d.\hfill \end{array}$$

Therefore, $I\left(u^{*}\right)=d$, that is, $u^{*}$ is a ground state solution of Equation (1). This finishes the proof.  □

4. Examples

For Equation (1), when given $s\in (0,1)$ and $N\in \mathbb{N}$, the function $\varphi$ defined by (2) can be selected from the following possibilities, each satisfying conditions $\left({\varphi }_1\right)$–$\left({\varphi }_2\right)$.

Case 1. Let $\varphi \left(t\right)={|t|}^{p-2}t$ for $t\ne 0$, $\varphi \left(0\right)=0$ with $1<p<\frac{N}{s}$. In this case, simple computations show that $l=m=p$.

Case 2. Let $\varphi \left(t\right)={|t|}^{p-2}t+{|t|}^{q-2}t$ for $t\ne 0$, $\varphi \left(0\right)=0$ with $1<p<q<\frac{N}{s}<\frac{pq}{q-p}$. In this case, simple computations show that $l=p,m=q$.

Case 3. Let $\varphi \left(t\right)=\frac{{|t|}^{q-2}t}{log(1+|t{|}^p)}$ for $t\ne 0$, $\varphi \left(0\right)=0$ with $1<p+1<q<\frac{N}{s}<\frac{q(q-p)}{p}$. In this case, simple computations show that $l=q-p,m=q$.

Moreover, we provide an additional case that satisfies condition $\left({\varphi }_1\right)$ but fails to satisfy condition $\left({\varphi }_2\right)$.

Case 4. Let $\varphi \left(t\right)={|t|}^{q-2}{tlog(1+|t|}^p)$ for $t\ne 0$, $\varphi \left(0\right)=0$ with $1<q<p+q<\frac{N}{s}<\frac{q(p+q)}{p}$. In this case, simple computations show that $l=q,m=p+q$.

For example, regarding Case 2, the operator in non-local problem (1) defined by (3) reduces to the following fractional $(p,q)$-Laplacian operator:

$$\begin{array}{ccc}\hfill {(-{\Delta }_p-{\Delta }_q)}^{s}u\left(x\right)& =& P.V.{\int }_{{\mathbb{R}}^N}\frac{{|u\left(x\right)-u\left(y\right)|}^{p-2}(u\left(x\right)-u\left(y\right))}{{|x-y|}^{N+ps}}dy\hfill \\ & & +P.V.{\int }_{{\mathbb{R}}^N}\frac{{|u\left(x\right)-u\left(y\right)|}^{q-2}(u\left(x\right)-u\left(y\right))}{{|x-y|}^{N+qs}}dy.\hfill \end{array}$$

Let $f(x,t)={qh\left(x\right)|t|}^{q-2}tlog(1+|t|)+\frac{{h\left(x\right)|t|}^{q-1}t}{1+|t|}$, where $h\in C({\mathbb{R}}^N,(0,+\infty ))$ is 1-periodic in x. Then, $F(x,t)={h\left(x\right)|t|}^{q}log(1+|t|)$ and $\widehat{F}(x,t)=\frac{{h\left(x\right)|t|}^{q+1}}{1+|t|}$. It is easy to check that f satisfies $\left(f_1\right)$-$\left(f_2\right)$, but does not satisfy the (AR) type condition ${\left(AR\right)}^{*}$. However, we can see that it satisfies $\left(f_3\right)$. Indeed, since $\frac{N}{s}<\frac{pq}{q-p}$, then there exists constant $k\in (\frac{N}{sp},\frac{q}{q-p})$ such that:

$$\begin{array}{c}\hfill \underset{|t|\to \infty }{\mathrm{lim\; sup}}{\left(\frac{|F(x,t)|}{{|t|}^l}\right)}^k\frac{1}{\widehat{F}(x,t)}=\underset{|t|\to \infty }{\mathrm{lim\; sup}}\frac{h^{k-1}{\left(x\right)(1+|t|)(log(1+|t|\left)\right)}^k}{{|t|}^{(p-q)k+q+1}}=0,\end{array}$$

which implies that condition $\left(f_3\right)$ holds. Therefore, by using Theorem 1, we obtain that Equation (1) has at least one ground state solution when potential V satisfies condition $\left(V\right)$.

In addition, let $f(x,t)=h\left(x\right)\gamma \left(t\right)$, where $h\in C({\mathbb{R}}^N,(0,+\infty ))$ is 1-periodic in x and:

$$\begin{array}{c}\hfill \gamma \left(t\right)=\left\{\begin{array}{c}\begin{array}{cc}0,\hfill & |t|\le 1,\hfill \\ \left({|t|}^{\frac{q+p^{*}-4}{2}}-\frac{1}{|t|}\right)t,\hfill & |t|>1.\hfill \end{array}\hfill \end{array}\right.\end{array}$$

Then, $F(x,t)=h\left(x\right)\Gamma \left(t\right)$, where:

$$\begin{array}{c}\hfill \Gamma \left(t\right)=\left\{\begin{array}{c}\begin{array}{cc}0,\hfill & |t|\le 1,\hfill \\ \frac{2}{q+p^{*}}{|t|}^{\frac{q+p^{*}}{2}}-|t|+\frac{q+p^{*}-2}{q+p^{*}},\hfill & |t|>1.\hfill \end{array}\hfill \end{array}\right.\end{array}$$

It is easy to check that f satisfies $\left(f_1\right)$ and $\left(f_4\right)$, but does not satisfy $\left(f_3\right)$ and the (Ne) type condition ${\left(Ne\right)}^{*}$. However, we can see that it satisfies $\left(f_5\right)$. Indeed, since:

$$\begin{array}{c}\hfill \frac{1-{\theta }^l}{m}tf(x,t)=\frac{1-{\theta }^p}{q}h\left(x\right)t\gamma \left(t\right) \mathrm{and} F(x,t)-F(x,\theta t)\le F(x,t)=h\left(x\right)\Gamma \left(t\right),\end{array}$$

(72)

for all $\theta \in \mathbb{R},(x,t)\in {\mathbb{R}}^N\times \mathbb{R}.$ Then, it is obvious that:

$$\begin{array}{c}\hfill \frac{1-{\theta }^l}{m}tf(x,t)\ge F(x,t)-F(x,\theta t), \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} \theta \in \mathbb{R},(x,t)\in {\mathbb{R}}^N\times [-1,1].\end{array}$$

(73)

Moreover:

$$\underset{|t|>1}{\inf }\frac{t\gamma \left(t\right)-q\Gamma \left(t\right)}{t\gamma \left(t\right)}=\underset{|t|>1}{\inf }\frac{\frac{p^{*}-q}{q+p^{*}}{|t|}^{\frac{q+p^{*}}{2}}+(q-1)|t|-\frac{q^2+q{p}^{*}-2q}{q+p^{*}}}{{|t|}^{\frac{q+p^{*}}{2}}-|t|}>0,$$

which implies that there exists a ${\theta }_0\in (0,1)$ such that:

$$\begin{array}{c}\hfill \frac{1-{\theta }^p}{q}h\left(x\right)t\gamma \left(t\right)\ge h\left(x\right)\Gamma \left(t\right), \mathrm{f}\mathrm{o}\mathrm{r} \mathrm{a}\mathrm{l}\mathrm{l} \theta \in [0,{\theta }_0],x\in {\mathbb{R}}^N,|t|>1.\end{array}$$

(74)

Then, combining (73) and (74) with (72), we can conclude that $\left(f_5\right)$ holds. Therefore, by using Theorem 2, we obtain that Equation (1) has at least one ground state solution when potential V satisfies condition $\left(V\right)$.

5. Conclusions

In this paper, we have explored the existence of ground state solutions for a non-local problem in fractional Orlicz–Sobolev spaces. This problem involves the fractional $\Phi$-Laplacian, a non-local, and a non-homogeneous operator. Our analysis did not rely on traditional assumptions such as the Ambrosetti–Rabinowitz type or Nehari type conditions on the non-linearity. Instead, we utilized a modified version of the mountain pass theorem and a Lions’ type result tailored for fractional Orlicz–Sobolev spaces. These techniques allowed us to demonstrate the existence of ground state solutions in the periodic case. This work extends and improves the existing results in the literature. Looking ahead, it is intriguing to consider the potential extension of our work to systems in fractional Orlicz–Sobolev spaces, presenting exciting prospects for future exploration and research.