## 1. Introduction

In this paper, we are initially motivated by the investigation of the class of variational-hemivariational inequalities considered in [

1]. Let

V be a reflexive Banach space. Consider an operator

$A:V\to {V}^{*}$, functions

$\phi :V\to \mathbb{R}$,

$j:V\to \mathbb{R}$ and a set

$K\subset V$. The constrained variational-hemivariational inequality studied in the paper reads as follows: find an element

$u\in K$ such that

Particular forms of problem (

1) contain various formulations investigated in the literature: the elliptic variational inequalities of the first and second kind, see [

2,

3,

4,

5], the elliptic hemivariational inequalities, see [

6,

7,

8,

9], and the elliptic equations, see [

6,

10,

11]. Moreover, the quasi-variational inequalities corresponding to problem (

1) and its variants can be treated by a fixed point technique, see [

1,

5]. In all aforementioned papers, the usual hypotheses for existence (and uniqueness) of a solution involve the function

$\phi $, which is supposed to be convex, the function

j which is locally Lipschitz and in general nonconvex, the operator

A pseudomonotone and strongly monotone, and a nonempty, closed and convex set of constraints

K.

In the current paper, we treat the counterpart of problem (

1) where the set

K represents a set of admissible constraints which is star-shaped with respect to a certain ball in

V. Note that for this class of nonconvex sets, some particular versions have been studied earlier in Section 7.4 of [

8] if

$j=0$, in Section 7.3 of [

8,

12] if

$\phi =0$, and Section 7.2 of [

8] when

$\phi =j=0$. The first novelty of our contribution is to study the class of fully variational-hemivariational inequalities involving nonconvex constraints. In contrast to contributions in [

8,

12,

13] which are based on surjectivity methods for the multivalued pseudomonotone operators for the existence proof of the penalized problem, in this paper we employ another method based on the Knaster, Kuratowski, and Mazurkiewicz (KKM) theory for the penalized problem. The second novelty is to study the constrained variational-hemivariational inequality on star-shaped sets without hypothesis on the strong monotonicity of the operator and without the relaxed monotonicity condition of the generalized subgradient. To the best of our knowledge, it is a new approach in the examination of this class of variational-hemivariational inequalities. We will use the celebrated lemma by Fan which is a milestone in the KKM theory and it is sufficient for our purpose.

For related results on variational-hemivariational equalities on nonconvex star-shaped constraint sets, we refer to [

12,

13] for stationary problems, and to [

14,

15] for evolution problems. Numerous applications of variational-hemivariational inequalities to problems of nonsmooth contact mechanics, economics, etc. can be found in classical monographs [

8,

9,

16], and in two recent books [

17,

18], and the references therein. A unified method, based on the hemivariational inequality formulation, to study contact problems of viscoelasticity is given in [

19], the abstract elliptic variational-hemivariational inequalities in reflexive Banach spaces with applications can be found in [

1], and the variational-hemivariational inqualities which model fluid flow in mechanics were treated in [

20,

21] and very recently in [

22,

23]. Other recent developments on variational methods in the study of existence and multiplicity of solutions, see [

24,

25].

The paper is structured as follows. Notations, basic definitions and preliminaries are provided in

Section 2.

Section 3 contains problem formulation with the first existence result, Theorem 1, whose proof is provided in

Section 4. Finally, the second existence theorem is proved in

Section 5.

## 2. Basic Material

In this part of the paper we recall the standard notations and definitions from [

10,

17,

26,

27].

Let $(X,\parallel \xb7{\parallel}_{X})$ be a Banach space. By $({X}^{*},\parallel \xb7{\parallel}_{{X}^{*}})$ we denote its dual space while the symbol ${\langle \xb7,\xb7\rangle}_{{X}^{*}\times X}$ stands for the duality pairing between ${X}^{*}$ and X.

**Nonlinear operators and the KKM lemma.** Let X be a reflexive Banach space and $T:X\to {2}^{{X}^{*}}$ be a multivalued mapping. A mapping $T:X\to {2}^{{X}^{*}}$ is called bounded if the image of each bounded set in X remains in a bounded subset of ${X}^{*}$. A mapping $T:X\to {2}^{{X}^{*}}$ is called pseudomonotone, provided the following conditions are satisfied

- (i)
T has nonempty, bounded, closed and convex values.

- (ii)
T is upper semicontinuous (u.s.c.) from each finite dimensional subspace of X into ${X}^{*}$ equipped with its weak topology.

- (iii)
if ${u}_{n}\in X$, ${u}_{n}\to u$ weakly in X, ${u}_{n}^{*}\in T{u}_{n}$ and $lim\; sup\phantom{\rule{0.166667em}{0ex}}{\langle {u}_{n}^{*},{u}_{n}-u\rangle}_{{X}^{*}\times X}\le 0$, then to each $y\in X$, there exists ${u}^{*}\left(y\right)\in Tu$ such that $\langle {u}^{*}\left(y\right),u-y\rangle}_{{X}^{*}\times X}\le lim\; inf\phantom{\rule{0.166667em}{0ex}}{\langle {u}_{n}^{*},{u}_{n}-y\rangle}_{{X}^{*}\times X$.

A mapping $T:X\to {2}^{{X}^{*}}$ is called generalized pseudomonotone, if for each sequences $\left\{{u}_{n}\right\}\subset X$, ${u}_{n}\to u$ weakly in X, $\left\{{u}_{n}^{*}\right\}\subset {X}^{*}$, ${u}_{n}^{*}\to {u}^{*}$ weakly in ${X}^{*}$, ${u}_{n}^{*}\in T{u}_{n}$ and $lim\; sup\phantom{\rule{0.166667em}{0ex}}{\langle {u}_{n}^{*},{u}_{n}-u\rangle}_{{X}^{*}\times X}\le 0$, we have ${u}^{*}\in Tu$ and ${\langle {u}_{n}^{*},{u}_{n}\rangle}_{{X}^{*}\times X}\to {\langle {u}^{*},u\rangle}_{{X}^{*}\times X}$.

The following relations concern the classes of pseudomonotone and generalized pseudomonotone mappings, see Propositions 1.3.65 and 1.3.68 in [

10]. If

$T:X\to {2}^{{X}^{*}}$ is a pseudomonotone mapping, then it is also generalized pseudomonotone. If

$T:X\to {2}^{{X}^{*}}$ is a bounded, generalized pseudomonotone mapping with nonempty, closed and convex values, then

T is pseudomonotone.

A single-valued mapping

$A:X\to {X}^{*}$ is said to be pseudomonotone, provided it is bounded and if

${u}_{n}\to u$ weakly in

X with

$lim\; sup{\langle A{u}_{n},{u}_{n}-u\rangle}_{{X}^{*}\times X}\le 0$ imply

$\langle Au,u-v\rangle}_{{X}^{*}\times X}\le lim\; inf{\langle A{u}_{n},{u}_{n}-v\rangle}_{{X}^{*}\times X$ for all

$v\in X$. Equivalently, see Proposition 3.66 in [

17], a single-valued mapping

A is pseudomonotone, if and only if it is bounded and

${u}_{n}\to u$ weakly in

X with

$lim\; sup{\u2329A{u}_{n},{u}_{n}-u\u232a}_{{X}^{*}\times X}\le 0$ imply

$lim{\u2329A{u}_{n},{u}_{n}-u\u232a}_{{X}^{*}\times X}=0$ and

$A{u}_{n}\to Au$ weakly in

${X}^{*}$. A mapping

$A:X\to {X}^{*}$ is demicontinuous, if it is continuous as a map from

X to

${X}^{*}$ furnished with the weak topology.

We recall below the KKM lemma in a version stated by Fan in Lemma 1 of [

28]. For various extensions to the Fan-KKM theorem, we refer to [

29] and the references therein.

**Lemma** **1.** Let X be a subset of a Haussdorf topological vector space Y. For any$x\in X$, let a set$F\left(x\right)$in Y be given such that:

- (a)
for every $x\in X$, $F\left(x\right)$ is a closed set in Y,

- (b)
convex hull of any finite set $\{{x}_{1},\dots ,{x}_{r}\}$ of X is contained in ${\bigcup}_{i=1}^{r}F\left({x}_{i}\right)$,

- (c)
$F\left(x\right)$ is a compact set at least for one $x\in X$.

Then ${\bigcap}_{x\in X}F\left(x\right)\ne \varnothing $.

**The Clarke generalized subgradient and tangent cones.** Let

X be a Banach space,

$h:X\to \mathbb{R}$ be a locally Lipschitz function, and

x,

$v\in X$. The Clarke generalized directional derivative of

h at

x in the direction

v is given by

The Clarke generalized subgradient of

h at

x is defined by

Let

$\mathbb{B}({u}_{0},\varrho )$ be the closed ball in a normed space

E with centre

${u}_{0}\in E$ and radius

$\varrho >0$. A nonempty set

$K\subset E$ is called star-shaped with respect to a ball

$\mathbb{B}({u}_{0},\varrho )$, if

$tv+(1-t)w\in K$ for all

$v\in K$,

$w\in \mathbb{B}({u}_{0},\varrho )$,

$t\in [0,1]$. When a set

K is star-shaped with respect to a closed ball, we just say that

K is star-shaped. Next, we denote by

$d:E\to \mathbb{R}$ the distance function of

K defined by

The Clarke generalized directional derivative of the function

d is well defined since

d is a Lipschitz continuous. Recall that for a star-shaped set, the Clarke directional derivative of the function

d enjoys the following discontinuity property, see Lemma 7.2, p. 224 in [

8].

**Lemma** **2.** Let E be a reflexive Banach space and $K\subset E$ be a closed set which is star-shaped with respect to a ball $\mathbb{B}({u}_{0},\varrho )$ with some ${u}_{0}\in K$ and $\varrho >0$. Then Finally, we shortly recall a material on tangent cones needed in what follows. Let

$K\subset E$ be a set of a Banach space

E and

$u\in K$. The Bouligand (contingent) cone to

K at the point

u is defined by

and the (Clarke) tangent cone to

K at

u is given by

It is well-known that

${T}_{K}\left(u\right)\subset {K}_{K}\left(u\right)$. The set

K is said to be regular at

$u\in K$ when

${T}_{K}\left(u\right)={K}_{K}\left(u\right)$. We also know that if

K is closed, convex and

$u\in K$, then

K is regular at

u, see Theorem 10.39 in [

30]. Further, it follows from Proposition 2.9 in [

30] that if

K is a convex set in

E and

$u\in K$, then

${K}_{K}\left(u\right)$ is a convex cone and

Equivalent definitions and properties of these and other cones can be found in [

26,

30] and Section 5.7 of [

27].

## 3. Formulation of the Problem

In this section we consider the constrained problem in which the set of admissible elements is nonconvex. The main goal is to prove the existence of solution.

Let

$(V,\parallel \xb7{\parallel}_{V})$ be a reflexive Banach space which is continuously and compactly embedded in a Hilbert space

$(H,\parallel \xb7{\parallel}_{H})$. The duality pairing between

${V}^{*}$ and

V is denoted by

$\langle \xb7,\xb7\rangle $, and

${\langle \xb7,\xb7\rangle}_{H}$ stands for the inner product in

H. Let

${K}_{0}$ be nonempty, closed, star-shaped with respect to a closed ball

$\mathbb{B}({u}_{0},\varrho )$ in

H, where

${u}_{0}\in V$ and

$\varrho >0$. Let

K and

${T}_{K}\left(u\right)$ for

$u\in K$ denote the realization of

${K}_{0}$ and

${T}_{{K}_{0}}\left(u\right)$ in

V, i.e.,

where

${T}_{{K}_{0}}\left(u\right)$ stands for the Clarke tangent cone of

${K}_{0}$ at

u.

**Problem** **1.** Find an element $u\in K$ such that The hypotheses on the data of Problem 1 are as follows.

- H(A):
$A:V\to {V}^{*}$ is a mapping such that

- (a)
A is pseudomonotone,

- (b)
A is strongly monotone with constant ${m}_{A}>0$, i.e.,

- H(j):
$j:V\to \mathbb{R}$ is a mapping such that

- (a)
j is locally Lipschitz,

- (b)
${\parallel \partial j\left(v\right)\parallel}_{{V}^{*}}\le {c}_{0}+{c}_{1}\phantom{\rule{0.166667em}{0ex}}{\parallel v\parallel}_{V}\phantom{\rule{4pt}{0ex}}\mathrm{for}\mathrm{all}\phantom{\rule{4pt}{0ex}}v\in V\phantom{\rule{4pt}{0ex}}\mathrm{with}\phantom{\rule{4pt}{0ex}}{c}_{0},{c}_{1}\ge 0$,

- (c)
$\mathrm{there}\mathrm{exist}\phantom{\rule{4pt}{0ex}}{\alpha}_{j}\ge 0\phantom{\rule{4pt}{0ex}}\mathrm{such}\mathrm{that}$

- H(φ):
$\phi :V\to \mathbb{R}$ is a convex and lower semicontinuous function.

- H(K):
K is a nonempty, closed and star-shaped in V.

- H(f):
$f\in {V}^{*}$.

In hypothesis

$H\left(j\right)$, the notation

${j}^{0}$ and

$\partial j$ stand for the Clarke generalized directional derivative and the Clarke generalized subgradient, respectively, of the function

j. We write

${\parallel \partial j\left(v\right)\parallel}_{{V}^{*}}=sup\{\parallel {v}^{*}{\parallel}_{{V}^{*}}\mid {v}^{*}\in \partial j\left(v\right)\}$. Condition

$H\left(j\right)$–(c) is known in the literature as a relaxed monotonicity condition, it holds with

${\alpha}_{j}=0$ when

j is a convex function, see [

1,

7,

17,

18] and the references therein. Examples of functions

j that satisfy

$H\left(j\right)$–(c) with single-valued and multivalued generalized subgradient can be found in Section 7.4 of [

17] and Examples 16 and 17 in [

1].

A sufficient condition for hypothesis

$H\left(K\right)$ reads as follows, see Theorem 7.4 of [

8] and [

12,

13]. Let

${K}_{i}$,

$i=1,\dots ,k$, be nonempty, closed, convex subsets of

V such that there is

${u}_{0}\in {\bigcap}_{i=1}^{k}\mathrm{int}{K}_{i}$. Then the set

$K:={\bigcup}_{i=1}^{k}$ is star-shaped with respect to a certain ball with center at

${u}_{0}$.

We state below our first existence result.

**Theorem** **1.** Let the hypotheses $H\left(A\right)$, $H\left(j\right)$, $H\left(\phi \right)$, $H\left(K\right)$, and $H\left(f\right)$ hold, and assume the following smallness condition Then, Problem 1 admits a solution $u\in K$.

The proof of this theorem will be given in the next section. The motivation to study Problem 1 is given in the remark below.

**Remark** **1.** If K is a nonempty, closed and convex set, then any solution to Problem 1 is a solution to the classical variational-hemivariational inequality: In fact, let $u\in K$ solve Problem 1 and $v\in K$. The set K is regular, being convex, and by (3), we have Therefore, we deduce that $v\in u+{T}_{K}\left(u\right)$. Then, $u\in K$ is a solution to problem (5). ## 4. Proof of Theorem 1

In this section, we assume the hypotheses of Theorem 1. The proof is based on the penalty method, where the penalty parameter is taken to be small and does not necessary tend to zero. Let $d\equiv {d}_{{K}_{0}}:H\to \mathbb{R}$ be the distance function of the set ${K}_{0}$ defined by ${d}_{{K}_{0}}\left(v\right)={inf\{\parallel v-w\parallel}_{H}\mid w\in {K}_{0}\}$. Let $\lambda >0$ represent a penalty parameter. Consider the penalized problem corresponding to Problem 1:

**Problem** **2.** Find an element ${u}_{\lambda}\in V$ such that We formulate a result on the generalized pseudomonotonicity property of a multivalued mapping.

**Lemma** **3.** Under the assumptions of Theorem 1, the multivalued mapping $A+\partial j+\frac{1}{\lambda}\partial d:V\to {2}^{{V}^{*}}$ is generalized pseudomonotone.

**Proof.** Let

$\left\{{w}_{n}\right\}\subset V$,

$\left\{{w}_{n}^{*}\right\}\subset {V}^{*}$ with

${w}_{n}^{*}\in A{w}_{n}+\partial j\left({w}_{n}\right)+\frac{1}{\lambda}\partial d\left({w}_{n}\right)$,

${w}_{n}\to w$ weakly in

V,

${w}_{n}^{*}\to {w}^{*}$ weakly in

${V}^{*}$ and

We need to show that ${w}^{*}\in Aw+\partial j\left(w\right)+\frac{1}{\lambda}\partial d\left(w\right)$ and $\langle {w}_{n}^{*},{w}_{n}\rangle \to \langle {w}^{*},w\rangle $.

We have

${w}_{n}^{*}=A{w}_{n}+{\xi}_{n}+\frac{1}{\lambda}{\eta}_{n}$ with

${\xi}_{n}\in \partial j\left({w}_{n}\right)$ and

${\eta}_{n}\in \partial d\left({w}_{n}\right)$. From hypothesis

$H\left(j\right)$–(b) and the estimate

${\parallel \partial d\left(v\right)\parallel}_{H}\le 1$ for all

$v\in V$, we infer that

where

$\xi \in {V}^{*}$ and

$\eta \in H$. Since the embedding

$V\subset H$ is compact, we get

Moreover, by the closedness of the graph of

$\partial d:H\to {2}^{H}$ in

$H\times \left(w-H\right)$-topology, see Proposition 3.23(v) in [

17], we deduce that

Subsequently, using the relation

we have

Take the upper limit of both sides,

Exploiting (

6)–(

8), the boundeness of the mapping

$A+\partial j+\frac{1}{\lambda}\partial d$, and the smallness condition (

4), we deduce

Since every single-valued pseudomonotone operator is demicontinuous, see Theorem 3.69(ii) in [

17], we infer

$A{w}_{n}\to Aw$ weakly in

${V}^{*}$. From the convergences (

7) and (

9), we have

$\xi \in \partial j\left(w\right)$. Hence, taking the limit in the equality

${w}_{n}^{*}=A{w}_{n}+{\xi}_{n}+\frac{1}{\lambda}{\eta}_{n}$, we get

Further, by (

9), it is obvious that

$\langle {w}_{n}^{*},{w}_{n}\rangle \to \langle {w}^{*},w\rangle $, which completes the proof. □

We continue the proof with three main steps.

Step 1. We show the existence of solution to Problem 2, for every

$\lambda >0$ fixed. For simplicity, we skip

$\lambda $ in this part of the proof. We define the multivalued mapping

$F:V\to {2}^{V}$ by

and

for

$v\in V$. We note that

Now, we prove that

${\bigcap}_{v\in V}F\left(v\right)\ne \varnothing $. We will apply the KKM lemma, see Lemma 1, with the space

V endowed with the weak topology. We shall verify that the mapping

F defined by (

10) enjoys the properties:

- (a)
for every $v\in V$, the set $F\left(v\right)$ is closed in V,

- (b)
for any finite set $\{{v}_{1},\dots ,{v}_{r}\}\subset V$, we have $\mathrm{co}\{{v}_{1},\dots ,{v}_{r}\}\subset {\bigcup}_{i=1}^{r}F\left({v}_{i}\right),$

- (c)
there is ${v}_{0}\in V$ such that $F\left({v}_{0}\right)$ is compact in V.

We show that the set

$F\left(v\right)$ in bounded in

V for all

$v\in V$. Let

$v\in V$ and

$u\in F\left(v\right)$. Thus

We show that

u stays in a bounded subset of

V. First, by hypothesis

$H\left(j\right)$–(b), (c), we have

Next, using

$H\left(\phi \right)$ and Proposition 5.2.25 in [

27], it follows that

$\phi $ admits an affine minorant, that is, we can find

$l\in {V}^{*}$ and

$b\in \mathbb{R}$ such that

$\phi \left(v\right)\ge \langle l,v\rangle +b$ for all

$v\in V$. Hence

We employ the strong monotonicity of the mapping

A, (

11)–(

13) to deduce

We exploit the global Lipschitz property of the function

d, see Lemma 2.1 in [

14], to infer that

${\parallel \partial d\left(v\right)\parallel}_{H}\le 1$ for all

$v\in V$. Hence

where

${c}_{e}>0$ denotes the embedding constant

$V\subset H$. Using (

14) in the estimate, we obtain

By the smallness condition (

4), since

$\lambda >0$ is fixed, we know that

${\parallel u-v\parallel}_{V}$ is bounded, and thus

${\parallel u\parallel}_{V}$ is also bounded. This completes the proof that

$F\left(v\right)$ is a bounded subset of

V.

First, we establish the property (a). Let

$\lambda >0$ and

$v\in V$ be fixed. It is enough to prove that the set

$F\left(v\right)$ is sequentially weakly closed in

V. Let

$\left\{{u}_{k}\right\}\subset F\left(v\right)$ and

${u}_{k}\to u$ weakly in

V, as

$k\to \infty $. We prove that

$u\in F\left(v\right)$. We have

${M}_{v}\left({u}_{k}\right)\ge 0$ for all

$v\in V$, that is,

From Proposition 3.23(ii) in [

17], there are

${\xi}_{k}\in V$,

${\eta}_{k}\in H$ such that

${\xi}_{k}\in \partial j\left({u}_{k}\right)$,

${\eta}_{k}\in \partial d\left({u}_{k}\right)$ with

Subsequently, by (

16), it follows

and

Here, we have used the fact that

$\phi $ is weakly sequentially l.s.c. (being convex and sequentially l.s.c. by

$H\left(\phi \right)$). Take

$v=u$ in (

17) to get

Let

${u}_{k}^{*}=A{u}_{k}+{\xi}_{k}+\frac{1}{\lambda}{\eta}_{k}\in A{u}_{k}+\partial j\left({u}_{k}\right)+\frac{1}{\lambda}\partial d\left({u}_{k}\right)$. By the boundedness of the mapping

$A+\partial j+\frac{1}{\lambda}\partial d$, we may assume that

for some

${u}^{*}\in {V}^{*}$. Summing up, we obtain

Hence, by the generalized pseudomonotonicity of the mapping

$A+\partial j+\frac{1}{\lambda}\partial d$, see Lemma 3, we have

By (

19), we know that

${u}^{*}=Au+{\xi}_{0}+\frac{1}{\lambda}{\eta}_{0}$ with some

${\xi}_{0}\in \partial j\left(u\right)$,

${\eta}_{0}\in \partial d\left(u\right)$ and

Using the latter, we can pass to the limit in inequality (

17) to get

Hence and by (

21) implies that

$u\in V$ satisfies

This means that $0\le {M}_{v}\left(u\right)$ for all $v\in V$, that is, $u\in F\left(v\right)$ for all $v\in V$. Thus, the set $F\left(v\right)$ is sequentially weakly closed in V.

Second, we show the property (b), that is,

$F:V\to {2}^{V}$ is a KKM map. Let

$r\in \mathbb{N}$,

$\{{z}_{1},\dots ,{z}_{r}\}\subset V$ be an arbitrary finite set. Let

$z={\sum}_{i=1}^{r}{\lambda}_{i}{z}_{i}$ with

${\lambda}_{i}\in (0,1)$ and

${\sum}_{i=1}^{r}{\lambda}_{i}=1$. We suppose by contradiction that

Then, for all

$i=1,\dots ,r$, we have

$z\notin F\left({z}_{i}\right)$. So, for all

$i=1,\dots ,r$, it holds

${M}_{{z}_{i}}\left(z\right)<0$. By this inequality and the convexity of the function

$z\mapsto {M}_{z}\left(u\right)$ for all

$u\in V$, we have

which is a contradiction. This proves (b).

We show property (c): the set $F\left(v\right)$ is weakly compact in V for all $v\in V$. Let $v\in V$ and $\left\{{w}_{n}\right\}\subset V$, ${w}_{n}\in F\left(v\right)$. Since the sequence $\{\parallel {w}_{n}{\parallel}_{V}\}$ is bounded in V by a constant independent of $n\in \mathbb{N}$, by the reflexivity of V, it is clear that there is a subsequence $\left\{{w}_{{n}_{k}}\right\}\subset \left\{{w}_{n}\right\}$ such that ${w}_{{n}_{k}}\to {w}_{0}$ weakly in V, with ${w}_{0}\in V$. Because $F\left(v\right)$ is sequentially weakly closed in V, see property (a), we have ${w}_{0}\in F\left(v\right)$. Therefore, $F\left(v\right)$ is weakly compact in V for all $v\in V$.

Having verified properties (a)–(c), for any $\lambda >0$ fixed, by the KKM lemma, we get that ${\bigcap}_{v\in V}F\left(v\right)\ne \varnothing $. This means that, for any $\lambda >0$ fixed, there exists ${u}_{\lambda}\in V$ solution to problem (2). This finishes Step 1.

Step 2. We show that the solution

${u}_{\lambda}\in V$ to Problem 2 obtained in Step 1 satisfies

${u}_{\lambda}\in K$ for

$\lambda >0$ sufficiently small. We claim that there is a constant

$M>0$ such that

$\parallel {u}_{\lambda}{\parallel}_{V}\le M$ for all

$\lambda >0$. In fact, to obtain the uniform estimate of

${\left\{{u}_{\lambda}\right\}}_{\lambda >0}$ in

V, we choose

$v:={u}_{0}$ in Problem 2, where

${u}_{0}\in V$ is the center of the ball

$\mathbb{B}({u}_{0},\varrho )$ to get

Analogously as in estimate (

15), we have

By Lemma 2, it follows that

${d}^{0}({u}_{\lambda};{u}_{0}-{u}_{\lambda})\le 0$, so we can skip the last term in estimate (

23). We deduce that

$\parallel {u}_{\lambda}{\parallel}_{V}\le M$ with

$M>0$ independent of

$\lambda >0$.

Let

${\lambda}_{0}:=\frac{2\rho}{{d}_{1}M+{d}_{2}}$. We claim that for all

$\lambda \in (0,{\lambda}_{0})$, it holds

${u}_{\lambda}\in K$. We continue by contradiction and assume that

Since

${u}_{\lambda}\notin K$, by Lemma 2, we have

Again, by (

23), we infer that

which implies

a contradiction with the choice of

$\lambda $. Hence

which proves the claim.

Step 3. Fix

$\lambda \in (0,{\lambda}_{0})$ with

${\lambda}_{0}$ defined in Step 2. We will show that

$u:={u}_{\lambda}$ solves Problem 1. From Step 1 we know that

${u}_{\lambda}\in V$ is a solution to Problem 2, and by Step 2, it is obvious that

${u}_{\lambda}\in K$. Thus

$u={u}_{\lambda}\in K$ satisfies

We choose

$v\in u+{T}_{K}\left(u\right)$ as the test function in the latter. Since

$v-u\in {T}_{K}\left(u\right)$, we have

${d}^{0}(u;v-u)=0$. Hence

Finally, $u\in K$ is a solution to Problem 1. This finishes the proof. □

## 5. Second Existence Result

In this section we deliver an existence result for Problem 1 under hypotheses different than the ones of

Section 3. We do not assume the strong monotonicity of the mapping

A, the relaxed monotonicity of the generalized subgradient

$\partial j$, and consider the nonconvex star-shaped admissible set of constraints.

We admit the following assumptions.

- H(A)
_{1}: $A:V\to {V}^{*}$ is a mapping such that

- (a)
for all $v\in V$, $V\ni u\mapsto \langle Au,v-u\rangle \in \mathbb{R}$ is weakly upper semicontinuous,

- (b)
for any $v\in V$, there exists ${m}_{v}>0$ such that $\langle Au,u-v\rangle \ge {\alpha}_{A}{\parallel u\parallel}^{2}$ for

all $\parallel u\parallel \ge {m}_{v}$, where ${\alpha}_{A}>0$.

- H(j)
_{1}: $j:V\to \mathbb{R}$ is a mapping such that

- (a)
j is locally Lipschitz,

- (b)
${\parallel \partial j\left(v\right)\parallel}_{{V}^{*}}\le {c}_{0}+{c}_{1}\phantom{\rule{0.166667em}{0ex}}{\parallel v\parallel}_{V}\phantom{\rule{4pt}{0ex}}\mathrm{for}\mathrm{all}\phantom{\rule{4pt}{0ex}}v\in V\phantom{\rule{4pt}{0ex}}\mathrm{with}\phantom{\rule{4pt}{0ex}}{c}_{0},{c}_{1}\ge 0$,

- (c)
$lim\; sup{j}^{0}({u}_{n};v-{u}_{n})\le {j}^{0}(u;v-u)$ for all $v\in V$ and ${u}_{n}\to u$ weakly in V.

The following example provides a sufficient condition for

$H{\left(j\right)}_{1}$–(c). Let

Y be a reflexive Banach space,

$\psi :Y\to \mathbb{R}$ be a locally Lipschitz function such that

$\psi $ or

$-\psi $ is regular, and

$M:V\to Y$ be given by

$Mv=Lv+{v}_{0}$, where

$L:V\to Y$ represents a linear, compact operator and

${v}_{0}\in Y$ is fixed. In this situation, the function

$j:V\to \mathbb{R}$ defined by

$j\left(v\right)=\psi \left(Mv\right)$ for

$v\in V$ satisfies

$H{\left(j\right)}_{1}$–(c). In applications,

M is a compact trace operator or a compact embedding operator, see, for instance,

Section 6.

We obtain the second existence result by employing the approach used in Theorem 1.

**Theorem** **2.** Under hypotheses $H{\left(A\right)}_{1}$, $H{\left(j\right)}_{1}$, $H\left(\phi \right)$, $H\left(K\right)$, $H\left(f\right)$ and the smallness condition Problem 1 has a solution $u\in K$.

**Proof.** We use the notation of

Section 4 and treat Problem 1 by the penalized inequality stated in Problem 2. We follow the three steps in the proof of Theorem 1. We only indicate below the new ingredients of the proof.

(i) We prove that the set

$F\left(v\right)$ in bounded in

V for all

$v\in V$, where the multivalued mapping

$F:V\to {2}^{V}$ is given by (

10). Let

$v\in V$ and

$u\in F\left(v\right)$. Hence

From this inequality, we use

$H{\left(J\right)}_{1}$–(b), (

13) and (

14) to obtain

By the condition

$H{\left(A\right)}_{1}$–(a) and the smallness condition (

24), we deduce that

${\parallel u\parallel}_{V}$ is bounded by a constant which depends on

v but is independent of

u. This completes the proof that

$F\left(v\right)$ is a bounded set in

V.

(ii) We prove that for

$\lambda >0$ and

$v\in V$ fixed, the set

$F\left(v\right)$ is sequentially weakly closed in

V. Let

$\left\{{u}_{k}\right\}\subset F\left(v\right)$ and

${u}_{k}\to u$ weakly in

V, as

$k\to \infty $. This means that

${M}_{v}\left({u}_{k}\right)\ge 0$ for all

$v\in V$, and

We show that

${M}_{v}(\xb7):V\to \mathbb{R}$ is sequentially weakly upper semicontinuous, that is,

We take upper limit in (

27) to get

Again, by the compactness of the embedding

$V\subset H$, we apply Proposition 3.23(ii) in [

17] to deduce

We use

$H{\left(A\right)}_{1}$–(a),

$H{\left(j\right)}_{1}$–(c),

$H\left(\phi \right)$ and (

29) in (

28) to obtain

Hence $0\le {M}_{v}\left(u\right)$ and $u\in F\left(v\right)$. This completes this proof.

(iii) We show that the solution

${u}_{\lambda}\in V$ to Problem 2 satisfies

$\parallel {u}_{\lambda}{\parallel}_{V}\le {M}_{1}$ for all

$\lambda >0$ with

${M}_{1}>0$ independent of

$\lambda $. We take

$v:={u}_{0}$ in the penalized Problem 2, where, recall,

${u}_{0}\in V$ is the center of the ball

$\mathbb{B}({u}_{0},\varrho )$. We have

Simarily as in (

26), we infer that

Again, by the condition

$H{\left(A\right)}_{1}$–(a) and the smallness condition (

24), we deduce that

$\parallel {u}_{\lambda}{\parallel}_{V}$ is bounded by a constant

${M}_{1}>0$ which is independent of

$\lambda $.

The remaining parts of the proof follow the lines in the proof of Theorem 1. □

From Remark 1 and Theorem 2, the deduce the following result on the existence of solution to the variational-hemivariational inequality with the convex constraint set.

**Corollary** **1.** Assume hypotheses $H{\left(A\right)}_{1}$, $H{\left(j\right)}_{1}$, $H\left(\phi \right)$, $H\left(f\right)$ and the smallness condition (24). If K is a nonempty, closed and convex set, then Problem 5 has a solution $u\in K$. We conclude the section with comments on the assumptions $H\left(A\right)$–(a) and $H{\left(A\right)}_{1}$–(a).

(1) If

$A:V\to {V}^{*}$ is monotone, bounded and

$\left(w-V\right)\times \left(w-{V}^{*}\right)$ continuous, then

$H\left(A\right)$–(a) holds. This follows by an observation that every

$\left(w-V\right)\times \left(w-{V}^{*}\right)$ continuous operator is demicontinuous. Then, the notions of demicontinuity and hemicontinuity coincide for monotone operators, see Exercise VI.9 in [

10]. Finally, by Theorem 3.69(i) in [

17], a bounded, monotone and hemicontinuous mapping is pseudomonotone, that is,

$H\left(A\right)$–(a) holds. Note that linear and bounded operators, and most quasilinear differential operators are weakly-weakly continuous, see e.g., [

11].

(2) If

$A:V\to {V}^{*}$ is monotone and

$\left(w-V\right)\times \left(w-{V}^{*}\right)$ continuous, then

$H{\left(A\right)}_{1}$–(a) holds. To prove this, first, we observe that if

A is monotone, then for all

${u}_{n}\to u$ weakly in

V, we have

$lim\; sup\langle A{u}_{n},u-{u}_{n}\rangle \le 0$. Indeed, we proceed by contradiction. Suppose that there are

$\left\{{w}_{n}\right\}\subset V$,

$w\in V$ such that

${w}_{n}\to w$ weakly in

V and

$lim\; sup\langle A{w}_{n},w-{w}_{n}\rangle >0$. The latter is equivalent to

$lim\; inf\langle A{w}_{n},{w}_{n}-w\rangle <0$. On the other hand by the monotonicity of

A, we have

We take lower limit in this inequality, and deduce

which is a contradiction. Second, to prove

$H{\left(A\right)}_{1}$–(a), let

$v\in V$,

${u}_{n}\to u$ weakly in

V. Then,

$A{u}_{n}\to Au$ weakly in

${V}^{*}$ and

where we have used the inequality

$lim\; sup\langle A{u}_{n},u-{u}_{n}\rangle \le 0$. Hence,

$H{\left(A\right)}_{1}$–(a) is verified.

(3) It is clear that if the mapping

A is strongly monotone with constant

${m}_{A}>0$, see

$H\left(A\right)$–(b), then

A is coercive in the sense that

and

compare with

$H{\left(A\right)}_{1}$–(b).

## 6. Semipermeability Model

In this section we provide an illustrative model which weak formulation leads to a constrained variational-hemivariational inequality. Based on this model, we justify the nature of operator M which may appear in applications.

Consider the following semipermeability model for the stationary heat conduction problem. Let $\Omega \subset {\mathbb{R}}^{d}$ be a bounded domain with the regular boundary $\Gamma $ which consists of three mutually disjoint and relatively open subsets ${\Gamma}_{1}$, ${\Gamma}_{2}$ and ${\Gamma}_{3}$ with $\Gamma ={\overline{\Gamma}}_{1}\cup {\overline{\Gamma}}_{2}\cup {\overline{\Gamma}}_{3}$ and $m\left({\Gamma}_{1}\right)>0$. Consider the following boundary value problem.

**Problem** **3.** Find a temperature $u:\Omega \to \mathbb{R}$ such that $u\in U$ and where $A:V\to {V}^{*}$ is a given linear mapping, ν denotes the outward normal on the boundary, and $\frac{\partial u}{\partial {\nu}_{A}}$ is the conormal derivative with respect to A and represents the heat flux through a part of the boundary. Further, $U\subset V$ is a nonempty, closed set of constraints which can be convex or nonconvex, and $V=\{v\in {H}^{1}(\Omega )\mid v=0\phantom{\rule{4pt}{0ex}}on\phantom{\rule{4pt}{0ex}}{\Gamma}_{1}\}$. We denote by $i:V\to {L}^{2}(\Omega )$ the embedding operator and by $\gamma :V\to {L}^{2}(\Gamma )$ the trace operator. It is well known that both operators are linear and compact.

Problem 3 is motivated by several kinds of semipermeability relations which arise in several situations in hydraulics, flow problems through porous media and electrostatics, the solution can represent temperature, pressure and the electric potential Chapter I in [

31] (where the monotone semipermeability relations were considered with convex potentials), and Chapter 5.5.3 of [

8,

16] (where nonmonotone subdifferential conditions were treated). Mappings

${g}_{1}$ and

${j}_{1}$ describe the interior semipermeability phenomena while

${g}_{2}$ and

${j}_{2}$ provide the boundary semipermeability relations in the subdifferential form. Additional constraints for the temperature (or the pressure of the fluid in a fluid flow model) are represented by the condition

$u\in U$. The set

U can be employed to introduce a bilateral obstacle which means that we look for the temperature within prescribed bounds in the domain

$\Omega $. The function

${f}_{1}$ corresponds to the density of heat sources in the domain. The multivalued subdifferential boundary conditions on

${\Gamma}_{2}$ (and

${\Gamma}_{3}$) describe the nonmonotone (and monotone, respectively) behavior of a semipermeable membrane (a wall) of finite thickness, and appear in a temperature control problem, see [

32].

We need the following hypotheses on the data.

- H(A)
_{2}: $A:V\to {V}^{*}$ is a mapping such that $A=-{\sum}_{i,j=1}^{d}{D}_{i}\left({a}_{ij}\left(x\right){D}_{j}\right)$, and

- (i)
${a}_{ij}\in {L}^{\infty}(\Omega )$ for i, $j=1,\dots ,d$.

- (ii)
${\sum}_{i,j=1}^{d}{a}_{ij}\left(x\right){\xi}_{i}{\xi}_{j}\ge {\alpha}_{0}{\parallel \xi \parallel}^{2}$ for all $\xi \in {\mathbb{R}}^{d}$, a.e. $x\in \Omega $ with ${\alpha}_{0}>0$.

- H(j
_{1}): ${j}_{1}:\mathbb{R}\to \mathbb{R}$ is such that

- (i)
${j}_{1}$ is locally Lipschitz.

- (ii)
$|\partial {j}_{1}\left(r\right)|\le {c}_{0j}+{c}_{1j}\left|r\right|$ for all $r\in \mathbb{R}$ with ${c}_{0j}$, ${c}_{1j}\ge 0$.

- (iii)
$(\partial {j}_{1}\left({r}_{1}\right)-\partial {j}_{1}\left({r}_{2}\right))({r}_{1}-{r}_{2})\ge -{\beta}_{1j}{|{r}_{1}-{r}_{2}|}^{2}$ all ${r}_{1}$, ${r}_{2}\in \mathbb{R}$ with ${\beta}_{1j}\ge 0$.

- H(j
_{2}): ${j}_{2}:\mathbb{R}\to \mathbb{R}$ is such that

- (i)
${j}_{2}$ is locally Lipschitz.

- (ii)
$|\partial {j}_{2}\left(r\right)|\le {c}_{2j}+{c}_{3j}\left|r\right|$ for all $r\in \mathbb{R}$ with ${c}_{2j}$, ${c}_{3j}\ge 0$.

- (iii)
$(\partial {j}_{2}\left({r}_{1}\right)-\partial {j}_{2}\left({r}_{2}\right))({r}_{1}-{r}_{2})\ge -{\beta}_{2j}{|{r}_{1}-{r}_{2}|}^{2}$ all ${r}_{1}$, ${r}_{2}\in \mathbb{R}$ with ${\beta}_{2j}\ge 0$.

- H(g
_{1}): ${g}_{1}:\mathbb{R}\to \mathbb{R}$ is such that

- (i)
${g}_{1}$ is convex and l.s.c.

- (ii)
$|\partial {g}_{1}\left(r\right)|\le {c}_{0g}+{c}_{1g}\left|r\right|$ for all $r\in \mathbb{R}$ with ${c}_{0g}$, ${c}_{1g}\ge 0$.

- H(g
_{2}): ${g}_{2}:\times \mathbb{R}\to \mathbb{R}$ is such that

- (i)
${g}_{2}$ is convex and l.s.c.

- (ii)
$|\partial {g}_{2}\left(r\right)|\le {c}_{2g}+{c}_{3g}\left|r\right|$ for all $r\in \mathbb{R}$ with ${c}_{2g}$, ${c}_{3g}\ge 0$.

- H(f):
${f}_{1}\in {L}^{2}(\Omega ),\phantom{\rule{4pt}{0ex}}{f}_{2}\in {L}^{2}\left({\Gamma}_{2}\right),\phantom{\rule{4pt}{0ex}}{f}_{b}\in {L}^{2}\left({\Gamma}_{3}\right).$

- (H
_{0}): ${\alpha}_{0}>{\beta}_{1j}{\parallel i\parallel}^{2}+{\beta}_{2j}{\parallel \gamma \parallel}^{2}.$

Assume that U is a convex set. Under the hypotheses above, by a standard procedure, we obtain the following weak formulation of Problem 3.

**Problem** **4.** Find $u\in U$ such that
where

$f\in {V}^{*}$ is given by

We introduce the functionals

${G}_{1}$,

${G}_{2}$,

${J}_{1}$,

${J}_{2}:V\to \mathbb{R}$ defined by

for

$v\in V$. Let

$\phi ={G}_{1}+{G}_{2}$ and

$j={J}_{1}+{J}_{2}$. We observe that from Propositions 3.37(i) and 3.46(iv) in [

17], we get

Using the last inequality, definitions of the convex subdifferential and the Clarke subgradient, we easily deduce that if

$u\in U$ solves the problem: find an element

$u\in U$ such that

then

u is a solution to Problem 4.

If the set of constraints U is a nonempty, closed and nonconvex subset of V, we can derive a weak formulation as in Problem 4. Then, Theorems 1 and 2 can be applied in this situation.