Localization Operators for the Linear Canonical Dunkl Windowed Transformation

Abstract

One of the best known time–frequency tools for examining non-transient signals is the linear canonical windowed transform, which has been used extensively in signal processing and related domains. In this paper, by involving the harmonic analysis for the linear canonical Dunkl transform, we introduce and then study the linear canonical Dunkl windowed transform (LCDWT). Given that localization operators are both theoretically and practically relevant, we will focus in this paper on a number of time–frequency analysis topics for the LCDWT, such as the $L^p$ boundedness and compactness of localization operators for the LCWGT. Then, we study their trace class characterization and show that they are in the Schatten–von Neumann classes. Then, we study their spectral properties in order to give some results on the spectrograms for the LCDWT.

1. Introduction

Charles Dunkl [1] introduced and explored Dunkl operators as part of an effort to expand the classical theory of spherical harmonics. They are used in quantum mechanics to examine one-dimensional harmonic oscillators that are subject to Wigner’s commutation principles, in addition to their mathematical significance.

For every function $f\in L_k^1({\mathbb{R}}^N)\cap L_k^2({\mathbb{R}}^N)$, its Dunkl transform ${\mathfrak{F}}_k(f)$ associated with a reflection group W and a non-negative multiplicity function k is given by

$$\begin{array}{c}\hfill {\displaystyle {\mathfrak{F}}_k(f)(\xi )={\int }_{{\mathbb{R}}^N}f(t)e_k(-it,\xi )d{\gamma }_k(t),\xi \in {\mathbb{R}}^N,}\end{array}$$

(1)

where $d{\gamma }_k$ is a weighted measure and $e_k(-i·,·)$ represents the Dunkl kernel. The Dunkl transform ${\mathfrak{F}}_D$ offers a natural extension of the standard Fourier transform for $k=0$ and may be uniquely extended to an isometric isomorphism on $L_k^2({\mathbb{R}}^N)$. We direct the reader to [2,3,4] and the references therein for a thorough understanding of the Dunkl transform.

Numerous significant advancements in the field of phase-space analysis have previously been studied within the Dunkl transform. For instance, the authors in [5,6] developed many uncertainty inequalities for the Dunkl windowed transform. Furthermore, in [7,8], the authors examined several relations of quantitative uncertainty principles related to the continuous Dunkl wavelet transform.

The recent advancements in deformed windowed transforms have greatly inspired us to investigate a new windowed transform employing a pair of generalized translation and modulation linked to the linear canonical Dunkl transform found in [9].

A generic homogeneous linear lossless mapping in phase space is related with the classical linear canonical transform (LCT), which is an integral transform that is independent of phase space coordinates and depends on three parameters. It was separately developed to investigate the conservation of information and uncertainty under linear mappings of phase space by Collins [10] in paraxial optics and Moshinsky-Quesne [11] in quantum mechanics.

The three parameters constitute an augmented matrix consisting of a uni-modular matrix of $2\times 2$. It maps the position x and the wave number y into

$$\begin{array}{c}\left(\begin{array}{c}{x}^{\prime }\\ y^{\prime }\end{array}\right)=\left(\begin{array}{cc}a& b\\ c& d\end{array}\right)\left(\begin{array}{c}x\\ y\end{array}\right),\hfill \end{array}$$

where $ad-bc=1.$

It is possible to convert any convex body into another convex body while preserving the body’s area. The homogeneous special group $SL(2,R)$ is composed of these modifications.

Given a real augmented matrix

$$M:=\left(\begin{array}{cc}a& b\\ c& d\end{array}\right)\in SL(2,\mathbb{R}).$$

Then the linear canonical transform of any function f with respect to M is defined by

$${\displaystyle {\mathfrak{F}}^{\mathbf{m}}(x)={\displaystyle \frac{1}{{(ib)}^{\frac{1}{2}}}}{\int }_{\mathbb{R}}{\mathcal{K}}^{\mathbf{m}}(x,y)f(y)dy,}$$

(2)

where

$${\mathcal{K}}^{\mathbf{m}}(x,y)=e^{{\displaystyle \frac{i}{2}}({\displaystyle \frac{d}{b}}{x}^2+{\displaystyle \frac{a}{b}}{y}^2)}{e}^{{\displaystyle \frac{-ixy}{b}}}.$$

(3)

The LCT is a generalized Fourier transform that is an extension of several well-known integral transformations, such as the Fourier transform, the Fresnel transform, the fractional Fourier transform, scaling operations, etc.

The LCT is more versatile than other transforms because of its additional degrees of freedom and straightforward geometrical representation, making it a useful and effective tool for examining deep issues in signal processing, quantum physics, and optics [12,13,14,15].

Recently, the LCT has gained considerable attention and has been extended to a wide class of integral transforms; see, for example [16,17,18,19,20], and the references therein.

Over the past few decades, the LCT’s application areas have grown rapidly, making it suitable for investigating complex problems in a variety of fields, including holographic three-dimensional television, quantum physics, pattern recognition, radar analysis, signal analysis, filter design, phase retrieval problems, and many other fields, such as uncertainty principles [21], Poisson summation formulae [22], convolution theorems [23], and sampling theorems [24].

The purpose of this document is threefold. First by using the main tools related to the linear canonical Dunkl transform (LCDT) [9], we present and study a new windowed-type transform, which we will call the linear canonical Dunkl windowed transform (LCDWT). Then, we will show the main theorems of the harmonic analysis of this transform. The windowed Fourier transformation has proven to be highly effective in a variety of physical and technical applications, such as quantum optics, signal processing, and wave propagation [25]. Moreover, it has seen several developments in recent years. For instance, we refer to [6,26,27,28,29,30] and the references therein.

The following are this article’s main contributions:

-

To derive a new inversion relation for the LCDWT.

-

To study the boundedness and compactness of the localization operators associated with the LCDWT in the Schatten classes.

-

To study the eigenvalues and eigenfunctions of the time–frequency Toeplitz operator.

-

To obtain some results on the spectrogram associated with the LCDWT.

The remaining part of this paper is organized as follows: In Section 2, we will state the necessary materials concerning the harmonic analysis related to the Dunkl transform and the linear canonical Dunkl transform studied in [9]. In Section 3, we will present and study a generalized windowed-type transform related to the LCDT. More precisely, we will prove a Plancherel-type, a Lieb-type theorem, and an inversion formula. Then, the study of localization operators theory in the context of the LCDWT is the focus of Section 4. Specifically, in the Schatten classes, we examine their boundedness and compactness, and in order to characterize functions with time–frequency content in a subset of finite measure, we lastly examine the spectral analysis related to the time–frequency Toeplitz operators. Additionally, we provide and study the spectrogram related to the LCDWT.

2. The Linear Canonical Dunkl Transform and Its Properties

We will outline the requirements for the Dunkl and linear canonical Dunkl transforms in this section. The primary sources include [1,2,3,4,31].

2.1. Dunkl Transform: Properties, Translation, and Convolution

Let $\{e_i,i=1,\dots ,N\}$ be the orthonormal basis of ${\mathbb{R}}^N$, and for $x\in {\mathbb{R}}^N$, we denote $\parallel x\parallel =\sqrt{⟨x,x⟩}$. The reflection ${\sigma }_{\alpha }$ in the hyperplane $H_{\alpha }\subset {\mathbb{R}}^N$ orthogonal to $\alpha \in {\mathbb{R}}^N\setminus \left\{0\right\}$ is defined by

$${\sigma }_{\alpha }(x)=x-2{\displaystyle \frac{⟨\alpha ,x⟩}{{\parallel \alpha \parallel }^2}}\alpha .$$

(4)

Let $\mathcal{R}$ be a finite subset of ${\mathbb{R}}^N\setminus \left\{0\right\}$. If $\mathcal{R}\cap \mathbb{R}\alpha =\{\pm \alpha \}$ and if for all $\alpha \in \mathcal{R}$, ${\sigma }_{\alpha }(\mathcal{R})=\mathcal{R}$, then $\mathcal{R}$ is a root system.

It is quite obvious that the reflections ${\sigma }_{\alpha },\alpha \in \mathcal{R}$ associated with root system $\mathcal{R}$ generate a finite group $W\subset O(N)$, called the reflection group. We note that there is a bijective correspondence between the conjugacy classes of reflections in W and the orbits in $\mathcal{R}$ under the natural action of W.

Let $\beta \in {\mathbb{R}}^N\setminus {\bigcup }_{\alpha \in \mathcal{R}}{H}_{\alpha }$. Fix a positive root system ${\mathcal{R}}_{+}=\left\{\alpha \in \mathcal{R}:⟨\alpha ,\beta ⟩>0\right\}$. We will assume that for all $\alpha \in {\mathcal{R}}_{+}$, we have $⟨\alpha ,\alpha ⟩=2$.

If a positive function k defined in $\mathcal{R}$ remains invariant under the action of the reflection group W, then it is considered as a multiplicity function.

In addition, the weight function ${\omega }_k$ is defined as

$${\omega }_k(x)=\prod _{\alpha \in {\mathcal{R}}_{+}}{|⟨\alpha ,x⟩|}^{2k(\alpha )},$$

(5)

and we introduce the index

$${\displaystyle ⟨k⟩=\sum _{\alpha \in {\mathcal{R}}_{+}}k(\alpha ).}$$

(6)

It is evident that ${\omega }_k$ is homogeneous of degree $2⟨k⟩$ and W-invariant.

Finally, we introduce the well-known Mehta-type constant [32]:

$${\displaystyle c_k={\int }_{{\mathbb{R}}^N}{e}^{-\frac{{\parallel x\parallel }^2}{2}}{\omega }_k(x)dx.}$$

(7)

For the finite reflection group W and for the multiplicity function k, the Dunkl operators $T_j,j=1,\dots ,d$ are defined as follows. For any function $f\in C^1({\mathbb{R}}^N)$,

$${\displaystyle T_{j}f(x):=\frac{\partial f}{\partial x_j}(x)+\sum _{\alpha \in {\mathcal{R}}_{+}}k(\alpha ){\alpha }_j\frac{f(x)-f({\sigma }_{\alpha }(x))}{⟨\alpha ,x⟩},x\in {\mathbb{R}}^N,}$$

(8)

where ${\alpha }_j=⟨\alpha ,e_j⟩$, and $C^p({\mathbb{R}}^N)$ is the space of functions of class $C^p$ on ${\mathbb{R}}^N$.

The Dunkl operators constitute a commutative system of differential-difference operators. Likewise, for each $f\in C^2({\mathbb{R}}^N)$, we may define the Dunkl–Laplacian operator ${▵}_k$ as

$$\begin{array}{cc}\hfill {▵}_{k}f(x)& {\displaystyle :=\sum _{j=1}^N{T}_j^{2}f(x)=▵f(x)+2\sum _{\alpha \in {\mathcal{R}}^{+}}k(\alpha )\left({\displaystyle \frac{⟨\nabla f(x),\alpha ⟩}{⟨\alpha ,x⟩}}-{\displaystyle \frac{f(x)-f({\sigma }_{\alpha }(x))}{{⟨\alpha ,x⟩}^2}}\right),x\in {\mathbb{R}}^N,}\hfill \end{array}$$

where Δ and ∇ denote the Euclidean Laplacian and the gradient operators on ${\mathbb{R}}^N$, respectively.

For $y\in {\mathbb{R}}^N$, the system

$$\left\{\begin{array}{ccc}{T}_{j}u(x,y)\hfill & =\hfill & y_{j}u(x,y),j=1,\dots ,d,\hfill \\ u(0,y)\hfill & =\hfill & 1,\hfill \end{array}\right.$$

(9)

admits a unique analytic solution on ${\mathbb{R}}^N$, called the Dunkl kernel. It has the following properties:

• $e_k(-i·,·)$ has a unique holomorphic extension to ${\mathbb{C}}^N\times {\mathbb{C}}^N$.
• For all $z,t,\in {\mathbb{C}}^N$ and $\lambda \in \mathbb{C}$, we have $e_k(z,t)=e_k(t,z)$, $e_k(z,0)=1$ and $e_k(\lambda z,t)=e_k(z,\lambda t)$.
• For all $\nu \in {\mathbb{N}}^N,x\in {\mathbb{R}}^N$ and $z\in {\mathbb{C}}^N$, we have

  $$|D_z^{\nu }{e}_k(-ix,z)|\le {\parallel x\parallel }^{|\nu |}exp\left(\parallel x\parallel \parallel \mathrm{Re}z\parallel \right),$$

  (10)

  where $D_z^{\nu }={\displaystyle \frac{{\partial }^{|\nu |}}{\partial z_1^{{\nu }_1}\cdots \partial z_N^{{\nu }_N}}}$ and $|\nu |={\nu }_1+\cdots +{\nu }_N$. In particular, $|e_k(ix,y)|\le 1$, for all $x,y\in {\mathbb{R}}^N$.
• $e_k(-i·,·)$ admits the following Laplace-type integral representation:

  $${\displaystyle e_k(-ix,z)={\int }_{{\mathbb{R}}^N}exp(⟨y,z⟩)d{\nu }_x(y),}$$

  (11)

  where ${\nu }_x$ denotes the positive probability measure on ${\mathbb{R}}^N$, with support in $B_N(0,\parallel x\parallel )$ of center 0 and radius $\parallel x\parallel$ (see [33]).

Let $L_k^p({\mathbb{R}}^N),1\le p<\infty ,$ the space of Borel measurable functions f on ${\mathbb{R}}^N$ satisfying

$$\begin{array}{ccc}{\parallel f\parallel }_{L_k^p({\mathbb{R}}^N)}\hfill & :=\hfill & {\left({\displaystyle {\int }_{{\mathbb{R}}^N}{|f(t)|}^{p}d{\gamma }_k(t)}\right)}^{1/p}<\infty ,if1\le p<\infty ,\hfill \\ {\parallel f\parallel }_{L_k^{\infty }({\mathbb{R}}^N)}\hfill & :=\hfill & {\displaystyle \mathrm{ess}\underset{t\in {\mathbb{R}}^N}{sup}|f(t)|<\infty ,}\hfill \end{array}$$

where $d{\gamma }_k(t):={c_k}^{-1}{\omega }_k(t)dt.$ Specifically, $L_k^2({\mathbb{R}}^N)$ is a Hilbert space that has the scalar product

$${\displaystyle {⟨f,g⟩}_{L_k^2({\mathbb{R}}^N)}:={\int }_{{\mathbb{R}}^N}f(x)\overline{g(x)}d{\gamma }_k(x).}$$

If $\mathfrak{R}$ denotes the space of all $\mathbb{C}$-valued functions on ${\mathbb{R}}^N$, then we set

$${\mathfrak{R}}_{\mathrm{rad}}:=\left\{g\in \mathfrak{R}:g\circ M=g,\forall M\in O(N,\mathbb{R})\right\},$$

as the subspace of radial functions in $\mathfrak{R}$. Therefore, for any $g\in {\mathfrak{R}}_{\mathrm{rad}}$, there exists a unique function $G:{\mathbb{R}}_{+}\to \mathbb{C}$ such that $g(t)=G(\parallel t\parallel )$, for every $t\in {\mathbb{R}}^N$.

Remark 1. 

In [31], it is proved that for a radial function $g\in L_k^1({\mathbb{R}}^N)$, the function G defined by $g(·)=G(\parallel ·\parallel )$ is integrable with regard to the measure $r^{2⟨k⟩+N-1}dr$. More precisely,

$${\displaystyle {\int }_{{\mathbb{R}}^N}g(t)d{\gamma }_k(t)=d_k{\int }_{{\mathbb{R}}^N}G(r)r^{2⟨k⟩+N-1}dr,}$$

(12)

where

$$d_k:={\displaystyle \frac{1}{2^{⟨k⟩+\frac{N}{2}-1}\mathrm{\Gamma }\left(⟨k⟩+{\displaystyle \frac{N}{2}}\right)}}.$$

(13)

The Dunkl transform of any function $f\in L_k^1({\mathbb{R}}^N)$ is defined by

$${\displaystyle {\mathfrak{F}}_k(f)(\xi )={\int }_{{\mathbb{R}}^N}f(x)e_k(-ix,\xi )d{\gamma }_k(x),\xi \in {\mathbb{R}}^N.}$$

(14)

The Dunkl transform satisfies the following properties (see [2,3]).

• For any $f\in L_k^1({\mathbb{R}}^N)$, we have

  $$\parallel {\mathfrak{F}}_k{(f)\parallel }_{L_k^{\infty }({\mathbb{R}}^N)}\le {\parallel f\parallel }_{L_k^1({\mathbb{R}}^N)}.$$

  (15)
• Inversion formula: If $f\in L_k^1({\mathbb{R}}^N)$ is a function such that its Dunkl transform ${\mathfrak{F}}_D(f)$ is in $L_k^1({\mathbb{R}}^N)$, then

  $${\mathfrak{F}}_k^{-1}(f)(·)={\mathfrak{F}}_k(f)(-·),a.e.$$

  (16)
• The Dunkl transform ${\mathfrak{F}}_k$ is a topological isomorphism from the Schwartz space $\mathcal{S}({\mathbb{R}}^N)$ onto itself. If $f\in \mathcal{S}({\mathbb{R}}^N)$ satisfies

  $$\overline{{\mathfrak{F}}_k}(f)(\xi )={\mathfrak{F}}_k(f)(-\xi ),\xi \in {\mathbb{R}}^N,$$

  (17)

  then,

  $${\mathfrak{F}}_k\overline{{\mathfrak{F}}_k}=\overline{{\mathfrak{F}}_k}{\mathfrak{F}}_k=Id.$$
• Plancherel-type relation: For all $f\in \mathcal{S}({\mathbb{R}}^N)$,

  $${\displaystyle {\int }_{{\mathbb{R}}^N}{|f(t)|}^{2}d{\gamma }_k(t)={\int }_{{\mathbb{R}}^N}{|{\mathfrak{F}}_k(f)(\xi )|}^{2}d{\gamma }_k(\xi ).}$$

  (18)
• Parseval-type relation: For all $f,g\in \mathcal{S}({\mathbb{R}}^N)$,

  $${\displaystyle {\int }_{{\mathbb{R}}^N}f(t)\overline{g(t)}d{\gamma }_k(t)={\int }_{{\mathbb{R}}^N}{\mathfrak{F}}_k(f)(\xi )\overline{{\mathfrak{F}}_k(g)}(\xi )d{\gamma }_k(\xi ).}$$

  (19)

Proposition 1. 

If $f\in D({\mathbb{R}}^N)$ (resp. $\mathcal{S}({\mathbb{R}}^N))$, then

$${\mathfrak{F}}_k(\overline{f})=\overline{{\mathfrak{F}}_k(\breve{f})},$$

(20)

and

$${\mathfrak{F}}_k(f)(·)={\mathfrak{F}}_k(\breve{f})(-·),$$

(21)

where $\breve{g}$ is the function given by $\breve{g}(·)=g(-·),$ and $D({\mathbb{R}}^N)$ is the space of $C^{\infty }$-functions on ${\mathbb{R}}^N$ which are of compact support.

Definition 1. 

The Dunkl translation operator $f\mapsto {\tau }_{y}f$ is defined on $L_k^2({\mathbb{R}}^N)$ by (see [31])

$${\mathfrak{F}}_k({\tau }_{y}f)=e_k(iy,.){\mathfrak{F}}_k(f),y\in {\mathbb{R}}^N.$$

(22)

The Wigner-type space of all functions that fulfill (22) point-wise is given by

$${\mathcal{W}}_k({\mathbb{R}}^N):=\left\{f\in L_k^1({\mathbb{R}}^N):{\mathfrak{F}}_k(f)\in L_k^1({\mathbb{R}}^N)\right\}.$$

The following properties of the Dunkl translation operator can be found in [31,34]:

• For all $f\in L_k^2({\mathbb{R}}^N)$, we have

  $$\parallel {\tau }_y{f\parallel }_{L_k^2({\mathbb{R}}^N)}\le {\parallel f\parallel }_{L_k^2({\mathbb{R}}^N)}.$$

  (23)
• For all $f\in {\mathcal{W}}_k({\mathbb{R}}^N)$,

  $${\displaystyle {\tau }_{y}f(x)={\int }_{{\mathbb{R}}^N}{e}_k(iy,\xi )e_k(ix,\xi ){\mathfrak{F}}_k(f)(\xi )d{\gamma }_k(\xi ),x\in {\mathbb{R}}^N.}$$
• For every $f\in {\mathcal{W}}_k({\mathbb{R}}^N)$,

  $${\tau }_{y}f(x)={\tau }_{x}f(y).$$

  (24)

Remark 2. 

Notice that Trimèche in [4] has defined the Dunkl translation on the space of $C^{\infty }$-functions on ${\mathbb{R}}^N$ as

$${\tau }_{x}f(y)={(V_k)}_x{(V_k)}_y[{(V_k)}^{-1}f(x+y)],x,y\in {\mathbb{R}}^N,$$

(25)

where $V_k$ is the Dunkl transmutation operator.

Until now, an explicit formula for the Dunkl translation operator is known only in the following cases: If $N=1$ and $W={\mathbb{Z}}_2$, then for all $x\in \mathbb{R}$ and $f\in C(\mathbb{R})$, we have

$$\begin{array}{ccc}{\tau }_{y}f(x)& =& {\displaystyle {\displaystyle \frac{1}{2}}{\int }_{-1}^{1}f\left(\sqrt{x^2+y^2-2xyt}\right)\left(1+\frac{x-y}{\sqrt{x^2+y^2-2xyt}}\right){\mathrm{\Phi }}_k(t)d{\gamma }_k(t)}\\ & +& {\displaystyle {\displaystyle \frac{1}{2}}{\int }_{-1}^{1}f\left(-\sqrt{x^2+y^2-2xyt}\right)\left(1-\frac{x-y}{\sqrt{x^2+y^2-2xyt}}\right){\mathrm{\Phi }}_k(t)d{\gamma }_k(t),}\end{array}$$

where

$${\mathrm{\Phi }}_k(t)={\displaystyle \frac{\mathrm{\Gamma }(k+\frac{1}{2})}{\sqrt{\pi }\mathrm{\Gamma }(k)}}(1+t){(1-t^2)}^{k-1}.$$

If $W={\mathbb{Z}}_2^N$, an explicit formula for the translation operator may also be obtained from the previous result. Additionally, we have the boundedness result shown below (see [34]).

Proposition 2. 

Let $1\le p\le \infty$. If $W={\mathbb{Z}}_2^N$, then for every $f\in L_k^p({\mathbb{R}}^N)$,

$$\parallel {\tau }_y{f\parallel }_{L_k^p({\mathbb{R}}^N)}\le 2^{{\displaystyle \frac{N}{2p}}|p-2|}{\parallel f\parallel }_{L_k^p({\mathbb{R}}^N)}.$$

(26)

Moreover, if $f\in {\mathcal{W}}_k({\mathbb{R}}^N)$ is radial, then

$${\displaystyle {\tau }_{y}f(x)={\int }_{B_N(0,\parallel x\parallel )}\tilde{f}(\sqrt{{\parallel x\parallel }^2+{\parallel y\parallel }^2+2⟨x,z⟩})d{\nu }_x(z),\forall x\in {\mathbb{R}}^N,}$$

where $\tilde{f}$ is the function defined by $f(·)=\tilde{f}(\parallel ·\parallel )$ and ${\nu }_x$ is the measure given by (11).

The subspace of radial functions in $L_k^p({\mathbb{R}}^N)$ is denoted by $L_{k,\mathrm{rad}}^p({\mathbb{R}}^N)$. Following that, we have (see [34]):

Proposition 3. 

1.

If $f\in L_{k,\mathrm{rad}}^1({\mathbb{R}}^N)$ is positive, then for all $x\in {\mathbb{R}}^N$

$${\tau }_{x}f\ge 0,{\tau }_{x}f\in L_k^1({\mathbb{R}}^N),$$

and

$${\displaystyle {\int }_{{\mathbb{R}}^N}{\tau }_{x}f(t)d{\gamma }_k(t)={\int }_{{\mathbb{R}}^N}f(t)d{\gamma }_k(t).}$$

(27)

2.

For all $f\in L_{k,\mathrm{rad}}^p({\mathbb{R}}^N)$, $1\le p\le \infty$, we have

$$\parallel {\tau }_x{f\parallel }_{L_k^p({\mathbb{R}}^N)}\le {\parallel f\parallel }_{L_k^p({\mathbb{R}}^N)},x\in {\mathbb{R}}^N.$$

(28)

The Dunkl convolution product may be defined using the Dunkl translation operator (see [4,34]).

Definition 2. 

The Dunkl convolution product of two functions $f_1,f_2\in D({\mathbb{R}}^N)$ is defined by

$${\displaystyle f_1{\ast }_k{f}_2(y)={\int }_{{\mathbb{R}}^N}{\tau }_y{f}_1(-t)f_2(t)d{\gamma }_k(t),y\in {\mathbb{R}}^N.}$$

(29)

This transformation is commutative, associative, and meets the following properties (see [4,34]).

Proposition 4. 

Let $1\le p,q,r\le \infty ,$ such that $\frac{1}{p}+\frac{1}{q}-\frac{1}{r}=1.$

• If $f\in L_{k,\mathrm{rad}}^p({\mathbb{R}}^N)$ and $g\in L_k^q({\mathbb{R}}^N),$ then $f{\ast }_{k}g$ belongs to $L_k^r({\mathbb{R}}^N)$ and

  $${∥f{\ast }_{k}g∥}_{L_k^r({\mathbb{R}}^N)}\le {∥f∥}_{L_k^p({\mathbb{R}}^N)}{∥g∥}_{L_k^q({\mathbb{R}}^N)}.$$

  (30)
• If $W={\mathbb{Z}}_2^N$, then for all $f\in L_k^p({\mathbb{R}}^N)$ and $g\in L_k^q({\mathbb{R}}^N)$, the function $f{\ast }_{k}g\in L_k^r({\mathbb{R}}^N)$ and

  $${∥f{\ast }_{k}g∥}_{L_k^r({\mathbb{R}}^N)}\le 2^{{\displaystyle \frac{N}{2p}}|p-2|}{∥f∥}_{L_k^p({\mathbb{R}}^N)}{∥g∥}_{L_k^q({\mathbb{R}}^N)}.$$

  (31)
• If $f,g\in L_k^2({\mathbb{R}}^N)$, then $f{\ast }_{k}g\in L_k^2({\mathbb{R}}^N)$ if and only if ${\mathfrak{F}}_k(f){\mathfrak{F}}_k(g)\in L_k^2({\mathbb{R}}^N)$, and in this case, we have

  $${\mathfrak{F}}_k(f{\ast }_{k}g)={\mathfrak{F}}_k(f){\mathfrak{F}}_k(g).$$

  (32)
• If $f,g\in L_k^2({\mathbb{R}}^N)$, then

  $${\displaystyle {\int }_{{\mathbb{R}}^N}|f{\ast }_k{g(x)|}^{2}d{\gamma }_k(x)={\int }_{{\mathbb{R}}^N}|{\mathfrak{F}}_k{(f)(\xi )|}^2{|{\mathfrak{F}}_k(g)(\xi )|}^{2}d{\gamma }_k(\xi ),}$$

  (33)

  where both sides are finite or infinite.

2.2. Linear Canonical Dunkl Transform

In this article, $\mathbf{m}:=\left(\begin{array}{cc}a& b\\ c& d\end{array}\right)$ denotes a matrix in $SL(2,\mathbb{R})$ with $b\ne 0.$ For all $z\in \mathbb{C}\setminus (-\infty ,0]$ and $r>0$, we choose the principal branch of the power function $z^r$ as $z^r=e^{rlnz}$ where $lnz=ln|z|+iargz,-\pi <argz<\pi$.

In this subsection, we recall some results proved in [9,35].

Definition 3. 

The linear canonical Dunkl transform of a function $f\in L_k^1({\mathbb{R}}^N)$ is defined by

$${\displaystyle {\mathfrak{F}}_k^{\mathbf{m}}(f)(x)=\frac{1}{{(ib)}^{\frac{2⟨k⟩+N}{2}}}{\int }_{{\mathbb{R}}^N}{\mathfrak{B}}_k^{\mathbf{m}}(x,y)f(y)d{\gamma }_k(y),}$$

(34)

where

$${\mathfrak{B}}_k^{\mathbf{m}}(x,y)=e^{{\displaystyle \frac{i}{2}}\left({\displaystyle \frac{d}{b}}{\parallel x\parallel }^2+{\displaystyle \frac{a}{b}}{\parallel y\parallel }^2\right)}{e}_k\left(-i{\displaystyle \frac{x}{b}},y\right).$$

(35)

Proposition 5. 

We denote by ${▵}_k^{\mathbf{m}}$ the differential-difference operator given by

$${▵}_k^{\mathbf{m}}:={▵}_k-i{\displaystyle \frac{d}{b}}\left(\sum _{j=1}^N{M}_j{T}_j-T_j{M}_j\right)-{\displaystyle \frac{d^2}{b^2}}{\parallel x\parallel }^2,$$

(36)

where $M_j(u(x)):=x_{j}u(x)$.

Then we have the following relations:

• ${▵}_k^{\mathbf{m}}$ and ${▵}_k$ are connected by

  $$e^{-{\displaystyle \frac{i}{2}}{\displaystyle \frac{d}{b}}{\parallel x\parallel }^2}\circ {▵}_k^{\mathbf{m}}\circ e^{{\displaystyle \frac{i}{2}}{\displaystyle \frac{d}{b}}{\parallel x\parallel }^2}={▵}_k.$$

  (37)
• We have for any $f,g\in \mathcal{S}({\mathbb{R}}^N)$,

  $${\displaystyle {\int }_{{\mathbb{R}}^N}{▵}_k^{\mathbf{m}}f(x)\overline{g(x)}d{\gamma }_k(x)={\int }_{{\mathbb{R}}^N}f(x)\overline{{▵}_k^{\mathbf{m}}g(x)}d{\gamma }_k(x).}$$

  (38)
• The kernel ${\mathfrak{B}}_k^{\mathbf{m}}(·,y)$ satisfies

  $$\left\{\begin{array}{c}{▵}_k^{\mathbf{m}}{\mathfrak{B}}_k^{\mathbf{m}}(·,y)=-{\parallel {\displaystyle \frac{y}{b}}\parallel }^2{\mathfrak{B}}_k^{\mathbf{m}}(·,y),\hfill \\ \\ {\mathfrak{B}}_k^{\mathbf{m}}(0,y)=e^{{\displaystyle \frac{i}{2}}{\displaystyle \frac{a}{b}}{\parallel y\parallel }^2}.\hfill \end{array}\right.$$

  (39)
• For every $f\in \mathcal{S}({\mathbb{R}}^N)$ we have

  $${\mathfrak{F}}_k^{\mathbf{m}}\left({\parallel y\parallel }^{2}f(y)\right)=-b^2{▵}_k^{\mathbf{m}}\left({\mathfrak{F}}_k^{\mathbf{m}}(f)\right),$$

  (40)

  and

  $${\parallel x\parallel }^2{\mathfrak{F}}_k^{\mathbf{m}}(f)=-b^2{\mathfrak{F}}_k^{\mathbf{m}}\left({▵}_k^{{\mathbf{m}}^{-1}}(f)\right).$$

  (41)
• For each $x,y\in {\mathbb{R}}^N$,

  $$|{\mathfrak{B}}_k^{\mathbf{m}}(x,y)|\le 1.$$

  (42)

2.2.1. Particular Cases

• In the case $\mathbf{m}:=\mathbf{m}(\tau )=\left(\begin{array}{cc}1& \tau \\ 0& 1\end{array}\right)$, $\tau \in \mathbb{R}$, ${\mathfrak{F}}_k^{\mathbf{m}}$ coincides with the Fresnel transform associated with the Dunkl transform (see [9]):

  $${\mathcal{W}}_k^{\tau }f(x)=\left\{\begin{array}{cc}{\displaystyle {\displaystyle \frac{1}{{(i\tau )}^{\frac{2⟨k⟩+N}{2}}}}{\int }_{{\mathbb{R}}^N}{E}_k^{\tau }(x,y)f(y)d{\gamma }_k(y),}\hfill & \tau \ne 0,\hfill \\ \\ f(x),\hfill & \tau =0,\hfill \end{array}\right.$$

  where $E_k^{\tau }(x,y)=e^{{\displaystyle \frac{i}{2\tau }}{(\parallel x\parallel }^2+{\parallel y\parallel }^2)}{e}_k\left(-i{\displaystyle \frac{x}{\tau }},y\right).$
• In the case $\mathbf{m}:=\left(\begin{array}{cc}0& -1\\ 1& 0\end{array}\right)$, ${▵}_k^{\mathbf{m}}$ is reduced to the Dunkl–Laplacian operator ${▵}_k$ and ${\mathfrak{F}}_k^{\mathbf{m}}$ coincides with the Dunkl transform ${\mathfrak{F}}_k$ (except for a constant unimodular factor ${(e^{i{\displaystyle \frac{\pi }{2}}})}^{{\displaystyle \frac{2⟨k⟩+N}{2}}}$) (see [9]).
• When $\mathbf{m}:=\mathbf{m}(\theta )=\left(\begin{array}{cc}cosh(\theta )& sinh(\theta )\\ sinh(\theta )& cosh(\theta )\end{array}\right)$, $\theta \in \mathbb{R},{\mathfrak{F}}_k^{\mathbf{m}}$ corresponds to the following integral transform (see [9]):

  $${\mathcal{V}}_k^{\theta }f(x)=\left\{\begin{array}{cc}{\displaystyle {\displaystyle \frac{1}{{(isinh(\theta ))}^{\frac{2⟨k⟩+N}{2}}}}{\int }_{{\mathbb{R}}^N}{R}_k^{\theta }(x,y)f(y)d{\gamma }_k(y),}\hfill & \theta \ne 0,\hfill \\ \\ f(x),\hfill & \theta =0,\hfill \end{array}\right.$$

  where $R_k^{\theta }(x,y)=e^{{\displaystyle \frac{i}{2}}{coth(\theta )(\parallel x\parallel }^2+{\parallel y\parallel }^2)}{e}_k(-i{\displaystyle \frac{x}{sinh(\theta )}},y).$
• In the case $\mathbf{m}:=\mathbf{m}(\theta )=\left(\begin{array}{cc}cos(\theta )& -sin(\theta )\\ sin(\theta )& cos(\theta )\end{array}\right)$, $\theta \in \mathbb{R},{\mathfrak{F}}_k^{\mathbf{m}}$ coincides with the fractional Dunkl transform ${\mathfrak{F}}_k^{\theta }$ (see [9]):

  $${\mathfrak{F}}_k^{\theta }f(x)=\left\{\begin{array}{cc}{\displaystyle {\displaystyle \frac{e^{i((\frac{2⟨k⟩+N}{2}))((\theta -2n\pi )-\widehat{\theta }\pi /2)}}{{|sin(\theta )|}^{\frac{2⟨k⟩+N}{2}}}}{\int }_{{\mathbb{R}}^N}{\mathcal{K}}_k^{\theta }(x,y)f(y)d{\gamma }_k(y),}\hfill & (2j-1)\pi <\theta <(2j+1)\pi ,\hfill \\ f(x),\hfill & \theta =2j\pi ,\hfill \\ f(-x),\hfill & \theta =(2j+1)\pi ,\hfill \end{array}\right.$$

  where $\widehat{\theta }=\mathrm{sgn}(sin(\theta ))$ and

  $${\mathcal{K}}_k^{\theta }(x,y)=e^{-{\displaystyle \frac{i}{2}}{cot(\theta )(\parallel x\parallel }^2+{\parallel y\parallel }^2)}{e}_k\left(-i{\displaystyle \frac{x}{sin(\theta )}},y\right).$$

2.2.2. Linear Canonical Dunkl Transform on $L_k^p({\mathbb{R}}^N)$, $1\le p\le 2$

Definition 4. 

For $s\in \mathbb{R}$, the dilation operator $D_s$ and the chirp multiplication operator ${\mathbf{L}}_s$ are defined by

$${\mathbf{L}}_{s}f(x)=e^{{\displaystyle \frac{is}{2}}{\parallel x\parallel }^2}f(x)\mathrm{and}{D}_{s}f(x)={\displaystyle \frac{1}{{|s|}^{\frac{2⟨k⟩+N}{2}}}}f(x/s),s\ne 0.$$

(43)

Proposition 6. 

The following equalities hold on $L_k^1({\mathbb{R}}^N)$.

• For all $s\in {\mathbb{R}}^{\ast }$, we have

  $$D_s\circ {\mathfrak{F}}_k={\mathfrak{F}}_k\circ D_{{\displaystyle \frac{1}{s}}}.$$

  (44)
• For $\mathbf{m}\in SL(2,\mathbb{R})$, we have

  $$e^{i({\displaystyle \frac{2⟨k⟩+N}{2}}){\displaystyle \frac{\pi }{2}}sgn(b)}{\mathfrak{F}}_k^{\mathbf{m}}={\mathbf{L}}_{{\displaystyle \frac{b}{d}}}\circ D_b\circ {\mathfrak{F}}_k\circ {\mathbf{L}}_{{\displaystyle \frac{a}{b}}}.$$

  (45)

Theorem 1 

(Riemann–Lebesgue’s lemma). For all $f\in L_k^1({\mathbb{R}}^N)$, the linear canonical Dunkl transform ${\mathfrak{F}}_k^{\mathbf{m}}(f)$ belongs to $C_0({\mathbb{R}}^N)$ and satisfies

$$\parallel {\mathfrak{F}}_k^{\mathbf{m}}{(f)\parallel }_{L_k^{\infty }({\mathbb{R}}^N)}\le {|b|}^{-{\displaystyle \frac{2⟨k⟩+N}{2}}}{\parallel f\parallel }_{L_k^1({\mathbb{R}}^N)}.$$

(46)

The following lemma is easily shown using basic computation.

Lemma 1. 

Given $\mathbf{m}:=\left(\begin{array}{cc}a& b\\ c& d\end{array}\right)$$\in SL(2,\mathbb{R})$, then ${\mathbf{m}}^{-1}:=\left(\begin{array}{cc}d& -b\\ -c& a\end{array}\right)\in SL(2,\mathbb{R})$.

Theorem 2 

(Plancherel’s Theorem).

• For every $f,g\in L_k^1({\mathbb{R}}^N)$,

  $${\displaystyle {\int }_{{\mathbb{R}}^N}{\mathfrak{F}}_k^{\mathbf{m}}(f)(x)\overline{g(x)}d{\gamma }_k(x)={\int }_{{\mathbb{R}}^N}f(x)\overline{{\mathfrak{F}}_k^{{\mathbf{m}}^{-1}}(g)(x)}d{\gamma }_k(x).}$$

  (47)
• If $f\in L_k^1({\mathbb{R}}^N)\cap L_k^2({\mathbb{R}}^N)$, then ${\mathfrak{F}}_k^{\mathbf{m}}(f)\in L_k^2({\mathbb{R}}^N)$ and we have

  $$\parallel {\mathfrak{F}}_k^{\mathbf{m}}{(f)\parallel }_{L_k^2({\mathbb{R}}^N)}={\parallel f\parallel }_{L_k^2({\mathbb{R}}^N)}.$$

  (48)
• The linear canonical Dunkl transform has a unique extension to an isometric isomorphism of $L_k^2({\mathbb{R}}^N)$. The extension is also denoted by ${\mathfrak{F}}_k^{\mathbf{m}}(f):L_k^2({\mathbb{R}}^N)\to L_k^2({\mathbb{R}}^N)$.
• For all $f,g\in L_k^2({\mathbb{R}}^N)$, we have

  $${⟨{\mathfrak{F}}_k^{\mathbf{m}}(f),g⟩}_{L_k^2({\mathbb{R}}^N)}={⟨f,{\mathfrak{F}}_k^{{\mathbf{m}}^{-1}}g⟩}_{L_k^2({\mathbb{R}}^N)}.$$

  (49)
• For all $f\in L_k^1({\mathbb{R}}^N)$ with ${\mathfrak{F}}_k^{\mathbf{m}}(f)\in L_k^1({\mathbb{R}}^N),$

  $$\left({\mathfrak{F}}_k^{\mathbf{m}}\circ {\mathfrak{F}}_k^{{\mathbf{m}}^{-1}}\right)(f)=\left({\mathfrak{F}}_k^{{\mathbf{m}}^{-1}}\circ {\mathfrak{F}}_k^{\mathbf{m}}\right)(f)=f,a.e.$$

  (50)

Definition 5. 

Let $\mathbf{m}\in SL(2,\mathbb{R})$ and $1\le p\le 2$. Then the linear canonical Dunkl transform on $L_k^p({\mathbb{R}}^N)$ is defined by

$${\mathfrak{F}}_k^{\mathbf{m}}=e^{-i({\displaystyle \frac{2⟨k⟩+N}{2}}){\displaystyle \frac{\pi }{2}}\mathrm{sgn}(b)}\left({\mathbf{L}}_{{\displaystyle \frac{d}{b}}}\circ D_b\circ {\mathfrak{F}}_k\circ {\mathbf{L}}_{{\displaystyle \frac{a}{b}}}\right),$$

where ${\mathfrak{F}}_k:L_k^p({\mathbb{R}}^N)\to L_k^{p^{\prime }}({\mathbb{R}}^N)$ is the Dunkl transformation on $L_k^p({\mathbb{R}}^N)$.

Theorem 3 

(Young’s inequality). For $1\le p\le 2$ and $\mathbf{m}\in SL(2,\mathbb{R})$, the linear canonical Dunkl transform ${\mathfrak{F}}_k^{\mathbf{m}}$ extends to a bounded linear operator on $L_k^p({\mathbb{R}}^N)$ and we have

$$\parallel {\mathfrak{F}}_k^{\mathbf{m}}(f){\parallel }_{L_k^{p^{\prime }}({\mathbb{R}}^N)}\le {|b|}^{\left({\displaystyle \frac{2⟨k⟩+N}{2}}\right)\left(2/p^{\prime }-1\right)}{\parallel f\parallel }_{L_k^p({\mathbb{R}}^N)}.$$

(51)

Proof. 

From the Theorem 1, we have for all $f\in L_k^1({\mathbb{R}}^N)$

$$\parallel {\mathfrak{F}}_k^{\mathbf{m}}(f){\parallel }_{L_k^{\infty }({\mathbb{R}}^N)}\le {|b|}^{-\left({\displaystyle \frac{2⟨k⟩+N}{2}}\right)}{\parallel f\parallel }_{L_k^p({\mathbb{R}}^N)},$$

(52)

and by (48), we have for every $f\in L_k^2({\mathbb{R}}^N)$

$$\parallel {\mathfrak{F}}_k^{\mathbf{m}}(f){\parallel }_{L_k^2({\mathbb{R}}^N)}={\parallel f\parallel }_{L_k^2({\mathbb{R}}^N)}.$$

(53)

By (52), (53) and the Riesz–Thorin interpolation theorem ${\mathfrak{F}}_k^{\mathbf{m}}$ may be uniquely extended to a linear operator on $L_k^p({\mathbb{R}}^N)$, $1\le p\le \infty$ and we have

$$\parallel {\mathfrak{F}}_k^{\mathbf{m}}(f){\parallel }_{L_k^{p^{\prime }}({\mathbb{R}}^N)}\le {|b|}^{\left({\displaystyle \frac{2⟨k⟩+N}{2}}\right)\left(2/p^{\prime }-1\right)}{\parallel f\parallel }_{L_k^p({\mathbb{R}}^N)}.$$

□

2.3. Generalized Convolution Product Associated with ${\mathfrak{F}}_k^{\mathbf{m}}$

In this subsection, we recall several results that were shown in [35].

Definition 6. 

Let $\mathbf{m}\in SL(2,\mathbb{R})$ such that $b\ne 0.$ For suitable function f, we define the generalized translation operators associated with the operator ${▵}_k^{\mathbf{m}}$ by

$$\begin{array}{c}\hfill T_x^{\mathbf{m},k}f(y)=e^{{\displaystyle \frac{i}{2}}{\displaystyle \frac{d}{b}}{(\parallel x\parallel }^2+{\parallel y\parallel }^2)}{\tau }_x\left[e^{-{\displaystyle \frac{i}{2}}{\displaystyle \frac{d}{b}}{||s||}^2}f(s)\right](y),\end{array}$$

(54)

where ${\tau }_x$ is the Dunkl translation operator.

We will rely on this definition for each function whose Dunkl translation is well defined. Thus from Definition 1, Proposition 2, Remark 2, and Proposition 3, we derive that the generalized translation (54) is well defined on the following spaces:

• $L_k^2({\mathbb{R}}^N)$.
• $L_{k,rad}^p({\mathbb{R}}^N)$, $1\le p\le \infty$.
• $C^{\infty }({\mathbb{R}}^N)$.
• $L_k^p({\mathbb{R}}^N)$, $1\le p\le \infty$ when $W={\mathbb{Z}}_2^N$.

Proposition 7. 

Let $\mathbf{m}\in SL(2,\mathbb{R})$ such that $b\ne 0.$ Then the operators $T_x^{\mathbf{m},k}$, $x\in {\mathbb{R}}^N$ satisfy

• $T_0^{\mathbf{m},k}=Id$ and $T_x^{\mathbf{m},k}f(y)=T_y^{\mathbf{m},k}f(x).$
• For all $x,y,z\in {\mathbb{R}}^N,$ we have the product formula

  $$\begin{array}{c}\hfill \begin{array}{c}{T}_x^{\mathbf{m},k}\left[{\mathfrak{B}}_k^{\mathbf{m}}(.,y)\right](z)=e^{-{\displaystyle \frac{i}{2}}{\displaystyle \frac{a}{b}}{\parallel y\parallel }^2}{\mathfrak{B}}_k^{\mathbf{m}}(x,y){\mathfrak{B}}_k^{\mathbf{m}}(z,y).\hfill \end{array}\end{array}$$

  (55)
• The operator $T_x^{\mathbf{m},k}$ is continuous from $C_{b,rad}({\mathbb{R}}^N)$ into itself, $L_k^2({\mathbb{R}}^N)$ into itself, and on $L_{k,\mathrm{rad}}^p({\mathbb{R}}^N)$. More precisely, if $f\in L_{k,\mathrm{rad}}^p({\mathbb{R}}^N)$ we have

  $$\begin{array}{c}\hfill {∥T_x^{\mathbf{m},k}f∥}_{L_k^p({\mathbb{R}}^N)}\le {∥f∥}_{L_k^p({\mathbb{R}}^N)}.\end{array}$$

  (56)
  Similarly, if $f\in L_k^2({\mathbb{R}}^N)$ we have

  $$\begin{array}{c}\hfill {∥T_x^{\mathbf{m},k}f∥}_{L_k^2({\mathbb{R}}^N)}\le {∥f∥}_{L_k^2({\mathbb{R}}^N)}.\end{array}$$

  (57)
• When $W={\mathbb{Z}}_2^N$, for any $f\in L_k^p({\mathbb{R}}^N)$, we have

  $$\begin{array}{c}\hfill {∥T_x^{\mathbf{m},k}f∥}_{L_k^p({\mathbb{R}}^N)}\le 2^{{\displaystyle \frac{N}{2p}}|p-2|}{∥f∥}_{L_k^p({\mathbb{R}}^N)}.\end{array}$$

  (58)
• For all $f\in L_{k,\mathrm{rad}}^1({\mathbb{R}}^N),$ (resp. $L_k^2({\mathbb{R}}^N)$) we have

  $$\begin{array}{c}\hfill {\mathfrak{F}}_k^{\mathbf{m}}\left[T_x^{{\mathbf{m}}^{-1},k}f\right](\lambda )=e^{{\displaystyle \frac{i}{2}}{\displaystyle \frac{d}{b}}{\parallel \lambda \parallel }^2}\overline{{\mathfrak{B}}_k^{\mathbf{m}}(x,\lambda )}{\mathfrak{F}}_k^{\mathbf{m}}(f)(\lambda ),\end{array}$$

  (59)

  where ${\mathbf{m}}^{-1}$ is the inverse matrix of $\mathbf{m}$ given in Lemma 1.
• For all $f\in L_{k,\mathrm{rad}}^p({\mathbb{R}}^N),p\in (1,2],$ we have

  $$\begin{array}{c}\hfill {\mathfrak{F}}_k^{\mathbf{m}}\left[T_x^{{\mathbf{m}}^{-1},k}f\right](\lambda )=e^{{\displaystyle \frac{i}{2}}{\displaystyle \frac{d}{b}}{\parallel \lambda \parallel }^2}\overline{{\mathfrak{B}}_k^{\mathbf{m}}(x,\lambda )}{\mathfrak{F}}_k^{\mathbf{m}}(f)(\lambda ),a.e.\end{array}$$

  (60)

Corollary 1. 

Let $\mathbf{m}\in SL(2,\mathbb{R})$ such that $b\ne 0$ and $x\in {\mathbb{R}}^N.$ Then for each $f\in \mathcal{S}({\mathbb{R}}^N)$ we have

$$\begin{array}{ccc}\hfill T_x^{{\mathbf{m}}^{-1},k}f(y)& =& {\displaystyle \frac{1}{{(-ib)}^{\frac{2⟨k⟩+N}{2}}}{e}^{-\frac{i}{2}\frac{a}{b}{\parallel y\parallel }^2}{\int }_{{\mathbb{R}}^N}{e}_k(i\lambda /b,y)\overline{{\mathfrak{B}}_k^{\mathbf{m}}(x,\lambda )}{\mathfrak{F}}_k^{\mathbf{m}}(f)(\lambda )d{\gamma }_k(\lambda ),}\hfill \end{array}$$

where ${\mathbf{m}}^{-1}$ is the inverse matrix of $\mathbf{m}$ given in Lemma 1.

Let $\mathbf{m}\in SL(2,\mathbb{R})$ such that $b\ne 0$. The generalized convolution product associated with ${\mathfrak{F}}_k^{\mathbf{m}}$ of two suitable functions f and g on ${\mathbb{R}}^N$, is the function $f{\ast }_{\mathbf{m},k}g$ defined by

$$\begin{array}{c}\hfill {\displaystyle f{\ast }_{\mathbf{m},k}g(x)={\int }_{{\mathbb{R}}^N}{T}_x^{\mathbf{m},k}f(-y)e^{-i\frac{d}{b}{\parallel y\parallel }^2}g(y)d{\gamma }_k(y)}\end{array}$$

(61)

for all x such that the integral exists. The elementary properties of convolutions are summarized in the following proposition.

Proposition 8. 

Assuming that all integrals in question exist, then we have

1.

$f{\ast }_{\mathbf{m},k}g=g{\ast }_{\mathbf{m},k}f.$

2.

$T_x^{\mathbf{m},k}\left(f{\ast }_{\mathbf{m},k}g\right)=\left(T_x^{\mathbf{m},k}f\right){\ast }_{\mathbf{m},k}g=f{\ast }_{\mathbf{m},k}\left(T_x^{\mathbf{m},k}g\right).$

Proposition 9 

(Young’s Inequality). Let $\mathbf{m}\in SL(2,\mathbb{R})$ such that $b\ne 0$ and let $1\le p,q,r\le \infty$ such that $p^{-1}+q^{-1}=r^{-1}+1.$ If $W={\mathbb{Z}}_2^N$, $f\in L_k^p({\mathbb{R}}^N)$ and $g\in L_k^q({\mathbb{R}}^N),$ then $f{\ast }_{\mathbf{m},k}g\in L_k^r({\mathbb{R}}^N)$ and we have

$$\begin{array}{c}\hfill {∥f{\ast }_{\mathbf{m},k}g∥}_{L_k^r({\mathbb{R}}^N)}\le 2^{{\displaystyle \frac{N}{2p}}|p-2|}{∥f∥}_{L_k^p({\mathbb{R}}^N)}{∥g∥}_{L_k^q({\mathbb{R}}^N)}.\end{array}$$

(62)

Moreover, if we assume that $f\in L_{k,rad}^p({\mathbb{R}}^N)$ and $g\in L_k^q({\mathbb{R}}^N),$ then

$$\begin{array}{c}\hfill {∥f{\ast }_{\mathbf{m},k}g∥}_{L_k^r({\mathbb{R}}^N)}\le {∥f∥}_{L_k^p({\mathbb{R}}^N)}{∥g∥}_{L_k^q({\mathbb{R}}^N)}.\end{array}$$

(63)

Proposition 10. 

Let $\mathbf{m}\in SL(2,\mathbb{R})$ such that $b\ne 0.$

• If $f\in L_{k,rad}^1({\mathbb{R}}^N)$ and $g\in L_k^1({\mathbb{R}}^N)$, then for all $x\in {\mathbb{R}}^N$,

  $$\begin{array}{c}\hfill \left(1/{(ib)}^{{\displaystyle \frac{2⟨k⟩+N}{2}}}\right){\mathfrak{F}}_k^{\mathbf{m}}\left(f{\ast }_{{\mathbf{m}}^{-1},k}g\right)(x)=e^{-{\displaystyle \frac{i}{2}}{\displaystyle \frac{d}{b}}{\parallel x\parallel }^2}{\mathfrak{F}}_k^{\mathbf{m}}(f)(x){\mathfrak{F}}_k^{\mathbf{m}}(g)(x),\end{array}$$

  (64)

  where ${\mathbf{m}}^{-1}$ is the inverse matrix of $\mathbf{m}$ given in Lemma 1.
• If $f\in L_{k,\mathrm{rad}}^1({\mathbb{R}}^N)$ and $g\in L_k^p({\mathbb{R}}^N),p\in (1,2]$, then

  $$\begin{array}{c}\hfill \left(1/{(ib)}^{{\displaystyle \frac{2⟨k⟩+N}{2}}}\right){\mathfrak{F}}_k^{\mathbf{m}}\left(f{\ast }_{{\mathbf{m}}^{-1},k}g\right)(x)=e^{-{\displaystyle \frac{i}{2}}{\displaystyle \frac{d}{b}}{\parallel x\parallel }^2}{\mathfrak{F}}_k^{\mathbf{m}}(f)(x){\mathfrak{F}}_k^{\mathbf{m}}(g)(x),a.e.\end{array}$$

  (65)
• If $f,g,h\in L_{k,\mathrm{rad}}^1({\mathbb{R}}^N)$, then we have

  $$\left(f{\ast }_{\mathbf{m},k}g\right){\ast }_{\mathbf{m},k}h=f{\ast }_{\mathbf{m},k}\left(g{\ast }_{\mathbf{m},k}h\right).$$
• If $W={\mathbb{Z}}_2^N$, the previous three results are valid without requiring the functions to be radials.

3. The Linear Canonical Dunkl Windowed Transform

In this section, we introduce the continuous linear canonical Dunkl windowed transform associated with the operator ${▵}_k^{\mathbf{m}}$ and we give some harmonic analysis properties for it. We will denote by $L_{{\mu }_k}^p({\mathbb{R}}^{2N}),p\in [1,\infty ],$ the space of Borel measurable functions $f:{\mathbb{R}}^{2N}\to \mathbb{C}$ such that

$$\begin{array}{ccc}\hfill {\parallel f\parallel }_{L_{{\mu }_k}^p({\mathbb{R}}^{2N})}& :=& {\left({\displaystyle {\int }_{{\mathbb{R}}^{2N}}{|f(x,t)|}^{p}d{\mu }_k(x,t)}\right)}^{{\displaystyle \frac{1}{p}}}<\infty ;1\le p<\infty ,\hfill \\ \hfill {\parallel f\parallel }_{L_{{\mu }_k}^{\infty }({\mathbb{R}}^{2N})}& :=& \begin{array}{c}esssup\\ (x,t)\in {\mathbb{R}}^{2N}\end{array}|f(x,t)|<\infty ,\hfill \end{array}$$

where

$$d{\mu }_k(x,t)=d{\gamma }_k(t)d{\gamma }_k(x).$$

Definition 7. 

Let $\varphi \in L_{k,\mathrm{rad}}^2({\mathbb{R}}^N)$ and $t\in {\mathbb{R}}^N.$ The modulation of the function ϕ by t is defined as follows:

$${\mathrm{M}}_t^{\mathbf{m}}\varphi :={\mathfrak{F}}_k^{{\mathbf{m}}^{-1}}\left(\sqrt{|T_t^{\mathbf{m},k}(e^{{\displaystyle \frac{id}{2b}}{\parallel z\parallel }^2}{|\varphi |}^2)|}\right).$$

(66)

In the following proposition, we state some properties of the modulation ${\mathrm{M}}_t^{\mathbf{m}}\varphi$.

Proposition 11. 

Let $\varphi \in L_{k,\mathrm{rad}}^2({\mathbb{R}}^N)$. Then

• For all $t\in {\mathbb{R}}^N$ we have

  $${\displaystyle {\int }_{{\mathbb{R}}^N}|{\mathfrak{F}}_k^{\mathbf{m}}({\mathrm{M}}_t^{\mathbf{m}}\varphi )(y){|}^{2}d{\gamma }_k(y)={\parallel \varphi \parallel }_{L_k^2({\mathbb{R}}^N)}^2.}$$

  (67)
• For all $y\in {\mathbb{R}}^N$ we have

  $${\displaystyle {\int }_{{\mathbb{R}}^N}|{\mathfrak{F}}_k^{\mathbf{m}}({\mathrm{M}}_t^{\mathbf{m}}\varphi )(y){|}^{2}d{\gamma }_k(t)={\parallel \varphi \parallel }_{L_k^2({\mathbb{R}}^N)}^2.}$$

  (68)

Proof. 

Relation (67) is as an immediate consequence of the relations (48), (54), (27) and (66). On the other hand, from the relations (48), (54), (27), (24), and (66), we have

$${\displaystyle {\int }_{{\mathbb{R}}^N}|{\mathfrak{F}}_k^{\mathbf{m}}({\mathrm{M}}_t^{\mathbf{m}}\varphi )(y){|}^{2}d{\gamma }_k(t)={\int }_{{\mathbb{R}}^N}{\tau }_t{(|\varphi |}^2)(y)d{\gamma }_k(t)={\int }_{{\mathbb{R}}^N}{\tau }_y{(|\varphi |}^2)(t)d{\gamma }_k(t)={\parallel \varphi \parallel }_{L_k^2({\mathbb{R}}^N)}^2.}$$

□

Definition 8. 

Let $\varphi \in L_{k,\mathrm{rad}}^2({\mathbb{R}}^N)$. We consider the family ${\varphi }_{x,t}^{\mathbf{m}}$, $x,t\in {\mathbb{R}}^N$, defined by

$$\forall \xi \in {\mathbb{R}}^N,{\varphi }_{x,t}^{\mathbf{m}}(\xi ):=\overline{T_x^{{\mathbf{m}}^{-1},k}({\mathrm{M}}_t^{\mathbf{m}}\varphi )(-\xi )}.$$

(69)

For any function $f\in L_k^2({\mathbb{R}}^N)$, we define its linear canonical Dunkl windowed transform (LCDWT) by

$${\displaystyle \forall x,t\in {\mathbb{R}}^N,{\mathfrak{W}}_{\varphi }^{\mathbf{m}}(f)(x,t)={\int }_{{\mathbb{R}}^N}f(\xi )\overline{{\varphi }_{x,t}^{\mathbf{m}}(\xi )}d{\gamma }_k(\xi ).}$$

(70)

Remark 3. 

• We have for all $x,t\in {\mathbb{R}}^N$,

  $$\parallel {\varphi }_{x,t}^{\mathbf{m}}{\parallel }_{L_k^2({\mathbb{R}}^N)}\le {\parallel \varphi \parallel }_{L_k^2({\mathbb{R}}^N)}.$$

  (71)
• The linear canonical Dunkl windowed transform can be written as

  $$\forall (x,t)\in {\mathbb{R}}^{2N},{\mathfrak{W}}_{\varphi }^{\mathbf{m}}(f)(x,t)=L_{{\displaystyle \frac{-2a}{b}}}f{\ast }_{{\mathbf{m}}^{-1},k}{\mathrm{M}}_t^{\mathbf{m}}\varphi (x).$$

  (72)

Using basic computation, we prove the following result.

Lemma 2. 

If $\varphi \in L_k^{\infty }({\mathbb{R}}^N)\cap L_{k,\mathrm{rad}}^2({\mathbb{R}}^N)$, then for every $f\in L_k^2({\mathbb{R}}^N)$,

$${\mathfrak{F}}_k^{\mathbf{m}}\left({\mathfrak{W}}_{\varphi }^{\mathbf{m}}(f)(·,t)\right)(y)={(ib)}^{{\displaystyle \frac{2⟨k⟩+N}{2}}}{e}^{-i{\displaystyle \frac{d}{2b}}{\parallel \xi \parallel }^2}{\mathfrak{F}}_k^{\mathbf{m}}(L_{{\displaystyle \frac{-2a}{b}}}f)(y)\sqrt{{\tau }_t{|\varphi |}^2(y)}.$$

(73)

Proposition 12. 

Let $\varphi \in L_{k,\mathrm{rad}}^2({\mathbb{R}}^N)$. Then for every $f\in L_k^2({\mathbb{R}}^N),{\mathfrak{W}}_{\varphi }^{\mathbf{m}}(f)$ belongs to $L_{{\mu }_k}^{\infty }({\mathbb{R}}^{2N})$ and we have

$$\parallel {\mathfrak{W}}_{\varphi }^{\mathbf{m}}{(f)\parallel }_{L_{{\mu }_k}^{\infty }({\mathbb{R}}^{2N})}\le {\parallel f\parallel }_{L_k^2({\mathbb{R}}^N)}{\parallel \varphi \parallel }_{L_k^2({\mathbb{R}}^N)}.$$

(74)

Proof. 

Let $\varphi \in L_{k,\mathrm{rad}}^2({\mathbb{R}}^N)$. Using (70), Cauchy–Schwartz’s inequality and Relation (71), we have for every $f\in L_k^2({\mathbb{R}}^N)$

$$\begin{array}{ccc}|{\mathfrak{W}}_{\varphi }^{\mathbf{m}}(f)(x,t)|\hfill & \le \hfill & {\parallel f\parallel }_{L_k^2({\mathbb{R}}^N)}{\parallel {\varphi }_{x,t}^{\mathbf{m}}\parallel }_{L_k^2({\mathbb{R}}^N)}\hfill \\ \hfill & \le \hfill & {\parallel f\parallel }_{L_k^2({\mathbb{R}}^N)}{\parallel \varphi \parallel }_{L_k^2({\mathbb{R}}^N)}.\hfill \end{array}$$

Thus, ${\mathfrak{W}}_{\varphi }^{\mathbf{m}}(f)$ belongs to $L_{{\mu }_k}^{\infty }({\mathbb{R}}^{2N})$ and we have

$$\parallel {\mathfrak{W}}_{\varphi }^{\mathbf{m}}{(f)\parallel }_{L_{{\mu }_k}^{\infty }({\mathbb{R}}^{2N})}\le {\parallel f\parallel }_{L_k^2({\mathbb{R}}^N)}{\parallel \varphi \parallel }_{L_k^2({\mathbb{R}}^N)}.$$

□

Theorem 4 

(Inversion-type Relation). Let ϕ be a function in $L_{k,rad}^2({\mathbb{R}}^N)$ such that ${\parallel \varphi \parallel }_{L_k^2({\mathbb{R}}^N)}=1$. Then, for any function f in $L_k^1({\mathbb{R}}^N)\cap L_k^2({\mathbb{R}}^N)$ such that ${\mathfrak{F}}_k^{\mathbf{m}}(f)$ belongs to $L_k^1({\mathbb{R}}^N),$ we have, for almost everywhere,

$$\begin{array}{c}\hfill {\displaystyle f(y)={\int }_{{\mathbb{R}}^N}{\int }_{{\mathbb{R}}^N}{e}^{-i\frac{d}{2b}{||\lambda ||}^2}{\mathfrak{F}}_k^{\mathbf{m}}\left({\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_{\frac{2a}{b}}f)(.,t)\right)(\lambda ){\mathfrak{F}}_k^{\mathbf{m}}(T_y^{{\mathbf{m}}^{-1},k}{\mathrm{M}}_t^{\mathbf{m}}\varphi )(\lambda )\frac{d{\mu }_k(\lambda ,t)}{{(ib)}^{\frac{2⟨k⟩+N}{2}}}.}\end{array}$$

(75)

To prove this theorem, we need the following lemma.

Lemma 3 

(${\mathit{L}}_{\mathit{k}}^{\mathbf{2}}$ inversion formula). Let ϕ be as above. Then, for any function f in $L_k^1({\mathbb{R}}^N)\cap L_k^2({\mathbb{R}}^N),$ we have

$$\begin{array}{c}\hfill {\displaystyle f(y)={(ib)}^{-\frac{2⟨k⟩+N}{2}}\underset{j\to \infty }{lim}{\int }_{B_N(0,j)}{\int }_{{\mathbb{R}}^N}{e}^{-i\frac{d}{2b}{||\lambda ||}^2}{\mathfrak{F}}_k^{\mathbf{m}}\left({\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_{\frac{2a}{b}}f)(.,t)\right)(\lambda ){\mathfrak{F}}_k^{\mathbf{m}}(T_y^{{\mathbf{m}}^{-1},k}{\mathrm{M}}_t^{\mathbf{m}}\varphi )(\lambda )d{\mu }_k(\lambda ,t)}\end{array}$$

(76)

where $B_N(0,j)$ is the ball of center 0 and radius $j\in \mathbb{N}$ and the limit is in $L_k^2({\mathbb{R}}^N).$

Proof. 

As f is in $L_k^1({\mathbb{R}}^N)\cap L_k^2({\mathbb{R}}^N)$, then from (72), we have

$${\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_{{\displaystyle \frac{2a}{b}}}f)(x,t)=f{\ast }_{{\mathbf{m}}^{-1},k}{\mathrm{M}}_t^{\mathbf{m}}\varphi (x).$$

Thus, using Proposition 4, (34), (35), and (65), we obtain

$$\begin{array}{c}\hfill {\mathfrak{F}}_k^{\mathbf{m}}({\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_{{\displaystyle \frac{2a}{b}}}f)(.,t))(\lambda )={(ib)}^{{\displaystyle \frac{2⟨k⟩+N}{2}}}{e}^{-i{\displaystyle \frac{d}{2b}}{||\lambda ||}^2}{\mathfrak{F}}_k^{\mathbf{m}}(f)(\lambda )\sqrt{{\tau }_t{|\varphi |}^2(\lambda )},\lambda \in {\mathbb{R}}^N.\end{array}$$

(77)

Using the hypothesis on $\varphi$ and by a standard analysis, we obtain

$$\begin{array}{c}\hfill {\displaystyle {\mathfrak{F}}_k^{\mathbf{m}}(f)(\lambda )={(ib)}^{-\frac{2⟨k⟩+N}{2}}{\int }_{{\mathbb{R}}^N}{e}^{i\frac{d}{2b}{||\lambda ||}^2}{\mathfrak{F}}_k^{\mathbf{m}}({\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_{\frac{2a}{b}}f)(.,t))(\lambda ){\mathfrak{F}}_k^{\mathbf{m}}({\mathrm{M}}_t^{\mathbf{m}}\varphi )(\lambda )d{\gamma }_k(t).}\end{array}$$

(78)

Thus, using this relation and the relation (59), we obtain

$$\begin{array}{c}\hfill {\displaystyle {\int }_{B_N(0,j)}\left({\int }_{{\mathbb{R}}^N}{\mathfrak{F}}_k^{\mathbf{m}}({\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_{\frac{2a}{b}}f)(.,t))(\lambda ){\mathfrak{F}}_k^{\mathbf{m}}(T_y^{{\mathbf{m}}^{-1},k}{\mathrm{M}}_t^{\mathbf{m}}\varphi (\lambda )d{\gamma }_k(t)\right)\frac{d{\gamma }_k(\lambda )}{{(ib)}^{\frac{2⟨k⟩+N}{2}}}=}\\ \hfill {\displaystyle {\int }_{B_N(0,j)}{\mathfrak{F}}_k^{\mathbf{m}}(f)(\lambda ){\mathfrak{B}}_k^{{\mathbf{m}}^{-1}}(y,\lambda )d{\gamma }_k(\lambda ).}\end{array}$$

(79)

Moreover, since f is in $L_k^2({\mathbb{R}}^N),$ we have

$${\displaystyle f(y)=\underset{j\to \infty }{lim}{\int }_{B_N(0,j)}{\mathfrak{F}}_k^{\mathbf{m}}(f)(\lambda ){\mathfrak{B}}_k^{{\mathbf{m}}^{-1}}(y,\lambda )d{\gamma }_k(\lambda ),y\in {\mathbb{R}}^N}$$

(80)

the limit is in $L_k^2({\mathbb{R}}^N).$ Thus, by this relation and (79), we derive that for $y\in {\mathbb{R}}^N,$

$${\displaystyle f(y)=\underset{j\to \infty }{lim}{\int }_{B_N(0,j)}\left({\int }_{{\mathbb{R}}^N}{\mathfrak{F}}_k^{\mathbf{m}}\left({\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_{\frac{2a}{b}}f)(.,t)\right)(\lambda ){\mathfrak{F}}_k^{\mathbf{m}}(T_y^{{\mathbf{m}}^{-1},k}{\mathrm{M}}_t^{\mathbf{m}}\varphi )(\lambda )d{\gamma }_k(t)\right)\frac{d{\gamma }_k(\lambda )}{{(ib)}^{\frac{2⟨k⟩+N}{2}}}.}$$

(81)

The limit is in $L_k^2({\mathbb{R}}^N).$ □

Proof of Theorem 4. 

Using (81), we derive that for almost every $y\in {\mathbb{R}}^N$

$$\begin{array}{c}\hfill {\displaystyle f(y)=\underset{j\to \infty }{lim}{\int }_{B_N(0,j)}\left({\int }_{{\mathbb{R}}^N}{\mathfrak{F}}_k^{\mathbf{m}}\left({\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_{\frac{2a}{b}}f)(.,t)\right)(\lambda ){\mathfrak{F}}_k^{\mathbf{m}}(T_y^{{\mathbf{m}}^{-1},k}{\mathrm{M}}_t^{\mathbf{m}}\varphi )(\lambda )d{\gamma }_k(t)\right)\frac{d{\gamma }_k(\lambda )}{{(ib)}^{\frac{2⟨k⟩+N}{2}}}.}\end{array}$$

By standard analysis, we obtain for almost every $y\in {\mathbb{R}}^N$

$$\begin{array}{c}\hfill {\displaystyle f(y)=\underset{j\to \infty }{lim}{\int }_{{\mathbb{R}}^N}\left({\int }_{B_N(0,j)}{\mathfrak{F}}_k^{\mathbf{m}}\left({\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_{\frac{2a}{b}}f)(.,t)\right)(\lambda ){\mathfrak{F}}_k^{\mathbf{m}}(T_y^{{\mathbf{m}}^{-1},k}{\mathrm{M}}_t^{\mathbf{m}}\varphi )(\lambda )d{\gamma }_k(\lambda )\right)\frac{d{\gamma }_k(t)}{{(ib)}^{\frac{2⟨k⟩+N}{2}}}.}\end{array}$$

(82)

We consider for $y\in {\mathbb{R}}^N$, the sequence ${\mathfrak{U}}_j$ given by

$${\displaystyle {\mathfrak{U}}_j(t)={\int }_{B_N(0,j)}{\mathfrak{F}}_k^{\mathbf{m}}\left({\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_{\frac{2a}{b}}f)(.,t)\right)(\lambda ){\mathfrak{F}}_k^{\mathbf{m}}(T_y^{{\mathbf{m}}^{-1},k}{\mathrm{M}}_t^{\mathbf{m}}\varphi )(\lambda )d{\gamma }_k(\lambda ).}$$

This sequence satisfies the following:

$${\displaystyle \forall t\in {\mathbb{R}}^N,\underset{j\to \infty }{lim}{\mathfrak{U}}_j(t)={\int }_{{\mathbb{R}}^N}{\mathfrak{F}}_k^{\mathbf{m}}\left({\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_{\frac{2a}{b}}f)(.,t)\right)(\lambda ){\mathfrak{F}}_k^{\mathbf{m}}(T_y^{{\mathbf{m}}^{-1},k}{\mathrm{M}}_t^{\mathbf{m}}\varphi )(\lambda )d{\gamma }_k(\lambda ).}$$

On the other hand, for all $t\in {\mathbb{R}}^N,$ we have

$${\displaystyle |{\mathfrak{U}}_j(t)|\le {\int }_{{\mathbb{R}}^N}|{\mathfrak{F}}_k^{\mathbf{m}}\left({\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_{\frac{2a}{b}}f)(.,t)\right)(\lambda ){\mathfrak{F}}_k^{\mathbf{m}}(T_y^{{\mathbf{m}}^{-1},k}{\mathrm{M}}_t^{\mathbf{m}}\varphi )(\lambda )|d{\gamma }_k(\lambda ).}$$

By standard analysis, we prove that the function

$${\displaystyle t\mapsto {\int }_{{\mathbb{R}}^N}|{\mathfrak{F}}_k^{\mathbf{m}}\left({\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_{\frac{2a}{b}}f)(.,t)\right)(\lambda ){\mathfrak{F}}_k^{\mathbf{m}}(T_y^{{\mathbf{m}}^{-1},k}{\mathrm{M}}_t^{\mathbf{m}}\varphi )(\lambda )|d{\gamma }_k(\lambda )}$$

is integrable on ${\mathbb{R}}^N$ with respect to the measure $d{\gamma }_k(t).$ Then, by applying the dominated convergence theorem to the relation (82), we obtain, for almost every $y\in {\mathbb{R}}^N$,

$${\displaystyle f(y)={(ib)}^{-\frac{2⟨k⟩+N}{2}}{\int }_{{\mathbb{R}}^N}{\int }_{{\mathbb{R}}^N}{\mathfrak{F}}_k^{\mathbf{m}}\left({\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_{\frac{2a}{b}}f)(.,t)\right)(\lambda ){\mathfrak{F}}_k^{\mathbf{m}}(T_y^{{\mathbf{m}}^{-1},k}{\mathrm{M}}_t^{\mathbf{m}}\varphi )(\lambda )d{\gamma }_k(\lambda )d{\gamma }_k(t).}$$

Thus, the theorem is proved. □

Proposition 13. 

If $\varphi \in L_{k,\mathrm{rad}}^2({\mathbb{R}}^N)$, then for every $f,g\in L_k^2({\mathbb{R}}^N)$,

$${\displaystyle {\int }_{{\mathbb{R}}^{2N}}{\mathfrak{W}}_{\varphi }^{\mathbf{m}}(f)(x,t)\overline{{\mathfrak{W}}_{\varphi }^{\mathbf{m}}(g)(x,t)}d{\mu }_k(x,t)={|b|}^{2⟨k⟩+N}{\parallel \varphi \parallel }_{L_k^2({\mathbb{R}}^N)}^2{\int }_{{\mathbb{R}}^N}f(x)\overline{g(x)}d{\gamma }_k(x).}$$

(83)

Proof. 

Using the relations (72), (49), and (65), we have

$$\begin{array}{ccc}\hfill {\displaystyle {\int }_{{\mathbb{R}}^{2N}}{\mathfrak{W}}_{\varphi }^{\mathbf{m}}(f)(x,t)\overline{{\mathfrak{W}}_{\varphi }^{\mathbf{m}}(g)(x,t)}d{\mu }_k(x,t)}& =& {\displaystyle {\int }_{{\mathbb{R}}^N}{\int }_{{\mathbb{R}}^N}{L}_{\frac{-2a}{b}}f{\ast }_{{\mathbf{m}}^{-1},k}{\mathrm{M}}_t^{\mathbf{m}}\varphi (x)\overline{L_{\frac{-2a}{b}}g{\ast }_{{\mathbf{m}}^{-1},k}{\mathrm{M}}_t^{\mathbf{m}}\varphi (x)}d{\gamma }_k(x)d{\gamma }_k(t)}\hfill \\ & =& {\displaystyle {\int }_{{\mathbb{R}}^N}{\int }_{{\mathbb{R}}^N}{\mathfrak{F}}_k^{\mathbf{m}}\left(L_{\frac{-2a}{b}}f{\ast }_{{\mathbf{m}}^{-1},k}{\mathrm{M}}_t^{\mathbf{m}}\varphi \right)(\lambda )}\hfill \\ & & \overline{{\mathfrak{F}}_k^{\mathbf{m}}\left(L_{{\displaystyle \frac{-2a}{b}}}g{\ast }_{{\mathbf{m}}^{-1},k}{\mathrm{M}}_t^{\mathbf{m}}\varphi \right)}(\lambda )d{\gamma }_k(\lambda )d{\gamma }_k(t)\hfill \\ & =& {\displaystyle {|b|}^{2⟨k⟩+N}{\int }_{{\mathbb{R}}^N}{\int }_{{\mathbb{R}}^N}{\mathfrak{F}}_k^{\mathbf{m}}(L_{\frac{-2a}{b}}f)(\lambda )\overline{{\mathfrak{F}}_k^{\mathbf{m}}(L_{\frac{-2a}{b}}g)(\lambda )}}\hfill \\ & & |{\mathfrak{F}}_k^{\mathbf{m}}({\mathrm{M}}_t^{\mathbf{m}}\varphi )(\lambda ){|}^{2}d{\gamma }_k(\lambda )d{\gamma }_k(t)\hfill \\ & =& {\displaystyle {|b|}^{2⟨k⟩+N}{\parallel \varphi \parallel }_{L_k^2({\mathbb{R}}^N)}^2{\int }_{{\mathbb{R}}^N}{\mathfrak{F}}_k^{\mathbf{m}}(L_{\frac{-2a}{b}}f)(\lambda )\overline{{\mathfrak{F}}_k^{\mathbf{m}}(L_{\frac{-2a}{b}}g)(\lambda )}d{\gamma }_k(\lambda )}\hfill \\ & =& {\displaystyle {|b|}^{2⟨k⟩+N}{\parallel \varphi \parallel }_{L_k^2({\mathbb{R}}^N)}^2{\int }_{{\mathbb{R}}^N}f(x)\overline{g(x)}d{\gamma }_k(x).}\hfill \end{array}$$

□

Corollary 2 

(Plancherel’s formula). Let $\varphi \in L_{k,\mathrm{rad}}^2({\mathbb{R}}^N)$. If $f,g\in L_k^2({\mathbb{R}}^N)$, then ${\mathfrak{W}}_{\varphi }^{\mathbf{m}}(f)\in L_{{\mu }_k}^2({\mathbb{R}}^{2N})$ and we have

$$\parallel {\mathfrak{W}}_{\varphi }^{\mathbf{m}}{(f)\parallel }_{L_{{\mu }_k}^2({\mathbb{R}}^{2N})}={|b|}^{{\displaystyle \frac{2⟨k⟩+N}{2}}}{\parallel \varphi \parallel }_{L_k^2({\mathbb{R}}^N)}{\parallel f\parallel }_{L_k^2({\mathbb{R}}^N)}.$$

(84)

Proposition 14. 

Let $\varphi \in L_{k,\mathrm{rad}}^2({\mathbb{R}}^N)$ and $2\le p<\infty$. Then for all $f\in L_k^2({\mathbb{R}}^N)$

$${\displaystyle {\int }_{{\mathbb{R}}^{2N}}|{\mathfrak{W}}_{\varphi }^{\mathbf{m}}{(f)(x,t)|}^{p}d{\mu }_k(x,t)\le {|b|}^{2⟨k⟩+N}{\parallel \varphi \parallel }_{L_k^2({\mathbb{R}}^N)}^p{\parallel f\parallel }_{L_k^2({\mathbb{R}}^N)}^p.}$$

(85)

Proof. 

We have

$${\displaystyle {\int }_{{\mathbb{R}}^{2N}}|{\mathfrak{W}}_{\varphi }^{\mathbf{m}}{(f)(x,t)|}^{p}d{\mu }_k(x,t)\le \parallel {\mathfrak{W}}_{\varphi }^{\mathbf{m}}{(f)\parallel }_{L_{{\mu }_k}^2({\mathbb{R}}^{2N})}^2{\parallel {\mathfrak{W}}_{\varphi }^{\mathbf{m}}(f)\parallel }_{L_{{\mu }_k}^{\infty }({\mathbb{R}}^{2N})}^{p-2}.}$$

Using Proposition 12 and Corollary 2, we obtain the desired result. □

Proposition 15. 

Let ϕ be in $L_{k,rad}^2({\mathbb{R}}^N)\cap L_k^{\infty }({\mathbb{R}}^N)$. Then, ${\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_k^2({\mathbb{R}}^N))$ is a reproducing kernel Hilbert space in $L_k^2({\mathbb{R}}^N)$ with kernel function

$${\displaystyle {\mathcal{K}}_{\varphi }^{\mathbf{m}}(x^{\prime },t^{\prime };x,t):=\frac{1}{{|b|}^{2⟨k⟩+N}{\parallel \varphi \parallel }_{L_k^2({\mathbb{R}}^N)}^2}{\int }_{\mathbb{R}}{\varphi }_{x^{\prime },t^{\prime }}^{\mathbf{m}}(y)\overline{{\varphi }_{x,t}^{\mathbf{m}}(y)}d{\gamma }_k(y).}$$

(86)

The kernel is pointwise bounded:

$$|{\mathcal{K}}_{\varphi }^{\mathbf{m}}(t^{\prime },x^{\prime };t,x)|\le {\displaystyle \frac{1}{{|b|}^{2⟨k⟩+N}}};\forall (x^{\prime },t^{\prime }),(x,t)\in {\mathbb{R}}^{2N}.$$

(87)

Proof. 

Let $f\in L_k^2({\mathbb{R}}^N)$. We have

$${\displaystyle {\mathfrak{W}}_{\varphi }^{\mathbf{m}}(f)(x,t)={\int }_{\mathbb{R}}f(y)\overline{{\varphi }_{x,t}^{\mathbf{m}}(y)}d{\gamma }_k(y),(x,t)\in {\mathbb{R}}^{2N}.}$$

Using Parseval’s relation (83), we obtain

$${\displaystyle {\mathfrak{W}}_{\varphi }^{\mathbf{m}}(f)(x,t)=\frac{1}{{|b|}^{2⟨k⟩+N}{\parallel \varphi \parallel }_{L_k^2({\mathbb{R}}^N)}^2}{\int }_{{\mathbb{R}}^{2N}}{\mathfrak{W}}_{\varphi }^{\mathbf{m}}(f)(x^{\prime },t^{\prime })\overline{{\mathfrak{W}}_{\varphi }^{\mathbf{m}}({\varphi }_{x,t}^{\mathbf{m}})(x^{\prime },t^{\prime })}d{\mu }_k(x^{\prime },t^{\prime }).}$$

On the other hand, using Proposition 4, one can easily see that for every $(x,t),(x^{\prime },t^{\prime })\in {\mathbb{R}}^{2N}$, the function

$${\displaystyle x^{\prime }\mapsto \frac{1}{{|b|}^{2⟨k⟩+N}{\parallel \varphi \parallel }_{L_k^2({\mathbb{R}}^N)}^2}{\mathfrak{W}}_{\varphi }^{\mathbf{m}}({\varphi }_{x,t}^{\mathbf{m}})(x^{\prime },t^{\prime })=\frac{1}{{|b|}^{2⟨k⟩+N}{\parallel h\parallel }_{L_k^2({\mathbb{R}}^N)}^2}{\int }_{\mathbb{R}}{\varphi }_{x^{\prime },t^{\prime }}^{\mathbf{m}}(y)\overline{{\varphi }_{x,t}^{\mathbf{m}}(y)}d{\gamma }_k(y)}$$

belongs to $L_k^2({\mathbb{R}}^N)$. Therefore, the result is obtained. □

4. Localization Operators for the Linear Canonical Dunkl Windowed Transform

Let ${(s_j(A))}_{j\in \mathbb{N}}$ be the singular values of the compact operator $T\in B(L_k^2({\mathbb{R}}^N))$, which are eigenvalues of the non-negative self-adjoint operator $|T|=\sqrt{T^{\ast }T}$.

All compact operators with singular values in ${\ell }^p(\mathbb{N})$ are called the Schatten class $S_p$, $1\le p<\infty$.

These spaces are equipped with the norm

$${\parallel T\parallel }_{S_p}:={\left(\sum _{i=1}^{\infty }{(s_i(T))}^p\right)}^{{\displaystyle \frac{1}{p}}}.$$

(88)

Moreover, let $S_{\infty }:=B(L_k^2({\mathbb{R}}^N))$, the set equipped with the norm,

$${\parallel T\parallel }_{S_{\infty }}:=\underset{u\in L_k^2({\mathbb{R}}^N):{\parallel u\parallel }_{L_k^2({\mathbb{R}}^N)}=1}{sup}{\parallel Tu\parallel }_{L_k^2({\mathbb{R}}^N)}.$$

(89)

Definition 9. 

The trace of an operator A in $S_1$ is defined by

$$tr(T)=\sum _{j=1}^{\infty }{⟨T{u}_j,u_j⟩}_{L_k^2({\mathbb{R}}^N)}$$

(90)

where ${(u_j)}_j$ is any orthonormal basis of $L_k^2({\mathbb{R}}^N)$.

Remark 4. 

If T is positive, then

$$tr(T)={\parallel T\parallel }_{S_1}.$$

(91)

Moreover, a compact operator T on the Hilbert space $L_k^2({\mathbb{R}}^N)$ is Hilbert–Schmidt if the positive operator $T^{\ast }T$ is in the space of trace class $S_1$. Then,

$${\parallel T\parallel }_{HS}^2:={\parallel T\parallel }_{S_2}^2=\parallel T^{\ast }{T\parallel }_{S_1}=tr(T^{\ast }T)=\sum _{j=1}^{\infty }{\parallel T{u}_j\parallel }_{L_k^2({\mathbb{R}}^N)}^2$$

(92)

for every orthonormal basis $(u_j)$ of $L_k^2({\mathbb{R}}^N)$.

Definition 10. 

Let ς be Borel measurable function on ${\mathbb{R}}^{2N}$. For $u,v$ two Borel measurable radial functions on ${\mathbb{R}}^N$, we define the two-wavelet localization operator related to the linear canonical Dunkl windowed transform on $L_k^p({\mathbb{R}}^N)$, $1\le p\le \infty$, by

$${\displaystyle {\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )(f)(y)={\int }_{{\mathbb{R}}^{2N}}\varsigma (x,t){\mathfrak{W}}_u^{\mathbf{m}}(f)(x,t)v_{x,t}^{\mathbf{m}}(y)d{\mu }_k(x,t),y\in {\mathbb{R}}^N.}$$

(93)

In accordance with the different choices of the symbols $\varsigma$ and the different continuities required, we need to impose different conditions on u and v, and then we obtain an operator on $L_k^p({\mathbb{R}}^N)$.

It is often more convenient to interpret the definition of ${\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )$ in a weak sense, that is, for f in $L_k^p({\mathbb{R}}^N)$, $p\in [1,\infty ]$, and g in $L_k^{p^{\prime }}({\mathbb{R}}^N)$,

$$\begin{array}{c}\hfill {\displaystyle {⟨{\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )(f),g⟩}_{L_k^2({\mathbb{R}}^N)}={\int }_{{\mathbb{R}}^{2N}}\varsigma (x,t){\mathfrak{W}}_u^{\mathbf{m}}(f)(x,t)\overline{{\mathfrak{W}}_v^{\mathbf{m}}(g)(x,t)}d{\mu }_k(x,t).}\end{array}$$

(94)

By straightforward calculus, one has the following result:

Proposition 16. 

The adjoint of the linear operator ${\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma ):L_k^p({\mathbb{R}}^N)\to L_k^p({\mathbb{R}}^N)$, $p\in [1,\infty )$, is ${\mathfrak{L}}_{v,u}^{\mathbf{m}}(\overline{\varsigma }):L_k^{p^{\prime }}({\mathbb{R}}^N)\to L_k^{p^{\prime }}({\mathbb{R}}^N)$. Moreover, we have

$${({\mathfrak{L}}_{u,v}^{\mathbf{m}})}^{\ast }(\varsigma )={\mathfrak{L}}_{v,u}^{\mathbf{m}}(\overline{\varsigma }).$$

(95)

For simplicity’s sake, we will refer to ${\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )$ as localization operators.

4.1. Boundedness of ${\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )$ on $S_{\infty }$

Our goal here is to show that

$${\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma ):L_k^2({\mathbb{R}}^N)\to L_k^2({\mathbb{R}}^N)$$

is a bounded operator for each symbol $\varsigma \in L_{{\mu }_k}^p({\mathbb{R}}^{2N})$, $1\le p\le \infty$. We first consider this problem for $\varsigma$ in $L_{{\mu }_k}^1({\mathbb{R}}^{2N})$ and next in $L_{{\mu }_k}^{\infty }({\mathbb{R}}^{2N})$ and we then conclude by using interpolation theory.

In this section, u and v will be two radial functions on ${\mathbb{R}}^N$ such that

$${\parallel u\parallel }_{L_k^2({\mathbb{R}}^N)}={\parallel v\parallel }_{L_k^2({\mathbb{R}}^N)}=1.$$

Proposition 17. 

Let $\varsigma \in L_{{\mu }_k}^1({\mathbb{R}}^{2N})$; then the localization operator ${\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )$ is in $S_{\infty }$ and we have

$$\parallel {\mathfrak{L}}_{u,v}^{\mathbf{m}}{(\varsigma )\parallel }_{S_{\infty }}⩽{\parallel \varsigma \parallel }_{L_{{\mu }_k}^1({\mathbb{R}}^{2N})}.$$

(96)

Proof. 

For every function f and g in $L_k^2({\mathbb{R}}^N)$, we have from the relations (94) and (74),

$$\begin{array}{cc}\hfill |{⟨{\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )(f),g⟩}_{L_k^2({\mathbb{R}}^N)}|& {\displaystyle ⩽{\int }_{{\mathbb{R}}^{2N}}|\varsigma (x,t)\parallel {\mathfrak{W}}_u^{\mathbf{m}}(f)(x,t)\parallel \overline{{\mathfrak{W}}_v^{\mathbf{m}}(g)(x,t)}|d{\mu }_k(x,t)}\hfill \\ & ⩽\parallel {\mathfrak{W}}_u^{\mathbf{m}}{(f)\parallel }_{L_{{\mu }_k}^{\infty }({\mathbb{R}}^{2N})}\parallel {\mathfrak{W}}_v^{\mathbf{m}}{(g)\parallel }_{L_{{\mu }_k}^{\infty }({\mathbb{R}}^{2N})}{\parallel \varsigma \parallel }_{L_{{\mu }_k}^1({\mathbb{R}}^{2N})}\hfill \\ & ⩽{\parallel f\parallel }_{L_k^2({\mathbb{R}}^N)}{\parallel g\parallel }_{L_k^2({\mathbb{R}}^N)}{\parallel \varsigma \parallel }_{L_{{\mu }_k}^1({\mathbb{R}}^{2N})}.\hfill \end{array}$$

Thus,

$$\parallel {\mathfrak{L}}_{u,v}^{\mathbf{m}}{(\varsigma )\parallel }_{S_{\infty }}⩽{\parallel \varsigma \parallel }_{L_{{\mu }_k}^1({\mathbb{R}}^{2N})}.$$

□

Proposition 18. 

Let $\varsigma \in L_{{\mu }_k}^{\infty }({\mathbb{R}}^{2N})$, then the localization operator ${\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )$ is in $S_{\infty }$ and we have

$$\parallel {\mathfrak{L}}_{u,v}^{\mathbf{m}}{(\varsigma )\parallel }_{S_{\infty }}⩽{|b|}^{2⟨k⟩+N}{\parallel \varsigma \parallel }_{L_{{\mu }_k}^{\infty }({\mathbb{R}}^{2N})}.$$

Proof. 

For all functions f and g in $L_k^2({\mathbb{R}}^N)$, we have, from Hölder’s inequality,

$$\begin{array}{cc}\hfill |{⟨{\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )(f),g⟩}_{L_k^2({\mathbb{R}}^N)}|& {\displaystyle ⩽{\int }_{{\mathbb{R}}^{2N}}|\varsigma (x,t)\parallel {\mathfrak{W}}_u^{\mathbf{m}}(f)(x,t)\parallel \overline{{\mathfrak{W}}_v^{\mathbf{m}}(g)(x,t)}|d{\mu }_k(x,t)}\hfill \\ & ⩽{\parallel \varsigma \parallel }_{L_{{\mu }_k}^{\infty }({\mathbb{R}}^{2N})}\parallel {\mathfrak{W}}_u^{\mathbf{m}}{(f)\parallel }_{L_{{\mu }_k}^2({\mathbb{R}}^{2N})}{\parallel {\mathfrak{W}}_v^{\mathbf{m}}(g)\parallel }_{L_{{\mu }_k}^2({\mathbb{R}}^{2N})}.\hfill \end{array}$$

Using Plancherel’s formula for ${\mathfrak{W}}_u^{\mathbf{m}}$ and ${\mathfrak{W}}_v^{\mathbf{m}}$, given by the relation (84), we obtain

$$\begin{array}{c}\hfill |{⟨{\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )(f),g⟩}_{L_k^2({\mathbb{R}}^N)}{|⩽|b|}^{2⟨k⟩+N}{\parallel f\parallel }_{L_k^2({\mathbb{R}}^N)}{\parallel g\parallel }_{L_k^2({\mathbb{R}}^N)}{\parallel \varsigma \parallel }_{L_{{\mu }_k}^{\infty }({\mathbb{R}}^{2N})}.\end{array}$$

Thus,

$$\parallel {\mathfrak{L}}_{u,v}^{\mathbf{m}}{(\varsigma )\parallel }_{S_{\infty }}⩽{|b|}^{2⟨k⟩+N}{\parallel \varsigma \parallel }_{L_{{\mu }_k}^{\infty }({\mathbb{R}}^{2N})}.$$

As desired. □

Consequently, we obtain the following result.

Theorem 5. 

Let ς be in $L_{{\mu }_k}^p({\mathbb{R}}^{2N})$, $1\le p\le \infty$. Then, there exists a unique bounded linear operator

$${\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma ):L_k^2({\mathbb{R}}^N)\to L_k^2({\mathbb{R}}^N),$$

such that

$$\parallel {\mathfrak{L}}_{u,v}^{\mathbf{m}}{(\varsigma )\parallel }_{S_{\infty }}⩽{|b|}^{{\displaystyle \frac{2⟨k⟩+N}{p^{\prime }}}}{\parallel \varsigma \parallel }_{L_{{\mu }_k}^p({\mathbb{R}}^{2N})}.$$

Proof. 

Let f be in $L_k^2({\mathbb{R}}^N)$. We consider the following operator

$$\begin{array}{ccc}{\mathcal{T}}_{\mathbf{m}}:L_{{\mu }_k}^1({\mathbb{R}}^{2N})\cap L_{{\mu }_k}^{\infty }({\mathbb{R}}^{2N})\hfill & \to \hfill & L_k^2({\mathbb{R}}^N),\hfill \end{array}$$

given by

$${\mathcal{T}}_{\mathbf{m}}(\varsigma ):={\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )(f).$$

Thus, by the Propositions 17 and 18,

$$\parallel {\mathcal{T}}_{\mathbf{m}}{(\varsigma )\parallel }_{L_k^2({\mathbb{R}}^N)}⩽{\parallel f\parallel }_{L_k^2({\mathbb{R}}^N)}{\parallel \varsigma \parallel }_{L_{{\mu }_k}^1({\mathbb{R}}^{2N})}$$

(97)

and

$$\parallel {\mathcal{T}}_{\mathbf{m}}{(\varsigma )\parallel }_{L_k^2({\mathbb{R}}^N)}\le {|b|}^{2⟨k⟩+N}{\parallel f\parallel }_{L_k^2({\mathbb{R}}^N)}{\parallel \varsigma \parallel }_{L_{{\mu }_k}^{\infty }({\mathbb{R}}^{2N})}.$$

(98)

Hence, using (97), (98) and the interpolation theorem ([36], Theorem 2.11) ${\mathcal{T}}_{\mathbf{m}}$ may be uniquely extended to a linear operator on $L_{{\mu }_k}^p({\mathbb{R}}^{2N})$ such that

$$\parallel {\mathfrak{L}}_{u,v}^{\mathbf{m}}{(\varsigma )(f)\parallel }_{L_k^2({\mathbb{R}}^N)}=\parallel {\mathcal{T}}_{\mathbf{m}}{(\varsigma )\parallel }_{L_k^2({\mathbb{R}}^N)}\le {|b|}^{{\displaystyle \frac{2⟨k⟩+N}{p^{\prime }}}}{\parallel f\parallel }_{L_k^2({\mathbb{R}}^N)}{\parallel \varsigma \parallel }_{L_{{\mu }_k}^p({\mathbb{R}}^{2N})}.$$

(99)

As (99) is true for arbitrary functions f in $L_k^2({\mathbb{R}}^N)$, then we obtain the desired result. □

4.2. Schatten–von Neumann Properties for ${\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )$

The main result of this subsection is to prove that the localization operators ${\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma ):L_k^2({\mathbb{R}}^N)\to L_k^2({\mathbb{R}}^N)$ are in $S_p$.

Proposition 19. 

If $\varsigma \in L_{{\mu }_k}^1({\mathbb{R}}^{2N})$, then ${\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )$ is in the Hilbert–Schmidt class $S_2$, such that

$$\parallel {\mathfrak{L}}_{u,v}^{\mathbf{m}}{(\varsigma )\parallel }_{S_2}⩽{\parallel \varsigma \parallel }_{L_{{\mu }_k}^1({\mathbb{R}}^{2N})}.$$

(100)

Proof. 

Let $\{h_j,j=1,2\dots \}$ be an orthonormal basis for $L_k^2({\mathbb{R}}^N)$. Then, by (94), Fubini’s theorem, Parseval’s identity, and the relations (70) and (95), we have

$$\begin{array}{ccc}{\displaystyle \sum _{j=1}^{\infty }{\parallel {\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )(h_j)\parallel }_{L_k^2({\mathbb{R}}^N)}^2}\hfill & =\hfill & {\displaystyle \sum _{j=1}^{\infty }{⟨{\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )(h_j),{\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )(h_j)⟩}_{L_k^2({\mathbb{R}}^N)}}\hfill \\ & =\hfill & {\displaystyle \sum _{j=1}^{\infty }{\int }_{{\mathbb{R}}^{2N}}\varsigma (x,t){⟨h_j,u_{x,t}^{\mathbf{m}}⟩}_{L_k^2({\mathbb{R}}^N)}{\overline{⟨{\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )(h_j),v_{x,t}^{\mathbf{m}}⟩}}_{L_k^2({\mathbb{R}}^N)}d{\mu }_k(x,t)}\hfill \\ & =\hfill & {\displaystyle {\int }_{{\mathbb{R}}^{2N}}\varsigma (x,t)\sum _{j=1}^{\infty }{⟨h_j,u_{x,t}^{\mathbf{m}}⟩}_{L_k^2({\mathbb{R}}^N)}{⟨{({\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma ))}^{\ast }{v}_{x,t}^{\mathbf{m}},h_j⟩}_{L_k^2({\mathbb{R}}^N)}d{\mu }_k(x,t)}\hfill \\ & =\hfill & {\displaystyle {\int }_{{\mathbb{R}}^{2N}}\varsigma (x,t){⟨{({\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma ))}^{\ast }{v}_{x,t}^{\mathbf{m}},u_{x,t}^{\mathbf{m}}⟩}_{L_k^2({\mathbb{R}}^N)}d{\mu }_k(x,t).}\hfill \end{array}$$

Thus, from (95), (96), and (71), we obtain

$${\displaystyle \sum _{j=1}^{\infty }\parallel {\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )(h_j){\parallel }_{L_k^2({\mathbb{R}}^N)}^2\le {\int }_{{\mathbb{R}}^{2N}}|\varsigma (x,t)|\parallel {({\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma ))}^{\ast }{\parallel }_{S_{\infty }}d{\mu }_k(x,t)\le {\parallel \varsigma \parallel }_{L_{{\mu }_k}^1({\mathbb{R}}^{2N})}^2<\infty .}$$

(101)

Therefore, by (101) and Proposition 2.8 in the book [36], by Wong,

$${\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma ):L_k^2({\mathbb{R}}^N)\to L_k^2({\mathbb{R}}^N)$$

is in the Hilbert–Schmidt class $S_2$ and is, hence, compact. □

Proposition 20. 

Let $\varsigma \in L_{{\mu }_k}^p({\mathbb{R}}^{2N}),$$1⩽p<\infty$. Then the localization operator ${\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )$ is compact.

Proof. 

Let ${({\varsigma }_j)}_{j\in \mathbb{N}}$ be a sequence in $L_{{\mu }_k}^1({\mathbb{R}}^{2N})\cap L_{{\mu }_k}^{\infty }({\mathbb{R}}^{2N})$ such that ${\varsigma }_j\to \varsigma$ in $L_{{\mu }_k}^p({\mathbb{R}}^{2N})$ as $j\to \infty$. Then, by Theorem 5,

$$\parallel {\mathfrak{L}}_{u,v}^{\mathbf{m}}({\varsigma }_j)-{\mathfrak{L}}_{u,v}^{\mathbf{m}}{(\varsigma )\parallel }_{S_{\infty }}\le {|b|}^{{\displaystyle \frac{2⟨k⟩+N}{p^{\prime }}}}{\parallel {\varsigma }_j-\varsigma \parallel }_{L_{{\mu }_k}^p({\mathbb{R}}^{2N})}.$$

Thus, in $S_{\infty }$, ${\mathfrak{L}}_{u,v}^{\mathbf{m}}({\varsigma }_j)\to {\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )$ as $j\to \infty$. Moreover, by Proposition 19, the operator ${\mathfrak{L}}_{u,v}^{\mathbf{m}}({\varsigma }_j)$ is in $S_2$; hence, it is compact. Consequently, ${\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )$ is compact. □

Theorem 6. 

If ς is in $L_{{\mu }_k}^1({\mathbb{R}}^{2N})$, then ${\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma ):L_k^2({\mathbb{R}}^N)\to L_k^2({\mathbb{R}}^N)$ is in $S_1$ and we have

$$\parallel \tilde{\varsigma }{\parallel }_{L_{{\mu }_k}^1({\mathbb{R}}^{2N})}⩽\parallel {\mathfrak{L}}_{u,v}^{\mathbf{m}}{(\varsigma )\parallel }_{S_1}⩽{\parallel \varsigma \parallel }_{L_{{\mu }_k}^1({\mathbb{R}}^{2N})},$$

(102)

where $\tilde{\varsigma }$ is given by

$$\forall (x,t)\in {\mathbb{R}}^{2N},\tilde{\varsigma }(x,t)={⟨{\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )u_{x,t}^{\mathbf{m}},v_{x,t}^{\mathbf{m}}⟩}_{L_k^2({\mathbb{R}}^N)}.$$

Proof. 

As $\varsigma$ is in $L_{{\mu }_k}^1({\mathbb{R}}^{2N})$, then by Proposition 19, ${\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )\in S_2$. Thus by the canonical form of compact operators (see [36], [Theorem 2.2]), there exists an orthonormal basis $\{h_j,j=1,2,\dots \}$ for the orthogonal complement of the kernel of the operator ${\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )$, consisting of eigenvectors of $|{\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )|$ and $\{{\phi }_j,j=1,2,\dots \}$, an orthonormal set in $L_k^2({\mathbb{R}}^N)$, such that

$$\begin{array}{c}\hfill {\displaystyle {\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )(f)=\sum _{j=1}^{\infty }{s}_j{⟨f,h_j⟩}_{L_k^2({\mathbb{R}}^N)}{\phi }_j,}\end{array}$$

(103)

where $s_j,j=1,2,\dots$ are the positive singular values of ${\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )$ corresponding to $h_j$. Then, we obtain

$$\begin{array}{c}\hfill {\displaystyle \parallel {\mathfrak{L}}_{u,v}^{\mathbf{m}}{(\varsigma )\parallel }_{S_1}=\sum _{j=1}^{\infty }{s}_j=\sum _{j=1}^{\infty }{⟨{\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )(h_j),{\phi }_j⟩}_{L_k^2({\mathbb{R}}^N)}.}\end{array}$$

Thus, by Fubini’s theorem, Cauchy–Schwarz’s inequality, Bessel inequality, and relations (71) and (70), we obtain

$$\begin{array}{ccc}\parallel {\mathfrak{L}}_{u,v}^{\mathbf{m}}{(\varsigma )\parallel }_{S_1}\hfill & =\hfill & {\displaystyle \sum _{j=1}^{\infty }{⟨{\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )(h_j),{\phi }_j⟩}_{L_k^2({\mathbb{R}}^N)}}\hfill \\ & =\hfill & {\displaystyle \sum _{j=1}^{\infty }{\int }_{{\mathbb{R}}^{2N}}\varsigma (x,t){\mathfrak{W}}_u^{\mathbf{m}}(h_j)(x,t)\overline{{\mathfrak{W}}_v^{\mathbf{m}}({\phi }_j)(x,t)}d{\mu }_k(x,t)}\hfill \\ & ⩽\hfill & {\displaystyle {\int }_{{\mathbb{R}}^{2N}}|\varsigma (x,t)|{\left({\displaystyle \sum _{j=1}^{\infty }{|{\mathfrak{W}}_u^{\mathbf{m}}(h_j)(x,t)|}^2}\right)}^{\frac{1}{2}}{\left({\displaystyle \sum _{j=1}^{\infty }{|{\mathfrak{W}}_v^{\mathbf{m}}({\phi }_j)(x,t)|}^2}\right)}^{\frac{1}{2}}d{\mu }_k(x,t)}\hfill \\ & ⩽\hfill & {\displaystyle {\int }_{{\mathbb{R}}^{2N}}|\varsigma (x,t)|\parallel u_{x,t}^{\mathbf{m}}{\parallel }_{L_k^2({\mathbb{R}}^N)}{\parallel v_{x,t}^{\mathbf{m}}\parallel }_{L_k^2({\mathbb{R}}^N)}d{\mu }_k(x,t)}\hfill \\ & ⩽\hfill & {\parallel \varsigma \parallel }_{L_{{\mu }_k}^1({\mathbb{R}}^{2N})}.\hfill \end{array}$$

Thus,

$$\parallel {\mathfrak{L}}_{u,v}^{\mathbf{m}}{(\varsigma )\parallel }_{S_1}⩽{\parallel \varsigma \parallel }_{L_{{\mu }_k}^1({\mathbb{R}}^{2N})}.$$

On the other hand, it is clear that $\tilde{\varsigma }$ belongs to $L_k^1({\mathbb{R}}^N)$, and by (103), we obtain

$$\begin{array}{cc}\hfill |\tilde{\varsigma }(x,t)|& =|{⟨{\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )(u_{x,t}^{\mathbf{m}}),v_{x,t}^{\mathbf{m}}⟩}_{L_k^2({\mathbb{R}}^N)}|\hfill \\ \hfill & {\displaystyle =|\sum _{j=1}^{\infty }{s}_j{⟨u_{x,t}^{\mathbf{m}},h_j⟩}_{L_k^2({\mathbb{R}}^N)}{⟨h_j,v_{x,t}^{\mathbf{m}}⟩}_{L_k^2({\mathbb{R}}^N)}|}\hfill \\ \hfill & {\displaystyle ⩽\frac{1}{2}\sum _{j=1}^{\infty }{s}_j\left(|{⟨u_{x,t}^{\mathbf{m}},h_j⟩}_{L_k^2({\mathbb{R}}^N)}{|}^2+|{⟨v_{x,t}^{\mathbf{m}},h_j⟩}_{L_k^2({\mathbb{R}}^N)}{|}^2\right).}\hfill \end{array}$$

Then from Fubini’s theorem, we obtain

$$\begin{array}{cc}\hfill {\displaystyle {\int }_{{\mathbb{R}}^{2N}}|\tilde{\varsigma }(x,t)|d{\mu }_k(x,t)}& {\displaystyle \le \frac{1}{2}\sum _{j=1}^{\infty }{s}_j({\int }_{{\mathbb{R}}^{2N}}{|{⟨u_{x,t}^{\mathbf{m}},h_j⟩}_{L_k^2({\mathbb{R}}^N)}|}^{2}d{\mu }_k(x,t)}\hfill \\ & {\displaystyle +{\int }_{{\mathbb{R}}^{2N}}{|{⟨v_{x,t}^{\mathbf{m}},h_j⟩}_{L_k^2({\mathbb{R}}^N)}|}^{2}d{\mu }_k(x,t)).}\hfill \end{array}$$

Thus, using Plancherel’s formula for ${\mathfrak{W}}_u^{\mathbf{m}}$, ${\mathfrak{W}}_v^{\mathbf{m}}$, we obtain

$${\displaystyle {\int }_{{\mathbb{R}}^{2N}}|\tilde{\varsigma }(x,t)|d{\mu }_k(x,t)\le \sum _{j=1}^{\infty }{s}_j={\parallel {\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )\parallel }_{S_1}.}$$

□

Corollary 3. 

The trace formula for ς in $L_{{\mu }_k}^1({\mathbb{R}}^{2N})$ is as follows:

$${\displaystyle tr({\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma ))={\int }_{{\mathbb{R}}^{2N}}\varsigma (x,t){⟨v_{x,t}^{\mathbf{m}},u_{x,t}^{\mathbf{m}}⟩}_{L_k^2({\mathbb{R}}^N)}d{\mu }_k(x,t).}$$

(104)

Proof. 

Let ${\left\{h_j\right\}}_{j\in \mathbb{N}}$ be an orthonormal basis for $L_k^2({\mathbb{R}}^N)$. Then by Theorem 6, ${\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )$ is in $S_1$. Therefore, by Relation (90) and Parseval’s identity,

$$\begin{array}{ccc}tr({\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma ))\hfill & =\hfill & {\displaystyle \sum _{j=1}^{\infty }{⟨{\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma )(h_j),h_j⟩}_{L_k^2({\mathbb{R}}^N)}}\hfill \\ & =\hfill & {\displaystyle \sum _{j=1}^{\infty }{\int }_{{\mathbb{R}}^{2N}}\varsigma (x,t){⟨h_j,u_{x,t}^{\mathbf{m}}⟩}_{L_k^2({\mathbb{R}}^N)}{\overline{⟨h_j,v_{x,t}^{\mathbf{m}}⟩}}_{L_k^2({\mathbb{R}}^N)}d{\mu }_k(x,t)}\hfill \\ & =\hfill & {\displaystyle {\int }_{{\mathbb{R}}^{2N}}\varsigma (x,t)\sum _{j=1}^{\infty }{⟨h_j,u_{x,t}^{\mathbf{m}}⟩}_{L_k^2({\mathbb{R}}^N)}{\overline{⟨h_j,v_{x,t}^{\mathbf{m}}⟩}}_{L_k^2({\mathbb{R}}^N)}d{\mu }_k(x,t)}\hfill \\ & =\hfill & {\displaystyle {\int }_{{\mathbb{R}}^{2N}}\varsigma (x,t){⟨v_{x,t}^{\mathbf{m}},u_{x,t}^{\mathbf{m}}⟩}_{L_k^2({\mathbb{R}}^N)}d{\mu }_k(x,t).}\hfill \end{array}$$

□

The main result of this subsection is shown in the following corollary.

Corollary 4. 

If $\varsigma \in L_{{\mu }_k}^p({\mathbb{R}}^{2N})$, $1⩽p⩽\infty$, then ${\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma ):L_k^2({\mathbb{R}}^N)⟶L_k^2({\mathbb{R}}^N)$ is in $S_p$ and we have

$$\parallel {\mathfrak{L}}_{u,v}^{\mathbf{m}}{(\varsigma )\parallel }_{S_p}⩽{|b|}^{{\displaystyle \frac{2⟨k⟩+N}{p^{\prime }}}}{\parallel \varsigma \parallel }_{L_{{\mu }_k}^p({\mathbb{R}}^{2N})}.$$

Proof. 

This is the consequence of interpolation theorems (see [36], [Theorems 2.10 and 2.11]), Proposition 18, and Theorem 6. □

Notice that, in the case that $u=v$ and $\varsigma \in L_{{\mu }_k}^1({\mathbb{R}}^{2N})$ is real valued and positive, we derive that ${\mathfrak{L}}_{u,v}^{\mathbf{m}}(\varsigma ):L_k^2({\mathbb{R}}^N)\to L_k^2({\mathbb{R}}^N)$ is non-negative. Then, by Corollary 3 and Relation (91), we have

$${\displaystyle \parallel {\mathfrak{L}}_{u,v}^{\mathbf{m}}{(\varsigma )\parallel }_{S_1}={\int }_{{\mathbb{R}}^{2N}}\varsigma (x,t){\parallel u_{x,t}^{\mathbf{m}}\parallel }_{L_k^2({\mathbb{R}}^N)}^{2}d{\mu }_k(x,t).}$$

(105)

Corollary 5. 

Let $u,v\in L_k^2({\mathbb{R}}^N)$ such that $u={v,\parallel u\parallel }_{L_k^2({\mathbb{R}}^N)}=1$, and let ${\varsigma }_1,{\varsigma }_2\in L_{{\mu }_k}^1({\mathbb{R}}^{2N})$ be any real-valued and positive function. Then, ${\mathfrak{L}}_{u,v}^{\mathbf{m}}({\varsigma }_1)$, ${\mathfrak{L}}_{u,v}^{\mathbf{m}}({\varsigma }_2)$ are positive trace class operators with

$$\begin{array}{ccc}\parallel {\left({\mathfrak{L}}_{u,v}^{\mathbf{m}}({\varsigma }_1){\mathfrak{L}}_{u,v}^{\mathbf{m}}({\varsigma }_2)\right)}^j{\parallel }_{S_1}\hfill & =\hfill & tr{\left({\mathfrak{L}}_{u,v}^{\mathbf{m}}({\varsigma }_1){\mathfrak{L}}_{u,v}^{\mathbf{m}}({\varsigma }_2)\right)}^j\hfill \\ & \le \hfill & {\left(tr\left({\mathfrak{L}}_{u,v}^{\mathbf{m}}({\varsigma }_1)\right)\right)}^j{\left(tr\left({\mathfrak{L}}_{u,v}^{\mathbf{m}}({\varsigma }_2)\right)\right)}^j\hfill \\ & =\hfill & \parallel {\mathfrak{L}}_{u,v}^{\mathbf{m}}({\varsigma }_1){\parallel }_{S_1}^j\parallel {\mathfrak{L}}_{u,v}^{\mathbf{m}}({\varsigma }_2){\parallel }_{S_1}^j,\hfill \end{array}$$

for every $j\in \mathbb{N}$.

Proof. 

Let $L_1={\mathfrak{L}}_{u,v}^{\mathbf{m}}({\varsigma }_1)$ and $L_2={\mathfrak{L}}_{u,v}^{\mathbf{m}}({\varsigma }_2)$. Then, by [37], [Theorem 1], we have for every $j\in \mathbb{N}$,

$$tr{(L_1{L}_2)}^j\le {\left(tr(L_1)\right)}^j{\left(tr(L_2)\right)}^j.$$

□

5. Spectral Analysis for the Generalized Concentration Operator

Here, we will assume that the window function $\varphi \in L_{k,rad}^2({\mathbb{R}}^N)\cap L_k^{\infty }({\mathbb{R}}^N)$ such that

$${\parallel \varphi \parallel }_{L_k^2({\mathbb{R}}^N)}={\displaystyle \frac{1}{{|b|}^{\frac{2⟨k⟩+N}{2}}}}.$$

We denote by

• ${({\mathfrak{W}}_{\varphi }^{\mathbf{m}})}^{\ast }:L_{{\mu }_k}^2({\mathbb{R}}^{2N})\to L_k^2({\mathbb{R}}^N)$ the adjoint of ${\mathfrak{W}}_{\varphi }^{\mathbf{m}}$ given by

  $${⟨{\mathfrak{W}}_{\varphi }^{\mathbf{m}}(f_1),f_2⟩}_{L_{{\mu }_k}^2({\mathbb{R}}^{2N})}={⟨f_1,{({\mathfrak{W}}_{\varphi }^{\mathbf{m}})}^{\ast }(f_2)⟩}_{L_k^2({\mathbb{R}}^N)},f_1\in L_k^2({\mathbb{R}}^N),f_2\in L_{{\mu }_k}^2({\mathbb{R}}^{2N}).$$
• $P_{\varphi }^{\mathbf{m}}:L_{{\mu }_k}^2({\mathbb{R}}^{2N})\to L_{{\mu }_k}^2({\mathbb{R}}^{2N})$ the orthogonal projection from $L_{{\mu }_k}^2({\mathbb{R}}^{2N})$ onto ${\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_k^2({\mathbb{R}}^N))$ defined by

  $$P_{\varphi }^{\mathbf{m}}:={\mathfrak{W}}_{\varphi }^{\mathbf{m}}{({\mathfrak{W}}_{\varphi }^{\mathbf{m}})}^{\ast }.$$

  (106)
• $P_U:L_{{\mu }_k}^2({\mathbb{R}}^{2N})\to L_{{\mu }_k}^2({\mathbb{R}}^{2N})$ the orthogonal projection from $L_{{\mu }_k}^2({\mathbb{R}}^{2N})$ onto the subspace of functions whose supports are in the subset $U\subset {\mathbb{R}}^{2N}$, that is,

  $$P_{U}F={\chi }_{U}F,F\in L_{{\mu }_k}^2({\mathbb{R}}^{2N}),$$

  where ${\chi }_U$ denotes the characteristic function of the subset U of ${\mathbb{R}}^{2N}$.

Our main focus in this section is on the concentration operator ${\mathcal{L}}_{\varphi }^{\mathbf{m}}(U)$, which can be written as

$${\mathcal{L}}_{\varphi }^{\mathbf{m}}(U):={\mathfrak{L}}_{\varphi ,\varphi }^{\mathbf{m}}(\varsigma ),$$

where $\varsigma ={\chi }_U$, and U is a subset of ${\mathbb{R}}^{2N}$, such that $0<{\mu }_k(U)<\infty$.

5.1. The Range of the LCDWT

As $P_{\varphi }^{\mathbf{m}}={\mathfrak{W}}_{\varphi }^{\mathbf{m}}{({\mathfrak{W}}_{\varphi }^{\mathbf{m}})}^{\ast }$, then $P_{\varphi }^{\mathbf{m}}$ is the integral operator such that

$${\displaystyle P_{\varphi }^{\mathbf{m}}F(z)={\int }_{{\mathbb{R}}^{2N}}F(x,t){\mathcal{K}}_{\varphi }^{\mathbf{m}}(z;x,t)d{\mu }_k(x,t),z=(x^{\prime },t^{\prime })\in {\mathbb{R}}^{2N},}$$

with integral kernel ${\mathcal{K}}_{\varphi }^{\mathbf{m}}$ given by (86).

As ${\mathcal{K}}_{\varphi }^{\mathbf{m}}$ is the integral kernel of an orthogonal projection, it satisfies

$${\mathcal{K}}_{\varphi }^{\mathbf{m}}(z;z^{\prime })=\overline{{\mathcal{K}}_{\varphi }^{\mathbf{m}}(z^{\prime };z)},z,z^{\prime }\in {\mathbb{R}}^{2N},$$

and

$${\displaystyle {\mathcal{K}}_{\varphi }^{\mathbf{m}}(z;z^{\prime })={\int }_{{\mathbb{R}}^{2N}}{\mathcal{K}}_{\varphi }^{\mathbf{m}}(z;z^{″}){\mathcal{K}}_{\varphi }^{\mathbf{m}}(z^{″};z^{\prime })d{\mu }_k(z^{″}),z,z^{\prime }\in {\mathbb{R}}^{2N}.}$$

(107)

If $\{v_j:j\in \mathbb{N}\}$ is an orthonormal basis of ${\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_k^2({\mathbb{R}}^N))$, ${\mathcal{K}}_{\varphi }^{\mathbf{m}}$ can be expanded as

$${\mathcal{K}}_{\varphi }^{\mathbf{m}}(z;z^{\prime })=\sum _{j=1}^{\infty }{v}_j(z)\overline{v_j(z^{\prime })},z,z^{\prime }\in {\mathbb{R}}^{2N}.$$

(108)

Definition 11. 

We define the spectrogram of f with respect to ϕ by

$${\mathbf{S}}_{\varphi ,\mathbf{m}}^D(f)(x,t)={|{\mathfrak{W}}_{\varphi }^{\mathbf{m}}(f)(x,t)|}^2,(x,t)\in {\mathbb{R}}^{2N}.$$

(109)

By (84), we have

$${\displaystyle {\int }_{{\mathbb{R}}^{2N}}{\mathbf{S}}_{\varphi ,\mathbf{m}}^D(f)(x,t)d{\mu }_k(x,t)={\parallel f\parallel }_{L_k^2({\mathbb{R}}^N)}^2,}$$

(110)

and by (94),

$${\displaystyle {⟨{\mathcal{L}}_{\varphi }^{\mathbf{m}}(U)f,f⟩}_{L_k^2({\mathbb{R}}^N)}={\int }_{{\mathbb{R}}^{2N}}\varsigma (x,t){\mathbf{S}}_{\varphi ,\mathbf{m}}^D(f)(x,t)d{\mu }_k(x,t).}$$

(111)

Definition 12. 

We define the deformed Calderón–Toeplitz operator

$${\mathcal{T}}_{U,\varphi ,\mathbf{m}}:{\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_k^2({\mathbb{R}}^N))\to {\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_k^2({\mathbb{R}}^N))$$

by

$${\mathcal{T}}_{U,\varphi ,\mathbf{m}}F=P_{\varphi }^{\mathbf{m}}{P}_{U}F.$$

(112)

Proposition 21. 

The operator ${\mathcal{T}}_{U,\varphi ,\mathbf{m}}:{\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_k^2({\mathbb{R}}^N))\to {\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_k^2({\mathbb{R}}^N))$ is trace class that fulfills

$$0\le {\mathcal{T}}_{U,\varphi ,\mathbf{m}}\le P_U\le I,$$

(113)

and

$${\mathcal{T}}_{U,\varphi ,\mathbf{m}}={\mathfrak{W}}_{\varphi }^{\mathbf{m}}{\mathcal{L}}_{\varphi }^{\mathbf{m}}(U){({\mathfrak{W}}_{\varphi }^{\mathbf{m}})}^{\ast }.$$

(114)

Proof. 

For all $F\in {\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_k^2({\mathbb{R}}^N)),$

$${\displaystyle {⟨{\mathcal{T}}_{U,\varphi ,\mathbf{m}}F,F⟩}_{L_{{\mu }_k}^2({\mathbb{R}}^{2N})}={⟨P_{\varphi }^{\mathbf{m}}(P_{U}F),F⟩}_{L_{{\mu }_k}^2({\mathbb{R}}^{2N})}={⟨P_{U}F,F⟩}_{L_{{\mu }_k}^2({\mathbb{R}}^{2N})}={\int }_U{|F(x,t)|}^{2}d{\mu }_k(x,t).}$$

(115)

Then we have (113), and the operator ${\mathcal{T}}_{U,\varphi ,\mathbf{m}}$ is bounded and non-negative.

On the other hand, the operator ${\mathcal{L}}_{\varphi }^{\mathbf{m}}(U):L_k^2({\mathbb{R}}^N)\to L_k^2({\mathbb{R}}^N)$ can be written as

$${\mathcal{L}}_{\varphi }^{\mathbf{m}}(U)(f)={({\mathfrak{W}}_{\varphi }^{\mathbf{m}})}^{\ast }(P_U{\mathfrak{W}}_{\varphi }^{\mathbf{m}}f),f\in L_k^2({\mathbb{R}}^N).$$

Therefore,

$$\left({\mathfrak{W}}_{\varphi }^{\mathbf{m}}{\mathcal{L}}_{\varphi }^{\mathbf{m}}(U){({\mathfrak{W}}_{\varphi }^{\mathbf{m}})}^{\ast }\right)F=P_{\varphi }^{\mathbf{m}}{P}_{U}F={\mathcal{T}}_{U,\varphi ,\mathbf{m}}F,F\in {\mathfrak{W}}_{\varphi }^{\mathbf{m}}\left(L_k^2({\mathbb{R}}^N)\right).$$

(116)

Consequently, the deformed Calderón–Toeplitz operator ${\mathcal{T}}_{U,\varphi ,\mathbf{m}}$ and the time–frequency operator ${\mathcal{L}}_{\varphi }^{\mathbf{m}}(U)$ are connected by

$${\mathcal{T}}_{U,\varphi ,\mathbf{m}}={\mathfrak{W}}_{\varphi }^{\mathbf{m}}{\mathcal{L}}_{\varphi }^{\mathbf{m}}(U){({\mathfrak{W}}_{\varphi }^{\mathbf{m}})}^{\ast }.$$

□

Based on the last proposition, we may conclude that ${\mathcal{T}}_{U,\varphi ,\mathbf{m}}$ and ${\mathcal{L}}_{\varphi }^{\mathbf{m}}(U)$ share the same spectral properties. Specifically, this leads to the following theorem.

Theorem 7. 

The deformed Calderón–Toeplitz operator ${\mathcal{T}}_{U,\varphi ,\mathbf{m}}$ is compact and even trace class with

$$tr({\mathcal{T}}_{U,\varphi ,\mathbf{m}})=tr({\mathcal{L}}_{\varphi }^{\mathbf{m}}(U))={\mathcal{M}}_k(U,\varphi ,\mathbf{m}),$$

(117)

where

$${\displaystyle {\mathcal{M}}_k(U,\varphi ,\mathbf{m}):={\int }_U{\parallel {\varphi }_{x,t}^{\mathbf{m}}\parallel }_{L_k^2({\mathbb{R}}^N)}^{2}d{\mu }_k(x,t).}$$

(118)

Proof. 

Recall that the operator ${\mathcal{T}}_{U,\varphi ,\mathbf{m}}:{\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_k^2({\mathbb{R}}^N))\to {\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_k^2({\mathbb{R}}^N))$ is bounded and positive. Now, let ${\left\{v_j\right\}}_{j=1}^{\infty }$ be an arbitrary orthonormal basis for ${\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_k^2({\mathbb{R}}^N)).$ Then, if we denote $u_j$ by $u_j={({\mathfrak{W}}_{\varphi }^{\mathbf{m}})}^{\ast }(v_j),$ then ${\left\{u_j\right\}}_{j=1}^{\infty }$ is an orthonormal basis for $L_k^2({\mathbb{R}}^N)$.

Thus, by (94) and Fubini’s theorem,

$$\begin{array}{ccc}{\displaystyle \sum _{j=1}^{\infty }{⟨{\mathcal{T}}_{U,\varphi ,\mathbf{m}}(v_j),v_j⟩}_{L_{{\mu }_k}^2({\mathbb{R}}^{2N})}}\hfill & =\hfill & {\displaystyle \sum _{j=1}^{\infty }{⟨{\mathcal{L}}_{\varphi }^{\mathbf{m}}(U){({\mathfrak{W}}_{\varphi }^{\mathbf{m}})}^{\ast }(v_j),{({\mathfrak{W}}_{\varphi }^{\mathbf{m}})}^{\ast }(v_j)⟩}_{L_k^2({\mathbb{R}}^N)}}\hfill \\ & =\hfill & {\displaystyle \sum _{j=1}^{\infty }{\int }_U{|{\mathfrak{W}}_{\varphi }^{\mathbf{m}}(u_j)(x,t)|}^{2}d{\mu }_k(x,t)}\hfill \\ & =\hfill & {\displaystyle {\int }_U\sum _{j=1}^{\infty }{|{\mathfrak{W}}_{\varphi }^{\mathbf{m}}(u_j)(x,t)|}^{2}d{\mu }_k(x,t)}\hfill \\ & =\hfill & {\displaystyle {\int }_U\sum _{j=1}^{\infty }{|{⟨u_j,{\varphi }_{x,t}^{\mathbf{m}}⟩}_{L_k^2({\mathbb{R}}^N)}|}^{2}d{\mu }_k(x,t)}\hfill \\ & =\hfill & {\displaystyle {\int }_U{\parallel {\varphi }_{x,t}^{\mathbf{m}}\parallel }_{L_k^2({\mathbb{R}}^N)}^{2}d{\mu }_k(x,t)}\hfill \\ & =\hfill & {\mathcal{M}}_k(U,\varphi ,\mathbf{m}).\hfill \end{array}$$

By Remark 4 and Definition 9, it follows that ${\mathcal{T}}_{U,\varphi ,\mathbf{m}}\in S_1$, with $\parallel {\mathcal{T}}_{U,\varphi ,\mathbf{m}}{\parallel }_{S_1}=tr({\mathcal{T}}_{U,\varphi ,\mathbf{m}})={\mathcal{M}}_k(U,\varphi ,\mathbf{m}).$ □

Let ${\mathbf{V}}_{U,\varphi ,\mathbf{m}}:L_k^2({\mathbb{R}}^N)\to L_k^2({\mathbb{R}}^N)$ be the operator defined by ${\mathbf{V}}_{U,\varphi ,\mathbf{m}}=P_{\varphi }^{\mathbf{m}}{P}_U{P}_{\varphi }^{\mathbf{m}}.$ Compared to ${\mathcal{T}}_{U,\varphi ,\mathbf{m}}$, ${\mathbf{V}}_{U,\varphi ,\mathbf{m}}$ has the benefit of being defined on $L_k^2({\mathbb{R}}^N)$, which makes it easy to link its spectral properties to its integral kernel.

Given that ${\mathcal{T}}_{U,\varphi ,\mathbf{m}}$ is trace class and non-negative, the decomposition

$$L_{{\mu }_k}^2({\mathbb{R}}^{2N})={\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_k^2({\mathbb{R}}^N))\oplus {\left({\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_k^2({\mathbb{R}}^N))\right)}^{\perp },$$

implies that ${\mathbf{V}}_{U,\varphi ,\mathbf{m}}$ is also positive and trace class with

$$tr({\mathbf{V}}_{U,\varphi ,\mathbf{m}})=tr({\mathcal{T}}_{U,\varphi ,\mathbf{m}})={\mathcal{M}}_k(U,\varphi ,\mathbf{m}).$$

(119)

In addition, we have the following result.

Proposition 22. 

The trace of ${\mathcal{T}}_{U,\varphi ,\mathbf{m}}^2$ is given by

$${\displaystyle tr({\mathcal{T}}_{U,\varphi ,\mathbf{m}}^2)={\int }_U{\int }_U{|{\mathcal{K}}_{\varphi }^{\mathbf{m}}(x,t;x^{\prime },t^{\prime })|}^{2}d{\mu }_k(x,t)d{\mu }_k(x^{\prime },t^{\prime }).}$$

(120)

Proof. 

As ${\mathbf{V}}_{U,\varphi ,\mathbf{m}}$ is positive, then

$$tr({\mathcal{T}}_{U,\varphi ,\mathbf{m}}^2)=tr({\mathbf{V}}_{U,\varphi ,\mathbf{m}}^2).$$

(121)

On the other hand, using the fact that the space ${\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_k^2({\mathbb{R}}^N))$ is a reproducing kernel Hilbert space with kernel ${\mathcal{K}}_{\varphi }^{\mathbf{m}}$, we obtain that for $F\in {\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_k^2({\mathbb{R}}^N))$

$${\displaystyle {\mathbf{V}}_{U,\varphi ,\mathbf{m}}F(x,t)={\int }_{{\mathbb{R}}^{2N}}F(x^{\prime },t^{\prime }){\int }_{{\mathbb{R}}^{2N}}{\chi }_U(s,y){\mathcal{K}}_{\varphi }^{\mathbf{m}}(x,t;s,y){\mathcal{K}}_{\varphi }^{\mathbf{m}}(s,y;x^{\prime },t^{\prime })d{\mu }_k(s,y)d{\mu }_k(x^{\prime },t^{\prime }).}$$

(122)

That is, ${\mathbf{V}}_{U,\varphi ,\mathbf{m}}$ has integral kernel

$${\displaystyle {\mathbf{N}}_{U,\varphi }^{\mathbf{m}}(x,t;x^{\prime },t^{\prime })={\int }_{{\mathbb{R}}^{2N}}{\chi }_U(s,y){\mathcal{K}}_{\varphi }^{\mathbf{m}}(x,t;s,y){\mathcal{K}}_{\varphi }^{\mathbf{m}}(s,y;x^{\prime },t^{\prime })d{\mu }_k(s,y).}$$

(123)

Therefore,

$$\begin{array}{ccc}\hfill tr({\mathbf{V}}_{U,\varphi ,\mathbf{m}}^2)& =& {\displaystyle {\int }_{{\mathbb{R}}^{2N}}{\int }_{{\mathbb{R}}^{2N}}{|{\mathbf{N}}_{U,\varphi }^{\mathbf{m}}(x,t;x^{\prime },t^{\prime })|}^{2}d{\mu }_k(x,t)d{\mu }_k(x^{\prime },t^{\prime })}\hfill \\ & =& {\displaystyle {\int }_{{\mathbb{R}}^{2N}}{\int }_{{\mathbb{R}}^{2N}}{\mathbf{N}}_{U,\varphi }^{\mathbf{m}}(x,t;x^{\prime },t^{\prime })\overline{{\mathbf{N}}_{U,\varphi }^{\mathbf{m}}(x,t;x^{\prime },t^{\prime })}d{\mu }_k(x,t)d{\mu }_k(x^{\prime },t^{\prime })}\hfill \\ & =& {\displaystyle {\int }_{{\mathbb{R}}^{2N}}{\int }_{{\mathbb{R}}^{2N}}{\chi }_U(z_1){\chi }_U(z_2){\mathfrak{K}}_{\varphi }^{\mathbf{m}}(z_1;z_2)d{\mu }_k(z_1)d{\mu }_k(z_2),}\hfill \end{array}$$

where

$$\begin{array}{ccc}\hfill {\mathfrak{K}}_{\varphi }^{\mathbf{m}}(z_1;z_2)& =& {\displaystyle {\int }_{{\mathbb{R}}^{2N}}{\int }_{{\mathbb{R}}^{2N}}{\mathcal{K}}_{\varphi }^{\mathbf{m}}(z_2;x,t){\mathcal{K}}_{\varphi }^{\mathbf{m}}(x,t;z_1){\mathcal{K}}_{\varphi }^{\mathbf{m}}(z_1;x^{\prime },t^{\prime }){\mathcal{K}}_{\varphi }^{\mathbf{m}}(x^{\prime },t^{\prime };z_2)d{\mu }_k(x,t)d{\mu }_k(x^{\prime },t^{\prime })}\hfill \\ & =& {\mathcal{K}}_{\varphi }^{\mathbf{m}}(z_2;z_1){\mathcal{K}}_{\varphi }^{\mathbf{m}}(z_1;z_2).\hfill \end{array}$$

By (107), we have

$${\mathfrak{K}}_{\varphi }^{\mathbf{m}}={|{\mathcal{K}}_{\varphi }^{\mathbf{m}}|}^2.$$

(124)

□

5.2. Spectral Properties

The spectral theorem provides the following spectral representation:

$${\displaystyle {\mathcal{L}}_{\varphi }^{\mathbf{m}}(U)(f)=\sum _{j=1}^{\infty }{s}_j(U){⟨f,{\phi }_j^U⟩}_{L_k^2({\mathbb{R}}^N)}{\phi }_j^U,f\in L_k^2({\mathbb{R}}^N),}$$

(125)

and this is because ${\mathcal{L}}_{\varphi }^{\mathbf{m}}(U)={({\mathfrak{W}}_{\varphi }^{\mathbf{m}})}^{\ast }{\chi }_U{\mathfrak{W}}_{\varphi }^{\mathbf{m}}$ is compact and self-adjoint. Here, ${\left\{{\phi }_j^U\right\}}_{j=1}^{\infty }$ is the orthonormal set of eigenfunctions, corresponding to the positive eigenvalues ${\left\{s_j(U)\right\}}_{j=1}^{\infty }$, which are arranged in a non-increasing manner. That is, $s_j(U)\searrow 0$, and from Relation (96),

$$s_j(U)\le s_1(U)\le 1,\forall j\ge 1.$$

(126)

We can infer from (126) and (114) that the deformed Calderón–Toeplitz operator

$${\mathcal{T}}_{U,\varphi ,\mathbf{m}}:{\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_k^2({\mathbb{R}}^N))\to {\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_k^2({\mathbb{R}}^N))$$

is diagonalizable as

$${\displaystyle {\mathcal{T}}_{U,\varphi ,\mathbf{m}}F=\sum _{j=1}^{\infty }{s}_j(U){⟨F,t_j^U⟩}_{L_{{\mu }_k}^2({\mathbb{R}}^{2N})}{t}_j^U,F\in {\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_k^2({\mathbb{R}}^N)),}$$

(127)

where $t_j^U={\mathfrak{W}}_{\varphi }^{\mathbf{m}}({\phi }_j^U)$.

Lemma 4. 

We have, for all $z=(x,t)\in {\mathbb{R}}^{2N},$

$${\displaystyle {\mathrm{\Theta }}_k^{\mathbf{m}}(z):={\int }_{{\mathbb{R}}^{2N}}{\chi }_U(\omega ){|{\mathcal{K}}_{\varphi }^{\mathbf{m}}(\omega ;z)|}^{2}d{\mu }_k(\omega )=\sum _{j=1}^{\infty }{s}_j(U){\mathbf{S}}_{\varphi ,\mathbf{m}}^D({\phi }_j^U)(z).}$$

(128)

Proof. 

From Proposition 15, we have, for all $z=(x,t)\in {\mathbb{R}}^{2N},$ the function ${\mathcal{K}}_{\varphi }^{\mathbf{m}}(.;z)$ is in ${\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_k^2({\mathbb{R}}^N)).$ Therefore, using the properties of the kernel of the reproducing kernel Hilbert space, we obtain

$$\begin{array}{ccc}{⟨{\mathcal{T}}_{U,\varphi ,\mathbf{m}}{\mathcal{K}}_{\varphi }^{\mathbf{m}}(.;z),{\mathcal{K}}_{\varphi }^{\mathbf{m}}(.;z)⟩}_{L_{{\mu }_k}^2({\mathbb{R}}^{2N})}\hfill & =\hfill & {⟨P_U{\mathcal{K}}_{\varphi }^{\mathbf{m}}(.;z),{\mathcal{K}}_{\varphi }^{\mathbf{m}}(.;z)⟩}_{L_{{\mu }_k}^2({\mathbb{R}}^{2N})}\hfill \\ & =\hfill & {\displaystyle {\int }_{{\mathbb{R}}^{2N}}{\chi }_U(\omega ){\mathcal{K}}_{\varphi }^{\mathbf{m}}(\omega ;z)\overline{{\mathcal{K}}_{\varphi }^{\mathbf{m}}(\omega ;z)}d{\mu }_k(\omega )}\hfill \\ & =\hfill & {\displaystyle {\int }_{{\mathbb{R}}^{2N}}{\chi }_U(\omega ){|{\mathcal{K}}_{\varphi }^{\mathbf{m}}(\omega ;z)|}^{2}d{\mu }_k(\omega ).}\hfill \end{array}$$

Let ${\left\{w_j^U\right\}}_{j=1}^{\infty }\subset {\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_k^2({\mathbb{R}}^N))$ be an orthonormal basis of $Ker({\mathcal{T}}_{U,\varphi ,\mathbf{m}})$ (eventually empty). Hence, ${\left\{t_j^U\right\}}_{j=1}^{\infty }\cup {\left\{w_j^U\right\}}_{j=1}^{\infty }$ is an orthonormal basis of ${\mathfrak{W}}_{\varphi }^{\mathbf{m}}(L_k^2({\mathbb{R}}^N))$; therefore, the reproducing kernel ${\mathcal{K}}_{\varphi }^{\mathbf{m}}$ can be written as

$${\displaystyle {\mathcal{K}}_{\varphi }^{\mathbf{m}}(x,t;x^{\prime },t^{\prime })=\overline{{\mathcal{K}}_{\varphi }^{\mathbf{m}}(x^{\prime },t^{\prime };z)}=\sum _{j=1}^{\infty }{t}_j^U(z)\overline{t_j^U(x^{\prime },t^{\prime })}+\sum _{j=1}^{\infty }{w}_j^U(z)\overline{w_j^U(x^{\prime },t^{\prime })}.}$$

(129)

Then

$$\begin{array}{ccc}{⟨{\mathcal{T}}_{U,\varphi ,\mathbf{m}}{\mathcal{K}}_{\varphi }^{\mathbf{m}}(.;z),{\mathcal{K}}_{\varphi }^{\mathbf{m}}(.;z)⟩}_{L_{{\mu }_k}^2({\mathbb{R}}^{2N})}\hfill & =\hfill & {⟨{\displaystyle {\mathcal{T}}_{U,\varphi ,\mathbf{m}}\sum _{j=1}^{\infty }\overline{t_j^U(z)}{t}_j^U,\sum _{n=1}^{\infty }\overline{t_n^U(z)}{t}_n^U}⟩}_{L_{{\mu }_k}^2({\mathbb{R}}^{2N})}\hfill \\ & =\hfill & {\displaystyle \sum _{j,n}\overline{t_j^U(z)}{t}_n^U(z){⟨{\mathcal{T}}_{U,\varphi ,\mathbf{m}}{t}_j^U,t_n^U⟩}_{L_{{\mu }_k}^2({\mathbb{R}}^{2N})}}\hfill \\ & =\hfill & {\displaystyle \sum _{j=1}^{\infty }{s}_j(U){|t_j^U(z)|}^2,}\hfill \end{array}$$

and the conclusion follows. □

Let $\epsilon \in (0,1)$ and define the quantity

$$r(\epsilon ,U,\mathbf{m}):=\mathrm{card}\left\{j:s_j(U)\ge 1-\epsilon \right\}.$$

(130)

The following estimate for the eigenvalue distribution is then obtained by the use of a simple adaption of the proof of Lemma 3.3 in [38].

Proposition 23. 

If $\epsilon \in (0,1)$, then

$$|r(\epsilon ,U,\mathbf{m})-{\mathcal{M}}_k(U,\varphi ,\mathbf{m})|\le max\left\{{\displaystyle \frac{1}{\epsilon }},{\displaystyle \frac{1}{1-\epsilon }}\right\}\left|{\displaystyle {\int }_U{\int }_U{|{\mathcal{K}}_{\varphi }^{\mathbf{m}}(x^{\prime },t^{\prime };x,t)|}^{2}d{\mu }_k(x,t)d{\mu }_k(x^{\prime },t^{\prime })-{\mathcal{M}}_k(U,\varphi ,\mathbf{m})}\right|.$$

5.3. Spectrogram of a Subspace

Let $P_V$ be the orthogonal projection onto V, with projection kernel ${\mathfrak{G}}_V$, that is,

$${\displaystyle P_{V}f(·)={\int }_{{\mathbb{R}}^N}{\mathfrak{G}}_V(·,t)f(t)d{\gamma }_k(t),}$$

(131)

where V is an n-dimensional subspace V of $L_k^2({\mathbb{R}}^N).$

Remember that if ${\left\{v_j\right\}}_{j=1}^n$ is any orthonormal basis of V, then

$${\displaystyle {\mathfrak{G}}_V(x,t)=\sum _{j=1}^n{v}_j(x)\overline{v_j(t)}.}$$

(132)

Notice that the choice of orthonormal basis for V has no effect on the kernel ${\mathfrak{G}}_V$.

Definition 13. 

Let V be an n-dimensional subspace V of $L_k^2({\mathbb{R}}^N).$ Then, the spectrogram of V with respect to ϕ is given by

$${\displaystyle {\mathbf{SPEC}}_{\varphi ,\mathbf{m}}^{k}V(x,t):={\int }_{{\mathbb{R}}^N}{\int }_{{\mathbb{R}}^N}{\mathfrak{G}}_V(s,z)\overline{{\varphi }_{x,t}^{\mathbf{m}}(s)}{\varphi }_{x,t}^{\mathbf{m}}(z)d{\gamma }_k(s)d{\gamma }_k(z).}$$

(133)

Proposition 24. 

The spectrogram ${\mathbf{SPEC}}_{\varphi ,\mathbf{m}}^{k}V$ satisfies

$${\displaystyle {\mathbf{SPEC}}_{\varphi ,\mathbf{m}}^{k}V=\sum _{j=1}^n{\mathbf{S}}_{\varphi ,\mathbf{m}}^D(v_j).}$$

(134)

Proof. 

As

$$\begin{array}{ccc}{\mathbf{SPEC}}_{\varphi ,\mathbf{m}}^{k}V(x,t)\hfill & =\hfill & {\displaystyle {\int }_{{\mathbb{R}}^N}{\int }_{{\mathbb{R}}^N}\sum _{j=1}^n{v}_j(s)\overline{v_j(y){\varphi }_{x,t}^{\mathbf{m}}(s)}{\varphi }_{x,t}^{\mathbf{m}}(y)d{\gamma }_k(s)d{\gamma }_k(y)}\hfill \\ & =\hfill & {\displaystyle \sum _{j=1}^n{⟨v_j,{\varphi }_{x,t}^{\mathbf{m}}⟩}_{L_k^2({\mathbb{R}}^N)}{\overline{⟨v_j,{\varphi }_{x,t}^{\mathbf{m}}⟩}}_{L_k^2({\mathbb{R}}^N)}}\hfill \\ & =\hfill & {\displaystyle \sum _{j=1}^n{\mathfrak{W}}_{\varphi }^{\mathbf{m}}(v_j)(x,t)\overline{{\mathfrak{W}}_{\varphi }^{\mathbf{m}}(v_j)(x,t)}}\hfill \\ & =\hfill & {\displaystyle \sum _{j=1}^n{|{\mathfrak{W}}_{\varphi }^{\mathbf{m}}(v_j)(x,t)|}^2,}\hfill \end{array}$$

then we have the desired result. □

Definition 14. 

Let V be an n-dimensional subspace V of $L_k^2({\mathbb{R}}^N).$ Then, the following defines the time–frequency concentration of V in U:

$${\displaystyle {\xi }_{U,\varphi }^{\mathbf{m}}(V):=\frac{1}{n}{\int }_U{\mathbf{SPEC}}_{\varphi ,\mathbf{m}}^{k}V(x,t)d{\mu }_k(x,t).}$$

(135)

Then, from (134),

$${\displaystyle {\xi }_{U,\varphi }^{\mathbf{m}}(V):=\frac{1}{n}\sum _{j=1}^n{\int }_U{\mathbf{S}}_{\varphi ,\mathbf{m}}^D(v_j)(x,t)d{\mu }_k(x,t).}$$

(136)

Theorem 8. 

Using the first n eigenfunctions of ${\mathcal{L}}_{\varphi }^{\mathbf{m}}(U)$, which correspond to the n biggest eigenvalues ${\left\{s_j(U)\right\}}_{j=1}^n$, the n-dimensional signal space $v_n=\mathrm{span}{\left\{{\phi }_j^U\right\}}_{j=1}^n$ maximizes the regional concentration ${\xi }_{U,\varphi }^{\mathbf{m}}(V)$, and we have

$${\displaystyle {\xi }_{U,\varphi }^{\mathbf{m}}(v_n):=\frac{1}{n}\sum _{j=1}^n{s}_j(U).}$$

(137)

Proof. 

We have

$${\displaystyle {\xi }_{U,\varphi }^{\mathbf{m}}(v_n):=\frac{1}{n}\sum _{j=1}^n{\int }_U{\mathbf{S}}_{\varphi ,\mathbf{m}}^D({\phi }_j^U)(x,t)d{\mu }_k(x,t).}$$

(138)

Moreover, the min–max lemma for self-adjoint operators states that (see, e.g., Sec. 95 in [39])

$${\displaystyle s_j(U)={\int }_U{\mathbf{S}}_{\varphi ,\mathbf{m}}^D({\phi }_j^U)(z)d{\mu }_k(x,t)=max\left\{{⟨{\mathcal{L}}_{\varphi }^{\mathbf{m}}(U)(f),f⟩}_{L_k^2({\mathbb{R}}^N)}:{\parallel f\parallel }_{L_k^2({\mathbb{R}}^N)}=1,f\perp {\phi }_1^U,\dots ,{\phi }_{j-1}^U\right\}.}$$

Thus, the number of orthogonal functions with a well-concentrated spectrogram in U is determined by the eigenvalues of ${\mathcal{L}}_{\varphi }^{\mathbf{m}}(U)$. Consequently,

$${\displaystyle {\xi }_{U,\varphi }^{\mathbf{m}}(v_n)=\frac{1}{n}\sum _{j=1}^n{s}_j(U).}$$

(139)

The time–frequency operator ${\mathcal{L}}_{\varphi }^{\mathbf{m}}(U)$ has optimum cumulative time–frequency concentration within U for its first n eigenfunctions, according to the min–max characterization of compact operator eigenvalues, that is,

$${\displaystyle \sum _{j=1}^n{⟨{\mathcal{L}}_{\varphi }^{\mathbf{m}}(U)({\phi }_j^U),{\phi }_j^U⟩}_{L_k^2({\mathbb{R}}^N)}=max\left\{\sum _{j=1}^n{⟨{\mathcal{L}}_{\varphi }^{\mathbf{m}}(U)v_j,v_j⟩}_{L_k^2({\mathbb{R}}^N)}:{\left\{v_j\right\}}_{j=1}^n\mathrm{orthonormal}\right\}.}$$

(140)

Therefore, any n-dimensional subset V of $L_k^2({\mathbb{R}}^N)$ cannot be better concentrated in U than $v_n,$ i.e.,

$${\xi }_{U,\varphi }^{\mathbf{m}}(V)\le {\xi }_{U,\varphi }^{\mathbf{m}}(v_n).$$

(141)

□

Notice that we have

$$s_n(U)\le {\xi }_{U,\varphi }^{\mathbf{m}}(v_n)\le s_1(U)\le 1.$$

(142)

5.4. Accumulated Spectrogram

Let ${\rho }_{(U,\varphi ,\mathbf{m})}^k:={\mathbf{SPEC}}_{\varphi ,\mathbf{m}}^k{V}_{n_k(U,\varphi ,\mathbf{m})},$ called the accumulated spectrogram, where we assume that $n_k(U,\varphi ,\mathbf{m})=[{\mathcal{M}}_k(U,\varphi ,\mathbf{m})]$ is the smallest integer greater than or equal to ${\mathcal{M}}_k(U,\varphi ,\mathbf{m})$, and

$$V_{n_k(U,\varphi ,\mathbf{m})}=\mathrm{span}{\left\{v_j^U\right\}}_{j=1}^{n_k(U,\varphi ,\mathbf{m})}.$$

Then,

$${\displaystyle {\rho }_{(U,\varphi ,\mathbf{m})}^k(x,t)=\sum _{j=1}^{n_k(U,\varphi ,\mathbf{m})}{|{\mathfrak{W}}_{\varphi }^{\mathbf{m}}(v_j^U)(x,t)|}^2.}$$

(143)

Note that

$$\parallel {\rho }_{(U,\varphi ,\mathbf{m})}^k{\parallel }_{L_{{\mu }_k}^1({\mathbb{R}}^{2N})}=n_k(U,\varphi ,\mathbf{m})={\mathcal{M}}_k(U,\varphi ,\mathbf{m})+O(1).$$

Moreover, as

$${\displaystyle \sum _{j=1}^{n_k(U,\varphi ,\mathbf{m})}{s}_j(U)\le tr({\mathcal{L}}_{\varphi }^{\mathbf{m}}(U))={\mathcal{M}}_k(U,\varphi ,\mathbf{m}),}$$

then

$$E_k^{\mathbf{m}}(\varphi ,U):=1-{\displaystyle \frac{{\displaystyle \sum _{j=1}^{n_k(U,\varphi ,\mathbf{m})}{s}_j(U)}}{{\mathcal{M}}_k(U,\varphi ,\mathbf{m})}}.$$

(144)

fulfills

$$0\le E_k^{\mathbf{m}}(\varphi ,U)\le 1.$$

(145)

Moreover, we have the following lemma.

Lemma 5. 

If $\epsilon \in (0,1)$, then

$$0\le E_k^{\mathbf{m}}(\varphi ,U)\le 1-(1-\epsilon )min\left(1,{\displaystyle \frac{r(\epsilon ,U,\mathbf{m})}{{\mathcal{M}}_k(U,\varphi ,\mathbf{m})}}\right).$$

(146)

Proof. 

Let $\epsilon \in (0,1)$ and define $l_k(\epsilon ,U,\mathbf{m})=min(n_k(U,\varphi ,\mathbf{m}),r(\epsilon ,U,\mathbf{m}))$. It follows that

$$s_j(U)\ge 1-\epsilon ,1\le j\le l_k(\epsilon ,U,\mathbf{m}).$$

(147)

As $n_k(U,\varphi ,\mathbf{m})\ge l_k(\epsilon ,U,\mathbf{m})$, we obtain

$${\displaystyle \sum _{j=1}^{n_k(U,\varphi ,\mathbf{m})}{s}_j(U)\ge \sum _{j=1}^{l_k(\epsilon ,U,\mathbf{m})}{s}_j(U)\ge (1-\epsilon )l_k(\epsilon ,U,\mathbf{m}).}$$

(148)

Therefore

$$0\le E_k^{\mathbf{m}}(\varphi ,U)\le 1-(1-\epsilon ){\displaystyle \frac{l_k(\epsilon ,U,\mathbf{m})}{{\mathcal{M}}_k(U,\varphi ,\mathbf{m})}}.$$

(149)

As $n_k(U,\varphi ,\mathbf{m})\ge {\mathcal{M}}_k(U,\varphi ,\mathbf{m}),$ we obtain the desired result. □

Consequently, when the eigenvalues ${\left\{s_j(U)\right\}}_{j=0}^{r(\epsilon ,U,\mathbf{m})}$ are close to 1, then $E_k^{\mathbf{m}}(\varphi ,U)\to 0.$ Moreover, we have the following result bounding the error between ${\rho }_{(U,\varphi ,\mathbf{m})}^k$ and ${\mathrm{\Theta }}_k^{\mathbf{m}}.$

Proposition 25. 

We have

$${\displaystyle \frac{1}{{\mathcal{M}}_k(U,\varphi ,\mathbf{m})}}{∥{\rho }_{(U,\varphi ,\mathbf{m})}^k-{\mathrm{\Theta }}_k^{\mathbf{m}}∥}_{L_{{\mu }_k}^1({\mathbb{R}}^{2N})}\le {\displaystyle \frac{1}{{\mathcal{M}}_k(U,\varphi ,\mathbf{m})}}+2E_k^{\mathbf{m}}(\varphi ,U).$$

(150)

Proof. 

Using Lemma 4,

$${\displaystyle {\rho }_{(U,\varphi ,\mathbf{m})}^k(·)-{\mathrm{\Theta }}_k^{\mathbf{m}}(·)=\sum _{j=1}^{\infty }(t_j-s_j(U)){|{\mathfrak{W}}_{\varphi }^{\mathbf{m}}(v_j^U)(·)|}^2,}$$

(151)

where $t_j=0$ if $j>n_k(U,\varphi ,\mathbf{m})$ and 1 otherwise. As

$${\displaystyle \parallel |{\mathfrak{W}}_{\varphi }^{\mathbf{m}}(v_j^U){{|}^2\parallel }_{L_{{\mu }_k}^1({\mathbb{R}}^{2N})}=1\mathrm{and}\sum _{j=1}^{\infty }{s}_j(U)={\mathcal{M}}_k(U,\varphi ,\mathbf{m}),}$$

we obtain

$$\begin{array}{ccc}{∥{\rho }_{(U,\varphi ,\mathbf{m})}^k-{\mathrm{\Theta }}_k^{\mathbf{m}}∥}_{L_{{\mu }_k}^1({\mathbb{R}}^{2N})}\hfill & \le \hfill & {\displaystyle \sum _{j=1}^{\infty }|t_j-s_j(U)|}\hfill \\ & =\hfill & {\displaystyle \sum _{j=1}^{n_k(U,\varphi ,\mathbf{m})}(1-s_j(U))+\sum _{j>n_k(U,\varphi ,\mathbf{m})}{s}_j(U)}\hfill \\ & =\hfill & {\displaystyle n_k(U,\varphi ,\mathbf{m})+\sum _{j=1}^{\infty }{s}_j(U)-2\sum _{j=1}^{n_k(U,\varphi ,\mathbf{m})}{s}_j(U)}\hfill \\ & =\hfill & {\displaystyle \left(n_k(U,\varphi ,\mathbf{m})-{\mathcal{M}}_k(U,\varphi ,\mathbf{m})\right)+2({\mathcal{M}}_k(U,\varphi ,\mathbf{m})-\sum _{j=1}^{n_k(U,\varphi ,\mathbf{m})}{s}_j(U))}\hfill \\ & \le \hfill & {\displaystyle 1+2({\mathcal{M}}_k(U,\varphi ,\mathbf{m})-\sum _{j=1}^{n_k(U,\varphi ,\mathbf{m})}{s}_j(U)),}\hfill \end{array}$$

and the estimate (150) follows. □

Remark 5. 

Let ϕ be in $L_{k,rad}^2({\mathbb{R}}^N)$. We proceed as in [26], and we define the modulation of ϕ by t otherwise, as follows:

$${\mathcal{M}}_t^{\mathbf{m}}(\varphi ):={\mathfrak{F}}_k^{{\mathbf{m}}^{-1}}\left(\sqrt{|T_t^{\mathbf{m},k}(e^{{\displaystyle \frac{id}{2b}}{\parallel z\parallel }^2}|{\mathfrak{F}}_k(\varphi ){|}^2)|}\right).$$

(152)

Subsequently, we define the generalized windowed transform ${\mathcal{W}}_{\varphi }^{\mathbf{m}}$ as follows:

$${\displaystyle \forall (y,t)\in {\mathbb{R}}^{2N},{\mathcal{G}}_{\varphi }^{\mathbf{m}}(f)(y,t):={\int }_{{\mathbb{R}}^N}f(\xi )T_y^{{\mathbf{m}}^{-1},k}({\mathcal{M}}_t^{\mathbf{m}}\varphi )(-\xi )d{\gamma }_k(\xi )=L_{\frac{-2a}{b}}f{\ast }_{{\mathbf{m}}^{-1},k}{\mathcal{M}}_t^{\mathbf{m}}\varphi (y).}$$

(153)

It is clear that

$${\mathcal{W}}_{\varphi }^{\mathbf{m}}={\mathfrak{W}}_{{\mathfrak{F}}_k(\varphi )}^{\mathbf{m}}.$$

(154)

Thus, by involving Plancherel’s formula (18), we derive that the two integral transforms are equivalent and then all results proved for one are valuables for the second. Therefore, we claim that all results proved in this paper for the LCDGT ${\mathfrak{W}}_{\varphi }^{\mathbf{m}}$ are valuables for the integral transform ${\mathcal{W}}_{\varphi }^{\mathbf{m}}$ and it is sufficient to replace ϕ by ${\mathfrak{F}}_k(\varphi )$ to derive the analog results.

6. Conclusions and Perspectives

In the present paper, we accomplished three major objectives. First, we investigated the linear canonical Dunkl windowed transform and studied its elementary properties. Then, we introduced the localization operators associated with the LCDWT and studied their trace class and Schatten–von Neumann class properties. As a side result, the spectrograms associated with the LCDWT were studied in detail. Finally, we indicate that in the future work, we will study other applications of the LCDWT, such as the uncertainty principles, which set limitations for a non-trivial signal and its LCDWT to be both well localized in the time–frequency plane.