Matrix Expression of Convolution and Its Generalized Continuous Form

Abstract

In this paper, we consider the matrix expression of convolution, and its generalized continuous form. The matrix expression of convolution is effectively applied in convolutional neural networks, and in this study, we correlate the concept of convolution in mathematics to that in convolutional neural network. Of course, convolution is a main process of deep learning, the learning method of deep neural networks, as a core technology. In addition to this, the generalized continuous form of convolution has been expressed as a new variant of Laplace-type transform that, encompasses almost all existing integral transforms. Finally, we would, in this paper, like to describe the theoretical contents as detailed as possible so that the paper may be self-contained.

1. Introduction

Deep learning means the learning of deep neural networks, called deep and if multiple hidden layers exist. Deep learning allows computational models that are composed of multiple processing layers to learn representations of data with multiple levels of abstraction [1]. The convolution in convolutional deep neural network (CNN) is the tool for obtaining a feature map from the original image data, it sweeps the original image with a kernel matrix, and transforms the original data into a different shape. This distorted image is called a feature map. Therefore, in CNN, the convolution can be regarded as a tool that creates a feature map from the original image. Herein, the concept of convolution in artificial intelligence is demonstrated mathematically.

The core concept of CNN is the convolution which applies the weight to the receptive fields only, and it transforms the original data into a feature map. This process is called convolution. This is a similar principle to integral transform. The method of Integral transform maps from the original domain to another domain to solve a given problem more easily. Since the matrix expression of convolution is an essential concept in artificial intelligence, we believe that this study would certainly be meaningful. In addition to this, the generalized continuous form of convolution has also been studied, and thus this form is expressed as a new variant of Laplace-type transform.

On one hand, the transform theory is extensively utilized in fields involving medical diagnostic equipment, such as magnetic resonance imaging or computed tomography. Typically, a projection data are obtained by an integral transform, and an image using an inverse transform is produced. Although plausible integral transforms exist, almost all existing integral transforms are not sufficiently satisfied with fullness, and can be interpreted as a Laplace-type transform. One of us proposed a comprehensive form of the Laplace-type integral transform in [2]. The present study is being conducted to investigate the matrix expression of convolution and its generalized continuous form.

In [2], a Laplace-type integral transform was proposed, expressed as

$$G_{\alpha }\left(f\right)=G\left(f\right)=u^{\alpha }{\int }_0^{\infty }{e}^{-\frac{t}{u}}f\left(t\right)dt.$$

(1)

For values of $\alpha$ as 0, $-1$, 1, and $-2$, we have, respectively, the Laplace [3], Sumudu [4], Elzaki [5], and Mohand transforms [6]. This form can be expressed in various manners. Replacing t by $ut$, we have

$$G\left(f\right)=u^{\beta }{\int }_0^{\infty }{e}^{-t}f\left(ut\right)dt,$$

where $\beta =\alpha +1$. In the form, $\beta$ values of 1, 0, 2, and $-1$ correspond to the Laplace, Sumudu, Elzaki, and Mohand transforms, respectively. If we substitute $u=1/s$ in (1), we then obtain the simplest form of the generalized integral transform as follows:

$$G\left(f\right)=s^{\gamma }{\int }_0^{\infty }{e}^{-st}f\left(t\right)dt,$$

where $\gamma =-\alpha$. In this form, the Laplace, Sumudu, Elzaki, and Mohand transforms have $\gamma$ values of 0, 1, $-1$, and 2, respectively. It is somewhat paved, but essentially a simple way to derive the Sumudu transform is to multiply the Laplace transform by s. Similarly, it can be obtained multiply by $s^{-1}$ to obtain the Elzaki transform, and multiply by $s^2$ to obtain the Mohand transform. The natural transform [7] can be obtained by substituting $f\left(t\right)$ with $f\left(ut\right)$. Additionally, by substituting $t=lnx$, the Laplace-type transform $G\left(f\right)$ can be expressed as

$$s^{-\alpha }{\int }_1^{\infty }f(lnx)x^{-s-1}dx.$$

As a similar form, there is a Mellin transform [8] of the form

$${\int }_0^{\infty }f\left(x\right)x^{s-1}dx.$$

As shown above, many integral transforms have their own fancy masks, but most of them can essentially be interpreted as Laplace-type transforms. From a different point of view, a slight change in the kernel results in a significant difference in the integral transform theory. Meanwhile, plausible transforms exist, such as the Fourier, Radon, and Mellin transforms. Typically, if the interval of integration and the power of kernel are different, it can be interpreted as a completely different transform. Studies using Laplace transform were conducted in [9,10]. The generalized solutions of the third-order Cauchy–Euler equation in the space of right-sided distributions has found [9], studied the solution of the heat equation without boundary conditions [10], and investigated further properties of Laplace-type transform [11]. As an application, a new class of Laplace-type integrals involving generalized hypergeometric functions has been studied [12,13]. As for research related to the integral equation, Noeiaghdam et al. [14] presented a new scheme based on the stochastic arithmetic. The scheme is presented to guarantee the validity and accuracy of the homotopy analysis method. Different kinds of integral equations such as singular and first kind are considered to find the optimal results by applying the proposed algorithms.

The main objective of this study is to investigate the matrix expression of convolution and its generalized continuous form. The generalized continuous form of the matrix expression was carried out in the form of a new variant of Laplace-type transform. The obtained result are as follows:

(1)

If the matrix representing the function (image) f is A and the matrix representing the function g is B, then the convolution $f\ast g$ is represented by the sum of all elements of $A\circ B$ and this is the same as $tr\left(A{B}^T\right)$ where ∘ is array multiplication, T is the transpose, and $tr$ is the trace. Thus, the convolution in artificial intelligence (AI) is the same as $tr\left(A{B}^T\right)$.

(2)

The generalized continuous form of the convolution in AI can be represented as

$$V\left(f\right)=\Phi \left(u\right){\int }_0^{\infty }{e}^{-t\Delta }f\left(t\right)dt,$$

where $\Phi \left(u\right)$ is an arbitrary bounded function and

$$\Delta =\Delta (\delta ,u)=\frac{ln[1+\frac{\delta -1}{u}]}{\delta -1}.$$

2. Matrix Expression of Convolution in Convolutional Neural Network (CNN)

Note that functions can be interpreted as images in artificial intelligence (AI). The convolution is changed from ${\int }_0^{t}f\left(\tau \right)g(t-\tau )d\tau$ to

$$\sum _{\tau =0}^{t}f\left(\tau \right)g(t-\tau )$$

by the discretization. The convolution in CNN is the tool for obtaining a feature map from the original image data, plays a role to sweeping the original image with kernel matrices (or filter), and it transforms original data into a different shape. In order to calculate the convolution, each $n\times n$ part of the original matrix is element-wise multiplied by the kernel matrix and all its components are added. Typically, the kernel matrix is using by $3\times 3$ matrix. On the one hand, the pooling (or sub-sampling) is a simple job, reducing the size of the image made by convolution. It is the principle that the resolution is increased when the screen is reduced.

Let the matrix representing the function f is A and the matrix representing the function g is B. For two matrices A and B of the same dimension, the array multiplication (or sweeping) $A\circ B$ is given by

$${(A\circ B)}_{ij}={\left(A\right)}_{ij}{\left(B\right)}_{ij}.$$

For example, the array multiplication for $2\times 2$ matrices is

$$\left(\begin{array}{cc}a& b\\ c& d\end{array}\right)\circ \left(\begin{array}{cc}e& f\\ g& h\end{array}\right)=\left(\begin{array}{cc}ae& bf\\ cg& dh\end{array}\right).$$

The array multiplication appears in lossy compression such as joint photographic experts group and the decoding step. Let us look at an example.

Example 1.

In the classification field of AI, the pixel is treated as a matrix. When the original image is

$$\left(\begin{array}{ccccc}1& 2& 3& 0& 0\\ 0& 1& 2& 1& 0\\ 3& 0& 1& 0& 1\\ 1& 0& 2& 1& 0\\ 0& 1& 1& 2& 0\end{array}\right),$$

array multiplying the kernel matrix

$$\left(\begin{array}{ccc}2& 0& 1\\ 0& 1& 2\\ 2& 0& 1\end{array}\right)$$

on the first $3\times 3$ matrix, we obtain the matrix

$$\left(\begin{array}{ccc}2& 0& 3\\ 0& 1& 4\\ 6& 0& 1\end{array}\right).$$

Now, adding all of its components, we obtain $17$. Next, if we array-multiply the kernel matrix to the $3\times 3$ matrix

$$\left(\begin{array}{ccc}2& 3& 0\\ 1& 2& 1\\ 0& 1& 0\end{array}\right)$$

on the right and add all the components, we get $8$ by stride $1$. If we continue this process to the final matrix

$$\left(\begin{array}{ccc}1& 0& 1\\ 2& 1& 0\\ 1& 2& 0\end{array}\right),$$

we get $6$. Consequently, the original matrix changes to

$$\left(\begin{array}{ccc}17& 8& 10\\ 8& 5& 10\\ 12& 8& 6\end{array}\right)$$

by using the convolution kernel. This is called the convolved feature map.

This is just an example for understanding, and in perceptron the output uses a value between $-1$ and 1 using the activation function. Note that the perceptron is an artificial network designed to mimic the brain’s cognitive abilities. Therefore, the output of neuron (or node) Y can be represented as

$$Y=sign[\sum _{i=1}^n{x}_i{w}_i-\Theta ]=\left\{\begin{array}{cc}1\hfill & X\ge \Theta \hfill \\ -1\hfill & X<\Theta \hfill \end{array}\right.,$$

where w is a weight, $\Theta$ is the threshold value, and X is the activation function with $X={\sum }_{i=1}^n{x}_i{w}_i$. In the backpropagation algorithm of deep neural network, the sigmoid function

$$Y^{sigmoid}=\frac{1}{1+e^{-x}}$$

is used as the activation function [15]. This function is easy to differentiate and ensures neuron output is in $[0,1]$. If max-pulling is applied to the above convolved feature map, the resulting matrix becomes $1\times 1$ matrix $\left(17\right)=17$.

As discussed above, convolution in AI can be obtained by array multiplication. We would like to associate this definition with matrix multiplication in mathematics.

Definition 1.

(Convolution in AI) If the matrix representing the function (image) f is A and the matrix representing the function g is B, then the convolution $f\ast g$ is represented by the sum of all elements of $A\circ B$ and this is the same as $tr\left(A{B}^T\right)$ where ∘ is array multiplication, T is the transpose, and $tr$ is the trace. Thus, the convolution in AI is the same as $tr\left(A{B}^T\right)$, the sum of all elements on the diagonal with the right side facing down in $A{B}^T$.

Typically, the convolution kernel is used as a $3\times 3$ matrix, but for easy understanding, let us consider a $2\times 2$ matrix.

Example 2.

If

$$A=\left(\begin{array}{cc}a& b\\ c& d\end{array}\right)$$

and

$$B=\left(\begin{array}{cc}e& f\\ g& h\end{array}\right),$$

then the convolution in AI is calculated as $ae+bf+cg+dh$ by the sweeping. On the other hand,

$$A{B}^T=\left(\begin{array}{cc}a& b\\ c& d\end{array}\right)\left(\begin{array}{cc}e& g\\ f& h\end{array}\right)=\left(\begin{array}{cc}ae+bf& ag+bh\\ ce+df& cg+dh\end{array}\right),$$

and

$$tr\left(A{B}^T\right)=ae+bf+cg+dh$$

for T is the transpose and $tr$ is the trace. This is the same result as in AI.

3. Generalized Continuous Form of Matrix Expression of Convolution

If the matrix representing a function f is A and the matrix representing a function g is B, then the convolution of the functions f and g can be denoted by $tr\left(A{B}^T\right)$. Intuitively, the diagonal part of $B^T$ corresponds to a graph of $g(t-\tau )$. The overlapping part of the graph can be interpreted as the concept of intersection, that is, the concept of multiplication. Thus, the generalized continuous form of the convolution in AI can be represented in a variant of Laplace-type transform given by

$$G_{\alpha }\left(f\right)=G\left(f\right)=u^{\alpha }{\int }_0^{\infty }{e}^{-\frac{t}{u}}f\left(t\right)dt$$

$$=u^{\alpha }{\int }_0^{\infty }\underset{\delta \to 1}{lim}{(1+\frac{\delta -1}{u})}^{-\frac{t}{\delta -1}}f\left(t\right)dt.$$

If $f\left(t\right)$ is a function defined for all $t\ge 0$, an integral of Laplace-type transform $vi\left(f\right)$ is given by

$$F(\Delta )=vi\left(f\right)={\int }_0^{\infty }{e}^{-t\Delta (\delta ,u)}f\left(t\right)dt$$

$$={\int }_0^{\infty }{[1+\frac{\delta -1}{u}]}^{-\frac{t}{\delta -1}}f\left(t\right)dt$$

for $\delta >1$ with

$$\Delta (\delta ,u)=\frac{ln[1+\frac{\delta -1}{u}]}{\delta -1}.$$

Additionally, let $\Phi \left(u\right)$ be an arbitrary bounded function and let $V\left(f\right)$ be a variant of Laplace-type transform of $f\left(t\right)$. If $f\left(t\right)$ is a function defined for all $t\ge 0$, $V\left(f\right)$ is defined by

$$V\left(f\right)=\Phi \left(u\right){\int }_0^{\infty }{e}^{-t\Delta }f\left(t\right)dt$$

$$=\Phi \left(u\right){\int }_0^{\infty }{[1+\frac{\delta -1}{u}]}^{-\frac{t}{\delta -1}}f\left(t\right)dt$$

for $\delta >1$ with $\Delta =\Delta (\delta ,u)$.

Based on the above two definitions, it is clear that the above variant of Laplace-type transform is represented as $V\left(f\right)=\Phi \left(u\right)·vi\left(f\right)$ for an arbitrary function $\Phi \left(u\right)$. If so, let us see the relation with other integral transforms. Since

$$V\left(f\right)=\Phi \left(u\right){\int }_0^{\infty }{e}^{-t\Delta }f\left(t\right)dt,$$

if $\delta \to 1+$ and $\Phi \left(u\right)=u^{\alpha }$, then it corresponds to the $G_{\alpha }$ transform. When we take $\delta \to 1$, $\Phi \left(u\right)=1$, and $u=1/s$, we get the Laplace transform. Similarly, when we take $\delta \to 1$, $\Phi \left(u\right)=u^{-1}(\delta \to 1,\Phi \left(u\right)=u)$, we get the Sumudu transform (Elzaki transform), respectively. In order to obtain a simple form of generalization, it is better to set $\varphi \left(u\right)$ to $u^{\alpha }$ for an arbitrary integer $\alpha$. However, it is judged that $\varphi \left(u\right)$ is better than $u^{\alpha }$ as a suitable generalization, where $\varphi \left(u\right)$ is a bounded arbitrary function. The reason is that $\varphi \left(u\right)$ can express more integral transforms.

Lemma 1.

(Lebesgue dominated convergence theorem [16,17]). Let $(X,M,\mu )$ be a measure space and suppose $\left\{f_n\right\}$ is a sequence of extended real-valued measurable functions defined on X such that

(a) ${lim}_{n\to \infty }{f}_n\left(x\right)=f\left(x\right)$ exists μ-a.e.

(b) There is an integrable function g so that for each n, $|f_n|\le g$μ-a.e.

Then, f is integrable and

$$\underset{n\to \infty }{lim}{\int }_X{f}_{n}d\mu ={\int }_{X}fd\mu .$$

Beppo Livi’s theorem is a special form of Lemma 1. Its contents are as follows:

$$\int \sum _{n=1}^{\infty }{g}_{n}d\mu =\sum _{n=1}^{\infty }\int g_{n}d\mu$$

for $\left(g_n\right)$ is a nondecreasing sequence. The details are can be found on page 71 in [16]. Note that the convolution of f and g is given by

$$(f\ast g)\left(t\right)={\int }_0^{\infty }f\left(\tau \right)g(t-\tau )d\tau .$$

The following theorem is as follows. Since the proof is not difficult, we would like to cover just a few.

Theorem 1.

(1) 

(Duality with Laplace transform) If $\pounds \left(f\right)=F_{\ast }\left(s\right)$ is the Laplace transform of a function $f\left(t\right)$, then it satisfies the relation of $V\left(f\right)=\Phi \left(u\right)·F_{\ast }(\Delta )$.

(2) 

(Shifting theorem) If $f\left(t\right)$ has the transform $F\left(u\right)$, then $e^{at}f\left(t\right)$ has the transform $\Phi \left(u\right)·F(\Delta -a)$. That is,

$$V\left[e^{at}f\left(t\right)\right]=\Phi \left(u\right)·F(\Delta -a).$$

Moreover, If $f\left(t\right)$ has the transform $F\left(u\right)$, then the shifted function $f(t-a)h(t-a)$ has the transform $e^{-a\Delta }·\Phi \left(u\right)F(\Delta )$. In formula,

$$V\left[f(t-a)h(t-a)\right]=e^{-a\Delta }·\Phi \left(u\right)F(\Delta )$$

for $h(t-a)$ is Heaviside function (We write h since we need u to denote u-space).

(3) 

(Linearity) Let $V\left(f\right)$ be the variant of Laplace-type transform. Then $V\left(f\right)$ is a linear operation.

(4) 

(Existence) If $f\left(t\right)$ is defined, piecewise continuous on every finite interval on the semi-axis $t\ge 0$ and satisfies

$$\left|f\left(t\right)\right|\le M{e}^{kt}$$

for all $t\ge 0$ and some constants M and k, then the variant of Laplace-type transform $V\left(f\right)$ exists for all $\Delta >k$.

(5) 

(Uniqueness) If the variant of Laplace-type transform of a given function exists, then it is uniquely determined.

(6) 

(Heaviside function)

$$vi\left[h(t-a)\right]={\int }_0^{\infty }{e}^{-t\Delta }h(t-a)dt={\int }_a^{\infty }{e}^{-t\Delta }·1dt$$

$$=e^{-a\Delta }/\Delta ,$$

where h is Heaviside function.

(7) 

(Dirac’s delta function) We consider the function

$$f_k(t-a)=\left\{\begin{array}{cc}1/k\hfill & ifa\le t\le a+k\hfill \\ 0\hfill & otherwise.\hfill \end{array}\right.$$

In a similar way to Heaviside, taking the integral of Laplace-type transform, we get

$$vi\left[f_k(t-a)\right]={\int }_0^{\infty }{e}^{-t\Delta }{f}_k(t-a)dt=-\frac{1}{k\Delta }{\left[e^{-t\Delta }\right]}_a^{a+k}$$

$$=-\frac{1}{k\Delta }(e^{-(a+k)\Delta }-e^{-a\Delta })=-\frac{1}{k\Delta }{e}^{-a\Delta }(e^{-k\Delta }-1).$$

If we denote the limit of $f_k$ as $\delta (t-a)$, then

$$vi\left(\delta (t-a)\right)=\underset{k\to 0}{lim}\left(vi\right)\left[f_k(t-a)\right]=e^{-a\Delta }.$$

(8) 

(Shifted data problems) For a given differential equation $y^{″}+a{y}^{\prime }+by=r\left(t\right)$ subject to $y\left(t_0\right)=c_0$ and $y^{\prime }\left(t_0\right)=c_1$, where $t_0\ne 0$ and a and b are constant, we can set $t=t_1+t_0$. Then $t=t_0$ gives $t_1=0$ and so, we have

$$y_1^{″}+a{y}_1^{\prime }+b{y}_1=r(t_1+t_0),y_1\left(0\right)=c_0,y_1^{\prime }\left(0\right)=c_1$$

for input $r\left(t\right)$. Taking the variant, we can obtain the output $y\left(t\right)$.

(9) 

(Transforms of derivatives and integrals) Let a function f is n-th differentiable and integrable, and let us consider the fraction Δ as an operator. Then $V\left(f\right)$ of the n-th derivatives of $f\left(t\right)$ satisfies

$$vi\left(f^{\left(n\right)}\right)={\Delta }^{n}vi\left(f\right)-\sum _{k=1}^n{\Delta }^{n-k}{f}^{(k-1)}\left(0\right)$$

(2)

and

$$V\left[{\int }_0^{t}f\left(\tau \right)d\tau \right]=\Phi \left(u\right)·\frac{1}{\Delta }V\left(f\right).$$

(10) 

(Convolution) If two functions f and g are integrable for * is the convolution, then $V(f\ast g)$ satisfies

$$V(f\ast g)=\Phi \left(u\right)·F(\Delta )G(\Delta )$$

for $V\left(f\right)=F(\Delta )$.

Proof. 

(5) Assume that $V\left(f\right)$ exists by $V\left(f_1\right)$ and $V\left(f_2\right)$ both. If $V\left(f_1\right)\ne V\left(f_2\right)$ for $f_1=f_2$, then

$$\begin{array}{ccc}\hfill V\left(f_1\right)-V\left(f_2\right)& =& \Phi \left(u\right){\int }_0^{\infty }{e}^{-t\Delta }{f}_1\left(t\right)dt-\Phi \left(u\right){\int }_0^{\infty }{e}^{-t\Delta }{f}_2\left(t\right)dt\hfill \\ & =& \Phi \left(u\right){\int }_0^{\infty }{e}^{-t\Delta }(f_1\left(t\right)-f_2\left(t\right))dt\hfill \\ & =& V(f_1-f_2)=0.\hfill \end{array}$$

This is a contradiction on $V\left(f_1\right)\ne V\left(f_2\right)$, and hence the transform is uniquely determined. Conversely, if two functions $f_1$ and $f_2$ have the same transform (i.e., if $V\left(f_1\right)=V\left(f_2\right)$), then

$$V\left(f_1\right)-V\left(f_2\right)=\Phi \left(u\right){\int }_0^{\infty }{e}^{-t\Delta }(f_1\left(t\right)-f_2\left(t\right))dt=0,$$

and so $f_1=f_2$ a.e. Hence $f_1=f_2$ excepting for the set of measure zero.

(9) Note that $vi\left(f\right)={\int }_0^{\infty }{e}^{-t\Delta }f\left(t\right)dt$, and let us approach the proof by induction. In case of $n=1$,

$$vi\left(f^{\prime }\right)={\int }_0^{\infty }{e}^{-t\Delta }{f}^{\prime }\left(t\right)dt.$$

Integrating by parts, we have

$$vi\left(f^{\prime }\right)={\left[e^{-t\Delta }f\left(t\right)\right]}_0^{\infty }+\Delta {\int }_0^{\infty }{e}^{-t\Delta }f\left(t\right)dt$$

$$=-f\left(0\right)+\Delta vi\left(f\right).$$

which is true by (2).

Next, let us suppose that $n=m$ is valid for some m. Thus,

$$vi\left(f^{\left(m\right)}\right)={\Delta }^{m}vi\left(f\right)-\sum _{k=1}^m{\Delta }^{m-k}{f}^{(k-1)}\left(0\right)$$

holds for $f^{\left(m\right)}$ is the m-th derivative of f. Let us show that

$$vi\left(f^{(m+1)}\right)={\Delta }^{m+1}vi\left(f\right)-\sum _{k=1}^{m+1}{\Delta }^{m+1-k}{f}^{(k-1)}\left(0\right).$$

(3)

Now we start with the left-hand side of (2).

$$vi\left(f^{(m+1)}\right)={\int }_0^{\infty }{e}^{-t\Delta }{f}^{\left(m\right)}\left(t\right)dt$$

$$=\Delta vi\left(f^{\left(m\right)}\right)-f^{\left(m\right)}\left(0\right)$$

$$=\Delta [{\Delta }^{m}vi\left(f\right)-\sum _{k=1}^m{\Delta }^{m-k}{f}^{(k-1)}\left(0\right)]-f^{\left(m\right)}\left(0\right)$$

$$={\Delta }^{m+1}vi\left(f\right)-\sum _{k=1}^m{\Delta }^{m+1-k}{f}^{(k-1)}\left(0\right)]-f^{\left(m\right)}\left(0\right)$$

$$={\Delta }^{m+1}vi\left(f\right)-\sum _{k=1}^{m+1}{\Delta }^{m+1-k}{f}^{(k-1)}\left(0\right).$$

Therefore, this theorem is valid for an arbitrary natural number n. Putting $g\left(t\right)={\int }_0^{t}f\left(\tau \right)d\tau$,

$$vi\left(f\left(t\right)\right)=vi\left(g^{\prime }\left(t\right)\right)=\Delta vi\left(g\right)-g\left(0\right)=\Delta vi\left(g\right)$$

follows.  □

As the direct results of (9), $vi\left(f^{\prime }\right)=\Delta vi\left(f\right)-f\left(0\right)$ and $vi\left(f^{″}\right)={\Delta }^{2}vi\left(f\right)-\Delta f\left(0\right)-f^{\prime }\left(0\right)$ are follow.

For example, we consider $y^{″}-y=t$ subject to $y\left(0\right)=1$ and $y^{\prime }\left(0\right)=1$. Taking the integral of Laplace-type transform on both sides, we have

$${\Delta }^{2}Y-\Delta y\left(0\right)-y^{\prime }\left(0\right)-Y=1/{\Delta }^2$$

for $Y=\left(vi\right)\left(y\right)$. Organizing this equation, we get $({\Delta }^2-1)Y=\Delta +1+1/{\Delta }^2$. Simplification gives

$$Y=\frac{1}{\Delta -1}+\frac{1}{{\Delta }^2-1}-\frac{1}{{\Delta }^2}.$$

From the relation of $V\left(f\right)=\Phi \left(u\right)·F_{\ast }(\Delta )$, we have the solution

$$y\left(t\right)=-t+2sinht+cosht=e^t+sinht-t,$$

where h is hyperbolic function.

Example 3.

(Integral equations of Volterra type) Find the solution of

$$\left(1\right)y\left(t\right)+{\int }_0^t(t-\tau )y\left(\tau \right)d\tau =1.$$

$$\left(2\right)y\left(t\right)-{\int }_0^{t}y\left(\tau \right)sin(t-\tau )d\tau =t.$$

$$\left(3\right)y\left(t\right)-{\int }_0^t(1+\tau )y(t-\tau )d\tau =1-sinht.$$

Solution.

(1)

Since this equation is $y+y\ast t=1$, taking the integral of Laplace-type transform on both sides, we have

$$Y+(Y·\frac{1}{{\Delta }^2})=\frac{1}{\Delta }$$

for $Y=vi\left(y\right)$. Thus

$$Y=\frac{\Delta }{{\Delta }^2+1}$$

and so, we obtain the solution $y=cost$.

Let us do the check by expansion. Expanding, we get $y^{″}\left(t\right)+y\left(t\right)=0$. Since ${\int }_a^{a}f=0$, we get $y\left(0\right)=1$ and $y^{\prime }\left(0\right)=0$. Thus, we obtain $y=cost$.

(2)

This is rewritten as a convolution

$$y\left(t\right)-y\ast sint=t.$$

Taking the integral of Laplace-type transform, we have

$$Y\left(u\right)-Y\left(u\right)\frac{1}{{\Delta }^2+1}=Y\left(u\right)(1-\frac{1}{{\Delta }^2+1})=\frac{1}{{\Delta }^2}$$

for $Y=vi\left(y\right)$. The solution is

$$Y\left(u\right)=\frac{1}{{\Delta }^2}+\frac{1}{{\Delta }^4}$$

and gives the answer

$$y\left(t\right)=t+\frac{1}{6}{t}^3.$$

(3)

Note that the equation is the same as $y-(1+t)\ast y=1-sinht$. Taking the transform, we get

$$Y-(\frac{1}{\Delta }+\frac{1}{{\Delta }^2})Y=\frac{1}{\Delta }-\frac{1}{{\Delta }^2-1},$$

and hence

$$Y(1-\frac{1}{\Delta }-\frac{1}{{\Delta }^2})=\frac{{\Delta }^2-\Delta -1}{\Delta ({\Delta }^2-1)}.$$

Simplification gives

$$Y=\frac{\Delta }{{\Delta }^2-1},$$

and so, we obtain the answer

$$y\left(t\right)=cosht$$

by the relation of $V\left(f\right)=\Phi \left(u\right)·F_{\ast }(\Delta )$.

Let us turn the topic to initial value problem of the convolution. The initial value problem

$$a{y}^{″}+b{y}^{\prime }+cy=f\left(t\right),y\left(0\right)=y_0,y^{\prime }\left(0\right)=y_0^{\prime }$$

gives

$$(a{\Delta }^2+b\Delta +c)Y(\Delta )-(a\Delta +b)y\left(0\right)-a{y}^{\prime }\left(0\right)=F(\Delta ),$$

where $Y(\Delta )=vi\left(y\right)$ and $F(\Delta )=vi\left(f\right)$. Simplification gives

$$Y(\Delta )=\frac{1}{a{\Delta }^2+b\Delta +c}·F(\Delta )+y_0·\frac{a\Delta +b}{a{\Delta }^2+b\Delta +c}$$

$$+y_0^{\prime }·\frac{a}{a{\Delta }^2+b\Delta +c}.$$

If we put the system function $H(\Delta )={(a{\Delta }^2+b\Delta +c)}^{-1}$, then

$$Y(\Delta )=H(\Delta )F(\Delta )+y_0(a\Delta +b)H(\Delta )+y_0^{\prime }aH(\Delta ).$$

Since $H(\Delta )F(\Delta )=vi\left(h\right)vi\left(f\right)=vi(h\ast f)$ for $H(\Delta )=vi\left(h\right(t\left)\right)$, taking the inverse transform, we have

$$y=(h\ast f)+y_{0}v{i}^{-1}\left\{(a\Delta +b)H(\Delta )\right\}$$

$$+y_0^{\prime }v{i}^{-1}\left\{aH(\Delta )\right\}.$$

Theorem 2.

(Differentiation and integration of transforms) Let us put $V=F(\Delta )$ and $Y=V\left(y\right)$. Then

$$\begin{array}{ccc}& \left(1\right)& V\left(t^{n}f\left(t\right)\right)=\Phi \left(u\right)·{(-1)}^n\frac{d^n}{d{s}^n}F(\Delta )\hfill \\ & \left(2\right)& V(f\left(t\right)/t)=\Phi \left(u\right){\int }_{\Delta }^{\infty }F\left(\delta \right)d\delta .\hfill \end{array}$$

Proof. 

This is an immediate consequence of $V\left(f\right)=\Phi \left(u\right)·F_{\ast }(\Delta )$ and $V\left(f\right)=\Phi \left(u\right)·vi\left(f\right)$. For this reason, detailed proofs are omitted.

The statements below are the immediate results of Theorem 2.

$$\begin{array}{ccc}& \left(1\right)& V\left(tf\left(t\right)\right)=-\Phi \left(u\right)F^{\prime }(\Delta )\hfill \\ & \left(2\right)& V\left(t{f}^{\prime }\left(t\right)\right)=-\Phi \left(u\right)(Y+\Delta \frac{dY}{d\Delta })\hfill \\ & \left(3\right)& V\left(t{f}^{″}\left(t\right)\right)=\Phi \left(u\right)·(-2\Delta Y-{\Delta }^2\frac{dY}{d\Delta }+y\left(0\right))\hfill \end{array}$$

Let us check examples for temperature in an infinite bar and displacement in a semi-infinite string by the variant of Laplace-type transform.  □

Example 4.

(Semi-infinite string) Find the displacement $w(x,t)$ of an elastic string subject to the following conditions [3].

(a) 

The string is initially at rest on the x-axis from $x=0$ to ∞.

(b) 

For $t>0$ the left end of the string is moved in a given fashion, namely, according to a single sine wave

$$w(0,t)=f\left(t\right)=\left\{\begin{array}{cc}sin2t\hfill & if0\le t\le \pi \hfill \\ 0\hfill & otherwise.\hfill \end{array}\right.$$

(c) 

Furthermore, $w(x,t)\to 0$ as $x\to \infty$ for $t\ge 0$.

Then the displacement w is

$$w(x,t)=f(t-\frac{x}{c})h(t-\frac{x}{c})=\left\{\begin{array}{cc}sin2(t-\frac{x}{c})\hfill & ifx/c<t<x/c+\pi \hfill \\ 0\hfill & otherwise,\hfill \end{array}\right.$$

where h is Heaviside function.

The proof is simple, and the interchangeability of limit and integral in the proof process guarantees its validity by the Lebesgue dominated convergence theorem.

Example 5.

(Temperature in an infinite bar) Find the temperature w in an infinite bar if the initial temperature is

$$f\left(x\right)=w(x,0)=\left\{\begin{array}{cc}{k}_0\left(constant\right)\hfill & if\left|x\right|<1\hfill \\ 0\hfill & otherwise\hfill \end{array}\right.$$

with $w(0,t)=0$.

Solution. Taking the integral of Laplace-type transform on both sides of $w_t=c^2{w}_{xx}$, we have

$$\Delta F-w(x,0)=c^2\frac{{\partial }^{2}F}{\partial x^2}$$

for $F(x,u)=vi\left[w\right(x,t\left)\right]$. Organizing the equality, we get

$$\frac{{\partial }^{2}F}{\partial x^2}-\frac{\Delta }{c^2}F=-\frac{1}{c^2}w(x,0).$$

Organizing this equality, we get

$$F(x,u)=A\left(u\right)e^{-\sqrt{\Delta }x/c}+B\left(u\right)e^{\sqrt{\Delta }x/c}$$

$$+e^{-\sqrt{\Delta }x/c}\int \frac{c}{2\sqrt{\Delta }}{e}^{\sqrt{\Delta }x/c}\frac{1}{c^2}w(x,0)dx$$

$$-e^{\sqrt{\Delta }x/c}\int \frac{c}{2\sqrt{\Delta }}{e}^{-\sqrt{\Delta }x/c}\frac{1}{c^2}w(x,0)dx,$$

(4)

where the Wronskian $W=2\sqrt{\Delta }/c.$ The value ${lim}_{x\to \infty }f\left(x\right)=0$ gives ${lim}_{x\to \infty }F(x,u)=0$, and hence $B\left(u\right)=0$. Thus, from $\left(4\right)$, we get

$$F(x,u)=A\left(u\right)e^{-\sqrt{\Delta }x/c}$$

$$+\frac{w(x,0)}{2c\sqrt{\Delta }}(e^{-\sqrt{\Delta }x/c}\int e^{\sqrt{\Delta }x/c}dx-e^{\sqrt{\Delta }x/c}\int e^{-\sqrt{\Delta }x/c}dx).$$

By the direct calculation, we have

$$F(x,u)=A\left(u\right)e^{-\sqrt{\Delta }x/c}+\frac{w(x,0)}{\Delta }.$$

From the formula of $vi\left(f\right)=F(\Delta )$ for $F\left(s\right)=\pounds \left(f\right)$ and $s=\Delta$, we know

$$vi\left(\frac{k}{2\sqrt{\pi t^3}}{e}^{-\frac{k^2}{4t}}\right)=e^{-k\sqrt{\Delta }}(k>0)$$

and $vi\left(1\right)=1/\Delta$. Taking the inverse transform, we obtain the temperature $w(x,t)$ as follows:

$$w(x,t)=A\left(t\right)\ast \frac{x}{2c\sqrt{\pi t^3}}{e}^{-\frac{x^2}{4c^{2}t}}+k_0$$

on $\left|x\right|<1$, and * is the convolution. In case of $\left|x\right|>1$, we have the solution

$$w(x,t)=A\left(t\right)\ast \frac{x}{2c\sqrt{\pi t^3}}{e}^{-\frac{x^2}{4c^{2}t}}.$$

In the above equality, we note that

$${vi}^{-1}\left[A\left(u\right)e^{-\sqrt{\Delta }x/c}\right]={vi}^{-1}[vi\left(A\left(t\right)\right)·vi\left(\frac{x}{2c\sqrt{\pi t^3}}{e}^{-\frac{x^2}{4c^{2}t}}\right)]$$

$$={vi}^{-1}\left[vi\{A\left(t\right)\ast \frac{x}{2c\sqrt{\pi t^3}}{e}^{-\frac{x^2}{4c^{2}t}}\}\right]=A\left(t\right)\ast \frac{x}{2c\sqrt{\pi t^3}}{e}^{-\frac{x^2}{4c^{2}t}}$$

because $vi(f\ast g)=F(\Delta )g(\Delta )$ for $vi\left(f\right)=F(\Delta )$.

4. Conclusions

In this study, the concept of convolution in convolutional neural networks (CNN) was presented mathematically and tried to connect with the concept of convolution in mathematics. As a continuous form of convolution in CNN, a new form of Laplace-type transform has been proposed. In the future, we will study the change of convolution in CNN by changing the stride. In addition to this, we shall also explore the possibility of our applying our newly defined Laplace-type transform in obtaining certain new and interesting results involving generalized hypergeometric functions that would certainly unify and generalized the results available in the literature and may be potentially useful from an applications point of view.