Deep Learning Approach for the Detection of Noise Type in Ancient Images

Abstract

Recent innovations in digital image capturing techniques facilitate the capture of stationary and moving objects. The images can be easily captured via high-end digital cameras, mobile phones and other handheld devices. Most of the time, captured images vary compared to actual objects. The captured images may be contaminated by dark, grey shades and undesirable black spots. There are various reasons for contamination, such as atmospheric conditions, limitations of capturing device and human errors. There are various mechanisms to process the image, which can clean up contaminated image to match with the original one. The image processing applications primarily require detection of accurate noise type which is used as input for image restoration. There are filtering techniques, fractional differential gradient and machine learning techniques to detect and identify the type of noise. These methods primarily rely on image content and spatial domain information of a given image. With the advancements in the technologies, deep learning (DL) is a technology that can be trained to mimic human intelligence to recognize various image patterns, audio files and text for accuracy. A deep learning framework empowers correct processing of multiple images for object identification and quick decision abilities without human interventions. Here Convolution Neural Network (CNN) model has been implemented to detect and identify types of noise in the given image. Over the multiple internal iterations to optimize the results, the identified noise is classified with 99.25% accuracy using the Proposed System Architecture (PSA) compared with AlexNet, Yolo V5, Yolo V3, RCNN and CNN. The proposed model in this study proved to be suitable for the classification of mural images on the basis of every performance parameter. The precision, accuracy, f1-score and recall of the PSA are 98.50%, 99.25%, 98.50% and 98.50%, respectively. This study contributes to the development of mural art recovery.

1. Introduction

Digital images play a vital role in many fields such as medical science, astronomy, geography, defense, and image animation [1]. In real life scenarios, it has been experienced that captured images or moving objects differ as compared to the original image. This has been experienced with high quality capturing media as well. The captured image is induced with unwanted effects such as grey scale, dense shade, improper light effects etc. This unwanted part of the captured image is termed as noise. Noise may cause incorrect results during image analysis. Image restoration is used to address the problem and manipulate the image in such a way that it will closely match the original [2]. Detecting and removing noise for preserving contents are requirements of current research in the image processing domain. There are different techniques to remove noise from the digital images, such as filtering and machine learning techniques. This is also termed as denoising. In the denoising process, the initial requirement is to detect and identify the type of noise from the given image. The detected noise type will give input to process further for the image restoration process. After noise detection (ND) and noise removal it can be used in many applications such as disease detection, object detection in astronomy etc. Image restoration (IR) is majorly required in many fields such as medical science and ancient historical document keeping [3].

In this paper, ancient mural images are used as an image dataset because these depict the cultural heritage and topographical status of a community. It gives information about historical details such as culture, lifestyle and nature at that time [4]. These provide important information to people regarding changes in ancient images and artistic style. These murals are of great interest to archaeologists and scientists for research work [5]. The preservation of these ancient images is a most challenging task, as most of these images are very old; some of these images became distorted due natural climatic disasters such as earthquakes, storms and rains [6]. It has also been noticed that improper handling of murals can also lead to distortion. These distortions in the murals have limited capabilities to perform research-oriented studies [7]. Degradation, shedding and cracking are common damage issues with ancient artworks. The distortions can lead researchers to identity and classify the murals and may result in inaccurate results. Restoration of the degraded artworks requires skilled artisans, which are difficult to find these days. Therefore, technique will be advanced to use digital image restoration to meet the requirements of the degraded mural images [8,9].

Restoration of such ancient images will be a massive contribution in historical research and study. This will help to permanently store and update mural information. In recent years researchers have been focusing on various interesting areas for preserving these historical details. This paper aims to focus its novel contribution in the noise detection (ND) and ancient image restoration (IR) area [7,10]. The different types of noise considered in this research include Poisson noise, speckle noise, Gaussian noise and impulse noise. There is a Probability Density Function (PDF) for Gaussian noise. Normal Gaussian distribution is used by Guassian noise, which is based on normal Gaussian distribution. The noise could be generated due to atmospheric conditions such as sensor noise or high temperature at the time of image acquisition. It is either additive or multiplicative. Poisson noise is similar to Gaussian distribution. The main reason for this type of noise generation is the number of photons present in the image which is captured by sensors and another reason is the discrete nature of electric charge. The speckle noise present in images is one type of granular noise, and exists inherently in an image. Signal gravity is the main reason for this type of degradation. It is more complex and harder to interpret the image [11,12,13]. A noise model study is essential for image noise removal, which is required for better accuracy results [14,15].

The primary research contribution starts with dominant features that have been extracted from the images. The feature extraction process is done by using wavelet transform. The wavelet transform is selected for the feature extraction due to the fact that it decomposes images to a high extent. Then the model is constructed by using convolution neural network.

Deep learning framework works analogous to human brain. This framework repeatedly performs tasks, tweaks, and optimizes during each run to get the improved results in comparison to traditional techniques [16]. The results are based on continuous learning, unlearning, and relearning without human intervention of framework. Deep learning has proven optimization in mission critical applications such as image tracing, semantic segmentation, image recognition and restoration [17].

Recent study has demonstrated detection of COVID-19 using patterns of Convolution Neural Networks and Long short-term memory (LSTM) through X-Ray images [18]. There are many kinds of deep learning models, among which convolutional neural network is commonly used [19].

The ancient mural images having the major challenges as these images are prone to be degraded. A PSA is designed based on multilevel neural network to identify and classify noise. The construction of this type of neural network not only gives high reliability but also works fine even if length of dataset is low. The results of PSA are then compared with existing with AlexNet, Yolo V5, Yolo V3, RCNN and CNN model. It only proves that the PSA is much robust than that of the existing one. The proposed model in this study proved to be suitable for the classification of mural images on the basis of every performance parameter. The precision, accuracy, f1-score and recall of the PSA are 98.50%, 99.25%, 98.50% and 98.50% respectively. This study contributes to the development of mural art analysis. Based on performance results, our model is more suitable for identifying and classifying mural image data sets. There is vast scope to use mural images for restoration. This will be significant support for archaeologist to study ancient mural images.

A list of abbreviations and acronyms are given in

Table 1. List of acronyms and abbreviations.

PSA DWT	Proposed System Architecture Discrete Wavelet Transform
ND	Noise Detection
IR	Image Restoration
PDF	Probability Density Function
ANN	Artificial Neural Network
RDN PSNR	Residual Dense Network Peak signal-to-noise ratio
IO	Ideal Observer
AMT	Automatic Machine Translation
DWT	Discrete Wavelet Transform
FDM	Frame Difference Method
AI WT RCNN	Artificial Intelligence Wavelet Transform Region-based Convolutional Neural Network

.

This paper contents are organized as: the literature survey of existing method is proposed in Section 2, Section 3 proposes noise identification flow; followed by performs implementation and analysis in Section 4; Section 5 gives results and in Section 6 conclusions are given.

2. Literature Survey

This section highlights existing methods and analyses strengths and weaknesses of existing methods. The challenges of research are highlighted towards the end of the section.

Qi Wang, Jing et al. [20] proposes preserving the information of low-frequency images to remove noise of high frequency. As a result, fractional calculus can effectively reduce noise, and the fractional calculus theory is well-established in digital image processing. This study proposes a new fractional differential gradient-based approach for detecting noise locations in images, as well as fractional integration based an enhanced image denoising algorithm. The proposed model can efficiently remove noise while preserving image textures and edges. The simplicity of the approach and the high degree of consistency are its advantages.

Ying Liufuetal [21], A reformative noise-immune neural network (RNINN) model has been developed based on this research, and it is compared to the classical gradient-based recursive neural network model, on convergence qualities and noise-resistance capabilities. The RNINN model’s efficiency is demonstrated by a relative numerical experiment simulation and application of images target detection.

Kushagra Yadav and colleagues [22] propose a two-stage object identification strategy, first stage is the image denoising process and second stage is the primary object detection, to upgrade prior detection pipelines. In image denoising step, proposed model uses the Residual Dense Network (RDN) to restore improved quality images from a noisy one. It extracts local characteristics from strongly connected convolutional layers while training hierarchical functions for image denoising. DNN-based denoising algorithms were validated using task-based IQ measures in paper [23]. Binary signal detection tasks were explicitly taken into account under SKE/BKS conditions. In order to assess the impact of denoising on task performance, the ideal observer (IO) and common linear numerical observers are assessed, and detection efficiency is estimated.

Studies examining modern denoising methods using objective methodologies are scarce [24]. To restore phase data when there is significant speckle decorrelation noise, it is critical to unwrap and de-noise the data. The two most common approaches to dealing with a noise wrapped phase are to de-noise before unwrapping or unwrap before de-noising. A comparison of the techniques’ resilience and efficiency is the goal of this study. Six combinations from various methodologies are compared in this study. The techniques are tested using ten simulated phase maps with escalating noise standard deviations. According to the simulation results, using windowed Fourier transform filtering before unwrapping with least-squares and iterations results in high accuracy and computation speed for restoring noisy data [25].

Varad A. Pimpalkhute and colleagues [26] proposed that in various image processing applications, such as denoising, compression, and video tracking, require accurate noise type and strength prediction. There are numerous approaches for estimating the kind and degree of noise in digital photographs. In most of these methods, images are transformed or their spatial domain information is used.

To evaluate the level of Gaussian noise in digital images, authors present a hybrid Discrete Wavelet Transform (DWT) and edge information removal-based approach. According to Kriengkri Langampol et al. [27] mixed noise happens when multiple types of noise present at the same time in images. Based on the switching bilateral filter, researchers present the detection switching bilateral filter (DSBF) where Gaussian and impulsive noise considered as mixed noise. The performance of filter is compared to NCSR, BM3D and SBF filters. The simulation results demonstrate that DSBF performs better as comparing it to other approaches, it provides the highest PSNR. Because this technique employs the best color parameter values for red, green, and blue. Furthermore, proposed technique’s complexity is similar to that of the SBF, which is the original filter in this field.

A two-stage SAID-END denoising algorithm was presented by Amandeep Singh et al. [28] to tackle the performance difficulties of existing denoising algorithms. This study by Danilo Gustavo Gil Sierra [29] proposes the sequential implementation of two bioinspired computational models to reduce impulsive noise and edge identification in grayscale images. This work, according to Piyush Joshi and Surya Prakash [30], based on noise detection, proposes a new technique for evaluating image quality.

Many applications rely on training images, such as face recognition and ear recognition. Poor-quality training photos result in inefficient training, which lowers the biometric system’s performance. This research proposes a method for estimating the quality of biometric images. For picture quality assessment, the technique does not require a reference image. The proposed technique’s efficacy has been demonstrated through experimental study.G. Maragatham et al. [31] describes noise detection and identification using statistical moment-based approach. This technique uses the discrete cosine transform to process frequency domain information of the image, which is grouped by bands. In order to determine whether noise exists, and more specifically whether impulsive noise exists, the statistical parameter kurtosis and the threshold sum of absolute deviation are used. The identification of noise is a key component of any digital image processing system that identifies the filters to smooth out the image for further processing.

For noise identification, an Artificial Neural Network (ANN)-based technique was presented [32]. The proposed method involved isolating the noise samples and extracting their statistical properties, which were then used to identify the noise using a neural network. The results showed that neural networks were more effective at detecting noise. The exact filter can be used to enhance the given images by recognizing the noise [33].

An approach based on neural networks for detecting noise types in noisy images is presented by Karibasappa K.G and colleagues [15]. There are many types of de-noising filters that can be applied to the proposed method. Based on the results of simulation experiments, the technology appears to be able to accurately determine the type of noise.

Hui Ying Khaw [34] proposes a model for recognizing image noise of various forms and levels, including Poisson, Speckle, Gaussian and Impulse noise as well as a combination of many types of noise.

The CNN method with back propagation algorithm is used to categories noise type. The authors’ investigations have shown that the suggested noise type recognition model is reliable, with an overall average accuracy as 99.3 percent when recognizing eight different types of noise. An image is a visual representation of something or someone, while a digital image is a numerical representation of that image. The technique that is used to change these pixel values in order to improve the image’s quality or to provide different viewpoints for human use [35].

According to Davood Karimia et al. [36], there is an increasing demand for such type of datasets for medical analysis applications. The influence of label noise, on the other hand, has gotten insufficient consideration. Label noise has been found to have a considerable impact on the performance of recent studies such as deep learning, machine learning. In medical science domain, as datasets often limited, domain expertise require for labeling and incorrect predictions may have a direct impact on human health.

In paper [19], study has shown machine leaning has achieved better performance on minimal sample datasets, whereas deep learning has significantly better performance on larger datasets. Deep learning techniques mimics human intelligence to understand audio sounds, text patterns and images. A smart deep learning framework is compared with human brain on object identifications and quick decision-making abilities. This is also known as deep neural setwork. This has widely used in the latest emerging techniques such as Driverless cars or self-driving and Automatic Machine Translation (AMT) [37].

In paper [38] implemented a photo classification system for cloud shapes using FDM extractor for detection and extraction cloudlike objects from large photos into small images, which can be sent further to a multi-channel CNN image classifier. Using this classification system, meteorologists will be able to categorize weather images more quickly, which will increase the efficiency of meteorological statistics. Weather forecasts can be improved by using comprehensive statistical data.

To reduce overfitting constraint of Deep CNN in paper [39] proposes a two-stage training method by optimizing the feature boundary. Pre-training, anomaly detection, and implicit regularization are three parts of two-stage training method. The implicit regularization training process begins with the anomaly detection using the pre-training model, followed by setting a regularization factor. This approach to implicit regularization at the data level offers innovative methods for AI, which can be applied to machine learning algorithms as well.

For automatic modulation classification in paper [40], proposed a spectrum interference-based two-level data augmentation method in DL. As deep CNN trained with data augmentation shows excellent performance, the proposed approach can be employed directly in cognitive radios if the model can be trained on more signals under different SNRs with various modulation schemes. Deep learning models can be improved by augmentation of data, but various factors, like model structure and weight initialization, may affect the generalization ability of neural networks.

Based on the literature discussed in this section and summary given in Table 2, the challenges of noise types in Ancient Images are:

• For removing noise from the images, it is necessary to detect the type of noise so that the content of the image will not get hampered while removing noise pixels.
• Also, there is a need to develop system in such a way that irrespective of the content of the image the type of noise should get detected.

In response to these challenges, the proposed system fills this literature gap. The proposed system is tested on different types of images, and it gives satisfactory results. In results and discussion chapter, results are given which got after comparison of PSA with various algorithms. The proposed system is evaluated on different processors for finding out the time complexity of the system as done by A.S. Ladkat et al. [41].

Table 2. Comparison with other detection applications.

Paper	Type of Detection	Technique Used	Dataset	Accuracy (%)
[9]	Classification of murals	MultiChannel seperable network model (MCSN)	China Dunhuang Murals	88.16
[16]	Biological images	Deep Learning	Wood boards	93
[18]	COVID-19	deep CNN-LSTM	X-ray images	99.4
[37]	Living or non living things	VGG16	ImageNet	99.8
[38]	Cloud shape	CNN and FDM	200 actual photos of real scenes a	94
[39]	Image detection	Two stage training	USPS, ILSVRC2012, MNIST, SVHN, CIFAR10, CIFAR100	98
Proposed System Architecture	Noise type	Wavelet Transform and CNN	Ancient mural images	99.25

Table 2. Comparison with other detection applications.

Paper	Type of Detection	Technique Used	Dataset	Accuracy (%)
[9]	Classification of murals	MultiChannel seperable network model (MCSN)	China Dunhuang Murals	88.16
[16]	Biological images	Deep Learning	Wood boards	93
[18]	COVID-19	deep CNN-LSTM	X-ray images	99.4
[37]	Living or non living things	VGG16	ImageNet	99.8
[38]	Cloud shape	CNN and FDM	200 actual photos of real scenes a	94
[39]	Image detection	Two stage training	USPS, ILSVRC2012, MNIST, SVHN, CIFAR10, CIFAR100	98
Proposed System Architecture	Noise type	Wavelet Transform and CNN	Ancient mural images	99.25

3. Proposed Noise Identification

Noise identification flow for the proposed system is shown in Figure 1 gives overall flow experiment. Some of Indian Heritage Image Retrieval Dataset images are processed to be reference images. A noisy image is generated by adding different types of noise to the reference image, such as Poission, Speckle, Gaussian and Impulse noise.

Then those classified by neural network classifier to obtain the classification accuracies. Furthermore, the differential accuracy of a classifier when given a reference image and images with noise was observed [42].

The PSA is shown in Figure 2 CNN is used because it gives maximum output. The PSA receives the labeled dataset as input in the form of coloured images. The wavelet transform has been used to extract the features from the photos (WT). The WT retrieves 1,572,864 characteristics in total. The system’s spatial complexity rises as a result. A dimensionality reduction technique is employed to lessen the complexity of the space, which reduces the number of features to 24,576. The 11 × 11 filter size convolution is chosen. The filter with 11-pixel height and width provides bigger spatial regions, which greatly aids in improving the classification’s accuracy. The pooling layer aims to reduce the spatial size of the representation that the convolutional layer collected. Essentially, it simplifies and condenses the information that has been obtained. The most used pooling technique is max pooling. The max pooling layer draws a window across the input, taking the maximum value inside the window and disregarding all other values. Similar to a convolutional layer the window size is specified (which corresponds to the filter size) and stride. Usually, a CNN’s last layers are fully connected layers. Their main responsibility is to categorize the features that the convolutional layer and pooling layer series have found and retrieved. Before feeding the fully connected layers, the feature maps are flattened into a single 1D vector which is then sent to the neural network for classification using the appropriate ratios of neurons and layers.

4. Algorithm Steps and Processes

PSA is implemented in two steps as described in the following steps:

• Step 1: Acquire ancient images from dataset

Dataset contains mural images from various heritage sites. Accept ancient images dataset and then images dataset is passed through wavelet transform.

• Step 2: Decompose image using wavelet transform and extract the features [43].

In digital image processing, feature extraction is the primary technique. After segmentation, this process executes in image processing Feature extraction is a crucial stage that represents the final result. It reduces the amount of resources needed to process a large set of data. It refers to the process of identifying and processing certain features within an image that are of interest. Features are described as functions of measurements. Each feature describes some quantifiable property of an object and is calculated in a way that estimates some essential characteristics. The features considered in PSA are Energy, Mean, Homogeneity, Contrast, Correlation, Variance, Skewness, Kurtosis, Entropy [20,44,45,46].

An energy measures how homogeneous image is, which means that it has a few gray levels.

$${{\displaystyle \sum }}_{\mathrm{i},\mathrm{j}}\mathrm{p}{\left(\mathrm{i},\mathrm{j}\right)}^2$$

(1)

Mean measures the intensity of the pixels in a region as a measure of the average value of their brightness.

$$\mathsf{\mu }=\sum _{\mathrm{i}=0}^{\mathrm{G}-1}\mathrm{i} \mathrm{p}\left(\mathrm{i}\right)$$

(2)

Homogeneity measures how close the distribution of elements in a matrix is to its diagonal matrix.

$${{\displaystyle \sum }}_{\mathrm{i},\mathrm{j}}\frac{\mathrm{p}\left(\mathrm{i},\mathrm{j}\right)}{1+\left|\mathrm{i}-\mathrm{j}\right|}$$

(3)

Contrast is the difference between the brightness of the objects or regions in a particular field of view and the brightness of their neighbors in that field of view.

$${{\displaystyle \sum }}_{\mathrm{i},\mathrm{j}}{\left|\mathrm{i}-\mathrm{j}\right|}^2\mathrm{p}\left(\mathrm{i},\mathrm{j}\right)$$

(4)

Correlation is degree and type of relationship between adjacent pixels throughout the whole image.

$${{\displaystyle \sum }}_{\mathrm{i},\mathrm{j}}\frac{\left(\mathrm{i}-{\mathsf{\mu }}_{\mathrm{i}}\right)\left(\mathrm{j}-{\mathsf{\mu }}_{\mathrm{j}}\right)\mathrm{p}\left(\mathrm{i},\mathrm{j}\right)}{{\mathsf{\sigma }}_{\mathrm{i}}{\mathsf{\sigma }}_{\mathrm{j}}}$$

(5)

Variance is a measure of how much the intensity of the image varies around the mean.

$${\mathsf{\sigma }}^2=\sum _{\mathrm{i}=0}^{\mathrm{G}-1}{\left(\mathrm{i}-\mathsf{\mu }\right)}^2 \mathrm{p}\left(\mathrm{i}\right)$$

(6)

When a histogram is symmetric about the mean, skewness is zero; otherwise, it is either positive or negative depending on whether the histogram is skewed above or below the average measure of asymmetry.

$${\mathsf{\mu }}_3={\mathsf{\sigma }}^{-3}=\sum _{\mathrm{i}=0}^{\mathrm{G}-1}{\left(\mathrm{i}-\mathsf{\mu }\right)}^3 \mathrm{p}\left(\mathrm{i}\right)$$

(7)

Kurtosis is a measure of flatness of the histogram or peak distribution relative to a normal distribution.

$${\mathsf{\mu }}_4={\mathsf{\sigma }}^{-4}=\sum _{\mathrm{i}=0}^{\mathrm{G}-1}\left({\left(\mathrm{i}-\mathsf{\mu }\right)}^4 \mathrm{p}\left(\mathrm{i}\right)\right)-3$$

(8)

Entropy is a measure of the overall disorder of the image or measure of randomness.

$$\mathrm{Entropy} =-\sum _{\mathrm{i}=0}^{\mathrm{G}-1} \mathrm{p}\left(\mathrm{i}\right){\log }_2\left[\mathrm{p}\left(\mathrm{i}\right)\right]$$

(9)

The low and high components are extracted from the given images. For the decomposition of the image level three, Daubechies 5 is selected. The $image\left(x,y\right)$ is passed through the wavelet transform to obtain $W_{\psi }^i\left(j,m,n\right)$ as a result, as illustrated in Equation (10).

$$W_{\phi }\left(j_0,m,n\right)=\frac{1}{\sqrt{MN}}\sum _{x=0}^{M-1N-1}image(x,y){\phi }_{j=0}(x,y)$$

(10)

$$W_{\psi }^i(j,m,n)=\frac{1}{\sqrt{MN}}\sum _{x=0}^{M=1}\sum _{y=0}^{N-1}image(x,y){\psi }^i{j}_{j,m,n}(x,y)\psi i=\{H,V,D\}$$

(11)

where $j_0$ is an arbitrary starting scale $W_{\phi }\left(j_0,m,n\right)$ coefficients define an approximation of $image\left(x,y\right)$ at scale $j_0{W}_{\psi }^i\left(j,m,n\right)$ parameters for the scales $j>=j_0.$ Given the $W_{\phi }$ and $W_{\psi }^i,image\left(x,y\right) \mathrm{is} \mathrm{as} \mathrm{follows}$:

$$\begin{array}{cc}\hfill image(x,y)=& \frac{1}{\sqrt{MN}}{\sum }_m{\sum }_n{W}_{\phi }\left(j_0,m,n\right){\phi }_{j_0,m,n}(x,y)\hfill \\ \hfill & +\frac{1}{\sqrt{MN}}{\sum }_{i=H,V,D}\sum _{j=j_0}^{\infty }{\sum }_m{\sum }_n{W}_{\psi }^i(j,m,n){\psi }_{j,m,n}^i(x,y)\hfill \end{array}$$

(12)

Equation (12) shows decomposition of the $image\left(x,y\right),$ which is nothing but the extracted features. The result of the decomposition of the image acquires higher space. To reduce the space complexity, the following mathematical model is used. This model is used to reduce the dimensionality of the extracted feature while keeping the logic behind the data unchanged.

• Step 3: Dimensional reduction of features

Now, retrieved decomposed values of the input image are passed through the equations which are given below, to obtain reduced dimensionality features:

$$w_{\left(1\right)}=\arg \underset{\left|\left|w\right|\right|=1}{\max }\left\{\left.{{\displaystyle \sum }}_i{\left(t1\right)}_{\left(i\right)}^2\right\}\right.=\arg \underset{\left|\left|w\right|\right|=1}{\max }\left\{{{\displaystyle \sum }}_i{\left(x_{\left(i\right)}.w\right)}^2\right\}$$

(13)

$$w_{\left(1\right)}=\arg \underset{\left|\left|w\right|\right|=1}{\max }\left\{\left.{\left|\left|Xw\right|\right|}^2\right\}\right.=\arg \underset{\left|\left|w\right|\right|=1}{\max }\left\{w^T{X}^{T}Xw\right\}$$

(14)

$$w_{\left(1\right)}=\arg \max \left\{\frac{w^T{X}^{T}Xw}{W^{T}W}\right\}$$

(15)

Equation (13) is there to obtain the maximum matching of the features with most suitable mathematical arguments. Equations (14) and (15) give the feature and find the ith component of the feature, which can be found by subtracting the first i-1 components,

$${\widehat{X}}_i=\mathrm{X}-\sum _{s=1}^{i-1}X{w}_{\left(i\right)}{w}_{\left(i\right)}^T$$

(16)

$$w_{\left(i\right)}=\arg \underset{\left|\left|w\right|\right|=1}{\max }\left\{{\left|\left|{\widehat{X}}_{k}w\right|\right|}^2\right\}=\arg \max \left\{\frac{w^T{\widehat{ X}}_i^T{\widehat{ X}}_i w}{w^{T }w}\right\}$$

(17)

To calculate the covariance of the feature set, the following equation is used:

$$\begin{array}{l}Q\left(P{C}_{\left(j\right) ,}P{C}_{\left(k\right)}\right)\propto {\left(X{w}_{\left(j\right)}\right)}^T\left(X{w}_{\left(k\right)}\right)\\ \mathrm{where} w_{\left(j\right)}^T{X}^{T}X{w}_{\left(k\right)}=w_{\left(j\right)}^T{\lambda }_{\left(k\right)}{w}_{\left(k\right)}={\lambda }_{\left(k\right)}{w}_{\left(j\right)}^T{w}_{\left(k\right)}\end{array}$$

(18)

Now Equation (17) is delivered as input to the neural network. Then, features represented by Equation (18) further feed to the neural network to obtain a classification. All these equations contribute to the one central idea, which is to find the logic between all the features and reduce the dimensionality of that. This leads to the lossless data compression.

• Step 4: Pass the features to Convolutional Neural Network

Step 4: Pass the features to Convolutional Neural Network

Neural network is summation of all the well-connected neurons along with their weights. The neural network with best suitable weight is called a trained neural network. Here, we consider x as input to the neural network layer, y as output of that layer and w is the weights of the connected neurons.

For all the ${\mathrm{x}}_{\mathrm{ij}}^{\ell }$ expressions in which ${\mathsf{\omega }}_{\mathrm{ab}}$ occurs,

$$\begin{array}{c}\hfill \frac{\partial \mathrm{E}}{\partial {\mathsf{\omega }}_{\mathrm{a}\mathrm{b}}}=\sum _{\mathrm{i}=0}^{\mathrm{N}-\mathrm{m}}\sum _{\mathrm{j}=0}^{\mathrm{N}-\mathrm{m}}\frac{\partial \mathrm{E}}{\partial {\mathrm{x}}_{\mathrm{i}\mathrm{j}}^{\ell }}\frac{\partial {\mathrm{x}}_{\mathrm{i}\mathrm{j}}^{\ell }}{\partial {\mathsf{\omega }}_{\mathrm{a}\mathrm{b}}}=\sum _{\mathrm{i}=0}^{\mathrm{N}-\mathrm{m}}\sum _{\mathrm{j}=0}^{\mathrm{N}-\mathrm{m}}\frac{\partial \mathrm{E}}{\partial {\mathrm{x}}_{\mathrm{i}\mathrm{j}}^{\ell }}{\mathrm{y}}_{(\mathrm{i}+\mathrm{a})(\mathrm{j}+\mathrm{b})}^{\ell -1}\end{array}$$

(19)

Here $\frac{\partial {\mathrm{x}}_{\mathrm{ij}}^{\ell }}{\partial {\mathsf{\omega }}_{\mathrm{ab}}}={\mathrm{y}}_{\left(\mathrm{i}+\mathrm{a}\right)\left(\mathrm{j}+\mathrm{b}\right)}^{\ell -1}$, which is used as a forward propagation equation.

To compute the gradient, it is necessary to obtain the values $\frac{\partial \mathrm{E}}{\partial {\mathrm{x}}_{\mathrm{ij}}^{\ell }},$ where the chain rule can be given as,

$$\frac{\partial \mathrm{E}}{\partial {\mathrm{x}}_{\mathrm{ij}}^{\ell }}=\frac{\partial \mathrm{E}}{\partial {\mathrm{y}}_{\mathrm{ij}}^{\ell }}\frac{\partial {\mathrm{y}}_{\mathrm{ij}}^{\ell }}{\partial {\mathrm{x}}_{\mathrm{ij}}^{\ell }}=\frac{\partial \mathrm{E}}{\partial {\mathrm{y}}_{\mathrm{ij}}^{\ell }}\frac{\partial }{\partial {\mathrm{x}}_{\mathrm{ij}}^{\ell }}\left(\mathsf{\sigma }\left({\mathrm{x}}_{\mathrm{ij}}^{\ell }\right)\right)=\frac{\partial \mathrm{E}}{\partial {\mathrm{y}}_{\mathrm{ij}}^{\ell }} {\mathsf{\sigma }}^{\prime }\left({\mathrm{x}}_{\mathrm{ij}}^{\ell }\right)$$

(20)

Therefore, the error at the current layer $\frac{\partial \mathrm{E}}{\partial {\mathrm{y}}_{\mathrm{ij}}^{\ell }}$ can be simply calculated by using deltas $\frac{\partial \mathrm{E}}{\partial {\mathrm{x}}_{\mathrm{ij}}^{\ell }}$ at the current layer by using the derivative of the activation function, ${\mathsf{\sigma }}^{\prime }\left(\mathrm{x}\right)$.

Due to the known errors at the current layer, it is now easy to compute the gradient with respect to the weights employed by this convolutional layer. Then it is necessary to transmit errors back to the previous layer in addition to computing the weights for the current convolutional layer.

$$\begin{array}{c}\hfill \frac{\partial E}{\partial y_{ij}^{\ell -1}}=\sum _{a=0}^{m-1}\sum _{b=0}^{m-1}\frac{\partial E}{\partial x_{(i-a)(j-b)}^{\ell }}\frac{\partial x_{(i-a)(j-b)}^{\ell }}{\partial y_{ij}^{\ell -1}}=\sum _{a=0}^{m-1}\sum _{b=0}^{m-1}\frac{\partial E}{\partial x_{(i-a)(j-b)}^{\ell }}{\omega }_{ab}\end{array}$$

(21)

This forward and backward moment gives rise to the improvement in the weights in such a way that the neural network will have maximum accuracy of classification. Then pooling operations are applied on the convolved feature set.

• Step 5: Flattening of the pooled features

Next, the procedure involves flattening. The generated 2-dimensional arrays from pooled feature maps are flattened into a single long continuous linear vector. The fully connected layer uses the flattened matrix as an input for noise identification.

• Step 6: Noise classification

The labeled classes are speckle noise, Poisson noise, Gaussian noise and impulse value noise. These four classes are trained and tested using CNN on the basis of extracted features. Then the detected image is noisy or not.

5. Results and Discussion

The results of the proposed system, ‘noise type detection of ancient images with features extraction and neural networks’, are presented in this section. The experiment is performed using TensorFlow and Keras for the neural network. The hardware environment in this experiment consists of 11th Gen Intel(R)Core (TM) i7-1165G7 @ 2.80GHzmemory 16GB, 64-bit operating system. The software environment includes freeware Microsoft visual studio code editor version 1.70.1, used with Python 3.6.3 for coding. OpenCV is used for image processing. The ancient mural images used in this paper are sourced from Indian Heritage Image Retrieval Dataset. It contains images from various heritage sites in Andhra Pradesh and Karnataka which describe various historical and mythological events. Image resolution is 10 M pixels and 18 M pixels, whose sizes range from 300 KB to 15 MB. Sample noisy images and non-noisy images are shown in Figure 3.

The metric employed for the analysis to improve effectiveness of the model is accuracy, precision, recall, and F1-score. The wavelet transforms of level 3 and db 5 are selected based on the accuracy of the classifier. The results of classification based on different levels are as tabulated in

Table 3. Accuracy of the PSA on the bases of feature extraction.

Wavelet Level	db Value	Accuracy (%)
2	1	89.23
2	2	92.34
2	3	95.23
2	4	93.12
2	5	92.59
3	1	94.93
3	2	92.34
3	3	95.23
3	4	97.23
3	5	98.93
4	1	96.34
4	2	95.23
4	3	97.52
4	4	98.29
4	5	97.42

.

Once the algorithm for the feature extraction is finalized, the performance parameters are calculated to check and compare PSA with other existing architectures.

The performance of algorithms has been measured by a number of metrics, each focusing on a different aspect. Appropriate metrics need to be applied to evaluate the performance of each machine learning problem. To determine the performance of algorithms and to make a comparison, here common metrics are used for classification problems which are precision, recall [47], f1-score [48], accuracy and confusion matrix [49]. These metrics are commonly used in many classification applications [50,51] for performance evaluation.

To find out the performance parameters, the confusion matrix is first calculated and then accuracy, precision, recall and F1 score is calculated, which is represented by Equations (22)–(25).

$$\mathrm{Accuracy}=\frac{\mathrm{TN}+\mathrm{TP}}{\mathrm{TN}+\mathrm{TP}+\mathrm{FN}+\mathrm{FP}}$$

(22)

$$\mathrm{Precision}=\frac{\mathrm{TP}}{\mathrm{FP}+\mathrm{TP}}$$

(23)

$$\mathrm{Recall}=\frac{\mathrm{TP}}{\mathrm{FN}+\mathrm{TP}}$$

(24)

$$\mathrm{F}1-\mathrm{Score}=\frac{2\mathrm{TP}}{2\mathrm{TP}+\mathrm{FN}+\mathrm{FP}}$$

(25)

where TP correctly predicts positive and is true, TN correctly predicts the negative and is true. Where FP predicts positive and it is false, and FN predicts negative and it is false.

From the table it is demonstrated that 89.23% is the minimum accuracy when the features are extracted by using WT of level 2 and db 1 and the maximum accuracy is there in case of the WT of level 3 and db 5. Different configurations of the WT are tried and tested with proposed neural network. The WT of level 3 and db 5 gives the maximum accuracy, as shown in Figure 4. Here, level 3 with db 5 is selected for the feature extraction.

After the consideration of the result of the classifications, the WT of level 3 and db 5 is selected. Then, using the same WT, the system is tested on four different classes (Figure 5).

The performance parameters of the proposed system are tabulated in

Table 4. The proposed system’s performance parameters.

Class Name	Accuracy	Precision	Recall	F1-Score
Poisson noise	98.75%	97%	98%	98%
Speckle noise	99.50%	99%	99%	100%
Gaussian noise	99%	98%	98%	97%
Impulse value noise	99.75%	100%	99%	99%

and its comparative graph is shown in Figure 6. The overall accuracy of the proposed system is 98.93%.

5.1. Comparative Methods

The effectiveness of the PSA model is demonstrated through the comparative analysis with different models like AlexNet, Yolo V5, Yolo V3, RCNN and CNN.

Table 5. Performance comparison of proposed system with other models.

Algorithm	Accuracy	Precision	Recall	F1-Score
PSA	99.25%	98.50%	98.50%	98.50%
AlexNet	97.30%	96.60%	95.70%	97.93%
Yolo V5	94.09%	94.58%	93.10%	94%
Yolo V3	92.24%	90.88%	91.20%	93.17%
RCNN	90.03%	92.08%	90.39%	92.13%
CNN	89.39%	89.53%	88.62%	88.82%

gives the results of PSA comparison with other models and visualization shown in Figure 7.

The comparative analysis of PSA model is based on performance metrics such as accuracy, F1-score, recall and precision. These performance parameters of the proposed system show better performance against the abovementioned models.

5.2. Comparative Analysis

A different number of hidden layers are checked for accuracy. When nine hidden layers are used, classification accuracy is observed to be the highest. The accuracy along with the number of hidden layers is given in

Table 6. Performance parameters of the proposed system based on number of hidden layers.

Number of Hidden Layers	Accuracy
1	86.23
2	87.43
3	89.56
4	90.88
5	92.66
6	94.09
7	95.69
8	97.29
9	98.93
10	97.29
11	96.23

. This shows that, as the number of hidden layers increases, accuracy increases gradually. However, once the number of hidden layers reaches the 9th layer, the accuracy starts decreasing.

Figure 8 is for the diagrammatical representation of noise classification using PSA. It represents the cluster of class and the target points labeled as noise type not detected, correctly detected and incorrectly detected.

5.3. Comparative Discussion

Comparative analysis of PSA models based on accuracy, precision, recall and F1-score is shown in Figure 9a. For every performance parameter the proposed system is superior to the AlexNet, Yolo V5, Yolo V3, RCNN and CNN, as shown in Figure 9b–f. These performance parameters of all the models are tabulated in

Table 7. Performance parameters of the proposed PSA, AlexNet, Yolo V5, Yolo V3, RCNN and CNN.

Algorithm	Class Name	Accuracy	Precision	Recall	F1-Score
PSA	Gaussian noise	99.00%	98.00%	98.00%	98.00%
	Impulse value noise	99.75%	100%	99.00%	100%
	Poisson noise	98.75%	97.00%	98.00%	97.00%
	Speckle noise	99.50%	99.00%	99.00%	99.00%
AlexNet	Gaussian noise	98.62%	97.13%	96.42%	98.62%
	Impulse value noise	97.41%	96.30%	95.23%	98.17%
	Poisson noise	97.32%	95.14%	93.62%	96.42%
	Speckle noise	96.21%	97.83%	97.53%	98.50%
Yolo V5	Gaussian noise	96.43%	94.72%	93.84%	94.24%
	Impulse value noise	95.72%	94.28%	93.63%	95.24%
	Poisson noise	92.52%	96.46%	91.30%	92.88%
	Speckle noise	91.69%	92.84%	93.61%	93.62%
Yolo V3	Gaussian noise	95.24%	92.43%	94.20%	93.53%
	Impulse value noise	89.63%	90.75%	91.64%	94.45%
	Poisson noise	92.48%	91.30%	87.53%	92.12%
	Speckle noise	91.59%	89.03%	91.41%	92.60%
RCNN	Gaussian noise	86.43%	89.35%	87.53%	90.70%
	Impulse value noise	89.53%	92.54%	90.51%	91.73%
	Poisson noise	93.15%	94.02%	92.09%	92.56%
	Speckle noise	91.00%	92.42%	91.42%	93.51%
CNN	Gaussian noise	87.35%	86.25%	84.62%	85.73%
	Impulse value noise	88.53%	87.53%	86.83%	87.62%
	Poisson noise	95.42%	96.42%	96.00%	96.19%
	Speckle noise	86.24%	87.92%	87.03%	85.72%

.

PSA utilizes only 12 kb memory for 50 kb image size and 0.0001 seconds as processing time. Similarly, time and space complexity for different image sizes of PSA is shown in

Table 8. Noise classification time and space complexity of PSA for different sizes of image.

	PSA
Image Size	Time (Seconds)	Required Memory for Processing (kb)
50 kb	0.0001	12
100 kb	0.0001	16
200 kb	0.000294	25
500 kb	0.000784	39
750 kb	0.001862	48
1 Mb	0.1274	74
5 Mb	1.0388	91
10 Mb	2.764	128
15 Mb	2.9543	381

.

On the different configurations, the time complexity is also evaluated. The required time for different hardware platforms to provide results is tabulated in

Table 9. Time required using PSA on different configurations.

Configuration
CPU/GPU	Processor	RAM	Required Time in Seconds
CPU	i3	8GB	0.393
CPU	i5	8GB	0.292
CPU	i7	8GB	0.286
GPU	Nvidia K80	24 GB	0.003

.

CPUs such as the i5 and i7 have similar time complexity, but GPUs require less time than CPUs. Accordingly, the processor comparison time requirement is different and GPU gives a better performance.

6. Conclusions

For restoring the ancient images, it is necessary to first identify the type of noise. The proposed study indicates optimized results for the noise identification and noise type classification of the noise present in the image. The PSA was tested against various parameters for reliability and accuracy. The time complexity of the system was compared against different hardware configurations. Additionally, the proposed system was compared with the existing AlexNet, Yolo V5, Yolo V3, RCNN and CNN models. The PSA accuracy was 99.25%, which is significantly better than the previous systems. Several noises can coexist at the same time, which can be classified in future work.