# Gaussian Mixture with Max Expectation Guide for Stacked Architecture of Denoising Autoencoder and DRBM for Medical Chest Scans and Disease Identification

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Literature Review

## 3. Background

#### 3.1. Expectation-Maximization Guided Gaussian Mixture Segmentation (EMGMM)

_{1}, i

_{2}, …, i

_{n}} is a stochastic random variable; w

_{j =}j = 1, …, q are the corresponding weights satisfying $\sum}_{\mathrm{j}\text{}=1}^{\mathrm{q}}{\mathrm{w}}_{\mathrm{j}}=1$, $\mathrm{n}\left(\mathrm{I}/{\varnothing}_{\mathrm{j}},\text{}{{{\displaystyle \sum}}^{\text{}}}_{\mathrm{j}}\right)$; and j = 1, …, q are the Gaussian densities; and (b) ${\varnothing}_{\mathrm{j}}$ is the mean vector while ${{{\displaystyle \sum}}^{\text{}}}_{\mathrm{j}}$ is the covariance matrix of the jth Gaussian. The GMMs maximize the probability of the parameters (e.g., means, covariance, and mixing coefficients). The multi-variant Gaussian mixture distribution can be defined by Equation (1):

- E-Step: suppose the parameter value is ${\varnothing}_{\mathrm{j}}$, calculate $\sum}_{\mathrm{j}\text{}=1}^{\mathrm{k}}{\mathrm{w}}_{\mathrm{j}}=1\text{}\mathrm{for$ all data points Ij, 1 ≤ j ≤ n and all mixture components 1 ≤ k ≤ K, which yields an n × K membership weights matrix.
- M-Step: in this step, the maximization of the parameters is reached based on the previous matrix. Let n = count of the membership weights, such that $\sum}_{\mathrm{j}\text{}=1}^{\mathrm{k}}{\mathrm{w}}_{\mathrm{j}}=1$ is the sum of the membership weights for the kth parameter; as seen in Equation (2), the values of weights and means are then updated respectively:$${\mathsf{\mu}}_{\mathrm{k}}^{\mathrm{updated}}=\frac{{\mathrm{n}}_{\mathrm{k}}}{\mathrm{n}},\text{}1\le \mathrm{k}\le \mathrm{K},$$$${\mathsf{\mu}}_{\mathrm{k}}^{\mathrm{updated}}=\left(\frac{1}{{\mathrm{n}}_{\mathrm{k}}}\right){\displaystyle \sum}_{\mathrm{k}\text{}=1}^{\mathrm{K}}{\mathrm{w}}_{\mathrm{jk}}{\mathrm{I}}_{\mathrm{j}},\text{}1\le \mathrm{k}\le \mathrm{K},$$$${\mathsf{\mu}}_{\mathrm{k}}^{\mathrm{updated}}=\left(\frac{1}{{n}_{\mathrm{k}}}\right){\displaystyle \sum}_{\mathrm{k}\text{}=1}^{\mathrm{K}}{\mathrm{w}}_{\mathrm{jk}}\left({\text{}\mathrm{I}}_{\mathrm{j}}-{\varnothing}_{\mathrm{k}}^{\mathrm{updated}}{}_{}\right){\left({\text{}\mathrm{I}}_{\mathrm{j}}-{\varnothing}_{\mathrm{k}}^{\mathrm{updated}}{}_{}\right)}^{\mathrm{T}},\text{}1\le \mathrm{k}\le \mathrm{K}.$$

#### 3.2. Autoencoder (AE)

_{d}), it would work as follows:

_{d}) by using a deterministic function h

_{l}= f

_{l}(i) = α(W

_{li}+ b

_{l}), where α is the hyperbolic tangent activation function (W

_{l}= weight matrix and b

_{l}= bias term for encoder).

_{l}(h) = α (W’

_{l}h + b’

_{l}) with l’ = lW’b (W’ = weight matrix and b’ = bias for the decoder).

_{j}= α (i*W

^{j}+ b

^{j})

_{h}and d

_{o}are the differences in the hidden and output layers, respectively. A stochastic gradient descent in Equation (6) modifies the weights, and the class label o″ is estimated by Equation (7).

#### 3.3. Deep Restricted Boltzmann Machine (DRBM)

_{1}, sv

_{2}… sv

_{i}) and arbitrary hidden variables rh = {rh

_{1}, rh

_{2}, rh

_{J}}. It is bidirectional, between sv and rh, where both pivot on the major features. This constrains direct neurons to being allied to the bipartite model. Mathematically, the RBM is a probabilistic energy algorithm that aims to reach a probability distribution (pd) association graph of sv and rh as shown in Equations (8)–(10).

_{ij}are weights sv

_{i}→rh

_{j}; (a

_{i}, b

_{j}) are biases for visible and hidden variables; and σ

_{i}is the standard deviation for the Gaussian noise.

_{1}is added to the one hot vector of labels, and its back propagating error constructs an optimal regression layer to classify the scans to either class C

_{1}(pneumonia) or class C

_{2}(normal).

## 4. Architecture and Development of the Proposed Model

## 5. Case Study and Experimental Results

#### 5.1. Data Preprocessing and Augmentation

#### 5.2. Feature Visualization and Model Results

#### 5.3. Adding DRBM Classifier

#### 5.4. Evaluation and Analysis of Results

_{n}: correct classification of normal), False Positive (F

_{P}: the misclassified existence of pneumonia), and False Negative (F

_{n}: the misclassified of normal cases) were computed for the model. Next, sensitivity (S

_{e}), specificity (S

_{p}), positive predictive value (PP

_{V}) and classification accuracy (ACC) were calculated to provide deeper insight into the performance of the classifier [see Equations (12)–(15)].

## 6. Conclusions and Future Work

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## References

- Lakhani, P.; Sundaram, B. Deep learning at chest radiography: Automated classification of pulmonary tuberculosis by using convolutional neural networks. Radiology
**2017**, 284, 574–582. [Google Scholar] [CrossRef] - Krishnamurthy, S.; Srinivasan, K.; Qaisar, S.M.; Vincent, P.D.R.; Chang, C.Y. Evaluating Deep Neural Network Architectures with Transfer Learning for Pneumonitis Diagnosis. Comput. Math Methods Med.
**2021**, 8036304. [Google Scholar] [CrossRef] - Siddiqi, R. Automated pneumonia diagnosis using a customized sequential convolutional neural network. In Proceedings of the 2019 3rd International Conference on Deep Learning Technologies, Xiamen, China, 5–7 July 2019; pp. 64–70. [Google Scholar] [CrossRef]
- Račić, L.; Popović, T.; Čakić, S.; Šandi, S. Pneumonia Detection Using Deep Learning Based on Convolutional Neural Network. In Proceedings of the 25th International Conference on Information Technology (IT), Zabljak, Montenegro, 16–21 February 2021; pp. 1–4. [Google Scholar] [CrossRef]
- Stephan, E.; Schlagenhaut, F.; Huys, Q.; Raman, S.; Aponte, E.; Brodersen, K.; Rigoux, L.; Moran, R.; Daunizeau, J.; Dolan, R.; et al. Computational neuroimaging strategies for single patient predictions. Neuroimage
**2017**, 145, 180–199. [Google Scholar] [CrossRef] [Green Version] - Schmidhuber, J. Deep learning in neural networks: An overview. Neural Netw.
**2015**, 61, 85–117. [Google Scholar] [CrossRef] [Green Version] - Krizhevsky, A.; Sutskever, I.; Hinton, G. ImageNet classification with deep convolutional neural networks. In Proceedings of the Advances in Neural Information Processing Systems; MIT Press: Cambridge, MA, USA, 2012; pp. 1106–1114. [Google Scholar]
- Cai, Y.; Landis, M.; Laidley, D.; Kornecki, A.; Lum, A.; Li, S. Multi-modal vertebrae recognition using transformed deep convolution network. Comput. Med. Imaging Graph.
**2016**, 51, 11–19. [Google Scholar] [CrossRef] - Xu, J.; Xiang, L.; Liu, Q.; Gilmore, H.; Wu, J.; Tang, J. Stacked sparse autoencoder (SSAE) for nuclei detection on breast cancer histopathology images. IEEE Trans. Med. Imaging
**2016**, 35, 119–130. [Google Scholar] [CrossRef] [Green Version] - Tsanev, G.S. Deep Multiconnected Boltzmann Machine for Classification. Am. J. Eng. Res.
**2017**, 6, 186–194. [Google Scholar] - Salakhutdinov, R.; Hinton, G. An efficient learning procedure for deep Boltzmann machines. Neural Comput.
**2012**, 24, 1967–2006. [Google Scholar] [CrossRef] [Green Version] - Larochelle, H.; Bengio, Y. Classification using discriminative restricted boltzmann machines. In Proceedings of the ACM International Conference on Machine Learning, Helsinki, Finland, 5–9 July 2008; pp. 536–543. [Google Scholar]
- Salakhutdinov, R.; Hinton, G. Deep Boltzmann machines. In Artificial Intelligence and Statistics; PMLR: Fort Lauderdale, FL, USA, 2009; pp. 448–455. [Google Scholar]
- Sankaran, A.; Goswami, G.; Vatsa, M.; Singh, R.; Majumdar, A. Class sparsity signature based restricted boltzmann machine. Pattern Recognit.
**2017**, 61, 674–685. [Google Scholar] [CrossRef] - Li, D.; Fu, Z.; Xu, J. Stacked-autoencoder-based model for COVID-19 diagnosis on CT images. Int. J. Res. Intell. Syst. Real Life Complex Probl.
**2021**, 51, 2805–2817. [Google Scholar] [CrossRef] - Vincent, P.; Larochelle, H.; Bengio, Y.; Manzagol, P. Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th International Conference on Machine Learning, Helsinki, Finland, 5–9 July 2008; pp. 1096–1103. [Google Scholar]
- Gomes, J.; Masood, A.; da Silva, L.H.S.; da Cruz Ferreira, J.R.B.; Freire Junior, A.A.; dos Santos Rocha, A.L. COVID-19 diagnosis by combining RTPCR and pseudo-convolutional machines to characterize virus sequences. Sci. Rep.
**2021**, 11, 11545. [Google Scholar] [CrossRef] [PubMed] - Iwendi, C.; Mahboob, K.; Khalid, Z.; Javed, A.; Rizwan, M.; Ghosh, U. Classification of COVID-19 individuals using adaptive neuro-fuzzy inference system. Multimed. Syst.
**2022**, 28, 1223–1237. [Google Scholar] [CrossRef] [PubMed] - Yee, S.; Raymond, W. Pneumonia Diagnosis Using Chest X-ray Images and Machine Learning. In Proceedings of the 10th International Conference on Biomedical Engineering and Technology (ICBET 20), Tokyo, Japan, 15–18 September 2020; pp. 101–105. [Google Scholar]
- Hashmi, M.F.; Katiyar, S.; Keskar, A.G.; Bokde, N.D.; Geem, Z.W. Efficient Pneumonia Detection in Chest Xray Images Using Deep Transfer Learning. Diagnostics
**2020**, 10, 417. [Google Scholar] [CrossRef] - Chouhan, V.; Singh, S.K.; Khamparia, A.; Gupta, D.; Tiwari, P.; Moreira, C.; Damaševičius, R.; Alburquerque, V.H. A Novel Transfer Learning Based Approach for Pneumonia Detection in Chest X-ray Images. Appl. Sci.
**2020**, 10, 559. [Google Scholar] [CrossRef] [Green Version] - Mahmud, T.; Rahman, M.; Fattah, S. CovXNet: A multi-dilation convolutional neural network for automatic COVID-19 and other pneumonia detection from chest X-ray images with transferable multi-receptive feature optimization. Comput. Biol. Med.
**2020**, 1, 122. [Google Scholar] [CrossRef] - Abiyev, R.H.; Ma’aitah, M.K.S. Deep convolutional neural networks for chest diseases detection. J. Healthc. Eng.
**2018**, 2018, 4168538. [Google Scholar] [CrossRef] [Green Version] - Stephen, O.; Sain, M.; Maduh, U.J.; Jeong, D. An efficient deep learning approach to pneumonia classification in healthcare. J. Healthc. Eng.
**2019**, 2019, 1–7. [Google Scholar] [CrossRef] [Green Version] - Kundu, R.; Das, R.; Geem, Z.W.; Han, G.-T.; Sarkar, R. Pneumonia detection in chest X-ray images using an ensemble of deep learning models. PLoS ONE
**2021**, 16, e0256630. [Google Scholar] [CrossRef] - Yao, J.C.; Wang, T.; Hou, G.H.; Ou, D.; Li, W.; Zhu, Q.D.; Chen, W.C.; Yang, C.; Wang, L.J.; Wang, L.P.; et al. AI detection of mild COVID-19 pneumonia from chest CT scans. Eur. Radiol.
**2021**, 31, 7192–7201. [Google Scholar] [CrossRef] - Wang, C.; Elazab, A.; Jia, F.; Wu, J.; Hu, Q. Automated chest screening based on a hybrid model of transfer learning and convolutional sparse denoising autoencoder. Biomed. Eng.
**2018**, 17, 63. [Google Scholar] [CrossRef] - Dhahri, H.; Rabhi, B.; Chelbi, S.; Almutiry, O.; Mahmood, A.; Alimi, A.M. Automatic Detection of COVID-19 Using a Stacked Denoising Convolutional Autoencoder. CMC-Comput. Mater. Contin.
**2021**, 69, 3259–3274. [Google Scholar] [CrossRef] - Farnoosh, R.; Zarpak, B. Image segmentation using Gaussian mixture model. IUST Int. J. Eng. Sci.
**2008**, 19, 29–32. [Google Scholar] - Hui, B.; Guanyu, Y.; Huazhong, S.; Dillenseger, J. Accurate Image Segmentation Using Gaussian Mixture Model with Saliency Map. In Pattern Analysis and Applications; Springer: Berlin/Heidelberg, Germany, 2018; Volume 21, pp. 869–878. [Google Scholar]
- Goodfellow, I.; Bengio, Y.; Courville, A. Deep Learning; MIT Press: Cambridge, MA, USA, 2016. [Google Scholar]
- Masci, J.; Meier, U.; Cireşan, D.; Schmidhuber, J. Stacked convolutional autoencoders for hierarchical feature extraction. In Artificial Neural Networks and Machine Learning. ICANN 2011; Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2011; Volume 6791, pp. 52–59. [Google Scholar]
- Vincent, P.; Larochelle, H.; Lajoie, I.; Bengio, Y.; Manzagol, P.A. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res.
**2010**, 11, 3371–3408. [Google Scholar] - Kaggle, P.M. Kaggle’s Chest X-ray Images (Pneumonia) Dataset. 2020. Available online: https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia (accessed on 30 January 2022).
- Acharya, A.; Satapathy, R. A Deep Learning Based Approach towards the Automatic Diagnosis of Pneumonia from Chest Radio-Graphs. Biomed. Pharmacol. J.
**2020**, 13, 449–455. [Google Scholar] [CrossRef] - Tobias, R.; De Jesus, L.C.M.; Mital, M.; Lauguico, S.; Guillermo, M.; Sybingco, E.; Bandala, A.; Dadios, E. CNN-based Deep Learning Model for Chest X-ray Health Classification Using TensorFlow. In Proceedings of the International Conference on Computing and Communication Technologies, Ho Chi Minh City, Vietnam, 14–15 October 2020. [Google Scholar]
- Emhamed, R.; Mamlook, A.; Chen, S. Investigation of the performance of Machine Learning Classifiers for Pneumonia Detection in Chest X-ray Images. In Proceedings of the IEEE International Conference on Electro Information Technology (EIT’ 20), Chicago, IL, USA, 31 July–1 August 2020; pp. 98–104. [Google Scholar]
- Rahman, T.; Chowdhury, M.; Islam, K.; Islam, K.; Mahbub, Z.; Kadir, M.; Kashem, S. Transfer Learning with Deep Convolutional Neural Network (CNN) for Pneumonia Detection Using Chest X-ray. Appl. Sci.
**2020**, 10, 3233. [Google Scholar] [CrossRef]

**Figure 1.**Illustration of the proposed EMGMM-AEDRBM model for a two class classification problem: “Visible layer” for the input layer and “h” for the hidden layer.

**Figure 3.**Presentation of convolution filters or kernels with different stride length and max pooling padding.

**Figure 7.**Visualization of two original scan samples, their corresponding denoised results from the convolutional DEA with noise factor 0.01, and their reconstructed mapped image.

**Figure 8.**The applied 3 × 3 filter and the corresponding 32-item set of extracted features for normal and pneumonia samples, in left and right part of the figure, respectively.

**Figure 9.**Effect of data augmentation on enhancing the proposed model classification’s loss and validation accuracies.

Before Preprocessing | Generated Cases | Updated | After Preprocessing | |||||
---|---|---|---|---|---|---|---|---|

Normal | Train | Test | Validate | 6332 | 7915 | Train | Test | Validate |

1341 | 234 | 8 | 5540 | 1978 | 397 | |||

Total = 1583 | ||||||||

Pneumonia | 3875 | 390 | 8 | 3205 | 7478 | 5235 | 1870 | 373 |

Total = 4273 |

Layers | Filter Size | Stride | Output Dimension |
---|---|---|---|

Conv2D | 3 × 3 | 2 | 224 × 224 × 128 |

Batch Normalization | - | - | 224 × 224 × 128 |

Conv2D | 3 × 3 | 1 | 224 × 224 × 64 |

MaxPooling | 2 × 2 | 2 | 112 × 112 × 64 |

Con2D | 3 × 3 | 1 | 112 × 112 × 32 |

Batch Normalization | - | - | 112 × 112 × 32 |

MaxPooling | 2 × 2 | 2 | 56 × 56 × 32 |

Author | Learning Technique | Features Method | Accuracy |
---|---|---|---|

[35] | Deep Siamese based neural network | CNN | 89.6% |

[36] | CNN | CNN | 96.65% |

[37] | CNN | Multiple features | 98.46% |

Rain forest | 97.6% | ||

KNN | 92.5% | ||

Adaboost | 95.6% | ||

[38] | CNN + transfer learning (AlexNet) | CNN | 94.5% |

ResNet18 | 96.4% | ||

DenseNet201 | 98% | ||

SqueezeNet | 96.1% | ||

Proposed model | Hybrid of denoising autoencoder + DRBM | Denoising autoencoder | 98.63% |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Jamjoom, M.; Mahmoud, A.M.; Abbas, S.; Hodhod, R.
Gaussian Mixture with Max Expectation Guide for Stacked Architecture of Denoising Autoencoder and DRBM for Medical Chest Scans and Disease Identification. *Electronics* **2023**, *12*, 105.
https://doi.org/10.3390/electronics12010105

**AMA Style**

Jamjoom M, Mahmoud AM, Abbas S, Hodhod R.
Gaussian Mixture with Max Expectation Guide for Stacked Architecture of Denoising Autoencoder and DRBM for Medical Chest Scans and Disease Identification. *Electronics*. 2023; 12(1):105.
https://doi.org/10.3390/electronics12010105

**Chicago/Turabian Style**

Jamjoom, Mona, Abeer M. Mahmoud, Safia Abbas, and Rania Hodhod.
2023. "Gaussian Mixture with Max Expectation Guide for Stacked Architecture of Denoising Autoencoder and DRBM for Medical Chest Scans and Disease Identification" *Electronics* 12, no. 1: 105.
https://doi.org/10.3390/electronics12010105