# Stochastic Capsule Endoscopy Image Enhancement

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Background

#### 2.1. Edge-Aware Smoothing and Random Walks

#### 2.2. Retinex-Inspired Envelope with Stochastic Sampling

## 3. Stochastic Cve Image Enhancement

#### 3.1. Smoothing

#### 3.2. Local Contrast Enhancement

#### 3.3. Image Decomposition

## 4. Implementation

Algorithm A1: Stochastic CVE image enhancement |

Data: - $Ycbcr:$ Input image
- $R1,R2:$ Inner and outer sampling circles, Figure 1
- $N,M:$ Number of samples and total number of iterations
Result: Enhanced imagePROCEDURE:G = color gradient Equation (4) foreachpixeldo |

## 5. Result and Evaluation

#### 5.1. Experimental Setup and Procedure

#### 5.2. Dataset

#### 5.3. Subjective Evaluation

#### 5.4. Objective Evaluation and Comparison

#### 5.5. The Effect of Parameter Selection

#### 5.6. Base Layer Estimation

#### 5.7. Applicability the Proposed Method and Computational Cost

## 6. Conclusions and Future Work

## Author Contributions

## Acknowledgments

## Conflicts of Interest

## Abbreviations

CVE | Capsule video endoscopy |

CCE | Colon capsule endoscopy |

CLAHE | Contrast limited adaptive histogram equalization |

CMOS | Complementary metal-oxide semiconductor) |

DFT | Discrete Fourier transform |

FICE | Flexible spectral imaging color enhancement |

HVS | Human Visual System |

JND | Just noticeable difference |

NBI | Narrow Band Imaging |

SSIM | Structural similarity index |

WLF | Weighted-level framework |

WLS | weighted least square |

## References

- Rimbaş, M.; Negreanu, L.; Ciobanu, L.; Benguş, A.; Spada, C.; Băicuş, C.R.; Costamagna, G. Is virtual chromoendoscopy useful in the evaluation of subtle ulcerative small-bowel lesions detected by video capsule endoscopy? Endosc. Int. Open
**2015**, 3, E615. [Google Scholar] [CrossRef] [PubMed] - Sakai, E.; Endo, H.; Kato, S.; Matsuura, T.; Tomeno, W.; Taniguchi, L.; Uchiyama, T.; Hata, Y.; Yamada, E.; Ohkubo, H.; et al. Capsule endoscopy with flexible spectral imaging color enhancement reduces the bile pigment effect and improves the detectability of small bowel lesions. BMC Gastroenterol.
**2012**, 12, 83. [Google Scholar] [CrossRef] [PubMed] - Lambert, R.; Kuznetsov, K.; Rey, J. Narrow-band imaging in digestive endoscopy. Sci. World J.
**2007**, 7, 449–465. [Google Scholar] [CrossRef] [PubMed] - Ramaraj, M.; Raghavan, S.; Khan, W.A. Homomorphic filtering techniques for WCE image enhancement. In Proceedings of the 2013 Computational Intelligence and Computing Research (ICCIC), Enathi, India, 26–28 December 2013; pp. 1–5. [Google Scholar]
- Ibrahim, H.; Kong, N.S.P. Brightness preserving dynamic histogram equalization for image contrast enhancement. IEEE Trans. Consum. Electr.
**2007**, 53, 1752–1758. [Google Scholar] [CrossRef] - Okuhata, H.; Nakamura, H.; Hara, S.; Tsutsui, H.; Onoye, T. Application of the real-time Retinex image enhancement for endoscopic images. In Proceedings of the 35th Annual International Conference of the Engineering in Medicine and Biology Society (EMBC), Osaka, Japan, 3–7 July 2013; pp. 3407–3410. [Google Scholar]
- Provenzi, E. Computational Color Science: Variational Retinex-Like Methods; John Wiley & Sons: Hoboken, NJ, USA, 2017. [Google Scholar]
- Gopi, V.P.; Palanisamy, P.; Niwas, S.I. Capsule endoscopic colour image denoising using complex wavelet transform. In Wireless Networks and Computational Intelligence; Springer: Heidelberg/Berlin, Germany, 2012; pp. 220–229. [Google Scholar]
- Liu, H.; Lu, W.S.; Meng, M.Q.H. De-blurring wireless capsule endoscopy images by total variation minimization. In Proceedings of the 2011 IEEE Pacific Rim Conference on Communications, Computers and Signal Processing (PacRim), Victoria, BC, Canada, 23–26 August 2011; pp. 102–106. [Google Scholar]
- Beck, A.; Teboulle, M. Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems. IEEE Trans. Image Process.
**2009**, 18, 2419–2434. [Google Scholar] [CrossRef] [PubMed] - Vu, H.; Echigo, T.; Yagi, K.; Okazaki, H.; Fujiwara, Y.; Yagi, Y.; Arakawa, T. Image-Enhanced capsule endoscopy preserving the original color tones. In Proceedings of the International MICCAI Workshop on Computational and Clinical Challenges in Abdominal Imaging, Cambridge, MA, USA, 14 September 2011; Springer: Heidelberg/Berlin, Germany, 2011; pp. 35–43. [Google Scholar]
- Imtiaz, M.S.; Wahid, K.A. Color enhancement in endoscopic images using adaptive sigmoid function and space variant color reproduction. Comput. Math. Methods Med.
**2015**, 2015, 3905–3908. [Google Scholar] [CrossRef] [PubMed] - Li, B.; Meng, M.Q.H. Wireless capsule endoscopy images enhancement via adaptive contrast diffusion. J. Vis. Commun. Image Represent.
**2012**, 23, 222–228. [Google Scholar] [CrossRef] - Attar, A.; Xie, X.; Zhang, C.; Wang, Z.; Yue, S. Wireless Micro-Ball endoscopic image enhancement using histogram information. In Proceedings of the 2014 36th Annual International Conference of the Engineering in Medicine and Biology Society (EMBC), Chicago, IL, USA, 26–30 August 2014; pp. 3337–3340. [Google Scholar]
- Srivastava, R.; Gupta, J.; Parthasarthy, H.; Srivastava, S. PDE based unsharp masking, crispening and high boost filtering of digital images. In Proceedings of the International Conference on Contemporary Computing, Aachen, Germany, 19–21 August 2009; Springer: Heidelberg/Berlin, Germany, 2009; pp. 8–13. [Google Scholar]
- Sheet, D.; Garud, H.; Suveer, A.; Mahadevappa, M.; Chatterjee, J. Brightness preserving dynamic fuzzy histogram equalization. IEEE Trans. Consum. Electr.
**2010**, 56, 2475–2480. [Google Scholar] [CrossRef] - Rukundo, O.; Pedersen, M.; Hovde, Ø. Advanced Image Enhancement Method for Distant Vessels and Structures in Capsule Endoscopy. Comput. Math. Methods Med.
**2017**, 2017. [Google Scholar] [CrossRef] [PubMed] - Kolås, Ø.; Farup, I.; Rizzi, A. Spatio-temporal Retinex-inspired envelope with stochastic sampling: A framework for spatial color algorithms. J. Imag. Sci. Technol.
**2011**, 55, 40503-1–40503-10. [Google Scholar] - Black, M.J.; Sapiro, G.; Marimont, D.H.; Heeger, D. Robust anisotropic diffusion. IEEE Trans. Image Process.
**1998**, 7, 421–432. [Google Scholar] [CrossRef] [PubMed][Green Version] - Estrada, F.J.; Fleet, D.J.; Jepson, A.D. Stochastic Image Denoising. In Proceedings of the British Machine Vision Conference, London, UK, 7–10 September 2009; pp. 1–11. [Google Scholar]
- Azzabou, N.; Paragios, N.; Guichard, F. Random Walks, Constrained Multiple Hypothesis Testing and Image Enhancement; Springer: Heidelberg/Berlin, Germany, 2006; pp. 379–390. [Google Scholar]
- Subr, K.; Soler, C.; Durand, F. Edge-preserving multiscale image decomposition based on local extrema. ACM Trans. Graph.
**2009**, 28, 147. [Google Scholar] [CrossRef][Green Version] - Wang, Z.; Wang, H.; Sun, X.; Zheng, X. An Image Enhancement Method Based on Edge Preserving Random Walk Filter; Springer: Heidelberg/Berlin, Germany, 2015; pp. 433–442. [Google Scholar]
- Perona, P.; Malik, J. Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Patt. Anal. Mach. Intell.
**1990**, 12, 629–639. [Google Scholar] [CrossRef][Green Version] - Easley, G.R.; Labate, D.; Colonna, F. Shearlet-based total variation diffusion for denoising. IEEE Trans. Image Process.
**2009**, 18, 260–268. [Google Scholar] [CrossRef] [PubMed] - Shih, A.C.; Liao, H.Y.; Lu, C.S. A new iterated two-band diffusion equation: theory and its application. IEEE Trans. Image Process.
**2003**, 12, 466–476. [Google Scholar] [CrossRef] [PubMed] - Rajpoot, K.; Rajpoot, N.; Noble, J.A. Discrete Wavelet Diffusion for Image Denoising; Springer: Heidelberg/Berlin, Germany, 2008; pp. 20–28. [Google Scholar]
- Fattal, R.; Agrawala, M.; Rusinkiewicz, S. Multiscale shape and detail enhancement from multi-light image collections. ACM Trans. Graph.
**2007**, 26, 51. [Google Scholar] [CrossRef][Green Version] - Farbman, Z.; Fattal, R.; Lischinski, D.; Szeliski, R. Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM Trans. Graph.
**2008**, 27, 67. [Google Scholar] [CrossRef] - Holm, J.; Tastl, I.; Johnson, T. Definition & use of the ISO 12640-3 reference color gamut. In Proceedings of the Color and Imaging Conference Society for Imaging Science and Technology, Scottsdale, AZ, USA, 6–10 November 2006; Volume 2006, pp. 62–68. [Google Scholar]
- Koulaouzidis, A.; Iakovidis, D. KID: Towards a Reference Database for Intelligent Capsule Endoscopy. In Proceedings of the IEEE Engineering in Medicine and Biology Conference (EMBC), Milan, Italy, 25–29 August 2015; pp. 1–2. [Google Scholar]
- Montag, E.D. Empirical formula for creating error bars for the method of paired comparison. J. Electr. Imag.
**2006**, 15, 010502. [Google Scholar] [CrossRef][Green Version] - Simone, G.; Pedersen, M.; Hardeberg, J.Y. Measuring perceptual contrast in digital images. J. Vis. Commun. Image Represent.
**2012**, 23, 491–506. [Google Scholar] [CrossRef] - Wang, Z.; Bovik, A.C.; Sheikh, H.R.; Simoncelli, E.P. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process.
**2004**, 13, 600–612. [Google Scholar] [CrossRef] [PubMed] - Zhang, L.; Zhang, L.; Mou, X.; Zhang, D. FSIM: A feature similarity index for image quality assessment. IEEE Trans. Image Proces.
**2011**, 20, 2378–2386. [Google Scholar] [CrossRef] [PubMed] - Wang, Z.; Li, Q. Information content weighting for perceptual image quality assessment. IEEE Trans. Image Process.
**2011**, 20, 1185–1198. [Google Scholar] [CrossRef] [PubMed] - Zuiderveld, K. Contrast limited adaptive histogram equalization. In Graphics Gems; Academic Press Inc.: Cambridge, UK, 1994; pp. 474–485. [Google Scholar]
- Xu, L.; Lu, C.; Xu, Y.; Jia, J. Image smoothing via L 0 gradient minimization. ACM Trans. Graph.
**2011**, 30, 174. [Google Scholar] [CrossRef] - Tomasi, C.; Manduchi, R. Bilateral filtering for gray and color images. In Proceedings of the Sixth International Conference on Computer Vision, Bombay, India, 7 Januray 1998; pp. 839–846. [Google Scholar][Green Version]
- Shim, K.N.; Jeon, S.R.; Jang, H.J.; Kim, J.; Lim, Y.J.; Kim, K.O.; Song, H.J.; Lee, H.S.; Park, J.J.; Kim, J.H.; et al. Quality indicators for small bowel capsule endoscopy. Clin. Endosc.
**2017**, 50, 148. [Google Scholar] [CrossRef] [PubMed] - Imagawa, H.; Oka, S.; Tanaka, S.; Noda, I.; Higashiyama, M.; Sanomura, Y.; Shishido, T.; Yoshida, S.; Chayama, K. Improved detectability of small-bowel lesions via capsule endoscopy with computed virtual chromoendoscopy: a pilot study. Scand. J. Gastroenterol.
**2011**, 46, 1133–1137. [Google Scholar] [CrossRef] [PubMed] - Hatogai, K.; Hosoe, N.; Imaeda, H.; Rey, J.F.; Okada, S.; Ishibashi, Y.; Kimura, K.; Yoneno, K.; Usui, S.; Ida, Y.; et al. Role of enhanced visibility in evaluating polyposis syndromes using a newly developed contrast image capsule endoscope. Gut liver
**2012**, 6, 218. [Google Scholar] [CrossRef] [PubMed] - Xu, L.; Ren, J.; Yan, Q.; Liao, R.; Jia, J. Deep edge-aware filters. In Proceedings of the 32nd International Conference on Machine Learning, Lille, France, 6–11 July 2015; pp. 1669–1678. [Google Scholar]
- Wang, Z.; Wang, H. Image smoothing with generalized random walks: Algorithm and applications. Appl. Soft Comput.
**2016**, 46, 792–804. [Google Scholar] [CrossRef] - Grady, L. Random walks for image segmentation. IEEE Trans. Patt. Anal. Mach. Intell.
**2006**, 28, 1768–1783. [Google Scholar] [CrossRef] [PubMed]

**Figure 1.**Concentric sampling regions for the target pixel at the center to characterize local visual context. As shown in the right-hand image, once the random walk is beyond the inner circle random sampling is done, marked by the red dots. All of the samples are aggregated to form one chain.

**Figure 2.**(

**a**) Stochastic sampling: Random walks capture low variance neighbors that are essential for smoothing whilst random sampling captures high variance intensity variations that are essential for local contrast enhancement Therefore, random walks are used for smoothing whilst random samples weights are set to zero. (

**b**) Weight estimation of random walks along path starting from target pixel. Pixels inside the inner circle are assigned weights according to Equation (1).

**Figure 3.**(

**a**) Y channel of input image; (

**b**) shows the smoothed image using random walk. From the intensity variation across, row 165 of the image, in (

**c**) the smoothing preserves edges and smooths only detail variation on the surface. Hence captures the surface texture of the tissues.

**Figure 4.**(

**a**) Input image from PillCam COLON camera showing a Polyp of 9 mm size (

**b**) Enhanced image using proposed method, in which color tone is kept whilst the details are enhanced; Compared to the original it can be seen that lumen and shadow details of the surface texture are more visible on the enhanced image; (

**c**) Shows different layers of image decomposition ${D}_{1},{D}_{2},$ and D of Equation (10) as compared to the original and enhanced image Equation (11). As it is shown the two detail layers captures low contrast shadows and surface details.

**Figure 5.**Psychometric experimental setup for subjective evaluation of diagnostic quality of an image. The observer gives letters from A (best) to D (worst) quality image for diagnosis.

**Figure 7.**Average observers preference for the sample images selected for the subjective experiment.

**Figure 8.**Comparison of different methods on PillCam COLON images. (

**a**) Input image showing a splenic flexure (

**b**) It is visually easy to see that our framework enhances the local contrast and the details of the tissue surface simultaneously. On both images, the proposed method gives a consistent result under different illumination variation. (

**c**) Shows result from CLAHE and (

**d**)-(

**g**) are results from different image decomposition techniques. (

**e**) WLS [29], (

**f**) L0 Gradient Minimization [38], (

**g**) Local Extrema [39]. Bilateral [28] (

**d**) image decomposition technique enhances the details, but creates a halo effect on edges which appear to widen the blood vessels and other tissue surfaces. Moreover; (

**c**) enhances the local contrast but the details of the tissue surface are lost.

**Figure 9.**The effect of number of samples (N) and iteration (M). The input image is size of 174 × 294. R1 and R2 are set to 3 and 200 respectively with ${\sigma}_{I}=9$ and ${\sigma}_{g}=4$.

**Figure 10.**Parameter setting for ${\sigma}_{I}$ and ${\sigma}_{g}$. The top row shows the input setup. The first image is noise free image input image size $255\times 255$. A Gaussian noise of $\sigma =0.44$ is added to the image. We converted the image to color for better visualization. Smaller value of ${\sigma}_{I}$ and ${\sigma}_{g}$ gives smaller weight to neighboring pixels intensity and gradient respectively. With higher values of ${\sigma}_{I}$ and ${\sigma}_{g}$ the neighboring pixels have higher weights Equation (5) and could result in blurring the edges. Hence, any value of ${\sigma}_{I}>{\sigma}_{g}$ and ${\sigma}_{g}>5$ gives reasonable estimate of the base layer of the image.

**Figure 11.**Comparison of smoothing methods: (

**a**) Input image. (

**b**) A Gaussian noise of $\sigma =0.09$ is added on the input image (

**c**) Shows intensity variation across row 165 of noisy image and smoothed image using bilateral filtering and the proposed method. From (

**d**–

**f**) we can see that the proposed method gives a better edge-aware filtering compared to bilateral filtering. Moreover, it is easier to see that bilateral filtering creates a halo effect around edges.

**Table 1.**Average performance of different methods using WLF, SSIM, CIE2000, FSIM, IW-SSIM metrics respectively. SSIM, FSIM and IW-SSIM represent full reference and WLS representes no-reference image quality metric.

Average Performance of Enhancement Methods Based on Objective Quality Metrics | |||||
---|---|---|---|---|---|

Methods | WLF Ratio | SSIM | CIE2000 | FSIM | IW-SSIM |

Bilateral [28] | 1.06 | 0.92 | 3.03 | 0.95 | 0.93 |

WLS [29] | 1.28 | 0.92 | 3.29 | 0.94 | 0.92 |

CLAHE [37] | 2.65 | 0.82 | 3.60 | 0.89 | 0.83 |

Proposed | 1.48 | 0.90 | 3.45 | 0.92 | 0.93 |

**Table 2.**PSNR values of different denoising algorithms for a Gaussian noise with a standard deviation $\sigma $.

Denoising | |||
---|---|---|---|

$\mathbf{\sigma}$ Value | Bilateral Filter | Anisotropic Diffusion | Random Walks |

$\sigma $ = 0.03 | 36.08 | 27.59 | 35.17 |

$\sigma $ = 0.09 | 33.40 | 27.29 | 32.45 |

$\sigma $ = 0.27 | 19.41 | 32.45 | 23.85 |

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Mohammed, A.; Farup, I.; Pedersen, M.; Hovde, Ø.; Yildirim Yayilgan, S. Stochastic Capsule Endoscopy Image Enhancement. *J. Imaging* **2018**, *4*, 75.
https://doi.org/10.3390/jimaging4060075

**AMA Style**

Mohammed A, Farup I, Pedersen M, Hovde Ø, Yildirim Yayilgan S. Stochastic Capsule Endoscopy Image Enhancement. *Journal of Imaging*. 2018; 4(6):75.
https://doi.org/10.3390/jimaging4060075

**Chicago/Turabian Style**

Mohammed, Ahmed, Ivar Farup, Marius Pedersen, Øistein Hovde, and Sule Yildirim Yayilgan. 2018. "Stochastic Capsule Endoscopy Image Enhancement" *Journal of Imaging* 4, no. 6: 75.
https://doi.org/10.3390/jimaging4060075