# Contrast Transfer Function-Based Exit-Wave Reconstruction and Denoising of Atomic-Resolution Transmission Electron Microscopy Images of Graphene and Cu Single Atom Substitutions by Deep Learning Framework

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Related Works

#### 2.1. Direct Exit-Wave Reconstruction from a Single Defocused Image

#### 2.2. Image Simulation Verification Method

- Prediction of the atomic structure from the raw data obtained from the CCD.
- Conversion of the expected atomic structure into the exit wave by using the multislice method.
- Conduction of the image simulation by using the exit wave and CTF (Equation (6)).
- Verification of the atomic structure by comparing the result of the image simulation to the raw data.
- Iteration of the structure modulation until the simulated results of the expected atomic structure become similar to the actual image up to the desired level.

#### 2.3. FFT-Based Image Deconvolution

#### 2.4. Autoencoder

## 3. Materials and Methods

#### 3.1. CDDAE Framework

_{amp}, D

_{phase}) and an image simulation part (Figure 1). The autoencoder part has an input image (X), which is an ARTEM image obtained from the CCD written in Equation (6) and described as ${\left|\varphi \left(u\right)\right|}^{2}$ and which has the role of finding the relationship between X and the exit wave ($\psi $) from Equation (2) for the direct reconstruction of the exit wave. It is the variation of the convolutional autoencoder, composed of one encoder (E) and two decoders, an amplitude decoder (D

_{amp}) and a phase decoder (D

_{phase}). E consists of the first four sequential layers, and each layer has a rectified linear unit (ReLU) as an activation function. Then, E

_{out}, an output of E, is the latent space representation of the X. Suppose A is the output of D

_{amp}with input E

_{out}, which decodes the amplitude of the exit wave from E

_{out}, and B is the output of D

_{phase}, which decodes the phase of the exit wave from E

_{out}. We set the activation function of each decoder to be a zero-centered hyperbolic tangent for the sensitive learning of each decoder to the sign of the wave. The end of each decoder has a convolutional layer, with a 1 × 1 size 1 kernel and no activation function, for merging results. Detailed information on the CDDAE network is provided in Table 1.

^{−5}as the initial learning rate and default parameters.

#### 3.2. Training Data

## 4. Results

#### 4.1. Direct Exit-Wave Reconstruction from Single Image of Monolayer Graphene

#### 4.2. Identification of Cu Single Atom from Single Image

#### 4.3. Denoising Performance Metrics

_{r}, the result of image simulation through CTF and the reconstructed exit wave in the CDDAE framework, is a nonlinear denoising solution of the input image. A conventional denoising method, the Wiener filter provides a linear denoising solution [14]. To compare the performance of the two methods mentioned earlier, an image simulation dataset of monolayer graphene has been built using MacTempas, and detector noise has been added, so that of the SNR is 9.1164 according to the actual TEM conditions, as shown in Figure 4a,b.

## 5. Conclusions

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Khan, K.; Tareen, A.K.; Aslam, M.; Wang, R.; Zhang, Y.; Mahmood, A.; Ouyang, Z.; Zhang, H.; Gou, Z. Recent developments in emerging two-dimensional materials and their applications. J. Mater. Chem. C
**2020**, 8, 387–440. [Google Scholar] [CrossRef] - Chang, Y.-Y.; Han, H.N.; Kim, M. Analyzing the microstructure and related properties of 2D materials by transmission electron microscopy. Appl. Microsc.
**2019**, 49, 10. [Google Scholar] [CrossRef] [Green Version] - Zhao, J.; Deng, Q.; Bachmatiuk, A.; Sandeep, G.; Popov, A.A.; Eckert, J.; Rümmeli, M.H. Free-Standing Single-Atom-Thick Iron Membranes Suspended in Graphene Pores. Science
**2014**, 343, 1228–1232. [Google Scholar] [CrossRef] [PubMed] - Chen, J.; Shi, T.; Cai, T.; Xu, T.; Sun, L.-T.; Wu, X.; Yu, D. Self healing of defected graphene. Appl. Phys. Lett.
**2013**, 102, 103107. [Google Scholar] [CrossRef] [Green Version] - Ryu, G.H.; Park, H.J.; Ryou, J.; Park, J.; Lee, J.; Kim, G.; Shin, H.S.; Bielawski, C.W.; Ruoff, R.S.; Hong, S.; et al. Atomic-scale dynamics of triangular hole growth in monolayer hexagonal boron nitride under electron irradiation. Nanoscale
**2015**, 7, 10600–10605. [Google Scholar] [CrossRef] [Green Version] - Allen, L.J.; McBride, W.; O’Leary, N.L.; Oxley, M.P. Exit wave reconstruction at atomic resolution. Ultramicroscopy
**2004**, 100, 91–104. [Google Scholar] [CrossRef] - Morgan, A.J.; Martin, A.; D’Alfonso, A.J.; Putkunz, C.T.; Allen, L. Direct exit-wave reconstruction from a single defocused image. Ultramicroscopy
**2011**, 111, 1455–1460. [Google Scholar] [CrossRef] - Kirkland, E.J. Improved high resolution image processing of bright field electron micrographs: I. Theory. Ultramicroscopy
**1984**, 15, 151–172. [Google Scholar] [CrossRef] - Wade, R.-H. A brief look at imaging and contrast transfer. Ultramicroscopy
**1992**, 46, 145–156. [Google Scholar] [CrossRef] - Zemlin, F.; Weiss, K.; Schiske, P.; Kunath, W.; Herrmann, K.H. Coma-free alignment of high resolution electron microscopes with the aid of optical diffractograms. Ultramicroscopy
**1978**, 3, 49–60. [Google Scholar] [CrossRef] - Bursill, L.A.; Wilson, A.R. Electron-optical imaging of the hollandite structure at 3 Å resolution. Acta Crystallogr. Sect. A: Found. Adv.
**1977**, 33, 672–676. [Google Scholar] [CrossRef] - Cowley, J.M.; Moodie, A.F. The scattering of electrons by atoms and crystals. I. A new theoretical approach. Acta Crystallogr.
**1957**, 10, 609–619. [Google Scholar] [CrossRef] - Goodman, P.; Moodie, A.F. Numerical Evaluation of N-Beam Wave-Functions in Electron-Scattering by Multi-Slice Method. Acta Crystallogr. Sect. A: Found. Adv.
**1974**, A30, 280–290. [Google Scholar] [CrossRef] - Ishizuka, K.; Uyeda, N. A new theoretical and practical approach to the multislice method. Acta Crystallogr. Sect. A: Found. Adv.
**1977**, A33, 740–749. [Google Scholar] [CrossRef] - Koch, C. Determination of Core Structure Periodicity and Point Defect Density Along Dislocations. Ph.D. Thesis, Arizona State University, Tempe, Arizona, April 2002. [Google Scholar]
- Kazubek, M. Wavelet domain image denoising by thresholding and Wiener filtering. IEEE Signal Process. Lett.
**2003**, 10, 324–326. [Google Scholar] [CrossRef] - Zhang, J.; Pan, J.; Lai, W.-S.; Lau, R.W.H.; Yang, M.-H. Learning Fully Convolutional Networks for Iterative Non-blind Deconvolution. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 6969–6977. [Google Scholar]
- Kruse, J.; Rother, C.; Schmidt, U. Learning to Push the Limits of Efficient FFT-Based Image Deconvolution. In Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; pp. 4596–4604. [Google Scholar]
- Kramer, M.A. Nonlinear principal component analysis using autoassociative neural networks. AIChE J.
**1991**, 37, 233–243. [Google Scholar] [CrossRef] - Vincent, P.; Larochelle, H.; Lajoie, I.; Bengio, Y.; Manzagol, P.A. Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion. J. Mach. Learn. Res.
**2010**, 11, 3371–3408. [Google Scholar] - Hinton, G.E.; Salakhutdinov, R.R. Reducing the dimensionality of data with neural networks. Science
**2006**, 313, 504–507. [Google Scholar] [CrossRef] [Green Version] - Zeng, K.; Yu, J.; Wang, R.; Li, C.; Tao, D. Coupled Deep Autoencoder for Single Image Super-Resolution. IEEE Trans. Cybern.
**2015**, 47, 27–37. [Google Scholar] [CrossRef] - Sakurada, M.; Yairi, T. Anomaly Detection Using Autoencoders with Nonlinear Dimensionality Reduction. In Proceedings of the MLSDA 2014 2nd Workshop on Machine Learning for Sensory Data Analysis (MLSDA’14), Gold Coast, QLD, Australia, 2 December 2014; pp. 4–11. [Google Scholar]
- Kingma, D.P.; Ba, J. Adam: A Method for Stochastic Optimization. arXiv
**2014**, arXiv:1412.6980. Available online: https://arxiv.org/abs/1412.6980 (accessed on 30 January 2017). - Castro Neto, A.H.; Guinea, F.; Peres, N.M.R.; Novoselov, K.S.; Geim, A.K. The electronic properties of graphene. Rev. Mod. Phys.
**2009**, 81, 109–162. [Google Scholar] [CrossRef] [Green Version] - Geim, A.K.; Novoselov, K.S. The rise of graphene. Nat. Mater.
**2007**, 6, 183–191. [Google Scholar] [CrossRef] [PubMed] - Ramasse, Q.M.; Zan, R.; Bangert, U.; Boukhvalov, D.W.; Son, Y.W.; Novoselov, K.S. Direct Experimental Evidence of Metal-Mediated Etching of Suspended Graphene. ACS Nano
**2012**, 6, 4063–4071. [Google Scholar] [CrossRef] [Green Version] - Kano, E.; Hashimoto, A.; Kaneko, T.; Tajima, N.; Ohno, T.; Takeguchi, M. Interactions between C and Cu atoms in single-layer graphene: Direct observation and modelling. Nanoscale
**2016**, 8, 529–535. [Google Scholar] [CrossRef] [PubMed] - Marconcini, P.; Cresti, A.; Roche, S. Effect of the Channel Length on the Transport Characteristics of Transistors Based on Boron-Doped Graphene Ribbons. Materials
**2018**, 11, 667. [Google Scholar] [CrossRef] [Green Version] - Wang, H.B.; Maiyalagan, T.; Wang, X. Review on Recent Progress in Nitrogen-Doped Graphene: Synthesis, Characterization, and Its Potential Applications. ACS Catal.
**2012**, 2, 781–794. [Google Scholar] [CrossRef]

**Figure 1.**The structure of the convolutional dual-decoder autoencoder (CDDAE) framework. Input images are decoded to A and B, which correspond to the amplitude and phase of the exit wave. I

_{r}is the result of the image simulation when applying the contrast transfer function (CTF) to the reconstructed exit wave.

**Figure 2.**Images and intensity profiles of the simulated (

**a,c,e**) and experimental (

**b,d,f**) data by the CDDAE framework. The intensity profiles are plotted along the solid red lines from left to right in each image. (

**a**) Image simulation result; (

**b**) Input image; (

**c**) Amplitude of simulated exit wave; (

**d**) Amplitude of reconstructed exit wave; (

**e**) Phase of simulated exit wave; (

**f**) Phase of reconstructed exit wave.

**Figure 3.**(

**a**) Phase image and (

**b**) amplitude image of the reconstructed exit wave, phase, and amplitude, from the atomic-resolution transmission electron microscopy (ARTEM) image by the CDDAE framework. Simulated (

**c**) phase image and (

**d**) amplitude image of the substituted Cu single atom in the graphene lattice. Simulated (

**e**) phase image and (

**f**) amplitude image of the substituted Si single atom in the graphene lattice. (

**g**,

**h**) The intensity profiles are plotted as solid red lines from left to right in each image.

**Figure 4.**(

**a**) Image simulation result of monolayer graphene using MacTempas; (

**b**) Noise-added simulation image; (

**c**) Denoising result of the noise-added image with the Wiener filter; (

**d**) Denoising result of the noise-added image with the CDDAE framework.

Encoder | Act | Output Shape | Parameters |
---|---|---|---|

Input image | - | 1 × 128 × 128 | - |

Convolution 3 × 3 | ReLU | 64 × 128 × 128 | 1048576 |

Convolution 3 × 3 | ReLU | 64 × 128 × 128 | 1048576 |

Downsample | 64 × 64 × 64 | ||

Convolution 3 × 3 | ReLU | 128 × 64 × 64 | 524288 |

Convolution 3 × 3 | ReLU | 128 × 64 × 64 | 524288 |

Downsample | 128 × 32 × 32 | ||

Decoder1 | Act | Output Shape | Parameters |

Upsample | 128 × 64 × 64 | ||

Convolution 3 × 3 | Tanh | 128 × 64 × 64 | 524288 |

Convolution 3 × 3 | Tanh | 128 × 64 × 64 | 524288 |

Upsample | 128 × 64 × 128 | ||

Convolution 3 × 3 | Tanh | 64 × 128 × 128 | 1048576 |

Convolution 3 × 3 | Tanh | 64 × 128 × 128 | 1048576 |

Convolution 1 × 1 | 1 × 128 × 128 | ||

Decoder2 | Act | Output Shape | Parameters |

Upsample | 128 × 64 × 64 | ||

Convolution 3 × 3 | Tanh | 128 × 64 × 64 | 524288 |

Convolution 3 × 3 | Tanh | 128 × 64 × 64 | 524288 |

Upsample | 128 × 64 × 128 | ||

Convolution 3 × 3 | Tanh | 64 × 128 × 128 | 1048576 |

Convolution 3 × 3 | Tanh | 64 × 128 × 128 | 1048576 |

Convolution 1 × 1 | 1 × 128 × 128 |

Method | SNR | PSNR | SSIM |
---|---|---|---|

Noise-added | 9.1164 | 17.5501 | 0.5443 |

Wiener filtered | 16.7889 | 25.2226 | 0.9229 |

CDDAE framework | 18.3390 | 26.7727 | 0.9440 |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Lee, J.; Lee, Y.; Kim, J.; Lee, Z.
Contrast Transfer Function-Based Exit-Wave Reconstruction and Denoising of Atomic-Resolution Transmission Electron Microscopy Images of Graphene and Cu Single Atom Substitutions by Deep Learning Framework. *Nanomaterials* **2020**, *10*, 1977.
https://doi.org/10.3390/nano10101977

**AMA Style**

Lee J, Lee Y, Kim J, Lee Z.
Contrast Transfer Function-Based Exit-Wave Reconstruction and Denoising of Atomic-Resolution Transmission Electron Microscopy Images of Graphene and Cu Single Atom Substitutions by Deep Learning Framework. *Nanomaterials*. 2020; 10(10):1977.
https://doi.org/10.3390/nano10101977

**Chicago/Turabian Style**

Lee, Jongyeong, Yeongdong Lee, Jaemin Kim, and Zonghoon Lee.
2020. "Contrast Transfer Function-Based Exit-Wave Reconstruction and Denoising of Atomic-Resolution Transmission Electron Microscopy Images of Graphene and Cu Single Atom Substitutions by Deep Learning Framework" *Nanomaterials* 10, no. 10: 1977.
https://doi.org/10.3390/nano10101977