Deep Encoder-Decoder Adversarial Reconstruction (DEAR) Network for 3D CT from Few-View Data
Abstract
:1. Introduction
- (1)
- DEAR-3D utilizes 3D convolutional layers to extract 3D information from multiple adjacent slices in a generative adversarial network (GAN) [23] framework. Different from reconstructing 2D images from 3D input data [2], DEAR-3D directly reconstructs a 3D volume, with faithful texture and image details; and
- (2)
- An extensive comparative study was performed between DEAR-3D and various 2D counterparts to demonstrate the merits of the proposed 3D network.
2. Methodology
2.1. Proposed Framework
2.2. Generator Network
2.3. Discriminator Network
2.4. Objective Functions for Generator
2.4.1. MSE Loss
2.4.2. Structural Similarity Loss
2.4.3. Adversarial Loss
2.5. Corresponding 2D Networks for Comparisons
3. Experimental Design and Results
3.1. Dataset and Pre-Processing
3.2. Hyperparameter Selection and Network Comparison
3.3. Comparison with Other Deep Learning Methods
3.4. Ablation Analysis
4. Discussion and Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Brenner, D.J.; Hall, E.J. Computed Tomography—An Increasing Source of Radiation Exposure. N. Engl. J. Med. 2007, 357, 2277–2284. [Google Scholar] [CrossRef] [Green Version]
- Shan, H.; Zhang, Y.; Yang, Q.; Kruger, U.; Kalra, M.K.; Sun, L.; Cong, W.; Wang, G. 3-D Convolutional Encoder-Decoder Network for Low-Dose CT via Transfer Learning From a 2-D Trained Network. IEEE Trans. Med. Imaging 2018, 37, 1522–1534. [Google Scholar] [CrossRef] [PubMed]
- Chen, H.; Zhang, Y.; Zhang, W.; Liao, P.; Li, K.; Zhou, J.; Wang, G. Low-dose CT via convolutional neural network. Biomed. Opt. Express 2017, 8, 679–694. [Google Scholar] [CrossRef]
- Shan, H.; Padole, A.; Homayounieh, F.; Kruger, U.; Khera, R.D.; Nitiwarangkul, C.; Kalra, M.K.; Wang, G. Competitive performance of a modularized deep neural network compared to commercial algorithms for low-dose CT image reconstruction. Nat. Mach. Intell. 2019, 1, 269–276. [Google Scholar] [CrossRef] [Green Version]
- Chen, H.; Zhang, Y.; Kalra, M.K.; Lin, F.; Chen, Y.; Liao, P.; Zhou, J.; Wang, G. Low-Dose CT with a Residual Encoder-Decoder Convolutional Neural Network (RED-CNN). IEEE Trans. Med. Imaging 2017, 36, 2524–2535. [Google Scholar] [CrossRef] [PubMed]
- Glick, S.J. Breast CT. Annu. Rev. Biomed. Eng. 2007, 9, 501–526. [Google Scholar] [CrossRef] [PubMed]
- Wallace, M.J.; Kuo, M.D.; Glaiberman, C.; Binkert, C.A.; Orth, R.C.; Soulez, G. Three-Dimensional C-arm Cone-beam CT: Applications in the Interventional Suite. J. Vasc. Interv. Radiol. 2008, 19, 799–813. [Google Scholar] [CrossRef]
- Jerri, A.J. The Shannon sampling theorem—Its various extensions and applications: A tutorial review. Proc. IEEE 1977, 65, 1565–1596. [Google Scholar] [CrossRef]
- Wang, G.; Ye, Y.; Yu, H. Approximate and exact cone-beam reconstruction with standard and non-standard spiral scanning. Phys. Med. Biol. 2007, 52, R1–R13. [Google Scholar] [CrossRef]
- Gordon, R.; Bender, R.; Herman, G.T. Algebraic reconstruction techniques (ART) for three-dimensional electron microscopy and X-ray photography. J. Theor. Biol. 1970, 29, 471–481. [Google Scholar] [CrossRef]
- Andersen, A. Simultaneous Algebraic Reconstruction Technique (SART): A superior implementation of the ART algorithm. Ultrason. Imaging 1984, 6, 81–94. [Google Scholar] [CrossRef] [PubMed]
- Dempster, A.P.; Laird, N.M.; Rubin, D.B. Maximum Likelihood from Incomplete Data via the EM Algorithm. J. R. Stat. Soc. Ser. B (Methodological) 1977, 39, 1–38. [Google Scholar]
- Wang, G. A Perspective on Deep Imaging. IEEE Access 2016, 4, 8914–8924. [Google Scholar] [CrossRef]
- Wang, G.; Butler, A.; Yu, H.; Campbell, M. Guest Editorial Special Issue on Spectral CT. IEEE Trans. Med. Imaging 2015, 34, 693–696. [Google Scholar] [CrossRef]
- Greenspan, H.; Ginneken, B.V.; Summers, R.M. Guest Editorial Deep Learning in Medical Imaging: Overview and Future Promise of an Exciting New Technique. IEEE Trans. Med. Imaging 2016, 35, 1153–1159. [Google Scholar] [CrossRef]
- Lecun, Y.; Bengio, Y. Convolutional networks for images, speech, and time-series. In The Handbook of Brain Theory and Neural Networks; MIT Press: Cambridge, MA, USA, 1995. [Google Scholar]
- Jin, K.H.; McCann, M.T.; Froustey, E.; Unser, M. Deep Convolutional Neural Network for Inverse Problems in Imaging. IEEE Trans. Image Process. 2017, 26, 4509–4522. [Google Scholar] [CrossRef] [Green Version]
- Lee, H.; Lee, J.; Kim, H.; Cho, B.; Cho, S. Deep-Neural-Network-Based Sinogram Synthesis for Sparse-View CT Image Reconstruction. IEEE Trans. Radiat. Plasma Med. Sci. 2019, 3, 109–119. [Google Scholar] [CrossRef] [Green Version]
- Chen, H.; Zhang, Y.; Chen, Y.; Zhang, J.; Zhang, W.; Sun, H.; Lv, Y.; Liao, P.; Zhou, J.; Wang, G. LEARN: Learned Experts’ Assessment-Based Reconstruction Network for Sparse-Data CT. IEEE Trans. Med. Imaging 2018, 37, 1333–1347. [Google Scholar] [CrossRef]
- Li, Y.; Li, K.; Zhang, C.; Montoya, J.; Chen, G. Learning to Reconstruct Computed Tomography (CT) Images Directly from Sinogram Data under A Variety of Data Acquisition Conditions. IEEE Trans. Med. Imaging 2019. [Google Scholar] [CrossRef]
- Xie, H.; Shan, H.; Cong, W.; Zhang, X.; Liu, S.; Ning, R.; Wang, G. Dual network architecture for few-view CT - trained on ImageNet data and transferred for medical imaging. In Developments in X-ray Tomography XII; International Society for Optics and Photonics: San Diego, CA, USA, 2019; Volume 11113, p. 111130V. [Google Scholar] [CrossRef] [Green Version]
- Liu, Z.; Bicer, T.; Kettimuthu, R.; Gursoy, D.; De Carlo, F.; Foster, I. TomoGAN: Low-Dose X-ray Tomography with Generative Adversarial Networks. arXiv 2019, arXiv:1902.07582. [Google Scholar]
- Goodfellow, I.J.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y. Generative Adversarial Networks. arXiv 2014, arXiv:1406.2661. [Google Scholar]
- Schaller, S.; Flohr, T.; Steffen, P. An efficient Fourier method for 3-D radon inversion in exact cone-beam CT reconstruction. IEEE Trans. Med. Imaging 1998, 17, 244–250. [Google Scholar] [CrossRef] [PubMed]
- Grangeat, P. Mathematical framework of cone beam 3D reconstruction via the first derivative of the radon transform. In Mathematical Methods in Tomography; Lecture Notes in Mathematics; Herman, G.T., Louis, A.K., Natterer, F., Eds.; Springer: Berlin/Heidelberg, Germany, 1991; pp. 66–97. [Google Scholar]
- Grangeat, P.P.; Masson, P.L.; Melennec, P.; Sire, P. Evaluation of the 3-D radon transform algorithm for cone beam reconstruction. In Medical Imaging V: Image Processing; International Society for Optics and Photonics: San Diego, CA, USA, 1991; Volume 1445, pp. 320–331. [Google Scholar] [CrossRef]
- Katsevich, A. An improved exact filtered backprojection algorithm for spiral computed tomography. Adv. Appl. Math. 2004, 32, 681–697. [Google Scholar] [CrossRef] [Green Version]
- Arjovsky, M.; Chintala, S.; Bottou, L. Wasserstein Generative Adversarial Networks. In Proceedings of the International Conference on Machine Learning, Sydney, Australia, 6–11 August 2017; pp. 214–223. [Google Scholar]
- Gulrajani, I.; Ahmed, F.; Arjovsky, M.; Dumoulin, V.; Courville, A.C. Improved Training of Wasserstein GANs. In Advances in Neural Information Processing Systems 30; Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R., Eds.; Curran Associates, Inc.: Dutchess County, NY, USA, 2017; pp. 5767–5777. [Google Scholar]
- Ronneberger, O.; Fischer, P.; Brox, T. U-Net: Convolutional Networks for Biomedical Image Segmentation. arXiv 2015, arXiv:1505.04597. [Google Scholar]
- Donoho, D.L. Compressed sensing. IEEE Trans. Inf. Theory 2006, 52, 1289–1306. [Google Scholar] [CrossRef]
- Huang, G.; Liu, Z.; van der Maaten, L.; Weinberger, K.Q. Densely Connected Convolutional Networks. arXiv 2016, arXiv:1608.06993. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep Residual Learning for Image Recognition. arXiv 2015, arXiv:1512.03385. [Google Scholar]
- Wolterink, J.M.; Leiner, T.; Viergever, M.A.; Išgum, I. Generative Adversarial Networks for Noise Reduction in Low-Dose CT. IEEE Trans. Med. Imaging 2017, 36, 2536–2545. [Google Scholar] [CrossRef]
- Wang, Z.; Bovik, A.C. Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures. IEEE Signal Process. Mag. 2009, 26, 98–117. [Google Scholar] [CrossRef]
- Zhao, H.; Gallo, O.; Frosio, I.; Kautz, J. Loss Functions for Image Restoration With Neural Networks. IEEE Trans. Comput. Imaging 2017, 3, 47–57. [Google Scholar] [CrossRef]
- Zhou, W.; Bovik, A.C.; Sheikh, H.R.; Simoncelli, E.P. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process. 2004, 13, 600–612. [Google Scholar] [CrossRef] [Green Version]
- Low Dose CT Grand Challenge. Available online: https://www.aapm.org/grandchallenge/lowdosect/ (accessed on 18 April 2019).
- Kak, A.C.; Slaney, M.; Wang, G. Principles of Computerized Tomographic Imaging. Med. Phys. 2002, 29, 107. [Google Scholar] [CrossRef]
- Abadi, M.; Agarwal, A.; Barham, P.; Brevdo, E.; Chen, Z.; Citro, C.; Corrado, G.S.; Davis, A.; Dean, J.; Devin, M.; et al. TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems. arXiv 2016, arXiv:1603.04467. [Google Scholar]
- Kingma, D.P.; Ba, L.J. Adam: A Method for Stochastic Optimization. arXiv 2015, arXiv:1412.6980. [Google Scholar]
- Cong, W.; Shan, H.; Zhang, X.; Liu, S.; Ning, R.; Wang, G. Deep-learning-based breast CT for radiation dose reduction. In Developments in X-ray Tomography XII; International Society for Optics and Photonics: San Diego, CA, USA, 2019; Volume 11113, p. 111131L. [Google Scholar] [CrossRef]
- Korhonen, J.; You, J. Peak signal-to-noise ratio revisited: Is simple beautiful? In Proceedings of the 2012 Fourth International Workshop on Quality of Multimedia Experience, Melbourne, Australia, 5–7 July 2012; pp. 37–38. [Google Scholar] [CrossRef]
- Willmott, C.J.; Matsuura, K. Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Clim. Res. 2005, 30, 79–82. [Google Scholar] [CrossRef]
- Cramer, A.; Hecla, J.; Wu, D.; Lai, X.; Boers, T.; Yang, K.; Moulton, T.; Kenyon, S.; Arzoumanian, Z.; Krull, W.; et al. Stationary Computed Tomography for Space and other Resource-constrained Environments. Sci. Rep. 2018, 8. [Google Scholar] [CrossRef]
# Parameters | DEAR-3D | DEAR-2D | DEAR-2D-i |
---|---|---|---|
5,123,617 | 3,459,749 | 5,519,329 |
MSE | SSIM | AL | |
---|---|---|---|
DEAR-2D | √ | ||
DEAR-2D | √ | √ | |
DEAR-2D-i | √ | √ | |
DEAR-3D | √ | √ | |
DEAR-3D | √ | √ | √ |
FBP | FBPConvNet | Residual-CNN | DEAR-3D | |
---|---|---|---|---|
PSNR | ||||
SSIM | ||||
RMSE |
FBP | DEAR-2D | DEAR-2D | DEAR-2D-i | DEAR-3D | DEAR-3D | |
---|---|---|---|---|---|---|
PSNR | ||||||
SSIM | ||||||
RMSE |
© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Xie, H.; Shan, H.; Wang, G. Deep Encoder-Decoder Adversarial Reconstruction (DEAR) Network for 3D CT from Few-View Data. Bioengineering 2019, 6, 111. https://doi.org/10.3390/bioengineering6040111
Xie H, Shan H, Wang G. Deep Encoder-Decoder Adversarial Reconstruction (DEAR) Network for 3D CT from Few-View Data. Bioengineering. 2019; 6(4):111. https://doi.org/10.3390/bioengineering6040111
Chicago/Turabian StyleXie, Huidong, Hongming Shan, and Ge Wang. 2019. "Deep Encoder-Decoder Adversarial Reconstruction (DEAR) Network for 3D CT from Few-View Data" Bioengineering 6, no. 4: 111. https://doi.org/10.3390/bioengineering6040111