Next Article in Journal
An Experimental Comparison between Deep Learning and Classical Machine Learning Approaches for Writer Identification in Medieval Documents
Next Article in Special Issue
Understanding the Effects of Optimal Combination of Spectral Bands on Deep Learning Model Predictions: A Case Study Based on Permafrost Tundra Landform Mapping Using High Resolution Multispectral Satellite Imagery
Previous Article in Journal
Spatial Spectral Band Selection for Enhanced Hyperspectral Remote Sensing Classification Applications
Previous Article in Special Issue
μXRF Mapping as a Powerful Technique for Investigating Metal Objects from the Archaeological Site of Ferento (Central Italy)
Open AccessArticle

Marching Cubes and Histogram Pyramids for 3D Medical Visualization

Department of Computer and Information Science, Faculty of Applied Science, King Mongkut’s University of Technology North Bangkok, Bangkok 10800, Thailand
J. Imaging 2020, 6(9), 88; https://doi.org/10.3390/jimaging6090088
Received: 1 July 2020 / Revised: 28 August 2020 / Accepted: 31 August 2020 / Published: 3 September 2020
(This article belongs to the Special Issue Robust Image Processing)

Abstract

This paper aims to implement histogram pyramids with marching cubes method for 3D medical volumetric rendering. The histogram pyramids are used for feature extraction by segmenting the image into the hierarchical order like the pyramid shape. The histogram pyramids can decrease the number of sparse matrixes that will occur during voxel manipulation. The important feature of the histogram pyramids is the direction of segments in the image. Then this feature will be used for connecting pixels (2D) to form up voxel (3D) during marching cubes implementation. The proposed method is fast and easy to implement and it also produces a smooth result (compared to the traditional marching cubes technique). The experimental results show the time consuming for generating 3D model can be reduced by 15.59% in average. The paper also shows the comparison between the surface rendering using the traditional marching cubes and the marching cubes with histogram pyramids. Therefore, for the volumetric rendering such as 3D medical models and terrains where a large number of lookups in 3D grids are performed, this method is a particularly good choice for generating the smooth surface of 3D object.
Keywords: marching cubes; histogram pyramids; volumetric rendering; 3D medical model; smooth voxel; isosurface marching cubes; histogram pyramids; volumetric rendering; 3D medical model; smooth voxel; isosurface

1. Introduction

3D visualization methods of human bone and organs were applied to diagnosis as long as 100 years ago [1]. Normally, 3D medical imaging system aims to provide both quantitative and qualitative information for diagnosis. 3D visualization system can be divided into 4 operations: (1) preprocessing that deals with the volume of interest and features extraction; (2) visualization processes generate 3D object from 2D images; (3) manipulation explains the geometry of object that can be distorted and deformed; (4) analysis that deals with methods of quantify 3D object [2]. For medical volumetric rendering where 2D binary images (CT-volume) are feed to construct 3D object, collision detection algorithm is used to manipulate the intersecting fragments and generate triangulated mesh models (see Figure 1a). The doctor can take this advantage of training by observing the demonstration of context of 3D manipulation of bone fragments and the resulting CT images [3]. More works of 3D visualization such as texture mapping and semi-automatic image segmentation. Texture mapping technique constructs 3D object by interpreting the spatial relationships of 2D binary images and generates 3D visualization of perspective-mapped from each image layer [4]. Whereas, semi-automatic image segmentation allows the doctor to make the segmentation of subject by the area of interest [5]. Therefore, the major drawback of these methods is the computational cost of real time resampling for making texture in 3D object reconstruction process [4], and infeasibility for each individual segmentation [5] (see Figure 1b,c). The alternative choice for 3D volumetric rendering is marching cubes. The marching cubes method keeps the coordinates conveyed by traversing the outline of 2D binary shape and marches them to construct 3D object [6,7,8,9,10]. The algorithm is based on the configuration of 15 fundamental cubes (see Section 2). Figure 2 shows 3D human head model constructed by marching cubes method; the method used 150 slides of 2D image as input source.
It seems like the marching cubes can reduce the computational time used for resampling in 3D reconstruction, but the problem still remains in observing the qualitative information of 3D surface constructed using marching cubes. One major problem of marching cubes is the unused voxels which can be generated during parsing the coordinates and the intensity values of 2D images, these unused voxels affect for the smoothness of 3D surface (the detail of the surface smoothness will be explained in Section 4 and Section 6). To overcome this drawback, this paper introduces histogram pyramids with marching cubes method for 3D medical volumetric rendering. The histogram pyramids organize the image entries to form the voxel related to the index values, the traversal of histogram pyramids is used to construct the point list that will be later used for generating the voxel. The organization of paper consists of (1) Introduction, (2) The brief concept of marching cubes method, (3) Reading the intensity value of CT images for 3D rendering, (4) Histogram pyramids, (5) Implementation, (6) Results, and (7) Conclusion and future works.

2. Marching Cubes

Original marching cubes algorithm was developed by [6], later works applied this algorithm for 3D rendering using 2D medical images (CT: Computed Tomography Scan, MRI: Magnetic Resonance Imaging, and SPECT: Single-Photon Emission Computed Tomography) sequence as an input. The marching cubes algorithm will be explained later in this section. More works of volume rendering techniques such as [7] used sampled scalar functions for displaying 3D surfaces, [8] presented the extended version of volume rendering that used to handle images with mixtures properties (color and texture). To deal with the boundary value problems which are the main task for detecting the outline of 2D binary shape, [9] applied the shortest path analysis to the speed function and used it to solve an unknown in the surface function. [10] used the endoscope images for 3D surface reconstruction; the sophisticated equipment was used as the fine-tuned measurement of object together with 3D grid that would be fit for the specific endoscope image.
As mentioned earlier, marching cubes algorithm is based on the configuration of 15 fundamental cubes [6]. 2D medical images (slides) contained the scalar value of gray scale will be used to construct the cubes based on their indexes correspond to the configuration of 15 fundamental cubes as shown in Figure 3. Based on [13,14], a cube consists of eight vertices that can contain two values of 0 and 1; note that the value of 0 implies that there is no information conveyed in the vertex (the vertex will not be used to form the next cube for 3D object rendering), whereas the value of 1 implies that the vertex contains the information and will be used to configure the next cube as the connecting vertex.
The first step of vertex configuration to form the cube begins with connecting the upper slice; the first four vertices locate in lower slice and the second four vertices will be in the upper one as illustrated in Figure 4. Next, the algorithm will determine the inner cut within the cube by using these criterions:
  • The vertex will be the inner vertex if the vertex value is greater than or equal to the isosurface value; then assign “0” to the vertex.
  • The vertex will be the outer vertex if the vertex value is less than the isosurface value; then assign “1” to the vertex.
Therefore, they will have 28 = 256 combinations per one cube (2 = two values of 0, 1; 8 = number of vertices in 1 cube).
The third step of marching cubes is the creating of index; the index will be used to match the configuration of 15 fundamental cubes in Figure 3. Figure 5a shows the configuration of index and Figure 5b shows the example of index that contains the value = 01000001, this index matches the cube no. 4 in Figure 3. It means that the cube will be sheared off at edge no. 1, 4, 9 (e1, e4, e9) and edge no. 6, 7, 12 (e6, e7, e12), respectively. In the last step of marching cubes, the algorithm will determine the direction of the cut using normal vector (see Figure 6).

3. Reading the Intensity Value of CT Images for 3D Rendering

The first step for obtaining the intensity value of medical images begins with reading the coordinates and values then store them in the data structure of the system, e.g., the binary image with dimension of 256 × 256 will be read and kept in the array. The input array will be manipulated for marching cubes and histogram pyramids. Figure 7 shows the binary image and grid structure used for dividing image into 128 × 128 segments.
The marching cubes algorithm will generate 3D visualization of 2D binary images by manipulating in form of width x height of image; e.g., in MRI, the resolution is determined by the number of pixels in a specified FOV (Field-of-view), for FOV = 320, the input image with dimension of 320 × 320 will obtain the amount of data = (320 + 1) × (320 + 1) = 103,041; these data will be stored in the array [0, …, 103,040]. The data that will be kept in the array is the intensity value of input image; in the gray-scale image, the range of gray-scale values are varying from 0 to 255 (256 intensity values). The rendering process will be computed starting from the first slide until the last slide of images (k, …, k + 1) as illustrated previously in Figure 4. Figure 8 shows the CT images sequence.

4. Histogram Pyramids

One major problem of manipulating the large amount of image data is sparse matrix. The main idea of histogram pyramids is used for preventing sparse matrix [16,17,18]. The sparse matrix can be occurred during the process of connecting image slides, many elements in sparse matrix can be spread out and most of them will not be used for generating the voxel. Figure 9 shows the index of histogram pyramids created from vertices, e.g., V000 → 0, V100 → 1, V010 → 2, V001 → 3, V101 → 4, V011 → 5, V110 → 6, V111 → 7, and the last index (index no. 8) will be kept as the starting point of the traversal of histogram pyramids.
If V101 (index no. 4) contains maximum intensity value, the histogram pyramids method can be traversed as shown in Figure 10. In this example, a 2D array contains nine elements (the index range = [0…8]) used to represent nine entries of 2D image. Note that x will be terminated when the summation of index pointer exceeds the maximum value of the index range (8). In the example in Figure 10, the traversal path is started with index no. 4, and L2 will be set to 0, 8 (0 + 8 ≥ 8; thus, this point is fit). The list will be lower down to L1 and retain start = 0. In L1: the upper left element, the index range will be set to 0, 3; the upper right element, the index range will be set to 3, 5 (3 + 5 ≥ 8; thus, this point is fit). The list will be lower down to L0 and retain start = 3. In L0: the upper left element = empty; the upper right element, the index range will be set to 3, 4; the lower left element, the index range will be set to 4, 5 (4 + 5 ≥ 8; this point is fit). The resulting point = (2, 1). The resulting point list will be used later in marching cubes process.

5. Implementation

The CT/MRI of 32/64/128 slides must be provided to use as input images. Figure 11 shows the MRI slides of human knee.
Two important parameters will be read: the coordinates and the intensity values; the point list of histogram pyramids will be computed for marching cubes process. The user interface of the program is shown in Figure 12, the program may prompt to the user for rotate the result 3D object and toggle to the wireframe mode. The screen captured of the program is also shown in Figure 12.

6. Results

As mentioned earlier in Section 4, the histogram pyramids can reduce the redundant voxels which are not used for generating 3D surface. These unused voxels affect for causes of surface roughness in 3D reconstruction. Figure 13 shows the comparison of surface rendering between the marching cubes [14] and the marching cubes with histogram pyramids, by reducing the unused voxel, the marching cubes with histogram pyramids can generate the smooth surface. To measure the smoothness of surface, [19] proposed the edge point computation and local window to calculate the distance between the vertices. The average curvature value is calculated for each vertex, and it corresponds to the mean of the curvature of all vertices from its local window. The curvature value is the parameter to measure the roughness of 3D mesh model [19]. Generally, the common parameter used to determine the curvature is mesh density (vertices). In [19,20], the work already proved that a higher mesh density associated to the roughness of 3D mesh model. Therefore, the model generated using the traditional marching cubes will have higher mesh density than the model generated using the marching cubes with histogram pyramids. Table 1 shows the comparison of mesh densities between the marching cubes [14] and the marching cubes with histogram pyramids.
The system was tested using six models; Table 1 shows the results of experiment, the rendering times of marching cubes compared with the rendering times of marching cubes with histogram pyramids. The paper also tested models by printing them with 3D printer using printing resolution = 200 µm. The results of rendering models and their 3D printings are shown in Figure 14.

7. Conclusions and Future Works

This paper presents the method for 3D visualization using marching cubes and histogram pyramids. The input image slides of CT/MRI were fed into the system; two important parameters, coordinates and intensity values of image, were computed using histogram pyramids to generate point list; the system later used point list for marching cubes process. The relationships of voxel among the layers of medical images (k and k + 1) were manipulated to create the sequence of cubes and form up 3D object. The paper also shows the comparison between the rendering times of marching cubes and the rendering times of marching cubes with histogram pyramids. The paper also shows the smoothness of 3D object generated from the marching cubes with histogram pyramids. The results show that the mesh density is the major factor affecting the experiment. Table 1 shows the time consuming for generating 3D model can be reduced by 15.59% in average. The complexity of algorithm of histogram pyramids for reducing sparse matrix gives O (N) + M (log N), where N is number of elements and M is number of lists. The recommendations of future works such as improve the histogram pyramids algorithm and conduct the research using terrain data in 3D computer game.

Funding

This research was funded by the King Mongkut’s University of Technology North Bangkok. Contract no. KMUTNB-GOV-60-53.

Conflicts of Interest

The author declares no conflict of interest.

References

  1. Nakayama, Y.; Tanida, Y. Atlas of Visualization; CRC Press: Boca Raton, FL, USA, 1997; pp. 213–214. [Google Scholar]
  2. Udupa, J.K. Three-dimensional visualization and analysis methodologies: A current perspective. Radiographics 1999, 19, 783–806. [Google Scholar] [CrossRef] [PubMed]
  3. Birkfellner, W. Applied Medical Image Processing: A Basic Course; CRC Press: Boca Raton, FL, USA, 2016; pp. 262–265. [Google Scholar]
  4. Helminen, H.; Alakuijala, J.; Laitinen, J.; Sallinen, S. Constant Z line texture mapping in interactive visualization of medical images. In Proceedings of the 17th International Conference of the Engineering in Medicine and Biology Society, IEEE, Montreal, QC, Canada, 20–23 September 1995; pp. 1039–1040. [Google Scholar]
  5. González Izard, S.; Sánchez Torres, R.; Alonso Plaza, Ó.; Juanes Méndez, J.A.; García-Peñalvo, F.J. Nextmed: Automatic Imaging Segmentation, 3D Reconstruction, and 3D Model Visualization Platform Using Augmented and Virtual Reality. Sensors 2020, 20, 2962. [Google Scholar] [CrossRef] [PubMed]
  6. Lorensen, W.E.; Cline, H.E. Marching cubes: A high resolution 3D surface construction algorithm. ACM SIGGRAPH Comput. Graph. 1987, 21, 163–169. [Google Scholar] [CrossRef]
  7. Levoy, M. Display of surfaces from volume data. IEEE Comput. Graph. Appl. 1988, 8, 29–37. [Google Scholar] [CrossRef]
  8. Drebin, R.A.; Carpenter, L.; Hanrahan, P. Volume rendering. ACM SIGGRAPH Comput. Graph. 1988, 22, 65–74. [Google Scholar] [CrossRef]
  9. Sethian, J.A. Fast Marching Methods. SIAM Rev. 1999, 41, 199–235. [Google Scholar] [CrossRef]
  10. Long, Z.; Nagamune, K. A Marching Cubes Algorithm: Application for Three-dimensional Surface Reconstruction Based on Endoscope and Optical Fiber. Information 2015, 18, 1425–1437. [Google Scholar]
  11. Messmer, P.; Matthews, F.; Jacob, A.L.; Kikinis, R.; Regazzoni, P.; Noser, H. A CT Database for Research, Development and Education: Concept and Potential. J. Digit. Imaging 2007, 20, 17–22. [Google Scholar] [CrossRef] [PubMed]
  12. Zhong, D.; Zhang, J.; Wang, L. Fast Implicit Surface Reconstruction for the Radial Basis Functions Interpolant. Appl. Sci. 2019, 9, 5335. [Google Scholar] [CrossRef]
  13. Gong, S.; Newman, T.S. A corner feature sensitive marching squares. In Proceedings of the IEEE Southeastcon, IEEE, Jacksonville, FL, USA, 4–7 April 2013; pp. 1–6. [Google Scholar]
  14. Visutsak, P.; Pensiri, F.; Chaowalit, O. Smooth voxel surface for medical volumetric rendering. In Proceedings of the International Conference on Image and Video Processing, and Artificial Intelligence. International Society for Optics and Photonics, Shanghai, China, 23–25 August 2019; p. 113210Y. [Google Scholar]
  15. Marching Cubes Algorithm. Available online: http://iqc.udg.es/cat/similarity/ASA/mca.html (accessed on 28 June 2020).
  16. Ziegler, G.; Theobalt, C.; Seidel, H.; Kobbelt, L.; Kuhlen, T.W.; Aach, T.; Westermann, R. On-the-fly Point Clouds through Histogram Pyramids. In 11th International Fall Workshop on Vision, Modeling and Visualization; IOS Press: Amsterdam, The Netherlands, 2006. [Google Scholar]
  17. Dyken, C.; Ziegler, G.; Theobalt, C.; Seidel, H.P. High-speed marching cubes using histopyramids. In Computer Graphics Forum; Blackwell Publishing Ltd.: Oxford, UK, 2008; pp. 2028–2039. [Google Scholar]
  18. Haque, H.F.; Rahman, A.; Ashraf, M.S.; Shatabda, S. Wavelet and pyramid histogram features for image-based leaf detection. In Emerging Technologies in Data Mining and Information Security; Springer: Singapore, 2019; pp. 269–278. [Google Scholar]
  19. Lavoué, G. A local roughness measure for 3D meshes and its application to visual masking. ACM Trans. Appl. Percept. (TAP) 2009, 5, 1–23. [Google Scholar] [CrossRef]
  20. Wang, K.; Torkhani, F.; Montanvert, A. A fast roughness-based approach to the assessment of 3D mesh visual quality. Comput. Graph. 2012, 36, 808–818. [Google Scholar] [CrossRef]
Figure 1. (a) Bone fracture displays in 3D visualization software, the Sobel-filter is used for detecting the outline of bone [11]; (b) CT image slides and (c) 3D visualization [4].
Figure 1. (a) Bone fracture displays in 3D visualization software, the Sobel-filter is used for detecting the outline of bone [11]; (b) CT image slides and (c) 3D visualization [4].
Jimaging 06 00088 g001
Figure 2. 3D human head model, constructed by marching cubes method (the original source file of this figure is [12]).
Figure 2. 3D human head model, constructed by marching cubes method (the original source file of this figure is [12]).
Jimaging 06 00088 g002
Figure 3. The configuration of marching cubes based on [6].
Figure 3. The configuration of marching cubes based on [6].
Jimaging 06 00088 g003
Figure 4. The connection of vertices to form the cube.
Figure 4. The connection of vertices to form the cube.
Jimaging 06 00088 g004
Figure 5. (a) The configuration of index; (b) The example of index contains value “01000001” and the triangular cross-section according to index value.
Figure 5. (a) The configuration of index; (b) The example of index contains value “01000001” and the triangular cross-section according to index value.
Jimaging 06 00088 g005
Figure 6. The direction of the triangular cross-section determined by normal vector [15].
Figure 6. The direction of the triangular cross-section determined by normal vector [15].
Jimaging 06 00088 g006
Figure 7. The input image with 128 × 128 segments.
Figure 7. The input image with 128 × 128 segments.
Jimaging 06 00088 g007
Figure 8. The CT images sequence used in 3D medical rendering.
Figure 8. The CT images sequence used in 3D medical rendering.
Jimaging 06 00088 g008
Figure 9. The vertices of cubes used for creating the histogram pyramids index.
Figure 9. The vertices of cubes used for creating the histogram pyramids index.
Jimaging 06 00088 g009
Figure 10. Histogram pyramids traversal.
Figure 10. Histogram pyramids traversal.
Jimaging 06 00088 g010
Figure 11. The image slides used as input data.
Figure 11. The image slides used as input data.
Jimaging 06 00088 g011aJimaging 06 00088 g011b
Figure 12. The screen captured of 3D medical visualization program.
Figure 12. The screen captured of 3D medical visualization program.
Jimaging 06 00088 g012
Figure 13. (a) The surface rendering using the traditional marching cubes [14]; (b) the surface rendering using the marching cubes with histogram pyramids.
Figure 13. (a) The surface rendering using the traditional marching cubes [14]; (b) the surface rendering using the marching cubes with histogram pyramids.
Jimaging 06 00088 g013
Figure 14. 3D rendering results and their 3D printing.
Figure 14. 3D rendering results and their 3D printing.
Jimaging 06 00088 g014aJimaging 06 00088 g014b
Table 1. The experimental results.
Table 1. The experimental results.
ModelMesh Density (Vertices)Rendering Times (Sec.)% Times Reduced
Marching Cubes [14]Marching Cubes with Histogram PyramidsMarching Cubes [14]Marching Cubes with Histogram Pyramids
Sphenoid45,44038,8000.3450.29514.49
Right-temporal39,60032,2000.3380.27718.04
Right-maxilla38,56031,9000.3240.26817.28
Ethmoid49,80043,8000.3820.33312.82
Ankle bone63,12052,9000.5950.50015.96
Teeth69,76059,2000.4670.39714.98
----Average15.59
Back to TopTop