# Multi-Sensor Image Fusion Method for Defect Detection in Powder Bed Fusion

^{1}

^{2}

^{3}

^{4}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. FMP Image Fusion Method

#### 2.1. FDST

^{h}, vertical cone C

^{v}, cross line of cone C

^{×}, and low-frequency C

^{0}[34]. The decomposition method is shown in Figure 1.

^{h}and C

^{v}. At the boundary of the cone ${\psi}_{j,k,m}^{h\times v}={\psi}_{j,k,m}^{v}+{\psi}_{j,k,m}^{h}+{\psi}_{j,k,m}^{\times}$, the discrete shear wave transform can be defined as:

#### 2.2. MSSTO

#### 2.3. Improved PCNN

_{ij}and connection input channel L

_{ij}through the branching tree. In the modulation domain, the neuron internal activity term U

_{ij}combines the decaying feedback input F

_{ij}and the connecting input channel L

_{ij}. Finally, by comparing the internal activity term U

_{ij}with the dynamic threshold E

_{ij}, the neuron decides whether to generate a spike or not. The mathematical model of PCNN neuron discrete is expressed as Equations (14) and (15):

_{ijkl}links the weight matrix, β

_{ij}is the link strength, and Y

_{ij}is the output item.

## 3. Experimental Studies and Discussion

#### 3.1. FMP Image Fusion Procedure

- Using the FDST to decompose the registered image A and image B into low-frequency sub-band coefficients and high-frequency sub-band coefficients, respectively.
- In the FDST transform domain, the MSSTO transform is used to extract the image detail bright and dark information in the low-frequency sub-band coefficients of image A and image B, respectively.
- The light and dark information of the image extracted by MSSTO are merged into the low-frequency coefficients after fusion, and the low-frequency fusion coefficients are obtained.
- In the FDST transform domain, the modified spatial frequency (MSF) is used to extract the gradient energy of the image in the vertical, horizontal, and diagonal directions, and the high-frequency sub-band coefficient MSF value is calculated, which is used as the external excitation of the PCNN.
- Using the PCNN criterion to obtain high-frequency fusion coefficients.
- The final fused image is reconstructed from the fused low-frequency sub-band fusion coefficients and the high-frequency sub-band fusion coefficients using the FDST inverse transform.

#### 3.2. Experimental Setup

#### 3.3. Defect Detection and Discussion

## 4. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- Peng, T.; Chen, C. Influence of energy density on energy demand and porosity of 316L stainless steel fabricated by selective laser melting. Int. J. Precis. Eng. Manuf. Green Technol.
**2018**, 5, 55–62. [Google Scholar] [CrossRef] - Echeta, I.; Feng, X.; Dutton, B.; Leach, R.; Piano, S. Review of defects in lattice structures manufactured by powder bed fusion. Int. J. Adv. Manuf. Technol.
**2019**, 106, 2649–2668. [Google Scholar] [CrossRef][Green Version] - Yang, L.; Lo, L.; Ding, S. Selective laser melting of aluminium components. J. Mater. Process. Technol.
**2011**, 211, 275–284. [Google Scholar] - Sing, S.; Huang, S.; Goh, G.; Tey, C.; Tan, J.; Yeong, W. Emerging metallic systems for additive manufacturing: In-situ alloying and multi-metal processing in laser powder bed fusion. Prog. Mater. Sci.
**2021**, 119, 100795. [Google Scholar] [CrossRef] - Wei, C.; Li, L. Recent progress and scientific challenges in multi-material additive manufacturing via laser-based powder bed fusion. Virtual Phys. Prototyp.
**2021**, 16, 347–371. [Google Scholar] [CrossRef] - Vasileska, E.; Demir, A.G.; Colosimo, B.M.; Previtali, B. Layer-wise control of selective laser melting by means of inline melt pool area measurements. J. Laser Appl.
**2020**, 32, 022057. [Google Scholar] [CrossRef] - Verhaeghe, F.; Craeghs, T.; Heulens, J.; Pandelaers, L. A pragmatic model for selective laser melting with evaporation. Acta Mater.
**2009**, 57, 6006–6012. [Google Scholar] [CrossRef] - Aboulkhair, N.T.; Simonelli, M.; Parry, L.; Ashcroft, I.; Tuck, C.; Hague, R. 3D printing of Aluminium alloys: Additive Manufacturing of Aluminium alloys using selective laser melting. Prog. Mater. Sci.
**2019**, 106, 100578. [Google Scholar] [CrossRef] - Yan, X.; Chen, C.; Chang, C.; Dong, D.; Zhao, R.; Jenkins, R.; Wang, J.; Ren, Z.; Liu, M.; Liao, H.; et al. Study of the microstructure and mechanical performance of CX stainless steel processed by selective laser melting (SLM). Mater. Sci. Eng.
**2020**, 781, 139227. [Google Scholar] [CrossRef] - Al-Rubaie, K.S.; Melotti, S.; Rabelo, A.; Paiva, J.M.; Elbestawi, M.A.; Veldhuis, S.C. Machinability of SLM-produced Ti6Al4V titanium alloy parts. J. Manuf. Process.
**2020**, 57, 68–786. [Google Scholar] [CrossRef] - Chan, Y.F.; Chen, C.J.; Zhang, M. Review of on-line monitoring research on metal additive manufacturing process. Mater. Rep.
**2019**, 33, 2839–2867. [Google Scholar] - Tapia, G.; Elwany, A. A Review on Process Monitoring and Control in Metal-Based Additive Manufacturing. J. Manuf. Sci. Eng.
**2014**, 136, 060801. [Google Scholar] [CrossRef] - Yakout, M.; Elbestawi, M.; Veldhuis, S.C. A Review of Metal Additive Manufacturing Technologies. Solid State Phenom.
**2018**, 278, 1–14. [Google Scholar] [CrossRef] - Zur Jacobsmühlen, J.; Kleszczynski, S.; Schneider, D.; Witt, G. High-resolution imaging for inspection of laser beam melting systems. In Proceedings of the IEEE International Instrumentation and Measurement Technology Conference, Minneapolis, MN, USA, 6–9 May 2013; pp. 707–712. [Google Scholar]
- Grasso, M.; Laguzza, V.; Semeraro, Q.; Colosimo, B.M. In-process monitoring of selective laser melting: Spatial detection of defects via image data analysis. J. Manuf. Sci. Eng.
**2017**, 139, 051001. [Google Scholar] [CrossRef] - Caltanissetta, F.; Grasso, M.; Petro, S.; Colosimo, B.M. Characterization of in-situ measurements based on layerwise imaging in laser powder bed fusion. Addit. Manuf.
**2018**, 24, 183–199. [Google Scholar] [CrossRef] - Zhang, Y.; Hong, G.S.; Ye, D.; Zhu, K.; Fuh, J.Y. Extraction and evaluation of melt pool, plume and spatter information for powder-bed fusion AM process monitoring. Mater. Des.
**2018**, 156, 458–469. [Google Scholar] [CrossRef] - Craeghs, T.; Bechmann, F.; Berumen, S.; Kruth, J.P. Feedback control of Layerwise Laser Melting using optical sensors. Phys. Procedia
**2010**, 5, 505–514. [Google Scholar] [CrossRef][Green Version] - Furumoto, T.; Egashira, K.; Munekage, K.; Abe, S. Experimental investigation of melt pool behaviour during selective laser melting by high-speed imaging. CIRP Ann.
**2018**, 67, 253–256. [Google Scholar] [CrossRef] - Berumen, S.; Bechmann, F.; Lindner, S.; Kruth, J.P.; Craeghs, T. Quality control of laser- and powder bed-based Additive Manufacturing (AM) technologies. Phys. Procedia
**2010**, 5, 617–622. [Google Scholar] [CrossRef][Green Version] - Gould, B.; Wolff, S.; Parab, N.; Zhao, C.; Lorenzo-Martin, M.C.; Fezzaa, K.; Greco, A.; Sun, T. In Situ Analysis of Laser Powder Bed Fusion Using Simultaneous High-Speed Infrared and X-ray Imaging. JOM
**2020**, 73, 201–211. [Google Scholar] [CrossRef] - Peng, X.; Kong, L.; Chen, Y.; Shan, Z.; Qi, L. Design of a Multi-sensor Monitoring System for Additive Manufacturing Process. Nanomanuf. Metrol.
**2020**, 3, 142–150. [Google Scholar] [CrossRef] - Gusarov, A.; Okun’Kova, A.A.; Peretyagin, P.; Zhirnov, I.V.; Podrabinnik, P.A. Means of Optical Diagnostics of Selective Laser Melting with Non-Gaussian Beams. Meas. Technol.
**2015**, 58, 872–877. [Google Scholar] [CrossRef] - Gaikwad, A.; Giera, B.; Guss, G.M.; Forien, J.B.; Matthews, M.J.; Rao, P. Heterogeneous sensing and scientific machine learning for quality assurance in laser powder bed fusion– A single-track study. Addit. Manuf.
**2020**, 36, 101659. [Google Scholar] [CrossRef] - Yakout, M.; Phillips, I.; Elbestawi, M.A.; Fang, Q. In-situ monitoring and detection of spatter agglomeration and delamination during laser-based powder bed fusion of Invar 36. Opt Laser Technol.
**2020**, 136, 106741. [Google Scholar] [CrossRef] - Liu, Z.; Feng, Y. Infrared and Visible Image Fusion Based on Compressive Sensing and OSS-ICA-Bases. In Proceedings of the 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece, 7–10 October 2018; pp. 1852–1856. [Google Scholar] [CrossRef]
- Shahdoosti, H.R.; Ghassemian, H. Combining the spectral PCA and spatial PCA fusion methods by an optimal filter. Inf. Fusion
**2016**, 27, 150–160. [Google Scholar] [CrossRef] - Xie, Z.; Zhang, S.; Yu, X.; Liu, G. Infrared and visible face fusion recognition based on extended sparse representation classification and local binary patterns for the single sample problem. J. Opt. Technol.
**2019**, 86, 408–413. [Google Scholar] [CrossRef] - Toet, A.; Van Ruyven, L.J.; Valeton, J.M. Merging Thermal and Visual Images By A Contrast Pyramid. Opt. Eng.
**1989**, 28, 287789. [Google Scholar] [CrossRef] - Srivastava, R.; Khare, A. Multifocus noisy image fusion using contourlet transform. Imaging Sci. J.
**2015**, 63, 1743131–1743422. [Google Scholar] [CrossRef] - Guo, B.; Zhang, Q.; Hou, Y. Region-based fusion of infrared and visible images using nonsubsampled contourlet transform. Chin. Opt. Lett.
**2008**, 2008, 338–341. [Google Scholar] [CrossRef][Green Version] - Kamstra, L. Nonlinear Discrete Wavelet Transforms over Finite Sets and an Application to Binary Image Compression. J. Math. Imaging Vis.
**2005**, 23, 321–343. [Google Scholar] [CrossRef] - Ioannidou, S.; Karathanassi, V. Investigation of the Dual-Tree Complex and Shift-Invariant Discrete Wavelet Transforms on Quickbird Image Fusion. IEEE Geosci. Remote Sens. Lett.
**2007**, 4, 166–170. [Google Scholar] [CrossRef] - Bai, Y.J.; Xiong, S.H.; Wu, X.Q. Infrared and visible images fusion based on FDST and MSS. Sci. Technol. Eng.
**2017**, 17, 215–219. [Google Scholar] - Bai, X. Infrared and visual image fusion through feature extraction by morphological sequential toggle operator. Infrared Phys. Technol.
**2015**, 71, 77–86. [Google Scholar] [CrossRef] - Lian, J.; Yang, Z.; Liu, J.; Sun, W.; Zheng, L.; Du, X.; Yi, Z.; Shi, B.; Ma, Y. An Overview of Image Segmentation Based on Pulse-Coupled Neural Network. Arch. Comput. Methods Eng.
**2019**, 28, 387–403. [Google Scholar] [CrossRef] - Garima, P.; Umesh, G. A compendious study of super-resolution techniques by single image. Optik
**2018**, 166, 147–160. [Google Scholar] - Bai, X. Morphological infrared image enhancement based on multi-scale sequential toggle operator using opening and closing as primitives. Infrared Phys. Technol.
**2015**, 68, 143–151. [Google Scholar] [CrossRef] - Li, M.J.; Yu, B.D.; Xiao, L.W. Image Fusion Algorithm Based on Wavelet Transform and Laplacian Pyramid. Adv. Mater. Res.
**2013**, 2013, 2846–2849. [Google Scholar] [CrossRef] - Toet, A. Image fusion by a ratio of low pass pyramid Pattern Recogn. Pattern Recogn. Lett.
**1989**, 9, 245–253. [Google Scholar] [CrossRef] - Lewis, J.J.; O’Callaghan, R.J.; Nikolov, S.G.; Bull, D.R.; Canagarajah, N. Pixel- and region-based image fusion with complex wavelets. Inf. Fusion
**2007**, 8, 119–130. [Google Scholar] [CrossRef] - Hyötyniemi, H. Multivariate Regression-Techniques and Tools; Helsinki University of Technology, Control Engineering Laboratory: Helsinki, Finland, 2001; 207p. [Google Scholar]
- Shu, Z. PCNN Model Automatic Linking Strength Determination Based on Geometric Moments in Image Fusion. J. Algorithms Comput. Technol.
**2014**, 8, 17–26. [Google Scholar] [CrossRef] - Zhao, L.M. Research on Insulator Defect Detection Method Based on Image Fusion; Xi’an Shiyou University: Xi’an, China, 2021. [Google Scholar]
- Liu, M.; Yi, W.C.; Zhao, Y.J. Research on remote sensing image dehazing based on generative adversarial networks. Spacecr. Recovery Remote Sens.
**2020**, 41, 14–20. [Google Scholar] - Ou, J.M.; Hu, X.; Yang, J.X. Low-light image enhancement algorithm based on improved retinex-net. Int. J. Pattern Recog.
**2021**, 34, 77–86. [Google Scholar]

**Figure 3.**The schematic diagram of the proposed FMP: FBIF: fusion of bright feature; FDIF: fusion of dark feature; FHFC: fusion of high-frequency coefficient; FLFC: fusion of low-frequency coefficient; C

_{low}: low-frequency coefficient; C

_{high}: high-frequency coefficient.

**Figure 4.**Schematic diagram of the experimental setup for defect detection in PBF processes: IC: Infrared channel imaging system; VC: Visible channel imaging system; BS: Beam-splitter; FL: Filters; IS: Infrared channel image sensor; VS: Visible channel image sensor; PC: computer.

**Figure 7.**Comparison of objective evaluation indicators of dataset 1 under different fusion algorithms.

**Figure 8.**Comparison of objective evaluation indicators of dataset 2 under different fusion algorithms.

Design Parameters | Visible System | Infrared System |
---|---|---|

Wavelength (μm) | 0.4–0.7 | 0.9–1.7 |

Image sensor type | CMOS | InGaAs |

pixel count | 7728 × 5368 | 320 × 256 |

Pixel size (μm) | 1.1 | 30 |

Focal length f (mm) | 50 | 50 |

F-number | 2.5 | 1.5 |

Object field size (mm) | 51.90 × 36.30 | 58.50 × 46.98 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Peng, X.; Kong, L.; Han, W.; Wang, S.
Multi-Sensor Image Fusion Method for Defect Detection in Powder Bed Fusion. *Sensors* **2022**, *22*, 8023.
https://doi.org/10.3390/s22208023

**AMA Style**

Peng X, Kong L, Han W, Wang S.
Multi-Sensor Image Fusion Method for Defect Detection in Powder Bed Fusion. *Sensors*. 2022; 22(20):8023.
https://doi.org/10.3390/s22208023

**Chicago/Turabian Style**

Peng, Xing, Lingbao Kong, Wei Han, and Shixiang Wang.
2022. "Multi-Sensor Image Fusion Method for Defect Detection in Powder Bed Fusion" *Sensors* 22, no. 20: 8023.
https://doi.org/10.3390/s22208023