A State-of-the-Art Review of Non-Destructive Testing Image Fusion and Critical Insights on the Inspection of Aerospace Composites towards Sustainable Maintenance Repair Operations

Abstract

Non-destructive testing (NDT) of aerospace structures has gained significant interest, given its non-destructive and economic inspection nature enabling future sustainable aerospace maintenance repair operations (MROs). NDT has been applied to many different domains, and there is a number of such methods having their individual sensor technology characteristics, working principles, pros and cons. Increasingly, NDT approaches have been investigated alongside the use of data fusion with the aim of combining sensing information for improved inspection performance and more informative structural health condition outcomes for the relevant structure. Within this context, image fusion has been a particular focus. This review paper aims to provide a comprehensive survey of the recent progress and development trends in NDT-based image fusion. A particular aspect included in this work is providing critical insights on the reliable inspection of aerospace composites, given the weight-saving potential and superior mechanical properties of composites for use in aerospace structures and support for airworthiness. As the integration of NDT approaches for composite materials is rather limited in the current literature, some examples from non-composite materials are also presented as a means of providing insights into the fusion potential.

1. Introduction

Competitiveness in aviation operations has depended on the constant adoption of continuous flights, requiring sustainable maintenance services. A maintenance and repair organisation (MRO) is a service establishment that provides maintenance, repair and overhaul functions to operators to keep their aircraft utilised. While airlines had their engineering departments to perform maintenance at the initial time of commercial aviation, outsourcing options have become popular due to the increment in operation volume [1]. In addition, regulatory authorities have forced companies to organise their cores as directives of regulations. MROs have introduced flexibility between aviation operation and maintenance processes, providing a variety of special maintenance centres that specialise in a particular component, such as engine or structural maintenance [2]. MROs have the vital mission in the aviation market of not only maintaining the sustainability of operations but also reducing costs because aircraft grounding impacts efficiency and “the ratio of” flight time to operating expenses. The daily cost of an aircraft being out of operation can be considered to be GBP 200,000 on average [3] (based on reasonable average estimates: a narrow body aircraft (200 passengers × 8 journeys × GBP 100 = GBP 160,000); wide body (350 passengers × 1 journey × GBP 800 = GBP 280,000)). Moreover, unscheduled maintenance due to unexpected problems also highly impacts expenses. According to the IATA, wide-body jets were predicted to have a cost of more than USD 6.5 billion in unplanned maintenance owing to delays and cancellations in 2017. Appropriately scheduled maintenance, including shorter downtimes, avoiding unnecessary disassembly and minimizing unexpected delays and possible human errors, is crucial to reducing the grounding time of aircraft.

Structural components are a significant element in terms of inspection and hence maintaining airworthiness. MROs spend significant time checking structural components, which accumulates to typically ≈30% of the costs [4]. Note that hidden (non-visible) defects (damages) inside the structure hinder operation in the medium to longer term. Several conventional inspection methods need a removal process that lengthens the maintenance time. Advanced non-destructive inspection/technique (NDI/NDT) methods are already popular in structural checks for detecting the occurrence of any defect in an early stage. The NDT is a powerful tool for detecting damages in the structures that formed during both the manufacturing and service stages [5,6]. However, it is a formidable task to design a generic inspection procedure, considering the restrictions caused by regulations, operations, capabilities and costs. Obtaining more informative and determinative inspection outcomes should provide cost-effective maintenance which reduces the time and effort for individual inspection methods, especially after the pandemic period, in which there was a high workforce loss of 2.3 million personnel in global aviation [7].

Composite structures have become attractive for many industrial sectors due to their good strength and stiffness per unit weight [8,9,10]. In aviation, the volume of the composite structures used in an aircraft, such as the fuselage, wings, and empennage, has risen day by day. Accordingly, it is a vital requirement to identify possible damage to aircraft structures regarding airworthiness. It should be noted that understanding the structure will help in exploring the problems and related solutions. For this reason, possible defect types that can occur in a composite structure must first be understood. Then, the characterisation and measuring of a defect in the structure must be performed precisely if possible. Here, measuring refers to correctly establishing the defect’s location, size and significance. Structural health monitoring (SHM) methods have ensured useful strategies for inspecting aerospace composites in various recent studies [11,12,13,14].

Defect localisation and characterisation are vital steps in NDT application, with their own parameters that affect maintaining sustainable aviation operation and airworthiness. That structural recognition and comprehensive knowledge of possible defects will ease maintenance operations, followed by sustainable aviation, is not in doubt. In this article, the fusion approaches already used in NDT studies have been investigated in terms of application potential for aerospace structures. The related algorithms and their evaluation are presented and discussed. It should be emphasised that NDT fusion is still a fresh research topic, and utilizing image fusion strategies for NDT defect detection application when applying the individual inspection methods is not satisfactory [15]. This comprehensive review study, which stems from a current PhD study exploring advanced NDT fusion of an ultrasonic phased array and pulsed thermography for inspection applications in composite structures, contributes the following:

• It introduces the current studies on NDT image fusion and evaluation strategies;
• It provides a reference source for the NDT-based fusion attempts;
• It reports the limited amount of work existing in fusing NDT techniques (particularly for composite structures);
• It fills the research gap of comprehensively reviewing image fusion for NDT inspection (particularly towards composite structures).

The rest of this paper is structured as follows. A brief discussion of the primary NDT techniques is given in Section 2. The basic principles of image fusion are discussed in Section 3. Section 4 includes fusion strategies and rules for various NDT techniques discussing principles and examples. Section 5 discusses the evaluation of fusion, focusing on performance metrics. Section 6 provides insights from the survey into NDT fusion and future prospects. Conclusions are drawn in Section 7.

2. NDT Method Preliminaries

There is a variety of inspection technology approaches. Their use depends on the structure or material to be inspected, cost and effectiveness. Method-wise, they offer pros and cons, and hence a requirement analysis can be a good starting point to be able to select an appropriate method for the relevant inspection application. Note that each technique utilises a different sensing technology and property. A useful categorisation of NDT methods relative to defect types has been seen in previous works [16,17,18].

Table 1. NDT method standings relative to defect types.

Defects	X-ray	Ultrasonic	Penetrant	Methods Magnetic Particle	Eddy Current	Thermography	Acoustic Emission
Porosity or Voids	✓	✓	⊙	⊙	⊙	⊙	✓
Delamination	⊙	✓	⊙	X	X	⊙	✓
Debonding	✓	✓	⊙	X	X	⊙	✓
Foreign Bodies	✓	✓	⊙	⊙	⊙	⊙	✓
Cracks	⊙	✓	⊙	✓	✓	⊙	✓
Surface	X	⊙	✓	✓	✓	✓	✓
Internal	✓	✓	X	⊙	⊙	⊙	✓
Limit	Orientation-dependent	Dead zone effect	Only open to surface defects	Ferromagnetic materials only	Conductive materials only	Small thickness	Lack of size and shape
Advantage	Inspection process is simple	Portable and good depth resolution	Suitable for mass-manufactured products	Rapid for complex surfaces	Suitable for hard-to-reach areas	Useful for quick response	Effective for active defects

Applicability → suitable (✓); weak (X); limited (⊙).

lists a snapshot of the standings of such methods, which relate to those presented in this review paper [17]. We also include an indicator of “applicability” as suitable, weak or limited. This relates to the level of information outcome of the NDT method for the said defect type and defect detection performance (ability). For example, X-ray-based NDT is challenged by detecting delamination defects in general, with open delamination possible to detect using X-ray computed tomography (CT). Moreover, we give a brief description of these main inspection methods.

X-ray radiography is a well-established technique which provides a permanent record (using the X-ray-sensitive film) and is used in NDT inspection. X-rays (or $\gamma$-rays, which have a better penetration ability) passing through the sample and the radiation intensity are different if any defects are encountered (see Figure 1c). This principle was successfully used to discriminate different materials for an object by using neutron and X-ray beams simultaneously [19]. However, it can provide a 2D line scan, and it might be difficult to identify the depth information in the case of defect characterisation or localisation. X-ray CT is a developed version that can give 3D images with depth information. The authors of [20] applied X-ray CT with terahertz testing to obtain 3D images for CFRP composite defect characterisation. Even though both can give 3D images using different wavelengths and frequencies of electromagnetic radiation, they were used together to obtain the optimum contrast and resolution. It should be noted that the inspection process is simple, while the equipment can be complex and have a high cost. Radiography also raises health concerns and is performed with care. In addition, both have limited capabilities because of the orientation of delamination or crack defects, as shown in Table 1 [17].

Another important method is ultrasonic inspection, which uses ultrasound wave propagation inside materials (Figure 1b). Using analysis of the reflections because of the abnormalities in an internal structure allows measuring the characteristics such as the thickness or depth of any defect. Although advanced ultrasonic tools provide 3D C-scans and cross-sectional B-scans to obtain sufficient information about a structure; quantifying and locating defects might be challenging and require complementary techniques when multiple close reflections [21] or near-surface defects [22] occur. The authors of [22] also showed the challenging points of processing ultrasonic data due to the multiple echoes caused by structural noise.

Figure 1. Basic representations of thermographic, ultrasonic, X-ray and eddy current NDT methods. (a) Schematic representation of pulsed thermography [23]. (b) Schematic representation of ultrasonic testing [24]. (c) Schematic representation of X-ray radiography. (d) Eddy current defect representation [25].

Figure 1. Basic representations of thermographic, ultrasonic, X-ray and eddy current NDT methods. (a) Schematic representation of pulsed thermography [23]. (b) Schematic representation of ultrasonic testing [24]. (c) Schematic representation of X-ray radiography. (d) Eddy current defect representation [25].

An efficient method for surface and near-surface defects is infrared thermography, which uses thermal wave propagation caused by temperature differences. Figure 1a presents a basic schematic representation of pulsed thermographic inspection. The authors of [26] presented an effective low-velocity impact characterisation for the matrix, delamination and fibre damage of a GFRP structure with thermography. Surface temperature maps have been used to observe local temperature changes and maximum temperature differences to help identify defect types. It should be noted that thin materials restrict infrared thermography due to the challenges of heat penetration into deeper regions of the material [27].

Liquid penetrant testing that uses the capillary action principle might provide a practical solution when surface examination of mass-produced items is performed [28]. A particular liquid, known as a penetrant, fills any discontinuities that may be present when applied to a specimen’s surface. Thus, it can be detected by fluorescence or human eyes according to the penetrant’s dye type. The main limitation of this technique is being restricted to surface or open-to-surface defects.

As can be seen from Table 1, the magnetic particle and eddy current (Figure 1d) methods are the other surface-effective techniques that use similar principles of forming regular magnetic fields on uniform materials. In the case of a discontinuity, the magnetic field deforms because it tends to follow the path having the least resistance [28,29]. However, magnetic particles and eddy currents are limited by the material types of ferromagnetic and conductive types, respectively.

Finally, acoustic emission is a reliable and real-time technique that detects the materials’ transient stress waves for early fault detection [30]. While the approach is highly sensitive to small changes in an active defect, it tends not to be suitable for constant (static) defects. Moreover, it seems to be an attractive method for electrical defects rather than mechanical ones, as well as having issues with the defect size and shape aspects [31].

3. Image Fusion

Fusion is a valuable process that combines or integrates information from different sensors or sources. Image fusion is the technique of integrating several input images of the same scene into a single fused image, intending to synthesise better quality and perception of the image features. Note that an image is viewed as a light intensity function $f(x,y)$, where x and y are the spatial coordinates of each pixel and f refers to the proportional value of the brightness at each coordinate [32]. Image fusion can combine the brightness, colour, distance, temperature and other scenery features of images. It should be noted that image fusion does not simply mean “matching” or “overlaying” of multiple images; it is a process followed to form a renewed and more meaningful image outcome [33]. Image fusion can reduce data dimensionality while keeping important information content, hence supporting a lower storage cost [34]. In principle, image fusion is seen as performing three main steps: registration, information fusion and representation (refer to Figure 2) [35].

NDT-based image fusion is a challenging topic due to the varied defect sensitivity and failure mechanisms of the inspection methods. However, these methods incorporate complementary features to obtain reliable and accurate defect characterisation and take the best action against an existing situation [36]. In such a fusion application, registration is the first significant step in having a proper reference for a fused image. The information from NDT sources should be aligned correctly to obtain the sense results. As well as the common feature point-based registration, such as for corners or edges, the authors of [37] presented a type of marker-based referencing which uses the highly reflective or emissive marks on the surface of the inspected material when the feature points are not detectable enough. As a vital matter, especially in the medical image fusion field, registration has been widely investigated with methods ranging from transformation or similarity-based methods [38,39] to novel AI-based strategies [40,41] in the literature.

Similar to other fields, NDT-based fusion can be classified as level-based by the forms of information flow: signal-level, pixel-level, feature-level and decision-level fusion [42]. In signal-level fusion, a fused signal has been formed to obtain better quality by combining raw sensor signals, which can be modelled as a random variable. As an example, the authors of [43] applied superposition-based fusion at the signal level to fuse simulated defect and experimental defect-free noise. Figure 3 shows a superposition outcome that is comparable to the presence of the defect in the sample. The authors have stated that the outcomes have accurate amplitudes for defects as well as back walls and shadowing effects due to the model’s destructive signal interfering with the back wall signal. This level of fusion has the highest registration needs in the time and space domain because of considering an optimal signal distribution problem [44].

Pixel-level fusion integrates the physical parameters of the source pixels given at specific spatial locations and their neighbourhoods and creates a pixel for the fused image. The authors of [45] applied the pixel-wise averaging, PCA and wavelet methods by considering the individual pixel values for each method and combining them. Although this combination has a better depth resolution and signal-to-noise ratio than the source images, it is restricted by the large size of the data to be calculated and is time-consuming.

The benefit of image fusion based on feature-level fusion, which acts on properties such as the edge, shape and texture, is that it achieves a certain amount of information compression and supports real-time processing. A color moment (including the mean, variance and skewness features) and texture (mean, standard deviation, smoothness, third-order moments, consistency and entropy features) fusion example was carried out in [46] to use the results as the input for a recognition training algorithm. The main advantage of this fusion is having fewer dimensional feature vectors thanks to the compression. In addition, the study proved that fusion has increased recognition accuracy and reduces the false detection rate.

Finally, decision-level fusion is a cognitive strategy where source images are converted to generate an independent target attribute estimation which is already a representative symbol for information extraction [44]. A partial density-based fusion was proposed in [47], using the computation of the single-sensor spatial density estimation. The calculated partial densities, instead of feature extraction or pixel-wise merging, are merged with a basic summation fusion rule ignoring the maximum value to provide false alarm reduction. The main advantage is that this technique explicitly treats the localisation uncertainty caused by registration problems. Consequently, the selection of the fusion level that will be applied depends on various parameters, such as the image sources, processed data type and information that the image has. This should be performed in a scrutinizing manner according to the overall expectations from the sources and fusion. Now, we will present current fusion applications that are already used for different materials and various fields within a generic classification according to the methods they are based on in the next section.

4. Image Fusion Applications in NDT Inspection

4.1. General Algebraic Approaches

General algebraic (basic mathematical) approaches have been utilised to combine the source signals, features or estimations pixel-wise as a fusion rule. The selection of the appropriate fusion rule might depend on the target that needs to be achieved and the nature of the data collected from different sources. For instance, while division and subtraction are suitable for detecting temporal changes in different measurements, the maximum approach might be more applicable to finding complementary information, as seen in [48]. The study performed a porosity and distribution analysis for concrete structures using ultrasonic and ground-penetrating radar (GPR). After the processing (weighting and filtering datasets) and reconstruction (synthetic aperture focusing technique (SAFT)) steps for both individual methods, the amplitude values of both methods are added to obtain improved depth information, adapting the depth point of the maximum amplitude.

The size of the dataset is another crucial parameter in selecting the fusion rule that will be implemented. For example, the data size must be the same for an addition or division operation. In addition, a data acquisition system might affect the selection of fusion methods due to the nature of the produced data. A bimodal imaging system may need supervision to prevent interference while using two separate imaging systems. Dual-imaging fusion based on X-ray and neutron images has been offered for material identification using attenuation differences [19]. It has been used as a single-electron linear acceleration system with a slight time delay to prevent the system from that type of interference. It should be noted that here, the acquisition systems for X-ray and neutron inspection have almost the same beam geometry, and data are ready to fuse directly. Pixel-wise matching has been performed using extracted neutron attenuation and photon attenuation values compared with the neutrons and photons of air. As a result of this pixel-wise fusion, a bivariate histogram has been provided to identify different materials and their thicknesses without prior knowledge.

The basic mathematical fusion rules can also be used mainly for comparison or complementary with other fusion rules. A good example application of this is in [49], which includes the basic rules, wavelets and decision-level fusion (kernel density estimation (KDE)) together. A steel-bearing shell specimen with 15 surface grooves and specific dimensions was used experimentally. Although the sum and max fusion rules provided poor performance, all other rules showed a satisfactory flaw detection capability rather than the single-sensor yield. Another study in the manufacturing field [50] proposed pixel-based image merging to improve object measurement using the multi-energy stacks of computed tomography (CT) images. The primary purpose behind the fusion process is to eliminate measurement difficulties caused by the sample’s high aspect ratio (thickness to length) and the variety of absorbing materials. Fusion was applied by finding outshined pixels of different energy stacks, followed by replacing those pixels. A mean deviation-based evaluation was applied, and the fusion results showed less deviation than the mono-energetic measurement.

Table 2. Brief information on current studies which use general algebraic fusion rules.

Sources	Material	Defect Types	Aim	Fusion Rule	Reference
Ultrasonic and ground penetration radar	Concrete	Porosity or voids	Investigating porosity and distribution of fibres	Sum, difference, quotient, maxima and average	[48]
X-ray and neutron images	* 1	Foreign body	Differentiating materials and thicknesses	Bivariate histogram	[19]
Ultrasonic simulation and experimental measurements	Copper	Holes	Avoiding costly physical samples	Summation (superposition)	[43]
Eddy current, active thermography and magnetic flux leakage	Steel	Cracks	Preventing single-sensor domination	Minimum, geometric mean, harmonic mean, product, median, sum, sumIgnoreMax and maximum	[49]
Computed tomography with different configurations	Mix material	Foreign body	Minimizing measurement issues	Threshold-based pixel replacement	[50]

¹ This study tried to detect different materials for an object.

presents information on the material, defect and source details of the aforementioned studies. There are more studies available in the papers [46,51,52,53] that used these basic rules in a complementary manner. However, the details provided in the papers are limited. In addition,

Table 3. Data fusion operations at signal level (amplitudes).

Method	Equation 1	Remark
Summation	$\tilde{F}={\displaystyle \sum _i}{F}_i$	Suitable for fusion of complete datasets (same size or resolution)
Difference	$\tilde{F}={\displaystyle \sum _i}{F}_i-{\displaystyle \sum _j}{F}_j$	As explained above and for fusion of simulated and real datasets
Quotient	$\tilde{F}={\displaystyle \sum _i}{F}_i/{\displaystyle \sum _j}{F}_j$	as explained above
Average	$\tilde{F}={\displaystyle \sum _i}{F}_i/k$	Fusion of incomplete datasets as well
Maximum	$\tilde{F}=max\bigcup _i{F}_i$	as explained above
Product	$\tilde{F}={\displaystyle \prod _i}{F}_i$	From quotient
SumIgnoreMax	$\tilde{F}={\displaystyle \sum _i}{F}_i-\underset{i}{max}{F}_i$	From summation

¹ $\tilde{F}$ = fused dataset; $F_i$, $F_j$ = i and j (source) datasets, respectively; k = number (size) of measurements of the i datasets.

has given mathematical expressions of related fusion rules.

4.2. Principal Component Analysis

Principal component analysis (PCA) is a well-known statistical technique for reducing the dimensionality of a dataset. This is carried out via appropriate coordinate transformation. Here, the main idea is to retain the maximum possible variation while reducing the dimensionality of the dataset. The new set of variables, referred to as the principal components (PCs), is generated in such a way that the first few PCs retain most of the variation (in more detail, the first component retains the maximum variance, with the second one existing in the subspace that is perpendicular to the first PC) [54,55,56]. PCs are linear combinations or mixtures of the original data. Figure 4 illustrates the first and second uncorrelated dimensions as PCs. PCA has already been used in various domains, such as pattern recognition, image compression, dimensionality reduction and feature extraction [57,58,59,60], as well as NDT applications [61,62,63,64,65].

In the context of NDT fusion, a pixel-level fusion using the PCA and DWT methods was proposed to detect inclusive defects (copper sheets) with different depths, sizes and shapes in a GFRP material [66]. Amplitude and phase images from active thermography, extracted using a Fourier transform, were fed for fusion as inputs. The principle components and wavelet coefficients shaped the input images for combination purposes. In terms of fusion efficiency, the metric used was the signal-to-noise ratio (SNR) between the inputs and fused images. It was highlighted in the work that the PCA fusion strategy increased the SNR for the inclusive defects considerably. A similar study [45], including Teflon inclusions of different depths and sizes for GFRP composites, applied pulse compression to thermographic images. For transmission, a low peak power and long-duration modulated wave was used, rather than a short duration and high peak power, to obtain a better depth resolution and SNR for the inhomogeneities in the structure. The study compared averaging-, PCA- and wavelet-based fusions, with the latter providing the highest SNR.

Within the remit of CFRP composite-related studies, (It is noted here that carbon fibre-reinforced polymer (CFRP) composites are the main composite structure used in aerospace structures rather than GFRP.) the authors of [67] proposed a fusion structure that utilises multi-frequency lock-in thermography. This approach explores the advantage of detection characteristics that provide different depth information for various frequencies. In addition, the authors proposed an optimum frequency selection approach considering the material thickness and thermal diffusivity. Phased images formed by the signal variation over time at each pixel were extracted in sequences that indicated thermal changes on the material surface. PCA was used to detect holes at different depths for a single image by decomposing a series of phase images to their principal components. A comparison between PCA-, wavelet- and multi-scale decomposition-based fusion was performed, and the former fusion approach provided the best edge sharpness and hole contrast detection.

Here, only the fusion strategies referring to composite materials are explained in detail. For completeness, we provide information on the fusion approaches that utilised different types of structures (however, the fusion application is a useful example to inform further work on composites) in

Table 4. A list of PCA-based studies for NDT image fusion.

Sources	Material	Defect Types	Aim	Reference
Lock-in thermography	GFRP composite	Foreign body or delamination	To eliminate non-uniform heating, low resolution and noise effects	[66]
Ultrasonic and vibro-acoustographic inspection	Steel pipe	Holes with different shapes	To detect holes and measure their dimensions	[21]
Eddy current pulse thermography	Aluminum alloy or stainless steel	Crack	To eliminate background interference effect	[68]
Thermal and visible images	Steel	Crack or split	To solve heat diffusion, reflection and illumination problems of constituent methods	[69]
Pulsed thermography	GFRP composite	Foreign body or delamination	To enhance the defect detection capability	[45]
Lock-in thermography	SS275 steel plate	Voids or corrosion	To inspect the contaminant liner plate (CLP) against radiation leakage	[70]
Lock-in thermography	CFRP composites	Holes	To find better depth resolution	[67]

with their relevant citations.

4.3. Fourier Transform-Based Approaches

Fourier analysis is a well-known approach in mathematics, covering a large spectrum of applications (approximating a general function with the sums of more straightforward trigonometric functions). Fourier analysis decomposes an image into its sine and cosine components, with the transformation output representing the image in the frequency domain (the input image is the spatial domain). For completeness, general forms of Fourier synthesis and analysis equations, respectively, for a periodic sequence of $x\left(n\right)$ [71] are shown below ($c_k$ and N refer to Fourier coefficients and period, respectively):

$$Synthesis:x\left(n\right)=\sum _{k=0}^{N-1}{c}_k{e}^{j2\Pi kn/N}$$

(1)

$$Analysis:c_k=1/N\sum _{n=0}^{N-1}x\left(n\right)e^{-j2\pi kn/N}$$

(2)

Working in the frequency domain enables easier processing for image fusion. The work in [72] presents a useful example of Fourier-based NDT image fusion. Using a Gaussian function, the authors presented a cone-shaped filter in the Fourier domain of X-ray computed tomography (CT) and ultrasonic testing (UT). The fusion aimed to combine the advantages of the inspection methods, hence minimising issues such as noise, artefacts and low resolution. A test sample containing the specific dimensions of the internal channels was used to perform CT and UT volume measurements. The fused image provided reduced artefacts and improved the edge contrast compared with using the inspection technologies independently. It is noted that, as seen in many works in the literature, the Fourier transform has often been used as a tool to preprocess data prior to feeding them to the fusion rule.

4.4. Wavelet-Based Approaches

The wavelet transform is an appropriate tool to decompose images into high- and low-frequency components. This makes possible analysis of the features of data over a certain scale, such as the edges and texture for an image or the frequency for a signal. Unlike the Fourier transform, which offers a specific frequency wave, it can provide data representation for multiple scales with different versions of wavelets. Shifted and scaled versions of a wavelet can positively impact the resolution in the time or frequency domain according to the selected parameters. The wavelet transform aims to maintain a good resolution in the time and frequency domain, contrary to the Fourier transform [73,74].

For an image, the row and column directions must be handled independently when the original image is divided into four sub-band images. The two-level decomposition structure is illustrated in Figure 5 for illustration purposes. Each row’s data are first used to utilise the high-pass filter H and the low-pass filter L, and then the high- and low-frequency components of the row are obtained by downsampling of the two. The next step is the same in the column base and continuous for the next decomposition levels iteratively. The low-pass portion (LL), called the approximation or wavelet coefficients, contains the average image information, and the high-pass portion (i.e., detail coefficients) has the directional information of the original image. By using this approach recursively, multi-resolution wavelet analysis can be performed [75].

The decomposition level and selection of the proper wavelet filter, one of the most important steps for wavelet application, should be decided according to the structure, aim of the transformation and the information that should be emphasised. The authors of [76] proposed a wavelet energy-based ($E_{min}$) filter selection method which considers that the minimum percentage of the approximation coefficients and detail coefficient will bring minimal noise. This is attributed to the minimum amount of low-frequency information in the approximation coefficient after decomposition. Additionally, a weighted risk factor-based decomposition, including automatic level selection, is used according to the maximum detail coefficients. The primary goal of the proposed strategy is to eliminate noise while maintaining appropriate defect information. The results show that the risk factor-based fusion provided more visual and qualitative detection than the minimum energy-based approach.

Moreover, a number of other studies in the literature attempted to interpret the effectiveness of the decomposition level and selected a filter using multiple options over a specific application [20,77]. For example, a hybrid image fusion structure of wavelet-based multi-scale transforms (W-MST) and saliency region analysis (SRA) was proposed to combine the advantages of the terahertz and X-ray CT techniques [20]. Here, the study compared and contrasted 36 fusion combinations, including various fusion rules, decomposition levels and basis functions. Experimental samples were formed using glass fibre (GFRB) with different-shaped inclusions and defects with various thicknesses and depths. First, saliency maps, which show the indication of the human eyes’ first focus in an image, were obtained for both methods. Then, the main outline in the low-frequency component and edge contour information of the high-frequency sub-bands were transmitted to the last fused image by W-MST. Assessment of these combinations was performed by calculating the standard deviation (SD), which shows the image’s grey level nature, and spatial frequency (SF), which provides the imaging resolution for the defect details. Among these combinations, the energy, average gradient and constant fusion strategies showed the highest increment between the source and fused images in terms of the two indicators. For completeness and useful information to the reader,

Table 5. A summary of the current wavelet applications for NDT image fusion.

Sources	Material	Defect Types	Aim	Reference
Eddy current images (at two different frequencies)	Stainless steel	Machined notches	To eliminate noise	[76]
Terahertz and X-ray CT techniques	GFRP	Foreign body or delamination	To combine higher contrast of terahertz and higher resolution of X-ray	[20]
Infrared, visible light and laser inspections	Copper	Foreign body, voids or cracks	To enhance system stability with the multi-sensor information	[78]
Visible camera images	Wall (non-defined)	Cracks	To minimise the complexity and diversity of wall images	[79]
Ground-penetrating radar (GPR) and ultrasonic echo array (UEA)	Concrete	Geometry or foreign body	To achieve enhanced images of the interior of concrete	[80]
Eddy current pulsed thermography	CFRP-steel structure	Cracks	To remove information on the texture that hides defects	[81]
Ultrasonic and Step-heating thermography inspections	Carbon/epoxy-aluminium	Delamination or debonding	To inspect patch and binding quality	[77]
Eddy current and pulsedthermography	CFRP	Impact	To provide comprehensive information from impacted CFRP structure	[82]
Ultrasonic, thermographic images and CAD model	* 1	Geometry or voids	To provide defect mapping in the partial CAD model	[83]
Eddy current, magnetic flux leakage and flying laser spot thermography inspections	Steel	Cracks	To improve near-surface defect detection in magnetisable and conductive materials	[84]
Terahertz images (in various configurations)	GFRP	Foreign body or delamination	To ease detecting deeper flaws	[85]

¹ The material is mentioned as aeronautical composites without any details.

lists important current studies on wavelet-based fusion strategies for different materials and defects. These include the ones discussed in more detail in this section.

4.5. Learning-Based Approaches

Learning-based probabilistic methods have already become popular in the image fusion field due to their direct link to computer vision problems and capability of automatic feature extraction. In addition, they have significant potential for representing the complex characteristics and relationships of the source images and fusion results. Learning enables more AI-based approaches in inspection applications and can be a useful tool to the inspection expert, supporting more autonomy in inspection applications. In this section, we present studies that are already used for the fusion of NDT systems, with some insights into the proposed methods.

4.5.1. Neural Networks

Artificial neural networks (ANNs) are one of the most useful forms of machine learning (ML) in several fields, such as classification and pattern recognition. The high-speed processing offered by a highly parallel implementation represents ANNs’ greatest potential. The ability to create simple and accurate models from complicated natural systems with vast inputs is a strong benefit of ANN applications. They can behave as a functionalised management system similar to the human brain’s nervous functions. ANNs mimic the operations of an interconnected arbitrary number of nodes by traversing a number of levels with the input characteristics [86]. The basic components of an ANN’s structure can be considered to consist of inputs, weights, a transfer function, an activation function and bias [87]. The inputs refer to the measurements of the features that will be modified. The weights are the scalar multiplications responsible for evaluating each input’s significance and directionality. The transfer function merges the inputs into a single output value in order to apply the activation function. An activation function captures the information from the transfer function and transforms that into a more useful form, which will be used in the next stage. Bias can be considered a systematic error, given as extra input to the system due to the assumptions during the process [88]. The basic artificial neuron architecture, the so-called threshold logic unit (TLU), is demonstrated in Figure 6.

ANNs have been constructed by multi-neurons, which use weights that can learn and be changed after each iteration through neurons, depending on the accuracy. An ANN structure is able to make predictions in the form of regression or classification, followed by a training step for a particular dataset. A recent study [90], as an example, used an ANN structure to train eddy current (EC) impedance and ultrasonic amplitude inputs, aiming to overcome problems as a result of the blind zone of ultrasonics on the near-surface and the electromagnetic skin effect of the EC on the subsurface. The authors employed a database that included a simulation of signal propagation into an aluminium block with holes for estimating the depths and radii of the holes in a regression. The ANNs were also used to characterise and validate the study using real holes experimentally. Mean absolute error (MAE) calculations were performed to assess the accuracy of the defect estimation parameters. The experimental results show that the error of the depths and radius estimates was limited to 4 percent on average, although it showed a peak of 11 percent for one hole. It is stated that this complementary approach might be promising for investigating more complex structures and various defect types.

In the theme of the neural network, the recent study on multi-modal infrared image fusion used a hybrid fusion structure that included a non-subsampled shearlet transform and a pulse-coupled neural network [91]. Two types of impact-defected fibre-metal hybrid composite laminates were inspected by vibro-thermography (VT) and pulse thermography (PT). Three-step feature extraction, feature selection and a fusion method were applied as shown in Figure 7. Fourier analysis for the frequency domain features and intensity contrast criterion for the feature selection were used. Feature selection in the time domain was made using a temperature curve that allowed for determining the thermal contrast over time. For the fusion, images were decomposed by a shearlet transform, followed by the weighted mean and pulse-coupled neural network fusion for the low- and high-frequency sub-bands, respectively. The fusion results indicated a conspicuous improvement in crack contrast and detecting the boundary of deformation rather than a single excitation mode.

Another important structure for fusion studies is the convolutional neural network (CNN). This is one of the most beneficial tools for image analysis applications due to its advanced feature extraction ability through convolution operations instead of standard matrix multiplication. A CNN may use multiple convolution layers to obtain features hierarchically, max pooling layers to minimise image sizes and batch normalisation for reducing internal covariance. A loss function is used to find the difference between the expected and predicted results and tries to minimise them [41]. Historically, after AlexNet [92], various CNN-based algorithms have been proposed to gain a better understanding of image analysis cases. U-Net achieved an efficient feature extraction method even in a small dataset using structural segmentation [93]. ResNet was developed to overcome gradient degradation problems in the training of deep neural networks using a residual network [94]. After that, DenseNet was proposed to obtain a densely connected structure by engaging all layers with each other [95]. GoogLeNet has used an inception module that has multiple kernels of different sizes for convolution to ensure more efficient computation [96]. A couple of studies that have been published recently have focused on the fusion of NDT methods through neural networks to inspect various materials, from wooden cultural heritages to common composites [97,98,99,100,101,102]. Here, the text will not give the details of these approaches. (The interested reader can refer to the specific papers cited.) However,

Table 6. A summary of the recent neural network-based fusion studies for NDT methods.

Sources	Material	Defect Types	Aim	Base Algorithm	Reference
Ultrasonic (US) and eddy current (EC)	Aluminum	Holes	To overcome individual sensor problems	ANNs	[90]
Vibro-thermography (VT) and pulse thermography (PT)	Aluminium-BFRP/GFRP	Impact	To use the synergistic effect of multi-excitation infrared fusion	Shearlet transform and a pulse-coupled neural network	[91]
Magnetic flux leakage and active thermography inspections	High-carbon steel wire rope	Split (broken wires)	To avoid single-sensor low precision	Neural networks and k-nearest neighbour	[97]
Accelerometer, microphone, current sensor and optical encoder	Cast iron	Wearing, cracks, etc.	To optimise feature extraction and selection at fusion level	Deep convolutional neural networks	[98]
Pulsed thermography and terahertz inspections	Plant fibre-reinforced polymer (PFRP)	Impact	To overcome the structural complexity of plant fibre composites	Unsupervised deep residual network	[99]
Industrial camera images	Stainless steel (reflective material)	BrightLine, deformations, scratches or dents	To investigate multi-light illumination effect on detection	Lightweight SqueezeNet architecture	[100]
Line scan active thermography and terahertz inspections	Fir wood	Missing and broken tesserae	To provide NDT inspection for preserving cultural heritage objects	Unsupervised deep residual network	[101]
Continuous-wave terahertz and long-wave infrared (LWIR) thermography inspections	Wooden	Holes	To introduce an autonomous dynamic line-scan inspection method	Unsupervised deep feature fusion	[102]

demonstrates the sources and aims of those neural network-based research works in various fields. The use of learning approaches in NDT inspection can greatly benefit the field (classify and learn image properties, defects characteristics and features). However, their use in this field is still in its infancy. This is an important part of the PhD study this review paper stems from.

4.5.2. Fuzzy Sets

Fuzzy logic is another technique that uses ambiguous or imprecise statements to simulate logical thinking. Fuzzy logic stems from the idea of fuzzy sets [103] and assigns a degree of membership to the elements of a system, often a real integer in the range of $[0,1]$ (where 0 denotes “completely false”, 1 denotes “completely true”, and values in between the range denote “partial truth”). This indicates a set of many-valued logics in which the truth values are viewed as levels of truth. The general workflow of a fuzzy logic approach is as follows [104]:

• Fuzzification: transform the data in the form of an uncertain language into the linguistic variables (fuzzy set or a membership function) together with the fuzzy components. In this way, the data can be processed using a classical binary method of “zero” and “one”.
• Inference: refer to a rule-based inference that the membership functions are combined with the fuzzy control rules, such as the mean of the maximum or minimum distance, to find the control outcome [105].
• Defuzzifcation: the fuzzy control outcome must be converted from linguistic variable to classical variable format.

From the point of view of NDT, defect characterisation is the primary implementation field to use entire methodologies, whether it is data collection or processing that data. An early study which introduced a fuzzy logic-based pixel fusion for three tomographic techniques that characterise different shape properties was presented at the end of the nineties [106]. Cylindrically shaped aluminium specimens that included two different thicknesses for the drilled side holes and a planar flaw were used. Here, the set was utilised to define the pixel’s membership level in a generated defect image. To be precise, the member pixels had a weight of 1, while others were between 0 and 1 according to the membership function. The weighted pixel amplitudes were summed to form a fused defect image. The study showed that the holes were clearly visible in the fused images, but the planar flaw was not evident in terms of size and shape. It was explained that the data acquisition configurations might change with the orientation of the defect. Lastly, the study does not contain any mathematical evaluation method, although it had sufficient discussion on the reasons for data acquisition issues.

Table 7. A summary of fuzzy set-based fusion approaches for NDT methods.

Sources	Material	Defect Types	Aim	Reference
Tomography (3 techniques)	Aluminum	Foreign body or holes	Shape characterisation	[106]
Ultrasonic and eddy currents	Aluminum (AA5083-H111)	Weld imperfections	To improve detection of friction stir welding imperfections	[107]
Laboratory-based measurements	Concrete	Foreign body or voids	To determine porosity, content and mechanical properties	[108]
Pulsed eddy current and ultrasonic methods	Aluminum alloy (AL-2024)	Stress	To determine stress measurement	[109]
Ground penetrating radar and square pulse thermography	Concrete	Foreign body or voids	To identify material content and characteristics	[110]

demonstrates that the current studies on NDT image fusion have investigated aluminium materials and concrete structures so far. The scarcity of NDT combination attempts applied to composite materials also shows the potential for a number of different algorithms to be used in this area.

Research on material properties is a different application field of NDT inspection technologies. Inspecting concrete characteristics and determining physical properties was proposed by the authors of [108]. Because of difficulties in understanding the porosity, water saturation rate, elastic modulus, compressive strength, depth of carbonisation and chloride content of concrete structures, a complementary fusion of non-destructive information might improve the reliability. For this purpose, a fuzzy set-based possibility theory was chosen to combine this heterogeneous information for controlled produced specimens with different compositions. The fusion results were compared with the destructive measurement and had several differences. It has been claimed that such a fusion technique can be helpful in understanding the conflicts of the NDT methods. This can make it possible to have a critical evaluation of the results and provides for the creation of a confidence index for inspected structures. The proposed method has been considered encouraging for the water saturation and porosity rates. However, it is problematic in estimating the elasticity value. With a similar purpose, the authors of [109] proposed stress measurement of an aluminium alloy material using fuzzy combinator-based fusion. It has been claimed that a fused result might have 50 percent less error than individual inspection outcomes. Even though the combination of NDT methods can give more deterministic clues regarding the mechanical properties of inspected structures, it is clear that these contemporary approaches are highly limited.

4.5.3. Bayesian and Dempster–Shafer Methods

The Dempster–Shafer (D-S) method, known as evidence theory or belief functions, can be seen as a generalised form of the Bayes theorem, which allows revision of the predicted probabilities of an occurrence with additional knowledge. While the Bayes theorem uses probabilities for weighting, the D-S method accomplishes this by assigning masses to all possible subsets [111]. For instance, the authors of [112] proposed a Bayes-based fusion method for tomography inspection, calculating wave velocity probability density functions to improve the impact localisation accuracy. Two sensors that have a 90${}^{\circ }$ angle difference were incorporated, and non-matched regions and measurement noise were minimised by the multiplication of related probabilities. Dempster–Shafer’s evidence theory can meet poor conditions better than probability theory can, using the belief functions instead of probability. It is also possible to capture cognition notions such as uncertainty. It presents the concept of attributing beliefs and plausibilities to potential measurement hypotheses and the necessary combination rule to combine them [113].

Basically, in the power set of a universal set X, $P\left(X\right)$ shows all possible subsets, including an empty $\left(\varphi \right)$ set with its mass function equal to zero:

$$m\left(\varphi \right)=0$$

(3)

The D-S theory employs the mass function (m), which reflects the level of support for the evidence, as well as the belief function and the plausibility function. The value of m(A) applies just to the set A and does not make extra claims about any subsets of A, each of which has its mass by definition. For the two different sensors $A_1$ and $A_2$, the combination rule of the D-S theory is as follows [114]:

$$m_{1,2}\left(A\right)=(\sum _{B\cap C=A\ne \varphi }m\left(A_1\right)m\left(A_2\right))/(1-K)$$

(4)

$$K=(\sum _{B\cap C=\varphi }m\left(A_1\right)m\left(A_2\right))$$

(5)

K indicates the amount of conflict between the two mass functions from the sensors. The equation means that the D-S combination procedure computes the orthogonal sum of each possible proposition and then combines these two beliefs to update the joint belief mass function $m_{1,2}$. In particular, the work in [115] used an image fusion strategy to investigate the bonding quality of images from ultrasonic and thermographic inspection. Three different types of artificial debonding defects formed, and the extracted features were fed to basic and statistical data fusion as the source. A multi-step preprocessing application that included noise reduction to eliminate error multiplication coordinate matching to remove the effects of different data collection environments, registration or interpolation to match values perfectly and normalisation was performed before the fusion of data. Here, the maximum amplitudes of the interface reflection and second principal component were saved as features indicating the defects for ultrasonic and induction thermography, respectively. In the fusion stage, three hypotheses—defected (positive), non-defected (negative) and unsure regions (doubt)—were identified, and their belief probabilities were calculated with global Gaussian curves of the local amplitude distributions. Figure 8 shows the calculation of the D-S belief percentage for a randomly chosen pixel and its surrounding area in the induction thermography feature matrix’s no-defect zone and global distribution. The results emphasised that the D-S combination might be an effective technique, even in the case of basic mathematical fusion rules that have worse results than those based on individual sensors.

Another study using the Bayes and D-S methods was recently introduced for multi-material image representation [116]. X-ray computed tomography was applied with various configurations to provide images for different materials of a particular object. This means that the D-S combination rule was implemented to produce a fusion image using three different energy images. Five different attenuation-based hypotheses have been chosen to apply the D-S combination because they give different grey values due to the energy range difference. It has been stated that the proposed method ensured higher image quality for the contrast-to-noise ratio, even though it had several artefacts. The results were also compared with the Bayes combination results, finding that it had a higher risk of conflict. In conclusion, it can be stated that the D-S theory has rarely been used for combination attempts of NDT methods and needs to find more space in NDT application due to its effective usability.

5. Evaluation Strategies and Performance Metrics

Evaluation plays an important part in NDT-based proposed methods towards validation, and it can be classified under quantitative and qualitative approaches. The simplest and most straightforward method is qualitative evaluation (mainly visual), which depends on the experience of the operator performing it. The aerospace industry has gradually moved towards both qualitative and quantitative evaluation, relying less on the operators’ skills and expertise [117]. Noting that NDT inspection is regarded as a quantitative analysis method, many papers in the current literature pay close attention to evaluation (and the use or proposal of relevant evaluation metrics) to support the efficacy of their approaches. We aim to systematise the presentation of evaluation strategies in this section, providing critical insights into NDT image fusion-related efficacy.

A useful example of the importance of evaluation in NDT-based inspection is in a highly noted recent paper [118], which looked into barely visible impact damage (BVID) in composite materials. The authors used NDT fusion of ultrasonic testing and X-ray computed tomography. Their particular fusion approach is shown graphically in Figure 9a, with a particular aim of investigating the true mapping of the defect from UT to the true defect from X-ray CT. (Figure 9b illustrates the scanning views obtained for the defect.) Their particular metric was the distance from the defect’s centre to the boundary points (see the example in Figure 9c, which illustrates a form of pseudo-segmentation of the defect area). The authors included statistical analysis to showcase the differences between the UT C-scans and B-scans relative to the defect mapping (estimation accuracy of the BVID), with the C-scan providing better overall results, as claimed by the authors of the work. An interesting outcome from this paper to consider is the transition from a qualitative aspect of defect analysis (images) to the statistical analysis of the boundary point distance (quantitative) viewpoint.

Given that defect analysis, detection and classification can be a very challenging task (depending on the nature and importance of the defect), researchers looked into a variety of evaluation metrics to support NDT analysis. There is no unique evaluation metric that can be applied in this domain (as is the case in many other engineering domains), with a number of evaluation criteria (metrics) introduced to facilitate a more informed outcome decision in NDT analysis of defects [119]. We present a series of important evaluation metrics (merely exhaustive) to assist the readers in understanding their nature, use and potential impact in NDT fusion works.

Common evaluation metrics seen in the literature are the root mean square error (RMSE), signal-to-noise ratio (SNR), correlation coefficient (CC), (fusion) mutual information (MFI) and structural similarity index (SSIM). It is possible to mention more metrics, but many of them are fairly correlated with each other, as can be seen with the 27 different metrics in [120]. Papers refer to these for evaluating the performance of fusion methods. In particular, a recent paper [119] closely utilised these criteria for their work on NDT fusion for defect analysis in Si-PV cells and comparison of the best algorithm outcomes.

We briefly present the nature of the five metrics mentioned above and their expressions, given in

Table 8. Fusion performance evaluation metrics.

Evaluation Metric	Expression	Characteristics
Root Mean Square Error	${\displaystyle \mathrm{RMSE}=\sqrt{\frac{1}{M\times N}\left(\sum _{i=1}^M\sum _{j=1}^N{\left[X(i,j)-Y(i,j)\right]}^2\right)}}$	$M\times N$ sizes of the images X and Y reference and source images
Correlation Coefficient	${\displaystyle CC=\frac{{\sum }_{i=1}^M{\sum }_{j=1}^N(X_{ij}-\overline{X})(Y_{ij}-\overline{Y})}{\left[{\sum }_{i=1}^M{\sum }_{j=1}^N{(X_{ij}-\overline{X})}^2\right]\left[{\sum }_{i=1}^M{\sum }_{j=1}^N{(Y_{ij}-\overline{Y})}^2\right]}}$	$M,N$ image sizes X reference and Y source
Fusion Mutual Information	${\displaystyle FM{I}_{\acute{X},\acute{F}}=\sum _{\acute{x}\acute{f}}{P}_{\acute{X},\acute{F}}(\acute{x},\acute{f})lo{g}_2(\frac{P_{\acute{X},\acute{F}}(\acute{x},\acute{f})}{P_{\acute{X}}\left(\acute{x}\right)P_{\acute{F}}(\acute{f})})}$	$P_{\acute{X},\acute{F}}(\acute{x},\acute{f})$ joint histogram of x (source) and f (fused image) $\acute{X},\acute{F}$ feature maps
Signal-to-Noise Ratio	${\displaystyle SNR=10{log}_{10}(\sum _{m=1}^{S_1}\sum _{n=1}^{S_2}z{(m,n)}^2/\sum _{m=1}^{S_1}\sum _{n=1}^{S_2}{[z(m,n)-o(m,n)]}^2)}$	$S_{1,2}$ image sizes $(m,n)$ location z and o estimated and original signals
Stractural Similarity Index	${\displaystyle SSIM(x,y)=\frac{(2{\mu }_x{\mu }_y+C_1)(2{\sigma }_x{\sigma }_y+C_2)}{({\mu }_x^2+{\mu }_y^2+C_1)({\sigma }_x^2+{\sigma }_y^2+C_2)}}$	$\sigma$ is standard deviation $\mu$ is mean value

, for an image set for completeness. The image example was obtained from the Northeastern University (NEU) surface defect dataset, which includes six types of surface defects [121]. From the original image, a second version had semi-blurred compressed corruption (pixelisation), and the third version was corrupted by Gaussian distribution noise. Figure 10 shows the three image snapshots, and Figure 11 gives the absolute differences. Blurring and noise have shaded defects at different levels, causing a visible loss of quality. The absolute difference images also indicate the corruption effects, showing how the defects were hidden. After this point, these images will be handled as corrupted and noisy.

Figure 12 shows the different images in a $2\times 2$ array format. The bottom-right image is a fused image version of the noisy and corrupted image versions via alpha blending with the (fused) image intensities together as a single dataset. It can clearly be seen that the fused image had a purer texture than the sources in terms of noise and corrupting effects. Here, we will explain the evaluation metrics over these example images using the calculated values between the reference image and sources, including the fused image.

Root mean square error (RMSE): The RMSE, given the pixel intensity nature of the image, is a measure of the proximity of the two images that are compared. From a statistical viewpoint, the RMSE refers to the standard deviation of the error under investigation. A smaller value for the RMSE indicates closer similarity of the images (in the case of a fused image, the similarity to the original one) [123]. It is noted here that the MSE is closely related to the RMSE. Hence, we do not explain the MSE metric in detail. A useful example, in this case, can be seen in [90], which calculated the MSE and RMSE values for the radius and depth estimations of holes and stated that the bigger gaps between the estimated and expected (real) values might represent an insufficiency of the fusion operation. The authors attributed these high MSE and RMSE values to the blind zone effect of the ultrasonic imaging method for their case. In addition, the authors of [70] agreed that the RMSE is a measurement of detectability, showing that lower values indicated less contrast between the estimations and expectations.

Table 9. A comparison of the evaluation metric values for the illustrative example for fusion metrics.

Metrics	Corrupted	Noisy	Fused
RMSE	12.960	30.910	16.607
CC	0.860	0.629	0.814
FMI	1.586	0.378	0.579
SNR	5.284	2.086	5.428
PSNR	15.165	11.940	15.310
SSIM	0.771	0.15	0.32
Thresholded
RMSE	0.164	0.192	0.148
CC	0.459	0.366	0.558
FMI	0.042	0.034	0.056
SNR	0.379	−5.390	0.822
PSNR	16.645	10.876	17.086
SSIM	0.846	0.406	0.745

shows the evaluation values for the example images, including corrupted, noisy and fused examples as a comparison. For our basic example application, it was not surprising to find less difference in the metrics between just the corrupted and original image, although the fused image had a considerably lower RMSE than the noisy image but slightly larger than the corrupted one. Even so, the lower RMSE value of the fused image indicates that the fusion successfully preserved the scratches’ information. On the other hand, thresholding of these images indicated that preprocessing might affect the fusion success, as can be seen from the values of the same metrics. After a basic segmentation by adaptive thresholding, the fused image had the lowest RMSE value, visibly showing the highest similarity.

Correlation coefficient (CC): This is an indicator that gives values in the range from −1 to 1, revealing the relationship intensity between the sources and the equivalent fused image. The best match between the fused and original images has the highest correlation value [124]. In our example of fusion application, the correlations of the fused and pixelised images were quite close to each other (Table 9). Similar to the RMSE calculation, we can consider that the corrupting operation does not affect the pixel intensities or the features of the image remarkably. However, the correlation of the fused image to the original image was notably higher than the noisy one, indicating that the fusion managed to merge the valuable components of the sources. Segmentation (thresholding) as a preprocessing step made the fusion result have the highest correlation with the original images eventually.

Fusion mutual information (FMI): This metric, based on the mutual information criterion, evaluates how much of the original images’ features, such as the edge, detail or contrast, are inherited in the fused image [125]. The FMI computes the degree of dependency between the input images and the fused image [126]. Smaller values for the FMI indicate worse quality for the fused image. In our case, the original image features were firmly maintained in the corrupted image, with a notably higher FMI value than those of the noisy and fused ones. That aside, the fused one also had a relatively higher FMI value than the noisy one. The story was different when the intensity became deterministic (black and white segmented areas) in the threshold images (Figure 13). In such a case, the corrupted image had the lowest FMI value, and the fused one had the largest FMI value, confirming the benefit of even a simplified fusion operation.

Signal-to-noise ratio (SNR): The SNR provides a metric of evaluating the quality of a signal quantitatively when comparing the fused and original signals [127], and it gives the proportion of signal power to noise power:

$$SNR=10{log}_{10}\left(Energ{y}_{Signal}\right)/\left(Energ{y}_{Noise}\right)$$

(6)

where $Energy$ denotes the sum of the squares of the signal values for the estimated signal and the difference (error or noise) between the estimated and original signals. With the SNR measuring the ratio between the information and noise of the fused image, a higher value reveals a greater similarity between the fused and reference (original) images. In an example fusion of terahertz and thermal imaging, the larger SNR values contributing to the achievement of suppressing the background noise showed that the salient features were extracted successfully [102]. Another interesting quantitative assessment with the SNR demonstrated that although the fusion provided a higher contrast and resolution, the SNR value of the fused image was relatively lower [99]. This might be attributed to calculating the average SNR from the SNR values of different depths. Another closely relevant metric the peak signal-to-noise ratio (PSNR), indicates the ratio of effective information to noise using the maximum squared signal intensity. The SNR and PSNR values of our example evaluation application indicated that the fused image had the greatest values, demonstrating that a successful fusion can provide more evident marks by reducing the noise effects. This gave the same results after the thresholding application, demonstrating that the SNR and PSNR values were not affected by this preprocessing operation, contrary to other metrics. Moreover, the negative SNR value of the noisy segmented image might be attributed to the noise density of the image being heavier than the signal density [128].

Structural similarity index (SSIM): The SSIM approach compares the luminance (l), contrast (c) and structure (s) in an independent manner using the mean and standard deviation values [129]. This means any change in one does not affect the other parameters:

$$S(x,y)=f\left(l\right(x,y),c(x,y),s(x,y\left)\right)$$

(7)

The application can be performed gradually in a square window which moves pixel by pixel across the image. The local statistics and the SSIM index are computed at each step in the range from 0 to 1. The SSIM is another positive indicator expressing that the higher the values, the greater the correlation. In our example, the corrupted images, as well as the thresholded (segmented) ones, had the highest similarity with the reference image, unsurprisingly. The fused image had a relatively higher similarity to the original image than the noisy one. This means that the fusion managed to compensate for the heavy disturbance effect of the noise partially. An example from the studies above [102] used the SSIM method to determine the defect preservation ability in the defective regions of raw images. The authors emphasised that the higher SSIM values of the proposed training algorithm indicate that it could extract adequate information from the input images.

6. Insights from the Survey and Future Prospects

This review paper presented, in a rigorous way, image fusion and evaluation strategies for NDT, taking into account various structures (albeit with a particular focus towards composites). It is clear that the studies indicate that using image fusion is promising for improving the reliability of inspections. An important highlight from this study is that various image fusion methods which are well-studied in other domains, such as remote sensing and image recognition, have not been extensively applied to NDT yet.

Individual fusion techniques’ performances with regard to a number of evaluation metrics were presented, including insights on how they have been perceived in studies in the literature. These metrics provide a quantitative means of identifying the best fusion approach for the specific or given inspection case. Each method is capable of revealing different aspects. For example, adding small noise to simulate a more realistic environment can significantly impact each technique with defect variability. Current studies have shown that the results depend on the acquisition system (i.e., being multi-sensor or multi-excitation). Different configurations impact the results. Another critical point is the applicability of the fusion method. This can differ depending on the level of fusion or whether it is pixel-wise or region-based. As one moves towards the use of machine learning tools, the importance of having sufficient datasets (of realistic defects) for training and validation is paramount.

The widespread use of high-performance AI (ML)-based fusion approaches in aviation has the potential to improve the reliability and profitability of aviation operations. This review paper sheds further light on utilising the fusion of NDT techniques appropriately in this field. This was shown by the highlights of applying fusion in other domains, which can inform the fusion-based inspection of aeronautical structures (particularly composites). The following points are noted from this rigorous survey study:

• There is no comprehensive review of NDT image fusion (this review paper fills this research gap).
• Fusion applications for composite structures are very limited.
• Low-level (signal or pixel) fusion applications require a precise registration that might be possible to reduce its effect on the higher-level fusion structures.
• General algebraic fusion rules have often been utilised as a main rule in low-level fusion or auxiliary for higher levels.
• PCA and wavelets are the most preferred methods among the current applications, but few relate to AI applications.
• Preprocessing operations before applying fusion are a vital part of the process.
• There is a plethora of evaluation metrics, and a series of these tends to be used in validation studies. This seems to be application-dependent.
• Even though ultrasonics and acoustic emission seem to outshine the other methods in terms of capability for various defect types, each technique has individual characteristics for their sensing ranges.

7. Conclusions

Airworthiness in aviation is vital for the safe operation of and guaranteeing the safety of both passengers and aircrew. In this context, MROs—typically complex processes—in the aerospace industry has strict and precise requirements defined by airworthiness authorities. Enabling more sustainable aviation utilisation of novel materials such as composites, given their potential for weight reduction of structures and attractive material properties, necessitates efficient MROs in order to avoid catastrophic failures. Structural inspection-wise (NDT) approaches are preferable, given their non-destructive testing nature, to ensure structural component integrity and reassure proper function expectations. Characterisation and localisation of possible defects at an early stage is one of the most crucial steps, given that even micro-cracks going unnoticed (and not addressed or treated) can result in catastrophic incidents.

Individual inspection techniques may be limited by their sensing technologies, and fusion provides a potential solution in improved inspection performance and timely information to the inspection expert for efficient MROs. This review paper provided comprehensive insights into the recent progress in image fusion and its potential impact on composite structure inspection, together with a rigorous analysis of the challenges in this new and challenging research field. It was noted in this research that PCA and wavelet decomposition-based fusion approaches seem to be preferred more often compared with learning-based methods. With the advent of AI and ML tools and the accumulation of extended datasets, more learning-based schemes will appear. There are limited examples of fusion-based inspection of composite materials, and the potential of fusion-based inspection in this area to positively impact aerospace MROs is vast, as presented in the discussion of this review paper.