Advances in TEE-Centric Intraprocedural Multimodal Image Guidance for Congenital and Structural Heart Disease

Percutaneous interventions are gaining rapid acceptance in cardiology and revolutionizing the treatment of structural heart disease (SHD). As new percutaneous procedures of SHD are being developed, their associated complexity and anatomical variability demand a high-resolution special understanding for intraprocedural image guidance. During the last decade, three-dimensional (3D) transesophageal echocardiography (TEE) has become one of the most accessed imaging methods for structural interventions. Although 3D-TEE can assess cardiac structures and functions in real-time, its limitations (e.g., limited field of view, image quality at a large depth, etc.) must be addressed for its universal adaptation, as well as to improve the quality of its imaging and interventions. This review aims to present the role of TEE in the intraprocedural guidance of percutaneous structural interventions. We also focus on the current and future developments required in a multimodal image integration process when using TEE to enhance the management of congenital and SHD treatments.


Introduction
Structural heart disease (SHD) refers to the heart abnormalities that affect the heart's valves, walls, chambers, or muscles [1].Its complex etiology includes congenital malformations, molecular or genetic anomalies, rheumatic valve disease [2,3], hypertension, cardiomyopathy, etc.In addition to its impact on quality of life, SHD often leads to severe complications such as heart failure [4], stroke [5], etc.Therefore, the diagnosis and treatment of SHD is a critical area of research in the field of cardiology and cardiac surgery.
The diagnosis of SHD usually requires a variety of imaging tests, including electrocardiography [6], echocardiography [7], computed tomography (CT) [8], magnetic resonance imaging (MRI) [8,9], and invasive cardiac catheterization [10,11].After diagnosis, SHD is increasingly treated via minimally invasive interventional procedures [12]-e.g., cardiac catheter occlusion or a closure of defects [13], valve replacement or repair, etc.-under imaging guidance [14][15][16][17].In a well-planned and executed cardiac catheterization procedure, imaging is fundamental for all aspects of the transcatheter treatment process, including preprocedural planning, the assessment of the extent of the disease, diagnosing and detecting risk factors, and intraoperative guidance, as shown in Figure 1.Echocardiography is established as an important tool for diagnosing and treating SHD because it can visualize the structure and function of the heart, as well as provide real-time feedback.In particular, transesophageal echocardiography (TEE) is frequently used during the treatment of SHD.TEE places an ultrasound probe through the esophagus to visualize the heart without any interposing structures between the probe and the heart, thus allowing for an excellent acoustic window that enables a clear visualization of the heart with minimal trauma.Furthermore, TEE is a low-cost, real-time, ionizing-radiation-free modality.As a result, under TEE guidance, interventionalists can accurately visualize the moving catheters or devices within the beating heart in real-time.
Although TEE has become an important means of imaging SHD, using other modalities to augment the image guidance offered by TEE is now standard practice.A list of commonly used image modalities can be found in Figure 2. As all medical imaging modalities have strengths and weaknesses, image fusion is widely accepted in the clinical workup of patients with SHD.The primary goals of this review paper are two-fold.First, this review paper summarizes the state-of-the-art clinical imaging technologies available for managing SHD as well as the limitations that are in need of further development of each modality.Second, our Echocardiography is established as an important tool for diagnosing and treating SHD because it can visualize the structure and function of the heart, as well as provide real-time feedback.In particular, transesophageal echocardiography (TEE) is frequently used during the treatment of SHD.TEE places an ultrasound probe through the esophagus to visualize the heart without any interposing structures between the probe and the heart, thus allowing for an excellent acoustic window that enables a clear visualization of the heart with minimal trauma.Furthermore, TEE is a low-cost, real-time, ionizing-radiation-free modality.As a result, under TEE guidance, interventionalists can accurately visualize the moving catheters or devices within the beating heart in real-time.
Although TEE has become an important means of imaging SHD, using other modalities to augment the image guidance offered by TEE is now standard practice.A list of commonly used image modalities can be found in Figure 2. As all medical imaging modalities have strengths and weaknesses, image fusion is widely accepted in the clinical workup of patients with SHD.Echocardiography is established as an important tool for diagnosing and treating SHD because it can visualize the structure and function of the heart, as well as provide real-time feedback.In particular, transesophageal echocardiography (TEE) is frequently used during the treatment of SHD.TEE places an ultrasound probe through the esophagus to visualize the heart without any interposing structures between the probe and the heart, thus allowing for an excellent acoustic window that enables a clear visualization of the heart with minimal trauma.Furthermore, TEE is a low-cost, real-time, ionizing-radiation-free modality.As a result, under TEE guidance, interventionalists can accurately visualize the moving catheters or devices within the beating heart in real-time.
Although TEE has become an important means of imaging SHD, using other modalities to augment the image guidance offered by TEE is now standard practice.A list of commonly used image modalities can be found in Figure 2. As all medical imaging modalities have strengths and weaknesses, image fusion is widely accepted in the clinical workup of patients with SHD.The primary goals of this review paper are two-fold.First, this review paper summarizes the state-of-the-art clinical imaging technologies available for managing SHD as well as the limitations that are in need of further development of each modality.Second, our The primary goals of this review paper are two-fold.First, this review paper summarizes the state-of-the-art clinical imaging technologies available for managing SHD as well as the limitations that are in need of further development of each modality.Second, our work also emphasizes the continued technological developments toward multimodality fusion that are being conducted to supplement the TEE guidance during treatment of SHD.The rest of this article is structured as follows.Section 2 discusses the limitations of the single-imaging modality that is frequently used during SHD treatment, which motivates multimodal image integration.In Section 3, the technological developments related to multimodal image integration are introduced, followed by a detailing of the clinical applications centered around TEE-based multimodality integration in Section 4. We close our work by outlining the challenges and opportunities for a multimodal integration in treating patients with SHD.A summary of the key articles reviewed in this paper is provided in Appendix A for the convenience of the readers.

Strengths and Weaknesses of Single-Modality Intraprocedural Imaging Guidance in the Treatment of SHD
Intraoperative imaging guidance of SHD is used to assist interventionalists and imagers in understanding the catheter and device manipulation required during the procedure [18].The instantaneous feedback provided through this guidance to the interventionalists is the key to procedural success.Also, providing comprehensive information enables the "interventional team" to avoid potential complications and to modify the surgical plan for better outcomes.
As is well understood in the literature [19], the available intraoperative imaging modalities (i.e., computed tomography, magnetic resonance imaging, X-ray fluoroscopy, and ultrasound) all have their advantages and disadvantages in terms of retrieving the required information, as summarized in the following few subsections.

X-ray Fluoroscopy and Angiography
X-ray fluoroscopy represents the mainstay for intraoperative imaging guidance [20] in patients undergoing many SHD procedures, such as transcatheter aortic valve replacement.Fluoroscopy helps to visualize the blood flow through the arteries and allows for interventionalists to examine vessel patency.Also, fluoroscopy can be used to guide catheter movement/placement, and allows for the accurate positioning/deployment of devices.
The advantages of X-ray fluoroscopy are its wide availability and real-time image guidance.The real-time capability is much needed for catheter movement and device implantation.However, it has significant limitations.First, the soft tissue contrast on fluoroscopy is poor, thus anatomical information is often lacking.The structural information of soft tissues is required to assess the device deployment in most interventions, e.g., transcatheter aortic valve replacement (TAVR).Second, when an X-ray fluoroscope is used for image guidance, patients and interventionalists are exposed to considerable ionizing radiation [21].The cumulative exposure to ionizing radiation may result in serious health risks, including cancers, and could impact the cognitive function of patients and medical professionals [22,23].The images obtained through the fluoroscopy of a dynamic three-dimensional structure such as the heart are limited by the planes and angles at which such images are captured.

MRI
In the last two decades, tremendous progress has been made in cardiac MRI.In terms of clinical workflow, cardiac MRI is increasingly used with advanced software and manual annotation so as to comprehensively assess heart structure, function, and hemodynamics [24].More importantly, cardiac MRI can accurately characterize soft tissues (e.g., myocardium, valve, aorta, etc.) without ionizing radiation [12].Specifically, MR imaging has multiple sequences-such as black-blood monophasic [25], bright blood imaging, MR angiography (MRA), and phase contrast-which are used to assess cardiac anatomy and function [26].The black-blood double-inversion recovery (DIR) sequence is useful for imaging myocardium and blood vessels (including the coronary arteries).The use of gadolinium-based agents can shorten T1 relaxation time, thus resulting in a high signal, also known as "hyper-enhancement", in the T1-weighted imaging areas of myocardial fibrosis.This is due to the increased extracellular compartments and can lead to increased gadolinium retention and delayed washout.MR angiography can be performed with or without contrast to evaluate non-cardiac structures [27].Phase contrast sequences generate quantitative velocity information for flow velocity and pressure gradient measurements, thus allowing for the quantification of cardiac output, shunt ratio, and valvular regurgitation [28].Modern cine imaging employs a steady-state free procession (SSFP) technique that allows for the acquisition of a series of images that show cardiac motion, and it can be segmented to measure the volume and ejection fraction.Cine SSFP imaging is the gold standard for assessments of cardiac function.Furthermore, MR imaging technology can provide additional information, such as myocardial fiber orientation and texture, which can aid in diagnosing and treating heart disease.
In short, MR imaging is widely used for pre-treatment assessments of SHD.However, long data acquisition times, the need for anesthesia in young children, and the limited availability of MR-compatible devices limit the use of MRI for intraoperative image guidance in treating SHD.

CT
CT is an imaging technique that uses X-ray scanning to obtain a large number of image slices and uses a computer to combine these slices into three-dimensional image datasets.It plays a fundamental role [29] in the preoperative evaluation of patients undergoing percutaneous and electrophysiological interventions.Modern multi-slice CT (MSCT) has been firmly established as a primary imaging tool in the context of structural heart disease interventions.MSCT and MDCT [30,31] can achieve high in-plane spatial resolution (0.3 mm or less), and the reconstructed slice thickness can be reduced to 0.5-0.6 mm.Furthermore, we can minimize cardiac motion artifacts during the data acquisition process when operating under an ECG-synchronized imaging protocol.MSCT imaging has been used [32] routinely to treat the transcatheter aortic valve in mitral, tricuspid, and pulmonary valve interventions in left atrial appendage (LAA) occlusion and paravalvular leak occlusion.
However, the absorption of X-rays among different soft tissues is comparable; thus, soft-tissue contrast under MSCT is not its strength.Furthermore, the slow frame rate of 3D CT does not allow for it to provide real-time feedback to operators during the treatment of patients with SHD.That is why MSCT has been limited primarily to pretreatment assessments of SHD interventions.Pre-procedural CT can provide important information such as on the most feasible/optimal access route, accurate measurements of the dimensions, and the optimal viewing angles of fluoroscopy imaging.At the same time, interventionalists depend on live X-ray fluoroscopic imaging for real-time feedback [33].
To address the concerns of ionizing radiation [30], many research efforts have been devoted to minimizing radiation exposure during pediatric CT imaging, including using low-tube-voltage techniques, adjusting tube current to body size, and using geometric dose modulation.Recent developments in CT have also demonstrated that soft-tissue contrast can be improved.Kaneko et al. [34] presented a method for imaging human cardiac structures via X-ray phase-contrast tomography.Compared with traditional X-ray absorption imaging technology, phase-contrast imaging technology has a higher resolution and imaging ability with respect to soft tissue.This method can provide high-quality 3D reconstruction images of human cardiac structures, as well as display fine cardiac structures and details.

Echocardiography
Echocardiography is a non-invasive, real-time, non-ionizing, and contrast agent-free imaging technique [35].It is widely used for pre-treatment SHD assessments, thereby providing real-time guidance for catheter interventional procedures.TEE imaging is a dynamic, gold-standard imaging method for guiding SHD intervention, allowing for the real-time observation of cardiac motion and blood flow [36,37].This imaging technique offers high-resolution images of the heart, thus helping doctors to assess the location and extent of lesions, as well as allowing them to choose the best surgical options more accurately.
In catheterization procedures, TEE provides the best spatial resolution and anatomical definition with high image resolutions and a sterile environment that does not interfere with the surgical area, thus making it commonly used in cardiac surgery [38].It has been used routinely to guide the closure of atrial septal defect (ASD), patent foramen ovale (PFO), and ventricular septal defect (VSD) [39].Although its use has been shown to reduce fluoroscopy time and radiation exposure, one major limitation of TEE is its small field of view [40] and the need for deep sedation or general anesthesia (GA).Recent reports have found that most patients had esophagus injuries after TEE-guided procedures due to probe manipulation [41].To a certain extent, frequent probe manipulation is a response to the limited field of view.
TEE has a high risk for contagious infections [42]; thus, prior to the procedure, the risk-benefit ratio in favor of the patient should be paramount.An alternative technique like the LA strain [43] as a predictor of the LA appendage thrombus in the setting of atrial fibrillation is considered by some authors instead of TEE.
In recent years, the role of intracardiac echocardiography (ICE) in SHD interventions has been expanding.Most cardiac structures treated with transcatheter SHD intervention can be imaged using ICE probes that are located at one or more of the five locations-such as the middle of the right atrium (RA), low RA, right ventricular (RV) inflow, RV outflow, and left atrium (LA)-to obtain better views [36].The effectiveness of ICE in guiding the closure of the percutaneous aortic fistula, aortic balloon valvuloplasty, and evaluating the function of artificial AV has been confirmed in a series of small cases.Studies have reported that percutaneous ASD closure under ICE guidance is safer and better than TEE guidance in shortening fluoroscopy and surgical time, as well as in reducing radiation dose [36].However, the scarcity of data, introduction of larger sheaths and extra access sites, increased cost, equipment limitations, and steep learning curve limit the use of ICE in SHD.Also, the advantages of ICE over TEE have not been generally accepted yet.

Review of Existing Multimodal Image Integration Techniques
Each imaging modality has its strengths and weaknesses and may be unable to provide interventionalists with complete information during treatment.Thus, multimodal image integration is a logical solution and has received increasing attention [44].
Without automated multimodal image integration, interventionalists are required to mentally overlay features from multiple modalities together [45] while performing a procedure.Such mental integration can be a daunting task.In contrast, multimodal image integration [46] automatically registers images together and offers excellent visualization.Figure 3 shows a generic process of multimodal image integration starting with the data acquisition of individual modalities, followed by image preprocessing, including image enhancement and denoising, image registration, and feature extraction.At the end of the integration processes, images are displayed for enhanced visualization.Modern computer graphic techniques such as volume rendering, virtual reality [47], and augmented reality have all been used for enhanced visualization.
One of the significant limitations of the current imaging technique is the specialist interpretation of images and the communication between the specialists (i.e., between interventionalist and imager).Automated image integration and advanced visualization remove this interphase through the 'democratization' of image interpretation to a common language.
By combining the strengths of each imaging modality, multimodality integration decreases procedure time and presumed radiation dose [37].Arguably, the amount of contrast agent used might also be reduced.
The major steps related to technological developments in multimodal image integration are summarized in Figure 3 and briefly explained below in this section.

Image Registration
One important step in multimodal image integration is image registration [48], as shown in Figure 4. Mathematically, image registration solves an optimization problem that arises when aiming at maximizing the similarity or minimizing the difference between two images that are acquired from the same subject.Image registration helps ensure spatial consistency between two images that are captured by different imaging modalities because their image formation processes differ vastly.Medical image registration remains a hot research topic, and existing medical image registration can be divided into two categories: (1) traditional and (2) deep-learning-based registration methods.In the registration process, it is necessary to specify the required registration accuracy and establish a corresponding mapping relationship between a reference image  and a target image  ; the mapping relationship is known as a parameter transformation  , and the mapped  maximizes its similarity to It via a pre-determined similarity measure.Formally, the goal of image registration is to determine an optimal transformation  by which transformed  (i.e.,  ( )) can maximize its similarity with the target image It.Formally, the optimization problem is written as follows

Image Registration
One important step in multimodal image integration is image registration [48], as shown in Figure 4. Mathematically, image registration solves an optimization problem that arises when aiming at maximizing the similarity or minimizing the difference between two images that are acquired from the same subject.Image registration helps ensure spatial consistency between two images that are captured by different imaging modalities because their image formation processes differ vastly.

Image Registration
One important step in multimodal image integration is image registration [48], as shown in Figure 4. Mathematically, image registration solves an optimization problem that arises when aiming at maximizing the similarity or minimizing the difference between two images that are acquired from the same subject.Image registration helps ensure spatial consistency between two images that are captured by different imaging modalities because their image formation processes differ vastly.Medical image registration remains a hot research topic, and existing medical image registration can be divided into two categories: (1) traditional and (2) deep-learning-based registration methods.In the registration process, it is necessary to specify the required registration accuracy and establish a corresponding mapping relationship between a reference image  and a target image  ; the mapping relationship is known as a parameter transformation  , and the mapped  maximizes its similarity to It via a pre-determined similarity measure.Formally, the goal of image registration is to determine an optimal transformation  by which transformed  (i.e.,  ( )) can maximize its similarity with the target image It.Formally, the optimization problem is written as follows Medical image registration remains a hot research topic, and existing medical image registration can be divided into two categories: (1) traditional and (2) deep-learningbased registration methods.In the registration process, it is necessary to specify the required registration accuracy and establish a corresponding mapping relationship between a reference image I r and a target image I t ; the mapping relationship is known as a parameter transformation T g , and the mapped I r maximizes its similarity to It via a pre-determined similarity measure.Formally, the goal of image registration is to determine an optimal transformation T g by which transformed I r (i.e., T g (I r )) can maximize its similarity with the target image It.Formally, the optimization problem is written as follows argmax S T g (I r ), where R is a regularization term, and adding the regularization term R allows for the solution of the transformation T g to be stable.
Among the traditional methods, an array of well-established methods have been defined and widely adopted.Loosely speaking, image registration methods can be performed with variable transformation models and different regularization terms (elastic, diffeomorphic, unbiased, etc.), as well as similarity metrics S(•) (cross-correlation, mutual information, etc.).
Under the category of deep-learning-based image registration methods, information at various spatial scales is extracted and aggregated together to transform the reference image to the target image in four subcategories: (1) deep similarity [49], (2) fully supervised, (3) unsupervised, and (4) dual-supervised methods.
Similar to other applications in deep-learning-based medical image analysis, supervised learning is also mainstream.Those attempts are well documented in the literature.Conceptually, under supervised learning, network models are trained to recognize transformations while minimizing the differences between those transformations and the known ground truth transformations, which is similar to the traditional deformable image registration (DIR) methods.The optimized loss function includes the mean squared error (MSE) and comparative loss so as to minimize the gap between the predicted value and the actual Tg (e.g., Equation ( 1)).Notably, Eppenhof et al. [59] proposed using synthetic random transformations in order to train a convolution neural network (CNN) to perform 3D CT lung deformable image registration (DIR).Eppenhof et al. [60] also improved the method by using U-net architecture.Both studies demonstrated the possibility of using CNN for direct transformation predictions.However, regarding registering multiple medical images, the ground truth transformations were often unknown.
Considering that, in actual supervised learning registration-due to the difficulty in obtaining real deformation field manually, and because the artificially synthesized label information may introduce unnecessary errors-there is increasing interest in developing unsupervised one-step transformation estimation frameworks [61].In the unsupervised learning methods of multimodal image registration, known transformations or registration labels are not available as training data.Thus, minimizing the loss functions that measure similarity (e.g., mutual information, normalized cross-correlation, etc.) and consistency (e.g., reciprocal motion consistency) between images are used to guide the training process.The Spatial Transformer Network (STN) [62] technique is one of the earliest methods.It performs spatial operations on data in the network, enabling it to actively perform spatial transformations on feature maps, with the feature maps themselves as conditions.This method is conducted without additional training supervision or modifications to the optimization process.Inspired by STN, more unsupervised registration methods have been developed.As one of the most promising methods of unsupervised learning on complex distributions in recent years, the image registration method based on Generative Adversarial Networks (GANs) has gradually become popular because it can introduce additional regularization, as well as perform image domain transformation so as to convert multimodal image registration to single-modal [63].
Another registration algorithm based on dual-supervised learning involves coupling the label information of samples with a measurement of the similarity between images in the training process.Fan et al. [64] proposed the Brain Image Registration Network (BIRNet), which uses the dual-supervised learning method to obtain the deformation field of 3D brain MRI images.The loss function measures the similarity between the template image and the deformed image to be registered, as well as the similarity between the predicted and actual deformation fields.Cao et al. [65] designed another dual-supervised learning method for cross-modality registration between CT and MRI.Specifically, using the aligned CT and MRI image pairs, the multimodal registration problem was transformed into a single-mode image registration problem, and similarity measures under a single modality were used instead of similarity measures between two modalities.In the loss function, the differences between CT-CT images and the similarities between MRI-MRI images were calculated.In general, compared with unsupervised learning, dual-supervised learning makes full use of labeled sample data information, which can make up for the shortcomings of unsupervised learning to a certain extent but still cannot avoid the restriction of using labeled information.

Image Registration
After multimodality images are spatially registered, we must combine the information with varying fusion technologies.The image fusion technologies in question can be further divided into three categories: (1) pixel-level fusion, (2) feature-level fusion, and (3) decisionlevel fusion.Pixel-level fusion is achieved by obtaining a fused image through directly processing the data acquired.This method requires a perfect registration of the source images to combine each pixel, but it is the most computationally efficient of the three categories [66]; it is also the basis of high-level image fusion and one of the focuses of current image fusion research.Due to its ability to preserve spatial details in fused images, this method is more suitable for medical imaging applications.
In contrast, feature-level fusion algorithms extract features of the source image and fuse them feature by feature.As a result, feature-level fusion may be more robust to noise and less sensitive to inconsistencies between two different modalities.Finally, decisionlevel fusion algorithms [67] allow for an effective combination of information at the highest level of abstraction, directly combining image descriptions and making decisions based on the ultimate purpose of the application.
Multimodal image integration in the context of clinical workflow (i.e., multimodal medical image fusion) is largely based on pixel-level fusion.Thus, our review below focuses on several mainstream fusion methods (as shown in Figure 5): spatial domain, transform domain, fuzzy logic, sparse representation fusion, and deep learning.
Diagnostics 2023, 13, x FOR PEER REVIEW 8 of 33 cross-modality registration between CT and MRI.Specifically, using the aligned CT and MRI image pairs, the multimodal registration problem was transformed into a single-mode image registration problem, and similarity measures under a single modality were used instead of similarity measures between two modalities.In the loss function, the differences between CT-CT images and the similarities between MRI-MRI images were calculated.In general, compared with unsupervised learning, dual-supervised learning makes full use of labeled sample data information, which can make up for the shortcomings of unsupervised learning to a certain extent but still cannot avoid the restriction of using labeled information.

Image Registration
After multimodality images are spatially registered, we must combine the information with varying fusion technologies.The image fusion technologies in question can be further divided into three categories: (1) pixel-level fusion, (2) feature-level fusion, and (3) decisionlevel fusion.Pixel-level fusion is achieved by obtaining a fused image through directly processing the data acquired.This method requires a perfect registration of the source images to combine each pixel, but it is the most computationally efficient of the three categories [66]; it is also the basis of high-level image fusion and one of the focuses of current image fusion research.Due to its ability to preserve spatial details in fused images, this method is more suitable for medical imaging applications.
In contrast, feature-level fusion algorithms extract features of the source image and fuse them feature by feature.As a result, feature-level fusion may be more robust to noise and less sensitive to inconsistencies between two different modalities.Finally, decision-level fusion algorithms [67] allow for an effective combination of information at the highest level of abstraction, directly combining image descriptions and making decisions based on the ultimate purpose of the application.
Multimodal image integration in the context of clinical workflow (i.e., multimodal medical image fusion) is largely based on pixel-level fusion.Thus, our review below focuses on several mainstream fusion methods (as shown in Figure 5): spatial domain, transform domain, fuzzy logic, sparse representation fusion, and deep learning.

Spatial Domain Fusion
Historically, spatial domain-based medical image fusion technology was of high interest.Its fusion methods were simple, and the fusion rules could be directly applied to Diagnostics 2023, 13, 2981 9 of 32 the source image pixels so as to obtain the fused image.The generic fusion methods in the spatial domain can be as follows: where F(•) represents a "fusion operator".Depending on the specific fusion operator selected, fusion methods include intensity-hue-saturation (IHS) transformation, high-pass filtering, principal component analysis (PCA), averaging, maximum selection, minimum selection, and Brovey transformation (BT) [68].
The remote-sensing research community widely uses the IHS image fusion technique to fuse multispectral (MS) and panchromatic (PAN) images.On the one hand, a panchromatic image is a high-resolution image with no colors, which shows a scene to the human eye (acquired using a single wavelength band).On the other hand, a multispectral image is a low-resolution image with many layers of the same scene that possess rich spectral information.Each layer is acquired at a particular wavelength band (for example, a redgreen-blue channel image) [69].Fusing MS and PAN images produces a combined image with a high spatial resolution and richer spectral information [70].
Principal component analysis (PCA) projects the data to its eigenspace.First, the images are divided into many matrix blocks.Second, for each matrix block, the principal eigenvector is calculated.Suppose x T and y T are eigenvectors of images I A and I B , respectively.Finally, fusion can be performed based on the rule IF = ω1 × I A + ω2 × I B , where ω1 and ω2 are weights based on x and y [71].
The Brovey (color normalization) transformation (BT) can be used to fuse the PAN and MS images.The BT preserves the high degree of spectral information of MS images, as well as the spatial information of PAN images.Each layer in the color of the MS image is multiplied by a ratio of the high-resolution PAN image data that has been divided by the sum of the color MS band images [72].
However, due to the spectral and spatial distortions in the fused image of the spatial domain, the popularity of this research area has declined in recent years.Researchers have started using spatial domain fusion strategies as a component of transformation domain methods to develop new research methods [67].

Transform Domain Fusion
Unlike the spatial domain fusion method, which directly processes image pixel values, the transform domain fusion method first converts the input image to a particular transform domain (e.g., frequency domain, wavelet domain, etc.).Then, it performs fusion operations in the transform domain.Finally, it transforms the fused representation to the spatial domain so as to obtain the fused image for visualization.The transformed domain-based methods can be summarized as follows, where T(•) represents a multiscale transformation, the superscript −1 denotes an inverse operation, and F(•) is the applied fusion operator.Many fusion algorithms adopt multiscale methods to reduce the algorithm's sensitivity to changes in the grayscale values of the source image, thereby improving the stability and effectiveness of the fusion process.Two classical transform domain fusion methods are briefly described below.
The traditional Laplacian pyramid-based fusion algorithm [73] can retain the edge details well, as well as make full use of the information at different scales.It represents an image as a series of quasi-bandpassed images, and it is localized in space and in a spatial frequency.Laplacian pyramid coding involves constructing a Gaussian pyramid, obtaining Laplacian pyramid images as the difference between successive Gaussian levels, quantizing them to yield the compressed code, and reconstructing the image via the expand-and-sum procedure.The Laplacian pyramid encoding scheme is computationally simple, local, and parallelizable, making it suitable for real-time processing.Based on this, Du proposed a method combining the Laplacian pyramid with multiple features to accurately transfer the salient features from the input of medical images to a single fused image, thus solving the problem of limited image contours and contrast representation [74].They used the Laplace pyramid to convert the input image into its multiscale representation, as well as extracted the contrast and contour feature maps from each scale of the image.This effective fusion scheme combines pyramid coefficients and obtains fused images through the reconstruction process of the inverted pyramid [74].
The method based on multiscale transformation is the most popular pixel-level fusion algorithm.The medical image fusion methods of the transform domain are mainly based on the multiscale transform (MST) theory, and they have also been the hotspots of research in recent years.Fusion methods are based on multi-resolution analysis, such as the discrete wavelet transform (DWT) [18,75] and the stationary wavelet transform (SWT) [10].The MST-based fusion method is divided into decomposition, fusion, and reconstruction.The source images are first decomposed into low-and high-frequency sub-bands via the appropriate MST method.Then, a fusion rule is applied to combine the sub-band coefficients of each source image.Finally, an inverse MST is performed to obtain the fused image [66].
Figure 6 shows the decomposition process of NSCT.NSCT is a shift-invariant, multiscale transform that decomposes the images based on the nonsubsampled pyramid (NSP) structure and nonsubsampled directional filter bank (NSDFB).The NSP decomposes the image into low-pass and bandpass-filtered images.Then, the bandpass images enter the NSDFB and are decomposed directionally into the sub-bands' 2D frequency domain.This method has been proven to effectively fuse CT and MRI scans [66].
Diagnostics 2023, 13, x FOR PEER REVIEW 10 of 33 frequency.Laplacian pyramid coding involves constructing a Gaussian pyramid, obtaining Laplacian pyramid images as the difference between successive Gaussian levels, quantizing them to yield the compressed code, and reconstructing the image via the expandand-sum procedure.The Laplacian pyramid encoding scheme is computationally simple, local, and parallelizable, making it suitable for real-time processing.Based on this, Du proposed a method combining the Laplacian pyramid with multiple features to accurately transfer the salient features from the input of medical images to a single fused image, thus solving the problem of limited image contours and contrast representation [74].They used the Laplace pyramid to convert the input image into its multiscale representation, as well as extracted the contrast and contour feature maps from each scale of the image.This effective fusion scheme combines pyramid coefficients and obtains fused images through the reconstruction process of the inverted pyramid [74].
The method based on multiscale transformation is the most popular pixel-level fusion algorithm.The medical image fusion methods of the transform domain are mainly based on the multiscale transform (MST) theory, and they have also been the hotspots of research in recent years.Fusion methods are based on multi-resolution analysis, such as the discrete wavelet transform (DWT) [18,75] and the stationary wavelet transform (SWT) [10].The MST-based fusion method is divided into decomposition, fusion, and reconstruction.The source images are first decomposed into low-and high-frequency sub-bands via the appropriate MST method.Then, a fusion rule is applied to combine the sub-band coefficients of each source image.Finally, an inverse MST is performed to obtain the fused image [66].
Figure 6 shows the decomposition process of NSCT.NSCT is a shift-invariant, multiscale transform that decomposes the images based on the nonsubsampled pyramid (NSP) structure and nonsubsampled directional filter bank (NSDFB).The NSP decomposes the image into low-pass and bandpass-filtered images.Then, the bandpass images enter the NSDFB and are decomposed directionally into the sub-bands' 2D frequency domain.This method has been proven to effectively fuse CT and MRI scans [66].
Non-subsampled shearlet transformation (NSST) is a non-subsampled multiscale transformation method based on the theory of shear wave transformation.NSST decomposes images into multiscale, multidirectional images through multiscale and multidirectional decomposition.First, an NSP is used as a multiscale decomposition filter to decompose the image into a low-frequency sub-band and a high-frequency sub-band.The high-frequency sub-bands are then decomposed using a shear filter (SF) to obtain multidirectional subbands.The NSST decomposition process does not sub-sample the NSP and SF, so the NSST is shift invariant.
The discrete wavelet transformation (DWT) decomposes the image into four high-and lowfrequency sub-bands by using a finite set of wavelets [76] that contain both spatial and temporal information (see Figure 7).Then, the source image bands are typically combined via averaging.Specifically, wavelet domain fusion methods use wavelet transformation to decompose an input image into sub-images of different scales and orientations.Then, fusion operations, such as weighted averaging, selection of maximum absolute value, etc., are performed on each subimage.Finally, the fused representation is transformed back to the spatial domain by an inverse wavelet transform approach.This method has limitations as it cannot capture the smoothness of the images and can have artifacts along the contour edges [66].

Nonsubsampled shearlet transformation.
Non-subsampled shearlet transformation (NSST) is a non-subsampled multiscale transformation method based on the theory of shear wave transformation.NSST decomposes images into multiscale, multidirectional images through multiscale and multidirectional decomposition.First, an NSP is used as a multiscale decomposition filter to decompose the image into a low-frequency sub-band and a high-frequency sub-band.The highfrequency sub-bands are then decomposed using a shear filter (SF) to obtain multidirectional sub-bands.The NSST decomposition process does not sub-sample the NSP and SF, so the NSST is shift invariant.

Discrete wavelet transformation.
The discrete wavelet transformation (DWT) decomposes the image into four high-and low-frequency sub-bands by using a finite set of wavelets [76] that contain both spatial and temporal information (see Figure 7).Then, the source image bands are typically combined via averaging.Specifically, wavelet domain fusion methods use wavelet transformation to decompose an input image into sub-images of different scales and orientations.Then, fusion operations, such as weighted averaging, selection of maximum absolute value, etc., are performed on each sub-image.Finally, the fused representation is transformed back to the spatial domain by an inverse wavelet transform approach.This method has limitations as it cannot capture the smoothness of the images and can have artifacts along the contour edges [66].

Fuzzy Logic Fusion
Fuzzy Logic Fusion [77] is an information fusion method based on fuzzy logic.Fuzzy logic is a mathematical framework handling information with uncertainty.It deals with this uncertain information by introducing fuzzy sets and fuzzy operations.The advantage of fuzzy logic fusion in medical image fusion lies in its ability to deal with uncertain, fuzzy, and incomplete information.Compared with traditional pixel-level fusion methods, fuzzy logic fusion can better preserve the information of the input image and reduce artifacts and distortions.In addition, the fuzzy logic fusion method has good interpretability and scalability, and it can be adjusted according to different application scenarios and requirements.
Fuzzy logic can be used in conjunction with other pixel-level techniques to enhance image contrast.Fuzzy logic uses a fuzzifier, an interference engine, a defuzzifier, fuzzy sets, and fuzzy rules to change the membership functions that determine the variable weight of each pixel.Javed et al. used wavelet decomposition and fuzzy logic with backpropagation and least mean square algorithms to successfully fuse MRI and PET scans of a higher quality than previous methods [78].

Fuzzy Logic Fusion
Fuzzy Logic Fusion [77] is an information fusion method based on fuzzy logic.Fuzzy logic is a mathematical framework handling information with uncertainty.It deals with this uncertain information by introducing fuzzy sets and fuzzy operations.The advantage of fuzzy logic fusion in medical image fusion lies in its ability to deal with uncertain, fuzzy, and incomplete information.Compared with traditional pixel-level fusion methods, fuzzy logic fusion can better preserve the information of the input image and reduce artifacts and distortions.In addition, the fuzzy logic fusion method has good interpretability and scalability, and it can be adjusted according to different application scenarios and requirements.
Fuzzy logic can be used in conjunction with other pixel-level techniques to enhance image contrast.Fuzzy logic uses a fuzzifier, an interference engine, a defuzzifier, fuzzy sets, and fuzzy rules to change the membership functions that determine the variable weight of each pixel.Javed et al. used wavelet decomposition and fuzzy logic with backpropagation and least mean square algorithms to successfully fuse MRI and PET scans of a higher quality than previous methods [78].

Sparse Representation Fusion
Sparse Representation Fusion is a multimodal technology based on sparse representation theory, and it is widely used [79] in medical image fusion.Sparse representation theory [80] states that a signal can be decomposed into a linear combination of a set of sparse vectors.This method first decomposes the image into sparse representations, combines these representations according to a specific fusion strategy, and then finally reconstructs the fused image.The advantage of sparse representation fusion in medical image fusion lies in its ability to effectively extract and preserve image feature information in the sparse domain.Through an appropriate fusion strategy, the complementarity of different image features can be achieved, thereby improving the quality of the fusion images.Yang was the first to apply coefficient theory to image fusion.He proposed an algorithm [81] derived from a sparse decomposition of multi-source images in the same dictionary.Specifically, the sparse coefficient calculated at the same (anatomical) position of multiple images is taken as the absolute value of the larger value to obtain the fused sparse coefficient, which is then combined with the dictionary to reconstruct the fused image.
The standard sparse representation method does not consider the inherent structure of sparse signals, i.e., non-zero elements that appear in clusters or groups (this structure is called group sparsity).To address this problem, Li et al. proposed a novel dictionarylearning method called Dictionary Learning with Group Sparsity and Graph Regularization (DL-GSGR) [82].This method ensures that the learned dictionary has low group coherence by alternating group sparse coding and dictionary updating, thus enabling the effective group sparse coding of any signal.In addition, the method uses graph regularization to model the geometric structure of atoms.
The limitations of fusion methods in this category are also noted in the literature.First, sparse representation fusion involves computationally intensive tasks such as dictionary learning, sparse coding, and reconstruction.Sparse representation fusion requires solving the L1 norm optimization problem, which leads to high computational complexity, thus increasing the computation time.Second, the performance of sparse representation fusion depends strongly on the learned dictionary.If the dictionary does not capture the image features well, the fusion may not be satisfactory.

Deep Learning Fusion
The medical image fusion methods in the spatial and transform domains resulted in a low correlation between the original and fused images [83].Deep learning methods have received rapid attention in the last decade.Compared with spatial domain and transform domain-based methods, deep learning methods have stronger adaptivity, robustness, and accuracy; in addition, they can complete the task in a shorter time and improve the quality of medical images.Therefore, deep learning methods have great potential in medical image fusion.Research on several common models, such as CNN, Restricted Boltzmann Machine (RBM), AutoEncoder (AE), and U-net, has been conducted in the field of medical image fusion.Among them, CNN is the most widely used.A general image fusion framework based on CNN is shown in Figure 8.
[80] states that a signal can be decomposed into a linear combination of a set of spars tors.This method first decomposes the image into sparse representations, combines representations according to a specific fusion strategy, and then finally reconstruc fused image.The advantage of sparse representation fusion in medical image fusion its ability to effectively extract and preserve image feature information in the sparse do Through an appropriate fusion strategy, the complementarity of different image fe can be achieved, thereby improving the quality of the fusion images.Yang was the f apply coefficient theory to image fusion.He proposed an algorithm [81] derived f sparse decomposition of multi-source images in the same dictionary.Specificall sparse coefficient calculated at the same (anatomical) position of multiple images is as the absolute value of the larger value to obtain the fused sparse coefficient, wh then combined with the dictionary to reconstruct the fused image.
The standard sparse representation method does not consider the inherent struct sparse signals, i.e., non-zero elements that appear in clusters or groups (this struct called group sparsity).To address this problem, Li et al. proposed a novel dictionarying method called Dictionary Learning with Group Sparsity and Graph Regularization GSGR) [82].This method ensures that the learned dictionary has low group coheren alternating group sparse coding and dictionary updating, thus enabling the effective sparse coding of any signal.In addition, the method uses graph regularization to m the geometric structure of atoms.
The limitations of fusion methods in this category are also noted in the liter First, sparse representation fusion involves computationally intensive tasks such a tionary learning, sparse coding, and reconstruction.Sparse representation fusion re solving the L1 norm optimization problem, which leads to high computational com ity, thus increasing the computation time.Second, the performance of sparse repre tion fusion depends strongly on the learned dictionary.If the dictionary does not ca the image features well, the fusion may not be satisfactory.

Deep Learning Fusion
The medical image fusion methods in the spatial and transform domains resul a low correlation between the original and fused images [83].Deep learning method received rapid attention in the last decade.Compared with spatial domain and tran domain-based methods, deep learning methods have stronger adaptivity, robustnes accuracy; in addition, they can complete the task in a shorter time and improve the q of medical images.Therefore, deep learning methods have great potential in medic age fusion.Research on several common models, such as CNN, Restricted Boltzman chine (RBM), AutoEncoder (AE), and U-net, has been conducted in the field of m image fusion.Among them, CNN is the most widely used.A general image fusion f work based on CNN is shown in Figure 8.  CNN is a typical deep learning model [84] that attempts to learn hierarchical feature representation mechanisms for signal/image data with different levels of abstraction.Performing activity level measurement and fusion rules is often difficult in existing image fusion methods.Through using a learning CNN, a multi-focus image fusion method [74] based on deep learning was proposed to solve this problem by directly mapping the relationship between the source and fused images.
In recent years, with the popularity of transformers, many researchers have also applied CNN in image fusion.Early methods applied a CNN-based multi-focus algorithm, creating a decision map based on the image focus that determines how each pixel is weighted [85].However, this method works only in the spatial domain and requires that the source images have similar local structures.Unfortunately, this requirement cannot be satisfied in multimodal image fusion under many circumstances.Liu et al. addressed this problem by decomposing the images via a Laplacian pyramid and a weight map [86] instead of direct coefficients so as to preserve source image detail.A combination of CNN and sparse autoencoders (SAE) was used to fuse multimodal neural image data from MRI and FDG-PET [87], thus achieving significant performance improvement in both directions compared with the single modality method.Zhao et al. proposed a novel Correlation-Driven Decomposition Fusion Network (CDDFuse) to address the challenges of cross-modal feature modeling and expected feature decomposition.The network employs Restormer blocks to extract shallow cross-modal features, a dual-branch Transformer CNN feature extractor with Lite Transformer (LT) blocks for processing low-frequency global features, and Invertible Neural Network (INN) blocks for extracting high-frequency local information.With a correlation-driven loss, CDDFuse can make the low-frequency features correlate while keeping high-frequency components uncorrelated, thus further improving image fusion performance.Experiments showed promising results for CDDFuses in various fusion tasks, as well as enhanced performance for downstream infrared-visible semantic segmentation and object detection.

Current Clinical Applications of TEE + for Intraprocedural Image Guidance
In 1972, two-dimensional TEE was first used to evaluate the results of mitral valve surgery [88].The development of matrix array transducers (50 rows by 50 columns) enabled real-time 3D echocardiographic data acquisition.Currently, major ultrasound vendors offer 3D/4D TEE through their matrix array transducers (Z6M True Volume 3D TEE Probe, Siemens Healthcare, Mountainview, CA, USA; X7-2t, Philips Healthcare, Andover, MA, USA, 6VT-D, GE Healthcare, Wauwatosa, WI, USA).As a result, modern TEE provides useful real-time three-dimensional morphologic assessment and hemodynamic interrogation.Technological developments, including multimodal integration with other modalities, have expanded TEE's clinical applications for examining cardiac function, structures, and hemodynamics [89].
Recent clinical guidelines indicate the importance of echocardiography in the perioperative evaluation of heart disease.For example, TEE images are used to confirm the position and size of perivalvar leaks, as well as to obtain measurements of the aortic annulus and aorta during surgery, evaluate the intraprocedural progress of aortic balloon valvuloplasty (ABV), and assist in locating the catheter and valve position during percutaneous transcatheter heart valve (THV) deployment [90].TEE guidance is also used in the interventional management of congenital heart defects like ASD and VSD closure, in the surgical repair of the aorta and aortic dissections, as well as in other applications such as valve repair, cardiopulmonary transplantation, the insertion and monitoring of ventricular assist devices, the assessment of the site and severity of paravalvular regurgitation [91], etc.
However, the field of view of TEE transducers is still relatively limited, making it challenging to examine the entire region of interest without moving the transducer.In many centers, fluoroscopy and TEE or cardiac CT are displayed on different screens in different orientations during an SHD procedure.Without automated live multimodal integration, interventionalists must mentally integrate multimodal images to understand the anatomy and spatial relationships during the procedure.Mental image integration may become problematic during communication amongst clinicians when mentally integrated images differ from person to person.In contrast, fused images improve eye-hand coordination as multimodal imaging information is overlaid.Furthermore, virtual markers can be added to fused images to improve visual coordination.These virtual markers in live integration allow for the democratization of multimodal imaging, in which multiple clinicians can view the same region of interest created by fused medical imaging, which may minimize misunderstandings, especially in cases involving complex or atypical anatomy.

TEE-fluoroscopy fusion has been tested on many SHD interventions in the clinical
setting.Recall that TEE's limitations include operator dependency, low image quality, and poor temporal resolution (if 3D data acquisition is used).Furthermore, TEE imaging alone can have a difficult time visualizing cardiac structures due to acoustic artifacts (e.g., prosthetic intracardiac devices and eructation), thus prohibiting imaging windows that may be essential to view specific regions of interest, such as identifying the location of paravalvular leaks after a prosthetic valve implantation.
Two integrated (TEE/fluoroscopy) systems are commercially available in the clinical workflow.The EchoNavigator software (Philips Healthcare, Best, The Netherlands) performs real-time 3D TEE and fluoroscopy fusion on a single screen.EchoNavigator starts with TEE probe localization and calibration.More specifically, EchoNavigator software [92] automatically finds and tracks the TEE probe's position and direction within the fluoroscopic image.Post-co-registration, the EchoNavigator system automatically aligns the two images and compensates for any change of position in the C-arm [69].In 2017, TrueFusion™ fusion-imaging technology (Siemens Healthineers, Erlangen, Germany) also received US FDA clearance for providing image guidance during SHD procedures.In a TrueFusion system, fluoroscopy images are registered with echo data by machine learning algorithm-based probe detection.Then, automated registration can integrate the two modalities.In both systems, markers labeled on echography are automatically transferred to the registered fluoroscopic data.
There are also some TEE-fluoroscopy prototype research systems.One approach [72] involves automatically relocating the 3D TEE image whenever the C arm of fluoroscopy moves by using image-based tracking and allowing the echocardiographer to manipulate the image through using a table-sided mouse.A discriminative learning-based method [90], which generates the required training data by automatically generating a single-volume TEE image, has been proposed to adapt to actual X-ray data, thereby avoiding manual labeling costs.This method uses unlabeled perspective images to estimate the differences in feature space density, and it allows for the correction of covariate bias through instance weighting so as to fuse image data from transesophageal ultrasound (TEE) and X-ray perspective imaging for use in the minimally invasive treatment of SHD.The evaluation shows that, compared with cross-validation on the test set, this method reduces the detection of faults by 95% and the positioning error from 1.5 mm to 0.8 mm [88][89][90][91][92][93].

Clinical Applications of TEE + Fusion Imaging 4.2.1. Transseptal Puncture
Transseptal puncture (TSP) is the most common transcatheter intervention for treating SHD.It is an interventional technique where a needle is used to puncture the fossa ovalis to access the left atrium.The procedure occurs typically in the initial stage of more complicated procedures such as left atrial appendage occlusion or percutaneous mitral valve repair/replacement.Several studies [94][95][96] have shown that combining echocardiographic images with X-ray fluoroscopy images in real time can aid in guiding the puncture procedure.Figure 9 shows that echocardiography provides excellent visualization of transeptal puncture.
In the literature, multiple reports found that using integrated TEE/fluoroscopy (i.e., only EchoNavigator) significantly decreased procedure time [97].However, some studies reported reduced radiation dosage, while other studies found the radiation dose did not change [98,99].In the literature, multiple reports found that using integrated TEE/fluoroscopy (i.e., only EchoNavigator) significantly decreased procedure time [97].However, some studies reported reduced radiation dosage, while other studies found the radiation dose did not change [98,99].

Left Atrial Appendage Occlusion
LAA occlusion, in which a device is implanted percutaneously to occlude the mouth of a left atrial appendage, is a procedure that requires multimodal imaging for planning and guidance.For these procedures, a live multimodal imaging system could be useful during three key stages: (1) the selection of a transseptal puncture site and subsequent guidance, (2) anatomical orientation and device deployment, and (3) post-deployment evaluation to ensure adequate sealing and device stability.The use of TEE-fluoroscopy systems for transseptal puncture was described in the previous section.After completing the transseptal puncture, there are two other major steps: transseptal crossing of the guiding catheter into the left atrium and the deployment of the occlusion device into the LAA.Some interventionalists perform these two steps using fluoroscopy alone.However, TEE-fluoroscopy integration offers additional benefits, as shown in Figure 10.Furthermore, 2D TEE can measure the orifice of the appendage in different 2D planes, while 3D TEE can more accurately assess the appendage's perimeter and landing zone (see Figure 11).As a result, the device size can be better selected [100], and visualizing the LAA landing zone allows the device to be appropriately implanted.

Left Atrial Appendage Occlusion
LAA occlusion, in which a device is implanted percutaneously to occlude the mouth of a left atrial appendage, is a procedure that requires multimodal imaging for planning and guidance.For these procedures, a live multimodal imaging system could be useful during three key stages: (1) the selection of a transseptal puncture site and subsequent guidance, (2) anatomical orientation and device deployment, and (3) post-deployment evaluation to ensure adequate sealing and device stability.The use of TEE-fluoroscopy systems for transseptal puncture was described in the previous section.After completing the transseptal puncture, there are two other major steps: transseptal crossing of the guiding catheter into the left atrium and the deployment of the occlusion device into the LAA.Some interventionalists perform these two steps using fluoroscopy alone.However, TEE-fluoroscopy integration offers additional benefits, as shown in Figure 10.Furthermore, 2D TEE can measure the orifice of the appendage in different 2D planes, while 3D TEE can more accurately assess the appendage's perimeter and landing zone (see Figure 11).As a result, the device size can be better selected [100], and visualizing the LAA landing zone allows the device to be appropriately implanted.In the literature, multiple reports found that using integrated TEE/fluoroscopy (i.e., only EchoNavigator) significantly decreased procedure time [97].However, some studies reported reduced radiation dosage, while other studies found the radiation dose did not change [98,99].

Left Atrial Appendage Occlusion
LAA occlusion, in which a device is implanted percutaneously to occlude the mouth of a left atrial appendage, is a procedure that requires multimodal imaging for planning and guidance.For these procedures, a live multimodal imaging system could be useful during three key stages: (1) the selection of a transseptal puncture site and subsequent guidance, (2) anatomical orientation and device deployment, and (3) post-deployment evaluation to ensure adequate sealing and device stability.The use of TEE-fluoroscopy systems for transseptal puncture was described in the previous section.After completing the transseptal puncture, there are two other major steps: transseptal crossing of the guiding catheter into the left atrium and the deployment of the occlusion device into the LAA.Some interventionalists perform these two steps using fluoroscopy alone.However, TEE-fluoroscopy integration offers additional benefits, as shown in Figure 10.Furthermore, 2D TEE can measure the orifice of the appendage in different 2D planes, while 3D TEE can more accurately assess the appendage's perimeter and landing zone (see Figure 11).As a result, the device size can be better selected [100], and visualizing the LAA landing zone allows the device to be appropriately implanted.TrueFusion technology has been found to significantly improve procedure efficiency and reduce contrast use [37].Similar findings were reported by Jungen et al. when using the EchoNavigator system [37].

Mitral Valve Repair
During the procedure, 3D-TEE is often used to evaluate the morphology and function changes of the mitral valve, including the leaflets, annulus, and subvalvular apparatus.TEE can also be used to evaluate the effectiveness of mitral valve repair [101].However, the orientation of the guiding system and the dedicated structures of valve implants (e.g., MitraClip ® ) are less visible through TEE.The fluoroscopy system serves implant tracking considerably better.Therefore, a multimodality approach for the guidance of MitraClip ® implantations when using 2D/3D TEE and fluoroscopy is important.
The EchoNavigator software was shown to reduce the radiation dose and treatment time for MitraClip procedures [102].As shown in Figures 12 and 13, TEE can already provide excellent information.Visualization can be further improved with the fusion imaging provided to interventionalists.Furthermore, the guidance provided by virtual marker placement on TEE images, with automatic transfer to fluoroscopic images, can help prevent injury to the left atrium and atrial septum during clip implantation [103].It has also been demonstrated that complex interventions with multiple clips have an increasingly higher reduction in radiation and treatment time when using TEE-fluoroscopy fusion software.TrueFusion technology has been found to significantly improve procedure efficiency and reduce contrast use [37].Similar findings were reported by Jungen et al. when using the EchoNavigator system [37].

Mitral Valve Repair
During the procedure, 3D-TEE is often used to evaluate the morphology and function changes of the mitral valve, including the leaflets, annulus, and subvalvular apparatus.TEE can also be used to evaluate the effectiveness of mitral valve repair [101].However, the orientation of the guiding system and the dedicated structures of valve implants (e.g., MitraClip ® ) are less visible through TEE.The fluoroscopy system serves implant tracking considerably better.Therefore, a multimodality approach for the guidance of MitraClip ® implantations when using 2D/3D TEE and fluoroscopy is important.
The EchoNavigator software was shown to reduce the radiation dose and treatment time for MitraClip procedures [102].As shown in Figures 12 and 13, TEE can already provide excellent information.Visualization can be further improved with the fusion imaging provided to interventionalists.Furthermore, the guidance provided by virtual marker placement on TEE images, with automatic transfer to fluoroscopic images, can help prevent injury to the left atrium and atrial septum during clip implantation [103].It has also been demonstrated that complex interventions with multiple clips have an increasingly higher reduction in radiation and treatment time when using TEE-fluoroscopy fusion software.

Atrial Septal Defect Occlusion
In the case of the percutaneous closure of atrial septal defect (ASD), TEE, i.e., real-time 3D-TEE, has now been widely established as a primary modality for intraprocedural guidance for ASD occlusion.In leveraging the availability of miniaturized matrix array transducers,

Atrial Septal Defect Occlusion
In the case of the percutaneous closure of atrial septal defect (ASD), TEE, i.e., realtime 3D-TEE, has now been widely established as a primary modality for intraprocedural guidance for ASD occlusion.In leveraging the availability of miniaturized matrix array transducers, 3D-TEE can offer image guidance in real time (see Figure 14), and it allows for accurate measurements of the ASD (i.e., morphology, size, position, and adequacy of the rims around the ASD defect), along with device visualization during the procedure and in post-deployment.The ASD measurements derived from 3D-TEE imaging are essential for determining which device(s) are suitable for ASD closure [104].Because it is a real-time modality, 3D-TEE can also be used to accurately localize guidewires and catheters [103] during the procedure, e.g., visualizing the wires crossing the ASD and the device's deployment (see Figure 14).Real-time 3D guidance has significantly reduced the procedure time by removing the need for balloon sizing and reducing the radiation dose.Real-time CT-fluoroscopy fusion has been attempted in PVL closures, especially in apical surgery [105].More recently, EchoNavigator was used to assist interventionalists for treating PVL [106,107].Ahmed et al. particularly noted that EchoNavigator helped identify the exact site, size, depth, and shape of the PVL, and it subsequently facilitated navigating a guidewire across the leak [107].TEE imaging can be useful in PVL cases, as shown in Figure 15; however, imaging may be further limited because the presence of the prosthetic valves creates acoustic artifacts (i.e., acoustic shadowing and reverberations).Thus, TEE image quality may be significantly affected [108].

Transcatheter Aortic Valve Replacement
Transcatheter aortic valve replacement (TAVR) is a valuable alter who are not good candidates for surgery (e.g., patients with severe, sym nosis).Interventionalists require essential information about the size and aortic root and its surrounding structures to plan the TAVR [109].Un TEE/fluoroscopy system, TEE is often used to identify significant land root, the aortic cusps, etc., which are invisible on fluoroscopic images.Th tion (e.g., the orientation of the prosthetic valve) is also critically impor ducted via the fluoroscopic system.
Under a TEE/fluoroscopy system, interventionalists can flag anato the echocardiography system, and those virtual markers can be transferr scopic system.For example, interventionalists can first mark points thro the valve via the TEE (see Figures 16 and 17), and they can then find ad are orthogonal to the annulus (i.e., finding a longitudinal direction of t

Transcatheter Aortic Valve Replacement
Transcatheter aortic valve replacement (TAVR) is a valuable alternative for patients who are not good candidates for surgery (e.g., patients with severe, symptomatic aortic stenosis).Interventionalists require essential information about the size and morphology of the aortic root and its surrounding structures to plan the TAVR [109].Under the integrated TEE/fluoroscopy system, TEE is often used to identify significant landmarks of the aortic root, the aortic cusps, etc., which are invisible on fluoroscopic images.The device visualization (e.g., the orientation of the prosthetic valve) is also critically important but often conducted via the fluoroscopic system.
Under a TEE/fluoroscopy system, interventionalists can flag anatomical landmarks in the echocardiography system, and those virtual markers can be transferred onto the fluoroscopic system.For example, interventionalists can first mark points through the annulus of the valve via the TEE (see Figures 16 and 17), and they can then find additional points that are orthogonal to the annulus (i.e., finding a longitudinal direction of the prosthetic valve when using fluoroscopy).The availability of the longitudinal direction facilitates the device deployment.
Under a TEE/fluoroscopy system, interventionalists can flag anatomical landmarks in the echocardiography system, and those virtual markers can be transferred onto the fluoroscopic system.For example, interventionalists can first mark points through the annulus of the valve via the TEE (see Figures 16 and 17), and they can then find additional points that are orthogonal to the annulus (i.e., finding a longitudinal direction of the prosthetic valve when using fluoroscopy).The availability of the longitudinal direction facilitates the device deployment.

Discussion
Percutaneous structural heart technologies may enable the treatment of patients with minimally invasive approaches.Right now, those percutaneous technologies enable the treating of SHD patients with a high surgical risk.The increasing use of percutaneous structural heart technologies demands a robust approach to intraprocedural guidance;

Discussion
Percutaneous structural heart technologies may enable the treatment of patients with minimally invasive approaches.Right now, those percutaneous technologies enable the treating of SHD patients with a high surgical risk.The increasing use of percutaneous

Discussion
Percutaneous structural heart technologies may enable the treatment of patients with minimally invasive approaches.Right now, those percutaneous technologies enable the treating of SHD patients with a high surgical risk.The increasing use of percutaneous structural heart technologies demands a robust approach to intraprocedural guidance; one that provides detailed, real-time 3D spatial orientation and a definition of cardiac structures and defects.That is why augmenting intraprocedural imaging must be one of the priorities when starting or growing a structural heart program.
Although intraprocedural 3D TEE guidance offers the best (overall) solution, at least in the SHD arena, it has not been accepted by all centers as a standard of care.The limitations of TEE (e.g., small field of view) can be mitigated by multimodal image integration.Clinically, evaluating the effectiveness of TEE-centric multimodal integration is determined by the following metrics: (1) increased procedural success rate, (2) reduced procedure time, (3) lower radiation exposure, (4) reduced complication rates, and (5) the need for lesser contrast media.According to TEE guidelines by the American Society of Echocardiography, the Society of Cardiovascular Anesthesiology, and the Society of Thoracic surgeons, TEE adds value to surgical decision making in the operating room, and it has contributed to improved outcomes during the treatment of patients with SHD [110].
Advances in imaging technology have begun to allow for the fusion of multiple imaging modalities.Reliance on a single modality is not always appropriate for cardiac interventions.As demonstrated by TEE-fluoroscopy and CT-fluoroscopy fusion software, multimodal imaging provides significant benefits by essentially merging the strengths of separate imaging modalities.Fusion imaging may provide the most useful diagnostic information for the interventionalist and the surgeon, consequently allowing the patients to receive the best care possible.Recent research has shown that the fusion of real-time TEE with fluoroscopy enhances cardiac interventions and catheter guidance.In contrast, TEE and CT fusion advance SHD diagnoses and interventional device sizing.TEE-fluoroscopy fusion has many advantages, as shown by the lowered procedure times and radiation doses in mitral valve repair and left atrial appendage occlusion (LAAO).Fluoroscopy and CT are the only other image scans being fused with TEE in a clinical setting.
While the current TEE-based multimodal imaging (i.e., EchoNavigator) is an excellent beginning, there are still limitations.The primary limitations of TEE-fluoroscopy imaging are two-fold.First, merging 3D TEE imaging volumes on a 2D angiographic plane is counter to the goal of multimodal imaging since it detracts from the utility of the 3D TEE imaging volumes.Second, the visualization of 3D imaging volumes on a 2D screen inhibits effective 3D spatial navigation of the anatomy.To realize immersive multi-modal 3D imaging, 3D volumes of different imaging modalities will need to be fused and then visualized on an augmented or mixed reality platform.In the multimodal image integration setting, if each modality's strength is independently realized and combined with an augmentation of characteristics that are preferred for that specific intervention, then multimodal image integration would have come to age.
Unfortunately, some controversies remain regarding the TEE+ fusion strategy.Several studies have shown that TEE+ fusion leads to shorter procedural duration, reduced radiation exposure, and improved patient outcomes.However, due to the lack of standardization, the discrepancies in those studies are difficult to reconcile.For instance, the procedure time was a variable in several studies, and a possible explanation could be that the fusion technology may be more helpful for novice teams in facilitating the learning curve of the procedures.Also, how such a TEE+ fusion system impacts routine clinical practice and patient outcomes compared with the current standard of care is not entirely clear.Furthermore, the current system enhances the use of anatomical information only.In other words, function information (e.g., hemodynamics and soft tissue information) has not been added so that the treatment of SHD can be improved.For instance, using the hemodynamic information available from TEE may enable interventionalists to eliminate the need to use pulmonary artery catheters, thus reducing patient safety risks during the procedure.Ultrasound-based elastographic information can also be used to assess the biomechanical properties of the myocardium, which could be important for some surgical SHD procedures, such as valve repair.We envision that developing a TEE+ fusion system that offers combined anatomical, functional, and automated ultrasound probe manipulation is a worthy direction to pursue.
We anticipate the research to propel the adaptation of artificial intelligence (AI) in the three major components of multimodal image integration (segmentation, registration, and fusion).Augmented and virtual reality following multimodal image integration will likely elevate the use of medical imaging for treating patients with SHD.Such an AI-powered software platform (from multimodal image integration to advanced visualization) could also be foundational in training future medical professionals.AI-guided visualization and automated skill assessments built into our system may accelerate clinicians' transition from novice to expert.Success stories in TEE training will be exponential if appropriate 3D visualization-based training on an integrated platform can be established [111,112].
Another imaging technique that aids in the planning and guidance of interventions for SHD is 3D printing (3Dp).The benefits of point-of-care 3Dp anatomic models have become increasingly recognized over the past decade.Patient-specific anatomic models allow for a unique, tactile understanding of anatomic features, especially in the context of complex congenital heart disease.Furthermore, 3Dp also allows for ex vivo simulations of transcatheter device deployment, providing another level of assurance during the interventional planning of atypical cases.The maturity and accessibility of 3D printing technology, as well as the application of virtual and augmented reality technology, can help physicians better understand the patient's cardiac anatomic relationships, thus providing unique opportunities to improve surgical training and pre-operative planning.However, the time and cost required to create models, the difficulty in segmenting finer cardiac structures such as cardiac valves, and the inability to reconstruct dynamic motion limits the overall utility of 3D printing for cardiac cases [113].However, AI-guided segmentation could assist in overcoming some of these obstacles.
The latest research shows that extended reality (XR) technology [79] Is expected to be an alternative to other 3D technologies in assisting in cardiac interventional planning and guidance.This technology has the potential to offer an intuitive and immersive experience in a three-dimensional space, enabling a thorough understanding of complex spatial relationships when planning and guiding cardiac procedures for congenital and structural heart diseases.This technology surpasses traditional 2D and 3D image displays, but must overcome several challenges to be applied safely and effectively in the clinical space.
As reviewed in this paper, multimodal image integration has advanced in the past decade, from inaccurate spatial transformations to highly complex deep learning algorithms.Now, commercially available packages (e.g., EchoNavigator (Philip]) and TrueFusion (Siemen])) can be used in the clinical workflow; moreover, augmented intraoperative image guidance is now accessible for SHD interventions.A limited number of SHD interventions are performed using the multimodal image fusion of TEE and another imaging technique.With the broad spectrum of available SHD treatments and the small amount of research on multimodal TEE-based guidance, there is still room for innovations in this field.Notably, it would be interesting to fuse MR images with TEE scans, as MRI produces a much-improved characterization of soft tissues or uses ultrasound-based tissue characterization to augment conventional TEE scans.
Furthermore, complex interventions needing image fusion may be more time consuming than a simple procedure that may not need image fusion.A simple ASD closure, for example, does not need image fusion and could be performed in a few minutes, whereas a complex ASD morphology may take much longer, even with multimodal image integration.
There is enormous potential for AI solutions in interventional cardiology.One such area is deep-learning-based image fusion.For instance, current state-of-the-art multimodal image integration is still at the pixel level and primarily limited by two complementary modalities.More specifically, AI can learn from examples without relying on program-defined rules to figure out complex associations from multimodal image data beyond the current clinical practices, thus becoming a disruptive force in advancing intraoperative image guidance for treating SHD.

Conclusions
This paper reviews multimodal medical imaging modalities, the development of medical image fusion techniques, and recent advances in SHD treatment based on TEE fusion.It offers some perspectives for future research in this area.
With the advent of minimally invasive and percutaneous interventions in treating complex SHD, it is essential to consider live multimodal integration with advanced visualization systems as the platform for such interventions.With the potential to provide lower radiation doses and treatment times, TEE fusion imaging provides an enhanced visualization of cardiac anatomy for interventional procedures.New fusion techniques, like deep learning, can align TEE with other images for real-time catheter guidance.Further research can test these AI-based fusion methods with TEE-CT and TEE-fluoroscopy imaging in a clinical setting.Furthermore, TEE-CT fusion can be introduced to real-time interventions following the completion of more research on their deep learning fusion.Echocardiography This paper discussed a method for the intraoperative echocardiographic measurement of mitral valve functions.The proposed method can be used for the postoperative evaluation of valvular functions, pericardial effusions, and cardiac outputs.[88] 1995 TEE This paper described the roles of TEE in cardiac operations; surgical decision-making processes; hemodynamic monitoring and problem-solving; mitral valve repair; and intraoperative and postoperative stages.Additionally, this study underscored clinicians' need to undergo training in TEE to manage situations adeptly.[101] 2004 Cardiovascular Ultrasound This article explained the impact of occupational exposure to ionizing radiation and ultrasound.It was observed that interventional cardiologists and electro-physiologists have a higher exposure compared to other health professionals. [30]

MR, CT, US, PET, and SPECT
This study is a concise overview of the fundamental principles of molecular imaging.It also discussed the sensitivity and spatial resolution aspects of imaging parameters, the balance between spatial resolution and image contrast, and the dimensions of the target.
[10] This review explained the complications of TEE probe manipulation in oropharyngeal, esophageal, gastric trauma, or other risks.The authors suggested that epicardial echocardiography could be an alternative imaging method for TEE when TEE probe placement is not feasible.[35] 2011 TEE This article discussed the role of real-time three-dimensional images of TEE.Topics include the accurate assessment of LAA anatomy, suitability for device implantation, continuous visualization of all intracardiac devices and catheters during procedures, and a clear delineation of device positioning in the LAA. [100]

CT fluoroscopy and MR fluoroscopy
This review explained the scope of MR fluoroscopy in invasive cardiac catheterization procedures.[11] 2012 TEE This article showed a case study of TEE.The authors believe that using 3D transesophageal echocardiography (TEE) provided additional valuable insights beyond 2D TEE, and aided in precisely identifying the dehiscence of the mitral valve repair.This enhanced clarity significantly contributed to the interpretation process and surgical preparation. [38]

TEE and X-ray fluoroscopy registration
This article showed a technique of TEE in X-ray fluoroscopy registration.The method was evaluated using phantom data and five clinical datasets from cardiac EP and TAVI procedures. [92]

TTE, TEE, CMR, MDCT, and angiography
This article analyzed the use and effectiveness of multi-modal images in pre-procedural, intraprocedural, and long-term transcatheter aortic valve replacement procedure follow-up.
[109] 2013 Echo, ICE, CT, and MR fluoroscopy fusion This article explored the practical uses of multimodal imaging tools in modern cardiac catheterization laboratories.These include 3D echocardiography, intracardiac echocardiography, and the overlay of CT scans with fluoroscopy.[18] 2013 CT This review offered a comprehensive outline of the noncoronary cardiac structures that can be assessed using multidetector CT.It delineated the well-established, appropriate clinical applications of multidetector CT for evaluating structural heart disease alongside emerging periprocedural clinical uses. [29]

TEE and X-ray fluoroscopy registration
This article discussed a discriminative learning-based approach to calculate the transformation of the TEE transducer in X-ray images.
The procedure was validated with more than 1900 images and concluded that image integration reduced detection failures by 95%, as well as improved the localization error from 1.5 to 0.8 mm. [93] 2014 EchoNavigator This article discussed an examination of EchoNavigator that encompassed two aspects: 1. the cumulative radiation dose and 2. fluoroscopy duration and overall procedure time.It was observed that EchoNavigator simplifies catheter manipulation through an automated, real-time fusion of echocardiography and fluoroscopy. [102]

TTE, TEE, CT, and MR
This paper assessed the function of 2D and 3D cardiac imaging approaches in the interventional treatment of patients dealing with symptomatic PVL. [108]

CT, TTE, and TEE
The effectiveness of minimally invasive surgery hinges on well-considered decisions concerning the nature and placement of incisions, the approach to cannulation, and the strategy for cardioprotection.These choices are made smoother with the assistance of preoperative imaging, which constitutes the central theme of this review.
[15] This article studied percutaneous left atrial appendage (LAA) closure that is guided by the automated image integration of 2D/3D-transesophageal echocardiography and fluoroscopy imaging.The image integration helps the intervention without increasing time and reducing radiation exposure.
[37] 2015 The EchoNavigator system to register echo and X-ray images This article discussed EchoNavigator in terms of registering the movement of the X-ray arc with the TEE probe for transseptal punctures.[94] 2015 EchoNavigator This article described the potential use of EchoNavigator in transseptal puncture and left atrial appendage closure.[98] 2015 EchoNavigator EchoNavigator is used for left atrial appendage, atrial septal defect, paravalvular leak closure, transaortic valve repair, and MitraClip procedures.Using EchoNavigator showed an enhanced assurance among interventional cardiologists when it came to directing and placing interventional tools, as well as in aiming to heighten the safety, precision, and effectiveness of percutaneous procedures in catheter procedures. [103]

TEE, CT, and MR
This article showed the importance of imaging proficiency, which plays a pivotal role in making decisions and managing patients with structural heart conditions.It is foreseeable that the field of interventional imaging will emerge due to a growing demand for top-tier imaging capabilities.[12] 2016 EchoNavigator This article showed that EchoNavigator helps the understanding of anatomical structures and that it is feasible and safe, improves the confidence of interventional cardiologists in the targeting and positioning of devices in percutaneous interventions, reduces procedural time, and reduces the dosage of contrast and radiation.
[99] 2016 Fluoroscopy, TTE, TEE, CT, and MR images This paper reviewed cardiac imaging that was used in the catheterization of SHD.

Magellan robotic system
This article explored the proficiency in performing TAVI surgery on a silicon phantom model when using traditional surgical instruments and the Magellan endovascular robotic platform.Furthermore, the abilities of experienced practitioners and those who were new to the procedure were compared. [111]

CT, TEE, and fluoroscopy
This review aimed to emphasize the importance of utilizing cardiac computed tomography, fluoroscopy, and fusion imaging as essential guiding tools in the implementation of transcatheter aortic valve replacement, as well as in the various percutaneous approaches for diagnosing and managing complex structural heart issues.[20] 2017 CMR This study aimed to proactively evaluate the effectiveness of utilizing cardiac magnetic resonance imaging (MRI) and programmed ventricular stimulation to predict adverse long-term outcomes in patients with premature ventricular contractions (PVCs) who were undergoing radiofrequency ablation.Before the ablation procedure, conducting cardiac MRI and programmed stimulation can provide valuable insights for risk assessment in patients experiencing frequent PVCs.Additionally, individuals with inducible ventricular tachycardia in the presence of structural heart disease might derive benefit from implantable cardioverter-defibrillator placement post-ablation, regardless of their left ventricular ejection fraction. [24]

X-ray phase-contrast CT
This article showed a study on five non-pretreated whole-heart specimens that were obtained by autopsy via X-ray phase-contrast CT.The obtained images distinctly depicted various structures accurately.X-ray phase-contrast tomography can achieve detailed, three-dimensional imaging of human heart specimens with exceptional resolution.This article described a few clinical applications of EchoNavigator, such as transseptal punctures, mitral valve repair when using the MitraClip device, LAA closure, and the percutaneous closure of PVLs. [96]

TEE and AI
This article discussed the artificial intelligence models employed to evaluate (based on criteria percentage and overall impression) TEE images.These AI assessments were juxtaposed against evaluations by human raters, and the AI's projected outcomes closely mirrored the expert assessments.This AI-driven approach also holds promise in aiding novice TEE trainees.
[112] 2018 3D-printing, 3D-VR headset, and desktop delivery This study assessed the efficiency of three distinct 3D techniques for preoperative planning and management.The results revealed that clinicians expressed a willingness to incorporate 3D virtual reality.[113] 2019 CT This article examined the preoperative CTs of 100 individuals who underwent a mini-sternotomy procedure.This study investigated the connections between access distance, access angle, annulus dimensions, and calcium volume with the duration of cardiopulmonary bypass (CPB) and aortic cross-clamp (AoX) times.
The findings showed that increased calcium volume is linked to more prolonged CPB and AoX times, with age and sutureless valve implantation independently influencing this association.Interestingly, contrary to prior research, the access angle was not associated with the procedure's complexity.[17] 2019 Fluoroscopy This article utilized thermoluminescent dosimeters to measure the radiation exposure across various body regions for patients and operating room medical staff.These findings represent an initial comprehensive comparison of the real-life radiation exposure among operating room personnel and patients when comparing navigation and fluoroscopy usage.[21] 2019 CT This document explained the necessity of establishing formalized, standardized, and measurable training prerequisites for CT imaging in structural heart diseases.[33] 2019

Critical care echocardiography
This study is a review of CCE.CCE is used in the ICU and emergency department, perioperative settings, and complete hemodynamic monitoring.Also, the article discussed training programs on both basic and advanced CCE, as well as international-level certifications for advanced CCEs. [89]

Echocardiography and fluoroscopy fusion
This article analysis elucidated the application of imaging technology that fuses echocardiography and fluoroscopy, thereby detailing its present applications in congenital and structural heart conditions.It also points toward forthcoming advancements that will further elevate the utility of this distinctive imaging technology for guiding interventional processes.
[105] This article showed the risk of occupationally exposed medical radiation for medical personnel that was related to workers handling radioisotopes.The results showed that the risk of thyroid cancer is higher in females than in males.[41]

TEE
This article addressed the potential risks of contracting infectious diseases while using transesophageal echocardiography (TEE), and delved into the necessity of thorough device cleaning protocols.[42] 2020 EchoNavigator This paper discussed EchoNavigator, which was applied to streamline the process of closing paravalvular leaks.This innovation holds the potential to accurately pinpoint the precise location, dimensions, depth, and morphology of the paravalvular leak, thus aiding in the accurate placement of the occluder device.
[107] 2020 TEE This article showed how TEE adds value in intraoperative settings in open, minimally invasive, or hybrid cardiac surgery procedures.[110] 2021 MR, nuclear imaging, and ultrasound This study discussed the use of cardiac magnetic resonance imaging to visualize the structure and function of the heart, as well as the issue of precise tissue characterization through a range of sequences, thereby offering [9] distinct perspectives on factors like fibrosis, infiltration, inflammation, edema, fat presence, strain, and other possible pathological attributes that can impact upcoming cardiovascular risks.

2021
3D intracardiac echocardiography, The 3D echocardiography rendering tool, and MDCT This review aimed to elucidate the present cutting-edge imaging methodologies applied in the context of the most prevalent percutaneous interventions alongside an exploration of emerging imaging tools.This article delved into using imaging strategies across pre-procedural planning, real-time procedural guidance, and subsequent follow-up assessments.Table A1.Cont.

CT
Van Praet introduced a classification system to anticipate the difficulty level in performing mini-thoracotomy aortic valve replacement (MT-AVR) by considering the positioning of the aorta within the thoracic cavity.This article assessed the significance of intricate computed tomography scan measurements in anticipating MT-AVR procedures' feasibility.[16] 2022 CMR This review delved into the physics underlying black-blood contrast and diverse strategies for achieving blood suppression.It also explained the technical complexities of black-blood contrast, how it is influenced by flow and motion conditions, its additional contrast weighting mechanisms (T1, T2, etc.), tissue magnetism, and spatial extent.[25] 2022 TTE and TEE This article discussed using TTE over TEE during COVID-19 for patients with nonvalvular atrial fibrillation.[43] 2022 TEE This study showed that intraoperative TEE improves the clinical outcomes of cardiac valve or proximal aortic surgeries. [90] 2023 Nuclear medicine and radiology This study showed the connection between penetrating ionizing radiation and mortality due to solid cancers within a combined cohort of nuclear workers in the USA.Regarding the relationship between ionizing radiation exposure and mortality from all solid cancers, an increased rate was observed between certain groups.[23] 2023 CT This review showed how CT contributes to the decision-making process for patients undergoing structural heart interventions.Also, the authors emphasized the need for multimodal image integration with CT. [32]

Figure 1 .
Figure 1.An illustration showing the critical role of imaging in SHD procedures.In the plot, CT, US, and Fluro are computed tomography, ultrasound (also known as echocardiography for cardiac applications), and fluoroscopy, respectively.

Figure 1 .
Figure 1.An illustration showing the critical role of imaging in SHD procedures.In the plot, CT, US, and Fluro are computed tomography, ultrasound (also known as echocardiography for cardiac applications), and fluoroscopy, respectively.

33 Figure 1 .
Figure 1.An illustration showing the critical role of imaging in SHD procedures.In the plot, CT, US, and Fluro are computed tomography, ultrasound (also known as echocardiography for cardiac applications), and fluoroscopy, respectively.

Figure 3 .
Figure 3.A schematic flowchart showing the steps in the multimodal image integration process.

Figure 4 .
Figure 4.An illustrative diagram showing the procedures related to traditional image registration methods.

Figure 3 .
Figure 3.A schematic flowchart showing the steps in the multimodal image integration process.

Figure 3 .
Figure 3.A schematic flowchart showing the steps in the multimodal image integration process.

Figure 4 .
Figure 4.An illustrative diagram showing the procedures related to traditional image registration methods.

Figure 4 .
Figure 4.An illustrative diagram showing the procedures related to traditional image registration methods.

Figure 5 .
Figure 5.A diagram showing the various multimodal medical image fusion techniques.Figure 5.A diagram showing the various multimodal medical image fusion techniques.

Figure 5 .
Figure 5.A diagram showing the various multimodal medical image fusion techniques.Figure 5.A diagram showing the various multimodal medical image fusion techniques.

Figure 8 .
Figure 8.A general framework of deep-learning-based image fusion based on CNN.Figure 8.A general framework of deep-learning-based image fusion based on CNN.

Figure 8 .
Figure 8.A general framework of deep-learning-based image fusion based on CNN.Figure 8.A general framework of deep-learning-based image fusion based on CNN.

Figure 9 .
Figure 9. 3D transesophageal echocardiography demonstrating a transseptal puncture from the left atrial perspective.

Figure 10 .
Figure 10.3D transesophageal echocardiography that demonstrates both sides of the left atrial appendage in profile: (a) an echocardiographic view and (b) a 3D reconstructed view.

Figure 9 .
Figure 9. 3D transesophageal echocardiography demonstrating a transseptal puncture from the left atrial perspective.

Figure 9 .
Figure 9. 3D transesophageal echocardiography demonstrating a transseptal puncture from the left atrial perspective.

Figure 10 .
Figure 10.3D transesophageal echocardiography that demonstrates both sides of the left atrial appendage in profile: (a) an echocardiographic view and (b) a 3D reconstructed view.

Figure 10 .
Figure 10.3D transesophageal echocardiography that demonstrates both sides of the left atrial appendage in profile: (a) an echocardiographic view and (b) a 3D reconstructed view.

Figure 11 .
Figure 11.(a) 3D transesophageal echocardiography that demonstrates the mouth of the left atrial appendage.(b) A deployed left atrial appendage closure device.

Figure 11 .
Figure 11.(a) 3D transesophageal echocardiography that demonstrates the mouth of the left atrial appendage.(b) A deployed left atrial appendage closure device.

33 Figure 12 .
Figure 12.The use of 3D transesophageal echocardiography (TEE) in demonstrating the mitral valve from the left atrial perspective: (left) TEE imaging planes of the mitral valve and (right) the 3D volume.

Figure 12 .
Figure 12.The use of 3D transesophageal echocardiography (TEE) in demonstrating the mitral valve from the left atrial perspective: (left) TEE imaging planes of the mitral valve and (right) the 3D volume.

Figure 12 .
Figure 12.The use of 3D transesophageal echocardiography (TEE) in demonstrating the mitral valve from the left atrial perspective: (left) TEE imaging planes of the mitral valve and (right) the 3D volume.

Figure 13 .
Figure 13.The use of 3D transesophageal echocardiography (TEE) in demonstrating a mitral valve replacement (MVR) from the left atrial perspective.The 2D TEE imaging planes of the MVR are shown to the left of the 3D volume.

Figure 13 .
Figure 13.The use of 3D transesophageal echocardiography (TEE) in demonstrating a mitral valve replacement (MVR) from the left atrial perspective.The 2D TEE imaging planes of the MVR are shown to the left of the 3D volume.

Figure 14 .
Figure 14.Variation in atrial septal defect morphology as illustrated by 3D TEE.Two catheters crossing the atrial septal defects-(a) one medium-sized and (b) one tiny defect.4.2.5.Paravalvular Leak Closure Paravalvular leak (PVL) is a known surgical and transcatheter valve replacement complication.Patients most commonly present with congestive heart failure and/or hemolysis, and thus historically require repeat surgery to correct the PVL with mixed success.Consequently, percutaneous PVL closure methods have been developed, and considerable em-

Figure 14 .
Figure 14.Variation in atrial septal defect morphology as illustrated by 3D TEE.Two catheters crossing the atrial septal defects-(a) one medium-sized and (b) one tiny defect.

4. 2 . 5 .
Paravalvular Leak Closure Paravalvular leak (PVL) is a known surgical and transcatheter valve replacement complication.Patients most commonly present with congestive heart failure and/or hemolysis, and thus historically require repeat surgery to correct the PVL with mixed success.Consequently, percutaneous PVL closure methods have been developed, and considerable emphasis has been placed on multimodal imaging for diagnosis and surgical guidance in cardiac catheterization laboratories.Several large series of percutaneous PVL closures have been published recently, with encouraging results regarding procedural success and clinical outcomes.

Figure 15 .
Figure 15.The 3D transesophageal echocardiography showing a paravalvular closure with a device: (a,b) two 3D reconstructed TEE images showing defects

Figure 15 .
Figure 15.The 3D transesophageal echocardiography showing a paravalvular defect and successful closure with a device: (a,b) two 3D reconstructed TEE images showing defects.

Figure 16 .Figure 17 .
Figure 16.The 2D transesophageal echocardiography in demonstrating the deployed transcatheter aortic valve replacement: (a-c) multi-view and the 3D reconstructed aortic valve of interest.

Figure 16 .Figure 16 .Figure 17 .
Figure 16.The 2D transesophageal echocardiography in demonstrating the deployed transcatheter aortic valve replacement: (a-c) multi-view and the 3D reconstructed aortic valve of interest.

Figure 17 .
Figure 17.The 3D transesophageal echocardiography in demonstrating the deployed transcatheter aortic valve replacement with endocarditis on follow-up: (a,b) the reconstructed aortic valve defects from the 3D TEE data from different viewing angles.
echo and X-ray images This review showed the advantages of echo and fluoroscopy registration during transseptal punctures.[95] coronary MRThis paper explored the contemporary clinical application and reasons for adopting non-contrast coronary magnetic resonance angiography.It focused on how these technologies can play a crucial role in facilitating the successful integration of coronary MRAs into routine clinical practice as a reliable imaging modality.[27] 2020 Phase-contrast MRI, MR elastography, MR angiography, and 4D flow imaging This article discussed phase-contrast MRI, which occupies a vital position in contemporary diagnostic imaging.Notably, it shows the applications in diagnosing heart conditions, assessing cerebrospinal fluid disorders, conducting angiography, and performing elastography.
the occurrence rate, various complication types, and the influencing factors related to esophageal or gastric injuries that result from transesophageal echocardiography (TEE) manipulation during structural cardiac interventions.A significant proportion of patients undergoing such interventions exhibited indications of injury that were linked to TEE usage.
This review aimed to delineate cutting-edge imaging methods, specifically in emphasizing echocardiography's role in guiding the most prevalent catheter-based interventions.It furnished insights, strategies, and recommendations that were tailored for interventional cardiologists to aid them in effectively navigating transseptal crossings, conducting left atrial appendage closures, executing transcatheter valve repairs or replacements, facilitating the percutaneous closure of patent foramen ovale and atrial defects, and in achieving the percutaneous closure of paravalvular leaks.[13]

Table A1 .
Cont.TEE) for guiding right heart catheterization (RHC) during atrial septal defect (ASD) closures and radiation exposure.The findings suggested that employing 3D TEE for RHC is viable without relying on fluoroscopy and can effectively lower radiation exposure in percutaneous ASD closures.

Table A1 .
Cont. described a user case study of the EchoNavigator system.Notably, the EchoNavigator was utilized to find the slit and wiring.This review summarized the role of cardiac CT and MRI in research and their use in clinical decision making.This review also touched on the guidelines relevant to current and future clinical practice.MR, PET, SPECT, and US This paper provided a comprehensive outline of multimodal medical image fusion techniques, including discussions of (1) contemporary fusion methodologies (particularly those rooted in deep learning), (2) the diverse imaging modalities applicable to medical image fusion, and (3) evaluation of the performance of medical image fusion, predominantly those based on datasets.