Multimodal Mutual Information Extraction and Source Detection with Application in Focal Seizure Localization

Abstract

Current multimodal imaging–based source localization (SoL) methods often rely on synchronously recorded data, and many neural network–driven approaches require large training datasets, conditions rarely met in clinical neuroimaging. To address these limitations, we introduce MieSoL (Multimodal Mutual Information Extraction and Source Localization), a unified framework that fuses EEG and MRI, whether acquired synchronously or asynchronously, to achieve robust cross-modal information extraction and high-accuracy SoL. Targeting neuroimaging applications, MieSoL combines Magnetic Resonance Imaging (MRI) and Electroencephalography (EEG), leveraging their complementary strengths—MRI’s high spatial resolution and EEG’s superior temporal resolution. MieSoL addresses key limitations of existing SoL methods, including poor localization accuracy and an unreliable estimation of the true source number. The framework combines two existing components—Unified Left Eigenvectors (ULeV) and Efficient High-Resolution sLORETA (EHR-sLORETA)—but integrates them in a novel way: ULeV is adapted to extract a noise-resistant shared latent representation across modalities, enabling cross-modal denoising and an improved estimation of the true source number (TSN), while EHR-sLORETA subsequently performs anatomically constrained high-resolution inverse mapping on the purified subspace. While EHR-sLORETA already demonstrates superior localization precision relative to sLORETA, replacing conventional PCA/ICA preprocessing with ULeV provides substantial advantages, particularly when data are scarce or asynchronously recorded. Unlike PCA/ICA approaches, which perform denoising and source selection separately and are limited in capturing shared information, ULeV jointly processes EEG and MRI to perform denoising, dimension reduction, and mutual-information-based feature extraction in a unified step. This coupling directly addresses longstanding challenges in multimodal SoL, including inconsistent noise levels, temporal misalignment, and the inefficiency of traditional PCA-based preprocessing. Consequently, on synthetic datasets, MieSoL achieves 40% improvement in Average Correlation Coefficient (ACC) and 56% reduction in Average Error Estimation (AEE) compared to conventional techniques. Clinical validation involving 26 epilepsy patients further demonstrates the method’s robustness, with automated results aligning closely with expert epileptologist assessments. Overall, MieSoL offers a principled and interpretable multimodal fusion paradigm that enhances the fidelity of EEG source localization, holding significant promise for both clinical and cognitive neuroscience applications.

1. Introduction

Neuroimaging is vital in advancing our understanding and treatment of brain disorders. Various imaging techniques each contribute unique insights into brain function. High-spatial-resolution imaging methods, such as Magnetic Resonance Imaging (MRI), Positron Emission Tomography (PET), and Computed Tomography (CT) scans, provide detailed anatomical views of the brain. These techniques are invaluable in identifying structural abnormalities or pathologies [1,2]. However, their utility is constrained by a low temporal resolution, making them less effective in capturing rapid neural dynamics. In contrast, high-temporal-resolution neuroimaging methods such as Electroencephalography (EEG), excel in monitoring real-time brain activity in a non-invasive and cost-efficient manner, offering high temporal resolution [3]. However, these methods have a significant limitation in their low spatial resolution, which restricts their ability to accurately pinpoint the exact locations of neural activity within the brain. Many neurological conditions, such as schizophrenia [4], Alzheimer’s disease [5], and epilepsy [6,7], necessitate an imaging approach that encompasses both high temporal and spatial resolution for accurate diagnosis and treatment. In these conditions, it is essential to first identify and distinguish the brain activities related to the neurological condition, such as an epileptic episode, from normal activities using high-temporal-resolution data [8,9]. Subsequently, determining the spatial location of these activities within the brain is crucial. In epilepsy, for example, an epileptologist must be able to differentiate characteristic seizure waves from normal brain activity. Interpreting EEG data visually to localize epileptic sources is time-consuming and demands substantial expertise. To enhance the quality of EEG data, several preprocessing methods have been proposed [10,11,12,13,14,15,16,17,18,19,20,21]. For spatial resolution, EEG’s low spatial resolution is addressed through EEG inverse problem solutions. These methods calculate the source location, orientation, and magnitude within the brain using electric potential measurements from the scalp and information about head geometry. This forward modeling process considers the conductivity properties of different brain layers and the head’s shape. The skull and other layers between the brain cortex and sensors reduce spatial resolution and complicate accurate source detection in noisy environments. The EEG measurements are spatially smoothed representations of neural activities due to the volume conduction properties of the skull [22]. Advanced data processing approaches aim to provide potential seizure zones, but these typically only identify larger areas such as the hemispheres or lobes, which lack precision [22,23,24,25,26]. Existing zoning algorithms struggle to precisely locate seizure sources, which is the focus of this paper. These methods often rely on either EEG or MRI data, limiting their ability to precisely locate seizure zones. Alternatively, invasive EEG approaches try to compensate for these disadvantages [27,28].

Existing multimodal fusion strategies fall broadly into early (input-level), intermediate (feature- or latent-level), and late (decision-level) approaches; however, all exhibit critical limitations when applied to epileptic source localization [29,30,31]. Early fusion methods, which concatenate raw or low-level features from EEG and MRI, assume matched noise characteristics and temporal compatibility between modalities and are therefore brittle in real clinical settings where EEG and MRI differ substantially in SNR and acquisition conditions. Intermediate fusion techniques—including recent EEG–MRI feature-fusion models—have shown promise for classification tasks such as epilepsy detection [32], but such methods do not guarantee anatomically precise or physiologically meaningful source localization. More clinically oriented multimodal pipelines that combine scalp or high-density EEG, intracranial EEG, structural MRI, PET/SPECT, or other modalities remain heavily dependent on expert manual integration; their diagnostic performance varies considerably across patients and lacks a standardized analytic fusion framework [33]. Even in concurrent EEG–fMRI studies, although functional network alterations can be captured, recent work shows that these approaches still fall short of delivering an automated, anatomically specific localization of the epileptogenic zone [34]. Consequently, despite the increased use of multimodal neuroimaging in epilepsy evaluation, a robust, noise-tolerant, and physiologically coherent multimodal fusion method tailored specifically for epileptogenic-source localization remains lacking. This gap motivates the proposed framework.

Current imaging methodologies, which focus on a single modality [35,36,37,38,39,40,41,42], fail to provide a comprehensive understanding of conditions requiring both spatial and temporal resolution. While machine learning approaches using neural networks (NNs) have been proposed for seizure patient classification and are successful in seizure pattern prediction, training an NN for this type of source localization is impractical due to the required amount of data for the purpose of convergence and precision [43,44,45,46,47,48].

Multimodal imaging and multimodal biomedical data processing have gained great attention for the purpose of more accurate diagnoses and decision making [49,50,51,52,53,54,55]. Many of these approaches either require a large amount of data and the use of machine learning algorithms with huge computational complexity, or are sensitive to the synchronicity of the multimodal data, i.e., they require the simultaneous recording of these multimodal data. However, there are practical scenarios in which the multimodal data, such as EEG and MRI, cannot be recorded simultaneously. In this case, the only option for source localization is to use the standardized low-resolution brain electromagnetic tomography (sLORETA). It has been shown that Efficient High-Resolution sLORETA (EHR-sLORETA) enhances spatial precision while preserving the mathematical guarantees and interpretability of sLORETA by efficiently reducing the noise effect in the MRI data [56]. On the other hand, the precision is also sensitive to the noise effects of the available EEG. Conventionally, the available methods of PCA and ICA can be used for the EEG denoising and source separation. Several methods of multimodal fusion for brain imaging utilize combinations of PCA and ICA [57]. These approaches usually impose strong statistical assumptions that may not align with the biological reality of the data and fail in precision when limited to a small number of subjects. Still, combining PCA and ICA in a traditional manner or in more recently developed multimodal forms is possible [49,58]. Note that the source separation methods require a method for the number of source estimation. We propose to use Mean Square Eigenvalue Error (MSEE), which has been shown to outperform other competitive methods that are mostly in the form of thresholding approaches [59]. In all these approaches, PCA and ICA are utilized separately and consecutively in the procedure. The goal of the proposed method is to utilize a method that can extract the mutual (common) information of the modalities in the denoising and source selection procedure. The notion of mutual information extraction strategy has been proposed for this form of joint or common feature information extraction [60]. Multimodal feature extraction is not for correlation detection but for the purpose of combining the modalities to extract the common information in the structure of the modals for the source separation, dimension reduction, and denoising of the modals. This work proposes a localization approach denoted Multimodal Information Extraction and Source Localization (MieSoL), which utilizes high-temporal data and high-spatial measures simultaneously. The method incorporates the optimum source number selection method MSEE and the optimum source localization method EHR-sLoreta. However, the use of conventional PCA/ICA denoising approaches is usually not effective enough and the remaining EEG noise can propagate into the source localization step. Therefore, the proposed method combines EEG and MRI to simultaneously denoise and extract the best basis. The method then employs source localization to accurately estimate the location and orientation of foci sources, which are then mapped onto the spatial data. The multimodal denoising step combines the basis extraction method, Unified Left Eigenvectors (ULeV) with the MSEE denoising approach [59,61]. This mutual information extraction, as opposed to the conventional PCA/ICA approaches, enables MieSoL to incorporate EHR-sLORETA to estimate the precise location and orientation of sources based on the denoised EEG dataset and the corresponding locations are mapped onto the MRI data [56]. Together, these elements constitute a unified and analytic multimodal fusion paradigm that (i) does not rely on large training datasets or deep models, (ii) is inherently robust to asynchronous and noise-mismatched modalities, and (iii) directly enhances the fidelity of EEG source localization by exploiting cross-modal evidence rather than treating modalities independently.

The foundation of important components of MieSoL for automatic hyperparameter selection is based on the information-theoretic approach for data-driven modeling inspired by innovative developments in statistical learning theory [62]. This method has shown impressive results in various applications such as LTI system modeling and time delay estimation [63], hyperparameter selection in clustering, ARMA modeling, and subspace identification [64,65,66]. MieSoL is compared with existing source localizing approaches for both synthetic and real datasets for the case of epilepsy using EEG as temporal data and MRI as spatial data. The results on the synthetic EEG and MRI data show great advantages of the proposed method in localizing sources in the sense of precision and robustness to the difference between EEG SNR and MRI SNR, which is a practical issue in many applications. The proposed automated method’s results for synthetic data are fully consistent with epileptologist-detected zones in 26 epileptic patients, highlighting its potential to assist in pre-surgical planning and in reducing surgical risk. It is important to mention that available EEG and MRI data are not recorded simultaneously, yet the proposed method is able to perform efficiently.

2. Multimodal Information Extraction and Source Localization (MieSoL)

The work considers utilizing two types of data, denoted as Modal 1 for high-temporal-resolution data and Modal 2 for high-spatial-resolution data, for the purpose of source localization. Examples of Modal 1 include EEG and EMG data, while examples of Modal 2 include structural MRI (sMRI) and CT scan data. High-temporal modalities, such as EEG, provide detailed information about the dynamic characteristics of neural activity, whereas high spatial modalities (e.g., sMRI) assist in precisely locating the source.

However, existing source localization methods often process each modality separately or fuse them at a later stage, which can lead to the loss of valuable cross-modal information and the introduction of noise artifacts. This limits the accuracy and robustness of source localization. Furthermore, the available temporal data is inherently noisy with an unknown signal-to-noise ratio (SNR), which poses additional challenges for accurate source estimation.

To address these issues, the proposed method consists of two main components. The first part is Multimodal Information Extraction (Mie), which integrates Unified Left Eigenvectors (ULeV) [61] and Mean Square Eigenvalue Error (MSEE) [59] to determine the optimal number of bases. This step effectively separates common foci source bases from noise and artifacts in the datasets. MSEE, an efficient SVD-based denoising approach rooted in information-theoretic principles [62], plays a critical role in this process. The novelty of MieSoL lies in its ability to utilize both available temporal and spatial data (EEG and MRI) to extract an optimal number of bases that best represent the underlying noiseless temporal data.

Figure 1 illustrates our proposed framework. Unlike existing methods, this approach denoises the two modalities simultaneously, enabling the extraction of the most informative and relevant features across domains. This integration enhances robustness to noise and improves the quality of the source signals.

The second component is Efficient Source Localization (ES), which applies our recently proposed Efficient High-Resolution sLORETA (EHR-sLORETA) [56] algorithm. This step uses the denoised data to accurately estimate the location of focal sources and map them onto the high-spatial-resolution data, achieving high precision in source localization while preserving the temporal dynamics.

2.1. Multimodal Information Extraction (Mie)

MieSoL involves two key parts: Basis Extraction and Number of Bases Selection. In the Basis Extraction step, Unified Left Eigenvectors (ULeV) processes the high-temporal datasets (such as EEG and MEG) to extract common bases that are significant for event-based source identification (such as epilepsy). Next, the Number of Bases Selection step, utilizing Mean Squared Eigenvalue Error (MSEE), estimates the optimal number of common bases. This estimation is crucial for the accurate representation of the underlying noiseless EEG data. Both steps are described below.

2.1.1. Basis Extraction: Unified Left Eigenvectors (ULeV)

ULeV is a novel basis extraction method that is able to extract the common bases among multimodal datasets [61]. This approach benefits from the theory of PCA and ICA to provide a novel method of best basis selection. However, it is able to combine these two theories in a totally new approach to simultaneously denoise and find the basis instead of implementing these two approaches in cascade. In a multimodal setting, ULeV allows the modals to have their own right eigenvectors, which preserves the stochastic properties of the modals, while the left eigenvectors are optimized over all the modals to extract the common basis between them. Here, ULeV uses the high-temporal data (Modal 1) along with the high-spatial data (Modal 2) to extract the common left bases related to the event-based sources. Letting each dataset have its own right eigenvectors while finding the common left eigenvectors has a two fold benefit. While the right eigenvectors in each of the datasets optimize for the best basis representation of the data itself, they preserve the statistical properties of the modal. On the other hand, the common left eigenvectors combine these two datasets and extract much of the information from them while simultaneously eliminating any redundancy.

The procedure is detailed as follows. Scalar values are denoted by lowercase non-bold letters, vectors by lowercase bold letters, and matrices by uppercase bold letters. Assume that there are J$(1\le j\le J)$ numbers of datasets for Modal 1 and Modal 2, each with different noise characteristics. The first j number of datasets, denoted by $j_1$$(1\le j_1\le j_b)$, are high-temporal datasets (Modal 1) and the rest are high-spatial datasets (Modal 2), denoted by $j_2$$(j_b<j_2\le J)$. Each dataset is represented by matrix ${\mathit{Y}}_j=\left[{\mathit{y}}_{j_1},{\mathit{y}}_{j_2},\dots ,{\mathit{y}}_{j_E}\right]\in R^{E\times N}$, where E is the number of observations and N is the length of the observations. The $i^{th}$ observation ($1\le i\le E$) in the $j^{th}$ dataset can be represented as follows:

$${\mathit{y}}_{ji}={\overline{\mathit{y}}}_{ji}+{\mathit{\omega }}_{ji}$$

(1)

where ${\mathit{\omega }}_{ji}$ is the additive noise and the noise-free data ${\overline{\mathit{y}}}_{ji}$ is linearly represented using M bases with a coefficient vector ${\overline{\mathit{w}}}_{ji}$:

$${\overline{\mathit{y}}}_{ji}={\overline{\mathit{w}}}_{ji}\mathit{B}$$

(2)

where $\mathit{B}$$\in R^{M\times N}$ is the unknown base dataset and ${\overline{\mathit{w}}}_{ji}\in R^{1\times M}$ denote coefficient vector. The focus of ULeV is on the extraction of a set of common bases, denoted by $\widehat{\mathit{B}}$ from the unknown base dataset by estimating the subspaces of the spanned vectors of the datasets. In contrast with conventional ICA, which typically assumes identical additive noise characteristics across datasets and thus struggles with multimodal data, ULeV distinguishes datasets based on their unique noise characteristics. This distinction allows ULeV to excel in extracting bases with significant influence across all datasets, as detailed in [61]. The left eigenvectors, $\widehat{\mathit{v}}$, are computed for the overall datasets, with the number of eigenvectors, m, determined after.

The procedure starts with initiating a set of right eigenvectors and optimizing for common left eigenvectors. The method iterates by changing the values of the initial right eigenvector and continuing this iterative algorithm to reach the optimum $common$ left eigenvectors as well as the resultant $adaptive$ right eigenvectors. To find the first common left eigenvector, ${\mathit{v}}_1$, the following optimization is solved:

$$\begin{array}{cccc}\hfill \underset{{\mathit{v}}_1,{\mathit{w}}_j}{max}& {\displaystyle \frac{1}{p}}\sum _{j=1}^J{\left|{\mathit{v}}_1^T{\mathit{Y}}_j{\mathit{w}}_j\right|}^p\mathrm{s}.\mathrm{t}.\hfill & \hfill {\mathit{v}}_1^T{\mathit{v}}_1=1& {\mathit{w}}_j^T{\mathit{w}}_j=1,1\le j\le J\hfill \end{array}$$

(3)

The power p, chosen to be a large number, for example, as large as 15 in [61], aids in discarding datasets whose spanned vectors weakly correlate with $\mathit{v}$. Starting with a random initial unmixing vector ${\mathit{w}}_j^{\left(0\right)}$, functioning as the adaptive right eigenvector, (3) solves for the initial value of the left eigenvalue ${\mathit{v}}_1^{\left(0\right)}$. In each kth iteration, the estimated left eigenvector is used to update the individual right eigenvectors:

$${\mathit{w}}_j^{(k+1)}={\displaystyle \frac{{{\mathit{v}}^{\left(k\right)}}^T{\mathit{Y}}_j}{{∥{{\mathit{v}}^{\left(k\right)}}^T{\mathit{Y}}_j∥}_2^2}}$$

(4)

The unmixing vectors and the left eigenvector are iteratively updated until two consecutive iteration values lie within an $ϵ$-distance of each other (e.g., $ϵ=0.001$) [61]. Once the desired ${\mathit{v}}_1$ is achieved, the part of the data that is generated by this left eigenvalue is subtracted from the data and (3) is repeated for the residuals to obtain ${\mathit{v}}_2$. The procedure is continued yielding the optimal common left eigenvectors $\mathit{V}=\left[{\mathit{v}}_1,{\mathit{v}}_2,\dots ,{\mathit{v}}_m\right]\in R^{m\times N}$. In the end, a conventional ICA is applied on the matrix $\mathit{V}$ to separate m numbers of bases in the form of $1\times N$ vector, ${\widehat{\mathit{B}}}_m=\left[{\widehat{\mathit{b}}}_1,{\widehat{\mathit{b}}}_2,\dots ,{\widehat{\mathit{b}}}_m\right]\in R^{m\times N}$ with the corresponding unmixing matrix ${\mathit{W}}_{jm}$:

$${\widehat{\mathit{B}}}_m={\mathit{W}}_{jm}^T{\mathit{Y}}_j$$

(5)

2.1.2. Number of Basis Selection: Mean Squared Eigenvalue Error (MSEE)

The Mean Squared Eigenvalue Error (MSEE) method is utilized to determine the optimal number of bases, m, for the ULeV algorithm, specifically for analyzing high-temporal data (Modal 1), such as epileptic EEG. The effectiveness of MSEE and its superiority over conventional basis selection methods are highlighted in [59,61]. In this context, each observation in the high-temporal data (Modal 1) is a combination of m bases superimposed with White Gaussian Noise (WGN) of zero mean and unknown variance, as outlined in (2). MSEE’s objective is to minimize the mean square error between the eigenvalues of an ideal, noiseless dataset (which is typically not directly accessible) and the eigenvalues derived from the available noisy dataset. This process involves computing the singular value decomposition (SVD) of the high-temporal dataset:

$${\mathit{Y}}_j=\mathit{U}{\Lambda }_{{\mathit{Y}}_j}{\mathit{U}}^{T}j=1,\dots ,j_b$$

(6)

Here, $\mathit{U}$ represents the eigenvector matrix, and ${\Lambda }_{{\mathit{Y}}_{\mathit{j}}}$ is the diagonal matrix of eigenvalues. The mean squared error between the eigenvalues of the hypothetical noise-free dataset and the estimated eigenvalues from the actual noisy data is quantified as follows:

$$z_m={\left|\left|{\Lambda }_{\overline{\mathit{Y}}j}-{\widehat{\Lambda }}_{{{\mathit{Y}}_j}^m}\right|\right|}_2^{2}j=1,\dots ,j_b$$

(7)

Here, ${\left|\left|.\right|\right|}_2^2$ represents norm-2 and $z_m$ is the mean squared error, ${\Lambda }_{\overline{Y}j}$ indicates the sorted eigenvalues of the theoretical noise-free dataset, and ${\widehat{\Lambda }}_{{{\mathit{Y}}_{\mathit{j}}}^{\mathit{m}}}$ refers to the first m sorted estimated eigenvalues from the noisy dataset. MSEE effectively identifies the number of bases that correspond to the hidden, underlying bases within the noisy dataset. A key strength of MSEE is its capability to automatically estimate the noise variance by utilizing the estimated eigenvalues and their statistics. It adopts a probabilistic approach to estimate the cost function ${\widehat{z}}_m$ for potential values of m (ranging from 1 to M) and selects the value that minimizes this cost:

$$m^{*}=\underset{1\le m\le M}{min}{\widehat{z}}_m$$

(8)

The optimal number of bases, $m^{*}$, ascertained through this process is then fed into the ULeV algorithm to extract a representative set of bases for the high-temporal data. The final extracted bases are denoted by ${\widehat{\mathit{B}}}_{{\mathit{m}}^{*}}$:

$${\widehat{\mathit{B}}}_{m^{*}}=\left[{\widehat{\mathit{b}}}_1,{\widehat{\mathit{b}}}_2,\dots ,{\widehat{\mathit{b}}}_{m^{*}}\right]$$

(9)

This process ensures that the basis selection for ULeV is optimal and specifically tailored to the characteristics of the event-based high-temporal data, such as epileptic EEG, enhancing the accuracy and reliability of the analysis. The importance of $z_m$ in (7) as a new form of loss function has recently been exercised and introduced in deep leaning medical image denoising [67].

2.2. Efficient Source Localization (ES)

Efficient Source Localization (ES) is a methodological approach encompassing both Forward Modeling and EHR-sLORETA techniques to accurately identify the sources of neural activity, particularly for events like epilepsy, using high-temporal and high-spatial data modalities.

2.2.1. Forward Model

The Forward Model in ES utilizes the optimized denoised common basis (9) from high-temporal data (Modal 1), such as event-based data, and high-spatial data (Modal 2) to generate a dataset tuned to the specific neural event of interest. The estimate of the desired noise-free data is

$${\widehat{\overline{\mathit{Y}}}}_j={\mathit{W}}_{j{m}^{*}}{\widehat{\mathit{B}}}_{m^{*}}j=1,\dots ,j_b$$

(10)

Here, ${\widehat{\overline{\mathit{Y}}}}_j\in {\mathbb{R}}^{E\times N}$ represents the event-tuned high-temporal data, which now contains clearer information about the source of the neural activity compared to the original dataset. This refined dataset is crucial for the subsequent source localization process.

2.2.2. Source Localization: Efficient High-Resolution sLORETA (EHR-sLORETA)

EHR-sLORETA is an advanced source localization method designed to pinpoint the location and magnitude of neural sources within the brain, particularly effective in the context of additive noise [56]. Its capabilities surpass those of traditional source localization methods in terms of accuracy and robustness, particularly regarding spatial dispersion and mean square error [56]. In applying EHR-sLORETA, a detailed mesh-grid model of the brain is constructed from the high-spatial data (Modal 2), typically derived from techniques like MRI. This model, composed of a large number of nodes, represents the neuron activity within the brain over time. Each node in this model correlates to a specific point of neural activity:

$${\widehat{\overline{\mathit{y}}}}_{jn}=\mathit{L}{\mathit{q}}_n+{\mathit{ϵ}}_{jn}n=1,\dots ,Nj=1,\dots ,j_b$$

(11)

In this equation, ${\widehat{\overline{\mathit{y}}}}_{jn}$ denotes the vector of the denoised observations at each sample time, ${\mathit{q}}_n\in {\mathbb{R}}^T$ represents the vector of neural activity at time n, $\mathit{L}\in {\mathbb{R}}^{E\times T}$ is the lead-field matrix that describes how neural sources inside the brain project onto external sensors (mapping source space to the measurement space-encoded Forward Model explained in the previous section and shown as unmixing matrix W). Lastly, ${\mathit{ϵ}}_{jn}$ is the additive WGN. It is important to note that the noise level in this denoised dataset is significantly reduced compared to the original dataset. EHR-sLORETA estimates the neural activity ${\mathit{q}}_n$ from the denoised observations ${\widehat{\overline{\mathit{y}}}}_{jn}$, assuming a linear relationship via a projection matrix $\mathit{H}$:

$${\widehat{\mathit{q}}}_n=\mathit{H}{\widehat{\overline{\mathit{y}}}}_{jn}n=1,\dots ,Nj=1,\dots ,j_b$$

(12)

Following this, EHR-sLORETA standardizes the estimated neural activities ${\widehat{\mathit{q}}}_n$ with respect to a normalization matrix ${\mathit{D}}_n^{tt}$ determined based on maximum and minimum magnates of the neuron activity at each time point. This standardized activity level serves as an indicator of each neuron’s involvement in the event generation, allowing for the differentiation between neurons actively engaged in the event (e.g., epileptic sources) and those that are not:

$${\widehat{\mathit{q}}}_{sLOR,n}^t={\displaystyle \frac{{\widehat{\mathit{q}}}_n^t}{{\mathit{D}}_n^{tt}}}n=1,\dots ,N$$

(13)

EHR-sLORETA then employs an automated thresholding model to select neurons with high activity levels, minimizing the mean squared error between the actual neural activity and the estimated activity. The optimal number of neural sources is determined, and their locations are marked on the high-spatial data (Modal 2), providing a precise map of the event sources within the brain.

$$z_t={\left|\left|{\mathit{q}}_n-{\widehat{\mathit{q}}}_{sLOR,n}^t\right|\right|}_2^2$$

(14)

$$t^{*}=arg\underset{t}{min}{z}_t$$

(15)

In summary, the ES approach, encompassing both Forward Modeling and EHR-sLORETA, provides a comprehensive and efficient framework for localizing the sources of neural events, utilizing the strengths of both high-temporal and high-spatial data modalities.

3. Simulation and Results

This section presents the evaluation of the Multimodal Information Extraction and Source Localization (MieSoL) method. The performance is assessed through synthetic datasets simulating both high-temporal and high-spatial resolution data and real-world patient data.

3.1. Multimodal Information Extraction Testing

In this subsection, we focus on assessing the denoising efficiency of the MieSoL method. Synthetic datasets representing high-temporal (similar to EEG data) and high-spatial (akin to MRI data) modalities were generated, each incorporating additive White Gaussian Noise at varying SNR levels. Three datasets ($J=3$) were generated as follows:

$${\mathit{Y}}_j={\mathit{W}}_j{\mathit{B}}_j+{\Omega }_j$$

(16)

where ${\mathit{Y}}_j$ is the generated dataset, ${\mathit{B}}_j$ is the basis matrix, ${\mathit{W}}_j$ is the mixing matrix and is generated randomly using independent identically distributed (IID) zero mean uniform distribution, and ${\Omega }_j\in R^{E\times N}$ is the additive White Gaussian Noise. It is important to note that the three dataset have been designed with different SNRs (5 dB, 10 dB, 15 dB). Assume that each dataset has three observations ($E=3$) with a sample length of 5000 ($N=5000$), as shown in Figure 2b. The mixing matrices are generated randomly, and each basis matrix contains four bases $(M=4)$.

In total, we have ten bases that are defined as follows:

$$\begin{array}{cc}\hfill & {\mathit{b}}_1=2cos\left(4\mathit{t}\right)+sin\left(3\mathit{t}\right)\hfill \\ \hfill & {\mathit{b}}_2=sin(3.1\mathit{t}+120)+0.25cos(1.9\mathit{t}-23)\hfill \\ \hfill & {\mathit{b}}_3=1.5cos\left(4.1\mathit{t}\right)sin\left(2.3\mathit{t}\right)\hfill \\ \hfill & {\mathit{b}}_4=5cos\left(3.2\mathit{t}\right)sin\left(5.1\mathit{t}\right)\hfill \\ \hfill & {\mathit{b}}_5=3sin\left(2.1\mathit{t}\right)+2.14cos\left(6.1\mathit{t}\right)\hfill \\ \hfill & {\mathit{b}}_6=sin\left(2.4\mathit{t}\right)+cos\left(3.3\mathit{t}\right)\hfill \\ \hfill & {\mathit{b}}_7=2cos\left(2.2\mathit{t}\right)+sin\left(2.9\mathit{t}\right)\hfill \\ \hfill & {\mathit{b}}_8=1.3cos\left(7.9\mathit{t}\right)+sin\left(6.3\mathit{t}\right)\hfill \\ \hfill & {\mathit{b}}_9=1.8cos\left(8.1\mathit{t}\right)sin\left(5.9\mathit{t}\right)\hfill \\ \hfill & {\mathit{b}}_{10}=0.5sin\left[\right(5\mathit{t}+25)sin\left(7\mathit{t}\right)]\hfill \end{array}$$

(17)

where $\mathit{t}\in R^{N\times 1}$ is the time vector. To evaluate the performance of MieSoL in terms of basis extraction, Average Correlation Coefficient (ACC) is selected as indicators. ACC evaluates the degree of correlation between the extracted bases and the true bases and measures the average degree to which the estimated common bases and the true bases are correlated. ACC illustrates a value between negative one and positive one, where an ACC value of positive one refers to a perfect positive correlation between two variables.

We consider three individual cases for basis matrices. In the first case, the datasets are generated from one common basis and three uncommon bases; in the second case, the datasets are generated from two common bases and two uncommon bases; and, in the third case, the datasets are generated from three common bases and one uncommon basis, as follows:

Case 1:

$$\begin{array}{cc}\hfill & {\mathit{B}}_1=\left[{\mathit{b}}_1,{\mathit{b}}_2,{\mathit{b}}_3,{\mathit{b}}_4\right]\hfill \\ \hfill & {\mathit{B}}_2=\left[{\mathit{b}}_1,{\mathit{b}}_5,{\mathit{b}}_6,{\mathit{b}}_7\right]\hfill \\ \hfill & {\mathit{B}}_3=\left[{\mathit{b}}_1,{\mathit{b}}_8,{\mathit{b}}_9,{\mathit{b}}_{10}\right]\hfill \end{array}$$

(18)

Case 2:

$$\begin{array}{c}\hfill {\mathit{B}}_1=\left[{\mathit{b}}_1,{\mathit{b}}_2,{\mathit{b}}_3,{\mathit{b}}_4\right]\\ \hfill {\mathit{B}}_2=\left[{\mathit{b}}_1,{\mathit{b}}_2,{\mathit{b}}_5,{\mathit{b}}_6\right]\\ \hfill {\mathit{B}}_3=\left[{\mathit{b}}_1,{\mathit{b}}_2,{\mathit{b}}_7,{\mathit{b}}_8\right]\end{array}$$

(19)

Case 3:

$$\begin{array}{c}\hfill {\mathit{B}}_1=\left[{\mathit{b}}_1,{\mathit{b}}_2,{\mathit{b}}_3,{\mathit{b}}_4\right]\\ \hfill {\mathit{B}}_2=\left[{\mathit{b}}_1,{\mathit{b}}_2,{\mathit{b}}_3,{\mathit{b}}_5\right]\\ \hfill {\mathit{B}}_3=\left[{\mathit{b}}_1,{\mathit{b}}_2,{\mathit{b}}_3,{\mathit{b}}_6\right]\end{array}$$

(20)

An example of the second case is shown in Figure 2. According to (19), there is a space of basis with eight bases: two common bases $(m^{*}=2)$ and six uncommon bases (Figure 2a), from which two are involved in each basis matrix ${\mathit{B}}_i$. Each dataset, shown in Figure 2b, is generated from two common bases and two uncommon bases.

The Average Correlation Coefficient (ACC) was employed as a metric to determine the denoising capability, with values closer to +1 indicating better performance. As is shown in

Table 1. ACC between the common bases and the estimated common bases as the number of common bases varies for Case 1, Case 2, and Case 3. (Averaged over 100 runs).

Number of	MSEE+	MSEE+	MSEE+
Common Bases	PCA+ICA	PCA+MICA	ULeV
1	0.65	0.74	0.96
2	0.78	0.89	0.97
3	0.83	0.91	0.98

, the performance of MieSoL is evaluated using ACC based on synthetic datasets, as the number of common bases varies. Table 1 shows the performance of three different methods of source separation, including the proposed method, in extracting the common bases for the generated three cases in (18), (19), and (20). The signal-to-noise ratios (SNRs) for the cases are 5 dB, 10 dB, and 15 dB, respectively, in this simulation, and each example uses three samples of the case for source separation. SNR measures the strength of the desired signal to the strength of the background noise. The higher SNR value indicates that the signal is less noisy, and, as the SNR value decreases, the noise power increases. MieSoL is compared with the well-known source separation method PCA/ICA, which combines PCA and ICA, as well as with a combination of PCA and Multiview ICA (MICA) [49], which is a more recent popular multimodal source separator. Note that both of these methods require the number of sources and, for that procedure, we used MSEE, which outperforms other existing numbers of source estimators, which are mostly thresholding approaches [59]. As the table shows, MieSoL outperforms the existing approaches by a noticeable difference and provides a very high accuracy, with ACCs close to one.

In the interesting second scenario, the considered data contain six samples: three are generated from Case 1 and three are generated from Case 2. To mimic the real life multimodal question, each case has its own SNR. The results in

Table 2. ACC between the common bases and the estimated common bases as the SNR distinction varies (first dataset has SNR of 5 dB).

SNR Difference	MSEE+	MSEE+	MSEE+
(dB)	PCA+ICA	PCA+MICA	ULeV
0	0.67	0.77	0.98
5	0.62	0.64	0.97
10	0.52	0.60	0.93
15	0.35	0.42	0.88

are shown as the SNR as the second case increases. In these simulations, the SNR of the first case is fixed to 20 dB and the SNR of the three samples generated from the second case varies. For example, the first row is for when the SNRs of both cases are 20 dB. In the second row, the SNR of the second case is reduced to 15 dB and therefore the SNR difference is 5 dB. The SNR of the second case is then reduced to 10 and 5 dB for the next rows. As the table shows, the existing methods degrade noticeably with the mismatch of SNRs, whereas the performance of MieSoL is extremely robust to the difference in statistical properties of the two cases and outperforms existing approaches by an even larger margin. Note that, in both Table 1 and Table 2, the standard deviation of the proposed method is in a range of 0.002 to 0.006 of its mean, while those of the existing methods are in a range of 0.0006 to 0.003 of their mean. While this value is lower for the existing methods, all methods have negligible standard deviation.

3.2. Synthetic EEG and MRI Data

To examine the performance of MieSoL, synthetic EEG and MRI datasets are generated using the BESA software (BESA^®|Brain Electrical Source Analysis—https://www.besa.de), version 7.0. This software has the ability to co-register EEG data with individual MRI data and to visualize dipole solutions developed by BESA Research in the individual anatomy. To generate the data, 10 sources for the EEG are used. Each EEG datapoint is generated with a resolution of 12 bits and sampling frequency 256 Hz, and the MRI is generated with 1.5l Tesla. The SNR of EEG is 5 dB, and the SNR of MRI is 12 dB. For the generated 12 cases, the locations of EEG sources are changed randomly.

To generate the required matrices in the form of ${\mathit{Y}}_j$ in Section 2.2.1, that is, the input of ULeV, the multimodal data must be preprocessed. Both EEG and MRI data are first normalized to have zero means and unit variances so that all modalities operate on a comparable numerical scale. Because ULeV requires all inputs to share the same matrix size, the EEG matrix and the MRI matrices are reshaped to a common fixed dimension, facilitating multimodal integration. After this size alignment, the matrices are concatenated to form a single unified data matrix, which is then provided as input to the ULeV module. This procedure ensures a consistent input structure across the modalities.

Average Error Estimation (AEE), which is commonly used for the comparison of source localizers, is shown in

Table 3. Estimated number of sources and Average Error Estimation (AEE) between the location of 10 true sources and the estimated sources. Numbers are shown in form of pairs as (Estimated number of sources, AEE).

#		MSEE	MSEE	MSEE	MieSoL
	EHR-sLORETA	+PCA+ICA	+PCA+MICA	ULeV
		+EHR-sLORETA	+EHR-sLORETA	+sLORETA
1	(7, 6.32)	(7, 5.30)	(10, 3.34)	(9, 2.08)	(10, 1.22)
2	(7, 7.68)	(7, 5.88)	(11, 4.29)	(9, 2.42)	(10, 1.44)
3	(4, 9.42)	(3, 8.62)	(13, 6.12)	(11, 3.65)	(10, 2.23)
4	(9, 7.12)	(7, 5.52)	(10, 4.03)	(9, 2.19)	(10, 1.21)
5	(8, 8.24)	(6, 6.07)	(11, 4.54)	(9, 2.72	(10, 1.80)
6	(6, 10.14)	(3, 8.14)	(12, 5.67)	(11, 3.34)	(10, 2.26)
7	(8, 7.02)	(7, 5.16)	(10, 3.61)	(10, 1.89)	(10, 1.01)
8	(7, 11.23)	(5, 7.97)	(12, 5.44)	(11, 3.55)	(10, 2.10)
9	(8, 7.78)	(7, 5.26)	(13, 6.62)	(10, 1.99)	(10, 1.22)
10	(8, 8.09)	(7, 5.93)	(11, 4.26)	(9, 2.55)	(10, 1.25)
11	(7, 7.23)	(6, 6.06)	(11, 4.48)	(9, 2.67)	(10, 1.22)
12	(5, 10.76)	(3, 8.45)	(10, 5.93)	(11, 3.28)	(10, 2.03)

for a comparison of methods. AEE measures the average estimated error between the estimated location of the sources and the true location of the sources. A low AEE value indicates that the coordination of the estimated location is close to the coordination of the true location, and a high AEE value indicates a remarkable difference between the estimated location and the desired location increases. The compared methods are source locator Efficient sLORETA (EHR-sLORETA), EHR-sLORETA with PCA/ICA preprocessing, or PCA/MICA Preprocessing for source separation and denoising procedure. Also, the proposed method is compared with common source extractor MieSoL with conventional sLORETA as well.

3.3. Real-World Patient Data

In this experiment, the performance of MieSoL is validated based on the real-world patients datasets, and the result of this experiment is verified by physicians. As we explain in Section 4, the results of this experiment are close to the medical diagnosis. EEG datasets and brain MRI scans of 26 patients are collected at the Epilepsy Clinic, Isfahan, Iran. Note that, due to the severity of the disease in these patients, they are candidates for epilepsy surgery according to the opinion of a specialist. EEG signals are recorded for 15 min through 19 electrodes at a sampling rate of 250 Hz with 12 bits of resolution, and MRI datasets are recorded with a resolution of 1.5 Tesla.

The MRI dataset of patients, with size $1024\times 1024$, are used to construct the patient’s brain mesh model with 15,000 vertices $(T=\mathrm{15,000})$, and the electrode placement of the patient’s EEG dataset is included in the brain model as well. In this experiment, MieSoL uses the EEG and MRI datasets of the subject to localize the epileptic sources in the subject’s brain, as shown in Figure 3, Figure 4 and Figure 5. In Figure 3, the green circle sign demonstrates the estimated location of the epileptic source (Figure 3a–c), and the topographic maps of these corresponding sources are represented as well (Figure 3d–f). In Figure 4, the red circles illustrate the estimated location of the epileptic sources, and the red condensed region represents the estimated location of the epileptic source in Figure 5. While the evaluation and performance results of the MieSoL for the synthetic datasets is shown in Table 3, a similar evaluation and the impact of MieSoL, compared with the existing methods, by measuring the AEE between the estimated location and the true location of the sources is represented in

Table 4. Estimated number of sources and Average Error Estimation (AEE) between the location of the true sources and the estimated sources (Estimated number of sources, AEE).

#	MSEE	MSEE	MSEE	MieSoL
	+PCA+ICA	+PCA+MICA	ULeV
	+EHR-sLORETA	+EHR-sLORETA	+sLORETA
1	(1, 5.65)	(3, 4.02)	(3, 2.23)	(2, 1.45)
2	(2, 6.61)	(3, 4.53)	(2, 2.75)	(2, 1.82)
3	(2, 8.74)	(4, 6.45)	(2, 3.98)	(1, 2.49)
4	(3, 5.85)	(3, 4.26)	(4, 2.42)	(2, 1.70)
5	(1, 6.40)	(3, 4.87)	(2, 3.05)	(1, 1.09)
6	(1, 8.47)	(4, 6.33)	(2, 3.63)	(2, 2.68)
7	(2, 5.35)	(2, 3.94)	(3, 2.21)	(3, 1.27)
8	(4, 8.30)	(4, 5.68)	(3, 3.50)	(3, 2.33)
9	(1, 5.59)	(3, 4.48)	(4, 2.32)	(3, 1.45)
10	(4, 6.26)	(4, 4.59)	(4, 2.92)	(3, 1.98)
11	(2, 6.39)	(4, 4.81)	(2, 3.03)	(2, 2.05)
12	(3, 8.78)	(3, 6.24)	(2, 3.61)	(1, 2.97)
13	(3, 5.57)	(5, 3.95)	(4, 2.27)	(2, 1.42)
14	(1, 6.15)	(3, 4.51)	(3, 2.70)	(1, 1.83)
15	(1, 8.43)	(2, 5.89)	(3, 3.24)	(1, 2.52)
16	(1, 6.19)	(3, 4.61)	(3, 2.82)	(1, 1.89)
17	(2, 8.44)	(2, 5.58)	(4, 3.04)	(2, 1.85)
18	(1, 8.15)	(2, 5.62)	(4, 3.39)	(3, 2.37)
19	(4, 5.49)	(2, 3.81)	(4, 2.16)	(3, 1.41)
20	(2, 7.86)	(3, 5.38)	(4, 3.10)	(2, 2.03)
21	(3, 5.82)	(4, 3.65)	(3, 1.93)	(3, 1.08)
22	(3, 5.36)	(3, 3.88)	(3, 2.01)	(3, 1.27)
23	(2, 5.61)	(4, 4.12)	(2, 2.43)	(3, 1.35)
24	(1, 6.11)	(5, 4.67)	(3, 2.99)	(2, 1.72)
25	(3, 7.56)	(5, 5.47)	(4, 3.20)	(2, 1.98)
26	(1, 8.56)	(3, 5.84)	(4, 3.23)	(3, 1.82)

. The locations of the true sources are obtained based on the medical diagnosis of expert physicians.

4. Discussion

An automated approach was proposed to localize the epileptic sources in the brain by using both the EEG signals and MRI scans of the epileptic patients. The performance of MieSoL was evaluated by using both synthesized and real-world datasets. The experimental results validated that MieSoL was able to localize epileptic sources in the brain with a high accuracy. The recorded neural activities were denoised to remove the contributed noise and motion artifacts, and to extract the mutual information between modalities (EEG and MRI). The multimodal seizure-tuned EEG signals were investigated to detect the excessive electrical impulses and estimate the location of the seizure discharges.

In this study, MieSoL was examined to remove the noisy part of the EEG signals without loosing the critical information and to extract the mutual information between EEG and MRI. This method showed a remarkable performance in terms of denoising compared to the other denoisers, such as MSEE+PCA+IC, and MSEE+PCA+MICA, as shown in Table 2. Moreover, in terms of extracting the mutual information, MieSoL performed significantly more accurately, compared to other methods, as shown in Table 1. The inverse problem, EHR-sLORETA, with a constructed head model with 15,000 vertices $(T=\mathrm{15,000})$ based on the MRI dataset, was used to localize the epileptic sources.

The performance of MieSoL was evaluated by computing the ACC between the estimated and desired common bases, as are shown in Table 1. According to the fact that, in real applications, different modalities have different SNR values, MieSoL was evaluated for varying SNR difference between the two datasets that are shown in Table 2. As is shown in the table, MieSoL performed amazingly robustly in the case of having a low number of common bases, as well as the case of having a huge SNR gap between two datasets, while the existing methods degrade drastically with the SNR difference between the datasets. As is illustrated in Table 1, three different methods, including MSEE+PCA+ICA, MSEE+PCA+MICA, and MieSoL, were used to extract the common bases among the synthetic datasets. As observed, the ACC value and the number of common bases were directly related. However, the decrease of ACC value for MieSoL was small compared to the two other methods. In addition, ACC values of MieSoL were higher than the intended methods for different numbers of common bases. According to the above observations, MieSoL was able to perform as a remarkable basis extractor by successfully extracting the common bases, which were highly correlated to the true bases from multiple datasets.

The same three methods that have been used in Table 1 are used in Table 2 as well. As observed in Table 2, the ACC between the extracted common bases and the true common bases decreased as the SNR difference between two datasets increased. However, the ACC value changed slightly for MieSoL compared to the two other methods. This indicated that MieSoL demonstrated a robust performance in the case of having a huge SNR difference between two datasets. In compliance with the obtained information from Table 1 and Table 2, MieSoL could be used as an exceptional multimodal simultaneous denoiser and best basis selection to estimate the noiseless dataset and to extract the mutual information between modalities. In the next part of the study, the performance of MieSoL was tested using the synthesized EEG and MRI datasets as the location and orientation of sources varied. As is shown in Table 3, five different methods, EHR-sLORETA, MSEE+PCA+ICA+EHR-sLORETA, MSEE+PCA+MICA+EHR-sLORETA, ULeV+SLORETA, and MieSoL, were used to localize the sources. The performance of these methods was evaluated by measuring AEE between the estimated location and the true location of the sources. According to Table 3, MieSoL exhibited a remarkable performance compared to the existing methods in the sense of the number of source identifications and AEE. This was accomplished by observing that MieSoL had the lowest AEE score for different sources. In addition, the minimal AEE values of MieSoL presented convincing evidence that this method was able to estimate the location of the sources with high confidence to the desired location. It is worth mentioning that the four other methods indicated inadequate performance when the sources were located in the underlying layers of the brain, such as case numbers 3, 6, and 12. As the table shows, another important advantage of MieSoL over the existing methods was in its reliability in finding the true number of sources. As was shown while for changes to the number of sources, MSEE+PCA+ICA+EHR-sLORETA and MSEE+PCA+MICA+EHR-sLORETA over- or underestimated the number of true sources, while our proposed method consistently chose the correct number of true sources in all of the scenarios. To this end, the performance of MieSoL was evaluated using the synthetic datasets.

Next, the performance of the proposed methods was validated using the EEG and MRI of 26 epileptic patients. MieSoL was used to localize the epileptic sources in an epileptic patient based on the EEG signals and MRI scans of the patient. As is shown in Figure 3a, MieSoL was able to detect a focal seizure in the left parietal lobe of Subject 1’s brain. Based on the corresponding topographic map (Figure 3d), the seizure originated in the inferior parietal lobe and oriented toward the right central. For Subject 2, a focal seizure was discovered in the superior parietal lobe of the brain (Figure 3b). For this subject, the seizure started in the superior parietal lobe and extended to the superior occipital lobe of the brain. For Subject 3, MieSoL localized a focal seizure in the right frontal lobe of the brain (Figure 3c). According to the corresponding topographic map (Figure 3f), the seizure was generated in the right frontal lobe and spread toward the right parietal lobe. Moreover, MieSoL detected two sources of seizure in the right central and the right frontal of the subject’s brain, which are indicated by red circles (Figure 4). By analyzing the density of the epileptic sources obtained through MieSoL, it was illustrated that the density of the seizure source in the right central was higher than the density of the seizure source in the right frontal; therefore, we could conclude that the seizure originated from the right central of the brain and went all the way to right frontal of the brain. In Figure 5, a similar situation was observed for another subject, where MieSoL localized the origin of the seizure source in the left lateral and followed its expansion to the frontal of the brain, which is indicated by a red condensed region. In addition, the beneath power graph represents the power of the source. The obtained findings for all of the subjects (including the above observations) were numerically compared with the outcomes of the intended methods by calculating AEE value (Table 4). As the table shows, the AEE value of the proposed method was minimal compared to the other approaches. It is interesting to note not only that the proposed method outperforms the other methods in the sense of AEE, but also that the provided minimum AEE is based on a lower number of sources (location) for the seizure. In other words, the proposed method detects a lower number of sources for the seizure, which is expected to be more realistic. Note that in all the above applications, the value of p in ULeV in (3) is adopted to be 15, similar to what is suggested in [61]. For all the cases, varying the value of p for a range between 1 and 20 confirmed that a value of 15 shows a good performance, with values growing up to 20 showing a negligible improvement.

We discussed our results with physicians and, in conclusion, our results were outstandingly close to the medical diagnosis. The detected seizure locations were consistently in the zones detected by the physicians, and the detected behavior of the seizure was considered to be informative. As a result, MieSoL was able to automatically localize the epileptic sources as the seizure location and size varied in the brain with a high accuracy. In addition, by analyzing the density of the estimated epileptic sources, it was shown that MieSoL was able to estimate the orientation of the epileptic sources as well. According to the findings, MieSoL was a reliable technique to estimate the location and orientation of the epileptic sources with a high accuracy. The MieSoL results are due to the efficient extraction of mutual information of both the EEG and the MRI datasets of an epileptic patient that could effectively localize the epileptic sources in the patient’s brain.

5. Conclusions

This paper proposed an automated epileptic localization method based on processing and analyzing the epileptic EEG signals and the epileptic brain MRI scans combined. Common seizure bases were extracted from the epileptic EEG and MRI datasets using ULeV, a novel and newly proposed multimodal basis extraction method. The results for both synthetic and real data were promising. The approach has a higher precision in locating the seizure sources compared to the state-of-the-art source localizing methods.

The proposed method has three main components. The first step performs dimension reduction, the second step concentrates on source separation and the selection of the number of sources, and the last one performs the source localization. The first and second components outperform (in the sense of ACC) the existing approaches. Also, due to the multimodality signal processing and extraction of the joint information of EEG and MRI, the source localization approach notably outperforms the available methods in the sense of AEE. The MieSoL has shown full accuracy in detecting the correct number of sources for the synthetic data. It is worth mentioning that, even though the method optimizes the desired AEE over the existing approaches, the average number of sources chosen by the proposed approach for the seizure is less than those proposed by the existing methods. In addition the method demonstrated robust performance in terms of reliability and accuracy for both the synthetic and real data, even when the multimodal data is not recorded simultaneously. MieSoL shows great potential for future applications in the diagnosis of other diseases by extracting and utilizing the mutual information of any combination of available signals such as EEG, MRI, ECG, and fMRI. By supporting both limited-data scenarios and asynchronous acquisitions, MieSoL is expected to provide an automated EEG–MRI multimodal analysis solution for cases where traditional approaches often fail.