Semi-Nonlinear Deep Efficient Reconstruction for Unveiling Linear and Nonlinear Spatial Features of the Human Brain

Abstract

Deep learning has become indispensable for identifying hierarchical spatial features (SFs), which are crucial for linking neurological disorders to brain functionality, from functional Magnetic Resonance Imaging (fMRI). Unfortunately, existing methods are constrained by architectures that are either linear or nonlinear, limiting a comprehensive categorization of spatial features. To overcome this limitation, we introduce the Semi-Nonlinear Deep Efficient Reconstruction (SENDER) framework, a novel hybrid approach designed to simultaneously capture both linear and nonlinear spatial features, providing a holistic understanding of brain functionality. In our approach, linear SFs are formed by directly integrating multiple spatial features at shallow layers, whereas nonlinear SFs emerge from combining partial regions of these features, yielding complex patterns at deeper layers. We validated SENDER through extensive qualitative and quantitative evaluations with four state-of-the-art methods. Results demonstrate its superior performance, identifying five reproducible linear SFs and eight reproducible nonlinear SFs.

1. Introduction

Spatial features (SFs) extracted from functional Magnetic Resonance Imaging (fMRI) showcase a specific brain region affected by neurological disorders, which is more straightforward than analyzing sequential time series in fMRI [1,2,3]. During the COVID-19 pandemic, investigators have reported an impaired spatial feature in a patient’s brain with olfactory loss [4].

With rapid developments in machine learning, the hierarchy of SFs in the human brain has been explored through various deep linear learning techniques, including Deep Independent Component Analysis (DICA) [5], Sparse Deep Dictionary Learning (SDDL) [6], and Deep Non-negative Matrix Factorization (DNMF) [7,8]. These methods have provided valuable insights into brain functionality, yet they are built on linear frameworks, i.e., without activation functions connecting adjacent layers. Meanwhile, the advent of deep learning has propelled the identification of SFs, introducing a suite of deep nonlinear methods capable of uncovering more intricate hierarchical SFs. In particular, techniques such as the Deep Convolutional Autoencoder (DCAE), Restricted Boltzmann Machine (RBM), Deep Belief Network (DBN), and Convolutional Neural Network (CNN) [9,10,11,12,13,14,15,16] have been employed extensively in functional Magnetic Resonance Imaging (fMRI) research. Among these, the RBM has demonstrated exceptional precision in extracting and reconstructing hierarchical temporal and spatial features [13,17,18,19]. Recent studies have also highlighted the effectiveness of deep nonlinear methods like DCAE and DBN in analyzing task-based fMRI time series, revealing task-evoked spatial features and their hierarchical temporal organization [10,17,18,20]. These methods usually leverage activation functions, such as Sigmoid and Rectified Linear Unit (ReLU) [21], positioning them as essential tools for advancing our understanding of the human brain’s functional architecture [14].

In particular, from a computational neuroscience perspective, methodologies for uncovering the hierarchy of spatial features in the brain can be broadly categorized into two main approaches. On one hand, Deep Linear Methods, such as DNMF and SDDL, focus on identifying linear spatial features. These features are typically conceptualized as linear combinations of shallow features, as described by Smith in 2009 [22] and elaborated on by Wylie in 2021 [5]. On the other hand, Deep Nonlinear Methods, such as Deep Neural Networks (DNNs), are designed to investigate nonlinear spatial features. Unlike linear methods, these approaches model nonlinear combinations of shallow features, often integrating multiple regions in a nonlinear fashion [23]. By leveraging nonlinear activation functions, such as ReLU or Sigmoid, nonlinear methods provide deeper insights into more intricate and complex SFs [12,14]. However, these nonlinear features are often located in deeper layers, making them more difficult to interpret and explain [12,14]. Notably, despite the strengths of both approaches, current deep learning methods are typically constrained to being either linear or nonlinear. This limitation poses significant challenges for researchers aiming to concurrently categorize spatial features as both linear and nonlinear, leaving a gap in the comprehensive understanding of brain functionality.

To address the existing challenges, we introduce the Semi-Nonlinear Deep Efficient Reconstruction (SENDER), a novel hybrid deep learning framework that is specifically designed to concurrently uncover both linear and nonlinear spatial features in the brain. Overall, SENDER incorporates the strengths of deep linear and nonlinear methods, leveraging the efficiency of linear models that require smaller training samples while employing the advanced perception capabilities of nonlinear methods through activation functions and efficient non-fully connected architectures.

Specifically, the methodological contributions of SENDER are detailed as follows: (1). Automatic Hyperparameter Tuning. To implement automatic hyperparameter tuning, such as the number of components (i.e., the number of layers and units/neurons) in each layer [24,25], we have developed a rank reduction technique called the Rank Reduction Technique (RRT) for SENDER. RRT employs orthogonal decomposition, specifically QR matrix decomposition, to estimate the rank of input matrices, e.g., feature matrices, which corresponds to the number of units. This estimation is achieved through three techniques: weighted ratio (WR), weighted difference (WD), and weighted correlation (WC). These techniques systematically reduce the size of feature matrices through all layers until the estimated number of features equals one. (2). Accurate Approximation. SENDER achieves approximation accuracy comparable to that of traditional DNNs, demonstrating its capabilities to deliver precise and reliable results. (3). Efficient Hybrid Architecture. Unlike any existing deep linear or nonlinear method, we demonstrated that the non-fully connected architecture of SENDER not only enhances efficiency, e.g., makes it easy to optimize, but also allows for precise spatial feature extraction at an individual level. (4). Reduced Accumulative Error via Matrix Backpropagation (MBP). When a deep learning model is constructed using a wide array of layers, the accumulative error could potentially deteriorate the accuracy. Thus, we utilize the MBP technique [7,8,26] to further improve the accuracy.

In summary, leveraging the capabilities of SENDER, we identified five linear and eight nonlinear SFs from its second linear and nonlinear layers, respectively. The five linear SFs indicate significant linear integration of various shallow SFs, while the eight nonlinear SFs demonstrate the integration by combining multiple partial regions from these shallow SFs. In contrast, even advanced deep learning methods like DBN can only identify five nonlinear SFs, which predominantly exhibit linear integration of shallow SFs. More importantly, all these linear and nonlinear SFs derived via SENDER showcase strong reproducibility. As our understanding of neuroscience advances, nonlinear SFs are expected to deepen our knowledge of brain functionality.

2. Materials and Methods

This section provides an in-depth description of SENDER, covering several critical aspects of the framework. The following Figure 1 illustrates the computational framework of SENDER.

In particular, Figure 1 shows the step-by-step process of SENDER identifying linear and nonlinear SFs. Steps include (a) collecting 4D fMRI raw data and (b) converting all 4D fMRI raw data into 2D matrices. (c1) and (c2): The first nonlinear layer. (d) RRT implements hyperparameter tuning. (e1) and (e2): The second nonlinear layer extracts time series and nonlinear SFs, with a visual example provided. (f1) and (f2): The first linear layer. (g1) and (g2): The second linear layer reveals time series and linear SFs, with a visual example provided.

2.1. Multiband Multi-Echo fMRI Data and Preprocessing

This fMRI data collection was performed using a 3T scanner (Signa Premier, GE Healthcare, Waukesha, WI, USA) equipped with a body transmit coil and a 32-channel NOVA (Nova Medical, Wilmington, MA) receive head coil. Overall, 29 participants’ MB and MBME fMRI data were acquired, and 19 of them returned within two weeks for repeat imaging [27]. Importantly, various works demonstrated that MBME fMRI can improve the spatial resolution and functional sensitivity [27,28,29]. Therefore, the proposed computational framework can benefit from MBME fMRI to present more complex and even never-reported SFs.

Specifically, the maximum gradient strength was 70 mT/m, and the maximum slew rate was 170 mT/m/ms. Each subject underwent two resting-state fMRI (rsfMRI) acquisitions: an MB scan and an MBME scan. The MB scan had the following parameters: TR/TE = 650/30 ms, FOV = 24 cm, matrix size = 80 × 80 with slice thickness = 3 mm (3 × 3 × 3 mm voxel size), 11 slices with a multiband factor of 4 (44 total slices), FA = 60°, and partial Fourier factor = 0.85. The MBME scan had the following parameters: TR/TE = 900/11, 30, 49 ms, FOV = 24 cm, matrix size = 80 × 80 with slice thickness = 3 mm (3 × 3 × 3 mm voxel size), 11 slices with a multiband factor of 4 (44 total slices), FA = 60° and partial Fourier factor = 0.85. Both scans used an EPI readout with in-plane acceleration (R) = 2. The resting-state scans lasted six minutes each, resulting in 400 volumes for the MBME scans. During the resting-state scans, subjects were instructed to close their eyes but remain awake, refrain from any motion, and not think about anything in particular [27,28,29,30,31]. Notably, recent research [32] suggests that having eyes closed tends to correlate with greater integration, while having eyes open correlates with greater specialization. In this work, to advance the identification of linear and nonlinear SFs, an integration of multiple shallow SFs (i.e., SFs identified from the first layer), we recommended that all participants keep their eyes closed.

The MBME fMRI data preprocessing followed the steps in previous works [27,28,31,33,34]. The preprocessing pipelines generally included skull removal, motion correction, slice time correction, spatial smoothing, and BOLD signal filtering. Finally, a brain mask was applied to extract all fMRI signals. Importantly, due to the limitations of ICA demonstrated in previous work [35], we decided to employ the preprocessing pipeline proposed by the works of [33,34] to reduce the influence of ICA by separating SFs at deep layers into independent patterns, which results in missing specific SFs.

Moreover, Figure 2 illustrates a novel hybrid structure for constructing the computational framework of SENDER. In particular, subfigure (a) demonstrates a linear structure without activation functions. In contrast, subfigure (b) showcases nonlinear structure that leverages activation functions (denoted as $\mathcal{F}$) to introduce nonlinearity. This hybrid architecture allows SENDER to identify more complex SFs from fMRI.

2.2. Semi-Estimated Nonlinear Deep Efficient Reconstructor

As previously introduced, SENDER utilizes a hybrid architecture that integrates both linear and nonlinear learning, incorporating an efficient non-fully connected structure [36] along with nonlinear activation functions typically used in DNNs. This section presents the objective function that governs SENDER, outlining how it integrates linear and nonlinear architectures to improve its performance in extracting holistic SFs in the human brain. The specific formulation of the optimization function is presented below:

$$\begin{array}{c}{\displaystyle \underset{S_i\in {\mathbb{R}}^{m\times n}}{min}}{\displaystyle \bigcup _{i=1}^k}{∥S_i∥}_0\\ \mathit{s}.\mathit{t}.\forall k\in [2,M],({\displaystyle \prod _{i=1}^k}{X}_i)·Y_k+({\displaystyle \prod _{i=1}^k}{U}_i)·{\mathcal{N}}_k\left(V_k\right)+S_k=I\\ X_i{Y}_i\leftarrow \mathcal{R}·Y_{i-1}\\ U_i{V}_i\leftarrow \mathcal{R}·{\mathcal{N}}_{i-1}\left(V_{i-1}\right)\end{array}$$

(1)

In the architecture of SENDER shown in Equation (1), ${\left\{X_i\right\}}_{i=1}^k$ denotes the hierarchical weight matrices (i.e., mixing matrices) of the linear SENDER; specifically, $X_i$ denotes the weight matrix at the $ith$ layer, with k indicating the total layers number. Similarly, ${\left\{U_i\right\}}_{i=1}^k$ refers to the weight matrices for the nonlinear method at each corresponding layer. Additionally, ${\left\{Y_i\right\}}_{i=1}^k$ represents deeper spatial features derived via the linear method, where $Y_i$ specifically denotes the deep feature matrix at the $ith$ layer. Concurrently, ${\left\{V_i\right\}}_{i=1}^k$ defines deeper features uncovered through the nonlinear method at each layer. Moreover, ${\left\{S_i\right\}}_{i=1}^k$ represents a series of matrices indicating background noise, calculated using the $l_0$ norm [37] due to their sparsity characteristics. The nonlinear activation function at each layer is denoted by ${\mathcal{N}}_i$, such as activation functions like Sigmoid or Rectified Linear Unit (ReLU), which are critical for enhancing nonlinearity within the model [12,14]. For instance, suppose the total number of layers is M, and considering the original input data I, the decomposition of data within SENDER can be expressed as follows:

$$I=(\prod _{i=1}^M{X}_i)·Y_M+(\prod _{i=1}^M{U}_i)·{\mathcal{N}}_M\left(V_M\right)+S_M$$

As shown in Equation (1), which is similar to the objective function in previous works [6,8,38], our fundamental assumption is that the previously revealed spatial features, e.g., $Y_{i-1}$ or $V_{i-1}$, can be further decomposed as a linear product of a deeper weight matrix $X_i$ and spatial features $Y_i$, or as a nonlinear representation involving a deeper weight matrix $U_i$ and spatial features $V_i$. Additionally, the objective function governing SENDER encompasses more variables than traditional deep linear/nonlinear methods such as DICA, DNMF, and SDDL [5,6,7,8], reflecting its complex aim of extracting a broader spectrum of spatial feature nuances. Therefore, we determine advanced gradient-based optimizer $STOR{M}^{+}$ [39] to optimize SENDER. Importantly, we employ an optimizer based on proximal gradient [40] to solve the $l_0$ norm penalty. Each adjacent layer of linear and nonlinear architectures is connected via non-fully connected layers [36].

2.3. Rank Reduction Technique for Automatic Tuning Hyperparameters

To implement automatic hyperparameter tuning and reduce the high dimensionality of the original input dataset, we introduce a novel technique named the Rank Reduction Technique (RRT). This method is designed to calculate and reduce the rank of the current pattern matrices, such as ${\left\{Y_i\right\}}_{i=1}^k$ and ${\left\{V_i\right\}}_{i=1}^k$, until their rank is equivalent to one.

Specifically, inspired by previous works [41,42], RRT incorporates rank-revealing techniques using orthogonal decomposition through QR factorization (e.g., $[Q,R]\leftarrow QR\left(Input\right)$ and $R\subseteq {\mathbb{R}}^{n\times n}$ is a upper triangular matrix) to efficiently estimate and reduce the rank of pattern matrices. Due to the effectiveness of QR factorization in handling sparse and overcomplete matrices, RRT is satisfactory for decomposing these data structures, which are often encountered in deep learning applications for neuroimaging. The application of RRT thus facilitates the handling of complex datasets by simplifying their structure and reducing dimensionality without significant loss of information, which is essential for enhancing the performance of SENDER. The mathematical representation of RRT involves iteratively applying QR factorization to adjust the eigenvalues and eigenvectors of the pattern matrices, effectively minimizing their rank while retaining the essential components within the data. In fact, RRT can reduce the dimensionality of the input data matrix continuously by layers. For instance, RRT can estimate the number of components as 25 at the first layer. Subsequently, RRT continuously estimates the number of components as 6 at the second layer. Finally, RRT reduces the number of components to 1 at the third layer. Lastly, the estimation is completed since it is impossible to decompose a single vector. Thus, RRT can present the number of layers as 3 and the size of components (i.e., the number of artificial neurons) as 25, 6, and 1 across 3 layers, respectively. The RRT can be summarized in the following formula:

$$\mathcal{R}\left[\begin{array}{c}{a}_1\\ a_2\\ ⋮\\ a_{n-1}\\ a_n\end{array}\right]=\left[\begin{array}{c}{a}_1^{\left(1\right)}\\ a_2^{\left(1\right)}\\ ⋮\\ a_{n-1}^{\left(1\right)}\end{array}\right]{\mathcal{R}}^k\left[\begin{array}{c}{a}_1\\ a_2\\ ⋮\\ a_{n-1}\\ a_n\end{array}\right]=\left[\begin{array}{c}{a}_1^{\left(1\right)}\\ a_2^{\left(1\right)}\\ ⋮\\ a_{n-k-1}^{\left(1\right)}\\ a_{n-k}^{\left(1\right)}\end{array}\right]$$

(2)

where the operator $\mathcal{R}$ represents the RRT, and ${\left\{a_i\right\}}_{i=1}^n$ is a series of vectors, with each $a_i$ representing an individual vector. When RRT is applied repeatedly to the series of vectors ${\left\{a_i\right\}}_{i=1}^n$ over k iterations, the rank of the matrix formed by these vectors decreases: $\mathrm{rank}({\mathcal{R}}^k·[a_1,a_2,\cdots ,a_n])<\mathrm{rank}\left([a_1,a_2,\cdots ,a_n]\right)$. Moreover, if k is sufficiently large, there exists a number $N\in \mathbb{N}$ such that if $k>N$, the rank of the matrix formed by these vectors satisfies $\mathrm{rank}({\mathcal{R}}^k·[a_1,a_2,\cdots ,a_n])=1$. In this scenario, k corresponds to the total number of layers in the model, as the rank of the pattern matrix at the $kth$ layer becomes one after repeatedly applying RRT. In detail, at first, the diagonal of matrix R, derived from the pattern matrix using QR decomposition, is non-increasing in magnitude [26,38,43].

Furthermore, along the main diagonal of the matrix R, three techniques named WR, WD, and WC are applied to calculate the maximum rank shown in Equations (3)–(5); then, I is replaced by $I-S_k$, $k=1,2,\cdots ,M$, iteratively. It indicates that the rank-reducing technique can yield a reasonable solution using QR factorization [26,38,43]. The following formulas provide details of WR, WD, and WC.

Let $d\in {\mathbb{R}}^{1\times r}$ and $r\in {\mathbb{R}}^{r-1}$, then WR can be calculated by Equation (3) [38,43]:

$$\begin{array}{c}{d}_i\leftarrow \left|R_{ii}\right|\\ w{r}_i\leftarrow {\displaystyle \frac{d_i}{d_{i+1}}}\end{array}$$

(3)

where $R_{ii}$ represents a diagonal element of matrix $R\subseteq {\mathbb{R}}^{n\times n}$ derived by QR decomposition and $w{r}_i$ represents a single value of WR. The value of each WR is calculated by the ratio of the current element of the diagonal and the following element.

Similarly, WD is calculated as

$$\begin{array}{c}w{d}_i\leftarrow {\displaystyle \frac{\left|d_i-d_{i-1}\right|}{{\sum }_{k=1}^i{d}_k}}\end{array}$$

(4)

In Equation (4), WD [38,43] is defined as the absolute difference between the current diagonal element and the previous one divided by the cumulative sum of all the previous diagonal elements.

Equation (5), including $D_{i-1}$ and $D_{i-2}$ representing adjacent columns of matrix R, after QR decomposition, describes the proposed WC as follows:

$$\begin{array}{c}w{c}_i\leftarrow {\displaystyle \frac{\left|corr(D_{i-2},D_{i-1})-corr(D_{i-1},D_i)\right|}{{\sum }_{k=1}^{i+1}\parallel D_k\parallel }}\end{array}$$

(5)

Since WD, WR, and WC are the cumulative difference, ratio, and correlation of two adjacent components, respectively, the matrix dimension can be reduced by one at least after rank estimation. Thus, the RRT iteratively determines the maximum value position from WD, WR, and WC to conduct the estimated rank.

2.4. Matrix Backpropagation

Another important technical contribution introduced in this work is the implementation of matrix backpropagation (MBP) in SENDER to further reduce potential accumulative error after updating all variables in the model. On the one hand, performing matrix backpropagation on the linear architecture of SENDER focuses on updating feature matrices. First, we group the feature matrix at the last layer and calculate the product of weight and the feature matrix at other layers. Subsequently, we calculate the product of each weight matrix in all layers. Furthermore, we calculate the residual error using the difference between input and nonlinear architecture. Lastly, all positive values within the feature matrix will be updated via integrating the positive and negative values within residual error, the product of all weight matrices, and current feature matrix. For the linear structure in SENDER, the product of the weight matrices ${\prod }_{i=1}^k{X}_i$, the term $X_k{Y}_k$, and the residual at the current layer (e.g., the $kth$ layer) are utilized to perform backpropagation [8,26].

$$\begin{array}{c}{\widehat{Y}}_k\leftarrow Y_k,k=M\\ {\widehat{Y}}_k\leftarrow X_{k+1}{\widehat{Y}}_{k+1},1<k<M\end{array}$$

(6)

In addition, we denote the product of hierarchical weight matrices as $\psi$ in Equation (7) [7,8]:

$$\begin{array}{c}\psi \leftarrow {\displaystyle \prod _{i=1}^k}{X}_i,1<k<M\end{array}$$

(7)

Then, the following formula describes the crucial steps of MBP to update variables of ${\left\{X_i\right\}}_{i=1}^M$ and ${\left\{Y_i\right\}}_{i=1}^M$, representing the hierarchical weight and pattern matrices, respectively. In Equation (8), ${[·]}^{+}$ and ${[·]}^{-}$ represent all positive and negative elements, respectively [7,8].

$$\begin{array}{c}{Z}_k\leftarrow I-{\displaystyle \prod _{i=1}^k}{U}_i·{\mathcal{N}}_i\left(V_k\right)\\ {\widehat{Y}}_k^{+}\leftarrow Y_k^{+}\odot \sqrt{{\displaystyle \frac{{\left[{\psi }^T{Z}_k\right]}^{+}+{\left[{\psi }^T\psi \right]}^{-}{\widehat{Y}}_k^{+}}{{\left[{\psi }^T{Z}_k\right]}^{-}+{\left[{\psi }^T\psi \right]}^{+}{\widehat{Y}}_k^{+}}}}\end{array}$$

(8)

On the other hand, the derivatives of the activation functions ${\mathcal{N}}_k$, weight matrices $U_k$, and feature matrices $V_k$ are adopted to perform backpropagation on the nonlinear architecture of SENDER. In brief, the MBP is similar to the backpropagation utilized in Deep Neural Networks [8]. First, we denote the residual error as the difference between input and linear architecture. Subsequently, we denote an important variable representing the product of weight matrix, its transpose, and the maximum value. Furthermore, we define the derivative of activation function and intermedium variables to update the weight and feature matrices. Lastly, we introduce an iterative framework, using a constant to reduce potential accumulative error in both weight and feature matrices [8,44].

$$\begin{array}{c}K\leftarrow {({\displaystyle \prod _{i=1}^M}{U}_k)}^T·(I-{\displaystyle \prod _{i=1}^M}{X}_i·Y_M)\\ P_k^{it}\leftarrow {\left(U_k^{it}\right)}^T{U}_k^{it}·{\displaystyle \prod _{i=1}^M}max\left(U_k\right)\end{array}$$

(9)

In Equation (9), two important variables are involved: deep weight matrices and pattern matrices in the nonlinear method, where $it$ represents the current iteration. These variables are updated using the following backpropagation techniques, as shown in Equation (10) [8,44]:

$$\begin{array}{c}{c}_k^{it}\leftarrow max\left(U_k\right)·{\displaystyle \frac{d{\mathcal{N}}_k}{ds}},s=U_k^{it}{V}_k^{it}\\ C_k^{it}\leftarrow {\left(U_k^{it}\right)}^T(P_k^{it}·{\mathcal{N}}^{-1}\left(s\right)-K)\odot c_k^{it}\\ D_k^{it}\leftarrow {\left(U_{k-1}^{it}\right)}^T(U_{k-1}^{it}·{\mathcal{N}}^{-1}\left(s\right)-V_{k-1}^{it})\odot {\displaystyle \frac{d{\mathcal{N}}_k}{s}}·{\left(V_k^{it}\right)}^T\end{array}$$

(10)

Finally, the following equations show the process of updating the weight matrix $U_k$ and the pattern matrix $V_k$ in the $kth$ layer. In addition, $\tau$ is a constant value determined as 0.01 [8].

$$\begin{array}{c}{V}_k^{it+1}\leftarrow V_k^{it}-{\displaystyle \frac{\tau }{2^{it}}}\left(C_k^{it}\right)\\ U_k^{it+1}\leftarrow U_k^{it}-{\displaystyle \frac{\tau }{2^{it}}}\left(D_k^{it}\right)\end{array}$$

(11)

Notably, we also provide pseudocode of SENDER (Algorithm) and other vital techniques in Appendix B.

2.5. The Accurate Approximation of SENDER

In this section, we theoretically analyze the property of the proposed SENDER. Due to SENDER being organized as a composition of linear and nonlinear architectures, the following theorem demonstrates that SENDER can accurately approximate any real function that is finite almost everywhere [45]. The detailed proof of Theorem 1 can be viewed in Appendix A.

Definition 1. 

(SENDER Operator) Denote the initialization operator as $\mathcal{I}:{\mathbb{R}}^{m\times n}\to R^{m\times n}$, the sparse operator as $\mathcal{S}:{\mathbb{R}}^{m\times m}\to R^{m\times m}$, the rank reduction operator as $\mathcal{R}:{\mathbb{R}}^{m\times n}\to R^{P\times Q}$, and the STORM operator as $\mathcal{T}:{\mathbb{R}}^{m\times n}\to R^{m\times n}$. Their norms can be represented as ${\displaystyle \frac{1}{i}}=∥\mathcal{I}∥$, ${\displaystyle \frac{1}{s}}=∥\mathcal{s}∥$, ${\displaystyle \frac{1}{r}}=∥\mathcal{R}∥$, and ${\displaystyle \frac{1}{t}}=∥\mathcal{T}∥$. SENDER can be treated as a combination of various operators as $SENDER={\left(\mathcal{S}\mathcal{R}\mathcal{T}\right)}^k·\mathcal{I}:{\mathbb{R}}^{m\times n}\to {\mathbb{R}}^{m\times n}$.

Theorem 1. 

(Accurate Approximation of SENDER) Consider a real function $f:{\mathbb{R}}^{m\times n}\to {\mathbb{R}}^{m\times n}\bigcup \{\pm \infty \}$ and $m\left(\{I\in {\mathbb{R}}^{m\times n}:f\left(I\right)=\pm \infty \}\right)=0$ where $m(·)$ represents the Lebesgue measure [45]. Given $I\in {\mathbb{R}}^{m\times m}$, SENDER includes a linear method and a nonlinear method with multiple activation functions denoted as ${\left\{P_k\left(I\right)\right\}}_{k=1}^{N_1}$ and ${\left\{{\mathcal{N}}_k\left(I\right)\right\}}_{k=1}^{N_2}$. If $P_k$ denotes a series of linear functions and ${\widehat{\mathcal{N}}}_k$ denotes a series of nonlinear functions, then we have $\forall ϵ>0,\exists N>0,N_1,N_2>N,∥{P_k\left(I\right)}_{k=1}^{N_1}+{{\widehat{\mathcal{N}}}_k\left(I\right)}_{k=1}^{N_2}-f\left(I\right)∥\le ϵ$.

3. Results

To validate the effectiveness of the proposed SENDER, we utilized publicly available fMRI signals from all healthy individuals in the Multiband Multi-echo fMRI dataset [46]. By augmenting the original 29 test–retest subjects [47], we created a dataset of 100 augmented subjects to validate the proposed SENDER, and the size of an individual fMRI signal matrix is 400 × 902,629. To minimize the heterogeneous effects arising from parameter tuning, all hyperparameters of other peer methods were aligned with the estimation results of SENDER. Specifically, SENDER automatically estimated the number of layers as 3 across all subjects, with the number of SFs per layer being $25.00\pm 0.96$, $16.00\pm 0.52$, and $1.00\pm 0.00$ ($Mean\pm StandardDeviation$). Meanwhile, the parameters for other peer methods were tuned according to guidelines from established studies [5,6,8,25] to ensure fairness and consistency in performance evaluation. The activation function for all layers within both SENDER and the DBN was set to ReLU [14]. Importantly, the cross-validation using training and testing is not a suitable evaluation metric for unsupervised learning [12] like SENDER. Thus, in medical research, reproducibility should be one of the most critical gold standards [48] for evaluating the performance of unsupervised methods. Thus, validating the global reproducibility of SENDER and other peer methods across all test–retest subjects is more suitable.

To begin, we validate the robustness, efficiency, and accuracy of SENDER by comparing it with peer methods. The results are quantified in

Table 1. Estimations (mean ± standard deviation) of layer number across 100 subjects using SENDER and DBN SearchNet.

Layers	Linear SENDER	Nonlinear SENDER	ENAS	DBN SearchNet
1st Layer	$25.00\pm 0.98$	$26.00\pm 0.96$	$31.28\pm 1.94$	$33.26\pm 2.11$
2nd Layer	$6.28\pm 0.48$	$7.16\pm 0.52$	$10.64\pm 1.06$	$11.17\pm 2.16$
3rd layer	$1.00\pm 0.00$	$1.00\pm 0.00$	$3.67\pm 0.96$	$3.87\pm 0.84$

and

Table 2. Time consumption comparison in seconds of SENDER and other peer methods.

Methods	Time Consumption at 1st Layer	Time Consumption at 2nd Layer
DICA	$1451.58\pm 93.86$	DICA $1287.42\pm 78.15$
DNMF	$1901.65\pm 196.96$	DNMF $1579.98\pm 53.41$
SDDL	$2562.50\pm 185.31$	SDDL $2228.70\pm 195.09$
DBN	$4538.50\pm 305.49$	DBN $3441.60\pm 547.12$
SENDER	$1938.50\pm 590.49$	SENDER $1541.60\pm 57.12$

, and Figure 3, demonstrating the advancements of SENDER over peer approaches.

In addition, shallow SF templates identified at the shallow layers of computational models [10,19,49] in the first row in Figure 4 and Figure 5 illustrate the fundamental brain functionality, such as auditory (in the fourth column in Figure 4) and primary visualization (in the first column in Figure 4) [10,22]. These have been identified over the past two decades through conventional computational frameworks [22] and continue to play a key role in advancing our understanding of more complex brain functionalities and validating innovative methods for fMRI analysis [2,3,10,19,35,49].

Furthermore, due to the lack of a concrete ground truth for linear and nonlinear SF templates, we employ multiple data-driven pipelines [35,50,51,52] to generate reproducible group-wise linear and nonlinear SFs. In particular, on the one hand, we applied two of the most representative approaches, SDDL [6] and Deep ICA [5] on each real MBME fMRI signal to identify linear SFs, with each real MBME fMRI matrix 400 × 906,629 in size. Importantly, SENDER’s hyperparameters are determined automatically, while those for Deep ICA and Deep SDL were tuned based on prior studies [5,6]. After removing noisy SFs and artifacts [51], we calculated spatial similarity [35] across all SFs identified in the second layer using other peer approaches. We then performed clustering (e.g., k-means) to categorize SFs based on spatial similarity [35,50]. Notably, the number of clusters is not manually designed in this work. In fact, we conduct a series of simulated experiments which are similar to our previous work [35], to present potential linear and nonlinear SFs via SENDER. These simulated experiments pave the way to determine a reasonable number of clusters. Additionally, we create group-wise SFs as templates via Gaussian smoothing [52] based on individual SFs within each dominant cluster, covering at least 90% of subjects [50]. This process yielded five group-wise linear SFs. We employed SENDER and DBN using a similar process. This led to the generation of eight group-wise nonlinear SFs for SENDER and four for DBN, demonstrating SENDER’s superior ability to uncover more complex SFs. These group-wise linear and nonlinear SFs are capable of further mitigating demographic sensitivity and inherent variability in identified SFs.

Finally, in Figure 4, Figure 5, Figure 6 and Figure 7, the values (ranging from red to yellow), e.g., intensities, within each SF reflect the activation intensity of different brain regions, with higher values indicating stronger activation [3]. This information is essential for understanding neural activity levels using fMRI [53,54,55].

3.1. The Robustness, Efficiency, and Accuracy of SENDER

In this section, we evaluate the performance of SENDER using data from 100 augmented subjects, comprising 29 original subjects and 71 augmented subjects derived from MBME fMRI data. The 71 augmented subjects were generated by introducing random noise. The validation process is structured into three key perspectives: (1) Robustness: Assessing the ability of SENDER to reliably estimate vital hyperparameters across all subjects. (2) Efficiency: Comparing the time efficiency of SENDER against other peer methods across all subjects. (3) Accuracy: Measurement of the reconstruction error of SENDER compared to peer methods for all subjects. The details of the computational environment are as follows: Dell PowerEdge R740XD Server with Intel Xeon Gold 6246R (32 Cascade Lake cores) 3.4 G, 35.75 M Cache.

Given the inclusion of noise in the 71 augmented subjects, this evaluation leverages noisy data to ensure rigorous testing of SENDER’s capabilities.

To begin, we incorporated MBME fMRI data from the 100 augmented subjects to estimate the hyperparameters of the data matrix corresponding to each subject. Recognizing the growing prominence of evolutionary search techniques, we employed the Evolutionary Neural Architecture Search (ENAS) [56] and DBN SearchNet [11] frameworks to determine the critical hyperparameters of the DBN. The following Table 1 showcases the robustness comparison of SENDER, ENAS, and DBNSearchNet when estimating critical hyperparameter using 100 augmented subjects.

Furthermore, we evaluate the efficiency of the proposed SENDER in comparison with other peer methods, including DICA [5], DNMF [8], SDDL [6], and DBN [10]. Efficiency is assessed based on the time consumption required by each method to identify spatial features (SFs). Since most methods lack automatic hyperparameter tuning, we standardized the layer number to three and determined the number of components following prior studies [5,6,7,8,10]. Table 2 summarizes the time consumption comparisons. It presents the mean and standard deviation of the time required by each method to extract SFs from 100 augmented subjects. The results reveal that DICA is the fastest method for extracting SFs, while DBN is the slowest, requiring the most time even for a single subject. Notably, SENDER demonstrates faster performance compared to SDDL and shows no significant difference in time consumption compared to DNMF. Meanwhile, a t-test comparing the time consumption of DNMF and SENDER for extracting SFs across 100 augmented subjects yields a p-value of 0.1631 and 0.2146 at the first and second layers, indicating that the efficiency of SENDER is statistically comparable to that of DNMF. These findings highlight SENDER’s efficiency while maintaining robust performance.

Moreover, we validate the accuracy of the proposed SENDER with other peer methods. We utilize the reconstruction loss, which is shown in Equation (12), to represent the accuracy.

$$\begin{array}{c}ReconstructionLoss={\displaystyle \frac{∥\left({\prod }_{i=1}^k{X}_i\right)·\left(Y_k\right)-I∥}{∥I∥}}\end{array}$$

(12)

In Equation (12), ${\prod }_{i=1}^k{X}_i$ denotes weight or mixing matrices, and $Y_k$ represents the identified SFs (i.e., the feature matrix) at the $kth$ layer. I indicates the input matrix. Within this evaluation, we set the maximum iteration as 5000. For the tuning of other parameters, refer to previous works [5,6,8,10].

In particular, Figure 3a highlights that SENDER achieves the most accurate reconstruction loss among all methods, with most reconstruction losses clustering around 0.00001. To further illustrate the accuracy of SENDER, Figure 3b presents a convergence curve that shows the average reconstruction loss at each iteration. The results demonstrate that SENDER not only converges faster than other peer methods but also achieves more precise reconstruction losses. To substantiate the accuracy of SENDER, we conducted a t-test comparing the reconstruction loss across 100 augmented subjects between SENDER and DBN. The resulting p-value of $0.000761$ indicates a statistically significant difference, underscoring SENDER’s superior accuracy.

Figure 3. A quantitative illustration of the identifiability of identified shallow, linear, and nonlinear SFs via SENDER and other peer methods, as well as a comparison of reconstruction accuracy.

Figure 3. A quantitative illustration of the identifiability of identified shallow, linear, and nonlinear SFs via SENDER and other peer methods, as well as a comparison of reconstruction accuracy.

Overall, this section demonstrates that SENDER outperforms other peer methods, as evidenced by various statistical measurements and a large sample size. First, leveraging advanced matrix decomposition techniques, SENDER exhibits greater robustness compared to prevalent evolutionary search methods. Notably, SENDER consistently estimates the average number of artificial neurons (i.e., the number of components) as 1, with a standard deviation of 0, across 100 augmented subjects. Second, thanks to its advanced optimizer, SENDER achieves faster decomposition compared to peer methods, with time consumption comparable to DNMF [7,8]. Third, SENDER delivers the most accurate reconstruction loss, excelling in the extraction of both linear and nonlinear spatial features (SFs). Its reconstruction loss is significantly more precise than that of other methods, underscoring its superior accuracy. Finally, given that the fMRI signal matrices of 71 augmented subjects were generated by adding random noise, the results—particularly in hyperparameter estimation and accuracy—highlight the robustness of SENDER. This robustness can be attributed to the integration of the $l_0$ norm, which effectively enhances denoising capabilities.

3.2. Qualitative Comparison of Identified Shallow and Linear SFs via SENDER with Other Peer Methods

In this section, we validate the performance of SENDER to reveal SFs with other peer methods. Briefly, we validate whether SENDER can identify more precise SFs compared with ground-truth and more linear as well as nonlinear SFs with higher reproducibility.

Figure 4 and Figure 5 showcase a comparative identification of shallow SFs [22] using SENDER at the first linear and nonlinear layers, alongside other peer methods, including DICA [5], DNMF [7,8], SDDL [6], and DBN [25]. The results from the linear and nonlinear architectures of SENDER and the other four peer methods are measured against ground-truth SFs [22]. This experimental validation illustrates that the identification of shallow SFs using SENDER aligns closely with established ground-truth SFs [22], demonstrating no significant variance.

Figure 4. This figure illustrates three representative slices of six shallow SFs derived via SENDER and four other peer methods at the first linear and nonlinear layer.

Figure 4. This figure illustrates three representative slices of six shallow SFs derived via SENDER and four other peer methods at the first linear and nonlinear layer.

Figure 5. This figure illustrates three representative slices of another six shallow SFs derived via SENDER and four other peer methods at the first linear and nonlinear layer.

Figure 5. This figure illustrates three representative slices of another six shallow SFs derived via SENDER and four other peer methods at the first linear and nonlinear layer.

Importantly, the parameters for other peer methods were tuned according to guidelines from established studies [5,6,8,25] to ensure fairness and consistency in performance evaluation. The activation function for all layers within both SENDER and the DBN was set to ReLU. The following figure illustrates qualitative results of identified shallow spatial features via SENDER and other peer methods. Meanwhile, leveraging the previously introduced data-driven pipelines [35,50,51,52], we identify and present twelve group-wise shallow SFs that exhibit the highest spatial similarity with established ground-truth templates [22]. This consistency underscores the robustness of SENDER in detecting reproducible shallow SFs.

Notably, Figure 6 demonstrates that the linear SFs identified from SENDER’s second linear layer, like other methods, capture similar group-wise linear SFs at the second layer. Due to the lack of concrete ground truth for linear SFs, we sourced five representative linear ground-truth SFs from several key studies [5,57]. Overall, in Figure 3, linear SFs derived via SENDER at its second linear layer showcase a significant spatial similarity compared with ground-truth SFs in the first row. In particular, SENDER identifies larger spatial areas than DICA, with linear SFs aligning with SDDL but differing from DNMF; however, SDDL misses some SFs in the fourth column that SENDER can capture.

Figure 6. This figure presents three representative slices of linear SFs revealed via SENDER at the second linear layer and three other peer methods at the second layer.

Figure 6. This figure presents three representative slices of linear SFs revealed via SENDER at the second linear layer and three other peer methods at the second layer.

3.3. Qualitative Comparison of Identified Nonlinear SFs via SENDER and Other Peer Methods

Moreover, we present the qualitative results of nonlinear SFs extracted by SENDER and DBN, as illustrated in Figure 4. Since there is no concrete ground truth for nonlinear SFs, we employ multiple data-driven pipelines [50,51,52] to conduct a ground truth for these networks. Specifically, after removing noisy SFs and artifacts [51], we calculate the spatial similarity [35] for identified SFs at the second layer across all test and retest subjects. We then perform clustering (e.g., k-means) to categorize SFs based on spatial similarity [35]. Finally, we derive group-wise nonlinear SFs using Gaussian smoothing [52] on individual SFs within each dominant cluster, including at least 90% of all subjects [50].

In particular, as shown in Figure 7, SENDER demonstrates a superior capability in revealing eight nonlinear SFs compared to DBN, one of the most widely used nonlinear learning methods [25]. For instance, in the first row and the first and second columns of Figure 7, two nonlinear SFs derived by SENDER highlight left and right partial regions with an SF, combined with major visual and auditory networks [22]. In contrast, all SFs extracted by DBN exhibit approximate linear integration of various shallow SFs. Notably, other linear methods, such as DICA [5], DNMF [8], and SDDL [6], fail to consistently identify these nonlinear SFs, with a low reproducibility.

Figure 7. This figure provides 8 nonlinear SFs identified via SENDER at the second nonlinear layer and 4 SFs extracted by DBN at the second layer.

Figure 7. This figure provides 8 nonlinear SFs identified via SENDER at the second nonlinear layer and 4 SFs extracted by DBN at the second layer.

3.4. Quantitative Comparison of Identified Shallow, Linear, and Nonlinear SFs via SENDER and Four Other Peer Methods

As discussed before, cross-validation and overfitting evaluation are not suitable metrics for unsupervised learning [12,14] like SENDER; thus, in medical research, reproducibility should be one of the most critical gold standards [27,48,58,59,60] for evaluating the performance of unsupervised methods. To validate the global reproducibility of SENDER across all test–retest subjects, we employed identifiability, a measure ranging from 0 to 1, where higher values indicate greater reproducibility [48] across all subjects. Identifiability was calculated based on the similarity of test and retest SFs [48], in particular, after converting each 3D SF into a vector [35,50]. The reproducibility results can be viewed in Figure 5. In addition, we examine the consistency of the hyperparameter estimator on all subjects.

Specifically, in Figure 8a,b, we used identifiability [48] by calculating the spatial similarity [35] of test–retest SFs identified by SENDER and other methods, using ground-truth SFs [22]. For shallow SFs identified in Figure 8a, SENDER achieves an identifiability value of approximately 0.70. Meanwhile, the identifiability of most linear SFs identified by SENDER in Figure 8b exceeds 0.80, demonstrating substantial reproducibility [48]. Notably, the identifiability of nonlinear SFs in Figure 8c reaches nearly 0.90, which is higher than that of SFs extracted by DBN. Lastly, compared with other methods, SENDER achieves the most accurate reconstruction error, below $1\times {10}^{-4}$.

In comparison, while other peer methods can identify twelve shallow features and five comparable linear spatial features at their first and second layers, respectively, the reproducibility of these features is notably lower. Specifically, the majority of the shallow and linear spatial features reported by these methods exhibit identifiability values below 0.70. This suggests that the features identified by these traditional methods are less consistent and may be more prone to variations across different datasets. The higher identifiability of features detected by SENDER underscores its superior performance in capturing consistent and reproducible features in the data, which is crucial for both basic neuroscience research and clinical applications. High reproducibility is particularly important in clinical settings, where consistent identification of brain features can lead to more accurate diagnoses and in-depth understanding of neurological conditions.

Figure 8. A quantitative illustration of the identifiability of identified shallow, linear, and nonlinear SFs via SENDER and other peer methods as well as a comparison of reconstruction accuracy.

Figure 8. A quantitative illustration of the identifiability of identified shallow, linear, and nonlinear SFs via SENDER and other peer methods as well as a comparison of reconstruction accuracy.

4. Discussion

We introduced an innovative computational framework, SENDER, designed to concurrently reveal reproducible linear and nonlinear spatial features (SFs) from MBME fMRI data. SENDER bridges the gap between traditional linear methods [5,6,7,8] and newer nonlinear approaches [10,18,19].

Notably, a key strength of SENDER lies in its ability to consistently, efficiently, and accurately reveal hierarchical SFs without requiring extensive fMRI datasets or high-performance computing resources such as GPUs or TPUs. Unlike DCAE [19] and DBN [10], SENDER is more explainable, as demonstrated by theoretical predictions validated through comparisons with peer methods [5,6,8,10]. In particular, SENDER achieves faster convergence and more accurate reconstruction loss by leveraging the advanced STORM optimizer [61], which is particularly well suited for hybrid problems combining linear and nonlinear architectures [61,62]. Additionally, SENDER employs the Rank Reduction Technique (RRT) for data-driven hyperparameter estimation, a significant improvement over peer methods that rely on manual tuning [5,6,7,8]. The integration of the $l_0$ norm penalty enhances denoising capabilities, enabling SENDER to outperform conventional methods, such as hierarchical clustering, in identifying linear and nonlinear SFs.

In addition, SENDER produces reproducible SFs at deeper layers, outperforming peer methods in terms of robustness and precision [5,6,8,10]. For example, hierarchical clustering methods often fail to integrate SFs with low spatial similarity, whereas SENDER effectively consolidates such features into unified SFs through its advanced framework [10,11,35]. Importantly, our research categorizes hierarchical SFs in the human brain into five linear and eight nonlinear SFs. Notably, these SFs align with established neurophysiological representations for resting-state fMRI [22,33,35]. For instance, the Salience Network (SN) modulates anticorrelated connectivity between the Default Mode Network (DMN) and the Executive Control Network (ECN) [63]. Similarly, functional coupling between vision and auditory networks, as observed in our results, reflects known neurophysiological dynamics [64]. By evaluating linear SFs identified by peer methods, SENDER demonstrates superior accuracy, attributable to its integration of advanced $l_0$ sparsity and RRT with the STORM optimizer. Additionally, SENDER reveals nonlinear SFs that exhibit asymmetrical organization, further showcasing its capability to uncover complex brain dynamics.

Furthermore, SENDER produces relatively reproducible SFs at the deeper layers compared with other peer methods [5,6,8,10]. This can be attributed to DICA’s relatively fast convergence rate and independence constraint [35]. The mathematical evaluation framework and the fMRI validation procedure provided in this work should enable further development of deep matrix decomposition optimized for different types of real-world applications in biomedical imaging, with SENDER as the current prevalent computational framework for fMRI analytics. Importantly, we also provided theoretical analyses as Theorem 1 to explain why SENDER has more accurate reconstruction loss, which is comparable to deep neural networks.

Moreover, in this research, we not only validate the proposed computational framework named SENDER but also successfully categorize hierarchical SFs within the human brain into five linear and eight nonlinear SFs. Meanwhile, both the linear and nonlinear SFs also showcase the superiority of MBME fMRI, attributable to the former’s better specificity for an in-depth understanding of neural activities. In this initial exploratory work on linear and nonlinear SFs using MBME fMRI, multiple derived linear and nonlinear SFs are consistent with neurophysiological representations between shallow SFs from previous studies [22,33,35] on resting-state fMRI. For instance, the Salience Network, one of the most important SFs, is known to modulate the anticorrelated connectivity of other SFs, such as the Default Mode Network (DMN) and the Executive Control Network (ECN) [63], hence the linkage of their nodes into a single linear SF (in Figure 3, in the third column). The functional coupling of vision networks with an auditory network, shown in Figure 3 in the fourth column, is also well known, given the role that the abnormal auditory stimulus plays in visual attention [64]. Future neuroscientific studies will be required to empirically validate these linear SFs using demographic, clinical, cognitive, behavioral, and/or electrophysiological data. By evaluating most linear SFs identified among peer methods, we notice that SENDER can accurately reveal linear SFs using MBME fMRI, whereas the identifications of other peer methods are not as accurate as SENDER. These results could be attributed to its joint use of advanced $l_0$ sparsity and RRT in conjunction with the STORM optimizer.

Specifically, in Figure 7, we discovered more nonlinear SFs than DBN. Notably, some nonlinear SFs showcase an asymmetrical organization. For example, in the first row and first and second columns, identified SFs only include partial regions of a linear SF, shown in the first row and column in Figure 7. Since the identification of these linear and nonlinear SFs does not require extensive training datasets nor specialized computing infrastructure, they can be easily applied to clinical research with the potential to generate novel SFs to play an important role as a wide array of biomarkers for neurodevelopmental, neurodegenerative, and psychiatric disorders [65], including for diagnosis, prognosis, and treatment monitoring. This is particularly significant given the recent observation that neuropathology and psychopathology often affect shallow SFs differently than linear and nonlinear SFs at deeper layers. For example, various psychiatric disorders have been found to decrease shallow sensory SFs uniformly across patients [66,67] while increasing distinctiveness among patients in either linear or nonlinear SFs at the deeper layers [65]. In fMRI studies of mild traumatic brain injury (TBI), altered brain functionality has been found early after concussion within the patient’s brain, such as the SN, DMN, and ECN [68]. The linear interactions of SFs, such as those between the SN and the DMN, are thought to be especially important for outcomes after TBI and can be used to guide personalized treatment [69,70]. Disordered coupling of the SN with the DMN and ECN has also been shown in mild cognitive impairment [71].

Hence, prevalent neurological disorders such as head trauma and neurodegenerative diseases are thought to affect multiple levels of the functional organization in human brains. Such linear and nonlinear interactions across multiple SFs can be investigated with deeper layers of an advanced deep learning model like SENDER that integrate their spatially distinct gray matter nodes into a linear or nonlinear SF, as shown in Figure 3 (refer to the top row). These examples show how more principled data-driven characterization of this hierarchy, particularly at its higher levels, holds great promise for providing clinically actionable diagnostics of neurological and psychiatric diseases.

Lastly, despite its advantages, SENDER has room for improvement. Enhancements to RRT, integration with Reinforcement Learning, and optimization of its computational efficiency are needed. For example, the current decomposition of a single fMRI signal matrix requires approximately two hours, highlighting the need for faster optimization techniques. Furthermore, SENDER currently lacks insight into the neurological representations of SFs and their associations with specific disorders. Future research will apply SENDER to diverse fMRI datasets, including those for Alzheimer’s and Parkinson’s diseases, to identify disorder-specific linear and nonlinear SFs. Improved fMRI sensitivity and resolution, such as SLIDER-SMS fMRI [72], will enable SENDER to uncover even more complex SFs, advancing our understanding of brain functionality.

To summarize, the benefits of SENDER gain importance as the spatial and temporal resolution and sensitivity of fMRI continue to increase with improved MR imaging hardware and pulse sequences. Briefly, SENDER delivers several advantages in detecting the hierarchical and overlapping organization of SFs compared to previously described methods in a data-driven manner, such as DICA [5], DNMF [8], SDDL [6], and DBN [10]. Notably, SENDER does not have the constraints of spatial independence that DICA has [35]. Since SENDER can reveal extensively overlapped SFs [10,11,35] and estimates the vital hyperparameters automatically, it is easy to leverage the number and size of each layer, i.e., the number of artificial neurons. Other peer methods, such as DICA [5], SDDL [6], DNMF [8], and DBN [10], require manual hyperparameter tuning. Moreover, compared to Deep Neural Networks, SENDER has several advantages: (a) fewer training samples, i.e., the capability to reconstruct a single individual’s scan; (b) fewer extensive computational resources; (c) guaranteed convergence to a unique fixed point; and (d) automatic hyperparameter estimation. In this research, our results showcase the benefits of SENDER for MBME imaging [27,28,29]. Additionally, we aim to push further beyond SENDER, advancing our understanding of brain functionality through innovative computational frameworks that further deepen and expand our insights in a comprehensive understanding of brain functionality. Moreover, SENDER is capable of incorporating Electroencephalogram (EEG) to explore physiological representation and potential association of linear and nonlinear SFs with neurological disorders. Meanwhile, by leveraging advanced norm penalty, SENDER can enable the detection of abnormal answering of students, such as rapid guessing.

5. Conclusions

The SENDER framework represents a groundbreaking hybrid learning approach, seamlessly integrating linear and nonlinear architectures with a data-driven Rank Reduction Technique (RRT) for automatic hyperparameter tuning.

In particular, leveraging its hybrid and efficient non-fully connected architectures, SENDER excels at uncovering comprehensive spatial features, including both linear and nonlinear components, with superior identifiability compared to other state-of-the-art methods. The RRT facilitates automatic determination of all hyperparameters in a data-driven manner, streamlining the model configuration process. Furthermore, qualitative analyses underscore SENDER’s ability to extract a greater number of nonlinear features than peer methods. Simultaneously, quantitative evaluations reveal that SENDER identifies highly reproducible features with exceptional identifiability. Notably, the framework achieves precise approximations of input data and demonstrates a rapid convergence rate, often outperforming algorithms such as Deep Neural Networks (DNNs). Moreover, the capacity of SENDER to concurrently extract reproducible linear and nonlinear spatial features opens avenues for synergistic advancements in research on neurodevelopmental, neurodegenerative, and psychiatric disorders. These features, characterized by high reconstruction accuracy, hold promise as clinical biomarkers, enabling personalized diagnostics, prognostics, and treatment monitoring [73,74]. In summary, SENDER emerges as a transformative deep hybrid learning method, with the potential to make significant contributions to deep learning, computational neuroscience, and clinical translational research. By bridging these fields, SENDER paves the way for pioneering advancements in machine learning and neuroscience applications.