1. Introduction
Spatial features (SFs) extracted from functional Magnetic Resonance Imaging (fMRI) showcase a specific brain region affected by neurological disorders, which is more straightforward than analyzing sequential time series in fMRI [
1,
2,
3]. During the COVID-19 pandemic, investigators have reported an impaired spatial feature in a patient’s brain with olfactory loss [
4].
With rapid developments in machine learning, the hierarchy of SFs in the human brain has been explored through various deep linear learning techniques, including Deep Independent Component Analysis (DICA) [
5], Sparse Deep Dictionary Learning (SDDL) [
6], and Deep Non-negative Matrix Factorization (DNMF) [
7,
8]. These methods have provided valuable insights into brain functionality, yet they are built on linear frameworks, i.e., without activation functions connecting adjacent layers. Meanwhile, the advent of deep learning has propelled the identification of SFs, introducing a suite of deep nonlinear methods capable of uncovering more intricate hierarchical SFs. In particular, techniques such as the Deep Convolutional Autoencoder (DCAE), Restricted Boltzmann Machine (RBM), Deep Belief Network (DBN), and Convolutional Neural Network (CNN) [
9,
10,
11,
12,
13,
14,
15,
16] have been employed extensively in functional Magnetic Resonance Imaging (fMRI) research. Among these, the RBM has demonstrated exceptional precision in extracting and reconstructing hierarchical temporal and spatial features [
13,
17,
18,
19]. Recent studies have also highlighted the effectiveness of deep nonlinear methods like DCAE and DBN in analyzing task-based fMRI time series, revealing task-evoked spatial features and their hierarchical temporal organization [
10,
17,
18,
20]. These methods usually leverage activation functions, such as Sigmoid and Rectified Linear Unit (ReLU) [
21], positioning them as essential tools for advancing our understanding of the human brain’s functional architecture [
14].
In particular, from a computational neuroscience perspective, methodologies for uncovering the hierarchy of spatial features in the brain can be broadly categorized into two main approaches. On one hand, Deep Linear Methods, such as DNMF and SDDL, focus on identifying linear spatial features. These features are typically conceptualized as linear combinations of shallow features, as described by Smith in 2009 [
22] and elaborated on by Wylie in 2021 [
5]. On the other hand, Deep Nonlinear Methods, such as Deep Neural Networks (DNNs), are designed to investigate nonlinear spatial features. Unlike linear methods, these approaches model nonlinear combinations of shallow features, often integrating multiple regions in a nonlinear fashion [
23]. By leveraging nonlinear activation functions, such as ReLU or Sigmoid, nonlinear methods provide deeper insights into more intricate and complex SFs [
12,
14]. However, these nonlinear features are often located in deeper layers, making them more difficult to interpret and explain [
12,
14]. Notably, despite the strengths of both approaches, current deep learning methods are typically constrained to being either linear or nonlinear. This limitation poses significant challenges for researchers aiming to concurrently categorize spatial features as both linear and nonlinear, leaving a gap in the comprehensive understanding of brain functionality.
To address the existing challenges, we introduce the Semi-Nonlinear Deep Efficient Reconstruction (SENDER), a novel hybrid deep learning framework that is specifically designed to concurrently uncover both linear and nonlinear spatial features in the brain. Overall, SENDER incorporates the strengths of deep linear and nonlinear methods, leveraging the efficiency of linear models that require smaller training samples while employing the advanced perception capabilities of nonlinear methods through activation functions and efficient non-fully connected architectures.
Specifically, the methodological contributions of
SENDER are detailed as follows: (1). Automatic Hyperparameter Tuning. To implement automatic hyperparameter tuning, such as the number of components (i.e., the number of layers and units/neurons) in each layer [
24,
25], we have developed a rank reduction technique called the Rank Reduction Technique (RRT) for
SENDER. RRT employs orthogonal decomposition, specifically QR matrix decomposition, to estimate the rank of input matrices, e.g., feature matrices, which corresponds to the number of units. This estimation is achieved through three techniques: weighted ratio (WR), weighted difference (WD), and weighted correlation (WC). These techniques systematically reduce the size of feature matrices through all layers until the estimated number of features equals one. (2). Accurate Approximation.
SENDER achieves approximation accuracy comparable to that of traditional DNNs, demonstrating its capabilities to deliver precise and reliable results. (3). Efficient Hybrid Architecture. Unlike any existing deep linear or nonlinear method, we demonstrated that the non-fully connected architecture of
SENDER not only enhances efficiency, e.g., makes it easy to optimize, but also allows for precise spatial feature extraction at an individual level. (4). Reduced Accumulative Error via Matrix Backpropagation (MBP). When a deep learning model is constructed using a wide array of layers, the accumulative error could potentially deteriorate the accuracy. Thus, we utilize the MBP technique [
7,
8,
26] to further improve the accuracy.
In summary, leveraging the capabilities of SENDER, we identified five linear and eight nonlinear SFs from its second linear and nonlinear layers, respectively. The five linear SFs indicate significant linear integration of various shallow SFs, while the eight nonlinear SFs demonstrate the integration by combining multiple partial regions from these shallow SFs. In contrast, even advanced deep learning methods like DBN can only identify five nonlinear SFs, which predominantly exhibit linear integration of shallow SFs. More importantly, all these linear and nonlinear SFs derived via SENDER showcase strong reproducibility. As our understanding of neuroscience advances, nonlinear SFs are expected to deepen our knowledge of brain functionality.
3. Results
To validate the effectiveness of the proposed
SENDER, we utilized publicly available fMRI signals from all healthy individuals in the Multiband Multi-echo fMRI dataset [
46]. By augmenting the original 29 test–retest subjects [
47], we created a dataset of 100 augmented subjects to validate the proposed
SENDER, and the size of an individual fMRI signal matrix is 400 × 902,629. To minimize the heterogeneous effects arising from parameter tuning, all hyperparameters of other peer methods were aligned with the estimation results of
SENDER. Specifically,
SENDER automatically estimated the number of layers as 3 across all subjects, with the number of SFs per layer being
,
, and
(
). Meanwhile, the parameters for other peer methods were tuned according to guidelines from established studies [
5,
6,
8,
25] to ensure fairness and consistency in performance evaluation. The activation function for all layers within both
SENDER and the DBN was set to ReLU [
14]. Importantly, the cross-validation using training and testing is not a suitable evaluation metric for unsupervised learning [
12] like
SENDER. Thus, in medical research, reproducibility should be one of the most critical gold standards [
48] for evaluating the performance of unsupervised methods. Thus, validating the global reproducibility of
SENDER and other peer methods across all test–retest subjects is more suitable.
To begin, we validate the robustness, efficiency, and accuracy of
SENDER by comparing it with peer methods. The results are quantified in
Table 1 and
Table 2, and
Figure 3, demonstrating the advancements of
SENDER over peer approaches.
In addition, shallow SF templates identified at the shallow layers of computational models [
10,
19,
49] in the first row in
Figure 4 and
Figure 5 illustrate the fundamental brain functionality, such as auditory (in the fourth column in
Figure 4) and primary visualization (in the first column in
Figure 4) [
10,
22]. These have been identified over the past two decades through conventional computational frameworks [
22] and continue to play a key role in advancing our understanding of more complex brain functionalities and validating innovative methods for fMRI analysis [
2,
3,
10,
19,
35,
49].
Furthermore, due to the lack of a concrete ground truth for linear and nonlinear SF templates, we employ multiple data-driven pipelines [
35,
50,
51,
52] to generate reproducible group-wise linear and nonlinear SFs. In particular, on the one hand, we applied two of the most representative approaches, SDDL [
6] and Deep ICA [
5] on each real MBME fMRI signal to identify linear SFs, with each real MBME fMRI matrix 400 × 906,629 in size. Importantly,
SENDER’s hyperparameters are determined automatically, while those for Deep ICA and Deep SDL were tuned based on prior studies [
5,
6]. After removing noisy SFs and artifacts [
51], we calculated spatial similarity [
35] across all SFs identified in the second layer using other peer approaches. We then performed clustering (e.g., k-means) to categorize SFs based on spatial similarity [
35,
50]. Notably, the number of clusters is not manually designed in this work. In fact, we conduct a series of simulated experiments which are similar to our previous work [
35], to present potential linear and nonlinear SFs via
SENDER. These simulated experiments pave the way to determine a reasonable number of clusters. Additionally, we create group-wise SFs as templates via Gaussian smoothing [
52] based on individual SFs within each dominant cluster, covering at least 90% of subjects [
50]. This process yielded five group-wise linear SFs. We employed
SENDER and DBN using a similar process. This led to the generation of eight group-wise nonlinear SFs for
SENDER and four for DBN, demonstrating
SENDER’s superior ability to uncover more complex SFs. These group-wise linear and nonlinear SFs are capable of further mitigating demographic sensitivity and inherent variability in identified SFs.
Finally, in
Figure 4,
Figure 5,
Figure 6 and
Figure 7, the values (ranging from red to yellow), e.g., intensities, within each SF reflect the activation intensity of different brain regions, with higher values indicating stronger activation [
3]. This information is essential for understanding neural activity levels using fMRI [
53,
54,
55].
3.1. The Robustness, Efficiency, and Accuracy of SENDER
In this section, we evaluate the performance of SENDER using data from 100 augmented subjects, comprising 29 original subjects and 71 augmented subjects derived from MBME fMRI data. The 71 augmented subjects were generated by introducing random noise. The validation process is structured into three key perspectives: (1) Robustness: Assessing the ability of SENDER to reliably estimate vital hyperparameters across all subjects. (2) Efficiency: Comparing the time efficiency of SENDER against other peer methods across all subjects. (3) Accuracy: Measurement of the reconstruction error of SENDER compared to peer methods for all subjects. The details of the computational environment are as follows: Dell PowerEdge R740XD Server with Intel Xeon Gold 6246R (32 Cascade Lake cores) 3.4 G, 35.75 M Cache.
Given the inclusion of noise in the 71 augmented subjects, this evaluation leverages noisy data to ensure rigorous testing of SENDER’s capabilities.
To begin, we incorporated MBME fMRI data from the 100 augmented subjects to estimate the hyperparameters of the data matrix corresponding to each subject. Recognizing the growing prominence of evolutionary search techniques, we employed the Evolutionary Neural Architecture Search (ENAS) [
56] and DBN SearchNet [
11] frameworks to determine the critical hyperparameters of the DBN. The following
Table 1 showcases the robustness comparison of
SENDER, ENAS, and DBNSearchNet when estimating critical hyperparameter using 100 augmented subjects.
Furthermore, we evaluate the efficiency of the proposed
SENDER in comparison with other peer methods, including DICA [
5], DNMF [
8], SDDL [
6], and DBN [
10]. Efficiency is assessed based on the time consumption required by each method to identify spatial features (SFs). Since most methods lack automatic hyperparameter tuning, we standardized the layer number to three and determined the number of components following prior studies [
5,
6,
7,
8,
10].
Table 2 summarizes the time consumption comparisons. It presents the mean and standard deviation of the time required by each method to extract SFs from 100 augmented subjects. The results reveal that DICA is the fastest method for extracting SFs, while DBN is the slowest, requiring the most time even for a single subject. Notably,
SENDER demonstrates faster performance compared to SDDL and shows no significant difference in time consumption compared to DNMF. Meanwhile, a
t-test comparing the time consumption of DNMF and
SENDER for extracting SFs across 100 augmented subjects yields a
p-value of 0.1631 and 0.2146 at the first and second layers, indicating that the efficiency of
SENDER is statistically comparable to that of DNMF. These findings highlight
SENDER’s efficiency while maintaining robust performance.
Moreover, we validate the accuracy of the proposed
SENDER with other peer methods. We utilize the reconstruction loss, which is shown in Equation (
12), to represent the accuracy.
In Equation (
12),
denotes weight or mixing matrices, and
represents the identified SFs (i.e., the feature matrix) at the
layer.
I indicates the input matrix. Within this evaluation, we set the maximum iteration as 5000. For the tuning of other parameters, refer to previous works [
5,
6,
8,
10].
In particular,
Figure 3a highlights that
SENDER achieves the most accurate reconstruction loss among all methods, with most reconstruction losses clustering around 0.00001. To further illustrate the accuracy of
SENDER,
Figure 3b presents a convergence curve that shows the average reconstruction loss at each iteration. The results demonstrate that
SENDER not only converges faster than other peer methods but also achieves more precise reconstruction losses. To substantiate the accuracy of
SENDER, we conducted a
t-test comparing the reconstruction loss across 100 augmented subjects between
SENDER and DBN. The resulting
p-value of
indicates a statistically significant difference, underscoring
SENDER’s superior accuracy.
Figure 3.
A quantitative illustration of the identifiability of identified shallow, linear, and nonlinear SFs via SENDER and other peer methods, as well as a comparison of reconstruction accuracy.
Figure 3.
A quantitative illustration of the identifiability of identified shallow, linear, and nonlinear SFs via SENDER and other peer methods, as well as a comparison of reconstruction accuracy.
Overall, this section demonstrates that
SENDER outperforms other peer methods, as evidenced by various statistical measurements and a large sample size. First, leveraging advanced matrix decomposition techniques,
SENDER exhibits greater robustness compared to prevalent evolutionary search methods. Notably,
SENDER consistently estimates the average number of artificial neurons (i.e., the number of components) as 1, with a standard deviation of 0, across 100 augmented subjects. Second, thanks to its advanced optimizer,
SENDER achieves faster decomposition compared to peer methods, with time consumption comparable to DNMF [
7,
8]. Third,
SENDER delivers the most accurate reconstruction loss, excelling in the extraction of both linear and nonlinear spatial features (SFs). Its reconstruction loss is significantly more precise than that of other methods, underscoring its superior accuracy. Finally, given that the fMRI signal matrices of 71 augmented subjects were generated by adding random noise, the results—particularly in hyperparameter estimation and accuracy—highlight the robustness of
SENDER. This robustness can be attributed to the integration of the
norm, which effectively enhances denoising capabilities.
3.2. Qualitative Comparison of Identified Shallow and Linear SFs via SENDER with Other Peer Methods
In this section, we validate the performance of SENDER to reveal SFs with other peer methods. Briefly, we validate whether SENDER can identify more precise SFs compared with ground-truth and more linear as well as nonlinear SFs with higher reproducibility.
Figure 4 and
Figure 5 showcase a comparative identification of shallow SFs [
22] using
SENDER at the first linear and nonlinear layers, alongside other peer methods, including DICA [
5], DNMF [
7,
8], SDDL [
6], and DBN [
25]. The results from the linear and nonlinear architectures of
SENDER and the other four peer methods are measured against ground-truth SFs [
22]. This experimental validation illustrates that the identification of shallow SFs using
SENDER aligns closely with established ground-truth SFs [
22], demonstrating no significant variance.
Figure 4.
This figure illustrates three representative slices of six shallow SFs derived via SENDER and four other peer methods at the first linear and nonlinear layer.
Figure 4.
This figure illustrates three representative slices of six shallow SFs derived via SENDER and four other peer methods at the first linear and nonlinear layer.
Figure 5.
This figure illustrates three representative slices of another six shallow SFs derived via SENDER and four other peer methods at the first linear and nonlinear layer.
Figure 5.
This figure illustrates three representative slices of another six shallow SFs derived via SENDER and four other peer methods at the first linear and nonlinear layer.
Importantly, the parameters for other peer methods were tuned according to guidelines from established studies [
5,
6,
8,
25] to ensure fairness and consistency in performance evaluation. The activation function for all layers within both
SENDER and the DBN was set to ReLU. The following figure illustrates qualitative results of identified shallow spatial features via
SENDER and other peer methods. Meanwhile, leveraging the previously introduced data-driven pipelines [
35,
50,
51,
52], we identify and present twelve group-wise shallow SFs that exhibit the highest spatial similarity with established ground-truth templates [
22]. This consistency underscores the robustness of
SENDER in detecting reproducible shallow SFs.
Notably,
Figure 6 demonstrates that the linear SFs identified from
SENDER’s second linear layer, like other methods, capture similar group-wise linear SFs at the second layer. Due to the lack of concrete ground truth for linear SFs, we sourced five representative linear ground-truth SFs from several key studies [
5,
57]. Overall, in
Figure 3, linear SFs derived via
SENDER at its second linear layer showcase a significant spatial similarity compared with ground-truth SFs in the first row. In particular,
SENDER identifies larger spatial areas than DICA, with linear SFs aligning with SDDL but differing from DNMF; however, SDDL misses some SFs in the fourth column that
SENDER can capture.
Figure 6.
This figure presents three representative slices of linear SFs revealed via SENDER at the second linear layer and three other peer methods at the second layer.
Figure 6.
This figure presents three representative slices of linear SFs revealed via SENDER at the second linear layer and three other peer methods at the second layer.
3.3. Qualitative Comparison of Identified Nonlinear SFs via SENDER and Other Peer Methods
Moreover, we present the qualitative results of nonlinear SFs extracted by
SENDER and DBN, as illustrated in
Figure 4. Since there is no concrete ground truth for nonlinear SFs, we employ multiple data-driven pipelines [
50,
51,
52] to conduct a ground truth for these networks. Specifically, after removing noisy SFs and artifacts [
51], we calculate the spatial similarity [
35] for identified SFs at the second layer across all test and retest subjects. We then perform clustering (e.g., k-means) to categorize SFs based on spatial similarity [
35]. Finally, we derive group-wise nonlinear SFs using Gaussian smoothing [
52] on individual SFs within each dominant cluster, including at least 90% of all subjects [
50].
In particular, as shown in
Figure 7,
SENDER demonstrates a superior capability in revealing eight nonlinear SFs compared to DBN, one of the most widely used nonlinear learning methods [
25]. For instance, in the first row and the first and second columns of
Figure 7, two nonlinear SFs derived by
SENDER highlight left and right partial regions with an SF, combined with major visual and auditory networks [
22]. In contrast, all SFs extracted by DBN exhibit approximate linear integration of various shallow SFs. Notably, other linear methods, such as DICA [
5], DNMF [
8], and SDDL [
6], fail to consistently identify these nonlinear SFs, with a low reproducibility.
Figure 7.
This figure provides 8 nonlinear SFs identified via SENDER at the second nonlinear layer and 4 SFs extracted by DBN at the second layer.
Figure 7.
This figure provides 8 nonlinear SFs identified via SENDER at the second nonlinear layer and 4 SFs extracted by DBN at the second layer.
3.4. Quantitative Comparison of Identified Shallow, Linear, and Nonlinear SFs via SENDER and Four Other Peer Methods
As discussed before, cross-validation and overfitting evaluation are not suitable metrics for unsupervised learning [
12,
14] like
SENDER; thus, in medical research, reproducibility should be one of the most critical gold standards [
27,
48,
58,
59,
60] for evaluating the performance of unsupervised methods. To validate the global reproducibility of
SENDER across all test–retest subjects, we employed identifiability, a measure ranging from 0 to 1, where higher values indicate greater reproducibility [
48] across all subjects. Identifiability was calculated based on the similarity of test and retest SFs [
48], in particular, after converting each 3D SF into a vector [
35,
50]. The reproducibility results can be viewed in
Figure 5. In addition, we examine the consistency of the hyperparameter estimator on all subjects.
Specifically, in
Figure 8a,b, we used identifiability [
48] by calculating the spatial similarity [
35] of test–retest SFs identified by
SENDER and other methods, using ground-truth SFs [
22]. For shallow SFs identified in
Figure 8a,
SENDER achieves an identifiability value of approximately 0.70. Meanwhile, the identifiability of most linear SFs identified by
SENDER in
Figure 8b exceeds 0.80, demonstrating substantial reproducibility [
48]. Notably, the identifiability of nonlinear SFs in
Figure 8c reaches nearly 0.90, which is higher than that of SFs extracted by DBN. Lastly, compared with other methods,
SENDER achieves the most accurate reconstruction error, below
.
In comparison, while other peer methods can identify twelve shallow features and five comparable linear spatial features at their first and second layers, respectively, the reproducibility of these features is notably lower. Specifically, the majority of the shallow and linear spatial features reported by these methods exhibit identifiability values below 0.70. This suggests that the features identified by these traditional methods are less consistent and may be more prone to variations across different datasets. The higher identifiability of features detected by SENDER underscores its superior performance in capturing consistent and reproducible features in the data, which is crucial for both basic neuroscience research and clinical applications. High reproducibility is particularly important in clinical settings, where consistent identification of brain features can lead to more accurate diagnoses and in-depth understanding of neurological conditions.
Figure 8.
A quantitative illustration of the identifiability of identified shallow, linear, and nonlinear SFs via SENDER and other peer methods as well as a comparison of reconstruction accuracy.
Figure 8.
A quantitative illustration of the identifiability of identified shallow, linear, and nonlinear SFs via SENDER and other peer methods as well as a comparison of reconstruction accuracy.
4. Discussion
We introduced an innovative computational framework,
SENDER, designed to concurrently reveal reproducible linear and nonlinear spatial features (SFs) from MBME fMRI data.
SENDER bridges the gap between traditional linear methods [
5,
6,
7,
8] and newer nonlinear approaches [
10,
18,
19].
Notably, a key strength of
SENDER lies in its ability to consistently, efficiently, and accurately reveal hierarchical SFs without requiring extensive fMRI datasets or high-performance computing resources such as GPUs or TPUs. Unlike DCAE [
19] and DBN [
10],
SENDER is more explainable, as demonstrated by theoretical predictions validated through comparisons with peer methods [
5,
6,
8,
10]. In particular,
SENDER achieves faster convergence and more accurate reconstruction loss by leveraging the advanced STORM optimizer [
61], which is particularly well suited for hybrid problems combining linear and nonlinear architectures [
61,
62]. Additionally,
SENDER employs the Rank Reduction Technique (RRT) for data-driven hyperparameter estimation, a significant improvement over peer methods that rely on manual tuning [
5,
6,
7,
8]. The integration of the
norm penalty enhances denoising capabilities, enabling
SENDER to outperform conventional methods, such as hierarchical clustering, in identifying linear and nonlinear SFs.
In addition,
SENDER produces reproducible SFs at deeper layers, outperforming peer methods in terms of robustness and precision [
5,
6,
8,
10]. For example, hierarchical clustering methods often fail to integrate SFs with low spatial similarity, whereas
SENDER effectively consolidates such features into unified SFs through its advanced framework [
10,
11,
35]. Importantly, our research categorizes hierarchical SFs in the human brain into five linear and eight nonlinear SFs. Notably, these SFs align with established neurophysiological representations for resting-state fMRI [
22,
33,
35]. For instance, the Salience Network (SN) modulates anticorrelated connectivity between the Default Mode Network (DMN) and the Executive Control Network (ECN) [
63]. Similarly, functional coupling between vision and auditory networks, as observed in our results, reflects known neurophysiological dynamics [
64]. By evaluating linear SFs identified by peer methods,
SENDER demonstrates superior accuracy, attributable to its integration of advanced
sparsity and RRT with the STORM optimizer. Additionally,
SENDER reveals nonlinear SFs that exhibit asymmetrical organization, further showcasing its capability to uncover complex brain dynamics.
Furthermore,
SENDER produces relatively reproducible SFs at the deeper layers compared with other peer methods [
5,
6,
8,
10]. This can be attributed to DICA’s relatively fast convergence rate and independence constraint [
35]. The mathematical evaluation framework and the fMRI validation procedure provided in this work should enable further development of deep matrix decomposition optimized for different types of real-world applications in biomedical imaging, with
SENDER as the current prevalent computational framework for fMRI analytics. Importantly, we also provided theoretical analyses as Theorem 1 to explain why
SENDER has more accurate reconstruction loss, which is comparable to deep neural networks.
Moreover, in this research, we not only validate the proposed computational framework named
SENDER but also successfully categorize hierarchical SFs within the human brain into five linear and eight nonlinear SFs. Meanwhile, both the linear and nonlinear SFs also showcase the superiority of MBME fMRI, attributable to the former’s better specificity for an in-depth understanding of neural activities. In this initial exploratory work on linear and nonlinear SFs using MBME fMRI, multiple derived linear and nonlinear SFs are consistent with neurophysiological representations between shallow SFs from previous studies [
22,
33,
35] on resting-state fMRI. For instance, the Salience Network, one of the most important SFs, is known to modulate the anticorrelated connectivity of other SFs, such as the Default Mode Network (DMN) and the Executive Control Network (ECN) [
63], hence the linkage of their nodes into a single linear SF (in
Figure 3, in the third column). The functional coupling of vision networks with an auditory network, shown in
Figure 3 in the fourth column, is also well known, given the role that the abnormal auditory stimulus plays in visual attention [
64]. Future neuroscientific studies will be required to empirically validate these linear SFs using demographic, clinical, cognitive, behavioral, and/or electrophysiological data. By evaluating most linear SFs identified among peer methods, we notice that
SENDER can accurately reveal linear SFs using MBME fMRI, whereas the identifications of other peer methods are not as accurate as
SENDER. These results could be attributed to its joint use of advanced
sparsity and RRT in conjunction with the STORM optimizer.
Specifically, in
Figure 7, we discovered more nonlinear SFs than DBN. Notably, some nonlinear SFs showcase an asymmetrical organization. For example, in the first row and first and second columns, identified SFs only include partial regions of a linear SF, shown in the first row and column in
Figure 7. Since the identification of these linear and nonlinear SFs does not require extensive training datasets nor specialized computing infrastructure, they can be easily applied to clinical research with the potential to generate novel SFs to play an important role as a wide array of biomarkers for neurodevelopmental, neurodegenerative, and psychiatric disorders [
65], including for diagnosis, prognosis, and treatment monitoring. This is particularly significant given the recent observation that neuropathology and psychopathology often affect shallow SFs differently than linear and nonlinear SFs at deeper layers. For example, various psychiatric disorders have been found to decrease shallow sensory SFs uniformly across patients [
66,
67] while increasing distinctiveness among patients in either linear or nonlinear SFs at the deeper layers [
65]. In fMRI studies of mild traumatic brain injury (TBI), altered brain functionality has been found early after concussion within the patient’s brain, such as the SN, DMN, and ECN [
68]. The linear interactions of SFs, such as those between the SN and the DMN, are thought to be especially important for outcomes after TBI and can be used to guide personalized treatment [
69,
70]. Disordered coupling of the SN with the DMN and ECN has also been shown in mild cognitive impairment [
71].
Hence, prevalent neurological disorders such as head trauma and neurodegenerative diseases are thought to affect multiple levels of the functional organization in human brains. Such linear and nonlinear interactions across multiple SFs can be investigated with deeper layers of an advanced deep learning model like
SENDER that integrate their spatially distinct gray matter nodes into a linear or nonlinear SF, as shown in
Figure 3 (refer to the top row). These examples show how more principled data-driven characterization of this hierarchy, particularly at its higher levels, holds great promise for providing clinically actionable diagnostics of neurological and psychiatric diseases.
Lastly, despite its advantages,
SENDER has room for improvement. Enhancements to RRT, integration with Reinforcement Learning, and optimization of its computational efficiency are needed. For example, the current decomposition of a single fMRI signal matrix requires approximately two hours, highlighting the need for faster optimization techniques. Furthermore,
SENDER currently lacks insight into the neurological representations of SFs and their associations with specific disorders. Future research will apply
SENDER to diverse fMRI datasets, including those for Alzheimer’s and Parkinson’s diseases, to identify disorder-specific linear and nonlinear SFs. Improved fMRI sensitivity and resolution, such as SLIDER-SMS fMRI [
72], will enable
SENDER to uncover even more complex SFs, advancing our understanding of brain functionality.
To summarize, the benefits of
SENDER gain importance as the spatial and temporal resolution and sensitivity of fMRI continue to increase with improved MR imaging hardware and pulse sequences. Briefly,
SENDER delivers several advantages in detecting the hierarchical and overlapping organization of SFs compared to previously described methods in a data-driven manner, such as DICA [
5], DNMF [
8], SDDL [
6], and DBN [
10]. Notably,
SENDER does not have the constraints of spatial independence that DICA has [
35]. Since
SENDER can reveal extensively overlapped SFs [
10,
11,
35] and estimates the vital hyperparameters automatically, it is easy to leverage the number and size of each layer, i.e., the number of artificial neurons. Other peer methods, such as DICA [
5], SDDL [
6], DNMF [
8], and DBN [
10], require manual hyperparameter tuning. Moreover, compared to Deep Neural Networks,
SENDER has several advantages: (a) fewer training samples, i.e., the capability to reconstruct a single individual’s scan; (b) fewer extensive computational resources; (c) guaranteed convergence to a unique fixed point; and (d) automatic hyperparameter estimation. In this research, our results showcase the benefits of
SENDER for MBME imaging [
27,
28,
29]. Additionally, we aim to push further beyond
SENDER, advancing our understanding of brain functionality through innovative computational frameworks that further deepen and expand our insights in a comprehensive understanding of brain functionality. Moreover,
SENDER is capable of incorporating Electroencephalogram (EEG) to explore physiological representation and potential association of linear and nonlinear SFs with neurological disorders. Meanwhile, by leveraging advanced norm penalty,
SENDER can enable the detection of abnormal answering of students, such as rapid guessing.