Next Article in Journal / Special Issue
Skin Lesion Segmentation Using Deep Learning with Auxiliary Task
Previous Article in Journal
A Comparative Analysis for 2D Object Recognition: A Case Study with Tactode Puzzle-Like Tiles
Previous Article in Special Issue
Investigating the Potential of Network Optimization for a Constrained Object Detection Problem
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Transfer Learning in Magnetic Resonance Brain Imaging: A Systematic Review

A.I. Virtanen Institute for Molecular Sciences, University of Eastern Finland, 70150 Kuopio, Finland
*
Author to whom correspondence should be addressed.
The order of these authors was randomized.
J. Imaging 2021, 7(4), 66; https://doi.org/10.3390/jimaging7040066
Submission received: 9 February 2021 / Revised: 26 March 2021 / Accepted: 29 March 2021 / Published: 1 April 2021
(This article belongs to the Special Issue 2020 Selected Papers from Journal of Imaging Editorial Board Members)

Abstract

:
(1) Background: Transfer learning refers to machine learning techniques that focus on acquiring knowledge from related tasks to improve generalization in the tasks of interest. In magnetic resonance imaging (MRI), transfer learning is important for developing strategies that address the variation in MR images from different imaging protocols or scanners. Additionally, transfer learning is beneficial for reutilizing machine learning models that were trained to solve different (but related) tasks to the task of interest. The aim of this review is to identify research directions, gaps in knowledge, applications, and widely used strategies among the transfer learning approaches applied in MR brain imaging; (2) Methods: We performed a systematic literature search for articles that applied transfer learning to MR brain imaging tasks. We screened 433 studies for their relevance, and we categorized and extracted relevant information, including task type, application, availability of labels, and machine learning methods. Furthermore, we closely examined brain MRI-specific transfer learning approaches and other methods that tackled issues relevant to medical imaging, including privacy, unseen target domains, and unlabeled data; (3) Results: We found 129 articles that applied transfer learning to MR brain imaging tasks. The most frequent applications were dementia-related classification tasks and brain tumor segmentation. The majority of articles utilized transfer learning techniques based on convolutional neural networks (CNNs). Only a few approaches utilized clearly brain MRI-specific methodology, and considered privacy issues, unseen target domains, or unlabeled data. We proposed a new categorization to group specific, widely-used approaches such as pretraining and fine-tuning CNNs; (4) Discussion: There is increasing interest in transfer learning for brain MRI. Well-known public datasets have clearly contributed to the popularity of Alzheimer’s diagnostics/prognostics and tumor segmentation as applications. Likewise, the availability of pretrained CNNs has promoted their utilization. Finally, the majority of the surveyed studies did not examine in detail the interpretation of their strategies after applying transfer learning, and did not compare their approach with other transfer learning approaches.

1. Introduction

Magnetic resonance imaging (MRI) is an non-invasive imaging technology that produces three dimensional images of living tissue. MRI measures radio-frequency signals emitted from hydrogen atoms after the application of electromagnetic (radio-frequency) waves, localizing the signal using spatially varying magnetic gradients, and is capable to measure various properties of the tissue depending on the particular pulse sequence applied for the measurement [1]. Particularly, anatomical (or structural) MRI is used to generate images of the anatomy of the studied region. Functional MRI (fMRI) demonstrates regional, time-varying changes in brain metabolism in the form of increased local cerebral blood flow (CBF) or in the form of changes in oxygenation concentration (Blood Oxygen Level Dependent, or BOLD contrast) that are both consequences of increased neural activity [2]. Diffusion MRI (dMRI) is used to assess the microstructural properties of tissue based on the diffusive motion of water molecules within each voxel [3]. The use of MRI is increasing rapidly, not only for clinical purposes but also for brain research and development of drugs and treatments. This has called for machine learning (ML) algorithms for automating the steps necessary for the analysis of these images. Common tasks for machine learning include tumor segmentation [4], registration [5], and diagnostics/prognostics [6].
However, the variability in, for instance, image resolution, contrast, signal-to-noise ratio or acquisition hardware leads to distributional differences that limit the applicability of ML algorithms in research and clinical settings alike [7,8]. In other words, an ML model trained for a task in one dataset may not necessarily be applied to the same task in another dataset because of the distributional differences. This difficulty emerges in large datasets that combine MR images from multiple studies and acquisition centers since different imaging protocols and scanner hardware are used, and also hampers the clinical applicability of ML techniques as the algorithms would need to be re-trained in a new environment. Additionally, partly because of the versatility of MRI, an ML model trained for a task could be useful in another, related task. For instance, an ML model trained for brain extraction (skull-stripping) might be useful when training an ML model for tumor segmentation. Therefore, developing strategies to address the variation in data distributions within large heterogeneous datasets is important. This review focuses on transfer learning, a strategy in ML to produce a model for a target task by leveraging the knowledge acquired from a different but related source domain [9].
Transfer learning (or knowledge transfer) reutilizes knowledge from source problems to solve target tasks. This strategy, inspired by psychology [10], aims to exploit common features between related tasks and domains. For instance, an MRI expert can specialize in computed tomography (CT) imaging faster than someone with no knowledge in either MRI or CT. There exist several surveys [9,11,12] of transfer learning in machine learning literature, including in medical imaging [13], but to our knowledge, no surveys have focused on its use in brain imaging or MRI applications. Pan and Yang [11] presented one of the earliest surveys in transfer learning, which focused on tasks with the same feature space (i.e., homogeneous transfer learning). Pan and Yang [11] proposed two transfer learning categorizations that are in wide use. One categorization divided approaches based on the availability of labels, and the other based on which knowledge is transferred (e.g., features, parameters). Day and Khoshgoftaar [9] surveyed heterogeneous transfer learning applications, i.e., tasks with different feature spaces, and, also based on the availability of labels, transfer learning approaches were divided into 38 categories. More recently, another survey [12] expanded the number of categories to over 40. These recent categorizations can be useful in a general context as they were derived from approaches from diverse areas. However, the large number of proposed categories can lead to their underutilization and the classification of similar strategies differently in specific fields, such as MR brain imaging. We chose the categorization proposed by Pan and Yang [11] since it divides transfer learning approaches into few categories, and because such categorization is the basis of categorization in other transfer learning surveys [9,12,13]. Furthermore, we introduced new subcategories that describe how transfer learning was applied, revealing widely used strategies within the MR brain imaging community.
Systematic reviews analyze methodologically articles from specific areas of science. Recent systematic reviews in biomedical applications of machine learning have covered such topics as predicting stroke [14], detection and classification of transposable elements [15], and infant pain prediction [16]. Here, we present the first systematic review of transfer learning in MR brain imaging applications. The aim of this review is to identify research trends, and to find popular strategies and methods specifically designed for brain applications. We highlight brain-specific methods and strategies addressing data privacy, unseen target domains, and unlabeled data—topics especially relevant in medical imaging. Finally, we discuss the research directions and knowledge gaps we found in the literature, and suggest certain practices that can enhance methods’ interpretability.

2. Transfer Learning

According to [11], we define a domain in transfer learning as D = { X , P ( X ) } , where X is the feature space, and P ( X ) with X = x 1 , , x n X is a marginal probability distribution. For example, X could include all possible images derived from a particular MRI protocol, acquisition parameters, and scanner hardware, and P ( X ) depend on, for instance, subject groups, such as adolescents or elderly people. Tasks comprise a label space Y and a decision function f, i.e., T = { Y , f } . The decision function is to be learned from the training data ( X , Y ) . Tasks in MR brain imaging can be, for instance, survival rate prediction of cancer patients, where f is the function that predicts the survival rate, and Y is the set of all possible outcomes. Given a source domain D S and task T S , and a target domain D T and task T T , transfer learning reutilizes the knowledge acquired in D S and T S to improve the generalization of f T in D T [11]. Importantly, D S must be related to D T , and T S must be related to T T [17]; otherwise, transfer learning can worsen the accuracy on the target domain. This phenomenon, called negative transfer, has been recently formalized in [18] and studied in the context of MR brain imaging [19].
Transfer learning approaches can be categorized based on the availability of labels in source and/or target domains during the optimization [11]: unsupervised transfer learning (unlabeled data), transductive (labels available only in the source domain), and inductive approaches (labels available in the target domains and, optionally, in the source domains). Table 1 illustrates these three types with examples in MR brain imaging applications.
Following the simple categorization described in Pan and Yang [11], transfer learning approaches can be grouped into four categories based on the knowledge transferred. Instance-based approaches estimate and assign weights to images to balance their importance during optimization. Feature-based approaches seek a shared feature space across tasks and/or domains. These approaches can be further divided into asymmetric (transforming target domain features into the source domain feature space), and symmetric (finding a common intermediate feature representation). Parameter-based approaches find shared priors or parameters between source and target tasks/domains. Parameter-based approaches assume that such parameters or priors share functionality and are compatible across domains, such as a domain-invariant image border detector. Finally, relational-based approaches aim to exploit common knowledge across relational domains.

Related Approaches

There exist various strategies to improve ML generalization in addition to transfer learning. In multi-task learning, ML algorithms are optimized for multiple tasks simultaneously. For instance, Weninger et al. [20] proposed a multi-task autoencoder-like convolutional neural network with three decoders—one per task—to segment and reconstruct brain MR images containing tumors. Zhou et al. [21] presented an autoencoder with three branches for coarse, refined, and detailed tumor segmentation. Multi-task learning differs from transfer learning in that transfer learning focuses on target tasks/domains whereas multi-task learning tackles multiple tasks/domains simultaneously; thus, considering source and target tasks/domains equally important [11]. Data augmentation, which adds slightly transformed copies of the training data to the training set, can also enhance algorithms’ extrapolability [22,23]. Data augmentation is useful when images with certain properties (i.e., a specific contrast) are scarce, and it can complement other techniques, such as multi-task or transfer learning. However, in contrast to multi-task or transfer learning, data augmentation ignores whether the knowledge acquired from similar tasks can be reutilized. Additionally, increasing the training data also increases computational costs, and finding advantageous data transformations is non-trivial. Finally, meta-learning aims to find parameters or hyper-parameters of machine learning models to generalize well across different tasks. In contrast to transfer learning, meta-learning does not focus on a specific target task or domain, but on all possible domains, including unseen domains. Liu et al. [24] showed that the model-agnostic meta-learning strategy [25] yields state-of-the-art segmentations in MR prostate images.

3. Methods

We followed the PRISMA statement [26]. The PRISMA checklist is included in the Supplementary Materials.

3.1. Search Strategy

We searched articles about transfer learning applied to MR images in the Scopus database (https://www.scopus.com, accessed on 19 October 2020). In addition, we searched for relevant articles from the most recent (2020) Medical Image Computing and Computer Assisted Interventions (MICCAI) conference from the Springer website (https://link.springer.com/search?facet-conf-event-id=miccai2020&facet-content-type=Chapter, accessed on 30 October 2020) because they were unavailable in Scopus at the time when the search was performed. To ensure we retrieved relevant results, we focused on finding certain keywords in either the title, abstract, or article keywords. We also searched for “knowledge transfer” and "domain adaptation" as these are alternative names or subclasses of transfer learning. Similarly, we searched for different terms related to MRI, including magnetic resonance and diffusion imaging. Table 2 shows the exact searched terms used. Note that “brain” was not one of the keywords as we observed that including it would have led to the omission of several relevant articles.

3.2. Study Selection

We excluded non-article records retrieved (e.g., entire conferences proceedings). We reviewed the abstracts of all candidate articles to discard studies that were not about MR brain imaging or did not apply transfer learning. For this, each abstract was reviewed by two co-authors. In more detail, we used the following procedure to assign the abstracts to co-authors. JMV assigned the abstracts randomly to six labels so to that each label corresponded to one co-author (all co-authors except JMV). JT assigned the labels with co-authors so that JMV was unaware of the true identity of each reviewer to guarantee anonymity and to reduce reviewer bias during the screening. Furthermore, to ensure the same criteria were applied during this review by each co-author, review guidelines written by JT and JVM and commented by other co-authors were distributed among the co-authors. Reviewers determined the relevance of each abstract they reviewed and, to support their decision, reviewers also included extra information from the screened abstracts, including the studied organs (e.g., brain, heart), MRI modality (e.g., anatomical, functional), and comments. If the two reviewers disagreed about the relevance of an abstract, JVM solved the disagreement.

3.3. Data Collection

Articles that were deemed relevant based on their abstracts were randomly and evenly divided among all seven co-authors for their full review (based on the complete article). We extracted information to categorize the surveyed papers, including the methods, datasets, tasks, types of input features, and whether the labels were available in source and/or in target domain. To ensure criteria homogeneity in the classification, JVM verified the collected data and discussed disagreements with the other co-authors. The information extracted in this process is in the Supplementary Materials.
We discarded studies that were irrelevant to this survey (i.e., not about transfer learning or MR brain imaging), too unclear to obtain relevant information, did not perform a proper experimental evaluation of the transfer learning approach, or we could not access. The authors of articles that we could not access were additionally contacted through https://www.researchgate.net/ (accessed on 4 December 2020) to obtain the article. Additionally, we included and examined other relevant articles coming to our attention during the review of the articles.

3.4. Screening Process

Figure 1 summarizes the screening process and the number of articles after each step. First, we retrieved 399 records from Scopus (19 October 2020) and 34 from Springer (30 October 2020). We excluded 42 records from Scopus: 41 entire proceedings of conferences and one corrigendum, leaving 391 journal and conference articles. After the abstract review, we excluded 220 articles that were either not about transfer learning or MR brain imaging. We reviewed the remaining 171 articles based on full paper, and 44 articles were discarded: 26 studies mixed data from the same subject to their training and testing sets, 8 were unrelated to transfer learning in MR brain imaging, 7 were unclear, 2 were inaccessible, and 1 was a poster. The studies that mixed data from the same subject typically performed ML tasks on slices of MRI instead of MRI volumes, and both training and testing sets appeared to contain data from the same volumetric MRI. Finally, we added two relevant articles coming to our attention while reading the articles, resulting in 129 articles that were included in this survey.

4. Results

4.1. Applications

We found 29 applications that we considered distinct. Table 3 summarizes the distinct tasks and applications within these tasks. Note that some articles studied more than one task/application. Figure 2 (left) shows the number of articles that addressed different brain diseases according to the 11th International Classification of Diseases (ICD-11) (https://icd.who.int/en, accessed on 18 November 2020).
Classification tasks were the most widely studied and, among these, dementia-related (neurocognitive impairment, Figure 2 (left)) and tumor-related (neoplasms, Figure 2) applications accounted for 45.59% and 14.71% of all classification tasks, respectively. Other applications included autism spectrum disorder diagnostics, and functional MRI decoding that is classification of, for instance, stimulus, or cognitive state based on observed fMRI [40]. Segmentation was the second most popular task, studied in one-third of the articles. Segmentation applications included anatomical, lesion, and tumor segmentation with each application studied in approximately one-third of the segmentation articles. Regression problems were considerably less common than classification and segmentation, and, within these, age prediction was predominant. Among the other applications, image reconstruction and registration were the most widely studied.
Figure 2 (right) shows the number of articles that employed anatomical, functional, diffusion MRI, and multimodal data. The majority of the surveyed articles (98) utilized anatomical MR images, including T1-weighted, T2-weighted, FLAIR, and other contrasts. The number of studies that utilized fMRI and diffusion MRI data was 18 and 6, respectively. Finally, 8 studies were multimodal, combining MRI with positron emission tomography (PET) or CT.

4.2. Machine Learning Approaches

Figure 3 (left) illustrates the number of articles that utilized specific machine learning and statistical methods. Convolutional neural networks (CNNs) were applied in the majority of articles (68.22%), followed by kernel methods (including support vector machines (SVMs) and support vector regression), multilayer perceptrons, decision trees (including random forests), Bayesian methods, clustering methods, elastic net, long short-term memory networks, and deep neural networks without convolution layers. Several other methods (Figure 3 (left), "Others") appeared only in a single article: deformable models, manifold alignment, graph neural networks, Fisher’s linear discriminant, principal component analysis, independent component analysis, joint distribution alignment, singular value decomposition, Pearson correlation, and Adaboost. Figure 3 (right) shows the number of articles that utilized CNNs, kernel methods, and other approaches across years. Since 2014, the number of articles applying transfer learning to MR brain imaging applications and the use of CNNs have grown exponentially. In the last two years 80% of the articles (68) utilized CNNs, and, during this period, their use and the total number of articles have started to converge. Finally, we computed a contingency table between the machine learning methods and the brain disease categories in the surveyed papers, but found no correlation between methods and disease categories (see Supplementary Figure S1).

4.3. Transfer Learning Approaches

Table 4 summarizes the transfer learning strategies found in the surveyed papers. Note that certain articles applied multiple strategies. We classified these strategies into instance, feature representation ("feature" for short), parameter, and relational knowledge [11]. Afterwards, we divided feature-based approaches into symmetric and asymmetric. Since the categorization described in [11] is general, we propose new subcategories, described below. These new subcategories, based on the strategies found during our survey, aim to reduce ambiguity and to facilitate the identification of popular transfer learning methods in the context of MR brain imaging.
We divided instance-based approaches into fixed and optimized sub-categories. Fixed weights are those weights assigned following certain preset criteria, such as assigning higher weights to target domain images. On the other hand, optimized weights are weights estimated by solving an optimization problem. Furthermore, we separated optimized weights approaches based on whether such optimization problem required labels—requirement not always feasible in medical imaging—into supervised and unsupervised.
We divided symmetric feature-based approaches that find a common feature space between source and target domain were into direct and indirect subcategories. Direct approaches are methods that operate directly on the feature representations of source and target domain data, thereby requiring these data simultaneously. We considered such requirement an important discriminative since it can limit the applicability of the approach. More precisely, approaches that require source and target domain data simultaneously may need more memory and, importantly, may not be applicable if source and target domain data cannot be shared due to privacy issues. As an example of this category, consider an approach that minimizes the distance between the feature representations of source and target domain images, aiming to transform the data into the same feature space. In contrast, indirect approaches do not operate directly on the feature representations and do not need simultaneous access to source and target domain data.
Parameter-based approaches consisted of two steps: first, finding the shared priors or parameters, and second, fine-tuning all or certain parameters in the target domain data. Since the approaches that fine-tuned all parameters assumed that their previous parameters were closer to the solution than their random initialization, we considered these approaches as prior sharing. Although sharing parameters could be also considered as sharing priors, separating these two approaches based on whether certain or all model parameters were fine-tuned revealed the popularity of different strategies. Furthermore, we propose to divide parameter-sharing approaches into two categories: approaches that only utilize one model, and approaches in which the shared parameters correspond to a feature-extracting model for optimizing separate models, thereby comprising multiple models.

4.3.1. Instance-Based Approaches

We found 16 approaches (11.63% of the studies) that applied transfer learning by weighing images. We propose to divide these approaches based on whether images’ weights were fixed or optimized, and in the latter case, distinguish between supervised or unsupervised optimization.
Fixed-weights strategies included sample selection (i.e., binary weights), such as in [41], where authors discarded certain images in datasets biased with more subjects with a given pathology. Likewise, Cheng et al. [42,43] trained a classifier to perform sample selection based on the images’ probability these images belong to the source domain. Similar probabilities and non-binary fixed weights were applied in [44,45,46], respectively, allowing the contribution of all images in the studied task.
Images’ weights that were optimized via unsupervised strategies relied on data probability density functions (PDFs). The surveyed articles that applied these strategies optimized images’ weights separately before tackling the studied task (e.g., Alzheimer’s diagnostics/prognostics). Images’ weights were derived by minimizing the distance (Kullback-Leibler divergence, Bhattacharyya, squared Euclidean, and maximum mean discrepancy) between the PDFs in the source and target domains [47,48,49,50]. On the other hand, supervised strategies optimized images’ weights and the ML model simultaneously by minimizing the corresponding task-specific loss [45,51,52,53,54]. Notably, supervised and unsupervised strategies can be combined, and approaches can also incorporate extra information unused in the main task. For instance, Wachinger et al. [55] also considered age and sex during the unsupervised optimization of PDFs for Alzheimer’s diagnostics/prognostics.

4.3.2. Feature-Based Approaches

We found 38 approaches (29.46% of the studies) that applied transfer learning by finding a common feature space between source and target domains/tasks. These approaches were divided into symmetric and asymmetric (see Section 2). Additionally, we propose to subdivide symmetric approaches based on whether the common feature space was achieved by directly operating on the source and target feature representations (e.g., by minimizing their distance), or indirectly.
We found 7 asymmetric approaches that transformed target domain features into source domain features via generative adversarial networks [56,57], Bregman divergence minimization [58], probabilistic models [59], median [60], and nearest neighbors [61]. Contrarily, Qin et al. [62] transformed source domain features into target domain features via dictionary-based interpolation to optimize a model on the target domain.
Among the surveyed 31 symmetric approaches, direct approaches operated on the feature representations across domains by minimizing their differences (via mutual information [63], maximum mean discrepancy [46,49,64], Euclidean distance [65,66,67,68,69,70,71], Wasserstein distance [72], and average likelihood [73]), maximizing their correlation [74,75] or covariance [36], and introducing sparsity with L1/L2 norms [42,76]. On the other hand, indirect approaches were applied via adversarial training [28,41,54,77,78,79,80,81,82,83,84,85], and knowledge distillation [86].

4.3.3. Parameter-Based Approaches

We found 87 approaches (65.89% of the studies) that applied transfer learning by sharing priors or parameters. Parameter-sharing approaches were further subdivided based on whether one model or multiple models were optimized.
The most common approach to apply a prior-sharing strategy—and, in general, transfer learning—was fine-tuning all the parameters of a pretrained CNN [29,31,32,33,35,39,71,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119] (80% of all prior-sharing methods). Other approaches utilized Bayesian graphical models [37,38,120,121], graph neural networks [122], kernel methods [64,123], multilayer perceptrons [124], and Pearson-correlation methods [125]. Additionally, Sato et al. [27] proposed a general framework to inhibit negative transfer. Within the prior-sharing group, 20 approaches utilized a parameter initialization derived from pretraining on natural images (i.e., ImageNet [126]) whereas 26 approaches pretrained on medical images.
The second most popular strategy to apply transfer learning was fine-tuning certain parameters in a pretrained CNN [34,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146]. The remaining approaches first optimized a feature extractor (typically a CNN or a SVM), and then trained a separated model (SVMs [30,45,147,148,149], long short-term memory networks [150,151], clustering methods [148,152], random forests [70,153], multilayer perceptrons [154], logistic regression [148], elastic net [155], CNNs [156]). Additionally, Yang et al. [157] ensembled CNNs and fine-tuned their individual contribution. Within the parameter-sharing group, 17 approaches utilized a ImageNet-pretrained CNN, and 15 others pretrained on medical images.
We found 40 studies that utilized publicly-available CNN architectures. The most popular were VGG [158] (23), ResNet [159] (15), and Inception [160,161,162] (11). Nearly three-fourths of these studies fine-tuned the ImageNet-pretrained version of the CNNs whereas one-third pretrained the networks in medical datasets. Finally, among these 40 studies, 13 articles compared the performance of multiple architectures, and VGG (4) and Inception (4) usually provided the highest accuracy.

4.4. Transfer Learning Approaches Relevant to MR Brain Imaging

We closely examined transfer learning approaches that were inherently unique to brain imaging. Additionally, we included strategies that considered data privacy, unseen target domains, and unlabeled data, as these topics are especially relevant to the medical domain.
  • Brain MRI-specificity
Aside from employing brain MRI-specific input features (e.g., brain connectivity matrices in fMRI, diffusion values in dMRI) or pre-processing (e.g., skull-stripping), we only found two transfer learning strategies unique to brain imaging. Moradi et al. [36] considered cortical thickness measures of each neuroanatomical structure separately, yielding multiple domain-invariant feature spaces—one per brain region. This approach diverges from the other surveyed feature-based strategies that sought a single domain-invariant feature space for all the available input features. Cheng et al. [42] extracted the anatomical volumes of 93 regions of interest in the gray matter tissue of MRI and PET images. Afterwards, authors applied a feature-based transfer learning approach with sparse logistic regression separately in MRI and PET images, yielding informative brain regions for each modality. Finally, authors combined these features with cerebrospinal fluid (CSF) biomarkers, and applied an instance-based approach to find informative subjects.
  • Privacy
We found two frameworks that considered privacy. Li et al. [54] built a federated-learning framework that optimized a global model by transferring the parameters of site-specific local models. This framework requires no database sharing, and the transferred parameters are slightly perturbed to ensure differential privacy [163]. Sato et al. [27] proposed a framework that applies online transfer learning and only requires the output—not the images—of the models optimized in the source domain data. Notably, this framework also tackles negative transfer. Besides these two frameworks, parameter-based approaches that were fine-tuned exclusively in target domain data also protected subjects’ privacy since they required no access to source domain images during the fine-tuning.
  • Unseen Target Domains
We found four studies that considered unseen target domains. van Tulder and de Bruijne [70], Shen and Gao [78] utilized multimodal images and proposed a symmetric feature-based approach that drops a random image modality during the optimization, avoiding the specialization to any specific domain. Hu et al. [38], Varsavsky et al. [84] considered that each image belongs to a different target domain, and presented a strategy that adapts to each image. Note that these approaches also protected data privacy as no access to source domain images was required to adapt to the target domain.
  • Unlabeled Data
We revisited transfer learning strategies that handled unlabeled data in the target domain and, optionally, in the source domain. Van Opbroek et al. [47] presented an instance-based approach that relied exclusively on the difference between data distributions, thereby requiring no labels. Goetz et al. [44] expanded this idea and also incorporated source domain labels to optimize a model for estimating images’ weights. On the other hand, feature-based approaches can include source domain labels while finding appropriate feature transformations. Following this idea, Li et al. [58] minimized the Bregman distance between the source and target domain features while optimizing a task-specific classifier. Similarly, Ackaouy et al. [72], Orbes-Arteaga et al. [82] sought a shared feature space while minimizing Dice loss and a consistency loss, respectively, with source domain labels. Additionally, various indirect symmetric feature-based approaches jointly optimized an adversarial loss and a task-specific loss on the source domain images [79,80,83]. Finally, Orbes-Arteainst et al. [86] used knowledge distillation, training a teacher model on the labeled source domain, and optimizing a student network on the probabilistic maps from the teacher model derived with the source and target domain images.

4.5. Tackling Transfer Learning Issues

The application of transfer learning has a few potential detrimental consequences that only two studies included in the survey have investigated. Kollia et al. [136] considered source and target domain images jointly during the optimization to avoid catastrophic forgetting—lower performance on the source domain after applying transfer learning to the target domain. Sato et al. [27] proposed an algorithm to detect aneurysms that directly inhibits negative transfer [18]—worse results on the target domain than if no transfer learning was applied.

5. Discussion

5.1. Research Directions of Transfer Learning in Brain MRI

We surveyed 129 articles on transfer learning in brain MRI, and our results indicate an increased interest in the field in recent years. Alzheimer’s diagnostics/prognostics, tumor classification, and tumor segmentation were the most studied applications (see Figure 2 (left), and Table 3). The popularity of these applications is likely linked to the existence of well-known publicly or easily available databases, such as ADNI [164], and the Brain Tumor Segmentation challenge (BraTS) datasets [4,165,166]. We would like to point out that there are also other large MRI databases available to researchers (e.g., ABIDE [167,168], Human Connectome Project [169]), but the number of articles in this review utilizing these other databases was considerably lower than ADNI or BraTS. CNNs were the most used machine learning method, utilized in 68.22% of all the reviewed papers, and 80% in the last two years (Figure 3). The demonstrations of outperformance of CNNs over other methods [170], and the availability of trained CNNs in ImageNet [126], such as AlexNet [171], VGG [158], and Inception [160], probably explains CNNs’ popularity.
We classified transfer learning approaches into instance, feature representation, parameter, and relational knowledge [11]. We noticed that this classification was too coarse, hindering the identification of popular solutions within the MR brain imaging community. As Pan and Yang [11] indicated, this categorization is based on "what to transfer" rather than "how to transfer." Therefore, based on the surveyed articles, we refined this categorization by introducing subcategories that define transfer learning approaches more precisely (see Section 4.3 and Table 4). Our categorization divided instance-based approaches based on whether images’ weights were fixed or optimized, and in the latter case, subdivided to separate supervised and unsupervised optimization. Symmetric feature-based approaches were split depending on whether the strategy operated directly or indirectly on the feature representations between domains. Parameter-sharing-based approaches were divided based on whether one or multiple models were optimized. With our categorization, we found that most of the strategies pretrained a model or utilized a pretrained model, and subsequently fine-tuned all, certain parameters, or a separate model on the target domain data. Among these studies, a similar number of approaches either utilized ImageNet-pretrained CNNs or pretrained on medical images.

5.2. Knowledge Gaps

Beyond showing accuracy gains, the surveyed articles rarely examined other approach-specific details. Only a few studies that optimized images’ weights [49,54,55] showed their post-optimization distribution. Interestingly, several of these weights became zero or close to zero, indicating that the contribution of their corresponding images to tackle the studied task was negligible. The incorporation of sparsity-inducing regularizers as in [42], and a closer view to these weights could lead to intelligent sample selection strategies and advances in curriculum learning [172]. Regarding feature-based transfer learning approaches, various studies [28,41,46,54,58,66,70,71,76] illustrated, typically with t-SNE [173], that the source and target domain images lied closer in the feature space after the application of their method. However, we found no articles that compared and illustrated the feature space after implementing different strategies. This also raises the question of how to properly quantify that a feature space distribution is better than another in the context of transfer learning.
With a limited number of training data in the target domain, fine-tuning ImageNet-pretrained CNNs has been demonstrated to yield better results than training such CNNs from scratch, even in the medical domain [174]. In agreement with this observation, nearly half of the parameter-based approaches followed this practice. Fine-tuning all parameters (prior-sharing) and fine-tuning certain parameters (parameter-sharing) were widely used methods, although in the latter case we rarely found justifications for choosing which parameters to fine-tune. Since the first layers of CNNs capture low-level information, such as borders and corners, various studies [34,134,135,137,145] have considered that those parameters can be shared across domains. Besides, as adapting pretrained CNNs to the target domain data requires, at least, replacing the last layer of these models, researchers have likely turn fine-tuning only this randomly-initialized layer into common practice, although we found no empirical studies that supported such practice. Four surveyed articles studied different fine-tuning strategies with CNNs pretrained on ImageNet [96,134] and medical images [129,130]. The approaches that utilized ImageNet-pretrained CNNs [96,134] reported that fine-tuning more layers led to higher accuracy, suggesting that the first layers of ImageNet-pretrained networks—that detect low-level image characteristics, such as corners and borders—may not be adequate for medical images. Furthermore, Bashyam et al. [101] reported that Inceptionv2 [161] pretrained on medical images outperformed its ImageNet-pretrained version in Alzheimer’s, mild cognitive impairment, and schizophrenia classification. Finally, a recent study [130] showed that fine-tuning the first layers of a CNN yielded better results than the traditional approach of exclusively fine-tuning the last layers, suggesting that the first layers encapsulate more domain-dependent information.
The number and which parameters require fine-tuning could depend on the target domain/task and on the training set size, as large models require more training data. Likewise, the training set size in the target domain could be task-dependent. Only a few studies investigated the application of transfer learning with different training set sizes [99,101,128,129,130,135,147,150]. Among these, most articles reported that with a sufficiently large training set, models trained from scratch achieved similar or even better [99,135] results than applying transfer learning.

5.3. Limitations

A limitation of this survey arises from the subjectivity and difficulty of classifying transfer learning methods into the sub-categories. Additionally, a few surveyed articles combined multiple transfer learning approaches, hindering their identification and classification. Occasionally, determining the source and target tasks/domains and whether labels were used exclusively in the transfer learning approach was challenging. Thus, the numbers of approaches belonging to specific categories might not be exact. Another limitation is that, especially with CNNs, it is sometimes ambiguous to decide whether an approach is a transfer learning approach. We based our literature search largely on the authors’ opinions on whether their article was about transfer learning. Additionally, when rejecting articles from review, we typically trusted authors’ judgments: we accepted if they indicated that their approach was transfer learning. Thus, borderline articles, which the authors themselves did not categorize as transfer learning, might have escaped our literature search, despite similar approaches possibly being included in this review.

6. Conclusions

The growing number of transfer learning approaches for brain MRI applications signals the perceived importance of this field. The need for transfer learning arises from the heterogeneity in large datasets that combine MR images from multiple studies and acquisition centers with different imaging protocols and scanner hardware. Additionally, transfer learning can increase ML clinical applicability by simplifying the training in new environments, hence the increased interest in transfer learning.
Well-known datasets that are easily available to researchers have clearly boosted certain applications, such as Alzheimer’s diagnostics/prognostics and tumor segmentation. Similarly, the availability of pretrained CNNs has contributed to their popularization in transfer learning. Indeed, we found that pretraining a CNN, or utilizing a pretrained CNN, to fine-tune it on target domain data was the most widely used approach to apply transfer learning. Additionally, we noticed that the studies that investigated different fine-tuning strategies in ImageNet-pretrained CNNs reported higher accuracy after fine-tuning all the parameters and not just the last layers, as many other approaches did.
We found various studies tackling issues relevant to the medical imaging community, such as privacy, although we only found two brain-specific approaches that, coincidentally, did not utilize CNNs. Besides, the surveyed studies rarely examined in depth their solutions after applying transfer learning. For example, instance-based approaches seldom interpreted images’ weights; feature-based approaches did not compare various feature spaces derived with other methods; and the majority of parameter-based approaches relied on assumptions to decide which parameters to share.
The importance of enhancing an algorithm’s capability to generalize large and heterogeneous datasets will likely continue to boost transfer learning in MR brain imaging. Particularly, the need for such large databases will probably motivate the collaborations among research institutions and the development of privacy-preserving methods that avoid data sharing [54]. Transfer learning in MR brain imaging can benefit from domain expertise for investigating brain-specific approaches, such as [36,42]. Therefore, we believe that the current lack of brain imaging-specific methods will motivate the development of such methods. Additionally, the number of recent studies that investigated which CNN layers to fine-tune suggests that there will be more effort put into demonstrating and understanding when and which parameters can be shared across domains.

Supplementary Materials

The following are available online at https://www.mdpi.com/article/10.3390/jimaging7040066/s1. Document S1: PRISMA checklist. Spreadsheet S1: Data gathered during our survey. Figure S1: Contingency matrix between the machine learning methods and the brain disease categories the methods have been applied to.

Author Contributions

Conceptualization, J.T.; methodology, J.M.V. and J.T.; abstract and paper reviews and analyses, J.M.V., V.I., A.A., R.D.F., M.P., R.C. and J.T.; writing—original draft preparation, J.M.V.; writing—review and editing, V.I., A.A., R.D.F., M.P., R.C. and J.T. All authors have read and agreed to the published version of the manuscript.

Funding

The work of J.M.V. Valverde was funded from the European Union’s Horizon 2020 Framework Programme (Marie Skłodowska Curie grant agreement #740264 (GENOMMED)). This work has been supported by the grant #316258 from Academy of Finland and grant S21770 from the European Social Fund.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data presented in this study are available in the Supplementary Materials.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

References

  1. Lerch, J.P.; van der Kouwe, A.J.; Raznahan, A.; Paus, T.; Johansen-Berg, H.; Miller, K.L.; Smith, S.M.; Fischl, B.; Sotiropoulos, S.N. Studying neuroanatomy using MRI. Nat. Neurosci. 2017, 20, 314–326. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  2. Glover, G.H. Overview of functional magnetic resonance imaging. Neurosurg. Clin. 2011, 22, 133–139. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Jones, D.K. Diffusion Mri; Oxford University Press: Oxford, UK, 2010. [Google Scholar]
  4. Menze, B.H.; Jakab, A.; Bauer, S.; Kalpathy-Cramer, J.; Farahani, K.; Kirby, J.; Burren, Y.; Porz, N.; Slotboom, J.; Wiest, R.; et al. The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans. Med. Imaging 2014, 34, 1993–2024. [Google Scholar] [CrossRef]
  5. Cao, X.; Fan, J.; Dong, P.; Ahmad, S.; Yap, P.T.; Shen, D. Image registration using machine and deep learning. In Handbook of Medical Image Computing and Computer Assisted Intervention; Elsevier: Amsterdam, The Netherlands, 2020; pp. 319–342. [Google Scholar]
  6. Falahati, F.; Westman, E.; Simmons, A. Multivariate data analysis and machine learning in Alzheimer’s disease with a focus on structural magnetic resonance imaging. J. Alzheimer’s Dis. 2014, 41, 685–708. [Google Scholar] [CrossRef]
  7. Jovicich, J.; Czanner, S.; Greve, D.; Haley, E.; van Der Kouwe, A.; Gollub, R.; Kennedy, D.; Schmitt, F.; Brown, G.; MacFall, J.; et al. Reliability in multi-site structural MRI studies: Effects of gradient non-linearity correction on phantom and human data. Neuroimage 2006, 30, 436–443. [Google Scholar] [CrossRef]
  8. Chen, J.; Liu, J.; Calhoun, V.D.; Arias-Vasquez, A.; Zwiers, M.P.; Gupta, C.N.; Franke, B.; Turner, J.A. Exploration of scanning effects in multi-site structural MRI studies. J. Neurosci. Methods 2014, 230, 37–50. [Google Scholar] [CrossRef] [Green Version]
  9. Day, O.; Khoshgoftaar, T.M. A survey on heterogeneous transfer learning. J. Big Data 2017, 4, 29. [Google Scholar] [CrossRef]
  10. Woodworth, R.S.; Thorndike, E. The influence of improvement in one mental function upon the efficiency of other functions (I). Psychol. Rev. 1901, 8, 247. [Google Scholar] [CrossRef] [Green Version]
  11. Pan, S.J.; Yang, Q. A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 2009, 22, 1345–1359. [Google Scholar] [CrossRef]
  12. Zhuang, F.; Qi, Z.; Duan, K.; Xi, D.; Zhu, Y.; Zhu, H.; Xiong, H.; He, Q. A comprehensive survey on transfer learning. Proc. IEEE 2020, 109, 43–76. [Google Scholar] [CrossRef]
  13. Cheplygina, V.; de Bruijne, M.; Pluim, J.P. Not-so-supervised: A survey of semi-supervised, multi-instance, and transfer learning in medical image analysis. Med. Image Anal. 2019, 54, 280–296. [Google Scholar] [CrossRef] [Green Version]
  14. Wang, W.; Kiik, M.; Peek, N.; Curcin, V.; Marshall, I.J.; Rudd, A.G.; Wang, Y.; Douiri, A.; Wolfe, C.D.; Bray, B. A systematic review of machine learning models for predicting outcomes of stroke with structured data. PLoS ONE 2020, 15, e0234722. [Google Scholar] [CrossRef]
  15. Orozco-Arias, S.; Isaza, G.; Guyot, R.; Tabares-Soto, R. A systematic review of the application of machine learning in the detection and classification of transposable elements. PeerJ 2019, 7, e8311. [Google Scholar] [CrossRef] [Green Version]
  16. Cheng, D.; Liu, D.; Philpotts, L.L.; Turner, D.P.; Houle, T.T.; Chen, L.; Zhang, M.; Yang, J.; Zhang, W.; Deng, H. Current state of science in machine learning methods for automatic infant pain evaluation using facial expression information: Study protocol of a systematic review and meta-analysis. BMJ Open 2019, 9, e030482. [Google Scholar] [CrossRef] [Green Version]
  17. Ge, L.; Gao, J.; Ngo, H.; Li, K.; Zhang, A. On handling negative transfer and imbalanced distributions in multiple source transfer learning. Stat. Anal. Data Min. ASA Data Sci. J. 2014, 7, 254–271. [Google Scholar] [CrossRef]
  18. Wang, Z.; Dai, Z.; Póczos, B.; Carbonell, J. Characterizing and avoiding negative transfer. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 16–20 June 2019; pp. 11293–11302. [Google Scholar]
  19. Leen, G.; Peltonen, J.; Kaski, S. Focused multi-task learning in a Gaussian process framework. Mach. Learn. 2012, 89, 157–182. [Google Scholar] [CrossRef]
  20. Weninger, L.; Liu, Q.; Merhof, D. Multi-task Learning for Brain Tumor Segmentation. In International MICCAI Brainlesion Workshop; Springer: Berlin/Heidelberg, Germany, 2019; pp. 327–337. [Google Scholar]
  21. Zhou, C.; Ding, C.; Wang, X.; Lu, Z.; Tao, D. One-pass multi-task networks with cross-task guided attention for brain tumor segmentation. IEEE Trans. Image Process. 2020, 29, 4516–4529. [Google Scholar] [CrossRef] [Green Version]
  22. Nalepa, J.; Marcinkiewicz, M.; Kawulok, M. Data augmentation for brain-tumor segmentation: A review. Front. Comput. Neurosci. 2019, 13, 83. [Google Scholar] [CrossRef] [Green Version]
  23. Zhao, A.; Balakrishnan, G.; Durand, F.; Guttag, J.V.; Dalca, A.V. Data augmentation using learned transformations for one-shot medical image segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 16–20 June 2019; pp. 8543–8553. [Google Scholar]
  24. Liu, Q.; Dou, Q.; Heng, P.A. Shape-aware Meta-learning for Generalizing Prostate MRI Segmentation to Unseen Domains. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Lima, Peru, 4–8 October 2020; pp. 475–485. [Google Scholar]
  25. Finn, C.; Abbeel, P.; Levine, S. Model-agnostic meta-learning for fast adaptation of deep networks. arXiv 2017, arXiv:1703.03400. [Google Scholar]
  26. Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G.; Prisma Group. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med. 2009, 6, e1000097. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  27. Sato, I.; Nomura, Y.; Hanaoka, S.; Miki, S.; Hayashi, N.; Abe, O.; Masutani, Y. Managing Computer-Assisted Detection System Based on Transfer Learning with Negative Transfer Inhibition. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, London, UK, 19–23 August 2018; pp. 695–704. [Google Scholar]
  28. Huang, Y.L.; Hsieh, W.T.; Yang, H.C.; Lee, C.C. Conditional Domain Adversarial Transfer for Robust Cross-Site ADHD Classification Using Functional MRI. In Proceedings of the ICASSP 2020—2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain, 4–8 May 2020; pp. 1190–1194. [Google Scholar]
  29. Martyn, P.; McPhilemy, G.; Nabulsi, L.; Martyn, F.; McDonald, C.; Cannon, D.; Schukat, M. Using Magnetic Resonance Imaging to Distinguish a Healthy Brain from a Bipolar Brain: A Transfer Learning Approach; AICS: Galway, Ireland, 6 December 2019. [Google Scholar]
  30. Attallah, O.; Sharkas, M.A.; Gadelkarim, H. Deep Learning Techniques for Automatic Detection of Embryonic Neurodevelopmental Disorders. Diagnostics 2020, 10, 27. [Google Scholar] [CrossRef] [Green Version]
  31. Si, X.; Zhang, X.; Zhou, Y.; Sun, Y.; Jin, W.; Yin, S.; Zhao, X.; Li, Q.; Ming, D. Automated Detection of Juvenile Myoclonic Epilepsy using CNN based Transfer Learning in Diffusion MRI. In Proceedings of the 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Montreal, QC, Canada, 20–24 July 2020; pp. 1679–1682. [Google Scholar]
  32. Chougule, T.; Shinde, S.; Santosh, V.; Saini, J.; Ingalhalikar, M. On Validating Multimodal MRI Based Stratification of IDH Genotype in High Grade Gliomas Using CNNs and Its Comparison to Radiomics. In International Workshop on Radiomics and Radiogenomics in Neuro-Oncology; Springer: Berlin/Heidelberg, Germany, 2019; pp. 53–60. [Google Scholar]
  33. Eitel, F.; Soehler, E.; Bellmann-Strobl, J.; Brandt, A.U.; Ruprecht, K.; Giess, R.M.; Kuchling, J.; Asseyer, S.; Weygandt, M.; Haynes, J.D.; et al. Uncovering convolutional neural network decisions for diagnosing multiple sclerosis on conventional MRI using layer-wise relevance propagation. Neuroimage Clin. 2019, 24, 102003. [Google Scholar] [CrossRef]
  34. Samani, Z.R.; Alappatt, J.A.; Parker, D.; Ismail, A.A.O.; Verma, R. QC-Automator: Deep learning-based automated quality control for diffusion mr images. Front. Neurosci. 2019, 13, 1456. [Google Scholar] [CrossRef] [Green Version]
  35. Dong, Q.; Zhang, J.; Li, Q.; Wang, J.; Leporé, N.; Thompson, P.M.; Caselli, R.J.; Ye, J.; Wang, Y. Alzheimer’s Disease Neuroimaging Initiative. Integrating Convolutional Neural Networks and Multi-Task Dictionary Learning for Cognitive Decline Prediction with Longitudinal Images. J. Alzheimer’s Dis. 2020, 75, 971–992. [Google Scholar] [CrossRef]
  36. Moradi, E.; Khundrakpam, B.; Lewis, J.D.; Evans, A.C.; Tohka, J. Predicting symptom severity in autism spectrum disorder based on cortical thickness measures in agglomerative data. Neuroimage 2017, 144, 128–141. [Google Scholar] [CrossRef] [Green Version]
  37. Huang, S.; Li, J.; Chen, K.; Wu, T.; Ye, J.; Wu, X.; Yao, L. A transfer learning approach for network modeling. IIE Trans. 2012, 44, 915–931. [Google Scholar] [CrossRef] [Green Version]
  38. Hu, L.S.; Yoon, H.; Eschbacher, J.M.; Baxter, L.C.; Dueck, A.C.; Nespodzany, A.; Smith, K.A.; Nakaji, P.; Xu, Y.; Wang, L.; et al. Accurate patient-specific machine learning models of glioblastoma invasion using transfer learning. Am. J. Neuroradiol. 2019, 40, 418–425. [Google Scholar] [CrossRef]
  39. Hermessi, H.; Mourali, O.; Zagrouba, E. Convolutional neural network-based multimodal image fusion via similarity learning in the shearlet domain. Neural Comput. Appl. 2018, 30, 2029–2045. [Google Scholar] [CrossRef]
  40. Naselaris, T.; Kay, K.N.; Nishimoto, S.; Gallant, J.L. Encoding and decoding in fMRI. Neuroimage 2011, 56, 400–410. [Google Scholar] [CrossRef] [Green Version]
  41. Dinsdale, N.K.; Jenkinson, M.; Namburete, A.I. Unlearning Scanner Bias for MRI Harmonisation. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Lima, Peru, 4–8 October 2020; pp. 369–378. [Google Scholar]
  42. Cheng, B.; Liu, M.; Zhang, D.; Munsell, B.C.; Shen, D. Domain transfer learning for MCI conversion prediction. IEEE Trans. Biomed. Eng. 2015, 62, 1805–1817. [Google Scholar] [CrossRef] [Green Version]
  43. Cheng, B.; Liu, M.; Suk, H.I.; Shen, D.; Zhang, D.; Alzheimer’s Disease Neuroimaging Initiative. Multimodal manifold-regularized transfer learning for MCI conversion prediction. Brain Imaging Behav. 2015, 9, 913–926. [Google Scholar] [CrossRef] [Green Version]
  44. Goetz, M.; Weber, C.; Binczyk, F.; Polanska, J.; Tarnawski, R.; Bobek-Billewicz, B.; Koethe, U.; Kleesiek, J.; Stieltjes, B.; Maier-Hein, K.H. DALSA: Domain adaptation for supervised learning from sparsely annotated MR images. IEEE Trans. Med. Imaging 2015, 35, 184–196. [Google Scholar] [CrossRef]
  45. Van Opbroek, A.; Ikram, M.A.; Vernooij, M.W.; De Bruijne, M. Transfer learning improves supervised image segmentation across imaging protocols. IEEE Trans. Med. Imaging 2014, 34, 1018–1030. [Google Scholar] [CrossRef] [Green Version]
  46. Wang, B.; Li, W.; Fan, W.; Chen, X.; Wu, D. Alzheimer’s Disease Brain Network Classification Using Improved Transfer Feature Learning with Joint Distribution Adaptation. In Proceedings of the 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Berlin, Germany, 23–27 July 2019; pp. 2959–2963. [Google Scholar]
  47. Van Opbroek, A.; Ikram, M.A.; Vernooij, M.W.; De Bruijne, M. A transfer-learning approach to image segmentation across scanners by maximizing distribution similarity. In International Workshop on Machine Learning in Medical Imaging; Springer: Berlin/Heidelberg, Germany, 2013; pp. 49–56. [Google Scholar]
  48. Van Opbroek, A.; Vernooij, M.W.; Ikram, M.A.; de Bruijne, M. Weighting training images by maximizing distribution similarity for supervised segmentation across scanners. Med. Image Anal. 2015, 24, 245–254. [Google Scholar] [CrossRef] [PubMed]
  49. Van Opbroek, A.; Achterberg, H.C.; Vernooij, M.W.; De Bruijne, M. Transfer learning for image segmentation by combining image weighting and kernel learning. IEEE Trans. Med. Imaging 2018, 38, 213–224. [Google Scholar] [CrossRef] [Green Version]
  50. Wang, B.; Prastawa, M.; Saha, A.; Awate, S.P.; Irimia, A.; Chambers, M.C.; Vespa, P.M.; Van Horn, J.D.; Pascucci, V.; Gerig, G. Modeling 4D changes in pathological anatomy using domain adaptation: Analysis of TBI imaging using a tumor database. In International Workshop on Multimodal Brain Image Analysis; Springer: Berlin/Heidelberg, Germany, 2013; pp. 31–39. [Google Scholar]
  51. Van Opbroek, A.; Ikram, M.A.; Vernooij, M.W.; de Bruijne, M. Supervised image segmentation across scanner protocols: A transfer learning approach. In International Workshop on Machine Learning in Medical Imaging; Springer: Berlin/Heidelberg, Germany, 2012; pp. 160–167. [Google Scholar]
  52. Tan, X.; Liu, Y.; Li, Y.; Wang, P.; Zeng, X.; Yan, F.; Li, X. Localized instance fusion of MRI data of Alzheimer’s disease for classification based on instance transfer ensemble learning. Biomed. Eng. Online 2018, 17, 49. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  53. Zhou, K.; He, W.; Xu, Y.; Xiong, G.; Cai, J. Feature selection and transfer learning for Alzheimer’s disease clinical diagnosis. Appl. Sci. 2018, 8, 1372. [Google Scholar] [CrossRef] [Green Version]
  54. Li, X.; Gu, Y.; Dvornek, N.; Staib, L.; Ventola, P.; Duncan, J.S. Multi-site fmri analysis using privacy-preserving federated learning and domain adaptation: Abide results. arXiv 2020, arXiv:2001.05647. [Google Scholar] [CrossRef]
  55. Wachinger, C.; Reuter, M.; Alzheimer’s Disease Neuroimaging Initiative. Domain adaptation for Alzheimer’s disease diagnostics. Neuroimage 2016, 139, 470–479. [Google Scholar] [CrossRef] [Green Version]
  56. Ali, M.B.; Gu, I.Y.H.; Berger, M.S.; Pallud, J.; Southwell, D.; Widhalm, G.; Roux, A.; Vecchio, T.G.; Jakola, A.S. Domain Mapping and Deep Learning from Multiple MRI Clinical Datasets for Prediction of Molecular Subtypes in Low Grade Gliomas. Brain Sci. 2020, 10, 463. [Google Scholar] [CrossRef]
  57. Tokuoka, Y.; Suzuki, S.; Sugawara, Y. An Inductive Transfer Learning Approach using Cycle-consistent Adversarial Domain Adaptation with Application to Brain Tumor Segmentation. In Proceedings of the 2019 6th International Conference on Biomedical and Bioinformatics Engineering, Shanghai, China, 13–15 November 2019; pp. 44–48. [Google Scholar]
  58. Li, W.; Zhao, Y.; Chen, X.; Xiao, Y.; Qin, Y. Detecting Alzheimer’s disease on small dataset: A knowledge transfer perspective. IEEE J. Biomed. Health Inform. 2018, 23, 1234–1242. [Google Scholar] [CrossRef]
  59. Wang, B.; Prastawa, M.; Irimia, A.; Saha, A.; Liu, W.; Goh, S.M.; Vespa, P.M.; Van Horn, J.D.; Gerig, G. Modeling 4D pathological changes by leveraging normative models. Comput. Vis. Image Underst. 2016, 151, 3–13. [Google Scholar] [CrossRef] [Green Version]
  60. Van Opbroek, A.; Achterberg, H.C.; de Bruijne, M. Feature-space transformation improves supervised segmentation across scanners. In Medical Learning Meets Medical Imaging; Springer: Berlin/Heidelberg, Germany, 2015; pp. 85–93. [Google Scholar]
  61. Van Opbroek, A.; Achterberg, H.C.; Vernooij, M.W.; Ikram, M.A.; de Bruijne, M.; Alzheimer’s Disease Neuroimaging Initiative. Transfer learning by feature-space transformation: A method for Hippocampus segmentation across scanners. Neuroimage Clin. 2018, 20, 466–475. [Google Scholar] [CrossRef]
  62. Qin, Y.; Li, Y.; Liu, Z.; Ye, C. Knowledge Transfer Between Datasets for Learning-Based Tissue Microstructure Estimation. In Proceedings of the 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), Iowa City, IA, USA, 3–7 April 2020; pp. 1530–1533. [Google Scholar]
  63. Mansoor, A.; Linguraru, M.G. Communal Domain Learning for Registration in Drifted Image Spaces. In International Workshop on Machine Learning in Medical Imaging; Springer: Berlin/Heidelberg, Germany, 2019; pp. 479–488. [Google Scholar]
  64. Zhou, S.; Cox, C.R.; Lu, H. Improving whole-brain neural decoding of fmri with domain adaptation. In International Workshop on Machine Learning in Medical Imaging; Springer: Berlin/Heidelberg, Germany, 2019; pp. 265–273. [Google Scholar]
  65. Van Tulder, G.; de Bruijne, M. Representation learning for cross-modality classification. In Medical Computer Vision and Bayesian and Graphical Models for Biomedical Imaging; Springer: Berlin/Heidelberg, Germany, 2016; pp. 126–136. [Google Scholar]
  66. Gao, Y.; Zhang, Y.; Cao, Z.; Guo, X.; Zhang, J. Decoding Brain States from fMRI Signals by using Unsupervised Domain Adaptation. IEEE J. Biomed. Health Inform. 2019, 24, 1677–1685. [Google Scholar] [CrossRef] [PubMed]
  67. Zhang, J.; Wan, P.; Zhang, D. Transport-Based Joint Distribution Alignment for Multi-site Autism Spectrum Disorder Diagnosis Using Resting-State fMRI. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Lima, Peru, 4–8 October 2020; pp. 444–453. [Google Scholar]
  68. Cheng, B.; Liu, M.; Shen, D.; Li, Z.; Zhang, D.; Alzheimer’s Disease Neuroimaging Initiative. Multi-domain transfer learning for early diagnosis of Alzheimer’s disease. Neuroinformatics 2017, 15, 115–132. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  69. Cheng, B.; Liu, M.; Zhang, D.; Shen, D.; Alzheimer’s Disease Neuroimaging Initiative. Robust multi-label transfer feature learning for early diagnosis of Alzheimer’s disease. Brain Imaging Behav. 2019, 13, 138–153. [Google Scholar] [CrossRef]
  70. Van Tulder, G.; de Bruijne, M. Learning cross-modality representations from multi-modal images. IEEE Trans. Med. Imaging 2018, 38, 638–648. [Google Scholar] [CrossRef]
  71. Li, Z.; Ogino, M. Augmented Radiology: Patient-Wise Feature Transfer Model for Glioma Grading. In Domain Adaptation and Representation Transfer, and Distributed and Collaborative Learning; Springer: Berlin/Heidelberg, Germany, 2020; pp. 23–30. [Google Scholar]
  72. Ackaouy, A.; Courty, N.; Vallée, E.; Commowick, O.; Barillot, C.; Galassi, F. Unsupervised domain adaptation with optimal transport in multi-site segmentation of multiple sclerosis lesions from MRI data. Front. Comput. Neurosci. 2020, 14, 19. [Google Scholar] [CrossRef] [Green Version]
  73. Hofer, C.; Kwitt, R.; Höller, Y.; Trinka, E.; Uhl, A. Simple domain adaptation for cross-dataset analyses of brain MRI data. In Proceedings of the 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017), Melbourne, Australia, 18–21 April 2017; pp. 441–445. [Google Scholar]
  74. Cai, X.L.; Xie, D.J.; Madsen, K.H.; Wang, Y.M.; Bögemann, S.A.; Cheung, E.F.; Møller, A.; Chan, R.C. Generalizability of machine learning for classification of schizophrenia based on resting-state functional MRI data. Hum. Brain Mapp. 2020, 41, 172–184. [Google Scholar] [CrossRef] [Green Version]
  75. Guerrero, R.; Ledig, C.; Rueckert, D. Manifold alignment and transfer learning for classification of Alzheimer’s disease. In International Workshop on Machine Learning in Medical Imaging; Springer: Berlin/Heidelberg, Germany, 2014; pp. 77–84. [Google Scholar]
  76. Wang, M.; Zhang, D.; Huang, J.; Yap, P.T.; Shen, D.; Liu, M. Identifying autism spectrum disorder with multi-site fMRI via low-rank domain adaptation. IEEE Trans. Med. Imaging 2019, 39, 644–655. [Google Scholar] [CrossRef]
  77. Zhang, J.; Liu, M.; Pan, Y.; Shen, D. Unsupervised Conditional Consensus Adversarial Network for Brain Disease Identification with Structural MRI. In International Workshop on Machine Learning in Medical Imaging; Springer: Berlin/Heidelberg, Germany, 2019; pp. 391–399. [Google Scholar]
  78. Shen, Y.; Gao, M. Brain tumor segmentation on MRI with missing modalities. In Proceedings of the International Conference on Information Processing in Medical Imaging, Hong Kong, China, 2–7 June 2019; pp. 417–428. [Google Scholar]
  79. Robinson, R.; Dou, Q.; de Castro, D.C.; Kamnitsas, K.; de Groot, M.; Summers, R.M.; Rueckert, D.; Glocker, B. Image-level Harmonization of Multi-Site Data using Image-and-Spatial Transformer Networks. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Lima, Peru, 4–8 October 2020; pp. 710–719. [Google Scholar]
  80. Guan, H.; Yang, E.; Yap, P.T.; Shen, D.; Liu, M. Attention-Guided Deep Domain Adaptation for Brain Dementia Identification with Multi-site Neuroimaging Data. In Domain Adaptation and Representation Transfer, and Distributed and Collaborative Learning; Springer: Berlin/Heidelberg, Germany, 2020; pp. 31–40. [Google Scholar]
  81. Mahapatra, D.; Ge, Z. Training data independent image registration using generative adversarial networks and domain adaptation. Pattern Recognit. 2020, 100, 107109. [Google Scholar] [CrossRef]
  82. Orbes-Arteaga, M.; Varsavsky, T.; Sudre, C.H.; Eaton-Rosen, Z.; Haddow, L.J.; Sørensen, L.; Nielsen, M.; Pai, A.; Ourselin, S.; Modat, M.; et al. Multi-domain adaptation in brain MRI through paired consistency and adversarial learning. In Domain Adaptation and Representation Transfer and Medical Image Learning with Less Labels and Imperfect Data; Springer: Berlin/Heidelberg, Germany, 2019; pp. 54–62. [Google Scholar]
  83. Kamnitsas, K.; Baumgartner, C.; Ledig, C.; Newcombe, V.; Simpson, J.; Kane, A.; Menon, D.; Nori, A.; Criminisi, A.; Rueckert, D.; et al. Unsupervised domain adaptation in brain lesion segmentation with adversarial networks. In Proceedings of the International Conference on Information Processing in Medical Imaging, Boone, NC, USA, 25–30 June 2017; pp. 597–609. [Google Scholar]
  84. Varsavsky, T.; Orbes-Arteaga, M.; Sudre, C.H.; Graham, M.S.; Nachev, P.; Cardoso, M.J. Test-time unsupervised domain adaptation. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Lima, Peru, 4–8 October 2020; pp. 428–436. [Google Scholar]
  85. Shanis, Z.; Gerber, S.; Gao, M.; Enquobahrie, A. Intramodality Domain Adaptation Using Self Ensembling and Adversarial Training. In Domain Adaptation and Representation Transfer and Medical Image Learning with Less Labels and Imperfect Data; Springer: Berlin/Heidelberg, Germany, 2019; pp. 28–36. [Google Scholar]
  86. Orbes-Arteainst, M.; Cardoso, J.; Sørensen, L.; Igel, C.; Ourselin, S.; Modat, M.; Nielsen, M.; Pai, A. Knowledge distillation for semi-supervised domain adaptation. In OR 2.0 Context-Aware Operating Theaters and Machine Learning in Clinical Neuroimaging; Springer: Berlin/Heidelberg, Germany, 2019; pp. 68–76. [Google Scholar]
  87. Grimm, F.; Edl, F.; Kerscher, S.R.; Nieselt, K.; Gugel, I.; Schuhmann, M.U. Semantic segmentation of cerebrospinal fluid and brain volume with a convolutional neural network in pediatric hydrocephalus—Transfer learning from existing algorithms. Acta Neurochir. 2020, 162, 2463–2474. [Google Scholar] [CrossRef] [PubMed]
  88. Chen, C.L.; Hsu, Y.C.; Yang, L.Y.; Tung, Y.H.; Luo, W.B.; Liu, C.M.; Hwang, T.J.; Hwu, H.G.; Tseng, W.Y.I. Generalization of diffusion magnetic resonance imaging–based brain age prediction model through transfer learning. NeuroImage 2020, 217, 116831. [Google Scholar] [CrossRef] [PubMed]
  89. Oh, K.; Chung, Y.C.; Kim, K.W.; Kim, W.S.; Oh, I.S. Classification and visualization of Alzheimer’s disease using volumetric convolutional neural network and transfer learning. Sci. Rep. 2019, 9, 1–16. [Google Scholar] [CrossRef]
  90. Abrol, A.; Bhattarai, M.; Fedorov, A.; Du, Y.; Plis, S.; Calhoun, V.; Alzheimer’s Disease Neuroimaging Initiative. Deep residual learning for neuroimaging: An application to predict progression to alzheimer’s disease. J. Neurosci. Methods 2020, 339, 108701. [Google Scholar] [CrossRef] [Green Version]
  91. Alex, V.; Vaidhya, K.; Thirunavukkarasu, S.; Kesavadas, C.; Krishnamurthi, G. Semisupervised learning using denoising autoencoders for brain lesion detection and segmentation. J. Med. Imaging 2017, 4, 041311. [Google Scholar] [CrossRef]
  92. Cui, S.; Mao, L.; Jiang, J.; Liu, C.; Xiong, S. Automatic semantic segmentation of brain gliomas from MRI images using a deep cascaded neural network. J. Healthc. Eng. 2018, 2018, 4940593. [Google Scholar] [CrossRef]
  93. Gao, F.; Yoon, H.; Xu, Y.; Goradia, D.; Luo, J.; Wu, T.; Su, Y. Alzheimer’s Disease Neuroimaging Initiative. AD-NET: Age-adjust neural network for improved MCI to AD conversion prediction. Neuroimage Clin. 2020, 27, 102290. [Google Scholar] [CrossRef]
  94. Han, Y.; Yoo, J.; Kim, H.H.; Shin, H.J.; Sung, K.; Ye, J.C. Deep learning with domain adaptation for accelerated projection-reconstruction MR. Magn. Reson. Med. 2018, 80, 1189–1205. [Google Scholar] [CrossRef] [Green Version]
  95. Amin, J.; Sharif, M.; Yasmin, M.; Saba, T.; Anjum, M.A.; Fernandes, S.L. A new approach for brain tumor segmentation and classification based on score level fusion using transfer learning. J. Med. Syst. 2019, 43, 326. [Google Scholar] [CrossRef]
  96. Swati, Z.N.K.; Zhao, Q.; Kabir, M.; Ali, F.; Ali, Z.; Ahmed, S.; Lu, J. Content-based brain tumor retrieval for MR images using transfer learning. IEEE Access 2019, 7, 17809–17822. [Google Scholar] [CrossRef]
  97. Xu, Y.; Géraud, T.; Bloch, I. From neonatal to adult brain MR image segmentation in a few seconds using 3D-like fully convolutional network and transfer learning. In Proceedings of the 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China, 17–20 September 2017; pp. 4417–4421. [Google Scholar]
  98. Ladefoged, C.N.; Hansen, A.E.; Henriksen, O.M.; Bruun, F.J.; Eikenes, L.; Øen, S.K.; Karlberg, A.; Højgaard, L.; Law, I.; Andersen, F.L. AI-driven attenuation correction for brain PET/MRI: Clinical evaluation of a dementia cohort and importance of the training group size. NeuroImage 2020, 222, 117221. [Google Scholar] [CrossRef] [PubMed]
  99. Dar, S.U.H.; Özbey, M.; Çatlı, A.B.; Çukur, T. A Transfer-Learning Approach for Accelerated MRI Using Deep Neural Networks. Magn. Reson. Med. 2020, 84, 663–685. [Google Scholar] [CrossRef] [Green Version]
  100. Aderghal, K.; Khvostikov, A.; Krylov, A.; Benois-Pineau, J.; Afdel, K.; Catheline, G. Classification of Alzheimer disease on imaging modalities with deep CNNs using cross-modal transfer learning. In Proceedings of the 2018 IEEE 31st International Symposium on Computer-Based Medical Systems (CBMS), Karlstad, Sweden, 18–21 June 2018; pp. 345–350. [Google Scholar]
  101. Bashyam, V.M.; Erus, G.; Doshi, J.; Habes, M.; Nasralah, I.; Truelove-Hill, M.; Srinivasan, D.; Mamourian, L.; Pomponio, R.; Fan, Y.; et al. MRI signatures of brain age and disease over the lifespan based on a deep brain network and 14 468 individuals worldwide. Brain 2020, 143, 2312–2324. [Google Scholar] [CrossRef]
  102. Xu, Y.; Géraud, T.; Puybareau, É.; Bloch, I.; Chazalon, J. White matter hyperintensities segmentation in a few seconds using fully convolutional network and transfer learning. In International MICCAI Brainlesion Workshop; Springer: Berlin/Heidelberg, Germany, 2017; pp. 501–514. [Google Scholar]
  103. Han, X. MR-based synthetic CT generation using a deep convolutional neural network method. Med. Phys. 2017, 44, 1408–1419. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  104. Deepak, S.; Ameer, P. Retrieval of brain MRI with tumor using contrastive loss based similarity on GoogLeNet encodings. Comput. Biol. Med. 2020, 125, 103993. [Google Scholar] [CrossRef]
  105. Li, H.; Parikh, N.A.; He, L. A novel transfer learning approach to enhance deep neural network classification of brain functional connectomes. Front. Neurosci. 2018, 12, 491. [Google Scholar] [CrossRef]
  106. Alkassar, S.; Abdullah, M.A.; Jebur, B.A. Automatic Brain Tumour Segmentation using fully Convolution Network and Transfer Learning. In Proceedings of the 2019 2nd International Conference on Electrical, Communication, Computer, Power and Control Engineering (ICECCPCE), Mosul, Iraq, 13–14 February 2019; pp. 188–192. [Google Scholar]
  107. Afridi, M.J.; Ross, A.; Shapiro, E.M. L-CNN: Exploiting labeling latency in a cnn learning framework. In Proceedings of the 2016 23rd International Conference on Pattern Recognition (ICPR), Cancun, Mexico, 4–8 December 2016; pp. 2156–2161. [Google Scholar]
  108. Chen, K.T.; Schürer, M.; Ouyang, J.; Koran, M.E.I.; Davidzon, G.; Mormino, E.; Tiepolt, S.; Hoffmann, K.T.; Sabri, O.; Zaharchuk, G.; et al. Generalization of deep learning models for ultra-low-count amyloid PET/MRI using transfer learning. Eur. J. Nucl. Med. Mol. Imaging 2020, 47, 2998–3007. [Google Scholar] [CrossRef]
  109. Mahmood, U.; Rahman, M.M.; Fedorov, A.; Lewis, N.; Fu, Z.; Calhoun, V.D.; Plis, S.M. Whole MILC: Generalizing learned dynamics across tasks, datasets, and populations. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Lima, Peru, 4–8 October 2020; pp. 407–417. [Google Scholar]
  110. Yang, Y.; Yan, L.F.; Zhang, X.; Han, Y.; Nan, H.Y.; Hu, Y.C.; Hu, B.; Yan, S.L.; Zhang, J.; Cheng, D.L.; et al. Glioma grading on conventional MR images: A deep learning study with transfer learning. Front. Neurosci. 2018, 12, 804. [Google Scholar] [CrossRef] [Green Version]
  111. Pravitasari, A.A.; Iriawan, N.; Almuhayar, M.; Azmi, T.; Fithriasari, K.; Purnami, S.W.; Ferriastuti, W. UNet-VGG16 with transfer learning for MRI-based brain tumor segmentation. Telkomnika 2020, 18, 1310–1318. [Google Scholar] [CrossRef]
  112. Tandel, G.S.; Balestrieri, A.; Jujaray, T.; Khanna, N.N.; Saba, L.; Suri, J.S. Multiclass magnetic resonance imaging brain tumor classification using artificial intelligence paradigm. Comput. Biol. Med. 2020, 122, 103804. [Google Scholar] [CrossRef]
  113. Zhao, X.; Zhang, H.; Zhou, Y.; Bian, W.; Zhang, T.; Zou, X. Gibbs-ringing artifact suppression with knowledge transfer from natural images to MR images. Multimed. Tools Appl. 2020, 79, 33711–33733. [Google Scholar] [CrossRef]
  114. Coupé, P.; Mansencal, B.; Clément, M.; Giraud, R.; de Senneville, B.D.; Ta, V.T.; Lepetit, V.; Manjon, J.V. AssemblyNet: A novel deep decision-making process for whole brain MRI segmentation. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Shenzhen, China, 13–17 October 2019; pp. 466–474. [Google Scholar]
  115. Zhou, Z.; Sodha, V.; Siddiquee, M.M.R.; Feng, R.; Tajbakhsh, N.; Gotway, M.B.; Liang, J. Models genesis: Generic autodidactic models for 3d medical image analysis. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Shenzhen, China, 13–17 October 2019; pp. 384–393. [Google Scholar]
  116. Tao, X.; Li, Y.; Zhou, W.; Ma, K.; Zheng, Y. Revisiting Rubik’s cube: Self-supervised learning with volume-wise transformation for 3D medical image segmentation. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Lima, Peru, 4–8 October 2020; pp. 238–248. [Google Scholar]
  117. Liu, Y.; Pan, Y.; Yang, W.; Ning, Z.; Yue, L.; Liu, M.; Shen, D. Joint Neuroimage Synthesis and Representation Learning for Conversion Prediction of Subjective Cognitive Decline. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Lima, Peru, 4–8 October 2020; pp. 583–592. [Google Scholar]
  118. Ataloglou, D.; Dimou, A.; Zarpalas, D.; Daras, P. Fast and precise hippocampus segmentation through deep convolutional neural network ensembles and transfer learning. Neuroinformatics 2019, 17, 563–582. [Google Scholar] [CrossRef]
  119. Afridi, M.J.; Ross, A.; Shapiro, E.M. On automated source selection for transfer learning in convolutional neural networks. Pattern Recognit. 2018, 73, 65–75. [Google Scholar] [CrossRef]
  120. Kouw, W.M.; Ørting, S.N.; Petersen, J.; Pedersen, K.S.; de Bruijne, M. A cross-center smoothness prior for variational Bayesian brain tissue segmentation. In Proceedings of the International Conference on Information Processing in Medical Imaging, Hong Kong, China, 2–7 June 2019; pp. 360–371. [Google Scholar]
  121. Kuzina, A.; Egorov, E.; Burnaev, E. Bayesian generative models for knowledge transfer in mri semantic segmentation problems. Front. Neurosci. 2019, 13, 844. [Google Scholar] [CrossRef] [Green Version]
  122. Wee, C.Y.; Liu, C.; Lee, A.; Poh, J.S.; Ji, H.; Qiu, A.; Alzheimer’s Disease Neuroimaging Initiative. Cortical graph neural network for AD and MCI diagnosis and transfer learning across populations. Neuroimage Clin. 2019, 23, 101929. [Google Scholar] [CrossRef]
  123. Fei, X.; Wang, J.; Ying, S.; Hu, Z.; Shi, J. Projective parameter transfer based sparse multiple empirical kernel learning Machine for diagnosis of brain disease. Neurocomputing 2020, 413, 271–283. [Google Scholar] [CrossRef]
  124. Velioglu, B.; Vural, F.T.Y. Transfer learning for brain decoding using deep architectures. In Proceedings of the 2017 IEEE 16th International Conference on Cognitive Informatics & Cognitive Computing (ICCI* CC), Oxford, UK, 26–28 July 2017; pp. 65–70. [Google Scholar]
  125. Li, W.; Zhang, L.; Qiao, L.; Shen, D. Toward a Better Estimation of Functional Brain Network for Mild Cognitive Impairment Identification: A Transfer Learning View. IEEE J. Biomed. Health Inform. 2019, 24, 1160–1168. [Google Scholar] [CrossRef] [PubMed]
  126. Deng, J.; Dong, W.; Socher, R.; Li, L.J.; Li, K.; Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 20–25 June 2009; pp. 248–255. [Google Scholar]
  127. Jónsson, B.A.; Bjornsdottir, G.; Thorgeirsson, T.; Ellingsen, L.M.; Walters, G.B.; Gudbjartsson, D.; Stefansson, H.; Stefansson, K.; Ulfarsson, M. Brain age prediction using deep learning uncovers associated sequence variants. Nat. Commun. 2019, 10, 1–10. [Google Scholar] [CrossRef] [Green Version]
  128. Ghafoorian, M.; Mehrtash, A.; Kapur, T.; Karssemeijer, N.; Marchiori, E.; Pesteie, M.; Guttmann, C.R.; de Leeuw, F.E.; Tempany, C.M.; Van Ginneken, B.; et al. Transfer learning for domain adaptation in mri: Application in brain lesion segmentation. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Quebec City, QC, Canada, 11–13 September 2017; pp. 516–524. [Google Scholar]
  129. Valverde, S.; Salem, M.; Cabezas, M.; Pareto, D.; Vilanova, J.C.; Ramió-Torrentà, L.; Rovira, À.; Salvi, J.; Oliver, A.; Lladó, X. One-shot domain adaptation in multiple sclerosis lesion segmentation using convolutional neural networks. NeuroImage Clin. 2019, 21, 101638. [Google Scholar] [CrossRef]
  130. Shirokikh, B.; Zakazov, I.; Chernyavskiy, A.; Fedulova, I.; Belyaev, M. First U-Net Layers Contain More Domain Specific Information Than The Last Ones. In Domain Adaptation and Representation Transfer, and Distributed and Collaborative Learning; Springer: Berlin/Heidelberg, Germany, 2020; pp. 117–126. [Google Scholar]
  131. Gao, Y.; Zhang, Y.; Wang, H.; Guo, X.; Zhang, J. Decoding Behavior Tasks From Brain Activity Using Deep Transfer Learning. IEEE Access 2019, 7, 43222–43232. [Google Scholar] [CrossRef]
  132. Naser, M.A.; Deen, M.J. Brain tumor segmentation and grading of lower-grade glioma using deep learning in MRI images. Comput. Biol. Med. 2020, 121, 103758. [Google Scholar] [CrossRef]
  133. Vakli, P.; Deák-Meszlényi, R.J.; Hermann, P.; Vidnyánszky, Z. Transfer learning improves resting-state functional connectivity pattern analysis using convolutional neural networks. GigaScience 2018, 7, giy130. [Google Scholar] [CrossRef]
  134. Jiang, H.; Guo, J.; Du, H.; Xu, J.; Qiu, B. Transfer learning on T1-weighted images for brain age estimation. Math. Biosci. Eng. 2019, 16, 4382–4398. [Google Scholar] [CrossRef]
  135. Kushibar, K.; Valverde, S.; González-Villà, S.; Bernal, J.; Cabezas, M.; Oliver, A.; Lladó, X. Supervised domain adaptation for automatic sub-cortical brain structure segmentation with minimal user interaction. Sci. Rep. 2019, 9, 1–15. [Google Scholar] [CrossRef] [Green Version]
  136. Kollia, I.; Stafylopatis, A.G.; Kollias, S. Predicting Parkinson’s disease using latent information extracted from deep neural networks. In Proceedings of the 2019 International Joint Conference on Neural Networks (IJCNN), Budapest, Hungary, 14–19 July 2019; pp. 1–8. [Google Scholar]
  137. Menikdiwela, M.; Nguyen, C.; Shaw, M. Deep Learning on Brain Cortical Thickness Data for Disease Classification. In Proceedings of the 2018 Digital Image Computing: Techniques and Applications (DICTA), Canberra, Australia, 10–13 December 2018; pp. 1–5. [Google Scholar]
  138. Kaur, B.; Lemaître, P.; Mehta, R.; Sepahvand, N.M.; Precup, D.; Arnold, D.; Arbel, T. Improving Pathological Structure Segmentation via Transfer Learning Across Diseases. In Domain Adaptation and Representation Transfer and Medical Image Learning with Less Labels and Imperfect Data; Springer: Berlin/Heidelberg, Germany, 2019; pp. 90–98. [Google Scholar]
  139. Liu, R.; Hall, L.O.; Goldgof, D.B.; Zhou, M.; Gatenby, R.A.; Ahmed, K.B. Exploring deep features from brain tumor magnetic resonance images via transfer learning. In Proceedings of the 2016 International Joint Conference on Neural Networks (IJCNN), Vancouver, BC, Canada, 24–29 July 2016; pp. 235–242. [Google Scholar]
  140. Stawiaski, J. A pretrained densenet encoder for brain tumor segmentation. In International MICCAI Brainlesion Workshop; Springer: Berlin/Heidelberg, Germany, 2018; pp. 105–115. [Google Scholar]
  141. Mahapatra, D.; Ge, Z. Training data independent image registration with gans using transfer learning and segmentation information. In Proceedings of the 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), Venice, Italy, 8–11 April 2019; pp. 709–713. [Google Scholar]
  142. Wang, L.; Li, S.; Meng, M.; Chen, G.; Zhu, M.; Bian, Z.; Lyu, Q.; Zeng, D.; Ma, J. Task-oriented Deep Network for Ischemic Stroke Segmentation in Unenhanced CT Imaging. In Proceedings of the 2019 IEEE Nuclear Science Symposium and Medical Imaging Conference (NSS/MIC), Manchester, UK, 26 October–2 November 2019; pp. 1–3. [Google Scholar]
  143. Wang, S.; Shen, Y.; Chen, W.; Xiao, T.; Hu, J. Automatic recognition of mild cognitive impairment from mri images using expedited convolutional neural networks. In Proceedings of the International Conference on Artificial Neural Networks, Alghero, Italy, 11–15 September 2017; pp. 373–380. [Google Scholar]
  144. Guy-Fernand, K.N.; Zhao, J.; Sabuni, F.M.; Wang, J. Classification of Brain Tumor Leveraging Goal-Driven Visual Attention with the Support of Transfer Learning. In Proceedings of the 2020 Information Communication Technologies Conference (ICTC), Nanjing, China, 29–31 May 2020; pp. 328–332. [Google Scholar]
  145. Khan, N.M.; Abraham, N.; Hon, M. Transfer learning with intelligent training data selection for prediction of Alzheimer’s disease. IEEE Access 2019, 7, 72726–72735. [Google Scholar] [CrossRef]
  146. Tufail, A.B.; Ma, Y.; Zhang, Q.N. Multiclass classification of initial stages of Alzheimer’s Disease through Neuroimaging modalities and Convolutional Neural Networks. In Proceedings of the 2020 IEEE 5th Information Technology and Mechatronics Engineering Conference (ITOEC), Chongqing, China, 12–14 June 2020; pp. 51–56. [Google Scholar]
  147. Castro, A.P.; Fernandez-Blanco, E.; Pazos, A.; Munteanu, C.R. Automatic assessment of Alzheimer’s disease diagnosis based on deep learning techniques. Comput. Biol. Med. 2020, 120, 103764. [Google Scholar] [CrossRef]
  148. Bodapati, J.D.; Vijay, A.; Veeranjaneyulu, N. Brain tumor detection using deep features in the latent space. J. Homepage 2020, 25, 259–265. [Google Scholar] [CrossRef]
  149. Kang, L.; Jiang, J.; Huang, J.; Zhang, T. Identifying early mild cognitive impairment by multi-modality mri-based deep learning. Front. Aging Neurosci. 2020, 12, 206. [Google Scholar] [CrossRef]
  150. Thomas, A.W.; Müller, K.R.; Samek, W. Deep transfer learning for whole-brain FMRI analyses. In OR 2.0 Context-Aware Operating Theaters and Machine Learning in Clinical Neuroimaging; Springer: Berlin/Heidelberg, Germany, 2019; pp. 59–67. [Google Scholar]
  151. Ebrahimi-Ghahnavieh, A.; Luo, S.; Chiong, R. Transfer Learning for Alzheimer’s Disease Detection on MRI Images. In Proceedings of the 2019 IEEE International Conference on Industry 4.0, Artificial Intelligence, and Communications Technology (IAICT), Bali, Indonesia, 1–3 July 2019; pp. 133–138. [Google Scholar]
  152. Zheng, J.; Xia, K.; Zheng, Q.; Qian, P. A smart brain MR image completion method guided by synthetic-CT-based multimodal registration. J. Ambient. Intell. Humaniz. Comput. 2019. [Google Scholar] [CrossRef]
  153. Svanera, M.; Savardi, M.; Benini, S.; Signoroni, A.; Raz, G.; Hendler, T.; Muckli, L.; Goebel, R.; Valente, G. Transfer learning of deep neural network representations for fMRI decoding. J. Neurosci. Methods 2019, 328, 108319. [Google Scholar] [CrossRef] [PubMed]
  154. Ren, Y.; Luo, Q.; Gong, W.; Lu, W. Transfer Learning Models on Brain Age Prediction. In Proceedings of the Third International Symposium on Image Computing and Digital Medicine, Xi’an, China, 24–26 August 2019; pp. 278–282. [Google Scholar]
  155. Han, W.; Qin, L.; Bay, C.; Chen, X.; Yu, K.H.; Miskin, N.; Li, A.; Xu, X.; Young, G. Deep Transfer Learning and Radiomics Feature Prediction of Survival of Patients with High-Grade Gliomas. Am. J. Neuroradiol. 2020, 41, 40–48. [Google Scholar] [CrossRef] [PubMed]
  156. He, Y.; Carass, A.; Zuo, L.; Dewey, B.E.; Prince, J.L. Self domain adapted network. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Lima, Peru, 4–8 October 2020; pp. 437–446. [Google Scholar]
  157. Yang, Y.; Li, X.; Wang, P.; Xia, Y.; Ye, Q. Multi-Source Transfer Learning via Ensemble Approach for Initial Diagnosis of Alzheimer’s Disease. IEEE J. Transl. Eng. Health Med. 2020, 8, 1–10. [Google Scholar] [CrossRef]
  158. Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv 2014, arXiv:1409.1556. [Google Scholar]
  159. He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27 June–1 July 2016; pp. 770–778. [Google Scholar]
  160. Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; Rabinovich, A. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 1–9. [Google Scholar]
  161. Szegedy, C.; Vanhoucke, V.; Ioffe, S.; Shlens, J.; Wojna, Z. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–1 July 2016; pp. 2818–2826. [Google Scholar]
  162. Szegedy, C.; Ioffe, S.; Vanhoucke, V.; Alemi, A. Inception-v4, inception-resnet and the impact of residual connections on learning. In Proceedings of the AAAI Conference on Artificial Intelligence, San Francisco, CA, USA, 4–9 February 2017; Volume 31. [Google Scholar]
  163. Dwork, C.; Roth, A. The algorithmic foundations of differential privacy. Found. Trends Theor. Comput. Sci. 2014, 9, 211–407. [Google Scholar] [CrossRef]
  164. Petersen, R.C.; Aisen, P.; Beckett, L.A.; Donohue, M.; Gamst, A.; Harvey, D.J.; Jack, C.; Jagust, W.; Shaw, L.; Toga, A.; et al. Alzheimer’s disease neuroimaging initiative (ADNI): Clinical characterization. Neurology 2010, 74, 201–209. [Google Scholar] [CrossRef] [Green Version]
  165. Bakas, S.; Akbari, H.; Sotiras, A.; Bilello, M.; Rozycki, M.; Kirby, J.S.; Freymann, J.B.; Farahani, K.; Davatzikos, C. Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Sci. Data 2017, 4, 1–13. [Google Scholar] [CrossRef] [Green Version]
  166. Bakas, S.; Reyes, M.; Jakab, A.; Bauer, S.; Rempfler, M.; Crimi, A.; Shinohara, R.T.; Berger, C.; Ha, S.M.; Rozycki, M.; et al. Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the BRATS challenge. arXiv 2018, arXiv:1811.02629. [Google Scholar]
  167. Di Martino, A.; Yan, C.G.; Li, Q.; Denio, E.; Castellanos, F.X.; Alaerts, K.; Anderson, J.S.; Assaf, M.; Bookheimer, S.Y.; Dapretto, M.; et al. The autism brain imaging data exchange: Towards a large-scale evaluation of the intrinsic brain architecture in autism. Mol. Psychiatry 2014, 19, 659–667. [Google Scholar] [CrossRef]
  168. Di Martino, A.; O’connor, D.; Chen, B.; Alaerts, K.; Anderson, J.S.; Assaf, M.; Balsters, J.H.; Baxter, L.; Beggiato, A.; Bernaerts, S.; et al. Enhancing studies of the connectome in autism using the autism brain imaging data exchange II. Sci. Data 2017, 4, 1–15. [Google Scholar] [CrossRef] [Green Version]
  169. Van Essen, D.C.; Smith, S.M.; Barch, D.M.; Behrens, T.E.; Yacoub, E.; Ugurbil, K.; Consortium, W.M.H. The WU-Minn human connectome project: An overview. Neuroimage 2013, 80, 62–79. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  170. Abrol, A.; Fu, Z.; Salman, M.; Silva, R.; Du, Y.; Plis, S.; Calhoun, V. Deep learning encodes robust discriminative neuroimaging representations to outperform standard machine learning. Nat. Commun. 2021, 12, 1–17. [Google Scholar] [CrossRef]
  171. Krizhevsky, A.; Sutskever, I.; Hinton, G.E. Imagenet classification with deep convolutional neural networks. Commun. ACM 2017, 60, 84–90. [Google Scholar] [CrossRef]
  172. Bengio, Y.; Louradour, J.; Collobert, R.; Weston, J. Curriculum learning. In Proceedings of the 26th Annual International Conference on Machine Learning, Montreal, QC, Canada, 14–18 June 2009; pp. 41–48. [Google Scholar]
  173. Maaten, L.v.d.; Hinton, G. Visualizing data using t-SNE. J. Mach. Learn. Res. 2008, 9, 2579–2605. [Google Scholar]
  174. Shin, H.C.; Roth, H.R.; Gao, M.; Lu, L.; Xu, Z.; Nogues, I.; Yao, J.; Mollura, D.; Summers, R.M. Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans. Med. Imaging 2016, 35, 1285–1298. [Google Scholar] [CrossRef] [Green Version]
Figure 1. Flowchart of the screening process.
Figure 1. Flowchart of the screening process.
Jimaging 07 00066 g001
Figure 2. (Left): Number of articles according to the ICD-11 category. (Right): Number of articles per MRI modality.
Figure 2. (Left): Number of articles according to the ICD-11 category. (Right): Number of articles per MRI modality.
Jimaging 07 00066 g002
Figure 3. (Left): Number of articles according to the ML method studied (methods enumerated in Section 4.2). (Right): Distribution of articles according to the publication year.
Figure 3. (Left): Number of articles according to the ML method studied (methods enumerated in Section 4.2). (Right): Distribution of articles according to the publication year.
Jimaging 07 00066 g003
Table 1. Types of transfer learning and examples in MR brain imaging. ∼ indicates “different but related”. The subscripts S and T indicate source and target, respectively.
Table 1. Types of transfer learning and examples in MR brain imaging. ∼ indicates “different but related”. The subscripts S and T indicate source and target, respectively.
TypePropertiesApproach Example
Unsupervised D S D T , T S = T T Transforming T1- and T2-weighted images into the same feature space with adversarial training.
Transductive D S D T , T S = T T Learning a feature mapping from T1- to T2-weighted images while optimizing to segment tumors in T2-weighted images.
Inductive D S D T , T S T T Optimizing a classifier on a natural images dataset, and fine-tuning certain parameters for tumor segmentation.
D S D T , T S = T T Optimizing a lesion segmentation algorithm in T2-weighted images, and re-optimizing certain parameters on FLAIR images.
D S = D T , T S T T Optimizing a lesion segmentation algorithm in T2-weighted images, and re-optimizing certain parameters in the same images for anatomical segmentation.
Table 2. Search terms in Scopus and Springer.
Table 2. Search terms in Scopus and Springer.
ScopusSpringer Website
(TITLE-ABS-KEY ((“transfer learning” OR “knowledge transfer” OR “domain adaptation”) AND (mri OR “magnetic resonance” OR “diffusion imaging” OR “diffusion weighted imaging” OR “arterial spin labeling” OR “susceptibility mapping” OR bold OR “blood oxygenation level dependent” OR “blood oxygen level dependent”)))(“transfer learning” OR “knowledge transfer” OR “domain adaptation”) AND (mri OR “magnetic resonance” OR “diffusion imaging” OR “diffusion weighted imaging” OR “arterial spin labeling” OR “susceptibility mapping” OR “T1” OR “T2”)
Table 3. Tasks and applications in the surveyed papers.
Table 3. Tasks and applications in the surveyed papers.
Task (Total)% of StudiesApplication
Classification (68)52.71%Alzheimer’s diagnostics/prognostics (31), Tumor (10), fMRI decoding (6), Autism spectrum disorder (5), Injected cells (2), Parkinson (2), Schizophrenia (2), Sex (2), Aneurysm [27], Attention deficit hyperactivity disorder [28], Bipolar disorder [29], Embryonic neurodevelopmental disorders [30], Epilepsy [31], IDH mutation [32], Multiple sclerosis [33], Quality control [34]
Segmentation (45)34.88%Tumor (16), Anatomical (15), Lesion (14)
Regression (12)9.30%Age (8), Alzheimer’s disease progression [35], Autism symptom severity [36], Brain connectivity in Alzheimer’s disease [37], Tumor cell density [38]
Others (15)11.63%Reconstruction (5), Registration (4), Image translation (3), CBIR (2), Image fusion [39]
Table 4. Transfer learning strategies categorization. Bold font highlights the proposed categories.
Table 4. Transfer learning strategies categorization. Bold font highlights the proposed categories.
Type% of StudiesSubtypeSubsubtypeApproaches
Instance (16)11.63%Fixed 6 (4.65%)
OptimizedUnsupervised5 (3.88%)
Supervised5 (3.88%)
Feature (38)29.46%Asymmetric 7 (5.43%)
SymmetricDirect17 (13.18%)
Indirect14 (10.85%)
Parameter (87)65.89%Prior sharing 50 (38.76%)
Parameter sharingOne model21 (16.28%)
Multiple16 (12.40%)
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Valverde, J.M.; Imani, V.; Abdollahzadeh, A.; De Feo, R.; Prakash, M.; Ciszek, R.; Tohka, J. Transfer Learning in Magnetic Resonance Brain Imaging: A Systematic Review. J. Imaging 2021, 7, 66. https://doi.org/10.3390/jimaging7040066

AMA Style

Valverde JM, Imani V, Abdollahzadeh A, De Feo R, Prakash M, Ciszek R, Tohka J. Transfer Learning in Magnetic Resonance Brain Imaging: A Systematic Review. Journal of Imaging. 2021; 7(4):66. https://doi.org/10.3390/jimaging7040066

Chicago/Turabian Style

Valverde, Juan Miguel, Vandad Imani, Ali Abdollahzadeh, Riccardo De Feo, Mithilesh Prakash, Robert Ciszek, and Jussi Tohka. 2021. "Transfer Learning in Magnetic Resonance Brain Imaging: A Systematic Review" Journal of Imaging 7, no. 4: 66. https://doi.org/10.3390/jimaging7040066

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop