Submit to Applied Sciences Review for Applied Sciences Propose a Special Issue

Journal Browser

Federated and Transfer Learning Applications

Print Special Issue Flyer
Special Issue Editors
Special Issue Information
Keywords
Published Papers

A special issue of Applied Sciences (ISSN 2076-3417). This special issue belongs to the section "Computing and Artificial Intelligence".

Deadline for manuscript submissions: closed (20 March 2023) | Viewed by 29682

Printed Edition Available!
A printed edition of this Special Issue is available here.

Special issue Federated and Transfer Learning Applications book cover image

Share This Special Issue

Special Issue Editors

Dr. George Drosatos

E-Mail Website
Guest Editor

Institute for Language and Speech Processing, Athena Research Centre, 67100 Xanthi, Greece
Interests: privacy-enhancing technologies (PETs); information security; distributed ledger technologies (DLTs); biomedical informatics; federated learning; transfer learning
Special Issues, Collections and Topics in MDPI journals

Prof. Dr. Pavlos S. Efraimidis

E-Mail Website
Guest Editor

Department of Electrical and Computer Engineering, Democritus University of Thrace, 67100 Xanthi, Greece
Interests: algorithms; social network analysis; federated learning; algorithmic aspects of privacy; algorithmic game theory

Prof. Dr. Avi Arampatzis

E-Mail Website
Guest Editor

Department of Electrical and Computer Engineering, Democritus University of Thrace, 67100 Xanthi, Greece
Interests: information retrieval; databases; data science; machine learning

Special Issue Information

Dear Colleagues,

The classic example of machine learning is based on isolated learning—a single model for each task using a single dataset. Most deep learning methods require a significant amount of labeled data, preventing their applicability in many areas where there is a shortage. In these cases, the ability of models to leverage information from unlabeled data or data that is not publicly available (for privacy and security reasons) can offer a remarkable alternative. Transfer learning and federated learning are such alternative approaches that have emerged in recent years. More precisely, transfer learning is defined as the set of methods that leverage data from additional fields or tasks to train a model with greater generalizability and usually use a smaller amount of labeled data (via fine-tuning) to make them more specific for dedicated tasks. Accordingly, federated learning is a learning model that seeks to address the problem of data management and privacy through joint training with this data, without the need to transfer the data to a central entity.

In this Special Issue, we seek research and case studies that demonstrate the application of federated and transfer learning approaches to support applied scientific research, in any area of science and technology. Example topics include (but are not limited to) the following:

Federated Learning (FL) Applications.
Distributed Learning Approaches.
Privacy-Preserving Techniques in FL.
Homomorphic Encryption Approaches in FL.
Differential Privacy Approaches in FL.
Incentive Mechanisms in FL.
Interpretability in FL.
FL with Unbalanced Data.
Selection of Appropriate FL Aggregation Function per Application.
Transfer Learning (TL) Applications.
Pre-Trained Models.
BERT-Like Models.
Federated Transfer Learning Approaches.
Applications of FL and TL in Biomedical Domain.
Applications of FL and TL in Cybersecurity.
Applications of FL and TL in Natural Language Processing.
Applications of FL and TL in Social Network Analysis.
Graph-based FL and TL.

Dr. George Drosatos
Prof. Dr. Pavlos S. Efraimidis
Prof. Dr. Avi Arampatzis
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Applied Sciences is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2400 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

federated learning
transfer learning
artificial intelligence
privacy
pre-trained models
BERT-like modes
distributed learning
homomorphic encryption
differential privacy

Published Papers (12 papers)

Download All Papers

Editorial

Jump to: Research

5 pages, 275 KiB

Open AccessEditorial

Federated and Transfer Learning Applications

by George Drosatos, Pavlos S. Efraimidis and Avi Arampatzis

Appl. Sci. 2023, 13(21), 11722; https://doi.org/10.3390/app132111722 - 26 Oct 2023

Viewed by 719

Abstract

The classic example of machine learning is based on isolated learning—a single model for each task using a single dataset [...] Full article

(This article belongs to the Special Issue Federated and Transfer Learning Applications)

► Show Figures

Figure 1

Research

Jump to: Editorial

23 pages, 7447 KiB

Open AccessArticle

Leveraging Dialogue State Tracking for Zero-Shot Chat-Based Social Engineering Attack Recognition

by Nikolaos Tsinganos, Panagiotis Fouliras and Ioannis Mavridis

Appl. Sci. 2023, 13(8), 5110; https://doi.org/10.3390/app13085110 - 19 Apr 2023

Cited by 6 | Viewed by 1562

Abstract

Human-to-human dialogues constitute an essential research area for linguists, serving as a conduit for knowledge transfer in the study of dialogue systems featuring human-to-machine interaction. Dialogue systems have garnered significant acclaim and rapid growth owing to their deployment in applications such as virtual assistants (e.g., Alexa, Siri, etc.) and chatbots. Novel modeling techniques are being developed to enhance natural language understanding, natural language generation, and dialogue-state tracking. In this study, we leverage the terminology and techniques of dialogue systems to model human-to-human dialogues within the context of chat-based social engineering (CSE) attacks. The ability to discern an interlocutor’s true intent is crucial for providing an effective real-time defense mechanism against CSE attacks. We introduce in-context dialogue acts that expose an interlocutor’s intent, as well as the requested information that she sought to convey, thereby facilitating real-time recognition of CSE attacks. Our work proposes CSE domain-specific dialogue acts, utilizing a carefully crafted ontology, and creates an annotated corpus using dialogue acts as classification labels. Furthermore, we propose SG-CSE BERT, a BERT-based model following the schema-guided paradigm, for zero-shot CSE attack dialogue-state tracking. Our evaluation results demonstrate satisfactory performance. Full article

(This article belongs to the Special Issue Federated and Transfer Learning Applications)

► Show Figures

Figure 1

14 pages, 7581 KiB

Open AccessArticle

Wireless Traffic Prediction Based on a Gradient Similarity Federated Aggregation Algorithm

by Luzhi Li, Yuhong Zhao, Jingyu Wang and Chuanting Zhang

Appl. Sci. 2023, 13(6), 4036; https://doi.org/10.3390/app13064036 - 22 Mar 2023

Cited by 3 | Viewed by 1273

Abstract

Wireless traffic prediction is critical to the intelligent operation of cellular networks, such as load balancing, congestion control, value-added service promotion, etc. However, the BTS data in each region has certain differences and privacy, and centralized prediction needs to transmit a large amount of traffic data, which will not only cause bandwidth consumption, but may also cause privacy leakage. Federated learning is a kind of distributed learning method with multi-client joint training and no sharing between clients. Based on existing related research, this paper proposes a gradient similarity-based federated aggregation algorithm for wireless traffic prediction (Gradient Similarity-based Federated Aggregation for Wireless Traffic Prediction) (FedGSA). First of all, this method uses a global sharing enhanced data strategy to overcome the data heterogeneity challenge of multi-client collaborative training in federated learning. Secondly, the sliding window scheme is used to construct the dual channel training data to improve the feature learning ability of the model; In addition, to improve the generalization ability of the final global model, a two-layer aggregation scheme based on gradient similarity is proposed. The personalized model is generated by comparing the gradient similarity of each client model, and the central server aggregates the personalized model to finally generate the global model. Finally, the FedGSA algorithm is applied to wireless network traffic prediction. Experiments are conducted on two real traffic datasets. Compared with the mainstream Federated Averaging (FedAvg) algorithm, FedGSA performs better on both datasets and obtains better prediction results on the premise of ensuring the privacy of client traffic data. Full article

(This article belongs to the Special Issue Federated and Transfer Learning Applications)

► Show Figures

Figure 1

19 pages, 946 KiB

Open AccessArticle

Towards Mobile Federated Learning with Unreliable Participants and Selective Aggregation

by Leonardo Esteves, David Portugal, Paulo Peixoto and Gabriel Falcao

Appl. Sci. 2023, 13(5), 3135; https://doi.org/10.3390/app13053135 - 28 Feb 2023

Cited by 2 | Viewed by 1590

Abstract

Recent advances in artificial intelligence algorithms are leveraging massive amounts of data to optimize, refine, and improve existing solutions in critical areas such as healthcare, autonomous vehicles, robotics, social media, or human resources. The significant increase in the quantity of data generated each year makes it urgent to ensure the protection of sensitive information. Federated learning allows machine learning algorithms to be partially trained locally without sharing data, while ensuring the convergence of the model so that privacy and confidentiality are maintained. Federated learning shares similarities with distributed learning in that training is distributed in both paradigms. However, federated learning also decentralizes the data to maintain the confidentiality of the information. In this work, we explore this concept by using a federated architecture for a multimobile computing case study and focus our attention on the impact of unreliable participants and selective aggregation in the federated solution. Results with Android client participants are presented and discussed, illustrating the potential of the proposed approach for real-world applications. Full article

(This article belongs to the Special Issue Federated and Transfer Learning Applications)

► Show Figures

Figure 1

17 pages, 3017 KiB

Open AccessArticle

Blockchain-Based Decentralized Federated Learning Method in Edge Computing Environment

by Song Liu, Xiong Wang, Longshuo Hui and Weiguo Wu

Appl. Sci. 2023, 13(3), 1677; https://doi.org/10.3390/app13031677 - 28 Jan 2023

Cited by 9 | Viewed by 1897

Abstract

In recent years, federated learning has been able to provide an effective solution for data privacy protection, so it has been widely used in financial, medical, and other fields. However, traditional federated learning still suffers from single-point server failure, which is a frequent issue from the centralized server for global model aggregation. Additionally, it also lacks an incentive mechanism, which leads to the insufficient contribution of local devices to global model training. In this paper, we propose a blockchain-based decentralized federated learning method, named BD-FL, to solve these problems. BD-FL combines blockchain and edge computing techniques to build a decentralized federated learning system. An incentive mechanism is introduced to motivate local devices to actively participate in federated learning model training. In order to minimize the cost of model training, BD-FL designs a preference-based stable matching algorithm to bind local devices with appropriate edge servers, which can reduce communication overhead. In addition, we propose a reputation-based practical Byzantine fault tolerance (R-PBFT) algorithm to optimize the consensus process of global model training in the blockchain. Experiment results show that BD-FL effectively reduces the model training time by up to 34.9% compared with several baseline federated learning methods. The R-PBFT algorithm can improve the training efficiency of BD-FL by 12.2%. Full article

(This article belongs to the Special Issue Federated and Transfer Learning Applications)

► Show Figures

Figure 1

18 pages, 3171 KiB

Open AccessArticle

Anomaly Detection and Early Warning Model for Latency in Private 5G Networks

by Jingyuan Han, Tao Liu, Jingye Ma, Yi Zhou, Xin Zeng and Ying Xu

Appl. Sci. 2022, 12(23), 12472; https://doi.org/10.3390/app122312472 - 6 Dec 2022

Cited by 3 | Viewed by 2329

Abstract

Different from previous generations of communication technology, 5G has tailored several modes especially for industrial applications, such as Ultra-Reliable Low-Latency Communications (URLLC) and Massive Machine Type Communications (mMTC). The industrial private 5G networks require high performance of latency, bandwidth, and reliability, while the deployment environment is usually complicated, causing network problems difficult to identify. This poses a challenge to the operation and maintenance (O&M) of private 5G networks. It is needed to quickly diagnose or predict faults based on high-dimensional data of networks and services to reduce the impact of network faults on services. This paper proposes a ConvAE-Latency model for anomaly detection, which enhances the correlation between target indicators and hidden features by multi-target learning. Meanwhile, transfer learning is applied for anomaly prediction in the proposed LstmAE-TL model to solve the problem of unbalanced samples. Based on the China Telecom data platform, the proposed models are deployed and tested in an Automated Guided Vehicles (AGVs) application scenario. The results have been improved compared to existing research. Full article

(This article belongs to the Special Issue Federated and Transfer Learning Applications)

► Show Figures

Figure 1

21 pages, 2937 KiB

Open AccessArticle

A Message Passing Approach to Biomedical Relation Classification for Drug–Drug Interactions

by Dimitrios Zaikis, Christina Karalka and Ioannis Vlahavas

Appl. Sci. 2022, 12(21), 10987; https://doi.org/10.3390/app122110987 - 30 Oct 2022

Cited by 1 | Viewed by 1294

Abstract

The task of extracting drug entities and possible interactions between drug pairings is known as Drug–Drug Interaction (DDI) extraction. Computer-assisted DDI extraction with Machine Learning techniques can help streamline this expensive and time-consuming process during the drug development cycle. Over the years, a variety of both traditional and Neural Network-based techniques for the extraction of DDIs have been proposed. Despite the introduction of several successful strategies, obtaining high classification accuracy is still an area where further progress can be made. In this work, we present a novel Knowledge Graph (KG) based approach that utilizes a unique graph structure in combination with a Transformer-based Language Model and Graph Neural Networks to classify DDIs from biomedical literature. The KG is constructed to model the knowledge of the DDI Extraction 2013 benchmark dataset, without the inclusion of additional external information sources. Each drug pair is classified based on the context of the sentence it was found in, by utilizing transfer knowledge in the form of semantic representations from domain-adapted BioBERT weights that serve as the initial KG states. The proposed approach was evaluated on the DDI classification task of the same dataset and achieved a F1-score of 79.14% on the four positive classes, outperforming the current state-of-the-art approach. Full article

(This article belongs to the Special Issue Federated and Transfer Learning Applications)

► Show Figures

Figure 1

21 pages, 683 KiB

Open AccessArticle

On a Framework for Federated Cluster Analysis

by Morris Stallmann and Anna Wilbik

Appl. Sci. 2022, 12(20), 10455; https://doi.org/10.3390/app122010455 - 17 Oct 2022

Cited by 3 | Viewed by 1956

Abstract

Federated learning is becoming increasingly popular to enable automated learning in distributed networks of autonomous partners without sharing raw data. Many works focus on supervised learning, while the area of federated unsupervised learning, similar to federated clustering, is still less explored. In this paper, we introduce a federated clustering framework that solves three challenges: determine the number of global clusters in a federated dataset, obtain a partition of the data via a federated fuzzy c-means algorithm, and validate the clustering through a federated fuzzy Davies–Bouldin index. The complete framework is evaluated through numerical experiments on artificial and real-world datasets. The observed results are promising, as in most cases the federated clustering framework’s results are consistent with its nonfederated equivalent. Moreover, we embed an alternative federated fuzzy c-means formulation into our framework and observe that our formulation is more reliable in case the data are noni.i.d., while the performance is on par in the i.i.d. case. Full article

(This article belongs to the Special Issue Federated and Transfer Learning Applications)

► Show Figures

Figure 1

23 pages, 3097 KiB

Open AccessArticle

Intelligent Arabic Handwriting Recognition Using Different Standalone and Hybrid CNN Architectures

by Waleed Albattah and Saleh Albahli

Appl. Sci. 2022, 12(19), 10155; https://doi.org/10.3390/app121910155 - 10 Oct 2022

Cited by 11 | Viewed by 3093

Abstract

Handwritten character recognition is a computer-vision-system problem that is still critical and challenging in many computer-vision tasks. With the increased interest in handwriting recognition as well as the developments in machine-learning and deep-learning algorithms, researchers have made significant improvements and advances in developing English-handwriting-recognition methodologies; however, Arabic handwriting recognition has not yet received enough interest. In this work, several deep-learning and hybrid models were created. The methodology of the current study took advantage of machine learning in classification and deep learning in feature extraction to create hybrid models. Among the standalone deep-learning models trained on the two datasets used in the experiments performed, the best results were obtained with the transfer-learning model on the MNIST dataset, with 0.9967 accuracy achieved. The results for the hybrid models using the MNIST dataset were good, with accuracy measures exceeding 0.9 for all the hybrid models; however, the results for the hybrid models using the Arabic character dataset were inferior. Full article

(This article belongs to the Special Issue Federated and Transfer Learning Applications)

► Show Figures

Figure 1

12 pages, 4567 KiB

Open AccessArticle

Comparisons Where It Matters: Using Layer-Wise Regularization to Improve Federated Learning on Heterogeneous Data

by Ha Min Son, Moon Hyun Kim and Tai-Myoung Chung

Appl. Sci. 2022, 12(19), 9943; https://doi.org/10.3390/app12199943 - 3 Oct 2022

Cited by 3 | Viewed by 1547

Abstract

Federated Learning is a widely adopted method for training neural networks over distributed data. One main limitation is the performance degradation that occurs when data are heterogeneously distributed. While many studies have attempted to address this problem, a more recent understanding of neural networks provides insight to an alternative approach. In this study, we show that only certain important layers in a neural network require regularization for effective training. We additionally verify that Centered Kernel Alignment (CKA) most accurately calculates similarities between layers of neural networks trained on different data. By applying CKA-based regularization to important layers during training, we significantly improved performances in heterogeneous settings. We present FedCKA, a simple framework that outperforms previous state-of-the-art methods on various deep learning tasks while also improving efficiency and scalability. Full article

(This article belongs to the Special Issue Federated and Transfer Learning Applications)

► Show Figures

Figure 1

15 pages, 703 KiB

Open AccessArticle

Privacy and Security in Federated Learning: A Survey

by Rémi Gosselin, Loïc Vieu, Faiza Loukil and Alexandre Benoit

Appl. Sci. 2022, 12(19), 9901; https://doi.org/10.3390/app12199901 - 1 Oct 2022

Cited by 26 | Viewed by 7626

Abstract

In recent years, privacy concerns have become a serious issue for companies wishing to protect economic models and comply with end-user expectations. In the same vein, some countries now impose, by law, constraints on data use and protection. Such context thus encourages machine learning to evolve from a centralized data and computation approach to decentralized approaches. Specifically, Federated Learning (FL) has been recently developed as a solution to improve privacy, relying on local data to train local models, which collaborate to update a global model that improves generalization behaviors. However, by definition, no computer system is entirely safe. Security issues, such as data poisoning and adversarial attack, can introduce bias in the model predictions. In addition, it has recently been shown that the reconstruction of private raw data is still possible. This paper presents a comprehensive study concerning various privacy and security issues related to federated learning. Then, we identify the state-of-the-art approaches that aim to counteract these problems. Findings from our study confirm that the current major security threats are poisoning, backdoor, and Generative Adversarial Network (GAN)-based attacks, while inference-based attacks are the most critical to the privacy of FL. Finally, we identify ongoing research directions on the topic. This paper could be used as a reference to promote cybersecurity-related research on designing FL-based solutions for alleviating future challenges. Full article

(This article belongs to the Special Issue Federated and Transfer Learning Applications)

► Show Figures

Figure 1

14 pages, 1400 KiB

Open AccessArticle

Lazy Aggregation for Heterogeneous Federated Learning

by Gang Xu, De-Lun Kong, Xiu-Bo Chen and Xin Liu

Appl. Sci. 2022, 12(17), 8515; https://doi.org/10.3390/app12178515 - 25 Aug 2022

Cited by 2 | Viewed by 1570

Abstract

Federated learning (FL) is a distributed neural network training paradigm with privacy protection. With the premise of ensuring that local data isn’t leaked, multi-device cooperation trains the model and improves its normalization. Unlike centralized training, FL is susceptible to heterogeneous data, biased gradient estimations hinder convergence of the global model, and traditional sampling techniques cannot apply FL due to privacy constraints. Therefore, this paper proposes a novel FL framework, federated lazy aggregation (FedLA), which reduces aggregation frequency to obtain high-quality gradients and improve robustness in non-IID. To judge the aggregating timings, the change rate of the models’ weight divergence (WDR) is introduced to FL. Furthermore, the collected gradients also facilitate FL walking out of the saddle point without extra communications. The cross-device momentum (CDM) mechanism could significantly improve the upper limit performance of the global model in non-IID. We evaluate the performance of several popular algorithms, including FedLA and FedLA with momentum (FedLAM). The results show that FedLAM achieves the best performance in most scenarios and the performance of the global model can also be improved in IID scenarios. Full article

(This article belongs to the Special Issue Federated and Transfer Learning Applications)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Federated and Transfer Learning Applications

Share This Special Issue

Special Issue Editors

Special Issue Information

Keywords

Published Papers (12 papers)

Editorial

Research

Further Information

Guidelines

MDPI Initiatives

Follow MDPI