A Federated Learning-Based Resource Allocation Scheme for Relaying-Assisted Communications in Multicellular Next Generation Network Topologies

Abstract

Growing and diverse user needs, along with the need for continuous access with minimal delay in densely populated machine-type networks, have led to a significant overhaul of modern mobile communication systems. Within this realm, the integration of advanced physical layer techniques such as relaying-assisted transmission in beyond fifth-generation (B5G) networks aims to not only enhance network performance but also extend coverage across multicellular orientations. However, in cellular environments, the increased interference levels and the complex channel representations introduce a notable rise in the computational complexity associated with radio resource management (RRM) tasks. Machine and deep learning (ML/DL) have been proposed as an efficient way to support the enhanced user demands in densely populated environments since ML/DL models can relax the traffic load that is associated with RRM tasks. There is, however, in these solutions the need for distributed execution of training tasks to accelerate the decision-making process in RRM tasks. For this purpose, federated learning (FL) schemes are considered a promising field of research for next-generation (NG) networks’ RRM. This paper proposes an FL approach to tackle the joint relay node (RN) selection and resource allocation problem subject to power management constraints when in B5G networks. The optimization objective of this approach is to jointly elevate energy (EE) and spectral efficiency (SE) levels. The performance of the proposed approach is evaluated for various relaying-assisted transmission topologies and through comparison with other state-of-the-art ones (both ML and non-ML). In particular, the total system energy efficiency (EE) and spectral efficiency (SE) can be improved by up to approximately 10–20% compared to a state-of-the-art centralized ML scheme. Moreover, achieved accuracy can be improved by up to 10% compared to state-of-the-art non-ML solutions, while training time is reduced by approximately 50%.

1. Introduction

The worldwide deployment of fifth-generation (5G) wireless networks is bringing in new and improved features such as enhanced mobile broadband (eMBB), ultra-reliable low latency communications (URLLC), and massive machine-type communications (mMTC) [1,2,3]. The confluence of these three pillars—eMBB, URLLC [4], and mMTC [5]—reaffirms 5G as a versatile and robust technology that extends beyond conventional mobile communication networks catalyzing innovation across diverse industries, facilitating the realization of novel applications and services that were hitherto unattainable with earlier generations of wireless technology.

However, the rapid evolution of networking technologies [6] (such as distributed computation, cloud computing, network function virtualization, network slicing, and software-defined networks (SDN)) and the need for reliable service provisioning for Internet of Things (IoT), Industry 4.0, and augmented/virtual reality (AR/VR) applications facilitated the need for next generations (NG) of wireless networks (such as beyond 5G (B5G) and sixth generation (6G) networks). Another factor that highlighted the need for B5G/6G networks and drove the start of the Third Generation Partnership Project (3GPP) standardization process [7,8] is the surge in data generation coupled with the user/service requirements for real-time network responses [9]. The overarching goal of B5G/6G networks is to evolve “connected intelligence” by deploying data-aided models across a wide span of tasks, applications, and open systems interconnection (OSI) levels. This marks a pivotal shift from mere connectivity to intelligent interactivity, shaping the trajectory of wireless communication networks toward an era characterized by enhanced responsiveness and interconnected intelligence [10,11,12].

From a physical layer perspective, the transition to NG wireless networks—which will be established with the expected International Mobile Telecommunications-2030 (IMT-2030) standard—involves the introduction of several novel technologies such as millimeter-wave (mmWave) transmission [13], non-orthogonal multiple access (NOMA) [14], massive multiple-input multiple-output orientations (m-MIMO) [15], physical layer security (PLS) [16], and reconfigurable intelligent surfaces (RIS) [17]. Moreover, network densification emerges as a pivotal strategy for ensuring seamless connectivity to an increasing number of simultaneously connected mobile devices. Within this framework, the conventional single-link (one-hop communications) concept, where the base station (BS) is directly linked to user equipment (UEs) is replaced by a network of potential connections emanating from access points (APs) and relay nodes (RNs), thereby enhancing the overall efficiency and coverage of the wireless communication infrastructure [15,18]. Such topologies also demonstrate significant performance gains in heterogeneous scenarios, such as mobile/flying ad hoc networks (MANETs/FANETs).

In the advancing landscape of B5G/6G networks, the vital role of artificial intelligence (AI) and machine learning (ML) as key tools for extracting valuable insights from vast and diverse data generated by networks is acknowledged [19,20]. These technologies are pivotal for supporting decision-making, automating diverse service requirements, and managing radio resources effectively. To achieve this, the substantial amount of data (big data) generated in the dynamically changing and densely populated B5G/6G environments plays a central role in training various learning algorithms, including artificial neural networks (ANNs) [21], support vector machines (SVMs) [22], reinforcement [23], and deep reinforcement learning (RL/DRL) models [24]. However, meaningful insights and the adjustment of important parameters require significant computational resources for the successive training and execution of these AI/ML tasks. Thus, accelerating the training phase of AI/ML models and minimizing response times emerge as crucial aspects, indispensable for the practicality and effectiveness of ML tasks within the overarching framework of NG networks [9,25].

ML techniques that rely on central entities to produce the learning outcome may face several difficulties when dealing with imbalanced, heterogeneous, or poor-quality data, such as B5G/6G channel state information (CSI) or m-MIMO-related ones [26]. Thus, both traditional ML (supervised, unsupervised learning) and centralized learning (CL) categories may be insufficient in achieving efficient resource orchestration for the increased number of interconnected devices, machines, and sensors in B5G/6G networks. The most significant drawbacks of such approaches when integrated into the B5G/6G networks’ domain are non-instant responses, local data reliance, and potential security vulnerabilities (e.g., single point of failure) [9,27]. For this purpose, decentralized and distributed ML solutions are proposed as an effective way to handle these problems.

This manuscript’s motivation stems from the need to reduce computational overload from B5G/6G major nodes (such as core network servers and BSs). These entities have to repetitively deal with multiple tasks that span across multiple OSI levels (from physical to application layer). In addition to that, RRM problems, such as subcarrier allocation, BS or RN selection, and placement are categorized as NP-hard ones, especially in dense and high-interference environments. Thus, this manuscript considers distributed ML training schemes to overcome both the data quality and the network overload problems.

On this framework, federated learning (FL), which was introduced in [28], and combines the principles of both ML and mobile edge computing (MEC), has recently been proposed as an efficient solution for efficient solutions in RRM problems. The key characteristic of FL, which diversifies this technique from CL ones, is the existence of a global model that is constituted by the—equal or not—local model contributions of several interconnected edge entities (e.g., devices, servers). As depicted in Figure 1, local datasets are not mitigating into a central entity, but only local-trained models are doing so to generate the global model [29]. FL can also operate without transferring local model updates to a central entity, when nearby devices form clusters to collaboratively exchange model parameters promoting both privacy and efficiency [30].

Recently, FL schemes have been proposed as a promising solution in different RRM tasks in B5G/6G networks, such as user management and profiling [31], interference mitigation [32], subcarrier allocation [33], and signal-to-interference-plus-noise (SINR) maximization [34]. This paper focuses on the joint RN selection and subcarrier allocation problem in two-hop relaying-assisted B5G/6G networks aiming to maximize both energy (EE) and spectral efficiency (SE). This problem is categorized into the NP-hard ones, as intense multipath propagation and non-linear channel representation phenomenon exist both for the BS-RN (first hop) and the UE-RN link.

Table 1. CL and FL in B5G/6G networks comparison.

Aspect	CL	FL
Data Privacy	May raise privacy concerns as centralized processing involves accessing raw data from multiple sources.	Raw data remain on local devices, and only model updates are exchanged, preserving data privacy.
Communication Overhead	Lower communication overhead as all data are in one location.	Higher communication overhead due to the need to send model updates across decentralized devices, impacting latency.
Scalability	May face scalability issues as the central entities handle all data.	Generally more scalable as decentralized learning can be distributed across a large number of devices in B5G/6G networks.
Data Efficiency	Efficient use of data since all data are available in one place.	May require more data due to decentralized training, potentially posing challenges in scenarios with limited data on individual devices.
Model Robustness	Centralized model may lack robustness if not trained on diverse data.	Encourages model robustness by training on diverse local data, potentially enhancing adaptability to heterogeneous network conditions.
Security	Vulnerable to security breaches as all data are in one location.	More secure in terms of data privacy since raw data stay on local devices, reducing the risk of a centralized data breach in 5G/6G networks.

presents a comparison of CL and FL schemes focusing on RRM in B5G/6G networks.

1.1. Related Works

Relaying-assisted wireless communications constitute an active area of research especially in B5G/6G cellular orientations, as multi-hop communications are proposed to extend the overall network coverage area, improve capacity without new hardware installation, and reduce energy consumption through the BS-UE link decongestion. These characteristics are vital for the new B5G/6G usage scenarios, such as industry automation, vehicle-to-vehicle (V2V) communications, and AR/VR applications, where even moving RNs are considered. Moreover, the RN-assisted communication gains can be extended even more when jointly utilized with advanced physical layer techniques, such as m-MIMO, NOMA, and RIS. In such scenarios, despite the NP-hard nature of the RRM optimization problem [35], ML-based schemes can be efficient enough, without the need for extensive search algorithms or numerous simulation rounds [18]. However, FL schemes have recently been proposed in order not just to solve the RRM problem efficiently but also to reduce training times and unstress both the BS-RN and the RN-UE links from overarching traffic load [36].

This subsection analyzes the current research activity in the field of FL-based frameworks for efficient RRM in relaying-assisted B5G/6G networks, focusing on performance metrics maximization (e.g., EE and SE).

In [37], the authors introduce the idea of FL-based relay-assisted communications. They propose a relay network that serves as a cooperative platform for transferring and trading model updates. In this setup, mobile devices create model updates from their training data and send them to the model owner through a cooperative relay network. The model owner, benefiting from the learning services provided by the mobile devices, reciprocates by compensating them with specific prices. To navigate the interference among mobile devices using the same relay node, rational mobile devices strategically choose relay nodes and determine their transmission powers. A Stackelberg game model is employed to analyze interactions among mobile devices and between mobile devices and the model owner. Performance evaluation reveals that devices with low computation power can benefit by strategically aligning with neighboring high-computation power, impacting model transmission time and overall energy consumption.

The authors of [38] first proposed a distributed FL framework to ensure secure data privacy in relaying-assisted 5G/6G air-BS networks. A distributed FL network architecture where user devices form clusters, following the MEC architecture paradigm, to efficiently relay ML training traffic between them. The aerial BS acts as the node that aggregates the local model updates. The proposed framework is compared to CL-based 5G/6G architectures, where the training phase of the AI/ML algorithms is coordinated by a central entity. Performance evaluation indicated that the proposed FL-based scheme improves classification accuracy by approximately 7% compared to state-of-the-art CL approaches. In the same way, the authors of [39] and [40] also focus on EE maximization when considering FL schemes for relay-assisted B5G/6G IoT networks. The optimization goal of the proposed approach was to reduce IoT device energy consumption while jointly meeting wireless transmission latency and model training calculation time constraints during the FL training phase. The key novelty of [40] was the integration of a weighted communication rate of all participating devices to maximize the convergence time for local model aggregation. To do so, the authors formed a maximum-weight independent problem that was solved approximately using graph theory techniques. In this way, a sub-optimal solution for the original RRM problem (NP-hard) that increases EE levels is proposed.

In [41], a novel relay-assisted cooperative FL scheme is introduced to effectively mitigate the straggler issue, which is caused in B5G/6G networks due to edge device heterogeneity. The proposed approach involves the deployment of multiple RNs to assist edge devices in uploading local model updates to the edge server. Due to the multi-parameter nature of the RRM problem in RN-assisted wireless topologies, an alternating-optimization algorithm is proposed, ensuring efficient optimization of transceiver and relay operations with low complexity. Performance evaluation is performed considering a two-hop orientation. Results indicate that the aggregation error in such a scenario decreases by approximately 10–20%, while the test set accuracy increases by a similar level (10–20%) compared to a state-of-the-art non-RN-based topology.

Considering, the joint EE and SE optimization in modern-era wireless communication networks, the authors of [42] model the joint optimization problem as a Markov decision process object to transmission power and QoS constraints. Integrating spatiotemporal CSI, they proposed a DRL framework (based on transmission power and QoS requirements as neural network inputs) to effectively solve the subsequent EE and SE optimization problem. Simulations demonstrate that the proposed method outperforms other state-of-the-art ML approaches as it demonstrates a five times increase in the joint EE-SE metric (Mbits/Joule). A similar approach from the same authors is presented in [43] focusing on cloud radio access network (CRAN) orientations. There, a double deep Q-network is proposed for joint EE-SE optimization. Performance evaluation indicated the effectiveness of the proposed approach compared to other DRL schemes.

Focusing on cloud-based architectures, the authors of [44] (which is an extended work of [45]) studied the resource allocation challenge in cloud data centers as a model-free DRL problem, considering dynamic system states and diverse user demands. An asynchronous advantage actor-critic-based method for efficient job scheduling, targeting improved QoS and energy efficiency, is proposed. Simulation experiments using real-world data from Google Cloud data centers validate the effectiveness of our approach, showcasing its superiority over traditional resource allocation methods (e.g., LJF, Tetris, SJF, and others) in terms of QoS metrics and energy efficiency. Moreover, the proposed method also outperforms recent alternatives under increased system load and demonstrates higher training efficiency compared to two advanced DRL-based methods (PG and DQL). In the same field, the authors of [46,47] proposed DRAW, a novel DRL-based RRM method with workload-time windows. DRAW considers both current and future workloads, leveraging a DQN prediction model trained on workload-time windows. This model accurately predicts management operations under different system states. An iterative feedback-control mechanism then constructs an objective resource allocation plan based on the current system state. Simulations demonstrate DRAW’s high prediction accuracy of 90.69% and its ability to achieve optimal/near-optimal performance, surpassing classic methods by 3 to 13% under diverse scenarios.

1.2. Contributions and Paper’s Structure

The goal of the study presented in this paper is to extend the work in [18]. In this paper, we focus on solving the joint problem of RN selection and subcarrier allocation by proposing a fully two-level fully ML-aided framework. The first level refers to a deep learning (DL)-based RN selection scheme that jointly maximizes EE and SE levels. The latter refers to a novel FL-based scheme for efficient ML algorithm training using MEC servers installed in topologies BSs and RNs, which minimizes training time by enabling a computation offloading mechanism between numerous edge devices. Our contributions are the following:

• We first formulate the problem of optimal RN selection and subcarrier allocation for each user served by the cooperative relaying-assisted system, which serves users that cannot be served directly from BSs, either for path loss or power management purposes.
• For the aforementioned problem, a DL-based algorithm is proposed. The primary goal is to maximize EE and SE levels both for each cell and for the whole coverage area.
• In order to relax the network’s computation load, the training of this DL scheme is performed using an FL-based computation offloading framework, where different edge devices (either UEs with certain computation capabilities and/or MEC servers installed in the BSs) are assigned with the execution of a portion the training phase. Afterward, the local updates are aggregated to form a global model—hosted at the central BS of the topology—that concerns the total system’s EE and SE maximization. Local models (before aggregation) utilize data from the cells that are hosted and utilize a local mechanism for local EE and SE maximization.
• The performance of the proposed framework is evaluated by extensive system and link-level simulations in different usage scenarios utilizing a B5G system and link-level simulator extending the work performed in [48]. Results indicate that the proposed FL-based RN selection and resource allocation scheme can overperform state-of-the-art approaches (both non-ML and CL ones) in improving various key performance indicators (KPIs) of interest (EE and SE). Furthermore, the proposed decentralized ML training scheme achieves the required training time minimization, which is a crucial aspect in B5G/6G network orientations.
• To sum up, the utilization of the proposed FL scheme for efficient RN selection and RRM in each cell of the multicellular topology, which provides a fully data-driven automated decision-making mechanism for high-interference dense B5G/6G environments and optimizes both network and ML performance metrics (focusing on EE, SE, and ML algorithms’ training time), is the key novelty of this paper.

The rest of this paper is organized as follows. In Section 2, the B5G multicellular orientation is described. In Section 3, the proposed FL framework for efficient RRM based on the optimization of EE and SE levels is analyzed. In Section 4, the overall simulation setup is presented along with simulation results. Finally, concluding remarks are outlined in Section 5.

2. B5G Multicellular Orientation

In this section, the system model of the B5G/6G multicellular topology under investigation is presented. Section 2.1 refers to the two-hop multicellular-topology architecture, where both BS-RN and RN-UE links are explained. Moreover, Section 2.2 refers to the FL architecture, where the edge devices, their capabilities, and the computation offloading optimization problem are analyzed.

2.1. System Model for RN Selection in B5G/6G Cooperative Networks

The downlink of a cooperative RN-assisted wireless B5G/6G multicellular topology is examined, as depicted in Figure 2. The system under consideration comprises two distinct node entities and their corresponding links. The Macro-BSs constitute the primary system, where UEs can directly connect and request service. In this scenario, only the BS-UE link is present. The secondary system is composed of cooperative amplify and forward (A&F) RNs, aiding the primary system in serving additional users and potentially enhancing coverage area. This is particularly beneficial for UEs initially rejected from the primary system, either due to high path loss or lack of radio resources, as they can be accommodated by the secondary system. In this case, both the BS-RN and RN-UE links are established.

As depicted in Figure 2, the cooperative system consists of the following sets:

• $S_{BS}=\{B{S}_1,B{S}_2,\dots ,B{S}_M\}$, where $M$ denotes the total number of Macro-BSs in the topology.
• $S_{RN}=\{{RN}_1,{RN}_2,\dots ,{RN}_R\}$, where $R$ denotes the total number of A&F RNs in the topology.
• $S_{UE}=\{{UE}_1,{UE}_2,\dots ,{UE}_N\}$, where $N$ denotes the total number of UEs that sequentially reach the topology.

The three different types of potential links that exist in the system are the following:

• $L_{b,u}$, where $b\in S_{BS}$ and $u\in S_{UE}$, which denotes a BS-UE link.
• $L_{b,r}$, where $b\in S_{BS}$ and $r\in S_{RN}$, which denotes a BS-RN link.
• $L_{r,u}$, where $b\in S_{RN}$ and $u\in S_{UE}$, which denotes a RN-UE link.

In the examined topology a constant number of RNs (denoted as $R_{BS}$) is deployed inside each macro-BS’s coverage area, to assist the primary system by processing the incoming signal and support service delivery. However, it should be mentioned that a UE can only be connected to one BS or RN and that channel orthogonality is assured for co-channel interference mitigation.

In the above two-hop B5G/6G communication orientation, the total available system bandwidth ($W$) is divided into $N_{sc}$ subcarriers to be allocated to accepted—either by the primary or the secondary system—UEs. We are focusing on the latter, which is the UEs that are influenced by the presence of the RNs and by the utilization of the RN-UE and BS-RN links. In this case, RNs have a twofold role, as they are acting as UEs in the BS-RN link, and as BSs in the RN-UE one. Furthermore, BSs/RNs are equipped with $M_t$ antennas, while UEs with $M_r$ ones. In an RN-assisted scenario, the SINR for the n^th UE ($1\le n\le N$) associated with the l^th subcarrier ($1\le l\le N_{sc}$) assuming independent BS-RN and RN-UE links, as well as a specific channel realization, is given by [18]:

$$SIN{R}_{n,l}=SIN{R}_{n,l}\left(BS\right)+SIN{R}_{n,l}\left(RN\right)$$

(1)

where,

$$SIN{R}_{n,l}(BS)=\frac{G_{n,n,l}}{{\mathit{r}}_{n,l}^H{\mathit{r}}_{n,l}{I}_0+\sum _{m\ne n,lϵS_m}{G}_{n,m,l}}$$

(2)

where $G_{n,m,l}=p_{n,l}{{\mathit{t}}_{m,l}}^H{{\mathit{H}}_{n,\sec \left(n\right),l}}^H{\mathit{r}}_{n,l}^H{\mathit{r}}_{n,l}{\mathit{H}}_{\mathrm{n},\sec \left(n\right),l}{\mathit{t}}_{m,l}$, ${\mathit{H}}_{n,\sec \left(n\right),l}$ represents the $M_r\times M_t$ channel matrix (flat Rayleigh fading) for the $l^{th}$ subcarrier of the $n^{th}$ UE relevant to its serving sector $\sec \left(n\right)$, ${\mathit{t}}_{n,l}$ is the $M_t\times 1$ transmission vector in diversity combining transmission mode, ${\mathit{r}}_{n,l}$ is the maximal ratio combining (MRC) multiplying vector [28], and $p_{n,l}$ denotes the transmission power allocated to the $l^{th}$ subcarrier of the $n^{th}$ UE. Moreover, the set $S_n$ indicates the subcarriers allocated to the $n^{th}$ UE and $I_0$ is the thermal noise level. Finally, ${\mathit{A}}^H$ is the conjugate transpose of matrix $\mathit{A}$.

Moreover,

$$SIN{R}_{n,l\left(RN\right)}=\frac{G_{n,n,l(RN-UE)}}{{\mathit{r}}_{n,l}^H{\mathit{r}}_{n,l}{I}_0+I_{B{S}_{n,l}}+I_{R{N}_{n,l}}}$$

(3)

where $I_{B{S}_{n,l}}=\sum _{b=1}^{N_{BS}}\sum _{mϵU{E}_b,l\epsilon S_m}{G}_{n,m,l}$ and $I_{R{N}_{n,l}}=\sum _{r=1}^{N_{RN}}\sum _{jϵU{E}_r,l\epsilon S_j}{G}_{n,j,l}$ are the cumulative interference levels of the $l^{th}$ subcarrier of the $n^{th}$ UE served by the $b^{th}$ BS or the $r^{th}$ RN. Moreover, $N_{BS}$, $N_{RN}$ are the total number of BSs and RNs deployed in the topology, respectively, $U{E}_r$ denotes the set of UEs served by the $r^{th}$ RN, while the notation x-y indicates all possible link connections. Moreover, $N_{BS}=k·R_{BS}$, where $k\in \mathbb{N}$.

Concerning the above, the total system throughput is given by [18,49]:

$$\begin{gathered} R={\sum }_{n=1}^N{\sum }_{s\in S_n}{r}_{n,s}=W\{{\sum }_{b=1}^{N_{BS}}{\sum }_{n\in U{E}_b}{\sum }_{sϵS_b}{\log }_2\left(1+SNI{R}_{n,s\left(BS\right)}\right)+ \\ {\sum }_{r=1}^{N_{RN}}{\sum }_{n\in U{E}_r}{\sum }_{sϵS_m}{\log }_2\left(1+SNI{R}_{m,s\left(RN\right)}\right)\} \end{gathered}$$

(4)

where $\left|S_n\right|$ indicates the length of the set $S_n$, $r_{n,s}$ is the corresponding throughput for the $s^{th}$ subcarrier and $B_{SC}$ is the bandwidth per subcarrier. Using (1)–(4), the overall system’s EE and SE levels are defined as:

$$EE=\frac{R}{{\sum }_{n=1}^N{\sum }_{sϵS_n}{p}_{n,s}}$$

(5)

$$SE=\frac{R}{W}$$

(6)

2.2. Federated Learning Architecture

The topology depicted in Figure 2 is illustrated from a FL architecture perspective in Figure 3. The $R$ RNs in the topology (equipped with $M_t$ antennas) act collaboratively in order to train a global/shared ML model by using only their local-available data. For this purpose, the global dataset is divided $R$ local datasets where $D_r=\{\left({\mathit{i}}_{r,i},o_{r,i}\right):1\le i\le D_r\}$ is the local dataset at RN $r$, where ${\mathit{i}}_{r,i}$ is the input training sample at this RN and $o_{r,i}$ is the relevant output.

The goal of the ML learning process is to find a model that minimizes the total empirical loss function for all the samples in the global dataset, i.e.,

$$\underset{w\in \mathbb{R}}{\min }F\left(w\right)=\frac{1}{D}{\sum }_{r=1}^R{\sum }_{i=1}^{D_k}f(w;{\mathit{i}}_{r,i},o_{r,i})$$

(7)

where $D={\sum }_{r=1}^R{D}_k$ is the total number of training samples coming from all $R$ devices and $f(w; {\mathit{i}}_{r,i},o_{r,i})$ is the loss function from the specific training sample ${(\mathit{i}}_{r,i},o_{r,i})$.

However, due to the nature of the B5G/6G communications domain, where intense multipath and high-interference environments exist, the close approximation of the aforementioned loss function is significantly difficult. Thus, in each algorithm’s iteration, a gradient-based optimization method is utilized for each device using the corresponding local dataset, based on:

$$w_{t+1}=w_t-{\lambda }_t\nabla F\left(w_t\right)$$

(8)

where ${\lambda }_t$ denotes the learning rate and $\nabla F\left(w_t\right)$ denotes the full gradient.

To sum up, the overall RRM policy will be applied to the system targets to (a) jointly maximize EE and SE levels, (b) maximize training set accuracy, and (c) minimize training time by the minimization of (8), subject to the following system and power constraints:

• (C1): $\sum _{sϵS_n}{p}_{n,s}\le p_m$, where $p_m$ is the maximum power limit per UE.
• (C2): $\sum _{nϵU{E}_b}\sum _{l\in S_n}{p}_{n,l}\le P_m$, where $P_m$ is the maximum power limit per BS and the set $U{E}_b$ consists of UE that are served by the $b^{th}$ BS.
• (C3): $SNI{R}_{n,l}\ge SNI{R}_{thr}$, where $SNI{R}_{thr}$ sets the minimum SNIR threshold for acceptable QoS and has different values based on the modulation level of each UE.
• (C4): $\sum _{n\in U{E}_b}\left|S_n\right|\le N_{sc}$, in order for all the BSs to have equal access to the available subcarriers.

3. Proposed Federated Learning Framework

3.1. Dataset Construction

Dataset construction is a fundamental step in the development of the training and validation phases of ML-based models. In pursuit of this goal, datasets utilized for learning objectives should be precise, current, and consistently subject to assessment.

In this paper, a MATLAB B5G/6G link and system-level network simulator are used to construct our global dataset. Dataset construction is achieved through multiple Monte Carlo (MC) simulation rounds in different A&F RN implementation scenarios, including both Inband and Outband orientations and considering both indoor and outdoor UEs. The simulator was developed in [18,48] and takes into consideration small- and large-scale fading, interference management, and cluster definition for each user of interest, as well as other physical layer aspects. The major improvements that have been performed in this simulator for the context of this paper are:

• UEs are not uniformly distrusted in the B5G/6G network’s coverage area.
• Algorithm 1 of [48] has been updated so that the best RN (out of the deployed ones in each cell) is selected based on the DRL scheme presented in [18]. Hence, the collaboration of m-MIMO and orthogonal frequency division multiple access (OFDMA) principles in RN-assisted multi-hop B5G cellular setups is integrated with contemporary ML techniques to optimize the overall system’s EE and SE levels.

Dataset formulation concerns only UEs that are served from the secondary system (not directly by the BSs) and contains 50.000 UE instances to adequately train ML models. As is also visible in

Table 2. Dataset parameters (per UE).

Features	Description
$U{E}_x$	x-axis position of the UE
$U{E}_y$	y-axis position of the UE
$U{E}_z$	z-axis position of the UE
$B{S}_{serve}$	ID of the server BS
$B{S}_{sec}$	Serving sector of the UE
$P{L}_{mat}$	$Rx1$ path loss matrix between the UE and all available RNs
$T{L}_{mat}$	$Rx1$ total losses matrix between the UE and all available RNs
${\mathit{H}}_{matrix}$	$M_{r}x{M}_t$ channel coefficient matrix
${RN}_{serve}$	ID of the RN that serves the UE

, both location/localization parameters (x, y, and z-axis position), serving BS, path loss, total losses, and MIMO parameters (channel coefficient matrix), are included in the global dataset that is generated by extensive MC simulations using the aforementioned simulators.

The whole dataset’s feature number (based on Table 2) is denoted as $D_{size}$ and calculated by the decomposition of ${\mathit{H}}_{matrix}$ feature into $M_{r}x{M}_t$ different features.

A DNN model is proposed to predict the best-performing RN for each UE entering the multicellular topology. The structure of this model is the following:

• A feature input later with z-score normalization of the input, where $D_{size}$ features are inserted.
• A fully connected layer with 50 × 1 output size, where the input is multiplied (feature input layer) by the corresponding weight matrix; also, the bias vector is added.
• A batch normalization layer, to normalize data across all observations for each channel independently, making training of the NN faster through re-centering and re-scaling.
• A rectified linear unit (ReLU) layer, using a rectified activation function to force the input directly to the output if it is positive, otherwise, to zero output.

Moreover, it should be mentioned that the exhaustive grid search algorithm is used for hyperparameter tuning [50] based on the training accuracy maximization for each local model.

3.2. FL for Training Optimization

In B5G/6G wireless communication network channel representations, interference models and multipath propagation are difficult to predict, especially in dense network orientations. Thus, for the sake of simplicity, we consider that the BSs and RNs know the CSI information and transmission power for each link. In this way, QoS levels can be estimated both for the BS-RN and the RN-UE link.

The problem of RN selection and resource allocation is categorized as an NP-hard one based on [35]. Thus, we propose an FL-based gradient aggregation algorithm to train.

The global model in distributed edge devices (placed at each RN) based on federated averaging (FedAvg) [28] is presented in Algorithm 1.

Algorithm 1: FL-based RN selection and RRM algorithm for EE, SE maximization
1	Input:$\mathrm{learning} \mathrm{rate} {\lambda }_t$$, \mathrm{number} \mathrm{of} \mathrm{BSs} M$$, \mathrm{number} \mathrm{of} \mathrm{UEs} N$$\mathrm{and} \mathrm{of} \mathrm{RNs} \mathrm{per} \mathrm{BS} R_{BS}$
2	Initialization:$t=0, w_0^r=0$
3	for$t=0$$\mathbf{t}\mathbf{o} t=T$ do
		BS Algorithm steps$(\forall m\in M$)
4		Step 1—Model dissemination:$\mathrm{the} \mathrm{BS} \mathrm{broadcast} \mathrm{the} \mathrm{updated} \mathrm{global} \mathrm{model} w_t$$\mathrm{and} \mathrm{the} \mathrm{corresponding} \mathrm{learning} \mathrm{rate} {\lambda }_t$ to all edge devices located to RNs
5		Step 2—Model aggregation:$\mathrm{BS} \mathrm{receives} \mathrm{the} \mathrm{corresponding} w_{t+1}^r$$\mathrm{from} \mathrm{each} \mathrm{edge} \mathrm{device} r$ located in the relevant RN
6			$\mathrm{The} \mathrm{BS} \mathrm{computes} w_{t+1}$$\mathrm{based} \mathrm{on} \left(8\right) \mathrm{and} \mathrm{the} \mathrm{contributions} \mathrm{of} \mathrm{each} \mathrm{RN} r$
7			$\mathrm{The} \mathrm{BS} \mathrm{estimates} \mathrm{the} \mathrm{actual} \mathrm{gradient} \mathrm{of} \mathrm{RN} (\mathrm{edge} \mathrm{device}) r$
8		Step 3—Performance Evaluation: BS computes the overall EE, SE levels
9			if$E{E}_{t+1}>E{E}_t \&\& E{E}_{t+1}>E{E}_t$ then
10				continue;
11			else
12				$\mathrm{reinitialize} w_0^r$ and go to line (3)
		RN Algorithm steps$(\forall r\in R$)—Edge device
1		Step 1—Model distribution:$\mathrm{the} \mathrm{RN} \mathrm{receives} \mathrm{updated} \mathrm{global} \mathrm{model} w_t$$\mathrm{and} \mathrm{the} \mathrm{corresponding} \mathrm{learning} \mathrm{rate} {\lambda }_t$ from the BS
2		Step 2—Local model update:$\mathrm{the} \mathrm{RN} \mathrm{receives} \mathrm{each} \mathrm{set} \left({\mathit{i}}_{r,i},o_{r,i}\right)$ from the connected UEs
3			$\mathrm{The} \mathrm{RN} \mathrm{computes} w_{t+1}^r$
4		Step 3—Local model distribution:$\mathrm{the} \mathrm{RN} \mathrm{broadcasts} w_{t+1}^r$ to the corresponding BS and UEs
		UE Algorithm steps$(\forall n\in N$)
1		Step 1—Sample selection:$\mathrm{the} \mathrm{UE} \mathrm{randomly} \mathrm{selects} \mathrm{samples} \mathrm{of} \mathrm{the} \mathrm{global} \mathrm{dataset} \mathrm{to} \mathrm{form} \left({\mathit{i}}_{r,n},o_{r,n}\right)$
2		Step 2—Local gradient computation
3		Step 3—Local update:$\mathrm{the} \mathrm{RN} \mathrm{receives} \mathrm{the} \mathrm{updated} w_{t+1}^r$ from the corresponding RN

As depicted in Algorithm 1, the procedures in BSs, RNs, and UEs are parallelly executed for each timestep $t$, where $1\le t\le T$ and $T$ are the total number of timesteps executed. The main processes that are performed by the three types of enrolled entities are the following:

• BSs: They broadcast the updated global model $w_t$ and the corresponding learning rate ${\lambda }_t$ to all edge devices located to RNs (Step 1—Model dissemination). Afterwards, they receive the corresponding $w_{t+1}^r$ from each edge device $r$ located in the relevant RN, compute $w_{t+1}$ based on (8) and the contributions of each RN $r$, and estimate the actual gradient of each RN (Step 2—Model aggregation). Finally, KPI evaluation is performed by the calculation of the overall EE and SE levels.
• RNs: They receive the updated global model $w\_t$ and the corresponding learning rate $\lambda \_t$ from the linked BS (Step 1—Model distribution). Afterward, they receive each set $\left({\mathit{i}}_{r,i},o_{r,i}\right)$ from the connected UEs and compute $w_{t+1}^r$ (Step 2—Local model update). Finally, the RN broadcasts $w_{t+1}^r$ to the linked BS and UEs (Step 3—Local model distribution).
• UEs: They randomly select samples of the global dataset to form $\left({\mathit{i}}_{r,i},o_{r,i}\right)$ (Step 1—Sample selection). Afterward, the compute the local gradient (Step 2—Local gradient computation) and, finally, receive the updated $w_{t+1}^r$ from the corresponding RN (Step 3—Local update).

4. Simulation Setup and Results

This section presents the performance of the proposed FL-based RN selection and resource allocation algorithm. The proposed approach is evaluated concerning a two-tier B5G/6G orientation, while the employed simulation parameters are depicted in

Table 3. Simulation parameters.

Features	Description
Tier/number of cells	2/19
Carrier frequency	28 GHz
Number of antennas per BS/RN/UE	4/2/1
Cell radius	$500\sqrt{3}$ m
BS/RN/UE antenna height	25/5/1.5 m
UE indoor-to-outdoor ratio	0.8/0.2
LOS Probability for BS-UE/BS-RN/RN-UE links	15%/11%/10% [52]
Maximum allowed path loss BS/RN	120/320 dB
Antenna gains BS/RN/UE	18/9/4
Requested subcarriers per UE	6/8/11
Number of subcarriers per BS	132
Subcarrier spacing	60 kHz

(similar to [18] and based on [51,52]).

On each occasion, the performance of the proposed algorithms is assessed against a state-of-the-art non-ML method outlined in [48], along with a relay-assisted DRL system introduced in [18].

Both Inband and Outband A&F RN scenarios are considered. Inband RNs use the same spectrum resources as their donner BS, while Outband RNs have been assigned a priori additional spectrum resources exclusively for RN usage [53]. In fact, Outband RN scenarios utilize an extra bandwidth of approximately 55 MHz to cater to initially rejected—from the primary system—UEs, aiming to mitigate interference levels, increase capacity gains over Inband ones [53], and extend the overall system’s coverage area.

The performed simulations have been implemented using MATLAB (R2023b release [54]) and the corresponding toolboxes (e.g., Communications Toolbox, Statistics and Machine Learning Toolbox, and Deep Learning Toolbox).

In this framework, this paper’s simulations consider the following RN deployment scenarios (in addition to the reference basis of no RN implementation): (1) No-RN: No RNs are deployed; (2) MLP-I: ML/DL-based Inband RN placement and DQL RN selection based on [18]; (3) MLP-O: ML/DL-based Outband RN placement and DQL RN selection based on [18]; (4) FL-I: FL-based Inband RN selection and RRM (see Section 3); (5) FL-0: FL-based Inband RN selection and RRM (see Section 3).

4.1. Network Metrics Evaluation

The overall system’s EE levels are depicted in Figure 4, while the corresponding SE levels are depicted in Figure 5 considering all the aforementioned simulation scenarios, along with the reference of no RN deployment.

As evident in Figure 4, the utilization of the FL-enabled RN selection and resource allocation scheme can significantly improve key B5G/6G network metrics, such as EE and spectral efficiency SE, in comparison to:

• The baseline scenario with no RN deployment: When six subcarriers are assigned per UE, EE can achieve up to 139.79/152.63 Mbps/W for FL-I/FL-O scenarios, respectively, while the No-RN scenario is limited to 23.45 Mbps/W. These figures indicate a nearly six times improvement in total EE through FL-based RN selection. For 11 subcarriers per UE, the corresponding values are 52.38/252.45/272.03 Mbps/W for No-RN/FL-I/FL-O scenarios, leading to a ~4–5 times EE enhancement. Similar enhancements are observed for SE, as shown in Figure 5, resulting in a ~4–5 times improvement.
• State-of-the-art CL-based ML approaches [18]: The EE values for scenarios in [18] are 76.95/139.79 Mbps/W for MLP-I/MLP-O scenarios, respectively. Thus, the overall EE is enhanced by ~1.5 times compared to the DRL scheme in [18]. Similarly, for 11 subcarriers per UE, the EE values for MLP-I/MLP-O scenarios are 110.34/252.45 Mbps/W, leading to a ~1–2 times EE improvement. Comparable improvements are observed for SE, as depicted in Figure 5, resulting in a ~1–2 times SE enhancement.

4.2. Impact on Training Accuracy and Time

A key aspect when designing FL-based schemes not only in the B5G/6G wireless communications domain but, in general, is the achieved accuracy compared to CL-based methods, as well as the achieved training time, decrease. The achieved training accuracy of the proposed FL model is depicted in Figure 6. Moreover,

Table 4. Achieved training and inference time comparison.

Evaluation Metrics	CL-Based Training [9,18]	FL-Based Training of This Paper
Training time	5 min 15 s	1 min 20 s
Inference time	50 ms	28 ms

depicts the achieved total training and inference times for the aforementioned RN implementation scenarios.

It is visible from both Figure 6 and Table 4 that the proposed FL-based approach can significantly improve both the achieved accuracy of the RN selection model, but also can minimize the training time needed for the model’s training phase compared to state-of-the-art CL-base ML techniques. To be more precise, the overall model’s accuracy is improved by ~10%, while the training time needed is reduced by ~50%.

5. Conclusions

The performance of an FL-based RN selection and resource allocation scheme for B5G/6G multicellular orientations has been evaluated via extensive system-level simulations. To this end, the goal was to reduce the training time of the corresponding ML models via the MEC nature of the FL approach. In particular, a DNN model trained in different edge devices (located at the RNs of the cellular topology) was considered in order to predict the best-performing RN for each user initially dropped by the primary system. The key novelty of the presented approach is that both EE, SE—as network metrics—and training time and accuracy were considered during DNN training and model aggregation. According to the presented results, both EE and SE levels can be significantly improved as RN edge devices execute a portion of the ML training task compared to CL-based approaches. Moreover, training accuracy is slightly improved by the FL method, while training time is much less.

As far as future work is concerned, a potential technique that may influence the proposed scheme even more is the implementation of DRL methods for RN selection. These methods promise more accuracy gains as they interoperate with the cellular environment and, thus, training accuracy can be further improved. In this framework, future work concerns among others the utilization of the deep Q-Learning scheme proposed in [18] in the FL framework presented in this paper. Moreover, on-device ML task execution is also of high interest as different mobile devices with different capabilities can be evaluated. Finally, B5G/6G non-terrestrial (NTN) scenarios are planned to be added to our wireless network simulators.