6G Wireless Communications and Artificial Intelligence-Controlled Reconfigurable Intelligent Surfaces: From Supervised to Federated Learning

Abstract

The new generation of wireless communication technologies is already in development. Sixth Generation (6G) mobile communications are designed to push the limits for more bandwidth, more connected devices with minimal power requirements, and better signal quality. Previous technologies used in Fifth Generation (5G) are inadequate to handle the new requirements alone. One of the proposed solutions is the use of Reconfigurable Intelligent Surfaces (RISs). These surfaces, when combined with Artificial Intelligence (AI), may be a very powerful means of achieving this. In this paper, we review studies that focus on the use of RISs controlled by AI in determining the concept of Smart Radio Environment (SRE) for use in 6G wireless networks. We examine applications that span from Supervised to Federated Learning (FL) as enabled by the rise in Edge Computing. As the new generation of mobile devices is expected to have enhanced capabilities to perform computing and AI locally, thus reducing the need to transfer the data to a central hub, more opportunities are created for the extensive use of FL. In this context, we focus on research in FL as used in RIS-aided SRE.

1. Introduction

In today’s world, mobile devices and applications are omnipresent, so much so that they have reshaped our society, especially the way people work, have fun, or even communicate. This change took the world by storm, increasing the requirements for more data traffic and more connected devices. By 2030, the latter are expected to reach 17.1 billion and the required data traffic may exceed 4300 Exabytes/month [1]. These numbers are significantly larger than the current capacities of our networks. Current technologies, such as massive Multiple-Input Multiple-Output (MIMO), Ultra-Dense Network (UDN), and millimeter Wave (mmWave) [2], are not enough to satisfy the requirements and novel approaches; thus, the employment of unconventional techniques may be needed. This leads to a next generation of wireless networks that will be able to address all new challenges and offer the necessary resources that will bring the future of telecommunications to the envisioned level of novelty.

The advent of 6G wireless networks represents a paradigm shift in telecommunications, characterized by the unprecedented integration of AI and advanced radio technologies. This evolution is not merely an incremental improvement from 5G but also a revolutionary leap towards intelligent, adaptive, and highly efficient communication systems. The synergy between AI and wireless communications in 6G is expected to enable novel applications such as holographic communications, tactile internet, and brain–computer interfaces, pushing the boundaries of what is possible in digital connectivity.

The discussion on future wireless networks (such as 6G) has already begun since 2018, even before the full implementation of 5G [3]. The next generation of wireless communications aims to unite the physical, digital, and biological worlds [4]. This will result in increasing Internet of Everything (IoE)-connected devices, from sensors and wearables to discreet devices such as phones and tablets. The application test-cases requirements for these devices alone set a new milestone in wireless communications. Moreover, a different network infrastructure is required, i.e., to unite all communications, from terrestrial to satellite, into a single entity. The nature of communications will also drastically change because a large number of these connected devices require Ultra Reliable Low Latency Connections (URLLCs) since the quality of the connection is critical for the applications.

To accommodate these changes, new frequency bands are required ranging from above 100 GHz to 300 GHz [5]. These frequencies allow for even smaller devices and require less energy to operate. However, they come with a significant caveat, i.e., the signal degradation is significantly higher. At these wavelengths electromagnetic (EM) waves suffer from high penetration and propagation losses, as well as higher absorption rates and shifts. This, along with the fact that the number of connected devices is expected to be vast, prompted researchers to consider using the radio environment to their advantage.

Until recently, the radio environment was considered a black box that could not change. In wireless communications, designers adopt two distinct approaches: (a) Shannon’s approach, according to which the model is static and can be formulated in terms of probabilities, requiring a proper initial planning [6], and (b) Wiener’s approach where the model is still the same, but the input is optimized according to feedback from the system’s output [7]. Some notable attempts have been made to use the nature of the radio environment as an advantage. Index modulation technologies, such as spatial modulation, scattering media modulation and media-based modulation, have been employed, albeit in all the aforementioned technologies the radio environment is used as is, with these methods just exploiting its characteristics [8,9,10].

One of the suggested solutions is more groundbreaking than attempting to solely leverage the characteristics of the propagation environment. Such an approach pursues the dynamic control of the radio propagation itself by creating SRE [11]. The latter is not static, but it can be transformed into such ways to facilitate communication between the transmitter and the receiver. Furthermore, it can be programmed according to ever-changing needs [12]. The possibility of controlling the signal after it leaves the transmitter and before it reaches its destination has for years been the holy grail of wireless communication engineers. The main objective is to introduce wireless communications into the new era of Smart Wireless Communications [13], that of Wireless 2.0 [14].

The concept of SRE, enabled by RISs, represents a paradigm shift in wireless communications. Although there have been solutions in the past (e.g., multi-antenna relays and phased antenna arrays) in order to control the transmitted signal along its route to its destination, such approaches are associated with many drawbacks (significant increase in the initial implementation cost, energy and maintenance costs, etc.). Thus, it is expected that the novel paradigm of RISs [15] may provide a preferable and cost-effective approach for the control of the radio propagation environment in a smart way.

Major benefits of RISs, as evidenced by the research results of the last few years, are related to cell coverage, communication data rates, physical security, etc., as well as to promising key use cases of RISs in various frequency bands. Thus, considering the 6G capabilities defined by the ITU-R [16],

Table 1. Capabilities of 6G compared to 5G.

Feature	5G	6G
User experienced data rate	Up to 200 Mbps (20 Gbps peak)	Up to 10 Gbps (1 Tbps + peak)
Coverage	Primarily terrestrial with limitations	Ubiquitous 3D coverage through space-air-ground integration (99%)
Connection density	Up to 1 million	106–108 devices/km2
Latency over the air interface	1–10 ms	0.1–1 ms
Frequency bands	Up to 100 GHz	0.1–10 THz
Security capacity	Robust (security-as-an-addition)	Enhanced (security-by-design)
New usage scenarios	Enhanced mobile broadband IoT Mission-critical applications	Integrated AI and Communication Ubiquitous Connectivity Integrated Sensing and Communication

depicts a selection of the foreseen 6G capabilities (compared to 5G) that will be positively influenced by the exploitation of RISs. It should be noted, however, that the (range of) values given for the capabilities in Table 1 are estimated targets for research and investigation of IMT-2030 (6G).

RIS technology offers the ability to manipulate electromagnetic waves in real-time, effectively turning the wireless environment into a programmable space. This capability, when combined with AI, creates a symbiotic relationship where the physical layer of communication becomes adaptive and intelligent. AI algorithms can optimize the configuration of RIS elements to enhance signal quality, mitigate interference, and improve overall network performance. For instance, deep RL may be employed to dynamically adjust RIS configurations based on changing channel conditions and user requirements, leading to significant improvements in spectral efficiency and energy consumption [17].

In this context, the integration of AI in wireless communications, particularly in the realm of 6G, becomes crucial. AI techniques, ranging from supervised and unsupervised deep learning to reinforcement learning (RL) and FL, offer powerful tools to address the complex challenges posed by next-generation networks. Supervised deep learning, for instance, may be employed for precise channel estimation and beamforming in high-frequency bands, while unsupervised learning algorithms may identify patterns in network traffic and optimize resource allocation dynamically. RL holds promise for adaptive network management, allowing systems to learn optimal policies for routing, spectrum allocation, and power control in real-time. FL, a decentralized machine learning (ML) approach, emerges as a key technology for preserving user privacy while enabling collaborative learning across distributed devices, crucial for the IoE paradigm [12,13,14,15,17].

In essence, the realization of SRE in the Wireless 2.0 era will be enabled by the synergistic integration of AI within the 6G technology, in line with the “Integrated AI and Communication” usage scenario of the IMT Framework for 2030 (6G) [16]. In such a framework, the evolution of ML models in a RIS-aided 6G communication system along with their critical analysis concerning their strengths, limitations, and challenges presented in this paper, is of great interest for future research efforts.

As we are becoming closer to the standardization of 6G, this paper aims to provide a comprehensive overview of the incorporation of AI and advanced ML techniques in a RIS-enabled wireless communication environment and constitutes a contribution for enabling the deployment of AI/ML towards the realization of a 6G native AI system. We present a framework that leverages the strengths of both centralized and decentralized ML methods, tailored specifically for the unique requirements of RIS-enabled 6G networks. By exploring the transition from supervised to FL in the context of RIS-aided networks, we seek to uncover new methodologies for enhancing network performance, security, and efficiency. Our study investigates the potential of these technologies to leverage opportunities, as well as to address key challenges, in 6G implementation, including ultra-high data rates, massive connectivity, and ultra-low latency. Our research lays the groundwork for future investigations into these novel possibilities that pave the way for the next generation of wireless innovation.

2. SRE Empowered by RISs

RISs represent a revolutionary approach to wireless communication that transforms traditionally passive wireless environments into programmable spaces. They are programmable structures consisting of numerous sub-wavelength elements that can dynamically manipulate electromagnetic waves. These elements may be configured independently, leading to the formulation of a surface that can alter in real-time the amplitude, phase, and polarization of the impinging wave, effectively creating customized propagation environments by reflecting, refracting, or absorbing waves in controlled directions [12,15,17].

The fundamental principle behind a RIS is to create favorable propagation conditions by intelligently reflecting or refracting incident signals towards desired directions while suppressing interference in others. This capability enables the RIS to establish virtual line-of-sight paths in non-line-of-sight scenarios, effectively bypassing obstacles that would otherwise block mmWave and terahertz frequencies. In other words, RIS has the potential to modify the environment in such a way that the environmental objects, which are typically considered as obstacles, are now used to form an EM response, thereby contributing to the communications channel and improving communication. By controlling the electromagnetic response of the environment itself, RIS can extend coverage, enhance signal quality, improve spectral efficiency, and reduce power consumption across the network. The implementation of a RIS includes arrays to reflect signals, liquid crystal surfaces, and meta-surfaces that are software controlled. The use of such surfaces may be complementary to other techniques, such as spatial modulation, by working together for signal improvement.

Unlike conventional relays or repeaters that actively process and amplify signals, RISs operate in a nearly passive manner, requiring minimal power only for reconfiguration control. However, RISs are not necessarily passive. There are three different classifications in respect of their way of functioning, i.e., passive, dynamic and active RISs. The former are programmable once, before installation, and they cannot be reprogrammed easily. The dynamic RISs are surfaces that can be easily reprogrammed to fit different scenarios; thus, they can provide a true SRE. The third category consists of active RISs that do not just reflect but also amplify the signal and can take the place of relays, with their own advantages and disadvantages. In this work, we focus on dynamic RISs and their use for creating the SRE.

RISs are not the only way to create an SRE. Alternative methods/techniques include relay-aided transmission and backscatter communications. In general, an SRE can be represented as shown in Figure 1. According to the latter, various network assets (such as base stations, mobile stations, satellite service stations, aircraft, and aeronautical communications) may be reconfigured by a control system. Data are being collected from a network of sensors or by feedback from the wireless network itself. These data are processed to extract the Channel Status Information that is used by the control system. SREs do not necessarily require a RIS to operate, although the RIS-approach is one of the most promising [15,17]. Its primary advantages over other solutions include low deployment cost, as it is relatively cost-effective, as well as its passive nature, which means that it does not require any power to operate. Other advantages are full band compatibility, as it can be programmed to work at various frequencies, and ease of deployment as it is a thin surface that can be mounted anywhere, i.e., from buildings to even clothes. The aforementioned advantages offer the possibility of creating complex solutions at minimal cost. New challenges are also set by these surfaces regarding their deployment, as there are still challenging aspects as regards their modeling. Accurate RIS models are lacking as in most cases they are assumed to be perfect reflectors, which is not completely true; a validated channel model is also lacking [17].

The most important challenge in implementing an SRE is the maintenance of a perfect balance between cost and effectiveness. As network designers work with limited resources, creating the ideal configuration is a more complex problem than prior designs. The first and most critical issue is that the nature of the design optimization problem for the creation of energy efficient SREs is non-convex [18]; thus, the solution may be computationally demanding. Another important issue is the ever-expansive complexity that is being introduced by the nature of the IoE. The number of devices and their presence and movement is more fluid than ever, as an increased number of static sensors and a large number of highly dynamic drones come into play.

There have been studies contributing to the simplification of setting up the RISs [19,20]. While the aforementioned methods simplify the use of the surfaces to constitute the technology usable, the design of a complete SRE is far more complex [21]. The number of parameters that we need to consider, and the fluidity of contemporary and future communication networks render such calculations at least computationally expensive if not impossible. Apart from the greater number of devices that are on the move and need to be connected constantly, the increasing number of obstacles has to be considered, too. As higher frequencies are used to increase the channels’ data throughput, we come across greater attenuation and other signal degradation phenomena, because higher frequencies are more susceptible to the effects caused by physical objects. As there is no effortless way to solve this design issue, engineers have steered their attention towards the use of AI to design the SREs. Figure 2 presents the simplest implementation of RIS-aided communications controlled by AI. The “AI Control” unit of Figure 2 is actually an AI-based joint optimizer; the latter performs the total optimization of the system by intervening to the transmission channel through RIS (which is the main objective of a SRE), by taking into account both the base station (BS) and the mobile device. The necessary Channel State Information (CSI) may be provided to “AI Control” either from the BS or from certain cell units of RIS that operate as sensors.

3. Use of AI in Wireless Communications

For many years computers have assisted humanity in solving a lot of challenging and time-consuming tasks. However, most of these tasks like solving an equation, beating humans at a strategy game, or in our case calculating the best BS placement for optimal coverage do not require any intelligent behavior from these systems. When AI came into the spotlight and the means to implement it were readily available, its potential applications were considered in all disciplines including telecommunications. AI has been used in the telecom industry [22,23,24,25,26,27,28,29] in multiple ways.

(1)

Network Optimization: AI algorithms are used to analyze reception feedback and predict needs before they arise, optimize the network topology as well as improve efficiency and performance.

(2)

Customer Service: AI is used to provide customer support with personalized interactions by employing virtual assistants and chatbots.

(3)

Fraud Detection: AI is used to analyze the behavior of users and patterns in data in order to detect fraudulent or exploitative use of the network and protect both the users and the providers.

(4)

Targeted Marketing: AI is used to create customized advertising and targeted marketing campaigns to maximize customer engagement and reach.

(5)

Security and Surveillance: Like fraud detection, AI is targeting security threats such as cyberattacks against user data and networks.

One of the most important applications of AI in telecommunications is Network Optimization. In the past, this term used to mean the configuration of network components to improve some basic characteristics, such as power or coverage. In 6G wireless communications, a huge next step is taken; it consists of applying the AI for the continuous reconfiguration of the hardware used in order to compensate for demands in the network like more connections or higher data speeds. This is envisioned as Intelligent Radio [22]. Nevertheless, the possibilities are even greater, as the environment itself can now be configured [30] to play an important role in the creation of Intelligent Radio. This is made possible with the creation of SREs. These environments are envisioned, as mentioned above, with the use of RISs.

4. RISs Controlled by AI

As the use of RISs for the creation of SRE is being examined, the problem of dynamically controlling the RIS occurs. This issue has been studied thoroughly, and as previously mentioned, one of the most promising ways of achieving this is through AI. There are two main approaches that most researchers have chosen to follow as described in [14]; (a) training-based and (b) training-free techniques. On the one hand, in training-based methods, the models are trained by using an existing set of data and can be reprogrammed according to different observations. These models can be developed by using either supervised or unsupervised learning approaches. Both approaches require that the dataset is large enough and the environment is not expected to change significantly. On the other hand, in training-free methods, the models are trained through feedback data collected by sensors or devices themselves and are continuously changing and improving to dynamically cover the needs of the network as they change. The most common method used is RL.

The integration of RISs into next-generation wireless networks promises to address key challenges in high-frequency communications while enabling more precise spatial multiplexing and beamforming. Deployment scenarios include mounting these surfaces on building facades, interior walls, ceilings, or even incorporating them into smart clothing and vehicles. Research is advancing towards making these surfaces increasingly intelligent through integration with AI, allowing them to autonomously adapt to changing environments and user needs while anticipating optimal configurations based on predicted mobility patterns and traffic demands.

4.1. Supervised Learning

Supervised learning is mainly used to solve problems for which we have significant amounts of data, and their future conditions are not expected to change considerably. This method can be very accurate when the necessary (big) dataset is available, but the caveat is that it must be labeled.

Table 2. Selection of research works using Supervised Deep Learning.

Ref.	Problem Description	Algorithm Used	Achievement
Taha et al., 2019 [31]	Reduction in the number of activated elements in a very large RIS	Multi-layer perceptron	Optimization of element activation/deactivation
Yang et al., 2021 [32]	Reduction in interference	Intelligent Spectrum Learning based on Supervised Deep Learning	Distinguish desired from interfering signals
Lu et al., 2022 [33]	Maximization of sum-rate for RIS hybrid precoding architecture	Multiple Discrete Classification based on Supervised Deep Learning	Reduction in runtime without any significant performance degradation
Zhang et al., 2021 [34]	Addition of signal processing units for RIS to overcome the difficulty of acquiring CSI	Backpropagation algorithm based on Supervised Deep Learning	Real-time channel state information acquisition

summarizes four different applications of Supervised Deep Learning in RIS, for wireless communications.

Deep Learning has been applied in order to reduce the number of activated elements in a large RIS system. A multi-layer perceptron (MLP) has been employed to optimize element activation and deactivation, achieving energy-efficient operation without significant performance degradation [31]. A Supervised Deep Learning approach has been used in [32] to reduce interference in RIS-assisted networks. The proposed method, termed as Intelligent Spectrum Learning (ISL), enables RIS to distinguish desired signals from interfering ones, improving SINR and system reliability. Moreover, a hybrid precoding architecture for THz communications has been presented, utilizing Deep Learning-based Multiple Discrete Classification (DL-MDC). The goal of this approach, compared with existing solutions, is the maximization of the sum-rate while significantly reducing runtime with negligible performance loss [33]. The last line of Table 2 refers to a study that focuses on overcoming the difficulty of acquiring CSI in RIS systems. A supervised learning-based backpropagation algorithm is employed, enhancing system efficiency and enabling real-time CSI acquisition [34].

The brief presentation of the previous paragraph demonstrates the contribution of each study to RIS-enabled wireless communications and showcases the potential of supervised deep learning for addressing computationally complex problems in real-time adaptive solutions. Energy efficiency is emphasized by reducing hardware demands and computational overhead [31,33], while the improvement of the system’s performance is achieved either by managing interference [32], or by optimizing sum-rate [33]. However, the reliance on supervised learning requires extensive labeled datasets, which may not always be feasible in dynamic wireless environments, a limitation that may be overcome by employing FL techniques, as will be shown further down.

4.2. Unsupervised Learning

Unsupervised, unlike supervised learning, does not require costly labeled datasets, as it uses a self-classification approach. Thus, its use is most appropriate for diverse datasets and/or unstructured data, or, generally, ineffectively labeled data. It may also be used to identify correlation between data and offers valuable insights. Unsupervised learning, when applied to RIS-enabled systems, may work well with big datasets, since they do not need to be previously labeled or structured.

Table 3. Selection of research works demonstrating Unsupervised Deep Learning.

Ref.	Problem Description	Algorithm Used	Achievement
Gao et al., 2020 [35]	Passive beamforming optimization in RIS-assisted systems	Customized deep neural network trained using unsupervised learning	Prediction of phase-shifts in real-time by using unlabeled collected data
Song et al., 2021 [36]	Beamforming in active and passive MISO systems	Two-stage Unsupervised Deep Learning	Optimization of both transmit beamforming and RIS phase shifts with reduced computational time compared to traditional iterative approaches
Al-Shaeli et al., 2023 [37]	Passive beamforming for RIS-aided MIMO systems	Neural network—Unsupervised Deep Learning	Low implementation complexity and greater time efficiency than conventional programming strategies

offers a view to three key studies on unsupervised deep learning for RISs.

All three works of Table 3 demonstrate the potential of unsupervised learning to address the inherent non-convexity and computational challenges of RIS beamforming and highlight the feasibility of deploying RISs in real-time wireless communication systems, particularly for latency-sensitive applications like 6G networks. Ref. [35] addresses the complexity of passive beamforming optimization in RIS-assisted systems. A customized deep neural network training using unsupervised learning is proposed in order to predict phase shifts in real-time. The achieved performance is comparable to semi-definite relaxation (SDR) methods albeit with significantly reduced computational complexity. The latter is a common asset for all approaches [35,36,37], since they manage to reduce significantly computational cost compared to traditional iterative methods; thus, they are suitable for real-time applications. The two-stage unsupervised learning algorithm presented in [36] applies to active and passive beamforming in multi-user multiple-input single-output (MISO) systems, while a deep neural network architecture tailored for passive beamforming has also been proposed for RIS-aided multi-user MIMO systems [36].

Unsupervised deep learning has demonstrated significant promise in optimizing RIS-enabled wireless systems by reducing complexity and eliminating the need for labeled datasets. However, unsupervised methods may be limited by centralized training constraints and transitioning towards FL could unlock new opportunities for scalable, efficient, and privacy-preserving solutions in the era of 6G communications.

4.3. Reinforcement Learning

RL is one of the most popular methods for training a model for RISs. The most important advantage of RL is adaptability as it does not require training on existing data because RL models learn and improve after deployment. Thus, the algorithm may be tailored to specific requirements of each scenario. Moreover, implementation of RL tends to be easier to deploy and integrate in existing networks.

A plethora of algorithms have been used for the application of RL in RIS-related scenarios. On the one hand, there exist value-based algorithms, such as Deep Q-Network (DQN) and Double Deep Q-Network (DDQN); these algorithms are rather simple in architecture, but as they work with values and actions they are constrained in specific values and actions spaces. On the other hand, policy-based algorithms are more complex in architecture by using optimal policies to make decisions; thus, they perform better in dynamic environments. Examples of such algorithms are the Deep Deterministic Policy Gradient (DDPG), the Proximal Policy Optimization (PPO) and the Twin Delayed DDPG (TD3). The latter may also be considered as hybrid. The studies presented in

Table 4. Selection of research works demonstrating RL (various approaches and different algorithm types).

Ref.	Problem Description	Algorithm Used	Achievement
Puspitasari and Lee, 2023 [42]	Review paper	Deep RL	Potential of Deep RL for RIS technologies
Wang and Zhang, 2022 [43]	Real-time phase control of IRS for SRE	Deep RL with DDQN	Model-free control of IRS and improvement of its adaptivity to different channel dynamics
Saleem et al., 2023 [38]	Network Security for IoT communications under trusted-untrusted device diversity	Deep RL with a DDPG	Maximize security for trusted devices while maintaining the QoS of all devices
Huang et al., 2020 [39]	Joint design of BS beamforming matrix and IRS phase shift matrix in Massive-MIMO wireless Communications systems	Deep RL with DDPG	Optimal performance in complex communication environments with high efficiency
Hashemi et al., 2022 [40]	Phase shift design for RIS-aided URLLC systems	Deep RL with TD3	Reduced overestimation of the action-value function that comes with DDPG
Huang et al., 2022 [44]	Optimization of multi-hop RIS-aided cooperative networks	Deep RL with PPO	Maximization of data rates in IoT applications
Nguyen et al., 2022 [41]	Wireless power transfer and RIS-assisted communication with IoT and UAVs	Deep RL with DDPG and PPO	Maintenance of power efficiency while addressing communication quality in dynamic environments

showcase diverse applications of RL, demonstrating the potential of the latter in addressing complex optimization problems in RIS-enabled systems and addressing challenges ranging from security enhancement [38] to beamforming optimization [39,40] and energy efficiency [41].

Puspitasari et al. [42] provide a broad perspective on the potential of Deep RL for RIS technologies, offering foundational insights. In [43], the conventional approaches have turned out to be insufficient in a RIS-aided SRE. Thus, a model-free design for controlling the Intelligent Reflecting Surface (IRS), independent of the sub-CSI, has been proposed in order to create SRE. Deep RL has been applied to perform real-time coarse phase control of IRS, followed by extremum seeking control (ESC) for the fine phase control. Optimization of beamforming has drawn strong interest over the last years and a lot of studies have turned to the application of RL in RIS-aided systems. Optimal performance has been achieved for a BS antenna beamforming matrix in conjunction with RIS phase shift matrix in massive-MIMO systems by applying DDPG [39], while phase-shift optimization for URLLC systems has been effectively handled through TD3 [40]. The latter reduced overestimations commonly found in DDPG and provided robust performance under short-packet communications.

The potential of RIS to enhance IoT network performance has been leveraged by several researchers [38,41,44]. The IoT network security has been addressed in [38], by diversifying the devices as trusted and untrusted while maintaining the quality of service (QoS) of all devices. Furthermore, the maximization of end-to-end data rates in multiple RIS-aided multi-hop cooperative networks has been explored by applying the PPO algorithm with exceptionally good long-term results [44]; the study emphasized the long-term benefits of PPO for maximizing data rates in IoT applications. By combining DDPG and PPO, satisfactory power efficiency, while addressing communication quality, has been achieved in wireless power transfer systems and RIS-assisted communication to IoT devices and Unmanned Aerial Vehicles (UAVs) [41].

Table 4 indicates that the studies presented so far explore a variety of deep RL approaches, including value-based (DDQN), policy-based (DDPG, PPO), and hybrid (TD3) methods, offering a comprehensive toolkit for RIS optimization. Each algorithm is tailored to specific challenges, such as network security [38], trajectory optimization [38], and URLLC [40]. It is worth noting that all these algorithms are model-free; thus, they do not require explicit knowledge of the system model. This is particularly advantageous in dynamic and uncertain wireless environments. Moreover, algorithms like PPO [44] balance long-term system performance with computational feasibility, indicating readiness for real-world applications.

However, deep RL methods require extensive training, as highlighted in [40, 43], which may limit their adaptability to fast-changing environments. In addition, value-based algorithms like DDQN may suffer from overestimation issues, while PPO and TD3 demand fine-tuning and involve greater computational complexity. As regards the performance of the aforementioned approaches, all studies have reported notable superiority over conventional methods, highlighting the effectiveness of RL approaches in RIS applications. While the approaches presented so far show promising results, most of them focus on relatively small-scale scenarios; thus, their scalability for larger, more complex networks and/or higher numbers of RIS elements remains a challenge.

5. The Use of AI for the Creation of SRE

In the previous section we studied the way RIS can be controlled by using AI, even in dynamic environments. However, having a small number of RIS elements to create a small-scale SRE is completely different than creating one that covers a massive area such as a city. In this situation, we have multiple environments with diverse characteristics [45], and they must all be coordinated together. Thus, two questions have to be resolved: (a) the placement of the RISs and (b) the channel estimation.

The placement of RISs is particularly important because it plays a key role in the final network design. However, it may be considered as a limited intervention parameter since the actual placement positions are basically denoted by the city’s architecture and buildings. It is an application-specific parameter, and it must be studied independently for each case.

The channel estimation over a massive network requires a lot of CSI. The required amount of data is so vast, and the time needed for its transmission is so impractical that a training session becomes rather unfeasible. Thus, a massive ML model able to be reconfigured in order to compensate for every minor change on the network may create a case of “butterfly effect”. To overcome this, efforts have been made to calculate the parameters of a network having imperfect CSI in the presence of multiple RISs in a dynamic environment [46].

In ML, we can also employ a learning method that breaks the larger model into a subset of smaller ones, each being calculated locally, keeping data where it is collected and reducing the needs for data exchange between the main hub and the local devices. This learning method is termed as FL [47].

5.1. Federated Learning

We are currently in the era of Edge Computing, where applications are executed locally on the end users’ devices, while the core service and processing are performed on cloud servers [48]. This offers two main advantages. The first one is privacy because local private data are not sent to central servers; the second one is responsiveness since data are being processed in the device where they reside. This also benefits the networks as less data needs to be transported.

Being in the era of Edge Computing affects all types of devices and applications, especially AI. More and more devices are capable of supporting AI applications locally, as advanced CPU and SOC designs come integrated with specific AI accelerators. From mobile phones, sensors, and personal computers to IoT, AI applications can run locally, either partially or completely, with a lot of progress made in this field over the last years [49].

One of the most important relevant learning methods is FL. FL relies on multiple entities called clients to collaborate on the training of a central model while all training data remain decentralized. This leads to individual smaller AI models controlling parts of a communications design while providing and receiving feedback to a global AI model handling the coordination. A high-level depiction of how FL may be implemented in the context of a RIS-aided 6G communication system, is presented in Figure 3. The blue lines, which connect the global large AI model (BS) with the local AI clients (User Equipment-UE), represent the BS-to-RIS and RIS-to-UE links. These links serve the communication rounds needed for the training of the global large AI model at the BS and the update of the local AI client models residing on the UE. This allows the network designer to create smaller models that control only a part of the network that contains a single RIS or a small number of them. Thus, the calculations become less time- and power- consuming and security is improved. FL is not only used in conjunction with RISs. There are also a lot of applications focusing on the creation of edge computing environments; for example, on multiple drones’ control, a swarm of drones can be centrally controlled as a single entity, and each drone can also be individually controlled locally. These applications can also benefit from the use of RISs for enhancing the capabilities of the network. The integration of RISs with FL in 6G wireless communications represents a significant advancement in the field. In this paper we focus on the implementation of FL on RISs and RIS-aided SREs. A selection of recent works is provided in

Table 5. Selection of research works demonstrating FL.

Ref.	Problem Description	Algorithm Used	Achievement
Das et al., 2023 [50]	Review paper	FL, FL with Deep Deterministic Policy Gradient (FL-DPPG)	Application of ML techniques in RIS-enhanced systems (focus on IoT)
Sejan et al., 2022 [51]	Review paper	ML techniques	Application of ML techniques in RIS-enhanced systems (focus on 6G)
Xiao et al., 2024 [52]	Review paper	OTA-FL	Application of ML techniques in RIS-enhanced systems (focus on OTA-FL performance and data privacy)
Zhou et al., 2024 [53]	Review paper	FL	Application of ML techniques in RIS-enhanced systems (focus on optimization approaches)
Ma et al., 2020 [54]	Privacy Preservation	Optimal Beam Reflection (OBR) based on FL	Minimization of centralized data dependencies to ensure robust user privacy
Li et al., 2020 [55]	Preservation of the privacy of user data in a RIS aided mmWave system	FL	Local models are trained, encrypted, and sent to the central server while ensuring private data remains localized on devices
Wang et al., 2022 [56]	Reduction in the fading channels effect	FL	Lower training loss and higher test accuracy
Yang et al., 2022 [57]	Poor spectrum efficiency in RIS-aided networks	FSL	Better spectrum prediction accuracy and enhanced system utility
Shen et al., 2022 [58]	Great transmission overhead that occurs in RIS-aided communication systems for the collection of Channel Status Information	FDReLNet	Significant reduction in transmission overhead
Elbir et al., 2022 [59]	Great transmission overhead in MIMO and RIS-aided MIMO systems	FL	Transmission overhead 16 times lower when compared to Centralized Learning
Zhong et al., 2022 [60]	Users sum rate optimization in a RIS-enhanced NOMA wireless network	FL-DDPG	Improvements in sum rate, training time, and system reliability
Hu et al., 2021 [61]	Minimization of the energy consumption of devices in a RIS-aided multi-antenna BS system	FL	Energy consumption reduced by 12% compared to FL without the presence of RIS
Chaaya et al., 2023 [45]	Model training on multi-RIS. Poor performance of heterogeneous communication and OoD environments	FL combined with Nash game theory	Up to 15% better performance in OoD Environments

.

The surveys listed in Table 5 provide a broad perspective on the application of ML techniques in RIS-enhanced systems, with a particular focus on IoT, FL and optimization approaches [50,51,52,53]. In [50], a detailed examination of ML-based RIS-enhanced Internet of Things (IoT) systems is presented and the potential of RIS to enhance IoT network performance is highlighted. A review of ML techniques for intelligent reflecting surface (IRS)-based wireless communication towards 6G is offered by Sejan et al. [51], emphasizing their role in channel estimation, beamforming optimization, and resource allocation. However, one primary challenge identified in both in [50,51] is the complexity of channel estimation in RIS-enhanced systems, particularly in dynamic environments. Although ML techniques show promise in addressing this issue, they often require substantial training data and computational resources, which may not always be available in practical scenarios. The over-the-air- (OTA-) FL is discussed in [52], and a comprehensive review of its current status, challenges, and future directions is provided. The potential of OTA-FL in preserving data privacy while enabling collaborative learning across distributed devices is highlighted, whereas the dependence of its performance on wireless channel conditions and device heterogeneity is examined, since these factors may cause model convergence issues. The optimization approaches surveyed in [53] are categorized into model-based, heuristic, and ML methods and are analyzed in the context of various RIS applications, such as coverage extension, interference management, and energy efficiency enhancement.

The other studies summarized in Table 5 provide various insights into the integration of FL within RIS-enabled wireless communication frameworks. The applicability of FL to heterogeneous and out-of-distribution (OoD) environments has been examined and up to 15% better performance has been achieved by leveraging Nash equilibrium strategies [45]. FL architectures that minimize centralized data dependencies have been proposed in order to protect user data during beam reflection and channel estimation [54,55]. The aforementioned approaches handle the problem of preserving the user’s data privacy, which is a fundamental requirement in 6G, by using small datasets to train local models that are sent to the central server to create a global model, while they ensure that private data remains localized on devices.

Other applications of FL in RIS-aided networks include channel estimation and spectrum optimization [56,57,58,59]. In [56], FL was employed to reduce the fading channels effect, resulting in lower training loss and higher test accuracy. A similar problem has been tackled in [57] by using Federated Spectrum Learning (FSL), while a novel Federated Deep Residual Learning Neural Network (FDReLNet) has been introduced in [58] to predict channel state information in RIS-aided networks. A significant reduction in transmission overhead has been observed due to the application of FL [58,59] and the employment of FDReLNet has led to better spectrum prediction accuracy and enhanced system utility, albeit slower convergence has been observed, compared to centralized learning approaches [57]. Improvements in users’ sum rate, training time and system reliability have been reported in [60] by employing FL-DDPG in a RIS-enhanced non-orthogonal multiple access (NOMA) network, whereas FL has also been used to minimize the energy consumption of users connected to a RIS-aided multi-antenna BS [61].

The studies presented so far underline FL’s versatility in addressing various use cases in RIS-aided wireless systems, like privacy-preserving, communication efficiency, and scalability. Nonetheless, further innovations, such as hybrid architecture, stronger security models, and more rapidly convergent algorithms, may be accommodated in the combination of 6G wireless technologies and AI, which is enabled by FL.

5.2. Recent Developments

AI and ML have been in the spotlight for quite some time now and they have started to decline from the over-hyped phase into a more stable one, solidifying the research in the field. The same holds for the employment of AI and ML, combined with RISs, in wireless communications. As research solidifies, the following directions draw strong interest.

Traditional approaches, such as RL, keep developing [62,63] in parallel with more innovative schemes such as the use of Large Language Models [64]. Security issues in RIS-enabled 6G systems still concern the research community and especially for frequencies as high as THz [65] or for Ground-to-Air IoT [66]. In addition, channel estimation in RIS-assisted communication systems has been achieved by employing a methodology that integrates principal component analysis (PCA)-based clustering with FL, tailored for heterogeneous users. The proposed technique has led to a significant reduction in the communication overhead, while enhancing the system security [67].

Furthermore, the feasibility of a RIS-aided Federated Edge Learning system has been investigated under the realistic supposition where only the statistical channel state information is available among devices [68]. FL has also been applied on the Internet of Vehicles; a RIS-assisted FL framework has been introduced in order to enhance communication reliability and diminish the straggler effect caused by heterogeneous computing and communication resources of vehicles [69]. A protocol that combines RISs, FL, and edge computing in order to enhance network performance in IoT-enabled mobile ad hoc networks with cell-free massive MIMO has been proposed in [70] with promising results as regards packet delivery ratio, routing delay, and control overhead. Recently, reconfigurable intelligent metasurfaces (RIMs) have emerged as a revolutionary technology, which, when combined with distributed learning mechanisms and especially FL, may offer a controllable means for the solution of key problems in RIM-aided wireless networks such as channel estimation, passive beamforming, and resource allocation [71].

6. Conclusions and Future Research

The survey presented so far has demonstrated that the transition from supervised to FL has offered several advantages. FL preserves data privacy because it enables decentralized training across devices without sharing raw data. Moreover, FL can handle larger datasets by distributing computation across edge devices. This decentralized architecture makes FL more robust and less vulnerable to single-point failures. Exchanging model updates instead of raw data leads to lower communication overhead and reduced bandwidth requirements. Thus, the communication burden is reduced significantly, compared to centralized learning approaches, particularly in large-scale RIS-enabled networks. Another asset of FL is its dynamic adaptation to heterogeneous environments, since it can continuously update models using local data unlike supervised models that require retraining for new environments.

However, FL introduces challenges that have to be addressed in order to fully leverage its potential in RIS-enabled systems. Although FL addresses privacy concerns, excessive data decentralization may lead to security vulnerability; the FL system may be susceptible to various attacks, such as model poisoning, which could compromise the integrity of RIS control mechanisms. Furthermore, convergence issues and model synchronization complexities may arise, potentially affecting the performance of RIS control algorithms. The model aggregation from devices, especially heterogeneous ones with varying computational capabilities and channel conditions, may also be a matter of concern as regards the final performance of the system.

It is beyond doubt that the transition from supervised to FL has led to more efficient, secure, and intelligent 6G wireless communication systems powered by AI-controlled RISs. Still, there remains ample opportunity for innovation and advancements in the field. Hybrid learning frameworks may be created by combining supervised and FL in order to balance centralized accuracy offered by supervised learning with decentralized adaptability found in FL. Emphasis may be given on developing FL techniques that enhance model generalization across heterogeneous environments and offer rapid adaptation of RIS control algorithms to new environments or user distributions. The possibility of constructing multi-level FL architectures that leverage the hierarchical nature of RIS-enabled networks may be investigated in order to allow for more efficient model aggregation and adaptation to local conditions. Furthermore, adaptive FL strategies may be developed by designing FL algorithms that dynamically adjust learning rates, aggregation frequencies, and client selection based on network conditions and RIS configurations. In addition, integration of blockchain technology with FL and possible incorporation of blockchain cryptographic hashes into FL algorithms may pave the way for increased security and trustworthiness of distributed learning in RIS-enabled networks. Finally, FL approaches that exploit the combination of multiple bands (e.g., mmWave, THz, visible light) in RIS-enabled systems may lead to greater beamforming accuracy and enhance channel estimation.

In conclusion, integrating FL with RISs offers great prospects; still, there is plenty of room for further investigation. FL-enabled RIS systems have the potential to revolutionize wireless network performance and user experience. Future research should focus on developing innovative FL algorithms tailored to the unique characteristics of RIS-enabled networks, paving the way for more efficient, secure, and intelligent 6G systems.