An Overview of Reinforcement Learning Algorithms for Handover Management in 5G Ultra-Dense Small Cell Networks

Abstract

The fifth generation (5G) wireless technology emerged with marvelous effort to state, design, deployment and standardize the upcoming wireless network generation. Artificial intelligence (AI) and machine learning (ML) techniques are well capable to support 5G latest technologies that are expected to deliver high data rate to upcoming use cases and services such as massive machine type communications (mMTC), enhanced mobile broadband (eMBB), and ultra-reliable low latency communications (uRLLC). These services will surely help Gbps of data within the latency of few milliseconds in Internet of Things paradigm. This survey presented 5G mobility management in ultra-dense small cells networks using reinforcement learning techniques. First, we discussed existing surveys then we are focused on handover (HO) management in ultra-dense small cells (UDSC) scenario. Following, this study also discussed how machine learning algorithms can help in different HO scenarios. Nevertheless, future directions and challenges for 5G UDSC networks were concisely addressed.

1. Introduction

Over the recent years, wireless technology is boosted with potential capabilities through research and innovation. The exponential increment of various wireless devices, more usage of data, improved quality of service, and the expansion of cellular network have gained importance. Main drivers are exponential increment of various wireless devices, data hunger applications, and providing improved quality of service/experience that required expansion of cellular network to support upcoming 5G use cases. This evolution of wireless networks include high speed in gigabits per sec, low latency, high throughput, and better efficiency of spectrum in contrast to 4G-LTE networks [1]. 5th generation of wireless technology supports all these use cases and can be achieved through heterogeneous behavior of networks [2,3]. Small cells are low-powered cellular radio access nodes and backbone of 5G wireless network architecture. Handover (HO) is a crucial challenge in presence of small cells paradigm such as number of barriers/hurdles may come between the user and small cells [4]. Hence, the small cell support from 10 m to a few kilometers range with mmWave and beamforming techniques that caused reliability and multiple HOs occurrence. Particularly, in the case of high-speed movement that may disrupt connection abruptly.

HO measures can be classified into four categories such that human obstacles in which pedestrians are the cause of HO. Moving obstacles, where passing vehicles are the cause of HO. While in rotations, hand movement and rotations of user is the cause of HO. At last category, fixed obstacles, the buildings, and constructions are mainly caused of HO [5]. Most important scenario for mmWave is the hotspot deployment in urban and semi-urban areas where Gigabit/s is the main concern of 5G wireless technology. In this regard following three main deployment scenarios are: street side service, stadium/concerts service, campus service. In street side scenario, the restaurants, shops, and pedestrians are the concerns where mmWave access point will deliver the required services with the underline effects of interference and signals blockage by large obstacles [6]. In campus/concerts type, the focus is the gathering places like seminar halls, concerts galleries, and hallways around meeting/classrooms etc. while in the case of stadiums such as football, tennis, cricket, rugby, and arena require high data rate for upcoming virtual, augmented, and mixed reality services.

Multiple technologies for 5G communication include massive multiple input multiple output (mMIMO), millimeter wave communication (mmWave), self-organized network (SON) and ultra-dense network (UDN) supports all these mentioned scenarios [7,8]. In 5G technologies, service operators and vendors experience different challenges for HO implementation. First challenge is imprecise signal measurement in which the bandwidth and frequency of mmWave is high which is essential to increase the capacity of network communication system. The signal path loss becomes high at high frequency bands due to atmospheric conditions, low diffraction around the walls or obstacles and rain absorption. These factors reduce the range of the signal. Moreover, signals operating at high frequency band can become a victim of fading easily which in turn creates error, lowering the overall switching rate of HO or unnecessary roaming. At all, it will reduce the customer’s experience. The next challenge is immediate HO where the size of the radius cell is smaller than the standard size in the UDN architecture. In every small cell, the total time interval of the user equipment becomes relatively small and caused frequent HOs. As a result, the overlapping time of two terminals also becomes relatively low. At last, 5G does not contain same network layers and technologies in contrast to 4G and 3G network communication and both vertical and horizontal HO process embrace HO issues in 5G [9,10]. Beside these challenges, there are number of other challenges exists due to heterogeneous communication behavior of network, merging of 5G technologies, and optimized automation of existing processes. Figure 1 shows the 5G NR communication system under the consideration of heterogeneous environment and required use cases i.e., eMBB, uRLLC, mMTC.

1.1. Motivation

Recently, several achievements and accomplishments have been perceived in the field of 5G mobile communication. This motivation encouraged many researchers and scholars to explore the machine learning techniques and its applications in the domains of upcoming wireless technology to help the communication system more intelligently. Deep reinforcement learning (DRL), convolutional and deep neural networks received more attention than basic algorithms. Their powerful optimization and convergence properties to make powerful 5G mobile communication system particularly in the case of UDSC networks.

1.2. Contribution

In this paper, a survey of HO management in 5G UDSC networks based on machine learning algorithms is presented. Summary of the contribution of this paper follows as: (1) a survey on HO management in 5G UDSC networks based on machine learning algorithms is provided. (2) A comparative analysis of machine learning based mobility management schemes is presented. (3) Comparative analysis identified and hits several open research challenges and future directions that must be acute for further wireless networks.

1.3. Road Map of the Survey

This paper presents a comprehensive review and analysis of machine learning based provisioning techniques for HO management in 5G UDSC networks environment. All included published papers are more closely to near future, many acronyms and symbols are used, tables and figures are also presented in the paper. The remainder of this paper is organized as follows. In Section 2, we provide the related surveys and summary of related surveys articles. In Section 3, we provide the preliminaries for 5G UDSC networks and the architectures as well as the mobility management and HO signaling, procedure, types, and requirements. Section 4 provides a discussion about how machine learning based algorithms helps HO management in 5G UDSC networks, we discussed reward base, policy based, values base strategies. We also present comparative summary of machine learning based HO management algorithms in this section. Then in Section 4, we present challenges and future direction in details and provide possible solutions. Finally, Section 5 concludes the paper.

2. Related Surveys

Number of surveys on mobility management in 5G wireless communication had already been published and cited in

Table 1. Summary of Related Surveys.

Authors	Improving Bandwidth Efficiency	Increasing Uplink Power Consumption	Quality of Service Provision	Efficient Spectrum Utilization	Mobility Handover	Reinforcement Learning
Yu	✓	✓	✓	✓	✓	✕
Luong et al.	✕	✕	✓	✕	✓	✕
Althamary et al.	✕	✕	✓	✕	✓	✕
Jiang et al.	✓	✕	✕	✕	✓	✕
Adedoyin and Falowo	✕	✕	✓	✕	✓	✕
tayyab et al.	✓	✕	✕	✓	✓	✕
Bithas et al.	✕	✕	✕	✕	✓	✕
Agiwal et al.	✓	✕	✕	✓	✓	✕
Van Quang et al.	✓	✕	✕	✓	✓	✕
Lee and Qin	✕	✕	✕	✕	✓	✕
Zaidi et al.	✕	✕	✕	✕	✓	✕
Ullah et al.	✕	✕	✕	✕	✓	✕
Mao et al.	✕	✕	✕	✕	✓	✕
Sharma et al.	✕	✕	✕	✕	✓	✕
Kibria et al.	✓	✓	✕	✕	✓	✓
Abdellah and Koucheryavy	✕	✕	✕	✕	✓	✓
Peng and Shen	✕	✕	✓	✕	✓	✓
Our Survey	✓	✓	✓	✓	✓	✓

. In [11], the survey reviews key mobility elements i.e., HO, cell selection, cell reselection and challenges of cellular communication due to transmitted power discrepancy and coverage area of cells in heterogeneous network (HetNet) deployment. Authors also introduces algorithms and strategies to mitigate these challenges. In [12], authors review the applications of DRL in modern cellular networks such as Internet of Things (IoT) and unmanned aerial vehicle (UAV) networks. Authors describe how network entities (UE, Drone) learn optimal policies for i.e., HO and offloading decision, cell and channel selection and reselection, reliable connectivity of multi-UAV network, and mobility pattern with the help of DRL. In [13], authors discussed research issues such as mobility management, resource allocation, data offloading and ultra-reliable low-latency communication (uRLLC) in multi-agent reinforcement learning (RL) framework for vehicle-to-everything (V2X) scenarios. Authors also discussed the prospective applications of MARL framework for decentralization and scalability in optimal decision policy. In [14], authors review the fundamentals of machine learning in 5G networks applications, i.e., small cells and heterogeneous networks, massive MIMO, massive MTC, energy harvesting, smart grid, and so on. Authors also emphasize the usage of machine learning algorithms for the future networks to explore the advance applications and services. In [15], authors present comprehensive review on 5G enabling technologies with UD networks. Authors also discuss the research challenges in intelligent management techniques and back haul solutions for 5G wireless networks such as unnecessary HO, unfairness in radio resource sharing, energy consumption, severe interference, and degraded quality-of-service (QoS). In [16], authors discuss in depth the mobility management in long term evolution (LTE) and 5G new radio (NR) with HetNets cellular networks. Authors also discuss HO performance metrics, HO failure types, differences from LTE to NR, mobility enhancers, and potential research challenges and techniques related to HO management. In [17], authors discuss UAV-based communications and characteristics with the help of ML techniques and present all relevant research works. Authors also consider physical layer issues such as channel modeling, interference management, transmission parameters configuration, physical layer security, resource management, and position related aspects.

In [18], authors discuss evolution of 5G wireless networks with architectural modifications combined with radio access network (RAN). Authors also discuss the mmWave technology comprising massive MIMO technologies, channel model estimation, beamforming algorithms, and directional antenna design. Authors focused on SON features, energy awareness and cost efficiency, QoS, QoE combined with the 5G evolution. At the end, authors discuss research issues with future directions, simulation experiments, relevant field trials, and drive tests. In [19], authors reviews HO algorithms for 60 GHz networks and with other bands. The comparison of HO algorithms is supposed for commendation of algorithm for each network. First, authors discuss 60 GHz based wireless systems then requirements, resource management, different schemes, and issues in 60 GHz based Wireless Systems. The research paper [20] focused on applications of DRL in resource management for 5G heterogeneous networks. Authors discussed the 5G architecture, heterogeneous networks, resource management functions for 5G HetNets. Then they discussed the DRL based resource management for 5G HetNets, comparative summary and analysis and open issues and future directions. In [21], authors discussed 3GPP based HO procedure with related KPIs, and challenges in UDSC mobile networks. Authors review the 5G mobility approaches, potential consequences of existing networks, technical challenges, and considerable opportunities using artificial intelligence in emergent UDSC networks.

In [22], authors discussed UAV communication, potential applications, and regulations. Author also discuss the UAVs standardization, energy harvesting technique, interference mitigation, optimal trajectory using DRL algorithms and propose number of regulations for UAV to make sure upcoming business opportunities. In [23], authors discussed the applications of DRL in communications and networking. Authors also discussed the advantages of DRL approaches, markov decision processes, RL, SARSA, deep learning and deep Q-learning, advanced deep Q-learning models, deep Q-learning for extensions of MDPs, network access and rate control, caching and offloading, network security and connectivity, preservation. In miscellaneous issues authors discussed the traffic engineering and routing, resource sharing and scheduling, power control and data collection, direction-of-arrival (DoA) estimation, signal detection, user association and load balancing, user localization, and access device detection. While in future research direction, state determination in density networks, knowledge of jammer’s channel information, multi-agent DRL in dynamic HetNets, training and performance evaluation of DRL framework. In [24], authors discussed the 5G for UAV and beyond communications. Authors explain the 5G based unmanned aerial vehicle types, single-tier drone’s deployment, multi-tier drone’s deployment, standardization of UAVs, 3GPP study item, 3GPP work item, UAVs standardization outside the 3GPP, cognition in 5G oriented UAVs, power optimization, security aspects, regulations, and open research issues.

In [25], authors review next generation wireless (NGW) radio technology for high data rates and new applications in adaptive learning and decision making environment using artificial intelligence tools i.e., machine learning to fulfill various requirements of underlying wireless network. Authors also discuss the 5G smart mobile terminals and transmission power considering their energy efficiency learning and autonomously access most meritorious spectral bands. At the end, authors discuss applications of 5G networks such as UDSC networks, massive MIMOs, device-to-device communications, and cognitive radios in the view of machine learning paradigm. In [26], authors discussed the 5G telecommunication networks and complexity of the processes of functioning by an order of magnitude compared to existing networks. Beside these issues, authors discuss the artificial intelligence support for these new applications of 5G wireless technology. In [27], authors discuss the learning and decisions making in vehicular networks using multi-agent system (MAS). The learning reliability of MARL is highly dynamic. Author also discuss the potential applications of MARL and research issues such as high mobility, uRLLC, resource allocation etc., in V2X scenarios.

3. Overview of HO Management in 5G UDSC Network

5G has almost same working condition as other cellular network has is composed of cell and sectors. Data can be transfer with the help of radio waves. These cells are connected either through wire or wireless with network at the back hand side. The encoding technique used by 5G technology is orthogonal frequency division multiplexing (OFDM) [28,29]. Mobile connectivity and mobile subscribers are continuously expanding to address service requirements, high data rates, changes in customer behavior and diverse use cases ranging from wireless nodes to robotics, chatbots and self directed transportation. Wireless industry and mobile operators need evolvement and unceasing growth to cope these challenges. Henceforth, by having a creative network design techniques and presence of advance modules, wireless industry can fulfill the exponential growth of data traffic which will reach to 77.5 exabytes per month by 2022 [30].

Likewise, GSM association estimated that total number of IoT devices will be around 25.1 billion by 2025 [31]. The statistics prove that existing mobile networks and operators are insufficient to complete the coverage and capacity demands of connected things. Mobile operating companies therefore must alter existing models, designs and operators to combat future challenges. Different research papers proposed the concept of MIMO to instigate the spectrum efficiency of 5G networks [32]. Similarly, mmWave communication was then introduced to improve the transmission bandwidth of 5G signals. But throughput and energy consumption were major challenges at that time.

3.1. 5G Technologies and Features

In 5G network, cells are divided into large and small cells to provide more network efficiency. Worldwide wireless web concept is used by 5G which is user-centric in contrast to service-centric used in 4G. By using user-centric concepts, 5G enable to support several applications and different services to connect the world. 5G mobile communication embed number of different technologies such as device-to-device (D2D), machine-to-machine (M2M), multiple-input multiple-output (MIMO), edge computing (EC), small cell (SC), beamforming (BF), convergence of WiFi and cellular, nonorthogonal multiple access (NOMA), subscribe software-defined networking (SDN), network functions virtualization (NFV) and channel coding (CC) [33]. 5G also introduce some features i.e., 5G architecture will include cloud computing, device centric and distributed system, provides higher data rates almost in gigabits, support large number of devices, utilization of battery is low, and it has low infrastructure cost [34]. To provide the ubiquitous communication in mobile systems, 5G include machine-to-machine communication technology which almost connect 100 billion devices. To provide higher throughput and gives higher spectrum efficiency, 5G uses MIMO technology [35]. 5G also take advantages of other technologies like OFDM, IoT, UDSC network, and mm Wave [36]. To gain the higher data rate and direct connectivity between devices, 5G use device to device communication technology. Succeeding four behaviors make sure D2D communication in 5G beyond scenarios i.e., device relaying with operator-controlled link establishment (DR-OC), direct D2D communication with operator-controlled link establishment (DC-OC), device relaying with device-controlled link establishment (DR-DC), and direct D2D communication with device-controlled link establishment (DCDC) [37,38].

Small cell technology is a promising yet economically driven approach to overcome the hurdles of mobile network operators, 5G coverage, capacity and IoT devices. Although small cells practice short range BSs, they have the potential to handle high data rates of mobile subscribers, monitor connected things and roll out both 5G and 6G. Small cells, for instance, picocells, femtocells and microcells, are deployed over the macrocell networks positioned within the same geographical region to form HetNets [39]. These HetNets have the capacity to deliver uninterrupted communications having high data rates and quality services, ensuring users to exploit reliable mobile networks services while moving from one cell to another. However, the deployment of large number of small cells to design HetNets creates issues, such as high inter-cellular interaction and expansion of cell boundaries, resulting in HO failures and radio link failure. All in all, the greater the number of small cells in HetNets, the greater will be the mobility management issues [40].

3.2. 5G Small Cells Architecture

3GPP (3rd Generation Partnership Project) has demonstrated the 5G structure to improve the cellular communication. Different service based communication models were deployed between control plane functions as proposed by 3GPP. Primary approaches are included but not limited to overcome dependencies among core network (CN) & access network (AN), boosting the access service, and obtaining a well defined separation between control plane (CP) & user plane (UP) to amplify the flexibility in deployment and scaling phases. 5G structure is also supporting concurrent access, which is highly paramount for low latency use cases [41]. The architecture of UDSC networks was proposed in [42] and compared with traditional network architecture. In traditional cellular network architecture, a tree network infrastructure is used in which BS managers are used to handle all macrocell BS in the core network, whereas all the back haul traffic is moved towards the core network from the gateway. A hybrid architecture is required to support the deployment of microcell in traditional cellular network. In hybrid architecture, microcell BS managers are used to handle microcell BS and the back haul traffic is moved to core network through fiber routes. In contrast to macrocell BS, microcell BS can deliver high speed wireless broadcast at home and hotspots use cases. Nonetheless, both have the potential to independently spread user data and management data to corresponding users [43]. In Figure 2, we show the 5G NR small cells architecture and its working model.

Mobile users can also HO in macrocell or microcell depending upon the requirements. Microcell network architecture therefore is used as a helping component for high speed wireless transmissions and communication purposes. Likewise, a similar approach is used to deploy small cells in 5G cellular networks, but it is quite challenging to forward back haul traffic from every small BS through fiber links or by using broadband internet, specifically in urban environments where cost and geographic installments are a major concern. Small cells BS cannot transmit back haul traffic to given gateway due to limitation of wireless transmission distance. To combat this challenge, a distributed UD 5G cellular network is proposed in which the functions of microcell BS and macrocell BS are distributed [44]. This means that the configuration of microcell BS is performed to handle management data and to monitor HO issues in small cells. While on the other side, transmission of user data is controlled by small cell BS. Furthermore, two different distribution architectures of UD cellular networks as follow:

• Using single gateway in UDSC network;
  A single gateway is deployed, configured, and installed at the macrocell BS having the capacity to embed large number of MIMO mmWave antennas. The purpose of integrating these antennas is to accept wireless back haul traffic coming from small cells located in macrocell. After collecting all the back haul traffic, it is moved to the macrocell BS through multi-hope mmWave links, which is then further forwarded to core network by using fiber links.
• Using multiple gateways in UDSC network;
  In this scenario, multiple gateways are used and deployed at massive microcell BSs by examining the conditions of back haul traffic and geographic scenarios. The reason to use multiple gateways instead of single gateway is that they are more flexible to handle back haul traffic and forward it to the core network. In this case, all the back haul traffic from small cells is dispersed over the multiple gateways in the macrocell, and then combined at gateway to move towards the core network by fibre-to-the-cabinet (FTTC) links [45].

Both the traffic capacity and network efficiency of 5G is 100x times than 4G-LTE, allowing it to deliver 20 Gbps peak data rates and 100+ Mbps average data rates. Small cell technology enables 5G networks to provide these benefits. With cell densification, connection distances are small, ensuring high network resources per UE. Large distribution of small cells BSs in UD networks make connection distance small compared to macrocell network, leading to offer better signal quality at every user equipment. UEs thus can support high mobile services, online gaming and streamline videos [46]. Besides, cells in UD networks have limited number of UEs, enabling them to experience more network resources.

States of RRC

On the air interface, 5G new radio technology is using a network layer protocol named radio resource control (RRC) protocol. This protocol is used between UE and base station that specified by 3GPP in TS 38.331. PDCP-Protocol is used for RRC message transportation. In general, the main functions of the RRC protocol follows as:

• Broadcasting system information;
• Paging notification and release;
• Connection establish and releasing;
• Reconfiguration and release;
• Outer loop power control;
• RRC connection mobility procedures.

According to the network status, RRC protocol configures user and control planes using signalling functions and grant radio resource management strategies for implementation. The operation of the RRC protocol is controlled by a state machine which characterize specific states of a UE and different amounts of radio resources correspondent with different state. Idle, connected, and inactive are the three states of RRC protocol. Among these three, idle and connected are introduced by 4G LTE, whereas inactive state is initiated in 5G cellular network.

• Idle State;
  In the RRC idle mode, the content of user equipment access stratum is not available in UE and network. It means that its main function is to save power and energy, for instance, when there is no need to transfer or receive data, the UE changes its mode to RRC idle by turning off both Tx and Rx. Similarly, when UE is following RRC idle mode, it regularly checks the call channel, handles incoming cases, and chooses the cell for camping by following mobility measurements [47]. Camping of user equipment performs the following purposes:
  • Getting system information for the camping in the cell;
  • Establishing RRC connection setup over the camped cell;
  • Getting call messages to shut the mobile calls within the camped cell;
  • Getting public warning system alerts.
• Connected State;
  In the RRC connected mode, the context of UE AS is present in both network and user equipment. When UE is present in RRC connected mode, it not only transmits or receives user plane data but also control plane signaling. It means that in this case, UE requires to monitor link quality of the former and target cell to get the radio link information. Subsequently, the consumption of battery is higher than idle mode [48]. However, DRX (discontinuous reception) is deployed in RRC connected mode to save power.
• Inactive State;
  Although when UE is present in RRC connected mode, networks do not suffer communication delays, yet high power consumption is a major challenge to address. And regular transition of UE from idle to connected or connected to idle creates undesirable signaling load, which amplifies latency. RRC inactive state or mode therefore is introduced by 5G NR to reduce power consumption, network signaling load and latency. In RRC inactive mode, the content of user equipment access stratum is stored in both core network and UE, whereas the connection between radio access network and core network is remained active to avoid power consumption along with the control plane delay. In this mode, the quantity of CN signals needed to paging a UE is estimated to be reduced, which is notable to improve latency performance. In the prior study, it is estimated that RC inactive states saves over 200% latency reduction in contrast to RRC idle states and 40% UE power consumption comparing to RRC connected mode [49].
• Registration and Paging of UE.
  Once the user equipment develops the potential to take benefit from the services and capabilities of network, its registration over the network must be initiated. The basic of registration process relies on the broadcasting of control messages amid UE, gNB and AMF, where gNB stands for NG NodeB and AMF is used for access & mobility management function. The registration of UE ensures that it can be controlled, handled and monitored over the network and reachable. Registration conditions including initial, periodic, mobility and emergency registration are essential to initiate the registration procedure. When the device is switched on, UE starts the initial registration to connect with the network. In periodic registration, network regularly monitors the UE to begin a new registration process. It enables the UE located in the registration area to determine whether its registration is eradicated without alerting the network or not [50].

UE performs the mobility registration whenever subscriber changes its position, and its linking tracking area cell is not available in the radio access list. Lastly, UE utilizes emergency registration while using emergency services and so. When the UE is switched on, it requires to select target cell and needs to launch the RRC connection with the gNB. Multiple input access operations are present to perform target cell synchronization along with the requirement to uncover public land mobile network (PLMN). Throughout the registration procedure, there exists a continuous exchange of NAS signal among UE and AMF, which is then captured and forwarded to gNB by RRC protocol [51]. By similar approach, the signal is moved towards the AMF through next generation application protocol (NGAP) in gNB. All these transmission and transactions enable the UE to receive the registration of 5G services.

3.3. 5G Small Cells Working Model

The reference model of 5G is depicted in Figure 1 in which the AMF or Access & Mobility Management Function acts as a brain of the model. It performs different management roles such as connection, mobility, and registration management. Like AMF, SMF or Session Management Function is also important part of model, and it is responsible for controlling, monitoring session context, editing PDU, and linking to decouple data plane. Connection between DN and architecture is provided by the user plane function (UPF). UPF is also responsible for PDU session anchor point, managing quality of service as well as the routing and forwarding packets [52]. The other counterparts are represented in the figure below.

3.4. Mobility Management in 5G UDSC Network

The purpose of mobility management in cellular networks is to find, identify and track users according to their locations to deliver them cellular services. It is primary function of the GSM and UMTS network to offer high quality of cellular service to their users irrespective of their locations. It means if the users are using their devices close to mobile terminal or present in hilly areas, all of them must be covered by GSM cellular networks. Henceforth, communication constancy, reliability, steadfast performance are all the ultimate goals of mobility management in the cellular networks [53,54]. Location management and HO are the two types of mobility management. In location management, UE informs the mobile station network that it is now associated to its specific location. After receiving the notification, location updating, and paging process starts. Network operates signaling process and paging progress through these location signals [55]. In B5G air based network, UAV path needs to change nearby to the terrestrial to reduce the communication delay as higher altitude of UAVs introduces loss of path in connectivity accordingly, so drone altitude depends upon use cases of applications and higher RSRP value and higher interference from neighbor base stations resultant in higher HO ratio [56]. Both air-to-ground and ground cellular network channel face high cochannel interference also on high frequencies UAV’s creates Doppler shift that causes inter carrier interference (ICI). Another issue is limited rechargeable battery power that limit the duration of UAV flight operation time. By contrast, UAV-BSs connecting with the macro base stations (MBSs) or the core network need high capacity wireless back haul links. Practically, the limited back hauls will become the bottleneck and affect the QoS of mobile users [57,58]. Back haul network is categorized by heterogeneous links for BS-wired links and UAV-wireless links and core network require high capacity wireless backhaul associations so the inadequate back hauls create the blockage and disturb the QoS. Satellite move on velocities differs for many services whether the task is fast data acquisition or real time information exchange [59,60].

Mobility in 5G beyond networks is mainstream for researcher due to the advance technologies range and directions are limited. In HO, the channel connection needs to be switch to other channel using information regarding reference signal strength and other network parameters. To optimize this process, it is very difficult to handle these parameters as they have equally influence on network. Some of them are interference management, energy consumption, load balancing, coverage, and capacity. These parameters also have the effect on ping-pong rate, call dropping/blocking probability, and early or late HOs [61]. Research have been proposed for the efficient HO mechanism using many theories and techniques/tools especially artificial learning to optimize the HO process [62]. 3GPP model explain the five basic UAV case studies based on velocities, mobility pattern, altitude, and movement. They also mentioned the maximum speed of 160 km/h and altitude 300 m, respectively. Depends upon application requirement UAV can exchange altitude, mobility, flight mode, and HO information with operator but under constraint of reliability, latency, and throughput [63]. The other problem with UAV is line of sight that create different interference conditions as terrestrial user would not face this problem. For management and operations of UAV system, identification, robust and efficient connectivity of UAV is a key interest. HO management and how we can acquire 5G HO through RL will be discussed in the remaining portion of this paper [64,65].

3.5. HO Management in 5G UDSC Network

To manage radio resources, HO play an important role in 5G. Transmission in mobile systems in 5G require greater bandwidth and greater communication service rate for distinct nodes or terminals. So, HO assure to provide good quality of service and continuous communications to maintain overall network performance in 5G [66]. Integrated UAVs communications into intelligent mobility and HO management/prediction with 5G and beyond using AI techniques will help to tackle frequent HOs with reliability and coverage. Different technologies embed in 5G; number of problems occur in HO process [67]. In HO, one need to implement well informed terminal to handle the higher data rates, base stations, and increased number of mobile devices. UAVs and user equipment’s are same by characteristics, but the HO characteristics are different and radio environment of both cases as their mobility models are different [68].

Performance depends upon the HO rates such as failed/successful HO. RACH, timer’s expiration, after a certain maximum number of re transmissions results in radio link failures. As the protocols for operations are different for UE and UAV so it is necessarily to decide first by elevation angle of the reference signal, velocity of user device, vertical location, and user device path loss/delay spread measurement. In the case of UAVs, they are not always connected to nearest BS as they are hunting for strongest RSRP so signal may come far from the drone also antenna beam concentrates to cover the ground coverage and drone usually fly on side lobes of antenna. UAVs fly in angular regions of 120°, 240°, and 270° and may connect to other BSs located in the region because they gain the inadequate signal strength. In this respect the UAV location and flight information help to make efficient HO i.e., enrich the existing signal report mechanisms, BS to help for HO using improved control of the RSRP measurement load [69]. The altitude of UAV matters a lot in terms of HOs as drone making height from 10–150 m then HO/minute also increase from 1–5 m. Integration of UAVs with beyond 5G wireless networks create the space for enhance cell selection process in 3D mobility patterns [70]. Following the guidelines in [71],

Table 2. Classification of HO Decision Schemes.

RSS Based Decision Schemes	QoS Based Decision Schemes	Function Based Decision Scheme	Intelligence Based Decision Scheme	Context Based Decision Schemes
Dwell Timer based Schemes	Available bandwidth based Schemes	Utility function based Schemes	Artificial neural based Schemes	Mobile agent based Schemes
RSS threshold based Schemes	SINR based Schemes	Cost function based Schemes	Fuzzy logic based Schemes	AHP based Schemes
Channel scanning based Schemes	User profile based Schemes	Network score function based Schemes	Intelligent protocol based Schemes	Mobility prediction based Schemes
Prediction based Schemes				Cooperation based Schemes MIH based Schemes

illustrate the classification of HO decision schemes, e.g., RSS based, QoS based, function based, intelligence based, and context based.

Advantages of small cell can be enlisting such as spectrum efficiency, high data rate, energy/money saving, less congestion, easy HO while there are also some disadvantages i.e., implementation cost and operational reliability, frequent authentication, and active or passive (on/off) state update. HO means that one can change its base terminal to its nearby terminal when moving from one point to another without interrupting the communication [72]. Innate challenges for HO in beyond 5G are following no services, improved routing, minimum latency, and security and these challenges become more harder in the scenario of multi-RATs, zero latency, network densification and high mobility. Beside these issues the load balancing for BS at the time of HO is also a main issue specially in the case of terrestrial network where the UEs move from houses to offices at morning and evening time respective of areas. HO is in different types according to connectivity such as intra macrocell, inter macrocell, and multi-Rat’s handoff. In dense HetNets HO mechanism is still an open issue, trade off between handoffs rates and interference level in the network and establish different types of interference to other UEs. Basically, there is network switching process occur in HO [73].

There are 3 basic steps in HO which are as follows: (1) discovery: in this step, network has to find such network which provide good quality of service to user. (2) decision: in this step HO process is initiated. If the initiation occurs in inaccurate time, then there is increase in call drop and thus reducing the QoS. (3) execution: to improve the QoS, decision should be executed at the right time to bypass the irrelevant HO. When both the second and third stage of HO is operated by mobile station or any other controller, then HO process is classified according to controller basis such as network controlled HO (NCHO), mobile controlled HO (MCHO) and mobile assisted HO (MAHO) [74]. Following the guidelines in [75],

Table 3. HO Information Gathering and Decision Making.

Handover Information Gathering		Handover Decision Making
Network Related	Mobile Terminal Related	Criteria Based	Strategy Based
Cost Based	Velocity Based	RSS Based	ANN Based
Coverage Based	Stations Based	Velocity Based	Function Based
Link Quality Based	User Preference Based	Security Based	Traditional Based
Quality of Service Based		QoS Parameters	Fuzzy Logic Based
		Bandwidth Based	User Centric Based
		Battery Usage Based	Context Aware Based
		Available RATs Based	Multiple Attribute Based
		User Preferences Based
		Operator Performance Based

illustrate the HO information gathering (network and mobile terminal related) and decision making (criteria based and strategy based) categories.

There are three parameters considers for the HO procedure i.e., system parameters, control parameters, performance parameters. First, Reference Signal Received Power and signal to noise and interference ratio comes under the system parameters. Second RSS HO Margin (Hysteresis), Time-to-Trigger (TTT) comes under the control parameters category. Third, HO failure ratio (HPIHOF), ping-pong HO ratio (HPIHPP), and call dropping ratio (HPIDC) comes under performance metrics [76].

3.6. Classification of HO Types

There are HOs types according to the network and according to the frequency and according to the techniques. The details of classification as followed:

• Horizontal HO; This type of HO is executed when the networks are same e.g., HO occur between 3G to 3G is called horizontal HO. This type of HO is also called intra-technology HO.
• Vertical HO; When HO executed between base stations of different network is called vertical HO. For example, HO is occurring between 3G to 4G. To proceed this type of HO, layer 2 and 3 of OSI model play an important role.
• Intra-frequency HO; When two distinct base stations work on the same operating frequency bands, then it supports intra-frequency HO.
• Inter-frequency HO; When two distinct base stations work on the different operating frequency bands, then it provides inter-frequency HO.
• Soft HO; It follows the make-before-make strategy where first new connections are built between UEs and wireless links before breaking the previous ones.
• Hard HO; It follows the break-before-make strategy where all the wireless links are first removed from UEs to build new wireless communication connections.
• Controller based HO; This type of HO is executed by mobile station. There are further three type of classifications: network controlled HO (NCHO), Mobile controlled HO (MCHO), and mobile assisted HO. In NCHO, the decision step is detained by a controller while mobile station takes initiation step, In MCHO, mobile station takes both steps initiation step as well as the decision step, while in a MAHO, network takes the decision and mobile only collect and send basic information i.e., received signal strength indication, and signal to interference-plus-noise ratio [77].

There are also some advantages and disadvantages of HO schemes based on spectrum, overhead, reliability, QoS, latency, and designated channels. Following the guidelines in [78],

Table 4. Advantages and Disadvantages of HO Schemes.

Schemes	Advantages	Disadvantages
Hard HO	Efficient user of spectrum No data overhead	Short interruption of service Sensitive to link transfer time (may result in dropped call)
Seamless HO	Reliable (no service interruption)	Inefficient use of spectrum Data overhead
Soft HO	Highly reliable No loss of QoS in HO	Data overhead Inefficient use of spectrum
Predictive Rerouting	Minimized HO latency	Signaling over-head Possible data overhead
Static GC	Reserved channels for HO	Possible under-utilization of spectrum
Dynamic GC	Reserved channels for HO More efficient use of spectrum	Signaling and computational overhead
Queuing Schemes	Easy to Implement (FIFO) Queue reorder according to degradation of channel	Degradation of channel disregarded (FIFO Queue) Signaling and computational overhead

illustrate advantages and disadvantages of HO schemes.

4. Reinforcement Learning Algorithms for HO

In 5G small cells paradigm, HO management is a critical challenge particularly in the case of mmWave frequencies. This, by its turn, with increasing number of HOs and signaling overhead are more likely with decreased the QoS and QoE of the user. Machine learning based HO solution are highly promising and optimized while the state-of-the-art algorithms only focused on event trigger parameters i.e., RSS, RSRP, RSRQ, Cell individual Offset, Time-to-Trigger, and Hysteresis.

4.1. Reinforcement Learning

RL means to inflate the rewards by taking number of actions in environment. This learning involves performing those actions which maximize these rewards. This type of learning behaves same as natural learning where agent must learn by himself through hit and trial mechanism for maximum reward [79]. Supervised, unsupervised, and semi-supervised are the classifications of ML. RL (semi-supervised) is different from supervised and unsupervised learning. In supervised learning, there are set of instruction for each action and the objective is to map the input correspondent output and learn the rules from labelled data. Regressive and classification model are used in this category depends upon whether the value is continuous or discrete. While in case of unsupervised learning, agent must discover the hidden structure for unlabeled data [80]. Unsupervised learning is vice versa to supervised learning and can be applied typically when data are insufficient and is not labelled. But in case of RL, the agent has initial and end points and to reach its destination, agent must find best possible actions by manipulating the environment. After reaching the final solution, agent receive the rewards but if he fails to reach, he does not get any reward, so agent must learn environment to receive maximum rewards [81]. In RL, the problem formulation is done using markov decision process (MDP) and the solution can be policy or model base and can be model free i.e., Q-learning, SARSA. In this technique, the agent interacts with environment and generate policy based on rewards and at the end, systems is trained and delivers improved performance [82,83].

$$\begin{array}{c}\hfill Q_{t+1}(s,a)\leftarrow (1-\alpha )Q_t(s,a)+\alpha [R_{t+1}+\lambda \frac{max}{{\alpha }^{{}^{\prime }}ϵA}{Q}_t(s^{\prime },a^{\prime })]\end{array}$$

(1)

In Equation (1), sum of old Q-value and learned Q-value provides an updated Q-value. Where Q${}_t$($s,a$) is a old Q-value, Q${}_{t+1}$$(s^{\prime },a^{\prime })$ is an updated Q-value, and R${}_{t+1}$+$\lambda \frac{max}{{\alpha }^{\prime }ϵA}$Q${}_t$$(s^{\prime },a^{\prime })$ is a learned value. Moreover $\alpha$ is a learning rate that can be set between 0 to 1, where 0 means never updated and 1 means quickly learning. While $\lambda$ is a discount factor that also can be set between 0 to 1, where 0 means considering current rewards and 1 means striving for a long-term high reward.

The intermediary scheme in which labeled and unlabeled data are exploited for the training is known as semi-supervised learning [84]. In deep learning (DL), the rules are established using neurons operation that approximate at the complex function. In mobile communication, DL has significant importance to tackle with complex nonconvex challenges and high computational problem [85]. As neural network is used for feature extraction and learning phase so this algorithm can be used in number of scenarios i.e., nonlinear model enhancement, continuously varying mobile environment evaluation, degree of overfitting and complexity reduction, and reconstruction error of the data minimization [86]. DRL is the revolutionary and emerging tool in many fields of sciences particularly in mobile communications for efficiently deliver the solution of various challenges [87]. Deep convolution neural network (DNN) intends to learn the characteristics of the channel and forecast the appropriate modulation coding scheme (MCS). For intelligent decision without human mediation the multiple layers employed to build an artificial neural network. For improving parameters of the network, the artificial intelligence, machine/deep learning techniques are the best approach as we supposed fewer physical intervention and advanced computational constraints [88].

Nowadays advances networks such as HetNets, IoT, and unmanned aerial vehicle (UAV) networks reshaped to autonomous, adhoc, and decentralized form in which mobile users, UAVs and IoT devices take decisions by theirself i.e., cell association, power control, data rat etc. In these scenarios, the problems sculpted by MDP has worth to make decisions accordingly and number of algorithms and learning techniques helped to solve MDP [89]. Computational complexity of advanced and large networks turns out to be very difficult. In this regard, DRL delivers some required benefits such as decision making independently, improves the learning speed with large state and action spaces, learn and develop network understanding about the communication and environment, sophisticated network optimizations, data offloading, interference management and cyber physical attacks modeled. DRL-based joint resource management functions for 5G HetNets, multi-objective DRL based resource management, flexible resource management design, DRL-based load balancing for 5g HetNets need to be investigated under the 5G context [90]. Figure 3 shows categories of HO optimization techniques using machine learning tools.

For the prediction analysis the artificial intelligence needs more matureness in channel modeling. The main issues are i.e., high dimensional search due to huge antenna, transmitted and received signal relationship, learning faster combination of transmitting and receiving beam, convergence in training the AI model. The advance techniques of AI/ML/DL energize the 5G and beyond 5G wireless networks to support emerging use cases introduces in the real world. However, despite of advancement the open research issues and future directions still need to be address. In practical implementation the efficiency of the training procedure needs matureness such as getting faster convergence with the best possible parameters of the learning algorithms. To acquire data from extensive measurement operations there is a still gap for real experiments results from dense urban propagation areas, high speed moving nodes on terrestrial areas and dynamically changing environments to justify the precision of the learning algorithms [91]. In hierarchical networks, architecture design, control of the communication parameters of network entities, computation proficiency, centralized or distributed control performance, and accuracy requirements still need to be explored using AI/ML/DL aspect [92]. Advance algorithms and techniques for cyber attack during the operation is also an open challenge in this field such as reliable communication of unmanned ariel systems, session hijacking, man-in-the middle attack etc. RL has two main features (i) trial and error search (ii) delayed rewards [93]. Following the guidelines in [94], Figure 4 shows both RL, and deep Q-learning schema.

Model is used to anticipate the nature of an environment. The ML designs which is using both planning and models are model based mechanism. If the model of environment is absent, then learning can be done through trial and error methods. Generally, there are three approaches to implement RL algorithm; Value based: In the value based RL algorithm, users try to achieve maximum value function, meaning that an agent is expecting a long period return of the existing states. Policy based: In this approach, users design a policy in which multiple actions are performed in every state to acquire a maximum reward in the future. Policy describe the method in which agent must act in certain environmental conditions [95]. Basically, policy always map the function of states and actions. Number of formats can be implemented as policy such as it can be a table, any searching process or can be a function. The idea of RL is way to maximize this policy. The signal rewards depicts the actions taken by agent is good or bad. The intention of this reward signal is to inflate the overall reward. Policy is dependent on signal reward in way that if agent receive bad reward, he must revise his policy and then again perform actions. Rewards can be categorizing as immediate reward or delayed rewards. In case of delayed rewards, the agent must find out which actions are cause of this reward. value function calculate the overall forthcoming reward. The core idea behind the value function is to figure out the states and perform actions accordingly. The basic diagram of RL is given above which shows the state and its relative actions [96]. Policy based methods are further divided into types:

• Deterministic; Same actions are performed for all the states and processed by the policy pie.
• Stochastic; Every action corresponds to a certain policy model based. In this method, a virtual model is designed for all types of surrounding atmosphere or environment. After creating a virtual model, learning process of agent begin to perform in that environment.

4.2. Related Contributions

Machine learning algorithms always relies on intelligence and environment awareness. It grows fast and extensively utilized in various research domains and applied fields. Related contributions in

Table 5. Related Contributions for RL-Based HO.

Reference	Algorithm	Model	Description
Minh-Thang Nguyen, et al. [97]	Reinforcement Learning	Model Free	Suggest an algorithm for seamless mobility management using optimized HO parameters in arbitrarily deployed small-cell networks.
S. S. Mwanje, et al. [98]	Reinforcement Learning	Model Free	Suggest a framework, enables an advance behavior of SON that learn the best possible configurations autonomously using reinforcement learning for mobility optimization and mobility load balancing.
K. T. Dinh, et al. [99]	Fuzzy and Reinforcement Learning	Model Free	Suggest a combined solution of Fuzzy Q-Learning Control and a heuristic Diff_load algorithm to optimize the HO and load balancing issue by adapting hysteresis and the time to trigger parameters for SON enabled networks.
Y. Koda, et al. [100]	Reinforcement Learning	Model Free	Suggest a reinforcement learning optimal HO decision-making policy in millimeter-wave (mmWave) communication networks to maximize throughput considering the velocities and locations of a pedestrians.
C. Lee, et al. [101]	Deep Learning	DNN Model	Suggest a policy for conditional HO by forecast the target cells get prepared for a forthcoming HO.
L. Yan, et al. [102]	Supervised Machine Learning	Model Free	Proposed to assist HO using existing chronological data such as channel state information (CSI) and K-nearest neighbor algorithm in mmWave vehicular networks for efficient HO decision.
C. Wang, et al. [103]	Deep Learning	Model Based	Suggest a multi-user multi-step trajectory prediction to predict user’s future location using the Long Short Term Memory (LSTM) for HO management.
Mollel, Michael S., et al. [104]	Deep Reinforcement Learning	Model free	Suggest an offline reinforcement learning algorithm that optimize the HO decisions considering existing user connectivity and throughput within both time and frequency domains.
Bahra, Nasrin, et al. [105]	Deep Reinforcement Learning	Model free	Suggest a hybrid approach to obtain the existing user mobility patterns and predict the future trajectory of a user.
Wang, C, et al. [106]	Deep Learning	Model free	Suggest a multi-user trajectory prediction using LSTM cells that learns the user’s historical mobility patterns.
Xu, J., et al. [107]	Deep Learning	Model Free	Suggest a model to understand the mobility patterns for trajectories destination prediction.
Bahra, N., et al. [108]	Deep Learning	Model free	Suggest a mobility model to simplify the user’s trajectory using recurrent neural network variations and eliminating the irrelevant data.
Sadri, A., et al. [109]	Deep Learning	Model Free	Suggest a mobility model of existing relations from all existing trajectories for future path prediction using a similarity metric.
Ozturk, M., et al. [110]	Deep Learning	Model Free	Proposed an analytical model to determine the holistic cost of HO i.e., latency, signaling overhead, and call dropping.
M. Alrabeiah, et al. [111]	Deep Learning	Model Free	Suggest a technique to predict obstruction and mmWave beam for mobility management considering sub-6 GHz channels.
C. Lee, et al. [112]	Deep Learning	Model Free	Suggest a policy to predict the upcoming cell for proactive conditional HO using deep neural network in mmWave networks.
Z. Wang, et al. [113]	Deep Learning	Model Free	For low latency mobile networks, hidden Markov process implemented for learning the optimal HO controllers and to prediction the next connected access point.
Chih-Lin I, et al. [114]	Deep Learning	Model Free	Proposed a proactive HO method based on novel data-driven intelligent radio access network. The technique decreases the number of service interruptions and the impact of ping-pong effect.

shows a comparison of efficient and in optimized machine learning algorithms for HO decision management.

4.3. Types of Reinforcement Learning

Positive and negative are the two types of RL, which are defined as follows:

• Positive;
  Positive RL is referred to event that happens because of the specific behavior. It amplifies the intensity and oscillation of behavior and impacts on the activities performed by the agent. It maximizes the performance of an event and maintain changes for a longer period while an excessive implementation of RL may create over optimization state that impacts the outcomes of actions.
• Negative;
  In this type of RL, actions are taken to improve the strength of behavior that happens because of the undesirable conditions. These undesirable conditions should be stopped or reduced to achieve the minimum standpoint of performance. Nevertheless, a lot of effort is needed to achieve the conditions of that standpoint [115].

5. Challenges and Future Research Directions

Many studies already have been conducted to address one of the biggest challenges of future wireless networks such as HO management in 5G small cell. Emerging technologies i.e., D2D, M2M, MIMO, EC, SC, BF, convergence of WiFi and cellular, SDN, NFV and CC and upcoming use cases and services such as mMTC, eMBB, and uRLLC introduces new challenges. Also high speed mobility, high data rate applications and limited resources in 5G UDSC networks faces numerous challenges. Still there are some significant challenges needs to be addressed in optimized way using advance machine learning algorithms. In this section, we will briefly discuss the upcoming challenges for HO management in small cell networks and future research direction.

• QoS/QoE for multimedia traffic; The requirements for quality of service and serving capability of multimedia traffic are different from the data and voice traffic. HO techniques deliver different QoS/QoE in different use cases to perform various types of multimedia traffic [116]. Providing the best machine learning solution while considering the QoS/QoE in HO management, is an active research area for beyond 5G wireless small cell networks where huge data will be drive with low latency and best connectivity.
• Controlling Communication Overhead; Existing HO solutions required complicated and frequent collaboration between all nodes available for communication i.e., Macrocell, small cells and the UEs. This phenomenon required large number of network resources to exchange the necessary information [117]. Providing the best machine learning solution for controlling communication overhead while considering is an active research area for beyond 5G wireless small cell networks.
• Network Performance in Outdoor Use Cases; Primarily, huge data traffic broadcast in indoor scenarios where wired and wireless connections are the best available option [118]. while providing the best machine learning solution for outdoor scenarios should be consider carefully is an active research area for beyond 5G wireless networks.
• Battery Life in Smartphone; Advance antenna, applications and optimized use case scenarios required huge processing and this killing behavior consuming the battery life of smart phones and wireless connected drones [119]. So, providing the best machine learning solution for limited energy supply is another critical challenge is an active research area for beyond 5G wireless networks.
• Wireless Back haul Spectrum Efficiency; In beyond 5G wireless networks, cell BSs requires wireless back haul network with massive capability to handle the large number of wireless connections and flexible deployment [120]. Hence, providing the best machine learning solution for spectrum resource management, networking complexity, and infrastructure cost to handle the large number of cells in beyond 5G wireless networks is an active research area.
• Advanced Techniques Integration; In 5G small cell networks, mmWave, massive MIMO, and mMTC are the key enablers to improve the network capacity up to 100 times. And massive signaling overhead of these advance technologies produce dense communication and processing [121]. Therefore, providing the resource efficiency, cost efficiency, and interference mitigation using machine learning in beyond 5G wireless networks is also an active research area.
• Security and Privacy Concerns; The most critical and crucial challenges in HO management for UD 5G small cell networks are security and privacy concerns since the high densification of the cells and UEs. Number of new functions and applications dealing with communication data pose new challenges for security compromise and privacy concern [122]. Hence, efficient counterstep using machine learning in beyond 5G small cell wireless networks also an active research direction.
• HO in Drone Mobility; According to 3GPP, unmanned ariel vehicles possibly experience weak signal-to-interference-plus-noise ratio (SINR) than terrestrial UEs. Because of obstacles occurring between the wireless signals and possibility of HOs increases so it is inevitable to improve these issues. As mentioned previously, small cell technology and cell densification also bring challenges for mobility and HO management. In UD networks, the coverage range of cells is limited and overlaid. As a result, UEs, covering mobility functions, need to move from one cell to another or face frequent HOs. Mobility management is a key feature of 5G infrastructure as it improves user experience and use cases that will be coming in future. Therefore, HO functions and operations must be completed without interference and interruption instances to perform the requirements of 5G mobility management. In this paper, we discuss challenges of mobility and HO management in 5G UD cellular network and scrutinize multiple ways to overcome these challenges. In drone communication, telepresence, dominance in line of sight (LoS), coordinated multi-point transmission, air-borne-base station required more efficient solutions to conduct different services [123,124]. Therefore, cost effective machine learning based solution in UD networks also an active research direction.
• Load Imbalance; Despite all the advantages of HetNets and cell densifications, small cell technology comes with hurdles that must be solved first. Load imbalance, for instance, occurs due to variation in transmitted power and coverage area from unlike tiers of cells. Henceforth, small cells will not serve substantial purpose by using traditional user association rules which revolve around only received power. Cell range expansion (CRE) or biasing is a persuasive technique to fight against this challenge [125].
• Inter-cell Interference; Inter-cell interference is yet another issue present in cell densification that can be solved through eICIC, an abbreviation used for cell interference Coordination. This mitigation technique exploits Almost blank subframes (ABS) to eradicate noises or interference from Macrocell BSs. ABSs are integrated in HetNets to optimize interference of high power nodes. However, low power nodes know the interference pattern, allowing the CRE to be embedded over the low power nodes and can serve large number of UEs without getting interference from high power nodes [126].
• Radio Resource Control (RRC). The mobile management and HO operation challenges can be controlled by RRC. Being a layer three network protocol, it is located between UE and BS nodes, used in UMTS, LTE and 5G and considered as a part of air interface control plane. Consequently, RRC has a potential to enhance the latency, power, and energy consumption in UD 5G cellular networks. Some other functions include transferring system information, initiating or emancipating RRC connections, paging, transferring nonaccess stratum (NAS) messages essential to handle communication between user equipment and core nets [127].

6. Conclusions

In this paper, we have presented an overview of the mobility management using 5G enabling technologies. We have presented the 5G wireless network structure supporting ultra-dense small cell networks. We discussed HO information and decision management in 5G UDSC network scenario. We also mentioned radio resource control, HO metrics, information gathering and classification of HO decision schemes. Finally, we have discussed how machine learning techniques can help to optimize the HO process in 5g network and related contribution of researchers accordingly. At the end, we discussed the mentioned challenges to be addressed with respect to 5G supported use cases using machine learning supported tools.