Resource Management in Converged Optical and Millimeter Wave Radio Networks: A Review

Abstract

Three convergent processes are likely to shape the future of the internet beyond-5G: The convergence of optical and millimeter wave radio networks to boost mobile internet capacity, the convergence of machine learning solutions and communication technologies, and the convergence of virtualized and programmable network management mechanisms towards fully integrated autonomic network resource management. The integration of network virtualization technologies creates the incentive to customize and dynamically manage the resources of a network, making network functions, and storage capabilities at the edge key resources similar to the available bandwidth in network communication channels. Aiming to understand the relationship between resource management, virtualization, and the dense 5G access and fronthaul with an emphasis on converged radio and optical communications, this article presents a review of how resource management solutions have dealt with optimizing millimeter wave radio and optical resources from an autonomic network management perspective. A research agenda is also proposed by identifying current state-of-the-art solutions and the need to shift all the convergent issues towards building an advanced resource management mechanism for beyond-5G.

1. Introduction

The efforts to define beyond-5G and 6G performance targets are emerging [1], envisioning high capacity links with low end-to-end latency, network softwarization, and massive connections for the future network architecture. In addition to network densification with small cells [2], millimeter wave (mmWave) radio and optical communication are considered as key enablers of an envisioned beyond-5G network ecosystem [1]. Merging the optical and radio channels can further increase the efficiency of the radio access network (RAN), and techniques, such as analog radio-over-fiber (ARoF) are promising solutions towards this direction [3]. These enablers, however, also increase the number and type of access point (AP) nodes (small cells, macro cells, remote radio heads (RRHs), roadside units, etc.) [4,5], diversifying the parameters to be included in resource management decisions together with the dynamism of massive communications. The complexity further increases as the multi-dimensional resources of dense networks include not only frequency, time, power, space, and multi-user diversity but also involve energy, computation and storage resources [4], making resource management a key topic in the next-generation communication systems.

The increase in data traffic and the number of users, devices and network components also lead to a huge increase in data which can be analyzed with machine learning (ML) techniques for optimization in user quality of service (QoS), network management, and service provisioning. The increasing number of unknowns in the system also creates an interest in the use of learning techniques in network management. This fact motivated us to conduct a review to determine the key performance indicators (KPIs), performance targets, and the optimization algorithms in the published works. Another goal of this review is to identify the methods adapted to converged optical and mmWave radio networks, understanding the complexity of the algorithms and the diversity of techniques implemented with an overview on related activities. Finally, Artificial intelligence (AI) contributions have a tremendous impact on the design and development of autonomic networking solutions [6]. Thus, we also aim to detect the steps to be taken from applying the ML techniques to providing a holistic autonomic (re)configuration for mmWave optical and radio networks.

The solution methods are categorized under the main objectives of throughput maximization, delay minimization, energy-efficiency, and virtual resource allocation. Technological enablers and infrastructural changes resulting from the implementation of these novel network paradigms also affect the way that resource allocation algorithms are designed. For this reason, we put special emphasis on the relation between the resource management solutions and the technological enablers that led to a paradigm shift in network management, such as software-defined networking (SDN), network function virtualization (NFV), and multi-access edge computing (MEC).

The remainder of this article is structured as follows: The background concepts of mmWave networks and AI/ML used in communication networks under autonomic network management (ANM) framework are presented in Section 2. The preparation, operation, and reporting stages of the survey are explained in Section 3 to reveal how we planned and conducted the survey. We also present an overview of the selected papers based on their optimization algorithms, performance metrics, and the evaluation criteria in this section. The results of our survey are categorized under the main optimization objective topics and presented in Section 4, Section 5, Section 6 and Section 7, respectively. A discussion specific to each of these optimization topics is also provided in these sections. Section 8 concludes the paper by summarizing our findings and providing a pointer for future research. The structure of the paper, and the noticeable concepts in the selected papers for the survey are graphically represented in Figure 1.

2. Background Concepts

In this section, we explain two of the background concepts that are expected to become the pillars of future networks, namely utilizing mmWave networks to boost capacity and ANM as a conceptual framework to make use of the potential gains of AI/ML towards higher resource efficiency. The optimization of the converged fiber-wireless mmWave network resources is the focus of this study, and this brief introduction aims to highlight the motivation, the main concepts, and the technological enablers behind the solutions provided in the presented resource management papers from Section 4, Section 5, Section 6 and Section 7.

2.1. Converged Optical and Mmwave Radio Networks

An essential source for capacity enhancement in dense networks is the use of mmWave spectrum bands around 28, 38, 71–76 and 81–86 GHz frequencies [7]. A total of 16 GHz bandwidth is available to use in these spectrum regions. In spite of the availability of large bandwidth, these bands were not used in cellular networks due to their high power consumption [8] and the challenging propagation characteristics due to high path loss, poor signal penetration and reflection, the sensitivity to blockage from various objects outdoors and the high Doppler effect observed in frequency bands deployed until long term evolution (LTE) technology [9].

Extensive studies are carried out in order to understand whether mmWave spectrum bands can be used in dense networks despite these unwanted characteristics. Interested readers are referred to [7,8,9,10,11,12,13,14,15,16,17,18,19,20]. Here, we only provide a list of some of the key challenges that are addressed by the resource management solutions in the selected studies: In order to reach the extreme data rates, beamforming with adaptive antenna arrays and highly directional transmissions are required for narrow beam operations [10,11], bringing in the problem of dynamic adjustment of antenna elements to support multiple directed beams with massive multiple input multiple output (MIMO) [12,13]. Synchronization and broadcasting should also be designed properly for these directional transmissions [14], as narrow beams can cause loss of connectivity in mmWave mobile networks when they are misaligned [15]. In addition, efficient beam-tracking and alternative directed spatial channels need to be provided to users in case of outage [16] or blockage [11]—one probable solution can be to serve a UE at the same time over several APs [7].

Integration of capacity enhancement techniques, namely the use of mmWave and massive MIMO is also complementary to the small cell evolution [13]. Utilizing higher frequency spectrum bands for capacity gains also drives toward use of more small cells, as the propagation characteristics of mmWave signals lead to higher attenuation and reduced coverage area, requiring the deployment of more APs. Ultra-dense networks (UDN) is an attractive technology to boost the capacity in a coverage area [17], as increasing the number of access nodes enables radio frequency reuse in a certain coverage area. To unfold the added capacity provided by UDN, end-to-end network management has to deal with increased dynamics of radio access due to the complexity of the architecture with increasing number of nodes, massive data generation, and dynamic topology changes requiring quicker network reconfiguration.

The introduction of mmWave frequency bands for wireless access with dense deployments also leads to a major increase in fronthaul capacity, and fiber network solutions can provide the required data rates for this fronthaul [21,22,23]. Radio-over-fiber (RoF) implementations for mmWave have long been considered to create distribute mmWave radio signals to dense APs from a central station [24,25]. Multiple wireless services can thus share the huge amount of bandwidth in the same optical fronthaul network, achieving optical-wireless convergence [3].

A data plane design for converged optical and wireless networks [26], and aligning this design with the control plane inside an ANM framework allows the dynamic reconfiguration of the end-to-end data plane channels with AI-ML-based solutions, such as beamforming and steering [27] or service-aware slicing [28].

2.2. Autonomic Network Management

Compared to existing wireless networks, a much more dynamic environment is foreseen due to mmWave propagation characteristics and massive deployments of small cells, leading to a dynamically changing large scale network topology. As a result, providing a stable network communication that is able to withstand varying network conditions will become more complex, pushing any human dependent network management approach out of the equation.

A number of approaches to transfer the intelligence to mobile networks have been around for many years from auto-configuration of network entities [29] to self-optimization of network operations [30], leading to the ANM paradigm for anticipating and diagnosing impairments in networks driven by operation and management (O&M) layer goals [31]. The emergence of SDN also contributed to network optimization efforts by offering programmability and network system reconfigurability to today’s networks by decoupling the control plane from data plane [32]. With the support of NFV and cloud technologies to SDN, softwarization and programmability are considered as the main enabling technologies to manage the dynamic topologies in a responsive way and achieve 5G traffic requirements [33].

An autonomic networking framework for 5G and beyond requires translating of high-level O&M goals into low-level technical parameters and then monitoring the network to adapt network status by making use of observations from the monitoring stage [6]. The management paradigm shifts towards avoiding the rigid properties and the limitations arising from the difficulties in modeling the entire state space beforehand. The type of data to be collected for monitoring, the possible actions and control rules, the learning algorithms and data analytics functions should also be defined properly, resulting in a control loop through which multiple stakeholders can become involved in decision-making inside a hierarchical functional decomposition.

All these ANM framework requirements and the increasing interest in AI/ML-assisted approaches to resource orchestration and optimization led to the efforts of ITU-T [34] and 3GPP [35] to define the architectural framework for integrating ML solutions to 5G and beyond networks, whereas exploiting AI solutions in every network segment possible to learn and adapt to network dynamics is conceptualized as the ‘AI Everywhere’ principle for networks [1,36]. The 3GPP has introduced the network data analytics function (NWDAF) and the management data analytics function (MDAF) for core services [37] that can be used for centralized optimization of the network resources. On the other hand, the 5G RAN data analytics function (DAF) is proposed for radio resource management [38], which targets managing machine learning and AI solutions in the RAN with open interfaces inside the O-RAN project [33]. RAN DAF can also be extended with local monitoring lightweight data analytics capabilities for decision-making at different types of RAN nodes inside a distributed and hierarchical framework [39], allowing local and distributed resource management optimizations to take place inside the network.

In light of these technological developments, the authors’ objective in this study are to identify the benefits and the barriers of the existing resource management methods for converged optical and mmWave wireless networks, and to discuss the open issues from an ANM perspective.

3. Research Method and Overview of Selected Articles

This section introduces the method employed for finding and selecting articles to be included in the study and subsequently provides information on the data extracted from the selected articles.

3.1. Research Method

In order to conduct this survey, we reviewed works published in the literature with a focus on those that cover most of the identified optimization requirements for converged optical fronthaul and mmWave wireless access networks. Before presenting these works, in the following we explain our research method based on the research steps given in [40]. The selection procedure is also illustrated in Figure 2.

The first step was the selection of the papers. We completed this step by conducting database searches in the ACM, Elsevier (Science Direct), IEEE, IET, MDPI, Optical Society/Optica (OSA), Springer, Taylor & Francis, and Wiley online library databases with keywords “resource allocation AND converged mmWave fiber wireless (FiWi)”, “resource management AND converged mmWave fiber wireless (FiWi)”, and “resource allocation AND converged fiber wireless (FiWi)”. The searches in all databases were completed in May 2021. The resulting collection was screened, to exclude non-scientific texts, book chapters, out of context papers, and survey papers. The full list of papers can be found in [41].

Among the remaining 189 papers found in our database search, our selection criteria was created to present the works that are most relevant to the target network architecture, providing novel implementation solutions to the requirements of the optimization objective. The criteria selected for our eligibility step can be summarized as follows:

• The study provided a sound research approach and published after a scholarly review process;
• The study had a resource management optimization objective for mmWave networks;
• The study explained the system model and proposed a well-defined optimization algorithm;
• The effects of the algorithm on a performance metric was reported and the different aspects of the performance metric were analyzed with different evaluation criteria.

This review is limited to the focus scope on converged optical and mmWave radio network solutions and to the databases taken into consideration. The prioritization of the works that address a well-defined optimization algorithm led to the omission of relevant papers. We did not include works that do not clearly define a resource management objective, e.g., a study that focuses on the the hardware implementation aspects of optical and mmWave radio networks with no resource management perspective. We manually excluded all studies that do not match these criteria with a simple scoring system, in which a point is deducted from an eligible paper for each missing criterion. After this screening process, we identified 37 papers that focused on at least one of the resource management objectives of throughput maximization (Section 4), delay minimization (Section 5), energy-efficiency (Section 6), and virtualized resource allocation (Section 7). The papers that have joint objectives are classified under their main optimization focus of that paper. Our target in this review is to understand the recent optimization techniques used in resource allocation for converged optical fronthaul and radio mmWave access network implementations, therefore we focused our search to the works completed in the last five years (between 2016 and 2021, both included), and approximately 95% of the selected papers fit in this category.

3.2. Overview of the Data Collected from Selected Papers

Before analyzing the papers classified under their objectives in Section 4, Section 5, Section 6 and Section 7, we present an overview of the data collected on the current trends in the algorithms for optimizing the performance of mmWave optical and radio networks, and understand the key network state parameters used to evaluate the performance of the presented algorithms. For this reason, in this section we provide answers to the following three questions with the data collected from the eligible studies:

• Question 1: Which algorithms are used more often in performance optimization in converged mmWave networks?
• Question 2: Which performance metrics are determined to show that the optimization method achieves the objective?
• Question 3: Which criteria are used to evaluate the solution method?

Regarding the first question, Figure 3 shows the distribution of the optimization algorithms used by the selected papers. Heuristic and iterative algorithms are frequently used in the literature, due to the fact that many optimization problems in this field are non-deterministic polynomial time hard (NP-hard) and thus require decomposition and simplification to create sub-optimal solutions which can then be solved with heuristic algorithms. Game theory and matching theory-based solutions are also attractive for distributed decision-making among entities, as centralized optimization is challenging in multi-stakeholder environments [42,43,44,45,46,47]. Finally, AI-based models (artificial neural networks (ANN) [48,49], Q-learning [50]) are also being utilized to resource management optimization problems. The distribution of the main performance metrics according to the resource optimization objectives is given in

Table 1. Distribution of the main performance metrics depending on optimization objectives.

Objective	Performance Metric	Total Papers
Throughput Maximization (Section 4)	Throughput/Sum rate (Bits/s)	7
	Spectrum efficiency (bits/s/Hz)	2
	Quality of Service (QoS)	2
	Fairness	2
	Bandwidth Utilization	1
Delay Minimization (Section 5)	Delay	3
	Average response time	2
	End-to-end delay	1
	Maximum delay	1
Energy Efficiency (EE) (Section 6)	EE gain	5
	Power consumption	3
	Achievable EE	1
	Revenue	1
Virtualized Resource Allocation (Section 7)	Virtual Network (VN) acceptance ratio/embedding success	4
	Resource utility/pooling gain	3
	Revenue/Profit	3
	Average sum Quality of Experience (QoE) of operators	1

and the objectives are also mapped to their sections in this review. Finally, the evaluation criteria to test the performances of the selected papers are grouped in

Table 2. Distribution of evaluation criteria.

Four Objectives		Three Objectives		Two Objectives		One Objective
User number	10	Transmit power	4	Coverage radius	3	Antenna number	3
Traffic load	9	Delay	3	Bandwidth	3	Queue length	1
Number of APs	8	User location	3	Rate of requests	3	Operator number	1
				Requested service class	2	Channel estimation error	1
				Service number	2	Flow arrival rate	1
				Computing capacity	2	Offload probability	1
						Fairness	1
						Data rate demand	1
						VN size	1

, which shows how many times each criterion is used together with how many of the resource management objectives use these criterion.

4. Throughput Maximization and Resource Allocation Algorithms

Higher throughput requirements of novel services defined under the enhanced mobile broadband (eMBB) category are one of the main drivers of creating resource allocation solutions with learning algorithms for throughput maximization. The learning methods for throughput optimization in mmWave radio and optical networks are presented Section 4.1. The existing solutions to throughput maximization problems are given in

Table 3. Analysis of resource allocation papers with throughput maximization objective.

Ref.	Optimization Algorithm	Advantages	Limitations	Performance Metrics
[42]	replicator dynamics to solve evolutionary game model	stable convergence in presence of time delay	dependent on information exchange among BSs and PON	guaranteed QoS
[51]	volume adjustable water filling-dynamic programming	able to adapt different coverage, blockage, and power conditions	UE high mobility scenario is not considered	throughput
[52]	deficit DBA and QoS parameter tuning for voice, video and best effort traffic models	guarantees high throughput for prioritized traffic and achieves fairness for low priority traffic	DBA algorithm has exponential time complexity, therefore not suitable for populated scenarios	QoS, average throughput, fairness
[53]	two heuristics for joint beam selection and user association	the algorithm converges with low number of iterations	user association conditions are simplified as extensive search leads to large delays	spectrum efficiency
[54]	interleaved polling with adaptive cycle time DBA scheme	improves bandwidth utilization, less average delay for high traffic load	solution depends on the reporting time between radio and optical nodes	bandwidth utilization
[43]	game based stochastic DBA	keeps fairness with more throughput and less delay for populated ONUs	not suitable for fully distributed scenarios as load balancing strategies of players are selected centrally	goodput, fairness
[55]	three iterative algorithms for fronthaul and access link optimization	improves QoS achievable data rate	the solution is convergent but has polynomial complexity	aggregate arrival rate
[56]	five matching theory based scheduling algorithms	achieves the optimal max min throughput performance for network	an approximation algorithm cannot be used under mutual interference	max–min throughput
[57]	Lagrange duality-based optimal algorithm and a greedy search-based heuristic	maximizes the weighted sum rate of all users and outperforms centralized scheduling	complexity increases exponentially with increasing number of RRHs, including the greedy heuristic	sum rate
[58]	differential evolution algorithm	higher achievable sum-rates for varying number of antennas and increasing number of RRHs	increasing user number affects the sum-rate negatively due to the increasing interference	spectrum efficiency (bps/Hz)
[59]	user selection strategy based on fuzzy clustering	reduces interference to achieve total data rate gains	complexity increases exponentially with the number of users	total data rate

. In Section 4.2, we discuss the potential improvements of applying existing AI-ML solutions that are not part of the existing literature for converged optical and radio networks.

4.1. Selected Papers

Regarding the technical challenges of converged optical and mmWave radio networks, the selected papers in Table 3 focus on solving the beamforming optimization and interference mitigation for throughput maximization. From the ANM perspective, the selected throughput optimization algorithms reveal that both centralized and distributed management systems are used in solving dynamic bandwidth allocation (DBA) problems. However, there is a shift towards distributed throughput optimization solutions, especially with the involvement of different network stakeholders in management decisions.

The integration of massive MIMO and beamforming solutions for throughput optimization in optical and mmWave radio networks leads to solving the multiple objective resource allocation algorithms joint with power allocation [51] or beam selection [55] optimization solutions. In [51], an algorithm called volume adjustable backhaul constrained water-filling dynamic programming method is developed to maximize the downlink throughput in a FiWi mmWave network. In [55], the downlink data arrival rate is maximized for a mmWave small cell cloud radio access network (C-RAN) with free space optical fronthaul between RRHs and baseband units (BBUs). A resource optimization solution with Lagrangian dual decomposition is proposed in which the sub-problems are iteratively solved with a combination of separate optical fronthaul beam selection, fronthaul link selection, access link power allocation, and UE-RRH association algorithms.

Interference reduction is a common objective for throughput maximization in mmWave networks, and ML-based beamforming is used in [56,59] to solve the traditional interference reduction problems. A hybrid beamforming design based on joint spatial division and multiplexing and fuzzy c-means (FCM) clustering is proposed in [59]. FCM gives membership grades to UEs so that they can belong to several clusters. Dual-connectivity with a macro cell and mmWave cells is also considered as an alternative architecture for fair scheduling in the presence of interference in [56] with approximation algorithms based on the fractional weighted vertex coloring of conflict graphs method are used for throughput optimization under mutual interference.

Different stakeholders become involved in the decision-making framework of converged optical and radio mmWave networks, such as competing network operators in [42], network services with different QoS demands in [52], and users in [57]. The interactions among these stakeholders are defined by distributed cooperative and competitive models depending on their relation. Evolutionary game theory is used to model the interactions among the BSs and the PON for DBA in [42] and BSs change their strategies based on replicator dynamics. Three algorithms are developed in [52] to assign appropriate bandwidth to each service with priority based differentiation of the QoS demands of different services. A user and network sub-channel resource allocation problem is analyzed in a joint algorithm in [57] and the interactions are defined with an optimal algorithm based on Lagrange duality and a greedy algorithm based sub-optimal solution.

Distributed resource allocation methods are used not only to define the interactions between the stakeholders as in [42], but also to enable information exchange at the network edge [43,54]. In [54], a distributed control plane shifts control tasks to FiWi access nodes with interleaved polling with adaptive cycle time DBA scheme. The nodes are able to exchange control information, such as queue states and transmission needs with each other. In [43], a load balancing game and a bandwidth allocation game based on a bidding system are designed to overcome the problem of mapping the channels to ONUs in a PON for different traffic scenarios.

Throughput maximization for eMBB is a two sided problem for mmWave radio and optical networks, as priority based QoS-aware solutions are discussed [52] to realize such services with QoS guarantees, whereas minimum throughput targets for other services are also considered by applying fair scheduling [56]. Among the existing works, centralized management decisions are mainly used to respond to the throughput maximization-resource allocation fairness trade-off. The optimization solution in [43] has a centralized regulator, as the optical line terminal (OLT) acts as the resource manager to keep optimal fairness values. Another centralized solution is provided in [58] for solving the UE-RRH and RRH-BBU uplink sum-rate optimization problem with a differential evolutionary algorithm. Apart from this general trade-off discussion, the problem of translating eMBB application specific requirements to mmWave radio and optical resource allocation is not discussed in detail in the existing works.

4.2. Discussion

The increasing number of external stakeholders and the need for information exchange among the mmWave radio and optical networks is already reflected in several works [42,43,52,54,57]. Considering the expanding level of information with the increasing number of users, nodes, antennas in massive MIMO mmWave networks, distributed federated learning approaches provide a novel optimization solution on top of the existing beamforming and DBA solutions. Among the joint optimization algorithms for radio and optical access and transport resources, we were not able to detect any federated learning models, which can overcome the information exchange dependencies of the stakeholder interaction models listed in Table 3. This might be due to the fact that federated learning is considered a bandwidth-consuming method with the model updates requiring the transmission of many parameters that scale with the size of the access node deployments [60]. However, the capacity boost provided by the optical transport network for massive UDN deployments can overcome this limitation, thus providing a future research direction for resource optimization in radio and optical mmWave networks.

Regarding interference mitigation, the selected algorithms grouped under this category in Section 4.1 have complexity issues with the increasing number of users. Predictive analytics can, therefore, help in interference handling by monitoring user behavior and traffic load variation in the mmWave radio and optical networks [38]. However, the computational cost of learning from inter-channel information to mitigate inter-cell interference at the access nodes of dense massive MIMO systems should also be considered for throughput maximization [61]. A future research direction for throughput maximization might therefore be creating a multi-layer learning framework [6] to adjust fronthaul resources by learning the aggregate radio access node interference behavior.

The evaluation criteria in Table 3 show that the number of users are monitored frequently in throughput maximization solutions, bringing the integration of user demand-driven ML solutions, such as the deep reinforcement learning throughput maximization algorithm used to achieve a minimum throughput target of 1 Mbit/s for 50 users [62]. The throughput scales well above 1 Mbit/s with the available bandwidth in mmWave frequencies, dense deployments and the use of MIMO; making eMBB applications such as V2X and mobile augmented reality the use case targets of mmWave radio and optical networks. For this reason, reinterpreting such ML solutions should be considered carefully as mmWave radio and optical network deployments aim to achieve significantly higher the throughput targets, e.g., 1 Gbit/s for eMBB use cases such as V2X collective perception [63].

Due to the scaling up of the network elements and the dependent increase in the amount of information, ML applications were discussed for massive MIMOs, cognitive radios, heterogeneous networks, and small cells at an early phase of the convergence of AI-ML methods and communications technologies [64]. The number of access nodes and antennas are also considered are evaluation parameters in the selected papers [53,55,58]. These parameters should be considered not only in the operations and management phases, but also during the cost-effective planning and pre-deployment phases of mmWave access and transport networks. Finally, the communication-efficient methods to push training processes of AI models to edge nodes [65] should also be considered to increase the distributed network control capability and make use of the increasing amount of information at the edge.

5. Delay Minimization and Resource Allocation Algorithms

Minimizing the delay caused by congestion or increased traffic has been a common management target throughout the evolution of mobile networks. However, the emphasis on latency increased with the challenging ultra-reliable low latency communication (URLLC) requirements [48], due to the “1 ms challenge” of the delay sensitive applications, such as tactile internet [66] and augmented/virtual reality. In Section 5.1, we summarize the optimization methods used to reach these delay targets in converged optical and mmWave radio networks. The list of the selected algorithms with their advantages and limitations can be found in

Table 4. Analysis of resource allocation papers with delay minimization objective.

Ref.	Optimization Algorithm	Advantages	Limitations	Performance Metrics
[67]	a distributed iterative algorithm and broadcasting of the Lagrange multipliers	improves average response time while meeting the delay thresholds	fairness is not considered for power consumption	delay, energy consumption
[68]	a priority based resource allocation scheme is created	reduces the delay of the high QoS class traffic	the algorithm is not studied under different traffic loads	delay, throughput
[69]	two-phase fronthaul and access delay minimization with two separate iterative algorithms	fronthaul link and access transmission delays converge to a lower bound in all scenarios with different parameters	reference simulation parameters such as RRH transmit power and real network parameters may differ	transmission delay
[70]	algorithm based on penalty dual decomposition framework	minimizes the maximum system delay of a multi-user mmWave MEC system	complexity increases exponentially with the number of users and BSs	maximum delay
[71]	user-driven offloading scheme with guaranteed low end-to-end latency	provides infrastructure sharing while reducing latency and increasing energy efficiency	test-bed results may differ from real network as continuous information exchange is required between users and APs	response time, energy consumption
[72]	users and MEC servers cooperatively set their offloading probabilities	improves average response time when compared to MEC- and cloud-only solutions	offloading probability of MEC servers should be tuned properly	average response time
[48]	ANN for decentralized bandwidth allocation	around 1ms end-to-end delay can be achieved for Tactile Internet	networks with arbitrary delays are not considered	end-to-end delay

. The potential research directions for delay minimization in converged optical and radio networks with AI-ML solutions are discussed in Section 5.2.

5.1. Selected Papers

Among the technological enablers of ANM, MEC is heavily adopted in the selected works as it contributes to delay minimization [71,72] and offloading. In [71], a FiWi enhanced two-level edge computing concept is developed to guarantee low end-to-end latency with offloading capabilities. The UEs send their computation offloading tasks to their associated ONU-APs in this the user-driven approach. Ref. [72] introduces a cooperative offloading strategy that allows users and MEC servers to iterate backhaul and user offloading probabilities until the minimized delay converges to a near-optimal solution. MEC is also seen as an enabler to reach URLLC target of 1 ms end-to-end delay for a tactile internet application in [48], and an artificial neural network is used at the network edge to minimize delay together with the offloading scheme used in [72].

Delay minimization in optical and mmWave radio networks is also considered in the scope of QoS restrictions in [67,68]. The delay restrictions of different services are used as constraint in the optimization problem in [67]. The problem is decomposed using Lagrange duality theory and solved with a distributed iterative algorithm to maximize the utility of all users and providing a better average response time. A resource management scheme for FiWi fronthaul is presented in [68] in which time slots are allocated in a way that the packets of a QoS class with higher priority are sent using more time slots to reduce the delay of the high QoS traffic.

In order to realize an optical and mmWave radio network, the relation between delay and beamforming is studied in the literature [69,70]. An iterative algorithm is proposed for the beamforming at the central processor and the RRHs to minimize the fronthaul transmission delay for mmWave C-RAN in [69]. The computational capabilities of MEC can be used for beamforming as well, such as the joint beamforming and resource allocation algorithm presented in [70] for system delay minimization. The proposed dual decomposition-based distributed algorithm also has an information exchange mechanism between the system components. Another work that investigates the relation between delay and the use of beamforming is [69], in which a two-phase fronthaul and access transmission delay minimization method is proposed.

5.2. Discussion

As seen in Section 5.1, several works consider traffic level and UE number as an evaluation criteria for delay minimization; however the complexity of the algorithms are bounded with the number of users and the level of traffic. To achieve the delay minimization objective without encountering complexity issues, AI/ML-based traffic forecasting methods are highly relevant to exploit the data collected from monitoring and to reveal patterns such as peak hours, thus simplifying the solution space. A forecasting example that uses neural networks for delay minimization is already provided for FiWi networks in [48]. By taking the optimal training time and forecast accuracy trade-off into account, different neural networks approaches can also be used to detect patterns and forecast the network traffic characteristics, such as the traffic forecasting with long-short term memory method in [73].

Regarding the deployment of MEC solutions, there deep reinforcement learning based delay minimization solutions [74] and edge caching [75] can achieve significant QoS improvements for applications, such as video streaming, but the impact of integrating these application-based decision-making mechanisms to optical and mmWave radio networks has not been studied in the existing papers, providing a novel research direction for delay minimization. The limitations of MEC solutions provided in Table 4 also reveal that the solutions depend on test-bed results and simulations; therefore impact of real network dynamics should be thoroughly studied in future works.

In delay optimization papers, users [67], network services [68], and cloud servers [72] are also involved in decision-making as external stakeholders. MEC servers could also be third party stakeholders who lease their computing resource blocks to process or store for dynamic function placement, providing AI-as-a-Service [1] or Security-as-a-Service to other network stakeholders and making service providers an external decision-maker in network management for service-specific decisions.

6. Energy Efficiency and Resource Allocation Algorithms

The objective of energy efficiency is defined as maximizing the amount of data transferred per unit energy consumed by the system [76]. In Section 6.1, we present the selected papers given in

Table 5. Analysis of resource allocation papers with energy-efficiency objective.

Ref.	Optimization Algorithm	Advantages	Limitations	Performance Metrics
[79]	heuristic algorithm to solve network formation game	optimizes energy under traffic load and the base energy consumed in sleep mode	fairness aspect is not considered for protection of the less demanding nodes	EE gain (Joules/s)
[44]	one-to-many matching game between users and subcarriers	maximizes EE gain and sum rate for both microwave and mmWave bands	the solution has polynomial time complexity	EE gain, sum rate
[80]	iterative algorithm with Dinkelbach method	improves EE by simultaneously assigning a subcarrier to multiple users	the solution assumes perfect channel state information	downlink EE, sum rate
[81]	SDN application monitors and estimates energy consumption	achieves minimum power consumption per flow for different arrival rates	the power gain diminishes with a high flow arrival rate	power consumption
[45]	iterative algorithm solves a non-cooperative transmission power game	reaches global optimization after a few iterations, has a memory factor to overcome estimation errors	delay of exchanging information among BSs is not considered	per spectrum EE, power consumption
[77]	iterative solution with alternating descent algorithm	jointly solves minimization of the total power and maximization of minimum rate for each user	EE decreases with the increased minimum rate for users	achievable EE (bits/Joule)
[46]	two layered game with frequency assignment and power allocation	joint EE and SE maximization under increasing number of users and BSs	complexity analysis and possible complexity reduction options are not provided	EE (revenue per cost)
[82]	a matching heuristic and a user reallocation heuristic	user association successful in distributing high and low data rate demanding users	multi-connectivity option is not considered for users	total power consumption
[78]	iterative algorithm with Lagrangian dual decomposition	proposed algorithm requires low fronthaul bandwidth for a given EE	does not consider any interference between the macro BS and the RRHs	EE gain (bits/Joule)
[49]	adaptive ant colony optimization	avoids frequent deactivation and reactivation of lightpaths for new traffic requests	The complexity of the algorithm increases exponentially with the number of nodes in the network	total power consumption
[83]	alternating direction method of multipliers algorithm	minimizes energy consumption consumed by content caching, data computing and traffic transmission	high resource demanding multimedia applications are not considered for optimization	energy consumption

, which provide EE solutions to converged optical and radio networks with AI-based techniques. A discussion on the future challenges and potential research directions are provided in Section 6.2.

6.1. Selected Papers

Resource allocation algorithms with EE optimization consider the trade-off between EE and bandwidth utilization. Thus, joint optical and radio resource and transmission power allocation algorithms are used to minimize the total consumed power in the entire network. For instance, in [45], the NP-hard problem of the joint uplink resource allocation of small cells, spectrum resources, and transmission power is decomposed into a potential game for small cell selection and a non-cooperative game for power allocation. The EE maximization problem is formulated in [77] in terms of number of bits delivered per unit of Joule subject to the QoS rate threshold for each user, and an alternating descent algorithm is applied to separate the energy efficiency optimization problem into two sub-problems of EE maximization problem and user throughput fairness. In [78], the EE maximization problem is modeled as a class of optimization problems called fractional programming to minimize the total power consumption of the entire system.

With the increased use of dense small cell deployments due to mmWave characteristics, activating and deactivating these small cells with sleep modes based on different parameters, such as the number of APs and network loads [79], flow arrival rates [81], under the presence of a macro cell [82] has become a frequently used energy saving method. In [79], the system to optimize transmission and sleep periods is modeled with a network formation game, in which every AP is a player, establishing connections with its neighbors to create energy efficient routes. The minimization of the total power consumption in user association is modeled as a capacitated facility location problem and solved with the selection and repetition based heuristic algorithms for sleep mode decisions in [82].

Dual-connectivity network architectures with both macro cell and mmWave small cell options are also considered for the trade-off analysis of joint energy efficiency and throughput gain maximization under varying number of users [44,46,80]. The coexistence of microwave and mmWave leads to a better performance in terms of both sum rate and EE when compared to mmWave and microwave only networks with a one-to-many matching game for frequency band selection in [44]. Downlink resource allocation is investigated for a non-standalone network with macrocell and small cells in [46], and a two layered hierarchical game approach is used for modeling the problem. A non-cooperative frequency assignment game is designed for small cells in the first layer and a power and subcarrier allocation via joint maximization of the revenue per cost based EE and spectral efficiency (SE) in the second layer. In [80], a joint access and fronthaul radio resource allocation method is proposed for downlink dual connectivity mode of a power domain non orthogonal multiple access-based C-RAN system with mmW and microwave carriers.

The network paradigm shift towards softwarized solutions provides novel options to implement EE algorithms in the optical and radio networks, as seen from the SDN-based solutions proposed in [49,81,83]. An SDN application called energy management and monitoring application, and a power consumption optimizer is developed in [81] to optimize the energy consumption of a C-RAN infrastructure with energy consumption estimation based on flow rates. An SDN controller-based power control framework with an adaptive ant colony optimization algorithm is proposed in [49] to avoid the frequent deactivation and reactivation of the lightpaths when new traffic request arrives, thus saving switching power. Finally, a joint caching, computing, and bandwidth resource allocation is designed for SDN in [83] to minimize the energy consumption consumed by content caching, data computing and traffic transmission.

6.2. Discussion

There are several network EE issues that should be taken into consideration for management in optical and mmWave radio networks. The massive small cell deployments increase the signalling cost as mmWave bands have smaller coverage radii. In addition, antenna processing for massive MIMO antenna systems consumes extra power [84]. The selected papers in Table 5 reveal that the use of sleep modes and information exchange among small cells are some of the methods to respond to EE requirements of mmWave UDN deployments. The EE optimization methods identified for optical and mmWave radio networks can be enhanced with cognitive networking methods, where each node seeks to “minimize its energy” by minimizing the cumulative neighborhood energy function, as in [85]. Adapting cluster-based protocols used to harvested energy utilization for wireless sensor networks [86] for UDN deployments can also provide a novel research direction for converged optical and radio networks.

Apart from these operation and maintenance level for EE, such as the sleep modes, the reduction in energy related costs should also be a target during the planning and pre-deployment phases [87], which can be optimized by selecting the optimal AP density and distribution. The relationship between the transmit power and the AP density is defined as a function with the use of stochastic geometry in [88], and solving this non-linear function gives the unique transmit power and density that maximizes the energy efficiency. Modeling the energy performance of APs and optical transport network together should also be considered in the planning phase [89] to increase the overall EE of the network.

Finally, creating SDN applications that aim to achieve energy minimization as in [81] can be considered as strategy to overcome EE issues; however the architectural changes for network management by implementing SDN can also increase power consumption as it introduces new components with controllers and SDN switches. To analyze the power consumption impact of the architectural changes in the C-RAN fronthaul, Ref. [90] measures the power consumption of SDN switches, RRHs, optical transceivers and control components, and the results show that this architecture increases the total power consumption of the network by about 20%. The optimal EE solution therefore also requires planning of the power consumption of the SDN components in the pre-deployment phase. Finally, the limitations of the SDN-based solutions in Table 5 show that used algorithms have diminishing power gains for high arrival rate [81], have complexity issues with the increasing number of users [49], and do not consider high resource demanding multimedia services [83]. An SDN-based energy consumption solution for beyond-5G dense mmWave deployments providing eMBB services to users is therefore a topic to consider for future research.

7. Virtual Resource Allocation Algorithms

The paradigm shift with network softwarization leads to the abstraction of network resources and makes it possible to dynamically allocate computation and storage resources. In Section 7.1, we present the selected virtual resource management optimization solutions for converged optical and radio networks. The existing solutions to virtualized resource allocation optimization issues are listed in

Table 6. Analysis of Virtualized Resource Allocation Papers.

Ref.	Optimization Algorithm	Advantages	Limitations	Performance Metrics
[91]	achieve social welfare with an approximation algorithm	temporal demand fluctuation is solved with short-term auctioning	exact graph sub-sectioning algorithm is not feasible for large solution space size	welfare, resource utility
[92]	Dijkstra algorithm with a protection scheme	protection scheme guarantees service connection and high resource occupancy under heavy traffic load	scalability of the protection scheme is not considered	transmission success, utility
[50]	traffic prediction with Q-Learning to maximize InP revenue	InP uses global view of physical resources to predict the traffic load	service behavior and dynamic demands are not part of the solution	revenue, VN acceptance ratio
[47]	Vickrey–Clarke–Groves auction solved with matching game	auction provides pay-off gains to operators with distributed slice allocation	the problem has exponential complexity dependent on the number of users and BSs	average sum QoE
[93]	QoS-aware region division algorithm	global monitoring is used to decrease energy consumption for service profit	the algorithm introduces extra VN embedding delay	profit, energy consumption
[94]	revenue-based VNE allocation with two greedy algorithms	increases the acceptance ratio and profit rate by considering the gains of service requests	dynamic service requests of nodes are not part of the VNE algorithm	VN acceptance ratio, profit
[95]	centralized and mobility-aware resource pooling algorithm	resource pooling gain increases with the average speed of users	overhead and delay caused by user location tracking are not considered	resource pooling gain
[96]	breadth first search channel allocation to maximize profit	algorithm provides a higher InP profit for varying average VN arrival rates	exponential complexity, not feasible for large network graphs	profit, VN acceptance ratio

. In Section 7.2, we discuss how these solutions can be enriched with network slicing and the use of agents for distributed decision-making.

7.1. Selected Papers

Infrastructure sharing is made possible with the help of virtualization technologies, and this fact transformed the network architecture itself into a novel stakeholder called the infrastructure provider (InP) [50,96]. To maximize InP profit in FiWi access networks [96], propose resource allocation in both wireless and optical subnetworks with a wireless channel allocation algorithm based on breadth first search and a DBA algorithm in FiWi for both radio and virtual resources. A revenue based bandwidth resource allocation is provided in [50] to map the idle resources of a virtualized FiWi access network architecture to the service provider requests. The InP uses Q-learning to predict idle bandwidth resource and the traffic load on each physical link. This algorithm enables InPs to accept more VN requests and obtain higher revenue.

Virtualized network resources pave the way for dynamic resource allocation between the InPs and the network operators over auctioning algorithms. An infrastructure sharing scenario is created for mmWave radio networks in [47], in which the operators use a distributed auction mechanism (Vickrey–Clarke–Groves) to allocate mmWave AP resources by obtaining the slices. The results showed that applying these algorithms after the auction provides pay-off gains to the network operators. Another resource allocation scheme based on auctioning is discussed in [91]. In this scheme, operators submit bids to capacitate a C-RAN subnetwork, and the infrastructure owner (auctioneer) aims to maximize the aggregate social welfare, defined as the sum of the aggregate operators’ utility and the C-RAN’s revenue.

Even though profit generation was always a factor in performance optimization for network operators, the shift in the architecture with network virtualization makes dynamic revenue gains an apparent performance criteria for resource allocation. This shift transforms the network into a market interaction between different stakeholders that aim to profit from the available physical and virtual resources [93,94] in a dynamic way. In [94], a VNE problem is analyzed for FiWi hybrid nodes that have abstracted physical optical and wireless resources. The main objective is to maximize profit by considering the gains of service requests and costs of the physical plane including networking and edge computing servers’ resources. Virtual resource allocation in a FiWi network is combined with an energy saving perspective in [93]. FiWi network resources are monitored globally to put the low-load devices to sleep mode. The results show that the algorithm managed to provide low energy consumption, high network service profit, and high network link utilization.

The integration of network virtualization technologies creates the incentive to dynamically reconfigure both the physical and virtual resources of converged optical and radio networks. For this reason, the joint optimization of both radio, optical and virtual resources is studied in the literature. The radio, optical, and fog resources are controlled with SDN in a cross-layer architecture for a fog-computing-based radio over fiber network in [92], in which the controller selects MEC nodes and establishes paths with spectrum and modulated radio frequency allocation. A mmWave 5G C-RAN pooling gain solution over an ARoF fronthaul design is presented in [95], in which both virtual and physical resources are allocated with a resource pooling algorithm.

7.2. Discussion

The softwarization of network functions with NFV and their live migration thanks to the technological enablers such as SDN and MEC makes it possible to dynamically allocate computation and storage resources. Physical layer abstraction is required for both optical and radio resources of the converged network to achieve joint optimization of physical and virtual network resources at the NFV orchestrator level. Achieving the abstraction of all physical resources is a step towards achieving end-to-end network slicing and translating high-level O&M goals to technical parameters in beyond-5G networks, as seen in the efforts, such as [28,97].

As seen from Table 6, the addition of the infrastructure provider to the increasing number of the external stakeholders in 5G networks is also taken into account in several works. In addition to the abstraction of all physical and virtual resources of the network, the common abstraction at the network management framework is also required between these stakeholders. A well-known abstraction for the design and implementation of intelligent stakeholders in a distributed fashion is the use of intelligent agents [6]. The collaboration, cooperation, and negotiation of multiple agents to achieve a common goal creates a multi-agent system, which is an ideal candidate to solve complex resource management problems between multiple stakeholders in a distributed fashion. As seen from Table 6, current multiple stakeholder algorithms do not provide solutions that are feasible for a large solution space [91], and complexity grows with the increasing number of users and BSs [47], therefore implementing these solutions require multi-agent architectures that can optimize local network sub-graphs.

8. Conclusions

8.1. Contributions of This Study

In this article, a literature survey is presented to understand the key concepts and the key network state parameters used to evaluate the performance of AI-based network optimization algorithms, to identify the future demands and to analyze the options for novel contributions and the limitations of resource management in converged optical and mmWave radio networks. We aimed to identify the main features, objectives, and the resource allocation solution methods in mmWave networks by also considering the relationship between the use of optimization algorithms for virtualized resource allocation and the use of the ANM technological enablers in the 5G and beyond-5G network architecture. Furthermore, a dedicated discussion is provided for each resource management objective to identify the gaps in the existing literature and to provide potential directions for future research.

8.2. Limitations

This review provides an initial overview on the subject matter and in future can be expanded upon with further research on the resource optimization algorithms. We selected to exclude works that do not present a concrete optimization algorithm; however it is well-known that many conceptual papers also provide a basis for solving resource management problems. It should also be emphasized that AI-ML solutions presented in the discussion subsections do not directly consider a converged radio and optical mmWave architecture; therefore the re-interpretation of their results might be costly for some specific optimization algorithms. Despite the limitations, the contributions of this article add value to the discussion with respect to integrating ML-based data analytics solutions with converged optical and mmWave radio networks, and motivate further research towards the autonomic resource management for 5G and beyond-5G performance optimization.

8.3. Agenda for Future Research

Converged optical and mmWave architectures provide additional capacity, and most works selected to this review try to make best use of this capacity boost with optimization algorithms to maximize QoS and EE. However, from an ANM perspective, the resource allocation solutions with different optimization objectives are not decomposed into clear functional components, but instead designed as solutions to specifically modeled problems. Converged optical and radio networks can greatly benefit from the integration of the provided AI-ML solutions to data analytics frameworks provided by ITU-T, 3GPP, and other regulatory bodies; however, further research and cooperation is required to define the boundaries of the protocols of these frameworks and create self-adaptive interaction mechanisms between the network components and enabling technologies in order not to suffer from rigid architectural structures. As given in Section 7.2, such a framework should target bringing all physical and virtual network resources together with common abstractions and well-defined interfaces.

The increasing amount of information can be exploited with a common framework that considers data measurement, storage, and processing libraries for diverse AI/ML methods, such as federated and deep learning methods discussed in Section 4.2, Section 5.2 and Section 6.2. Furthermore, as ML components provide mechanisms generating knowledge with data, researchers cannot overlook privacy and security related issues during the acquisition, storage and transfer of these data when developing ML-based solutions. Applying “security by design” at each data related operation in a service-based softwarized network design should therefore be considered as a critical future research direction.