A Coalitional Model Predictive Control for the Energy Efficiency of Next-Generation Cellular Networks

Abstract

Next-generation cellular networks are large-scale systems composed of numerous base stations interacting with many diverse users. One of the main challenges with these networks is their high energy consumption due to the expected number of connected devices. We handle this issue with a coalitional Model Predictive Control (MPC) technique for the case of next-generation cellular networks powered by renewable energy sources. The proposed coalitional MPC approach is applied to two simulated scenarios and compared with other control methods: the traditional best-signal level mechanism, a heuristic algorithm, and decentralized and centralized MPC schemes. The success of the coalitional strategy is considered from an energy efficiency perspective, which means reducing on-grid consumption and improving network performance (e.g., number of users served and transmission rates).

1. Introduction

Large-scale systems such as telecommunication networks and power systems are comprised of numerous lower-dimensional subsystems that inter-operate to achieve a common goal. The subsystems are usually distributed over a wide geographical area, and it is necessary to develop control strategies to guarantee optimal system performance. From the control viewpoint, the main challenges of large-scale systems are to collect information from geographically distributed elements, coordinate control actions taken by different types of agents, and handle the coupling between subsystems.

Mobile cellular networks are large-scale systems that are expecting exponential growth in 5G and 6G systems due to the increasing number of subscribers and mobile devices. Concerning [1], over 70% of the global population will have mobile connectivity by 2023, and 5G devices and connections will be over 10% of global mobile devices and connections by 2023. The consequences of such growth in the demand for network capacity are also very well known. For the telecommunications industry, this implies a need to increase infrastructure deployment for the core and Radio Access Network (RAN), which means higher capital and operational expenditures. Note that the energy consumption costs are crucial in the operational expenditures.

In the 5G roadmap, environmentally friendliness is a mandatory characteristic defined by the International Telecommunication Union (ITU). In this sense, different alternatives have been studied to increase energy efficiency in the network segments. For example, the Study Group 5 of ITU Telecommunication Standardization Sector (ITU-T) is responsible for the environment and circular economy recommendations, and it is developing a standard called "L.1380: Smart energy solution for telecom sites" [2]. The ITU L.1380 objective is to describe techniques for the automated control of power feeding from multiple energy sources, to improve energy efficiency and reduce operating costs, e.g., using schemes that prioritize the intake of power from renewable energy sources.

Another critical issue is related to the energy consumption of the multi-functional user equipment in 5G and beyond. In this scenario, the multiband, high processing, and ultra-reliability capacities demand novel hardware solutions that facilitate cost-effective and energy-efficient user equipment design [3]. Although this paper focuses on the user-BS association process, it is essential to note that 5G is an ecosystem where each element and process plays a significant role.

Previous studies have shown that cellular networks consume about $0.5$% of the global energy supply [4,5]. This phenomenon has motivated several kinds of research focused on reducing on-grid consumption, e.g., Artificial Intelligence for Green Networks (AI4GREEN) [6] and Sustainable Cellular Networks Harvesting Ambient Energy (SCAVENGE) [7]. A common conclusion is that Base Stations (BSs) cause most on-grid consumption in cellular networks. Moreover, this consumption has a direct relationship with the traffic load [8]. From the energy efficiency perspective, mobile networks can be optimized by (i) redesigning the system to reduce energy consumption while achieving the same goals [9], (ii) reducing the energy losses [10], and (iii) changing the energy source to one with lower environmental impact [11,12]. In this paper, we focus on the energy sources and the optimization of the energy management policy to reduce consumption.

In the last few years, renewable energies have been promoted as the energy source for BSs. Various studies show that using renewable energy sources in Heterogeneous cellular Networks (HetNets) reduces network costs [13,14,15] and improves environmental factors [16]. Another appealing aspect is deploying infrastructure in off-grid and connection limited scenarios such as developing countries and disaster zones. Therefore, Next-Generation Cellular Networks (NGCNs) can become self-sustainable systems thanks to renewable energy sources [17].

The design of policies to reduce grid consumption in NGCNs is deeply connected with the user-BS association algorithms [18] since it determines how the network uses its resources to serve users. In the literature, it can be found several strategies to solve the load-balancing problem in HetNets. For example, Andrews et  al. [19] presented a survey of approaches for load-balancing in HetNets. There is a need for exploring new load-balancing mechanisms because the problem of associating users with BSs is Non-deterministic Polynomial-time hard (NP-hard) and may not be tractable even for small-sized HetNets. From the energy efficiency perspective, Zhou et al. [20] proposed a heuristic algorithm for target-cell selection combined with a power control algorithm for coverage optimization to guide users towards BSs with renewable energy supply in the handover process. Likewise, Han and Ansari proposed to optimize the use of renewable energy by cell-size optimization [21]. To this end, they divided the problem into two parts: a multi-stage energy allocation problem, and a multi-BS energy balancing problem. In [22], an off-line algorithm to optimize the renewable energy allocation across different time instants was proposed to minimize the on-grid energy consumption of a base station. In [23], the classic optimal-transportation approach to study the mobile association problem in cellular networks was employed. In [24], a profit-driven user association scheme by jointly considering the traffic delivery latency and renewable energy application was proposed to maximize the benefits of network providers in HetNets. These approaches differ from our proposal, which considers that the network structure can be dynamically clustered in sets of cooperating BSs (also known as coalitions) to optimize consumption and coordination overheads with energy and traffic forecast information.

In the control field, Model Predictive Control (MPC) stands for a family of control methods based on the receding horizon principle and can deal with multiple variables systems and complex issues such as delay times, constraints, and uncertainties [25]. A system model is used to predict its behavior, and the controller optimizes its evolution according to a specific cost function and the given constraints. For example, in [26], an MPC was implemented to handle the power signal in base stations and decrease the unfavorable influence of disturbances in the transmission process. Moreover, co-channel interference via distributed power adaptation with MPC was reduced in [27]. Nevertheless, the significant computation burden required to solve a centralized MPC can make it impossible or greatly difficult to implement in real time. For that reason, the divide-and-rule policy of distributed MPC techniques has been widely established over the last few decades to deal with large-scale problems [28]. A halfway approach is the coalitional MPC [29,30], which considers a dynamic system partition to find a trade-off between local performance and coordination overheads.

Relevant challenges such as network and energy management appear when renewable energy is used in NGCNs [31]. In this context, coalitional MPC becomes a suitable control strategy. The motivation of this work is to provide an efficient controller that holds the quality of service while reducing on-grid consumption. Moreover, the scalability of the method is also important to deal with the large spatial distribution of cellular networks. Hence, the main contribution of the article is to assess how Long-Term Evolution (LTE) systems powered by hybrid energy sources can be more sustainable in a scalable manner by using coalitional MPC. Likewise, unlike other studies such as [9,32,33], we propose a more realistic internal model for the controller that considers that the transfer of users between base stations is limited to neighboring base stations.

Furthermore, the proposed coalitional approach is compared with the traditional best-signal-level strategy, a reactive control scheme based on an energy-aware user-BS association mechanism [33] and decentralized and centralized MPC methods. These strategies are tested in two simulated scenarios: (i) an academic example with 9 BSs, and (ii) a complex scenario with 37 BSs. Results show that the proposed control scheme becomes a suitable option to reduce on-grid consumption while maintaining proper service levels to users.

Finally, a much earlier and shortened version of the proposed method was accepted for a conference presentation [34]. There are significant differences between the current article and the conference paper: (i) the conference version does not include the large scenario with 37 BSs, which represents the complexity of real problems; (ii) we present here new, improved methods for the selection of topology; (iii) a more realistic internal model for the controller is used that only considers user transfer between neighboring base stations; (iv) several refinements in the main algorithm are introduced in the current version; and (v) the experiments here performed are new and include the previously mentioned novelties.

The outline of the rest of the article is as follows. The problem statement is described in Section 2. The coalitional MPC algorithm and other assessed user-BS association techniques are detailed in Section 3. The control schemes are assessed in two simulated scenarios in Section 4. Finally, the authors discuss the results and give concluding remarks in Section 5.

Notation: ${\mathbb{N}}_{0+}$ and ${\mathbb{R}}_{>0}$ are, respectively, the sets of non-negative integers and positive real numbers. ${\mathbb{R}}^n$ represents an n-dimensional set of real numbers. For set $\mathcal{X}\triangleq \{1,\dots ,X\}\subseteq {\mathbb{R}}^n$, its cardinality is $\left|\mathcal{X}\right|=X$, and the subtraction of an element $i\in \mathcal{X}$ from $\mathcal{X}$ is denoted as $\mathcal{X}\backslash \left\{i\right\}$. The modulo operation is $\mathrm{mod}(a,b)$, with $a,b$ being positive integers. Given a set $\mathcal{C}\in \mathcal{M}$, the generalized union of sets is represented as ${\bigcup }_{\mathcal{C}\in \mathcal{M}}\mathcal{C}$.

2. Problem Description

Let us define a two-tier downlink HetNet as depicted in Figure 1, which is divided into $\mathcal{B}=\{1,2,\dots ,B\}$ base stations: a high-power Macro-cell Base Station (MBS) and numerous low-power Small-Cell Base Stations (SCBSs). In particular, the MBS is full-time available and connected to the electricity grid, while each SCBS is locally fed by renewable energy and equipped with a storage battery. Although primary coverage is provided by the MBS, the implementation of SCBSs broadens the network capacity since they receive data traffic from the MBS.

For simplicity, we follow the setting of [35,36] so that the inter-BS interference is modeled as a static value that includes the influence of other BSs in the network. This value varies depending on the activities in the interfering BSs, which can be coordinated via time-domain, frequency-domain, and power-control techniques [37]. Additionally, this simplification allows us to model the network as an MBS and multiple SCBSs without lacking generality.

2.1. Base Station Dynamics

The discrete-time dynamics of each base station $i\in \mathcal{B}$ is:

$$x_i(k+1)=A_i{x}_i\left(k\right)+\sum _{j\in {\mathcal{C}}_i\left(k\right)}{B}_{ij}{u}_{ij}\left(k\right)+w_i\left(k\right),$$

(1)

where $k\in {\mathbb{N}}_{0+}$ is the time instant; $x_i={[x_i^{\mathrm{z}},x_i^{\mathrm{b}}]}^{\top }$ is the state vector comprised by $x_i^{\mathrm{z}}$, which represents the users served by i, and $x_i^{\mathrm{b}}$, which is the energy stored in the battery of i. Input $u_{ij}$ is user flow, i.e., the amount of users exchanged between i and j, which belongs to the set of subsystems cooperating with i, namely ${\mathcal{C}}_i$. Hence, it is satisfied that $u_{ij}=-u_{ji}$ for $i,j\in {\mathcal{C}}_i$, and $u_{ij}=0$ for all $j\notin {\mathcal{C}}_i$. Moreover, $A_i$ is the state matrix, and $B_{ij}$ is the input-to-state matrix from neighbors $j\in {\mathcal{C}}_i$. The disturbance vector $w_i\in {\mathbb{R}}^2$ consists of the traffic generated by the incoming and outgoing users and the atmospheric conditions that influence the production of renewable energy.

2.2. Coalition Dynamics

Each subsystem $i\in \mathcal{B}$ is governed by a local controller with partial information regarding the overall system state. The set of controllers can cooperate in a topology network described by an undirected graph $(\mathcal{B},\mathcal{L})$, where $\mathcal{L}$ is the set of links or edges $\mathcal{L}\subseteq {\mathcal{L}}^{\mathcal{B}}=\left\{\{i,j\}\left|\right\{i,j\}\subseteq \mathcal{B},i\ne j\right\}$. Each link provides a bidirectional information flow between BSs i and j that can be enabled or disabled by the control scheme. Each enabled link involves a fixed cooperation cost ${\mathrm{c}}_{\mathrm{link}}\in {\mathbb{R}}_{>0}$, which will be selected large enough to stimulate cooperation, considering that a small value and a huge value will always lead to a centralized and a decentralized network, respectively. The active links $L\le \left|\mathcal{L}\right|$ define the network topology $\mathrm{\Lambda }\in \mathcal{T}$ at time instant k. The set of all possible topologies is $\mathcal{T}\triangleq \{{\mathrm{\Lambda }}_0,{\mathrm{\Lambda }}_1,\dots ,{\mathrm{\Lambda }}_{2^{\left|\mathcal{L}\right|-1}}\}$, where the decentralized topology is ${\mathrm{\Lambda }}_{\mathrm{dec}}={\mathrm{\Lambda }}_0$ and the centralized one is ${\mathrm{\Lambda }}_{\mathrm{dec}}={\mathrm{\Lambda }}_{2^{\left|\mathcal{L}\right|-1}}$. Should there be a substantial number of links, a subset of topologies can be considered to avoid combinatorial explosion issues.

Thanks to the capacity of the MBS to deal with all incoming users (at the cost of reducing its quality of service), coalition problems can be decoupled. Given topology $\mathrm{\Lambda }$, the disjoint set of coalitions is represented as $\mathcal{B}/\mathrm{\Lambda }$ with ${\bigcup }_{\mathcal{C}\in \mathcal{B}/\mathrm{\Lambda }}\mathcal{C}=\mathcal{B}$. The discrete-time dynamics of each coalition $\mathcal{C}$ is:

$$x_{\mathcal{C}}(k+1)=A_{\mathcal{C}}{x}_{\mathcal{C}}\left(k\right)+\sum _{j\in \mathcal{C}\left(k\right)}{B}_{\mathcal{C}}{u}_{\mathcal{C}}\left(k\right)+w_{\mathcal{C}}\left(k\right),$$

(2)

where $x_{\mathcal{C}}={\left(x_i\right)}_{i\in \mathcal{C}}$ and $u_{\mathcal{C}}={\left(u_{ij}\right)}_{i,j\in \mathcal{C}}$ are, respectively, the aggregated state and input vectors of all base stations in $\mathcal{C}$, and disturbance $w_{\mathcal{C}}$ represents disturbances caused by the effect of the weather and the coupling with neighboring coalitions.

2.3. Energy Consumption Model

The energy consumption model here employed was proposed by project EARTH and has been widely used in studies related to energy efficiency in cellular networks [35,38,39]. According to EARTH, the energy consumed by base station $i\in \mathcal{B}$ is given by dynamic and static energy consumption as in [8]:

$$E_i\left(k\right)={\Delta }_i{\delta }_i\left(k\right)T_i+E_i^{\mathrm{S}},\forall i\in \mathcal{B},$$

(3)

where ${\Delta }_i$ is the slope of the load-dependent energy consumption; ${\delta }_i$ denotes the traffic load; $T_i$ is the transmission power; and $E_i^{\mathrm{S}}$ is the static energy consumption, which is relevant to the amount demanded for the appropriate operation of a base station. Note that the dynamic energy consumption is the additional energy required to cope with the traffic load.

Since the total consumption is the sum of the on-grid consumption of the MBS ($i=1$) and the renewable energy consumption of all SCBSs $i\in \mathcal{B}\backslash \left\{1\right\}$, the reduction of grid consumption is crucial to increase energy efficiency. Finally, the available renewable energy $E_i^{\mathrm{G}}$ in SCBS i at time instant k is:

$$E_i^{\mathrm{G}}\left(k\right)=E_i^{\mathrm{G}}(k-1)-E_i(k-1)+\phi \left(k\right),$$

(4)

where $E_i^{\mathrm{G}}(k-1)$ and $E_i(k-1)$ are, respectively, the stored and consumed renewable energy at time instant $k-1$, and $\phi \left(k\right)$ is the renewable energy generation rate at time instant k.

One of the main issues of renewable energy is the management of the stored energy, especially how to maintain appropriate quality of service levels according to renewable energy availability. In this sense, it is crucial to design a mechanism that guarantees the continuity of the service in periods without renewable energy arrival.

2.4. Data Traffic Model

Let us define the geographical area $\mathcal{A}$ where BSs are spatially distributed at positions $p\in \mathcal{A}$ and a set of available base stations ${\mathcal{B}}_z\left(k\right)\subseteq \mathcal{B}$ that provides service to user $z\in \mathcal{Z}\triangleq \{1,2,\dots ,Z\}$ at time instant k. The spatial variability of traffic requests is considered and modeled as a Non-Homogeneous Poisson Process (NHPP) as in [36]. The traffic size, the arrival rate per area ${\lambda }^p$, and the average traffic size ${\mu }^p$ for all p are independently distributed.

The transmission rate $r_{zi}^p$ of a mobile user $z\in \mathcal{Z}$ can be generally calculated according to the Shannon–Hartley theorem [36] as:

$$r_{zi}^p\left(k\right)={BW}_i{\log }_2\left(1+{\psi }_{zi}^p\left(k\right)\right),\forall z\in \mathcal{Z},i\in {\mathcal{B}}_z\left(k\right),p\in \mathcal{A},$$

(5)

with ${BW}_i$ being the working bandwidth for all $i\in {\mathcal{B}}_z\left(k\right)$; and ${\psi }_{zi}^p$ being the received signal at location p from i, which is given by the Signal-to-Interference-plus-Noise Ratio (SINR) as:

$${\displaystyle {\psi }_{zi}^p=\frac{T_i{g}_i^p}{{\sigma }^2+\sum _{j\in {\mathcal{B}}_z\backslash \left\{i\right\}}{T}_j{g}_j^p},\forall i,j\in {\mathcal{B}}_z\left(k\right),}$$

(6)

where $T_i$ is the transmission power of i; the channel gain between BS i and the user at position p is $g_i^p$, which captures the slow fading signal containing path loss and shadowing; and finally, ${\sigma }^2$ is the noise power level. Notice that the denominator depicts the interfering BSs transmission towards user z at location p as Gaussian noise. Moreover, ${\psi }_{zi}^p$ must be higher than a threshold ${\psi }_{\min }$ so that user z has enough signal level and can be associated with i. For simplicity, the location indicator p will be omitted when referring to user z.

Each SCBS $i\in {\mathcal{B}}_z\backslash \left\{1\right\}$ is assumed to serve a fixed number $z_i^{\max }$ of users simultaneously. Conversely, the service is full-time guaranteed because the MBS offers an unlimited user service. Considering users uniformly distributed in $\mathcal{A}$, the traffic load ${\delta }_i$ is:   

$${\delta }_i\left(k\right)=\{\begin{array}{c}{\displaystyle \frac{{\sum }_{z\in \mathcal{Z}}{y}_{zi}\left(k\right)}{z_i^{\max }},\forall i\in {\mathcal{B}}_z\left(k\right)\backslash \left\{1\right\}}\hfill \\ \frac{{\sum }_{z\in \mathcal{Z}}{y}_{zi}\left(k\right)}{\left|\mathcal{Z}\right|},\mathrm{otherwise}\hfill \end{array}$$

(7)

where $y_{zi}$ represents the user association indicator, i.e., $y_{zi}=1$ when user z is attended by i and $y_{zi}=0$ otherwise. Additionally, the traffic load is bounded by $0\le {\delta }_i\left(k\right)\le 1$ for all $i\in {\mathcal{B}}_z\left(k\right)$.

2.5. On-Grid Energy Consumption Optimization Problem

Reducing on-grid consumption is a design requirement in NGCNs. Regarding Equation (3), the energy consumption of $i\in \mathcal{B}$ depends on traffic load ${\delta }_i$, i.e., the number of active users connected to a base station. Let us define the vector of MBS-user association indicators as $Y_{z1}\left(k\right)\triangleq [y_{11}\left(k\right),\dots ,y_{Z1}\left(k\right)]$. The objective is to minimize the number of users connected to the MBS as:

$$\begin{array}{c}\hfill {Y_{z1}\left(k\right)}^{*}=arg\underset{Y_{z1}\left(k\right)}{min}\sum _{k=1}^N\sum _{z\in \mathcal{Z}}{y}_{z1}\left(k\right),\end{array}$$

(8a)

subject to

$$\begin{array}{cc}\hfill \sum _{z\in \mathcal{Z}}{y}_{zi}\left(k\right)& \le z_i^{\max },\forall i\in \mathcal{B}\backslash \left\{1\right\},\hfill \end{array}$$

(8b)

$$\begin{array}{cc}\hfill y_{zi}\left(k\right)·{\psi }_{zi}{\left(k\right)}^p& \ge {\psi }_{\min },\forall z\in \mathcal{Z},i\in {\mathcal{B}}_z\left(k\right),\hfill \end{array}$$

(8c)

$$\begin{array}{cc}\hfill \sum _{i\in \mathcal{B}}{y}_{zi}\left(k\right)& \le 1,\forall z\in \mathcal{Z},\hfill \end{array}$$

(8d)

$$\begin{array}{cc}\hfill y_{zi}\left(k\right)& \in \{0,1\},\forall z\in \mathcal{Z},i\in \mathcal{B},\hfill \end{array}$$

(8e)

where $k\in [1,N]\cap {\mathbb{N}}_{0+}$ with N being the simulation length, Equation (8a) is the cost function to minimize, and inequality Equations (8b)–(8e) are the problem constraints: Equation (8b) establishes that SCBSs can serve a maximum of $z_i^{\max }$ users simultaneously; Equation (8c) is the user’s received signal level constraint; Equation (8d) requires that a user is served only by one BS in a time instant; and finally, Equation (8e) establishes that $y_{zi}$ is a binary variable. Note that the optimization problem is solved according to the specific network characteristics at each time instant k. This fact allows considering the variability of renewable energy sources in the solution.

The optimization problem Equation (8a) is a Mixed-Integer Linear Problem (MILP), a well-known $\mathcal{NP}$-hard problem [40]. The coalitional MPC control strategy proposed in this paper relaxes this problem to provide a suitable alternative to achieve a sub-optimal solution with reduced computation and cooperation burden. The next section shows how user-BS association decisions are made by a supervisory controller, which relates the number of users at the BS with the corresponding battery level. Thus, users will be initially associated with their closest base station. Later on, sets of cooperating BSs will decide whether to exchange users to obtain a trade-off between energy efficiency and service levels.

3. Control Methods

This section details the proposed coalitional MPC algorithm and other assessed methods: (i) centralized and decentralized MPC, which provide, respectively, an upper and lower bound of the performance; (ii) an energy-aware heuristic method; and (iii) the best-signal-level technique, which is a well-known user BS-association strategy. Unlike other studies such as [9,32,33], we here propose a more realistic internal model for the controller that considers that the transfer of users between base stations is limited to neighboring base stations.

3.1. Coalitional MPC

This strategy promotes dynamic cooperation between local controllers to improve coordination, giving rise to the so-called coalitions, which implement an MPC controller to calculate the most appropriate control actions at the coalition level and optimize the overall system performance. At each time instant, optimal inputs are calculated for a prediction horizon of $N_{\mathrm{p}}$ time steps according to a cost function. The first component of the optimal input sequence is applied, and the remaining elements are then discarded. This procedure can be depicted as a sliding window optimization process over time because it is repeated at each time instant [28].

For the centralized control, the problem is solved by a single agent with full information. Nevertheless, this setup scales very poorly due to the increase of the computational burden with the problem size. By contrast, non-centralized architectures are often preferred due to to their inherent redundancy in the decision making process. One possible solution is to solve the centralized problem in a distributed fashion using distributed algorithms, but this can only be done at the expense of increasing the cooperation effort, which can be measured in terms of computation and communication burden. In general, the more agents involved in a negotiation, the more iterations required to attain an agreement. Hence, one may be willing to pay a cost in terms of performance if the cooperation burden can be decreased. To this end, we introduce a communication cost penalty in the problem setup.

Each communication link $l\in \mathcal{L}$, which involves a cost $c_{\mathrm{link}}\in {\mathbb{R}}_{>0}$, can be either enabled or disabled. Note that the role played by this cost is crucial in this context. Otherwise, there would be no incentive to disable links, and full communication would be used because it provides the best control performance. However, this latter fact may cause the overall control problem to be significantly large to be solved in real-time, and the use of specific links is restricted due to the energy consumption constraints.

The objective of coalition $\mathcal{C}$ is to optimize the cost function $J_{\mathcal{C}}(·)$, which is the sum of a stage cost function $l_{\mathcal{C}}\left(x_1^{\mathrm{z}}\left(k\right)\right)$ and a penalty function, at each time instant k:

$$\begin{array}{c}{J}_{\mathcal{C}}\left(x_1^{\mathrm{z}}\left(k\right),{\delta }_{\mathcal{C}}\left(k\right)\right)={\displaystyle \sum _{t=0}^{N_{\mathrm{p}}-1}}\underset{l_{\mathcal{C}}\left(x_1^{\mathrm{z}}\left(k\right)\right)}{\underbrace{x_1^{\mathrm{z}}{(k+t)}^{\top }Q{x}_1^{\mathrm{z}}(k+t)}}+{\delta }_{\mathcal{C}}{(k+t)}^{\top }{\alpha }_{\mathcal{C}}{\delta }_{\mathcal{C}}(k+t),\end{array}$$

(9)

where $x_1^{\mathrm{z}}(k+t)$ is the amount of users served by the MBS at the corresponding time slot t of the prediction horizon; $Q>0$ is a constant weighting parameter that penalizes the number of users at the MBS; ${\alpha }_{\mathcal{C}}\in {\mathbb{R}}_{+}$ is a weighting parameter; and ${\delta }_{\mathcal{C}}(k+t)$ is a slack variable used as a soft constraint.

The proposed coalitional MPC algorithm presents a hierarchical control architecture [29], where an upper layer computes the best new topology every $T_{\mathrm{up}}$ instants, considering the current disturbances. Specifically, it explores the set of potential successor topologies ${\mathcal{T}}_{\mathrm{new}}\subseteq \mathcal{T}$ with a search strategy to select the new topology. The set ${\mathcal{T}}_{\mathrm{new}}$ is formed by those topologies $\mathrm{\Lambda }\in \mathcal{T}$ that only differ one link with respect to the current topology ${\mathrm{\Lambda }}_{\mathrm{cur}}$, i.e., those whose Hamming distance is one, as follows:

$$\begin{array}{cc}\hfill {\mathcal{T}}_{\mathrm{new}}\triangleq & \{\mathrm{\Lambda }\subset {\mathrm{\Lambda }}_{\mathrm{cur}}:\left|\mathrm{\Lambda }\right|=|{\mathrm{\Lambda }}_{\mathrm{cur}}|-1\}\cup \{{\mathrm{\Lambda }}_{\mathrm{cur}}\subset \mathrm{\Lambda }:\left|\mathrm{\Lambda }\right|=|{\mathrm{\Lambda }}_{\mathrm{cur}}|+1\}.\hfill \end{array}$$

(10)

Once the topology ${\mathrm{\Lambda }}_{\mathrm{new}}\in {\mathcal{T}}_{\mathrm{new}}$ is selected, it is sent to the lower control layer where coalitions must be formed in accordance with this topology.

Let us define the vector of user flow of coalition $\mathcal{C}\in \mathcal{B}/\mathrm{\Lambda }$ as:

$$U_{\mathcal{C}}\left(k\right)\triangleq [u_{\mathcal{C}}\left(k\right),u_{\mathcal{C}}(k+1),\dots ,u_{\mathcal{C}}(k+N_{\mathrm{p}}-1)].$$

The objective is to minimize the user flow from coalition C to the MBS at each time instant k:

$$\begin{array}{c}\hfill \{U_{\mathcal{C}}{\left(k\right)}^{*},{\delta }_{\mathcal{C}}{\left(k\right)}^{*}\}=arg\underset{U_{\mathcal{C}}\left(k\right),{\delta }_{\mathcal{C}}\left(k\right)}{min}{J}_{\mathcal{C}}\left(x_1^{\mathrm{z}}\left(k\right),{\delta }_{\mathcal{C}}\left(k\right)\right),\end{array}$$

(11a)

subject to Equation (2) and:   

$$\begin{array}{c}\hfill \sum _{i,j\in \mathcal{C}}{u}_{i,j}(k+t)\le F_{i,j}(k+t),t=0,\dots ,N_{\mathrm{p}}-1,\end{array}$$

(11b)

$$\begin{array}{c}\hfill \sum _{j\in \mathcal{C}}{u}_{i,j}(k+t)=x_i^{\mathrm{z}}(k+t),i\in \mathcal{B},t=0,\dots ,N_{\mathrm{p}}-1,\end{array}$$

(11c)

$$\begin{array}{c}\hfill u_{i,j}(k+t)\ge 0,i,j\in \mathcal{B},t=0,\dots ,N_{\mathrm{p}}-1,\end{array}$$

(11d)

$$\begin{array}{c}\hfill {\delta }_{\mathcal{C}}(k+t)\ge 0,\forall i\in \mathcal{C},t=0,\dots ,N_{\mathrm{p}}-1,\end{array}$$

(11e)

$$\begin{array}{c}\hfill x_{\mathcal{C}}(k+t)+{\delta }_{\mathcal{C}}(k+t)\ge 0,t=1,\dots ,N_{\mathrm{p}},\end{array}$$

(11f)

$$\begin{array}{c}\hfill G_{x_{\mathcal{C}}}\left(x_{\mathcal{C}}(k+t)+{\delta }_{\mathcal{C}}(k+t)\right)\le g_{x_{\mathcal{C}}},t=1,\dots ,N_{\mathrm{p}},\end{array}$$

(11g)

where Equation (11b) indicates that the sum of user flows between two BSs cannot pass an upper limit; Equation (11c) imposes flow conservation, that is that the sum of user flows must be equal to the served users $x_i^{\mathrm{z}}$; Equation (11d) and Equation (11e) define, respectively, that user flows and the slack variable must be positive; Equation (11f) imposes that the sum of the state vector and slack variable must be positive; and finally, limits $G_{x_{\mathcal{C}}}$ and $g_{x_{\mathcal{C}}}$ define a convex set containing the origin in its interior, i.e., Equation (11g) constrains the number of users served and the battery level of SCBSs as $0\le x_i^{\mathrm{z}}\le z_i^{\max }$ and $0\le x_i^{\mathrm{b}}\le b_i^{\max }$, respectively.

Finally, the proposed Algorithm 1 is summarized as follows.

Algorithm 1: Coalitional MPC.

Initial data:$N,T_{\mathrm{up}},Q,F_{i,j},z_i^{\max },b_i^{\max },c_{\mathrm{link}},{\alpha }_{\mathcal{C}}$. Starting point:$k=0,\mathrm{\Lambda }={\mathrm{\Lambda }}_{\mathrm{cen}},x_i=x_i\left(0\right)$. Main program:  for$k\le N-1$ do   if$mod(k,T_{\mathrm{up}})=0$ then    1: The upper control layer measures states $x_{\mathcal{C}}\left(k\right)$ and disturbances $w_{\mathcal{C}}\left(k\right)$, $\forall \mathcal{C}\in \mathcal{B}/\mathrm{\Lambda }$.    2: The upper layer computes the set ${\mathcal{T}}_{\mathrm{new}}$ (Equation (10)) and solves Equation (11) subject    to Equation (2) for each ${\mathcal{T}}_{\mathrm{new}}\subseteq \mathcal{T}$. Afterwards, the topology $\mathrm{\Lambda }\in {\mathcal{T}}_{\mathrm{new}}$ with the lowest cost    is selected.    3: Send the new topology $\mathrm{\Lambda }$ to the lower control layer.    4: Apply the first element of $U_{\mathcal{C}}^{*}\left(k\right)$ to coalition system (Equation (2)), for all $\mathcal{C}\in \mathcal{B}/\mathrm{\Lambda }$.   else    5: In the lower layer, each coalition $\mathcal{C}$ solves Equation (11) subject to its dynamics and    constraints.    6: Apply the first element of $U_{\mathcal{C}}^{*}\left(k\right)$ to coalition system (Equation (2)), and obtain $x_{\mathcal{C}}(k+1)$,    $\forall \mathcal{C}\in \mathcal{B}/\mathrm{\Lambda }$.   end if   7: Set $k\leftarrow k+1$.  end for

3.2. Centralized and Decentralized MPC

In centralized MPC, full communication is enabled, and hence, there is only one cooperation component $\mathcal{C}=\mathcal{B}$. The discrete-time dynamics of the overall system is:

$$x_{\mathcal{B}}(k+1)=A_{\mathcal{B}}{x}_{\mathcal{B}}\left(k\right)+\sum _{j\in \mathcal{B}}{B}_{\mathcal{B}}{u}_{\mathcal{B}}+w_{\mathcal{B}}\left(k\right).$$

(12)

Therefore, Equation (11) is computed each k for the great coalition $\mathcal{C}=\mathcal{B}$. Since there is full system information, the sheer performance is better than coalitional control. However, this technique needs full cooperation each time instant, leading to a higher computation burden and cooperation costs, which may hinder its implementation.

Conversely, when there is no cooperation between agents, Equation (11) is solved for $\mathcal{C}=\left\{i\right\},\forall i\in \mathcal{B}$ at each time instant k. This fact occurs in decentralized MPC, where each coalition is formed by a single local controller that manages each base station (Equation (1)). A lack of full information leads to worse performance than a coalitional strategy at the expense of reducing the computation burden.

3.3. Energy-Aware Heuristic

In [33], the authors proposed a renewable energy-aware user equipment-BS association (named green association) to reduce the on-grid energy consumption in HetNets supplied with hybrid (grid and renewable) energy sources.

Given the impact of active users on the overall energy consumption, the proposal focuses on minimizing the on-grid consumption by balancing the downlink traffic loads among BSs with an appropriate user-BS association scheme. The reactive control scheme modifies the BS-selection procedure by prioritizing renewable-powered BSs over a grid-powered MBS.

3.4. Best-Signal-Level Policy

Traditional cellular networks employ the best-signal-level policy, which consists of mobile users connecting to the BS with the best SINR. Issues such as power transmission, path loss, and interference with other BSs can influence the SINR of a base station. Conversely, the best-signal-level mechanism (referred to as the traditional scheme) is not fully suitable for hybrid-energy HetNets because available SCBSs can be ignored by users who receive a stronger signal from the MBS [41]. The traditional scheme will be used as the baseline to assess the proposed coalitional MPC algorithm and other MPC-based schemes.

4. Illustrative Examples

Controllers are assessed in the MATLAB environment. To this end, two case studies that consist of a two-tiered downlink HetNet composed of an MBS and multiple SCBSs are considered. Mainly, it is the impact on grid energy consumption and stored energy management that we want to analyze.

4.1. Case Studies

The following two simulated HetNets are proposed to test the proposed coalitional MPC algorithm:

• An academic scenario with a set of $\mathcal{B}=\{1,\dots ,9\}$ base stations (one MBS and eight SCBSs), which is sophisticated enough to show the potential of the proposed method (see Figure 2a). The complexity of the association process is caused by the number of BSs and active users.
• A large scenario with a set of $\mathcal{B}=\{1,\dots ,37\}$ base stations (one MBS and 36 SCBSs), which represents the complexity of real problems (see Figure 2b). In this case, it is only possible to compare the proposed method with a heuristic approach.

In both cases, the MBS is fed by on-grid energy to ensure coverage over the area. On the other hand, SCBSs are locally fed by renewable energy and equipped with storage batteries. Only path losses between users and BSs are considered. Furthermore, user requests follow a Homogeneous Poisson Process (HPP) with ${\lambda }^p=\lambda$ for simplicity.

4.2. Simulation Parameters

The implementation of user-BS association controllers is assumed to be faster than the dynamics of user traffic. When new information is available, the calculations are updated accordingly. Moreover, changes in renewable energy arrival and the spatial variability in the behavior of active users are considered in a simulation of $N=50$ time instants. According to [8], a summary of the simulation parameters is displayed in

Table 1. Summary of the main parameters.

	Description	Value	Units
$\mathcal{A}$	Coverage area	$3.5$	${\mathrm{km}}^2$
$BW$	Bandwidth of the LTE system	20	MHz
$d_{i,j}$	Inter-site distance	500	m
$T_1$	Transmission power of the MBS	43	dBm
$T_i$	Transmission power of the SCBSs	22	dBm
$E_1^{\mathrm{S}}$	Static power consumption of the MBS	130	W.h
$E_i^{\mathrm{S}}$	Static power consumption of the SCBSs	$6.8$	W.h
${\Delta }_1$	Consumption slope of the MBS	$4.7$	–
${\Delta }_i$	Consumption slope of the SCBSs	$4.0$	–
$F_{i,j}$	Max. users flow between the BSs	100	–
$z_i^{\max }$	Max. users served by the SCBSs	200	–
$b_i^{\max }$	Max. battery capacity of the SCBSs	200	W.h
$N_{\mathrm{p}}$	Prediction horizon	5	–
$T_{\mathrm{up}}$	Upper layer execution	10	–
Q	Weighting parameter	100	–
${\alpha }_i$	Weighting factor	${10}^7$	–
$c_{\mathrm{link}}$	Cost per cooperation link	${10}^4$	–
-	Path loss between the MBS and z	Cost 231 model	–
-	Mobility model	Random walk	–
-	Mobility speed	4	km/h

. The impact of tuning parameters in coalitional control has been explored in other references such as [29,42,43]. In this case, we keep the parameters constant so as to focus on the difference of performance between the centralized optimization and the proposed method, which is the main goal of the article.

Furthermore, 5G defines three general bands for the deployment of its services, named low band (<1 GHz), mid-band (between 1 GHz and 6 GHz), and high band (<6 GHz). In this work, we consider the scenarios in mid-band frequencies, based on the fact that these frequencies were thought to improve simultaneously capacity and coverage.

4.3. Disturbances

In addition to the input and output of users, base stations are affected by disturbances such as weather, which determines renewable energy generation. The renewable energy arrival, which determines clusters of SCBSs during specific periods, can be estimated with sensor measurements. In this way, active BSs change according to a pre-defined sequence while users are moving. In Figure 3, the average renewable energy arrival is represented, and Figure 4 depicts the users’ balance for the BS $\#9$ in both scenarios. Notice that outgoing users are also taken into account over the simulation. The MPC controller uses this information to reduce on-grid consumption.

4.4. Key Performance Indicators

Furthermore, the following Key Performance Indicators (KPIs) are proposed to compare the control schemes presented in Section 3:

• Grid consumption (kWatts-hour).
• Reduction of grid consumption (%) in comparison with the best-signal level mechanism.
• Average stored energy (Watts-hour).
• Users served (%).
• Average transmission rate (%) in comparison with the best-signal level mechanism.
• Performance cost, which is defined as the sum of the accumulated stage cost over the simulation:

  $$\begin{array}{c}{J}_{\mathrm{perf}}=\sum _{k=1}^N\sum _{\mathcal{C}\in \mathcal{B}/\mathrm{\Lambda }}{l}_{\mathcal{C}}\left(x_1^z\left(k\right)\right).\end{array}$$

  (13)
• Cooperation cost, which is defined as the sum of the cooperation cost over the simulation:

  $$\begin{array}{c}{J}_{\mathrm{coop}}=\sum _{k=1}^N\sum _{\mathcal{C}\in \mathcal{B}/\mathrm{\Lambda }}{g}_{\mathcal{C}}\left(L_{\mathrm{\Lambda }}\left(k\right)\right),\end{array}$$

  (14)

  with $g_{\mathcal{C}}\left(L_{\mathrm{\Lambda }}\left(k\right)\right)=c_{\mathrm{link}}{L}_{\mathrm{\Lambda }}\left(k\right)$, where we recall that $c_{\mathrm{link}}\in {\mathbb{R}}_{>0}$ is the cost per active link, and $L_{\mathrm{\Lambda }}\left(k\right)$ denotes the number of enabled links in topology$\mathrm{\Lambda }$ at time instant k.

4.5. Results and Discussion

In this section, the results and their interpretation are discussed for the proposed simulated systems: the academic, and the large scenario. In particular, the results are discussed from the perspectives of the energy management, user experience, and MPC performance.

4.5.1. Academic Scenario

On evaluating the on-grid consumption, it is shown how schemes based on MPC balance their traffic load to reduce the use of the MBS (Figure 5b) and increase the number of users served by the SCBSs (Figure 5a). Furthermore,

Table 2. Numerical results in the 9-BS network.

	Energy Management			User Experience		MPC Costs
Control Scheme	Grid Consumption (kW.h)	Grid Consumption Improvement (%)	Average Stored Energy (W.h)	Users Served (%)	Transmission Rate (Mbps)	${\mathit{J}}_{\mathbf{perf}}$	${\mathit{J}}_{\mathbf{coop}}$	${\mathit{J}}_{\mathbf{total}}={\mathit{J}}_{\mathrm{perf}}+{\mathit{J}}_{\mathrm{coop}}$
Best-signal-level	$938.1$	−	$65.5$	$95.2$	$4.2$	−	−	−
Energy-aware heuristic	$830.3$	$11.5$	$61.2$	$94.7$	$3.9$	−	−	−
Centralized MPC	130	$86.1$	$56.6$	$96.1$	$4.0$	$4.44·{10}^4$	$1.25·{10}^7$	$1.25·{10}^7$
Decentralized MPC	$550.7$	$41.3$	$58.5$	$96.5$	$3.8$	$5.34·{10}^6$	$4.00·{10}^6$	$9.34·{10}^6$
Coalitional MPC	$272.8$	$70.9$	$57.9$	$96.3$	$4.0$	$1.64·{10}^5$	$1.14·{10}^7$	$1.16·{10}^7$

displays that centralized and coalitional MPC have, respectively, an $86.1$% and $70.9$% of reduction in grid consumption with respect to the traditional scheme. Similarly, decentralized MPC has a $41.3$% reduction, being a suitable option to solve the energy consumption problem, but with less efficiency than the centralized and coalitional schemes. The obtained results show the efficiency of MPC-based methods to decrease grid consumption.

Regarding stored energy management, centralized MPC attains $56.6$ W.h as the average stored energy; coalitional MPC stores an average energy of $57.9$ W.h; and decentralized MPC achieves an average of $56.6$ W.h. Compared to the result obtained with the traditional scheme ($65.5$ W.h), an improvement close to 15% is obtained regarding the use of renewable energy. Generally, the MPC methods have similar behavior in terms of energy management. However, the reduction of the grid consumption and traffic load of SCBSs is better with the centralized and coalitional schemes. Nevertheless, the coalitional strategy is more suitable for non-centralized architectures.

The MPC-based schemes are compared in Figure 6, where the evolution of the four SCBSs is depicted. The first top row shows the number of users served by the SCBS. The second and third ones represent, respectively, the expected users flow over the simulation and the renewable energy arrival for the SCBSs. They are disturbances for controllers, which compute appropriate control inputs to reduce on-grid consumption and efficiently manage the stored energy. The fourth row depicts the energy stored in the battery. Thus, the response of the controller in periods without renewable energy arrival can be analyzed. For example, in the time interval 0–5 s, the users served by SCBS $i=7$ are zero because there is no renewable energy arrival nor energy stored in its battery. Notice the result consistency regarding the users served by the MBS between 0 s and 5 s, as presented in Figure 5b.

Centralized MPC achieves the best performance at the expense of using full cooperation between subsystems. Conversely, the coalitional MPC technique adapts the topology every $T_{\mathrm{up}}$ instants considering changes in user traffic and weather conditions. Note that centralized MPC has $72.92$% better performance cost than coalitional MPC. Even so, the latter method presents a cost reduction of $7.2$% regarding centralized MPC when cooperation costs are also considered.

The percentage of users served and the average transmission rate are the KPIs used to assess user experience. In MPC schemes, the former KPI is higher than in the best-signal-level strategy, as shown in Table 2. Regarding average transmission rates, MPC mechanisms have reductions close to 4% compared to the traditional scheme. In the energy-aware heuristic case, the percentage of users served is lower than the MPC schemes because it prioritizes the energy criterion over the signal level received. As stated previously, these results stem from the relationship between the signal level and the user rate, being an expected outcome. However, the percentage of users served in the MPC-based schemes remains fairly steady ($96\%$). All this, combined with the improvements previously presented, makes coalitional control an efficient strategy to deal with energy management in the next-generation cellular networks.

4.5.2. Large Scenario

Numerical results for the 37-BS network are displayed in

Table 3. Numerical results in the 37-BS network.

	Energy Management			User Experience		MPC Costs
Control Scheme	Grid Consumption (kW.h)	Grid Consumption Improvement (%)	Average Stored Energy (W.h)	Users Served (%)	Transmission Rate (Mbps)	${\mathit{J}}_{\mathbf{perf}}$	${\mathit{J}}_{\mathbf{coop}}$	${\mathit{J}}_{\mathbf{total}}={\mathit{J}}_{\mathrm{perf}}+{\mathit{J}}_{\mathrm{coop}}$
Best-signal-level	$252.8$	−	$63.5$	$95.3$	$3.3$	−	−	−
Energy-aware heuristic	$227.5$	$10.1$	$60.2$	$94.3$	$2.8$	−	−	−
Centralized MPC	$130.8$	$48.3$	$71.1$	$95.1$	$3.1$	$4.5\times {10}^4$	$1.37\times {10}^8$	$1.3705\times {10}^8$
Decentralized MPC	$4732.8$	$-1772$	$70.8$	$95.5$	$2.9$	$3.24\times {10}^8$	$1.8\times {10}^7$	$3.42\times {10}^8$
Coalitional MPC	$182.2$	$27.9$	$70.5$	$95.1$	$3.1$	$5.02\times {10}^5$	$1.36\times {10}^8$	$1.365\times {10}^8$

. As shown, the centralized and coalitional MPC schemes have, respectively, a $48.3$% and $27.9$% of grid consumption improvement. Moreover, both methods do the best management of stored energy with $71.1$ W.h in centralized MPC and $70.5$ W.h in coalitional MPC. Hence, these methods serve more users in the SCBSs with lower consumption of stored energy. Conversely, decentralized MPC cannot manage the user flows between SCBSs when the stored energy is spent. Thus, the number of users served by the MBS is increased, and higher on-grid energy is consumed than other methods.

Figure 7 presents the distribution of users between base stations for the MPC schemes. The centralized and coalitional MPC strategies have better user management over the simulation time, while decentralized MPC can only transfer users to the MBS when the SCBSs do not have stored energy in their batteries. Thus, decentralized MPC is not an appropriate scheme to deal with consumption problems in NGCNs.

Additionally, Figure 8 presents the behavior of several SCBSs with different renewable energy arrival profiles. First, it is observed how the SCBSs without renewable energy arrival use the energy stored in their batteries to guarantee the user service. However, in the centralized and coalitional MPC schemes, users are transferred to other BSs when there is no stored energy. When renewable energy arrives, the batteries maintain a reserve level to serve users and balance the consumed energy according to the arrival forecast. This behavior shows a desirable energy management scheme because the average energy stored in batteries is a 35% of its capacity, the percentage of load in SCBSs is 55%, and the average occupation of the MBS is $8.35$%, as depicted in Figure 7.

5. Conclusions

The broad spatial distribution of cellular networks and the exponential growth of 5G mobile devices promote distributed control methods and alternative sources such as solar, hydro, and wind energy. This article proposes a hierarchical coalitional MPC algorithm to dynamically manage the user-BS association in two simulated hybrid-energy HetNets of different scales. Its assessment regarding the best-signal-level and energy-aware heuristic methods has shown an efficient way to hold the quality of service while reducing on-grid consumption.

Furthermore, the coalitional strategy has been compared with centralized and decentralized MPC. As is known, centralized MPC always has the best performance at the expense of full communication between base stations. Since the viewpoint is to find a trade-off between performance and communication costs, coalitional MPC becomes a suitable method to adapt the network topology dynamically. It mainly evaluates whether a communication link between two base stations contributes to enhancing the overall performance, and otherwise, the link is disabled to save battery storage. Finally, scalability is also achieved for large-scale cellular networks.

Open research lines from this work are a fully distributed implementation of coalitional MPC and its application to the network resource management in 5G systems. The network resource management approach can be studied not only from the energy efficiency perspective, but also from the quality of service approach to accomplish the 5G characteristics. Additionally, we will explore new linear models and stochastic formulations for the problem dynamics.

6. Materials and Methods

Simulated examples were performed using MATLAB^® R2017a on Windows 10 with a PC Intel^® Core™ i7-8700 CPU at $3.20$ GHz and 16 GB RAM.