Cross-Entropy-Based Energy-Efficient Radio Resource Management in HetNets with Coordinated Multiple Points

Energy efficiency and spectrum efficiency are the most important issues for future mobile systems. Heterogeneous networks (HetNets) with coordinated multiple points (CoMP) are wildly approved as a promising solution to meet increasing demands of mobile data traffic and to reduce energy consumptions. However, hyper-dense deployments and complex coordination mechanisms introduce several challenges in radio resource management (RRM) of mobile communication systems. To address this issue, we present an RRM approach for CoMP-based HetNets, which aims to maximize weighted energy efficiency while guaranteeing the data rate of each transmission. The proposed RRM approach is based on a cross-entropy (CE) optimization method that is an effective and low-complexity heuristic algorithm. Furthermore, we also give the implementations of the proposed RRM approach in centralized and decentralized mode, respectively. At last, extensive simulations are conducted to validate the effectiveness of the proposed schemes.


Introduction
Heterogeneous networks (HetNets) [1,2] are proposed in long-term evolution-advanced (LTE-A) systems by the 3rd Generation Partnership Project (3GPP) to not only meet the rapidly increasing demands of mobile data traffic, but also to reduce the huge energy consumptions caused by data transmissions. A HetNet is a hierarchical network where low-power access points (APs, also named small cell sites [3,4]) are included in a macro-cell, in order to provide highly qualified services. Typical small cell sites include pico, femto, relay and so forth. Femto [5] is mainly for indoor transmissions, while pico [6] is used outdoors in a crowded place, such as a university. Relay [7] is usually built in a remote area far from the macro-cell site (named the Evolved Node B, or eNodeB, in LTE-A HetNet) to extend the coverage of the macro-cell. Figure 1 presents a simple example of a HetNet. Although HetNets are widely considered as the main trend in the development of mobile communication systems [3], several challenges should be dealt with before implementation in the real world. One of the major challenges is severe intercell interference aggravated by the dense deployment, which crucially affects the performance of HetNets. As a promising measurement to deal with intercell interference, coordinated multiple point (CoMP) [8] techniques are considered to be helpful for HetNets. The basic idea of CoMP techniques is to coordinate coordinated neighboring cells, so that intercell interference can be reduced to a large extent. According to the different ways to cooperate, CoMP techniques are classified into two categories, coordinated scheduling/coordinated beamforming (CS/CB) and joint processing (JP) [9]. CS/CB CoMP makes neighboring cells jointly pre-code according to global channel state information (CSI) to avoid potential interference, while JP CoMP allows neighboring cells to jointly process signals intended for a specific user in the overlapping area. Joint transmission (JT) is a typical JP CoMP technique that requests of cooperating cells to transmit the same data packets to a user independently and simultaneously. In this paper, we take JT CoMP as an example to discuss the performance of CoMP-based HetNet, and the study can also be extended to other CoMP techniques.
To fulfill the requirements of the future mobile networks, CoMP-based HetNets need to provide qualified services as much as possible while maintaining acceptable energy consumption. According to report [10], energy consumption caused by information and communications technology (ICT) has contributed up to 5% of the world-wild power supply at present. To avoid the further increase of energy consumption, mobile networks are required to significantly improve the utility efficiency of power resources. Radio resource management (RRM), which adaptively allocates radio resources, such as frequency and power, according to CSIs, is beneficial for CoMP-based HetNets to enhance the energy efficiency.
In this paper, we focus on RRM in a CoMP-based HetNet for the purpose of improving the energy efficiency while guaranteeing high data rates, as well as user fairness. An optimization problem aiming at maximizing weighted energy efficiency is formulated, where the weights are employed for maintaining the fairness of users's data rates. Several crucial constraints in practice are taken into consideration in the formulated problem. Besides the limitation on total power at each transmitter, backhaul links, which connect small cells to the eNodeB for exchanging data and control information, are considered to have restricted capacity. Additionally, the lowest data rate of each transmission is defined, in order to guarantee the quality of transmissions and avoid wasting of energy. Since the formulated problem is unsolvable mixed integer programming (MIP), we separate the whole problem into a scheduling subproblem under the assumption of equal power allocation and a power allocation subproblem with known scheduling results. We first proposed a centralized scheduling algorithm based on cross-entropy (CE) and a corresponding power allocation algorithm. Since centralized algorithms involve numerous calculations, the time delay could be intolerable in a large-scale network. An alternative method to decrease the time delay is to conduct resource allocation in a decentralized way, where calculations are distributed to small cells. For this reason, we also propose modified algorithms that can be used in a decentralized mode.
The rest of this paper is organized as follows. In Section 2, we present the considered system model. Then, the mathematical formulation of the discussed problem is described in Section 3. Section 4 proposes a centralized strategy of resource allocation, which includes a CE-based scheduling algorithm and a power allocation algorithm. Section 5 modifies the proposed algorithms, so that they can be utilized in a decentralized system. Simulation results and relevant analysis are shown in Section 6. At last, we conclude our work in Section 7.

System Model
For the sake of simplicity, each independent transmitter, including the macro-cell site and micro-cell sites, is hereby designated as a transmit point (TP) in this paper. We consider a downlink system in a CoMP-based HetNet with M TPs and K user elements (UEs), as shown in Figure 2. N T and N R represent the number of antennas on each TP and each UE, respectively. A control unit (CU) is assumed to be located at the center of the network, which is responsible for managing data information, as well as collecting all CSI in this network. TPs are connected to the CU by backhaul links, whereby control information and data packets are delivered to TPs from the CU. Since JT CoMP is employed in our work, there are a large number of data packets that need to be transmitted via backhaul links. CU )) ) ) ) ) ) )))))) ) ) ) ) ) ) )) ) ) ) ) )) ) ) ) ) ))) )) )) ) ) ) ) )) ) ) ) ) )) ) ) ) ) ) ) ) ) B a c k h a u l The unit of the radio resource in both time and frequency dimensions is referred to as the resource block (RB). As defined in the LTE standard, an RB consists of 12 consecutive subcarriers for a duration of a transmit time interval (TTI) [12]. In this paper, we assume all TPs share the same spectrum bandwidth, which is divided into N RB RBs in total. Major notations used in this paper are listed in Table 1. Several other important assumptions are considered in our work: channel fading is considered to be quasi-static, so that channel coefficients remain constant per TTI; perfect CSI acknowledgment is assumed at both receivers and transmitters; TPs are synchronized in terms of time, frequency and phase, which is reasonable in the considered system thanks to backhaul connections.

CoMP Set Selection
Ideally, a UE can achieve the optimal data rate if all TPs cooperatively transmit to it. However, the corresponding power consumption and computational complexity are unaffordable. An alternative is to select a CoMP set for the UE according to channel conditions. TPs in the CoMP set can provide the UE a favorable data rate at a much lower cost.
A UE-specific selection of the CoMP set includes three steps: 1. TPs broadcast reference signals (RSs) periodically. 2. A UE hears the channels and measures them according to the strength of the received RSs. Based on the measurements and a given selection rule, the UE can decide its own CoMP set. 3. The UE acknowledges its decision to the CU.
Denote M k as the CoMP set of UE k and M as the set including all TPs in the network. UE k decides its M k following the rule below: where RS indicates the strength of the reference signal and ∆ is a threshold in dB. UE k distinguishes the strongest RS in the first place and adds the corresponding TP to the CoMP set M k . Other TPs will be added to M k only if the strengths of their RSs are no less than (RS(the strongest) − ∆) dB. As suggested in LTE releases, a rational ∆ is in the range of 5 − 6 dB [11].
In the case where M k includes only 'the strongest TP', UE k is referred to as a non-CoMP UE in this paper, since no cooperation occurs during downlink transmissions towards it. On the contrary, UE k is referred to as a CoMP UE if its CoMP set M k includes more than one TPs. A CoMP UE is possibly is located at an overlapping area of neighboring cells where intercell interference seriously damages transmissions. To combat interference, TPs in M k are asked to conduct JT CoMP transmissions to UE k for strengthening transmit signals and reducing inter-cell interference.

Dynamic JT CoMP Transmission
JT CoMP allows TPs in UE's CoMP set to simultaneously transmit the desired data signal to it. Owing to the spatial separation of the transmit antennas, multiple versions of the desired signal will be received by the UE, which generates extra spatial diversity gain and strengthens the signal. According to information theory [18], the obtained data rate of a JT CoMP transmission to UE k on RB n is given as: where H n m,k is an N R × N T channel matrix between TP m and UE k on the n-th RB. M \ M k is the complementary of M k in M, which includes all of the interfering TPs. w n m is the precoding vector with dimensions of N T × 1, which maps data stream s n m onto the transmit antennas of TP m, and (w n m ) H w n m = 1, E s n m = 1, ∀m, n. p n m is the power used by TP m for transmitting on RB n, and n n k ∼ CN (0, σ 2 I N R ) is the corresponding complex Gaussian noise vector. b represents the bandwidth of an RB, which is standardized to be 180 kHz in LTE-A systems [12].
Equation (2) implies a static coordinated strategy where TPs in M k are all required to serve UE k all of the time. However, static strategies are not always optimal due to the time variation of wireless channels. To further improve the network performance, we use a dynamic JT CoMP strategy where a subset of each M k rather than M k is adaptively determined to perform JT CoMP transmission. Define a scheduling index β n m,k ∈ {0, 1} to indicate scheduling results, where β n m,k = 1 means that TP m is chosen to transmit to UE k on the n-th RB. Then, the data rate of a dynamic JT transmission can be given as: where ∑ M m=1 β n m,k ≥ 1.

Problem Formulation
In this work, we consider a practical RRM problem in terms of both spectrum and power in a CoMP-based HetNet modeled in the last section. To improve the synthesis performance, the objective involves data rates, power consumption and fairness among UEs at the same time.
The optimal data rate of the network can be achieved if resources are allocated to UEs with better channel conditions, regardless of those in "poor" condition. A side-effect of this scheme is unfavorable fairness of UEs' data rates. In order to enhance the fairness, we introduce the concept of proportional fairness into the objective of the RRM problem. As in [13], we weighted UE's data rate by its average data rate determined by: where 0 < α < 1 is the forgetting factor. The introduced weights bring UEs with worse channel conditions more of a possibility to occupy resources and, therefore, increase the UEs' data rates. The fairness of UEs' data rates in the network will be improved in this way. To quantify the degree of this fairness, [14] introduces a fairness factor, defined as: Additionally, for the purpose of conserving energy, energy efficiency should be thoughtfully considered. The energy efficiency is defined by the ratio of the obtained data rate to the total power consumed correspondingly. Combining with proportional fairness principle, the objective of the RRM problem is formulated as, Equation (6) can be considered to maximize the weighted energy efficiency of the network. In a practical network, system performance is restricted by several factors. In addition to limited transmit power at each TP, the finite capacity of backhaul links defines the upper limit of throughput achieved by a TP during a TTI. Furthermore, to guarantee the quality of transmissions, we impose a threshold to the data rate of each transmission. In summary, the RRM problem of the considered system can be formulated as: In Equation (7), C1 shows the power constraint at each transmission where S is the largest transmit power allowed by the system; C2 ensures each index β n m,k to be a bit number, so that a TP can serve no more than one UE on each RB; C3 demonstrates the constrained throughput of a TP caused by the limited capacity of backhaul connections to the CU, where C m represents the capacity of the backhaul link connecting TP m and the CU; and C4 guarantees the data rate of each ongoing transmission, where R thres is the given threshold of the data rate.

Centralized Algorithm
Since the problem in Equation (7) is NP-hard, it is unpractical to achieve the optimal solution. An alternative method is to consider the problem as a combination of a scheduling problem under the consumption of equal power allocation and a power allocation problem with a given scheduling result. In this way, an approximated solution of the problem can be obtained in polynomial time.
In the rest of this section, we propose a heuristic algorithm based on CE for RB scheduling with equal transmit power at the first place. Then, a Karush-Kuhn-Tucker-method to solve the power allocation problem is presented. The algorithm involving both RB scheduling and power allocation proposed in this section is centralized, which means that the resource allocation is operated at the CU with global CSI. The centralized algorithm is capable of achieving a favorable system performance, but it requires significant computational effort of the CU. In the next section, we also propose a decentralized algorithm of resource allocation with lower complexity and delay of computation, which leads to decreased performance unavoidably.

CE-Based Scheduling Algorithm
The objective of the considered RB scheduling problem becomes: where S is the fixed transmit power. The objective is constrained by C2, C3 and C4 in Equation (7). We first propose a CE-based algorithm to solve the RB scheduling problem described above. The CE method is a typical heuristic algorithm to estimate the probabilities of rare events in complex stochastic networks [15] and to deal with linear programming. The basic idea of the CE method is to generate sufficient samples under a given strategy and then update the generating strategy according to samples. After iteratively repeating this procedure, generated samples will converge to the optimal solution. The proposed CE-based scheduling algorithm follows three major stages, including initialization, iteration and a complementary stage to close unfavorable transmissions, as shown in Figure 3.  . For the sake of accelerating convergence, we design the initial probability distribution according to estimated data rates. We first defined P{x m (n) = 0} = Pr 0 where Pr 0 is a given value. This probability gives an opportunity to TP m for scheduling no transmission on RB n. For each u ∈ U m , the data rate R n u is estimated based on the fixed JT CoMP transmission, as given by (2), at the initialization stage. As suggested by C4 in (7), the transmission is considered to be unqualified if R n u < R thres . To save energy, the proposed algorithm sets the corresponding probability to be zero, i.e., q m,n u = 0. For the rest of the optional UEs, the probability is defined as q m,n u = (R n u R m tot ) × (1 − Pr 0 ), where R tot indicates the sum data rate of all possible qualified transmissions, defined as: Summarily, the initial distribution of x m (n) is:

Iteration
Each iteration of the proposed algorithm includes three steps. First, the algorithm generates adequate samples according to a given strategy. Then, it is necessary to exclude those samples that do not satisfy the constraints and to select "good" samples for the next stage. At last, the probability distribution needs to be updated according to the selected samples, so that "better" samples will be generated in the next iteration. After sufficient iterations, the algorithm gradually approaches the optimal solution.
Let N SAM denote the number of samples generated in each iteration for each TP and X m 1 , · · ·, X m i , · · ·, X m N SAM denote the corresponding samples, where X m i = [x m i(1) , · · ·, x m i(n) , · · ·, x m i(N RB ) ]. Each sample can map into a scheduling index set {β n m,k , ∀k, n}, and leads to a weighted energy efficiency of TP m given as: where ∑ N RB n=1,x m i (n) =0 S represents the total power consumption at TP m according to sample X m i . The sum data rate of TP m can be estimated as R m X m . A qualified sample should satisfy two requirements. First, the sum data rate R m cannot exceed the backhaul capacity of TP m. Therefore, those samples lead to overlarge sum data rates, i.e., R m X m i > C m , will be removed. Second, the value of f m X m i should be high enough. Samples whose weighted energy efficiency f m X m i < f = N x m j(n) = u /N IM (12) where N x m j(n) = u represents the number of times that UE u appears in N IM samples on the n-th RB. The proposed iteration algorithm is summarized as Algorithm 1.

Algorithm 1 Iteration in the CE-based scheduling algorithm.
1: f max = f max_pre = 0; counter = 1; 2: while counter ≤ N SAM do 3: Generate samples X m according to the distribution q m,n .

4:
Calculate utility function of X m i , i.e., f X m i , according to (11).

5:
Calculate the sum data rate of X m , i.e., R m (X m ) = ∑ k∈U m R n k . 6: if f (X m ) < f counter = counter + 1 14: end while 15: for i = 1 to N SAM do 16: Calculate f X m i according to (11). 17 22: end if 23: Update q m,n according to (12) 24: Map X m out into β n m,k , and recalculate R n k according to (3)

Removing Unqualified Transmissions
The algorithm proposed above cannot ensure that the data rate of each transmission is as high as the given threshold R thres . Since we put our purpose on improving energy efficiency, it is reasonable to close those transmissions that are estimated to be unqualified.
Algorithm 2 summaries the entire RB scheduling algorithm proposed above.

Power Allocation Algorithm
With the obtained RB scheduling result, the power allocation problem can be given as: The objective function in (13) is a well-known non-convex function for which the optimal solution does not exist. In this work, we use an analytical method on the base of the KKT-condition to approach a local optimal solution of power allocation. A solution of Equation (13) can be achieved by solving its dual function in terms of backhaul and data rate constraints given as: where {λ m , ∀m} and {µ n,k , ∀n, k} are non-negative Lagrangian multipliers. However, the dual function given above is still hard to solve, since it involves too many variables (MN RB variables and Lagrangian multipliers). To further simplify the problem, we intend to decompose Equation (14) into independent subproblems with fewer variables. Let β m = ∑ N RB n=1 ∑ K k=1 β n m,k . β m = 0 indicate the situation where no transmission is scheduled for TP m. This is unreasonable in the high-loading network we considered. Thus, we suppose that β m = 0; then C m can be rewritten as: After substituting (15) into (14), the problem can be decomposed into N RB independent subproblems, where each subproblem is given as: Each subproblem in Equation (16) involves M variables with Lagrangian multipliers only and can be solved independently on each RB. The computational complexity is significantly cut in this way. In the rest of this subsection, we propose an iterative method to address each subproblem.
Let f n (p n m , λ m , µ n,k ) denote the objective function of Equation (16). Take the first order derivative in terms of p n m , and make it equal to zero; then, a possible value of power allocation, denoted by p n m , can be obtained as follows, where p n tot = ∑ M m=1 p n m and k * is the UE scheduled on RB n of TP m, i.e., β n m,k * = 1. The derivative ∂R n k ∂p n m and A n m are given by: and: respectively. The obtainedp n m above may not be in the range of [0, S]. Therefore, the real power allocation needs to be adjusted following the rule given by: where t indicates the times of the iteration.
Lagrangian multipliers can be updated in each iteration by the sub-gradient method [16] as follows, µ are the step sizes used in the current iteration for updating λ m and µ n,k .

Decentralized Algorithm
The centralized strategy of resource allocation proposed above is processed on the CU with the global CSI at the beginning of each TTI. It is possible that the time delay caused by processing is too long to guarantee the effectiveness of a large-scale system involving numerous TPs and UEs. An alternative method for shortening the time delay is a decentralized strategy that distributes calculations to each TP instead of the CU. Under a decentralized strategy, global CSIs are shared between TPs at the first place. Then, the resource allocation is processed at each TP independently and simultaneously, according to the known CSIs and a given strategy. In this way, the time delay of processing can be significantly decreased, even in a large-scale network. However, due to the lack of knowledge about the scheduling results of other TPs, the accuracy of the decentralized one is unavoidably worse than the centralized one.
In this section, we propose a decentralized strategy with the similar CE-based scheduling and KKT-based power allocation to the centralized proposed above. Simulation results presented in Section 6 will prove that the decline of system performance under the proposed decentralized strategy is acceptable.

Decentralized RB Scheduling Algorithm
A decentralized RB scheduling algorithm based on the CE method is proposed in this subsection. The same as the centralized one proposed in Subsection 4.1, the decentralized scheduling algorithm initializes the probability distribution q m at the first place. Then, the iteration procedure is processed to obtain RB scheduling results. It should be noticed that the decentralized strategy cannot accurately estimate data rates of ongoing transmissions due to the lack of information about the scheduling results of other TPs. Therefore, the decentralized scheduling algorithm deletes the procedure of removing unqualified transmissions (as described in Section 4.1.3). In the simulation, we consider unqualified transmissions as failures, which waste energy and contribute nothing to the data rates of the system. Algorithm 3 summarizes the decentralized RB scheduling based on the CE method.

Power Allocation Algorithm
In this subsection, we modify the power allocation algorithm proposed in Subsection 4.2 to be decentralized, so that it can be processed at each TP independently and simultaneously. The individual power allocation problem of each TP is given by: As in Subsection 4.2, we constitute and solve the dual function of Equation (22), instead of solving it directly. The dual function in terms of constraints C3 and C4 is given as: Substituting Equation (15) into Equation (23), we can decompose the power allocation problem into N RB independent subproblems. Let f m,n denote the objective of the subproblem on RB n of TP m, which is given as: β n m,k = 0 means that TP m does not schedule any transmissions on RB n. In this case, power allocation is not required, i.e., p n m = 0. In the case where K ∑ k=1 β n m,k = 1, transmit power p n m can be obtained by an iterative method proposed in the rest of this subsection.
Let k * denote the UE of TP m scheduled on RB n (i.e., ∑ K k=1 β n m,k = β n m,k * = 1). Taking the first order derivative of f m,n with respect to p n m , we obtain: where: Let ∂ f n ∂p n m = 0. An expression of p n m in the t-th iteration can be obtained by solving the equation, which is given as: where multipliers λ m and µ n,k * should be updated according to Equation (21). A suboptimal p n m can be approached after sufficient iterations.

Simulation Results
We consider a HetNet downlink system with 37 TPs, where only 19 TPs of these conduct actual communications to UEs, and the others wrap them around to produce virtual interference. The radius of each small cell is 250 m, since a dense deployment is considered. The system includes 100 RBs, each of which is under a bandwidth of 180 kHz. Therefore, the overall bandwidth of the system is 18 MHz. Additionally, 2 × 2 MIMO links are created using the space channel model (SCM) [17]. Each simulation lasts 20 TTIs, where a TTI is 1 ms. Important parameters used in the simulation are listed in Table 2. The simulation is carried out to prove the proposed algorithms effectiveness. A greedy algorithm, named max capacity, is also simulated as a benchmark. The max capacity algorithm tends to allocate resources to UEs with good channel conditions, in order to reach the optimal throughput of the network. In this way, UEs with worse channel conditions may no chance to communicate. Therefore, the fairness of the max capacity algorithm is unfavorable. Figures 4 and 5 compare the performances of the max capacity algorithm to that of the proposed one. Figure 4 demonstrates the average throughput per TP and fairness factor (as defined by Equation (5)) of the system under different resource allocation algorithms, when the transmit power of each TP is 20 Watts. It is obvious that max capacity algorithm achieves an outstanding throughput and a much worse fairness factor than the proposed one. The future mobile communication system targets to provide not only high throughput of the network, but also quality service to every UE. Therefore, the max capacity is no longer appropriate. The proposed algorithms have much better fairness factors. More importantly, as shown in Figure 5, the proposed centralized algorithm can also achieve an energy efficiency as good as that of max capacity.     Figures 4 and 5 also compare the performance of the centralized algorithm proposed in Section 4 and that of the decentralized algorithm in Section 5. The results show that both energy efficiency and throughput are decreased when the decentralized algorithm is used. Figure 6 shows the system performances of the proposed algorithms when the transmit power of each TP is 40 Watts. Comparing the results to Figures 4 and 5, it can be seen that the energy efficiency of the centralized algorithm significantly decreases when the transmit power of each TP is up to 40 Watts, while the throughput does not increase. This proves that high-level transmit power is not appropriate in a dense network. Additionally, the results demonstrate that the decentralized algorithm is more robust, since both energy efficiency and throughput are changed a little when different transmit powers are used. Simulations are also conducted under infinite, 100-Mbps and 80-Mbps backhaul limits, respectively, with a fixed R thres of 360 kbps, to clearly illustrate the effect on system performance caused by backhaul constraints. Restricted backhaul capacity leads to a low throughput per TP, as shown in Figure 7a, since fewer transmissions are scheduled in this case. However, the tendency is different in terms of energy efficiency. Figure 7b shows that the energy efficiencies of the proposed algorithms under different backhaul limits are almost the same. This is explained by the fact that the power consumed by transmissions is also reduced when backhaul capacity is restricted. At last, we conduct simulations when R thres is 180 kbps, 360 kbps and 540 kbps, respectively, with the fixed backhaul capacity of 100 Mbps. This shows that the throughput of our proposals grows as the increase in R thres , as shown in Figure 8a. This is because more resources are assigned to the UEs with better channel conditions, which are possible to achieve for quality transmissions. Figure 8b illustrates that the value of R thres hardly affects the energy efficiency of the system. Since those transmissions estimated to be inferior to the given R thres are closed, no (or little) power is wasted. Therefore, that energy efficiency of the system can be maintained at a high level.

Conclusions
In this paper, we have studied a constrained RRM problem aiming at improving energy efficiency in a CoMP-based HetNet. To solve the problem, we first propose a CE-based RB scheduling algorithm under the assumption of equal power allocation. Then, a KKT-based algorithm for power allocation is presented. The proposed algorithms are considered to be used in a centralized way at the first place. Since the centralized strategy for RRM takes a long time delay in large-scale networks, we modified the proposed one in order to adapt to a decentralized system in order to shorten the time delay for processing. Simulation results compare performances of both the centralized and the decentralized and discuss the influence on system performance caused by the considered constraints.