Improving Power and Resource Management in Heterogeneous Downlink OFDMA Networks

: In the past decade, low power consumption schemes have undergone degraded communication performance, where they fail to maintain the trade-o ﬀ between the resource and power consumption. In this paper, management of resource and power consumption on small cell orthogonal frequency-division multiple access (OFDMA) networks is enacted using the sleep mode selection method. The sleep mode selection method uses both power and resource management, where the former is responsible for a heterogeneous network, and the latter is managed using a deactivation algorithm. Further, to improve the communication performance during sleep mode selection, a semi-Markov sleep mode selection decision-making process is developed. Spectrum reuse maximization is achieved using a small cell deactivation strategy that potentially identiﬁes and eliminates the sleep mode cells. The performance of this hybrid technique is evaluated and compared against benchmark techniques. The results demonstrate that the proposed hybrid performance model shows e ﬀ ective power and resource management with reduced computational cost compared with benchmark techniques.


Introduction
Significant voice and data requirements have placed pressure on wireless radio links and the infrastructure of wireless networks globally. Hence, serving an increasing number of users has become a major challenge for network operators even if the bandwidth is wider. Several solutions for wireless communications have been previously proposed; however, they are ineffective in terms of capital and operational costs. Other techniques, such as cell densification on heterogeneous wireless networks, reduced the cost of small cell deployment with enhanced coverage and capacity [1]. However, the distribution of unplanned small cells is associated with serious threats that lead to coand cross-tier interference [2]. Densification of small cells leads to co-tier interference and generates multiple overlapped cell boundaries [3]. Macrocells cause extreme cross-tier interference and leads to performance degradation of low power cells [4]. Such interference issues are avoided through radio resource management (RRM), which coordinated intercells through the allocation of users orthogonally in the frequency and time domains, where interference is avoided [5]. utilization of resources. The ICI is controlled through the incorporation of resource partitioning (RP) with small cell/users' orthogonalization. The reuse is reduced with orthogonalization using a selection algorithm that identifies the base station (BS) with the highest reduction in reuse. The information is fed into an RP scheme to determine whether the deactivation of a small cell can improve resource utilization. The hybrid method improves the throughput and power consumption with reduced computation cost. Also, the hybrid facilitates improved deactivation decision behavior with limited transmission between the core and BSs. A suitable mathematical model analyses the performance of the hybrid model to reveal superior performance and a closer match between the simulated and analytical results.
The main contributions of the hybrid model include: • Design of a hybrid model that identifies interferences during which the switching off or sleeping mode of some small cells can enhance the capacity of the heterogeneous network. The remainder of the paper is structured as follows. Section 2 provides a brief description of the network model and its assumptions. Sections 3 and 4 thoroughly discuss the hybrid model. Results and discussions are presented in Section 5. Final remarks are presented in Section 6.

Interference Map (IM) Approach
An IM identifies the conflicts [25] in a heterogeneous system. The measurement report (MR) from UE is used to build the IM, and the MR is triggered by a radio resource control (RRC) message. For the determination of physical cell identity (PCI), UE is used to scan the nearby BS and the power reference signal that is received (reported to serving BS). The local IM, ζ x , of serving BS x is expressed as: where z is referred to as the user served by x and its nearby BS y. β is referred to as the signal to interference (SIR) threshold. F x,z is referred to as the power received from BS x to UE z, F j,k is referred to as the interference power received from BS y to UE z.
The threshold value is selected based on the optimization objective. Typically, a higher threshold maximizes the performance, and a lower threshold maximizes the throughput. If the local IM is updated by BS, then local IM ζ x is sent to Femtocell Management System (FMS), iff ζ' x , which is different from local IM ζ x [25].
The universal IM of the heterogonous network with ζ∈ Z×B is represented as: ζ(z, x) = w, x serves BS k ζ(z, x) = 0 y doesn t interfere z ζ(z, x) = 1 y interferes z where w is a positive integer number Information 2020, 11, x FOR PEER REVIEW 3 of 17 allows increased utilization of resources. The ICI is controlled through the incorporation of resource partitioning (RP) with small cell/users' orthogonalization. The reuse is reduced with orthogonalization using a selection algorithm that identifies the base station (BS) with the highest reduction in reuse. The information is fed into an RP scheme to determine whether the deactivation of a small cell can improve resource utilization. The hybrid method improves the throughput and power consumption with reduced computation cost. Also, the hybrid facilitates improved deactivation decision behavior with limited transmission between the core and BSs. A suitable mathematical model analyses the performance of the hybrid model to reveal superior performance and a closer match between the simulated and analytical results. The main contributions of the hybrid model include:


Design of a hybrid model that identifies interferences during which the switching off or sleeping mode of some small cells can enhance the capacity of the heterogeneous network.  Formulation of an SLS mode-based RRS for providing an enhanced gain of reused resources.  Presentation of a model that captures the interference received and the resultant performance of upper bounds accurately verifies the estimated gains.
The remainder of the paper is structured as follows. Section 2 provides a brief description of the network model and its assumptions. Sections 3 and 4 thoroughly discuss the hybrid model. Results and discussions are presented in Section 5. Final remarks are presented in Section 6.

Interference Map (IM) Approach
An IM identifies the conflicts [25] in a heterogeneous system. The measurement report (MR) from UE is used to build the IM, and the MR is triggered by a radio resource control (RRC) message. For the determination of physical cell identity (PCI), UE is used to scan the nearby BS and the power reference signal that is received (reported to serving BS). The local IM, x, of serving BS x is expressed as: where z is referred to as the user served by x and its nearby BS y.  is referred to as the signal to interference (SIR) threshold. Fx,z is referred to as the power received from BS x to UE z, Fj,k is referred to as the interference power received from BS y to UE z.
The threshold value is selected based on the optimization objective. Typically, a higher threshold maximizes the performance, and a lower threshold maximizes the throughput. If the local IM is updated by BS, then local IM x is sent to Femtocell Management System (FMS), iff 'x, which is different from local IM x [25].
The universal IM of the heterogonous network with  ZB is represented as: where w is a positive integer number 1 and 0.

The Proposed Method
The main objective is to identify whether deactivating some definite small cells can improve the wider perspective of the network. Here, the centralized controller is selected as FMS due to unplanned deployment of femtocells. Determining the deactivation of such potential cells can result in improvements to the capacity and wide reuse of frequency. Attention is given to the re-association of network users to relocate to the nearby cells during the deactivation process. This resolves the co-tier interferences through orthogonal RRB. The scenario of a cell being switched off is shown in Figure 1. The main objective is to identify whether deactivating some definite small cells can improve the wider perspective of the network. Here, the centralized controller is selected as FMS due to unplanned deployment of femtocells. Determining the deactivation of such potential cells can result in improvements to the capacity and wide reuse of frequency. Attention is given to the re-association of network users to relocate to the nearby cells during the deactivation process. This resolves the cotier interferences through orthogonal RRB. The scenario of a cell being switched off is shown in Figure  1.

Sleep Mode for Maximizing Capacity
The stochastic optimization problem is formulated using the semi-Markov SLS (SMD-SLS) decision process that considers MU density, spatial distribution, and scheduling in a heterogeneous network. The SMD-SLS model achieves a favorable trade-off between performance and power consumption in multi-cell heterogeneous networks. The stochastic optimization problem is solved through an optimal set of power scheduling policies. Finally, an iterative algorithm solves the SMD-SLS-based stochastic optimization and yields an optimal decision for minimizing power consumption in BS.
The main objective involves analyzing the maximum capacity in a network after the deactivation of the BS set. This leads to the formulation of the capacity maximization problem using the semi-Markov SLS (SMD-SLS) decision process. Consider a BS to be in SL or RE and that the BSs state is indicated by a variable φx ∈ {0,1}; where, φx = 0 denotes the SL state and vice versa. The attainable capacity, ck,n, using a semi-Markov SLS (SMD-SLS) decision at a physical resource block is expressed as: cz,n= W⋅log(Yz,n + 1) where W denotes the physical resource block bandwidth n, which is similar to all physical resource blocks. The capacity maximization problem is thus formulated with constraint functions and a stochastic reward that minimizes the consumption of power with respective performance constraints.
The SMD-SLS model is applied using a state-space definition and activation functions. The SMD-SLS state, Nx(t), at time t is characterized by the density of each cell. The location of the user state is Lx(t) and each cell's transmit power state is Ex(t). Then, the SMD-SLS state of cell x is sx (t) = {Nx(t), Lx(t), Ex(t)} and the probability matrix of state transition is [δsxs′x(t)] = [ψx(t), Θx(t), Φx(t)] ≅ φ(t)ϕ(t)ψ(t). The SMD-SLS state transition probability changes from the state sx to state s′x.
The state-space of the SMD-SLS model is denoted as S ≅ {D, L, E}. The activation state is A, which is denoted by A = {a1(t), a2(t), a3(t), a4(t)}, where a1(t) is the action of turning off the FMS without changing the nearby femtocells, a2(t) is the action of turning off the FMS and increasing the

Sleep Mode for Maximizing Capacity
The stochastic optimization problem is formulated using the semi-Markov SLS (SMD-SLS) decision process that considers MU density, spatial distribution, and scheduling in a heterogeneous network. The SMD-SLS model achieves a favorable trade-off between performance and power consumption in multi-cell heterogeneous networks. The stochastic optimization problem is solved through an optimal set of power scheduling policies. Finally, an iterative algorithm solves the SMD-SLS-based stochastic optimization and yields an optimal decision for minimizing power consumption in BS.
The main objective involves analyzing the maximum capacity in a network after the deactivation of the BS set. This leads to the formulation of the capacity maximization problem using the semi-Markov SLS (SMD-SLS) decision process. Consider a BS to be in SL or RE and that the BSs state is indicated by a variable ϕ x ∈ {0,1}; where, ϕ x = 0 denotes the SL state and vice versa. The attainable capacity, c k,n , using a semi-Markov SLS (SMD-SLS) decision at a physical resource block is expressed as: where W denotes the physical resource block bandwidth n, which is similar to all physical resource blocks. The capacity maximization problem is thus formulated with constraint functions and a stochastic reward that minimizes the consumption of power with respective performance constraints.
The SMD-SLS model is applied using a state-space definition and activation functions. The SMD-SLS state, N x (t), at time t is characterized by the density of each cell. The location of the user state is L x (t) and each cell's transmit power state is E x (t). Then, the SMD-SLS state of cell x is s x (t) = {N x (t), L x (t), E x (t)} and the probability matrix of state transition is [δ sxs The SMD-SLS state transition probability changes from the state s x to state s x .
The state-space of the SMD-SLS model is denoted as S {D, L, E}. The activation state is A, which is denoted by A = {a 1 (t), a 2 (t), a 3 (t), a 4 (t)}, where a 1 (t) is the action of turning off the FMS without changing the nearby femtocells, a 2 (t) is the action of turning off the FMS and increasing the transmission power of two nearby femtocells by a multilevel power control in FSS, a 3 (t)is the action of unchanged FMS and increasing the transmission power of two nearby femtocells by a multilevel power control in FSS, and, finally, a 4 (t) is the action of decreasing the power of FSS due to reduced MU without changing nearby cells. The current state of SMD-SLS, s(t)∈S, and action a(t)∈A represent the probability matrix of state transition for the next state s(t + 1), which is P r {a(t), s(t + 1)|s(t)}. This Markovian function probability represents the state transition of the previous state and is independent of the next stage when specified in the present stage.
In SMD-SLS, during each epoch, power control scheduling decides the power allocation to the femtocells, as per the power and traffic profiles. Power control scheduling uses the activation function a(t) at each epoch. Since the objective is to minimize power consumption, a reward function is used as the power consumption function for the femtocells. Thus, the reward function is defined as: where P(S,A) is the total power consumed by all BSs that acts as a function of the SMD-SLS state and SMD-SLS activation for obtaining optimal trade-off between performance and energy efficiency. Thus, an optimal policy is selected, denoted by π, which has maximized the total reward under the threshold constraint with SINR function on the zth user's SINR g k (x) within x. This is formulated as in Equation (6): where: v π (S) is referred to as the maximum rewards under the policy π over the present state S, gth is referred to as the targeted SINR, and λ is referred to as the discount factor g k (x) is referred to as the SINR of z user under cell x. where: h is referred to as the propagation coefficient, P tx (x) is referred to as the transmit power of cell x, x is referred to as the attenuation coefficient, which is set between 2 and 5, d z (x) is referred to as the distance from user z to BS cell i, N 0 is referred to as the AWGN function.
Information 2020, 11, x FOR PEER REVIEW 5 of 17 transmission power of two nearby femtocells by a multilevel power control in FSS, a3(t)is the action of unchanged FMS and increasing the transmission power of two nearby femtocells by a multilevel power control in FSS, and, finally, a4(t) is the action of decreasing the power of FSS due to reduced MU without changing nearby cells. The current state of SMD-SLS, s(t)S, and action a(t)A represent the probability matrix of state transition for the next state s(t + 1), which is Pr{a(t), s(t + 1)|s(t)}. This Markovian function probability represents the state transition of the previous state and is independent of the next stage when specified in the present stage.
In SMD-SLS, during each epoch, power control scheduling decides the power allocation to the femtocells, as per the power and traffic profiles. Power control scheduling uses the activation function a(t) at each epoch. Since the objective is to minimize power consumption, a reward function is used as the power consumption function for the femtocells. Thus, the reward function is defined as: where P(S,A) is the total power consumed by all BSs that acts as a function of the SMD-SLS state and SMD-SLS activation for obtaining optimal trade-off between performance and energy efficiency. Thus, an optimal policy is selected, denoted by , which has maximized the total reward under the threshold constraint with SINR function on the zth user's SINR gk(x) within x. This is formulated as in Equation (6): where: v  (S) is referred to as the maximum rewards under the policy  over the present state S, gth is referred to as the targeted SINR, and  is referred to as the discount factor gk(x) is referred to as the SINR of z user under cell x. where: h is referred to as the propagation coefficient, Ptx(x) is referred to as the transmit power of cell x, x is referred to as the attenuation coefficient, which is set between 2 and 5, dz(x) is referred to as the distance from user z to BS cell i, N0 is referred to as the AWGN function.
,, , where: k,n = 1 when user z is allocated in n resource block, and vice versa. k represents the total resources allocated to user z.
Equation (8) assures that each resource block is allocated to a user within the same femtocell. Equation (9) guarantees that zUE is allocated with the z resource block. Equation (7) is an integer linear programming problem (ILP) that is solved by ILP solvers.
Efficient methods are required to solve the running time of ILP solvers, which is uncertain, where: χ k,n = 1 when user z is allocated in n resource block, and vice versa. Ψ k represents the total resources allocated to user z.
Equation (8) assures that each resource block is allocated to a user within the same femtocell. Equation (9) guarantees that zUE is allocated with the z resource block. Equation (7) is an integer linear programming problem (ILP) that is solved by ILP solvers.
Efficient methods are required to solve the running time of ILP solvers, which is uncertain, thereby leading to computational complexity. To reduce this, a definite solution is provided that possess two parts. In the initial part, the FMS with a definite set of criteria determines the BSs for deactivation. Additionally, resource partitioning with the condition of the selected BS's switching state determines whether resource utilization can be increased. The final part is the allocation of resources that is independently solved by/within each BS using the Channel Quality indicator (CQI) user reports. Thus, the output of the initial part consists of an active BS set and an allocated resource block within a permissible level per user. Therefore, the orthogonal property is ensured with resource partitioning, which determines the best resources allocated per user.

Iteration Algorithm
An iterative algorithm is used to solve the problem of optimization that is specified by Equation (7). Specifically, the iterative algorithm derives the optimal policy and the expected reward function. The algorithm is expressed as: Step 1: Set the value for each state s(t) as v 0 π (S) = 0. Denote ε > 0 and σ is initialized to 0.
Step 5: For each state, s(t)∈S, optimal policy π is computed.
The computational complexity is thus calculated as O( S 2 A ); · represents cardinality set.

Femtocell Deactivation Condition
The BS selection is performed using the femtocell deactivation condition. The deactivation trials are described as follows: Trial-1: The total number of conflicts (NoC) of an individual BS can be inferred as total UEs interfered with by BS x. The total number of x connected UEs is interfered with through neighboring BSs. Hence, the NoC is calculated by the FMS, which is associated with individual BS based on IM: where x = a and x= b; if a = b then x = 1 and a b then x = 0. Trial-2: If entire UEs belong to BS, then the UE proportion located in the handover region is expressed as: where H x -handover ratio of BS x, S x -total UEs linked to x L x -total UEs served by x, which is located inside the region overlapped with nearby femtocells.
Trial-3: The total UEs connected to x, which is expressed as: Trial-4: The total received power (its sum) of all UEs connected to x is expressed as: Trial-5: The interference received power (its sum) by x from nearby BSs is expressed as:

Deactivation Algorithm
The deactivation algorithm for small cells incorporates a deactivation scheme and a resource partitioning scheme. Deactivating the selected or unwanted BSs reduces the interference with increased resource reuse and capacity. Measurements are taken to prevent outage probability of BSs, which are connected to cells in sleep mode. The deactivation algorithm for small cells is performed as follows: • Prior to executing the tests for deactivation, the sum of resource blocks allocated to each user is determined and compared with the updated deactivated value after deactivation policy.
where Ψ tot and Ψ tot -present and previous resources utilized, respectively.

•
The reuse efficiency is significantly affected by NoC, which is applied with Trial-1 to determine the maximum NoC in BSs: where ℘-preferred BS.

•
If ℘ = 1, BS is switched off, or else if ℘ > 2, then BS having maximum handover ratio is applied with Trial-2: • If ℘ = 0, which means BS is not nominated in the handover region, the deactivation policy is selected based on Trial-4 for all BSs with equal users: Equation (18) is used to find whether ℘ possesses less received power than other BSs and whether it enhances the overall capacity. Otherwise, Trial-3 selects the BS with minimum users for reducing the UEs dropping: • The dropping UEs are avoided by selecting the largest handover ratio on ℘, which is located between the other cells within the handover region. If ℘ > 1 and ℘ = H, and if users lie within the handover region of nearby cells, Trial-5 is used for selecting ℘, which is shown below: Trial-5 in Equation (20) selects a BS that receives the maximum interference from neighboring BSs, because such UEs achieve enhanced signal quality when handed over to the cells. Otherwise, the selection of ℘ is based on Equation (18).

•
Finally, ℘ is deactivated and the nearby BSs in an active state with maximum power receive disconnected users. However, a user will be dropped if it lies outside the coverage range. • ζ is updated and then the Ψ tot is calculated.

•
The Ψ tot (updated state) is compared with the Ψ tot (previous state) and ℘ = SL mode if Ψ tot > Ψ tot . The process continues to searching for the other users/BSs for deactivation.

Allocation of Resource
Each BS/user is adopted independently using the frequency resource allocation algorithm that is measured from total resource per user. BSs use the CQI report from each user for selecting the preferred resource block and-after determination of the preferred resource block-an update is periodically sent to BSs. The resource management allocates user data packets in the frequency domain according to the implemented scheduling rule using reinforcement learning for improving the QoS provisioning for the frequency resource allocation algorithm towards improving the QoS of BSs.

Performance Analysis
In order to study the impact of a downlink signal-to-interference ratio (SIR) on the user location with certain required parameters, the simulation results can be verified using the probability density function (PDF). To order to model accurately the received interference, a grid design of static BS structures and random users is presumed.

Signal to Interference Model
The received interference power from BS, x, to UE, z, is P t d −ϕ x,z , where P t represents the transmission power of x, φ represents the path loss exponent and d x,z represents the distance between x and z. Assume that the value P t is similar for all BSs and, hence, the user z is not interfered with by y when the interference of y < threshold: where β represents the threshold of SIR, and d x,z /d y,z or D is the ratio of the distance between z, x, and y.

Conflict Analysis
The deactivation decision is majorly impacted through NoC since the grid scenario is considered with BSs (F) and UEs (S f ), which is connected with BS (f ). The total interfered users in the detected coverage BS (f ) area are given as: where F D is the cumulative density function (CDF) of f D . Further, the total users of 'f' interfered with through its nearby small cells is: and Figure 2 shows their effect of the SIR threshold on NoC between the nominated cell and neighboring cells under different distance criteria. As shown in Figure 2, NoC is proportional to the SIR threshold, as NoC increases with the increase in the SIR threshold. A greater number of interfered users is detected within a coverage area by increasing the NoC.

Reuse Efficiency
The SIR threshold (β) effect completely influences reuse efficiency and, hence, the β effect on utilization of resources using PDF is studied in this section. Consider a system with f detected users. Sf users per F BSs is equal to: Total resources available per user is Ψz = N/Tf and total resources allocated per BS is Ωf = k/Sf, as follows: 1 1 The impact of SIR threshold on reuse efficiency is shown in Figure 3; it can be inferred that reuse efficiency decreases as the SIR threshold increases, since NoC is found to be higher in the network. Hence, it can be seen that the SIR poses a serious threat to reuse efficiency due to NoC constraints.

Reuse Efficiency
The SIR threshold (β) effect completely influences reuse efficiency and, hence, the β effect on utilization of resources using PDF is studied in this section. Consider a system with f detected users. S f users per F BSs is equal to: Total resources available per user is Ψ z = N/T f and total resources allocated per BS is Ω f = k/S f , as follows: The impact of SIR threshold on reuse efficiency is shown in Figure 3; it can be inferred that reuse efficiency decreases as the SIR threshold increases, since NoC is found to be higher in the network. Hence, it can be seen that the SIR poses a serious threat to reuse efficiency due to NoC constraints. where dm represents the total bits to encode the IM.
The centralized FMS requires the feedback of the local IM from its individual BS, which extracts required information including NoC, H, and S. If ℘ > 1, then the signal power received by BSs of the users is sent as feedback to the centralized FMS: where dt represents deactivating the ℘ and τ represents the total bits required to send the feedback of CQI (4 bits). Conversely, the FMS is restricted in sending the feedback to the BSs, since allocation is independently performed by each individual BS.

Computational Complexity
An optimal and exhaustive benchmark approach uses an iterative mechanism to check the impact of BS deactivation with the available resources. In contrast, the proposed hybrid method selects the optimal BSs without the need of an exhaustive mechanism. The exhaustive approach requires O(M) iterations to find the deactivated cell whereas the proposed hybrid approach does not use such a mechanism to find the deactivated cell. M defines the overall BSs for testing over M iterations with M possibilities.

Performance Evaluation
This section presents the metrics of homogeneous and heterogeneous solution. The parameters for simulation are presented in Table 1.

Signaling Overhead
The total exchanged information between the BSs and the centralized FMS is: where d f i represents the total bits for sending the IM from BS f i to centralized FMS. Assume f i BS with neighboring J i BSs and S i users, then: where d m represents the total bits to encode the IM. The centralized FMS requires the feedback of the local IM from its individual BS, which extracts required information including NoC, H, and S. If ℘ > 1, then the signal power received by BSs of the users is sent as feedback to the centralized FMS: where d t represents deactivating the ℘ and τ represents the total bits required to send the feedback of CQI (4 bits). Conversely, the FMS is restricted in sending the feedback to the BSs, since allocation is independently performed by each individual BS.

Computational Complexity
An optimal and exhaustive benchmark approach uses an iterative mechanism to check the impact of BS deactivation with the available resources. In contrast, the proposed hybrid method selects the optimal BSs without the need of an exhaustive mechanism. The exhaustive approach requires O(M) iterations to find the deactivated cell whereas the proposed hybrid approach does not use such a mechanism to find the deactivated cell. M defines the overall BSs for testing over M iterations with M possibilities.

Performance Evaluation
This section presents the metrics of homogeneous and heterogeneous solution. The parameters for simulation are presented in Table 1. The proposed system is analyzed by a comparative evaluation with the Manchester technique [25]. Both methods use IM for indicating the interference between the users and the BSs. The Manchester technique makes a similar assumption as SMD-SLS except for the use of the SMD algorithm. The proposed method is also compared to the optimal approach with upper bound performance. Figure 4 shows the mean rate at various cell densities, which reveals that the proposed method is significantly better than the Manchester method. The results in Figure 4 show that there is a significant improvement in reuse efficiency due to reduced interference in neighboring cells. This is due to the minimization of the number of femtocells that are unnecessarily active. The performance gap between the proposed SMD-SLS and Manchester techniques is increased with a denser network formation, where improvements have increased to 23%, 22%, and 25% for a certain number of femtocells, for example, 12, 13 and 14, respectively. Furthermore, the performance of the Feedback Shift Register (FSR)-Manchester method shown in Figure 4 approaches the performance of the Manchester technique with increasing cell density since, in the FSR-Manchester technique, the entire resources are allocated to BSs, and the users are protected from interference, thereby tending to yield a higher data rate than the Manchester method with ICI. The proposed system is analyzed by a comparative evaluation with the Manchester technique [25]. Both methods use IM for indicating the interference between the users and the BSs. The Manchester technique makes a similar assumption as SMD-SLS except for the use of the SMD algorithm. The proposed method is also compared to the optimal approach with upper bound performance. Figure 4 shows the mean rate at various cell densities, which reveals that the proposed method is significantly better than the Manchester method. The results in Figure 4 show that there is a significant improvement in reuse efficiency due to reduced interference in neighboring cells. This is due to the minimization of the number of femtocells that are unnecessarily active. The performance gap between the proposed SMD-SLS and Manchester techniques is increased with a denser network formation, where improvements have increased to 23%, 22%, and 25% for a certain number of femtocells, for example, 12, 13 and 14, respectively. Furthermore, the performance of the Feedback Shift Register (FSR)-Manchester method shown in Figure 4 approaches the performance of the Manchester technique with increasing cell density since, in the FSR-Manchester technique, the entire resources are allocated to BSs, and the users are protected from interference, thereby tending to yield a higher data rate than the Manchester method with ICI.  Figure 5 shows the Energy Consumption Rating (ECR) with various cell densities, which indicates that there is an improvement in the data rate as compared to the existing method. The performance of ECR in the proposed SMD-SLS is better than in the Manchester method, even with   Figure 5 shows the Energy Consumption Rating (ECR) with various cell densities, which indicates that there is an improvement in the data rate as compared to the existing method. The performance of ECR in the proposed SMD-SLS is better than in the Manchester method, even with higher densities, since the aim of the benchmark technique [26] is to reduce the consumption of energy.
higher densities, since the aim of the benchmark technique [26] is to reduce the consumption of energy.  Figure 6 shows a comparison of power consumption between the SMD-SLS technique and the other methods. As can be seen in Figure 6, the proposed SMD-SLS technique consumes less power than the Manchester technique. This is because switching off the hardware parts during SL mode further reduced power consumption in the proposed technique [12]. Alternatively, it is found that the benchmark method consumes low power as compared to the other methods, since additional BSs are further switched off. An evaluation of throughput vs CDF with a total of seven deployed femtocells is shown in Figure 7. The throughput results of the proposed SMD-SLS technique are better than the results of the Manchester technique, with a rate increase of approximately 30%-32%. The overall interference rate is significantly reduced due to activation of the SL mode in unnecessary femtocells. The avoidance of such disturbances further improved resource utilization by the nearby BSs. The BS receives a stronger signal due to the open-access policy of BSs with the users. Total deactivated cells in all three techniques are shown in Figure 8 Figure 6 shows a comparison of power consumption between the SMD-SLS technique and the other methods. As can be seen in Figure 6, the proposed SMD-SLS technique consumes less power than the Manchester technique. This is because switching off the hardware parts during SL mode further reduced power consumption in the proposed technique [12]. Alternatively, it is found that the benchmark method consumes low power as compared to the other methods, since additional BSs are further switched off.
Information 2020, 11, x FOR PEER REVIEW 12 of 17 higher densities, since the aim of the benchmark technique [26] is to reduce the consumption of energy.  Figure 6 shows a comparison of power consumption between the SMD-SLS technique and the other methods. As can be seen in Figure 6, the proposed SMD-SLS technique consumes less power than the Manchester technique. This is because switching off the hardware parts during SL mode further reduced power consumption in the proposed technique [12]. Alternatively, it is found that the benchmark method consumes low power as compared to the other methods, since additional BSs are further switched off. An evaluation of throughput vs CDF with a total of seven deployed femtocells is shown in Figure 7. The throughput results of the proposed SMD-SLS technique are better than the results of the Manchester technique, with a rate increase of approximately 30%-32%. The overall interference rate is significantly reduced due to activation of the SL mode in unnecessary femtocells. The avoidance of such disturbances further improved resource utilization by the nearby BSs. The BS receives a stronger signal due to the open-access policy of BSs with the users. Total deactivated cells in all three techniques are shown in Figure 8,   An evaluation of throughput vs CDF with a total of seven deployed femtocells is shown in Figure 7. The throughput results of the proposed SMD-SLS technique are better than the results of the Manchester technique, with a rate increase of approximately 30%-32%. The overall interference rate is significantly reduced due to activation of the SL mode in unnecessary femtocells. The avoidance of such disturbances further improved resource utilization by the nearby BSs. The BS receives a stronger signal due to the open-access policy of BSs with the users. Total deactivated cells in all three techniques are shown in Figure 8, in which the highest rate of deactivation occurred in the proposed SMD-SLS technique compared to the benchmark and Manchester technique. The dropping ratio of total users vs total UEs is shown in Figure 9, which shows that the proposed SMD-SLS technique displayed an increased dropped ratio with increasing user density, since dropping UEs are reduced against the Manchester technique. The average capacity of UEs (shown in Figure 10) using the proposed method is improved by a rate of 14%-20% compared to the Manchester technique, based on total BSs. Information 2020, 11, x FOR PEER REVIEW 13 of 17 ratio of total users vs total UEs is shown in Figure 9, which shows that the proposed SMD-SLS technique displayed an increased dropped ratio with increasing user density, since dropping UEs are reduced against the Manchester technique. The average capacity of UEs (shown in Figure 10) using the proposed method is improved by a rate of 14%-20% compared to the Manchester technique, based on total BSs.    Information 2020, 11, x FOR PEER REVIEW 13 of 17 ratio of total users vs total UEs is shown in Figure 9, which shows that the proposed SMD-SLS technique displayed an increased dropped ratio with increasing user density, since dropping UEs are reduced against the Manchester technique. The average capacity of UEs (shown in Figure 10) using the proposed method is improved by a rate of 14%-20% compared to the Manchester technique, based on total BSs.     For a heterogeneous solution, the femtocell access threshold μ = 12 dB offloads the macrocells with the femtocells. Figure 11 shows the average throughput vs CDF for evaluating the performance of SMD-SLS compared to Manchester and the benchmark technique. It can be inferred from the results that the performance of SMD-SLS is superior to the other techniques. The results show that the improvement in performance is less apparent in a heterogeneous network compared to a homogeneous network, since the femtocells are randomly deployed in the coverage area of the macrocells, which reduces the overlapped area by eliminating the deactivated cells.   For a heterogeneous solution, the femtocell access threshold μ = 12 dB offloads the macrocells with the femtocells. Figure 11 shows the average throughput vs CDF for evaluating the performance of SMD-SLS compared to Manchester and the benchmark technique. It can be inferred from the results that the performance of SMD-SLS is superior to the other techniques. The results show that the improvement in performance is less apparent in a heterogeneous network compared to a homogeneous network, since the femtocells are randomly deployed in the coverage area of the macrocells, which reduces the overlapped area by eliminating the deactivated cells.  For a heterogeneous solution, the femtocell access threshold µ = 12 dB offloads the macrocells with the femtocells. Figure 11 shows the average throughput vs CDF for evaluating the performance of SMD-SLS compared to Manchester and the benchmark technique. It can be inferred from the results that the performance of SMD-SLS is superior to the other techniques. The results show that the improvement in performance is less apparent in a heterogeneous network compared to a homogeneous network, since the femtocells are randomly deployed in the coverage area of the macrocells, which reduces the overlapped area by eliminating the deactivated cells.

Limitations, Practical Implications and Future Work
The major limitation is the failure in deploying advanced machine learning models for the selection of sleep modes. The semi-Markov model designed cannot be applied for other network architectures and is purely limited to this model.
The throughput results for the homogeneous model and heterogonous model shows the application of the proposed method on advanced networks with high bandwidth constraints. It further can provide better adoption on small cell networks and offers wide benefits in the allocation of resources using a semi-Markov process.
In future, the proposed small cell deactivation method may be modelled with a machine learning or a heuristic model that can enhance the strategy of selecting the small cell in an optimal manner. This would improve the network throughput of the entire system.

Conclusions
A hybrid semi-Markov sleep mode selection with a small cell deactivation strategy is shown to improve the efficiency of power and resource utilization in small cell OFDMA networks. The semi-Markov process optimizes sleep mode selection, thereby minimizing the overall power consumption in networks using operational mode transitions. Network performance is increased by deactivating the small cells located at undesirable spots. Comparison with conventional non-sleeping mode cells demonstrated that the proposed hybrid model increased the efficiency of data rate and power efficiency by 38.5% and 19.2%, respectively. The CDF of the proposed system demonstrated that the throughput performance is increased up to 34% compared to non-sleeping mode heterogeneous cell systems. Finally, the proposed SMD-SLS system approaches the benchmark performance with lowered computational cost.

Limitations, Practical Implications and Future Work
The major limitation is the failure in deploying advanced machine learning models for the selection of sleep modes. The semi-Markov model designed cannot be applied for other network architectures and is purely limited to this model.
The throughput results for the homogeneous model and heterogonous model shows the application of the proposed method on advanced networks with high bandwidth constraints. It further can provide better adoption on small cell networks and offers wide benefits in the allocation of resources using a semi-Markov process.
In future, the proposed small cell deactivation method may be modelled with a machine learning or a heuristic model that can enhance the strategy of selecting the small cell in an optimal manner. This would improve the network throughput of the entire system.

Conclusions
A hybrid semi-Markov sleep mode selection with a small cell deactivation strategy is shown to improve the efficiency of power and resource utilization in small cell OFDMA networks. The semi-Markov process optimizes sleep mode selection, thereby minimizing the overall power consumption in networks using operational mode transitions. Network performance is increased by deactivating the small cells located at undesirable spots. Comparison with conventional non-sleeping mode cells demonstrated that the proposed hybrid model increased the efficiency of data rate and power efficiency by 38.5% and 19.2%, respectively. The CDF of the proposed system demonstrated that the throughput performance is increased up to 34% compared to non-sleeping mode heterogeneous cell systems. Finally, the proposed SMD-SLS system approaches the benchmark performance with lowered computational cost.

Conflicts of Interest:
The authors declare no conflict of interest.