# Solution for Interference in Hotspot Scenarios Applying Q-Learning on FFR-Based ICIC Techniques

^{*}

## Abstract

**:**

## 1. Introduction

- Performance comparison of classic FFR-based ICIC algorithms that take into account the number of users and their distributions along the cell (edge and center);
- Statistical analysis of the strict frequency reuse scheme that identifies which parameters do not need to be dynamically controlled;
- Analysis of the strict frequency reuse algorithm’s performance in a mixed scenario with hotspot and homogeneous user distribution;
- Q-Learning algorithm that continually operates in the network to dynamically mitigate the performance loss (SINR) that results from the appearance of hotspots (densely populated areas).

## 2. Interference in Hotspot Scenarios

#### 2.1. Classic Solutions for Interference in Hotspot Scenarios

#### 2.1.1. MIMO Systems

#### 2.1.2. Small Cells

## 3. Fractional Frequency Reuse

#### 3.1. ICIC Techniques

#### 3.1.1. Strict Frequency Reuse

#### 3.1.2. Soft Frequency Reuse

#### 3.1.3. Soft Fractional Frequency Reuse

## 4. ICIC in 3GPP Standards

## 5. Related Works

## 6. System Model

#### 6.1. Simulation Software

#### 6.2. LTE Module and ICIC on ns-3

#### 6.3. Q-Learning on ns-3

## 7. Preliminary Analysis A: Performance Comparison of ICIC Algorithms

#### 7.1. Evaluation Scenario

**Scenario 1:**UEs are concentrated in a 100 m radius, representing a classic hotspot scenario;**Scenario 2:**UEs are less concentrated, distributed over a 500 m radius circle.

#### 7.2. Results and Discussions

- FFR-based ICIC techniques can improve the performance of a mobile network, given the high interference suffered by its users. It is possible to enhance cell-edge performance without compromising cell-center users;
- Simply changing the user density can strongly impact on system performance, especially if the bandwidth is divided into sub-bands. The concentration or spreading of the users can result in different occupation of the sub-bands, leading to overloaded or nearly empty sub-bands;
- There is no scheme that performs best in every situation. However, the Strict FR and the SFFR schemes have a good performance in different scenarios, and they are both efficient at reducing ICI, especially for cell-edge users;
- The Strict Frequency Reuse (Strict FR) algorithm has the best compromise between performance and complexity, considering SFFR has a higher number of sub-bands, which leads to more complexity when adjusting the parameters. Therefore, the Strict FR is the object of the study presented in the following sections.

## 8. Preliminary Analysis B: Factorial Design using the Strict Frequency Reuse Algorithm

#### 8.1. Evaluation Scenario

#### 8.2. ${2}^{k}$ Factorial Design

#### 8.3. Full Factorial Design

## 9. Preliminary Analysis C: Evaluating the Hotspot Scenario

#### 9.1. Evaluation Scenario

#### 9.2. Simulation Results

## 10. Proposed Solution: Background, Implementation, and Simulation Results

#### 10.1. Machine Learning Techniques

#### 10.2. Reinforcement Learning

#### 10.3. Q-Learning

#### 10.4. The Q-Learning Implementation

- A set of available actions, $A={a}_{1},{a}_{2},...,{a}_{n}$;
- A set of possible states of the system, $S={s}_{1},{s}_{2},...,{s}_{n}$;
- A $Q(s,a)$ matrix to store the estimated rewards;
- $\alpha $ and $\gamma $.

- State 1: $SINR<24\phantom{\rule{3.33333pt}{0ex}}dB$;
- State 2: $24\phantom{\rule{3.33333pt}{0ex}}dB<SINR\le 27\phantom{\rule{3.33333pt}{0ex}}dB$;
- State 3: $27\phantom{\rule{3.33333pt}{0ex}}dB<SINR\le 29\phantom{\rule{3.33333pt}{0ex}}dB$;
- State 4: $29\phantom{\rule{3.33333pt}{0ex}}dB<SINR\le 30\phantom{\rule{3.33333pt}{0ex}}dB$;
- State 5: $30\phantom{\rule{3.33333pt}{0ex}}dB<SINR\le 32\phantom{\rule{3.33333pt}{0ex}}dB$;
- State 6: $SINR>32\phantom{\rule{3.33333pt}{0ex}}dB$.

#### 10.5. Evaluation Scenario

Algorithm 1: Pseudo-code of the proposed Q-learning algorithm. |

**Scenario 1**has 60 users distributed uniformly and 10 users on each hotspot. A total of 52 RBs are allocated for the common sub-band, and 16 RBs are allocated to the private sub-band.

**Scenario 2**has the double of users on each hotspot and 2/3 of the uniform-distributed users when compared to Scenario 1. Hence, there are 40 users distributed uniformly and 20 users on each hotspot, which results in a higher number of users in the system. A total of 28 RBs are allocated for the common sub-band, and 24 RBs are allocated to the private sub-band. Therefore, the available bandwidth for users allocated in the common sub-band is significantly smaller. Other simulation parameters are summarized in Table 10.

#### 10.6. Data Collection on ns-3

#### 10.7. Proof-of-Concept Simulation Results

## 11. Conclusions

## 12. Future Works

- Use other metrics to define the QL states and actions, such as the BandwidthDistribution. As presented in Section 10.3, these metrics can also be the combination of different variables. As a result, the algorithm is expected to improve in flexibility and efficiency when using two parameters, adapting to a broader set of scenarios. For example, the throughput could be included as part of the reward;
- Expand proposed scenarios: vary number of users, add mobility, varying number and location of hotspots (which could also appear in random locations), and test different values for BandwidthDistribution;
- Evaluate the system using different metrics, such as the relation between convergence speed, the amount of state/action pairs, or the Packet Loss Ratio (PLR);
- Apply the solution on a system without isotropic antennas, such that transmission is made in sectors within a cell;
- Apply the presented solution for 5G New Radio (NR) or the LTE UL, given that the interference in the UL has different characteristics, when compared to DL;
- Provide a similar solution, replacing the RL algorithm for, e.g., multi-armed bandit, providing a simpler solution.

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## References

- Cisco; Cisco Systems, I. Cisco Visual Networking Index: Forecast and Trends, 2017–2022. Cisco White Paper
**2019**. [Google Scholar] - GSMA Association. The Mobile Economy, 2019; GSMA Association: London, UK, 2019. [Google Scholar]
- Shannon, C. Communication In The Presence Of Noise (Republished). Proc. IEEE
**1998**, 86, 447–457. [Google Scholar] [CrossRef] - Zheng, K.; Li, Y.; Zhang, Y.; Jiang, Z.; Long, H. Performance analysis and evaluation of deployment in small cell networks. KSII Trans. Internet Inf. Syst.
**2015**, 9, 886–900. [Google Scholar] [CrossRef] - Cavalcanti, F.R.P.; Maciel, T.F.; Freitas, W.C., Jr.; Silva, Y.C.B. Comunicação Móvel Celular, 1st ed.; Elsevier: Rio de Janeiro, Brazil, 2018. [Google Scholar]
- Diggavi, S.N.; Al-dhahir, N.; Member, S. Great Expectations: The Value of Spatial. Wirel. Netw.
**2004**, 92, 219–270. [Google Scholar] - Adhikary, A.; Nam, J.; Ahn, J.Y.; Caire, G. Joint spatial division and multiplexing-The large-scale array regime. IEEE Trans. Inf. Theory
**2013**, 59, 6441–6463. [Google Scholar] [CrossRef] - Adhikary, A.; Dhillon, H.S.; Caire, G. Massive-MIMO meets HetNet: Interference coordination through spatial blanking. IEEE J. Sel. Areas Commun.
**2015**, 33, 1171–1186. [Google Scholar] [CrossRef] [Green Version] - Muirhead, D.; Imran, M.A.; Arshad, K. A Survey of the Challenges, Opportunities and Use of Multiple Antennas in Current and Future 5G Small Cell Base Stations. IEEE Access
**2016**, 4, 2952–2964. [Google Scholar] [CrossRef] - Yaacoub, E.; Husseini, M.; Ghaziri, H. An overview of research topics and challenges for 5G massive MIMO antennas. In Proceedings of the 2016 IEEE Middle East Conference on Antennas and Propagation (MECAP), Beirut, Lebanon, 20–22 September 2016; pp. 1–4. [Google Scholar] [CrossRef]
- Khandekar, A.; Bhushan, N.; Tingfang, J.; Vanghi, V. LTE-Advanced: Heterogeneous Networks. In Proceedings of the 2010 European Wireless Conference (EW), Lucca, Italy, 12–15 April 2010. [Google Scholar]
- Damnjanovic, A.; Montojo, J.; Wei, Y.; Ji, T.; Luo, T.; Vajapeyam, M.; Yoo, T.; Song, O.; Malladi, D. A survey on 3GPP heterogeneous networks. IEEE Wirel. Commun.
**2011**, 18, 10–21. [Google Scholar] [CrossRef] - Erlinghagen, K.; Dusza, B.; Wietfeld, C. Dynamic Cell Size Adaptation and Intercell Interference Coordination in LTE HetNets. In Proceedings of the 2013 IEEE 78th Vehicular Technology Conference (VTC Fall), Las Vegas, NV, USA, 2–5 September 2013. [Google Scholar]
- Chandrasekhar, V.; Andrews, J.G.; Gatherer, A. Femtocell Networks: A Survey. IEEE Commun. Mag.
**2008**, 46, 59–67. [Google Scholar] [CrossRef] [Green Version] - Al-Turjman, F.; Ever, E.; Zahmatkesh, H. Small cells in the forthcoming 5G/IoT: Traffic modelling and deployment overview. IEEE Commun. Surv. Tutorials
**2019**, 21, 28–65. [Google Scholar] [CrossRef] - Mudassir, A.; Akhtar, S.; Kamel, H. Survey on Inter-cell Interference Coordination in LTE-Advanced Heterogeneous Networks. In Proceedings of the 2016 Sixth International Conference on Innovative Computing Technology (INTECH), Dublin, Ireland, 24–26 August 2016. [Google Scholar]
- Ree, M.D.; Mantas, G.; Radwan, A.; Mumtaz, S.; Rodriguez, J.; Otung, I.E. Key Management for Beyond 5G Mobile Small Cells: A Survey. IEEE Access
**2019**, 7, 59200–59236. [Google Scholar] [CrossRef] - Chang, B.J.; Liou, S.H.; Liang, Y.H. Cooperative communication in ultra-dense small cells toward 5G cellular communication. In Proceedings of the 2017 8th IEEE Annual Information Technology, Electronics and Mobile Communication Conference, IEMCON, Vancouver, BC, Canada, 3–5 October 2017; pp. 365–371. [Google Scholar] [CrossRef]
- Gawłowicz, P.; Baldo, N.; Miozzo, M. An Extension of the ns-3 LTE Module to Simulate Fractional Frequency Reuse Algorithms. In Proceedings of the Workshop on ns-3-WNS3 2015, Barcelona, Spain, 13–14 May 2015. [Google Scholar]
- Hamza, A.; Khalifa, S.; Hamza, H.; Elsayed, K. A Survey on Inter-Cell Interference Coordination Techniques in OFDMA-Based Cellular Networks. IEEE Commun. Surv. Tutorials
**2013**, 15, 1642–1670. [Google Scholar] [CrossRef] - Kimura, D.; Seki, H. Inter-Cell Interference Coordination (ICIC) Technology. Fujitsu Sci. Tech. J.
**2012**, 48, 89–94. [Google Scholar] - Xie, Z.; Walke, B. Enhanced Fractional Frequency Reuse to Increase Capacity of OFDMA Systems. In Proceedings of the 3rd International on New Technologies, Mobility and Security (NTMS 2009), Cairo, Egypt, 20–23 December 2009. [Google Scholar]
- Sesia, S.; Toufik, I.; Baker, M. LTE—The UMTS Long Term Evolution: From Theory to Practice, 2nd ed.; Wiley: Chichester, UK, 2011. [Google Scholar]
- Holma, H.; Toskala, A. LTE Advanced: 3GPP Solution for IMT-Advanced, 1st ed.; Wiley: Chichester, UK, 2012. [Google Scholar]
- 3GPP. TS 36.213: Technical Specification Group Radio Access Network; Evolved Universal Terrestrial Radio Access (E-UTRA); Physical Layer Procedures (Release 8); Technical Report; 3GPP: Valbonne, France, 2009. [Google Scholar]
- 3GPP. 5G; NR; Overall Description; Technical Specification 38.300 version 15.3.1 Release 15; Technical Report; 3GPP: Valbonne, France, 2018. [Google Scholar]
- Soret, B.; De Domenico, A.; Bazzi, S.; Mahmood, N.H.; Pedersen, K.I. Interference Coordination for 5G New Radio. IEEE Wirel. Commun.
**2018**, 25, 131–137. [Google Scholar] [CrossRef] [Green Version] - Singh, R.; Siva Ram Murthy, C. Techniques for Interference Mitigation Using Cooperative Resource Partitioning in Multitier LTE HetNets. IEEE Syst. J.
**2018**, 12, 843–853. [Google Scholar] [CrossRef] - López-Pérez, D.; Ding, M.; Claussen, H.; Jafari, A.H. Towards 1 Gbps/UE in Cellular Systems: Understanding Ultra-Dense Small Cell Deployments. IEEE Commun. Surv. Tutorials
**2015**, 17, 2078–2101. [Google Scholar] [CrossRef] [Green Version] - Shirakabe, M.; Morimoto, A.; Miki, N. Performance evaluation of inter-cell interference coordination and cell range expansion in heterogeneous networks for LTE-Advanced downlink. In Proceedings of the 2011 8th International Symposium on Wireless Communication Systems, Aachen, Germany, 6–9 November 2011; pp. 844–848. [Google Scholar] [CrossRef]
- Shen, L.H.; Feng, K.T. Joint beam and subband resource allocation with QoS requirement for millimeter wave MIMO systems. In Proceedings of the 2017 IEEE Wireless Communications and Networking Conference (WCNC), San Francisco, CA, USA, 19–22 March 2017; pp. 1–6. [Google Scholar] [CrossRef]
- Rüegg, T.; Hassan, Y.; Wittneben, A. User cooperation enabled traffic offloading in urban hotspots. In Proceedings of the IEEE International Symposium on Personal, Indoor and Mobile Radio Communications, PIMRC, Valencia, Spain, 4–8 September 2016. [Google Scholar] [CrossRef]
- Khan, S.A.; Colak, S.A.; Kavak, A.; Kucuk, K. A User Location Distribution Based FFR Strategy for Efficient Utilization of Radio Resources in LTE-A HetNets. In Proceedings of the 1st International Informatics and Software Engineering Conference: Innovative Technologies for Digital Transformation, IISEC 2019-Proceedings, Ankara, Turkey, 6–7 November 2019. [Google Scholar] [CrossRef]
- Li, X.; Liu, Z.; Qin, N.; Jin, S. FFR Based Joint 3D Beamforming Interference Coordination for Multi-Cell FD-MIMO Downlink Transmission Systems. IEEE Trans. Veh. Technol.
**2020**, 69, 3105–3118. [Google Scholar] [CrossRef] - Zheng, C.; Hailin, Z. A Quasi-Perfect Resource Allocation Scheme for Optimizing the Performance of Cell-Edge Users in FFR-Aided LTE-A Multicell Networks. IEEE Commun. Lett.
**2019**, 23, 918–921. [Google Scholar] [CrossRef] - Mitsolidou, C.; Vagionas, C.; Mesodiakaki, A.; Maniotis, P.; Kalfas, G.; Roeloffzen, C.G.; van Dijk, P.W.; Oldenbeuving, R.M.; Miliou, A.; Pleros, N. A 5G C-RAN optical fronthaul architecture for hotspot areas using OFDM-based analog IFoF waveforms. Appl. Sci.
**2019**, 9, 4059. [Google Scholar] [CrossRef] [Green Version] - Ma, B.; Yang, B.; Zhu, Y.; Zhang, J. Context-aware proactive 5g load balancing and optimization for urban areas. IEEE Access
**2020**, 8, 8405–8417. [Google Scholar] [CrossRef] - Abdullahi, S.U.; Liu, J.; Mohadeskasaei, S.A. Analytical evaluation of FFR-aided heterogeneous cellular networks with optimal double threshold. KSII Trans. Internet Inf. Syst.
**2017**, 11, 3370–3392. [Google Scholar] [CrossRef] - ns-3. ns-3 Documentation. Página na Internet, ns-3. 2020. Available online: https://www.nsnam.org/ (accessed on 24 October 2021).
- ns-3. Design Documentation. Página na Internet, ns-3. 2019. Available online: https://www.nsnam.org/ (accessed on 24 October 2021).
- Hamza, A.S.; Khalifa, S.S.; Hamza, H.S.; Elsayed, K. A Survey on Inter-Cell Interference Co-ordination Techniques of LTE Networks. IEEE Commun. Surv. Tutor.
**2013**, 4, 1–29. [Google Scholar] - 3GPP. TS 36.133: Technical Specification Group Radio Access Network; Evolved Universal Terrestrial Radio Access (E-UTRA); Requirements for Support of Radio Resource Management (Release 8); Technical Report; 3GPP: Valbonne, France, 2010. [Google Scholar]
- de Santana, P.M.; de Sousa, V.A., Jr.; Abinader, F.M., Jr.; de C. Neto, J.M. DM-CSAT: A LTE-U/Wi-Fi coexistence solution based on reinforcement learning. Telecommun. Syst.
**2019**, 1, 1–12. [Google Scholar] [CrossRef] - Montgomery, D.C. Estatística Aplicada e Probabilidade Para Engenheiros, 4th ed.; ETC: Leusden, The Netherlands, 2009. [Google Scholar]
- Mathematics, E. Chi-Squared Distribution; Web page; Springer: Berlin/Heidelberg, Germany, 2020. [Google Scholar]
- Mathematics, E. Fisher-F-Distribution; Web page; Springer: Berlin/Heidelberg, Germany, 2020. [Google Scholar]
- Trejo Narváez, O.A.; Miramá Pérez, V.F. Machine learning algorithms for inter-cell interference coordination. Sist. TelemáTica
**2018**, 16, 37–57. [Google Scholar] [CrossRef] - Haykin, S.S. Neural Networks and Learning Machines, 3rd ed.; Pearson Education: Upper Saddle River, NJ, USA, 2009. [Google Scholar]
- Richard, S.; Sutton, A.G.B. Reinforcement Learning: An Introduction; MIT Press: Cambridge, MA, USA, 2018. [Google Scholar]
- Chen, Z.; Chuai, G.; Gao, W. A novel ICIC scheme for reducing intra cluster interference in LTE-Hi network. In Proceedings of the 2016 IEEE International Conference on Network Infrastructure and Digital Content (IC-NIDC), Beijing, China, 23–25 September 2017; Volume 129, pp. 262–267. [Google Scholar] [CrossRef]
- Simsek, M.; Bennis, M.; Czylwik, A. Dynamic Inter-Cell Interference Coordination in HetNets: A reinforcement learning approach. In Proceedings of the 2012 IEEE Global Communications Conference (GLOBECOM), Anaheim, CA, USA, 3–7 December 2012; pp. 5446–5450. [Google Scholar] [CrossRef]
- Morozs, N.; Clarke, T.; Grace, D. Distributed Heuristically Accelerated Q-Learning for Robust Cognitive Spectrum Management in LTE Cellular Systems. IEEE Trans. Mob. Comput.
**2016**, 15, 817–825. [Google Scholar] [CrossRef] - de Santana, P.M.; de Sousa, V.A., Jr.; Abinader, F.M., Jr.; de C. Neto, J.M. GTDM-CSAT: An LTE-U self Coexistence Solution based on Game Theory and Reinforcement Learning. J. Commun. Inf. Syst. (JCIS)
**2019**, 34, 169–177. [Google Scholar] [CrossRef] [Green Version] - Watkins, C.; Dayan, P. Q-learning. Mach. Learn.
**1992**, 8, 279–292. [Google Scholar] [CrossRef] - Watkins, C. Learning From Delayed Rewards. Ph.D. Thesis, King’s College, London, UK, 1989. [Google Scholar]
- Bazzo, J.J.; de Sousa, V.A., Jr. Method and Apparatus for Admission Control and Forced Handover in a Multi-Layer Network Configuration. U.S. Patent US8660086B2, 11 October 2011. [Google Scholar]

**Figure 1.**Scenario with reuse factor of 3. In this topology, no adjacent cells share bandwidth, i.e., their frequency bands are disjoint.

**Figure 2.**Bandwidth distribution and power allocation for the Strict frequency reuse algorithm in a cluster of three cells.

**Figure 3.**Bandwidth distribution and power allocation for the soft frequency reuse algorithm in a cluster of three cells.

**Figure 4.**Bandwidth distribution and power allocation for the soft fractional frequency reuse algorithm in a cluster of three cells.

**Figure 5.**LTE-EPC data plane protocol stack (taken from the ns-3 documentation [40]).

**Figure 6.**Scenario for preliminary analysis a: performance comparison of ICIC algorithms. The Radius R is set to 100 m in Scenario 1 and 500 m in Scenario 2, as described in Section 7.1.

**Figure 7.**Scenario 1 from preliminary analysis A: 10th percentile throughput for concentrated users (R = 100m). Each curve represents a different ICIC algorithm.

**Figure 8.**Scenario 1 from preliminary analysis A: SINR CDF for concentrated users (R = 100 m). Each curve represents a different ICIC algorithm.

**Figure 9.**Scenario 2 from preliminary analysis A: 10th percentile throughput for less concentrated users (R = 500 m). Each curve represents a different ICIC algorithm.

**Figure 10.**Scenario 2 from preliminary analysis A: SINR CDF for less concentrated users (R = 500 m). Each curve represents a different ICIC algorithm.

**Figure 11.**Scenario for preliminary analysis B, which is similar to the scenario in Figure 6, but without the radius R, since all users are randomly distributed according to a uniform distribution.

**Figure 12.**Full factorial design using the strict frequency reuse: results for the average throughput. Each curve is a different bandwidth distribution.

**Figure 13.**Full factorial design using the strict frequency reuse: results for the 10th percentile throughput. Each curve is a different bandwidth distribution.

**Figure 14.**Full factorial design using the strict frequency reuse: results for the SINR CDF. Each curve is a different bandwidth distribution.

**Figure 15.**Scenario for preliminary analysis C with the approximate position for each hotspot, which are not randomly located.

**Figure 16.**Simulation results in terms of the 10th percentile of throughput. The dashed lines represent the scenario with hotspots.

**Figure 19.**Scenario 1 of the proposed solution: throughput results for each group of users, with and without Q-Learning.

**Figure 20.**Scenario 2 of the proposed solution: throughput results for each group of users, with and without Q-Learning.

**Table 1.**Parameters used for the comparison of the ICIC algorithms in preliminary analysis A [19].

Hard Frequency Reuse | |
---|---|

Bandwidth: cells 1 and 2 | 8 RBs |

Bandwidth: cell 3 | 9 RBs |

Strict Frequency Reuse | |

Bandwidth (center/edge) | 6 RBs each |

RsrqThreshold | 32 |

Power offset (center) | –6 dB |

Power offset (edge) | 3 dB |

Soft Frequency Reuse | |

Bandwidth (center) | 25 RBs |

Bandwidth (edge): cells 1 and 2 | 9 RBs |

Bandwidth (edge): cell 3 | 9 RBs |

RsrqThreshold | 32 |

Power offset (center) | –6 dB |

Power offset (edge) | 3 dB |

Fractional Soft Frequency Reuse | |

Bandwidth (center/edge) | 6 RBs each |

RsrqThreshold (center) | 37 |

RsrqThreshold (edge) | 32 |

Power offset (center) | –6 dB |

Power offset (middle) | –1.77 dB |

Power offset (edge) | 3 dB |

**Table 2.**Simulation parameters for preliminary analysis a: performance comparison of ICIC algorithms.

Parameter | Value |
---|---|

Bandwidth (RBs) | 25 |

UE distribution | Uniform |

Total UEs | 40 to 240 |

Cell-edge UEs | 10 to 60 |

Inter-eNBs distance | 1000 m |

Scheduling algorithm | Proportional Fair |

Channel model | Friis model |

Error model | MIESM |

UE mobility | No mobility |

Traffic model | Non-GBR TCP-based |

Video (Buffered Stream) |

**Table 3.**Simulation parameters for preliminary analysis B: parametric analysis of strict frequency reuse.

Parameter | Value |
---|---|

Bandwidth (RBs) | 25 |

UE distribution | Uniform |

Number of UEs | 80 |

Distance between eNBs (m) | 1000 |

Scheduling algorithm | Proportional Fair |

Channel model | Friss model |

Error model | MIESM |

UE mobility | No mobility |

Traffic model | Non-GBR TCP-based |

Video (Buffered Stream) |

Parameter (Average Throughput) | F0 |
---|---|

CenterPowerOffset (A) | 0.084 |

RsrqThreshold (B) | 39040.4 |

BandwidthDistribution (C) | 4301.4 |

BandwidthDistribution * CenterPowerOffset | 0.0015 |

BandwidthDistribution * RsrqThreshold | 6264.7 |

RsrqThreshold * CenterPowerOffset | 0.0012 |

A * B * C | 0.00032 |

Parameter (10th Percentile) | F0 |
---|---|

CenterPowerOffset (A) | 0.00059 |

RsrqThreshold (B) | 2688.37 |

BandwidthDistribution (C) | 6359.40 |

BandwidthDistribution * CenterPowerOffset | 0.035 |

BandwidthDistribution * RsrqThreshold | 695.91 |

RsrqThreshold * CenterPowerOffset | 0.0033 |

A * B * C | 0.00034 |

Parameter | Values |
---|---|

RsrqThreshold | 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34 |

Common/Private | Bandwidth Distribution 1: 6 / 18 |

Sub-band (RBs) | Bandwidth Distribution 2: 12 / 12 |

Bandwidth Distribution 3: 18 / 6 |

Scenario 1 |
---|

60 uniformly distributed users and no hotspots |

Scenario 2 |

135 uniformly distributed users and no hotspots |

Scenario 3 |

135 users: 60 uniformly distributed users and |

5 hotspots with 15 users each |

Parameter | Value |
---|---|

Bandwidth (RBs) | 100 |

UE distribution | Uniform / Hotspots |

Number of UEs | 60 / 135 |

Distance between eNBs (m) | 1000 |

Scheduling algorithm | Proportional Fair |

Simulation duration | 6000 subframes |

Channel model | Friis Model |

Error model | MIESM |

UE mobility | No mobility |

Traffic model | Non-GBR TCP-based |

Video (Buffered Stream) |

Scenario 1 |
---|

60 users uniformly distributed |

10 users on each hotspot |

BandwidthDistribution: 52/16 |

Scenario 2 |

40 users uniformly distributed |

20 users on each hotspot |

BandwidthDistribution: 28/24 |

Parameter | Value |
---|---|

Bandwidth (RBs) | 100 |

UE distribution | Uniform / Hotspots |

Number of UEs | 110 / 140 |

Distance between eNBs (m) | 1000 |

Scheduling algorithm | Proportional Fair |

Simulation duration | 60,000 subframes |

Channel model | Friis model |

Error model | MIESM |

UE mobility | No mobility |

Traffic model | Non-GBR TCP-based |

Video (Buffered Stream) |

Interval | Scenario 1 (UEs) | Scenario 2 (UEs) |
---|---|---|

0 to 10 s | 60 + 0 on HS | 40 + 0 on HS |

10 to 20 s | 60 + 10 on HS | 40 + 20 on HS |

20 to 30 s | 60 + 20 on HS | 40 + 40 on HS |

30 to 40 s | 60 + 30 on HS | 40 + 60 on HS |

40 to 50 s | 60 + 40 on HS | 40 + 80 on HS |

50 to 60 s | 60 + 50 on HS | 40 + 100 on HS |

**Table 12.**Best and worst results for SINR and throughput of the Q-Learning solution, for each group of users.

Best SINR Gain | Worst SINR Gain | Best Tput Gain | Worst Tput Gain | |
---|---|---|---|---|

Scenario 1 | ||||

Hotspot users | 94% | 35.3% | 59.8% | 5.4% |

Outside hotspots | 99.1% | 11.1% | 6.5% | 0.2% |

All Users | 96.7% | 12.8% | 10.3% | 0.2% |

Scenario 2 | ||||

Hotspot users | 180.2% | 25.7% | 12.5% | 1.7% |

Outside hotspots | 140.9% | 8.6% | 1.9% | −3.8% |

All Users | 167.1% | 10.3% | 6.7% | −0.6% |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Diógenes do Rego, I.; de Sousa, V.A., Jr.
Solution for Interference in Hotspot Scenarios Applying Q-Learning on FFR-Based ICIC Techniques. *Sensors* **2021**, *21*, 7899.
https://doi.org/10.3390/s21237899

**AMA Style**

Diógenes do Rego I, de Sousa VA Jr.
Solution for Interference in Hotspot Scenarios Applying Q-Learning on FFR-Based ICIC Techniques. *Sensors*. 2021; 21(23):7899.
https://doi.org/10.3390/s21237899

**Chicago/Turabian Style**

Diógenes do Rego, Iago, and Vicente A. de Sousa, Jr.
2021. "Solution for Interference in Hotspot Scenarios Applying Q-Learning on FFR-Based ICIC Techniques" *Sensors* 21, no. 23: 7899.
https://doi.org/10.3390/s21237899