Network Long-Term Evolution Quality of Service Assessment Using a Weighted Fuzzy Inference System

Abstract

The United Nations has pushed for improved mobile connectivity, ensuring that 97% of the world’s population lives within reach of a mobile cellular signal. This is within the framework of objective nine regarding industry, innovation, and infrastructure for sustainable development. The next challenge is for users to know the quality of this service. The Long-Term Evolution (LTE) network’s quality of service (QoS) is evaluated with key performance indicators (KPI) that only specialists can interpret. This work aims to assess the QoS and effectiveness of the fourth-generation (4G) LTE network using a weighted fuzzy inference system. Analytic Hierarchy Process (AHP) is integrated to rank the fuzzy rules. The KPIs that are considered for the evaluation are download speed, upload speed, latency, jitter, packet loss rate, reference received signal power (RSRP), and reference received signal quality (RSRQ). The evaluated data were collected collaboratively with end-user equipment (UEs). Different usage scenarios are contemplated to define the importance according to the positive impact of the QoS of the LTE mobile network. The advantage of the weighted fuzzy inference system concerning the fuzzy inference system is that each KPI is assigned a different weight, which implies having rules with hierarchies. In this way, the weighted fuzzy inference system provides two indices of quality and effectiveness. It can be a valuable tool for end users and regulatory bodies to identify the quality of the LTE mobile network.

1. Introduction

Mobile network operators (MNOs) face the challenge of growing user numbers. They demand multiple data and high speeds for various applications, such as high-quality content streaming and the ability to work remotely effectively. Therefore, the challenge for MNOs is to ensure a reliable and high-quality experience for users on the mobile network [1,2]. The mobile broadband network’s quality of service (QoS) is improved by having quality control and network performance monitoring [3,4]. In addition, it impacts the quality perceived by the user (QoE). Therefore, an alternative to evaluating the quality of service (QoS) of Long-Term Evolution (LTE) mobile networks is through crowdsourcing measurements with end-user equipment (UE) [5,6,7].

On the other hand, the fifth generation 5G network is being implemented in two phases or two deployment approaches. The first approach is 5G Non-Standalone (NSA) or 5G NSA; it consists of deploying 5G on the 4G infrastructure. Therefore, the existing fourth-generation (4G) infrastructure is being used. The second approach is 5G standalone (SA) or 5G SA, whose Next-Generation Core Network (NGCN) network core is based on network software and the cloud; it is a unique development for 5G without the need to adapt legacy technologies [8,9]. In addition, according to the Global Mobile Suppliers Association (GSA), 94 mobile network operators in 48 countries worldwide had invested in 5G SA networks as of November 2021 [10,11]. Mexico has decided to start differently and implemented the first 5G NSA approach [9,12]. To date, no open data have been found on this network implemented in Mexico that allows the development of the QoS evaluation of the 5G network, so it has been decided to continue working with data from the 4G network.

Various research focus on presenting reviews and analyses of state-of-the-art networks to understand or solve the problems and challenges faced by mobile networks such as 5G or later. For example, the state-of-the-art coexistence between traffic in the enhanced mobile broadband (eMBB) and ultra-reliable low latency communication (URLLC) scenarios for resource scheduling has been investigated [13]. On the other hand, a review of transfer management in future ultra-dense mobile networks has been carried out, finding methodologies, such as dual connectivity (DC) and handover management algorithms, that have the potential to be used in improving data speed, coverage, and spectral efficiency [14]. Similarly, the analysis of the standardization activities in 3rd generation partnership program (3GPP) and open–radio access network (O-RAN) for artificial intelligence and machine learning (AI/ML) applications and the future technological trends of AI/ML in mobile systems was carried out, to improve the KPIs between the network and the UE [15].

Among the works that aim to improve mobile networks, a service classifier using ML based on supervised learning (SL) is proposed, making use of the KPIs and key quality indicators (KQIs) of the different services to improve the management of 5G and beyond 5G (B5G) networks [16]. Meanwhile, in [17], a hidden Markov model (HMM) is proposed to proactively predict cellular degradation using KPI real-time time series data. On the other hand, to optimize the handover control parameters (HCP) configurations based on the network experiences experienced by UEs in 5G network mobility, an artificial intelligence multiple linear regression algorithm (AI-MLR) is proposed [18]. To support the future implementation of large-scale 5G network performance in vehicular services, the design, development, and evaluation of four categories of 5G-enabled services/use cases in the Transport vertical were proposed [19]. Similarly, a mobile network simulator is presented with the realistic recreation of the dual-link enabled Hybrid Vehicular Network (HNV) in a simulator running with dual-link radio access technologies (RATs), in addition to proposing a parameterized data traffic steering algorithm [20]. Finally, proposing the use of multi-attribute decision-making based on fuzzy logic (MADM-FL), a scalable video coding—Dynamic adaptive streaming over HTTP (SVDC-DASH) method is presented over the vehicular environment based on the Multi-access Edge Computing (MEC) architecture [21]. These works are an excellent contribution to emerging mobile networks.

Regarding the QoS of mobile networks, several researchers have proposed various methodologies. First, a machine learning (ML) model is proposed that analyzes the vertical sector quality of service and predicts the degradation of the perceived quality (QoP) in the next 5 min [22]. Second, a cost-effective hybrid analytical model was proposed to obtain the user quality index (UQI) and know the quality perceived by the user of the eMBB scenario in the 5G SA network. ML and statistical tools were applied [9]. Third, to provide a better QoS, the cubic Hermit spline (CHS) method is applied, which allows knowing the impact of the KPIs on a key quality indicator (KQI) [23]. Finally, to know the characteristics of the QoS in the flow transfer between the Ethernet Time-Sensitive Networks (TSN) network exchange and 5G, a QoS mapping algorithm (QoS-MAN) is proposed [24]. Although these investigations are valuable for improving QoS, they focus on benefiting mobile network operators and not users.

Particularly for the LTE mobile network, different investigations have addressed the evaluation of the quality of service of the LTE mobile network, such as the study of graphical comparisons of the key performance indicators (KPI) with the signal parameters [25], as well as other works that aim to improve the quality of service and the quality of experience during video transmission by evaluating different configurations [26,27,28,29,30]. However, their main objective is to serve mobile network operators (MNOs), not users.

In addition, other works focus on optimizing LTE mobile networks using various computational models and simulation-based techniques. For example, improvements in downlink scheduling aim to optimize resource allocation using algorithms such as Knapsack and Priority-only [31]. The QoS-aware downlink scheduling algorithm (QuAs) improves the user-perceived quality of service (QoE) at the network edge [32]. Simulators are often used to assess network capacity and propose improvements in the scheduling process, which affects QoS and QoE, leading to increased spectral efficiency [33,34,35,36,37]. In handover scenarios, Sugeno-type fuzzy models have been applied to evaluate KPIs in LTE networks, with comparisons made against adaptive neuro-fuzzy inference systems (ANFIS) tailored for QoS [38,39]. The results of these simulations are often based on non-real-world data, which can lead to a discrepancy with real-world conditions [38,39].

It represents a significant challenge when designed with numerous inputs and membership functions in fuzzy inference systems [40]. The design of the FIS can be based on expert knowledge, data, or a hybrid using expert knowledge and data [41]. In particular cases, expert knowledge is applied when data are insufficient [42]. Models based on expert knowledge are a good candidate for evaluations of different specific domains and are interpretable for users (non-experts) [43].

On the other hand, the Analytic Hierarchy Process (AHP) is a multi-criteria decision-making method developed by Thomas L. Saaty [44]. It allows for interpreting levels of importance in mathematical weights that amplify or reduce the behavior of each proper parameter in a decision system [45]. In other words, AHP sections the problem into hierarchies of decision elements [46]. At each hierarchical level, the weights of the components are calculated. The final decision is made based on these weights [44]. The fuzzy inference system (FIS) uses the weights to hierarchize the fuzzy rules according to the importance given to each parameter [47]. Several investigations have applied the weighted fuzzy inference system to evaluate air quality, the environmental risk of artificial nanomaterials, water quality, beach tourism factors, and the safety of coastal sites, among other things [47,48,49,50,51,52].

Within mobile networks, a simple additive weighting (SASAW) vertical handover algorithm based on the signal-to-interference ratio (SINR) and AHP has been proposed, allowing the consideration of the QoS of different attributes according to the traffic [46]. Although a good performance is achieved for the network and the user, this work uses simulated data. On the other hand, a multiple handover parameter selection system has been proposed, using the AHP and FIS to determine when the handover is activated between an LTE and Worldwide Interoperability for Microwave Access (WiMAX) network [53]. The relevance of this work prioritizes the connectivity between heterogeneous networks, giving more importance to the base stations than to the users.

This paper proposes a new solution for evaluating the quality of service and the effectiveness of data transfer in the LTE mobile network. A FIS is proposed, taking into account that LTE network users mostly use over-the-top (OTT) applications [54]. Therefore, it is decided to assign levels of importance to the KPIs according to their relevance to the performance of these applications [55].

• In this FIS, the International Telecommunication Union (ITU)-recommended QoS KPIs are evaluated using a weighted fuzzy inference system, which will allow for the identification of desirable quality situations from the different combinations between the KPIs.
• The Analytic Hierarchy Process identifies the importance of each KPI in the form of a weight for each rule, increasing the efficiency of the evaluation.
• Finally, two indices are obtained for evaluating the QoS and the effectiveness of data transfer in the LTE network, in a range of [0–1] consistent with the QoE, Very good, Good, Acceptable, Poor, or Very Poor.

This work is organized as follows: Section 2 presents the Materials and Methods, followed by the Results in Section 3, and the discussion in Section 4. Finally, Section 5 presents the conclusions.

2. Materials and Methods

The proposed methodology is shown in the block diagram in Figure 1, inspired by [47,49]. In the first block, reference is made to the information repository collected through crowdsourcing with user equipment (UE). The KPIs recommended by the ITU for the evaluation of the QoS of the LTE network are shown: download speed (DS), upload speed (US), latency (L), jitter (J), packet loss rate (PLR), the reference received signal power (RSRP), and the reference received signal quality (RSRQ). Next is the fuzzification block, where each input’s membership functions and membership degrees are determined. Then, the inference rules block is observed, and the fuzzy rules are proposed based on expert knowledge. On the other hand, the analytical hierarchy process block proposes the importance of weighting for each input KPI, aiming to obtain the importance weights corresponding to the weight application block, which are to be applied to the fuzzy rules. Finally, in the last block, the QoS and Effectiveness indices are obtained through Aggregation and Defuzzification. Each of the blocks is detailed in Section 2.1, Section 2.2, Section 2.3, Section 2.4, Section 2.5 and Section 2.6.

Figure 1. Block diagram of weighted fuzzy inference system.

Figure 2 presents a complementary diagram to the block diagram in Figure 1, detailing the architecture of the weighted fuzzy inference system. The first column contains the information repository (IR), where the input variables (KPI) used to obtain the fuzzy QoS index are DS, US, L, J, RSRQ, and PLR. The input variables for the fuzzy effectiveness index are PLR and RSRP. The second column shows the fuzzification, where each input KPI’s functions and degrees of membership are determined, with three linguistic variables for each: Low, Medium, and High. Next, in the third column, the Weighted Inference Process is observed. Here, the fuzzy rules are proposed as follows: 433 to obtain the fuzzy QoS index and 9 for the effectiveness index. In addition, the weight vector or eigenvector corresponding to the input variables is obtained for each output index. For quality of service, important comparisons between DS, US, L, J, PLR, and RSRQ are considered, while for effectiveness, importance comparisons between PLR and RSRP are considered. Subsequently, the result of each fuzzy rule is multiplied by the weight of the KPI that determines that same rule (weight that applies). Finally, in the output values column, the quality of service and effectiveness indices are obtained with aggregation and defuzzification. The above is detailed in Section 2.1, Section 2.2, Section 2.3, Section 2.4, Section 2.5 and Section 2.6.

Figure 2. Architecture of the weighted fuzzy inference system for evaluating the LTE mobile network’s quality of service (QoS) and data transfer effectiveness.

2.1. Information Repository

The information repository used in this research is the same one used in [56]. The variables of interest are the seven key performance indicators recommended by the ITU for evaluating the quality of service of the LTE network: download speed, upload speed, latency, jitter, packet loss rate, RSRP, and RSRQ [55]. The repository’s data dictionary is in Table A1. The data samples were obtained collaboratively with mid-range user equipment in the Alameda Central area of Mexico City during the first two months of 2021.

The information repository analysis found that data samples from five mobile network operators are reported in 39 geolocated points of the study area. It is relevant to indicate that each geolocated point does not have data from each operator or measurements from each day of the collection period. This implies that the dataset of the information repository is not balanced. However, most data classification tools require balanced data or a balancing process. In the case of the FIS, the advantage is that when being designed, they can be based on expert knowledge and not on data. In addition, starting the FIS with expert knowledge is possible, and the parameters should be adjusted based on the available data, as was conducted in [43].

Considering that we propose to evaluate and not classify the KPIs to obtain the LTE mobile network’s service quality and effectiveness indices, the statistics of mean, variance, and standard deviation shown in Table 1 were obtained. It is essential to mention that for this calculation, from the total 607 samples of the information repository, filters were applied with the valid ranges for each KPI in Table 2 and Table 3, leaving 385 valid data samples. In this way, the characteristics of the input variables were determined. The statistics show that the variables are not linear and change over time; mathematically, these variables follow a stochastic process. Based on what was explained in the previous paragraph and this one, it was considered that this evaluation should be approached with a weighted fuzzy inference system, which is detailed in Section 2.2, Section 2.3, Section 2.4, Section 2.5 and Section 2.6.

Table 1. Key performance indicators dataset statistics.

KPI of QoS	Units	Mean	Variance	Standard Deviation
Download speed	Mbps	34.59	1006.35	31.72
Upload speed	Mbps	18.50	188.20	13.71
Latency	ms	29.40	246.83	15.71
Jitter	ms	8.05	23.34	4.83
Packet loss rate	%	180.06 × 10−6	1 × 10−6	1 × 10−3
RSRP	dBm	–96.68	130.89	11.4
RSRQ	dB	–11.07	8.98	2.99

Table 2. Classification of KPI ranges with linear units for evaluating the quality of service and effectiveness of the LTE network. Each range is defined by the valid limits lower ($Q_l$), upper ($Q_u$), and the optimal value represented by $Q_m$.

KPI	Classification	Ranges	Fuzzy Limits
			Ql	Qm	Qu
Download speed	Low	[0 to 21]	–	4	38
	Medium	[21 to 55]	4	38	72
	High	[55 to 300]	38	72	–
Upload speed	Low	[0 to 7.5]	–	1	14
	Medium	[7.5 to 20.5]	1	14	27
	High	[20.5 to 100]	14	27	–
Latency	Low	[0 to 25.5]	–	1	50
	Medium	[25.5 to 74.5]	1	50	99
	High	[74.5 to 100]	50	99	–
Jitter	Low	[0 to 7.5]	–	0	15
	Medium	[7.5 to 20.5]	0	15	30
	High	[20.5 to 100]	15	30	–
Packet loss rate	Low	[0 to 7.5]	–	0	0.005
	Medium	[7.5 to 20.5]	0	0.005	0.01
	High	[20.5 to 100]	0.005	0.01	–

Table 3. Classification of KPI ranges with nonlinear units for evaluating LTE service quality and network effectiveness. Each range is defined by $Q_{a_n}$ and $Q_{c_n}$ boundaries, which determine the slope and the crossover point value, respectively.

KPI	Classification	Ranges	Fuzzy Limits
			${\mathit{Q}}_{{\mathit{a}}_1}$	${\mathit{Q}}_{{\mathit{c}}_1}$	${\mathit{Q}}_{{\mathit{a}}_2}$	${\mathit{Q}}_{{\mathit{c}}_2}$
RSRP	Low	[–120 to –96]	–	–	–25	–101
	Medium	[–96 to –70]	25	–101	–25	–63
	High	[–70 to –44]	25	–63	–	–
RSRQ	Low	[–19.5 to –14]	–	–	–25	–15.375
	Medium	[–14 to –8.5]	25	–15.375	–25	–7.125
	High	[–8.5 to –3]	125	–7.125	–	–

On the other hand, the valid ranges for each key performance index with linear units of the LTE mobile network, download speed, upload speed, latency, jitter, and packet loss rate, are shown in Table 2. It is observed that the fuzzy limits $Q_l$ lower, $Q_u$ upper, and the optimal point $Q_m$. These are values with which the trapezoidal membership functions of these KPIs are determined. Meanwhile, sigmoid membership functions are considered for the KPIs with nonlinear units RSRP and RSRQ. Thus, the values $Q_{a_n},$ that determine the slope, and $Q_{c_n},$ the crossing point value found in Table 3, are essential fuzzy parameters for the sigmoid functions. The above is vital for the development of fuzzification in Section 2.2.

2.2. Fuzzification

In this section, the membership degrees of each input KPI are determined according to the input membership functions. In Section 2.1, two types of membership functions, trapezoidal and sigmoid, were assigned for KPIs with linear and nonlinear units, respectively. This is because linear membership functions (triangular and trapezoidal) are inappropriate in many practical situations. Instead, nonlinear membership functions may better reflect reality [57,58]. Three linguistic terms, Low, Medium, and High, were proposed for each KPI. Then, the boundaries of each term, $Q_l$ lower, $Q_m$ medium, and $Q_u$ upper, were assigned to the trapezoidal functions ${\mu }^k$, Equations (1)–(3), where $k$ represents the sample data of the KPIs. In Equation (1), the $k$ ranges are defined by the upper and lower boundaries, see Table 2.

$${\mu }_I^k=max\left\{min\left[{\displaystyle \frac{Q^k-Q_l^k}{Q_m^k-Q_l^k}},1,{\displaystyle \frac{Q_h^k-Q^k}{Q_h^k-Q_m^k}}\right],0\right\}$$

(1)

where $Q^k$ denotes the sample value of the KPI data, $l$ is the lowest allowed value, $u$ is the upper allowed value, and m is the central value. Meanwhile, Equation (2) occurs when the fuzzy function is defined with only the lower bound.

$${\mu }_I^k=max\left\{min\left[{\displaystyle \frac{Q^k-Q_l^k}{Q_m^k-Q_l^k}},1\right],0\right\}$$

(2)

On the other hand, when a range is defined only with the upper bound, the fuzzy function can be obtained as follows.

$${\mu }_I^k=max\left\{min\left[1,{\displaystyle \frac{Q_h^k-Q^k}{Q_h^k-Q_m^k}}\right],0\right\}$$

(3)

Equations (1)–(3) are applied to each KPI: download speed, upload speed, latency, jitter, and packet loss rate, using the three linguistic terms Low, Medium, and High.

Similarly, Equations (4)–(6) apply to the KPIs with non-line units, RSRP and RSRQ. The values defining these sigmoid membership functions ${\mu }^k$ are observed in Table 3. The same linguistic terms, Low, Medium, and High, are considered. Where the limits of each term are defined by $Q_{a_l}^k$ and $Q_{c_l}^k$ slope and lower crossing point, respectively. Also, $Q_{a_u}^k$ and $Q_{c_u}^k$ slope and upper crossing point are considered.

$${\mu }_I^k=max\left\{min\left[{\displaystyle \frac{1}{1-e^{-Q_{a_l}^k\left(Q^k-Q_{c_l}^k\right)}}},1,{\displaystyle \frac{1}{1-e^{-Q_{a_u}^k\left(Q^k-Q_{c_u}^k\right)}}}\right],0\right\}$$

(4)

where $c_l$ is the lowest crossing point, and $c_u$ is the highest. Meanwhile, in Equation (5), the fuzzy function is defined with the lower limit.

$${\mu }_I^k=max\left\{min\left[{\displaystyle \frac{1}{1-e^{-Q_{a_l}^k\left(Q^k-Q_{c_l}^k\right)}}},1\right],0\right\}$$

(5)

Finally, when the membership function is defined with the upper bound, it is denoted as:

$${\mu }_I^k=max\left\{min\left[1,{\displaystyle \frac{1}{1-e^{-Q_{a_2}^k\left(Q^k-Q_{c_2}^k\right)}}}\right],0\right\}$$

(6)

These membership functions allow for the definition of fuzzy rules, which are discussed in detail in Section 2.3.

2.3. Inference Rules

To process each rule, the use of fuzzy logic operations is required. A fuzzy operator is used to process the results of membership functions within an inference rule, see Equation (7) [59]. In this work, the fuzzy intersection operator (AND) has been used:

$${\mu }_{A\bigcap B}\left(x\right)=min\left\{{\mu }_A\left(x\right),{\mu }_B\left(x\right)\right\}$$

(7)

For evaluating the quality of service and the effectiveness of the LTE mobile network. Fuzzy logic-based systems are helpful due to the handling of subjectivity, which allows the interpretation of any knowledge and mapping them into several parallel evaluations. In the LTE mobile network data transfer, it is essential to identify all possible scenarios generated with the different key performance indicators that evaluate the QoS. In this sense, the combinations of the KPIs of the LTE network quality of service can be expressed with phrases commonly used by experts, such as, for example: if the download speed is high, the download speed is medium, the latency is high, the jitter is medium, and the packet loss rate is low, then the quality of service (QoS) is acceptable. This state can be expressed in fuzzy logic, as shown in the examples of the fuzzy rules in Table 4.

Table 4. Example of the fuzzy rules defined in the FIS according to the expert knowledge of the LTE network.

Fuzzy Rules
R1: If download speed is Low and upload speed is Low and latency is High and jitter is High and packet loss ratio is High then QoS is Very poor.
R18: If download speed is Low and upload speed is Low and latency is Medium and jitter is High and packet loss ratio is High and RSRQ is High then QoS is Poor.
R189: If download speed is Medium and upload speed is Low and latency is Low and jitter is Low and packet loss ratio is High then QoS is Acceptable.
R433: If download speed is High and upload speed is High and latency is Low and jitter is Low and packet loss ratio is Low then QoS is Very Good.
R435: If packet loss ratio is High and RSRP is Medium then Effectiveness is Poor.
R438: If packet loss ratio is Medium and RSRP is Medium then Effectiveness is Acceptable.
R441: If packet loss ratio is Low and RSRP is Medium then Effectiveness is Good.

For this work, the set of fuzzy rules covering all possible scenarios was demonstrated. Once the rules are defined, their mathematical calculation can be performed in two phases, taking as an example the rules for obtaining the effectiveness index.

First, all membership functions for the effectiveness index are evaluated with Equation (8).

$${\mu }_R=min\left\{{\mu }_{PLR}^k,{\mu }_{RSRP}^l\right\}$$

(8)

where $PLR$ and $RSRP$ are the packet loss rate and RSRP is the reference signal received power. Furthermore, $k$ and $l \in \{Low, Medium, High\}$ and ${\mu }_R$ is known as the rule’s antecedent. Finally, the consequent or output of the rule ${\mu }_{out}$ is obtained with Equation (9).

$${\mu }_{out}=min\left\{{\mu }_R,{\mu }_{Effectiveness}^l\right\}$$

(9)

where $l\in \{Very poor, Poor, Acceptable, Good, Very good\}$. The membership function of the QoS and effectiveness indices can be calculated using Equation (10).

$${\mu }_{out}\left(x\right)=max\left\{min\left\{{\displaystyle \frac{1}{1+e^{-a_1(x-c_1)}}},1,{\displaystyle \frac{1}{1+e^{a_2(x-c_2)}}}\right\},0\right\}$$

(10)

where $a_1$, $c_1$, $a_2$, and $c_2$ are the parameters defined by the output membership function; see Table 5 and Figure 3.

Table 5. Fuzzy function parameters are used to determine the FIS output.

QoS or Effectiveness	Parameter	Value
Very poor	$c$	−50
	$c_1=c_2$	0.125
Poor	$a_1={-a}_2$	50
	$c_1$	0.125
	$c_2$	0.375
Acceptable	$a_1={-a}_2$	50
	$c_1$	0.375
	$c_2$	0.625
Good	$a_1={-a}_2$	50
	$c_1$	0.625
	$c_2$	0.875
Very poor	$a_1=a_2$	50
	$c_1=c_2$	0.825

Figure 3. Membership functions, (a) Example of a sigmoid membership function, with its defining parameters. (b) Output membership functions used to obtain both the quality of service (QoS) and effectiveness (E) indices of the LTE mobile network.

2.4. Analytical Hierarchy Process

The Analytical Hierarchy Process (AHP), a multi-criteria decision-making tool, can solve a problem by decomposing related factors in a hierarchy and evaluating the priorities of the hierarchy in simple pairwise comparison judgments [60]. Thus, the fuzzy inference system (FIS) is robust using the AHP of L. Saaty in 1980 [61].

2.4.1. Hierarchy Assignment

After establishing the rules between KPIs, hierarchies are assigned by applying the Analytic Hierarchy Process (AHP). To verify the order of hierarchy between the seven key performance indicators (download speed, upload speed, latency, jitter, packet loss rate, RSRP, and RSRQ), the dependency links must be identified to establish these hierarchies. Therefore, the relative importance scale proposed by Saaty was used to select the level of importance from a comparison between KPIs. Table 6 shows the relative importance scale for obtaining the QoS and effectiveness indices.

Table 6. Interpret the relative importance scale for obtaining the service quality and effectiveness indices according to [44].

Scale	1	2	3	4	5	6	7	8	9
Importance	Equal	Weak	Moderate	Moderate+	Strong	Strong+	Very	Very+	Extreme
KPI to QoS *	Download Speed/Upload Speed	Packet Loss Rate	Jitter	Latency	–	–	–	–	RSRQ
KPI to E **	Packet Loss Rate	–	–	–	–	–	–	RSRP	–

* Key performance indicators are used to obtain the quality of service index. ** Key performance indicators are used to obtain the effectiveness index.

The level of importance assigned to each KPI focused on the applications that general LTE network users use daily. These users take advantage of the cost reduction in the mobile network service to access over-the-top (OTT) applications [54]. These are applications for video streaming, video calls, messaging, voice, audio, social networks, data transmission, cloud services, and browsers [55,62]. To evaluate the network quality aspects of these applications, the ITU in [55] recommends the relevance of quality parameters in the performance of these applications. The download and upload speed is most relevant, followed by the packet loss rate, jitter and latency, and RSRP and RSRQ. It is essential to evaluate the degradation based on these considerations.

Considering these services, the upload speed and download speed KPIs are determined to be paramount for QoS. Meanwhile, for the packet loss rate, the level of importance is weak compared to upload and download. Jitter is moderate, and latency is moderate +. Finally, RSRQ was assigned a much lower level of importance because this relationship is considered lower regarding upload and download.

The packet loss rate KPI was found to be most important for the effectiveness index due to its influence on evaluating data transfer effectiveness. However, the RSRP was assigned the lowest importance concerning the packet loss rate.

2.4.2. Pairwise Comparison Matrix

Once the importance has been assigned to the KPIs, pairwise comparisons between each KPI are performed. First, a pairwise matrix $(A=a_{ij})$ is used to store the comparisons of each index. $A$ is a positive $nxn$ reciprocal matrix and is constructed as follows:

$$A=\begin{array}{cc}& \begin{array}{ccc}{KPI}_1& {KPI}_2& \begin{array}{cc}\cdots & {KPI}_n\end{array}\end{array}\\ \begin{array}{c}{KPI}_1\\ {KPI}_2\\ \begin{array}{c}⋮\\ {KPI}_n\end{array}\end{array}& \left[\begin{array}{ccc}{a}_{11}& a_{12}& \begin{array}{cc}\cdots & a_{1n}\end{array}\\ a_{21}& a_{22}& \begin{array}{cc}\cdots & a_{2n}\end{array}\\ \begin{array}{c}⋮\\ a_{n1}\end{array}& \begin{array}{c}⋮\\ a_{n1}\end{array}& \begin{array}{cc}\begin{array}{c}\ddots \\ \cdots \end{array}& \begin{array}{c}⋮\\ a_{nn}\end{array}\end{array}\end{array}\right]\end{array}=\left[\begin{array}{ccc}{\displaystyle \raisebox{1ex}{$w_1$}\!\left/ \!\raisebox{-1ex}{$w_1$}\right.}& {\displaystyle \raisebox{1ex}{$w_1$}\!\left/ \!\raisebox{-1ex}{$w_2$}\right.}& \begin{array}{cc}\cdots & {\displaystyle \raisebox{1ex}{$w_1$}\!\left/ \!\raisebox{-1ex}{$w_n$}\right.}\end{array}\\ {\displaystyle \raisebox{1ex}{$w_2$}\!\left/ \!\raisebox{-1ex}{$w_1$}\right.}& {\displaystyle \raisebox{1ex}{$w_2$}\!\left/ \!\raisebox{-1ex}{$w_2$}\right.}& \begin{array}{cc}\cdots & {\displaystyle \raisebox{1ex}{$w_2$}\!\left/ \!\raisebox{-1ex}{$w_n$}\right.}\end{array}\\ \begin{array}{c}⋮\\ {\displaystyle \raisebox{1ex}{$w_n$}\!\left/ \!\raisebox{-1ex}{$w_1$}\right.}\end{array}& \begin{array}{c}⋮\\ {\displaystyle \raisebox{1ex}{$w_n$}\!\left/ \!\raisebox{-1ex}{$w_2$}\right.}\end{array}& \begin{array}{cc}\begin{array}{c}\ddots \\ \cdots \end{array}& \begin{array}{c}⋮\\ {\displaystyle \raisebox{1ex}{$w_n$}\!\left/ \!\raisebox{-1ex}{$w_n$}\right.}\end{array}\end{array}\end{array}\right]$$

(11)

where $w_i$ is the importance scale of the $i^{th}$ $KPI$, substituting the assigned values from Table 6 in Equations (11)–(13) for the $QoS$ and effectiveness indices, respectively.

$$A_{QoS}=\begin{array}{cc}& \begin{array}{ccc}\begin{array}{cc}DS& US\end{array}& \begin{array}{cc}L& J\end{array}& \begin{array}{cc}PLR& RSRQ\end{array}\end{array}\\ \begin{array}{c}\begin{array}{c}DS\\ US\end{array}\\ \begin{array}{c}L\\ J\end{array}\\ \begin{array}{c}PLR\\ RSRQ\end{array}\end{array}& \left[\begin{array}{cccccc}1& 1& 4& 3& 2& 9\\ 1& 1& 4& 3& 2& 9\\ 1/4& 1/4& 1& 3/4& 2/4& 9/4\\ 1/3& 1/3& 4/3& 1& 2/3& 9/3\\ 1/2& 1/2& 4/2& 3/2& 1& 9/2\\ 1/9& 1/9& 4/9& 3/9& 2/9& 1\end{array}\right]\end{array}$$

(12)

where $DS$ is the download rate, $US$ is the upload rate, $L$ is the latency, $J$ is the jitter, $PLR$ is the packet loss rate, and $RSRQ$ is the reference signal received quality. Similarly, the matrix for effectiveness is defined by:

$$A_{Effectiveness}=\begin{array}{cc}& \begin{array}{cc}PLR& RSRP\end{array}\\ \begin{array}{c}PLR\\ RSRP\end{array}& \left[\begin{array}{cc}1& 9\\ 1/9& 1\end{array}\right]\end{array}$$

(13)

where $PLR$ is the packet loss rate, and $RSRP$ is the reference signal received power.

2.4.3. Square Matrix of A

The square of the pairwise comparison matrix is then calculated using the expression in Equation (14).

$$B=A\times A=\left[\begin{array}{ccc}{a}_{11}& a_{12}& \begin{array}{cc}\cdots & a_{1n}\end{array}\\ a_{21}& a_{22}& \begin{array}{cc}\cdots & a_{2n}\end{array}\\ \begin{array}{c}⋮\\ a_{n1}\end{array}& \begin{array}{c}⋮\\ a_{n1}\end{array}& \begin{array}{cc}\begin{array}{c}\ddots \\ \cdots \end{array}& \begin{array}{c}⋮\\ a_{nn}\end{array}\end{array}\end{array}\right]\times \left[\begin{array}{ccc}{a}_{11}& a_{12}& \begin{array}{cc}\cdots & a_{1n}\end{array}\\ a_{21}& a_{22}& \begin{array}{cc}\cdots & a_{2n}\end{array}\\ \begin{array}{c}⋮\\ a_{n1}\end{array}& \begin{array}{c}⋮\\ a_{n1}\end{array}& \begin{array}{cc}\begin{array}{c}\ddots \\ \cdots \end{array}& \begin{array}{c}⋮\\ a_{nn}\end{array}\end{array}\end{array}\right]$$

(14)

Substituting values from Equation (12) into Equation (14) gives the square of the matrix $A$ for the $QoS$.

$$B_{Qos}=\left[\begin{array}{cccccc}6& 6& 24& 18& 12& 54\\ 6& 6& 24& 18& 12& 54\\ 1.5& 1.5& 6& 4.5& 3& 13.5\\ 2& 2& 8& 6& 4& 18\\ 3& 3& 12& 9& 6& 27\\ 0.667& 0.667& 2.667& 2& 1.3333& 6\end{array}\right]$$

(15)

The square of the effectiveness matrix $A$ is calculated similarly.

$$B_{Effectiveness}=\left[\begin{array}{cc}2& 18\\ 0.2222& 2\end{array}\right]$$

(16)

2.4.4. Obtaining the Vector C

Next, vector C is obtained, which will be used to obtain the weight vector. To do this, the rows of matrix B are added using Equation (17) to calculate each element of vector $C$.

$$C_i=\sum _{j=1}^n{B}_{ij}\forall i=\mathrm{1,2},\dots ,n$$

(17)

Using Equation (17), vector $C$ for the quality of service is calculated.

$$C_{QoS}=\left[\begin{array}{c}\begin{array}{c}120\\ 120\end{array}\\ \begin{array}{c}30\\ 40\end{array}\\ \begin{array}{c}60\\ 13.3333\end{array}\end{array}\right]$$

For effectiveness, the corresponding steps are taken.

$$C_{Effectiveness}=\left[\begin{array}{c}20\\ 2.2222\end{array}\right]$$

2.4.5. Weight Vector or Eigenvector

After obtaining the weight vector, the vector $C_i$ must be normalized by applying Equation (18).

$$w_i={\displaystyle \frac{C_i}{{\sum }_{j=1}^n{C}_j}}\forall i=\mathrm{1,2},\dots ,n$$

(18)

where $w_i$ is the weight vector, known as the eigenvector, proposed by Perron in [63]. This process is repeated until the eigenvector solution is unchanged from the previous iteration. The priority weights for each KPI concerning $QoS$ and effectiveness are shown in Equations (19)–(20), respectively.

$$w_{QoS}=\left[\begin{array}{c}\begin{array}{c}0.3130\\ 0.3130\end{array}\\ \begin{array}{c}0.0783\\ 0.1043\end{array}\\ \begin{array}{c}0.1565\\ 0.0348\end{array}\end{array}\right]$$

(19)

$$w_{Effectiveness}=\left[\begin{array}{c}0.9\\ 0.1\end{array}\right]$$

(20)

2.4.6. Iteration

The iteration starts by repeating the process described in Section 2.4.3, Section 2.4.4 and Section 2.4.5. It ends when there are changes of less than four decimal places between the elements of the eigenvector of the current iteration and those of the eigenvector of the previous iteration. The matrix $B_i$ is initially obtained by calculating the square matrix of $B_{i-1}$, obtained in the last iteration.

$$B_{1\_QoS}=B_{QoS}\times B_{QoS}=\left[\begin{array}{cccccc}216& 216& 864& 648& 432& 1944\\ 216& 216& 864& 648& 432& 1944\\ 54& 54& 216& 162& 108& 486\\ 72& 72& 288& 216& 144& 648\\ 108& 108& 432& 324& 216& 972\\ 24& 24& 96& 72& 48& 216\end{array}\right]$$

$$B_{1\_Effectiveness}=B_{Effectiveness}\times B_{Effectivveness}=\left[\begin{array}{cc}8& 72\\ 0.8889& 8\end{array}\right]$$

Then, the vector $C_i$ of each index is obtained:

$$C_{1\_QoS}=\left[\begin{array}{c}\begin{array}{c}4320\\ 4320\end{array}\\ \begin{array}{c}1080\\ 1440\end{array}\\ \begin{array}{c}2160\\ 480\end{array}\end{array}\right]$$

$$C_{1\_Effectiveness}=\left[\begin{array}{c}80\\ 8.8889\end{array}\right]$$

Next step is to calculate the weight vector $w_i$.

$$w_{1\_QoS}=\left[\begin{array}{c}\begin{array}{c}0.3130\\ 0.3130\end{array}\\ \begin{array}{c}0.0783\\ 0.1043\end{array}\\ \begin{array}{c}0.1565\\ 0.0348\end{array}\end{array}\right]$$

$$w_{1\_Effectiveness}=\left[\begin{array}{c}0.9\\ 0.1\end{array}\right]$$

Finally, the difference between the current iteration’s eigenvector and the previous iteration’s eigenvector is obtained. The difference in eigenvectors for the $QoS$ is calculated:

$$\left[\begin{array}{c}\begin{array}{c}0.3130\\ 0.3130\end{array}\\ \begin{array}{c}0.0783\\ 0.1043\end{array}\\ \begin{array}{c}0.1565\\ 0.0348\end{array}\end{array}\right]-\left[\begin{array}{c}\begin{array}{c}0.3130\\ 0.3130\end{array}\\ \begin{array}{c}0.0783\\ 0.1043\end{array}\\ \begin{array}{c}0.1565\\ 0.0348\end{array}\end{array}\right]=\left[\begin{array}{c}\begin{array}{c}0\\ 0\end{array}\\ \begin{array}{c}0\\ 0.1388\times {10}^{-16}\end{array}\\ \begin{array}{c}0\\ 0\end{array}\end{array}\right]$$

Meanwhile, for effectiveness:

$$\left[\begin{array}{c}0.9\\ 0.1\end{array}\right]-\left[\begin{array}{c}0.9\\ 0.1\end{array}\right]=\left[\begin{array}{c}0\\ 0\end{array}\right]$$

when the result for both eigenvectors is less than four decimal places, the iteration stops, and the process continues to Section 2.4.7.

2.4.7. Consistency

Finally, the Consistency Ratio ($CR$) of the matrix “$A$” was calculated. According to Saaty [44], a limit of 0.1 is defined as the maximum acceptable inconsistency. Equation (21) is used to calculate $CR$.

$$CR={\displaystyle \frac{{\mathsf{\lambda }}_{max}-n}{\left(n-1\right)RI}}$$

(21)

where ${\mathsf{\lambda }}_{max}$ is the maximum eigenvalue of the comparison pair matrix, $n$ is the matrix’s size, and $RI$ is the random consistency index, which can be determined according to Table 7.

Table 7. Random consistency index [44].

Matrix Size (n)	1	2	3	4	5	6	7	8	9	10
$RI$	0	0	0.52	0.89	1.11	1.25	1.35	1.4	1.45	1.49

If the ratio (called the consistency ratio $CR$) between the consistency index and that of the random matrices is significantly tiny (it is carefully specified that it should be 10% or less), we accept the estimate of $w$. Otherwise, we try to improve the consistency.

The Equation (22) is applied to obtain the ${\mathsf{\lambda }}_{max}$.

$${(\mathsf{\lambda }}_{max})w=Aw$$

(22)

where $A$ is the pairwise comparison matrix proposed in Section 2.4.2, ${\mathsf{\lambda }}_{max}$ is the maximum eigenvalue of the matrix, and $w$ is the eigenvector or vector of resulting weights obtained at the end of the iteration of Section 2.4.6. In addition, the equality shown in Equation (23) is fulfilled.

$$Aw=\left[\begin{array}{ccc}1& a_{12}& \begin{array}{cc}\cdots & a_{1n}\end{array}\\ 1/a_{12}& 1& \begin{array}{cc}\cdots & a_{2n}\end{array}\\ \begin{array}{c}⋮\\ 1/a_{1n}\end{array}& \begin{array}{c}⋮\\ 1/a_{2n}\end{array}& \begin{array}{cc}\begin{array}{c}\ddots \\ \cdots \end{array}& \begin{array}{c}⋮\\ 1\end{array}\end{array}\end{array}\right]\left[\begin{array}{c}\begin{array}{c}{w}_1\\ w_2\end{array}\\ \begin{array}{c}⋮\\ w_n\end{array}\end{array}\right]=cw$$

(23)

It is verified whether the matrix $A$ for the $QoS$ is consistent by applying Equation (23).

$$A_{QoS}{w}_{QoS}=\left[\begin{array}{cccccc}1& 1& 4& 3& 2& 9\\ 1& 1& 4& 3& 2& 9\\ 1/4& 1/4& 1& 3/4& 2/4& 9/4\\ 1/3& 1/3& 4/3& 1& 2/3& 9/3\\ 1/2& 1/2& 4/2& 3/2& 1& 9/2\\ 1/9& 1/9& 4/9& 3/9& 2/9& 1\end{array}\right]\times \left[\begin{array}{c}\begin{array}{c}0.3130\\ 0.3130\end{array}\\ \begin{array}{c}0.0783\\ 0.1043\end{array}\\ \begin{array}{c}0.1565\\ 0.0348\end{array}\end{array}\right]=\left[\begin{array}{c}\begin{array}{c}1.8783\\ 1.8783\end{array}\\ \begin{array}{c}0.4696\\ 0.6261\end{array}\\ \begin{array}{c}0.9391\\ 0.2087\end{array}\end{array}\right]$$

$$\therefore c_{QoS}{w}_{QoS}=\left[\begin{array}{c}\begin{array}{c}1.8783\\ 1.8783\end{array}\\ \begin{array}{c}0.4696\\ 0.6261\end{array}\\ \begin{array}{c}0.9391\\ 0.2087\end{array}\end{array}\right]$$

Furthermore, by relating Equations (22)–(23), it is observed that there is equality between $c$ and ${\mathsf{\lambda }}_{max}$; therefore:

$$c={\mathsf{\lambda }}_{max}={\sum }_{j=1}^n{w}_j$$

(24)

Calculating ${\mathsf{\lambda }}_{max}$ for the quality of service we have:

$${\mathsf{\lambda }}_{\mathrm{m}\mathrm{a}\mathrm{x}\_QoS}=6$$

To verify that the correct value of ${\mathsf{\lambda }}_{max}$ is obtained, Equation (25) is applied.

$$c{\mathsf{\lambda }}_{max}w={\mathsf{\lambda }}_{max}\left[\begin{array}{c}\begin{array}{c}{w}_1\\ w_2\end{array}\\ \begin{array}{c}⋮\\ w_n\end{array}\end{array}\right]$$

(25)

$${\mathsf{\lambda }}_{\mathrm{m}\mathrm{a}\mathrm{x}\_QoS}{w}_{QoS}=6\times \left[\begin{array}{c}\begin{array}{c}0.3130\\ 0.3130\end{array}\\ \begin{array}{c}0.0783\\ 0.1043\end{array}\\ \begin{array}{c}0.1565\\ 0.0348\end{array}\end{array}\right]=\left[\begin{array}{c}\begin{array}{c}1.8783\\ 1.8783\end{array}\\ \begin{array}{c}0.4696\\ 0.6261\end{array}\\ \begin{array}{c}0.9391\\ 0.2087\end{array}\end{array}\right]$$

It is observed that the result is the identical resulting vector of $A_{QoS}{w}_{QoS}$; therefore, ${\mathsf{\lambda }}_{max}$ can be used to calculate the consistency ratio for quality. The consistency ratio is calculated by substituting values.

$${CR}_{QoS}={\displaystyle \frac{6-6}{\left(6-1\right)1.25}}=0$$

It is verified that ${CR}_{QoS}<0.1$; therefore, the matrix $A_{QoS}$ is consistent. For effectiveness, the matrix $A_{Effectiveness}$ is of dimension $2\times 2$. Thus, the consistency is verified by demonstrating that ${\mathsf{\lambda }}_{max}=2$ for each positive reciprocal matrix of 2 by 2. Therefore, each positive reciprocal matrix of 2 by 2 is a positive reciprocal matrix, as shown in Equation (26).

$$\left[\begin{array}{cc}1& \alpha \\ {\alpha }^{-1}& 1\end{array}\right]\left[\begin{array}{c}1+\alpha \\ \left(1+\alpha \right){\alpha }^{-1}\end{array}\right]=2\left[\begin{array}{c}1+\alpha \\ \left(1+\alpha \right){\alpha }^{-1}\end{array}\right]$$

(26)

As the proof is fulfilled, it is verified that the matrix proposal is consistent. Therefore, the weight vector for the $QoS$ is:

$$w_{QoS}=\begin{array}{cc}\begin{array}{c}\begin{array}{c}DownloadSpeed\\ UploadSpeed\end{array}\\ \begin{array}{c}Latency\\ Jitter\end{array}\\ \begin{array}{c}PacketLossRate\\ RSRQ\end{array}\end{array}& \left[\begin{array}{c}\begin{array}{c}0.3130\\ 0.3130\end{array}\\ \begin{array}{c}0.0783\\ 0.1043\end{array}\\ \begin{array}{c}0.1565\\ 0.0348\end{array}\end{array}\right]\end{array}$$

(27)

Furthermore, the weight vector for effectiveness is:

$$w_{Effectiveness}=\begin{array}{cc}\begin{array}{c}PacketLossRate\\ RSRP\end{array}& \left[\begin{array}{c}0.9\\ 0.1\end{array}\right]\end{array}$$

(28)

2.5. Weight Applying

The FIS works by giving equal importance to the input variables, in this case, to the KPIs. However, the significance of some KPIs can be weighted to benefit the response of the indices. In this sense, some KPIs must have a different mathematical importance for this hierarchy to impact each index. The weights of these variables were obtained in Section 2.4. It is essential to mention that the assignment of these weights cannot be integrated separately into the rules since they would unbalance the calculation of the output rule; however, when the result of a function defined for a KPI establishes the result of the rule, then the weight corresponding to the KPI of this function is multiplied to the result according to the following expression.

$${\mu }_w=w_i\times {\mu }_{R\_out}$$

(29)

where $w$ is the selected weight of the $i^{th}$ KPI that defined ${\mu }_w$. When two or more KPIs have the same membership value, the KPI with the highest priority establishes the weight to be used. Particularly for $QoS$, the rules are defined with six KPIs: download speed ($DS$), upload speed ($US$), latency ($L$), jitter ($J$), packet loss rate ($PLR$), and $RSRQ$. Thus, the weight to be used is defined with Equation (30).

$$w_{QoS}=\mathrm{m}\mathrm{a}\mathrm{x}\{w_{DS},w_{US},w_L,w_J,w_{PLR},w_{RSRQ}\}$$

(30)

Similarly, the effectiveness index rules are generated with two $KPIs$, the $PLR$ and the $RSRP$. Therefore, the weight to be used is assigned using Equation (31).

$$w_{Effectiveness}=\mathrm{m}\mathrm{a}\mathrm{x}\{w_{PLR},w_{RSRP}\}$$

(31)

Figure 4 shows how the $KPI$ influences the exit rule with the highest importance, exemplifying the process of obtaining the effectiveness index.

Figure 4. Example of the operation of a fuzzy rule to obtain the effectiveness index. The assignment of weights $w_{PLR}$ and $w_{RSRP}$ is chosen according to the membership function determining the output rule. In this case, the packet loss rate membership function and $w_{PLR}$ define the maximum output rule.

2.6. QoS and Effectiveness Indices (Aggregation and Defuzzification)

After the output rules (${\mu }_{out}$) have been calculated, the area of all these rules is combined by superposition, known as the aggregation process. Thus, a final membership function (${\mu }_g$) is obtained that integrates all the particular evaluations (${\mu }_{out}$). The aggregation for the QoS index is presented as an illustrative example (see Figure 5).

Figure 5. Fuzzy inference architecture for 4G LTE network QoS assessment using rules 240 and 241. The output rule membership values are used to truncate the QoS membership function. Then, all the truncated functions (${\mu }_{QoS}$) are combined to create a final membership function (${\mu }_g$). Subsequently, the centroid method ($CM$) is used to calculate the output score.

3. Results

3.1. KPI Analysis

It is essential to analyze the LTE network’s KPIs to understand their behavior in terms of quality of service and effectiveness. Each KPI was analyzed using the corresponding fuzzy functions based on Table 1 and Table 2. Three fuzzy states were determined for each KPI: Low, Medium, and High. It considered the repository of information collected collaboratively through user teams in the first two months of 2021 in the Alameda Central area of Mexico City.

The data samples from the download speed, upload speed, latency, jitter, packet loss rate, RSRP, and RSRQ information repository are shown in Figure 6a–g, respectively. Figure 6a shows that the download speed presents 42% of the data samples between the ranges of 21 and 55 Mbps; that is, it is classified as Medium speed, leading to an Acceptable quality of service. On the other hand, 40% is between the range of 0 and 21 Mbps, which makes it susceptible to having mostly a Poor or Very Poor quality of service. The remaining 18% is between the limits of 55 and 300 Mbps, which have a Good or Very Good QoS.

Figure 6. Analysis of each KPI of the repository of information collected with user equipment, (a) download speed, (b) upload speed, (c) latency, (d) jitter, (e) packet loss rate, (f) RSRP, and (g) RSRQ.

In Figure 6b, the download speed shows 39% of data within the range of 20.5 to 100 Mbps, meaning the QoS is Good or Very Good. In addition, 34% of the samples are in the 7.5 to 20.5 Mbps range, which results in an Acceptable QoS. The remaining 27% range is from 0 to 7.5 Mbps, generating a Poor or Very Poor QoS. On the other hand, Figure 6e shows the rate of lost packets where 96% of the data is in the range of 0 to 0.25%, which benefits from having good results in both indices, but especially in Effectiveness since the weighting for this index is very high, presenting 3% in the range 0.25 to 0.75 and 1% within 0.75 to 1, which limits an unfavorable response for both indices.

On the other hand, in Figure 6d, it can be observed that jitter accounts for 55% of the data samples in the range of 0 to 7.5 ms, benefiting from a good evaluation of the quality of service. Meanwhile, 44% is within the limits of 7.25 to 22.5 ms, promoting an acceptable QoS, and the remaining 2% is in the range of 22.5 to 30 ms, generating a poor QoS evaluation. On the other hand, in Figure 6c, latency accounts for 52% of data in the range of 0 to 25.5 ms, which promotes a better QoS, while 45% of the data is between 25.5 and 74.5 ms, impacting an acceptable quality. Finally, the remaining 3% is 74.5 to 100 ms, causing a poor QoS.

On the other hand, in Figure 6f of the RSRP, it is observed that 60% of the data is in the lowest range, while 38% is in the middle range, and 2% is in the highest, −120 to −96 dBm, −96 to −70 dBm, and −70 to −44 dBm, respectively. This affects the effectiveness index to a lesser extent by the assigned weighting. Finally, Figure 6g presents the RSRQ with 57% in the range of −14 to −8.5 dB, contributing to having an acceptable QoS. In comparison, 22% presents a better level in the range of −8.5 to −3 dB, contributing to having an enhanced evaluation of the QoS; on the other hand, the remaining 21% is in the lowest level of −19.5 to −14 dB, affected by having a poor quality of service of the LTE network.

3.2. Quality of Service and Effectiveness Index

Previously, the evaluation of the QoS and the effectiveness of the data transfer of the LTE network was carried out with an FIS in [56] using seven KPIs: download speed, upload speed, latency, jitter, packet loss rate, RSRP, and RSRQ. On the other hand, this work proposes strengthening the FIS by adding the AHP (FIS-AHP). Although both works propose a tool for evaluating QoS and effectiveness, which general users of the LTE network can interpret, both propose their design using the 385 samples of the information repository, considering the same valid ranges of Table 2 and Table 3, as well as the functions and membership degrees for each KPI. The differences lie in that the FIS model of [56] considered the same level of importance in five KPIs: download speed, upload speed, latency, jitter, and packet loss rate, but did not consider RSRP and RSRQ KPIs as the same levels of importance. This is because they evaluate the LTE network’s transmission as medium (RF); therefore, RSRP and RSRQ were assigned a much lower importance than the rest of the KPIs.

The proposal for the weighted FIS-AHP considered different levels of importance for the KPIs, considering that general users of the LTE network mostly access over-the-top (OTT) applications daily. In this way, the ITU recommendation [55] for quality assessment in OTT applications was considered. It mentions the relevance of the KPIs in these applications. The most relevant are the download and upload speeds, followed by the packet loss rate, jitter, latency, and RSRP and RSRQ. In addition, 372 rules were proposed in the FIS, and 441 were proposed in the FIS-AHP.

The results of both evaluations for the quality of service are compared in Figure 7. It is observed that both quality of service indices applying the Fuzzy System Inference (QoS-FIS) and the Analytic Hierarchy Process (QoS-AHP) have similar behavior. However, due to the importance assigned to the upload speed and download speed KPIs in the QoS-AHP index, there are differences concerning the QoS FIS index. Table 8 summarizes these differences and the minimum and maximum values obtained for both indices.

Figure 7. Comparison between the evaluations obtained from QoS-FIS and the proposed QoS-AHP.

Table 8. Numerical comparison between LTE network quality of service indices.

Key Performance Indicators							Quality of Service
No.	Download Speed	Upload Speed	Latency	Jitter	Packet Loss Rate	RSRQ	QoS-FIS	QoS-AHP	Difference QoS
	(Mbps)	(Mbps)	(ms)	(ms)	×102 (%)	(dB)	×102 (%)	×102 (%)	×102 (%)
1	21.92	26.436	31	9.2	0	−8	0.69075442	0.68954949	0.0012049
2	67.943	27.148	32	7.8	0	−8	0.83907463	0.83940658	–0.0003319
17	42.258	18.604	53	7.1	0	−6	0.68695612	0.78890377	–0.1019476
49	66.157	38.405	16	3.3	0	−15	0.89710123	0.79683236	0.1002688
63	70.729	47.714	12	1.9	0	−7	0.93022936	0.9215899	0.0086394
64	73.728	40.81	14	4.4	0	−7	0.88947785	0.88242199	0.0070558
99	71.401	38.848	22	5	0	−5	0.86634704	0.96546328	–0.0991162
116	15.578	13.683	14	2.3	0	−16	0.6872645	0.58977728	0.0974872
228	7.2	9.535	15	3.1	0	−18	0.60138247	0.46628529	0.1350971
303	16.239	2.307	42	13.1	0	−12	0.54413438	0.54499786	–0.0008634
304	4.474	4.241	21	19.3	0.003597122	−17	0.33756652	0.33931445	0.0419891
305	74.017	45.95	18	3.3	0	−9	0.89356243	0.90416858	–0.0106061
365	70.889	36.202	16	6.5	0	−3	0.86994519	0.96871253	–0.098767
378	26.193	4.579	17	8.7	0	−12	0.6169446	0.62113169	–0.004187
379	56.694	27	21	15.1	0	−13	0.84751376	0.838701	0.008812
380	22.186	9.66	19	13.9	0	−9	0.65859332	0.66158947	–0.002996

The weighting of the QoS-AHP index amplifies the effect of the most representative KPIs. Generally, the quality of service index obtained with AHP averaged 67%, one percentage point higher than the average obtained with the QoS-FIS index.

On the other hand, Figure 7 shows in detail the data sample 228, where the QoS-AHP evaluation presents an improvement of 13.5% concerning the QoS-FIS due to weighting in the AHP. Sample 228 has the following input linguistic values about each KPI: download speed value, upload speed, latency, jitter, packet loss rate, and RSRQ of Low, Medium, Low, Low, Low, and Low, respectively, which translates to obtaining a Good classification in the FIS evaluation. The above is why the values of the latency, jitter, and packet loss rate KPIs benefit from good quality. Meanwhile, for the AHP, an Acceptable classification is obtained due to the importance assigned to the upload and download speed KPIs, penalizing the evaluation result more. In addition, the difference between the highest peak value of each index is 3.84%, where the QoS-AHP has the highest score. Meanwhile, the difference between the lowest peak value of each index is 0.17%, where the QoS-FIS also has the lowest score, see Table 8.

In the same way, the effectiveness indices evaluated with the fuzzy inference system (E-FIS) and the analytical hierarchy process (E-AHP) are compared. Figure 8 shows both indexes, which have similar behavior with a slight difference due to the importance of the packet loss rate KPI rules. The average obtained by the E-AHP is higher than that of the E-QoS by 1%.

Figure 8. Comparison between the evaluations obtained from Effectiveness-FIS and the proposed Effectiveness-AHP.

Also, the evaluation of the effectiveness of sample 84 can be seen, where the linguistic values of packet loss rate are High, and that of the RSRP is Medium. This implies that both evaluation systems should obtain a result classified in the Very Poor category. However, the difference between both evaluations is 9%; this is due to the critical weighting assigned to the packet loss rate in the AHP because this KPI strongly impacts the effectiveness of the data transfer of the LET network. Table 9 summarizes the results of the comparison between both effectiveness indices. On the other hand, the difference between the high peak value of both indices is 2.26%, where the E-AHP index is higher.

Table 9. Numerical comparison between indices of LTE network effectiveness.

Key Performance Indicators			Effectiveness
No.	Packet Loss Rate	RSRP	Effectiveness-FIS	Effectiveness-AHP	Difference Effectiveness
	×102 (%)	(mdB)	×102 (%)	×102 (%)	×102 (%)
1	0	−104	0.81553327	0.8124495	0.00308377
2	0	−104	0.81553327	0.8124495	0.00308377
6	0	−95	0.91944493	0.94246453	−0.0230196
84	0.009950249	−95	0.01934556	0.10958865	−0.09024309
157	0	−81	0.99994155	0.99897333	0.00096821
216	0	−81	0.99994155	0.99897333	0.00096821
217	0	−81	0.99994155	0.99897333	0.00096821
222	0	−77	0.99975917	0.99954272	0.00021645
254	0	−77	0.99975917	0.99954272	0.00021645
313	0.006711409	−110	0.21242474	0.20021829	0.01220645
363	0.01	−75	0.02058827	0.03952786	−0.01893959
372	0	−77	0.99975917	0.99954272	0.00021645

4. Discussion

The information repository was collected collaboratively with mid-range user equipment during the first two months of 2021. Data from five mobile network operators are reported at 39 geolocated points. It is essential to clarify that each geolocated point does not have data from all operators or measurements from all collection period days. In addition, after statistically analyzing the data, it was found that the KPI values are variants over time, so the behavior of these variables refers to a stochastic process. Therefore, it will be determined that the data in the repository are not balanced. Subsequently, the information repository was analyzed regarding the range of valid values for each KPI, as shown in Table 1, and 385 valid data were found. Therefore, to evaluate the quality of service and effectiveness, the weighted fuzzy inference system (FIS-AHP) is used to strengthen the first FIS carried out [38].

The relevance of this model is the ability to offer LTE mobile network users a pair of indices that allows them to assess the quality of service and effectiveness without being experts. In other words, an additional tool for assessment is proposed that does not replace the KPIs recommended by regulators. Particularly for QoS, the weighting assigned to the download speed and upload speed KPIs in the FIS-AHP is because this pair is decisive in the data transfer of over-the-top services. Similarly, importance was assigned to the KPI of the packet loss rate for effectiveness, considering that this KPI is essential in evaluating the effectiveness of data transfer.

To discuss the results, an example is presented in Figure 9. It can be observed that the evaluation coefficient of the absolute values (ρ) in (a) and (b) is positive. In addition, in both sections, there is a strong relationship between the evaluation models due to the coincidence in the behavior of the input KPIs of the evaluation with the FIS and the importance weights assigned in each AHP. The differences in Figure 9a between the QoS-FIS and the QoS-AHP can be seen in different data samples. This is caused because, in the AHP evaluation, the importance weights assigned to the upload and download speed amplify or attenuate the behavior of the quality of service concerning the FIS. In the AHP evaluation, the importance levels assigned to each KPI were based on the relevance of the quality parameters in the performance of OTT applications recommended in the ITU Quality of Service Regulator Manual. Download and upload speed are the most relevant, followed by packet loss rate, jitter, latency, RSRP, and RSRQ.

Figure 9. Correlation between index (a) assessment between the quality of service index obtained with the proposed Analytic Hierarchy Process (QoS-AHP) and the quality of service index obtained with the fuzzy inference system (QoS-FIS). (b) estimation between the effectiveness index obtained with the proposed Analytic Hierarchy Process (E-AHP) and the effectiveness index obtained with the fuzzy inference system (E-FIS).

In other words, if the data samples from the information repository have values in the ranges of 55 to 300 Mbps and 20.5 to 100 Mbps for download and upload speed, respectively, or in linguistic terms, they have a high value for both, they will influence more than the rest of the KPIs to amplify the result of the evaluation. On the contrary, if the data samples have low linguistic values or values in the range of 0 to 21 Mbps for the upload speed or 0 to 7.5 Mbps for the download speed, they will weaken the result of the Quality of Service Index.

Meanwhile, Figure 9b shows a more significant difference between the E-FIS and E-AHP models in the effectiveness evaluations with values of less than 1% because the influence of the weighting on the packet loss rate is more critical than RSRQ’s. In this case, the AHP evaluation is amplified. The rest of the values compared between E-FiS and EAHP present an average difference of 1% due to the similarity between both evaluation models.

5. Conclusions

In this work, a new computational model was proposed to evaluate the quality of service and the effectiveness of data transfer in the LTE mobile network, using the fFuzzy inference system (FIS) and an Analytic Hierarchy Process. The KPIs of download speed, upload speed, latency, jitter, packet loss rate, RSRP, and RSRQ were evaluated to provide an extra tool to users and regulators of mobile networks. Different works have been developed around the evaluation of QoS, but the importance of developing specialized models depends on the types of data to be analyzed; in this case, data are collaboratively collected by user equipment. Firstly, telecommunications experts carry out QoS evaluations, and users only have a subjective perception of this quality of service. On the other hand, providing an additional tool to users or regulators will allow them to identify undesirable quality situations based on the different values of the KPIs. It is essential to mention that the proposed model can be adapted to other information repositories collected with specialized equipment, not necessarily with user equipment or other more advanced generation mobile networks such as 5G or sixth generation (6G). Therefore, for future work, it is considered that the computational model should be adapted with other information repositories from other geographical areas, in addition to integrating other artificial intelligence techniques such as Adaptive Neuro-fuzzy Inference Systems (ANFIS), multilayer perceptron networks (MLP), recurrent networks (RNN), or generative adversarial networks (GANs) to evaluate the eight KPIs and five key quality indicators (KQI) in the scenario of enhanced mobile broadband use of the 5G network.