Dynamic Regressor/Ensemble Selection for a Multi-Frequency and Multi-Environment Path Loss Prediction

Abstract

Wireless network parameters such as transmitting power, antenna height, and cell radius are determined based on predicted path loss. The prediction is carried out using empirical or deterministic models. Deterministic models provide accurate predictions but are slow due to their computational complexity, and they require detailed environmental descriptions. While empirical models are less accurate, Machine Learning (ML) models provide fast predictions with accuracies comparable to that of deterministic models. Most Empirical models are versatile as they are valid for various values of frequencies, antenna heights, and sometimes environments, whereas most ML models are not. Therefore, developing a versatile ML model that will surpass empirical model accuracy entails collecting data from various scenarios with different environments and network parameters and using the data to develop the model. Combining datasets of different sizes could lead to lopsidedness in accuracy such that the model accuracy for a particular scenario is low due to data imbalance. This is because model accuracy varies at certain regions of the dataset and such variations are more intense when the dataset is generated from a fusion of datasets of different sizes. A Dynamic Regressor/Ensemble selection technique is proposed to address this problem. In the proposed method, a regressor/ensemble is selected to predict a sample point based on the sample’s proximity to a cluster assigned to the regressor/ensemble. K Means Clustering was used to form the clusters and the regressors considered are K Nearest Neighbor (KNN), Extreme Learning Trees (ET), Random Forest (RF), Gradient Boosting (GB), and Extreme Gradient Boosting (XGBoost). The ensembles are any combinations of two, three or four of the regressors. The sample points belonging to each cluster were selected from a validation set based on the regressor that made prediction with lowest absolute error per individual sample point. Implementation of the proposed technique resulted in accuracy improvements in a scenario described by a few sample points in the training data. Improvements in accuracy were also observed on datasets in other works compared to the accuracy reported in the works. The study also shows that using features extracted from satellite images to describe the environment was more appropriate than using a categorical clutter height value.

1. Introduction

Most of modern-day information is sent through the wireless channel due to its low cost and ease of installation when compared with the wired channel [1]. However, as signals propagate through the wireless channel, the strength diminishes in free space due to expansion and presence of obstacles, resulting in reflection, refraction, diffraction, and absorption of the signals, and hence, multipath [2,3,4]. The reduction in signal strength is termed path loss and its computation/prediction is important, particularly during network design, maintenance, or interference analysis, as it helps in setting up network parameters such as transmitter/base station antenna height, cell radius, and transmitting power [5,6]. Prediction of path loss is done using either empirical models, deterministic models, or Machine Learning models (ML) [7,8]. Empirical models are mostly used due to their ease of use and availability in Radio Frequency Planning Softwares [9]. However empirical models have lower accuracy than deterministic models. The problem with deterministic models is their high computational complexity, and requirements for detailed environmental description [8,10,11]. ML models rival deterministic models, having a faster prediction, with high accuracy [12]. Machine Learning algorithms such as K Nearest Neighbor (KNN) [13], Multiple Layer Perceptron (MLP) [8], Support Vector Regression (SVR) [14], Generalized Regression Neural Network (GRNN) [15], Radial Basis Function Neural Network (RBFNN) [16], and Random Forest (RF) [17] have been used in developing path loss models. However, such models were developed for specific environments, with fixed frequencies and antenna heights. Such use of machine learning models is at its early stage of adoption for use in path loss prediction and cannot replace empirical models such as the COST 231 Hata model because most empirical models have parameter ranges in which they are valid in terms of frequency, distance, antenna heights, as well as parameters describing the type of environment, whether rural, or urban, which the ML models lack [6]. Due to this flexibility inherent in empirical models, they are widely used, and sometimes tuned for a specific environment, when measurements in the environment are taken [18,19]. This tuning feature is available in Radio Frequency Planning Softwares [9]. Therefore, development of an ML model with the adaptive features of empirical models is required so as to have a trained model that can be used to make accurate predictions for a network under design/maintenance even if measurements for that network are unavailable. The ability of ML in capturing nonlinearities, coupled with the inclusion of features such as frequency, heights of antennas, and environment that are available in most empirical models will enable the development of such a robust and accurate path loss model.

Attempts were made by scholars at developing path loss models valid for some frequency ranges and environment. Ayadi et al. developed an MLP model based on data at several frequency values in the Ultra High Frequency (UHF) band. The effect of the propagation environment was included by features measuring the distance crossed in each type of clutter by the propagating signal [20]. Nguyen and Cheema also developed an MLP model with data measured at some frequency values within the interval of 0.8 GHz to 70 GHz. A categorical feature was used to specify the environment type as urban or suburban. Another categorical feature was used to describe the propagation scenario as below or above rooftop based on the heights of the antennas [21]. If the transmitter and receiving antennas’ heights are below the surrounding rooftops, the scenario is below rooftop, whereas if any of them is higher, then it is an above-rooftop scenario. Sani et al. [22] developed a multiple parameter and multiple environment machine learning path loss predictive model in which a clutter height value was used in specifying the environment type. The dataset used was a combination of datasets comprising path loss measurements from different environments and network parameters, which can be viewed as a dataset with various cases. The case of a sample point within the dataset can be defined by the type of environment, frequency band, and heights of antennas. Five different algorithms were compared and KNN was the best amongst them. A single clutter value was used for each environment type, which does not give the details of locations points. As such, satellite images were further used to represent the environment type [23]. This gives a better representation of the environment. Features were extracted from the satellite images with a Convolutional Neural Network. The satellite image features were combined with other numeric features and used in developing models using ML algorithms and compared in terms of overall accuracy. This was further improved in [24] by the use of features extracted using Gray Level Co-occurrence Matrix in addition to the features used in [23], and replacing the CNN with a less complex one. However, the overall accuracy could misrepresent accuracy in the various cases, as observed in [22], where the model’s prediction is poor in the suburban environment, compared with other cases. Even if an algorithm has the overall best accuracy, there might be cases in which accuracies are low, especially when the data are imbalanced. Thus, proper model selection should be done such that there is good accuracy for the different cases of data. One way is to examine the accuracies of the cases for all algorithms considered. An alternative is to use Dynamic Regressor Selection (DRS), or Dynamic Ensemble Selection (DES) [25]. In DRS, a regressor model is automatically selected for each sample point to be predicted. Likewise in DES, an ensemble is selected for the prediction of a sample point. DRS or DES is used because a model’s performance differs on localized regions of data space and, as such, selecting models suited for each sample point will result in a better accuracy [26,27,28]. We developed DRS and DES schemes and compared performances per case of input data. The contributions of this work are:

• We show that for a path loss dataset with multiple cases, a model could have the overall best accuracy but can be poor in prediction of certain data cases.
• We show that for a path loss dataset with multiple cases, a model not having the overall best accuracy could have better accuracy for cases with a low number of sample points.
• A DES scheme was developed to improve accuracy of cases with a low number of sample points.

2. Related Works

In an Ensemble, a collection of models is used for prediction such that the final predicted value can be obtained by voting in classification or averaging in regression. Because each model is different, there is diversity leading to an improved prediction accuracy through the reduction in variance or bias, or both [29]. Ensembles have been used in path loss prediction, and the results were good. Moraitis et al. compared the performances of MLP, SVR, RF, and a bagging of KNN regressors for path loss prediction in the rural environment for a Fixed Wireless Access (FWA) at a frequency of 3.7 GHz. Bagging based on the KNN regressors had the highest accuracy, followed by RF, which is also an ensemble [30]. Random Forest outperforms other algorithms in path loss predictions. The algorithms outperformed include MLP, KNN, SVR, and AdaBoost [17,31,32,33]. In a study by Sotiroudis et al., XGBoost performed better than Random Forest [34]. However, these ensemble approaches were developed with a single dataset meant for predictions in specific environments such as rural [30], campus (suburban) [17], urban [34], aircraft cabin [33], and air-to-air interface in an urban environment [32]. A Stacking ensemble was reported to perform better than the individual base regressor models in [35,36], likewise, a Bagging ensemble outperformed a Blending Ensemble as well as the base regressor models in [37]. Table 1 presents a summary of works that used ensemble models for path loss prediction. Each of the works considered a different set of features for model development.

In this study, four datasets were merged into a single dataset in order to develop a model valid for some frequency bands, environments, and antenna heights, and was adopted from [22]. As the constituent datasets were from different sources, sizes and of different parameter values, developing a single model can result in erroneous predictions on some regions of the data space. This problem can be tackled using a DRS or DES such that a regressor or an ensemble suitable for a test sample point is selected for prediction. Alternatively, sampling techniques such as oversampling, undersampling, and synthetic data generation such as Synthetic Minority Over Sampling Technique for Regression (SMOTER), SMOGN, and Generative Adversarial Networks (GAN) can be used when there is data imbalance in target values. Undersampling and oversampling techniques are ineffective when the population of the minority case is small. Undersampling can also lead to removal of useful data and oversampling can result in addition of noise to the data [38]. SMOTER has been used in many applications including synthetic data generation for path loss model development, but SMOTER and SMOGN were designed for applications where there are rare extreme target values posing an imbalance in the data [39,40,41,42]. Likewise, GAN have also been used to generate synthetic data for both tabular and image data and to solve class imbalance problems [43,44]. This work deals with rarity of values within the features rather than target values because the rare cases have no extreme target values.

Table 1. Summary of previous works that used ensemble models for path loss prediction.

Reference	Input Data Type	Model(s)	Environment	Frequency (MHz)
[17]	Numeric	RF	University campus (Suburban)	1800
[31]	Numeric	RF	Urban	2021.4
[32]	Numeric	RF	Urban (air to air interface)	2400
[33]	Numeric	RF and AdaBoost	Air craft cabin	2400
[45]	Numeric	RF	Complex environment with vegetation	2400
[34]	Numeric	RF and XGBoost	Urban	n/a
[46]	Numeric	RF	Urban	900
[46]	Numeric	RF	Urban	1800
[30]	Numeric	Bagging of KNN base models	Rural	3700
[22]	Numeric	RF, Gradient Boosting, and XGBoost	Rural, urban, suburban, and urban highrise	868, 1800, 1835.2, 1836, 1840.8, 1864
[23]	Numeric + Image	RF, Gradient Boosting, and XGBoost	Rural, urban, suburban, and urban highrise	868, 1800, 1835.2, 1836, 1840.8, 1864
[24]	Numeric + Image	RF, Gradient Boosting, Extreme Learning Trees, and XGBoost	Rural, urban, suburban, and urban highrise	868, 1800, 1835.2, 1836, 1840.8, 1864
[47]	Image	RF, XGBoost, and LightGBM	Urban	900
[35]	Numeric	Stacking, RF, XGBoost, LightGBM, and AdaBoost	Urban	900
[36]	Numeric	Stacking, voting, bagging with KNN base regressors, and Gradient Boosting	Rural	3700
[37]	Numeric	Bagging and Blending	Rural, suburban, and urban	2400
This study	Numeric + Image	RF, Gradient Boosting, Extreme Learning Trees, XGBoost, DRS, and DES	Rural, urban, suburban, and urban highrise	868, 1800, 1835.2, 1836, 1840.8, 1864

n/a was used where information was not provided by the author.

Table 1. Summary of previous works that used ensemble models for path loss prediction.

Reference	Input Data Type	Model(s)	Environment	Frequency (MHz)
[17]	Numeric	RF	University campus (Suburban)	1800
[31]	Numeric	RF	Urban	2021.4
[32]	Numeric	RF	Urban (air to air interface)	2400
[33]	Numeric	RF and AdaBoost	Air craft cabin	2400
[45]	Numeric	RF	Complex environment with vegetation	2400
[34]	Numeric	RF and XGBoost	Urban	n/a
[46]	Numeric	RF	Urban	900
[46]	Numeric	RF	Urban	1800
[30]	Numeric	Bagging of KNN base models	Rural	3700
[22]	Numeric	RF, Gradient Boosting, and XGBoost	Rural, urban, suburban, and urban highrise	868, 1800, 1835.2, 1836, 1840.8, 1864
[23]	Numeric + Image	RF, Gradient Boosting, and XGBoost	Rural, urban, suburban, and urban highrise	868, 1800, 1835.2, 1836, 1840.8, 1864
[24]	Numeric + Image	RF, Gradient Boosting, Extreme Learning Trees, and XGBoost	Rural, urban, suburban, and urban highrise	868, 1800, 1835.2, 1836, 1840.8, 1864
[47]	Image	RF, XGBoost, and LightGBM	Urban	900
[35]	Numeric	Stacking, RF, XGBoost, LightGBM, and AdaBoost	Urban	900
[36]	Numeric	Stacking, voting, bagging with KNN base regressors, and Gradient Boosting	Rural	3700
[37]	Numeric	Bagging and Blending	Rural, suburban, and urban	2400
This study	Numeric + Image	RF, Gradient Boosting, Extreme Learning Trees, XGBoost, DRS, and DES	Rural, urban, suburban, and urban highrise	868, 1800, 1835.2, 1836, 1840.8, 1864

n/a was used where information was not provided by the author.

Performance of each model varies at localized regions of the data space. DRS or DES is used to select the best model/ensemble for each sample point to be predicted because a model/ensemble with the best overall performance can be worst at predicting sample points from a localized region compared to others. This is particularly useful in datasets generated from a fusion of datasets of different sources. DRS and DES improve prediction accuracy even in situations where the dataset is from a single source. Although DRS and DES have not been used in path loss prediction, such schemes were developed and tested across various datasets and performances were observed to depend on the dataset such that DRS outperforms DES in few cases [48,49,50,51]. The algorithm used as the base regressor/model also determines the prediction accuracy of the DES [52]. A DES was proposed by Ghoneimy et al. for predicting death counts due to influenza-like illnesses [53], which performed better than individual models. A Dynamic Ensemble Selection with Instantaneous Pruning (DESIP) was proposed by Dias and Windeatt for application in signal calibration, which performed better than some static methods. However, it failed on some public datasets [54]. Other applications in which dynamic ensemble regression was applied include prediction of mass flow rate and volume fraction [55], water quality prediction, and in predicting the temperature of molten steel in a ladle furnace [56]. We therefore investigate on DRS and DES to find out which of them is suitable for a path loss dataset.

3. Methodology

Two datasets were used in this study. The first dataset is adopted from [22] and is composed of 11 features, including a clutter height feature for differentiating environments. The second dataset is adopted from [24] and is made up of eight numerical features, and other features extracted from satellite images using CNN and Grey Level Co-occurrence Matrix (GLCM). Each of the datasets has 12,369 samples which were split into training, validation, and testing sets based on the ratio 70:15:15, resulting in 8658, 1855, and 1856 samples, respectively. Details of the datasets are presented in

Table 2. Summary of datasets.

Dataset	Size	Features
Dataset 1	12,369	Distance, elevation of transmitter position, elevation of receiver position, frequency, height of transmitting antenna, height of receiving antenna, clutter height, latitude, longitude, distance in latitude between transmitting and receiving antenna, and distance in longitude between transmitting and receiving antenna.
Dataset 2	12,369	Distance, elevation of transmitter position, elevation of receiver position, frequency, height of transmitting antenna, height of receiving antenna, distance in latitude between transmitting and receiving antenna, distance in longitude between transmitting and receiving antenna, and others extracted from satellite images using a CNN and GLCM.

.

3.1. Dynamic Regressor Selection (DRS)

The proposed DRS technique is different from those in previous studies in that it uses a heterogeneous ensemble rather than homogeneous one, thereby improving the diversity of the base regressors. Although [51] made use of a heterogeneous ensemble, classification was used for ensemble selection rather than clustering as in this case. In this study, a high accuracy in base models was ensured by optimizing hyper-parameters such that the focus is on selecting the best model to use in predicting a test sample point. Having known that the prediction accuracy of a model varies across localized regions of the data space, the method exploits the accuracy of the base models and their diversity in making a fair model selection. The training and testing process are outlined as follows:

3.1.1. Training

(i)

Optimize hyper-parameters of learning algorithms (K Nearest Neighbor, Extreme Learning Trees (ET), Random Forest, Gradient Boosting (GB), and Extreme Gradient Boosting (XGBoost)).

(ii)

Train models with optimized hyper-parameters.

(iii)

Test each model with the validation set.

(iv)

For each validation sample point, select the model with the lowest absolute error. This is done to identify the model that predicted each validation sample point with the least error.

(v)

Develop a cluster for each model based on validation sample points in which they predicted with the least error. K means clustering with a K value of 1 is used for cluster formation. A value of 1 was used because the cluster members were already selected based on the minimum absolute error as shown in Figure 1, and thus, the centroids were determined by K Means. Algorithm 1, gives additional description of the DRS training phase.

Algorithm 1 Dynamic Regressor Selection Training Phase

1: Split dataset into train, validation, and test sets with ratio 70:15:15 to obtain $N_{train}$, $N_{validation}$, and $N_{test}$ sample points, respectively. 2: Train j models with the train set based on different algorithms with their optimized hyper-parameters to form a set Model. 3: for i in $N_{validation}$ do 4:    for j in Model do 5:      Compute $absolute\_erro{r}_j=|P_{mi}-P_{p\left(ij\right)}|$, where: $absolute\_erro{r}_j$ represent the absolute error of the jth model, $P_{mi}$ is the measured path loss of the ith sample point in the validation set, and $P_{p\left(ij\right)}$ is the path loss predicted by the jth model using the ith validation sample point as input. 6:    end for 7:    Find $min(absolute\_error)$ 8:    if $min(absolute\_error)=absolute\_erro{r}_{\left(j\right)}$ then 9:      Append sample point i to cluster j 10:    end if 11: end for

3.1.2. Testing

(i)

Measure the distance of the test sample point to each cluster. This measures the similarity between the test sample point and validation samples in each cluster [57]. Euclidean distance is used for this purpose.

(ii)

Select the cluster with the least distance and use the model designated for the cluster to make a prediction. Further details on the DRS testing phase is presented in Algorithm 2, and description of the the whole DRS process is shown in Figure 1.

Algorithm 2 Dynamic Regressor Selection Testing Phase

1: for k in $N_{test}$ do 2:    for j in cluster do 3:      Measure cluster distance $\left(d_{c\left(j\right)}\right)$ 4:    end for 5:    Find $min\left(d_c\right)$ 6:    if $min\left(d_c\right)=d_{c\left(j\right)}$ then 7:      Make prediction using $Mode{l}_{\left(j\right)}$ 8:    end if 9: end for represent the absolute

3.2. Dynamic Ensemble Selection (DES)

The training and testing processes in DES are the same with that of DRS described in Section 3.1.1 and Section 3.1.2. However, ensembles of models are included in addition to the single regressors/models as shown in Figure 2, which is its difference with DRS described in Figure 1. The Ensemble combinations in

Table 3. Ensemble combinations.

Two Algorithms	Three Algorithms	Four Algorithm
KNN+RF	KNN+RF+GB	KNN+RF+GB+XGBoost
KNN+GB	KNN+RF+XGBoost	KNN+RF+GB+ET
KNN+XGBoost	KNN+RF+ET	KNN+RF+XGBoost+ET
KNN+ET	KNN+GB+XGBoost	KNN+GB+XGBoost+ET
RF+GB	KNN+GB+ET	RF+GB+XGBoost+ET
RF+XGBoost	RF+GB+XGBoost
RF+ET	RF+GB+ET
GB+XGBoost	XGBoost+ET+KNN
GB+ET	XGBoost+RF+ET
XGBoost+ET

were considered, and are all possible combinations of either two, three or four algorithms to exploit the accuracy improvement characteristic of ensembles resulting from the diversity provided by the base models. The DES scheme selects either a regressor or an ensemble suitable for predicting a test sample point. The ensembles are formed by averaging the predictions of the models concerned. For example, KNN+RF+GB refers to an ensemble in which the predictions of KNN, RF, and GB are summed and divided by 3.

4. Results and Discussion

The overall performance of each regressor/model, DRS and DES are examined as well as the performance per case of data.

4.1. Results Based on Dataset 1

The absolute error of the various model’s predictions on 10 random sample points from the validation set are presented in Figure 3 so as to clearly show how model performance depends on individual sample points. Figure 3 shows variations in the model suitable for predicting each sample points. ET was observed to be better for the first sample point, KNN for the second, and the models performed almost equally on the third point. Likewise, there were variations in terms of the least accurate model per sample, with KNN, GB, and XGBoost having the highest absolute error for the first, second, and ninth samples, respectively. This showed that there is need to dynamically select the model/ensemble to predict each point so as to minimize the prediction error per point. Figure 4, Figure 5, Figure 6 and Figure 7 present the performance metrics in dataset 1, Figure 8 and Figure 9 present the percentage that each model/ensemble is selected for the total number of predictions made, and

Table 4. Performance on the different cases in dataset 1.

Algorithm	Environment	Band	MAE (dB)	RMSE (dB)	MAPE (dB)	${\mathit{R}}^2$
KNN	All	All	2.9275	3.9998	2.2583	0.9246
	Rural	800	2.7144	3.7026	2.1330	0.9426
	Suburban	1800	2.2862	3.0839	1.6192	0.8822
	Urban	1800	3.4695	4.5189	2.7069	0.8324
	Urban	2100	6.5301	8.1102	5.5187	−0.1391
	Urban highrise	800	3.2147	4.4396	2.5911	0.9164
ET	All	All	3.0745	4.1078	2.3708	0.9204
	Rural	800	2.5712	3.4693	2.0220	0.9496
	Suburban	1800	2.7102	3.5857	1.9239	0.8409
	Urban	1800	3.7666	4.8619	2.9531	0.8060
	Urban	2100	6.2294	7.6968	5.2703	−0.0265
	Urban highrise	800	3.1145	4.2043	2.5288	0.9251
RF	All	All	2.8815	3.8908	2.2217	0.9286
	Rural	800	2.5706	3.4623	2.0231	0.9483
	Suburban	1800	2.3280	3.1457	1.6524	0.8775
	Urban	1800	3.5618	4.5681	2.7833	0.8287
	Urban	2100	6.3881	7.7169	5.3807	−0.0317
	Urban highrise	800	3.0375	4.1677	2.4474	0.9263
GB	All	All	2.8426	3.8350	2.1867	0.9307
	Rural	800	2.5310	3.4443	1.9902	0.9504
	Suburban	1800	2.3529	3.1345	1.6669	0.8784
	Urban	1800	3.3723	4.3239	2.624	0.8465
	Urban	2100	6.6264	8.0565	5.6001	−0.1262
	Urban highrise	800	3.0486	4.1866	2.4559	0.9257
XGBoost	All	All	2.9690	3.9983	2.2851	0.9246
	Rural	800	2.5450	3.4336	2.0023	0.9498
	Suburban	1800	2.5150	3.3566	1.7824	0.8605
	Urban	1800	3.6179	4.6663	2.8194	0.8212
	Urban	2100	6.5289	8.0822	5.5532	−0.1364
	Urban highrise	800	3.1112	4.2227	0.5110	0.9244
DRS	All	All	3.1692	4.3757	2.4396	0.9102
	Rural	800	2.7258	3.7340	2.1339	0.9386
	Suburban	1800	2.5977	3.6892	1.8464	0.8342
	Urban	1800	4.0672	5.3260	3.1717	0.7665
	Urban	2100	8.9912	10.8677	7.5460	−0.4363
	Urban highrise	800	3.2588	4.4808	2.6139	0.9139
DES	All	All	2.9677	4.0145	2.2865	0.9243
	Rural	800	2.5001	3.3901	1.9599	0.9494
	Suburban	1800	2.5173	3.4173	1.7887	0.8579
	Urban	1800	3.6356	4.7048	2.8484	0.8178
	Urban	2100	8.0006	9.6230	6.7360	−0.1095
	Urban highrise	800	3.2326	4.4335	2.5864	0.9156

shows the performance for each data case.

Based on the results, GB had the best overall performance as it has the least values of the error metrics and the highest Coefficient of Determination, and DRS had the least accuracy. Even though DRS and DES were not the best in terms of overall accuracy, they could be when tested on datasets in other domains [48,49].

From Table 4, it was observed that the performances in all environments and frequencies were good except in the urban at 2100 MHz band. This happened because of the few number of sample points representing the case. Although GB had the overall best accuracy based on least error and $R^2$ value, it was not the best in this data case as ET, RF, and DES were better than it, with ET as the best, and DRS the last. DES performance was surpassed by only ET and RF. Thus, even though it did not have the best performance, it gave an intermediary performance and could be used when avoiding the check on the individual algorithms to save time. Figure 8 shows that XGBoost was the most selected model for prediction in dataset 1 using the DRS technique, while ET is the least selected. In the case of DES in Figure 9, RF was selected the most for prediction while other single models had the least, including XGBoost that had the highest selection in DES. This showed that the ensemble of models by the combination of models in 2’s, 3’s, and 4’s, resulted in a variety of ensembles with better accuracy than the individual models and their clusters were selected for prediction due to their proximity to the sample points.

The negative coefficient of determination values for the urban environment at 2100 MHz case shows the high variation of the predicted values from the original. Under normal circumstances, it is supposed to have values between 0 and 1. However, it is negative because the Sum of Squared Error (SSE) is high to the extent that it exceeds the Total Sum of Squares as described by Equations (1)–(4). This is because the model/models learned less of this case and learned much of others, as it is represented by few sample points in the dataset.

$$R^2=1-\frac{SSE}{TSS}$$

(1)

$$R^2=1-\frac{{\sum }_{i=1}^N{(P_{m\left(i\right)}-P_{p\left(i\right)})}^2}{{\sum }_{i=1}^N{(P_{m\left(i\right)}-\overline{P_p})}^2}$$

(2)

$$SSE=\sum _{i=1}^N{(P_{m\left(i\right)}-P_{p\left(i\right)})}^2$$

(3)

$$TSS=\sum _{i=1}^N{(P_{m\left(i\right)}-\overline{P_p})}^2$$

(4)

$P_{m\left(i\right)}$ represents the ith measured path loss, $P_{p\left(i\right)}$ the ith predicted path loss, $\overline{P_p}$ the average of predicted path loss, and N the total number of observations [58].

4.2. Results Based on Dataset 2

Figure 10, Figure 11, Figure 12 and Figure 13 present the performances of models on dataset 2, Figure 14 and Figure 15 present the percentage that each model/ensemble is selected for the total number of predictions made, and

Table 5. Performance on the different cases in dataset 2.

Algorithm	Environment	Band	MAE (dB)	RMSE (dB)	MAPE (dB)	${\mathit{R}}^2$
KNN	All	All	2.8328	4.2307	2.1997	0.9163
	Rural	800	3.2276	3.2429	1.9116	0.9572
	Suburban	1800	3.2429	2.7357	1.4274	0.9085
	Urban	1800	3.5979	5.7336	2.8139	0.7278
	Urban	2100	3.2622	5.4613	2.8540	0.5099
	Urban highrise	800	3.2276	4.3935	2.6034	0.9151
ET	All	All	2.9058	4.3627	2.2577	0.9097
	Rural	800	2.5786	3.3885	2.0477	0.9533
	Suburban	1800	2.0481	2.7567	1.4506	0.9071
	Urban	1800	3.4708	5.8460	2.7077	0.7171
	Urban	2100	3.6348	6.0512	3.1511	0.3940
	Urban highrise	800	3.5089	4.7502	2.8289	0.9007
RF	All	All	2.8086	4.1584	2.1721	0.9189
	Rural	800	2.4753	3.3440	1.9460	0.9525
	Suburban	1800	2.1655	2.8948	1.5350	0.9006
	Urban	1800	3.3866	5.3520	2.6453	0.7589
	Urban	2100	4.4750	6.2959	3.7403	0.2986
	Urban highrise	800	3.1117	4.2989	2.4969	0.9199
GB	All	All	2.6485	3.8459	2.0491	0.9298
	Rural	800	2.3876	3.2406	1.8802	0.9573
	Suburban	1800	2.0517	2.7599	1.4552	0.9068
	Urban	1800	3.0806	4.7584	2.4096	0.8125
	Urban	2100	3.9586	6.4257	3.4343	0.3257
	Urban highrise	800	3.0938	4.2769	2.4810	0.9195
XGBoost	All	All	2.5848	3.6680	2.0013	0.9372
	Rural	800	2.4384	3.2622	1.9201	0.9557
	Suburban	1800	1.9314	2.6362	1.3675	0.9150
	Urban	1800	3.5975	5.4851	2.8129	0.7509
	Urban	2100	5.0399	6.7689	4.2441	0.2529
	Urban highrise	800	3.1088	4.2639	2.4914	0.9200
DRS	All	All	2.6593	3.7806	2.0589	0.9324
	Rural	800	2.5577	3.4003	2.0296	0.9481
	Suburban	1800	1.9843	2.6724	1.4027	0.9096
	Urban	1800	2.8381	4.1750	2.2091	0.8469
	Urban	2100	3.8718	5.6792	3.3268	0.5023
	Urban highrise	800	3.1815	4.4501	2.5533	0.9129
DES	All	All	2.7452	4.0631	2.1262	0.9219
	Rural	800	2.4446	3.2896	1.9307	0.9514
	Suburban	1800	2.0217	2.7029	1.4285	0.9076
	Urban	1800	3.1749	5.0792	2.4751	0.7732
	Urban	2100	3.0924	5.1577	2.6625	0.5885
	Urban highrise	800	3.2138	4.4460	2.5860	0.9131

shows the performance per environment case. XGBoost had the highest accuracy in this case, followed by DRS, unlike in dataset 1, where DRS had the least accuracy. This is due to the difference of the features in the two datasets. Dataset 1 has 11 features and dataset 2 has 191 features due to the presence of CNN and GLCM features extracted from satellite images which are not present in dataset 1. Each type of ML algorithm has its strengths and weaknesses. Some are good with large datasets and others are better with small datasets [59]. This caused the variation in performance of ML algorithms in the two datasets.

XGBoost had the overall highest accuracy and the least accurate in terms of the minority case, whereas KNN had the highest accuracy for this data case amongst the five algorithms with DRS and DES exclusive. DES had the highest accuracy with an RMSE value of 5.1577 dB for the case as it outperformed KNN (RMSE = 5.4613 dB). Thus, there was an improvement in performance in the case with the least sample points as RMSE values were below the 7 dB and 15 dB threshold values in the urban and rural environments, respectively. Although DES has higher computational complexity than the individual models, it can be used when there is imbalance in data due to its ability to select the best model/ensemble to predict for a particular test sample point. Performance based on dataset 2 is better than that of dataset 1 as observed from results obtained for the various cases of data. This is because the features in dataset 2 were rich enough to provide better description of the environments, unlike in dataset 1 where a single integer was assigned as clutter height for the environment. Figure 14 and Figure 15 present the percentage selection of the models/ensembles in dataset 2 based on DRS and DES, respectively. XGBoost was the highest selected model, and KNN was the least selected model for DRS, while an ensemble of KNN+RF+GB was the most selected in DES. DES was better than DRS in predicting the case with the least number of sample points because XGBoost was selected most in DRS. However, XGBoost has the lowest accuracy for this case (urban at band of 2100 MHz) as presented in Table 5. The addition of ensembles in DES provided more options and proper selections were made that led to improvement in prediction accuracy, unlike in DRS where there were few options.

Figure 16, Figure 17, Figure 18, Figure 19 and Figure 20 present the measured and predicted path loss values versus distance by the proposed DRS and DES techniques in rural, suburban, urban at 1800 MHz, urban at 2100 MHz, and urban highrise, respectively. Good correlation between the measured and predicted path loss values was observed in all cases except for Figure 19 that represents the urban environment at the 2100 MHz band. This reflects their error metrics presented in Table 4 and Table 5. In each of the figures, not many differences were observed that will validate the preference of one technique over the other. The superiority of one over the other can only be ascertained from Table 4 and Table 5.

4.3. Comparison with Empirical Models

Comparison was made with the three empirical models shown in

Table 6. Empirical models and their input parameter limit.

Empirical Model	Frequency Range (MHz)	Transmitting Antenna Height Range (m)	Receiving Antenna Height Range (m)	Distance Range (km)
Egli [60]	[40, 1000]	n/a	$\le$10 and $\ge$10	[1, 80]
Ericsson 999 [61]	[150, 1900]	[30, 300]	[1, 10]	[1, 20]
COST-231 Hata [62]	[1500, 2000]	[30, 200]	[1, 10]	[1, 20]

. Each of the models has a range of frequencies, antenna height, and distance within which it is valid. The Egli model was used for comparison in the rural and urban highrise environments because it is valid for 800 MHz band networks and can be used in situations where the receiving antenna height is above 10 m as in this case. Ericsson 999 and COST-231 Hata were used for suburban and urban environments because the frequency and antenna heights are within the operating ranges of the models.

Figure 21 presents performances of the empirical models in terms of RMSE in the various environments and frequency bands, and performance of the DES built from dataset 2. Comparison was done with the model developed from dataset 2 because its performance was good across the various environments and frequency bands due to the satellite image features in the dataset as well as DES ability to select the regressor/ensemble to predict a test sample point. The proposed model’s performance is better than that of the empirical models in all situations, and a wide performance gap was experienced in the rural and urban highrise environments. RMSE values of empirical models were lowest in the urban environment. The Ericsson 999 model and COST-231 Hata were lower in the urban environment than in the suburban environment. This must have been due to resemblance of environments from which the empirical models were developed with the current urban environment. The Ericsson 999 model had a better accuracy than COST-231 Hata at the two bands in the urban environment, whereas in the suburban environment, COST-231 had a better accuracy. The main determinant of the accuracy of the empirical models is the resemblance in data based on which the empirical model was developed with the data it is tested against.

4.4. Testing on Other Datasets

The proposed method was tested on datasets used in [63,64,65,66,67], and comparisons were made with results in the original works in terms of RMSE, as presented in

Table 7. Performance of proposed method on other datasets.

Dataset	RMSE in Original Work (dB)	RMSE with Proposed DES Method (dB)
[63] at 811 MHz	≈4.1000	3.4420
[63] at 2630 MHz	≈4.2500	2.2058
[64]	1.7600	4.2061
[65] for Rural	10.2844	3.2203
[65] for Urban highrise	6.4300	4.3679
[66,68]	6.2700	2.6780
[67]	3.8000–6.5400	3.2415–5.1490

. Although RMSE values were not presented in [65], the path loss exponent, mean, and standard deviation of the shadowing component for each of the two environments were presented. These values were substituted into the Log-distance path loss model and RMSE values of the predictions were computed. RMSE values in [63] were presented in the form of bar charts for the 811 MHz and 2630 MHz frequencies. The values were slightly above 4 dB, having approximate values of 4.25 dB and 4.1 dB, respectively. In [68], a dataset analysis was presented and the dataset was used as a subset in [66] to develop a model that resulted in an RMSE value of 6.27 dB. For the work by Carvalho et al. [67], results of 4 routes were presented, 3 of which are for suburban environments, and the fourth one is an urban environment. RMSE values for each route lies within the range 3.80 dB–6.54 dB.

Improvements were made to results in [63,65,66,67] by the proposed method and a reduction of accuracy was observed for [64] because the RMSE value was greater than that of the original work, as presented in Table 7. However, the RMSE value of 4.2061 dB by our proposed method is acceptable as it is less than 7 dB. It should be noted that results in [63,64,67] were obtained when the search space of 1:190 was used for the maximum features’ hyper-parameter in RF, ET, and GB. Meanwhile, good results were obtained from other datasets with just a search space of 1:10. Hence, the method is recommended for path loss model development, especially if the process involves combining datasets from various sources and of different settings of measurements.

5. Conclusions

Path loss models are used during the design of networks and during maintenance. Machine Learning path loss models provide predictions at a faster speed than Deterministic models without compromising accuracy. However, Deterministic models and Empirical models are adaptive to frequencies, antenna heights, and environments, while most machine learning models are developed with data from a single environment, frequency, and antenna height. Therefore, Deterministic and Empirical models have an advantage of versatility over machine learning. To develop machine learning models with such characteristics, data from various environments and with different network parameters are required. Such data can be obtained from various sources, which come in varied sizes. It is known that machine learning model performance differs on distinct parts of a dataset. Thus, this effect will be intense if several datasets were merged, as the accuracy of one of the classes in the merged dataset can be low. This was observed in the dataset used such that the accuracy of one class of the data was low compared to others. This class of data had the least size amongst the combined data. A Dynamic Regressor/Ensemble Selection model was used to solve the problem such that a model/ensemble is selected during the prediction of each data sample point. In the proposed method, a regressor/ensemble is selected to predict a sample point based on its proximity to a cluster assigned to the regressor/ensemble. The regressor/ensemble for the closest cluster is selected for prediction purposes. This improved the accuracy of the minority data class. The method’s accuracy was better than that of some empirical models it was compared against. The technique performed better when features extracted from satellite images were included compared to a case in which integer values representing clutter height were used to describe environments. Hence, the satellite image features were very qualitative for environment description which improved the minority case accuracy of the individual models, and with the Dynamic Regressor/Ensemble technique applied, the accuracy of the minority case was enhanced. The method was tested on five datasets used in other works, and an improvement in accuracy beyond that reported in the original works was observed in four out the five datasets.