Intelligent Feature Selection Ensemble Model for Price Prediction in Real Estate Markets

Abstract

Real estate is crucial to the global economy, propelling economic and social development. This study examines the effects of dimensionality reduction through Recursive Feature Elimination (RFE), Random Forest (RF), and Boruta on real estate price prediction, assessing ensemble models like Bagging, Random Forest, Gradient Boosting, AdaBoost, Stacking, Voting, and Extra Trees. The results indicate that the Stacking model achieved the best performance with an MAE (mean absolute error) of 14,090, MSE (mean squared error) of 5.338 × 10⁸, RMSE (root mean square error) of 23,100, R² of 0.924, and a Concordance Correlation Coefficient (CCC) of 0.960, also demonstrating notable computational efficiency with a time of 67.23 s. Gradient Boosting closely followed, with an MAE of 14,540, R² of 0.920, and a CCC of 0.958, requiring 1.76 s for computation. Variable reduction through RFE in both Gradient Boosting and Stacking led to an increase in MAE by 16.9% and 14.6%, respectively, along with slight reductions in R² and CCC. The application of Boruta reduced the variables to 16, maintaining performance in Stacking, with an increase in MAE of 9.8% and a R² of 0.908. These dimensionality reduction techniques enhanced computational efficiency and proved effective for practical applications without significantly compromising accuracy. Future research should explore automatic hyperparameter optimization and hybrid approaches to improve the adaptability and robustness of models in complex contexts.

1. Introduction

The real estate market represents an essential component of the global economy [1], substantially impacting macroeconomic and microeconomic dynamics [2,3]. The importance is evidenced by its ability to influence economic growth [4,5], mainly through changes in real estate prices [2]. A paradigmatic example of the inherent vulnerability of this market was the mortgage crisis of 2007 and 2008, which triggered a global recession [6]. That crisis revealed the structural fragility of the real estate sector [7] and its deep interconnection with the international financial system [8,9], underscoring the necessity for more accurate price forecasting models.

There is a clear need to refine predictive tools in the real estate market [10,11], particularly in contexts of accelerated growth. Accurate prediction of real estate prices would not only reduce the risk of forming speculative bubbles and financial collapses [12,13] and promote local economic development by facilitating more efficient planning [14]. This predictive capability would drive socioeconomic growth locally and globally and support informed decision-making by governments, real estate agents, financial institutions, and market analysts [15,16,17].

To solve the problem, traditional models for price prediction have been applied, such as the Comparative Market Method [8], the Income Capitalization Method (Income or Rent Method) [18], the Replacement Cost or Replacement Value Method, the Automatic Valuation Method (AVM) [19], and the Dynamic Residual Value Method (Discounted Cash Flow) [20]. These models have proven effective in stable markets, where price trends typically follow predictable patterns based on historical data [21]. However, they have significant limitations, particularly in environments with high volatility or when prices react to more complex economic dynamics. Traditional models often underestimate or overestimate prices in highly speculative markets or during crises, as they fail to adequately consider macroeconomic factors, regulatory changes, large data volumes, or sudden fluctuations in supply and demand [22,23].

In recent years, the increasing availability and storage of large volumes of data have created new opportunities to tackle the complex issue of price prediction in the real estate market [24,25,26,27]. Traditional valuation models, primarily based on statistical methods, have been outperformed in accuracy and predictive capability by machine learning algorithms [28,29], which are proving to be a highly effective alternative for enhancing real estate price estimates [30,31].

Numerous empirical studies have investigated the predictive capability of various machine learning algorithms, yielding promising results in real estate price prediction [24]. Among the most notable methods are Random Forest (RF) [32,33], Support Vector Machines (SVMs) [34], Multiple Linear Regression (MLR) [35], and regularization techniques such as LASSO (Least Absolute Shrinkage and Selection Operator) [36]. Additionally, algorithms like K-Nearest Neighbors (KNNs) [37] and decision trees (DTs) [38] have demonstrated remarkable performance across different dataset configurations.

Research on machine learning algorithms applied to regression problems has gained significant importance in the real estate sector in recent years [31,39]. However, regression issues in real-world contexts often involve highly complex internal and external factors [40]. Furthermore, different machine learning algorithms exhibit considerable variations in terms of scalability and predictive performance [41,42], which presents additional challenges for their effective application in practical settings by tuning their parameters to select the best model [40,43]. Although this strategy is commonly used, it faces three critical challenges [44,45,46]. First, defining the “best” model is complicated when multiple candidates show comparable performance, complicating the final choice [47]. This issue is exacerbated when the algorithm is sensitive to local optima and the amount of training data are limited [48]. Second, excluding less successful models may lead to the loss of valuable information that could enhance prediction accuracy [49]. Third, variable selection is essential in mitigating the curse of dimensionality, as using too many features may introduce noise and redundancy, impacting model generalization [50].

As a result, research in the real estate market encounters methodological and computational challenges that demand a rigorous and systematic approach. The lack of studies examining these issues highlights the necessity of developing innovative strategies that effectively enhance models’ predictive accuracy and generalizability, ensuring their relevance in complex and dynamic scenarios within the real estate sector [51,52].

In this context, this study aimed to develop and compare ensemble models with optimized feature selection for price prediction in real estate markets. The proposed methodology integrates multiple base algorithms and employs advanced dimensionality reduction strategies, including RF, Recursive Feature Elimination (RFE), and Boruta, to identify and retain only contributing variables. This approach aims to maximize predictive accuracy, model robustness, and generalizability, ensuring optimal performance in highly complex and structurally variable environments.

2. Literature Review

An extensive literature review on price prediction in real estate markets was conducted, emphasizing machine learning algorithms. The analysis identified and assessed the most pertinent scientific articles in this field. In this context, Park et al. [53] developed a housing price prediction model utilizing C4.5, Repeated Incremental Pruning to Produce Error Reduction (RIPPER), Naïve Bayes, and AdaBoost, comparing their classification accuracy. The results indicate that RIPPER surpasses the other algorithms in predictive performance.

In the same line of research, Varma et al. [54] implemented a weighted average approach using multiple regression techniques to enhance the accuracy of predicting real estate values. This method reduces error and outperforms individual models in stability and accuracy, optimizing valuations by incorporating environmental contextual information. Conversely, Rafiei et al. [55] developed an innovative approach based on a Restricted Boltzmann machine (RBM) combined with a genetic algorithm that does not involve mating. Their model evaluates the viability of the real estate market by incorporating economic variables, seasonality, and temporal effects, achieving computationally efficient optimization on standard workstations. The validity and applicability of the model were verified through a case study, demonstrating its accuracy in predicting real estate prices.

An effective strategy to tackle this problem is to implement ensemble methods, which combine multiple base models to enhance the stability and generalizability of the final model [56]. This approach addresses the individual limitations of each algorithm, reducing both variance and bias, thereby minimizing the risk of overfitting and optimizing predictive accuracy in real estate price estimation [57]. In this context, models based on ensemble methods, such as RF, have demonstrated superior performance in predicting prices in real estate markets due to their ability to decrease variance, improve stability, and capture nonlinear relationships in the data [32].

In a related study, Phan [45] evaluated the performance of SVM, RF, and Gradient Boosting Machines (GBMs) in real estate valuation in Hong Kong, discovering that RF and GBMs surpassed SVM regarding accuracy. However, while SVM exhibited lower performance, it excelled in efficiency for rapid predictions. The authors concluded that machine learning presents a promising alternative for property valuation.

Arabameri et al. [57] employed data mining techniques and regression models, including LASSO, RF, and GBMs, to predict prices in the Wroclaw real estate market, achieving 90% accuracy and demonstrating these approaches’ effectiveness in modeling real estate prices. Regarding the enhancement in capturing nonlinear relationships with high accuracy, Ribeiro and dos Santos [58] evaluated several machine learning techniques for real estate price prediction, highlighting the superiority of RF over traditional models such as linear regression and SVM. Furthermore, the authors discussed potential future advancements in real estate estimation.

Adetunji et al. [33] obtained concordant results by employing RF to predict housing prices using the Boston dataset (UCI). Their model achieved an accuracy margin of ±5%, demonstrating its effectiveness in estimating individual real estate values. Ensemble methods like Bagging, GBMs, and Extreme Gradient Boosting (XGBoost) have shown promising outcomes in predicting real estate prices. They are noted for their capacity to enhance predictive accuracy and model robustness across various contexts [42,58,59].

Sibindi et al. [59] evaluated the effectiveness of XGBoost in predicting real estate prices, achieving an accuracy of 84.1% compared to the 42% obtained by hedonic regression. Their study, which analyzed 13 variables, reaffirms the superiority of machine learning approaches over traditional models in this field. Similarly, Gonzales [60] examined land price prediction in Seoul from 2017 to 2020 using Random Forest (RF) and XGBoost. This study considered 21 variables and assessed their impact on predictive accuracy. The results indicated that XGBoost outperformed RF, demonstrating greater generalizability and accuracy in estimating real estate values.

Meanwhile, Sankar et al. [61] applied regression techniques and assembly methods to predict real estate prices, considering key variables such as location and demographics. The models developed achieved an accuracy of 94%, evidencing their usefulness in real estate investment decision-making. Consistent with these findings, Kumkar et al. [62] conducted a comprehensive comparative analysis of ensemble methods, including Bagging, RF, Gradient Boosting, and XGBoost, in Mumbai real estate valuation, focusing on hyperparameter optimization. To improve the accuracy of the models, they implemented advanced data preprocessing techniques, which mitigated biases and reduced variance [63]. The results confirm that ensemble methods are robust and efficient tools for price estimation in real estate markets characterized by high complexity and volatility [59].

In a bibliometric analysis, Takouabou et al. [10] examined 70 articles indexed in Scopus, noting that scientific production in this field primarily comes from the USA, China, India, Japan, and Hong Kong—regions known for their high levels of digitization and well-established research ecosystems. However, the review revealed recurring methodological limitations, such as dependence on small datasets and a preference for straightforward machine learning models, complicating the issue of the curse of dimensionality. These limitations highlight the need to integrate more advanced methodologies that enhance predictive performance and improve the interpretability of models applied to the real estate market.

In recent years, machine learning techniques for predicting real estate prices have significantly advanced, enhancing the models’ accuracy and efficiency. However, critical challenges remain that restrict these methods’ interpretability, generalizability, and robustness, particularly in situations with vast data volumes and high market heterogeneity. One major gap in the literature is the identification, selection, and optimization of relevant variables for predictive models in real estate. Although advancements in machine learning have enabled the development of more sophisticated approaches, challenges remain in integrating multiple data sources and addressing biases stemming from the structural complexity of real estate markets.

In recent years, machine learning techniques for real estate price prediction have advanced significantly, improving the accuracy and efficiency of predictive models. Despite the increasing adoption of machine learning methods in forecasting prices and behaviors within the real estate market, critical challenges persist that limit the interpretability, generalizability, and robustness of these models [26]. These challenges are especially pronounced in contexts characterized by large data volumes, high market heterogeneity, and structural complexity in the factors influencing property value [20].

One of the main knowledge gaps identified in the current literature is selecting and optimizing relevant features [21]. This issue not only compromises model accuracy but also constrains the replicability and transferability of models to other geographical or socioeconomic contexts. The literature indicates that, although there are algorithms capable of ranking variables to some extent, these approaches still face limitations when addressing nonlinear interactions, latent variables, and multiscale contextual effects [38].

Another critical challenge is integrating multiple data sources, such as urban cadastres, public records, satellite imagery, socioeconomic indicators, and transactional databases [44]. This integration requires advanced data fusion, normalization, and hierarchical modeling techniques, which have not yet been standardized or empirically validated in the real estate domain. Consequently, structural biases and multicollinearity issues persist, undermining the quality of inferences [54].

This gap in the literature underscores the need for research to develop integrated methodological frameworks that concurrently address feature selection, heterogeneous data integration, and bias mitigation. Such frameworks are essential to ensure more transparent, replicable, and effective predictive models.

In this context, this study tackles this gap by creating a model based on ensemble methods while optimizing the selection of key variables with advanced machine learning techniques. The goal is to enhance predictive accuracy and reduce potential biases, thereby contributing to the development of more robust and generalizable models in real estate valuation.

3. Methodology

This study follows the positivist paradigm, employs a quantitative approach, uses experimental design, and functions at a predictive level. As detailed below, the methodology for developing the machine learning model describes the design of machine learning systems.

3.1. Data Acquisition

The Ames Housing dataset, utilized in various competitions organized by Kaggle (https://www.kaggle.com/datasets/ahmedmohameddawoud/ames-housing-data?select=Ames_Housing_Data.csv, accessed on 20 October 2024), was employed. This dataset contains structured information on 2930 real estate transactions recorded in Ames, Iowa. Eighty-one variables are included, 79 of which pertain to structural, spatial, and qualitative attributes of the properties, while the dependent variable (SalePrice) represents the sales price in U.S. dollars. The included variables described essential aspects of the real estate market, such as the location and zoning of the lot, structural characteristics of the building, distribution of living space, and availability of additional amenities, including garages, swimming pools, and fireplaces. Details on building materials and remodeling were also captured. It was noted that the SalePrice variable exhibited an asymmetric distribution, necessitating the application of mathematical transformations to enhance the stability and fit of the regression models. Methodologically, we dealt with a high-dimensional dataset, which facilitated the implementation of advanced feature selection and dimensionality reduction techniques.

3.2. Preprocessing

An exploratory analysis was carried out with the functions pandas.info() and pandas.describe(). These tools allowed the identification of null values, data type inconsistencies, and possible distribution anomalies. For data integration, merging and concatenation techniques were implemented, ensuring structural coherence through standardizing variable names and types [62]. Missing values were managed through elimination and imputation strategies with SimpleImputer, applying methods based on the mean, median, or mode, depending on the nature of each variable. Outlier detection was performed using Z-score and interquartile range, while redundant records were eliminated using drop_duplicates(), ensuring the integrity of the dataset [63].

Categorical variables were transformed using one-hot encoding through OneHotEncoder and pd.get_dummies(), optimizing their representation for integration into machine learning models. Normalization and scaling techniques were utilized to enhance data homogeneity: StandardScaler for standardization. These strategies enhanced the model’s stability and reduced the impacts of data heterogeneity, ensuring more accurate and robust predictions for estimating real estate prices.

3.3. Selection Methods

The importance of the features was evaluated using the varImp() function in the Python language (Version: 3.11.12), which employs the RF algorithm to estimate the relative contribution of each variable in the model prediction. This method allowed us to identify influential characteristics by randomly permuting each variable and measuring the decrease in accuracy or increase in impurity. The RFE technique was implemented to optimize the variable selection using sklearn.feature_selection.RFE, which iteratively eliminates those variables with lower relevance until an optimal subset [64,65] is reached. Additionally, the Boruta algorithm was incorporated using BorutaPy, an extension of RF based on statistical hypothesis testing for feature selection. Boruta compares each variable with randomized synthetic attributes (shadow features), providing a robust criterion for their relevance. Its ability to capture nonlinear interactions and relationships makes it a key tool for improving the interpretability and generalization of models in high-dimensional environments [66]. In constructing decision trees within the RF model, the quality of each partition was evaluated using impurity metrics, which quantify the reduction of heterogeneity after each split. This criterion ensured that the groups formed were as homogeneous as possible, optimizing the model’s predictive capacity. In this context, the impurity function $i\left(t\right)$ measures the heterogeneity of the node $t$, and the optimal partition is selected by maximizing the impurity reduction, given by the following equation:

$$\mathsf{\Phi }\left(s,t\right)=i\left(t\right)+P_{R^{i\left(t_R\right)}}-P_{L^{i\left(t_L\right)}}$$

where

• $\mathsf{\Phi }\left(s,t\right)$ is the goodness of partitioning at node t using the partitioning criterion s.
• $i\left(t\right)$ is the impurity of the parent node.
• $i\left(t_R\right)$ and $i\left(t_L\right)$ are the impurities of the child nodes after partitioning.
• $P_R$ and $P_L$ represent the proportions of data assigned to the right and left nodes, respectively.

The most commonly used impurity functions in decision trees are as follows:

Gini index (Gini(t)):

The Gini index measures the probability that an element is incorrectly classified if it is randomly chosen according to the distribution of classes in the node.

It is calculated as follows:

$$Gini\left(t\right)=1-\sum _{i=1}^c{p}_i^2$$

where $p_i$ is the proportion of elements of the class $t$ in the node.

Entropy (H(t)):

Entropy measures the amount of information held in the node.

It is defined as follows:

$$H\left(t\right)=-\sum _{i=1}^c{p}_i{\log }_2{p}_i$$

Evaluating the goodness of partitioning and impurity in RF allows optimal variable selection using well-founded mathematical criteria. Methods such as RFE and Boruta use this structure to select the most relevant variables, optimize predictive models, and reduce the problem’s dimensionality.

3.4. Learning Algorithm Selection

Ensemble models are chosen for land price prediction because they can improve the accuracy and robustness of predictions by combining the results of multiple base models. This is especially advantageous in applications such as real estate appraisal, where data can have high variability and nonlinear characteristics.

3.4.1. AdaBoost

AdaBoost for regression, known as AdaBoost.R, extends the boosting methodology by minimizing a continuous error instead of a binary ranking function [67]. At each iteration, a weak regressor is trained on a dynamically adjusted weight distribution according to the absolute or quadratic error of the prediction [68]. The error is normalized and used to compute a weighting coefficient, where regressors with lower error have a more significant influence on the final combination [69]. AdaBoost (Adaptive Boosting) optimizes the error by weighting instances that are difficult to classify. In regression, an exponential cost function is minimized by iteratively updating weights:

$$F_m\left(x\right)={\sum }_{m=1}^M{\alpha }_m{h}_m\left(x\right),$$

where $h_m\left(x\right)$ are base estimators (commonly weak regressors such as decision trees), and ${\alpha }_m$ represents their weighting coefficients.

3.4.2. Gradient Boosting

Gradient Boosting is a boosting-based machine learning method that optimizes an additive model by minimizing the loss function using downward gradients in the functional space [70]. It aims to build a strong model as a sequential combination of weak models, where each new model is designed to correct the errors of the previous models as follows [71]:

$$F_m\left(x\right)=F_{m-1}+{\gamma }_m{h}_m\left(x\right)$$

where ${\gamma }_m$ is the learning coefficient obtained by minimizing the following:

$${\gamma }_m=arg\underset{\gamma }{\min }{\sum }_{i=1}^{n}L(y_i,F_{m-1}\left(x_i\right))+\gamma h_m(x_i)$$

Unlike AdaBoost, which adjusts sample weights according to their error, Gradient Boosting builds a sequential model by fitting each new estimator to the residual gradients. This formulation allows flexibility using various loss functions, such as mean square error for regression or classification [59].

3.4.3. Random Forest Regressor

Random Forest Regressor is a decision tree-based ensemble method that improves prediction accuracy and stability by combining multiple trees trained on random subsets of the data [57]. The central idea of RF is to reduce the variance of individual models by taking advantage of the diversity of multiple predictors, which makes it less prone to overfitting compared to a single decision tree as follows [56]:

$$F\left(x\right)={\displaystyle \frac{1}{T}}\sum _{t=1}^T{h}_t\left(x\right)$$

where $h_t\left(x\right)$ are individual regression trees trained with bootstrap sampling.

3.4.4. Extra Trees Regressor

Extra Trees Regressor is a variant of RF that introduces more randomization into the construction of the trees, reducing the variance of the model and improving its robustness to noise in the data [72]. Unlike RF, where the split thresholds in each node are selected by optimization, Extra Trees assigns these thresholds completely randomly within the subset of selected characteristics.

$$F(x)={\displaystyle \frac{1}{B}}\sum _{t=1}^B{h}_b\left(x\right)$$

The function $h_b\left(x\right)$ denotes the prediction made by the b-th tree. This strategy introduces a higher bias compared to RF but reduces the variance and correlation between the trees, improving generalizability.

3.4.5. Bagging Regressor

Bagging Regressor is an ensemble method that decreases the variance of the base models by training them on multiple subsets of data generated by sampling with replacement and combining their predictions [73]. Each base model $h_b\left(x\right)$ is fitted independently, and the final ensemble prediction is obtained as the average of the individual predictions as follows:

$$\widehat{y}={\displaystyle \frac{1}{B}}\sum _{b=1}^B{h}_b\left(x\right)$$

where each $h_b\left(x\right)$ is a base model trained on a random subset of the data.

The relationship explains the impact of Bagging on variance reduction:

$$Var\left(\widehat{y}\right)={\displaystyle \frac{1}{B}}Var\left(h_b\right)+\left(1-{\displaystyle \frac{1}{B}}\right)Cov(h_b,h_{b^{\prime }})$$

where the decrease in variance is more significant when the model’s $h_b$ values are less correlated. Bagging improves model stability and generalizability, reducing overfitting without significantly increasing bias [58].

3.4.6. Stacking Regressor

Stacking Regressor is an ensemble method that combines multiple regression models in a two-level hierarchical architecture to improve predictive ability [74]. Its theoretical foundation lies in the optimal combination of base estimators using a meta-model that learns to correct its biases and exploit its strengths:

$$\widehat{y}=g\left(h_1\left(x\right),h_2\left(x\right),...,h_M\left(x\right)\right),$$

where $g$ is a meta-model trained with the outputs of the base regressor s$h_m\left(x\right)$. Stacking can be seen as an optimal predictor combination problem, where the meta-model learns to minimize the loss function L (y,$\widehat{y}$) more efficiently than a simple aggregation by averaging (as in Bagging) or weighting (as in Voting). Its flexibility allows the integration of heterogeneous models with different biases and variances, achieving a more robust ensemble [75].

3.4.7. Voting Regressor

Voting Regressor is an ensemble method that combines the predictions of multiple regression models to improve stability and generalizability [76,77]. Its underlying principle is that by merging several estimators with different biases and variances, a more robust prediction is obtained that is less sensitive to fluctuations in the training data. Voting by simple averaging, one obtains the following:

$$\widehat{y}={\displaystyle \frac{1}{B}}\sum _{b=1}^B{h}_b\left(x\right)$$

where all contributions have the same weight.

Weighted voting:

$$\widehat{y}=\sum _{b=1}^B{w}_b{h}_b\left(x\right),\sum _{b=1}^B{w}_b=1,$$

Each model $h_b$ receives a weight $w_b$ proportional to its performance, which is usually determined by validation metrics such as the coefficient of determination ${\mathbb{R}}^2$ [78].

3.5. Model Training

The program was developed using the Scikit-learn (Version: 1.6.1) Python package, employing a rigorous approach centered on ensemble models and k-fold cross-validation (k = 10) to ensure optimal training and evaluation. This alternating procedure offered robust evaluation, minimizing bias and variance related to specific data partitions while reducing the risk of overfitting [79]. A sample split of 70% for training and 30% for testing was employed, ensuring final model evaluation on fully independent data. During training, ensemble models iteratively optimize errors by combining predictions from multiple base models to enhance accuracy and reduce bias. In addition, a systematic hyperparameter optimization process was implemented using techniques such as Grid Search and Randomized Search, integrated into the cross-validation framework. This allowed the identification of optimal parameter combinations that maximize model performance while balancing complexity and generalization capacity.

3.6. Model Evaluation

To evaluate the accuracy, robustness, and computational efficiency of the predictive models for land prices, we employed a set of standard regression performance metrics, including mean squared error (MSE), mean absolute error (MAE), and the coefficient of determination (R²). Additionally, we incorporated Lin’s Concordance Correlation Coefficient (CCC) to assess the degree of agreement between predicted and observed values, capturing both precision and accuracy. To complement the performance analysis, we also reported the execution time of each model as a proxy for computational efficiency, providing a more holistic comparison across different model configurations and feature selection strategies.

3.6.1. Coefficient of Determination (R²)

Statistical metric to evaluate the quality of a regression model. It indicates what proportion of the variability in the dependent variable (y) is explained by the model as a function of the independent variables (X). The formula for R² is as follows:

$${\mathrm{R}}^2=1-\mathrm{S}\mathrm{S}\mathrm{E}/\mathrm{S}\mathrm{S}\mathrm{T}$$

where SSE is the sum of squared errors (${\sum }_{i=1}^n{(y_i-{\widehat{y}}_i)}^2$), which represents the variability not explained by the model; SST is the total sum of squares (${\sum }_{i=1}^n{(y_i-\overline{y})}^2$), representing the total variability in the real data; and $\overline{y}$ is the mean of the real values ($y_i$).

3.6.2. Mean Squared Error (MSE)

Mean Squared Error is an evaluation metric that measures the average squared difference between the actual values ($y_i$) and the model predictions (${\widehat{y}}_i)$; the formula is as follows:

$$MSE\left(y,\widehat{y}\right)={\displaystyle \frac{1}{n}}\sum _{i=1}^n{|y_i-{\widehat{y}}_i|}^2$$

where n is the total number of observations; $y_i$ is the actual value of the i-th observation; and ${\widehat{y}}_i$ is the value predicted by the model for the i-th observation.

3.6.3. Root Mean Squared Error (RMSE)

Root mean squared error is an evaluation metric used mainly in regression models. It represents the square root of the average of the squared errors between the actual values $y_i$ and the predictions of the model ${\widehat{y}}_i$; the formula is as follows:

$$RMSE\left(y,\widehat{y}\right)=\sqrt{{\displaystyle \frac{1}{n}}\sum _{i=1}^n{|y_i-{\widehat{y}}_i|}^2}$$

3.6.4. Mean Absolute Error (MAE)

Mean absolute error is a metric used to evaluate the accuracy of a model in regression tasks. It represents the average of the absolute differences between the actual values $\left(y_i\right)$ and the predictions of the model ${(\widehat{y}}_i)$. The formula is as follows:

$$MAE\left(y,\widehat{y}\right)={\displaystyle \frac{1}{n}}\sum _{i=1}^n|y_i-{\widehat{y}}_i|$$

4. Results and Discussion

Accurate estimation of real estate prices is crucial for investment and urban planning [16,17], given the sector’s significance in the global economy [1,15]. Dimensionality reduction improves computational efficiency, enhances model interpretability, and reduces overfitting, resulting in more robust predictions [49,50]. The comparative analysis of ensemble models in machine learning and dimensionality reduction techniques, such as RFE, RF, and Boruta [65,66], shows that effective feature selection maximizes the predictive accuracy and stability of the model, confirming its efficacy in predicting real estate prices.

A dataset from the Kaggle machine learning repository (https://www.kaggle.com/datasets/ahmedmohameddawoud/ames-housing-data (accessed on 20 October 2024) was utilized. The Ames Housing dataset, which is well known in the scientific literature and machine learning competitions, contains detailed information on 2930 homes in Ames, Iowa, described by 81 variables, including structural, spatial, and contextual characteristics. A thorough process of preprocessing and data cleaning was implemented to predict the properties’ sale prices. First, the dataset’s structure was examined to evaluate the types of variables and the existence of missing values. A threshold of 5% was set for the removal of variables with a high proportion of missing data, ensuring a balance between reducing bias and preserving information. Consequently, the database was narrowed down to 69 variables.

To mitigate the impact of null values, we applied differentiated imputation strategies. We used the median to reduce sensitivity to outliers for numerical variables, while for categorical variables, we employed the mode to maintain class coherence. Additionally, we identified and removed duplicate records to minimize potential redundancies in the analysis. We transformed categorical variables using one-hot encoding, ensuring appropriate numerical representation and avoiding collinearity issues by excluding the first category of each variable. Next, we performed a normalization process for the numerical variables using StandardScaler, ensuring each variable had a distribution with a zero mean and a unit standard deviation. Data preprocessing resulted in a final dataset of 2930 records and 227 variables derived from the transformation of categorical variables through one-hot encoding. This strategy provided a structured and fitting representation for subsequent analysis, facilitating the application of feature selection techniques and machine learning models with an optimized and uniform database.

The research evaluated the impact of the RFE algorithm in conjunction with an RF classifier to identify the most relevant variables in a structured dataset. The results indicated that the progressive elimination of irrelevant features reduced the complexity of the model without significantly affecting its accuracy. The selected features had the highest contribution to predicting the target variable. Additionally, an importance ranking was generated, allowing visualization of the relative influence of each variable in the model, as shown in Figure 1.

A database was reduced to fifteen variables after conducting attribute engineering (feature selection). For the comparative analysis, seven widely recognized ensemble models were evaluated: AdaBoost, Gradient Boosting, Random Forest, Extra Trees, Bagging, Stacking, and Voting. These were chosen for their ability to enhance prediction accuracy by combining multiple base models, thus optimizing the robustness and generalization of the results.

The predictive performance of each model was quantified using standard metrics for regression problems. Among these metrics, the coefficient of determination (R²) indicated the proportion of variability in the data explained by the model, serving as a crucial indicator of its fit. The root mean square error (RMSE) directly measures the magnitude of error and is expressed in the same units as the dependent variable, allowing for intuitive interpretation of precision. The mean absolute error (MAE) assesses the average absolute deviation between predictions and actual values, providing a complementary metric that does not excessively penalize extreme errors. Lastly, the mean squared error (MSE) was used to impose stricter penalties on significant errors, offering additional insight into model quality by identifying highly discrepant predictions.

The analysis of these metrics enabled a thorough comparison of the models, revealing their strengths and limitations in predicting real estate prices. The observed differences underscore the importance of choosing the correct algorithm based on the problem’s specific characteristics and the analysis goals.

Table 1. Performance of machine learning ensemble models without variable selection and with variable selection through RFE.

Model	Features	MAE	MSE	RMSE	R2	CCC	Time (seg)
AdaBoost	227	23,630	1.044 × 109	32,320	0.851	0.918	1.709
RFE + AdaBoost	15	25,390	1.174 × 109	34,260	0.833	0.904	0.757
Gradient Boosting	227	14,540	5.643 × 108	23,750	0.920	0.958	1.756
RFE + Gradient Boosting	15	17,450	6.754 × 108	25,990	0.904	0.947	1.231
Random Forest	227	15,190	6.066 × 108	24,630	0.914	0.953	6.445
RFE + Random Forest	15	17,540	7.455 × 108	27,300	0.894	0.943	2.992
Extra Trees	227	15,480	6.574 × 108	25,640	0.906	0.945	5.409
RFE + Extra Trees	15	16,760	6.783 × 108	26,040	0.904	0.946	1.100
Bagging	227	15,130	6.391 × 108	25,280	0.909	0.950	1.964
RFE + Bagging	15	17,430	7.575 × 108	27,520	0.892	0.939	0.761
Stacking	227	14,090	5.338 × 108	23,100	0.924	0.960	67.231
RFE + Stacking	15	16,510	6.473 × 108	25,440	0.908	0.951	20.946
Voting	227	15,720	5.971 × 108	24,440	0.915	0.953	13.433
RFE + Voting	15	17,510	6.940 × 108	26,340	0.901	0.944	4.752

presents the results for each model, along with a critical analysis that evaluates their relative performance in terms of accuracy, robustness, and generalizability. This analysis establishes a solid foundation for identifying the optimal configurations for practical applications and future research.

This study evaluated the performance of various machine learning models in predicting land prices and analyzed the impact of dimensionality reduction using Recursive Feature Elimination (RFE). Baseline models employing 227 features were compared with their optimized counterparts, in which 15 features were selected through RFE.

The results indicate that the Stacking model achieved the highest overall perfor-mance, with a Mean Absolute Error (MAE) of 14,090, a Mean Squared Error (MSE) of 5.338 × 10⁸, a Root Mean Squared Error (RMSE) of 23,100, an R2 of 0.924, and a Lin’s Concordance Correlation Coefficient (CCC) of 0.960. It was closely followed by Gra-dient Boosting, which recorded an MAE of 14,540 and an R2 of 0.920, highlighting strong predictive accuracy and generalization capabilities. In contrast, the Adaboost and RFE + Adaboost models demonstrated the weakest performance, with MAE values exceeding 23,000 and R2 scores below 0.85, suggesting difficulties in adequately fitting the training data.

Including the CCC metric provided a comprehensive assessment of both the precision and accuracy of the models’ predictions, while execution time served as a robust indicator of computational efficiency. In this regard, models such as RFE + Bagging (0.76 s) and RFE + AdaBoost (0.76 s) demonstrated significantly reduced computation times compared to more complex models such as Stacking (67.23 s) and Voting (13.43 s).

Although dimensionality reduction enhances efficiency and interpretability, the findings suggest that feature elimination through RFE did not consistently improve predictive accuracy. Specifically, the Gradient Boosting model without RFE significantly outperformed its reduced counterpart, achieving a 16.9% reduction in MAE and a 1.6% increase in R². A similar pattern was observed in the Stacking model, where the full-feature version attained a 14.6% lower MAE than its RFE-optimized version, implying that the removed features may have contained relevant information for the prediction task.

This trend was also observed in models such as Random Forest, Extra Trees, Bagging, and Voting, where RFE increased prediction errors (MAE, MSE, and RMSE) and led to a decline in R². This confirms that these algorithms benefit from access to more input features. Nevertheless, dimensionality reduction resulted in significant improvements in execution time, which is a critical advantage in resource-constrained environments.

From a computational perspective, RFE-based feature selection can be an effective strategy for balancing accuracy and efficiency. In scenarios where model interpretability is crucial—such as in regulatory environments or explanation-driven decision-making—limiting the input to 15 features enhances the understanding of each variable’s impact on the model without significantly compromising predictive performance.

The method outlined involves calculating the significance of each variable using RF. Figure 2 illustrates the results of the variable selection.

The results obtained in the evaluation of different ensemble algorithms for price prediction show apparent differences in their performance, according to the indicators of root mean square error (RMSE), explained variance (EV), mean absolute error (MAE), and coefficient of determination (R²). The results are presented in

Table 2. Performance of machine learning ensemble models without variable selection and with variable selection by RF.

Model	Features	MAE	MSE	RMSE	R2	CCC	Time (seg)
AdaBoost	227	23,630	1.044 × 109	32,320	0.851	0.918	1.709
RF + AdaBoost	16	24,370	1.124 × 109	33,530	0.840	0.901	0.705
Gradient Boosting	227	14,540	5.643 × 108	23,750	0.920	0.958	1.756
RF + Gradient Boosting	16	17,310	6.725 × 108	25,930	0.904	0.947	1.261
Random Forest	227	15,190	6.066 × 108	24,630	0.914	0.953	6.445
RF + Random Forest	16	17,170	7.265 × 108	26,950	0.897	0.944	2.057
Extra Trees	227	15,480	6.574 × 108	25,640	0.906	0.945	5.409
RF + Extra Trees	16	16,780	6.684 × 108	25,850	0.905	0.945	1.109
Bagging	227	15,130	6.391 × 108	25,280	0.909	0.950	1.964
RF + Bagging	16	17,650	7.408 × 108	27,220	0.895	0.939	0.742
Stacking	227	14,090	5.338 × 108	23,100	0.924	0.960	67.231
RF + Stacking	16	16,590	6.388 × 108	25,280	0.909	0.951	22.465
Voting	227	15,720	5.971 × 108	24,440	0.915	0.953	13.433
RF + Voting	16	17,390	6.870 × 108	26,210	0.902	0.944	4.005

.

This study evaluated the performance of various machine learning models in predicting real estate prices. It compared full-feature versions (227 features) with their respective reduced versions obtained through Random Forest-based feature selection (RF-FS), which limited the input space to 16 variables. The analysis focused on the trade-off between predictive accuracy and computational efficiency resulting from dimensionality reduction.

Among all models, Stacking achieved the best overall performance, with a Mean Absolute Error (MAE) of 14,090, a Mean Squared Error (MSE) of 5.34 × 10⁸, a Root Mean Squared Error (RMSE) of 23,100, and a coefficient of determination (R²) of 0.924, along with Lin’s Concordance Correlation Coefficient (CCC) of 0.960, confirming its robustness and excellent generalization capacity. The Gradient Boosting model followed closely, with an MAE of 14,540 and an R² of 0.920, further supporting its high predictive power.

The impact of RF-based dimensionality reduction varied significantly across models. In the case of Gradient Boosting, the RF-optimized version exhibited a 19.1% increase in MAE and a 1.6% reduction in R² compared to its full-feature counterpart. A similar pattern was observed in the Stacking model, where the reduced version resulted in a 17.7% increase in MAE, suggesting that key predictive variables may have been excluded during feature selection.

Other ensemble models, such as Random Forest and Extra Trees, also experienced decreased accuracy after dimensionality reduction. Specifically, RF + Random Forest increased MAE by 13.1%, while RF + Extra Trees increased MAE by 8.3%, indicating that reducing the feature space negatively affected these models’ predictive capabilities. Conversely, RF + AdaBoost slightly outperformed the full-feature AdaBoost version, yielding a 5.3% reduction in MAE, which suggests that RF-FS may help mitigate overfitting and enhance performance in models prone to overparameterization.

From a computational perspective, dimensionality reduction through RF-FS led to significant reductions in execution time across all models. For instance, RF + Bagging and RF + AdaBoost required only 0.74 s and 0.70 s, respectively, compared to 1.96 s for full Bagging and 1.71 s for full AdaBoost. The computational gains were even more pronounced in complex models such as Stacking, where runtime decreased from 67.23 s to 22.47 s after feature selection, and Voting, from 13.43 s to 4.00 s.

Despite occasional losses in predictive accuracy, the RF-FS optimized models maintained acceptable performance with only 16 features, compared to their original versions with 227 features. These findings highlight the value of RF-based feature selection as a method to reduce computational complexity and enhance interpretability, particularly in resource-constrained environments or fields where explainability is crucial.

Concerning the Boruta model, 16 variables were selected based on the ranking established, as shown in Figure 3.

Similar to the prior models, the performance of several machine learning models in predicting land prices was assessed using metrics such as mean absolute error (MAE), mean squared error (MSE), root mean squared error (RMSE), and the coefficient of determination (R²), as shown in

Table 3. Performance of machine learning ensemble models without variable selection and with variable selection using Boruta.

Model	Features	MAE	MSE	RMSE	R2	CCC	Time (seg)
AdaBoost	227	23,630	1.04 × 109	32,317	0.851	0.918	1.710
Boruta + AdaBoost	16	23,529	1.04 × 109	32,277	0.852	0.920	0.560
Gradient Boosting	227	14,537	5.64 × 108	23,755	0.920	0.958	1.760
Boruta + Gradient Boosting	16	16,073	6.53 × 108	25,555	0.907	0.950	0.830
Random Forest	227	15,186	6.07 × 108	24,629	0.914	0.953	6.440
Boruta + Random Forest	16	15,841	6.86 × 108	26,195	0.902	0.949	1.750
Extra Trees	227	15,476	6.57 × 108	25,639	0.907	0.945	5.410
Boruta + Extra Trees	16	15,899	7.23 × 108	26,893	0.897	0.946	1.030
Bagging	227	15,134	6.39 × 108	25,281	0.909	0.950	1.960
Boruta + Bagging	16	15,576	6.87 × 108	26,204	0.902	0.946	1.120
Stacking	227	14,092	5.34 × 108	23,104	0.924	0.960	67.230
Boruta + Stacking	16	15,472	6.45 × 108	25,401	0.908	0.951	21.970
Voting	227	15,724	5.97 × 108	24,436	0.915	0.953	13.430
Boruta + Voting	16	16,228	6.58 × 108	25,658	0.906	0.949	3.760

.

This study systematically evaluates the influence of applying the Boruta algorithm for feature selection across various ensemble-based machine learning models. The original models were trained using 227 predictors, while their optimized versions utilized a reduced subset of 16 variables selected by Boruta as the most relevant. The assessment focused on predictive accuracy—measured through MAE, MSE, RMSE, R², and the Concordance Correlation Coefficient (CCC)—and computational efficiency, represented by training time.

The findings indicate that Boruta achieved a 92.9% reduction in the number of variables without significantly compromising predictive performance. In most models, both R² and CCC remained close to the values obtained with the full feature set, thereby highlighting the effectiveness of the feature selection process in maintaining predictive quality with a more compact dataset.

The Stacking model emerged as the top performer in both configurations. The full model achieved an MAE of 14,092, an RMSE of 23,104, an R2 of 0.9241, and a CCC of 0.960. Its reduced counterpart (Boruta + Stacking) preserved a high level of agreement (CCC = 0.951), despite a modest decline in R2 (0.908) and an increase in MAE (15,472). Notably, computational time decreased from 67.23 to 21.97 s—a reduction of 67.3%—demonstrating that the model retains strong predictive performance even after a significant reduction in dimensionality.

Similarly, Gradient Boosting demonstrated resilience to feature reduction. The CCC slightly decreased from 0.958 to 0.9502, with a marginal increase in MAE (from 14,537 to 16,073) and a controlled decline in R2 (from 0.920 to 0.907). This stability in R2 and CCC underscores Boruta’s ability to preserve key structural relationships between predictors and the target variable.

Conversely, tree-based models such as Random Forest and ExtraTrees exhibited greater sensitivity to feature reduction. In the case of Random Forest, CCC decreased from 0.953 to 0.949, while ExtraTrees saw a drop from 0.945 to 0.946. Although these changes are modest, they indicate that such models benefit from broader feature sets to maintain the diversity of decision trees. The dimensionality reduction may compromise the robustness and consistency of their predictions.

A particularly noteworthy case is the Adaboost model. The optimized version performed nearly identically to its full-feature counterpart, with CCC values of 0.920 and 0.918, MAE values of 23,529 and 23,630, and R2 values of 0.851 and 0.851, respec-tively. This result aligns with existing literature suggesting that Adaboost is sensitive to data noise and prone to overfitting, thereby benefiting from more selective and parsimonious input spaces. Computational time was also substantially reduced—from 1.71 to 0.56 s.

Similarly, the Bagging model demonstrated minimal performance changes following dimensionality reduction. Its CCC slightly decreased (from 0.9496 to 0.9464), accompanied by only moderate increases in MAE and RMSE. However, the training time was nearly halved (from 1.96 to 1.12 s), highlighting the advantages of feature selection in scenarios where time constraints and interpretability are critical.

Overall, incorporating the Concordance Correlation Coefficient (CCC) as an additional evaluation metric provides a more comprehensive understanding of each model’s predictive capacity. Unlike R², which only assesses the proportion of variance explained, CCC captures both precision and accuracy, enabling a more thorough identification of potential biases or systematic misalignments in predictions.

The computational advantages of dimensionality reduction were significant across all models. This increase in efficiency is particularly valuable in situations where rapid model iteration is necessary or where computational resources are limited. Moreover, by concentrating the analysis on a smaller and more informative subset of variables, model interpretability is enhanced, which is a desirable characteristic in high-stakes applications such as real estate valuation or medical diagnosis, where the rationale behind predictions must be clearly justified.

Dimensionality reduction significantly improved computational efficiency. Decreased variables optimized training time and model responsiveness, which is crucial in prediction applications in massive data environments [50]. Dimensionality reduction also effectively mitigated the adverse effects of the exponential increase in complexity as the number of variables in the dataset increased [47].

Feature selection enhances model accuracy by reducing dimensionality and removing irrelevant or redundant variables that introduce noise and increase the risk of overfitting. This process optimizes model variance and boosts generalization capacity, consistent with the bias–variance tradeoff. From a statistical perspective, it improves the signal-to-noise ratio and lowers model complexity, resulting in decreased estimator variance.

However, accuracy may decline if predictively relevant variables are eliminated, introducing bias and resulting in underfitting. Additionally, some selection methods struggle to capture nonlinear relationships or interactions between variables, and if they are not properly integrated within a cross-validation framework, they may lead to overfitting in the training set.

The findings indicate that no single model is universally superior for predicting real estate prices. While Gradient Boosting and Stacking stand out for their accuracy and generalizability, models like Random Forest and Extra Trees offer robustness and stability, especially when utilizing variable selection techniques. Future research could focus on automated hyperparameter optimization and incorporate hybrid methods to improve predictive performance in highly complex situations.

Accurate land value prediction using ensemble machine learning models, supported by feature selection and dimensionality reduction processes, significantly enhances credit risk assessment by providing more precise valuations of mortgage-backed assets. At the same time, it strengthens data-driven urban planning, enabling informed decision-making regarding land use and infrastructure allocation. Integrating dimensionality reduction techniques captures complex interactions among spatial, structural, and socioeconomic variables, thereby increasing model robustness and generalization capacity. This approach positions price prediction not merely as a technical task but as a strategic tool for territorial management and evidence-based decision-making in both public and private sectors.

Dimensionality reduction in real estate price prediction models enhances interpretability and computational efficiency. By reducing the number of features, the most relevant variables, such as location, size, and structural characteristics of properties, become more easily identifiable, thereby improving the model’s transparency and understanding. This is especially valuable for professionals in the real estate sector, such as appraisers and urban planners, who can make more informed decisions based on clear and comprehensible results.

Regarding computational efficiency, dimensionality reduction optimizes training time and resource usage, resulting in faster and more cost-effective models that facilitate implementation in financial institutions and real estate companies. This approach enables quicker and more accurate appraisals, enhancing credit risk assessment and decision-making in urban development. Collectively, these benefits position dimensionality reduction as a crucial factor in improving operational effectiveness and the practical adoption of predictive models in the real estate sector.

5. Conclusions

The results of this study underscore the remarkable effectiveness of ensemble learning models in predicting land prices, with Stacking and Gradient Boosting emerging as the most accurate and robust methods. These models consistently achieved the highest coefficients of determination (R²) and Concordance Correlation Coefficients (CCCs), along with the lowest prediction errors (MAE and RMSE), establishing themselves as reliable tools for capturing complex, high-dimensional patterns. Stacking demonstrated outstanding stability across all scenarios, even after dimensionality reduction using RFE, Random Forest, and Boruta, while maintaining an optimal balance between predictive accuracy and computational efficiency.

The findings also highlight the strategic relevance of dimensionality reduction techniques for enhancing the performance of ensemble models. Methods such as RFE and Boruta reduced the number of predictive features by over 90%, significantly lowering computational complexity while maintaining satisfactory levels of predictive performance. This efficiency was particularly evident in Gradient Boosting and Stacking, where variable reduction led to only marginal losses in accuracy, making these approaches highly suitable for applications that require model scalability, interpretability, and operational speed.

Conversely, models such as AdaBoost, Random Forest, and Extra Trees demonstrated greater sensitivity to feature reduction, highlighting the need to carefully implement these strategies in contexts where information loss could impair model performance. Interestingly, the combination of Boruta with AdaBoost resulted in minimal performance differences between the full and reduced models, suggesting that this pairing may offer valuable advantages in resource-constrained environments.

The practical implications of this research are significant, particularly in real estate valuation, financial risk assessment, and spatial planning, where predictive accuracy and computational efficiency are crucial. Nevertheless, several directions for future work are identified. These include evaluating the generalizability of ensemble models across more diverse datasets spanning different geographic regions and temporal settings, and incorporating advanced hyperparameter optimization strategies to further enhance model performance in a fully automated manner.

Finally, exploring hybrid approaches that combine the strengths of multiple ensemble algorithms and variable selection methods, such as adaptive combinations of Stacking, Voting, and Boosting with Boruta or RFE, represents a promising avenue for developing more robust and flexible methodological frameworks. Such advancements would contribute to the design of high-impact predictive solutions that are particularly suited to real-world contexts characterized by high dimensionality, uncertainty, and the demand for interpretable and reproducible outcomes.