Rethinking Machine Learning Evaluation in Waste Management Research

Abstract

Reliable model evaluation is critical in waste management research, where machine learning is increasingly used to inform policy, circular economy strategies and progress towards the United Nations Sustainable Development Goals. However, common evaluation practices often fail to account for key methodological challenges, risking misleading conclusions. This study presents a theoretical analysis supplemented with a practical example of municipal solid waste generation in Ireland to demonstrate how standard evaluation metrics can produce distorted results. In particular, the widespread use of the $R^2$ in waste management/sustainability machine learning is examined, showing its susceptibility to inflation when data exhibit strong correlations, temporal dependence or non-linear model structures. The findings show that reliance on the $R^2$ misrepresents model performance under conditions typical of waste datasets. In the Irish example, the $R^2$ often suggested a degradation of predictive ability even when error-based metrics, such as root mean squared error (RMSE) and mean absolute error (MAE), indicated improvement or stability. These results demonstrate the need for evaluation frameworks that move beyond single, correlation-based metrics. Future work should focus on developing and standardising robust practices to ensure that machine learning can support transparent, reliable and effective decision-making in waste management and circular economy contexts.

1. Introduction

Machine learning (ML) is a subset of artificial intelligence that enables computers to improve performance on specific tasks by learning from data, without being explicitly programmed [1]. Its ability to generalise from examples has made it a powerful tool for prediction, classification and decision-making across disciplines. With advances in computational power and data availability, ML has increasingly been applied to sustainability research, offering new approaches for modelling resource flows, predicting waste generation and informing circular economy (CE) strategies. Reliable forecasting of municipal solid waste is essential for achieving Sustainable Development Goal (SDG) 12 (responsible consumption and production) and for implementing the 3Rs principle (reduce, reuse, recycle) that underpins CE policy [2].

Municipal solid waste (MSW) presents a particular challenge for both policymakers and researchers. Globally, MSW generation is projected to reach between 2.89 and 4.54 billion tonnes by 2050, driven by urbanisation and economic growth [3]. In the European Union, MSW management is central to achieving CE objectives and compliance with recycling targets set under Directive 2008/98/EC as amended by 2018/851 [4]. Accurate forecasting of MSW is therefore essential for infrastructure planning, investment decisions and monitoring progress towards SDG 12.

As interest in ML grows, more researchers are turning to these methods to address environmental and resource management challenges [5]. However, the rapid uptake of ML has brought methodological risks that can undermine the credibility of findings. A central concern is model evaluation. The choice of evaluation metric directly shapes how results are interpreted, and in sustainability contexts, may influence policy and resource allocation. Misapplied metrics risk producing misleading conclusions and suboptimal decisions.

As highlighted by Pennington et al. [6], to further the use of data science in sustainability applications, focus must be applied to methodological knowledge and underlying assumptions, as well as possible pitfalls that researchers may encounter. While Zhu et al. [7] cautions against relying on a single metric, the limitations of $R^2$ in this context remain largely unexamined. The objective of this study is to provide a critical assessment of the $R^2$ in ML applications to waste management, given that recent studies that forecast MSW using ML techniques report the $R^2$ as an evaluation metric [8,9,10,11,12,13,14].

To this end, three contributions are made. Firstly, a practical example of Irish MSW generation demonstrates how reliance on the $R^2$ can distort conclusions about model performance under the data constraints typical in this field. Secondly, the theoretical limitations of the $R^2$ are examined, with emphasis on why its assumptions are frequently violated in waste management and sustainability applications. Thirdly, the broader implications for researchers and policymakers are considered, with recommendations made on evaluation metrics and practices better suited to sustainability data. While the statistical limitations of the $R^2$ are documented in the statistical contexts, the statistic is yet to be criticised in the ML literature. The instability of the statistic has not been systematically linked to the data characteristics typical of waste management ML. The contribution of this study is a theoretical analysis showing how these features undermine the stability of the $R^2$ for ML waste management research, supported by an illustrative Irish MSW example and a practical workflow for more reliable model evaluation in sustainability applications.

2. Evaluation Metrics in the Literature

This section reviews the evaluation metrics most commonly used in waste management-related ML research, with particular attention to MSW studies. Across the 18 studies identified, the three dominant metrics were the coefficient of determination ($R^2$), the root mean squared error (RMSE) and the mean absolute error (MAE). Their prevalence is consistent with broader patterns in environmental ML applications [7] and with findings from Allen et al. [15], who observed similar metric choices in ML work aligned with the SDGs. Thus, while this section focuses on MSW forecasting, the observations made here are likely generalisable across sustainability research. Table 1 summarises the metrics used in each study. Only one of the 18 did not report at least one of $R^2$, RMSE, or MAE [16]. It is important to emphasise that Table 1 reflects the popularity of metrics rather than their validity. Frequency of use reflects convention rather than validation and therefore does not constitute evidence of their suitability for waste management applications.

Although the $R^2$, RMSE and MAE remain the dominant metrics in waste management ML, recent developments in probabilistic and interval-based model evaluation offer alternative perspectives on predictive performance. These include prediction interval metrics [17], as well as probabilistic scoring rules such as the continuous ranked probability score (CRPS), which has been refined and operationalised in recent statistical and ML literature [18]. Quantile-based scoring rules, including the pinball loss used in distributional and multi-horizon forecasting, have also gained increasing attention in ML contexts [19]. While such approaches are not yet common in waste management ML, they represent promising directions for future methodological development, particularly where predictive uncertainty is relevant to decision-making.

Table 1. References to the metrics used in the identified literature.

Metrics	References
$R^2$	[8,9,10,11,12,13,14,20,21,22,23,24,25,26,27]
RMSE	[8,10,12,13,14,20,23,25,26,28]
MAE	[10,13,14,20,22,23,25,27,28,29]

Table 1. References to the metrics used in the identified literature.

Metrics	References
$R^2$	[8,9,10,11,12,13,14,20,21,22,23,24,25,26,27]
RMSE	[8,10,12,13,14,20,23,25,26,28]
MAE	[10,13,14,20,22,23,25,27,28,29]

While the focus of this research is on waste management, the $R^2$ has been used to evaluate machine learning models across sustainability and environmental research. Recent reviews in areas such as water quality/hydrology [30,31], air quality [32], renewable energy [33] and ecological modelling [34] have all documented the use of the $R^2$. Despite its prevalence, these reviews do not assess the suitability of the $R^2$ itself. This further highlights the need for work that examines the use of the $R^2$ in applied modelling contexts, such as waste management.

2.1. The $R^2$

The coefficient of determination ($R^2$) is a measure used to assess the goodness of fit of a model. For linear regression models fitted by ordinary least squares, it quantifies the proportion of variance in the dependent (what is being predicted) explained by the covariates (inputs to the models). Hence, it should take a value between 0 and 1 [35]. The formula for the $R^2$ is shown in Equation (1).

$$R^2=1-{\displaystyle \frac{{\sum }_i{(y_i-\widehat{y_i})}^2}{{\sum }_i{(y_i-\overline{y})}^2}}.$$

(1)

where $y_i$ are the dependent values, $\widehat{y_i}$ are the values predicted by the model (fitted values) and $\overline{y}$ is the mean of the dependent.

The $R^2$ is used extensively in research applying ML to waste management. Of the 18 studies identified, 15 of them used the $R^2$ for evaluating their ML models. This makes it the most popular metric for assessing ML models among the studies identified. However, as will be discussed in Section 5, the application of $R^2$ in ML has significant limitations that can lead to misleading conclusions.

2.2. Error-Based Metrics

Error-based metrics provide a direct measure of prediction accuracy, which is crucial for assessing real-world applicability. Of the error metrics used in the literature, the two most common were the root mean squared error (RMSE) and mean absolute error (MAE), which both appeared 10 times in the 18 studies considered. The RMSE measures the average magnitude of the residuals (errors) [36]. It is defined in Equation (2). The MAE is defined in Equation (3) and measures the average magnitude of residuals, without consideration of their direction. RMSE is more sensitive to large errors due to the squared residuals, while MAE provides a more balanced assessment by averaging absolute errors. Both RMSE and MAE are on the same scale as the dependent, allowing for interpretation of the errors in model prediction. RMSE is defined in Equation (2) while MAE is defined in Equation (3).

$$\mathrm{RMSE}=\sqrt{{\displaystyle \frac{1}{n}}\sum _{i=1}^n{(y_i-{\widehat{y}}_i)}^2}$$

(2)

$$\mathrm{MAE}={\displaystyle \frac{1}{n}}\sum _{i=1}^n|y_i-{\widehat{y}}_i|$$

(3)

In Equations (2) and (3), n is the number of predictions made by the model, $y_i$ is the ith dependent value and ${\widehat{y}}_i$ is the ith predicted value.

Other error-based metrics were also used in the literature; however, they were not as common as the RMSE and MAE. Examples include the mean absolute percentage error (MAPE) and the mean squared error (MSE, which is the same as the RMSE, without the square root). The MAPE and MSE were used six and four times, respectively.

2.3. Criticism of the $R^2$ in the Literature

Numerous publications have outlined the limitations and problems associated with using $R^2$s for non-linear regression models [37,38,39,40,41,42]. Many of these publications are from the 20th century, and it seems their lessons have been forgotten. More recently, Speiss and Neumeyer [43] used a Monte Carlo approach to demonstrate the inadequacy of $R^2$s for selecting non-linear models in a pharmacological setting. These authors found that the $R^2$ struggled to select the optimal model and was biassed towards models with more parameters.

Despite these observed limitations in a more classical regression setting, there is a surprising lack of research addressing the misuse of the $R^2$ in machine learning, particularly within the context of waste management.

3. Methods

Having identified and discussed three commonly used evaluation metrics in waste management ML research, their appropriateness is examined through a practical example on Irish MSW. The MSE is also included for completeness. The purpose of this example is not to provide an exhaustive cross-country analysis but rather to offer a representative case that mirrors the type of data, modelling practices and constraints frequently encountered in the literature. The structural issues that affect the behaviour of the $R^2$ are widespread across sustainability and waste management datasets. The Irish case therefore serves as an illustrative example of broader methodological challenges.

In this section, four models are constructed to predict municipal waste generation in Ireland using socio-economic variables as covariates. These models include a mean baseline model that always predicts the training mean, a naive baseline model that predicts the last observed value, a support vector machine (SVM), and a neural network. The comparison of these models highlights the importance of selecting appropriate evaluation metrics and demonstrates that the $R^2$ can behave inconsistently and misleadingly when applied to ML models. All data preparation, modelling and evaluation procedures were conducted in R version 4.4.1 [44].

3.1. Data Acquisition and Processing

Annual Irish municipal solid waste (MSW), recorded in kilograms per capita, was obtained from Eurostat [45]. The dataset comprises 19 observations for 2000–2020, with values for 2013 and 2015 missing. Eurostat is the source used in many MSW forecasting studies identified in Section 2 [11,12,21,22]. The two missing values were linearly interpolated prior to splitting the data into training and test sets, following common practice for addressing small, isolated gaps in time series data [24,46]. Because these are nationally compiled statistics used for EU target monitoring, alternative values cannot be obtained, and interpolation ensures continuity of the series. The full 2000–2020 MSW trajectory, which serves as the dependent variable in the modelling, is shown in Figure 1. The length of this time series is typical of nationally reported MSW datasets used in the literature [8,9,23].

Figure 1. Plot of annual municipal solid waste generation of Ireland [45].

The covariates selected were gross domestic product (GDP), population and unemployment [47,48,49]. More details on the covariates can be seen in Table 2. These covariates were chosen as they capture socio-economic and environmental influences on waste. The studies identified in Section 2 frequently use such covariates when building their models [8,9,11,12,21,23,27].

Table 2. Explanation of variables used for modelling.

Variable	Units	Source
Municipal waste	kilograms/capita	[45]
GDP	constant to 2015 US$	[47]
Population	total population	[48]
Unemployment	% of total labour force	[49]

At this stage, trend and multicollinearity analyses are deliberately omitted, as is commonly executed in the 18 studies identified in Section 2. These analyses will be conducted and discussed in Section 5.

Prior to training the models, the data was split into training and test datasets according to a chronological 70/30 split. The models were trained on the years 2000–2016 and tested on the years 2017–2020. A 70/30 split was chosen, as this is a common and well-accepted train/test split [50,51]. When the dependent data is recorded over time (also known as a time series [52]), it is best practice to employ a chronological split as opposed to a random split. Random splits in time series can lead to future values influencing past predictions, which is unrealistic in real-world forecasting scenarios [53].

The training data was standardised by subtracting the mean and dividing by the standard deviation. The same scaling was applied to the test data. Each value in the test set was scaled by subtracting the training mean and dividing by the training standard deviation. Standardisation prevents variables with larger magnitudes (e.g., GDP) from disproportionately influencing the model compared to smaller values (e.g., unemployment) [54]. This process is described mathematically in Equation (4). Standardisation was used by studies identified in Section 2 [9,14].

$$x_{i,j}^{*}={\displaystyle \frac{x_{i,j}-{\overline{x}}_j^{\mathrm{train}}}{{\sigma }_j^{\mathrm{train}}}}$$

(4)

where $x_{i,j}$ is the ith observation of the jth variable, ${\overline{x}}_j^{\mathrm{train}}$ is the training mean of the jth variable and ${\sigma }_j^{\mathrm{train}}$ is the training standard deviation of the jth variable.

3.2. Mean Baseline Model

The mean baseline model is a simple reference model that predicts all future values as the average of the training data. It does not account for variation over time or relationships with covariates but instead assumes that the mean is the best predictor. Its purpose is to provide a lower bound for model performance. Any predictive model should outperform the baseline model if it is capturing meaningful patterns in the data. By including it, distinctions can be made between genuine predictive ability and performance that simply reflects the central tendency of the dataset.

3.3. Naive Baseline Model

The naive baseline model predicts each future value as equal to the most recent observed value. For example, this model will predict the 2019 value as being the 2018 value. In the context of time series data, this is equivalent to a random walk benchmark, making it a strong reference model when the dependent variable exhibits trend or persistence.

3.4. Support Vector Machine

An SVM was built using the ‘e1071’ package in R version 4.4.1 [55]. It took the variables in Table 2 as covariates and predicted MSW. The data was scaled prior to the modelling (as described in Section 3.1). A radial basis kernel was employed, based on choices made in the literature [9,11,20]. This kernel is defined in Equation (5).

$$K(x_i,x_j)=exp(-\gamma \parallel x_i-x_j{\parallel }^2).$$

(5)

Given the limited sample size, we adopted a conservative hyperparameter specification. The SVM hyperparameters were set as follows: The gamma value was 1/3, corresponding to the inverse of the number of predictors and determining the influence radius of individual data points. The cost value was set to 1, which balances model complexity against margin violations. The epsilon was set to 0.1, defining the width of the insensitive zone in regression. These choices represent values that prioritise generalisation and avoid the risk of overfitting. To ensure robustness, alternative hyperparameter values were also explored, and the results of the analysis remained consistent. The SVM was chosen as it was commonly used in the literature identified in Section 2 [8,9,11,20,25].

3.5. Neural Network

A feedforward neural network was built using the ‘neuralnet’ package in R [56]. It took the variables in Table 2 as covariates and predicted MSW. The data was scaled prior to the modelling (as described in Section 3.1). Given the limited sample size, a simple architecture was used to avoid overfitting. The network had 1 hidden layer with 2 neurones. The activation function was a logistic function. The hyperparameters were as follows, and were chosen to reduce the risk of the model overfitting: The error threshold was set to 0.01, and the network was trained using resilient backpropagation with the maximum number of steps for the training set to 5000. The number of training repetitions was set to 10. The network was trained with 10 independent random initialisations.

4. Results

The training and test metrics of the mean baseline model, naive baseline model, SVM and neural network can be seen in Table 3. To complement the numerical metrics in Table 3, Figure 2 presents a visual comparison of the test set predictions from each model.

Table 3. Evaluation metrics for models.

Model	Train ${\mathit{R}}^2$	Train RMSE	Train MAE	Train MSE	Test ${\mathit{R}}^2$	Test RMSE	Test MAE	Test MSE
Mean Baseline	0	75.07	68.84	5635.53	−5.02	59.67	54.49	3560.34
Naive Baseline	0.72	39.75	29.19	1580.28	0.39	19.02	17.75	361.75
SVM	0.87	26.76	20.44	716.11	−0.31	27.81	25.24	773.34
Neural Network	0.75	37.85	20.69	1432.51	−0.64	31.17	25.20	971.49

Notes: Lower RMSE, MAE and MSE values indicate better model performance. $R^2$ values closer to 1 indicate better model performance.

Figure 2. Plots of the observed (black line) and fitted (red line) test set values for the (a) mean baseline model, (b) naive baseline model, (c) support vector machine (SVM) model and (d) neural network.

The results in Table 3 highlight several important contrasts between the models. For the mean baseline, the $R^2$ behaves as expected on the training set, taking a value of 0 since the model predicts the training mean. On the test set its $R^2$ falls to −5.02, reflecting the fact that the test mean differs from the training mean. In absolute terms, the RMSE (59.67) and MAE (54.49) are interpretable as error magnitudes, but the negative $R^2$ illustrates how the statistic can be uninterpretable for trivial models once applied out of sample. As shown in Figure 2, the mean baseline produces a flat prediction line that fails to match the dynamics of the series.

Although the $R^2$ of the naive baseline model declines from 0.72 in training to 0.39 on the test set, its error metrics improve substantially. The RMSE decreases from 39.75 to 19.02, the MAE decreases from 29.19 to 17.75, and the MSE decreases from 1580.28 to 361.75. Thus, while $R^2$ suggests a loss of explanatory power on the test set, the absolute prediction errors indicate that the naive model generalises well. Figure 2 shows that the naive baseline tracks year-to-year variation well, which explains the low errors despite a decline in $R^2$. This highlights that the $R^2$ can understate the performance of simple, effective forecasting methods.

For the SVM, the $R^2$ suggests a collapse of the SVM on the test set. The model achieves a high $R^2$ of 0.87 on the training set, but this drops sharply to −0.31 on the test set. Judged by $R^2$ alone, one might conclude that the model generalises poorly and is of little practical use. Yet the RMSE and MAE remain relatively stable between train and test (26.76 vs. 27.81 for RMSE, 20.44 vs. 25.24 for MAE and 716.11 vs. 773.34 for MSE). In other words, the decline in performance is modest if not acceptable in absolute terms, but the $R^2$ greatly exaggerates it. Figure 2 illustrates this. The model captures the overall level and broad shape of the test series, contradicting the impression of collapse implied by the test set $R^2$.

The neural network exhibits a negative $R^2$ on the test set, which would typically indicate performance worse than the mean baseline. However, the error metrics tell a different story: the RMSE and MAE remain moderate and well within the range seen for the other models. As shown in Figure 2, the neural network captures the general level of the series but does not follow year-to-year fluctuations closely, resulting in prediction errors but not catastrophic failure. The negative $R^2$ therefore overstates the weakness of the model and fails to reflect its actual predictive accuracy.

Overall, these findings demonstrate that the $R^2$ can be a misleading guide to the performance of machine learning models in sustainability applications, as well as the complications involved with its use for even extremely simple models. In this example, the SVM and neural network appear to collapse on the test set if judged by $R^2$, despite maintaining error magnitudes similar to those observed in training. The naive baseline appears weak according to its test $R^2$, even though it achieves the lowest predictive errors of all models. These results show that $R^2$ can mislead in both directions, overstating the failure of complex models and understating the success of simple baselines. To understand why the $R^2$ behaves in this way, it is necessary to examine its underlying assumptions. While the statistic is well defined in the context of linear regression, the conditions required for valid interpretation are frequently violated in ML and time series settings. The next section outlines these theoretical limitations and explains why the $R^2$ can distort perceptions of both model weakness and model strength in practice.

5. Theoretical Reasons for the ${\mathit{R}}^{\mathbf{2}}$s Failings

The $R^2$ has several limitations that reduce its reliability as an evaluation metric in ML applications, including those used for waste forecasting. This section explains its theoretical limitations. The findings from the Irish practical example generalise to a wide range of sustainability and waste management applications, as the behaviour of the $R^2$ examined here stems from fundamental statistical properties (non-linearity of ML models, trend and multicollinearity).

While the assumptions underlying the $R^2$ in linear regression are well known, this section is not a restatement of classical statistics. Instead, it extends those assumptions to demonstrate why they are systematically violated in ML contexts, such as waste management. Although similar issues have been documented in the statistical and econometric literature, they have not been systematically acknowledged in ML applications, where the $R^2$ remains the dominant metric. Thus, this section highlights how well-known statistical concerns become overlooked in the specific context of ML waste management forecasting, contributing to the discussions started by [6,7].

5.1. The $R^2$ for Linear Regression

Linear regression is a classical statistical technique which uses a linear combination of covariates to predict the dependent [35]. As with most statistical tools, it comes with a set of assumptions [57]. These assumptions can be expressed as follows: (1) The data is observed with no error; (2) the relationships between the dependent and covariates are linear; (3) the residuals (errors) are random and not correlated; (4) the covariates are not correlated; and (5) the dependent is conditionally normally distributed. When these hold, the $R^2$ is a meaningful performance measure of the linear regression. Otherwise, it may be miscalculated and misleading.

In ML models, some of these conditions can be relaxed. ML models are designed to capture complex, non-linear relationships, meaning they do not require assumption 2 [58]. However, other assumptions, like independent errors, still matter. If these are not met, the $R^2$ may not provide a reliable measure of model accuracy.

5.2. The Effect of Hyperparameters on the $R^2$

Unlike linear regression, ML models have hyperparameters, which are design choices made before or during training [59]. These hyperparameters influence how the model ‘learns’ from the data and can affect the independence of the residuals. In a linear regression model, the total variation in the data can be split into two parts, the variation explained by the model and the unexplained variation. This is explained mathematically in Equation (6), which is commonly referred to as the total sum of squares identity. While it provides a formal basis for understanding the $R^2$, the key takeaway is that it assumes a clean partitioning of variance in the data. It is also worth noting that Equation (6) can be rearranged to obtain the definition of the $R^2$ in Equation (1).

$$\sum _i{(y_i-\overline{y})}^2=\sum _i{(\widehat{y_i}-\overline{y})}^2+\sum _i{(y_i-\widehat{y_i})}^2.$$

(6)

where $y_i$ is the ith observed dependent value, $\overline{y}$ is the mean of the dependent and $\widehat{y}$ is the ith value predicted by the model.

Since ML models are non-linear (because of their complexity and use of hyperparameters) [58], they introduce a ‘cross term’ that disrupts variance partitioning [38]. This makes residuals non-random, violating assumption 3. This issue can lead to $R^2$ values below 0 or above 1 [39]. This challenges the $R^2$’s reliability for ML models. The reader is referred to Book and Young [38] for the mathematical derivation of the cross term.

5.3. The Effect of Trend on the $R^2$

When data follow a steady upward or downward trajectory, the $R^2$ can give a misleading impression of model quality [60]. This issue is particularly acute in waste management, where long-term trends in MSW generation or recycling rates are directly linked to progress towards SDG 12. In such cases, even a very simple model that merely reproduces the trend may obtain a high $R^2$, not because it captures the underlying drivers, but because it aligns with the overall direction of the series. As shown in Section 4, the naive baseline achieved a train $R^2$ of 0.72 despite achieving nothing more than projecting the previous observation forward. This illustrates how the $R^2$ rewards the replication of trend rather than genuine explanatory or predictive insight.

In sustainability research, the dependent is often a time series (data collected over time [52]), such as annual waste generation or pollution levels. Many studies in this field use the $R^2$ to assess ML models applied to such data but do not check whether a trend is present. Of the 18 studies identified in Section 2, only 1 performed a check for trend [16].

Trend can cause a model’s residuals to be correlated over time, breaking assumption 3 [57]. This suggests the model has not fully captured the patterns in the data, making the $R^2$ unreliable. Furthermore, autocorrelation can negatively affect the model’s forecasting ability.

A simple way to check for a trend is to plot the data and visually inspect whether it consistently moves upward or downward. More formally, an Augmented Dickey–Fuller (ADF) test can be used to test for trend [61]. However, for small datasets, the test may be underpowered, so it should always be complemented with a visual inspection [62]. Looking back to Figure 1, a clear pre-2007 rise in MSW can be identified, as well as a fall post-2007. This clearly increases the naive baseline’s $R^2$ as it follows these trends without actually modelling the dynamics.

Another method is to examine the residuals of a model. Plotting the fitted values (predictions) against the residuals should show a random scatter of points. A visible pattern, such as a consistent upward or downward slope, a curved shape or wave-like movement, suggests the presence of a trend [63]. Trend can also cause heteroscedasticity, where the spread of residuals changes across the fitted values (that is, the residuals exhibit non-constant variance) [64]. Figure 3 plots the fitted values against the residuals on the training set for the SVM built in Section 3.4. These residuals are not ideal; there is an outlying value at around 675 on the x axis, and there seems to be a non-linear tail to the right of the plot. This would suggest a violation of the necessary assumptions for the $R^2$s correct calculation.

Figure 3. Fitted vs. residuals plot of the training set for the SVM built in Section 3.4.

To correct for trend, a process called differencing can be used. This is where the change in the data is modelled, rather than the data itself (for example, the change in MSW from one year to the next, rather than the quantity of MSW). Sometimes, multiple rounds of differencing are needed to fully remove the trend.

5.4. The Effect of Multicollinearity on the $R^2$

Multicollinearity occurs when covariates are highly correlated with each other [65]. In sustainability contexts, this is common: for example, economic activity, consumption and waste generation often move together, making it difficult to disentangle drivers of waste and weakening the worth of $R^2$ for CE-related modelling. In the presence of multicollinearity, correlated predictors can cause unstable estimates and may give an inflated impression of model fit on the training data [66]. This effect can be thought of as the model counting the same information twice. Multicollinearity can also lead to overfitting [67]. Multicollinearity causes the model to learn specific patterns in the training set rather than general relationships. This makes the model sensitive to small changes in the data, reducing its ability to make reliable predictions [68].

To identify multicollinearity, one can create a correlation matrix, which shows the correlation coefficients between covariates. If high levels of correlation are identified (typically greater than 0.7 in magnitude), then multicollinearity is a concern. Table 4 shows a correlation matrix for the covariates used to build the SVM in Section 3. As shown in boldface, there is one correlation which is greater than 0.7 in magnitude. This is evidence that multicollinearity could be influencing the $R^2$s in Section 4, giving a false impression of model ability.

Table 4. Correlation matrix for the covariates.

	GDP	Population	Unemployment
GDP	1	-	-
Population	0.84	1	-
Unemployment	−0.08	0.43	1

Multicollinearity can be dealt with by omitting one covariate of a correlated pair. Alternatively, one could use a technique such as Principal Component Analysis (PCA). PCA transforms correlated covariates into a smaller set of uncorrelated components (linear combinations of the covariates) while preserving as much information as possible [69].

6. Discussions

The theoretical limitations of the $R^2$ discussed in Section 5 are not merely academic concerns. Issues related to the misevaluation of ML models have tangible, real world impacts when ML is used as part of the decision-making process in sustainability and waste management planning.

The policy relevance of these findings is particularly strong in the context of EU waste legislation. Core instruments such as the Waste Framework Directive (2008/98/EC) [70] (which establishes the waste hierarchy and sets binding targets for recycling and recovery) and the Packaging and Packaging Waste Directive (94/62/EC) [71] and its subsequent amendments, rely on accurate national statistics. These policy frameworks inform investment decisions, extended producer responsibility (EPR) obligations and the monitoring of Member State compliance. If ML models used in such policy analyses are evaluated using inappropriate metrics, there is a genuine risk of misjudging whether countries are on track to meet EU targets. For example, a model that appears strong under $R^2$ may systematically under-predict municipal waste arisings, affecting assessments of compliance with recycling requirements. Conversely, a negative test set $R^2$ may lead policymakers to dismiss models that in fact produce acceptable errors. Such distortions can influence planning for recycling infrastructure, evaluations of EPR performance and the implementation timelines associated with the Packaging and Packaging Waste Regulation [72]. By demonstrating how easily model performance can be misinterpreted under realistic data conditions, this study highlights the potential policy risks that arise when evaluation metrics are not chosen with care.

While the practical example focuses on Irish MSW data, the implications of the findings extend more broadly. The limitations of the $R^2$ highlighted in Section 5 arise from the non-linearity of ML models and the structural properties of sustainability datasets rather than from characteristics specific to Ireland. As a result, the mechanisms through which the $R^2$ becomes unreliable are expected to generalise across diverse contexts, even though this study deliberately restricts itself to a single representative case. The practical example should therefore be viewed as illustrative rather than exhaustive, with the theoretical argument providing the basis for generalisability.

When evaluating models, error-based metrics like RMSE and MAE should be prioritised, as they provide concrete error assessments in the same units as the prediction and do not rely on assumptions that are often violated in ML. Moreover, because RMSE and MAE are in the same units as the predictions, they offer a direct and interpretable measure of error, helping policymakers understand the accuracy of forecasts. It is also good practice to visualise predictions made by ML models, to ensure that the predictions look feasible (i.e., the model is not predicting unrealistic values or making repeated predictions).

To make these considerations operational, we outline a simple and practical workflow for evaluating ML models in sustainability and waste management research. First, model performance should always be assessed using a suite of metrics, such as the MAE and RMSE, rather than relying on a single indicator such as the $R^2$. Second, evaluation should incorporate basic diagnostic tools, such as residual plots, checks for autocorrelation and visualisations of fitted and observed values, to identify issues that scalar metrics may obscure. Third, for time series data, validation procedures should respect temporal ordering, avoiding random train/test splits that can unrealistically mix past and future information. Finally, multicollinearity should also be checked for among the covariates. If present, it can be dealt with by omitting covariates or using techniques such as PCA. These steps provide a transparent and reproducible procedure that can be readily implemented using standard ML workflows and help ensure that model assessment remains methodologically sound.

The credibility of waste management research is pivotal in maintaining public trust and ensuring stakeholder engagement. Policymakers, practitioners, and the public rely on transparency in the modelling process to justify decisions and secure funding. By ensuring the selected evaluation metrics are appropriate, researchers can offer a more nuanced and accurate picture of model performance. This not only mitigates the risks of misinterpretation but also promotes better decision-making. Demonstrating awareness of the limitations of traditional metrics and addressing them can foster a culture of methodological rigour in ML-based waste management research, advancing the field and supporting robust CE strategies.

A limitation of this study is the size of the Irish MSW dataset, which contains 19 annual observations from 2000 to 2020. Short time series are common in national-level waste management research but can affect the stability of ML models and contribute to variability in performance metrics, including the $R^2$. Two missing values were linearly interpolated, introducing a small degree of uncertainty. However, as can be seen from Figure 1, the interpolated years (2013 and 2015) fall within a period where the series is relatively stable (2012–2017), reducing the likelihood of distortion. To mitigate risks of overfitting, we restricted the models to three covariates and used conservative hyperparameter settings.

Importantly, these limitations reinforce the central argument of the paper. Under realistic data constraints typical of sustainability applications, the $R^2$ becomes particularly unreliable as an evaluation metric, whereas error-based measures offer a more stable and informative basis for model assessment.

In summary, evaluating ML models with appropriate error metrics and diagnostic checks is not just a methodological concern but a prerequisite for evidence-based waste management. A thorough evaluation framework supports the achievement of waste management objectives and helps ensure that waste policies contribute effectively to SDG 12.

7. Conclusions

This study examined the use of evaluation metrics in machine learning applications to waste management, with a focus on the limitations of the $R^2$ statistic. Through a theoretical analysis and a practical example of Irish MSW forecasting, it is demonstrated how reliance on $R^2$ can misrepresent model performance under the data characteristics typical of waste management research, including temporal dependence and correlated predictors.

These findings show that error-based measures such as RMSE and MAE offer more stable and interpretable assessments of predictive accuracy, while residual diagnostics and checks for multicollinearity are essential for ensuring robust models. The novelty of this work lies in linking recognised statistical limitations of $R^2$ to the specific context of sustainability and waste management ML, where these concerns remain underappreciated.

Building on these insights, this study provides a set of operational recommendations for improving ML evaluation in waste management research. First, model performance should be assessed using a suite of complementary metrics, prioritising assumption-free error measures such as RMSE and MAE, and avoiding the use of the $R^2$ for model evaluation. Second, researchers should implement a minimum diagnostic protocol that includes visual inspection of trends in both the raw data and residuals and checks for autocorrelation using tools such as residual ACF plots and Durbin–Watson statistics. Multicollinearity should be assessed through correlation matrices, with corrective steps such as variable omission or PCA implemented where necessary. Time series validation must respect temporal ordering by using chronological train/test splits rather than random partitions. Finally, numerical metrics should always be paired with visual diagnostics (such as plotting observed vs. predicted values and out-of-sample forecast trajectories) to detect implausible behaviour that scalar metrics may obscure. Taken together, these steps form a practical and reproducible evaluation workflow suitable for sustainability applications.

Future research should prioritise the development and standardisation of evaluation frameworks that move beyond single metrics, ensuring transparency, reliability and policy relevance. Strengthening methodological practice in this way is a prerequisite for advancing CE and waste management strategies, contributing to achieving SDG 12.