A Multi-Stage Feature Selection and Explainable Machine Learning Framework for Forecasting Transportation CO₂ Emissions

Abstract

The transportation sector is a major consumer of primary energy and is a significant contributor to greenhouse gas emissions. Sustainable transportation requires identifying and quantifying factors influencing transport-related CO₂ emissions. This research aims to establish an adaptable, precise, and transparent forecasting structure for transport CO₂ emissions of the United States. For this reason, we proposed a multi-stage method that incorporates explainable Machine Learning (ML) and Feature Selection (FS), guaranteeing interpretability in comparison to conventional black-box models. Due to high multicollinearity among 24 initial variables, hierarchical feature clustering and multi-step FS were applied, resulting in five key predictors: Total Primary Energy Imports (TPEI), Total Fossil Fuels Consumed (FFT), Annual Vehicle Miles Traveled (AVMT), Air Passengers-Domestic and International (APDI), and Unemployment Rate (UR). Four ML methods—Support Vector Regression, eXtreme Gradient Boosting, ElasticNet, and Multilayer Perceptron—were employed, with ElasticNet outperforming the others with RMSE = 45.53, MAE = 30.6, and MAPE = 0.016. SHAP analysis revealed AVMT, FFT, and APDI as the top contributors to CO₂ emissions. This framework aids policymakers in making informed decisions and setting precise investments.

1. Introduction

1.1. Background

Currently, the transportation sector accounts for 21% of global carbon emissions, and according to the International Energy Agency (IEA), the transport demand is anticipated to double by 2070, specifically, a 60% rise in car usage rate and a threefold increase in freight aviation [1]. Road travel accounts for approximately 75% of transport, or 15% of global carbon emissions, of which 45.1% is attributed to passenger automobiles and 29.4% is derived from lorries engaged in freight transportation. Aviation, shipping, rail, and other systems contribute 11.6%, 10.6%, 1%, and 2.2%, respectively [2].

Figure 1 displays the historical data of the United States (U.S.) CO₂ emissions since 1990. The transport field was the second biggest sector until 2016, when it surpassed electric power, turning into the greatest origin of CO₂ emissions [3]. Containing CO₂ emissions in the transport field is crucial in minimizing climate change and attaining sustainability. Precisely forecasting upcoming CO₂ emissions is a necessary initial phase, as dependable forecasts allow stakeholders and policymakers to establish powerful techniques and apply cleaner technologies, as well as encourage sustainable transport options.

The development of big data and artificial intelligence (AI) has triggered an astonishing revolution in many research fields. A range of Machine Learning (ML) Methods were used to forecast the level of CO₂ emission around the world, such as Artificial Neural Networks (ANN) [4,5,6], Support Vector Machine (SVM) [5], regression methods [4,6,7,8,9,10,11], time series analysis [12,13,14], Deep Learning (DL) [5], decision trees [4,15], and hybrid methods [16,17,18].

Although several researchers have utilized different AI and empirical techniques to predict CO₂ emissions, limited attention has been concentrated on its relationship with the transportation field in the USA. Furthermore, prior research on CO₂ emission prediction has either focused on a limited number of preselected features or utilized short-term data collection methods. For example, four ML algorithms were developed by Chukwunonso et al. [19] to evaluate total CO₂ emissions in the United States with data from 1973 to 2022, with eight indicators included. Ahmed et al. [20] explored the energy usage and greenhouse gas (GHG) emissions of the United States, India, China, and Russia using ML methods based on features including gas, petroleum liquids, coal, and renewable power usage. Mishra et al. [21] implemented the wavelet transform method to examine the relationship among economic development, the transport sector, tourism development, and CO₂ emissions, evaluating a period of 16 years after 2001. Table 1 summarizes some of the latest research addressing the forecasting of CO₂ emissions in different areas inside and outside of the United States. It is observed that feature selection (FS) was less concerned, which is a crucial factor due to ML forecasting fully relies on and is sensitive to features selected [22,23,24]. Additionally, a proper interpretation of feature importance is vital for decision/policy making. In general, precise forecasting of CO₂ emission based on the right features serves as the fundamental factor in supporting policymaking.

1.2. Research Novelty

The novelty of this study can be concluded in the following points:

• This research proposed a sophisticated multi-stage ML framework for an accurate forecasting of transportation CO₂ emission in the U.S. potential factors were selected based on a comprehensive literature review, including energy usage, economic and demo-graphic parameters like population, unemployment rate, Gross Domestic Product (GDP), urban population, total fossil fuels consumed, etc. This broad and varying information allows the model to comprehend driving contributors to CO₂ emissions in the transport field, which also provides a comprehensive method in comparison with conventional methods that concentrate exclusively on energy metrics.
• Given the extremely multicollinear features like total primary energy production, GDP, fossil fuel consumption, population, air transport freight, road mileage, and so on, conventional forecasting usually faces issues associated with decreased precision. To this, a main concentrate of the study was a two-stage FS procedure starting with time series clustering to determine groups of correlated factors, accompanied by an advanced FS voting mechanism based on Boruta FS and Spearman correlation analysis to identify the best appropriate features. This arranged method ensured the robustness and reliability of forecasting results.
• Four different ML models, like SVR, eXtreme Gradient Boosting (Xgboost), ElasticNet, and Multilayer perceptron (MLP), were employed to improve the accuracy of CO₂ emissions forecasting in the transportation field. The SHAP method was then applied to uncover the black box of ML, identifying the global and local relationship between input features and CO₂ emissions. The explainable ML methods ensured offering valuable insights for policymakers to support efficient techniques regarding decreasing CO₂ emissions in the USA. Figure 2 depicts the conceptual framework for forecasting transportation CO₂ emissions based on the above procedure. All machine learning coding was implemented based on python 3.11.9, scikit-learn 1.5.1, xgboost 2.1.0 and shap 0.47.0.

The rest of the paper is arranged as below: Section 2 briefs the related literature review in transportation CO₂ emission forecasting, and Section 3 describes data collected and the methodology. Section 4 presented the final results and discussion, followed by Section 5 indicating policy recommendations and limitations, and Section 6 finalized the conclusion and future research.

2. Literature Review

Much research on CO₂ emission in the transportation sector has been developed utilizing ML algorithms and statistical methods. An overview of the recent research concerning the abovementioned topic with a variety of methods is provided as an initial step in the review process.

2.1. Machine Learning for CO₂ Prediction

Qiao et al. [22] attempted to utilize ML techniques to predict the CO₂ emissions and energy usage in the United kingdom’s transport field. The experimental outcomes revealed that carbon intensity in the road was the most influential factor for CO₂ emissions and energy usage, while GDP per capita and population were less crucial. Chukwunonso et al. [19] considered CO₂ emissions from 1973 to 2022 in the USA based on four ML algorithms, of which the Recurrent Neural Network (RNN) algorithm outperformed all others. Nevertheless, this method is generally recognized to be challenging in long-term dependencies because of vanishing gradient issues, positioning it less powerful in capturing long-range temporal behavior. Additionally, the importance of features was not discussed in their research. To predict transportation-related GHG emissions of China, Yin et al. [4] introduced a new method that combined information extraction and managed ML techniques. The results recommended that ANN had exceptional prediction precision in comparison with other models. Fu et al. [23] suggested integration of ML, localized emissions information, and satellite imaging to create an accurate and applicable system for checking GHG emissions throughout roads. The outcomes revealed the capability of integrating ML techniques with satellite imagery in effectively checking GHG emissions. Similarly, Javanmard et al. [16] used hybrid multi-objective ML models to forecast transport emissions and energy needs, which indicated an increase in CO₂ emissions and energy by 0.02% and 50.02% from 2019 to 2048 throughout the Canadian transport field. Ahmed et al. [20] reviewed the energy usage of the USA, India, China, and Russia, as well as its pattern in GHG emissions using ML methods. The predicted outcomes with the long-short term memory technique verified a rise in CO₂, methane, and Nitrous oxide (N₂O) emissions regarding India and China and a slowdown pattern in the USA and Russia. Although Ahmed et al. [20] utilized ML methods to forecast GHG emissions. Ulussever et al. [24] analyzed sector-based energy usage indicators from 1973 to 2021 as independent features to calculate CO₂ emissions in the USA. The empirical results show the outperformance of ML over the time series econometric models. Three ML algorithms are generally utilized to predict transport-based CO₂ emissions by Li et al. [25], where the top 30 CO₂ emissions-producing nations from 1960 to 2020 were selected. The results showed that the gradient boosting regression model with variables mixing transport and socioeconomic elements has the greatest efficiency for transport CO₂ emission prediction. Although Li et al. [25] integrated both socioeconomic and transportation variables for CO₂ emission forecasting, their research concentrated on a worldwide range as well as did not particularly evaluate the USA transport field. Three ML algorithms were utilized by the Ağbulut [5] to predict the transport CO₂ emission and energy needs throughout Turkey. It was predicted that the yearly growth rate regarding carbon dioxide emission and transport energy needs within Turkey would cumulatively increase by 3.65% and 3.7%, respectively, and will be almost 3.4 times greater in the year 2050 compared to those of today. Although Ağbulut [5] effectively predicted upcoming trends, mainly concentrating on Turkey’s transport field utilizing a restricted set of input parameters. To predict the peak of transportation CO₂ emissions throughout China, a new bio-inspired forecasting model was suggested by Wang and Wang [26], namely, the Manta Rays Foraging Optimization-Extreme Learning Machine (MRFO-ELM) with the mean impact value technique used to examine and identify the significance of 13 important variables. The scientific outcomes show that the suggested MRFO-ELM has outstanding functionality regarding the optimization searching speed and forecasting precision. Alfaseeh et al. [27] employed an LSTM method to forecast GHG Emission Rate (ER) depending on the most important features, for example, velocity, density, and GHG ER of prior time period steps. Compared with clustering and the autoregressive integrated moving average (ARIMA) method, the LSTM model with velocity, GHG ER, density, and velocity from three prior minutes carried out the greatest action. Qin and Gong [28] used RF and decision trees to figure out the variables influencing CO₂ emissions in the eastern areas with greater economic development throughout China, for example, GDP, foreign investment, and general financial budget revenue can impact CO₂ emissions. The results show that locations with severe CO₂ emissions, like Chongqing, Tianjin, and Shanghai, ought to be prioritized as locations for low-carbon economic development.

Despite the abovementioned research, their efforts mainly concentrated on using existing ML approaches without integrating enhanced FS methods or taking into consideration a broad variety of impacting variables. Moreover, their strategy lacks the level of interpretability, which is a crucial part of current research, promoting its applicability in real-world policy decisions.

2.2. Feature Selection for the CO₂ Prediction

FS technique can be utilized in data pre-processing to obtain effective data reduction while maintaining the informative features. A suitable FS process is essential in forecasting CO₂ emission. We provide an overview of several of the latest research that considers FS techniques within the fields of energy and CO₂ emission.

The effects of FS techniques on ML models for prediction of CO₂ emission and energy consumption throughout the UK were evaluated by Qiao et al. [22]. A novel voting strategy for FS was introduced, including both integrated and filter methodologies. Wang et al. [29] introduced a hybrid model merging scenario analysis with the ML method. Prior to the training models, FS (i.e., stepwise regression) was a substantial step to simplify the model and prevent overfitting. Ma et al. [30] suggested a multivariate evaluation of the most essential elements for the national level of air quality from 171 features varying in economic, energy, environmental, meteorological, and demographical aspects. To tackle huge information, the FS method and Extreme Gradient Boosting were used to model the connection and evaluate the feature significance. Li and Sun [31] utilized a set of open access information and ML algorithms to forecast CO₂ emissions throughout China. Eighteen out of thirty-one factors were chosen based on the FS technologies to create forecasting models of carbon dioxide emissions. They discovered that the statistical indicators of city environment pollution were generally the most essential features in terms of the urban-level carbon dioxide emissions within China.

Van Zyl et al. [32] evaluated the effectiveness of explainable AI techniques, particularly Shapley Additive Explanations and Gradient-weighted Class Activation Mapping, for the FS method regarding national energy need prediction. Amiri et al. [33] used FS for forecasting household transport energy utilization. A new predicting energy model depending on the SVR algorithm with a wrapper FS method utilizing multi-objective optimization method was designed by Karasu et al. [34]. A modeling scheme by Tang et al. [35] developed a DL for predicting the NO_x emission depending on a mix of FS and the JAYA optimization algorithm. Although prior research (Van Zyl et al. [32], Amiri et al. [33], Karasu et al. [34], and Tang et al. [35]) has made significant contributions to FS, they generally ignore the challenge of multicollinearity that can skew model forecasts as well as decrease the performance of FS techniques.

Regarding the above review of prior research works, Table 1 summarizes reviews from all over the world related to CO₂ evaluation in the field of transportation. Given the sophistication of transport emissions, which are generally affected by different technological, environmental, and socioeconomic variables, the insufficient, considerable study, including FS, highlights a significant gap. Existing research mostly concentrated on traditional forecasting methods without methodically optimizing input parameters, possibly resulting in decreased predictive precision and model interpretability. This highlights the requirement for more extensive research that examines FS techniques tailored to CO₂ emission prediction, providing strong and reliable predictive frameworks. Based on such premises, we proposed a comprehensive feature selection framework to address the multicollinearity issue while maintaining the informative features, providing an advantage that balances precision, strength, and interpretability, and generating it extremely appropriately regarding predicting CO₂ emissions as well as offering more dependable inputs for policymaking.

Table 1. Summary of the literature review in predicting CO₂ emissions.

Article	Country	Time Period	Feature Selection	Machine Learning	Applied Models
					ML	Hybrid	Others
Qiao et al. [22]	UK	1990–2019	Yes	Yes	LSTM	SVR-RBF	---
Javanmard et al. [16]	Canada		No	Yes	---	multi-objective mathematical model with data-driven ML	---
Ahmed et al. [20]	China, India, the USA, and Russia	1992–2018	No	Yes	SVM, ANN, LSTM	---	---
Qin and Gong [28]	China	2000–2019	No	Yes	RF, decision tree	---	---
Ulussever et al. [24]	USA	1973–2021	No	Yes	MLP, SVM, RF, MARS, k-NN	---	Time Series Econometric
Li et al. [25]	30 Countries	1960–2020	No	Yes	OLS, SVM, GBR	---	---
Ağbulut [5]	Turkey	1970–2016	No	Yes	DL, SVM, ANN	---	---
Li and Sun [31]	China	2010	Yes	Yes	GBM, SVM, RF, and XGBoost	---	---
Chukwunonso et al. [19]	USA	1973–2022	No	Yes	RNN, FFNN, CNN	---	---
Wang and Jixian [26]	China	2000–2012	No	Yes	---	MRFO-ELM	---
Alfaseeh et al. [27]	Canada	N/A	No	Yes	LSTM	---	ARIMA
Yin et al. [4]	China	N/A	No	Yes	Decision Tree, MLR, ANN	---	MLR
Fu et al. [23]	worldwide	2021	No	Yes	GNN, CNN	---	---
Wang et al. [29]	China	1980–2014	Yes	Yes	SVM, GPR, BPNN	PSO-SVM	---
Peng et al. [36]	China	N/A	No	Yes	---	GAPSO-SVR	---
Chadha et al. [37]	India	N/A	No	Yes	XGBoost, (SVR), RF, and Ridge Regression	---	---
Anonna et al. [38]	USA	1970–2020	No	Yes	RF, Support Vector Classifier, and Logistic Regression	---	---
Jha et al. [39]	USA	2004–2023	No	Yes	11 AI algorithms	---	---
Li et al. [40]	USA	N/A	No	Yes	DL-LSTM	---	---
Tian et al. [41]	USA	1973–2022	No	Yes	k-nearest neighbors (KNN), RF, MLR, gradient boosting, decision tree and SVR	---	---
Ajala et al. [42]	China, India, the USA, and the EU27&UK	2022–2023	No	Yes	14 AI algorithms	---	---
Current Research	USA	1990–2023	Yes	Yes	XGBoost, MLP, SVR, Elastic Net	----	----

Note: Deep Learning (DL), Feed-Forward Neural Network (FFNN), Convolutional neural network (CNN), Multinomial Logistic Regression (MLR), Generalized Neural Network (GNN).

Table 1. Summary of the literature review in predicting CO₂ emissions.

Article	Country	Time Period	Feature Selection	Machine Learning	Applied Models
					ML	Hybrid	Others
Qiao et al. [22]	UK	1990–2019	Yes	Yes	LSTM	SVR-RBF	---
Javanmard et al. [16]	Canada		No	Yes	---	multi-objective mathematical model with data-driven ML	---
Ahmed et al. [20]	China, India, the USA, and Russia	1992–2018	No	Yes	SVM, ANN, LSTM	---	---
Qin and Gong [28]	China	2000–2019	No	Yes	RF, decision tree	---	---
Ulussever et al. [24]	USA	1973–2021	No	Yes	MLP, SVM, RF, MARS, k-NN	---	Time Series Econometric
Li et al. [25]	30 Countries	1960–2020	No	Yes	OLS, SVM, GBR	---	---
Ağbulut [5]	Turkey	1970–2016	No	Yes	DL, SVM, ANN	---	---
Li and Sun [31]	China	2010	Yes	Yes	GBM, SVM, RF, and XGBoost	---	---
Chukwunonso et al. [19]	USA	1973–2022	No	Yes	RNN, FFNN, CNN	---	---
Wang and Jixian [26]	China	2000–2012	No	Yes	---	MRFO-ELM	---
Alfaseeh et al. [27]	Canada	N/A	No	Yes	LSTM	---	ARIMA
Yin et al. [4]	China	N/A	No	Yes	Decision Tree, MLR, ANN	---	MLR
Fu et al. [23]	worldwide	2021	No	Yes	GNN, CNN	---	---
Wang et al. [29]	China	1980–2014	Yes	Yes	SVM, GPR, BPNN	PSO-SVM	---
Peng et al. [36]	China	N/A	No	Yes	---	GAPSO-SVR	---
Chadha et al. [37]	India	N/A	No	Yes	XGBoost, (SVR), RF, and Ridge Regression	---	---
Anonna et al. [38]	USA	1970–2020	No	Yes	RF, Support Vector Classifier, and Logistic Regression	---	---
Jha et al. [39]	USA	2004–2023	No	Yes	11 AI algorithms	---	---
Li et al. [40]	USA	N/A	No	Yes	DL-LSTM	---	---
Tian et al. [41]	USA	1973–2022	No	Yes	k-nearest neighbors (KNN), RF, MLR, gradient boosting, decision tree and SVR	---	---
Ajala et al. [42]	China, India, the USA, and the EU27&UK	2022–2023	No	Yes	14 AI algorithms	---	---
Current Research	USA	1990–2023	Yes	Yes	XGBoost, MLP, SVR, Elastic Net	----	----

Note: Deep Learning (DL), Feed-Forward Neural Network (FFNN), Convolutional neural network (CNN), Multinomial Logistic Regression (MLR), Generalized Neural Network (GNN).

3. Materials and Methods

Figure 2 systematically describes the research outline of the proposed multi-step FS based explainable ML approach for forecasting total CO₂ emissions of transportation TCT (t + 1). The study in total involved 4 steps, with the first step being data identification and collection, features deemed crucial based on literature review were then collected from multiple official websites and merged properly in CSV files. Considering the likelihood of including redundant and irrelevant variables collected in step 1. A comprehensive multi-step FS method based on hierarchical feature clustering, Boruta FS and Spearman correlation was established in step 2 to mitigate data irrelevancy and redundancy challenges. Inspired by the Autoregressive model, the study employed autocorrelation analysis to determine the number of historical TCTs that were the most relevant to TCT (t+1) as additional independent variables. Step 3 involved ML forecasting based on three feature set scenarios, e.g., a feature subset based on the proposed FS method, the original feature set, and a feature subset generated by Recursive feature elimination-Random Forest (RFE-RF) [38]. Data was split into training and testing datasets at a ratio of 80:20 without shuffle. The main reason for an 80:20 ratio is not only due to it being a typical ratio for ML applications but also due to the concern of limited sample size; 80% of training data primarily ensures ML models fully learn the data. Ten-Fold Time Series Split Grid Search Cross Validation (TSGSCV) by Liu and Zhou [39] was also employed to tune and determine the best hyperparameter for each ML model based on different feature setting scenarios. Three evaluation metrics, namely Root Mean Square Error (RMSE), Mean Absolute Error (MAE), and Mean Absolute Percentage Error (MAPE), were introduced to validate the trained ML models performance using the rest of the 20% of the testing dataset. Feature subsets and ML models that indicate the best performance were selected for Shapley analysis to quantify the contribution of the most relevant and important variables to TCT (t + 1). The source code and the data of the research could be found in the GitHub repository (https://github.com/qiaoqingyao/CANADACO2/tree/master (accessed on 14 July 2025)).

3.1. Data Collection

Figure 3 provides the record of CO₂ emissions in the transportation sector in the USA from 1990 to 2023. An overall rising trend was observed from almost 1600 (Mt) in 1990 to 2026 (Mt) in 2007. Followed by a moderate decrease until 2012, probably affected by financial elements. After that, emissions increased once again from 2012 (1776 Mt) to 2019 (1924 Mt). In 2020, there was a sharp decrease (i.e., 1633 Mt, which is lower than 1994), probably because of decreased transportation activities during the COVID-19 pandemic. After 2020, emissions started to rebound despite staying under the pre-pandemic crisis level until 2023 (1858 Mt).

In this research, 25 variables, including 24 independent and one dependent variable, were selected.

Table 2. List of variables, abbreviations, and descriptive statistics.

Variables	Abbr.	Unit	Mean	Std	Min	Max
Total CO2 by Transportation	TCT	MtCO2	1814.09	125.51	1565.00	2026.00
Total Fossil Fuels Consumed (transportation field)	FFT	Trillion Btu	25,439.21	1691.27	21,994.76	28,142.77
Biomass Energy Consumed (transportation field)	BET	Trillion Btu	722.10	612.08	60.42	1788.41
Electricity Sales to Ultimate Customers (transportation field)	EUT	Trillion Btu	22.11	4.18	16.06	27.89
Total Energy Consumed (Transportation field)	TET	Trillion Btu	26,224.57	1982.71	22,114.32	28,810.91
Total Primary Energy Production	TPEP	Quadrillion Btu	76.31	10.78	66.20	102.85
Total Primary Energy Consumption	TPEC	Quadrillion Btu	92.94	4.47	82.21	98.97
Population	---	---	297,726,251.53	26,600,955.72	249,623,000.00	343,477,335.00
Urban Population rate	UPR	%	79.90	2.24	75.30	83.30
GDP Per Capita	GDP	USD	46,036.55	15,435.30	23,888.60	81,632.25
Total Unemployment rate	UR	%	5.77	1.63	3.65	9.63
Retail Gasoline Prices	GP	USD per Gallon	2.17	0.94	1.03	3.95
Total Petroleum, Excluding Biofuels, Imports	TPI	Quadrillion Btu	22.18	3.84	16.35	29.20
Total Primary Energy Imports	TPEI	Quadrillion Btu	26.02	4.81	18.33	34.68
Air Passengers-Domestic and international	APDI	Million	668,920,157.79	138,802,316.46	369,501,000.00	926,737,000.00
Air transport, freight	APF	million ton-km	32,826.18	9782.80	14,486.20	47,716.00
Railways Passengers	RPC	million passenger-km	28,411.97	5851.35	12,460.00	36,393.11
Railways, goods transported	RGT	million ton-km	2,330,457.76	227,003.11	1,906,206.0	2,702,736.0
Number of motor vehicles registered	MVR	1000 s	242,272.71	28,968.71	192,314.00	286,300.00
Annual Vehicle Miles Traveled	AVMT	Miles	2,811,389.09	342,132.55	2,117,716.00	3,284,596.00
Road Milage	RM	Mile	4,032,513.13	111,366.49	3,866,926.00	4,200,000.00
Motor vehicle licensed drivers	MVLD	Number	202,325,783.05	20,679,848.46	167,015,250.00	236,404,000.00
Average annual temperature	AAT	°F	53.97	1.64	50.30	56.60
Crude Oil First Purchase Price	COPP	Dollars per Barre	46.70	29.21	10.87	95.99
Annual Percent Change	APC	%	0.03	0.02	0.00	0.08

Abbr.: Abbreviation.

lists these variables along with their abbreviations and statistical descriptions. Generally, we divide all the input features into four categories: transportation-based, socioeconomic, environment-based, and energy-based features. Transportation-based features comprise air passengers—domestic and international—air transport freight, railway passengers, railway goods transported, number of motor vehicles registered, annual vehicle miles traveled, road mileage, and motor vehicle licensed drivers. Socioeconomic features include population, urban population rate, GDP, unemployment rate, gasoline prices, crude oil first purchase price, and annual percent change. Environmental factors include average annual temperature. Energy-based features consist of fossil fuels consumed, biomass energy consumed, electricity sales to ultimate customers, primary energy production, primary energy consumption, petroleum (excluding biofuel imports), and primary energy imports.

In the current study, all data was extracted from the World Bank Group, https://data.worldbank.org/ (accessed on 26 July 2023), International Energy Agency, https://www.iea.org/data-and-statistics/, (accessed on 14 July 2025), U.S. Energy Information Administration, https://www.eia.gov/ (accessed on 14 July 2025), U.S. Department of Energy, https://afdc.energy.gov/data (accessed on 1 May 2023), U.S. Department of Transportation, https://www.bts.gov/, (accessed on 14 July 2025), National Weather Service, https://www.weather.gov/ (accessed on 1 January 2025), Federal Reserve Bank of Minneapolis, https://www.minneapolisfed.org/ (accessed on 1 January 2025).

3.2. Feature Selection Methods

3.2.1. Hierarchical Clustering

In order to deal with multicollinearity of independent variables, a hierarchical clustering was first implemented, which is a clustering technique in data analysis that aims to construct a tree-like structure of clusters [43]. This method involves grouping data points into a hierarchy of clusters, where each cluster is formed by combining smaller clusters or individual data points. There are two main approaches to hierarchical clustering: agglomerative (bottom-up), which involves merging data points into larger clusters, and divisive (top-down), which entails splitting a cluster into smaller sub-clusters. Agglomerative clustering approach was employed in this study. The key task of the agglomerative method is an effective cluster combination, which in general is achieved by comparing the dissimilarity between observation pairs based on some distance metrics; for instance, in this study, Ward’s Minimum variance method:

$${\displaystyle \frac{\left|A\right|·\left|B\right|}{\left|A\cup B\right|}}{‖{\mu }_A-{\mu }_B‖}^2={\sum }_{x\in A\cup B}{‖x-{\mu }_{A\cup B}‖}^2-{\sum }_{x\in A}{‖x-{\mu }_A‖}^2-{\sum }_{x\in B}{‖x-{\mu }_B‖}^2$$

(1)

when the cluster tree is established, the next step in hierarchical clustering is determining the optimal number of clusters. Silhouette score was employed to decide the best number for clustering.

$$S={\displaystyle \frac{b_i-a_i}{max \left(a_i{,b}_i\right)}}$$

(2)

where $a_i$ signifies the average distance of point $i$ to other data points within the same clusters and $b_i$ calculates the average distance of point $i$ to all other clusters.

The number with the smallest Silhouette is regarded as the optimal number of clusters and the independent variables will be clustered accordingly.

3.2.2. Boruta Feature Selection (BFS) and Spearman Correlation

The independent variables in each cluster implied a significant multicollinearity, which, in other words, represents similar information. In this step, the task is to determine whether each cluster is relevant to TCT (t + 1) and, meanwhile, choose the best candidate variable from each cluster. BFS and Spearman correlation were employed for this task.

The BFS algorithm is generally a wrapper method around the RF. It considers the variations in the average accuracy loss of trees in the forest as well as utilizing the average drop precision, which is the Z score, to calculate the significance. Generally, depending on the full variables, the correlation among the variable and the forecasted magnitude is removed by generating a combined shadow variable prior to selection, which is considerably more beneficial for processing information with more powerful variable correlation [44]. A comprehensive process for BFS is iterated below:

• Add randomness to the variables by generating shuffled copies (shadow variables) of the whole variables and then combine the shadow variables using original variables to create extended variables.
• Set up an RF procedure for the extended variables as well as calculate the significance variable (the mean decreased precision Z value). The greater the Z value, the more significant the variable; the greatest Z value of the shadow variable is determined as Zmax.
• Throughout every iteration, if the Z value of the variable is greater than Zmax, after that, the variable is taken into account as significant and will be retained. Otherwise, the variable is considered highly insignificant as well as will be eliminated through the variables.
• The earlier mentioned procedure stops whenever either all variables are generally rejected or verified, or BFS gets to the highest range of iterations [42].

Spearman correlation is a nonparametric statistical technique that evaluates the rank correlation between two variables by assessing the extent to which their rankings are related through a monotonic function [43]. It provides insights into the strength and direction of the association between variables based on their ordinal rankings. Monotonical nature enables Spearman correlation to effectively measure the nonlinear relationship between variable pairs.

For each cluster, a conservative FS mechanism based on BFS and Spearman correlation analysis was proposed as follows:

• If multiple variables are tested to be relevant in a cluster, the variable that has the highest correlation score with statistical significance is determined as the best candidate variable for this cluster. In case no variable indicates statistical significance, a compromise is that the variable that has the highest correlation score is still regarded as the best one.
• If only one variable is either tested to be relevant or indicates the highest correlation score with statistical significance in a cluster, then this variable is the chosen variable for this cluster.
• If variables in a cluster show neither relevancy nor statistical significance in Spearman correlation, then this cluster is discarded and no variable will be selected for the next step of ML.

3.3. Machine Learning Models

3.3.1. eXtreme Gradient Boosting (Xgboost)

Xgboost is an ensemble technique designed to amalgamate inadequate learning models to formulate a considerably more resilient model via a repetitive process. At every repetition, the residual from the prior estimate may be utilized to modify as well as improve the loss function. This approach often uses a binary decision tree as the principal learner, integrating regularization into the loss function to enhance performance and reduce overfitting, as seen in Equation (3). In this regard, l as a loss function will calculate the distinction between the real magnitude $y_i$ and forecasting ${\widehat{y}}_i$ regarding every sample i.

$$L={\sum }_{i=1}^{n}l\left(y_i.{\widehat{y}}_i\right)+{\sum }_k\Omega \left(k\right)$$

(3)

Since this method pertains to a classification/regression issue, the Mean Squared Error (MSE) can be used as the loss function. The word Ω indicates a control technique and is also used to penalize the intricacy of every model. The intricacy of a decision tree k is identified through its framework and may be expressed as Equation (4). Here, $w_j$ indicates the rating of every leaf j and T denotes the quantity of leaves integrated within a decision tree.

$$\Omega \left(K\right)=\gamma .T+{\displaystyle \frac{1}{2}}\lambda {\sum }_{i=1}^T{w}_j^2$$

(4)

Typically, both γ and λ serve as the factors for scaling the penalty. Let ${\widehat{y}}_i^{\left(t\right)}$ represent the outcome of the i-th tree at the t-th reputation that will be reformulated as Equation (5), with $q_t$ denoting the additive instance at the t-th reputation. According to this specific method of substitution, the goal function will alter Equation (6). Subsequently, to assist in the optimization procedure, a second-order Taylor expansion is carried out to approximate the specific objective function, as seen in Equation (7).

$${\widehat{y}}_i^{\left(t\right)}={\widehat{y}}_i^{(t-1)}+q_t(x_i)$$

(5)

$$L^{\left(t\right)}={\sum }_{i=1}^{n}l\left(y_i.{\widehat{y}}_i^{\left(t-1\right)}+q_t\left(x_i\right)\right)+\gamma .T+{\displaystyle \frac{1}{2}}\lambda {\sum }_{j=1}^T{w}_j^2$$

(6)

$$L^{\left(t\right)}\approx {\sum }_{i=1}^N\left[l\left(y_i,{\widehat{y}}_i^{\left(t-1\right)}\right)+g_i{q}_t\left(x_i\right)+{\displaystyle \frac{1}{2}}{h}_i{q}_t^2(x_i)\right]+\gamma .T+{\displaystyle \frac{1}{2}}\lambda {\sum }_{j=1}^T{w}_j^2$$

(7)

Given that previous iterations (t − 1) have been completed, the initial term l ($y_i,{\widehat{y}}_i^{(t-1)}$) is effectively constant and will be excluded from the optimization process. Moreover, each example will finally reside in a single leaf node; hence, the function will be expressed as Equation (8).

$${\widehat{L}}^{\left(t\right)}={\sum }_{j=1}^T\left[\left({\sum }_{iϵI_j}{g}_i\right)w_j+{\displaystyle \frac{1}{2}}\left({\sum }_{iϵI_j}{h}_i+\lambda \right)w_j^2\right]+\gamma .T$$

(8)

where $I_j$ represents the sample sets that correspond to leaf node j [45].

3.3.2. Multilayer Perceptron (MLP)

The MLP consists of several layers, starting with the input layer as well as ending with the output layer, having intermediate layers known as hidden layers. The common framework of an MLP is represented within Figure 4. Neurons at various levels are generally linked, and every link has a specific weight. Every node inside the hidden layer is certainly capable of carrying out two basic procedures, including summation and activation [46,47]. The summing procedure utilizes the multiplication of weights, input, and bias, as presented in Equation (9).

$$S_j={\sum }_{i=1}^n{w}_{ij}{I}_i+{\beta }_j$$

(9)

Here, n indicates the quantity of inputs, I_i signifies the i-th input magnitude, $w_{ij}$ signifies the link weight, and lastly, ${\beta }_j$ indicates the bias. The examination of the ANN’s efficiency is generally proven via the use of a loss function. A common technique consists of utilizing the MSE as the specified loss function [46,47]. The MSE calculates the cumulative sum of squared variances between the real and predicted magnitudes, as mentioned in Equation (10):

$$MSE={\sum }_{\mathrm{i}=1}^{\mathrm{n}}{\displaystyle \frac{{\left(y_i-\widehat{y_i}\right)}^2}{\mathrm{n}}}$$

(10)

3.3.3. Support Vector Regression (SVR)

This is a supervised learning method depending on the statistical learning theory presented by Vapnik [48]. The essential concept of SVR is the mapping of input features in a greater dimensional variable space utilizing a nonlinear mapping method. A linear regression method in the corresponding variable space is then acquired [49,50]. For a set of training datasets, {(x₁, y₁), …, (x_n, y_n)} from the true model, where x_i ϵ Rⁿ is the feature vector, and y_i ϵ R¹ is the output information. The regular form of the support vector regression under the provided variables C > 0 and ε > 0 could be explained in Equation (11):

$${min}_{w,b,\xi , {\xi }^{\ast }}{\displaystyle \frac{1}{2}}{w}^{T}w+C{\sum }_{i=1}^l{\xi }_i+C{\sum }_{i=1}^l{\xi }_i^{*}$$

(11)

Subject to the constraints outlined as follows:

$$\left\{\begin{array}{c}{w}^T \phi \left(x_i\right)+b-y_i\le \epsilon +{\xi }_i\\ y_i-w^T\phi \left(x_i\right)-b\le \epsilon +{\xi }_i^{*}\\ {\xi }_i{,\xi }_i^{*}\ge 0, i=1, \dots , l\end{array}\right.$$

(12)

where w is the vector feature, C indicates the trade-off between the tube violation and also regularization, and ξ_i* and ξ_i denote the lower and upper tube violations. ε is the vertical tube width, b is the parameter that determines the linear parameterization, and φ(x_i) maps x_i into a higher-dimensional space. The dual problem acquired utilizing the optimization technique (Equation 13) is given as:

$${min}_{\alpha ,{\alpha }^{*}}{\displaystyle \frac{1}{2}}{\left(\alpha -{\alpha }^{*}\right)}^{T}Q(\alpha -{\alpha }^{*})+\epsilon {\sum }_{i=1}^l({\alpha }_i+{\alpha }_i^{*})+{\sum }_{i=1}^l{y}_i({\alpha }_i+{\alpha }_i^{*})$$

(13)

with the constraints (i.e., α* and α are the dual features regarding every data point):

$$\left\{\begin{array}{c}{e}^T\left(\alpha -{\alpha }^{*}\right)=0\\ 0\le {\alpha }_i,{\alpha }_i^{*}\le C, i=1, \dots , l\end{array}\right.$$

(14)

Following the dual problem is certainly solved, the SVR function can be composed as

$$f\left(x\right)={\sum }_{i=1}^l\left(-a_i+a_i^{*}\right)K\left(x_i,x\right)+b$$

(15)

where K(x_i, x) is the Kernel function. Accordingly, the Sigmoid kernel was utilized as the Kernel function, that can be composed as

$$K\left(x_i, x\right)=tanh(a{x}_i^{T}x+r)$$

(16)

where r is the shifting variable that controls the threshold of mapping and also indicates the scaling variable of the input information. In the training of SVR, the optimization of the penalty component and Kernel function variable was acquired through cross-validation [50].

3.3.4. Elastic Net

This method is an ML and statistical approach, which includes the advantages of Lasso (L1) and Ridge (L2) regularization to enhance model interpretability and forecasting performance. It is specifically well-suited for handling information having remarkably correlated variables or when the range of predictors surpasses the range of real data. Prior to utilizing Elastic Net, it is important to comprehend the two primary elements it integrates:

• L2 as ridge regression reduces coefficients in the direction of zero through penalizing their squared degree. It performs properly when variables are extremely correlated, however, it does not carry out FS.
• L1 as Lasso promotes sparsity through penalizing the total magnitudes of coefficients, frequently establishing some coefficients to zero as well as carrying out FS. Nevertheless, this method is challenged when predictors are usually extremely correlated, as it randomly chooses one and neglects some others.

Elastic Net addresses the restrictions of Lasso and Ridge by merging their penalties (Equation (17)):

$$Minimize: {‖y-X\beta ‖}_2^2+{\lambda }_1{‖\beta ‖}_1+{\lambda }_2{‖\beta ‖}_2^2$$

(17)

where β: the coefficients to be approximated, X: the independent features (predictors), y: the dependent feature (target), λ₂: controls the L2 penalty (Ridge-like behavior), and λ₁: controls the L1 penalty (Lasso-like behavior).

Elastic Net presents an extra parameter, α, that ascertains the harmony between L1 and L2 penalties:

$$Penalty= \alpha {‖\beta ‖}_1+(1-\alpha ){‖\beta ‖}_2^2$$

(18)

where α = 0: Equal to Ridge regression, α = 1: Equal to Lasso regression, 0 < α < 1: A combination of both, providing Elastic Net its flexibility [51].

The hyperparameter matrix space of the above models, i.e., Xgboost, MLP, SVR and Elastic Net, is detailed in

Table 3. Summary of the hyperparameter matrix space of ML models.

Model	Hyperparameter
Xgboost	n_estimators: [10, 30, 50, 100, 200] learning_rate: [0.001, 0.005, 0.01, 0.05, 0.1] max_depth: [1, 2, 3, 4, 5]
MLP	hidden_layer_size: [hidden_layer_size: [5, 10, 50, 80, 100] activiation: [‘relu’, ‘tanh’, ‘logistic’] alpha: [0.0001, 0.001, 0.01, 0.1, 1, 10, 100] learning_rate: [‘aptive’, ‘invscaling’, ‘constant’]
SVR	C: [1, 10, 12, 14, 16, 18, 20, 22] gamma: [0.001, 0.01, 0.1, 1, 2, 5] epsilon: [0.001, 0.01,0.1, 1, 2, 4] kernel: [‘rbf’, ‘poly’, ‘sigmoid’]
Elastic Net	alpha: [0.0001, 0.001, 0.01, 0.1, 1, 10, 100] l1_ratio: [0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0] tolerance: [0.0001, 0.001, 0.01]

.

3.4. Model Evaluation

To examine the functionality of the ML methods, statistical indices, for example, Standard Deviation (StD) of Mean Error (ME), Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and Mean Absolute Percentage Error (MAPE) (Equations (19)–(21)), were used, where ${\widehat{y}}_i$ is the forecasted CO₂ emission, $y_i$ indicates the actual information, and n indicates sample size.

$$ME={\displaystyle \frac{1}{n}} {\sum }_{i=1}^n{y}_i-{\widehat{y}}_i$$

(19)

$$StD=\sqrt{{\sum }_{\mathrm{i}=1}^{\mathrm{n}}{\displaystyle \frac{{(y_i-{\widehat{y}}_i-ME)}^2}{\mathrm{n}}}}$$

(20)

$$MAE={\displaystyle \frac{1}{n}} {\sum }_{i=1}^n\left|y_i-{\widehat{y}}_i\right|$$

(21)

$$\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}=\sqrt{{\sum }_{\mathrm{i}=1}^{\mathrm{n}}{\displaystyle \frac{{\left(y_i-{\widehat{y}}_i\right)}^2}{\mathrm{n}}}}$$

(22)

$$\mathrm{M}\mathrm{A}\mathrm{P}\mathrm{E}={\displaystyle \frac{1}{n}}{\sum }_{i=1}^n\left|{\displaystyle \frac{y_i-{\widehat{y}}_i}{y_i}}\right|$$

(23)

3.5. Shapley Analysis

The ML model that indicates the best performance was determined based on the SHAP method. This method can be utilized in ML to evaluate the contribution of all variables in the model, which collectively delivers the forecasting [52]. The Shapley value regarding variable X_j within a model is generally provided by:

$$Shapley X_j={\sum }_{S\subseteq N\backslash \left\{j\right\}}{\displaystyle \frac{K!\left(P-K-1\right)!}{p!}}(f\left(S\cup \left\{j\right\}\right)- f(S))$$

(24)

where p is actually the overall range of variables, N\{j} is a set of entire feasible mixtures of variables not including X_j, S is a parameter in N\{j}, f(S) is generally the model forecasting with variables in S, as well as f (S∪{j}) is the model forecasting with variables in S plus variable X_j.

The interpretation of Equation (20) is that the Shapley value of a particular variable is generally its marginal contribution to model forecasting averaged over all feasible models with distinct combinations of variables. Shapley value has a quantity of beneficial properties like symmetry, efficiency, additivity, and dummy [53,54]. Symmetry indicates that two variables possess similar Shapley values if they play an equal role in the model. Efficiency pertains to the requirement that all variable contributions total up to the distinction between the forecasting and the average. Additivity requires that the aggregate of forecasting through specific models be identical to the forecasting from the mixture of almost all models. Dummy indicates that a variable has a Shapley value of actually zero when its marginal contribution to all feasible models is zero. The results of this method are reasonable as well as special if all the above attributes are satisfied [55,56].

4. Results and Discussion

4.1. Multicollinearity Analysis and FS Method

The annual rate of the dependent/independent variables between 1990 and 2023 is visualized in Figure 5. Considering the dramatic impact of COVID-19 on almost every aspect of daily life, in this study, data in 2020 was excluded to avoid any potential adverse impact of outliers on CO₂ emission forecasting. The dependent variable, i.e., total CO₂ by transportation TCT (t + 1), indicates a similar pattern with many independent variables such as Total Fossil Fuels Consumed by the Transportation (FFT), Total Energy Consumed by the Transportation (TET), Total Primary Energy Consumption (TPEC), etc. In addition, similar patterns were also frequently observed within independent variables (e.g., Population, Urban Population rate (UPR), GDP, Motor vehicle licensed drivers (MVLD)). For ML, even though the redundant features would not necessarily sacrifice the forecasting performance. However, the multicollinearity issue, to a certain extent, hinders proper interpretation of feature importance, which plays a significant role in decision-making for stakeholders/policymakers.

In order to mitigate the multicollinearity in the dataset, hierarchical feature clustering was implemented on independent variables only, and the Silhouette score was also employed to determine the optimal number of clusters. The results are illustrated in Figure 6. As shown in Figure 6a, variable pairs with a minimum ward distance were iteratively combined into hierarchical feature clusters with different colors until all variables were clustered. For instance, TPI and TPEI, which indicate a significant similarity in Figure 4, had the smallest Ward’s distance as shown on the right side of the purple clusters. Silhouette score of Figure 6b confirms the optimal number of clusters is 8. Detailed feature cluster is presented in Figure 7. Specifically, time series scaling was implemented in order to have a comparable range for the input variables. As the unit of variables in the same clusters varied, the unit description was not presented.

After clustering independent features, the next step is to determine the best candidate from each cluster. The results of the conservative feature selection mechanism based on Boruta feature selection and Spearman correlation are listed in

Table 4. Summary of feature selection.

Variable	Correlation	p-Value	Boruta Ranking	Cluster	Decision
TPI	0.684	<0.001	2	1	○
TPEI	0.711	<0.001	1	1	●
FFT	0.881	<0.001	1	2	●
TPEC	0.789	<0.001	1	2	○
BET	0.44	<0.05	1	3	○
TPEP	0.075	0.68	1	3	○
Population	0.449	<0.01	1	3	○
UPR	0.449	<0.01	1	3	○
GDP	0.45	<0.01	1	3	○
APF	0.517	<0.01	1	3	○
MVR	0.486	<0.01	1	3	○
AVMT	0.572	<0.001	1	3	●
RM	0.418	<0.05	2	3	○
MVLD	0.448	<0.05	1	3	○
AAT	0.245	0.18	7	4	○
RPC	−0.056	0.76	9	5	○
RGT	0.194	0.29	10	5	○
EUT	0.459	<0.05	3	6	○
GP	0.295	0.1	4	6	○
APDI	0.503	<0.01	1	6	●
COPP	0.272	0.13	5	6	○
UR	−0.483	<0.01	6	7	●
APC	−0.18	0.32	8	8	○

Note: ● implies inclusion and ○ signifies exclusion.

. Based on the proposed multi-step feature selection framework (hierarchical clustering-Boruta feature selection-Spearman correlation), variables including TPEI, FFT, AVMT, APDI and UR were determined as the selected independent feature subset for the following TCT (t + 1) forecasting task.

For one-step forecasting, i.e., TCT (t + 1), previous studies by Qiao et al. [22] also included the last historical value of CO₂ emission, specifically TCT (t) as an independent variable to improve ML model performance, which, however, did not statistically consider the autocorrelation of time series data. Inspired by Autoregressive model, this study measured the autocorrelation of TCT (t + 1), and the result indicated a significant relation with the last 3 steps of historical TCTs, i.e., $TCT(t+1)~TCT\left(t\right)+TCT(t-1)+TCT(t-2)$ as illustrated in Figure 8.

Based on the above analysis, the final feature subset was determined with a total number of 8, including TPEI, FFT, AVMT, APDI, UR, TCT (t), TCT (t − 1), and TCT (t − 2). In order to examine the performance of our proposed multistep feature selection framework, this study additionally employed another popular embedded feature selection method named 10-fold Cross Validated Recursive Feature Elimination with RF as the base model (RFE-RF) for comparison. The selected feature subset by RFE-RF was a total of 10 features, including FFT, TET, TPEP, TPEC, Population, UPR, GDP, TPI, MVLD, TCT (t), and TCT (t − 1). The original feature set without any feature selection was also employed as a baseline.

4.2. Modeling and Evaluation

The four popular ML methods, including Xgboost, SVR, MLP and ElasticNet, were employed for forecasting tasks. The dataset was split with a training-testing ratio of 80:20 without data shuffle. A grid search with 10-fold time series split was concluded to determine the best hyperparameter setting of each ML method based on different feature scenarios (the proposed, RFE-RF and original feature sets). The performance in terms of TCT (t + 1) forecasting for each ML model based on the best hyperparameter setting was listed in

Table 5. The performance of ML methods on TCT prediction based on different feature sets.

ML	Metrics	The Proposed FS		Original		RFE-RF
		Training	Testing	Training	Testing	Training	Testing
Xgboost	StD	0.045	58.823	0.003	39.562	0.002	39.334
	RMSE	0.046	59.280	0.003	47.849	0.002	58.574
	MAE	0.046	45.227	0.002	44.540	0.002	52.498
	MAPE	1.99 × 10−5	0.0243	1.32 × 10−6	0.023	1.22 × 10−6	0.027
MLP	StD	135.220	37.638	169.130	37.408	143.540	38.363
	RMSE	1796.848	1861.018	1760.353	1848.11	1788.91	1860.122
	MAE	1792.046	1860.637	1752.73	1847.738	1783.030	1859.74
	MAPE	0.987	0.993	0.964	0.986	0.982	0.993
SVR	StD	8.880	94.998	16.890	63.669	20.707	170.830
	RMSE	8.992	96.994	16.978	89.629	26.676	89.045
	MAE	5.433	79.123	8.346	63.084	17.325	64.5
	MAPE	0.002	0.042	0.004	0.034	0.009	0.035
Elastic Net	StD	28.977	45.257	13.837	41.652	35.404	48.434
	RMSE	28.977	45.533	13.837	136.464	36.873	55.999
	MAE	22.462	30.6	9.821	129.952	29.393	54.405
	MAPE	0.012	0.016	0.005	0.069	0.015	0.029

. Figure 9 visualized the performance regarding RMSE. It is observed that despite Xgboost showing the best training performance, the significant increase in test error suggests an overfitting issue of Xgboost in all scenarios, which means this ML method hard remembered all training information but failed to learn the general pattern from the dataset. MLP surprisingly indicated a bad prediction performance no matter the feature settings. It may perhaps be related to the difficulty in hyperparameter tuning and the small dataset. Further research is required to understand such an unsatisfying performance of MLP. SVR in general reflected the same overfitting problem as Xgboost. Despite a higher training error compared to Xgboost and SVR, the Elastic Net indicated the best testing performance based on the proposed FS framework, and the testing error was slightly higher than the training error, which means Elastic Net learned the general pattern of the training dataset and proved its generalization capability. Comparing different datasets, it is noticed that the performance of MLs based on the feature set generated by the proposed FS framework is comparable to the original and RFE-RF feature sets without significant compromise. However, considering the significant reduction in the number of independent variables, the requirement for data acquisition and computation power/time is modified from the original 24 features to now only five features. Additionally, the redundant features were also eliminated compared to RFE-RF which mitigated the multi-collinearity issue and improved the interpretation capability.

The optimal hyperparameter setting of Elastic Net based on 10-fold TSGSCV tuning and the proposed feature subset was: ‘alpha’: 0.1, ‘l1_ratio’: 0.9 and ‘tolerance’: 0.01. Detailed training and testing performance of Elastic Net based on different feature settings is presented in Figure 10. All feature set scenarios showed, in general, a promising training performance, with using the original feature set outperforming the other two feature settings. While the dramatic decrease in testing indicates the overfitting of the feature set. It, on the other hand, proved the philosophy of “garbage in, garbage out”. Learning information about redundant or irrelevant features would significantly deteriorate the capability of ML models in capturing the main pattern within the data. In comparison, RFE-RF determined a sub-optimal feature subset, which, despite using fewer variables, had overall performance on both the training and testing datasets that was acceptable without significant deviation from the actual value of TCT (t + 1). When modelling with a feature subset generated by the proposed feature selection framework, the best testing performance was achieved without overfitting or underfitting.

4.3. SHAP Analysis

Considering the outperformance of Elastic Net in TCT (t + 1) forecasting, this study only employed Elastic Net for explainable ML. The result of SHAP analysis is illustrated in Figure 11. The vertical color bar at the right side of the Figure 11a,b signifies the value of independent variables (e.g., red indicates a higher value, and blue indicates a lower value); the x-axis reflects the SHAP value or the contribution of an independent variable to the change of TCT (t + 1) compared to its average value. The ordering of variables from top to bottom indicates the feature contribution rank. It is observed that AVMT, FFT and APDI were the top three major contributors to TCT (t + 1), with AVMT and FFT contributions positively associated with TCT (t + 1), while a negative association existed between APDI contribution and TCT (t + 1). UR and TPEI contribute the least with, respectively, negative and positive associations to TCT (t + 1). It is reasonable to observe the importance of historical TCTs in predicting TCT (t + 1)_. However, it is interesting to find that TCT (t − 1) and TCT (t − 2) contributed negatively to TCT (t + 1), which is in contrast to the positive association between TCT (t) and TCT (t + 1). Figure 11c depicted the feature distribution of independent variables and their individual SHAP values. A linear relation between independent variables and their associated SHAP contribution to TCT (t + 1) was detected. Such a linear relationship may probably relate to the linear nature of the Elastic Net method, which also emphasized that the embedded model in SHAP analysis has a critical and fundamental impact on SHAP values.

In this study, we proposed a multistep feature selection framework that combined the hierarchical clustering method, Boruta feature selection, and spearman correlation analysis. The core idea of the framework is to remove the redundant/multicollinear and irrelevant variables that hinder a proper interpretation of feature importance in ML applications. Based on the proposed multistep feature selection framework, a total of 24 original variables were effectively reduced to 5 of the most representative variables. The Elastic Net with the selected five variables outperformed all other models in all feature set scenarios. The results proved the feasibility and practicality of the proposed framework.

It is noticed that, for small datasets, the simple linear Elastic Net outperformed the more advanced ML methods, including SVR, Xgboost and MLP. Perhaps due to that, advanced ML is more capable of learning information in training datasets, which, to a certain degree, compromises its generalization capabilities. This also emphasizes the theory of Occams’ Razor by Webb [57]. Relying on more advanced and complicated ML or DL methods, which may not be the optimal model for a small dataset like in this study with a dataset of 34 samples. Meanwhile, small datasets may also amplify the importance of individual independent variables. In other words, feature selection and ML prediction may be highly sensitive to the value of individual variables, especially some extreme values. The reason is that no matter the feature selection (measuring the distance between variables) or model prediction (gradient descent for SVR, Elastic Net, and MLP, and enthalpy for Xgboost), the core is variance measurement, which is very sensitive to the extreme value, especially when it comes to small datasets. A proper feature selection primarily determines the quality of feature importance interpretation.

5. Policy Recommendations and Limitations

5.1. Policy Recommendations

The results of this particular research presented an extensive framework for comprehending and predicting CO₂ emissions throughout the transport industry, providing useful insights pertaining to market stakeholders, urban planners, and policymakers. By discovering important contributors, including AVMT, FFT, APDI, TPEI, and UR, this study highlighted the interconnected characters of micro- and macro-level variables in creating emissions trends.

At the macro stage, variables, for example, FFT and TPEI emphasized the crucial requirement for shifting from fossil fuel power systems to alternative and low-carbon power resources. Policymakers must prioritize opportunities in clean energy infrastructure, for example, green hydrogen creation as well as electrified transport systems, to decrease reliance on non-renewable sources. Economic parameters, such as the UR, additionally underline the significance of developing environmental plans with socioeconomic policy to make sure that techniques for emission decline are generally sustainable and equitable.

As a micro stage, variables such as AVMT and APDI to CO₂ emissions signify an urgent request to address transport efficiency and need. Policies promoting multimodal transport networks, such as improved public transportation systems and shared mobility methods, as well as active transport (e.g., infrastructure for walking and biking), will considerably decrease individual automobile reliance. Motivating the ownership of low-emission and electric automobiles via financial assistance, tax rewards, as well as charging infrastructure development will additionally minimize emissions at the community and individual stages.

Furthermore, the incorporation of data-driven methods, such as the suggested multi-stage predicting system in policymaking, will allow live monitoring and dynamic modifications to strategies. The 33-year information utilized in this particular study offers a powerful base for analyzing the long-term influences of historical policy interventions, as well as predicting upcoming developments under different circumstances. Policymakers are able to leverage this kind of examination to model adaptive procedures, which take into account economic and technical development and demographic adjustments in the transport sector.

By concentrating on both systemic and immediate options, the structure offered in the following research provides policymakers with the methods required to create knowledgeable, data-driven judgements. These techniques will assist eco-friendly advancement through attaining substantial reductions in CO₂ emissions, contributing to worldwide attempts to overcome climate change as well as changeover to a low-carbon future.

Besides its functional worth in predicting United States transport emissions, the suggested explainable ML structure and multi-stage FS are inherently adaptable, as well as can be used in other application domains, regions, and datasets. Its scientific incorporation of SHAP-based model interpretation, Boruta-Spearman evaluation, and hierarchical clustering guarantees powerful efficiency even in situations with limited data availability and huge feature multicollinearity. Consequently, the structure can function as a general application for environmental forecasting projects beyond the United States, allowing worldwide policymakers to extract valuable information from localized data.

5.2. Limitations

While this particular research delivers useful information on CO₂ emissions forecasting in the transport industry, some restrictions need to be considered.

The study first incorporated an extensive dataset of 24 explanatory parameters and one dependent parameter, addressing a broad variety of transport, macroeconomic, environmental, and energy elements. Nevertheless, through FS techniques (Boruta-Spearman correlation and Spearman correlation analysis), only 5 essential parameters were kept for the forecasting system. Though this specific technique improves model performance and decreases multicollinearity, it might unintentionally exclude some other possibly substantial parameters. For example, components including BET, EUT, RGT, or AAT could have had an indirect, however substantial, impact on CO₂ emissions and were not investigated completely. Although the chosen parameters (TPEI, FFT, AVMT, APDI, and UR) were demonstrated to be the most influential regarding prediction, this concentrates on essential predictors and restricts the capability to discover relationships among less notable parameters. This specific restriction limits the comprehensive knowledge of emissions dynamics, especially within complicated systems with various interconnected variables.

The research depends on historical information for a period of 33 years from 1990. Although this extensive dataset delivers a strong foundation for evaluation, it may not completely account for emerging trends, for example, quick progress in modifications in urbanization behavior or electric automobile ownership. These kinds of development could impact the precision and applicability of the suggested predicting system in upcoming scenarios.

Finally, the research only focuses on the transport industry throughout the United States. Because of this, the results might not be applicable to other nations or areas without some changes that consider differences in transport systems, energy resources, and socioeconomic circumstances. Upcoming investigations could discover cross-country evaluations or broaden the examination to consist of further areas, for example, business or residential energy utilization.

Despite these types of restrictions, the suggested framework displays the possibility of combining FS and ML methods to forecast CO₂ emissions, paving the way for more enhanced and flexible models in upcoming research.

6. Conclusions and Future Research

In this research, we proposed a multi-stage framework for forecasting transportation CO₂ emissions using FS and ML in the United States. A total of 25 variables related to transportation, socioeconomics, environment, and energy were originally selected. Hierarchical clustering, Boruta FS and Spearman were employed to select representative but not repeated variables. Eight important variables, including TPEI, FFT, AVMT, APDI, UR, TCT (t), TCT (t − 1), and TCT (t − 2), were selected as the independent feature subset for the following TCT (t + 1) forecasting task. For CO₂ emissions forecasting task, 4 popular ML methods, including Xgboost, SVR, MLP and ElasticNet, were employed. The results suggested an overfitting issue of Xgboost and SVR. MLP implied tuning difficulty. Elastic Net in general indicated the best testing performance (RMSE = 45.53, MAE = 30.6, and MAPE = 0.016) based on the proposed FS framework, which means Elastic Net learned the general pattern of the training dataset and proved its generalization capability. Accordingly, it is noticed that the performance of MLs based on feature sets generated by the proposed FS framework is comparable to the original and RFE-RF feature sets without significant compromise. The results of the SHAP analysis showed that AVMT, FFT and APDI were the top three major contributors to TCT (t + 1), with AVMT and FFT contributions positively associated with TCT (t + 1), while a negative association existed between APDI contribution and TCT (t + 1). UR and TPEI contribute the least with, respectively, negative and positive associations to TCT (t + 1). It is interesting to find that TCT (t − 1) and TCT (t − 2) contributed negatively to TCT (t + 1), which is in contrast to the positive association between TCT (t) and TCT (t+1). Upcoming investigation could discover the components of extra factors, for example, alternative energy adoption rates, electric automobiles, traffic jam factors, and policy involvement results (e.g., fuel performance specifications, carbon taxes) to improve the comprehensiveness of the predicting model. Performing comparative research throughout various nations with different financial conditions and transport infrastructures could offer further information into the worldwide trends of transport emissions. Additionally, upcoming research could utilize more superior AI-based techniques, for example, deep learning and hybrid methods, to capture intricate temporal dependencies as well as non-linear associations among parameters.