Explainable Machine Learning for Urban Carbon Dynamics: Mechanistic Insights and Scenario Projections in Shanghai, China

Abstract

Using Shanghai as a case study, this paper estimates multi-sector urban carbon emissions by integrating multi-source statistical data from 2000 to 2023 with IPCC guidelines. Via rolling-window time-series validation, XGBoost is the most reliable model. To better understand the underlying drivers, explainable machine-learning approaches, including SHAP and the Friedman H-statistic, are applied to examine the nonlinear effects and interactions of population scale, industrial energy efficiency, investment structure, and infrastructure. The results suggest that Shanghai’s emission pattern has gradually shifted from a scale-driven process toward one dominated by structural change and efficiency improvement. Building on an incremental framework, four scenarios, Business-as-Usual, Green Transition, High Investment, and Population Plateau, are designed to simulate emission trajectories from 2024 to 2060. The simulations reveal a two-stage pattern, with a period of rapid growth followed by high-level stabilisation and a weakening path-dependence effect. Population agglomeration, economic growth, and urbanisation remain the main contributors to emission increases, while industrial upgrading and efficiency gains provide sustained mitigation over time. Scenario comparisons further indicate that only the Green Transition pathway supports early peaking, a steady decline, and long-term low-level stabilisation. Overall, this study offers a data-efficient framework for analysing urban carbon-emission dynamics and informing medium- to long-term mitigation strategies in megacities.

1. Introduction

The rapid growth of carbon emissions has become one of the most pressing global challenges of the modern era [1]. Since the Industrial Revolution, human activities—most notably the large-scale use of fossil fuels—have been the dominant driver of rising greenhouse gas emissions [2]. According to the International Energy Agency (IEA), carbon dioxide (CO₂) accounts for nearly 80% of total greenhouse gas emissions worldwide [3]. The IPCC Sixth Assessment Report further highlights carbon emission control as a core issue in global climate change mitigation.

Cities play a central role in this process. As hubs of population, economic activity, and infrastructure, urban areas concentrate both energy consumption and carbon emissions, making them critical arenas for climate and carbon governance [4]. Current estimates suggest that cities are responsible for approximately 71–76% of global carbon emissions [5]. With around half of the world’s population now living in urban areas, urban emissions could rise to nearly 80% of the global total by 2100 in the absence of effective mitigation measures [6]. Excessive urban carbon emissions not only contribute to global warming but also increase the frequency and intensity of extreme meteorological, climatic, and hydrological events [7,8]. As a result, reducing emissions in cities is widely recognised as a key pathway for addressing climate change.

Against this backdrop, accurately quantifying urban carbon emissions and examining their spatiotemporal patterns and driving mechanisms are essential for supporting regional sustainable development, refining low-carbon strategies, and advancing national “dual-carbon” goals. Identifying the key drivers of carbon emissions is a fundamental prerequisite for effective mitigation [7]. Rigorous analysis of emission drivers can provide an empirical basis for targeted policy design. Previous studies have applied methods such as remote sensing, energy accounting, and economic–statistical modelling to estimate urban emissions, analyse temporal dynamics, and explore mitigation pathways. An expanding body of literature therefore underscores the pivotal role of cities in achieving carbon reduction targets [8].

Existing studies on urban carbon-emission forecasting have largely concentrated on emissions from single sources, such as urban transport systems or energy sectors [9]. By contrast, relatively limited attention has been paid to the evolution of urban carbon emissions under the combined influence of multiple drivers, particularly in the context of megacities. For instance, Azizalrahman developed a model to examine the effects of urbanisation on carbon emissions by decomposing emission drivers into residential, commercial, and industrial components, thereby highlighting the interactions among different factors [9]. While some studies have begun to address the multi-factoral nature of urban carbon emissions, much of the existing literature remains focused on emission accounting, with insufficient exploration of the coupled and nonlinear effects among multiple drivers. Other contributions have analysed urban carbon emissions and their drivers from perspectives such as production, consumption, and income [10], yet an integrated understanding of their joint dynamics is still lacking. Further studies have analysed urban carbon emissions and their driving forces from multiple perspectives, including production, consumption, and income [11].

To address these gaps, this study selects Shanghai as a representative megacity and systematically investigates the driving mechanisms of urban carbon emissions and their future evolution using machine-learning approaches under constrained feature conditions. First, urban carbon emissions in Shanghai from 2000 to 2023 are estimated using multi-source statistical data. Second, a comparative evaluation of multiple modelling approaches is conducted, and XGBoost is selected in combination with rolling out-of-sample (OOS) validation to identify both single-factor and multi-factor mechanisms influencing urban carbon emissions. Based on the SHAP (Shapley Additive Explanations) framework, the marginal contributions and nonlinear interaction effects of explanatory variables are quantified, allowing for an examination of how individual factors and their interactions shape emission dynamics. This analysis captures the dynamic roles of population, industrial structure, socioeconomic conditions, public services, and other multidimensional elements. Third, building on an incremental modelling framework and a set of alternative scenario assumptions, this study simulates carbon-emission trajectories under different development pathways, including Business-as-Usual, Green Transition, High-Investment, and Population Plateau scenarios. These simulations reveal distinct emission-response patterns across development stages and provide insights into feasible pathways and optimisation strategies for future carbon reduction in Shanghai.

2. Materials and Methods

2.1. Study Area

Shanghai is located on the eastern coast of China, at the estuary of the Yangtze River. It is a core national hub of economy, finance, shipping, and technological innovation, and one of the world’s most representative megacities. As of 2023, Shanghai’s permanent population reached approximately 24.75 million, with a regional GDP of 4.72 trillion RMB and a per capita GDP exceeding 190,000 RMB, ranking among the highest globally. The urbanisation rate exceeds 89%, and the built-up area spans 1297 km². Figure 1 illustrates the geographical location of Shanghai. The city’s terrain is predominantly flat, characterised by the alluvial plain of the Yangtze River Delta. Spatially, Shanghai occupies a central position within the Yangtze River Delta urban agglomeration, functioning as a key regional hub for economic activity, technological innovation, and international trade. Its high-density urban form, complex industrial structure, and extensive transportation networks make it a representative case for examining the carbon-emission characteristics and low-carbon transition pathways of global megacities.

In recent years, Shanghai has actively responded to the national “dual-carbon” strategy [12,13], establishing the country’s first local carbon-emissions trading market and implementing multi-level pilot programs in areas such as energy structure optimisation, green building promotion, and carbon reduction through urban regeneration. As a critical node city within global networks, Shanghai faces a typical dual constraint between high-intensity economic activity and ambitious carbon-reduction targets. Its low-carbon transition pathway, therefore, holds significant relevance for other megacities. Figure 2 presents the technical roadmap of this study.

2.2. Data Sources and Preprocessing

2.2.1. Categories of Urban Carbon Emission Accounting

This paper applies the Intergovernmental Panel on Climate Change (IPCC) methodology to analyse HCF data and assess the associated environmental impacts. The IPCC method is widely recognised for its convenience and efficiency in carbon emissions calculation, featuring advantages such as ease of data acquisition, standardised computation procedures, broad applicability, and rapid updates [14]. This approach converts the consumption of various products into standard coal equivalents, applies IPCC emission factors to derive corresponding carbon-emission coefficients, and subsequently calculates their carbon emissions through a series of established formulas. As a unified, scientific, and transparent tool for assessing global carbon emissions, the IPCC method facilitates a clearer understanding of greenhouse gas emissions and supports global climate change mitigation efforts.

The formula used for calculating carbon emissions under the IPCC method is presented in Equation (1). For categories included in the energy balance sheets, units have already been converted into standard coal equivalents. For categories without energy balance data, such as residential activities, electricity consumption across different sectors is used as a proxy for estimating energy use. Electricity data are particularly suitable due to their accessibility, reliance on a mature grid-monitoring system, capacity to reflect real-time energy-consumption dynamics, and close linkage to economic activities.

$$CE=\sum _{i}MiTi$$

(1)

In Equation (1), CE denotes carbon emissions, M represents standard coal consumption, and T refers to the carbon-emission factor of standard coal. This paper employs a conversion factor that assumes that 1 kg of standard coal produces 2.4567 kg of CO₂.

The calculation in this paper covers the primary, secondary, and tertiary sectors. The primary sector comprises agriculture, forestry, animal husbandry, and fisheries; the secondary sector encompasses industry and construction; the tertiary sector encompasses transportation, storage, postal services, wholesale and retail trade, as well as accommodation and catering. Transportation includes road, rail, waterway, and aviation. Carbon emissions from residential areas are estimated based on the consumption of urban and rural households.

For the calculation of carbon emissions in the primary sector (Table 1), agricultural emissions mainly account for the carbon released during crop cultivation, including emissions from the planting process and fertiliser application. Because agriculture in the study area primarily involves annual herbaceous crops, the biomass residues are usually returned to the soil after the growing cycle, thereby forming part of the carbon cycle; thus, carbon sequestration is not a significant factor. Given that rice and wheat together account for approximately 80% of grain production in the study area, agricultural carbon-emission accounting concentrates on emissions from rice and wheat cultivation. Their emission factors encompass production, packaging, product transportation, pesticide production, agricultural film production, and tool production, as well as storage/milling activities.

Carbon emissions from pasture and animal husbandry primarily originate from enteric fermentation in livestock, with a particular emphasis on methane emissions from pigs, cattle, and sheep. Emissions from fisheries encompass the carbon emissions of aquaculture, which varies depending on rearing methods and can be categorised into intensive aquaculture and ecological free-range systems. Emission factors for the primary sector are sourced from the National Greenhouse Gas Inventory of the People’s Republic of China [15] and the 2006 IPCC Guidelines for National Greenhouse Gas Inventories [16]. In cases of missing data, approximate-year production values or relevant policy-based estimates are applied.

Table 1. Calculation of Carbon Emissions in the Primary Sector.

Category	Factor	Emission Factor	Description/Source
Agriculture	Fertiliser production, transportation, and application	5.24 kg CO2e/kg	The fertiliser factor is based on compound fertiliser, utilising nitrogen (N), phosphorus (P2O5), and potassium (K2O) application rates and ratios from the 2023 Spring Scientific Fertilisation Guidelines for Vegetables [17]. The N:P:K ratio is maintained at approximately 1:0.2:1.2. Emission factors for nitrogen, phosphorus, and potassium fertilisers are derived from IPCC calculations [16].
	Rice	1.08 kg CO2e/kg	Shanghai’s total grain output in 2024 was 983,000 tons, with summer grain (mainly wheat) accounting for 16.17% (approximately 159,000 tons). Although rice output was not directly reported, rice and wheat together account for over 80% of grain production in the study area [18]. Emission factors follow national inventory and IPCC guidelines [15]. Emission factor from the National Greenhouse Gas Inventory of the People’s Republic of China [15].
	Wheat	0.91 kg CO2e/kg
Animal Husbandry	Enteric fermentation—cattle	1232 kg CO2e/head·year	Emission factor from the National Greenhouse Gas Inventory of the People’s Republic of China [15].
	Enteric fermentation—goats	182 kg CO2e/head·year	[15]
	Enteric fermentation—pigs	35 kg CO2e/head·year	[15]
Fisheries	Aquaculture area	Intensive aquaculture (294.37)	[19]

Table 1. Calculation of Carbon Emissions in the Primary Sector.

Category	Factor	Emission Factor	Description/Source
Agriculture	Fertiliser production, transportation, and application	5.24 kg CO2e/kg	The fertiliser factor is based on compound fertiliser, utilising nitrogen (N), phosphorus (P2O5), and potassium (K2O) application rates and ratios from the 2023 Spring Scientific Fertilisation Guidelines for Vegetables [17]. The N:P:K ratio is maintained at approximately 1:0.2:1.2. Emission factors for nitrogen, phosphorus, and potassium fertilisers are derived from IPCC calculations [16].
	Rice	1.08 kg CO2e/kg	Shanghai’s total grain output in 2024 was 983,000 tons, with summer grain (mainly wheat) accounting for 16.17% (approximately 159,000 tons). Although rice output was not directly reported, rice and wheat together account for over 80% of grain production in the study area [18]. Emission factors follow national inventory and IPCC guidelines [15]. Emission factor from the National Greenhouse Gas Inventory of the People’s Republic of China [15].
	Wheat	0.91 kg CO2e/kg
Animal Husbandry	Enteric fermentation—cattle	1232 kg CO2e/head·year	Emission factor from the National Greenhouse Gas Inventory of the People’s Republic of China [15].
	Enteric fermentation—goats	182 kg CO2e/head·year	[15]
	Enteric fermentation—pigs	35 kg CO2e/head·year	[15]
Fisheries	Aquaculture area	Intensive aquaculture (294.37)	[19]

In the secondary sector, the energy consumption of industry and construction is calculated using the following formula:

$${CE}_{ind}=C{E}_{ind-coal}+C{E}_{ind-elec}$$

(2)

$$C{E}_{ind-elec}=E{C}_{ind}\times E{F}_{grid}\left(t\right)$$

(3)

$$C{E}_{ind-coal}=S{C}_{ind}\times E{F}_{coal}$$

(4)

where ${CE}_{ind}$ denotes carbon emissions from the secondary sector; $C{E}_{ind-coal}$ refers to carbon emissions from industrial coal consumption; $C{E}_{ind-elec}$ represents carbon emissions from electricity consumption in the construction sector; $E{C}_{ind}$ denotes annual electricity consumption in the construction industry; $E{F}_{grid}\left(t\right)$ is the power-grid emission factor for year t; $S{C}_{ind}$ represents standard coal consumption in the industrial sector (kg of standard coal); and $E{F}_{coal}$ corresponds to 2.4567 kg CO₂ per kg of standard coal.

Energy consumption in industry and construction consists of electricity consumption and standard coal consumption. For electricity consumption, the associated carbon emissions have shown a continuous decline with the increasing adoption and penetration of clean energy. In Shanghai, the grid emission factors for 2010, 2012, and 2018 were 0.7934 t CO₂/MWh, 0.6241 t CO₂/MWh, and 0.5641 t CO₂/MWh, respectively. It is important to note that, as one of China’s pilot carbon markets, Shanghai had long adopted a grid emission factor of 0.788 t CO₂/MWh, calculated from the 2010 Shanghai Energy Balance Sheet and the municipal greenhouse gas inventory.

In February 2022, the Shanghai Municipal Bureau of Ecology and Environment issued the Notice on Adjusting the Emission Factors in the Municipal Greenhouse Gas Accounting Guidelines [20], which revised the grid emission factor used for calculating carbon emissions from purchased electricity from 0.788 t CO₂/MWh to 0.42 t CO₂/MWh, reflecting the significant increase in the share of clean and renewable electricity in Shanghai over the past decade [21].

For other years, grid emission factors are obtained from relevant power grid policies and official documents. In this paper, the carbon-emission calculations use the electricity emission factor corresponding to each specific year. Due to data unavailability for 2000 and 2005, the 2005 grid emission factor is used as a substitute for both years. The power grid factors before 2017 are from reference [22], of which the 2008 data are from reference [23]. The Shanghai power grid factors from 2000 to 2023 are shown in Figure 3.

In the calculation of transportation-related carbon emissions:

$$C{E}_{\mathrm{trans}}=\sum _{i}T{P}_i/C{O}_i\times EF{G}_i+T{G}_i\times EF{G}_i$$

where $C{E}_{\mathrm{trans}}$ represents the carbon emissions generated by urban transportation activities (tCO₂), and i denotes the type of transportation sector (including rail, road, waterway, and aviation). $T{P}_{\mathrm{i}}$ denotes passenger turnover (passenger-kilometres), and $C{O}_i$ represents the conversion coefficient between passenger turnover and freight turnover. $T{G}_i$ denotes freight turnover (ton-kilometres). $EF{G}_i$ is the turnover-based carbon emission factor. Passenger turnover for rail and waterway transport is converted using 1 passenger-kilometre = 1 ton-kilometre; for road transport, 10 passenger-kilometres = 1 ton-kilometre; and for aviation, 13.89 passenger-kilometres = 1 ton-kilometre [24].

In the calculation of residential consumption

$$C{E}_{res}={EC}_{res}\times E{F}_{grid}\left(t\right)$$

(5)

where $C{E}_{res}$ refers to carbon emissions from residential consumption (tCO₂), ${EC}_{res}$ denotes household electricity consumption (kWh), and $E{F}_{grid}\left(t\right)$ represents the annual grid CO₂ emission factor (kgCO₂/kWh).

2.2.2. Machine Learning–Based Prediction of Urban Carbon Emission Drivers

This paper employs two ensemble-learning methods, LightGBM and XGBoost. Both approaches are variants of the Gradient Boosting Machine (GBM), which train a sequence of weak learners, each attempting to correct the errors of the preceding learner. This iterative process enables the model to progressively capture complex relationships within the data, thereby enhancing its predictive performance. In gradient boosting, weak learners are sequentially fitted to the ensemble model, and the training dataset is updated at each iteration to reflect the strengths and weaknesses of the current ensemble more effectively.

The dataset is partitioned using a holdout method, with 80% allocated for training and 20% for testing. Model training is conducted using the Scikit-learn library. Model performance is evaluated using the coefficient of determination (R²) and mean squared error (MSE), and the optimal model is selected through hyperparameter tuning.

Given the relatively small dataset (24 years of observations, 77 features, and 1848 data points), this paper employs annual rolling validation (Rolling Forecast Origin Evaluation) in conjunction with internal validation sets and hyperparameter optimisation to further enhance the performance of machine-learning models.

In each forecasting iteration, the model is trained exclusively on historical data available up to the current year and is then evaluated on a subsequent future year, thereby enabling multiple independent out-of-sample (OOS) tests. Compared with a single train–test split, this approach provides a more reliable assessment of the model’s true generalisation performance under small-sample conditions and allows the stability of parameter estimation to be evaluated through variations in prediction errors across different forecasting horizons.

Rolling-window out-of-sample (OOS) validation can be regarded as an approximate test of model stability under different training-window lengths. In the rolling procedure, each forecasting step is based on a slightly different training period (e.g., training on 2000–2015 to predict 2016, training on 2000–2016 to predict 2017, and so forth). As a result, the model is repeatedly subjected to out-of-sample evaluation under varying training-sample sizes, which indirectly reflects the influence of limited sample size on model stability. Such approaches to assessing model robustness in time-series settings have been widely supported in the machine-learning literature [25,26,27]. By progressively expanding the training window, rolling validation further enables the examination of how prediction errors and model parameters evolve with increasing sample size, consistent with previous studies [28,29].

In the annual rolling time-series validation scheme adopted in this study (Rolling Forecast Origin Cross-Validation), the sample is ordered chronologically by year t = 1, 2, …, T, with each year corresponding to a feature vector Xt∈Rp and a target value ${\widehat{\mathrm{y}}}_t$.

For each test year t = k + 1, k + 2, …, T, all preceding years are used as the training set.

$$D^{\left(t\right)}train=\left\{\right(x_i,y_i\left)\right|i<t\}$$

(6)

$$D^{\left(t\right)}test=\left\{\left(x_t,y_t\right)\right\}$$

(7)

For each rolling window, the model is retrained and used to generate the corresponding annual carbon-emission prediction ${\widehat{\mathrm{y}}}_t$. It produces a continuous sequence of rolling forecasts:

$${\widehat{\mathrm{y}}}_t=f^{\left(t\right)}\left(x_t\right),t=k+1,k+2,\dots ,T$$

(8)

where $f^{\left(t\right)}\left(x_t\right)$ denotes the model trained using all data prior to year t. Model performance is evaluated using the coefficient of determination R² and the root mean square error (RMSE) for LightGBM-TPE, defined as follows:

$$R^2=1-{\displaystyle \frac{{\sum }_t{\left(y_t-{\widehat{\mathrm{y}}}_t\right)}^2}{{\sum }_t{\left(y_t-\overline{\mathrm{y}}\right)}^2}}$$

(9)

$$RMSE=\sqrt{({\displaystyle \frac{1}{n}})\sum _t{\left(y_t-{\widehat{\mathrm{y}}}_t\right)}^2}$$

(10)

where $\overline{\mathrm{y}}$ is the mean of the observed values and n is the number of test samples.

Within each rolling training set, an inner temporal split is applied, using the final 20% of the training period years as the validation set. This validation subset is used for hyperparameter optimisation via the Tree-structured Parzen Estimator (TPE) algorithm. TPE constructs probabilistic models to estimate the distribution of the objective function within the hyperparameter space, selects promising hyperparameter configurations, evaluates their performance, and iteratively updates the probability model. By concentrating on performance improvement while exploring previously untested regions of the hyperparameter space, TPE can identify near-optimal configurations with relatively few iterations. This makes it particularly advantageous for optimising complex machine-learning models, substantially reducing computational cost and time. This paper utilises the TPE algorithm to optimise model hyperparameters, aiming to achieve the best possible predictive performance.

2.2.3. XGBoost Model with SHAP-Based Interpretability

To enhance model transparency and policy relevance, this paper incorporates an Explainable Machine Learning framework to analyse the internal decision-making mechanisms of the predictive model quantitatively. Explainable machine learning aims to maintain predictive accuracy while ensuring that variable contributions, marginal effects, and interaction relationships are traceable, thereby overcoming the “black-box” characteristics of tree-based and other nonlinear models. This paper applies SHAP (Shapley Additive Explanations) and feature-importance ranking to interpret the key drivers of carbon emissions under different scenarios, ensuring the interpretability and policy credibility of the results.

This study uses the Shapley Additive exPlanations (SHAP) library to interpret the best-performing trained model and to determine the contribution of each feature to the model output. SHAP originates from cooperative game theory and computes the marginal contribution of each feature to the predicted outcome through its Shapley value. SHAP constructs an additive explanation model in which all features are treated as contributors to the final prediction. For each sample, the method assigns a Shapley value to every feature, indicating its contribution or importance to the prediction. SHAP-based feature-importance ranking is used to interpret the optimal model and to guide feature selection during the model training and prediction processes.

2.2.4. Model Training and Rolling Validation Strategy

To fully capture the temporal dependence and inertia characteristics of urban carbon-emission indicators, the model adopts a rolling-origin evaluation strategy. In this approach, data from several preceding years are used as the training set to predict the carbon-emission level of the subsequent year, and the window is progressively advanced. This method enables simultaneous assessment of the model’s temporal robustness and predictive capability.

On the basis of the original explanatory variables, this paper further introduces lagged features of the target variable, including the one-period lag $y_{t-1}$, the two-period lag $y_{t-2}$, and the two-period moving average ${\overline{\mathrm{y}}}_{t-2,t-1}$. Incorporating these lag terms enhances the model’s ability to capture trends and fluctuations in time-series regression while avoiding data leakage that could occur from using future information directly. Given the relatively large number of input variables and the potential for multicollinearity and noise, this paper computes the Spearman rank correlation between each predictor and the target variable within each rolling training window. It retains the top K features with the highest absolute correlations (with K set to 20 in this study), while forcibly retaining the lagged terms. This approach preserves the core explanatory information and improves model robustness and generalisation under small-sample conditions.

The transferability of the proposed methodology across cities depends primarily on the structure of data availability in the target city, rather than on its population size or stage of development. Specifically, the modelling framework can be fully replicated as long as a city can provide the following: (1) continuous, multi-year carbon-emission or energy-consumption accounting data compiled under consistent statistical definitions; (2) basic statistical indicators covering key dimensions such as population, industrial structure, investment activities, and public services; and (3) relatively consistent temporal scales and statistical standards. Under these conditions, the complete analytical workflow proposed in this study—including IPCC-based emission accounting, XGBoost modelling within a rolling time-validation framework, SHAP-based explainable analysis, and Δ-Model–based scenario extrapolation—can be directly implemented. In this sense, the proposed approach exhibits strong applicability and scalability for megacities with well-established statistical systems and long-term data continuity, such as Beijing, Guangzhou, Shenzhen, Tokyo, and Seoul.

3. Results

3.1. Spatiotemporal Evolution of Carbon Emissions in Shanghai

From 2000 to 2023 (Figure 4a), Shanghai’s total carbon emissions exhibited an overall pattern of “continuous growth, phased fluctuations, and stabilisation with slight decline.” Total emissions increased from 128.7857 million tons in 2000 to 310.4005 million tons in 2023, representing an approximately 1.4-fold increase, with an average annual growth rate of 3.8%. In general, the period from 2000 to 2010 corresponds to a phase of rapid increase, during which emissions rose from 129 million tons to 270 million tons. Industrialisation, a coal-dominated energy structure, and the rapid growth of vehicle ownership primarily drove this surge. The period from 2010 to 2016 is characterised by high-level fluctuations, with emissions oscillating slightly within the range of 270–290 million tons. This is likely related to the initial effects of industrial restructuring and energy efficiency improvements, as Shanghai’s Eleventh Five-Year Plan highlighted the city’s significant potential for energy conservation through optimization of its industrial structure [30]. After 2017, emissions entered a relatively stable phase, remaining between 290 and 310 million tons, indicating a gradual transition toward a plateau near peak levels.

Regarding carbon sinks, Shanghai’s annual carbon sink capacity increased from 0.3038 million tons in 2000 to 3.9895 million tons in 2023, representing an approximately 12-fold increase. However, its proportion relative to total emissions consistently remained below 2%, insufficient to produce a significant offset at the citywide scale. This suggests that, although progress has been made in urban green-space expansion and ecological restoration [31,32], the contribution of carbon sinks to overall mitigation remains limited.

Sectoral analysis (Figure 4b,c) reveals that the industrial sector has consistently been the dominant source of carbon emissions, accounting for between 45% and 55% of total emissions. Industrial emissions rose from 86.1520 million tons in 2000 to 145.8408 million tons in 2010. Although the absolute volume increased, the share of total emissions declined from 66.9% to 54.0%, reflecting a pattern of “high but decreasing proportion.” This indicates that, despite intensified industrial restructuring and the growing share of the tertiary sector, the sheer scale of industrial activity continues to make it central to efforts aimed at reducing emissions.

During the same period, emissions from transportation, storage, and postal services increased from 14.6901 million tons to 59.0347 million tons, an approximately threefold rise, demonstrating a sustained upward trend in transport-related energy consumption. Emissions associated with the construction sector and residential consumption also increased, suggesting that urban lifestyles, building energy use, and mobility patterns have become emerging drivers of emission growth. In contrast, emissions from agriculture, animal husbandry, and fisheries are relatively small and declining; agricultural emissions decreased from 6.0300 million tons in 2000 to 1.3284 million tons in 2023, indicating the progressive marginalisation of primary-sector emissions during the urbanisation process. Overall, Shanghai’s sectoral emission structure has undergone a transition from an “industry-dominated” pattern to a more diversified “industry–transport–residential composite” structure.

The characteristics of per capita carbon emissions (Figure 4a) indicate a sustained upward trend in Shanghai from 2000 to 2023, with a rise from 8.01 to 12.48 tons per person, representing a 55.8% increase. The period from 2000 to 2010 exhibited relatively rapid growth, with an average annual increase of approximately 3.8%, reflecting simultaneous improvements in per capita energy consumption and living standards. After 2010, growth slowed and stabilised at approximately 11–12 tonnes per person. Notably, Shanghai’s per capita emissions have been significantly higher than the national average (approximately 9.09 t/person) since 2007 [33], yet still retain potential for further reduction compared with other major international cities (such as New York in the United States). The stage-wise stabilisation of per capita emissions at high levels indicates the growing contribution of urban lifestyle, mobility structure, and building energy consumption to total emissions, while also suggesting that rising consumption-related emissions have partially offset the achievements of industrial emission reduction and technological progress.

Carbon intensity (carbon emissions per unit GDP) serves as a key indicator for assessing the decoupling of economic development from carbon emissions. From 2000 to 2023 (Figure 4a), Shanghai’s carbon intensity decreased from 2.68 to 0.66 t/10,000 RMB, representing a 75.3% reduction, indicating a pronounced downward trend. During the Eleventh and Twelfth Five-Year Plan periods, carbon intensity declined by 30.5% and 23.0%, respectively, highlighting the significant contributions of energy efficiency improvements and low-carbon industrial transition. Overall, the evolution of carbon intensity can be divided into three stages: (1) 2000–2008: a period of high-intensity but gradual decline, with carbon intensity falling from 2.68 to 1.69 t/10,000 RMB, primarily driven by efficiency improvements within the secondary sector; (2) 2009–2016: a phase of rapid decline, from 1.56 to 0.98 t/10,000 RMB, indicating marked effects of economic restructuring and the rising share of the service sector; (3) 2017–2023: a phase of stable low intensity, fluctuating between 0.65 and 0.88 t/10,000 RMB, suggesting that Shanghai has largely achieved relative decoupling between economic growth and carbon emissions.

Despite the substantial decline in carbon intensity, total emissions have remained high, revealing a dual pattern of “declining intensity yet persistently high emissions.” This indicates that, while Shanghai has achieved notable improvements in technological energy efficiency, indirect emissions associated with consumption and transportation continue to pose significant challenges.

In summary, from 2000 to 2023, Shanghai’s carbon-emission trajectory is characterised by “stabilising total emissions, diversified structural transformation, and markedly declining intensity.” The industrial sector has achieved significant emission reductions, yet the rapid growth of transportation and residential consumption has emerged as new drivers of emission increases. The continued decline in carbon intensity reflects progressive decoupling between economic development and emissions, whereas the persistently high per capita emissions underscore the need to strengthen green lifestyles and energy-management mechanisms on the consumption side. Overall, Shanghai has entered a pre–peak transition stage, characterised by improvements in energy efficiency, structural optimisation, and lifestyle transformation.

3.2. Driving Mechanisms of Carbon Emissions

3.2.1. Model Selection

This paper aims to develop an urban carbon-emission driving mechanism and prediction model under conditions of limited data availability. Due to the relatively small dataset, a time-series–oriented training/validation partition (rolling validation) is employed, and lagged features are incorporated. A rigorous comparison is conducted across four models, LightGBM, LightGBM-TPE, XGBoost, and XGBoost-TPE, to determine the optimal estimator. Ensemble-learning approaches are adopted to simulate the nonlinear relationships between 77 features and Shanghai’s urban carbon footprint. Recent studies have demonstrated the advantages of ensemble methods for carbon-emissions prediction [34,35].

To address the time-series characteristics and multidimensional driving mechanisms of urban carbon emissions, this study constructs a predictive modelling framework based on Gradient Boosting Decision Trees (GBDTs), selecting LightGBM and XGBoost as the core algorithms to systematically compare performance differences and parameter optimisation effects in sequence forecasting. Compared to traditional linear regression or neural network models, GBDT-based methods offer a strong nonlinear representation capacity, enhanced interpretability, and robustness, making them well-suited for modelling multidimensional heterogeneous variables and nonstationary temporal data.

XGBoost (Extreme Gradient Boosting) demonstrates superior overall performance in this study. Its core advantages include the following: (1) the use of second-order gradient information for loss optimization, enabling more stable convergence and higher predictive accuracy; (2) built-in regularization terms (L1 and L2) that effectively suppress overfitting and enhance cross-period generalization; (3) support for temporally ordered incremental training, which facilitates dynamic model updating under the rolling-validation framework; and (4) improved robustness to feature fluctuations through combined column and row subsampling, making it particularly suitable for forecasting annual carbon-emission series. Considering the significant temporal dependence and trend characteristics of carbon-emission data, this study does not employ randomly shuffled cross-validation; instead, it uses Rolling Time-series Validation, which predicts subsequent years using data from earlier years. This approach follows the authentic predictive logic of “using the past to forecast the future” and provides a more reliable assessment of temporal stability and generalisation capacity.

Figure 5 displays the prediction results under rolling time-series validation, illustrating the relationship between observed and predicted carbon-emission values for the four models (LightGBM, LightGBM-TPE, XGBoost, XGBoost-TPE). The horizontal axis represents observed emissions, while the vertical axis denotes predicted emissions. The dashed line indicates the ideal fit line (y = x), and the red line shows the linear regression fit, reflecting the direction and magnitude of prediction deviations.

The rolling time-series validation results indicate that XGBoost achieves the best performance among all models, with the baseline version attaining an R² of 0.66 and an RMSE of 989, demonstrating the highest predictive accuracy and trend consistency under the rolling validation framework. In contrast, the LightGBM and LightGBM-TPE models exhibit systematic underestimation in certain years, with larger overall fitting deviations, resulting in R² values of −3 and 0.29, respectively. Although hyperparameter tuning slightly improves the accuracy of LightGBM-TPE, it remains insufficient to capture the year-to-year variations in carbon emissions effectively.

It is noteworthy that the XGBoost-TPE model displays significant overfitting under the current parameter configuration. Its predicted values substantially exceed the observed values in high-emission years, resulting in R² = −6.31 and an RMSE of 4584. This indicates that excessively high learning rates and deeper tree structures impair the model’s cross-period generalisation ability within the rolling time-series framework. Overall, the baseline XGBoost model offers the best balance between fitting accuracy and generalisation stability, accurately capturing the annual fluctuation patterns of carbon emissions from 2015 to 2023.

It is worth noting that, under the current hyperparameter configuration, the XGBoost-TPE model exhibits pronounced overfitting. In years with high emission levels, its predicted values are substantially higher than the observed values, resulting in a negative coefficient of determination (R² = −6.31) and a large root mean square error (RMSE = 4584). This outcome indicates that, in time-series settings characterised by limited sample size and marked interannual variability, highly complex XGBoost-TPE configurations tend to overfit the training period and struggle to maintain generalisation performance in rolling forecasts for future years.

Accordingly, the TPE-optimised variant is not adopted as the final forecasting model in this study. Instead, the baseline XGBoost model, which demonstrates greater structural stability and stronger cross-period generalisation ability, is selected. Overall, the baseline XGBoost model achieves the most favourable balance between fitting accuracy and generalisation robustness, and can track the year-to-year fluctuations in carbon emissions over the period 2015–2023 with relatively high accuracy.

To further verify the model’s stability and prediction error characteristics, this paper conducts a statistical analysis of the XGBoost model’s prediction residuals using the Rolling Time-series Validation framework, as shown in Figure 6. Figure 6a illustrates the relationship between prediction residuals and predicted values. Most residuals fall within the ±1000 range and exhibit an essentially random distribution, without any systematic deviation associated with increasing predicted values. This indicates that the model’s prediction bias remains stable across different carbon-emission levels. Figure 6b presents the histogram of residuals, showing a mean residual of 411.3 and a standard deviation of 929.0. The distribution is approximately symmetric, suggesting the absence of significant systematic overestimation or underestimation; only slight positive skewness appears in a few years, corresponding to abnormal fluctuations during rapid emission-growth phases. Figure 6c depicts the relationship between relative error and actual values, with the vast majority of samples concentrated within ±5%, demonstrating strong annual-scale predictive stability and robustness.

Overall, the XGBoost model achieves an R² of 0.66 and an RMSE of 989.05 under rolling validation. The residual mean is close to zero, and the distribution is symmetric, indicating that the model effectively captures temporal variation patterns and nonlinear relationships in carbon-emission dynamics. Prediction errors primarily stem from year-to-year fluctuations rather than structural bias, reflecting strong cross-period generalisation ability and robustness. These results verify the reliability of the rolling-validation framework for time-series carbon-emission prediction and further confirm the stability and accuracy advantages of the XGBoost model when handling high-dimensional, multivariate, and nonlinear driving-factor data.

3.2.2. SHAP-Based Influencing Factors

To characterise the marginal contributions of different driving factors to Shanghai’s annual carbon emissions, this study applies SHAP (SHapley Additive exPlanations) decomposition to the year-by-year out-of-sample (OOS) predictions of the XGBoost model within a strict rolling time-validation framework. Rolling OOS SHAP avoids information leakage caused by random data splitting and, therefore more accurately reflects marginal effects under a “using the past to explain the future” setting. At the same time, SHAP values represent local attribution rather than causal effects. In particular, under conditions of strong multicollinearity, the model may allocate similar contributions across correlated features, leading to substitution effects among SHAP values. In other words, when two or more variables exhibit highly synchronous variation in a statistical sense, SHAP may be unable to precisely disentangle their independent marginal effects and instead tends to reflect their joint contribution. Consequently, SHAP values capture the internal attribution structure of the predictive model rather than structural causal effects in a strict sense.

Urban carbon emissions inherently exhibit substantial structural overlap. For example, population scale, investment activities, and infrastructure expansion often evolve simultaneously with economic growth and spatial development. As a result, interactions and overlapping contributions among variables are unavoidable in model interpretation, and their direction and magnitude are influenced by sample distribution and multicollinearity. For scale-related variables such as population, investment, and infrastructure, relatively large mean absolute SHAP values may therefore partly reflect their extensive indirect association networks rather than purely independent effects.

Figure 7 illustrates how SHAP values quantify the incremental contribution of a single variable relative to the baseline prediction, conditional on the remaining features. The sign of the value indicates its enhancing or suppressing direction, and the absolute magnitude represents its influence strength. For each year, the absolute SHAP values (mean|SHAP|) of OOS predictions are averaged and used as the cross-period robust importance measure, from which the top 20 dominant factors are identified. In Figure 7, red points represent higher feature values, while blue points represent lower values; the horizontal axis denotes SHAP values (i.e., the direction and magnitude of the feature’s impact on carbon emissions). Points located further to the right indicate that the feature increases the predicted value, whereas points further to the left indicate a suppressing effect; greater dispersion implies higher variability in the feature’s influence.

Overall, the factors affecting Shanghai’s carbon emissions can be grouped into four major categories: population scale, industrial energy efficiency, investment structure, and infrastructure. (1) Population scale (X3 total population, X4 permanent population, X5 urban permanent population) shows the most prominent weights and dominates all other factors; (2) Industrial structure and energy efficiency (X18 secondary-industry employment, X16 share of secondary industry, X76_lag2 lagged electricity consumption per unit GDP) exhibit suppressing or moderating effects at different stages; (3) Investment and price/demand variables (X28 industrial investment, X30 share of industrial investment, X26 retail price index, X27 consumer price index, X28_lag2 lagged industrial investment) display positive stimulation effects in current or lagged years; (4) Infrastructure/energy structure (X50 urban road area, X57 water-supply production capacity, X74 coal share) generally contribute positively, while public infrastructure factors (X8 number of physicians, X12 number of schools, X34 investment in public administration and social security, X56 per capita urban green space) have relatively minor effects on carbon emissions.

(1)

Scale effect: the dominant role of population-related variables.

X3 total population (mean|SHAP| ≈ 3956) ranks first, far exceeding all other factors, indicating that population stock changes exert the most substantial marginal pulling effect on annual emissions; X4 permanent population and X5 urban permanent population reinforce this conclusion. The rolling OOS SHAP beeswarm decomposition (Figure 8) shows that high-value population samples (red points) cluster predominantly in the positive region, with SHAP values mostly above zero, suggesting that population expansion significantly elevates annual emissions through channels such as household energy consumption, travel demand, and infrastructure construction. This suggests that a larger population scale has a substantial positive impact on predicted carbon emissions, i.e., “greater population leads to higher emissions.” Conversely, low-population samples (blue points) lie in the region of near-zero values, reflecting lower and less volatile emission levels in years with smaller population sizes.

(2)

Industrial energy efficiency: the sustained upward effect of industrial investment share.

X28 industrial investment and X30 share of industrial investment in total fixed-asset investment both rank within the top fifteen. Their high-value samples (red points) are clearly positioned on the right side of the SHAP axis, indicating that an increasing proportion of industrial investment raises current-year carbon emissions; conversely, low-value samples (blue points) cluster on the left side, suggesting that a slowdown in industrial investment corresponds to a decline in emissions. The out-of-sample SHAP values of X20 tertiary-industry employment share are predominantly negative for high-value samples, indicating that a higher share of the service sector suppresses current-year emissions. X28_lag2 (industrial investment lagged by two periods) continues to exert a positive contribution, suggesting that the production capacity and capital stock generated by investment not only affect emissions in the current year but also extend their energy-consumption effects into subsequent years.

(3)

Investment structure: supplementary effects of price and demand-side indicators.

X18 secondary-industry employment and X16 share of secondary industry both enter the top twenty. Figure 8 shows that their SHAP values span both sides of zero, indicating heterogeneous effects on carbon emissions: when the secondary-industry scale expands without sufficient energy-efficiency improvement, the effect is positive; however, as the industrial structure shifts toward technology-intensive and low-energy-consumption sectors, the effect weakens or becomes negative. X76_lag2 electricity consumption per unit GDP (lagged by two periods) is harmful primarily, indicating that years with higher energy efficiency (lower electricity intensity) yield delayed mitigation benefits in the subsequent two years. X26 retail price index and X27 consumer price index are included in the top twenty but exhibit relatively weak influence. Their SHAP scatter points fluctuate slightly around zero in both directions, indicating that price changes primarily serve as auxiliary signals of demand cycles and investment rhythms, indirectly affecting emissions through these channels. This suggests that the impact of price variation on carbon emissions is bidirectional but limited in magnitude, with effects transmitted mainly through indirect consumption and investment pathways.

(4)

Infrastructure: indirect “energy spillover” under rigid service provision.

X50 urban road area and X57 water-supply production capacity mostly show positive effects on Shanghai’s carbon emissions, indicating that expansions in municipal and water-utility infrastructure lead to increased operational energy use and electricity consumption, thereby generating positive marginal contributions to emissions. This likely reflects the additional electricity and chemical reagent requirements of water supply systems. These results suggest that expansions in public infrastructure tend to exert upward pressure on carbon emissions. The energy-structure variable X74 (coal consumption share) also displays consistently positive SHAP values across the plot, highlighting the long-term emission-increasing effect of fossil-energy dependence. Although X66 (the number of gas users) also tends to show positive values, the SHAP magnitudes are small, indicating a limited marginal impact. Within public-infrastructure indicators, X8 number of physicians, X12 number of schools, and X34 investment in public administration and social security show generally small SHAP values, with slight rightward skew in some years, suggesting that these variables function more as proxies for urban-development stages and public-service expansion, contributing mild, pro-cyclical positive marginal effects. This indicates that education and social-service investments reflect broader development stages rather than direct emission drivers, though they may produce pro-cyclical increases during expansion periods. The X56 per capita urban green-space area shows a weak average influence, with scatter points near zero or slightly negative, suggesting that green-space expansion may exert limited mitigation effects through pathways such as microclimate improvement and optimised travel patterns.

3.2.3. Nonlinear Threshold Effects of Individual Factors

To further clarify how different factors influence Shanghai’s carbon emissions, SHAP dependence plots are generated (Figure 8). These plots intuitively illustrate the specific contribution of each variable to Shanghai’s carbon emissions across its full value range. The x-axis represents the feature value, while the right y-axis shows the SHAP value, which quantifies the contribution of the corresponding feature to carbon emissions in each sample. The LOWESS (locally weighted scatterplot smoothing) trend line displays the average change in SHAP values, and the surrounding light-grey shaded area indicates the 95% confidence interval (CI). The left y-axis corresponds to a vertical bar chart (distribution), showing the sample distribution of each variable in the dataset. Vertical dashed lines denote critical thresholds at which the direction of influence shifts.

A multidimensional analysis of these influencing factors reveals the following insights:

(1)

Population Dimension

Figure 8a–c,e,h,l shows that population-related variables exhibit pronounced two-stage effects on carbon emissions. Overall, population expansion follows a three-phase pathway—scale effect, intensification effect, and congestion rebound—reflecting the nonlinear responses of urban systems across different development stages.

First, the SHAP curves for total population (X3) and resident population (X4) both display an “increase–decrease” pattern. The total population (X3) reaches its peak SHAP value at approximately 14.6 million, indicating the strongest positive contribution to carbon emissions. As population size further approaches the model-identified threshold (around 15.16 million), SHAP values shift from increasing to gradually declining. This suggests that population growth initially drives emissions through scale effects whereas, at higher density levels, infrastructure sharing and improvements in energy-use efficiency begin to generate intensification effects that slow emission growth. Similarly, the resident population (X4) exhibits a comparable turning point at around 24.5 million, beyond which the SHAP curve flattens and then declines, indicating notable marginal efficiency gains in urban operations at high population levels.

This transition reflects a structural shift from intensification effects to congestion effects and is consistent with the policy orientation of Shanghai’s 14th Five-Year Plan, which emphasises reducing population density in central urban areas, improving urban functions and the living environment, optimising spatial structure, and promoting population redistribution. These measures aim to guide central districts toward high-quality development while enhancing the carrying capacity of suburban areas. When SHAP values turn from negative to positive, congestion effects emerge, indicating that, beyond a certain density threshold, per capita carbon emissions begin to rise. This pattern aligns with Shanghai’s recent policies on relocating non-essential functions from the city centre and promoting a multi-centre urban structure. In high-density areas lacking sufficient ecological space and public infrastructure, building energy consumption, traffic pressure, and overall energy demand tend to increase.

Second, registered population density (X7) exhibits a clear positive turning point at approximately 3600 persons per km², where SHAP values shift from a declining to an increasing trend. This indicates that, once urban density exceeds this threshold, per capita carbon emissions begin to rise rather than fall. This structural transition from intensification effects to congestion effects is highly consistent with the objectives outlined in the Shanghai Master Plan (2017–2035) [36], which calls for strict control of population size in megacities, continuous optimisation of population structure and spatial distribution, adjustment of population density and per capita construction land, and improvements in urban liveability. The plan also advocates policies on land use, employment, and housing supply to alleviate excessive population concentration in central areas while increasing population and employment density and spatial efficiency in new towns and suburban districts.

Within the moderate-density range (approximately 2000–3600 persons per km²), compact urban form and efficient public transport systems can reduce per capita energy consumption, reflecting the energy-efficiency dividend of compact cities. However, when density exceeds this optimal range, the combined effects of concentrated building energy demand, deteriorating ventilation and green-space conditions, and traffic congestion lead to rising energy use and carbon emissions. Therefore, urban density enhancement should operate within an optimal interval, beyond which measures such as spatial decentralisation, building energy-efficiency upgrades, and the integration of ecological spaces are required to avoid a reversal toward “high-density, high-emission” outcomes.

Figure 8e,h further shows that the urban resident population (X5) and urbanisation level (X6) exhibit interlinked two-stage effects on carbon emissions. At the scale level, the SHAP curve for X5 shows an overall downward trend. When the urban population is below approximately 22.26 million, SHAP values remain strongly positive, indicating that population concentration and urban expansion significantly promote carbon emissions in the early stage. Once population size approaches this threshold, SHAP values decline sharply and stabilise, suggesting that improvements in infrastructure, energy efficiency, and shared public services generate a clear intensification-driven mitigation effect. This demonstrates that urban population growth does not drive emissions linearly but instead exhibits diminishing marginal effects and efficiency turning points beyond a certain scale.

At the structural level, the urbanisation rate (X6) shows a negative turning point beyond 88.2%, where SHAP values gradually shift from a declining to an increasing trend. This indicates that, once urbanisation reaches an extremely high level, its marginal effect on carbon emissions changes from suppressive to mildly promotive. This pattern reflects a carbon rebound effect at advanced stages of urbanisation. During the intermediate-to-high urbanisation phase (approximately 75–88%), energy structure optimisation, improved public transport systems, and industrial service-oriented transformation contribute to reductions in carbon intensity. However, when urbanisation enters a saturated range (>88%), high-energy-consumption lifestyles, urban spatial expansion, and energy-intensive service-sector agglomeration lead to a slight rebound in total emissions.

Accordingly, 88.2% can be regarded as a critical threshold marking the transition from “structural dividends” to “energy rebound” in the urbanisation process. At this stage, mitigation strategies should shift from infrastructure expansion toward greener lifestyles and consumption patterns, including the promotion of green mobility, energy-efficient buildings, and shared public services, to offset the energy-demand rebound associated with ultra-high urbanisation levels.

(2)

Industrial Dimension

Figure 8d,q illustrates the effects of industry-related variables on carbon emissions. First, the SHAP curve for employment in the secondary sector (X18) exhibits a clear turning point at approximately 4.4 million workers. Below this threshold, SHAP values remain consistently negative, indicating that, at low-to-moderate levels of industrial employment, industrial expansion is accompanied by technological upgrading and improvements in energy-use efficiency, thereby exerting a suppressive effect on carbon emissions. However, once industrial employment exceeds this critical level, SHAP values rapidly turn positive and increase, suggesting that output expansion and spillover energy demand associated with additional employment become the dominant forces, resulting in a pronounced positive contribution to carbon emissions. This transition from negative to positive SHAP values reveals a nonlinear threshold effect of industrial employment scale: when employment remains below the threshold, efficiency gains derived from technological upgrading and process optimisation can be further consolidated; when employment approaches or exceeds the threshold, it becomes necessary to simultaneously implement energy-consumption caps, promote clean-energy substitution, and encourage shifts toward higher value-added activities to prevent emission rebound driven by further scale expansion.

The mitigation effect observed at lower employment levels reflects the benefits of industrial renewal, technological transformation, and capacity optimisation, whereas the positive contribution beyond the threshold indicates that continued expansion of industrial employment leads to energy-demand spillovers. This pattern is highly consistent with the industrial upgrading direction emphasised in the Shanghai Master Plan (2017–2035), which highlights accelerating the transformation and upgrading of manufacturing, promoting high-end and service-oriented manufacturing, gradually phasing out labour-intensive low-end manufacturing, and increasing employment in high-technology sectors.

The SHAP curve for the share of the secondary industry (X16) displays a critical threshold at approximately 29.8%. Below this level, carbon emissions increase with a rising industrial share, representing a typical “industrialisation-driven” phase. Once the share exceeds 29.8%, SHAP values shift from positive to negative and gradually stabilise, indicating that structural optimisation and energy-efficiency improvements increasingly dominate changes in carbon emissions. This result suggests that, when the secondary-industry share surpasses roughly 30%, the regional economy begins to transition from energy-intensive manufacturing toward higher value-added industries and services, forming a progressive mitigation pathway characterised by scale, efficiency, and structural upgrading.

This trend is consistent with Shanghai’s transition toward a post-industrial, service-oriented development stage. The Outline of the 14th Five-Year Plan for Economic and Social Development and the Long-Range Objectives through 2035 emphasise the building of Shanghai into a service-economy hub, promoting the integration of service outsourcing with advanced manufacturing and producer services, and accelerating the formation of high-end industrial clusters driven by strategic emerging industries and digital transformation of traditional sectors. The data-driven results of the model thus provide empirical evidence of the endogenous link between industrial-structure optimisation and carbon-emission dynamics.

(3)

Economic Dimension

Figure 8g,i,j,m,o depicts the dependence of carbon emissions on economic variables, revealing an overall nonlinear pattern characterised by “increase–turning point–stabilisation.” This structure reflects the dynamic transition of economic development from expansion-driven growth toward structural optimisation.

Both industrial investment (X28) and its two-period lag (X28_lag2) exert significant influences on Shanghai’s carbon emissions, highlighting the asymmetry between the immediate and lagged effects of investment. The SHAP curve for contemporaneous industrial investment (X28) reaches a threshold at approximately RMB 126.8 billion. Below this level, SHAP values increase rapidly, indicating a strong short-term emission-raising effect associated with investment expansion. Once investment exceeds the threshold (around RMB 126.84 billion), SHAP values level off or slightly decline, suggesting that diminishing marginal returns to capital and technological upgrading gradually emerge, thereby weakening the marginal contribution of investment to carbon emissions. This pattern is consistent with Shanghai’s policy orientation during the 14th Five-Year Plan period, which emphasises expanding investment while optimising its structure.

By contrast, the second-order lagged investment variable (X28_lag2) exhibits a clear sign reversal at approximately RMB 140 billion, where SHAP values decline sharply from positive to negative. This indicates that, after a two-year accumulation period, earlier investments begin to generate structural mitigation effects through improved capacity utilisation and equipment upgrading. This “positive construction-period effect with negative lagged feedback” reveals the dual role of industrial investment: while it raises carbon emissions in the short term, it can contribute to reductions in carbon intensity over the longer term through technology diffusion and the formation of greener productive capacity. This mechanism provides empirical support for policy measures advocated in the Outline of the 14th Five-Year Plan for Economic and Social Development and the Long-Range Objectives through 2035, which emphasise promoting green investment, developing green finance, and leveraging the demonstration role of national green development funds.

These findings underscore the critical importance of the temporal structure of investment for carbon-emission dynamics. When capital is primarily directed toward energy-intensive manufacturing, lagged effects continue to manifest as persistently high emissions. In contrast, when investment is oriented toward energy-efficiency retrofits and low-carbon infrastructure, the lagged SHAP values become negative, indicating that “green investment payback” generates a pronounced emission-mitigation effect over time.

Finally, the SHAP curve for the share of industrial investment in fixed-asset investment (X30) reveals a threshold at approximately 15.6%. Below this level, SHAP values are significantly positive, suggesting that, when industrial investment accounts for a relatively small share of total investment, incremental capital flows mainly into industrial production, thereby driving emission growth. Once the share exceeds this threshold, SHAP values decline rapidly and approach zero, indicating that the fixed-asset investment structure becomes more diversified, with an increasing proportion allocated to the tertiary sector and service industries. As a result, the carbon-intensity effect dominated by industrial expansion is gradually weakened. This pattern highlights the critical role of the “deindustrialisation–servitisation” transition in achieving regional carbon-emission mitigation.

(4)

Public Service Dimension

Figure 8i–k,p,r indicates that public-service-related variables exert significant threshold and stage-dependent effects on carbon emissions, reflecting nonlinear feedback mechanisms among consumption behaviour, fiscal investment, and the public service system.

Both the retail price index (X26) and the consumer price index (X27) exhibit pronounced two-stage effects (see Figure 8i,m). Taking X26 as an example, when the price index remains below approximately 100.8, SHAP values are negative, suggesting that, at low price levels, suppressed market demand and insufficient production activity lead to relatively low carbon emissions. As the index approaches the threshold, SHAP values rapidly turn positive and increase, indicating that the expansion of consumption and production drives higher emissions through increased energy use in transportation, manufacturing, and related sectors. Similarly, the SHAP curve for the consumer price index (X27) turns positive at around 102, following a pattern broadly consistent with X26. This further confirms that, when price levels are low, consumption and production activities have not fully recovered, and carbon emissions remain weakly responsive; once prices rise into the critical range, consumption-driven expansion significantly amplifies carbon emissions.

Investment in public administration, social security, and social organisations (X34) displays a typical “low-level stability–mid-level increase–high-level stabilisation” pattern in its SHAP curve (Figure 8). When investment remains below approximately RMB 4.9 × 10⁶, SHAP values are close to zero, indicating that limited public-sector investment has a negligible impact on carbon emissions. Once investment exceeds this threshold, SHAP values rise sharply, reflecting short-term increases in energy consumption and emissions associated with public infrastructure construction and the expansion of social security systems. However, when investment further increases beyond approximately RMB 0.95 × 10⁷, the SHAP curve levels off, suggesting that high levels of public investment enter a “stable-effect” phase in which additional capital generates diminishing marginal emission impacts. This pattern reveals the dual role of public investment: while it creates emission spillovers through construction activities in the short term, it can indirectly promote carbon mitigation in the longer term by improving service efficiency and facilitating the development of green infrastructure.

Public service provision variables further exhibit a similar “increase–turning point–stabilisation” pattern. The number of physicians (X8) shows a negative-to-positive turning point at approximately 157,000. At low levels (<150,000), SHAP values are relatively high, reflecting the high energy intensity associated with concentrated operation of large hospitals under conditions of medical resource scarcity. As the number of physicians approaches the threshold, improved resource distribution and service balance enhance operational efficiency, reducing SHAP values to their minimum and indicating an energy-efficiency threshold in healthcare system expansion. When the number of physicians continues to increase beyond high levels (>180,000), SHAP values rise slightly again, suggesting that excessive concentration and facility redundancy may introduce new operational energy burdens. This pattern is highly consistent with the policy objectives outlined in the Outline of the 14th Five-Year Plan for Economic and Social Development and the Long-Range Objectives through 2035, which emphasises strengthening emergency public health capacity and promoting the expansion and balanced allocation of high-quality medical resources, while also revealing—at the model level—the potential risk of energy rebound associated with overconcentration.

Similarly, the number of schools (X12) exhibits a threshold at approximately 1678 institutions, beyond which SHAP values gradually increase. When the number of schools exceeds roughly 1725, SHAP values turn negative and continue to decline, indicating that the expansion and spatial balancing of educational resources significantly reduce carbon emissions at the urban scale and generate a clear structural mitigation effect. Overall, the expansion of educational and medical resources follows a dynamic pattern of “initial promotion–mid-term suppression–long-term stabilisation,” suggesting the existence of an optimal range for public service density. Moderate expansion enhances urban energy efficiency and residents’ quality of life, whereas excessive construction may lead to energy-consumption rebound effects.

(5)

Environmental Dimension

Figure 8f,i,t indicates that improvements in environmental infrastructure generate significant marginal mitigation effects once development reaches medium-to-high levels. First, the SHAP curve for urban road area (X50) exhibits a positive threshold at approximately 1.5 × 10⁴ m², where SHAP values increase rapidly. This suggests that, during the early stage of transport infrastructure development, road expansion is accompanied by substantial material inputs and construction-related energy consumption, thereby exerting a pronounced positive effect on carbon emissions. However, as the road area further expands to around 2.4 × 10⁴ m², SHAP values reach a peak and then begin to decline. This pattern indicates that, as the road network becomes more complete, the combined effects of congestion alleviation, improved commuting efficiency, and optimisation of travel structure gradually slow the growth rate of transport-related emissions. This turning point reflects a typical transition from “high emissions during the construction phase” to “stabilised emissions during the operational phase.” The observed pattern is highly consistent with the planning objectives of the Shanghai Master Plan (2017–2035), which emphasises the development of a more sustainable and resilient eco-city and the construction of a green and low-carbon infrastructure system.

The SHAP curve for water supply production capacity (X57) shows a clear positive activation threshold at approximately 10.88 million tonnes per day (Figure 8i). Below this level, SHAP values remain close to zero, indicating that, at smaller system scales, water supply infrastructure has a limited impact on carbon emissions. Once production capacity exceeds this threshold, SHAP values turn positive and continue to rise, suggesting that the expansion of basic public service infrastructure enters a high-energy-consumption phase. In this stage, processes such as water treatment plant operation, pumping energy demand, and pressure maintenance in transmission and distribution networks substantially increase energy use, thereby intensifying carbon emissions. This trend reveals a typical “scale expansion–energy consumption escalation” mechanism: while expanding water supply facilities beyond the threshold improves public service provision, it is also associated with a relatively high operational carbon cost.

Finally, per capita park green space (X56) exhibits persistently negative SHAP values once it exceeds approximately 8.45 m² per person, indicating a significant reduction in carbon emissions as green space availability increases. This pattern reflects the positive role of urban ecological space in carbon sequestration and thermal environment regulation. Specifically, the expansion of green space and increased vegetation cover enhance urban carbon sinks and reduce energy demand, thereby generating a pronounced “ecological suppression effect” on carbon emissions.

3.2.4. Multi-Factor Interaction Analysis

This study further evaluates the interactions among multiple influencing factors to uncover the complex dynamic effects of different variable combinations on Shanghai’s carbon emissions. Figure 9 presents the interaction strength of various feature combinations. Given the large number of possible interactions, only the top 20 are displayed, and the six strongest pairs are analysed in detail. The results show that combinations involving total population, permanent population, household-registered population density, secondary-industry employment, urban permanent population, and urbanisation level exhibit the most substantial interactive effects on Shanghai’s carbon emissions.

For each annual model generated through rolling training, the pairwise SHAP interaction values (interactions = True) are computed using that year’s OOS samples. The absolute values of these interactions are treated as interaction strength, and their averages across all OOS years are calculated to produce an “interaction strength matrix.” Interaction strength can be interpreted as the magnitude of the joint effect of two features on carbon-emission prediction, beyond their individual contributions. Larger values indicate more substantial interaction effects. Figure 9 displays the top 20 feature pairs with the highest averaged interaction strength, enabling the identification of “significant co-movement” pathways that affect Shanghai’s carbon emissions.

To further verify the degree of synergistic influence among different factor combinations under higher-order nonlinear conditions, this study adopts Friedman’s H-statistic to quantify the contribution of interaction terms [37]. A comparative modelling system is constructed using a shallow model (Baseline, max_depth = 4) and a deep model (Deep, max_depth = 8). A larger H value indicates a more substantial interaction effect between two variables.

Specifically, based on both the shallow (baseline, max_depth = 4) and deep (max_depth = 8) XGBoost models, the partial dependence surface $g\left(x_i,x_j\right)$ for each pair of features, as well as the single-variable partial dependence curves $f_i\left(x_i\right)$, and $f_{j\left(x_j\right)}$, are computed.

Friedman’s H is defined as follows:

$$H=\sqrt{{\displaystyle \frac{Var\left(g\left(x_i,x_j\right)-f_i\left(x_i\right)-f_{j\left(x_j\right)}\right)}{Var\left(g\left(x_i,x_j\right)\right)}}}$$

(11)

$$g(x_i,x_j)=E_{X\left\{i,j\right\}}\left[f\right(X\left)\right|x_i,x_j]$$

(12)

$$f_i\left(x_i\right)=E_{X\backslash \left\{i\right\}}\left[f\right(X\left)\right|x_i]$$

(13)

$$f_j\left(x_j\right)=E_{X\backslash \left\{j\right\}}\left[f\right(X\left)\right|x_j]$$

(14)

where $g\left(x_i,x_j\right)$ denotes the two-dimensional partial dependence surface (2D Partial Dependence Surface), representing the model’s average predicted response when features $x_i$ and $x_j$ vary simultaneously. $f_i\left(x_i\right)$ is the one-dimensional partial dependence function (1D Partial Dependence of $x_i$), describing the average effect on the prediction when only feature $x_i$), varies independently; $f_{j\left(x_j\right)}$ is the corresponding 1D partial dependence function of $x_j$, reflecting the independent contribution of feature $x_j$. A higher HHH value indicates stronger interaction effects between the two variables.

$$∆H=H_{deep}-H_{baseline}$$

(15)

When ΔH > 0, it suggests that, under higher-order nonlinear modelling, the interaction strength of the feature pair is amplified, exhibiting a super-additive “1 + 1 > 2” effect.

Figure 9 presents the top 20 feature pairs ranked by interaction amplification effects (ΔH). The bar length represents the absolute magnitude of ΔH, and the colour gradient, from red (strong amplification) to blue (weak amplification), indicates the relative strength of amplification. The results show that specific feature pairs (e.g., X3 × X30, X4 × X50) exhibit markedly strengthened interactions in the deep model, revealing strong synergistic behaviour under nonlinear conditions.

The pair X4 × X5 (permanent population × urban permanent population) exhibits the largest ΔH (0.0249), with an amplification factor of 2.36, indicating that the nonlinear coupling between population scale and urbanisation rate is significantly intensified in the higher-order model. In other words, the influence of population urbanisation on carbon-emission dynamics is reflected not only in linear growth effects, but also in compounded amplification through the joint processes of population agglomeration and urban expansion.

The interaction X3 × X57 (total population × water-supply production capacity) yields a ΔH value of 0.0162, indicating a pronounced nonlinearity between population size and infrastructure capacity. As the population increases, the energy consumption and carbon emissions associated with expanding water-supply systems do not grow linearly but follow an accelerated trajectory. The interaction X3 × X30 (total population × industrial investment share) has a ΔH value of 0.0131, indicating that population growth and industrial structure expansion exert a strong amplification effect. This suggests that, during rapid industrialisation, the combined intensification of population-driven demand and industrial energy consumption significantly elevates emissions. The interaction X3 × X28_lag2 (total population × lagged industrial investment) also displays strong amplification (ΔH = 0.0085), implying that investment behaviour has a sustained temporal influence on the nonlinear response of the carbon-emission system.

Other interactions, such as X3 × X26 (total population × consumer price index) and X4 × X18 (urban population × secondary industry employment), also exhibit positive amplification, indicating substantial multi-factor coupling among population urbanisation, consumption structure, and industrial labour structure within the carbon-emission formation mechanism. In contrast, some energy-related pairs, such as X3 × X74 (total population × coal-consumption share) and X3 × X50 (total population × urban road area), show ΔH < 0, meaning that their interactions weaken in the deep model and become more additive. This may reflect the declining marginal amplification effects of high-carbon energy consumption and the diminishing marginal influence of transportation infrastructure development.

Overall, approximately 70% of variable pairs exhibit positive amplification in the deep model, suggesting that multidimensional socio-economic factors display widespread synergistic amplification under nonlinear frameworks. In particular, the nonlinear coupling among population, urbanisation, industrial structure, and infrastructure systems constitutes a key mechanism that shapes the spatial differentiation and temporal evolution of carbon emissions.

Figure 9 provides a comprehensive illustration of the extent to which multiple factors interactively influence carbon-emission intensity. Taking X3 × X30 as an example, the horizontal axis represents the value range of the primary feature total population (X3). In contrast, the vertical axis represents the SHAP interaction effect between the total population and the share of industrial investment in total fixed-asset investment (X30), indicating the joint impact of these factors on carbon emissions. The colour of each scatter point corresponds to the magnitude of the industrial-investment share: red represents higher industrial-investment levels (X30 > 23.25), while blue represents lower levels (X30 ≤ 23.25).

In the low industrial-investment stage (blue regression line), the marginal effect of population growth on carbon emissions is weak and tends to stabilise. In contrast, under the high industrial-investment stage (red regression line), as the population increases from approximately 1.35 × 10⁷ to 1.42 × 10⁷, the SHAP interaction values rise rapidly. This indicates that the combination of population expansion and industrial-investment growth significantly intensifies carbon emissions.

Such a phenomenon reveals a marked amplification effect, indicating that, when a large population scale and high industrial capital input coexist, carbon emissions are significantly magnified. This reflects a synergistic driving mechanism between industrial-structure expansion and population growth (Figure 10).

(1)

Interaction between population and infrastructure (X4 × X5)

The Figure 10a, representing the interaction between permanent population and urban permanent population, shows the highest ΔH value (0.0249), indicating the most substantial structural amplification effect. As total population increases, a rising level of urbanization significantly enhances the marginal impact on carbon-emission intensity: in the early stage, where population size is relatively small, urban agglomeration effects are not yet fully formed, and the emission pressure induced by population growth remains modest; however, once the permanent population exceeds 21.47 million, the rising proportion of urban residents markedly amplifies energy demand and lifestyle-related emissions. This result demonstrates that the coupled variation of total population and urban population concentration is a major driver of accelerated emission growth, reflecting how population spatial agglomeration magnifies pressure on energy systems and transportation activities. This indicates a typical “population–urbanisation amplification effect,” where high population density, combined with rapid urbanisation, substantially increases energy consumption and lifestyle-related emissions.

(2)

Interaction between population and residential energy use (X3 × X57, X3 × X26)

The ΔH values of Figure 10b (total population × water-supply production capacity) and X3 × X26 (total population × retail price index) are 0.0162 and 0.00127, respectively, both demonstrating consumption-driven amplification effects.

For the interaction between the total population (X3) and water-supply production capacity (X57), the marginal effect of population growth on carbon emissions is significantly higher under low water-supply capacity (X57 ≤ 11.525 million tons/day), indicating that substantial energy inputs are associated with early-stage infrastructure expansion. In the high-capacity range (X57 > 11.525 million tons/day), as the system approaches saturation, the interaction effect stabilises or slightly declines. This indicates a nonlinear pattern in the impact of infrastructure development on emissions across stages, transitioning from a driving phase to a dampening phase.

For the interaction between total population (X3) and retail price index (X26) (Figure 10e), when price levels are low (X26 ≤ 101.10), the influence of population growth on emissions is limited. In contrast, during high-price periods (X26 > 101.10), the interaction SHAP values increase sharply as the population expands, indicating that higher consumption levels, combined with population growth, jointly drive increases in emissions. This “living-standard amplification effect” suggests that population growth indirectly strengthens emission intensity through consumption upgrading and increased energy demand.

(3)

Interaction between population and industrial development (X3 × X30, X3 × X28_lag2, X4 × X18)

This group of variables reflects the dynamics of emissions along the “population–industrialisation” pathway. The ΔH values of Figure 10c (total population × industrial investment share) and Figure 10d (total population × lagged industrial investment) are 0.0131 and 0.00854, respectively, both indicating a clear pattern of synergistic amplification. As the population expands and fixed-asset investment accelerates, the energy demand of the industrial sector intensifies further in the lagged stage, reinforcing emission growth. For Figure 10f (permanent population × secondary-industry employment), the ΔH value is 0.00107, suggesting that industrial employment has a strong positive amplification effect under a high-population context.

In the interaction between total population (X3) and lagged industrial investment (X28_lag2), population growth has relatively stable impacts on emissions in areas with low lagged investment (X28_lag2 ≤ 1179.34). In contrast, in areas with higher lagged investment (X28_lag2 > 1179.34), the interaction SHAP values rise sharply, indicating that historical investment activity continues to amplify current emissions. This reveals the time-lag effect of industrial capital accumulation and early investment expansion, which, when coupled with continued population growth, further strengthens carbon-emission levels.

In the interaction between the permanent population (X4) and secondary-industry employment (X18), under low industrial employment conditions (X18 ≤ 4.375 million), the effect of population growth on emissions is limited. However, when industrial employment exceeds this threshold (X18 > 4.3750 million), the interaction effect strengthens rapidly as the population expands. This indicates that the combination of industrial labour concentration and population agglomeration significantly amplifies carbon emissions, reflecting a deeper structural coupling between demographic expansion and industrial composition.

Overall, when the population exceeds approximately 14 million and urban employment is concentrated in industrial sectors, the emission response becomes markedly steeper, demonstrating the mutually reinforcing effects of population agglomeration and industrial capital accumulation. This confirms that population size acts as an “amplifier” of carbon emissions: when high population levels coincide with high industrial investment intensity, emissions exhibit an escalating response. In contrast, interaction terms involving efficiency and structural-optimisation variables (e.g., lagged electricity consumption intensity and secondary industry share) exhibit “offsetting amplification,” forming endogenous stabilising mechanisms. This suggests that a single factor does not drive urban carbon-emission dynamics, but instead emerges from the nonlinear synergies among population, investment, industrial structure, and energy efficiency.

Taken together, the amplification patterns revealed by Friedman’s ΔH demonstrate nonlinear interactions between “population factors” and the three categories of “infrastructure–consumption–industrial development.” Population size is not only a direct driver of carbon emissions but also an amplifier of infrastructure expansion and industrial growth. When urbanisation rate, water-supply capacity, and industrial investment rise concurrently with population growth, the system’s marginal amplification effect is substantially enhanced, suggesting that linear single-variable estimates may underestimate the actual compound impacts.

3.3. Future Carbon Emission Forecasting for Shanghai

To evaluate the effects of different combinations of driving factors on model performance and to systematically characterise the contribution of cumulative features to model accuracy and generalisation, this paper develops a Stepwise Feature Expansion Evaluation framework. Guided by SHAP-based feature importance and theoretical interpretability, a set of core driving factors is identified and incorporated into the XGBoost model in a predetermined sequence. At each step, only one additional predictor is introduced while all other model parameters remain unchanged, allowing an assessment of the marginal effects of feature accumulation on explanatory power and predictive accuracy.

During model training and validation, a rolling expanding window strategy is adopted to prevent information leakage and align with the logic of real-world time-series forecasting. Specifically, for any prediction year t, only historical observations up to year t − 1 are used as the training set, while data for year t serve as the independent test set. The training window is then expanded year by year, and this procedure is repeated until the whole study period (2002–2023) is covered. For each step, both the in-sample goodness of fit (R²_train) and rolling validation accuracy (R²_ROLLING) are computed. This approach maintains temporal independence and effectively reflects the model’s stability and generalisation capacity in cross-year forecasting.

As shown in Figure 11, the R²_ROLLING value exhibits an overall upward trajectory as features are incrementally added, indicating that newly introduced factors enhance the model’s temporal predictive performance. In the early stage (k ≤ 5), the predictive ability remains limited, with R²_ROLLING stabilising at around 0.54. As variables related to energy consumption and industrial structure, such as X28, X50, X6, and X26, are progressively incorporated, cross-period prediction accuracy improves substantially. When the feature count reaches ten to eleven variables, R²_ROLLING enters a relatively stable range, achieving its optimal value at k = 11 (R²_ROLLING = 0.6198). Beyond this point, the marginal gains from adding additional variables diminish noticeably, and some indicators even cause slight fluctuations, suggesting that, once the core feature set is established, the model can already capture the primary mechanisms driving carbon emission dynamics.

Taking into account model fit, cross-period stability, and feature interpretability, this paper ultimately selects the k = 11 configuration as the primary model. This configuration includes eleven core factors, X3, X4, X7, X18, X5, X50, X28, X6, X26, X30, and X8, and achieves R²_ROLLING = 0.6198 under rolling validation. This result indicates that the model maintains strong explanatory capacity while demonstrating robust year-to-year predictive performance. Its accuracy level is consistent with that of the full XGBoost model under the rolling validation framework (R² ≈ 0.66), confirming the effectiveness of the stepwise feature expansion strategy in balancing feature selection and model robustness.

For the forward-looking carbon emission forecasting analysis, and to maximise predictive rationality and policy operability while ensuring model interpretability and cross-period robustness, this paper adopts a scenario extrapolation approach based on XGBoost, with the k = 11 feature configuration identified through a rolling validation framework. To further avoid excessive historical path dependence in emission forecasting, this study constructs an XGBoost-based “increment model” scenario extrapolation framework. Without introducing emission lag terms, the model learns from annual differences of the driving variables, thereby enabling dynamic simulation of Shanghai’s carbon emission trajectory for 2024–2060.

Unlike directly regressing on emission levels, the increment model uses the annual change in emissions (ΔX78) as the prediction target and incorporates the k = 11 feature set (X3, X4, X7, X18, X5, X50, X28, X6, X26, X30, X8) obtained from stepwise expansion and rolling validation. This substantially mitigates the non-stationarity and trend components inherent in the original series, allowing changes in emissions to be explained more by real variations in industrial structure and investment activity rather than by the self-propagating inertia of historical emissions.

Model training follows a rolling time-series cross-validation strategy, using 2000–T as the training set and T + 1 as the validation set. The process iteratively rolls T within the range [2005, 2022] to capture temporal stability, error structure, and extrapolation capability. Results indicate that the increment model maintains high goodness-of-fit and robustness for most years, avoiding the systematic drifts that typically arise when trend components dominate level-based models.

Future scenarios are constructed using a bottom-up pathway design approach. Specifically, four scenarios—Business-as-Usual (BAU), Green Transition (GREEN), High Investment/Manufacturing Reshoring (HIGH_INV), and Population Plateau (POP_PLATE)—are defined by assigning stage-specific growth rates to key driving factors. These growth paths are applied to the 2023 baseline to generate time series of driving variables for the period 2024–2060. The detailed trajectories of the driving factors are illustrated in Figure 12.

The assumed ranges of population size and urbanisation rate are consistent with the Shanghai Master Plan (2017–2035), which emphasises strict control over the scale of megacity population growth, continuous optimisation of population structure and spatial distribution, and adjustments to population density and per capita construction land to promote a more liveable city. The contraction pathways for industrial investment and manufacturing shares correspond to policy orientations such as “high-end, intelligent, and green manufacturing transformation” and the strict control of new capacity in energy-intensive industries. Assumptions regarding infrastructure and public-service variables are informed by planning objectives, including the “urban renewal initiative,” the “15-min community life circle,” and the construction of a green infrastructure system, reflecting a transition from expansion-oriented development toward efficiency-oriented improvement.

From a methodological perspective, parameter values in all scenarios are anchored to the statistical ranges and growth rates observed in historical data. Smoothed growth trends over the past 5–10 years are adopted as the baseline, upon which directional adjustments—classified as conservative, neutral, or strengthened—are introduced according to policy intensity. This design avoids unrealistic extreme assumptions. All variable changes are constrained within historically observed ranges or policy-admissible intervals, ensuring the feasibility and interpretability of the scenario simulations. The resulting emission trajectories across scenarios are consistent with the mechanisms identified through SHAP and nonlinear interaction analyses, namely the transition from scale effects to intensification and structural transformation. Specifically, the High-Investment scenario exhibits a hump-shaped pattern characterised by a construction-phase emission surge followed by delayed decline; the Green Transition scenario forms a pronounced downward pathway in the medium to long term; and the Population Plateau scenario reflects a low-growth steady-state trajectory dominated by scale constraints and efficiency gains.

(1)

Business-as-Usual (BAU) Scenario: an “inertia-continuation” pathway following the existing trend

This scenario extrapolates population, urbanisation rate, and investment structure based on the smoothed growth rates of the past 5–10 years, reflecting a continuation of the recent trajectory. It assumes a stable, slightly decelerating evolution, where population size and urbanisation maintain moderate growth (X3 and X4 increase by +0.6% → +0.6% → +0.5% and +2.0% → +2.0% → +1.0%, respectively). Population density grows at a pace consistent with recent years (X7 +2%). Employment in the secondary sector continues to decline gradually (X18 −1.0% → −1.0% → −2.0%). To reflect the inertia of economic expansion, industrial investment and infrastructure remain neutral to mildly expansionary (X28 +6% → +6% → +5%; urban road area X50 +7% → +6% → +5%). The share of industrial investment in fixed assets remains broadly stable before slightly decreasing (X30 +1.0 pp → 0.0 pp → −1.0 pp). The price level remains stable (X26 +0.2%).

Overall, the BAU scenario represents a development pattern characterised by a relatively stable economic structure, limited efficiency improvements, and a carbon-emission trajectory that slowly increases along the path of economic inertia, a typical “stable–slightly decelerating” continuation scenario.

(2)

Green Transition Scenario (GREEN): “slight early increase, rapid later decline,”

Under enhanced green transition and emission reduction policies, population and density growth slow down (X3 and X4 increase by +0.4% → +0.3% → +0.2% and +1.2% → +1.0% → +0.8%, respectively), mitigating the high-density “congestion rebound.” Population density declines significantly (X7 −1.5% → −2.0% → −2.0%). Employment in the secondary sector continues to shrink (X18 −2.0% → −2.5% → −3.0%). Industrial investment follows a “slight rise, sharp decline” trajectory (X28 +1.5% → −2.5% → −4.0%); the industrial investment share decreases (X30 −2.0 pp → −4.0 pp → −6.0 pp). Urbanisation converges more rapidly toward a high level (X6 +0.8 pp → +0.6 pp → +0.5 pp). Public service expansion and green-efficiency improvements strengthen simultaneously (X8 +3% → +5% → +4%).

This scenario represents a green development pathway characterised by proactive emission reduction and rapid structural transformation. It presents the most substantial reductions in industrial investment and employment, with carbon emissions consistently remaining below those of the Population Plateau scenario.

(3)

High Investment/Manufacturing Reshoring Scenario (HIGH_INV): a typical “surge, delayed decline” pattern driven by a construction-period shock

This scenario captures the short-term surge associated with manufacturing reshoring and investment stimulus, showing a hump-shaped “surge, retreat” trajectory. Population and resident population grow rapidly (X3 +3% → +3% → +3%; X4 +4% → +4% → +4%), and population density increases even more quickly (X7 +3% → +5%). Industrial investment and its share exhibit a sharp early rise followed by a delayed decline (X28 +10% → +8% → +5%; X30 +1.5 pp → +8 pp → −5 pp). Urban infrastructure investment rises in parallel (X50 +8% → +6%). Employment in the secondary sector increases in the short term before stabilising (X18 +1.5% → +5% → +4%).

This scenario explicitly embeds the previously identified mechanism of “positive construction-period effects, negative delayed feedback,” wherein short-term investment expansion raises emissions, followed by a slower subsequent decline as capacity utilisation stabilises.

(4)

Population Plateau Scenario (POP_PLATE): a “scale-constrained, intensity-driven” low-growth steady-state structure

In this scenario, both total population and resident population growth approach zero (X3: 0.0%/0.0%/0.0%; X4: +0.1%/0.0%/0.0%), while the resident population growth rate converges to zero (X4 +0.8% → 0% → 0%). Population density steadily declines (X7 −2% → −2% → −2%). Employment in the secondary sector continues to decline, albeit at a moderate rate (X18: −1.0% → −1.2% → −1.2%). Investment activities contract (X28 −1.5% → −3.0% → −3.0%), and the share of industrial investment exhibits a mild downward trend (X30 −0.5 pp → −0.5 pp → −0.3 pp). Urbanisation gradually converges toward its saturation level (X6 +0.4 pp). Public services and price levels remain in a neutral steady state (X26 +1.5%; X8 +2% → +2% → +2%).

Overall, the four scenarios diverge rapidly after 2025 (Figure 13). The HIGH_INV scenario exhibits the steepest and longest-lasting “investment–capacity surge” trajectory, crossing into a higher emission plateau around 2045 and showing only mild deceleration after 2050. This pattern reflects the mutually reinforcing effects of manufacturing reshoring, heavy capital investment, and road infrastructure expansion, as well as the associated risk of high-level emission lock-in. In contrast, the BAU scenario presents a moderate yet continuously increasing trajectory, assuming structural inertia and slow efficiency improvements. Although the growth rate tapers slightly after 2040, no structural decline emerges, indicating that incremental improvements alone are insufficient to meet stringent emission-reduction targets without additional policy interventions.

The POP_PLATE scenario maintains an almost linear, low-slope upward trend after 2024, consistently remaining below BAU. Its low-emission pattern is driven by stagnant population and density, persistent contraction of secondary-sector employment, and investment convergence. However, due to the limited marginal effect of efficiency gains, the emission curve does not naturally fall below the 2023 baseline. This suggests that “zero population growth” does not equate to “absolute emission reduction.” Building on this, the GREEN scenario follows a designed pathway of “slight initial increase, peaking around 2028, then steady decline.” The magnitude and persistence of this decline result from significant pullbacks in industrial investment and its share, a contraction of secondary-sector employment, the convergence of urbanisation toward its upper bound, and increased investments in public services and energy efficiency. Consequently, after 2030, the gap between GREEN and BAU/HIGH_INV expands rapidly. By 2050, GREEN stabilises at a substantially lower emission level and achieves an absolute reduction relative to the 2023 baseline before 2060, thereby meeting the objectives of carbon peaking and carbon neutrality.

4. Discussion

4.1. Spatiotemporal Evolution of Carbon Emissions

From 2000 to 2023, Shanghai’s carbon emissions exhibited an overall pattern of “rising before stabilisation, structural diversification, and pronounced intensity reduction.” Total emissions increased by approximately 1.4-fold, with an average annual growth rate of 3.8%. The period from 2000 to 2010 was characterised by rapid emission growth, likely driven by industrial expansion, a coal-dominant energy mix, and a sharp increase in motor vehicle ownership. Between 2010 and 2016, emissions underwent a phase of high-level fluctuation, oscillating within the range of 270–290 million tons, indicating that industrial restructuring and energy efficiency improvements had begun to take effect. After 2017, emissions entered a relatively stable stage, remaining between 290 and 310 million tons, marking a transition from growth-driven dynamics toward a near-peak equilibrium platform. Although carbon sinks increased substantially, their contribution to the overall carbon balance remained limited. Between 2000 and 2023, Shanghai’s carbon sink increased from 303,800 tons to 3.99 million tons, representing an approximate 12-fold rise. However, its proportion relative to total emissions consistently remained below 2%, which is insufficient to produce a meaningful offset in the overall carbon budget.

In terms of sectoral structure, Shanghai transitioned from a high-emission configuration dominated by industrial concentration to a more diversified, multi-sector emission profile. The industrial sector remained the primary contributor throughout the study period. Industrial emissions were 86.15 million tons in 2000 (66.9% of the total) and rose to 145.84 million tons in 2010, yet their share declined by nearly 12%, reflecting a structural shift from “high share, rising scale” to “high share, declining proportion.” This indicates that, under industrial restructuring and an expanding tertiary sector, industrial emission intensity decreased significantly, though its absolute scale continued to exert a decisive influence. The results reveal a dual pattern in which industrial emission reductions and efficiency improvements coexist with rising consumption-side emissions. While industry remained the primary driver of mitigation, the growing contributions from transportation and household consumption gradually formed new sources of emission growth.

Overall, Shanghai has entered a pre-peak transition stage, centred on energy efficiency improvements, structural optimisation, and consumption-side transformation. From 2000 to 2023, carbon intensity declined from 2.68 to 0.66 tons per 10,000 yuan of GDP, representing a 75.3% reduction, indicating a sustained downward trend. Although total emissions remained at a high level, the continued decline in carbon intensity indicates a gradual decoupling between economic growth and carbon emissions. This demonstrates that Shanghai has made notable progress in technological upgrading and structural optimisation; however, considerable pressure persists in controlling indirect emissions from consumption and transportation. This suggests that the benefits of industrial emission reduction and technological improvement are being partially offset by growth in consumption, underscoring the need to strengthen green lifestyles and energy-management mechanisms in urban daily life.

4.2. Carbon Emission Impact Mechanisms

Shanghai’s carbon-emission mechanism exhibits a three-stage dynamic pattern of “scale constraint → structural optimisation → efficiency enhancement.” The rolling-out-of-sample (OOS) SHAP results indicate that population and investment variables exert the strongest immediate upward pressure on emissions, while infrastructure, energy structure, and efficiency-related variables generate delayed feedback in the mid- and long-term. This reflects pronounced temporal heterogeneity and path dependence. In other words, population and investment expansion constitute the primary drivers of emissions in the initial stage, industrial upgrading and efficiency improvements produce mid-term mitigation effects, and ecological restructuring of the energy and infrastructure systems determines long-term low-carbon stability. Overall, scale expansion sets the upper bound of emissions, structural optimisation shapes emission intensity, and technological progress determines the pace of decarbonization.

Accordingly, policy interventions should advance along three coordinated routes:

(1) Controlling Scale Effects. Population-related variables exhibit the highest contribution among all factors, highlighting population expansion as the fundamental driver of urban carbon emissions. High population density and rapid urbanisation generate a strong “scale effect” in the early stage through increased household energy consumption, mobility demand, and infrastructure expansion, thereby pushing emissions upward. Once population density surpasses certain constraint thresholds, emissions enter a nonlinear regime characterised by “intensive equilibrium → congestion rebound,” reflecting dual constraints of resource carrying pressure and diminishing marginal energy efficiency. Future policy should therefore guide population distribution and investment timing to build a low-carbon infrastructure that supports a sustainable carrying system. In high-density stages, cities should achieve a balanced coupling among population, energy, and space through spatial restructuring and enhanced sharing of public resources.

(2) Optimising Structural Effects. During early industrialisation, expanding the secondary sector supports economic growth, but it simultaneously intensifies carbon emissions. Once industrial employment and sectoral shares exceed critical thresholds, technological innovation, productivity gains, and industrial restructuring become dominant forces, shifting emissions from growth toward stabilisation or decline. Industrial investment and its lagged effects show a precise combination of short-term positive and medium- to long-term negative influences. Immediate investment expansion generates high construction-period emissions, while SHAP values of the two-period lagged investment terms turn negative, indicating long-term mitigation benefits from green-capacity expansion and efficiency upgrading. This pattern demonstrates that the low-carbon transition should shift from controlling the scale of activity to improving its quality, advancing green manufacturing, strengthening energy-efficiency retrofits, and promoting substitution by the service sector, thereby achieving a dynamic balance between industrial expansion and emission reduction.

(3) Strengthening Technological Effects. Infrastructure development and energy-system expansion are associated with high emissions in their initial stages, reflecting a “high-carbon construction phase → stable-carbon operation phase” trajectory. High values of road area and water-supply capacity correspond to positive SHAP values, indicating that infrastructure scaling produces indirect “energy-consumption spillovers” through operating electricity demand, material use, and system maintenance. As system capacity approaches saturation thresholds, however, reduced congestion and improved operational efficiency attenuate the additional emission burden. Similarly, the coal share in the energy mix and the lagged unit GDP electricity consumption reveal the time-lagged mitigation effects of efficiency upgrades, demonstrating that coal phase-out and electrification generate sustained long-term decarbonization benefits across economic cycles. Accelerating coal displacement, end-use electrification, and efficiency retrofits is therefore essential to lowering the carbon intensity of the energy and engineering chains and facilitating the transition from an expansion-driven to a stability-driven emissions regime.

4.3. Future Carbon Emission Trajectories and Optimization Pathways for Shanghai

Multi-scenario simulations indicate that Shanghai’s future carbon emission trajectory exhibits a three-stage pattern characteristic of a megacity, i.e., “inertial escalation, structural divergence, and policy convergence.” This evolution is not merely the outcome of scale expansion, but rather the combined result of Shanghai’s transition in development stage, the upgrading of its industrial structure, and increasingly binding spatial and institutional constraints.

Under the baseline scenario (BAU), if existing development inertia and the current pace of energy-efficiency improvement persist, carbon emissions are projected to maintain a moderate upward trend and gradually approach a plateau after 2040, without achieving a substantive decline. This outcome suggests that, as a megacity entering a phase of population stabilisation and strict land-development constraints, Shanghai can no longer rely on natural structural evolution or marginal efficiency improvements to deliver significant emission reductions. Path dependence embedded in the stock-based economy and existing energy systems remains pronounced.

The high-investment scenario (High-Investment) exhibits a more prominent construction-phase shock. In the short term, large-scale infrastructure investment and capital deepening in manufacturing substantially elevate energy demand, driving emissions to rise rapidly and remain locked in at a high level during the medium term. This effect is particularly evident in Shanghai, where stringent land-use constraints mean that investment expansion is primarily manifested as high-intensity, capital-intensive redevelopment, thereby amplifying the energy consumption and emission response per unit of investment.

The population plateau scenario (Population Plateau) significantly weakens scale effects by driving both population size and density growth toward a zero-growth range, while simultaneously tightening investment expansion and promoting industrial intensification. Under this scenario, the emission curve becomes flatter and stabilises at a relatively low level, reflecting the upper limit of emission mitigation that Shanghai can achieve through scale constraints and spatial efficiency improvements once population and construction scale have largely peaked. However, this pathway alone remains insufficient to trigger absolute emission reductions, indicating that reliance on “scale capping” is inadequate to support deep decarbonisation objectives.

By contrast, the green transition scenario (Green) represents the optimisation pathway most consistent with Shanghai’s development orientation, characterised by “short-term peaking and long-term decline.” Under this scenario, Shanghai is projected to reach a phased emission peak before 2040 and reduce emissions below the 2023 level by 2060. The effectiveness of this pathway lies in the formation of a mitigation system centred on “structural optimisation and efficiency enhancement,” built upon a service-oriented economy and advanced manufacturing base through industrial upgrading, sustained energy-efficiency improvements, and the low-carbon transformation of public service systems. This trajectory is also aligned with the planning principles articulated in the Shanghai Master Plan (2017–2035), which emphasise optimising urban spatial structure, strictly controlling city scale, and pursuing a development path that integrates economic growth, social well-being, and ecological sustainability.

Overall, the optimal pathway for Shanghai’s future carbon emissions should follow a megacity-specific, coordinated strategy of “scale constraint, structural transformation, and efficiency-driven improvement.” Specifically, this involves the following: (1) scale constraint: strengthening boundary management of population and investment growth to curb the inertial expansion of construction-related emissions, with the permanent resident population targeted at approximately 25 million by 2035; (2) structural transformation: accelerating the greening of manufacturing and increasing the share of the service sector, while promoting the relocation of energy-intensive activities toward low-carbon value chains, with R&D expenditure reaching 5.5% of Shanghai’s GDP by 2035; and (3) efficiency-driven improvement: advancing energy-efficiency technologies and end-use electrification through coal phase-out, smart-grid development, and building energy-efficiency systems, aiming to reduce total carbon emissions by 5% from the peak level by 2035. This synergistic governance pathway, centred on technological upgrading, structural optimisation, and scale control, can facilitate a strategic shift from growth-driven to efficiency-driven development, providing scientific support and decision-making guidance for Shanghai to achieve a stable emission peak around 2030 and progress toward carbon neutrality before 2060.

4.4. Limitations and Prospects

This study has several limitations. First, although machine-learning methods are employed to identify the main driving mechanisms of Shanghai’s carbon emissions and their scenario-based evolution under constrained feature conditions, the analysis is confined to a single city over the period 2000–2023. The relatively short temporal span and limited spatial coverage make it difficult to fully capture long-term structural dynamics or cross-regional heterogeneity. The issue of “small samples” is, however, common in long-term urban carbon-emission studies. Owing to constraints such as statistical yearbook publication cycles, changes in accounting standards, and the lack of long-term continuous emission factors at the city scale, most existing studies necessarily rely on annual time series of comparable length. This structural limitation inevitably restricts the parameter stability and long-horizon extrapolation capacity of both traditional statistical models and machine-learning approaches.

Second, although rolling-window out-of-sample validation is adopted to partially assess model robustness under varying training-sample sizes, the spatial generalisability and long-term stability of the results still require verification using broader datasets. In addition, several socioeconomic and energy-related indicators are derived from official statistical accounts, which may involve cumulative measurement errors or representativeness bias. Finally, the scenario simulations in this study should be understood as policy-oriented quantitative extrapolation tools designed to characterise relative trends under alternative development pathways, rather than as precise numerical forecasts of specific policy outcomes. Given current data availability, the time series remains insufficient to fully capture structural evolution across multiple climate-policy cycles.

Future research should therefore extend both the temporal and spatial dimensions of the dataset by constructing multi-city, multi-scale spatiotemporal carbon-emission databases. Incorporating higher-frequency and more disaggregated energy statistics (e.g., monthly energy use, gridded consumption estimates), remote-sensing observations, and auxiliary multi-source data (such as nighttime light intensity and industrial activity proxies), together with expert consultation or scenario co-design, would allow for more refined calibration of key parameters. Such efforts would enhance the ability to identify long-term institutional and structural transitions and provide more timely and scalable decision support for regional and national low-carbon transitions.

5. Conclusions

Drawing on multi-source statistical data for Shanghai from 2000 to 2023, this paper integrates machine-learning modelling (XGBoost) with rolling out-of-sample back-testing (Rolling OOS) to systematically identify the single-factor and multi-factor mechanisms influencing urban carbon emissions. Through SHAP (Shapley Additive exPlanations), the marginal contributions and nonlinear interaction effects of each feature variable are quantitatively interpreted, revealing the dynamic relationships through which multidimensional factors, population size, industrial energy efficiency, investment structure, and infrastructure, shape carbon-emission outcomes. Furthermore, based on incremental modelling and a set of multi-scenario assumptions (BAU, Green, High-Investment, Population Plateau), this paper simulates emission trajectories under different policy and structural conditions, identifying the emission-response patterns and optimisation directions across Shanghai’s development stages.

(1)

Overall evolution of carbon emissions

Between 2000 and 2023, Shanghai’s carbon emissions followed a trajectory of “continuous growth, periodic fluctuations, and gradual stabilisation,” transitioning from a growth-driven pattern to a steady-state equilibrium. Although carbon sinks increased substantially, their offsetting effect remained limited, with the overall carbon balance still constrained by the energy structure and emissions on the consumption side. The industrial structure shifted from “industry-dominated” to a “mixed industry, transport–household” pattern, while carbon intensity declined by approximately 75%, indicating an emerging decoupling between economic growth and emissions.

(2)

Driving mechanisms of carbon emissions

SHAP analysis indicates that population size, investment structure, infrastructure, and the energy mix are the dominant variables. Population-related factors consistently exhibit the highest contributions, forming a pronounced “scale effect.” Industrial investment elevates emissions in the short term but shows negative feedback in lagged periods, confirming a dynamic mechanism of “positive effects during the construction phase, mitigation effects during the lag phase.” Infrastructure development and coal-share variables exhibit marginal attenuation once they reach high levels, reflecting a “high-carbon construction, stable-carbon operation” stage pattern. Public service and social investment variables exhibit a composite evolution characterised by “initial promotion, mid-term suppression, long-term stabilisation”.

(3)

Multi-factor interaction effects

Interaction-SHAP results show that roughly 70% of variable pairs produce positive amplification. The ΔH values for population–urbanisation, population–infrastructure, and population–industrial investment are the highest, indicating that population factors serve as key “amplifiers” of carbon emissions. When high population scale coincides with high investment intensity, emission elasticity rises sharply, whereas interaction terms involving energy efficiency and industrial upgrading produce stable offsetting effects, forming a “structure–technology” balancing mechanism.

(4)

Future evolution and optimization pathways

Scenario analysis indicates that, under the Business-as-Usual (BAU) scenario, carbon emissions continue to rise gradually and fail to reach a clear peak. The High-Investment scenario produces a typical “surge–high-level lock-in” trajectory, while the Population Plateau scenario forms a low-level steady state under scale constraints, but with relatively limited mitigation effects. In contrast, the Green Transition scenario follows a “short-term peak–long-term decline” pathway and falls substantially below the BAU baseline before 2060, demonstrating the greatest emission-reduction potential among all scenarios.

Integrating the multi-scenario results with the key drivers identified through SHAP analysis, the optimisation of Shanghai’s future carbon-emission pathway should follow a tripartite, synergistic strategy of “scale constraint–structural transformation–efficiency enhancement.” First, at the scale-constraint level, policies should focus on regulating population size and spatial density, imposing carbon- and energy-intensity constraints on major investment projects, and strengthening restrictions on high-energy-consuming construction activities, thereby preventing the amplification of emissions once population agglomeration and investment expansion exceed critical thresholds. Second, at the structural-transformation level, efforts should accelerate the shift toward service-oriented and high-end manufacturing sectors, guide industrial investment and employment toward low-carbon and high–value-added activities, and weaken path dependence on traditional industrial structures through optimised industrial-park layouts and differentiated energy-use policies. Third, at the efficiency-enhancement level, priority should be given to improving end-use energy efficiency and advancing energy-system decarbonisation, including building retrofits, expansion of public transport and transport electrification, clean-energy substitution, and efficiency optimisation of public-service facilities.

Overall, Shanghai’s low-carbon transition should be jointly driven by scale constraints that define boundary conditions, structural transformation that shapes medium- and long-term trajectories, and efficiency improvements that unlock sustained mitigation potential, thereby facilitating a shift from a growth-driven to an efficiency- and structure-driven emission regime.