Performance of CMIP6 Models in Capturing Summer Maximum Temperature Variability over China

Abstract

Previous research has primarily focused on assessing seasonal mean or annual extreme climate events, whereas intraseasonal variability in extreme climate has received comparatively little attention, despite its importance for understanding short-term climate dynamics and associated risks. This study evaluates the performance of nine climate models from the Coupled Model Intercomparison Project Phase 6 (CMIP6) in reproducing summer maximum temperature (Tmax) variability across China during 1979–2014, with the variability defined as the standard deviation of daily Tmax anomalies for each summer. Results show that most CMIP6 models fail to reproduce the observed north–south gradient of Tmax variability with significant regional biases and limited agreement on temporal trends. The multi-model ensemble (MME) outperforms most individual models in terms of root-mean-square error and spatial correlation, but it still under-represents the observed temporal trends, especially over southeastern and central China. Taylor diagram analysis reveals that EC-Earth3, GISS-E2-1-G, IPSL-CM6A-LR, and the MME perform relatively well in capturing the spatial characteristics of Tmax variability, whereas MIROC6 shows the poorest performance. These findings highlight the persistent limitations in simulating intraseasonal Tmax variability and underscore the need for improved model representations of regional climate dynamics over China.

1. Introduction

The World Climate Research Programme (WCRP) identifies alterations in extreme weather and climate events as a primary scientific challenge. Similarly, the Intergovernmental Panel on Climate Change (IPCC) has also highlighted this issue in its assessment reports [1,2]. Surface air temperature serves as a fundamental and widely used indicator for assessing both regional and global climate changes, as well as the occurrence of extreme weather and climate events [3,4]. Extreme weather and climate events pose growing risks to societies and ecosystems [5,6]. As a major agricultural nation, China is particularly vulnerable to extreme high temperatures, which can severely impact agriculture, public health, labor productivity, and natural resources, resulting in substantial economic losses [7]. These risks underscore the urgent need to quantify and understand the variability in extreme high temperatures over China.

Climate models have long been leveraged as essential tools for analyzing both historical and projected changes in climate extremes [8,9,10,11]. The recent development of a new generation of climate models for the Coupled Model Intercomparison Project Phase 6 (CMIP6) experiment provides an opportunity to assess their performance [12]. Numerous studies have assessed the capability of climate models from different CMIP phases to simulate temperature extremes at both global and regional scales [13,14,15,16,17]. For instance, CMIP6 models could effectively reproduce the spatial distributions and temporal variations of temperature extremes and compound warm/wet extremes in some regions [18,19,20]. However, CMIP5 and CMIP6 models consistently overestimate the magnitude of temperature anomalies for both hot and cold extremes, with CMIP6 showing modest improvements over CMIP5 [21,22]. CMIP5 and CMIP6 models reproduce warm extremes reasonably well but underestimate the spatial heterogeneity and temporal trends of cold extremes across China with CMIP6 exhibiting reduced biases and inter-model variability [23].

Previous research has predominantly concentrated on assessing seasonal mean or annual extreme climate events at both global and regional scales, with less emphasis on intraseasonal variability in extreme climates. This intraseasonal variability is strongly associated with the occurrence and intensity of extreme climates, which in turn have significant impacts on human health and energy demands [5,6,7]. However, few studies have explicitly addressed the intraseasonal variability in extreme climates, particularly in temperature extremes. This represents a critical research gap, especially for regions like China, where intraseasonal climate variability is strongly influenced by monsoonal dynamics and land–atmosphere interactions.

The IPCC’s Special Report on Managing Extreme Events and Disaster Risks to Promote Climate Change Adaptation (SREX) suggests that shifts in extreme temperature events may result from changes in the mean state and alterations in the probability density function, including metrics such as skewness, kurtosis, and variations in intraseasonal standard deviation [24]. Recent studies have demonstrated that interannual and intraseasonal fluctuations in daily temperature variability can account for more than half of the regional differences in warming trends, potentially outweighing mean-state warming in driving changes in heatwave severity and occurrence rates [6,25]. This suggests that climate models with limited representation of extreme temperature variability may struggle to accurately simulate the frequency and intensity of extreme temperature events. Since variability is strongly associated not only with the occurrence of extreme temperature events but also with their temporal clustering and duration, its accurate simulation is crucial for improving the prediction of temperature extremes. Therefore, a robust evaluation of CMIP model simulations of maximum temperature variability is critical to quantify and correct biases in the magnitude and temporal structure of temperature extremes.

This research aims to evaluate the performance of nine CMIP6 climate models in simulating summer maximum temperature (Tmax) variability, which is calculated by the standard deviation of the daily Tmax in each year’s summer, across China during 1979–2014. The remainder of this study is organized as follows. Section 2 presents the data and methodology. Section 3 shows the performance of nine CMIP6 climate models in reproducing summer Tmax variability over China for the period of 1979–2014. Section 4 summarizes the main conclusions of the present study.

2. Data and Methodology

2.1. Data

Daily Tmax data for the period 1979–2014 from historical simulations of 9 climate models (

Table 1. The list of 9 CMIP6 models in the present study and their horizontal spatial resolutions.

Model Name	Resolution	Country
AWI-CM-1-1-MR	384 × 192	Germany
CanESM5	128 × 64	Canada
CNRM-CM6-1	256 × 128	France
EC-Earth3	512 × 256	Sweden and 9 other European countries
GISS-E2-1-G	144 × 90	USA
IPSL-CM6A-LR	144 × 143	France
MIROC6	256 × 128	Japan
MPI-ESM1-2-HR	384 × 192	Germany
MRI-ESM2-0	320 × 160	Japan

) that participated in the CMIP6 project are utilized in this study. The CN05.1 daily Tmax data with a horizontal resolution of 0.25° × 0.25° during 1979–2014, which is interpolated from quality-controlled 2416 meteorological observations [26], is used to evaluate the performance of CMIP6 models. The CN05.1 data was developed by the National Climate Center of China Meteorological Administration. It is generated using an improved “anomaly approach” with thin-plate spline interpolation, which is widely used in climate studies over China. The CN05.1 dataset has been extensively evaluated against independent station data and other gridded products. It performs well in representing temperature variability as well as temperature extremes, although uncertainties may exist in areas with sparse station coverage or complex topography [26]. Note that the CMIP6 and CN05.1 data is re-gridded to a resolution of 1° × 1° in this research for ease of comparison.

2.2. Methodology

Here we mainly focus on the performance of CMIP6 models in reproducing summer intraseasonal Tmax variability during 1979–2018. Daily Tmax anomalies were first computed for each day from 1979 through 2014. Specifically, the climatological daily Tmax for each calendar day (i.e., day 1 to day 365) was calculated over the period 1979–2014. This was accomplished by averaging the Tmax values for each specific day across all years in the study period. For example, the climatological mean Tmax for 1 January was obtained by averaging the Tmax values for 1 January from 1979 to 2014. The daily Tmax anomaly for each day in each year was calculated by subtracting the corresponding climatological mean Tmax value on that particular day. Then the summer intraseasonal Tmax variability was quantified as the standard deviation of Tmax anomalies in each year’s summer (June–August, 92 days) during 1979–2018. In this research, a multi-model ensemble (MME) was constructed by assigning equal weights to all 9 models, treating each as equally plausible contributors to the ensemble’s prediction.

The correlation analysis, root-mean-square error (RMSE), mean bias, and Taylor diagrams are utilized to assess the performances of CMIP6 simulations [19,22,27,28]. The temporal trends of Tmax variability in observations and CMIP6 outputs are estimated via linear least squares regression analysis [29]. The significances of correlation coefficients and mean bias are estimated via the Student’s t test.

Taylor diagrams offer an integrated graphical approach for evaluating the performance of climate models by simultaneously presenting three key statistical metrics: the Pearson correlation coefficient, the normalized standard deviation, and the centered root-mean-square error. These diagrams provide a unified visualization that facilitates direct comparison between model outputs and observations in terms of pattern similarity, amplitude, and overall error. In this polar-coordinate system, the reference observation is typically plotted along the horizontal axis, while each model is represented as a point whose angular and radial positions reflect the correlation and standard deviation, respectively. The distance from the reference point corresponds to the centered root-mean-square error, enabling a concise assessment of model fidelity across multiple dimensions in a single framework. This method is particularly useful for diagnosing the ability of multiple models to reproduce observed spatial or temporal variability [19,27].

3. Results

3.1. Comparison of Spatial Distribution and Trend of Summer Tmax Variability over China

To understand the variations in Tmax variability across China, the spatial distribution and temporal trends of summer Tmax variability during 1979–2014 are investigated. This analysis allows us to identify regions with pronounced variability and to detect potential hotspots of increasing or decreasing trends. By exploring these spatial and temporal characteristics, we can gain valuable insights into the regional distribution patterns of Tmax variability over China.

Figure 1 presents climatological summer Tmax variability, quantified as the standard deviation of Tmax anomalies, over China during 1979–2014. The observed spatial distribution of summer Tmax variability from CN05.1 exhibits an evident north–south gradient, with higher variability over northern and central China, where the standard deviation ranges approximately from 2.5 to 4 °C. In contrast, the variability is lower over the Tibetan Plateau and southern China, with standard deviations around 2 °C (Figure 1a). The MME generally exhibits a pronounced Tmax variability peak in the northern region, with a gradual decrease towards the southwest. In particular, the MME overestimates Tmax variability over the Tibetan Plateau and northern China with evident regional bias compared with the CN05.1 (Figure 1b). Individual CMIP6 climate models exhibit diverse performances with substantial inter-model differences in both the magnitude and spatial pattern. GISS-E2-1-G, MIROC6, and MRI-ESM2-0 exhibit a pronounced overestimation in Tmax variability across China especially over northern areas. AWI-CM-1-1-MR, CanESM5, and IPSL-CM6A-LR show a moderate underestimation in Tmax variability over southeastern China. CNRM-CM6-1, EC-Earth3, and MPI-ESM1-2-HR slightly overestimate Tmax variability over northern China and underestimate Tmax variability over southern China. Overall, most CMIP6 climate models are unable to reproduce the general north–south gradient of Tmax variability, and notable biases remain both in the magnitude and spatial distribution.

Figure 2 displays temporal trends of summer Tmax variability from CN05.1, MME, and individual climate models that participated in CMIP6. Summer Tmax variability derived from CN05.1 exhibits pronounced positive trends over southeastern and central China, particularly around the middle and lower reaches of the Yangtze River Basin, with rates ranging from 0.1 to 0.2 °C per decade. In contrast, negative trends are detected across parts of southwestern China (Figure 2a). The MME does not capture the overall trend pattern of summer Tmax variability across China. Apart from some regions over northeastern China, MME presents insignificant negative trends of summer Tmax variability (Figure 2b). AWI-CM-1-1-MR exhibits a marked decreasing trend for summer Tmax variability around northeastern and southeastern China and an increasing trend around the central Tibetan Plateau (Figure 2c). Contrary to CN05.1, CanESM5 shows a negative trend around lower reaches of the Yangtze River Basin and a positive trend around the Tibetan Plateau and northeastern China (Figure 2d). Except for limited areas, CNRM-CM6-1, GISS-E2-1-G, MIROC6, MPI-ESM1-2-HR, and MRI-ESM2-0 show insignificant trends over the most regions of China with substantial discrepancies comparing to that of the CN05.1 (Figure 2e,g,i–k). EC-Earth3 exhibits a marked negative trend for summer Tmax variability over northern and central China (Figure 2f). Significant decreasing trends are detected over northeastern China in IPSL-CM6A-LR (Figure 2h). Compared with the CN05.1 model, CMIP6 MME and individual models show poor performances in simulating the trends of Tmax variability in summer over China with divergent trend distributions inconsistent with that of the observation.

3.2. Evaluation of CMIP6 Models in Simulating Summer Tmax Variability over China

Accurate simulation of climate extremes is essential for reliable climate projections and impact assessments. In this section, we evaluate the performance of the selected 9 CMIP6 climate models in reproducing summer Tmax variability over China. By comparing model outputs with CN05.1 data, we assess the strengths and limitations of the models and identify areas where improvements are needed. This evaluation provides a basis for selecting appropriate models for future climate projections and for understanding model uncertainties.

Differences in summer Tmax variability between CMIP6 MME, individual climate models, and CN05.1 over China during 1979–2014 are given in Figure 3. The MME displays positive biases in summer Tmax variability over western and northeastern China and negative biases around southeastern China. The most pronounced biases are primarily confined to fragmented regions in western and northern China, where the MME overestimates Tmax by approximately 1.0 to 1.5 °C compared to CN05.1 (Figure 3a). Compared to the CN05.1, both AWI-CM-1-1-MR and MPI-ESM1-2-HR show marked negative biases around middle and lower reaches of the Yangtze River, and with significant positive biases around northwestern Tibetan Plateau with MPI-ESM1-2-HR being less significant (Figure 3b,i). Summer Tmax variability is significantly overestimated around some areas over western China and underestimated over southeastern China in CanESM5 (Figure 3c). The differences in summer Tmax variability between CNRM-CM6-1 and CN05.1 over most regions of China are insignificant except for limited areas over northwestern China (Figure 3d). A large bias in summer Tmax variability between EC-Earth3 and CN05.1 is detected over northeastern China, where EC-Earth3 overestimates Tmax variability by approximately 1–2 °C (Figure 3e). Apart from southeastern China, positive biases in summer Tmax variability occur across most regions of China for GISS-E2-1-G, MIROC6, MPI-ESM1-2-HR, and MRI-ESM2-0 compared to CN05.1. The pronounced biases are predominantly located in western and northern China, with MIROC6 exhibiting the most substantial bias and MRI-ESM2-0 showing a less pronounced one (Figure 3f,h,j). For IPSL-CM6A-LR, significant positive biases in summer Tmax variability are observed over the central Tibetan Plateau, with overestimations of approximately 1.0–1.5 °C. In contrast, pronounced negative biases are present over the western Tibetan Plateau and southeastern China, where the model underestimates variability by about 1.0–2.0 °C (Figure 3g). Overall, CMIP6 individual models exhibit distinct regional biases in summer Tmax variability over China with AWI-CM-1-1-MR, CNRM-CM6-1, and MPI-ESM1-2-HR demonstrating relatively better performance.

RMSE is a widely used measure that quantifies the average magnitude of the differences between model-simulated values and actual observed values. The RMSE values of summer Tmax variability from CMIP6 MME and individual climate models compared against the CN05.1 data over China during 1979–2014 are shown in Figure 4. The MME shows higher RMSE values, ranging from 0.9 to 1.5 °C, in the lower reaches of the Yangtze River as well as in western and northern China. The MME has a lower RMSE compared to individual CMIP6 climate models with values ranging from 0 to 1.5 (Figure 4a). AWI-CM-1-1-MR and MPI-ESM1-2-HR perform relatively better than other climate models with lower RMSE values. However, large RMSE values, ranging from 0.9 to 1.5 °C, are also detected around middle and lower reaches of the Yangtze River as well as the Tibetan Plateau in AWI-CM-1-1-MR and MPI-ESM1-2-HR (Figure 4b,i). For CanESM5 and CNRM-CM6-1, large RMSE values are scattered across China with the most pronounced errors occurring in the northwest (Figure 4c,d). Compared to the CN05.1 model, EC-Earth3 exhibits large RMSE values over northern China, ranging from 1.5 to 2.1 °C, while showing lower RMSE values over southeastern China (Figure 4e). For IPSL-CM6A-LR, larger RMSE values are evident, ranging from 0.9 to 1.8 °C, around southeastern China and the Tibetan Plateau (Figure 4g). The GISS-E2-1-G, MIROC6, and MRI-ESM2-0 models all yield higher RMSE values than the other models across most regions of China, with GISS-E2-1-G showing the largest errors. Notably, in certain regions, the RMSE for GISS-E2-1-G exceeds 3 °C (Figure 4f,h,j).

Correlation coefficients between summer Tmax variability from CMIP6 MME, individual climate models, and CN05.1 over China during 1979–2014 are displayed in Figure 5. As shown in Figure 5a, marked positive correlation coefficients (exceeding 0.3) between summer Tmax variability from CMIP6 MME and CN05.1 are found over southeastern China, with smaller but still significant areas around the western Tibetan Plateau and the northernmost margins of China. In contrast, significant negative correlation coefficients are mainly distributed around southwestern and northeastern China, indicating poor performance of the MME in capturing summer Tmax variability there. Except for a few scattered areas, correlation coefficients between summer Tmax variability simulated by the majority of CMIP6 climate models and CN05.1 are insignificant over China (Figure 5b–d,f,h). For EC-Earth3, significant positive correlation coefficients are observed over southeastern and northeastern China, indicating that EC-Earth3 performs particularly well in these regions (Figure 5e). IPSL-CM6A-LR exhibits pronounced positive correlation coefficients around southeastern China (Figure 5g). For MPI-ESM1-2-HR, significant positive correlation coefficients appear over southeastern China and southern Tibetan Plateau whereas marked negative correlation coefficients occur around northeastern China (Figure 5i). MRI-ESM2-0 shows pronounced positive correlation coefficients over western China (Figure 5j).

The Taylor diagram [28] is employed to estimate the skill of the CMIP6 MME and climate models in simulating the spatial pattern of climatological summer Tmax variability over China during 1979–2014. Figure 6 displays Taylor diagrams for summer Tmax variability from CMIP6 MME and individual climate models as compared with CN05.1 over China during 1979–2014. The performances of CMIP6 MME and individual climate models in capturing summer Tmax variability are distributed loosely, indicating the inter-model spread of summer Tmax variability is large. The MME, EC-Earth3, GISS-E2-1-G, IPSL-CM6A-LR, and MRI-ESM2-0 exhibit relatively higher spatial correlation coefficients, generally exceeding 0.5, indicating a spatial pattern of climatological summer Tmax variability that is more similar to that of CN05.1. MIROC6 shows the lowest spatial correlation coefficient among those nine climate models, with a value of approximately 0.35, suggesting a poor performance of MIROC6 in capturing spatial pattern of climatological summer Tmax variability over China. The normalized spatial standard deviations of the CMIP MME and individual climate models range from 1.2 to 1.5, exceeding that of CN05.1. The normalized spatial standard deviation simulated by IPSL-CM6A-LR is the closest to that of CN05.1. However, MIROC6 and MRI-ESM2-0 represent relatively large normalized spatial standard deviations, which is much larger compared with the CN05.1. The centered pattern root-mean-square differences range from 1.2 to 1.4 with IPSL-CM6A-LR showing the lowest value and MIROC6 exhibiting the largest value. Therefore, the MME, EC-Earth3, GISS-E2-1-G, and IPSL-CM6A-LR perform relatively well in capturing the spatial characteristics of climatological summer Tmax variability over China, outperforming the other climate models. However, MIROC6 possesses a fairly poor ability in simulating the spatial patterns of the climatological summer Tmax variability over China.

Despite advances in model physics and increased resolution, most CMIP6 models still exhibit substantial limitations in accurately simulating Tmax variability over China. One of the main reasons for the limited performance is that many models still have difficulties in accurately representing key physical processes and the land–atmosphere interactions that strongly influence surface air temperature variability. Particularly, those physical processes related to surface energy partitioning and evapotranspiration processes during heat extremes [30,31,32,33]. Many models struggle to capture the spatial heterogeneity of land cover, vegetation dynamics, and soil moisture feedbacks, all of which play crucial roles in modulating local and regional heat extremes. Furthermore, biases in simulating cloud cover, precipitation, and radiative fluxes can propagate into errors in surface temperature variability [21,30,31,32,33]. These combined limitations underscore the complexity of the processes governing extreme high temperature events and highlight the need to further refine both physical parameterizations and coupled land–atmosphere interactions in climate models to improve the fidelity of future projections of summer temperature extremes.

4. Conclusions

This study conducted a comprehensive evaluation of nine CMIP6 climate models in simulating the intraseasonal summer Tmax variability across China during the historical period of 1979–2014. The variability was quantified as the standard deviation of daily Tmax anomalies for each summer season (June–August), and model performance was assessed relative to the CN05.1 dataset. A suite of statistical metrics, including RMSE, mean bias, and Taylor diagrams, was employed to assess both the spatial distribution and temporal evolution of summer Tmax variability over China during 1979–2014.

The results reveal that most CMIP6 models struggle to replicate the observed spatial gradient of summer Tmax variability, particularly the prominent north–south contrast. The MME generally provides improved performance relative to individual models, capturing broad features of the observed variability and exhibiting lower RMSE and higher correlation coefficients in several regions. Among the individual models, EC-Earth3 and IPSL-CM6A-LR perform relatively well in reproducing both the magnitude and spatial structure of Tmax variability, while MIROC6 displays the poorest performance. In terms of temporal trends, the observed increase in summer Tmax variability over southeastern and central China is poorly captured by most models, which tend to underestimate or misrepresent the spatial pattern and significance of these trends.

Overall, this study provides a systematic assessment of the capabilities and limitations of CMIP6 models in reproducing the intraseasonal variability of summer Tmax over China. The substantial inter-model spread and regional inconsistencies highlight that, despite advancements in model resolution and physical parameterizations, significant uncertainties remain in simulating sub-seasonal temperature variability. The added value of the MME demonstrates the utility of ensemble approaches in reducing individual model biases and enhancing the robustness of simulated climate variability. These findings underscore the ongoing challenges in simulating sub-seasonal temperature variability over complex terrains such as China and highlight the necessity of further improving climate model representations of land–atmosphere interactions and regional-scale processes. Future work should extend the analysis to a broader ensemble of CMIP6 models, incorporate alternative observational datasets for more robust validation, and explore the physical mechanisms underlying the model deficiencies to guide targeted model improvements.