The Retrospective and Predictive Effectiveness of ESG Ratings: Evidence from China

Abstract

With the development and proliferation of sustainable investing, ESG ratings have gradually become an important basis for measuring corporates’ ESG performance and influencing investors to make investment decisions. However, the validity of ESG ratings has also raised public concerns due to the differences in the evaluation systems and standards of ESG rating agencies. This paper analyzes the effectiveness of ESG rating data provided by Chinese rating agencies in terms of retrospective and predictive effectiveness. It assesses how well these data reflect the past ESG performance of Chinese companies and its ability to predict future ESG performance. The study focuses on China Securities Index 300 companies from 2016 to 2020 and benchmarks their ESG ratings against five indicators derived from negative events. Through regression analysis, this paper studies the association between these indicators and ESG ratings. The results indicate that domestic ESG ratings in China can capture the past ESG performance of Chinese companies, but they can only partially predict the future ESG performance.

1. Introduction

Under the mounting sustainability pressure from regulators, the market, and various stakeholders, investors’ interests in the green and sustainable finance sector have been steadily increasing [1]. As a result, environmental, social, and governance (ESG) issues have become important considerations for global investors when making decisions [2]. The importance and proliferation of ESG investing is manifested by the scale of the signatories of the United Nations (UN) Principles for Responsible Investment (PRI), a premier international organization promotes a global sustainable development financial system. As of 31 March 2024, the UN PRI has 5345 signatories with an estimated assets under management (AUM) value exceeding $121 trillion. It is forecast that ESG-oriented AUM value will continue to grow and constitutes more than 20% or even 25% of the global AUM [3].

ESG ratings play a critical role in promoting ESG investing by providing investors with valuable insights into a company’s sustainability, ethical practices and risks, and off-the-shelf tools for investing. These ratings are usually provided by for-profit rating agencies such as MSCI, Sustainalytics, S&P Global, etc. [4]. The ESG ratings assess a firm’s performance on various ESG criteria, such as its environmental impact, labor practices, diversity and inclusion, and corporate governance [5,6,7,8]. In addition to equities, ESG ratings can also be constructed for other assets, such as bonds and funds [9,10]. Investors use ESG ratings to make informed decisions about allocating capital to companies that align with their values and sustainability goals, through strategies such as negative screening [11].

While ESG ratings have become increasingly important in the business world, there are some notable challenges and areas of confusion surrounding their effectiveness. First, different rating agencies often provide varying ESG scores for the same company [12]. This inconsistency can lead to confusion for investors and decision-makers. Chatterji et al. (2016) argues that “low agreement implies all or almost all of the ratings have low validity [13]”. Second, ESG ratings heavily rely on companies’ self-reported data, which are not always audited by third-party sources or ESG providers and potentially introduce biases [14]. Third, there are concerns about “greenwashing”, as some companies may exaggerate their ESG efforts and practices to appear more sustainable than they actually are [15]. Inconsistency between different rating agencies, lack of third-party assurance, and the presence of greenwashing lead to concerns about the ratings’ effectiveness [16,17]. If the ESG ratings were ineffective, they would result in misallocation of resources for investors who rely on these ratings to develop investment portfolio strategies [18]. Therefore, it is crucial to examine the effectiveness of ESG ratings for global sustainable development.

This study examines the retrospective and predictive effectiveness of ESG ratings provided by domestic rating agencies in China. Retrospective effectiveness refers to whether ESG rating can accurately capture the real-world ESG impact in the past. Predictive effectiveness assesses whether ESG ratings can predict real-world outcomes related to environmental and social performance [14]. All ESG ratings are retrospective to some degree, in the sense that they provide information about ESG performance during a prior time period. Since 2020, the governing and regulatory bodies in China have been promoting the ESG agenda in the country’s financial market and corporate world [19,20]. The expansion of the ESG landscape in China is accompanied by a group of nascent domestic rating agencies [21]. However, in an emerging economy like China it raises the question whether these agencies can accurately evaluate and predict a company’s ESG performance based on the information they collect and the rating criteria they establish, that is, the validity of ratings from domestic rating agencies has raised concerns among investors. In this paper, the measurement of effectiveness involves evaluating whether ESG ratings effectively align with the real-world impacts of corporate ESG practices. The usual method is to compare the ESG ratings with an external criterion of ESG performance [14]. This study benchmarks ESG ratings from two influential domestic rating agencies in China, SynTao Green Finance and SINO Securities, against the occurrence and impact of negative ESG events both retrospectively and prospectively.

The examination of the effectiveness of ESG ratings in this study involves two perspectives. First, it assesses whether Chinese ESG ratings can accurately reflect the past ESG performance of focal companies, in other words, whether current ESG ratings can accurately reveal the ESG risks of enterprises in the past. Another critical dimension of effectiveness is the predictive validity, which assesses whether ESG ratings predict the future ESG performance of the enterprise based on the current status of company management. This research makes two contributions. First, the prior literature extensively uses ESG ratings to proxy the firms’ sustainability performance and studies their influence on corporate innovation and financial performance. However, there is no guarantee that the ratings can accurately reflect the firms’ performance. This study delves into the effectiveness of ESG ratings in the Chinese market, expanding the scope of the ESG ratings literature. Second, this study demonstrates that Chinese ESG ratings provided by domestic agencies have attained a certain level of effectiveness and can provide distinctive information for policymakers and business managers.

The remaining part of the paper proceeds as follows. Section 2 reviews previous research. Section 3 presents the research hypotheses. Section 4 introduces data sources and variable definitions and describes the methods. Section 5 presents the results. Section 6 conducts robustness tests. Section 7 discusses policy recommendations, research limitations, and future research directions.

2. Literature Review

2.1. The Current Development Status of ESG Ratings

Firms’ ESG ratings are extensively employed to measure their sustainability and social responsibility performance and constitute the cornerstone of a wide range of research efforts, including but not limited to the relationship between sustainability and financial performance [22], governance practices and sustainability [23], and sustainability and innovation [24]. ESG ratings reflect the ESG performance and ESG disclosure level of a company or financial product. The results of the ESG ratings are intended to support investors in their decision-making and operational strategies [2,25]. Investors can use these ratings to understand and identify the ESG performance of a company and as a reference to assess or construct their portfolios, thereby helping to alleviate information asymmetry between investors and firms [26,27]. The results of ESG ratings rely heavily on firms’ disclosed ESG information. Rating agencies define ESG elements according to their own understanding and assign different weights to the relevant indicators based on their judgment of the importance in order to construct their ESG rating system. This system is used to measure and assess the collected ESG information.

The current ESG rating systems of mainstream international rating agencies have the following characteristics: first, the assessments all involve sustainability and risk management [28]. Second, the indicator system contains both qualitative and quantitative indicators. Third, the ESG data cited by them mainly comes from the information voluntarily disclosed by enterprises [29]. Fourth, it emphasizes collaborating with third parties to validate its ratings for businesses or financial products [30]. Fifth, major international ESG index rating companies have expanded their influence by launching ESG index products to promote the harmonization of ESG rating standards. Currently, most scholarly studies on ESG ratings focus on their correlation with ESG performance. These studies have concluded that a company or financial product with better ESG performance will receive a higher ESG score. Although many scholars’ studies emphasize the positive performance and impact of ESG, in reality, investors commonly apply negative screening strategies when focusing on ESG ratings. This means that investors tend to prioritize the ESG risks of investment targets.

The development of the ESG rating system in China started relatively late. At this stage, local ESG rating agencies in China primarily evaluate the disclosure and regulation of corporate ESG information based on China’s national conditions. They continuously update and adjust the rating standards using this information [31]. This means that there is no uniform mandatory evaluation standard for ESG rating agencies in China [32]. For example, two prominent ESG rating agencies in China, Sino-Securities Index Information Service (Shanghai, China) Co., Ltd. (hereafter CSI) and SynTao Green Finance Co., Ltd. (Beijing, China) (hereafter SynTao) assess the environmental, social, and governance dimensions [33,34]. CSI’s ESG rating system includes more than 130 sub-indicators, while SynTao’s ESG rating system covers more than 200 sub-indicators. Due to variations in the evaluation criteria used by different ESG rating agencies in China, it is possible for the ESG scores of the same listed company to yield completely different rating results [35]. Therefore, as they explore the establishment of local rating standards, Chinese ESG rating agencies are also dedicated to aligning with international ESG rating systems to ensure that their ratings accurately and effectively reflect the ESG performance of companies [36]. Current research on ESG ratings of Chinese firms primarily focuses on the influence of ESG ratings on firms’ performance, encompassing innovation performance, financial performance, green transformation efficiency, and the quality of corporate earnings [20,37].

2.2. The Validity of ESG Ratings

As researchers increasingly rely on ESG ratings as a proxy for corporate sustainability, it is essential to assess the extent to which ESG ratings accurately measure the sustainability performance of firms. A conspicuous phenomenon noticed by researchers in recent years is the striking discrepancy of different ESG ratings. Semenova and Hassel (2015) found that on the aggregate environmental metric, different ESG ratings do not converge. However, a high correlation can be observed for specific environmental dimensions [38]. Chatterji et al. (2016) show that there is a lack of consensus among six leading ESG raters (KLD, Asset4, DJSI, Innovest, FTSE4Good, Calvert) by analyzing the pairwise correlations for a common universe of firms, and further argues that the ESG ratings have low validity. Chatterji (2016) also argues that if there are significant differences in the ratings provided by ESG rating agencies and they are in a state of low consistency. In this case, these ratings provided by these agencies fail to address the issue of information asymmetry between investors and enterprises [13]. In other words, they do not meet the investors’ need to utilize the rating results to comprehend the ESG performance of enterprises, indicating a lack of validity in their rating results. Divergence of ESG ratings is a rather common phenomenon and also observed in countries other than the United States [39].

Another strand of the literature directly studies the effectiveness of the ESG ratings in capturing the ESG performance of the firms and aligning with the real-world impacts of corporate behavior on the ESG practices. The studies in this domain typically benchmark ESG rating against objective real-world metrics such as sustainability violations and investments. Chatterji et al. (2009) found that the widely used KLD rating can capture firms’ past environmental performances fairly well and can also predict future environmental compliance violations [14]. Kang (2015) examines the effectiveness of the KLD rating in diversity and governance, and discovered that the rating is effective in summarizing past performance and predicting future performance in both dimensions [40].

After analyzing the leading international ESG rating agencies, researchers assert that discrepancies in ESG ratings are primarily attributed to the divergence of agencies on the concept of ESG, the transparency of ESG disclosure, the complexity of ESG data measurement, and the geographic locations and industries of the evaluated company [12,41]. Dimson et al. (2020) examines the degree of inconsistency among leading ESG raters and contends that one reason of disagreement is that the raters assign different weights to each pillar of ESG [42]. Lin and Cui (2023) believe that for domestic ESG rating agencies in China, in addition to the reasons mentioned above, the speed at which ESG rating agencies receive negative news from companies and update their rating results can also lead to differences in ESG rating results [43]. Some papers also analyze the reasons behind the divergence of ESG ratings from the perspectives such as weighting and measurement construct [12,44]. Overall, the discrepancy of ESG ratings calls into question the effectiveness of the rating to accurately reflect the sustainability performance of the firms [13].

In response to low consistency in ESG ratings, scholars can reduce its impact by analyzing the results of multiple ESG rating agencies simultaneously [45], selecting a specific ESG rating for research, and building hypotheses around specific ESG rating sub-indicators (such as greenhouse gas emissions or environmental governance). For ESG rating agencies, enhancing the clarity of ESG performance definitions and the transparency of measurement methodologies can be utilized to improve the low consistency of rating results. Regarding rating investors, they can reallocate the data range and weight according to their own needs, based on the indicators of multiple rating agencies, in order to reduce the differences between rating results. In addition, investors should also consider factors such as the industry type and the country’s environment when making investment decisions based on ESG ratings [12,41]. Mobius and Ali (2021) investigated the efficacy of ESG ratings in emerging markets and concluded that companies that excel in the environmental, social, and governance dimensions simultaneously do not necessarily have greater growth potential. Investors in emerging markets should prioritize identifying the core dimension of a company’s value, which means that being able to discern the importance of ESG metrics is also one of the key factors in reducing the misleading nature of ESG ratings [46].

This paper is also related to studies of ESG ratings in China, a fast-growing market for ESG investing. Studies in this domain have explored various research topics similar to the grand research genre [47,48,49]. In addition, similar to other markets, ESG rating disagreement has also be observed for samples of Chinese firms by recent studies, and it has been found to negatively impact stock returns [50,51,52,53,54]. While studies on ESG ratings in China abound, no research has examined the effectiveness of the ESG ratings for Chinese firms.

Since the concept of ESG started late in China, domestic scholars usually study the environment, society, and governance as three independent dimensions, and seldom discuss their impact on enterprises from a holistic perspective. At the same time, because the current ESG data base in China is weak and the time span is insufficient, the ESG rating data that can be accessed is relatively limited, while the sample data in this paper covers multiple dimensions such as enterprise, industry, macro, etc., and the data are more complete. Taking into account the current actual situation in China and based on the analysis of the previous literature, this paper concludes that the following gaps exist in the existing research. First, existing studies of rating effectiveness focus on established ratings for firms in developed countries. Second, the literature on effectiveness usually focuses on effectiveness of specific dimensions. This study aims to fill these gaps.

3. Hypothesis Construction

Stakeholder theory emphasizes that while pursuing profits, enterprises should also consider the impact on all stakeholders in order to achieve sustainable development and social responsibility [55]. In such a context, there may be a short-term game between corporate stakeholders and ESG information demanders, i.e., under the framework of stakeholder power asymmetry, corporations tend to prioritize meeting the needs of stakeholders with immediate bargaining power, and they adopt strategic disclosures or policies in order to achieve short-term improvements in ESG rating scores, rather than formulating a long-term plan to address ESG issues at their root causes. Such symbolic compliance behavior will lead to a systematic deviation from actual ESG performance, creating a “greenwash” trap [56]. As the current ESG rating system relies heavily on non-financial data disclosed by companies themselves and suffers from a methodological lag, early highly rated companies may be exposed to historical performance deficiencies in dynamically evolving ESG standards or third-party data validation [57]. In addition, the instrumental use of ESG ratings in the capital market exacerbates the agency problem, whereby firms’ rating gaming behaviors to obtain financing facilities often come at the expense of deep ESG improvements such as supply chain management. Under the combined effect of decisions made by firms in pursuit of short-term rating incentives and the failure of stakeholder monitoring mechanisms, high historical ESG scores have instead resulted in a lack of substantive corporate responsibility. Therefore, this paper proposes the following hypothesis:

H1. 

There is a significant negative correlation between firms’ past ESG performances and their ESG rating scores.

Based on signaling theory, a higher ESG score can serve as an effective quality signal [58], which not only reflects that firms have followed the legitimacy requirements of social norms [59], but also pushes back the optimization of the internal governance mechanism through institutional commitment. First, ESG-leading firms usually establish a more comprehensive compliance risk control system, which can effectively reduce the probability of environmental infringements, labor disputes, and other illegal events; second, the accumulated reputational capital can have a deterrent effect, so that potential plaintiffs are inclined to settle or reduce their claims in the trade-off between the costs and benefits of litigation [60]. Existing research suggests that firms with superior ESG performance will be involved in lower amounts of environmental administrative penalties, product liability lawsuits, and other negative consequences than other firms in the same industry [61] and that this risk-mitigating effect stems from the fact that the preventive governance mechanisms of ESG practices can enhance the marginal costs incurred as a result of the offending behavior. Thus, ESG ratings can reduce the expenditures incurred by firms due to litigation by enhancing their social legitimacy.

Based on the resource-based view and institutional theory, firms with higher ESG scores can build up scarce legitimacy resources through institutionalized practices in the environmental, social, and governance dimensions [62], which not only reduces the risk of triggering judicial interventions due to non-compliance, but also strengthens the firms’ abilities to withstand and deal with controversial events. When controversial events such as lawsuits arise, such firms are more inclined to internalize and deal with conflicts through non-litigation mechanisms such as mediation and social responsibility compensation [59], thus reducing the generation of adjudicative instruments. Existing research shows that the number of litigation decisions by ESG-rated leading firms is much lower than that of other firms in the industry, and the rate of case dismissal is also higher, which suggests that ESG practices can reduce the actual impacts of illegal events by enhancing organizational resilience and social trust.

According to stakeholder theory, companies with high ESG scores usually incorporate stakeholders’ needs into their daily operational governance considerations [55] and front-load the identification and handling of negative events that may cause disputes, such as environmental infringement and labor discrimination [63]. At the same time, ESG-leading companies tend to establish a multi-dimensional risk warning system to intervene in potential conflicts in advance before they evolve into legal events, minimizing the probability of corporate involvement. Therefore, ESG management can systematically compress the living space of illegal behaviors and prevent potential risks from turning into actual legal cases by restructuring the organizational decision-making framework. In summary, this paper proposes the following hypotheses:

H2a. 

There is a significant negative correlation between ESG rating scores and the amount of money involved in enterprises as defendants in the future.

H2b. 

There is a significant negative correlation between ESG rating scores and the number of judgment documents involved in enterprises as a defendant in the future.

H2c. 

There is a significant negative correlation between ESG rating scores and the number of all cases involved in enterprises in the future.

4. Method and Data

4.1. Data and Variables

The sample in this study consists of China Securities Index 300 (CSI 300) constituent firms for the period 2016–2020. Due to the limited availability of ESG rating for smaller firms in China, this study focuses on the larger and more influential CSI 300 companies to ensure an appropriate sample. This paper further screens the sample according to the following steps. First, this study excludes companies with ST (special treatment, which refers to listed companies with negative net profit for two consecutive fiscal years) or ST* (special treatment*, which represents delisting warning due to loss of listed companies for three consecutive fiscal years). Second, this article selects companies that have been rated by two prominent ESG rating agencies in China, namely SynTao Green Finance and SINO Securities, at the same time throughout the entire sample period. Third, the paper removes data points with missing values. In the end, the study collects a sample of 1494 annual observations for 300 companies. In addition, to mitigate the impact of outliers, this study performs winsorization on all continuous variables at the upper and lower 1% levels.

4.1.1. ESG Ratings

This study extracts the ESG ratings provided by two domestic rating agencies, SynTao Green Finance and SINO Securities, from the WIND database. Both agencies rate ESG performance via a similar framework. Take SynTao Green Finance as example. First, it computes the ESG management scores and the ESG risk exposure scores independently. It then combines these scores to calculate an overall score that assesses the company’s ESG performance. It is worth noting that SynTao Green Finance’s ESG rating system places significant emphasis on the adverse impact of negative events. As this study focuses primarily on the negative performance of companies, this study has selected the SynTao Green Finance ESG risk exposure scores as the research variable, with each company’s score ranging from 0 to 100 points.

4.1.2. ESG Performance

Chatterji et al. (2009) measured firms’ environmental performances in terms of the amount of penalties and the number of penalties imposed on firms for violating environmental regulations to explore the impact of social ratings on CSR [14]. Dyck et al. (2019) argued that the number of judgment documents associated with a firm can be used to measure ESG performance [64]. Bhagat et al. (2015) use the number of litigation cases (e.g., fraud, mis-selling) of financial institutions as a measure of their governance performance to examine the risks that can be posed by negative ESG events [65].

Based on the above literature, and in conjunction with the phenomenon that investors commonly adopt negative screening strategies to avoid ESG risks in ESG investment practices, this paper refers to the research of Shen et al. (2022) and selects negative events of enterprises as the key variable for measuring the ESG performance of enterprises [66]. Specifically speaking, this paper utilizes a dataset of negative events as indicators to assess ESG performance and benchmark the effectiveness of the ratings. The dataset includes: (I) the amount of money involved as a defendant, (II) the proportion of money involved as a defendant, (III) the number of judgment documents involved as a defendant, (IV) the proportion of judgment documents involved as a defendant, and (V) the number of all cases involved. To ensure the objectivity and authenticity of the data, this article manually collects data disclosed by government bodies, such as the Ministry of Justice and the Ministry of Ecology and the Environment, through official portals.

4.1.3. Control Variables

The existing literature typically collects data on firms’ industry and financial metrics, including industry code, revenue, total assets, and net income as control variables to examine issues related to the effectiveness of ESG ratings [14,23,67,68]. Based on a summary of the previous literature and combining the data that could be collected, this study selects firm size, income, year, and industry dummies as control variables. Specifically, firm size is measured using the natural logarithm of total assets. Income is measured as the natural logarithm of total revenue. The data are extracted from the China Stock Market and Accounting Research (CSMAR) database.

Table 1. Definition of variables.

Variables	Definition	Data Source
ESG rating	SynTao Green Finance ESG risk exposure scores ranging from 0 to 100, SINO Securities	WIND
ESG performance	Amount of money involved as a defendant	Government disclosure
	Proportion of money involved as a defendant
	Number of judgment documents involved as a defendant
	Proportion of judgment documents involved as a defendant
	Number of all cases involved
Controls	The logarithm of total assets	CSMAR
	The logarithm of total income
	Year dummies
	Industry dummies

presents a comprehensive description of the variables employed in this study.

4.2. Method

Chatterji et al. (2009) verified the effectiveness of social ratings in reflecting CSR by constructing an OLS regression model. Specifically, they constructed models that verify the following questions: First, to what extent environmental ratings reflect the past environmental performance of firms. Second, whether environmental ratings can predict the future environmental performance of enterprises [14]. This article argues that for the ESG ratings of Chinese domestic companies to be effective, they must meet the following criteria. First, ESG ratings should accurately reflect the past ESG performance of local companies, and second, they should be capable of predicting future ESG performance based on the current investment activities and management decisions of the companies. Referring to Chatterji et al.’s study, this section will construct two regression models to evaluate the retrospective and predictive effectiveness of China’s domestic ESG ratings.

4.2.1. Retrospective Effectiveness

First, this paper uses the OLS regression model (1) to examine the extent to which ESG ratings reflect a company’s past ESG performance, i.e., retrospective effectiveness to verify Hypothesis 1. The model is as follows:

$${\mathrm{ESG}\mathrm{Rating}}_{\mathrm{i},\mathrm{t}}={\mathsf{\alpha }}_0+{\mathsf{\beta }}_1{\mathrm{ESG}\mathrm{Performance}}_{\mathrm{i},\mathrm{t}-1}+{\mathsf{\beta }}_2{\mathrm{Controls}}_{\mathrm{i},\mathrm{t}-1}+{\mathsf{\epsilon }}_{\mathrm{i},\mathrm{t}}$$

(1)

In (1), ${\mathrm{ESG}\mathrm{Rating}}_{\mathrm{i},\mathrm{t}}$ represents the ESG ratings of company i in year t. ${\mathrm{ESG}\mathrm{Performance}}_{\mathrm{i},\mathrm{t}-1}$ is the negative performance of company i, which will be measured through 5 proxy variables: (I) Amount of money involved as a defendant, (II) Proportion of money involved as a defendant, (III) Number of judgment documents involved as a defendant, (IV) Proportion of judgment documents involved as a defendant, and (V) Number of all cases involved. ${\mathrm{Controls}}_{\mathrm{i},\mathrm{t}-1}$ are control variables, including year dummy variables, industry dummy variables, and company size (logarithm of assets and logarithm of income). Considering that the construction of model (1) aims to estimate whether the ESG ratings accurately reflects the focal company’s past ESG performance, this study uses the values from the year before SynTao Green Finance’s release of the ESG ratings to represent the other variables in the model. If the regression coefficient ${\mathsf{\beta }}_1$ is negative and statistically significant, then it is considered that the ESG ratings effectively reflect the company’s past ESG performance.

4.2.2. Predictive Effectiveness

Next, this study uses regression model (2) to assess the effectiveness of ESG ratings in predicting future ESG performance of companies, which is Hypothesis 2 of this paper. The model constructed in this paper is as follows:

$${\mathrm{ESG}\mathrm{Performance}}_{\mathrm{i},\mathrm{t}}={\mathsf{\alpha }}_1+{\mathsf{\beta }}_3{\mathrm{ESG}\mathrm{Rating}}_{\mathrm{i},\mathrm{t}-1}+{\mathsf{\beta }}_4{\mathrm{Controls}}_{\mathrm{i},\mathrm{t}-1}+{\mathsf{\epsilon }}_{\mathrm{i},\mathrm{t}}$$

(2)

In the above text, ${\mathrm{ESG}\mathrm{Performance}}_{\mathrm{i},\mathrm{t}}$ represents the ESG performance of enterprise i in year t, measured by the five proxy variables mentioned earlier; ${\mathrm{ESG}\mathrm{Rating}}_{\mathrm{i},\mathrm{t}-1}$ represents the rating of SynTao Green Finance for enterprise i in year t − 1. Since the construction of model (2) aims to validate the effectiveness of ESG ratings in predicting future ESG performance of enterprises, this research uses the ESG performance values from the previous year to represent the other variables in the model. If the regression coefficient of model (2) is negative and statistically significant, then it is considered that the ESG ratings can effectively predict the future ESG performance of enterprises.

5. Analysis and Results

5.1. Descriptive Statistics

The descriptive statistics of the main research variables in this paper are shown in

Table 2. Descriptive statistics.

Variables	N	Mean	Std. Dev	Min	Max
ESG Rating	1494	88.968	8.102	60.94	100
Log(+1) amount of money involved as a defendant	1494	9.302	3.487	9.210	18.329
Proportion of money involved as a defendant	1494	0.205	0.373	0	1
Number of judgment documents involved as a defendant	1494	11.459	22.847	0	143
Proportion of judgment documents involved as a defendant	1494	0.376	0.334	0	1
Number of all cases involved	1494	63.790	210.343	0	1616
Log Income	1494	23.925	1.556	20.621	27.684
Log Assets	1494	25.118	1.769	22.142	30.688

. Among them, the mean value of the ESG score is 88.968 and the standard deviation is 8.102, indicating that although the ESG performance of each enterprise is basically at the medium to high level, there are significant differences in their ESG performance. Additionally, the average values of Proportion of money involved as a defendant and Proportion of judgment documents involved as a defendant are 20.5% and 37.6%. These two pieces of data indicate that, for most of the sample companies selected for this study, the amount of money and the number of judgement documents in which they are involved as defendants are at a low level. This suggests that they are less exposed to legal proceedings compared to other companies. The range of the remaining relevant control variables is large, indicating that the control variables have a significant impact on the explained variables. For specific results refer to Table 2.

5.2. Pearson Correlation Analysis

To examine the correlation between variables, this study conducted Pearson correlation coefficient tests on all selected variables, and the test results are shown in

Table 3. Pearson correlation matrix.

	ESG Rating	Log(+1) Amount of Money Involved as a Defendant	Proportion of Money Involved as a Defendant	Number of Judgment Documents Involved as a Defendant	Proportion of Judgment Documents Involved as a Defendant	Number of All Cases Involved	Log Income	Log Assets
ESG Rating	1
Log(+1) amount of money involved as a defendant	−0.289 ***	1
Proportion of money involved as a defendant	−0.208 ***	0.673 ***	1
Number of judgment documents involved as a defendant	−0.194 ***	0.412 ***	0.166 ***	1
Proportion of judgment documents involved as a defendant	−0.110 ***	0.179 ***	0.254 ***	0.212 ***	1
Number of all cases involved	−0.090 ***	0.164 ***	−0.0280	0.464 ***	−0.165 ***	1
Log Income	−0.581 ***	0.288 ***	0.186 ***	0.169 ***	0.125 ***	0.112 ***	1
Log Assets	−0.508 ***	0.255 ***	0.138 ***	0.209 ***	0.067 ***	0.240 ***	0.736 ***	1

Note: *** p < 0.01.

. From Table 3, it is evident that the correlation coefficients between ESG ratings and Amount of money involved as a defendant, Proportion of money involved as a defendant, Number of judgment documents involved as a defendant, Proportion of judgment documents involved as a defendant, and Number of all cases involved, are −0.289, −0.208, −0.194, −0.110, and −0.090, respectively. All of these coefficients show a negative correlation at the 1% significance level, which provides preliminary evidence of the negative correlation between ESG ratings and ESG performance. According to the data in the table, the control variables Log Income and Log Assets both show significant negative correlations with ESG ratings. This suggests that larger companies are more likely to receive higher ESG ratings compared to smaller enterprises. Additionally, the correlation coefficients between the main variables are all less than 0.7, and the variance inflation factors (VIF) in

Table 4. Collinearity diagnostics.

Variable	Collinearity Statistics
	VIF	1/VIF
Log(+1) amount of money involved as a defendant	2.30	0.435
Proportion of money involved as a defendant	1.98	0.504
Number of judgment documents involved as a defendant	1.65	0.605
Proportion of judgment documents involved as a defendant	1.22	0.822
Number of all cases involved	1.47	0.679
Log Income	2.28	0.439
Log Assets	2.31	0.432
Mean VIF	1.89

are all well below 10, indicating the absence of multicollinearity among the explanatory variables.

5.3. Regression Analysis

Based on the panel data of the selected sample firms for this study, a regression analysis of model (1) was conducted to examine the correlation between Chinese ESG ratings and the past ESG performance of local firms. The regression results are shown in

Table 5. Regression results of ESG ratings on past ESG performance and other determinants.

	M(1)	M(2)	M(3)	M(4)	M(5)
	ESG Rating	ESG Rating	ESG Rating	ESG Rating	ESG Rating
Log(+1) amount of money involved as a defendant, lagged 1 year	−0.349 *** (−3.77)
Proportion of money involved as a defendant, lagged 1 year		−1.824 ** (−3.16)
Number of judgment documents involved as a defendant, lagged 1 year			−0.032 ** (−3.27)
Proportion of judgment documents involved as a defendant, lagged 1 year				−1.237 * (−1.97)
Number of all cases involved, lagged 1 year					0.000 (0.01)
Log Income, lagged 1 year	−1.732 *** (−4.84)	−1.833 *** (−5.13)	−1.895 *** (−5.30)	−1.859 *** (−5.19)	−1.850 *** (−5.15)
Log Assets, lagged 1 year	−1.390 *** (−3.43)	−1.332 ** (−3.28)	−1.294 ** (−3.18)	−1.332 ** (−3.26)	−1.409 *** (−3.46)
Constant	168.9 *** (32.04)	166.6 *** (31.26)	167.1 *** (31.47)	167.5 *** (31.34)	168.8 *** (31.79)
Year FE	Yes	Yes	Yes	Yes	Yes
Industry FE	Yes	Yes	Yes	Yes	Yes
N	1004	1004	1004	1004	1004
R2	0.458	0.467	0.464	0.465	0.461

Note: * p < 0.1, ** p < 0.05, *** p < 0.01.

. Specifically, in column (1) of Table 5, the correlation coefficient of Amount of money involved as a defendant is −0.349, which is a significant negative correlation at the 1% level. In columns (2)–(3), the correlation coefficients of Proportion of money involved as a defendant and Number of judgment documents involved as a defendant are −1.824 and −0.032, respectively, with a significant negative correlation at the 5% level. The correlation coefficient of Proportion of judgment documents involved as a defendant in column (4) is −1.237, which is significantly negatively correlated at the 10% level. However, the correlation coefficient for Number of all cases involved in column (5) is neither negative nor statistically significant. The results of this paper show that overall, there is a significant negative correlation between Chinese ESG ratings and the past ESG performance of local firms. Since the ESG ratings framework consists of publicly disclosed specific indicators that measure the level of ESG risk, a company with fewer negative events and the lower its exposure to litigation, the higher its ESG rating will be.

Although there are some differences between the regression results of different explanatory variables, overall, there is a strong correlation between the rating results and the ESG performance of the firms in terms of negative events, accurately measuring the performance of the firms over the past 1 year in the four aspects of the amount of money involved as a defendant, the proportion of money involved as a defendant, the number of judgment documents involved as a defendant, and the proportion of judgment documents involved as a defendant. This result can also be reflected by typical corporate rating events in the capital market, e.g., TongRenTang triggered a series of negative ESG effects in December 2018 due to quality and safety issues, while its ESG performance is basically in line with the C+ rating given by SynTao Green Finance in June 2018, and its ESG score is much lower than the industry standard. KangDe Xin was forced to be delisted for four consecutive years of financial fraud from 2015–2018, and in 2017 and 2018, SynTao Green Finance had already given low ratings of B- and C, which is also basically in line with the actual performance of the enterprise. In summary, it can be seen that Chinese ESG ratings can effectively capture the risks that exist in enterprises and reflect them through the rating data, which verifies the hypothesis H1 of this paper.

This study employs model (2) to examine the effectiveness of China’s domestic ESG ratings in predicting the future ESG performance of firms. The regression results are displayed in

Table 6. Regression results of future ESG performance on the ESG rating and other determinants.

	M(1)	M(2)	M(3)	M(4)	M(5)
	Log(+1) Amount of Money Involved as a Defendant	Proportion of Money Involved as a Defendant	Number of Judgment Documents Involved as a Defendant	Proportion of Judgment Documents Involved as a Defendant	Number of All Cases Involved
ESG Rating, lagged 1 year	−0.0374 ** (−2.96)	−0.0230 (−1.82)	−0.0569 *** (−7.51)	−0.0242 (−1.60)	−0.0471 *** (−5.93)
Log Income, lagged 1 year	0.130 (0.93)	0.162 (1.11)	−0.106 (−1.13)	0.172 (1.14)	−0.178 (−1.83)
Log Assets, lagged 1 year	0.211 (1.35)	0.221 (1.38)	0.259 ** (2.59)	0.329 * (1.87)	0.263 * (2.50)
Constant	5.125 * (1.76)	−9.076 ** (−3.06)	2.054 (1.15)	−9.896 ** (−2.85)	3.148 * (1.70)
Year FE	Yes	Yes	Yes	Yes	Yes
Industry FE	Yes	Yes	Yes	Yes	Yes
N	1004	963	1004	857	1004
R2	0.271	0.174	0.077	0.192	0.097

Note: * p < 0.1, ** p < 0.05, *** p < 0.01.

. Specifically, in column (1) of Table 6, the correlation coefficient of Amount of money involved as a defendant is −0.0374 and is significant negative correlation at the 5% level. This suggests that for every one standard deviation unit increase in a firm’s ESG ratings, the amount of money in which a firm is involved as a defendant in the future will be reduced by approximately 1.87% (−0.0374 × 0.0126/0.0252 = 0.0187). The correlation coefficients in columns (3) and (5) of Table 6 are −0.0569 and 0.0471, respectively. They are significantly negatively correlated at the 1% level, which implies that for every one-point increase in the ESG scores, the number of judgment documents in which the firm is involved as a defendant in the future will be reduced by 5.73% (1−e^−0.0569 = 0.0573), and the number of all the firm’s involvement in the case will be reduced by 4.6%. The correlation coefficients of columns (2) and (4) in Table 6 are −0.0230 and −0.0242, respectively. However, the regression results are not significant. The regression results in Table 6 indicate that Chinese ESG ratings can predict the future ESG performance of local firms to a certain extent. Specifically, they can predict the amount of money involved as a defendant, the number of judgment documents involved as a defendant in the future, and the number of firms’ all-involved events. However, they cannot accurately predict the proportion of money involved as a defendant and the proportion of judgment documents involved as a defendant in the following year. Hypothesis 2 proposed in this paper has been verified.

The reason for the above results may be due to the fact that the percentage of the number of defendants in the coming year is affected by other factors such as the number of defendant and plaintiff cases. Among them, the number of plaintiff cases is largely influenced by a combination of subjective factors such as the willingness of the company’s board of directors and the company’s level of inclusiveness, whereas most of the rating agency indicators are objective, so the information asymmetry between the rating agency and the company will reduce the correlation between the indicator and the score, which will directly or indirectly affect the strength of the rating’s prediction.

The existing literature on ESG validity mainly focuses on the environmental dimension, and current scholars’ research indicates that environmental scores can better reflect the past environmental performance of firms, i.e., environmental scores are better at achieving retrospective effectiveness, but are slightly less effective at predictive effectiveness [14]. The studies in the diversity and governance dimensions, on the other hand, show that the relevant scores achieve both retrospective and predictive effectiveness [40]. This paper obtains similar conclusions after extending the research on the effectiveness of scores to the overall ESG level. The results of this paper show that the rating systems established by Chinese domestic ESG organizations are able to accurately capture the risks that firms have been exposed to and give accurate rating scores. However, rating agencies have not yet constructed a more scientific and comprehensive methodology to measure the future ESG performance of firms with respect to the potential risks they face.

6. Robustness Check

6.1. Replace Variables

In order to ensure the robustness of the empirical results, this paper conducts a robustness test by altering the sample. In this study, the ESG rating data of the selected samples from SynTao Green Finance were replaced with the rating data from SINO Securities for the purpose of conducting a robustness test and re-estimating the regression model. The results of the regression are presented in

Table 7. Robustness checks of model (1) using SINO Securities’ ESG ratings.

	M(1)	M(2)	M(3)	M(4)	M(5)
	ESG Rating	ESG Rating	ESG Rating	ESG Rating	ESG Rating
Log(+1) amount of money involved as a defendant, lagged 1 year	−0.031 * (−2.20)
Proportion of money involved as a defendant, lagged 1 year		−0.662 * (−2.01)
Number of judgment documents involved as a defendant, lagged 1 year			−0.004 * (−2.29)
Proportion of judgment documents involved as a defendant, lagged 1 year				−0.087 (−0.84)
Number of all cases involved, lagged 1 year					−0.0001 (−0.73)
Log Income, lagged 1 year	0.073 (1.30)	0.067 (1.18)	0.067 (1.18)	0.070 (1.23)	0.069 (1.21)
Log Assets, lagged 1 year	0.120 * (1.86)	0.123 * (1.90)	0.123 * (1.91)	0.116 * (1.80)	0.113 * (1.75)
Constant	−0.003 (−0.00)	−0.208 (−0.25)	−0.224 (−0.27)	−0.083 (−0.10)	−0.010 (−0.01)
Year FE	Yes	Yes	Yes	Yes	Yes
Industry FE	Yes	Yes	Yes	Yes	Yes
N	995	995	995	995	993
R2	0.239	0.237	0.239	0.235	0.235

Note: * p < 0.1.

and

Table 8. Robustness checks of model (2) using SINO Securities’ ESG ratings.

	M(1)	M(2)	M(3)	M(4)	M(5)
	Log(+1) Amount of Money Involved as a Defendant	Proportion of Money Involved as a Defendant	Number of Judgment Documents Involved as a Defendant	Proportion of Judgment Documents Involved as a Defendant	Number of All Cases Involved
ESG Rating, lagged 1 year	−0.013 (−0.15)	0.138 (1.71)	−0.149 ** (−3.11)	0.200 * (2.05)	−0.110 * (−2.18)
Log Income, lagged 1 year	0.139 (0.97)	0.041 (0.29)	0.067 (0.69)	0.179 (1.18)	−0.051 (−0.50)
Log Assets, lagged 1 year	0.333* (2.04)	0.382 * (2.36)	0.292 ** (2.73)	0.357 * (2.02)	0.289 ** (2.59)
Constant	−1.162 (−0.55)	−13.14 *** (−6.16)	−7.344 *** (−5.42)	−13.78 *** (−5.54)	−4.246 ** (−3.09)
Year FE	Yes	Yes	Yes	Yes	Yes
Industry FE	Yes	Yes	Yes	Yes	Yes
N	995	950	995	854	993
R2	0.257	0.171	0.072	0.197	0.094

Note: * p < 0.1, ** p < 0.05, *** p < 0.01.

. According to the data in the tables, it is evident that the results of the robustness test are similar to the results of the main model of this study. The direction of the coefficients of the variables involved has not changed, which confirms the robustness of the conclusions of this research and the reliability of the results. This provides strong evidence for testing the validity of the local ESG ratings in China.

6.2. Increase Control Variables

In this paper, ROI, ROA, and ROE were also added as control variables to test the robustness of the regression results. As shown in

Table 9. Robustness test with the addition of control variables for model (1).

	M(1)	M(2)	M(3)	M(4)	M(5)
	ESG Rating	ESG Rating	ESG Rating	ESG Rating	ESG Rating
Log(+1) amount of money involved as a defendant, lagged 1 year	−0.162 ** (0.079)
Proportion of money involved as a defendant, lagged 1 year		−0.864 * (0.49)
Number of judgment documents involved as a defendant, lagged 1 year			−0.003 * (0.00)
Proportion of judgment documents involved as a defendant, lagged 1 year				−0.415 (0.50)
Number of all cases involved, lagged 1 year					0.000 (0.00)
Log Income, lagged 1 year	−0.211	−0.070	−0.123	−0.008	−0.032
	(1.79)	(1.79)	(1.79)	(1.78)	(1.78)
Log Assets, lagged 1 year	−7.228 *	−7.168 *	−7.719 **	−7.473 *	−7.521 *
	(3.79)	(3.79)	(3.86)	(3.84)	(3.86)
L.ROI, lagged 1 year	−0.000 **	−0.000 **	−0.000 **	−0.000 *	−0.000 ***
	(0.00)	(0.00)	(0.00)	(0.00)	(0.00)
L.ROA, lagged 1 year	0.439	0.468	0.559	0.542	0.553
	(0.37)	(0.37)	(0.40)	(0.38)	(0.40)
L.ROE, lagged 1 year	−3.160	−3.523	−3.371	−3.780	−3.558
	(5.57)	(5.54)	(5.56)	(5.56)	(5.57)
Constant	169.6 ***	167.4 ***	173.8 ***	170.1 ***	170.7 ***
	(41.20)	(41.07)	(41.75)	(41.54)	(41.84)
Year FE	Yes	Yes	Yes	Yes	Yes
Industry FE	Yes	Yes	Yes	Yes	Yes
N	794	794	794	794	794
R2	0.049	0.047	0.043	0.043	0.041

Note: * p < 0.1, ** p < 0.05, *** p < 0.01.

and

Table 10. Robustness test with the addition of control variables for model (2).

	M(1)	M(2)	M(3)	M(4)	M(5)
	Log(+1) Amount of Money Involved as a Defendant	Proportion of Money Involved as a Defendant	Number of Judgment Documents Involved as a Defendant	Proportion of Judgment Documents Involved as a Defendant	Number of All Cases Involved
ESG Rating, lagged 1 year	−0.054 ** (0.03)	−0.005 (0.00)	−0.417 ** (0.92)	−0.000 (0.00)	−2.381 * (3.03)
Log Income, lagged 1 year	1.435 (1.47)	0.031 (0.20)	−52.520 (−45.90)	0.086 (0.16)	−110.000 (−152.70)
Log Assets, lagged 1 year	2.648 (1.82)	0.580 ** (0.27)	30.410 (31.25)	0.312 (0.26)	−236.600 (214.70)
L.ROI, lagged 1 year	0.000 (0.00)	0.000 (0.00)	0.000 (0.00)	0.000 (0.00)	0.000 (0.00)
L.ROA, lagged 1 year	0.176 (0.20)	0.025 (0.03)	1.571 (1.55)	0.100 *** (0.03)	20.790 (16.11)
L.ROE, lagged 1 year	−0.681 (3.36)	−0.180 (0.48)	25.960 (54.91)	−0.775 (0.50)	416.500* (229.30)
Constant	17.100 (16.24)	−6.251 *** (2.27)	182.800 (184.50)	−3.807 (2.34)	3514 *** (1.32)
Year FE	Yes	Yes	Yes	Yes	Yes
Industry FE	Yes	Yes	Yes	Yes	Yes
N	794	794	794	794	794
R2	0.138	0.044	0.021	0.029	0.052

Note: * p < 0.1, ** p < 0.05, *** p < 0.01.

, the significance and direction of the coefficients of the core variables after the addition of the control variables are basically consistent with the results of models 1 and 2, which further confirms the robustness of the findings (The added control variables can be referred to in the Supplementary Materials).

7. Discussion and Conclusions

7.1. Discussion

The rise of ESG investing has led to an increasing demand for ESG information from market participants. As investors are unable to comprehensively and specifically understand the ESG information of companies, ESG ratings have gradually become an important reference basis for evaluating their ESG performance and a cornerstone of ESG investing. ESG rating agencies first define the concept of ESG factors, and then assign different weights to the indicators based on their assessments of the importance of each indicator. After completing the construction of the ESG rating system following the aforementioned steps, the agencies will compile and assess the gathered ESG information based on its rating system, ultimately yielding the ESG rating results for the enterprise. Due to the subjective nature of human judgment in the ESG rating agencies’ process of defining concepts and determining the importance of indicators, as well as each agency’s varying capabilities in collecting ESG information, there may be significant disparities in the rating results among ESG rating agencies. This variance has sparked public discussion about the validity of ESG ratings. If the current effectiveness of ESG ratings is limited or invalid, it may lead to a mismatch of investor funds. It also introduces a lot of uncertainty into academic research based on ESG ratings at the current stage. Therefore, it is crucial to assess the validity of ESG ratings, both theoretically and in practical application.

Since the development of ESG in China is still in its early stages, research on the effectiveness of ESG ratings for the Chinese market is limited. This study offers a potential avenue for future research by utilizing an Ordinary Least Squares (OLS) model to examine the correlation between the negative ESG performance of CSI 300 constituent companies and the rating data from Chinese ESG rating agencies for the period of 2016–2020 in order to assess the validity of ESG ratings in the Chinese market. This paper considered five factors reflecting the negative performance of the enterprise: (I) the amount of money involved as a defendant, (II) the proportion of money involved as a defendant, (III) the number of judgment documents involved as a defendant, (IV) the proportion of judgment documents involved as a defendant, and (V) the number of all cases involved. Correspondingly, there is a focus on researching and analyzing the ESG risk scores of enterprises.

This study has observed that ESG ratings in the Chinese market reflect the past ESG performance of local firms. In other words, the fewer negative events a firm is involved in, the lower the risk of litigation it faces and the higher its ESG rating scores. The results of this study show that ESG ratings in the Chinese market can effectively and accurately detect negative events happening in local businesses and utilize them as a basis for evaluating the firms’ ESG performances. However, there is a discrepancy in the ESG rating results provided by Chinese rating agencies when predicting the future ESG performance of local firms. The reason for this is that a company’s exposure to litigation risk is influenced by various factors, including the legal environment, industry dynamics, and the subjective willingness of managers. All of these factors will impact the company’s future ESG performance. Therefore, ESG ratings in the Chinese market can only partially predict the future ESG performance of local companies.

This study has the following limitations. First, this paper mainly focuses on the negative events of companies and their corresponding ESG risk scores, and there is a lack of research on the positive performance of companies. Meanwhile, this paper’s study does not break down the specific performance of the E, S, and G dimensions, so it is not possible to make specific recommendations for these three dimensions. Second, this paper only selects two representative ESG rating agencies in China, and the research sample only focuses on CSI 300 constituent companies. Due to the limitations of the research sample selection, it may have an impact on the final regression results.

7.2. Conclusions

Based on the theories of stakeholder theory, signaling theory, and legitimacy theory, this paper systematically demonstrates the effectiveness of the two current mainstream ESG rating systems in China. At the theoretical level, it reveals the degree of accuracy of ESG ratings in measuring firms’ past ESG performances and their predictive effect on future ESG performance, enriches the existing research on the effectiveness of ESG ratings, and provides support for further exploration of Chinese firms’ ESG investment strategies. At the practical level, this paper confirms whether ESG ratings can clearly reflect the actual ESG performance of firms, and serves as a guide for Chinese ESG organizations to adjust their rating systems to enhance their accuracy and usefulness.

The results obtained in this paper bear the following implications for business executives and government in China. First, because the development of ESG is still in its early stages in China and there is a lack of a more comprehensive ESG disclosure mechanism, the authorities should formulate ESG-related policies, make efforts to promote the establishment of their own ESG systems, and standardize ESG disclosure and rating criteria to enhance the transparency, comparability, interoperability, and applicability of ESG ratings. Secondly, since many enterprises still lack a deep understanding of ESG concepts, they should make an effort to incorporate ESG concepts into their strategy formulation and daily operation management. Furthermore, companies should actively disclose their ESG social responsibility reports to enable the public to monitor their ESG performance. Finally, the coverage of product ratings provided by Chinese ESG rating data providers is relatively limited, with most of them focusing solely on large Chinese listed companies. To improve the credibility of ESG ratings, providers should broaden the range of their ratings and extend the duration of rating coverage. At the same time, ESG ratings providers need to focus on standardizing and subdividing each criterion when rating companies. They should also enhance the diversity and precision of the criteria in order to improve the quality of ESG ratings.