Was Chinese “Double-First Class” Construction Policy Influential? Analysis Using Propensity Score Matching

Abstract

The “double first-class” construction policy is an educational policy in China for building the world first-class universities and disciplines. The research objective of this paper is to verify the effect of “double first-class” construction policy. The Propensity Score Matching–Difference in Differences (PSM–DID) method was employed to gauge the academic influence of the policy. PSM–DID is a metering method specifically used to analyze the effectiveness of the policy to determine whether the impact of the policy has significant statistical significance. The panel data, covering the years 2016–2020, was derived from 122 universities in China and the United States, indicating the experimental and control data, respectively. The results show that the “double first-class” construction policy promotes the international academic influence of Chinese universities, but this promotion effect is not sustainable.

1. Introduction

The “double first-class” construction policy is an educational policy in China for building the world first-class universities and disciplines. In 2015, the State Council of China promulgated the “Overall Plan for Coordinated Promotion of the initiative of World-Class Universities and First-Class Disciplines”. It proposed that universities which offer a number of high-level disciplines and maintain a dominant position in the Chinese education system, should add more first-class disciplines around their main disciplines, strengthen their characteristics, expand their international influence, and drive their schools to the forefront of the global education system. In January 2017, China’s Ministries of Education, and Finance and National Development and Reform Commission issued the “Implementation Measures for Coordinating the initiative of World-Class Universities and First-Class Disciplines (Provisional)”. The general provisions of this program mention that the international influence of universities should be highlighted and the strength of Chinese higher education, in terms of scientific research and international exchange and cooperation, should be comprehensively improved. In September 2017, 42 Chinese universities were titled as “double first-class” universities by the Ministries of Education and Finance and the National Development and Reform Commission. By this time, the “Double first-class” initiative had officially entered the implementation phase. As globalization proliferates further, governments aim to establish world-class universities within the country, to enhance national competitiveness in education, and increase their country’s global academic standing [1]. To accelerate the transformation of certain national universities into world-class universities, Korea implemented the “Brain Korea 21 Project” in 1999 [2], and Japan initiated the “21st Century Center of Excellence Program” in 2002 [3]. Similar programs were created in Europe, such as the 2010 “Initiatives d’Excellence” program by France [4], the “Project 5–100” by Russia and the 2005 “Initiative for Excellence” by Germany [5]. China was no exception to this trend. From the early “985” and “211” projects, to the current “double first-class” construction project, the overall goal has been to improve the level of development of higher education in China, and enhance international competitiveness [6]. The Chinese higher education system has become a more developed after its ruination during the Cultural Revolution. The academic influence of universities plays a key role in enhancing Chinese higher education, which will enable China to gain a competitive advantage at the international level [7]. From 2016 to 2020, China completed the first stage of the “double first-class” initiative. During this time, Chinese universities made corresponding reforms around the goal of attaining the “double first-class” title. The main objective of the “double first-class” initiative was to promote Chinese universities in international academic centers. Therefore, the international academic influence of Chinese universities, is important to gauge the effectiveness of the “double first-class” construction policy.

Considering the impact of the time accumulation effect which refers to the additional effect brought by the “double first-class” policy to universities, this study uses panel data of 122 universities in China and the United States, from 2016 to 2020. Further, the Propensity Score Matching–Difference in Differences method (PSM–DID) is adopted to empirically analyze the impact of “double first-class” construction policy on the international academic influence of Chinese universities. This study also compares and analyzes the difference in the international academic influence of Chinese universities before and after the implementation of the “double first-class” construction policy, to accurately evaluate its global effect. This study aims to answer the following research questions: 1. Does public policy aimed toward Chinese higher education, improve the international academic influence of Chinese universities? 2. Does the “double first-class” policy effect last long?

2. Related Work

2.1. “Double First-Class” Construction Initiative and University Academic Achievements

Most studies examining the “double first-class” initiative have not focused on the effectiveness of its implementation in selected universities. The research examining the initial stage of the “double first-class” initiative, is mainly focused on its connotation, objectives, and tasks, academic innovation, opportunities, challenges, and developmental strategies. In their study Lee [8] concluded that the contents of the “double first-class” initiative in China focus on the traditions and characteristics of universities, cultivation and attraction of talents, intensive financial support, and the introduction of a competitive system. Fu et al. [9] argued that the initiative to encourage talent is related to the development of university-level education and the process of reforming national university education. Therefore, it plays a very important role in the national “double first-class” initiative. Sun [10] evaluated the status of faculty and staff using the (Grey-AHP) model, and concluded that these factors also play an important role in accomplishing the initiative of “first-class” universities. They suggested that to improve the competitiveness in Chinese “double-class” universities, it is necessary to conduct rational and scientific research, examining the current status of the staff. Furthermore, Yuan et al. [11] analyzed the risks of discipline layout planning in China’s top universities, from the perspective of group behavior theory. They argued that in planning the disciplines of the “double first-class” initiative, the dominant disciplines are prone to involution caused by excessive investment of resources. However, the weak disciplines face risks of passive adjustment or abolition, induced by insufficient resources.

As the “double first-class” initiative continues to advance, research surrounding it is gradually becoming enriched and refined. Jing and Hua [12] used social network analysis to understand the development of collaborative networks in the clinical research publications promoted by Chinese “double first-class” medical universities. Shi et al. [13] presented a similar study. Li et al. [14] used the SBM-DEA evaluation model to evaluate the teaching efficiency across 36 “double first-class” universities. Wei and Zhang [15] compared the number of scientific publications by the “double first-class” universities from 2006 to 2018, and found that the absolute total number of articles published by Chinese universities increased in terms of quantity. To evaluate the performance of universities in the “double first-class” construction list, Xu [16] proposed a theoretical framework using a new evaluation model based on economic theory and hierarchical analysis (AHP). Their data were collected from universities in the “double first-class” initiative, and the results showed that the performance, input, and output rankings were inconsistent with each other. Therefore, the best-ranked universities are far from being the most efficient. These studies illustrate the results of the universities’ efforts to achieve the academic goals and characteristics outlined in the “double first-class” initiative. This suggests that the this initiative plays an important role in the development of Chinese universities. However, previous studies have evaluated the impact of “double first-class” initiative on universities based on the increment of absolute value. After excluding the cumulative effect of time on Chinese universities, we examine the actual impact of the “double first-class” construction policy.

2.2. Policy Evaluation

Following the initiation of the world-class university construction projects in various countries, significant emphasis was placed on policy evaluation to ensure the efficient use of resources and quality of education. Most studies in this field focus on the evaluation of policy effects from the perspective of research performance. Shin [2] compared the frequency of publications of SCI journals, and found that the BK21 project did not narrow the gap between Korean and American universities. Turko et al. [17] demonstrated that, universities under the 5-100 program, significantly improved their academic performance. Möller et al. [18] found that, the “Excellence Programme” has not improved the German research system, from the perspective of highly cited publications. It can be observed that, these studies focus on the description of the achievements of the educational development programs. However, the simple description of the end result does not explain the effectiveness of the university’s achievements, which depends upon the implementation of policy.

Substantial quantitative research has also been applied in conducting policy evaluation. Sohn et al. [19] used the structural equation model to evaluate the BK2I program. Their results indicated that, the BK21 program did not increase the satisfaction of Korean graduate students. Agasisti et al. [20] used propensity score matching and data envelopment analysis to evaluate the effectiveness of the Project 5-100 program and found that it had a positive effect on collegiate performance. Given the practical contribution of policy implementation, the natural experiment approach has been used by numerous scholars, for policy evaluation. Using the difference-in-differences (DID) approach, a study examined the impact of funding on the performance of undergraduate students following four-year courses in Historically Black Colleges and Universities (HBCUs) [21]. However, it was found that funding has no significant impact on the academic performance of the related students. Another study explored the impact of the program “GEAR UP Iowa” on college enrollment and persistence, using a DID method. The findings showed that the “GEAR UP Iowa” increased college enrollment rates among high school graduates by 3–4 percentage points. However, it had no impact on college persistence [22]. These studies used naturalistic experimental methods to assess the actual effects of policy interventions on academic performance. Fu et al. [23] also used the DID approach to evaluate the World Class Universities program, and found that Taiwanese universities had developed their research outcomes, but were far from attaining their desired policy goals. While countries have implemented various policies to build world-class universities, the PSM-DID methodology is rarely used to evaluate such policies.

3. Theory, Data, and Method

3.1. Resource Dependency Theory and University Performance

This study applies the resource dependency theory to elucidate the impact of “double first-class” construction initiative on Chinese universities. Examining the dependence of public institutions on national government funding is crucial in understanding the application of resource dependency theory on higher public education [21]. According to the resource dependency theory, universities regulate the direction of their core mission to the greatest extent possible, to sustain their operations [24]. Resource dependency theorists argue that, those in control of resources can exploit the fear of environmental uncertainty and economic volatility of those receiving the resources. Thus, forcing the recipients to comply with their regulations. The more dependent an institution is on resources, the more likely it is to comply with the resource controller, to meet its needs [25].

The universities in the “double first-class” list exclusively include public universities, and public funding accounts for a major part of their income. This dependence on government funding exposes these institutions to changes in the state funding, which makes them more likely to conform to national expectations, so that they can protect their own interests [26]. The “double first-class” construction initiative is a national educational strategy following the “211” and “985” projects, which have contributed greatly to the improvement of China’s academic level. The “double first-class” construction initiative specifies further requirements for the academic development of universities. The core motive of this initiative, is to integrate Chinese universities into the global higher education system, and to significantly impact in the overall process of global university development [27]. From the perspective of academic achievements, this includes the completion of university academic output, enhancement of academic influence, and expansion of academic communication. The purpose of this study is to determine whether China’s support of “double first-class” education policies has led to an increase in the international academic influence of Chinese universities. This analysis will demonstrate the effectiveness of “double first-class” construction policy.

3.2. Data

This study uses an institution-level balanced panel dataset including 610 observations for Chinese and American universities, spanning from 2016 to 2020. These universities were ranked in top 1000 in U.S. news “Best National University Rankings” in 2016. The reason behind this sample period is that China announced the list of the “double first-class” universities in 2017. American universities were selected as comparison sample because of there were no related higher education policy shock in this period. The data included in this study were obtained from InCites database, the official websites of universities, China Education Statistics, and the NCES’s Integrated Postsecondary Education Data System (IPEDS).

The selection of variables was influenced by the theoretical framework of this study, focusing on the goals, measurements, and incentives of the “double first-class” construction policy. Recently, varies studies examining the “double first-class” initiative, have focused on academic achievements. The choice of controls was also informed by the prior research on the “double first-class” initiative. Hugo Horta and Shen [28] highlighted that the number of publications in the International Index literature is not a satisfactory measure of academic achievement. Therefore, they suggested a shift from research quantity to research impact. Fan et al. [29] showed that China is in the stage of quantitative and not qualitative development, and universities should utilize their resource advantages to enhance the academic impact of scientific research. Therefore, the scholarly influence of each institution for each academic year, is considered as the outcome variable (goal) in this study. With the institution as the main body, the main vehicle of academic impact is the academic research process, such as the development of talent through research projects, scientific communication, and degree of dissemination of research results. The citation-based bibliometric methods are widely used to assess the scientific publication records of authors, institutions, and journals [30]. Hirsch [31] proposed the h-index, which is superior to other indicators in predicting future scientific achievement. Kovba and Gribovod [32] found that the academic strength of an institution is often reflected in the international academic mobility preferences of students. Therefore, in this study, the academic impact (outcome variable) was measured using the international student ratio, international collaborations, network of scientific papers, documents in the top 1%, citation impact, frequently cited papers, and H-Index.

In this study, the international academic influence of universities is related to the internationalization of universities, quality of teaching, ability to conduct scientific research and open access rights for scientific research results [33,34]. Horta [35] argued that the international academic impact of universities is largely based on research. Moreover, the internationalization of the academic staff has been confirmed to have a positive impact on the academic performance of universities. Therefore, increasing the proportion of international academic staff at leading universities, could improve competitiveness at international level. The academic staff are the mainstay of the University’s academic output. However, increased number of students and heavy workloads have deteriorated the quality of the academic staff [36]. The teacher-student ratio has become a key factor in gauging the quality of teaching and learning [37]. Moreover, the universities’ reputations also play an important role in attracting students and improving the global academic competitiveness of the university [38]. The citation data regarding university research publications, is also an important indicator in assessing the academic impact of universities [39]. Open access research holds great potential for accelerating the acknowledgment and diffusion of research findings [40]. Therefore, the following variables were introduced as control variables: proportion of international teachers to total teaching staff, proportion of teaching staff to number of students, quality of teachers, reputation of scientific research, reputation of teaching, and number of open access research papers [41,42].

3.3. Method

This study aims to analyze whether the “double first-class” construction policy has affected the international academic influence of universities. To address this question, this study treats the introduction of the “double first-class” policy initiative, as a quasi-natural experiment. Further, the PSM-DID model is used to quantify the effect of the policy implementation, by selecting a homogeneous treatment group and a comparison group. Accordingly, the difference between the two groups indicate the international impact of Chinese universities before and after the policy introduction.

The DID method requires the treatment and comparison groups to meet the common trend assumption, otherwise a sample selection bias may arise out of individual heterogeneity, and affect the evaluation results [43]. To minimize the differences between the treatment and comparison group, this study adopts the propensity score matching method (PSM) that enables the construction of a homogeneous treatment group and a comparison group. Accordingly, it controls for the systematic differences in observable variables between “double first-class” and non- “double first-class” universities to address the problem of sample bias [44]. Because the propensity score matching method (PSM) cannot avoid the endogeneity problem caused by the omission of variables, the DID method is needed. Additionally, because the sample data are panel data, individual heterogeneity and time-varying yearly accumulation effects must be considered. Therefore, the institutional and annual fixed effects should be added to ultimately derive the “policy treatment effect”.

As mentioned above, this study uses the PSM-DID method to estimate the policy effect of “double first-class” initiative, and determine whether the “double first-class” policy affects the international academic influence of Chinese universities. In the PSM method, two types of universities are selected from the sample. The first group or the treatment group consists of 31 universities not included in the “double first-class” initiative list published in 2016. However, these universities have been motivated by the “double first-class” initiative since 2017. The other group or the comparison group comprises 91 universities that were not affected by the “double first-class” initiative from 2016 to 2020. Finally, after the PSM treatment, the samples were subjected to the DID analysis. The final econometric model was set as follows

$$Y_{i,t}={\beta }_0+{\beta }_{1}Treate{d}_{i,t}\times T_{i,t}+\lambda {\displaystyle \sum Contro{l}_{i,t}}+{\mu }_i+{\eta }_t+{\epsilon }_{i,t}$$

(1)

The explanatory variable Y_i,t represents the degree of international academic influence of the selected universities. $Treate{d}_{i,t}$ is a dummy variable for “double first-class” universities, which is used to distinguish the treatment group from the comparison group. $Treate{d}_{i,t}=1$ indicates the universities affected by “double first-class” initiative policy, $Treate{d}_{i,t}=0$ indicates the universities not affected by “double first-class” initiative policy. $T_{i,t}$ is the year of the introduction of the policy, $T_{i,t}=1$ is the year after the policy was introduced and $T_{i,t}=0$ is the year before the policy was introduced. The cross multiplier $Treate{d}_{i,t}\times T_{i,t}$ is the core explanatory variable that determines whether the “double first-class” initiative policy impacted the international academic influence of Chinese universities. If the coefficient is significantly positive, the “double first-class” initiative policy is proven to significantly promote the international academic influence of Chinese universities. However, if it is insignificant the policy did not promote the international academic influence of Chinese universities. $C\mathrm{o}\mathrm{n}\mathrm{t}\mathrm{r}\mathrm{o}{l}_{i,t}$ refers to a series of control variables, ${\mu }_{\mathrm{i}}$ denotes institutional fixed effects, ${\eta }_{\mathrm{t}}$ denotes year fixed effects, and ${\epsilon }_{i,t}$ is a random disturbance term.

Second, to further analyze whether the impact of the “double first-class” initiative on universities is sustainable, we test the dynamic effect of the policy. Accordingly, the dynamic effect regression model is constructed by introducing the interaction term between the dummy and policy implementation variables, for the years 2018–2020, indicating the period following the implementation of the policy. This effect is given in Equation (2), as follows

$$Y_{i,t}={\beta }_0+{\displaystyle \sum _{n=2018}^{2020}{\beta }_n}{T}_n+{\displaystyle \sum _{n=2018}^{2020}{\gamma }_n}Treate{d}_{i,t}\times T_n+\lambda {\displaystyle \sum Contro{l}_{i,t}}+{\mu }_i+{\epsilon }_{i,t}$$

(2)

T_n is the year-time dummy variable for years 2018–2020, $Treate{d}_{i,t}\times T_n$ is the interaction term between the dummy and policy implementation variables for years 2018–2020, and ${\gamma }_n$ is the coefficient of the interaction term $Treate{d}_{i,t}\times T_n$.

3.4. Robustness Checks

To test the robustness of the results presented in this study, we randomly generate an experimental group [45]. A placebo test conducted by randomly generating an experimental group, can determine whether the changes in the international academic influence of universities are caused by other randomness factors. In this study, the pseudo-policy effect estimates were obtained by randomly selecting 31 colleges and universities from the sample, as the pseudo-experimental group. The remaining 91 colleges and universities formed the pseudo-control group, to test the policy effects. After 500 repetitions of random sampling, the placebo outcome coefficients were extracted and plotted in a graph. It was checked whether the randomized DID term coefficients were concentrated around 0, or significantly deviated from their true values. If the randomized DID coefficients are concentrated around 0 and significantly deviate from their true values, the change in the international academic influence of Chinese universities, is caused by the implementation of the “double first-class” initiative and not by other random factors This indicates that the conclusion is robust; otherwise, the conclusion is not robust.

3.5. Limitations

There are three major limitations to this study that should be noted when interpreting its findings. First, there is no unified consensus regarding the exact definition of academic influence. Many studies on academic impact vary based on their measures and descriptions of different subjects. Although this study measures academic impact from different perspectives derived from previous research, measuring academic impact solely in terms of quantifiable metrics, is not considered comprehensive [46]. Second, in this study, the comparison of the academic impact was conducted at the international level; therefore, it is mainly based on international research databases and does not consider Chinese research databases. This may, to a certain extent, weaken the performance of Chinese universities and affect the judgment of policy effectiveness. Third, differences exist among different categories of universities, and if these factors are ignored to directly compare all universities at the same level, it is easy to produce bias. For example, comparing comprehensive universities with science and engineering institutions will inevitably lead to bias.

4. Results

4.1. Propensity Score Matching Processing

Table 1. Balance Test Before and After Propensity Matching for experimental and control groups.

Variable	Unmatched/Matched	Mean		%Bias	%Reduct |Bias|	t-Test
		Treated	Control			t	p > t
International staff/staff	U	46.307	52.554	−31.2	95.4	−1.5	0.137
	M	46.307	46.594	−1.4		−0.06	0.953
Staff/student	U	54.404	57.482	−13.8	89.3	−0.58	0.562
	M	54.404	54.732	−1.5		−0.06	0.951
Research reputation	U	8.0029	28.394	−79.8	98.7	−3.34	0.001
	M	8.0029	7.7413	1.0		0.08	0.937
Teaching reputation	U	8.9241	30.129	−83	99.9	−3.5	0.001
	M	8.9241	8.8987	0.1		0.01	0.994
Free to read documents	U	315.94	647.48	−62.4	99.5	−2.51	0.013
	M	315.94	317.53	−0.3		−0.03	0.978
The quality of teachers	U	11.177	24.495	−83.9	98.4	−3.58	0.000
	M	11.177	10.963	1.4		0.07	0.941

Note. U denotes before matching; M denotes after matching.

shows the results of the balance test for the treatment and comparison groups, after propensity matching. Compared to the pre-matching period, the differences between the treatment and comparison groups were significantly lower after propensity score matching. These differences were significantly lower for the following variables: the proportion of international teachers to total teaching staff, proportion of teaching staff to student staff, quality of teachers, reputation of scientific research, reputation of teaching, and number of open access studies. The absolute value of the standard deviation for each variable was significantly less than 10. Rosenbaum and Rubin [41] stated that when the standard deviation values of the matched variables and absolute value of the standard deviation of the matching variables, are greater than 20, they are considered to be poor. Figure 1 shows the density distribution of the propensity scores for the treatment and comparison groups, before and after propensity matching. The kernel density plots of the propensity scores of the treatment and comparison groups almost overlap after the completion of matching. Therefore, it can be assumed that the matched treatment and comparison groups selected in this study, show a largely consistent trend in 2016. Therefore, they can be compared with each other.

4.2. Average Effect of “Double First-Class” Construction Policy on Scholarly Influences

Table 2. “Double First-Class” Initiative Effect on Scholarly Influences of Chinese Universities.

	SI	IC	WSD	DT	CI	HCP	HI
Treatedi,t × T	2.218 **	294.9 ***	28,668.7 ***	0.415 ***	5.099 ***	33.51 ***	11.33 ***
	−2.28	−5.63	−6.24	−4.21	−7.94	−5.96	−5.37
Control Variables	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Year fixed effects	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Institution fixed effects	Yes	Yes	Yes	Yes	Yes	Yes	Yes
_cons	30.98 ***	1060.1 ***	−105,827.7 ***	1.620 ***	9.086 ***	10.24	40.92 ***
	−7.78	−6.49	(−5.10)	−5.06	−3.83	−0.61	−4.53
N	550	550	550	550	550	550	550
R2	0.986	0.978	0.913	0.828	0.94	0.938	0.956
adj. R2	0.983	0.972	0.888	0.78	0.923	0.921	0.943

Note. SI = International Student Ratio; IC = International Collaborations; WSD = Web of Science Documents; DT = % Documents in Top 1%; CI = Citation Impact; HCP = Highly Cited Papers; HI = H-Index. ** p < 0.01 *** p < 0.001.

shows the average effect of the “double first-class” initiative on the global academic influence of Chinese universities. The regression results of the core variables show that the implementation of the “double first-class” initiative, has a significant positive effect on the international academic influence of Chinese universities. First, in terms of international academic exchanges, after the implementation of the “double first-class” initiative, the treatment group universities increased the average proportion of international students to total students, by 2.218%. Additionally, the average number of international collaborative studies increased by 295, compared with the comparison group universities. Second, in terms of international academic publications, universities in the treatment group published an average of 28,668 more academic publications than universities in the comparison group, after the implementation of the “double first-class” initiative. Lastly, the universities in the treatment group published 0.415% more high-level papers, 34 more highly cited papers, 5 more citation impact papers, and 11.33 more h-index papers, than the universities in the comparison group.

4.3. Dynamic Effect of “Double First-Class” Construction Policy on Scholarly Influences

As shown in

Table 3. Dynamic Effect of “double first-class” initiative on Scholarly Influences of Chinese.

	SI	IC	WSD	DT	CI	HCP	HI
$Treate{d}_{i,t}\times T_{2018}$	1.682 **	250.9 ***	19,840.7 ***	0.257 ***	3.252 ***	26.69 ***	8.288 ***
	−2.4	−7.51	−9.52	−3.14	−7.23	−6.51	−5.17
$Treate{d}_{i,t}\times T_{2019}$	2.224 ***	494.0 ***	41,552.5 ***	0.447 ***	5.638 ***	52.53 ***	8.871 ***
	−2.84	−11.68	−15.53	−5.79	−11.7	−11	−4.64
$Treate{d}_{i,t}\times T_{2020}$	2.224 ***	640.8 ***	59,924.6 ***	0.937 ***	6.334 ***	83.90 ***	7.858 **
	−2.84	−10.08	−14.81	−8.65	−10.29	−9.23	−2.49
Control Variables	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Year fixed effects	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Institution fixed effects	Yes	Yes	Yes	Yes	Yes	Yes	Yes
_cons	46.49 ***	1361.1 ***	39,304.6 ***	2.191 ***	18.71 ***	79.51 ***	85.03 ***
	−286.64	−150.55	−45.94	−113.57	−156.95	−80.62	−191.28
N	550	550	550	550	550	550	550
R2	0.985	0.983	0.915	0.846	0.941	0.956	0.952
adj. R2	0.981	0.979	0.892	0.805	0.925	0.945	0.939

Note. SI = International Student Ratio; IC = International Collaborations; WSD = Web of Science Documents; DT = % Documents in Top 1%; CI = Citation Impact; HCP = Highly Cited Papers; HI = H-Index. ** p < 0.01 *** p < 0.001.

, the policy effects of the “double first-class” initiative show some fluctuations in terms of the yearly impact. In the first year after the implementation of the “double first-class” initiative, Chinese universities showed significant increases in all variables, among which, an increase in the number of publications was the most expected. Additionally, the proportion of international students increased by 1.68%, and the citation of academic results also showed a small promotion effect. In the second year of the policy, the percentage of international students was 2.224% more than the comparison group, number of international collaborative papers was 490 more, and number of papers published was 41,553 more than that of the comparison group. However, in terms of citation, the growth rate is 0.2% more than last year, indicating the percentage of high-level papers, citation impact is 2.4 more, number of highly cited papers is 26 more, and the h-index is 0.6 more than that in the last year. In the third year of the policy’s implementation, the impact of the “initiative on the universities, is 0.2. In the third year of the “double first-class” initiative, its effect on universities began to decrease. Accordingly, the proportion of international students stopped rising, and number of international cooperation studies increased, compared with those in the previous year. The citation impact dropped by 1.5 compared to that in the previous year, and the effect on h index also decreased. It is worth noting that in terms of citation frequency, the number of highly cited papers and ratio of high-level studies showed a significant increase, with the ratio of high-level papers reaching 0.94% and volume of highly cited papers reaching 84.

The validity of the present results is further supported by the placebo test. Figure 2 and Figure 3 show the distribution of the policy effect estimates after 500 random repetitions of the selected pseudo-treatment groups. The x-axis indicates the estimated value, the y-axis represents the corresponding p-value, horizontal dashed line demarcates the 10% significance level, and vertical dashed line gives the true estimate of the DID model. As shown in Figure 2 and Figure 3, the corresponding p-values determining the pseudo-policy effect estimates of the seven indicators and measuring the academic influence, were all greater than 0.1, after random sampling of the treatment groups. However, the true estimates of these indicators did not intersect with the pseudo-estimates. Therefore, the results of this study, demonstrating the effectiveness of the “double-class” initiative, are not obtained by chance.

5. Discussion and Conclusions

Chinese universities have experienced varying degrees of growth in international academic outcomes and collaborations, after the implementation of the “double first-class” initiative, suggesting an expansion of their global academic influence. The objective of the “double first-class” initiative is to build world-class universities for global competitiveness. Accordingly, increasing international exchange and cooperation is an effective development path. The internationalization of scientific research cooperation has been conducive to the growth of China’s international discourse in the academic field, improving the international academic influence of Chinese universities [47]. The data results from this study also indicate that first-class universities built in China, are gaining global recognition. This academic influence has been attained by research cooperation and university selection.

However, the output of high-quality academic results from Chinese universities, remains limited. The improvement of academic quality is the core of academic development of universities. The results of this study show that despite the “double first-class” initiative having a significantly positive impact on the citation of academic results in universities, its growth rate is much lower than the growth of academic publications. The number of academic publications of universities can reflect their academic ability to a certain extent, but the ultimate pursuit is to elevate their academic quality, not quantity. Through the textual analysis of the early initiative programs of first-class universities, it can also be found that the initiative programs of Chinese first-class universities exhibit tendencies of utilitarianism [48]. In the area of scientific research, there is a tendency to pursue materialized and quantifiable results. From the analysis process, it is apparent that the promotion effect of the “double first-class” initiative on the quantity of academic publications of universities, is much higher than the academic quality in the early stages of initiative. However, universities can over report the quantity of their academic publications, which is not conducive to the development of international competitiveness. Moreover, it hampers the progress of academic quality and limits the speed of scale expansion.

The promotion effect of the “double first-class” construction policy does not last long. A university becomes globally competitive if it can consistently attract talented researchers, faculty and students, has adequate physical and financial resources, a modern infrastructure base, and operates under an effective management model [49]. The “double first-class” universities mainly rely on state funding, and the current study found that Chinese universities have a wide range of financial investments in the initiative process. Therefore, the projects are crossed and duplicated, the focus is not prominent, and the performance management concept is lacking, along with performance assessment and other phenomena [50]. However, it can be related to the lagging effect of education policies, caused by the expression of policy needs and development, implementation and dissemination of solutions [51].

The first stage of “double-first class” policy was from 2016–2020. Some of the effects of this policy may not appear. The second stage of the double -class construction project has been implemented in 2021. More policy effects will also be discovered in the future.

The current findings support the effectiveness of the “double first-class” construction policy in achieving its intended outcomes, but more research is needed to explain the effects of these efforts and how they can be sustained. The “Project 5–100” by Russia aims to at least 5 universities enter the top 100 in the world’s rankings in 2020. Judging from the three major world’s rankings in 2019–2020, this project has not achieved the expected goal. But from the perspective of participating in universities, the level of internationalization of universities has been significantly improved, and international scientific research cooperation has made rapid progress. Many universities have been able to enhance international competitiveness and international influence by virtue of this plan. The goal of the “Initiative for Excellence” by Germany is to increase the funds of universities and scientific research institutions, so that German universities and research institutions can improve their research ability and academic level. The results showed that the social awareness, international influence, and teacher -student attraction of funded German universities have been greatly improved. The number and quality of scientific research have made a significant breakthrough, and the world rankings of high -level universities in Germany have also improved. Similarly, because the goals of the “double first-class” program are multidimensional and its effects of implementation vary significantly, further research is needed to explore the impact of the “double first-class” program.