Categorical Evaluation of Scientific Research Efficiency in Chinese Universities: Basic and Applied Research

Abstract

The categorical evaluation of scientific research efficiency is of great significance to technological innovation and research management. It is also helpful to promote the sustainable development of basic research and applied research in universities. In this study, 32 “Double First-Class” universities directly under the Ministry of Education in China were evaluated with the research efficiency evaluation system of basic research and applied research used, which is constructed based on the “research efficiency classification evaluation”. The empirical results show that the efficiency of basic research is low but total factor productivity grows faster, while the efficiency of applied research is high but total factor productivity grows slowly, and the gap between the two will be further reduced in the future. At the same time, scientific research efficiency depends on the type of university and disciplinary strengths: Comprehensive and normal universities are good at basic research while scientific and agricultural and forestry universities are more efficient in applied research. Universities should consolidate their strengths while making key breakthroughs on their shortcomings, optimize the structure of research inputs and outputs, and improve the efficiency of research resources utilization to actively promote the national innovation system and the construction of a powerful nation of science and technology.

1. Introduction

The outline of the 14th Five-Year Plan proposes that China should adhere to the core position of innovation in the overall situation of modernization and take scientific and technological self-reliance and self-improvement as the strategic support for development. To meet the major needs of national economic development, China should deeply implement the strategy of rejuvenating the country through science and education, the strategy for developing a quality workforce, as well as the strategy of innovation-driven development [1]. As the combination point of science, talent and innovation, universities have more than 60% of the key state laboratories, undertake more than 80% of the National Natural Science Foundation projects, and gather more than 40% of the academicians of the two academies. Therefore, they need to strengthen basic research and original innovation, improve the common technology supply system, and help the national innovation system and the construction of a scientific and technological power by taking advantage of their resources.

According to the statistics of science and technology of the Ministry of Education, the investment in science and technology in China’s universities increased from 153.701 billion yuan to 245.820 billion yuan (an increase of 59.93%) from 2016 to 2019; the full-time equivalent of research and development personnel increased from 234,700 to 509,700 person-years (an increase of 117.17%). The convergence of resources has laid a solid foundation for the scientific research output of universities. As a result, 13,600 monographs were issued, 1,083,300 papers were produced, 206,000 patents were authorized, and 13,900 technology transfer contracts and 6.75 billion yuan were signed in 2019, an increase of 197.29, 17.99, 42.71, 45.1 and 152.78%, respectively, over 2016. While the increase in resource input has brought about an increase in scientific research output, people cannot help but think, in the face of large-scale investment in scientific research resources, how should the resources of universities be allocated between basic and applied research to promote a sustainable research system. How efficient are they in their respective research at present? What are the problems of each? Are there differences between different universities?

The purpose of this study is as follows. Firstly, a scientific research efficiency evaluation index system for basic research and applied research was built from the perspective of scientific research classification evaluation to explore the rationality of scientific research classification evaluation. Then, the BCC model and Malmquist index were applied to analyze the scientific research efficiency of basic research and applied research in “Double First-Class” universities directly under the Ministry of Education of China from both static and dynamic aspects. We hope to find out the differences and existing problems between individuals and groups and put forward countermeasures and suggestions to promote the further optimization of scientific research management in sample universities. Meanwhile, we hope that readers can use our study to update previous perceptions and to build a sustainable concept of universities’ research systems. we also expect university administrators to take measures to promote the sustainable development of university research systems based on the concept of classification and evaluation.

The rest of this paper is arranged as follows. Section 2 is a review of domestic and foreign literature. Section 3 is a research design, including research ideas, samples and data sources, and the method model used in the research. Section 4 is the classification construction of the evaluation system. Section 5 is the empirical analysis results, which are elaborated using the three aspects of basic research, applied research and differences between universities. Section 6 summarizes the research conclusions and proposes countermeasures and suggestions on this basis. Section 7 is a discussion that describes the limitations of this study and helps the reader to better understand the article.

2. Literature Review

At present, scholars have conducted a lot of research on the scientific research efficiency of universities. Kempkes G. used DEA and SFA to measure the scientific research efficiency of German public universities, finding that the average efficiency score of universities in the western region is higher, while the universities in the eastern region were better at total factor productivity [2]. Rhaiem M. used the two-stage Bootstrap DEA methodology to assess the academic research efficiency of Canadian business school scholars, showing that there were differences between academics in different disciplines. The accreditation, qualifications, funding sources of independent institutions, business schools with doctoral programs, as well as the prestige and reputation of affiliated universities have a significant impact on the efficiency of scientific research [3,4]. The network DEA feedback model was applied to evaluate the research efficiency of Korean private universities by Shamohammadi M. indicating that the efficiency of scientific research universities was higher than that of scientific research and teaching universities and teaching universities [5]. Exposito-Garcia evaluated the level of scientific efficiency of several Spanish state universities from the perspective of desirable and undesirable outputs and gave improvement measures under the natural and management efficiency schemes [6]. Xu X. measured the research efficiency of universities in 29 provinces in China using the DEA-Window model. The results showed that the comprehensive efficiency of scientific research in universities increased significantly and the growth rate slowed down from 2016 to 2018. Provinces at the forefront of comprehensive efficiency account for less than 1/3 of the whole country, so the level of scientific research efficiency in universities was generally not high [7]. Jiang J. examined the research efficiency of Chinese higher education institutions. The result showed that the research efficiency of “Project 211” universities was lower than that of “Project 985” and general universities. The research efficiency of the selected universities in China differed by region and the type of universities [8].

Xue W. used the three-stage DEA and Malmquist index methods to evaluate the scientific research efficiency of universities directly under the Ministry of Education of China, indicating that the overall efficiency of scientific research activities in universities increased year by year, and from the perspective of efficiency segmentation, scale efficiency optimization played a significant role in efficiency improvement [9]. Su W. believed that most of the traditional scientific research evaluation methods started from a macroscopic perspective and ignored the differences between disciplines, thus a multi-criteria decision-making (MCDM) non-radial super-efficient data envelopment analysis (NRSDEA) model was proposed to effectively solve the management non-solution problem and integer decision variable constraints [10]. Wang L. used the DEA model to study the scientific research efficiency of 31 provincial universities in Shandong Province, demonstrating that the sample universities were not high and showed a downward trend. There was a positive correlation between the scientific research strength and technical efficiency for universities, and the regional economic development had a significant positive impact on the efficiency of the scientific research scale [11]. Zong X. used the super-efficiency BCC model and the Malmquist index method to evaluate the scientific research efficiency and its changes in China’s “Double First-Class” universities, finding that the overall scientific research efficiency was low. A slightly increasing trend was observed year by year. The differences in efficiency depended on the region and type of university [12]. Chen X. explored the Influencing factors of universities’ scientific & technological innovation performance, based on the Tobit-DEA model. It was highly related to the input quality and matching structure of scientific research elements, government relevance, and industry-academia-research collaboration level. Among these main factors, the high proportion of full-time teachers with senior titles, reasonable resource allocation structure and government support had a significant positive impact, whereas the number of participants in international academics had a negative impact [13].

In conclusion, study on the efficiency of research in universities is on the ascendant. From the perspective of research content, it mainly focuses on the empirical analysis of scientific research efficiency [14,15,16] and the exploration of influencing factors [17,18,19]; from the aspect of research objects, it generally includes universities in different countries or specific regions and different types of universities [20,21,22]; from the perspective of research methods and models, most of them use classical DEA models and super-efficient DEA models improved on their basis [23,24,25]. However, existing research is often an overall evaluation of scientific research efficiency, that is, basic research and applied research are generalized, and no specific analysis is carried out using the evaluation system of “categorical evaluation of scientific research efficiency”. Meanwhile, the selection of indicators tends to focus more on the output of science and engineering, and the output of humanities and social sciences is less concerned. Consequently, it is often unfair to unify science and humanities and social science into a set of evaluation indicators for evaluation due to the significant differences in the input and output between basic research and applied research and the respective advantages between science and humanities and social science. Furthermore, it is found that the recommendation to simply increase or decrease scientific research investment loses the directivity of basic or applied research in operation when the evaluation results are applied to practice. Therefore, the classification and evaluation of the scientific research efficiency not only is favorable to enrich the current scientific research efficiency evaluation system but also makes for guiding the practice of scientific research management.

3. Research Design

3.1. Research Ideas

Scientific research efficiency generally refers to the efficiency level of universities that make full use of various scientific research resources to engage in scientific research activities and produce scientific research results. In other words, the maximum amount of scientific research output that universities can obtain with the given scientific research inputs. It is calculated by the ratio of weighted output to weighted input. The research idea of this paper is to construct two sets of evaluation indicators that distinguish basic research and applied research based on theoretical analysis. Various sample universities are selected to conduct empirical analysis via using the panel data and explore the rationality and scientificity of the indicator system of “categorical evaluation of scientific research efficiency”.

At the same time, in order to scientifically explore the deep-seated variability that exists within the sample universities, we propose to link university type to research efficiency heterogeneity for comparative analysis. The reasons for this design are as follows. Firstly, we refer to the study of Rhaiem M., et al. [3], Jiang J., et al. [8], Zong X., et al. [12] to fuzzy classify the research subjects according to attributes. More importantly, the reason is that universities have different attributes and thus their disciplinary strengths are different, and there are large differences between basic and applied research inputs and outputs. The division by attributes can better reflect the differences between sample groups and enable targeted improvement measures to promote the sustainable development of university research systems.

In addition, the current situation and problems of scientific research management are summarized to provide advantageous suggestions for the improvement of the scientific research evaluation theory of universities and the practice of scientific research performance management and resource input adjustment of universities. In order to clearly express the research thinking, we draw a concise thinking diagram, as shown in Figure 1.

3.2. Samples and Data

University scientific research is a multi-input and multi-output activity. In this study, 32 “Double First-Class” universities directly under the Ministry of Education are selected for the following reasons. Firstly, they are directly managed by the Chinese Ministry of Education and play an exemplary role in teaching, research and social services. Secondly, the sample universities are widely representative. On the one hand, the sample universities come from the four major regions of central, eastern, western and northeastern China in terms of economic regions; on the other hand, the sample universities include comprehensive universities, science & technology universities, normal universities and agriculture & forestry universities in terms of university types. Thirdly, in terms of scientific research capability, these universities, as outstanding research universities in China, have an excellent performance in basic and applied research with strong research teams, sufficient funds and resources. Therefore, their research input and output data are more informative and can provide better data support for this study. In conclusion, the sample universities, as “Double First-Class” universities, can identify problems in their research management through the study, which will contribute to the sustainable development of the “Double First-Class” project in China.

Considering the time lag between the input and output of science and technology in universities, this study sets the time lag period for the characteristics of basic research and applied research when calculating the efficiency: the cycle of basic research is long, and the project generally goes through 2–5 years from the beginning to the output of the results; the applied research cycle is relatively short, and the project time is generally 1–3 years; after scientific and reasonable consideration, the basic research sets a 3-year time lag period, and the applied research sets a 2-year delay period. The data source of the article is from the “Compilation of Scientific and Technological Statistics of Colleges and Universities” published by the Ministry of Education, the “Compilation of Statistical Data of Universities Directly Affiliated to the Ministry of Education” and the panel data of the Web of Science and CNKI Academic Publication and Citation Database, considering the time lag of input and output, the basic research and applied research input data are from 2012–2014, 2013–2015, and the corresponding output data are from 2015–2017.

3.3. Research Model

3.3.1. BCC Model

DEA, namely data envelopment analysis, was first established and used by American operational researchers Charnes and Rhodes [26]. Due to its advantages in evaluating the performance of multi-input and multi-output decision-making units, DEA has been widely applied in operations research, management and other fields. Considering the variable scale income and the actual management of university research, the output-oriented BCC model is applied for scientific research efficiency analysis. The output-oriented BCC model expression is:

$$\begin{array}{c}Z=max\theta \\ S.t\left\{\begin{array}{c}{\displaystyle {\sum }^{}_n^{j=1}}{x}_j{\lambda }_j+S^{-}=x_{j_0}\\ {\displaystyle {\sum }^{}_n^{j=1}}{y}_j{\lambda }_j-S^{+}=\theta y_{j_0}\\ {\displaystyle {\sum }^{}_n^{j=1}}{\lambda }_j=1,j=2,3,\cdots ,n\\ \theta \mathrm{is}\mathrm{unrestrained},S^{+}\ge 0,S^{-}\le 0\end{array}\right.\end{array}$$

(1)

where x_j is the input variable, y_j is the output variable, and S⁺, S⁻ is the relaxation variable.

The BCC model is based on the scale of variable income. Comprehensive efficiency (Crste) can be decomposed into pure technical efficiency (Vrste) and scale efficiency (scale), that is, Crste = Vrste × Scale. The meaning of each efficiency is different: comprehensive efficiency indicates the static scientific research efficiency level of universities, pure technical efficiency indicates the scientific research operation mechanism and management level of universities, while scale efficiency indicates the rationality of input-output structure and resource allocation. For the model calculation results, if Crste = 1, it means that the decision unit DEA is valid, and if Crste < l, it means that the decision unit DEA is invalid [27].

3.3.2. Malmquist Index

The traditional DEA model is to measure input-output efficiency from a static point of view, while the Malmquist index is an improved and developed method for measuring input-output efficiency from a dynamic point of view, which was first proposed by Swedish economist Malmquist [28]. The Malmquist index reflects the change in the productivity of the decision unit in the time series, and the Malmquist index has the expression total factor productivity (Tfpch) as follows:

$$M_0\left(x_t,y_t,x_{t+1},y_{t+1}\right)={\left[\frac{D^t\left(x_t,y_t\right)D^t\left(x_{t+1},y_{t+1}\right)}{D^{t+1}\left(x_t,y_t\right)D^{t+1}\left(x_{t+1},y_{t+1}\right)}\right]}^{1/2}$$

(2)

where $\left(x_t,y_t\right)$ and $\left(x_{t+1},y_{t+1}\right)$ are input variables and output variables in the t period and t + 1 period, respectively, D^t and D^t+1 are distance functions for the corresponding period.

Total factor productivity (Tfpch) can be decomposed into technological progress efficiency (Techch) and technical efficiency (Effch). That is, Tfpch = Techch × Effch, the meaning of each efficiency is different, the total factor production efficiency indicates the dynamic scientific research efficiency level, the technological progress efficiency indicates the progress of scientific research technology, and the technical efficiency indicates the improvement of scientific research management level and scientific research investment. For the model calculation results, if the Tfpch > 1, it implies that the total factor productivity change of the sample unit shows an upward trend, indicating that the scientific research level has improved in this period; if the Tfpch < 1, it shows that the change of total factor productivity of the sample unit shows a downward trend, indicating that the scientific research level of the sample unit is decreasing during this period.

4. Evaluation Index Construction

The construction of the indicator system should be “adapted to the research conditions”. Compared to the generalization in the existing evaluation, our study attempts to distinguish input-output metrics from basic and applied research. The input indicators start from the three aspects of personnel, funding and projects, but highlight the division between basic research and applied research. Furthermore, the construction of output indicators is different: basic research is considered from the two aspects of scientific and technological achievements and achievement awards, and the applied research should also take into account technology transfer on this basis.

4.1. Evaluation Index Construction of Basic Research Efficiency

Basic research refers to experimental or theoretical research conducted to acquire new knowledge about the basic principles, such as phenomena and observable facts, which is not aimed at any specific application. Basic research can be divided into natural science and humanities and social science. The basic research of natural science mainly reveals the nature of objective things through experimental analysis or theoretical research, explores the laws of object movement, or puts forward and verifies various assumptions, theories or laws. Basic research in the humanities and social sciences serves the national strategic needs and the inheritance of human civilization by discovering new problems, putting forward new ideas and constructing new theories based on adhering to the correct political direction.

Output indicators of basic research include scientific and technological achievements and achievement awards: Scientific and technological achievements contain SCI papers, scientific works, national invention patent authorization and acceptance of national basic research projects in natural sciences, and acceptance of SSCI papers, monographs and national basic research projects in humanities and social sciences. The acceptance of national basic research projects mainly refers to the acceptance of the National Key Basic Research Development Program of Natural Science (973 Program) projects, the National Natural Science Foundation of China projects, and the National Philosophy and Social Science Foundation of Humanities and Social Sciences projects. The national awards mainly refer to the National Natural Science Award for the natural sciences and the Outstanding Achievement Award for scientific research in the humanities [29]. It can be seen from

Table 1. Evaluation index system of basic research efficiency.

	First-Level Indicators	Second-Level Indicators	2015 (2012)	2016 (2013)	2017 (2014)	Average Annual Rate
Inputs	Personnel	Personnel input in that year	18,479	20,050	21,371	7.54%
	Fundings	R&D expenditure of that year	713,494	845,472	884,517	11.54%
	Projects	Research projects approved in that year	33,789	35,410	38,791	7.17%
Outputs	Scientific and technological achievements	SCI/SSCI papers	172,870	191,626	211,090	10.50%
		Scientific works	1161	1266	1276	4.92%
		Number of National invention patent authorization	13,070	16,843	19,055	21.00%
		Number of national projects accepted	951	966	1041	4.67%
	Achievement awards	Number of national awards	188	208	220	8.20%

The data in each year is the sum of each index data of 32 sample universities; and the R&D expenditure unit is ten thousand yuan.

that from 2015 to 2017, the resource investment in basic research of the sample universities maintained an annual growth rate of more than 7%, and the annual growth rate of funding expenditure and project investment in that year was more than 10%. The output of scientific basic research showed different degrees of growth: invention patents showed a good performance, maintaining an annual growth rate of 21%, SCI/SSCI papers and national awards increased by more than 8%, monograph and national project acceptance remained an annual growth rate of more than 4%.

4.2. Evaluation Index Construction of Applied Research Efficiency

Applied research refers to creative research for obtaining new knowledge, determining the possible use of basic research results, or exploring new approaches to achieve predetermined goals. Applied research can also be divided into natural sciences and humanities and social sciences: The applied research of natural sciences is mainly demand-oriented and promotes the transformation of scientific and technological achievements and the development of industry industries by breaking through key and core technologies and expanding the application value of knowledge. Comparatively speaking, applied research in the humanities and social sciences is driven by important social issues, it solves important social issues by applying new methods, using new materials and forming new countermeasures, and provides decision-making suggestions for the Party and government.

The output indicators of applied research are carried out from three aspects: scientific and technological achievements, achievement awards and technology transfer. Scientific and technological achievements include EI and Chinese Core Journals papers on natural science, national utility model patent authorization and acceptance of national applied research projects, while the achievements of humanities and social sciences contain CSSCI and Chinese Core Journals papers as well as research reports. Among them, the acceptance of national applied research projects mainly refers to the national science and technology project and the national high-tech Research and Development Program (863 Program) project on natural science. Achievement awards mainly include the National Invention Award for natural science, the National Science and Technology Progress Award and the consultation service Report Award for Outstanding Achievements in university scientific research for humanities. Technology transfer mainly refers to the technology transfer contract of natural sciences, humanities and social sciences and the actual income of technology transfer in that year.

The comparison between Table 1 and

Table 2. Evaluation index system of applied research efficiency.

	First-Level Indicators	Second-Level Indicators	2015 (2013)	2016 (2014)	2017 (2015)	Average Annual Rate
Inputs	Personnel	Personnel input in that year	18,040	17,605	18,777	4.25%
	Fundings	R&D expenditure of that year	1,044,220	1,088,860	1,210,942	7.74%
	Projects	Research projects approved in that year	35,522	37,173	39,214	5.07%
Outputs	Scientific and technological achievements	EI/CSSCI/Chinese Core Journals papers	209,036	209,122	218,049	2.15%
		Number of research reports adopted	1558	1692	1763	6.40%
		Number of national utility model patent authorization	5087	5675	6062	9.19%
		Number of national projects accepted	436	534	461	4.40%
	Achievement awards	Number of national awards	104	115	109	2.68%
	Technology transfer	Number of technology transfer contracts	2611	1917	1637	−20.59%
		Income of technology transfer that year	107,639	84,035	85,554	−10.06%

The data in each year is the sum of each index data of 32 sample universities; and the R&D expenditure unit is ten thousand yuan.

showed that from 2015 to 2017, the expenditure on applied research in sample universities was higher than that on basic research. It was worth noting that the gap was gradually closing, and the input of human resources and projects was lower than that of basic research. The average annual growth rate of scientific research resources inputs in applied research was lower than that in basic research, also reflecting the national education department’s emphasis on basic research. Especially scientific research inputs have been continuously tilted to basic research in recent years, which has injected a “strong heart” into scientific and technological innovation in basic research. In terms of research output, the growth of applied research output was mixed: Utility model patents authorized by more than 9% a year, the study adopted the national project acceptance growth remains above 4%, the average annual growth rate of national awards and publications remains around 2%, technology transfer contract with the real income had substantial negative growth, this also reflected the multiplicity and uncertainty of university scientific research output.

5. Empirical Analysis Results

5.1. Efficiency Evaluation of Basic Research in Universities

The output-oriented BCC model and Malmquist index in DEAP 2.1 were used to calculate the static and dynamic research efficiency of basic research in 32 sample universities from 2015 to 2017, and the results were shown in

Table 3. Average value of basic research efficiency of sample universities from 2015 to 2017.

Universities	BCC Model			Malmquist Index			Universities	BCC Model			Malmquist Index
	Crste	Vrste	Scale	Tfpch	Techch	Effch		Crste	Vrste	Scale	Tfpch	Techch	Effch
PKU	0.721	1	0.721	1.154	1.257	0.918	XMU	1	1	1	1.237	1.237	1
RUC	1	1	1	0.814	0.814	1	SDU	0.519	0.796	0.652	1.029	1.301	0.791
THU	0.699	1	0.699	1.488	1.511	0.985	OUC	0.326	0.847	0.385	0.949	1.179	0.805
CAU	0.751	0.99	0.728	1.019	1.141	0.893	WHU	1	1	1	1.073	1.226	0.875
BNU	1	1	1	0.954	1.073	0.889	HUST	1	1	1	1.062	1.293	0.821
NKU	0.587	1	0.587	0.837	0.817	1.025	HNU	0.541	0.582	0.929	1.054	1.305	0.808
TJU	0.516	0.772	0.668	1.106	1.345	0.822	CSU	1	1	1	0.916	1.102	0.831
DUT	0.617	0.983	0.628	1.056	1.266	0.834	SYSU	0.554	0.75	0.738	1.153	1.294	0.891
NEU	0.725	1	0.725	0.888	1.113	0.798	SCUT	0.537	0.737	0.729	1.403	1.643	0.854
JLU	0.428	0.635	0.67	1.040	1.323	0.786	CQU	0.584	0.663	0.881	1.165	1.523	0.765
FUDAN	0.677	0.952	0.711	1.183	1.276	0.927	SCU	0.549	1	0.549	0.780	1.08	0.722
TONGJI	0.512	0.925	0.553	0.940	1.41	0.667	UESTC	0.618	0.947	0.653	1.133	1.455	0.779
SJTU	0.829	1	0.829	0.843	1.158	0.728	XJTU	0.634	1	0.634	1.039	1.294	0.803
ECNU	0.675	0.951	0.71	0.896	0.862	1.04	NWAFU	0.542	0.922	0.588	0.832	1.145	0.727
NJU	0.725	1	0.725	1.118	1.396	0.801	LZU	0.689	0.713	0.967	1.086	1.369	0.793
SEU	1	1	1	1.357	1.357	1	Average	0.695	0.911	0.761	1.054	1.250	0.849
ZJU	0.708	1	0.708	1.111	1.436	0.774

Crste indicates total (overall) efficiency; Vrste indicates pure technical efficiency, and Scale indicates scale efficiency, respectively. Tfpch indicates total factor productivity; Techch indicates technological progress efficiency; and Effch indicates technical efficiency, respectively.

.

From the perspective of static efficiency, in terms of the average overall efficiency, the average static scientific research comprehensive efficiency, pure technical efficiency mean and scale efficiency of the basic research of the sample universities from 2015 to 2017 were 0.696, 0.911 and 0.761, respectively. Although the basic research of the “Double First-Class” universities directly under the Ministry of Education of China has made a number of important original achievements in many aspects, such as high temperature superconductivity, multi-photon entanglement and neutrino experiments, and some important disciplines have entered the world’s advanced ranks. However, it could not be ignored that the scientific research efficiency of basic research needs to be further improved. Notably, pure technical efficiency was higher than scale efficiency, which inferred that the irrationality of the input-output structure was the main reason for the low overall efficiency of basic research [30]. For effective decision-making units, seven universities including the Renmin University of China were in a DEA effective state, accounting for only 21.88% of the total number of decision-making units, and the research input and output of these universities reached the relatively optimal level. The remaining 25 universities were in a non-DEA effective state, of which the basic scientific research efficiency value of seven universities, such as Peking University was higher than the average of the sample universities, and the basic scientific research efficiency of 18 universities, such as Nankai University was lower than the average of the sample universities, indicating that there was a large gap in the basic scientific research efficiency between the “Double First-Class” universities directly under the Ministry of Education.

From the perspective of dynamic efficiency, as for the overall efficiency, the average annual total factor productivity, technological progress efficiency and technical efficiency of basic research of the sample universities from 2015 to 2017 were 1.054, 1.250 and 0.849, respectively. The total factor productivity of the sample universities increased by 5.4% annually during the study period, of which the average annual growth rate of technological progress efficiency was 25%, while the average annual decline rate of technical efficiency was 15.1%. It could be concluded that from 2015 to 2017, with the increase in resource input, the efficiency of sample universities’ scientific research innovation has been effective and greatly improved. Scientific research and technological innovation were continuously improved and the innovation effect was obvious. However, there were still deficiencies in scientific research management and input-output ratio, and the catch-up effect is insufficient. In terms of individuals, the total factor productivity of 21 universities was greater than 1, while that of 11 universities was less than 1 among the sample universities. Furthermore, universities with total factor productivity greater than 1 account for 65.63% of the decision-making units, and the scientific research efficiency of basic research as a whole showed an increasing good trend year by year.

5.2. Efficiency Evaluation of Applied Research in Universities

As in the case of basic research, we also measured the static dynamic research efficiency of applied research, and the results were shown in

Table 4. Average value of applied research efficiency of sample universities from 2015 to 2017.

Universities	BCC Model			Malmquist Index			Universities	BCC Model			Malmquist Index
	Crste	Vrste	Scale	Tfpch	Techch	Effch		Crste	Vrste	Scale	Tfpch	Techch	Effch
PKU	1	1	1	0.769	0.62	1.241	XMU	0.322	0.482	0.669	1.017	0.833	1.221
RUC	0.845	1	0.845	0.664	0.664	1	SDU	0.836	0.909	0.92	1.008	0.845	1.193
THU	1	1	1	1.026	1.026	1	OUC	1	1	1	1.042	0.679	1.535
CAU	0.711	0.878	0.81	1.295	0.736	1.76	WHU	0.955	1	0.955	1.064	0.784	1.357
BNU	0.347	0.588	0.59	0.985	0.69	1.428	HUST	1	1	1	0.795	0.795	1
NKU	1	1	1	1.376	0.833	1.652	HNU	0.552	0.656	0.842	0.779	0.792	0.983
TJU	0.845	1	0.845	2.254	1.786	1.262	CSU	0.697	0.781	0.892	1.031	0.641	1.608
DUT	1	1	1	0.769	0.783	0.982	SYSU	0.658	0.983	0.669	1.064	0.926	1.149
NEU	1	1	1	1.041	0.962	1.082	SCUT	1	1	1	0.911	0.911	1
JLU	0.786	1	0.786	0.789	0.91	0.867	CQU	0.598	1	0.598	1.049	0.875	1.199
FUDAN	1	1	1	0.809	0.784	1.032	SCU	1	1	1	1.415	0.905	1.563
TONGJI	1	1	1	0.874	0.797	1.096	UESTC	0.727	0.949	0.766	0.828	0.638	1.298
SJTU	1	1	1	0.950	0.859	1.106	XJTU	1	1	1	1.265	0.946	1.337
ECNU	0.42	0.614	0.684	1.025	0.91	1.126	NWAFU	1	1	1	1.090	0.758	1.438
NJU	0.643	1	0.643	1.052	0.75	1.402	LZU	0.596	0.65	0.917	1.109	1.083	1.024
SEU	0.718	0.833	0.862	0.739	0.651	1.135	Average	0.821	0.916	0.884	1.027	0.844	1.227
ZJU	1	1	1	0.988	0.828	1.193

Crste indicates total (overall) efficiency; Vrste indicates pure technical efficiency, and Scale indicates scale efficiency, respectively. Tfpch indicates total factor productivity; Techch indicates technological progress efficiency; and Effch indicates technical efficiency, respectively.

.

From the perspective of static efficiency, in terms of the average overall efficiency, the average static scientific research comprehensive efficiency, average pure technical efficiency and the average scale efficiency of applied research from 2015 to 2017 were 0.821, 0.916 and 0.884, respectively. Compared with basic research, applied research efficiencies of most target universities were relatively higher, and scale efficiencies were generally lower, indicating that scientific research resources of applied research have not been effectively utilized due to the irrational management of scientific research and unbalanced proportion of input and output. It was also relevant to the research nature: Basic research has greater challenges and difficulties, more risks and uncertainties of scientific research output, as well as longer periodicity than applied research [31]. Therefore, basic research is less efficient than applied research. For effective decision-making units, the applied research efficiency of 15 universities including Peking University is in a DEA effective state, accounting for 46.88% of the total number of decision-making units, and the inputs and outputs of scientific research were relatively optimal. In contrast, the other 17 universities were in a non-DEA effective state. Although the number of universities in a non-DEA effective state has decreased compared with that of basic research, there was still a gap between universities.

From the perspective of dynamic efficiency, for overall efficiency, the average annual total factor productivity, technological progress efficiency and technical efficiency of applied research in the sample universities from 2015 to 2017 were 1.027, 0.844 and 1.227, respectively, indicating that the total factor productivity of applied research of item universities increased by 2.7% annually during the study period. The efficiency of scientific research innovation in applied research is effective and significantly improved, mainly due to the increase in technical efficiency, which was different from basic research. However, the efficiency of technological progress showed an annual decline. That is to say, the sample universities made continuous progress in scientific research management and input-output ratio control of applied research, while the improvement of scientific research and technological innovation was weak from 2015 to 2017. For an individual, the total factor productivity of 14 universities was less than 1, and that of 18 universities was greater than 1, accounting for 56.25% of DMU. The overall efficiency of applied research was also increasing year by year. In the future, key breakthroughs should be made based on maintaining advantages. While these advantages are being consolidated, their scientific research management and innovation capabilities should be comprehensively improved [32].

5.3. Analysis of Differences between Sample Universities

5.3.1. Individual Differences

As shown in Figure 2, the static and dynamic research efficiency of basic research and applied research was distinguished by quadrifid graphs, in which the vertical coordinate represented the static research efficiency and the horizontal coordinate represented the dynamic research efficiency.

The static scientific research efficiency of universities in region I was higher than the average of sample universities, and the total factor productivity was greater than 1, showing the best performance. The static scientific research efficiency of universities in region I was high, and the total factor productivity showed an increasing trend, indicating that the scientific research constructive effect of these universities was admirable, the management level and scientific research innovation ability were also high. The static scientific research efficiency of universities in region II was higher than the average of sample universities, but the total factor productivity was less than 1. Although the current level of universities in region II was high, their advantages have been declining year by year. The decrease in dynamic scientific research efficiency was caused by the decline of technological progress efficiency or technological efficiency. Therefore, under the premise of ensuring their advantages, universities should enhance their ability of scientific research innovation and technological progress by improving scientific research innovation systems and introducing high-level talents. Moreover, universities should optimize the structure of input and output and improve scientific research management to solve excessive investment in scientific research resources [33]. The static scientific research efficiency of universities in region III was lower than the average of the sample universities, and the dynamic scientific research efficiency was less than 1, showing that the problems of scientific research management in this region need to be solved urgently. Universities in region III should take precise measures to optimize the operation mechanism of scientific research. The static scientific research efficiency of universities in region IV was lower than the average, but the dynamic scientific research efficiency was greater than 1. Although the scientific research efficiency of universities in region IV was low, it increased over years. The universities in region IV would properly adjust the input of scientific research resources, strictly enforce the scientific research management system, and constantly improve their scientific research efficiency based on consolidating the existing advantages, to get closer to the universities in region I.

5.3.2. Group Differences

Fuzzy clustering was performed according to the type of school, and the 32 sample universities were divided into comprehensive, science and engineering, teacher training and agriculture and forestry. The static and dynamic research efficiency of basic and applied research were subjected to separate one-way ANOVAs and all p-values were less than 0.05. This would also be able to indicate a statistically significant difference between research efficiency and university types.

As shown in Figure 3, both static research efficiency of basic research in normal universities and comprehensive universities gave an excellent performance. The proportion of basic research resources input and output of these universities is larger, so it was not difficult to explain that their basic research was more efficient. Meanwhile, the total factor productivity of basic research in comprehensive universities increased rapidly, and the gap with normal universities narrowed year by year. It was predicted that they would become the leader in the field of basic research via their abundant funding, balanced discipline layout as well as strong research team before long. However, the static efficiency of applied research in normal universities and comprehensive universities was low, and the total factor productivity grew slowly or even declined. It was recommended to strengthen the scientific research management of applied research, transform the input-output structure, improve the utilization rate of resources, and actively build a platform for the transformation of scientific and technological achievements, to combine the advantages of universities with the needs of enterprises and social science and technology, and enhance the social service capabilities of universities [34].

The efficiency of applied research in universities of science and technology and agriculture and forestry was good, and the total factor productivity of applied research increased strongly. The applied research efficiency of science & engineering and agriculture & forestry universities continued to lead due to the nature of universities and the large proportion of research resources. Furthermore, their dominant disciplines focus on applied natural sciences, scientific research output was quicker than basic research, and the transformation of scientific and technological achievements was easier, so the scientific research efficiency of applied research was higher than basic research [35]. The static efficiency of basic research in science and engineering and agriculture and forestry universities was low, but the total factor productivity of science and engineering was growing, and the gap with other universities would be narrowed before long. In contrast, the total factor productivity of agriculture and forestry declined. It was suggested that they attach importance to the management of basic research and improve the basic research efficiency, to lay a foundation for solving the “three rural issues” and serving the construction of rural revitalization.

6. Conclusions and Suggestions

6.1. Research Conclusions

This study first started from the theoretical analysis and constructed an evaluation index system of “Categorical evaluation of scientific research efficiency “ to distinguish the scientific research efficiency of basic research and applied research. Secondly, based on the panel data from 2015–2017, the DEA model and Malmquist index was applied to empirically analyze the scientific research efficiency of 32 “Double First-Class” universities directly under the Ministry of Education in China. The research conclusions and suggestions were as follows:

First, although the basic research of the “Double First-Class” universities directly under the Ministry of Education in China has made some important original achievements in recent years, the overall level of static scientific research efficiency needs to be further improved. From the efficiency segmentation, the input-output structure is not reasonable, resulting in low efficiency. The strong increase in the efficiency of technological progress has led to rapid growth in the average annual total factor productivity. It can be predicted that the basic research innovation efficiency will improve rapidly along with the growth of the resources. Nevertheless, we should also pay attention to basic research management to make up for the lack of technical efficiency.

Second, compared with basic research, the overall static research efficiency of applied research is relatively high, the results of efficiency segmentation are generally consistent with basic research, and the scientific research management is not reason enough to lead to an imbalanced input-output ratio and insufficient utilization of resources. The strong increase in the efficiency of technological progress has led to rapid growth in the average annual total factor productivity. The efficiency of applied research will also increase obviously in the future, but the growth rate is not as fast as that of basic research. Meanwhile, based on consolidating its advantages, we should pay attention to improving the ability of scientific research innovation and technological progress, to better fulfill the function of social service in universities.

Last but not least, there are obvious differences in scientific research efficiency between the two-first-class universities directly under the Ministry of Education: although the efficiency of basic research and applied research varies among universities, it can be found that the basic research efficiency of comprehensive and normal universities is higher on the whole, while the applied research efficiency of science & technology and agriculture & forestry universities has an excellent performance. Furthermore, through the comparison of total factor productivity, it is speculated that in terms of basic research efficiency, the scientific research efficiency of comprehensive colleges and universities will rapidly improve until they catch up with normal colleges and universities, and the gap between science and engineering and other colleges and universities will be further narrowed. In terms of applied research, the scientific research efficiency of science & technology and agriculture & forestry will continue to lead, normal universities will slowly improve, while comprehensive universities will maintain the status quo or even decline.

6.2. Suggestions

First, universities should change the traditional way of scientific research evaluation, distinguish basic research from applied research, and identify and solve problems in a targeted manner.

The traditional evaluation of scientific research cannot reflect the specific reality of basic research and applied research at the same time. Concrete analysis can effectively solve the problem that the huge output of one research always covers the much smaller output of the other. This study verified the scientific nature of the scientific research evaluation method based on research conditions through empirical research. Specific analysis can more directly find the problems of scientific research activities and management of universities, and specifically, adjust the allocation of scientific research resources between basic research and applied research, to achieve the purpose of classification management and common improvement [36]. From the research results, comprehensive universities and normal universities have higher efficiency of basic research and lower efficiency of applied research, while science and engineering and agriculture and forestry universities have higher efficiency of applied research and lower efficiency of basic research. In the future, colleges and universities should make key breakthroughs in their weaknesses while consolidating their advantages: comprehensive and normal colleges and universities should strengthen the output guidance of applied research, be market-oriented, rely on scientific research projects, and combine their advantages with the needs of social science and technology. Science and engineering and agriculture and forestry universities should increase the scientific research management of basic research and improve the discipline layout and interdisciplinary integration to promote their basic research level and ability.

Second, universities practically increase investment in scientific research, build a scientific investment evaluation mechanism, and ensure the efficient use of scientific research resources.

Compared with applied research, the investment in scientific research resources in basic research is growing faster, which also reflects China’s “strengthening basic research, Focus on original innovation” policy orientation. Nevertheless, compared with developed countries, there is still a big gap in the total investment and the proportion of basic research in Chinese universities. So, it is necessary to continue to increase investment to ensure the smooth progress of basic research. Compared with basic research, applied research can predict the practicability and value of research. Therefore, colleges and universities strive to promote university-enterprise cooperation, broaden the funding and project sources of applied research, promote the in-depth integration of industry, university and research, and promote the smooth transformation of scientific and technological achievements in universities [37]. At the same time, based on increasing scientific research investment, it is necessary to improve the utilization efficiency of resources. The results of empirical analysis show that the current resource utilization rate of colleges and universities is low. Universities should improve the scientific research investment evaluation mechanism and establish a coordinated allocation mode of stable and competitive investment. Stable input provides a stable guarantee and support for scientific research activities, while competitive input adjusts and reallocates resources according to the supervision of the scientific research process, reduces inefficient personnel and expenditure, and effectively guarantees the efficient use of resources.

Third, the scientific and technological innovation capabilities of colleges and universities should be enhanced, basic frontier research and application technology exploration should be strengthened constantly, and the construction of a scientific and technological power should be solidly promoted.

As the vanguard of scientific and technological innovation, “Double First-Class” universities must continue to strengthen basic frontier research and application technology exploration, and steadily improve their scientific and technological innovation capabilities: Above all, they must organize and implement many major national scientific research projects, such as artificial intelligence, quantum information, life sciences, aerospace Technology and other forward-looking and strategic fields, trying to break the international monopoly of scientific research and technology, solving the “stuck neck” problem that affects China’s economic and social development, and striving to achieve “overtaking on curves” in some important areas and seize the future [38]. Then, we must strengthen the construction of scientific research bases and laboratories, improve the original state key laboratories, and encourage universities, research institutes and corporate scientific research departments to join forces to actively prepare for the establishment of experimental bases in emerging cross-cutting frontier fields, and actively promote the optimal allocation of scientific research resources and The sharing of technical equipment provides a strong hardware guarantee for improving the ability of scientific and technological innovation. Last, we need to cultivate and bring in outstanding scientific and technological personnel. Under the principle of “respecting talents, knowledge and innovation”, we need to deepen the reform of the personnel development system, train more first-class research leaders and scientific and technological innovation teams, and reserve more science and technology reserves for basic and applied research. Last but not least, we must strengthen the cultivation of advantageous disciplines and the cross-integration of disciplines, scientific and technological innovation is often backed by the dominant disciplines of colleges and universities, and basic frontier research also relies on the close cross-integration between disciplines, so “Double First-Class“ universities should cultivate their advantageous disciplines, take into account the expansion of discipline layout through exchanges and cooperation between colleges and universities, and lay the disciplinary foundation for scientific research and innovation.

7. Discussion

This study adopts a categorical evaluation method to evaluate the research efficiency of basic and applied research in 32 “Double First-Class” universities in China and makes targeted advice. It can not only promote the sustainable development of the research system of the sample universities but also provide a new idea for the evaluation of research efficiency. Meanwhile, this study still has some limitations worth discussing. First, limited by the availability of sample data, this paper only analyzes data from 2015 to 2017 and does not include sample data from longer periods. The timeliness of interpretation can be improved by continuous research in the future. Second, this paper explored the group variability of research efficiency in the sample universities only in terms of the types of university. However, factors, such as regional economic level also have an impact on the differences in research efficiency. Third, in response to the findings of the study, the article focuses on the suggestions for the sustainable development of universities’ research activities from a macro perspective, but some studies based on a micro perspective are also very meaningful. For instance, Dongbin Kim examined Chinese faculty members’ perceptions of the pursuit of “World-Class” universities from a glocal (global, national and local) heuristic perspective and how they struggled with the multiple dimensions of academic life [39]. Through a quantitative study, Yan H found that there were significant differences in research funding access for researchers with different mentorship identities. The academic identity of the tutor played a leading role in the acquisition of scientific research funds, while the administrative identity of the tutor was in a subordinate position [40].