Coordinated Evaluation of Technological Innovation and Financial Development in China: An Engineering Perspective

Abstract

Innovation-driven development is the main driving strategy for promoting high-quality economic development. Technological innovation is the core of innovation-driven development. Financial innovation is an important aspect of promoting financial development. As such, the coupling and coordination of the technological innovation and financial development in developing countries, such as China, is an important issue. The topic has been extensively studied over the last decade in the context of China, and a dominating method has emerged on how to model the technological innovation subsystem and the financial development subsystem, and how to quantitatively determine the degree of coupling and coordination of the two subsystems. A variety of predictors have been proposed to model each subsystem. The coupling degree and the coordination degree are then calculated, and then they are used to analyze the current development status for potential issues. However, we make an effort to validate the calculated degree of coupling and coordination before the results are used for the analysis.Without validation, the outcomes of the analysis not only might not be useful but also could lead to inappropriate governmental policies. That said, it is tremendously challenging to validate the results due to the lack of the ground truth. The goal of this study is to work towards objectively determining the reliability of the degree of coupling and coordination from an engineering perspective. Specifically, we accomplish this task by evaluating the regression performance and projection performance. We demonstrate that the use of a carefully crafted set of predictors for each subsystem is the foundation for deriving the reliable coordination degree of the two subsystems.

1. Introduction

Since early 1980s, China’s economy has achieved rapid growth and has become the world’s second largest economy. The economic structure in China has been continuously shifting from the original extension-driven growth predominantly driven by low-cost labor, to endogenous growth driven by innovation. In order to accelerate the construction of the national innovation system, the Chinese government has enacted policies that encourage innovation-driven development. Financial innovation has become a key driving force for financial development in recent years.

From the perspective of system theory [1], technological innovation can be regarded as a subsystem, and financial development can also be regarded as a subsystem. If there is a benign coupling and coordinated interaction between the two systems of technological innovation and financial development, and they promote each other and develop together, they create a win–win situation. However, in practice, the two subsystems may suffer from a number of issues, such as insufficient interaction, mismatch between supply and demand, and uncoordinated development. Therefore, it is of great significance to examine the coupling and coordinated development of the two systems, promote the combination of technology and finance, and achieve coordinated development, in order to achieve stable economic development. As such, the relationship between technological innovation and financial development has been extensively studied over the last decade [2,3,4,5,6,7,8,9], and a dominating method has emerged on how to model the technological innovation subsystem and the financial development subsystem, and how to quantitatively determine the degrees of coupling and coordination of the two subsystems. A variety of predictors have been proposed to model each subsystem. The degree of coupling and the degree of coordination are then calculated, and then they are used to analyze the current development status for potential issues.

However, we make an effort to validate the calculated degrees of coupling and coordination before the results are used for the analysis.Without validation, the outcomes of the analysis not only might not be useful but also could lead to inappropriate governmental policies. That said, it is tremendously challenging to validate the results due to the lack of the ground truth. The goal of this study is to work towards objectively determining the reliability of the degree of coupling and coordination from an engineering perspective. Specifically, we accomplish this task by evaluating the regression performance and projection performance. We demonstrate that the use of a carefully crafted set of predictors for each subsystem is the foundation for deriving a reliable coordination degree of the two subsystems.

The remainder of this article is organized as follows. Section 2 outlines the related work and highlights the differences between our study and the related studies. Section 3 describes the mainstream theory on modeling the coupling and coordination of two correlated subsystems. In this section, we also elaborate the set of 12 predictors for the technological innovation subsystem, and the set of 12 predictors for the financial development subsystem that we propose for the current study. Section 4 reports the data source that we use to calculate the predictors. Section 5 elaborates the details of data processing using the model we describe in Section 3, and the results, including the comprehensive evaluation index for each of the two subsystems, the coupling degree, and the coordination degree. To demonstrate the importance of using the right set for each subsystem, all calculations are performed, and all results are presented side by side with the predictors that we introduce and the predictors proposed in the first significant study on the coupling and coordination of the two subsystems [3]. This section also presents the objective evaluation on the quality of the calculated degrees of coupling and coordination. Section 6 reports an in-depth discussion on several aspects of our study, including three alternative configurations (using a reduced predictor set, using alternative imputation methods, and using an alternative weighting method), and statistical comparison of the outcomes derived from different configurations, including using our dataset and the reference dataset. Section 7 concludes this article and identifies the limitations of the current study. Furthermore, four appendices are included to report detailed information that might not be critical for understanding our study, but could be of interest to readers, including the predictors introduced in related studies, the coupling degree matrices, and the coordination degree matrices.

2. Related Work

Our study falls under the umbrella of the general system theory [1], where a system is modeled as consisting of cohesive groups of interrelated and interdependent components (or subsystems). Naturally, the coupling and coordination of the subsystems would be of interest in understanding the system. One of the first publications on the coupling of two systems in the field of socio-economy is [10] (published in 1947), where a quantitative model was proposed to calculate the degree of coupling between two correlated systems, particularly focusing on the markets with production lags. The concept of loosely coupled systems and the necessity of evaluating the strength of loosely coupled systems in the context of educational organizations was proposed in [11].

2.1. Coupling and Coordination Studies

The coupling and coordination between a variety of correlated subsystems are seen in the field of finance and economic. These studies have considered the coupling and coordination between environmental pollution and economic development [12], human capital and economic growth [13,14], tourism industry and regional economic development [15], resource endowment, ccological environment, and economic development [16], digital economy and green finance [17], intellectual property and economy [18], urbanization and ecological environment [19]. It is also interesting to note that in [20], the coupling and coordination calculation is used to determine predictors that indicate obstacles in economic development by comparing the coupling and coordination between different predictors (a total of 10 subsystems are considered based on first-level predictors).

Several studies focused on proposing coupling mechanisms between technological innovation and financial development. In [21], the important of the coupling and coordination between technological innovation and financial development is recognized and advocated appropriate policies to facilitate the coupling and coordination. In [22], three levels of mechanisms are proposed to enhance the coupling and coordination of technological innovation and financial development: (1) financing of science and technology industrial parks; (2) setting up technology financial institutions; and (3) subsidizing market-oriented operation. In [23], the coupling mechanisms proposed include the setup of independent regulatory Institutions, encourage the integration of research and development in enterprises, provide support for big data and wide-band connectivity, and enact suitable governmental policies. In [24], a dynamic model to facilitate the coupling and coordination of the two subsystems is proposed. The model consists four feedback loops with similar steps: financial investment on research institutions, which would produce more papers and patents, then in turns, the intellectual properties could lead to new products and the growth of the local economy. The improved economy would generate more taxes, which makes it possible for the local government to invest more.

Several studies [2,3,4,5,6,7,8,9] adopted the same set of models and formula for calculating the degree of coupling and coordination between technological innovation and financial development. Some studies [25] described the formula used to calculate the comprehensive evaluation index, the coupling degree, and the coupling and coordination degree with citations but did not explicitly provide the formulas. From the context, it appears that the formulas are likely the same.

In [2], a total of six predictors for both the technological innovation and financial development are used to model the coupling between new industry and traditional industry, instead of treating technological innovation and financial development as two subsystems.

In [3], the focus is on applying the set of formulas to discover the economical development in China based on the 2002–2012 data, while in [4], the focus is on examining the impact on economic efficiency without presenting the detailed outcomes of the degree of coupling and coordination. In [3], two categories of seven predictors are used to model the technological innovation subsystem, and three categories of seven predictors are used to model the financial development subsystem. The proposed model is applied to data collected in China between 2002 and 2012.

In [5], two categories of four predictors are used for the technological innovation subsystem, and three categories of three predictors (one per category) are used for the financial development subsystem. The model is applied to the 2008–2013 data.

In [26], four predictors for the technological innovation subsystem are introduced (no category is created), and three predictors for the financial development subsystem (no category is created). The way the coupling degree C is calculated is non-mainstream. The range for C ranges between −1 and 1 instead of between 0 and 1. Furthermore, the model assumes that the comprehensive index for each subsystem would significantly change over the period of time of interest.

In [6], the model is applied to four regions around the Yangtze river using the 2005–2016 data. For the technological innovation subsystem, three categories of eight predictors are used. For the financial development subsystem, also three categories of eight predictors are used.

In [27], the coupling and coordination between technological innovation and industrial structure optimization, as well as the coupling and coordination financial innovation and industrial structure optimization, are studied. The formulas used are not explicitly provided but are presumably similar to those of [3,4,28]. For the technological innovation subsystem, three categories of eight predictors are used. For the financial development subsystem, two categories of nine predictors are used. This study examines the China 2013–2017 data.

In [8], the coupling and coordination of three subsystems instead of two are studied, including culture, technological innovation, and financial development. For the technological innovation subsystem, five predictors of three categories are used. For the financial development subsystem, also five predictors of three categories are used. This study uses the China 2005–2016 data.

In [9], the coupling and coordination between technological innovation, green finance, and industrial policies are studied. Seven predictors of two categories are used to model the technological innovation subsystem. This study uses the China 2008–2015 data.

In [25], 10 predictors for three categories of technological innovation, and 12 predictors for three categories of financial development are used to study the coupling and coordination between different geographical regions instead of those of the two subsystems using China 2006–2016 data.

In [29], the relationship between finance policy and the innovation performance from small-to-medium high-tech businesses is studied. The focus of this study is quite different from (i.e., much narrower scope than) other studies that aim to investigate the correlation between financial development and technological innovation in general. This study uses five predictors for finance policy, and four predictors for the innovation performance. The model used to compute the coupling and coordination degree is slightly different from that adopted in [3,4].

2.2. Coupling and Coordination Modeling

To our best knowledge, the first modern theory on system coupling and coordination in the field of socio-economy is attributed to [30]. In later studies, the theory is refined and extended in a large array of studies [3,12,15]. The actual model is presented in Section 3.

The coupling and coordination requires the calculation of a comprehensive index for each subsystem. Various methods have been proposed. In [14], a genetic algorithm is used to estimate the weights. In [31], principal component analysis is used to determine the weights. In [32], in addition to entropy-based weighting, another method called Criteria Importance Through Intercriteria Correlation (CRITIC) [33] is also used to calculate the weights. CRITIC measures the weight of an indicator by evaluating its volatility and conflict. The standard deviation and correlation coefficient are employed to describe volatility and conflict, respectively. Furthermore, machine learning has also been used to predict the economic development level [34], where the ARIMA time series model, LSTM model, XGBoost model, and a hybrid ARIMA-LSTM-XGBoost model based on them are used. Entropy-based weighting appears to be the most popular method by far [35,36,37,38,39].

2.3. Comparison with Related Studies

The coupling and coordination between technological innovation and financial development, which is the focus of the current study, has been extensively studied previously as we have elaborated above. Furthermore, we also choose to use the dominating model for coupling and coordination. Despite the similarity to existing studies, we make a number of research contributions:

• We use a more comprehensive set of predictors to describe each subsystem than those in existing studies. In Table 1, we summarize the predictors used for related studies and ours. In Section, we demonstrate that by using a comprehensive set of predictors, the predictability and consistency of the coupling and coordination degree are stronger than previous studies.
  Table 1. Comparison of the number of predictors used for the technological innovation subsystem and the financial development subsystem.

  Reference	Technological Innovation Subsystem	Financial Development Subsystem	Data
  [3]	2 categories of 7 predictors	3 categories of 7 predictors	China 2002–2012
  [4]	3 categories of 7 predictors	2 categories of 6 predictors	35 Countries 2004–2010
  [5]	2 categories of 4 predictors	3 categories of 3 predictors	China 2008–2013
  [26]	4 predictors (no category)	3 predictors (no category)	China 2002–2012
  [6]	3 categories of 8 predictors	3 categories of 8 predictors	China 2005–2016
  [27]	3 categories of 8 predictors	2 categories of 9 predictors	China 2013–2017
  [8]	3 categories of 5 predictors	3 categories of 5 predictors	China 2005–2016
  [9]	2 categories of 7 predictors	not applicable	China 2008–2015
  [25]	3 categories of 10 predictors	3 categories of 10 predictors	China 2006–2016
  Our study	3 categories of 12 predictors	3 categories of 12 predictors	China 2002–2023

• We perform pre-processing of the data, and use regression to fill the missing data. Although this step is essential in machine learning-based studies, it has not been used in the competing studies to our knowledge. By filling the missing data, we are able to investigate the coupling and coordination degree for a wider range of durations.
• We perform regression-based prediction of future trend on the coupling and coordination degree. Again, this is not reported in competing studies. The reliable prediction of future trend is instrumental to enacting optimal government policies for financial development. A side benefit of doing this is that the quality of the selected predictors can be indirectly assessed by the errors of the regression. The rationale is that if the predictors can truly reflect the development stage of the subsystem, and there is no drastic change of governmental policies, the development of the subsystem would change gradually and easily fit into a regression model.

3. Coupling and Coordination Modeling

We adopt the mainstream coupling and coordination modeling in the field of socio-economy [3,12,15,30]. In this section, we introduce the specific predictors for the technological innovation subsystem and those for the financial development subsystem, the method to calculate the comprehensive evaluation index of each subsystem based on the predictors, the method to calculate the degree of coupling between the two subsystems, and the coordination degree of the two subsystems.

3.1. Predictors

A predictor is a metric that can capture a specific aspect of the subsystem. As we will demonstrate in this paper, the selection of appropriate predictors is essential to establishing robust comprehensive evaluation index, which will ultimately impact the coupling and coordination degree for the two subsystems. The construction of predictors for the technological innovation system focuses on the facts that reflect innovation efficiency [40,41]. The predictors for the financial development are based on the theory of financial structure [42], and the integration of quantitative finance and qualitative finance [43]. Ideally, we want to have a full set of predictors that could reflect all aspects of the subsystem. Unfortunately, that is not always possible, e.g., due to the limited public data. As such, we decide to use 12 predictors (in 3 categories) for each subsystem as described in Table 2.

Table 2. Categories and predictors used for the two subsystems in our model.

Subsystem	Category	Predictors	Unit
Technological Innovation	Environment ($X_1$)	Gross Domestic Product (GDP) per capita ($X_{11}$)	Yuan/person
		Average number of students enrolled in higher education institutions per 100,000 people ($X_{12}$)	Person
		Number of Internet broadband access ($X_{13}$)	${10}^4$
		Number of high-tech enterprises in each region ($X_{14}$)
		Number of research and development (R&D) institutions by region ($X_{15}$)	Item
	Investment ($X_2$)	Local fiscal expenditure on science and technology ($X_{21}$)	${10}^8$ Yuan
		R&D funding intensity ($X_{22}$)	%
		Internal expenditure on R&D ($X_{23}$)	Yuan
		Full-time equivalent of R&D personnel by region ($X_{24}$)	Person
	Output ($X_3$)	Technology market transaction amount ($X_{31}$)	${10}^4$ Yuan
		Number of patents granted ($X_{32}$)	Item
		Number of scientific papers indexed by major international publication repositories by region ($X_{33}$)	Item
Financial Development	Scale ($Y_1$)	Proportion of financial practitioners ($Y_{11}$)	%
		Bank balance ($Y_{12}$)	${10}^8$ Yuan
		Bank loan balance ($Y_{13}$)	${10}^8$ Yuan
		Total market value of stocks ($Y_{14}$)	${10}^8$ Yuan
		Premium income ($Y_{15}$)	${10}^8$ Yuan
	Structure ($Y_2$)	Savings per capita ($Y_{21}$)	Yuan/person
		Number of securities industry institutions ($Y_{22}$)	Item
		Insurance depth ($Y_{23}$)	%
		Insurance density ($Y_{24}$)	Yuan/person
	Efficiency ($Y_3$)	Loan-to-deposit ratio ($Y_{31}$)	%
		Securitization rate ($Y_{32}$)	%
		Insurance claims ratio ($Y_{33}$)	%

3.2. Comprehensive Evaluation Index

The comprehensive evaluation index U quantifies how well the subsystem is developed based on the setup predictors. More specifically, U is defined by Equation (1):

$$U=\sum _{i_1}^n{w}_i{x}_i$$

(1)

where $x_i$ ($i=1,2,\dots ,n$) is the i-th normalized predictor for the subsystem, $w_i$ is the weight for predictor $x_i$, and n is the total number of predictors for the subsystem. As such, U is in the range of 0 and 1, with 0 being the worst and 1 being the best. A key step is to estimate the weight for each predictor. The most dominating method is the entropy weighting method [44]. The first step in calculating the weight for each predictor is to normalize the data across all the years in the dataset. Several normalization methods could be used. In the context of our study, the min-max normalization is used, where each data point is normalized into the range of [0, 1] as defined in Equation (2):

$$p_{ij}={\displaystyle \frac{x_{ij}-min\left(x_j\right)}{max\left(x_j\right)-min\left(x_j\right)}}$$

(2)

where $x_{ij}$ is the i-th value for the j-th predictor, $p_{ij}$ is the corresponding normalized value, and $x_j$ is the vector for the j-th predictor.

The second step is to calculate the entropy for each predictor according to Equation (3):

$$e_j=-k\sum _{i=1}^m{p}_{ij}ln\left(p_{ij}\right)$$

(3)

where $k=1/ln\left(m\right)$ is a constant to scale $e_j$ to the range of [0, 1], m is the number of values for each predictor, and $e_j$ is the entropy of the j-th predictor.

The third step is to calculate the degree of diversification calculated via Equation (4):

$$d_j=1-e_j$$

(4)

Finally, the weight for each predictor is calculated via Equation (5):

$$w_j={\displaystyle \frac{d_j}{{\sum }_{j=1}^n{d}_j}}$$

(5)

It is worth noting that the weight calculation requires the use of the entire dataset for all predictors in the subsystem. Predictors that have higher variability would have low entropy and therefore higher weight. On the other hand, predictors that have less variability would have high entropy and therefore lower weight. The structure of the comprehensive evaluation index and the overall steps in calculating the index are illustrated in Figure 1.

Figure 1. The structure of the comprehensive evaluation index and the overall steps in calculating the index.

3.3. Coupling Degree

The coupling degree C between two subsystems 1 and 2 quantifies how well the development level of the two subsystems are synchronized with each other. When both subsystems have the same U value, then C would be 1. C is first defined in [30] by Equation (6):

$$C={\displaystyle \frac{U_1{U}_2}{{((U_1+U_2)/2)}^2}}=4{\displaystyle \frac{U_1{U}_2}{{(U_1+U_2)}^2}}$$

(6)

Although C is guaranteed to be within the range of 0 and 1 by Equation (6), it works only when U is strictly unitless. Hence, it is not surprising in later publications [12,15] that C is calculated by making a square root of the right-hand side of Equation (6) as defined in Equation (7):

$$C=2{\displaystyle \frac{\sqrt{U_1{U}_2}}{(U_1+U_2)}}$$

(7)

3.4. Coordination Degree

The coordination degree focuses on the subtle relationship between the two subsystems in that the two subsystems should not only be synchronized in their development but also have a symbiotic relationship that makes each other better. D is defined in [30] by Equation (8):

$$D=\sqrt{C·T}$$

(8)

where T is the combined evaluation index for $U_1$ and $U_2$, and it is defined as $\alpha U_1+\beta U_2$, and $\alpha +\beta =1$. If system 1 and system 2 are of equal importance, which is the case of our current study, $\alpha =\beta =0.5$. As can be seen, unlike C, only when both $U_1$ and $U_2$ are 1 would D be 1. If both $U_1$ and $U_2$ are at a low level of development, D would be close to 0 as well.

4. Data Source

The data for the scientific and technological innovation evaluation index system come from the “China Statistical Yearbook” [45] and the “China Science and Technology Statistical Yearbook” [46]; the data of the financial development evaluation index system come from the “China Finance Yearbook” [47], “China Statistical Yearbook” [45], and “China Regional Financial Operation Report” [48]. The data corresponding to each indicator are processed using the range standardization method, and the four decimal places are uniformly taken, and all the data that are 0 after standardization are replaced with 0.0001 to facilitate logarithmic calculation. The data collected span 2002–2023 and 30 provinces (including autonomous regions and municipalities) in China.

5. Data Processing and Results

To demonstrate our research contribution, it is necessary to compare some of the previous studies that used a different set of predictors. We choose to compare with the first study that adopted the same approach [3], which has 7 predictors for each subsystem. Hence, in each of the following subsections, we present the findings of each dataset side by side. The original study [3] only includes data between years 2002 and 2012. To facilitate direct comparison, we extend the data to 2002–2023 based on the data source.

More precisely, our dataset O consists of the data for all 30 regions between years 2002 and 2023 for each predictor. Recall that we have 12 predictors ($X_{11}$, $X_{12}$, $X_{13}$, $X_{14}$, $X_{21}$, $X_{22}$, $X_{23}$, $X_{24}$, $X_{31}$, $X_{32}$, and $X_{33}$) for the technological innovation subsystem, and 12 predictors ($Y_{11}$, $Y_{12}$, $Y_{13}$, $Y_{14}$, $Y_{21}$, $Y_{22}$, $Y_{23}$, $Y_{24}$, $Y_{31}$, $Y_{32}$, and $Y_{33}$) for the financial development subsystem.

The reference dataset R consists of the data for all 30 regions between years 2002 and 2023 for each predictor. There are seven predictors ($X_{11}$, $X_{12}$, $X_{13}$, $X_{14}$, $X_{15}$, $X_{16}$, and $X_{17}$) for the technological innovation subsystem. There are also seven predictors ($Y_{11}$, $Y_{12}$, $Y_{13}$, $Y_{14}$, $Y_{15}$, $Y_{16}$, and $Y_{17}$) for the financial development subsystem.

The compiled datasets O and R are saved as two separate Excel sheets. All data pre-processing and processing are carried out using the Python 3.9 programming language on an iMac computer with 64 GB memory. The datasets and all the Python scripts are made available as a GitHub project (https://github.com/wenbingcsu/ccd).

5.1. Data Pre-Processing

Data pre-processing involves two steps: (1) outliner detection, and (2) fill the missing data. For outlier detection, we consider two statistical methods, where one is the z-score method, and the other is the Interquartile Range (IQR) method. The outlier detection gives us an idea of the quality of the data. In the z-score method, a z-score is calculated for each data value based on how many standard deviations it is from the mean. If the z-score for a data item is greater than 3, then it is considered an outlier. The IQR method determines the outliers based on the spread of the middle 50% of the data, and the outliers are values outside of a range bounded by $Q1-1.5\times IQR$, and $Q3+1.5\times IQR$, where $Q1$ is the first quantile value, $Q3$ is the third quantile value, and $IQR$ is the difference between $Q3$ and $Q1$. It is apparent that the $IQR$ method would label more data items as outliners than the z-score method. We perform outlier detection for each predictor for each year of data that are available. Unlike developed countries, regions within China differ significantly in their financial and technological development levels. As such, even if some data items are identified as outliers statistically, they might not be real outliers due to measurement errors. As such, we choose not to replace the values of the outliers for the sake of authenticity of the study.

We log the number of outliers and produce plots for each predictor detected by both outlier detection methods. The maximum number of outliers is 1 in each year of data (among the 30 regions) based on the z-score method, while the number of outliers ranges between 0 and 5 based on the IQR method. Because we choose to use 12 predictors for each subsystem, there are a total of 48 plots for our dataset O. Readers who are interested in viewing these plots may generate these plots via an our source code. Here we only show in Figure 2 two plots for the predictor $X_{11}$, one on the number of outliers based on the z-score method, and the other based on the IQR method.

Figure 2. The number of outliers determined by the z-score method (left), and those by the IQR method (right), both for predictor $X_{11}$.

As much as we all would like to have access to all the data, it is inevitable that some data points are not available or invalid. In Table 3, we report the missing data status for each predictor in our dataset O. In Table 4, we report the missing data status for the predictors in the reference dataset R. The detailed definitions of the predictors in the reference dataset R are provided in Appendix A as part of Table A1 and Table A2.

Table 3. Missing data status in our dataset O.

X Predictor	Missing Data Status	Y Predictor	Missing Data Status
$X_{11}$	No missing data	$Y_{11}$	No missing data
$X_{12}$	2002, 2003 data missing	$Y_{12}$	2023 data missing
$X_{13}$	2002, 2003, 2004 data missing; 2006 partial data missing	$Y_{13}$	2023 data missing
$X_{14}$	2002, 2003, 2004 data missing	$Y_{14}$	2002, 2003, 2004, 2005, 2006 partial data missing; 2023 data missing
$X_{15}$	2002, 2003, 2004, 2020, 2021, 2022, 2023 data missing	$Y_{15}$	2023 data missing
$X_{21}$	No missing data	$Y_{21}$	2023 data missing
$X_{22}$	No missing data	$Y_{22}$	2022, 2023 data missing; 2004, 2005 partial data missing
$X_{23}$	No missing data	$Y_{23}$	2002, 2003, 2022 partial data missing; 2023 data missing
$X_{24}$	No missing data	$Y_{24}$	2002, 2003, 2022 partial data missing; 2023 data missing
$X_{31}$	No missing data	$Y_{31}$	2023 data missing
$X_{32}$	No missing data	$Y_{32}$	2002, 2003, 2004, 2005, 2006 partial data missing; 2023 data missing
$X_{33}$	2023 data missing	$Y_{33}$	2023 data missing

Table 4. Missing data status in the reference dataset R.

X Predictor	Missing Data Status	Y Predictor	Missing Data Status
$X_{11}$	No missing data	$Y_{11}$	No missing data
$X_{12}$	No missing data	$Y_{12}$	2023 data missing
$X_{13}$	No missing data	$Y_{13}$	2023 data missing
$X_{14}$	No missing data	$Y_{14}$	2023 data missing
$X_{15}$	No missing data	$Y_{15}$	2002, 2003, 2004, 2005, 2006 missing partial data; 2023 data missing
$X_{16}$	No missing data	$Y_{16}$	2023 data missing
$X_{17}$	2002, 2003, 2004, 2008 missing partial data	$Y_{17}$	2023 data missing

We decide to use regression to fill the missing data. For each predictor, a Python script is developed to automatically identify missing data on a per row basis (i.e., for each region over the years 2002–2023). If any missing data item is found, the available data for the same region are used to train the regression model, and the trained regression model is then used to fill the missing data items.

Considering that there are numerous regression models that we can choose from, we need to decide one that would produce the best estimated missing data. Because we are filling missing data, there is no direct way of determining the accuracy of the estimated missing data via regression. The best that we can do is to make the following hypothesis: if the mean squared error (MSE) of the regression on the available data for the region is smallest, then the estimated missing data would have the best accuracy. To be practical, we experiment with two regression models: (1) linear regression with polynomial features (LRPF), and (2) gradient boosting regression (GBR). For LRPF, we use three-fold cross-validation to tune the polynomial degree as the only tunable parameter. For GBR, we choose to use the default parameters. The training MSE is collected for both LRPF and GBR for whenever a missing data item is present. The histograms (with 100 bins) of MSE for LRPF and GBR are shown in Figure 3. As can be seen, although the shapes of the histograms are quite similar, with the great majority of MSE close to the smallest bins, the MSE for GBR is ${10}^4$ smaller than that for LRPF. Hence, it is obvious that we should use GBR to generate missing data. Nevertheless, the presence of a few occurrences at very large MSE bins indicate that some of the generated data could be of low quality. How to further enhance the quality of the generated data is left for future research because the fraction of potentially bad data is very small (which would have a negligible negative impact on the results).

Figure 3. Comparison of MSE for linear regression (with polynomial features) and gradient boosting regression on the raw data for all predictors in the dataset O.

For completeness, we also show the MSE distribution of regression for generating the missing data for the reference dataset in Figure 4. The primary purpose is still to demonstrate which regression model should be used to generate the missing data. Similar to the case for generating missing data in the dataset for our current study, GBR again outperforms LRPF with much smaller MSE for the reference dataset.

Figure 4. Comparison of MSE for linear regression (with polynomial features) and gradient boosting regression on the raw data for all predictors in the reference dataset R.

5.2. Determination of the Weights of Predicators

The objective-weighting Python library (https://pypi.org/project/objective-weighting/) (accessed on 1 March 2025) is used as the basis for the calculation of the weights of the predictors. The reason why we cannot use that library as it is is because the entropy weighting function uses a sum normalization instead of the min-max normalization. The purpose of sum normalization is to scale the values in a vector so that the sum of the scaled values equals to 1. On the other hand, the min-max normalization would scale all values to a range between 0 and 1. Hence, for our study, the min-max normalization is used. The source code of the entropy weighting function in the objective-weighting python library is modified accordingly. The weights calculated this way are shown in Figure 5 for our dataset, and shown in Figure 6 for the reference dataset.

Figure 5. The weights for 12 predictors for the technological innovation subsystem (left), and the weights for the 12 predictors for the financial development subsystem (right) in the dataset O.

Figure 6. The weights for 7 predictors for the technological innovation subsystem (left), and the weights for the 7 predictors for the financial development subsystem (right) in the dataset R.

As can be seen, for the technological innovation subsystem in dataset O, the largest weight is 0.138 (for $X_{12}$), and the smallest weight is 0.035 (for $X_{31}$). For the financial development subsystem in dataset O, the largest weight is 0.149 (for $Y_{23}$), and the smallest weight is 0.046 (for $Y_{32}$).

For the technological innovation subsystem in the dataset R, the largest weight is 0.227 (for $X_{16}$), and the smallest weight is 0.079 (for $X_{17}$). For the financial development subsystem in the dataset R, the largest weight is 0.229 (for $Y_{17}$), and the smallest weight is 0.059 (for $X_{16}$).

The ratio between the largest weight and the smallest weight for all subsystems and for both datasets is roughly between 3 and 4. Hence, no predictor can be apparently ignored during the calculation for the comprehensive evaluation index. In other words, every predictor would make some impact to the comprehensive evaluation index to the subsystem, and eventually may exert some impact on the degree of coupling and degree of coordination.

5.3. Determination of Comprehensive Evaluation Index for Technological Innovation

In this section, we report the comprehensive evaluation index for the technological innovation subsystem. The complete calculated indices for all 30 regions in years 2002–2023 for dataset O is provided in Appendix B, Table A3. The complete indices for the reference dataset R are provided in Appendix B, Table A5.

First, we provide an overview of the indices for all 30 regions in 2002, 2012, and 2023, for the technological innovation subsystem calculated using dataset O in Figure 7.

Figure 7. Comprehensive evaluation indices for the technological innovation subsystem for all 30 regions in China in 2002, 2012, and 2023 calculated using dataset O.

To show the evolution of the indices over the years, we show two regions, Beijing, which consistently has the highest index, and Qinghai, which has the lowest index among all regions, in Figure 8. As can be seen, the general trend for both regions is that the development level (as indicated by the index) is increasing over time, with the exception of occasional slight drops (from 2002 to 2003 for Beijing) or stagnancy (from 2010 to 2011 for Beijing; from 2004 to 2005 and from 2014 to 2015 for Qinghai).

Figure 8. Comprehensive evaluation indices for the technological innovation subsystem for Beijing and Qinghai over the years 2002–2023 calculated using dataset O.

For readers who are not familiar with China, it is necessary to provide some explanation regarding as to why Qinghai consistently has the lowest index and Beijing consistently has the highest index. Unlike many developed countries, regions in China may have huge differences in their development levels due to historical governmental policies, locations, natural and human resources. We contrast the two regions from four dimensions that could impact their development levels:

• Economic and financial infrastructure: Beijing is the capital of China with a large concentration of financial institutions, including international financial headquarters. Qinghai, on the other hand, is located in the far western region of China and much less economically developed with very limited financial infrastructure.
• Technological development: Beijing is home to many top universities in China, and as such, Beijing is leading in patent filings and technology startups. In contrast, Qinghai does not have a top university and lacks the high-tech industrial base.
• Government policy and investment: Being the capital of China, Beijing receives substantial central government funding and policy support. The support for Qinghai is limited to renewable energy projects.
• Human capital and talent pool: Due to the special location and the rich opportunities, Beijing attracts top-tier latent from across China. Beijing also has the best talent pool in China due to its top universities. In contrast, Qinghai faces significant challenges in attracting and retaining skilled workers due to the lack of opportunities and possibly harsh weather. Qinghai also lacks a talent pool.

As can be seen in Figure 9, the development levels across different regions appear rather similar when we use the set of seven predictors proposed in [3]. What is also odd is that Henan has the highest index value in 2002, and Hubei has the highest index value in 2023. The calculated distribution of the indices clearly fail to match the reality in China. Beijing, the capital city of China, has a concentration of the best higher educational institutions in China. As such, Beijing should be expected to have the highest technological innovation index. Henan and Hubei are nowhere near the level of Beijing. This demonstrates that the choice of the set of predictors is very important to deriving the development levels of each subsystem. The issue of using the set of seven predictors is further evident from the evolution of the indices over time as shown in Figure 10. Unlike the gradual but consistent increase in the development levels for Beijing and Qinghai from 2002 to 2023 shown in Figure 8, what is shown in Figure 10 implies that the development levels for Beijing and Qinghai in recent years (2007–2023) are basically stagnant, which clearly fail to reflect the reality because we all know the rapid technological development in China during that period.

Figure 9. Comprehensive evaluation indices for all 30 regions in China in 2002, 2012, and 2023 calculated using the reference dataset R.

Figure 10. Comprehensive evaluation indices for the technological innovation subsystem for Beijing and Qinghai over the years 2002–2023 calculated using the reference dataset R.

5.4. Determination of Comprehensive Evaluation Index for Financial Development

In this section, we report the comprehensive evaluation index for the financial development subsystem. The complete calculated index for all 30 regions in years 2002–2023 using our dataset O is provided in Appendix B, Table A4. The complete index using the reference dataset R for comparison is provided in Appendix B, Table A6. We provide an overview of the indices for all 30 regions in China in 2002, 2012, and 2023, for the financial development subsystem that is estimated using our set of 12 predictors in Figure 11. The calculated indices align with common knowledge of China’s financial development where Beijing, Shanghai, and Guangdong are the top three regions. In recent years, Jiangsu and Zhejiang are getting close to the levels of the top three regions.

Figure 11. Comprehensive evaluation indices for the financial development subsystem for all 30 regions in China in 2002, 2012, and 2023 calculated using the dataset O.

To show the evolution of the indices over the years, we show two regions, Beijing, which consistently has the highest index, and Qinghai, which has the lowest index among all regions in year 2023, in Figure 12. As can be seen, the general trend for Beijing is that the development level (as indicated by the index) is increasing over the time but with more noticeable jittering compared with that of technological innovation. For Qinghai, even though the general trend is also roughly increasing over the years, we see episodes of declines (from 2005 to 2006; and from 2008 to 2010) and stagnancy (from 2018 to 2023).

Figure 12. Comprehensive evaluation indices for the financial development subsystem for Beijing and Qinghai over the years 2002–2023 calculated using the dataset O.

In comparison, the estimated development levels as shown in Figure 13 calculated using the reference dataset R are quite different from ours. Only Beijing still ranks the first in index value. Shanghai does not even rank among the top three. In year 2023, Shanghai is even ranked below Anhui, which does not match the reality. As can be seen in Figure 13, the development levels across different regions appear to be rather similar when we use the reference dataset R.

Figure 13. Comprehensive evaluation indices for the financial development subsystem for all 30 regions in China in 2002, 2012, and 2023 calculated using the reference dataset R.

Again, the issues of using the set of seven predictors are evident from the evolution of the indices over time, as shown in Figure 14. For Beijing, the drastic increase in the development level from 2006 to 2007 is unrealistic. For Qinghai, there are also several apparent issues: (1) It is odd that the development level is already relatively high in 2002 (i.e., the second-highest index second only to the year 2020 value); (2) the drastic drop in development level from 2002 to 2003 cannot be explained; and (3) similarly, the huge drop in development level from year 2020 to year 2021 is also hard to explain.

Figure 14. Comprehensive evaluation indices for the financial development subsystem for Beijing and Qinghai over the years 2002–2023 calculated using the reference dataset R.

5.5. The Coupling Degree Between Technological Innovation and Financial Development

Figure 15 shows the coupling degrees between the technological innovation and the financial development subsystems calculated based on our set of 12 predictors for each subsystem for all 30 regions in 2002, 2012, and 2023.

Figure 15. The coupling degree between the two subsystems for all 30 regions in China in 2002, 2012, and 2023 calculated using dataset O.

As can be seen, except Qinghai, and to some extent Ningxia, the coupling degrees are very close to 1.0 in 2023. Back in 2002, the western areas in China have apparent imbalance in the development levels of the two subsystems. This imbalance has been improved over time. Overall, the coupling degrees appear to be consistent with the actual development in different regions in China. The evolution of the coupling degrees for Beijing and Qinghai over the years 2002–2023 is shown in Figure 16. Even in 2002, Beijing already had a high coupling degree between the two subsystems, and it stays high close to one through 2023. For Qinghai, the coupling degree rises from below 0.6 to over 0.9 over the years, with a noticeable increase over the span of three years (2008, 2009, and 2010).

Figure 16. The coupling degree between the two subsystems for Beijing and Qinghai over the years 2002–2023 calculated using dataset O.

The coupling degrees for the 30 regions in 2002, 2012, and 2023 using the seven predictors proposed in [3] for each subsystem are shown in Figure 17.

Figure 17. The coupling degree between the two subsystems for all 30 regions in China in 2002, 2012, and 2023 calculated using the reference dataset R.

Although it is somewhat counterintuitive for Beijing to have a lower coupling degree than that for Tianjin, Hebei, and Shanxi, there is no strong evidence there is something wrong with the model itself. The evolution over the years for Beijing and Qinghai revealed serious issues as shown in Figure 18. First, one may notice that the coupling degree for Beijing is gradually reducing over the years, with that for 2023 apparently below that of year 2022. This does not match the reality where all regions in China have been improving their development levels in technological innovation as well as in financial development. Second, the coupling degree between the two subsystems for Qinghai appears to drop significantly from 2002 and 2003, which is not explainable. Both issues are evidence that the seven predictors ($X_{11}$ to $X_{17}$) failed to fully capture the development levels of the technological innovation subsystem, and the seven predictors ($Y_{11}$ to $Y_{17}$) failed to fully capture the development levels of the financial development subsystem.

Figure 18. The coupling degree between the two subsystems for Beijing and Qinghai over the years 2002–2023 calculated using the reference dataset R.

5.6. The Coordination Degree Between Technological Innovation and Financial Development

Figure 19 shows the coordination degrees between the technological innovation and the financial development subsystems calculated based on our dataset O for all 30 regions in 2002, 2012, and 2023. As can be seen, all regions have been improving their coordination degrees over the 21 years. The patterns of the coordination degrees for the regions are roughly aligned with expectation. In 2002, Beijing, Shanghai, and Guangdong have the highest coordination degrees and are higher than the remaining regions by a big margin. In 2012, Jiangsu and Zhejiang were catching up with Shanghai. In 2023, Jiangsu already surpassed Shanghai.

Figure 19. The coordination degree between the two subsystems for all 30 regions in China in 2002, 2012, and 2023 calculated using our dataset O.

Figure 20 shows the evolution of the coupling degrees for Beijing and Qinghai over the years 2002–2023. Both Beijing and Qinghai had a steady increase in their coordination degree over the years. Beijing increased from slightly above 0.5 to close to 0.8. Qinghai increased from under 0.2 to close to 0.35.

Figure 20. The coordination degree between the two subsystems for Beijing and Qinghai over the years 2002–2023 calculated using our dataset O.

Figure 21 shows the coordination degrees for the 30 regions in 2002, 2012, and 2023 using the reference dataset R. Although the overall patterns of the coordination degrees contain no apparent issue, the increase in the coordination degrees over the years is less prominent than that predicted using our set of predictors.

Figure 21. The coordination degree between the two subsystems for all 30 regions in China in 2002, 2012, and 2023 calculated using the reference dataset R.

As shown in Figure 22, the evolution over the years for Beijing and Qinghai revealed more issues. First, for Beijing, other than a significant jump in coordination degree from 2006 to 2007, the coordination degrees are fairly stagnant, i.e., there were less than 0.1 increases from 2008 to 2023. This does not appear to match the rapid technological and financial development during this time period. Second, for Qinghai, the coordination degrees drop in three periods: (1) from 2002 to 2006, (2) from 2009 to 2011), and (3) from 2020 to 2023. There does not appear to be any reason for the drop in the first two periods. Perhaps the third period may be attributed to the pandemic. Again, the issues could be due to the set of predictors failing to capture the actual development level of each subsystem.

Figure 22. The coordination degree between the two subsystems for Beijing and Qinghai over the years 2002–2023 calculated using the reference dataset R.

5.7. Projection of Future Coordination Degree

Highly reliable projection of future coordination degree could be very useful for enacting conducive policies to encourage coherent development in technological innovation and financial systems. In this subsection, we report our effort in making such projections. The findings prove that making the correct projection of future coordination degree is very challenging.

In our investigation, we would like to project for the most recent five years in our dataset, i.e., 2019, 2020, 2021, 2022, and 2023, using the remaining data as the training set for a regression model. The projection accuracy is evaluated using a metric called $R^2$. This metric quantifies the normalized error between the predicted values and the actual values with one caveat, i.e., $R^2$ would be a negative number if the prediction is so bad that it is worse than simply making the prediction using a straight horizontal line. A perfect regression would have a $R^2=1$ (i.e., the regression model explains all the variability in the actual data around the mean), and a regression would have a $R^2=0$ if the model fails to explain any of the variability in the actual data.

We experiment with two regression models. One is linear regression with polynomial features. The other is gradient boosting regression. Again, we compare the results of the two sets of coordination degrees computed using the dataset O and the reference dataset R. The regression study is performed with three-fold cross-validation with hyperparameter tuning using grid search. For linear regression, a single hyperparameter, i.e., polynomial degree, with possible values of 1, 2, and 3, is tuned with cross-validation. For GBR, five hyperparameters are tuned with cross-validation:

• Number of estimators: with possible values of 50, 100, and 200;
• Learning rate: with possible values of 0.01, 0.1, and 0.2;
• Max depth: with possible values of 3, 4, 5;
• Minimum samples split: with possible values of 2, 5, and 10;
• Minimum samples leaf: with possible values of 1, 2, and 4.

5.7.1. Linear Regression with Polynomial Features

Figure 23 shows the results of the training and testing performance using $R^2$ using our dataset O. As can be seen, the high training performance does not translate to high testing performance at all. All but one region has a training $R^2$ greater than 0.9. However, 12 regions have negative testing $R^2$. Only seven regions score $R^2$ greater than 0.8. This result means that one would have to be cautious when interpreting the projected coordination degrees, particularly when enacting development policies.

Figure 23. LRPF training $R^2$ and testing $R^2$ for the coordination degrees using our dataset O.

In contrast, the training performance and testing performance for the coordination degrees using the reference dataset R are far worse as shown in Figure 24. Eight regions have very low training $R^2$. Only six regions have training $R^2$ higher than 0.9. Furthermore, only two regions have positive testing $R^2$. Again, our findings demonstrate that using inappropriate predictors would render unreliable results.

Figure 24. LRPF training $R^2$ and testing $R^2$ for the coordination degrees using the reference dataset R.

For completeness, we show the actual coordination degree and the predicted value for two regions, Beijing and Sichuan, using our dataset O in Figure 25. Beijing has reasonably good project performance, while Sichuan has negative $R^2$ value. The insets are zoomed-in for the five-year projection.

Figure 25. The actual data and the LRPF-predicted values using our dataset O for Beijing and Sichuan.

Figure 26 shows the actual coordination degree and the predicted value for two regions, Hainan and Sichuan, using the reference dataset R. Hainan has the best project performance using dataset R, while Sichuan has a negative $R^2$ value. As can be seen, the regression errors are much larger than those using our dataset O.

Figure 26. The actual data and the LRPF-predicted values using the reference dataset R for Hainan and Sichuan.

Besides $R^2$, the regression and projection performance can also be evaluated in terms of mean absolute error (MAE) and root mean squared error (RMSE). For completeness, we show the performance evaluation in terms of MAE and RMSE for both our dataset O and the reference dataset R in Figure 27, Figure 28, Figure 29 and Figure 30.

Figure 27. LRPF training MAE and testing MAE for the coordination degrees using our dataset O.

Figure 28. LRPF training MAE and testing MAE for the coordination degrees using the reference dataset R.

Figure 29. LRPF training RMSE and testing RMSE for the coordination degrees using our dataset O.

Figure 30. LRPF training RMSE and testing RMSE for the coordination degrees using the reference dataset R.

Using MAE as the performance evaluation metric, we can see from Figure 27 and Figure 28 that the error is several folds bigger for the reference dataset R than for our dataset O. Furthermore, regardless of which dataset is used, the testing error is also bigger than the training error by several folds. Nevertheless, the difference is not enormous because the coordination degree is normalized within the range of [0,1].

Using RMSE as the performance evaluation metric, we can see from Figure 29 and Figure 30, the patterns are similar to those for MAE. However, the contrast is less for RMSE than for MAE due to how the error is calculated.

If we compare the evaluation results based on $R^2$, we can conclude that $R^2$ is a more distinctive metric on the quality of regression and projection because the contrast between training and testing, as well as the contrast between the performance of the two datasets, is much more prominent.

5.7.2. Gradient Boosting Regression

Figure 31 shows the results of the training and testing performance using $R^2$ using our dataset O. As can be seen, the training performance is superb, with all but one region has nearly perfect $R^2$. However, the generalizability is extremely poor, with the testing $R^2$ being negative for all regions. Obviously, the gradient boosting regression is not suitable for making projections.

Figure 31. GBR training $R^2$ and testing $R^2$ for the coordination degrees using our dataset O.

The patterns of performance on training and testing remain similar for the coordination degrees using the reference dataset R as shown in Figure 32. While the training performance is noticeably improved over the linear regression with polynomial features, the testing performance is actually worse, with no region having positive $R^2$.

Figure 32. GBR training $R^2$ and testing $R^2$ for the coordination degrees using the reference dataset R.

For completeness, we show the actual coordination degree and the predicted value for the same two regions, Beijing and Sichuan, using our dataset 0, in Figure 33. The insets are zoomed-in for the five-year projection. As can be seen, GBR has excellent regression performance for the training set but performs poorly for projection using the testing set.

Figure 33. The actual data and the GBR-predicted values using our dataset O for Beijing and Sichuan.

Figure 34 shows the actual coordination degree and the GBR-predicted value for two regions, Hainan and Sichuan, using the reference dataset R. As can be seen, again, the regression errors are much larger than those using our dataset O.

Figure 34. The actual data and the GBR-predicted values using the reference dataset R for Hainan and Sichuan.

Similar to LRPF, besides $R^2$, the regression and projection performance can also be evaluated in terms of mean absolute error (MAE) and root mean squared error (RMSE). For completeness, we show the performance evaluation in terms of MAE and RMSE, for both our dataset O and the reference dataset R in Figure 35, Figure 36, Figure 37 and Figure 38.

Figure 35. GBR training MAE and testing MAE for the coordination degrees using our dataset O.

Figure 36. GBR training MAE and testing MAE for the coordination degrees using the reference dataset R.

Figure 37. GBR training RMSE and testing RMSE for the coordination degrees using our dataset O.

Figure 38. GBR training RMSE and testing RMSE for the coordination degrees using the reference dataset R.

Using MAE as the performance evaluation metric, we can see from Figure 35 and Figure 36 that the training error is several folds bigger for the reference dataset R than for our dataset O. However, the testing errors for the two datasets are on the same scale.

Using RMSE as the performance evaluation metric, we can see from Figure 37 and Figure 38, the patterns are roughly similar to those for MAE. The testing error for the reference dataset R is slightly larger than that for our dataset O.

Again, we can see that $R^2$ is a more distinctive metric on the quality of regression and projection because the contrast between training and testing, as well as the contrast between the performance of the two datasets are much more prominent.

6. Discussion

In this section, we engage in in-depth discussion on several aspects of our study, including three alternative configurations (using a reduced predictor set, using alternative imputation methods, and using an alternative weighting method), and statistical comparison of the outcomes derived from different configurations, including using our dataset and the reference dataset.

6.1. Reduced Predictor Set

We are using 12 predictors for each subsystem, which is more than most other studies. It is desirable to determine which predictors are the most influential to the study of coupling and coordination degrees. This is typically performed via a sensitivity analysis. Because the weight for each predictor is calculated as the first step in our study, we choose to use weights as the indication for the most influential predictors. Because the reference dataset that we use employs seven predictors to model each subsystem, we decide to use the seven predictors that have the largest weights to calculate the comprehensive evaluation index for each subsystem, the degree of coupling, the degree of the coordination, and subsequently to make projection. The results are in general rather similar to those produced using the 12-predictor set for each subsystem. In Figure 39, we show the training performance and the testing performance for the coordination degrees using our dataset O using the reduced seven-predictor set for each subsystem. Compared with the result obtained using the full 12-predictor set, 12 regions have positive $R^2$ with the 7-predictor set while there are 18 regions have positive $R^2$. Nevertheless, the result is significantly better than that using the reference dataset R, where only two regions have positive $R^2$.

Figure 39. LRPF training $R^2$ and testing $R^2$ for the coordination degrees using our dataset O using the reduced 7-predictor set for each subsystem.

6.2. Alternative Imputation Methods

In our study, we compare two imputation methods [49] for filling missing data, one is LRPF, and the other is GBR. We use the regression performance as an indicator to select which one is better. Because GBR has significantly smaller errors, we choose to use GBR to fill the missing data. Because other imputation methods are available [49], it will be interesting to experiment with these imputation methods. It is difficulty to compare the performance of imputation methods because we do not have complete dataset. We experiment with using the Kolmogorov–Smirnov (KS) test [50] to gauge the performance of imputation methods. If the imputation is doing a good job, the expectation is that the imputed data would have similar distribution to the original data, which would lead to a KS score of closing to 0. We test the following imputation methods: mean imputed, median imputed, mode imputed, forward fill, backward fill, KNN inputted, and interpolated. All imputation methods have rather similar low KS scores. Hence, comparing distributions is not informative. Instead, we single out the interpolated imputation method and compute the regression performance based on missing data filled by this method. The regression performance for LRPF and GBR is shown in Figure 40. As can be seen, the performance is degraded noticeably for both training and testing.

Figure 40. LRPF training $R^2$ and testing $R^2$ for the coordination degrees using our dataset O, where the missing data are filled with the interpolated imputation method.

6.3. Alternative Weighting Method

Besides the entropy weighting, several other objective weighting methods are available. To examine the impact of weighting method to the results, we additionally experiment with the CRITIC method [33]. We visually compare the resulting comprehensive evaluation index for both subsystems, the degree of coupling, the degree of coordination, as well as the projection performance. They are rather similar to those computed using the entropy weighting method. In Figure 41, we show the training and testing regression performance for making projection.

Figure 41. LRPF training $R^2$ and testing $R^2$ for the coordination degrees using our dataset O, where the weights are calculated using the CRITIC weighting method.

6.4. Statistical Comparison of Regression Performance

In previous sections, we visually compared the regression results on the degree of coordination for the two datasets. In this subsection, we present a statistical comparison of the two datasets and demonstrate that indeed our dataset O has statistical better regression performance $R^2$. Due to the presence of negative $R^2$ values in the testing results, we can only perform the statistical comparison using the training performance. We first check the $R^2$ values for each dataset for normality. If the distributions of $R^2$ for both datasets are both normal, then we can use the t-test. Otherwise, we compare using a form of non-parametric statistical method, i.e., confidence interval. The normality checks on the regression performance for both datasets for both LRPF and GBR fail. Hence, we have to resort to using confidence interval for statistical comparison.

We also take this opportunity to compare with several other configurations: (1) using the seven-predictor set for each subsystem based on our dataset O; (2) using the interpolated imputation method to fill the missing data; and (3) using CRITIC weighting method to calculate the weights. The comparison is made for both LRPF and GBR. As shown in Figure 42, all four different configurations using our dataset O are statically better than R because there is no overlap between the two confidence intervals. Three configurations (the original configuration with 12-predictor set for each subsystem, and alternative configurations 1 and 3) are statistically the same. However, the results are sensitive to the alternative interpolated imputation method, which lowers the performance when LRPF is used for regression. When GBR is used for regression, all four configurations are equivalent.

Figure 42. Statistical comparison of the training performance $R^2$ for our dataset O in four different configurations and the reference dataset R, for both LPRF (left) and GBR (right).

6.5. Heterogeneity of Coordination Degrees and Heterogeneity of Projection Performance

As we have shown in previous sections, there is an apparent heterogeneity of the coordination degrees among different regions, and there is also an apparent heterogeneity of the projection performance. It is interesting to see if there is any correlation between the heterogeneity. We note that the regression performance in terms of $R^2$ is rather uniform across almost all regions. Hence, we choose to focus on the heterogeneity of the testing performance (i.e., project performance). The top 10 regions for the degree of coordination and the top 10 regions for the testing performance are shown in Figure 43 side-by-side. As labeled using colored arrows, four regions (Beijing, Zhejiang, Hubei, and Tianjin) are present in both plots. This demonstrates that the heterogeneity of the degree of coordination is not strongly correlated with the heterogeneity of the projection performance.

Figure 43. The top 10 regions for the degree of coordination and the top 10 regions for the projection performance. The four regions that are present in both are highlighted.

7. Concluding Remarks

In this paper, we make a critical look at the mainstream approach in estimating the coupling and coordination of two or more subsystems from the engineering perspective. The mainstream approach is to define a set of predictors for each subsystem. Then, the weight for each predictor is calculated towards establishing the development level of the subsystem (referred to as comprehensive evaluation index in our study). Subsequently, the coupling degree and the coordination degree are computed based on the comprehensive evaluation indices of the subsystems. A big issue with this approach is the lack of validation of the results. The set of predictors is usually subjectively defined base on the authors’ expertise and perspective. As such, there is no guarantee that the set of predictors can fully model the subsystem. Consequently, the development level of each subsystem, and the following coupling degree and the coordination degree could be biased and misleading. While people who are familiar with the subsystems may have subjective perceptions regarding which regions are more developed, there is no established ground truth for validation.

While in this paper we focus on the coupling and coordination of technological innovation and financial development, the issue identified above is applicable to all similar studies. Hence, the impact of the issue is wider than the specific scope of our current study. The goal of this study is to explore the feasibility of establishing an objective evaluation method on the quality of the models for the subsystems via the regression and projection performance of the coordination degree between the subsystems. The hypothesis is that if the set of predictors defined is an accurate model for each subsystem, then the intrinsic interaction between the subsystems may be reliably estimated via regression. This would lead to small regression error on the training data, and good projection performance for the near future. By performing a side-by-side comparison between our dataset O (each subsystem is modeled with 12 predictors) and a reference dataset R (each subsystem is modeled with 7 predictors), we demonstrate the feasibility of objectively evaluate alternative models. Indeed, the set of predictors that we propose lead to significantly better regression and project performance than the approach that we use as a reference.

Limitations. Our study is the first step in establishing an objective evaluation method for the coupling and coordination studies of two or more subsystems. Due to the limited time and resources, we only compare our model with one reference system. Ideally, we should compare with all related studies, which would make the result much more compelling. Furthermore, we present a qualitative comparison. It is desirable to develop a systematic quantitative evaluation method where one or more metrics are defined. Additionally, the scope could be extended beyond regions in China to other developing countries, provided that the data are available.

We also acknowledge that the selection of 12 predictors for each subsystem is quite subjective. How to determine what predictors to use to fully describe the subsystem is still an open research issue. Furthermore, assessment on the potential redundancy and multicollinearity of the predictors is also desirable. Unfortunately, in the context of financial development, it is unclear how to perform the assessment due to the lack of ground truth regarding the degree of coupling and the degree of coordination.

The current study is limited in its analysis of the relationship between the development levels of two subsystems. In future work, we plan to explore potential causal relationships development levels of the two subsystems and their coordination degree could deepen the study’s implications. For instance, Granger causality tests [51] could help determine whether changes in one subsystem can forecast changes in the other or in the coordination degree. Furthermore, given the panel structure of the data (regions over time), applying panel regression techniques [52] (e.g., fixed or random effects models) could help account for unobserved regional heterogeneity and yield more robust estimates of the interrelationships. This would also clarify whether the determinants of coordination are consistent across regions.

Last, but not least, the use of regression performance as a benchmark for evaluating the models for the coordination of subsystems for financial development, although intuitive, lacks theoretical foundations. In particular, high regression performance alone might not be a fair evaluation of the coordination degrees of a model. Additional information should be collected, such as that related to policy relevance and external consistency, to fully evaluate a model.