A DANP-Based NDEA-MOP Approach to Evaluating the Patent Commercialization Performance of Industry–Academic Collaborations

Abstract

The industry–academic collaboration (IAC) in developed and developing countries enables these economies to gain momentum in continuous innovation and, thus, economic growth. Patent commercialization is one major channel of knowledge flow in IAC. However, very few studies consider the flow of knowledge between industrial firms and universities. Moreover, ways that the patent commercialization performance of IACs can be evaluated are rarely discussed. Therefore, defining an analytic framework to evaluate the performance of IAC from the aspect of patent commercialization is critical. Traditionally, data envelopment analysis (DEA) models have widely been adopted in performance evaluation. However, traditional DEA models cannot accurately evaluate the performance of IACs with complex university–industry interconnections, the internal linkages, or linking activities of knowledge-flow within the decision-making units (DMUs), i.e., the IACs. In order to solve the abovementioned problems, this study defines a multiple objective programming (MOP)-based network DEA (NDEA), with weighting derived from the decision-making trial and evaluation laboratory (DEMATEL)-based analytic network process (ANP), or the DANP. The proposed analytic framework can evaluate the efficiency of decision-making units (DMUs) with a network structure (e.g., supply chains, strategic alliances, etc.) based on the weights that have been derived, based on experts’ opinions. An empirical study based on the performance of the patent commercialization of Taiwanese IACs was used to demonstrate the feasibility of the proposed framework. The results of the empirical research can serve as a basis for improving the performance of IAC.

1. Introduction

In the era of the knowledge-based economy, knowledge has become the major factor driving the economic growth of major and fast-catching-up minor economies. The accumulation, creation, and application of knowledge have become the main sources of national competitiveness. Industry–academic collaboration (IAC) enables firms to obtain new techniques and patents from university research [1]. Thus, over the past few decades, pharmaceutical, engineering, and software firms have increasingly enhanced their cooperation with universities [2]. In addition to their own research and development (R&D), firms also need universities’ R&D of new technologies.

IACs have a long tradition in numerous countries all over the world [3]. The logic behind IAC refers to the cooperation between firms and universities. Such collaborations between firms and universities have gradually become a key component of an effective national innovation system. The motivations of policymakers and higher-education institutes to develop “third missions” besides the two conventional basic missions of lecturing and academic research, and to commercialize academic knowledge through various approaches like patent applications and commercialization, technology licensing and transfer, incubation has strengthened the appropriateness of IACs [4,5,6]. IAC also promotes technological innovation, and the transfer of scientific technology can be promoted through the acquisition and use of knowledge created by the IAC, innovative start-ups, and spin-offs [7].

Much evidence has demonstrated that effective knowledge and technology transfer are vital for a successful IAC [8]. An effective transfer is able to enhance innovation performance [9], improve the technology’s novelty [10], and enhance product development [6,11]. Thus, the evaluation of effective knowledge and technology transfer in IAC is very important. Performance evaluation is the formal process of measuring employees’ productive work and achievements, based on job responsibilities. In general, performance management is a systematic management process used to organize the way that individuals achieve common goals. It can provide directions for organizations or employees to improve or communicate the results being achieved [12].

Numerous scholars have tried to evaluate the performance of IAC. For example, Anderson et al. [13] evaluated the performance of university technology transfers, while Rast et al. [14] defined an evaluation framework for assessing IAC. Crescenzi et al. [15] used co-patenting to detect the closeness and innovative collaborations of university–industry linkages. Furthermore, Crescenzi et al. [15] argued that collaborative research projects, scientific publications, and patenting are three major channels for shaping knowledge transmission in IAC. Apparently, patents are an important indicator of IAC.

Therefore, scholars have started to investigate the related issues of patent commercialization by universities, due to the low percentage rate of patents that have been commercialized by universities [16]. Whether university patents can successfully be commercialized is very important for the sustainable economic growth of certain fast-growing economies like that of mainland China. Recently, according to Hsu et al. [17], in spite of the economic importance of patented university research, it is hard to quantify the monetary value of patents by universities and to observe the extent to which higher-education institutes are capable of capturing such value from patent commercialization. Apparently, the way that IAC can be evaluated according to the performance of patent licensing and commercialization is very important for evaluating the knowledge transmission in AIC and is worthy of further investigation.

However, almost no studies consider the flow of knowledge between industrial firms and universities; nevertheless, this knowledge flow is critical to IAC in a knowledge-based economy. In reality, there is a growing awareness of the very limited understanding that we have about the connections that link universities and industrial firms together [18]. Consequently, a growing number of scholars have mentioned the need to investigate further the appropriateness of current sources for evaluating “third missions” or IAC performance, as well as for the development of novel possible evaluation criteria [19].

Traditionally, data envelopment analysis (DEA) was widely adopted for tackling performance evaluation problems. However, the conventional DEA models (e.g., the CCR [20] and BCC [21] models) have some limitations in solving IAC problems, especially when taking into account the complexity of university–industry interconnections [19]. According to Tone and Tsutsui [22], many organizations comprise numerous linked divisions (see Figure 1a, from [22], for a typical example of an organization consisting of three linked divisions). The traditional DEA models usually adopt either the aggregation (e.g., the CCR [20] and BCC [21] models) or the separation (e.g., Seiford and Zhu [23]) methods. (Typical configurations for the aggregation and separation models of DEA can be found in Figure 1b,c). For the aggregation models, the decision-making unit (DMU) is usually regarded as a black box (see Figure 1b), while the internal linkages or linking activities within the DMU are neglected; therefore, the performance of individual divisions within the DMUs cannot be evaluated [24]. In contrast, the DEA models based on the separation method usually treat each division as an independent unit (see Figure 1c for a typical example); linkages among other divisions are completely neglected [24].

Thus, the multiple objective programming (MOP)-based NDEA approach, or the NDEA-MOP, proposed by Kao et al. [24], is adopted to resolve the abovementioned problem in evaluating the patent commercialization performance of IACs. Based on the CCR model [20] or BCC model [21], the total and divisional efficiencies of a DMU are defined according to different objective functions to be measured cohesively in the NDEA-MOP. Furthermore, very few works on MOP problems in general, and NDEA-MOP problems in particular, adopt the real weights associated with goals. Thus, the authors propose a decision-making trial and evaluation laboratory (DEMATEL)-based analytic network process (ANP) [25] or DANP framework to derive the influence weights associated with the objectives, based on the opinions of experts familiar with IAC. The analytic results derived by the DANP-based NDEA-MOP model with real weights will be very important for solving real-world management problems. An empirical study, based on the open database of the Ministry of Education, Taiwan, will be used to evaluate the IAC performance of Taiwanese technical universities.

The rest of this paper is organized as follows. In Section 2, past works on IAC, patent commercialization, performance evaluations of IAC, and NDEA are reviewed. In Section 3, the DANP-based NDEA-MOP analytic model is proposed. In Section 4, an empirical study based on the performance evaluation of the patent commercialization of Taiwanese IACs is introduced to verify the feasibility of the proposed analytic framework. Then, a discussion will be presented in Section 5. Finally, Section 6 will conclude the paper with observations, conclusions, and recommendations for further study.

2. IAC, Patent Commercialization and Performance Evaluation

In the era of the knowledge economy, IAC has already become one of the most important methods for diffusing knowledge that is embedded in techniques or inventions, from higher institutes to industrial firms. During recent years, the patent commercialization of universities has become a research focus due to its dominant role in enhancing economic growth [26]. Therefore, the way in which the performance of the patent commercialization of IACs can be evaluated is very important. Specifically, the way that the inputs and outputs of the IACs should be determined so that the performance of IAC in general, and the evaluation of the patent commercialization in IACs in particular, can be evaluated is worth further investigation. Traditionally, DEA-based methods have already been adopted widely in performance evaluation. However, the conventional DEA methods neglect the interconnections within DMUs [24]. Thus, NDEA-based methods that include the divisions inside the DMU and the connections between the DMUs can cross the research gap and evaluate the efficiency of patent commercialization correctly. The NDEA-MOP further advances the NDEA models from the aspect of considering multiple objectives of management in performance evaluation. Thus, the NDEA-MOP will be very useful in evaluating patent commercialization performance in IACs. This section first reviews the definitions regarding IAC. Then, it reviews the literature on patent commercialization. After that, past works on the performance evaluation of IAC and on NDEA are reviewed.

2.1. IAC

Collaboration between universities and industries can expand the research relationships that public enterprises carry out, promote the commercialization of public R&D results, increase labor mobility between the public and private sectors, and contribute to new products and patents [1]. Thus, IAC has widely been adopted. Such cooperation is conducive to knowledge-cooperation so that different innovative capabilities can be integrated to create new and useful items [27]. There are many different types and goals and categories of ways in which IAC can be formulated [28]. Cooperation can be formal or informal; the former includes formal equity partnerships, signing contracts, setting research projects, and licensing a variety of patents [29]. The IAC is a win-win approach for both universities and industrial firms. Universities can improve the quality of their research and teaching through their practical cooperation experiences with industrial firms, while the industrial firms can increase the differentiation of their products and/or services, as well as reducing operating costs via innovative concepts developed through collaborative research. Universities and industries share not only their knowledge and abilities but also their R&D facilities and staff members [30].

2.2. Patent Commercialization by Universities

Patent commercialization is a vital stage in enhancing organizational performance [31]. The scope of patent commercialization consists of the sales, transfer, or license of patented technologies to industrial firms; founding start-ups that are based on the grounds of licensed patent technologies; or commercializing patented technologies in designs, products, and manufacturing or the service process of licensees [31]. The patents held by firms or universities are, by nature, different [16]. On the one hand, patents held by firms are driven primarily by possible market benefits. On the other hand, patents held by universities are more dependent on the governmental patent policy and the missions or characteristics of a specific university [16,32,33]. Effective patent commercialization increase returns and thus improves the innovation performance of specific firms or universities [31].

One vital mechanism by which universities contribute to economic growth is the conversion of academic inventions into patent commercialization [34]. Consequently, over the past three decades, research interests on the transfer of academic knowledge to industrial firms have increased radically [35] in developed countries like the United States (e.g., [36]), Spain (e.g., [37]), The Netherlands (e.g., [38]), Portugal (e.g., [39,40]), Japan (e.g., [41]), and in quickly evolving economies like China (e.g., [42]). Furthermore, this transfer has been regarded as a natural phase in the development of modern higher-education institutes [35,43]. Thus, researchers have started to investigate the commercialization of university patents [44,45,46] and the formation of spin-offs [35,47,48,49]. Giuri et al. [35] also presented a similar finding that patent sales, licensing, and academic spinoffs are three exploitation channels of academic patents.

2.3. Performance Evaluation of the IAC

The concept of “performance evaluation” was proposed by Drucker [50]. In 1954, he promoted the phrase “management by objectives”, with the goal of assessing performance by comparing the performances of personnel, whether individually or as members of a team. According to Ahmed et al. [51], performance evaluation is crucial for the effective management of the manpower behind an organization and the evaluation of employees, which can assist in the development of staff, improve organizational effectiveness, and feed into business planning. Recently, Murphy [52] defined performance evaluation as a process in which one or more individuals (typically, supervisors or managers) in organizations observe and gain information about the employees’ work performance.

Under the leadership of Drucker, many scholars have proposed specific methods that focus on setting goals and then comparing performance against these goals. For example, Huang et al. [53] proposed a multiple objective programming-based DEA model for evaluating the performance of leading fabless integrated-circuit design houses. Wu et al. [54] developed a professional performance evaluation system for pre-service automobile repair vocational high-school teachers in Taiwan. Han and Trimi [55] proposed a fuzzy technique for order preference according to its similarity to the ideal solution (TOPSIS) method. This method was used to evaluate the performance of reverse logistics in social commerce platforms. Kao et al. [56] evaluated the performance of systemic innovation problems regarding the Internet of Things (IoT) in manufacturing industries, using novel MCDM methods.

Performance evaluation can address both general (e.g., employee performance evaluation [57]) and specific (e.g., leanness evaluation metrics for manufacturing organizations [58]) issues. The resources for performance measurement are usually part of the organizational infrastructure, where the assessment is targeted. Performance evaluation can focus on goal setting, which is the basis of traditional performance management processes. Goals enable the alignment of individual and team performance with organizational-level performance. Goal setting can help managers increase engagement by clarifying expectations, leading to more challenging work [59].

When discussing the relative performance of the DMUs of a decision-making problem in a specific field, the main input and output variables should be considered at the very beginning. The possible inputs that can be used to evaluate IAC problems include manpower, financial resources, material input, and human input. Manpower includes the number of teachers, such as the professors and lecturers, as well as support personnel. Financial resources include government funds, tuition fees, private grants, self-raised income by universities, and other incomes. Material investment mainly includes fixed assets and equipment investment. The educational output primarily refers to the university’s achievements in the process of operation, education, teaching, and talent training, and the rewards and achievements of academic and scientific research [60].

Three input factors of IAC, namely, resources, researchers’ capabilities, and researchers’ motivations, are usually those that are considered. Furthermore, expense is a key input factor in any type of R&D activity [61]. The easiest way to measure researchers’ capabilities is with bibliographic metrics. In academia, one of the major criteria for evaluating researchers’ quality is the number of journal publications they have. However, the quality of journals and individual journal articles varies, so the publication number is not a trustworthy method of evaluating the influence of a researcher; therefore, the number of citations is a better method for measurement. Citation calculations record the number of times a writer’s publications have been cited by others in the bibliographical databases, e.g., the Science Citation Index (SCI) and Social Science Citation Index (SSCI), being developed by the Institute for Scientific Information [62]. Sometimes, measuring researchers’ motivation directly is very tough, but the academic atmosphere and other factors associated with the academic department can be adopted for assessment. Previous studies have shown that the atmosphere is one of the most important predictors of participation in industry activities [63,64].

According to Perkmann et al. [2], IAC has three major categories of outputs. First, the technological output of IAC projects can be measured by patent applications or patents granted. Second, publications in peer-reviewed journals can serve as a major performance indicator in academic institutions. Third, several staffing indicators have also been successfully evaluated in terms of staff skills and training [2]. Crescenzi et al. [15] argued that patenting is one of the major channels for shaping knowledge transmission in AIC. Apparently, patent applications or patents granted should be one of the major indicators for evaluating the performance of IAC [35]. One typical example is the recent exploration by Nugent et al. [65] of the impact of university–industry-targeted (U-I-targeted) grants by governments, based on the economic benefit from patent applications that have been filed and granted.

Albats et al. [66] summarized seven factors to be considered when evaluating the performance of an IAC. (1) The evaluation system should take into account the peculiarities of particular collaborations. (2) Individual and expert assessments should be conducted in high-tech and/or niche markets, rather than by official indicators collected. (3) The introductions of agile principles that have been widely adopted in the iterative development process of the information and communications technology (ICT) products to other more traditional industries may help shorten the time-frame of IAC project implementation and assessment [66]. (4) Especially in an era when an open innovation approach is emerging rapidly, or if the company is moving into science-intensive sectors, the prominence of publications in academic journals, as well as the entire strategic cooperation with academia, must be well-thought-out. (5) The cooperation amongst the performers, such as round-tables, conferences, meetings on funding, regional events, etc., should be included. (6) As a suggestion for policymakers and researchers, a number of regional background checks should be conducted on a large-scale basis to respond to the link between the actors’ priorities and relevant key performance indicators for IAC. (7) Finally, based on new indicators developed and related to previous indicators, optional indicators should be provided so that actors can select the most relevant indicators [66].

Albeit evaluating IAC is very important, according to Perkmannet al. [2], the performance of IAC is often hard to evaluate for the following four reasons. First, the outputs from projects are often intangible and are thus not possible to measure directly. While this know-how is relevant to achieving specific goals, it is difficult to quantify its value. Second, outputs from these projects are very complicated, combining the goals of the university and firm. Third, the benefits from these projects may only be comprehended over the long term [67]. It takes a long time to realize the benefits of these projects. Fourth, it is difficult to judge performance according to the reference value or target with which the performance indicator should be compared. The objectives, outputs, and organizational structure of IAC in different fields of science vary widely.

2.4. NDEA

The DEA is a method proposed by Charnes et al. [20] to mathematically contrast the productivities of distinct DMUs, according to various inputs and outputs. A DMU will be regarded as efficient when the relative efficiency is one and all the slack variables are zero [20]. Other DMUs with ratios lower than 1 are regarded as “less efficient” with respect to the efficient ones. (Please refer to the original work by Charnes et al. [20] for the fundamental concepts of how the relative efficiencies of DMUs can be derived.) Due to the fact that the weights associated with the input and output variables of DMUs are derived to maximize the ratio and are then compared with the most efficient DMUs, the measured productivity is also called “relative efficiency” [68].

One disadvantage of conventional DEA models is their ignorance of internal activities. For example, suppose the target firm has three departments that are closely interrelated. Each department uses its own input resources to generate its own output. However, in most cases, a department may use the partial or full outputs of other departments as its inputs. Such a scenario cannot be formulated using the traditional DEA model, where each input or output belongs only to one specific department [22].

To measure the efficiency of systems with linkage(s) inside, the NDEA models proposed by Färe and Grosskopf [69] can provide complete access to the underlying information of the black box and can evaluate the overall performance of the DMUs and individual components thereof. Following the efforts by Färe and Grosskopf [69], various scholars have proposed methods to tackle NDEA problems. Tone and Tsutsui [22] proposed a generic slack-based NDEA method, or the so-called network SBM, which can formally tackle intermediate products. Cook et al. [70] proposed a specific two-stage NDEA model, where the output from the first stage is the sole input to the second stage. Moreover, Cook et al. [71] explored the generic problem of an open multi-stage process by presenting overall efficiency as an additive-weighted average of the efficiencies of every single component or stage. Following Tone and Tsutsui’s effort [22], Fukuyama and Mirdehghan [72] proposed an NDEA method for deriving the efficiency of each DMU and the divisions inside, to prevent incorrect decisions caused by multiple optima [24].

MOP features a set of objective functions that should be optimized at the same time, and a set of well-defined constraints to be fulfilled [73]. In recent years, several scholars have proposed the NDEA-MOP model. Kao et al. [24] proposed models for NDEA-MOP methods. Despotis et al. [74] adopted an MOP model for evaluating efficiency through a two-stage process. Recently, Koronakos et al. [75] re-formulated some fundamental NDEA methodologies in a generic MOP framework. Compared with traditional NDEA approaches, NDEA-MOP has been verified as effective and discriminatory in an evaluation and manifests the following merits [76]. First, with MOP, the efficiency of the DMU and of its divisions can be optimized simultaneously [76]. Second, from the perspective of multi-level decision-making, the DMU level and the divisional level can work collaboratively to optimize their efficiencies [76]. Due to the advantages of the NDEA-MOP model in tackling real-world performance evaluation problems, an analytic framework based on this method will be introduced to tackle the performance evaluation of patent commercialization in IACs.

3. Research Methods

The first subsection introduces the modified Delphi method. The second subsection introduces NDEA. The third subsection defines the parameters and model of the NDEA-MOP, which will serve as the research method of this study, and evaluates the performance of IAC. Finally, the structure of the NDEA model is configured, which will serve as the basis for the empirical case study in Section 4.

3.1. Modified Delphi Method

The Delphi method, a research method for collective decision-making by anonymous experts [77], was proposed by the RAND Corporation in 1950. The way to solve a problem by the Delphi method is to repeat interviews or dispatch questionnaires for several iterations until a consensus can be reached. Although applicable to solving real-world problems, the traditional Delphi method wastes time and provides uncontrollable front and rear views of the expert group on a contradictory situation. Thus, Murry and Hommons [78] proposed a modified Delphi method to conquer the shortcomings of the traditional Delphi method. Based on this modified Delphi method, any criterion or alternative should be included in the decision-making problem when more than 75% of the experts invited to provide opinions agree with the specific criterion or alternative.

3.2. DEMATEL

DEMATEL was first proposed by Gabus and Fontela [79] to resolve complicated problems in the world. This graph theory-based method can be adopted to understand the influencing relationships among the criteria of an MCDM problem. In the past decade, the DEMATEL-based analytic framework and hybrid MCDM methods have been extensively implemented to resolve many decision-making problems, which include policy, management (e.g., [80,81,82]), education (e.g., [83,84,85,86]), engineering (e.g., [87]), medical devices (e.g., [88,89]), and other social problems (e.g., [90]).

The analytic procedure of DEMATEL can be formulated according to Tzeng and Huang [81], Yang et al. [91], and Huang et al. [87], as follows. At first, the initial direct relation matrix (IDRM) can be constructed according to the expert opinion questionnaires. The IDRM can be defined as ${\mathit{A}}_d=[a_{i_d{j}_d}], i_d,j_d\in [1,\cdots ,t_d]$, where $t_d$ is the number of objectives. Here, a 5-point Likert scale is adopted, where 5 indicates the highest degree of influence, while 1 indicates the lowest degree of influence. Then, the influence from the objective $i_d$ to the objective $j_d$ is denoted as $a_{i_d{j}_d}$, and is filled into the elements of the $i_d\mathrm{th}$ row and $j_d\mathrm{th}$ column in the IDRM. After that, the IDRM is normalized by multiplying the matrix with the factor ${\rho }_d$, i.e., ${\mathit{N}}_d={\rho }_d{\mathit{A}}_d$, where the maximum row sum and the maximum column sum can be selected; ${\rho }_d$ is equal to the reciprocal of the larger one of the two numbers. That is:

$${\rho }_d=\underset{}{\min }\left\{1/\underset{i_d}{\max }{\displaystyle \sum _{j_d=1}^{t_d}{a}_{i_d{j}_d}},1/\underset{j_d}{\max }{\displaystyle \sum _{i_d=1}^{t_d}{a}_{i_d{j}_d}}\right\},i_d,j_d\in \left\{1,2,\dots ,t_d\right\}.$$

(1)

The total relation matrix (TRM), ${\mathit{T}}_d={\left[t_{i_d{j}_d}\right]}_{t_d\times t_d}$, can be derived accordingly as follows:

$$\begin{array}{ll}{\mathit{T}}_d& ={\mathit{N}}_d+{\mathit{N}}_d^2+{\mathit{N}}_d^3\cdots +{\mathit{N}}_d{}^{\varsigma }\\ & ={\mathit{N}}_d({\mathit{I}}_d+{\mathit{N}}_d+{\mathit{N}}_d^2+\cdots +{\mathit{N}}_d{}^{\varsigma -1})({\mathit{I}}_d-{\mathit{N}}_d)({\mathit{I}}_d-{\mathit{N}}_d{)}^{-1}\\ & ={\mathit{N}}_d({\mathit{I}}_d-{\mathit{N}}_d{}^{\varsigma }){({\mathit{I}}_d-{\mathit{N}}_d)}^{-1}\\ & ={\mathit{N}}_d{({\mathit{I}}_d-{\mathit{N}}_d)}^{-1}, \end{array}$$

(2)

where $\varsigma \to \infty$, ${\mathit{I}}_d$ is the identity matrix. The row sum and column sum vectors of the TRM can be derived as ${\mathit{r}}_d$ and ${\mathit{c}}_d$, respectively. The sum of elements belonging to the $i_d\mathrm{th}$-row and the $j_d\mathrm{th}$-column can be denoted by $r_{i_d}$ and $c_{j_d}$, respectively. The influence relation map (IRM) of all the objectives can be derived by demonstrating the influence relationships, where $r_{i_d}+c_{i_d}$ and $r_{i_d}-c_{i_d}$ represent the horizontal and vertical axes of the IRM.

3.3. DANP

DANP, a method proposed by Prof. Gwo-Hshiung Tzeng [92,93,94], is an analytic method that integrates DEMATEL and ANP. The traditional ANP-based analytic framework requires a pre-defined structure of the decision-making problem. Usually, an IRM that is being defined by DEMATEL (e.g., [80]) or a structure being derived by other decision-making methods will be adopted. The derivations of such a structure need extra time. The DANP method that transforms the TRM being derived by DEMATEL into the supermatrix of the AN, as proposed by Saaty [25], can reduce the time needed for structure derivations. Here, the process of the DANP method is summarized below, based on [87,95].

Let ${\mathit{T}}_{\mathrm{C}}$ be equal to the transposed matrix of ${\mathit{T}}_d$ derived in Section 3.2. That is ${\mathit{T}}_{\mathrm{C}}={\mathit{T}}_d^t$. The ${\mathit{T}}_d$ can be divided into $m$ submatrices, according to the number of aspects ($m$) to which the objectives belong. In other words, ${\mathit{T}}_{\mathrm{C}}={\left[{\mathit{T}}_{C_{ij}}\right]}_{m\times m}.$ The submatrices can be denoted as ${\mathit{T}}_{C_{ij}}={\left[t_{i_u{j}_v}\right]}_{n_i\times n_j}$, where $1\le i_u\le i_{n_i}$ and $1\le j_v\le j_{n_j}$. Here, n_i and n_j are the numbers of objectives that belong to the ith aspect, ${\mathit{D}}_i$, and the jth aspect, ${\mathit{D}}_j$, respectively (see Equation (A11) of [87]). Then, every column of the submatrix, ${\mathit{T}}_{C_{ij}}$, should further be normalized by $d_{j_v}={\displaystyle \sum _{u=1}^{n_i}{t}_{i_u{j}_v}},v=1,\cdots ,j_{n_i}$ (see Equation (A12) of [87]). The normalized submatrix of ${\mathit{T}}_{C_{ij}}$ can be formulated as ${\mathit{T}}_{{{}_c}_{{}_{ij}}}^{(N)}={\left[\frac{t_{i_u{j}_{\nu }}}{d_{j_{\nu }}}\right]}_{{}_{n_i\times n_j}}$ according to Equation (A13) of [87]. The normalized TRM, ${\mathit{T}}_C^{(N)}$, can serve as the unweighted supermatrix, $\mathit{W}$, according to Equation (A6) of [87]. To derive the weighted supermatrix, the values of the elements belonging to each submatrix ${\mathit{T}}_{C_{ij}}$ that belongs to the matrix ${\mathit{T}}_C$ can be summed and filled into the matrix ${\mathit{T}}_D={\left[t_c{}_{{}_{i_d{j}_d}}\right]}_{m\times m}$, in which $t_c{}_{{}_{i_d{j}_d}}$ is the sum of all the elements belonging to the submatrix ${\mathit{T}}_{C_{i_d{j}_d}}$. Then, the matrix ${\mathit{T}}_D$ can be normalized as ${\mathit{T}}_D^{(N)}={\left[\frac{t_c{}_{{}_{i_d{j}_d}}}{d_{j_d}}\right]}_{m\times m}$ by normalizing each column to unity as follows, where $d_{j_d}={\displaystyle \sum _{i=1}^m{t}_c{}_{{}_{i_d{j}_d}}}$. The weighted supermatrix, $\mathit{\Pi }$, can thus be calculated by multiplying the transposed matrix, ${\mathit{T}}_D^{(N)}$, to $\mathit{W}$, i.e., $\mathit{\Pi }={\mathit{T}}_D^{(N)}{}^t\mathit{W}$. Then, the weighted supermatrix can be calculated by raising the weighted supermatrix to its limiting powers, or $\underset{{\theta }_n\to \infty }{\lim }{\mathit{\Pi }}^{{\theta }_n}$. Detailed further explanations of the above process can be found in [87]. The global priority vectors can be derived accordingly, along with the weights associated with every objective.

3.4. NDEA-MOP Model

In this study, the NDEA model proposed by Kao et al. [24] will be adopted as the basis for developing the theoretic framework, where the multiple-objective decision-making method is used to develop the NDEA-MOP model. In this section, the MOP model for the compensatory solution is shown below.

The MOP Model is based on the BCC model. The notations are defined, following the work performed by Tone and Tsutsui [22] and Kao et al. [24]. We assume a total of $n$ DMUs$(j=1,\cdots ,n)$ can be found in the performance evaluation problem. Each DMU consists of $K$ divisions $(k=1,\cdots ,K)$. We also assume that $m_k$ and $r_k$ are the numbers of inputs to and outputs from the $k\mathrm{th}$ division. The link streaming from Division $k$ to Division $h$ can be represented by $(k,h)$ and the set of links by $L$. The observed input resources to $DM{U}_j$ at Division $k$ are $\{{\mathit{x}}_j^k\in R_{+}^{m_k}\}$ $(j=1,\cdots ,n;k=1,\cdots ,K)$; the output products from $DM{U}_j$ at Division $k$ are $\{{\mathit{y}}_j^k\in R_{+}^{r_k}\}$ $(j=1,\cdots ,n;k=1,\cdots ,K)$; the linking intermediate products from Division $k$ to Division $h$ are $\{{\mathit{z}}_j^{(k,h)}\in R_{+}^{t_{(k,h)}}\}$ $(j=1,\cdots ,n;(k,h)\in L)$, where $t_{(k,h)}$ is the number of items in Link $(k,h)$. Based on the above notations, the formula for the BCC-MOP model can be defined as follows:

$$\begin{array}{c}\mathrm{Max} E_o^{} (\mathrm{DMU}\text{:} \mathrm{firm} \mathrm{level})\hfill \\ \mathrm{Max} E_o^k=\frac{{\displaystyle \sum _{r=1}^{r_k}{u}_r^k{Y}_{ro}^k+{\displaystyle \sum _{\forall (k,h)}^{}{\displaystyle \sum _{p=1}^{t_{(k,h)}}{\mu }_h^k{Z}_{op}^{(k,h)}}}-{\alpha }_k}}{{\displaystyle \sum _{i=1}^{m_k}{v}_i^k{X}_{io}^k+{\displaystyle \sum _{\forall (g,k)}^{}{\displaystyle \sum _{p=1}^{t(g,k)}{\omega }_g^k{Z}_{oq}^{(g,k)}}}}}, k=1,\dots ,K. (\mathrm{Divisional} \mathrm{Level})\hfill \\ s.t.\hfill \\ \frac{{\displaystyle \sum _{r=1}^{r_k}{u}_r^k{Y}_{rj}^k+{\displaystyle \sum _{\forall (k,h)}^{}{\displaystyle \sum _{p=1}^{t_{(k,h)}}{\mu }_h^k{Z}_{jp}^{(k,h)}}}-{\alpha }_k}}{{\displaystyle \sum _{i=1}^{m_k}{v}_i^k{X}_{ij}^k+{\displaystyle \sum _{\forall (g,k)}^{}{\displaystyle \sum _{q=1}^{t_{(g,k)}}{\omega }_g^k{Z}_{jq}^{(g,k)}}}}}\le 1,\hfill \\ {\displaystyle \sum _{i=1}^{m_k}{v}_i^k{X}_{ij}^k+{\displaystyle \sum _{g=1}^{t_{(g,k)}}{\omega }_g^k{Z}_{jq}^{(g,k)}}}=1,\hfill \\ j=1,2,\dots ,n; k=1,2,\dots ,K.\hfill \\ u_r^k,v_i^k,{\mu }_n^k,{\omega }_g^k\ge \epsilon \text{>}0; {\alpha }_k^{} \mathrm{is} \mathrm{unrestricted} \mathrm{in} \mathrm{sign}; k=1,2,\mathrm{\dots },K.\hfill \\ r=1,2,\dots {,r}_k; i=1,2,\dots ,m_k; all (a,h),(g,k)\in L.\hfill \end{array}$$

(3)

Here, the objective equation $E_0^k$ is defined to evaluate the efficiency of $DM{U}_O$ in Division $k$. $X_{ij}^k$ and $Y_{rj}^k$ are the $i\mathrm{th}$ input and the $r\mathrm{th}$ outputs of the $DM{U}_k$. $v_i^k$ is the weight of the input $X_{ij}^k$. $u_r^k$ is the weight of the output $Y_{rj}^k$. ${\omega }_g^k$ is the weight associated with the incoming link $Z_{jq}^{(g,k)}$. ${\mu }_h^k$ is the weight associated with the outgoing link $Z_{jp}^{(k,h)}$. $\epsilon$ is a non-negative minimum that ensures that all weights are non-negative. ${\alpha }_k^{}$ is a constant with no positive or negative restrictions. To solve the mathematical programming problem, ${\displaystyle \sum _{i=1}^{m_k}{v}_i^k{X}_{ij}^k+{\displaystyle \sum _{g=1}^{t_{(\mathrm{g},\mathrm{k})}}{\omega }_g^k{Z}_{jq}^{(g,k)}}}=1$ will be added to the original BCC model, so that the nonlinear programming problem can be transformed into a linear mathematical programming model. A feasible solution can thus be derived easily.

In order to solve the MOP problem, this research uses the NDEA-MOP method proposed by Kao et al. [24] to develop the following algorithms and proposes the following solutions. At first, the ideal solution for each objective is optimized independently. For $DM{U}_o$, $E_o^k$ is maximized independently, where $k=1,2,\dots ,K$. Thus, the maximum values, $E_o^{k_{}^{*}}$, can be derived accordingly. After that, the anti-ideal solution for every objective will be derived independently; that is, for every objective equation, $E_o^k$ is minimized as $E_o^{k_{}^{-}}$. Then, the membership function for every objective can be defined as

$${\mu }_{E_0^k}^{}(E_o^k)=(E_o^k-E_o^{k_{}^{-}})/(E_o^{k_{}^{*}}-E_o^{k_{}^{-}}), k=1,2,\cdots ,K.$$

(4)

Finally, the solutions can be derived according to Lee and Li [96]. The compensatory solution maximizes the weighted mean membership function of all objectives, as follows:

$$\begin{array}{c}Max\lambda ={\displaystyle \sum _{k=1}^k{w}_k{\mu }_{E_o^k}(E_o^k)}\hfill \\ \mathrm{s}.\mathrm{t}. \hfill \\ {\displaystyle \sum _{K=1}^K{w}_k=1};\hfill \\ {\mu }_{E_o^k}^{}(E_o^k)=(E_o^k-E_o^{k_{}^{-}})/(E_o^{k_{}^{*}}-E_o^{k_{}^{-}}), k=1,2,\cdots ,K,\hfill \end{array}$$

(5)

where $w_k$ is the weight for Division $k$. In this work, we adopt the structural model proposed by [22], as seen in Figure 2. The problem can easily be solved by using LINGO.

In general, the analytic process can be summarized as follows. First, users define DMUs, based on the literature review results. Then, the input and output data are formulated, according to the NDEA-MOP model defined in Section 3.3. The solution can be defined by LINGO, accordingly. Then, the performance of the DMUs can be evaluated. The process is demonstrated in Figure 3.

4. Empirical Study

In this section, the IAC performance of major Taiwanese universities of technology will be evaluated, based on the NDEA-MOP introduced in Section 3. First, the development and current status of IAC and the patent commercialization of higher-education institutes in Taiwan will be introduced in Section 4.1. Then, the modified Delphi method will be introduced to confirm the input and output variables to be fed into the NDEA-MOP models in Section 4.2. The weights associated with the objectives of the NDEA-MOP models will be derived in Section 4.3, using the DANP method. The patent commercialization problem in IAC will be modeled in Section 4.4. Finally, the analytic results will be demonstrated in Section 4.5.

4.1. IAC and Patent Commercialization in Taiwan

The IACs of Taiwan originated from the establishment of work-study programs provided by vocational senior high schools in 1969. Over the past 50 years, these work-study programs have already provided sufficient workforces to the industrial firms and have thus accelerated the economic growth and competitiveness of the state. During the past three decades, the need to upgrade Taiwanese industries to technology and service ones [97] has given birth to the IAC model. In the initiation of IACs in Taiwan, numerous limitations were placed on the collaboration between industrial firms and universities, due to the government’s restrictions on the development of such collaborations [98]. However, the Taiwanese government changed its policy and has encouraged collaboration between industrial firms and academic institutes since the 1980s [98]. To cross the gap between university education and the industrial requirements for manpower, and to enhance the capabilities of industrial innovation, the Taiwanese government aggressively encourages IAC. Nowadays, the Taiwanese Ministry of Science and Technology (MOST) plays the dominant role in developing IAC, focusing on patent applications, technology transfer, and commercialization [98,99].

Currently, most of the patented technologies developed by Taiwanese universities are being transferred to enterprises in the form of IAC [100]. However, most schools still need to rely on external resources for help with patent commercialization [100]. According to Wang et al. [101], in comparison with top foreign universities, Taiwanese universities are weaker when it comes to commercializing their research results into products; moreover, these products are usually less valuable in the market. According to the Taiwanese Ministry of Economic Affairs [102], about 5,000 patents were granted to Taiwanese universities in 2016. In general, only three out of every one thousand patents granted to Taiwanese universities can be commercialized [102]; thus, the patent commercialization rate of Taiwanese universities is on around the same scale and is still very limited [102]. It is clear that an evaluation of IAC’s patent commercialization performance can meaningfully enhance Taiwan’s innovation capability.

Higher education in Taiwan features a dual-track system of general universities alongside universities of technology [103]. The universities of technology aim to develop a higher-level workforce in the fields of technology, engineering, and management [104]. For some of the leading institutions, like the Taiwan University of Science and Technology, AIC is always the university’s main mission [105]. Thus, 20 Taiwanese universities of technology will be adopted as the focus for empirical studies of the performance evaluation of patent commercialization in IAC, based on the DANP-based NDEA-MOP.

4.2. Confirmation of Input and Output Variables Using the Modified Delphi Method

First, the structure model was adapted from the work by Tone and Tsutsui [22]. The indicators for IAC patent commercialization were collected from the Ministry of Education, Taiwan (https://udb.moe.edu.tw/ReportCategories; accessed on 6 July 2021). Then, the modified Delphi method was introduced to confirm the input and output of and linkages between the divisions of the structure model, based on the opinions collected from industry experts (see

Table 1. The list of experts.

No.	Gender	Exp. (Years)	Education	Industry	Title
1	Male	27	Ph.D.	Academia	Professor
2	Male	10	Ph.D.	Academia	Assistant Professor
3	Male	23	Ph.D.	Institute	Director
4	Female	19	Ph.D.	Institute	Principal Engineer
5	Male	20+	Ph.D.	Business	C.O.O.
6	Male	30	Master	Business	Consultant
7	Male	15	Master	Business	Manager
8	Male	20+	Master	Business	Senior Engineer

). Based on the modified Delphi method, all these inputs, outputs, or linkages are agreed upon by more than 75% of the experts polled and are regarded as acceptable (see

Table 2. The result from the questionnaire for the input and outputs.

Expert			1	2	3	4	5	6	7	8	A	D	A%	D%
Structure Model			A	A	A	A	A	A	A	A	8	0	100.000	0.000
Inputs	Faculty	Instructors	A	A	A	A	A	A	A	A	8	0	100.000	0.000
		Staff	D	D	D	D	D	D	D	D	0	8	0.000	100.000
	Financial resources	Budget	A	A	A	A	A	A	A	A	8	0	100.000	0.000
		Tuition	D	D	D	D	D	D	D	D	0	8	0.000	100.000
		Gov. Grants	A	A	A	A	A	A	A	A	8	0	100.000	0.000
		Ind. Grants	A	A	A	A	A	A	A	A	8	0	100.000	0.000
		Other Grants	A	A	A	A	A	A	A	A	8	0	100.000	0.000
Outputs	IPR	Patents	A	A	A	A	A	A	A	A	8	0	100.000	0.000
		Patent Licensed	A	A	A	A	A	A	A	A	8	0	100.000	0.000
		License Fee and Royalties	A	A	A	A	A	A	A	A	8	0	100.000	0.000

Note: A: agree; D: disagree.

) for the NDEA-MOP model.

4.3. Derivations of the Weights Associated with the Divisions

Based on the opinions provided by the polled experts, the influence of division $i_d$ on division $j_d$ is denoted as $a_{i_d{j}_d}$ in the IDRM. By adopting the process introduced in Section 3.2, the IRM can be derived as shown in

Table 3. IRM.

	D1	D2
D1	5.000	4.000
D2	3.625	5.000

Note: D₁ and D₂ are the abbreviations of Divisions 1 and 2.

, while the TRM can be derived as shown in

Table 4. TRM.

	D1	D2
D1	0.488	0.488
D2	0.512	0.512

. Then, the row sum and column sum vectors of the TRM can be derived as ${\mathit{r}}_d$ and ${\mathit{c}}_d$, respectively. All these aspects, as well as $r_{i_d}+c_{i_d}$ and $r_{i_d}-c_{i_d}$ versus each aspect, are demonstrated in

Table 5. Variables for $r_{i_d}+c_{i_d} ,$$r_{i_d}-c_{i_d}$ and weights versus each aspect.

Division 1	${\mathit{r}}_{{\mathit{i}}_{\mathit{d}}}$	${\mathit{c}}_{{\mathit{i}}_{\mathit{d}}}$	${\mathit{r}}_{{\mathit{i}}_{\mathit{d}}}+{\mathit{c}}_{{\mathit{i}}_{\mathit{d}}}$	${\mathit{r}}_{{\mathit{i}}_{\mathit{d}}}-{\mathit{c}}_{{\mathit{i}}_{\mathit{d}}}$	Weight
Division 1	47.000	44.750	91.750	2.250	48.800%
Division 2	44.750	47.000	91.750	−2.250	51.200%

. The IRM is demonstrated in Figure 4. Furthermore, the influence weights versus each division can be derived according to the procedure outlined in Section 3.3. Here, the academic department is defined as Division 1, while the university is defined as Division 2 of an IAC.

The significant confidence test proposed by Lu et al. [106] was introduced to verify whether the experts’ opinions are consistent, based on the IRM in Table 3. The significant confidence level $({\alpha }_c)$ can be derived according to Lu et al. [106], where ${\alpha }_c=\frac{1}{t_d^2}{\displaystyle {\sum }_{i_d=1}^{t_d}{\displaystyle {\sum }_{j_d=1}^{t_d}\frac{\left|d_{i_d{j}_d}^p-d_{i_d{j}_d}^{p-1}\right|}{d_{i_d{j}_d}^p}}}\times 100\%=0.343\%,$ which is smaller than 5%. That is, the significant confidence is 99.997%. Here, $p=7$ denotes the number of experts; $d_{i_d{j}_d}^p$ is the average influence of the objective $i_d$ on the objective $j_d$; and $t_d$ denotes the number of objectives. Thus, the experts’ opinions are consistent.

4.4. The NDEA-MOP Model with Variables and Data

The NDEA-MOP problem can thus be formulated. In the structure model, two divisions, which include the department and the university, are included (see Figure 5). The inputs to the academic department include government grants and industry grants in New Taiwan Dollars (NTD). The output of the university division is the number of patents, which includes, but is not limited to, contractual research, authorization fees, royalties, and technology transfer fees. However, due to the availability of the open data, only the license fee and royalty, as well as the number of patents licensed, are included in the model. The unit of “license fee of IP” is in NTD. $Lin{k}_{12}$ is the output from Division 1 to Division 2. The variable is defined as the total number of patents granted to the faculty belonging to a specific DMU or the technical university under investigation. In general, all the inputs, outputs, and linkages are summarized in

Table 6. Input, output, and link variables of IAC.

Division	Input	Linkage	Output
Department	Gov. Grants ($x_{10}^{(1)}$) Ind. Grants ($x_{20}^{(1)}$)	Patents ($z_{01}^{(1,2)}$)	N.A.
University	Other Grants ($x_{10}^{(2)}$) Faculty ($x_{20}^{(2)}$) Budget ($x_{30}^{(2)}$)	N.A.	Patent Licensed ($y_{10}^{(2)}$) License Fee and Royalties ($y_{20}^{(2)}$)

Note: N.A. = not applicable.

.

After configuring the research model, the open data for these DMUs were collected from the Ministry of Education, Taiwan (https://udb.moe.edu.tw/ReportCategories; accessed on 6 July 2021). The original data for inputs, outputs, and linkages are given in

Table 7. The original data of inputs, outputs and linkages (as of the year 2016).

DMU	Inputs					Outputs		Linkage
	Gov. Grants (1)	Industry Grants (1)	Other Grants (1)	Faculties	Budget	License Fee & Royalty	Patent Licensed	Link12 (Patents)
A1	92.948	213.667	60.708	462	2893.740	40.336	13	133
A2	250.753	71.437	20.443	370	2167.332	7.048	1	49
A3	136.947	151.589	36.694	445	3423.780	38.810	4	59
A4	111.685	51.201	28.601	266	1563.000	16.001	7	30
A5	59.714	44.240	15.167	355	1822.636	14.174	2	57
A6	62.452	75.942	22.663	330	1553.903	11.428	5	34
A7	7.222	28.916	9.244	286	1391.950	2.805	6	80
A8	23.441	13.705	6.328	403	1666.169	0.074	1	6
A9	29.108	77.585	42.432	576	3035.482	10.038	1	140
A10	64.649	34.758	13.583	376	1767.798	12.536	7	88
A11	52.318	48.049	0.721	493	2064.406	2.796	1	25
A12	2.515	68.857	5.419	328	1638.733	4.721	5	14
A13	3.615	11.554	6.246	325	1308.637	0.848	5	7
A14	38.041	165.729	26.331	440	2209.844	19.427	9	18
A15	7.259	12.231	2.279	249	1198.237	0.807	2	25
A16	10.354	13.887	4.827	275	934.030	0.469	1	34
A17	6.683	17.921	2.739	268	1184.800	0.122	2	2
A18	21.349	38.288	4.398	243	1074.339	10.548	9	67
A19	0.500	13.545	1.251	228	1206.356	0.264	2 (2)	1
A20	4.700	7.067	55.841	354	1728.185	0.250	3	3

Note: ⁽¹⁾ Million NTD; ⁽²⁾ the patent is licensed twice.

. The data are further normalized by dividing by the maximum number of their respective columns, as shown in

Table 8. Normalized inputs, outputs, and linkage.

DMU	Inputs					Outputs		Linkage
	Gov. Grants (1)	Industry Grants (1)	Other Grants (1)	Faculties	Budget	License Fee & Royalty	Patent Licensed	Link12 (Patents)
A1	0.371	1.000	1.000	0.802	0.845	1.000	1.000	0.950
A2	1.000	0.334	0.337	0.642	0.633	0.175	0.077	0.350
A3	0.546	0.709	0.604	0.773	1.000	0.962	0.308	0.421
A4	0.445	0.240	0.471	0.462	0.457	0.397	0.538	0.214
A5	0.238	0.207	0.250	0.616	0.532	0.351	0.154	0.407
A6	0.249	0.355	0.373	0.573	0.454	0.283	0.385	0.243
A7	0.029	0.135	0.152	0.497	0.407	0.070	0.462	0.571
A8	0.093	0.064	0.104	0.700	0.487	0.002	0.077	0.043
A9	0.116	0.363	0.699	1.000	0.887	0.249	0.077	1.000
A10	0.258	0.163	0.224	0.653	0.516	0.311	0.538	0.629
A11	0.209	0.225	0.012	0.856	0.603	0.069	0.077	0.179
A12	0.010	0.322	0.089	0.569	0.479	0.117	0.385	0.100
A13	0.014	0.054	0.103	0.564	0.382	0.021	0.385	0.050
A14	0.152	0.776	0.434	0.764	0.645	0.482	0.692	0.129
A15	0.029	0.057	0.038	0.432	0.350	0.020	0.154	0.179
A16	0.041	0.065	0.080	0.477	0.273	0.012	0.077	0.243
A17	0.027	0.084	0.045	0.465	0.346	0.003	0.154	0.014
A18	0.085	0.179	0.072	0.422	0.314	0.261	0.692	0.479
A19	0.002	0.063	0.021	0.396	0.352	0.007	0.154	0.007
A20	0.019	0.033	0.920	0.615	0.505	0.006	0.231	0.021

Note: ⁽¹⁾ Million NTD.

.

4.5. The Efficiency of DMUs

In this subsection, the efficiency of the DMUs associated with each division will be derived separately. First, the ideal solution $E_0^{k_{}^{*}}$ and the anti-ideal solution $E_0^{k_{}^{-}}$ will be derived by using the BCC and CCR models, respectively, with LINGO. For Division 1, the inputs are government grants ($x_{10}^{(1)}$) and industry grants ($x_{20}^{(1)}$), while the output is the number of patents ($z_{01}^{(1,2)}$) granted to the faculty. The number of patents ($z_{01}^{(1,2)}$) granted to the faculty also serves as the linkage and one of the inputs to Division 2. Except for the number of patents, other grants ($x_{10}^{(2)}$), the number of faculty ($x_{20}^{(2)}$), and budgets ($x_{30}^{(2)}$) will serve as the inputs of Division 2, while patents licensed to industrial firms and other institutes ($y_{10}^{(2)}$), as well as license fees and royalties ($y_{20}^{(2)}$), will serve as the outputs. The mathematical programming equations for the BCC of DMU1 are provided below as an example. The compensatory results for other DMUs can be derived in the same style:

$$\begin{array}{c}\mathrm{Max}{E}_0^1=0.950z_{01}^{(1,2)}-{\alpha }_1;\hfill \\ 0.371x_{10}^{\left(1\right)}-1.000x_{20}^{\left(1\right)}=1.000;\hfill \\ 0.950z_{01}^{(1,2)}-0.371x_{10}^{\left(1\right)}-1.000x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.350z_{01}^{(1,2)}-1.000x_{10}^{\left(1\right)}-0.334x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.421z_{01}^{(1,2)}-0.546x_{10}^{\left(1\right)}-0.709x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.214z_{01}^{(1,2)}-0.445x_{10}^{\left(1\right)}-0.240x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.407z_{01}^{(1,2)}-0.238x_{10}^{\left(1\right)}-0.207x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.243z_{01}^{(1,2)}-0.249x_{10}^{\left(1\right)}-0.355x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.571z_{01}^{(1,2)}-0.029x_{10}^{\left(1\right)}-0.135x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.043z_{01}^{(1,2)}-0.093x_{10}^{\left(1\right)}-0.064x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 1.000z_{01}^{(1,2)}-0.116x_{10}^{\left(1\right)}-0.363x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.629z_{01}^{(1,2)}-0.258x_{10}^{\left(1\right)}-0.163x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.179z_{01}^{(1,2)}-0.209x_{10}^{\left(1\right)}-0.225x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.100z_{01}^{(1,2)}-0.010x_{10}^{\left(1\right)}-0.322x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.050z_{01}^{(1,2)}-0.014x_{10}^{\left(1\right)}-0.054x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.129z_{01}^{(1,2)}-0.152x_{10}^{\left(1\right)}-0.776x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.179z_{01}^{(1,2)}-0.029x_{10}^{\left(1\right)}-0.057x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.243z_{01}^{(1,2)}-0.041x_{10}^{\left(1\right)}-0.065x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.014z_{01}^{(1,2)}-0.027x_{10}^{\left(1\right)}-0.084x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.479z_{01}^{(1,2)}-0.085x_{10}^{\left(1\right)}-0.179x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.007z_{01}^{(1,2)}-0.002x_{10}^{\left(1\right)}-0.063x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.021z_{01}^{(1,2)}-0.019x_{10}^{\left(1\right)}-0.033x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ x_{10}^{\left(1\right)},x_{20}^{\left(1\right)},z_{01}^{(1,2)}\ge 0.000001;\hfill \end{array}$$

The CCR results for every DMU can also be derived by removing ${\alpha }_1$ from the above equations. The results derived by both the CCR and BCC models are demonstrated in

Table 9. The results of the separation models.

DMU	CCR				BCC
	Division 1		Division 2		Division 1		Division 2
	Effcy.	Rank	Effcy.	Rank	Effcy.	Rank	Effcy.	Rank
A1	0.225	10	1.000	1	0.336	9	1.000	1
A2	0.248	9	0.304	17	0.248	10	0.304	17
A3	0.140	17	1.000	1	0.140	18	1.000	1
A4	0.211	11	1.000	1	0.211	12	1.000	1
A5	0.465	7	1.000	1	0.465	7	1.000	1
A6	0.162	14	0.715	13	0.162	15	0.715	13
A7	1.000	1	0.567	15	1.000	1	0.567	15
A8	0.159	15	0.200	19	0.159	16	0.200	19
A9	0.651	4	0.271	18	1.000	1	0.271	18
A10	0.912	2	0.666	14	1.000	1	0.666	14
A11	0.188	12	1.000	1	0.188	13	1.000	1
A12	0.508	6	1.000	1	0.508	6	1.000	1
A13	0.130	18	1.000	1	0.219	11	1.000	1
A14	0.431	8	1.000	1	0.431	8	1.000	1
A15	0.043	19	0.455	16	0.043	19	0.455	16
A16	0.884	3	0.170	20	0.884	4	0.170	20
A17	0.039	20	0.819	12	0.039	20	0.819	12
A18	0.633	5	1.000	1	0.633	5	1.000	1
A19	0.178	13	1.000	1	0.178	14	1.000	1
A20	0.150	16	0.833	11	0.150	17	1.000	1

Note: DMUs with values in bold are efficient units. Effcy. is the abbreviation of efficiency.

. For the evaluations of the DMUs with the CCR model from the Division 1 aspect (academic departments), DMU A₇ is the most efficient unit (Table 9). For the evaluations of the DMUs with the CCR model from the Division 2 aspect (university level), DMUs A₁, A₃, A₄, A₅, A₁₁, A₁₂, A₁₃, A₁₄, A₁₈, and A₁₉ are efficient. Further, regarding DMU evaluations with the BCC model from the Division 1 aspect (academic departments), DMUs A₇, A₉, and A₁₀ are the efficient units, while A₁, A₃, A₄, A₅, A₁₁, A₁₂, A₁₃, A₁₄, A₁₈, A₁₉, and A₂₀ are efficient for Division 2 evaluations (Table 9).

Following the performance evaluations of independent divisions by the BCC and CCR models, the compensatory solution for evaluating the performance of two linked divisions will be derived. First, the performance will be evaluated using the BCC model, based on the structure model demonstrated in Figure 5 and the normalized inputs, outputs, and linkage in Table 8. Based on Equations (3)–(5), as well as the ideal solutions demonstrated in Table 9, the compensating solution to maximize a weighted average of all the membership functions of the target can be derived. (Here, the anti-ideal solutions are all 0s.) Because the overall performance, $E_0={\displaystyle {\sum }_{k=1}^K{w}_k{E}_0^k}$, will be measured for every DMU, by adopting the weights derived using the DANP in Section 4.3, the weights associated with Divisions 1 and 2, which are 0.488 and 0.512, will be adopted. The equation is:

$$overall efficiency=0.488\times Divison1+0.512\times Division2.$$

(6)

Thus, the mathematical programming model for DMU A_1, based on the BCC model, can be formulated as follows:

$$\begin{array}{c}\mathrm{Max}\lambda =w_1{\mu }_{E_0^1}^{}\left(E_0^1\right)+w_2{\mu }_{E_0^2}^{}\left(E_0^2\right)\hfill \\ w_1+w_2=1;\hfill \\ {\mu }_{E_0^1}^{}\left(E_0^1\right)=(E_0^1-0)/(0.336-0)\hfill \\ {\mu }_{E_0^2}^{}\left(E_0^2\right)=(E_0^2-0)/(1.000-0)\hfill \\ E_0^1=0.950z_{01}^{(1,2)}-{\alpha }_1;\hfill \\ 0.371x_{10}^{\left(1\right)}-1.000x_{20}^{\left(1\right)}=1.000;\hfill \\ E_0^2=1.000y_{10}^{\left(2\right)}+1.000y_{20}^{\left(2\right)}-{\alpha }_2;\hfill \\ 1.000x_{10}^{\left(2\right)}+0.802x_{20}^{\left(2\right)}+0.845x_{30}^{\left(2\right)}=1.000;\hfill \\ 0.950z_{01}^{(1,2)}-0.371x_{10}^{\left(1\right)}-1.000x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.350z_{01}^{(1,2)}-1.000x_{10}^{\left(1\right)}-0.334x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.421z_{01}^{(1,2)}-0.546x_{10}^{\left(1\right)}-0.709x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.214z_{01}^{(1,2)}-0.445x_{10}^{\left(1\right)}-0.240x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.407z_{01}^{(1,2)}-0.238x_{10}^{\left(1\right)}-0.207x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.243z_{01}^{(1,2)}-0.249x_{10}^{\left(1\right)}-0.355x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.571z_{01}^{(1,2)}-0.029x_{10}^{\left(1\right)}-0.135x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.043z_{01}^{(1,2)}-0.093x_{10}^{\left(1\right)}-0.064x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 1.000z_{01}^{(1,2)}-0.116x_{10}^{\left(1\right)}-0.363x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.629z_{01}^{(1,2)}-0.258x_{10}^{\left(1\right)}-0.163x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.179z_{01}^{(1,2)}-0.209x_{10}^{\left(1\right)}-0.225x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.100z_{01}^{(1,2)}-0.010x_{10}^{\left(1\right)}-0.322x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.050z_{01}^{(1,2)}-0.014x_{10}^{\left(1\right)}-0.054x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.129z_{01}^{(1,2)}-0.152x_{10}^{\left(1\right)}-0.776x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.179z_{01}^{(1,2)}-0.029x_{10}^{\left(1\right)}-0.057x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.243z_{01}^{(1,2)}-0.041x_{10}^{\left(1\right)}-0.065x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.014z_{01}^{(1,2)}-0.027x_{10}^{\left(1\right)}-0.084x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.479z_{01}^{(1,2)}-0.085x_{10}^{\left(1\right)}-0.179x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.007z_{01}^{(1,2)}-0.002x_{10}^{\left(1\right)}-0.063x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 0.021z_{01}^{(1,2)}-0.019x_{10}^{\left(1\right)}-0.033x_{20}^{\left(1\right)}-{\alpha }_1<=0;\hfill \\ 1.000y_{10}^{\left(2\right)}+1.000y_{20}^{\left(2\right)}-1.000x_{10}^{\left(2\right)}-0.802x_{20}^{\left(2\right)}-0.845x_{30}^{\left(2\right)}-0.950z_{01}^{\left(1,2\right)}-{\alpha }_2<=0;\hfill \\ 0.175y_{10}^{\left(2\right)}+0.077y_{20}^{\left(2\right)}-0.337x_{10}^{\left(2\right)}-0.642x_{20}^{\left(2\right)}-0.633x_{30}^{\left(2\right)}-0.350z_{01}^{\left(1,2\right)}-{\alpha }_2<=0;\hfill \\ 0.962y_{10}^{\left(2\right)}+0.308y_{20}^{\left(2\right)}-0.604x_{10}^{\left(2\right)}-0.773x_{20}^{\left(2\right)}-1.000x_{30}^{\left(2\right)}-0.421z_{01}^{\left(1,2\right)}-{\alpha }_2<=0;\hfill \\ 0.397y_{10}^{\left(2\right)}+0.538y_{20}^{\left(2\right)}-0.471x_{10}^{\left(2\right)}-0.462x_{20}^{\left(2\right)}-0.457x_{30}^{\left(2\right)}-0.214z_{01}^{\left(1,2\right)}-{\alpha }_2<=0;\hfill \\ 0.351y_{10}^{\left(2\right)}+0.154y_{20}^{\left(2\right)}-0.250x_{10}^{\left(2\right)}-0.616x_{20}^{\left(2\right)}-0.532x_{30}^{\left(2\right)}-0.407z_{01}^{\left(1,2\right)}-{\alpha }_2<=0;\hfill \\ 0.283y_{10}^{\left(2\right)}+0.385y_{20}^{\left(2\right)}-0.373x_{10}^{\left(2\right)}-0.573x_{20}^{\left(2\right)}-0.454x_{30}^{\left(2\right)}-0.243z_{01}^{\left(1,2\right)}-{\alpha }_2<=0;\hfill \\ 0.070y_{10}^{\left(2\right)}+0.462y_{20}^{\left(2\right)}-0.152x_{10}^{\left(2\right)}-0.497x_{20}^{\left(2\right)}-0.407x_{30}^{\left(2\right)}-0.571z_{01}^{\left(1,2\right)}-{\alpha }_2<=0;\hfill \\ 0.002y_{10}^{\left(2\right)}+0.077y_{20}^{\left(2\right)}-0.104x_{10}^{\left(2\right)}-0.700x_{20}^{\left(2\right)}-0.487x_{30}^{\left(2\right)}-0.043z_{01}^{\left(1,2\right)}-{\alpha }_2<=0;\hfill \\ 0.249y_{10}^{\left(2\right)}+0.077y_{20}^{\left(2\right)}-0.699x_{10}^{\left(2\right)}-1.000x_{20}^{\left(2\right)}-0.887x_{30}^{\left(2\right)}-1.000z_{01}^{\left(1,2\right)}-{\alpha }_2<=0;\hfill \\ 0.311y_{10}^{\left(2\right)}+0.538y_{20}^{\left(2\right)}-0.224x_{10}^{\left(2\right)}-0.653x_{20}^{\left(2\right)}-0.516x_{30}^{\left(2\right)}-0.629z_{01}^{\left(1,2\right)}-{\alpha }_2<=0;\hfill \\ 0.069y_{10}^{\left(2\right)}+0.077y_{20}^{\left(2\right)}-0.012x_{10}^{\left(2\right)}-0.856x_{20}^{\left(2\right)}-0.603x_{30}^{\left(2\right)}-0.179z_{01}^{\left(1,2\right)}-{\alpha }_2<=0;\hfill \\ 0.117y_{10}^{\left(2\right)}+0.385y_{20}^{\left(2\right)}-0.089x_{10}^{\left(2\right)}-0.569x_{20}^{\left(2\right)}-0.479x_{30}^{\left(2\right)}-0.100z_{01}^{\left(1,2\right)}-{\alpha }_2<=0;\hfill \\ 0.021y_{10}^{\left(2\right)}+0.385y_{20}^{\left(2\right)}-0.103x_{10}^{\left(2\right)}-0.564x_{20}^{\left(2\right)}-0.382x_{30}^{\left(2\right)}-0.050z_{01}^{\left(1,2\right)}-{\alpha }_2<=0;\hfill \\ 0.482y_{10}^{\left(2\right)}+0.692y_{20}^{\left(2\right)}-0.434x_{10}^{\left(2\right)}-0.764x_{20}^{\left(2\right)}-0.645x_{30}^{\left(2\right)}-0.129z_{01}^{\left(1,2\right)}-{\alpha }_2<=0;\hfill \\ 0.020y_{10}^{\left(2\right)}+0.154y_{20}^{\left(2\right)}-0.038x_{10}^{\left(2\right)}-0.432x_{20}^{\left(2\right)}-0.350x_{30}^{\left(2\right)}-0.179z_{01}^{\left(1,2\right)}-{\alpha }_2<=0;\hfill \\ 0.012y_{10}^{\left(2\right)}+0.077y_{20}^{\left(2\right)}-0.080x_{10}^{\left(2\right)}-0.477x_{20}^{\left(2\right)}-0.273x_{30}^{\left(2\right)}-0.243z_{01}^{\left(1,2\right)}-{\alpha }_2<=0;\hfill \\ 0.003y_{10}^{\left(2\right)}+0.154y_{20}^{\left(2\right)}-0.045x_{10}^{\left(2\right)}-0.465x_{20}^{\left(2\right)}-0.346x_{30}^{\left(2\right)}-0.014z_{01}^{\left(1,2\right)}-{\alpha }_2<=0;\hfill \\ 0.261y_{10}^{\left(2\right)}+0.692y_{20}^{\left(2\right)}-0.072x_{10}^{\left(2\right)}-0.422x_{20}^{\left(2\right)}-0.314x_{30}^{\left(2\right)}-0.479z_{01}^{\left(1,2\right)}-{\alpha }_2<=0;\hfill \\ 0.007y_{10}^{\left(2\right)}+0.154y_{20}^{\left(2\right)}-0.021x_{10}^{\left(2\right)}-0.396x_{20}^{\left(2\right)}-0.352x_{30}^{\left(2\right)}-0.007z_{01}^{\left(1,2\right)}-{\alpha }_2<=0;\hfill \\ 0.006y_{10}^{\left(2\right)}+0.231y_{20}^{\left(2\right)}-0.920x_{10}^{\left(2\right)}-0.615x_{20}^{\left(2\right)}-0.505x_{30}^{\left(2\right)}-0.021z_{01}^{\left(1,2\right)}-{\alpha }_2<=0;\hfill \\ x_{10}^{\left(1\right)},x_{20}^{\left(1\right)},x_{10}^{\left(2\right)},x_{20}^{\left(2\right)},x_{30}^{\left(2\right)},y_{10}^{\left(2\right)},y_{20}^{\left(2\right)},z_{01}^{(1,2)}\ge 0.000001.\hfill \end{array}$$

The CCR model can easily be formulated simply by removing ${\alpha }_1$ and ${\alpha }_2$ from the above equations. The results derived by both the BCC and CCR models are given in

Table 10. The results of compensatory models.

DMU	CCR						BCC
	Division 1		Division 2		Overall Perf.		Division 1		Division 2		Overall Perf.
	Effcy.	Rank	Effcy.	Rank	Effcy.	Rank	Effcy.	Rank	Effcy.	Rank	Effcy.	Rank
A1	0.225	11	1.000	1	0.622	5	0.336	10	1.000	1	0.676	4
A2	0.248	10	1.000	1	0.633	4	0.248	11	1.000	1	0.633	5
A3	0.140	18	1.000	1	0.580	11	0.140	18	1.000	1	0.580	11
A4	0.211	12	1.000	1	0.615	6	0.211	12	1.000	1	0.615	6
A5	0.465	9	0.258	17	0.359	17	0.465	9	0.258	17	0.359	17
A6	0.162	15	0.710	10	0.443	13	0.162	15	0.710	10	0.443	13
A7	0.619	5	0.545	14	0.581	10	0.619	5	0.545	14	0.581	10
A8	0.159	16	0.189	18	0.174	20	0.159	16	0.189	18	0.174	20
A9	0.651	3	0.173	19	0.406	15	0.651	3	0.173	19	0.406	15
A10	0.574	7	0.615	12	0.595	9	0.574	7	0.615	12	0.595	9
A11	0.188	13	1.000	1	0.604	7	0.188	13	1.000	1	0.604	7
A12	0.508	8	0.903	9	0.710	3	0.508	8	1.000	1	0.760	3
A13	0.043	19	0.538	15	0.296	19	0.043	19	0.538	15	0.296	19
A14	0.742	1	1.000	1	0.874	1	0.742	1	1.000	1	0.874	1
A15	0.742	1	0.270	16	0.500	12	0.742	1	0.270	16	0.500	12
A16	0.586	6	0.148	20	0.362	16	0.586	6	0.148	20	0.362	16
A17	0.039	20	0.604	13	0.328	18	0.039	20	0.604	13	0.328	18
A18	0.633	4	1.000	1	0.821	2	0.633	4	1.000	1	0.821	2
A19	0.178	14	1.000	1	0.599	8	0.178	14	1.000	1	0.599	8
A20	0.150	17	0.663	11	0.413	14	0.150	17	0.663	11	0.413	14

Note: DMUs with values in bold are efficient units. Effcy. is the abbreviation of efficiency.

. Based on the analytic results derived by the CCR model, A₁₄ and A₁₅ are efficient DMUs in Division 1 (academic department level) (Table 10). For Division 2 (university level) evaluations, DMUs A₁, A₂, A₃, A₄, A₁₁, A₁₄, A₁₈, and A₁₉ are efficient. For the evaluation of overall performance by the CCR model, DMUs A₁₄ and A₁₈ top other DMUs and are ranked first and second, respectively. From the perspective of the BCC model (Table 10), A₁₄ and A₁₅ are efficient DMUs in Division 1. For Division 2 (university level) evaluations, DMUs A₁, A₂, A₃, A₄, A₁₁, A₁₂, A₁₄, A₁₈, and A₁₉ are efficient. Finally, the evaluation of overall performance by the BCC model generates the same results as the ones generated by using the CCR model. DMUs A₁₄ and A₁₈ are ranked as the first and second, respectively (Table 10).

5. Discussion

The traditional DEA models have been widely adopted for solving performance evaluation problems. Though these models are very popular, they cannot correctly evaluate the performance of systems, e.g., supply chains, value networks, industrial–academic collaborations, etc. In particular, the linkages between the components of the systems have always been neglected. Thus, the traditional DEA models are not suitable for calculating the efficiency of such systems. The NDEA-MOP, an NDEA framework considering multiple objectives proposed in this work, can resolve this kind of problem and can evaluate the relative performance of systems. Especially, the integration of the DANP with the NDEA-MOP is very useful for solving performance evaluation problems in the real world. Compared with traditional DEA models, this approach is indeed a better one. In the following section, the managerial implications, advances in research methods, as well as limitations and future research possibilities, will be discussed.

5.1. Managerial Implications

According to the empirical results in Section 4, the managerial implications can be discussed from the aspect of the necessity of analyzing IAC problems in patent commercialization from the NDEA-MOP perspective, the advancement of research in the patent commercialization of IAC, the discussion of school management, and the influence of the relationship between the university administration and academic departments.

We cannot ignore the role of the intermediate data when we evaluate the relative efficiency of DMUs, in the case of multiple inputs and multiple outputs [107]. It is common in IAC that part of the outputs of Division 1 become the inputs of Division 2. Although traditional DEA methods have been applied to measure the relative efficiency of DMUs, based on their inputs and outputs, the DEA model is unable to evaluate the transfer of inputs and outputs inside a DMU. NDEA-MOP takes the input/output transfer among divisions inside a DMU into consideration. To provide a more precise performance evaluation of IAC, we suggest that decision-makers should use the NDEA-MOP model.

There are at least two divisions in a DMU in IAC. It is important to determine the appropriate weights in different divisions in order to evaluate the overall performance of IAC. This study suggests that it is appropriate to adopt the weights derived by the DANP in Section 4.3, instead of letting decision-makers determine the weights themselves.

The overall patent commercialization performance of IAC in DMUs is affected by the efficiency of both Divisions 1 and 2. We find that the overall performance may not be good when one division is efficient but the other one is inefficient, for example, in the results derived by the compensatory models (Table 10), A₁₅ is an efficient DMU in Division 1. However, the DMU is inefficient from both the aspect of Division 2 and the overall performance. Similarly, the DMU A₁₉ is efficient from the perspective of Division 2. However, the DMU is inefficient, from the perspectives of Division 1 and the overall performance. Thus, we suggest that universities that want to improve their overall performance in IAC patent commercialization should simultaneously monitor the efficiency of Divisions 1 and 2. The way to improve the efficiency of Division 1, or the academic department, is to encourage the faculty to apply for patents after they complete the IAC. To increase the possibility of obtaining patents, government administrators or firm managers who decide to provide grants to universities should evaluate the past performance of IAC, the capabilities of the research teams, and the potential of proposals from the patent commercialization aspect. In addition, it is important to continue to monitor performance during the lifespan of projects. From the Division 2 perspective, one way to improve efficiency is via budget-planning for patent commercialization. Inventions in the faculties should be checked regularly (e.g., quarterly or annually) for patentable technologies. Furthermore, key performance indicators can be defined to help monitor the performance of university patent commercialization in general, and the patent commercialization performance of IACs in particular.

Regarding the performance evaluation results, we found that about one-half of the university administrations were already efficient, whether seen from the perspective of the separation model or of the compensatory solution.

Table 11. Efficient DMUs in different analytic models.

Method		Division 1	Division 2	Overall Performance
Separation	CCR	A7	A1, A3, A4, A5, A11, A12, A13, A14, A18 and A19	N.A.
	BCC	A7, A9, A10	A1, A3, A4, A5, A11, A12, A13, A14, A18, A19 and A20	N.A.
Compensatory Solution	CCR	N.A.	A1, A2, A3, A4, A11, A12, A14, A18, A19	A14
	BCC	N.A.	A1, A2, A3, A4, A11, A12, A14, A15, A18, A19	A14

Note: N.A. = not applicable.

below summarizes the results of the separation model and the compensatory solution from Table 9 and Table 10.

Based on the separation model, 1 DMU of the CCR model and 3 DMUs of the BCC model are efficient from the perspective of Division 1 (see Table 11). Based on Division 2, 10 DMUs of the CCR model and 11 DMUs of the BCC model are efficient. However, no single example of the DMUs is efficient. Since each division in a DMU is treated as an independent unit and the linking activities between other divisions are totally neglected, the separation model is insufficient for evaluating the efficiencies [24]. Therefore, the results derived from the separation model can only serve as a contrast, which can help demonstrate the effectiveness of the compensatory solution as a way to derive meaningful performance evaluation results.

Based on the compensatory solution, 9 DMUs of the CCR model and 10 DMUs of the BCC model are efficient, from the perspective of Division 2 (see Table 11). However, no DMU was efficient from the perspective of Division 1. The analytic results may be due to the universities’ aggressive pursuit of every kind of government grant from the Ministry of Education (e.g., the Aim for the Top University Project, Higher Education SPROUT Project, etc.). Furthermore, due to the historical faculty promotion system, most Taiwanese professors were dedicated to pursuing academic publications in indexed journals (e.g., SCI and SSCI journals). Few faculty members were dedicated to applying for patents. The analytic results are also consistent with the work of Hsu et al. [108], who argued that Taiwan’s universities are dedicated to basic research in pursuit of academic publications rather than to applied research; the criteria of university faculty promotion in Taiwan tend to emphasize academic publications [108]. Thus, the performances of academic departments in IAC were less satisfactory.

From the aspect of the overall performance of IAC, A₁, A₂, A₄, A₁₁, A₁₂, A₁₄, and A₁₈ is higher than 0.6. The results suggest that these DMUs are more efficient. However, the overall performance of all DMUs is still far from satisfactory. The results are consistent with the current status of Taiwanese patent commercialization. As has already been mentioned in Section 4.1, only three out of every one thousand patents granted to Taiwanese citizens can be commercialized [102], not to mention the commercialization of patents granted to Taiwanese universities. Furthermore, the overall performance of some DMUs is poor because their academic divisions are less efficient, which may be due to the overemphasizing of academic publications in the current system.

Except for the aspect of performance evaluation, the current study has advanced the research on higher education, especially on vocational education. In the past, very few works have tried to study the influence of relationships between the university administration (Division 2) and academic departments (Division 1). Furthermore, very few works have tried to explore the dominance of both parties. In this work, we found that the academic department is playing a more influential role in developing industrial–academic cooperation. In contrast, the university administration is influenced by the academic departments. In Taiwan, traditionally, it was widely recognized that professors initiated the IAC projects. The university administration helped finish the administrative work. This analytic result is consistent with traditional cognition. Furthermore, although academic departments play a more influential role, both parties have almost equal weights in the collaboration. Even so, the university administration is a little more important in such performance evaluation or in resolving decision-making problems. The result may be due to the influence of universities’ resources on industry cooperation, as was argued by Nam et al. [109]. Another possible way that the university administration can influence the performance of academic departments by aggressively pursuing IAC is the compensation policy initiated by several Taiwanese universities in recent years. In this arrangement, the faculty will receive monetary and other rewards. Such reward policies encourage the faculty to aggressively initiate or pursue IAC projects and commercialize patents. Such an influence relationship can also be reflected in the weights of both parties in the IAC.

5.2. Advances in the Research Method

This work mainly advances research methods in the integration of MCDM and MODM methods, in the introduction of real weights into the NDEA-MOP models for performance evaluation in general, and in the evaluations of IAC in particular. Solving MCDM problems by combining different methods is not new, as many authors have used this approach. For example, Lu et al. [106] adopted DEMATEL, DANP and VIseKriterijumska Optimizacija I Kompromisno Resenje (VIKOR) to evaluate the factors that influence the adoption of radio frequency identification (RFID). Yang et al. [91] integrated DEMATEL, DANP, and VIKOR to evaluate and select disaster-recovery sites for information systems of academic big data. Huang et al. [88] evaluated the investment projects of spinal medical device firms, using the real option and DANP, with modified VIKOR (DANP-mV)-based MCDM methods. In fact, the DANP method has been widely adopted for weighting criteria in other studies. However, very few scholars have tried to integrate DANP to derive the weights associated with the objectives of an MODM problem. However, the mutual influence relationships among objectives can always be found in an MODM problem in general, and MOP-based DEA problems in particular. Such influential relationships among the objectives are always crucial and cannot be neglected. For example, in the empirical case in this work, the mutual influence relationships between the objective of academic departments and the objective of a university (see Figure 4) are very significant and meaningful (see Section 5.1 for an analysis of the influences of university administrations on the IAC-performance of academic departments).

Furthermore, people were used to dealing with MCDM and MODM problems separately. As mentioned above, numerous works have been dedicated to DEMATEL- or DANP-based research. However, the influence of relationships and the weights derived therefrom were seldom introduced into real-world mathematical programming problems with multiple objectives. However, such integrations are very important. So, the integration of real weights into the MODM programs crosses the research gap of fusing MCDM and MODM problems in resolving real-world problems.

5.3. Limitations and Future Possibilities

In this work, only Taiwanese universities of technology were investigated. Furthermore, due to the very low availability of financial information from the industry, the patent commercialization performance of IAC was only investigated based on the open data from the Taiwanese Ministry of Education. Thus, the results may be controversial. In the future, the outputs of the university division may include more detailed information (e.g., authorization fees, royalties, and technology transfer fees) when available. The introduction of industry data into the analytic framework, as well as analyses based on the data of multiple periods, will also be very helpful in offering more insights into performance evaluation.

6. Conclusions

IAC is mutually beneficial to firms and universities. Through industrial collaboration, universities obtain funding to undertake research and have the opportunities to take an invention or product from conception to market. Through cooperation with universities, firms that are aligned with early-stage research may find opportunities to get a head start on their competition. IAC has become a trend in recent years. Patent commercialization is one of the major channels for transferring the academic knowledge embedded in technology to industrial firms or for realizing the benefits of IAC. However, the rates of the successful commercialization of academic patents belonging to higher-education institutions in most countries are still low. Thus, finding ways to evaluate the patent commercialization performance of IAC becomes a critical issue.

Universities that undertake industry–academic projects vary in size, resources, and faculty capabilities. In addition, at least two divisions (departments and universities) are interrelated. The outputs of IAC from departments, such as patents, are the inputs of universities. These characteristics make the DEA unsuitable for evaluating the performance of IAC. In order to evaluate the performance of IAC, this study not only considers the variables of the inputs and outputs from the industry–university cooperation but also uses NDEA-MOP to consider the link between the university and its departments.

In this work, the performance of the patent commercialization of major Taiwanese universities of technology was evaluated, based on the NDEA-MOP model. From interpreting the analytic results, it is shown that the academic departments and the faculty play a more influential role in the patent commercialization in IAC. About half of the university administrations have been efficient. However, the performance of most academic departments should be enhanced. The reasons for the inefficiency of academic departments may be due to the overemphasis of journal paper publications in the promotion system. Further, university administrations play a more important role than academic departments in IAC, due to the compensation policy initiated by Taiwanese universities in recent years. The policy can encourage the faculty to aggressively pursue IACs and commercializing patents.

This study has various suggestions for future research. First, this research uses only one year’s data, obtained from the government’s public information, for the analyses. Future research can collect multi-year data to evaluate the performance of IAC. Second, this study uses 20 universities of technology as the sample. Thus, the outputs of IAC mainly consider the number of patents, patents licensed, license fees, and royalties. Future research can use corporate-sponsored research projects in business schools in universities as a sample to evaluate the overall performance of IAC. These studies should choose other outputs for their analyses.