# A Rough Hybrid Multicriteria Decision-Making Model for Improving the Quality of a Research Information System

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Literature Review

#### 2.1. Statistical Methods for the D&M Model

#### 2.2. MCDM Models

#### 2.3. Research Gaps

## 3. Methodology

#### 3.1. The Rough Number

- Step 1: Conform lower and upper approximations of rough number for each crisp scale.

- Step 2: Compute the interval value of the rough number.

_{i}and y

_{i}are denoted the elements in the lower approximation or upper approximation of z

_{Θ}, respectively. In addition, N

_{L}and N

_{U}are represented the total number of respondents involved in the lower and upper approximations of z

_{Θ}, respectively.

- Step 3: Drive operations for two rough numbers.

_{Θ}), as shown in Equation (4):

- Step 4: Transfer into crisp value from rough interval value. When needing to compare analysis for criteria or alternatives ranking, the de-roughness of the rough number into a crisp value can be used by:

#### 3.2. The RDANP Method

- Step 1: Build a rough original influence relationship matrix $\overline{\mathit{M}}$ on a measuring scale of 0–4 ranging from “no influence (0)” to “very high influential (4)”.

_{ij}. We checked the consistency of the raw data, using the formula as follows:

_{n}

_{xn}= ${\left[{m}_{ij}^{\nabla},{m}_{ij}^{\mathsf{\Delta}}\right]}_{n\times n}$, where is n the number of criteria.

- Step 2: Obtain the rough initial influence relationship matrix $\overline{\mathit{P}}$ = [${\overline{p}}_{ij}$]
_{nxn}, which is the multiplication of $\overline{\mathit{M}}$ and v.$$\overline{\mathit{P}}=v\times \overline{\mathit{M}}$$$$v=\mathrm{min}\left[\begin{array}{cc}\frac{1}{\underset{i}{\mathrm{max}}{\displaystyle {\sum}_{j}^{n}\left|{m}_{ij}^{u}\right|}},& \frac{1}{\underset{j}{\mathrm{max}}{\displaystyle {\sum}_{i}^{n}\left|{m}_{ij}^{u}\right|}}\end{array}\right]$$

- Step 3: Calculate the rough total influence relationship matrix $\overline{\mathit{T}}$ with Equation (11). The element ${\overline{t}}_{ij}$ indicates the rough interdependent effects that criteria i has on criteria j, where I is an identity matrix.$$\overline{\mathit{T}}=\overline{\mathit{P}}+{\overline{\mathit{P}}}^{2}+\cdots +{\overline{\mathit{P}}}^{\mathsf{\Omega}}=\overline{\mathit{P}}{(\mathit{I}-\overline{\mathit{P}})}^{-1}\text{}\mathrm{when}\text{}\mathsf{\Omega}\to \infty $$

- Step 4: Derive each column sum (${\overline{c}}_{i}$) and row sum (${\overline{r}}_{i}$) from the rough total influence relationship matrix $\overline{\mathit{T}}$ as follows:$${\overline{c}}_{j}={\left({\overline{c}}_{j}\right)}_{1\times n}={\left({\overline{c}}_{j}\right)}_{n\times 1}^{\prime}={\left[{\displaystyle \sum _{i=1}^{n}{\overline{t}}_{ij}}\right]}^{\prime}$$$${\overline{r}}_{i}={\left({\overline{r}}_{i}\right)}_{n\times 1}=\left[{\displaystyle \sum _{j=1}^{n}{\overline{t}}_{ij}}\right]$$

- Step 5: Get the RINRM for whole evaluation model.

- Step 6: Derive rough total influence relationship matrix ${\overline{\mathit{T}}}_{C}$ based on the criteria and ${\overline{\mathit{T}}}_{D}$ based on the dimensions.

_{i}denotes the ith dimension; c

_{ij}denotes the jth criteria in the ith dimension. For example, we get a crisp value by averaging ${\overline{\mathit{T}}}_{c}^{mn}$, where ${\overline{\mathit{T}}}_{c}^{mn}$ means the extent to which the criteria in the mth dimension affect the criteria in the nth dimension. Then, we get the ${\overline{\mathit{T}}}_{D}$ by averaging the ${\overline{\mathit{T}}}_{c}^{mn}$ in the ${\overline{\mathit{T}}}_{C}$.

- Step 7: Obtain the rough unweighted supermatrix.

- Step 8: Derive the rough weighted supermatrix.

_{n}is the number of criteria in dimension n; and j

_{m}is the number of criteria in the dimension m.

- Step 9: Obtain the rough influential weights.

#### 3.3. The COPRAS-R Method with Aspiration Level

- Step 1: Build a rough decision matrix.

- Step 2: Obtain an aspirated rough decision matrix.

- Step 3: Calculate the rough proximity degree of gray relation.

_{j}is obtained from Equation (23).

- Step 4: Integrate the aspirated proximity index.

_{s}represents the de-roughness degree of satisfaction on each criterion for alternative s. The relative proximity H

_{s}of the criteria are calculated as shown in Equation (27):

- Step 5: Calculate the utility ratio for each alternative.

## 4. Case Study

#### 4.1. Identification Dimensions and Criteria for Evaluation of a Research Information System

#### 4.2. Measuring the Relationship between Dimensions and Criteria by RDANP Method

_{11}on C

_{12}is denoted by ${C}_{11}-{C}_{12}=\left\{3,3,3,3,3,3,4,3,3,3\right\}$, which can be converted into a rough number through Equations (1)–(8) as follows:

_{12}–C

_{11}= {[3, 3.1], [3, 3.1], [3, 3.1], [3, 3.1], [3, 3.1], [3, 3.1], [3.1, 4], [3, 3.1], [3, 3.1], [3, 3.1]}.

_{12}–C

_{11}(i.e., m

_{12}) is as follows:

_{i}− c

_{i}) and (r

_{i}+ c

_{i}) values are related to system quality, meaning that system quality not only has the greatest total impact of the four dimensions, but also has the most profound impact on the other three dimensions. Therefore, system quality affects information quality, service quality, and intention to use; furthermore, it is also affected by information quality and service quality, and is the key to the quality of the scientific research information system. Intention to use (C

_{4}) has the smallest (r

_{i}+ c

_{i}) value (1.11), and its (r

_{i}− c

_{i}) is negative (−0.22), meaning that it is greatly influenced by other factors and is the resulting element in the evaluation system. The influential network relationship map (INRM) (Figure 1) of the four dimensions and their respective subsystems can be drawn according to Table 3, Table 4 and Table 5. As shown in Figure 1, the arrow source represents the cause element, and is pointed to the result destination. System quality (C

_{1}), information quality (C

_{2}), and service quality (C

_{3}) are the three main factors that affect use by users. From Figure 1, we can also see the most important criteria for each dimension. For example, organization design (C

_{33}) affects IS training (C

_{31}) and assurance (C

_{32}); this shows that it is the most critical criterion in the dimension of service quality (C

_{3}). Therefore, to maintain the enthusiasm of users in using the system, we need to improve the quality of systems, services, and information. Additionally, service quality and system quality have an important impact on information quality, and service quality and system quality show an interactive relationship.

#### 4.3. Obtaining the Weights of Each Dimension and Criterion

_{4}, 0.174) is the most important dimension, followed by system quality (C

_{1}, 0.150), information quality (C

_{2}, 0.143), and service quality (C

_{3}, 0.137). Among the 15 criteria, efficiency (C

_{44}, 0.051), effectiveness (C

_{43}, 0.050), and frequency of use (C

_{41}) are the three most important criteria.

#### 4.4. Evaluating and Improving School Information System Using the COPRAS-R Method

_{42}, 0.018), timeliness (C

_{24}, 0.025), and response time (C

_{14}, 0.026) are the three criteria that have smaller relative significance value than the others.

#### 4.5. Discussion

_{1}) has the largest (r

_{i}+ c

_{i}) value and (r

_{i}− c

_{i}) value, and only intention to use (C

_{4}) has a negative (r

_{i}− c

_{i}) value. The results indicate that system quality is the most important reason for influencing intention to use, and information quality (C

_{2}) and service quality (C

_{3}) also have a significant impact on intention to use (C

_{4}). This result means that intention to use (C

_{4}) is the result variable for the entire model, which reflects the usage of the information system, and it is the value of user satisfaction with system performance. The ease of use (C

_{11}), timeliness (C

_{23}), and organization design (C

_{33}) are the three main elements of the respective subsystems, according to Figure 1. Therefore, if we want to improve the frequency of use (C

_{41}), effectiveness (C

_{43}), and efficiency (C

_{44}) of the information system, improving system usability, reducing system response time, and optimizing system design are objectives worth considering.

_{4}) has the greatest weight, followed by system quality (C

_{1}), information quality (C

_{2}), and service quality (C

_{3}). These results are consistent with the results of RDEMATEL analysis, and system quality (C

_{1}) and intention to use (C

_{4}) are the two priorities in the evaluation information system. Considering the criteria, efficiency (C

_{44}, 0.051), effectiveness (C

_{43}, 0.050), and frequency of use (C

_{41}, 0.047) are the three most important criteria, followed by assurance (C

_{31}, 0.047), IS training (C

_{32}, 0.046), and organization design (C

_{33}, 0.044). From their point of view, users are highly concerned with the effectiveness and efficiency of the system, which play a major role in user satisfaction with the system. In addition, service quality is becoming increasingly important for information system products. Users care whether the system design can fully meet their needs and pay attention to the developer’s service commitment and careful training.

_{44}) and effectiveness (C

_{43}) ranked in the top two in terms of relative significance value. This result reflects that the research information system of XMUT can improve the efficiency of teachers and researchers. Most teachers are quite satisfied with the effectiveness and efficiency of the system. As the informatization of Chinese universities deepens, the use of information systems has become a trend. University staff must change the way they work using the internet to improve work efficiency and effectiveness. The navigation pattern (C

_{42}) is the ranking with a relative significance value. As with the development of mobile internet technology, users want to access the system through mobile terminals. However, the speed of development of mobile APPs in Chinese universities is slow, and cannot meet the needs of teachers and researchers. Timelines (C

_{23}) and response time (C

_{14}) are ranked lower in the performance evaluation of this system. As we know, users are quite disgusted with the slow response of the system and slow updating of content. If users find that the system is not responding, they are likely to give up on using the system. Therefore, improving the response speed of the system itself and speeding up the content updating process are important prerequisites for satisfying users. Assurance (C

_{31}), IS training (C

_{32}), and organization design (C

_{33}) are urgently needed to improve performance, according to Table 10. The service quality of the system has long been a problem in the construction of information technology in Chinese universities. Many system developers focus on improving system performance while ignoring user guidance and training. Therefore, the poor quality of service is also an important reason for user dissatisfaction with the system.

## 5. Conclusions

_{4}) and system quality (C

_{1}) are the two most significant indicators for the evaluation of the quality of research information systems for universities in China, making up 60% of the total weight, and system quality (C

_{1}) is the main factor affecting the intension to use (C

_{4}). Therefore, ease of use (C

_{11}), integration (C

_{12}), reliability (C

_{13}), and response time (C

_{14}) have an important influence on the frequency of use (C

_{41}), effectiveness (C

_{42}), and efficiency (C

_{43}) of the system. This study offers several contributions to the literature. First, this paper attempts to apply the D&M model to the university research platform using the hybrid MCDM method, and fills the gaps in the current research using confirmatory statistical analysis without analysis from expert decision management. Second, we applied the R-DANP approach, which analyzes the causal relationship between indicators and obscures the subjective limitations of experts to obtain the weights of the dimensions and criteria. Taking the scientific research system of Chinese universities as a case, we proposed corresponding strategies to improve the performance of scientific research systems.

## Author Contributions

## Funding

## Conflicts of Interest

## Appendix A

Criterion | No. 1 | No. 2 | No. 3 | No. 4 | No. 5 | No. 6 | No. 7 | No. 8 | No. 9 | No. 10 |
---|---|---|---|---|---|---|---|---|---|---|

C_{11}–C_{11} | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |

C_{11}–C_{12} | 3 | 3 | 3 | 3 | 3 | 3 | 4 | 3 | 3 | 3 |

C_{11}–C_{13} | 2 | 3 | 3 | 3 | 2 | 3 | 3 | 2 | 3 | 1 |

C_{11}–C_{14} | 3 | 3 | 4 | 4 | 4 | 3 | 4 | 4 | 4 | 3 |

C_{11}–C_{21} | 1 | 2 | 0 | 0 | 3 | 3 | 2 | 1 | 3 | 1 |

C_{11}–C_{22} | 4 | 1 | 1 | 1 | 2 | 3 | 3 | 1 | 3 | 1 |

C_{11}–C_{23} | 2 | 1 | 4 | 1 | 3 | 3 | 1 | 2 | 2 | 2 |

C_{11}–C_{24} | 3 | 0 | 1 | 1 | 0 | 3 | 3 | 1 | 2 | 1 |

C_{11}–C_{31} | 4 | 3 | 4 | 4 | 3 | 3 | 4 | 3 | 3 | 1 |

C_{11}–C_{32} | 4 | 3 | 4 | 3 | 3 | 3 | 4 | 2 | 2 | 1 |

C_{11}–C_{33} | 4 | 3 | 2 | 3 | 4 | 3 | 4 | 2 | 1 | 0 |

C_{11}–C_{41} | 0 | 1 | 2 | 3 | 3 | 3 | 1 | 3 | 3 | 2 |

C_{11}–C_{42} | 2 | 2 | 0 | 0 | 1 | 2 | 1 | 0 | 0 | 0 |

C_{11}–C_{43} | 0 | 2 | 3 | 1 | 3 | 1 | 1 | 2 | 2 | 1 |

C_{11}–C_{44} | 1 | 1 | 2 | 2 | 3 | 1 | 1 | 2 | 2 | 1 |

## References

- Perc, M. Growth and structure of Slovenia’s scientific collaboration network. J. Informetr.
**2010**, 4, 475–482. [Google Scholar] [CrossRef] - Perc, M. Zip’s law and log-normal distribution in measures of sicentific output across fields and institutions: 40 years of years of Slovenia’s research as an example. J. Informetr.
**2010**, 4, 358–364. [Google Scholar] [CrossRef] - Lužar, B.; Levnajić, Z.; Povh, J.; Perc, M. Community structure and the evolution of interdisciplinarity in Slovenia’s scientific collaboration network. PLoS ONE
**2014**, 4, e94429. [Google Scholar] [CrossRef] [PubMed] - Helbing, D.; Brockmann, D.; Chadefaux, T.; Donnay, K.; Blanke, U.; Woolley-Meza, O.; Moussaid, M.; Johansson, A.; Krause, J.; Schutte, S.; et al. Saving Human Lives: What Complexity Science and Information Systems can Contribute. J. Stat. Phys.
**2015**, 158, 735–781. [Google Scholar] [CrossRef] [PubMed] - DeLone, W.H.; McLean, E.R. Information systems success: The quest for the dependent variable. Inf. Syst. Res.
**1992**, 3, 60–95. [Google Scholar] [CrossRef] - DeLone, W.H.; McLean, E.R. The DeLone and McLean Model of Information Systems Success: A Ten-Year Update. J. Manag. Inf. Syst.
**2003**, 19, 9–30. [Google Scholar] - Pitt, L.F.; Watson, R.T.; Kavan, C.B. Service quality: A measure of information systems effectiveness. MIS Q.
**1995**, 19, 173–187. [Google Scholar] [CrossRef] - Seddon, P.B.; Staples, S.; Patnayakuni, R.; Bowtell, M. Dimensions of information systems success. Commun. Assoc. Inf. Syst.
**1999**, 2, 2966–2976. [Google Scholar] [CrossRef] - Yeo, H.J. Information System Success Disparity between Developer and Users. Indian J. Sci. Technol.
**2016**, 9, 1–6. [Google Scholar] - Zheng, Y.M.; Zhao, K.X.; Stylianou, A. The impacts of information quality and system quality on users’ continuance intention in information-exchange virtual communities: An empirical investigation. Decis. Support Syst.
**2013**, 56, 513–524. [Google Scholar] [CrossRef] - Azeroual, O.; Saake, G.; Schallehn, E. Analyzing data quality issues in research information systems via data profiling. Int. J. Inf. Manag.
**2018**, 41, 50–56. [Google Scholar] [CrossRef] - Dwivedi, Y.K.; Wastell, D.; Laumer, S.; Henriksen, H.Z.D.; Myers, M.; Bunker, D. Research on information systems failures and successes: Status update and future directions. Inf. Syst. Front.
**2014**, 17, 143–157. [Google Scholar] [CrossRef] [Green Version] - Grudzień, Ł.; Hamrol, A. Information quality in design process docunment of quality management systems. Int. J. Inf. Manag.
**2016**, 36, 599–606. [Google Scholar] [CrossRef] - Lin, H.F. An application of fuzzy AHP for evaluating course website quality. Comput. Educ.
**2010**, 54, 877–888. [Google Scholar] [CrossRef] - Hsu, C.C.; Lee, Y.S. Exploring the Critical Factors Influencing the Quality of Blog Interfaces Using the Decision-Making Trial and Evaluation Laboratory DEMATEL Method. Behav. Inf. Technol.
**2014**, 33, 84–194. [Google Scholar] [CrossRef] - Tsai, W.H.; Chou, W.C.; Leu, J.D. An effectiveness evaluation model for the web-based marketing of airline industry. Expert Syst. Appl.
**2011**, 38, 15499–15516. [Google Scholar] [CrossRef] - Su, H.S.; Tzeng, G.H.; Hu, S.K. Cloud e-learning service strategies for improving e-learning innovation performance in a fuzzy environment by using a new hybrid fuzzy multiple attribute decision-making model. Interact. Learn. Environ.
**2015**, 24, 812–1835. [Google Scholar] [CrossRef] - Ayyash, M.M. Scrutiny of Relationship between E-Banking Information Quality Dimensions and Customer Satisfaction. J. Comput. Sci.
**2017**, 13, 78–90. [Google Scholar] [CrossRef] [Green Version] - Molla, A.; Licker, P.S. E-commerce systems success: An attempt to extend and respecify the Delone and McLean model of IS Success. J. Electron. Commer. Res.
**2001**, 2, 131–141. [Google Scholar] - Bernroider, E.W.N. IT governance for enterprise resource planning supported by the DeLone-McLean model of information systems success. Inf. Manag.
**2008**, 45, 257–269. [Google Scholar] [CrossRef] - Chen, C.Y.; Chang, R.E.; Hung, M.C.; Lin, M.H. Assessing the quality of a Web-based Learning System for Nurses. J. Med. Syst.
**2009**, 33, 317–325. [Google Scholar] [CrossRef] [PubMed] - Bossen, C.; Jensen, L.G.; Udsen, F.W. Evaluation of a comprehensive EHR based on the DeLone and McLean model for IS success: Approach, results, and success factors. J. Med. Syst.
**2013**, 82, 940–953. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Kyoung, W.C.; Sung, K.B.; Ji, H.R.; Kyeong, N.K.; Chang, H.A.; Yong, M.C. Performance Evaluation of Public Hospital Information Systems by the Information System Success Model. Health Inf. Res.
**2015**, 21, 43–48. [Google Scholar] [Green Version] - Chiu, P.S.; Chao, I.; Kao, C.C.; Pu, Y.H.; Huang, Y.M. Implementation and evaluation of mobile e-books in a cloud bookcase using the information system sucess model. Libr. Hi Tech
**2016**, 34, 207–223. [Google Scholar] [CrossRef] - Yang, M.; Shao, Z.; Liu, Q.; Liu, C.Y. Understanding the quality factors that influence the continuance intention of students toward articipation in MOOCs. Educ. Technol. Res. Dev.
**2017**, 65, 1195–1214. [Google Scholar] [CrossRef] - Lin, H.H.; Wang, Y.S.; Li, C.R.; Shih, S.J.; Lin, S.J. The Measurement and Dimensionality of Mobile Learning Systems Success: Two-Stage Development and Validation. J. Educ. Comput. Res.
**2017**, 55, 1–22. [Google Scholar] [CrossRef] - Wu, C.H.; Chen, S.C. Understanding the relationships of critical factors to Facebook educational usage intention. Internet Res.
**2015**, 25, 262–278. [Google Scholar] [CrossRef] - Wang, Y.S. Assessing e-commerce systems success: A respecification and validation of the DeLone and McLean model of IS success. Inf. Syst. J.
**2008**, 18, 529–557. [Google Scholar] [CrossRef] - Sebetci, Ö.; Aksu, G. Evaluating e-government systems in Turkey: The case of the ‘e-movable system’. Inf. Polity
**2014**, 19, 225–243. [Google Scholar] [CrossRef] - Hu, X.L. Effectiveness of information technology in reducing corruption in China: A validation of the DeLone and McLean information systems success model. Electron. Libr.
**2015**, 33, 52–64. [Google Scholar] - Tzeng, G.H.; Chiang, C.H.; Li, C.W. Evaluating intertwined effects in e-learning programs: A novel hybrid MCDM model based on factor analysis and DEMATEL. Expert Syst. Appl.
**2007**, 32, 1028–1044. [Google Scholar] [CrossRef] - Tsai, W.H.; Chou, W.C. Selecting management systems for sustainable development in SMEs: A novel hybrid model based on DEMATEL, ANP, and ZOGP. Expert Syst. Appl.
**2009**, 36, 1444–1458. [Google Scholar] [CrossRef] - Liou, J.J.H.; Yen, L.; Tzeng, G.H. Building an effective safety management system for airlines. J. Air Transp. Manag.
**2008**, 14, 20–26. [Google Scholar] [CrossRef] - Chen, J.K.; Chen, I.S. Using a novel conjunctive MCDM approach based on DEMATEL, fuzzy ANP, and TOPSIS as an innovation support system for Taiwanese higher education. Expert Syst. Appl.
**2010**, 37, 1981–1990. [Google Scholar] [CrossRef] - Sun, C.C. Identifying critical success factors in EDA industry using DEMATEL method. Int. J. Comput. Intell. Syst.
**2015**, 8, 208–218. [Google Scholar] [CrossRef] - Hsu, C.C.; Liou, J.J.H.; Chuang, Y.C. Integrating DANP and modified grey relation theory for the selection of an outsourcing provider. Expert Syst. Appl.
**2013**, 40, 2297–2304. [Google Scholar] [CrossRef] - Huang, C.N.; Liou, J.J.H.; Chuang, Y.C. A method for exploring the interdependencies and importance of critical infrastructures. Knowl. Based. Syst.
**2014**, 55, 66–74. [Google Scholar] [CrossRef] - Liou, J.J.H. Building an effective system for carbon reduction management. J. Clean. Prod.
**2015**, 103, 353–361. [Google Scholar] [CrossRef] - Shao, Q.G.; Liou, J.J.H.; Weng, S.S.; Chuang, Y.C. Improving the Green Building Evaluation System in China Based on the DANP Method. Sustainability
**2018**, 10, 1173. [Google Scholar] [CrossRef] - Bai, C.G.; Sarkis, J. A grey-based DEMATEL model for evaluating business process management critical success factors. Int. J. Prod. Econ.
**2013**, 146, 281–292. [Google Scholar] [CrossRef] - Lin, R.J. Using fuzzy DEMATEL to evaluate the green supply chain management practices. J. Clean. Prod.
**2013**, 40, 32–39. [Google Scholar] [CrossRef] - Govindan, K.; Khodaverdi, R.; Vafadarnikjoo, A. Intuitionistic fuzzy based DEMATEL method for developing green practices and performances in a green supply chain. Expert Syst. Appl.
**2015**, 42, 7202–7220. [Google Scholar] [CrossRef] - Song, W.Y.; Cao, J.T. A rough DEMATEL-based approach for evaluating interaction between requirements of product-service system. Comput. Ind. Eng.
**2017**, 110, 353–363. [Google Scholar] [CrossRef] - Pamučar, D.; Mihajlović, M.; Obradović, R.; Atanasković, P. Novel approach to group multi-criteria decision making based on interval rough numbers: Hybrid DEMATEL-ANP-MAIRCA model. Expert Syst. Appl.
**2017**, 88, 58–80. [Google Scholar] [CrossRef] [Green Version] - Pawlak, Z. Rough sets. Int. J. Comput. Inf. Sci.
**1982**, 11, 341–356. [Google Scholar] [CrossRef] - Zhai, L.Y.; Khoo, L.P.; Zhong, Z.W. A rough set enhanced fuzzy approach to quality function deployment. Int. J. Adv. Manuf. Technol.
**2008**, 37, 613–624. [Google Scholar] [CrossRef] - Shyng, J.Y.; Shieh, H.M.; Tzeng, G.H.; Hsieh, S.H. Using FSBT technique with Rough Set Theory for personal investment portfolio analysis. Eur. J. Oper. Res.
**2010**, 201, 601–607. [Google Scholar] [CrossRef] - Han, S.; Jin, X.N.; Li, J.X. An assessment method for the impact of missing data in the rough set-based decision fusion. Intell. Data Anal.
**2016**, 20, 1267–1284. [Google Scholar] [CrossRef] - Liou, J.J.H.; Chuang, Y.C.; Hsu, C.C. Improving airline service quality based on rough set theory and flow graphs. J. Ind. Prod. Eng.
**2016**, 33, 1231–1233. [Google Scholar] [CrossRef] - Chiu, W.Y.; Tzeng, G.H.; Li, H.L. A new hybrid MCDM model combining DANP with VIKOR to improve e-store business. Knowl. Based. Syst.
**2013**, 37, 48–61. [Google Scholar] [CrossRef] - Song, W.; Ming, X.; Liu, H.C. Identifying critical risk factors of sustainable supply chain management: A rough strength-relation analysis method. J. Clean. Prod.
**2017**, 143, 100–115. [Google Scholar] [CrossRef] - Shen, K.Y.; Yan, M.R.; Tzeng, G.H. Combining VIKOR-DANP model for glamor stock selection and stock performance improvement. Knowl. Based. Syst.
**2014**, 58, 86–97. [Google Scholar] [CrossRef] - Zavadskas, E.K.; Kaklauskas, A.; Šarka, V. The new method of multicriteria complex proportional assesment projects. Technol. Econ. Dev. Econ.
**1994**, 1, 131–139. [Google Scholar] - Zavadskas, E.K.; Turskis, Z.; Tamosaitiene, J.; Marina, V. Multicriteria Selection of Project Managers by Applying Grey Criteria. Technol. Econ. Dev. Econ.
**2008**, 14, 462–477. [Google Scholar] [CrossRef] - Podvezko, V. The comparative analysis of MCDA methods SAW and COPRAS. Eng. Econ.
**2011**, 22, 134–146. [Google Scholar] [CrossRef] - Zolfani, S.H.; Rezaeiniya, N.; Zavadskas, E.K.; Turskis, Z. Froset roads location based on AHP and COPRAS-G method: An emprical study based on Iran. E M Ekon. Manag.
**2011**, 14, 6–21. [Google Scholar] - Liou, J.J.H.; Tamosaitiene, J.; Zavadskas, E.K.; Tzeng, G.H. New hybrid COPRAS-G MADM Model for improving and selecting suppliers in green supply chain management. Int. J. Prod. Res.
**2016**, 54, 114–134. [Google Scholar] [CrossRef] - Chatterjee, K.; Pamucar, D.; Zavadskas, E.K. Evaluating the performance of suppliers based on using the R’AMATEL-MAIRCA method for green supply chain implementation in electronics industry. J. Clean. Prod.
**2018**, 184, 101–129. [Google Scholar] [CrossRef] - Xie, M.; Wu, L.F.; Li, B.; Li, Z.C. A novel hybrid multivariate nonlinear grey model for forecasting the traffic-related emissions. Appl. Math. Model.
**2019**. [Google Scholar] [CrossRef]

**Table 1.**Evaluation indicators for the scientific research information system. IS: information system.

Dimension | Criteria | Description |
---|---|---|

System quality (C_{1}) | Ease of use (C_{11}) | Does not require excessive professional guidance |

Integration (C_{12}) | System function integration level | |

Reliability (C_{13}) | System robustness, few system crashes | |

Response time (C_{14}) | The reaction time after users make a request to the system | |

Information quality (C_{2}) | Accuracy (C_{21}) | Accuracy of the information delivered by the system |

Completeness (C_{22}) | Integrity of the information supplied by the system | |

Timelines (C_{23}) | System information update speed | |

Usefulness (C_{24}) | Value of the information produced by the system | |

Service quality (C_{3}) | Assurance (C_{31}) | Frequency and effect of enterprise maintenance system |

IS training (C_{32}) | Effect of training scientific research personnel | |

Organization design (C_{33}) | Service awareness and management improvement for system design | |

Intention to user (C_{4}) | User frequency (C_{41}) | Number of times the user uses the system |

Navigation patterns (C_{42}) | How users access the system (computer or mobile phone) | |

Effectiveness (C_{43}) | Does accessing the system help to improve job performance? | |

Efficiency (C_{44}) | Does productivity increase after accessing the system? |

C_{11} | C_{12} | C_{13} | C_{14} | C_{21} | … | C_{44} | |
---|---|---|---|---|---|---|---|

C_{11} | [0.00, 0.00] | [2.40, 3.17] | [2.71, 3.64] | [2.94, 3.65] | [2.12, 3.27] | [2.81, 3.78] | |

C_{12} | [3.01, 3.19] | [0.00, 0.00] | [2.35, 3.06] | [2.48, 3.13] | [2.12, 3.27] | [3.25, 3.75] | |

C_{13} | [2.13, 2.85] | [2.33, 3.62] | [0.00, 0.00] | [2.83, 3.37] | [0.74, 2.13] | [3.25, 3.75] | |

C_{14} | [3.36, 3.84] | [2.47, 3.52] | [2.28, 3.48] | [0.00, 0.00] | [0.24, 0.98] | [2.75, 3.64] | |

C_{21} | [0.86, 2.34] | [1.07, 2.71] | [1.33, 2.60] | [0.57, 1.62] | [0.00, 0.00] | [3.02, 3.76] | |

C_{22} | [1.35, 2.68] | [1.59, 2.60] | [1.09, 2.46] | [0.38, 1.67] | [1.68, 3.42] | [2.87, 3.52] | |

C_{23} | [1.52, 2.73] | [1.35, 2.06] | [1.65, 2.60] | [1.41, 2.35] | [1.95, 3.42] | [3.13, 3.85] | |

C_{24} | [0.80, 2.23] | [2.12, 3.27] | [1.69, 2.51] | [0.78, 2.20] | [2.83, 3.75] | [3.16, 3.64] | |

C_{31} | [2.71, 3.64] | [1.31, 2.87] | [2.61, 3.18] | [1.81, 2.99] | [1.54, 3.23] | [1.66, 3.14] | |

C_{32} | [2.28, 3.48] | [1.12, 2.27] | [1.49, 2.31] | [0.52, 1.73] | [1.24, 2.32] | [2.36, 3.25] | |

C_{33} | [1.68, 3.42] | [1.75, 3.55] | [1.90, 3.42] | [1.68, 3.42] | [2.63, 3.36] | [2.00, 3.00] | |

C_{41} | [1.42, 2.72] | [1.49, 2.64] | [0.38, 1.67] | [0.75, 2.08] | [0.45, 2.25] | [2.24, 2.98] | |

C_{42} | [0.30, 1.32] | [0.25, 1.43] | [0.16, 1.11] | [0.58, 2.21] | [0.16, 1.32] | [1.28, 2.48] | |

C_{43} | [1.02, 2.19] | [1.40, 2.17] | [0.38, 1.67] | [0.54, 1.86] | [0.65, 1.94] | [2.75, 3.64] | |

C_{44} | [1.24, 1.98] | [1.49, 2.31] | [0.66, 2.14] | [0.86, 2.34] | [0.75, 2.08] | [0.00, 0.00] |

C_{11} | C_{12} | C_{13} | C_{14} | C_{21} | … | C_{44} | |
---|---|---|---|---|---|---|---|

C_{11} | [0.04, 0.24] | [0.08, 0.30] | [0.09, 0.29] | [0.09, 0.28] | [0.07, 0.29] | [0.11, 0.37] | |

C_{12} | [0.10, 0.29] | [0.04, 0.23] | [0.08, 0.27] | [0.08, 0.26] | [0.07, 0.28] | [0.12, 0.35] | |

C_{13} | [0.07, 0.27] | [0.08, 0.28] | [0.03, 0.20] | [0.08, 0.26] | [0.04, 0.24] | [0.11, 0.33] | |

C_{14} | [0.10, 0.27] | [0.08, 0.26] | [0.07, 0.25] | [0.03, 0.18] | [0.03, 0.21] | [0.10, 0.31] | |

C_{21} | [0.04, 0.25] | [0.05, 0.26] | [0.05, 0.24] | [0.03, 0.22] | [0.02, 0.20] | [0.10, 0.33] | |

C_{22} | [0.05, 0.26] | [0.06, 0.25] | [0.04, 0.24] | [0.03, 0.22] | [0.06, 0.26] | [0.10, 0.32] | |

C_{23} | [0.06, 0.27] | [0.06, 0.25] | [0.06, 0.25] | [0.05, 0.24] | [0.06, 0.27] | [0.11, 0.34] | |

C_{24} | [0.04, 0.24] | [0.07, 0.26] | [0.06, 0.23] | [0.04, 0.22] | [0.08, 0.26] | … | [0.11, 0.32] |

C_{31} | [0.08, 0.27] | [0.05, 0.26] | [0.08, 0.25] | [0.06, 0.24] | [0.05, 0.26] | [0.08, 0.31] | |

C_{32} | [0.07, 0.24] | [0.04, 0.22] | [0.05, 0.21] | [0.03, 0.19] | [0.04, 0.22] | [0.08, 0.28] | |

C_{33} | [0.06, 0.29] | [0.06, 0.29] | [0.06, 0.28] | [0.06, 0.27] | [0.08, 0.28] | [0.09, 0.34] | |

C_{41} | [0.05, 0.22] | [0.05, 0.22] | [0.02, 0.19] | [0.03, 0.19] | [0.02, 0.21] | [0.07, 0.27] | |

C_{42} | [0.01, 0.15] | [0.01, 0.15] | [0.01, 0.13] | [0.02, 0.15] | [0.01, 0.14] | [0.04, 0.20] | |

C_{43} | [0.04, 0.20] | [0.05, 0.20] | [0.02, 0.18] | [0.03, 0.18] | [0.03, 0.19] | [0.08, 0.27] | |

C_{44} | [0.04, 0.21] | [0.05, 0.22] | [0.03, 0.20] | [0.03, 0.20] | [0.03, 0.21] | [0.03, 0.21] |

${\overline{\mathit{r}}}_{\mathit{i}}$ | ${\overline{\mathit{c}}}_{\mathit{i}}$ | ${\overline{\mathit{r}}}_{\mathit{i}}+{\overline{\mathit{c}}}_{\mathit{i}}$ | ${\overline{\mathit{r}}}_{\mathit{i}}-{\overline{\mathit{c}}}_{\mathit{i}}$ | ${\overline{\mathit{r}}}_{\mathit{i}}$ | ${\overline{\mathit{c}}}_{\mathit{i}}$ | ${\overline{\mathit{r}}}_{\mathit{i}}+{\overline{\mathit{c}}}_{\mathit{i}}$ | ${\overline{\mathit{r}}}_{\mathit{i}}-{\overline{\mathit{c}}}_{\mathit{i}}$ | ||
---|---|---|---|---|---|---|---|---|---|

C_{1} | [0.27, 1.07] | [0.21, 0.94] | [0.47, 2.01] | [−0.67, 0.86] | C_{11} | [1.10, 4.36] | [0.87, 3.67] | [1.97, 8.03] | [−2.57, 3.50] |

C_{12} | [1.15, 4.21] | [0.83, 3.65] | [1.98, 7.85] | [−2.50, 3.38] | |||||

C_{13} | [0.89, 3.92] | [0.74, 3.42] | [1.64, 7.33] | [−2.52, 3.17] | |||||

C_{14} | [0.90, 3.59] | [0.67, 3.30] | [1.57, 6.89] | [−2.40, 2.92] | |||||

C_{2} | [0.22, 1.01] | [0.18, 0.95] | [0.41, 1.96] | [−0.73, 0.82] | C_{21} | [0.79, 3.77] | [0.69, 3.52] | [1.48, 7.29] | [−2.73, 3.08] |

C_{22} | [0.83, 3.79] | [0.69, 3.58] | [1.52, 7.37] | [−2.74, 3.10] | |||||

C_{23} | [0.95, 3.94] | [0.64, 3.41] | [1.60, 7.36] | [−2.46, 3.30] | |||||

C_{24} | [0.84, 3.68] | [0.70, 3.71] | [1.54, 7.40] | [−2.87, 2.98] | |||||

C_{3} | [0.21, 0.99] | [0.15, 0.88] | [0.36, 1.87] | [−0.67, 0.83] | C_{31} | [0.82, 3.75] | [0.63, 3.39] | [1.45, 7.15] | [−2.57, 3.12] |

C_{32} | [0.69, 3.28] | [0.58, 3.32] | [1.27, 6.61] | [−2.63, 2.71] | |||||

C_{33} | [0.92, 4.15] | [0.51, 3.22] | [1.43, 7.38] | [−2.30, 3.64] | |||||

C_{4} | [0.13, 0.77] | [0.28, 1.05] | [0.41, 1.82] | [−0.93, 0.49] | C_{41} | [0.54, 3.15] | [1.22, 4.26] | [1.76, 7.41] | [−3.72, 1.93] |

C_{42} | [0.24, 2.24] | [0.41, 2.52] | [0.66, 4.76] | [−2.28, 1.82] | |||||

C_{43} | [0.56, 2.97] | [1.27, 4.46] | [1.83, 7.43] | [−3.90, 1.70] | |||||

C_{44} | [0.56, 3.17] | [1.34, 4.55] | [1.89, 7.72] | [−4.00, 1.83] |

${\mathit{r}}_{\mathit{i}}$ | ${\mathit{c}}_{\mathit{i}}$ | ${\mathit{r}}_{\mathit{i}}+{\mathit{c}}_{\mathit{i}}$ | ${\mathit{r}}_{\mathit{i}}-{\mathit{c}}_{\mathit{i}}$ | ${\mathit{r}}_{\mathit{i}}$ | ${\mathit{c}}_{\mathit{i}}$ | ${\mathit{r}}_{\mathit{i}}+{\mathit{c}}_{\mathit{i}}$ | ${\mathit{r}}_{\mathit{i}}-{\mathit{c}}_{\mathit{i}}$ | ||
---|---|---|---|---|---|---|---|---|---|

C_{1} | 0.67 | 0.57 | 1.24 | 0.09 | C_{11} | 2.73 | 2.27 | 5.00 | 0.47 |

C_{12} | 2.68 | 2.24 | 4.91 | 0.44 | |||||

C_{13} | 2.41 | 2.08 | 4.48 | 0.33 | |||||

C_{14} | 2.25 | 1.98 | 4.23 | 0.26 | |||||

C_{2} | 0.62 | 0.57 | 1.18 | 0.05 | C_{21} | 2.28 | 2.10 | 4.38 | 0.17 |

C_{22} | 2.31 | 2.13 | 4.44 | 0.18 | |||||

C_{23} | 2.45 | 2.03 | 4.48 | 0.42 | |||||

C_{24} | 2.26 | 2.21 | 4.47 | 0.06 | |||||

C_{3} | 0.60 | 0.52 | 1.12 | 0.08 | C_{31} | 2.29 | 2.01 | 4.30 | 0.28 |

C_{32} | 1.99 | 1.95 | 3.94 | 0.04 | |||||

C_{33} | 2.54 | 1.87 | 4.41 | 0.67 | |||||

C_{4} | 0.45 | 0.67 | 1.11 | −0.22 | C_{41} | 1.85 | 2.74 | 4.58 | −0.89 |

C_{42} | 1.24 | 1.47 | 2.71 | −0.23 | |||||

C_{43} | 1.77 | 2.87 | 4.63 | −1.10 | |||||

C_{44} | 1.86 | 2.94 | 4.80 | −1.08 |

C_{11} | C_{12} | C_{13} | C_{14} | C_{21} | … | C_{44} | |
---|---|---|---|---|---|---|---|

C_{11} | [0.128, 0.213] | [0.333, 0.275] | [0.289, 0.267] | [0.356, 0.281] | [0.255, 0.260] | [0.286, 0.255] | |

C_{12} | [0.285, 0.268] | [0.129, 0.215] | [0.298, 0.280] | [0.289, 0.274] | [0.279, 0.266] | [0.314, 0.261] | |

C_{13} | [0.290, 0.262] | [0.269, 0.258] | [0.099, 0.197] | [0.261, 0.260] | [0.283, 0.250] | [0.192, 0.243] | |

C_{14} | [0.297, 0.256] | [0.269, 0.252] | [0.314, 0.256] | [0.094, 0.184] | [0.183, 0.224] | [0.208, 0.241] | |

C_{21} | [0.252, 0.248] | [0.247, 0.246] | [0.249, 0.242] | [0.166, 0.229] | [0.123, 0.202] | [0.234, 0.248] | |

C_{22} | [0.235, 0.243] | [0.278, 0.255] | [0.247, 0.260] | [0.207, 0.256] | [0.272, 0.265] | [0.229, 0.248] | |

C_{23} | [0.263, 0.247] | [0.231, 0.237] | [0.240, 0.235] | [0.343, 0.249] | [0.230, 0.250] | [0.260, 0.242] | |

C_{24} | [0.250, 0.262] | [0.244, 0.262] | [0.265, 0.263] | [0.284, 0.266] | [0.375, 0.283] | … | [0.278, 0.262] |

C_{31} | [0.366, 0.340] | [0.315, 0.333] | [0.399, 0.346] | [0.379, 0.344] | [0.394, 0.347] | [0.361, 0.335] | |

C_{32} | [0.304, 0.329] | [0.337, 0.332] | [0.292, 0.334] | [0.326, 0.335] | [0.360, 0.335] | [0.351, 0.335] | |

C_{33} | [0.330, 0.331] | [0.348, 0.335] | [0.309, 0.320] | [0.295, 0.320] | [0.246, 0.319] | [0.288, 0.330] | |

C_{41} | [0.292, 0.269] | [0.282, 0.270] | [0.276, 0.272] | [0.291, 0.266] | [0.284, 0.272] | [0.329, 0.295] | |

C_{42} | [0.111, 0.168] | [0.095, 0.158] | [0.098, 0.158] | [0.118, 0.165] | [0.089, 0.155] | [0.094, 0.162] | |

C_{43} | [0.288, 0.277] | [0.308, 0.284] | [0.312, 0.283] | [0.288, 0.278] | [0.323, 0.286] | [0.417, 0.307] | |

C_{44} | [0.309, 0.286] | [0.315, 0.288] | [0.313, 0.286] | [0.302, 0.291] | [0.303, 0.288] | [0.161, 0.236] |

C_{11} | C_{12} | C_{13} | C_{14} | C_{21} | … | C_{44} | |
---|---|---|---|---|---|---|---|

C_{11} | [0.034, 0.051] | [0.088, 0.066] | [0.076, 0.064] | [0.094, 0.068] | [0.056, 0.063] | [0.070, 0.063] | |

C_{12} | [0.075, 0.065] | [0.034, 0.052] | [0.078, 0.067] | [0.076, 0.066] | [0.061, 0.064] | [0.076, 0.064] | |

C_{13} | [0.076, 0.063] | [0.071, 0.062] | [0.026, 0.047] | [0.069, 0.063] | [0.062, 0.060] | [0.047, 0.060] | |

C_{14} | [0.078, 0.062] | [0.071, 0.061] | [0.083, 0.062] | [0.025, 0.044] | [0.040, 0.054] | [0.051, 0.059] | |

C_{21} | [0.053, 0.061] | [0.052, 0.061] | [0.053, 0.060] | [0.035, 0.057] | [0.027, 0.049] | [0.048, 0.061] | |

C_{22} | [0.050, 0.060] | [0.059, 0.063] | [0.052, 0.064] | [0.044, 0.063] | [0.061, 0.065] | [0.046, 0.061] | |

C_{23} | [0.056, 0.061] | [0.049, 0.059] | [0.051, 0.058] | [0.072, 0.062] | [0.051, 0.061] | [0.053, 0.060] | |

C_{24} | [0.053, 0.065] | [0.051, 0.065] | [0.056, 0.065] | [0.060, 0.066] | [0.083, 0.069] | … | [0.057, 0.065] |

C_{31} | [0.067, 0.080] | [0.058, 0.078] | [0.074, 0.081] | [0.070, 0.081] | [0.070, 0.080] | [0.082, 0.080] | |

C_{32} | [0.056, 0.077] | [0.062, 0.078] | [0.054, 0.078] | [0.060, 0.079] | [0.064, 0.078] | [0.080, 0.080] | |

C_{33} | [0.061, 0.078] | [0.064, 0.079] | [0.057, 0.075] | [0.054, 0.075] | [0.044, 0.074] | [0.065, 0.079] | |

C_{41} | [0.099, 0.075] | [0.096, 0.075] | [0.094, 0.076] | [0.099, 0.074] | [0.108, 0.077] | [0.107, 0.079] | |

C_{42} | [0.038, 0.047] | [0.032, 0.044] | [0.034, 0.044] | [0.040, 0.046] | [0.034, 0.044] | [0.031, 0.043] | |

C_{43} | [0.098, 0.077] | [0.105, 0.079] | [0.106, 0.079] | [0.098, 0.077] | [0.123, 0.081] | [0.136, 0.082] | |

C_{44} | [0.105, 0.079] | [0.107, 0.080] | [0.107, 0.080] | [0.103, 0.081] | [0.115, 0.081] | [0.052, 0.063] |

Local Weight | De-Roughness | Local Weight | De-Roughness | Global Weight | De-Roughness | ||
---|---|---|---|---|---|---|---|

C_{1} | [0.054, 0.245] | 0.150 (2) | C_{11} | [0.283, 0.262] | 0.273 (1) | [0.015, 0.064] | 0.040 (7) |

C_{12} | [0.275, 0.260] | 0.267 (2) | [0.015, 0.064] | 0.039 (8) | |||

C_{13} | [0.228, 0.243] | 0.235 (3) | [0.012, 0.060] | 0.036 (12) | |||

C_{14} | [0.214, 0.235] | 0.224 (4) | [0.012, 0.058] | 0.035 (14) | |||

C_{2} | [0.047, 0.249] | 0.143 (3) | C_{21} | [0.247, 0.247] | 0.247 (3) | [0.012, 0.062] | 0.037 (11) |

C_{22} | [0.250, 0.251] | 0.250 (2) | [0.012, 0.062] | 0.037 (10) | |||

C_{23} | [0.245, 0.241] | 0.243 (4) | [0.012,0.060] | 0.036 (13) | |||

C_{24} | [0.258, 0.261] | 0.259 (1) | [0.012, 0.065] | 0.038 (9) | |||

C_{3} | [0.042, 0.231] | 0.137 (4) | C_{31} | [0.366, 0.342] | 0.354 (1) | [0.015, 0.079] | 0.047 (4) |

C_{32} | [0.338, 0.334] | 0.336 (2) | [0.014, 0.077] | 0.046 (5) | |||

C_{33} | [0.296, 0.324] | 0.310 (3) | [0.012, 0.075] | 0.044 (6) | |||

C_{4} | [0.073, 0.275] | 0.174 (1) | C_{41} | [0.283, 0.270] | 0.277 (3) | [0.021, 0.074] | 0.047 (3) |

C_{42} | [0.099, 0.160] | 0.129 (4) | [0.007, 0.044] | 0.026 (15) | |||

C_{43} | [0.300, 0.282] | 0.291 (2) | [0.022, 0.078] | 0.050 (2) | |||

C_{44} | [0.318, 0.288] | 0.303 (1) | [0.023,0.079] | 0.051 (1) |

No. 1 | No. 2 | No. 3 | No. 4 | No. 5 | No. 6 | No. 7 | No. 8 | No. 9 | No. 10 | |
---|---|---|---|---|---|---|---|---|---|---|

C_{11} | 75 | 71 | 90 | 80 | 75 | 75 | 80 | 90 | 80 | 85 |

C_{12} | 70 | 75 | 85 | 85 | 75 | 75 | 60 | 80 | 90 | 80 |

C_{13} | 70 | 84 | 80 | 80 | 75 | 85 | 80 | 80 | 90 | 90 |

C_{14} | 60 | 77 | 76 | 75 | 70 | 80 | 50 | 90 | 95 | 95 |

C_{21} | 65 | 71 | 71 | 85 | 83 | 85 | 80 | 90 | 85 | 85 |

C_{22} | 70 | 88 | 62 | 90 | 85 | 85 | 71 | 80 | 85 | 85 |

C_{23} | 70 | 72 | 77 | 75 | 70 | 60 | 70 | 80 | 90 | 80 |

C_{24} | 70 | 76 | 80 | 80 | 70 | 85 | 70 | 80 | 85 | 95 |

C_{31} | 75 | 66 | 87 | 70 | 70 | 75 | 60 | 80 | 80 | 80 |

C_{32} | 60 | 62 | 88 | 70 | 81 | 60 | 50 | 70 | 80 | 80 |

C_{33} | 70 | 69 | 77 | 70 | 70 | 60 | 60 | 70 | 85 | 88 |

C_{41} | 70 | 80 | 70 | 80 | 70 | 70 | 30 | 70 | 70 | 80 |

C_{42} | 70 | 61 | 64 | 80 | 80 | 70 | 30 | 80 | 85 | 85 |

C_{43} | 70 | 70 | 77 | 85 | 80 | 70 | 70 | 80 | 90 | 90 |

C_{44} | 75 | 80 | 80 | 80 | 70 | 70 | 60 | 80 | 90 | 90 |

Global Weight | Aspiration-Level | Rough Evaluation Scores | Relative Significance | |
---|---|---|---|---|

C_{11} | [0.015, 0.064] | [100, 100] | [76.054, 84.427] | 0.030 (10) |

C_{12} | [0.015, 0.064] | [100, 100] | [71.487, 83.015] | 0.029 (7) |

C_{13} | [0.012, 0.060] | [100, 100] | [77.298, 85.415] | 0.027 (4) |

C_{14} | [0.012, 0.058] | [100, 100] | [65.851, 86.906] | 0.026 (3) |

C_{21} | [0.012, 0.062] | [100, 100] | [74.431, 84.755] | 0.027 (5) |

C_{22} | [0.012, 0.062] | [100, 100] | [73.702, 85.560] | 0.028 (6) |

C_{23} | [0.012,0.060] | [100, 100] | [69.070, 80.022] | 0.025 (2) |

C_{24} | [0.012, 0.065] | [100, 100] | [74.070, 84.447] | 0.029 (8) |

C_{31} | [0.015, 0.079] | [100, 100] | [68.763 79.569] | 0.033 (13) |

C_{32} | [0.014, 0.077] | [100, 100] | [61.654, 78.396] | 0.031 (12) |

C_{33} | [0.012, 0.075] | [100, 100] | [66.126, 77.937] | 0.030 (9) |

C_{41} | [0.021, 0.074] | [100, 100] | [62.271, 74.900] | 0.031 (11) |

C_{42} | [0.007, 0.044] | [100, 100] | [58.679, 79.539] | 0.018 (1) |

C_{43} | [0.022, 0.078] | [100, 100] | [73.076, 83.480] | 0.036 (14) |

C_{44} | [0.023,0.079] | [100, 100] | [71.458, 83.187] | 0.037 (15) |

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Shao, Q.-G.; Liou, J.J.H.; Weng, S.-S.; Chuang, Y.-C.
A Rough Hybrid Multicriteria Decision-Making Model for Improving the Quality of a Research Information System. *Symmetry* **2019**, *11*, 1248.
https://doi.org/10.3390/sym11101248

**AMA Style**

Shao Q-G, Liou JJH, Weng S-S, Chuang Y-C.
A Rough Hybrid Multicriteria Decision-Making Model for Improving the Quality of a Research Information System. *Symmetry*. 2019; 11(10):1248.
https://doi.org/10.3390/sym11101248

**Chicago/Turabian Style**

Shao, Qi-Gan, James J. H. Liou, Sung-Shun Weng, and Yen-Ching Chuang.
2019. "A Rough Hybrid Multicriteria Decision-Making Model for Improving the Quality of a Research Information System" *Symmetry* 11, no. 10: 1248.
https://doi.org/10.3390/sym11101248