Improving the Sustainability of Open Government Data

: In the context of big data, opening government data to society for development and utilization would improve the transparency of government work and promote harmonious social development. Improving the sustainability of open government data (OGD) is an important and challenging issue, and has received the attention of multiple governments. The goals of this study are to construct a new evaluation model for the sustainability of OGD that explores the main inﬂuential factors and amends the priorities. We used a combination of the Decision-Making Trial and Evaluation Laboratory (DEMATEL) method and the Analytical Network Process (ANP) method known as DANP to construct an inﬂuential network relationship map (INRM) of the evaluation system and calculate the criterion weights. The INRM and weights can help decision makers understand the complex assessment system and set priorities for improving OGD. The results demonstrate that the improvement priorities of dimensions have the following order: impact (D4), use (D3), data (D2), and environment (D1). Economic, social, and political values are the three critical factors for improvement of the sustainability of OGD. Some implications for management are also provided.


Introduction
Governmental departments generate, acquire, and preserve large amounts of data resources in the process of fulfilling their administrative duties. These data are public resources for the whole society. Open government data (OGD) refers to government data that anyone or any organization can access, use, and share freely [1,2]. Continuously opening government data to society for development and utilization will help improve the transparency of government work and promote public participation in governance [3]. OGD is easy for the public to reuse and redevelop for any purpose, to generate new services, stimulate innovation, and provide new economic opportunities, thus creating social and economic value [4,5]. OGD can stimulate the public to participate in government events and affairs, so that political value can be created by improving the transparency and public service quality of the government departments [6]. Hence, OGD is one of the key paths for sustainable development [7].
The US government launched the open data platform data.gov in 2009, and, since then, other countries have also established web portals to promote the sustainability of OGD, which has been progressing rapidly all around the world. As of 2016, 106 of the 193 member states of the United opening data [4,5]. Sustainable OGD can contribute in many ways to the achievement of sustainable development goals, and can drive progress towards sustainable development [7]. Sustainable OGD is an important way to promote sustainable development. The ways to improve sustainability of OGD remain a very meaningful research issue. Thus, this study aims to reveal the influential factors and impact effects of sustainable OGD, as well as to propose strategies for improvement. In sum, the investigation aimed to: (1) Define the evaluating factors and establish the evaluation model for the sustainability of OGD.
(2) Investigate the influence of degrees between factors.
(3) Put forward suggestions to improve the sustainability of OGD.
The improvement of sustainable OGD is a multiple-attribute decision-making problem. Relevant variables usually have certain degrees of correlation, especially the interaction between important variables, such as dependence and feedback effects. The Decision-Making Trial and Evaluation Laboratory (DEMATEL) method combined with the Analytic Network Process (ANP) method can solve the problem by constructing an interaction between the evaluation dimensions and the criteria of sustainable OGD. This method has been widely used to examine emergency strategies, transportation, green buildings, network reputation, service quality, and other applications [31][32][33][34][35][36][37]. However, it is rarely used on OGD. The DANP is a new method to construct evaluation models of sustainability of OGD. Based on the DEMATEL and ANP methods, the relationships and weights of each influential factor can be calculated. The derived influential network relationship map (INRM) is used to represent the influence of relationships between dimensions and criteria in sustainable OGD. This will help government administrators understand the influencing factors in complex relationships and propose suggestions to improve the sustainability of OGD.
The remainder of this paper is organized as follows. Section 2 reviews the relevant literature to form the key factors for sustainability of OGD. Section 3 introduces the DANP method that was used to build the INRM and find the weights of the criteria. The data analysis and results are presented in Section 4. A discussion and the implications are discussed in Section 5, and Section 6 presents the conclusions.

Literature Review on Establishment Indicator Framework
This study adopts the OGD assessment draft framework proposed by ODI and New York University [17] and integrates OGD ecosystem theory [38]. We evaluate the sustainability of OGD from environment, data, use, and impact dimensions.

Definition of Sustainability of OGD
Sustainability refers to the ability to sustain or support objective things permanently or indefinitely. From a dynamic point of view, it means that healthy existence and development of the system can be continuously or indefinitely maintain or supported [38]. Sustainability was first proposed by ecologists, and this is now called ecological sustainability [39].
An ecosystem can be understood as "a system of people, practices, values and technologies in a particular local environment" [40]. There are various ecosystem analogies, such as the business ecosystem, digital ecosystem, and open government ecosystem [38,[40][41][42]. The concept of a data ecosystem has seen growing use in research on data-driven government. In open data ecology, there are four archetypal sustainable value generating mechanisms. They are information transparency, collective impact, data-driven efficiency, and data-driven innovation. The sustainable value generation is influenced by: the incentives provided, the level of technical and legal openness of data, the maturity of resource (data) governance, the general data-related capabilities in society, and the technological maturity and prevalence [43]. In general, open data ecosystems can be seen as being composed of a number of elements, such as participants (actors), data resources, design, and interdependencies [41]. We also think of OGD as an ecosystem. The sustainability of OGD is a new area of research that links the concepts of sustainability and OGD. Generally, sustainable OGD emphasizes the realization of long-term continuous OGD and the reuse of data to generate political, economic, and social value [12,41]. Opening and using data encounters numerous impediments, such as availability and access, findability, usability, understandability, quality, metadata, and so on [44]. The OGD ecosystem includes stakeholders, processes of data, and benefits associated [6,38]. The components of the OGD ecosystem mainly include stakeholders, the process of utilization, benefit, and external environment [42]. An OGD ecosystem is defined as one driven by the realization of value [40]. The environment is the infrastructure of OGD ecosystem. The data is the basic and most important resource of the OGD ecosystem [6,38]. Impacts on politics, economy, and society will be created by using data resources [45]. The impacts are the sustainable value that sustainable OGD will generate [40,43]. This study adopts the OGD ecosystem and the OGD assessment draft framework proposed by ODI and New York University [16]. The research framework includes the environment, data, use, and impact dimensions with 18 criteria, which are discussed below.

Environment Dimension
The environment dimension refers to the background conditions for providing sustainable OGD. It is the premise for OGD implementation and provides protection and a technical foundation, including laws and regulations, political will and leadership, and technical capacity [28,37]. Laws and regulatory factors provide an implementation framework for sustainable OGD, to ensure that government data are continuously open for a long time, to improve the transparency of government information, and to stimulate the use and value creation of OGD [25,28].
OGD is open data that was generated by government management, and the open data is stored in various parts of the government [2,46]. Political will and leadership will strengthen collaboration among government departments to build a unified data set that can be sustainably opened to the public [45]. If leaders do not pay attention, sustainable OGD cannot be effectively developed [47]. As a basic policy to improve government transparency, sustainable OGD requires time and money to implement. OGD requires strong government support and a strong political will to mobilize government data and other related resources [48].
Public supervision, consultation, and participation in OGD will promote government transparency and social equity, as well as trust in society and social connectedness [3]. The more active the public is in OGD, the more sustainable OGD becomes [10]. Enthusiasm among the public for the access and reuse of open data is a prerequisite for OGD to create value [29].
Technical capacity provides technical support for improving the quality of sustainable OGD and promoting its effective use. Linked data technology realizes the association and integration between different datasets, data, and OGD platforms, as well as the semantic expression and integration of open data [49][50][51]. Ontology enhances the discovery capability of unstructured data and the reusability of mapping information [52]. Thus, this study uses laws and regulations, political will and leadership, general public, and technical capacity as indicators to assess the sustainability of OGD under the environmental dimension.

Data Quality Dimension
Data quality has significant direct effects on OGD adoption and sustainable value generation [43,53]. Data quality is a measure of the data content and reusability provided by sustainable OGD, which is the basis for the public reuse of data [22]. Moreover, data quality is critical in the implementation of sustainable value generation [40,43]. OGD should provide complete, accurate, timely, and easy-to-use data to improve the quality of OGD services and increase public satisfaction [17,54]. Open data may conflict with privacy, security, and transparency. When opening government data, the data should be densified and access controlled [55]. Completeness refers to the type of OGD and the richness of datasets. The richer the open data, the more value can be generated by reusing data, while protecting national security, trade secrets, and personal privacy [18,20,22,23,56,57]. The value of big data in social domain can only be created if the outcomes are correctly interpreted [58].
Accuracy is a measure of the correctness of data. Only suitably accurate data can be used to make decisions for the public. Correct data can help the public to make scientific decisions [20,22,23,54,57]. Timeliness refers to whether OGD provides timely data updates, whether the data is up-to-date, the frequency and cycle of data updates, and when the last data update has been made. Data timeliness is one of the important indicators for data quality evaluation [22,23,54,56,57]. Understandability means that keywords, titles, and descriptions in OGD are easy to understand not only by experts, but also by the general public [18,20,22,53]. The public can only use the data after understanding OGD.
Accessibility means that the public has equal access to open data without having to be questioned about the desired purpose, and the use of the data is not restricted [15,56]. Data usability refers to the basic data format provided to facilitate public access, downloading, and reuse [18,20]. Data usability can be divided into three levels: non-machine format (such as PDF), automatically processed structured formats (such as XLS), and inclusion elements, structured format descriptions, and semantic formats of data (such as XML and RDF) [57]. Non-machine-formatted data could potentially hamper the efficient reuse of open data [22]. This study uses completeness, accuracy, timeliness, understandability, accessibility, and usability as indicators to assess the openness of government data under the data quality dimension.

Use Dimension
The use dimension evaluates the convenience of public access to an OGD platform. Simple and convenient operation results in a good experience for the public. The interface refers to the user's experience when visiting an OGD website, and is evaluated based on the design of the platform. If the interface experience is good, the public's satisfaction with OGD will increase [18,53]. The data index is used to classify the OGD platform dataset by subject, institution, data format, or other qualities, as well as keyword search and advanced search operation, whether the evaluation data classification directory is reasonable, the convenience of retrieval [18], and whether the online help function will be user-friendly [20].
Data acquisition refers to whether the public requires complicated operations, such as registration or filling in personal information. The public's use of OGD is affected by whether the platform's operation steps are simple and whether obtaining the data is complicated [18]. Data display refers to whether the OGD is displayed to users in various forms, provides personalized previews and information displays, and provides multi-language support, which all affect the reuse of OGD [20]. Interactive communication refers to whether the OGD website allows providers to collect dataset ratings and comments. The OGD platform should provide forums and other interactive communication functions to encourage the public to use OGD and improve service satisfaction [18]. Thus, this study uses the interface, index, acquisition, presentation, and instructiveness factors as indicators of the openness of government data in the use dimension.

Impact Dimension
Impact refers to the effects and value generated after the data are opened and utilized. This is the goal of OGD [24,44] and promotes sustainable OGD [12]. OGD generates benefits in social, political, and economic aspects [20]. Social impact is generated through public participation and their use of OGD. The OGD platform should allow the public to query relevant information quickly, simplify government workflow, improve work efficiency, and improve quality of life [20,27]. OGD can change the public from passively communicating to actively participating in the model by letting them supervise, negotiate, and participate in governance [3,59]. The government is the provider of OGD. OGD can improve the transparency of public administration and its procedures, reduce the abuse of public power, and help fight corruption. OGD stimulates and improves the government's serviceability and improves public satisfaction [27,59]. OGD could advance the democratic process, promote sustainable development, and thus could have a political impact [20]. An enterprise information system could directly access the OGD platform and process the data to create new products, provide new services, and create new jobs, economic growth, and enterprise innovation to generate economic value [60]. For example, using the government's open traffic data, enterprises could develop a "mobile bus" application, and enterprises could gain economic benefits. This study uses social impact, political influence, and economic impact as indicators to assess the openness of government data under the impact dimension. Table 1 shows all of the discussed indicators from the relevant literature used to assess sustainable OGD. Whether the operation to obtain data is complex. [18,20] Data is presented in many forms. [18,20] Interactive (C 35 ) OGD platform provides interaction with users. [20,52] Impact (D 4 )

Social (C 41 )
The impact of OGD on society. [20,27] The impact of OGD on politics.

Methodology
This study established an evaluation model of sustainable OGD by introducing the DANP model, which integrates DEMATEL with ANP. The DANP method has been applied to construct evaluation models in education, marketing, traffic, and other fields [34][35][36][37]61]. It is the new method for building an evaluation model for the sustainability of OGD. We calculated the influential weights (IWs) of DANP to find the causal relationships associated with each dimension and criterion. We used the results to derive the INRM. Both the weights and the INRM could help Chinese government administrators to improve their priorities for improving OGD. The details of this have been described in the literature [34,[61][62][63][64][65][66].

DEMATEL Method
The Geneva Battelle Research Center [67,68] first proposed DEMATEL to construct structural analysis models to solve specific societal problems. DEMATEL is used to identify the correlation between factors. An INRM is constructed to clarify the sources of effect indexes, and can be used to improve system performance [69][70][71]. The details are as follows.
Step 1: Build the average direct influence relation matrix A = [a ij ] m×n . Questionnaires were used to collect data, and indicators were compared in pairs. Experts were invited to select the degree of influence, as well as positive and negative directions according to the scale. The results were recorded in the form of a matrix. Scales of 0-4 were used to represent the range from "absolutely no influence" (0) to "very high influence" (4). The number of experts is H, and the number of factors (the criteria) is n. The average scores of the H experts were a ij , x ij is the degree of direct influence of the criterion i on the criterion j, and x k ij is the score of the expert k. The direct influence-relation average matrix A is obtained by calculating the average of the same factor in N matrices of the respondents. A is expressed in Equation (1), where n is the number of criteria.
Step 2: Formulate the normalizing direct influence-relation average matrix X. We obtained the normalized initial direct influence-relation matrix X by normalizing the average matrix A. X is the product of s and A, and is obtained from Equations (2) and (3).
Step 3: Construct the total influence matrix T. In Equation (4), the total influence relation matrix T is an n × n matrix formed as T = t ij n×n , and I is an n × n unit matrix. Element t ij is the degree of direct influence of the criterion i on criterion j.
Step 4: Obtain the INRM using the total matrix T. Using Equations (5) and (6), we can calculate each column sum (c j ) and row sum (r i ) from matrix T.
The c j is the sum of the direct and indirect influences of factor j on the other factors in matrix T, which is called the degree of influence. r i is the sum of direct and indirect influences of other factors on factor i in matrix T, which is called the degree of effect. When i = j, r i + c i reflects the intensity affected given by and received from factor i; that is, the centrality of index i. r i -c i is called the degree of causality. Clearly, if r i -c i is greater than zero, the criterion i is an influencing element, but if r i -c i is less than zero, the criterion i is an effective element [33]. Consequently, based on r i + c i and r i -c i , we can draw the INRM.
Step 5: Derive the total influence relation matrices T C , based on criteria, and T D , based on dimensions. The matrix T can be transformed into T C (as shown in Equation (7)) based on the criteria and T D (as shown in Equation (8)) based on the dimensions. n is the number of dimensions, and m i is the number of criteria in dimension i.

Obtaining the Influence Weights through the DANP
In 1996, Satty proposed ANP based on AHP. The ANP technique is used to build un-weighted and weighted super-matrices that consider the mutual influence of various factors or adjacent layers. Nevertheless, there are some weaknesses in the original ANP technique. First, we must assume the relationship structure of the evaluation system before using ANP. Second, the ANP questionnaire is difficult to understand, due to its complexity [72] and the time-consuming process of pairwise comparison. Thus, consistent results are difficult to obtain. Third, the hypothesis that every dimension has the same weight seems unreasonable because of conflicts of influence among the dimensions, or because the clusters are different in the real world [73]. Accordingly, the DEMATEL-based ANP (DANP) was proposed by Tzeng et al. [60,61,74] to solve these problems. The DANP method is used to solve real-world problems with feedback and interaction between criteria or dimensions, and to determine the affecting weights [61][62][63]75,76]. The DANP technique is summarized as follows.
Step 6: Construct the unweighted super-matrix W. The matrix T ω C can be obtained by normalizing matrix T C , which is shown in Equation (7). For example, T pq C , which is a submatrix of T C , can be normalized into T ωpq c , as shown in Equations (9) and (10). The matrix T ωpq c can be transposed to an unweighted matrix super-matrix W pq , as shown in Equation (12).
For example, T pq C , which is a submatrix of T C , can be normalized into T ωpq C , as shown in Equation (9) and Equation (10).
The matrix pq C  T can be transposed to unweighted matrix super-matrix Wpq , as shown in Equation (12).
( ) Step 7: Calculate the weighted super-matrix . Referring to step 5, we can calculate the total influence-relation matrix of the dimension's matrix, TD. Matrix can be obtained by The matrix T ωpq c can be transposed to unweighted matrix super-matrix W pq , as shown in Equation (12).
Step 7: Calculate the weighted super-matrix W δ . Referring to step 5, we can calculate the total influence-relation matrix of the dimension's matrix, T D . Matrix T δ D can be obtained by dividing the matrix T D by δ i = n j=1 t ij D , i = 1, 2, . . . , n, as shown in Equations (13) and (14). The weighted super-matrix W δ can be obtained from the normalized total influence-relation matrix T δ D and the un-weighted super-matrix W pq , as shown in Equation (15). The Step 7: Calculate the weighted super-matrix . Referring to step 5, we can calculate the total influence-relation matrix of the dimension's matrix, TD. Matrix can be obtained by dividing the matrix TD by = ∑ =1 , i = 1, 2, ..., n, as shown in Equations (13) and (14). The weighted super-matrix can be obtained from the normalized total influence-relation matrix and the un-weighted super-matrix Wpq, as shown in Equation (15).
The weighted super-matrix W δ can be gained by the normalized total influence-relation matrix T δ D and the un-weighted super-matrix W pq , as described in Equation (15).
Step 8: Limit weighted super-matrix W w . If the weighted super-matrix is multiplied by itself many times, we can obtain the limit super-matrix, which converges and becomes stable. Therefore, we can find the IWs (also called the DANP weights) by using Equation (16), where λ is the number of multiplication iterations.

Data Analysis and Results
Data were collected from 10 experts to obtain the average matrix (Table A1 of Appendix A). The details of the questionnaire for the data collection are illustrated in Appendix B. The DEMATEL method was then used to calculate the influential relationships among dimensions and criteria based on pairwise comparisons (Table A1), and the INRM was constructed. Finally, the influential weights of the dimensions and criteria were determined by DANP.

Descriptive Analysis of Data Collection
We collected data from 10 experts who have more than 10 years of experience in the OGD industry. Four of the experts were government officials engaged in OGD management, three were scholars engaged in OGD-related research, and three were senior managers who use OGD for application-development enterprises. We collected all standard data through an expert questionnaire in July 2018. Every expert took 40-60 min to complete the survey. The average matrix A was calculated from the means of all experts' responses to the questionnaires, and is presented in Table A1 in Appendix A. The confidence level of the 10 questionnaires was 95.3% (a little more than 95%), and consistency gaps were 4.7% (little smaller than 5%). The results show that there is a good consistency, and the 10 questionnaires can reflect most real situations.

Assembling the Influence Network via the DEMATEL Method
Using Equations (2) and (3), we can obtain the normalized directed-relation matrix. Consequently, the total influence matrix T can be calculated with Equation (4). Table A2 in Appendix A presents the equations, which show that all of the criteria of the relationships have complex interrelationships. We can calculate the matrix of overall influence dimensions by averaging the influence matrix T within each dimension. Table A3 in Appendix A shows the detailed steps. Table A4 indicates the sum of influences given by and received from the criteria and dimensions. We obtained the sum using Equations (5) and (6) in step 4. Table A4 shows that "environment" (D 1 ) (r i -c i ) had the largest positive value (0.234), which indicates the most influential dimension in the OGD evaluation system. Furthermore, the environment has a deep influence on the other dimensions. However, "impact" (D 4 ) (r i -c i ) had the smallest negative value (−0.306), which indicates it is the most easily influenced by other dimensions. Accordingly, "environment" should be considered a crucial factor in OGD improvement, and "impact" should be considered as the affected dimension. In addition, "data" (D 2 ) (r i + c i ) had the highest positive value (1.129). "Data" should be considered the most interactive relationship with other dimensions, and had the most interactive dimension from the perspective of experts.
It is well acknowledged that the environment of laws, policies, political will, and leadership influence the completeness of opening data, the usability of data, and other factors [29]. By reusing government data, stakeholders create new value that is impacted by data attributes, such as completeness, accuracy, timeliness, understandability, accessibility, and usability of data [58]. Therefore, data affects the other dimensions, such as use and impact, and is also affected by the environment. In contrast, "environment" (D 1 ) (r i + c i ) had the lowest relation (0.867) to other dimensions, meaning that it has the smallest total degree of influence among the dimensions. From Table A4, the INRM can be drawn by illustrating the influence network relationship of the four dimensions and 18 criteria, as shown in Figure 1.
The major influence route between dimensions can also be formed, as shown in Figure 2, where the primary cause is "environment" (D 1 ), and "impact" (D 4 ) is located in the center of the OGD sustainability evaluation system. This means that "environment" (D 1 ) is the most important influential factor. Therefore, the environment is the basis of OGD that provides useful conditions. To improve the sustainability of OGD, the government should strengthen environmental development. Furthermore, "impact" (D 4 ) was affected by other dimensions, which means that it is an important element. Using the DANP method, we can calculate the IWs shown Table 2. "Impact" (D 4 ) and "use" (D 3 ) are the two most important dimensions, and the cumulative total proportion is 61.5% in the OGD sustainability evaluation system. Therefore, the government should consider the usability of the OGD platform, and make it easy to use for public operation and data reuse. Improving the impact of OGD will create social value, political value, and economic value, and will promote sustainable OGD.  Figure 1 shows that the factors that affect other indicators in each dimension criterion are "political will and leadership" (C12), "understandability" (C24), "presentation" (C34), and "political" (C42). Notably, "political will and leadership" (C12) has the highest priority, because OGD requires strong government political will and leadership [8,35]. In the "environment" (D1) dimension, "political will and leadership" (C12) influences "laws and policy" (C11), "social public" (C13), and "technical capacity" (C14), while "general public" (C13) is affected by other criteria. Thus, to improve the sustainability of OGD, government administrators must have a  Figure 1 shows that the factors that affect other indicators in each dimension criterion are "political will and leadership" (C 12 ), "understandability" (C 24 ), "presentation" (C 34 ), and "political" (C 42 ). Notably, "political will and leadership" (C 12 ) has the highest priority, because OGD requires strong government political will and leadership [8,35]. In the "environment" (D 1 ) dimension, "political will and leadership" (C 12 ) influences "laws and policy" (C 11 ), "social public" (C 13 ), and "technical capacity" (C 14 ), while "general public" (C 13 ) is affected by other criteria. Thus, to improve the sustainability of OGD, government administrators must have a strong will and political leadership.
Moreover, "understandability" (C 24 ) also deserves further attention, because users can only utilize data if they understand it. Therefore, the data description should be simple and easy for the public to understand. Government offices should also consider "presentation" (C 34 ) because the data format is very important for the reuse of data. The OGD platform should provide a machine-readable format such as XLS, CSV, JSON, or XML. Citizens and organizations that access the data interface could develop applications to create value and to achieve the goals of OGD [3]. Furthermore, additional attention should be given to the "political" factor (C 42 ). Political values, such as government transparency and public participation, are conducive to creating social and economic value.
The IWs between dimensions were calculated by DANP to evaluate OGD sustainability (Table 2), and the dimensions have the following ranking in terms of priority for making improvements: "impact" (D 4 ), "use" (D 3 ), "data" (D 2 ), and "environment" (D 1 ). For government officials, the result is clear and simple, and provides improved prioritization for many complex criteria. "Economic" (C 43 ), "social" (C 41 ), and "political" (C 42 ) are the three most significant criteria, with weights of 0.123, 0.1122, and 0.1077, respectively. These indicators constitute the "impact" (D 4 ) dimension. The format and timeliness of OGD are the basis of data utilization. The government should provide a variety of machine-readable data formats and update the data promptly to eliminate "zombie" data. Citizens and organizations should be able to reuse the open data easily to create economic value by downloading or directly importing it to their computer systems. The government should open as much data as possible in all areas to improve government transparency and generate political value.
Furthermore, the following three criteria are important: "interactive" (C 35 ), "general public" (C 13 ), and "interface" (C 31 )", which have weights of 0.0676, 0.0666, and 0.0664, respectively. Hence, government officials should pay attention to the interactive functions and improve the user-friendliness of the OGD platform's interface operations. Strengthening personal privacy protection in the process of OGD could enhance users' trust and increase the degree of public support and participation in OGD. The three indicators with the weakest weights were "political will and leadership" (C 12 ), "laws and policy" (C 11 ), and "technical capacity" (C 14 ), which had weights of 0.0218, 0.0282, and 0.0299, respectively. Thus, these three indicators play relatively small roles in the OGD sustainability evaluation system.
For long-term enhancement, the government should play a leading role in the continuous opening of government data, which requires strong political will and leadership in the government. Therefore, the government should pay full attention to OGD, actively promote it, and provide corresponding policy and technical support. This study shows that social, political, and economic value are important indicators for the evaluation of OGD sustainability. Therefore, government officers should promote the value generated by data.

Conclusions
This study used 18 criteria to establish an evaluation system for OGD sustainability. The results indicated that "environment" (D 1 ) is the main factor that influences other dimensions, and "impact" (D 4 ) is an important result. The three most significant criteria were the "economic" (C 43 )," "social" (C 41 )," and "political" (C 42 ) criteria, which had about 34% of the total evaluation weight. A new mixed model was constructed as an evaluation system of OGD sustainability by integrating the DEMATEL and DANP methods. We identified complex relationships among criteria dimensions, and the INRM was constructed using the DEMATEL method. The INRM can reflect the causal relationships between the dimensions and criteria. The IWs of the dimensions and criteria can be calculated by applying the DANP method. Some limitations remain that require further study. First, this study only established an evaluation system for OGD sustainability, but the model was not applied to evaluate an OGD project. Therefore, in the future, to validate the model and improve the sustainability of OGD, we will apply this system to evaluate existing cases of OGD. Second, the assessment model included environment, data, use, and impact dimensions. The other possible dimensions of sustainability of OGD were not considered, such as data security or privacy protection. The other dimensions could be included to improve the evaluation model in the future work. Third, to help government managers identify the key factors and priorities for improving the sustainability of OGD, we calculated the weights of dimensions and criteria using the DANP method. Future studies could use additional multi-criteria approaches, such as data envelopment analysis (DEA), to estimate the relative IWs for the sustainability of OGD. The results of these studies could then be compared with those presented here.  Acknowledgments: The authors are extremely grateful for the Sustainability Journal editorial team's valuable comments on improving the quality of this article.

Conflicts of Interest:
The authors declare no conflict of interest.

Appendix A. Results in Detail
We used the DEMATEL-based ANP (DANP) method to establish the evaluation structure for the sustainability of OGD and analyze the relationships and degree of influence of the four dimensions and 18 criteria. We collected questionnaires from 10 experts and obtained an 18 × 18 average initial direct influence matrix A to form the average responses, as shown in Table A1. Tables A2 and A3 show the degree of influence among the 18 criteria and the influential relations among the four dimensions. Table A4 presents the sum of influences received or given from four dimensions and criteria within their dimension. The dimension of "environment" (D2) had the largest net influence (ri -ci) among all dimensions, so it is the main influencing dimension in the OGD evaluation system. The weights of criteria were calculated according to the specific steps of DANP. Table A5 presents the unweighted super-matrix, which was obtained by transposing the normalized influence matrix Tc, based on Equations (9)- (12). The weighted super-matrix W can be obtained using Equations (13)-(15), as shown in Table A6.

Environment
Legal and policy Related laws and policy on OGD.
Political will and leadership Willingness and leadership of government agencies.

Social public
The degree of public support and participation in OGD.
Technical Capacity Technological advancement in OGD.

Completeness
The number of OGD sets.

Accuracy
The data is accurate.

Timeliness
Timely update of data.

Understandability
The data description is easy for the public to understand.

Accessibility
The data can be accessible to anyone for any purpose.

Usable
Open data structure for easy to reuse.

Use
Interface User experience of operating OGD platform.

Index
Information systems or search engines.

Acquisition
Whether the operation to obtain data is complex.

Presentation
Data is presented in many forms.
Interactive OGD platform provides interaction with users.

Social
The impact of OGD on society.

Political
The impact of OGD on political.

Economic
The impact of OGD on economic.

III. Instructions
Indicator description: 0 = no influence; 1 = low influence; 5 = middle influence; 3 = high influence; 4 = extremely high influence. For example, the degree of influence of A on B is extremely high, so fill in the position 4.

III. Instructions
Indicator description：0 = no influence；1 = low influence；5 = middle influence；3 = high influence；4 = extremely high influence. For example, the degree of influence of A on B is extremely high, so fill in the position 4. 1. The influence of Legal and policy on Political will and leadership is (low influence), so fill in 1 in the box 2. The influence of Political will and leadership on Legal and policy is (extremely high), so fill in 4 in the box Please note that the influence relationship may not be equal. Examples:

IV. Pairwise comparisons between criteria
1. The influence of Legal and policy on Political will and leadership is (low influence), so fill in 1 in the box 2. The influence of Political will and leadership on Legal and policy is (extremely high), so fill in 4 in the box