# Empirical Research on the Evaluation Model and Method of Sustainability of the Open Source Ecosystem

^{1}

^{2}

^{3}

^{4}

^{5}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Related Work

## 3. Construction of the Evaluation Indicators System

#### 3.1. Definition of Sustainability

#### 3.2. Evaluation Indicators System for the Sustainability of OSE

#### 3.2.1. Openness

#### 3.2.2. Stability

#### 3.2.3. Activity

#### 3.2.4. Extensibility

#### 3.3. Sustainability Measurement Method

#### 3.3.1. Weight Calculation Method Based on Information Contribution Value

_{ij})

_{m}

_{×n}, and the normalized matrix is X

^{′}= (x

_{ij}

^{′})

_{m}

_{×n.}The specific formula is:

_{ij}

^{′}= (x

_{ij}− min x

_{j})/(max x

_{j}− min x

_{j}),

_{ij}, x

_{ij}

^{′}respectively represent the original and standardized value of the j-th sample of the i-th indicator; max x

_{j}, min x

_{j}respectively represent the standardized maximum value and standardized minimum value of the j-th sample.

_{i}is between [0,1].

_{ij}is 0. The value of p

_{ij}depends on the value of x

_{ij}

^{′}, so we need to judge x

_{ij}

^{′}. There are two cases:

- When x
_{ij}^{′}> 0 of the original data x_{ij}is normalized, p_{ij}> 0, so setting $\epsilon $ = 0; - When x
_{ij}^{′}= 0 of the original data x_{ij}is normalized, p_{ij}= 0, the value of lnp_{ij}cannot be calculated, so $\epsilon $ = 0.1 is set, and the purpose is to eliminate the case of p_{ij}= 0.$$\{\begin{array}{cc}{p}_{ij}=\frac{{x}^{\prime}{}_{ij}+\epsilon}{{\displaystyle \sum _{j=1}^{n}({x}^{\prime}{}_{ij}+\epsilon )}},& {x}^{\prime}{}_{ij}=0\\ {p}_{ij}=\frac{{x}^{\prime}{}_{ij}}{{\displaystyle \sum _{j=1}^{n}{x}^{\prime}{}_{ij}}},& {x}^{\prime}{}_{ij}>0\end{array}.$$

_{i}of the i-th indicator can be calculated according to the following formula:

_{k}(k = 1, 2...). Thus, the weight of the indicator of the corresponding guideline level is

#### 3.3.2. Sustainability Assessment Model

_{i}—the weight of the i-th guideline level; w

_{ij}—the weight of the j-th evaluation level indicator in the i-th guideline level; t—the total number of indicators in the guideline level; r—the number of indicators of evaluation level contained in the i-th guideline level. The larger the F value, the better the sustainable development of the open source ecosystem.

## 4. SO Introduction

#### 4.1. User Activities in SO

#### 4.2. Data Description

#### 4.3. Evaluation Indicators System for Sustainability of the SO Ecosystem

## 5. Sustainability Analysis of the SO Ecosystem

#### 5.1. Evaluation Indicators Analysis

#### 5.1.1. Openness Analysis

^{(i)}is the number of new users in the i-th year.

^{(i)}is the number of questions with answers in the i-th year.

_{1}is the weight of the number of new users and β

_{1}is the weight of the number of questions with answers. The α

_{1}, β

_{1}values are calculated according to the Equations (2)–(4) proposed in Section 3.3.1.

#### 5.1.2. Stability Analysis

^{(i)}is the number of stable users in the i-th year.

#### 5.1.3. Activity Analysis

_{2}is the weight of the response rate of questions and β

_{2}is the weight of the average number of comments. The α

_{2}, β

_{2}values are calculated according to Equations (2)–(4) proposed in Section 3.3.1.

#### 5.1.4. Extensibility Analysis

^{(i)}is the set of the top 10 languages of SO posts in the i-th year, and G

^{(i)}is the set of the top 10 languages of GitHub project in the i-th year.

^{(i)}is the number of questions in an emerging language in the i-th year.

_{3}is the weight of the popularity and β

_{3}is the weight of the growth force. The α

_{3}, β

_{3}values were calculated according to Equations (2)–(4) proposed in Section 3.3.1.

#### 5.2. SO Sustainability Measurement Results and Analysis

#### 5.2.1. Calculate the Weight of the Indicator

#### 5.2.2. Results of Comprehensive Assessment and Analysis

- (1)
- the openness index showed an overall upward trend from 2010 to 2016, increasing from 0.0081 in 2011 to 0.791 in 2016, indicating that the openness of the SO ecosystem was in a good state from 2010 to 2016.
- (2)
- The overall curve of the stability index was at its peak during the period from 2010 to 2016. The stability index rose from 0.0013 in 2010 to 0.0132 in 2013 with a sharp peak in 2013 and decreased to 0.0101 in the following three years but with a smaller decrease, meaning that the stability of the ecosystem maintained its level of development with minor fluctuations.
- (3)
- The overall activity index shows a downward trend, indicating that the current active status of the SO ecosystem has deteriorated, and the interaction of the user should be strengthened. From 2010 to 2015, the activity index decreased from 0.0646 to 0.0347; then, from 2015 to 2016, the activity index increased from 0.0347 to 0.0372.
- (4)
- The extensibility index of the SO ecosystem increased from 2010 to 2016 as a whole, except for 2013. The expansibility index increased year by year, and the level of development increased rapidly. From 2010 to 2012, the extensibility index increased from 0.0084 to 0.0419. From 2013 to 2016, the activity index increased from 0.0407 to 0.0855. The increase of the extensibility index was conducive to the improvement of the SO ecosystem’s sustainable development.

#### 5.3. Prediction Results and Analysis

- Data preprocessing: The original array x
^{(0)}= (x^{(0)}(1), x^{(0)}(2), …, x^{(0)}(n)) is accumulated and a new sequence x^{(1)}= (x^{(1)}(1), x^{(1)}(2), …, x^{(1)}(n)) is obtained, where each data in x^{(1)}(t) represents the accumulation of the corresponding previous data.$${x}^{(1)}(t)={\displaystyle \sum _{k=1}^{t}{x}^{(0)}}(k),t=1,2,\dots ,n.$$ - Establishing first order linear differential:$$\frac{{dx}^{(1)}}{dt}+a{x}^{(1)}=\mathrm{u},$$
- Average-generating arithmetic operators: The mean of the series x
^{(1)}, which generates B and the constant vector Y_{N}:$$B=\left[\begin{array}{c}0.5({x}^{(1)}(1)+{x}^{(1)}(2))\\ 0.5({x}^{(1)}(2)+{x}^{(1)}(3))\\ \dots \\ 0.5({x}^{(1)}(n-1)+{x}^{(1)}(n))\end{array}\begin{array}{c}1\\ 1\\ \dots \\ 1\end{array}\right].$$$${Y}_{N}={({x}^{(0)}(2),{x}^{(0)}(3),\dots ,{x}^{(0)}(n))}^{T}.$$ - Calculating a, u: Calculated using the least square method$$\widehat{a}={{(\mathrm{B}}^{\mathrm{T}}\mathrm{B})}^{-1}{B}^{T}{Y}_{N}={[a,u]}^{T}.$$
- Time response model: The grey parameter gray parameter $\widehat{a}$ is substituted into $\frac{{dx}^{(1)}}{dt}+a{x}^{(1)}=\mathrm{u}$, and the time response model is obtained.$${\widehat{x}}^{(1)}(t+1)=\langle {x}^{(0)}(1)-\frac{u}{a}\rangle {e}^{-at}+\frac{u}{a}.$$
- Establishing prediction model: Discretize the function expressions ${\widehat{x}}^{(1)}(t+1)$ and ${\widehat{x}}^{(1)}(t)$ make the difference to restore the original sequence of x
^{(0)}to obtain the approximate data sequence ${\widehat{x}}^{(0)}(t+1)$ as follows:$${\widehat{x}}^{(1)}(t+1)=\langle {x}^{(0)}(1)-\frac{u}{a}\rangle {e}^{-at}+\frac{u}{a}.$$

## 6. Discussion and Conclusions

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- Bosch, J.; Bosch-Sijtsema, P. From Integration to Composition: On the Impact of Software Product Lines Global Development and Ecosystems. J. Syst. Softw.
**2010**, 83, 67–76. [Google Scholar] [CrossRef] - Manikas, K.; Hansen, K.M. Reviewing the Health of Software Ecosystems—A Conceptual Framework Proposal. CEUR Workshop Proc.
**2013**, 987, 26–37. [Google Scholar] - Jansen, S. Measuring the health of open source software ecosystems: Beyond the scope of project health. Inf. Softw. Technol.
**2014**, 56, 1508–1519. [Google Scholar] [CrossRef] - Lu, Y.; Wang, R.; Zhang, Y.; Su, H.; Wang, P.; Jenkins, A.; Ferrier, R.; Bailey, M.; Squire, G. Ecosystem health towards sustainability. Ecosyst. Health Sustain.
**2016**, 1, 1–15. [Google Scholar] [CrossRef] - Wan, J.; Zhu, S.; Huali, S. Some Considerations of China’s Software Industry Business Ecosystem. Sci. Technol. Manag. Res.
**2009**, 29, 130–132. [Google Scholar] - Jin, Z.; Zhou, M.; Zhang, Y. Open source software and its ecosystems: Today and tomorrow. Sci. Technol. Rev.
**2016**, 34, 42–48. [Google Scholar] - Penzenstadler, B. Towards a definition of sustainability in, and for, software engineering. In Proceedings of the 28th Annual ACM Symposium on Applied Computing, Coimbra, Portugal, 18–22 March 2013; pp. 1183–1185. [Google Scholar]
- Durdik, Z.; Klatt, B.; Koziolek, H.; Krogmann, K.; Stammel, J.; Weiss, R. Sustainability guidelines for long-living software systems. In Proceedings of the IEEE International Conference on Software Maintenance, Trento, Italy, 23–28 September 2012; pp. 517–526. [Google Scholar]
- An, L.; Mlouki, O.; Khomh, F.; Antonio, G. SO: A code lSUndering platform? In Proceedings of the 24th IEEE International Conference on Software Analysis, Evolution, and Reengineering (SANER), Klagenfurt, Austria, 20–24 February 2017; pp. 283–293. [Google Scholar]
- Dhungana, D.; Groher, I.; Schludermann, E.; Biffl, S. Software ecosystems vs. natural ecosystems: Learning from the ingenious mind of nature. In Proceedings of the Software Architecture, European Conference, Ecsa 2010, Copenhagen, Denmark, 23–26 August 2010; pp. 96–102. [Google Scholar]
- Srba, I.; Bielikova, M. Why SO Fails? Preservation of Sustainability in Community Question Answering. IEEE Softw.
**2016**, 33, 1. [Google Scholar] [CrossRef] - Srba, I. Promoting Sustainability and Transferability of Community Question Answering. Inf. Sci. Technol. Bull. ACM Slovakia
**2011**, 8, 1–7. [Google Scholar] - Sethanandha, B.D.; Massey, B.; Jones, W. Managing open source contributions for software project sustainability. In Proceedings of the Technology Management for Global Economic Growth, Phuket, Thailand, 18–22 July 2012; pp. 1–9. [Google Scholar]
- Fotrousi, F.; Fricker, S.A.; Fiedler, M.; Le-Gall, F. KPIs for Software Ecosystems: A Systematic Mapping Study. In Proceedings of the International Conference of Software Business. Springer International Publishing, Paphos, Cyprus, 16–18 June 2014; pp. 194–211. [Google Scholar]
- Monteith, J.Y.; Mcgregor, J.D.; Ingram, J.E. Scientific Research Software Ecosystems. In Proceedings of the 2014 European Conference on Software Architecture Workshops (ECSAW ’14). ACM, New York, NY, USA, 25–29 August 2014; pp. 1–6. [Google Scholar] [CrossRef]
- Gamalielsson, J.; Lundell, B. Sustainability of Open Source software communities beyond a fork: How and why has the LibreOffice project evolved? J. Syst. Softw.
**2014**, 89, 128–145. [Google Scholar] [CrossRef] - Gamalielsson, J.; Lundell, B. Long-Term Sustainability of Open Source Software Communities beyond a Fork: A Case Study of LibreOffice. In Proceedings of the IFIP International Conference on Open Source Systems; Springer: Berlin/Heidelberg, Germany, 2017; pp. 29–47. [Google Scholar]
- Nyman, L.; Mikkonen, T.; Lindman, J.; Fougère, M. Perspectives on Code Forking and Sustainability in Open Source Software. IFIP Adv. Inf. Commun. Technol.
**2012**, 378, 274–279. [Google Scholar] [Green Version] - Matragkas, N.; Williams, J.R.; Kolovos, D.S.; Paige, R.F. Analysing the ‘biodiversity’ of open source ecosystems: The GitHub case. In Proceedings of the 11th Working Conference on Mining Software Repositories, Hyderabad, India, 31 May–1 June 2014. [Google Scholar]
- Sahu, T.P.; Nagwani, N.K.; Verma, S. An Empirical Analysis on Reducing Open Source Software Development Tasks using SO. Indian J. Sci. Technol.
**2016**, 9. [Google Scholar] [CrossRef] - Sun, L.; Li, J. A role model and quality model of software ecosystems. Shanxi Univ. Sci. Technol.
**2011**, 29, 93–95. [Google Scholar] - Zhang, D.; Li, B.; He, P.; Zhou, H. Characteristic Study of Open-source Community Based on Software Ecosystem. Comput. Eng.
**2015**, 41, 106–113. [Google Scholar] - Liao, Z.; Dayu, H.; Chen, Z.; Fan, X.; Zhang, Y.; Liu, S. Exploring the Characteristics of Issue-related Behaviors in GitHub Using Visualization Techniques. IEEE Access
**2018**, 6, 24003–24015. [Google Scholar] [CrossRef] - Lee, C. A Model of Sustainable Ecosystem for Software Development: Software Business and Music Education. 2013. Available online: http://repository.lib.eduhk.hk/jspui/handle/2260.2/16637 (accessed on 1 December 2018).
- Joo, J.; Eom, M.; Shin, M. Executive practices for corporate sustainability: A business ecosystems perspective. Int. J. Bus. Res.
**2016**. [Google Scholar] [CrossRef] - Qiang, F.U.; Fan, D.P. Green Values and the Holistic Optimization of Socio-ecological Systems: From the Perspective of Philosophy of Complexity Science. Stud. Dialectics Nat.
**2017**, 33. [Google Scholar] [CrossRef] - Wang, S.; Loreau, M. Biodiversity and ecosystem stability across scales in metacommunities. Ecol. Lett.
**2016**, 19, 510–518. [Google Scholar] [CrossRef] [Green Version] - Ramírez-Carrillo, E.; López-Corona, O.; Toledo-Roy, J.C.; Lovett, J.C.; de León-González, F.; Osorio-Olvera, L.; Equihua, J.; Robredo, E.; Frank, A.; Dirzo, R.; et al. Assessing sustainability in North America’s ecosystems using criticality and information theory. PLoS ONE
**2018**, 13, e0200382. [Google Scholar] [CrossRef] - Aguilar, B.J. Applications of Ecosystem Health for the Sustainability of Managed Systems in Costa Rica. Ecosyst. Health
**1999**, 5, 36–48. [Google Scholar] [CrossRef] - Ahkami, A.H.; White, R.A., III; Handakumbura, P.P.; Jansson, C. Rhizosphere Engineering: Enhancing Sustainable Plant Ecosystem Productivity in a Challenging Climate. Rhizosphere
**2017**, 3, 233–243. [Google Scholar] [CrossRef] - Wen-Jing, M.A.; Zhang, Q.; Niu, J.M.; Kang, S.; Liu, P.T.; He, X.; Yang, Y.; Zhang, Y.N.; Wu, J.G. Relationship of ecosystem primary productivity to species diversity and functional group diversity: Evidence from Stipa breviflora grassland in Nei Mongol. Chin. J. Plant Ecol.
**2013**, 37, 620–630. [Google Scholar] - Forsius, M.; Akujärvi, A.; Mattsson, T.; Holmberg, M.; Punttila, P.; Posch, M.; Liski, J.; Repo, A.; Virkkala, R.; Vihervaara, P. Modelling impacts of forest bioenergy use on ecosystem sustainability: Lammi LTER region, southern Finland. Ecol. Indic.
**2016**, 65, 66–75. [Google Scholar] [CrossRef] - De Kruijf, H.A.M.; Van Vuuren, D.P. Following Sustainable Development in Relation to the North-South Dialogue: Ecosystem Health and Sustainability Indicators. Ecotoxicol. Environ. Saf.
**1998**, 40, 4. [Google Scholar] [CrossRef] [PubMed] - Niu, M.; Wang, J.; Binduo, X.U. Assessment of the ecosystem health of the Yellow River Estuary based on the pressure-state-response model. Acta Ecol. Sin.
**2017**, 37, 942–952. [Google Scholar] - Neto, E.D.A.L.; de Carvalho, F.D.A. Centre and Range method for fitting a linear regression model to symbolic interval data. Comput. Stat. Data Anal.
**2008**, 52, 1500–1515. [Google Scholar] [CrossRef] - Oprime, P.C.; Mendes, G.H.D.S. The X-bar control chart with restriction of the capability indices. Int. J. Q. Reliab. Manag.
**2017**, 34, 38–52. [Google Scholar] [CrossRef] - Mohamad, D.; Ibrahim, S.Z. Decision making procedure based on jaccard similarity measure with Z-numbers. Pertanika J. Sci. Technol.
**2017**, 25, 561–574. [Google Scholar] - Kuang, L.; Yu, L.; Huang, L.; Wang, Y.; Ma, P.; Li, C.; Zhu, Y. A Personalized QoS Prediction Approach for CPS Service Recommendation Based on Reputation and Location-Aware Collaborative Filtering. Sensors (Basel)
**2018**, 18, 1556. [Google Scholar] [CrossRef] - Gwozdz-Lason, M. Analysis by the Residual Method for Estimate Market Value of Land on the Areas with Mining Exploitation in Subsoil under Future New Building. In IOP Conference Series: Earth and Environmental Science; IOP Publishing: Bristol, UK, 2017. [Google Scholar]
- Guo, T.; He, B.H.; Chen, J.J. Study on soc forecast model in regions of hilly purple soil by water erosion. Adv. Mater. Res.
**2011**, 391-392, 982–987. [Google Scholar] [CrossRef] - Schrepp, M.; Hinderks, A.; Thomaschewski, J. Construction of a Benchmark for the User Experience Questionnaire (UEQ). Int. J. Interact. Multimed. Artif. Intell.
**2017**, 4, 40–44. [Google Scholar] [CrossRef] - Bader, F.; Schön, E.M.; Thomaschewski, J. Heuristics Considering UX and Quality Criteria for Heuristics. Int. J. Interact. Multimed. Artif. Intell.
**2017**, 4, 48–53. [Google Scholar] [CrossRef]

**Figure 2.**Evaluation Indicators System for Sustainability of OSE (the indicators system establishes the target level, guideline level, and evaluation level based on the Pressure-State-Response (PSR) model, determines indicators at the guideline level through measures of natural ecosystem sustainability, and determines evaluation level indicators through participant activities.).

**Figure 3.**(

**a**) User activities in the SO ecosystem (Step 1 indicates that the user creates a question; Step 2 indicates that the answerer answers the question; Step 3 indicates that the reviewers comment on the question and answers; Step 4 indicates that the user accepts a satisfactory answer; Step 5 indicates that the searcher searches for the question through question content and tags); (

**b**) post in SO (users, questions, answers, tags, comments, vote are marked by arrows).

**Figure 4.**Openness analysis (in which the blue line with diamonds represents the number of new users registered for the SO system each year, and the orange line with triangles represents the number of questions with the answers per year).

**Figure 7.**Top 10 popular languages for each years of 2010–2016 in GitHub and SO. (The bar chart on the left of each year shows the top 10 popular languages in GitHub, ranked from top to bottom according to the number of languages used by the GitHub project; the bar chart on the right represents the top 10 most popular languages in SO, ranked from top to bottom according to the number of questions raised by users in the SO system. Grey represents different popular languages in the GitHub and SO, where O-C stands for Objective-C).

Influencing Factors | Meaning |
---|---|

openness | The ability of the entire ecosystem to communicate and transformation with the outside environment or within the ecosystem. |

stability | The anti-interference ability of the structure, state and behavior of the ecosystem. Including avoidance, tolerance and resilience. |

integrity | The internal composition, structure and function of an ecosystem and the integrity of its external biophysical environment. |

productivity | The biological production capacity of the ecosystem. |

regulation | The ecosystem has certain resistance to interference and has a certain ability to recover after being disturbed. It can coordinate and maintain stability. |

resilience | The ability of ecosystems to maintain function under pressure. |

diversity | Rich and balanced species within the ecosystem. |

drivers | Natural or man-made disturbances or stress on ecosystems that change ecosystems. |

organizational structure | Components and structures in the ecosystem. |

a propensity for growth | Ecosystem growth is good. |

conform to development trend | The ability to meet the needs of contemporary people. |

Year | The Number of New Users | Total Number of Posts | Total Number of Comments | Tag |
---|---|---|---|---|

2010 | 199,547 | 2,174,195 | 2,847,812 | <css>,<html>,<java>... |

2011 | 360,035 | 3,510,622 | 5,135,270 | <java>,<JavaScript>,<c++>... |

2012 | 686,547 | 4,518,274 | 7,217,973 | <c++>,<c>,<c#>... |

2013 | 1,126,738 | 5,425,822 | 9,172,187 | <java>,<c++>,<c>... |

2014 | 1,181,162 | 5,406,334 | 9,209,145 | <css>,<js>,<html>... |

2015 | 1,261,603 | 5,412,464 | 9,328,299 | <ruby>,<python>,<c++>... |

2016 | 1,569,322 | 5,617,974 | 9,765,734 | <java>,<c++>,<c#>... |

Target Level | Guideline Level | Evaluation Level |
---|---|---|

Sustainability of SO ecosystem | openness | the number of new users |

the number of questions with the answers | ||

stability | the number of stable users | |

activity | the response rate of questions | |

the average number of comments | ||

extensibility | popularity | |

growth force |

Year | 2010 | 2011 | 2012 | 2013 | 2014 | 2015 | 2016 |

Similarity | 0.7 | 0.9 | 0.9 | 0.8 | 0.9 | 0.9 | 0.9 |

Target Level | Guideline Level | Evaluation Level | Guideline Level (weight) | Evaluation Level (weight) |
---|---|---|---|---|

Sustainability of SO ecosystem | openness | the number of new users | 0.2861 | 0.1728 |

the number of questions with the answers | 0.1085 | |||

stability | the number of stable users | 0.1482 | 0.0889 | |

activity | the response rate of questions | 0.2814 | 0.1104 | |

the average number of comments | 0.2184 | |||

extensibility | popularity | 0.2842 | 0.2031 | |

growth force | 0.0978 |

**Table 6.**Comprehensive assessment value of the sustainability and the composite index of the guideline indicators.

Year | Openness Index | Stability Index | Activity Index | Extensibility Index | Comprehensive Assessment Index |
---|---|---|---|---|---|

2010 | 0.0081 | 0.0013 | 0.0646 | 0.0086 | 0.0825 |

2011 | 0.0180 | 0.0066 | 0.0638 | 0.0334 | 0.1218 |

2012 | 0.0396 | 0.0111 | 0.0618 | 0.0419 | 0.1544 |

2013 | 0.0639 | 0.0132 | 0.0567 | 0.0407 | 0.1744 |

2014 | 0.0665 | 0.0113 | 0.0440 | 0.0623 | 0.1840 |

2015 | 0.0693 | 0.0104 | 0.0347 | 0.0708 | 0.1852 |

2016 | 0.0791 | 0.0101 | 0.0372 | 0.0855 | 0.2120 |

Year | Actual Value | Predicted Value | Relative Error | Average Relative Error |
---|---|---|---|---|

2010 | 0.0825 | 0.0825 | 0.0000 | 0.0439 |

2011 | 0.1218 | 0.1359 | 0.1157 | |

2012 | 0.1544 | 0.1486 | 0.0373 | |

2013 | 0.1744 | 0.1626 | 0.0678 | |

2014 | 0.1840 | 0.1778 | 0.0339 | |

2015 | 0.1852 | 0.1944 | 0.0496 | |

2016 | 0.2120 | 0.2126 | 0.0030 |

Open Source Community | Community Function | Community Introduction |
---|---|---|

Github | open source code hosting | GitHub is a web-based hosting service that uses Git for version control, providing access control and multiple collaboration features for each project, such as bug tracking, feature requests, task management, and wiki. |

Sourceforge | open source code hosting | Sourceforge is a web-based service that provides software developers with a centralized online platform to control and manage open source software projects. |

Quora | Q&A Community | Quora is a knowledge market, Quora brings together many questions and answers, allowing users to collaboratively edit questions and answers. |

Topcoder | crowdsourcing platform | Topcoder is a mass outsourcing company with an open global community of designers, developers, data scientists and competitive programmers, and pays for the work of community members on projects, as well as providing services to businesses, medium-sized enterprises and small business customer sales communities. |

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Liao, Z.; Deng, L.; Fan, X.; Zhang, Y.; Liu, H.; Qi, X.; Zhou, Y.
Empirical Research on the Evaluation Model and Method of Sustainability of the Open Source Ecosystem. *Symmetry* **2018**, *10*, 747.
https://doi.org/10.3390/sym10120747

**AMA Style**

Liao Z, Deng L, Fan X, Zhang Y, Liu H, Qi X, Zhou Y.
Empirical Research on the Evaluation Model and Method of Sustainability of the Open Source Ecosystem. *Symmetry*. 2018; 10(12):747.
https://doi.org/10.3390/sym10120747

**Chicago/Turabian Style**

Liao, Zhifang, Libing Deng, Xiaoping Fan, Yan Zhang, Hui Liu, Xiaofei Qi, and Yun Zhou.
2018. "Empirical Research on the Evaluation Model and Method of Sustainability of the Open Source Ecosystem" *Symmetry* 10, no. 12: 747.
https://doi.org/10.3390/sym10120747