Knowledge Graph in Smart Education: A Case Study of Entrepreneurship Scientific Publication Management

Abstract

In recent years, with the rapid growth of science and innovation, plenty of constantly-updated scientific achievements containing innovative knowledge can be acquired and used to solve problems. However, most undergraduate students and non-researchers cannot use them efficiently. In traditional teacher-centric education, education for sustainability is often marginalized and the interdisciplinary demand is neglected. Additionally, it fails to provide education for learners to connect abstract knowledge with actual world problems. This paper presents the design of a scientific publication management model to integrate scientific metadata based on the knowledge graph and data analysis technologies. Based on this model, an interdisciplinary transregional multiple application platform could be realized for scientific resource retrieval and analysis, the purpose of which is to enhance scientific retrieval efficiency and reduce learning difficulty in the scientific domains and encourage non-researchers to utilize scientific resources in their study and work. Finally, to evaluate this model, the use of the case of an entrepreneurship scientific publication management prototype system was implemented. This work not only favors student’s learning for sustainability through analysis and knowledge management functions, but also promotes their awareness, comprehensive thinking, and the skills to deal with the issues of sustainability in their future work.

1. Introduction

1.1. Smart Education for Sustainability

In 1987, the World Commission on Environment and Development defined development sustainable (SD) as an ethical concept and has become the major definition of SD: “Sustainable Development is a development that meets the needs of the present without compromising the ability of future generations to meet their own needs.” [1]. Since the release of the report, this definition has been subject to several modifications and was reformulated according different point of views. One widely-accepted view presented it as the intersection between environment, society and economy [2]. The United Nations Education, Scientific, and Cultural Organization (UNESCO) wished to achieve sustainable development through education [3]. According to UNESCO, education for sustainable development fosters learning to make decisions that secure the economic, environmental, and social future. The basic meaning of sustainable education could be considered as education that can contribute to sustainable development, the target of which is to help students acquire knowledge, learning capacity, life style, and values for sustainable development and further promote the sustainable development of society, economy, environment, and culture. It is necessary to educate individuals and groups to live and act sustainably and give them an understanding of the environmental, social, and economic issues involved [4].

Many institutions around the world have emphasized the importance for education to achieve sustainability, however, only limited progress has been made. There are some problems in sustainable education that need to be solved as soon as possible.

First, UNESCO points out that “this vision of education emphasizes a holistic, interdisciplinary approach to developing the knowledge and skills needed for a sustainable future as well as changes in values, behavior, and lifestyles.” [5]. Sustainability is a cross-curricula theme, however, in traditional curricula, it is always neglected because of the academic divisions of knowledge which separate the natural and social sciences and the humanities [6].

Second, learning-based education that includes ‘learning to know’, ‘learning to do’, ‘learning to live together’, and ‘learning to be’ were emphasized in sustainable education, one of the important points of which is to learn how to learn, acquire, construct, and manage knowledge (learning to know) and the way of putting knowledge into action (learning to do) [7]. At present, university education depends on traditional classroom learning [8], which is a teacher-centric education model emphasizing basic knowledge studies and exam scores. It fails to provide education for learners to connect abstract knowledge with actual world problems, which promotes narrow and individualistic activities and corresponding world views among the students [9].

Finally, many learners meet the problem of isolation and lack of access to information and knowledge. They do not have a chance to accept sustainable education because of the unbalanced development of education. In fact, education for sustainability is often marginalized in national classes [6].

In the modern age, the increase of information technologies has provided an opportunity to overcome the problems of lack of access to information and knowledge. It can provide learning resources for students without geographical and time constraints, which promote educational and socioeconomic development. Some technologies can also help learners explore concepts, engage in problem-based and authentic learning, enhance meta-cognitive skills, and present information using multiple media [4]. All these are closely related to the goals, themes, and learning objectives addressed by education for sustainability [10,11,12,13,14]. However, one review in the field of education for sustainability showed that most education institutions do not exploit the potential of information technologies in education for sustainability, especially Web 2.0 technologies and the use of open education resources available on the Web [15].

Therefore, it is beneficial to import emerging information technology into sustainable education. Several studies have implemented smart education using information technologies and acquired good effects in education for sustainability. One study developed an information and technologies supported pedagogical framework, which combined information technology with a problem-based learning approach in order to promote awareness, critical thinking, and the skills to deal with the issues among middle and high school children [16]. One work designed a virtual learning environment that aimed to offer the chance for students to interact asynchronously and synchronously, negotiate meaning and reflect on their learning and viewpoints through collaborative problem solving. Some researchers combined gamification and splitting and combining to help student understand knowledge structure, and the work also improved students’ learning effect [17].

This paper focuses on integrated academic entitles (not only scientific papers, but also other entities: researchers, journal, conference, research institution, and domain topic) and provides learners with an interesting analysis and retrieval service. In this work, an academic network based on knowledge graph technology was built for students to search and analyze knowledge. The purpose of the work was similar: to provide access to information and knowledge without geographical and time constraints, engage student in problem-based and authentic learning to make them acquire capabilities of awareness, critical thinking, and skills to deal with the issue, and eliminate discipline boundary.

Some achievements in sustainability have focused on resource integrated and knowledge management to support sustainable education. Some researchers have put forward that information and communication technology (ICT) should be used to integrate education resources and have proposed that higher education institutions should integrate and manage knowledge resources to provide a learning environment for students [18].

Compared with those works, this paper holds two innovations. First, this work designed a management model based on knowledge graph and data mining technologies. This model could support more clear analysis and visualization of the relationship of research entities and research topics, which is helpful in understanding knowledge structure and relationships between knowledge in sustainability. It is significant because sustainability is an interdisciplinary domain between the environment, economic, and social issues. Second, this paper regards academic achievements as the main learning resource of sustainable education, the reasons of which are as follows:

First, scientific papers are the most important innovation achievement in the modern age. With the beginning of the era of open innovations, how to manage, learn, and utilize the innovation achievements, and promote innovation for the sustainable development of the entire society are important questions. From the perspective of education and education systems, some researchers have considered a few related questions: What are the challenges related to innovation and how do we employ innovation in view of building and exploiting innovation networks [19]? This work realized a method to manage and link scientific entities as well as provide people with analysis and retrieval of these innovation achievements.

Second, different from other learning resources, there are plenty of interdisciplinary achievements in scientific publications that meet the requirements of sustainable education: an emphasis on a holistic, interdisciplinary, and cross-disciplinary approach to developing the competences needed for building a sustainable future. Additionally, the natural advantage of a knowledge graph is to maintain, analyze and visualize the relationships between the subjects and knowledge.

Finally, the thinking model of research is problem-based. Many studies have analyzed the semantic structure of scientific papers where the basic think logic in scientific papers includes the background, problem, method, and conclusions [20,21]. As above-mentioned, problem-based learning can promote awareness, critical thinking and the skills to deal with the issue and has been emphasized in the education of sustainability.

Scientific resources with innovative research and knowledge can service plenty of applications in the world, which would not only promote a student’s learning capabilities, but also promote innovation sharing and research cooperation. Some researchers have found that the academic environment is a wealth of knowledge, but it is not organized properly and hence utility is also lacking and a cause for repetitions of the activity [22]. According to interviews of this research, at present, only researchers use these scientific achievements efficiently, while other undergraduate students and non-researchers lack the ability to search and use scientific papers to solve problems. There are several reasons that obstruct non-researchers in learning from scientific papers. First, knowledge in scientific papers is professional knowledge, which is challenging for students to learn and understand by themselves. In addition, in recent years, science and scientific research achievements have increased immensely, which makes scientific retrieval difficult for students. To acquire suitable scientific resources, students need to search on many different retrieval platforms, which can waste time. Due to the lack of comprehension regarding scientific entities such as scientific journals, scientific institutions, and scientific domains, students do not know how to choose high-quality related papers. Moreover, without the help of teachers, sometimes students do not know which search terms should be used for searching. These barriers make students think that searching and understanding scientific publications is difficult and inconvenient, so they may prefer traditional classroom education.

Therefore, to help non-researchers access scientific entities, study, understand sustainability from scientific resources and acquire the capability to put knowledge into action in the future, it is significant for smart education to provide an academic knowledge management model and help students learn and utilize knowledge and methods regarding sustainability from scientific papers as well as form problem-based research thinking. This work realized a scientific entities management model and implemented a case study to train students learning and researching about entrepreneurship ecosystems about this system to acquire scientific publications and domain knowledge. The results showed that students understood more about sustainability in entrepreneurship management, and they presented more confidence and capability to find and solve problems than before.

1.2. Knowledge Graphs in Scientific Entities Management

In 2012, Google published the knowledge graph project and announced its intention to build an intelligent search engine based upon it [23]. A knowledge graph is a structured knowledge base consisting of the triple “entity-property-entity,” which is used to describe entities, concepts, and the relationships between them in the physical world [24]. The superiority of the knowledge graph is that it can implement concept retrieval through inference and provide users with structured knowledge directly, while the traditional search engine is based on string matching the return links of webpages.

In addition, based on the knowledge graph, many applications, including retrieval and management, can be realized. There are a variety of researchers that have applied the knowledge graph and knowledge management in the field of education. Some researchers have recommended a learning path for students based on the knowledge graph [25], and others have managed learning data for students using the knowledge graph [26]. In scientific retrieval, the knowledge graph has promoted semantic society web development, and some researchers have integrated scientific publications and author data from different databases and provided a retrieval service for users [27]. Furthermore, knowledge graphs can also be used for micro-learning, and some researchers built a knowledge base to support the decision-making process of micro open educational resources adaptation. A construction of augmented ontologies oriented to micro open learning will be illustrated first, followed by a data processing strategy [28].

In this work, a management model for scientific resources was designed and a use case for entrepreneurship scientific publication management was implemented based on a knowledge graph. More specifically, in the use case, this work extracted related entities in scientific resources, such as scientific papers, authors, journals, research institutions, and so on, and the relationships among them. Next, some applications, such as scientific entity retrieval, analysis, recommendation, and management were implemented, which aimed at reducing the difficulty of scientific retrieval and understanding and helping students search for scientific entities and to understand domains efficiently. This paper also realized semantic annotation for abstracts to facilitate searching and understanding and provided a semantic search for students.

Unlike traditional scientific publication retrieval, this work intended to integrate and link scientific entities based on a knowledge graph, which meant that not only scientific papers could be retrieved, but also other entities, such as journals, researchers, and institutions that could also be retrieved and analyzed. Moreover, this work attempted to show more useful details of the content of an article to help students retrieve and choose sources. For example, geographic entity and theory terms were extracted from abstracts and shown in this use case system as they are important terms in the field of entrepreneurial ecosystems. Meanwhile, some data analysis services were also provided including hot issues, authoritative journals, researchers, and institutions in one field. The final purpose of this research was to enhance the utilization of scientific resources for undergraduate students and non-researchers, which is an advantage for their sustainable learning and development. In addition, the knowledge graph can integrate heterogeneous scientific data distributed in different databases, which benefits knowledge sharing and scientific research cooperation worldwide to promote the sustainable development of education.

2. Materials and Methods

2.1. Scientific Publication Management Model

Nowadays, most students use digital library platforms to search for scientific papers. This kind of system can satisfy the basic needs of retrieval for students. With the development of information technology and data analysis, a new scientific publication management model based on the knowledge graph could support better service for students.

Figure 1 shows the management model, which would not be limited to paper retrieval, but have a scientific resource management function, including retrieval, analysis, and recommendations. The bottom layer of it is based on the knowledge graph, which integrates data from different platforms and extracts and manages related scientific entities such as paper, author, research institution, and journal to form a scientific society web. Based on the knowledge graph, data analysis could be implemented to show more details about one research domain. For example, this work could analyze research hotspots in one field and research domains for one author. The top layer is the application platform that provides the student with abundant services.

2.2. Data Schema Design

The knowledge graph is a structured knowledge base composed of entities, concepts, and the relationships among them. This work aimed to extract entities and concepts regarding scientific papers from scientific resources and establish a scientific resource knowledge graph.

Knowledge graphs can be regarded as having two parts: one is the data schema at the bottom, also called the concept level; and the other is the data level. Data schema should be designed by experts, and the data schema in this work is shown in Figure 2. In this paper, these kinds of entities, concepts, and their property relations were defined to describe scientific resources and form a knowledge graph. The entity class in this work represented objects that are important in scientific resources, such as papers, researchers, journals, and organizations, and were linked together through their properties. The concept class referred to the abstract concept. In this work, they were always the domain terminologies extracted from scientific papers, and concepts were also linked with properties.

At the data level of the knowledge graph, every class in the concept level holds a variety of instances, which is the specific entity or concept of the class. Every instance holds properties defined in a corresponding class and their own values for these properties. There were two types of properties in this work: the data property links the instance to a string or number value, and the object property, such as “written_by,” links the instance to another instance. In this way, the knowledge graph could be regarded as a knowledge network composed of triples of entity(concept)-property-entity(concept). “Paper,” “Researcher”, and their instances were selected as an example in Figure 3. “Paper” and “Researcher” are two classes, while a specific paper or an author is the instance of a corresponding class.

In Figure 2 and Figure 3, three kinds of object-property relations were found.

• “Is_a” represents an inheritance relationship; for example, “Researcher” is a “Person.” In an inheritance relationship, a subclass holds all properties of the superclass and adds their own properties. For example, “Researcher” holds properties of “Person” such as “name” and has its own properties, such as work in some research institutions, and so on. An inheritance relationship gives a hierarchical structure to the data schema.
• “Instance_of” represents the instance relationship. For example, “Paper” is the class of scientific paper, and an actual scientific paper entity is an instance of “Paper,” which has its own value of properties defined for the “Paper” class.
• Other special properties represent relationships between entities and concepts such as “published_on” and “written_by.” This work defined abundant and extensible properties for every entity and concept to describe scientific resources. For example, the properties for the “Paper” entity are shown in

  Table 1. Properties for “Paper” entity.

  Property Name	Property Type	Value Type
  identifier	ID	INTEGER
  doi	Data Property	STRING
  title	Data Property	STRING
  type	Data Property	STRING
  keyword	Data Property	[]STRING
  abstract	Data Property	STRING
  citation_number	Data Property	INTEGER
  publication_year	Data Property	DATE
  language	Data Property	INTEGER
  cite	Object Property	[]Paper
  written_by	Object Property	[]Author
  published_on	Object Property	Source
  research_object	Object Property	[]Entity
  research_theme	Object Property	[]Domain_Concept

  , and properties of other entities defined in this work can be found in

  Table A1. Properties for the “Researcher” entity.

  Property Name	Property Type	Value Type
  identifier	ID	INTEGER
  name	Data Property	[]STRING
  email	Data Property	[]STRING
  talent_title	Data Property	[]STRING
  gender	Data Property	INTEGER
  home_page	Data Property	[]STRING
  nationality	Object Property	[]Country
  work_on	Object Property	[]Research_Affiliation
  research_area	Object Property	[]Domain_Concept

  ,

  Table A2. Properties for the “Journal” entity.

  Property Name	Property Type	Value Type
  identifier	ID	INTEGER
  name	Data Property	[]STRING
  ISSN	Data Property	STRING
  eISSN	Data Property	STRING
  language	Data Property	INTEGER
  impact_factor	Data Property	[]STRING
  published_by	Object Property	[]Organization
  research_area	Object Property	[]Domain_Concept

  ,

  Table A3. Properties for the “Conference” entity.

  Property Name	Property Type	Value Type
  identifier	ID	INTEGER
  name	Data Property	[]STRING
  date	Data Property	DATE
  located_at	Object Property	[]Location
  published_by	Object Property	[]Organization
  research_area	Object Property	[]Domain_Concept

  ,

  Table A4. Properties for the “Country” entity.

  Property Name	Property Type	Value Type
  identifier	ID	INTEGER
  name	Data Property	[]STRING
  latitude	Data Property	STRING
  longitude	Data Property	STRING
  part_of	Object Property	[]Location

  ,

  Table A5. Properties for the “Research_Affiliation” entity.

  Property Name	Property Type	Value Type
  identifier	ID	INTEGER
  name	Data Property	[]STRING
  address	Data Property	[]STRING
  email	Data Property	[]STRING
  part_of	Object Property	[]Organization
  located_at	Object Property	[]Location

  and

  Table A6. Properties for the “Domain_Concept” entity.

  Property Name	Property Type	Value Type
  identifier	ID	INTEGER
  name	Data Property	[]STRING
  related_with	Object Property	[]Domain_Concept
  sub_domain_of	Object Property	[]Domain_Concept

  .

2.3. Building a Knowledge Graph

Building a knowledge graph is the process by which entities and properties are extracted and stored at the concept and data levels of the knowledge graph based on some automated or semiautomatic methods. This paper chose scientific paper resources in the field of entrepreneurial ecosystems to build a knowledge graph; this procedure can be seen in Figure 4. First, the metadata of the scientific paper were acquired from scientific databases. Then, data extraction, entity links, and ontology learning were implemented to acquire entities, concepts, and properties. Finally, they were added to the knowledge graph.

The resource data in this work for knowledge graph building were metadata extracted from papers from three scientific databases: Web of Science, Engineering Village, and EBSCO. The metadata of scientific papers include the title, author, journal, and so on, which can be acquired from the databases directly. In the retrieval, “Entrepreneurial Ecosystem” was used as the search term, and the language was set to English.

Table 2. Data sources of this work.

Source	Number of Papers	Search Words
Web of Science	158	Entrepreneurial Ecosystem
Engineering Village	50	Entrepreneurial Ecosystem
EBSCO	45	Entrepreneurial Ecosystem

presents the sources of metadata in this work.

Table 3. Meta data acquired.

Name	Abbreviation
Title	TI
Author	AF
Source	SO
Keyword	DE
Abstract	AB
Language	LA
Number of reference	NR
Publish year	PY
Digital object identifier	DOI

shows what kind of metadata were acquired in this work.

After acquiring the metadata of the papers, this work extracted and defined entities, concepts, and properties according to the model introduced in the previous chapter. The metadata of a paper is semi-structured and some properties, such as title, abstract, and author name, can be downloaded directly. However, the values of ‘research_object’ and ‘research_theme’ should be extracted from the title, keywords, and abstract as they are the research content of scientific papers that are not always provided by a traditional database. These are significant in this work as this paper focused on general education, and more content extraction could reduce the difficulty of retrieval and learning not only for researchers, but for every student and non-researcher to use in their learning and work.

In the field of entrepreneurial ecosystems, some entities in the case study, such as research locations and institutions, can reveal some research content. Domain concepts, such as theory, method, and domain keyword, can also inform the searchers of the main content of the paper. Thus, as shown in

Table 4. Data extracted from metadata.

Value	Property	Example
Location entity	research_object	China, Europe
Institution entity	research_object	Cambridge University
Domain theory	research_theme	Stakeholder theory
Domain method	research_theme	Case study, Questionnaire
Domain keywords	research_theme	Entrepreneurial ecosystem

, in this work, these entities and concepts were extracted from the metadata to provide students with more annotated information. To implement automatic extraction, this work established a geographic dictionary based on Wordnet and established a theory and item dictionary in the field of entrepreneurial ecosystems based on Wikipedia, the scientific paper metadata, and expert guidance. Finally, a dictionary and some machine-learning algorithms were used to extract these related entities and concepts from the metadata.

The newest entities and concepts extracted from the metadata should be linked in the knowledge graph. An important task is to map them to corresponding entities and concepts in the knowledge graph, and if the same entity or concept does not exist in the knowledge graph, a new entity or concept should be built. Given that natural language always has polysemy and synonyms, this makes this mapping difficult. For example, for researchers, name repetition is common, so it is advisable to judge whether the same name represents the same person. Meanwhile, “United States of America,” “America,” “USA,” and “U.S.A.” are different strings, but they should be mapped to the same country entity. Today, there is a variety of technologies of entity disambiguation and entity alignment for data linking, which always relies on the similarity calculation of words in document content. In addition, authors with the same name could be distinguished by some probabilistic models, which provide judgment according to their email, work institution, citing papers, other authors in their papers, and so on.

After entity linking, new extracted entities and concepts can be mapped to the knowledge graph. If new entities and concepts are out of the scope of the definition of the data schema, the schema should be supplemented and updated automatically or artificially through ontology learning. It is observed that knowledge graph building is an iterative procedure.

This knowledge management system is composed of four modules: the User Interface (UI) module, natural language processing (NLP) module, ontology module, and entity module. The UI module is the user interface of this platform, and the NLP module is responsible for natural language processing, which could extract concepts and entities related with scientific publications from natural language that is input by users. The ontology module and entity module are the databases that store the knowledge graph; the ontology module stores concepts in the knowledge graph, and the entity module stores the entities. When the UI module receives query information from a user, it will transmit the query to the NLP module to extract concepts and entities from natural language. Then, in the ontology module and entity module, the concepts and entities will be mapped to the corresponding nodes of the knowledge graph. Finally, based on some rules, the related nodes of entities or concepts that the user wants to obtain will be returned to the UI module.

2.4. Semantic Annotation for Scientific Paper Abstract

To provide semantic retrieval for students for convenient retrieval and understanding, this work realized semantic annotation for abstracts. Scientific papers are always written with a logical sequence. First, the research background is introduced. Then, the research question and research method are described. Finally, the conclusions and evaluation of the research are provided. The annotation model of scientific papers is presented in

Table 5. Annotation model for abstracts.

Annotations	Simplified Annotations	Comments
Hypothesis	Background	Research background and research motivation
Motivation
Background
Objective	Objective	Overview of research objective and research content
Experiment	Content	Detail of research, including research procedure, method, model, experiment, and so on
Model
Method
Observation
Result	Conclusion	Research conclusion, including findings, suggestions, and significance
Conclusion
Related work
Future work

, and this paper uses the simplified version for abstract annotation.

According to the semantic features of scientific abstracts and the domain of entrepreneurial ecosystems, this paper chose four tags to perform automatic annotation for abstracts: research background, objective, content, and conclusion. Sentences that describe the research background and research motivation were labeled ‘background’. The overview of the research objective and research content were labeled ‘objective’. The description of the details of the research method, research content, and research procedure were labeled ‘content’. Finally, research conclusion sentences that included findings, suggestions, and significance were labeled ‘conclusion’.

This paper regarded automatic semantic annotation as a text classification problem in machine learning. For each sentence in one abstract, machine-learning algorithms were used to classify it to one semantic target. As entrepreneurial ecosystems are a newly-developing research field, there is insufficient large-scale data for training. Thus, this work used the traditional classification method, designed some features, represented sentences as a feature vector, and then performed classification using machine-learning algorithms. After the experiment, the following types of features could support classification effectively.

• Location: The semantic tag of a sentence is related to its location in the abstract. For example, the background sentence is always at the beginning of an abstract, while the conclusion is always at the end of an abstract. Therefore, this work calculated the distance between the location of each sentence to the average location of each tag, then, four different feature values were given to the sentence according to the distance for the four tags.
• Pointer word: Pointer words can be regarded as words with a high frequency in one classification sentence but hold low frequencies in other classifications. Four pointer word dictionaries for the four tags were established based on the chi-square test and artificial statistics. Then, this work built four features for each sentence according to whether it held a pointer word for each tag.
• Relative location with other sentences: The relative location with other sentences includes classification of the previous sentence and the relative location between this sentence and the first sentence, which holds the objective or conclusion pointer word.

The experiment results of this work can be seen in

Table 6. F-score of the annotation model.

Training Sentences Number	Algorithm	F-Score
1412	Support Vector Machine	0.83
1412	K-Nearest Neighbor	0.75
1412	Naive Bayes Model	0.80
1412	Logistic Regression	0.81

. In this experiment, 253 abstracts were used for training and testing, and 1412 sentences were trained using four classification algorithms, including support vector machine, k-nearest neighbor, naive bayes model, and logistic regression. The results showed that under-fitting is a problem for small datasets, but this work still held a good F-score.

In addition, this model could also be used in semantic annotation for abstracts in other fields. The annotation of abstracts could be used by students for exhibition and semantic retrieval to facilitate reading and searching.

3. Entrepreneurship Scientific Publication Management Use Case

This chapter introduces a use case of the entrepreneurship ecosystem scientific publication management. Entrepreneurial ecosystem comprises many kinds of entrepreneurship participants and entrepreneurial environments, and there are complex interactions that occur between them to enhance the overall level of entrepreneurship. This paper acquired metadata, performed semantic annotation for abstracts, and built a knowledge graph according to the method introduced in the previous chapter. Finally, an entrepreneurial ecosystem scientific resource management platform was implemented to provide service for undergraduate and master’s students in the school of management.

The retrieval page of this platform is shown in Figure 5. Compared with traditional retrieval systems for scientific publications, according to the search terms provided by the students, this platform supported the retrieval of a variety of scientific entities including papers, sources, researchers, and research institutions, which was based on string matching or recommendations. Meanwhile, to provide students with plentiful visualization, two display modes showed the retrieval results: the list and the network diagram. The meaning of the displayed combined list and visualization is that the information for all the data could be shown for students through the visualization of a graph, while the list displays the details of every entity. The abstract annotation and the entity, theory, and method annotation were displayed on the information detail page of each paper to help student reading and could be used in the semantic retrieval in future work.

Based on the knowledge graph, this platform provided some basic data analysis services, such as research hotspots, theory and method lists, influential journals and institutions, and so on. In this paper, three visualization analysis diagrams are shown in Figure 6, Figure 7 and Figure 8. Figure 6 shows the results of the location entities extracted from the content of scientific papers. It was observed that America was the most popular research object, followed by Western European countries and China. Figure 7 presents the scientific publication number for every year from 2006 to 2017. ‘Entrepreneur ecosystem’ is a new research direction, and in recent years, more researchers are paid attention to it. Figure 8 shows the journals with the most of scientific papers in this field. The purpose of the visualization analysis was to allow non-researchers to gain information on one research field efficiently and easily.

4. Results and Discussion

To evaluate and improve this work, student feedback should be obtained and analyzed. Fifteen undergraduate students and eight master’s students who researched entrepreneurial ecosystems in the College of Management used this system for four weeks. Most of them gained some knowledge regarding entrepreneurial ecosystem, however, did not access to the special class of sustainability. Most of them held narrow conceptions of “Sustainable Development” through other traditional curriculums. During this time, the teacher assigned one research question on entrepreneurial ecosystems and students searched related learning resources, learnt knowledge, understood concepts, and solved this question based on reading scientific achievements using this system.

The research question was: “Though management systems are fueling a growing body of scholarly literature on ‘corporate greening’, very little effort has been made to understand the role that young, entrepreneurial firms can contribute towards a more sustainable society through innovation [29,30,31]. Please explain the concept of sustainability and discuss how an entrepreneurial community could potentially evolve into a ‘sustainable valley’?” After four-week study, students presented their report answering and accepted evaluation of their knowledge of sustainability and research capabilities. Then, they completed the questionnaire survey. The questions were as follows:

• Does this system appeal to you?
• Do you think this model is helpful for sustainable education?
• When you have questions, are you inclined to utilize scientific publications more than before?
• Does this system improve your query efficiency?
• Do the analysis and visualization functions help you understand more about this field?
• What part of the function is helpful for your study? Why?
• How do you feel about this system, and what suggestion do you have for this model?
• What capacities have you improved after learning in last month? Why(if have)
• Write the detailed method and procedures when you learn and solve problems you meet in studies or work. (before and after)

The top five questions were single choice and the option values were −1, −0.5, 0, 0.5, and 1. Positive numbers represented a positive response, while negative numbers represented a negative response, and the absolute value showed a positive or negative degree. Question 1 was about the user experience of this service. Questions 2 and 3 were aimed to obtain viewpoints about sustainable education. Questions 4 and 5 represented concrete help about retrieval and analysis for students. The average scores of those questions are shown in

Table 7. Opinion score for every question.

Question Number	Average Score (Undergraduate)	Average Score (Master)
1	0.615	0.563
2	0.808	0.688
3	0.538	0.313
4	0.462	0.500
5	0.731	0.375
All	0.631	0.489

. This model played a positive role in encouraging students to utilize scientific resources and enhance sustainable development for education and personal use, especially for undergraduate students.

Questions 6 and 7 aimed to obtain the attitudes and suggestions of students for this model. Through the result, 15 students viewed this work in a positive light, while others held a negative attitude. Some supporters mentioned that the knowledge graph could integrate data and entities stored in different databases, which was convenient when searching for information. In addition, the analysis was helpful to understand the research status. Other students thought that these visualizations were interesting and could not be found in traditional retrieval platforms. Most of them mentioned that this system enhanced their enthusiasm of searching for scientific papers. However, some students thought that many functions of this system were not necessary as they were accustomed to using traditional retrieval platforms. Someone stated that the recommendation and analysis functions are simple and not sufficiently mature at present.

To further analyze the action mechanism of this learning style, two questions are put forward for the further discussion:

Question 1: What kind of students are suited to this learning style? What kind of students are against this learning style?

To answer these questions, a background investigation was conducted for all interviewees. Above all, the interviewees were categorized in different ways, and the use of the system by each category of interviewees was analyzed. Upon comprehensive consideration, interviewees were categorized according to the gender, education level, objective learning ability, subjective learning ability, learning activity level, and learning style.

Concerning the evaluation of learning ability in the entrepreneurial ecosystem field, this work not only required their instructors to grade them according to their test results, reading and publishing of papers, as well as usual performance, but also asked them to evaluate themselves. It can be seen from the two results that students were more discreet and humble in the evaluation of their abilities when compared with the teachers. It was found in the interview that all students were from management-related specialties, and only two of them have had contact with the research of sustainability. Students were less confident in evaluating their learning ability of strange fields. The score of the activity level was based on the length of learning, times of participating in discussions, initiatively, times of communicating with instructors, and it was hoped that this index could reflect the students’ enthusiasm and interest in learning. Through further interviews, it was found that character would influence the judgment, and some introverted students were less active in expressing their ideas, but it could not suggest that they had no interest in learning this field. Therefore, the weight of learning time was increased. Finally, the inclined learning styles of students were investigated, namely, reception learning, independent learning, and cooperative learning. According to the interviews, students of reception learning preferred traditional classroom learning. Students of independent learning would select the learning contents, and regardless of classroom learning or extracurricular learning, they preferred learning what they were interested in, thus featuring strong information acquisition ability.

After the categorization of students, the number of excellent papers on entrepreneurial ecosystems read by students within a month was tested and counted. Meanwhile, with the help of teachers from the School of Management, their grasp of knowledge about the entrepreneurial ecosystem and sustainability, as well as their understanding of sustainability in entrepreneurial ecosystems was examined according to their report on the research issue. Their professional competence after learning was quantized by six indexes, namely A+, A, B, C, D, and D–, in comparison with their objective professional competence, for instance, the objective professional competence of a student was evaluated as B, while their professional competence was A+, the score of learning effect (LE) of this student would be 2; if the objective professional competence of a student was evaluated as B, while their professional competence was C, then the score of learning effect of this student would be −1. Finally, the mean value of the emotional score and learning effect score of each category of students was counted, and results are shown in

Table 8. Category of students and their average learning effect (LE).

Feature	Category Labels (Number)	LE
Gender	Female (14)	0.643
	Male (9)	0.556
Educational Level	Undergraduate (15)	0.733
	Master (8)	0.375
Professional Competence (Objective)	A: Very Good (7)	0.429
	B: Good (10)	0.700
	C: General (6)	0.667
	D: Weak (0)	-
Professional Competence (Subjective)	A: Very Good (3)	0.668
	B: Good (11)	0.636
	C: General (8)	0.625
	D: Weak (1)	0.000
Activity Degree	A: Active (8)	0.500
	B: General (11)	0.727
	C: Not Active (4)	0.500
Learning Style	A: Reception Learning (8)	0.625
	B: Independent Study (10)	0.700
	C: Cooperative Learning (5)	0.400

.

It is clear that this system might be more helpful for undergraduates, students whose learning activity level was B and students whose activity degree was B, and objective professional competence is B or C. It is probable that these students lacked in paper research and reading. It was of limited help for students presenting high professional competence and activity levels, but the learning effect of all students was a positive integer, suggesting that this learning style, as an auxiliary learning style of traditional education, could not get negative impact on studies. One problem is that the sample size is small and may lead to statistical limitations. In future work, more experiments will be done to analyze the learning effect.

Attention was mainly paid to the eight students who thought poorly of this system, who held negative opinions on the subjective issue, and two main reasons were extracted from their evaluation. One was the limitation of technology: three master’s students and one undergraduate student whose learning ability was A mentioned more or less that the system functions were insufficient to meet their learning demands, that this system was still simple, and more and abundant data analysis and recommendation functions needed to be applied. In future interviews, most of them indicated that the concept of the system and learning style was worthy of being confirmed, and that they would support it if the technology was improved. Another four undergraduates with levels of B and C in professional competence mentioned that papers were difficult and that they did not like reading papers, of them two preferred reception learning, and indicated that they did not have the habit of searching for learning materials, especially scientific publications, while one student preferred independent learning, and indicated that he only loved reading books and Internet discussions; all these students showed that they had not encountered conditions of searching for papers to solve problems at present, and they did not like training in this form of learning, although the system was interesting.

It was concluded through the analysis that the learning method of reading papers proposed by the system was also attractive to scientific researchers and excellent students, but was still in the research and development period, and that functions should be further optimized. In addition, the opposition of many students may be greatly related to the fact that they have not been in contact with scientific research and papers. Independent research and the use of papers is ignored in university education, and more attention is paid to teachers’ lecturing of knowledge. Furthermore, students are not attached to problem-based education, and many undergraduates have not previously read scientific papers. Therefore, the system is compatible with traditional classroom education, and if traditional classroom education could be transferred towards independent learning and research, the system would be able to play a greater role.

Question 2: Comparing with the traditional classroom learning, what changes happened on students’ learning behavior? What impacts does it have on students’ learning?

The process of problem-based learning is similar to that of scientific research, which is a series of processes to find and research problems in real life as well as conducting innovation and solving problems. According to Question 8 and 9 in the interview, students were asked to describe the process of learning and solving problems before and during the use of the system to research. The purpose was to analyze what changes took place in the learning process and the overall abilities of the students after using the system to further analyze the impact on students’ learning.

It was found that most students had similar learning steps before. When encountering a problem, they usually sought help from their knowledge base (people or physical knowledge base), and then summarized the knowledge they just learnt and tried to solve the problem by using the new knowledge. It was also found that students, especially undergraduates who have not been trained in scientific research, tended to choose faster methods, such as asking others directly or asking people on the Internet, and the second choice was to look up learning resources in and out of class. Most of the extra-curricular learning materials that students would like to read were authoritative books, public study notes obtained from the Internet, and videos of classes. There were only four undergraduates with the habit of looking up the thesis before studying. The students’ learning approach are summarized in

Table 9. Approaches of achieving knowledge.

Approach to Learning	Number of Students (Before)	Number of Students (After)
Help from other people	21	21
Class learning materials	16	16
Learning materials out of class	18	20
Scientific papers	9	15
No processing	7	4

.

After using this system—the exercise to read the thesis—it was seen that the number of students who were willing to search for a thesis increased from 9 to 15 in a month. After a month of study, they were willing to consult a thesis on their own, and also showed a confident attitude in solving problems. There was little change in other items, which indicated that the thesis study and the acquisition of traditional learning resources were not contradictory in the process of knowledge learning but added a new cognitive approach. It also promoted students’ enthusiasm to look up extracurricular resources and reduced the negative emotions of students that took no action when meeting problems.

The students’ reports of the research issue were also analyzed. Before the research was conducted in that month, they had never related entrepreneurship ecosystems and economics with sustainability. Their understanding of sustainability was limited to environmental protection based on their textbook in traditional classrooms. However, after researching a concrete issue using this system, their reports showed a more comprehensive and diversified understanding of sustainability, moreover, more students tried to combine environmental, economic, and social issues to give a reasonable answer, even though they never attended traditional courses on sustainability.

Furthermore, students also had to explain their thinking process during learning and writing the research report in this month. Most of them expressed that, compared with their traditional homework, it was more difficult. The former always had a standard answer, while the latter did not, so more time was needed to search related publications, analyze and combine knowledge from many different subjects and domains, and try to give suggestions and conclusions. They were aware that many issues in real life need to be researched and solved based on methods from different subjects and domains, so it was necessary to learn and analyze as part of holistic thinking. One-third of the students described that the system could eliminate their fearful feelings of learning theses to some extent. The functions of analysis and visualization were useful for them to understand knowledge structure and the state of art for this domain. Students also showed more confidence in their information collection and problem-solving capabilities.

This system played a positive role in sustainable education: the diversified retrieval, analysis, and visualization functions based on knowledge graphs helped students understand sustainability in an interdisciplinary view; the process of learning and research based on scientific publications promoted their comprehensive abilities. After this case study, students showed more understanding of sustainability in an interdisciplinary view, they were more inclined to acquire knowledge from scientific publications, and most of them could use abstract knowledge to solve problems of the entrepreneurial ecosystem. However, it should be noted that although the students’ tendency to use theses were increased, their interests in reading theses were not obvious, which was probably because short-term training could not change the students’ ability and interest in reading theses. It also showed that this system needs to be integrated with traditional classroom education. In the traditional classroom education, if teachers can give more guidance and related activities to encourage students to query and learn scientific publications that enlighten a problem-based thinking, the learning effect would be better. Additionally, according to the suggestions from students, the technologies of analysis and recommendation should be further improved and support more complex applications.

5. Conclusions

In the twenty-first century, humankind has entered the information and scientific age, while students still often depend on traditional classroom education and knowledge from textbooks. To encourage undergraduate students and non-researchers to search and take advantage of scientific publications, this paper proposed a scientific knowledge management model based on the knowledge graph and data analysis, which combined scientific entities and concepts and provided many kinds of services for students. In this paper, a use case for an entrepreneurial ecosystem scientific management system was built and used by students at the College of Management. Compared with other works, this work combined scientific entity retrieval, analysis, and recommendations based on the knowledge graph, and provided more useful details of paper content to help students retrieve and choose sources. It is aimed at reducing the retrieval and learning difficulty of scientific papers for undergraduate students and non-researchers.

The contributions of this paper to sustainable education are as follows. First, systems thinking and an interdisciplinary approach are essential for sustainable development and sustainability education. Fulfilling this requirement requires the connection of a large number of traditional disciplines and the implementation of a more holistic way of thinking. This paper realized an academic entity network based on knowledge graphs, which could analyze and display scientific entities, in particular, research topics, knowledge, and their relationships. It was helpful for students to search, learn, and understand knowledge about sustainability in a holistic way of thinking. Second, this paper imported scientific papers into an education resource and focused on helping students access scientific knowledge efficiently and effectively to assist in investigating issues, solving problems, and decision-making. Reading and learning problem-based research achievements could improve their awareness, critical thinking, and skills to deal with the issue in the real world. The results of the case study showed that the analysis and visualization functions displayed interdisciplinary knowledge in a comprehensive, intuitional, and interesting way to help students understand knowledge structures, and students’ tendency to use scientific publications were increased. Additionally, students showed more comprehensive understanding and interdisciplinary thinking of sustainability; they not only learnt more basic knowledge of sustainability, but also utilized knowledge in a real research issue of the entrepreneurial ecosystem.

There are some deficiencies in this paper and improvements will be realized in future work. First, the system should be regarded as supplementary for university classroom sustainable education, in the future, more teacher work should be designed to combine problem-based learning and the understanding of scientific publications. Teachers should be retrained in the use of technology to create and develop different activities for learners. Second, the scale of the knowledge graph should be expanded to cover all research domains between the environmental, economic, and social issues regarding sustainability. Finally, some technologies should be improved: more data-mining algorithms should be researched to provide more services of data analysis and recommendation functions, and this work intends to add a domain ontology to the knowledge graph and to realize future semantic retrieval. In the future, it will be published online to provide retrieval, analysis, and visualization services to researchers and students for sustainability or other domain learning and researching.