Detection of the Perceptual Gap between Experts and Public for the Successful Implementation of New Policies: Evidence from the Educational Policy Development

Abstract

Understanding the difference between the viewpoints of experts and the public and communicating are very important processes to prevent wasteful conflicts and achieve successful implementation of new policies. This study compares the structures of perceptions representing the views of experts and the public as subjects in research papers and newspaper articles of national education policy using semantic network analysis and overlay mapping. Data are collected from the Korean education system, where it is convenient to collect national-level educational policy data. The results show the group of education experts showed great interest in the core concepts of the revised curriculum and its implementation, focusing on abstract theories, whereas the public showed greater interest in practical problems and the outcomes of the revision rather than the process of revision of the curriculum. It clearly highlights the differences in and causes of the reactions of experts and the views of the public to newly developed national education policies, thereby presenting implications for the successful design of important national programs that include education policies for the future.

1. Introduction

The argument that there is a vast difference in the risk perceptions of experts and the public about the introduction and change of new policies or technologies is accepted as common knowledge. Although experts objectively evaluate and judge the impact of new changes, there is a widespread perception that the public is gripped with emotionally irrational concerns about these changes [1]. Furthermore, the “information gap” between experts’ evaluation of changes and the public’s perception of the same has been often regarded as a key mediating factor in social and political conflicts triggered by new policies or technologies in various fields, and the resolution of these differences is crucial to the successful introduction of them [2,3,4].

In general, in the process of change, for instance, reflected in the development of new and important policies of the country, the determination of social risk and change is highly dependent on expertise. Thus, the process of policy creation and development tends to be led by experts [5], and individuals demanding for policy are relatively excluded. Regarding this phenomenon, Gilbert and Terrel [6] argue that it is necessary to minimize the errors attributed to the group of experts that excludes major users of policy by reflecting the public’s point of view during policy development. As a policy consumer, the public has a more accurate understanding of the actual situation due to the comprehensive interaction of psychological, social, institutional, and cultural aspects [7,8]. Therefore, efforts to narrow the gap by identifying the level of awareness and perspectives of the public pertaining to key policies under development in advance, compared to incorporating opinions of experts, can be the basis of successful policy [9,10].

Among various national policies, education policy receives a lot of attention from the public. In the case of the revision of educational policy, not only education authorities, who are the main actors, and schools and teachers, who are in charge of implementation, but also diverse groups with different viewpoints and understanding express their respective positions in the formulation and application of education policy. In general, to formulate policies at the national level, the general outline and the subject’s curriculum are developed through basic research by a group of experts. Therefore, experts’ perspectives have a great influence in determining the direction and content of education policy and analyzing the intellectual structure of their research is an essential task for nurturing insight into successful policy. In addition, the views of the general public, including students and parents, as consumers of educational policies, as well as expert groups, can determine the success or failure of an established education policy. The public’s point of view can be formed and communicated through mass media such as the press. Among them, newspapers are the oldest and most common mass media and have the power to reflect and shape public opinion. In academia, research that analyzes newspaper articles related to a specific issue is often done to understand the trend of public opinion on the issue and to explore the social implications.

To achieve the goal of this study, various perceptions of experts and the public were compared and analyzed with respect to the development of national curriculum in Korea. Semantic Network Analysis (SNA) based on text-mining was used to compare the semantic relationship of opinions of experts and the public on the development of national education policy. The semantic network analysis method [11] refers to visualizing and interpreting the overall meaning in the form of a network by grasping the structural relationship between words. Linking words that appear together in a document with a line (link) to visualize the relationship between words in the form of a semantic network makes it possible to identify features that are not revealed by themselves; thus, it is useful for structural analysis of specific topics. In addition, compared to the existing research methods based on qualitative analysis and survey analysis, the application of the SNA technique, an automated analysis technique, has the advantage that it can be extended to a research method that can effectively detect differences in real-time in the future.

This study compares the perceptions of experts and the public during the development of major education policies in a country and attempts to analyze, in depth, the causes of the differences. This analysis is expected to increase the possibility of successfully developing an education policy by checking, in advance, various perceptual differences that may occur in the development of national education policies and addressing risks. The policy pertaining to the national curriculum of the Republic of Korea, which is the subject of research, functions in a centralized national-level curricular system. Therefore, the contents and methods of implementation of the national-level curriculum, operational support, and evaluation directly affect the school curriculum. However, not only the Ministry of Education, which is the main developmental agent, and the school, which is in charge of implementation, but also diverse groups with different viewpoints and interests put forth their respective arguments. In addition to this context, it can be said that the revision of the Korean curriculum has a higher impact on school education than ordinary times in that it requires a new basic framework of education by emphasizing classes and evaluations centered on competency and achievement standards. Thus, it is appropriate as a target for the analysis of policy change, which is the subject of this study.

To sum up, detecting and attenuating the gap between the viewpoints of experts and the public are very important processes to prevent wasteful conflicts and achieve successful implementation of new policies. This study tries to detect the structural gap of perceptions representing the views of experts and the public as subjects in research papers and newspaper articles of a target new policy using semantic network analysis and overlay mapping. The result highlights the differences in and causes of the reactions of experts and the views of the public to newly developed national education policies, thereby presenting implications for the successful design of important national policies that include education policies for the future.

2. Reform of the National Curriculum of Korea in 2015

Korea’s education system has grown into a strong national curricular system since 1945. When the central government develops and announces a new curriculum, provincial offices of education provide administrative and financial support, and teachers teach students in schools with textbooks based on this curriculum [12]. The national curriculum serves as a common and general standard for the school curriculum to be operated by in elementary and secondary schools [13,14].

The revision of the national curriculum in Korea has been carried out in a way that responds to national and social changes, the needs of students, the opinions of subject-matter experts, and resolves problems in the implementation of the existing curriculum. With the advent of the knowledge information society of the 21st century, the most recently revised curriculum of 2015, in response to the demand for nurturing creative talents through school education, was announced in October 2013, and preparations for reorganization began in earnest. Thereafter, it was formalized and announced in September 2015, and has been applied to schools since 2017. The major directives of the revised curriculum are: the cultivation of basic knowledge in the humanities, social sciences, and science technology; the development of a curriculum that nurtures the dreams and talents of students; the development of a curriculum that can cultivate the competencies required by the futuristic society; and the overall improvement of education policy in connection with the curriculum. The revised curriculum (draft) emphasizes the development of learners more than ever by focusing on the cultivation of basic knowledge, the development of students’ dreams and talents, and the nurturing of competencies [15].

On 27 August 2013, the basic draft of the revised curriculum focused on “integration of humanities and science” to simplify the process of university admissions, and public hearings and meetings to gather public opinion by region were held. This was followed by the “Major Matters of the Liberal Arts and Science Integrated Curriculum” being announced, and the curriculum for each subject was immediately developed. After two public hearings and deliberation, the “Major Matters of the General Outline (draft)” was passed and announced on 23 September 2015. From 24 September 2015, until March 2017, when the revised curriculum was first applied to schools, textbooks, guidance for teachers, pedagogical methods, and methods of evaluation were developed based on the general outline. During this process, the Ministry of Education of Korea developed new educational policies and systems such as “Advanced Schools”, “Free Semester System”, “Improvement of Teacher Training Course”, and “Digital Textbooks for Elementary and Secondary Schools”.

3. Data Collection and Scope

A total of 248 articles were extracted through a search engine, Google (http://news.google.com (accessed on 23 May 2021)), and the news content of articles of renowned newspapers were analyzed to understand the structure of public perception. The subject of analysis was articles related to the relevant education policy from October 2013 to February 2017. The “2015 Revised Curriculum” and “2015 Integrated Curriculum for Humanities and Science” were selected as keywords for searching for relevant material. For the analysis of the structure of experts’ knowledge, 308 papers published in journals registered in the Research Information Sharing Service (RISS) of Korea were collected. To extract the articles related to the 2015 revised curriculum, the keywords selected were “2015 revised curriculum”, “2015 revision”, “Common Curriculum for Humanities/Sciences”, and “Liberal Arts/Science common”. The period of analysis was set from August 2013, when the initial draft was announced, to February 2017, when the revised curriculum was implemented.

The distribution of the number of research papers and newspaper articles by year is shown in Figure 1. From 2013, when the plan of change in policy was announced, the number of research papers attracting experts’ interest steadily increased, whereas the number of newspaper articles drawing the public’s interest was relatively small. Importantly, the public’s interest has remained relatively unchanged. However, although the number of newspaper articles sharply increased when the policy was applied thereafter, the attention of experts did not increase significantly. This means that the attention of experts was mainly focused on the developmental process of the new policy, whereas the public, its target consumer, had greater interest in the application of the policy itself than in its development.

4. Methodology

4.1. Text Data Refinement and Keyword Extraction

The process of refining data was applied to the collected data to enable natural language processing (NLP) for semantic network analysis. NLP refers to the technology of artificial intelligence that deals with the understanding, generation, and analysis of human language using computers. The first step adopted in this study was data pre-processing. In this step, the R programming language (R) was used to separate paragraphs, exclude meaningless characters and special characters, and deal with compound nouns. R is a software environment for statistical computation and graphics. It is widely used for statistical software development and data analysis because of the easy package development it affords.

For extracting refined text data, morphemes were classified, and target stems were automatically extracted using the stemming analyzer of R, based on its natural language processing library (KoNLP). The extracted stem and morpheme were compared with the previously classified morphemes and reviewed, and letter-level errors of morpheme analysis (singular/plural, abbreviation, spacing, part-of-speech, etc.) were automatically identified; subsequently, morphological analysis errors and stemming errors were corrected through the experts’ review. After the error stem was corrected, the case of high similarity between stems was automatically identified, and the experts directly determined the semantic similarity of the stems to select and correct similar stems. Finally, the final target keywords were selected by excluding stems that had no meaning in the context of this analysis (e.g., stems of less than two letters, stems that are too general, and stems that are meaningless in the context of this analysis) and stems that appeared less frequently (less than 150 times).

4.2. Semantic Network Analysis and Overlay Mapping

For semantic network analysis (SNA), first, the qualitative relationship between keywords is quantitatively changed. To this end, this study utilized a co-occurrence matrix (Co-occurrence/Co-word Matrix), which is one of the techniques used to identify the relationship between keywords in keyword-based semantic network analysis for unstructured text analysis. Co-occurrence refers to the significant existence of objects (people, words, etc.) within the same range/region based on a specific criterion (unit of analysis, sentence, document, etc.). In text-based network analysis, this co-occurrence could be interpreted as a meaningful relationship between objects [16,17]. In this study, the criteria and scope of co-occurrence were defined as abstracts of research papers and newspaper articles. VOSviewer (https://www.vosviewer.com/ (accessed on 10 August 2021)), a distance-based network visualization tool (Distance-based maps), was used for mapping and clustering. The VOSviewer presents mapping results in the form of network maps and density maps. In the network map, the distance between two keywords indicates the strength of the relationship between the two keywords, that is, the degree of similarity [18,19]. Therefore, the higher the similarity between the two keywords, the shorter the distance between the two keywords [20]. The mapping result for this study was displayed by classifying the clusters of the network according to the semantic relevance through the VOSviewer. In this case, the popular modularity-based clustering technique was deployed [21].

In this study, the Overlay Mapping technique was introduced as a technique to compare and analyze the mapping and clustering results of several comparison groups. Overlay mapping is a method proposed by [22,23] to uniformly compare the results of semantic network analysis in various situations and contexts. The network structure is not easy to compare because the target node and the visibility and location are not constant in various networks. To overcome this, a basemap that can be uniformly applied to the comparison of target networks is prepared in advance, and the target network to compare is visualized on this basemap frame. In general, the basic map is also called a global map because it is produced based on the visualization result of a network including all nodes to be compared. When the basic map is completed, the nodes constituting it and the position of each node are determined, and the nodes of each network are placed at the predetermined positions, with only the node value and connection relationship changing for each network. The structure of the network can be compared and analyzed based on the same location in the network of all nodes of the target networks to compare.

5. Results

5.1. Result of Keyword Analysis

Important keywords were extracted to compare and understand the main contents of interest in the research papers and newspaper articles on curriculum revision policy.

Table 1. Comparison of Core Keywords: Researcher Paper vs. Newspaper (Based on Total Weight).

Rank	Research Paper	Newspaper
1	curriculum revision	textbook
2	competency	math course
3	domain	Science
4	curriculum education	history course
5	Chinese letter education	course
6	achievement	integrated
7	Instructor	site
8	method	apply
9	evaluation	evaluation
10	course	elementary school
11	core content	announcement
12	elementary school	nationalization
13	culture	convergence education
14	textbook	middle School
15	process focus	Korean SAT
16	integrated education	creative
17	understand	society
18	humanistic education	core
19	knowledge	high school
20	creative convergence	problem

shows the 20 most frequently mentioned keywords. Research papers frequently included keywords related to essential concepts of education such as “curriculum revision”, “competency”, “curriculum education”, “achievement”, “instructor”, “method”, “evaluation”, “core content”, “process focus”, “integrated education”, and “creative convergence”, while newspaper articles included keywords pertaining to relatively direct and realistic educational issues, such as “textbook”, “math course”, “science”, “course”, “site”, “nationalization”, “Korean SAT”, “elementary/middle/high school”, “society”, and “problem”.

In other words, experts’ interest and discussions were focused on abstract and higher-level concepts as revealed by an analysis of research papers with high levels of professional knowledge, whereas news that reflected public interest was relatively low on intellectual content and professional knowledge, and discussions were centered on practical problems. The peculiar aspect is that the keywords “history course” and “nationalization” were related to important social/political issues in Korea at the time; they are thus frequently mentioned in newspaper articles, but they do not appear in research papers. This indicates that although the target data included socio-political issues, the interest of experts did not always consider non-educational factors such as socio-political issues.

5.2. Semantic Cluster Configuration

In this study, the structure of perceptions and points of view of experts and the public about the development process of curriculum revision was investigated through a semantic network and cluster analysis by calculating the co-occurrence of words appearing in these perceptions. As a result of the analysis of keywords selected from research papers and newspapers representing the perceptions and viewpoints of experts and the public, respectively, three major semantic clusters were found as demonstrated in Figure 2.

Table 2. Main Keywords of Three Semantic Clusters.

Topic (Number of Keywords)	Keywords
A Consequences and Impact of Curriculum Revision (118)	textbook, history course, middle School, society, high school, reinforce, math course, Science, Necessity, Information course, integrated, apply, convergence education, safety course, announcement, experience activities, nationalization, Korean SAT, creative, core, problem, elective courses, common, college entrance, talented person, university, adopt, common subjects, propel, recipe, suggestion, burden, science and humanities Integration, fostering, SW education, operation, private tutoring, certification, comment, research, draft, confirmation, core concept, integrated science, integrated social studies, English course, writing, career education, plan, training, science and engineering, reflection, notice, standard, Review, learning volume, participation, enlargement, future, Korean education, Chinese letter usage, style, implement, emphasis, opposition, expected, math abandoned student, necessary, goal, organization, important, newly established, concept, English, progress, exam, public hearing, explanation, reorganization, Korean SAT reformation, modification, amendment, opinion, preparation, Declaration, Comprehensive Student Admission, downsize, free semester system, admission, concern, development, confusion, argument, completion of course, forum, compilation, committee, reduction, drawback, absolute criteria, learning burden, number of lectures, democratic citizenship education, advanced, establishing, literacy education, research team, formation, Happiness, calculus, physical exercise course, full revision, publishing, hastily, multicultural Education, early admission, environment, alternatives
B Curriculum Development to Improve Learning Experience (77)	curriculum revision, competency, domain, curriculum education, achievement, evaluation, elementary school, core content, process focus, knowledge, integrated education, humanistic education, Korean language course, creative convergence, academic ability, technology course, diversification, core competencies, demand, Connectivity, school, art course, music course, concept understanding, change, systematization, activity focus, general overview, practical course, social studies course, learner, subsistence, teacher, traditional music education, recognition, fit, home Economics course, value relationship, courses, support system, Improving, student, science course, talent nurturing, agricultural education, level, Execution, nation, ability to think, perspective, ethics subject, experience, reconstruction, special education, dietary education, design, assignment, sense of community, perform, curriculum Competency, regular assessment, education with robots, system of content, criticism, communication skills, instructional design, students, policy, grammar, religion, vocational education, ethics course, general school, computer education, particulars, grade, teacher training
C Strengthening the Basic Humanities Curriculum (16)	Chinese letter education, course, method, Instructor, understand, site, culture, configuration, subject learning, material, basic knowledge, discussion education, traditional culture, vocabulary, attitude, problem-solving ability

shows the keywords included in each major cluster and the associated topics that could cover all keywords belonging to each cluster and reflect previous studies. Examining the derived topics, Topic A was analyzed as “Consequences and Impact of Curriculum Revision”, Topic B as “Curriculum Development to Improve Learning Experience”, and Topic C as “Strengthening the Basic Humanities Curriculum”.

Topic A, “Consequences and Impact of Curriculum Revision”, included the meaning and necessity of the “2015 Revision Curriculum”, which was the core of the revised curriculum, and the results of the curriculum revision. Keywords such as “newly established” and “public hearing” related to the development process, those related to historical issues that were controversial in the early stages of development, and keywords related to college admissions, such as “history course”, “reformation Korean SAT”, “university”, and “textbook”, were frequently mentioned. Topic B “Curriculum Development to Improve Learning Experience” emphasized the practical improvement of the curriculum with a view to enhance the learning experience rather than the organization and reorganization of the general outline with the vision of the revised curriculum “improving the quality of the learning experience”. Moreover, keywords related to a learning experience, such as “competence”, “core content”, “concept understanding”, and “process focus”, were included. In addition, practical subjects in elementary and middle schools such as “technology course”, “computer education”, and “Korean language course”, which have become emphasized since the revised curriculum, were included. Topic C, “Strengthening the Basic Humanities Curriculum”, emphasized basic knowledge as a common learning experience for all students, as presented in the revised curriculum, and activated a variety of student-participatory classes suitable for the subject, making learning fun. Keywords, such as “Chinese letter education”, “basic knowledge”, “discussion education”, “traditional culture”, and “subject learning”, were included.

5.3. Result of Semantic Network Analysis and Overlay Mapping

In this study, the opinions of experts and the public on curriculum revision were mapped, clustered, and visualized as a network map to analyze the intellectual structure of these opinions, and a basemap was established to compare and analyze the intellectual structure of the two groups. During the process, the node properties (color and size) were analyzed to be proportional to the number of appearances of newspapers and research papers and the strength of the connection between keywords. Therefore, it was possible to compare the size of nodes according to the number of appearances of keywords and the strength of their connections in newspapers and research papers. However, as there was a difference in the number of newspapers and research papers, the relative distribution of keyword appearances was reflected.

Figure 3 shows the structure of public perception articulated in newspaper articles. The most emphasized area in the network map of the structure of expressions of public awareness was Topic A grouped around “textbook” and “field”. After announcing the revised curriculum, the public’s interest in the impact on school sites and students was high, and public attention focused on urging changes in classroom instruction and evaluation before applying the new curriculum. Therefore, it means that keywords such as “integration”, “free semester system”, and “SW education”, which were factors that directly affected curriculum revision in the field, played an important role in the structure of public perception. Additionally, keywords related to the university admission system centered on “university admission”, “Korea SAT”, and “private education” occupied a significant proportion of Topic A. Topics B and C, centered on “class” and “evaluation” and formed a wide area in the map but had a relatively low weight in the structure of public perception. Therefore, the public paid more attention than the experts to the issues related to student participation and practical issues being strengthened through the revision of the new curriculum.

Figure 4 shows the intellectual perspective of experts. In Figure 4, the most remarkable issue is Topic B, “Curriculum Development to Improve Learning Experience” clustered around “Curriculum Revision”, “Competence”, “Curriculum Education”, and “Core Content”. This result means that after the announcement of the revised curriculum, studies on the development of the actual curriculum based on the revision rather than the research related to the revision itself have been active. In addition, after the announcement of the revised curriculum, various themes of study to build on the principles of curriculum development, as well as the learning process and methods to improve the quality of learning experiences that emerged while developing the general outline of the curriculum and the subject’s curriculum. Studies in Topic C, which develop and apply learner-centered participatory classes that can cultivate convergence thinking skills by using various methods such as project and problem-centered learning, according to the characteristics of the subject, also represented a relatively large proportion. Furthermore, studies related to reflection and improvement tasks in the process of curriculum development were conducted.

However, interest in research topics related to Topic A was found to be the least. This is in stark contrast to the results that showed curriculum revision receiving the greatest public attention in the previous analysis. This means that the intellectual structure of experts is biased toward the topic areas of B and C, and the strength of the connection between the two topics is much greater than that of Topic A, which garnered intense public attention. Therefore, it can be inferred that there is a considerable difference and gap between the perceptions of the public and the intellectual activities of experts in curriculum revision and development.

6. Conclusions and Implications

This study compares and analyzes the perspectives of experts and the public in the process of the revision of curriculum of Korea. Research papers by experts and newspaper articles that appropriately reflect public perspective were analyzed and compared by applying the semantic network analysis technique. Through this analysis, we presented implications for the successful development of education policy by assessing the differences in the reaction and formation of experts’ perspectives on and public perceptions of new education policies and suggested ways to resolve them. Consequently, it was possible to clarify the difference in the scope of understanding between education experts and the public about the revised curriculum. The group of education experts showed great interest in the core concepts of the revised curriculum and its implementation, focusing on abstract theories, whereas the public showed greater interest in practical problems and the outcomes of the revision rather than the process of revision of the curriculum.

The most emphasized features of the revised curriculum included creative convergence, competency, and integration of liberal arts and science. In the area of interest of education experts, these characteristics were reflected as they were, and interest in developing a curriculum that takes advantage of the purpose of the revised curriculum was highlighted [14], whereas “mathematics”, “nationalization of history textbooks”, and “SW education” garnered a high level of interest among the public in issues related to grades and entrance exams. These perspectives are directly encountered in school sites, rather than interest in the core issues of the revision of the curriculum [13]. In other words, although education experts understand the revised curriculum in the context of development-application-improvement, the public tends to understand it piecemeal, focusing on application. This indicates that there is a problem with the role of the media on key issues of curriculum policy reform. Although it is natural for the media to reflect public perceptions and public opinion, it is sometimes necessary to highlight the main characteristics or purposes of major educational policies as they are, such as the revised curriculum, and for educational experts to shape public opinion through the media.

It is very important to attenuate the conflict attributed to the group of experts that excludes major users of policy by understanding the difference between the viewpoints of experts and the public and communication [6,7,8]. This study sheds light on the way to prevent wasteful conflicts and achieve successful implementation of new policies. If the framework of analysis proposed in this study is used to analyze the difference in perception and knowledge structures between stakeholder groups in the policy development process, it will be possible to understand the differences in perceptions between various stakeholders, as well as to resolve conflicts and problems based on understanding the differences. In this way, the proposed framework can contribute to successful policy development.

Among the limitations of this study, there are still difficulties surrounding identifying the subjective keywords. As defining and classifying the keyword is crucial to achieving the validity of the research, more objective and advanced semantic techniques, such as deep learning for natural language processing, can be deployed in future research. Although this study found a difference in perceptions of experts and the public in the process of developing an important education policy and inferred its cause, the effect of these differences and their relationship have not yet been studied. Therefore, further studies are needed to analyze differences in perceptions among key stakeholders of policies in the future and the relationship between their characteristics and performance. Finally, it can be extended to compare the perceptional gap among all the key players in the specific situation, such as experts, decision-makers, and policy consumers, to understand their successful interactions.