Information System for Monitoring and Managing the Quality of Educational Programs

Abstract

The article explores the issues of the objective approach to managing the educational process allowing to effectively update and monitor it. The study is aimed at organizing the processes of control and management of the quality of educational programs with reduced time costs. The authors have researched the processes of methodological support of the educational process and the corresponding documents of an educational organization. An information system for monitoring and managing the quality of educational programs has been developed. The authors have developed models of the problem area, allowing one to determine the bottlenecks in organizing education quality control. The results of the study are meant for the internal audit of educational services and carrying out the procedures of public control, accreditation, or licensing of an educational organization. They allow for the quantitative evaluation of the educational program content and unambiguous interpretation of the obtained result. For this, the authors have developed an objective system for assessing the quality of educational programs. The assessment includes a set of objective indicators with a certain significance: for example, compliance with certain standards, the use of relevant peer-reviewed literature, agreed forms of education, a point-rating system, etc. All criteria are divided into blocks, each of which has a corresponding conformity assessment system. The developed assessment scale unifies the quality monitoring procedure, reduces subjective evaluations, and ensures less time-consuming monitoring of the quality of an educational program at the different stages of its life cycle.

1. Introduction

The basis for the modernization of education in the Russian Federation is formed by ensuring the quality of educational services in accordance with the needs, requests, and expectations of the state, society, and the individual [1]. The importance of the quality of education is determined at the state, regional, and municipal levels, as well as at the level of the organization implementing multiple educational programs. Solving the strategic and operational tasks of managing the quality of education requires objective information on the resources, processes, and results of the functioning of an educational organization or its educational directions [2]. In turn, it requires creating and updating the system of monitoring and evaluation of the quality of educational services [3]. The process of creating a system of monitoring and evaluation of the quality of educational services raises the problem of developing algorithms, methods, and the corresponding means of information support [4].

The quality of an educational program includes a list of requirements for the educational content considering the achievement of scientific and technological progress and international standards, request from a specific consumer and/or customer (due to competition in the labor market, the education system must be mobile and dynamic), and the results of internal and external monitoring [5]. A quality assessment system should have a quantitative assessment that allows one to determine the significance of each of the quality parameters, the level of their influence, and the degree of dependence on each other.

The present study demonstrates theoretical significance since it identifies the main elements of the regulatory legal component of the educational process and its structure and presents an integrated system for assessing the quality of the content of an educational program considering various legal and other requirements. The above-mentioned ideas allow for forming a theoretical base for developments in educational process automatization in the spheres of preparing recommendations for students, preparing reporting documents from the teaching staff, and unifying the requirements for internal documentation of various educational organizations.

The practical significance of the study lies in the expeditiousness and objectivity of monitoring educational programs, the reduction of labor expenses for its monitoring and the unification of its content, and the organization of unified information space of the educational organization with a decreased information redundancy [6].

The hypothesis of the study states that the quality of any educational program presents a quantitative value obtained as a result of objective evaluation.

2. Literature Review

Russian and foreign experience in research in the field of education quality management comes down to the following aspects:

1. The development of education management technologies and mechanisms. A comprehensive analysis of the state of processes is characterized by a variety of applied research methods [5]. As noted by Tsvelik, the resulting methods do not imply their widespread use in the development of process automatization algorithms [7]. This is confirmed by Zang and is related to the lack of an unambiguous result of the analysis of educational organization processes and a generally recognized system of performance indicators for both the educational organization and its structural units [8].

2. The identification of education management problems. As noted by Mikhaleva and Tsvelik, research in this area is typically short-term as in Russia an active reformation of the education system is taking place [7,9]. Researchers note the need for ensuring the effective functioning of an educational organization, the quality of educational services, the competitiveness of education, and the development of science in general [9,10]. Thus, Liamin points to problems in decision-making associated with the contradictory nature of educational services which in essence present inherently socially significant benefits [10]. Management requires taking into account not only the economic development factors but also the need to fulfill social obligations [7].

3. The development of systems for educational results quality evaluation. The main conclusions of such studies is that modern education is going through an evolutionary transition from the “knowledge” paradigm to the “competency” paradigm [4,11]. Stain, Atkinson, and Al-Gabri state that this transition requires substantial changes both in the methods of educational activity management and the system of evaluation of the results of students. In most cases, they are examined from the point of psychology and pedagogics [3,12,13].

The research aimed at studying and developing unified approaches, methods, and algorithms for assessing the level of formation of competencies throughout the training is especially worth noting. Such works have a technical profile and only rely on the results obtained in pedagogical studies to a small extent. They evaluate the volume of information being processed and substantiate the need for developing automated or information systems. In this respect, they propose certain methods and mathematical models for evaluating students’ marks obtained as a result of an academic period [14,15,16].

The developed models (for example, by de Mello, Pedro, Gevorkian, Savenkov, Levitski, Narikbayeva, and Chirtsov) systematize the indicators allowing to determine the level of formation of competencies (or knowledge or skills) in accordance with the developed assessment scale [17,18,19]. The acquired models are used for the implementation of algorithms in the software of information or automated systems.

4. The substantiation of problems of educational organization management are examined from the point of determining the relation between the state and social components [20,21,22,23]. The authors of such studies develop the models of education management organization and the methods for assessing the effectiveness of various models and their positioning in relation to one another and create new models of educational organizations. However, the results of these studies do not make it possible to identify effective management models and the conditions of their possible distribution [4,10,24].

Studies focusing on the problems of education management emphasize the development of engineering education. According to these studies, modern engineering education in Russia does not correspond to the dynamically evolving conditions of the socio-economic and professional environment. The main proposition of such studies is the formation of the corresponding information and pedagogical environment combining the technologies and principles like automated learning automated training systems, automated complexes, laboratory research facilities, computer technologies, and educational resources [25,26,27].

Research in this area is aimed at the implementation of information processes in a way that would enable the proper organization of labor resources and the optimization of network resources and the resources of the material and technical support of the educational process in the process of organizing various forms of learning (including distance learning) [11,28,29]. Moreover, the organization of labor resources should be carried out in such a way as to obtain the best results possible to attract financing for an educational organization from various sources.

5. The identification of specific characteristics of educational process support and the formation and evaluation of competencies in the students of an educational organization. For example, researchers believe that the formation of students’ competencies requires the educational process to be practice-oriented [30,31]. Scientific studies develop a method of structuring educational material for improving the practice-oriented component of learning. Such works typically lie in the field of pedagogics and are mostly theoretical. These studies result in methods allowing one to organize the educational process or the educational material in such a way as to achieve the goal of its mastery by the student. However, the technical characteristics required for the realization of the acquired study results are not taken into account. The authors note that educational standards and, therefore, educational programs have a weak practical focus. Industry employers do not possess technologies that allow developing requirements for the level of formation and content of competencies in educational programs [31,32,33,34]. This allows us to talk about practice-oriented education. Most often, educational organizations draw up curricula for several years in advance, without changing them for a specific period of student learning. This is a serious problem, since the modern educational market is dynamic and technology is changing rapidly. Thus, upon completion of the training, the specialist is not ready to perform the work assigned to them at the level that has formed in the industry. This situation is typical not only for most Russian but also foreign educational institutions. Burlea-Schiopoiu notes that the connection between curricula and the industry should be strengthened (based on the example of student accountants) [35,36].

Another part of scientific works focuses on the study of distance learning and the specific characteristics of developing electronic educational resources. Active attempts at developing electronic educational resources are made both in Russia and in foreign countries, although the accumulated experience and the obtained results are not systematized. Electronic learning tools duplicate the content of the traditional ones and most educational organizations do not have an idea of how to effectively implement the available resources in their activities [19,37].

The subjects of the educational activity (mostly teachers but also students) typically associate the automatization of said activity with the attempts to standardize the educational process, abandon the individual approach, and completely replace the teacher. This is the creation of the information support of the educational process. There are few software environments and shells aimed at creating interactive electronic educational resources for basic disciplines [38,39,40].

6. Increase in the demand for educational services and the demand for graduates of educational institutions. Researchers have developed the models and methods for predicting the demand for staffing. Some studies examine this issue from the point of recruitment of applicants accounting for the demand in the educational services market and the demand of the regional economy in the labor market [41,42,43]. Zang, Al-Gabri, and Işık note that each educational organization develops its methods of resolving this problem [8,13,44]. Research in this area is typically sociological or psychological. Nevertheless, the acquired practical results are formalized poorly and the responsible managers make the decisions on the realization of applicant attraction programs based on their experience rather than with the use of automated decision support tools [3,45,46].

Summing up the literature analysis, the experience acquired in the sphere of educational process organization has high theoretical and practical significance, but the individual educational process elements are viewed in it as independent objects isolated from the educational organization [47,48,49] or educational processes [50,51,52]. We conclude that the results of the wide variety of research in the sphere of education are autonomous and do not unite into one system. The integration of the obtained results allows creating a single system with consistent and relevant data, available both to scientists or experts for new research and to end users—all participants in the educational process.

3. Methods

3.1. Research Design and Hypothesis

The development of algorithms of the information system’s functioning requires determining the specific features of the work of specialists that execute the educational program quality control. As part of the study, we developed questionnaires and interviewed methodologists of educational organizations, department heads, experts in public control of the quality of education, experts in professional and public accreditation and independent assessment of qualifications, and researchers of the Institute for the Development of Education of the Russian Academy of Education [53]. Their number was eight, which corresponds to the ratio of the confidence indicators of the obtained result and the maximum permissible error of the standard deviation share before the start of the survey. Interviewing was carried out orally with recording in special forms of answers. This was done during the accreditation procedure of the educational organization so that the expert could not only answer the question but also demonstrate the actions they performed, as well as, if necessary, explain them.

Each specialist answered a set of questions determining important factors of the audit of the educational program content. Such questions included:

• How much time (on average) does it take you to evaluate one document?
• What elements of the document under evaluation are important (arrange in descending order: title page, references, competencies, assessment tools, conditions for implementation, distribution of the number of teaching hours, didactic units, etc.)?
• Do you consider the presented fragment of the educational program content a mistake? Explain your decision.

The answers obtained as a result of interviews make it possible to formalize the work of specialists executing the control of the educational program content. This stage of the study allowed us to identify:

• the stages of evaluation;
• the fragments of documents the specialists focus on;
• the time required to evaluate the separate sections of a document and the whole document;
• the importance of the elements of the educational program content;
• the composition of the reference material and the timing of its use.

This result allows for developing a universal algorithm of educational program evaluation for high-quality automatization of the largest number of stages of this work.

The presented method was important and labor-intensive. Preparing interview questions, developing survey content, choosing the time and place for working with experts, and processing and interpreting the acquired results were the steps fundamental to this method and its importance is difficult to overestimate. This allows one to understand the methodology for checking the content of educational programs, determine the significance of each element of its content, etc. As a result, we determined not only the algorithms of work but also tested the reliability and randomness of the acquired results, established the optimal and minimal number of experts that have to be recruited for the development of the criterion and evaluation complex of the educational program. This step was the one making all further stages of the study possible. The results are necessary for the implementation of the method of expert assessments, which makes it possible to obtain the weight values of each criterion for the assessment complex of the information system. The results are the basis for drawing up formal models presented in the Results section of this article.

3.2. Data Collection and Processing

The study uses the method of structural analysis to formalize aspects of the problem area. The main goal of structural analysis is the transformation of general, imprecise knowledge about the original problem area into exact models describing the subsystems of the modeled processes or objects. The implementation of this method allows to:

• identify all the subjects participating in the process of the audit of the educational program content;
• establish the key objects of processes;
• identify the relations between objects and processes;
• determine the methods and means of information system modeling.

The acquired results are presented in the form of graphic models:

(a)

Euler diagrams demonstrating the relations between the objects of the subject area. In the study, these diagrams allow us to illustrate the influence of documents and requests on the content of an educational program, the relations between the educational program elements, etc.

(b)

Directed graphs demonstrating the links between the key elements of an educational program. The presence of links between elements allows determining their subordination, isolation levels, and possible iterations of algorithms for evaluating the content of an educational program.

(c)

Event-driven process chains allowing us to visualize the sequence of events and actions, their participants, performers, and document flows. This type of model illustrates the specific characteristics of interactions between all process subjects in using an information system. Said subjects include information system modules, specialists of an educational organization, and files necessary to execute processes and obtained as a result of completing them.

(d)

Radar charts demonstrating the interdependence of numerous values. This approach is used for demonstrating the results of educational program content evaluation. The values marked in the coordinate system correspond to the assessments of the aspects of the educational program quality criteria. The values of the aspects are determined by the information system automatically according to the developed algorithms.

After structural analysis, we obtained an array of elements that define the key aspects of the content of any educational program. The interviewed experts were tasked to range these elements. To do this, they were asked to fill out questionnaires, where all objects are grouped in structural blocks by certain characteristics [54]. Thus, the method of expert assessments is implemented. The implementation of this method results in criterion an evaluation complex of an educational program consisting of criteria and their aspects. Each criterion and aspect has its significance value. The acquired evaluation results are tested for consistency across different experts [55] using Kendall’s coefficient of concordance (Formula (1)).

$$W=\frac{s}{\frac{1}{12}{m}^2\left(n^3-n\right)-m {\sum }_{i=1}^m{T}_i},$$

(1)

where W—coefficient of concordance of experts on the structural element in question;

S—the sum of squared deviations from the mean;

m—the number of experts;

n—the sample size (the number of items evaluated);

T_i—the number of ties (types of repeating elements) in the set of ranks for expert i.

To ensure that the obtained coefficient W is not random and the obtained result is reliable, its significance is tested [55] using Pearson’s chi-squared test χ² calculated by Formula (2).

$$X^2=\frac{S}{\frac{1}{12}mn\left(n+1\right)+\frac{1}{n-1}{\sum }_{i=1}^m{T}_i},$$

(2)

To determine the significance of experts’ consistency in assessing the aspects, we used significance level α = 0.05. The significance level used for determining the consistency in the assessment of criteria was α = 0.01.

Storage and processing of the received data and their use for the functioning of an information system require a database. The design of the database structure must preserve the balance between the performance of the information system and the integrity of the stored data. For this, methods of the theory of relational databases are used. The database is normalized based on the subject area and the technical capacities of potential clients. As a result of the implementation of methods for relational database design, an entity–relationship model (ER model) was obtained. The ER model allows us to realize all the capabilities of the information system implied by the goal and objectives of the study.

4. Results

The educational process is regulated by numerous federal and regional regulatory documents and the local documents of a specific educational organization. Figure 1 presents a list of documents affecting the content of an educational program.

This model reflects the results of a change in the approach to education in Russia that has occurred over the past five years. The main approach to training now is the use of practice-oriented methods meeting the employer’s needs. Therefore, a separate block should be allocated to documents, the content of which is determined by external organizations.

Any educational program must comply not only with an educational standard, which may not change over the years but also, for example, with professional standards that define the labor functions of modern professions and industry specialists. The constantly changing approaches and content of practice-oriented and demonstration exams according to World Skills standards oblige an educational organization to consider these facts in the content of implemented educational programs.

An educational program has a clear structure regardless of the educational organization or the educational direction. The conducted study allows us to establish the key elements of this structure and their interrelation. These results are presented in Figure 2.

If every element of an educational program is formed, it can be concluded that the educational program is compiled correctly. Thus, the basis of the criterion and evaluation complex is formed by the structural elements of the educational program (criteria). Each criterion comprises certain aspects.

The lists of aspects and possible mistakes in its content were obtained as a result of studying the results of the survey and interviews of specialists participating in the expert evaluation of the quality of the provided educational services. All possible errors in the educational program can be classified as follows:

• Higher-order mistakes (category 1) arising from non-compliance with the requirements of regulatory documents (educational and professional standards, laws on education, etc.).
• Middle-order mistakes (category 2) appearing in the text of an educational program due to non-compliance with the requirements of the documents of the educational organization (academic plans, competency matrices, etc.).
• Lower-order mistakes (category 3) emerging as a result of not following the educational program file layout template.
• Content mistakes (category 4)—mistakes made in the formation of didactic units (topics do not correspond to the academic discipline, the recommended literature does not allow studying the declared topics, etc.).
• Technical mistakes (category 5)—mistakes made by the typesetter when typing due to carelessness or low level of proficiency in the software.

Mistakes belonging to category 4 can be checked through the developed information system. The content of the topics of academic disciplines is attributed to the creative part of the educational program. For various reasons, every teacher can view the substantial part of the program in their own way. Such reasons can include:

• the teacher’s professional level;
• the material and technical base of the educational organization;
• the level of students’ training;
• lobbying the interests of the educational organization, industrial partners, or political actors.

Therefore, this part of an educational program has to be evaluated by a specialist. The evaluation of aspects of this criterion can be formed based on the conclusion of the head of the educational program, external reviewers, and the conclusion following the meeting of the department or faculty council.

The remaining categories of mistakes can be identified automatically without the support of a specialist. Once the file of the educational program (or its separate parts) is uploaded, the algorithms of the developed information system can autonomously conduct the assessment by the aspects of criteria implying the possibility of mistakes from categories 1–3 and category 5 and provide access to responsible specialists to check for errors from category 4. After the evaluation is completed, the system calculates indicators based on the assessment rules determined by the criterion and evaluation complex. A part of the criteria and their aspects is presented in

Table 1. A fragment of the results of expert evaluation of the significance of criteria of educational program quality evaluation.

№.	Criterion	Aspect	Value, %
			of the Aspect	of the Criterion
1	The passport of programs of educational disciplines is filled in correctly	Corresponds to the template	8.1	9.5
2		The type of professional activity is indicated correctly	6.4
3		Professional competencies are indicated correctly	17.3
4		General cultural and general professional competencies are indicated correctly	17.3
5		Knowledge is indicated correctly	11.4
6		Skills are indicated correctly	11.4
7		Practical experience is indicated correctly	11.4
8		The indicated number of academic hours corresponds to the academic plan	16.7
9	The reverse side of the title page is filled in	Corresponds to the educational standard	8.5	14.5
10		The educational program is approved	23.7
11		The educational program is confirmed	27.9
12		The educational program is authorized	18.3
13		The columns “Author/Compiler” are filled in correctly	12.8
14		The form completion corresponds to the template	8.8

.

The indicators presented in Table 1 were obtained using the expert assessment method based on the results obtained in the process of interviewing and questioning specialists participating in the examination of educational programs’ content.

The structure of such an information system is presented in general form in Figure 3.

The information system for monitoring the quality of educational programs comprises the following modules:

• Converter. Allows for converting file formats into the ones used by the corresponding system modules (for instance, a pdf file of an educational program and the list of criteria must be converted to XML).
• Import/Export. Input/output of the parts of files for the correct functioning of modules of the current system or external systems.
• Assessment of aspects/criteria. Determine all categories of mistakes in the formal part of the educational program or its parts.
• Control. Forms the list of experts for evaluating the categories of mistakes from the creative part of the educational program or its parts, provides access to assessment materials, and records the received expert answers.
• Monitoring. Allows us to adjust the parameters of scales for assessing the aspects, criteria, or the integral indicator, convert the primary result of the assessment of aspects and criteria into a result corresponding to a certain scale, and form a list of recommendations for corrections.
• Statistics. Records the acquired results and allows creating reports on the given parameters.
• Designer. Creates diagrams based on the given parameters or the created reports (an example of such a diagram is presented in Figure 4).
• “Monitoring” Database. Stores reference files, report files, and other data necessary for the functioning of the information system (an ER model of data is presented in Figure 5).

Figure 6 demonstrates a model of the process of interaction of a teacher with the information system to obtain the results of the assessment of the formal part of the operational program of discipline.

The obtained model of an information system allows one to exclude the human factor as much as possible when checking the content of the educational program. It is possible to quickly obtain a list of recommendations for correction and keep documents up to date. The resulting structure of the information system corresponds to the accepted hypothesis.

5. Discussion: Information System for Education Program, and Open Innovation

5.1. Discussion: Information System for Managing the Quality of Eudcational Program

Any educational organization has to ensure the interaction of three primary processes: obtaining resources, using them to achieve the goals of the educational organization, and transferring the results of work to the external environment. The system of education quality management has to take into account the pattern of interactions between various authorities the interests of which lie in the field of education [14,56]. The following characteristics are considered as the components of the quality of education: quality, motivation, and achievements of the teaching staff, the state of the material and technical base of an educational organization, the quality of educational programs, the quality of students’ knowledge, the composition of students, the introduction of innovations, the demand for the organization’s graduates in the labor market, the achievements of students and graduates, the innovative activity of the heads of an educational organization [14]. Thus, the quality of education can be viewed as a multifaceted concept and studying it requires the complex process approach.

The complex approach to studying the problem of the quality of education can significantly reduce the time and financial expenses of organizing the educational process and lower the degree to which the human factor affects the obtained results. The conducted studies do not demonstrate satisfactory results in determining the relations between the indicators of the state of educational programs, the educational process, and the organization of the work of the teaching staff. The existing methods are not oriented on operational management in improving the quality of education, organizing the self-control of the teaching staff, predicting the development of an educational direction, and the objective assessment of the obtained results [57,58].

The results attained in the present study have not only theoretical but also practical significance. The reliability of the conducted research and the validity of the acquired results are ensured by the correct use of research methods, the approbation of various aspects of the work at scientific and practical conferences, obtaining the certificates of state registration of a software product, and the implementation of the criterion and evaluation complex in several educational organizations [6,55]. The functioning of the developed information system involves phased execution of the operations the results of which can be corrected. The stages of the information system functioning include determining the composition of experts ↑, establishing the value of aspects and criteria ↑, obtaining an educational program for evaluation, evaluating the aspects and criteria, forming the recommendations and the final result, output of the final result, and notifying the specialists about the results of the assessment [6]. The arrow symbol (↑) marks the stages allowing corrections to be made. The presented approach meets the requirements of the quality loop of the educational process.

A significant difference can be detected when comparing the results of educational program evaluation conducted using the means of the developed information system and manually.

Studies in the area of statistical assessment of the quality of typing demonstrate that when processing text, there are always mistakes [59], the number of which a regular flow [37]. However the proofreading process does not completely prevent errors only reducing their number [60,61]. The number of mistakes depends on experience with both texts and text editors. Specialists, monitoring the content of educational programs are not specially trained in the sphere of publishing. Moreover, the volumes of the documentation being evaluated are estimated in hundreds of pages, requiring constantly referencing other documents or reference materials. Thus, high-quality monitoring of the educational program content either becomes highly time-consuming or becomes faster due to the adoption of the formal approach to the evaluation of individual documents.

The developed information system quickly finishes the evaluation task: the specialist only needs to upload the source file and wait for the result. However, it should be borne in mind that only the formal indicators of the educational program are assessed in the specified time, the evaluation of the creative part occurs simultaneously. Thus, the load is evenly distributed, increasing the time for classes and research and methodical work.

The modern studies conducted in the sphere of education allow practical implementation of the results presented in them only in the management of human resources, the assessment of the development of educational material, or the automatization of certain types of activities of structural divisions of educational organizations [40]. The obtained results focus on convenience, personality development, and students’ education from the point of psychology and pedagogics. Meanwhile, the developments aimed at the automatization of the work of the teaching staff or other specialists related to the methodical support of the educational process provide additional workload [22,30,38]. The acquired results can be applied to specific educational organizations in which the study is conducted. This conclusion is related to the fact that every educational organization has the right to regulate the content of the educational process independently. Moreover, even two educational organizations of the same region with the same direction of training will differ in the content of educational programs. If we examine educational organizations not only from the point of the content of educational programs but also from the point of the educational process, the differences become even more acute.

5.2. Discussion: Information System, and the Open Innovation in Education

The concept developed within the framework of the study provides an open innovative approach to education in terms of using the information system to determine the quality of educational programs. The developed information system provides the continuity of management and the consistency and interconnection of individual educational processes, as well as their interaction [62]. With an open innovative approach, educational organizations can interact with each other to develop common quality criteria, use targeted knowledge flows, and identify promising ideas or developments for further common use. The system of education quality management has to be integrated and focus on modern information technologies the implementation of which has to be assessed through both qualitative and quantitative indicators. Such indicators should be developed not only within educational organizations (or between them), but with the involvement of industrial partners. Industrial partners should participate in determining the development trajectory of any educational program, ensuring, among other things, culture of dynamics of open innovations in education.

Changes in the criteria or content of an educational program (including any educational processes) should occur only upon completion of the full training cycle. This makes it possible to assess the effectiveness of each process and determine its macrodynamics in the education system, pedagogy, and industry innovations, as well as the microdynamics of the processes themselves within the educational organization. The criterion and evaluation complex developed in the course of this study are “flexible” and easily adjustable, meeting the requirements of an open innovative approach. Moreover, the results acquired in different educational organizations will be relevant and can be used for integrated management of the education system not only at the level of a specific region but also across the country [3,58].

The developed information system for monitoring and control of the quality of educational programs takes into account both the specific characteristics of the educational organization and a wide variety of other external factors influencing the content of the educational program. This advantage is achieved through the use of information system modules. Studies demonstrate that to achieve effective management, obtain the desired result, and adapt the system to various kinds of external changes, it is required to use a modular structure of software tools. The same refers to educational organizations. As demonstrated by the studies in the field of informatization and automatization of education, individual modules of the developed information systems allow to determine the level of formation of students’ competencies, gather and process statistical data on their performance, determine the levels of knowledge or the effectiveness of the work of the teaching staff of the educational organization, plan lessons, identify the gaps in students’ knowledge, plan the load of independent work, improve the methods of teaching and testing students’ knowledge, etc. The resulting information system does not exclude the developed modules or distort the results obtained at the output of each information system. On the contrary, it serves as a means that would allow combining the developed information systems or their separate modules as much as possible to design a unified information system of the educational organization [14,31]. This makes it possible to ensure the availability of open innovation between higher education and any branch of the national economy (for example, industry, tourism, etc.).

As a result of the study, it was established that the educational process is complex and comprises a great number of interrelated objects, with data flows emerging between them, control actions, etc. The main object of the educational process is the educational program. Studies related to the educational process organization note that the educational program consists of heterogeneous information that needs to be systematized via special methods. In addition, the process of accumulating and updating said information has to be continuous [59,60]. This requirement is supported by the results of our study. Meanwhile, the structural links between the elements of the educational program and the documents allowing to carry out continuous monitoring of its content are established.

6. Conclusions

As a result of the research, we developed a universal model of the information system. It allows us to acquire objective evaluation results, promptly make changes to its structure, keep the educational programs up-to-date, and inform and take corrective actions in managing the process of developing regulatory documents of an educational organization. This presents an important part of the educational process since educational standards, the requirements and demands of the industry, the teaching staff, educational and reference literature, and material and technical support of the educational process change almost annually. For a specialist to decide to improve the quality of an educational program, it is necessary to consider numerous factors that can be unapparent or subjective. The conducted analysis of the educational process support processes demonstrated that educational organizations currently use software tools that work autonomously from each other. Moreover, even if the results of the work of one system are used in another one, data export or import is executed manually with the adjustment of additional parameters. Therefore, operational management largely depends on the human factor: the level of proficiency in the software demonstrated by specialists administering the corresponding systems and their personal interest in obtaining the result. To enable the customization and scaling of the developed information system, process models that allow for showing the key objects, and the relationships between them, have been created. A visual demonstration allows one to evaluate the bottlenecks, as well as the possible options for their modernization. The structural approach enables quick adaptation to the changes in the external environment. The above-mentioned model serves as a basis for the formation of the criterion and evaluation complex lying at the base of the information system for monitoring and managing the quality of educational programs. The developed structure of such a system is “flexible” and allows one to adjust the parameters of the criterion and evaluation complex and determine the values of parameters and the degree of their importance. All of these adjustments require minimal participation of specialists in the process of educational program content evaluation. The resulting information system presents an independent unit of the material and technical base of an educational organization; however, it is also an element that unites several heterogeneous information systems. Thus, an integrated information system ensuring the exhaustive and competitive functioning of an educational organization in modern market conditions is created.

Summing up the conducted study, we can argue that its objectives were completed and the goal was achieved. The formulated hypothesis was confirmed.

The directions for further research that should be noted include further improvement of the objectivity of the educational program content evaluation, the development of algorithms for automatic formation of the content of individual elements of the educational program content, and the improvement of the algorithms for educational program content evaluation to reduce the amount of time it takes.

7. Research Limitations

The methods used in the research are universal and generally accepted. During the study, we did not have any difficulties in using them and interpreting the results. The developed algorithms allow for creating an evaluation method for educational programs in different study areas and an information system based on it. Such software is a powerful tool in the hands of the participants in the educational process involved in its provision and control. The criterion-assessment complex is customizable and allows one to change not only the number of criteria or involved processes, but also experts. The mathematical apparatus used allows maintaining the model in a consistent state. The model of an information system makes it possible to unambiguously process any input data and obtain a result based on those indicators that are embedded in it by an educational organization or any other specialized department. This indisputable advantage allows the model to be used both in the Russian education system and abroad.

In conclusion, we did not set a goal to develop an optimal model of the system, but to reduce the costs of implementing certain processes in the field of education quality management. At this stage, the goal has been achieved. We admit that with the further development of this topic, the use of new software tools will further reduce certain categories of costs.