1. Introduction
The current speed of change requires people to act as quickly as possible and respond to any social transformation. A modern and proactive person can no longer afford to stop at what has been achieved in the direction of the acquired knowledge; for this he must be open to new trends, use modern technologies and develop his professional knowledge. At the forefront of progress will always be countries, companies and people capable of rapidly changing and adapting. Any kind of personal change requires diversified development—it is achieved through training. However, the problem is that approaches to learning are also changing: online courses are developing, educational programs are increasingly relying on modern competencies, the ratio of theoretical and applied components in learning is changing, the concept of LLL (lifelong learning) is developing, etc. Under these conditions, the value of the operational assessment of the effectiveness of the training is significantly increased [
1,
2].
Economic science from the second half of the 20th century has created many forms and methods for assessing the effectiveness of training. The application of these methods is used in their original “author’s” form, whereas others are adapted and combined, actually creating unique methods that correspond to the specifics and needs of the teacher and the student.
Over the past decades, the education system has changed dramatically, but approaches to assessing the effectiveness of training have remained the same. Relying on the fact that educational organizations are a place for collecting a huge amount of personal information about students, an important factor is their correct collection, analysis and use, which can improve the process of providing such services and thus improve the end result for students. Modern technologies make it possible to take into account not only information about learning outcomes but also changes in behavior and performance of a trainee. By comparing detailed information about the cost of resources with the results obtained during training, you can create a system for assessing its effectiveness. Some educational organizations are actively working in this direction; having studied their experience, we can identify the most promising approaches to assessing the effectiveness of training.
Dasenbrock [
3] sees the corporate university as an opportunity to focus on the practical aspects of the industry, since at the moment the main problem and disadvantage of traditional education is that students entering traditional universities do not receive education, but a certificate, and the universities themselves are guided by the problem of attracting applicants.
Walton [
4] emphasized that the main goal of the corporate university is to inspire the student with an original idea, as exemplified by the experience of the Walt Disney Corporation. In 1963, Walt Disney created Disney University in Anaheim, California, so that all new hires would understand and deliver the services he conceived [
5].
According to the Corporate University of Exchange (CUX), provided to a New York-based consulting firm that specializes in supporting the development of corporate universities, there were about 400 corporate universities in the United States in 1990, and by 2001 that number had grown to 2000.
At the beginning of the history of the United States, there was no formal education system for the development of the emerging workforce. Out of necessity, American businesses have become employers and teachers of the workforce [
6]. Competition and technological advancement today require modern corporations to increasingly become the main educators for the workforce [
6,
7].
Internationally, four corporate university associations predominate: (a) CorpU, (b) Corporate University Enterprise (CUE) and (c) Global Association of Corporate Universities and Academies (G-AUCA) and (d) Global Council of Corporate Universities (Global CCU). CorpU and the Corporate University Exchange are located in the United States, and the Global Association of Corporate Universities and Academies (G-ACUA) and the Global Council of Corporate Universities (Global CCU) are located in Western Europe.
Among a number of problems in this area, it should also be noted that corporate universities are subject to serious scientific criticism. In an effort to enhance the prestige of their academic departments, some business leaders have cleverly added the word “university” to the name [
8]. For some who work in a traditional university environment, it is difficult to support the concept of a corporate university [
9]. Blass [
10] has argued that corporate adoption of the name “university” has “muted” the term, making it acceptable for use in various inappropriate ways. This casts doubt on the legitimacy of the adoption of the term “university” by corporate organizations.
In defense of the term “corporate university”, Wills [
11] suggested that corporations using the term “university” are trying to gain a positive association with the learning environment. She went on to state that corporations do indeed generate new knowledge in the form of research and development. Finally, Wills suggested that the use of academic terms in the corporate environment will continue to grow as partnerships with higher education expand. In addition, a corporate university can be a way to strategically develop the HR department in an organization [
12].
Betof [
13] described six key benefits of corporate education: (a) it helps to achieve results; (b) stimulates the development of leaders and like-minded people; (c) improves the leadership skills of those who teach; (d) strengthens organizational culture and communication; (e) fosters positive organizational change and (f) reduces costs by attracting the best talent.
Walton [
14] argued that the traditional university is becoming more corporate in its views, whereas corporations rely more on the knowledge they need. Kiely [
15] argued that it is time to recognize that corporate universities are very different from traditional academic universities. Training departments focus their efforts on employee training [
16,
17]. In contrast, corporate universities are focused on supporting an organization’s strategy, which can include more than just training.
In our study, we focused on corporate training, which, due to its significant attachment to business, greater flexibility, short educational cycles and practice orientation, is able, unlike the state education system, to quickly introduce and use innovations, i.e., an accelerator in the field of education. The object of this research is corporate universities. The purpose of this work is to analyze the possibilities and experience of evaluating the effectiveness of training at a corporate university, highlight the tools that can be used to evaluate various performance indicators, and determine which of them can be transferred to other educational organizations.
To achieve this goal, we will consider the general characteristics of the system of corporate universities in our country, determine the optimal tools for assessing the effectiveness of corporate training and evaluate its applicability in practice.
2. Materials and Methods
Conducting training in the traditional format for employees of some organizations is time-consuming and financially expensive. Therefore, one of the ways to organize a single developmental space is training in a digital format. The most popular in the context of digitalization are electronic and distance learning. Online learning (e-learning) and distance learning are considered identical by some researchers [
18,
19,
20]. But in our opinion, these two types of learning must be distinguished on the basis of the subject of action. E-learning (e-learning) is one of the digital learning formats that involves the use of the Internet and modern multimedia technologies to access educational resources. The specificity of online learning lies in the optional nature of the interaction of people with each other and the use of interactive electronic tools for working with information. Thus, e-learning involves subject–object and subject–subject forms of interaction. The main way to gain knowledge in online learning is self-education of students, i.e., self-filling gaps in knowledge in a convenient form. So, e-learning is informal, unlike other digital learning methods.
In modern conditions, traditional teaching methods are acquiring new forms and content based on the introduction of digital technologies into this process. The use of software products and information technologies in the traditional format of conducting classes allows you to improve the quality of the results of the educational process, optimize training costs and ensure accessibility for each employee of the company. For example, today it is impossible to conduct lectures and practical classes without using such elements of digital learning as a presentation, an electronic training course, testing students based on computer programs, analyzing cases in digital format and conducting online quests. Thus, the use of digital technologies makes it possible to transfer traditional forms of education into an electronic format. Distance learning is a way of implementing the learning process based on the use of modern information and telecommunication technologies that allow learning at a distance without direct personal contact between the teacher and students. The peculiarity of this training is that the interaction of participants occurs indirectly through a program that determines the algorithm of the training system and all participants. This type of training is characterized by greater formalization than e-learning [
21,
22]. The authors of the article do not share the opinions of individual researchers regarding the classification of distance learning as a method. It is a digital learning format that includes several different methods (
Table 1).
As part of the study, we have identified the advantages and disadvantages of using distance learning methods.
So, one of the main HR trends in a pandemic is the development of e-learning and distance learning. Many foreign companies have switched to a remote work format and digital training of employees. In the practice of training and development of personnel, new forms have been actively introduced—scribing, animated infographics, educational 3D games, microlearning, virtual and augmented reality, artificial intelligence and machine learning. Gamification as an integral part of business information solutions today has become an indispensable element in the development of corporate training. The results of our study show that companies save on training budgets (approximately 50–70%) due to the replacement of face-to-face forms with electronic and distance methods.
Digital learning during the coronavirus pandemic is becoming increasingly popular among companies. Statistics show that the share of companies that create their own internal online learning systems is only 20%. Most Russian enterprises prefer to train their employees using digital platforms. Thus, for many companies there is a difficult question about choosing an effective digital platform, because more than 1000 providers are represented on the Russian market for the development and implementation of digital learning systems.
According to the most general estimates, there are more than 4000 corporate universities in the world [
23]. According to CUX data, by 2010 alone, more than 3700 such educational organizations were registered in the United States; they are currently being actively created in Europe, Asia and Russia. Note that in the world science of management and education there is still no unambiguous interpretation of the concept of “corporate university”; oftentimes, under it, various forms of corporate educational structures (from departments and training centres to outside educational organizations) are combined. J.K. Meister [
24] understands a corporate university as “a strategic tool for providing training and education of personnel, customers and suppliers in order to meet the requirements of the organization’s business strategy.” This interpretation somewhat vaguely defines the educational organization we are considering.
Doug Guthrie gives a more detailed definition. From his point of view, a corporate university is “a strategic developer of human capital in a company that meets its business goals, a tool for broadcasting corporate culture and a catalyst for the creation and transfer of knowledge” [
25]. It is this interpretation that we propose to use as a basic one. Note that a theoretical review of the literature on corporate universities in the world was carried out in sufficient detail in the work of A.D. Chanko, A.A. V. Basner [
26]; therefore, we will not pay much attention to this issue.
Ukraine is also influenced by Western corporate learning culture. This trend has been picked up in recent years by leading domestic companies. Thus, among the largest corporate universities in Ukraine are the following:
- (1)
DTEK Academy (Year of foundation—2010, number of students—1600 people). Studying at DTEK Corporate University focuses on the training of middle and senior managers. There are 4 training programs—“Successor”, “Integrator”, “Energy of Knowledge”, “Energy of the Leader”—designed for managers at different levels. The program is designed for 2 years without interruption from production. After graduation, graduates defend their dissertations and receive a certificate.
- (2)
Ukrainian Agricultural School (Mriya Agricultural Holding, year of foundation 2011, number of students—100 people). The educational project of Mriya agricultural holding is designed for undergraduate students and young professionals with up to 3 years of experience. The school has 3 educational programs: Agricultural School, Mechanical School and Accounting School. Since 2013, Mriya has had an MBA program for the company’s top management.
- (3)
Privat University (PrivatBank, year of foundation—2003, number of students—30,000 people). Studying at PrivatBank’s corporate university is mandatory for all its employees. Immediately after the interview, the “newcomers” are sent for training—a training program that covers 29 banking specialties. After this stage, the company selects 50–70% of the most promising students who get into the staff of the bank. Periodically, employees undergo refresher courses and undergo certification. Tuition at the university is completely free and involves the payment of a scholarship in the amount of the initial rate of the bank employee.
- (4)
Career Development Program (EPAM Systems, year of foundation—2012, number of students—1409 people). The IT integrator EPAM Systems established a CDP Global training department in 2005. Last year, a similar structure was created in IPAM Ukraine. CDP UA has training programs for managers, employees and students. Training takes place during working hours, without interruption from production. There are no fixed trainings; they are created at the request of project managers and include the development of technical skills and effective communication skills. Training is free.
- (5)
Ernst & Young Academy of Business (Year of foundation—1995, number of students—1370 external clients and 250 employees). Unlike other corporate educational institutions, the academy is designed primarily for external clients. Free training is provided for Ernst & Young employees. The company also offers its employees about 2000 online courses—both mandatory and optional.
- (6)
Deloitte Academy (Year of foundation—2011, number of students—356 people). At Deloitte Academy, as at Ernst & Young Corporate University, students from other companies study. The basics of auditing, accounting and other specialties are offered for training. Deloitte employees spend up to 20% of their time studying at the academy. There are also training programs for students in the company—a set of bachelors is recruited, who are taught in accordance with the standards and requirements of the company [
27].
However, despite the presence of corporate universities in leading Ukrainian companies, most do not have employee training systems. Thus, according to a study by the portal rabota.ua, the practice of improving the knowledge and skills of employees is implemented in only a third of Ukrainian companies (36%). And 53% of domestic companies do not have staff training systems.
However, despite the presence of corporate universities in leading Ukrainian companies, most do not have employee training systems. Thus, according to a study by the portal rabota.ua, the practice of improving the knowledge and skills of employees is implemented in only a third of Ukrainian companies (36%). Accordingly, 53% of companies do not have staff training systems.
Only 3% of companies have a corporate university, and every 5th has a training centre. Another 3% of Ukrainian employers can afford to maintain both a corporate university and a training centre. Moreover, 57% of companies do not have the appropriate tools for staff training.
Our review will not be complete if we do not reflect on the possible shortcomings of the implementation of corporate training in the format of a corporate university as a separate legal entity:
- -
separation of training from business—the structure is always made for the business task and it must be constantly updated when the task changes. The corporate university, busy mainly with finding new clients, satisfying requests “from the outside”, often does not have time to meet new realities, the needs of the basic enterprise;
- -
availability of service for the studied organizations is characterized by a large number of service personnel (legal department, security, accounting) in comparison with the training centres in the structure of the organization;
- -
lack of a single customer—there is often a duplication of functions of the corporate university and HR-department in enterprises. It is possible that regional divisions apply to a corporate university at their own discretion; as a result, some regions will develop better than others, there may be internal competition between divisions and the university itself and there may be competition for university resources between regions;
- -
the need to make money—having such a goal can lead to the fact that the corporate university will try to train more employees than the organization needs.
3. Results
To describe the possibilities of evaluating the effectiveness of corporate training, we identified a methodological basis, in the role of which was the Kirkpatrick model. As you know, it was formulated in the late 50’s by Donald Kirkpatrick. During this time, this model has proven its practicality and ease of use. Most importantly, it is characterized by the presence of an internal logic that allows customers and performers to evaluate various aspects of the work not in terms of inputs (how much money was spent on education, which teachers were involved), but from the standpoint of results (how satisfied the students are, how their behaviour has changed since enrolment and what effect the organization has received). The Kirkpatrick model is actively used in the practice of corporate universities; in particular, it is implemented in the educational process in KU Citibank, KU Deloitte, KU American Express, KU Wells Fargo, etc.
Note that according to the CEB study “Learning Analytics: Measurement Innovations to Support Employee Development” (2016), according to a survey of 111 companies, 73.7% of respondents use the Kirkpatrick model in their practice [
28]. This is the most common assessment tool effectiveness of corporate training in the world.
The chosen model for evaluating the effectiveness of corporate training includes four levels of measurement:
Level 1: reaction. On this level, it is possible to learn to what extent participants of training—after its carrying out—react to training events.
Level 2: learning. At this level, how the participants were able to acquire the expected knowledge, ideas and attitudes after participating in the training event is assessed.
Level 3: behavior. It is assumed that at this level, how participants apply in their workplaces what they have learned during the training should be assessed.
Level 4: results. This is the most complex level of assessment, which examines the extent to which changes in the company’s performance have resulted from training activities and subsequent post-training events.
Note that this model does not stand still, and recently its author presented a new Kirkpatrick model.
What tools can be used to assess the effectiveness of training at each level? From our point of view, the following sets of tools can be offered:
Level 1: reaction. The simplest option is direct observation, as well as pre-program survey and questionnaire during the educational program.
Level 2: learning. To assess the level of learning, you can use tests, quizzes, assessment of solutions to individual tasks and the task of developing a plan for development/change.
Level 3: behavior. This level involves more sophisticated assessment, which can be done through a post-program survey of participants and staff who interact with them, monitoring in the workplace, post-program survey of student leaders, monitoring of the action plan, key performance indicators, interviews with the participant, follow-up sessions (meetings, discussions, consolidation of acquired skills in a few weeks after training), reports/essays of the participant, post-program focus groups and evaluations of work behavior.
Level 4: results. This level is considered one of the most difficult to assess, as it is necessary to assess the immediate learning outcomes. For this you can use tools such as perception of the effect of training of the participant by his leader, customer satisfaction, partners, business results (profit growth, sales, productivity, cost reduction), the perception of the effect on the business of the participant, the measurement of performance indicators of the participant, indicators of turnover and progress and ROI.
Note that experts in the field of corporate training highlight some shortcomings of this model. Among them are: the gradation of assessment is not justified; you can measure the end result of training without assessing all intermediate indicators at different levels and the lack of an actual correlation between the complexity of the assessment and its usefulness [
29]. Despite these shortcomings, the Kirkpatrick model is one of the most widely used in practice by domestic and foreign corporate universities.
Next, we consider what exactly are the methods of assessing the effectiveness of training in various educational programs used in the Corporate University and what are the results.
Tools for assessing the effectiveness of training on the indicators of the Kirkpatrick model for the 1st and 2nd levels are implemented in most educational programs of the Corporate University. Questionnaires during the program are implemented in 100% of educational courses. It is necessary to assess the 1st level—the response to training, expressed in the degree of satisfaction of participants. To study the expectations and experiences of participants, as well as to form an attitude to effective work during training in almost all programs (except for training for trainers), a pre-program survey is used. Slightly less common are testing and the formation of individual assignments, which reflect the effectiveness of training at the 2nd level of the Kirkpatrick model [
30].
As an example, here is a list of questions that is used to assess the 1st level with a pre-program survey to study the expectations/experiences of participants in the program “Corporate Culture and Emotional Intelligence-I”:
1. What do you like about the corporate culture of Corporate University?
2. What do you dislike about the corporate culture of Corporate University?
3. Evaluate on a 10-point scale how much your behavior corresponds to the corporate culture of the Corporate University.
4. Evaluate on a 10-point scale how the existing corporate culture corresponds to the stated one.
5. Do you believe that your daily behavior can influence the change in the corporate culture of the Corporate University?
6. What will be a good result of the program for you personally?
Similarly, taking into account the specifics of the training, questionnaires were compiled for all other programs. The results of the participants’ satisfaction assessment are summarized in
Figure 1.
As can be seen from
Figure 1, satisfaction is assessed in several areas, compared to the previous year. Note that the data in the figure reflect the results of surveys of 40 groups (1130 listeners). As noted earlier, Level 2 indicators are less common, but they are used in the practice of the Corporate University in many programs—for example, in educational programs of the Corporate University.
But at this level, the features of the program itself determine the applicability of certain tools for assessing the effectiveness of training. The most “run-through” in terms of performance assessment is the Leaders Team program (for teams of heads of regional banks and functional blocks). It uses tools such as polls of participants, post-program survey of the head, interviews with a participant, follow-up sessions and participants’ essays [
31].
As an example of using a tool for assessing effectiveness using a survey of participants before and after training (level 3), we will describe the features of the questionnaire for the Leaders Team program. It includes open-ended questions and assessment on 33 indicators. Each indicator is evaluated on a 10-point scale twice: retrospectively and now. The questions are grouped into blocks:
1. Formulation and transmission of the team result vision.
2. Team coordination.
3. Formation of commitment to the overall result.
4. Giving and receiving feedback.
5. Building trust.
6. Translating conflict into a productive way of finding ideas and solutions.
Examples of interpretations of answers to the questions of this questionnaire for the Leaders Team program are shown in
Figure 2,
Figure 3,
Figure 4,
Figure 5,
Figure 6 and
Figure 7. A total of 661 people who studied in 2020 took part in the assessment. Of the total number of students, 24% were involved in the survey. The questions were slightly different depending on whether they were interviewing the leader or team members.
The purpose of this survey for the Leaders Team program is to assess the level of team interaction before and after the program. Judging by the results of the assessment, we can conclude that the program provides managers with a new view of the assessment of their own skills related to team interaction [
32]. Confirmation is the increased awareness of the participants regarding the self-assessment of the level of skills before the program, which in this example is presented retrospectively: the minimum score is 5.46, the maximum is 7.9 on a 10-point scale. For example, prior to the program, the main areas of team development (range of estimates 5.46–5.99) are skills related to:
with feedback: providing feedback to the superior manager;
building trust: team members openly admit mistakes in front of colleagues, are aware of each other’s privacy;
coordination of team actions: team members work in concert with each other, team members with different functionalities coordinate work processes with each other;
translating conflict into new ideas: team members openly disagree with the leader’s ideas.
Relatively high (in the range of 7.33–7.90) are assessed skills related to:
If you look at the scores for all the questions, you can see that the difference between the retrospective average score (6.34) and the current score (7.95) was 1.61. Evaluation of the survey results shows that the program contributes to significant development of team interaction skills. So, when assessing “skills after the program”, an increase in values is noted: the minimum score is 7.00, the maximum is 9.14 on a 10-point scale. In addition, after the program, there is a qualitative dynamic of skills development in all indicators. The average delta of changes, according to team members, was 1.61; according to team leaders, it was 1.93.
The maximum absolute values of the delta of changes are from 2 to 3 points and are noted for skills related to:
with the coordination of team actions: team members work in concert with each other (3.00—according to the team leaders and 2.05—team members);
the formulation and transmission of the team result vision: the team’s activities are aimed at achieving priority tasks (2.57—team leaders and 2.26—team members), team members have a common vision of what we want to achieve in the future (2.29—team leaders), team members have a common understanding of the image of success (2.57—team leaders).
The average skill gains are shown in
Figure 8. At the same time, against the background of positive dynamics, the following skills remain as areas for further development (range of estimates 7.01–7.18):
with feedback: in the last month I have provided feedback to a superior manager;
building trust: team members are aware of each other’s privacy;
translating conflict into new ideas: team members openly disagree with the leader’s ideas.
A detailed description of the 4th level tools and examples of their use in the practice of the Corporate University is difficult due to the complexity of the methods and the lack of data about them in open sources. The 4th level involves assessing the effect on the business results of the customer of the training, which are usually confidential.
In conclusion, it should be noted that some corporate universities are moving away from basic learning efficiency metrics, replacing them with traditional marketing indicators that are adapted to the specifics of corporate training. One of the most common is the NPS (Net Promoter Score) loyalty index, which is calculated quite simply: respondents are prompted to answer one question: “What is the likelihood that you will recommend the curriculum to your colleagues?” This indicator is assessed on a 10-point scale. For its calculation, the number of those who chose options from 0 to 6 (they are called critics) and those who chose options 9–10 (promoters) are used. The formula for calculating this indicator is as follows:
NPS = 100 × (Number of promoters − Number of critics)/Total number of respondents.
To some extent, this trend demonstrates the idea that students are key clients of the educational process, and corporate universities should focus on the most effective work with them, which in the end will still lead to an increase in the economic indicators of the business customer.
This section may be divided by subheadings. It should provide a concise and precise description of the experimental results and their interpretation as well as the experimental conclusions that can be drawn.
A significant factor influencing these processes is the overall digitalization of management processes, including in the field of personnel training and development. The penetration of new technologies is accompanied by the entry into the market of generations brought up in the conditions of digitalization. There is a transformation of means and methods of training aimed at achieving the competitiveness of companies in a changing environment. Cloud technologies are being introduced to systematize human resource management activities, including the educational aspect, and direct it to achieve the strategic goals of companies. At the organizational level, companies respond to changes in the environment by increasing investment in training programs, a significant expansion of the forms and methods of staff training.