Thrown into Deep Water: Feedback on Student Satisfaction—A Case Study in Hungarian and Romanian Universities

Abstract

As a result of the COVID-19 health epidemic, online life has exploded into our daily lives, forcing most of us to move previously seemingly irreplaceable “face-to-face” activities into “non-face-to-face” meetings and activities in many sectors. One of the biggest challenges has been in the field of education: This sector, compared to other sectors, was less digitized. Under these circumstances, the entire education process was transferred to online space overnight, which was/is a major challenge for everyone. Thus, a questionnaire survey was conducted among students from two universities in Hungary and Romania, the results of which are included in the present article. The aim of the research was to measure students’ satisfaction and to examine the benefits of online education, for example, in terms of introducing hybrid education over the long term. Descriptive statistics as well as the Wilcoxon rank-sum test were used to analyze the database. The results showed that, from a practical point of view, there was no significant difference between the Hungarian and Romanian respondents who had a fundamentally positive view of digital education. In this respect, positive feedback can be seen as encouraging, especially for those individuals and social strata who may find online education much more attractive than traditional physical teaching. Online education could be an attractive, accessible, sustainable form of further education in the long run.

1. Introduction

It is indisputable that mankind is living in a world that is changing very fast, in circumstances which are very unpredictable. In general, when rapidly changing, unpredictable circumstances are mentioned, economic, social, and natural circumstances come to mind most frequently; they are less often associated with health problems that affect the whole world. There have been several worldwide epidemics of pandemic diseases, e.g., the Black Death (in 14th century), smallpox and tuberculosis (17th–18th century), or the 1918 influenza pandemic (Spanish flu), and, currently, HIV/AIDS and COVID-19. These epidemics have demanded, and are demanding, rapid, decisive, unexpected, and often “unpleasant” measures.

Today, to curb the COVID-19 epidemic, countries have been trying to combat the virus by introducing many common and similar measures. The effects of the pandemic and the series of measures introduced cover activities from all sectors, including tourism, economics, social activities, health, and cultural institutions. The pandemic has also had a major impact and brought about a “drastic” change, in the field of education, where, during March–April 2020 most countries switched completely or partially to online education, a process which has fundamentally transformed traditional educational activities. In the past, this form of education (online/distance learning) was not—or was only rarely—used, and thus its introduction was a great challenge for pupils/students, teachers, institutions, and parents as well.

In general, higher education responded quickly and positively to the transition to online education, with the introduction and operation of an existing platform for online education within a few weeks. This study examines how students in two nationally and internationally recognized universities (the University of Debrecen, Hungary, and “G. E. Palade” University of Medicine, Pharmacy, Science and Technology of Târgu Mureș, Romania) expressed their views on various issues related to online education at their universities one year after its introduction. Through this, the institutions received valuable feedback about students’ experiences, and the strengths of the activity and the elements which need to be developed. This, in turn, leads to an increase in the quality of the education provided, to better customer satisfaction, and also to a sustainable form of education.

• Review of the academic literature: the concepts of homework/online/distance/digital education/learning

The concept of so-called homework or telework also existed in the pre-pandemic period, the description and regulation of which were defined by legal rules. Telework is defined both at the European Union level and by the laws of Romania [1] and Hungary [2] as that form of work which takes place in another place than the work organized by the employer, using existing information and communication technologies. In these regulations, the keyword is the “place” where the work is performed, which should be “another” place than the employer’s headquarters or office location. Of course, we can only talk about teleworking if the nature of the activity allows it.

In the field of education, and for educational activities, it is difficult to define the concept of telework. In this regard, “another place” actually refers to somewhere far away from the students. For this new kind of (non-personal contact) education, society uses several terms, including ‘online education’, ‘distance learning’, ‘digital education’, etc. These terms basically refer to the same thing, i.e., any form of education that does not require a personal presence; however, there are still some differences among them.

Experts from the Organization for Economic Co-operation and Development (OECD) distinguish between online learning and distance learning: Online learning refers to learning aids that are used in a non-tangible form, i.e., online, while distance learning refers primarily to educational activity which takes place away from the educational institution or workplace [3]. According to a series of annual reports on the state of online education among U.S. institutions of higher education made by the Babson Survey Research Group [4], an online course is a course in which more than 80% of the information is provided online. In another definition of online education, formulated by Panova and Erakovich [5], the focus is on the use of Internet technology for a course. Distance learning can be defined as: “any approach to education delivery that replaces the same-time, same place, face-to-face environment of a traditional classroom”, or “a form of distributed learning enabled by the Internet” [6]. Komenczi [7] highlights three factors that define e-learning: a common set of three source elements, which are computer-based learning, web-based learning, and distance learning. These three components are the foundations of e-learning, which can provide data storage, data processing, network communication, and time and spatial independence.

• Advantages/disadvantages of online education activities

Even in the period before the coronavirus epidemic, several researchers had addressed the issue of the acceptance of online education, but this topic came to the fore with the advent of the COVID-19 pandemic. We can already find a very rich literature on this subject, highlighting the advantages/disadvantages of online education. In the first half of 2020, most countries switched to online education, mainly in higher education. This transition to online education has posed new challenges for educational institutions as well as for teachers and students. Some countries/institutions already had some experience in this field, while others had almost no experience at all.

There is both positive and negative feedback about students’ perceptions of online learning. Muthuprasad et al. [8] highlights the flexibility and convenience which are characteristic of online classes. This makes it an attractive option to students. McPartlan et al. [9] came to the same conclusion after a study conducted with 999 students: The most important reason why students would choose online courses rather than face-to-face courses is the flexibility of online courses. Most studies indicate the instructor’s central role regarding the impact of online education [10,11,12]. Bruggerman et al. [13] identified a series of skills and characteristics of teachers which are necessary to provide an effective online or blended teaching activity, of which the most important would be recognizing and accepting the need for change and creativity. Xin Xie et al. [14] lists six benefits (flexibility, information accessibility, global reach, equity, innovation, and efficiency) and four drawbacks (network instability and technological constraints, lack of a sense of belonging and connectedness, presence of distractions, and lack of engagement) related to online education. The disadvantages or negative aspects of online education include loneliness [15,16] and technical problems [17].

Volery and Lord [6] conducted a survey at an Australian university and identified the following three factors which can directly influence the success of online courses: technology, the instructor, and previous experience of using the online platform. Similar thinking can be found in Dillon and Gunawardena [18] who, based on their studies, identified three factors directly affecting the success of the online activity (technology, the characteristics of the instructor and the characteristics of the students). Two of the factors identified are identical in the two studies mentioned above: technology and the instructor.

• Evolution of online courses/digital education

Technological development, globalization, digitalization, and taking advantage of the myriad benefits provided by the Internet did not bypass education, either. Even before the global epidemic started in March 2020, there was some experience of online courses and some initiatives undertaken to introduce them in order to increase the use of digital technology in education as well. However, compared with other sectors, education is one of the least digitalized [19].

According to a study conducted by the National Centre for Education Statistics [20], the number of students enrolled on at least one online course at a US University is growing. Western Governors University has the most students enrolled on at least one online course, followed by Southern New Hampshire University; in both cases, the increase was over 70% between 2018 and 2015. Other universities (with fewer students enrolled on online courses) also registered an important growth by 2018 of over 50% compared to 2015 (e.g., Arizona State University, Colorado Technical University, Ohio State University, Central Washington University, etc.).

The National Centre for Education Statistics provides data regarding the number and percentage of students having enrolled or not enrolled on any distance learning course(s) in the USA before the COVID-19 pandemic [21]: in 2017, 66.5% of all students had never taken any distance learning courses (i.e., 33.5% of all students had taken a distance learning course). One year later, in 2018, the share of students who had taken a distance learning course(s) had grown to 35.3% (with 16.6% of these students taking exclusively distance learning course(s)).

The proportion of (undergraduate and graduate) students who have enrolled or not enrolled in a distance learning course(s) in the USA changed from 2012 to 2018 (Figure 1): The percentage of students who have never taken a distance learning course has been declining, while the percentage of undergraduate and graduate students who have enrolled in at least one distance course has increased to 34.5% and 39.8%, respectively [14].

It is noticeable that among the graduate students, distance learning courses are more popular and there is a faster growth in the proportion of students who have taken exclusively online courses (in 2018, this percentage had already risen to 31% from 22% in 2012).

Statistical data from Eurostat [22] show that in 2019 (before the pandemic), only 8% of people of working age from the European Union had enrolled in an online course. This is quite a low proportion, but it is double the figure for 2010 (Figure 2).

In a few states more than 13% of the population had done an online course, including Finland (21%), Iceland (20%), Sweden (18%), Spain (15%), Estonia (14%), Ireland, and the Netherlands (13%). In contrast, there were states such as Bulgaria (2%), Romania (3%), and Latvia (4%) where the popularity of online courses was very low in 2019 [23]. In 2020, after the start of the pandemic, this proportion jumped, reaching 32% in Iceland, 29% in Finland, and 12% in Hungary, although it remained very low in Romania at only 3%, according to data from Eurostat (Figure 2).

Regarding other aspects of distance learning, such as using online materials and communicating with instructors or students using educational websites, the situation is better: Iceland and Finland have very high figures (Figure 3 and Figure 4). Regarding the use of online materials, the remainder of the countries have proportions averaging between 20 and 40% (Denmark 39%, Finland 37%, Spain 35%, Sweden 31%, Estonia 28%, Portugal 27%, and Slovenia 26% in 2020). The countries with the lowest proportions for both aspects are Bulgaria, Greece, and Romania.

• E-learning in Romania (in an academic environment) before and after the start of the COVID-19 pandemic

Although distance learning appeared in Romania in 1995, by 2007, only 58% of Romanian higher educational institutes had used eLearning solutions in their activities [24]. The most popular eLearning platforms are Blackboard Learn (Learning Management System) and platforms designed by Microsoft and Google (MTeams, G Suite). The Romanian universities implemented the following eLearning platforms after the pandemic started: Babes-Bolyai University—Moodle; Lucian Blaga University of Sibiu—Google G Suite; Vest University of Timisoara—Moodle; Bucharest University of Economic Studies—Google G Suite; and George Emil Palade University of Medicine, Pharmacy, Science and Technology of Târgu Mureș—Blackboard.

• E-learning in Hungary (in academic environments) before and after the COVID-19 pandemic started

In Hungary, as in other European countries, a series of steps were introduced in the early 1990s and early 2000s to launch digital, distance learning (e.g., the “schoolnet program”), which by 2012 had already achieved some positive results, although purely e-learning exists in only a few institutions in Hungary (in 6 out of 39 institutions, i.e., 15.38%), accounting for 1.52% of all forms of education [25]. The Hungarian universities implemented the following eLearning platforms after the pandemic started: Corvinus University of Budapest—Microsoft Teams; the University of Public Service—Moodle; Semmelweis University—Moodle; University of Debrecen—Webex; and Eötvös Loránd University—Canvas/Moodle.

2. Materials and Methods

The database of the study was compiled by means of a questionnaire survey, in the course of which the experiences of digital education were measured at the Faculty of Economics of a higher education institution, one in Hungary and one in Romania, in the summer and autumn of 2020. The two institutions are the University of Debrecen, Hungary, and the “G.E. Palade” University of Medicine, Pharmacy, Science and Technology of Târgu Mureș, Romania.

The University of Debrecen, Hungary (UNIDEB), is one of the most recognized universities, both nationally and internationally, undergoing dynamic development. The Faculty of Economics is currently its largest faculty with a student population of nearly 5000. It has a varied course offer, which includes economics and sports courses, among others. Lectures are given in Hungarian and English in 8 undergraduate, 11 master’s, and 1 doctoral programs.

“G.E. Palade” University of Medicine, Pharmacy, Science and Technology of Târgu Mureș, Romania (UMFST), is an accredited public education institution, which is dynamic, multicultural, and is the only Romanian university in the field of health with study programs in Romanian, Hungarian, and English. More specifically, UMFST has more than 10,000 students, 35 bachelor study programs, 38 master programs, and 7 doctoral fields of study. UMFST is recognized at the national and international level, being evaluated in 9 international ranking databases. Within the UMFST structure, there are currently 7 faculties, including the Faculty of Economics and Law, with over 1200 students on bachelor’s and master’s degree programs.

The advantages of a questionnaire survey are that the questions are simple to apply, the questions are fixed in advance (closed questions), they take a relatively short time to answer, and the interviewer can obtain important information by evaluating the answers [26]. A disadvantage is that respondents may not be able and willing to respond, and responses may be deliberately dishonest. In the present study, online questionnaires were chosen because they can be filled in by a wider audience, in larger numbers; moreover, in the current situation, there was no opportunity to carry out a questionnaire survey in any other form. The questions were formulated statements, thus they belong to the group of closed questions, and several subtypes can be found in the questionnaire. There are single-answer questions, such as questions about age and department, and also seven-point Likert scale questions measuring satisfaction with digital education. The Likert scale is a scale with seven response categories on which the respondent must choose a number from the range between “completely dissatisfied” and “completely satisfied”. It is an easy-to-apply, popular measurement scale [27]. The questionnaire was prepared using the EvaSys system, and the students of the two institutions answered it on the interface provided by this system. The EvaSys software system enables fast and efficient surveys in the education sector using state-of-the-art data processing and Internet technology. With the help of EvaSys, a wide range of surveys can be conducted in courses and study programs.

The sample consisted of 1396 completed and evaluable questionnaires. Based on the answers, the gender, age, form of course, and department of the respondents can be determined. Among the respondents of both institutions, a higher proportion of women completed the questionnaire (approximately 70%). In terms of their level of education, the majority of respondents were in undergraduate courses, and full-time courses were the most frequent course type (

Table 1. Number and proportion of respondents at the two universities.

	UNIDEB	UMFST
Variable	N = 1.032 a	N = 364 a
Gender
Male	331 (32%)	105 (28%)
Female	701 (68%)	259 (72%)
On what level of course do you study?
Undergraduate course	762 (74%)	296 (81%)
Higher education vocational course	161 (16%)	56 (15%)
MSc course	107 (10%)	12 (4%)
On what form of course do you study?
Correspondence course	187 (18%)	75 (20%)
Full-time course	845 (82%)	289 (80%)

^a n (%).

).

A significant portion of the data was measured on an ordinal scale, which meant a 1 to 7 rating. In these cases, the Wilcoxon rank-sum test was used, which compares mean rank values and can be interpreted as a non-parametric version of the independent two-sample t-test. Since the Wilcoxon rank-sum test does not calculate a mean value, it is not the difference between the means which is examined but the relationship between the two distributions.

In addition, a chi-square χ² test was used to compare categorical responses. In these cases, the frequencies are compared. The χ² test examines the extent to which the observed and expected frequencies differ within the categories (1):

$$x^2={\displaystyle \sum }_i{\displaystyle \sum }_j\frac{\left(f_{ij}-e_{ij}\right){ }^2}{e_{ij}}$$

(1)

where f_ij is the observed frequency in row i and column j, while e_ij is the theoretical frequency in row i and column j. The test statistic follows a χ² chi-square distribution with (n−1) (m−1) degrees of freedom, where n is the number of rows and m is the number of columns. To examine significant differences, standardized Pearson residuals and mosaic diagrams were used. For standardized Pearson residuals, there was a significant deviation from the expected frequency in categories where the residual was lower than −2 or higher than +2 [28].

As these methods are standard in statistical analyses, they are not described in detail. Major mathematical and computational details can be found in the work by Field et al., [29]. In addition, classical descriptive statistical methods were applied, mainly frequency tables due to the categorical nature of the data.

3. Results and Discussion

During their online education, the vast majority of students had adequate Internet access, thus this was not a barrier to the transition. The proportions of respondents with access was 94% in Hungary and 87% in Romania. In the case of the former, 73% had both broadband and mobile Internet. In the case of Romanian respondents, 52% said that both types were available to them. Almost 100% of the students of both institutions had computers.

Student satisfaction was measured on a 7-point ordinal scale, therefore, the difference between the Hungarian and Romanian responses could not be examined with t-test type mean comparative tests. To eliminate this, the Wilcoxon rank-sum body test was applied. Most of the time, the test showed significant differences; however, the difference was never really large. Median values usually differed by a level of 1. Due to the diversity of the questions, the analysis presented here is limited to the most important questions. There was no difference between the Hungarian and Romanian respondents in the case of two questions:

• How satisfied were you with the distance learning?
• How helpful were the lecturers if a problem occurred?

The lack of a significant difference in this case can be evaluated positively, as this meant that there was no difference between the median evaluation of the Hungarian and Romanian respondents. The median values were higher for the Romanian respondents in all cases. There was also a significant correlation between curriculum and assessment issues. The median value was usually five in Hungary and six in Romania. Contact with instructors and their assistance was considered favorable by both groups, with a median value between five and six. Instructors were considered prepared by both groups, with the vast majority of responses (50–70%) being five or above. The same trends have characterized responses to administrative and data-protection-related questions. Satisfaction with the online interface also showed similar values. In this case, the evaluation of Romanian respondents was also significantly higher. However, the difference was not large here either; in both groups, 80% of the respondents gave a rating of at least five or better. The median value was six in both cases, and the p-value of the difference was around the threshold.

It is worth noting that although a statistically significant difference was presented in almost all cases, from a practical point of view there was no significant difference between Hungarian and Romanian respondents. Most responses were above five, and significant differences were provided by minor differences in these groups. The Wilcoxon rank-sum test results for the most important questions are presented in

Table 2. Comparison of ordinal type questions with the Wilcoxon rank-sum body test.

	W	p
How satisfied were you with the distance learning?	186,005.00	0.97
How satisfied were you with the overall quality of the material provided by your lecturers?	240,484.00	0.00
How satisfied were you overall with the amount of material delivered by your lecturers?	236,564.00	0.00
How satisfied were you with the availability of learning materials?	237,313.00	0.00
To what extent has the ease or difficulty of completing the course changed?	218,608.00	0.00
How do you rate the consistency of the curriculum provided and the exam?	200,035.00	0.01
In the case of subjects where the conditions for fulfilment have changed, has it been easier to complete the course?	215,202.00	0.00
How satisfied were you with the contact with the lecturers?	198,760.00	0.02
How helpful were the lecturers if a problem occurred?	182,332.00	0.83
In general, how prepared did the lecturers seem for online administration?	235,291.00	0.00
In general, how prepared did the administrative staff seem for online administration?	229,261.00	0.00
In general, how satisfied are you with the online administration?	201,330.00	0.01
How satisfied are you with the e-learning system?	205,957.00	0.00
How secure do you consider the interface in terms of data protection?	215,570.00	0.00
How manageable do you find the interface?	164,767.00	0.00
How useful do you find the interface?	171,423.00	0.05
How interactive did you find the online lectures?	212,780.00	0.00
How satisfied were you with the interface you marked?	168,972.00	0.05

Note: Scales were numbered from 1 to 7. Significant differences are below or equal to 5%.

.

One of the most important questions that arises in terms of distance learning is how students assessed distance learning as a function of teacher skills. For easier understanding, the responses on the seven-level ordinal scale were divided into three categories. In levels 1–3, the answers were marked with B, at level 4 N was used, and at levels 4–7, G. Thus, in practice, B meant a negative rating, N a neutral rating, and G a positive rating (Figure 5).

Based on the above, there was a significant correlation between satisfaction with distance learning and the general preparedness of teachers in the case of Hungarian respondents (χ² = 336.56 with 4 degrees of freedom, p ≤ 0.05). The same can be stated for the Romanian respondents (χ² = 75.762 at 4 degrees of freedom, p ≤ 0.05). The reasons for the significant correlation can be examined using standardized Pearson residues, where a standardized residue lower than −2 or higher than 2 indicates a significant correlation (in the sense that the observed frequency was more or less than the expected frequency).

In the case of Hungarian respondents, the value of the standardized residue exceeded the threshold value in practically all categories (except in the case of the N-B pairing). The values on the diagonal were positive, while those outside the diagonal were negative. This meant that the number of respondents in the two response categories moved relatively well together, i.e., those who rated teacher skills as worse were also less satisfied with distance learning. In addition, (statistically) more students than expected were satisfied with distance learning and also rated the skills of their teachers as good. From the relative blocks of the mosaic figure, it can be observed that the most significant proportion of the respondents was provided by this group. However, there were very few who were satisfied with distance learning but were not satisfied with the skills of their teachers (or vice versa).

The same trend was observed for Romanian respondents, albeit it was less pronounced. All blocks in the mosaic figure were significant except for the B-N, N-B, and N-G pairings. The largest block was also provided by the number of respondents who were satisfied both with distance learning and the skills of their teachers. Similar to the Hungarian respondents, there were fewer respondents than expected in the opposite groups.

These results draw attention to the fact that satisfaction was associated with responses. The test is unable to show which variable may be the cause and which is the effect, but it was clear that the correlation between distance learning and teacher skills is significant.

4. Conclusions

The situation caused by COVID-19 hit higher education institutions and students unexpectedly, and in the absence of previous experience they had to adapt to the changed situation. There was a short transition from face-to-face education to online education, which poses a new challenge for both students and faculty. Students from both universities mostly had the technical and info-communication background required for the transition. It can be said that although there was a statistically significant difference in almost all cases, there was no significant difference between the Hungarian and Romanian respondents from a practical point of view. Most responses were above five, with significant differences provided by minor differences in these groups. In this respect, positive feedback can be seen as encouraging, especially for those individuals and social strata who may find online education much more attractive than the traditional face-to-face learning system. Online education could be an attractive, accessible, sustainable form of further education in the long run.

The results of the survey clearly showed a positive result in the assessment of digital education, but it is important to note that the results will actually become measurable only later on. The positive evaluation made by the students regarding the online educational activity is not necessarily and directly reflected in a positive change in students’ knowledge. Measuring the level of knowledge of students during the online period can be done by analyzing—in the first place—the students’ grades, and their graduation level compared to the average level of the previous period.

Regardless of the results of the survey, there is always place for improving e-learning, for which our suggestions and recommendations especially for teachers are: using more own made digital videos and making them available to students, regularly collect feedback from students, requesting regular and frequent independent work from students, thus providing feedback about the level of knowledge.

Distance learning is typically seen as a challenge by every participant in the educational system and whether the effects of the process are more positive or more negative may vary not only among groups and institutions but also among individuals. At the same time, distance learning can also be considered a kind of opportunity, since the success of education can typically be measured in multiple ways. This could include, for example, the performance of students in exams and, in terms of educational success, the extent to which a university can prepare its students to succeed in further studies and in life in general. Different levels of education should also prepare graduates to successfully meet educational expectations. In this context, it cannot be ignored that, despite all the unexpectedness, immaturity, and difficulty of the first phase of “quarantine learning”, it can lead students to acquire learning methodological tools that they will be able to take advantage of later in their studies parallel with work. Taking all this into account, we can conclude with certainty that online/hybrid education can indeed become a possible form of sustainable education in the future.