Two Years of Hybrid Education in Romania: A Comparative Overview of the Students’ Expectations for the Online Educational Platforms

Abstract

During the COVID-19 pandemic, due to the high infection rates, in some periods, a hybrid or totally online educational system was used. In both types of fully online and hybrid educational processes, the need for online educational platforms arose. This research aims to provide a comparative overview of the expectations for these platforms using the responses of two successive generations of students as input by applying the Kano methodology. Furthermore, this article performs a Fong test as a relevance check to identify the features for which the Kano analysis results are statistically significant. The results of the analysis show that the expectations of students are more related to the features that describe their access to administrative resources and the technical-related features. This can be explained through a permanent need, independent of the chosen educational system (i.e., face-to-face, fully online, or hybrid) and their field of study. The use of virtual reality (VR) technology in creating laboratory experimental lessons is identified as the only possible mechanism, from the proposed features in the engagement category, to keep students engaged during periods when fully online and hybrid educational processes are needed.

1. Introduction

The COVID-19 pandemic forced the transition to a fully online educational system throughout the world. This led to a decrease in student engagement and motivation, while anxiety has increased because of this fast change. Anxiety continued to increase when students experienced a lack of learning atmosphere during online classes, the lack of a collaborative face-to-face environment, and feelings of loneliness caused by the lack of socialization [1]. From this comes the need for online educational platforms that address these issues.

Compared to other countries that continued the online educational process, in Romania, a hybrid educational system has been employed after six months of the COVID-19 pandemic. Universities have been allowed to manage the return of students to the university and, in the case of an increase in the number of COVID-19 infections, to impose the online educational system for all students. There are also existing some similar works analyzing the perceptions of students related to the COVID-19 pandemic in Romania [2,3,4,5], but there is a lack of studies proposing new solutions regarding the future use and development of new online educational platforms. Regarding the experiences of other countries from the same region of Central and Eastern Europe as Romania, most of them faced COVID-19 with “emergency remote teaching” at the beginning (spring 2020), when teachers and students have been trained to use Microsoft Teams, Google Classroom, and Zoom platforms. In the Czech Republic and Slovakia, afterward, the learning management system (LMS) Moodle has been commonly considered (autumn 2020), as argued by Guncaga et al. [6]. Although Ukraine has experienced the need to implement different forms of distance learning in the past with the 2009 flu quarantine and 2015–2016 online teaching due to low temperatures, it did not fully incorporate distance learning [7]. Croatia struggled with too many ICT tools and platforms such as Loomen, Google classroom, Yammer, Microsoft Teams, YouTube videos, Office 365 tools, Genially, wizer.me, bookwidgets, Powtoon, Geogebra, and so on. Baksa and Luić [8] considered that a correlation between remote teaching, the duration of the tasks assigned to students, and the real-time lesson plan is mandatory. In Hungary and Serbia, the popularity of online education has grown, and even completely online environments have been developed before the pandemic. In Hungary, the K-MOOC project (Carpathian Basin Online Education Center) offered online courses in Hungarian for the higher education system and summed 46 courses from 2016 until 2020, and in Serbia, the ITACADEMY also provided completely online courses [9].

Mitescu-Manea et al. chose Romania, Bulgaria, the Republic of Moldova, and Hungary in their analysis of the education policy debates during the first wave of COVID-19. In all of these countries in the region of Central and Eastern Europe, the policies considered two main directions, one focusing on governing “through” the crisis with a strong orientation toward complying with sanitary measures, and the other using the crisis as an opportunity to intervene to reduce inequities in the educational system [10]. The authorities of Bulgaria, the Czech Republic, and Hungary made changes through new guidelines for the adaptation of the learning process and evaluation, while the education-oriented guidelines of Romania focused the most on addressing delays in student learning [11]. However, until now, there is no clear overview of which Central Eastern European country adapted the educational system in the most successful way during the COVID-19 pandemic. The results of the decision taken by the authorities can be better evaluated in the future, especially when the students who started university during the pandemic crisis will graduate and face the demands of the labor market. This research continues the work of Pop et al. [12] and uses the same online questionnaire (

Table A1. Questionnaire designed according to Kano methodology to identify the student’s perceptions about the functionalities of a new online educational platform (dataset) * [59].

Faculty:
Year of Study:
□ I acknowledge that no personal data will be collected by completing this form.
□ I acknowledge that the options expressed in this form will only be analyzed and used for scientific purposes.
Students engagement requirements
1. How do you feel if an online educational platform:
	I like it that way	It must be that way	I am neutral	I can live with it that way	I dislike it that way
PROVIDES personalized suggestions for other educational resources (e.g., other online courses, articles in scientific journals, practical applications, etc.) based on previous enrollments in various courses?	○	○	○	○	○
DOES NOT PROVIDE personalized suggestions for other educational resources (e.g., other online courses, articles in scientific journals, practical applications, etc.) based on previous enrollments in various courses?	○	○	○	○	○
2. How do you feel if an online educational platform:
	I like it that way	It must be that way	I am neutral	I can live with it that way	I dislike it that way
PROVIDES quizzes during courses (live/recorded courses) with a chance to win badges?	○	○	○	○	○
DOES NOT PROVIDE quizzes during courses (live/recorded courses) with a chance to win badges?	○	○	○	○	○
3. How do you feel if an online educational platform:
	I like it that way	It must be that way	I am neutral	I can live with it that way	I dislike it that way
PROVIDES the possibility of using VR (Virtual Reality) technology in laboratory experiments?	○	○	○	○	○
DOES NOT PROVIDE the possibility of using VR (Virtual Reality) technology in laboratory experiments?	○	○	○	○	○
4. How do you feel if an online educational platform:
	I like it that way	It must be that way	I am neutral	I can live with it that way	I dislike it that way
PROVIDES the possibility to evaluate each course (course rating)?	○	○	○	○	○
DOES NOT PROVIDE the possibility to evaluate each course (course rating)?	○	○	○	○	○
Technical and security requirements
5. How do you feel if an online educational platform:
	I like it that way	It must be that way	I am neutral	I can live with it that way	I dislike it that way
PROVIDES automatic real-time subtitle generation during the course?	○	○	○	○	○
DOES NOT PROVIDE automatic real-time subtitle generation during the course?	○	○	○	○	○
6. How do you feel if an online educational platform:
	I like it that way	It must be that way	I am neutral	I can live with it that way	I dislike it that way
PROVIDES the same functionalities on any device (desktop, tablet, mobile phone)?	○	○	○	○	○
DOES NOT PROVIDE the same functionalities on any device (desktop, tablet, mobile phone)?	○	○	○	○	○
7. How do you feel if an online educational platform:
	I like it that way	It must be that way	I am neutral	I can live with it that way	I dislike it that way
BLOCKS screenshot and video desktop recording services while the online platform is open?	○	○	○	○	○
DOES NOT BLOCK screenshot and video desktop recording services while the online platform is open?	○	○	○	○	○
8. How do you feel if an online educational platform:
	I like it that way	It must be that way	I am neutral	I can live with it that way	I dislike it that way
PROVIDES the availability on the platform of the course recording after it took place?	○	○	○	○	○
DOES NOT PROVIDE the availability on the platform of the course recording after it took place?	○	○	○	○	○
9. How do you feel if an online educational platform:
	I like it that way	It must be that way	I am neutral	I can live with it that way	I dislike it that way
PROVIDES automatic registration of student attendance after the full viewing of the course in live format or later based on its registration available on the platform?	○	○	○	○	○
DOES NOT PROVIDE automatic registration of student attendance after the full viewing of the course in live format or later based on its registration available on the platform?	○	○	○	○	○
Communication
10. How do you feel if an online educational platform:
	I like it that way	It must be that way	I am neutral	I can live with it that way	I dislike it that way
PROVIDES options for creating and customizing a user profile (the profile will contain fields such as: name, surname, profile picture, cover photo, interests, appreciations, badges obtained)?	○	○	○	○	○
DOES NOT PROVIDE options for creating and customizing a user profile (the profile will contain fields such as: name, surname, profile picture, cover photo, interests, appreciations, badges obtained)?	○	○	○	○	○
11. How do you feel if an online educational platform:
	I like it that way	It must be that way	I am neutral	I can live with it that way	I dislike it that way
PROVIDES messaging service with other users (students, teachers, secretariat)?	○	○	○	○	○
DOES NOT PROVIDE messaging service with other users (students, teachers, secretariat)?	○	○	○	○	○
12. How do you feel if an online educational platform:
	I like it that way	It must be that way	I am neutral	I can live with it that way	I dislike it that way
PROVIDES a discussion forum to which both students and teachers have access?	○	○	○	○	○
DOES NOT PROVIDE a discussion forum to which both students and teachers have access?	○	○	○	○	○
13. How do you feel if an online educational platform:
	I like it that way	It must be that way	I am neutral	I can live with it that way	I dislike it that way
PROVIDES the possibility to create discussion groups between any categories of users (students, teachers, administrative—secretariat)?	○	○	○	○	○
DOES NOT PROVIDE the possibility to create discussion groups between any categories of users (students, teachers, administrative—secretariat)?	○	○	○	○	○
Requirements for access to administrative resources
14. How do you feel if an online educational platform:
	I like it that way	It must be that way	I am neutral	I can live with it that way	I dislike it that way
PROVIDES access to the updated school situation (e.g., grades, credits obtained)?	○	○	○	○	○
DOES NOT PROVIDE access to the updated school situation (e.g., grades, credits obtained)?	○	○	○	○	○
15. How do you feel if an online educational platform:
	I like it that way	It must be that way	I am neutral	I can live with it that way	I dislike it that way
PROVIDES digitized administrative communication (description: allows the upload of various documents—study contracts, applications for the issue of certificates, the certificates issued by the secretariat, etc.)?	○	○	○	○	○
DOES NOT PROVIDE digitized administrative communication (description: allows the upload of various documents—study contracts, applications for the issue of certificates, the certificates issued by the secretariat, etc.)?	○	○	○	○	○
16. How do you feel if an online educational platform:
	I like it that way	It must be that way	I am neutral	I can live with it that way	I dislike it that way
PROVIDES the possibility to pay online the study fees (e.g., study fee for those enrolled in the form with fee, failed exams fees, fee for the 3rd/special presentation)?	○	○	○	○	○
DOES NOT PROVIDE the possibility to pay online the study fees (e.g., study fee for those enrolled in the form with fee, failed exams fees, fee for the 3rd/special presentation)?	○	○	○	○	○
17. How do you feel if an online educational platform:
	I like it that way	It must be that way	I am neutral	I can live with it that way	I dislike it that way
PROVIDES the possibility to report administrative problems directly to the responsible person (e.g., electrician, IT technician, carpenter, cleaning manager if the transmission is made from the faculty or the education is in a hybrid system)?	○	○	○	○	○
DOES NOT PROVIDE the possibility to report administrative problems directly to the responsible person (e.g., electrician, IT technician, carpenter, cleaning manager if the transmission is made from the faculty or the education is in a hybrid system)?	○	○	○	○	○

* Importance-related questions are not considered for current research and were not specified in this table.

) for further data collection and analysis. The study of Pop et al. [12] among students of the Politehnica University of Timisoara managed to receive 57 responses from five faculties and various years of study (from the first year of the Bachelor’s study program to the second year of the Master’s study program). The initial study analyzed the results collected between November and December 2021. The questionnaire was shared with the students via Zoom during online activities and through Facebook groups. The interesting results of the analysis performed by Pop et al. [12] opened the opportunity for the extension of that study and the improvement of the applied methodology.

The current research presents the results collected in the same manner from November 2021 to April 2022 among the students of the Politehnica University of Timisoara and additionally discusses from a statistical perspective the characteristics of the respondents. In this regard, this study neglected the faculties from which the number of responses was not considered statistically valid compared to the corresponding population (i.e., the total number of students enrolled at that faculty).

1.1. Context and Research Purpose

The COVID-19 pandemic caused a forced and immediate change in the Romanian educational system at all levels. The Politehnica University of Timisoara also applied legal policies that comply with pandemic restrictions and moved all educational activities of the university year 2019–2020 to a fully online format, initially for two weeks starting from 11 March 2020 [13]. The increased number of COVID-19 infections led to an extension of this fully online education period, until the end of the university year. The Politehnica University of Timisoara permanently monitored the national evolution of the pandemic and adapted its decisions regarding the educational process according to the legal restrictions decided by the national government. In this sense, the students joined a hybrid educational process aimed at increasing student engagement and developing collaborative relationships between the students. This hybrid educational system was first adopted on 14 September 2020 by the Senate of the Politehnica University of Timisoara for the university year 2020–2021 [14]. Due to several waves of COVID-19 infections, it was mandatory to switch several times between fully online and hybrid educational processes and vice versa.

This context shows that in both educational processes, the online educational aspect is present. Here, arises the need for online educational platforms that can offer access to educational materials, support the evaluation process, and at the same time keep students engaged, especially in the time of a pandemic.

The purpose of this research is to evaluate the proposal for online education that contains the features described previously by Pop et al. [12] and to address the following research questions:

• How are the proposed features perceived by students from the Politehnica University of Timisoara?
• Are there significant differences in perceptions based on the students’ field of study?
• Is there a correlation between student perceptions and their previous experience with the online educational process?

1.2. Related Works

The Kano methodology [15] applies to various domains as a mechanism to identify the features of a product or a service that will delight potential customers. Wu and Lin [16] mentioned this method as suitable for future use in assessing student expectations with e-learning. Fujs et al. [17] applied the Kano methodology to analyze possible efficiency improvements in the use of remote conference tools in higher education. Even if a statistical test was not performed on the results of the analysis, a great contribution represents the improvement of the Berger et al. [18] approach consisting of the incorporation of reverse Kano quality categories in the calculation of satisfaction and dissatisfaction coefficients. Several studies neglected the use of statistical validation in Kano evaluation results [12,17,19,20,21,22]. However, a small number of studies applied a validation based on the Fong test [23,24] or Pearson’s coefficient correlation [25].

Some recent studies discussed the psychological impact of the COVID-19 pandemic and the use of online educational platforms. The rapid change to the online educational system caused anxiety in the student community. This can be seen as the result of student concerns about their health, the social distancing restrictions that deprived them of university life experiences [5], the existence of computer anxiety (technophobia) in the case of e-learning [26], or the lack of a learning atmosphere in online classes [1]. Anxiety also has a strong connection to online educational platforms, especially with the use of the gamification concept. Wang and Tahir [27] took as a case study the analysis of Kahoot! use, which is debatable among researchers in terms of influence on student anxiety. They found that many existing studies reported a reduction in student anxiety, and only two studies that were also statistically validated “showed a statistically significant reduction of student anxiety, and one study reported that Kahoot! could produce agitation” [27].

The impact of the use of online educational platforms on student achievements represents a key factor in an online or hybrid/blended educational system. Learning motivation and student engagement directly influence their achievements, especially on educational platforms based on VR [28] or augmented reality (AR) [29,30]. Furthermore, these learning platforms facilitate the achievement and improvement of student soft skills such as time management, commitment, collaboration, teamwork, and creative, critical, and affective thinking [31,32,33,34,35].

Other studies focus on the practical implementation of online educational platforms from a technical perspective. Liu et al. [36] proposed the use of cloud computing in the development of a new online educational platform as an efficient mechanism to store and provide educational resources through large-scale distributed environments for student communities. The “Hstar” platform also used cloud services to provide hardware infrastructure services and the main functional features (e.g., online forum, assignment, curriculum resources, classroom management, etc.) [37]. “Aoki Cloud” uses cloud technology in the development process of an online educational platform customized to address business domain needs [38]. This customized platform allows small and medium companies to train their future employees at relatively affordable costs, providing a win-win environment for both trainees and businesses.

Sometimes, it is better to focus on needs and development costs rather than blindly following recent trends in technical solutions. As Yuan [39] argued, “technology application should focus on the time demand, effective application and cost-effectiveness”, and Web-based solutions are preferred in some cases instead of the current trend of cloud computing resource platforms. Chen et al. [40] addressed the need for decentralized online education platforms that use blockchain technology to manage educational resources. The strengths of this solution consist of providing a more secure environment for users and resource storage and solving existing issues on many of the current platforms related to centralized management of educational resources.

Abuhassna et al. [41] applied “Transactional Distance Theory (TDT) and Bloom’s Taxonomy Theory (BTT) to investigate the levels of student satisfaction and academic achievements by analyzing the background, experience, collaborations, interaction, and autonomy of students” [12]. The outcome of this research was to propose a guideline for the development of online educational platforms. They also highlighted the importance of previous experience with such a platform and the good knowledge of the class instructor about the features of the online platform. Zhou et al. [42] analyzed the intention of students to use an online educational platform from the perspective of four external variables “Online Course Design, Perceived System Quality, and Perceived Enjoyment, along with an additional perceived variable (Perceived Interaction)”.

An important step in identifying student perceptions is the chosen data acquisition method. Many researchers prefer the use of questionnaires as a method of collecting data for the analysis of student perceptions related to the evaluation of the learning process. Emanuel et al. [32] used questionnaires to assess the influence of an online educational platform on the achievement of soft skills. Ibáñez et al. [29] also used this method to collect data needed to characterize the impact of AR on the academic achievement and motivation of students. In addition to collecting data for a quantitative analysis, Teo et al. [43] used the focus-group interview method for the qualitative analysis. Potra et al. [5] performed a qualitative analysis based on the grounded theory approach and adopted the asynchronous online interview method for data acquisition for safety reasons during the COVID-19 pandemic. Iosif et al. [4] used a Google Forms questionnaire to identify the perceptions of students related to psychological and educational impact during the COVID-19 pandemic.

2. Materials and Methods

To get a better overview of the methodology applied in this research, Figure 1 illustrates the steps followed in this research, from the formulation of research questions to their completion. After defining the research questions, the next step is to identify the potential features of a new online educational platform. The list of these features is also used in the preparation of an online questionnaire that contains questions and possible response options consistent with the Kano methodology, which will be described in detail in Section 2.3.

After sharing the designed questionnaire with the student community, the analysis of the collected data begins. The first step in this analysis consists of identifying the total population and calculating the sample size, which provides a better statistical overview of the relevance of this study from the perspective of confidence level and average error. Section 2.4 describes this statistical methodology.

The Kano evaluation represents the second step of the analysis of the results and aims to classify the proposed feature according to the approach detailed in Section 2.3. This classification provides an overview of the student’s expectations regarding the online educational process and emphasizes the features worth investing in. In addition to the previous statistical analysis, this Kano evaluation performs a statistical test on the classification of features using the Kano quality attributes. This significance test, usually neglected by most Kano-based evaluation studies, aims to provide a validation of the Kano classification. In this way, this research provides a double validation of the results obtained (statistical check based on population characteristics and statistical check based on the relevance of the collected results by applying the Fong test [44]).

The last step in the proposed research methodology is to verify the relevance of the significant results obtained from the analysis of the data collected with the formulated research question. Discussions in this regard will be made in Section 4.

2.1. Identification of Potential Features

This article uses potential features defined in a previous study by Pop et al. [12].

Table 1. Definition of the features for a new proposed online educational platform—entirely retrieved from Pop et al. [12] (pp. 10373–10374).

Feature No.	Features	Definition	Category
1.	New educational resources suggestion	personalized suggestions for other educational resources (e.g., other online courses, articles in scientific journals, practical applications, etc.) based on previous enrollments in various courses	Students engagement requirements
2.	Interactive quizzes (intrinsic motivation)	quizzes during courses (live/recorded courses) with a chance to win badges
3.	VR enabling laboratory experiments	the possibility of using VR technology in laboratory experiments
4.	Course evaluation	the possibility of evaluating each course (course rating)
5.	Automatic real-time subtitles generation	automatic real-time subtitle generation during the course	Technical requirements
6.	Multiple devices availability (desktop, tablet, mobile phone)	the same functionalities on any device (desktop, tablet, mobile phone)
7.	Blocking unauthorized recording/screenshots	blocks screenshot and video desktop recording services while the online platform is open
8.	Course recording availability	the availability on the platform of the course recording after it took place
9.	Automatic presence after full visualisation of live/recorded course	automatic registration of student attendance after full viewing of the course in live format or later based on its registration available on the platform
10.	Customized user profile	provides options for creating and customizing a user profile (the profile will contain fields such as name, surname, profile picture, cover photo, interests, appreciations, badges obtained)	Communication tools
11.	Messaging service	provides a messaging service with other users (students, teachers, secretariat)
12.	Discussions forum	discussion forum to which both students and teachers have access
13.	Groups creation service	the possibility to create discussion groups between any categories of users (students, teachers, administrative—secretariat)
14.	Access to the updated scholar situation	access to the updated school situation (e.g., grades, credits obtained)	Requirements for access to administrative resources
15.	Online administrative documents/requests evidence and submission	digitized administrative communication (description: allows the upload of various documents—study contracts, applications for the issue of certificates, the certificates issued by the secretariat, etc.)
16.	Online studies related payments possibility	the possibility to pay online the study fees (e.g., study fee for those enrolled in the form with a fee, failed exams fees, the fee for the 3rd/special presentation)
17.	Administrative responsible-based feedback	the possibility to report administrative problems directly to the responsible person (e.g., electrician, IT technician, carpenter, cleaning manager if the transmission is made from the faculty or the education is in a hybrid system)

presents the list of features, their description, and a classification of them into four categories according to their purpose. In addition to standard features, such as availability on multiple types of devices, custom profiles, communication-related features, and features related to administrative resources were proposed. In addition, other functionalities that influence student engagement and motivation were identified based on the related scientific literature, and the reason behind these choices is presented further.

Student engagement represents a big challenge in the design of a new online educational platform. The difficulty of keeping students engaged becomes greater in a fully online educational system because it is harder to monitor and control their activities. The face-to-face learning system facilitates the identification of specific characteristics that influence student engagement, and professors can directly appeal to their didactic and pedagogical competencies.

A good way to engage students during online learning periods is to implement game-based features that have proven their efficiency in various cases [43,45,46,47]. For this reason, this research aims to address the intrinsic motivation of students by implementing a reward system based on quizzes. This allows the students to answer some questions during the courses (a feature also present for the students who are watching the course recording) and gives them the chance to win badges. In this way, the student’s attention and engagement during course activities can be increased from a competitive perspective. Ross et al. [48] proposed the use of adaptive quizzes that attempt to address the individual needs of each student. As expected, this approach did not influence student scores and achievements because final evaluations consider all concepts presented for a discipline, and these customized quizzes allowed the neglect of concepts that did not arouse students’ curiosity. However, an increase in student motivation and engagement was observed. As a result of the research by Göksün and Gürsoy [47] who compared two quiz-based assessment tools (i.e., Kahoot! and Quizizz), characteristics such as “presentation of questions, feedbacks, progression speed and method of the questions, technical requirements” can influence student achievement and engagement.

The level of student satisfaction and their skills can increase by implementing the virtual reality feature. Even if VR usage increases engagement through spatial presence in a mediated learning environment, this has a negative impact on information recall [28]. As argued by Ahn et al. [28], this has a strong connection with individual differences in terms of user experience with spatial presence and is independent of VR content segmentation. The application of this approach to engineering classes showed that it positively influenced communication between students and teamwork skills and increased their motivation to develop their practical skills [49]. A similar study applied the theory of intrinsic motivation in the case of using AR [30]. The researchers developed an AR-based mobile educational application and collected feedback from 78 students using questionnaires before and after the use of the created application. The results show that the use of AR technology increased user satisfaction, attention, and confidence, keeping them motivated during the learning process [29,30].

The availability of video recordings after the courses took place is also an important topic, even if there are concerns from both lecturing staff and students about this functionality [50]. During the COVID-19 pandemic, Schmitz et al. [51] made a comparison between video-based learning in the medical field and learning based only on textbooks provided in terms of student achievement. Video recordings were uploaded and made available to students on their university video-based online platform. The exam results show significantly higher scores for students who used the video-based online platform in their preparation for the exam instead of the classical method based on the usage of textbooks. The experiment performed by Fujs et al. [17] showed that the availability of the course recordings did not significantly influence attendance at live online presentations. On the other hand, the research by Morris et al. [52] identified a significant increase in attendance at lectures that were not recorded. Furthermore, they observed that “students have high expectations about the availability and quality of recordings” [52].

2.2. Design of the Questionnaire

The questionnaire was designed according to the Kano methodology (Section 2.3), creating two questions for each feature proposed in Table 1 to cover both its functional and dysfunctional characteristics, in short terms, the underlying of its presence and absence.

In addition to Kano-related questions, this questionnaire (see Table A1) asks for details about the faculty and year of study. Furthermore, the questionnaire also includes informed consent that no personal data will be collected and that the responses will be used only for research purposes.

2.3. Kano-Based Analysis

Kano et al. [15] proposed a methodology to characterize the non-linear relationship between customer satisfaction and the features of a product or service. This approach allows classification of the proposed features using the following quality attributes: attractive (A), one-dimensional (O), must-be (M), indifferent (I), reverse (R), and questionable (Q).

The attractive quality attribute (A) describes the customer’s excitement about a feature. Although it is unexpected and manages to surprise the potential customer, the implementation of this functionality will certainly lead to greater customer satisfaction. A one-dimensional quality attribute (O) emphasizes the clearest perception expressed by a potential customer through a straightforward reaction that describes that the absence of it will bring dissatisfaction; the potential customer likes to have this feature and dislikes not having it. The must-be quality attribute (M) shows the cases where the presence of a feature is tolerated, expected, or the potential customer feels neutral with respect to its presence but expresses straightforward disagreement about its absence. An indifferent quality attribute (I) describes the case of a feature that has no positive or negative impact on customer satisfaction, a clear reaction of “like” or “dislike” not being expressed by the potential customer for the presence or absence of a feature. The reverse quality attribute (R) expresses a backward influence on customer satisfaction [53], the reactions of a potential customer showing that he or she likes not having the feature or dislikes having it. The questionable quality attribute (Q) shows contradictions in the reactions of a potential customer to the existence or absence of a feature. According to Wittel et al. [54], this situation is related to skeptical reactions, and the understanding of the feature by a potential customer is debatable.

The Kano evaluation process illustrated in Figure 2 shows that the first step is to formulate two questions for each feature proposed for a new product or service to capture the reaction of a potential customer to the existence (functional question) or absence (dysfunctional question) of that feature. Each potential customer should answer both questions by rating their reaction using the following options: “I like it that way”, “It must be that way”, “I am neutral”, “I can live with it that way”, and “I dislike it that way”. The evaluation of each potential customer reaction is analyzed on the basis of the pair of reactions (functional, dysfunctional).

A pair (“I like it that way”, x), where x = {“It must be that way”, “I am neutral”, “I can live with it that way”} leads to a categorization of that feature as attractive (A). The pair (“I like it that way”, “I dislike it that way”) categorizes the evaluated feature as one-dimensional (O). A pair (x, “I dislike it that way”), where x = {“It must be that way”, “I am neutral”, “I can live with it that way”} classifies the feature as must-be (M). Classification of a feature as reverse (M) results from the pairs (“I like it that way”, x), where x = {“It must be that way”, “I am neutral”, “I can live with it that way”, “I dislike it that way”}, or (x, “I dislike it that way”), where x = {“I like it that way”, “It must be that way”, “I am neutral”}. A feature is classified as questionable (Q) if reactions consist of pairs (“I like it that way”, “I like it that way”) or (“I dislike it that way”, “I dislike it that way”). The other remaining pairs of reactions lead to the classification of that feature as indifferent (I).

After counting the frequency of each quality attribute for each feature, the quality attribute with the maximum frequency will generally describe the reactions of potential customers for a certain feature. If two quality attributes are tied in the scoring of a given feature, Berger et al. [18] recommended the selection of the category that would have the greatest impact on customer satisfaction, determined by the following ordering: M > O > A > I. In addition, identification of market segmentation differences and follow-ups with potential customers can be carried out to identify additional information.

Berger et al. [18] proposed the usage of the satisfaction index (SI), also known as better, and the dissatisfaction index (DI), also known as worse, in the ranking of the features proposed for a new product or service. The results obtained from the calculation of SI using Equation (1) [18] indicate the level of satisfaction corresponding to each feature, the highest values being achieved by providing the features classified as A and O after performing the Kano evaluation.

$$\mathrm{SI}=\frac{\mathrm{A}+\mathrm{O}}{\mathrm{A}+\mathrm{O}+\mathrm{M}+\mathrm{I}}$$

(1)

Equation (2) [18] describes the calculation of DI, the lowest values obtained being strongly related to the missing features classified as O and M according to the Kano methodology.

$$\mathrm{DI}=\left(-1\right) \times \frac{\mathrm{O}+\mathrm{M}}{\mathrm{A}+\mathrm{O}+\mathrm{M}+\mathrm{I}}$$

(2)

A general overview of the proposed product or service is determined by computing the total satisfaction index (TSI), defined in Equation (3) [18]. This will provide a more accurate overview of the features that deserve investment in the development stage and should be present in the final product or service. Negative results indicate that the lack of those features will bring dissatisfaction to the potential customer, while positive results are related to the features that will provide satisfaction through their presence. Following this approach, the features with higher TSI values yield a greater impact.

$$\mathrm{TSI}=\mathrm{SI}+\mathrm{DI}$$

(3)

The interpretation of TSI values also provides a validation of the Kano evaluation results by also considering the values of the other categories in addition to the category that got the most reactions. Following this approach, a TSI < 0 describes a one-dimensional (O) feature, a TSI = 0 shows a reverse (R) feature, and if TSI > 0.1, the feature is attractive (A).

These metrics have been further researched and improved. Matzler et al. [56] introduced the notion of positive customer satisfaction (CS) for SI and negative CS for DI. Later, Jang et al. [57] introduced the average satisfaction coefficient (ASC) to describe the general impact of each feature. A drawback of this metric, defined by Equation (4), arises in the case of functionalities with low SI and high DI, where the ASC for that will be higher than for other features with higher SI and lower DI. For this reason, these metric results will not be analyzed in this research.

$$\mathrm{ASC}=\frac{\left|\mathrm{SI}\right|+ \left|\mathrm{DI}\right|}{2}$$

(4)

The statistical significance of Kano evaluation is another important aspect that unfortunately is missing in several studies aiming to assess the quality attributes of various products or services. This metric has been proposed by Fong [44] and is defined as follows [24]:

$$\left|\mathrm{a} - \mathrm{b}\right|{ < \mathrm{z}}_{\mathsf{\alpha }} \times \sqrt{\frac{\left(\mathrm{a}+\mathrm{b}\right) \times \left(2 \times \mathrm{n} - \mathrm{a} - \mathrm{b}\right)}{2 \times \mathrm{n}}}$$

(5)

where a represents the total frequency of the Kano attribute given most often, b represents the total frequency of the Kano attribute with the second scoring, ${\mathrm{z}}_{\mathsf{\alpha }}$ is the standard deviation corresponding to the confidence level $\mathsf{\alpha }$, and n represents the number of respondents.

Only the features identified as significant show that the study is relevant to that feature and that the quality attribute was identified with high precision.

2.4. Data Acquisition and Analysis

Data collection was carried out using the online questionnaire Table A1 designed according to the first step of the Kano methodology previously presented. The online questionnaire was shared with students via Zoom during laboratory activities and through Facebook groups. In the university year 2021–2022, the online questionnaire was shared with the students only during laboratory activities (Zoom video conferences or face-to-face in some cases, depending on university decisions related to the educational process [13,14]) to avoid receiving answers from the same students interviewed in the university year 2020–2021.

Taking into account the research questions defined previously for this study, it is mandatory to provide a statistical overview of the interviewed population. The determination of the sample size uses an optimal stratified survey similar to the methodology applied by Potra et al. [5]. Equation (6) defines the division of a population N into k sub-populations, further called layers.

$$\mathrm{N}={{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{k}}{\mathrm{N}}_{\mathrm{i}}{=\mathrm{N}}_1{+\mathrm{N}}_2{+\dots +\mathrm{N}}_{\mathrm{k}}$$

(6)

The stratified survey depends on the dimension of the ${\mathrm{n}}_{\mathrm{k}}$random levels of each layer according to Equation (7).

$${\mathrm{n}}_1{+\mathrm{n}}_2{+\dots +\mathrm{n}}_{\mathrm{k}}={{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{k}}{\mathrm{n}}_{\mathrm{i}}=\mathrm{n}$$

(7)

The determination of the final value of the sample size n considers the linear characteristics as follows [58]:

$$\mathrm{n} \cong \frac{{\mathrm{N} \times \mathrm{z}}_{1-\mathsf{\alpha }}^2{ \times \mathsf{\sigma }}_{\mathrm{b}}^2}{{\mathrm{N} \times \mathsf{\delta }}^2{+\mathrm{z}}_{1-\mathsf{\alpha }}^2{ \times \mathsf{\sigma }}_{\mathrm{b}}^2}$$

(8)

where:

• ${\mathrm{z}}_{1-\mathsf{\alpha }}$ is the value of the Laplace variable for a probability $1 - \mathsf{\alpha }$, α representing the significance level;
• ${\mathsf{\sigma }}_{\mathrm{b}}^2$ is the variance of the binary characteristic, computed as ${\mathsf{\sigma }}_{\mathrm{b}}^2=\mathrm{p} \times \left(1 - \mathrm{p}\right)$;
• ${\mathsf{\delta }}^2$ represents the probable error.

To ensure the reduction of statistical uncertainty, this research applies maximum variance, and the sample design considers the worst-case scenario, consisting of a binary characteristic’s variance ${\mathsf{\sigma }}_{\mathrm{b}}^2=0.5 \times \left(1 - 0.5\right)=0.25$.

The optimal value results from dimensioning the ${\mathrm{n}}_{\mathrm{j}}$level at a maximum efficiency level using Equation (9).

$${\mathrm{n}}_{\mathrm{j}}=\frac{{\mathrm{n} \times \mathrm{N}}_{\mathrm{j}}{ \times \hat{\mathsf{\sigma }}}_{\mathrm{j}}}{{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{k}}{\mathrm{N}}_{\mathrm{i}}{ \times \hat{\mathsf{\sigma }}}_{\mathrm{i}}},\hspace{1em}\mathrm{j}=\left\{1, 2, \dots , \mathrm{k}\right\}$$

(9)

3. Results

This section provides an overview of the results obtained. It starts with a statistical characterization of the students who answered the questionnaire in Table A1 and continues with the presentation of the collected data and their classification based on the Kano methodology. Furthermore, this section also presents the calculated satisfaction metrics and emphasizes the relevance of this study based on the application of the Fong test [24,44].

3.1. Distribution of the Collected Data

Similarly to the research by Potra et al. [5], the analysis of the results begins with the calculation of the sample size, which will provide a better description of the distribution of the respondents.

Table 2. Distribution of questionnaire respondents (dataset) [59].

Layers	Description	Number of Respondents	Percentage (%)
${\mathrm{n}}_1$	Automation and Computing (AC)—university year 2020–2021	183	46.45
${\mathrm{n}}_2$	Management in Production and Transportation (MPT)—university year 2020–2021	124	31.47
${\mathrm{n}}_3$	Automation and Computing (AC)—university year 2021–2022	52	13.20
${\mathrm{n}}_4$	Management in Production and Transportation (MPT)—university year 2021–2022	35	8.88
Total		394	100

shows the distribution of the students in four layers ${\mathrm{n}}_1$, ${\mathrm{n}}_2$, ${\mathrm{n}}_3$, and ${\mathrm{n}}_4$; the calculated sample size is n = 394 students (number of respondents to the questionnaire from Table A1).

The total population is approximately N = 6000 students if the total number of AC and MPT students is summed. The value approximately considers the approach that at each time during this study, the number of students remained approximately constant (the students who graduated after the university year 2020–2021 were replaced by the same number of first-year students in the university year 2021–2022). This research considers a significance level α = 0.05, the confidence level being (1 − α = 95%), and therefore, ${\mathrm{z}}_{0.95}=1.96$. The worst-case scenario was considered in the sample design process; therefore, the variance of the binary characteristic is ${\mathsf{\sigma }}_{\mathrm{b} }^2=0.50 \times \left(1 - 0.50\right)=0.25$. This approach leads to an average error of 4.77% for our study.

3.2. Kano Evaluation Results

The data collected were analyzed according to the Kano methodology described in Section 2.3. Kano-based analysis.

Table 3. Distribution of the student’s reactions to the defined features for a new online educational platform for the period 2020–2022 (dataset) [59].

Feature No.	Features	A	M	I	O	R	Q
1.	New educational resources suggestion	65	14	70	28	4	2
2.	Interactive quizzes (intrinsic motivation)	62	6	82	19	13	1
3.	VR enabling laboratory experiments	84	7	53	36	2	1
4.	Course evaluation	45	26	58	47	6	1
5.	Automatic real-time subtitles generation	46	5	99	32	1	0
6.	Multiple devices availability (desktop, tablet, mobile phone)	37	24	27	95	0	0
7.	Blocking unauthorized recording/screenshots	1	1	50	3	127	1
8.	Course recording availability	24	25	17	117	0	0
9.	Automatic presence after full visualisation of live/recorded course	63	13	71	33	2	1
10.	Customized user profile	42	6	117	17	1	0
11.	Messaging service	34	25	50	73	0	1
12.	Discussions forum	39	15	65	64	0	0
13.	Groups creation service	55	10	69	48	0	1
14.	Access to the updated scholar situation	13	31	14	125	0	0
15.	Online administrative documents/requests evidence and submission	17	31	27	107	1	0
16.	Online studies related payments possibility	28	24	23	107	1	0
17.	Administrative responsible-based feedback	39	20	76	48	0	0

shows a parallel overview of the data collected in two successive university years for two engineering programs of studies, one more technical and the other oriented to the management area according to the description from Table 2. A detailed overview of the distribution for each layer is available in the Appendix A (

Table A2. Distribution of the students’ reactions to the defined features for a new online educational platform for the university year 2020–2021 (dataset) [59].

Feature No.	$\mathbf{Layer} {\mathbf{n}}_1$						$\mathbf{Layer} {\mathbf{n}}_2$
	A	M	I	O	R	Q	A	M	I	O	R	Q
1.	65	14	70	28	4	2	11	3	10	10	0	1
2.	62	6	82	19	13	1	11	2	10	10	1	1
3.	84	7	53	36	2	1	15	1	7	12	0	0
4.	45	26	58	47	6	1	7	4	12	11	1	0
5.	46	5	99	32	1	0	8	4	12	11	0	0
6.	37	24	27	95	0	0	8	7	3	17	0	0
7.	1	1	50	3	127	1	0	2	4	0	29	0
8.	24	25	17	117	0	0	5	5	4	21	0	0
9.	63	13	71	33	2	1	7	3	14	9	2	0
10.	42	6	117	17	1	0	10	4	15	4	2	0
11.	34	25	50	73	0	1	8	6	7	14	0	0
12.	39	15	65	64	0	0	7	1	11	15	1	0
13.	55	10	69	48	0	1	5	1	16	13	0	0
14.	13	31	14	125	0	0	2	3	1	29	0	0
15.	17	31	27	107	1	0	3	6	6	20	0	0
16.	28	24	23	107	1	0	5	4	6	20	0	0
17.	39	20	76	48	0	0	3	3	13	14	1	1

and

Table A3. Distribution of the students’ reactions to the defined features for a new online educational platform for the university year 2021–2022 (dataset) [59].

Feature No.	$\mathbf{Layer} {\mathbf{n}}_3$						$\mathbf{Layer} {\mathbf{n}}_4$
	A	M	I	O	R	Q	A	M	I	O	R	Q
1.	13	7	21	10	0	1	11	3	10	10	0	1
2.	15	1	21	14	1	0	11	2	10	10	1	1
3.	24	2	11	12	3	0	15	1	7	12	0	0
4.	8	5	18	20	1	0	7	4	12	11	1	0
5.	17	2	15	16	2	0	8	4	12	11	0	0
6.	7	10	6	29	0	0	8	7	3	17	0	0
7.	1	1	12	0	38	0	0	2	4	0	29	0
8.	10	5	4	33	0	0	5	5	4	21	0	0
9.	12	3	18	19	0	0	7	3	14	9	2	0
10.	20	2	18	10	2	0	10	4	15	4	2	0
11.	12	5	11	24	0	0	8	6	7	14	0	0
12.	15	3	14	19	0	1	7	1	11	15	1	0
13.	17	3	25	7	0	0	5	1	16	13	0	0
14.	5	12	2	32	1	0	2	3	1	29	0	0
15.	10	12	7	22	1	0	3	6	6	20	0	0
16.	5	12	6	29	0	0	5	4	6	20	0	0
17.	13	8	17	14	0	0	3	3	13	14	1	1

).

The frequency of each Kano quality attribute for each feature was counted by analyzing the entire sample size of n = 394 students, considered as potential customers of the proposed online educational platform. Identification of the specific Kano category for each feature for the entire sample size is shown in

Table 4. Kano evaluation results and satisfaction metrics overview for the period 2020–2022 (dataset) [59].

Feature No.	${\mathbf{n}}_1{+\mathbf{n}}_3$				${\mathbf{n}}_2+{\mathbf{n}}_4$				$\mathbf{n}$
	Kano Category	SI	DI	TSI	Kano Category	SI	DI	TSI	Kano Category	SI	DI	TSI
1.	I	0.51	−0.26	0.25	A	0.57	−0.30	0.27	I	0.53	−0.27	0.26
2.	I	0.50	−0.18	0.32	I	0.54	−0.21	0.33	I	0.52	−0.19	0.33
3.	A	0.68	−0.25	0.43	A	0.68	−0.34	0.34	A	0.68	−0.28	0.40
4.	I	0.53	−0.43	0.10	I	0.54	−0.40	0.14	I	0.53	−0.42	0.11
5.	I	0.48	−0.24	0.24	I	0.56	−0.34	0.22	I	0.51	−0.28	0.23
6.	O	0.71	−0.67	0.04	O	0.60	−0.68	−0.08	O	0.67	−0.68	−0.01
7.	R	0.07	−0.07	0.00	R	0.12	−0.12	0.00	R	0.09	−0.09	0.00
8.	O	0.78	−0.77	0.01	O	0.62	−0.72	−0.10	O	0.72	−0.75	−0.03
9.	I	0.55	−0.29	0.26	O	0.59	−0.43	0.16	I	0.56	−0.35	0.21
10.	I	0.38	−0.15	0.23	I	0.46	−0.28	0.18	I	0.41	−0.20	0.21
11.	O	0.61	−0.54	0.07	O	0.56	−0.56	0.00	O	0.59	−0.55	0.04
12.	O	0.59	−0.43	0.16	I	0.55	−0.44	0.11	O	0.57	−0.43	0.14
13.	I	0.54	−0.29	0.25	I	0.55	−0.39	0.16	I	0.54	−0.33	0.21
14.	O	0.75	−0.85	−0.10	O	0.71	−0.79	−0.08	O	0.73	−0.83	−0.10
15.	O	0.67	−0.74	−0.07	O	0.67	−0.74	−0.07	O	0.67	−0.74	−0.07
16.	O	0.72	−0.74	−0.02	O	0.67	−0.73	−0.06	O	0.70	−0.73	−0.03
17.	I	0.49	−0.38	0.25	I	0.53	−0.46	0.07	I	0.50	−0.41	0.09

. Additionally, this table contains the satisfaction metrics (SI, DI, and TSI) calculated according to (1), (2), and (3). The TSI will be considered as the main metric in the section allocated for discussions of the results of this research. Identification of the specific Kano category and the calculated metrics for each feature for the four layers of the sample size are available in the Appendix A (

Table A4. Kano evaluation results and satisfaction metrics overview for the university year 2020–2021 (dataset) [59].

Feature No.	$\mathbf{Layer} {\mathbf{n}}_1$				$\mathbf{Layer} {\mathbf{n}}_2$				${\mathbf{n}}_1{+\mathbf{n}}_2$
	Kano Category	SI	DI	TSI	Kano Category	SI	DI	TSI	Kano Category	SI	DI	TSI
1.	I	0.53	−0.24	0.29	A	0.56	−0.27	0.29	A	0.54	−0.25	0.29
2.	I	0.48	−0.15	0.33	I	0.52	−0.16	0.36	I	0.49	−0.15	0.34
3.	A	0.67	−0.24	0.43	A	0.65	−0.33	0.32	A	0.66	−0.27	0.39
4.	I	0.52	−0.41	0.11	I	0.54	−0.39	0.15	I	0.53	−0.41	0.12
5.	I	0.43	−0.20	0.23	I	0.57	−0.32	0.25	I	0.49	−0.25	0.24
6.	O	0.72	−0.65	0.07	O	0.57	−0.68	−0.11	O	0.66	−0.66	0.00
7.	R	0.07	−0.07	0.00	R	0.14	−0.08	0.06	R	0.10	−0.08	0.02
8.	O	0.77	−0.78	−0.01	O	0.59	−0.71	−0.12	O	0.70	−0.75	−0.05
9.	I	0.53	−0.26	0.27	O	0.62	−0.45	0.17	I	0.57	−0.33	0.24
10.	I	0.32	−0.13	0.19	I	0.47	−0.29	0.18	I	0.38	−0.19	0.19
11.	O	0.59	−0.54	0.05	O	0.54	−0.56	−0.02	O	0.57	−0.55	0.02
12.	I	0.56	−0.43	0.13	I	0.52	−0.43	0.09	I	0.55	−0.43	0.12
13.	I	0.57	−0.32	0.25	I	0.56	−0.39	0.17	I	0.56	−0.35	0.21
14.	O	0.75	−0.85	−0.10	O	0.66	−0.75	−0.09	O	0.72	−0.81	−0.09
15.	O	0.68	−0.76	−0.08	O	0.67	−0.73	−0.06	O	0.68	−0.75	−0.07
16.	O	0.74	−0.72	0.02	O	0.65	−0.74	−0.09	O	0.71	−0.73	−0.02
17.	I	0.48	−0.37	0.11	I	0.54	−0.45	0.09	I	0.50	−0.40	0.10

and

Table A5. Kano evaluation results and satisfaction metrics overview for the university year 2021–2022 (dataset) [59].

Feature No.	$\mathbf{Layer} {\mathbf{n}}_3$				$\mathbf{Layer} {\mathbf{n}}_4$				${\mathbf{n}}_3+ {\mathbf{n}}_4$
	Kano Category	SI	DI	TSI	Kano Category	SI	DI	TSI	Kano Category	SI	DI	TSI
1.	I	0.45	−0.33	0.12	A	0.62	−0.38	0.24	I	0.52	−0.35	0.17
2.	I	0.57	−0.29	0.28	A	0.64	−0.36	0.28	I	0.60	−0.32	0.28
3.	A	0.73	−0.29	0.44	A	0.77	−0.37	0.40	A	0.75	−0.32	0.43
4.	O	0.55	−0.49	0.06	I	0.53	−0.44	0.09	O	0.54	−0.47	0.07
5.	A	0.66	−0.36	0.30	I	0.54	−0.43	0.11	I	0.61	−0.39	0.22
6.	O	0.69	−0.75	−0.06	O	0.71	−0.69	0.02	O	0.70	−0.72	−0.02
7.	R	0.07	−0.07	0.00	R	0.00	−0.33	−0.33	R	0.05	−0.15	−0.10
8.	O	0.83	−0.73	0.10	O	0.74	−0.74	0.00	O	0.79	−0.74	0.05
9.	O	0.60	−0.42	0.18	I	0.48	−0.36	0.12	I	0.55	−0.40	0.15
10.	A	0.60	−0.24	0.36	I	0.42	−0.24	0.18	I	0.53	−0.24	0.29
11.	O	0.69	−0.56	0.13	O	0.63	−0.57	0.06	O	0.67	−0.56	0.11
12.	O	0.67	−0.43	0.24	O	0.65	−0.47	0.18	O	0.66	−0.45	0.21
13.	I	0.46	−0.19	0.27	I	0.51	−0.40	0.11	I	0.48	−0.28	0.20
14.	O	0.73	−0.86	−0.13	O	0.89	−0.91	−0.02	O	0.79	−0.88	−0.09
15.	O	0.63	−0.67	−0.04	O	0.66	−0.74	−0.08	O	0.64	−0.70	−0.06
16.	O	0.65	−0.79	−0.14	O	0.71	−0.69	0.02	O	0.68	−0.75	−0.07
17.	I	0.52	−0.42	0.10	O	0.52	−0.52	0.00	I	0.52	−0.46	0.06

).

In addition to the statistical overview of the population interviewed using the Kano-based questionnaire (Table 2), this research also performs a significance test of the results obtained after the Kano evaluation by applying the Fong test according to Equation (5). This calculation considers a standard deviation ${\mathrm{z}}_{0.95}=1.96$ corresponding to a confidence level of 95%. For each layer of the sample, the worst-case scenario was considered by considering the total population as N = 6000 students. Consequently, the statistical evaluation in

Table 5. Significance * of the evaluation of features based on the Fong test. Not significant is represented as “no” and significant as “yes”.

Feature No.	Features	Significance
		$\mathbf{Layer} {\mathbf{n}}_1$	$\mathbf{Layer} {\mathbf{n}}_2$	$\mathbf{Layer} {\mathbf{n}}_3$	$\mathbf{Layer} {\mathbf{n}}_4$	Sample n
1.	New educational resources suggestion	no	no	no	no	no
2.	Interactive quizzes (intrinsic motivation)	yes	no	no	no	yes
3.	VR enabling laboratory experiments	yes	yes	yes	no	yes
4.	Course evaluation	no	no	no	no	no
5.	Automatic real-time subtitles generation	yes	no	no	no	yes
6.	Multiple devices availability (desktop, tablet, mobile phone)	yes	no	yes	yes	yes
7.	Blocking unauthorized recording/screenshots	yes	yes	yes	yes	yes
8.	Course recording availability	yes	yes	yes	yes	yes
9.	Automatic presence after full visualisation of live/recorded course	no	no	no	no	yes
10.	Customized user profile	yes	yes	no	no	yes
11.	Messaging service	yes	no	yes	no	yes
12.	Discussions forum	no	no	no	no	no
13.	Groups creation service	no	no	no	no	yes
14.	Access to the updated scholar situation	yes	yes	yes	yes	yes
15.	Online administrative documents/requests evidence and submission	yes	yes	yes	yes	yes
16.	Online studies related payments possibility	yes	yes	yes	yes	yes
17.	Administrative responsible-based feedback	yes	no	no	no	yes

* The complete analysis results are available in Table A6, Table A7 and Table A8.

shows the relevance, also called “significance”, of the Kano categories identified for each feature. This validates current research by focusing on positive and negative results from a relevance perspective. In practice, this highlights the features that need further research (e.g., market research, costs analysis, etc.) before taking a final decision regarding their implementation.

4. Discussion

This study used as input the responses of 394 students compared to the 57 responses collected and analyzed in the previous study by Pop et al. [12]. The results of the Kano evaluation for the entire study are illustrated in Table 3 and Table 4. The increase in sample size led to greater precision in identifying the right quality attribute. Previous research had deficiencies in the classification of feature 1 (i.e., New educational resources suggestion) and feature 13 (i.e., Groups creation service) due to the reduced number of responses that led to the equality of two quality attributes, attractive (A) and indifferent (I), respectively, one-dimensional (O) and indifferent (I). In this case, Berger et al. [18] recommended the selection of the Kano category considering that the following order in terms of customer satisfaction M > O > A > I would lead to a wrong implementation decision. The current study shows that features 1 and 13 can be clearly categorized as indifferent (I), and implementation costs can be reduced by neglecting these features. Feature 9 (i.e., Automatic presence after full visualisation of live/recorded course) and feature 17 (i.e., Administrative responsible-based feedback) were both classified as attractive (A) by Pop et al. [12] but with small differences to the second most chosen quality attribute (i.e., indifferent (I) in both cases) became also indifferent (I) features after the current Kano evaluation. This last change can also be the outcome of the alternation of fully online with a hybrid education system, with students’ lived experiences in both cases making them no longer interested in these features.

Figure 3 shows the general comparative overview of the expectations for the online educational platforms of the layers ${\mathrm{n}}_1$, ${\mathrm{n}}_2$, ${\mathrm{n}}_3$, and ${\mathrm{n}}_4$ of the chosen sample for the features that were significant after applying the Fong test (Table 5). In the Appendix A, Figure A1, Figure A2, Figure A3 and Figure A4 illustrate different comparisons between layers that will be discussed in Section 4. Discussion. Furthermore, the TSI was also considered for the entire sample size. These results demonstrate that the biggest needs of the students are related to the requirements describing their access to administrative resources, this category of requirements having most of the features rated as significant according to the Fong test. This is understandable because these needs are permanently independent of the chosen educational system (i.e., face-to-face, fully online, or hybrid) and the field of study.

The analysis of the TSI for feature 7 (i.e., Blocking unauthorized recording/screenshots) demonstrates a clear classification as reverse (R), for the students from AC (layers ${\mathrm{n}}_1$ and ${\mathrm{n}}_3$) and for the entire sample. Even if MPT students classified this feature as reverse (R) according to the Kano evaluation, the TSI analysis shows a deviation compared to the TSI values obtained for AC students. This shows a better overview and understandability of the possible consequences related to this feature by the background in computer and information technology of AC students. The change in TSI between the university year 2020–2021 and 2021–2022 of MPT students shows a strong relationship with their experience with the online educational system. In this way, the students represented in layer ${\mathrm{n}}_4$ are completely dissatisfied with the existence of this feature compared with layer ${\mathrm{n}}_2$, two years of online education making them realize that taking screenshots or short recordings of some important explanations during the course can be helpful in their professional career or for exams.

According to the calculated TSI, feature 8 (i.e., Course recording availability) is generally considered one-dimensional (O) for the entire sample. The TSI shows a tendency of the students from layer ${\mathrm{n}}_4$ to consider this feature as reverse (R) even if this was not directly expressed in their responses in the Kano questionnaire. The reason behind this is the longer periods of the hybrid education system compared to the students representing the other layers that made students consider this feature useless because the courses were better understood in the face-to-face laboratory or seminar activities. In the university year 2020–2021, both categories of students (layers ${\mathrm{n}}_1$ and ${\mathrm{n}}_2$) had negative values for TSI, these values showing the one-dimensional (O) characteristic. Respondents find their appearance uncomfortable in course recordings from online video conference sessions, and this explains why the values of DI are greater than SI. One year later, this feeling changed after putting in balance the advantage of having all-time access to these recordings that contain additional explanations in many cases compared to the received written material. The TSI obtained for the layer ${\mathrm{n}}_3$ describes a tendency for this feature to become attractive (A) for this category of students even if they expressed it as one-dimensional (O) in the Kano questionnaire.

Features 14 (i.e., Access to the updated scholar situation), 15 (i.e., Online administrative documents/requests evidence and submission), and 16 (i.e., Online studies related payments possibility) are one-dimensional (O) also according to TSI analysis. These features show that there is no correlation between the need for access to administrative resources and previous experience with the online educational system or the field of study. Small deviations are observable for feature 16 in the case of layers ${\mathrm{n}}_1$ and ${\mathrm{n}}_4$, explainable for the latter through face-to-face interactions with the faculty secretariat as a need to know and integrate with the university environment during the periods when students had face-to-face activities.

5. Conclusions

This work extended the research by Pop et al. [12] by providing a comparison between the data collected from two engineering study faculties, one related to computer science and information technology and the other oriented to the management field. Furthermore, this study used as input the answers of 394 students compared to 57 responses collected and analyzed in the previous study by Pop et al. [12]. From this arises the need for choosing the right sample size in order to have accurate results that better describe the population’s perceptions on a product or service.

The research context and related work were presented. Furthermore, the methodology applied in this research was presented in detail to facilitate the reproducibility of the experiment or the identification of possible improvements.

This research used online surveys as a method to collect data between November 2020 and April 2022 from the student community of the Politehnica University of Timisoara. The questionnaire design followed the Kano methodology and considered 17 possible features of a new online educational platform. The feature selection process used a literature review to identify features that have a direct impact on student engagement and motivation. The other features focused on basic technical or administrative needs, such as availability on multiple types of devices, the creation of a custom profile, the existence of messaging services, and access to personal documents handled by the university. The analysis results emphasize the need of students for online educational platforms even in the post-pandemic times.

This study performed a double statistical validation, one on the choice of sample size considering relevance compared to population size, and the other on the significance of the results obtained after applying the Kano evaluation. The second validation is lacking in most studies that use the Kano methodology to assess the expectations of potential customers about the proposed features of a new product or service. This article presented both positive and negative results obtained in this study and responded to the research questions formulated only based on the validated results. The implementation of online educational platforms following the proposed methodology leads to sustainable platforms designed according to the customer’s (i.e., students’) needs and allows efficient management of resources involved in the development of this type of software product.

Further work can address a discussion on the correlation between current results and the results of a self-stated importance questionnaire. Furthermore, a deeper analysis of the features marked as indifferent (I) and reverse (R) should be performed.