Preparedness Indicator System for Education 4.0 with FUCOM and Rough Sets

Abstract

In view of the recent education sectoral transition to Education 4.0 (EDUC4), evaluating the preparedness of higher education institutions (HEIs) for EDUC4 implementation remains a gap in the current literature. Through a comprehensive review, seven criteria were evaluated, namely, human resources, infrastructure, financial, linkages, educational management, learners, and health and environment. This work offers two crucial contributions: (1) the development of an EDUC4 preparedness indicator system and (2) the design of a computational structure that evaluates each indicator and computes an aggregate preparedness level for an HEI. Using the full consistency method (FUCOM) to assign the priority weights of EDUC4 criteria and the rough set theory to capture the ambiguity and imprecision inherent in the measurement, this study offers an aggregate EDUC4 preparedness index to holistically capture the overall preparedness index of an HEI towards EDUC4. An actual case study is presented to demonstrate the applicability of the proposed indicator system. After a thorough evaluation, the results indicate that human resources were the most critical criterion, while health and environment ranked last. Insights obtained from the study provide HEIs with salient information necessary for decision making in various aspects, including the design of targeted policies and the allocation of resources conducive to implementing EDUC4 initiatives. The proposed indicator system can be a valuable tool to guide HEIs in pursuing EDUC4, resulting in a more effective and efficient implementation of this educational paradigm.

1. Introduction

Higher education is crucial in adapting to new technology-based teaching and learning systems to address social concerns and transitions [1]. Education 4.0 (EDUC4) is considered the recent educational transition, which refers to the adaptation of innovative technologies to the current social environment [2]. The shift towards EDUC4 is being driven by technological progress and industrial growth, which has generated attention worldwide [3]. The primary goal of EDUC4 is to integrate information and communication technologies into university curricula to prepare students for the Fourth Industrial Revolution (4IR) [4], which is centered around smart technology, artificial intelligence (AI), and robotics. To achieve this, universities are developing systems for analyzing and adapting learning based on big data, machine learning, and artificial intelligence for improving and personalizing learning—a key component of EDUC4 [5]. With such advances in technology use, traditional school environments are insufficient for higher and vocational education, and integrating real-world challenges and work–life dynamics into the learning process of the future workforce is a critical agenda [6].

Given its nature, EDUC4 has become a desired learning approach that facilitates alignment with the 4IR to meet labor market demands for customized, flexible, accessible, and skill-based learning [7]. This approach transforms learning through innovative technologies such as immersive technologies, augmented reality, and simulation case studies [4,8]. Within this domain, digital strategies, digital security, and appropriate infrastructure are essential to EDUC4, an important pivot that effectively advances traditional education [9]. Due to the complexity inherent in EDUC4, the interconnectedness of knowledge, industry, and human resources forms its ecosystem [10]. As graduates become labor market responsive with the advent of EDUC4, universities play a pivotal role in promoting the social and cultural transitions required for the 4IR [11].

Various empirical evidence highlights the benefits of EDUC4 within the overarching ecosystem in which it operates. For instance, implementing EDUC4 benefits students’ learning process, and is deemed more efficient than conventional learning approaches. It is particularly important in settings where students’ learning is personalized, and specific attempts were reported to appropriately design a curriculum for student learning [12]. Due to its inevitability, several countries have sought to integrate EDUC4 into the higher education sector. For example, Malaysia revised its educational and instructional programs to accommodate the unmet 4IR requirements. With this, the Ministry of Higher Education launched an influential reference [13] that aims to develop and enhance individual potential and meet the nation’s aspirations within the domain of EDUC4. Similarly, the Ministry of Education in Thailand launched Thailand 4.0, which seeks to promote economic prosperity, social well-being, raising human values, and environmental protection, which the education sector is instrumental in its implementation [14]. Additionally, Singapore started the “Smart Nation” initiative, which promotes the widespread adoption of digital and smart technology across the country [15]. Furthermore, the government of Ghana has enabled digitization as one of its primary policy goals and has recently unveiled several initiatives aiming to create a more digitally accessible public sector and promote efficiency, and the education sector serves as its driver [16]. According to Dzandza [17], seven of the nine libraries in Ghana’s public universities have begun digitization efforts. In Korea, Srivani et al. [12] reviewed the impact of EDUC4 chatbots (i.e., bots designed to converse with humans) on Korean university students’ foreign language learning. They found that EDUC4 chatbots positively affect Korean learners’ learning of a foreign language.

In an attempt to fully embed EDUC4 in higher education institutions (HIEs), the emerging literature identifies some directions on how particular features of EDUC4 can be implemented in HEIs. These include opportunities to implement augmented reality/virtual reality (AR/VR) technologies in the classrooms [18], Internet of Things (IoT) in education with AI-enhanced biosensors and wearables [19,20], and preparing future engineers with the trends of cyber-physical systems [21]. Some works, including those by Stachová et al. [22], Giesenbauer and Müller-Christ [23], and Mourtzis et al. [24], offered a bird’s eye view on insights to successfully overcome the challenges in the misalignment of the skill sets learned in schools and those skills needed in the industry. They argue that universities must adopt a multidimensional, multi-networked managerial model to successfully implement EDUC4 (e.g., business cooperation within universities, factory-to-classroom concepts, and external partnerships in the context of employee education). In managing its implementation, myriad policy directions for a successful implementation of EDUC4 have been put forward in various works (e.g., [2,7,23,25]). They contend that the re-alignment of the curriculum must be proactive, the transformation in higher education must be sustainable, and the assessment to overcome the implementation barriers should be consistent in order to gain the benefits of EDUC4.

Despite these advances, from a broader perspective, a clear evaluation tool to assess the preparedness of HEIs for EDUC4 implementation is missing in the current literature. Evaluating the preparedness of HEIs represents an important step in gaining insights into the following: (1) retrospectively assessing how the organization is collectively heading in view of EDUC4, (2) determining the current performance of the organization in the different facets of EDUC4 and building baseline conditions thereafter, (3) identifying critical “hotspots” in the organization for advancing its agenda regarding EDUC4, and (4) comparing the aggregate performance of different HEIs or units within HEIs in their efforts toward EDUC4 implementation. Numerous scholarly works delve into the discussion of the development of indicators and indices that assess the preparedness of HEIs in various contexts. They encompass a wide range of aspects, highlighting the following key areas: a framework for evaluating the long-term environmental sustainability of HEIs [10,26,27], examining preservice teacher preparedness for education for sustainable development [28], knowledge and attitude assessment in disaster preparedness in schools [29], measuring STEM teachers’ instructional readiness [30], assessing e-learning platforms for blended learning [31], espousing a robust college readiness agenda [32], the data-driven decision making and big data analytics capability of HEIs [33], and service quality evaluation based on students’ satisfaction [34]. The development of indicators or indices is popular due to its criticality in generating important holistic insights into the organization’s performance in achieving a desired agenda. These insights offer organizations critical information for decision making regarding policy development, strategy formulation, and resource allocation in progressing toward a goal. In this particular focus in the literature on the EDUC4 agenda, Jamaludin et al. [10] reported initial low-resolution insights into the readiness of HEIs, at least in the ASEAN region. Their work identified 16 indicators spread over personal, curriculum, and pedagogical readiness, 7 for industry readiness, and 6 for humanity readiness. Their findings suggest the following: (1) high personal readiness among stakeholders (e.g., policymakers, lecturers, and students) for EDUC4, (2) high curriculum readiness, with areas attuned to the requirements of EDUC4, (3) low pedagogical readiness, (4) technical, management, and financial readiness of institutions are low, and (5) institutions are significantly not culturally ready for EDUC4. Although these insights provide an overview of the readiness of HEIs in the ASEAN, a number of limitations become apparent. First, having over 263 stakeholders from three groups (i.e., policymakers, lecturers, and students) to accomplish the task of describing HEIs from 10 ASEAN countries can be viewed as an overstretch to its objective and may not thoroughly provide a representative view of all HEIs in the region. Secondly, the limited number of indicators present in their evaluation fails to capture the overarching preparedness level of HEIs. Finally, their approach, as well as those described in other studies, could not provide HEIs with an overview of specific areas that require their attention to leapfrog to EDUC4.

Thus, this work advances on these limitations in the literature and offers a comprehensive evaluation approach by building an indicator system that holistically captures the overall preparedness index of an HEI in its route to EDUC4. It builds upon the prior work of Costan et al. [7] on seven areas HEIs need to examine in their implementation of EDUC4: human resources, infrastructure, financial, linkages, educational management, learners, and health and environment. With these areas, this current work attempts to determine the list of appropriate indicators, each with a measurement scale that decision makers in HEIs can efficiently measure. Subsequently, it offers a systematic approach that aggregates these indicators into a single-valued index that represents the preparedness level of an HEI in its EDUC4 implementation. In assigning the priority weights of EDUC4 areas necessary in the proposed computational framework, the full consistency method (FUCOM) proposed by Pamučar et al. [35] was adopted, as it augments the high cognitive workload requirement of its more popular counterparts, such as the analytic hierarchy process (AHP) and the best-worst method (BWM). Ocampo [36] demonstrated that the weights of attributes derived from FUCOM are highly consistent with those of the more prominent BWM. To capture the ambiguity and imprecision inherent in the measurements, we adopted the rough set theory introduced by Pawlak [37], with the basic concepts and operations presented in Pawlak [38]. While fuzzy set theory and its extensions (e.g., intuitionistic fuzzy sets, spherical fuzzy sets, and linear Diophantine fuzzy sets) are more popular in handling uncertainty in judgments, they require prior information (e.g., membership functions) and assumptions. Rough sets, on the other hand, overcome these limitations by representing imprecise judgments into a concept based on lower and upper approximations [38]. The applications of rough sets for performance evaluation are gaining prominence in the recent literature, including stroke indicators [39], predicting sustainability performance [40], assessing ecological environment sustainability of islands [41], and evaluating potential accidents on a mountain road [42], among others. Zhang et al. [43] reported a review of rough sets and their applications.

An actual case study in a Philippine HEI is presented to demonstrate the applicability of the proposed indicator system. To the best of our knowledge, this study is the first to offer such a system and its corresponding systematic computational framework to evaluate the preparedness of HEIs in their EDUC4 implementation. The main contribution of this work is the development of such an indicator system under a rough set environment that allows HEIs to assess their preparedness efforts in implementing EDUC4 and identifies specific actionable hotspots to leapfrog its progress towards EDUC4. The adoption of rough sets in this work is beneficial for handling uncertainty in the evaluation process while minimizing the cognitive workload of evaluators or decision makers by limiting additional information and assumptions required from them, such as the shape of the membership functions, as in fuzzy sets. The rest of the paper is arranged as follows. Section 2 presents some preliminary concepts of rough set theory and FUCOM. Section 3 illustrates the proposed indicator system for preparedness evaluation in EDUC4 implementation. Section 4 demonstrates an application of the indicator system in an actual case of a Philippine HEI. Some insights and limitations are identified in Section 5. It ends with concluding remarks in Section 6.

2. Preliminaries

This section details the preliminary concepts of rough set theory and the full consistency method.

2.1. Rough Set Theory

The rough set theory that Pawlak [37] put forward serves as a powerful mathematical technique for addressing information and knowledge that are imprecise, inconsistent, and incomplete without any assumptions and additional adjustments. Due to its innovative approach, distinct methodology, and straightforward operation, rough set theory has gained prominence in various fields such as intelligence information processing (e.g., [44,45]), pattern recognition (e.g., [46,47]), knowledge acquisition (e.g., [48]), and decision support analysis (e.g., [49]), among others. Note that this list is not intended to be comprehensive. A survey of the applications of rough set theory can be found elsewhere [43].

Rough set theory enables the representation of ambiguous concepts through a combination of precise concepts determined by two crisp numbers [37], defined as follows.

Definition 1. 

Let $U$ be the universe of discourse, $Y$ be an arbitrary object of $U$ , and $A=\left\{A_1,A_2,\dots ,A_t\right\}$ be a set of $t$  classes ordered in a manner of $A_1<A_2<\dots <A_t$ . Then, $A_q\in A, \forall Y\in U$ and $1\le q\le t$. The lower approximation, upper approximation, and boundary region of $A_q$, denoted as $\underset{¯}{Apr}\left(A_q\right)$, $\overline{Apr}\left(A_q\right)$, and $Bnd\left(A_q\right)$, respectively, are defined as follows.

$$\underset{¯}{Apr}\left(A_q\right)=\bigcup \left\{Y\in U:A\left(Y\right)\le A_q\right\},$$

(1)

$$\overline{Apr}\left(A_q\right)=\bigcup \left\{Y\in U:A\left(Y\right)\ge A_q\right\},$$

(2)

$$Bnd\left(A_q\right)=\bigcup \left\{Y\in U:A\left(Y\right)\ne A_q\right\}=\left\{Y\in U:A\left(Y\right)<A_q\right\}\cup \left\{Y\in U:A\left(Y\right)>A_q\right\}.$$

(3)

Definition 2. 

Any ambiguous class  $A_q$  of  $U$  can be represented by a rough number  $RN\left(A_q\right)$  which is defined by its lower and upper limits, denoted as  $\underset{¯}{Lim}\left(A_q\right)$  and  $\overline{Lim}\left(A_q\right)$, respectively, and defined as follows.

$$RN\left(A_q\right)=\left[\underset{¯}{Lim}\left(A_q\right),\overline{Lim}\left(A_q\right)\right],$$

(4)

$$\underset{¯}{Lim}\left(A_q\right)=\frac{1}{\underset{\_}{M}}\sum A\left(X\right):X\in \underset{¯}{Apr}\left(A_q\right),$$

(5)

$$\overline{Lim}\left(A_q\right)=\frac{1}{\overline{M}}\sum A\left(X\right):X\in \overline{Apr}\left(A_q\right).$$

(6)

where  $\underset{\_}{M}$  and  $\overline{M}$  are the number of objects included in  $\underset{¯}{Apr}\left(A_q\right)$and  $\overline{Apr}\left(A_q\right)$, respectively.

Definition 3. 

Suppose  $RN\left(A_q\right)$ and  $RN\left(B_q\right)$  are two rough numbers, and  $\mu$  is a nonzero constant; then, the interval arithmetic operations are carried out as follows:

$$RN\left(A_q\right)+RN\left(B_q\right)=\left[\underset{¯}{Lim}\left(A_q\right)+\underset{¯}{Lim}\left(B_q\right),\overline{Lim}\left(A_q\right)+\overline{Lim}\left(B_q\right)\right]$$

(7)

$$RN\left(A_q\right)÷RN\left(B_q\right)=\left[\underset{¯}{Lim}\left(A_q\right)÷\underset{¯}{Lim}\left(B_q\right),\overline{Lim}\left(A_q\right)÷\overline{Lim}\left(B_q\right)\right]$$

(8)

$$RN\left(A_q\right)\times RN\left(B_q\right)=\left[\underset{¯}{Lim}\left(A_q\right)\times \underset{¯}{Lim}\left(B_q\right),\overline{Lim}\left(A_q\right)\times \overline{Lim}\left(B_q\right)\right]$$

(9)

$$\mu RN\left(A_q\right)=\left[\mu \underset{¯}{Lim}\left(A_q\right),\mu \overline{Lim}\left(A_q\right)\right]$$

(10)

Various aggregation methods for rough numbers are proposed in the literature. For instance, Stević et al. [50] provided a Rough Hamy aggregator for group decision making that simultaneously considers mutual correlations among multiple arguments.

Theorem 1. 

Let  $RN\left({\alpha }_j\right)=\left[\underset{¯}{Lim}\left({\alpha }_j\right),\overline{Lim}\left({\alpha }_j\right)\right]$ $\left(j=1,2,\dots ,n\right)$ represent a set of rough numbers in  $R$. The aggregate rough number can be determined as follows.

$$\begin{gathered} RNHM\left\{RN\left({\alpha }_1\right),RN\left({\alpha }_2\right),\dots ,RN\left({\alpha }_n\right)\right\}=\left[\underset{¯}{Lim}\left({\alpha }_{RNHM}\right),\overline{Lim}\left({\alpha }_{RNHM}\right)\right] \\ =\left[\frac{{{\displaystyle \sum }}_{1\le i_1<\dots <i_k}{\left({{\displaystyle \prod }}_{j=1}^k\underset{¯}{Lim}\left({\alpha }_{i_j}\right)\right)}^{\frac{1}{k}}}{\left(\begin{array}{c}n\\ k\end{array}\right)},\frac{{{\displaystyle \sum }}_{1\le i_1<\dots <i_k}{\left({{\displaystyle \prod }}_{j=1}^k\overline{Lim}\left({\alpha }_{i_j}\right)\right)}^{\frac{1}{k}}}{\left(\begin{array}{c}n\\ k\end{array}\right)}\right] \end{gathered}$$

(11)

where $\left(i_1,i_2,\dots ,i_k\right)$ includes all  $k$-tuple combinations of  $\left(1,2,\dots ,n\right)$.

2.2. Full Consistency Method

To generate priority weights for a set of predefined attributes, Pamučar et al. [35] developed the FUCOM approach as a comparison-based multi-attribute decision making (MADM) method that incorporates some of the features of the AHP and BWM. Two well-known categories of constraints are used by FUCOM: (1) mathematical transitivity and (2) consistency in the relationships between attribute weights and the relative priorities of the attributes. The deviation from the full consistency (DFC) metric associated with these two constraint groups is optimized to establish the distribution of the priority weights of the attributes. Such metric measures the reliability of the resulting attribute weights. The major takeaway of FUCOM over AHP and BWM is the minimal number of pairwise comparisons that decision makers must make, reducing their mental effort throughout the judgment–elicitation process. A classic MADM problem can be solved with FUCOM when the goal–criteria–alternative hierarchical structure is employed to produce the weights of the attributes and the priority weights of the alternatives for each attribute.

The integration of FUCOM has been widely used in prioritizing transportation demand management measures (e.g., [51,52]), public sector supply chain [53], evaluating the sustainability of farm tourism sites [36], determining drivers for investing in cryptocurrencies [54], and landfill site selection [55], among others. The computational step of FUCOM was introduced by Pamučar et al. [35]. Here, the word “criteria” refers to representational standards. They can be referred to as any collection of homogeneous elements in general (e.g., attributes, factors).

Step 1: Consider the collection $C=\left\{c_1,c_2,\dots ,c_n\right\}$ of decision criteria. Rank them in order of their level of importance. Based on this step, the following illustrates the ranking of the set of criteria based on their estimated weights:

$$c_{j\left(1\right)}>c_{j\left(2\right)}>\dots >c_{j\left(s\right)}$$

(12)

where $s$ denotes the rank of a criterion $j$, where $j\ne j^{\prime }$ and $j,j^{\prime }=1,\dots ,n$. Whenever $j$ and $j^{\prime }$ are perceived to have equal weights, then $c_{j\left(s\right)}=c_{j^{\prime }\left(s\right)}$.

Step 2: Generate the comparative priorities $\left\{{\phi }_{s/\left(s+1\right)},s=1,2,\dots ,n\right\}$, such that the vector of relative priorities of the criteria, represented by $\mathsf{\Phi }$, is obtained as follows:

$$\mathsf{\Phi }=\left({\phi }_{1/2},{\phi }_{2/3},\dots ,{\phi }_{s/\left(s+1\right)}\right)$$

(13)

where ${\phi }_{s/\left(s+1\right)}$ denotes the priority ratio of the criterion $c_{j\left(s\right)}$ over the criterion $c_{j\left(s+1\right)}$. By construction, ${\phi }_{s/\left(s+1\right)}\ge 1$.

Step 3: Obtain the final weight vector of the evaluation criteria, denoted as $\left\{w_j:j=1,\dots ,n\right\}$. In generating the criteria weights, the following conditions must be satisfied: (1) the ratio of the weights of two criteria is equal to their comparative priority ${\phi }_{s/\left(s+1\right)}$, specified in Step 2. Symbolically:

$$\frac{w_s}{w_{s+1}}={\phi }_{s/\left(s+1\right)}$$

(14)

In addition, (2) the final weight values must also meet the transitivity condition, i.e., ${\phi }_{s/\left(s+1\right)}\times {\phi }_{\left(s+1\right)/\left(s+2\right)}={\phi }_{s/\left(s+2\right)}$. Since ${\phi }_{s/\left(s+1\right)}=\frac{w_s}{w_{s+1}}$ and ${\phi }_{\left(s+1\right)/\left(s+2\right)}=\frac{w_{s+1}}{w_{s+2}}$, then $\frac{w_s}{w_{s+1}}\times \frac{w_{s+1}}{w_{s+2}}=\frac{w_s}{w_{s+2}}$. Thus, the following condition is required:

$$\frac{w_s}{w_{s+2}}={\phi }_{s/\left(s+1\right)}\times {\phi }_{\left(s+1\right)/\left(s+2\right)}$$

(15)

To attain full consistency, these two conditions must be met. As such, the weight assignments $\left\{w_j:j=1,\dots ,n\right\}$ must satisfy $\left|\frac{w_s}{w_{s+1}}-{\phi }_{s/\left(s+1\right)}\right|\le \chi$ and $\left|\frac{w_s}{w_{s+2}}-{\phi }_{s/\left(s+1\right)}\times {\phi }_{\left(s+1\right)/\left(s+2\right)}\right|\le \chi$, where $\chi$ represents the DFC. Thus, obtaining the weight vector ${\left(w_j:j=1,\dots ,n\right)}^T$ requires solving the following optimization problem:

$$\min \chi$$

(16)

subject to:

$$\left|\frac{w_s}{w_{s+1}}-{\phi }_{s/\left(s+1\right)}\right|\le \chi , \forall s$$

$$\left|\frac{w_s}{w_{s+2}}-{\phi }_{s/\left(s+1\right)}\times {\phi }_{\left(s+1\right)/\left(s+2\right)}\right|\le \chi , \forall s$$

$${{\displaystyle \sum }}_{j=1}^n{w}_j=1 w_j>0, \forall j.$$

3. Development of the Proposed EDUC4 Preparedness Indicator System

The process of generating the set of indicators involves four distinct steps. Firstly, the areas or themes of EDUC4 implementation were identified based on a previously published systematic literature review by Costan et al. [7]. For convention, we treat these areas as the upper-level criteria in the proposed indicator system. These criteria, including human resources, infrastructure, financial, linkages, educational management, learners, and health and environment, are recognized by Costan et al. [7] as dimensions associated with known implementation barriers in operationalizing EDUC4 in HEIs. Secondly, preliminary indicators specific to each theme were formulated based on the results of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement outlined in Costan et al. [7]. Their systematic literature review identified 30 articles from which they established the themes of EDUC4 implementation. After obtaining the references, they were thoroughly examined with the intention of capturing possible indicators of a specific criterion. The initial inclusion rule is that an indicator must be quantifiable, such as those expressed through percentages, actual data, or espousing a minimum threshold requirement. They were systematically organized into different sub-criteria of the seven previously identified criteria.

In order to create a more robust identification of indicators in each criterion, the third step involved identifying the overlapping and irrelevant indicators based on the core components of EDUC4, as illustrated by Miranda et al. [2]. These components highlighted nine essential concepts, such as (A1) heutagogical, peeragogical, and cybergogical; (A2) mentorship, collaboration, and reference; (A3) active, independent, and trajectory designing; (A4) mostly student-centered education; (A5) training soft and hard key competencies; (A6) utilizing ICT tools and platforms powered by IoT; (A7) based on online sources, (A8) cyber and physical spaces, shared and individual; and (A9) connectivity, digitalization, and virtualization. This list forms the overarching agenda of EDUC4 that separates itself from the previous revolutions in education. This step ensures the relevance of all indicators in all components of EDUC4, an important direction to not over-generate the list of indicators in the proposed indicator system. Finally, the indicators were augmented and finalized through brainstorming and focus group discussions (FGDs) in the fourth and final step. This step was implemented with an expert group of eight members with extensive backgrounds in the education sector and university core processes. They all hold Ph.D. degrees in education, administration, governance, and research management. Their experiences range from 19 to 26 years, with an average of 22 years. The task of the expert group was to ensure the comprehensiveness and relevance of the indicators in each criterion. During the FGD, brainstorming, and augmentation efforts to establish a relevant indicator system, initially identified items were consolidated and evaluated based on redundancy and relevance. Figure 1 illustrates the steps in generating the list of final indicators relevant to EDUC4 implementation.

Figure 1. The process of generating the list of indicators.

Results show that out of 129 identified indicators obtained from the review of the literature, 31 were considered redundant and irrelevant indicators to the core components of EDUC4 and were consequently removed from further analysis. At the same time, seven indicators were added during the augmentation process. The summary of the transitions of indicator lists across the steps in Figure 1 is found in Table 1. To provide a clear overview of the process, for instance, the indicators identified to measure the infrastructure for online learning (C22) and other infrastructure in line with the EDUC4 implementation (C23), internet connectivity was found to be redundant. Thus, the measurement derived from this indicator was revised to cut across both criteria. The final list of 105 indicators is presented in Table A1. By adopting these indicators and engaging key stakeholders, universities can better align their teaching and learning practices with the demands of the 4IR and EDUC4 framework.

Table 1. Generation of indicators per criterion.

Code		Criterion Description		No. of Indicators after Step 2 (A)	No. of Indicators after Step 3 (B)	No. of Indicators after Step 4 (C)	Final Indicators (A) − (B) + (C)
C1		Human resources		22	16	0	6
C2		Infrastructure
	C21		ICT infrastructure	6	0	0	6
	C22		Infrastructure for online learning	7	0	0	7
	C23		Other infrastructure relevant to EDUC4 implementation	5	0	0	5
	C24		Science laboratories	6	0	0	6
	C25		Library 4.0	13	0	0	13
C3		Financial
	C31		Faculty capability enhancement	2	0	0	2
	C32		Technology and infrastructure alignment	3	0	0	3
	C33		Physical facilities and supplies	2	0	1	3
C4		Linkages
	C41		Education and training	8	0	1	9
	C42		Collaboration	7	1	0	6
C5		Educational management
	C51		Educational leaders’ commitment	6	2	0	4
	C52		Relationship with stakeholders	5	3	0	2
	C53		Management support toward EDUC4 implementation	7	4	0	3
C6		Learners
	C61		Learners’ experience	6	0	0	6
	C62		Teaching and learning experience	5	0	0	5
	C63		Students and curriculum design	4	0	1	5
	C64		Technology-based monitoring of the system of student performance	3	1	0	2
	C65		Student service	1	0	0	1
C7		Health and Environment
	C71		Screen viewing	1	0	0	1
	C72		Physical risk	1	0	0	1
	C73		Emotional risk	1	0	0	1
	C74		Cognitive risk	1	0	0	1
	C75		Time constraints for material preparation	1	0	0	1
	C76		Classroom layout	6	4	0	2
	C77		E-waste management	0	0	4	4
	TOTAL			129	31	7	105

3.1. Case Study Background

To demonstrate the applicability of the proposed indicator system, a case in a state university in the Philippines, i.e., Cebu Technological University (CTU), is presented in this work. Note that the case study considered only assumes one of the primary functions of the proposed indicator system, which is to evaluate the aggregate performance of an HEI and identify certain “hotspots”. CTU is a state-funded HEI in central Philippines. The Board of Regents is the highest governing authority of CTU, chaired by a Commissioner of the Philippine Commission on Higher Education, and the university president sits as the Vice-Chair of the board. Five Vice Presidents assist the university president in carrying out the university’s core functions, including instruction, research, extension, and production. The budget utilized by the university is mainly taken from the subsidies granted by the Philippine government through the annual release of the General Appropriations Act and from other incomes granted by several institutions, locally and internationally. Recently, CTU has strengthened its endeavor to become a premier university and forged various partnerships with universities abroad. It has nine external campuses, a main campus, and extension campuses. CTU-Danao is one of the external campuses of CTU, located in the northern part of Cebu. As a fast-growing institution, it has implemented massive improvements in its physical and technological infrastructure. The campus envisions leading the educational community by upgrading its facilities to improve the quality of instruction and thereby producing world-class graduates by adhering to the intricacies of EDUC4. Concerted efforts of various campus departments are underway toward a holistic approach to developing a campus that is EDUC4-ready. With its current EDUC4 trajectory, it is vital to determine its preparedness to realign efforts, human and financial resources, and development programs for full EDU4 implementation.

3.2. Application of the Proposed Methodology

This section details the application of the proposed methodology in the case of CTU-Danao. Figure 2 features the proposed methodology, which comprises two phases. Phase I utilizes rough FUCOM in obtaining the rough weight of each criterion, while Phase II obtains the index that describes the degree of preparedness of CTU-Danao in EDUC4 implementation.

Figure 2. The proposed framework for adopting the proposed indicator system.

Phase I. Generate criteria weights using rough-FUCOM.

Step 1. Establish the set of evaluation criteria $C=\left\{c_1, \dots , c_f,\dots ,c_F\right\}$ to measure the preparedness of an institution for EDUC4 implementation. The evaluation criteria, namely, (1) human resources, (2) infrastructure, (3) financial, (4) linkages, (5) educational management, (6) learners, and (7) health and environment, were identified from Costan et al. [7]. In our work, a group of experts $\left(k=1,2,\dots ,K\right)$ was asked to elicit judgments on the evaluation criteria. The criteria were ranked according to their degree of significance to EDUC4 implementation.

Step 2. Construct the vector of the comparative priorities of the criteria for each expert. Arrange the evaluation criteria in decreasing level of importance. In our work, the expert group was again asked to elicit their judgments on the comparative priorities of the criteria set.

Step 3. Obtain the priority weights of the criteria. The priority weights of the evaluation criteria were obtained using Equation (16).

Step 4. Transform the priority weights into rough numbers. Assume that $W$ is a set containing all the weight coefficients carried out by experts’ evaluations, and $w_x$ is an arbitrary object of $W$. A set of weight coefficients $\omega =\left(w_1^k, w_2^k,\dots ,w_F^k\right)$ are obtained from Step 1 to Step 3 and arranged in a manner $w_1^k<w_2^k<\dots <w_F^k$; then, $w_j^k\in \omega ,\forall w_x\in W$. The lower approximation $\underset{¯}{Apr}\left(w_j^k\right)$ and upper approximation $\overline{Apr}\left(w_j^k\right)$ are obtained using Equations (1) and (2), respectively. Then the rough number of $RN\left(w_j^k\right)=\left[\underset{¯}{Lim}\left(w_j^k\right),\overline{Lim}\left(w_j^k\right)\right]$ is determined following Equations (5) and (6). The optimal values of the rough weight coefficients of the criteria are calculated using Equation (17). The resulting rough weight coefficients are presented in Table 2.

$$RN\left(w_j\right)=\left\{\begin{array}{c}\underset{¯}{Lim}\left(w_j\right)=\frac{1}{K}{\displaystyle {\displaystyle \sum }_{k=1}^K}\underset{¯}{Lim}\left(w_j^k\right)\\ \overline{Lim}\left(w_j\right)=\frac{1}{K}{\displaystyle {\displaystyle \sum }_{k=1}^K}\overline{Lim}\left(w_j^k\right)\end{array}\right.$$

(17)

Table 2. Rough priority weights of the criteria.

Criteria	Rough Weights
Human resources	[0.1167, 0.1545]
Infrastructure	[0.1545, 0.1684]
Financial	[0.1409, 0.1680]
Linkages	[0.1106, 0.1680]
Educational management	[0.1091, 0.1466]
Learners	[0.1191, 0.1540]
Health and environment	[0.1063, 0.1568]

Phase II. Calculate the degree of preparedness of CTU-Danao for EDUC4 implementation.

Step 5. Determine the list of indicators for each evaluation criterion. A literature survey was conducted to gather the initial list of criterion indicators. The process described in Figure 1 was implemented to obtain a final set of indicators. The same group of experts who conducted Step 1 was involved in the discussion. The outcome of the discussion resulted in the list of indicators presented in Table A1. These indicators were then sorted according to the stakeholders associated with them. The following are the identified stakeholders that have a significant role in EDUC4 implementation: human resource office, faculty, dean, IT manager, registrar, Vice-President for Administration, librarian, finance office, OJT coordinators, Research and Development (R&D) office, internalization office, community extension office, student affairs office, educational leaders, Vice-President for Academics, secretary of the top management, campus administrator, evaluators, and students. Separate survey questionnaires were created for each stakeholder. Table A2 also shows each indicator and stakeholder’s specific measurement scale and sampling plan.

Step 6. Calculate the aggregate rough number per criterion for each stakeholder. Let $A$ be a set of all evaluations elicited by $L \left(l=1,2,\dots ,L\right)$ survey participants based on a set of $n$ indicators characterizing $f$th criterion and ${\alpha }_x$ be an arbitrary object of $A$. A set of evaluations $\alpha =\left({\alpha }_1^l, {\alpha }_2^l,\dots ,{\alpha }_n^l\right)$ elicited by $l$th respondent are arranged in a manner ${\alpha }_1^l<{\alpha }_2^l<\dots <{\alpha }_n^l$ then ${\alpha }_j^l\in \omega ,\forall {\alpha }_x\in A$. The lower approximations $\underset{¯}{Apr}\left(a_j^l\right)$ and upper approximation $\overline{Apr}\left(a_j^l\right)$ are obtained using Equations (1) and (2), respectively. Then, the rough number of $RN\left(a_n^l\right)=\left[\underset{¯}{Lim}\left(a_j^l\right),\overline{Lim}\left(a_j^l\right)\right]$ is determined following Equation (5) and Equation (6). Then the aggregate rough numbers $RN\left({\alpha }_j\right)$ are obtained using the expression: $RN\left({\alpha }_j\right)=\left\{\begin{array}{c}\underset{¯}{Lim}\left({\alpha }_j\right)=\frac{1}{L}{\displaystyle \sum }_{l=1}^L\underset{¯}{Lim}\left(a_j^l\right)\\ \overline{Lim}\left({\alpha }_j\right)=\frac{1}{L}{\displaystyle \sum }_{l=1}^L\overline{Lim}\left(a_j^l\right)\end{array}\right.$. The resulting aggregate rough numbers are presented in Table A2.

Step 7. Normalize $RN\left({\alpha }_j\right)$ relative to predefined ideal values. A series of interviews with the concerned stakeholders were carried out to finalize quantifiable measures of the existing practices of the case university and the minimum standard based on statutory and regulatory requirements. Other standards were determined based on the emerging literature (e.g., [20,56,57]). The group of experts provided a set of ideal rough values per $n$th indicator, denoted as $RN\left({\alpha }_j^{*}\right)=\left[\underset{¯}{Lim}\left(a_j^{*}\right),\overline{Lim}\left(a_j^{*}\right)\right]$ and defined as follows.

For maximization criteria:

$$RN\left({\tilde{\alpha }}_j\right)=\left\{\begin{array}{c}\underset{¯}{Lim}\left({\tilde{\alpha }}_j\right)=\underset{¯}{Lim}\left({\alpha }_j\right)÷\overline{Lim}\left(a_j^{*}\right)\\ \overline{Lim}\left({\tilde{\alpha }}_j\right)=\overline{Lim}\left({\alpha }_j\right)÷\underset{¯}{Lim}\left(a_j^{*}\right)\end{array}\right.$$

(18)

For minimization criteria:

$$RN\left({\tilde{\alpha }}_j\right)=\left\{\begin{array}{c}\underset{¯}{Lim}\left({\tilde{\alpha }}_j\right)=\underset{¯}{Lim}\left(a_j^{*}\right)÷\overline{Lim}\left({\alpha }_j\right)\\ \overline{Lim}\left({\tilde{\alpha }}_j\right)=\overline{Lim}\left(a_j^{*}\right)÷\underset{¯}{Lim}\left({\alpha }_j\right)\end{array}\right.$$

(19)

Step 8. Aggregate the indicators and generate the weighted $RN\left(\tilde{\alpha }\right)$. The $RN\left({\tilde{\alpha }}_j\right)$, where $j=\left(1, 2,\dots , n\right)$, are aggregated using Equation (11). Then, the weighted $RN\left({\tilde{\alpha }}_j\right)$ is obtained through Equation (20).

$$RN\left({\widehat{\alpha }}_j\right)=RN\left({\tilde{\alpha }}_j\right)\times RN\left(w_j\right)=\left\{\begin{array}{c}\underset{¯}{Lim}\left({\tilde{\alpha }}_j\right)\times \underset{¯}{Lim}\left(w_j\right)\\ \overline{Lim}\left({\tilde{\alpha }}_j\right)\times \overline{Lim}\left(w_j\right)\end{array}\right.$$

(20)

where $RN\left(w_j\right)$ is the rough weight coefficients obtained from Phase I.

Step 9. Obtain the crisp value of the preparedness index. The weighted $RN\left({\widehat{\alpha }}_j\right)$ is transformed into crisp ${\widehat{\alpha }}_j$ using Equation (21). The crisp scores are normalized using Equation (22). The results are presented in Table 3.

$${\widehat{\alpha }}_j=\frac{\underset{¯}{Lim}\left({\widehat{\alpha }}_j\right)\times \left(1-\overline{Lim}\left({\widehat{\alpha }}_j\right)\right)+\overline{Lim}\left({\widehat{\alpha }}_j\right)\times \overline{Lim}\left({\widehat{\alpha }}_j\right)}{1-\overline{\underset{¯}{Lim}\left({\widehat{\alpha }}_j\right)+Lim}\left({\widehat{\alpha }}_j\right)}$$

(21)

$${\tilde{a}}_j=\frac{{\widehat{\alpha }}_j}{{{\displaystyle \sum }}_{j=1}^n{\widehat{\alpha }}_j}$$

(22)

Table 3. Rough and normalized crisp preparedness index per criterion.

Criteria	Rough Preparedness Index	Normalized Crisp Preparedness Index
Human resource	[0.6991, 0.8589]	0.2500
Infrastructure	[0.2654, 0.2667]	0.1169
Financial	[0.3434, 0.3545]	0.1393
Linkages	[0.4968, 0.7411]	0.1790
Educational management	[0.3515, 0.4011]	0.1122
Learners	[0.4976, 0.6891]	0.1817
Health and Environment	[0.0635, 0.1624]	0.0205

The single-valued aggregate EDUC4 preparedness index (EPI) for the case university is obtained as follows:

$$RN\left({\tilde{\alpha }}_j^w\right)=RN\left(w_j\right)\times RN\left({\widehat{\alpha }}_j\right)$$

(23)

$$a_j^w=\frac{\underset{¯}{Lim}\left({\tilde{\alpha }}_j^w\right)\times \left(1-\overline{Lim}\left({\tilde{\alpha }}_j^w\right)\right)+\overline{Lim}\left({\tilde{\alpha }}_j^w\right)\times \overline{Lim}\left({\tilde{\alpha }}_j^w\right)}{1-\overline{\underset{¯}{Lim}\left({\tilde{\alpha }}_j^w\right)+Lim}\left({\tilde{\alpha }}_j^w\right)}$$

(24)

$$w_j=\frac{\underset{¯}{Lim}\left(w_j\right)\times \left(1-\overline{Lim}\left(w_j\right)\right)+\overline{Lim}\left(w_j\right)\times \overline{Lim}\left(w_j\right)}{1-\overline{\underset{¯}{Lim}\left(w_j\right)+Lim}\left(w_j\right)}$$

(25)

$$EPI={{\displaystyle \sum }}_{j=1}^n{a}_j^w/{{\displaystyle \sum }}_{j=1}^n{w}_j$$

(26)

Thus, the case university has an $EPI=0.393$.

4. Sensitivity and Comparative Analysis

Sensitivity analysis was carried out to determine the extent to which the most influential criterion performance affects the ranking of the entire criteria set. The analysis was performed by changing the criteria weights to determine the model’s sensitivity to weight changes. For this case, seven scenarios were formed in which the criteria weights were modeled. Scenarios were defined based on the weights, whereas the weights were assigned based on a certain criterion’s domination. For instance, in the first scenario, human resources (C1) criterion dominates the ideal weights of (0.70, 0.70), and other criteria were assigned (0.05, 0.05). Table 4 presents the weights based on the domination of a certain criterion for seven scenarios.

Table 4. Sensitivity weights based on the domination of an identified criterion.

	Scenario 1	Scenario 2	Scenario 3	Scenario 4	Scenario 5	Scenario 6	Scenario 7
C1	[0.7000, 0.7000]	[0.0500, 0.0500]	[0.0500, 0.0500]	[0.0500, 0.0500]	[0.0500, 0.0500]	[0.0500, 0.0500]	[0.0500, 0.0500]
C2	[0.0500, 0.0500]	[0.7000, 0.7000]	[0.0500, 0.0500]	[0.0500, 0.0500]	[0.0500, 0.0500]	[0.0500, 0.0500]	[0.0500, 0.0500]
C3	[0.0500, 0.0500]	[0.0500, 0.0500]	[0.7000, 0.7000]	[0.0500, 0.0500]	[0.0500, 0.0500]	[0.0500, 0.0500]	[0.0500, 0.0500]
C4	[0.0500, 0.0500]	[0.0500, 0.0500]	[0.0500, 0.0500]	[0.7000, 0.7000]	[0.0500, 0.0500]	[0.0500, 0.0500]	[0.0500, 0.0500]
C5	[0.0500, 0.0500]	[0.0500, 0.0500]	[0.0500, 0.0500]	[0.0500, 0.0500]	[0.7000, 0.7000]	[0.0500, 0.0500]	[0.0500, 0.0500]
C6	[0.0500, 0.0500]	[0.0500, 0.0500]	[0.0500, 0.0500]	[0.0500, 0.0500]	[0.0500, 0.0500]	[0.7000, 0.7000]	[0.0500, 0.0500]
C7	[0.0500, 0.0500]	[0.0500, 0.0500]	[0.0500, 0.0500]	[0.0500, 0.0500]	[0.0500, 0.0500]	[0.0500, 0.0500]	[0.7000, 0.7000]

Figure 3 presents the results of the comparative analysis after assigning priority weight to each criterion. It shows that the changes in the criteria ranking are observed to be directly dependent on the dominating criterion. For example, in the first scenario, assigning ideal weights on C1 generates an outcome that incurs utmost prioritization on the same criterion, with a notable increase of at least triple the original amount. The same behavior is evident in the succeeding scenarios where the dominating criterion takes the most priority in the ranking of the preparedness index. This remark is even observed with C7, originally having the lowest local preparedness index. Additionally, all the other criteria aside from the dominating criterion retain their order from the original hierarchy of the criteria. Figure 3 also shows that if all the weights of the dominating criteria from all the scenarios are to be collated, the result reflects the original ranking or index scores. This observation is consistent with the behavior of the remaining criteria, apart from the dominating criterion, which remain in the same order for all the scenarios. The findings in this analysis demonstrate that assigning priority criteria weights is crucial as it greatly influences the resulting aggregate index.

Figure 3. Ranking of results from the sensitivity analysis.

5. Results and Discussion

Given the overall results, an $EPI=0.393$ out of 1.000 suggests that the case university still requires extensive effort to be prepared to implement EDUC4. This score indicates significant gaps in the university’s current infrastructure, initiatives, and teaching–learning processes that must be addressed before it can fully embrace the intricacies of EDUC4. Regarding impact, the ranking of the identified criteria shows that infrastructure, financial resources, and learners emerge as the top three factors the case university must focus on to improve and implement EDUC4 effectively. These findings can be explained in parallel with the theoretical standpoint of the Unified Theory of Acceptance and Use of Technology 2 (UTAUT2). This suggests that the adoption and use of new technologies (e.g., cyber-physical systems, smart classrooms, AR/VR) are influenced by several factors, including facilitating conditions, social influence, and motivation [58]. Infrastructure and financial resources can be seen as facilitating conditions that can influence the adoption and use of new educational technologies [9]. The lack of infrastructure and financial resources can limit the availability and accessibility of educational technologies, prompting barriers to their use [59]. Learners who perceive that other students are using and benefiting from new educational technologies and find them enjoyable and pleasurable are more likely to adopt and use them.

Several strategies can be considered to improve the implementation of EDUC4. For instance, the university can secure funding to support infrastructure development and facilities improvement through various sources, such as government grants, industry partnerships, and international collaborations [60]. More attention must be paid to ensure that the infrastructure and technology systems are in place, which can be achieved through proper planning, implementation, and continuous evaluation. Furthermore, the university can adopt learner-centered approaches to improve the degree of use of EDUC4 tools such as cloud computing, AI, and IoT, among others. This direction is highly relevant, especially since the results show that learners, although affected by the health and environment factor, have high screen time, indicating that if addressed, the utilization of EDUC4 technologies would help optimize the implementation. Hence, providing personalized and interactive learning experiences that engage the students using adaptive learning technologies and project-based learning approaches could facilitate the rapid adoption of EDUC4. On the other hand, faculty development and training programs can be established to equip the university faculty with the skills and knowledge needed to use these new technologies and practices effectively.

In light of these findings, it could be deduced that improving infrastructure, accessing financial resources, and employing learner-centered approaches are critical directions to successfully implement EDUC4. These initiatives require careful planning, implementation, and evaluation, which can be supported through funding, partnerships, and collaborations with other institutions and stakeholders. By implementing them, the case university can effectively embrace the benefits of EDUC4 and contribute to advancing education, especially in the developing economy. Overall, the proposed preparedness indicator system can effectively measure in an encompassing view the preparedness of HEIs in their direction of implementing the EDUC4 as a disruptive paradigm in education. The proposed system considers the overarching scope of EDUC4, and the computational procedure allows a measurable metric that can determine the overall preparedness level of a university. The system offers two general uses: (1) it can be used to compare the overall individual EDUC4 preparedness levels of universities or units within a university, and (2) it can be used to identify hotspots or areas within the university that require greater attention and investment. The first function allows the ranking of universities or their units for the performance evaluation or inputs to reward systems. The second function provides information for resource allocation decisions and long-range planning of universities.

6. Conclusions

This work offers a comprehensive evaluation approach that aims to capture the overall preparedness index of an HEI in its implementation of EDUC4. The proposed indicator system considers a range of criteria that impact an HEI’s preparedness, which allows for a more accurate and in-depth assessment of its readiness for EDUC4. This comprehensive approach can help HEIs identify areas where they need to improve, thus facilitating the development of targeted strategies for the successful implementation of EDUC4. Seven criteria became benchmarks in the proposed indicator system: human resources, infrastructure, financial, linkages, educational management, learners, and health and environment. In each criterion, a set of indicators were identified through a rigorous four-step process. A total of 105 indicators were identified, each with a quantifiable measure designed to be accessible by HEIs. To capture the relative importance of the seven criteria and the inherent uncertainty of the evaluation process, FUCOM and rough set theory were used. FUCOM assigns the priority weights of the criteria, while rough sets handle the uncertainty of the computational process. To demonstrate its application, an actual case study in a state university in the Philippines was implemented.

After a rigorous evaluation of the indicator system within the proposed computational framework, results indicate that human resources are considered top tier, while infrastructure, financial resources, and learners are among the top three that need re-evaluation for EDUC4 implementation. It is noteworthy that technological processes and industrial growth drive EDUC4; however, it is not apparent from the findings. This may imply that the case university is unprepared to implement the educational paradigm. Inadequate infrastructure and financial resources can act as barriers to using educational technologies, limiting their availability and accessibility. Educational technologies may not function optimally without proper infrastructure, such as reliable Internet connectivity and access to appropriate hardware and software. Moreover, the scarcity of financial resources makes it difficult for institutions to acquire and maintain the necessary technologies. These barriers can impact the quality of education, particularly in areas with limited educational resources. Thus, addressing these infrastructure and financial limitations is crucial in ensuring equitable access to educational technologies and ultimately improving educational outcomes, especially during the transition.

This study has limitations that require future consideration. First, the domain literature on EDUC4 is evolving, and some criteria and their indicators may not be identifiable at present. However, future real-life conditions and scenarios require additional consideration of future criteria that are not yet straightforward. At present, the nexus between disaster risk management and EDUC4 is not yet fully understood. Future work may consider other criteria and indicators in the computational framework of the indicator system in order to provide a more holistic overview of EDUC4. Secondly, the measurement for each indicator may be enhanced to translate subjective 7-point scale measures to more measurable indicators. This agenda is a rich resource for future work. Identifying the ideal values for indicators may contain idiosyncrasies in view of the background of the expert group. Thus, future research may consider re-evaluation of indicator measures. Lastly, other emerging priority allocation tools aside from FUCOM may be used to assign the priority weights of the criteria. In addition, the use of fuzzy sets and their extensions may be considered a framework for handling uncertainty in decision making within the computational structure of the EDUC4 preparedness indicator system.