1. Introduction
The digital transformation of higher education has radically changed the structure and logic of the learning process. The integration of artificial intelligence technologies, educational data analytics, adaptive learning management systems, and personalization algorithms has enabled the transition from a standardized teaching model to individualized educational trajectories. Within this paradigm, adaptive learning is seen as one of the most promising areas of development for educational systems, allowing for dynamic adjustments to the content, level of difficulty, and pace of learning based on real-time analysis of data on student progress and behavior.
Despite significant achievements in optimizing knowledge acquisition and improving academic performance, the problem of systematic diagnosis of higher cognitive skills, especially critical thinking, remains underdeveloped. Most existing assessment models focus on test results and the level of mastery of the learning material, while critical thinking is interpreted indirectly through indicators of academic success. This approach does not take into account the multidimensional and processual nature of this construct, which is formed in the course of cognitive, metacognitive, and behavioral interaction between the learner and the educational environment.
Critical thinking is recognized as one of the key meta-competencies of the 21st century, necessary for making informed decisions, analyzing information, reflecting, and developing strategies for action in conditions of uncertainty. Modern digital educational environments generate significant amounts of process data: task completion time, frequency of content access, nature of errors, interface interaction trajectories, system operating modes, and other behavioral parameters. This data opens up fundamentally new opportunities for diagnosing the cognitive characteristics of learners, but there is no formalized model in the scientific literature that allows such indicators to be interpreted as indicators of critical thinking development.
The active introduction of intelligent and generative AI systems into the educational process makes this problem even more relevant. On the one hand, these technologies expand the possibilities for personalization and automated learning support, but on the other hand, they create risks of cognitive dependence and the phenomenon of “cognitive offloading,” in which part of the analytical activity is transferred to algorithmic tools. In these conditions, it is particularly important to develop models capable of distinguishing between improved academic performance and the genuine development of students’ metacognitive and analytical skills.
Critical thinking in this study is understood as a multidimensional cognitive construct that includes analytical reasoning, reflective judgment, and self-regulation in the learning process. In the context of adaptive educational environments, critical thinking is manifested not only through final academic outcomes, but also through the way learners interact with tasks, evaluate alternatives, regulate their decisions, and adjust their learning strategies. Therefore, the present study adopts a process-based perspective, according to which critical thinking can be identified through a combination of behavioral, metacognitive, and strategic indicators generated during interaction with the digital learning environment.
Critical thinking is widely described in the literature as a multidimensional construct that includes analysis, evaluation, inference, and self-regulation [
1,
2,
3]. These frameworks emphasize both cognitive and metacognitive aspects of thinking, including reflective judgment, decision-making, and the regulation of learning processes.
In this study, these dimensions are operationalized through three latent components of the proposed model. Analytical depth reflects core cognitive skills such as analysis and evaluation; metacognitive awareness corresponds to self-regulation and reflective monitoring; and strategic learning captures goal-oriented planning and the organization of learning activity.
Thus, the proposed model is conceptually aligned with established theoretical frameworks of critical thinking while extending them into a process-based analytical context within adaptive learning environments.
Behavioral indicators such as time on task, interaction frequency, and user activity patterns are widely used in learning analytics as proxies for cognitive engagement and learning processes. These findings are supported by recent studies [
4,
5], which demonstrate that adaptive and interaction-based learning environments can significantly enhance student motivation, engagement, and learning outcomes.
In particular, adaptive gamification approaches highlight the importance of behavioral interaction data as meaningful indicators of learning activity and cognitive involvement in digital educational systems.
In response to these challenges, this study proposes a multi-criteria hierarchical model for identifying and quantitatively assessing critical thinking based on individual process indicators of learning activities in an intellectual information and education system. Unlike traditional approaches, which focus primarily on test results, critical thinking is viewed as a latent multidimensional construct that manifests itself through the characteristics of a student’s interaction with the digital educational environment.
The methodological basis of the study consists of a systematic approach, principles of educational measurement, and multi-criteria decision-making methods. The work formalizes a set of individual indicators, including initial and achieved levels of knowledge, the degree of complexity of learning elements (LE), time characteristics, intensity and frequency of content use, error rate, the degree of doubt (metacognitive uncertainty), parameters of the user’s information style, and system operating modes. These variables are aggregated into a personal matrix of the learner’s status, which serves as the basis for constructing a hierarchical model for assessing critical thinking.
To prevent critical thinking from being replaced by academic performance indicators, corrective and restrictive mechanisms have been introduced into the model to ensure a balance between cognitive results and metacognitive characteristics.
The model was empirically tested in a quasi-experimental longitudinal study involving 500 students from higher education institutions, divided into control and experimental groups. During the learning process, process indicators of learning activity were automatically recorded, which made it possible to perform a comprehensive statistical analysis. The reliability and construct validity of the model were assessed using Cronbach’s coefficient and confirmatory factor analysis, which provided statistical justification for its stability and three-component structure.
The scientific novelty of the study lies in the transition from results-oriented assessment to process-oriented diagnosis of cognitive changes in adaptive educational systems. The proposed model combines the capabilities of educational analytics and psychometric modeling, forming an explainable and interpretable tool for assessing critical thinking in a digital environment.
The practical significance of the work lies in the possibility of integrating the developed model into adaptive and intelligent educational platforms for monitoring the development of critical thinking in dynamics, supporting management decisions, and designing personalized educational trajectories. Thus, the research contributes to the formation of intelligent educational ecosystems focused not only on improving academic performance but also on developing independent, reflective, and critically thinking professionals.
2. Literature Review
Adaptive learning is a multifunctional and multifaceted approach to education that significantly changes traditional teaching methods. The theoretical foundations of adaptive learning are based on the interaction of various pedagogical, psychological, and technological aspects and are aimed at individualizing the educational process to most effectively meet the needs of each learner.
One of the key theories on which adaptive learning is based is the concept of individual differences. Each learner is unique: they have their own learning styles, prerequisites, interests, and pace of learning. From the perspective of Howard Gardner’s theory of multiple intelligences (1983) [
6], it should be taken into account that students may have different types of intelligence, and learning can be more effective if its content and methods are adapted to these differences. Thus, adaptive learning allows for the use of a variety of approaches and tools to promote the development of each participant in the educational process.
One of the central sources in this area is the work of Gransden et al. [
7]. This work examines the conceptual foundations of adaptive learning, methods for its implementation in higher education, and examples of the successful introduction of such technologies into the educational process.
In addition, the work of Asmara [
8] is of great importance. It shows how modern digital technologies influence the personalization of the learning process and what tools can be used to create adaptive educational environments. Asmara examines the impact of adaptive learning systems on improving educational outcomes in the context of EdTech development. The author analyzes the concept of adaptive learning as a technology based on algorithms for analyzing student data, dynamic diagnosis of the level of preparation, and automatic personalization of content. It is emphasized that adaptive systems provide individual learning trajectories and vary the complexity of tasks and the pace of material delivery depending on the student’s cognitive profile.
Ahmad et al. [
9] present a review of the use of data-driven AI-based applications in education. These applications are used for student grading and assessment, prediction of student retention and dropout, sentiment analysis, intelligent tutoring systems, classroom monitoring, and recommender systems. In a similar line of research, Strielkowski et al. [
10] describe how artificial intelligence can be used to analyze student data and design individualized educational trajectories.
The technological aspects of adaptive learning also form an important theoretical basis. The development of information technologies and learning management systems expands the possibilities for implementing adaptive approaches. Intelligent educational systems that use machine learning algorithms can analyze educational data, identify patterns in student learning, and recommend individualized learning resources. This makes it possible to organize the learning process more effectively. Such technological platforms focus on analyzing the behavior and performance of individual learners, allowing timely adjustments and improvements to the structure of educational materials.
Adaptive learning is one of the most promising areas in education, ensuring personalization of the learning process, which is especially relevant in a rapidly changing world. Adaptive learning systems enable each student to receive materials that correspond to their level of preparation and pace of learning.
In recent years, various technologies have been used to implement adaptive learning. Platforms based on big data analysis and machine learning make it possible to construct individualized educational trajectories. According to Thompson et al. [
11], machine learning tools are successfully integrated into the process of assessing and monitoring student performance, enabling timely adjustments to teaching materials.
Adaptive learning management has become a relevant and important topic in the modern educational process. It involves the individualization of educational trajectories while taking into account the characteristics of each student, including their level of knowledge and learning pace. In recent years, this topic has attracted increasing attention from researchers and practitioners, which is reflected in the growing body of scientific literature.
Adaptive learning systems that consider individual learner characteristics are widely used in international practice. Such systems analyze student behavior and performance, adapting the content and complexity of learning materials in real time. Adaptive learning therefore involves the use of technological solutions such as learning management systems (LMS) as well as methodological approaches that take into account the psychological and pedagogical aspects of knowledge formation.
Adaptive learning can be described as a personalized educational process within an intelligent learning system (ILS) that dynamically changes depending on the learner’s progress and needs. Unlike traditional approaches, adaptive systems use student data to tailor educational content and learning strategies.
According to Katsaris and Vidakis [
12], adaptive learning systems increase student engagement and improve learning effectiveness through individualized learning trajectories.
The main components of adaptive learning include the following:
Knowledge assessment—analysis of the student’s current level of preparation.
Content adaptation—personalization of learning materials based on the student’s abilities.
Interactive learning process management—use of algorithms and artificial intelligence to dynamically adjust learning tasks.
Assessment of results and feedback—analysis of student progress data.
El-Sabagh [
13] demonstrated that intelligent learning systems can improve student outcomes through personalized approaches and rapid analysis of learner progress.
Adaptive learning management systems (LMS) can tailor content and learning tasks based on student progress. Bagustari and Santoso [
14] describe the key features of modern adaptive LMS, including adaptive testing mechanisms and automatic adjustment of task difficulty levels.
Bahroun et al. [
15] present a comprehensive bibliometric and content analysis of research on generative artificial intelligence in educational environments. The authors report a sharp increase in publication activity after 2020 and identify several key research directions, including automated feedback generation, personalization of learning, generation of educational content, and issues related to academic integrity. The study emphasizes that generative AI is becoming a transformative tool for educational ecosystems, although its implementation is accompanied by ethical and methodological challenges.
Barker et al. [
16] examine the use of generative artificial intelligence as an innovative pedagogical tool in teaching professional communication in the field of dietetics. Their results demonstrate that AI-assisted learning contributes to the development of clinical communication skills, supports formative assessment, and increases student engagement, highlighting the practical potential of these technologies in professional education.
A systematic analysis of artificial intelligence applications in higher education is presented by Zawacki-Richter et al. [
17]. The authors identify a significant imbalance between technological developments and pedagogical research, concluding that the role of teachers must be strengthened in the design and implementation of AI-driven educational solutions. Earlier, Popenici and Kerr [
18] conceptually analyzed the influence of artificial intelligence on teaching and learning processes. Their study highlights the potential of AI technologies to transform assessment methods, teacher–student interaction, and institutional models of higher education, emphasizing the need to reconsider the pedagogical role in the context of digital transformation.
Mustafa et al. [
19] present a meta-review of literature reviews in the field of Artificial Intelligence in Education (AIED). Their work proposes a roadmap for future research priorities, including improving algorithm transparency, developing inclusive AI solutions, strengthening interdisciplinary collaboration, and conducting empirical studies to evaluate the effectiveness of AI-based educational systems.
The strategic and organizational aspects of digital transformation in education are discussed by Meng and Qin [
20], who propose a model for managing an innovative educational network based on artificial intelligence technologies. The system is designed to optimize management processes, coordinate resources, and increase the effectiveness of educational initiatives. Similarly, George and Wooden [
21] consider artificial intelligence as a tool for the strategic transformation of higher education, emphasizing the importance of leadership, digital culture, and institutional strategy for successful technology integration.
The methodological foundations for assessing educational outcomes are presented in the classical works of Brown [
22] and Ebel and Frisbie [
23]. Brown’s early research on correlations among mental abilities laid the foundation for factor analysis and psychometric modeling, which remain essential for modern intelligent assessment systems. Ebel and Frisbie systematized the principles of educational measurement, including reliability, validity, and test standardization, forming the methodological basis for the development of automated AI-based knowledge assessment systems.
Kassenkhan et al. [
24] present a review of modern approaches in which gamification and artificial intelligence are considered synergistic tools for developing critical thinking. The authors systematize existing gamification models (point systems, scenario-based quests, and problem-oriented games), compare them with AI technologies (adaptive recommendation systems, interactive tutors, and natural language processing tools for reasoning analysis), and demonstrate how the combination of game mechanics and algorithmic personalization can stimulate motivation, maintain engagement, and guide learners toward tasks of increased cognitive complexity.
Serbin and Kassenkhan [
25] present an applied study in the field of sensor analytics for mobile devices. The authors propose and validate a behavioral authentication model based on gyroscope and accelerometer data aimed at improving secure access to mobile educational resources. Their work also demonstrates the possibility of linking behavioral sensor patterns with cognitive indicators, for example by analyzing correlations between device interaction patterns and reasoning behavior or time spent on tasks.
Practical confirmation of the effectiveness of mobile learning technologies is presented by Wigley [
26], who demonstrates increased employee engagement and productivity through the implementation of mobile learning in the Jaguar Land Rover corporate training environment. Trede et al. [
27] examine learning in hybrid spaces and propose a framework for developing mobile technology competencies for workplace learning. The authors emphasize that modern professional training occurs at the intersection of physical and digital spaces, with mobile technologies acting as a bridge between formal, informal, and practice-oriented learning.
A systematic review of artificial intelligence applications in higher education is presented by Zawacki-Richter et al. [
17]. The authors conclude that most research focuses on learning analytics, automated feedback systems, and intelligent tutoring systems, while the pedagogical dimension and the role of educators remain comparatively underexplored. They emphasize the importance of integrating educational theory into the development of AI-based learning technologies.
The issue of digital transformation in Kazakhstan is explored in several studies. Bokayev et al. [
28] analyze the experience of implementing distance learning during the COVID-19 pandemic and identify institutional and regulatory gaps, including insufficient infrastructure readiness and policy inconsistencies that affected the effectiveness of remote learning. Abilkhairova et al. [
29] investigate academic analytics in the context of online education in Kazakhstan and emphasize the importance of educational data analysis for monitoring learning quality. Ibrayeva and Yegemberdiyeva [
30] evaluate the level of digital transformation in Kazakhstan’s education system, noting the steady growth of digital initiatives alongside the need for stronger strategic coordination and improved digital competencies among teachers.
The development of 21st-century skills is analyzed by Thornhill-Miller et al. [
31], who examine methods for assessing and certifying creativity, critical thinking, communication, and collaboration. Their study emphasizes the importance of reliable measurement tools and interdisciplinary methodologies for evaluating these competencies. Hao et al. [
32] further explore the mediating role of critical thinking in the relationship between motivation and creativity among learners in AI-supported educational environments.
Experimental models aimed at developing critical thinking through gamification and artificial intelligence are presented in several studies. Naatonis et al. [
33] evaluate the Problem-Based Gamification Learning (PBGL) model integrated with artificial intelligence approaches and the ChatGPT API. Their experimental results demonstrate a statistically significant increase in the level of critical thinking among students in the experimental group. Nazneen [
34] investigates the integration of gamification into curriculum design and shows that game-based mechanisms positively influence the development of decision-making, problem-solving, and critical analysis skills among adolescents. Daghestani et al. [
35] analyze the adaptation of gamified learning systems using educational data mining methods and demonstrate that intelligent adaptation of game scenarios improves both learning personalization and academic performance.
Xu and Liu [
36] provide a comparative analysis of AI-integrated educational tools, specifically Duolingo and ChatGPT, in terms of their influence on learner motivation, enjoyment, autonomy, and critical thinking development. The study demonstrates that Duolingo’s gamified mechanisms enhance engagement and sustained motivation, while the use of ChatGPT promotes learner autonomy and analytical thinking when supported by appropriate pedagogical strategies.
Roopaei and Roopaei [
37] propose a gamified approach to artificial intelligence education for children through the TransAI project. Their work highlights the importance of early AI literacy and demonstrates how game-based scenarios, quests, and interactive tasks can support the development of algorithmic thinking and critical reasoning skills. The authors show that gamified representations of complex technological concepts increase both accessibility and motivation among young learners.
Priante and Tsekouras [
38] analyze the impact of game-based response systems in traditional classrooms. The authors find that the integration of digital interactive tools into the physical classroom increases student engagement, improves feedback quality, and enhances the depth of understanding of the material, thereby creating a hybrid model of educational interaction.
A systematic review by Zhai et al. [
39] addresses the problem of excessive dependence on AI dialogue systems. The authors warn about the risks of cognitive “outsourcing”, a decline in the ability to analyze information independently, and the potential degradation of metacognitive skills resulting from uncontrolled AI usage. They conclude that a balance between automated support and independent cognitive activity is essential.
The pedagogical foundations of critical thinking development are examined by Athanassiou et al. [
40], where Bloom’s taxonomy is considered a framework for designing learning activities in management education. The study demonstrates that structuring learning tasks according to levels of cognitive complexity contributes to the development of analytical and evaluative skills.
Hao and Tasir [
41] propose a theoretical framework for Massive Open Online Courses (MOOCs) incorporating gamification elements aimed at developing higher-order thinking skills. The authors emphasize the importance of integrating game mechanics with well-designed pedagogical architectures of online courses to enhance learner cognitive engagement.
A theoretical analysis of gamification in education is presented by Astashova et al. [
42], where gamification is considered a systemic resource for modernizing educational environments. The authors highlight the motivational, cognitive, and social effects of game mechanics and emphasize the importance of scientifically grounded design of gamified learning environments. In a related context, Gerlich [
43] examines the phenomenon of cognitive offloading associated with the widespread use of AI tools. The author analyzes how automated assistants influence critical thinking development and highlights the dual nature of artificial intelligence: while it can enhance analytical capabilities, passive use may reduce cognitive independence.
Alam [
44] explores the opportunities and challenges associated with the integration of artificial intelligence in education. The study identifies significant opportunities for learning personalization, intelligent analytics, and automated assessment while also emphasizing risks related to unequal access, ethical dilemmas, and data privacy concerns. Bucchiarone [
45] proposes a model for integrating gamification and virtual reality technologies within digital twin-based learning systems. The author describes the architecture of digital twin learning environments and identifies technological and organizational challenges, including system complexity and the need for interdisciplinary collaboration.
An interdisciplinary perspective on the development of artificial intelligence is provided by Dwivedi et al. [
46]. The authors propose a research agenda addressing technological, managerial, social, and policy-related aspects of AI development, emphasizing the need for comprehensive approaches to regulating and implementing AI technologies.
Finally, Gunning and Aha [
47] describe the DARPA Explainable Artificial Intelligence (XAI) program aimed at developing transparent and interpretable AI models. The authors highlight the importance of algorithm transparency for building user trust and enabling the responsible deployment of AI systems in socially significant domains such as education.
The systematization of scientific approaches to adaptive learning and digital transformation of education is presented in
Table 1. The analysis shows that adaptive learning represents a multi-level system formed at the intersection of theoretical, methodological, technological, and managerial components. Its conceptual foundation is based on psychometric approaches, principles of educational measurement, and pedagogical models of cognitive skill development that ensure valid knowledge assessment and objective evaluation of educational outcomes. At the same time, the technological basis of adaptive learning relies on artificial intelligence, machine learning, educational data analytics, and intelligent learning management systems that enable dynamic adaptation of educational content, learning pace, and task complexity.
Artificial intelligence has become a key driver of personalization in education, enabling the creation of individualized learning paths in real time. By analyzing data on student progress and learning behavior, adaptive systems support the transition from standardized instructional models to individualized approaches tailored to the needs of each learner [
7,
8,
10]. The cognitive and motivational effects of adaptive learning are further strengthened through the integration of gamification, mobile technologies, and hybrid learning formats, which increase learner engagement and support the development of higher-order skills [
24,
34,
35,
38].
Contemporary research pays particular attention to the development of critical thinking as a key meta-competence of the 21st century [
31,
32]. At the same time, scholars emphasize that the formation of higher cognitive skills is possible only when digital technologies are integrated into pedagogically grounded learning designs rather than implemented mechanically. The scientific literature also highlights the risks of cognitive dependence on intelligent systems and the phenomenon of “cognitive offloading”, which requires ensuring the transparency, explainability, and controllability of algorithms [
39,
43,
47].
The successful implementation of adaptive learning also depends on the strategic management of digital transformation, the development of digital culture in educational organizations, and the improvement of teachers’ digital competencies [
21,
28,
30]. The current stage of development of educational technologies is characterized by a transition from the local use of digital tools toward the creation of intelligent adaptive educational ecosystems. In such ecosystems, artificial intelligence technologies are combined with pedagogical theories and management mechanisms, ensuring a balance between automation and the development of learner autonomy [
19,
46].
While the existing literature provides a broad overview of adaptive learning and AI in education, relatively limited attention has been given to process-based indicators for diagnosing critical thinking. This gap motivates the development of the proposed model, which focuses on behavioral and metacognitive indicators within intelligent learning environments.
3. Materials and Methods
The research methodology is based on systematic, qualimetric, and multi-criteria approaches to managing the learning process in intelligent educational systems. The research is aimed at developing and formalizing a multi-criteria decision-making model that provides adaptive learning management based on individual student indicators, including knowledge level, behavioral characteristics, degree of doubt (metacognitive uncertainty), and critical thinking.
3.1. General Research Outline
The research was carried out in several stages:
Analysis of existing approaches to adaptive learning, psychometric methods of knowledge assessment, and multi-criteria decision-making models in education.
Formalization of individual student indicators characterizing cognitive, behavioral, and metacognitive features.
Development of a mathematical multi-criteria model for assessing the level of knowledge and making management decisions in an information and training system.
Designing algorithms for adaptive control of learning elements and system operating modes.
Experimental verification of the model in the conditions of an intelligent information and training system.
The methodological basis of the research is presented at the level of user data interaction, multi-criteria assessment of the state of the learning element, and logical rules for decision-making, which ensures the integrity and reproducibility of the results obtained.
The experimental design and the differences between the control and experimental groups are described in detail in
Section 3.2.
3.2. Experimental Intervention Design
The quasi-experimental study was conducted in two parallel instructional conditions. Both the control group (CG, n = 250) and the experimental group (EG, n = 250) studied the same academic content within the same intelligent learning environment and over the same instructional period. The difference between the groups concerned the pedagogical design of interaction with the system rather than the subject content itself.
In the control group, students used the standard adaptive learning mode. This mode included sequential work with learning elements, routine knowledge assessment, and conventional feedback based mainly on correctness of answers and progress through the material. Adaptation was limited to the ordinary adjustment of content sequence and difficulty according to current performance indicators.
In the experimental group, students used an extended adaptive learning mode specifically designed to foster critical thinking. This mode incorporated a structured set of higher-order tasks, including problem-analytical, diagnostic, interpretive, argumentative, reflective, and metacognitive activities. In addition to correctness-based feedback, the system provided reflective prompts encouraging students to justify their answers, reconsider errors, compare alternatives, and monitor their own confidence and doubt during task completion. The system also applied dynamic regulation of task complexity and learning trajectories using process indicators such as error rate, time to mastery, number of learning steps, level of doubt, and interaction patterns.
Thus, the experimental intervention was defined as a combination of pedagogical scaffolding, reflective feedback, and process-aware adaptive support aimed at strengthening metacognitive awareness, analytical depth, and strategic learning activity. This design distinguishes the EG condition from the standard adaptive instruction used in the CG and provides the causal basis for interpreting intergroup differences in the critical thinking indicator.
The control and experimental groups differed in terms of task structure, feedback mechanisms, and adaptive support. These differences are systematically presented in
Table 2.
3.3. Methods of Data Collection and Presentation
Behavioral and interaction-based indicators used in the model are grounded in the field of learning analytics, where variables such as time on task, frequency of interaction, error rate, and user activity patterns are widely employed as proxies for cognitive engagement and learning processes. These indicators reflect how learners interact with content, make decisions, and regulate their activity within the educational environment.
It should be noted that such indicators are not treated as direct measures of critical thinking. Instead, they are interpreted as indirect, process-based features that provide insight into cognitive and metacognitive processes underlying learning behavior. This approach allows capturing dynamic aspects of thinking that are not accessible through traditional outcome-based assessments.
The relationship between the selected indicators and critical thinking is established through their connection to underlying cognitive and metacognitive processes. For example, indicators such as error rate, time to mastery, and number of solution steps reflect analytical reasoning, problem-solving strategies, and depth of information processing.
At the same time, interaction-based indicators, including user activity patterns and the “doubt” parameter, capture aspects of metacognitive monitoring, uncertainty recognition, and self-regulation. These processes are widely recognized as core components of critical thinking.
Thus, the proposed model does not directly measure critical thinking as an outcome, but rather captures its process-level manifestations through observable learning behavior in adaptive environments.
The initial dataset used in this study includes the results of initial, intermediate, and final testing, as well as behavioral data obtained from students during their interaction with the information and learning system. Data collection is performed automatically and includes the following indicators:
Results of responses to test tasks with varying levels of complexity;
Time characteristics of task completion;
Frequency and periodicity of access to learning elements;
Parameters of user interaction with the interface (information interaction patterns);
Indicators reflecting the level of doubt and confidence in decision-making.
All indicators presented in
Table 3 are aggregated into a personal matrix representing the learner’s status. This matrix is dynamically updated at each stage of the learning process and serves as the input data for the proposed multi-criteria evaluation model.
The “doubt” parameter S is interpreted as a proxy for metacognitive uncertainty rather than a direct measure of metacognitive awareness. It reflects observable behavioral patterns such as hesitation, repeated attempts, and instability in decision-making during interaction with the learning environment.
It should be noted that this parameter represents an exploratory construct and has not yet been fully validated against established metacognitive assessment instruments. Further empirical validation is required and represents an important direction for future research. Among these indicators, particular attention is given to the “doubt” parameter as a representation of metacognitive processes.
The computational representation of the learner’s mastery of educational material in the information and learning system is constructed from the personal matrix
W according to Equation (
1):
where
W is the matrix representing the learner’s status in the Intelligent Learning System (ILS), and
n denotes the number of learning elements.
The model of user mastery of training material is based on the identification of their individual characteristics during interaction with the system. The system analyzes the personal properties of each user, forming their unique profile. This profile serves as a tool for determining the current status of the learner, including their level of knowledge, error rate, nature of interaction with information, and other parameters.
Upon registration, the user takes an entrance test, the results of which are evaluated according to several criteria and automatically saved in their personal database (agent matrix). This data is used to ensure a comfortable educational process, adapt content, manage the level of learning complexity, and make decisions while working with the system.
During the learning process, information about the user is dynamically updated and adjusted. The system continuously monitors personal parameters at each stage of learning, updating the learner’s profile.
In this context, the model incorporates a range of behavioral and interaction-based indicators derived from the learning process. Although the model includes multiple indicators, they are systematically grouped into three latent components, which reduces the overall complexity at the conceptual level.
In addition, the model is designed as a modular framework, where the set of indicators can be adapted, reduced, or extended depending on the specific educational context and data availability. This flexibility ensures both analytical expressiveness and practical applicability of the approach. Future work may also explore data-driven dimensionality reduction techniques or indicator selection methods to further optimize the model.
3.4. Formation of Individual Student Indicators
The methodology involves the use of a set of quantitative and latent indicators that reflect the process of mastering learning elements. Key indicators include initial and achieved levels of knowledge, the degree of complexity of the learning element, the intensity and frequency of its use, the time taken to master it, the error rate, the system’s operating mode, and the level of critical thinking.
Particular attention is paid to assessing the level of doubt of the learner, which is considered a hidden metacognitive parameter. Its quantitative assessment is based on the analysis of decision-making time, repeated answer selections, inconsistencies between decisions on questions of varying levels of complexity, and user behavioral responses. This approach allows for cognitive uncertainty that is not reflected in traditional knowledge assessment models.
3.4.1. Initial Knowledge Level of the Learning Element
The initial knowledge level of learning element n, denoted by , is entered by the user or calculated based on the results of an entrance test.
The initial knowledge level of the next learning element,
, is assumed to be equal to the final knowledge level of the previous element:
The condition for transition to subsequent learning elements is defined in terms of the achieved (final) knowledge levels as follows:
The procedure for calculating the percentage of mastery for each learning element is summarized in Algorithm 1.
| Algorithm 1 Procedure for calculating the percentage of mastery. |
The entrance test is divided into learning elements (paragraphs or subtopics). Each learning element contains n questions of varying difficulty levels (A—easy, B—medium, C—difficult, Z—advanced/Olympiad level, etc.). Points are awarded for correct answers according to the corresponding difficulty level. As a result of the test, the percentage of mastery of each learning element is determined using Equation ( 4).
|
where
is the percentage of mastery characterizing the level of knowledge of the learning element.
Let denote the number of points obtained for the correct i-th element of difficulty level a (easy), the number of points for the correct i-th element of level b (medium), the number of points for the correct i-th element of level c (difficult), and the number of points for the correct i-th element of level n (advanced level). Let denote the number of questions corresponding to each level of difficulty. is the maximum possible number of points.
The value of
is determined by the formula:
where
is the maximum number of points for easy questions (level
a),
is the maximum number of points for medium-difficulty questions (level
b), and
is the maximum number of points for difficult questions (level
c).
3.4.2. Achieved Level of Knowledge of the Learning Element
The achieved (final) knowledge level of the learning element n, denoted by , is determined using a multi-criteria evaluation model that integrates the indicators described above.
The final knowledge level
is computed according to Equation (
4).
3.4.3. Estimation of the Degree of Doubt for Learning Elements
The user’s level of doubt is a latent measurement parameter that cannot be directly observed. It plays a significant role in knowledge assessment, as it reflects the psychological characteristics of human behavior during decision-making. The structure of the doubt level includes latent characteristics such as decision-making time, frequency and intensity of actions, reaction to external disturbances, the selection process, behavior in artificially created situations, and questionable user actions.
To evaluate these characteristics, quantitative indicators are required, including the number of missed operations, the amount of unconfirmed information, the frequency of interruptions in the logical decision chain, and other behavioral metrics.
The proposed method for accounting for the level of doubt is based on identifying fluctuations in the decision-making process by comparing them with similar decisions. Within this model, the latent parameter is estimated based on quantitative characteristics based on quantitative characteristics, including the subject domain, task complexity, and the decisions made by the learner.
A test is organized consisting of x questions, which are divided into t groups according to a specific topic. Each question has a single answer option. All x questions are distributed across n levels of difficulty, and each question corresponds to a particular difficulty level.
The distribution of questions by difficulty level must satisfy the following condition: when , easy questions constitute , medium questions , and difficult questions .
Each difficulty level is assigned a weighting coefficient F, where corresponds to the most difficult level, represents the next level of difficulty, and so forth.
Questions that are answered correctly at higher difficulty levels while lower-level questions remain unanswered may indicate the presence of doubt. The doubt associated with answering a question of difficulty level
F is calculated based on responses to questions of the same topic (i.e., from the same group) that have lower difficulty than the current question.
where
m is the number of difficulty levels below the one for which the calculation is being made.
F is the weight coefficient of the current level of complexity;
is the weighting coefficient reflecting the contribution of lower-level responses to the degree of doubt.
—total weight of doubts for question
i-th level:
where
g is the number of questions of the
i-th level of difficulty on the same topic, and
denotes the response to the
l-th question at this level:
Given that there may be several questions of the same level of difficulty in the test, it is necessary to find the root mean square value of doubt at each level:
The final coefficient characterizing the degree of doubt is equal to
where
is the degree of doubt associated with answers to questions of difficulty level
F,
is the weighting coefficient of doubt associated with difficulty level
F.
where
is the maximum weight of the highest level.
3.4.4. Degree of Complexity and Weight of the Learning Element
The system implements an adaptive algorithm for self-regulation of learning content complexity, allowing the learning process to dynamically adjust to the learner’s current level of preparation.
A flowchart of the adaptive complexity adjustment algorithm is shown in
Figure 1.
The main variables of the model are defined as follows:
—current level of content complexity;
—learner’s level of preparation;
—performance mismatch (cognitive overload or underload);
—control parameter for regulating complexity;
The algorithm operates according to the principle of negative feedback. After each learning step, the system evaluates the learner’s performance based on the mismatch . If the learner successfully handles the task, the complexity level is increased. If the performance declines or errors occur, the complexity is reduced to prevent cognitive overload. In cases of stable performance, the complexity level remains unchanged. This mechanism ensures gradual adaptation and prevents abrupt fluctuations in task difficulty.
The key objective of the algorithm is to achieve the equilibrium state , which corresponds to a balance between content difficulty and learner ability. In this state, the cognitive load is optimal, and the learner operates within the zone of proximal development.
From a mathematical perspective, the model can be interpreted as a discrete dynamic system in which the level of content complexity is iteratively adjusted based on the current mismatch . This process ensures convergence toward a stable learning trajectory and supports personalized adaptation.
The adaptive logic of the algorithm can be formalized as follows:
where
is a threshold parameter defining the acceptable deviation between the content complexity and the learner’s level of preparation.
3.4.5. Intensity of LE Use
Intensity of use is characterized by the frequency of visits to the LE over a certain period of time:
where
N is the number of learning steps;
is the total time interval spent on mastering the learning element, denoted as
T in subsequent sections.
3.4.6. Frequency of Use of the Learning Element
The frequency Freq of use of the LE is characterized by the following formula:
where
is the time of the
i-th interaction with the learning element;
n is the number of the learning element;
k is the number of observation intervals.
Freq reflects the average time interval between consecutive interactions with the learning element and is inversely related to the intensity of use.
3.4.7. Time Spent on Mastering the LE
During the study of a learning element, the time T spent by the user on mastering it is recorded.
3.4.8. Number of Learning Steps
The number of learning steps N represents the number of successful interactions required to achieve mastery of the learning element.
3.4.9. User’s Information Signature
The user’s information signature
D is a model of “user–mouse” interaction, since virtually all actions are performed using this device. The selection of ten parameters for the pointer’s trajectory on the monitor screen is justified based on the results of previous studies of keyboard signatures [
24].
—time of movement of the manipulator before stopping;
L—length (linear) of the trajectory:
where
and
are coordinates, and
is the number of sampled trajectory points, including the end of the trajectory;
represents the average instantaneous speed of the pointer estimated over discrete time intervals based on sampled trajectory data:
where
is the step along the time axis at which trajectory readings are taken. In this work, the minimum value for PC
ms is used.
—initial speed of the manipulator, speed of the first 10% of the trajectory length:
where
is the time of movement of the pointer at the initial interval of the trajectory;
i corresponds to the first point at which the cumulative trajectory length reaches at least 10% of the total trajectory length.
is the length of the initial section of the trajectory: ;
—the time the button is held down on the manipulator;
—maximum speed of the pointer along the trajectory:
—time between the pointer stopping and the confirmation button press on the manipulator:
where
is the time interval from the start of pointer movement to pressing the manipulator button;
—angle between the direction of initial movement (to the third vertex of the trajectory) and the line connecting the initial and final points of the trajectory:
—the root mean square deviation of the trajectory from the linear path of the pointer. It is defined as the distance between the actual path of movement and the straight line connecting the first and last points of this trajectory.
In this case, the shortest path is the segment connecting the starting and ending points of the trajectory, representing the optimal route between them.
where
is the distance from the
i-th point of the trajectory to the shortest path line,
are the coordinates of the
i-th point of the trajectory, and
are the coordinates of the intersection point of the perpendicular line dropped from the
i-th point of the trajectory with the shortest path line.
Vector D is characterized by .
Aggregation of the information interaction signature.
The vector of interaction parameters is transformed into a scalar indicator through normalization and aggregation.
Each parameter is first normalized to the interval
to ensure comparability. The aggregated indicator
is then calculated as the average value of the normalized components:
where
are the normalized values of the corresponding parameters and
M is the number of interaction parameters.
This approach provides a compact representation of user interaction behavior while preserving the relative contribution of each parameter. More advanced aggregation methods (e.g., weighted aggregation or dimensionality reduction techniques) may be considered in future research.
This formulation allows the use of the model across different interaction modalities with appropriate adaptation of input parameters.
3.4.10. Error Rate During LE Verification
where
is the number of errors in testing,
is the number of correct answers,
is the number of incorrect answers.
E is normalized to the interval [0, 1].
3.4.11. Information and Training System Operating Modes
When registering in the system, the user is prompted to select the system operating mode. There are 10 alternative options available:
Training (closed mode). This involves sequential study of the material from the beginner level to full mastery of the course.
Training (open mode). The content, task complexity, and assessment are adapted based on the aggregated performance of learners within a defined group.
Corrective mode. Analyzes the student’s current level of knowledge and allows gaps in knowledge to be corrected.
Practice mode. Training is based on constant feedback from the user and is focused on practicing tested knowledge.
Self-education mode. Automation of learning content generation is disabled. The user receives only key information and then deepens their knowledge independently.
Examination mode. Generates an exam based on the initial and final level of knowledge, as well as the user’s personal characteristics. Allows checking achievements and assigning a rating and assessment.
Adaptive mode. Optimizing the learning process and potentially reducing the time required to master the material. The system analyzes the user’s level of knowledge and skips topics that have already been mastered with a minimum probability of errors.
Mode with learning level setting. Allows setting the required level of knowledge to be achieved during the learning process.
Learning mode. The system adapts continuously based on user interactions, analyzing progress, access frequency, task responses, and other behavioral parameters.
Intelligent mode. Uses elements of artificial intelligence, adjusting algorithms to the individual characteristics of the learner, generating personalized content, and optimizing the learning process.
A brief description of the intelligent learning environment is provided below to contextualize the implementation of the proposed model.
The intelligent e-learning system used in this study provides adaptive content delivery, real-time monitoring of learner performance, and dynamic adjustment of task complexity based on user interaction data.
The system maintains an evolving learner profile that is continuously updated throughout the learning process, enabling personalized learning trajectories and supporting the implementation of adaptive learning strategies.
3.5. Critical Thinking Assessment Model Based on Individual Indicators
Critical thinking is considered a latent variable, manifested through:
M—the level of metacognitive awareness of the learner, reflecting the ability to reflect, question, and analyze their own mistakes;
A—the level of analytical depth of thinking, characterizing the degree of meaningful processing of educational material;
—the level of strategic learning activity, reflecting the stability and awareness of the learning strategy in the information and educational environment.
Let us denote the aggregated indicator of critical thinking as
where
denotes the normalized critical thinking score, normalized in the range
.
denotes the final aggregated critical thinking score obtained by weighting individual learning elements.
All variables are normalized to the interval , including inversion where appropriate (e.g., for error-related indicators).
A conceptual representation of the proposed approach is shown in
Figure 2.
The detailed structural model is presented in
Figure 3.
3.5.1. Structure of the Formal Model (Hierarchical)
Submodel of metacognitive awareness (
M):
where
are the weight coefficients of the indicators that form the metacognitive component;
S is the degree of doubt (metacognitive uncertainty); E is the normalized error rate when testing knowledge of the educational content (inverted during normalization);
is the user’s information signature, reflecting individual characteristics of interaction with educational content (speed, sequence, returns, pauses).
Interpretation: the ability to doubt, analyze errors, and demonstrate a conscious style of work.
Analytical depth submodel (
A):
where
are the weight coefficients of the indicators that form the analytical component A;
is the initial level of knowledge of the learning element (LE) recorded before the start of its mastery;
is the level of knowledge of the learning element achieved after completion of training;
C is the degree of complexity of the learning element;
T is the time spent by the learner on mastering the learning element;
N is the number of learning steps required to master the learning element.
Interpretation: depth of information processing, taking into account the complexity of the learning element.
Submodel of learning strategy (
):
where
are the weight coefficients of the indicators that form the strategic component ;
I is the intensity of use of the learning element (frequency of use per unit of time);
Freq is the frequency of use of the learning element (regularity of access);
R is the mode of operation in the information and educational environment (independent, adaptive, supported, mixed, etc.). The variable R is treated as a categorical parameter and is encoded using a normalized scale in the interval , where different modes are assigned values based on their level of adaptivity and system involvement. This allows integrating R into the quantitative model. The differentiation between system modes (e.g., adaptive vs. intelligent) is based on the level of autonomy of the system and the degree of personalization and decision-making support provided to the learner.
Interpretation: the ability to develop a sustainable and conscious learning strategy.
The coefficients , , and used in the sub-models were selected based on normalization principles and empirical balancing of indicator contributions within the multi-criteria modeling framework. This ensures comparability of indicators and stability of the model outputs. All coefficients are explicitly defined within the model specification to ensure transparency and reproducibility.
Aggregated model of critical thinking:
where
—weight coefficients of latent components of the model.
Recommended scale configuration (based on expert-informed multi-criteria modeling):
The weight coefficients = 0.4, = 0.4, = 0.2 were determined based on expert-informed reasoning within the framework of multi-criteria decision-making (MCDM). Greater weights were assigned to metacognitive awareness and analytical depth, as these components are considered fundamental to critical thinking development, while strategic learning activity was assigned a lower weight due to its more supportive and context-dependent role in the learning process. This weighting scheme reflects the relative importance of cognitive and metacognitive processes in critical thinking formation.
Limitations and Adjustments
To prevent the model from degenerating into a performance-based assessment, the following constraint is introduced:
where
is a tolerance coefficient reflecting acceptable variation between model components.
In cases where the condition
is observed, the value of
A is capped as
The selected range of ensures model stability and prevents the analytical component from disproportionately dominating the overall critical thinking assessment.
Sensitivity analysis indicates that moderate variations of do not significantly affect the overall behavior of the model, confirming its robustness.
Aggregation across learning elements:
where
is the number of learning elements.
The weights represent the relative importance of individual learning elements in the overall aggregation of the critical thinking score. These weights are assigned based on instructional significance, content complexity, and the position of each learning element within the learning sequence.
This weighting scheme accounts for the non-uniform contribution of different elements to the overall learning outcome and ensures a more accurate and interpretable aggregation of the model results. In this study, the weights are determined based on expert-informed structuring of the learning process.
3.5.2. Validation of the Critical Thinking Model
The developed critical thinking model was empirically evaluated to assess its reliability, construct validity, and practical applicability in an information and educational environment. The model considers critical thinking as a latent multidimensional construct formed on the basis of process indicators of educational activity, which required a comprehensive approach to its verification.
The model demonstrated satisfactory internal consistency (Cronbach’s ) and acceptable construct validity, as confirmed by Confirmatory Factor Analysis (CFA). The goodness-of-fit indices indicated an adequate model fit, supporting the multidimensional structure of critical thinking.
Sensitivity Analysis
To evaluate the robustness of the proposed model, a sensitivity analysis was conducted by varying the weight coefficients within a range of to . The results showed that such variations do not substantially affect the relative differences between groups or the overall trends of the critical thinking indicator, indicating the stability of the model.
3.6. Statistical Analysis
Statistical analysis was performed to evaluate differences between the control and experimental groups and to assess the reliability and validity of the proposed model. Descriptive statistics (mean values and standard deviations) were calculated for all key indicators.
To compare the pre-test and post-test results between groups, parametric statistical tests were applied (independent samples t-test (two-tailed)). Statistical significance was evaluated at the levels of p < 0.05 and p < 0.01. Effect sizes were additionally estimated using Cohen’s d to assess the magnitude of observed differences.
The reliability of the model was evaluated using Cronbach’s alpha coefficient, while construct validity was assessed through confirmatory factor analysis (CFA). The CFA was conducted using standard maximum likelihood estimation.
To verify the applicability of parametric methods, the distribution of key variables was examined using standard normality diagnostics. The results indicated that the data were approximately normally distributed, allowing the use of parametric statistical tests.
4. Experimental Setup
4.1. Setting up the Experiment
The purpose of the experiment is to empirically verify the validity, stability, and diagnostic validity of the developed model of critical thinking of students, based on process indicators of educational activity in an information and educational environment.
The experiment was conducted on a sample of 500 students from higher education institutions studying disciplines in a digital information and educational environment. The study included students enrolled in educational programs in the fields of information technologies and management. The total sample size provided sufficient statistical power for factor analysis and model reliability assessment. All participants in the experiment worked in a unified information and educational environment with identical functional capabilities and educational content structure.
The experiment had a quasi-experimental longitudinal design and included three consecutive stages: initial assessment, main training stage, and final assessment. At the initial assessment stage, the initial level of knowledge of the learning elements was recorded, as well as the initial behavioral characteristics of the students’ work in the system. The main stage of the experiment involved independent and adaptive mastery of a set of learning elements of varying degrees of complexity, with the system automatically recording the process indicators of learning activity. At the final stage, the achieved level of knowledge was recorded and all observed indicators were aggregated to calculate the aggregated critical thinking score.
As part of the experiment, students were offered learning elements that varied in complexity, structure, and the depth of analysis required. The tasks included elements of problem-based learning that required a meaningful choice of solution strategy, error analysis, and revisiting the material. At the same time, students were not given a rigid algorithm of actions, which made it possible to identify individual learning strategies and the characteristics of the user’s information style.
Table 4 presents the structure of the learning tasks used in the experiment, indicating their type, cognitive level, and a brief description of their content. The tasks encompass analytical, problem-solving, research, and reflective forms of activity and are aimed at identifying students’ thinking strategies under conditions of uncertainty and choice. Differentiation by type and level of cognitive complexity allows for the assessment of not only outcome indicators but also process characteristics of learning activities, enabling their use in calculating the integrated critical thinking indicator.
An analysis of the task structure suggests that the experimental module is purposefully oriented toward the development and assessment of higher cognitive functions. Most tasks relate to the levels of analysis, synthesis, evaluation, and reflection, which correspond to the upper levels of cognitive taxonomy and go beyond the reproductive retrieval of knowledge. This suggests that the learning model is focused not on testing factual knowledge, but on identifying the depth of information processing, the ability to reason, make decisions, and work under uncertainty.
The structural diversity of the tasks ensures the capture of various aspects of mental activity: strategic choice, error analysis, hypothesis formation, critical evaluation of sources, and reflective revision of decisions. This distribution of task types enhances the validity of the measurement, since the integral indicator is formed based on a set of multidirectional cognitive manifestations, rather than a single dominant parameter. Furthermore, the inclusion of tasks with uncertainty and alternative solutions creates conditions for the development of individual learning strategies, allowing for the analysis of not only the level of achievement but also the characteristics of the process dynamics. The table illustrates the systematic nature of the experimental design and confirms that critical thinking is measured through a multidimensional behavioral-cognitive model, rather than traditional testing.
During the experiment, indicators of the learning process were automatically recorded, including the time taken to master learning elements, the number of learning steps, the intensity and frequency of access to content, the error rate, the degree of doubt in responses, and the parameters of the operating mode in the information and educational environment. In addition, the initial and achieved levels of knowledge were recorded and used for normalization and contextual interpretation of the results.
4.2. Sample Distribution
The total sample size was participants. The control group (CG) included students, and the experimental group (EG) also included students.
The equal group distribution (1:1 ratio) ensures sufficient statistical power for confirmatory factor analysis (CFA) and reliable intergroup comparison.
The demographic characteristics of the sample are presented in
Table 5. No statistically significant differences between the groups were observed at baseline (
), which confirms the comparability of the control and experimental groups before the intervention. Group differences were evaluated using independent samples
t-tests.
The post-test values of the main process indicators are shown in
Table 6. The experimental group demonstrates higher values for several indicators reflecting metacognitive activity and learning engagement.
The dynamics of the latent components of critical thinking are summarized in
Table 7. The most significant increase is observed in the experimental group in the metacognitive (
M) and analytical (
A) components.
The overall dynamics of critical thinking development are illustrated in
Figure 4.
The aggregated critical thinking score (
) before and after the experiment is presented in
Table 8. The increase in the experimental group significantly exceeds the natural dynamics observed in the control group.
The reliability of the measurement scales is shown in
Table 9. All Cronbach’s
coefficients exceed the recommended threshold of
, indicating good internal consistency of the proposed model.
Confirmatory factor analysis.
The confirmatory factor analysis (CFA) was conducted to evaluate the construct validity of the proposed three-component model of critical thinking, including metacognitive awareness, analytical depth, and strategic learning activity. The analysis was performed using the maximum likelihood estimation method.
Model fit was assessed using standard fit indices, including the Comparative Fit Index (CFI), Tucker–Lewis Index (TLI), Root Mean Square Error of Approximation (RMSEA), Standardized Root Mean Square Residual (SRMR), and Chi-square statistics.
The results of the confirmatory factor analysis are presented in
Table 10, where all standardized factor loadings exceed 0.60, supporting the adequacy of the measurement model.
Finally, the overall quality indices of the CFA model are shown in
Table 11. The obtained fit indices indicate a good model fit and confirm the stability of the model for the sample size
.
The obtained values indicate an acceptable model fit (CFI = 0.94, TLI = 0.92, RMSEA = 0.049, SRMR = 0.043), confirming the adequacy of the proposed three-factor structure.
The three-factor structure was defined based on the theoretical framework of the study and reflects the conceptual decomposition of critical thinking into its core components. While alternative model structures were not explicitly tested in this study, this is considered a limitation and a direction for future research.
A structural representation of the proposed model is shown in
Figure 5.
4.3. Experimental Results
Analysis of the input data showed no statistically significant differences between the groups in terms of initial knowledge level (
in the control group and
in the experimental group,
), which allows subsequent changes in the indicators to be attributed to the experimental intervention. The baseline characteristics of the sample are presented in
Table 5.
A comparative analysis of the process indicators of learning activities at the final stage of the experiment revealed significant differences between the groups (
Table 6). In the experimental group, the average degree of doubt in their own knowledge was
, while in the control group this indicator did not exceed
. At the same time, the error rate in the experimental group decreased to
compared to
in the control group. The increase in the time spent on mastering the learning elements (
vs.
) and the increase in the number of learning steps (
vs.
) indicate a more in-depth and analytical processing of the learning material by the students in the experimental group. The information interaction signature indicator was also significantly higher in the experimental group (
vs.
), which indicates the formation of stable individual learning strategies.
Analysis of the latent components of critical thinking demonstrated fundamentally different dynamics in the control and experimental groups (
Table 7). In the control group, the values of the metacognitive component increased from 0.43 to 0.47, the analytical component from 0.45 to 0.49, and the strategic component from 0.47 to 0.50, which corresponds to the natural effect of learning. In the experimental group, the increase was much more pronounced: the metacognitive component increased from 0.44 to 0.64 (
), the analytical component from 0.45 to 0.67 (
), and the strategic component increased from 0.46 to 0.58 (
). The greatest increase was observed in the metacognitive and analytical components, which confirms the effectiveness of the experimental intervention focused on the development of reflection and analytical thinking.
The dynamics of the critical thinking indicators are illustrated in
Figure 4.
The aggregated critical thinking score also showed significant intergroup differences (
Table 8). In the control group, the average value of the indicator increased from
to
(
), while in the experimental group, the increase was from
to
(
). Thus, the increase in the aggregated critical thinking score in the experimental group was substantially higher than in the control group, with no comparable difference in the level of final academic performance.
The reliability and validity of the model were further confirmed by statistical indicators (
Table 9). Cronbach’s
coefficients were 0.84 for the metacognitive component, 0.81 for the analytical component, 0.77 for the strategic component, and 0.88 for the model as a whole, indicating high internal consistency of the indicators.
The results of confirmatory factor analysis (
Table 10) demonstrated adequate model fit. The goodness-of-fit indices also confirm the stability of the model (
,
,
,
,
) as shown in
Table 11.
Taken together, the quantitative results indicate that the developed model of critical thinking is a sensitive and robust tool for analyzing cognitive and metacognitive changes in learners. The model allows to identify qualitative differences in learning strategies and depth of thinking that cannot be reduced to knowledge indicators, confirming its applicability for monitoring the development of critical thinking and evaluating the effectiveness of pedagogical and technological interventions in the digital educational environment.
5. Discussion
This paper proposes and empirically substantiates a multi-criteria model for identifying and assessing critical thinking in students in adaptive learning environments based on individual process indicators of learning activity. Unlike traditional approaches that reduce critical thinking to test results or the level of knowledge acquisition, the developed model considers it as a latent multidimensional construct formed in the process of interaction between the student and the information and educational environment.
The key scientific result of the study is the formalization of critical thinking through the integration of three interrelated components: metacognitive awareness, analytical depth, and strategic learning activities. The proposed hierarchical model allows the model to account for not only the cognitive results of learning, but also behavioral, temporal, and metacognitive characteristics, including the level of doubt, the user’s information style, and the features of the system’s operating modes. This provides a more precise and interpretable assessment of qualitative changes in thinking that cannot be reduced to performance indicators.
The obtained results are consistent with contemporary research emphasizing the multidimensional nature of critical thinking in digital learning environments. Previous studies have demonstrated that critical thinking cannot be reliably measured using knowledge-based indicators alone, as it also includes metacognitive regulation, reflective analysis, and strategic behavior in problem solving. In this context, the proposed model expands existing approaches by integrating process indicators of learning activity with latent cognitive characteristics. Unlike traditional assessment frameworks based primarily on test performance, the presented model captures dynamic behavioral patterns of interaction with the educational environment, including indicators of uncertainty, learning strategies, and temporal learning parameters. This approach makes it possible to detect qualitative transformations in thinking processes that remain invisible within conventional assessment models.
The results of experimental verification on a sample of 500 students confirmed the stability, reliability, and diagnostic validity of the model. The experimental group demonstrated a statistically significant increase in all latent components of critical thinking, and the integral indicator increased substantially compared to the natural dynamics observed in the control group. Internal consistency indicators (Cronbach’s ) and the results of confirmatory factor analysis confirm the adequacy of the three-component structure of the model and its high construct validity. The obtained CFA fit indices confirm the adequacy of the model for use in real digital learning conditions.
The practical significance of the study lies in the possibility of integrating the proposed model into intelligent and adaptive educational systems for monitoring the development of critical thinking in dynamics. The model can be used as a tool to support management decisions in education, set adaptive trajectories, evaluate the effectiveness of pedagogical interventions, and design personalized educational content. Of particular value is the ability to detect early changes in the metacognitive and strategic characteristics of learners, which expands the potential of formative assessment.
The limitations of the study include the use of a sample of students mainly from the information technology and management disciplines, which may narrow the possibilities of generalizing the results to other subject areas.
Another limitation is the absence of external criterion validation using established critical thinking assessment instruments, such as the Watson–Glaser Critical Thinking Appraisal or the Cornell Critical Thinking Test. While the proposed model suggests satisfactory internal consistency and construct validity, it has not yet been directly compared with standardized measures of critical thinking.
It should be noted, however, that the proposed approach differs conceptually from traditional assessment tools. Conventional instruments typically evaluate critical thinking as a static outcome in controlled testing conditions, whereas the present model captures it as a dynamic, process-based construct manifested through behavioral, metacognitive, and interactional indicators in real-time learning environments.
Further research should consider interdisciplinary testing of the model, expanding the set of latent indicators, as well as conducting convergent validation studies that combine the proposed model with established psychometric instruments in order to strengthen the empirical grounding and interpretability of the results.
In addition, validation of the “doubt” parameter against established metacognitive assessment instruments is required to confirm its interpretability and further support the robustness of the model. Future work may also focus on studying the long-term effects of critical thinking development in adaptive educational environments using machine learning and intelligent analytics methods.
6. Conclusions and Future Work
This study developed and empirically tested a multi-criteria hierarchical model for identifying and quantitatively assessing critical thinking in students in adaptive learning environments. Unlike traditional approaches, which focus primarily on test results and academic performance, the proposed model considers critical thinking as a latent, multidimensional construct that is formed in the process of cognitive, metacognitive, and behavioral interaction between the learner and the digital information and educational environment.
The key scientific result of the work is the formalization of critical thinking through the integration of three interrelated components: metacognitive awareness, analytical depth, and strategic learning activities. The model is based on a set of individual process indicators, including the level of doubt, knowledge dynamics, time characteristics, intensity and frequency of work with content, error rate, parameters of the user’s information style, and system operating modes. This combination of psychometric and behavioral indicators enables the transition from static knowledge assessment to process-based diagnosis of qualitative changes in thinking.
Empirical verification of the model on a sample of 500 students demonstrated statistical stability, high internal consistency, and satisfactory fit within the framework of confirmatory factor analysis. The results indicate that the aggregated critical thinking score is sensitive to metacognitive and analytical changes, while not being reduced to indicators of academic performance. The experimental group showed a significant increase in all latent components of the model, supporting the diagnostic validity of the proposed tool in an adaptive educational environment.
The practical significance of the study lies in the possibility of integrating the developed model into intelligent learning systems for monitoring the development of critical thinking over time. The model can be used as a tool to support management decisions, design personalized educational trajectories, and evaluate the effectiveness of pedagogical and technological interventions. Of particular value is the ability to identify latent cognitive characteristics of learners based on the analysis of their behavioral patterns in a digital environment.
The limitations of the study include the fact that the sample consisted mainly of students with an information technology and management disciplines, which may limit the transferability of the results to other subject areas. In addition, the model was tested within a specific information and educational environment, which requires further verification of its universality when integrated into alternative platform solutions.
Future research should focus on testing the model in other subject domains, expanding the set of latent indicators, and integrating machine learning methods for more advanced analysis of learning behavior and the development of critical thinking in adaptive educational environments. In addition, future work may include comparison with alternative factor structures (e.g., single-factor or two-factor models) to further validate the robustness of the proposed model.