Search Results (25)

Pursuit–evasion games are a fundamental framework for advancing autonomous decision-making and cooperative control in multi-UAV systems. However, the application of reinforcement learning to pursuit–evasion games involving fixed-wing UAVs remains challenging due to constraints, such as minimum velocity, limited turning radius, and high-dimensional continuous action spaces. To address these issues, this paper proposes a method that integrates automatic curriculum learning with multi-agent proximal policy optimization. A self-play mechanism is introduced to simultaneously train both pursuers and evaders, enabling dynamic and adaptive encirclement strategies. In addition, a reward structure specifically tailored for the encirclement task was designed to guide the pursuers in gradually achieving the encirclement of the evader while ensuring their own safety. To further improve training efficiency and convergence, this paper develops a subgame curriculum learning framework that progressively exposes agents to increasingly complex scenarios, facilitating experience accumulation and skill transfer. The simulation results demonstrate that the proposed approach improves learning efficiency and cooperative pursuit performance under realistic fixed-wing UAV dynamics. This work provides a practical and scalable solution for multiple fixed-wing UAV pursuit–evasion missions in complex environments. Full article

This study proposes an unmanned combat aerial vehicle (UCAV)-oriented hierarchical reinforcement learning framework to address the temporal abstraction challenge in autonomous within-visual-range air combat (WVRAC) for UCAVs. The incorporation of maximum-entropy objectives within the MEOL framework facilitates the optimization of both autonomous low-level tactical discovery and high-level option selection. At the low level, three tactical policies (angle, snapshot, and energy tactics) are designed with reward functions informed by expert knowledge, while the high-level policy dynamically terminates current tactics and selects new ones through sparse reward learning, thus overcoming the limitations of fixed-duration tactical execution. Furthermore, a novel automatic curriculum generation mechanism based on Wasserstein Generative Adversarial Networks (WGANs) is introduced to enhance training efficiency and adaptability to diverse initial combat conditions. Extensive experiments conducted in UCAV air combat simulations have shown that MEOL not only achieves significantly better win rates than other policies when training against rule-based opponents, but also that MEOC achieves superior results in tests against tactical intra-option policies as well as other option learning policies. The framework facilitates dynamic termination and switching of tactics, thereby addressing the limitations of fixed-duration hierarchical methods. Ablation studies confirm the effectiveness of WGAN-based curricula in accelerating policy convergence. Additionally, the visual analysis of UCAVs’ flight logs validates the learned hierarchical decision-making process, showcasing the interplay between tactical selection and manoeuvring execution. This research provides novel methodologies combining hierarchical reinforcement learning with tactical domain knowledge for the autonomous decision-making of UCAVs in complex air combat scenarios. Full article

Advancements in modern technology have significantly increased the availability of educational data, presenting researchers with new challenges in extracting meaningful insights. Educational Data Mining offers analytical methods to support the prediction of student outcomes, development of intelligent tutoring systems, and curriculum optimization. Prior studies have highlighted the potential of semi-supervised approaches that incorporate feature selection to identify factors influencing academic success, particularly for improving model interpretability and predictive performance. Many feature selection methods tend to exclude variables that may not be individually powerful predictors but can collectively provide significant information, thereby constraining a model’s capabilities in learning environments. In contrast, Deep Learning (DL) models paired with Automated Machine Learning techniques can decrease the reliance on manual feature engineering, thereby enabling automatic fine-tuning of numerous model configurations. In this study, we propose a reproducible methodology that integrates DL with AutoML to evaluate student performance. We compared the proposed DL methodology to a semi-supervised approach originally introduced by Yu et al. under the same evaluation criteria. Our results indicate that DL-based models can provide a flexible, data-driven approach for examining student outcomes, in addition to preserving the importance of feature selection for interpretability. This proposal is available for replication and additional research. Full article

The field of autonomous air combat has witnessed a surge in interest propelled by the rapid progress of artificial intelligence technology. A persistent challenge within this domain pertains to autonomous decision-making for dogfighting, especially when dealing with intricate, high-fidelity nonlinear aircraft dynamic models and insufficient information. In response to this challenge, this paper introduces reinforcement learning (RL) to train maneuvering strategies. In the context of RL for dogfighting, the method by which opponents are sampled assumes significance in determining the efficacy of training. Consequently, this paper proposes a novel automatic opponent sampling (AOS)-based RL framework where proximal policy optimization (PPO) is applied. This approach encompasses three pivotal components: a phased opponent policy pool with simulated annealing (SA)-inspired curriculum learning, an SA-inspired Boltzmann Meta-Solver, and a Gate Function based on the sliding window. The training outcomes demonstrate that this improved PPO algorithm with an AOS framework outperforms existing reinforcement learning methods such as the soft actor–critic (SAC) algorithm and the PPO algorithm with prioritized fictitious self-play (PFSP). Moreover, during testing scenarios, the trained maneuvering policy displays remarkable adaptability when confronted with a diverse array of opponents. This research signifies a substantial stride towards the realization of robust autonomous maneuvering decision systems in the context of modern air combat. Full article

Arctic sea ice has a significant effect on global climate change, ship navigation, Arctic ecosystems, and human activities. Therefore, it is essential to produce high-resolution sea ice maps that accurately represent the geographical distribution of various sea ice types. Based on deep learning technology, many automatic sea ice classification algorithms have been developed using synthetic aperture radar (SAR) imagery over the last decade. However, sea ice classification faces two vital challenges: (1) it is difficult to distinguish sea ice types with close developmental stages solely from SAR images and (2) an imbalanced sea ice dataset has a significantly negative effect on ice classification model performance. In this article, a spatio-temporal deep learning model—the Dynamic Multi-Layer Perceptron (MLP)—is utilized to classify 10 sea ice types automatically. It consists of a SAR image branch and a spatio-temporal branch, which extracts SAR image features and spatio-temporal distribution characteristics of sea ice, respectively. By projecting similar image features to different positions in the spatio-temporal feature space dynamically, the Dynamic MLP model effectively distinguishes between similar sea ice types. Furthermore, to reduce the impact of data imbalance on model performance, the dynamic curriculum learning (DCL) method is used to train the Dynamic MLP model. Experimental results demonstrate that our proposed method outperforms the long short-term memory (LSTM) network approach in distinguishing between sea ice types with similar developmental stages. Moreover, the DCL training method can also effectively improve model performance in identifying minority ice types. Full article

Curriculum learning is a method of prioritizing learning data to improve learning performance. In this paper, we propose a new algorithm that determines how to select learning data and when to start and stop curriculum learning by considering learning errors. We use entropy to select data samples with less consistent predictions and automatically determine the warming-up period based on the characteristics of the data. Additionally, to mitigate learning bias, we introduced a variable that adjusts the range of sample selection according to the progress of the training. To validate our method, we conducted extensive experiments on both balanced and imbalanced data classification tasks, and our proposed approach showed an average improvement of about 1.8%, with a maximum improvement of up to 4.4%, compared to previously suggested methods. Full article

The level of difficulty of mathematical test items is a critical aspect for evaluating test quality and educational outcomes. Accurately predicting item difficulty during test creation is thus significantly important for producing effective test papers. This study used more than ten years of content and score data from China’s Henan Provincial College Entrance Examination in Mathematics as an evaluation criterion for test difficulty, and all data were obtained from the Henan Provincial Department of Education. Based on the framework established by the National Center for Education Statistics (NCES) for test item assessment methodology, this paper proposes a new framework containing eight features considering the uniqueness of mathematics. Next, this paper proposes an XGBoost-based SHAP model for analyzing the difficulty of mathematics tests. By coupling the XGBoost method with the SHAP method, the model not only evaluates the difficulty of mathematics tests but also analyzes the contribution of specific features to item difficulty, thereby increasing transparency and mitigating the “black box” nature of machine learning models. The model has a high prediction accuracy of 0.99 for the training set and 0.806 for the test set. With the model, we found that parameter-level features and reasoning-level features are significant factors influencing the difficulty of subjective items in the exam. In addition, we divided senior secondary mathematics knowledge into nine units based on Chinese curriculum standards and found significant differences in the distribution of the eight features across these different knowledge units, which can help teachers place different emphasis on different units during the teaching process. In summary, our proposed approach significantly improves the accuracy of item difficulty prediction, which is crucial for intelligent educational applications such as knowledge tracking, automatic test item generation, and intelligent paper generation. These results provide tools that are better aligned with and responsive to students’ learning needs, thus effectively informing educational practice. Full article

This study endeavors to automate the assessment of competencies within the domain of entrepreneurship, specifically targeting the augmentation of entrepreneurial cognition and conduct within universities in German rural regions, like Lower Franconia. Employing methods, including literature analyses and expert interviews, we formulated and validated an entrepreneurship competence profile and accompanying self-assessment tool. The ensuing evaluative framework is poised for seamless integration into learning management systems, thereby facilitating intelligent competence monitoring within educational environments. Purpose: The aim of this thesis is to develop an automated competence assessment procedure in the field of entrepreneurship. This can be used in the university environment in the long term to promote and teach entrepreneurial thinking and behavior in order to sustainably improve the quality of learning outcomes and achieve targeted promotion of entrepreneurship. Methodology: Based on a relevant literature analysis, four guideline-based expert interviews were created and conducted. The results of the interviews were compiled and validated in a structured competence profile (entrepreneurship competence profile). Based on this competence catalog for entrepreneurs, an empirically valid self-test was created using standard psychometric questionnaire construction methods. Results: The entrepreneurship competence profile and a consequential empirically validated self-test for competence assessment were created. This test provides the basis for the long-term competence development of students and can further be embedded automatically into a learning management system (LMS) as part of intelligent competence monitoring, which allows for the recording of competencies for each student and the individual incorporation of gap closure into the curriculum. Originality/value: In previous research, there were no competence profiles or competence assessment procedures in the field of entrepreneurship that derived relevant competencies directly from actors within this environment. This work illustrates the development of a competence assessment procedure for entrepreneurs and shows which methods can be used to close prevailing research gaps in the field of intelligent competence monitoring. Full article

Collaborative autonomous landing of a quadrotor Unmanned Aerial Vehicle (UAV) on a moving Unmanned Ground Vehicle (UGV) presents challenges due to the need for accurate real-time tracking of the UGV and the adjustment for the landing policy. To address this challenge, we propose a progressive learning framework for generating an optimal landing policy based on vision without the need of communication between the UAV and the UGV. First, we propose the Landing Vision System (LVS) to offer rapid localization and pose estimation of the UGV. Then, we design an Automatic Curriculum Learning (ACL) approach to learn the landing tasks under different conditions of UGV motions and wind interference. Specifically, we introduce a neural network-based difficulty discriminator to schedule the landing tasks according to their levels of difficulty. Our method achieves a higher landing success rate and accuracy compared with the state-of-the-art TD3 reinforcement learning algorithm. Full article

Maneuver decision-making is essential for autonomous air combat. However, previous methods usually make decisions to aim at the target instead of hitting the target and use discrete action spaces instead of continuous action spaces. While these simplifications make maneuver decision-making easier, they also make maneuver decision-making more unrealistic. Meanwhile, previous studies usually rely on handcrafted reward functions, which are troublesome to design. Therefore, to solve these problems, we propose an automatic curriculum reinforcement learning method that enables agents to maneuver effectively in air combat from scratch. On the basis of curriculum reinforcement learning, maneuver decision-making is divided into a series of sub-tasks from easy to difficult. Thus, agents can gradually learn how to complete a series of sub-tasks, from easy to difficult without handcrafted reward functions. The ablation studies show that automatic curriculum learning is essential for reinforcement learning; namely, agents cannot make effective decisions without curriculum learning. Simulations show that, after training, agents are able to make effective decisions given different states, including tracking, attacking, and escaping, which are both rational and interpretable. Full article

Arguing and communicating are basic skills in the mathematics curriculum. Making arguments in written form facilitates rigorous reasoning. It allows peers to review arguments, and to receive feedback about them. Even though it requires additional cognitive effort in the calculation process, it enhances long-term retention and facilitates deeper understanding. However, developing these competencies in elementary school classrooms is a great challenge. It requires at least two conditions: all students write and all receive immediate feedback. One solution is to use online platforms. However, this is very demanding for the teacher. The teacher must review 30 answers in real time. To facilitate the revision, it is necessary to automatize the detection of incoherent responses. Thus, the teacher can immediately seek to correct them. In this work, we analyzed 14,457 responses to open-ended questions written by 974 fourth graders on the ConectaIdeas online platform. A total of 13% of the answers were incoherent. Using natural language processing and machine learning algorithms, we built an automatic classifier. Then, we tested the classifier on an independent set of written responses to different open-ended questions. We found that the classifier achieved an F1-score = 79.15% for incoherent detection, which is better than baselines using different heuristics. Full article

The process of learning about a student’s knowledge and comprehension of a particular subject is referred to as student knowledge assessment. It helps to identify areas where students need additional support or challenge and can be used to evaluate the effectiveness of instruction, make important decisions such as on student placement and curriculum development, and monitor the quality of education. Evaluating student knowledge assessment is essential to measuring student progress, informing instruction, and providing feedback to improve student performance and enhance the overall teaching and learning experience. This research paper is designed to create a machine learning (ML)-based system that assesses student performance and knowledge throughout the course of their studies and pinpoints the key variables that have the most significant effects on that performance and expertise. Additionally, it describes the impact of running models with data that only contains key features on their performance. To classify the students, the paper employs seven different classifiers, including support vector machines (SVM), logistic regression (LR), random forest (RF), decision tree (DT), gradient boosting machine (GBM), Gaussian Naive Bayes (GNB), and multi-layer perceptron (MLP). This paper carries out two experiments to see how best to replicate the automatic classification of student knowledge. In the first experiment, the dataset (Dataset 1) was used in its original state, including all five properties listed in the dataset, to evaluate the performance indicators. In the second experiment, the least correlated variable was removed from the dataset to create a smaller dataset (Dataset 2), and the same set of performance indicators was evaluated. Then, the performance indicators using Dataset 1 and Dataset 2 were compared. The GBM exhibited the highest prediction accuracy of 98%, according to Dataset 1. In terms of prediction error, the GBM also performed well. The accuracy of optimistic forecasts on student performance, denoted as the performance indicator ‘precision’, was highest in GBM at 99%, while DT, RF, and SVM were 98% accurate in their optimistic forecasts for Dataset 1. The second experiment’s findings demonstrated that practically no classifiers showed appreciable improvements in prediction accuracy with a reduced feature set in Dataset 2. It showed that the time required for related learning objects and the knowledge level corresponding to a goal learning object have less impact. Full article

With the continuous development of deep reinforcement learning in intelligent control, combining automatic curriculum learning and deep reinforcement learning can improve the training performance and efficiency of algorithms from easy to difficult. Most existing automatic curriculum learning algorithms perform curriculum ranking through expert experience and a single network, which has the problems of difficult curriculum task ranking and slow convergence speed. In this paper, we propose a curriculum reinforcement learning method based on K-Fold Cross Validation that can estimate the relativity score of task curriculum difficulty. Drawing lessons from the human concept of curriculum learning from easy to difficult, this method divides automatic curriculum learning into a curriculum difficulty assessment stage and a curriculum sorting stage. Through parallel training of the teacher model and cross-evaluation of task sample difficulty, the method can better sequence curriculum learning tasks. Finally, simulation comparison experiments were carried out in two types of multi-agent experimental environments. The experimental results show that the automatic curriculum learning method based on K-Fold cross-validation can improve the training speed of the MADDPG algorithm, and at the same time has a certain generality for multi-agent deep reinforcement learning algorithm based on the replay buffer mechanism. Full article

The International Classification of Diseases (ICD) has an important role in building applications for clinical medicine. Extremely large ICD coding label sets and imbalanced label distribution bring the problem of inconsistency between the local batch data distribution and the global training data distribution into the minibatch gradient descent (MBGD)-based training procedure for deep multi-label classification models for automatic ICD coding. The problem further leads to an overfitting issue. In order to improve the performance and generalization ability of the deep learning automatic ICD coding model, we proposed a simple and effective curriculum batching strategy in this paper for improving the MBGD-based training procedure. This strategy generates three batch sets offline through applying three predefined sampling algorithms. These batch sets satisfy a uniform data distribution, a shuffling data distribution and the original training data distribution, respectively, and the learning tasks corresponding to these batch sets range from simple to complex. Experiments show that, after replacing the original shuffling algorithm-based batching strategy with the proposed curriculum batching strategy, the performance of the three investigated deep multi-label classification models for automatic ICD coding all have dramatic improvements. At the same time, the models avoid the overfitting issue and all show better ability to learn the long-tailed label information. The performance is also better than a SOTA label set reconstruction model. Full article

Objectives: Accurate self-assessment and -reflection of competencies are crucial skills for all health professions. The National Competence-Based Learning Objectives Catalogue (NKLM) guiding medical faculties in Germany points out reflection as a non-technical skill and competency-based medical education (CBME) as important approaches. In this context, the role and structure of curricula and skills labs evolved. Especially in peer-assisted trainings, reflection of competencies is important to improve self-regulated learning. Traditionally, we assume self-reflection skills to evolve automatically with learners’ experience. This approach aims to find empirical evidence for this assumption and implements self-reflection of competencies in clinical skills education. Here, we quantify the influence of age and semester of study and its interaction on the concordant self-reflection of students’ own competencies. Methods: Investigation was based on a retrospective analysis of evaluation data from peer-assisted “first aid” and “physical examination” courses in the skills labs of the medical faculty at the Ruhr-University Bochum, Germany. Participants were asked for self-assessed competencies before (pre) and after (post) the course. Additionally, they were asked to retrospectively re-rate their “before” competencies after completing the course (post-pre). Differences between pre and post-pre competencies were assessed as the concordant self-reflection in a moderated regression analysis. Group means and standard deviation were depicted using univariate analysis of variance (ANOVA) with post-hoc Tukey HSD testing in IBM SPSS Statistics V.28. Moderated regression and simple slope analyses were conducted to calculate interaction effects of age and semester of study on the concordant self-reflection. Results: As expected, participants (n = 168) showed significant progress in subjective self-assessment (pre vs. post) in all 18 assessed domains in the course (all p < 0.001). Additionally, participants self-assessed their previous competencies after the course (post-pre) differently than before the course (pre) in 11 out of 18 domains. Hereby, the interaction of age and semester of study explained a significant part of variance in the first aid course (∆R² = 0.008, ∆F (1;1020) = 8.53, p < 0.005) and in the physical examination course (ΔR² = 0.03, ΔF (1;10,280) = 10.72, p < 0.001). Conclusions: We quantified that interaction of age and semester has a significant influence on concordant self-reflection skills using a moderated regression analysis. Assumed as an indicator, we conclude that advanced and older students show less differences in pre- vs. post-pre-ratings. This has implications for curriculum development, postulating that an exposure to self-reflection as a metacognitive process should be introduced early in order to train competencies in health professionals. Prospective studies with competency-based assessments are necessary to validate findings. Full article