AI-Based Online Education Systems Integrating Real-Time Affective Computing: A Design-Oriented Conceptual Framework
Abstract
1. Introduction
2. Theoretical Background and Related Work
2.1. Learning Analytics in Online Education
2.2. Affective Computing and Emotion Modeling
2.3. AI-Driven Adaptive Learning Systems
2.4. Gaps and Research Opportunities
2.5. Contribution of Research
3. Research Design and Conceptual Framework Development
3.1. Research Design and Paradigm
3.2. Framework Development Process
3.3. Conceptual Framework Description
3.4. Conceptual Illustration of Framework Operation
Conceptual Illustration of Data Flow
3.5. Conceptual and Analytical Assessment
3.6. Implementation and Ethical Considerations
4. Proposed Framework and System Architecture
4.1. Overview of the Framework
4.2. Framework Conceptual Layers
4.2.1. Data Acquisition Layer
- Behavioral interaction logs (click-streams, time on task).
- Textual input (forum discussions, written assignments).
- Optional audiovisual streams (via webcam and microphone).
4.2.2. Preprocessing and Feature Engineering Layer
- Data cleaning and normalization;
- Temporal alignment;
- Feature extraction.
4.2.3. Machine Learning Layer and Affective Computing
4.2.4. Learning Analytics and Adaptation Layer
- Content sequence;
- Timing of feedback;
- Adjusting difficulty;
- Alerts and warnings.
4.2.5. The Presentation and Feedback Layer
- Learner-facing feedback;
- Instructor dashboards;
- Institutional-level aggregated analytics.
4.3. Characteristics and Evaluation of the Framework
4.4. Contribution of the Framework
- It integrates affective computing and learning analytics in a unified architecture;
- It enables real-time affect-aware adaptation in online learning systems;
- It provides a reusable reference model for future empirical and system-oriented research.
5. Discussion and Implications
5.1. Theoretical Contributions
5.2. Practical Implications
5.3. Limitations
5.4. Future Research Directions
5.5. Conclusions of Discussion
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Anwar, Aamir, Ikram Ur Rehman, Moustafa M. Nasralla, Sohaib Bin Altaf Khattak, and Nasrullah Khilji. 2023. Emotions Matter: A Systematic Review and Meta-Analysis of the Detection and Classification of Students’ Emotions in STEM during Online Learning. Education Sciences 13: 914. [Google Scholar] [CrossRef]
- Azevedo, Roger, and Dragan Gašević. 2019. Analyzing Multimodal Multichannel Data About Self-Regulated Learning with Advanced Learning Technologies: Issues and Challenges. Computers in Human Behavior 96: 207–10. [Google Scholar] [CrossRef]
- Bosch, Nigel, Sidney K. D’Mello, Ryan S. Baker, Jaclyn Ocumpaugh, Valerie Shute, Matthew Ventura, Lubin Wang, and Weinan Zhao. 2016. Detecting Student Emotions in Computer-Enabled Classrooms. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI. New York: ACM, pp. 4125–29. [Google Scholar]
- Cîrneanu, Andrada-Livia, Dan Popescu, and Dragoș Iordache. 2023. New Trends in Emotion Recognition Using Image Analysis by Neural Networks, A Systematic Review. Sensors 23: 7092. [Google Scholar] [CrossRef] [PubMed]
- D’Mello, Sidney K., and Arthur C. Graesser. 2012. Dynamics of Affective States During Complex Learning. Learning and Instruction 22: 145–57. [Google Scholar] [CrossRef]
- D’Mello, Sidney K., Blair Lehman, Reinhard Pekrun, and Arthur C. Graesser. 2014. Confusion Can Be Beneficial for Learning. Learning and Instruction 29: 153–70. [Google Scholar] [CrossRef]
- Fan, Yizhou, Luzhen Tang, Le Huixiao, Shen Kejie, Shufang Tan, Yueying Zhao, Yuan Shen, Xinyu Li, and Dragan Gašević. 2024. Beware of Metacognitive Laziness: Effects of Generative Artificial Intelligence on Learning Motivation, Processes, and Performance. British Journal of Educational Technology 56: 489–530. [Google Scholar]
- Feldman-Maggor, Yael, Mutlu Cukurova, Carmel Kent, and Giora Alexandron. 2025. The Impact of Explainable AI on Teachers’ Trust and Acceptance of AI-EdTech Recommendations. International Journal of Artificial Intelligence in Education 35: 2889–922. [Google Scholar] [CrossRef]
- Fernández-Herrero, Jorge. 2024. Evaluating Recent Advances in Affective Intelligent Tutoring Systems: A Scoping Review of Educational Impacts and Future Prospects. Education Sciences 14: 839. [Google Scholar] [CrossRef]
- Gašević, Dragan, George Siemens, and Shazia Sadiq. 2023. Empowering Learners for the Age of Artificial Intelligence. Computers and Education: Artificial Intelligence 4: 100130. [Google Scholar] [CrossRef]
- Gašević, Dragan, Shane Dawson, and George Siemens. 2015. Let’s Not Forget: Learning Analytics Are About Learning. TechTrends 59: 64–71. [Google Scholar]
- Gašević, Dragan, Yi-Shan Tsai, Shane Dawson, and Abelardo Pardo. 2019. How Do We Start? An Approach to Learning Analytics Adoption in Higher Education. International Journal of Information and Learning Technology 36: 100722. [Google Scholar] [CrossRef]
- Geetha, A. V., T. Mala, D. Priyanka, and E. Uma. 2024. Multimodal Emotion Recognition with Deep Learning: Advancements, Challenges, and Future Directions. Information Fusion 105: 102218. [Google Scholar] [CrossRef]
- Giannakos, Michail, Roger Azevedo, Peter Brusilovsky, Mutlu Cukurova, Yannis Dimitriadis, Hernández-Leo Davinia, Sanna Järvelä, Manolis Mavrikis, and Bart Rienties. 2024. The Promise and Challenges of Generative AI in Education. Behaviour and Information Technology 44: 1–27. [Google Scholar] [CrossRef]
- Henrie, Curtis, Lisa Halverson, and Charles Graham. 2015. Measuring Student Engagement in Technology-Mediated Learning: A Review. Computers and Education 90: 36–53. [Google Scholar] [CrossRef]
- Ifenthaler, Dirk, and Jane Yau. 2020. Utilising Learning Analytics to Support Study Success in Higher Education: A Systematic Review. Educational Technology Research and Development 68: 1961–90. [Google Scholar] [CrossRef]
- Jaakkola, Elina. 2020. Designing Conceptual Articles: Four Approaches. AMS Review 10: 18–26. [Google Scholar] [CrossRef]
- Järvelä, Sanna, Andy Nguyen, and Allyson Hadwin. 2023. Human and Artificial Intelligence Collaboration for Socially Shared Regulation in Learning. British Journal of Educational Technology 54: 1057–76. [Google Scholar] [CrossRef]
- Khalil, Mohammad, Paul Prinsloo, and Sharon Slade. 2023. The Use and Application of Learning Theory in Learning Analytics: A Scoping Review. Journal of Computing in Higher Education 35: 573–94. [Google Scholar]
- Kizilcec, René F., Andrew J. Saltarelli, Justin Reich, and Geoffrey L. Cohen. 2017. Closing Global Achievement Gaps in MOOCs. Science 355: 251–52. [Google Scholar]
- Kizilcec, René F., Chris Piech, and Emily Schneider. 2013. Deconstructing Disengagement: Analyzing Learner Subpopulations in Massive Open Online Courses. In Proceedings of the Third International Conference on Learning Analytics and Knowledge 2013. New York: ACM, pp. 170–79. [Google Scholar]
- Kollias, Dimitrios, Panagiotis Tzirakis, Mihalis A. Nicolaou, Athanasios Papaioannou, Guoying Zhao, Björn Schuller, Irene Kotsia, and Stefanos Zafeiriou. 2019. Deep Affect Prediction in-the-Wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyond. International Journal of Computer Vision 127: 907–29. [Google Scholar] [CrossRef]
- Lek, Jeniffer Xin-Ying, and Jason Teo. 2023. Academic Emotion Classification Using FER: A Systematic Review. Human Behavior and Emerging Technologies 2023: 9790005. [Google Scholar] [CrossRef]
- Li, Shan, and Weihong Deng. 2022. Deep Facial Expression Recognition: A Survey. IEEE Transactions on Affective Computing 13: 11959–1215. [Google Scholar] [CrossRef]
- Liu, Hsin-Lan, Tao-Hua Wang, Hao-Chiang Koong Lin, Chin-Feng Lai, and Yueh-Min Huang. 2022. The Influence of Affective Feedback Adaptive Learning System on Learning Engagement and Self-Directed Learning. Frontiers in Psychology 13: 858411. [Google Scholar] [CrossRef] [PubMed]
- Matcha, Wannisa, Dragan Gašević, Nora’ayu Ahmad Uzir, Jelena Jovanović, Abelardo Pardo, Lisa Lim, Jorge Maldonado-Mahauad, Mar Pérez-Sanagustín, and Yi-Shan Tsai. 2020. Analytics of Learning Strategies: Role of Course Design and Delivery Modality. Journal of Learning Analytics 7: 45–71. [Google Scholar] [CrossRef]
- Molenaar, Inge. 2022. Towards Hybrid Human–AI Learning Technologies. European Journal of Education 57: 632–45. [Google Scholar] [CrossRef]
- Ng, Davy Tsz Kit, Chee Wei Tan, and Jac Ka Lok Leung. 2024. Empowering Student Self-Regulated Learning and Science Education Through ChatGPT. British Journal of Educational Technology 55: 1328–53. [Google Scholar] [CrossRef]
- Ouyang, Fan, and Pengcheng Jiao. 2021. Artificial Intelligence in Education: The Three Paradigms. Computers and Education: Artificial Intelligence 2: 100020. [Google Scholar] [CrossRef]
- Pei, Guanxiong, Haiying Li, Yandi Lu, Yanlei Wang, Shizhen Hua, and Taihao Li. 2024. Affective Computing: Recent Advances, Challenges, and Future Trends. Intelligent Computing 3: 0076. [Google Scholar] [CrossRef]
- Pekrun, Reinhard. 2024. Control-Value Theory: From Achievement Emotion to a General Theory of Human Emotions. Educational Psychology Review 36: 83. [Google Scholar] [CrossRef]
- Shingjergji, Krist, Deniz Iren, Corrie Urlings, and Roland Klemke. 2025. Affective Computing in Online Higher Education: A Systematic Literature Review. Computers and Education: Artificial Intelligence 10: 100499. [Google Scholar] [CrossRef]
- Wang, Shan, Fang Wang, Zhen Zhu, Jingxuan Wang, Tam Tran, and Zhao Du. 2024. Artificial Intelligence in Education: A Systematic Literature Review. Expert Systems with Applications 252: 124167. [Google Scholar] [CrossRef]
- Xia, Qi, Xiaojing Weng, Fan Ouyang, Tzung-Jin Lin, and Thomas K. Chiu. 2024. A Scoping Review on How Generative Artificial Intelligence Transforms Assessment in Higher Education. International Journal of Educational Technology in Higher Education 21: 40. [Google Scholar] [CrossRef]
- Zawacki-Richter, Olaf, Victoria I. Marín, Melissa Bond, and Franziska Gouverneur. 2019. Systematic Review of Research on Artificial Intelligence Applications in Higher Education—Where Are the Educators? International Journal of Educational Technology in Higher Education 16: 39. [Google Scholar] [CrossRef]


| Dataset Type | Illustrative Purpose | Framework Layer | Expected Output |
|---|---|---|---|
| FER-2013/AffectNet | Shows affect recognition input | Machine learning/affective computing | Predicted affective states |
| Learning analytics dataset | Shows learner behavior input | Analytics and adaptation | Integrated learner profile |
| Conceptual multimodal flow | Shows end-to-end conceptual data flow | Full framework | Adaptive pedagogical response |
| Framework Layer | Example Input | Processing | Output |
|---|---|---|---|
| Data Acquisition | FER-2013 image | Capture facial data | Raw affective data |
| Preprocessing | FER-2013 image | Normalization and feature extraction | Feature vector |
| ML and Affective Computing | Feature vector | Affective classification | Predicted affective states |
| Learning Analytics | Clickstream data | Behavioral analysis | Engagement profile |
| Adaptation | Combined profile | Decision logic | Adaptive feedback |
| Feedback | Adaptive strategy | Presentation | Learner feedback |
| Framework Layer | Function | Data Input | Output |
|---|---|---|---|
| Data Acquisition | Collect multimodal learner interactions | Webcam, microphone, logs, text | Structured raw data |
| Preprocessing and Feature Engineering | Clean, normalize, and extract features | Raw multimodal streams | Feature vectors |
| Affective Computing and ML | Infer affective states | Feature vectors | Predicted affective states |
| Learning Analytics and Adaptation | Analyze data, generate interventions | Predicted states + performance metrics | Adaptive strategies |
| Presentation and Feedback | Deliver adaptive responses | Adaptive strategies | Dashboards, content adjustment |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.
Share and Cite
Jaffri, S.U.; Koo, A.-C.; Hussain, S.; Ting, C.-Y. AI-Based Online Education Systems Integrating Real-Time Affective Computing: A Design-Oriented Conceptual Framework. Soc. Sci. 2026, 15, 421. https://doi.org/10.3390/socsci15070421
Jaffri SU, Koo A-C, Hussain S, Ting C-Y. AI-Based Online Education Systems Integrating Real-Time Affective Computing: A Design-Oriented Conceptual Framework. Social Sciences. 2026; 15(7):421. https://doi.org/10.3390/socsci15070421
Chicago/Turabian StyleJaffri, Syed Uzair, Ah-Choo Koo, Salman Hussain, and Choo-Yee Ting. 2026. "AI-Based Online Education Systems Integrating Real-Time Affective Computing: A Design-Oriented Conceptual Framework" Social Sciences 15, no. 7: 421. https://doi.org/10.3390/socsci15070421
APA StyleJaffri, S. U., Koo, A.-C., Hussain, S., & Ting, C.-Y. (2026). AI-Based Online Education Systems Integrating Real-Time Affective Computing: A Design-Oriented Conceptual Framework. Social Sciences, 15(7), 421. https://doi.org/10.3390/socsci15070421

