An Exploratory Study on the Efficacy and Inclusivity of AI Technologies in Diverse Learning Environments
Abstract
:1. Introduction
2. Literature Review
- 1
- How do students perceive the impact of AI technologies on their learning experiences?
- 2
- To what extent do AI technologies address the diverse needs of students from various cultural and linguistic backgrounds?
- 3
- What are some of the challenges in integrating AI in learning from students’ perspectives, and how can AI technologies be optimized to better support individual learning?
3. Materials and Methods
3.1. Participants and Context
3.2. Procedure and Instrument
3.3. Data Analysis
4. Results
4.1. Codes and Emergent Themes
4.2. Inter-Rater Reliability and Agreement
4.3. Theme Patterns and Connections
4.4. Enhancing Learning and Research Capacities
4.4.1. Linguistic and Cultural Inclusivity
4.4.2. Learning Adaptability
4.4.3. Study Aid
4.4.4. Research Assistance
4.4.5. Cognitive Expansion
4.5. Challenges and Risks Associated with AI in Education
4.5.1. Dependency and Reduced Critical Thinking
4.5.2. Inaccuracy and Misinformation Risks
4.5.3. Ethical and Academic Integrity Concerns
4.6. Improving AI: Inclusivity, Specialization, and User Co-Design
4.7. A Model for Enhanced Learning and Ethical Engagement with AI
5. Discussion
6. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Chang, D.H.; Lin, M.P.-C.; Hajian, S.; Wang, Q.Q. Educational design principles of using AI chatbot that supports self-regulated learning in education: Goal setting, feedback, and personalization. Sustainability 2023, 15, 12921. [Google Scholar] [CrossRef]
- Kılınç, S. Embracing the future of distance science education: Opportunities and challenges of ChatGPT integration. Asian J. Distance Educ. 2023, 18, 205–237. [Google Scholar] [CrossRef]
- Lin, M.P.C.; Chang, D. CHAT-ACTS: A pedagogical framework for personalized chatbot to enhance active learning and self-regulated learning. Comput. Educ. Artif. Intell. 2023, 5, 100167. [Google Scholar] [CrossRef]
- Atlas, S. ChatGPT for Higher Education and Professional Development: A Guide to Conversational AI. 2023. Available online: https://digitalcommons.uri.edu/cgi/viewcontent.cgi?article=1547&context=cba_facpubs (accessed on 30 September 2024).
- Baidoo-Anu, D.; Ansah, L.O. Education in the era of generative artificial intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning. J. AI 2023, 7, 52–62. [Google Scholar] [CrossRef]
- Grassini, S. Shaping the future of education: Exploring the potential and consequences of AI and ChatGPT in educational settings. Educ. Sci. 2023, 13, 692. [Google Scholar] [CrossRef]
- Chi, N.; Lurie, E.; Mulligan, D.K. Reconfiguring diversity and inclusion for AI ethics. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, Association for Computing Machinery, Virtual, 19–21 May 2021; pp. 447–457. [Google Scholar] [CrossRef]
- Holmes, W.; Porayska-Pomsta, K.; Holstein, K.; Sutherland, E.; Baker, T.; Shum, S.B.; Santos, O.C.; Rodrigo, M.T.; Cukurova, M.; Bittencourt, I.I.; et al. Ethics of AI in education: Towards a community-wide framework. Int. J. Artif. Intell. Educ. 2022, 32, 504–526. [Google Scholar] [CrossRef]
- Jobin, A.; Ienca, M.; Vayena, E. The global landscape of AI ethics guidelines. Nat. Mach. Intell. 2019, 1, 389–399. [Google Scholar] [CrossRef]
- Memarian, B.; Doleck, T. Fairness, Accountability, Transparency, and Ethics (FATE) in Artificial Intelligence (AI) and higher education: A systematic review. Comput. Educ. Artif. Intell. 2023, 5, 100152. [Google Scholar] [CrossRef]
- Raji, I.D.; Scheuerman, M.K.; Amironesei, R. You can’t sit with us: Exclusionary pedagogy in AI ethics education. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, Association for Computing Machinery, Virtual, 3–10 March 2021; pp. 515–525. [Google Scholar] [CrossRef]
- Borenstein, J.; Howard, A. Emerging challenges in AI and the need for AI ethics education. AI Ethics 2021, 1, 61–65. [Google Scholar] [CrossRef]
- Sikdar, S.; Lemmerich, F.; Strohmaier, M. GetFair: Generalized fairness tuning of classification models. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, Association for Computing Machinery, Seoul, Republic of Korea, 21–24 June 2022; pp. 289–299. [Google Scholar] [CrossRef]
- Khosravi, H.; Shum, S.B.; Chen, G.; Conati, C.; Tsai, Y.-S.; Kay, J.; Knight, S.; Martinez-Maldonado, R.; Sadiq, S.; Gašević, D. Explainable artificial intelligence in education. Comput. Educ. Artif. Intell. 2022, 3, 100074. [Google Scholar] [CrossRef]
- Woolf, B. Introduction to IJAIED special issue, FATE in AIED. Int. J. Artif. Intell. Educ. 2022, 32, 501–503. [Google Scholar] [CrossRef]
- Islam, S.R.; Russell, I.; Eberle, W.; Dicheva, D. Incorporating the concepts of fairness and bias into an undergraduate computer science course to promote fair automated decision systems. In Proceedings of the 53rd ACM Technical Symposium on Computer Science Education V. 2, Providence, RI, USA, 3–5 March 2022; Association for Computing Machinery: New York, NY, USA, 2022; p. 1075. [Google Scholar] [CrossRef]
- Li, C.; Xing, W.; Leite, W. Using fair AI to predict students’ math learning outcomes in an online platform. Interact. Learn. Environ. 2024, 32, 1117–1136. [Google Scholar] [CrossRef]
- Shin, D.; Rasul, A.; Fotiadis, A. Why am I seeing this? Deconstructing algorithm literacy through the lens of users. Internet Res. 2022, 32, 1214–1234. [Google Scholar] [CrossRef]
- Beerkens, M. An evolution of performance data in higher education governance: A path towards a ‘big data’ era? Qual. High. Educ. 2022, 28, 29–49. [Google Scholar] [CrossRef]
- Kong, S.-C.; Cheung, W.M.-Y.; Zhang, G. Evaluating an Artificial Intelligence Literacy Programme for Developing University Students’ Conceptual Understanding, Literacy, Empowerment and Ethical Awareness. Educ. Technol. Soc. 2023, 26, 16–30. [Google Scholar]
- Prinsloo, P.; Slade, S. An elephant in the learning analytics room: The obligation to act. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference, in LAK ’17, Vancouver, BC, Canada, 13–17 March 2017; Association for Computing Machinery: New York, NY, USA, 2017; pp. 46–55. [Google Scholar] [CrossRef]
- Friesen, P.; Kearns, L.; Redman, B.; Caplan, A.L. Rethinking the Belmont report? Am. J. Bioeth. 2017, 17, 15–21. [Google Scholar] [CrossRef]
- Bearman, M.; Ryan, J.; Ajjawi, R. Discourses of artificial intelligence in higher education: A critical literature review. High. Educ. 2023, 86, 369–385. [Google Scholar] [CrossRef]
- Ayeni, O.O.; Al Hamad, N.M.; Chisom, O.N.; Osawaru, B.; Adewusi, O.E. AI in education: A review of personalized learning and educational technology. GSC Adv. Res. Rev. 2024, 18, 261–271. [Google Scholar] [CrossRef]
- Schei, O.M.; Møgelvang, A.; Ludvigsen, K. Perceptions and use of AI chatbots among students in higher education: A scoping review of empirical studies. Educ. Sci. 2024, 14, 922. [Google Scholar] [CrossRef]
- Wang, K.; Ruan, Q.; Zhang, X.; Fu, C.; Duan, B. Pre-service teachers’ GenAI anxiety, technology self-efficacy, and TPACK: Their structural relations with behavioral intention to design GenAI-assisted teaching. Behav. Sci. 2024, 14, 373. [Google Scholar] [CrossRef]
- Pagallo, U. From automation to autonomous systems: A legal phenomenology with problems of accountability. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, Melbourne, Australia, 19–25 August 2017; IJCAI-17. Sierra, C., Ed.; International Joint Conferences on Artificial Intelligence: Melbourne, Australia, 2017; pp. 17–23. [Google Scholar] [CrossRef]
- Salas-Pilco, S.Z.; Xiao, K.; Oshima, J. Artificial intelligence and new technologies in inclusive education for minority students: A systematic review. Sustainability 2022, 14, 13572. [Google Scholar] [CrossRef]
- Yang, H.; Kyun, S. The current research trend of artificial intelligence in language learning: A systematic empirical literature review from an activity theory perspective. Australas. J. Educ. Technol. 2022, 38, 180–210. [Google Scholar] [CrossRef]
- Ungerer, L.; Slade, S. Ethical considerations of artificial intelligence in learning analytics in distance education contexts. In Learning Analytics in Open and Distributed Learning: Potential and Challenges; Prinsloo, P., Slate, S., Khalil, M., Eds.; Springer Nature Singapore: Singapore, 2022; pp. 105–120. [Google Scholar] [CrossRef]
- Akinrinola, O.; Okoye, C.C.; Ofodile, O.C.; Ugochukwu, C.E. Navigating and reviewing ethical dilemmas in AI development: Strategies for transparency, fairness, and accountability. GSC Adv. Res. Rev. 2024, 18, 050–058. [Google Scholar] [CrossRef]
- Fu, Y.; Weng, Z. Navigating the ethical terrain of AI in education: A systematic review on framing responsible human-centered AI practices. Comput. Educ. Artif. Intell. 2024, 7, 100306. [Google Scholar] [CrossRef]
- Petousi, V.; Sifaki, E. Contextualising harm in the framework of research misconduct. Findings from discourse analysis of scientific publications. Int. J. Sustain. Dev. 2021, 23, 149–174. Available online: https://www.inderscienceonline.com/doi/10.1504/IJSD.2020.115206 (accessed on 30 September 2024). [CrossRef]
- Mahmud, H.; Islam, A.K.M.N.; Ahmed, S.I.; Smolander, K. What influences algorithmic decision-making? A systematic literature review on algorithm aversion. Technol. Forecast. Soc. Chang. 2022, 175, 121390. [Google Scholar] [CrossRef]
- Klimova, B.; Pikhart, M.; Kacetl, J. Ethical issues of the use of AI-driven mobile apps for education. Front. Public Health 2023, 10, 1118116. [Google Scholar] [CrossRef]
- Salloum, S.A. AI perils in education: Exploring ethical concerns. In Artificial Intelligence in Education: The Power and Dangers of ChatGPT in the Classroom; Al-Marzouqi, A., Salloum, S.A., Al-Saidat, M., Aburayya, A., Gupta, B., Eds.; Springer Nature Switzerland: Cham, Switzerland, 2024; pp. 669–675. [Google Scholar] [CrossRef]
- Airaj, M. Ethical artificial intelligence for teaching-learning in higher education. Educ. Inf. Technol. 2024. [Google Scholar] [CrossRef]
- Smuha, N.A. Pitfalls and pathways for trustworthy artificial intelligence in education. In The Ethics of Artificial Intelligence in Education; Holmes, W., Porayska-Pomsta, K., Eds.; Routledge: New York, NY, USA, 2022. [Google Scholar]
- Zawacki-Richter, O.; Marín, V.I.; Bond, M.; Gouverneur, F. Systematic review of research on artificial intelligence applications in higher education—Where are the educators? Int. J. Educ. Technol. High. Educ. 2019, 16, 39. [Google Scholar] [CrossRef]
- Kwet, M. Digital colonialism: US empire and the new imperialism in the Global South. Race Class 2019, 60, 3–26. [Google Scholar] [CrossRef]
- Moran, T.C. Racial technological bias and the white, feminine voice of AI VAs. Commun. Crit./Cult. Stud. 2021, 18, 19–36. [Google Scholar] [CrossRef]
- Antonenko, P.D.; Dawson, K.; Cheng, L.; Wang, J. Using technology to address individual differences in learning. In Handbook of Research in Educational Communications and Technology: Learning Design; Bishop, M.J., Boling, E., Elen, J., Svihla, V., Eds.; Springer International Publishing: Cham, Switzerland, 2020; pp. 99–114. [Google Scholar] [CrossRef]
- Gibson, D.; Kovanovic, V.; Ifenthaler, D.; Dexter, S.; Feng, S. Learning theories for artificial intelligence promoting learning processes. Br. J. Educ. Technol. 2023, 54, 1125–1146. [Google Scholar] [CrossRef]
- Moser, A.; Korstjens, I. Series: Practical guidance to qualitative research. Part 3: Sampling, data collection and analysis. Eur. J. Gen. Pract. 2018, 24, 9–18. [Google Scholar] [CrossRef]
- Braun, V.; Clarke, V. Using thematic analysis in psychology. Qual. Res. Psychol. 2006, 3, 77–101. [Google Scholar] [CrossRef]
- Tie, Y.C.; Birks, M.; Francis, K. Grounded theory research: A design framework for novice researchers. SAGE Open Med. 2019, 7. [Google Scholar] [CrossRef]
- Cohen, L.; Manion, L.; Morrison, K. Research Methods in Education, 7th ed.; Routledge: London, UK, 2011. [Google Scholar]
- Martin, P.Y.; Turner, B.A. Grounded theory and organizational research. J. Appl. Behav. Sci. 1986, 22, 141–157. [Google Scholar] [CrossRef]
- Holloway, I.; Todres, L. The status of method: Flexibility, consistency and coherence. Qual. Res. 2003, 3, 345–357. [Google Scholar] [CrossRef]
- Nemorin, S.; Vlachidis, A.; Ayerakwa, H.M.; Andriotis, P. AI hyped? A horizon scan of discourse on artificial intelligence in education (AIED) and development. Learn. Media Technol. 2023, 48, 38–51. [Google Scholar] [CrossRef]
- Kasneci, E.; Sessler, K.; Küchemann, S.; Bannert, M.; Dementieva, D.; Fischer, F.; Gasser, U.; Groh, G.; Günnemann, S.; Hüllermeier, E.; et al. ChatGPT for good? On opportunities and challenges of large language models for education. Learn. Individ. Differ. 2023, 103, 102274. [Google Scholar] [CrossRef]
- Holstein, K.; Doroudi, S.; Holstein, K.R.; Doroudi, S. Equity and artificial intelligence in education. In The Ethics of Artificial Intelligence in Education; Holmes, W., Porayska-Pomsta, K., Eds.; Routledge: New York, NY, USA, 2022; pp. 151–173. [Google Scholar]
- Lin, M.P.C.; Chang, D. Exploring inclusivity in ai education: Perceptions and pathways for diverse learners. In Generative Intelligence and Intelligent Tutoring Systems; Sifaleras, A., Lin, F., Eds.; Springer Nature Switzerland: Cham, Switzerland, 2024; pp. 237–249. [Google Scholar]
- Johnston, H.; Wells, R.F.; Shanks, E.M.; Boey, T.; Parsons, B.N. Student perspectives on the use of generative artificial intelligence technologies in higher education. Int. J. Educ. Integr. 2024, 20, 2. [Google Scholar] [CrossRef]
- Lin, M.P.C.; Chang, D.H.; Winne, P.H. A proposed methodology for investigating student-chatbot interaction patterns in giving peer feedback. Educ. Technol. Res. Dev. 2024. [Google Scholar] [CrossRef]
- Lin, M.P.C.; Chang, D. Enhancing post-secondary writers’ writing skills with a chatbot: A mixed-method classroom study. J. Educ. Technol. Soc. 2020, 23, 78–92. [Google Scholar] [CrossRef]
- Zhang, K.; Aslan, A.B. AI technologies for education: Recent research & future directions. Comput. Educ. Artif. Intell. 2021, 2, 100025. [Google Scholar] [CrossRef]
- Sullivan, M.; Kelly, A.; Mclaughlan, P. ChatGPT in higher education: Considerations for academic integrity and student learning. J. Appl. Learn. Teach. 2023, 6, 31–40. [Google Scholar] [CrossRef]
- Lin, M.P.C. A Proposed Methodology for Investigating Chatbot Effects in Peer Review. Doctoral Dissertation, Simon Fraser University, Burnaby, BC, Canada, 2020. Available online: https://summit.sfu.ca/item/20533 (accessed on 8 September 2024).
- Schiff, D. Education for AI, not AI for education: The role of education and ethics in national AI policy strategies. Int. J. Artif. Intell. Educ. 2022, 32, 527–563. [Google Scholar] [CrossRef]
- Tariq, M.U. Equity and inclusion in learning ecosystems. In Preparing Students for the Future Educational Paradigm; IGI Global: Hershey, PA, USA, 2024; pp. 155–176. [Google Scholar] [CrossRef]
Number | Question |
---|---|
Q1 | How do you perceive the role of AI technologies, such as chatbots, in your learning experience? Please describe any positive or negative aspects you’ve noticed. |
Q2 | In your experience, do you feel that AI technologies (Grammarly or ChatGPT) are designed to cater to the diverse needs of all students, including those from different cultural or linguistic backgrounds? Please elaborate on your response. |
Q3 | Please describe how AI technologies have impacted your sense of belonging and self-efficacy (confidence) in the learning environment? Feel free to share specific instances or general impressions. |
Q4 | What improvements or changes would you suggest to make AI technologies more inclusive and responsive to the needs of a diverse student population? |
Q5 | How do you think AI technologies could be better utilized to support your individual learning needs and goals, especially considering your unique cultural or linguistic background? |
Theme | Definition | Example |
---|---|---|
Linguistic and cultural inclusivity | Accommodating students from diverse linguistic and cultural backgrounds in the learning community to ensure language use does not become a barrier to learning or course participation, particularly for students who are non-English speakers. | “It can polish our writing assignments. As an international student, it can give many suggestions to make my articles more ‘local’”. “AI technologies can help me correct some mistakes, especially grammar”. “English as second language learners could also benefit from this because it can translate and simplify texts”. |
Learning adaptability | Tailoring educational content to individual learning needs and preferences through simplifying complex concepts, breaking down difficult queries, and providing explanations at appropriate levels of complexity to enhance comprehension. | “I can ask the AI to explain to me like I am 10 for instance and it will simplify the biggest terms into an easy explanation that works for me”. “It has helped me a lot through academia… I can use AI technology to help me simplify it and break it down”. “It really aids in breaking down any complex queries, which often makes things easier for me to comprehend and process”. |
Study aid | Assisting students in their studying by simplifying, summarizing, organizing learning content for better understanding, and creating practice questions for retention. | “AI Chatbots… I would voice record the lecture and use Rewind to summarize the voice lecture and give me some essential notes”. “AI Chatbots are very useful in terms of studying and giving my study notes”. “I have asked AI technology to give me practice prompts to practice writing long answers for a certain topic and this was a great study method for me”. |
Research assistance | Supporting students in their research process by identifying keywords, determining the relevance of sources, and summarizing articles to facilitate effective and efficient research experience. | “In terms of research, AI chatbots is a great starting point. I would put the research question in an AI chatbot and ask for some key words…” “When we ask ChatGPT questions it gives us a good foundation of where we need to start our research”. “If I do see an article, after reading the introduction myself, I can input that article into an AI chatbot to help me summarize it for me and do a simple explanation so I can decide whether that article will be useful to me or not”. |
Cognitive expansion | Stimulating curiosity or the desire to discover knowledge about unfamiliar topics, inspiring new ideas and creativity, and supporting exploration of alternative perspectives and complex concepts. | “For my COGS 110 (Learning in Everyday Life: The Art and Science of Hacking your Brain), I learned more about AI technologies such as AI and AGI. In that course, I was able to research more into that realm of AI that connects to learning and how it can impact the human experience and our brain”. “I personally find AI helpful in allowing students to brainstorm ideas, think of research topics, and expand the scope of thinking needed for assignments”. “AI could offer perspectives similar to what the person might have (to provide validation and comfort) as well as provide diverse perspectives to challenge and broaden the person’s lens of the world”. |
Dependency and reduced critical thinking | The over-reliance on AI tools that leads to a decreased ability to think critically and a lack of motivation to develop understanding of the subject matter and problem-solving skills. | “We can easily become dependent on artificial intelligence. This can also lead to laziness in thinking or digging deeper into the problem”. “AI is able to produce so much in so little time that students may start to question why they need to try so hard. Thus, jeopardizing their motivation to learn”. “Despite these positive aspects, the negative aspects of ChatGPT are that it makes students rely on AI for quick answers rather than allowing us to think critically and problem-solve as we would without AI technology”. |
Inaccuracy and misinformation risks | The risks of AI tools to provide potentially inaccurate or false information, which can hinder learning and lead to misunderstandings if not critically evaluated. | “AI sometimes provides false information which requires me to be critical and verify the information by myself.” “The level of accuracy varies from one AI system to another. Especially when the student does not understand the context needed to get the answer given, the student will not learn or will be taught wrong information without knowing that it is wrong”. “Something negative that I experienced was the inconsistency of formatting issues and misinformation about topics that I tried to study”. |
Ethical and academic integrity concerns | Concerns regarding unethical use of AI tools in educational contexts such as cheating on assignments and exams or using AI to complete work expected to be completed independently. | “AI can also be misused by students in regard to cheating, and being lazy”. “With the rise of AI technologies in education, there are things that are academically wrong, such as having ChatGPT write an essay for you or using it to answer questions on tests or quizzes”. “Personally, I try not to use AI technologies throughout my education and learning. I worry that I will be penalized for improper or unethical use of these tools, when that is not my intent”. |
Suggested future AI improvements | Students’ suggestions for improving AI technologies to address gaps in their current functionality or to better serve the needs of diverse learners. Suggestions were on features, capabilities, or design changes to improve the adaptability, inclusivity, and effectiveness of AI tools in education. | “For example, if we need help solving a math equation, then asking an AI specialized in that area would be more helpful instead of a general answer”. “Diversify training data, incorporating various cultural, linguistic, and socioeconomic perspectives to mitigate biases”. “Focus on understanding diverse user communities…co-designing solutions with them, not for them”. |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Lin, M.P.-C.; Liu, A.L.; Poitras, E.; Chang, M.; Chang, D.H. An Exploratory Study on the Efficacy and Inclusivity of AI Technologies in Diverse Learning Environments. Sustainability 2024, 16, 8992. https://doi.org/10.3390/su16208992
Lin MP-C, Liu AL, Poitras E, Chang M, Chang DH. An Exploratory Study on the Efficacy and Inclusivity of AI Technologies in Diverse Learning Environments. Sustainability. 2024; 16(20):8992. https://doi.org/10.3390/su16208992
Chicago/Turabian StyleLin, Michael Pin-Chuan, Arita Li Liu, Eric Poitras, Maiga Chang, and Daniel H. Chang. 2024. "An Exploratory Study on the Efficacy and Inclusivity of AI Technologies in Diverse Learning Environments" Sustainability 16, no. 20: 8992. https://doi.org/10.3390/su16208992
APA StyleLin, M. P. -C., Liu, A. L., Poitras, E., Chang, M., & Chang, D. H. (2024). An Exploratory Study on the Efficacy and Inclusivity of AI Technologies in Diverse Learning Environments. Sustainability, 16(20), 8992. https://doi.org/10.3390/su16208992