The Effects of Generative AI Platforms on Undergraduates’ Narrative Intelligence and Writing Self-Efficacy
Abstract
:1. Introduction
RQ1. Does creating digital stories using generative AI platforms have a significant impact on the narrative intelligence of undergraduate students?
RQ2. Does creating digital stories with generative AI platforms have a significant effect on the writing self-efficacy of undergraduate students?
2. Materials and Methods
2.1. Research Design
2.2. Sampling
2.3. Measures
2.3.1. Narrative Intelligence Scale
- Emplotment involves the process of organizing events, especially those that occur less frequently or are of high significance, into a coherent and meaningful order. It helps create a narrative structure that makes sense to the audience;
- Characterization focuses on the ability to vividly depict and create a mental image of the thoughts, emotions, and personalities of the participants or characters involved in a narrative. It adds depth and relatability to the story;
- Narration is the skill of effectively conveying information and engaging in a dialogue with others about the events taking place in a narrative. It involves presenting events logically and making assumptions that allow for meaningful communication;
- Generation entails the process of arranging events within a narrative in a way that makes them predictable and coherent. It involves creating a structured and understandable sequence of events;
- Thematization refers to the ability to recognize and be aware of recurring patterns or themes within specific events. It involves identifying common elements or structures that help shape the overall meaning of a narrative.
2.3.2. Situated Academic Writing Self-Efficacy Scale
2.4. Procedure
- Promoting inclusivity and equity: This study recognizes that digital storytelling has the potential to democratize content creation and media production. By examining how these technologies are used by undergraduate students from diverse sociodemographic backgrounds, this study contributes to understanding whether story creation tasks can help bridge gaps in inclusivity and equity in the digital media landscape;
- Impact of storytelling training courses: This study underscores the importance of training courses in shaping students’ attitudes towards story creation technologies. This finding has implications for educational institutions and policymakers who may consider integrating Greek curricula to prepare students for the evolving digital media landscape;
- Quality and reliability: By assessing students’ contentment with the reliability of creative storytelling technologies, this study highlights the importance of ensuring that story content meets certain quality standards based on lessons taught from the Greek curriculum. This is significant for media producers and policymakers maintaining the integrity of multimedia platforms;
- Policy considerations: This research emphasizes the need for considerations and policy guidance in instructional contexts to create a fair and equitable digital media environment. This is particularly relevant in an era where story creation technologies are becoming increasingly prevalent in media production.
2.5. Data Analysis
3. Results
4. Discussion and Conclusions
5. Implications
- Enhancement of narrative intelligence and writing self-efficacy: This study’s results provide evidence that the use of generative AI platforms can significantly improve undergraduate students’ narrative intelligence scores and writing self-efficacy. This finding aligns with the growing recognition of the potential benefits of technology-assisted learning in enhancing core competencies. Educators and institutions can leverage these tools to empower students with the skills and confidence necessary for effective communication;
- Effects of generative AI platforms on storytelling: The positive outcomes observed in the experimental group highlight the evolving role of AI technologies in education. Generative AI platforms can augment traditional teaching methods by offering personalized feedback, suggesting improvements, and facilitating the writing process. The study suggests that these technologies can serve as valuable educational aids, supporting students in their journey to become proficient writers;
- Human creativity vs. generative AI assistance: Educators and curriculum designers should consider incorporating generative AI platforms into writing instruction. On the one hand, while generative AI demonstrated clear benefits in terms of narrative intelligence and writing self-efficacy, the absence of a significant difference in the “Creative identity” factor suggests that human creativity remains a distinctive and irreplaceable aspect of storytelling. This finding underscores the importance of striking a balance between generative AI assistance and human creativity, particularly in fields where originality and creative expression are highly valued. On the other hand, it is crucial to do so thoughtfully, recognizing the strengths and limitations of AI tools. These platforms can be especially useful for tasks involving grammar, structure, and organization, leaving space for students to focus on the creative aspects of their writing.
6. Limitations and Future Work
- Long-term effects: Conducting longitudinal studies to assess the long-term impact of digital storytelling and generative AI technologies on narrative intelligence and writing self-efficacy. This would provide a more comprehensive understanding of the sustained benefits or potential drawbacks of using these tools;
- Diversity and inclusion: Investigating how digital storytelling and AI technologies affect students from diverse backgrounds, including those with varying levels of writing proficiency, linguistic diversity, and accessibility needs, would ensure that the benefits are accessible to a broad range of learners;
- Pedagogical strategies: Exploring different pedagogical approaches and instructional designs that maximize the benefits of digital storytelling and AI technologies in educational contexts. Developing guidelines and best practices for educators to effectively integrate these tools into their teaching;
- Interdisciplinary research: Collaborating with experts from the fields of psychology, education, and computer science to gain a more holistic understanding of the cognitive processes involved in digital storytelling and generative AI writing. This interdisciplinary approach can shed light on the underlying mechanisms at play;
- Ethical considerations: Investigating the ethical implications of using AI in education, especially regarding issues such as plagiarism detection, privacy, and potential bias in AI-generated content. Developing ethical guidelines for the responsible use of AI technologies in educational settings.
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Rodríguez, C.C.; Romero, C.N. Digital Storytelling in Primary Education: A Comparison of Already Made Resources and Story Creation Tools to Improve English as a Foreign Language; Octaedro: Barcelona, Spain, 2023; p. 387. [Google Scholar]
- Nurlaela Ilham, M.J.; Lisabe, C. The Effects of Storyjumper on Narrative Writing Ability of EFL Learners in Higher Education. J. Educ. FKIP UNMA 2022, 8, 1641–1647. [Google Scholar]
- Güvey, A. Writing a Folktale as an Activity of Written Expression: Digital Folktales with Storyjumper. Educ. Policy Anal. Strateg. Res. 2020, 15, 159–185. [Google Scholar] [CrossRef]
- Dai, J.; Wang, L.; He, Y. Exploring the Effect of Wiki-based Writing Instruction on Writing Skills and Writing Self-efficacy of Chinese English-as-a-foreign Language Learners. Front. Psychol. 2023, 13, 1069832. [Google Scholar] [CrossRef]
- Ispir, B.; Yıldız, A. An Overview of Digital Storytelling Studies in Classroom Education in Turkey. J. Qual. Res. Educ. 2023, 23, 187–216. [Google Scholar] [CrossRef]
- Zakaria, S.M.; Yunus, M.M.; Nazri, N.M.; Shah, P.M. Students’ Experience of Using Storybird in Writing ESL Narrative Text. Creative Educ. 2016, 7, 2107–2120. [Google Scholar] [CrossRef]
- Fitriyani, R.; Ahsanu, M.; Kariadi, M.T.; Riyadi, S. Teaching Writing Skills through Descriptive Text by Using Digital Storytelling “storyjumper”. J. Engl. Educ. List. 2023, 4. [Google Scholar]
- Fang, X.; Ng, D.T.K.; Leung, J.K.L.; Chu, S.K.W. A Systematic Review of Artificial Intelligence Technologies Used for Story Writing. Educ. Inf. Technol. 2023, 1–37. [Google Scholar] [CrossRef]
- Lambert, J.; Stevens, M. Chatgpt and Generative AI Technology: A Mixed Bag of Concerns and New Opportunities. Comput. Sch. 2023, 1–25. [Google Scholar] [CrossRef]
- Han, A.; Cai, Z. Design Implications of Generative AI Systems for Visual Storytelling for Young Learners. In Proceedings of the 22nd Annual ACM Interaction Design and Children Conference, Chicago, IL, USA, 19–23 June 2023; pp. 470–474. [Google Scholar]
- O’Meara, J.; Murphy, C. Aberrant AI Creations: Co-creating Surrealist Body Horror Using the DALL-E Mini Text-to-image Generator. Converg. Int. J. Res. New Media Technol. 2023, 29, 1070–1096. [Google Scholar] [CrossRef]
- Bender, S.M. Coexistence and Creativity: Screen Media Education in the Age of Artificial Intelligence Content Generators. Media Pract. Educ. 2023, 1–16. [Google Scholar] [CrossRef]
- Chaudhary, S. 15 Best Free AI Content Generator & AI Writers for 2023. 2023. Available online: https://medium.com/@bedigisure/free-ai-content-generator-11ef7cbb2aa0 (accessed on 10 October 2023).
- Lee, J.H.; Shin, D.; Noh, W. Artificial Intelligence-based Content Generator Technology for Young English-as-a-foreign-language Learners’ Reading Enjoyment. RELC J. 2023, 54, 508–516. [Google Scholar] [CrossRef]
- Im, H.; Jeon, S.; Cho, H.; Shin, S.; Choi, D.; Hong, H. AI Story Relay: A Collaborative Writing of Design Fiction to Investigate Artificial Intelligence Design Considerations. In Proceedings of the 2023 ACM International Conference on Supporting Group Work, Hilton Head, SC, USA, 8–11 January 2023; pp. 6–8. [Google Scholar]
- Pellas, N. The influence of sociodemographic factors on students’ attitudes toward AI-generated video content creation. Smart Learn. Environ. 2023, 10, 57. [Google Scholar] [CrossRef]
- Sun, T.; Wang, C. College Students’ Writing Self-efficacy and Writing Self-regulated Learning Strategies in Learning English as a Foreign Language. System 2020, 90, 102221. [Google Scholar] [CrossRef]
- Teng, M.F.; Wang, C. Assessing Academic Writing Self-Efficacy Belief and Writing Performance in a Foreign Language Context. Foreign Lang. Ann. 2023, 56, 144–169. [Google Scholar] [CrossRef]
- Campbell, C.W.; Batista, B. To Peer or Not to Peer: A Controlled Peer-editing Intervention Measuring Writing Self-efficacy in South Korean Higher Education. Int. J. Educ. Res. Open 2023, 4, 100218. [Google Scholar] [CrossRef]
- Randall, W.L. Narrative intelligence and the novelty of our lives. J. Aging Stud. 1999, 13, 11–28. [Google Scholar] [CrossRef]
- Ziaei, S.; Ghonsooly, B.; Ghabanchi, Z.; Shahriari, H. Proposing a Cognitive EFL Writing Model Based on Personality Types and Narrative Writing Intelligence: A SEM Approach. Issues Lang. Teach. 2019, 8, 163–185. [Google Scholar]
- Golabi, A.; Heidari, F.F. Prediction of Iranian EFL Learners’ Learning Approaches Through Their Teachers’ Narrative Intelligence and Teaching Styles: A Structural Equation Modelling Analysis. Iran. J. Engl. Acad. Purp. 2019, 8, 1–12. [Google Scholar]
- Pishghadam, R.; Baghaei, P.; Shams, M.A.; Shamsaee, S. Construction and Validation of a Narrative Intelligence Scale with the Rasch Rating Scale Model. Int. J. Educ. Psychol. Assess. 2011, 8, 75–90. [Google Scholar]
- Ge, J.; Lai, J.C. Artificial Intelligence-based Text Generators in Hepatology: ChatGPT Is Just the Beginning. Hepatol. Commun. 2023, 7, e0097. [Google Scholar] [CrossRef]
- Yuan, A.; Coenen, A.; Reif, E.; Ippolito, D. Wordcraft: Story Writing with Large Language Models. In Proceedings of the 27th International Conference on Intelligent User Interfaces, Helsinki, Finland, 22–25 March 2022; pp. 841–852. [Google Scholar]
- Pishghadam, R.; Shams, M.A. Hybrid Modeling of Intelligence and Linguistic Factors as Predictors of L2 Writing Quality: A SEM Approach. Lang. Test. Asia 2012, 2, 1–24. [Google Scholar] [CrossRef]
- Cohen, L.; Manion, L.; Morrison, K. Research Methods in Education; Routledge: New York, NY, USA, 2007. [Google Scholar]
- Rogers, J.; Revesz, A. Experimental and Quasi-Experimental Designs; Routledge: London, UK, 2019; pp. 133–143. [Google Scholar]
- Cennamo, K.; Ross, J.; Ertmer, P. Technology Integration for Meaningful Classroom Use. A Standard-based Approach; Wadsworth: Belmont, CA, USA, 2010. [Google Scholar]
- Collins, A.; Joseph, D.; Bielaczyc, K. Design Research: Theoretical and Methodological Issues; Psychology Press: Portland USA, 2016. [Google Scholar]
- Mitchell, K.M.; McMillan, D.E.; Lobchuk, M.M.; Nickel, N.C.; Rabbani, R.; Li, J. Development and Validation of the Situated Academic Writing Self-Efficacy Scale (SAWSES). Assess. Writ. 2021, 48, 100524. [Google Scholar] [CrossRef]
- Cortina, J.M. What Is Coefficient Alpha? an Examination of Theory and Applications. J. Appl. Psychol. 1993, 78, 98–104. [Google Scholar] [CrossRef]
- Garth, A. Analyzing Data Using SPSS; Sheffield Hallam University: Sheffield, UK, 2008. [Google Scholar]
- Cha, E.-S.; Kim, K.H.; Erlen, J.A. Translation of Scales in Cross-cultural Research: Issues and Techniques. J. Adv. Nurs. 2007, 58, 386–395. [Google Scholar] [CrossRef] [PubMed]
Groups | Pretest | Treatment | Posttest |
---|---|---|---|
G1 (NR) | O1 | O2 | |
G2 (NR) | O3 | X | O4 |
Gender | Control Group | Experimental Group | Total | |||
---|---|---|---|---|---|---|
n | % | n | % | n | % | |
Female | 20 | 60.4 | 23 | 69.8 | 43 | 64.1 |
Male | 12 | 36.6 | 9 | 31.2 | 21 | 35.9 |
Total | 32 | 100 | 32 | 100 | 64 | 100 |
Group | Gender | Total | X2 | df | p | ||||
---|---|---|---|---|---|---|---|---|---|
Female | Male | ||||||||
n | % | n | % | n | % | ||||
Control | 19 | 29.7 | 13 | 20.3 | 32 | 50 | 0.58 | 1 | 0.455 |
Experimental | 22 | 34.4 | 10 | 15.6 | 32 | 50 | |||
Total | 41 | 64.1 | 23 | 35.9 | 64 | 100 |
Group | n | M | SD | df | t | p |
---|---|---|---|---|---|---|
Control Group | 32 | 3.66 | 0.53 | 62 | −1.88 | 0.11 |
Experimental Group | 32 | 3.77 | 0.44 |
Test | Group | Kolmogorov–Smirnov | Shapiro–Wilk | ||
---|---|---|---|---|---|
N | p | N | p | ||
Writing Self-Efficacy Scale pretest | Control | 32 | 0.23 | 32 | 0.71 |
Experimental | 32 | 0.23 | 32 | 0.17 | |
Writing Self-Efficacy Scale posttest | Control | 32 | 0.23 | 32 | 0.76 |
Experimental | 32 | 0.23 | 32 | 0.12 | |
Narrative Intelligence Scale posttest | Control | 32 | 0.23 | 32 | 0.73 |
Experimental | 32 | 0.13 | 32 | 0.16 |
Group | N | M | SD | df | t | p | Cohen’s d | |
---|---|---|---|---|---|---|---|---|
Emplotment | Control | 32 | 4.52 | 0.72 | 62 | −2.57 | 0.02 * | 0.61 |
Experimental | 32 | 4.91 | 0.66 | |||||
Characterization | Control | 32 | 4.55 | 0.71 | 62 | −2.87 | 0.01 * | 0.74 |
Experimental | 32 | 4.88 | 0.62 | |||||
Generation | Control | 32 | 4.22 | 0.63 | 62 | −2.74 | 0.01 * | 0.71 |
Experimental | 32 | 4.53 | 0.66 | |||||
Narration | Control | 32 | 4.66 | 0.65 | 62 | −2.64 | 0.01 * | 0.73 |
Experimental | 32 | 4.84 | 0.64 | |||||
Thematization | Control | 32 | 4.44 | 0.77 | 62 | −3.61 | 0.00 * | 0.93 |
Experimental | 32 | 4.88 | 0.61 |
SAWSES | Group | Pretest | Posttest | ||||
---|---|---|---|---|---|---|---|
N | M | SD | N | M | SD | ||
Writing essentials | Control | 32 | 3.33 | 0.55 | 32 | 3.67 | 0.66 |
Experimental | 32 | 3.53 | 0.56 | 32 | 4.13 | 0.56 | |
Reflective writing | Control | 32 | 3.17 | 0.62 | 32 | 3.57 | 0.67 |
Experimental | 32 | 3.42 | 0.45 | 32 | 3.88 | 0.52 | |
Creative identity | Control | 32 | 3.64 | 0.78 | 32 | 3.78 | 0.79 |
Experimental | 32 | 3.88 | 0.67 | 32 | 4.38 | 0.62 | |
General writing self-efficacy | Control | 32 | 3.33 | 0.49 | 32 | 3.78 | 0.68 |
Experimental | 32 | 3.64 | 0.59 | 32 | 3.93 | 0.64 |
Variable | Source | Sum of Squares | df | Mean Square | F | p | Partial Eta Squared (η2) |
---|---|---|---|---|---|---|---|
Writing essentials | Group | 3.089 | 1 | 3.197 | 6.129 * | 0.018 | 0.09 |
Error | 32.998 | 62 | 0.524 | ||||
Scores | 0.762 | 1 | 0.768 | 8.698 * | 0.004 | 0.13 | |
Group * Score | 0.388 | 1 | 0.389 | 4.792 * | 0.037 * | 0.08 | |
Error | 5.287 | 62 | 0.091 | ||||
Total | 41.552 | 127 | |||||
Reflective writing | Group | 4.698 | 1 | 4.762 | 8.693 * | 0.005 | 0.12 |
Error | 35.875 | 62 | 0.555 | ||||
Scores | 1.974 | 1 | 1.778 | 23.512 * | 0.000 | 0.27 | |
Group * Score | 0.566 | 1 | 0.569 | 6.846 * | 0.016 * | 0.10 | |
Error | 4.936 | 62 | 0.079 | ||||
Total | 46.867 | 117 | |||||
Creative identity | Group | 2.298 | 1 | 2.368 | 2.790 | 0.113 | 0.04 |
Error | 55.348 | 62 | 0.866 | ||||
Scores | 0.936 | 1 | 0.911 | 3.995 | 0.056 | 0.06 | |
Group * Score | 0.003 | 1 | 0.003 | 0.007 | 0.968 | 0.00 | |
Error | 14.698 | 62 | 0.241 | ||||
Total | 73.241 | 117 | |||||
General writing self-efficacy | Group | 4.293 | 1 | 3.197 | 6.54 * | 0.017 | 0.09 |
Error | 4.624 | 62 | 0.521 | ||||
Scores | 0.991 | 1 | 0.984 | 14.559 * | 0.000 | 0.19 | |
Group * Score | 0.308 | 1 | 0.298 | 4.548 * | 0.039 * | 0.07 | |
Error | 4.298 | 62 | 0.078 | ||||
Total | 40.469 | 117 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Pellas, N. The Effects of Generative AI Platforms on Undergraduates’ Narrative Intelligence and Writing Self-Efficacy. Educ. Sci. 2023, 13, 1155. https://doi.org/10.3390/educsci13111155
Pellas N. The Effects of Generative AI Platforms on Undergraduates’ Narrative Intelligence and Writing Self-Efficacy. Education Sciences. 2023; 13(11):1155. https://doi.org/10.3390/educsci13111155
Chicago/Turabian StylePellas, Nikolaos. 2023. "The Effects of Generative AI Platforms on Undergraduates’ Narrative Intelligence and Writing Self-Efficacy" Education Sciences 13, no. 11: 1155. https://doi.org/10.3390/educsci13111155