Integrating Mobile Technologies: Two Stage Study on Academic Performance and Social Presence

Abstract

The ubiquity of mobile technologies has sparked interest in their potential impact. However, current research predominantly concentrates on adults, neglecting the younger demographic that is more profoundly immersed in mobile technologies. To bridge this gap, this study introduces a specialized instructional design, integrating the widely used application Duolingo into English classes, to explore the ramifications of mobile technologies on the academic performance and social presence of high school students in their initial years. Conducted within a private school where English is mandatory, the research adopts a mixed-method approach, with quantitative and qualitative facets. Data collection comprises a needs analysis, a standardized English test (KET), a social presence scale and follow-up interviews. Participants were selected using purposive sampling, 16 students each for the experimental and control groups. Semi-structured interviews were conducted with eight voluntary participants. Analysis techniques, including t-tests for the social presence scale, descriptive analysis for interviews, and grading for exam results were applied. Noteworthy distinctions in students’ academic performances were revealed through effect size measurements in writing (0.78), listening (1.01), and speaking (0.82), with a minor change in reading (−0.1). Additionally, a significant variation in social presence emerged (t = −9.194; p < 0.05). Qualitative themes emerged, highlighting Duolingo’s entertainment value, curriculum alignment, content diversity, accessibility, parental consent, and teacher guidance. Quantitative results demonstrated Duolingo’s positive impact on academic performance, while qualitative insights reflected students’ enthusiasm for its integration. Findings invite a critical examination of the role of such technologies, prompting inquiries into their potential advantages and challenges.

1. Introduction

Coined the ‘global village’, the world is becoming more connected as technology advances day by day. With the growing number of features they possess, mobile technologies have already become an integral part of the lives of millions of people worldwide with no time or place limit [1]. They have gone far beyond being devices that allow users to make calls and send messages by allowing their users to obtain any type of information imaginable [2]. The significant characteristics of mobile technologies such as ease of access to information, ability to create and share content, and ability to share information have paved the way for alternative approaches to the formation of a teaching and learning setting, and there is growing interest in exploring their potential contributions [3]. However, despite the swift proliferation of mobile technologies, the quantity of research endeavors exploring their utilization beyond conventional settings remains rather limited, particularly in comparison to the rapid expansion of these technologies [4]. Yet, existing research tends to concentrate on adults, neglecting younger age groups, who are more immersed in mobile technologies [5]. While efforts have been made to incorporate mobile technology-based learning within school education, as exemplified by studies such as [6], the actual adoption and integration of such approaches remain constrained [7,8]. The body of research addressing barriers to mobile technology-based learning in primary and secondary education schools remains relatively restricted [9].

Needless to say, communication is the cornerstone of language education opening paths to more sharing and collaboration among people of almost all ages. Users prefer mobile learning because they want to communicate with one another in online communities by exchanging messages, opinions, information, images, and videos, which facilitates communication easily [10]. In this context, mobile learning refers to an educational experience that takes place through mobile devices, allowing students to participate regardless of their physical location or time [11]. As the name suggests, MALL (mobile assisted language learning) is using mobile technologies to study languages [12]. Considering the popularity and spread of devices such as tablets and smart phones, it can be claimed that exploring the efficacy and impact of them has become vital among researchers, especially when the research regarding implementation of MALL is rather limited [13] and more research is needed [14].

In terms of English language education in the EFL context, typically the majority of English language instruction occurs within formal classroom settings, offering limited opportunities for students who are not within those walls and lacking the necessary exposure to casual use of the language. Talking about Duolingo, in particular, as it is the application this study is interested in investigating, Duolingo’s features are said to facilitate interaction. According to the participants’ responses in a study interested in foreign language education, conducted by Falk and Götz [15], n = 212, the automated commentary and interaction formed with Duolingo were beneficial and facilitative. Taking into account the research in the field, it would be unfair to claim that there are not plausible implications received from the studies in the field but would be fair to claim that although mobile technologies have been growing rapidly, the number of research studies investigating their use outside the classroom is relatively small, especially when compared to their growth [16,17]. This is especially true for research exploring the impacts of Duolingo and other mobile technologies for L2 development [18].

In the case of social presence theory, which has gained more attention thanks to the significance of communication and socialization, there are many studies investigating it in educational settings, some suggesting a positive impact while some others refer to a high correlation [19,20]. In general, the degree to which a person feels ‘real’ when using media platforms for communication is defined as social presence. To put it another way, the level of self-awareness someone has when using media technologies to communicate with others is known as their social presence. According to [21], social presence is the evaluation of a student’s degree of sense of belonging to a group. In this regard, what makes social presence worth investigating in language classes may be connected to its potential impact on learning outcomes. This is because if social presence exists, in other words if learners are ‘there’ and they are ‘real’, a positive relationship may be created leading to positive effect in their learning [22,23].

While possessing many unique features and considering the possible contributions of mobile technologies, it is high time for researchers to focus on more unique studies not using these technologies as self-study supports but investigating their effects on students’ academic performance and social presence with the hope of opening up new spaces without a time and place limit while encouraging communication. By the same token, it is crucial to recognize that unique challenges must be considered when designing and delivering instruction [24]. These challenges encompass understanding and addressing the diverse requirements and needs of students, recognizing the contributions of partners to each other’s improvement, and contextual factors that influence the learning environment [25]. To ensure the effectiveness of these technological advancements in education, thoughtful consideration and adaptation are essential to meet the dynamic demands of learners and foster meaningful and collaborative learning experiences.

The learning environment plays a vital role in shaping the outcomes of the learning process. To create effective learning environments, innovative perspectives are necessary. One crucial feature of such environments is the formation of learning communities that promote student participation and facilitate effective learning [26]. The significance of learning environment design becomes even more apparent when considering its impact on quality, especially in the context of learners not physically present in a classroom [27]. By April 2023, the global count of internet users reached 5.18 billion, constituting 64.6 percent of the worldwide population [17]. Nevertheless, despite the significance of mobile learning applications, limited investigation has been conducted on the factors influencing their adoption [28]. In the current landscape, while mobile technologies have been expanding rapidly, there remains a substantial dearth of research encompassing diverse perspectives, particularly in light of their exponential growth [4,12]. Therefore, the use of technological tools for educational purposes requires further investigation in various learning contexts and with diverse tools. Exploring these technological tools in different settings may enhance our understanding of their potential for supporting effective learning outcomes.

On the whole, keeping in mind how prevalent mobile technologies are and the scarcity of studies regarding younger age groups, one of the significances of this study lies in its exploration of the academic performance and social presence of language learners within learning environments that incorporate a mobile application, aligned with a robust instructional design tailored to the course objectives. To put it another way, instead of using a mobile application as a self-study tool targeting language learning in isolated chunks, this study finds it significant to investigate whether and how mobile tools like Duolingo enrich learning environments facilitating increased academic performance as well as social presence by forming a specific instructional design. The study examines the academic outcomes and social presence of high school students in their initial years as well as their perspectives on using Duolingo in language education in the EFL context. The results of this study are anticipated to have a significant impact on the discussion surrounding the use of mobile technologies in language education, and the effects they may have on academic performance and social presence in the EFL context. Therefore, the questions this study seeks to answer are:

• What are the effects of Duolingo-based activities on students’ academic performance in English classes?
• What are the effects of Duolingo-based activities on students’ social presence in English classes?
• What are students’ opinions regarding Duolingo use in English classes?

2. Theoretical Framework

2.1. Mobile Learning and Mobile Assisted Language Learning

As portable mobile technologies have spread worldwide and the majority of internet usage now occurs on mobile devices [29], so has the understanding that these tools can be used to learn at anytime and anywhere replacing conventional desktop and laptop computers. The increasing number of mobile devices and related technology development have made mobile learning (m-learning) more interesting for educational settings [30,31]. In this respect, how mobile learning is defined needs to be explained. In its simplest form, mobile learning refers to a kind of learning that allows the learner to be independent by using mobile tools [32]. In another definition, mobile learning is expressed as the ability to gain knowledge without time and space limits through any service or facility [33]. The main features mobile learning provides are cited as freedom of choosing time and place to study using a variety of devices like mobile phones and tablets with easy access, as well as flexibility with regard to the individual’s study habits [10,30].

In this regard, mobile assisted language learning is considered to be a subcategory that is of particular interest in language learning, and the advantages are noted as supporting communication between teachers and students, both ways, as well as facilitating certain skills such as speaking and listening [34,35]. Yet, mobile materials like web services and apps are reported to be preferred because of their facilitative role in autonomy in language learning as they give the learners the opportunity to study outside of the classroom [12]. In recent times, there has been an increasing recognition of technology’s role as a pedagogical tool and a valuable resource for language learning, driven by continuous and rapid technological advancements in language education [36]. To make the most of these opportunities, it is crucial for research to prioritize the investigation of technology’s affordances in language learning [37]. Understanding the unique capabilities and potentials that technology offers can enable educators and learners to harness its full benefits for more effective language learning experiences.

2.2. Social Presence

Social presence as a new phrase was conceived by Short, Williams and Christie [38] when they investigated human communication through the media of that time forming a social presence theory. They defined social presence as “degree of salience of the other person in the interaction and the consequent salience of the interpersonal relationships” (p. 65). Gunawardena and Zittle [39] define social presence as “the degree to which a person is perceived as “real” in mediated communication” (p. 8). In time, Garrison, Anderson and Archer [40] defined social presence in a similar manner in their community of inquiry model by focusing on the individual being ‘real’ when presenting themselves to other people (p. 94). Social presence is a long-debated concept and researchers have provided many different definitions of it [41,42,43]. Although there has not been an agreement on the definition of the term social presence among researchers, they agree that it focuses on the degree of an individual’s being “real” and “there”.

Considering many factors involved in forming social presence between users, it may be worth suggesting that perceiving the level of social presence involves both ambient and personal elements, as well as participant characteristics [44].

This is especially important in that if there are problems related to social presence and if there is no social presence, learners may be more frustrated and critical toward the instructor’s efficacy [45]. As aforementioned, the reasons why social presence in the language classroom merits investigation may be related to its potential effects on learning as it may foster good relationships and significantly affect student learning outcomes.

Social presence is commonly recognized as a crucial element that contributes to the establishment of trusting relationships among participants, thereby positively influencing student learning outcomes [23]. On the other hand, the absence of social presence may lead to increased levels of irritation, a more critical perception of the instructor’s competence, and a decrease in affective learning [45]. Specifically, the body of research on facilitating innovative applications for use both within and beyond educational settings is relatively new [46]. This holds especially true in the realm of social presence, as while the social presence survey has found application in studies involving adult learners, a noticeable gap exists in research focusing on high school students. [47,48]. Therefore, further research in this area, similar to this study, could offer valuable insights into the effectiveness of these tools in enhancing language learning, with a specific focus on younger students. Exploring the impact of innovative applications on social presence among high school students can contribute significantly to our understanding of their potential for promoting meaningful language learning experiences in this age group.

2.3. Duolingo

Duolingo is a free language education application that provides users worldwide access through the internet founded by Luis von Ahn and Severin Hacker in 2011 [49]. When users decide on the language they are interested in learning, depending on their preference, they sit a placement test and make their learning objectives clear including the points, rewards and bonuses provided in the application. They can set a daily objective of a particular number of experience points and if they reach it, they are awarded with bonuses. When learners achieve their goal, they are awarded with streaks, showing how long the learner studied consecutively. They are also able to use Duolingo’s digital currency, allowing access to different features of the application. Here, it is worth mentioning that Duolingo is said to include many gamification features with a variety of tasks to complete as well as challenges, badges, achievements and awards it presents, which is expected to lead to a more engaging and enjoyable learning process, and thus has been a focus of interest for many researchers studying MALL [50,51]. There are a variety of themes to introduce lessons which focus on mostly grammar and cultural concepts to teach vocabulary items and drills with little explanation. Spelling, word recognition and translation are some of the exercise types provided. Duolingo offers learning objectives for skills and lessons are separated into each skill. To illustrate, two lessons are on asking questions while three lessons are on family in a linear order. When learners finish their previous task, then they are allowed to access the new tasks. The activities that Duolingo uses vary depending on the course.

Regarding the specific use of Duolingo, the application has been regarded as convenient for English language learning based on digital resources standards [52]. Additionally, the literature supports Duolingo’s effectiveness in enhancing listening skills and vocabulary [53]. The game-like features of Duolingo, such as gaining streaks, have been found to boost students’ motivation [54], potentially contributing to improved performance among the participants in this study. Furthermore, students’ perception of the application’s ease of use aligns with previous research findings in the field [55]. Duolingo is claimed to keep users engaged and more motivated, which is supported by a considerable amount of research [56,57]. Duolingo, being the application of interest in this study, is known for its features facilitating interaction in foreign language education. Loewen and colleagues [18] (2019) identified a relationship between utilizing Duolingo for learning a language and enhanced results on a comprehensive examination evaluating listening, speaking, writing, reading and lexico-grammatical proficiency. Users of Duolingo reported a notably high level of contentment derived from the application’s utilization [55,58]. Several studies highlighted participants’ positive perceptions of Duolingo as a beneficial tool [59]. Although there are plausible implications derived from research in this field, it is fair to acknowledge that the number of studies investigating the use of mobile technologies outside the classroom remains relatively limited, especially when compared to the rapid growth of such technologies [12,60]. This is particularly true for research exploring the impacts of Duolingo and other mobile-assisted language learning (MALL) apps on second language development [18]. More research in this area can shed further light on the potential benefits and challenges of integrating Duolingo and similar apps into language learning practices beyond the traditional classroom setting.

3. Methodology

This study used a mixed method model utilizing both quantitative and qualitative features based on the idea that mixed design research acts as a kind of bridge between qualitative and quantitative design [61].

3.1. The Program

This study was carried out in a private school which has six campuses in Istanbul. The school followed the guidelines of MoNE (Ministry of National Education) as all the other schools regardless of being private or state but formed its own curriculum giving special attention to language proficiency. English was compulsory at all levels and grade 9 students had English 12 h a week. It can be said that English is one of the most important priorities of parents sending their children to a private school in Turkey, so English was taught as a foreign language as a high priority of the school. All the classes were equipped with smart boards, computers and internet connection. The school had its own LMS (learning management system) and students were permitted to use their tablets in class with the approval of their teachers. There were also spare tablets provided for students in case they faced a technical problem. It was compulsory for each student to take the Cambridge KET (Key English Test), which measured students’ performance in four skills at the end of grade nine, unless they presented a valid excuse or their teacher referred the student to a different exam higher or lower than their current level like Cambridge PET (Preliminary English Test) or Fliers depending on their performance.

In-class activities of the school included teaching and practicing new vocabulary words, as well as grammar exercises, in addition to listening, speaking, reading and writing skills. After examining the replies of the students in the needs analysis, an instructional design was planned that included this widely used application, named Duolingo, to be used as part of the English classes. To illustrate, the study’s execution encompassed a specifically formed instructional design, following the steps of needs analysis, instructional design development, implementation and subsequent evaluation, details of which are explained below.

3.2. Participants

The participants were selected using the purposive sampling method. Purposive sampling is employed to enhance alignment between the sample and research goals, thereby bolstering study rigor, data trustworthiness and result credibility [62]. Participants were drawn from a private school where English is obligatory, and both students and teachers have mobile technology access allowing the intervention to be carried out. Given the distinctive challenge of working with younger age groups, which is one of the distinguishing factors of this study, an ethical committee report was applied for and obtained. This included obtaining consent forms from the parents as well due to the inclusion of young age groups who are under 18. The maximum number of students in each class was 18 in the school and the participants of the experimental group were 16 students who were in their first year of high school. The control group also consisted of 16 students from another campus of the school who attended their school as usual and did not know about the intervention. Both groups’ teachers were well-qualified with teaching experience of seven and nine years. All the participants attended the same school for their primary and elementary education. The school had already assigned them to their classes as mixed ability groups according to the results of KET, so the experimental and control groups are not expected to be significantly different from each other.

3.3. Instructional Design

In-class instruction: Before the intervention, the experimental group was given a needs analysis to explore their expectations and design the course accordingly. The intervention was designed to fit in the current syllabus of the school while trying to answer the students’ needs as they appeared in the needs analysis. Hence, Duolingo-based learning activities intended to get students involved in encouraging learning environments while also explicitly focusing on grammar and vocabulary. Students were referred to the activities presented in Duolingo not only during school hours but also twice weekly after school for two hours.

After-class instruction: The Duolingo-based activities were created to be in line with the topics covered in class. The activities were formed in various ways to engage students with language parallel to the topics applied in class be it grammar, reading or vocabulary. The majority of the tasks offered to the students were aimed at enhancing their overall proficiency in English as the students did not mention one specific skill to be focused on but required more fun activities with an opportunity to improve their grammar and vocabulary in the needs analysis.

3.4. Intervention

The intervention aimed to provide a supportive learning experience for the students by integrating Duolingo into their English language instruction. The activities were designed to be aligned with the syllabus and meet the needs and expectations of the students as identified in the needs analysis. The use of Duolingo allowed for a dynamic and interactive learning experience that engaged students with the language and topics covered in class. The intervention began in the third week of the second semester and lasted for eight weeks. Students were requested to use the activities assigned in Duolingo both in and after school hours according to their teacher’s referral.

The participants in the group were provided with support for 8 weeks. The tasks were updated every week, and students had an additional two hours during the school week to work on the Duolingo activities. To give an example, the students were exposed to the text via seeing a film about helping the planet when the topic was protecting the environment, and one of the objectives was to uncover main ideas in their course book. They were then introduced to the intensive reading part while also being introduced to the vocabulary items and target grammar points. To further illustrate, the students were introduced to the topic “environment” and asked to read a text about green life. They were asked some comprehension questions and to deduce the meaning of new vocabulary items like “protect” and “awareness”, as well as the grammar point “should”. So far this seems like a conventional classroom. However, the main difference lied in the fact that students were introduced to the activities via Duolingo and their progress was expected to continue online after class in a supposedly more engaging way after school. The students were encouraged to share their ideas on the application as well as in class through pair-work, group work or discussion. The control group also had similar assignments to do in and after class but without Duolingo.

The teacher encouraged the students to participate more by regularly keeping track of their progress in the tasks and activities. It is worth mentioning here that this intervention required extra-time on the part of the teachers. To convince teachers to join the study, they were offered exemption from the preparation of end-of-year language fair preparations with the approval of the school administration. The teachers were provided with training sessions about the application and its integration in the syllabus by the researcher. The students were already aware that their progress was going to affect their performance grades at the end of the year. Those who successfully participated and completed the tasks in Duolingo would obtain higher grades. The teachers uploaded the course tasks and had a checklist to follow students’ performance while they also required the students to participate in discussions and complete the tasks related to the topics each evening as part of their assignments. There is no exact measurement of how much time each student spent on the exercises as they continued studying after school as well, but the teacher gave encouraging feedback the day after the tasks were given to encourage additional engagement. By including a performance rating for the students for conducting the tasks and following the intervention process, the participation was supported. Figure 1 illustrates a sample of Duolingo use.

3.5. Data Collection

Data were collected from a needs analysis, a standard English test (KET), social presence scale and follow-up interviews.

3.5.1. Needs Analysis

At the start of the second semester, a needs analysis was handed out to students to identify their needs and design the course accordingly by integrating Duolingo. The needs analysis was formed in students’ native language, Turkish, by the researcher and reviewed by another expert in the same field. The needs analysis focused on features such as identifying students’ objectives to study English apart from its being compulsory in all the schools in the country, students’ future goals, the challenges they face in studying English, the areas they would specifically require support in and their additional comments. Examples from the needs analysis can be seen in

Table 1. Example items of the needs analysis.

Item	Question	Indications
1	Number the items you are good at from the strongest to the weakest with 1 to 7	Grammar ( ) Vocabulary ( ) Listening ( ) Speaking ( ) Reading ( ) Writing ( ) Pronunciation ( )
2	What kind of activities help you the most when learning English? Number the activities that help you the most from the most helpful to the least helpful with 1 to 10	individual work, ( ) pair/group work, ( ) reading activities, ( ) school trips, ( ) grammar activities, ( ) vocabulary activities, ( ) listening, ( ) chatting, ( ) pronunciation activities, ( ) watching movies ( )

below.

3.5.2. Exams

KET is a universally accepted standard test by Cambridge University, which was considered a reliable source to determine students’ English performance for each language skill. There are reading, writing, listening and speaking parts in KET. As aforementioned, the students were already given a KET that served as a placement test by the school. The school did not approve of placing the students according to their exam results in levels based on possible problems that might occur so students were assigned to the classes as mixed ability groups. The school kept the results for themselves to check students’ progress individually and also refer the students to the most suitable exam at the end of the year. Therefore, no placement test was administered to the students prior to the start of the intervention. However, since the students are required to take a KET (Key English Test) at the end of the year, both experimental and control groups were given an adapted version of KET to compare their progress after the intervention ended. There are reading and writing, listening and speaking sections in the exam [63]. The outline and the format of KET with sample questions can be seen in

Table 2. Outline and format of KET with sample questions.

Skill	Duration	Purpose	Sample Question
Reading and Writing	1 h	Shows that participants understand simple written information	Read the text and choose the correct answer for each question
Listening	30 min	Shows that participants understand spoken material.	Listen and fill in the gaps according to the text.
Speaking	8–10 min for each pair	Shows that participants can carry out a conversation exchanging questions and answers.	Look at the pictures that show places to shop and talk together to choose the best one.

below.

3.5.3. Social Presence Scale

To determine students’ social presence perceptions, a 5-point Likert social presence scale [42] was used before and after the intervention in both groups. Among the three dimensions the scale had social presence, teaching presence and cognitive presence; this study utilized the social presence dimension that consisted of nine items. The scale was already translated into Turkish for better understanding of the high school students considering their possible English language challenges. Cronbach’s alpha of the translation of the scale was 0.96 and KR-20 was 0.86.

3.5.4. Follow-Up Interviews

In addition to the quantitative data findings, this study further aimed to gain a comprehensive understanding of students’ perceptions of their experiences using Duolingo by conducting semi-structured interviews with eight volunteering students from the experimental group, which consisted of 16 students. With their class teacher present, the students were asked for their willingness to participate in an interview concerning the intervention. Since the participants were under 18 years old, their parents’ consent was also obtained. The interview questions were meticulously crafted and referred to another expert in the field before finalized to elicit the participants’ reflections on their use of Duolingo, both within and beyond school activities as a part of their English lessons. Although the term “social presence” was not directly introduced or explicitly questioned due to the absence of a universally agreed-upon definition among researchers, and considering the relatively younger age group of the participants, the questions aimed to uncover their opinions and perspectives on the implementation of Duolingo in their learning journey. The interviews focused on their experience with Duolingo and whether they found it beneficial and would like to use it again in the following years. The interviews lasted about 10 min for each student and they were recorded. Participants were encouraged to share their thoughts openly and candidly during the interviews to provide valuable insights into their experiences with the educational tool. The outline of the interview questions is:

• Do you think Duolingo is beneficial in your English studies?
• Do you think Duolingo helps you with socialization?
• What do you think about using Duolingo in English classes?

3.6. Data Analysis

As aforementioned, data came from diverse sources, which included a needs analysis, a standard English test (KET), social presence scale and follow-up interviews. Therefore, data were subjected to analyses tailored to their respective types within the study. To analyze the needs analysis, percentages were computed. For the KET exam, skill-specific grading was conducted, which included a distinct evaluation for the speaking component with the involvement of two certified young learners’ examiners. For the social presence scale, t-tests were applied and for the follow-up interviews, the descriptive analysis technique was facilitated. Comprehensive explanations for each analysis and their corresponding outcomes are provided in separate sections below.

4. Results

4.1. Results Regarding Needs Analysis

Since the number of students was not high (n = 16), percentages are calculated neglecting the small number of items. Only the highest percentages for each item are given because of the richness of the items and limited space. For the question that asked about students’ objectives and their future goals, 88.4% of students focused on finding a well-paid job, followed by continuing their education academically with 77.6% and having a wide capacity of vocabulary at 74.2%. For the item that asked students about what they found as the most challenging, 89.2% chose tests in exams and 78.4% learning new vocabulary, and 77.2% answered comprehension questions. In the next item, students were asked about the parts they required more support in and expressed their additional opinions. Students’ responses varied but the most frequently mentioned part was exam preparation including grammar and vocabulary. The students’ emphasis on exams may come from their being frequently assessed through quizzes and common exams at regular intervals and the exam results playing an important role.

4.2. Results Regarding Exams

The first question of this study focused on students’ academic performance in English after using Duolingo. As mentioned before, no placement test was applied to the students at the beginning as the school had already assigned them to their classes as mixed groups and they were not expected to be significantly different. When the adapted version of KET was applied at the end, particular attention was given to the assessment of speaking skill, which is thought to require extra emphasis for consistency. For the results to be reliable, the speaking exams were evaluated by two teachers who were also formally certified young learners speaking exams. Due to the fact that both raters were present for the application of the test at the same time, precisely like they were for the actual KET speaking tests, the interrater reliability was not calculated. As for the writing part, two scorers were assigned. The interrater reliability for the raters was 0.85 at the significance level of 0.001. For the other parts of the test, the responses of students were indicated as they appeared in the answer key provided. The results of the KET scores for each skill after Duolingo for the experimental and control groups can be seen in

Table 3. Results of KET scores for experimental and control groups.

KET	Experimental	Control	Effect Size	Sd (Exp.)	Sd (Cont.)	p
Reading	28.8	29.7	−0.1	4.9	4.9	0.59
Writing	15.2	13.2	0.78	2.2	1.8	0.2
Listening	17.8	14.3	1.01	4.7	3.7	0
Speaking	9.4	7.3	0.82	2.5	2.4	0

below.

As can be seen in Table 3, experimental group students outperformed those of the control group and the results were statistically significant on writing, listening and speaking as revealed by the effect sizes. The reading scores were not significantly different; they were nearly the same.

4.3. Results Regarding Social Presence Scale

The second question was on students’ social presence perceptions before and after using Duolingo. The Shapiro–Wilk test was used to check homogenic distribution and the results revealed that data were distributed normally. This can be seen in the experimental group’s results for social presence pre-test (z = 0.961; p > 0.05) and post-test (z = 0.924; p > 0.05). The results for the control group also revealed normal distribution for social presence pre-test (z = 0.912; p > 0.05); post- test (z = 0.947; p > 0.05).

For the social presence scale, paired samples t-test was used to compare students’ results before and after the intervention in both the experimental and control groups. According to the paired samples t-test conducted to determine the difference between the pre- and post-tests of the experimental group, the results regarding the arithmetic mean were significant with (t = −9.194; p < 0.05). This can be interpreted as after the intervention the experimental group’s social presence level increased and it was significant. The experimental group’s pre- and post-test results can be seen in

Table 4. Experimental Group’s Pre- and Post-Tests.

Points	Groups	N	$\overline{\mathbf{x}}$	ss	$\mathbf{S}{\mathbf{h}}_{\overline{\mathbf{x}}}$	t Test
						t	Sd	p
Social Presence Social Presence	Pre-test	16	25.39	5.08	1.24	−9.194	5.42	0.000
	Post-test	16	38.33	1.75	4.48

below.

As for the control group, a paired samples t-test was conducted to determine the difference between the pre- and post-tests. The results regarding the arithmetic mean were not significant with (t = −1.486; p < 0.05). This can be interpreted as the control group’s social presence level did not increase significantly. The control group’s pre- and post-test results can be seen in

Table 5. Control Group’s Pre- and Post-Test Points.

Points	Groups	N	$\overline{\mathbf{x}}$	ss	$\mathbf{S}{\mathbf{h}}_{\overline{\mathbf{x}}}$	t Test
						t	Sd	p
Social Presence Social Presence	Pre-test	16	23.15	1.30	0.314	−1.486	6.23	0.170
	Post-test	16	25.34	5.73	1.40

below.

4.4. Results Regarding Follow-Up Interviews

For the semi-structured interviews, descriptive analysis was applied. The interviews focused on students’ reflections on using Duolingo and whether they found it beneficial and would like to continue using it in the future. Among the eight volunteering interviewees, for the question that asked whether they found Duolingo beneficial, 75% of the students (n = 6) mentioned that Duolingo was beneficial while for the second question 87.5% of the students (n = 7) found Duolingo helpful for socialization. In the future, 87.5% of the students (n = 7) expressed that they would like to use Duolingo in their classes.

For the open-ended question of the interview, themes were created according to the responses of students. As seen in the table below, the number of codes in some themes is more than the number of responses obtained in the interviews. Instead of names, students are referred with numbers such as S1 (S = student) and S2. The themes created according to the interviews can be seen in

Table 6. Themes Created from the Interviews.

Themes	Frequency
It is entertaining	18
It follows the school program	14
It provides a variety	12
It is easily accessed (including being able to access out of school hours as well as the interface of the application)	12
Parents give consent to go on the internet after school hours easily	11
It is strictly followed by the teacher	11

below.

According to students’ responses, Duolingo’s being beneficial may be attributed to the content of the tasks being similar to what they have studied in class at school while being entertaining, which had the highest frequency in student interviews. This is mentioned by S2 as, “There are lots of sites on the net. Our teacher talks about them. Also, my friends tell me but I get lost in them and feel confused. In Duolingo, I remembered what we were doing in class for example words… I couldn’t study words on my own but when I saw them in Duolingo, I didn’t forget about it and it was fun. I got lots of lingots (digital currency of Duolingo)”. Duolingo’s being entertaining is seen in another quotation in S3 as, “What I liked about studying with this app was it didn’t feel like work or study or homework you know. It was kind of fun”.

For the second most frequently occurring theme following the school program, the students indicated their approval of the activities being in line with the program they were following, and thus contributing to their growth in terms of English language learning. This can be identified in the expressions of S5 as, “I didn’t feel like I had to do anything extra because it already covered what we were doing in class”.

In terms of providing a variety as the third most frequently occurring theme, S7 mentioned, “I could choose and do whatever I liked. For example, I always find it difficult to learn new vocabulary. With the activities provided, I didn’t have to memorize words because we had lots of different exercises and from there I could form new sentences elsewhere”.

For being easily accessed, the students mentioned the accessibility of the app as it did not require them to place extra effort to reach the activities provided there, as well as having a friendly interface. S5 mentioned, “I have uploaded the app on my mobile and all I needed was an internet connection. At school hours, I spent from my internet package and it was still fine but at home it was easier because we had lots of internet in our home internet package”. S6 expressed, “It was like very easy to use this app”.

Moreover, when students mentioned about the activities in Duolingo helping them in terms of easy access, they did not only mention about the application’s interface and studying after school but they also mentioned about being allowed to go on the internet by their parents’ consent. In S4′s expression, this is revealed as, “I don’t normally want to study after school you know. I like going on the internet doing some other stuff like surfing, chatting and gaming but my mum says no. She wants me to do something useful. With Duolingo, I had fun and I was studying for the exams. Mum couldn’t get me”.

For the last theme of being strictly followed by the teacher, students mostly focused on their work being regularly checked by their teacher. Students found this beneficial because they think that the teacher’s monitoring encouraged them to do more studies. To illustrate, S1 mentioned, “I like Ms X (referring to their teacher) and she is really helpful. She always supports us in class as well. I mean, when I couldn’t continue with an exercise or an activity, she explained it again and gave me an easier one and then more difficult one. I knew she wanted me to get a good grade in the exam”.

5. Conclusions

When addressing inquiries about the impacts of Duolingo-based activities on students’ academic performance and their perspectives on its incorporation in English classes, the findings unveiled a favorable influence of using Duolingo. Specifically, the use of Duolingo revealed a positive influence on students’ comprehensive academic performance, encompassing skills such as listening, speaking and writing. Additionally, a majority of students expressed a positive view of Duolingo’s utility and a desire for its integration into their English classes. The positive effects of Duolingo can be supported with many studies in the field [64,65]. The observed improvements in students’ academic performance in writing, listening and speaking can be attributed to various factors, including but not limited to their exposure to a substantial amount of visuals and the effective design of instruction that specifically addresses the requirements related to these skills. Moreover, certain participants expressed a willingness to partake in comparable learning endeavors in the future, as observed by James and Mayer [58] (2019).

According to the results of this study in terms of academic performance, it can be claimed that integrating a tool like Duolingo in English classes with a specific instructional design formed in line with the syllabus may act as an effective agent and support students’ academic performance in listening, speaking and writing, although the positive effects may not be attributed to the impact of Duolingo solely.

Considering the skills in English language education, for improved academic performance in writing, the findings may be interpreted as constructive peer criticism that may be used to re-evaluate both ideas and organization, and may have helped students make substantive adjustments as they had the chance to evaluate and revise the writing of other group members and were able to identify their writing issues, such as improper language use and poor structure thanks to the helpful feedback they received from their peers in collaboration, as well as the teachers. The same applies to performance of the students in listening and speaking. Although listening is usually found to be a challenging skill [66], which is also true for speaking [67], Duolingo has shown a facilitative role in these skills in this study. Similar results are seen in the literature attaining an effective role to the application to improve listening and communication [7,16,38,68]. Practice types such as listening to a text and then transcribing the sentences they heard in the listening text may have been advantageous for the students [5,14]. It should be noted that the application provides students with a microphone for their speaking activities, which may have supported students in communication. Mobile technologies can act as an effective agent to facilitate oral proficiency [39,69]. Results also showed that using constructive comments and timely feedback can improve the standards of student discussion replies, which is in line with many other earlier studies [70,71].

The lingering question revolves around understanding why students exhibited enhanced academic performance in three skills while reading did not display a substantial improvement. It’s important to acknowledge that reading comprehension is a multifaceted task that entails the concurrent integration of several cognitive processes [72]. A potential rationale for the absence of a significant improvement in students’ reading skills could be linked to various factors, including, but not restricted to, the instructional design’s potential inability to stimulate interest and curiosity when it comes to reading texts, as well as the inclusion of activities pertaining to this specific skill.

Moreover, the texts may have not been attractive enough to get the attention of students or the level may have been too high for them, making the texts not understandable. In addition, the students were already assigned to read and prepare a project as part of their extended reading projects, which may have overloaded the homework on the part of the students. Besides, Turkey does not hold a high percentage among the countries with a high number of readers. Students are not already keen on reading in their own language, which may lead to their being indifferent to reading as a skill in another language.

In seeking an answer regarding social presence within social presence theory, Duolingo has shown to be effective in increasing students’ social presence, which may be attributed to Duolingo’s allowing for interaction between people [18]. This resembles another study which highlighted the benefits of Duolingo with EFL students [73]. In the study of De Castro et al. [73], the effects of Duolingo in learning English are investigated and the results suggested a significant effect in facilitating communication in language practice. This is also true for the course design when social presence is concerned as functional communication practices are associated with the clarity of the instructional design [74]. Increased social presence results may also be attributed to students’ being kind of bombarded with feedback and discussions that were carried out both in and after school hours, as well as the activities’ being novel. In a way, students were made to be involved in the topics discussed as part of the activities as a result of which Duolingo may have served as a supplementary tool that encouraged communication. It can also be suggested that the activities involved in Duolingo highlight more interactive and social features of the target language including more authentic cases [75]. This emphasizes how crucial it is to incorporate language learning into effective surroundings so that users can interact and socialize using the language outside formal education settings [32,40]. Furthermore, activities conducted via Duolingo may have supported students by providing opportunities to meet regularly and collaborate with their peers. This is especially significant because social presence is regarded as a crucial component of collaboration within learner communities [40].

When looked at from a wider angle, the number of active internet users being more than 5 million worldwide as of April 2023 [17], new technologies continue spreading worldwide. Naturally, today’s generation differs greatly from the previous ones, which makes questioning educational settings more significant. It is beyond one’s capacity to imagine how technological tools would further develop, and undoubtedly the overall objective of this study is not to advocate for a particular tool. Although the current study has a limited scope, the results indicate the potential of applications like Duolingo for facilitating language learning when accompanied with a sound instructional design. Hence, finding alternatives to integrate new technologies to language classes should be taken more seriously for better language education.

As aforementioned, research has emphasized the significance of examining the educational potential of technology [76] and how new generations are more likely to pay attention to interaction due to their intense contact with technology outside classroom. This is because learning a foreign language is influenced by a variety of factors. In this context, mobile technologies may be considered as effective supplementary tools when learning a language [77]. By designing activities that promote social presence through interaction, learning experience of the students can be enriched. Also, as suggested in the social presence theory, it may be possible to foster a sense of ‘being real’ and ‘being there’ in the language learning environment and aid students in engaging in interpersonal learning [74].

It should also be noted that this study was conducted in a private school setting, where class sizes were already limited to a maximum of 18 students and the restricted class size made it impractical to implement the program in larger groups. The relatively small class sizes in the school might have influenced the dynamics of the learning environment and interactions among students. Hence, while the study provides valuable insights into the perceptions of the participants regarding the use of Duolingo, it is essential to acknowledge that individual experiences and perspectives may vary among a larger and more diverse population through extended intervention durations. When interpreting the findings of this study and applying them to other settings, researchers and educators should be mindful of the potential influence of class size on the outcomes. It is important to avoid excessive claims about the potential contributions of mobile technologies while striving for enhancement.

Overall, this study is of limited scope, and it would not be prudent to extrapolate extensive expectations or broad generalizations. While the findings of this study indicate that mobile devices might improve academic performance and social presence in language education, the true impact of mobile learning programs should be augmented through extended intervention durations, closer integration of technology and curriculum, and a more comprehensive evaluation of advanced skills. There are drawbacks and the results can only be suggestive so it is important to avoid excessive claims about the potential contributions of mobile technologies while striving for enhancement. It should be noted that this study took place in the Turkish EFL context, and so the sample size and duration were rather limited. Conducting the study in different contexts might reveal different results and may eliminate the questions that might appear regarding findings. Future research should consider conducting the study with a higher number of participants over a longer period of time.

On the whole, as in every study, there is a need for more context-specific research to effectively determine whether the findings are generally applicable in a wider context. Using technology in the classroom is definitely not a panacea: However, it can provide new opportunities for the conduct of education [78]. By giving students access to educational materials and resources at any time and place, mobile learning can break down traditional barriers to learning and create new opportunities for students to engage with the subject matter. Additionally, the use of mobile devices can also help to foster collaboration and social presence among students, which can further enhance the learning experience. Hence, it can be concluded that effectively utilizing mobile devices has the potential to transform the educational settings for a better learning experience, especially considering the advantages and potential learning affordances they offer when they are aligned and integrated with a substantial instructional design.

6. Implications

In order to adapt to the changes occurring in the new century, new technologies need to be adopted to reach today’s learners. Integrating a mobile application may be more challenging than it seems as it requires teachers’ doing more work in terms of creating suitable content, as well as checking and monitoring student progress more frequently in and out school hours, which means a heavy workload for teachers. Hence, teachers’ schedule needs to be arranged accordingly before integrating technology. Regular and timely feedback, as well as a performance grade, may increase student participation in integrating technology.