Managing Social Presence in Collaborative Learning with Agent Facilitation

Abstract

As interest in online learning has increased, studies utilizing a social system for the innovation of lecture/learning environments have attracted attention recently. To establish a sustainable social environment in the online learning system, prior research investigated strategies to improve and manage the social presence of collaborators (e.g., students, AI facilitators, etc.) in an online lecture. Nevertheless, the negative effect of social presence was often neglected, which leads to a lack of comprehensiveness in managing social presence in an online lecturing environment. In the study, we intend to investigate the influence of social presence with both positive (student engagement) and negative (information overload) aspects on the learning experience by formulating a structural equation model. To test the model, we implemented an experimental online lecture system for the introductory session of human–computer interaction, and data from 83 participants were collected. The model was analyzed with Partial Least Square Structural Equation Modeling (PLS-SEM). The result shows the social presence of the collaborators influences both student engagement (other learners: β = 0.239, t = 2.187) and information overload (agent facilitator: β = 0.492, t = 6.163; other learners: β = 0.168, t = 1.672). The result also supports that student engagement is influenced by information overload as well (β = −0.490, t = 3.712). These positive and negative factors of social presence influence learning attainment (student engagement: β = 0.183, t = 1.680), satisfaction (student engagement: β = 0.385, t = 3.649; information overload: β = −0.292, t = 2.343), and learning efficacy (student engagement: β = 0.424, t = 2.543). Thus, it corroborates that a change in the level of social presence influences student engagement and information overload; furthermore, it confirms that the effect of changes in social presence is reflected differently depending on learning attainment and experience.

1. Introduction

As interest in online learning has increased, diverse internet lecture programs have appeared. According to Class Central, one of the representative Massive Open Online Course (MOOC) sites in the USA, the number of MOOC courses available in the USA has exceeded 194,000, and online-based learning services are growing rapidly, as seen from the participation of 950 universities [1]. However, regardless of the quantitative growth of internet lecture services, many problems arise concerning the continued use of such services by users and the learning attainment of students in academic circles and work sites [2,3].

The biggest issues among the problems raised are those related to learning sustainability. According to the study performed by Angelino et al. [4], the online learning dropout rate of students is confirmed to be higher than that of off-line lectures by 10 to 20% on average. Many studies explain that the reason for the drop-out rate is attributable to the fact that students’ interest in, and concentration on, the learning and service drop because of the characteristics of online learning, from which direct face-to-face communication is excluded [5,6]. Concerning this, studies on a mechanism that can motivate students to use online lecture services have been performed. The intent is to design a system and analyze changes in the learning experience utilizing social presence, one of the concepts that have recently received attention among the diverse elements of learning motives [7,8]. Social presence can be defined as the salience of another person that one can feel in a virtual space or indirect communication environment [9]. This includes not only humans but also all the objects with which the user can communicate. The level of social presence varies depending on the user’s cognition.

As interest in computer-assisted learning increased after the 2000s, the systems that utilize game elements and communities have been introduced steadily, and the implementation of social characteristics in learning environments has started to be regarded as important [10,11,12]. In addition, as open learning systems such as MOOC attract attention, thus increasing interest in the cognition and utilization of social elements, social presence is emerging again [13,14,15,16]. In the existing studies, social presence is understood to have a big effect on learning satisfaction, learning attitude, and student view activities [16,17]. The interaction between learners that is strengthened by enhancing social presence affects learning satisfaction, which can be represented by the increase in student engagement in learning [8,14,18,19].

Student engagement in learning can be said to be an important factor in evaluating the learning experience using learner interest in task execution in a learning environment [6,20]. Student engagement has been utilized as one of the main constructs that connect social presence and learning experience in the existing studies, and it is believed to have a positive effect on learner experience [21]. Prior research confirmed that the presence level derived from the increase in the number of interactions with teachers and the user community leads to high student engagement [6,22]. Moreover, interactivity and anthropomorphism within e-learning provide a feeling of being together that helps users to concentrate on the learning activity [19,23].

Unlike the studies exploring the positive link between social presence and student engagement, the number of studies exploring the negative effect of social presence is relatively small. Among such negative factors, information overload is emerging as an important factor in the modern online environment where the complexity of information exchange is becoming higher [18,24]. The number of studies relating social presence and information overload is limited, and it is difficult to declare whether the two are directly linked. Prior research addressed the information overload issue as the amount of information and the provision of more choices. Therefore, the strategies involving learning recommendations and reducing the amount of information presentation or the choice complexity were typically explored [18,25]. These studies also point out that the social setting aggravates such a vast quantity of information and choices, yet managing the social aspect itself was not considered in depth. In the case of social presence, construction, setup, and information understanding affect the cognition of social presence, changes in the experience. Accordingly, the results from information overload are required to be considered together [26,27]. This means a vast amount of information is necessary to create the opportunity to provide cues for the users to recognize the saliency of other people. Therefore, it can be predicted that the amount and complexity of information will be high in an environment where the users can sense a high level of social presence.

In past studies on social presence, teaching behavior based on a regular learning frame or analysis of cause and effect through manipulation of the communication between students and teachers, rather than studies on system environment and interaction, was the mainstream [6,8,28]. The number of studies on the deliberations on effective interaction based on learner experience, and the design factors that realize it, is low despite their importance [18,29]. Although diverse opinions on the proper function of social presence are being presented, deliberations on the adverse function that can appear as social presence increases are insufficient. In this study, we intend to organize a social presence-based learning environment using the interaction between the learner and system, to identify the effect of the factors of information overload and student engagement that improve through social presence, and to verify its influence on the experience and attainment of the learner.

In the next section, we discuss the studies performed to this point concerning the above research goals and questions, based on which we develop our hypotheses. Then, the explanation of the research model and hypotheses, the research method, the data analysis, and the discussion will follow accordingly.

2. Related Work

In this section, the prior studies related to the research domain are discussed.

Table 1. Summary for each related work literature domain.

Related Work Domains	Summary
Online Learning Environment	- Online learning utilizing a smart learning environment is prevailing but the drop-out rate is still a problem [3,30]. - As a result, the implementation of teaching guides and social factors is getting more attention [29,31].
Social Presence	- The concept of social presence has been studied to enhance the learning experience [8,32,33]. - In online learning, the social presence of conversational agents and other learners was specially studied to examine teaching presence and cognitive presence [15,19].
Student Engagement	- Student engagement has been studied frequently with social presence in the online learning domain [7,21]. - Student engagement enhanced by social presence influences learning experience and outcomes [8,17].
Information Overload	- The number of research related to the negative effect of social presence is limited [18,34,35]. - Information overload has been studied IT domain but its relation with social presence has not been intensively studied in the online learning field [36,37].
Evaluation of Online Learning Experience	- To investigate the effect of both the positive and negative effects on the learning experience, the representative indexes to evaluate online learning experience were studied. - Based on the prior research learning attainment, user satisfaction, and efficacy were selected [8,38,39].

briefly summarizes the research domain explored and the brief trend explained in each subsection.

2.1. Online Learning Environment

Online learning environments are remote learning environments that use the internet and can be said to be part of e-learning. E-learning can be defined as learning through multimedia format content provided by a digital platform, such as web-based learning, computer-assisted learning, virtual classroom, and more [40]. Nowadays, the term “smart learning” is prevailing as technological dominance has moved to smart devices with high-level interaction, intelligence, and personalization [30]. The smart learning environment includes all the physical aspects related to enhancing the learning experience enriched with digital devices whose aim is to improve and accelerate training [41]. Smart devices such as electronic blackboards, intelligent tutoring systems with a combination of innovative online technologies such as the Internet of things (IoT), and social networking-stimulated smart learning have become widespread [30,42]. The recent COVID-19 pandemic boosted the dispersion and adoption of smart learning to provide learning services in remote conditions, especially in higher education. Consequently, the concerns of industry and academics have moved to developing a technological environment to connect and enrich the learners’ environs with the advantages of conventional learning and smart learning [30,41]. Accordingly, this study addresses research that focuses on online learning as a part of smart learning and uses online learning as a representative term.

As online learning utilizing a smart learning environment has achieved quantitative growth in service with the prevailing development of IT technology, diverse issues have emerged. In particular, students’ dropout rate has been studied as a primary issue [3]. Prior studies reported that the dropout rate of online learners tends to be higher than that of off-line learners by 10 to 20%. Moreover, learning interest and motivation deteriorates because the remote learning environment itself impedes direct interaction between learners [4,18]. In the study performed by Jun [40], the causes of learner dropout in online learning environments were classified into five factors: individual background, motivation, academic integration, social integration, and technological support. Later studies also categorized factors for online learning dropout in a similar manner by focusing on stakeholders and the learning environment [2,43]. In this study, we focus on the absence of motivation and academic integration in this research. The reason for disregarding the other three factors is that those factors cannot be controlled in the online learning system. To investigate the factors’ level and their influence on the learning experience related to dropout, it is important to focus on systemically implementable factors and secure the parsimony of the research.

The motivation factor includes the absence of motivation features or functional failure, which causes insufficient grant of incentive or platform design. It includes extrinsic motivation through an external factor other than learning and motivation by the learner in person, and we can see that outside involvement, including direct rewards, physical punishments, guideline presentation, and others, has an effect on learning attainment and platform use [31]. Academic integration is a factor related to learning and teaching, which includes class interaction and learning information. These are mostly related to the integration of teaching methods and learners. These two factors are closely connected to the service or platform environment and can be cultivated through environmental manipulation, which is suitable to draw and test design features and measure the learning experience. In the existing system-related studies, the factors related to learning motivation and support for teaching that can be manipulated environmentally within the system were employed. Many studies inducing learner motive and interest by directly implementing the social factors in the system have emerged recently as open learning systems attract increasing attention [29]. Based on the above, we aim in this study to design a system which manipulates social presence with the factors of environmental motivation and academic integration.

2.2. Social Presence

In this study, we considered social presence as the conceptual element for the design of a systematic element that controls the cognition of social connection in learning platforms. Social presence is a concept introduced in the study by Short et al. [44] for a socio-psychological analysis in a virtually mediated or remote environment. In the study, it was defined as the degree of the saliency of another person, or the relationship between persons in a virtual space, which varies depending on how much the other person or medium encountered by the user in a virtual space is actually perceived. Social presence received attention later as a major evaluation element that measures online media services in the Computer-Mediated Communication (CMC) field, and it has so far been utilized in diverse fields [8]. The first major case of utilizing social presence in relation to online learning is the study by Gunawardena [45]. In the study, social presence was indicated as the factor that affects the learner in remote learning circumstances. It was also implied that higher social presence and learning experience can be perceived through interaction with the medium, rather than the medium itself. In addition, Gunawardena verified through additional studies that social presence is different from existing social interaction in the sense that the learner (or user) can actually feel social presence, and it can be utilized as a major element across the entire attitude of the learner and learning experience [9].

Later, studies based on social presence in an online learning environment were performed [8,32]. The main research fields are the analysis of the factors that give rise to social presence in a learning environment, the setup of the evaluation system thereof, and changes in the attitude and behavior of the learner that appear through the enhancement of social presence. Short et al. [44] regarded intimacy and immediacy as the major elements that comprise social presence. Intimacy is maintaining friendliness through eye contact, physical activities, and the attitude of the other party [46]; immediacy can be regarded as adjusting the psychological distance between colloquists [47]. Although there is a slight difference, the existing studies were performed essentially based on the above content; the intent was to grasp the cause of social presence and produce it within the study mostly by directly connecting the method of interacting with others within the system (Text-Base vs. Graphic Base), a sense of the real person in the CMC environment, and the organization of the learner team or the construction of a virtual community (Team & Community Work) and off-line-based features (e.g., video chatting or showing the social connection itself) [8,32,48,49,50,51]. The relevant studies implied that it is important to arrange an opportunity for the user to continuously interact with another person or object in a virtual space, which can be also applied to the interaction between the user and the system [33,52].

2.3. Social Presence Manipulation in Online Education

Scollins-Mantha [53] put in order, and recommended, the studies on the elements required for the formation of social presence. The author also introduced auxiliary guidelines, feedback setup, and discussion inducements as matters helpful for the enhancement of social presence. Scollins-Mantha also discussed the necessity for lounges, message boards, discussion bulletins, etc. as design elements for such matters. Recently, the factors related to interlocutors in online learning systems (communication style, anthropomorphism level, interaction modes, etc.) have been receiving more attention as a conversational agent (e.g., chatbot, virtual instructor) was applied to the domain [19,54,55]. The existing studies approached social presence in learning circumstances largely by dividing it into cognitive presence and teaching presence [56]. Cognitive presence has a precondition that the learner should be able to feel a specific significance through interaction within the service, and teaching presence is allowing the learner to positively participate in learning activities through a guideline or scenario. The study by Kirschner et al. [15] asserted similar notions that environmental education property and sociality play important roles in the construction of social presence. They also argued that, in addition to educational property and sociality as environmental elements, the teaching skills of the teacher affect social presence. Furthermore, Kirschner et al. indicated that the cognition direction on social presence can vary depending on the character model of the individual that appears within the system. Based on such research frames, we intend to manipulate social presence within the system by implementing the connection between the system elements related to teaching methods and the learner as well as information presentation features. The previous research mainly focused on either teaching behavior based on a regular learning frame or analysis of cause and effect through manipulation of the communication between students [28]. However, we implemented and manipulated design elements representing each type of social presence to control the level of presence. This research frame distinguishes our research from the previous research works by manipulating the system features representing the content source to establish a social presence environment, without changing the information element of learning content.

2.4. Student Engagement

Engagement means the level of personal importance or interest induced and perceived by a stimulus in specific circumstances [57]. More specifically, student engagement means how much interest the learner has in task execution in a learning environment, and it can be regarded as an important element in evaluating the learning experience [58]. The study by Shernoff et al. [21] evaluated student engagement as a concept similar to the concept of flow and measured student engagement through the degree of interest and fun the students perceived in the learning task and environment and their level of concentration. In addition, in studies on the learning environment and social presence, we can frequently observe cases of utilizing learner engagement as a major element that connects learner behavior and social presence. In the study by Gunter and Kenny [7], it could be confirmed that student engagement was enhanced along with social presence by adjusting multi-tasking opportunities in a learning environment. Moreover, in the study by Kim et al. [17], it could be confirmed that learning attainment varied depending on social interaction and engagement, which were strengthened by social presence. Studies in diverse fields, such as community or broadcasting environments, that confirm the relationship between social presence and engagement in tasks did not appear until recently in diverse fields, and attempts are required to verify this in a more diverse environment [8,59]. In this study, we measure changes in the level of student engagement influenced by social presence based on the above implications. We especially focus on the relationship between social presence and student engagement in individual e-learning environments rather than class environments, which is a difference from previous research.

2.5. Information Overload in Online Learning Platform

Thanks to the development of IT technologies, the traditional desktop-based online learning environment has been expanded into diverse forms, including mobile devices. Correspondingly, the amount of information presented in the learning system has also increased as multiple social tools and social connections have been implemented in the existing environment [18]. Now, users are facing a flood of information as the size of the online learning ecosystem grows, creating the major issue of information overload. According to the study by Thatcher et al. [24], parallelism is regarded as the major cause of information overload in a virtual environment. Prior research also warned that simultaneous dialogues or exchanges of information in a communication environment with multiple modes and screens can lead to information overload [60]. Moreover, mobile devices are prevailing as primary media for online learning. It was reported that users are confused when the information in the existing online environment is introduced into the mobile environment since it limits the opportunity for presenting information and the number of system components required [34,35].

Thus, the situation is changing in the direction of deepening information overload, and the e-learning domain is at the center of it. Recent studies have reported the negative effect of information overload on the learning experience. It was reported that such a phenomenon leads to a decline in learning intention and interaction in an online learning environment [36,37], and the effect is deepened in a new media environment where the complexity of information exchange increases [61]. From this perspective, it can be seen that a more prudent approach is required to constitute social presence concerning information overload. Social presence is about making users feel the presence of another person by presenting information about other persons and their relationships via diverse sensory interfaces [52]. There have been limited numbers of distinct shreds of evidence that support the direct relationship between social presence and information overload. However, based on the assumption that social presence depends on the delivery method and the amount of information carried in the system element, we can assume their relation. In this study, we attempt to verify the direct relationship between social presence and information overload and their influence on the learning experience by experiment testing the above assumption.

2.6. Evaluation of Online Learning Experience

A traditional method of evaluating online learning experience in the existing studies is the measurement of changes in the results of examinations. Junco and Cotten [62] measured the result of online activities by comparing learning attainment through changes in grade, and Jansen et al. [38] measured changes in multi-tasking by students by comparing the examination results and the course completion rate. Furthermore, because good note-taking requires high learning concentration and attention, and it is an activity related to learning comprehension and memory, cases where the quantity and quality of note-taking are utilized as a method of measuring learning attainment can be also found [39]. In addition to attainment, learning satisfaction is utilized as a major construct to evaluate the learning experience. Learning satisfaction is utilized as a criterion for learning experience in the study by Gunawardena and Zittle [9] introduced earlier, and it is utilized in the diverse studies on social presence after that. Learning satisfaction and learning attainment, along with social presence, are evaluated as important elements in understanding and evaluating a learning system, and this matter is required to be discussed continuously [8].

In this study, we intend to draw ideas for the implementation of social presence on the basis of the advantages and disadvantages of the above studies and to deliberate on the influence of the relevant design element in a mobile environment. In particular, we intend to concentrate on a method to obtain a positive effect on the behavior of the learner by strategically utilizing information overload and student engagement in order to effectively deliver the learning experience in a social presence environment. There are many extrinsic factors that affect student motivation other than those stated above, including the culture and habits of the individual, social life, and more [63,64]. However, this research focuses on investigating the universal effect of social presence, which is a factor that can be controlled by the features of the online learning platform. Thus, two representative variables—student engagement and information overload—are selected to pursue our research goal based on the trend of prior research concerning the social presence effect as explained above and to secure research model parsimony. In the next chapter, we explain the detailed direction in which the study is to unfold by setting up a research model with SEM and a hypothesis for each path.

3. Research Model and Hypotheses Setting

In this study, we intend to develop the design elements that can control social presence within a learning platform and verify the effect of social presence on the learning experience and attainment in an experimental setting. In particular, we plan to manipulate the level of social presence and measure the changes in learner experience and learning attainment that appear through student engagement and information overload. By organizing an experiment employing an online lecture environment for adults, a structural equation model to verify the relation among the proposed constructs was set and tested.

3.1. Research Model Formulation

Structural equation modeling (SEM) refers to a family of statistical procedures for testing whether obtained data are consistent with a theoretical model. They are particularly useful when the phenomenon under investigation involves a complex system of interrelationships among variables [65,66]. This study aims to investigate and provide a holistic view of how changes in social presence levels influence users’ learning experience and learning attainment with the mediation of user engagement and information overload. Therefore, the SEM approach was utilized to set up the research model. This model includes two types of social presence levels. First, we plan to implement the system based on the elements of teaching presence explained earlier and the cognitive presence that uses interaction with other people; we also plan to measure the change that results from social presence.

A typical system element related to a teaching presence is the learning assistance system that uses an agent. Recently, the application of conversational agents in the educational sector has increased [67]. These agents have been replacing the roles of the teacher as an instructor, facilitator, and manager, and research has been conducted to maximize their efficacy in sustaining the learning experience by designing socially appropriate interactions [15,68]. Along with this trend, we intended to implement and utilize a lecture assistance agent that can provide real-time feedback resulting from changes in the learner to study how student engagement and the perceived level of information overload vary upon the social presence of the agent and the social factors implemented.

As result, the research model was formulated as in Figure 1. Two social presences, the agent’s presence and the presence of other learners, are linked with student engagement and information overload. Further, information overload is linked to student engagement, verifying the negative influence of information overload derived from social presence. Then, both student engagement and information overload are linked to the three learning output variables: learning attainment, user satisfaction, and efficacy. Thus, the influences of positive and negative factors derived from social presence can be corroborated. In the next subsection, the hypotheses which describe our assumption on each relation among constructs will be discussed.

3.2. Hypotheses Setting

The level of social presence varies depending upon the psychological distance between the service user and the other entities and their behavior [44]. Agent-based services have been utilized as a tool to enhance service reliability and satisfaction by providing an environment more familiar to users in the CMC field [69]. Furthermore, in the online field, we can see studies that utilized an avatar or virtual agent and confirmed its effect on the platform’s reliability and adoption by the learner, which could confirm that learner attitude changes depending on the characteristics and behavior of the agent [50,70]. It is also reported that behaviors such as calling the user’s name or presenting and recommending additional information, as well as emotional expressions such as looks and gestures, helped the users to understand and concentrate on the information provided by the agents [71,72,73]. This kind of mechanism can facilitate social presence as the presentation elements of the virtual agents provide a sense of intimacy and immediacy to the system. In an online learning environment, the guidelines and the expression provided by the learning assistance agent will act as the factors to feel the intimacy and immediacy necessary for the users to follow the learning process and concentrate on the learning content. In this way, social presence will reinforce the functions of the learning system that helps learners to be motivated and to stick to the learning content. Therefore:

H1.1. 

The higher the level of the presence of the learning assistance agent is, the higher student engagement will also be.

As technology evolves, the amount of information that can be carried by a virtual agent has dramatically increased. In other words, as agent utilization increases, the ways for providing information are diversified, and the amount of information grows, thus increasing the possibility for the user to be exposed to the state of information overload. Since the level of social presence depends on the level of saliency created by various types of information, we can assume that a high level of social presence usually includes a vast amount of information [74]. Prior research reported that an increase in the quality of agent representation causes misunderstanding and confusion [63,75,76]. A learning assistance agent with a high level of social presence presents the information in various types of representation elements such as additional graphical posture, gesture, and expression methods. These design elements may cause learners to become exhausted with the design elements that distract the learner’s attention. Accordingly,

H1.2. 

The higher the presence level of the learning assistance agent is, the higher will be the level of information overload that the learner feels.

Apart from the learning assistance agent, the method frequently utilized for the application of social elements in an Internet learning environment is to allow the learner to confirm the relationship with other persons [51,77]. In existing studies, it was found that the user can learn the behavior of another person by referring to, and mimetically adapting it in, a virtual environment [78]. It is stated in the existing studies that such mimetic behavior is utilized as a type of guideline, and it is further strengthened based on the reliability of the other person [79,80].

In this study, we plan to adjust presence by exposing the behavior and information of other people, who are watching the same learning content, to fit the context of the online learning platform, and by adjusting the possibility that the learner can feel a change in the behavior and information of other people connected in the service. The prior research reports that the relationship between users helps to facilitate social interaction by forming a group of people such as an online community [77,81]. In this study, we designed system features which express the behavior and the personal information of other learners studying the same content. In the system, the feature would facilitate a sense of community by creating high saliency for the other learners with various forms of information and expression. As a result, the feeling of distance in the relationship between other persons and the system environment decreases, and the learners will have a better chance to be engaged in the content provided by the learning system. Accordingly,

H2.1. 

The higher the presence levels of other learners rise, the higher the student engagement becomes.

The effect of the social presence of other learners connected in the network on information overload can be explained in a similar way to how the effect of the social presence of learning assistance influences total information. The core information, which determines the saliency of the other learners, is the information about the learner itself. As the amount of information and number of information types increase, the level of social presence also increases. In the case of the presence level of the other learners being high, the influence can be much stronger than in the assistance agent case due to the fact that multiple people can be involved and even stronger based on the size of the network implemented in the system environment [17,51,82]. In this study, the information of other learners includes the current behavior and study history of multiple people, which users can utilize as study guidelines to feel their presence. Users may feel that the amount and the types of presented information are too many due to the network’s size. In other words, the information which is presented to facilitate a high level of social presence can disrupt the user‘s processing of the necessary information, resulting in information overload. Accordingly,

H2.2. 

The higher the presence levels of other learners rise, the higher will be the level of information overload felt by the learner.

Student engagement is based on the learner’s physical time consumed in educational activities and has been utilized as a major criterion for evaluating the quality of education. The higher student engagement is, the more the student concentrates on learning tasks and makes efforts for the promotion of activities related to the learning tasks [18,20]. Because student engagement is important in an online learning environment, we deliberated over the best way of asking the learner for diverse activities, increasing student engagement, and enhancing learning attainment [18,68]. The student engagement that appears through presence has been confirmed to strengthen the link between the learners in the relevant learning environment, as well as increasing online activities [83]. The presence of the learning agent and the relationship between learners mentioned earlier also increases engagement in the system and another person within a learning environment; moreover, such presence will have a positive effect on learning attitude and attainment by increasing learner activities. Accordingly,

H3.1. 

Perceived student engagement has a positive effect on the learning attainment of the learner.

In addition, student engagement is based on the premise that the learner is interested and has fun in the overall use of the service along with the learning activities [21]. The enjoyment and the interest of the user are the core element of evaluating user satisfaction. A prior study also reported that social presence helps mental learning based on the trust and intimacy of the network [84]. In our research design, the intimacy and the trust cultivated with the learning assistance agent and other learners will help focus on the learning with a high-interest level. Thus, high intimacy and trust also increase the general satisfaction level of the learner. Accordingly,

H3.2 

. Perceived student engagement has a positive effect on the learning satisfaction of the learner.

Furthermore, an improvement in student engagement means that the student is more absorbed in the learning activities. As such, an increase in the time and efforts invested by the student appears as a result of utilizing the content of learning in diverse activities, and the efficacy of learning also increases [83]. A high level of student engagement also leads the users to focus on study by repeating the presentation of important information. For this reason, users will be able to fully understand the content and consider future implications, which will also increase learning efficacy. Accordingly,

H3.3. 

Perceived student engagement has a positive effect on the learning efficacy of the learner.

The issue of information overload is also closely related to learner behavior. Information overload is related to user cognition load; information overload appears as a deficiency in attention and leads to the loss of acquired information [85]. The study by Lindsay and Norman [86] explained that such an information-dealing process in humans also continues ceaselessly in the cognitive dimension, and the study confirmed that information overload and cognition load appearing through such a process have a negative effect on learning attainment. In addition, as information overload affects learner attention and memory, it is regarded as a matter that should be considered particularly for teaching behavior and organizing the learning system [37,56,87]. Accordingly,

H4.1. 

Information overload for the learner has a negative effect on the learning attainment of the learner.

Information overload in terms of the existing user is shown to have different effects on the attainment and satisfaction of the user [87]. The study of Agnew and Szykman [88] also showed that there are differences in the results of information overload and user satisfaction depending on the knowledge level of the user on the task. If the quality and quantity of the information exceed the level that can be utilized by the user, it has a negative effect, and in a learning environment, if the amount of information exceeds the level that can be assimilated by students, it is expected to harm learning satisfaction. Accordingly,

H4.2. 

Information overload for the learner has a negative effect on the learning satisfaction of the learner.

Information overload deteriorates comprehension and utilization of the learning content as well as learning satisfaction [36,37]. The negative influence of information overload is closely related to memory. When a person feels information overload, it means the amount of information has overwhelmed the brain’s power to process the information [85]. In this case, the memory process is disrupted, and the person feels difficulty recalling or acting based on certain memories. Then, the user will feel a low level of confidence in the acquired information and strategy to implement it, which will decrease learning efficacy. Accordingly,

H4.3. 

Information overload for the learner has a negative effect on the learning efficacy of the learner.

Another negative of information overload reported in prior research is loss of concentration [85]. When the user senses information overload, the memory becomes corrupted, and it is difficult to process the information properly [89]. Therefore, the user will not be able to search and focus on the information, resulting in a dispersion of attention. In other words, the user will not be able to pay attention to the learning content because of the disruption produced by information overload. Accordingly,

H4.4. 

A level of information overload lowers perceived student engagement for the learner.

4. Research Methodology

As briefly mentioned in the previous section, the research model and hypotheses were planned to be tested by organizing an experiment employing an online lecture environment. In this section, the methodology related to experiment set-up, measurement development, experiment procedures, and analysis approach will be explained.

4.1. Experiment Environment

An experiment was performed by developing a virtual online lecture platform to implement the presence of a lecture agent, as well as other learners. The lecture platform was produced in the form of a mobile web page that uses HTML5 with a display resolution of 1024 × 768. The system set up for exposure to the stimulus was produced using JavaScript, and the basic lecture video was exposed through YouTube API. All experiment stimuli exposed using Apple iPad Mini, and manipulation of the learner system is conducted through touch manipulation of the same device.

The experiment participants (learners) are supposed to conduct online learning activities using the relevant system in the designated space of the experiment. Although the basic lecture time is 30 min, because the participants can execute rewinding, fast forwarding, and pausing during playback of the lecture content, the total learning time varies by 5 to 10 min in some cases, depending on learner manipulation. All the learner behaviors and activities in the platform were recorded in video, for which a camera recorded learner behavior, and the activities in the platform were recorded utilizing the iPad screen capture program. Although a considerable number of existing experiment studies related to the learners were set up in the form of a lecture room where multiple numbers of subjects participated simultaneously to identify the difference between attainment and experience depending on changes in the lecture environment, it is difficult to control the influence of the presence of other people and social elements in the relevant condition of the learner; furthermore, social presence changes quite sensitively depending on such external factors [28,49]. Accordingly, in this study, influences other than those intended were required to be controlled in measuring the effects of the elements of social presence, and the experiment was performed for individual learners in a separate experiment space.

4.1.1. Online-Learning Content and Learning Setting

A class lecture video of an introduction to human–computer interaction (HCI) taught by a university professor was utilized as the lecturing content of this experiment. The length of the lecture clip was modified to fit into an experiment session. In this study, we focused on the higher education setting as the target study platform for practical reasons. Recently the introduction of online lectures has been accelerated in universities due to the COVID-19 pandemic. However, dissatisfactions have arisen one after another despite their quantitative and qualitative growth [36]. In line with such a trend, the experimental environment and content were selected for university students. The reason for the selection of the HCI lecture was due to its accessibility and academic characteristics: it is from a multidisciplinary research domain open to students with various academic backgrounds. An introductory class was selected so that the experiment participants would be able to understand the class without extensive prior experience or knowledge. The features of the online lecture platform were designed by referring to the features of the system suggested by prior research related to higher education. However, we aimed to keep the system and the lecture simplified to maximize the effect of the social presence stimuli.

4.1.2. E-Learning Platform Layout

The overall layout of the learning platform used for the experiment is shown in Figure 2. The main content of the lecture is displayed at the top left, and the information about the learners attending the lecture together is displayed at the bottom. The right side of the screen is an agent area, in which the auxiliary information related to the lecture and feedback for learner activities are displayed. The learners’ information and agent information change based on the point in time; the content of the lecture video and guidelines or information that correspond to the relevant point in time are displayed.

4.1.3. Independent Variable Stimuli

The conditions of the presence of the agent and participating learners were divided into high and low levels, and accordingly, the experimental stimuli were set up with a total of four conditions (2 × 2 design). To adjust the learning effect and individual differences depending on continuous exposure to the stimulus, each subject was exposed to only one type of experiment stimulus. All the stimuli of variables were only exposed visually, mostly through texts or subtitles, in order to prevent confusion with the main lecture content. The effect of manipulating the individual variables was measured through a survey, and the presence of the relevant agent or other learners was checked by asking questions based on the existing studies [90]. To check whether the effects of stimuli were correctly applied, we compared the means of each presence level and conducted an independent sample T-test accordingly. If the perceived social presence level responded to by the participants of the high social presence condition was higher than the level of the low social presence condition, it could be assumed that the stimuli were correctly applied. The T-test was conducted to check the statistical significance of the mean difference. As shown in

Table 2. Manipulation check result.

Agent Present (AP)
Presence Level		N	Mean	Std. Dev	SE	T-Value	p-Value
	Low	37	2.81	1.05	0.173	−2.34	0.022 *
	High	37	3.38	1.037	0.17
Other Leaners’ Present (OP)
Presence Level		N	Mean	Std. Dev	SE	t	p
	Low	38	1.95	1.114	0.181	−2.392	0.019 *
	High	36	2.61	1.271	0.212

Notes. * p < 0.05.; N: number of participants assigned to the condition, Std. Dev: Standard deviation, Mean: average of perceived social presence level (1–5).

, the presence level mean value of the high social presence condition was higher than that of the low social presence condition for both agent presence and other learner presence, and the absolute t-value was higher than 1.99, which is a critical value for t-distribution (two-sided α = 0.05, df = 73).

The agent facilitator played the role of expatiating the insufficient part of the main lecture content or emphasizing the important concepts. The agent provided information feedback in real-time based on the learning content and learner behavior separately from the main lecture at a specific point in time. It also exposed guiding phrases or comments for the lecture based on the content and atmosphere of the lecture to sustain the learner’s concentration. The differential points of the cases with high and low presence were manipulated by the external element of the agent that expressed information. In the case of an agent in the condition of high presence, a 3D-woman character expressed the information in spoken style through subtitles and additionally exposed corresponding gestures and looks. In the condition of low presence, the agent exposed the content of the information in the style of the presentation adopting the form of a blackboard. In other words, in the case where the agent presence is high, the sense of existence is made evident in comparison with a text-based agent by utilizing a 3D-based animation agent. The agent graphic is set in such a way that the features of the virtual character, rather than a real human, are noticeable, and it is implemented with lowered reality. The reason for low agent avatar reality is due to the uncanny valley effect [91]. According to the related literature, the excessive realism expressed by the agent may hurt user experience and interaction with the agent. Figure 3 presents the form of agent representation utilized in each manipulation.

In the relevant functional area, the number of persons attending the lecture at a specific point in the playback, and the changes in the main behaviors of the relevant persons, are informed spontaneously. As major behaviors, those related to learning, such as note-taking, memo, and searching of related sites (sharing of the presented links), and those related to platform utilization, such as video rewinding, fast forwarding, pausing, and ending of learning, are exposed; moreover, detailed information on the activities can be checked by moving the mouse pointer over the icon. For example, when a student performs a search, the searched word is shown, or if notes are taken, the content of major note-taking is indicated. The expression methods were differentiated to distinguish the learner presence levels; in the case of high presence, all the individuals attending the class at present were expressed as icons. This was implemented in such a way that the behavioral information was exposed by the icon, and the sequence and location of the icons changed according to changes in the information and circumstances. On the other hand, in the case of low presence, it was implemented in such a way that the information related to the number of other learners was expressed only through text. The intention is to emphasize the sense of the existence of each learner in a condition with high learner presence by utilizing the expression of each individual and the visual effect using icons. Figure 4 presents the form of design describing other learners’ behavior in the experiment system utilized.

4.2. Measurement of Dependent Variables

In this study, the measurement of dependent variables largely comprises the survey measurements for the overall learning experience and the task evaluation for measuring attainments. The survey included the levels of social presence, student engagement, and information overload perceived while using the learning platform, and learning satisfaction and efficacy were measured as constructs for the final evaluation of the learning experience. The survey measurement was performed after the learning process was completed, and all the questions were organized using a five-point Likert scale. The survey questions were applied by correcting the questions used in the existing related studies to fit this experiment; the questions utilized to measure each construct are listed in Table 3.

In this study, to measure learner attainment, the experiment participants were asked to take notes of important parts of the learning content and solve a subjective quiz on the learning content after closing. The quiz was composed of 12 short answer-type subjective questions to which the answers are the names of the concepts or constructs explained in the lecture. To control the influence depending on differences in the ability of individuals, a free test was conducted using a platform to which no independent variable stimulus was applied to measure the baseline scores and finally record the difference between the scores of this experiment. For the score of note-taking, the following criteria were created based on the examples utilized in the studies of Vessey and Mumford [92]. The evaluation was performed by granting one point whenever the content of note-taking conformed to each criterion (from a total of five points). The evaluation was conducted independently by two evaluators, and the relevant content of note-taking was discussed and re-evaluated when the score difference was large. To check evaluation consistency, an inter-rater reliability analysis was conducted. The measurement was made using the Intraclass Correlation Coefficient (ICC) suggested by Shrout and Fleiss [93]. The ICC is a value between 0 and 1, where values below 0.5 indicate poor, between 0.5 and 0.75 moderate, between 0.75 and 0.9 good, and any value above 0.9 indicates excellent in reliability [94]. The ICC is calculated based on the equation (variance of interest)/(variance of interest + unwanted variance) [95]. In this study, the ICC of two evaluators was calculated resulting in a value of 0.95, which was presumed to have sufficient reliability.

Table 3. Scale items of dependent variables.

Construct	Reference	Measurement
Student Engagement	[21]	1. I felt the learning experience was interesting. 2. I could concentrate on learning content. 3. I enjoyed this learning experience.
Information Overload	[96]	1. I felt the amount of information in the system is too much. 2. I felt difficulty in understanding all the information presented. 3. I suggest decreasing the amount of information presented in the system.
Satisfaction	[97,98]	1. The system delivered the exact information necessary. 2. The system delivered the information clearly. 3. The system presented learning information in a proper time frame.
Efficacy	[99]	1. I felt confident about what I have learned. 2. I can apply what I have learned to other tasks. 3. I can easily recall the content I learned.

Table 3. Scale items of dependent variables.

Construct	Reference	Measurement
Student Engagement	[21]	1. I felt the learning experience was interesting. 2. I could concentrate on learning content. 3. I enjoyed this learning experience.
Information Overload	[96]	1. I felt the amount of information in the system is too much. 2. I felt difficulty in understanding all the information presented. 3. I suggest decreasing the amount of information presented in the system.
Satisfaction	[97,98]	1. The system delivered the exact information necessary. 2. The system delivered the information clearly. 3. The system presented learning information in a proper time frame.
Efficacy	[99]	1. I felt confident about what I have learned. 2. I can apply what I have learned to other tasks. 3. I can easily recall the content I learned.

4.3. Experiment Procedure

The experiment was divided into a free test to measure the standard value of the learner in the situation where the platform is not utilized and the learner is not exposed to the stimulus, and in the main experiment that measures changes in the learner after being exposed to an actual stimulus. The experiment participants prepared a lecture note while attending a 30 min video lecture, and they were asked to solve survey and quiz questions after the lecture ended. A 10 min break was given between the free test and the main experiment, and before and after all the experiments an explanation and interviews on the experiment and lecture system were performed.

In this study, in order to motivate the experiment participants, a cash reward worth $8 (USD) was given; furthermore, an additional monetary reward was provided based on the task-performing record of the subjects. In addition, to control the influence of the personal interest and prior knowledge of the participants on learning attainment, a lecture on a study model recently under development was selected as the learning content; the subjects were asked to participate in the experiment after their knowledge of the learning content had been checked in advance. To exclude the learning effect that results from the experiment platform, all subjects were asked to participate in the experiment for only one condition and were assigned to the condition at random without any advance information provided on the condition to be presented. A proctor was present with the subjects during the experiment to check and correct expected errors during the experiment by recording the experiment.

4.4. SEM Analysis

This study developed a research model based on SEM to investigate the relationships among the variables related to social presence and online learning experience in an experimental setting. To analyze SEM models, two modeling methods, the covariance structure-based modeling method (LISREL) and the partial least squares-based path modeling method (PLS), are often applied. Generally, it has been known that PLS has more advantages in dealing with small-sample data and non-normal distribution [100]. Moreover, the PLS analysis for which the variable setting is free was believed to be more suitable than LISREL. In our model, given that not only survey questions but also an evaluation of learning attainment are included in the measurement variables (the constructs are measured by the score of notetakings and the score of the quiz), PLS would be the better option for analysis. For actual analysis, the statistical application named “Smart PLS (version 3.1.7)” was utilized.

5. Analysis Results

In this section, the results of the analysis are presented. The first subject analysis was conducted to explain the composition of experiment participants. Then, the measurement reliability and validity were tested to check the psychometric property of applied survey measurement items. As a main analysis, paths analyses between the constructs in the proposed SEM were conducted to corroborate the relation among the constructs. Lastly, indirect effect analysis was conducted to check the level of influence of social presence on learning outputs (learning attainment, satisfaction, efficacy).

5.1. Subject Analysis

A total of 83 subjects participated in the experiment, and the data of 74 subjects were analyzed, excluding insincere respondents and outliers. The participants were male and female adults aged between 21 and 39, and those with experience using online lectures were recruited as subjects. Since the lecture presented in the experiment is for those with at least an undergraduate academic background, the recruit limited age to at least 19. We did not limit the upper bound of the age range, but the participants older than 40 did not apply to the results. We assumed that the place of the experiment, which was the university laboratory, caused a problem. The demographic characteristics of the experiment participants are listed in

Table 4. Descriptive statistics of the participants (N = 74).

Participants Info.		Freq.	Ratio (%)
Age	19–24	33	44.6
	25–29	24	32.4
	30–34	12	16.2
	35–39	5	6.8
Gender	Male	36	48.6
	Female	38	51.4
Education	Under Graduate	52	70.3
	Bachelor	15	20.3
	Graduate Student	4	5.4
	Masters or higher	3	4.1

Notes. Age: participants’ age; Gender: participants’ gender; Education: Participants’ current educational background; Freq: number of participants, Ratio: ratio of participants numbers according to the information type.

. The target value of the subject sample size was set using the G*Power3.1.7 program, which refers to the Required Sample Size Calculation Table for regression analysis presented by [101]. As the sample size for the two independent variables of the study model, a sample of 68 persons (Effect Size = 0.15, Alpha Error = 0.05, Power = 0.8) was primarily set, based on which the details of the experiment schedule and process were arranged.

5.2. Measurement Validity and Reliability

Before the research model analysis, question verification was performed through a Confirmatory Factor Analysis. As a result of measuring question reliability, Averages Variance Extracted (AVEs) were all 0.600 or higher, which is statistically significant, and the consistency of the questions was confirmed to have been secured because Cronbach’s Alpha also recorded 0.650 or higher. In addition, each composite reliability was 0.700 or higher, by which we can say that the concentration validity is high [102]. The discriminant validity was measured utilizing the standard of Fornell and Larcker [103], and the squared root of the AVE value of each construct was measured to be higher than the coefficient of correlation with other constructs by 0.1 or higher. The discriminant validity could be also secured because the Cross-Loading for each question was 0.7 or higher in the relevant construct.

5.3. Structural Model Analysis

Verification of each research hypothesis was performed through the calculation of the path coefficient and T-value in the research model, as shown in Figure 5. This is equipped with the explanatory power of the final dependent variable based on learning attainment (R² = 0.031), satisfaction (R² = 0.322), and learning efficacy (R² = 0.157). The Standardised Root-Mean-square Residual (SRMR) standard coefficient for the entire model also shows low model adequacy with 0.69, which is thought to be low because of the utilization of a single question to measure the attainment score. Verification for the hypothesis on each path in the model was performed by calculating each path coefficient and performing statistical verification for each coefficient using the Bootstrap method. The verification content of each hypothesis is ordered in

Table 5. Model path coefficient analysis results.

Hypothesis	Path Coefficient	T-Value	Statistical Significance (95%)
H1.1 PA → ENG	0.115	0.789	Non-Significant
H1.2 PA → IO	0.492 ***	6.163	Significant
H2.1 PO → ENG	0.239 *	2.187	Significant
H2.2 PO → IO	0.168 *	1.692	Marginal
H3.1 ENG → ATT	0.183 *	1.680	Marginal
H3.2 ENG → SAT	0.385 ***	3.649	Significant
H3.3 ENG → EFF	0.424 **	2.543	Significant
H4.1 IO → ATT	0.127	0.843	Non-Significant
H4.2 IO → SAT	−0.292 **	2.343	Significant
H4.3 IO → EFF	0.097	0.646	Non-Significant
H4.4 IO → ENG	−0.490 ***	3.712	Significant

Notes. * p < 0.1, ** p < 0.01, *** p < 0.001.; PA: Presence of Agent, PO: Presence of Other Learner, ENG: Student Engagement, IO: Information Overload, ATT: Learning Attainment, SAT: Satisfaction, EFF: Efficacy.

. Six hypotheses were found to be fully supported (p < 0.05; H1.2, H2.1, H3.2, H3.3, H4.2, and H4.4), two hypotheses were marginally supported (p < 0.1; H2.2 and H3.1), and three hypotheses were not supported (p > 0.1; H1.1, H4.1, and H4.3).

As presented in Table 5, the statistical significance of each path explains the validity of each hypothesis proposed in Section 3.2. The presence level of the agent has a significant influence only on information overload (β = 0.492, t = 6.163). We assume this result is attributable to the characteristics of the content presented by the agent. In the experiment setting, the role of the agent was to provide auxiliary information necessary to understand the main lecture content. Therefore, the presence level of the agent did not help users to engage. This result is consistent with the result of Thatcher et al. [24], which explained parallelism as the major cause of information overload. On the other hand, the presence of other learners has a significant influence on engagement (β = 0.239, t = 2.187) and a marginal influence on information overload (β = 0.168, t = 1.692). This result implies the social cues of others helped the users to be more engaged. As Sabah [51] asserted, a collaborative learning environment enhances educational usage. Similarly, the social cues presented by other learners constituted a collaborative learning-like environment, leading students to be more engaged in the learning content. Regarding the marginal influence on information overload, we believe the agent’s social presence is responsible for the result. According to the manipulation check result, the presence level from the agent (Low: 2.81, High:3.38) perceived by users was higher than the level from the other learner (Low: 1.95, High 2.61). Therefore, it can be inferred that the strong influence of the agent’s social presence diminished the effect of social presence of other learners.

The results in Table 5 show the effect of student engagement and information overload influence on learning outcomes in a different manner. Unlike the student engagement effect, which influences all three outcomes (Attainment: β = 0.183, t = 1.680; Satisfaction: β = 0.385, t = 3.649; Efficacy: β = 0.239, t = 2.187), the information overload affects only user satisfaction (β = −0.292, t = 2.343), rejecting H4.1 and H4.3. This result supports the finding that user engagement is important in online learning, as prior research also claimed [18,83]. Based on the result, we can presume that information overload is concerned with experiences directly related to the learning platform rather than the learning outcome. Its support of H4.4 (β = −0.490, t = 3.172) provides the endorsement for such an assumption.

5.4. Indirect Effect Analysis

To further examine the mediating role of secondary traits in the hierarchy, we compared the sum of the indirect effect of the path independent variable (IV) to the final dependent variable (DV). The indirect effect was evaluated by multiplying the path coefficient of the paths between each IV and DV. Then, we used the sum of all the indirect effects for each IV to compare the total indirect effect. This comparison is conducted disregarding statistical significance to predict only the indirect effects of the IVs on the DVs. The detailed values are in

Table 6. Result of Indirect Effect Analysis.

	Presence of Agent			Presence of Other Learners
	PA-> ENG->	PA-> IO-> ENG->	PA-> IO->	PO-> ENG->	PO-> IO-> ENG->	PO-> IO->
->Attainment	0.021	−0.044	−0.031	0.044	−0.015	−0.010
Sum	−0.054			0.018
->Satisfaction	0.044	−0.093	0.070	0.092	−0.032	0.024
Sum	0.022			0.084
->Efficacy	0.049	−0.102	0.023	0.101	−0.035	0.008
Sum	−0.030			0.074

Note: PA: Presence of Agent, PO: Presence of Other Learner, ENG: Student Engagement, IO: Information Overload.

. The result indicates that the indirect effect of other learners’ social presence on learning outcomes was greater (Attainment: 0.018; Satisfaction 0.084; Efficacy: 0.030) than the effect of agent presence (Attainment: −0.054; Satisfaction 0.022; Efficacy: −0.074). This result implies that the importance of other learners is greater than that of agent facilitators in the online learning setting.

6. Discussion

Social presence in online learning has been evaluated as a factor that motivates learning and has a positive effect on the learning experience and learning attainment [14]. Both interests in the implementation of social factors within a platform and the importance of social presence have also risen as open online education systems attract attention. In addition, in the existing studies on social presence, focus has been placed on invigorating the communication between teachers and students in a general lecture environment, concentrating on the effect of social presence and the factors that cultivate it; on the other hand, discussion on changes in the learning experience that depend on the system environment in a general platform, and manipulating social presence, has been relatively insignificant [28]. In this study, to supplement the restrictions of existing studies, social presence was manipulated by implementing a virtual learning platform as an experimental environment and selecting the key constructs that can be most greatly influenced by this to analyze its influence on the experience of learners and learning attainment. As the key constructs, student engagement that sensitively responds to changes in social presence and information overload expected in accordance with an increase in the information related to presence level were selected. The key findings of this study as seen through the measurement data can be largely divided into two, as discussed in the following subsection.

6.1. Relationship between Social Presence, Student Engagement, and Information Overload

The first key finding is the varying influence of social presence on student engagement and information overload. The result of analyzing the experimental data has shown that an increase in social presence has a positive effect on student engagement, but only the presence of other learners has shown statistically significant results in engagement improvement. The existing studies have reported a finding that, when exposed to the information of other people, the learner elicits interest in, and concentration on, learning while evaluating and comparing the activities of the other person [51,78]. It is believed that the opportunity for the learner to perceive and grasp information on other people increases as social presence rises in the experiment environment of this study, thus producing the effect that results from the above comparison appear well.

However, the presence of the agent has a limited effect on student engagement and is not statistically significant. The reason for this result can be explained based on the characteristics of the learning assistance agent’s design. The agent carries information which supports and elaborates the content of the main lecture. This means the basic amount of information carried in the agent is quite large. Thus, the influences of additional information produced by the social presence element may have been suppressed because the participants were encountering severe information overload. The result of the prior study also supports the assumption that information overload hinders the interactive behavior of service users in the CMC environment [77,87]. Therefore, this result can also be interpreted as a side effect of information overload produced by social presence. The result of the experiment supports that high information overload mitigates the level of student engagement. Moreover, it was reported during the post-hoc interview that many human-like features disrupted the participants’ attention and made them feel irritated.

Concerning information overload, it was confirmed that the influences of both agents and other learners applied. Though there was no existing study that analyzed social presence and information overload connecting them directly, when the attribute of social presence is considered, we can expect that the higher the level of presence, the more the amount of information, and the higher the risk of information overload. Social presence is known to be correlated with the intimacy and immediacy of an object. By granting more opportunities to interact, more information needs to be recognized and processed [44,68,84]. The results of analyzing the research model showed that the influence that presence level has on the level of information overload has been exerted well. However, the statistical significance of the presence of another learner was only marginal. The co-efficient value was also comparatively lower than the presence of the agent. We can assume the reason for this result based on the type of information carried by each stimulus. According to visual attention theory, human beings utilize attention to selectively recognize information based on the purpose of the task. Therefore, it can be predicted that learners can disregard the information which they feel is less important [104]. In the post-hoc interview, the participants reported that they tend to focus on information regarding the learning content rather than checking the behavior and information of the others. This supports the low influence of the presence of another learner.

6.2. Relationship between Learning Platform and Overall Learning Outcomes

Another interesting point in this study is the effect of the student engagement and information overload that the learner experiences while using the learning platform on learning attainment and overall learning experience. First, student engagement is shown to have a positive effect on both learning attainment and experience. This is similar to the result of the existing studies where increases in student engagement promote interaction between the learner in the learning platform and the system and contribute to a positive learning experience. However, in the case of student learning attainment, the statistical difference in the influence of student engagement was shown to be insignificant. The characteristics of the examination asked as an assignment can be regarded as the reason for this. The examination questions were organized in subjective order of briefly describing the main concepts and features, and because quiz questions of such type are closely related to the Working Memory Span of the learner, the difference in the abilities of individuals had a greater effect than student engagement [83,105]. However, it can be confirmed that student engagement has an effect on qualitative improvement in note-taking, and considering the fact that the quality of the learner’s note-taking was found to have a positive effect on the examination score in the existing studies, student engagement is eventually expected to indirectly have a positive effect on learning assessment tests [18,58,84].

When seen in the result of analyzing the experimental data of this study, the only variable with a statistically significant effect on the level of information overload was service satisfaction. The effect of information overload on learning attainment, including note-taking and quiz scores, was not statistically significant, and the effect of the path coefficient was also of an insignificant level. This shows that information overload is more closely related to service experience, rather than learning attainment. Such a result has an important meaning in the information construction of online learning system design. The reason is that, because the amount of learning information does not affect learning attainment and efficacy in an online learning environment, even when diverse system elements are implemented and expressed simultaneously, it has no negative effect on learning itself. This can be utilized as a basis to support the hypothesis that a multi-media lecture with high complexity can be more effective than a video-centered online lecture if the main objective of the service is to improve learning attainment. However, because an increase in the level of information overload could have a negative effect on the satisfaction of the entire service or the platform, prudence is required when selecting and setting up learning information.

6.3. Agent Presence vs. the Other Learners’ Presence

In analyzing the comparison of the indirect effect of the independent variable to each final dependent variable, it is discovered that the utilization of other learners may be a better solution to stimulate the learning experience with social presence than utilizing a virtual agent. According to Table 6, the indirect effect of the presence of other learners is greater than that of the presence of the agent. Moreover, the influence of the agent is negative, while the other learner’s influence is positive. We can assume that the reason for this difference in the indirect effect is due to their influence level on each mediating construct. The agent has a greater influence on information overload, while the other learner has a greater influence on student engagement. As result, the indirect effects of each IV are varied based on the characteristics of student engagement and information overload, which represent the positive and adverse effects of the learning environment. Thus, it is recommended that the implementation of cognitive presence should be considered in higher priority than teaching presence.

7. Implications

Based on the discussion and the analysis results presented, we present several implications for researchers as well as for the industry as follows.

7.1. Implication for Social Presence Research

From a theoretical perspective, we provide a more complete view of the positive and negative effects on the learning experience, which were studied separately. Our study results support our conceptualization that the appraisals of social presence from agent facilitators and other learners are distinct, playing different roles in engagement and information overload perceived by the users to vary online learning outcomes including learning attainment, satisfaction, and efficacy. Consistent with previous research [8,17,21,38] that claimed the importance of students’ engagement and information overload on learning experiences and outcomes, our model supports the finding that those two factors, derived from social presence, are the important antecedent in inducing online learning satisfaction, efficacy, and attainment.

7.2. Strategies for Social Presence Management According to the Learning Outcomes

Our study findings also have notable practical implications for managing the social presence level of social artifacts when developing an online learning system utilizing a smart learning environment. We found that:

• Learning attainment and learning efficacy are mainly influenced by student engagement, which is derived from the social presence of other learners in the system.
• User satisfaction is influenced by both student engagement and information overload.
• Information overload is influenced by the social presence level of both agents and other learners, but the agent’s influence is greater.
• Information overload deteriorates student engagement.
• The indirect influence of other learners’ social presence is greater than that of the agent’s social presence.

These findings can be easily translated into actionable items such as implementing an agent with lower design saliency for minimizing information overload and presenting more social cues from another learner to enhance student engagement. Further, according to the design focus set by the developers, the priority could be set according to the target variable presented in the model. For example, if the system design aims to achieve higher learning attainment and efficacy, the social presence of other learners should be considered in higher priority than the agent’s design saliency. Thus, the capital and social resources of online learning developers and designers can be efficiently managed by setting up guidelines that are prioritized based on their focus on the learning experience leading to more sustainable learning.

8. Limitations and Future Work

This study investigated the influence of social presence with both positive (student engagement) and negative (information overload) aspects on the learning experience by formulating a structural equation model. In particular, we focused on factors that can be controlled within the online learning system features, which were related to social presence. However, prior research also asserted the importance of characteristics of teachers and learners such as gender, age, and cultural backgrounds on social presence in the online learning environment [64,77]. We did not include those factors in our analysis to maintain our research focus and model parsimony, yet more implications will be derived by utilizing more parameters and expanding the model explanation.

Another limitation of this research is the content and lecture setting. We used an introductory lecture on human–computer interaction given by a university professor in the experiment. Therefore, the study could provide implications for higher education settings. However, the lecture content plays a great role in the learning experience, and the different contents of various lecture settings should be tested [106,107]. Thus, we suggest testing the model in different learning settings (e.g., K-12, labor training, etc.) in future research to secure the generalization of the model.

Lastly, the experiment was conducted in a university laboratory. Therefore, the recruitment condition was limited in that age group and educational background mainly reflected the characteristics of university students including undergraduate and graduate students. Furthermore, as the learning experience was analyzed through a one-time experiment, experiments and analyses in the long-term aspect are required to more accurately analyze attainment and grasp changes in behaviors. Based on the above limitations, we propose as a future subject of study a longitudinal study that uses a multimedia learning system in which the diverse design elements of presence are implemented. In addition, as provisions of information on the other learner and agent designated in this study have a strong nature of a scenario in a pre-made experimental environment, more behavioral patterns and feedback are required to be considered in preparation for actual application.

9. Conclusions

In this study, we intended to draw a framework for platform design that can help with learner experience and learning attainment using social presence, a motivation element in the online learning environment. To this end, we grasped the design elements of social presence that can help with the learning experience based on existing studies, created a research model by selecting the elements that affect user experience as social presence increases, and analyzed through an experiment the relationship between each construct and its effect on user experience and attainment. The result showed that the social presence of the collaborators influenced both student engagement (other learners: β = 0.239, t = 2.187) and information overload (agent facilitator: β = 0.492, t = 6.163; other learners: β = 0.168, t = 1.672). The result also supported that student engagement was influenced by information overload (β = −0.490, t = 3.712). These positive and negative factors from social presence influenced learning attainment (student engagement: β = 0.183, t = 1.680), satisfaction (student engagement: β = 0.385, t = 3.649; information overload: β = −0.292, t = 2.343), and learning efficacy (student engagement: β = 0.424, t = 2.543).

Based on the result, we drew a design plan of the relevant element while setting up a virtual learning experiment environment using social presence and analyzed the effect of changes in the social presence of the agent and other learners through the levels of student engagement and information overload on learning attainment and experience. The significance of this study can be found in that, through its findings, it has theoretically expanded existing social presence and, at the same time, has revived a point of discussion on learning system design study that has been decreasing recently. Reviving such discussions on changes in the learning experience depending on the system environment and manipulation of social presence in a general platform separated from the traditional interaction between lecturer and students can also be said to be an advantage of this study.