The Appraisal Principle in Multimedia Learning: Impact of Appraisal Processes, Modality, and Codality

Abstract

This paper presents two experiments examining the influences of media-specific appraisal and attribution on multimedia learning. The first experiment compares four different versions of learning material (text, text with images, animation with text, and animation with audio). Results reveal that the attributed type of appraisal, (i.e., the subjective impression of whether a medium is easy or difficult to learn with) impacts invested mental effort and learning outcomes. Though there was no evidence for the modality effect in the first experiment, we were able to identify it in a second study. We were also able to replicate appraisal and attribution findings from study 1 in study 2: if media appraisal leads to the result that learning with a specific medium is difficult, more mental effort will be invested in information processing. Consequently, learning outcomes are better, and learners are more likely to attribute knowledge acquisition to their own abilities. Outcomes also indicate that the modality effect can be explained by avoidance of split-attention rather than modality-specific information processing in working memory.

1. Introduction

Current research investigates the design of learning material in multimedia learning and animations focusing on the design of learning material based on human cognitive architecture. Information processing theory from a working memory perspective is a major theoretical approach for explaining the benefits of mixed visual/textual or visual/auditive information. Based on the working memory model—as suggested by Baddeley and colleagues [1,2]—several subsequent theories propose approaches on how to use these limited resources by exploiting working memory space appropriately. Cognitive load theory provides a major theoretical framework for designing cognitively adequate learning material (CLT; [3,4,5]). This theory suggests that several different kinds of cognitive resources are required during information processing: intrinsic cognitive load, extraneous cognitive load, and germane cognitive load. Intrinsic cognitive load (ICL) is mainly influenced by the interactivity of elements. That is, elements that have to be kept and processed simultaneously in working memory increase ICL. Extraneous cognitive load (ECL) is mainly affected by the inappropriate design of learning material for instance resulting in a split-attention effect or lost-in-information problems [6,7]. Germane cognitive load (GCL) occupies working memory resources by activating prior knowledge and schemata in order to integrate new information. New information also has to be stored and processed in working memory; hence, cognitive resources are occupied.

Based on cognitive load theory, several approaches to reduce ECL were developed and empirically proven. Mayer and colleagues [8,9,10], for example, suggest using multi-modal instead of uni-modal information presentation. The theory regarding the modality effect [9] states that using auditive information in combination with images contributes to the reduction in cognitive load because working memory resources are used adequately. The working memory model—as suggested by Baddeley [11,12,13]—proposes two separate subsystems for audio and image processing: a phonological loop and a visuospatial sketchpad. By using complementary information represented in both subsystems, working memory resources are used more efficiently. Thus, ECL is expected to be smaller than when using only one sub-symbolic channel (auditory or visual only). Even though empirical evidence for the modality effect is convincing [14], it could not be fully replicated in any case so far [15,16,17]. Rummer et al. [18] assume that this effect is not based on modality-specific information processing in working memory, but that the model by Baddeley addresses codality instead of modality. Consequently, verbal material is processed in the phonological loop; that is, visually presented language information is first processed in the phonological store. In contrast, the modality effect assumes that visual language information is first processed in the visuospatial sketchpad, and only additively presented information is immediately represented in the phonological loop. On the other hand, the visuospatial sketchpad is not used for audio information. Rummer et al. [18] also assume that information entering the phonological loop is preceded by acoustic sensory and visual sensory representations that are not part of the working memory. Therefore, the modality effect can be explained via restricted visual perception. Better learning performance through the use of acoustic sensory information can be explained via a memory-specific type of modality effect, which is caused by an echo of acoustic sensory information. This effect is independent of processing text or text with images. However, the modality effect vanishes if combined acoustic–visuals override working memory, (e.g., with too long and complex texts, with schematic or redundant information [19]).

Another advantage of multimedia learning derives from the complementary dual coding of combined image/verbal information. For instance, Wang suggests that multi-modal data analysis provides vital information enabling the reconciliation of different modalities during problem-solving processes (Wang, 2021). Similar findings are reported by Denton et al. (2015) and Baumgartner et al. (2018). Separate sub-symbolic and interactive processing of multi-code information is theoretically and empirically advantageous over the mono-code information presentation [20]. Empirical evidence for these phenomena is convincing [21], though, mainly cognitive issues in media learning are addressed. Research suggests that cognitive variables directly involved in information processing, analysis of cognitive issues not directly involved in information processing, and non-cognitive parameters, (e.g., meta-cognition, attribution, motivation, etc.), can contribute to understanding multimedia learning.

However, not only instructional design but also self-regulation capacity substantially impacts learning outcomes and academic achievement. This concept is closely interlinked with working memory and can be described as the responsibility learners assume for their learning by setting appropriate goals, monitoring and controlling learning processes, and reflecting.

1.1. Learning with Media: An Appraisal and Attribution Perspective

Our view of the world and our understanding of its underlying mechanisms are often subjective. For instance, appraisal or attribution theories [22,23] describe factors that influence information processing from a subjective perspective.

In media-based learning, appraisal and attribution processes play a central role in information processing. Here, Salomon and colleagues and their research on appraisal and attribution reading television form a major contribution. Salomon [24,25] examined how children’s prior expectations of the ease of learning with text or television interact with the amount of invested mental effort during learning with either of these media and their objective learning outcomes. Results suggest that children expect learning with television to be easier than learning with text. However, learning with printed text led to better learning outcomes than watching a television film containing the same information [26]. Similar results were found concerning multimedia learning environments. For example, Wilson et al. [27] showed that individuals using either print or multimedia material performed equally well. Children also invested more mental effort in processing print material than in television film. This difference in media evaluation seems to be the result of an enculturation process as demonstrated by Bordeaux and Lange [28]. Moreover, these studies revealed that media also affects attribution processes. Children learning successfully with film interpreted their success as a function of the media, whereas participants successfully learning with texts attributed their success to their own effort and abilities.

When media, such as printed texts and television, are exposed to specific attribution and appraisal processes, effects related to multimedia-based learning could also be expected. Here, more research is needed in order to examine possible appraisal and attribution media interaction effects.

1.2. Learning with Text, Pictures and Multimodal Information: The Influence of Appraisal and Attribution

The following research examines the appraisal and attribution of information that is presented via different codes and modalities. We expect that appraisal and attribution of the form of representation play a central role regarding the investment of mental effort and consequently influence the activation of germane cognitive load. Based on the model proposed by Gerjets and Scheiter [29], we propose a modified version of this model (see Figure 1).

The original model suggests that self-regulated learning is influenced by several factors that cause different kinds of cognitive load in the working memory [30]. The activated types of cognitive load substantially impact learning outcomes. Central aspects of the original model are: the design of learning environments, learner conceptions, and information processing that affect each other, (e.g., positive or negative attitudes towards computers that might influence information processing in computer-mediated learning [31,32]). Concerning these aspects, it is necessary to specify the model thereby focusing on two processes that we already discussed: appraisal and attribution. The original model illustrates learners’ expectations as part of their conceptions, and it highlights the influence of these expectations on cognitive processes, (e.g., elaboration processes) and goal development. However, the influence of learners’ expectations needs to be elaborated further. Therefore, we suggest that appraisal and attribution processes do not only influence immediate learning activities, but that they also affect learners’ perception of the learning environment, invested mental effort, cognitive load, and thus, learning outcomes. This is more of an iterative feedback process than a unidirectional relationship. First, learners work with a specific medium—for instance, a textbook—and assume this medium to be difficult to learn with. Hence, learning environments that mainly refer to textbooks as learning resources are expected to be more demanding than other learning environments, such as learning software [33]. Consequently, greater mental effort is invested in processing the information presented in the learning environment. This, in turn, leads to the activation of prior knowledge. Then, active and elaborated information processing activates germane cognitive load. Finally, this process leads to better learning outcomes than an appraisal process that judges a specific medium as easy to learn with. Moreover, this appraisal process is likely to interact with attribution processes [34]. As depicted in the work by Salomon and colleagues [25,35], success in learning outcomes can be attributed in different ways. If learners invest greater mental effort (especially germane cognitive load as highlighted in Figure 1) in processing the content of a medium, they are more likely to attribute successful learning to their own abilities and effort rather than to the medium itself. In contrast, an unfamiliar medium that is estimated to be easy to learn with requires greater mental effort for information processing. This external attribution leads to overgeneralization and appraisal processes that support a generalized and disadvantageous surface information processing strategy concerning these specific media (self-fulfilling prophecy).

In sum, the suggested model assumes that appraisal and attribution processes substantially impact learning with different media. The purpose of this research is to examine the influence of such processes on multimedia learning in comparison to text-based learning (dynamic vs. static representations) as well as the influence of information presented in different modalities (dynamic representations with different modalities).

1.3. Dynamic versus Static Representation: Supplantation in Multimedia Learning

In multimedia learning, media are divided into two groups: static and dynamic media. Dynamic media are characterized by visualization of different information and visualization of the transition between these separate parts of information. This characteristic of dynamic media supports a process known as supplantation [36,37]. Supplantation means that external representations enable learners to conduct mental operations they would not be able to conduct internally and/or without external representations. Consequently, external representations can replace covert mental operations [36,38,39]. Media, as an external representation, presents a functional link between objects that learners cannot construct on their own. Supplantation occurs when learners internalize this link and reconstruct a relationship between internally represented objects. This process does not require greater mental effort but an active cognitive knowledge reconstruction process. The mapping of external and internal representations is connected to learners’ prior knowledge, because learners have to understand the coding system as well as the objects and the relationships between them. Since learners are able to understand the relations between objects, they do not need to know these relations beforehand. For instance, Salomon [37] used different versions of film in order to show the effect of supplantation. In supplantation versions, the operation between two states in the film was presented by zooming in and out (the material was about the overall impression and details of Breughel paintings). In control group versions, these operations were short-circuited by only showing the initial and the final state of an operation. Both of these conditions were compared to an activation condition in which only the initial state of an operation was presented to learners. Then, they were required to activate the appropriate mediating operation from their own mental repertoire. Results revealed that the supplantation version and the activation version led to better results in singling out details of the presented paintings. There was also a strong ATI effect showing that poor scorers on cue-attendance (important for singling out details) and verbal reasoning benefited more from filmic supplantation than did learners with high scores in these aptitudes. A subsequent study using a different task (laying out solid objects and folding them back again)—also comparing supplantation vs. short-circuited film—showed a strong effect proving that filmic codes can be internalized to be used covertly. In a study by Seel and Dörr [40] sets of images integrated into a computer program were used to show different objects as well as their transformations in a supplantation condition. The task was to watch a three-dimensional object, and then, imagine the corresponding orthogonal projection and the other way around. This was compared to an imagery condition without the visualization of the transformation. They found strong evidence for the supplantation hypothesis. That is, showing the operations within both types of visualizations leads to superior task performance compared to simply presenting single object states without showing the transformation process. Similar results are reported by Vogel, Girwidz, and Engel [39] on using a computer program for interpreting data and graphs. In short, the supplantation effect supports the idea of favoring multimedia learning. Learners are more likely to internalize connections between objects and underlying operations if dynamic instructions are provided than they are when learning with static media.

1.4. Open Research Questions and Hypotheses

Research on multimedia learning, for instance by Bremer [41] and Kramer et al. [42], shows that in specific contexts this type of learning is more effective than learning with static media. One basic principle supporting these findings is the modality principle in multimedia learning [9]. Nevertheless, there are also doubts regarding this principle. First, there is criticism from a theoretical perspective arguing that not only auditive verbal information is processed in the phonological loop but also textual verbal information (see above) [43]. Second, the modality effect is not always replicable and seems to be influenced by additional variables, for instance, the complexity and length of the learning material or additional instructional strategies such as pacing [44,45]. Third, it remains unclear how appraisal and attribution processes influence learning with media that differ in modality, codality, and dynamics. Especially, the benefits of dynamics in external representations in supporting supplantation are not or only marginally addressed [46]. Finally, appraisal and attribution towards media processes might additionally influence mental effort and learning outcomes in the media-based learning [47].

In order to examine the influence of these variables on multimedia learning, we conducted two independent studies. The first study examined the influence of different representations (modality and codality) and appraisal processes on knowledge acquisition and cognitive load. Therefore, we created four learning scenarios on the eutrophication of lakes (text-only, text with images from an animation, animation with spoken text, and animation with written text). We presupposed text-based learning material to be estimated as being more difficult to learn with than computer-based animations. Therefore, we also expected greater invested mental effort in this condition. Following a supplantation assumption, we anticipated dynamic learning material (animations) to be superior to text-based static information presentation. If the modality effect is in operation here, in study 1 the animation with pictures and auditive verbal information would be superior to the one with textual verbal information, text, or text with images. That is, knowledge acquisition would be greater and cognitive load would be reduced. The second study examined the modality effect via using complex and extended learning material (routing algorithms in network data exchange). Here, learners were assigned either a text-only version, a text with images from an animation, an animation with spoken text, or an animation with fields of written text. Both animations covered the same content and were of the same length (5 min).

2. Material and Method

2.1. Experiment 1

2.1.1. Participants

Eighty-six participants (69 female, 17 male, mean age = 19.85; SD = 5.58) took part in this study voluntarily. Forty-three participants were university students and received a study relevant certificate for participation. Forty-three participants were high school students in their final year. Participants were randomly assigned to one of four conditions (a text-only version, a text with images, an animation with textual verbal information, and an animation with auditive verbal information).

2.1.2. Material

Learning material. The learning material used in this study covered the topic “eutrophication of lakes”, presenting seasonal cycles over five years with different parameters and their development over time. The topic was presented in the form of four different conditions: a text-only version, a text version with images from an animation, an animation with spoken text, and an animation with written text fields. All versions covered the same content which was presented in different codes or modalities (see Figure 2). The length of the two animations used was 17 min and 19 s. Text and images were designed to be complementary to the text explaining the situation and its underlying parameters. In the text-only condition, the images were removed but the text was kept coherent. Participants were assigned one of the four conditions and worked with their respective learning material.

Knowledge acquisition. In order to measure individual learning success, a knowledge pre- and post-test were conducted. They included an 18-item multiple-choice test measuring learners’ knowledge and understanding of the topic “eutrophication of lakes”.

Cognitive load. The NASA Task Load Index (NASA-TLX; [48]) was used to assess cognitive load. This index evaluates the mental workload of the situation in which a person is involved. The instrument consists of six bipolar sub-scales (mental demand, temporal demand, performance, effort, frustration level, and physical demand). For this study, we reduced the original version to a five-point Likert scale. Its internal consistency after the exclusion of one item was 0.67 (Cronbach’s alpha). In addition, the mental effort rating scale (MERS; [49]) was applied.

Appraisal and Attribution. A questionnaire adapted from Salomon [24] was used to measure media-specific attribution. The perception of realism attributed to each medium and the causal attributions of failure/success to different media were also rated. A four-item long five-point Likert scale was used to assess how easy or difficult learning with computers and/or text was estimated to be. The scale also served to determine participants’ appraisal of the learning material presented within their respective experimental conditions (Cronbach’s alpha = 0.52). Three other sub-scales were employed to determine the locus of attribution when being successful in learning with a certain type of media. Four items assessed learners’ ability (internal, stable attribution; Cronbach’s alpha = 0.58), four items measured their effort in information processing (internal, unstable attribution; Cronbach’s alpha = 0.58), and four items determined how well information was designed within each medium (external, stable attribution; Cronbach’s alpha = 0.43).

Design and Procedure. First, participants gave informed consent, then they filled in a questionnaire assessing socio-demographic data (age, sex, major of studies). Next, they were informed about the procedure of the experiment. Then, they filled in a pre-test assessing knowledge and a questionnaire on media-specific appraisal and attribution. Following the pre-test, participants were randomly assigned to one of the four conditions. The treatment time was constant taking 20 min. Finally, participants filled in a post-test measuring knowledge acquisition and cognitive load. Overall, participation took about 50 min.

2.1.3. Results Experiment 1

For analyzing the results between all four conditions, a MANCOVA was computed. Here, the condition itself served as an independent variable, prior knowledge as assessed in the pre-test as a covariate, and performance in the knowledge post-test as well as the cognitive load measures as dependent variables. The results revealed a one-sided overall effect of the independent variable (F (3, 81) = 2.56), p = 0.03; η² = 0.08) but not of the covariate (F (3, 79) = 1.77), p = 0.16; η² = 0.063). Analysis of between-subject effects showed no significant differences of the different conditions on cognitive load measures (NASA-TLX: F (3, 81) = 0.73), p = 0.54; η² = 0.03; MERS: (F (3, 81) = 0.27), p = 0.85; η² = 0.01) but a one-sided effect on performance in knowledge post-test (F (3, 81) = 2.55), p = 0.031; η² = 0.09). Pairwise comparisons revealed that the animation with audio led to significantly better performance than the text-only (p = 0.02) and the text with images conditions (p = 0.02). However, the animation including audio text did not outperform the animation with written text (p = 0.31). There were no significant differences between the two text conditions (p = 0.97), the animation with written text as well as the text only (p = 0.19), and the text with images conditions (p = 0.19). Descriptive results reveal that the animation with spoken text produced the highest performance in the knowledge post-test, followed by animation with written text, and text-only. The lowest performance was found in the condition with text and images (see

Table 1. Descriptive data of study 1 in group comparisons.

Measure	Text Only		Text with Images		Animation with Written Text		Animation with Spoken Text
	M	SD	M	SD	M	SD	M	SD
Knowledge pre-test	5.33	2.78	4.32	2.17	4.95	2.31	4.11	2.56
Knowledge post-test	9.08	2.23	8.86	2.36	9.90	2.30	10.47	2.25
NASA-TLX	2.95	0.73	2.66	0.67	2.86	0.89	2.86	0.74
Mental Effort Rating Scale	3.25	1.11	3.46	1.01	3.19	1.57	3.37	1.07

). The highest self-reported cognitive load occurred in the text-only condition, followed by both animation conditions. The lowest self-reported cognitive load was found in the condition with text and images (see Table 1).

Linear regression analyses were computed, in order to analyze the effects of appraisal processes on the different learning environments. These analyses were conducted separately for participants of the text and animation conditions. The results of the knowledge post-test were used as dependent variables in the regression model. In contrast, results of the knowledge pre-test and the four sub-scales from the appraisal/attribution scale were used as independent variables. Results of the text conditions revealed that only appraisal of the difficulty of a medium is a significant predictor (t = 2.24, p = 0.03); on the other hand, prior knowledge is only marginally influential (t = 1.95; p = 0.058). This leads to the significant overall model (F (2, 43) = 4.03, p = 0.03; R² = 0.16) which can be described as follows:

Performance in knowledge post-test = 0.98 + 2.26 × Learning with text is difficult + 0.33 × Performance in knowledge pre-test

(1)

The results of the animation conditions revealed—similar to the text conditions—that appraisal of the difficulty of the medium is a significant predictor (t = 1.99, p = 0.05). In this model, prior knowledge is influential (t = 2.15; p = 0.038). This leads to the significant overall model (F (2, 37) = 5.57, p = 0.008; R² = 0.23) that can be described as follows:

Performance in knowledge post-test = 4.71 + 2.03 × Learning with computers is difficult + 0.33 × Performance in knowledge pre-test

(2)

A second ANOVA with repeated measurements was conducted in order to test general differences in appraisal and attribution of learning with text vs. learning with computers. Here, the four appraisal and attribution sub-scales for judging text and computers separately were used. Results revealed an overall effect of the model (F (7, 76) = 7.46, p < 0.001; η² = 0.41). Test of within-subjects contrasts showed no significant differences neither concerning internal stable attribution (ability in learning with text or computers; F (1, 83) = 3.76, p = 0.06; η² = 0.04) nor regarding effort in processing information from text or computers (internal, unstable attribution; F (1, 83) = 0.04, p = 0.96; η² < 0.001). Significant differences were found relating to judgments of how well information is designed within a specific medium (external, stable attribution; F (1, 83) = 12.13, p = 0.001; η² < 0.13) and the appraisal of difficulty between text and computer-mediated learning material (F (1, 83) = 23.97, p < 0.001; η² < 0.23). Descriptive results revealed that learning with text is generally regarded to be more difficult than learning with computer-mediated media (see

Table 2. Descriptive data of study 1 in media comparison.

Measure: Learning Is Successful Due to	Learning with Text		Learning with Computers
	M	SD	M	SD
High ability in learning (internal, stable attribution)	3.81	0.81	3.62	0.80
High effort in information processing (internal, instable attribution	3.66	0.88	3.54	0.78
Good information design (external, stable attribution)	3.52	0.75	3.72	0.75
Learning with this medium is difficult (appraisal)	3.24	0.47	3.20	0.51

). In addition, computer-mediated learning material is generally assumed to have a better information design than print learning material. Mean values also indicate that success in learning from text is predominantly attributed to ability and effort whereas learning with computer-mediated learning material is not.

2.1.4. Discussion of Experiment 1

Results do not confirm the modality effect in multimedia learning. Although animation combined with audio produced the highest learning outcomes, several facts provide an alternative explanation. First, both animation conditions (text and audio) led to high performance in the knowledge post-test. Thus, it seems not to be an effect of modality but rather an effect of dynamic presentation. The learning material used for this study showed different states and parameters of a lake that is undergoing eutrophication. Dynamic visualizations were able to present transitions between these different states while static text-based learning material was not able to depict these changes. Therefore, dynamic multimedia learning material was able to supplant these transitions—regardless of using either text or audio animation. Concerning the two print versions of the learning material, the condition without images led to better performance in the knowledge post-test than the condition including images. This outcome was unexpected since different models of text and picture comprehension all indicate that the opposite is the case. However, these findings can be explained via invested mental effort, since the self-reported cognitive load was significantly higher in the text condition. For instance, participants had to re-construct different states, their relations, and operations from the learning material, consequently leading to deeper elaboration processes and increased knowledge acquisition. Concerning cognitive load theory, animations led to decreased intrinsic cognitive load by reducing the number of elements that have to be memorized and worked with simultaneously in working memory. This was achieved by providing different states of the lake system, underlying operations, and transitions. Concerning both text conditions, these operations had to be conducted by the learners themselves. Participants in the text with image condition did not draw these inferences; however, participants in the text-only condition had to make these inferences in order to be able to understand the learning material. Thus, they actively used free working memory resources thereby increasing usage of available germane cognitive load.

The appraisal hypothesis was confirmed. Results of both regression analyses revealed that the influence of media-specific assumptions has a greater impact on learning outcomes than prior knowledge. This outcome supports prior findings [24,25] claiming that the attitude towards specific media directly or indirectly influences the way of information processing either on a surface level or in an elaborated manner. Further evidence is provided by results from the attribution scales. These results reveal that learning with text is generally regarded as more difficult than learning with computer-supported media. Differences in appraisal and attitude towards the design of media support the idea that media-specific perceptions influence information processing strategies. Generally, participants estimated that information design is better in computer-mediated learning environments than in print material. If the design of the material is better, less mental effort is necessary to process it. Consequently, successful learning with a badly designed medium, which is estimated to be more difficult to process, is more likely to be attributed to ability and effort. In addition, these perceptions and attitudes neither lead to deeper elaborations nor do they foster knowledge acquisition. However, deeper elaborations and knowledge acquisition can be fostered by additional instructional strategies.

The most important outcomes of this experiment are: that the modality effect does not occur automatically and learning with multimedia (either text-based animations or narrated audio) has the potential to support cognitive processes. The dynamics of the presentation showed greater effects than reducing cognitive load by using multi-modal learning resources. However, the impact of media-specific appraisal and attribution on learning outcomes and the learning process is of interest. Moreover, there is evidence that appraisal directly influences knowledge acquisition and consequently attribution processes related to these learning outcomes. Our first study compared text and animations. In addition, a subsequent study replicating and extending these findings was designed. The second study aimed at directly measuring the influence of appraisal processes on learning outcomes within one medium. Therefore, participants were divided into two groups: learners that assumed computer-based learning material to be easier to work with and learners that assumed this type of material to be more difficult to learn with. An inference statistical procedure tested the following hypothesis: the more difficult the learning material is judged in media appraisal, the more mental effort is invested in learning with this material, the higher is subsequent knowledge acquisition, and the more learners attribute learning performance to their own abilities. Hence, the effect of different modalities in multimedia learning on attribution outcomes was examined. Another major aim of the second study was to re-visit the modality effect. While in most studies examining this effect the learning material was not as complex as in the animation used in this study, animation from a simpler domain with fewer complex parameters was developed. In addition, working with the learning material in the first experiment took about 50 min. Thus, interference processes between long-term memory and working memory are likely. Consequently, a shorter animation might be able to replicate the modality effect. Additionally, in the first experiment text fields changed their position within the animation with text. Each text field was located at the place of the corresponding image that the information referred to. Following the assumption that the modality effect can be explained by the split-attention effect [18], locating text fields continuously at the bottom of an animation might contribute to this effect. Finally, an interaction effect is expected here. An audio narrated animation leads to a reduction in cognitive load, compared to a text-based animation. In turn, a reduction in cognitive load leads to free working memory capacity being available for elaborated information processing, which is the case in the condition where learning with computers is judged to be more difficult.

2.2. Experiment 2

2.2.1. Participants

One-hundred and two participants (all male, mean age = 19.18; SD = 0.96) volunteered for this study. All participants were recruits of the Austrian Federal Army. Participants were randomly assigned to one of two conditions (an animation with textual verbal information with N = 45, and an animation with auditive verbal information with N = 57).

2.2.2. Material

Learning material. All participants had to work with the learning material assigned to their condition. The topic was routing algorithms in network data exchange. Two different versions of the same content were developed: an animation with spoken text, and an animation with written text. Both versions had the same content and were 5 min long.

Knowledge acquisition. In order to measure individual learning success, a knowledge pre- and post-test was conducted. It included a 12-item test with seven multiple-choice and five open questions measuring learner’s knowledge and understanding of the subject “network data exchange and routing algorithms”.

Cognitive load. The amount of invested mental effort, which is nearly identical to the mental effort rating scale (MERS; [49]), was applied for the purpose of assessing cognitive load.

Appraisal and attribution. A modified version of the questionnaire that was used in experiment 1 was applied to evaluate media-specific attribution. This modified version served to measure the perception of realism attributed to each medium and to identify causal attributions of failure/success to different media directly and separately for learning with text or computers. Two items on a five-point Likert scale evaluated how easy or difficult learning with computers was estimated to be in comparison to learning with text and to determine participants’ appraisal of the learning material presented within their experimental conditions (Cronbach’s alpha = 0.9). Three other items were used to identify the locus of attribution when being successful in learning with computers (learners’ ability, mental effort, and information design). The overall scale had an internal consistency of Cronbach’s alpha = 0.87.

2.2.3. Design and Procedure

First, participants gave informed consent followed by a questionnaire assessing socio-demographic data (age, sex). Then, they were informed about the procedure of the experiment. Next, the pre-test assessing knowledge and the questionnaire on media-specific appraisal were applied. After the pre-test, participants were randomly assigned to one of the two conditions. The treatment time was constant at 5 min. Finally, the post-test measuring knowledge acquisition, attribution, and cognitive load were applied. Overall, participation took about 35 min.

2.2.4. Results Experiment 2

First, the mean value of the two items on whether learning with computers is easy or difficult was calculated and a median split was conducted (median = 3.00; mean = 3.38). Participants with the same value as the median were excluded from further analyses (see

Table 3. Descriptive data of experiment 2.

	Appraisal Condition
Modality Condition	Learning with Computers Is Easy	Learning with Computers Is Difficult
Animation with text	13	21
Animation with audio	14	26

). Next, a MANCOVA was computed with the condition modality (text animation vs. audio animation) itself as an independent variable and appraisal as a second independent variable. Prior knowledge served as a covariate, whereas performance in the knowledge post-test, cognitive load, and the media-specific attribution items were dependent variables. The results reveal an overall effect of the independent variable “modality” (F (5, 65) = 2.38), p = 0.048; η² = 0.16) and of the independent variable “appraisal” (F (5, 65) = 5.40), p < 0.001; η² = 0.29), and of the covariate (F (5, 65) = 13.15), p < 0.001; η² = 0.50). There was no significant interaction effect (F (5, 65) = 0.71), p = 0.62; η² = 0.05).

Analysis of between subject effects showed significant differences concerning the condition “modality” in the post-test (F (1, 69) = 5.57, p = 0.02; η² = 0.08) and cognitive load, (i.e., invested mental effort) in the attribution scale (F (1, 69) = 5.65, p = 0.02; η² = 0.08). There was a marginal effect on the judgment of information design (F (1, 69) = 3.96, p = 0.05; η² = 0.05) and no effect on the ability item (F (1, 69) = 2.54, p = 0.12; η² = 0.04). Descriptive results revealed that both groups increased their knowledge test performance from pre- to post-test (see

Table 4. Descriptive data of study 2 in group comparison.

	Modality Condition				Appraisal Condition
Measure	Animation with Text		Animation with Audio		Learning with Computers Is Difficult (Compared with Text)		Learning with Computers Is Easy (Compared with Text)
	M	SD	M	SD	M	SD	M	SD
Knowledge pre-test	7.53	3.67	9.68	4.55	8.44	4.59	8.83	4.13
Knowledge post-test	14.65	4.21	16.33	3.67	16.18	4.81	14.80	3.69
Ability	2.79	1.38	2.38	0.94	3.07	1.30	2.11	0.05
Amount of invested mental effort (Cognitive Load)	3.08	1.34	2.46	0.86	3.12	1.28	2.43	0.99
Information design	2.93	1.36	2.39	0.92	3.01	1.24	2.31	1.04

). The group with the audio animation performed better in the knowledge post-test than the group with the text animation. However, cognitive load was higher in the condition with text. Here, participants also reported higher invested mental effort.

Analysis of between-subject effects showed significant differences of the condition “appraisal” on post-test performance in the knowledge test (F (1, 69) = 4.04, p = 0.04; η² = 0.06), on cognitive load as measured with the amount of invested mental effort in the attribution scale (F (1, 69) = 7.47, p = 0.008; η² = 0.10), on the judgment of information design (F (1, 69) = 7.15, p = 0.009; η² = 0.10), and on the ability item (F (1, 69) = 14.96, p < 0.001; η² = 0.18). Descriptive results reveal that both groups increased their results in knowledge test performance from pre- to post-test (see Table 4). The group that estimated learning with computers to be easier than learning with text performed worse in the knowledge post-test than the group that estimated it to be more difficult. Cognitive load was lower, but the reported amount of invested mental effort was higher in the condition that estimated learning with computers to be difficult. Information design with computers is judged to be better than with text compared to the other condition. Beliefs in their own abilities are higher in the condition where learning with computers is judged as easier than learning with text.

The covariate “prior knowledge”, as assessed with the knowledge pre-test, had significant influence on post-test performance in the knowledge test (F (1, 69) = 60.07, p < 0.001; η² = 0.47), and on the ability item (F (1, 69) = 5.91, p < 0.018; η² = 0.08). There was no significant influence on the amount of invested mental effort (cognitive load) in the attribution scale (F (1, 69) = 3.56, p = 0.06; η² = 0.05) and on the judgment of information design (F (1, 69) = 0.42, p = 0.52; η² = 0.01).

2.2.5. Discussion of Experiment 2

The results of experiment 2 replicate findings from experiment 1 and extend them. Instead of comparing text and animations, in this experiment, two different animations (with text and with narrated audio) were compared. Outcomes reveal that the animation with audio leads to better performance in knowledge acquisition measures than the animation with screen text. Interestingly, the audio group also reported lower cognitive load as measured with the MERS and the amount of invested mental effort. This supports the modality principle in multimedia learning. Nevertheless, it remains questionable if this effect is really caused by using auditory and visual information processing simultaneously thus reducing extraneous cognitive load. In contrast to experiment 1, we used an animation that was less complex regarding the number of elements that have to be processed simultaneously. Moreover, the treatment time was shorter. Finally, screen text fields were constantly displayed at the bottom of the screen and not, like in experiment 1, directly at the corresponding place within the graphics. In sum, these findings can be explained by the modality effect but also by the split-attention effect. Learners with the text condition had to continuously match the text (at the bottom of the screen) with the graphical information. In terms of CLT, this might result in increased extraneous cognitive load. The difference to existing explanations is that the modality effect does not result from a decrease in extraneous cognitive load by using audio and graphics simultaneously. Instead, the modality effect occurs because of an increase in this kind of cognitive load due to dividing the locus of attention into two parts of the computer screen. This might also explain why this effect was not replicable in experiment 1. Nevertheless, the data does not finally prove these considerations but indicates that further research is needed to clarify the basic causes for the modality effect.

A central finding of experiment 2 that supports the model suggested in this work is the influence of appraisal processes and attribution on learning with different media. Learners who estimated learning with computers to be difficult performed significantly better in the knowledge post-test than learners judging this kind of learning to be easy. Hence, appraisal directly influences knowledge acquisition and attribution processes related to these learning outcomes. We assumed that the more difficult learning material is judged in media appraisal, the more mental effort is invested, the higher is subsequent knowledge acquisition, and the more likely learning performance is attributed to learners’ abilities. Our results support these assumptions. In addition to the differences in knowledge acquisition, learners that estimated learning with computer-based learning material to be difficult reported greater invested mental effort. They were also more likely to attribute their learning success to their own abilities. This can be explained by the fact, that these learners experienced only their mental effort and abilities to be predictors of their progress (internal attribution strategies). Learners judging this kind of learning to be easier preferred an external attribution strategy. Unfortunately, this was not supported by the results from the question on the quality of the information design. Learners in the condition “Learning with Computer-based Learning Material is easy” also estimated that the quality of information design is worse than learners in the other condition. Two considerations explain this effect. First, the overall experienced quality of information design might not directly influence mental effort and, thus, learning success in a supportive manner when attributing learning success internally. Second, if learning with computers is judged to be easy and learning success is attributed externally, the quality of information design is also used for external attribution purposes in a causal relationship. That is, learners judged learning with computers to be easy, but after the treatment, they realized that their performance in the post-test was not as successful as expected. Thus, an external attribution style contributes to compensating for this experience.

The missing interaction effect indicates that information processing under a modality effect perspective is a more separate process than information processing under an appraisal and attribution perspective. We assumed that with an audio narrated animation and, thus, reduced cognitive load compared to a text-based animation, there would be greater working memory capacity available for elaborate information processing in the condition where learning with computers is judged to be more difficult. This effect did not occur, because not only working memory processes are involved here but also long-term memory and meta-cognitive processes. In addition, motivational aspects of appraisal and attribution could be the reason why a direct and unambiguous interaction effect was not measurable.

3. Discussion

In this paper, an extended model of self-regulated learning with multimedia learning material was presented. Based on this model, we assumed that successful multimedia learning is determined mainly by working memory capacity. Moreover, working memory capacity and different types of cognitive load are regarded as dependent from other influences. Though most of the research on cognitive load theory is dedicated to the design of learning resources, not only the design itself plays an important role in information processing but also processes of appraisal and attribution. In two experiments, the influence of appraisal of media on cognitive load and learning outcomes was examined. Results revealed that appraisal outcomes related to the ease of information processing with different types of media influence the amount of invested mental effort. The more difficult a medium is judged to be, the higher the invested mental effort in processing the information presented within the respective medium. Following prior work [24], these findings undermine the assumption that the attitude towards specific media might directly or indirectly influence the way of information processing. This influence does not only seem to affect mental effort before and during learning, but it also has an effect on attribution patterns related to the type of medium. Results from both studies showed that learning success is more likely to be attributed to learners’ ability and effort if a specific medium is judged to be more difficult. Learners regarding a specific medium to be easier to learn with preferred an external attribution strategy, (i.e., the medium) for explaining successful learning. There is also evidence that judgment of the quality of media-specific information design influences this attribution. Regarding direct media comparisons, as conducted in experiment 1, information design is generally judged to be better in computer-mediated learning material than in printed media. This also influences the mental effort invested in information processing. Results indicate that, if learning is successful with a specific medium, this medium is judged to be not ideally designed and also estimated to be more difficult. Moreover, learning success is more likely to be attributed to learners’ ability and effort.

Another aim of this research was to identify the influence of modality in multimedia learning. Though experiment 1 did not confirm the modality effect in multimedia learning, the effect was replicated in experiment 2. Though animations with audio in both experiments led to the highest learning outcomes, several aspects provide alternative explanations for these findings. First, a general advantage of animations as dynamic visualizations is possible. Using dynamic multimedia learning material enables learners to supplant transitions between different stages of animation independent of modality. Second, in complex animations, interactive processes between long-term memory and working memory are likely, thus making it impossible to find effects based on separate modality-specific information processing in working memory. One of the most reasonable explanations is that the modality effect can be explained by split-attention instead of modality. With a shorter animation and the screen text separated from the image, there was an advantage of an auditive animation versus an animation with screen text. In the animation with text from experiment 1, the text changed positions to be located directly at the place in the animation that the text referred to. In experiment 2, text fields were permanently located at the bottom of the animation. This indicates that the benefits of audio/image animations result from avoiding split-attention. Nevertheless, this has to be re-examined in further research.

Missing interaction effects between modality and appraisal indicate that information processing in multimedia learning is not restricted to mere working memory processes. Instead, working memory interacts with long-term memory and meta-cognitive processes. From a cognitive load theory perspective, this implies that it is not sufficient to keep extraneous cognitive load low in order to have free capacity for germane cognitive load. This capacity for germane cognitive load is used effectively if learning and elaboration processes are conducted on purpose. Active learning also depends on appraisal and attribution processes related to the media, as both experiments showed. Disadvantageous appraisal and attribution strategies could be compensated by specific instructions, but further research has to be carried out in order to find effective compensation strategies.

Both studies show promising results, but limitations regarding the sample have to be discussed. The participants of the first experiment were mainly female university and high school students, whereas the second sample consisted only of males from the army. Due to the heterogeneity of the sample, the results are not generalizable and representative of other populations. For further studies on this topic, samples should be more balanced regarding gender and level of education.

Regarding practical implications, this research shows that when planning multimedia-based instruction the role or at least the attitude toward the domain has to be taken into consideration. This could also imply the use of adaptive instruction that might contribute to reducing mental burdens towards the learning material by e.g. introducing students in an adaptive manner to the learning content depending on their attitudes towards the domain. Nevertheless, the key to successful multimedia learning is active information processing by elaboration. Despite the learning material itself, this could be fostered by supplementing instruction such as prompting, where learners are required to elaborate on the learning content by providing summaries or conclusions. Another possibility could be to use testing effects by providing a pre- and post-test, e.g., when animations are integrated within a web-based training.

4. Further Directions

Results of this study reveal the importance of appraisal and attribution processes to media regarding mental effort investment and learning outcomes. Nevertheless, recent work in this field shows that positive emotions also play a central role in increasing learners’ engagement. This process could directly influence learners’ investment of cognitive resources allocated to carry out a specific task and consequently improve learning. Thus, further work is needed to investigate the impact of positive emotions, attitudes, and mental effort on learning outcomes in multimedia learning environments. In addition, the use of additional instruction during learning with animations seems to be a promising research field. Especially, how to foster elaboration processes and, thus, active learning by means of instructional scaffolds seems to be worth examining. Additionally, the way such effects as reported here contribute to effects in longer-lasting learning environments should be examined. This refers to the validity of such instructional approaches as examined here that mainly address working memory effects. The question remains how these effects impact learning in learning environments that are much more complex than animations used in standardized and controlled experiments.