Next Article in Journal
Investigating the Effectiveness of Group Work in Mathematics
Previous Article in Journal
Using Institutional Resources and Agency to Support Graduate Students’ Success at a Hispanic Serving Institution
Open AccessArticle

Hypertext or Textbook: Effects on Motivation and Gain in Knowledge

Centre of Maths and Science Education, University of Bayreuth, University Campus, 95447 Bayreuth, Germany
*
Author to whom correspondence should be addressed.
Academic Editor: James Albright
Educ. Sci. 2016, 6(3), 29; https://doi.org/10.3390/educsci6030029
Received: 4 April 2016 / Revised: 16 July 2016 / Accepted: 16 August 2016 / Published: 24 August 2016

Abstract

Computers are considered innovative in classrooms, raising expectations of increased cognitive learning outcomes or motivation with effects on Deeper Learning (DL). The “new medium”, however, may cause cognitive overloads. Combined with gender-related variations in ability, self-efficacy or self-confidence, computers may even diminish learning effects. Our empirical study used a quasi-experimental design and the Intrinsic Motivation Inventory (IMI) to monitor efficacy in knowledge gain and motivation when using computer-aided versus textbook-based educational units. Our sample consisted of 393 eighth graders. One objective focused on gender effects associated with autonomous teacher-assisted learning via interactive software or an appropriate textbook. Both groups finished with a recapitulation with the teacher. A third group concluded a computer-aided lesson with a computer quiz. To provide evidence for DL we tested long-term memory after six weeks and examined its correlation with intrinsic motivation factors. In general, our intervention affected the girls’ but not the boys’ intrinsic motivation. We recorded significantly higher post-test scores in the textbook-based lesson, but the differences vanished in the retention test. The teacher-assisted consolidation phase increased long-term knowledge and positively intervened with the students’ interest. Thus, we found evidence for DL.
Keywords: Deeper Learning; e-Learning effectiveness; gender issues; intermedia comparison; learning motivation; hypertext; evaluation of e-Learning; cognitive load Deeper Learning; e-Learning effectiveness; gender issues; intermedia comparison; learning motivation; hypertext; evaluation of e-Learning; cognitive load

1. Introduction

Compared to textbooks, computers are regarded as innovative new media in classrooms leading to a substantial increase in motivation scores and cognitive learning outcomes. Students seem to regard computers as a new tool which motivates them and promotes cognitive learning [1], although several problems could interfere with success in learning [2] by causing cognitive load problems [3].
Almost 30 years ago, Salomon assumed that students might see computers as toys and thus underestimate the importance of the subject matter [4]. On the other hand, students may regard print media as difficult-to-handle learning tools needing much concentration, while in contrast audios and films may offer relaxation rather than encouraging hard work; this may reduce the attention level for the subject matter and consequently reduce cognitive learning achievements. Meanwhile, schools are reaching the digital age. For the last ten years e-learning has been discussed not only as an enrichment of teaching materials but also as leading to improvement in personal skills, and to Deeper Learning (DL), a concept describing a more meaningful and self-dependent way of learning [5,6,7]. Therefore, teachers change their role-perception from a broadcaster of knowledge to a tutor and change their view of their teaching: students have to see themselves as the responsible persons for their learning [6].
The present study analyses learning success and intrinsic motivation in the context of a self-regulated learning unit involving either textbooks or computers with varying amounts of teacher supervision.

1.1. What is Hypermedia and Hypertext?

The terms “hypermedia” and “hypertext” were originally defined in 1965 by Ted Nelson; “Hypertext is a method to create and access nonlinear text” [8] (p. 4). Hypermedia describes a version of hypertext by emphasizing multimedial aspects. Thus, hypermedia can be understood as a combination of the words “hypertext” and “multimedia.”
The primary characteristic of hypertext is the information unit labelled as a “node”. A node is a basic unit used in computer science. From a semiotic point of view the type of the node is a concrete symbol. Thus a node is a central element in dividing information into chunks of knowledge. This may be pure text (single character, row or a passage) in the basic hypertext or a graphic, audio or video-element in hypermedia [9,10]. The greatest and most common usage of hypermedia is the World Wide Web (www): links connect information units (nodes), and organise the information via the relationships among the nodes [10].
Often the terms hypertext and hypermedia are used synonymously (e.g., [11]). Since we compare the effects of a text, presented in a book or in computer-aided multimedia, we decided to use the original term hypertext, although the multimedia aspects of the computer-aided learning environment could justify the use of the broader term hypermedia.

1.2. Advantages of Learning with Hypertext

1.2.1. Learning Motivation and Deeper Learning

Computer-aided learning is deemed to support interest and learning success because of a learner’s (inter-)activity (e.g., [12]). Dale (2008) reported “the relative newness and coolness” of computers [7]. Since the iPod assignments in his study did not feel like work to the students, it was easier to motivate students and draw them into the instructional process.
There are complex computer programs that enable interactivity, such as communication with other learners or adaptive learning programs: common examples are “Wiki”s, collaborative websites like the encyclopaedia “Wikipedia” [13] or “LEO” (Link Everything Online) [14], an internet-based electronic dictionary with integrated training functions. In contrast to this, in conventional learning software, learners still have passive roles by mostly interacting with navigation features [15,16]. Just by itself, the dynamic of hypertexts promotes interactivity since the opportunity of self-regulated learning encourages exploration of the individual’s interests. Thus hypertext is a constructivist learning environment, offering a suitable learning setting which specifically supports constructing knowledge and preventing boredom [17]. Information offered in nodes is not essential for the present text, although the information reached via nodes offers the possibility to deepen one’s knowledge of a topic. Simply having freedom of choice may help to satisfy curiosity and thus foster joy of studying [18] (p. 195). Any hypertext offering the option to deal in more detail with the subject requires additional involvement with it, so that the outcome is less likely to be just linear learning but a networked thinking process. Students have to process the learning matter, resulting in meaningful knowledge unachievable by rote-learning alone [12].
Even if activity is reduced, a wide range of information of variable complexity relevant to one’s interest is offered, thus providing individualised learning [12,15]. The need for navigation and the didactic interaction with feedback may support individual motivation and activate thinking abilities, intensify elaboration, permit creativity and cultivate learning by discovery [12].
More incidental advantages of online learning could be the increase of personal responsibility for learning and personal development, especially within the social communities of the platforms [6] (p. 311). These are valuable skills, summarised in the concept of Deeper Learning (DL): Learners are not passive recipients. Piaget [19] puts emphasis on the importance of action and problem-solving in learning [20]. Learning does not occur by listening to the teacher in silence but is constructed through active participation. Similarly, Papert describes such cognitive autonomy as “intellectual self-determinism,” as children “act as creators rather than consumers of knowledge” [21] (p. 12). Knowledge that is created in this way is crosslinked and meaningful, which is why learners forget less.
Several factors enhancing DL have been identified: degree of interest in, the relevance of and the challenge provided by the subject content; a workload which is perceived as comfortable by students; provision of a framework which demonstrates interrelationships e.g. through the use of concept maps—or hypertexts; assessment instruments which reward Deeper Learning (e.g., a quiz), and student involvement in their own learning through the use of strategies such as group work or negotiation of topics for subject assessment tasks [22].
A learning environment fostering DL gives students the ability to practise their Learning Lab skills in the context of meaningful projects that foster critical thinking, problem solving, collaboration and learning per se [23]. According to Dale, the hypertext environment is particularly suitable for DL, meeting all these demands with the opportunity of self-regulated explorative learning [7].
Thus with self-regulated computer-aided learning we expect higher gain in intrinsic motivation factors such as interest, in turn associated with higher knowledge gain.

1.2.2. Learning Success

The developers Nelson and Engelbart, the fathers of hypertext, were inspired to evolve the hypertext technology by Bush’s (1945) essay “As We May Think” [24], which describes a sort of collective memory machine, called “Memex” (Memory Extender). This precursor of hypertext-organised information in a manner comparable to the organisation structure of human memory as a semantic network of inter-connected concepts [24]. It was thus capable of changing a flood of information into knowledge. Similarly, Collins and Quillian [25] described a “semantic memory” as “the net-structure of hypertext reflecting the mental presentation of knowledge,” as Ausubel did with his “assimilation theory of cognitive learning” [26]. This analogy of hypertext and human memory may support the learning process, as students need networked thinking for meaningful understanding [27], which is why hypertexts are regarded as an effective educational tool with the additional benefit of allowing conceptual learning. Similar to concept maps, hypertexts may provide a perfect tool for visual presentation of interrelations [28].
Computers also provide an opportunity to access multimedial and multicodal information for learners using various learning styles [29,30]. Hypermedial learning environments are therefore considered helpful tools to teach even complex, interdisciplinary subject matters with complex structured concepts, as in science education. The multimodality and multicodality may help the student to understand a structure, and may support cognitive processing [31]. Learning via nodes in a hypertext reveals relationships between different concepts: this helps learners in the active meta-cognitive progress of long-term knowledge, an essential part of meaningful learning—a step to DL.
DL is “not memorising only to forget and it is not reciting or regurgitating what really is not understood and cannot be applied” [32]. DL is thought to lead to long-term learning because you forget less; meaningful knowledge is learned when intrinsically motivated, probably with fun. Computer-aided learning environments are considered to support DL, as students acquire knowledge by exploring according to their interest [6]. In accordance with PISA progressive teaching methods, like problem-based, inquiry-based learning, an individual and group project work can be used to foster self-regulated learning and deep understanding and prepare students to apply their knowledge in novel situations [33].

1.3. Disadvantages

As early as 1987 Conklin found two main disadvantages of hypertext: “disorientation” (“lost in hyperspace”) and “cognitive overhead” [34] (p. 40), which may coincide with the theory of “cognitive (over-)load” [35]. Nowadays with web 2.0 the problem has become more complex than ever!
Disorientation may occur with malfunctioning navigation design, supporting associative but not target-oriented exploration. This can lead to problems especially when hypertext data are highly complex, shallowly structured and/or a user has little pre-knowledge of handling computers and hypertext [36,37]: “Navigation in its narrow sense means to move through space; in its broader sense, navigation also includes virtual movement through cognitive space made up of data and of knowledge emerging from those data” [38] (p. 63). With inappropriate navigation design, a learner has no mental map of the hypertext, and may rapidly lose track where he/she is manoeuvring within the hypertext and where nodes interlink. Some learners, named “apathetic hypertext users” [39] will always get lost in hyperspace. “Knowledge-seekers” or “book lovers” navigate in a consequential, logical, systemic and strategic manner [39]. As the name suggests, these students could learn equally well with textbooks as with hypertext. “Feature explorers” or “resource junkies” are more interested in the style of the hypertext and what kind of screens it contains, than in the important substance of the written text. Those learners will use the hypertext exploratively. Workbook usage might remind such learners to take in information.
Since the iPod assignments were not seen as work by the students, it was easier to motivate students and draw them into the instructional process [7]. On the other hand, this very point could be a disadvantage: Salomon describes an underestimation of the matter as students think “print is hard, television is easy” [4].
Another crucial problem hindering learning success may lie in cognitive overhead/overload. The “Generative Theory of Multimedia Learning” [40] maintains that learners are limited-capacity dual encoders who actively process information in order to integrate it meaningfully into their existing knowledge. Working memory is needed to integrate new information and handle the learning aid. Problematical at this juncture is the limited information capacity of the mental activity of working memory, labelled cognitive load (CL) (e.g., [35,41]). Sweller et al. [3] assume three CL components: (CL1) intrinsic load caused by content complexity, (CL2) extraneous load caused by the instructional mode, and (CL3) germane load necessary for individually processing information in preparation for long-term memory. As all three components are assumed to be additive [42], an increase of component (CL1) and/or (CL2) without a decrease of the other would lead to the loss of (CL3), causing Cognitive Overload. Reducing the capacity for (CL3) germane load would consequently reduce cognitive learning of the subject matter. Applied to the CL theory (CL2) extraneous load refers to the instruction design. Thus computers and textbooks could cause different CLs. A hypermedia design following the principles of multimedia learning can enhance a learner’s achievement by avoiding cognitive overload [40,43]. The advantages of self-regulated and inquiry-based learning complement each other [43]. Many issues fundamental to the use of computers and hypermedia have been studied; preeminent are the principles of helpful hypermedia use, identified by Mayer [40]; for instance, avoiding split attention which might increase cognitive load [35]; additionally, the modality of information has been shown to have an effect on learning owing to its limitation of the working memory capacity [41,44]. Probably computer handling is not a relevant cognitive load factor for “digital natives” [45]. Dale refers to the youth of today as the “iPod generation,” one well prepared for e-learning [7].

1.4. Gender

A number of studies of gender effects in hypermedia instruction have yielded inconsistent results; some report no differences at all [46], others report large differences with regard to interest in computers [47]. This may be an effect of the participants’ age: children monitored within the project “Hole-in-the-Wall Education Ltd” (New Delhi, India) (HIWEL; www.hole-in-the-wall.com) learned computer handling as a required skill (e.g., the internet language English) through practice. Adolescents and adults seem to have a natural “timidity”, a scepticism towards new methods [48]. Previous studies have shown that educational software motivates girls less than boys [2]. This might be explained by the fact that designers of the software tend to be male.
In the 1990s, studies in Germany reported boys to be more experienced with computers than girls [49]. Decker [50] drew the conclusion that boys have higher competence in this context, a conclusion that could have been premature. Furthermore, the rapid development of children’s computer handling needs consideration.
Our research questions were the following:
  • Do students learn with hypertext?
  • Do students gain more long-term knowledge with hypertext?
  • Does lack of teacher’s tutoring effect long-term learning success?
  • Does autonomous learning without teacher assistance affect intrinsic motivation?
  • Does higher intrinsic motivation correlate with higher knowledge test results?
  • Are there gender differences in learning emotions and success between the learning environments?

2. Materials and Methods

2.1. The Teaching Unit

Our study consisted of a quasi-experimental before-after-control-impact (BACI) design [51] (read more 2.4) with the instructional mode as the independent variable:
(1)
Computer-aided lesson with computer-aided consolidation phase (CALC) (n = 51);
(2)
Computer-aided lesson with teacher-assisted consolidation phase (CALT) (n = 114);
(3)
Textbook-based lesson with teacher-assisted consolidation phase (TAL) (n = 172);
(4)
Control group, not participating in the unit (n = 55).
We designed a computer-aided lesson about honeybees consistent with the Bavarian syllabus for the age group. The module “Honeybee—Social Insects” highlighted the following subjects and paid particular attention to evolutionary contexts:
Superorganism “bee colony” (e.g., social organisation, bee language);
Timelines (e.g., life history of a working bee);
Products (e.g., beeswax);
Matter for humans (e.g., economy).

2.2. The Hypertext Design

We used a common navigation bar used in web 2.0 today (Figure 1). Nodes already displayed on pages are marked by changed font colour. An additional site map comparable to the table of contents in the textbook helps create a mental map of the hypertext [37]. A glossary is implemented as a tabular overview of the most important terms, each linking to the corresponding screen. The glossary is not so much a navigational aid through the hypertext as a pointer to the learning target. A working book was accompanied by an optional guided tour through the hypertext, as recommended by Jonassen [9]. The working book supports the self-regulated learners to reach the educational objective.

2.3. Sample and Design

We selected eighth graders (N = 393) at the top level (“Gymnasium”) in Bavaria, Germany. A total of 20 biology classes participated in the study. This cohort already had sufficient experience with computers since information technology is part of the regular syllabus. The participants’ mean age was 13.6 (SD = 0.63) years. The gender distribution revealed a higher number of boys (54% to 46%).
A single teacher unknown to all students tutored all lessons. After a ten-minute introductory phase students worked cooperatively in groups of two, or three at most, for 60 min. The students could choose their team. Students with CALC and CALT used an informative internet homepage with pictures, texts, videos and animations. Students with TAL used an equivalent textbook, with the same texts and pictures as the homepage but with the obvious lack of multimodality.
As a consolidation phase, the class reviewed the subject matter for ten minutes. In CALT and TAL, students recapitulated on the basis of a quiz presented with a PowerPoint presentation with teacher’s tutoring. To exclude teacher’s assistance in CALC, students used an equivalent computer-aided quiz (cf. Table 1).
A workbook provided a guideline through the lessons. Questions in the workbook had to be answered autonomously by the team using research in the homepage and the textbook respectively. Students were requested to complete their own workbook. There was no teacher support needed, except with regard to technical problems with the in-school computer network.

2.4. Quasi-Experimental Before-After-Control-Impact Design (BACI) Design

A study with students at schools in everyday conditions is comparable to an ecological “open field experiment”: however strict the design, in educational settings there are always many uncontrollable factors. This is why we call this study “quasi-experimental”. We use a before-after-control-impact design (BACI) (Smith [51]): We measure cognitive knowledge before and after the treatment.
In our study we focused on the effect of media: a text was provided in a hypermedia environment using computers or with an equivalent textbook. In a third treatment group we excluded the teacher’s tutoring completely. The consolidation phase was a quiz with computer feedback instead of the teacher-assisted recapitulation. We collected data on students’ cognitive learning achievement and intrinsic motivation. We examined the effects of computer-aided and textbook-guided learning on cognitive knowledge and learning emotions, and took gender into account.

2.5. Dependent Variable

2.5.1. Knowledge Test

Previous knowledge and changes in the level of knowledge were measured by means of a pre-test (K1) two weeks before, a post-test (K2) immediately after the lesson and a retention test (K3) 6 weeks after that (Table 1). The questionnaire covered the content of the learning targets of the lesson. The 16 items were in multiple-choice format with three distractors and one correct answer (examples Table 2). Although multiple choice knowledge tests do not test meaningful learning this tool can test long-term learning, which is considered as an indicator for Deeper Learning [32].
As students had hardly any knowledge (sum score M = 5.4 of 16, Figure 2), K1 had a low Cronbach’s alpha = 0.05. Reliability was, however, acceptably high in K2 (Cronbach’s alpha = 0.73) and quite good for the learning tests in K3 (Cronbach’s alpha = 0.59). Generally, knowledge tests are difficult to test for reliability although reliability coefficients less than 0.6 are used for differentiating groups [52] (p. 213).
Item difficulty, defined as the percentage of correct answers [53] was normally distributed for the K1, K2 and K3. Content validity of the knowledge test is given due to the syllabus-based subject. Furthermore, all knowledge items were constructed according to the learning goals of the intervention.
Since the knowledge sum scores were not normally distributed (Kolmogorov–Smirnov with Lilliefors significance correction, p < 0.001), we used non-parametric tests and box plots. We presume that the different sample sizes and the quasi-experimental design of the study affected the non-normal distributed data (Shapiro-Wilk p = 0.000).

2.5.2. Intrinsic Motivation Inventory (IMI)

The “Intrinsic Motivation Inventory” (IMI) [54] is a multidimensional measurement device to assess participants’ subjective experience related to a target activity (e.g., a lesson). IMI has been applied effectively in many experiments related to intrinsic motivation and self-regulation (e.g., [55,56,57,58]). We applied four of the six subscales of IMI: The subscale interest/enjoyment (IMI-I) is deemed to be the self-report measure of intrinsic motivation (seven items; e.g., “I enjoyed doing this activity very much.”). The concept of perceived competence (IMI-C) is considered to be a positive predictor of both self-report and behavioural measures of intrinsic motivation (6 items; e.g., “I think I am pretty good at this activity.”). The subscale effort (IMI-E) is relevant to some motivation questions and is thought to measure activity’s relevance (five items; e.g., “I put a lot of effort into this.”). The felt tension/pressure (IMI-T) is theorized to be a negative predictor of intrinsic motivation (five items; e.g., “I was very relaxed in doing this.”). We used the German version of IMI, which has previously been successfully applied, for instance, by Girwidz et al. [1]. Students scored their answer on a five-digit Likert scale ranking from 1 (not true at all) to 5 (very true). The test was integrated in the post-test K2 after the knowledge questionnaire, and applied immediately after the intervention (Table 1).

2.6. Analysis

For statistical analyses we used SPSS 21.0 (IBM SPSS Statistics, IBM Deutschland GmbH, Ehningen, Germany). Due to the partially non-normally distributed variables (Kolmogorov-Smirnov test: p > 0.05) we used non-parametric analysis. Knowledge items were scored as correct (1 point) or incorrect (0 point) and summed to yield a score. In the case of IMI, we calculated mean scores for each subscale. We calculated inter-group differences with the Friedman test (FT) and between-group differences with Mann-Whitney U test (MWU) for two independent samples, and the Kruskal-Wallis test (KW) to analyse k independent samples. The significance of learning success was calculated using the Wilcoxon test (WT).

3. Results

3.1. Cognitive Achievement

There was no test effect as the control group did not gain knowledge (FT: Chi-Square = 2.995, df = 2, p = 0.224). The K1 showed neither differences between treatment groups (KW Chi-Square = 2.769, df = 3, p = 0.429) nor between the genders (MWU: Z = −0.864, p = 0.388).
Students of TAL gained significantly higher sum scores than students of CALT immediately after the lesson (MWU: K2 Z = −4.430, p < 0.001; Figure 3—K2) and yielded the highest knowledge scores among all students. After six weeks both groups achieved equivalent cognitive sum scores (MWU: K3 Z = −1.098, p = 0.272; Figure 3—K3). In CALT as well as in TAL girls gained significantly higher sum scores than boys in K2 (MWU: CALT Z = −2.385, p = 0.017; TAL Z = −3.784, p < 0.001; Figure 3—K2). This was still found after six weeks in K3 (MWU: CALT Z = −3.777, p < 0.001; TAL Z = −2.339, p = 0.019; Figure 3—K3). Girls forgot less knowledge after six weeks with CALT, which caused significant differences in oblivion rates (K3–K2) (girls MWU: Z = −3.393, p = 0.001; all MWU: Z = −3.036, p = 0.002; Figure 4).
All students learned with computers in both CALC and CALT. Cognitive achievement immediately after the lesson was equivalent independent of the consolidation phase of CALC or CALT (MWU: K2 Z = −1.12, p = 0.265). In respect to long-term learning, students of CALT gained significantly higher knowledge scores than students of CALC (MWU: Z = −2.23, p = 0.026).
The lack of teacher guidance is disadvantageous notably for girls: in CALC—without teacher-guided consolidation phase—all students, even the girls, yielded lower knowledge scores in K3 than in CALT or TAL. The treatment effect is significant for girls with TAL (K2 MWU: Z = −2.858, p = 0.004). But there is no treatment effect with the boys. Accordingly, gender effects were reduced in CALC: there is no gender effect in K3 (MWU: Z = −0.949, p = 0.343) and hardly any in K2 (MWU: K2 Z = −2.065, p = 0.039).
This finding is remarkable as girls seemed to learn more in general as both genders have similar pre-knowledge scores (MWU K1: Z = −0.66, p > 0.5), yet girls gain higher sum scores than boys in K2 (MWU K2: Z = −2.685, p = 0.007) and K3 (MWU K3: Z = −3.974, p = 0.000; Figure 2).
For the indication of DL we are interested in the oblivion rate, which is calculated from the difference of K3 and K2. There is a significant treatment effect with lowest oblivion rates found with CALT (Table 3; Figure 4). Girls forgot most with TAL, whereas boys forgot more with CALC.

3.2. Intrinsic Motivation Inventory (IMI)

We found the usual acceptable Cronbach’s alpha for interest/enjoyment IMII (seven items; Cronbach’s alpha = 0.744), perceived competence IMI-C (six items; Cronbach’s alpha = 0.673) and tension/pressure IMI-T (five items; Cronbach’s alpha = 0.622). We rejected the subscale Effort IMI-E due to a questionable Cronbach’s alpha (five items; Cronbach’s alpha = 0.545). Furthermore, this subscale is not affected at all.
There were no significant treatment effects in the IMI subscales without correlation to gender. However, in the IMI-I (interest), students yielded the highest mean scores in CALT (Figure 5: IMI interest). This is due to the fact that girls’ intrinsic motivation is affected by the treatments but not the boys’. The IMI-I mean scores are significantly lower in CALC than in CALT (MWU: Z = −2.744, p = 0.006) or in TAL (MWU: Z = −1.794, p = 0.073). The difference between CALT and TAL is not significant (MWU: Z = −1.448, p = 0.148). This difference between CALC and CALT in IMI-I is affected by the girls’ increase in IMI-I. Although girls are more interested in the subject with CALT than with TAL (not significant) they are significantly less interested with CALC (MWU: CALC CALT: Z = −2.130, p = 0.033).
Girls’ perceived competence IMI-C is higher with CALC than with CALT (MWU: Z = −2.300, p = 0.021) but equivalent in CALT and TAL.
Lastly, girls reported least tension (IMI-T) in TAL with significant difference to CALT (MWU: Z = −3.075, p = 0.002) but IMI-T is similar in CALC and CALT. The boys reported similar intrinsic motivation in all treatments.

4. Discussion

Our examination of the effects of CAL and equivalent TAL in biology lessons focused on the differences in cognitive achievement and the intrinsic motivation with regard to gender. An underestimation of the subject matter’s difficulty [4] was excluded by the regular CAL in the eighth grade and workbook usage.
In line with Schaal [59] we found the highest mean scores for interest for the subject matter in CALT. This is notable as there can hardly be any novelty effect as participants were sufficiently accustomed to CAL. However, in contrast to Schaal [59], but in line with Conradty and Bogner [60], the higher interest scores did not result in higher knowledge scores in K2. This may result from learning-inhibiting factors, namely extraneous load (CL2) [3]. But firstly, all students had sufficient experience in computers because of regular IT lessons. Secondly, students reported highest perceived competence (IMI-C) in CALC. As the software design followed the principles of multimedia learning, we conclude that cognitive overload was reduced to a large extent [61].
Contrary to the expectations of Dillon and Gabbard [29] and Rasmussen and Davidson-Shivers [30], the multicodality of CAL with its opportunity of appealing to various learning styles did not enhance cognitive achievements in K2. However, retention rate was higher in CALT than in TAL, indicating that computer-aided learning could support learning. Thus the oblivion rate was lowest with CALT, indicating a meaningful learning success. Students learning with CALT learned less but then forgot less. This could be caused by the fact that interest in the subject (IMI-I) with CALT is higher, probably owing to the use of multimedia such as teacher’s presence. The learning success we monitored is in line with the findings of Dale [7] who perceives this learning success as DL, according to Askew and Carnell [62]. It could be argued that the augmentation of personal responsibility for learning and personal development [6] is in the tradition of Piaget [19], who challenged the importance of action and problem solving in learning [20] (p. 9). In other words, learning is constructed through active participation as children engage in the learning process to act out solutions to problems faced. Similarly, Papert [21] describes such cognitive autonomy as “intellectual self-determinism,” as children “act as creators rather than consumers of knowledge.”
Our results point to the relevance of the teacher according to Conradty [60]: CAL as an instructional enrichment increased a student’s interest in the subject matter. Without the teacher-assisted consolidation phase the perceived competence was higher but both interest and cognitive achievement were reduced dramatically. The role of the teacher is vital for long-term learning. The mixture of “intellectual self-determinism” [21] and the social contact with a teacher in the role of a tutor instead of a “broadcaster of knowledge” fosters long-term learning success [60], as we found in CALT. Authentic learning is embroiled with issues of socialization and identity, not only─but especially─with teenagers, and it would be a mistake to attempt to reify or separate these.
In line with Dillon and Jobst [63], we found gender effects with participants older than fifth graders. At this age, students are adolescent, with gender-related behaviour determined by their already developed gender identity [64]. As girls learned more in this study, this might be due to their preference for the subject “biology”: for years, about 63%–66% of the first-semester students in biology have been women [65,66]. In general, girls’ motivation and learning success was affected by our intervention, but not the boys’. In particular, in CALC learning about the bee dance failed for both genders; indeed, girls learned even less than the boys, reflecting perfectly their loss in interest. Probably the lack of contact with the teacher served to decrease interest scores even more than the computer increased them. These findings contradict Passing and Levin [2] and Conradty [60] whose educational software did not motivate girls, whereas in our study, only girls’ learning emotions were affected. This might be explained by the software and textbook which in our case were designed by a female designer. But in both the present study and previous work of Conradty and Bogner [60], the teachers were women.
A good reason for a computer implementation in regular lessons is the students’ opportunity to learn required skills casually as monitored within the project “Hole-in-the-Wall Education Ltd” [48,67]. These incidental skills are not only computer handling or a foreign language but also DL skills. According to Turvey [6] the type of long-term learning is one that is based upon a deep understanding, actively created by students taking responsibility for their learning consistent with their interests.

5. Conclusions

In the present study, our findings provide evidence that students who are used to computers can adequately learn with them. Computers may provide instructional enrichment by increasing a student’s interest in the subject matter and providing additional skills such as the ability for Deeper Learning. Therefore, computer learning supports meaningful and long-term learning. Important for both interest and learning is the subsequent tutoring by a teacher with a refined role-perception. With respect to gender in our study, it was the girls’ intrinsic motivation who were most affected. Notably, girls’ learning success with CALT—computer-aided learning with teacher tutoring—increased with their interest in the subject matter, though this was not the case with the boys. Therefore, our results could be viewed as reflecting the Deeper Learning and motivating effects of CALT. Further studies should focus on the effect of teacher’s contact on students’ interest and learning effect. Continuative indicators for Deeper Learning have to be specified to design appropriate studies.

Author Contributions

Cathérine Conradty and Franz X. Bogner conceived and designed the experiments; Cathérine Conradty performed the experiments and analyzed the data; Cathérine Conradty and Franz X. Bogner wrote the paper.

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

The following abbreviations are used in this manuscript:
CLCognitive load
IMIIntrinsic Motivation Inventory
CALComputer-aided learning
CALCComputer-aided learning with computer-centred consolidation phase
CALTComputer-aided learning with teacher-assisted consolidation phase
TALTextbook-based learning with teacher-assisted consolidation phase
K1pre-test two weeks before intervention
K2post-test immediately after intervention
K3retention test six weeks after intervention
DLDeeper Learning
MWUMann-Whitney U test
FTFriedman test

References

  1. Girwidz, R.; Bogner, F.X.; Rubitzko, T.; Schaal, S. Media assisted learning in science education: An interdisciplinary approach to hibernation and energy transfer. Sci. Educ. Int. 2006, 17, 95–107. [Google Scholar]
  2. Passig, D.; Levin, H. Gender preferences for multimedia interfaces. J. Comput. Assist. Learn. 2000, 16, 64–71. [Google Scholar] [CrossRef]
  3. Sweller, J.; van Merrienboer, J.; Paas, F. Cognitive architecture and instructional design. Educ. Psychol. Rev. 1998, 10, 251–296. [Google Scholar] [CrossRef]
  4. Salomon, G. Television is “easy” and print is “tough”: The differential investment of mental effort in learning as a function of perceptions and attributions. J. Educ. Psychol. 1984, 76, 647–658. [Google Scholar] [CrossRef]
  5. VanderArk, T.; Schneider, C. How Digital Learning Contributes to Deeper Learning. Available online: http://cdno.gettingsmart.com/wp-content/uploads/2012/12/Digital-Learning-Deeper-Learning-Executive-Summary.pdf (accessed on 14 March 2016).
  6. Turvey, K. Towards deeper learning through creativity within online communities in primary education. Comput. Educ. 2006, 46, 309–321. [Google Scholar] [CrossRef]
  7. Dale, C. iPods and Creativity in Learning and Teaching: An Instructional Perspective. Int. J. Teach. Learn. High. Educ. 2008, 20, 1–9. [Google Scholar]
  8. Kommers, P.A.M.; Grabinger, R.S.; Dunlap, J.C. Hypermedia Learning Environments. Instructional Design and Integration; L. Erlbaum Associates: Mahwah, NJ, USA, 1996. [Google Scholar]
  9. Jonassen, D.H. Hypertext/Hypermedia; Educational Technology Publications: Englewood Cliffs, NJ, USA, 1989. [Google Scholar]
  10. Jonassen, D.H. Computers in the Classroom; Mindtools for Critical Thinking: Englewood Cliffs, NJ, USA, 1996. [Google Scholar]
  11. Nielsen, J. Multimedia and Hypertext, The Internet and Beyond; Morgan Kaufmann: Burlington, MA, USA, 1995. [Google Scholar]
  12. Strzebkowski, R.; Kleeberg, N. Interaktivität und Präsentation als Komponenten multimedialer Lernanwendungen. In Information und Lernen mit Multimedia und Internet; BeltzPVU: Weinheim, Germany, 2002; pp. 229–246. [Google Scholar]
  13. Wikimedia. Wikipedia. Available online: https://en.wikipedia.org/wiki/Main_Page (accessed on 18 March 2016).
  14. TU München. Leo. Available online: https://www.leo.org/ende/index_de.html (accessed on 18 March 2016).
  15. Haak, J. Interaktivität Als Kennzeichen von Multimedia und Hypermedia. In Information und Lernen Mit Multimedia und Internet; BeltzPVU: Weinheim, Germany, 2002; pp. 127–136. [Google Scholar]
  16. Hammond, N. Learning with Hypertext: Problems, Principles and Prospects. In Hypertext A Psychological Perspective; Ellis Horwood Ltd.: Hertfordshire, UK, 1993; pp. 153–168. [Google Scholar]
  17. Tulodziecki, G.; Hagemann, W.; Herzig, B.; Leufen, S.; Mütze, C. Neue Medien in den Schulen. Projekte-Konzepte-Kompetenzen; Bertelsmann Stiftung: Gütersloh, Germany, 1996. [Google Scholar]
  18. Spitzer, M. Lernen. Gehirnforschung und die Schule des Lebens, 1st ed; Spektrum Akademischer Verlag: München, Germany, 2007. [Google Scholar]
  19. Piaget, J.; Elkind, D. Six Psychological Studies; Vintage Books: New York, NY, USA, 1968. [Google Scholar]
  20. Wood, D. How Children Think and Learn. In The Social Contexts of Cognitive Development, 2nd ed.; Blackwell: Oxford, UK; Malden, MA, USA, 1998. [Google Scholar]
  21. Papert, S. The Children’s Machine. In Rethinking School in the Age of the Computer; BasicBooks: New York, NY, USA, 1993. [Google Scholar]
  22. Johnston, C. Fostering Deeper Learning. Available online: http://fbe.unimelb.edu.au/__data/assets/pdf_file/0007/634309/DEEPL1.pdf (accessed on 14 March 2016).
  23. Chow, B. The Quest for “Deeper Learning”. Available online: http://www.hewlett.org/newsroom/news/quest-deeper-learning-barbara-chow (accessed on 14 March 2016).
  24. Bush, V. As We May Think. Reproduced with Their Permission on. The Atlantic Monthly, July 1945. [Google Scholar]
  25. Collins, A.M.; Quillian, M.R. Experiments on semantic memory and language comprehension. In Cognition in Learning and Memory; Wiley: Chichester, UK, 1972. [Google Scholar]
  26. Ausubel, D.P.; Novak, J.D.; Hanesian, H. Educational Psychology. In A Cognitive View, 2nd ed.; Holt, Rinehart and Winston: New York, NY, USA, 1978. [Google Scholar]
  27. Vester, F. Die Kunst Vernetzt zu Denken. Ideen und Werkzeuge für Einen Neuen Umgang mit Komplexität; ein Bericht an den Club of Rome, 8th ed.; Dt. Taschenbuch-Verl.: München, Germany, 2011. [Google Scholar]
  28. Novak, J.D. Concept maps to facilitate teaching and learning. Prospects 1995, 25, 95–111. [Google Scholar] [CrossRef]
  29. Dillon, A.; Gabbard, R. Hypermedia as an Educational Technology: A Review of the Quantitative Research Literature on Learner Comprehension, Control, and Style. Rev. Educ. Res. 1998, 68, 333–349. [Google Scholar] [CrossRef]
  30. Rasmussen, K.L.; Davidson-Shivers, G.V. Hypermedia and learning styles: Can performance be influenced? J. Educ. Multimed. Hypermed. 1998, 7, 291–308. [Google Scholar]
  31. Spiro, R.J.; Feltovich, P.J.; Jacobson, M.J.; Coulson, R.L. Cognitive flexibility, constructivism and hypertext: Random access instruction for advanced knowledge acquisition in ill-structured domains. In Constructivism and the Technology of Instruction; Routledge: London, UK, 1995. [Google Scholar]
  32. Weimer, M. Deep Learning vs. Surface Learning: Getting Students to understand the Difference. Available online: http://www.facultyfocus.com/articles/teaching-professor-blog/deep-learning-vs-surface-learning-getting-students-to-understand-the-difference/ (accessed on 16 March 2016).
  33. OECD. PISA 2012 Assessment and Analytical Framework. Mathematics, Reading, Science, Problem Solving and Financial Literacy; OECD Publishing: Paris, France, 2013. [Google Scholar]
  34. Conklin, J. Hypertext: A survey and introduction. IEEE Comput. 1987, 20, 17–41. [Google Scholar] [CrossRef]
  35. Tarmizi, R.A.; Sweller, J. Guidance during mathematical problem solving. J. Educ. Psychol. 1988, 80, 424–436. [Google Scholar] [CrossRef]
  36. Smith, P.A.; Wilson, J.R. Navigation in hypertext through virtual environments. Appl. Ergon. 1993, 24, 271–278. [Google Scholar] [CrossRef]
  37. Tergan, S.-O. Was macht E-Learning Erfolgreich? Grundlagen und Instrumente der Qualitätsbeurteilung; Springer: Berlin, Germany, 2004. [Google Scholar]
  38. Whitaker, L.A. Human navigation. In Human Factors and Web Development; Forsythe, C., Grose, E., Ratner, J., Eds.; Lawrence Erlbaum Associates: Mahwah, NJ, USA, 1998. [Google Scholar]
  39. Lawless, K.A.; Kulikowich, J.M. Understanding hypertext navigation through cluster analysis. J. Educ. Comput. Res. 1996, 14, 385–399. [Google Scholar] [CrossRef]
  40. Mayer, R.E. The Cambridge Handbook of Multimedia Learning, 2nd ed.; Cambridge University Press: Cambridge, UK, 2014. [Google Scholar]
  41. Baddeley, A. Working Memory. Science 1992, 255, 556–559. [Google Scholar] [CrossRef] [PubMed]
  42. Sweller, J. How the human cognitive system deals with complexity. In Handling Complexity in Learning Environments: Theory and Research; Emerald Group Publishing Limited: Bingley, UK, 2006. [Google Scholar]
  43. Blumstengel, A. Entwicklung hypermedialer Lernsysteme; Wiss. Verl. Berlin: Berlin, Germany, 1998. [Google Scholar]
  44. Plass, J.L.; Chun, D.M.; Mayer, R.E.; Leutner, D. Supporting visual and verbal learning preferences in a second-language multimedia environment. J. Educ. Psychol. 1998, 90, 25–36. [Google Scholar] [CrossRef]
  45. Prensky, M. Digital natives, digital immigrants. Horizon 2001, 9, 1–2. [Google Scholar]
  46. Teh, G.P.L.; Fraser, B.J. Gender Differences in Achievement and Attitudes among Students Using Computer-Assisted Instruction. Int. J. Instr. Media 1995, 22, 111–140. [Google Scholar]
  47. Vichaporn, R.; William, J.G. Computerized drill and practice: Design options and learner characteristics. Int. J. Instr. Med. 1995, 22, 59–77. [Google Scholar]
  48. Mitra, S.; Rana, V. Children and the Internet: Experiments with minimally invasive education in India. Br. J. Educ. Technol. 2001, 32, 221–232. [Google Scholar] [CrossRef]
  49. Jansen-Schulz, B.; Kastel, C. “Jungen arbeiten am Computer, Mädchen können Seil springen…”. In Computerkompetenzen von Mädchen und Jungen—Forschung, Praxis und Perspektiven für die Grundschule; Kopaed: München, Germany, 2004. [Google Scholar]
  50. Decker, M. Kinder vor dem Computer: Die Herausforderung von Pädagogen und Eltern durch Bildschirmspiele und Lernsoftware; KoPäd Verlag: München, Germany, 1998. [Google Scholar]
  51. Smith, E.P. BACI design. In Encyclopedia of Environmetrics; El-Shaarawi, A.H., Ed.; Wiley: Chichester, UK, 2002; pp. 141–148. [Google Scholar]
  52. Lienert, G.A.; Raatz, U. Testaufbau und Testanalyse, 6th ed.; Psychologie VerlagsUnion: Weinheim, Germany, 1998. [Google Scholar]
  53. Bortz, J.; Döring, N. Forschungsmethoden und Evaluation für Human-und Sozialwissenschaftler, Mit 87 Tabellen, 4th ed.; Springer: Berlin/Heidelberg, Germany; New York, NY, USA, 2006. [Google Scholar]
  54. Deci, E.L.; Schwartz, A.J.; Sheinman, L.; Ryan, R.M. An instrument to assess adults’ orientations toward control versus autonomy with children: Reflections on intrinsic motivation and perceived competence. J. Educ. Psychol. 1981, 73, 642–650. [Google Scholar] [CrossRef]
  55. Ryan, R.M.; Koestner, R.; Deci, E.L. Varied forms of persistence: When free-choice behavior is not intrinsically motivated. Motiv. Emot. 1991, 15, 185–205. [Google Scholar] [CrossRef]
  56. Deci, E.L.; Eghrari, H.; Patrick, B.C.; Leone, D.R. Facilitating internalization: The self-determination theory perspective. J. Personal. 1994, 62, 119–142. [Google Scholar] [CrossRef]
  57. Schaal, S.; Bogner, F.X. Human visual perception—Learning at working stations. J. Biol. Educ. 2005, 40, 28–39. [Google Scholar] [CrossRef]
  58. Sturm, H.; Bogner, F.X. Student-oriented versus Teacher-centred: The effect of learning at workstations about birds and bird flight on cognitive achievement and motivation. Int. J. Sci. Educ. 2008, 30, 941–959. [Google Scholar] [CrossRef]
  59. Schaal, S. Fachintegratives Lernen Mit Neuen Medien; Kovac: Hamburg, Germany, 2006. [Google Scholar]
  60. Conradty, C.; Bogner, F.X. Computer-Aided Learning: Unguided versus Guided Instruction. Adv. Sci. Lett. 2011, 4, 3310–3316. [Google Scholar] [CrossRef]
  61. Paas, F.G.; Van Merriënboer, J.J.; Adam, J.J. Measurement of cognitive load in instructional research. Percept. Mot. Skills 1994, 79, 419–430. [Google Scholar] [CrossRef] [PubMed]
  62. Askew, S.; Carnell, E. Transforming Learning. Individual and Global Change; Cassell: London, UK; Washington, DC, USA, 1998. [Google Scholar]
  63. Dillon, A.; Jobst, J. Multimedia Learning with Hypermedia. In The Cambridge Handbook of Multimedia Learning; Cambridge University Press: Cambridge, UK, 2005. [Google Scholar]
  64. Stoller, R.J. Sex and Gender: On the Development of Masculinity and Femininity; Science House: New York, NY, USA, 1968. [Google Scholar]
  65. Federal Statistical Office of Germany. Hochschulstandort Deutschland 2005; Federal Statistical Office of Germany: Wiesbaden, Germany, 2005. [Google Scholar]
  66. Statista. Women’s Quota. Available online: http://de.statista.com/themen/873/frauenquote/ (accessed on 14 March 2016).
  67. Hole-in-the-Wall Education Ltd. (HIWEL). Available online: www.hole-in-the-wall.com (accessed on 16 March 2016).
Figure 1. Screenshot of the hypertext.
Figure 1. Screenshot of the hypertext.
Education 06 00029 g001
Figure 2. Knowledge sum scores of boys and girls for all instructions without control (K1: pre-test; K2: post-test; K3: retention test; gender differences: ***: p ≤ 0.001; **: p ≤ 0.01).
Figure 2. Knowledge sum scores of boys and girls for all instructions without control (K1: pre-test; K2: post-test; K3: retention test; gender differences: ***: p ≤ 0.001; **: p ≤ 0.01).
Education 06 00029 g002
Figure 3. Knowledge sum scores after gender. CALC: computer-aided learning with computer-aided consolidation; CALT: computer-aided learning with teacher-assisted consolidation; TAL: textbook learning with teacher-assisted consolidation; K1: pre-test; K2: post-test; K3: retention test; black asterisk: treatment effects; blue asterisk: gender effects; ***: p ≤ 0.001; **: p ≤ 0.006; *: p ≤ 0.015).
Figure 3. Knowledge sum scores after gender. CALC: computer-aided learning with computer-aided consolidation; CALT: computer-aided learning with teacher-assisted consolidation; TAL: textbook learning with teacher-assisted consolidation; K1: pre-test; K2: post-test; K3: retention test; black asterisk: treatment effects; blue asterisk: gender effects; ***: p ≤ 0.001; **: p ≤ 0.006; *: p ≤ 0.015).
Education 06 00029 g003
Figure 4. Oblivion (K3–K2) after gender. CALC: computer-aided learning with computer-aided consolidation; CALT: computer-aided learning with teacher-assisted consolidation; TAL: textbook learning with teacher-assisted consolidation).
Figure 4. Oblivion (K3–K2) after gender. CALC: computer-aided learning with computer-aided consolidation; CALT: computer-aided learning with teacher-assisted consolidation; TAL: textbook learning with teacher-assisted consolidation).
Education 06 00029 g004
Figure 5. Intrinsic Motivation Inventory grouped by treatment, selected by gender: Interest (* p = 0.033), Competence (* p = 0.021), Tension (* p = 0.002) grouped into treatment, selected by gender; CALC: computer-aided learning with computer-aided consolidation; CALT: computer-aided learning with teacher-assisted consolidation; TAL: textbook learning with teacher-assisted consolidation.
Figure 5. Intrinsic Motivation Inventory grouped by treatment, selected by gender: Interest (* p = 0.033), Competence (* p = 0.021), Tension (* p = 0.002) grouped into treatment, selected by gender; CALC: computer-aided learning with computer-aided consolidation; CALT: computer-aided learning with teacher-assisted consolidation; TAL: textbook learning with teacher-assisted consolidation.
Education 06 00029 g005
Table 1. Quasi-Experimental (BACI) Design a of the Study.
Table 1. Quasi-Experimental (BACI) Design a of the Study.
GroupTime SpanInstruction 1—CALCInstruction 2—CALTInstruction 3—TALControl
Pre-test K1K1K1K1
Delay2 weeks
Instruction b70 minCALCALTAL
Consolidation10 minComputer-AidedTeacher-AssistedTeacher-Assisted
Post-test10 minK2 + IMIK2 + IMIK2 + IMIK2
Delay6 weeks
Retention test K3K3K3K3
Note: a This design implied a one-way multivariate analysis of variance with repeated measures which had to be rejected on the grounds of the non-normal distribution of variables; b including a 10-min introduction.
Table 2. Examples for multiple-choice questionnaires.
Table 2. Examples for multiple-choice questionnaires.
Subject MatterQuestionMultiple Choice Distractors
(a) superorganism bee colonyHow does a working bee tell its colleague what food it has found?
By performing the “bee dance”
The bee smells like the food
This is not told, only the location of the food
With specific buzzing sounds
(b) timelinesWhat is the first job of a new-born working bee?
Cleaning
Doing sentry duty
Collecting food
Building comb
(c) productsWhich material is not collected by bees?
Nectar
Honey dew
Pollen
Wax
Table 3. Statistic Data of the Knowledge Tests.
Table 3. Statistic Data of the Knowledge Tests.
EffectTreatmentTestStatistics
Test EffectControlK1, K2, K3FT: Chi-Square = 2.995, df = 2, p = 0.224
Treatment Group DifferencesTAL—CALT—CALCK1
K2
K3
Oblivion
KW Chi-Square = 2.769, df = 3, p = 0.429
KW Chi-Square = 21.141, df = 2, p < 0.000
KW Chi-Square = 11.298, df = 2, p = 0.004
KW Chi-Square = 17.456, df = 2, p < 0.000
TAL—CALTK2
K3
Oblivion
MWU: Z = −4.430, p < 0.000
MWU: Z = −1.098, p = 0.272
MWU: Z = −3.036, p = 0.002
CALT—CALCK2
K3
Oblivion
MWU: K2 Z = −1.12, p = 0.265
MWU: Z = −2.23, p = 0.026
MWU: Z = −3.963, p < 0.000
TAL—CALCK2
K3
Oblivion
MWU: Z = −21.141, p < 0.000
MWU: Z = −11,298, p = 0.004
MWU: Z = −17.456, p < 0.000
Girls‘ Treatment EffectsTAL—CALT—CALCK2
K3
Oblivion
KW Chi-Square = 24.070, df = 2, p < 0.000
KW Chi-Square = 9.802, df = 2, p = 0.007
KW Chi-Square = 12.213, df = 2, p = 0.002
TAL—CALTK2
Oblivion
MWU: Z = −3.915, p < 0.000
MWU: Z = −3.393, p = 0.001
CALT—CALCK3MWU: Z = −2.858, p = 0.004
TAL—CALCK2
K3
MWU: Z = −4.024, p < 0.000
MWU: Z = −3.106, p = 0.002
Boys’ Treatment EffectsTAL—CALT—CALCK2KW Chi-Square = 12.890, df = 2, p = 0.002
TAL—CALTK2
K3
MWU: Z = −3.418, p = 0.001
MWU: Z = −2.457, p = 0.014
CALT—CALCK2
Oblivion
MWU: Z = −2.774, p = 0.006
MWU: Z = −2.488, p = 0.013
TAL—CALC No sig. differences!
Gender Effectsall treatmentsK1
K2
K3
MWU: Z = −0.864, p = 0.388
MWU: Z = −2.685, p = 0.007
MWU: Z = −3.974, p < 0.000
TALK2
K3
MWU: Z = −3.784, p < 0.001
MWU: Z = −2.339, p = 0.019
CALTK2
K3
MWU: Z = −2.385, p = 0.017
MWU: Z = −3.777, p < .001;
CALCK2
K3
MWU: Z = −2.065, p = 0.039
MWU: Z = −0.949, p = 0.343
K1: pre-knowledge test; K2: post-knowledge test; K3: retention knowledge test KW: Kruskal Wallis test; MWU: Mann-Whitney U test; FT: Friedman test; Oblivion: lost knowledge = K3—K2.
Back to TopTop