Effect of Technology on Student Class Performance and Class Absence

Abstract

Background: This study examined the effect of instructional technology availability on the performance of students enrolled in a medical physiology course at a podiatric medical school. Methods: Multiple linear regression analysis was used to predict student overall test performance based on instructional technology, Medical College Admission Test score, undergraduate grade point average, and class absence. Results: The availability of instructional technology was associated with a small decline in mean test performance and a small increase in class absence. Class absence had a negative effect on test performance only when the technology was available. Total Medical College Admission Test score and grade point average were positively correlated with performance. Conclusions: Instructional technology did not enhance absentee student course performance and, indeed, hurt it. Its use as a means of providing access to additional lecture material needs to be reevaluated. (J Am Podiatr Med Assoc 102(6): 471–476, 2012)

The use of technology in the pedagogy at schools and universities is widespread. It is being used in the context of increasing access to education and with a view to improving student performance. However, although examinations of various affective parameters abound, objective examination of the effect on student performance in courses is sparse and inconclusive. An important confounding factor that must be considered in such studies is the effect of absenteeism.

Although cognitive factors play the major role in student performance, [1] there is much research forthcoming regarding the effect of various affective parameters, including absenteeism, on the performance of students. [2–4] Absenteeism is an oft-cited problem on university campuses and has been repeatedly addressed in the literature. [5–7] It is a proxy for a range of affective influences, eg, conscientiousness, that influence test performance.

The negative impact of class absence has been known for a long time. [8,9] In the intervening years, although increasingly refined methods have been used to examine student performance by incorporating cognitive factors (eg, grade point average [GPA], graduate record examination score [GRE], and Medical College Admission Test [MCAT] score) into the analyses, affective factors (eg, personality, openness, agreeability, and conscientiousness) have only relatively recently begun to receive much attention. Among these affective factors, absenteeism has been shown to have a considerable negative impact. [4,10,11]

In an extensive study on the impact of various course-related activities on student performance, Schmidt [10] found that the most influential affective activity was the time spent in the classroom. Time spent studying for examinations was the least influential. Therefore, it would seem that time spent in class would be an activity that has effects far beyond the mere presence and whose beneficial interactions would not be realized by an absent student. Furthermore, Peverly et al [12] found a significant correlation between good note taking skills and test performance.

Of all of the factors that can affect student performance, the one most under the student’s control is class attendance. Given the perennial problem of student absenteeism, [8,9] faculty has sought to use technology as a means of countering the negative influence of absenteeism on performance. With the advent of increasingly sophisticated computer-mediated teaching methods, eg, PowerPoint (Microsoft Corp, Redmond, Washington) presentations, recording lectures (as delivered) for later viewing (lecture capture), and message boards, it is possible to deliver the material to even the most reticent of students and, therefore, to attenuate the negative effects of their absenteeism from class.

However, data regarding the effect of such interventions on subsequent student performance are sparse and, where available, are of a survey type rather than objective and provide little conclusive evidence. [13,14] Estimates of the impact of technology are likely to be biased by the omission of absenteeism, an important confounding influence.

In this study, we examined the hypothesis that providing access to technology would improve the test performance of all students. This article represents the first contribution to the understanding of the relationship between instructional technology and performance in a medical school.

Materials and Methods

Sample

Podiatric medical students enrolled in a medical physiology course between 2006 and 2011 were the subjects of this study. The initial sample included 302 students; however, 108 (36%) were dropped because of missing MCAT scores or undergraduate GPA data, which were needed for the subsequent analysis. Therefore, the available sample was 194 students. Fifty-two percent of these students were men, and 8% were not US citizens. The sample was 43% White, 17% Hispanic, 17% African American, and 11% Asian, with 12% other race/ethnicity or not reported. The mean ± SD age of the sample was 24.6 ± 3.6 years. The underrepresented racial/ethnic minorities, Hispanic and African American, were overrepresented, composing 34% of the sample but only 14% of the population of podiatric medical students since 2007. The sample was divided into two cohorts: the instructional technology (IT) cohort that had access to audiovisual recordings of all lectures that were given during the course via learning management software (Blackboard; Blackboard Inc, Washington, DC), and the control (CON) cohort that received no such assistance. Only students taking the course for the first time were included in the analysis. Students who dropped out of the course during the year were excluded.

Lecture Capture

Lectures were recorded using Windows Media Encoder version 9 (Microsoft Corp). Each 50-min lecture in the course was accompanied by a PowerPoint slideshow. Each lecture also had associated hard copy notes that the students were given at the beginning of the semester. The PowerPoint slides and the instructor’s simultaneous audio commentary were recorded as they were delivered. Any discussion on a topic occurring during the lecture and annotations made to the slides were also recorded by the software. Therefore, when students reviewed the lecture, the instructor’s commentary, discussion, and annotations were synchronized with the slides. Students could rewind or fast forward the recordings, but the software did not permit searching by keywords or phrases.

The recordings were uploaded to a server at the institution and then were uploaded to Blackboard, the institution’s learning management software. Students were provided a link on the course Web site on Blackboard through which they had unlimited and unrestricted access to audiovisual recordings. This arrangement was made known to the students at the beginning of the course.

Student achievement was assessed by four tests during the semester. Performance on each test was measured by the proportion of correct responses. The final course score was a weighted average of the four scores where the weights were determined by principal components analysis. The distribution of class attendance was severely positively skewed because most students had few or no absences. To improve the extent to which the variable satisfied the required assumptions for multiple regression analysis, class attendance was made operational as the natural log of the proportion of classes missed.

Statistical Analysis

To estimate the impact of the intervention on student test performance while statistically controlling for the potentially confounding influence of class absence and educational background, multiple linear regression analysis was used to predict student test performance in the course based on instructional technology, class absence, measures of educational background (MCAT score and undergraduate GPA), and demographic characteristics (gender, race/ethnicity, age, and citizenship status). The statistical significance of the intervention and of other potential predictors was evaluated using t tests.

Essentially, performance was “adjusted” for differences in the strength of a student’s educational background so that every student’s adjusted performance is an estimate of the test performance a student would have achieved if they had an average background. This adjustment improves the rigor of the analysis and the precision of measuring the magnitude of the effects of the intervention and of class absence.

Results

Except for race/ethnicity, there were no statistically significant differences in demographic characteristics or educational background between the CON and IT cohorts (

Table 1. Demographic Characteristics of Students in the Two Experimental Cohorts

). χ [2] analysis showed a difference between the CON and IT cohorts in the distribution of race/ethnicity (χ [2]₄ = 9.99, P = .04). Hispanic and Asian students were disproportionately represented in the IT cohort. However, since test performance was not related to race/ethnicity, it is unlikely that differences in racial/ethnic distribution are a source of bias in the analysis. Mean proportion of classes missed (absenteeism) was slightly higher for the IT cohort (15.2%) than for the CON cohort (11.2%), and the difference was significant (t₂₄₄ = 2.30, P = .02).

Multiple regression analysis was conducted to evaluate the extent to which instructional technology was associated with mean test performance. The predictors of test performance were instructional technology, the interaction between the use of instructional technology and class attendance, biology MCAT subscore, and undergraduate GPA. The attendance variable was dropped from the analysis because it was unrelated to test performance (t₁₈₉ = 0.516, P = .61). Collectively, the predictors were significantly related to test performance (F_4,189 = 24.55, P < .001). The correlation between test performance and the predictors collectively was high (R = 0.59). Approximately 34% (R [2] = 0.34; see the note at the bottom of

Table 2. Regression Analysis Summary for Instructional Technology, Attendance, and Educational Background Predicting Test Performance

) of the sample variance in student performance could be accounted for by the predictors.

The intervention had a small negative effect (−0.59 SD) on mean test performance after controlling for educational background and class attendance. After controlling for educational background, absenteeism had a small negative effect (r =−0.17) on test performance only for the IT cohort (Fig. 1). The biology MCAT score (r =0.525) and the undergraduate GPA (r = 0.310) were found to be correlated with test performance.

Discussion

The present study provides an objective measure of the impact of the availability of technology on overall test performance in a traditionally difficult medical physiology course. This is a scenario in which the availability of such technology would seem highly appropriate. It also addresses the interaction of this technology with class absence. The general premise was that the performance of students would be improved by providing them with Blackboard-mediated access to audiovisual records of the lectures that they wanted to review. We also suspected that lecture capture technology would mitigate the effect of class absence by providing a virtual substitute for the missed class.

These data showed that technology had a small negative effect on mean test performance after controlling for educational background and class attendance (Table 2). Absenteeism was higher, on average, for the IT than the CON cohort. After controlling for educational background, class absence had a small negative effect on test performance only for the IT cohort and class absence was higher on average for this group. The mediating effect of instructional technology on the negative impact of absenteeism and the lower test scores for the IT cohort was a surprising finding since the IT cohort had more learning resources available to them than did the CON cohort. Consequently, the expectation was that more resources would lead to better test performance. Apparently, students did not use the technology effectively and were hurt by their “ineffective use.” Also unexpected was the finding that absenteeism did not hurt test performance for the CON cohort. An explanation for this may be that students in the CON cohort compensated for their absence from class in ways that were more effective, having developed these compensatory behaviors over several years of experience with traditional lecture pedagogy.

Although, it has been stated that the use of technology can enhance student understanding, [15] student participation, [16] and teamwork, [17] it has also been stated to be a hindrance to learning, [18] to cause diminished understanding due to the increase in cognitive load, [19] and to require students to spend more time learning and keeping up with technology. [20] The results of the present study tend to support the latter and do, indeed, raise the question of whether students in the IT cohort spent more time on mastering the technology than on mastering course subject matter and, consequently, performed poorly.

We speculate that class absence is a “proxy” for some unobserved influences on student performance, such as motivation and study time. Perhaps these factors were magnified by the technology, eg, the technology is a bigger impediment for those already hurt by their low motivation. It is well documented that motivational factors play a role in student performance both positively (conscientiousness and openess) and negatively (introversion and neuroticism). [2–4] The results of this study indicate that those same motivational factors are brought to bear in the use of technology and act similarly to produce the observed discrepancies in performance.

A limitation of this study is that it did not track the use of technology by students. Consequently, we have no evidence regarding how or how often students used the technology. Nonetheless, ineffective use explains the negative impact of IT and the negative effect of class absence only for the IT cohort. Student training in the technology used also needs to be part of the equation.

If students who are already at a disadvantage because of ineffective learning behaviors (such as missing class) are put at a further disadvantage by their ineffective use of the instructional technology, it seems that mere availability of technology is not enough to change student performance outcomes. Its appropriate use also needs to be addressed.

Another implication is that technology in its present form is not equally suitable for all academic settings. In the absence of formal training in the effective use of technology as a learning aid, students in medical programs may, indeed, do better with the tried and tested methods of information assimilation rather than be distracted by technology and, thus, have less time to use these “established” learning methods. Students’ high comfort level with technology should not be mistaken for the skillful use of technology.

Students generally perceive technology positively in terms of affective criteria. [21] However, when they are asked to rate the efficacy of technology in improving their performance, they rate it significantly lower. [21] The same study also reported that not all aspects of technology were equally liked or perceived to be effective. Thus, students are sufficiently discriminatory about what technology helps and what is just “eye candy.”

It would, therefore, seem appropriate that for technology to have the desired educational effect, it needs to be introduced with some training in its optimal use. Perhaps with the incorporation of these modifications in the teaching pedagogy, the expectations of improvements in student performance will then be realized.

In conclusion, providing technology to students did not show the expected improvement in course performance and, in fact, slightly diminished this performance. Furthermore, providing the technology slightly increased student class absence and created an environment in which class absence hurt test performance. More data on use of technology would help refine this conclusion. In the interim, we believe that training in technology use is required to reap the sought-after benefits of technology on student academic performance.