Next Article in Journal
Longitudinal Socio-Emotional Learning Intervention for Autism via Smartglasses: Qualitative School Teacher Descriptions of Practicality, Usability, and Efficacy in General and Special Education Classroom Settings
Previous Article in Journal
The Effects of Using Mobile Devices on Student Achievement in Language Learning: A Meta-Analysis

Educ. Sci. 2018, 8(3), 106; https://doi.org/10.3390/educsci8030106

Article
Using the Learner-Generated Digital Media (LGDM) Framework in Tertiary Science Education: A Pilot Study
Faculty of Science, University of Technology Sydney, Building 7, Thomas St, Ultimo, NSW 2007, Australia
*
Author to whom correspondence should be addressed.
Received: 28 June 2018 / Accepted: 8 July 2018 / Published: 24 July 2018

Abstract

:
Learner-Generated Digital Media (LGDM) has become prevalent in higher education. Frameworks have been developed for video-making in the classroom that consider technical requirements, pedagogies, and the combination of both. However, missing is a practical model to guide academics and students on the implementation of LGDM assignments. This research aims to test a model to design, implement, and evaluate LGDM as an assessment tool. The model was built based on research gaps and it considers the following elements: (1) pedagogy, (2) student training, (3) hosting of videos, (4) marking schemes, (5) group contribution, (6) feedback, (7) reflection, and (8) evaluation. For this purpose, five science subjects (N = 270) were used to test the model as a guide to implementing LGDM assignments. Data was gathered using a validated 33-step questionnaire instrument. Additionally, group contributions were received using the SPARKPlus peer review application, and marks attained were gathered. Methodological triangulation of the datasets suggested that students have a positive attitude toward LGDM for science learning. Students enjoyed the group work and creativity, and they identified digital media support as a critical component of their learning experience. Preliminary data support using the LGDM framework to design digital media assignments for science education.
Keywords:
Learner-generated digital media; video as an assessment tool; digital media; digital media literacies; blended media; digital media as an assessment tool

1. Introduction

Learner-Generated Digital Media (LGDM) can be defined as a digital artefact developed by students to learn the subject content [1]. This approach emerged in the field of education more than a decade ago [2,3,4]. Currently, it is gaining momentum in the higher education landscape [5,6,7]. Using digital media as an assessment tool has been made possible by the wide availability of digital applications [8] and electronic devices such as smartphones, tablets, video cameras, and so on [9,10]. These new technological tools create opportunities for new approaches to curriculum and pedagogies in the classroom [11,12,13].
The pedagogical approach behind LGDM use is the promotion of student reflection, engagement in active learning, fostering of collaboration and creativity [14], and the creation of an environment for deep learning [15,16]. Learner-generated content has the potential to add value to hands-on experience and peer-driven learning [17]. Other benefits of LGDM include the development of graduate attributes such as interpersonal communication skills, project planning and time management skills [18], critical thinking, report writing, research skills, and digital literacy [19]. Nevertheless, research on LGDM in higher education is considered to be in the embryonic stage [20,21,22]. Thus, there is a need for rigorous studies using large student samples to evaluate the effectiveness of LGDM in different disciplines [2,6,22,23].
The authors reviewed the research in LGDM using educational databases including Education Research Complete, ERIC (EBSCO), Education Database (ProQuest), A+ Education (Informit), and LearnTechLib. A research gap was identified in the literature: the need for a practical model linking pedagogy and technology to guide academics and students on the implementation of digital media assignments. Consequently, the aims of this research paper are (1) to use the LGDM framework to guide the implementation of digital presentations as assessment tools in tertiary learning and (2) to explore student perceptions about the use of Learner-Generated Digital Media (LGDM) assignments for learning scientific concepts.

2. Literature Review

Research conducted in the last decade in the field of education has described the use of digital media assignments. The main focus has been on reflective practices for pre-service teachers [24,25]. In contrast, in science disciplines, digital media assignments are a novel approach focusing on the development of research skills, inquiry, and active learning [22]. The affordability of new technology allowed digital media assignments to spread the repertoire of the traditional assignments such as writing a lab report, a scientific abstract, a literature review, and so on. Examples of digital media assignments have been documented in life sciences such as biology [26], health sciences [27], and pharmacology [28,29,30]. Other disciplines using digital media assignments include computer programming [31,32], geology [33], mathematics [34,35], and engineering [36]. There are challenges to designing digital media assignments in the science discipline due to the lack of systematic approaches and theoretical models to guide the implementation in the classroom. Additionally, educators outside the field of visual design, multimedia, filming, and digital media do not have a basic understanding of digital media production workflow and digital media principles.
Learning mediated by digital media assignments has not been rigorously explored. The semiotic theory described by Hoban et al. (2015), the self-explanation principle [37], and the internalisation principle [38] have been theorised as mediating learning when creating digital media assignments. When creating LGDM assignments, learning takes place in three different steps—preparing, representing, and reinforcing. When students prepare by searching for subject content to build their storyboards, they learn about the topic [26,39]. Representing occurs when they look at their scripts and think in a multimodal way to present the content [26,40,41]. Finally, reinforcing takes place during the digital media production task, which is iterative and time-consuming [42]. For example, creating an animation to explain a biochemical reaction will require students to prepare their storyboards [43]. Students will need to engage in reading and understanding the material before they can summarise it for the script. The next step will be to think about the best way to represent the reaction, whether it be a whiteboard animation [44], slowmation [45], or PowerPoint animation [46]. The final step will be to play and refine their animations until they run smoothly, which will reinforce student learning [47].
Existing models to guide the design and implementation of digital media assignments in the classroom focus on technical aspects such as development, pre-production, production, post-production, and distribution. In these technology-driven models, there is no emphasis on teachers’ and learners’ roles [25,40]. These models have been heavily influenced by professional video-makers and they lack pedagogical substance [43]. In teacher education, a nine-stage model that includes teacher strategies and peer-learning structures [2] has been proposed. Later, a learning design for learner-generated digital stories was proposed based on the previous model [25]. Although this framework is very comprehensive, it has been contextualised for teacher education, and extrapolation to other disciplines can be difficult. The CASPA model (Consume, Analyse, Scaffold, Produce, and Assess) [5] is a novel instructional design framework for implementing multimedia creation in the classroom. A recent study used the ICSDR model (Identify, Conceptualise and Connect, Storyboard, Develop, Review, Reflect, and Revise) to inform the use of LGDM in the classroom [6].
However, none of these models considered, for example, communicating to students the assessment task rationale, training the students in digital media principles, using an accurate marking rubric, ensuring healthy groupwork, or evaluating the intervention. Therefore, we identified that a simple model combining pedagogies, digital media training, video hosting, marking schemes, group contribution assessment, feedback, reflection, and evaluation was required to implement digital media assignments outside of the Education discipline. This paper uses the Learner-Generated Digital Media (LGDM) Framework [48] for the design of digital media as an assessment tool. This model is flexible enough to be applied to any digital media type, including a podcast, digital story, animation, video, and blended media. The LGDM Framework is student-centred because it helps students to understand the benefits of learning using digital media and guides them in how the digital assignment is structured. From the educator’s perspective, the LGDM Framework could be an excellent approach for designing digital media assignments in the classroom.

3. The LGDM Implementation Framework

The LGDM model has eight elements, starting with pedagogy and ending with an evaluation to inform future improvements (Figure 1). These elements were chosen based on previous models of digital media as an assessment tool and identified research gaps in the literature. Aspects of the LGDM Framework are explained above and linked to a set of questions which students need to understand before undertaking the digital media assessment. From the academic perspective, the model blends theory and good practice. From the student’s viewpoint, the model informs them about the benefits of learning using digital media and about how the assessment task is structured. Communicating this information is crucial to ensure that students will buy into the task and to set clear expectations of the requirements for success in the assessment task.

3.1. Pedagogy

The starting point of LGDM assignments is the pedagogic and instructional strategies. Most of the research in the field of LGDM is guesswork and does not rest on a solid educational foundation. In other words, educators improvise digital media assignments without taking into consideration that the starting point is the pedagogical approach, for example, students working together using Active Learning [49], Problem-Based Learning [50], Collaborative Learning [51], Cooperative Learning [52], and Peer-Assisted Learning [53]. The objective is for students to engage with the subject content and use the technology as a vehicle for learning. When developing LGDM assessments, it is also essential to align the subject learning objectives with graduate attributes and with the digital media tasks the students will undertake [54,55].

3.2. Student Training

Storyboard creation, digital media principles, and production technique training have in most cases been neglected in LGDM research. Several studies in different disciplines did not consider student training and support [5,6,15,25,27,56]. The assumption is always that students know more than educators regarding technology because they own it and use it more in everyday life [22]. However, the literature reported student apprehension, anxiety, and poor digital media skills in LGDM assignments [27,57,58]. The issue with technology in first world countries is not ownership or access, but fluency in its use [12,59]. Our framework identified digital media support for students as essential. For this purpose, training on how to create compelling digital presentations needs to be planned and delivered to students. The Digital Media Literacies Framework (DMLF) [1] is used to train students in the conceptual domain (storyboard), functional domain (software), and audiovisual domain (digital media principles). Intellectual property and copyright issues must also be considered in training to ensure that students understand their importance and application to their digital media projects.

3.3. Hosting and Distribution

Research has described the importance of audience awareness to get students motivated about LGDM assignments [2,26,38]. The prospect of creating LGDM content that can help the learning of students elsewhere gives students a sense of agency and students reported high levels of accomplishment and ownership from digital media assignments [25]. Hosting video on open platforms such as YouTube and Vimeo [60,61] can also promote student agency and accomplishment. Using an open platform to upload the LGDM assignments can provide student access even after they finish their studies, so that they can showcase their assignments as evidence of teamwork, digital media skills, and achievement of graduate attributes highly regarded in the workforce [26,55]. Nevertheless, some higher education institutions are moving to closed video platforms such as Kaltura. Uploading LGDM assignments to open platforms is a vital feature of student-created digital media.

3.4. Marking Schemes

Digital media production is time-consuming [62], iterative [63], and resource-intensive [64], with variations depending on the media type [65]. For example, creating an audio podcast is less onerous than creating a digital story or video. Educators need to have a good understanding of the complexity of different digital media types before designing LGDM assignments. The Taxonomy of Digital Media Types Framework is used [65] to guide assignment design and weighting. For example, for a video group assignment, it is recommended that it be worth at least 20% of the total subject mark [28]. Finally, the development of marking rubrics that consider communication in the digital space is crucial to ensure that the effort students put into their assignments is recognised.

3.5. Group Contribution

Digital media production is teamwork by nature. It is unrealistic to ask students to produce, for example, video individually. Some media types, such as an audio podcast or blog posting, could be developed individually because they are less complicated. A mechanism needs to be implemented to ensure groupwork is optimal and that all group members contribute to their projects. The best approach, in this case, is self and peer-assessment [66,67]. Development of a marking rubric for contributions to group work will be necessary, as well as a peer-review application allowing students to rate each other’s contributions to the project. SPARKPlus is an example of a group work moderation application that provides feedback and quantitative ratings for group contributors [33]. The use of other tools, such as Google Forms or even paper-based forms in small classrooms, could substitute for online peer-review tools.

3.6. Feedback

Feedback on LGDM assignments is crucial from two perspectives: content and digital media production. Students need to feel supported, and this has a positive effect on their engagement with the task. As theorised previously, the purpose of feedback is to reduce discrepancies between understanding, performance, and the goal [68]. Students will need feedback on the storyboard structure early in the semester. Week 2–3 is preferable. Then, they will need feedback on the digital media tools they are planning to use. It is ideal to provide feedback on students’ prototypes two to three weeks before assignment submission. This level of feedback will allow students to produce a useful digital artefact and minimise anxiety about the LGDM task [27,58].

3.7. Student Reflection

Research has shown that when students do not reflect on a learning task using technology, they do not see the value of it [69]. Adding a reflective task after the assignment will help the students to rethink if they have gained additional knowledge by engaging in the digital media project. This can be implemented by using a reflective journal inside the Learning Management System and by asking students questions in the classroom such as “What do you feel you learned from this task?” and “How could you use the skills you developed?”

3.8. Evaluation

Evaluation is an integral part of any educational intervention. The purpose of the evaluation is to produce data that will help to improve the LGDM assignment in the next iteration. The process of evaluation involves (1) identifying the activity/task, (2) developing questions (for students and tutors), (3) determining the sources of data, (4) collection and analysis, (5) making the adjustments required, and (6) starting again. Sources of data can be teacher reflections, student perceptions (via surveys, interviews, and focus groups), student assessment performance (grade attained), and student action (group contribution) (Phillips & Gilding, 2002).

4. Materials and Methods

This research project used a mixed-methods approach [70]. A 33-step online questionnaire (28 Likert scales and five open-ended questions) was used to gather data. The questionnaire items were validated using factor analysis [71]. Factor analysis is a statistical methodology that allows the researcher to test the hypothesis that there is a relationship between observed variables and their underlying latent constructs. The research also collected group contribution data (SPARKPlus) and grades attained. Methodological triangulation [72] of datasets was performed to confirm student perceptions of LGDM as an assessment tool. Questionnaire data were analysed using IBM Corp. Released 2016. IBM SPSS Statistics for Windows, Version 24.0. Armonk, NY, USA: IBM Corp., and open-ended questions were analysed using NVivo (Version 11, QSR International, Melbourne, Australia, 2016).

4.1. Selection of Subjects

The study was conducted during Spring 2016 and included five Science subjects (Table 1).
The Faculty’s learning designer liaised with subject coordinators to adopt authentic assessments for the University’s strategic implementation of blended learning. LGDM assignments are considered authentic because (1) students need to research to write their storyboards, (2) they require teamwork and applied problem-solving and communication skills, and (3) they reflect tasks that students will face in their professional life as scientists—for example, showcasing research using digital media to communicate to the scientific community or to attract funding. Subject coordinators from ten subject areas were presented with the LGDM framework and had the process to design the assessment task explained to them. Five of them decided to implement the task and be part of the pilot study.

4.2. LGDM Assessment Design

Design, implementation, and evaluation of the digital media assignments was done using the LGDM Framework. Pedagogy was the starting point of the design process. Active learning and small group work were the drivers of the assignment. Constructive alignment ensured that the digital media task addressed the subject learning objectives and the Faculty of Science’s graduate attributes—for example, (1) Disciplinary knowledge and its appropriate application, (2) An inquiry-oriented approach, (3) Professional skills and their appropriate application, and (4) Communication skills.
Student training was carefully designed and delivered via a face-to-face lecture on “Basic Digital Media Principles to Communicate Science”. The lecture followed the Digital Media Literacies Framework (DMLF) and addressed the conceptual, functional, and audiovisual domains [65]. The conceptual domain was addressed by introducing principles of storyboarding, while the functional domain was covered with a general explanation of how video editing works in Movie Maker (Windows platform) and iMovie (Macintosh). The audiovisual domain was addressed using the Digital Media Principles Framework [73]. This section of the lecture covered concepts such as layout design, colour theory, typography, use of images, and basic video principles. A workshop on storyboarding for digital media creation was also delivered to the students. In this session, students were able to apply the conceptual domain discussed in the classroom and develop a preliminary structure for their storyboards. Online resources on digital media creation were embedded inside the Learning Management System (LMS) to further support students with the LGDM assignment. For the subject Introductory Pharmacology and Microbiology, an interactive module on “Visual Design Principles to Communicate Science” was deployed inside the LMS. This subject had a different type of digital media task, creating a poster/brochure, and did not have time to allocate to a lecture and workshop.
Video hosting was on YouTube, as our institution supports it. Individual YouTube accounts for each subject were created, and instructions were developed for students on how to upload their videos to the subject channel. Video-sharing services were considered the best for students to upload their LGDM assignments to and also watch videos created by their peers. This also helps tutors to mark the videos quickly, as they are all in one place.
The project developed a comprehensive marking rubric for each of the subjects. The rubrics aligned with the subject learning objectives and graduate attributes. The intervention used the SPARKPlus application, a peer-review tool, to ensure strong group contributions. SPARKPlus helped students to focus on the task and contribute to their groups. A basic marking rubric for group work was designed and used across all subjects, with the following criteria: (1) subject input for the project; (2) punctuality and time commitment, (3) contribution of original ideas, (4) communication skills and working effectively as part of the team, and (5) focus on the task and what needs to be done.
Students developed a structure for their storyboards during the workshops and feedback was provided by the subject coordinator and the learner designer during weeks 1 and 2 of the semester. Students also had access to a page in the LMS that contained the following sections: (1) a “welcome to LGDM assignments” video, (2) an interactive lecture on digital presentations/brochure design, (3) frequently asked questions on digital media assignments, (4) examples of LGDM developed in previous years, (5) the marking rubric, and (6) instructions on how to upload digital presentations to YouTube channels.
Toward the end of the semester, when students had completed their digital media projects, they were prompted to reflect about their experience in the classroom. Students discussed with their peers the importance of using digital media to learn and communicate science. This data was not captured for the research, as it would require observational studies, video or audio recordings, and further analysis. Finally, in the last two weeks of the semester, students were asked via announcements to participate in the online questionnaire.

4.3. Data Gathering and Questionnaire Design

Student attitudes toward LGDM for learning were captured with an online questionnaire (33-step questionnaire, Likert scale) that considered demographics, digital media support, attitude toward technology, understanding of the assignment, knowledge construction, and open-ended questions (Table 2). Factor analysis was performed to validate the questionnaire items.
Data from the online questionnaire were analysed using frequencies and descriptive statistics and combined for the five subjects, as there were no significant statistical differences between the subjects. The software IBM Corp. Released 2016. IBM SPSS Statistics for Windows, Version 24.0. Armonk, NY: IBM Corp., was used to analyse the data. For the open-ended questions (Q29–Q33), thematic analysis was used to find the categories, and open-ended question responses from all subjects were consolidated using NVivo (Version 11, QSR International, 2016). SPARKPlus results for group contribution and grades attained were analysed using descriptive statistics. Data were interpreted using methodological triangulation [72].

5. Results

5.1. Demographics

The overall sample was 62.8% female and 37.2% male. Most participants were in the age bracket of 18–29 (87.2%), with 30–49 (11.3%) and 50–64 (1.5%). Sixty-five percent were high school graduates, and 24% already had a university degree. Twenty percent of participants had English as an Additional Language (EAL). Table 3 presents the detailed demographic characteristics of participants.

5.2. Questionnaire Validation Using Factor Analysis

Factor analysis (FA) (N = 270) was performed to verify the factor structure of the questionnaire items. The online questionnaire aimed to gauge (1) digital media support, (2) attitude toward technology, (3) understanding of the assignment, and (4) knowledge construction. The extraction method used was ‘principal components’. The Kaiser-Meyer-Olkin (KMO) Test for Sampling Adequacy was calculated at 0.909, which indicated that the sample size was suitable. Bartlett’s Test of Sphericity was calculated as p < 0.001, which allowed us to conclude that there were relationships between the variables. Table 4 presents the loading factors for the questionnaire items tested. Items 10 (Overall, the technical support to complete my project was good), 20 (Overall, I was happy about the digital media presentation assignment), and 24 (The digital presentation helped me to work as a part of a team) had low loading factors and were withdrawn from the analysis.

5.3. Attitude toward Technology

This section of the questionnaire had six items measuring the use of technology for personal or recreational matters and learning. Ninety-one percent of participants agreed that they enjoy using technology for personal/recreational matters, 85% were confident using technology for that purpose, and 93% had a positive attitude toward technology for personal/recreational matters. Similar results were observed when participants were asked about enjoying technology for learning (93%), their confidence in using technology for learning (85%) and having a positive attitude toward technology for learning (93%). Table 5 summarises the frequencies of student responses. Figure 2 is a visual representation of the data.

5.4. Digital Media Support

This data does not include the subject Introductory Pharmacology and Microbiology (N =34), as the delivery of the training for this subject was entirely online. Instead, we recorded module completion rates by looking at the Learning Management System (LMS) logs at the end of the semester. Eighty-two percent of students completed the interactive modules, but their questionnaire did not contain digital media support questions. Additionally, records from the LMS were gathered to track student activity within the supporting material in the digital media tab. Data showed extensive engagement with the support content for the LDGM assignment.
Eighty-six percent of participants found the digital presentation lecture engaging, while 88% applied concepts from the lecture to their assignments, including storyboarding (73%). Eighty percent of students thought they would recommend the digital media lecture to their peers. Finally, 73% of participants believed they needed a better understanding of digital presentation principles (Table 6, Figure 3).

5.5. Understanding the Assignment

Eighty-five percent of participants thought the instructions for the digital media assignment were clear, while 91% thought the timeframe to complete the assignment was good. Ninety-seven percent of participants understood the importance of communicating concepts/ideas in the digital world (Table 7, Figure 4).

5.6. Knowledge Construction

Of the participants, 82% thought the digital presentation assignment helped them to understand the topic, develop critical thinking skills (75%), develop communication skills (86%), exercise creativity (90%), and learn additional skills (83%). Seventy-five percent of students thought digital presentations were an excellent way to assess students’ understanding of the topic. Seventy-six percent of students would encourage academics to use similar assignments in other subjects (Table 8, Figure 5).

5.7. Open-Ended Questions

Responses to open-ended questions (Q29–Q33) were analysed using thematic analysis (NVivo 11), coded, and classified. For the first open-ended question Q29, (Did you experience any issue with the assignment?) we received 163 responses (60%). Table 9 presents the frequencies of student responses.
For the second open-ended question (Q30 What did you like most about the assignment?), we received 169 responses (62%). Table 10 presents the frequencies of student responses.
Quotes from students are presented below:
Student 1
“To be able to be creative rather than just another boring report assignment. It was also a fun way to present information and learn about the content. Sufficient time was also given to complete the assignment which was great!”
“I like that there were a few topics that had current cultural/social relevance, e.g., study drugs in universities, abortion drugs, cannabis issues, etc. Would be great to see if all the topics could have some real world social relevance to things happening currently in society and being addressed in the media”.
Student 2
“The most enjoyable aspect was definitely the group work (which is rare). It was simply fun to come up with ideas on how to include a joke or how to present a particular point. It’s rare that we as students get such enjoyable and fun assignments at this late point in the degree. A much-needed breath of fresh air”.
Student 3
“The entire concept was brilliant. At first, I was somewhat skeptical about how useful such an approach would be in learning about a topic. However, I found that by essentially forcing students to break down the general concepts of a given topic for ease of explanation for those of a non-science background, it made understanding the more complex components much easier to not just learn, but actually remember. This was particularly helpful for the final exam in that it made study that much easier, with an incredibly solid foundation on at least one area of study”.
For the third open-ended question (Q31 What did you like least about the assignment?), we received 143 responses (52%). Table 11 presents the frequencies of student responses.
Quotes from students are presented below:
Student 4
“Group work is always a challenge in that it is hard to know how each individual works and different styles and ways people work”.
Student 5
“Groups of four is too big. The assignment should be in pairs so that people learn better. Often groups of four end up affording at least one student to be slacking off”.
Student 6
“The effort and time you have to put into making a 5-min video is a lot, you have to make a script, figure out where and when you’re going to film (if you’re filming), edit etc. It’s a lot of time that goes into it. And when there are a lot of other subjects’ projects going on at the same time, it’s a hard thing to do. With all that effort put in the amount it was worth for the subject was very small, and it seemed like a lot of wasted time”.
For the fourth open-ended question (Q32 Do you have any feedback on how to improve this assignment?), we received 129 responses (47%). Table 12 presents the frequencies of student responses.
Quotes from students are presented below:
Student 7:
“No, it seemed extremely fluid and was easy to understand what was needed to be done, the assignment allowed students to take control of their learning which I believed got us to be more engaged and keen to produce a good piece of work. It helped in my case in particular that I was quite interested in the topic chosen”.
Student 8:
“It was the best assignment I have done throughout my whole university degree and believe other subjects should take a similar approach to learning”.
Student 9:
“I think that allowing the students to focus on the social issues associated with drug use and prescription results in a much greater level of critical thinking on the issues. It is too easy to ask a student to present how a drug functions. Asking ethical questions, the answers to which aren’t clear-cut or can be found with a simple google search, ensures a greater level of engagement”.
For the last open-ended question (Q33 Is there anything that you would like to say that has not been covered in the previous questions?), we received 55 responses (20%). Table 13 presents the frequencies of student responses.
Quotes from students are presented below:
Student 10
“Really great assessment, again speaking personally, not having any group troubles meant that this assignment was a breeze and a pleasure to complete (something that I would not have seen myself saying about a pharmacology assignment!)”.
Student 11
“Both the subject coordinator and the digital media person were excellent; they showed a genuine interest in the education of their students”.
Student 12
“I strongly support the idea to use digital media for some part of course studies or assignments. Skills on using technologies are important skills for students when they seek employment or develop their own business”.

5.8. Group Contribution Data

Group contribution data was captured using the SPARKPlus application for three subjects—Geological Processes, Introductory Pharmacology & Microbiology, and Pharmacology 2. The two other subjects had some technical issues implementing the application, and there was missing data. The Relative Performance Factor (RPF) is a measure of the degree of contribution to group work. This factor is calculated from a peer review of group members. Table 14 presents descriptive statistics for the three subjects. Table 15 shows the percentages of students who had optimum, acceptable, or poor performance. Only 6.3%, 7.5%, and 3% of students had a poor contribution level for the subjects Geological Processes, Introductory Pharmacology & Microbiology, and Pharmacology 2, respectively.

5.9. Grades Attained

Grades were corrected, using the RPF factor, for students who had RPF < 0.8. For example, if a student had an RPF = 0.6, the assessment task was 30 marks, and the group got 25 marks, the final mark would be 25 × 0.6 = 15 marks. The marks were converted to percentages to see how students performed across the different subjects (Table 16). Grade comparison was not possible, as every subject had slightly different rubrics, different learning objectives for the digital media assignment, and various different markers. The marks data were used to triangulate questionnaire responses. The majority of students thought they learnt from the assessment task, as shown by the findings on the knowledge construction data from the questionnaire (Table 8, Graph 4). Table 16 shows that this was the case for most students by displaying the means and standard deviations of marks attained.

6. Discussion

This study is one of the first which has used a comprehensive and practical framework to systematically approach the design, implementation, and evaluation of LGDM assignments. Previous studies did not use frameworks to guide the implementation of LGDM assignments [27,58,74,75]. Other studies used semiotic theory to conceptualise learning with digital media, but included no model to guide the task design, implementation, and evaluation [29,39,76,77]. However, most of these studies are restricted to qualitative surveys and open-ended questions [15,25,27,56] or purely qualitative comments from interviews [43,78]. While students’ perspectives provide a valuable dataset in educational research, they cannot be relied on solely to evaluate an intervention. Along with small sample size (from 3 to 79 students), the lack of standardised evaluation approaches and the qualitative nature of these investigations make comparisons between the current study and previous studies problematic. Also, different media types used in LGDM require different production skills, whether they be audio podcast, digital story, animation, screencast, or video [65], adding an extra layer of complexity when comparing studies.
The demographics of participants in this study showed a high percentage of females (63%) from 18 to 29 years old (87%) and high school graduated (65%). They had a positive attitude toward technology for personal/recreational use and for learning (Table 5). Their gender, age, and socioeconomic status could influence their perceptions [79,80,81]. These results could reflect student exposure to the use of technology for learning at high school, giving them a positive attitude to the LGDM assignment. With the current data, we cannot elucidate if this was the case. Focus groups and individual interviews would be of value.
Triangulating the data from the questionnaire items, the open-ended questions, the group contribution data, and the grades attained indicated that the use of the LGDM Framework enhanced the student learning experience with digital media assignments. Current literature in science education has found similar results, using qualitative surveys and interviews [82,83,84]. Overall, student attitude toward digital media support was highly positive. Seventy-three percent of students used a storyboard to inform their digital media projects. Storyboards are essential to ensure the production of a quality digital artefact [85] but also to develop conceptual skills for digital media production [38,65,86]. These digital media skills have been highlighted as desirable graduate attributes across all disciplines in the 21st century [54,87,88]. Nevertheless, in LGDM assignments the primary benefit of producing a storyboard is to learn the subject content. Producing a storyboard requires students to search for information, think critically, and summarise their findings, and this is the first step of learning through digital media production [65]. Seventy-three percent of students thought they needed a better understanding of digital media principles. This finding was confirmed by responses to open-ended Q31, where 20 students (14%) said they had poor digital media skills, and to Q32, where 31 students (24%) said they needed additional software training and 12 students (9%) said they needed additional instructions for the assessment. Understanding digital media principles from a one-hour lecture can be overwhelming, and students need post-lecture online activities to reinforce the concepts covered, for example, an annotated online video to highlight the digital media principles applied. Tools such as Kaltura can be used for this purpose. These findings open opportunities to further engage students in additional training during the semester and maybe run digital media drop-in clinics.
Research on LGDM assignments does not often include providing digital media support to students [29,76,77]. There are problems with assessing students on a skill that is not formally taught and relying on the myth of ‘digital natives’ that postulated that young students who grew up in the digital age are fluent with the use of technology. This notion has been disproved in the literature [89]. Current research in the field of digital literacy has identified that the issue with technology is not ownership or access, but fluency of use [59]. Research has pointed to the need for student support with LGDM assignments [27,54,58]. In this regard, the LGDM Framework offers extensive support to students in acquiring digital media production skills like storyboarding and includes training on software and digital media principles.
An important aspect missing in the current literature on LGDM assignments is the opinions of students on the adequacy of the digital media assignment instructions. In this study, a high percentage of students seemed to understand the assignment (Table 7, Graph 3), although open-ended question Q29 showed that 11 students (7%), and Q31 showed that 14 students, (10%) had issues understanding the assignment. Perhaps those students did not come to the lectures or visit the digital media resources. Communicating the task well is essential because it has been reported that students can become anxious about digital media assignments [57,58,90,91]. Explaining the assignment at the beginning of the semester using the LGDM Framework (Figure 1) and providing early feedback on storyboards from the content and digital media perspectives seem to have a positive impact on students’ engagement. None of the responses to the qualitative questions (Q29 to Q33) showed the student task anxiety described in other previous studies [27,55,57,58]. Ninety-seven percent of students understood the importance of communicating concepts/ideas in the digital world. This figure is crucial, as this understanding could act as a factor motivating students to self-regulate their learning using LGDM assignments. Motivational factors such as self-efficacy, goal orientation, task value, attribution for failure, and anxiety are considered in educational psychology the sine qua non of self-regulation processes [92].
Students’ attitudes toward LGDM and knowledge construction were highly positive (Table 8, Graphic 4). These findings were confirmed by Q30, where students reported positive attitudes about LGDM regarding creativity (28%), development of teamwork (23%), learning of subject content (10%), and learning about digital media (21%) (Table 10). For the last open-ended question, Q33, 55% of students gave positive comments on LGDM, reinforcing these findings. Creativity was a feature of LGDM reported approvingly by students previously [27,57], as was teamwork [58]. Learning of subject content [26,78,93] and learning about digital media [58] were also highlighted as attractive by previous research. Triangulating the data from this study with the marks attained by participants, it seems that students had an overall positive learning experience using LGDM as an assessment tool. The data available on group contributions (SPARKPlus) (Table 14 and Table 15) showed students had positive group work experiences. On average, 94% of students had a healthy groupwork experience. These findings are reinforced by the responses to open-ended question Q30, where twenty-three percent of students said they enjoyed the teamwork. These results are similar to results previously reported for LGDM in science education [27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58], but those studies only used qualitative responses to open-ended questions. The present study is the first to gather group contribution data from the whole cohort of students by using the SPARKPlus peer-review application in LGDM assignments. We postulated that, due to the nature of digital media projects where group members have different roles, students bring different skills complementing each other to the project. A focus group would be required to elucidate if this is the case. Responses to Q31 (What did you like least about the digital media assignment?) contradict these findings, as 22% of students’ responses to the survey mentioned group issues. Students who perceived they did not perform well in groups or did not like the idea to work in groups could contribute to this figure. It could also be the case that students receive good SPARKPlus feedback from their peers, but somehow, they felt unsatisfied. Analysing these responses further, three themes were found. Communicating ideas with the group, members’ availability, and group conflicts were the issues highlighted by students. Students that contributed to this figure with the open-ended question could be the ones who did not perform well in groups or who were not satisfied with it. Educators often ask students to work in groups, but training on how to work in groups effectively is usually overlooked. A video of ‘how to’ tips on working in groups could provide support to students in the future.
This study had several limitations which minimise the generalisability of its findings. First, the sample only included science students from a single institution. Future research should consider using large cohorts from a wide range of disciplines and university settings to provide more generalisable findings. Second, the study did not use interviews and focus groups to gain an in-depth understanding of students’ attitudes toward LGDM assignments. Third, it would be ideal to compare cohorts of students who developed LGDM assignments using the LGDM Framework with those who did not, to elucidate if there are differences. However, such research would be challenging to design and implement because it could potentially disadvantage some students. Finally, interviews with academics on their perceptions of the validity of the LGDM framework would add an all-around perspective to the results.
The next step in this project is to map the LGDM Framework against self-regulation subscales [94] and measure how students adapt to learning with LGDM assignments. Because LGDM assignments require a high level of autonomy [38] and are time-consuming [63], iterative, and resource-intensive processes [64], self-regulation could be a useful theoretical model for further research in the field [95,96]. Understanding how students self-regulate their learning with LGDM would allow educators to help students acquire and master the necessary skills [95,96] and increase the personalisation of the learning experience [97]. Investigating the self-regulated learning processes of LGDM could lead to future research exploring group work dynamics and co-regulation. It is required to understand self-regulation first as in group work students bring these skills that will affect the group dynamics and therefore how they co-regulate with their peers.
With the affordability of digital technologies and new assessment tools, the challenge now is how to embed them in curricula and how to evaluate their impact on student learning and performance. Higher education institutions should be encouraged to adopt a systematic approach to introducing LGDM assignments. The LGDM Framework offers educators the opportunity to align digital media tasks to learning outcomes and graduate attributes, to plan student training on digital media principles and production, to ensure effective teamwork, to develop rubrics for evaluation, and so on. From the learner’s perspective, LGDM assignments require further consideration for successful implementation. For example, 50% of students have never produced a video for assessment purposes [27], and they are more familiar with written tasks. Student training and scaffolding in storyboarding, digital media principles, and digital media production is essential to support them when undertaking the assessment task and learning experience. The LGDM Framework has been designed as a student-centred approach to engaging students with their learning while also developing digital media literacy. Also, considering intellectual property and copyright issues in the student training about LGDM assignments [61,98] will help foster ethical behaviour in the digital space. Further studies across different disciplines and university settings to further develop the LGDM framework would be highly desirable.

7. Conclusions

Methodological triangulation of the datasets shows evidence that the LGDM Framework helped students to learn using digital media by communicating the assessment design, scaffolding their learning experience, putting in place a mechanism to ensure effective group work, and providing them with relevant feedback. The present study is the first to use a systematic approach to LGDM assignments that included communicating the task to students, formal student training, mechanisms to ensure group contribution, and evaluation of the learning experience. Academics can use the validated survey developed for this study to improve their LGDM assignments. In conclusion, students have a generally positive attitude toward LGDM as an assessment tool. Students highlighted creativity, teamwork, digital media support, learning of subject content, and self-expression as the main features of the assessment.

Author Contributions

Conceptualization, J.R.; Formal analysis, J.R.; Investigation, J.R.; Methodology, J.R.; Supervision, P.M.; Validation, J.R.; Writing—original draft, J.R.; Writing—review & editing, J.R. and P.M.

Funding

This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Reyna, J.; Hanham, J.; Meier, P. Theoretical Considerations to Design Learner-Generated Digital Media (LGDM) Assignments in Higher Education. In Proceedings of the 12th annual International Technology, Education and Development Conference, Valencia, Spain, 5–7 March 2018. [Google Scholar] [CrossRef]
  2. Kearney, M.; Schuck, S. Students in the director’s seat: Teaching and learning with student-generated video. In Proceedings of the Ed-Media 2005 World Conference on Educational Multimedia, Hypermedia and Telecommunications, Montréal, QC, Canada, 27 June–2 July 2005. [Google Scholar]
  3. Crean, D. QuickTime streaming: a gateway to multi-modal social analyses. In Proceedings of the Apple University Consortium Conference, James Cook University, Townsville, Australia, 23–26 September 2001. [Google Scholar]
  4. Ludewig, A. iMovie. A student project with many side-effects. In Proceedings of the AUC Conference, James Cook University. Townsville, Australia, 23–26 September 2001. [Google Scholar]
  5. Blum, M.; Barger, A. The CASPA Model: An Emerging Approach to Integrating Multimodal Assignments. In EdMedia: World Conference on Educational Media and Technology 2017; Johnston, J.P., Ed.; Association for the Advancement of Computing in Education (AACE): Washington, DC, USA, 2017; pp. 709–717. [Google Scholar]
  6. Campbell, L.O.; Cox, T.D. Digital Video as a Personalized Learning Assignment: A Qualitative Study of Student Authored Video Using the ICSDR Model. J. Scholarsh. Teach. Learn. 2018, 18, 11–24. [Google Scholar] [CrossRef]
  7. Spicer, S. Perspectives on the Role of Instructional Video in Higher Education: Evolving Pedagogy, Copyright Challenges, and Support Models. In The Routledge Companion to Media Education, Copyright, and Fair Use; Routledge: New York, NY, USA, 2018; pp. 37–58. [Google Scholar]
  8. Reynolds, C.; Stevens, D.D.; West, E. “I’m in a Professional School! Why Are You Making Me Do This?” A Cross-Disciplinary Study of the Use of Creative Classroom Projects on Student Learning. Coll. Teach. 2013, 61, 51–59. [Google Scholar] [CrossRef]
  9. Devine, T.; Gormley, C.; Doyle, P. Lights, Camera, Action: Using Wearable Camera and Interactive Video Technologies for the Teaching and Assessment of Lab Experiments. Int. J. Innov. Sci. Math. Educ. 2015, 23, 22–23. [Google Scholar]
  10. Nilsen, S. Use of a GoPro® camera as a non-obtrusive research tool. J. Play. Pract. 2017, 4, 39–47. [Google Scholar] [CrossRef]
  11. Handley, F.J. Developing Digital Skills and Literacies in UK Higher Education: Recent developments and a case study of the Digital Literacies Framework at the University of Brighton. UK Publ. 2018, 48, 109–126. [Google Scholar] [CrossRef]
  12. Becker, S.A.; Pasquini, L.A.; Zentner, A. Digital Literacy Impact Study: An NMC Horizon Project Strategic Brief. 2017, The New Media Consortium. In Rebooting Learning for the Digital Age: What Next for Technology-Enhanced Higher Education? Davies, S., Mullan, J., Feldman, P., Eds.; Higher Education Policy Institute: Oxford, UK, 2017; pp. 1–58. [Google Scholar]
  13. Davies, S.; Mullan, J.; Feldman, P. Rebooting Learning for the Digital Age: What Next for Technology-Enhanced Higher Education? Higher Education Policy Institute: Oxford, UK, 2017. [Google Scholar]
  14. Barra, E.; Herrera, S.A.; Caño, J.Y.P.; Vives, J.Q. Using multimedia and peer assessment to promote collaborative e-learning. New Rev. Hypermedia Multimedia 2014, 20, 103–121. [Google Scholar] [CrossRef][Green Version]
  15. Cox, A.M.; Vasconcelos, A.C.; Holdridge, P. Diversifying assessment through multimedia creation in a non-technical module: reflections on the MAIK project. Assess. Eval. High. Educ. 2010, 35, 831–846. [Google Scholar] [CrossRef]
  16. Hamm, S.; Robertson, I. Preferences for deep-surface learning: A vocational education case study using a multimedia assessment activity. Australas. J. Educ. Technol. 2010, 26, 961–965. [Google Scholar] [CrossRef]
  17. Berardi, V.; Blundell, G.E. A learning theory conceptual foundation for using capture technology in teaching. Inf. Syst. Educ. J. 2014, 12, 64–73. [Google Scholar]
  18. Morel, G.; Keahey, H. Student-generated multimedia projects as a multidimensional assessment method in a health information management graduate program. In Proceedings of the Society for Information Technology and Teacher Education International Conference, Savannah, GA, USA, 21 March 2016; Chamblee, G., Langub, L., Eds.; Association for the Advancement of Computing in Education (AACE): Chesapeake, VA, USA, 2016. [Google Scholar]
  19. Ohler, J. New-media literacies. Academe 2009, 95, 30–33. [Google Scholar]
  20. Hakkarainen, K. A knowledge-practice perspective on technology-mediated learning. Int. J. Comput. Support. Collab. Learn. 2009, 4, 213–231. [Google Scholar] [CrossRef][Green Version]
  21. Potter, J.; McDougall, J. Digital Media, Culture and Education: Theorising Third Space Literacies; Palgrave Macmillan: London, UK, 2017; ISBN 978-1-137-55315-7. [Google Scholar]
  22. Hoban, G.; Nielsen, W.; Shepherd, A. Student-Generated Digital Media in Science Education: Learning, Explaining and Communicating Content; Routledge: Abingdon-on-Thames, UK, 2015; pp. 1–254. [Google Scholar]
  23. Duffy, T.M.; Jonassen, D.H. Constructivism and the Technology of Instruction: A Conversation; Routledge: Abingdon-on-Thame, UK, 2013; pp. 1–232. [Google Scholar]
  24. Rich, P.J.; Hannafin, M. Video annotation tools technologies to scaffold, structure, and transform teacher reflection. J. Teach. Educ. 2009, 60, 52–67. [Google Scholar] [CrossRef]
  25. Kearney, M. Learner-generated digital video: Using Ideas Videos in Teacher Education. J. Technol. Teach. Educ. 2013, 21, 321–336. [Google Scholar]
  26. Pirhonen, J.; Rasi, P. Student-generated instructional videos facilitate learning through positive emotions. J. Biol. Educ. 2017, 51, 215–227. [Google Scholar] [CrossRef]
  27. Pearce, K.L.; Vanderlelie, J.J. Teaching and evaluating graduate attributes in multimedia science-based assessment task. In Proceedings of the Australian Conference on Science and Mathematics Education, Brisbane, Australia, 28–29 September 2016. [Google Scholar]
  28. Reyna, J.; Meier, P.; Geronimo, F.; Rodgers, K. Implementing Digital Media Presentations as Assessment Tools for Pharmacology Students. Am. J. Educ. Res. 2016, 4, 983–991. [Google Scholar]
  29. Nielsen, W.; Hoban, G.; Hyland, C. Pharmacology Students’ Perceptions of Creating Multimodal Digital Explanations. Chem. Educ. Res. Pract. 2017, 18, 329–339. [Google Scholar] [CrossRef]
  30. Henriksen, B.; Henriksen, J.; Thurston, J.S. Building Health Literacy and Cultural Competency through Video Recording Exercises. INNOVATIONS Pharm. 2016, 7, 1–2. [Google Scholar] [CrossRef]
  31. Powell, L.; Robson, F. Learner-generated podcasts: a useful approach to assessment? Innov. Educ. Teach. Int. 2014, 51, 326–337. [Google Scholar] [CrossRef]
  32. Vasilchenko, A.; Green, D.P.; Qarabash, H.; Preston, A.; Bartindale, T.; Balaam, M. Media Literacy as a By-Product of Collaborative Video Production by CS Students. In Proceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education, Bologna, Italy, 3–5 July 2017. [Google Scholar] [CrossRef]
  33. Reyna, J.; Horgan, F.; Ramp, D.; Meier, P. Using Learner-Generated Digital Media (LGDM) as an Assessment Tool in Geological Sciences. In Proceedings of the 11th Annual International Technology, Education and Development Conference, Valencia, Spain, 6–8 March 2017. [Google Scholar] [CrossRef]
  34. McLoughlin, C.; Loch, B. Engaging students in cognitive and metacognitive processes using screencasts. In EdMedia: World Conference on Educational Media and Technology 2012; Amiel, T., Wilson, B., Eds.; Association for the Advancement of Computing in Education (AACE): Denver, CO, USA, 2012; pp. 1107–1110. [Google Scholar]
  35. Calder, N. The layering of mathematical interpretations through digital media. Educ. Stud. Math. 2012, 80, 269–285. [Google Scholar] [CrossRef]
  36. Anuradha, V.; Rengaraj, M. Storytelling: Creating a Positive Attitude toward Narration among Engineering Graduates. IUP J. Engl. Stud. 2017, 12, 32–38. [Google Scholar]
  37. Johnson, C.I.; Mayer, R.E. Applying the self-explanation principle to multimedia learning in a computer-based game-like environment. Comput. Hum. Behav. 2010, 26, 1246–1252. [Google Scholar] [CrossRef]
  38. Hobbs, R. Create to Learn: Introduction to Digital Literacy; Wiley-Blackwell: Hoboken, NJ, USA, 2017; pp. 1–296. [Google Scholar]
  39. Yeh, H.-C. Exploring the perceived benefits of the process of multimodal video making in developing multiliteracies. Lang. Learn. Technol. 2018, 22, 28–37. [Google Scholar]
  40. Nelson, M.E. Mode, meaning, and synaesthesia in multimedia L2 writing. ICFAI J. Engl. Stud. 2006, 2, 69–91. [Google Scholar]
  41. Shin, D.-S.; Cimasko, T. Multimodal composition in a college ESL class: New tools, traditional norms. Comput. Compos. 2008, 25, 376–395. [Google Scholar] [CrossRef]
  42. Nelson, M.E.; Hull, G.A. Self-presentation through multimedia: A Bakhtinian perspective on digital storytelling. In Digital Storytelling, Mediatized Stories: Self-Representations in New Media; Peter Lang: New York, NY, USA, 2008; pp. 123–144. [Google Scholar]
  43. Hoban, G.; Nielsen, W.; Carceller, C. Articulating constructionism: Learning science through designing and making “Slowmations” (student-generated animations). In Conference of the Australasian Society for Computers in Learning in Tertiary Education; University of Queenland: Brisbane, Australia, 2010; pp. 433–443. [Google Scholar]
  44. Türkay, S. The effects of whiteboard animations on retention and subjective experiences when learning advanced physics topics. Comput. Educ. 2016, 98, 102–114. [Google Scholar] [CrossRef]
  45. Hoban, G.; Loughran, J.; Nielsen, W. Slowmation: Preservice elementary teachers representing science knowledge through creating multimodal digital animations. J. Res. Sci. Teach. 2011, 48, 985–1009. [Google Scholar] [CrossRef]
  46. Miller, S.T.; James, C.R. The Effect of Animations Within PowerPoint Presentations on Learning Introductory Astronomy. Astron. Educ. Rev. 2011, 10, 107–119. [Google Scholar] [CrossRef]
  47. Jacobs, B.; Clark, J.C. Create to critique: Animation creation as conceptual consolidation. Teach. Sci. J. Aust. Sci. Teach. Assoc. 2018, 64, 26–36. [Google Scholar]
  48. Reyna, J.; Hanham, J.; Rodgers, K.; Meier, P. Learner-Generated Digital Media (LGDM) Framework. In Proceedings of the INTED2017, Valencia, Spain, 6–8 March 2017. [Google Scholar]
  49. Van Dijk, A.M.; Lazonder, A.W. Scaffolding students’ use of learner-generated content in a technology-enhanced inquiry learning environment. Interact. Learn. Environ. 2016, 24, 194–204. [Google Scholar] [CrossRef]
  50. Hmelo-Silver, C.E. Problem-based learning: What and how do students learn? Educ. Psychol. Rev. 2004, 16, 235–266. [Google Scholar] [CrossRef]
  51. Goodsell, A.S. Collaborative Learning: A Sourcebook for Higher Education; National Center on Postsecondary Teaching, Learning and Assessment: Washington, DC, USA, 1997; Volume II. [Google Scholar]
  52. Doolittle, P.E. Understanding cooperative learning through Vygotsky. In Proceedings of the Lily National Conference on Excellence in College Teaching, Colombia, SC, USA, 2–4 June 1995. [Google Scholar]
  53. Foot, H.; Howe, C. The psychoeducational basis of peer-assisted learning. In Peer-Assisted Learning; Topping, K.J., Ehly, S.W., Eds.; Lawrence Erlbaum Associates: Mahwah, NJ, USA, 1998; pp. 27–43. [Google Scholar]
  54. Fuller, I.C.; France, D. Does digital video enhance student learning in field-based experiments and develop graduate attributes beyond the classroom? J. Geogr. High. Educ. 2016, 40, 193–206. [Google Scholar] [CrossRef]
  55. Frawley, J.K.; Dyson, L.E.; Tyler, J.; Wakefield, J. Building graduate attributes using student generated screencasts. Globally connected, digitally enabled. In Proceedings of the Ascilite 2015, Perth, Australia, 30 November–2 December 2015. [Google Scholar]
  56. Greene, H.; Crespi, C. The value of student-created videos in the college classroom—An exploratory study in marketing and accounting. Int. J. Arts Sci. 2012, 5, 273–283. [Google Scholar]
  57. Pearce, K.L. Undergraduate creators of video, animations and blended media: The students’ perspective. In Proceedings of the Australian Conference on Science and Mathematics Education (formerly UniServe Science Conference), Sydney, Australia, 29 September 2014. [Google Scholar]
  58. Coulson, S.; Frawley, J.K. Student-generated multimedia for supporting learning in an undergraduate physiotherapy course. In Proceedings of the ASCILITE 2017: 34th International Conference on Innovation, Practice and Research in the Use of Educational Technologies in Tertiary Education, Toowoomba, Southern Queensland, Australia, 4–6 December 2017. [Google Scholar]
  59. Alexander, B.; Becker, S.A.; Cummins, M. Digital Literacy: An NMC Horizon Project Strategic Brief; The New Media Consortium: Austin, TX, USA, October 2016; Volume 3.3. [Google Scholar]
  60. Sturges, M.; Reyna, J. Use of Vimeo on-line video sharing services as a reflective tool in higher educational settings: A preliminary report. In Proceedings of the ASCILITE-Australian Society for Computers in Learning in Tertiary Education Annual Conference, Sydney, Australia, 5–8 December 2010. [Google Scholar]
  61. Snelson, C. YouTube Across the Disciplines: A Review of the Literature. MERLOT J. Online Learn. Teach. 2011, 7, 159–169. [Google Scholar]
  62. Kearney, M. Towards a learning design for student-generated digital storytelling. In Proceedings of the Future of Learning Design Conference, University of Wollongong, Sydney, Australia, 10 December 2009. [Google Scholar]
  63. Musburger, R.B.; Kindem, G. Introduction to Media Production: The Path to Digital Media Production; Focal Press: Burlington, MA, USA, 2012; pp. 74–78. [Google Scholar]
  64. Sørensen, B.H.; Levinsen, K.T. Digital Production and Students as Learning Designers. Des. Learn. 2014, 7, 54–74. [Google Scholar] [CrossRef]
  65. Reyna, J.; Hanham, J.; Meier, P. A taxonomy of digital media types for Learner-Generated Digital Media assignments. e-Learn. Digit. Med. 2017, 14, 309–322. [Google Scholar] [CrossRef]
  66. Willey, K.; Gardner, A. Investigating the capacity of self and peer assessment activities to engage students and promote learning. Eur. J. Eng. Educ. 2010, 35, 429–443. [Google Scholar] [CrossRef][Green Version]
  67. Hanrahan, S.J.; Isaacs, G. Assessing self-and peer-assessment: The students’ views. High. Educ. Res. Dev. 2001, 20, 53–70. [Google Scholar] [CrossRef]
  68. Hattie, J.; Timperley, H. The power of feedback. Rev. Educ. Res. 2007, 77, 81–112. [Google Scholar] [CrossRef]
  69. Phillips, R.; McNaught, C.; Kennedy, G. Evaluating e-Learning: Guiding Research and Practice; Routledge: Abingdon-on-Thames, UK, 2012. [Google Scholar]
  70. Tashakkori, A.; Teddlie, C. Sage Handbook of Mixed Methods in Social & Behavioural Research; Sage: Thousand Oaks, CA, USA, 2010; pp. 27–35. [Google Scholar]
  71. Beavers, A.S.; Lounsbury, J.W.; Richards, J.K.; Huck, S.W.; Skolits, G.J.; Esquivel, S.L. Practical considerations for using exploratory factor analysis in educational research. Pract. Assess. Res. Eval. 2013, 18, 1–13. [Google Scholar]
  72. Gorissen, P.; Bruggen, J.V.; Jochems, W. Methodological triangulation of the students’ use of recorded lectures. Int. J. Learn. Technol. 2013, 8, 20–40. [Google Scholar] [CrossRef]
  73. Reyna, J.; Hanham, J.; Meier, P. The Internet explosion, digital media principles and implications to communicate effectively in the digital space. e-Learn. Digit. Media 2018, 15, 36–52. [Google Scholar] [CrossRef]
  74. Powell, L.M. Evaluating the Effectiveness of Self-Created Student Screencasts as a Tool to Increase Student Learning Outcomes in a Hands-On Computer Programming Course. Inf. Syst. Educ. J. 2015, 13, 106–111. [Google Scholar]
  75. Braun, M. Comparative Evaluation of Online and In-Class Student Team Presentations. J. Univ. Teach. Learn. Pract. 2017, 14, 1–21. [Google Scholar]
  76. Hoban, G.; Nielsen, W. Learning Science through Creating a ‘Slowmation’: A case study of preservice primary teachers. Int. J. Sci. Educ. 2013, 35, 119–146. [Google Scholar] [CrossRef]
  77. Georgiou, H.; Nielsen, W.; Doran, Y.; Turney, A.; Jones, P. Analysing student-generated digital media in science. In Proceedings of the Australian Conference on Science and Mathematics Education, Brisbane, Australia, 28–29 September 2016. [Google Scholar]
  78. Anderson, J. Evaluating student-generated film as a learning tool for qualitative methods: geographical “drifts” and the city. J. Geogr. High. Educ. 2013, 37, 136–146. [Google Scholar] [CrossRef][Green Version]
  79. Collins, A.; Halverson, R. Rethinking Education in the Age of Technology: The Digital Revolution and Schooling in America; Teachers College Press: New York, NY, USA, 2018. [Google Scholar]
  80. Pomerantz, J.; Brooks, D.C. ECAR Study of Faculty and Information Technology; ECAR: Louisville, CO, USA, 2017; pp. 1–43. [Google Scholar]
  81. Hatlevik, O.E.; Throndsen, I.; Loi, M.; Gudmundsdottir, G.B. Students’ ICT self-efficacy and computer and information literacy: Determinants and relationships. Comput. Educ. 2018, 118, 107–119. [Google Scholar] [CrossRef]
  82. Hoban, G.; Nielsen, W. Using ‘Slowmation’ to Enable Preservice Primary Teachers to Create Multimodal Representations of Science Concepts. Res. Sci. Educ. 2012, 42, 1101–1119. [Google Scholar] [CrossRef]
  83. Jablonski, D.; Hoban, G.F.; Ransom, H.S.; Ward, K.S. Exploring the use of “slowmation” as a pedagogical alternative in science teaching and learning. Pac.-Asian Educ. J. 2015, 27, 5–20. [Google Scholar]
  84. Banner, O.; Ostherr, K. Design in Motion: Introducing Science/Animation. Discourse J. Theor. Stud. Media Cult. 2015, 37, 175–192. [Google Scholar] [CrossRef]
  85. Stockman, S. How to Shoot Video That Doesn’t Suck: Advice to Make Any Amateur Look Like a Pro; Workman Publishing: New York, NY, USA, 2011; pp. 45–56. [Google Scholar]
  86. Hashimoto, A.; Clayton, M. Visual Design Fundamentals: A Digital Approach; Charles River Media: Hingham, MA, USA, 2009; pp. 29–33. [Google Scholar]
  87. Beetham, H.; Sharpe, R. Rethinking Pedagogy for a Digital Age: Designing for 21st Century Learning; Routledge: Abingdon-on-Thames, UK, 2013; p. 311. [Google Scholar]
  88. Bates, T. Teaching in a Digital Age; University of British Columbia, Tony Bates Associates: Vancouver, BC, Canada, 2016; p. 606. [Google Scholar]
  89. Bennett, S.; Maton, K.; Kervin, L. The ‘digital natives’ debate: A critical review of the evidence. Br. J. Educ. Technol. 2008, 39, 775–786. [Google Scholar] [CrossRef][Green Version]
  90. Yang, I.; Lau, B.T. Undergraduate Students’ Perceptions as Producer of Screencast Videos in Learning Mathematics. In Redesigning Learning for Greater Social Impact; Springer: New York, NY, USA, 2018; pp. 277–286. [Google Scholar]
  91. Yang, X.; Guo, X.; Yu, S. Student-generated content in college teaching: Content quality, behavioural pattern and learning performance. J. Comput. Assist. Learn. 2016, 32, 1–15. [Google Scholar] [CrossRef]
  92. Zimmerman, B.J.; Schunk, D. Motivational sources and outcomes of self-regulated learning and performance. In Handbook of Self-Regulation of Learning and Performance; Routledge: New York, NY, USA, 2011; pp. 49–64. [Google Scholar]
  93. Graybill, J.K. Teaching energy geographies via videography. J. Geogr. High. Educ. 2016, 40, 55–66. [Google Scholar] [CrossRef]
  94. Zimmerman, B.J. Self-Efficacy: An Essential Motive to Learn. Contemp. Educ. Psychol. 2000, 25, 82–91. [Google Scholar] [CrossRef] [PubMed][Green Version]
  95. Azevedo, R.; Cromley, J.G. Does training on self-regulated learning facilitate students’ learning with hypermedia? J. Educ. Psychol. 2004, 96, 523. [Google Scholar] [CrossRef]
  96. Dabbagh, N.; Kitsantas, A. Supporting self-regulation in student-centered web-based learning environments. Int. J. e-Learn. 2004, 3, 40–47. [Google Scholar]
  97. Pannabecker, V.; Barroso, C.S.; Lehmann, J. The Flipped Classroom: Student-Driven Library Research Sessions for Nutrition Education. Int. Ref. Serv. Q. 2014, 19, 139–162. [Google Scholar] [CrossRef][Green Version]
  98. Hofer, M.; Owings-Swan, K. Digital moviemaking—The harmonization of technology, pedagogy and content. Int. J. Technol. Teach. Learn. 2005, 1, 102–110. [Google Scholar]
Figure 1. The Learner-Generated Digital Media (LGDM) framework. DMP stands for Digital Media Project.
Figure 1. The Learner-Generated Digital Media (LGDM) framework. DMP stands for Digital Media Project.
Education 08 00106 g001
Figure 2. Student attitude toward technology in the LGDM assignment for five Science subjects (N = 274).
Figure 2. Student attitude toward technology in the LGDM assignment for five Science subjects (N = 274).
Education 08 00106 g002
Figure 3. Student perceptions of digital media support in the LGDM assignment for five Science subjects (N = 240).
Figure 3. Student perceptions of digital media support in the LGDM assignment for five Science subjects (N = 240).
Education 08 00106 g003
Figure 4. Student understanding of the LGDM assignment instructions, timeframe, and importance for five Science subjects (N = 274).
Figure 4. Student understanding of the LGDM assignment instructions, timeframe, and importance for five Science subjects (N = 274).
Education 08 00106 g004
Figure 5. Student perceptions of knowledge construction in the LGDM assignment for five Science subjects (N = 274).
Figure 5. Student perceptions of knowledge construction in the LGDM assignment for five Science subjects (N = 274).
Education 08 00106 g005
Table 1. Subjects which implemented LGDM as an assessment tool in the Faculty of Science in 2016. B = Blended, O = Online.
Table 1. Subjects which implemented LGDM as an assessment tool in the Faculty of Science in 2016. B = Blended, O = Online.
SubjectYrNSample Size (%)Digital Media TypeAssessment WeightMode of Instruction
Pharmacology 2216998 (58%)Digital story, video, blended media30%B
Geological Processes210173 (72%)Digital story, video, blended media10%B
Animal Behaviour and Physiology210634 (31%)Digital story, video, blended media10%B
Evaluating TCM: Theory, Practice and Research34335 (81%)Digital story, video, blended media20%B
Introductory Pharmacology and Microbiology33934 (87%)Brochure, poster design25%O
TOTAL 458274 (60%)
Table 2. Students’ online questionnaire on attitude toward digital media for learning.
Table 2. Students’ online questionnaire on attitude toward digital media for learning.
CategoryItem
Demographics1. Gender
2. Age
3. Education
4. English as an additional language (EAL)
Digital media support5. I found the digital presentation lecture engaging.
6. I applied concepts from the lecture to my assignment.
7. I need a better understanding of digital presentation principles.
8. I will recommend that my peers attend this lecture.
9. I used a storyboard to structure my project.
10. Overall, the technical support to complete my project was good.
Attitude towards technology11. I Enjoy using technology for personal/recreational matters.
12. I am confident using technology for personal/recreational matters.
13. I have a positive attitude towards technology for recreational matters.
14. I enjoy using technology for learning.
15. I am confident using technology for learning.
16. I have a positive attitude towards technology for learning.
Understanding of the assignment17. I believe instructions on the assignment were clearly provided.
18. The timeframe to complete the project was good.
19. I understand the importance of communicating concepts/ideas in the digital world.
20. Overall, I was happy about the digital media presentation assignment.
Knowledge construction21. I believe using digital presentations helped me to understand the topic.
22. The digital presentation assignment helped me to develop critical thinking skills.
23. The digital presentation assignment helped me to develop communication skills.
24. The digital presentation helped me to work as a part of a team.
25. The digital presentation helped me to exercise my creativity.
26. I believe digital presentations are a good way to assess students’ understanding of a topic.
27. I will encourage academics to use similar assignments in other subjects.
28. I believe I learnt additional skills by doing this assignment.
Open-ended Questions29. Did you experience any issue with the assignment?
30. What did you like most about the assignment?
31. What did you like least about the assignment?
32. Do you have any feedback on how to improve this assignment?
33. Is there anything that you would like to say that has not been covered in the previous questions? If so, please feel free to provide additional feedback in the space below:
Table 3. Demographic characteristics of participants in the LGDM assignment for five Science subjects (N = 274).
Table 3. Demographic characteristics of participants in the LGDM assignment for five Science subjects (N = 274).
CharacteristicN%
Gender
 Male10237.2
 Female17262.8
Age bracket
 18–2923987.2
 30–493111.3
 50–6441.5
Level of education
 High school graduate17965.3
 Some college155.5
 College graduate51.8
 University degree6624.1
 Trade/technical/vocational training93.3
English as an Additional Language (EAL)
 Yes5520.1
 No21979.1
Table 4. Standardised solutions by Factor Analysis to measure questionnaire construction.
Table 4. Standardised solutions by Factor Analysis to measure questionnaire construction.
Factor
ItemDigital Media SupportAttitude Toward TechnologyUnderstanding the AssignmentKnowledge Construction
50.724
60.714
90.687
80.607
70.561
13 0.874
12 0.855
15 0.851
14 0.823
16 0.823
11 0.809
18 0.742
17 0.682
19 0.561
26 0.833
27 0.817
28 0.784
22 0.762
21 0.749
23 0.744
25 0.681
Table 5. Student attitude toward technology in the LGDM assignment, for five Science subjects (N = 274).
Table 5. Student attitude toward technology in the LGDM assignment, for five Science subjects (N = 274).
QuestionFrequencies
SDDASA
I enjoy using technology for personal/recreational matters.2 (0.7%)24 (8.8%)107 (39.1%)141 (51.5%)
I am confident using technology for personal/recreational matters.2 (0.7%)40 (14.6%)116 (42.3%)116 (42.3%)
I have a positive attitude towards technology for recreational matters.1 (0.4%)19 (6.9%)124 (45.3%)130 (47.4%)
I enjoy using technology for learning.2 (0.7%)16 (5.8%)136 (49.6%)120 (43.8%)
I am confident using technology for learning.-41 (15%)122 (44.5%)111 (40.5%)
I have a positive attitude towards technology for learning.-20 (7.3%)129 (47.1%)125 (45.6%)
SD = Strongly Disagree; D = Disagree; A = Agree; SA = Strongly Agree.
Table 6. Student perceptions of digital media support in the LGDM assignment for five Science subjects (N = 240).
Table 6. Student perceptions of digital media support in the LGDM assignment for five Science subjects (N = 240).
QuestionFrequencies
SDDASA
I found the digital presentation lecture engaging.2 (0.7%)33 (13.8%)147 (61.3%)58 (24.2%)
I applied concepts from the lecture to my assignment.3 (1.3%)25 (10.4%)140 (58.3%)72 (30.0%)
I used a storyboard to structure my project.10 (4.1%)55 (22.9%)111 (46.3%)64 (26.7%)
I will recommend that my peers attend this lecture.3 (1.3%)44 (18.3%)124 (51.7%)69 (28.7%)
I need a better understanding of digital presentation principles.10 (4.1%)55 (22.9%)111 (46.3%)64 (26.7%)
SD = Strongly Disagree; D = Disagree; A = Agree; SA = Strongly Agree.
Table 7. Student understanding of the LGDM assignment instructions, timeframe, and importance for five Science subjects (N = 274).
Table 7. Student understanding of the LGDM assignment instructions, timeframe, and importance for five Science subjects (N = 274).
QuestionFrequencies
SDDASA
I believe instructions on the assignment were clearly provided.5 (1.8%)38 (13.9%)39 (50.7%)92 (33.6%)
The timeframe to complete the project was good.11 (4%)15 (5.5%)121 (44.2%)127 (46.4%)
I understand the importance of communicating concepts/ideas in the digital world.2 (0.7%)7 (2.6%)121 (44.2%)144 (52.6%)
Table 8. Student perceptions of knowledge construction in the LGDM assignment for five Science subjects (N = 274).
Table 8. Student perceptions of knowledge construction in the LGDM assignment for five Science subjects (N = 274).
QuestionFrequencies
SDDASA
I believe using digital presentations helped me to understand the topic.8 (2.9%)41 (15%)134 (48.9%)91 (33.2%)
The digital presentation assignment helped me to develop critical thinking skills.8 (2.9%)60 (21.9%)134 (48.9%)72 (26.3%)
The digital presentation assignment helped me to develop communication skills.10 (3.6%)28 (10.2%)137 (50%)99 (36.1%)
The digital presentation helped me to exercise my creativity.4 (1.5%)23 (8.4%)127 (46.4%)120 (43.8%)
I believe digital presentations are a good way to assess students’ understanding of a topic.20 (7.3%)49 (17.9%)117 (42.7%)88 (32.1%)
I will encourage academics to use similar assignments in other subjects.17 (6.2%)48 (17.5%)124 (45.3%)85 (31%)
I believe I learnt additional skills by doing this assignment.14 (5.1%)34 (12.4%)125 (45.6%)101 (36.9%)
Table 9. Categories derived from thematic analysis of open-ended Q29.
Table 9. Categories derived from thematic analysis of open-ended Q29.
ThemeFrequency%
No issues11068
Inadequate skills in digital media creation2012
Not understanding the assignment117
Short time to complete the assignment106
Other127
Table 10. Categories derived from thematic analysis of open-ended Q30.
Table 10. Categories derived from thematic analysis of open-ended Q30.
ThemeFrequency%
Creativity4728
Teamwork3823
Learning digital media
Learning subject content
35
16
21
10
Different to other assignments1610
Self-expression106
Other72
Table 11. Categories derived from thematic analysis of responses to open-ended Q31.
Table 11. Categories derived from thematic analysis of responses to open-ended Q31.
ThemeFrequency%
Nothing
Group issues
46
32
32
22
Inadequate digital media skills2014
Understanding the assignment1410
Time-consuming139
Not enough time to produce the assignment54
Other139
Table 12. Categories derived from thematic analysis of responses to open-ended Q32.
Table 12. Categories derived from thematic analysis of responses to open-ended Q32.
ThemeFrequency%
No feedback4837
Additional software training3124
More assignment instructions129
Small group size65
Start task earlier in the semester75
Equipment available to students54
Ability to choose group members54
More topics to choose from54
Other108
Table 13. Categories derived from thematic analysis of responses to open-ended Q33.
Table 13. Categories derived from thematic analysis of responses to open-ended Q33.
ThemeFrequency%
Positive comments about assignment3055
No comment1832
Other713
Table 14. Descriptive statistics for the RPF—SPARKPlus group contribution.
Table 14. Descriptive statistics for the RPF—SPARKPlus group contribution.
SubjectNMinMaxMeanS. DVariance
Geological Processes960.191.180.990.140.019
Introductory Pharmacology & Microbiology400.601.120.990.110.012
Pharmacology 21670.291.150.990.090.007
Total303
Table 15. Group contribution ranking from three subjects which used the SPARKPlus application to moderate marks.
Table 15. Group contribution ranking from three subjects which used the SPARKPlus application to moderate marks.
Contribution
Level (%)
Geological ProcessesIntroductory Pharmacology & MicrobiologyPharmacology 2
Optimum (RPF > 1.0)60.062.549.1
Acceptable (RPF = 0.8–1.0)33.730.047.9
Poor (RPF < 0.8)6.37.53.0
Table 16. Mark distribution across the five subjects which implemented LGDM. As the assignment weighting varies across the five subjects, marks were converted to a percentage.
Table 16. Mark distribution across the five subjects which implemented LGDM. As the assignment weighting varies across the five subjects, marks were converted to a percentage.
SubjectNMinMaxMeanS. D
Pharmacology 21693396799.25
Geological Processes10167100957.51
Animal Behaviour and Physiology106531007714.45
Evaluating TCM: Theory, Practice & Research437095847.83
Introductory Pharmacology and Microbiology3961978212.48
Total458

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Back to TopTop