Strategies for Enhancing Assessment Information Integrity in Mobile Learning

Abstract

Mobile learning is a global trend, which has become more widespread in the post-COVID-19 pandemic era. However, with the adoption of mobile learning comes new assessment approaches to evaluate the understanding of the acquired information and knowledge. Nevertheless, there is scant knowledge of how to enhance assessment information integrity in mobile learning assessments. Due to the importance of assessments in evaluating knowledge, integrity is the sine qua non of online assessments. This research focuses on the strategies universities could use to improve assessment information integrity. This research adopts a qualitative design, employing interviews with academics as well as teaching and learning support staff for data collection. The findings reveal five strategies that academics and support staff recommend to enhance assessment information integrity in mobile learning. The theoretical and practical implications are discussed, as well as future research directions.

1. Introduction

The worldwide closures of universities in response to the outbreak of the COVID-19 pandemic generated significant challenges for university administrators and academics. Arguably, the inaccessibility of learning information was one of the more serious challenges, as it deprived students of opportunities for learning and development. As a result, the adoption of educational technologies is receiving renewed attention from academia, government and the business sectors [1], thus leading to a proliferation of research output addressing various facets of e-learning. Mobile ICTs in particular have become indispensable to bridging and binding students and academics together in the midst of the chasms which are a consequence of the social distancing regulations. For example, Yuan et al. [2] highlight that Chinese universities introduced mandatory mobile learning courses so as to enable students to remotely access learning information. The shift in how academic information is acquired, accessed, exchanged and stored has had a marked influence on integrity. Further compounding information integrity are new software and mobile technologies making cheating easier, thereby exacerbating an old problem with new techniques [3].

Particularly, one of the key challenges, one that remains pertinent today, is the need to ensure the academic integrity of online assessments. Through a comprehensive literature review of studies related to online assessments, Butler-Henderson and Crawford [4] found that cheating in examinations was the most prevalent focus of the majority of the studies. Concerns around assessment information integrity are not new, with evidence of research around the topic dating back to about three decades [3]. Nevertheless, the global mass migration to online assessment has provided a new impetus to promote and maintain assessment information integrity. The academic integrity of online assessments is still critical to the future of e-learning in all its forms. Currently, online assessments continue to be necessary due to the prevalence of the highly infectious COVID-19, which, although subsided somewhat, remains a concern. Yet, cheating and other forms of academic dishonesty have been reported to be more widespread during the COVID-19 era than before [5], with some calling it “at an alarming level” [6]. In studies undertaken in some parts of the world such as Germany [7], students confirmed that they cheated more frequently during online assessments than during traditional forms of face-to-face assessments, with some studies [8] establishing that up to 95% of students cheated. Cheating manifests itself in different forms, such as impersonation, forbidden aids, peeking, peer collaboration, outside assistance and student–staff collusion [9]. As a result, there has been a crescendo of voices calling on stakeholders to “rescue academic integrity and combat academic dishonesty” [5]. Furthermore, some scholars [10] argue that, beyond the COVID-19 pandemic, there will be a continued push towards the use of online assessments in order to guard against future COVID-19-types of disruptions to assessments.

The main objective of this paper is to determine the strategies adopted by universities to promote assessment information integrity in mobile learning. Whilst there is evidence of research on academic integrity in online learning environments, to the best of our knowledge, there is little evidence of studies that focused on assessment information integrity in mobile learning. The need for such studies cannot be overstated, particularly in a mobile-centric continent such as Africa [11], where mobile devices are the most commonly used digital device by students to access learning information [12]. Understanding mobile learning assessment information integrity strategies can cultivate the confidence of various stakeholders in the quality of mobile learning qualifications. Since evidence suggests that students’ performance improved during the COVID-19 online-mediated assessments [10], until concrete reasons for the improved performance are advanced, suspicions of potential academic dishonesty are likely to persist. Such suspicions are vindicated by studies such as that of Harmon and Lambrinos [13] and Arnold [14], who found that students scored significantly less in supervised assessments as opposed to the high marks attained when there was no supervision. The study is situated in Namibia, a middle-income country with a mobile penetration rate of over 100 percent.

2. Literature Review

2.1. Academic Integrity

Assessment is defined as “the systematic collection of information about student learning, using the time, knowledge, expertise and resources available, in order to inform decisions that affect student learning” ([15], p. 2). Through assessments, educators can establish the skill or knowledge level of students [16]. As noted by Comas-Forgas and colleagues [6], the assessment results obtained by students play a critical part in shaping their future career as well as economic and social prospects. Consequently, the pressure to score high marks may make cheating an attractive option for many students. In traditional assessments, cheating is discouraged through the physical presence of invigilators, something which is a near impossibility during online assessments. Due to the distributed geography of students enrolled for online learning, questions around the authenticity and integrity of online assessments remain a topical issue among scholars and practitioners. As a result, some employers often dismiss online education as sub-standard, and often discriminate against holders of online degrees, regardless of how competent they may appear to be [17]. There is a stigma attached to online degrees, one which could disadvantage holders of such qualifications when applying for employment opportunities.

On the other hand, institutions that ensure that staff and students act in an integrous manner reinforce their reputation [18], which in turn cultivates confidence in the quality of their online degrees. In a study conducted among employers in Tanzania, Kisanga [17] found that the reputation of the institution awarding a qualification was an important consideration that influenced the hiring decisions of otherwise sceptical employers. Against the foregoing, it is apparent that the enforcement of assessment information integrity is beneficial not only to universities, but to students and graduates as well.

2.2. Proctoring in Online Assessments

A popular solution to the cheating phenomena is the use of proctoring. The frenzied efforts to adopt proctoring tools has been labelled a technological arms race as universities attempt to address assessment integrity concerns associated with online learning [19]. Proctoring may involve the use of live webcams to observe student behaviour during online assessments. Bilen and Matros [20] argue that in the absence of proctoring, it is likely that cheating among students will increase. Consequently, the authors propose that universities devise and implement uniform policies that inform the use of cameras to monitor student behaviour during online assessments.

Whilst proctoring has been proven to reduce cheating and inflated assessment performance [7], it is costly, compromises privacy [19,21], communicates educators’ lack of trust in their students [8] and it may compromise students’ mental health [6]. In a study undertaken in the United Arab Emirates on students’ attitudes and concerns regarding the use of e-proctoring tools during their final exams, Kharbat and Abu Daabes [22] found that more than 86% of students deemed online proctoring as impractical and invasive of their privacy. Furthermore, the majority of the students (91.6%) felt that being watched via webcams made them feel anxious and negatively affected their exam performance. In addition to privacy concerns, some proctoring software has been found to discriminate against students based on their skin colour [18]. Furthermore, proctoring may be hindered by a lack of access to proctoring software [7], particularly in developing countries.

2.3. Mobile Devices and Learning

In traditional face-to-face assessments, mobile devices have been identified as one of the chief enablers of cheating and academic dishonesty during assessments [6]. In a study undertaken among accounting students in Saudi Arabia, El-Sayed Ebaid [23] found that the most common type of cheating behaviour during examinations was the use of cell phones to text exam questions and answers between students, as well as consulting personal class notes. The role of cell phones in cheating was further elucidated by Bernardi and Higgins [24] in the United States of America. To mitigate against the use of mobile devices to cheat, interventions such as the use of mobile phone detectors and jammers have been developed [25].

Although mobile devices have gained notoriety due to their ability to act as an enabler of cheating, they remain an invaluable tool for enhancing the accessibility of learning information. The role of mobile devices in knowledge acquisition, or mobile learning, was further highlighted due to the social distancing regulations necessitated by the COVID-19 pandemic. Several studies have elucidated the significant role played by mobile learning in various contexts such as Greece [26], Namibia [12] and Bangladesh [27]. The popularity of mobile devices such as smartphones and tablets is increasing at an exponential pace, particularly in developing countries [26]. Due to the limited number of computers in households in developing countries, the prevalence of mobile devices plays a critical role in enhancing access to the Internet. For example, in South Africa, only 22% of households have access to a computer, and of these, only 10–12% have access to the Internet [28,29]. In such a case, mobile devices such as smartphones become an indispensable tool for accessing online information.

Whilst the interest in mobile learning has grown, there is scant literature that focuses on mobile learning assessment, or m-assessment [26]. This is despite the fact that the lack of a specific monitoring system has been identified as a major weakness of mobile learning. However, with the COVID-19 pandemic transforming mobile learning from an option to a necessity, there is a need to respond to the vacuity in the literature and devise measures that ensure that assessment integrity in mobile learning is not compromised.

3. Materials and Methods

3.1. Methodology

This paper was underpinned by a qualitative design. Qualitative research questions the taken-for-granted variables [30], allowing researchers to derive explanations of processes and human behavioural patterns that may be otherwise difficult to capture quantitatively. Qualitative approaches are most appropriate where little is known about a phenomenon, which we believe to be the case with assessment integrity in mobile learning. Additionally, a qualitative study of this nature can unearth critical information that is useful for guiding policy-making [31]. In mobile learning research, some studies [11,32] have found that quantitative studies are more prevalent than qualitative studies. Therefore, in adopting a qualitative approach, we chose to take the road less travelled. Qualitative studies are good at eliciting thick descriptions and insider perspectives when compared to quantitative studies, which are predominantly used in the Global North. A qualitative approach allowed us to gain a richer understanding of interviewees’ lived experiences and perceptions regarding mobile learning assessment integrity. This was particularly important, given the scant mobile learning research emanating from Namibia. Interviews were conducted with academics and e-learning system administrators from 2 public universities in Namibia, as well an Australian mobile learning expert who previously worked in the Southern African region. The goal of the interviews was to elicit respondents’ perceptions (and lived experiences) of how assessment integrity could be enhanced in mobile learning. In order to achieve our goal, the following research question was asked: what strategies could be employed to mitigate against cheating and enhance assessment integrity in mobile learning?

3.2. Participants

In selecting participants for this study, we sought participants who were deemed to have the best knowledge of the subject and thus able to provide a rich and detailed account of the phenomenon [33]. The interviews were conducted with a total of 12 participants as depicted in

Table 1. Demographic profile of interviewees.

Interviewee	Position	Department	Courses Taught
AW	Administrator	Centre of Open and Lifelong Learning	Web Communications, Communication Skills
BC	Administrator	Centre for Open and Lifelong Learning	Microteaching, Classroom and Workshop Management
LK	Administrator/Academic	Technical and vocational education and training	ICT for Educational Management, Psychology of Learning, Microteaching
ES	Academic/Management	Marketing and Logistics	Customer Care, Product Innovation, Project Management
ET	Academic	Accounting	Labour Economics, Econometrics, Advanced Microeconomics Theory
AB	Academic	Information Technology	Python, C#
RM	Academic	Land and Property Sciences	Information Systems Law, Land Policy and Development, Project Management for Land Administration
TD	Academic	Architecture, Planning and Construction	Construction Economics, Building and Engineering Law, Project Management
JL	Academic	Department of Biology, Chemistry and Physics	Analytical Principles and Practice, Molecular Spectroscopy and Chemometrics
EG	Academic	Department of Communication	Professional Communication, English in Practice
CK	Administrator/Academic	Department of Higher Education and Lifelong Learning	Educational Research, Instructional Design, Contemporary Social Issues
ND	Academic	Department of Mathematics, Statistics and Actuarial Science	Advanced Calculus, Engineering Mathematics, Business Statistics

, which were made up of 8 academics and 4 teaching and learning technical support staff from 2 public universities in Namibia. The 4 support (administrative) staff members also had lecturing experience (only 1 was not teaching at the time of the study). The purposive sampling technique was adopted in the selection of participants. The number of participants in this study seems sufficient, given that scholars have indicated that even as low as 4 participants may be enough in a qualitative study due to the large quantities of data produced.

We wish to highlight that in some cases, the list of courses taught by interviewees is not exhaustive. Many of the interviewees indicated that their courses often changed, depending on the semester and level of class(es) that they are allocated in that particular year. However, the department where the academic works rarely changes; therefore, the department variable provides a better idea of the field the academic belongs to.

3.3. Data Analysis

The interview recordings were transcribed by one of the researchers and shared with a professional transcriber for quality checks to ensure the accuracy of the transcripts. Then, the interview data were coded and analysed using inductive thematic analysis. Thematic analysis is useful where researchers seek to understand participants’ feelings, experiences and behaviours from a given dataset, as opposed to a single set or data item [34].

3.4. Dependability and Trustworthiness

Whilst validity and reliability are common terms in research, they are more suited to quantitative research designs rather than qualitative research. In qualitative research, it is more common to use the terms dependability and trustworthiness. To ensure trustworthiness and rigor, we adopted the member checking technique to verify the results. As noted by Brady [35], member checking is commonly used in qualitative research to ensure the integrity and accuracy of results.

4. Results

Our analysis revealed five key themes that could be used to inform approaches aimed at enhancing assessment information integrity in mobile learning. The themes and supporting verbatim quotations are presented next.

4.1. Theme One: Assessment Approach

A number of the respondents indicated that educators should strive to set assessment questions that require learners to draw on higher-order thinking skills, rather than merely recall questions. This approach was also suggested by a number of the interviewees. For example, interviewee BC argued that “you can give them open book tests… open book does not necessarily mean that the questions come from the book, you can have case studies for example”. Interviewee ND added that academics “…could maybe bring the concept of applications more”. Through application questions, students are given questions that pose “a world problem”. In this way, students are forced to apply their understanding, and searching for answers online is less likely to be useful to them. Rather, students are forced to draw from their “critical and analytical thinking in the process” (interviewee TD).

However, for this to happen, it is important that educators are capacitated with the requisite skills to formulate the right questions. Interviewee BC added:

“I definitely feel that there is a need for the teachers themselves to be capacitated to learn more about mobile learning because they think [it] is still the traditional way of teaching.”

The educators’ lack of capacity to formulate assessments suitable for mobile learning had some negative repercussions. As highlighted by interviewee BC:

“there were quite a number of challenges, our students actually performed better but the reason why they performed better during the past two years is because they cheated and…that I can say comes from the fact that our lecturers or tutors…do not know how to set the assessments or the types of assessments.

Interviewee LK, who has acted in various capacities as an educator, mobile learning researcher and other middle management roles, corroborated the above, arguing that

…it all depends on how the questions are set up, you can set up [questions] and then that brings another dimension; how are you setting up your questions because if you are setting up a question where the student can easily find the answer or copy and paste the answer then yah, student will automatically cheat but…, if the questions are designed in such a way that they have to apply their mind and really think through and they are not going to find the answer in the book or even by googling, that would make it a bit more of a challenge.

Furthermore, assessments integrity could also be enhanced through what interviewee AW, a distance learning division senior administrator, called “near transfer”. The interviewee argued:

I believe in what we call near transfer and far transfer…if I receive content right now and I get an activity where I need to interact, or an instruction where I need to interact with a certain topic whether an activity [or what] and it is done in class, as [in] right now, it is like the formative assessment, that is my near transfer, because what you taught me I can immediately apply or I can immediately give feedback on it.

Such an approach calls for continuous formative assessments to be administered. This viewpoint is shared by interviewee BC, who opined that “I would even suggest that you do not do summative assessments you do formative assessments, I mean continuously”. AW added that some assessments, the “far transfer” format, should be “alive”, that is, include aspects such as “a problem based assignment, project based assignment, I would want to see that assignment to be a living document”.

Respondents stated that if questions were framed properly, then open book assessment could be a useful approach that eliminated the need to monitor students.

4.2. Theme Two: Monitoring

Some respondents suggested that educators could require that students switch on their cameras for them to be monitored when writing certain assessments such as examinations. When cameras are on, students’ movements and actions during the course assessment are monitored, and thus they may be dissuaded from engaging in any behaviour that could compromise assessment integrity. Interviewee JL argued that “as long as the student knows that they are being watched somehow, they cannot go beyond what is prescribed…”. Monitoring could also be through inviting students to write assessments under controlled environments such as regional centres, where invigilators are utilised. Interviewee ES stated that “although we have the mobile learning option, we can have regional centers or regional places where they go” to write assessments. The interviewee argued that technical fields such as engineering and health sciences “…also need the physical presence somehow, so it’s a matter of blending the two…”. The interviewee acknowledged that whilst mobile learning apps could be used for simulation, “that component of physical presence is also important”. Interviewee JL shared similar sentiments, but also adding that “this can only apply in a normal situation where you do not have an issue like [the] pandemic, where people could not be [put] together”.

4.3. Theme Three: Technical Tools

Technical features could also be built into the mobile system to minimise opportunities for cheating. Interviewee LK stated that mobile learning can “have a few security features which you can put in and…once you start the quiz it does not allow you to copy and paste and it does not allow you to get out of the browser window…”. Proctoring was proposed as a tool that could be used to enhance mobile learning assessment integrity. According to ES, proctoring tools “can monitor the movement…as to where the student was actually browsing at the time of the assessment. So that is really recorded. It’s analysed and provided with the submission. So when the student is submitting [the assessment] also the records of such [navigation patterns during assessment] are there. I can actually verify and the student is aware of that”. Not only can students’ browsing activities be monitored, but they can be restricted, whereby students are not able to open other online pages during the test. This functionality is sometimes referred to as safe exam browser. Interviewee EG stated that “safe exam browser is to make sure that they do not leave the screen where they currently are”. Nevertheless, the interviewee indicated that in some instances the tool was not functional when students accessed their tests using mobile devices.

Interviewee CK shared how the university e-learning is configured to make it possible to identify the location or device used to take a particular test. Scrutinising IP addresses is often a reactive measure that is taken if there are suspicions of cheating activities. Narrating a previous incident where the university used an IP address to scrutinise network activity, interviewee CK stated that the system detected that many “students were taking the test at the same IP address”. She added that “we learned that it may have been one person doing it for them”.

4.4. Theme Four: Awareness Creation

Assessment integrity can also be enhanced through awareness campaigns that are meant to change students’ attitudes towards cheating during assessments. To cultivate awareness, “…academic integrity workshops were conducted” and some academics “often reminded students of the consequences of cheating in examinations” (AB). This approach was supported by RM, who argued that students need to be taught to maintain “a certain level of ethics knowing that they should maintain and uphold”. Nevertheless, she expressed doubts over the effectiveness of this method, adding that “I do not know how that can be administered, how do you ensure that everyone is ethical since the interest of most students is that we want to pass”. The interviewee went on to suggest that students may be made aware of the implications of cheating behaviours on their future career prospects. She argued that there is a need for “learning for students in terms of the impact of, what the impact of cheating would have on their future careers, like the moment you are not able to master a certain concept or subject and then you get into the workplace it means it is going to be challenging”.

4.5. Theme Five: Verification of Identity

Sometimes it is possible that the various measures that are put in place to promote integrity do not work as they are intended to work. Some academics indicated they had their suspicions raised when there was a significant discrepancy between assignment marks (not under controlled conditions) and test marks (assessed under controlled conditions). One interviewee stated:

I even had a lot of zeros in my assessments but in the assignments I will see people who have 74, 75 in the assignments, having zero in the test, so it was a puzzle that what happens, what is happening in the background? So I experienced that a lot, when you give an assignment they perform but when you give an assessment of up to 2 h, suddenly some people who were even passing, who passed their assignment are having zero in a test or in the assessment.

(ET)

The interviewee indicated that they resorted to taking steps to authenticate the identity of the student who wrote the assessment. The interviewee argued:

I could also verify because when I have received term papers from students and I just call them that they should come and present their work to see if, I really want to authenticate whether they are the ones who did the work.

(ET)

The interviewee (ET) was confident of their authentication approach, and proceeded to narrate how his approach had worked in the recent past. When suspicious of compromised integrity, the academic would call the student for an oral assessment, whereby he asks questions from the student’s assignment script to test their knowledge. He shared the following experience:

I called the student in my office [to] ask one or two questions, he could not even answer one question, he did not even know anything then at the end…the conclusion was that it was not written by him…

5. Discussion

This study sought to identify the strategies adopted by academics to ensure assessment information integrity in mobile learning in the Namibian higher education landscape. The results of this study have important implications for scholars and practitioners alike, as well the broad future of mobile learning.

5.1. Theoretical Implications

Namibian academics and support staff revealed that academic and integrity in mobile learning could be enhanced by firstly accepting that, if given a chance, students will consult illegal information such as notes when writing an assessment without close supervision. Therefore, they argued that academics need to set assessments in cognisance of this fact. The approach adopted by some Namibian academics, whereby they set their assessment with an open-book mindset, is supported by authors such as Butler-Henderson and Crawford ([4], p. 7), who argue that “online examinations should be oriented towards the same theory of open book examinations”. This approach places the student in the role of an expert witness, and seeks to validate their knowledge by setting questions that do not require rote answers. Furthermore, the approach usually involves setting essay questions, and more importantly, citing real examples [21]. Reedy and others [36] also suggest setting higher-order questions that require students to use information instead of merely regurgitating it. With mobile learning characterised by the high mobility of the learners, this approach could prove useful and practical to incorporate into mobile learning implementation plans.

Our findings also indicate that academics believe that the monitoring of students through remote proctoring using webcams could enhance assessment integrity in mobile learning. This is an approach that has been used in various contexts, such as Australia [36], and the United States of America [37]. This approach could be useful where there is contract cheating, that is, where students pay for someone to take the assessment on their behalf and then present it as their own effort [21]. However, it is worth noting that some students have complained of the intrusive nature of monitoring, feeling that their privacy was being violated [38], which then raises ethical issues. On the other hand, arguments have been made that in-person invigilation is also invasive of students’ privacy [39]. One of the software tools that have been used in e-learning is Respondus [38,40], although its collection of sensitive student personal information, whilst providing “nebulous terms of service…” ([40], p. 719) is strongly criticised. Concerns regarding monitoring through proctoring software have led some to suggest that other approaches of enhancing integrity be considered instead [19].

Technical tools may also be appropriated and used to improve integrity in mobile learning assessments. Some of the academics argued that a screen-lock or browser lock-down software can be used to prevent students from moving from the test website or app to another app or website, as well as preventing copying, screen-sharing or printing any information using their device during the time of the assessment.

Our findings reveal that academics and support staff believe that appealing to the students’ sense of morality could help to reduce instances of academic dishonesty. According to Hsu [41], the most important determinant of whether a student cheats or not is their moral anchor, which could be enhanced through explicit discussions on the importance of academic integrity. Indeed, such an approach has been found to be effective in some instances. In a study conducted by Reedy [36] among Australian students, many students indicated that although cheating was relatively easy, their own moral compass and beliefs made it impossible for them to cheat. Consequently, this necessitates further research on strategies that could be used to inculcate a sense of morality among mobile learning users.

However, cheating may still occur regardless of all measures enacted. In that case, our findings reveal that some Namibian academics choose to interview students to verify their ownership of the work in question. Such an approach, whilst effective, may be difficult in cases where mass cheating is suspected as it is practically impossible to individually interview each and every student. Furthermore, this approach relies on an educator’s subjective evaluation of potential cheating. If the instructor fails to identify any cues that suggest cheating took place, they are unlikely to initiate the interview process.

5.2. Practical Implications

This study has a number of practical implications. Firstly, the issue of academic dishonesty is a problem that persists, and the ubiquitous nature of mobile learning probably worsens it. The findings once again highlight that some students continue to cheat. The perpetuation of this malpractice is likely to reinforce the attitudes of naysayers who do not have confidence in knowledge acquisition through mobile learning and other forms of ICT-mediated learning. Nevertheless, academics use various methods to enhance information integrity in mobile learning assessments. This suggests that efforts to enhance assessment information integrity should take a multi-pronged approach, using various tools at the disposal of academics. Some tools are preventative (before the fact), whilst post hoc approaches are diagnostic and aimed at seeking to identify whether any dishonesty took place. Some of the approaches (e.g., design of open-book type of assessments) necessitate for academics to be trained on how to develop questions can test a student’s own knowledge with reasonable accuracy. Perhaps mobile learning can borrow the concept of reasonable assurance, rather than seeking absolute academic information integrity.

5.3. Limitations and Future Work

This study, however, is not without its limitations. Firstly, this study was conducted in Namibia. Added to the qualitative nature of the study, this means that the results cannot be easily generalized to other contexts. Therefore, future studies may consider conducting similar studies in other cultural and country contexts as we suspect that such variations may yield other interesting insights. Furthermore, other researchers may utilise a qualitative approach for more generalizable results, as well as being afforded the opportunity to use much larger sample sizes.

6. Conclusions

Cheating or academic dishonesty remains a concern for all stakeholders. As technology evolves, it is likely that newer ways to cheat will emerge. Enforcing assessment information integrity is not a unidimensional exercise, but rather one that could be approached from different angles. Mobile learning practitioners have various options that they could consider to improve confidence in the integrity of the information given by remote students during online assessments. The pursuit of absolute certainty in assessment integrity may be an elusive dream in mobile learning, but then, there is no evidence that there is absolute assessment integrity in offline assessment either.