User Experience and Usability Evaluation of an Educational Mobile Application Developed for Fostering Ethics Literacy

Abstract

The world is constantly challenged by complex crises—from the COVID-19 pandemic and geopolitical tensions to economic uncertainty and severe environmental disasters. During these critical times, individuals need to reflect on ethical values and demonstrate responsible decision-making, integrity, and preparedness to mitigate the impact of future crises. Education can play an instrumental role in these endeavours. This study presents the user experience and usability evaluation of PREPARED App—an educational mobile application developed to raise users’ awareness on the ethical dimensions of global challenges through real-life case studies. The captivating narratives, clear structure, ease-of-use, and multimedia content were reported as key strengths of the mobile app by both users (n = 54) and experts (n = 4). Suggestions were also captured for enriching the learning experience through enhanced customisation options, personalised feedback mechanisms, and accessibility features. A set of pedagogical guidelines is extracted to enable instructional designers, educators, and mobile application developers to create accessible and engaging mobile learning experiences.

1. Introduction

The world is facing an alarming increase in the frequency, scale, and complexity of global crises [1], ranging from severe climate changes and armed conflicts to financial shocks and public health emergencies [1]. Coupled with rapid technological developments, including advancements in Artificial Intelligence (AI), these trends constitute major drivers of social disruption and are expected to radically transform the way we live, learn, and work by 2030 [2,3]. These cascading global crises are threatening humanity [2] and are dramatically slowing down the progress towards a fair and sustainable world [4]. In the latest Sustainable Development Goals (SDGs) progress report, António Guterres (Secretary-General of the United Nations) emphasised that we have the obligation to “act with urgency and responsibility” [4], highlighting that education is one of the critical areas where immediate action is required [5]. UNESCO reinforces this vision, stating that “Access to education is not only a fundamental right, but also lifesaving during times of crisis” [1].

During these turbulent times, educating people about the consequences and impact of past and ongoing global crises, activating their awareness on responsible decision-making, and encouraging reflection on ethical dilemmas are no longer considered peripheral actions; they are central for the betterment of our society. As the social order faces such multidimensional crises, ethics education is imperative. Nevertheless, critical reflection requires re-consideration of the role of education in promoting a shared understanding of ethical values, integrity, and responsible behaviours. Educational initiatives can contribute to raising awareness about privacy, equality, diversity, and inclusion; recognising the role of ethics in research and science; and understanding the impact of sociotechnical trends and ongoing transformational changes on responsible decision-making. Equally, the role of education is instrumental in sharpening a society’s preparedness and resilience to mitigate the impact of future crises [6].

While dedicated courses in Higher Education Institutions (HEIs) may address these educational objectives within formal curricula [7,8], innovating pedagogies—such as mobile learning [9], micro-credentials [10], and learning in conversation with AI [11]—can complement traditional approaches and enhance the learning outcomes. Although the popularity of mobile devices and mobile learning (mLearning) increased in recent years, especially following the outbreak of the COVID-19 pandemic [12,13], there is still a limited number of mobile educational apps dedicated to ethics education. This presents both a pedagogical gap in ethics-focused mLearning and an empirical gap in exploring how users experience learning about ethics and integrity through a mobile application.

This study aims to address the identified gaps by conducting an exploratory study to evaluate the User Experience (UX) and usability of the PREPARED App—an educational mobile application developed in the context of the EU-funded project ’Pro-active Pandemic Crisis Ethics and Integrity Framework’ (PREPARED) [14]. The purpose of the app is to raise learners’ awareness on contemporary topics on ethics and integrity, advance their decision-making competencies, and help them critically reflect on ethical dilemmas pertinent to science and global crises [14]. The study makes a methodological contribution to the current literature by adopting a mixed-methods design. A combination of UX and cognitive-walkthrough evaluation is conducted, involving 54 users and four experts, respectively. The study’s empirical novelty is demonstrated through its three-fold emphasis on the educational effectiveness, user experience, and usability of the app. Finally, the study’s pedagogical contribution is culminated through the empirically-informed ’ETHICS’ pedagogical guidelines developed based on the study findings. The guidelines emphasise the necessity for Engaging, Tailored, Human-centred, Inclusive, Customisable, and Sustainable learning, aligned with the principles of Quality Education for (SDG 4) [15]. Following the in-depth discussion of the study findings, these guidelines are presented with the view to enable instructional designers, educators, and mobile application developers to create accessible and engaging mobile educational apps. The following research questions are formulated to guide this research:

1.

How do users experience learning about ethics and integrity related to global crises through the PREPARED App?

2.

What are experts’ perspectives on the usability of the PREPARED App?

3.

What are the key strengths, weaknesses, and areas of improvement of the PREPARED App?

Following the description of the problem context and the study’s aim and contributions, the remaining article is structured as follows: Section 2 presents a review of the related literature discussing the interplay between mobile learning, ethics education, and learning experience design. Section 3 describes the educational purpose, interactive features, and learning content featured in the PREPARED App. Section 4 outlines the research methodology, followed by the discussion of key findings, study limitations and suggestions for further research in Section 5. Finally, Section 6 synthesises the key findings into a set of pedagogical guidelines extracted based on user and expert insights.

2. Background and Related Work

The current literature is critically reviewed and analysed to explore how contemporary pedagogical needs, technological transformations, usability and accessibility considerations, as well as the affordances and constraints of mobile learning, can be synergistically combined to inform the design of educational mobile apps focusing on ethics literacy. The discussion of the related literature is organised into thematic areas covering each of these key domains.

2.1. Contemporary Pedagogical Needs and the Role of Ethics Education

A decade has passed since the United Nations (UN) 2030 Agenda for Sustainable Development (SD) was launched [16]. The 17 SDGs were established to promote core human values like equality and diversity, good health and well-being, quality education for all [15], decent work, justice, and global peace [4]. During this period, millions of people on the planet have acquired access to essential services, gained social protection, and enjoyed a better quality of life [5]. Despite these important gains, escalating geopolitical conflicts, socioeconomic shocks, technological transformations, and severe environmental disasters obstruct progress at the pace and scale needed to meet the 2030 target [5]. Leading global authorities such as UNESCO [17], the United Nations’ Department of Economic and Social Affairs [5], the European Commission (EC), and the World Economic Forum [3], amongst others, unanimously emphasise the need to invest in education.

Ethics education is increasingly recognised as a vital component in preparing individuals to confront and resolve the complex global crises of the 21st century. Ethics-based attitudes, including environmental stewardship, social responsibility, and global citizenship, are considered fundamental skills required for succeeding in an ever-changing and constantly challenged world [3]. Moreover, environmental stewardship is among the top 10 skills with the highest expected increase for the period 2025–2030 [3]. Gaining a robust understanding of ethical decision-making and integrity is crucial for navigating challenges such as climate change, humanitarian emergencies, and technological advancements [18]. Therefore, individuals need to be educated about the consequences and impact of past global crises while also sharpening their preparedness to combat future crises [6].

Integrity and ethical values, cultivated through awareness raising, training and education, are paramount for navigating ongoing and future crises [19]. By fostering ethical reasoning skills, reflecting on ethical dilemmas [17], and exploring different viewpoints, ethics education empowers people to move beyond self-interest and consider the broader societal impact of their actions. Ethical dilemmas constitute an inevitable element in decision-making [18]. Hence, presenting learners with carefully designed instructional material capturing ethical dilemmas can help raise their awareness, activate their reflection and critical thinking, and ultimately lead to positive changes in their behaviours and attitudes. Addressing biases and promoting integrity and fair decision-making is an important aspect of ethics education [20].

Studies have shown that embedding ethics into curricula can cultivate a sense of moral judgment and social responsibility, which are essential for collective action [21]. Furthermore, recent research highlights the role of ethical leadership in fostering transparency and trust during crises, such as the COVID-19 pandemic [22]. Scholars have also proposed novel approaches such as the Planetary Social Pedagogy (PSP) [23], which emphasises the need to establish healthy connections and interactions between people, society, and the planet, while also highlighting the importance of ethical actions that strengthen the health and integrity of all entities [11]. Furthermore, initiatives and frameworks, such as the TRUST code [19,24,25] and the PREPARED code, have been established with the vision to create a healthier and equitable environment, governed by fairness, respect, care and honesty [19]. These initiatives are underpinned by globally understood moral values [25] and inform educational and training materials, policy briefs, and guidelines to enable global awareness on areas pertinent to ethics and integrity [19].

Ultimately, ethics education is not merely about theoretical knowledge; it is a practical and indispensable platform for developing a generation of leaders, learners, researchers, and responsible citizens equipped with the right mindset to make conscious, ethical decisions that contribute to a more just and sustainable world.

2.2. Technological Transformations and Ethics Education

The focus on ethics has been intensified in recent years due to rapid technological advancements, specifically in the area of AI. The world is experiencing a sharp increase in sophisticated, AI-driven cybersecurity attacks, while the dissemination of mis/disinformation and deepfakes on social media has been accelerated, raising severe privacy, safety, and ethical concerns. Human–AI interactions are rapidly evolving, and the impact of using (un)trustworthy AI agents for formulating decisions is growing [8,17,20,26,27,28]. While ethical values, responsibility and accountability are inextricably interwoven in all aspects of life and drive decision-making, they must be carefully reconsidered amid these technological developments.

As a result of these trends, transformative changes are happening at a national, European, and global scale. UNESCO’s recent publication [17] explores the philosophical, ethical, and pedagogical dilemmas posed by the disruptive influence of AI in education, challenges assumptions, exposes frictions, provokes controversies, and sparks new visions for equitable human–AI ’co-creation’. Additionally, the European Commission has established ‘Ethics Guidelines for Trustworthy AI’ [29] and the ‘EU AI Act’ [30]—the first ever comprehensive legal framework on AI worldwide—to foster trustworthy AI use. The key principles underpinning these frameworks focus on ensuring AI-enabled applications are lawful, respecting all applicable laws and regulations; ethical, respecting ethical principles and values; and risk-free, being robust both from a technical perspective and considering the social environment.

Despite these actions, the underlying ethical values are often overlooked, peripherally covered, or approached from a theoretical stance in traditional curricula. This presents a pressing need to rethink education while considering these transformational changes, foster critical thinking, and activate a deeper reflection on the ethical dimension of past, ongoing, and future crises [6,19]. Educating the public about these values (with and about AI) will play a crucial role in guiding ethical decision-making in everyday life. This has informed the instructional design of dedicated case studies included in the PREPARED App.

2.3. Mobile Learning: Affordances, Constraints, and Learner Experience Design

Mobile learning involves utilising mobile devices, such as tablets and smartphones, for formal and/or informal educational purposes. Mobile technology serves as a dynamic medium enabling ubiquitous interaction, collaboration, and learning anytime and anywhere [9]. The educational potential of mobile technology is not a recent discovery; it was captured more than three decades ago [31] and was accelerated during the COVID-19 pandemic, revealing its wide-ranging capabilities as teaching, learning, and social devices [32]. As foreseeably suggested in the Innovating Pedagogies report [33], “Pedagogies change and develop in response to changes in society. They open up new possibilities […] that could produce radical changes in education” (pp. 6–7). Evidently, the COVID-19 pandemic led to a social crisis that disrupted not only what people learn, but also how, when, and where they learn [13]. During the emergency transition from traditional to remote education, the role of mobile computing technologies has been vital on social and educational fronts alike [12,13]. Today, mobile technologies offer a dynamic medium for establishing social connections, maintaining ongoing interactions, and dynamically co-creating an online community. Users leverage mobile devices to stay connected and learn about what is happening in the world around them. At the same time, current trends have contributed to the increasing utilisation of mobile devices for more formal educational purposes, including mobile Game-Based Learning (mGBL) and microlearning, establishing mLearning as one of the most omnipresent pedagogies across all levels of education, as well as in workplaces and social settings [34,35,36,37,38,39]. The key affordances and constraints of mLearning are discussed next.

2.3.1. Affordances of Mobile Learning

Beyond increasing user performance, enhancing user capabilities, and offering direct access to an array of social and entertainment platforms, mobile apps have a wide range of affordances and can support collaborative and individual learning [9], knowledge sharing, information acquisition, skills development, and decision-making [40].

Continuous and seamless learning. The portability and affordability of smartphones and tablets, compared to personal computers [41], make their use naturally more flexible, convenient and accessible [42,43,44]. These technical characteristics enable users to access learning content and educational resources across different contexts, platforms, and environments, seamlessly embedding educational experiences into their daily routines while at work, at home, or on the go [9,41,45]. These features have also made learning seamless—both when an individual is learning independently through mobile technology (self-directed or autonomous learning) [46] and when learning with others (collaborative or competitive learning) [47,48]. Seamless learning implies that the learning process can be continuous, adaptive, and accessible whenever and wherever this is relevant [13,49,50], according to the learners’ real-time context and needs [44,49,51]. The seamless nature of mLearning can, therefore, help advance learners’ awareness of trending challenges pertaining to ongoing global crises.

Customisation and personalisation features. Unlike traditional, classroom-based learning or Massive Open Online Courses (MOOCs) following the ’one-to-many’ mode of instructional delivery, well-designed mobile apps can make learning experiences feel more personal, enabling a stronger connection with the learning content. mLearning enables users to learn at their own pace, adjust the learning speed or revisit learning content according to their learning preferences, track their individual progress, and choose specific content to focus on based on their context, unique needs and personal interests [49,51]. Furthermore, most mobile apps include User Interface (UI) personalisation or content customisation features that enable self-regulation [43], enhance student engagement [52] and boost comprehension [53]. When learners have the option to orchestrate their own (mobile) learning, this helps sustain their interest [49], increase their participation and achievement [45], and raise their self-esteem, self-confidence, and learner autonomy [46]. These qualities are instrumental, especially in educational applications like the PREPARED App, which focuses on raising awareness on complex concepts (such as ethical dilemmas underpinning contemporary global challenges) or developing higher-order skills and competencies (such as critical thinking, ethical reasoning, and sustainability literacy).

Structured content. Educational mobile apps and quiz-based mobile educational games typically present narratives and learning content in a hierarchical or thematically organised structure that can be easily accessed on a mobile device. Unlike online learning through Web browsing, which is typically unstructured and hence may yield an overwhelming amount of information or irrelevant content, leading to confusion and increased cognitive load, mobile educational apps with a well-designed UI and a stepwise, structured path through the learning content can help reduce cognitive load and increase learning effectiveness [41,47,54,55,56]. This affordance is evident in the design of the PREPARED App.

Multimodality and Interactivity. In educational applications, multimodality encompasses diverse interactions, extending beyond traditional text to include video, the Internet, music, and other modes of interaction, and can therefore cater for diverse learning dis(abilities) and preferences [57]. Mobile apps can offer enhanced features beyond simple content delivery [39], including content interactivity and social interactivity. Interactive design enables immediate and multimodal on-screen feedback based on users’ actions or responses to in-app games or quizzes (including visual, auditory or haptic feedback), while social interactivity is possible through embedded user messaging functionality, competitive game modes and leaderboards, content sharing, and other social capabilities enabling collaborative learning [52,58].

Engagement, motivation, and playful experiences. Educational mobile apps provide more dynamic and unique learning experiences compared to traditional methods [43]. Moreover, many mobile educational apps include captivating gamification features and game mechanics, like points, leaderboards, badges, and rewards, yielding engaging learning experiences [52,58,59,60]. Such affordances help maintain learners’ interest and motivation while increasing their participation in educational activities [45]. These features are conducive to learner engagement [13,44,49,50,51], constituting mobile technology as an appealing learning medium in both formal and informal learning contexts [46,48].

2.3.2. Constraints of Mobile Learning

In addition to the strengths associated with mobile learning, several weaknesses and constraints are reported in the literature, which negatively impact learner engagement and the educational efficacy of mobile apps. Exploring these constraints and reviewing existing mobile apps specialising in ethics-related topics was an important step during the requirements specification phase of the PREPARED App and subsequently informed the design and development phases.

Technical constraints and cognitive load. Mobile apps and mLearning are inevitably constrained by the inherent technical characteristics of mobile devices (e.g., small screen size, low display resolution, limited processing capabilities, single input modalities, and remote context of use). These factors influence the way users interact with mobile apps [40,42]. Although most users can read short, simple textual content on mobile devices with comparable ease as on any computer screen, they may slow down when reading difficult text on a mobile device [56]. A practical challenge is the small screen size of mobile devices, which imposes the use of a smaller font size. This may lead to dense, hard-to-read text or compact graphics, which altogether reduce readability or obstruct the reading flow due to the need to zoom in and out, especially in the case of detailed diagrams [40,61,62,63].

Furthermore, due to the limited screen space, users can see a smaller fraction of text at any given time and, thus, must rely more on their memory to access contextual information needed during reading. This results in a higher working-memory load and requires excessive mental resources [64], which cannot be sustained for a long period of time. Increased cognitive load is associated with several negative consequences [61,64], including hindering information transfer [41] and users’ comprehension [56], reducing learners’ enjoyment and sense of engagement [65], and degrading the overall learning experience [13]. These processes may, in turn, lead to poor learning performance [66], negatively impacting the effectiveness of the learning process altogether [67].

Additionally, although mobile devices are easy to use, they are less convenient than using a personal computer for certain learning-related tasks [68], such as note-taking or typing, searching for information in long textual content [37,69], multi-tasking or switching between multiple files or screens. Additionally, learning tasks which require a more complex interaction process, such as activities deployed on a Learning Management System (LMS) [41], lab-based tasks, or collaborative project-based learning (PBL), could be ineffective on a mobile device. Other technological limitations must also be considered [40,42], including platform compatibility or the dependency on stable Internet connectivity. Such technical aspects must be carefully considered for mobile interactions to be effective [40,44], especially in the case of educational apps.

Pedagogical and UI design constraints. The instructional design of the learning content embedded in the app and the app’s user interface design are two inter-related yet distinct challenges. Often, learning theories and instructional design principles are overlooked during the mobile application development lifecycle [70], leading to text-heavy, complex, or poorly designed mobile educational apps. Poor UI design can make learning more cognitively demanding [41]. Therefore, it is important to consider who the target audience is, how they learn, and what could motivate them to use the app. These considerations are key for sustaining, not merely attaining, their engagement [71]. This entails understanding how users’ first impressions may affect the overall user experience [72], figuring out how to reach each individual learner more deeply [73], and catering to the varying digital proficiency levels of diverse learners [44].

Privacy and social distractions. As we delve deeper into an increasingly interconnected, automated, and AI-driven digital world, users are becoming highly dependent on their personal mobile devices. This poses two key challenges. Firstly, mobile devices carry valuable personal data. Users need to be informed about what personal data are collected by the mobile apps they use, how these are processed, and by which entities. Therefore, privacy, security, and safety have become paramount [42,74]. Unquestionably, raising user awareness on these principles is imperative. In the context of the present study, these principles are reflected both in the development of the PREPARED App and the instructional design of the learning content embedded in the app. Secondly, constant connectivity and interactions through social media mobile apps pose several challenges learning-wise. Social media interactions may lead to increased distractions that can hinder learners’ concentration. This is, in turn, associated with frustration and eventually inhibits mobile learning. Such continuous and ad hoc mobile-enabled social interactions may act as barriers to the learning process as a whole [13]. It is therefore crucial to collectively consider technological, social, contextual, and pedagogical aspects and how they impact the effectiveness of mobile learning.

2.3.3. Usability, Accessibility, and Learner Experience Design Considerations

To complement the insights gathered from the in-depth exploration of the affordances and constraints of mobile learning, usability and accessibility aspects also informed the design of the PREPARED App. Attending to these human-centred aspects ensures that the mobile educational app demonstrates both educational efficacy (i.e., achieves the intended learning outcomes) [58] and engagement efficacy [52] (i.e., enriches the overall learner experience).

Usability. Usability refers to ensuring that interactive products are easy to use and enjoyable from the user’s perspective, enabling them to successfully carry out any work-related, educational, or everyday activities. Usable applications satisfy the following key goals: effectiveness, efficiency, safety (safe to use), utility (having a meaningful purpose), learnability (easy to learn), and easy to remember how to use (memorability) [75]. The same principles apply to mobile apps. Usable mobile apps enable users to achieve their tasks effectively and efficiently within a specified context of use [76] and are associated with increased user satisfaction.

In the context of educational mobile apps, specifically, designers must prioritise brevity, clarity, and efficiency [56,63], eliminating unnecessary content and ensuring learning material is well-organised. This will help avoid negative emotions like frustration and confusion, which lead to disengagement. The mobile app must be free of design-oriented or navigation flaws that may hinder learning [77], and the UI design must follow usability guidelines to enable positive learning experiences [41,45,56]. Attending to the usability of mobile educational apps can enable knowledge transfer through appealing visual communication [78], straightforward app navigation, and well-designed learning content, which helps reduce the negative impact on the learner’s cognitive load [45,55,56,61], leading to an engaging and enjoyable learning experience. Evidently, usability, user interface design (UID), and User Experience Design (UXD) considerations are central when designing and developing educational mobile apps since they affect user engagement, adoption, and sustained use [40,52,58,79].

Accessibility. Mobile environments present unique interaction patterns primarily enabled through touch gestures; hence, they differ from traditional Web or desktop-based interactions [80]. Developers include several accessibility options ranging from adjusting the colour palette or theme (e.g., between dark, light, custom, or system default) to match users’ preferences to allowing users to zoom in/out, adjust the font size and orientation, and enable or disable certain features such as audio support, word-by-word reading, notifications, timer, vibration, etc. Through these features, mobile devices enhance accessibility and play a supportive role in improving educational outcomes for students with disabilities [57]. Several sets of heuristics and accessibility guidelines (e.g., Web Content Accessibility Guidelines (WCAGs) [81], Game Accessibility Guidelines [82], and Nielsen’s heuristics [83]) are strategically adjusted for mobile apps [80,84] to achieve an improved experience for all mobile users, including those with disabilities.

Learning theories and multidisciplinary perspectives. Educational mobile applications cannot be created using the same principles or standards as entertainment, performance-oriented, or generic mobile apps because those are neither intended nor optimised for teaching and learning [41]. Instead, the design and development of educational technology must be founded on sound learning and engagement theories [70,85] and guided by pedagogical constructs and instructional design principles. This can be achieved by forming multidisciplinary development teams comprising domain/subject experts, educators, instructional designers, system analysts, UI/UX designers, mobile application designers and programmers, amongst other professionals. Learning experience design (LXD) is an emerging research field addressing these issues. It resides at the intersection of instructional and usability design, UX perspectives, and human–computer interaction (HCI) [86,87,88]. Under LXD, the emphasis is on designing quality learning experiences rather than on the design of the materials and tools to be used [86]. Learning experience is seen as the combination of cognitive engagement, active participation with the learning tasks, and affective responses to the learning context [85,87]. These human-centred aspects are important for designing effective, impactful, and engaging educational tools and ensuring positive learning outcomes. These aspects are particularly relevant when developing mobile apps to educate learners on domain areas that cut across all disciplines and subjects, like ethics and integrity.

2.4. Existing Mobile Apps Focusing on Ethics

Today, mobile apps are used by millions of users globally and cover a wide range of domains [40]. An example particularly relevant to ethics and integrity, and thus the current study, is the Dilemma Game App [89], which was designed to challenge students, scientists, researchers, and professional services staff to reflect on their choices and moral boundaries. It is founded on the view that integrity is an important part of an open, safe, and inclusive academic culture and that genuine reflection and discussion can foster a stronger academic community [89]. Researchers are commonly confronted with moral dilemmas or face delicate issues relating to the professional code of conduct and integrity in research. These include questions related to whether certain results can be omitted from a research study, dealing with unwanted behaviour in a research team, or deciding whether to add colleagues as co-authors if their contribution was trivial, among other situations. Through the Dilemma Game App, users are confronted with such thought-provoking dilemmas and can discuss these issues with others in an open and safe setting, allowing them to think critically about different situations. Its aim is to support users in further developing their own moral compass [89]. The app shares a new dilemma every month, and users can submit their preferred course of action, after which they can see what others have voted for. There is also an option to read a relevant review by an integrity expert. From an educational point of view, this is a key strength of the app, as it provides an in-depth analysis of the dilemma at hand while also capturing the underlying principles that apply in the specific dilemma. In this way, these principles can inform users’ future decisions and actions. Users can also submit their own dilemma and receive feedback from the research community. Finally, the game incorporates group and lecture modes, facilitating collaborative learning and critical dialogue. This app differs from the PREPARED App, as it focuses primarily on research-related dilemmas that are classified in terms of three categories (researcher position, research strategy, and research phase) [90], whereas the PREPARED App focuses on an extensive range of real-world challenges, as discussed in the next section.

Other apps were also reviewed during the analysis phase to generate ideas and identify gaps in existing solutions. A key observation was that the majority of existing mobile apps primarily include content pertaining to professional codes of ethics or business ethics. We identified a set of domain-specific mobile educational apps that touch upon specialised ethics-related subjects, such as AI ethics and privacy, business law and ethics, human values and ethics, military and healthcare ethics, among others. Most of these apps feature a simple design and layout and include informational learning content. Some also include trivia quizzes or gamification features, offering a more interactive format. However, the quest did not yield any mobile apps specifically focusing on the role of ethical values and integrity pertinent to real-world crises, presenting a gap and a genuine opportunity for conducting the current study.

3. The PREPARED Mobile App

Awareness raising, education, and training are central to becoming better prepared for future crises [6,25]. The purpose of the PREPARED App is to provide a digital platform to complement available learning resources developed to promote research ethics and integrity decision-making during global crises [91]. The app addresses the two-fold aim of ethics training, namely, to increase understanding and raise awareness of ethics topics, and enhance engagement in deep ethical reflection and the development of ethical competencies. The app engages learners in a reflective learning experience, allowing them to navigate ethical dilemmas pertaining to global crises [14]. This is achieved by integrating multimedia content with illuminative, real-life thematic case studies to enable transformative learning [92]. Through this unique combination of features, the app aims to enhance learners’ familiarity with key terminology associated with the ethical code of conduct in research, science and beyond; activate their reflection on biases, ethics, and integrity in decision-making; and sharpen their knowledge of contemporary topics associated with ensuring that international research is equitable and conducted based on ethical grounds. The following sub-sections discuss the educational purpose, interactive features, and types of learning content featured in the PREPARED App.

3.1. Pedagogical Framework and Learning Experience Design

The purpose of the PREPARED App is to provide a digital platform to complement ethics training in terms of research ethics and integrity decision-making during global crises. Cross-platform development was employed to support Android and iOS, which are the most widely used smartphone platforms. The app contains two types of training materials, case studies and short courses, captured as stories. Stories are designed to present users with authentic scenarios put together by experts in the field (instructional designers and experts in ethics). The completion of these stories (along with associated quizzes, videos, polls, etc.) requires users to engage in the learning process, use their critical thinking skills, and encourages them to weigh various perspectives in different activities and contexts.

The educational design is based on the principles of microlearning [93,94,95,96] and constructivism. ‘Mobile microlearning’ [94] empowers learners by delivering targeted, concise, and accessible learning experiences [96], while ‘constructivist mobile learning’ [97] places an emphasis on learner profiles and how learners construct their own knowledge through their experiences [98]. The combination of these pedagogical theories is realised through real-world problems presented as stories, as well as quizzes and live polls, encouraging deep learning, and diverse interactive activities intended to promote knowledge construction, critical thinking, and reflection.

Each story is broken into smaller, manageable segments called pages, which progressively introduce the story, characters, and different perspectives or activities. This makes both the stories and the app itself more modular, allowing the definition of various types of content and their use on mobile devices. Furthermore, this modular approach is advantageous for learners, as it prevents cognitive overload [41,47,54,55,56,66] by reducing the content shown to the users and pacing the learning process throughout each story.

The instructional designers can develop stories that leverage several pedagogical principles in line with the studied literature [43,49,53]. Firstly, the stories facilitate learner-centred design, encouraging learners to be autonomous and make progress at their own pace [46]. This is facilitated by allowing learners to navigate back and forth between pages at will, revisit material, and make choices that can influence or control the flow of the presented content.

The interactive features of the app (discussed in detail below) also foster active learning [99]. Through simulated dialogues presented in a conversational narrative format, users can construct knowledge through considering and questioning different perspectives. Equally, components such as polls enable learners to provide their own opinions and, after submitting their answer, to be able to see what others have voted. This real-time feedback can boost learner engagement compared to passively consuming learning content. The dilemmas posed during the stories present different and frequently opposing views of controversial topics. These are often presented in a dialogue form that stimulates reflection, discussion, and critical thought. Furthermore, different modalities, such as text, audio, images, video, transcripts, etc., are used to accommodate diverse learning needs under different contexts, thereby enhancing the accessibility and inclusiveness of the app. Additionally, several stories reward learners with badges when they complete stories with a certain score, which can provide extrinsic motivation and enable them to showcase their achievements beyond the application and the stories they have engaged with. The learning material is designed to be informative and scientifically grounded while challenging the users to question their own assumptions and potentially come to new conclusions.

3.2. Interactive Features

The component-based software architecture of the app allows it to support the development of dynamic, extensible, re-usable, and interactive content types, enabling instructional designers to deliver content in an engaging way.

Firstly, HTMLComponent (Figure 1) supports basic content that is traditionally composed of text, pictures, and hyperlinks that connect the story’s material to external resources. VideoComponent provides a way for instructional designers to embed videos in the story. Additional content can be defined to supplement the video using text and images. Users can interact with videos through controls such as seeking, full screen mode, subtitles, and adjusting the video speed. In addition, the app prevents learners from skipping to the next step without watching most of the video. AudioComponent allows playback of audio files relevant to the stories, and it is accompanied by supplementary resources, like a transcript of the audio in text form, to make learning more accessible.

Several components add interactivity to the stories by allowing learners to make decisions and provide their opinions through different activities. DiscussionComponent simulates a discussion taking place between two users using chat messages. The format (Figure 2) looks very similar to chat applications, showing the actors’ names and avatars, as well as speech bubbles. Typically, the two actors involved discuss two opposing points of view, attempting to engage the learner in the discussion and facilitate critical thinking. Another interactive feature is PollComponent, which presents learners with an ethical dilemma related to the case study. In such components, there is typically no right or wrong answer, as it is often a matter of one’s ethical interpretation. After submitting their responses, users receive individualised feedback based on their selection or more generic feedback (depending on the instructional designer’s preference), and they can also see how other learners have responded to the polls, allowing them to reflect on the popularity of their response. A similar component is MCQComponent, allowing instructional designers to create Multiple-Choice Questions (MCQs) to assess learners’ understanding of the material delivered during a story. Feedback is provided to the learners after their responses, allowing them to reflect on how well they are doing during the story’s progression (Figure 2).

Another interactive element is SortingComponent, which allows users to sort options into the correct categories by picking them up and dragging them to the corresponding buckets (i.e., drop-points on the screen). To finish this activity and move on to the next stage, learners must match all options to a category and press ‘Submit’ to receive their accuracy score, depending on how many items were categorised correctly. ExamComponent can pose several MCQs in the form of a quiz or exam. In this component, learners are assessed on the concepts presented by the underlying story. Such components are strategically placed at the end of each story and are coupled with BadgeComponent. A learner who manages to attain more than a certain percentage of correct answers (e.g., 80%) can opt to receive a badge related to the story they have completed. By inserting their name and email address, they will receive a link to the badge (Figure 3). Finally, an important utility component is BranchComponent, which allows instructional designers to branch out their stories to different paths based on how a learner progresses through the story. One of the motivations behind the branch component is to support a user-centred learning experience by allowing learners to customise how they go through each story based on their own knowledge.

3.3. Availability and Performance

Mobile devices, such as smartphones and tablets, constitute one of the primary media for consuming and communicating information [100]. The two most popular mobile platforms are Google Play and Apple App Store, offering about two million apps each. The PREPARED App uses a cross-platform development approach, where the application is developed using a single codebase and then compiled into the corresponding platform by the underlying framework. This enhances the development process, as it results in lower development time and overheads, faster time-to-market, and a more maintainable code base [101]. Cross-platform application development frameworks, such as Flutter, React Native, and Xamarin, have become increasingly popular in the last five years and have several benefits in terms of developing educational applications. One of their most important advantages is the ability to provide educational content on multiple platforms with a single codebase, thereby increasing the coverage of the app in terms of the number of learners. Secondly, the cross-platform approach can enable educators to offer a consistent learner experience across multiple types of devices. With this approach, developers do not need to conform to platform-specific designs and adaptations that might introduce inconsistencies in style, navigation, and the overall interaction design of an app [102,103]. Finally, various studies have proven that the performance gap between native and cross-platform applications has shrunk significantly, making the development of educational apps—which are not performance-critical software—using this approach even more beneficial [104].

3.4. Data Privacy

Data privacy and compliance with General Data Protection Regulation (GDPR) [105] are central to the PREPARED App—both in terms of instructional content and implementation. The app does not use any cookies for tracking user activity, nor does it collect, store, or share any personal user data. Using the app is governed by an explicit Data Policy, which is accessible via the PREPARED App through a direct link [106], as well as via the corresponding app stores listing it. The users are often asked to vote among proposed actions. This results in submitting data on a cloud service so that the aggregated results can be reported back to the users to encourage further reflection and engagement. This information is only linked via a randomly generated Universally Unique Identifier (UUID) that can be reset by the user at any time. If users opt to reset it, then all their poll selections are also reset to maintain privacy. Lastly, users who utilise the Badges functionality have the option to specify their name and email so the generated badge can mention them. Again, this is optional. If the option is selected, the users still have control of the generated badge. The Data Policy also explicitly states, “You should refrain from sharing, uploading, displaying, or sending your or any other person’s personal data to us, in the App and/or Services” [106].

4. Research Methodology

The purpose of this study was to evaluate the user experience and usability of the PREPARED App. A mixed-methods approach was employed to gather insights from users (adult learners) and UI/UX design experts. Gathering user perceptions and experiences and exploring prominent usability aspects can provide an assessment of the degree to which mobile apps meet users’ needs and expectations [40].

4.1. Informed Consent and Ethical Approval

The research described in this paper was assessed and approved by the Cyprus National Bioethics Committee—reference number 2025.01.38. Written informed consent was obtained from all participants in the study.

4.2. Research Participants

Both users (n = 54) and experts (n = 4) were invited to use and evaluate the PREPARED App, provide insights about their experience and overall satisfaction with the app, identify positive and negative elements, and give suggestions and feedback to improve the app’s UI design, overall quality, and educational efficacy. To calculate the sample size in advance, we used G*power (version 3.1.9.7). Even though we ran other tests as well, we used G*power for the Mann–Whitney test, with a large effect size of 0.80, a significance level of 0.05, and a power of 0.80, to get an ideal total sample size of 60.

Insights from users were gathered in two phases. During the first phase, the anonymous questionnaire was shared among the project consortium partners. This purposive convenience sample yielded 23 responses. Subsequently, the questionnaire was widely disseminated through email and social media channels, generating an additional set of 31 responses. Preliminary analysis did not reveal major differences between the two groups. The data was collectively analysed, guided by the research questions. In addition to user evaluation, four UI/UX experts were invited to evaluate the PREPARED App. The combination of user and expert perspectives provided rich insights into the app’s strengths and highlighted prominent areas and suggestions for improvement and future development.

Basic demographic information was captured through the questionnaire, ensuring that diverse perspectives and contexts are represented. Providing a country was an optional question. Among those who declared their country, most respondents were based in Cyprus, but users from Kenya, Germany, Italy, and the UK also evaluated the app. Participants from all age clusters (18 to 50+) took part, with 33% being in the age range of 28–39 years old, followed by the age groups 40–49 (30%), 18–22 (19%), 23–27 (11%) and 50+ (7%) years. The participation of female (n = 29, 54%) and male (n = 24, 44%) users was recorded (one respondent preferred not to disclose their gender).

Among the respondents, 22 (41%) are Higher Education students, including undergraduate (Bachelor’s) (n = 12, 44%), postgraduate (Master’s) (n = 8, 30%), and PhD students (n = 3, 11%). Out of the remaining 32 participants, the majority are academics (n = 17, 53%), followed by industry professionals (n = 7, 22%), public organisation or service sector employees (n = 5, 16%) (the remaining responded “Other”). Participants from a wide range of fields took part, including pure and applied sciences (n = 33, 61%), humanities and social sciences (n = 11, 20%), engineering (n = 4, 7%), and medical and health sciences (n = 2, 4%), amongst others. Although most of the respondents (n = 43, 80%) reported having previously used a mobile app to learn a new skill or learn something new, their level of expertise varied, both in terms of using educational mobile apps (Figure 4) and in terms of UID/UXD experience (Figure 5). This distribution was constructive, as it brought together diverse perspectives based on users’ different levels of familiarity with these aspects.

4.3. Research Methods

The app is publicly available and free to download [92]. Simple instructions were shared with the participants, guiding them to download, use, and, in turn, evaluate the app. For consistency, the participants were provided with the research protocol, which included a list of activities to follow and the link to the questionnaire. They were first guided to explore and navigate the PREPARED App freely (3–5 min); browse the various case studies and navigate to the various pages (5–8 min) to familiarise themselves with the UI layout before switching to the learning mode; and find and read the privacy and help documentation embedded in the app (2–3 min). Following this orientation phase, they were invited to choose among the short courses and case studies to learn more about ethics, read the text, watch the videos, interact with the app by submitting their answers to the quizzes or polls, and switch between different case studies. After using and testing the app (for a minimum of one hour), the participants were instructed to evaluate its usability and their user experience by honestly responding to the evaluation questionnaire (10–15 min). The questions addressed key areas such as: user experience and satisfaction; prioritisation of key features based on how much they impacted users’ learning experience; educational effectiveness and usefulness of the app; enjoyability and engaging features; ease of use; and quality of instructional multimedia content. Several types of questions were included: five-point Likert scales, sorting (drag-and-drop), and multiple-choice questions. Additionally, the questionnaire included three open-ended questions to capture the respondents’ views and perceptions on the most negative and most positive aspects of the PREPARED App, plausible suggestions for improving its usability, educational efficacy, or new features they would like to be added. Thematic and frequency analysis of these qualitative responses revealed interesting patterns and offered rich insights from users. Selective verbatim quotes are presented in the next section to enrich and contextualise the discussion of the findings.

A similar research protocol to that used with the users was employed with the experts to ensure consistency. Based on this, experts were provided with an indicative list of activities to perform. This included free app navigation during the orientation phase, exploration of selected case studies, and overall evaluation of the app’s usability and UI design. Instead of submitting a questionnaire, the cognitive walkthrough approach [107,108] was utilised with the experts. Cognitive walkthrough is an efficient and affordable usability inspection method [107,108,109] that helps uncover system usability issues [110,111]. Evaluators follow a procedure simulating a user’s problem-solving process at each step, checking if the simulated user’s goals, intentions, assumptions, and memory content can successfully lead to the next correct action and ultimately achieve the required outcome while using the system [107]. Any discrepancies are noted during the inspection process, informing subsequent design cycles. Following this approach, experts were invited to “walk through” the app and articulate their thoughts, share their impressions while using the PREPARED App, identify any navigation deficiencies or design flaws impacting the intuitiveness and effectiveness of the app, and highlight successful features and areas of improvement. Each cognitive walkthrough session was conducted independently to maintain anonymity and encourage the collection of unique perspectives. Each session lasted approximately 45 min. The experts downloaded the app on their personal devices to trigger more diverse perspectives and identify platform-specific or other technical issues. The experts’ responses were captured through field notes based on the experts’ verbal feedback as well as visual cues observed while they were using the app. Additionally, the experts took screenshots on their devices while using the app, which they shared along with brief written feedback or screen annotations to highlight a particular feature of interest or provide suggestions for improvement. The gathered evaluation data were initially coded and analysed by one of the authors, and, subsequently, intra-team validity checks were performed to reach team consensus. To achieve common ground among all researchers, coding reviews took place to validate the coding structure and adjust, merge, or rename thematic categories. The resulting analysis was confirmed during a consensus meeting. This ensured that the final data coding, themes, and interpretation reflect the team’s collective analysis.

5. Results and Discussion of the Findings

The purpose of this study was to explore users’ and experts’ experience using the PREPARED App, evaluate its usability, and invite suggestions and feedback to improve its design, overall quality, and educational efficacy. User feedback was combined with cognitive walkthrough sessions with experts, providing a detailed review and evaluation of the app. The gathered questionnaire responses and field notes provided useful insights, which are discussed next.

5.1. User Experience Evaluation

The first research question focuses on exploring how users experience learning about ethics and integrity pertaining to global crises through the PREPARED App. In the context of this study, user experience encompasses aspects such as learning experience, user engagement, and overall user satisfaction with the app, including enjoyability, comfort, and the match between user expectations and reality. The results show that users positively rated the PREPARED App across all these dimensions. The analysis also delves deeper into the gathered data to analyse subtle user perspectives—even when these were reported by a minority of the respondents. This provided a wider perspective on the gathered insights and a source of inspiration for further improving the app.

Multimedia content (videos, text, audio, and images) and a combination of features and educational activities (e.g., quizzes, live polls, dialogue simulation, sorting, etc.) are integrated to achieve the purpose of the app, that is, to provide a digital platform for raising awareness on the ethical dimensions of global challenges. Consequently, users were invited to rate which of these they consider the most impactful for their learning experience. Unsurprisingly, videos represent the most popular feature resonating with users, with 33% of the participants selecting this as their first choice (Figure 6) shows the weighted totals, where the first choice has a weight of 10, whilst the last choice has a weight of 1. The second most impactful feature is text, which was the first choice for 31% of the participants, closely following videos in terms of impact. This finding indicates that the importance of textual learning content is comparable to that of video-based content, suggesting that a balance between these is essential. Pictures and audio clips were perceived as less impactful than text, with 11% and 2% selecting them as their first choice, respectively.

Another observation is that among the interactive features (quizzes, grouping/sorting, live polls, and dialogue simulation), the most popular were the quizzes. Users tend to express their preference for learning tasks that are more familiar to them, especially in novel applications. Moreover, the badges feature was the only one not selected by any participant as their first or second choice, landing at the bottom of the list. Given the fact that gamification features and mGBL are conducive to learner engagement [52,59,60], this finding suggests the need for considering the multidimensional nature of learner engagement with educational technology [85,112]. There was no difference in the preferred features between age groups or users with different levels of experience with mobile educational apps.

Similar insights were captured when students were asked to select the most engaging aspects of the PREPARED App (Figure 7). The aspect considered most engaging is multimedia content (n = 43), confirming that users appreciate a good balance of text, videos, images, quizzes, polls, dialogues, etc. This is important, as different content types and learning methods will resonate better with different learners according to their preferences and learning styles. The step-by-step approach was also highlighted in the results (n = 37). The principle of breaking the content down into small and manageable steps was satisfied [56,63], making it easy to follow on a mobile device. The interactive nature of the case studies embedded in the app (e.g., ‘Nuclear accident in Fukushima’, ‘Scientific collaboration during war’, and ‘Ethical challenges during COVID-19’) and the use of clear, concise language, making the content easy to understand, were two factors that were also highly perceived by users (n = 34). Badges were again listed as the least engaging feature. This was an intriguing finding, indicating that for reward systems to work and engage learners, they need to be properly contextualised to attract learner interest and motivation [73,85]. This emphasises the importance of attending to the human factors pertaining to LXD and learner engagement when designing educational technology [86,88,113].

In addition to user preferences and engagement, user experience with an educational technology depends on the quality of interactions, which includes the quality of the multimedia content. Inspired by the studied literature, the evaluation aimed to explore the users’ perspectives on the quality of the design and multimedia elements embedded in the app. Being an educational application, aspects such as information clarity and information organisation are crucial for achieving the expected educational outcomes and the app’s mission. This is even more critical when the content focuses on non-trivial or controversial topics, ethical dilemmas, and topics requiring deep learning and engagement.

Equally, in acknowledging the inherent constraints and weaknesses of mobile devices and mobile apps (as discussed in Section 2), it was deemed important to evaluate the user experience regarding these qualities. The analysis of the participants’ responses indicates a positive experience across all aspects. The highest-ranked aspect was the visual quality of the videos (embedded in the case studies), with 40% rating it as Excellent and 35% as Very Good, followed by information clarity (39% and 43%, respectively) and design elements (37% and 35%, respectively). While the visual and sound quality of the videos was ranked high, conversely, the pace of speech in the videos received the highest percentage of respondents who considered it Poor (almost 4%) or Fair (13%). Combined with the earlier finding indicating that videos are the most popular feature of the app, this signifies that aspects affecting the quality of their experience may impact learner engagement in the long term. For instance, if learners find the video playback pace too slow or too fast, then it will be difficult to sustain their interest in the learning content, even if videos constitute their preferred learning medium. Additionally, although there was an option to change the playback speed, this was apparently not identified by some users; hence, customisation features must be easy to discover.

High-resolution screenshots of videos, as shown in Figure 8, provide a better insight into the app UI as experienced by the users. The first screenshot shows the video in normal play mode. As is common in most video players, the controls are hidden from the UI at this point to maximise the video visibility. When touched once, the controls appear for a few seconds, as shown in the second screenshot. Even though standard conventions are used, like the ‘full screen’ icon at the bottom right, during the early prototypes, some users found it difficult to understand or access it. For this reason, we implemented an additional button with the explicit affordance to ‘View video in full screen’, which is always visible. This was well received by the prototype users. Finally, some users indicated that the video pace was slow. Even though the video playback speed is configurable, as shown in the third screenshot, discovering and activating this feature proved lacking. The three-dots icon on the top right follows the standard convention for overflown menu options. However, this has the inherent limitation that only users willing to explore the UI will become aware of the additional functionality. The results concerning all aspects of quality of accessibility are summarised in Figure 9.

Users were also invited to rate the educational effectiveness and usefulness of the PREPARED App (Figure 10). The findings are encouraging, as most users agreed or strongly agreed that using the PREPARED App increased their awareness about ethics and ethical decision-making (91%); the app was effective in teaching them about ethics (91%); it is useful (89%); they have acquired new knowledge about ethics through the app (83%); it provides engaging, concise content for optimal training and impact (83%); the sense of accomplishment when completing the various tasks (case studies, quizzes, etc.) was high (76%); it satisfies their specific learning needs and expectations (74%); and they have enjoyed learning through the app (74%). Finally, although 72% of respondents agree or strongly agree that they have developed new skills in ethics, which is positive overall, looking closer at the findings shows that this feature received the highest collective percentage of neutral, disagree, and strongly disagree (28%) responses. One plausible explanation for this could be the limited time and effort some users devoted to learning by using the mobile app before evaluating it. At the same time, it indicates that developing skills is considerably more cognitively demanding than acquiring new factual knowledge. Further research is needed to better understand this phenomenon in the context of mobile learning.

In addition to evaluating their learning experience and the educational effectiveness of the app, participants were asked to rate their level of satisfaction with it (Figure 11). The results show that most users agreed or strongly agreed they feel comfortable using the app (87%), that the UI is pleasant to use (83%), and that the app works as expected (83%). The results are positive, with 50% strongly agreeing and an additional 33% stating that they are satisfied with the app. Considering that enjoyment and satisfaction with a mobile app influence the users’ engagement with it upon initial adoption [114], the evaluation findings show the potential of the PREPARED App for offering an engaging learning experience. This applies to all users, irrespective of their academic background or level of familiarity with mobile learning.

5.2. Usability Evaluation

The second research question focuses on evaluating the usability of the PREPARED App. Both users and experts found the mobile application usable, easy to use, and enjoyable. Participants agreed or strongly agreed that using the app is straightforward, without the need for additional help or instructions (89%), and easy to use (87%). Users can efficiently navigate the app (85%), are satisfied with its usability overall (85%), did not notice inconsistencies (78%), and could efficiently accomplish tasks by taking the fewest steps possible (74%) (Figure 10).

When delving deeper into the gathered data, a noteworthy observation is that the aspect that received the highest combined percentage of neutral/disagree responses was flexibility and customisation (13% disagree, 35% neutral, and only 52% agreed or strongly agreed). There seems to be no connection between negatively rating this aspect and the respondents’ level of expertise, since out of the seven (13%) participants who responded that they disagree, three are competent users of mobile educational apps, three are beginners, and one declared themselves as a novice. The same applies to age group, profession, and gender (three women and four men disagreed). Interestingly, however, six out of the seven who disagreed are in the field of sciences, and the other one is in the field of Engineering. Although the number of respondents is small for reaching any generalisable conclusions, this highlights the importance of gaining a deeper understanding of the role of flexibility, personalisation, and customisation in mobile learning across diverse user groups and user characteristics. As discussed in the literature, flexibility, customisation, and personalisation features are aligned with qualities conducive to effective learning, such as self-regulation [43], self-confidence and learner autonomy [46], sustained interest [49], learner engagement [52], improved comprehension [53], and increased participation and achievement [45]. This was further captured by the responses to the open-ended questions presented in the Tables below. Therefore, effective apps must provide diverse customisation options allowing users to adjust the UI and the learning content presentation format based on their preferences, learning styles, or context. This is essential, considering that even the same individual may learn or engage differently at different times of the day, based on the location or context of mobile app use [114] or other aspects (e.g., available time, required effort, intrinsic/extrinsic motivation, technical constraints, etc.) [113] that may influence their engagement.

The data analysis also revealed that the only statement to which some respondents strongly disagreed (although a small percentage) relates to the absence of noticeable inconsistencies while using the app. Specifically, three participants strongly disagreed, and an equal number disagreed with this statement. The thematic analysis of the qualitative data captured through the open-ended questions also provided some areas where minor inconsistencies were noticed. Although this is a small percentage, this constructive feedback helps identify areas of concern from the users’ perspective. Again, no pattern was revealed between this response and any demographic group. Moreover, among all study participants, only two participants rated both features (i.e., flexibility and consistency) negatively.

Consistency is a key design principle that was also brought up by the experts. Three out of the four experts touched upon the internal and external consistency design principle, highlighting that although internal consistency was high (i.e., consistency among the pages and parts of the app), external consistency could be improved in relation to the menu and icons. One illustrative example was the reference to the tick icon appearing on the submit button, which was considered counterintuitive, as this typically shows that the answer has been successfully submitted or that the process is complete, rather than showing that the submit button is ready to be pressed.

Overall, the usability of the PREPARED App was positively rated across all usability aspects (Figure 11, Figure 12, Figure 13 and Figure 14). Specifically, participants agreed or strongly agreed that the app satisfies key usability goals, including memorability (93%), safety (93%), learnability (89%), adherence to privacy (89%) and provision of easy-to-locate privacy policy information (87%), trustworthiness (83%), the clarity and informative nature of error messages (76%), availability of expected functions and capabilities (76%), recoverability (74%), and, finally, provision of help and instructions (72%). These elements were also confirmed by the experts during the cognitive walkthrough sessions.

5.3. Accessibility Assessment

In addition to user experience and usability evaluation, the PREPARED App was assessed against a list of success criteria linked to UI accessibility guidelines. The criteria are pulled from W3C (World Wide Web Consortium) Content Accessibility Guidelines (WCAG) version 2.2, which are commonly employed as a standard for mobile accessibility [81]. The guidelines are organised under four main principles known as POUR: Perceivable (information and UI components must be presentable to users in ways they can perceive), Operable (UI components and navigation must be operable), Understandable (information and the operation of the UI must be understandable), and Robust (content must be robust enough that it can be interpreted by a wide variety of user agents, including assistive technologies).

Under each principle, a number of guidelines are proposed, and each guideline is associated with a set of Success Criteria (SC). Several guidelines apply specifically to mobile apps and can be used to assess the degree to which accessibility is addressed.

Table 1. WCAG compliance overview for the PREPARED App.

Guidelines	Success Criteria (SC)	Status	Comments
1.1. Text Alternatives: Provide text alternatives for any non-text content.	1.1.1. Non-text Content	Implemented	Informative images and iconography include meaningful alternative text. Icon buttons include assistive labels when long pressed. All videos are accompanied by descriptive text, transcripts, or both.
1.2. Time-based Media: Provide alternatives for time-based media.	1.1.2. Captions (Prerecorded)	Implemented	All embedded videos have subtitles that are automatically enabled by default.
1.3. Adaptable: Create content that can be presented in different ways (for example, simpler layout) without losing information or structure.	1.3.1. Info & Relationships	Implemented	Content is accompanied by a textual description, and its structure is preserved through different layouts. Users can adapt video playback speed to meet their preferences. Users can switch between different views (e.g., normal or full screen in videos) without the loss of content.
	1.3.4. Orientation	Partially Implemented	The app’s user interface is locked to Portrait mode to ensure the content’s visibility. For videos, the orientation can be adapted to either portrait or landscape (full screen).
1.4. Distinguishable: Make it easier for users to see and hear content, including separating foreground from background.	1.4.3. Contrast (Minimum)	Implemented	Normal text meets the minimum contrast ratio of 4.5:1, and large-scale text and images have a contrast ratio of at least 3:1, critical for reading content in various lighting conditions (e.g., outdoor use).
	1.4.4. Resize Text	Partially implemented	Text content can be zoomed up to 200% while also respecting the operating system’s font scaling accessibility settings. At maximum scaling, some quiz buttons clip or overlap at maximum (200%) zoom.
2.2. Enough Time: Provide users with enough time to read and use content.	2.2.2. Pause, Stop, Hide	Implemented	The app does not contain any components that automatically start (such as videos), no moving banners or blinking elements, or any other time-based distractions that can interfere with the user’s understanding of the content.
2.5. Input modalities: Make it easier for users to operate functionality through various inputs beyond keyboard.	2.5.5. Target Size (AAA)	Partially implemented	Touch targets for multiple-choice answers are smaller than the recommended 44x44 px size and are too close to each other for people with reduced dexterity or motor impairments. Normal buttons meet the size criteria.
3.1. Readable: Make text content readable and understandable.	3.1.5. Reading Level	Implemented	The text is written at an appropriate educational level. Case study descriptions and content are broken into smaller, more readable sections and paragraphs.
3.2. Predictable: Make pages appear and operate in predictable ways.	3.2.3. Consistent Navigation	Implemented	Navigation components, such as those for going to the previous or next pages, are repeated on multiple screens and are placed at the same locations. Other navigational components are clearly visible and indicated as interactive components through elevation and colouring.
3.3. Input Assistance: Help users avoid and correct mistakes.	3.3.1. Error Identification	Implemented	If an input error is detected, the app identifies the respective field and provides clear feedback to the user in text. When a user leaves a required field empty, the app provides an informative message, rather than a generic error. Specific error messages are included for lack of Internet connectivity and general system issues.
	3.3.2. Labels or Instructions	Implemented	Form fields and other interactive elements contain clear labels or hints, along with instructions in each screen, which are visible before the user provides any input.
4.1. Compatible: Maximize compatibility with current and future user agents, including assistive technologies.	4.1.2. Name, Role, Value	Partially implemented	Interactive UI elements (e.g., form elements, links, buttons, etc.) use native components that correctly broadcast their Role (e.g., “Button”) and Value (e.g., “On/Off”). Primary navigation links have unique Names. However, assistive technologies (e.g., screen readers) are not embedded.

provides an overview of the guidelines and success criteria and whether these are fully, partially, or not implemented by the PREPARED App.

5.4. Strengths, Weaknesses, and Suggestions for Improvement

The third research question aims to capture the most and least enjoyed features of the PREPARED App, as well as general feedback and suggestions for improvement. Constructive and creative ideas for enhancing the educational impact of the app and improving its UI design, learning content, and learning experience and effectiveness were captured both in the user evaluation questionnaire and during the discussion with the experts. The provided answers were coded and grouped under thematic areas. Frequency analysis was performed to illuminate the most prominent areas and provide a more informative discussion (the number in brackets indicates how many responses were thematically classified under the respective thematic category). In cases where a participant’s response covered multiple points, each point was coded as a separate entry, and the respective thematic code was assigned (hence, the numbers resulting from the frequency analysis shown in the tables below are higher than the actual number of participants who responded to each question).

5.4.1. Strengths

The first open-ended question invited participants to identify the most positive aspects of the app. A total of 50 responses were received (out of 54 participants), which is a good indication of users’ willingness to report the aspects they enjoyed the most. The responses varied in terms of length and detail, ranging from some comprising a single word (e.g., “UI”, “Informative”, and “Interactive”) to brief answers (e.g., “Nice UI”, “Easy to use”, and “Well structured”) to more extensive answers commenting on various elements.

The word cloud shown in Figure 15 was created using M365 Copilot with the prompt, “Create a word cloud excluding all words with one, two, or three letters.” This provides a quick indication of frequency analysis and an initial visualisation of prominent keywords based on the participants’ responses. Evidently, “Easy to use”, “Video”, “Informative”, and “Information” emerged as the most popular answers. However, undertaking a comprehensive thematic analysis revealed that, by merging the references to information into a single theme (i.e., informative content, information relevance, useful information, etc.), in fact, this was the most prominent aspect, captured by more than half of the participants in their answers to this open-ended question (58%).

The thematic analysis of the participants’ most enjoyed aspects yielded seven thematic areas, representing the main strengths of the app: informative content, relevance to ethics, educational efficacy (58%); easy to use (26%); multimedia content and variety of learning tasks (24%); UI design and visual appeal (24%); engaging, captivating, and enjoyable (18%); clarity and well-organised structure (16%); and interactive learning elements (14%).

Table 2. Positive aspects of the PREPARED App.

Thematic Categories (Frequency)	Representative Verbatim Quotes from Users
Informative content, relevance to ethics, educational efficacy (n = 29)	“This is a very powerful tool for learning purposes.” “The different characters in the cases help to understand the ethical challenges.” “Very interesting and relevant case studies. The content provides the key insights per case study and really triggers reflection.” “The topics are very important. This app helps to learn more, to think and to rethink more about these topics.” “Excellent topics, very relevant and timely topics.” “Informativeness and relevance of the information to ethics topics.” “Very educational and I like the push about ethics because it’s very important.” “Interesting subjects and modules to learn about.” “Very informative, fast learning.” “Informative short videos.” “Highly informative.” “A lot of stories.” “Learn new skills/improve or refresh your skills.” “Well organized case studies with insightful information on ethical problems and decisions.” “The user interface is easy to use and the app is very informative. I can see this being widely adopted in higher education as a training course.” “New knowledge.”
Ease of use (n = 13)	“Easy to use.” “Simple and easy to use.” “Intuitive to navigate and use.”
Multimedia content and variety of learning tasks (n = 12)	“Very interesting activities and stories, which have high-quality graphics and videos.” “Engaging multimedia content, like videos and interactive elements.” “The content is interesting and there is a good mix of different types of multimedia.” “Easy to access and has very important information and videos to watch and learn.” “Graphics and videos.” “Option to read the text in several places.” “The sorting buckets.”
UI design and visual appeal (n = 12)	“Nice UI.” “Easy to navigate.” “Clean design as an app.” “Clarity in design.” “The videos and the progress bar at the bottom of every section.” “Nice appearance.”
Engaging, captivating, and enjoyable (n = 9)	“Captivating case studies with a variety of interactive components and storytelling.” “Useful content and information which invites reflecting on ethical aspects; important knowledge is provided; useful external sources through hyperlinks allow further exploration and extend the value of the app.” “Interesting subjects.” “Entertaining as well as educational.”
Clarity and well-organised structure (n = 8)	“Clear information.” “To the point.” “Well-structured.” “Simple structure.” “Step-by-step information.” “Very well organized activities appearance, and also interesting case studies.”
Interactive learning elements (n = 7)	“I loved the fact that you see what each user voted (in the polls).” “Interesting videos and interactive elements.” “Engaging, informative, interactive.”

presents the thematic categories and the respective frequencies, along with a representative sample of verbatim quotes selected among participants’ responses.

Participants from diverse academic backgrounds, age groups and degrees of familiarity with educational mobile apps and UI/UX evaluation agreed on the informative nature and educational efficacy of the PREPARED App, as indicated by their responses. Strong positive statements were provided by participants to express their satisfaction with the app, including “Powerful”, “Excellent”, “Captivating”, “Engaging”, “Highly informative”, “Very interesting”, and “High quality”, amongst others.

Instructional design and UI design formed an inherent part of the mobile application development lifecycle. Equally, learner engagement and LXD were prioritised during the instructional design, UI design and development, and testing and evaluation phases. These aspects will continue to drive further research into the aspects triggering learners’ interest and engagement with ethics education through a mobile app.

The analysis of the gathered data is aligned with the insights drawn from the studied literature. Firstly, the findings demonstrate how high-quality instructional design can lead to informative, well-designed and relevant learning content that can increase learning effectiveness and reduce cognitive load [39,41,47,54,55,56]. Secondly, perceived usability and UI design can impact users’ first impressions, which, in turn, affect the overall user experience [72]. Thirdly, ease of use, convenience, and flexibility play a positive role in promoting a seamless and autonomous learning experience [46,49,51].

Fourthly, the combination of multiple types of learning activities in the app (e.g., bucket sorting, polls, quizzes, etc.) and multimedia content (e.g., videos, text, images, etc.) was closely linked with increased learner engagement. In the participants’ responses, the engaging UI design of the app was discussed alongside its capacity to enable reflection, thinking and rethinking. This confirms the ability of educational mobile apps to provide a dynamic learning experience [43] and increase interest and motivation [13,45,46,48,49,50,51].

5.4.2. Weaknesses

The respondents were invited to describe any weaknesses or negative elements they identified while using the app. A total of 41 participants provided a response, while the remaining 13 participants included comments like “None” or “No negative aspects”. Some of these responses were brief, including a single word or short phrase (e.g., “Design” or “Layout” or “Small number of case studies”), while others were more comprehensive and raised multiple issues.

The word cloud shown in Figure 16 was created using M365 Copilot with the same prompt as before, but using the set of textual responses received for this question as input. This provides an early visualisation of prominent keywords based on the participants’ responses. Clearly, “Video” received the most mentions when participants were asked about the negative aspects they identified. Comparing this with the earlier word cloud, an appealing observation is that videos were listed both as the most negative aspect and one of the most positive aspects of the PREPARED App. This highlights the importance that users attach to visual content and/or their familiarity with video content, as well as the role of such content in mobile learning. This is aligned with the findings discussed earlier, showing that one-third of participants selected videos as their first choice among all features (Figure 6).

The participants’ responses were categorised under three thematic areas: learning experience, including educational content and human factors (

Table 3. Negative aspects concerning the learning experience.

Thematic Codes (Frequency Count)	Representative Verbatim Quotes from Users
Video quality (n = 14)	“Speech in videos is too slow” “The videos are very slow” “Pace/speed of narration in videos is too slow which impacts engagement negatively” “Video quality and video playback issues” “Poor audio quality in videos” “Many videos cannot load properly” “Long videos without interactivity are boring”
Lack of flexibility and customisation (n = 7)	“Forced to watch videos in order to proceed to the next page” “Unable to skip content” “Forbidden skipping” “Cannot resubmit questions or polls again when revisiting content” “Lack of customisation options” “My only issue as someone who has returned to the app several times to look at different courses and case studies is that it can be frustrating to have to go through a whole case again when I might just want to check one screen”
Learning content quantity and quality (n = 7)	“Small number of case studies” “Limited references to external sources” “The case study topics appear ad hoc with no central theme or clear vision”
Text-heavy content (n = 6)	“Too much text to read” “A lot of information”
Feedback, help, and hints (n = 4)	“In some quizzes the correct answers are not shown before proceeding” “Lack of hints while doing some interactive activities” “Could provide better help messages” “Unclear where the chosen nickname is used and how”
Language barrier (n = 3)	“Content only available in English” “No translation options or subtitles in my native language”

); UI design aspects (

Table 4. Negative aspects concerning UI design.

Thematic Codes (Frequency Count)	Representative Verbatim Quotes from Users
Visual appeal (colours, layout, graphics, icons) (n = 9)	“The images selected for the case studies are not always very engaging (e.g., there is a white human figure on a black background but it does not relate to the topic). Using more relevant, attractive, impactful graphics would be better.”
	“When selecting an option in the quiz a tick (check icon) appears on the submit button which is confusing. Initially I thought it had already registered the answer.”
	“The icon with the questionnaire mark on MCQs is misleading.”
	“Design, colours, icons are not distinguishable.”
	“Colours.”
	“Layout.”
Navigability (n = 3)	“Some bugs when scrolling in main menu.”
	“Scrolling at the end turns the page.”
	“You cannot skip through pages, for example if you accidentally go back to the previous page you will have to repeat the step if you want to move on.”

); and technical issues (

Table 5. Technical issues.

Thematic Codes (Frequency Count)	Representative Verbatim Quotes from Users
Internet connectivity (n = 7)	“I’m not sure if it’s the app or not, but on the quizzes it kept disconnecting. I checked my WiFi and it’s not the problem.”
	“Internet connection is lost even though the app is connected.”
	“The network error shows up even with stable Internet connection.”
	“The video loading would show a pop up that I had no Internet connection.”
	“An error appearing for ‘No Connectivity’ especially when playing videos.”
Interactivity and responsiveness (n = 5)	“Some interactive elements may lag or not function smoothly.”
	“Loading for too long.”
	“Many of the videos aren’t working properly—they just don’t load.”
	“I couldn’t watch all the videos due to errors.”
	“During video playing the mobile was going to sleep mode.”

). The tables demonstrate the thematic groupings, along with a representative sample of verbatim quotes selected among participants’ responses. The respective frequency of each theme is shown in the first column of each table.

Under the first thematic category addressing learning experience aspects, approximately one-third of the respondents (34%) commented on some aspect associated with the videos embedded in the case studies. This is also illustrated by the word cloud above. The most prominent comment relates to the slow pace of narration in the videos. Even though there is an easily accessible feature embedded in the app for adjusting the playback speed of each video (the option is accessible at the top right corner of the video frame), this was potentially missed or undiscovered by some users. Surprisingly, this comment was also received from users who are experts in the use of mobile educational apps. Customisation options are crucial, and they should be easily discoverable to allow users to enjoy a personalised learning experience.

Some participants (17%) also raised issues related to learning content quantity and quality, such as the number of case studies available and the lack of a clear theme connecting the case studies. This perspective is important from a learner experience point of view, as it cannot be assumed that learners will access any related external resources to which they may be directed for further reading. Another relevant perspective is that when using the app, most participants were not familiar with the project or the importance of ethics education. The application’s ‘Help’ and ‘Information’ pages provide some high-level information, but these could be enhanced to ensure the educational purpose, learning aims and objectives, and expected learning outcomes are clearly communicated to all users. This will help support learners who may prefer a self-contained, stand-alone mobile educational app that incorporates rich educational content and useful features.

An equal number of participants (17%) also pointed out issues related to flexibility and customisation options when browsing the case studies, including the ability to skip steps or multimedia content without restrictions, revisit content, and re-answer quizzes and polls. Some comments used emotionally triggered expressions such as “forced”, “forbidden”, and “frustration”. Although these were expressed by very few participants, they still demonstrate how the lack of customisation and flexibility may be perceived as a negative aspect hindering learner engagement and the learning experience with a mobile educational app. These affective aspects are aligned with the findings reported in the studied literature. Other negative aspects captured by the participants, albeit with less prominence, include text-heavy content (15%), the need for more explicit feedback, help, and hints while using the app (10%), and the language barrier (not having the option to choose the content in one’s native language) (7%).

The second thematic category captured negative aspects concerning UI design, including visual appeal (colours, layout, graphics, and icons) (22%) and navigability while progressing through case study content and application pages (7%).

Finally, the third thematic category, shown in Table 5, captures some technical issues identified by users. The most prominent relates to a message that is displayed mainly when users select to play a video. The error message reads “No Internet. Please make sure you have an Internet connection” and includes a “Retry” button. Although this instantly restores the flow of the app, it can still hinder learner engagement or create frustration if it persists.

5.4.3. Suggestions from Users

A total of 25 responses were provided to the concluding open-ended question, asking participants to share any suggestions they have for improving the usability of the PREPARED App or propose new features they would like to be added. Although fewer responses were received compared to the previous two open-ended questions, many respondents provided more extensive and comprehensive answers (up to 138 words in length) to share their recommendations. As expected, most participants reframed the negative aspects highlighted in the previous question into suggestions for improvement. Additional recommendations and creative ideas were also provided.

While frequency analysis provided useful insights when analysing the positive and negative aspects, this method was not applicable when analysing the participants’ suggestions for improvement, since their answers were largely unique in this question. Therefore, the gathered recommendations are thematically grouped into three broad categories: learning experience and human factors; usability and UI design; and multimedia learning content. These thematic categories are not mutually exclusive. In the participants’ responses, UI design features appeared to be correlated with the overall learning experience, highlighting the impact that a well-designed UI can have on learning and engagement. Equally, human factors like learning styles and preferences play a central role in the successful adoption of the app and in sustaining its use in the long term.

Usability and UI design (

Table 6. Suggestions on usability and UI design.

Thematic Category	Representative Verbatim Quotes from Users	Analysis and Interpretation
Customisation options	“Dark theme.” “Offer a wider range of customisation options and perhaps offer a simple tutorial when the first poll/quiz/sorting activities take place to guide the user.”	Customisation, personalisation, and flexibility are inextricably associated with technology acceptance, adoption, and learner engagement.
Icons, buttons, navigation, and interaction modes	“The UI should be more engaging/flexible. The icons and placement of information can be a bit misleading, but a quick rearrangement will fix it!” “The submit button on the quizzes area is misleading users if they have already submitted or not.” “During a course to be able to swipe left or right, instead of only using the buttons.” “I’m not sure how the app performs on mobile phones, as I don’t have an Android device. I tested it on my tablet in landscape mode, but the app remained locked in portrait. I checked the auto-rotate settings, but it still didn’t switch. Only the videos rotate when viewed in full screen.”	Three ideas can be extracted from these statements. Firstly, technical issues or deficiencies that may cause negative emotions, such as frustration or confusion, should be eliminated, as they hinder learner engagement. Secondly, the absence of expected functionality, interactive modes, or shortcuts (typically available in other mobile apps) can influence the app’s intuitiveness. Thirdly, the use of icons and menus should be efficient, effective, memorable, learnable, and have reasonable utility, in line with key usability principles [75]. If a user needs to click on an icon to remember what information or page it links to, this violates the principle of “recognition rather than recall” [115].
Accessibility	“In terms of accessibility, I would recommend adding subtitles to the videos. Many users might access the app in noisy environments or while on public transport. Some pages contain only a few lines of text followed by large blank white space—this may be due to viewing it on a tablet. However, it’s worth considering that many institutions in higher education use tablets in classroom sessions, so optimizing the layout for tablet use could be beneficial.”	Accessibility is key—both in terms of technology compliance and compliance with accessibility guidelines (e.g., WCAG). Designing inclusive and accessible learner experiences involves offering alternative modes of interaction and adjustable settings to allow diverse users, including those with disabilities, to be able to use the mobile app effectively and efficiently. Cross-platform development was employed for the PREPARED App to ensure wider adoption and access to this technological resource. Still, the increasing variability of device sizes, specifications, and user contexts must be considered to ensure the app is inclusive and accessible.

): The majority of the responses revolve around UI design and usability-related features, including the need to provide better or more informative help messages, making the UI more engaging as well as flexible, and carefully considering the use of icons and menus.

Learning experience and human factors (

Table 7. Suggestions related to learning experience and human factors.

Thematic Category	Representative Verbatim Quotes from Users	Findings
Explicit learning objectives and guidelines	“Add an Introduction paragraph to explain what the app includes.” “It would help to add specific learning goals.” “Provide a few guidelines like user info/guidelines to highlight the key benefits of the app and key functionality.”	These add-on features can help manage learners’ expectations and boost their confidence during first use, while also sustaining their attention and reducing the risk of losing interest.
Inclusive and accessible learning	“It would help to add subtitles in order to make the videos accessible to those with hearing problems.” “Add a feature to allow in-app note-taking, or to allow me to favourite a screen/page I see so I can revisit it, or to be able to get a snapshot of what I read, rather than going through the whole case again.” “Offer a text alternative to the audio/videos.”	Accessibility, equality, diversity, and inclusion are of paramount importance in learning contexts [39]. Attending to diverse learning styles and human factors supports the vision of inclusive education.
Feedback and features	“Have a separate section for quizzes, independent from the case studies, to test general knowledge or current understanding. Another feature is to add a screen which shows the percentage of completing each case study to encourage users to continue and complete it. Another idea is to have daily informative notifications or daily challenges e.g., quizzes to engage the users.” “Add a progress bar below each case study so that every time I open the app I remember the status and track my progress easier.” “It would be nice to have a dashboard in the app to show which case studies are complete, incomplete, or not started. This way, each completed case study could be assigned a badge showing on the dashboard.” “There should be a notification system or reminder options, which could be customised to encourage users and keep them engaged.” “It would be nice to see the titles of the case studies in the menu so I can choose which one to view without scrolling all the way down on the main page every time. A progress bar could also be shown in this menu to encourage me to complete the remaining case studies.” “To be able to share this app to increase visibility or to suggest it to others.”	These suggestions show how enhanced interaction features embedded in the mobile app can encourage and engage users. They also highlight the social nature of mobile apps, even in educational contexts. Promoting the badge/certificate feature and incorporating additional gamification mechanics can strengthen the role of social computing and social media [58]. This aligns with trends in professional networks, where users share certifications or challenge results (e.g., Wordle). Embracing similar features can increase interest and influence the overall learning experience.

): A set of suggestions was received in relation to the learning experience, including hints to help learners when learning through the interactive activities. Hints are a typical gamification feature applied to educational games and MGBL, alongside other game mechanics such as points, rewards, leaderboards, and badges. This indicates there is space for further exploring gamification for an improved learning experience and sustained engagement [42,44,52,58,59,60].

Multimedia learning content (

Table 8. Suggestions on multimedia learning content.

Thematic Category	Representative Verbatim Quotes from Users	Findings
Video & graphical content	“Video controls could be improved to add flexibility.” “Fast paced videos.” “Shorter and more engaging videos.” “Offer the option to watch the entire content instead of pieces.” “Play video highlights (like short videos) as an intro instead of showing static video.” “It should be clear/visible how to change the pace of the video. I discovered it much later accidentally.” “Using more relevant, attractive, impactful graphics would be better and more engaging.”	Visual content can strongly influence the effectiveness of an educational mobile app. Graphical elements must be carefully selected. Adding video previews, breaking videos into smaller parts, and adding headings or tags can support microlearning. Making playback-speed controls more obvious can also enhance engagement.
Personalisation and multimodal interactions	“Text under videos is useful but could be better formatted or structured to highlight keywords.” “Personalisation options on whether the text appears plain or formatted.” “It would be better to have the option to see the content in other languages or have choice of subtitles in other languages.” “Add a podcast option, to make the app allow the videos to play in the form of podcasts, and the videos to start one after the other. Also, give the option to change languages.”	These recommendations demonstrate that multimedia and multimodal interactions resonate with users, offering flexible ways to consume the same content. For an inclusive and personalised learning experience, it is important to allow users to access content in their preferred modality.
Content quality	“Provide more content on ethics and wellness.” “It would be nice if there was a way to propose/suggest/contribute new case studies or new topics to be considered by the developing team. Or, some suggested topics which users can vote (as upcoming case studies for example).” “Add something to make the app not a ’single use’ app, to have continuity, maybe something to revisit the quizzes or notification if new case studies or info is added.” “For people that want to know more trusted links on more thorough info or research can be given.” “Additional reference to external resources for expanded credibility.” “Need more validation. Need to be Multilingual. Good to have Progress Tracker. More quizzes. AI feedback is good to have. AR or 360° video for more immersive scenarios is good to have.”	Learners value credible, validated, and high–quality content presented in multiple formats. They also seek more interaction with the material, including content in additional languages, podcast modes, AI-supported feedback, more case studies and quizzes, and access to immersive or alternative formats.

): The use of multimedia, combining videos, text, and images with diverse learning activities, was well-received and highlighted by users. Their suggestions focused primarily on improving the flexibility and enhancing the customisation of these resources and highlighted “mobile microlearning” trends. Flexibility is commonly associated with learner engagement and improved learning outcomes; hence, it needs to be carefully attended to when designing video-based learning content for mobile devices [67,116,117].

The suggested functionalities would allow users to better control the content and access the most relevant content in their preferred mode at any given time. This would give more flexibility, both during the first time they complete the case study, as well as when revising the content as a recap. The findings are both positive and important, as they illuminate the potential of the PREPARED App as a useful and dynamic educational tool rather than a one-off educational resource that becomes obsolete after consuming the learning material. As indicated by one of the academics who evaluated the app, “The user interface is easy to use and the app is very informative. I can see this being widely adopted in higher education as a training course.”

5.4.4. Statistical Analysis

In addition to the descriptive statistics, statistical analysis was performed using SPSS (Version 31) based on the users’ responses to the questionnaire (see Supplementary Materials). Demographics (Q.2–8) were compared with categorical variables (Q9, Q12, and Q14) using the chi-square test. However, for these tests, the sample size was not sufficient for extracting significant conclusions. Specifically, for all tests, more than 20% of the cells had expected counts of less than 5. Even on the Q5 vs. Q9 test, where 25% of the cells have an expected frequency of less than 5, the p-value is high (0.741), and we get a statistically insignificant clue for a homogeneous sample. No composite scales were formed for this study. Each Likert-type item was analysed as an individual variable.

The Mann–Whitney test was used to compare the replies of Higher Education students and the rest of the respondents to questions 13, 15, 16, 17, and 18. Statistically significant answers (with significance level 0.05) were given on items 13.1 (’The PREPARED app is useful’), 13.2 (’ I have acquired new knowledge about ethics through the PREPARED app’), 13.5 (’Using the PREPARED app increased my awareness about ethics and ethical decision-making’), 16.2 (’I feel comfortable using the app’), and 16.4 (’Overall, I am satisfied with the PREPARED mobile app’). The Mann–Whitney test can be considered reliable, since both groups contain more than 5 data items. The effect size calculated using the formula r = Z/7.35 (where Z is the Mann–Whitney statistic and 7.35 is the square root of the sample size) is around 0.30 for these 5 cases, which is considered a medium effect. Kruskal–Wallis comparisons for the rest of the demographics were avoided since we have groups smaller than 5 in all other demographics.

5.4.5. Insights from Experts

The cognitive walkthrough sessions with UI/UX experts provided complementary, rich insights into the usability and user experience aspects of the PREPARED App. Many of their suggestions were already captured by the users, demonstrating a convergence in their overall satisfaction and experience with the app. The experts’ insights helped to confirm and validate the importance of key factors such as the following:

UI design: “I would add a welcome screen or greeting page, and a main menu making it easier to navigate to key content or functionality”; “The icons on the top right are unclear, you cannot tell just by looking at them what information will pop up when you click on them. Although trial-and-error would work here, UI design can be more intuitive by choosing more distinguishable and recognisable icons. Similarly, the check icon appearing on the submit button is misleading because it acts as feedback that I have registered my answer, while the app requires an additional click to proceed”; “Some design improvements could include using higher contrast in the colours used in the quizzes, polls, and dialogues. Also, use different colours for polls and quizzes to allow the user to easily distinguish between different activities”.

Flexibility and accessibility: “Allow users to use their preferred interaction modes with the mobile UI”; “Accessibility means understanding variability across devices in terms of models and sizes as well as different operating systems”.

Trends in social computing and social media: “Some features can leverage evolving trends like podcasts, generative AI, and sharing updates, badges, and certificates in professional social networks.”

Personalisation and customisation features: “Having a nickname assigned to the user is a cool feature to have, it can make learning more personal, it helps users connect with this experience. But to have an impact it should be embedded in the learning journey, it should be used to personalise greeting messages or congratulating users after successfully achieving a milestone, or showing their name in a leaderboard based on their learning performance since this is essentially an educational app”; “I would like to see additional personalisation features and customisation options regarding the difficulty level to address diverse users groups, quiz length to adjust to available time or learning on the go, option to set a timer on/off to make it more intriguing, adjusting frequency of notification settings, add competitive or social features, etc.”

Progress tracking and feedback: “Human–Computer Interaction is mediated by feedback. Designing effective feedback mechanisms is crucial for a usable mobile app”; “There is a progress bar at the bottom which indicates the progress, but this is very subtle even for the expert eye. This will go unnoticed by most users, especially novice users.”

Multimedia content and alternative representations of learning material: “Add a feature which allows users who completed the mini courses or case studies to revisit certain parts without having to sequentially visit all pages again, like adding a virtual bookmark. Similarly, for those considering a new case study allow the option to browse through or offer a ‘trailer video’ or short summary of learning outcomes”; “There is a Help and Information page. These could be embedded in the main menu, and the content could be enhanced to highlight the need, the mission and vision, the learning outcomes of the app.”

These suggestions are aligned with current trends and transformations in education. For instance, embedding podcasts in teaching and learning practices is an evolving, innovative pedagogy, associated with many advantages, including flexibility, control over how and when the podcast is listened to by the learners, inclusivity, and the delivery of up-to-date content [118]. The characteristics of this pedagogy are aligned with the nature of the PREPARED App, which involves critical reflection, self-paced learning, and flexible content delivery.

The experts also captured additional, subtle aspects of UI design, such as internal and external consistency: “Although the first few case studies have a consistent graphical design and follow a particular style, there are some that break this consistency, like the black-and-white image. The overall consistency in terms of graphical components is important. External consistency is equally important. If icons are similar to each other or resemble icons commonly used in other contexts this can be misleading. For instance, the question mark symbol ‘?’ is traditionally associated with ‘Help’ functionality. Hence using it as a decorative element or assigning a different function to it can be misleading and confusing for users—even proficient ones. Some of them will click on it, but nothing happens so this can be confusing.”

Another aspect that surfaced prominently during the cognitive walkthrough sessions was the need to leverage gamification features and mGBL. Although some gamification aspects were also mentioned by users (e.g., hints and badges), experts explained that these aspects need to be more explicitly embedded in the learning activities: “Everything should count towards winning a place at the top of the leaderboard! From percentage of case study progression to scores in quizzes to active engagement time.” The concept of ‘engagement by design’ [52] is crucial in this context, as it captures the necessity to strategically consider which game mechanics, mGBL aspects, and human factors affect learner engagement. These aspects should be embedded early in the design phase, alongside the technological and pedagogical considerations.

When discussing personalisation and customisation features, all experts projected the role that GenAI will play in the future development of mobile learning. AI-driven visualisations and summarisation features can help produce personalised content based on each learner’s profile, (dis)abilities, and learning styles. Embedded mobile AI assistants can boost user satisfaction [119] and offer authentic learning experiences for learners. The discussion with one of the experts also led to the recognition of a key reciprocal relationship: “On one hand, a mobile educational app like PREPARED (focusing on ethics education) can educate learners about the ethical use of AI in research, education and beyond, and how to recognise deepfakes, untrustworthy sources, mis/disinformation, and, on the other hand, leveraging AI as an embedded component in the app can enhance the learning experience through scaffolding, argumentation, and personalised content generation” (Source: author’s field notes). This displays the appropriateness of leveraging GenAI for learning about ethics in science and developing critical thinking and integrity problem-solving skills, which are aligned with the purpose of the PREPARED App. Nevertheless, issues of trust, integrity, and accountability need to be carefully considered in ethical decision-making processes. This becomes increasingly relevant with the adoption of AI for ethical decisions [26,27].

Overall, the positive learning outcomes associated with mobile learning make mobile apps an appealing medium for raising learners’ awareness about topics requiring deep engagement and reflection, such as ethics literacy and integrity decision-making in the context of global crises [91]. Moreover, they can help neutralise the increasing mis/disinformation happening through social media channels, allowing users to stay informed about the rapid pace of global crises, be educated about the impacts it can have on people’s lives, and be steered to view global crises through an ethical lens. Furthermore, the strengths of mobile apps can be easily combined with advanced technologies such as GenAI for an enhanced learning experience. Learning in conversation with AI agents is rapidly expanding as a dialogic, real-time pedagogical method providing learners with a diverse range of interactive experiences [11]. The capabilities of GenAI, including role-playing, dialogue simulation, and real-time interaction, provide a unique opportunity for learners to engage with ethical decision-making, reflect on ethical dilemmas, and practice their argumentation, negotiation, critical thinking, and problem-solving skills. As argued in the latest ‘Innovative Pedagogy 2024’ report, “GenAI can also be helpful in taking on the role of an intellectual sparring partner, particularly in fields requiring robust argumentation like law, philosophy, and ethics. By challenging learners to defend their viewpoints, identify logical fallacies, and counter opposing arguments, GenAI can help them both improve a specific argument to be made and sharpen their critical thinking and debate skills” [11] p. 25. The same trends that are transforming education can be utilised to enhance learning and raise awareness about these transformations. Mobile apps provide an effective means to achieve this goal.

Finally, during the discussions with the experts, it became obvious that “to collectively address all these aspects there is a genuine need for taking a ‘systemic’ approach to developing educational technology. This approach entails formulating multidisciplinary teams bringing together instructional designers and domain experts, software engineers/mobile application developers, UI/UX experts, and GenAI experts” (Source: author’s field notes). This combination of knowledge, skills, and interdisciplinary expertise is crucial for crafting impactful, personalised, authentic learning experiences.

5.5. Study Limitations and Suggestions for Future Research

The use of a dual approach, gathering insights from users as well as experts, provides a good basis for the usability and user experience evaluation and strengthens the validity of the study findings. At the same time, we acknowledge some limitations in our study.

Sample representativeness. While we aimed for demographic variability, the sample size was geographically skewed and relatively small (n = 54) to allow for the generalisability of the results. To address this gap, there is a need for conducting more systematic quantitative studies with a larger or more representative sample size, reporting more detailed statistical analysis and confidence intervals for key quantitative results.

Questionnaire validation. The emphasis of the user questionnaire employed in this study was to gather and analyse users’ perspectives and learning experience with the PREPARED App. The questionnaire invited users to rate the educational effectiveness and usefulness of the app, the aspects which they find more engaging, its ease of use, usability and overall quality. The questionnaire is provided in Supplementary Materials to allow for the reproducibility of the study results and enable subsequent studies to validate and verify the study findings. Additional studies are also needed, employing more comprehensive research designs or deployed in diverse contexts in order to validate the results. Alternative research designs are also worth exploring, including a pre-post experimental study measuring ethical reasoning gains with validated instruments (e.g., a Defining Issues Test or a bespoke ethics decision-making rubric) and retention over time (e.g., after 3–6 months or a year).

Cognitive walkthrough documentation and thematic analysis. During the cognitive walkthrough with the experts, a simple protocol was employed, including a checklist of initial steps to perform, in order to ensure consistency across evaluation sessions. However, the cognitive walkthrough sessions were not recorded or fully transcribed, which may have led to potential information loss. While field notes were used, this introduces a risk of missing nuances or expert rationale compared to full transcriptions or recordings. To mitigate this, written feedback and annotated screenshots which evaluators took during the evaluation session were also gathered and analysed to capture certain UID elements and usability aspects requiring attention. The screenshots captured experts’ views and complemented the researchers’ field notes. Still, it may be possible that some subtle aspects were not captured for subsequent analysis. Additionally, by nature, self-reported data can be unreliable due to inherent biases (e.g., during the session, an expert may have under/over-reported something or not accurately expressed their opinion). Nevertheless, the findings from the analysis of the expert usability evaluation yielded a wide coverage of topics, as well as overlapping themes. The number of evaluators involved (n = 4) is aligned with industry standards, especially with application evaluations of this scale. Furthermore, although all researchers were involved in subsequent analysis and intra-team validity checks were performed, initial coding was conducted by one of the authors. Future research will aim to triangulate the results with more systematically recorded expert notes or conduct independent coding among multiple researchers to avoid single-coder limitations. Another direction for future research is to conduct interdisciplinary evaluations involving experts from diverse fields in the evaluation process, including educational psychologists, ethics experts, instructional designers, and UI designers, to gather interdisciplinary perspectives.

Research design. Although there was a recommendation for the amount of time users should have spent on the app regarding exploration before switching to learning mode to complete the case studies, there is no way of knowing how much time each participant engaged with the app before submitting the evaluation questionnaire. While the insights and suggestions gathered indicate a sufficient degree of involvement—indicated by the rich responses to the open-ended questions—future research should also assess the long-term learning outcomes and explore learning retention. Therefore, longitudinal studies or experimental tests are recommended in future research endeavours, where the duration and engagement of the learners can be more accurately measured, and engagement metrics (e.g., time on task and feature usage logs) can be precisely tracked to correlate usage patterns with reported satisfaction and learning outcomes. A related insight was that skills development may require more effort compared to acquiring new factual knowledge or raising awareness. Therefore, longitudinal empirical research is needed to better understand and measure skills development in the context of ethics education.

Accessibility and assistive technologies. Accessibility was captured as an important aspect by both users and experts. In line with WCAG [81], during the app’s design and evaluation, the goal was to ensure that distractions are eliminated and that the app’s UI does not interfere with learners’ ability to process the information or respond to the quiz questions. Nevertheless, some accessibility success criteria were not fully implemented. Future research and development are needed to fully comply with accessibility guidelines. Another area for furthering the research is to explore how assistive technologies can be embedded to enhance UI accessibility and the overall user experience with educational mobile apps.

Suggestions for employing emerging technologies for ethics education. In addition to the suggestions discussed above, the insights extracted from the critical review of the studied literature combined with the findings from the present study, illuminate the vital role that mobile educational apps, in particular, and educational technology, in general (including serious games, GenAI-enabled online learning platforms, and educational immersive worlds in the Metaverse), can play in cultivating the skills and competencies required to foster ethical literacy. Mobile educational apps can complement all levels of formal education, training programmes and lifelong learning initiatives [43,51], helping learners develop the necessary competencies to combat current and future crises. Equally, AI systems are increasingly adopted by society, governments, and organisations in various decision-making contexts [20]. The same technologies used for decision-making support in real life, including mobile apps [40] and AI tools [20], can also be utilised for learning and ethical skills development, creating a dynamic, reciprocal relationship between technology and ethics education. Future research can explore the capabilities of such innovative pedagogies for fostering ethics literacy.

6. Conclusions

While the popularity of mobile learning is on the rise, there are limited educational initiatives embracing mobile technology for fostering ethics literacy, specifically in relation to global crises. The purpose of this study was to address the gaps in ethics-focused mLearning and explore the user experience and usability of the PREPARED App. The app is dedicated to raising learners’ awareness of contemporary topics on ethics and integrity, advancing their decision-making competences, and helping them critically reflect on ethical dilemmas pertinent to science and global crises [14].

Education and awareness-raising initiatives are instrumental for mitigating the impact of ongoing crises and preparing for upcoming global challenges. The usability and user experience evaluation presented in this study suggests that the PREPARED App is a useful educational tool offering well-structured, informative, engaging, and interactive learning content. The findings demonstrate the beneficial role the app can play in promoting integrity and ethics and raising awareness of global challenges. Based on the findings, this is achieved by integrating multimedia content with informative case studies that enable interactive and reflective learning. The scenarios and topics of the case studies were well received by users. Coupled with the inherent strengths of mobile technology, the PREPARED App enables an overall positive learning experience accessible anytime and anywhere.

The use of a dual approach strengthens the validity of the study findings. On the one hand, the users’ suggestions are aligned with the studied literature and offer insights into the human factors affecting their learning experience with the app while emphasising usability, UI design, and multimedia learning content. On the other hand, the experts’ insights gathered during the cognitive walkthrough sessions provided in-depth insights into usability-oriented aspects and helped validate the user evaluation results. Constructive ideas and suggestions for enhancing the app’s features and impact were also captured. These included embracing trends in social computing and social media, enriching personalised feedback and customisation features by leveraging GenAI, enabling progress tracking and leveraging multimedia and gamification features. The findings collectively provide a good understanding of users’ and experts’ experience with the PREPARED App while also revealing appealing areas for further research and development in ethics education and beyond. These insights were combined into the set of ’ETHICS’ pedagogical guidelines. The guidelines were extracted based on the study results and capture the key areas for informing future research and development in ethics-focused mobile learning:

• E: Engaging and Relevant Learning Content: Add informative and relevant content in the form of captivating narratives or stories to activate reflection, and aim to capture and sustain learner engagement. Ensure learning objectives are clear and explicitly stated, and that the learning outcomes are rewarding in order to enhance educational efficacy. Gamification and game mechanics (e.g., badges, certificates, points, leaderboards, etc.) can be embedded to activate engagement.
• T: Tailored Responses and Personalised Feedback: Tailor the responses and adapt the feedback or hints provided to match users current context (e.g., based on the user’s actions or response to a quiz question, previous answers or accumulated user interactions, progression level, or personal traits and user profiles). AI technology can be utilized for achieving personalized interactions.
• H: Human-centred and Multimedia Approach: Embrace multimedia content, multimodal interactions, and a variety of interactive learning tasks to activate the development of diverse skillsets and offer a rounded learning experience. Eliminate text-heavy content and provide alternative representations (e.g., subtitles, visual content, animations, etc.) to enhance user satisfaction.
• I: Inclusive and Accessible Design: Design inclusive learner experiences with accessibility guidelines in mind, offer alternative modes of interaction (e.g., text to speech) and adjustable features to allow diverse users, including those with disabilities or from diverse cultural backgrounds, to be able to use the mobile app seamlessly, effectively, and efficiently.
• C: Customisable and Flexible User Interactions: Support diverse needs and preferences by allowing users to customise mobile app settings (e.g., pace of videos, colour contrast, difficulty level, etc.) based on their preferred interaction modes. Offering flexible learning paths is conducive to mobile earning.
• S: Sustainable and Structured Learning Content: Design the learning content, interactive activities, and user interface by employing a simple structure to enable easy navigation on mobile devices, and ensure content remains relevant, reflecting the sustainability challenges that underpin the fast-paced and constantly changing world.

As the world is constantly disrupted by social and technological changes, this study demonstrates that mobile technology can be leveraged for good, enabling new engaging, personalised, and multimdela learning experiences that can afford inclusive, accessible, user-centred learning in a structured yet customisable way. The study makes a methodological contribution to the current literature by adopting a mixed-methods design involving both users and UI/UX experts. Guided by the research questions, the study’s empirical novelty is demonstrated through its three-fold emphasis on the educational effectiveness, user experience, and usability of the app. The study illuminated the app’s strengths, weaknesses, and suggestions from users and experts. This has shaped the study’s pedagogical contribution, which was culminated through the empirically-informed ’ETHICS’ pedagogical guidelines. The guidelines capture the key study findings, address practical implications, and emphasise the necessity for inclusive learning and alignment with the principles of Quality Education for all (SDG 4) [15]. They can guide instructional designers, educators, and mobile application developers to co-create accessible, inclusive, and engaging mobile educational apps. The presented mobile app and study findings contribute to ongoing educational endeavours to foster ethics literacy and develop the skills and competencies required to mitigate the impact of current and global crises, including navigating ethical dilemmas, reflective thinking, responsible behaviours, ethical decision-making, and integrity.