Online and Blended Learning Courses for Healthcare Professionals and Policymakers on Patients’ Perspectives on Medicine: A Project Report

In order for healthcare professionals to better engage with patients, they need to understand and integrate the perspectives of patients into their daily work. In this project, we developed two courses for healthcare professionals on patients’ perspectives on medicine. One course was an online course that introduced the patients’ perspectives on medicine and explained its importance for healthcare and health policy. The second course was a blended learning course, consisting of online modules and face-to-face webinars, which specified how to explore patients’ perspectives in qualitative interviews, and how to develop implementation plans. Patients participated in the development, execution, and evaluation of both courses. Overall, more than 2000 healthcare professionals enrolled in the first course and, in just over a year, 191 participants completed the online course; 57 healthcare professionals registered in the second blended learning course and six participants completed both components of the course. The relevance of knowledge gained was positively evaluated. Participants especially appreciated the participation of patients. Based on the feedback, the second blended learning course was adapted to run online and both courses continue to be freely available to all interested healthcare professionals on the Coursera platform.


Introduction
For years, there have been many challenges associated with implementing rational use of medications which deserve careful attention [1,2] and now, more than ever, rational use of medications is challenged by the ever-growing use of the Internet and social media [3]. Patients adapt, change, and substitute their treatments often without the involvement of healthcare professionals, based on patient-to-patient interaction and a rapidly increasing amount of information received through the Internet and social media [4]. These situations can, in some cases, cause life-threatening medication errors and waste of resources within healthcare systems [5]. There is also an increasing trend encouraging patients to take more responsibility for their own treatment [6][7][8]. It has been shown that when patients, caregivers, and healthcare professionals jointly decided how to best use medicine, medicine materials; discussion learning activites promoted discussion; and peer-and instructorreviewed assignments covered the practice, investigation, production and collaboration types of learning.
Two academic staff members were assigned to review and scrutinize the finalised content of all the modules to assure a logical flow and appropriate use of terminology. An expert from COBL with a journalist background conducted the final review of the manuscripts for the video lectures, created visual backgrounds, and filmed the lectures. The preparation of the courses lasted from January to September 2020.

Content
An overview of the two courses is presented in Figure 1. The online course (Modules 1-4) was developed as a Massive Open Online Course (MOOC) on the Coursera platform [12]. The blended learning course consisted of: (1) online modules including staff interactions and (2) face-to-face webinars. The blended learning course was provided on the online learning platform Canvas/Absalon (Salt Lake City, UT, USA) at the University of Copenhagen, and participants were required to sign up to get access. The online modules of the blended learning course (Modules 5-8) were held from October to November 2020. The face-to-face webinars were held on three days during one week in November 2020, i.e., Monday, Wednesday, and Friday, from 9 am to 1 pm. The online course (Modules 1-4) introduced how the patients' perspectives on medicine could be understood and explained why it was important to explore and apply patients' perspectives in healthcare practice and in health policy. The online component of the blended learning course (Modules 5-8) specified how qualitative interviews could be used to explore patients' perspectives on medicine, while the face-to-face webinars focused on the implementation of patients' perspectives in healthcare and health policy. A detailed description of the aims, learning objectives, learning materials, and assessments of both courses and their components is provided below. The courses were free of charge.

Aim and Learning Objectives
The first module, entitled "Why is the patient perspective important?", provided an overview of society-wide challenges related to medicine use. The learning objectives of the module were:

•
To explain why it is important to understand patients' perspectives on medicine from a society's point of view; • To provide examples of problems related to medicine use.

Teaching Materials
The teaching materials included video lectures, compulsory readings, a discussion prompt, a quiz, and additional readings. The three video lectures lasted, in total, 18 min, and were entitled: • "Introduction to the course" (1 min); • "Medicine-related problems from professional and political viewpoints" (10 min); • "Why we need to understand patients' perspectives" (7 min).
The compulsory readings were estimated to take, in total, 1 h and 45 min, and dealt with: Adherence problems [2]; • Shared decision making [4].
Then, the participants had an opportunity to discuss the challenges of medicine use experienced by patients in their daily living, in a discussion prompt, as well as to engage with some additional readings detailing medication safety and adherence problems [13][14][15].

Participant Assessment
The module was passed by correctly answering 80% of an eight-question quiz based on the video lectures and compulsory reading materials.

Aims and Learning Objectives
In the second module, entitled "Patient stories", the participants learned directly from patients about their views on medicine and how medications are incorporated into patients' daily living. Furthermore, the module introduced a theoretical model for summarizing different aspects of patients' perspectives on medicine. The learning objectives of the module were:

•
To provide examples of the different factors influencing patients' experiences with medications in daily living; • To explain why patients sometimes self-regulate their medications.

Teaching Materials
The teaching materials included video lectures, compulsory readings, a discussion prompt, a quiz, and additional readings. The three video lectures lasted, in total, 26 min, and included:

•
A lecture presenting and illustrating the Patient Lived Experiences with Medicine (PLEM) model [16] (14 min); • An interview with an older woman with arthritis, in which the practicalities related to the use of medications are discussed, as well as about the "silent knowledge" a patient often possesses and not always discloses (9 min); • An interview with a younger woman with multiple chronic diseases, who described her coping strategies and expectations of healthcare professionals (13 min).
Compulsory reading materials included: • A review article on the PLEM model [16], estimated to be read in 45 min; • Additional videos with patients taking pain medicines [17], estimated to take about 30 min to watch.
Then, the participants had the opportunity to discuss their views on patients' selfregulation of their medications, in a discussion prompt, and to read some additional studies, based on qualitative interviews with patients [18][19][20].

Participant Assessment
The module was passed by correctly answering 80% of a nine-question quiz based on the video lectures and compulsory reading materials.

Aims and Learning Objectives
The third module, entitled "The professional perspective", taught the participants that patients' rationalities regarding medicine use can be, and often are, different from the rationalities expressed by professionals. The module also introduced various ways that healthcare professionals evaluate patients' perspectives on medicine, and enabled participants to reflect upon their own professional assumptions about patients' perspectives on medicine. Specific learning objectives of the module were:

•
To provide examples of how healthcare professionals and policymakers view and detect patients' perspectives on medicine, using questionnaires on (a) satisfaction, (b) patients' attitudes towards medicine, and (c) knowledge of, or behaviours related to, medications; • To reflect on how the assumptions and perceptions of the participants concerning patients' perceptions on medicine are different or similar to those of patients.

Teaching materials
The teaching materials consisted of video lectures, compulsory readings, a discussion prompt, a quiz, and additional readings. The three video lectures lasted, in total, 19 min, and were entitled: • "The professional vs. the patient perspective" (3 min); • "Interactions between patients and professionals" (8 min); • "Evaluation methods used by professionals" (8 min).
Compulsory readings were estimated to take, in total, 1 h and 40 min and included: • A report on different aspects of the concept "patient perspective" provided in the literature, prepared by the academic staff from the University of Copenhagen [21]; • Readings on medication adherence measures [22]; • An example of the medication burden questionnaire [23].
Then, the participants had an opportunity to discuss their own perspectives on medications as compared with those of patients (in a discussion prompt), and to read more about questionnaires used by healthcare professionals to assess different aspects of medical treatment (in additional readings).

Participant Assessment
The module was passed by correctly answering 80% of a nine-question quiz based on video lectures and compulsory reading materials.

Aims and Learning Objectives
The fourth and final module of the online course was entitled "How to apply and explore patients' perspectives". In this module, the participants were shown examples of how patients' perspectives have been implemented into health policy and healthcare practice, and participants were acquainted with different aspects of a qualitative interview as a method that could be used for an in-depth exploration of patients' perspectives. Lastly, the participants reflected on how they could use the knowledge obtained from the course in their professional settings. The learning objectives of this module were:

•
To provide examples of how to apply patients' perspectives on medicine in professional settings; • To provide examples of the methods that can be used for in-depth exploration of a patient's perspective on medicine; • To explain the ethical aspects when exploring patients' perspectives on medicine.

Teaching Materials
The teaching materials included video lectures, compulsory readings, discussion prompts, additional readings, and a written assignment. The four video lectures lasted, in total, 32 min. The contents of the lectures are described below:

•
The first lecture provided examples of how patients' perspectives on medicine can be applied in a healthcare setting (5 min); • The second lecture was an interview with a patient representative who described their experiences of being involved as a patient in different professional settings related to the development and use of medications (16 min); • The third lecture was devoted to qualitative methods and provided an overview of these methods (6 min); • The fourth lecture was also devoted to qualitative methods and presented related ethical principles (5 min).
The compulsory readings in this module consisted of literature on how to conduct qualitative research [24] and was estimated to take 30 min to read.
Then, the participants discussed what they thought about the methods, and what they had learned during the course in two discussion prompts. They could also watch additional videos related to patient involvement, as one of the ways to apply patients' perspectives [25].

Participant Assessment
The participants had to submit a written assignment, describing how patients' perspectives were currently applied in their professional setting, and how this situation might be improved. After submitting their own assignment, the participants had to evaluate the assignments of at least two other participants. The module and the online course were passed if the peer evaluators gave at least eight out of maximum ten points, for the written assignment.

Aim and Learning Objectives
The module entitled "How do we design a qualitative interview study?" started with a presentation of the different stages of a qualitative interview study. Then, insight into the different relevant themes that could be explored via interviews was introduced. Finally, the participants designed a small qualitative interview study to be carried out throughout the remaining modules of the online component of the blended learning course, in which each module covered a separate phase in the process. The learning objectives of this module were:

•
To understand the relevance of the different stages of a qualitative interview study; • To define a research question within the area of patients' perspectives on medicine, which was relevant to a participant's own professional setting; • To design a small qualitative interview study to answer this research question.

Teaching Materials
The teaching materials included video lectures, a quiz, a discussion prompt, and an instructor-reviewed assignment. The five video lectures lasted, in total, 1 h and presented the following:

•
The theoretical basis of a qualitative interview study (i.e., an outline and short presentation of the seven phases of a qualitative interview study); • An interview with a researcher on how and why interview studies can be carried out; • An interview with a pharmacist who described their experience carrying out an interview study in a pharmacy; • An interview with a policymaker about what kind of interview studies with patients they had found relevant from a health policy point of view; • An interview with a patient who explained his reasons for recommending interview studies to be carried out.
After the theoretical part, the participants had to answer at least 80% of a sevenquestion quiz. Then, they discussed their research questions with peers in a discussion prompt and looked at some additional interview studies as additional readings [26,27].

Participant Assessment
The module was passed by submitting the research question to be worked with during the course, along with the design of a relevant interview study for the course instructors to provide feedback.

Aims and Learning Objectives
This module, entitled "Preparing for an interview", aimed at supporting the participants in identifying relevant materials to prepare an interview guide. The module illustrated other central elements to consider when creating the interview guide. In addition, ethical considerations, strategies, and practical advice for recruiting participants were covered. The specific learning objectives of the module were:

•
To create an interview guide; • To explain different recruitment strategies and to select the most suitable strategy for participants' own studies.

Teaching Materials
The teaching materials included video lectures, additional readings, a discussion prompt, compulsory readings, and an instructor-reviewed assignment. The five video lectures lasted, in total, 36 min. The content of the lectures covered: • Preparation for an interview study (13 min); • Construction of the interview guide (7 min); • Informed consent (3 min); • Recruitment (7 min); • Sampling strategies for an interview study (6 min).
Additionally, the participants were provided with a YouTube video which provided tips on how to design a semi-structured interview guide [28]. They could also discuss their interview guides with peers in a discussion prompt. Compulsory reading materials consisted of papers relevant to individual participants' interview studies, selected by the course instructors, estimated to be read in a maximum of 2 h.

Participant Assessment
To pass the module, the participants had to submit their interview guide and informed consent form for the course instructors to provide feedback.

Aims and Learning Objectives
This module was entitled "How to conduct an interview". During the week allocated to this module, the participants had to conduct at least one interview to understand patients' perspectives on medicine, using the guide from the previous module. Additionally, the module supported the participants in developing their skills as interviewers, emphasizing body language, empathy, and active listening. This module also introduced the most common pitfalls in interview situations and provided time for reflections on participants' own interviewing skills. The specific learning objectives of the module were:

•
To explain the do's and don'ts of conducting interviews; • To conduct a qualitative interview; • To discuss the participants' own roles as interviewers.

Teaching Materials
The teaching materials included a video lecture, a quiz, additional readings, a discussion prompt, and an instructor-reviewed assignment. The video lecture lasted 12 min and was devoted to interview preparation, phases, and skills. The lecture was followed by a two-question quiz. Further, the participants were provided with additional literature on interviewing [29,30], and they could discuss their interviewing skills with peers in a discussion prompt.

Participant Assessment
To pass the module, the participants had to conduct at least one interview, with an estimated 2 h for preparation (e.g., arrangement) and execution of this interview.

Aims and Learning Objectives
This module was entitled "Analysing a qualitative interview" and provided participants with an understanding on how to analyse qualitative interview data. The participants learned the most important points to consider when transcribing interviews and they were provided with an overview of different types of commonly used qualitative analyses. Then, the participants practised their skills by transcribing and analysing parts of their own interview. Finally, they reflected on their experiences throughout Modules 5 to 8. The specific learning objectives of this module were: • To transcribe an interview; • To describe approaches when analysing qualitative interview data; • To discuss experiences with interviewing a patient and with analysing and interpreting the interview findings.

Teaching Materials
The teaching materials consisted of two video lectures, quizzes, compulsory readings, a peer-reviewed assignment, and a discussion prompt. The two video lectures lasted, in total, 12 min, and were entitled: • "The transcription process" (5 min); • "Data analysis and coding" (7 min).
The lectures were both augmented with three-and four-question quizzes, respectively. Compulsory reading was estimated to take 1 h and included the analysis part from the qualitative interview guide, prepared by the University of Copenhagen academic staff [31].

Participant Assessment
To pass the module and the online part of the blended learning course, the participants had to submit an assignment in which they transcribed and analysed their own interview data, spending a maximum of 2 h on the job. Further, they had to evaluate the assignments of at least two peers. Finally, to prepare for the face-to-face webinars of the blended learning course, in a discussion prompt, they had to discuss their experiences throughout the course associated with conducting an interview concerning a patient's perspective on medicine.

Aim and Learning Objectives
The overall aim of the face-to-face webinars (which, before the COVID-19 pandemic, were planned to be carried out as physical workshops at the University of Copenhagen) was to teach the participants how to apply patients' perspectives to develop services contributing to better use of medications. The learning objectives of the face-to-face webinars were:

•
To discuss and clarify questions and challenges related to participants' interview studies concerning patients' perspectives on medicine, which were conducted during Modules 5-8; • To become acquainted with implementation strategies; • To develop an implementation plan for how to incorporate the learnings from the interview study conducted during Modules 5-8, into daily practice.

Teaching Content
The teaching content of each day is specified below. Day 1 was allocated to introduce each other's projects (1.5 h) and to discuss the challenges while conducting individual interviews (1.5 h). During the last hour, the subject of implementation and the readings to prepare for the next session were introduced by the course instructors. The readings included: • A paper on Consolidated Framework for Implementation Research (CFIR) [32]; • A paper on Systems Engineering Initiative for Patient Safety (SEIPS) [33].
Day 2 started with the lectures on implementation strategies (1 h and 15 min), where the participants were provided with a template to be used for making an implementation plan. Then, the participants worked individually to develop their individual implementation plans, which were then shared and discussed in groups with peers and course instructors (1 h and 45 min). During the next day free of teaching, the participants had to finalise their individual implementation plans.
Day 3 was the last day of face-to-face webinars and the blended learning course, and it started with group work, where the finalised individual implementation plans were presented, and commented upon by peers, course instructors, and patients (3 h). The last hour of the day, and of the entire blended learning course, was devoted to course evaluation.

Evaluation Aims
The aims of the evaluation of the online course were: (1) to assess the activity levels in the course, (2) to describe the demographics of the participants completing the course, and (3) to assess how the participants evaluated the course in terms of teaching methods and content.

Evaluation Methods
The activity levels were assessed based on the information provided by the Coursera statistics. The demography of participants and their evaluations of the teaching methods and content were based on a questionnaire, including open-ended questions (Appendix A), constructed by the University of Copenhagen academic staff and delivered using the Sur-veyXact online survey system (Aarhus, Denmark) to the participants completing the course.

Evaluation Results
According to Coursera statistics, starting from the end of September 2020 until the middle of December 2021 (i.e., for slightly more than a year), the course page had 10,405 unique visitors, of which 2249 people enrolled into the course. Among the people who enrolled, 1053 were participants who started the course, of which 191 participants completed the course, and 90 of the participants who completed the course responded to at least one question in the evaluation questionnaire. Among the respondents, the majority were (a) women, (b) employed in the health care field, (c) who had worked in their field up to 10 years. The median age of the participants was 30 years old, and they came from four continents around the world, mostly from Europe and Asia (Table 1). The results of the evaluation of the online course were excellent. The relevance of the knowledge provided in the course was rated the highest. Among the course elements, the videos received the best evaluations. When evaluating the individual modules, Module 2 received the best evaluation; almost all participants "completely agreed" that they "got insight into different factors influencing patients' experiences with medicine in daily living" ( Table 2). Table 2. Results of the survey evaluating the online course (Modules 1-4). Accordingly, the answers to the open-ended questions revealed that the participants considered the course to be "extremely interesting", "inspiring", "excellent", "relevant" and "engaging". They especially liked the presentations of the patient stories, and the variety of teaching materials, i.e., the combination of videos, readings, quizzes, discussions, and assignments. They also appreciated the opportunity "to write to the peers" and put their "reflections and reality there". Regarding possible improvements, three main issues were mentioned: a need for a more specific introduction to the course; some replacement of workload from Module 4 to Module 3; and a suggestion for the discussions to be mandatory.

Evaluation Aims
Similar to the aims of the online course evaluation, the aims of the evaluation of the blended learning course were: (1) to assess activity levels in the course; (2) to describe demographic characteristics of the course participants; and (3) to assess how the participants evaluate the course in terms of teaching methods and content.

Evaluation Methods
The activity levels in the course were assessed based on the number of people registering (by e-mail) for the course, and the number of people completing the course assignments using the University of Copenhagen teaching platform. The participants' demographics and their evaluations of the content and teaching methods in the online modules of the blended learning course (Modules 5-8) were based on a survey provided by EIT Health (i.e., the funder) and adapted by the University of Copenhagen academic staff. Similar to the survey used to evaluate Modules 1-4 of the online course, the survey for the Modules 5-8 of the blended learning course included both closed and open-ended questions. The evaluation of the content and teaching methods of the face-to-face webinars of the blended learning course were based on discussions with the course participants during the last hour of the face-to-face webinars. The contents of these discussions were analysed qualitatively.

Evaluation Results
In total, 57 persons registered for the blended learning course, among whom, 19 people logged in to access the materials for the online modules (Modules 5-8). Seven people completed all the assignments here, among whom, six people participated in the face-toface webinars. One person dropped out from the webinars due to personal reasons. Among the six participants, one was male and five were female. Three participants were aged 35-44 years, two were younger, and one was older. Three participants were from Europe, two participants were from Africa, and one participant was from Asia. One participant was a postgraduate student, and the other five participants were pharmacists employed in academia, administration or pharmacies.
Overall, the evaluation of the online modules of the blended learning courses were positive: half of the participants were "extremely satisfied" with the content of the course, two thirds of the participants were "extremely satisfied" with other participants, and all participants were "extremely satisfied" with the level of the course instructors, as well as the usefulness of the program. While answering the open-ended question "What did you like the most?", participants noted teaching approaches (e.g., variability, content of the videos, interaction with the teachers) and usability or practical aspects of the course where patients were involved in the teaching. One of the participants wrote, "Putting the learning into practice really helped me to understand the topic better. Also including real patients was very good".
While answering the open-ended question "What did you like the least?", the participants mentioned work load (i.e., it seemed to be too heavy for a short period of time for participants with presumably full-time jobs) and the administration of the course material (the materials for the forthcoming week were only opened in the beginning of this week and participants would have liked to have an overview of all the materials at once).
One of the participants wrote, "It was not always clear what work was involved for the next week. That made planning for myself in one week challenging (because of holidays). Maybe for the next time posting a schedule for the whole course might have helped." A content analysis of the discussion after the face-to-face webinars revealed four main themes: • General feedback Participants thought that this was an interesting course, with strong content. The course for some participants exceeded their expectations. Participants thought that the level of complexity was right, and the knowledge gained was helpful to understand and apply the patients' perceptions. • Structure Participants liked being introduced to each other, which made the teaching more personal as compared with the online courses. Some thought that the face-to-face webinars could have been placed earlier in the course. • Content Participants appreciated having patients involved giving feedback and adding new insights to their implementation plans. Some participants would have liked to have had more references to scientific materials, including books. • Administration The group size of six students was thought to be very fitting/appropriate. The participants noted that all the administrative questions were quickly solved. A few participants said that they would have liked to have the readings sent by e-mail ahead of the course.

Reflections and Perspectives
We developed, implemented, and evaluated two innovative online (Modules 1-4) and blended learning (Module 5-8 and 3-day face-to-face webinars) courses for healthcare professionals and policymakers, to raise awareness of the importance of patients' perspectives on medicine and to teach qualitative interviews as a method to research and to understand patients' perspectives on medicine. The attendance and evaluations of the online course to introduce patients' perspectives on medicine were excellent. Despite good evaluations, participants' attendance in the blended learning course's teaching interview methods was low. This may be related to the high intensity of the course and the need to be present for the teaching at certain hours of the day during the three-day face-to-face webinars.
We plan to continue working on attracting a broader audience of healthcare professionals and policymakers to the courses, and thereby, create awareness of the importance of patients' perspectives on medicine for healthcare practice and health policy. The decisions on how to reach this goal were taken during academic staff discussions in the final evaluation meeting, taking into consideration the suggestions of the evaluation panel, including external experts in designing educational activities for healthcare professionals and a patient representative.
In order to reach a broader audience for the online course (Modules 1-4), it was decided, firstly, to intensify the marketing of the course, to try to reach the USA, and to make a promotional video [34], and secondly, to use the course for teaching internal and external university students. The latter decision has already been implemented, and the course is used for teaching pharmacy students at the University of Copenhagen and the University of Oslo. In order to reach a broader audience for the online modules of the blended learning course (Modules 5-8), it was decided to transfer the course to a more user-friendly educational platform, i.e., Coursera, after replacing instructor reviews with peer reviews and quizzes, which the University of Copenhagen academic staff took responsibility for. The course is already running on Coursera [35] and it is being marketed with a promotional video [36]. Similar to Modules 1-4, Modules 5-8 are also being used for teaching both our own and external university students (i.e., the videos from the course were recently used in a summer school by the European Drug Utilisation Research Group (EuroDURG)). We decided to not continue providing the face-to-face webinars, due to the very small number of participants and the heavy workload for the instructors.

Conclusions
In the two courses on patients' perspectives on medicine and how these perspectives can be explored in qualitative interviews, healthcare professionals learned to appreciate patients' perspectives on medicine. This awareness is important, so that medications can be tailored to individual patient needs, potentially reducing medication harm and leading to improvements in health outcomes. Currently, the courses are available on the Coursera platform, at no cost, to all interested healthcare professionals, filling a gap in the existing Coursera portfolio of courses for healthcare professionals. The courses have, so far, received positive feedback from participants from several continents.   Free text comments about the course generally: Please write what you think was good and bad, what you would suggest improving _________________ Now, we will ask you about the specific modules of the course.
To refresh your memory, please visit the description of the modules clicking the link below.
The content of the modules can be found under "Syllabus." Please note that the link will open in a separate window. https://coursera.org/learn/patient-perspectives-on-medications Module 1. On a scale from 1 to 5, please assess the if the learning objectives of Module 1 were met.    (1) Be acquainted with how to apply patients' perspectives on medicine in professional settings.
(2) Be acquainted with the methods that can be used for an in-depth exploration of patients' perspectives on medicine.
(3) Be aware of ethical aspects when exploring patients' perspectives on the use medicine.