4.1. First Round of Qualitative Research
The detailed research design presented in the previous section has supported the collection of a significant amount of qualitative data from higher education administrators. In this section, the qualitative analysis of the data collected has a threefold objective:
First, to analyze the basic perceptions of higher education administrators and professors in terms of the complementary value components of quality. The objective is that the integration of the perceived complementary aspects will enlighten a detailed mapping of quality metrics;
Second, to reveal several concerns and limitations as perceived by administrators and professors related to the integration of the quality value components to the design of learning content and academic programs;
Third, to emphasize the understanding of hidden or existing relationships between quality perceptions and performance indicators from different perspectives. Thus, the next methodological step will lead to the identification of several qualitative key performance indicators.
Several value components are revealed, and their interpretation may guide the justification of various initiatives in higher education organizations. Additionally, several quality perceptions of interviewees and the main arguments outlined in their statements have been mentioned repeatedly in the clear majority of responses. At the same time, the opinions of respondents on their perceptions have coincided in all responses.
In a synthesis of their perceptions, respondents’ arguments have been clustered to formulate a set of aspects of perceptions considered critical in integrating quality in the educational process. The key arguments provided refer to teaching qualifications instructors need to hold, together with research activity they demonstrate every academic year. Thus, well-qualified and research-active faculty are able to inform their teaching through research in their field and at the same time assist students in reaching learning outcomes at the course and program level. Additionally, respondents argue that teaching content should be customized to address course learning outcomes and different student learning styles. In this way, students will be motivated to engage in active learning and consequently develop skills in team work, problem-solving, technology and innovation among others. Moreover, most respondents pointed out the importance of the integration and application of theoretical knowledge into addressing real-life problems and situations. This could be achieved through innovative assessments and student engagement with the industry and job market.
It was interesting to observe that respondents with academic administration experience and service in university committees agree that quality can only be maintained through an established quality assurance system, with clear, automated procedures geared toward promoting quality outcomes.
The synthesis of the previous perceptions provides numerous interesting insights. A first interpretation of the commentary aspect for quality perceptions is provided in the proposed model in Figure 2
. A three-dimensional value integration space for quality value components is well defined and is linked with the critical theoretical model that was presented in previously published work. According to Varouchas and Sicilia [18
], the dimensions and the value ingredients of this value space include three dimensions and 20 value components which require further investigation (see Figure 2
4.2. A Second Round of Qualitative Research—Drafting KPIs for Quality Measurement
Having developed the key contribution summarized in Figure 2
above, a subsequent thread of qualitative research strategy is required to elaborate and confirm several quality measurements. For this reason, a second round of focused structured interviews with key stakeholders (academic deans and department heads) was conducted. As mentioned in the introduction, the main purpose of this qualitative research is to understand some metrics or key performance indicators for the main dimensions of the tested model.
Once more, the constant comparison method was used to analyze the qualitative data gathered from 13 interviews with academic administrators and professors in higher education. Following the analysis and codification of a detailed research agenda, a summary of the main findings which include constructive responses for the formulation of constructs and candidate KPIs is presented below.
Construct 1: Time for Preparation of courses/effort invested in design
Question 1:How much effort do you make in the preparation to teach a required course in your discipline? Do you believe that the time you invest in the preparation of content is a key ingredient of quality? Elaborate on this statement.
|Construct #1 Summary of Findings—Key Quotes|
To teach a course effectively, one would need over 3 h of preparation per credit hour per week; in addition to this, a faculty member needs to be constantly informing oneself on developments in their field of expertise, which adds significantly to the minimum preparation time cited above.
I would guess that it would take me between 25 to 35 h to teach a required course in economics or finance. This time differs for the principles classes, which I have taught for decades and which are easier for me to prepare, while upper level classes typically require more time. Yes, I certainly do believe that the time spent is an indicator of quality.
The main finding is that it is evident that time devoted for preparing a course is critical as well as the time devoted to updating material and to engaging students with the learning content and context. One generic KPI which will further be developed in future research is recommended:
Construct 2: Technology-enhanced learning utilization
Question 2:Which are the main technologies you deploy in your classroom? Can you elaborate on the added value contribution of the use of information & communication technologies (ICTs) in your classes? For example, what do you think about the use of educational videos from YouTube? Are there any prerequisite factors for the use of the technology in the classroom?
|Construct #2 Summary of Findings—Key Quotes|
Blackboard tools (i.e., journals, blogs, and discussion forum) contribute to making the class more interactive and facilitate exchanges both between the instructors and among students. They also allow for class discussions to be extended online, facilitate the supervision of projects (work in progress), peer review, coordination of group assignments between students. As far as teaching is concerned, I use PowerPoints in which I frequently embed audio-visuals, slinks to interesting articles or research findings, as well as educational videos. The use of ICT is essential for today’s teaching environment: It can be used to illustrate elaborate ideas or concepts in a student-friendly way; it promotes a more interactive approach to teaching and learning; it can facilitate class discussions; and it is compatible with the habits of the generation of “digital natives” and our culture’s emphasis on the visual.
The main technologies that I use are a blackboard/canvas; excel and video content from sources like Khan Academy, YouTube, TED talks and Merlot. Current media such as CNN, CNBC etc. are also used.
The main finding is that it is evident in the responses that technology-enhanced learning solutions are perceived as key quality drivers in higher education. There are a variety of approaches and technologies available. One generic KPI which will further be developed in future research is recommended:
Construct 3: Academia–industry partnerships
Question 3:To which extent do you use industry project engagements in your classes? Can you name some transferable skills acquired by students through these engagements?
|Construct #3 Summary of Findings—Key Quotes|
My aim is to use live assignments in all my courses, but I try also to maintain the relevant equilibrium in the themes of the assignment, and the topics. Transferable skills could be professionalism, teamwork, and leadership.
Executives from the company deliver the project brief to students, deliver company presentations, provide support to student teams, and they attend the final student presentations. Transferrable skills are built through these projects in varying degrees: communication skills, reporting skills, presentation skills and teamwork skills, and leadership, time management, and negotiation skills.
The main finding is that it is evident that most respondents recognize the need to align their course with industry requirements and real-world problems. Thus, a critical component in the proposed KPIs is related to industry orientation and alignment. Two generic KPIs which will further be developed in future research are recommended:
Industry alignment = number of case study analyses per course x time allocated per analysis/total course teaching hours in an academic term.
Interaction with practitioners = number of interactions per course per academic term
Construct 4: Students’ research outcomes and quality
Question 4:Do you have any criteria for measuring the quality of the research work of your students? Are you interested in measuring the dissemination of their work? For example, how many research papers are published from students’ coursework?
|Construct #4 Summary of Findings—Key Quotes|
Important criteria are the appropriate use of suggested research methodology quality and appropriate use of sources of sources (updated bibliography, classic works, relevance to the specific topic, referencing/citations), concept use (terminology) and concept development, sociological relevance, application of theory, connection of research finding to relevant bibliography, organization and focus of the paper, development and clarity of argument, use of language and technical issues.
The main finding is that Ii is evident that most respondents recognize that student research works improve the quality perceptions of a course and its impact. Thus, a critical component in the proposed KPIs is related to research works and depth. Two generic KPIs which will further be developed in future research are recommended:
Construct 5: Engagement
Question 5:Do you promote discussion on a given topic among students in your classroom? Are you interested in the generation of new ideas on the topic discussed coming from students? How do you balance critical thinking and knowledge transfer in your lectures? Do you have any good recommendations: for example, 50% knowledge transfer and 50% critical thinking?
|Construct #5 Summary of Findings—Key Quotes|
It is difficult to balance critical thinking and knowledge on 50:50 basis, but given the nature of our discipline, that is, philosophy, and the relevant courses, it is fundamental to combine both these two components. I try at least to have a 40% (critical thinking) and knowledge (60%) balance.
Elaborating business cases serves the purpose of in-class discussion and exchange of ideas. In order to gain critical knowledge, students should have grasped theory as well as alternative interpretative frameworks. Thus, balancing knowledge and critical thinking is not a task which is easily accomplished.
I use educational videos as the starting point of a discussion or alternatively a case study, a graph some visual asking students to interpret and elaborate on the relevant topic. I ask students to contribute as I am presenting new material, to express their views, share experiences, and provide illustrations. Allocating about 1/3 of class time to class discussion should be appropriate.
The main finding is that it is evident that most respondents recognize that the engagement of students in interactive discussions promote the quality and the impact of teaching and learning. Additionally, most respondents replied that balancing knowledge and critical thinking is not a task which iseasily accomplished. For this reason, we have excluded references to balance from the proposed KPI. Thus, a critical component in the proposed KPIs is related to engagement. One generic KPI which will further be developed in future research is recommended:
Construct 6: Competencies and skills
Question 6:Do you constantly associate learning objectives to transferable skills? Do you assign a specific number of teaching assignments to students? Can you give an example stating key elements in such an assignment? For example: in order to promote critical thinking, I design the X assignment.
|Construct #6 Summary of Findings—Key Quotes|
In my field, that is, English, learning outcomes are directly related to transferable skills. All assignments require that students exercise their critical thinking skills by unpacking layers of meaning in various types of texts.
In my technology introductory course, students are assigned the development of a video which they share with their classmates through Blackboard. Then, based on a rubric I give them, they evaluate and rate their classmates’ videos.
The main finding is that most respondents recognize that practical and transferable skills as well as skills and competencies promote the quality and the impact of teaching and learning. Thus, a critical component in the proposed KPIs is related to engagement. One generic KPI which will further be developed in future research is recommended:
Construct 7: Inter/multi-disciplinary character
Question 8:What about the interdisciplinary character in the courses you teach? Can you name how many contributions from different disciplines you utilize in teaching your courses? For example, in the X course I teach, I use main contributions from four disciplines: Computer Science, Sociology, Psychology and News Media.
|Construct #7 Summary of Findings—Key Quotes|
The field I teach is interdisciplinary by its nature. We use concepts from different disciplines and emphasize the importance of economic, social and ecological dimensions of environmental issues. I try to present as many perspectives as possible so that students make connections with their disciplines. I ask students to reflect on how each discipline could help study a problem and help towards its solution. Information technology, math, different branches of natural sciences, social sciences (sociology, economics), law, ethics, and policy making are some of the disciplines that are involved in the study of the topics I present.
We live our lives in an interdisciplinary, multicultural and global fashion and our students should be educated like that to be successful citizens and employees. All of my classes have content from Politics, Geopolitics and Sociology.
The main finding is that it is evident that most respondents recognize that the multi/inter-disciplinary character of a course promotes the quality and the impact of teaching and learning. Thus, a critical component in the proposed KPIs is related to its inter/multi-disciplinary character. One generic KPI which will further be developed in future research is recommended:
Construct 8: Metrics
Question 9:If you were asked to write down a formula for quality in higher education, what factors would you include? For example, quality = time allowed for preparation + pedagogy + student engagement.
Question 10. Name one metric from your own perception for the quality of education in higher education. For example, “quality metric #1 = # of students passing a course / # of total students enrolled in this course” or “quality metric #2 = # of papers presented in conferences / # of papers delivered in a course assessment from students”.
Construct #8 Findings
In response to question 9, the interview participants suggested different formulas for the measurement of quality (QFs), based on their teaching experience and active involvement in curriculum design and review:
QF #1 = Time allowed for preparation + scholarship/academic expertise + pedagogies + student engagement;
QF #2 = Selected students + meaning of knowledge + engagement + dedication;
QF #3 = Faculty expertise + pedagogies + high academic standards;
QF #4 = Planning + preparation + personality + pedagogy + physical environment + assessment.
Similarly, in response to question 10, the following quality metrics (QMs) were suggested by interview participants and are summarized below:
QM #1 = Papers presented in conferences;
QM #2 = Job positions which business students get 5 years following graduation;
QM #3 = Successful teaching of transferable skills;
QM #4 = Synthesis of concepts;
QM #5 = Ability for independent study;
QM #6 = Ability to solve problems;
QM #7 = Ability to collaborate in teams;
QM #8 = Number and quality of faculty publications;
QM #9 = Student satisfaction and happiness;
QM #10 = Student engagement;
QM #11 = Number of students with high/good performance in course assessments.
In the Table 1
below, nine generic KPIs derived from the research findings are listed.
In Figure 3
below, the integrative model for the study of quality perceptions in higher education is introduced and, together with the nine generic KPIs shown in Table 1
, will provide the basis for future research. These KPIs will be applied to measure quality dimensions and produce quality metrics which will eventually be used by academic administrators and decision makers for quality enhancements leading to the sustainability of higher education curricula.