Next Article in Journal
Experimental Study on the Preparation of a Highly Active Bacterial Suspension for MICP in the South China Sea
Next Article in Special Issue
COVID-19 Impact: A Case Study at the School of Agricultural Engineering and Environment of the Universitat Politècnica de València
Previous Article in Journal
Comprehensive Assessment of Geopolitical Risk in the Himalayan Region Based on the Grid Scale
Previous Article in Special Issue
COVID-19 and E-Learning Adoption in Higher Education: A Multi-Group Analysis and Recommendation
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Attitudes and Opinions of Biomedical Students: Digital Education Questionnaire

by
Tamara Nikolic Turnic
1,2,*,
Sara Mijailovic
3,
Maja Nikolic
4,
Jelena Dimitrijevic
3,
Olivera Milovanovic
1,
Katarina Djordjevic
1,
Marko Folic
1,
Ljiljana Tasic
1,
Vladimir Reshetnikov
2,
Maria Mikerova
2,
Dragan Milovanovic
5 and
Vladimir Jakovljevic
4,6
1
Department of Pharmacy, Faculty of Medical Sciences, University of Kragujevac, Svetozara Markovića Street 69, 34000 Kragujevac, Serbia
2
N.A. Semashko Public Health and Healthcare Department, F.F. Erismann Institute of Public Health, I.M. Sechenov First Moscow State Medical University (Sechenov University), 119435 Moscow, Russia
3
Department of Medical Statistics and Informatics, Faculty of Medical Sciences, University of Kragujevac, Svetozara Markovića Street 69, 34000 Kragujevac, Serbia
4
Department of Physiology, Faculty of Medical Sciences, University of Kragujevac, Svetozara Markovića Street 69, 34000 Kragujevac, Serbia
5
Department of Pharmacology and Toxicology, Faculty of Medical Sciences, University of Kragujevac, Svetozara Markovića Street 69, 34000 Kragujevac, Serbia
6
Department of Human Pathology, 1st Moscow State Medical, University IM Sechenov, 8 Trubetskaya Street, Str. 2 119991, 119435 Moscow, Russia
*
Author to whom correspondence should be addressed.
Sustainability 2022, 14(15), 9751; https://doi.org/10.3390/su14159751
Submission received: 29 June 2022 / Revised: 2 August 2022 / Accepted: 4 August 2022 / Published: 8 August 2022

Abstract

:
(1) Background: the purpose of this study was a preliminary analysis of current methods of online teaching at the Faculty of Medical Sciences, University of Kragujevac, Republic of Serbia, in order to define the attitudes of biomedical students about education during the COVID-19 pandemic and to validate of the education medical questionnaire (eMedQ), a new tool for the assessment of the students’ perceptions about digital education. (2) Methods: this was a qualitative cross-sectional observational study that used the originally developed 45-item questionnaire (eMedQ) as an assessment instrument conducted on biomedical students (n = 209) of all study years at the Faculty of Medical Sciences, University of Kragujevac (Republic of Serbia), during winter semester or between September 2021 and January 2022. (3) Results: In this study, a Cronbach’s alpha value of 0.904 was obtained, which indicated good internal consistency; the correlation matrix revealed many coefficients greater than 0.3, denoting high correlations between the items. The Kaiser–Meyer–Olkin measure of sampling adequacy was 0.866 and Bartlett’s test of sphericity was significant (p < 0.001). The PCA revealed the presence of seven components with characteristic values over one, while three factors explained the highest percentage of variance. (4) Conclusions: this research developed and validated a new tool for evaluation of biomedical student perceptions about digital education.

1. Introduction

Changes in the world are constant and accelerated, which requires our parallel adaptation to novel modes of living, showing how humanity, although vulnerable, may also be very strong. One of the biggest health threats in human history, which has recently affected physical, psychological, and social lives, is the coronavirus disease 2019 (COVID-2019) pandemic, whose outbreak occurred in December 2019 in Wuhan, China [1,2]. On 31 January 2020, the World Health Organization (WHO) declared the global public health threat, while on March 11, the accelerated spreading of COVID-19 disease reached pandemic proportions [1]. The first COVID-19 case in the Republic of Serbia was reported on 6 March 2020, and on 15 March 2020 the Serbian government declared a state of emergency. Confrontation with the COVID-19 pandemic brought the inevitable need for immediate adaptation to the necessary restrictive measures, including the closure of international borders, suspension of education, tourism, hospitality, and other public institutions [2].
Having in mind that education is a human right [3], the negative effects of the newly emerging health risk situation at all levels of the educational system have not gone unnoticed. In fact, it has been reported that the closure of educational institutions due to the COVID-19 pandemic globally affected more than 1.5 billion students [4,5]. However, educational institutions all over the world made great efforts in order to accelerate the modification of the teaching process by switching classical teaching with online teaching [6]. From that point, the quality and efficiency of this virtual mode of education became an important subject of intensive research and continuous improvement. Various studies have pointed out numerous advantages of virtual education, including cost-effectiveness, schedule flexibility, and the absence of physical and temporal limitations, but also on many disadvantages, such as technological difficulties, low quality of teaching, and an inability to provide high-quality education in the field of applicable disciplines [6].
An additional challenge was to implement an online teaching process among biomedical students that would provide them with quality teaching content on one side but halt the face-to-face interaction with both teaching staff and patients during clinical practice. At the beginning of the pandemic, the Association of American Medical Colleges released the guidelines considering novel educational rules, including suspension of clinical rotations for medical students during the COVID-19 pandemic [7]. However, it is well known that acquiring practical knowledge and skills during clinical rotations is of crucial importance for future medical professionals [8]. A first study investigating the perceptions of medical students on this novel mode of learning, conducted in the United Kingdom during May 2020, reported that students were less satisfied with online compared to in-person teaching, with concerns about whether they will be adequately prepared for their future professions [9]. However, properly adjusted, the long-term, hybrid learning, which includes a mixed model of on-site/online teaching, appears to be adequate for teaching in biomedical sciences [10].
In the Republic of Serbia, the suspension of in-person classes forced all stages of educational institutions to adopt different modes of remote teaching. In high educational institutions, various existing platforms have been used for these purposes, such as MS Teams, Zoom, etc. Some institutions even constructed their own online platforms in order to improve the accessibility of educational materials to all students.
The core of our research is based in the fact that it gives comprehensive and valid information that can create favorable conditions for achieving more efficient online teaching methods at the faculties of medical sciences in the Republic of Serbia. In such a manner, we will provide insight into previous circumstances in students’ education in the time of the pandemic induced by the COVID-19 virus. Observed information and identified deficiencies can be used as a working directive for increasing the quality of online education in a specific area of biomedical sciences.
The purpose of this study was the examination of the current approach of learning online at the Faculty of Medical Sciences, University of Kragujevac, Serbia, in order to determine the opinions of biomedical students (medicine, pharmacy, and dentistry) about learning methods in the time of the COVID-19 pandemic, their previous experiences, and validation of the education medical questionnaire (eMedQ), a new tool for assessment of the students’ perceptions about digital education.

2. Materials and Methods

2.1. Ethical Concerns

This study’s protocol was approved by the institutional Ethics Committee, Faculty of Medical Sciences, University of Kragujevac (no. 143/19), and it is in accordance with the principles of the Declaration of Helsinki (revision 2013). The participation in this examination was voluntary and anonymous.

2.2. Research Design and Participants

This was a qualitative, cross-sectional, and observational study that used the originally developed questionnaire as an assessment instrument. The whole data were collected from biomedical students (n = 209) of all study years at Faculty of Medical Sciences, University of Kragujevac (Republic of Serbia), during winter semester or between September 2021 and January 2022. The study sample consisted of representative number of biomedical students from central part of Republic of Serbia who were active students during pandemic.
Criteria for the inclusion of respondents were as follows: students who were at undergraduate or vocational level at all biomedical faculties in Central Serbia, the presence of a study subject’s user account on the specific online platform (Licensed Moodle Platform for Online Education 27655/2), or adequate online approach and voluntary participation. For our study, there were not foreseen any special exclusion criteria.

2.3. Pilot Study for Item Generation and Targeting Population

In constructing the final instrument (45-item), we used the data from our previous pilot questionnaire (17-item) study, which was conducted between December 2020 and January 2021 at Faculty of Medical Sciences [11]. That pilot study was designed as a qualitative study, which included 332 participants (students and academics staff) at the Faculty of Biomedical Sciences, University of Kragujevac, during the pandemic of SARS-CoV-2 infection in Serbia. Pilot version of questionnaire consists of 17 items, and the aim of the first phase was to determine the share of participation in relation to the total population of potential respondents, to determine whether the questions are appropriate and understandable, what is the distribution of answers, and the suitability for their statistical analysis [11].

2.4. Moodle Open-Source Learning Platform

The Faculty of Medical Sciences had a prepared and elaborated strategy for all working conditions, including the pandemic situation in which we found ourselves globally. The Moodle platform allows students to access electronic textbooks, an electronic library, and subject pages of all study programs, as well as all other materials necessary for successful mastering of the material. Moreover, there is all the necessary information related to the faculty and schedules of lectures and exams, but also, thanks to the multifunctionality of the platform, there is the possibility of organizing online tests (colloquia and exams).

2.5. Selecting the Size of Study Sample

According to the results of pilot study [11] and the allocation of subjects regarding gender, but also gender representation in general among student and teaching population of the Faculty of Medical Sciences in January 2021, we made target research sample which presents a similar gender distribution to the official student/academic staff population at the Faculty of Medical Sciences in Kragujevac. The total number of students at Faculty of Medical Sciences is 1572, and for development and validation of eMedQ, 209 undergraduate students were selected as a representative number (7.5%) of all students from Faculty of Medical Sciences, University of Kragujevac.

2.6. Constructed eMedQ Instrument for Evaluation of Attitudes and Perceptions among Biomedical Students

The Serbian version of eMedQ is comprised of 45 closed-ended questions with graduated answers divided into 7 domains as follows: demographic characteristics, experience with online teaching, education process (teaching organization), aspects of mental functioning, clinical skills, technical aspects, and quality of life. Students filled out a questionnaire through an online learning platform, which served as educational tool during COVID-19 pandemic at this institution. Study attendance in our research was on voluntary base and entirely unidentified, and the identification details of the technical approach of the study sample were available only to the server administrator, who handles the data in line with the regular working practice of the institution. The dimensions of questionnaire were constructed according to similar research methods [12,13].

2.7. Validation of Serbian Version of eMedQ (Reliability Testing and Factor Analysis)

Internal consistency as a measure of inter-correlation of the items of the questionnaire was accessed by calculating the Cronbach alpha coefficient. Moreover, the split-half reliability testing was conducted. This method involved dividing the questionnaire into two parts and calculating Cronbach’s alpha for each part. The Spearman–Brown formula was used to determine the correlation of the scores on the scales from the two parts.
In order to discover principal factors, 33 items underwent principal component analysis (PCA). Prior to conducting the analysis, the suitability of the data for analysis was assessed by examining the correlation matrix, observing the values of the Kaiser–Meyer–Olkin indicator and the significance of the Bartlett sphericity test. Firstly, the factors were extracted without rotation based on eigenvalues greater than 1, scree plot, and factor loadings. Then, oblimin rotation was conducted.

2.8. Statistical Analysis

IBM SPSS, version 26.0, was used for statistical processing of the obtained data. The study sample was established according to the design of the survey type, based on the presumption of an error margin of 5% and confidence limits of 95%. The complete calculated obtainable study respondents at the Faculty of Medical Sciences, University of Kragujevac, were 1572 respondents, and the share of participation is estimated at 30%. Using an appropriate online calculator [14], a study pattern of 190 participants was determinate so that it was defined to the smallest sample from 200 people for the research. Descriptive statistics included the presenting of category characteristics as absolute and relative frequencies of individual categories. To test the difference between the obtained and expected frequencies (frequencies), a matching test or a chi-square test was used to examine the quality of the match. A chi-square independence test was used to examine the relationship between categorical variables. The data are presented in tabular and graphical form.

3. Results

3.1. Basic Demographic Characteristics of Participants

In Table 1, the basic characteristics of the study population are shown. We have observed that the mean age of the study group was 21.83 ± 4.163, with predominantly female students present. Most of the participants were pharmacy students from the second year of study with very similar skills in the use of electronic devices (Table 1).

3.2. Validation of Serbian Version of eMedQ

By using the internal consistency as a measure of inter-correlation of the items of the questionnaire, the Cronbach alpha coefficient was calculated. Moreover, the split-half reliability testing was conducted. This method involved dividing the questionnaire into two parts and calculating Cronbach’s alpha for each part. The Spearman–Brown formula was used to determine the correlation of the scores on the scales from the two parts.
In this study, we observed the Cronbach’s alpha at 0.904, which indicates good internal consistency. When the questionnaire was divided by the split-half method into two parts, Cronbach’s alphas were 0.852 and 0.906. The value of the Spearman–Brown coefficient for the questionnaire was 0.555.

3.3. Factor Analysis

In order to discover principal factors, 33 items underwent principal component analysis (PCA). Prior to conducting the analysis, the suitability of the data for analysis was assessed by examining the correlation matrix and observing the values of the Kaiser–Meyer–Olkin indicator and the significance of the Bartlett sphericity test. Firstly, the factors were extracted without rotation based on eigenvalues greater than one, scree plot, and factor loadings. Then, oblimin rotation was conducted.
In this research, the correlation matrix revealed many coefficients greater than 0.3, denoting high correlations between the items. The Kaiser–Meyer–Olkin measure of sampling adequacy was 0.866, and Bartlett’s test of sphericity was significant (p = 0.000). The PCA revealed the presence of seven components with characteristic values over one, while three factors explained the highest percentage of variance. The scree plot also suggested a three-factor solution. After oblimin rotation was conducted, the rotated solution confirmed three factors explaining 47.7% of the variance. Two items were excluded due to having low loadings. The distribution of questions into factors after direct oblimin rotation is shown in Table 1. Cronbach’s alpha for the remaining items was 0.913. Cronbach’s alpha for the first, second, and third factor was 0.912, 0.878, and 0.781, respectively.
By using the factor analysis, we observed two negative factors in our instrument, such as questions Q35 and Q40, which are excluded from further analysis and the final form of the instrument (Figure 1).

3.4. Analyzing of the Perceptions Regarding the Online Education among Biomedical Students

Table 2 presents the answers from biomedical students related to the main perceptions regarding the online education during the COVID-19 outbreak. As we can see, most of the students who used the Zoom tool for online teaching expressed progress during online education (39.9%). Half of students used presentation with narration and half of them did not, and almost all students did not use online consultations with the lecturers (Figure 2). From Figure 2, we can see that there were statistically significant differences between answers regarding the forms of online teaching that students experienced during the COVID-19 pandemic (p < 0.001, chi-square test).
Interesting, about 57.4% of biomedical students think that the use of online education must be improved, while 42.6% of students do not think that. Perceptions about the online education modalities that could improve education and additional material for teaching are similar among all included students (Figure 3).

3.5. Analyzing of the Attitudes Regarding the Online Education among Biomedical Students

The characteristics of the study population related to attitudes toward online education from different aspects are presented in Table 3. Regarding the education process (questions 13–22), most of the biomedical students were satisfied with the organization and type of learning (Table 3). This type of education induced some type of changed mental functioning, as we can see from the next part of the questionnaire (questions 23–29). A large number of students had a drop in motivation and concentration (Q23–34.4%; Q24–35.1%) and difficulties memorizing lectures (Q25–26.3%). On the other hand, online education during the COVID-19 outbreak did not induce mental stress in a large number of students (Q26–26.1%), but we could not diminish the significant number of the students with some type of stress during digital education (Q26–22.2%). Depressive emotion was not dominant amount biomedical students, and we also have an indicative number of students who reported that they had emotions that contributed to depression and anxiety (Q27–15.3%; Q28–17.3%). Moreover, the occurrence of insomnia in students was very significant, so about 39.2% of biomedical students reported the presence of insomnia during the COVID-19 outbreak and studying (Q29) (Table 3).

3.6. Multifactorial Analysis of Different Variables on the Perception and Attitudes among Biomedical Students

Table 4 shows the mean scores and median of scores of factors one to three. As we can see, the means of the first two factors are similar, as well as the medians, while the third factor has the lowest mean. In that sense, to analyze which variable has the most significant impact on the factor separately, we carried out the analytical statistical test shown in Table 5. Study program and year of study are two statistically significant variables that influenced factors one to three. There are differences between IASM vs. IASF: Factor I (0.023) and Factor III (0.003), IASF vs. IASS: Factor I (0.000), Factor II (0.020), and Factor III (0.002), and IASF vs. OSS: Factor I (0.006), Factor II (0.001), and Factor III (0.001) (Table 5). Moreover, the year of study determines the factors as follows: II vs. IV: Factor I (0.003), Factor II (0.025), and Factor III (0.010); IV vs. VI: Factor I (0.008), Factor II (0.001), and Factor III (0.000).
Moreover, progress during online education during the COVID-19 pandemic was observed as significant variables in determining the scores of each part of eMedQ. Interestingly, further improvement of the current form of online teaching is something that is also observed as a significant variable. Finally, the type of online application/platform was also something that statistically influenced factors one and three, where students choose one of the mentioned (Zoom/Teams/Webex/Google Meet meeting) (Table 5).
From our point of view, the most important variable extracted after statistical analysis was progress in education among biomedical students, where it is statistically confirmed that for the students it is important how much they can progress through digital education in this specific pandemic environment and in medical education (Figure 4). We observed the next significant differences: Very much vs. Moderate: Factor I (0.037), Factor II (0.002), and Factor III (0.002); Very much vs. Little: Factor I (0.010), Factor II (0.000), and Factor III (0.000); Very much vs. I have not progressed: Factor II (0.000) and Factor III (0.014); Much vs. Moderate: Factor I (0.016) and Factor III (0.039); Much vs. Little: Factor I (0.005), Factor II (0.002), and Factor III (0.001); Much vs. I have not progressed: Factor II (0.002); Moderate vs. Little: Factor II (0.027) and Factor III (0.040); Moderate vs. I have not progressed: Factor II (0.004). On the other hand, between the next answers there were no differences: Very much vs. Much: no difference; Little vs. I have not progressed: no difference (Figure 4).

4. Discussion

The purpose of this questionnaire study was an evaluation of current teaching approaches at the Faculty of Medical Sciences, University of Kragujevac, Serbia, in order to determine the attitudes of biomedical students about learning methods in the time of the COVID-19 pandemic, as well as their previous experiences. Moreover, the second aim was to establish the tool for determining the attitudes and perceptions of students from medical faculties regarding digital education in pandemic conditions.
All over the world, the impacts of the pandemic in recent years have been observed. In higher education, institutions responded innovatively and rapidly to the unusual conditions in each region. However, the impact varied according to the particular system, and for each of them this pandemic was a chance to answer social needs. How each region responds to the unexpected and provides the necessary medical education focused on students in health care professions are excellent representations of how the regional and national higher-education communities responded to the pandemic. Moreover, it is still important to find a reason for barriers to student learning that could be related to the learning environment [15,16,17,18].
The first part of the study is designed to develop and validate the eMedQ, a 45-item global questionnaire for determining the students’ perceptions of the learning environment. Our results suggested that eMedQ is an acceptable instrument and provided the evaluation of different aspects of students’ learning and medical education, such as demographic characteristics, experience with online teaching, education process (teaching organization), aspects of mental functioning, clinical skills, technical aspects, and quality of life. Developing the eMedQ in these seven dimensions, we represented a tool for measuring the overall perception of the learning environment. Initially, the target population for this study was undergraduate students and academic staff. However, during the development process, we expanded the previous scope related to academic staff, considering the large differences between previous skills, experience, and age among academic staff [4,19]. Furthermore, we included only students, but undergraduate, since that level of study requires more intensive, continuous, and practical education. Moreover, this eMedQ is focused on students from biomedical faculties and this tool could be relevant for students of health care professions (e.g., medicine, pharmacy, dentistry, and nursing sciences).
There are some other constructed surveys that were used in the measuring of the impact of COVID-19 in higher-education institutions. The first global survey conducted by The International Association of Universities under UNESCO was conceived in order to better understand the disruption caused by COVID-19 on higher education and to investigate the first measures undertaken by higher-education institutions around the world in response to the crisis. This instrument provides only the information about the learning but not the psychological aspects of ours.
The second aim of this research was to evaluate the study programs in the form of digital education related to biomedical sciences in the Republic of Serbia. Definitely, the COVID-19 pandemic suddenly transformed the ways of working, living, and relating to each other on a global level. This research was focused on education in medical faculties, and we showed how higher-education institutions transformed their own educative and training processes. We constructed the self-assessment survey, which examined and founded barriers and challenges that students have. Based on that information, we plan to transform higher education and apply in full the digitalization of education in the foreseeable future. According to our results, remote learning did not disrupt the quality of education in a greater manner, even in this last short time. Moreover, the system posed unprecedent challenges for students, but they had to keep assisting and learning in the same manner as before the pandemic. This technological revolution in higher education in the Republic of Serbia was very fast, but young people such as students from our research led these fundamental changes and accelerated them. The motivation of the most students was not decreased and even increased in some students with the requirements for new educational material and methods (e.g., chats, interaction with professors, recorded lectures, gamification, quizzes, etc.).
In the last two years, there have been many studies that examined the impact of technology on higher education during the age of COVID-19 [20,21,22,23,24,25].
Carrillo and Flores [26] carried out a literature review between January 2000 and April 2020 on online teaching and learning practices in teacher education to explore how and why online teaching and learning in teacher education occur and also to discuss its implications in the context of the pandemic. The authors emphasized the complicated character of the teaching method, point out specific elements such as social, cognitive, and teaching issues and the need for a comprehensive view of the pedagogy of online technology-based education used to support teaching and learning.
Di Pietro et al. [27] made a work that examined the direct and indirect ways in which the COVID-19 pandemic may influence education. Based on the existing literature and pre-COVID-19 data, it made predictions about the impact on and future of education. The results observed four major findings: (1) learning, on average, is expected to experience stagnation; (2) the impact on academic performance is likely to vary with socio-economic levels; (3) different socio-economic positions may manifest in an emotional response, as those from less privileged backgrounds may be under more environmental stress; (4) the broadening social gap may persist and have permanent consequences.
Since that phenomenon is still recent, there are just a few studies that discuss the direct effects of digital education in higher education caused by the pandemic, especially on pros, cons, and future consequences. Up to today, there are just literature reviews that systematically report about the transition from face-to-face education or traditional classroom education to the new remote or digital education [28,29,30].
Remote medical education is a specific situation and required the multidisciplinary approach. Undergraduate medical students have a lot of practical education, and the aim of their study is archiving skills. For example, radiology residents have to see many patients and radiographs in order to learn how to recognize different types of pneumonia. A very interesting previous report about the clinical skills of residents during the COVID-19 pandemic examined how the learning process was organized [31]. Mc Roy and coworkers describe a novel cloud-based HIPAA-compliant and accessible education platform that simulates a live radiology workstation for continued education of first-year radiology (R1) residents, with an emphasis on call preparation and peer-to-peer resident learning. Methods such as this and many others must be developed, because remote medical education must closely mimic face-to-face learning [31].
In Spain, a prospective study was conducted to examine the learning experiences in education during pandemic. The authors concluded that the imposition of e-learning provides limitations for older students and for students from rural areas, based on technical limitations [32].
In China, Huang et al. examined the effects of online education on providing the skills from chemistry. In this research, 56 teachers and 432 students were tested, and the authors concluded that the new way of education was a huge challenge for both students and teachers and that teachers must be more familiar with new technologies [33].
Moreover, previous authors examined the effects of the COVID-19 pandemic on the learning process among 242 students in Nigeria. They concluded that this pandemic has affected students’ well-being, behavior, and learning [34]. Similarly, research conducted by Russian teachers tended to examine the effects of the pandemic on the learning process but from the other point of view [35]. From the answers from 87 university teachers, the study provided the conclusion that many professional development programs at universities helped to minimize the negative impact of the rapid changes of the educational process [35]. Definitely, the COVID-19 pandemic has induced multilevel changes in the learning process among universities in different countries.
This study was the first research from this region that examined the impact of the COVID-19 pandemic on the perceptions and attitudes of biomedical students. Definitely, we observed the most important variables that influenced the global perception of remote medical education, such as type of study, year of study, progression in education, and striving for further improvement. Other difficulties have been solved, as we observed in this research.
The limitation of this study could be a lack of viewpoints from learners or educators regarding the switch from offline to online education among medical students. Moreover, a limitation is the unicentric placement of the study and including students from one institution, such as the University of Kragujevac. That will be the plan of the next research, when we will evaluate the lecturers’ acceptance of online education.

5. Conclusions

This research developed and validated a new tool (eMedQ) for the evaluation of biomedical students’ perceptions about digital education. It goes without saying that the COVID-19 pandemic has had profound impacts on society and the way humans organize themselves in the real world. The real consequences will be seen in the distant future. Our higher-education institutions invested heavily in technical infrastructure to remove the many barriers in shifting from conventional to a blended education model in biomedical sciences. Moreover, learning assessment and other approaches must be observed in order to comply with online medical teaching and learning pedagogy. Many other high-quality remote tools must be implemented for students of health professions who require practical skills.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/su14159751/s1, Supplementary Files S1 and S2 (original and translated eMedQ).

Author Contributions

Conceptualization, methodology, and software, L.T., D.M. and V.J.; validation, S.M., J.D., T.N.T. and M.N.; writing—original draft preparation, O.M., K.D. and M.F.; writing—review and editing, M.M. and V.R.; supervision and project administration, T.N.T., K.D. and V.J. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

This study’s protocol was approved by the institutional Ethics Committee, Faculty of Medical Sciences, University of Kragujevac (no. 143/19), and it is in accordance with the principles of the Declaration of Helsinki (revision 2013). The participation in this study was voluntary and anonymous.

Informed Consent Statement

Not applicable.

Data Availability Statement

All data are available on the request.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Cucinotta, D.; Vanelli, M. WHO declares COVID-19 a pandemic. Acta Biomed. Atenei Parm. 2020, 91, 157–160. [Google Scholar]
  2. Djurović, I. Epidemiological control measures and predicted number of infections for SARS-CoV-2 pandemic: Case study Serbia march–april 2020. Heliyon 2020, 6, e04238. [Google Scholar] [CrossRef] [PubMed]
  3. UN. Universal Declaration of Human Rights. United Nations. 1948. Available online: https://www.un.org/en/universaldeclaration (accessed on 19 April 2022).
  4. Unesco. COVID-19 Education Response. 2020. Available online: https://en.unesco.org/covid19/educationresponse/globalcoalition (accessed on 9 March 2022).
  5. UNICEF. UNICEF and Microsoft Launch Global Learning Platform to Help Address COVID-19 Education Crisis. 2020. Available online: https://www.unicef.org/press-releases/unicef-and-microsoft-launch-globallearning-platform-help-address-covid-19-education (accessed on 9 March 2022).
  6. Masalimova, A.R.; Khvatova, M.A.; Chikileva, L.S.; Zvyagintseva, E.P.; Stepanova, V.V.; Melnik, M.V. Distance Learning in Higher Education During Covid-19. Front. Educ. 2022, 7, 822958. [Google Scholar] [CrossRef]
  7. Association of American Medical Colleges. Important Guidance for Medical Students on Clinical Rotations During the Coronavirus (COVID-19) Outbreak. [Press Release]. 2020. Available online: https://www.aamc.org/news-insights/press-releases/important-guidance-medical-students-clinical-rotations-during-coronavirus-covid-19-outbreak (accessed on 1 April 2022).
  8. Chakladar, J.; Diomino, A.; Li, W.T.; Tsai, J.C.; Krishnan, A.R.; Zou, A.E.; Kharidia, K.; Baig, F.A.; Householder, S.; Kuo, S.Z.; et al. Medical student’s perception of the COVID-19 pandemic effect on their education and well-being: A cross-sectional survey in the United States. BMC Med. Educ. 2022, 22, 149. [Google Scholar] [CrossRef]
  9. Dost, S.; Hossain, A.; Shehab, M.; Abdelwahed, A.; Al-Nusair, L. Perceptions of medical students towards online teaching during the COVID-19 pandemic: A national cross-sectional survey of 2721 UK medical students. BMJ Open 2020, 10, e042378. [Google Scholar] [CrossRef] [PubMed]
  10. Goddard, L.; Bruinsma, F.J.; Giles, G.G. Informed consent and internet-based research in epidemiology. Med. J. Aust. 2018, 208, 424. [Google Scholar] [CrossRef] [PubMed]
  11. Turnic, T.N.; Tasic, L.; Jakovljevic, V.; Folic, M.; Zaric, M.; Milovanovic, O.; Simovic, S.; Ognjanovic, I.; Zdravkovic, N.; Mijailovic, S.; et al. Online Teaching and Learning in Biomedical Sciences: Students’ and Staff’ Attitudes and Experiences on the New Educational Environment Imposed by the COVID 19 Outbreak. Serbian J. Exp. Clin. Res. 2021. [Google Scholar] [CrossRef]
  12. Rajab, M.H.; Gazal, A.M.; Alkattan, K. Challenges to Online Medical Education During the COVID-19 Pandemic. Cureus 2020, 12, e8966. [Google Scholar] [CrossRef]
  13. Althwanay, A.; Ahsan, F.; Oliveri, F.; Goud, H.K.; Mehkari, Z.; Mohammed, L.; Javed, M.; Rutkofsky, I.H. Medical Education, Pre- and Post-Pandemic Era: A Review Article. Cureus 2020, 12, e10775. [Google Scholar] [CrossRef]
  14. Raosoft Inc. RaoSoft Sample Size Calculator. 2004. Available online: http://www.raosoft.com/samplesize.html (accessed on 1 June 2020).
  15. Murphy, M.P.A. COVID-19 and emergency eLearning: Consequences of the securitization of higher education for post-pandemic pedagogy. Contemp. Secur. Policy 2020, 41, 492–505. [Google Scholar] [CrossRef]
  16. Crawford, J.; Percy, A.; Kelder, J.-A. JUTLP Editorial 17.3: Connection, digital education, and student-centric teaching practice before COVID-19. J. Univ. Teach. Learn. Pract. 2020, 17, 1–5. [Google Scholar]
  17. Butler-Henderson, K.; Crawford, J.; Rudolph, J.; Lalani, K.; Sahu, K.M. COVID-19 in Higher Education Literature Database (CHELD V1). Inst. Res. Innov. 2020. [Google Scholar] [CrossRef]
  18. Bonk, R.; Kefalaki, M.; Rudolph, J.; Diamantidaki, F.; Rekar Munro, C.; Karanicolas, S.; Kontoleon, P.; Pogner, K. Pedagogy in the time of pandemic: From localisation to glocalisation. J. Educ. Innov. Commun. 2020, 2, 17–64. [Google Scholar]
  19. Available online: https://www.iau-aiu.net/IMG/pdf/iau_covid-19_regional_perspectives_on_the_impact_of_covid-19_on_he_july_2020_.pdf (accessed on 1 March 2022).
  20. Choi, B.; Jegatheeswaran, L.; Minocha, A.; AlHilani, M.; Nakhoul, M.; Mutengesa, E. The impact of the COVID-19 pandemic on final year medical students in the United Kingdom: A national survey. BMC Med. Educ. 2020, 20, 206. [Google Scholar] [CrossRef] [PubMed]
  21. Aquifer (n.d.). About Advancing Clinical Teaching & Learning. Available online: https://aquifer.org/about-aquifer/ (accessed on 26 June 2022).
  22. A Babchuk, W. Fundamentals of qualitative analysis in family medicine. Fam. Med. Community Healh 2019, 7, e000040. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  23. Harrison, A.G.; Lin, T.; Wang, P. Mechanisms of SARS-CoV-2 transmission and pathogenesis. Trends Immunol. 2020, 41, 1100–1115. [Google Scholar] [CrossRef]
  24. Rose, S. Medical Student Education in the Time of COVID-19. JAMA 2020, 323, 2131–2132. [Google Scholar] [CrossRef]
  25. Masic, I. E-Learning as New Method of Medical Education. Acta Inform. Medica 2008, 16, 102–117. [Google Scholar] [CrossRef] [Green Version]
  26. Carrillo, C.; Flores, M.A. COVID-19 and teacher education: A literature review of online teaching and learning practices. Eur. J. Teach. Educ. 2020, 43, 466–487. [Google Scholar] [CrossRef]
  27. Di Pietro, G.; Biagi, F.; Costa, P.; Karpiński, Z.; Mazza, J. The Likely Impact of COVID-19 on Education: Reflections Based on the Existing Literature and Recent International Datasets; Publications Office of the European Union: Luxembourg, 2020.
  28. Khalaf, K.; El-Kishawi, M.; Moufti, M.A.; Al Kawas, S. Introducing a comprehensive high-stake online exam to final-year dental students during the COVID-19 pandemic and evaluation of its effectiveness. Med. Educ. Online 2020, 25, 1826861. [Google Scholar] [CrossRef]
  29. Mulenga, E.M.; Marbán, J.M. Prospective Teachers’ Online Learning Mathematics Activities in The Age of COVID-19: A Cluster Analysis Approach. Eurasia J. Math. Sci. Technol. Educ. 2020, 16, em1872. [Google Scholar] [CrossRef]
  30. Rodriguez-Segura, L.; Zamora-Antuñano, M.A.; Rodríguez-Reséndiz, J.; Paredes-García, W.J.; Altamirano-Corro, J.A.; Cruz-Pérez, M.Á. Teaching Challenges in COVID-19 Scenery: Teams Platform-Based Student Satisfaction Approach. Sustainability 2020, 12, 7514. [Google Scholar] [CrossRef]
  31. McRoy, C.; Patel, L.; Gaddam, D.S.; Rothenberg, S.; Herring, A.; Hamm, J.; Chelala, L.; Weinstein, J.; Smith, E.; Awan, O. Radiology Education in the Time of COVID-19: A Novel Distance Learning Workstation Experience for Residents. Acad. Radiol. 2020, 27, 1467–1474. [Google Scholar] [CrossRef]
  32. Ramos-Morcillo, A.J.; Leal-Costa, C.; Moral-García, J.E.; Ruzafa-Martínez, M. Experiences of Nursing Students during the Abrupt Change from Face-to-Face to e-Learning Education during the First Month of Confinement Due to COVID-19 in Spain. Int. J. Environ. Res. Public Healh 2020, 17, 5519. [Google Scholar] [CrossRef] [PubMed]
  33. Huang, J. Successes and Challenges: Online Teaching and Learning of Chemistry in Higher Education in China in the Time of COVID-19. J. Chem. Educ. 2020, 97, 2810–2814. [Google Scholar] [CrossRef]
  34. Cifuentes-Faura, J.; Obor, D.O.; To, L.; Al-Naabi, I. Cross-cultural impacts of COVID-19 on higher education learning and teaching practices in Spain, Oman, Nigeria and Cambodia: A cross-cultural study. J. Univ. Teach. Learn. Prac. 2021, 18, 135–151. [Google Scholar] [CrossRef]
  35. Almazova, N.; Krylova, E.; Rubtsova, A.; Odinokaya, M. Challenges and Opportunities for Russian Higher Education amid COVID-19: Teachers’ Perspective. Educ. Sci. 2020, 10, 368. [Google Scholar] [CrossRef]
Figure 1. Factor analysis among all tested questions in eMedQ. Extraction method: principal component factoring; rotation Method: oblimin rotation with Kaiser normalization.
Figure 1. Factor analysis among all tested questions in eMedQ. Extraction method: principal component factoring; rotation Method: oblimin rotation with Kaiser normalization.
Sustainability 14 09751 g001
Figure 2. Form of online teaching experienced during the COVID-19 pandemic. Results are presented as frequency in percentage from total number of participants (%).
Figure 2. Form of online teaching experienced during the COVID-19 pandemic. Results are presented as frequency in percentage from total number of participants (%).
Sustainability 14 09751 g002
Figure 3. Online educational modalities that would significantly improve the acquisition of practical knowledge and skills in biomedical sciences. Results are presented as frequency in percentage from total number of participants (%).
Figure 3. Online educational modalities that would significantly improve the acquisition of practical knowledge and skills in biomedical sciences. Results are presented as frequency in percentage from total number of participants (%).
Sustainability 14 09751 g003
Figure 4. Progress in education among biomedical students during the COVID-19 pandemic in relation to factors. Results are presented as means.
Figure 4. Progress in education among biomedical students during the COVID-19 pandemic in relation to factors. Results are presented as means.
Sustainability 14 09751 g004
Table 1. The basic demographic characteristics of study population. Data are presented as mean ± SD, median (min–max), or frequency in percent (%). Chi-square test was used to test the differences between selected variables with the significance level of 0.05.
Table 1. The basic demographic characteristics of study population. Data are presented as mean ± SD, median (min–max), or frequency in percent (%). Chi-square test was used to test the differences between selected variables with the significance level of 0.05.
CharacteristicN (%)Chi-Square Test/p
Gender
Male42 (20.4)72.252/0.000
Female164 (79.6)
Study program
IASM39 (18.8)91.760/0.000
IASF80 (38.5)
IASS24 (11.5)
OSS65 (31.3)
Year of study
I26 (12.4)230.321/0.000
II115 (55.0)
III5 (2.4)
IV27 (12.9)
V18 (8.6)
VI18 (8.6)
Level of skills in using electronic devices (computers, smartphones, tablets…)
Inadequate2 (1.0)91.856/0.000
Acceptable15 (7.2)
Good57 (27.4)
Very good64 (30.8)
Excellent70 (33.7)
Experience with online education before COVID-19
Very great experience15 (7.3)58.709/0.000
Great experience18 (8.7)
Moderate experience42 (20.4)
Little experience63 (30.6)
Without any experience68 (33.0)
Table 2. Perception regarding the online education among biomedical students. Data are presented as mean ± SD, median (min–max), or frequency in percent (%). Chi-square test was used to test the differences between selected variables with the significance level of 0.05.
Table 2. Perception regarding the online education among biomedical students. Data are presented as mean ± SD, median (min–max), or frequency in percent (%). Chi-square test was used to test the differences between selected variables with the significance level of 0.05.
VariablesN (%)Chi-Square Test/p
Progress during the online education during the COVID-19 pandemic
Very much30 (14.8)72.148/0.000
Much54 (26.6)
Moderate81 (39.9)
Little25 (12.3)
I have not progressed13 (6.4)
Further improvement of the current form of online teaching
It is necessary109 (57.4)4.126/0.042
It is not necessary81 (42.6)
Area in which it would be useful to have additional teaching materials compared to existing online materials
Preclinical subjects (anatomy, physiology, histology, genetics, chemistry, and other fields...)41 (21.6)71.568/0.000
Subjects of clinical medicine, clinical pharmacy, and clinical dentistry118 (62.1)
Other subjects:31 (16.3)
Table 3. The attitudes regarding the online education among biomedical students. Data are presented as mean ± SD, median (min–max), or frequency in percent (%). Chi-square test was used to test the differences between selected variables with the significance level of 0.05. The full names of questions are presented in Supplementary File S1.
Table 3. The attitudes regarding the online education among biomedical students. Data are presented as mean ± SD, median (min–max), or frequency in percent (%). Chi-square test was used to test the differences between selected variables with the significance level of 0.05. The full names of questions are presented in Supplementary File S1.
QuestionStrongly Disagree N (%)Somewhat Disagree
N (%)
Neither Agree nor Disagree
N (%)
Somewhat
Agree
N (%)
Strongly Agree
N (%)
Chi-Square Test/p
EDUCATION PROCESS (TEACHING ORGANIZATION)
Q138 (3.8) 16 (7.7)44 (21.1)70 (33.5)71 (34.0)82.794/0.000
Q149 (4.3)31 (15.0)49 (23.7)59 (28.5)59 (28.5)44.329/0.000
Q156 (2.9)17 (8.1)28 (13.4)62 (29.7)96 (45.9)129.971/0.000
Q166 (2.9)19 (9.1)30 (14.4)36 (17.3)117 (56.3)183.394/0.000
Q1712 (5.7)18 (8.6)47 (22.5)64 (30.6)68 (32.5)63.656/0.000
Q188 (3.8)12 (5.7)37 (17.7)55 (26.3)97 (46.4)126.191/0.000
Q199 (4.3)19 (9.1)51 (24.4)67 (32.1)63 (30.1)66.144/0.000
Q2011 (5.3)12 (5.8)36 (17.3)52 (25.0)97 (46.6)120.702/0.000
Q2128 (13.6)15 (7.3)50 (24.3)42 (20.4)71 (34.5)44.340/0.000
Q2287 (41.6)23 (11.0)27 (12.9)32 (15.3)40 (19.1)64.947/0.000
ASPECTS OF MENTAL FUNCTIONING
Q2328 (13.6)21 (10.0)49 (23.4)39 (18.7)72 (34.4)38.153/0.000
Q2435 (16.8)21 (10.1)37 (17.8)42 (20.2)73 (35.1)35.462/0.000
Q2536 (17.2)24 (11.5)53 (25.4)41 (19.6)55 (26.3)15.569/0.004
Q2654 (26.1)34 (16.4)43 (20.8)30 (14.5)46 (22.2)8.870/0.064
Q2764 (30.6)24 (11.5)56 (26.8)33 (15.8)32 (15.3)28.344/0.000
Q2864 (30.8)25 (12.0)53 (25.5)30 (14.4)36 (17.3)25.798/0.000
Q2982 (39.2)20 (9.6)52 (24.9)18 (8.6)37 (17.7)66.622/0.000
CLINICAL SKILLS
Q3047 (22.6)27 (13.0)66 (31.7)47 (22.6)21 (10.1)31.038/0.000
Q3134 (16.4)40 (19.3)67 (32.4)40 (19.3)26 (12.6)22.976/0.000
Q3210 (4.8)9 (4.3)36 (17.4)45 (21.7)107 (51.7)154.135/0.000
Q3333 (15.9)23 (11.1)51 (24.6)46 (22.2)54 (26.1)16.454/0.000
Q34115 (55.8)25 (12.1)31 (15.0)16 (7.8)19 (9.2)168.466/0.000
Q3511 (5.4)7 (3.4)39 (19.1)48 (23.5)99 (48.5)134.137/0.000
Q366 (2.9)17 (8.3)58 (28.2)75 (36.4)50 (24.3)80.748/0.000
TECHNICAL ASPECTS
Q3710 (4.9)19 (9.2)54 (26.2)64 (31.1)59 (28.6)59.874/0.000
Q3829 (13.9)27 (13.0)66 (31.7)49 (23.6)37 (17.8)25.077/0.000
Q3957 (27.5)29 (14.0)71 (34.3)28 (13.5)22 (10.6)44.184/0.000
Q4019 (9.1)23 (11.1)66 (31.7)43 (20.7)57 (27.4)40.654/0.000
QUALITY OF LIFE
Q4149 (23.6)25 (12.0)49 (23.6)41 (19.7)44 (21.2)9.404/0.052
Q4219 (9.1)10 (4.8)36 (17.3)41 (19.7)102 (49.0)124.740/0.000
Q4315 (7.2)15 (7.2)48 (23.2)36 (17.4)93 (44.9)99.739/0.000
Q4468 (32.9)29 (14.0)46 (22.2)37 (17.9)27 (13.0)26.792/0.000
Q4562 (30.0)25 (12.1)53 (25.6)30 (14.5)37 (17.9)23.604/0.000
Table 4. The total score for each factor of eMedQ. The score for each factor was obtained by summing the points of all questions that are classified into a certain factor.
Table 4. The total score for each factor of eMedQ. The score for each factor was obtained by summing the points of all questions that are classified into a certain factor.
FactorMean ± SDMedian (Min–Max)
Factor I40.48 ± 13.12 40 (14–68)
Factor II38.36 ± 7.8040 (15–50)
Factor III21.34 ± 6.2822 (7–35)
Table 5. Comparison of each variable related to the factor. Statistically analysis was conducted by using Mann–Whitney U test or Kruskal–Wallis test with the significance level of 0.05.
Table 5. Comparison of each variable related to the factor. Statistically analysis was conducted by using Mann–Whitney U test or Kruskal–Wallis test with the significance level of 0.05.
CharacteristicFactor I
Median (IQR)
pFactor II
Median (IQR)
pFactor III
Median (IQR)
p
Gender *
Male41.5 (18)0.70838.0 (12)0.64420.5 (10)0.219
Female39.5 (21)40.0 (12)22.0 (9)
Study program **
IASM34.0 (21)0.00140.0 (18)0.00620.0 (9)0.000
IASF45.0 (20)42.0 (11)23.5 (8)
IASS36.5 (15)36.5 (14)18.5 (9)
OSS38.0 (17)37.0 (9)20.0 (7)
Year of study **
I38.0 (14)0.00544 (11)0.00421.5 (10)0.006
II40.0 (20)39.0 (10)21.0 (9)
III47.0 (23)39.0 (9)21.0 (5)
IV52.0 (23)43.0 (9)24.0 (6)
V40.0 (17)38.0 (16)20.0 (7)
VI29.5 (20)31.5 (14)17.0 (10)
Level of skills in using electronic devices (computers, smartphones, tablets…) **
Inadequate33.5 (NA)0.54532.5 (NA)0.10213.5 (NA)0.249
Acceptable35.0 (14)39.0 (13)19.0 (10)
Good42.0 (14)39.0 (12)22.0 (8)
Very good38.5 (22)38.0 (11)21.0 (9)
Excellent38.5 (27)42.5 (13)22.0 (10)
Experience with online education before COVID-19 **
Very great experience46.0 (26)0.12940.0 (13)0.27726.0 (8)0.074
Great experience43.0 (23)37.0 (12)21.5 (12)
Moderate experience42.0 (15)42.5 (9)21.5 (10)
Little experience38.0 (21)40.0 (12)21.0 (10)
Without any experience37.0 (18)38.5 (14)21.0 (8)
Progress during the online education during the COVID-19 pandemic **
Very much45.5 (23)0.01143.5 (9)0.00025.0 (9)0.000
Much44.5 (21)41.0 (10)23.0 (6)
Moderate38.0 (18)39.0 (11)21.0 (8)
Little32.0 (11)35.0 (12)18.0 (8)
I have not progressed42.0 (19)30.0 (11)17.0 (9)
Form of online teaching experienced during the COVID-19 pandemic
Zoom/Teams/Webex/Google Meet meeting *
Yes41.0 (21)0.02140.0 (12)0.08420.0 (8)0.046
No38.0 (15)36.0 (14)22.0 (8)
Pre-recorded videos *
Yes36.0 (21)0.10437.0 (14)0.21322.0 (9)0.944
No42.0 (20)40.0 (11)21.0 (8)
Digital platform (Moodle, etc.) *
Yes39.0 (23)0.99536.0 (14)0.19522.0 (10)0.597
No40.0 (20)40.0 (10)21.5 (9)
Presentation with narration *
Yes38.5 (20)0.96340.0 (14)0.99322.0 (9)0.580
No42.0 (22)40.0 (11)21.0 (9)
Short online consultations in writing (chat consultations) *
Yes46.0 (33)0.88144.0 (25)0.74124.0 (14)0.544
No40.0 (20)40.0 (11)21.5 (9)
Test questions *
Yes42.5 (23)0.26240.0 (13)0.29122.0 (6)0.384
No40.0 (20)40.0 (11)21.0 (10)
Further improvement of the current form of online teaching *
It is necessary37.0 (19)0.00637.0 (12)0.00021.0 (8)0.017
It is not necessary44.0 (21) 43.0 (9)23.0 (10)
Online educational modalities that would significantly improve the acquisition of practical knowledge and skills in biomedical sciences *
Virtual Classroom
Yes42.0 (23)0.96642.0 (12)0.09321.0 (9)0.170
No39.0 (21)39.0 (12)22.0 (7)
System Simulations of practical skills *
Yes37.0 (18)0.24939.0 (12)0.79821.0 (9)0.304
No42.5 (24)40.0 (12)22.0 (9)
Educational Games/Gamification *
Yes42.0 (22)0.95635.0 (14)0.09321.0 (8)0.443
No39.0 (21)40.0 (11)22.0 (9)
Clinical Scenarios, Virtual Patients, Clinical Vignettes *
Yes38.0 (21)0.99540.0 (13)0.59122.0 (9)0.811
No41.5 (22)39.0 (13)21.0 (8)
Multimedia content/Educational multimedia streaming *
Yes48.0 (18)0.02240.0 (10)0.28023.0 (10)0.092
No38.0 (19)39.0 (12)21.0 (9)
Area in which it would be useful to have additional teaching materials compared to existing online materials **
Preclinical subjects 38. 0 (15)0.27237.0 (14)0.31921.0 (9)0.103
Subjects of clinical medicine, clinical pharmacy, and clinical dentistry39.0 (22)40.0 (11)21.5 (8)
Other subjects:42.0 (21)40.0 (11)23.0 (12)
* Mann–Whitney U test; ** Kruskal–Wallis test.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Nikolic Turnic, T.; Mijailovic, S.; Nikolic, M.; Dimitrijevic, J.; Milovanovic, O.; Djordjevic, K.; Folic, M.; Tasic, L.; Reshetnikov, V.; Mikerova, M.; et al. Attitudes and Opinions of Biomedical Students: Digital Education Questionnaire. Sustainability 2022, 14, 9751. https://doi.org/10.3390/su14159751

AMA Style

Nikolic Turnic T, Mijailovic S, Nikolic M, Dimitrijevic J, Milovanovic O, Djordjevic K, Folic M, Tasic L, Reshetnikov V, Mikerova M, et al. Attitudes and Opinions of Biomedical Students: Digital Education Questionnaire. Sustainability. 2022; 14(15):9751. https://doi.org/10.3390/su14159751

Chicago/Turabian Style

Nikolic Turnic, Tamara, Sara Mijailovic, Maja Nikolic, Jelena Dimitrijevic, Olivera Milovanovic, Katarina Djordjevic, Marko Folic, Ljiljana Tasic, Vladimir Reshetnikov, Maria Mikerova, and et al. 2022. "Attitudes and Opinions of Biomedical Students: Digital Education Questionnaire" Sustainability 14, no. 15: 9751. https://doi.org/10.3390/su14159751

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop