The Contribution of the Flipped Classroom Method to the Development of Information Literacy: A Systematic Review

: With the arrival of the Internet and information and communication technologies (ICTs), today’s society has driven many changes in which the digital channel is one of the main information consumption channels for young university students. Due to the turbulent landscape full of fake news and confusing information we have entered, there is a need to promote information literacy among quality students. To this end, the use of innovative strategies and methods such as Flipped Classroom can be a novel alternative to receive this content. Based on these ideas, the objective of this work was framed in the methodological design corresponding to the systematic review. Thus, empirical work was compiled to put into practice the Flipped Classroom methodology and to promote informational teaching from di ﬀ erent knowledge disciplines at the university level. The results showed that the application of the methodology is multivariate, and that it can be administered to students in various ways (video-tutorials, animated videos, training modules, etc.) depending on the creativity of the teacher. Likewise, the results compiled in the review turn towards the improvement of the students’ informational skills, as well as the increase in diverse personal and academic variables.


Introduction
In recent years, the demands on future teachers in the face of the multitude of changes that the current information society is experiencing are continually increasing. With the arrival of information and communication technologies (ICTs), today's young people have incorporated new habits when it comes to the consumption of information. Currently, the Internet is the source of reference for young people, and in many cases, the only one [1,2]. This has led to the use of this channel for disseminating fake news, a phenomenon that has gradually gained prominence and whose appearance has caused a negative impact on various spheres of life, by producing a distortion of reality. Likewise, this action has led to a controversial social atmosphere and a feeling of confusion among citizens [3,4].
In consideration of this situation, the education system shall promote a line of action that allows young people to develop a critical perspective on the process of selection, evaluation and information management [5].
In response to this, the development of digital competence has become one of the most demanded educational challenges by the educational community. It is necessary to train current and, above all, future generations to develop a minimum of digital skills so that they can safely navigate the web, extract information from it, and share it safely [6]. In this sense, digital training constitutes one of the lines of action for the promotion and fulfilment of the objectives proposed by the Agenda 2030 for Sustainable Development [7]. Specifically, objective number 4 responds to the purpose of improving the quality of education, especially by updating the processes of teaching and learning through the the web, extract information from it, and share it safely [6]. In this sense, digital training constitutes one of the lines of action for the promotion and fulfilment of the objectives proposed by the Agenda 2030 for Sustainable Development [7]. Specifically, objective number 4 responds to the purpose of improving the quality of education, especially by updating the processes of teaching and learning through the integration of technology and providing students with digital and informational skills. To this end, it is necessary to promote training that helps to face the technological change that society, and therefore the educational system, is undergoing. In the face of this challenge, digital competence, and more specifically, information literacy, is presented as a compendium of skills that every citizen must possess in order to create sustainable generations that interact smoothly on the digital network and are able to manage the wide range of information it presents [3].
This training takes as its main reference the following conceptual framework: The European Digital Competence Framework for Citizens [8,9] (DIGCOMP). The digital competence is distinguished by five dimensions that structure the concept: 1: Information and Information Literacy; 2: Communication and Collaboration; 3: Digital Content Creation; 4: Digital Security; 5: Problem Solving. Specifically, when reference is made to information literacy, the specialized literature refers to the ability to "identify, locate, obtain, store, and organize digital information, data and digital content, assessing their purpose and relevance for teaching tasks". Likewise, there are complementary definitions that understand this concept as the ability to think critically and give reasoned opinions on any information we find and use [10] (Figure 1). In relation to this concept, the Association of College & Libraries [11] claims that informational teaching is common to all disciplines, all learning contexts and all levels of education. Developing an optimal level of these skills would allow students to better understand content, to expand their searches, to become more autonomous, to promote critical development and to exercise greater control over their own learning.
The ability to judge whether the information obtained is reliable or not is a competence that younger generations do not fully possess. This is why it is a challenge that remains to be completed, especially from the Higher Education stage [12]. In relation to this concept, the Association of College & Libraries [11] claims that informational teaching is common to all disciplines, all learning contexts and all levels of education. Developing an optimal level of these skills would allow students to better understand content, to expand their searches, to become more autonomous, to promote critical development and to exercise greater control over their own learning.
The ability to judge whether the information obtained is reliable or not is a competence that younger generations do not fully possess. This is why it is a challenge that remains to be completed, especially from the Higher Education stage [12].
In this sense, the curricula at this stage have changed in order to provide students with the essential skills to carry out a process of familiarization of digital information. Thus, and based on what is established by the European Higher Education Area (EHEA), current students and future professionals need to acquire informational skills, regardless of the degree they are studying, as it is a Sustainability 2020, 12, 7273 3 of 13 transversal skill [13]. In this sense, the Assessment and Teaching of 21st Century Skills (ATC21S) need to be taken into account, which advocates the need to acquire these skills in order for young people to succeed in a rapidly changing digital society [14].
Similarly, there are also studies in which digital competence, and more specifically, levels of information literacy in different sectors of learners, are assessed [15][16][17]. The results of these studies stated that there is still a need to improve the training plans in this area in order to increase the skills of the students in this subject in question.

The Effectiveness of the Flipped Classroom Method
In recent years, the education system has been in a phase of constant transformation, in which multiple changes have broken through regarding how we understand the concept of education until then. Today's students are characterized by different skills, interests and motivations than those of previous years. This implies that, from the education sector, a point of adoption shall be established for the new interests and behaviors arisen by the current students.
Based on these ideas, there are many active methodologies that have recently emerged. The understanding of those provides a leading role to the student, who is able to learn by himself, thanks to the role of mediator and facilitator of content that occupies the teacher [18]. Among these ideas is where the Flipped Classroom or inverted classroom is presented: a pedagogical model in which a traditional learning environment and its activities are reformed. In this case the teaching/learning process takes place in two phases: (1) Outside the classroom, where students must watch videos on the subject, and; (2) when the student is in the classroom, a period dedicated to resolving doubts and to carry out activities and tasks of assimilation and consolidation [19].
The application of this model implies a drastic change in the temporal and spatial organization that we have had until now. However, this is a necessary change that makes us forget the traditional expository teaching, which placed the students in a passive position, to give place to a way of teaching in which the team works, and the sharing of contents and practical application have a greater place [20].
Nowadays, the Flipped Classroom approach can accept multiple variants, ranging from teaching and learning entirely in the classroom to fully online. There is the case of Flipped Classroom dynamics that uses videoconferences, interactive readings or video-tutorials instead of a video recorded by the teachers themselves [21]. The purpose entrusted with the proposal of this methodology is to facilitate the continuous, supervised, experiential learning that promotes the autonomy of the student and the construction of knowledge [22].
In this sense, the application of Flipped Classroom has constituted a novel alternative for numerous teachers from different branches of knowledge, in which after the implementation with their students, they have observed different academic variables, such as performance, motivation, autonomy or self-regulation towards the learning they have benefited from [23][24][25][26].
On the other hand, regarding the application of this method, Flipped Classroom constitutes an alternative and emerging method, in which several teachers are elucidating a dynamic and novel possibility of being able to teach informational-digital competences [27].
In relation to previous studies in line with the present one, the scientific literature provides both studies linked to the analysis and the systematic paradigm on the use of the Flipped Classroom methodology [28] and its influence on variables such as the academic performance [29], motivation and satisfaction of the students [30], or their use within different branches of knowledge, such as education [31] or health [32,33]. In the case of information literacy, there are some review studies which, as referred to above, relate the concept of information literacy to that of digital competence [34,35], in which the need to promote learning these skills in the Higher Education level is determined, with the aim of creating professionals who are critical of the information that surrounds them.
Based on this paradigm, the main objective of the present piece of work was to locate quality educational experiences that applied the Flipped Classroom method to promote information literacy in Higher Education classrooms. Based on these objectives, the following research questions were proposed: RQ1. What are the main experiences from the international context in which the Flipped Classroom method is being implemented to achieve information literacy development?
RQ2. In what areas or disciplines of knowledge do these types of studies redound? RQ3. Which journals have published scientific papers on this subject? RQ4. What was the objective of implementing the Flipped Classroom methodology? RQ5. How was the implementation of the methodology carried out? RQ6. What effects have I observed on the students analyzed and on their information competence?

Method
Based on the above ideas, the present piece of work was framed within the method of systematic literature review, conceived as one which analyzes information provided to generate an overview of a given object of study, specifically on the information provided in scientific databases or reports, which compile data on publications [36]. This type of research allows us to categorize the findings to date on the subject, as well as to quantify the data based on different criteria established on those relevant aspects that need to be pointed out [37,38].
To this end, the methodological process consists of a series of steps that go from the definition of the scope to the classification of the data obtained. For this piece of work, the proposed phases were followed in the PRISMA statement (Preferred Reporting Items for Systematic reviews and Meta-Analyses) [39].
The review process carried out was divided into two phases: Planning: This protocol consisted in defining the research questions, the inclusion and exclusion criteria and the establishment of descriptors and databases from where the scientific documents would be collected.
Action: We proceeded to locate references in the selected databases, to refine the data through filters and extract the information to finally perform the data representation. With regard to the formulation of the inclusion and exclusion criteria, the following were configured based on the objectives of the study and the indications contained in the PRISMA statement.

Search Strategy
The search for scientific publications took place in the databases Web of Sciences (WoS) and Scopus. These repositories were chosen because of their potential and international prestige, as well as the quality standards they use for indexing their articles [40]. In the case of Web of Sciences, the search was carried out in the indexes Social Sciences Citation Index (SSCI), Science Citation Index Expanded (SCIE) and Arts and Humanities Citation Index (AHCI). To do this, the search equation was used, which is composed of the following descriptors "Flipped Classroom OR Flipped Learning", and "Information literacy OR informational literacy".
The descriptors were applied in the search engine of both databases for further filtering. For this purpose, a series of inclusion and exclusion criteria were established to limit the study sample ( Table 1). The reason for which only scientific articles were used was due to the peer-review process involved in order to be published, which guarantees the necessary rigor to the study. As for the rest of the criteria, we included them as a reference to previous systematic reviews of great importance in the educational field [33,40,41].
To avoid bias in the study selection, apart from other systematic review works [41], two researchers undertook the systematic review using the same descriptors and the inclusion and exclusion criteria. The degree of agreement on the inclusion of articles was of 95%. The disagreement was addressed by a third researcher who chose to include 100% of the extracted scientific literature.

Procedure
Taking the Prism Declaration as a reference [39], the literature selection process was carried out distinguishing four phases. The first phase, called "Identification", which consisted of applying the equation of searching for documents in the databases, and filtering the search in scientific articles (IC1, EX1) in English or Spanish (IC3, EX3), obtaining a total of 88 documents (WoS = 35; Scopus = 53). Consequently, at the revision phase, a large part of the inclusion criteria (IC2, IC4, IC5) and the exclusion criteria (EX2, EX4, EX5) were applied. Finally, the duplicate articles were eliminated so that the "included" phase was finally the result of the application of multiple criteria throughout the refining process (n = 11).
In summary, the flow chart shows the process until the conformation of the definitive sample followed and the scrutiny of the scientific articles ( Figure 2).

Data Analysis Procedure
In order to answer the research questions posed previously, an analysis of the diachronic productivity of the selected manuscripts was first established, with the aim of observing the upward or downward trend in productivity. After this, the focus was established on investigating the journals in which these articles were published and their origin, as well as the branch of knowledge in which the research can be framed and the origin of the authors, in order to determine whether the line of research could be considered a global one. After this, the research objectives pursued by the researchers with the implementation of Flipped Classroom in their classrooms were analyzed. In this way, the goal was to observe the possible multifunctionality of the methodology according to the branch of research and the teacher who carried it out. Finally, we investigated how the Flipped The data analysis was carried out by a Review Manager V5.3. Software (Copenhagen, Denmark), in which each of the documents that made up the final sample were stored to be examined through a content analysis [42].

Data Analysis Procedure
In order to answer the research questions posed previously, an analysis of the diachronic productivity of the selected manuscripts was first established, with the aim of observing the upward or downward trend in productivity. After this, the focus was established on investigating the journals in which these articles were published and their origin, as well as the branch of knowledge in which the research can be framed and the origin of the authors, in order to determine whether the line of research could be considered a global one. After this, the research objectives pursued by the researchers with the implementation of Flipped Classroom in their classrooms were analyzed. In this way, the goal was to observe the possible multifunctionality of the methodology according to the branch of research and the teacher who carried it out. Finally, we investigated how the Flipped Classroom method was carried out and which resources were used to put it into practice among the different investigations that made up the systematic review.

Results
In the first instance, the studies were grouped according to the year of publication ( Figure 3). It should be noted that the vast majority of them were published in 2018, followed by three papers in 2016, and one each in 2014, 2015 and 2019.

References Journal
Country h-Index [43] Communications in Information Literacy United States 14 [44] Journal of the Medical Library Association United States 57 [45] Pixel-Bit. Revista de Medios y Educación Spain 2 [46] Journal of Information Literacy United Kingdom 9 [47] Journal of Information Literacy United Kingdom 9 [48] Journal of Information Literacy United Kingdom 9 [49] Journal of the Medical Library Association United States 57 [50] Journal of Information Literacy United Kingdom 9 [51] Australian Academic & Research Libraries Australia 25 [52] The Journal of Academic Librarianship United Kingdom 55 [53] Communications in Informational Literacy United States 14 In terms of the discipline of knowledge to which the analyzed studies belonged (Table 3), they mainly belonged to the branch of knowledge "Information Science and Library Science" followed by "Computer Sciences" with "Medical Informatics" and "Educational Research". As for the institutions from which the authors come, they are varied, a large part of them coming from the United States. However, it is necessary to appreciate the territorial variety of the institutions of origin, coming from different countries and continents (Europe, America, Asia and Oceania). On the other hand, taking into account the journals that are waiting for the scientific papers (Table 2), it can be seen that the papers were published in different informative platforms. The Journal of Information Literacy stands out, containing a total of four scientific papers. As for the origin of the journals, it can be seen that there is homogeneity, as most of them belong to either the United States or the United Kingdom. Finally, the h-index reflects the level of citation acquired per published article held by each journal. In this case, the Journal of the Medical Library Association and the Journal of Academic Librarianship are those that acquire a higher level of citation.
In terms of the discipline of knowledge to which the analyzed studies belonged (Table 3), they mainly belonged to the branch of knowledge "Information Science and Library Science" followed by "Computer Sciences" with "Medical Informatics" and "Educational Research". As for the institutions from which the authors come, they are varied, a large part of them coming from the United States. However, it is necessary to appreciate the territorial variety of the institutions of origin, coming from different countries and continents (Europe, America, Asia and Oceania). Table 2. Journals to which the works belong and country of origin.

References
Journal Country h-Index [43] Communications in Information Literacy United States 14 [44] Journal of the Medical Library Association United States 57 [45] Pixel-Bit. Revista de Medios y Educación Spain 2 [46] Journal of Information Literacy United Kingdom 9 [47] Journal of Information Literacy United Kingdom 9 [48] Journal of Information Literacy United Kingdom 9 [49] Journal of the Medical Library Association United States 57 [50] Journal of Information Literacy United Kingdom 9 [51] Australian Academic & Research Libraries Australia 25 [52] The Journal of Academic Librarianship United Kingdom 55 [53] Communications in Informational Literacy United States 14 Finally, with reference to the implementation of Flipped Classroom, Table 4 shows the development of each of the studies that made up the study sample. Practically the majority of the scientific works obtained positive results in favor of the application of Flipped Classroom in the different teaching rooms, affecting the increase in personal and academic variables such as motivation and interest towards the content or academic performance. On the other hand, with regard to the implementation of the methodology, it was observed that there are different ways to apply it, such as the use of video tutorials [43], animated videos [48], training modules distributed through learning management system (LMS) platforms [49,52], interactive presentations through Power Point [46] or the elaboration of video guides [51] for the use of academic libraries. Finally, with respect to the nature of the research, a high degree of heterogeneity is observed, with research of quasi-experimental design (with application of pre-test and post-test), qualitative (through interviews, semi-structured interviews or discussion groups), quantitative (through surveys, rubrics or evaluation tests) and mixed (applying surveys and interviews). Table 4. Development of the collected scientific works.

Reference
Objective Flipped Classroom Implementation Instrument Effect of Flipped Classroom Application [43] To promote the improvement of information skills to encourage better research projects The application of video tutorials and quizzes Survey Students improved their skills for finding and evaluating information sources, increased their satisfaction, autonomy and interactivity. [44] To encourage the improvement of information skills and critical thinking 6 self-made flipped videos (duration: less than 10 min)

AACU Information Literacy Rubric
Students improved in the scientific search of sources of information and learnt about databases (PubMed). [45] To develop the level of digital competence of future teachers through FC Analysis of external videos and reading of articles; discussions, debates and reflections through a digital blog

Interview
Students improved their knowledge about the dimensions of digital teaching competence and their interest presented towards the subject arose. [46] To determine the learners' perceptions and expectations regarding the flipped teaching model, and to determined what can be learned from a learner's response to a pre-class PowerPoint with personalized and active elements Training in information literacy through interactive PowerPoint presentations; in-class discussions and group work

Interview
Despite the students' interest in the FC model, there were moments of confusion during the development of the experience. An increase in the rates of collaboration between the students was observed. Students perceived a more personalized learning. [47] To check whether students experience an improvement in scientific information-seeking skills Training modules through long videos (15 min onwards); the training was complemented by lectures and seminars; evaluation through tests

Interviews and one essay
Students perceived an improvement in the knowledge of scientific search skills. Students valued the application of CF positively. However, there was no correlation between students' attitudes and the evaluation of the trials. [48] To promote a change of the library through the FC methodology to encourage an improvement of the information literacy Animated videos (less than 7 min) were produced to transmit the knowledge; an interactive "pet" of the University was used as an introductory element of the informational contents

Not specified
The students rated the videos very positively, reiterating their viewing on several occasions. They considered the option of making animated videos to be a fun way to learn. They also valued their perception of the usefulness of the library. [49] To determine if FC would improve the ability to find, evaluate, and use appropriate evidence for research tasks Training modules distributed through an LMS platform; use of videos, streams and textual information; quizzes were completed in class with small group presentations on the contents, and the first two teams were rewarded AACU Information literacy Rubric/Semi-structured interviews Slight improvement in information capabilities, in the value given to the library as a collaborative network.  [50] To test the improvement of informational knowledge through quasi-experimental design Prior to the practical activities, students watched a video (7 min) and performed subsequent tasks outside of class; these same activities were done by the control group in class, as well as the practical part Pre-test and post-test The experimental group scored better than the control group. In particular, it was noted that students in this set were more proficient in key concepts (visualized through the videos) than students in the control group. This made them perform better when doing the practical activities. [51] To encourage improved information and academic literacy of students through an upside-down library model Personal guidance through the library was replaced by a Flipped self-guide that students could perform autonomously; quizzes were designed to motivate the students to take the tour

Survey
Students improved their search and information management skills through the flipped library model. This increased interactivity among students and confidence in the library as a learning resource. [52] To examine the needs linked to information literacy and the perception of the learners about the training module provided through FC Viewing short videos, completing quizzes, tests, evaluation tasks and online exams; all managed through an LMS platform Preliminary needs analysis/survey and discussion groups Improved confidence in interacting with databases and locating external information sources. An increase in self-regulation towards learning was observed. [53] To determine if through the application of FC there are significant differences in the search of academic sources Training video tutorials; use of a control and experimental methodology Pre-and post-test No significant differences on search skills improvements were detected between one group and another.

Discussion
The emergence of active methodologies that incorporate technology has had an impact on the way we understand teaching. The emergence of methodologies such as Flipped Classroom has caused different teaching professionals to incorporate into their educational practice this way of understanding the teaching-learning process. As a result of this, the present piece of work aimed to configure a systematic review of educational experiences in which the Flipped Classroom method was used to teach contents related to information literacy.
In this way, the results of the systematic review carried out have allowed us to establish an approach with the state of the art regarding the application of this method at the time of teaching informational competencies in different branches of knowledge. Firstly, the initial results indicated that this is a global, current line of research, as well as a practice common to different branches of knowledge. Especially, and in line with the main theme, the majority of pieces of work came from the field of documentation and communication, although more and more studies from other fields such as education and health sciences are emerging in the scientific literature [31,33].
Consequently, the review carried out on the articles that made up the sample showed that the concept around the Flipped Classroom is very varied. Currently, there are multiple conceptions of this methodology and various ways of applying it [20]. Thus, studies were identified that used the traditional concept (self-made videos where the teacher explains in it), however, it also results in another type of implementation, through video-tutorials, interactive presentations and the distribution of videos through training modules. This is undoubtedly a positive aspect in favor of this methodology, which does not restrict creativity and openness to new ideas and which is therefore completely adaptable to the teacher's ideas and proposals.
On the other hand, concerning the effectiveness of its implementation around the development of information literacy, the results collected in the articles that made up the review were positive, in which it was observed not only how the students' information skills improved (especially linked to the search, evaluation and management of information sources), however, as indicated in previous studies, other types of personal and academic variables were increased, such as their motivation indexes, autonomy, self-regulation towards learning, interactivity among peers or interest towards the content or subject taught [23,24,29,30]. There were few studies that showed the ineffectiveness of this methodology. Based on these premises, it could be stated that the application of Flipped Classroom favors the learning of contents associated to the phenomenon of information literacy and coincides with the point of view expressed in previous studies [34,35] to continue promoting literacy as a transversal and common field to all areas of knowledge.
Therefore, from the findings of the present piece of work, it is possible to verify the effectiveness of the application of active methodologies in promoting the information literacy in university students. Thus, these results show how the integration of technology in the teaching-learning processes is promoting the improvement of educational quality, and therefore, the promotion of sustainable generations at an educational, digital and informational level, as the objectives of Agenda 2030 are being pursued [7].

Conclusions
The development of information literacy is postulated as one of the priority challenges to be met by current society. It is a transversal discipline that every citizen shall possess in order to live in the information society in which we find ourselves. For this reason, it is necessary that universities implement a quality practice that gives significance to the teaching of this compendium of skills. In view of this, the appearance of the Flipped Classroom method is presented as a dynamic and innovative alternative for students to work on the various contents and skills that make them up.
From this piece of work, we tried to establish a state of the question about the implementation of the Flipped Classroom in the work of informational skills, in order to be able to elucidate to the educational and scientific community what is the current line of practice and research of this branch of research. The findings of the work allowed us to know that the teaching of information literacy constitutes transversal content common to several branches of knowledge, as well as the implementation of Flipped Classroom, which is used by different professionals in very varied ways.
In reference to the limitations of this work, we observed that we could not include the revision of those articles that are not in open access, which would have added a greater quantity to the final sample, and therefore, would have helped to analyze the existing literature even further. On the other hand, as future lines of research, the need to continue deepening this thematic line is advocated through the proposal of empirical studies that verify the effectiveness of the Flipped Classroom method in the development of informational skills and the subsequent meta-analytical studies that can establish a reliable opinion on the optimal performance of the implementation of this methodology when developing this topic.
In conclusion, the teaching of information skills must provide new variants that are motivating and attractive to students, so that they acquire learning in an active and meaningful way. To this end, it will be essential that teachers are in a process of continuous training, which allows them to be updated on the latest educational trends and consequently, can offer their students attractive and functional opportunities to learn knowledge.