Educational Challenges of Higher Education: Validation of the Information Competence Scale for Future Teachers (ICS-FT)

Abstract

The irruption of information and communication technologies has brought about an abrupt change in the demands placed on future professionals. In this sense, in recent years, information competencies have gained importance in university education from a cross-cutting perspective, which advocates as its main purpose the training of young people in information search, evaluation, processing and communication skills, especially through a digital network. Based on this idea, the present work aims to develop the Information Competence Scale for Future Teachers (ICS-FT), in order to measure the level of self-perceived skills in this area. For this purpose, a work design is presented below concerning an empirical validation, divided into different phases: the validation of content, construct and reliability, which has taken as a pilot sample a total of 259 university students studying the degree of Primary Education. The results of the validation determined the optimal conditions of content, construct and reliability that allowed the application of this scale as a generic approach to determine the level of competence in the information skills of future teachers.

1. Introduction

In recent years, the irruption of technology has monopolized a compendium of modifications that have produced an abrupt change in the way we understand daily processes. Specifically, since its emergence, the Internet has become one of the most major pastimes used by the population, and more specifically, when exercising both generic and advanced information queries and searches [1,2]. This is due, to a greater extent, to the number of possibilities offered by this resource and the multitude of variants it offers the user with just a “click” [3]. As a consequence, from the educational landscape, a substantial paradigm shift is taking place, reorienting teaching/learning processes and reconsidering the role of students, teachers, future demands and demands that revolve around these factors [4].

In this characteristic context of the information society in which we find ourselves, the need for future professionals to possess information competencies is a priority common to all disciplines, all learning environments and all levels of education [5]. Its acquisition is already justified in different educational plans, and its correct development encourages university students to have a greater understanding of knowledge, in order to possess a greater metacognitive capacity and, in short, to assume greater control over their own learning process [6].

The “Conferencia de Rectores Universitarios Españoles of ICT” (Crue-ICT) Joint Commission and Rebiun [7] define information competencies as “the set of knowledge, skills, attitudes and behaviors that enable individuals to recognize when they need information, where to locate it, how to evaluate its suitability, and how to use it appropriately according to the problem at hand” (p. 6). In view of the constant change that revolves around the current context, the relevance of training in information skills, as well as the general promotion of information literacy in the population, is presented as one of the challenges that contemporary society must face in educational terms [8]. Especially, this skill becomes more relevant in different education professionals, whose objectives include training their students to have a critical spirit and responsible attitude towards the functions of locating, evaluating, configuring and communicating information [9,10].

This term is closely related to that of digital competence in teaching. The development of digital competence has become one of the most demanding educational challenges faced by the educational community. In this sense, digital training constitutes one of the lines of action for the promotion and fulfilment of the objectives proposed by the Agenda 2030 for Sustainable Development [11]. This training takes as its main reference the following conceptual framework: The European Digital Competence Framework for Citizens [12] (DIGCOMP). The digital competence is distinguished by five dimensions that structure the concept: 1—Information and Information Literacy; 2—Communication and Collaboration; 3—Digital Content Creation; 4—Digital Security; and 5—Problem Solving. Specifically, when reference is made to information literacy, the specialized literature refers to the ability to “identify, locate, obtain, store, and organize digital information, data and digital content, assessing their purpose and relevance for teaching tasks”. Likewise, there are complementary definitions that understand this concept as the ability to think critically and give reasoned opinions on any information we find and use [13]. As far as the training of future teachers is concerned, the latest research shows low levels of information literacy skills [14,15], content creation [16,17], or skills in informational and digital communication and collaboration [18,19].

Upon reaching university, students require basic training to interact with the informational process, since the development of their skills is not enough to be able to locate, access, retrieve, evaluate, and critically use information autonomously [20]. Throughout their university careers, there are many situations in which these competencies are required, and not having adequate training leads to unsatisfactory results, and thus to a deficit in this type of knowledge. Therefore, the justification of this concept in educational terms is agreed upon regarding the importance of achieving an integration in the university curricula of this discipline. This is considered common and transversal to all formative degrees in any type of subject [21] in order to promote a comprehensive learning that augurs a better and greater professional development [22]. In this sense, Cortés et al. (2004) cited by Gallardo and Lau [23], point out a series of priority informational skills to work on during the university stage:

(a)

Understanding of the structure of knowledge and information: identifying the information cycle (generation, processing, organization and dissemination).

(b)

Determine the nature of an information need: be able to identify and communicate your information needs.

(c)

Develop effective strategies to search for and find information: carry out an orderly search process to ensure success in obtaining information.

(d)

Information retrieval: develop strategies for effective information retrieval from different sources.

(e)

Analyze and evaluate the information: determine the scope and depth of the information.

(f)

Integrate, synthesize and use information: incorporate previous knowledge and make the corresponding transfers in different disciplinary fields.

(g)

Presenting the results of the information obtained: understanding the information obtained and being able to express it adequately.

(h)

Respect intellectual property and copyrights: behave ethically in the use and application of information.

Thus, there are several studies that analyzed the level of informational skills in different university students. From the implementation of intervention programs, substantial improvements were encouraged in aspects such as formal data searches in official [24,25] and informal repositories [26]; in the writing of academic papers [27], the citations and referencing of scientific sources [28,29]; critical thinking and autonomous learning [30,31,32]; quality assessment of electronic sources [33]; or in shaping new scholarly elaborations and their communication [34]. Accordingly, the benefits of working on these competencies within university education are noted [35].

With regard to instruments related to the evaluation of self-efficacy in these competencies, it is necessary to highlight those that were referents when configuring the competencies established in this work. (i) Firstly, there is the IL-HUMASS 21 scale [36], which was designed for a population of students, librarians and teachers based on four dimensions (information seeking, evaluation, processing and communication/dissemination) and three self-report dimensions (motivation, self-efficacy and favorite source of learning). (ii) Secondly, there is the information literacy self-efficacy scale (ILSE), developed by Kurbanoglu et al. [37], which measured these competencies in terms of seven basic skills: (a) Defining the need for information; (b) Initiating the search strategy; (c) Locating and accessing resources; (d) Evaluating and understanding information; (e) Interpreting, synthesizing and using information; (f) Communicating information; and (g) Evaluating the product and the process. Undoubtedly, these are two rigorously configured instruments that were used in multiple investigations and taken as references for the elaboration of the questionnaire used in the present work.

With regard to the justification of this instrument, its configuration is intended to be useful for students who are in the process of becoming future teachers in order to self-evaluate their level of information competencies. To this end, the wording of the items is simple, and the number of items configured is brief, following the recommendations of experts [38,39] regarding the development of questionnaires aimed at a young population, in order to obtain more rigorous results.

Therefore, the main objective of this work is to construct and empirically validate a self-perception instrument for the development of the information competencies of university students.

2. Materials and Methods

The present study was developed following a cross-sectional, correlational, quantitative design [40], framed in the validation of an ad hoc scale through a content and internal consistency analysis [41]. To this end, the study was divided into different phases (Figure 1) from its initial configuration to the final drafting of the instrument and subsequent statistical validation [42]. Therefore, two large stages can be distinguished. The first is linked to the process of developing the instrument and the second to its validation.

2.1. Sample

A total of 259 university students studying for a degree in Primary Education at the University of Granada (Spain) participated in the pilot study. The age range was between 18 and 15 years, with a mean age of 23.29 years (SD = 3.49), of which 63.3% were female and 36.7% male. A greater number of women were included in the study due to the fact that, generally, for teacher training degrees, the female population is more prevalent [43]. The sample selection procedure was based on a non-probabilistic or convenience sample [44]. On the other hand, the sample size was greater than 200 cases, above the minimum threshold established in different simulation studies to set up structural equation models [45]. See the details at Appendix A and Appendix B.

2.2. Instrument Development and Content Validation

Firstly, an exhaustive review of the literature on the topic in question was carried out. From this point, in accordance with Wilson [46], the operational definition of information competencies was drafted. In this case, the definition expressed by the (CRUE-ICT) Joint Commission and Rebiun [7] was taken as the main definition. Next, the drafting of the scale items took place. Throughout, the main priority was that the items were simple, and specific references to their configuration were followed [42,47]. Likewise, the type of scale chosen in this case was Likert type 7 with a frequency response type (1 = never/7 = always).

Consequently, content validation was carried out through expert judgments [38]. The expert judges were asked to evaluate, in a total of two rounds, different aspects of the measurement scale, the questionnaire items, and an overall assessment of the questionnaire [39], as well as the pilot scale presented in this work. Therefore, the overall assessment was made up of a qualitative analysis (analyzing the degree of comprehension, adequacy of the wording according to the target population, etc.) and a quantitative assessment (scale 1 to 10), in order to evaluate the degree of belonging to the object of study, i.e., to what extent each of the items should form part of the scale. As for the panel of the expert judges, it was composed of four university professors belonging to the branch of Didactics and School Organization. Specifically, they worked within the lines of research related to educational technology, digital competence, continuing teacher training and inclusion of information and communication technologies (ICT) in the classroom. They had an extensive curriculum in terms of research activity, as well as in the validation of multiple scales, several of which are similar in nature to those presented here (especially those associated with digital competence in teaching and development). On the other hand, the degree of understanding by the target population in the pilot study was also assessed by means of an open-ended item in which the participants of the study could show those strengths or weaknesses they considered important in order to improve the wording of the scale items.

2.3. Data Analysis

In the first instance, an exploratory factor analysis was performed using the principal components method. The factors obtained were orthogonally rotated using the Varimax method with Kaiser normalization. Then, following the recommendations of Muthén and Muthén [48], the characteristics of the data distribution were analyzed: descriptive statistics and linearity or normality of the extracted factors. Likewise, possible differences in the distribution of data according to gender were analyzed using the Mann–Whitney U test and the Kruskal–Wallis H test for independent samples. Spearman’s bivariate correlation analysis technique was applied to analyze the links between the dimensions.

Once the number of factors was determined, finally, confirmatory analysis was carried out to check whether the theoretical measures of the model were consistent through the modeling of diagrams and the use of structural equations [49]. In summary, the aim was to check whether the data obtained fit the hypothetical model produced by the exploratory factor analysis. Finally, the reliability of the scale and of the different dimensions configured were analyzed using Cronbach’s alpha coefficient. The different analyses were performed with the SPSS v.25 and AMOS v.24 statistical packages.

3. Results

In the first instance, with regard to content validity,

Table 1. Qualitative assessment of expert judgment.

Experts	Report
Expert 1	It is necessary to specify whether students are aware of when they need information on a certain topic, in order to cover in greater depth all the aspects expressed in the operational definition of the construct.
Expert 2	Some items need to be reworded to make them easier for university students to understand.
Expert 3	I would specify the different formats through which information can be presented (audio, social network, video, textual, etc.). Similarly, when referring to social networks, I would allude to all of them.
Expert 4	I would replace the word sex with gender. I would also change the word “biases” to “errors” when referring to information that may not be correct, to improve understanding by undergraduates.

shows the qualitative assessment of the scale expressed by the expert judges. These contributions were useful for making significant modifications to the different items that made up the final version of the scale, and for clarifying some conceptual issues, as in the case of identifying the need for information (judge 1), clarifying items by including examples (judge 3), or using a simpler vocabulary in order to adapt the instrument to the target population (judge 4).

Secondly, the results of the quantitative assessment of the scale items (

Table 2. Qualitative assessment of expert judgment.

Item Reference	Expert 1	Expert 2	Expert 3	Expert 4	Mean
I.1	8	9	9	8	8.5
I.2.	7	8	8	8	7.75
I.3.	7	7	6	7	6.75 (Retired)
I.4.	9	9	10	10	9.5
I.5.	6	7	6	6	6.25 (Retired)
I.6.	8	7	8	7	7.5
I.7.	9	9	10	10	9.5
I.8.	9	9	9	9	9
I.9.	9	9	10	9	9.25
I.10	9	9	10	9	9.25
I.11	9	9	9	9	9
I.12	9	9	9	9	9
I.13	8	8	8	8	8
I.14	8	8	8	8	8
I.15	9	8	9	8	8.5
I.16	8	8	8	8	8
I.17	9	9	8	9	8.75
I.18	8	9	9	9	8.75
I.19	7	8	8	7	7.5
I.20	6	7	6	6	6.25 (Retired)

) showed that, in general, all the items were considered optimal by the committee of experts. Those items that received an average score equal to or lower than 7 were eliminated, as was the case with items 3 and 5 and 20.

On the other hand, with respect to the evaluation of comprehension by the study sample based on the item open to suggestions, no suggestions for profound improvement were suggested, indicating only that the comprehension of the scale was optimal.

3.1. Factor Analysis

For the grouping of the questionnaire items into second-order dimensions, the principal component extraction method with Varimax rotation was applied. A three-dimensional solution was fixed.

Table 3. Matrix of rotated components and factorial weight.

	Factor 1	Factor 2	Factor 3	Factorial Weight
I.1. I double check about the veracity of the information I check.	0.799	0.097	0.073	0.615
I.2. I compare a piece of information in several sources to verify its accuracy.	0.791	0.219	0.162	0.630
I.3. I am able to identify if a piece of information has mistakes.	0.705	0.085	0.321	0.548
I.4. I am able to identify if the resources of a piece of information are up to date.	0.627	0.472	0.104	0.456
I.5. I am able to identify if a piece of information is useful for my learning process.	0.489	0.333	0.448	0.537
I.6. I share the information after I have checked its reliability.	0.436	0.362	0.203	0.557
I.7. I use specialized resources (such as scientific databases) to look up for specialized information about a topic.	0.201	0.718	0.029	0.580
I.8. Before I present a piece of information, I share the sources I have used.	0.235	0.715	0.118	0.417
I.9. I index the pieces of information I use following a specific scientific citation, such as APA.	−0.048	0.682	0.384	0.362
I.10. I acknowledge what it implies to share fake information.	0.263	0.491	0.229	0.700
I.11. I am able to write a text with a clear structure (such as an instruction, its development, and a conclusion).	0.109	0.079	0.782	0.652
I.12. I am capable of acknowledging the main ideas of a text.	0.115	0.067	0.721	0.626
I.13. I am capable of selecting the right format to present a piece of information, (whether it is text, audio, image, or video).	0.229	0.269	0.650	0.607
I.14. I am capable of distinguishing the veracity of the pieces of information I receive from different sources (such as emails, news on social media, etc.).	0.242	0.385	0.499	0.551
I.15. When facing a problem, I am capable of knowing where and how to look for information that I need.	0.307	0.366	0.435	0.362
Variance	2.980	2.626	2.593	8.199
% Variance	19.865	17.509	17.288	54.662

shows the rotated component solution with the respective communalities. Factor 1 defines the information evaluation competencies. It is made up of six items that explain 19.87% of the total variance. Factor 2 refers to the information search and communication skills. It is made up of four items that explain 17.51% of the total variance. Factor 3 evaluates competency in information processing and elaboration, explaining 17.29% of the total variance with five items. The total variance explained by the three factors is 54.66%. In short, of the 17 total items resulting from the content analysis, after the application of the exploratory factor analysis, the scale obtained a total of 15 items distributed in three factors.

The Kaiser–Meyer–Olkin test (KMO = 0.89) indicates that there is a high correlation between the variables. Barlett’s test of sphericity was significant (χ2 = 1344.02; gl = 105; p < 0.001) indicating that a factor analysis was appropriate.

Therefore, once the factors and variance percentages were extracted, we proceeded to analyze the characteristics of the data distribution. The results of the normality test indicated that the data did not follow a normal distribution. In this sense, the use of a U-Mann–Whitney test was advocated, which determined that men and women had a similar level of competence in the four second-order dimensions (

Table 4. Descriptive analysis by gender.

	Male (n = 95)			Female (n = 164)			U Mann–Whitney
	M	SD	Me	M	Dt	Me	Z	p	d
Factor 1	5.73	0.93	5.83	5.66	0.83	5.83	−0.849	0.396	0.05
Factor 2	6.02	0.98	6.25	6.03	0.86	6.25	−0.409	0.682	0.03
Factor 3	6.10	0.71	6.20	6.23	0.59	6.40	−1.388	0.165	0.09

).

Table 5. Spearman’s Rho correlations.

		Factor 1	Factor 2	Factor 3
Factor 2	Rho	0.545
	p	<0.001
Factor 3	Rho	0.600	0.537
	p	<0.001	<0.001
Age	Rho	−0.063	0.087	−0.099
	p	0.316	0.162	0.111

shows that the competency of evaluating information is directly and significantly associated with the competency of working with information sources and with the competency in comprehension and elaboration of texts. The latter two are significantly related. Age is not related to any of the second-order dimensions.

3.2. Reliability Analysis

The reliability analysis (

Table 6. Descriptive statistics and reliability of factors.

	M	SD	α
Factor 1	5.69	0.87	0.83
Factor 2	6.02	0.91	0.70
Factor 3	6.18	0.64	0.75

) shows that the first factor has a high Cronbach’s alpha coefficient compared to a moderate-high value for factor 3 and a moderate value for factor 2. The lowest level of competence is found in information evaluation skills.

3.3. Confirmatory Analysis

In order to validate the factor structure extracted after the exploratory factor analysis, a confirmatory factor analysis was carried out using the maximum likelihood assumption. The model consists of fifteen observed variables explained by three second-order dimensions that correspond to those specified after the exploratory factor analysis. The structure of the model and the standardized solution is presented in Figure 2.

The chi-square index of absolute goodness-of-fit of the model to the data (χ287 = 224.336; p < 0.001) shows that there is a significant discrepancy between the data and the model. The coefficient χ2/gl = 2.579 is above 2, which is in line with what was reported for the chi-square value, which indicated that the model does not fit the data perfectly. However, the CFI = 0.89 and NFI = 0.84 have values close to 0.90, while the RMSEA = 0.078 is less than 0.08. These three indicators do show that the model is appropriate for explaining the data from the proposed structure.

Figure 2 shows that the second-order dimensions are highly correlated. In turn, each of these dimensions explains, according to the standardized regression coefficients, a significant proportion of the variance of the observed variables. In short, it can be affirmed that the model is adequate for explaining the factor structure of the observed variables; although, future research of this structure should be refined in order to achieve a better fit to the data.

4. Discussion

Information literacy has become one of the main challenges of the information society. With the arrival of the Internet in our lives, and the large amount of time we spend interacting with digital networks, this subject has become very important in the transversal training of young university students in order to promote sustainable processes of search, evaluation, processing and communication of information skills [5]. In this context, the information competence scale for future teachers was configured in order to measure the levels of information competence for those university students who are in their initial training period for their profession and are able to self-diagnose their level of information competence in this discipline. It is a discipline common to all areas of knowledge, but in the case of the future teacher, it is even more important, since he/she is in charge of transmitting knowledge to a new generation whose interaction with digital and informative media is daily and continuous [14].

For this purpose, this work was framed in an empirical validation design grouped around different phases [42]: initial design and content analysis through expert judgment [38], configuration of the pilot test and dissemination to the target population, and construct (exploratory and confirmatory factorial) and reliability analysis. Finally, and after all these procedures, a scale of 15 items grouped into three dimensions was obtained: (I)—Competences in information evaluation; (II)—Information search and communication skills; and (III)—Competencies in information processing and elaboration. There are three dimensions that comprise the basic processes understood at the time of information collection: Firstly, there is the evaluation of the information coming from the digital network, in order to know how to locate information correctly; secondly, there is a deepening work in specialized sources, in order to be able to classify the sources of information according to the quality of the information offered; and finally, from this information, it is necessary that future teachers are able to elaborate their own information from the specialized information, so that emphasis is placed on the process of creation, elaboration and the processing of the information. Although it is true that the scale is brief, it is intended to establish a generic view of the students’ self-perceived level of this set of skills and abilities. Therefore, from its application in populations, it can elucidate initial approaches to promoting educational interventions in this regard and promote an improvement in competency indexes. Likewise, it is a scale that, unlike several of its predecessors [36,37], is not contextualized in the library landscape, in favor of establishing the focus on transversal actions of an academic nature carried out by any university student, and in this case, more specifically, of future teachers.

The results obtained in the present study showed that the scale obtained favorable scores in terms of its internal structure, as well as in terms of reliability indices. Likewise, the analyses presented did not reflect significant differences in the self-perceptions of the pilot population studied in terms of the variables gender and age, which according to Haladyna and Rodriguez [41], is a positive index in favor of the scale. Likewise, the SEM model obtained from the data distribution indicated high levels of correlation and covariance between the constructs that make up the scale, which is another factor in favor of the scale [49].

5. Conclusions

In recent years, the exponential increase in information and communication technologies has become an empowering phenomenon that has made the Internet the main option for searching and interacting with information. As a result, today’s society needs future professionals who are able to identify, evaluate and critically communicate the information they find on the digital network. In education, there is a demand for teachers and professors who are experienced in digital competence, and more specifically, in information literacy, in order to continue fostering future generations trained in critical thinking and with the skills to search, locate, evaluate, process and communicate information. Therefore, from this work, we tried make progress towards this goal; towards a prospective work, in which information literacy becomes important in the training of the future teachers, understood as one of the current and future challenges that the educational system must face during the digital boom and digital transformation that is happening in today’s society. However, these types of competencies should be considered as transversal to all areas of knowledge.

Finally, with regard to future lines of research, it is necessary for the scientific community to continue investigating the level of informational competence presented by future professionals in different disciplines of knowledge, as well as to develop intervention programs in this area to promote an improvement in information searches, selection and evaluation, elaboration and communication skills. It is, therefore, one of the challenges that Higher Education must address, in order to ensure that future generations of young people present competencies in accordance with the needs demanded by today’s 21st century Society.