Embedded Information Problem-Solving Instruction to Foster Learning from Digital Sources: Longitudinal Effects on Task Performance

Abstract

This research paper is based on a longitudinal study to find out how long-term embedded whole-task instruction can help students to develop more efficient information problem-solving (IPS) skills that could lead to a better use of internet information for learning and solving digital tasks more effectively. To this end, we designed, implemented and evaluated a three-year instruction programme to promote students’ development of key IPS skills in real-life classroom settings. This research involved sixty-one secondary education students. Forty-two of them received the IPS instruction and their results were analysed longitudinally and subsequently compared to a control group which received the regular courses. The results showed that students who received the IPS instruction improved their performance significantly in tasks in which the use of IPS skills was needed and these students organised and presented the information found on the internet critically and gave personal arguments. The findings also revealed that during the three-year project, the scores of IPS task performance were statistically higher in the instructed students than those obtained in control group students. Our study then provides an insight into how secondary students develop IPS skills throughout long-term instructional support and shows a series of educational implications.

1. Introduction

The impact of the Internet Age has prompted a paradigm shift in education. Nowadays, most of our everyday learning is characterised by drawing knowledge from a wide variety of electronic resources. Learners from different levels are required to search for, collect and understand information from digital external sources and construct a solution to solve a task. This shift has never been more noticeable than amidst the current coronavirus pandemic. In this context, it is important to remember that educational research has identified information problem solving (henceforth IPS) as a complex process that requires the unfolding of complex higher-order cognitive skills, e.g., [1,2,3].

Although it is undeniable that younger generations of students appear to master the skills needed to navigate online digital resources, educational research confirms that, without explicit instruction, students underuse or even lack the IPS skills to find correct and reliable online resources and construct knowledge from them [4,5,6,7]. Therefore, educational research sees the need to provide students with adequate IPS skills to learn from online and digital resources. Furthermore, [8] claim that IPS skills instruction is crucial to promote quality, equality and sustainable education because it has been found that students’ performance in digital skills is initially associated with their socio-economic background, academic achievement and residence location.

Various theoretical models have been proposed to characterise the phases and the cognitive processes involved in IPS that are needed to transform the retrieved web information into knowledge [9]. However, these models describe the stages and cognitive competences involved in the process, but fail to show which students’ specific activities are in each stage and how to best support them. As a consequence, educational institutions and teachers find it difficult to teach the key IPS skills that could help students take full advantage of the opportunities the internet provides for learning and building knowledge autonomously from online digital resources and in finding a suitable place and time in the curriculum [3,5].

In recent years, research has been carried out to analyse the effectiveness of teaching IPS using the internet, e.g., [10,11,12,13]. However, further research is still needed to tailor the existing IPS models to specific groups of students and in specific learning contexts [3,9] and, by so doing, promote quality and sustainable education for all students.

This paper takes a first step towards supporting teachers in embedding IPS skills in educational curricula with the description of the design and empirical testing of instruction for IPS skills. Inspired by the Four-Component Instructional Design (4C/ID) model [14] for teaching complex skills, we have designed a long-term, embedded, whole-task IPS instruction programme to foster learning and meaning-making from digital sources and investigated its effects on students’ task performance.

1.1. Information Problem Solving

Information problem solving (IPS) is a complex cognitive process considered as an important 21st century skill in combination with critical thinking [15]. The authors of [1,2,3] have defined a five-step approach to solving information problems based on a decomposition of the IPS process into constituent skills and subskills. This approach highlights the fact that during the implementation of all skills, it is essential to activate regulation activities, such as orientation, monitoring, steering and evaluating [16,17].

Figure 1 shows this IPS model. Basically, it represents that, when students are confronted with an information problem or challenge, as a first step, they have to define the problem, activate previous knowledge which will help detect what information is needed and formulate a clear question that will lead to the information searching process and the selection of searching terms. Research shows that teenagers have problems when they reach the stage of formulating questions [18], clarifying task requirements, activating prior knowledge and determining what information needs searching [16]. In addition, very few students start searching with a prior reflection on the task at hand and clear outlining of their search [10,19,20].

The searching skill then starts off with the definition of relevant search terms to be used in the search engine. Many students struggle with this second stage and tend to introduce long sentences that may reduce their success in finding relevant and reliable sources [18,21]. Therefore, skills related to the formulation of the search terms and their fine tuning are, indeed, influential abilities to solve a problem in a satisfactory manner [22,23].

To accomplish the skill of selecting the relevance and reliability of information, students must tackle evaluation abilities. However, they do not systematically judge the search results on the search engine results page (SERP) and often find it difficult to choose between reputable and questionable sources [16,20,21,24]. Sometimes, the source selected may well come from a commercial source [11]. As these activities are often challenging, novice students tend to pay little attention to reliability of the sources and the information found [24,25].

The fourth skill, i.e., the one needed to analyse, select and process the information found, comes into play. By virtue of that, useful information is compared, contrasted and elaborated with students’ previous knowledge and information from other resources. It has often been claimed that this skill requires a high level of effort and motivation [26], since it is essential in accomplishing an IPS task [27] successfully. In fact, some studies point out that learners can be instructed to generate more relevant search terms, enhance their evaluation habits and select better information and sources (e.g., [28]).

Finally, the fifth skill in the students’ constructed information is organising and integrating in a personal manner using organisation and communication tools to answer the question or challenge posed at the beginning of the process. Regarding this skill, it is reported that students mainly use ineffective strategies such as copying and pasting information from the web [29], and tend to find a simple answer on a single web page as opposed to reading the web information in a critical and thorough way [12].

To sum up, considering that the resolution of the task as the solution to an information problem from online sources implies a complex cognitive process [12,16], in which secondary students face many challenges, it is essential for them to receive guidance and supervision through a well-designed educational intervention.

1.2. Information Problem-Solving Instruction

It is often claimed that IPS skills are underdeveloped or absent without explicit instruction, even among “digital natives” [1,3,4,5,6,16]. However, educational research shows that students can be instructed to better define the problem and the information needed, generate more relevant search queries, adopt more evaluation criteria, select higher quality resources and deeply process and present information to answer an informational problem [10,25].

Over recent decades, much effort has been made to investigate efficient instructional approaches for IPS and incorporate effective support for guiding students’ activity in searching, retrieving, evaluating and integrating information from multiple web sources (e.g., [3,4,10,11,12,13,16,30]). However, despite the researchers’ efforts made so far, their attempts have proved insufficient and further research is still needed in order to face and shed light on how formal IPS skills training could be designed in order to have a positive impact on students’ learning.

Our study is built on the basis of the four-component instructional design (4C/ID, for short) model [14] to design, implement and empirically test innovative IPS instruction in secondary education. The 4C/ID model advocates the design of four components:

• Learning tasks are understood as authentic real-life tasks and their solution requires the integration and coordination of skills, knowledge and attitudes.
• Both supportive information and guidance are needed to develop cognitive models and strategies in order to complete the learning task.
• Procedural information has to be carefully designed by providing step-by-step instruction and explicit skills and procedures.
• Part-task practice should be included to provide enough training for recurrent skills.

In the arena of IPS instruction, these four components have been translated according to the following principles: whole-task, embedded and long-term instruction.

1.2.1. Whole-Task IPS Instruction

Whole-task instruction proposes the resolution of ill-structured, authentic and complex real-life situations in which students have to perform all the steps of the IPS process, from beginning to end, and students can find different ways to solve the task. Whole-task instruction has proved to be more effective to teach IPS complex skills than part-task fragmented instruction [10,14]. Whole-task instruction offers the possibility to provide support for all the IPS skills and practice them as a whole process in which one skill relates to and impacts on the others. By contrast, instructional approaches that focus only on practicing specific searching or evaluating skills, e.g., [23], offer students very few occasions to coordinate and integrate all of the five IPS skills [14], and also to transfer [31].

Regarding the support needed, research provides evidence that it is possible to build on whole-task support to improve students’ IPS skills in demanding learning and learning that is difficult to be achieved successfully [11,12,32,33,34]. The main approaches for giving support in IPS instruction and the outcomes obtained are the following five: driving questions, prompting, content representation tools, processing worksheets and writing and communicating support.

• Driving questions. Driving questions are open questions given in an initial phase. They guide students to accomplish the key phases of the whole task, help learners to activate prior knowledge and offer relevant resources to solve tasks efficiently. IPS research shows that driving questions turn out to be a useful supporting tool for assisting students through the phases and skills required to solve IPS tasks [35,36]. In this line, [23] found that driving questions had positive effects in regulating the IPS process in higher education. The authors of [36] also used driving questions to aid law students in the proper use of electronic resources and guide them through their learning process.
• Prompting. The authors of [3] define prompts as a simple and yet effective method to provide instructional feedback in an online environment. These researchers summarise the effectiveness of three timing prompts: anticipative prompts delivered before execution of the targeted skill, instructional prompts provided during skill performance and reflection prompts given after execution of the targeted skill. Similarly, [12] used efficiently computer-embedded prompts in secondary education to help students assess and select the relevant information, but also to reflect and regulate the IPS process. In addition, [35] found that the use of metacognitive prompts in collaborative settings efficiently provides guidance in searching information skills. The authors of [37] also highlighted that prompting laypersons when consulting the internet proved to be a positive tool to acquire knowledge on health issues.
• Content representation tools. Content representation tools provide learners with category elements of an underlying ontology [38] and can aid students to structure a task domain. Different studies have used content representation tools to guide the students’ search and navigation across multiple information sources [39] and organise ill-structured web information [40], concept maps [41] or data tables [42] to assist in understanding and structuring web information retrieved from multiple digital resources.
• Process worksheets. Activities such as breaking up the task into parts or steps and completing process worksheets that focus on key processes to solve the whole tasks are effectively used in IPS instruction research [3,43] with two main benefits: first, completing process worksheets stimulates the active processing of essential information to solve the IPS task. Second, they ensure that students will follow a correct systematic approach and generate schemas of correct solution strategies themselves [44]. In this line, [23] included process worksheets as scaffolds in IPS instruction in higher education and discovered that intervention students regulated the problem-solving process and judged the information found more often.
• Writing and communication support. Most IPS tasks asked students to articulate a written response constructed from multiple digital resources. Writing from multiple documents entails complex cognitive processes and is a challenging activity itself [12]. Despite this difficulty and importance, this aspect of IPS skill has been marginally considered in most IPS studies [39]. Therefore, supporting the productive process of writing is necessary [35,37]. Our study tackles this issue by giving support to students’ writing processes when solving an IPS task and providing templates of how to organise information in a leaflet, flyer, brochure, letter or argumentative essay given to the students.

1.2.2. Embedded Instruction

Embedding IPS training within a meaningful context with domain-specific instruction has proved more effective than standalone courses [37,45,46]. Embedding instruction has the potential to increase engagement, motivation, transfer and deep learning [47]. Previous studies investigating embedded instruction have shown good results in primary education [48,49], secondary education [12,31,50] and higher education [13,51].

A literature review offers theoretical and empirical evidence on the effectiveness of whole-task and embedded IPS instruction. However, there are still scarce studies combining these two key instructional approaches. For instance, [10] investigated an embedded IPS course designed according to a whole-task approach and instructed ten student teachers in a quasi-experimental intervention study, finding positive results in the development of IPS skills and task performance. In another study, [23] successfully applied embedded IPS instruction with psychology students. In this study, students obtained good learning outcomes and increased the frequency of the use of some of their IPS constituent skills and regulation activities. More recently, [52] investigated student teachers’ IPS skills through embedded whole-task instruction in a 20-week course and reported that the instruction succeeded in developing cognitive strategies to tackle an information problem.

1.2.3. Long-Term Instruction

Long-term instruction for learning has been considered as instruction that lasts over a quarter of the academic year [53], or even as an instructional course that may take place over two or three weeks [54]. In the specific field of IPS, long-term instruction has been related to a curriculum-wide approach [6,30,52]. Most IPS intervention studies apply short-term instruction and these studies report that some of the improvements in IPS skills reached by the participants disappeared after completing the course [10]. In this vein, researchers claimed the need for “a scaled-up version with more content, more task classes containing tasks of increasing complexity, offered over a longer period of time and embedded in a multitude of contexts, might prove very effective.” [3] (p. 101). This claim is also shared by other studies, in which it is assumed that the whole-task approach to complex learning requires more learning tasks over longer periods than other kinds of instruction, but such practice will lead to better transfer to new settings when designed and conducted adequately [10,55].

In summary, despite the existence of studies confirming that embedded whole-task IPS instruction improves students’ IPS skills, there is still the need to know to what an extent the period of instruction of the IPS skills might have a positive impact on students’ learning and performance results [22,32,56]. Furthermore, while most educational institutions acknowledge that IPS is an essential academic skill in this digital and knowledge era, they struggle with its implementation, and specifically in finding a suitable place and sizeable time in the curriculum for IPS integration [3,39]. IPS skills require domain-specific knowledge and, in order to guarantee their transfer to daily activities, long-term, embedded and supported IPS practice throughout the whole curriculum is needed [13,57].

Notwithstanding this necessity, most IPS instruction is often implemented as a separate course and loosely connected to the curricular contents (e.g., [11]) and secondary education students still face difficulties in their daily school activities [58,59]. Therefore, it is desirable to further investigate how to embed IPS research and instruction in real secondary classrooms and learn curricular contents to provide best practices, approaches and conclusive results of quality education for all students. To this end, this paper tackles this objective and provides answers to this educational challenge by discussing the design, development and empirical testing of a long-term, embedded, whole-task IPS instruction programme in secondary education. Specifically, our research investigates the longitudinal effects of a three-year IPS instruction programme on students’ task performance when solving complex digital problems.

1.3. The Study

The present study is grounded on research by [50], who started to investigate the effects of long-term, embedded, whole-task instruction on the development of IPS skills in secondary education. Our study then follows up on this research and takes a longitudinal approach that aims to answer the following question: what are the effects of long-term, embedded and whole-task IPS instruction on students’ task performance? While research shows discrete short-term learning effects, it is unclear whether there may be higher potential in a long-term situation [3,10]. Our study aims to contribute with new data. With this purpose, a three-year embedded IPS skills intervention programme was designed, during which students solved whole-task projects related to daily life challenges as well as science, technology, mathematics (STEM) and social science curricular contents. In our quasi-experimental design, the digital task performance of students following the regular curriculum (i.e., control group) was compared to that of students following the three-year IPS instruction (i.e., experimental group).

As this long-term training makes use of whole tasks that address and support all constituent IPS skills, our expectations are that those students who follow IPS instruction will display deeper meaning-making from digital sources and better task performance than their counterparts who follow the regular curriculum. With a view to obtaining a more detailed description on the effects of IPS instruction on task performance results, the four evaluation tests carried out to assess task performance over the three-year project included three different tasks of varying difficulty, namely: (a) fact-finding task, (b) information-gathering task and (c) final essay. Our research aims to confirm or reject the following four hypotheses:

Hypothesis 1 (H1).

Students following IPS instruction will carry out a fact-finding task better, as measured by the number of correct answers presented within.

Hypothesis 2 (H2).

Students following IPS instruction will solve an information-gathering task better, as measured by the number of correct argumentative concepts presented within.

Hypothesis 3 (H3).

Students following IPS instruction will write a better final essay, as measured by the level of explanation of the ideas written up.

Hypothesis 4 (H4).

There will be longitudinal differences between the two groups—control and experimental groups—on IPS task performance throughout the three-year project. These differences will be more noticeable in more complex tasks that involve information gathering as well as in the final essay.

2. Materials and Methods

2.1. Participants

The participants of our study were involved in a larger research project that aimed to promote digital literacy in secondary education students and in real-life classroom settings. For this reason, we only recruited a sample of sixty-one students (32 girls and 29 boys). It must be said, though, that these participants were fully committed during our ambitious three-year IPS instruction and completed the four tests of the longitudinal research throughout this period of time. From these, 42 of them corresponded to the experimental group and followed the long-term IPS instruction, while the remaining 19 were members of the control group and followed regular classes. At the beginning of the project, the students’ ages were 12/13, and by the end of the project, their ages were 15/16. They belonged to three urban schools from the city of Lleida (Spain). In order to preserve the natural classroom environment and due to ethical issues to ensure that all students of the same school could benefit from our long-term IPS instruction, one school was established as the control group, while the other two schools acted as the intervention group.

In the control group, the students did not follow the IPS instruction and this group was used to study the natural development of IPS skills by a group of students who live in a digital society and use digital information in their daily life. Therefore, the control group students were free to use internet information to solve school assignments depending on the teacher’s learning objectives. However, the teachers did not provide guided internet use and neither did students participate in any specific instruction, course or workshop related to IPS skills nor internet navigation.

The research complied with the ethical code by requiring the school authorities and parental consent to allow participation of their children in the study, and that of the teachers. The research team guaranteed confidentiality and data protection to all participants.

2.2. Study Design and Procedure

This is a longitudinal study with a quasi-experimental design, including an experimental group, i.e., the group of interest, and a control group, used to establish the natural development of students who did not receive explicit IPS instruction. All the participants completed four tests carried out at four different moments during the three-year project.

The longitudinal study design process consisted of three main actions, as seen in Figure 2:

• Action 1. Initial evaluation of control and experimental students at the beginning of the research project, namely, Test 1.
• Action 2. Implementation of the IPS instruction: only the experimental group followed the three-year intervention.
• Action 3. Follow-up evaluation: at the end of every academic year, control and experimental students were tested, namely, Test 2, Test 3 and Test 4.

To carry out the three-year project, we also counted on the collaboration of eighteen secondary teachers of four school disciplines (namely, science, technology, maths (STEM) and social sciences) who worked hand in hand with our research group in designing the IPS tasks, the supporting tools provided for each task to promote the development of IPS skills and the embedding of the digital tasks in the school curriculum. During this collaboration, our research group ensured that the teachers became aware of the importance of IPS skills to better solve information problems and promoting them in a real-life classroom setting.

2.3. Materials and Characteristics of the IPS Instruction

The long-term, embedded and whole-task IPS instruction consisted in the resolution of 24 web-based learning tasks. Each task consisted in an authentic, ill-structured whole task embedded within four sessions of 60 min each. Therefore, students received approximately 96 hours of sustained and maintained IPS instruction for a period of three years.

All the IPS learning tasks were designed following the three key instructional and methodological principles grounded on the IPS literature review presented previously: long-term, embedded and whole task.

• Long-term instruction. Students were exposed to a wide range of learning tasks and domain-specific instruction [60], and had plenty of opportunities to practice the different skills and subskills in an extensive curriculum over a period of three academic years, which makes the transferability of the IPS skills an easier task [13].
• Embedded instruction. This principle aims to design authentic curricular tasks and teach the current subjects more meaningfully, because the learning tasks must be fully integrated in the regular curriculum. Since the role of the domain knowledge is an important factor that can be analysed in IPS [61] and instruction could be more effective by tackling more subjects and settings to facilitate the transfer of IPS skills [3,10], our long-term instruction was embedded in the contents of four curricular areas: science, technology, maths (STEM) and social science.
• Whole-task instruction. Students solved problems and challenges in which they covered the entire IPS process and had to use all the constituent skills and subskills from beginning to end, including the practice of IPS as a whole process in which one skill was coordinated and integrated with the rest of the skills [10]. The instruction provided students with the five-step structure, whereby each skill and its respective subskills functioned in an ordered and iterative way [62]. In addition, the learning tasks designed during the longitudinal project were optimised by means of technological support displayed on screens during the learning process. All these measures guaranteed students proper guidance and support to learn specific IPS skills and subskills [11,12,33] and the following five types of educative support were included: driving questions, prompting, content representation tools, process worksheets and writing and communicating support. Figure 3 shows and illustrates the five types of educational support designed to promote the IPS development.

2.4. Data Collection

The instrument to collect the data of this study was a web-based authentic whole task that required unfolding learning skills. We designed two versions of the web-based task; both were about astronomy and were similar in terms of style, complexity and structure. Web-based task 1 was used as the basis for Test 1 and Test 3; its content was about the planet Mars. Web-based task 2 was used as the basis for Test 2 and Test 4, its content being about the Moon. All the participants solved the web-based tasks individually, in a real classroom context within 50 min and the students’ answers were all stored in a webserver.

2.5. Measurements

In order to obtain a more detailed description about the effects of the IPS instruction in solving learning tasks of different levels of complexity, the design of the web-based task was divided into three inter-related tasks of varying difficulty, namely: (a) fact-finding task; (b) information-gathering task and (c) final essay; resulting in different variable types, respectively, quantitative and qualitative nominal, each one with different grading scores.

Dependent variable 1: Fact-finding task performance. This consisted in a carefully structured task, in which its 61 participants were asked to complete a conceptual map that involved searching for factual information about a planet (e.g., physical characteristics, orography). This task focused on searching and locating relatively simple pieces of information that was usually found on one single website [59]. Students received 1 point for each question answered correctly, and 0 points for incorrect answers. This was a quantitative variable with a total score of 14 points.

Dependent variable 2: Information-gathering task performance. This was an ill-structured task, in which its 61 participants had to answer seven questions argumentatively. Specifically, this task encouraged students to gather and integrate information from different web sources to find an answer. Information-gathering tasks are more difficult to solve because collecting and integrating information from different sources requires remembering pieces of information while searching. Besides, this type of task involves complex cognitive processes and IPS skills to generate meaning out of complex, electronic documents [33]. Students received 1 point for each question answered correctly, and 0 points for incorrect answers. This variable was quantitative, with a total score of 7 points.

Dependent variable 3: Final essay task performance. This task was a conclusion or sum-up task whereby each student was asked to write a short argumentative essay (of approximately 300 words) that integrated and used comprehensive web information to write a personal argument. To be specific, students were asked to hypothesise whether it would be possible to establish a human colony on Mars or the Moon and, if so, what problems would humans encounter. This variable was qualitative nominal; a rubric scale was constructed to capture the mean level of explanation of the content ideas presented in the essay. The rubric scale started with the category “no answer”, which meant that students had failed to write the essay, followed by two categories related to describing facts: (1) “separate pieces of facts” and (2) “organised facts”; the rubric also established two categories related to explaining: (3) “partial explanation” and (4) “explanation” [19,63].

The study considered reliability and validity issues. For variables 1 and 2, two raters, familiar with both the IPS tasks and the materials, coded 15% of all the answers protocols. Interrater reliability computed on this subsample of protocols yielded a Cohen’s kappa higher than 80. One rater scored the remaining protocols. For variable 3, each participant essay was reviewed by two raters. Discrepancies were solved using a consensus-based approach. Member checking is a well-established procedure to build up “trustworthiness” in qualitative research [64].

2.6. Data Analysis

The statistical methods used in the analyses are described as follows:

• Descriptive statistics were computed for each dependent variable and each test. For quantitative variables, the following were shown: median, mean, minimum and maximum values. For qualitative variables, the following were shown: counts and percentages.
• In order to compare the results shown by the control and intervention groups, each test was subjected to a bivariate analysis. A non-parametric Wilcoxon test for comparing medians was used for quantitative dependent variables, and a chi-squared test for qualitative dependent variables.
• To analyse the longitudinal effect of long-term IPS instruction on the different dependent variables, a general linear model with repeated measures, e.g., [65], was established for the quantitative dependent variables. The results were given in terms of least square mean differences. For qualitative nominal dependent variables, on account of three nominal categories (no answer, facts, explanation), three logistic regression models, e.g., [66], were established. For these models, the results were displayed using the odds ratio (OR) and the corresponding 95% confidence interval reported.

The statistical significance was defined as p < 0.05. All the results were obtained using SAS software, v9.3, SAS Institute Inc. (Cary, NC, USA), 2007.

3. Results

We have organised the presentation of the results into two subsections: (1) descriptive results obtained by the control and experimental group students in IPS task performance variables in each test and bivariate analyses; (2) longitudinal analyses of long-term instruction effects on task performance.

3.1. Descriptive Results and Bivariate Analyses

Table 1. Fact-finding and information-gathering task performance (dependent variables 1 and 2): descriptive and Wilcoxon test results.

		Control				Experimental				Wilcoxon Two-Sample Test
Dependent Variable	Test	Mean	Median	Min.	Max.	Mean	Median	Min.	Max.	p Value
1. Fact-finding task-performance (Total value: 14 points)	1	11	11	7	14	10.3	11	5	14	n.s
	2	9	10	1	12	10.5	11	2	13	400.5 (0.0028)
	3	13	13	10	14	12.9	13	8	14	n.s.
	4	11.4	12	8	14	12.2	12.5	8	14	n.s
2. Information-gathering task performance	1	1.5	1	0	3	1.7	1.7	0	5	n.s
	2	2.4	2	0	5.5	3.7	3.7	0	7	418 (0.0074)
	3	2.5	2.5	0	5	3.6	3	0	7	n.s.
(Total value: 7 points)	4	3.3	3	0	7	4.5	5	1	7	421 (0.0085)

Note: N = 61 students; N control group = 19; N experimental group = 42.

and

Table 2. Final essay task performance (dependent variable 3): descriptive and chi-squared test results.

		Control		Experimental		Chi Square Test
Test	Categories	Count	%	Count	%	P Value
1	No answer	14	73.68	20	47.62
	Facts	5	26.32	22	52.38
	Explanation	0	0	0	0
						3.603 (0.0577)
2	No answer	6	31.58	10	23.81
	Facts	13	68.42	24	57.14
	Explanation	0	0	8	19.05
						6.531 (0.0382)
3	No answer	2	10.53	6	14.29
	Facts	13	68.42	25	59.52
	Explanation	4	21.05	11	26.19
						n.s.
4	No answer	8	42.11	6	14.29
	Facts	10	52.63	19	45.24
	Explanation	1	5.26	17	40.48
						10.059 (0.0065)

Note: N = 61 students; N control group = 19; N experimental group = 42.

present the main descriptive statistics for the three IPS task performance variables obtained by the students in the four tests.

The results presented in Table 1 and in Figure 4 show that both groups of students obtained lower scores in information-gathering than in fact-finding tasks (in line with Hypotheses 1 and 2). From these results, we can infer that information-gathering tasks are more complex and difficult than fact-finding tasks.

As the project progressed (see Figure 4), on average, both groups obtained better scores in fact-finding as well as in information-gathering tasks. However, this increase was significantly higher in the experimental group that followed the long-term IPS instruction. The experimental group also performed better in solving the information-gathering tasks; thus, from Test 1 through to Test 4, the median increased by 3.3 points (out of 7) in this type of task. By contrast, the increase for the control group students in this task was only of 2 points.

Figure 5 shows the results of the different categories identified in the final essay, these results show that at the outset of the study, no student had any explanation category whatsoever for Test 1. As the project progressed, we identified different pattern responses in control and experimental students. While the experimental group students reduced drastically the percentage of no answer category and increased firmly the percentage of explanation category (from 0 in Test 1 to 40.48% in Test 4), the control group students maintained high rates of no answer and very low rates of explanation throughout the project.

Bivariate analyses were carried out to study the effect of IPS instruction on students’ IPS task performance in each evaluation point. At the outset of the study, in Test 1, no statistically significant differences were found between groups (control vs. experimental) neither for fact-finding nor for information-gathering task performance (see Table 1 and Figure 4).

When solving fact-finding tasks, the bivariate analyses hardly showed statistically significant differences between groups, with the exception of Test 2 (Wilcoxon two-sample test = 400.5, p = 0.0028). However, in solving information-gathering tasks, experimental students obtained higher scores than control students in Tests 2, 3 and 4, and these differences were statistically higher in Test 2 (Wilcoxon two-sample test = 418, p = 0.0074) and Test 4 (Wilcoxon two-sample test = 421, p = 0.0085), compared with the control group (see Table 1 and Figure 4).

Regarding the final essay (Hypothesis 3), the bivariate analyses revealed statistically significant differences between the quality of the responses between the groups in Tests 1, 2 and 4 (see Table 2).

In summary, these results point out that those students who followed the long-term, embedded, whole-task IPS instruction showed higher improvement in the performance of complex IPS tasks and were more capable of generating explanations from web information than control group students.

3.2. Longitudinal Analyses of IPS Instruction Effects on Task Performance

A stratified general linear model with repeated measures analysis was used in order to investigate how the long-term, embedded, whole-task IPS instruction improved the experimental students’ IPS task performance throughout the project. Three analyses were carried out and the results obtained in Test 1 were compared to those in Test 2 (T1 vs. T2), Test 3 (T1 vs. T3) and Test 4 (T1 vs. T4). The results of these three analyses are presented in

Table 3. Fact-finding and information-gathering task performance (dependent variables 1 and 2): stratified general linear models with repeated measures for different evaluation points.

		Control				Experimental
Quantitative Dependent Variable	Comparison between Tests	LSM Differences	95% CI Lower	95% CI Upper	p Value	LSM Differences	95% CI Lower	95% CI Upper	p Value
1. Fact-finding task-performance (Total value: 14 points)	T1 vs. T2	1.7406	0.2176	3.2635	0.0195	−0.3098	−1.2670	0.6474	0.8320
	T1 vs. T3	−1.9694	−3.4282	−0.5106	0.0043	−2.8788	−3.6522	−2.1054	<0.0001
	T1 vs. T4	−0.8748	−2.3740	0.6244	0.4115	−2.3111	−3.1602	−1.4619	<0.0001
2. Information-gathering task- performance (Total value: 7 points)	T1 vs. T2	−1.1292	−2.1397	−0.1187	0.0232	−1.6525	−2.4248	−0.8802	<0.0001
	T1 vs. T3	−1.0808	−1.9732	−0.1883	0.0121	−1.3526	−1.9944	−0.7107	<0.0001
	T1 vs. T4	−1.4960	−2.4916	−0.5003	0.0013	−2.3852	−3.0685	−1.7018	<0.0001

.

The longitudinal analysis points out that IPS instruction has a highly statistically significant impact on the evolution of experimental students’ IPS fact-finding and information-gathering task performance (Hypotheses 1 and 2). This positive impact is significantly higher in the resolution of complex tasks. Thus, the estimated least square mean (henceforth, LSM) difference between Test 1 and Test 4 is 2.31 units (out of 14) for fact-finding tasks and 2.38 units (out of 7) for information-gathering tasks.

Control group students also showed a longitudinal increment in IPS task performance scores; the estimated LSM difference between Test 1 and Test 4 is only 0.87 (out of 14) for fact-finding tasks and 1.5 units (out of 7) for information-gathering tasks. However, these increments are lower than the ones obtained by intervention group students.

To verify whether there were any statistical differences between control and intervention groups in IPS task performance scores across time (Hypothesis 4), a stratified general linear model with repeated measures was performed. Statistically significant differences between the two groups were found in information-gathering task performance; the estimated LSM difference between the two groups in this variable was 0.9695 (−0.9695, CI 95% = (−0.386, −1.553)). However, no statistically significant differences between control and experimental groups were found in fact-finding task performance. This result reveals that the positive longitudinal effect of long-term IPS instruction was higher when solving complex IPS tasks.

A logistic regression model was carried out to investigate the longitudinal effect of long-term, embedded, whole-task IPS instruction on final essay task performance (Hypothesis 3). As this is a qualitative nominal variable with three nominal categories (no answer, facts, explanation), three logistic regression models were established. Model 1 analysis (no answer vs. answer) showed that in the control group, the odds ratio of no answer was almost two times higher than that of the intervention group. Model 2 analysis (no answer versus explanation) showed that the odds of obtaining no answer in the control group were five times higher than in the intervention group. Finally, Model 3 analysis (facts vs. explanation) revealed that intervention group students were 3.39 times more likely to generate explanation responses in their essays (see all the results in

Table 4. Final essay task performance (dependent variable 3): three logistic regression models established for the qualitative nominal dependent variable.

	Control vs. Experimental
Qualitative Nominal Dependent Variable 3	OR Estimate	CI 95% OR Lower	CI 95% OR Upper	p Value
Model 1. No answer vs. Answer	1.96	1.09	3.51	0.0250
Model 2. No answer vs. Explanation	5.14	1.75	15.07	0.0035
Model 3. Facts vs. Explanation	3.39	1.14	10.08	0.0286

) than control group students. This finding shows a positive longitudinal effect of IPS instruction on integrating and using web information comprehensively.

4. Discussion and Conclusions

Even though schools experience difficulties in embedding IPS skills across the curriculum, our study successfully implemented and evaluated long-term, embedded, whole-task IPS instruction in real-life classroom settings. Consequently, the instructional approach used in this study can make a valuable contribution to providing good practice and quality education for all students and to overcome some of the difficulties stated by previous research [59,60].

Moreover, this paper has researched the longitudinal effect of IPS instruction on the students’ task performance and learning in real-life classroom settings. Our results reveal that long-term, embedded, whole-task IPS instruction has a highly positive effect on students’ IPS performance as time goes by. Thus, students following IPS instruction produced a better fact-finding task (Hypothesis 1), information-gathering task (Hypothesis 2) and a better final essay (Hypothesis 3), as verified by the quality of the explanation of the ideas expressed. In addition, we found longitudinal differences between control and experimental groups regarding IPS task performance throughout the three-year project, because instructed students outperformed their counterparts in more complex tasks, namely the information-gathering task and final essay (Hypothesis 4).

These findings coincide with the results obtained in previous research in which the participants’ engagement in web-search instruction could improve their learning and efficiency in solving the task [12,50]. Other authors also pointed out that the better the strategy and reflection on each skill, the better the efficiency, e.g., [22,25].

Our results also reveal that, as the project progressed, those students who followed the long-term, embedded, whole-task IPS instruction solved information problems better than the control students. Hence, the positive effects of the students’ participation in IPS instruction are higher in solving complex tasks. In information-gathering tasks, whereby complex cognitive processes come into play and which encourage students to extract meaning out of complex digital documents, experimental students outperformed control students as the project progressed and these differences increased over time. This is in line with previous studies that had shown that the more complex the task, the deeper the processing of information. Likewise, a more expert pattern was required to solve the problem successfully [10,32,33]. We can then conclude that the IPS instruction designed in this study promoted the students’ development of those informational skills that helped them learn from internet resources.

Besides, our research contributes with experimental evidence to the fact that the IPS instruction designed in this study helped students to generate more and better explanations in their final essay. Thus, our students learned to integrate information from digital resources and build meaning-making to provide a successful answer to an informational problem. Previous research had already demonstrated that the process of explanation increased critical thinking and understanding by pushing an agent to explain the consequences of his/her view and to search for new information needed for answering questions and achieving his/her cognitive goals [67]. To conclude, our study extends previous results and gives experimental evidence that long-term, embedded, whole-task instruction in real-life classroom settings is desirable to efficiently help students to learn while solving complex digital tasks.

In the light of the results obtained in our study, we can claim that embedded support across long-term instruction boosts students’ IPS skill development. The students receiving the instruction mastered IPS skills that helped them draw relevant conclusions out of internet resources and succeeded in using this information meaningfully to construct their own arguments in the final essay. This confirms previous studies by [68], in that such factors as the students’ use of adequate search strategies, the adoption of assessment strategies towards online information and the quality of online resources obtained by students were essential to explain the successful development of science-related conceptual understandings expressed in a final essay. Along the same line of argument, [11] related greater source evaluation reached by instructed students in a secondary school with a deeper level of information comprehension to solve an information task.

Our study strengthens the claim highlighted by previous research in which natural interaction with digital information and non-explicit support is not enough to furnish our students with those IPS skills needed to solve complex information problems [3,14,30,69].

Limitations and Implications for Future Research

Important difficulties were overcome in order to recruit the students and teachers to participate in this demanding three-year study. Especially worth noting are the difficulties to recruit the control group which had less benefits from participating in the project. This resulted in having a small sample, which in some of the analyses could be a weakness, and may not allow us to draw general and transferable conclusions. Furthermore, our research did not study the impact of pedagogical variables, such as school teaching approaches or school teaching programmes, on students’ results. In future studies, possible differences of the participant schools in key variables related with teaching should be monitored and considered.

One could argue that our study did not analyse the students’ results in relation with their initial IPS skills, prior knowledge and interest in the topic at hand, even though these variables had been considered as important in previous IPS skills research, i.e., [11,12]. Despite this limitation, the deep long-term analyses of the progression of individual students regarding IPS skills show that students’ participation in a sheltered learning environment over time can be beneficial to all students, regardless of their background knowledge. Indeed, as the project progressed, we could see that the intervention students performed better in complex tasks. This result is in line with previous research that revealed that providing scaffolds to develop cognitive skills is beneficial to children with higher and lower skills [11].

Another limitation of this study is that, although our IPS instruction offered different and diverse supporting tools to promote the development of IPS skills, other support could have been more effective, such as modelling examples, as previous educational research has shown their efficacy in promoting IPS learning [30] and web page evaluation [70]. A modelling example involves an expert solving a problem while thinking aloud and describing the details and decisions related with each skill during the IPS process. A recorded video can show the screen of and actions made by the expert while concurrently playing the expert’s explanation. Online learning instruction provides an easy opportunity for embedding modelling example videos [30]. In subsequent studies, modelling examples can therefore be included in the repertoires of tools to support learning tasks, and even eye-movement modelling examples (EMMEs), consisting of videos showing a dot representing the eye movements in the modelling example videos [71], which are also effective in a digital educational context [72].

Although we have hypothesised that progressive development of IPS skills in the experimental group could account for the effectiveness of their task performance throughout the project [20,58], in this study we have not analysed the impact of IPS instruction on the development of IPS skills and subskills. In future research work, our intention is to obtain more empirical evidence to support this statement, using a series of techniques to collect and analyse the data, such as log files, eye tracking, think-aloud protocols, cued retrospective reports or a combination of them all [73], and by so doing, be able to apprehend the strengths and weakness of the execution of the IPS process, skills and subskills in greater detail and/or in a qualitative way. The ongoing learning activities during the web-search process could also be analysed in a more in-depth manner [74].

Despite its limitations, our research is significant for educational research in the topic of IPS instruction because it shows that long-term, embedded, whole-task IPS instruction for learning curricular content in real classroom settings is feasible. In a nutshell, our study can contribute to education with the design of a series of learning tasks and supporting tools that can be effective in developing IPS skills in a secondary education curriculum that have a positive impact on students’ learning.