The Research of Educational Innovation: Perspective and Strategies

Abstract

Innovation processes in the Spanish and international educational models have evolved, especially over the last 40 years. Technological expansion and development have been integrated, albeit belatedly, into the field of education. There have been changes to the educational model: school practices have been modified so that, while not entailing a completely generalized impact, significant advances have been introduced. This has led to a change in the spaces, times and ways in which these practices are implemented, thus paving the way for a complete shift in the model. Yet this has not led to a systematic and organised reflection on the processes of change, nor on how to approach evaluation and research in education. This paper aims to provide some ideas about the correct targets of these transformations, and it does so by presenting several examples.

1. Introduction

Technological development has conditioned and will continue to condition the processes of change in schools since, from a social perspective, it causes the transformation of social uses and of the very concept of society. Within the framework of human development, the technological revolution has shaped the actions, behaviours and profiles of all of us who participate in the technological environment [1], an influence which, as the latter author points out, permeates all our vital actions in a fluid and liquid manner. We are surrounded by a huge number of actions, facts and relational and communicative processes that take place over the Internet [2].

Education is one of those fields of human action that is subject to such influences. In fact, we are generating innovation when we transform educational processes through the use of technology, even if sometimes the way we use the latter is not entirely positive. In fact, the transformation of educational and innovation-geared processes by means of technology has also involved a procedural makeover as well as a change in our approach to evaluating/researching the benefits that these processes bring about.

To talk about how educational innovation has developed over the last 40 years is to associate educational changes with teaching methodologies and technology integration [3]. Basic ideas, such as those proposed by Carbonell [4] when he defines the concept of innovation in relation to the school setting, construe these changes as a series of intentional and systematic interventions, decisions, and processes aimed at modifying attitudes, ideas, cultures, contents, models and pedagogical practices.

Currently, understanding this process involves rethinking the school model as a whole so that it meets the needs of both the knowledge society and the students who are immersed daily in new technologies and new forms of communication and learning, as well as in the creative construction of knowledge [5].

Changes occur spontaneously, often because of the emergence of a new type of technology that facilitates a process or action that had not been possible before. In other cases, we witness the reuse of a specific technology that had originally been deployed for one type of human activity in different contexts, including school settings. There are many examples of this occurrence, but a case in point may well be the use of geopositioning in the world of mobile learning [6,7].

Furthermore, how are these changes caused by the integration of technology into the educational activities of a school from any level (primary schools, secondary schools or universities) evaluated and researched? The dynamics of evaluation and research, which in this paper are closely connected because of the need for both to plan actions and develop instruments for data collection, require procedures for data gathering and analysis that support an informed response to the issue or change under examination [8].

From our point of view, if we evaluate or research these new processes with the aid of technologies, we must also rethink our designs. In this case, data sources, by the very nature of educational activities, are nourished by qualitative information, as well as by the perspective of the subjects who participate in it. However, it can also be nourished by the information provided by technological resources, usually quantitative or quantifiable. The answer to the question of how to plan evaluation and research around the current use of technology in education inevitably steers our focus in the direction of mixed designs and analyses [9].

It is necessary to introduce research dynamics that go beyond classical educational inquiries based on either qualitative or quantitative methodologies. We need to move towards mixed designs: firstly, because the reasonable epistemology of this current of pragmatic thinking can help us break the dichotomous struggles between post-positivism and constructivism [10,11]; secondly, because when we introduce technology into educational innovation processes, it provides us with both quantitative and qualitative information that needs to be incorporated into our research designs.

In this paper we want to present an overview of the incidence of educational innovation in terms of the incorporation of new methodologies, experiences and technologies, which in turn will lead us to the search for their value as tools for improvement in research processes that combine data sources, mixed analyses and the construction of complex conclusions. In this way, this article attempts to provide an outline of how technology-aided educational innovation is being understood at present. We will also provide examples of how such proposals for change are being designed, researched and implemented.

We hope that this contribution can be helpful in guiding new styles of inquiry and bridging a long-standing gap in innovation dynamics: the absence of robust research into the value of what we propose as changes targeted at having an impact on processes and on individuals. Although much emphasis is placed on innovation, little attention is paid to the process of analysing its value as a driver of transformation. Perhaps because we are specialists in our educational disciplines, but we are not very experienced in research or evaluation processes.

2. Educational Innovation and the Integration of Technology

Innovation in education, as well as in all other fields of social development and knowledge, is important, but it is particularly relevant in this field of study, where it has largely become the groundwork for our research processes as educators. Educational innovation is what makes teaching action change and improve. The foundation of Educational Innovation in Technology rests, as Mike Sharples et al. [12] argue, on the educational research actions and projects that academics undertake. This was made evident when members of the Open University in the UK, in collaboration with several tertiary education institutions worldwide, published one of the first annual reports on educational innovation. This work was particularly focused on educational technology, understood as a framework for methodological development, innovative experiences and the development of educational resources. Since then, a report on the orientations and outcomes of educational innovation has been published annually (http://www.open.ac.uk/blogs/innovating/ (accessed on 20 November 2022).

From the time these reports were issued, and long before that, important work has been done in terms of promoting technology as a fundamental axis of innovation. All innovations should start by responding to some questions, as claimed by [13]. First, we should make sure that what we are trying to change involves an approach that enables the individual and general extension of innovative school practices; innovation is not just “proof of concept”. An example of this was the initial use of resources such as VLEs [14,15], which originated in the previous century even though their introduction in schools in Castile and León only began in the academic year 2018–2019. Secondly, we must answer the question of whether people make structures or structures make people. In this case, we are in a situation where the social structure entails an important transformation in the school institution. It is not that there are changes in society that the school must take on board, but rather that the school is at a time of fundamental social change. Additionally, this has a critical impact on education, since traditionally the school incorporates many social processes by means of knowledge and information (it used to organize content in a static and analogue form, and now this is done in a dynamic and digital form); it employs society’s basic resources to interact through written communication (previously the classic reading-writing systems and now the digital systems). Therefore, educational institutions and the people who run them are pervaded by the sense of ongoing change: the teacher, formerly a repository of knowledge, is now a guide in a dynamic world [13].

If we analyse the proposals put forward by the Open University report as trends in educational innovation in 2022 [16], we see that they focus on the use of hybrid models. After the pandemic, we witness a combination of workspaces in the school timetable and those outside the classroom. The “new normal” that followed the COVID-19 pandemic has made it possible to understand the virtuality of this model, where many more people can participate in training, whether or not they are physically close, by relying on flexible processes.

The report also highlights the importance of dual training jointly provided by schools and companies/industries. Another aspect it underlines is the pedagogies related to micro-credentials are a new type of qualification with specific characteristics oriented towards career, workplace, and professional competencies. Micro-credentials aim to open opportunities for new groups of learners so that people studying them are likely to have a different profile from those who sign up for other forms of traditional education or training. One other aspect worth mentioning is the construction of training in a self-regulated way.

These approaches to training or education have a lot to do with the generation of learning autonomy in what some authors have termed learning ecologies [17]. Essentially, this is so because each person has a specific environment that teaches them and shapes their own learning.

The report likewise emphasises the support of video viewing in general, which leads to closed environments agreed upon by the teacher and the students, but which can also take place in autonomous settings strongly associated with learning processes—an aspect worth analysing when it comes to evaluation. Similarly, mention is made of the management of learning processes through the agency of “educational influencers”.

Finally, the document discusses suggestive views on pedagogies for home education, critical pedagogies of contestation, or movement-based pedagogies.

All the above issues reveal the impact of technology and the fundamental pillars that support its implementation. However, what does this mean in terms of the paradigm shift in education? Furthermore, how should the school be transformed on all educational levels? In this sense, the school has to readjust:

• The role of the teacher, who is nowadays more of a guide and process manager.
• The way in which the school teaches because it must change its textbook-centred training models.
• The very substance of teaching, which should focus on students learning to navigate a networked society, rather than being the guarantor of stable content.
• Content, which is dynamic and therefore should not be taught for a static society, but for one that is diverse, ever-changing and participatory. Therefore, the school must focus on teaching life skills.
• The ways of assessment, which are essentially not suitable for a changing society and must therefore perform a more complex role adapted to such social complexity.

The above dynamics of education have been the object of study in the scientific community for a long time, especially since the technological revolution took shape, and even more so in the university environment and in view of the transformations that European higher education systems have undergone in recent decades.

3. Evaluation and Research Models for this Type of Experience: The Mixed Perspective

In order to carry out research or evaluation processes in new spaces of innovation, we need to define how they should be specifically implemented. Let us broadly define a framework for the analysis and assessment of what these processes contribute against the backdrop of innovative proposals in the field of education.

Both in this and the following section, information about current developments in innovation-led activities is characteristically nourished by quantitative and qualitative data. We can argue that innovative experiences involving technology require the use of both sources of information for evaluation and research purposes. Our attempt goes beyond defining an epistemological position for educational research [18,19] and is instead supported by a series of methodological implications associated with the quantitative and qualitative traditions. In 1989, starting with [11], a new perspective began to take shape: the “mixed” conception of research processes. This perspective emerged as a response to the classic controversy between an extensive and generalizing focus of research and an in-depth and comprehensive focus on the question under examination.

When technology is carried over into the field of educational innovation, the data it collects about current practices provide real quantifiable information about educational activities. No doubt this information contributed evidence of the steps a user takes, but in the analysis of innovation practices, we can also incorporate qualitative data in the form of opinions, observations and reflections. By resorting to mixed methodologies, we can more easily ensure that in any research process five fundamental principles can be observed so as to secure the quality of analysis, one which is complex, yet also close to human situations, and supported by data that represent the various positions of participants. In this sense, we seek to improve by “triangulating” the information, which determines or corroborates the observations achieved by any method, but even more so if two contrasting methods are used. We also sometimes seek “complementarity” as a means of clarifying results when a single qualitative or quantitative perspective does not sufficiently help. Thus we “develop” a process of understanding by shifting from one method to the other. We may occasionally be interested in “expanding” and broadening the research outcomes by seeking to identify particular elements or collateral topics in connection with our research problem. Finally, we may “initiate” a process of extended search for contradictions or complex issues, sometimes simply by broadening our observational framework. These five principles, which Greene first developed in 1989, are fleshed out in her monograph on mixed methodologies published in 2007.

Greene’s works, in addition to those signed by authors like Creswell [20] Creswell and Creswell [21], or Tassacori and Teddly [22], provide a research framework in the area of the Social Sciences and Education that leads us to complex designs which combine quantitative and qualitative data and analytical processes and furnish us with a sound grounding for discussions of technological innovation.

We can produce a number of simple arguments that underpin the importance of this kind of approach. In the first place, when we introduce technology in an educational environment, the processes that are set in motion involve people who participate in activities, form opinions, can be observed or can themselves engage in the observation of their own experiences. Moreover, through the use of technology, they also leave traces of their narrative creations, opinions, and qualitative contributions in technological formats that remain as evidence of their views. Social media, Virtual Learning Environments, simple contributions in the form of school assignments, etc., are clear examples of this valuable input. In addition, the use of technology generates quantitative information that can be analysed: trajectories, systematic action processes, user profiles, interactions with other users, targeted actions, etc., all of which are stored in technological devices. Although this “digital footprint” often raises significant concern, voiced by authors like Marta Peirano [23], who denounced the misuse of data from telephone communications and social media, this does not refute the general benefits that may be drawn by research from processes that are already being implemented in the treatment of information in the global market and in the world at large.

Thus, when planning research actions in technological environments and in educational contexts, we must bear in mind that as soon as we use technology, we start generating quantitative data in addition to the feedback we are able to collect from a qualitative perspective; that these data cannot be ignored because the information on how learning takes place is enriched by additional inputs from the initial, intermediate and final steps of our inquiry as well as by other non-aligned sources (information about the retrieval of information itself from the Web).

In this sense, we have to carefully consider how to draw up our research designs and what our general approach will be: do we mean to be extensive and explanatory, or do we want instead to delve deeper into the analysis of the specific situation under examination? We likewise need to consider when and how to collect and analyse each piece of information, the sequence of data-gathering, the evidence and the method that inform each step in that design. Finally, we must ensure that the results and conclusions obtained provide a truer picture of the facts and the object of our research. Because the integration of technological resources means that educational activities can be experienced at two different levels: one observable by the naked eye and another that is virtually accessed thanks to the technology that envelops the real world.

In the following section, we will briefly present some cases to illustrate the basic typologies of this type of design where research that has generated an innovation proposal is also supported by mixed methodologies and where the research work itself involved not only the monitoring of innovation but also, and ultimately, its evaluation.

4. What Experience Tells Us: Innovation Initiatives for the Use of Technology in the Classroom

In the research group where we have been working for more than two decades, investigative processes related to the use of technology and the evaluation of its potential uses in education have been carried out within the mixed methods framework. The latter findings have proved useful in constructing concepts for discussion and developing a theoretical framework for the analysis and implementation of standard or good practices to exemplify our conception of educational technology.

First, it is necessary to make a clear definition of what is understood by good practices, sometimes understood as the space where traditional practices are developed with innovative elements that help improve their results [24]. Or, as Carrington [25] puts it, those spaces of educational development open the door to the inspiration of new ones.

Our first example will focus on collaborative processes involving technology. Its context is the kind of research carried out during the first years of the 21st century, when the integration of technology helped to carry out analyses of collaboration as a didactic tool and the design of spaces for the development of digital environments: what at that time was called, and still is, CSCL [26], an approach that facilitated the integration into the educational experience of different types of technological environments, as well as collaborative processes based on the use of digital-platform technology [27]. This study was based on a process of innovation in the Computer Architecture subject within the Telematics Engineering studies at the University of Valladolid, where students had to start working in groups and by means of the project technique. In this case, and by resorting to the BSCW platform, we developed instructional dynamics informed by face-to-face processes and technological aids. This innovative experience made it possible to show the importance of group processes for the development of professional knowledge in a collaborative dynamic in the training of computer engineers.

The results of this action were researched and evaluated with a mixed enquiry design. The role that technology played in the follow-up and evaluation of such an innovative practice was crucial. Additionally, above all, the shift in the educational research perspective involved in this approach allowed for qualitative and quantitative processes that enriched and enabled analysis from complementary perspectives.

In the graph below (Figure 1) we can see how contributions from different data sources helped researchers gather information on the qualitative perception by participants of the implemented innovation (QUEST-open questionnaire and Observations) as well as on the digital resources themselves, which were analysed quantitatively (SAMSA). Finally, a comparison of qualitative and quantitative data was carried out at the end of the process by means of a focus group discussion.

The second example, from a few years later, is a study on mobile learning. It is framed in the context of research carried out between 2014 and 2017 [6,7]. This example of innovation consisted of the application of different learning processes in mobility where the space became a part of the classroom. In a specular process of carrying out activities outside the classroom, as if it were a mirror, these activities ended up including in the classroom and were transferred as part of the training activity. This example provides an illustration of how by using the “project technique” [27] it was possible to develop several technological planes that facilitated the connection of spaces in teacher training. In this particular case, initial teacher training and the implementation of an innovation process in a practice school were connected. This type of process allows learning with the support of technological tools and virtual learning environments that facilitate, expand and enrich learning itself in a natural setting outside the classroom. Additionally, the incorporation of technological resources and collaborative activities involved the use of the mobile devices of the students/school. More specifically, a research framework was constructed by using two technological platforms, GLUEPS-AR and LEARNING BUCKETS [29,30], which integrate through VLE (Moodle) different resources, including Augmented Reality (AR-Augmented Reality) and other web-based tools.

As we can see in the figure below (Figure 2), the processes are structured in two layers that are respectively placed on a physical and a virtual level. We can also see how the spatial contexts are distributed between the classroom, the home and open spaces outside the classroom. The arrangement of resources in “learning buckets”, or activity frameworks for teaching and learning, confronts us with educational designs that the teacher creates for his or her class. They are the tool used by environments such as GLUEPS-AR to mediate in the technological layer, which is the one that provides the content resources and the activity resources (spaces where the proposed activity can be carried out), and they enable the association of other technologies that are in the VLE environment or generally on the Internet (including applications that have Smart systems inside phones or tablets). This example foregrounds the possibility of moving outside the school, but also of using alternatives to the classic resources employed for the development of training activities: for example, moving from a physical map to an interactive virtual map, or more generally using resources to guide a schedule of events, which in itself is also an educational action insofar as it procedurally timetables the teaching/learning activities of students. This is therefore a fundamental tool in shaping a global technology-aided educational model that goes beyond the integration of the computer in the classroom. From the point of view of research and evaluation of innovative experiences, it also relies on a mixed design.

Following Stake’s [32] model of qualitative research design, we can argue that in this case two types of data sources and a mixed design were consistently used. In the design diagram (Figure 3) we can see qualitative techniques in operation, but also, in the slot dedicated to documentary sources, quantitative data provided by the LOGs of events recorded by the digital platforms. These sources were compared and contrasted during the process, while a perception study of the participants and their interactions with technology was likewise conducted. The result was a number of deeper insights into the process as a whole.

In our third example, we will focus on a recent study that has not yet been finalised where we used smart technology to guide learning in an educational innovation setting. The mix of SMART technology and the dynamics of Learning Analytics foregrounds the transformation of today’s technology-enhanced learning environments. In this way, we can provide learners with the right support, at the right time and in accordance with their needs by analysing their learning behaviour, performance, and context [33]. The experience followed from the development of the concept of Learning Mobility (m-learning), which has been recognised for the unique opportunity it provides for authentic learning experiences anytime and anywhere [34]. The widespread use of smart devices and online technologies (cloud data, Internet of Things, social media) has led to the evolution of m-learning into smart learning supported by smart technology (anytime, anywhere) [35]. This example highlights the potential of combining specific features and benefits of “Ubiquitous Learning” with social networks and the digital interaction between web content and services [36].

Innovation here means that, in order to support students to learn in the real world in an intelligent way, it is necessary to take into account several factors when designing and developing smart learning systems. This is done with the intent of generating educational innovations where learning is designed differently from the traditional approaches and also to facilitate the orchestration of resources, situations, spaces, etc., which in turn involves monitoring, awareness, (self-)regulation and evaluation of learning activities and outcomes. Both goals stem from the fact that obtaining the information needed to make decisions about the (re)design and orchestration of non-trivial learning situations is beyond the reach of participants given the large number of learners and the diversity of devices that may be involved.

For this piece of innovation, which was carried out on SCARLeT [37,38], measures of student motivation, participation, collaboration, dropout and satisfaction were collected and analysed in order to efficiently discuss learning processes and outcomes. A mixed research design was put together that included a quantitative data collection procedure, as well as the analysis with participants, through a survey format and a focus group, of the results in terms of motivation and participation levels (Figure 4).

The analysis ultimately targeted affects, attention and expectations, as well as measures of support and retention in order to predict learning pathways. More precisely, we were able to analyse attitudes and competencies—in short, educational performance. The combination of all these variables in the analysis can provide a complex yet efficient inquiry into learning processes, all of which is actually provided by Smart technology. To this effect, the identification of the digital traces of students and their learning outcomes has to contain a tightly structured arrangement of information around the several items under consideration. In short, this is what we in the educational profession were always keen to develop: a comprehensive evaluation process which was our ultimate goal. Exploring learners’ digital footprints will enable us to harvest many sources of information, as well as a highly diverse number of data (e.g., access to online materials, learners’ digital records, grades and length of interaction with the learning environment), which leads us in the direction of moving towards the exploration of multiple, complex and information-rich data sources and sophisticated digital environments that employ mobile and smart devices and are also inspired by real-world contexts. Thus, Learning Analytics is defined as “the measurement, collection, analysis and reporting of data about learners and their contexts in order to understand and optimise learning and the environments in which it occurs” [39].

After presenting these three examples, which represent practices that have evolved into the present, we would like to conclude by summarising their contribution to evaluation design and mixed research.

5. Consequences for the Conception of the New Model

To conclude, the contributions of these three studies, which could be just a few examples of how this type of design can help, represent advances in the analysis of innovation experiences. It is worth highlighting the contributions made by the automatic information collection systems, which in some way show us students’ unconscious actions, as well as processes that influence educational dynamics and which are not usually considered. Another aspect has to do with the possibilities of contrasting the participants’ visions when they are confronted with what they think they have done, and what the information reports made by the technology show them. Finally, the possibilities of analysing all levels of reality, which is more complex with technology.

The practical consequences of technology-based educational innovation models have been clear to us for a long time. There are basically two main impacts on the classroom and the teachers which are worth stating here because of their relevance. The first implies that the classroom doors are no longer closed, but instead, access is provided to as much content as possible—content that is ethically acceptable and complies with the principle of human rights. However, alongside this, there is the possibility that, if the professionals, the school institution or the parents do not surveil this process of content construction, schools could become ethically unacceptable spaces. This is a fundamental consequence that challenges the teaching staff, who must become guides in the selection of the school’s content. Additionally, in this guiding role—in the development of active methodologies—teachers must be conductors and not wells of wisdom.

However, are we equally perceptive about the evaluation and research processes involved in these innovative experiences? From the point of view of innovation, research and evaluation, we believe that these processes do not generally include the analysis of their value for potential improvement. It is much more common to engage in analyses of satisfaction, performance, or educational change, but not of the in-depth consequences of innovation on the basis of all available data. The evaluation of initiatives does not, in many cases, consider the possibilities provided by the rich space provided by the Learning Analytic community [40]. This article has attempted to steer the debate along that path and to promote real changes in evaluation, monitoring and research processes, which are currently dispersed in terms of their results and value.