The Socio-Economic Evaluation of a European Project: The DIYLab Case

Abstract

This paper builds on the results of a 3-year long European project, the main aim of which was to deeply and sustainably transform teaching and learning practice in primary and secondary schools and higher education, by introducing Do it Yourself (DIY) philosophy in order to expand digital competence and foster students’ agency and collaborative learning. Three universities and three primary and secondary schools have been involved in a Collaborative Action Research (CAR) process in order to analyse their current institutional context and perceive needs, strengths and weaknesses; to undertake professional development activities and the design of DIYLabs; implement DIYLabs in the selected courses; and reflect upon ways of improving the institution’s performance. This paper offers a global vision of the research and implementation processes and the results achieved, from the perspective of the socio-economic dimensions involved in a project aiming to make a difference in teaching and learning to meet the challenges of a society highly permeated by digital technology (DT).

1. Introduction

This paper builds on the results of the 3-year long European project Do It Yourself in Education: Expanding Digital Competence to Foster Student Agency and Collaborative Learning (DIYLab). Responding to the European Commission’s call for Lifelong Learning Programme Key Activity 3: 3.1.1. Reinforcing key competences, and taking into account the importance of research-practice partnership in education [1], the project brought together three universities, three primary and three secondary schools with the common aim of combining educational innovation with today’s cultural and technical realities. The partners were the University of Barcelona (UB), Spain; the University of Oulu. Extension School and Teacher Training School (UOULU), Finland; Charles University in Prague (CUNI), Czech Republic; Escola Virorai, Spain; and ZŠ Korunovační (KORUNKA) School, Czech Republic.

1.1. Project Background

At present, there is a growing need to understand and foster knowledge and skills required for education and training better suited to meet society’s challenges, to better equip citizens with key competences and to develop 21st century lifelong and life-wide learning skills [2]. For the Organisation for Economic Co-operation and Development (OECD’s) DeCeSo project [3], it is clear that competence implies several dimensions starting from the ability to (a) use a wide variety of tools; (b) interact with others in heterogeneous groups; and (c) take responsibility for one’s life and act autonomously. At the heart of the key competences, the document suggests, lies reflectiveness. Consistent with the initial diagnosis of a globalised world, the ability to handle differences and deal with contradictions so defining for non-standard, complex situations is clearly evident. “Today’s diverse and complex world demands that we do not necessarily rush to a single answer, to an either-or solution, but rather handle tensions [...] by integrating seemingly contradictory or incompatible goals as aspects of the same reality” [3] (p. 9). Although most policy documents then continue to list specific skill sets, reflectiveness often remains ill-defined.

In any case, many of the abilities related to competences are transversal—cutting across different subjects and educational system levels—and digital technologies can help to support them. However, although competences may be quite concisely defined, they are less evident in practical terms and “difficult to be transferred to the context of schools” [4] (p. 58) and [5]. Fostering such competences requires novel strategies and teaching approaches based on active modes of learning, such as collaborative learning, peer-learning communities, creative problem solving, learning by doing, experiential learning, and the development of critical thinking and creativity.

Digital competence is considered as a core skill for life and employability. Today, the question is not whether digital technology (DT) should be used, but rather what is its impact on the way people learn and approach and value knowledge; and why, how, where and for what educational goals should it be used. Accordingly, our attention turns to pedagogical support, learning opportunities and assessment approaches that will encourage the acquisition of digital and other key competences, because “competences are associated with successful behaviour in non-standardized situations” [6] (p. 86), and are “more than just knowledge and skills. They involve the ability to meet complex demands, by drawing on and mobilising psychosocial resources (including skills and attitudes) in a particular context” [3] (p. 3) and [7]. In this light, the DIYLab project assumes that in order to address digital competence in its complexity, we should consider current teaching practices and institutional cultures [8,9]. This is why the project closely worked with schools and universities in a Collaborative Action Research process [10] to foster a lasting commitment to innovation.

In 2006, the European Commission (EC) defined eight key competences—understood as a combination of knowledge, skills and attitudes—necessary for personal fulfilment, active citizenship, social cohesion and employability in a knowledge society. One of them is digital competence. The EC recommendation suggests that: “Digital competence involves the confident and critical use of Information Society Technology (IST) for work, leisure and communication. It is underpinned by basic skills in ICT: the use of computers to retrieve, assess, store, produce, present and exchange information, and to communicate and participate in collaborative networks via the Internet.” [11] (p. 15). Similarly, in different European countries, the government has established basic competences for all infant, primary and secondary school curricula. Universities have also adopted this logic [12].

As suggested earlier, for different authors [13,14] digital competence may be quite concisely defined, although this definition is less evident in practical terms and difficult to be transferred to the institutional contexts. In fact, Ferrari, Punie & Redecker [15] have identified up to fifteen frameworks to approach digital competence, while Hatlevik & Christophersen [16] consider different concepts around the use of digital tools, such as digital competence, digital literacy, media literacy, digital skill, Internet skills and others. Likewise, Mills [17] (p. 246) refers to the “increased attention to new literacy practices in digital environments across a variety of social contexts” as the digital turn.

While for policy-makers, digital competence consists of having the ability to search, obtain, handle and communicate information, and to transform it into knowledge. It incorporates different skills, ranging from access to information to its transmission through various media once it has been processed, including the use of ICT as essential for informing oneself, learning and communicating. It is associated with the search, selection, recording and processing or analysis of information, using a variety of techniques and strategies to access it according to the source and support used (oral, print, visual, digital or multimedia). It requires a basic command of specific languages (textual, numerical, iconic, visual, graphic and audio) and guidelines for decoding, transferring and applying knowledge in different situations and contexts. Digital competence involves making regular use of the technological resources available to solve real problems efficiently. At the same time, it enables the evaluation and selection of new information sources and technological innovations as they appear, in terms of their usefulness for undertaking specific tasks or objectives [4].

It also means placing special emphasis on the idea that being digitally competent means being autonomous, efficient, responsible, critical and reflective in selecting, processing and using information and its sources, as well as using different technological tools. It also implies having a critical attitude and a reflective assessment of the information available, contrasting if necessary, and respecting the socially agreed rules of conduct to regulate the use of the information and its sources in the different media [4].

Due to the difficulty of implementing competence-based curricula in an educational tradition grounded in subject-based curricula [9,18]), there is a pressing need to better understand and support the learning of digital competence equally in formal, informal and non-formal learning. Linking the three kinds of learning experiences is paramount to contributing to the development of lifelong and life-wide learning. Here, we must take into account that initial and in-service teacher professional development is not paying enough attention to new knowledge and skills, especially those related to digital competence.

Today, DT is present in schools and universities. eBooks, interactive blackboards, tablets or notebooks are now commonplace in Europe’s classrooms. However, when using this technology, students are typically positioned as consumers [19]. This is why our project aims to reposition them as prosumers, as people who not only consume but produce media, and do it from a critical perspective [20]. The pedagogical approach underpinning the project builds on the experience gained by partners in European, national and international projects. It also draws on lessons offered by DIY culture, and requires students to become producers of educational digital objects, rather than only consumers of what teachers or educational software companies offer [21].

The Emergence of DIY Culture

In the last decade, young people’s relationship with DT has evolved. Our understanding of digital competences has shifted from considering youth’s critical understanding of new media as one key aspect of digital literacy [22,23] to seeing young people not only as consumers but as content producers contributing to blogs, designing animations, graphics, and video productions [24]. This has generated alternative approaches to digital competence to develop “creative designs, ethical considerations, and technical skills to capture youths’ expressive and intellectual engagement with new media.” [25] (p. 89).

From this perspective, the DIYLab project took into account the change (and its educational effects) occurring in the last decade regarding digital competencies, especially the emergence of a culture of collaboration connected to youth learning environments, technology and DIY philosophy [20,21,22,23,24,25]. Young people’s efforts to create and disseminate digital media have been associated with the growing do-it-yourself (DIY) movement [26]. Starting in the 90‘s [27] with arts, crafts, and new technologies [28,29], it is now prevalent in curriculum contents [30,31] giving educators and students the opportunity to create, share and learn in collaboration. In recent years, the DIY philosophy has found a formidable push in the Maker movement, deeply based on DT, more focused on software and supported in open-source hardware. It relishes the creation of new devices as well as mending and playing with existing ones [32]. Heavily founded projects such as the Fab Lab (http://fab.cba.mit.edu/) initiative, launched by the Media Lab in the Massachusetts Institute of Technology [33] or The Digital Media and Learning Research Hub (http://dmlhub.net/), of the University of California, Humanities Research Institute, have considerably approached educational institutions with these open learning cultures. We have also considered the countless DIY and Maker communities, taking into account that our aim was to develop DIY cultures in formal institutions—in other words, in highly prescriptive and inertial contexts, and that we wanted to do it across the curriculum, not necessarily to produce objects, but to foster autonomous and reflective learning in collaboration.

1.2. Objectives

The main objectives of the project were:

• Analyse how digital competence can be better integrated into curricula and connected to learning outcomes.
• Build a conceptual and technical approach through a collaborative professional development process—with researchers, teachers and administrators—to allow participants to move from being information consumers to knowledge producers, by fostering digital competence.
• Use a digital hub, for sharing DIY digital objects, to support the growth of an open, cross-cultural learning community.
• Draw on the DIY philosophy to create DIYLabs, understood as flexible spaces for developing cross-curriculum projects where participants introduce, develop and use inquiry-based projects connecting different class subjects and students’ interests.
• Through an action research process, assess the design and implementation of DIYLabs—with researchers, teachers, administrators and students—in order to make sustainable improvements in each institutional context.
• Undertake a socio-economic evaluation to assess the costs and benefits of the project.

The project also aimed to deeply and sustainably transform teaching and learning practices in the primary and secondary schools and higher education institutions involved, as well as provide scalability guidelines to foster the development of key competences. In this respect, taking into account research on educational change [9,34], this can be only achieved by involving teachers and students in the decision-making process and anchoring new practices in the most promising aspects of teachers’ professional knowledge and practice. If teachers and students felt their current knowledge and skills were being dismissed instead of built upon—as so often happens—they would revert to their old practices and forget the new skills and resources acquired once the project was over.

1.3. The Focus of the Paper

In this paper, given our awareness and interest for the socio-economic issues related to DT-driven educational innovations, we focus the discussion section on these aspects of the project.

By Luke Georghiou, John Rigby and Hugh Cameron [35] (p. 220):

Socio-economic evaluation has the major function of identifying the outcomes and impacts of governmental programmes in all their variety and scale. Such programmes need endorsement by the democratic institutions which provide the resources which allow them to occur. Furthermore, such programmes require management to ensure appropriateness and the validity of the premises upon which such programmes are based. They also need management to ensure efficiency, control and effectiveness in their delivery. Evaluation must serve all these ends. It is a vital task, and the more difficult to achieve because of the complexity of the social and economic processes upon which programmes act. Without evaluation activities, governmental initiatives are blind, lacking the means to justification and to learning, to improvement and to excellence.

However, they also highlight its limits and expectations:

Because intervention in the form of programme initiatives is intended to change the world, programmes which operate on a significant scale necessarily destroy the possibility of comparison, making the assessment of programme impacts difficult and measurements of net programme impact doubly so. Attempts to establish true net impacts therefore must invoke hypothetical and counterfactual constructions—to focus on what might have happened without the programme. Despite the difficulty of carrying through such an aim, it is essential to focus on additionally or net programme impact to obtain any sense and measure of programme action.

[35] (p. 220)

DIYLab is a relatively small educational project partially supported by European funds, but in its design and implementation, different assessment levels are embedded, a fundamental one being the socio-economic evaluation to weigh the impact of the introduction of a DIYLab into schools and universities.

Table 1. DIYLab embedded evaluation levels.

Level	Place	Procedure	Time	Results
European	External	Assessment of deliverables and general project’s development.	Months 18th, 36th	Report highlighting project’s strengths and weaknesses and recommendations.
Consortium	External	A quality assurance committee appointed by each participating country has assessed local reports.	Months 25th, 35th	Reports identifying project’s strengths, weaknesses, at country level, and recommendations.
Partners	Internal	Collaborative action research (curriculum evaluation, focus groups.)	Throughout the entire project	Imbedded in the corresponding deliverables.
Teacher	Internal	Rubrics and cross-country assessment of DIY digital objects	Throughout the implementation process	Students’ marks.
Consortium	Internal	Socio-economic dimensions	Data collected during the implementation process as a whole	Reports highlighting the socio-economic dimension

summarizes the different appraisal stages implanted across the project, highlighting the socio-economic assessment conducted at consortium level.

At the different stages of the project implementation, diverse evaluation procedures have been enacted, each of them with different purposes and methods.

2. Materials and Methods

Taking into account that the hardest limits of school change, including those promoted by DT, seems to be in the “grammar” of schooling [36], we have carefully considered the different elements that shape educational institutions, in Foucault’s terms, as powerful social dispositive [37]. To foster stakeholders’ involvement and the sustainability of the project, we have used a methodology based on the principles of Collaborative Action Research (CAR).

A participatory, democratic process, concerned with developing practical knowing in the pursuit of worthwhile human purposes, grounded in a participatory worldview which we believe is emerging at this historical moment. It seeks to bring together action and reflection, theory and practice, in participation with other, in the pursuit of practical solutions to issues of pressing concern to people, and more generally the flourishing of the individual persons and their communities.

[10] (p. 1)

As represented in Figure 1, each project’s work package (WP) represents a circle of the CAR process involving all participants.

Step 1 consisted of:

• The analyses of the official documents that prescribed the primary and secondary education curricula, the syllabi of the participating schools and the university degrees involved.
• Fifteen focus groups [38,39]; twelve with primary and secondary Spanish, Finnish and Czech students, teachers and parents, and three with Spanish and Czech higher education students and teachers.

Step 2 involved the development of five professional development workshops (three for primary and secondary and two for university teachers) understood as an on-going dialogical process of analysis and practice. The main aim of these workshops was to design key aspects for implementing DIYLabs in the different institutional contexts.

Step 3 involved the design, implementation, and training for the participants, of the DIYLabHub (http://hub.diylab.eu/); DIY digital objects produced in the implementation of DIYLabs at schools and university are publically shared.

Step 4 consisted of the implementation of DIYLabs designed in step 2, in the five participating institutions following the DIY teaching and learning philosophy, based on inquiry-based projects, collaborative learning, creative problem solving, learning by doing, experiential learning, the development of critical thinking and creativity and the extensive use of digital technologies [40,41,42,43,44]. As stated above, DIYLabs were understood as flexible spaces for developing cross-curriculum projects, where participants introduce, develop and use inquiry-based projects based on their interests, connecting different subjects and topics, and using different kinds of educational resources, in particular DT.

Step 5, continuing the CAR cycle started in step 1, entailed the organisation once again of focus groups in order to evaluate from the participants’ point of view the implementation of the DIYLabs. In each primary and secondary school, three focus groups for parents, teachers and students, and in each higher education institution, two for teachers and students were implemented.

Step 6, the socio-economic evaluation draws on the analyses of the data collected through all project’s steps.

In addition to data collected for the internal and public deliverables related to each WP, through all the project’s stages, we have kept textual, visual and audio-visual field notes [45,46].

3. Results

To be able to understand the complexity of DT-driven educational projects and foresee their socio-economic dimensions, we summarise the set of results obtained in each CAR circle.

3.1. Step 1. Building DIYLab from Participants’ Experience and Expertise

This phase of the project was the foundation stone for the conversion of the consortium into a community of learning and practice. Participating teachers (students, parents) and institutions felt that DIYLab was not a top-down project aimed at implementing other people’s ideas. From the very beginning, everybody’s knowledge, experience, working condition and expectations were recognised and were the basis of the new educational practice to be implemented.

3.1.1. Digital Competence and DIY Learning in National Curricula and Local Syllabi

At primary and secondary education level, in the case of Spain, we identified four key points in the official curricula in line with the main aims of the DIYLab project. Both the prescribed curricula and the school syllabi put special emphasis on:

• Autonomous and self-regulated learning.
• Interdisciplinary knowledge.
• Inquiry-based teaching and learning.
• Digital competence was understood not as a technical skill, but as general (and critical) literacy across a range of media use. The school also considered it a transversal competence, which supported the project aims of implementing DIYLabs [47].

In Finland, the new national curriculum for primary and secondary education emphasized the importance of key competences development. Because key competences are transversal, this opened up space for imagining the role of the DIYLabs, mainly based on Multiliteracy and ICT. The curriculum remade to suit the needs of future students appeared to adopt a number of shared values also expressed within the DIYLab project.

In Czech curricula there was little or no national incentive to promote the main educational aims established in the DIYLab project. In the analysed documents, there was no mention of autonomous learning or collaborative learning, and very little mention of inquiry-based practices. Digital competency did not appear as a transversal issue, only as subject matter within two specific courses. More flexibility appeared within the “cross-curricular topics”, namely Media Education, as well as Environmental Education and Personality and Social Education, which tended to focus on projects and teamwork.

Regarding higher education, in the case of Spain, the analyses of the syllabi of the five degrees (Fine Arts, Pedagogy, Social Education, Infant Education and Primary education) involved, showed a clearly discipline-based and compartmentalised notion of knowledge. Digital competence was not one of the six transversal competences considered at the University of Barcelona [48]. However, competences such as learning and responsibility, creativity and entrepreneurship, teamwork and communication, were deeply connected with the project. The syllabi of different courses of the identified degrees also showed important connections with DIYLab philosophy.

In the Faculty of Education of Charles University, ICT courses were not compulsory for student teachers. These courses have been, and still are, taught and guaranteed by the Department of Information Technology and Technical Education (DITTE), which is involved in the DIYLab project.

3.1.2. Introducing DIYLabs into Participating Institutions

In Virolai School, both primary and secondary school-based curricula revealed key spaces and moments to implement DIYLabs, especially for the commitment of the school to promote inquiry-based cross-curriculum projects.

The primary school teachers considered the pupils as the focus of the learning, and pointed out situations that promoted this viewpoint. They placed special emphasis on promoting meaningful learning and set it as a challenge. The pupils were guided to take into account what they learn outside school. Their families believed the school encouraged learning, and pointed out things learnt by their children they found surprisingly mature. Students and families identified areas within school where a sort of DIY learning was already taking place.

For the secondary school teachers, the school had made a clear and firm decision to use learning and knowledge technologies (LKT) to favour learning experiences and the development of digital competence. The secondary school pupils pointed out a series of contexts as a substantial part of their learning process, some of which, accessible in and outside school, were highly connected with DIY principles.

Finnish schools saw the new national curriculum framework as a great opportunity to implement DIY teaching and learning philosophy and DIYLabs.

Primary school teachers spoke about specific shortcomings of the school that, if addressed, would greatly improve the project’s outcomes, such as the traditional classrooms of the school, which echoed making teamwork difficult. In addition, logistically, the school needed to resolve issues like where to store DIYLab projects (which will be new), how to source extra material for more hands-on, inquiry-based learning, etc.

Secondary teachers gave a few examples about successful school projects that reflected DIYLab aims. Efforts had been made to connect home time, personal skills and the school framework, for example, while making use of the presentation from the vacation trip and in the recording of nature observations.

In Czech primary and secondary school, whereas the national curricula appeared distant from the philosophy of DIY learning, the particular culture of a so-called “creative school” was more closely aligned with the principles of the project. In the school curriculum, there were six compulsory cross-curricular topics, organised in collaboration by teachers of various school subjects. Through these cross-curricular topics, the school pursued many important activities that corresponded to the idea and purposes of DIYLab. The obstacles were based on the fact that not all teachers were ready to use advanced DT, the lack of financial budget for equipment, and the social gaps among children.

The analysis of the teaching plans of the five abovementioned degrees of the University of Barcelona, and of the content of the focus groups held with students and teachers, highlighted the spaces and subjects to implement DIYLabs. All of them shared inquiry-based methodological approaches and some were directly related to the educational use of DT and the understanding and creation of new learning situations and possibilities related to these technologies.

In Charles University, some departments—Maths Education, Art Education, Czech Language Education, Biology and Environmental Studies, Chemistry Education, etc.—tried to introduce DT into teaching subjects. At the masters’ level, they offered two courses, which could anchor DIYLabs.

At this stage, the biggest challenge faced was the task of imagining the new, of trying to develop a model that responded to the needs of each context yet managed to provide a comprehensive and innovative structure. Therefore, the upcoming design of the DIYLabs in different educational contexts, including the professional development of those involved in the project, struggled against the tradition of educational cultures that tends to tell teachers what they must do and how they must do it. We challenged all project participants to develop a professional atmosphere of learning and cooperation that assisted the development of proposals integrating the three principles of the DIY philosophy: creating, sharing and learning in collaboration.

3.2. Formation in Support of DIY Education and Design of the DIY Lab

The main results of this action relate to the five professional development workshops (three for primary and secondary and two for university teachers) put into practice. The workshops focused on the following topics. (a) DIY philosophy and its pedagogical principles; (b) From the vision to the classroom—what DIYLab means in our institutions; (c) The available tools, devices, etc.; (d) Relation between DIYLab and curriculum—How the actions we have planned support the goals of the curriculum. In the following paragraphs, we highlight the results around the second topic, as it was the basis of the design of DIYLabs by making decisions about:

• WHERE: times and contexts to implement DIYLabs.
• HOW: ideas about how DIYLabs will be implemented.
• WHO: who should be involved
• WHAT: what we need to be able to work.
• EVALUATION: how we envisage it.
• DIFFICULTIES AND ADVANTAGES: of implementing DIYLabs

In Spanish primary school, the decision was to implement the DIYLabs in the context of two transversal and interdisciplinary projects: “Tivo Creativo” and “We geographers”. In the secondary school, the decision was to implement the DIYLab into the “Smart Cities” interdisciplinary project that involved languages, natural and social sciences, and technology. It addressed the challenges faced by 21st century cities through classroom activities that pose real or similar situations to the ones found in a modern city. In both cases, the evaluation will be based on rubrics in order to assess the work process, evaluate the final product, self-assess the working groups and co-evaluate among students.

In Finnish schools, the theoretical and methodological framework of the DIYLab was well accepted and from this point forward, teachers’ way of thinking was straightforward towards the implementation. The challenge was how to maintain an innovative approach in a pressure of school routines and curriculum boundaries. In the primary school the designed DIYLab would be implemented through an interdisciplinary topic: “Force”. While the secondary school decided to do it across the following inquiry-based projects: “Europe yesterday and today”, “Traffic Behaviour, road safety, environmentalism”, “North-America and immigration throughout history “Light as wave motion”.

Both Spanish and Finnish schools had quite good ICT-infrastructure, widely used in activities. Teachers made plans to document the work processes and products by videos and photos, bearing in mind that the practices will be shared through DIYLabHub.

In the primary and secondary Czech school, the internal project scheme facilitated the involvement of all lower secondary school pupils in a specific way. The school-based curriculum enabled, in subjects such as Fine Art, Labour Activities and Informatics, to make a choice of cross-curriculum topics by a teacher and the pupils. Nevertheless, teachers worried about the lack of up-dated technical resources.

At the University of Barcelona, the decision was to implement DIYLabs in eleven different subjects from five different degrees. Despite the variability of these subjects, they all shared a conception of teaching and learning based on: (a) Self-regulated learning; (b) Autonomous learning; (c) Collaborative learning; (d) Meta-reflection; (e) Transference; (f) Evaluation of the learning; (g) The use of all learning tools and resources (digital or not) was available and relevant in each case. In all subjects, the students developed inquiry-based projects in groups.

Regarding Charles University, DIYLabs were implemented with the students of the Faculty of Pedagogy within compulsory or optional courses provided by the departments. The outcomes had to be parts of assessed tasks, in most cases necessary for the successful completion of the course. Solving DIYLabs activities always had to require the cooperation of students (at least a team of two), whereas the distribution of roles and responsibilities was left primarily to the students themselves. Great attention was paid to the improvement of the digital literacy of both students and teachers.

3.3. The Launch Digital Hub

The result of step 3 was the implementation of DIYLabHub (http://hub.diylab.eu) and the training of all project participants to be able to upload and share DIY digital objects produced in the implementation of DIYLabs at schools and university. At present, the number of digital objects available in the hub is 217.

The consortium defined the characteristics of the DIY digital objects produced by students to be included in the DIYLabHub. They had to last for a maximum of five minutes. The content had to be comprehensible for a person not connected with the project. They had to have a visual component, had to move between description and reflection and had to have a storyboard as a base. The title and author or authors had to be specified, and they had to provide a summary and subtitles in English.

3.4. DIY Labs in Action at School and Higher Education

As stated in the proposal submitted to the European Commission, one of the aims of the project was to involve (approximately):

• 150 5th grade students and 12 teachers (in 3 primary schools).
• 180 9th grade students and 28 teachers (in 3 secondary schools).
• 160 university students and 20 teachers (in 2 universities—2 Faculties of Education and 1 Faculty of Fine Arts).

By the end of this phase, at least 150 DIY digital objects developed by students had to be available in the DIYLabHub.

As shown in

Table 2. DIYLab activities in primary and secondary schools described in numbers.

Primary and Secondary School	Num. of Pupils	Num. of Teachers	Num. of Subjects	Num. of Activities	Num. of Digital Objects Published on the Hub
Spain	95	15	9	2	32
Finland	114	14	18	9	56
Czech Republic	269	-	13	20	20
Total	478	36	40	31	108

and

Table 3. DIYLab activities in higher education described by numbers.

University	Faculty	Field of Study	Num. of Students	Num. of Teachers	Num. of Courses or Activities	Num. of Digital Objects
UB (Spain)	Education Fine Arts	Pedagogy	228	11	5	53
		Social Education	79	4	3	1
		Early Childhood and Primary Education	12	2	1	4
		Fine Arts	152	3	2	18
CUNI (Czech Rep.)	Education	ICT	23	6	13	9
		Education	196	-	-	-
		Biology	23	1	3	7
		Fine Arts	23	1	1	17
Total		7	713	28	28	109 a

^a Not all objects have been published on DIYLabHub—depending on the quality and criteria set by the project.

, this goal has been clearly achieved.

The primary and secondary pupils were highly motivated and enjoyed solving DIYlabs problems. DIYLabs activities were implemented in the majority of school subjects; they had a cross-subject character. They were done both in and outside school including home preparation for school. The majority of DIY digital objects uploaded in the DIYLabHub were realised in teamwork. In the three countries, students proved to be incredibly creative, had many original ideas and had to learn to follow through with their projects, and to communicate their processes and results in a refined manner. The DIYLabs activities demonstrated that students had no substantial problems using DT; they were able to start solving problems with unknown technology in projects that were attractive and meaningful for them.

The role of primary school teachers differed from that of secondary ones. The primary school teachers faced the problem that for pupils it was not so easy to do research by themselves or use DT for researching a particular topic. Another issue was to make sure pupils went beyond what they already knew. The secondary school teachers realised that their students were able to be much more independent in their learning than expected.

Even if the implementation process in all primary and secondary schools met the project’s objectives, it also faced tensions, contradictions and problematic situations. The most relevant challenges to introduce DIYlabs in these educational levels were:

• According to teachers. Time to plan the project with all teachers involved—including the familiarisation with evolving digital technologies. The need to define and plan the different issues students can set out, especially for primary education pupils as they have not yet developed the necessary skills to carry out a completely open project. The organization of school time, schedules and spaces. More time was needed to make a deeper analysis of the learning process during the project.
• According to students. The difficulties related to collaborative teamwork and organisation and planning. Time management (lack of time to work in class, the need to wait for teacher’s help, etc.). The need for more digital resources.

In other words, the biggest challenges for teachers and students were time (time for motivation, discussion, brainstorming, doing a review, etc.), teamwork (social relations, to bring activities to a conclusion, concentration to the work, etc.), and access to digital resources.

The organizational dimensions of universities (subject-based curriculum, semesters, traditional spaces…) made it difficult for teachers to introduce transversal inquiry-based projects. The two universities proved to be rather rigid and formal institutions with little space to introduce the educational approach underlying the project. However, the content of all subjects was outlined from DIY philosophy, especially regarding the design of learning environments, the notion of knowledge, the need to develop collective wisdom in any group and the use of digital technologies. Teachers developed different strategies to meet the specific contexts’ needs. Most of the students involved evaluated the DIY teaching and learning approach positively due to its bond with the punk, anti-consumerist and self-generated movements [31,49].

Nevertheless, as in the case of primary and secondary education, several issues, contradictions and difficulties were identified. Teachers and students involved in the project pointed out the following challenges and proposals.

• Teachers. There is a need to place emphasis on the HOW and WHY of learning, to explore further the collaborative dimension, to reflect on the educational interest a DIY digital object can have beyond the course, and to fully integrate the DIYLabs philosophy into the courses, trying not to see it as something added, but something that actually merges with the subject matter and the teaching and learning methodologies inside and outside the classroom. Due to the huge proliferation of digital technologies, more time is needed to explore them personally and with the students.
• Students. The need for more time to work in the classroom –including teachers’ feedback and advice; of clarifying the link between the DIYLabs and the development of the inquiry-based project of the course; of increasing the level of technological knowledge. For them, teachers should present and explain all existing digital resources so that students can choose the most appropriate for their work. They should also provide clearer guidelines regarding the content and objectives of the assignment.

At this educational level, the biggest challenges for teachers and students were also time (for preparation, working in the classes, exploring new digital resources, etc.) and the development of pedagogical resources and predisposition to foster students and digital technology-centred teaching and learning approaches.

3.5. Building on Experience: Making Improvements to the DIYLab

The analysis of the focus groups implemented in this phase of the project allowed the identification of the degree of satisfaction among teachers, students and parents for the students learning outcomes, motivation and engagement. It has also provided a global view of the challenges involved for its institutional sustainability and its socio-economic costs and benefits. The participants in the focus groups, but especially all level teachers, underlined as a fundamental result of the project the closed and collaborative relationship among them. Primary and secondary school teachers highly valued the possibility of exchanging knowledge, views, perspectives and concerns with university teachers while feeling accompanied by them. For university teachers, many of them involved in the initial and in-service professional development of primary and secondary teachers, was an unmatched opportunity to learn about the real situation of schools.

Coherent with the aim of this paper, the following section discusses the results from the socio-economic dimensions of the project. In particular, the project’s potential for reducing institutional disaffection and dropout rates; the mitigation of teachers’ fear and anxiety regarding digital technologies; and the impact on the digital divide. We also discuss the economic impact in terms of setting up and maintaining a DIYLab; the equipment and software needed; and the time investment needed by teachers and other educational actors.

4. Discussion

DIYLab is a relatively small European project that aims to introduce a relatively big change in three primary and secondary schools and two universities from three European countries. In this kind of project, cost-efficacy is understood as the ability of the partners to meet the project objectives and perform the planned activities in time and with an acceptable level of quality. However, when one of the aims of the project is to foster its sustainability beyond the implementation time, the socio-economic aspects must not only indicate the immediate economic requirements for starting, but also the capacity for providing a lasting impact beyond initial funding. So when speaking about socio-economic evaluation we have to avoid “an overly reductionist approach […] by acknowledging the social, human, professional and institutional costs alongside the purely economic costs of educational change”. This is a first step “in unlocking the complex nature of such an undertaking.” [50] (p. 1177).

The DIYLab project identified a set of possible educational-social benefits and a set of costs to be taken into account to foster its sustainability and possible scalability to other contexts (see

Table 4. DIYlab educational-social benefits.

Educational-Social Benefits	Collateral Effects
- Potential mitigation of teachers’ fear and anxiety regarding digital technologies	- The proliferation of DT resources and applications overwhelms teachers with no time to be updated.
- Potential reduction of school/university disaffection and dropout rates	- Tensions between a self-managed philosophy and the obligations imposed by educational institutions.
- Potential reduction of the digital divide.	- The intensive technological development and slow investments in education can increase social inequalities and the digital divide.

and

Table 5. DIYlab costs.

Costs	Implies
- Setting up and maintaining a DIYLab	- Cost related to the analysis of current teaching practices, the professional development of educators, and the design of the DIYLabs (how, why, where, who, which equipment).
- Time investment needed by educators and other educational actors, also taking students into account.	- Who will pay the cost related to the time invested in the different phases of the innovation? Will it be the responsibility of teachers or the institution? Should the teaching and learning schedule be changed? Should the institutional space be modified?
- Equipment and software needed	- Cost related to the acquisition and maintenance of the technological infrastructure. Previewing the social impact of bring-your-own-devise philosophy.

).

The first foundations of the project sustainability were laid in the CAR process that converted university researchers, schoolteachers, students, and to some extent parents, into learning communities. Teachers and students explored their current institutional contexts and perceived needs. Schoolteachers and researchers implemented the professional development workshops in collaboration and decided the content and shape of the DIYLabs for their educational level. Periodical meetings at country and consortium levels followed up the implementation of the DIYLabs and the project as a whole.

The setting up and maintenance of a DIYLab involves a large amount of time, knowledge and skills. In our case, after investing six months in exploring, in each institution, the initial strengths and weaknesses to implement DIYLabs, another six months were committed to the professional development of teachers, which included pedagogical and technological knowledge and skills. The implementation or the DIYLabs throughout a whole year involved an extra concern, time and work for teachers and students. Both of them had to develop new knowledge and skills recognised as thrilling, but also as challenging ways of learning. The time issue was repeatedly pointed out by teachers and students; a question that will gain weight once the partially financed project is over if the involved institutions want to sustain the innovation implemented. This dimension has to be carefully taken into account by any institution wanting to achieve the same aims.

Authors such as Eva Illouz [51] reject the conventional idea that capitalism has designed a society dominated by rationality and bureaucracy in which economic behaviour conflicts with emotions and intimate, authentic relationships; in which the public and private spheres are antagonistic and that true love is opposed to calculation and self-interest. On the contrary, for this author, capitalism has fostered an intensely emotional culture—in the workplace, in the family, and in our own relationships.

In this context, despite the constant attempts of converting education into an effective bureaucratic machine, education is profoundly permeated by emotions, love (or hate), and the idea of care, service and contribution to societal improvement underpins most educators’ behaviour, even at higher levels. As a result, due to their feeling of responsibility, educators may end up paying an important part of the cost of the innovation with their time and extra-emotional involvement. At this point fundamental questions are:

• Who will pay the cost related to the time invested in the different phases of the innovation?
• Will it be the responsibility of the teachers or the institution?
• Should the teaching and learning schedule and spaces be changed?
• If so, what will be the implications for teachers, students, parents and families?

Another set of costs relates to the need of updating equipment and software. Institutions participating in the project were relatively well equipped and, in most cases, students have access to digital resources outside the institutions. However, some teachers felt overwhelmed and complained about the extra work that the project represented and the lack of finance for equipment. This kind of cost is particularly worrying at a time of intense technological development and slow investment in education. In this context, educational institutions may have a hard time keeping their equipment updated and paying the maintenance bills. On the other hand, the bring-your-own-devise movement, which shifts the economic pressure onto families, is gaining ground. As a result, social inequalities do not only remain but can be increased by widening the digital divide.

The professional learning context created by the project contributed to mitigating teachers’ fear and anxiety regarding digital technologies. However, as it has been glimpsed at in the previous section, teachers felt somehow more pressurised when realizing they would need much more time and more knowledge about inquiry-based teaching and learning and about DT. The speed and the amazing proliferation of DT resources and applications overwhelmed not only all educational level teachers, but also some students and parents who could not find the time to be updated. The need for institutions to provide time and professional development for teachers increases the cost. This is particularly significant at university level, due to the fact that academics have to add research and publications to their professional agenda.

Students felt they had more freedom to pursue their own interest and be more engaged inside and outside institutional activities. This is an indicator of the project’s potential to reduce institutional disaffection and dropout rates. However, those students that have found their comfort zone [52,53] in highly regulated educational contexts, teacher, and factual knowledge and test-centred, had a hard time when asked to think and act by themselves. On the other hand, it also opens a fundamental challenge related to the social and digital divide, as not all institutions and families have access to the same kind of equipment, not all students have the same cultural and social capital [54] and some of them may suffer a new kind of social and educational discrimination.

5. Conclusions

A long time ago, writers such as Seymour Sarason [55] pointed out the predictable failure of educational reforms. Michael Fullan [56] argued that transforming a secondary school requires more than 6 years. While Bjon Stensaker, Jussi Välimaa and Claudia Sarrico showed up the complexities of transforming higher education institutions [57].

At the same time, since the computer age, authors such as Seymour Papert [58] and many others, followed by corporations, are granting digital technologies with the magic power for transforming education.

Digital technologies are defining new landscapes for all human spheres, posing unprecedented challenges for education and training. In this context, DT, more than a straightforward answer to educational challenges is becoming a new problem to be solved by people, sometimes with little knowledge and understanding of these technologies. The challenges of this new scenario involve rethinking the organizational metaphor of educational institutions as a whole, putting into question the notion of knowledge, the visions about teaching and learning, the role of teachers, students and administrator, the organization of time and space, the function of assessment, the concept of leadership, etc. [59,60].

In the DIYLab project, we aimed to deeply and sustainably transform teaching and learning practice in primary and secondary schools and higher education institutions by introducing enhancing synergy between these three educational levels and the DIY philosophy, in order to expand digital competence and foster student agency and collaborative learning. For this reason, we paid special attention to the institutional dimensions of innovation and considered the DIY philosophy as a way of making the most of digital technology and learning.

Considering the socio-economic dimensions of the project, it has allowed us to make explicit the possibilities, tensions and difficulties we have encountered. This knowledge will guide our next steps in innovation, once the European project is over, and can help educators and institutions to responsibly implement digital technology-driven innovations.