“Sustainable City”: A Steam Project Using Robotics to Bring the City of the Future to Primary Education Students

Abstract

This article presents a STEAM project, “Sustainable City”, based on educational robotics whose objective is to bring the problems of climate change closer to primary school students. The participants were 30 students (aged 10–11) from 5th year of Primary Education and the project consisted of 14 sessions in which different active methodologies such as project-based learning, collaborative learning and the flipped classroom were carried out. A pre-test and a post-test were carried out and the results were better in the post-test, which indicates that the students improved their skills and increased their motivation. As a conclusion, this STEAM project, which is adapted to the Spanish LOMCE curriculum, can be a reference for other teachers to bring their students closer to the problem of environmental change through educational robotics and the use of active methodologies.

1. Introduction

Climate change, together with global warming caused by the increased concentration of greenhouse gases in the atmosphere, is probably the greatest environmental threat facing today’s society.The greenhouse effect is a natural process in which certain gases in the atmosphere retain part of the energy emitted by the ground when heated by solar radiation allowing the Earth to maintain the conditions that make life possible. Since the Industrial Revolution, human activities related to the use of fossil fuels, such as coal and oil, have exponentially increased the emission of greenhouse gases, producing a warming of the atmosphere that is triggering a change in the climate of the planet. In addition to environmental consequences such as rising temperatures, rising sea levels, the extinction of species or extreme weather events with natural disasters, climate change will also have social consequences such as population displacement, conflicts over the use and exploitation of natural resources, increased poverty, spread of diseases, food and water shortages.The forecasts are not optimistic, but the Intergovernmental Panel on Climate Change [1] estimates that, with urgent actions, the consequences can still be mitigated, therefore it intends to do this by reducing the emission of greenhouse gases and the burning of fossil fuels, restoring habitats and ecosystems, creating protected areas, promoting renewable energy, properly managing water, prioritizing measures for the most vulnerable and promoting climate education and literacy. The IPCC proposed, for the first time in 2019 [1], climate education and literacy to enhance society’s learning and resilience and preserve the environment; it also recommended climate literacy to improve awareness, understanding and social learning about the specific risks of climate change for communities. These proposals pose a challenge for educational systems and schools, which must adapt their educational models to promote education and literacy on climate change. In this work, based on a broader one [2], we develop the design, implementation and validation of a STEAM project, “Sustainable City”, based on educational robotics and whose objective is to bring the problem of climate change closer to Primary Education students. The project presents an inclusive, resilient and sustainable model for a city of the future, capable of adapting to the social and environmental changes that climate change may cause by using resources efficiently and providing a reduced environmental impact. Active methodologies are used in this project, such as project-based learning, collaborative learning and the flipped classroom (FC), in which the students become the protagonists of the teaching-learning process and build their own knowledge.

2. Theoretical Framework

2.1. Steam Projects with Educational Robotics

2.1.1. Steam Learning

The Fourth Industrial Revolution, or Industry 4.0, marked by the convergence of digital, physical and biological technologies, is changing society, modifying “the way we live, work and interact” [3]. One of the aspects being changed is the labour market and the way to access it, in such a way that, in the near future, technological knowledge will be essential for 80% of workers and the required qualifications to join the labour market will be assessed by technological skills [4], many jobs will disappear or be replaced by machines and new types of unthinkable jobs will be created [5]. These changes are forcing current educational systems, many of them designed to meet the needs of the last century, to face the challenges of the future [6], redefining their educational models and curricula to provide students with competencies, attitudes and experiences, in a digital, innovative and creative way, in order to adapt to the new society. In response to this situation, STEM learning (Science, Technology, Engineering and Mathematics) was developed in the first decade of this century to provide students with the acquisition of scientific-technological skills. Subsequently, to overcome the integration problems of the four disciplines that emerged in the implementation of STEM learning and to provide greater creativity and innovation. Ref. [7] proposed to incorporate Art as a new discipline, thus emerging STEAM learning (Sciences, Technology, Engineering, Art and Mathematics). In this way, STEAM learning is an educational model whose objective is to integrate and develop scientific-technological and artistic subjects in a single interdisciplinary framework. In this interdisciplinary approach, each discipline provides its characteristics and particularities to STEAM learning: Science provides the scientific method, Technology (T) its transversal character and its predisposition to adapt to social demands, Engineering (E) develops the design of systems, components and processes, Art (A) provides its creative and integrating character and Mathematics (M) provides a common language to the rest of the disciplines. Ref. [7] defined STEAM learning as the learning of “Science and Technology interpreted by Engineering and Art all based on mathematical elements“ and summarized its theoretical framework in a pyramid with five layers: she placed the contents in isolation at the base; in the second stratum, she placed, also in isolation, the five STEAM disciplines; in the third stratum she linked the STEM disciplines and kept Art separate; in the fourth stratum she placed STEAM learning, which uses Art as a transversal and integrating element; and at the top she placed the holistic theories that represent learning in a totally integral and interdisciplinary way.

2.1.2. Educational Robotics

An effective proposal to introduce STEAM learning into educational contexts is educational robotics, which is presented as an interdisciplinary teaching environment that promotes the development of scientific-technological skills and competencies in order to develop the construction, programming and manipulation of robotic platforms [8]. Educational robotics is based on the constructivism of [9], who considers that manipulation is key in the construction of knowledge, and especially on the constructionism of [10], who states that the construction of knowledge is more effective when the students can build their own resources. The expansion of robotics in various areas of society has also been reflected in the classrooms of Primary Education where educational robotics has been continuously introduced and developed. The generalization of low-cost platforms such as Arduino, Raspberry Pi or Kinect and mobile robots such as Adept, Moway, Epuck and Lego Mindstorms has contributed to this development. Although STEAM projects based on educational robotics are still in the empirical testing phase, recent research presents educational robotics as a didactic tool capable of improving learning, developing technological competencies and creative skills, and increasing educational performance, interest and motivation, collaborative work and social skills [2]. Several studies have shown a considerable improvement in the students’ performance when they use robots in their learning: [11] quantified this improvement in student performance by 57% and [12] by more than a 37%. Other studies have shown a considerable increase in interest and motivation by students when they work on STEAM subjects with robots [13,14] and indicate as motivating factors the inclusion of Art in STEAM projects [15] and the construction and assembly of the robot [16]. In addition, the use of robots and robotic platforms encourages students to discuss topics among peers, solve problems together, and share common goals, making educational robotics an effective tool for developing collaborative work and social skills [17,18].

2.1.3. Active Methodologies

STEAM projects in general, and in particular those that use educational robotics, promote the use of so-called active methodologies encouraging the active participation of the student, who becomes the protagonist of the teaching-learning process and develops his/her own knowledge. Active methodologies such as project-based learning, problem-based learning, collaborative learning or the flipped classroom, are revealed as effective tools to generate meaningful learning and to train critical, creative people who will be prepared to face current and future challenges and be able to work in a team, communicate, discuss, evaluate.

Project-based learning is the active methodology inherent in educational robotics projects, since the aim is to develop a final product from an initial challenge. The development of the project, which creates the final product, is carried out in four phases: presentation of the challenge, implementation of the project, presentation of the product and final reflection and evaluation.

Thus, project-based learning is defined as a set of tasks that culminate in a final product and that students must solve autonomously through research processes [19]. It is a holistic strategy in which interaction with reality beyond the classroom is enhanced [20].

In the development of the final product, new challenges or problems arise and require problem-based learning, in which students, while solving the problems, inductively build new knowledge. The problem-based learning process begins with the recognition and identification of the problem, continues with the discussion of possible hypotheses and the choice and justification of solutions, and ends with the presentation of the solution obtained.

Problem-based learning places the student in a confusing, unstructured situation, before which they assume the leading role: they identify the real problem and learn, through research, what is necessary to reach a viable solution [21,22,23]. In [21], problem-based learning within primary education is a pedagogical experience developed to investigate and solve problems, it is an organizer of the curriculum and also a teaching strategy [24,25,26].

Both methodologies use collaborative learning, so that work is carried out in small groups in which students with different skills and levels share situations and address common goals. In cooperative learning, each student is responsible for their own learning and that of their peers, so that they will reach their goals if, and only if, the peers also achieve theirs. The flipped classroom is often used in educational robotics projects, in which students learn the contents at home, usually from online tutorials, and then carry out classroom activities and tasks guided by the teacher. In the flipped classroom, although a greater student involvement is required in learning, it is adapted better to the student’s pace, providing deeper and more meaningful learning. Other active methodologies that can be used in STEAM projects with educational robotics are competency-based learning, contract learning, gamification, design thinking, or case study, [27,28]. The implementation of active methodologies in STEAM projects with educational robotics implies a new organizational structure of the classroom, a different way of managing time and assessment systems, and a change in the teacher’s role and training, which represent a challenge for the teachers.

The use of these active methodologies in the context of educational robotics as a tool to develop STEAM learning situations opens up a range of possibilities to implement different projects in the classroom so that students learn in a motivating way. One of these motivating projects for students can be the design and building of sustainable cities.

2.1.4. Evaluation Tools

If an evaluation tool is considered to be any pedagogical action that informs the student of their progress and leads them to a reflective and critical review, it can be understood that project-based learning is a methodology that incorporates frequent evaluation processes by constantly including activities of reflection, criticism and revision, [29]. The same happens with problem-based learning or cooperative learning activities, all of them active methodologies that also facilitate the emergence of evaluation processes through reflection, criticism and review, developing a continuous and formative evaluation, [30,31,32].

In the context of a STEAM learning project, an initial evaluation, a continuous evaluation, also called formative, and a final evaluation should be incorporated. Adapting this scheme to the methodological scheme defined for these projects, (1) the initial evaluation will be carried out during the project presentation phases and must provide the necessary information to adapt the rest of the process to the student’s characteristics, (2) the formative evaluation will be carried out during the development of the STEAM project and will be marked, especially, by reflection, criticism and review activities and (3) the final evaluation will be carried out in the product presentation phase and will include activities on learned and reflective concepts and global assessment of the project. The inclusion of the final evaluation within this last phase does not prevent it from having a summative aspect that includes results from the other two evaluation phases, since, complying with the legal provisions, this last phase will determine a final numerical grade.

2.2. Sustainable Cities

Cities, despite occupying 3% of the earth’s surface, are home to 56% of the world’s population, consume between 60 and 80% of energy and produce 75% of CO₂ emissions and other greenhouse gases (United Nations, 2015). They are, therefore, the largest contributor to climate change, mainly due to the generation and consumption of energy, transport and industry, [33]. Cities, in addition to being the largest contributor to climate change, are the most vulnerable environment, since many of them will be affected by rising sea levels, by rainfall and floods, and by the increase in extreme temperatures. These impacts will unleash important consequences in the life quality of many citizens, especially in the most vulnerable population. From this perspective, the dual role of cities in climate change, as an impact agent and as an environment vulnerable to negative effects, makes the participation of cities in the challenge of sustainability fundamental. The 2030 Agenda, approved by the United Nations General Assembly in 2015, lists 17 sustainable development goals and among these, objective 11 has been highlighted, which aims to “make cities and human settlements inclusive, safe, resilient and sustainable” and which proposes in objective 11b that cities “adopt and implement integrated policies and plans to promote inclusion, efficient use of resources, mitigation of climate change [34,35]. A sustainable city is a resilient city that can adapt to economic, social and environmental changes and uses resources efficiently. Among the aspects addressed in the sustainability agenda for cities, climate change is, in some way, the common denominator since in order to achieve cities with zero emissions and with reduced environmental impact, cities must rely on four factors for climate action: (1) sustainable mobility, (2) biodiversity and green areas, (3) renewable energy and energy efficiency, and (4) infrastructure and services. The number of vehicles, 1.2 billion today and about 2 billion in 2035, causes, especially in large urban centres, mobility problems, such as air pollution, noise, congestion. Therefore, solutions are required that help reduce emissions and fight against climate change; these solutions to improve mobility range from promoting public transport and building connections to promoting the use of electric cars and bicycles. The proper planning that will improve the life quality of citizens, encourage innovation and production, increase resilience to environmental impacts and reduce vulnerability and risks linked especially to climate change must include green areas. These will improve biodiversity, control pollution, reduce noise, regulate the urban climate, favouring air circulation and lowering temperatures, and work as carbon sinks. Improvements in energy efficiency in cities are associated with the production of renewable energies and the decrease in consumption. The energy consumption of buildings is responsible for 40% of greenhouse gas emissions and it is estimated that in 2030 it could be reduced by half by changing lighting systems, insulating homes and using alternative and more efficient cooling and heating systems. On the other hand, the promotion of alternative fuels and the efficient integration of renewable energies will be essential. Through existing infrastructures, companies and systems, a sustainable city will offer citizens services that allow them to adopt and maintain a healthy and sustainable lifestyle. In this way, a sustainable city must have an impact on providing citizens with effective waste management, efficient and non-polluting public transport, affordable housing.This STEAM project of “Sustainable City” that uses educational robotics as a didactic tool, presents the students with six challenges related to these aspects: The six challenges are related to fundamental aspects of sustainability such as mobility, biodiversity, efficient and renewable energy, and infrastructure and services. Good mobility in cities will help them to have a sustainable development and the challenge carried out with this project will be the building of a marquee for traffic regulation. To work on biodiversity the students will build a sustainable roof. In relation to efficient and renewable energies, the students will build a wind turbine, a photovoltaic field and streetlights for urban lighting. Finally, with regard to infrastructure and services, students will build a waste sorting plant. One of the main novelties of this STEAM project is that it adapts to the Spanish curriculum (LOMCE), which implies that it can be used as a substitute for traditional teaching.

3. Project Details

3.1. Participants and Context

In the implementation of the project, 30 students from 5th year of Primary Education participated with the collaboration of the tutor teacher. The project was carried out during 15 sessions of 50 min, distributed in two weekly sessions linked to the areas of Natural Sciences and Social Sciences. The studends were divided by groups. The groups were as heterogeneous as possible to avoid that students with low social skills coincided in the same group, so the comments of the teacher-tutor were very much taken into account.

The project was developed as another curricular activity within school hours. Due to the age of the students, from 5th year of Primary Education, the study had to be developed in the intermediate range of planned sessions, from 17 to 20 sessions. However, two factors accelerated the implementation process: on the one hand, the continuous presence of two teachers in the classroom, the regular teacher of the students and the researcher, and on the other hand, 14 of the 30 students had already worked after school hours with the robotics components used in the project and with Bitbloq, the software used in programming. Of the 30 students, 13 were boys and 17 girls.

3.2. Especialization, Challenges and Roles

In the project, the participants must build a board with different elements that will make up a sustainable city and program a robot to follow a line marked on the board and activate the different elements of the sustainable city. The implementation of the project is carried out in two phases. In the first phase, the specializations (experts) and individual challenges are assigned to each of the three components of each base team:

• Expert in biodiversity and energy
  • Challenge 1. Build a sustainable garden or orchard in a city (Sustainable roof).
  • Challenge 2. Create an energy source that takes advantage of wind gusts (Wind Turbine).
• Expert in mobility and energy
  • Challenge 3. Build a marquee that reacts to traffic and emits light (Marquee).
  • Challenge 4. Create an energy source that takes advantage of sunlight. (Photovoltaic field).
• Expert in efficiency and recycling
  • Challenge 5. Create a lighting system that optimizes and saves light (Urban lighting).
  • Challenge 6. Make a building for recycling (waste separation).

In the second phase, the functions or roles to be performed in the final assembly are assigned:

• Designer, who must design the final appearance of the product and agree on the decisions that imply a certain degree of creative freedom.
• Assembler, who must assemble all the pieces of the board.
• Programmer, who must program the control panels of the board and the robot.

3.3. Materials and Resources

• Robotics kit (Figure 1)
  • Robot adapted from BQ’s PrintBot Renacuajo.

    –

    A line follower sensor.

    –

    3D printed PLA plastic casing.

  • Board made of PLA plastic printed in 3D.

    –

    Removable crosspieces structure.

    –

    25 tiles sized 15 × 15 cm.

    –

    Components to build the six challenges.

  • ZUM robotics kit compatible with Arduino Uno.

    –

    ZUM Control Panel.

    –

    Three IR sensors.

    –

    Three LEDs.

    –

    Two miniservos.

    –

    One continuously rotating servo.

    –

    USB cable.

• Bitbloq online software, developed by BQ.
• Virtual learning environment.
  • http://ciudadsostenible.adoce.es
  • Learning guides

    –

    Presentation guide.

    –

    Research guide.

    –

    Implementation guide.

    –

    Product presentation guide.

The total production cost of the robotics kit (robot and board) amounts to 177.99 €, of which 115.73 € can be recovered corresponding to the electronics items for other projects.

3.4. Project Timing

Table 1. Summary of project development.

Stage	Session	Tasks	Grouping
Project presentation	1	1. Pre-test: Competence test in sustainability	Individual
		2. Triggering activity (video 1)	Large team
	2	3. Challenge presentation (video 2)	Large team
Project implementation		4. Formation of base teams	Large team
		5. Introductory FC (video 3)	Individual
	3	6. Presentation of material	Large team
		7. Distribution of specialization roles	Base team
		8. Specialization FC (Expert roles)	Individual
	4	9. Expert groups formation	Large team
		10. Brainstorming	Expert Group
		111.a. Assembly and programming of challenges	Individual.
	5	111.b. Assembly and programming of challenges	Individual.
	6	111.c. Assembly and programming of challenges	Individual.
		12. Self-assessment group of experts and individual	Individual.
	7	13. Presentation of experts to the base team	Base team
		14. Evaluation of experts in the base team	Base team
		15. Implementation FC (virtual environment guides)	Individual
	8	16. Brainstorming	Base team
		17. Task sharing	Base team
		18.a. Programming, design and assembly	Individual
	9	18.b. Programming, design and assembly	Individual
	10	19. Presentation of individual progress to the base team	Base team
		20. Brainstorming and improvement proposals	Base team
		21.a. Programming, design and assembly	Individual.
	11	21.b. Programming, design and assembly	Individual.
Presentation of project	12	22. Presentation preparation	Base team
		23. Presentation practice	Base team
Evaluation	14	25. Evaluation and self-evaluations	Individual
		26. Final test	Individual
		27. Post-test: Competence test in sustainability	Individual

summarizes the development of the project, specifying the sessions, the stages, the tasks and the groupings.

3.5. Development by Sessions

Session 1

• The project begins with a pre-test or “Competence test in sustainability” (questionnaire 1), which is also used as a post-test at the end of the project and is used to compare results and evaluate the effectiveness of the project.
• As a triggering activity, the video “The wake-up call” is played and a debate on sustainability is carried out. (https://www.youtube.com/playlist?list=PLkzKI1ucDKjr0jTANtCNUN-iqqzzZU8LI (Video 1)).

Session 2

• The video “The Sustainable City” is watched and discussed and the teacher shows how the robot travels a previously assembled board. (https://www.youtube.com/playlist?list=PLkzKI1ucDKjr0jTANtCNUN-iqqzzZU8LI (Video 2)).
• The teacher, based on the pre-test results and the students’ knowledge, makes up 10 heterogeneous teams of three students each.
• At home, students watch the video “5 components of sustainability” (green areas, renewable energy, mobility, efficiency and waste management) and answer a questionnaire (questionnaire 2) on sustainable cities. (https://www.youtube.com/watch?v=FP0Wwyv2h9U&list=PLkzKI1ucDKjr0jTANtCNUN-iqqzzZU8LI&index=3 (Video 3)).

Session 3

• The robotics kit (robot, board and guides) and the virtual learning environment are presented and distributed to the students, assigning the access passwords.
• The expert roles are distributed to each student of the base team and the individual challenges of each specialty:
  • Biodiversity and energy expert (sustainable roof and wind turbine).
  • Mobility and energy expert (traffic marquee and photovoltaic field).
  • Efficiency and waste expert (urban lighting and waste separation).
• 8. At home, each student accesses the virtual environment, watches the video related to their specialty “What is... (biodiversity and renewable energy, mobility and renewable energy or efficiency and waste management)?” and answers a questionnaire (questionnaire 3).
  • https://vimeo.com/199938416 (vídeo 4)
  • https://vimeo.com/199960641 (vídeo 5)
  • https://vimeo.com/199997024 (vídeo 6)

Session 4

• Groups of experts are formed (six groups of three students and three groups of four students with the same specialization, three groups for each specialty).
• The corresponding material is distributed to each student who studies the instructions in the research guides for their two challenges; within each group of experts, a brainstorm is held: each student writes and draws their proposal for mounting the assigned challenges, explains their ideas to their colleagues, discusses the ideas and selects the best option.
• Students begin to individually assemble their two challenges with the help of the research guide and the other peers in their team of experts.

Session 5

• Students continue to develop their challenges that began in session 4-3.

Session 6

• Students finish assembling their challenges and present them to their peers in the expert team.
• The self-evaluations of the team of experts and the student’s own are carried out individually, answering two questionnaires (questionnaires 4 and 5).

Session 7

• The students are regrouped in the base teams and each expert presents to their classmates the assembly of their two challenges and explains the theoretical concepts learned.
• Each student expresses their opinion about the assembly and the explanation of their other two peers from the base team through a questionnaire (questionnaire 6)
• At home and individually, each student studies the implementation guides (design, assembly and programming of the board) found in the virtual environment and answers a new questionnaire (questionnaire 7).

Session 8

• After studying the implementation guide, brainstorming is held in the base groups: each student writes and draws their proposal for the design, assembly and programming of the board, they discuss the ideas and select the best option.
• The tasks to be carried out are distributed and the roles of each student in the base group are assigned:
  • Programmer
  • Designer
  • Assembler
• The students individually carry out the tasks assigned in the base team according to their role.

Session 9

• Students continue to work on tasks assigned in task 8-3

Session 10

• Each student presents their progress to the rest of the base team.
• A new brainstorm is held in each base group: students write and draw proposals to improve the work of their classmates and the best proposals are selected.
• The students resume the tasks cooperatively incorporating the selected proposals.

Session 11

• The students finish the programming, the design and the assembly of the board and check its operation which started in tasks 10-2

Session 12

• Each base team, following the presentation guide, reviews their board and prepares the presentation in which the team will show their final product to all the classmates.
• The three students practice the presentation of the final product, jointly agree on the final script of the presentation and continue practicing

Session 13

• Before the presentation of each group, the student who will carry it out is chosen by lots. The representatives of the ten base groups present, for 5 min, the final product: they show and explain how the board works, place the robot on the board and make it go through the established itinerary and finally explain the characteristics of the design, assembly and programming of their product.

Session 14

• Each student evaluates the work of the base team and their own work within the base team, with the same questionnaire used to evaluate the group of experts (questionnaires 4 and 5) and the teacher evaluates the presentation of the final product of each base team (questionnaire 8).
• The students take, individually, the final test (questionnaire 9).
• Students take the “competence test in sustainability” (questionnaire 1) again.

3.6. Active Methodologies Used

The main active methodologies used in the project are project-based learning, problem-based learning, collaborative learning and the flipped classroom. The initial challenge of the project is the construction, programming and presentation of a robotic platform composed of a board on which a line-following robot circulates activating the elements that make the city a sustainable place (sustainable roof, wind turbine, traffic marquee, photovoltaic field, urban lighting and waste separation), so methodologically project-based learning is used. In addition, the project development follows the four phases of project-based learning: project presentation, project implementation, final product presentation, and discussion and evaluation. There are two phases in the project: in the first phase each student assembles their two individual challenges within the expert groups and in the second phase they program, design or assemble the boards within the base teams. In both phases the structure of problem-based learning is followed: identification of the problem, discussion of possible solutions, choice and justification of solutions and presentation of solution to peers. Throughout the project, in addition to individual work and the large team or class group, two other groups are used: expert teams and base teams. In the expert groups the individual challenges are assembled and in the base groups the board is programmed, designed and assembled and the product is subsequently presented. In both groups, collaborative learning is promoted, especially in the base teams, since each student takes responsibility for their own learning and their peers’ learning and achieves their objectives if, and only if, their classmates also achieve them.Three flipped classroom (FC) sessions are included in the project: introductory, specialization and implementation. The introductory FC provides information on the conceptual framework of the project, the specialization FC defines the different specialties and individual challenges, and the implementation FC explains the design, assembly and programming of the board.

3.7. Evaluation

The project incorporates nine questionnaires with which the teacher evaluates the knowledge of the students and the work groups and each student evaluates their own work, that of their peers’ base teams and that of their own teams. The questionnaires were developed by the authors. Each one contains 15 questions of different types: six for essay, four for multiple choice, four for matching with arrows, and one for true or false. Regarding the project contents and its comprehension goals, the questionnaires were organized around five concepts: efficiency, renewable energies, mobility, green areas and waste management:

• Questionnaire 1. Pre-test and post-test: STEAM competence test in sustainability (S1-T1 and S14-T27).
• Questionnaire 2. Introductory FC: Five components of the sustainable City (S2-T5).
• Questionnaire 3 Specialization FC: Questionnaires for experts (S3-T8).
• Questionnaire 4. Group self-assessments: expert teams (S6-T12) and base teams (S15-T25).
• Questionnaire 5. Individual self-assessments: in expert teams (S6-T12) and in base teams (S15-T25).
• Questionnaire 6. Evaluation of the explanation of each expert to the base team (S7-T-14).
• Questionnaire 7. Implementation FC Questionnaire (S7-T15).
• Questionnaire 8. Evaluation by the teacher of final product presentation of each base team (S14-T25).
• Questionnaire 9. Final test on the development of the project (S14-T26).

4. Results

Pictures of the process can be seen in the Appendix A. The students were given a sustainability competence test, see Appendix B.1. This test was intended to measure the sustainability knowledge of the students at the beginning of the project (pre-test) and at the end of the project (post-test) and their progress throughout the project. The test consists of 15 questions of different types (six essay tests, four multiple choice tests, four matching tests and one true/false test) which are organized around five concepts: green areas, renewable energy, urban mobility, waste efficiency and management.

Table 2. Results of the competence test in sustainability.

	Pre-Test	Post-Test	Increase
			Score	Coefficient
Green areas	3.4	6.9	+3.5	2.0
Renewable energies	2.9	5.0	+2.0	1.7
Urban mobility	3.9	6.5	+2.7	1.7
Efficiency	1.9	6.2	+4.3	3.3
Waste management	2.3	6.0	+3.7	2.6
Complete test	2.9	6.1	+3.2	2.1

shows the average score of the results obtained by the 30 students in the five concepts and in the complete test. We can see that the overall performance doubled, from 2.9 in the pre-test to 6.1 in the post-test, with the efficiency concept being the one that increased the most and those of renewable energies and mobility the least.

The reliability of the test was carried out through the internal consistency study using Cronbach’s Alpha coefficient and the Kurder-Richardson 20 method (KR 20). The values of both oscillate between 0 and 1, with 1 being maximum reliability and considering as very high reliability the values higher than 0.80.

Table 3. Reliability of the competence test in sustainability.

Aspects	Cronbach’s Alpha	KR 20
Green areas	0.62	0.71
Renewable energies	0.88	0.88
Urban mobility	0.66	0.73
Efficiency	0.88	0.89
Waste management	0.88	0.89

shows the values obtained in both studies, observing high or very high results in the concepts evaluated, and very high results in the complete test in which a 0.96 was obtained in both studies; these results sufficiently guarantee the reliability of the test and its internal consistency.

4.1. Results of the Evaluation System Incorporated in the Project

Table 4. Results of the evaluation system of the project.

Quest.	Session	Task	Evaluation Activity	Grades	Coefficient	Grades
2	2	5	Introductory Flipped Classroom	7.00	0.05	0.35
3	3	8	Flipped classroom of experts	7.60	0.05	0.38
4 and 5	6	12	Expert team and individua self-evaluation 1	7.10	0.10	0.71
6	7	14	Explanation by expert in base team	7.60	0.10	0.76
7	7	15	Implementation Flipped Classroom	8.00	0.05	0.40
8	15	25	Presentation of final product	6.80	0.35	2.38
4 and 5	15	25	Base team and individual self-evaluationl 2	7.20	0.10	0.72
9	15	26	Final test	6.50	0.20	1.30
			Total	7.23	1.00	7.00

shows the average results obtained by the students in the evaluation activities carried out throughout the project, the weighting coefficient of each questionnaire and the final grade of the project.

The scores of the questionnaires are between 6.5 of the final test and 8 of the implementation FC, the average being 7.23. The penultimate column shows the weighting coefficient assigned to each activity according to its importance in the development of the project, and the last column shows the contribution of each activity to the overall rating of the project, which is 7 out of 10. The activity with the highest weighting in the project was the presentation of the final product with 35% of the project’s rating and a rating of 6.80. In the presentation, although the boards had different designs and resolutions, all the robots completed the circuit on all the boards following the line and all the mechanisms were activated. The second activity with the highest weighting was the final test, Appendix B.2, with 20%. The questionnaire consisted of 10 multiple-choice questions: one question on sustainability in general and two on each of the aspects studied, except efficiency with one question. The average grade was 6.5 and the grades for all questions were above 5. The average grade of the three questionnaires related to the flipped classroom sessions was 7.53 and the average of the evaluations and self-evaluations of the students was 7.3.

Figure 2 shows the numerical grades and the ones according to LOMCE (Organic Law for the Improvement of Educational Quality), obtained by each of the 30 students. In the numerical qualifications it is observed that all the qualifications are equal to or greater than 5 and that the most repeated qualification has been 7, and according to LOMCE all qualifications are equal to or greater than Pass grade and the most repeated has been Merit grade.

4.2. Comments

Throughout all the sessions, a very high degree of motivation and interest has been observed in all the students who have felt highly identified with the project, have discussed among their peers, have solved problems in a collaborative way and have shared objectives.The results of the introductory FC session were used to form the base teams and those of the expert FC to form the expert teams. All groups closely followed the expected pace in collaborative activities, showing a high level of individual and group responsibility. The pace of individual tasks varied from one student to another, and those who finished earlier helped the slowest ones, allowing everyone to present their work on time. The students’ degree of commitment throughout the project was very satisfactory, except in the tasks to be carried out at home, since 2 students did not carry out any of the activities proposed for home and 6 did not carry out any of the tasks; the students who did not do the activities at home did them eventually in the classroom.Some students had time problems in the design, assembly and programming phase of the board, although all of them finished their work, with greater or lesser perfection, to incorporate it into the presentation of the final product. The students who demanded more help in this phase were the assemblers, who had difficulties in fastening and connecting cables; some programmers also needed help with program compilation problems, and designers often consulted their peers about their likes and opinions on aesthetics. The time provided for each presentation and demonstration of the product was five minutes and some teams had problems with time, so the session had to be extended.

The review of the state of educational robotics by [36] includes three research studies that are focused exclusively on students aged 9 to 12, those by [12,37,38].Of these studies, only the one by [12] is a practical work of learning through robotics with a certain degree of reliability, while the other two studies work with surveys without actually putting robotics into practice. Similarly, in the reviews by [39,40] only the publication by [12] is shown, which guarantee a certain objectivity and work with students of these ages. The age range from 9 to 12 is, therefore, one of the least studied in the practical empirical studies collected in the reviews consulted regarding the state of educational robotics.

5. Conclusions

This work presents the design, implementation and validation of a STEAM project using educational robotics, in which a sustainable city model is described, capable of adapting to the social and environmental changes that climate change may cause and in which the resources are used efficiently providing a reduced environmental impact. To compare the students’ theoretical and conceptual knowledge, a competence test in sustainability was carried out at the beginning and at the end of the project. The results obtained show that the level of knowledge doubled, increasing from 2.9 to 6.1. The project included a comprehensive and complete evaluation system with eight questionnaires covering three flipped classroom sessions, two group and two individual self-evaluations, an explanation to the base team, the presentation of the final product and a final test. The average rating of the questionnaires was satisfactory, obtaining an average of 7.23, which was reduced to 7 after weighting according to the importance assigned to each questionnaire. The knowledge and concepts about sustainability were especially evaluated in the final test that consisted of 10 multiple-choice questions on sustainability in general and on each of its aspects. The overall result of the test was 6.5. This result was similar to that obtained in the competence test in sustainability applied as a post-test. We can see how the use of robots improved the students’ performance and increased their interest and motivation. This STEAM project can be used by other teachers as a reference to bring their students closer to the problem of environmental change through educational robotics and the use of active methodologies.

The present work leaves the door open to experimentation with new robotics kits that, using or not the same technology, offer sufficient design freedom to be adapted to didactic needs. In this sense, the redesign of the robotics kit used, the “Sustainable City” board, would be one of the lines of work resulting from the research and its simplification to reduce the number of project sessions and costs would be an objective to be included in research that was focused entirely on kit development.

Another line of research that would complete the present work is related to measuring the degree of interdisciplinarity for this type of project. A research study focused on this objective should measure whether there is an improvement in knowledge not only in the main opportunity area, but in each and every one of the conceptual opportunity areas involved. This type of evaluation would allow the extraction of quantitative learning data and the rigorous and totally objective analysis of the degree of interdisciplinarity.

The inclusion of art (A) as part of STEM learning deepens in the improvement of the interdisciplinarity of the project and the emergence of creative solutions. The STEAM analysis of the official curriculum has revealed the scarce curricular variety of artistic content, as the international scientific literature has already confirmed. In this sense, a final line of research has been opened, focusing on the arts education curriculum, completing it, diversifying it and turning it into a true source of creativity in the classroom either by updating its contents or by expanding its field of knowledge.