Contribution of Citizen Science to SDG 4: A Systematic Review of the Evaluation of Learning Outcomes in Citizen Science Projects in Compulsory Education

Abstract

The contribution that the inclusion of CS in the curriculum can make to address SDG 4—Quality Education—which fosters convergence between Science Education and Education for Sustainable Development, essential for addressing the sustainability challenges currently facing humanity, has been widely recognized. This recognition is driving the inclusion of CS in formal education. However, to ensure that the use of CS in formal education contributes to this objective, a systematic and rigorous evaluation of its benefits in terms of participants’ learning outcomes (LO) is necessary. This study presents a systematic review of the published literature on CS projects implemented in compulsory education to examine whether students’ LO from participation in CS projects are evaluated, and if so, how this evaluation is performed. The results indicate a lack of systematic evaluation of LO from participating in CS projects. Moreover, although in 79% of cases, where some LO is evaluated, the evaluation reported positive results, in most of them, the results may have been influenced by the voluntary or mandatory nature of participation in the projects and the design of the evaluation itself. This may bias the results, leading to an over-optimistic view of the contribution of CS to SDG 4. In order to obtain solid evidence of the benefits, or lack thereof, for learners of participation in CS activities, which can guide the designers and educators in improving the CS projects to maximize their educational and sustainability impacts, some recommendations for future studies are presented.

1. Introduction

Education is key to advancing towards a sustainable planet; therefore, SDG 4 aims to ensure quality education. Promoting scientific literacy in society is one of the objectives of quality education [1], as scientific literacy is essential for addressing many of the sustainability challenges humanity faces. However, students’ interest in science decreases as they progress through the educational system [2,3] due to different factors, such as the lack of mentors as well as the use of teaching methodologies that are not stimulating for students [4,5]. To address this problem, the Next Generation Science Standards [6] emphasize the need to rethink science teaching, stressing the need to integrate science content with the practice of doing science. The inclusion of Citizen Science (CS) in formal education could play a significant role in this regard, as it allows students to engage in the scientific process through participation in real scientific projects, and at the same time, learn scientific content [7].

Due to the technological advances to promote citizen participation in science, and the potential of CS to raise the scientific literacy of society [8], there has been a significant increase in CS activities in recent years in informal settings [9,10]. However, the implementation of CS projects in informal settings has a limitation, which is that they do not reach the entire population. CS projects in informal settings face inequality issues endemic throughout society, with innate barriers to participation for minorities and disadvantaged communities [11]. In addition, participants in CS projects in informal settings are often volunteers, so they are likely to have an intrinsic motivation and interest in science that may be lacking in other groups of citizens [11]. Therefore, there is a push to include CS in formal education, with the aim of bringing all the potential benefits of participation in CS to all citizens [12,13].

1.1. Potential of the CS in Formal Education

The contribution that the inclusion of CS in the curriculum can make to address SDG 4—Quality Education—which fosters convergence between Science Education and Education for Sustainable Development, essential for addressing the sustainability challenges currently facing humanity, has been widely recognized [14,15,16,17]. Participation in CS can contribute to increasing students’ interest and motivation [7,18] because it gives the students the opportunity to participate in scientific activities in authentic contexts, which allows them to understand the relationships between scientific concepts and everyday life [19,20]. Through this type of context-based learning activity, learning becomes more relevant for students because they observe phenomena immersed in their close context and contribute to the solution of real problems [21,22]. Moreover, the integration of CS in the curriculum can contribute to improving students’ understanding of the Nature of Science, as well as to the development of scientific skills [23,24]. Similarly, different studies claim that participation in CS is an excellent way to stimulate a positive attitude towards science [25] and critical thinking [26]. Finally, the contribution of CS to increasing environmental awareness and participation in environmental conservation has also been recognized [27,28].

1.2. Challenges for the Inclusion of CS in Formal Education

The integration of CS into the curriculum is a relatively new approach as it poses a great challenge [10,11]. One of the biggest challenges for the inclusion of CS in formal education is making scientific goals converge with educational goals [12,29,30,31]. While the scientific community is motivated by data and concerned about data accuracy, teachers are concerned about time demands and the alignment of goals and activities with curriculum standards.

For the integration of CS activities into formal education, from a pedagogical perspective, CS interventions must be meaningful and engaging for students, feasible for teachers, and must pursue teaching objectives aligned with curriculum standards [12,29,31,32]. However, in many CS projects, while science objectives are clearly defined, learning objectives [33] and connections to the curriculum are not [34]. Furthermore, once learning objectives are established, quality assessment of learning outcomes (LO) should also be planned and designed [11,35]. Numerous authors argue that a systematic and rigorous evaluation of the benefits of CS in terms of the LO of participants is necessary, since without it, any conclusion about the impact on students of participation in CS activities is premature [10,30]. However, very few CS projects contemplate an evaluation of LO [33,36], because participants’ learning in many cases is not the focus of the project [10,37], and many of the commonly expected LO from participation in CS activities are difficult to evaluate [31,33,38].

1.3. Previous Revisions of the Use of CS in Formal Education

Recent literature reviews have examined the use of CS in formal education, but they present gaps in terms of the evaluation of students’ LO in CE. Pizzolato and Tsuji [34] analyzed 73 CS interventions in K-12 settings but omitted any analysis of the evaluation of students’ LO. Similarly, Lüsse and colleagues [30] reviewed several CS initiatives in schools and proposed potential LO (e.g., motivation, knowledge, and communication skills) but did not examine whether or how these LO were assessed. Despite this, they called for a more systematic evaluation of LO in formal science education. Tsivitanidou and Ioannou [39] focused their review on the evaluation of students’ LO, but their systematic review only included CS projects facilitated by digital technologies, finding only 12 studies conducted in formal CE that evaluated the impact of the project on students’ LO, and concluding that empirical evidence on the effects of technology-facilitated CS projects on students’ LO remains scarce. Hadjichambi and colleagues [40], in their review of CS projects in K-12 education, identified only eight initiatives implemented within formal education settings, and, although the study examined which LO were expected, it did not investigate whether these outcomes were actually evaluated, nor the methods employed for such assessment. A similar limitation is observed in another review of CS projects in K-12 education [41].

The same gaps have been revealed in other reviews not focused on formal CE. Abourashed and colleagues [42], in their review, mainly analyzed the quality of data and student participation in CS projects in life sciences in formal education. However, they reported that, of the 23 projects identified, only three addressed LO, and none were carried out in CE. Vance-Chalcraft and colleagues [10] reviewed the use of CS in post-secondary education and noted that only a few of the 15 studies they found reported on LO evaluations.

In summary, although numerous reviews exist, to the best of the author’s knowledge, none has focused on the evaluation of the students’ LO from CS participation in CE. However, understanding and improving LO evaluation practices is crucial for enhancing the educational value of CS projects in CE and, therefore, their contribution to SDG 4. To address this need, the present study provides a systematic review of the published literature on CS projects implemented in CE to examine whether students’ LO are evaluated, and, if so, how these evaluations are carried out, in order to identify possible limitations and good practices, with the aim of providing clues for future designers of CS project in CE to obtain as reliable as possible evaluations of the students’ LO. The initial hypothesis of this study is that there is no systematic and rigorous assessment of students’ LO in CS projects.

2. Methods

This study presents a systematic review of the publications on the use of CS in CE. For this purpose, the PRISMA method was used (Figure 1).

2.1. Search Strategy

The search for articles was conducted in June 2023 and focused on articles published in English up to December 2022 in two important databases due to their wide coverage and the quality of their content: Web of Science and Scopus. Based on previous studies [10,34], the following combination of terms and Boolean operators, which include different ways of referring to CS, were used: (“citizen science” OR “public science” OR “community science” OR “public participation in science”) AND (School* OR Education* OR Class*) AND (Student* OR Pupil* OR Learn* OR Youth* OR Child*).

The search led to the retrieval of 1161 articles. After eliminating duplicates (n = 393), 768 items remained to be analyzed.

2.2. Article Screening

The articles were screened in two phases based on exclusion and inclusion criteria. For the first screening, the following exclusion criteria were used:

• Articles that did not present educational experiences
• CS experiences outside formal education
• No first-hand data (e.g., reviews)
• CS experiences in high or early childhood education

Many of the articles analyzed met the first exclusion criteria as some of them had a theoretical approach, e.g., [44,45], while others presented CS projects in a general way, indicating that students could participate, e.g., [46], or presented tools, applications, techniques, etc. to facilitate data collection and interpretation in CS projects, e.g., [47]. Finally, some articles focused on analyzing the quality of data obtained through CS, e.g., [48]. In the first screening, 578 articles were eliminated.

For the second screening, an exhaustive review of the remaining 190 full articles was conducted, using the following inclusion criteria:

• Articles that present CS experiences implemented within formal education
• Articles that present CS experiences implemented within CE (for each paper, the educational system of the country was considered, including it as CE or not)
• Articles that present an evaluation of the LO

All articles that met the first two criteria and made no reference to LO, or mentioned LO without presenting the evaluation of them, were counted but not included in the subsequent analysis.

Of the 190 articles, six could not be analyzed because access to the full articles was unavailable, and 58 did not meet either of the first two inclusion criteria. The remaining 126 articles presented CS experiences implemented in formal CE, but only 45 presented an evaluation of the LO (see Appendix A).

2.3. Analysis and Coding

For the coding, a content analysis based on a full and exhaustive reading of each article was performed. The 45 articles were analyzed for coding based on the following information: year of publication, country, CS project topic, type of CS project (contributory/collaborative/co-produced) [8], duration of the intervention, type of student participation (voluntary/mandatory), evaluated LO, basis of evaluation (perception - self-reported data/objective data), and evaluation design (experimental/quasi-experimental/non-experimental) [49].

3. Results

The systematic review identified 126 articles that present CS projects implemented in compulsory formal education and showed a significant increase in these projects since 2016 (Figure 2).

Of the 126 identified articles, only 45 presented an evaluation of the LO achieved by the students. Thus, 50 of the articles did not refer to the LO achieved by the participating students. Most of these articles focused on disseminating the results of the research, presenting the students as a mere data collection tool, e.g., [50,51], or on analyzing the reliability/rigidity/usability of data collected by students, e.g., [52,53]. Of the remaining 76 articles, 31 mentioned the LO that students would obtain, but in a theoretical or hypothetical way, without conducting an evaluation of the LO, e.g., [54,55]. Some of these articles even indicated that in the future it would be necessary to design an LO evaluation system, e.g., [56,57].

3.1. Characteristics of the CS Projects Analyzed

Regarding the geographical distribution of the 45 articles analyzed, most of them present CS projects carried out in European countries (51%) and in the USA (31%), and the most common study topics were biodiversity (33%) and pollution (mainly water and air pollution) (33%).

As for the type of CS project, following the classification of Bonney and colleagues [8], the projects were classified as contributory (student participation is restricted to obtaining data for the research), collaborative (students participate in other phases of the project such as data analysis or communication of results), and co-produced (the projects are designed in collaboration with the participating students, who participate in most phases of the research). Of the 45 articles analyzed, in 62% of them student participation was contributory, with 31% collaborative, while only 7% presented co-produced projects.

Finally, with regard to the duration of the students’ participation in the CS projects, in 18% of the articles, it was not possible to extract this information. In most of the projects the duration of the students’ participation was short (1–4 sessions) (40%) or medium (1–2 months) (29%), with only six projects (13%) where the intervention took more than 2 months.

3.2. Evaluated LO

Following the LO classifications proposed by Friedman [58] and Phillips and colleagues [49], the LO evaluated were grouped into six categories (

Table 1. Frequency of evaluated LO and evaluation results.

Category	Learning Outcomes	No. Evaluations	Results *
			+	0	−
Interest/attitude towards science and the environment	Interest in and/or positive attitude towards science	11	8	2	1
	Positive attitude towards living beings	5	4	1	--
	Interest in environmental conservation	4	3	1	--
	Interest in scientific careers	8	4	4	--
	Highlight of scientific activity	4	3	--	1
	Interest in learning about scientific topics	4	3	--	1
	Connection with nature	2	1	1	--
Knowledge and awareness of scientific issues	Awareness of the problem under study	11	8	2	1
	Knowledge about the study topic	33	32	1	--
Knowledge of the Nature of Science	Knowledge about the Nature of Science	6	6	--	--
Behavior	More sustainable behaviors	4	2	1	1
	Participation in community projects and/or actions (intention)	1	1	--	--
Skills for scientific inquiry	Data collection	2	1	1	--
	Data analysis/graph creation and interpretation skills	7	6	1	--
	Critical thinking	3	3	--	--
	Raising research questions	1	1	--	--
	Formulating hypothesis	2	1	1	--
	Communication/argumentation	4	4	--	--
	Research design	3	3	--	--
	Use of dichotomous keys	1	1	--	--
Self-efficacy	To solve problems	3	1	1	1
	To participate in scientific activities	8	5	1	2
	To learn/study science	1	1	--	--
Others	Less stereotypical image of scientists	2	2	--	--
	View of oneself as a scientist	2	--	1	1
TOTAL		132	104	19	9

* positive (+), no effect (0), negative (−).

): (1) interest and attitude towards science and the environment, (2) knowledge and awareness of scientific topics, (3) knowledge of the Nature of Science, (4) competencies for scientific inquiry, (5) behavior; and (6) self-efficacy.

The most evaluated LO were cognitive and attitudinal. For cognitive LO, the most evaluated LO was ‘Knowledge about the study topic’ (73% of the articles), followed by ‘Awareness of the problem under study’ (24%). For attitudinal LO, ‘Interest in and/or positive attitude towards science’ (24%) was the most evaluated. The procedural LO related to the acquisition of skills for scientific inquiry, and the behavioral LO were much less evaluated. Among these, the ones that were evaluated the most were ‘Self-efficacy to participate in scientific activities’ (18%) and ‘Data analysis/graphs creation and interpretation skills’ (16%).

Regarding the results of the evaluation, in 79% of cases, the evaluation reported positive results, such as more knowledge about the study topic, e.g., [59,60,61], or the development of data analysis skills, e.g., [38,62]. Moreover, 14% of the evaluations showed a lack of effect of the intervention, and 7% showed a negative effect, such as a reduction in self-efficacy for participating in scientific activities [13].

3.3. Reported LO by Study Topic and Type of Project (Contributory/Collaborative/Co-Produced)

The comparison of the reported results of the LO evaluation in function of the study topic does not show any effect of this factor. For both biodiversity and pollution projects, around 70% of evaluations reported positive results and 30% negative or no effect. The type of project (contributory/collaborative/co-produced) also shows no effect in the reported evaluations (see Appendix B).

3.4. Reported LO by Type of Participation (Voluntary vs. Mandatory)

As for the type of student participation, it was not possible to extract this information in 18% of the articles. In 18% of the studies, the CS project was presented as a complementary activity, and students’ participation was voluntary. In 64% of the studies, CS activity was presented as another class activity, and entire classes participated.

The analysis of the reported results of the LO evaluation shows that in projects where student participation was voluntary, the evaluation reported mainly positive results (94%). However, in the projects where student participation was mandatory, the evaluation reported negative results in 12% of the cases and no effect of the intervention in 18% of the cases (Figure 3).

3.5. LO Reported by Evaluation Design

Following the classification proposed by Phillips and colleagues [49] on LO evaluation design in CS projects, the evaluation was classified into three groups: (a) with non-experimental design, only with post-intervention testing (test, interviews, papers…) and without control; (b) with quasi-experimental design, with pre- and post-tests and without control; and (c) with experimental design, with control.

In the 45 analyzed articles, 36% of the evaluations of the LO presented a non-experimental design, 56% a quasi-experimental design, and only 8% an experimental design.

Both projects with non-experimental, quasi-experimental, and non-experimental evaluation designs included projects on different topics, of different duration, and of different types (contributory/collaborative/co-produced) (see Appendix C).

3.5.1. Results of the Evaluation of LO with a Non-Experimental Design

All LO reported in the evaluations with non-experimental design were positive, such as increased knowledge about the study topic, e.g., [63,64], or critical thinking, e.g., [65,66], with one exception, in which a lack of effect of the intervention on the students’ view of themselves as scientists was reported [67] (Figure 4a).

However, the evaluations had several limitations in most cases. In 50% of the cases, participation in the evaluation was voluntary, e.g., [67,68,69]. Furthermore, in many cases, evaluations were based on self-reported learning by students (

Table 2. Examples of questions in the post-intervention questionnaires to obtain self-reported LO by students.

Ask	Article
Do you think your knowledge of Excel has improved? Y/N	[62]
To what extent did this project help you develop research skills?	[72]
At what level was the project useful for acquiring the following competencies (a lot, quite, moderately, a little, nothing): Understanding of scientific and biosafety protocols	[73]
Express your degree of agreement with the following sentences: Now I understand better the problem of antibiotic resistance Now I think I know more things than I knew before participating in the project	[74]
Did you know before the project how radioactivity is measured? Yes/No/Don’t know. Do you after the project know how radioactivity is measured? Yes/No/Don’t know.	[64]

), on perceptions of teachers, e.g., [67,70], or on informal tests or observations, e.g., [69,71].

These limitations were recognized in several papers:

‘When analyzing the participants’ questionnaires, the frequency of response and possible biases must be taken into account........ In this case, the frequency of response is around 18%. It must be assumed that some teachers who respond to the questionnaire also encourage their students to do so, resulting in a certain bias based on mutual experiences. It can also be assumed that those teachers who enjoyed participating are more likely to respond to the questionnaire and encourage their students to do the same.’ [67] (pp. 5–6)

‘The main limitation of our study was that we did not track actual learning outcomes but rather students’ perceptions and reports of what they had learned. As our study is based on the students’ self-described experiences, own interpretations, and self-reflection of what has happened, we are not able to assess the factual impact of our project in relation to their learning.’ [7] (p. 335)

3.5.2. Results of the Evaluation of LO with a Quasi-Experimental Design

The evaluation of LO with a quasi-experimental design showed a greater diversity of results, with positive results in 69% of the cases, showing no effect of the intervention in 19% of the LO reported, and reporting negative results in 12% of the evaluations (Figure 4b).

The most evaluated LO was “Knowledge about the study topic” (70% of the articles), with positive results in all cases, e.g., [5,75], and measured objectively except for one case where it was self-reported by the students [61]. The rest of the LO were evaluated by objective and/or previously validated questionnaires from other research, with the exception of three cases [13,61,76], where the evaluation was based on student self-reported or teacher-perceived learning. Excluding LO related to the acquisition of knowledge about the topic of study, the results presented show a greater balance between positive effects of interventions (58%) and neutral or negative effects (42%).

As for student participation in the evaluation, the percentage of cases in which participation was voluntary was less than in evaluations with a non-experimental design, falling to 26%, e.g., [77,78]. In the remaining cases in which this information could be extracted, all the students who took part in the projects participated in the evaluation, or at least entire classes in the projects in which the coverage was extremely large, e.g., [79].

3.5.3. Results of the Evaluation of LO with an Experimental Design

Of the 45 articles analyzed, only four presented an evaluation of LO with experimental design [80,81,82,83]. These studies report 14 LO evaluations, with the results being positive on half of the occasions and showing no extra beneficial effect of the intervention on the other half (Figure 4c).

Focusing only on the most evaluated LO, evaluations with non-experimental design reported positive LO in all cases, and evaluations with more rigorous designs did not support many of those results (

Table 3. Results of the evaluation of the most reported LO by the type of evaluation design.

	Non-Experimental			Quasi-Experimental			Experimental
Result of the Evaluation *	+	0	−	+	0	−	+	0	−
Interest and/or attitude towards science	6	--	--	1	2	1	1	--	--
Interest in scientific careers	2	--	--	2	1	--	--	3	--
Awareness of the problem under study	4	--	--	3	2	1	1	--	--
Knowledge about the study topic	15	--	--	15	--	--	2	1	--
Self-efficacy to participate in scientific activities	3	--	--	1	--	2	1	1	--

* positive (+), no effect (0), negative (−).

).

While the three evaluation designs supported the contribution of CS in increasing knowledge about the study topic, the studies with quasi-experimental and experimental design did not support an increase in interest or the development of a more positive attitude towards science, increase in the awareness of the problem under study, increase in self-efficacy to participate in scientific activities, or increase in interest in scientific careers, due to participation in CS projects.

3.6. Reported LO by Duration of the Intervention

The comparison of the reported results of the LO evaluation in function of the duration of the interventions shows an unexpected result, with the projects with longer interventions reporting less positive LO than the ones with shorter interventions (Figure 5).

In order to interpret these results, it is worth pointing out that the use of more rigorous evaluation designs was more common in the longer projects than in the shorter ones (Figure 6).

4. Discussion

The potential social benefits for the participants in CS projects are driving the inclusion of CS activities in formal education, with the aim that these benefits reach the entire citizenry [12,13]. This review has evidenced this drive for CS in CE, having identified more than 100 articles published between 2016 and 2022 on CS projects conducted in CE.

Projects involving students in formal education must ensure that the participating students achieve LO aligned with the school curriculum [12,31,84]. Therefore, they should have well-defined educational objectives and rigorous evaluation systems to assess whether the project really contributes to the achievement of those objectives. However, this review has shown the lack of LO evaluations in most of the articles on CS activities in CE, confirming the initial hypothesis of this study. Many articles (n = 31) refer to the LO that students would achieve through participation in the CS project, without presenting any evidence. This is common in CS projects, which usually do not integrate an evaluation of their educational value, assuming that it is high [25]. However, some authors indicated that ‘Educational outcomes are an important component of school-based citizen science projects, so the project’s ability to deliver them should be formally evaluated rather than assumed’ [85] (p. 11). This argument is supported by different authors, such as Wichmann and colleagues [83], who reported a lack of effect of the intervention on environmental behavior and indicated that ‘The results suggest that a change in environmentally favorable behavior cannot be expected from participation alone’ [83] (p. 1). Other authors [5] also reported a lack of effect of the intervention on several of the expected LO and indicate that ‘Citizen science projects should not be assumed to increase scientific self-efficacy, attitudes toward Science, or interest in Science careers among students’ [5] (p. 1048).

Although there seems to be a consensus on the benefits for students of participating in CS activities, these benefits often remain more assumed than empirically demonstrated, since, as this review has shown, the publications on the use of CS in CE provide little rigorous data on students’ LO. Other authors, such as Vance-Chalcraft and colleagues [10], have reported similar results. These authors, in their review on the use of CS in postsecondary education, found that most of the articles reported many LO, on the basis of anecdotal evidence and the perceptions of the students and instructors, with only two articles that evaluated any LO beyond student perceptions or reflections. Tsivitanidou and Ioannou [39], in their review of technology-enabled CS projects involving school students, also found only 12 projects implemented in formal education that reported any LO, some of which were not supported by meaningful measures to authenticate the results.

To increase the empirical evidence for these assumptions, it is crucial to formally and rigorously evaluate the LO of student participation in CS projects [25,83]. However, the results of this review have shown that in most of the articles that evaluated LO, the results reported may have been influenced by the voluntary or mandatory nature of participation in the projects and by the design of the evaluation itself.

SDG 4—Quality Education—emphasizes the need to foster scientific literacy, critical thinking, and active citizenship [86], all of which align closely with the pedagogical potential of CS. Robust evaluation of LO in CS projects is therefore essential to ensure that they contribute meaningfully to the broader educational targets established under SDG 4.

4.1. Effects of the Type of Student Participation on LO Evaluation Results

This review has shown that projects in which students had volunteered reported a positive effect of the intervention in almost all cases. However, these results may be due to the bias produced by the type of participating students [7].

The contribution of participation in CS activities to the achievement of LO should not be evaluated only with students who have volunteered to participate in a project, as these students may not be representative of the whole class. They may be more motivated students who have a personal interest in learning or science [13], which the rest of the students who do not volunteer may not have. This may condition the benefits of the intervention [5,13], as student motivation may contribute to greater learning [87]. This would explain why, in projects in which whole classes participated, a lack of effect or even negative results were reported on many occasions. Therefore, taking into account that activities carried out in CE should ensure that they bring benefits to all the students, and not only of those who have a predisposition to learn, the suitability of CS projects for implementation in CE should be evaluated through projects that involve whole classes, to ensure the inclusion of students of different characteristics (e.g., social class, culture, learning skills, attitudes and interests, etc.).

The evaluation should also involve all students, since the voluntary or mandatory nature of participation in evaluation processes can also affect the results. It is possible that the students who voluntarily participate in the evaluation are those who have been motivated by the project and, therefore, have been more involved in it. This can contribute to greater real or perceived learning. The students who do not participate in the evaluation are possibly those who have not been as motivated by the project and, therefore, have not obtained or perceived the same learning.

Regarding the phases of the research in which students participate, student participation may be restricted only to data collection (contributory project) or may extend to other phases of the research, such as data analysis or communication of results, (collaborative project), or even to all of them (co-produced project). Authors such as Roche and colleagues [11] recommend that new CS projects should be co-produced, as these projects are more likely to achieve the scientific and educational goals of the project. However, this review has not shown an effect of project type on the reported LO evaluations, with all three types of projects reporting very similar frequencies of positive and negative results or no effect.

Finally, as for the duration of the participation in the CS project, it might be expected that projects of longer duration would achieve more LO than those of shorter duration, as there are LO that take longer to achieve. However, this review has shown the opposite result, with a lower proportion of positive evaluations in long projects than in those of shorter duration. This could be explained by the evaluation design, with more rigorous evaluation designs (quasi-experimental/experimental) being used more frequently in the longer-duration projects.

4.2. Effects of the Evaluation Design on LO Evaluation Results

In addition to the inclusion of all students in the evaluation of LO, the design of the evaluation can also condition the results, as this review has evidenced. Evaluations based only on post-intervention tests do not allow for determining objectively whether there has been an improvement, because there is a lack of diagnosis of the actual baseline situation. Regarding the tool used, although a rigorous analysis of the specific tools used to evaluate the different LO has not been carried out, as this was beyond the scope of this study, this review has shown that evaluations are often not based on robust or validated instruments. Frequently, evaluations rely on students’ self-reported learning or teacher perceptions, e.g., [67,70]. These evaluations may be biased due to social desirability [88,89]; that is, students’ responses may be influenced by what they believe the teachers or researchers expect them to respond. This highlights the need for future research focused on the development, standardization, and validation of assessment tools to evaluate different LO in the context of CS projects.

So, in order to carry out an evaluation of the LO of participation in such projects that is as rigorous as possible, an evaluation with an experimental design based on objective data would be the most appropriate one. Only an experimental design can ensure that an innovative methodology, such as the use of CS, will result in higher LO than other more conventional methodologies. Evaluations with quasi-experimental design (pre- and post-test) based on objective data can evaluate whether the expected LO have been achieved due to the intervention but cannot ensure that this was due to the methodology or simply to having worked on the subject [5,25,90].

4.3. Evaluated LO

Finally, this review has shown a clear bias of evaluations towards cognitive LO, with ‘knowledge about the study topic’ and ‘awareness of the problem under study’ being the most evaluated LO. However, evaluations should go beyond merely assessing effects on knowledge acquisition and awareness of an issue [13,91] as scientific literacy goes beyond this. It includes the ability to apply scientific knowledge to solve problems, identify questions, generate new knowledge, explain phenomena, and draw evidence-based conclusions about science-related issues, as well as valuing science and a positive attitude and interest in scientific topics [92]. So, to ensure that participation in CS activities contributes to students’ comprehensive scientific literacy, deeper LO must also be evaluated [33,35], such as the development of scientific inquiry skills, a deeper understanding of the Nature of Science [14,93,94], or changes in attitudes and behaviors of participating learners [95]. However, this review has shown that the evaluation of this type of LO is uncommon. In the vast majority of cases, and independently of the participating students and the design of the evaluation, the level of knowledge about the study topic increased, so it could be said that there is evidence of the contribution of CS to increasing student knowledge. However, regarding other types of LO, the results reported when using more rigorous evaluation systems have shown different results; therefore, the available evidence of the effects of CS projects on deeper learning remains limited, which offers a clear avenue for future research.

5. Conclusions and Recommendations

The use of CS in CE has proliferated in recent years; however, the lack of systematic and rigorous evaluation of LO may lead to an overly optimistic view of its positive effects on learners and, therefore, on its contribution to SDG 4—Quality Education. This review has shown two main trends: a) many studies assume LO without evidence, and b) many others base their evidence on projects and evaluations that, due to their design, may bias the results. In order to obtain solid evidence of the beneficial effects, or lack thereof, for students of participation in CS activities, that can guide the designers and educators in improving the CS projects to maximize their educational and sustainability impacts, future studies should formally evaluate student LO in a rigorous manner, for which it is recommended that:

• The CS project has clear and well-defined learning objectives;
• The project design includes an LO evaluation system;
• Whole classes participate in the project;
• All participating students participate in the evaluation;
• The evaluation has an experimental design with pre-test, post-test, and control group;
• The evaluation excludes self-reported LO.

Given that SDG 4 emphasizes not only knowledge acquisition but also the development of scientific competencies and problem-solving skills [86], the absence of robust evaluation frameworks limits our ability to determine whether CS projects meaningfully support these broader educational goals. Strengthening LO assessments in CS projects is therefore essential not only for improving educational practice but also for verifying and enhancing their potential contribution to the targets of SDG 4 [93,96].