Next Article in Journal
The Restorative Quality of the Work Environments: The Moderation Effect of Environmental Resources between Job Demands and Mindfulness
Previous Article in Journal
The Effects of the COVID-19 Pandemic “Crisis” on Unaccompanied Minors Navigating US Removal Proceedings
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Social Sciences
Volume 12
Issue 7
10.3390/socsci12070374
socsci-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Assessing Portuguese Elementary School Students’ Scientific Literacy: Application of the ALCE Instrument

by
Marcelo Coppi
*,
Isabel Fialho
and
Marília Cid
Center for Research in Education and Psychology, University of Évora, 7000-803 Évora, Portugal
*
Author to whom correspondence should be addressed.
Soc. Sci. 2023, 12(7), 374; https://doi.org/10.3390/socsci12070374
Submission received: 20 April 2023 / Revised: 15 June 2023 / Accepted: 22 June 2023 / Published: 27 June 2023
Browse Figures
Review Reports Versions Notes

Abstract

:
The study aimed to identify the scientific literacy level of students at the end of the third cycle of Portuguese basic education and verify how the variables sex, failure, a liking for the subject, socio-economic level of the schools and course option in secondary education were related to the students’ scientific literacy levels. Data from the ALCE instrument, answered by 516 ninth-grade students from 20 public schools in mainland Portugal, revealed that 64.14% were classified as being scientifically literate and that most students were had a moderate level of scientific literacy. On average, significant differences were observed only between the students’ scientific literacy levels and the variables related to a liking for natural science and physical chemistry subjects, course option in secondary education and socio-economic level of the schools. Among the investigated variables, only the option of vocational courses, the option of science and technology and a liking for physical chemistry were able to predict, partially, the students’ scientific literacy levels. In summary, the results of the scientific literacy level of the students and the number of scientifically literate students were positive and similar to those verified in external evaluations and in other studies assessing the scientific literacy of this education cycle or similar.

1. Introduction

In 2015, the World Economic Forum, through the report New Vision for Education—Unlocking the Potential of Technology (World Economic Forum 2015), published a list of 16 skills needed by students in the 21st century. Among them, six skills were identified that came to constitute the group of what were then called foundational literacies, which “serve as the base upon which students need to build more advanced and equally important competencies and character qualities” (World Economic Forum 2015, p. 2), namely, reading literacies, numeracy, scientific literacy, ICT literacy, financial literacy and cultural and civic literacy.
The development of these multiple literacies, especially at school, as a space for learning and skills development (Martins et al. 2017), should be promoted in order to respond to current demands and requirements. These are largely due to the intensification of the pace of scientific and technological development and exponential growth in information on a global scale, consequently bringing multiple challenges to education, related, for example, to identity and security, sustainability, interculturality, innovation and creativity (Martins et al. 2017).
As stated by Vizzotto and Del Pino (2020), it is expected that students, upon completing compulsory education, will have developed skills that will enable them to understand and act in their daily lives in a critical and responsible way. According to the authors, in general, scientific literacy shares this goal and contributes significantly to this process.
Defined by the Organisation for Economic Co-operation and Development (OECD) as “the ability to engage with science-related issues, and with the ideas of science, as a reflective citizen” (OECD 2017, p. 22), scientific literacy is recognised as the main goal of science education in several countries around the world (DeBoer 2000; Odegaard et al. 2015; Wang and Zhao 2016; World Economic Forum 2015; Yacoubian 2018). For Miller (1983), one of the first researchers to propose a definition for scientific literacy, a scientifically literate person is one who, in addition to understanding scientific nomenclatures and concepts, understands the process of scientific knowledge production and also how science and technology impact society and the environment.
In Portugal, the education system is compulsory from 6 to 18 years of age and comprises basic education (organised in three cycles: first cycle from first to fourth grade, second cycle from fifth to sixth grade and third cycle from seventh to ninth grade) and secondary education (from tenth to twelfth grade). In particular, in the third cycle of basic education, the last cycle in which all students must participate in all subjects aimed at addressing scientific literacy, these skills are developed in the two subjects that compose the area of physical and natural sciences: natural sciences and physical chemistry. Both subjects are ruled by two curricular reference documents, the Essential Learnings and the Students’ Profile by the End of Compulsory Schooling, in which the acquisition of scientific literacy skills in students is notably and clearly recommended.
In the Essential Learnings of Physical Chemistry (DGE 2018d, 2018e, 2018f), scientific literacy is stated as its main goal. According to the document, the subject of physical chemistry “aims to contribute to the development of scientific literacy in students, arousing curiosity about the world around us and interest in Science” (DGE 2018d, p. 1; DGE 2018e, p. 1; DGE 2018f, p. 1). The Essential Learnings in Physical Chemistry also states that students who complete the subject at the end of the third cycle will be equipped with scientific literacy skills that will allow them to mobilise their understanding of scientific processes and phenomena for decision-making, aware of the implications of science in today’s world, in order to exercise a participatory citizenship (DGE 2018d, p. 2; DGE 2018e, p. 2; DGE 2018f, p. 2).
Similarly, the Essential Learnings of Natural Sciences (DGE 2018a, 2018b, 2018c), although not making explicit reference to the term scientific literacy, assumes it as the subject objective. As stated in the document, the subject of natural sciences aims to broaden the horizons of learning, providing students with access to relevant products of science and its processes, through the understanding of the limits and potentialities of science and its technological applications in society. On the other hand, it is sought after that students become aware of the impact of human intervention on Earth and the need to adopt behaviours of active and fair citizenship, consistent with sustainable development (DGE 2018a, p. 2; DGE 2018b, p. 2; DGE 2018c, p. 2)
The Students’ Profile by the End of Compulsory Schooling (Martins et al. 2017), a document presented as a reference for the organisation of the entire Portuguese education system, mentions the need for the development of multiple literacies, including scientific literacy. According to the Profile, it is intended that, by the end of compulsory education, the school should be able to promote the development of multiple literacies, enabling students to “critically analyse and question reality, evaluate and select information, formulate hypotheses and make informed decisions in their daily lives” (Martins et al. 2017, p. 15).
From this perspective, it can be seen that the Portuguese curriculum documents ruling on the subjects of physical and natural sciences of the third cycle of basic education are in line with the global trend of declaring scientific literacy as the main goal of science education. Analysing the continuous trend improvement in Portugal’s results in the international assessments Programme for International Student Assessment (PISA) and Trends in International Mathematics and Science Study (TIMSS), it is assumed that a development in students’ scientific literacy skills is taking place and producing positive effects.
However, despite the continuous improvement, Galvão et al. (2017) point out that the results are below expectations. According to the authors, “a discrepancy persists between what would be expected in terms of students’ performances in international examinations (such as PISA, constructed in the light of international recommendations) and what students actually achieve” (Galvão et al. 2017, p. 15).
In this sense, it becomes necessary to monitor the scientific literacy level of Portuguese students more periodically than just every three or four years, as is the case with the results of these international assessments. This implies using different instruments for assessing scientific literacy, at local, regional and national levels, in order to collect the necessary information to propose more frequent and punctual changes in education plans, methodologies and practices of the subjects of natural sciences and physical chemistry, aspiring to a development in scientific literacy skills in students.
It was in this context that the study presented in this article aimed to assess the scientific literacy level of students at the end of the third cycle of Portuguese basic education. This survey will allow us to envisage future paths for adequate scientific and technological education in schools in Portugal.
Therefore, the following research questions were designed: (1) What is the scientific literacy level of students at the end of the third cycle of basic education? (2) How many students are scientifically literate? (3) Is there a difference in the scientific literacy level between male and female students? (4) Is there a difference in the scientific literacy level between students who have already failed natural sciences and physical chemistry and those who have never failed them? (5) Is there a difference in the scientific literacy level between students who intend to choose different courses in secondary education? (6) Is there a difference in the scientific literacy level among students from schools with favourable, intermediate and unfavourable socio-economic levels? (7) Is there a difference in the scientific literacy level among students who like and do not like the subjects of natural sciences and physical chemistry? (8) Which one or which of these variables are able to predict the scientific literacy level of students?
The between-group comparisons needed to answer these research questions were based on differences already noted in the literature in studies conducted in different contexts. For example, with regard to sex, both the PISA 2018 and TIMSS 2019 Portugal reports indicate that boys have higher average scores than girls. Such results are in accordance with those presented by most countries participating in the two tests, in which girls present the highest average score (Duarte et al. 2020; Lourenço et al. 2019).
In terms of failure in subjects, several studies highlight the fact that retention is a factor that impairs the performance of students in international assessments (Pinto et al. 2019). The study by Moreira et al. (2010), for example, reveals that, in PISA 2006, there was a 44-point difference in favour of the group of non-retained Portuguese students in the average score of the science test in relation to students who had been retained once, twice or three times.
Regarding the variable course option in secondary education, in Portugal, the TIMSS 2019 report showed that one in two boys with high performance in science intend to follow a profession in science and, in the case of girls, one in seven intend to do so (Duarte et al. 2020).
In the case of groups of students from schools with favourable, inter-mediate and unfavourable socio-economic levels, several studies show that students with lower performance are found in the most unfavourable socio-economic contexts, with these two variables being associated (Pinto et al. 2019). Furthermore, the results of Portuguese students in TIMSS 2019 revealed that students from more socio-economically advantaged schools scored better than those from more-disadvantaged schools.
Finally, regarding the variable of liking or not liking the subjects of natural sciences and physical chemistry, the study of Pinto et al. (2019), by analysing the relationship between student performance in science and a liking for scientific subjects, revealed that a liking for science has a positive impact on student performance.
From this perspective, it is expected that the answers to the research questions will contribute to contextualising the results of the application of the instrument and to the discussions about science teaching and a development in scientific literacy in primary education.

2. Methodology

2.1. The Instrument

From a methodological point of view, the study applied the Avaliação da Literacia Científica Essencial (ALCE) instrument to Portuguese 9th-grade students. The ALCE consists of 34 items in “true-false-don’t know” format, divided into three subtests—nature of science (NS), impact of science and technology on society (ISTS) and content of science (CS)—as shown in Table 1.
For the purposes of this study, two groups of questions were added. The first, consisting of four items, was added to collect sociodemographic data: (1) sex (male/female); (2) age; (3) school year and (4) school where you study. The second group consisted of five items of covariates, whose purpose it was to contextualise the students’ performance results: (1) Do you like the subject of natural sciences (Likert scale between 1 = I don’t like it and 5 = I adore it)? (2) Have you ever failed the subject of natural sciences (yes/no)? (3) Do you like the subject of physical chemistry (Likert scale between 1 = I don’t like it and 5 = I adore it)? (4) Have you ever failed the subject of physical chemistry (yes/no)? (5) Which course do you intend to choose in secondary education (science and technology course/socio-economic sciences course/languages and humanities course/visual arts course/professional courses)?
The data on the socio-economic level of the schools were taken from the Infoescolas Portal (Ministério da Educação e Ciência 2022) of the Portuguese Ministry of Education and Science. This information classifies schools into three distinct groups, namely, favourable, in which less than 25% of students are covered by the School Social Action programme; intermediate, in which 25% to 50% of students are covered by the School Social Action programme; and unfavourable, in which more than 50% of students are covered by the School Social Action programme (DGEEC 2023).
The 34 items that make up, properly, the ALCE assess three cognitive domains of the revised Bloom Taxonomy (Anderson et al. 2001): understanding, analysis and evaluation of problems and everyday situations involving skills from the area of physical and natural sciences to find a solution. This set of skills appears in the following Portuguese curriculum documents of physical and natural sciences of the 3rd cycle of basic education, up to the time it was developed in force: Curricular Guidelines for the 3rd cycle of Basic Education—Physical and Natural Sciences (DEB 2001), Essential Learnings in Natural Sciences and Physical Chemistry (DGE 2018a, 2018b, 2018c, 2018d, 2018e, 2018f) and Students’ Profile by the End of Compulsory Schooling (Martins et al. 2017).
The items of the ALCE are interpretative, consisting of one or more highlighted phrases in italics or bold, which should be taken as true, followed by a sentence without any highlight, which should be judged as true or false. An example item from each subtest is presented below:
Laws and theories are part of science. While laws describe natural phenomena, theories use them to propose an explanation of such phenomena. Once a theory has been established, scientists in the field set aside research related to that theory and start studying other phenomena, since, because it is a scientific theory, it is considered as an incontestable truth by the scientific community.
(NS subtest item)
The techno-scientific revolution associates scientific knowledge with the use of technology in industrial production. In regard to this association, advances in science enable a development in new technologies and these enable the production of more scientific knowledge, making science and technology inseparable areas.
(ISTS subtest item)
Rocks formed in the past and preserved can be considered as the geological memory of planet Earth. The fact that some rocks have fossil records allows scientists to perform relative dating. Thus, it is possible to date the layers of rocks on the basis of the fossils that they contain, allowing the age of the rock to be related to the age of the fossil preserved in it.
(CS subtest item)

2.2. Validation of the Instrument

The instrument’s validation process was carried out through the collection of validity evidence based on the content (Coppi et al. 2022) and validity evidence based on the internal structure, as proposed by Pasquali (2009). The first group of evidence was obtained from the definition of the cognitive domains; the definition of the universe of content; the definition of the representativeness of the content; the preparation of the specification table; the construction of the instrument; the theoretical analysis of the items, designed by a panel composed of ten experts in the areas of educational sciences, biology, geology, physics and chemistry, who verified the representativeness, relevance and quality of the items; and the empirical analysis of the items, which consisted of the psychometric evaluation of the items, using item response theory (IRT), by submitting the instrument to a pilot test, consisting of 35 items, applied to a sample of students in the 10th grade, recently graduated from the 9th grade, selected “by convenience” (Ghiglione and Matalon 1992; Hill and Hill 2005). This application was carried out at the beginning of the 2020/2021 school year, in digital format, by means of the LimeSurvey software, version 3.25.17, in the classroom and in the presence of teachers. The IRT technique was also used to collect validity evidence based on the internal structure, obtained through the IRT information functions, the Kernel density estimate of the subtests and the students’ proficiency level (θ).
After analysing the results of the collection of validity evidence based on the content and on the internal structure of the instrument, it was found that some items were classified as very easy and others were classified as very difficult, when compared to the samples’ proficiency level. In this sense, and with the purpose of better adapting the instrument to the characteristics of students in this education cycle, these items were again submitted to the panel of experts, who contributed to changes in terms of understanding, relevance and clarity of the items and to the exclusion of one item, leading to the final version of the ALCE, composed of a total of 34 items. In this version, it was found that the items were adequate for the students’ proficiency level, validating the use of the results for the proposed objectives.

2.3. Proceedings

The application of the ALCE final version was authorised by the Directorate General for Innovation and Curriculum Development, through the School Survey Monitoring System, under the registration no. 0740900001.
At the end of the 2021/2022 school year (between April and June 2022), the link to access the instrument on the LimeSurvey platform was sent via e-mail to the directors of all public schools in mainland Portugal, who were asked to forward it to the teachers of the subjects of natural sciences and physics–chemistry. The e-mail contained instructions for accessing, reading and applying the instrument on the LimeSurvey platform. The directors and teachers were asked to have the students answer the instrument in the school’s computer room and in the presence of the natural sciences and physical chemistry teachers.

2.4. Participants

The ALCE was answered by 516 9th-grade students from 20 public schools in mainland Portugal. Of these students, 259 (50.2%) were male and 257 (49.8%) were female. The average age of the respondents was 14.69 years (SD = 0.88).
It is important to note that the sample is not representative and, therefore, the results cannot be generalised to the population of Portuguese students in the 3rd cycle of basic education.

2.5. Data Analysis Procedure

The data were analysed using statistical procedures comprising frequency analysis, t-Student test, analysis of variance (ANOVA) and regression analysis, carried out using SPSS software, v. 27. Frequency analysis was carried out to identify the number of correct answers and the students’ scientific literacy levels. The t-Student test for independent samples was used to analyse the comparison of the means of the students’ scientific literacy levels by sex, by failure, meaning whether or not they had already failed the subjects natural sciences and physical chemistry, and by the course they intended to choose in secondary education.
The ANOVA, whose employed post hoc was Tukey’s, was applied for the analysis of comparison of the means of the students’ scientific literacy levels according to their liking for the subjects of natural sciences and physical chemistry and to the socio-economic level of the schools. This last analysis took into account the scientific literacy level and the number of correct answers by the students. Finally, multiple linear regression analysis was employed in order to describe the relationship between the independent variables sex, failure, a liking for the subject, socio-economic level of the school and course option in secondary education and the dependent variable scientific literacy level of the responding students. It is worth noting that all of the prerequisites of the statistical analyses used were met.

2.6. Instrument Scoring and Classification Criteria

Regarding the format of the items and the attribution of the score of the ALCE items, it was decided that it be performed dichotomously: one point awarded per item for a correct answer, and zero awarded for a wrong answer. The correct number of answers were summed in order to form the total score per subtest which, when added, formed the overall ALCE score.
The option “don’t know” was added in order to be marked if the student had no knowledge of the content or skill requested by the item. According to Ebel and Frisbie (1991), adding this option in survey questionnaires significantly reduces the number of correct answers obtained only through guessing. In addition, the inclusion of the “don’t know” option was also aimed at collecting important data for teachers regarding students’ knowledge. Items with large numbers of answers in this option were able to inform teachers that certain content or a certain skill were not effectively assimilated by the students, or even that it was not worked on in the classroom or was worthy of greater attention from teachers in the reformulation of their classes and teaching strategies. Therefore, in terms of scoring and classification criteria, items marked with the option “don’t know” also received a score of zero.
Taking into account the logic of the instrument, the students’ classification in each subtest was determined in five levels of scientific literacy—very low, low, moderate, high and very high—according to the percentage and number of correct answers in the items, as shown in Table 2.
This distribution follows the assessment scoring model adopted in the Portuguese 3rd cycle of basic education (Despacho Normativo n.º 1-F/2016 do Ministério da Educação 2016). In this model, the tests are graded on a percentage scale from 0 to 100, and the final classification of the test is converted into a scale from 1 to 5, following the following distribution: level 1, from 0 to 19%; level 2, from 20 to 49%; level 3, from 50 to 69%; level 4, from 70 to 89% and level 5, from 90 to 100%. To be approved, students need to reach at least level 3 on the scale.
The same categories and scale were used for the classification of the students’ general scientific literacy level. This was obtained by averaging the scientific literacy levels of the three subtests, as presented in Table 3. Students who reached at least the moderate level of scientific literacy were considered to be scientifically literate.

3. Results and Discussion

3.1. Students’ Scientific Literacy Levels

The results revealed that most of the students had a moderate level of scientific literacy. Of the 516 responding students, 255 (49.4%) were classified at this level, 184 (35.7%) were classified at the low level, 74 (14.3%) were classified at the high level and 3 (0.6%) were classified at the very high level (Figure 1). No students were rated as having a very low level of scientific literacy.
It is observed that, in accordance with the logic of the instrument and the established classification criteria, more than half of the responding students, 331 (64.14%), were classified as being scientifically literate, as they reached at least the moderate level of scientific literacy. The characteristics of the scientifically literate students are described in Table 4.
Despite the fact that the assessed result related to the level of scientific literacy of students in the third cycle of basic education was positive, that is, that 64.14% of students were classified as being scientifically literate, the data revealed that more than a third of students (36%) did not reach the minimum performance to be classified as being scientifically literate. Furthermore, it was observed that only 14.9% of the students were classified in the above-moderate levels of scientific literacy.
Although it is not possible to make a strict and direct comparison between classification levels, the results support those released in the latest reports of Portugal’s performance in TIMSS 2019 (Duarte et al. 2020) and PISA 2018 (Lourenço et al. 2019). In the TIMSS 2019 assessment, 73% of Portuguese students reached the intermediate level of performance in science (Duarte et al. 2020). The report displays that 34% of students reached the high level and only 7% reached the advanced level on the TIMSS scale.
According to Duarte et al. (2020), the overall result of the Portuguese students shows that the majority present intermediate-level skills, demonstrating some knowledge in the areas of biology and physics. However, due to the low number of students at the high and advanced levels, a minority showed skills to apply the knowledge of biology, chemistry, physics and earth sciences and to characterise the concepts of such areas in a contextual plurality (Duarte et al. 2020).
The same could be verified in the ALCE. Among the items with a wrong answer rate above 50%, two belonged to the NS and ISST subtests and twelve belonged to the CS subtest. Of these twelve, one corresponded to the area of natural sciences, one corresponded to the area of geology, six corresponded to the area of physical chemistry and four corresponded to the area of biology. Only analysing the items with a wrong answer rate above 70%, six items were identified: two in the area of chemistry, three in the area of physics and one in the area of biology, indicating a certain difficulty encountered by students in these areas.
Comparing the results with those of the PISA 2018 report, it can be seen that the performance of Portuguese students in the international assessment was higher than in the present study, although the results showed variations according to the region of the country. As the report points out, in the 2018 assessment, 80% of the students reached level 2 of proficiency in the science assessment (Lourenço et al. 2019). At this level, students have basic skills for understanding science (Saraiva 2017) and for using scientific knowledge to appropriately explain everyday events scientifically (Marôco et al. 2016), skills that are similar to those assessed by the ALCE.
However, in line with the results obtained in the ALCE and TIMSS 2019, only 5.1% and 0.5% of students reached levels 5 and 6 on the PISA scale, respectively. According to the PISA 2018 grading rubric, levels 5 and 6 demand the mastery of skills of higher complexity, in which students are able, for example, to “apply their knowledge of and about science autonomously and creatively to a wide variety of situations, even the least familiar” (Lourenço et al. 2019, p. vii).
For Soobard and Rannikmäe (2011), a small number of individuals were expected to occupy higher levels in assessments that addressed all levels of scientific literacy. The authors argue that “it is not yet clear whether students are capable to give answers at higher levels of scientific literacy where questions require this, or whether students are not able to answer at this level” (Soobard and Rannikmäe 2011, p. 141).
The results of the ALCE are close to those shown in the studies of Oliveira and Silva-Forsberg (2012) and Özdem et al. (2010), who applied an adapted version of the Test of Basic Scientific Literacy—developed by Laugksch and Spargo (1996), and based on the same assumptions of Miller’s (1983) scientific literacy that were used for the construction of the ALCE—to Brazilian and Turkish primary school students, respectively. In Oliveira and Silva-Forsberg’s (2012) study, while 66% of the students achieved at least the minimum number of correct answers to be classified as being scientifically literate, only 23.5% obtained ratings at the higher levels of satisfactory, good and very good. In the research of Özdem et al. (2010), on the other hand, the authors identified that the student respondents had a moderate level of scientific literacy.
When compared with other studies whose objective was to assess the scientific literacy of students in the third cycle of basic education, or of similar school years (Coppi and Sousa 2019; Jufri et al. 2019; Koedsri and Ngudgratoke 2018; Rachmatullah et al. 2016; Vizzotto et al. 2020), it is evident that the ALCE results are more positive than the others, since they all revealed that the majority of the students assessed did not obtain the minimum correctness on the items of the instruments to be considered as being scientifically literate. It should be noted, however, that the contexts of application, the assessment instruments used and the number of participants of these studies were not the same as those of the present study. Coppi and Sousa (2019), for example, applied the Test of Basic Scientific Literacy in 189 ninth-grade students from a private school in Brazil. Vizzotto et al. (2020) applied a simplified version of the Test of Basic Scientific Literacy in 125 ninth-grade students from a public school, convened with the Federal University of Rio Grande. Rachmatullah et al. (2016), in Indonesia, applied the Scientific Literacy Assessment in 223 students of the third cycle of basic education. And, Koedsri and Ngudgratoke (2018) applied a scientific literacy assessment instrument, built by the authors themselves, to 270 ninth-grade students from a school in Thailand.

3.2. Scientific Literacy Level According to the Variable Sex

No significant differences were found in the average level of scientific literacy between male and female students. Moreover, when analysing the amount of scientifically literate male and female students, presented in Table 4, it is observed that there are slightly more boys than girls, 169 (33%) and 162 (31%), respectively. However, when analysing the number of male and female students at the two highest levels of the scale (high and very high), it can be observed that the number of boys is greater than that of girls, 45 and 32, respectively.
These results are consistent with those presented in the PISA 2018 (Lourenço et al. 2019) and TIMSS 2019 reports, in which the mean score of boys was higher than that of girls, being significant in the case of TIMSS (Duarte et al. 2020). However, they contradict the trend in the average performance results of OECD participant countries and TIMSS 2019 countries, in which girls have the highest average scores in science tests.

3.3. Scientific Literacy Level According to the Variable Failure in the Subjects of Natural Sciences and Physical Chemistry

With regard to the characteristics of the scientifically literate students in relation to the variable failure in the subjects of natural sciences and physical chemistry, Table 4 shows that most of the scientifically literate students were those who never failed the two subjects, 61% and 59%, respectively. Nevertheless, considering the difference in the average level of scientific literacy between the groups, significant differences were observed only between the average of the students who had failed the subject of physical chemistry and those who had not failed it. The average scientific literacy level of students who had not failed the subject physical chemistry (M = 2.80; SD = 0.74) in previous years was significantly higher [t(514) = −2.702, p < 0.01] than that of students who had already failed it (M = 2.51; SD = 0.67).
Several studies related to the effects of school failure show that it does not promote benefits, but on the contrary, produces negative effects on students’ performance (Rebelo 2009). Jimerson and Ferguson (2007), for example, analysed the school performance of students in the fourth, fifth, eighth, tenth and eleventh grades and found that retained students obtained, on average, significantly lower performances than those who passed. Fernandes et al. (2018), meanwhile, analysing the school performance of Brazilian students at the end of primary education, a cycle equivalent to the end of the third cycle of basic education in Portugal, showed that the higher the number of failures, the lower the students’ school performance. The authors also revealed that the variable with the greatest weight on students’ school performance was school failure, showing a negative association with it (Fernandes et al. 2018).
Moreira et al. (2010), for their part, claim that retention is a more predictive factor than socio-economic and cultural background, arguing that in the PISA 2006 science test there was a 44-point difference in favour of the group of non-retained Portuguese students in the mean score of the science test, in relation to students who had already been retained once, twice or three times.

3.4. Scientific Literacy Level According to the Variable Liking for the Subjects of Natural Sciences and Physical Chemistry

The results presented in Table 4 reveal that, for the subjects of natural sciences and physical chemistry, the largest number of scientifically literate students belongs to the groups of students who like the subjects. It is also observed that, for both subjects, the number of students who like them a lot and who adore them is greater than those who do not like them or do not like them very much.
Regarding the distribution of students’ scientific literacy levels on the variable of a liking for the subject of natural sciences, statistically significant differences [F(4, 511) = 5.463, p < 0.001] were found for the mean literacy level of students who were in the five categories of the liking for the subject. The analysis revealed that the average level of scientific literacy of students who adored the subject of natural sciences (M = 3.05; SD = 0.75) was significantly higher (p < 0.01, p < 0.05 and p < 0.05, respectively) than the students who only liked it (M = 2.70; SD = 0.72), did not like it very much (M = 2.67; SD = 0.72) or did not like the subject (M = 2.41; SD = 0.51) (Figure 2).
Statistically significant results were also found in the case of the physical chemistry subject [F(4, 511) = 8.39, p < 0.001]. The average scientific literacy level of students who liked it (M = 2.70; SD = 0.72), liked it a lot (M = 2.92; SD = 0.74) and adored the subject (M = 3.05; SD = 0.75) was significantly higher (p ≤ 0.05, p < 0.001 and p < 0.001, respectively) than that of students who did not like it (M = 2.41; SD = 0.51). The mean of students who liked it a lot (M = 2.92; SD = 0.74) and adored physical chemistry (M = 3.05; SD = 0.75) was also statistically significantly higher (p < 0.001, and p < 0.01, respectively) than that of students who did not like it very much (M = 2.67; SD = 0.72). And, the mean of students who liked the subject a lot (M = 2.92; SD = 0.74) was also statistically significantly higher than that of students who only liked it (M = 2.70; SD = 0.72) (Figure 3).
To a certain degree, the observed result was expected. The positive impact of the student’s motivational effect on their performance has already been described in the literature (Cavalcanti 2009; Kpolovie et al. 2014; Oliveira 2021). According to Kpolovie et al. (2014), students’ interest is directly related to their choices of engagement with subjects and the study time devoted to them, for example. Consequently, students tend to dedicate more and make an effort to study the subjects that they enjoy the most.
The study of Cavalcanti (2009) supports the above. According to the author, the student’s interest in learning was the factor that showed the most positive and moderate correlation index with the intrinsic motivation to learn. Corroborating this, Silva et al. (2018), in research in which they analysed the relationship between liking chemistry and the students’ difficulty in the subject, conclude that the students’ positive or negative relationship with the subject is closely linked to their learning. This means that in the same way that liking the subject of chemistry facilitates students’ learning, the reverse is also true, that is, the fact that the student does not like the subject will make it difficult for them to understand it. Furthermore, Pereira (2019), analysing a liking for science, measured through interest and enjoyment when learning and carrying out assignments in science and liking to read about scientific content, evidenced that the more students demonstrate these behaviours, the better their performance in science subjects.

3.5. Scientific Literacy Level According to the Course Option in Secondary Education

Regarding the characteristics of the scientifically literate students with reference to the variable course option in secondary education, it can be observed in Table 4 that most of the scientifically literate students intended to choose the science and technology course (27%), followed by professional courses (14%), languages and humanities course (10%), socio-economic sciences course (7%) and visual arts course (5%). However, considering the difference in the average level of scientific literacy between the groups intending or not to choose these courses in secondary education, significant differences were found only between the mean of students who did or did not intend to choose the science and technology course and the vocational courses.
The average scientific literacy level of students intending to choose the science and technology course (M = 3.08; SD = 0.73) was statistically higher [t(514) = 323.50, p < 0.001] than that of students not intending to choose it (M = 2.62; SD = 0.69). Conversely, the average scientific literacy level of students aspiring to opt for vocational courses (M = 2.48; SD = 0.68) was statistically lower [t(514) = −6.234, p < 0.001] than that of students not intending to pursue such courses (M = 2.90; SD = 0.73).
The data reinforce the previous results related to the students’ liking for the subjects of natural sciences and physical chemistry. Moreover, these results are in line with the conclusions of the studies of de Camargo et al. (2011), Nascimento-Schulze et al. (2006) and Vizzotto and Del Pino (2020), which identified an interest for the scientific–technological context as being one of the factors that best explained the difference in students’ performance in scientific literacy tests. They are also supported by the research of Noronha and Ambiel (2009), who demonstrated the existence of relationships between the variables school performance and professional interests of the sample studied, and the Portuguese PISA 2018 report, which showed that 48% of boys with high performance in science intend to follow a profession in science and engineering and, in the case of girls, only 15% think of choosing this area (Duarte et al. 2020).

3.6. Scientific Literacy Level According to Socio-Economic Level of the Schools

The results shown in Table 4 reveal that approximately one third of the scientifically literate students study in schools with an intermediate socio-economic level (33%), followed by those studying in schools with unfavourable (16%) and favourable (15%) socio-economic levels, respectively. Moreover, the results of the ANOVA involving the scientific literacy level and the variable socio-economic level of the schools revealed statistical differences between students belonging to schools with favourable and unfavourable socio-economic contexts [F(2, 513) = 3.32, p < 0.05]. On average, the scientific literacy level of students studying in schools with a favourable socio-economic background (M = 2.93; SD = 0.77) was statistically higher (p < 0.05) than the level of students whose background was unfavourable (M = 2.69; SD = 0.65). There was no significant difference between the average scientific literacy level of students from schools with an intermediate socio-economic background (M = 2.76; SD = 0.76) and those with unfavourable and favourable levels.
In Portugal, there is constant concern regarding the socio-economic equity of students. The effect of the school’s socio-economic background on students’ achievements, related to timely conclusion and success in national tests, is one of the factors that has been explored by the Directorate for Statistics on Education and Science of Portugal. In the last report on school achievement and equity available, it was observed that, in the third cycle of basic education, the results of students from schools with an unfavourable socio-economic context were slightly lower than those from schools with a favourable context (DGEEC 2023). Furthermore, according to the 2022 report, although the differences are not very expressive, schools in more favourable socio-economic contexts also tend to show slightly higher results in the educational achievement of students compared with those from unfavourable socio-economic backgrounds (DGEEC 2022).
According to the 2021 and 2022 reports, the impact of the school’s socio-educational context has also been observed in international studies (DGEEC 2021, 2022). Studies and reports of international assessments, such as PISA and TIMSS, show that socio-economically disadvantaged students have a greater tendency to obtain lower performances, to repeat school years and to show early school leaving, when compared to their peers (Savvides et al. 2021; Sudbrack and Fonseca 2021).
According to the PISA 2018 report (Lourenço et al. 2019), in Portugal, “the probability that a student from among the 25% most disadvantaged obtain a score below proficiency level 2 is approximately three times higher than that of a student with higher socio-economic status obtaining that score” (p. VIII). Furthermore, the difference between the economic, social and cultural statuses (ESCSs) of the most-advantaged and least-advantaged students, that is, those with an ESCS above the 95th percentile of the socio-economic and cultural status distribution and those with an ESCS below the 5th percentile, respectively, is 3.6 points. This difference is above the 2.9 points of the OECD countries as a whole, positioning Portugal among the countries with the greatest economic, social and cultural disparity in terms of ESCS distribution (Lourenço et al. 2019).
In the TIMSS 2019 report (Duarte et al. 2020), the results were similar. It was found that, in the science assessment, Portuguese students from schools with a more-advantaged socio-economic composition scored, on average, 34 points more than students attending more-disadvantaged schools, 16 points less than the average of the participating countries. In the present study, and analogous to the results of the scientific literacy level, there was a significant difference [F(2, 513) = 3.23, p < 0.05] only between the mean score of the correct answers of students coming from schools with favourable (M = 17.32; SD = 5.14; p < 0.05) and unfavourable socio-economic status (M = 15.79; SD = 4.11), with the former being higher than the latter.
Furthermore, several studies have demonstrated the association between socio-economic inequalities and high levels of school failure (Azevedo 2021; Couto et al. 2020; Jaloto and Primi 2021; Melo et al. 2021; Pinto 2021; Sudbrack and Fonseca 2021). Jaloto and Primi (2021), for example, state that in all of the studies reviewed in their research, socio-economic status was positively associated with student performance. They further reiterate that “this shows that academic performance in general has been associated with the resources (whether cultural or material) available to the student’s family” (Jaloto and Primi 2021, p. 138).
For Azevedo (2021), socio-economic inequalities have a very clear expression in the education system, translating into high levels of failure and early school leaving. Corroborating this idea, the results of the study by Melo et al. (2021) show that the socio-economic status of families has a strong relationship with student performance, reflecting such an association. Similarly, the research of Couto et al. (2020) revealed that students’ families’ socio-economic status and socio-economic level of the school have significant predictive power regarding students’ performance in science, mathematics and reading, whose explained variation ranges between 20% and 24%, depending on the domain.

3.7. Multiple Linear Regression Analysis

Finally, multiple linear regression analysis also resulted in a statistically significant model [F(10, 505) = 8.13; p < 0.001; R2 = 0.14]. It was noticed that only the variables opting for vocational courses (β = −0.171; t = −3.27; p < 0.01), opting for a science and technology course (β = 0.159; t = 2.81; p < 0.01) and a liking for the physical chemistry subject (β = −0.138; t = −2.73; p < 0.01) were predictors of students’ scientific literacy level, according to the following formula: (y = b0 + b1X1 + b2X2 + b3X3): scientific literacy level = 2.31 − 0.28 × (opting for vocational courses) + 0.25 × (opting for the science and technology course) + 0.10 × (a liking for the physical chemistry subject).
It can be observed that, among the three independent variables, opting for vocational courses explains a major percentage of the variation in the dependent variable, scientific literacy level, 17.1%, followed by the variables opting for a science and technology course and a liking for the subject of physical chemistry, which were 15.9% and 13.8%, respectively. As previously reported, there were statistically significant differences between the average scientific literacy level of the students when analysed in relation to these three variables, corroborating the discussion of the latter data.
However, it is worth noting that this multiple linear regression model is able to explain, approximately, 15% of variability in the scientific literacy level of the ALCE student respondents, showing that other factors that were not analysed in this study could be more efficient in predicting it, such as the study time for the two subjects and the interest in answering the instrument, for example. It was expected that the socio-economic level of the school would also be one of the factors that could predict the students’ scientific literacy levels and back up the study of Couto et al. (2020). Nevertheless, this fact was not verified, strengthening the idea that other variables can better explain and predict the students’ scientific literacy levels.

4. Conclusions

In order to produce a balance of the students’ scientific literacy levels at the end of the third cycle of Portuguese basic education, this study applied the ALCE instrument to 516 students in the ninth grade from 20 public schools in mainland Portugal. The results showed that 49.4% were classified as being at the moderate level, 35.7% were classified at the low level, 14.3% were classified at the high level, 0.6% were classified at the very high level of scientific literacy and 64.14% were classified as being scientifically literate.
When analysing the differences between the average scientific literacy level of the students and the independent variables considered, it was noticed that there were statistically significant differences for the following variables: failure in the subject of physical chemistry, in which the average of those who had not failed was higher than that of students who had already failed; a liking for the subjects of natural sciences and physical chemistry, whereby, as this increases, the higher the average scientific literacy level is; course option in secondary education, whereby students opting for the science and technology course and those not opting for vocational courses obtained the highest average; and socio-economic level of the school, revealing that the average of students from schools whose socio-economic level was favourable was higher than that of students studying in schools where the socio-economic level was unfavourable.
Among all of the variables investigated, the only ones which proved to be capable of predicting the students’ scientific literacy level, according to the multiple linear regression analysis, were as follows: opting for vocational courses, opting for the science and technology course and a liking for the physical chemistry subject. Although this is a correlational study, which ought to be free of casual inference, from these results, it is possible to infer that the area that they intended to continue studying, related to their professional interests, and the liking for the subjects might be associated with better development in students’ scientific literacy skills and could be taken into consideration by school institutions and the government in an attempt to improve science teaching in this educational cycle.
In summary, the results are positive and similar to those verified in external assessments and in other studies assessing scientific literacy in this education cycle or similar. However, it should be noted that approximately one third of the students assessed did not obtain the minimum number of correct answers to be classified as being scientifically literate. Taking into account that these students may not opt for a science and technology course in secondary education, this fact becomes even more relevant, since they, at least during the rest of compulsory education, will no longer have contact with subjects aimed at developing scientific literacy.
This study aimed to address the need for research related to the assessment of scientific literacy and the development of new assessment instruments, as referred to in the systematic literature review conducted by Coppi et al. (2023). The collection and presentation of these data are of great importance at local and national levels in order to produce diagnostic indicators about the situation of science teaching in Portugal, with regard to scientific literacy, so that measures can be taken to improve it. In the international as well as national context, the results of this study and similar studies may contribute to the establishment of comparisons and correspondences, and encourage the development of new instruments capable of assessing students’ scientific literacy.
The study did not include analyses comparing the students’ scientific literacy levels with their respective school grades in the subjects of natural sciences and physical chemistry. This limitation was due to some reasons, such as the time of year that the instrument was applied, in which the schools did not yet have the students’ final grades, the bureaucracy required for accessing this type of information and, mainly, as a result of the application format (online) due to the pandemic caused by COVID-19, which made it impossible for researchers to go to schools to code the students. It is therefore suggested that further studies be able to produce this analysis, since it may provide data and information about the way in which scientific literacy is assessed in these subjects, verifying whether or not there is a correlation between the students’ scientific literacy levels and their grades, contributing to the debate on the scientific literacy level of students in the third cycle of basic education and, consequently, collaborating towards an improvement in scientific and technological education in schools in Portugal.
A further limitation refers to the fact that this study did not address the potential im- pact of the results of applying the ALCE to teachers’ classroom practice. It is therefore suggested that future research could establish such an analysis by relating, for example, the results of the students’ performance in the ALCE with the students’ final averages in the subjects of natural sciences and physical chemistry. Similarly, future studies may, for example, carry out case studies in order to compare the classroom practices of natural sciences and physical chemistry teachers between the schools whose students obtained the highest and the lowest levels of scientific literacy.
The possible limitation related to the format of the item chosen can also be highlighted. Although experts in item development advise the use of true–false items for educational tests and that the validation process of the instrument have a “don’t know” option added to true–false items, this item format is highly criticised by researchers for, from their perspective, encouraging the memorisation of trivial facts and, especially, for the high percentage (50%) of random correct answers in the items. It is suggested, therefore, that the option “don’t know” be well explained by the teachers who apply the instrument and students be encouraged to answer with it when they are not sure of the correct answer, and that certain caution be taken in the use of the assessment results.
Finally, considering that the linear regression model presented in this study is able to explain 15% of the variability in the students’ level of scientific literacy, it is suggested that future studies investigate factors that can more efficiently explain this variability.

Author Contributions

Conceptualization, M.C. (Marcelo Coppi), I.F. and M.C. (Marília Cid); methodology, M.C. (Marcelo Coppi); software, M.C. (Marcelo Coppi); validation, M.C. (Marcelo Coppi), I.F. and M.C. (Marília Cid); formal analysis, M.C. (Marcelo Coppi); investigation, M.C. (Marcelo Coppi); resources, M.C. (Marcelo Coppi), I.F. and M.C. (Marília Cid); data curation, M.C. (Marcelo Coppi); writing—original draft preparation, M.C. (Marcelo Coppi); writing—review and editing, M.C. (Marcelo Coppi); visualization, M.C. (Marcelo Coppi), I.F. and M.C. (Marília Cid); supervision, I.F. and M.C. (Marília Cid); project administration, M.C. (Marcelo Coppi); funding acquisition, M.C. (Marcelo Coppi), I.F. and M.C. (Marília Cid). All authors have read and agreed to the published version of the manuscript.

Funding

This work was funded by national funds through FCT—Fundação para a Ciência e a Tecnologia—I.P., within the scope of the research grant with reference UI/BD/151034/2021 and the project UIDB/04312/2020.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Informed consent was obtained from all of the subjects involved in the study.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses or interpretation of data; in the writing of the manuscript or in the decision to publish the results.

References

  1. Anderson, Lorin, David Krathwohl, Peter Airasian, Katheleen Cruikshank, Richard Mayer, Paul Pintrich, James Raths, and Merlin Wittrock. 2001. A Taxonomy for Learning, Teaching and Assessing: A Revison of Bloom’s Taxonomy of Educational Objectives. New York: Addison Wesley Longman. [Google Scholar]
  2. Azevedo, Joaquim. 2021. A qualidade da educação na perspetiva dos excluídos da escola. In (Re)pensar a Qualidade das Organizações Educativas: Olhares Sobre a Educação Básica, Secundária e Superior. Edited by Manuela Gonçalves, Dora Fonseca, António Neto-Mendes, Alexandre Ventura and Jorge Adelino Costa. Aveiro: UA Editora, pp. 27–50. [Google Scholar]
  3. Cavalcanti, Maria Mônica Pinheiro. 2009. A Relação Entre Motivação para Aprender, Percepção do Clima de Sala de aula Para Criatividade e Desempenho Escolar de Alunos do 5º ano do Ensino Fundamental. Dissertação de Mestrado, Universidade de Brasília, Brasília, Brazil. Available online: https://repositorio.unb.br/handle/10482/4399 (accessed on 1 August 2022).
  4. Coppi, Marcelo Alves, and Clarilza Prado Sousa. 2019. Estudo da alfabetização científica dos alunos do 9º ano do ensino fundamental de um Colégio Particular se São Paulo. Debates Em Educação 11: 169–85. [Google Scholar] [CrossRef]
  5. Coppi, Marcelo Alves, Isabel Fialho, and Marília Cid. 2022. Evidências de validade baseadas no conteúdo de um instrumento de avaliação da literacia científica. Cadernos de Pesquisa 29: 99–127. Available online: https://periodicoseletronicos.ufma.br/index.php/cadernosdepesquisa/article/view/17211/11126 (accessed on 20 July 2022).
  6. Coppi, Marcelo, Isabel Fialho, and Marília Cid. 2023. Scientific literacy assessment instruments: A systematic literature review. Educação Em Revista 39: e37523. [Google Scholar] [CrossRef]
  7. Couto, Alcino Pinto, Patrícia Costa, and Maria Eugénia Ferrão. 2020. Territorial differences in student performance in Portugal: The role of family characteristics and school composition. Revista Portuguesa de Estudos Regionais 57: 89–103. [Google Scholar]
  8. DEB. 2001. Orientações Curriculares Para o 3° ciclo do Ensino Básico—Ciências Físicas e Naturais; Lisboa: Ministério da Educação.
  9. DeBoer, George. 2000. Scientific literacy: Another look at its historical and contemporary meanings and its relationship to science education reform. Journal of Research in Science Teaching 37: 582–601. [Google Scholar] [CrossRef]
  10. de Camargo, Andrea Norema Bianchi, Fabiana Dias Pilar, Marcus Eduardo Maciel Ribeiro, Mirian Fantinel, and Maurivan Güntzel Ramos. 2011. Alfabetização científica: A evolução ao longo da formação de licenciandos ingressantes, concluintes e de professores de química. Momento Diálogos Em Educação 20: 19–29. [Google Scholar]
  11. Despacho normativo n.º 1-F/2016 do Ministério da Educação. 2016. Diário da República, 2.ª série-N.º 66-5 de abril de 2016. Available online: http://www.dge.mec.pt/sites/default/files/Noticias_Imagens/2016_dn_1f.pdf (accessed on 13 June 2023).
  12. DGE. 2018a. Aprendizagens Essenciais, Articulação com o Perfil dos Alunos—Ciências Naturais-7.° ano-3.° ciclo do Ensino Básico. Available online: http://www.dge.mec.pt/sites/default/files/Curriculo/Aprendizagens_Essenciais/3_ciclo/ciencias_naturais_3c_7a_ff.pdf (accessed on 4 March 2022).
  13. DGE. 2018b. Aprendizagens Essenciais, Articulação com o Perfil dos Alunos—Ciências Naturais-8.° ano-3.° ciclo do Ensino Básico. Available online: http://www.dge.mec.pt/sites/default/files/Curriculo/Aprendizagens_Essenciais/3_ciclo/ciencias_naturais_3c_8a_ff.pdf (accessed on 4 March 2022).
  14. DGE. 2018c. Aprendizagens Essenciais, Articulação com o Perfil dos Alunos—Ciências Naturais-9.° ano-3.° ciclo do Ensino Básico. Available online: http://www.dge.mec.pt/sites/default/files/Curriculo/Aprendizagens_Essenciais/3_ciclo/ciencias_naturais_3c_9a_ff.pdf (accessed on 4 March 2022).
  15. DGE. 2018d. Aprendizagens Essenciais, Articulação com o Perfil dos Alunos—Físico-Química-7.° ano-3.° ciclo do Ensino Básico. Available online: http://www.dge.mec.pt/sites/default/files/Curriculo/Aprendizagens_Essenciais/3_ciclo/fisico-quimica_3c_7a_ff.pdf (accessed on 4 March 2022).
  16. DGE. 2018e. Aprendizagens Essenciais, Articulação com o Perfil dos Alunos—Físico-Química-8.° ano-3.° ciclo do Ensino Básico. Available online: http://www.dge.mec.pt/sites/default/files/Curriculo/Aprendizagens_Essenciais/3_ciclo/fisico-quimica_3c_8a_ff.pdf (accessed on 4 March 2022).
  17. DGE. 2018f. Aprendizagens Essenciais, Articulação Com o perfil dos Alunos—Físico-Química-9.° ano-3.° ciclo do Ensino Básico. Available online: http://www.dge.mec.pt/sites/default/files/Curriculo/Aprendizagens_Essenciais/3_ciclo/fisico-quimica_3c_9a.pdf (accessed on 4 March 2022).
  18. DGEEC. 2021. Resultados Escolares—Indicador de Equidade|Ensino Básico e Secundário—2019; Lisboa: DGEEC.
  19. DGEEC. 2022. Resultados Escolares: Sucesso e Equidade|Ensino Básico e Secundário; Lisboa: DGEEC.
  20. DGEEC. 2023. Resultados Escolares: Sucesso e Equidade|Ensino Básico e Secundário; Lisboa: DGEEC.
  21. Duarte, Alexandra, Alexandra Nunes, Joana Tavares, Madalena Mota, and Tânia Venâncio. 2020. TIMSS 2019—Portugal. Resultados a matemática e a ciências—8.° ano—Volume 2. Lisboa: IAVE. [Google Scholar] [CrossRef]
  22. Ebel, Robert, and David Frisbie. 1991. Essentials of Educational Measurement, 5th ed. Englewood Cliffs: Prentice Hail International, Inc. [Google Scholar]
  23. Fernandes, Luana de Mendonça, Vanessa Barbosa Romera Leme, Luciana Carla dos Santos Elias, and Adriana Benevides Soares. 2018. Preditores do desempenho escolar ao final do ensino fundamental: Histórico de reprovação, habilidades sociais e apoio social. Temas Em Psicologia 26: 215–28. [Google Scholar] [CrossRef]
  24. Galvão, Cecília, Sofia Freire, Cláudia Faria, and Mónica Baptista. 2017. Avaliação do Currículo das Ciências Físicas e Naturais: Percursos e Interpretações. Lisboa: Instituto de Educação da Universidade de Lisboa. [Google Scholar]
  25. Ghiglione, Rodolphe, and Benjamin Matalon. 1992. O Inquérito. Teoria e Prática. Oeiras: Celta Editora. [Google Scholar]
  26. Hill, Manuela Magalhães, and Andrew Hill. 2005. Investigação por questionário. Lisboa: Edições Sílabo. [Google Scholar]
  27. Jaloto, Alexandre, and Ricardo Primi. 2021. Fatores socioeconômicos associados ao desempenho no Enem. Em Aberto 34: 125–41. Available online: http://www.emaberto.inep.gov.br/ojs3/index.php/emaberto/article/view/5002 (accessed on 24 August 2022).
  28. Jimerson, Shane, and Phillip Ferguson. 2007. A longitudinal study of grade retention: Academic and behavioural outcomes of retained students. School Psychology Quarterly 14: 314–39. [Google Scholar] [CrossRef]
  29. Jufri, Abdul Wahab, Alifman Hakim, and Agus Ramdani. 2019. Instrument development in measuring the scientific literacy integrated character level of junior high school students instrument development in measuring the scientific literacy integrated character level of junior high school students. International Seminar on Science Education 1233: 012100. [Google Scholar] [CrossRef]
  30. Koedsri, Anusorn, and Sungworn Ngudgratoke. 2018. Diagnostic assessment of scientific literacy of lower secondary school students using g-dina model. Paper presented at the Re-Thinking Teacher Professional Education: Using Research Findings for Better Learning—61st World Assembly ICET 2017, Brno, Czechia, June 28–30; pp. 268–73. [Google Scholar]
  31. Kpolovie, Peter James, Andy Igho Joe, and Tracy Okoto. 2014. Academic achievement prediction: Role of interest in learning and attitude towards school. International Journal of Humanities, Social Science and Education 1: 73–100. [Google Scholar]
  32. Laugksch, Rüdiger, and Peter Spargo. 1996. Construction of a paper-and-pencil test of basic scientific literacy based on selected literacy goals recommended by the american association for the advancement of science. Public Understanding of Science 5: 331–59. [Google Scholar] [CrossRef]
  33. Lourenço, Vanda, Alexandra Duarte, Alexandra Nunes, Ana Amaral, Conceição Gonçalves, Madalena Mota, and Rosário Mendes. 2019. PISA 2018—Ortugal. Relatório Nacional. 156. Available online: http://www.pnl2027.gov.pt/np4/file/1205/RELATORIO_NACIONAL_PISA2018_IAVE.pdf (accessed on 26 July 2021).
  34. Marôco, João, Conceição Gonçalves, Vanda Lourenço, and Rosário Mendes. 2016. PISA 2015—Portugal. Volume I: Literacia Científica, Literacia de Leitura & Literacia Matemática. Lisboa: IAVE. [Google Scholar]
  35. Martins, Guilherme d’Oliveira, Carlos Alberto Sousa Gomes, Joana Maria Leitão Brocardo, José Vítor Pedroso, José León Acosta Carrilho, Luísa Maria Ucha Silva, Maria Manuela Guerreiro Alves da Encarnação, Maria João do Vale Costa Horta, Maria Teresa Carmo Soares Calçada, Rui Fernando Vieira Nery, and et al. 2017. Perfil dos Alunos à Saída da Escolaridade Obrigatória; Lisboa: Ministério da Educação e Ciência—DGE.
  36. Melo, Rafael Oliveira, Anne Caroline de Freitas, Eduardo de Rezende Francisco, and Marcelo Tadeu Motokane. 2021. Impacto das variáveis socioeconômicas no desempenho do Enem: Uma análise espacial e sociológica. Revista de Administração Pública 55: 1271–94. [Google Scholar] [CrossRef]
  37. Miller, Jon. 1983. Scientific literacy: A conceptual and empirical review. Daedalus 112: 29–48. [Google Scholar]
  38. Ministério da Educação e Ciência. 2022. Estatísticas do Ensino Básico e Secundário. Available online: https://infoescolas.medu.pt/bds.asp (accessed on 13 June 2023).
  39. Moreira, Isabel, Inês Santos, and Joseph Conboy. 2010. A retenção académica e performance a ciências em Portugal evidenciadas no PISA 2006. Available online: https://repositorio.ul.pt/bitstream/10451/7782/1/MoreiraSantosConboy2010.pdf (accessed on 30 May 2023).
  40. Nascimento-Schulze, Clélia Maria, Brígido Camargo, and João Wachelke. 2006. Alfabetização científica e representações sociais de estudantes de ensino médio sobre ciência e tecnologia. Arquivos Brasileiros de Psicologia 58: 24–37. [Google Scholar]
  41. Noronha, Ana Paula Porto, and Rodolfo Augusto Matteo Ambiel. 2009. Relações entre desempenho escolar e interesses profissionais de estudantes do ensino médio—Desempenho escolar e interesses profissionais. Psychologica 50: 277–94. [Google Scholar] [CrossRef]
  42. Odegaard, Marianne, Berit Haug, Sonja Mork, and Gard Ove Sorvik. 2015. Budding science and literacy. A classroom video study of the challenges and support in an integrated inquiry and literacy teaching model. Procedia Social and Behavioral Sciences 167: 274–78. [Google Scholar] [CrossRef] [Green Version]
  43. OECD. 2017. PISA 2015—Assessment and Analytical Framework: Science, Reading, Mathematic, Financial Literacy and Collaborative Problem Solving. Paris: OECD Publications. [Google Scholar]
  44. Oliveira, Carmem Fruhauf. 2021. Práticas Pedagógicas e Desempenho Escolar no Brasil. Dissertação de Mestrado, Universidade Federal de Viçosa, Viçosa, Brazil. Available online: https://poseconomia.ufv.br/wp-content/uploads/2022/02/texto-completo-Carmem.pdf (accessed on 15 October 2022).
  45. Oliveira, Will Fadul Alencar, and Maria Clara Silva-Forsberg. 2012. Níveis de alfabetização científica de estudantes da última série do ensino fundamental. Atas Do VIII Encontro Nacional de Pesquisa Em Educação Em Ciências—VIII ENPEC, 00–11. Available online: http://www.nutes.ufrj.br/abrapec/viiienpec/resumos/R0671-1.pdf (accessed on 26 April 2021).
  46. Özdem, Yasemin, Pınar Çavaş, Bülent Çavaş, Jale Çakıroğlu, and Hamide Ertepınar. 2010. An investigation of elementary students’ scientific literacy levels. Journal of Baltic Science Education 9: 6–19. [Google Scholar]
  47. Pasquali, Luiz. 2009. Psicometria Teoria dos Testes na Psicologia e na Educação, 4th ed. Petrópolis: Vozes. [Google Scholar]
  48. Pereira, Leonor Rebelo. 2019. Instrução, empenho e desempenho dos alunos em ciências. Dissertação de Mestrado, Universidade do Minho, Braga, Portugal. Available online: https://repositorium.sdum.uminho.pt/handle/1822/62315 (accessed on 12 June 2023).
  49. Pinto, Andreia Alexandra da Silva. 2021. A Influência das Condições Socioeconómicas das Famílias na Aprendizagem dos/as Alunos/as—Estudo de caso. Dissertação de mestrado, Politécnico de Coimbra, Coimbra, Portugal. Available online: https://comum.rcaap.pt/handle/10400.26/36549 (accessed on 24 August 2022).
  50. Pinto, Joaquim, Teresa Neto, and Jaime Carvalho e Silva. 2019. Fatores influenciadores do desempenho de estudantes portugueses, singapurenses, holandeses, espanhóis e brasileiros em literacia matemática no PISA: Revisão Integrativa. Revista de Estudios y Experiencias En Educación 18: 41–60. [Google Scholar] [CrossRef]
  51. Rachmatullah, Arif, Sariwulan Diana, and Nuryani Rustaman. 2016. Profile of middle school students on scientific literacy achievements by using scientific literacy assessments (SLA). AIP Conference Proceedings 1708: 1–5. [Google Scholar] [CrossRef] [Green Version]
  52. Rebelo, José. 2009. Efeitos da retenção escolar, segundo os estudos científicos, e orientações para uma intervenção eficaz: Uma revisão. Revista Portuguesa de Pedagogia 43: 27–52. [Google Scholar] [CrossRef]
  53. Saraiva, Leonor. 2017. A aprendizagem das ciências em Portugal: Uma leitura a partir dos resultados do TIMSS e do PISA. Mediações 5: 4–18. [Google Scholar]
  54. Savvides, Nicola, Sandrina Milhano, Catarina Mangas, Carla Freire, and Sara Lopes. 2021. ‘Failures’ in a failing education system: Comparing structural and institutional risk factors to early leaving in England and Portugal. Journal of Education and Work 34: 789–809. [Google Scholar] [CrossRef]
  55. Silva, Rauã Bezerra, Jerônimo Ferreira Mendes, Thainá Maria da Silva Carvalho, and Sanderson Hudson da Silva Malta. 2018. O gostar do aluno e sua dificuldade em química. In Anais Do Anais Do V Congresso Nacional de Educação—CONEDU. Campina Grande: Realize Eventos Científicos & Editora. [Google Scholar]
  56. Soobard, Regina, and Miia Rannikmäe. 2011. Assessing student’s level of scientific literacy using interdisciplinary scenarios. Science Education International 22: 133–44. Available online: https://eric.ed.gov/?id=EJ941672 (accessed on 26 April 2021).
  57. Sudbrack, Edite Maria, and Dora Maria Ramos Fonseca. 2021. A avaliação em larga escala no bojo da regulação supranacional. In (Re)pensar a Qualidade das Organizações Educativas: Olhares Sobre a Educação Básica, Secundária e Superior. Edited by Manuela Gonçalves, Dora Fonseca, António Neto-Mendes, Alexandre Ventura and Jorge Adelino Costa. Aveiro: UA Editora, pp. 149–69. [Google Scholar]
  58. Vizzotto, Patrick Alves, and José Cláudio Del Pino. 2020. Medida del nivel de alfabetización científica en alumnos recién ingresados y del último año de los cursos de física. Revista de Ensañnza de La Física 32: 21–30. [Google Scholar]
  59. Vizzotto, Patrick Alves, Liane Sera Rosa, Valesca de Matos Duarte, and Luiz Fernando Mackedanz. 2020. O uso do Teste de Alfabetização Científica Básica em estudantes do ensino fundamental: Análise da confiabilidade de medida nesse grupo. Research, Society and Development 9: e79932447. [Google Scholar] [CrossRef] [Green Version]
  60. Wang, Jingying, and Ying Zhao. 2016. Comparativeresearch on the understandings of nature of science and scientific inquiry between science teachers from Shanghai and Chicago. Journal of Baltic Science Education 15: 97–108. [Google Scholar] [CrossRef]
  61. World Economic Forum. 2015. New Vision for Education—Unlocking the Potential of Technology. Geneva: World Economic Forum. [Google Scholar]
  62. Yacoubian, Hagop. 2018. Scientific literacy for democratic decision-making. International Journal of Science Education 40: 308–27. [Google Scholar] [CrossRef]
Socsci 12 00374 g001 550
Figure 1. Students’ scientific literacy levels. Source: elaborated upon by the authors.
Figure 1. Students’ scientific literacy levels. Source: elaborated upon by the authors.
Socsci 12 00374 g001
Socsci 12 00374 g002 550
Figure 2. Students’ scientific literacy levels by category of a liking for the subject of natural sciences. Source: elaborated upon by the authors.
Figure 2. Students’ scientific literacy levels by category of a liking for the subject of natural sciences. Source: elaborated upon by the authors.
Socsci 12 00374 g002
Socsci 12 00374 g003 550
Figure 3. Students’ scientific literacy levels by category of a liking for the subject of physical chemistry. Source: elaborated upon by the authors.
Figure 3. Students’ scientific literacy levels by category of a liking for the subject of physical chemistry. Source: elaborated upon by the authors.
Socsci 12 00374 g003
Table
Table 1. Distribution of the ALCE items by subtest.
Table 1. Distribution of the ALCE items by subtest.
SubtestNumber of Items
NS06
ISTS06
CS22
Source: elaborated upon by the authors.
Table
Table 2. Levels of scientific literacy according to the percentage and number of correct answers per subtest.
Table 2. Levels of scientific literacy according to the percentage and number of correct answers per subtest.
LevelsPercentageNSISTSCS
1—Very low<20%≤1≤1≤4
2—Low20 to 49%225 to 10
3—Moderate50 to 69%3 to 43 to 411 to 15
4—High70 to 89%5516 to 19
5—Very high≥90%66≥20
Source: elaborated upon by the authors.
Table
Table 3. General scientific literacy levels according to the average of the subtest levels.
Table 3. General scientific literacy levels according to the average of the subtest levels.
LevelsAverage of the Subtest Levels
Very low<1.00
Low1.00 to 2.49
Moderate2.50 to 3.49
High3.50 to 4.49
Very high≥4.50
Source: elaborated upon by the authors.
Table
Table 4. Characteristics of scientifically literate students, according to the variables analysed.
Table 4. Characteristics of scientifically literate students, according to the variables analysed.
VariableScientifically Literate StudentsNon-Scientifically Literate Students
n%n%
SexMale16933%9017%
Female16231%9519%
Liking for the subject of natural sciencesI do not like it71%102%
I do not like it very much438%275%
I like it15530%10721%
I like it a lot7414%275%
I adore it5210%143%
Liking for the subject of physical chemistryI do not like it224%296%
I do not like it very much6513%479%
I like it14528%7915%
I like it a lot7014%204%
I adore it296%102%
Failure in the subject of natural sciencesYes183%143%
No31361%17133%
Failure in the subject of physical chemistryYes275%245%
No30459%16131%
Course option in secondary educationScience and technology13827%337%
Socio-economic sciences377%214%
Languages and humanities5210%357%
Visual arts255%234%
Professional courses7414%7815%
Socio-economic level of the schoolsUnfavourable8416%5310%
Intermediate16833%10420%
Favourable7915%285%
Source: elaborated upon by the authors.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Coppi, M.; Fialho, I.; Cid, M. Assessing Portuguese Elementary School Students’ Scientific Literacy: Application of the ALCE Instrument. Soc. Sci. 2023, 12, 374. https://doi.org/10.3390/socsci12070374

AMA Style

Coppi M, Fialho I, Cid M. Assessing Portuguese Elementary School Students’ Scientific Literacy: Application of the ALCE Instrument. Social Sciences. 2023; 12(7):374. https://doi.org/10.3390/socsci12070374

Chicago/Turabian Style

Coppi, Marcelo, Isabel Fialho, and Marília Cid. 2023. "Assessing Portuguese Elementary School Students’ Scientific Literacy: Application of the ALCE Instrument" Social Sciences 12, no. 7: 374. https://doi.org/10.3390/socsci12070374

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop