Errors concerning Statistics and Probability in Spanish Secondary School Textbooks

Abstract

Textbooks are considered essential, providing a hierarchical organisation of knowledge, forging the intellectual scaffolding of students and teachers alike, and playing a crucial role in compulsory education. In this paper we discuss, by means of a content analysis, the systematic errors detected in the presentation of questions related to statistics and probability in Spanish secondary school textbooks on mathematics. We found some errors appear systematically in the texts, and the most common are: faulty differentiation between quantitative and qualitative variables, between discrete and continuous variables and between randomness and determinism, confused examples for the bar charts, uncritical choice for graphic representations, inaccuracies in specific vocabulary, and ignoring prior probabilities and a poor consideration about representativeness. We classify the observed errors considering that some of these errors arise from the inherent difficulty of the content and others arise from differences between mathematical and statistical thinking as well as from judgments based on heuristic rules. Knowing the existence of these errors and the reasons why they occur are key points to make them disappear from statistical lessons and to help citizens achieving true statistical literacy.

1. Introduction

There is wide acknowledgement of the need for statistical literacy among nonexperts. In consequence, mathematics curricula have long included aspects of statistics and probability as part of compulsory secondary education. In Spain, this inclusion began over 30 years ago with the establishment of minimum requirements in this respect, and it has since been modified in successive educational laws, to the extent that stochastic notions are now addressed, in response to the increasingly competence-focused approach of recent curricular reforms.

However, the teaching of statistics and probability in mathematics classes has been criticised as not really ‘fitting in’, for several reasons, but chiefly the fact that statistical thinking presents notable differences from purely mathematical considerations. Thus, observing statistics under a mathematical lens can lead to the nonconsideration of elements that are crucial in statistics, but which in mathematics do not play a significant role. In addition, mathematics teachers often have only limited training in statistics, and so this discipline is often abbreviated or omitted.

Consequently, we consider that equipping teachers with good instructional materials (i.e., appropriate, effective textbooks) could be a determinant factor in achieving statistical literacy among the general population.

In the present study, we analyse a range of mathematics textbooks for first and second year students in secondary education (Grades 7–8). We find that, despite the time and effort invested in developing statistical literacy, certain errors systematically appear in statistics and probability chapters within mathematics textbooks. Thus, much remains to be done in developing generalised statistical literacy and a stochastic sense.

1.1. Statistical Literacy, Statistical Thinking and Intuition

Already since 1990s, numerous authors have highlighted the need for education in statistical literacy as a fundamental aspect of modern life, enabling citizens to understand and make use of numerical data. In this respect, Snee [1] argues that within this need to acquire a better understanding of statistics, it is important to focus on statistical thinking in terms of the concepts or systems involved, and not merely to consider the knowledge and use of statistical tools. Other authors have also remarked on the need to address a serious problem, namely that most people do not properly understand statistical reasoning and fail to appreciate the value of the discipline [2,3,4].

This referred need is still present. We can found in literature sentences like “it is necessary to continuously insist on improving statistical knowledge and skills, through all forms of education”, “Statistical literacy is recognised as a key tool for a proactive and critical society in a modern environment”, “Statistics still seem to be unfairly neglected”, and “due to insufficient understanding and insufficiently widespread culture of statistical literacy, there is a misinterpretation of information presented daily in different type of media” [5]. In [6] remains the basic idea that “citizens must be capable of evaluating such statistical information thoroughly before making any decision” and “Statistical literacy is a necessary skill for adults”. Other authors are even currently arguing for a better statistical education and statistical literacy among citizens [7,8,9].

Statistical literacy has been defined as “the ability to understand and critically evaluate statistical results that permeate our daily lives coupled with the ability to appreciate the contributions that statistical thinking can make in public and private, professional and personal decisions” [10], p. 1. According to Moore [11], adults should develop a positive view of themselves as individuals capable of statistical and probabilistic reasoning and have a willingness and interest to “think statistically” in relevant situations. Thus, adults should appreciate the power of statistical processes and accept that properly planned studies have the potential to lead to better or more valid conclusions than those drawn from anecdotal data or personal experience.

There seems to be a consensus [12] that statistics should be present in school education and that it should be taught as part of the mathematics subject. However, various authors have observed that statistics and statistical thinking do not fully fit within the mathematical discipline and its reasoning and hold that statistics is a mathematical science in itself, not just a branch of the main body. One of the forerunners in identifying this dividing line between mathematics and statistics was Tukey [13].

Following Gal and Gardfiel [14], Burrill and Biehler [15] compiled a catalogue of fundamental ideas in statistics, highlighting the points of conflict between mathematical and statistical thinking, in areas such as how data are managed and presented, the nature of variation, the notion of distribution (fundamental to statistical theory), the role of the association between variables, and the construction and use of probabilistic models. These ideas were subsequently developed by Rossman and Medina [16] and Scheaffer [12].

Another important consideration is that in statistics and probability over-reliance on intuition often leads to erroneous conclusions being drawn [17,18,19]. Moore [11] expressed this idea indicating that statistical thinking does not come naturally to most people.

1.2. Teaching Statistics and Probability

According to Rossman and Medina [16], the preparation needed to teach statistics is different from that needed to teach mathematics. Furthermore, students react in different ways to the two types of operation.

As a consequence, although successive education laws have included the study of concepts related to statistics and probability within mathematics courses, the former are frequently ignored by teachers, many of whom do not feel comfortable with the subject, and often leave it as the last to be addressed during the academic year, and end up not teaching it [18,20].

Fernández de Carrera [21], in a study of mathematics teachers, observed that the majority did not teach statistics and probability, and among those who did, only part of the syllabus was taught. Moreover, many of the teachers interviewed had confused ideas about concepts such as randomness and independence.

1.3. Errors in Mathematics Education

Erroneous knowledge, conditions that make it possible and the role that it can play in the domain and advancement of science have occupied an important part of the reflections of philosophers of science and epistemologists. Rico [22] presents a review on the study of errors in the teaching and learning of mathematics until the 1990s, showing that it has been a constant object of concern in mathematics education, which has been characterised by different approaches and interests, conditioned by the goals and ways of organisation of mathematics curricula as well as by certain predominant currents of psychology and pedagogy.

This interest in error has focused, fundamentally, on studies on the errors that students make when learning mathematics. Concretely, the observed fact that students fail when they are asked to complete some tasks by providing wrong responses or, simply, by answering nothing. Two main approaches have been considered regarding the use of errors. The former considers errors as an opportunity to promote student learning. The latter emphasises the analysis of the error by the teacher who identifies the error and diagnoses the student to improve their performance [22,23].

From this issue, a specific vocabulary related to the study of errors, in which error, obstacle, and misconception are the best known specific terms, has arisen. However, in the literature, some studies demonstrated that people cannot distinguish between error and misconception. On one hand, misconceptions emerge as a result of experiences and wrong beliefs of individuals, becoming a subset of errors, which means one can define all misconceptions as errors but all errors may not be misconception [24]. In other words, error is the result of misconception, or misconception is a type of perception which systematically produces error [25].

On the other hand, obstacles emerged in philosophy of science in the seminal works of Bachelard, and Brousseau was the first to transpose this concept to the didactics of mathematics. Instead of considering the error as a result of ignorance of chance, he highlighted the effect of prior successful knowledge on the performance of students when facing new tasks. Among the obstacles to learning, Brousseau distinguishes indeed the ontogenic obstacles, the didactical obstacles, and the epistemological obstacles [26].

The purpose of the characterisation of mathematical conceptions/misconceptions and obstacles is that this allows us to identify the different components that are implied in the understanding of a given concept [27]. Since both misconceptions and obstacles have a marked cognitive character and the present research does not focus on students but textbooks, in the current work we use error as a generic term that allows us to detect precise situations in which there is correct knowledge that opposes an alternative.

Additionally, due to errors do not arise in an unpredictable way, several authors have tried to organise errors in order to explain their causes (e.g., [28,29]). Also in statistics and probability education, some classifications can be found. Garfield and Alhgren [30], Batanero [27], and Batanero et al. [31] refer to obstacles or misconceptions to claim reasons for errors in teaching and learning statistics and probability. Among the causes they propose, we underline:

• Existence of false intuitions or heuristics rules
• Differences between mathematical and statistical thinking

1.4. Textbooks and Errors in Mathematics Education

According to the Spanish National Association of Book and Teaching Material Publishers [32], textbooks still play a very important role in teaching and learning. Teachers need them to guide the approach taken in the classroom and students need them to underpin the learning experience [33]. Although most books are created with the student in mind, in practice, they are mainly used by the teacher.

In most education contexts, textbooks represent an expression of power, contributing to the standardisation of a discipline, to cultural levelling and to the propagation of dominant ideas [34]. Another fundamental consideration is that textbooks often bear a stronger influence on teaching practices than legislative regulations [35]. In view of this crucial role, an increasing number of research studies in this respect are being conducted, as evidenced in several recent conferences on the use of textbooks, such as the four editions of the International Conference on Mathematics Textbooks Research and Development and also in the studies by Fan et al. [36] and Schubring and Fan [37].

Many studies of learning in general and of mathematics learning in particular have focused on the mistakes made by students. However, considerably less is known about the errors committed in textbooks and how they may influence learning. On reviewing the proceedings of the last three International Conferences on Mathematics Textbooks Research and Development (years 2014, 2017, 2019), the working group of the Congress of European Research on Mathematics Education (CERME) and the 38th Topic Study Group of the 13th International Congress on Mathematical Education (ICME—13), we only found one paper referring to this topic [38]. This study examined how students react to errors that appear in textbooks, concluding that these errors have a negative impact on student performance. In our opinion, the absence of research into errors in mathematics textbooks reflects the implicit assumption that no such errors exist.

1.5. Research Goal

In this paper, our study aim is to analyse, classify, and explain the systematic errors contained in the pages dedicated to statistics and probability in Spanish mathematics textbooks for students in the first and second years of compulsory secondary education (Grades 7–8).

2. Method

Our approach consists of a qualitative, exploratory content analysis that defines a strict and systematic set of procedures and steps for a rigorous analysis of written data, textbooks in this case [39]. There is no universally accepted method for performing content analysis, so we adapt the stages proposed by Cohen et al. [40].

2.1. Context

In the Spanish educational system, statistics and probability topics take part of the mathematics curriculum from the initial grades of primary to the end of secondary. Consequently, these contents are taught within the mathematics subject. In grades 7th–8th, the contents described in the regulations are: Population and individual. Sample. Statistical variables. Qualitative and quantitative variables. Absolute and relative frequencies. Organisation in tables of data collected in an experiment. Bar and sector diagrams. Frequency polygons. Measurements of central tendency. Measurements of dispersion. Deterministic and random phenomena. Formulation of conjectures about the behavior of simple random phenomena and design of experiments for their verification. Relative frequency of an event and its approximation to probability through simulation or experimentation. Equiprobable and not equiprobable elementary events. Sample space in simple experiments. Simple tables and tree diagrams. Calculation of probabilities using Laplace’s rule in simple experiments [41].

2.2. Sample

We analysed eighteen mathematics textbooks for first and second-year students in compulsory secondary education (Grades 7–8). The analysed books are listed in

Table 1. Textbooks analysed; grade, publisher, year of publication, and ISBN.

Grade	Publisher	Year	ISBN
7th	Anaya	2020	978-84-698-6933-8
7th	Bruño	2020	978-84-696-1956-8
7th	Editex	2020	978-84-9161-837-9
7th	McGraw Hill	2019	978-84-486-1542-0
7th	Oxford	2020	978-01-905-3509-4
7th	Santillana	2020	978-84-9132-556-7
7th	SM	2020	978-84-1318-507-1
7th	Teide	2020	978-84-307-7029-8
7th	Vicens Vives	2020	978-84-682-5781-5
8th	Anaya	2021	978-84-698-7920-7
8th	Bruño	2021	978-84-696-3147-8
8th	Edebé	2021	978-84-683-5182-7
8th	Edelvives	2021	978-84-140-3100-1
8th	Editex	2021	978-84-1321-205-0
8th	Oxford	2021	978-01-905-3956-6
8th	Santillana	2021	978-84-9132-721-9
8th	Teide	2018	978-84-307-8914-6
8th	Vicen Vives	2021	978-84-682-7813-1

.

All of these texts were the most recent editions available, published in Spain and in compliance with applicable rules and regulations [41]. The sample was obtained according to availability and included seven books in this field from the ten largest publishers of educational texts [42].

For each book, information is collected from chapters dedicated to statistics and probability, computing the basic characteristics of each text, such as: number of pages of the book, number of pages devoted to statistics and probability; number of units in which these topics are treated and position they occupy within the organisation of the book; number of pages devoted to statistics topics; number of pages devoted to probability topics; number of examples or solved exercises for each topic; and number of exercises proposed. In addition, we also gathered data on the date of publication of the book, number of authors, group according to market share and other general characteristics. By way of summary, the mean of the number of pages in the books is 287, of which 36 (12.5%) are dedicated to statistics and probability, which, in most cases (16 books, 89%) include these topics at the end of the book. We summarise some information collected during the analysis of the textbooks in

Table 2. Textbooks analysed; mean, standard deviation, minimum, and maximum calculated for the total number of pages, number of proposed exercises, and number of solved examples for each statistics and probability units.

	Mean	Stand. Dev.	Min.	Max.
Total textbook pages	287	32.25	252	365
Pages on Descriptive Statistics	9.75	3.28	4	18
Pages on Probability	5	4.12	0	16
Exercises on Descriptive Statistics	53.13	12.10	27	69
Examples on Descriptive Statistics	12.36	7.94	4	31
Exercises on Probability	30.94	23.86	0	44
Examples on Probability	5.38	4.51	0	14

.

2.3. Units of Analysis

For each book, all the chapters or sections related to statistics and probability were reviewed in depth. To carry out the analysis, three types of analysis units have been considered: (a) paragraphs that develop the topics, (b) examples, and (c) tasks.

We compared all the terms—and how they are used—with the standard definitions found in The OECD Glossary of Statistical Terms [43]. In case of discrepancy, we compare it with other sources [44,45] in order to determine if this conflicting use is an error or not. We consider it to be an error if its meaning does not match the one given in references [43,44,45]. If this is the case, we note for each text how many times it is occurring, describe it, and analyse it. Once this has been carried out, the so-called systematic errors are detected. The following criteria are applied to categorise a systematic error as such: (1) it must appear several times in the same book; (2) it must appear in at least one third of the texts examined.

2.4. Categories for Analysis

The categories for analysis were initially organised by content blocks: descriptive statistics and probability. When a systematic error was detected, its was written down. Once the errors were gathered, through an inductive reasoning, by which themes emerge from the data through the researcher examinations and constant comparison [46]. Results section includes the emerging themes and the analysis carried out.

Finally, we rearrange the themes into the list of difficulties provided in the introduction section in order to suggest some causes of the described errors.

3. Results

3.1. Errors Detected in Texts on Descriptive Statistics

3.1.1. Between the Quantitative and the Qualitative

The definitions of quantitative and qualitative variables offer a dichotomous view: variables will be of either one type or another depending on whether their values are numeric or not. In the proposed tasks, statements such as “Classify the following variables as qualitative or quantitative: Age, Favourite colour, Number of customers in a bar in one day” suggest that there is an evident partition and that the variables are naturally and intrinsically in one type or another, and therefore will belong to one or another category unequivocally.

However, the quantitative or qualitative character is really assigned to a variable depending on how it is observed and on the context. For example, “age” could be obtained as “number of years completed” or as belonging to one of the two categories “older/younger”, quantitatively in the first case and qualitatively in the second.

In addition, this type of task is usually followed by a second part asking students to perform exercises such as the following: “Provide an example of a variable that can be studied quantitatively and qualitatively”. However, this task suggests there is a contradiction between the two statements. We cannot make a classification as requested in the first part of the exercise if we accept that one or another nature may be assigned to the variables in question depending on how they are observed. Furthermore, a variable cannot belong to both types, because by definition it is assigned to just one category.

The above example illustrates a common situation in these texts, where the reader is led to believe that a brief description of the variable, such as “Age”, “Favourite colour” or “Number of customers in a bar in a day” is sufficient to determine the measurement scale that should be used.

Consideration of the context, as observed by Wild and Pfannkuch [47], is one of the key dimensions of statistical thinking. Scheaffer [12] emphasises that, in statistics, the context is crucial, adding that one of the major differences between statistics and mathematics is that measurements play a fundamental role in statistics. According to this author, the way in which data are obtained is not relevant from a mathematical standpoint but is of fundamental importance from a statistical perspective.

3.1.2. Bar Charts Lacking Frequencies

The mere use of bars in a representation does not mean that the graph should be called a bar chart. This term is reserved for a chart where the bars, one for each modality of the variable, represent the frequency of the modalities. Surprisingly, we found that in many texts, nonbar graphs are proposed—and they are nonbar despite the fact that bars are used in the representation—and are called bar diagrams.

This type of confusion usually occurs in the examples given and in exercises proposed, with the role of a frequency distribution being taken, in many cases, by a time series. Thus, the distributions of the modalities together with their frequencies $\{x_i,n_i\}$ are replaced by the observed values (whose frequency is unitary) over time $\{t_i,X\left(t_i\right)\}$.

Another example where this type of confusion appears is in exercises like the one illustrated below: “Ana has 7 books, Daniel has 12, Julia 10, and Luis has 8. Represent these data in a bar chart”. In this case, when students are asked to represent the information by means of a bar graph, this means they should organise the information within a graph containing four bars—one for each individual—with heights varying according to the number of books possessed. However, in this case, a bar graph should have four bars, each representing one of the observed quantities, all with the same height; this is because the height is determined by its frequency, which in the case in question is unitary, for each student. Figure 1 shows what the expected solution looks like and what would actually be obtained according to the definition of a bar graph.

3.1.3. Uncritical Choice of Graphic Representation

Errors can also be found in the type of graph used to represent the information. Thus, although many books use ordinal variables to exemplify the pie chart, it would be more appropriate to select a bar chart for this type of variable and to reserve the pie chart for nonordinal qualitative variables.

None of the textbooks analysed provide any indications about the adequacy of one or another form of graphic representation according to the observed data, nor about the resulting legibility and interpretability. No warnings were given, either, about the difference between the use of bar charts for qualitative or for quantitative variables, or about what type of scale should be considered for the horizontal axis.

Moreover, the textbook examples usually consider the application of both types of figure for both types of variables, and the exercises frequently require the student to obtain the two graphic representations for the same set of data.

In addition to this problem regarding criteria for selecting the type of graphic representation, the frequency polygon is included in the curriculum for these courses. This concept is confused in all the texts, which provide a definition that corresponds to a polygonal line that joins the highest points of the bars from a bar chart. None of the texts provides an interpretation of this graph, and the polygon is never completed by closing the ends with segments that join the X axis with the first and last of the bars. Obviously, using a frequency polygon or part of it to represent a nonordinal qualitative variable—a common feature of these texts—is hard to interpret and contributes very little to understanding the question.

3.1.4. Measures, Estimators and Parameters

The term “parameter” is sometimes used in reference to the arithmetic mean or other measures of position or dispersion. In this context, the word “parameter” is extremely inappropriate because the text does not deal with population data but with small data sets. The correct expression, then, would be “measures”. The difference between these two terms is the fundamental basis of statistical inference and transcends this topic to become a core element of statistical versus mathematical thinking. Moreover, at higher levels, the difference between the two concepts becomes part of the content. Specifically, in the second year of high school (Grade 12), the curriculum subject of Mathematics Applied to Social Sciences II includes “Parametric statistics: parameters of a population and statistics obtained from a sample”.

In their analysis of university entrance exam performance in activities related to statistics, Espinel et al. [48] found that high school students (in the subject “Mathematics Applied to Social Sciences”) had difficulty discerning between parameter, estimator, and estimation. The fact that students’ textbooks label the basic measurements of position and dispersion as “parameters” may be a relevant factor in the failure to make this necessary distinction.

3.1.5. Discrete Measures for Continuous Variables and Vice Versa

One error that is frequently observed is to consider a numerical variable, which could be taken as continuous, and then making discrete observations of it, thus generating intervals and diluting the discrete information into intervals for a continuous variable. Subsequently, the measures are calculated using the class mark, which rediscretises the values, but does so inaccurately.

In this respect, consider the following example: “A group of children take the following time (in minutes) to travel from their home to the school: 34, 12, 9, 21, 5, 2, 20, 18, 4, 9, 16, 18, 14, 7, 32, 10, 21, 6, 11, 13. Summarise the information in a table containing intervals of 5 minutes, starting at 0. With the table obtained, calculate the position measurements and dispersion”.

For this type of task, it should be noted that if the variable is really continuous the values given (all integers) will not actually be observed. It might have been more appropriate to propose the intervals from the beginning of the study and then observe the time interval corresponding to each individual. If the variable “number of minutes” is really the variable of interest, then the creation of intervals is not very informative, only approximating the value of the closest class mark for the calculation of the measurements, instead of using the true values observed.

Hence, fitting these observations into a scheme proposed for continuous variables does not improve the calculation of the measures. Such an approach might provide a resource enabling the information to be presented in the form of a histogram and thus more userfriendly, but this is not the case with the calculation of measures. The opposite is true when obviously discrete variables are considered and the information is presented in tables divided into intervals, such as “number of weekly hours dedicated to studying” or “age (in years) of the members of a club (values between 13 and 19)”.

3.2. Errors Detected in Textbook Chapters on Probability

3.2.1. Between Randomness and Determinism

Closely related to the errors found in the texts on descriptive statistics, those addressing probability also contain ambiguities regarding certain differentiations, such as the classification of an experiment as either random or deterministic. To illustrate the ambiguity, let consider a statement such as “Classify the following experiments as random or deterministic: measure the volume of the content of a drinks can, draw a card from a deck, …”.

It is important to note that the nature of the experiment is not determined merely by this brief description. A more complete description of the context is needed, telling us what type of measurement instrument is used and under what conditions the experiment is performed. As observed by Vermette and Savard [49], it is precisely the consideration of variability that indicates the lack of determinism. According to Wozniak [50], variability is a key concept in statistical thinking. Indeed, statistics can be defined as the science of variability within natural and social phenomena in the world around us.

In the first of the examples quoted, the drinks firm’s quality control department will undoubtedly consider the volume of liquid to be a random variable, with very little variability to be controlled. However, the description of a problem in any other area addressed in the same mathematics textbook would not consider any possibility other than that a can of x capacity actually contains exactly x units of capacity.

In the second case of our example, the extraction of a card from a deck, some aspects remain unclear. Is the deck ordered or disordered? Are the cards face up or face down during the extraction? Is the number and suit of the card extracted the result of the experiment or is some other phenomenon being studied?

Since the experiments are poorly described, the lack of detail about the context in the exposition of the experiments can lead to a single case being considered random and deterministic simultaneously.

3.2.2. Proposal for Random Experiments That Are Not, in Fact, Random Experiments

Another type of confusion often observed in these texts is the failure to distinguish between random experiment and event. As an example: “Classify the following experiments as random or deterministic: throw a totem and it lands feet-first, spin a roulette wheel and the ball lands on a certain number, …”. These events (the totem landing feet-first, the roulette ball falling on a particular number) are specified as if they were part of the random experiment. In the first case, the experiment is to throw the totem pole and see how it falls, while in the second, it is to spin the wheel and see where the ball falls. The totem landing feet-first and the ball falling on a specific number actually are events associated with these experiments.

With respect to randomised experiments, ambiguity may also be caused by inadequate description and lack of context in the examples given and the tasks proposed. For instance, in the following statement: “A bowl contains 3 red balls, 2 blue ones and a white one. Describe the sample space”. There is an obvious omission in the description of the random experiment, namely the assumption that whenever there is a bowl containing balls, the experiment will consist in taking one out, after mixing up the balls, without looking and without any additional information about the balls or how to choose a certain one. Equivalently, whenever there is a die the experiment will consist in throwing it, whenever there is a deck of cards the experiment will involve drawing one, etc. The failure to define the experimentation completely results in a lack of information, which students are then required to overcome by guessing.

Furthermore, the richness with which the same starting situation can give rise to different random experiments and to different measurements of nonidentical random phenomena is diminished.

This ambiguity also has effects in the opposite direction, that is, when the statement provides an overdefinition of the random experience, turning the definition of the experiment into the definition of the experiment together with a random variable. This situation is illustrated in a statement such as: “Consider the random experiment consisting in tossing three coins and observing the number of heads that come up”.

Sometimes this type of situation is accompanied by information on other aspects, for example, proposing the set $\{0,1,2,3\}$ as a sample space, in a clear supplanting of the true sample space $\mathsf{\Omega }=\left\{\right(h,h),(h,t),(t,h),(t,t\left)\right\}$ over the path of a discrete random variable X defined as “number of heads observed”.

3.2.3. Ignore A Priori Probabilities

The error of ignoring a priori probabilities is frequently present. For example, in the statement: “In a television repair company, 15 televisions of brand A, 35 of brand B and 10 of brand C are received in one week. Which brand seems to be more defective?” It is clear that without information on a priori probabilities, this question cannot be properly answered.

As observed by Tversky and Kahneman [17], ignoring a priori probabilities is a heuristic that leads the practitioner to instinctively suggest an erroneous solution to a probability calculation problem. If the question or exercise omits this information, the a priori probability will seem to be irrelevant in the search for the solution, resulting in the establishment of the heuristic and obscuring the appropriate way to solve this type of problem (not usually taught until a more advanced level, in which the Bayes rule is studied).

3.2.4. Use of Inappropriate Vocabulary

Some of the texts examined use inconsistent and sometimes inappropriate vocabulary, for example, in the definitions of complementary and elementary events, naming them “opposite event” and “case” respectively; in addition, some texts use “possible” instead of “probable” and “randomly” instead of “randomly and equiprobable”.

It is not always valid to use the term “complementary” instead of “opposite” in the negation of the logical quantifier. Their meanings may coincide in some circumstances, but generally, they do not. This type of conflict occurs in cases where terms such as “all” are used, whereby the opposite is “none” and yet the complement is “not all” or “some do not”.

Using the term “case” instead of “elementary event” prevents the enrichment of the specific vocabulary, which can lead to poor differentiation between event and elementary event. Thus, forming a sample space with all the “cases” can lead students to consider events that are not elementary, which may later give rise to errors in the application of Laplace’s rule.

In the case of using “possible” as a synonym for “probable”, expressions such as “the probability of an event is the degree of possibility of its occurrence” can be found. Something is possible when it can happen, without gradation; possible means that it can happen, that is, that its probability of occurrence is different from 0. Impossible and possible are concepts that refer only to whether the probability is null or not, but in no case do they refer to the measure of the uncertainty about real occurrence in the experimentation.

Finally, in a flagrant misuse of language, the expression “randomly” is used in statements such as “a card is randomly drawn”. If the deck is ordered and you draw a card from the top half, even if it is “random”, not all cards have the same probability of being selected. A lottery within families, where a letter is chosen and with it the family whose surname begins with that letter is selected, in alphabetical order, this is a random lottery and the selection is “random” since we do not know the result in advance. However, this is clearly far from being an equiprobable raffle.

3.2.5. Proportional Allocation as a Guarantor of Sample Representativeness

The representativeness of a sample is no trivial matter. In order to study the issue in question from a statistical perspective, there must be no doubt about the randomness of the sample. This characteristic is influenced by multiple factors, including sample size, the variability in the population and the maximum admissible error. Trivialising this issue facilitates the erroneous interpretation that any sample of any size is equally representative, an error that may lead the practitioner to accept the law of small numbers as valid [51].

Beyond the question of sample size, issues such as the consideration of population strata underlie statements such as the following: “If eight boys and one girl are interviewed about their musical preferences, can we say that the responses are representative of the entire city in which they live?” In later explanations, the book merely states that the fact making the results representative is that the determination of the strata marked (boys and girls) is not proportional.

In other texts, this conclusion is more explicit, “If there are groups in the population, the sample should contain elements of all the groups, in the same proportion”. The question then arises, is this always so? The statement made does not consider the full context, and therefore it might not be appropriate for the population proportions to be respected in the sample, or for the strata to be more complex than those proposed.

Although it is not established as a sufficient condition, the fact that it is held to be necessary, and the emphasis laid on knowing and controlling the number and identity of the individuals who will constitute the sample, rather than knowing how they are selected and what valid information they can provide, ultimately reduce the sampling problem to a collection of biased indications which appear to be sufficient, and may or may not be so.

3.3. Other Errors

Other less common errors that also constitute a clear source of possible didactic errors were also detected as follows.

3.3.1. Linguistic Inaccuracies

This type of error arises from avoiding the use of the word “modality”, using “data” or “value” instead. Example: “To calculate the arithmetic mean we can multiply each datum by its absolute frequency, add these amounts and divide the result by the number of data”. Two different meanings are confounded in the same sentence: data as observation and data as modality of the variable.

3.3.2. Graphic Representations of Pictograms

Here, the resulting figures are not proportional to the frequency, because the allegorical figure has been enlarged or reduced in both dimensions instead of only expanding the height or stacking the figure.

3.3.3. Use of Nonexclusive Modalities

Nonexclusive modalities are used in statements such as “A study was carried out to find out what kind of pets the students had. The results were: 12 students had a dog, 11 had a cat, 4 had birds, and 5 had fish or turtles”. The implicit assumption that each student had a pet and, in each case, only one, is completely unrealistic, as is well known to students of the age addressed.

3.3.4. Use Nonstandard Notation for Events

Diagrams or figures are sometimes used to represent the elementary events of the sample space. For example, considering the experiment in which two dice are thrown, an event could be represented by an image of the two resulting faces: (,). Events must be clearly defined and are usually expressed with capital letters beginning with the first letters of the alphabet A, B, C, … Although elementary events can be represented more simply by using the observed numerical value—in this case, for example, expressed as (2,5)—compound events do not generally follow this form. The event in question, therefore, should be described with the usual notation or otherwise clarified precisely.

3.3.5. Taking Ordinary Qualitative Variables as Quantitative Variables

Finally, let us point out an error that, while not frequent, is important enough to be taken into account in this classification. This error consists in taking ordinal qualitative variables as quantitative ones, and using concepts of measurement and quantity for numerical values that really only work to specify ordered categories. Such is the case in the following: “The degree of cleanliness observed in the centre is studied (1—very dirty, 2—dirty, 3—clean, 4—very clean). With the data collected, calculate the mean, the median and the mode”.

4. Discussion

Our review of mathematics textbooks for children in first and second-year secondary education identified errors in the exposition and use of tools and basic concepts of statistics and probability. We then classified these errors according to the nature of the concept. In a slightly different approach, we now interpret these errors in terms of statistical thinking and its characteristics.

4.1. Classification of Errors

4.1.1. Errors Due to Differences between Statistical and Mathematical Thinking

The differences between mathematical and statistical thinking are a major source of error. Thus, consideration of the context is one of the main dimensions of statistical thinking [47]. Similarly, Scheaffer [12] emphasised that in statistics the context is crucial, adding that another important difference between the two approaches is that measurements play a fundamental role in statistics. Finally, this author observed that the way in which data are obtained is not relevant from a mathematical standpoint but is fundamental from a statistical perspective.

We also include in this category the errors and/or confusion that may arise between quantitative and qualitative data, such as when ordinal qualitative variables are used as quantitative ones, when continuous data are measured as if they were discrete and vice versa, when experiments described as random experiments are not in fact random experiments.

4.1.2. Errors Due to Judgments Based on Heuristic Rules

In statistical and probabilistic reasoning, individuals are apt to rely on a limited number of heuristic principles. This reliance limits their reasoning to the use of simpler procedures that, may give rise to errors in resolving problems related to statistics and, especially, to probability. Such heuristics include insensitivity to sample size, to prior probabilistic results and/or predictability and the illusion of validity [17]. Among the errors detected in these textbooks, a priori probabilities are sometimes ignored and proportionality has been used as a guarantor of sample representativeness.

4.1.3. Errors Associated with the Complexity of Concepts

Let us now consider a third group of errors that are not specific to statistics and probability but are directly related to the complexity of the meanings of the concepts.

• Structural errors, such as confusion between measure, estimator, and parameter; considerations of nonexclusive modalities; the use of inappropriate vocabulary; and the use of nonstandard notation for events.
• Errors related to representation, such as attaching the label “bar chart” to a chart lacking frequencies; the uncritical choice of graphic representations; or errors in pictogram representations. Figurative representations express certain aspects and properties of concepts and enable their use for certain functions. If careful attention is not paid to the way in which this information is represented, elements that may be needed for proper understanding of the content may be ignored.

4.2. Research Implications

First, we must point out that this is the first study, as far as we know, which addresses errors on statistics and probability in textbooks. In addition, we have not only detected systematic errors but have also classified them into difficulties, which can explain the errors in learning, cognitive as well as conceptual terms.

This valuable finding is the starting point to be able to eliminate these errors—since they hinder the statistical literacy of citizens—and to propose renewed lessons that are correct and in line with the basic pillars of statistical thinking. Correcting the errors is essential to properly redirect the statistics and probability teaching–learning process. On one hand, textbooks play a crucial role in compulsory education as they contribute to the standardisation of the disciplines and they influence on teaching practices more than legislative regulations do [34,35]. On the other hand, students deserve rigorous textbooks, including formal concepts, precise representations, and suitable contexts and applications [43]. Consequently, remove errors and inaccuracies in textbooks is necessary, and it will provide new learning opportunities.

Beyond correcting the books, this finding raises a question about how the professionals who wrote the texts—mostly mathematicians—understand the discipline and how their point of view may skew the one of the teachers. If teachers have not questioned the contents, they will assume the errors that appear in the textbooks. This means that the teaching sequence may be contaminated; books condition teachers and they spread errors among students. These errors may have been inherited in different generations and it is possible that teachers acquired them in their student stage.

We must reflect and research about the causes and effects of making errors in textbooks. We can infer neither that errors in textbooks are the cause of the lack of statistical literacy nor the reciprocal, that is, the lack of statistical literacy is the reason why authors make these errors. The most plausible is that both events feed back to each other, so that learning errors during the school stage cause teachers to reproduce them when they have to teach statistics in the classroom.

Literature suggests some rational ideas to explain the difficulties to achieving statistical literacy. First, there exists differences in the preparation needed to teach statistics and the one required to teach mathematics [16]. Second, in statistics is usual the existence of false intuitions or heuristic rules [19]. Finally, there exists a conflict between mathematical and statistical thinking [14,15]. Nevertheless, the issue is still open and our results can help experts to deep in the causes.

Open questions arise from our result, such as whether teachers also make the same mistakes genuinely and independently of the textbooks, even with a training without errors. If so, researchers should investigate teachers’ misconceptions and provide strategies to remove errors regarding statistics and probability from classrooms.

It seems clear that textbooks needs supervision of statistics topics by experts in the field, who can contribute the elements of statistical thought that are different from the points of mathematical thought.

5. Conclusions

Statistics is a complex discipline. Its teaching involves difficulties not usually encountered in other areas of mathematics, for various reasons. First, there is the nature of the discipline itself, which differs profoundly from mathematics but falls within its sphere. Second, statistical reasoning is subject to numerous paradoxes and misintuitions. Finally, many teachers lack the necessary specialised knowledge of this field.

Against this backdrop, and despite the fact that statistical education has formed part of compulsory education for over thirty years, mathematics textbooks for first and second year students in compulsory secondary education in Spain systematically include errors, reflecting the three difficulties expressed before.

Education is intended to develop statistical literacy, but this process is hampered by the errors detected. They need to be identified and eliminated so that students and teachers can have reliable material, enabling them to effectively study and understand the basic concepts of statistics and probability.

Although it is not always possible to provide comprehensive, rigorous definitions, due to the need to take into account the cognitive development of the students addressed, this didactic transposition must not be at the cost of accuracy in the statements made in examples and exercises. The textbook must be absolutely trustworthy, and cannot be allowed to present contradictory or erroneous material.

Knowing the errors, why they occur, and in what form they appear is essential to improve the desired statistical literacy in students.

We consider as future work the study of textbooks on statistical topics in other grades and in other countries. Finally, we propose the question: are teacher conceptions related to these errors?