Classification and Analysis of Pre-Service Teachers’ Errors in Solving Fermi Problems

Abstract

Fermi problems are useful for introducing modelling in primary school classrooms, although teachers’ difficulties in problem solving may hinder their successful implementation. These difficulties are associated with the modelling process, but also with the estimation and measurement skills required by Fermi problems. In this work, a specific categorization of errors for Fermi problems was established, and it allowed us to analyse the errors of N = 224 pre-service primary school teachers. The results showed that prospective teachers make a large number of errors when solving this type of task, especially conceptual ones, which are associated with the process of simplifying/structuring the real situation and the mathematization process. They also showed that there is a significant relationship between the characteristics of the problem context and the error categories. Knowing the types of errors that prospective teachers make and designing task sequences that make them emerge so that prospective teachers learn from them could be an effective way to improve initial teacher education in modelling and estimation problem solving.

1. Introduction

Modelling tasks pose a problem related to a real-world situation that requires formulating, interpreting, and solving a mathematical model, the answer to which must be validated both mathematically and in its own context [1]. In recent years, there has been increasing interest in educational proposals that incorporate mathematical modelling at different educational levels [2]. Fermi problems are accessible modelling tasks that allow students to connect their mathematical knowledge with real-world phenomena [3,4]. In a Fermi problem, by its realistic nature, the statement presents a situation where little concrete information is known, requiring students to make assumptions and estimations in order to obtain a solution to the initial question [5]. In this work, we used a specific type of Fermi problem: those that require estimating a large number of elements enclosed in a delimited area, such as knowing the number of people that fit in a public square.

It is important to know how future teachers solve tasks that allow modelling to be introduced in primary school in order to improve their initial training and promote their effective practice in the classroom [6,7]. The introduction of mathematical modelling activities in primary school classrooms is a challenge for teachers.

Hagena [8] argued that measurement sense and estimation are necessary to successfully solve many modelling tasks. By the nature of the situation they pose, a sense of measurement and estimation is not only necessary, but central to solving Fermi problems [9]. Different studies [10,11,12] have found that prospective teachers have difficulties and make many errors when solving modelling problems. On the other hand, there are studies on the deficiencies in the estimation skills of prospective teachers [13,14], especially when they must reason about estimation and measurement in complex situations [15].

In this work, we studied errors in the productions of N = 224 pre-service primary school teachers when they were confronted with a sequence of Fermi problems. The Fermi problems they faced required them to be competent in the modelling process [16], but also to have acquired skills in measurement and estimation. Thus, the first objective of this paper was, based on a review of the background, to develop a specific error categorisation for Fermi problems, which considers both the phases of the modelling process and the concepts and procedures involved in estimation and measurement of surfaces and lengths. The second objective of this work was to analyse the prospective teachers’ resolutions using the above categorisation system. The third objective was to analyse whether there is a relationship between the characteristics of the context of the Fermi problem and the types of errors made. This information is useful for improving initial teacher training programmes in mathematics and for effective teaching of modelling and problem solving. In fact, it allows us to design sequences of modelling tasks based on the errors made and control the problem-context characteristics to emphasise one type of difficulty or another.

2. Theoretical Framework

2.1. Mathematical Modelling

Although there are different didactic approaches to defining what a modelling task is, there is a consensus that they are problems that involve transitions back and forth between reality and mathematics [17,18]. It involves mathematising real-world situations and elaborating mathematical models to describe the phenomena studied, often conceptualised as the result of having engaged in a complex modelling process [1]. Lesh and Harel [19] defined a mathematical model as a system consisting of mathematical concepts, symbolic representations of reality, relationships, and regularities or patterns, as well as the procedures, mathematical or otherwise, associated with their use. From this definition, we understand that to create and develop mathematical models intended to abstractly describe or represent a certain phenomenon or reality is a complex task. This process is made up of phases, and there is a consensus that these phases form a cycle [17,20,21]. During the modelling process, solvers must go through different stages in which they move from reality to the mathematical domain, each time re-evaluating the phenomenon under study. The modelling cycle, from a cognitive perspective, puts the focus on the solver’s cognitive processes during the modelling process [6], which make up its different phases. Thus, it starts from a real and open situation that has to be simplified and structured, selecting the relevant elements of reality to solve the problem and making assumptions if necessary. This process leads to the real model, which is a representation of the problem that prefigures the mathematical model [6]. In order to build the mathematical model, the real model must be mathematised, that is, it must be translated into mathematical language and its representations [22]. The next phase requires working mathematically within the model to find a solution to the problem in mathematical terms. Finally, the mathematical result must be interpreted within the real model and validated in the real situation.

2.2. Fermi Problems as Modelling Tasks

In everyday life, we are confronted with many situations that pose questions where an estimate is the best answer, either because one does not have the means to answer accurately or because not all the necessary information is available. Estimation tasks can be used as a means of initiation into mathematical modelling [6]. A specific case of estimation tasks from real contexts where explicit information is missing are Fermi problems: the lack of data requires a process of simplification and mathematisation of reality [5]. Following Ärlebäck [3], Fermi problems are:

Open, non-standard problems requiring the students to make assumptions about the problem situation and estimate relevant quantities before engaging in, often, simple calculations.

(p. 331)

According to Sriraman and Knott [23], Fermi problems are estimation problems that aim at encouraging students to make educated guesses. There is a clear link between Fermi’s problem-solving process and the work developed during the modelling cycle for the construction of a mathematical model [3,4]. Indeed, the detailed analysis of the real situation allows the problem to be decomposed into simpler ones in order to arrive at the solution from reasoned conjectures. The process of identifying the essential variables of the problem and their relationships involves the synthesis of a model [24].

In this study, we used a subset of Fermi problems, those that consist of estimating a large number of elements in a delimited surface, i.e., estimating how many people can fit in Times Square. Based on Lesh and Harel’s definition of a mathematical model and Albarracín, Ferrando, and Gorgorió [25] categorisation types of solving strategies, in previous works we developed the notion of the resolution plan for the analysis of the productions of pre-service teachers when they are asked to elaborate a solution scheme for this type of Fermi problem [26]. The resolution plan is formed by two components (that are dependent on each other): an initial model that corresponds to the real model, and a solution strategy that corresponds to the mathematical work within the model. The initial model refers to the essential simplifications and assumptions that the solver must make about the configuration and arrangement of the elements whose number is to be estimated, i.e., how the solver distributes the elements on the surface. Actually, when obtaining a reasoned estimate of the number of objects that fit in a bounded enclosure, the first step is to fix their arrangement in space. One way to accomplish this is to assume that the elements are arranged in rows and columns; this leads us to reduce the initial problem (of areas) to a problem of lengths, and in this case we say that the initial model is one-dimensional. Otherwise, the elements can be arranged directly on the surface and this necessarily implies using procedures linked to the measurement of areas, which is why we say that we are dealing with two-dimensional initial models. The strategy is the chain of procedures related to the concepts of measurement and proportionality that the solver applies to obtain the estimate of the result. In [26], the following resolution plans were categorised:

• Counting: these solution plans are not based on a model of the situation, but on a straightforward strategy that is unrealisable for a large number of elements.
• Linearization: productions with an initial one-dimensional model in which the elements are distributed by rows. For instance, in order to estimate the number of people that fit in a rectangular square, the solver assumes than the persons are organised in rows and columns. Then, the solution is obtained using the width and length of the porch and the estimation of the width and length of a person.
• Base unit: productions with an initial two-dimensional model, based on the procedure of dividing the total area by the area of an element taken as a unit.
• Density: productions with an initial two-dimensional model, based on the procedure of multiplying the total area by an estimated density.

Because of this variety of types of resolution plans, Fermi problems have been used in primary school pre-service teachers’ training as activities to promote flexibility in problem solving [27]. These studies show that using Fermi problems is useful for understanding students’ problem-solving proficiency and, in particular, for promoting mathematical modelling competence.

2.3. Difficulties and Errors in Modelling Tasks

Some researchers suggest that teachers should experience problem solving from the problem solver’s perspective before they can adequately approach teaching it [28]. Primary school teachers need to be competent as solvers of modelling tasks. In particular, teachers need to be trained to guide the mathematical process from a variety of real-world situations in order to improve teaching in modelling problem solving. However, many difficulties have been identified in implementing such tasks in classrooms [29]. Indeed, several studies indicate that prospective teachers do not acquire problem-solving skills during their initial training. For example, Widjaja [10] found that prospective teachers have difficulties in identifying the variables involved in their mathematical models and show difficulties in comparing and validating solutions.

The assessment of the mathematical errors made by prospective teachers can be helpful to obtain a diagnosis of the way in which they use mathematical thinking when solving a modelling task and to know their competence as a solver of modelling tasks [12,30,31,32]. Kaiser, Schwarz, and Tiedemann [33] identified difficulties in understanding the real situations to be modelled and also in determining the relationship between the real situation and the mathematical knowledge required for its solution. Crouch and Haines [11] identified four main types of errors and difficulties in the resolution of a modelling task:

• inconsistency between the real world and the mathematical model;
• applying an inappropriate model in a context;
• incorrect or incomplete development of mathematical concepts or procedures;
• failure to validate the resolution in the real world.

Klock and Siller [34] developed a comprehensive categorisation that provides an overview of the difficulties at each stage of the modelling cycle. Moreno, Marín, and Ramírez-Uclés [12] (p. 121), based on the work of Crouch and Haines, established the following system of error categories during the modelling process:

• Simplification error: incomplete real model associated with failure to consider elements of reality; incomplete real model due to inconsistencies in the relationships between the elements of reality considered; failure to develop an objective function for the real model; failure to build a real model.
• Mathematization error: mathematical model inconsistent with the real one; incomplete mathematical model; no mathematical model is built.
• Resolution error: conceptual errors; procedural errors; incomplete resolution.
• Interpretation error: results are not interpreted; failure to identify or raise possible limitations of the model.

This category system is similar to the categorisation by Klock and Siller [34], but simplified. Moreno, Marín, and Ramírez-Uclés [12] applied their categorisation in the analysis of the resolutions of prospective Secondary Mathematics teachers when faced with a modelling task. They found that all of them made some errors during the modelling process. Most of the errors occurred during the simplification phase (translation of the real situation into a real model) and the validation phase. The conclusion is that future teachers are not very familiar with modelling processes, so specific initial training is needed to improve their modelling competence.

2.4. Errors in Estimation and Measurement of Areas and Lengths

As has been pointed out, in this work, we focused on one type of Fermi problem, those that consist of estimating a large number of elements in a delimited surface. They are appropriate for introducing modelling in primary education, and they are problems in which the sense of estimation and measurement plays a central role. When we talk about estimation we refer to metric estimation, which involves different skills: understanding the concept of unity, mental image of unity, and use of estimation strategies [35]. Estimation is a complex activity whose teaching has been limited and superficial [36]. However, measurement estimation is rich from a didactic point of view, as it allows the development of flexible strategies to obtain results, to interpret them, and to validate them in real contexts, strengthening the relationship between the real world and the mathematical world [37]. This allows overcoming didactic obstacles such as the abuse of algorithmic procedures in the change of units or the idealisation of simplified objects. Moreover, the use of non-conventional or informal units of measurement (steps, strings, etc.) helps to solve problems of scale and to internalise appropriate referents of magnitude [38]. If the teaching of measurement has not been further developed and its didactic potential has not been developed in depth in the classroom, it is because teachers have not felt competent in the subject and have not had the tools to develop and evaluate estimation activities in a real context [39].

In fact, there are studies on the deficiencies in the estimation skills of future teachers [13]. There is a lack of basic understanding of quantity management, which prevents students from being able to reason about estimation and measurement in complex situations [15]. Research by Castillo-Mateo et al. [14] provided categorisation of errors during the process of estimating quantities of length and area: E1. Miscalculation of operations; E2. Error in magnitude perception; E3. Error in the meaning of terms proper to the magnitude; E4. Absence of units of measurement; E5. Use of inappropriate units of measurement; E6. Error in conversion of units of measurement; E7. Inadequate internalisation of referents of the quantity to be estimated; E8. Inadequate internalisation of units of measurement of the I.S. of the quantity to be estimated; E9. Use of incorrect calculation procedures.

3. Methodology

This section is divided into two subsections. The first one describes and justifies the design of the experiment, and the second one establishes a system of error categories specific to Fermi problems, associated with the phases of the modelling cycle and to the concepts and procedures involved in the estimation and measurement work.

3.1. Description of the Experience

The starting point of the experience was the design and validation of a sequence of four problems requiring a reasoned estimate of a sufficiently large number of elements that cannot be obtained directly [27]. The four problems are contextualised in rectangular enclosures located in areas close to the Faculty of Education.

The task sequence was proposed to a total of N = 224 students in their fourth year of study in Primary School Education at the Universitat de València during the 2017/18 and 2018/19 academic years. The experience was developed during a 90-min session in the regular classroom, with the regular teacher supervised by the researcher. Each prospective teacher was provided with a booklet containing the problem statements and with enough space to write. Although the context of the problem was familiar to all prospective teachers, each problem statement was accompanied by a photograph of the situation. For each of the problems, prospective teachers were not required to provide a complete solution, but to outline a resolution plan indicating the data needed to solve the problem, as well as the mathematical procedures needed to arrive at the requested estimate, but it was not necessary to quantify or perform the calculations.

The statement and the relevant contextual characteristics for each problem, which are discussed further, based on Kilpatrick’s [40] task variables, are detailed in

Table 1. Task sequence and relevant contextual characteristics.

Statement	Contextual Characteristics
P1—People. How many students can stand on the faculty porch when it rains?	Element size: medium Element shape: irregular Element arrangement: disordered Area size: medium
P2—Tiles. How many tiles are there between the education faculty building and the gym?	Element size: medium Element shape: regular Element arrangement: ordered Area size: big
P3—Grass. How many blades of grass are there in this space?	Element size: small Element shape: irregular Element arrangement: disordered Area size: medium
P4—Cars. How many cars can fit in the faculty parking?	Element size: big Element shape: regular Element arrangement: ordered Area size: big

.

The productions of prospective teachers were categorised according to the theoretical framework: counting, linearisation, base unit, and density. The results of previous work show that there is a statistically significant relationship between the context characteristics of the problems and the resolution plans proposed by the students [41]. Some context features (see Table 1) influenced the solving plans of future teachers: problems that present a situation with an ordered arrangement of elements (such as P2—Tiles) promoted linear initial models, while disordered arrangement (such as P1—People or P3—Grass) gave rise to two-dimensional initial models. In addition, a large element size (such as P4—Cars) increased the number of solving plans based on reasoning from the area of the elements, while a small element size (such as P3—Grass) promoted the strategy based on element density.

3.2. Categorisation of Error Types in Fermi Problem-Solving

In this research, we focused on one type of modelling problem (problems of estimation of large quantities on delimited surfaces), with the aim of classifying and analysing the future teachers’ types of errors, considering both the errors inherent in the modelling process and the errors in the process of estimating measurements (in this type of problem, lengths, and areas) and quantities. An analysis of errors from the perspective of the problem-solving knowledge of future teachers contributes to finding out whether they are competent in modelling and whether they are good estimators.

The following categorisation (

Table 2. Error category system in Fermi problem-solving.

Category	Category Values
Simplification error	E1. Incomplete initial model associated with the lack of consideration of elements of the real situation. E2. Incorrect initial model due to error of perception of the magnitude. E3. Incorrect initial model due to inadequate internalisation of referents of the magnitude to be estimated. E4. Does not build an initial model.	Conceptual
Mathematization error	E5. Mathematical model incoherent with the initial model due to an error in the meaning of the terms of the magnitude. E6. Mathematical model incoherent with the initial model due to inadequate internalisation of units of measurement of the S.I. of the magnitude to be estimated. E7. Mathematical model incoherent with the initial model due to the use of unsuitable units of measurement. E8. Mathematical model is not constructed or is incomplete because elements of the initial model are not quantified.	Conceptual
Mathematical working error	E9. Use of incorrect calculation procedures or calculation errors. E10. Error in conversion of measurement units. E11. Incomplete resolution.	Procedural
Interpretation error	E12. Absence of measurement units in the results. E13. The estimate is clearly incompatible with the real situation.	Conceptual

) is therefore a synthesis of the category system of Moreno, Marín, and Ramírez-Uclés [12], based on the modelling cycle, and the classification of errors in estimation and measurement of lengths and areas by Castillo-Mateo et al. [14]. This categorisation of errors, specific to Fermi problems, was elaborated a priori, based on the previous analysis of the solving plans of future teachers and on the background review. It takes into account the essential processes in solving a modelling task: simplifying to obtain the initial/real model, mathematising to build the mathematical model, solving by applying a strategy to find a solution/estimate, and interpreting the result. In addition, it takes into account the essential processes of estimation and sense of measurement: concept of unit of measurement, mental referent of the unit of measurement, and the use of estimation procedures.

The analysis of the N = 224 pre-service teachers’ productions on the basis of this error categorisation combined a quantitative analysis and a qualitative analysis of the types of errors in the resolution plans. Following the qualitative analysis methodology of Moreno, Marín, and Ramírez-Uclés [12] for modelling tasks, this analysis was also not performed in comparison with a given solution, but rather analysed the coherence of the resolution plans (initial model and associated strategy) presented. The categorisation was carried out by the authors of this paper, with meetings with another researcher to ensure consistency. It was carried out on the basis of the aforementioned works and an initial exploratory analysis of the N = 224 pre-service teachers’ productions, in which they detected errors associated with different phases of the modelling process and related to estimation and measurement, of a conceptual and procedural nature. Based on the category system (Table 2), the authors and another researcher analysed the productions again, peer-reviewing and establishing consensus in the classification of the types of error. The following section discusses the results.

4. Results and Discussion

4.1. Results and Discussion of the Descriptive and Qualitative Analysis of Error Types

Table 3. Frequency of each type of error in N = 224 pre-service teacher resolution plans.

Error Type	Frequency
E1	27 (5, 86%)
E2	100 (21, 69%)
E3	4 (0, 87%)
E4	41 (8, 89%)
Simplification error	172 (37, 31%)
E5	66 (14, 32%)
E6	7 (1, 52%)
E7	24 (5, 21%)
E8	87 (18, 87%)
Mathematization error	184 (39, 91%)
E9	44 (9, 54%)
E10	1 (0, 22%)
E11	49 (10, 63%)
Mathematical working error	94 (20, 39%)
E12	2 (0, 43%)
E13	9 (1, 95%)
Interpretation error	11 (2, 39%)
Total	461

shows the overall results of the analysis of the N = 224 pre-service teachers’ resolution plans based on the system of error categories specific to Fermi problems. It should be stressed that, when analysing each of the productions, all the errors that appeared in each of them were counted.

It was observed that prospective teachers made a large number of errors (461) in solving the proposed sequence of four Fermi problems. In fact, they made an average of 2.06 errors per prospective teacher. Of the N = 224 pre-service teachers who participated in the experiment, 166 made at least one error, which represents 74.11% of the sample with an average of 2.78 errors per solver.

The majority of errors were concentrated in the categories of simplification error (37.31%) and mathematization error (39.91%). This indicates that there are two phases that generated the greatest difficulties for prospective teachers. One is the phase in which the real situation is understood and an initial (real) model of the space and the distribution of the elements to be estimated is established (through simplification and structuring). The other was the phase in which this initial model is quantified and mathematised (through the geometrisation of the space and the elements, and the estimation of the measurement of areas and/or lengths). These are errors of a conceptual nature.

Prospective teachers also made errors of a procedural nature related to the phase of mathematical work and resolution of the model (20.39%), although this is a significantly lower number.

As for the category interpretation error, its frequency was very low (2.39%). This is not because this phase generates fewer difficulties in future teachers, as shown in the work of [12], but because of the schematic nature of the resolution plan, which did not require them to make numerical estimates of the problem, and therefore arrive at a numerical result to be interpreted, but to describe step by step how the solution would be reached.

Having discussed the overall results of the descriptive analysis, we present the results of the qualitative analysis for each type of error, which allows us to explain their characteristics.

4.1.1. Simplification Errors

Error 1 (E1).

Incomplete initial model associated with the lack of consideration of elements of the real situation.

It accounts for 5.86% of the errors made by prospective teachers. In the structuring phase of the real situation, the solver does not identify the model variables or the relevant aspects of the real situation that should be incorporated into the model. In the case of sequence problems, most errors of this type occur because the solver does not consider that the space occupied by the elements is a variable in the model to obtain the estimate of their number. For example, in the following transcript of a resolution plan for problem P1—People, the space occupied by each person is not taken into account, nor are people distributed over the porch area:

“We need to know the total size of the porch first. To do this, we would have to estimate the total from the length and width.” (Here ends the solver’s resolution plan).

This type of error makes it impossible to develop a strategy to obtain an estimate of the number of elements, and the resolution plans remain incomplete.

Error 2 (E2).

Incorrect initial model due to error of perception of the magnitude.

The magnitudes involved in the initial model of the real situation are confused, confusing length and area, or area and volume. This is the most numerous error in the process of solving Fermi problems, as it appeared in 100 resolution plans, which represents 21.69% of the total number of errors. This is an indicator of profound deficiencies in the magnitude estimation sense, as many prospective teachers did not seem to differentiate well between the magnitudes involved in a Fermi problem or to choose the correct one. For example, in P2—Tiles, it was a very frequent error to confuse the surface area between the gymnasium and the Faculty of Education (which is what fits the problem situation) with the distance between the gymnasium and the Faculty of Education, as can be seen in the following transcript of a resolution plan:

“I would calculate how many tiles there are in x metres, or if I wasn’t exact, I would count x tiles and measure how long that group is [in length]. Example: 20 tiles = 1 m. Then I would measure the metres between the two buildings and multiply it by the tiles in one metre”.

In this case, the resolution plan is complete: the strategy is correct, but in the initial model the space has been structured on the magnitude length.

This type of error also appeared in other problems, affecting the magnitude chosen to express the space occupied by the element to be estimated. For example, in P3—Grass, some future teachers confused the area of the blade with the width of the blade, or in P1—People, we found resolution plans in which the area occupied by a person was confused with the height occupied by a person. For example:

“You would have to measure the entrance porch [it does not indicate how to do this or what magnitude measurement you want to obtain] and measure the average height of a person, and divide the measurement of the porch and the person.”

In this case, the resolution plan is incorrect, and there are serious deficiencies in the magnitude sense.

Error 3 (E3).

Incorrect initial model due to inadequate internalisation of referents of the magnitude to be estimated.

Occurs when the solver has not internalised an adequate measure of some referent of the magnitude to be estimated, for example, he/she thinks that a large tile on the porch floor can fit one person (when it can fit three or four). This represents a small number of errors (0.87% of the total), perhaps due to the schematic nature of the resolution plan (most solvers do not venture numerical estimates of the magnitude referent, but only propose its use in a qualitative way), but it is also influenced by the fact that the sequence of Fermi problems is set in a familiar environment for the students. For example, in P2—Tiles, we found the following measurement error of the tile referent in relation to the foot referent, because clearly one foot does not cover two tiles, and the solver himself expressed his doubts once proposed:

“(...) as the tile is smaller than a foot, i.e., it is not that long, what I would do is divide the number of steps I have counted by two, and that would give me the number of tiles. I think that would be wrong, so I would have to measure a tile, both its width and its length.”

Error 4 (E4).

Does not build an initial model.

This type of error corresponds to blank answers and those showing that the situation posed by the problem has not been understood. It was numerous (8.89% of the total) because there were many blank answers (26) to the problem P3—Grass.

4.1.2. Mathematization Errors

Error 5 (E5).

Mathematical model incoherent with the initial one due to an error in the meaning of the terms of the magnitude.

E5 is the third most frequent error (14.32% of the total). During the process of mathematising the variables involved in the initial model, this error occurs when a term related to the magnitude being estimated is used inappropriately. This inappropriate use may be related to mixing different magnitudes in a procedure without respecting dimensional homogeneity. For example, there were some resolution plans that proposed that, in order to obtain the number of elements, the measurements of the magnitudes of different dimensions (for instance, area of the total space divided by the width of the element to be estimated) should be divided:

“Data [measurements he/she needs to obtain/estimate] →how much a car occupies and the length of the car park. [Process:] I would take the measurement of the car park [the length] and divide it by the area of the car.”

In the above transcript we note that, in addition to an error of perception of magnitude (E2) by considering that the space of the problem corresponds to a linear magnitude, an error was made in terms of magnitude by pretending to divide the length of the space by the area of the car.

Another example of E5 is the confusion of the concept of area and the concept of perimeter. The frequent confusion between area and perimeter is well known, as their calculation is often accompanied by formulas that stereotype the understanding and relationship of the spatial foundations of these two concepts [42]. We found several resolution plans that made this mistake, for example, (Figure 1):

“[Transcript of Figure 1] In order to know how many people fit under the porch, I think we should solve the perimeter of the area, so that, through this resolution, we could get the meters (sic) that the porch occupies (...).”

Errors associated with the linearisation resolution planes appeared very frequently. Most linearisation-based resolutions were found in P2—Tiles, due to the ordered arrangement (rows and columns) of the small tiles. We could call this error “incomplete linearisation”, and it consists of estimating the number of elements in length and width, but the solver forgets that the surface is a two-dimensional magnitude and the Cartesian product is not done. This is clear in the following transcript of a resolution plan:

“Calculate [measure or estimate] the metres of the basis [of the rectangle that forms the total space between the gymnasium and the Faculty], from one side of the gymnasium to the other, find out [count] how many tiles there are in one metre, and multiply it [to get the number of tiles at the base]. Do the same with the height [of the rectangle].”

In the previous resolution plan, we observed an initial model based on the linearisation of the distribution of the elements in space (it considers the tiles by rows), and a correct strategy to estimate the number of tiles in the width and length of the rectangular surface. However, the magnitude “number of tiles” is incomplete, as it is the result of the Cartesian product of the number of tiles in width by the number of tiles in length.

Error 6 (E6).

Mathematical model incoherent with the initial one due to inadequate internalisation of units of measurement of the S.I. of the magnitude to be estimated.

E6 is caused by taking as a conventional unit of measurement a quantity that does not correspond to that unit of measurement, e.g., when one step is considered to be equivalent to one metre. Another example can be found in Figure 2:

“[Transcript of Figure 2] (...) we need to know what the area of each blade is (...) Blade →Width = 1 m, Length = 3 m, [Area] = 1 × 3 = 3m².”

It can be seen from the above example that the conventional quantities for the length of a blade of grass were not well internalised by the solver. E6 is not a frequent error, although the schematic nature of the resolution plan may explain that it represented only 1.52% of the total errors. It is to be expected to increase in resolutions requiring numerical estimates.

Error 7 (E7).

Mathematical model incoherent with the initial one due to the use of unsuitable units of measurement.

It appeared in 24 resolution plans (representing 5.21% of the total errors), when units of measurement of another magnitude are used, e.g., metres instead of square metres. This type of error can be seen (in several cases, metre was used as a unit of measurement of area) in the following transcript of a resolution plan for P1—People:

“First of all, I would measure the width and length of the porch floor to calculate its area and thus know how many metres we have available. Then I would establish a measure [of area] per person, for example, one person occupies one metre, in order to calculate how many people can fit on the porch in an approximate way, dividing the total metres of the floor by the measure [occupied area] of the people.”

Error 8 (E8).

Mathematical model is not constructed or is incomplete because elements of the initial model are not quantified.

E8 was the second most numerous type of error, representing 18.87% of the total. This error is made when not enough variables or aspects of the initial model of the real situation are mathematised/quantified to provide a strategy to obtain an estimate of the number of elements enclosed in the delimited surface of the problem. We categorised resolution plans based on direct counting as E8, as their solution is not based on a mathematical model that allows a reasoned estimate. For example, in P4—Cars:

“Count the number of cars that fit in the car park.”

On the other hand, there are numerous examples of resolution plans that did not fully develop the mathematical model. In the following transcription of a resolution plan for P4—Cars, it can be seen that measurements were mentioned, but the magnitude was not established, nor were procedures noted for obtaining the estimate of the number of cars from the variables considered (measurement of the car park and measurement of the car):

“We would need to know the measurements of the car park and the size of one car in order to know how many cars could fit.”

The same is applicable in the following resolution plan for P2—Tiles:

“- To know the distance between the gymnasium and the building (both width and length).

- [To know] Measurements of the tile.”

It is not known which magnitudes and procedures will be used in the mathematical model—will it be based on linearisation, or will it calculate areas? There is a lack of definition of relevant variables for the resolution of the problem and the dependencies between these variables, which indicates difficulties in understanding the mathematical concepts involved in the initial model of the situation.

4.1.3. Mathematical Working Errors

Error 9 (E9).

Use of incorrect calculation procedures or calculation errors.

These errors are made when the solver uses an inappropriate formula, or an incorrect calculation procedure, e.g., that the area of a rectangle is base plus height. It was a numerous error in the resolution of the Fermi problem sequence: it accounted for 9.54% of the total.

In this typology, the most recurrent error is the use of the inverse algorithm [43], which consists in that for a situation with a measurement division structure, a multiplication is used. In the case of the base unit resolution plan, the procedure $surface area÷ element area=number of elements$ is inverted to $surface area \times element area=number of elements$. We see an example in the following transcript of a base unit resolution plan for P1—People:

“First we must measure the width and length of the porch to find out the area of the space. Next, we would average [the area] that a person occupies, and multiply the two data.”

In the case of density-based resolution plans, the error of using the inverse algorithm consists of changing the calculation procedure total surface area × number of elements in a sample surface $÷$ sample surface area by total surface area × number of elements in a sample surface × sample surface area.

Reversal errors [44] also appeared frequently, although less frequently in measurement division, which involves reversing the order of the variables. In the case of base-unit resolution plans, a reversal error involves changing surface area $÷$ element area = number of elements by element area $÷$ surface area = number of elements. For example, in the following transcript of a resolution plan for P4—Cars:

“First of all I would need to know [the area] the total space of the car park and how much [area] a [car] space measures. So we could divide [the area of] the space by [the area of] the whole car park, and then we could know how many cars would fit without leaving any space.”

There may be an explanation for this phenomenon, i.e., the coincidence in word order [45], which would be due to a literal conversion of the statement’s word order to mathematical procedures without a clear mental representation of the situation. Indeed, in the statement of P4—Cars, the word car appears before the word parking. Reversal error also occurs in density-based resolution plans, where the calculation procedure is surface area $\times$number of elements in a sample surface $÷$ sample surface area is reversed by the procedure sample surface area $\times$ number of elements in a sample surface $÷$ surface area (see Figure 3).

“[Transcript of Figure 3] (...) Divide the number of blades in [a sample area of] 1 cm² by the total area.”

Reversal errors also appeared in the incorrect use of the rule of three, when the order of the three known quantities is altered. For example, in the following transcript of a resolution plan for P3—Grass, we found the following reasoning based on a rule of three:

“(...) since we assume that a blade of grass is 25 cm [instead of cm²]. We would then apply a rule of three, in which if we assume that the total area [of the surface] is 1500 cm [cm²]:

1500 cm →25 cm

1 blade →x

This gives the total number of blades.”

The application of this rule results in dividing blade area by total area. Such mechanical problem-solving procedures are an obstacle to understanding the concepts involved [46]. In this case, this was the proportionality of area ratios.

Error 10 (E10).

Error in conversion of measurement units.

E10 occurs when one unit of measurement is not converted correctly from one unit to another, e.g., from metres to centimetres. It only appeared in one resolution plan, for P3—Grass:

“(...) we count the blades in that space and multiply, knowing that one square metre is 100 cm² or 10 dm².”

The number of errors was almost non-existent because very few prospective teachers work with numerical data when they outline the resolution process. It is to be expected that in resolutions in which they have to estimate or measure numerical data of the problem situation, this type of error could increase.

Error 11 (E11).

Incomplete resolution.

E11 is made when the mathematical procedures on which the solving strategy is based are not sufficiently completed. It is due to the fact that the solver implicitly assumes these procedures but does not write them down, although they were instructed that all steps should be made explicit. There was a high number of this type of error (10.63% of the total). In all of them, the solver had not explicitly written that he/she must divide the total area by the area occupied by an element to find the number of elements. For example, in the following transcript:

“First, you would need to find the area of the porch, assuming it is rectangular, by multiplying the width and length, and the space occupied by an average person. After that, I would calculate the number of people that fit in that area.”

This type of error is a procedural oversight, as the written resolution plan shows that the base unit strategy is being used. This is different from E8, because an incomplete mathematical model does not allow interpretation of how the estimate of the number of elements would be arrived at.

4.1.4. Interpretation Errors

Error 12 (E12).

Absence of measurement units in the results.

E12 is committed when the solver estimates a numerical value or the result of a measurement without disclosing the unit of measurement to which it refers. It only appeared in two resolution plans because of the schematic nature of the plans. We found an example in this transcript of a resolution plan for P3—Grass:

“If a blade is 2 × 3 [it does not specify units] we would multiply and then take the total area and divide [and it does not say what estimate is obtained or in what units].”

Error 13 (E13).

The estimate is clearly incompatible with the real situation.

E13 is made when the mathematical result is not interpreted in the real situation. It occurs when a clearly implausible estimate is obtained, which is directly perceived as too high or too low. It also occurs when the numerical nature of the result is incompatible with reality, for example, when the solver estimates the number of people or cars with a decimal number. The schematic nature of the resolution plan also influences the low number of this type of error (1.95% of the total). In Figure 2, we see that the estimated number of blades of grass is 10 (“R: 30: 3 = 10 blades”), which is clearly incompatible with the real situation, since in a space like P3—Grass, it is obvious at a glance that there are many more than 10 blades of grass.

4.2. Results and Discussion of the Quantitative Analysis of the Relationship between Error Categories and Problem Characteristics

Previous studies, as indicated above, found a statistically significant relationship between some characteristics of the Fermi problems used in the sequence and the types of resolution plan most chosen by pre-service teachers [41]. Kilpatrick [40] called them task variables and distinguished three categories: format, structure, and context. The characteristics that we took into account for the design of the Fermi problem sequence are contextual. We took as context variables, and for each problem we determined values, as can be seen in Table 1. The context variables are: the element size (big, medium, small); the size of the rectangular surface (big, medium, small); the order or disorder in the elements’ arrangement on the surface; and the regularity or irregularity of the elements’ shape. These characteristics of the context are relevant because they have to be taken into account to develop the initial model (how are the elements to be estimated placed in the problem space) and the associated strategy (what sizes and shapes do the elements have and how does this influence the method of estimating their quantity).

In order to extend the study of the context influence on this type of Fermi problem, it would be interesting to analyse whether there is a relationship between the context variables and the number and type of errors made by prospective teachers.

Table 4. Frequency of each type of error for each Fermi problem in the sequence.

Error Type	P1—People	P2—Tiles	P3—Grass	P4—Cars
E1	9	4	10	4
E2	33	46	17	4
E3	1	2	0	1
E4	4	4	26	7
E5	20	17	14	15
E6	3	2	2	0
E7	5	7	8	4
E8	27	21	22	17
E9	8	11	13	12
E10	0	0	1	0
E11	18	3	13	15
E12	0	0	2	0
E13	3	3	2	1
Total	131	120	130	80

shows the frequency of each type of error for each task in the P1-P2-P3-P4 sequence of Fermi problems.

In order to carry out a statistical analysis of the correlation between the variables in the context of each problem and the errors made, we grouped the types of errors by category according to the phase of the modelling cycle, as shown in

Table 5. Frequency of each error category for each Fermi problem in the sequence.

Error Category	P1—People	P2—Tiles	P3—Grass	P4—Cars
Simplification error	47	56	53	16
Mathematisation error	55	47	46	36
Mathematical working error	26	14	27	27
Interpretation error	3	3	4	1
Total	131	120	130	80

.

We performed an inferential analysis based on the Chi-Square Test for independence (DF = 9, N = 461). We assumed as null hypothesis that there is no relationship between the context variables of the problems and the error categories. We fixed α = 0.01 and the test yielded a result for X2 = 23.2873 and p-value = 0.006 that led us to reject the null hypothesis. Since in four cells in Table 5 the frequency is less than five, which is 25% of the cells, and frequencies less than five should not exceed 20% of the total, the result may or may not be reliable. To confirm reliability, we must base our results on the Likelihood Ratio Chi-square test (LR), which admits cells with frequencies lower than five [47,48]. Thus, setting a significance level of α = 0.01, the Likelihood Ratio the Chi-Square test was LR = 24.117 with an asymptotic (bilateral) significance of 0.004. This confirms that there is a statistically significant relationship between the type of problem and the error category. In addition, we measured the strength of the correlation with Cramer’s V, obtaining V = 0.13 with a significance of 0.006. This result informs us that, although the correlation is significant, the effect size is small, but we should keep in mind that Cramer’s V tends to produce relatively low correlation measures, even for highly significant results. Furthermore, Pearson’s contingency coefficient had a value of 0.22, and the phi coefficient was 0.23. Both results confirm that the correlation between the contextual characteristics and the error category exists, although the association is weak.

The correlation between problem context and errors should be discussed and interpreted. We observe that P1—People and P3—Grass were those with a significantly higher number of errors. Considering the context variables that have been taken into account for their design, P1 and P3 differed from the rest in that they are problems with irregularly shaped elements and disordered distribution. In fact, we know that P3—Grass correlates with the density-based resolution plan [41], and that this resolution plan is usually more complex than the others for students [25], hence the presence of many unsolved problems (E4) in P3 (see Table 4). In fact, there were 26 compared to 4, 4, and 7 in the other problems. On the other hand, in P1—People the high number of errors may be related to the irregularity of persons’ size and shape. Thus, the number of errors E2, confusing area, and width of person, and of E5, related to dividing area of the porch by width of person, was very high, as can be seen in Table 4. It is also related to the fact that the porch is a more complex space to model, due to the presence of columns and revolving doors, as there are many incomplete mathematical models E8 (27 versus 21, 22, and 17, as can be seen in Table 4). This is consistent with another previous study [49], in which it was found that P1—People is the problem in the sequence in which prospective teachers incorporate more complex factors into their mathematical model. Many solvers incorporate obstacles in the surface, subtracting from the total area the area occupied by columns and revolving doors to obtain a useful area in which to estimate the number of people.

As for P2—Tiles, the number of errors was also high, with E2 errors standing out (as shown in Table 4, there are 46 versus 33, 17, and 4). This E2 error was due in all cases to confusion between distance and area. We know from previous work [41] that P2—Tiles correlates with the resolution plan linearisation, so the confusion between distance and area could be related to the use of this type of plan. In fact, the order in the distribution of the bricks (grid distribution), and their regularity, promotes a “linear look” that can generate difficulties with the perception of magnitude.

Finally, P4—Cars was clearly the easiest problem for pre-service teachers. Considering its context variables, it seems that posing Fermi problems in which the elements to be estimated occupy a big area are regular and are arranged in the order of a clear rectangular surface makes it easier for solvers to make fewer errors.

5. Conclusions

The first result of this study is the categorisation of errors proposed from the review and adaptation of specific studies focused on modelling activities [12] and also of categorisations of errors in procedures connected with magnitudes measurement [14]. Thus, we consider that this categorisation, which was carried out on the basis of the qualitative analysis of a considerable number of productions, may be useful for future research focused on the use of Fermi problems with students at different educational levels.

Indeed, the development of a system of error categories specific to Fermi problems allowed us to address the second objective: to analyse in detail what type of error occurs at each phase of the resolution process. Most pre-service teachers make errors when solving Fermi problems. They are accessible problems because a low number of them remain incomplete, but the errors reflect serious difficulties in the modelling competence and in the estimation and measurement skills of prospective teachers, as most of them are conceptual. The resolution plans involve conceptual difficulties associated with the first two phases of the modelling cycle: simplification/structuring to obtain a real model (which, in the resolution plans, we have called the initial model) of the problem situation, and mathematisation to construct a mathematical model.

During the process of simplifying and structuring the situation of the Fermi problem (in particular, configuring the space and the distribution of the elements to be estimated), most errors were related to shortcomings in the sense of estimating surface measurements. It is due to an incorrect perception of the magnitude (area, length) to be considered.

In the mathematisation phase, the most frequent errors were of two types. The first type derives from the erroneous perception of the magnitude, since the terms of the magnitude in the mathematical model are handled in an inadequate manner, mixing measurements of different dimensions. The second is due to leaving the mathematical model unfinished, as the situation’s mathematisation does not allow one to associate a strategy to obtain the estimate.

In the mathematical work phase, procedural errors were numerous, although fewer than the conceptual errors associated with the two previous phases. The most frequent procedural errors were: using an inverse algorithm, especially multiplying areas instead of dividing areas; and not making sufficient notes on the calculation procedures to be carried out.

There were few interpretation errors because, being resolution plans, few solvers provided a numerical estimate that must be interpreted and contrasted with the real situation. Because of this, in this work, there is a lack of knowledge of the potential errors that prospective teachers could make in the interpretation and validation phase. It is necessary to study future resolutions of the Fermi problem sequence in which prospective teachers make measurements and calculate a numerical estimate from real data obtained or estimated. Will there be a greater number of errors in the interpretation and validation of the results? Other studies [12,33] suggest so.

Knowing the categories and types of errors made by pre-service teachers facilitates the design of initial training in problem resolution and modelling that takes them into account in order to incorporate them into the teaching and learning process. In this sense, the third objective, to verify that there is a significant statistical relationship between the context characteristics of the problems and the categories of error in each modelling phase, allowed us to design sequences of Fermi problems similar to P1-P2-P3-P4. These sequences contribute to reinforce one or another category of errors, depending on the problems’ context characteristics. For example, to plan initial training for prospective teachers in modelling and estimation, it is convenient to design sequences of Fermi problems that begin with problems with the contextual characteristics of P4, as they are simpler, and end with problems with the contextual characteristics of P3 and P1, which are the most complicated. Additionally, if, for example, we want to reinforce errors associated with the confusion of length and area, it is advisable to use problems similar to P2. Designing didactically efficient task sequences to scaffold learning in modelling and problem resolution is a fertile and important field of research.

Regarding limitations and future issues, in the present study we focused on analysing the written productions that include the resolution plans of students faced with solving estimation problems. This forced us, when analysing each of the resolutions, to consider exclusively the errors that appeared written in the productions. No doubt an analysis based on semi-structured interviews or recordings of the resolutions could complete this study. However, the techniques used, in particular the described categorisation of errors, can also be very useful in the temporal analysis of students’ resolutions, using tools such as those described in [50,51]. Indeed, identifying, during the resolution process, at what point certain errors appear can be key to finding relationships of dependence or hierarchies between them. Certainly this may be helpful in future research focusing on students’ solving processes and difficulties when confronted with Fermi problems.