Understanding Map Misinterpretation: Factors Influencing Correct Map Reading and Common Errors

Abstract

Misinterpreting maps can have serious consequences, especially in situations requiring quick decisions like using car navigation systems. Studies indicate that a map reader’s experience is crucial for understanding maps, but factors such as age, education, and gender can also influence interpretation. However, understanding only the proportion of correctly interpreted information is not enough. It is essential to investigate the types of mistakes made and their causes. To address this, we conducted a study available in six languages with 511 participants who completed an online questionnaire testing their map reading skills. The questions focused on scale usage, mental rotation, and recognizing map categories (relief, line and point symbols, and geographic names). Gender had significant relation with one skill, qualification with two and age with three. Experience was associated to the highest number of skills, a total of four, confirming previous findings. When making mistakes, participants tended to overestimate distances and struggled with conceptual similarities in symbol recognition. Experienced readers often misplaced reference locations of geographic names. The results of the research could be used in the design of large-scale maps (e.g., car navigation), as they allow to reduce typical map reading errors by careful selection of symbol types and placements.

1. Introduction

The maps that people use during navigation are most likely to appear on the screen of a cell phone or car navigation device [1,2]. These maps can be at any scale, depending on the zoom level, but for navigation and wayfinding, they are typically viewed at a large scale. Navigation is the process of following a planned course, whereas the goal of wayfinding is to maintain an accurate knowledge of one’s spatial position [3]. The two scenarios often overlap in real life, and neither of them necessarily requires a map, except when travelling in unfamiliar places [3,4]. Maps displayed on digital devices can be used for both navigation and wayfinding, but since the former typically allows less time, only one to two seconds, to read a map [5], correct understanding of the map is crucial. To ensure safe map use, map symbols and the associated textual information (typically audio guides in navigation devices) should be clear [6].

Map interpretation and wayfinding are distinguishable competences [7,8]. Someone may recognize the signs on a map but not be able to identify their location or the route to follow. The accuracy of map reading, however, affects the success of wayfinding and navigation [9]. Studies investigating map reading accuracy have examined a wide range of skills, from sign recognition to visual and verbal abilities to mental rotation [10,11]. These skills can be considered as statistical variables and are most commonly represented by the seven competencies defined by Muir in 1985: sign recognition, sense of directions, distance- coordinate- and scale reading, recognition of perspective depicts and morphology [12]. The types of tests that can quantify the map reading skills can be divided based on studies e.g., done by Gary L. Allen and his colleagues [13]. The test questions usually measure: (1) surveying (memorizing details on a large area), (2) recognition (spotting specific objects in a messy context), (3) completing (recognition of an object from a detail) and (4) mapping (memorizing the place of objects relative to each other).

Errors made during map reading depend on factors that are only partially known despite a large number of studies [10,11]. These can be considered as partly learned errors that evolved during map use through inaccurate interpretation of map-related concepts. Examples of the latter can best be found when reading small- and medium-scale maps, especially when investigating the matter of map projections [14,15]. On the other hand, age, gender and education of the map reader also influence the occurrence of errors [11,16,17,18], but most importantly by the frequency of map use [12,19,20,21]. Thus, the symbol system of maps constructed on the basis of habits and traditions of representation can “condition” the map reader, and therefore he/she cannot always adapt to other maps, which causes misinterpretation of symbols. Another reason for errors may be the reliance on intuitive recognition of “familiar” symbols. Map representations can only be read correctly according to the accepted sign system of the time and culture, and for example from old maps we can only interpret what is intuitive or what is the same as today [22]. By focusing on map reading errors, we can therefore understand what map readers of the present age are reading incorrectly and why.

In general, today’s map representation paradigms have a well-defined set of symbols, which are usually used in some way on maps (e.g., linear elements, contour lines, point symbols, place names, etc. considered as map data types), but they do not have the same meaning for the readers [23], and in addition, mental transformations (e.g., travel time estimation from scale) and rotation are the most important factors that influence the interpretation of the map. In the present study, these skills and map data types were tested for large scale maps.

In addition to exploring the interdependence of difficult questions and skill sets, we focused on whether there was a pattern behind the incorrect answers, which could be interpreted as a learned/cultural background error. We expect that these results may help map design and data visualization for navigation purposes. We aimed to find these results by answering the following research questions:

• Which of the demographic variables (e.g., age, gender, etc.) are most related to map reading competencies?
• Is the map reading performance related to the test subjects’ language?
• Does screen size affect map-reading performance?
• Are there specific patterns in the error types based on the test results?

Related Studies

There have already been several studies on research questions 1–3, and prior experiments conducted by the Research Group on Experimental Cartography (RGEC) at ELTE University (of which the authors are members) have established a foundation for our anticipated outcomes. However, the nature of errors in map reading, which is our 4th research question, has been investigated only by few, and to our knowledge has not been specifically investigated for large scale digital maps.

Research on demographic variables goes back decades and the number of studies on this topic is very large, but useful summaries are still being produced [10,11]. However, this paper does not aim to provide a comprehensive overview of the literature as these works do, so we will only focus on those that provide the key findings on the research questions. Demographic variables are usually gender, age and education, but can also include cultural background.

The recognition of the gender difference in map reading was one of the earliest achievements of user studies for cartographic purposes. This difference is mainly related to the practical application of spatial concepts, and on maps that contain contour lines or require mental rotation for interpretation, males have generally performed better in many studies [17,24,25]. However, even the early studies have shown that the gender gap in map reading is not significant for most competence indicators [26,27].

The age of the map reader is a variable examined in almost all studies, but it is usually correlated with map reading experience, so it is often difficult to distinguish which of the two variables is related to the competence under investigation. A significant number of studies have focused on younger age groups and have shown an improvement in map reading performance with increasing age already in the teenage and young adult years [28,29,30]. Improvements can also be found when looking at older age groups [18]. The authors explain the improved results mainly by the acquisition of geographic experience.

People with higher education must have reached a certain age, so the positive correlation between age and map reading experience is also likely to be true for the education level, and for example, a relationship between these three parameters can be shown for the ability to relief interpretation and mental rotation [20]. However, as studies on map reading ability are rare for the whole population, less information is known about this.

Culture and language can define how people use maps. Studies using multilingual map reading tests have shown the existence of cultural differences [20,24,31,32], but the cause of this phenomenon is generally not well explained. Cautious explanations generally include differences in the quality and methods of education, different social backgrounds, and the symbol key and quality of map materials available in a given language.

The effect of screen size on map use performance in navigation tasks has also been investigated [33], and the results showed that displaying a smaller map size reduces performance and slows down responses, although the study was not conducted on a real small screen device and the maps were not dynamically zoomable. However, experiments conducted by the RGEC with digital maps with dynamically zoomable content [34] showed that the problem is more complex than this: screen zooming is used differently by users depending on the task and their experience. Zooming in, i.e., viewing the map at a larger scale, is typical in navigation and orientation tasks, while a smaller scale was preferred by test subjects for sign recognition and name interpretation. An experiment with pedestrian navigation also suggests that users zoomed in depending on the visibility of the decision point and landmarks, i.e., the size of the area displayed on the screen depended on the task at hand [35]. In essence, screen size is irrelevant if the content can be dynamically zoomed in, but as people with little experience are less likely to zoom in, this also affects their performance. In-car navigation maps are also small-screen, dynamically zoomable, large-scale maps, but to reduce risk, zooming is usually done automatically depending on the position of the car [36].

The nature of user errors due to map display has also been investigated in car navigation tasks [37]. Here, the experiment was conducted on simple maps with a simulator with a small number of participants and found that although users preferred richer map content (e.g., landmarks), it slowed them down and made the navigation more uncertain.

2. Materials and Methods

2.1. Test Settings

To investigate the map reading skills of the test subjects, we utilized a preexisting map reading test of the RGEC operated at the ELTE university (ktk.elte.hu), which already provided insights about university students’ map reading skills [20]. Compared to the preexisting study we collected new fillings. Before starting the test, the participants were noted that they are taking part of a research survey measuring the map reading skill and how long the test takes approximately. They were also informed that the test is anonymous, does not collect personal data and that the data collected is used for research purposes only. The test involved 4 maps and 8 questions. One of them was an open-ended question, the rest were multi-choice questions with 4 + 1 answers (4 possibilities and an “I don’t know” option) where the participants could select only one option. The questionnaire was available in 6 languages: Hungarian, English, German, Spanish, Bulgarian and Romanian. Both the questions and the maps were translated to these languages. The translation or proofreading was done by native speaking colleagues. In the course of the translation, the placement of geographical names has not been changed on the maps and has followed the general name placement rules [38] as much as possible. In the next part of the test, participants completed a questionnaire on their map-using habits. At the end their results and the correct answers were presented to the participants. The research design and data collection methods were approved by the Council on Talent Support and the Research Ethics Committee of the Faculty of Informatics at ELTE University.

The test was available online and the data was stored in a MariaDB database (another open source version of MySQL) hosted by the research group. Thanks to the virtual setting, it was possible to randomize the sequence of questions for the participants, using the Node.js backend. The questions focused on the map reading skills defined by Muir [12] and Clarke [19] but were narrowed down to competences for large scale maps (

Table 1. The competencies examined in the research and their relationship with primary map use scenarios and test question types.

No. of Question	No. of Map	Competence Targeted by Test Question	Primary Scenario of Use	Type of Test Question
Q1	1. (Figure 1)	Interpretation of Hypsography	Wayfinding	Surveying, Mapping
Q2	1. (Figure 1)	Orientation Skill and Mental Rotation	Wayfinding	Mapping
Q3	1. (Figure 1)	Distance and Travel Time Estimation	Wayfinding	Mapping, Completing
Q4	2. (Figure 2)	Interpretation of Hypsography and Mental Rotation	Wayfinding	Surveying, Mapping
Q5	3. (Figure 3)	Interpretation of Map Symbols	Navigation & wayfinding	Recognition
Q6	3. (Figure 3)	Interpretation of Geographic Names	Navigation	Recognition, Mapping
Q7	4. (Figure 4)	Interpretation of Topographic Objects	Navigation & wayfinding	Recognition, Surveying
Q8	4. (Figure 4)	Use of a Scale Bar, and Distance Estimation	Navigation & wayfinding	Completing

).

The maps used in this research were created for this purpose only and show fictitious areas. The symbol key was monochrome and contained symbols that are found on most European topographic maps [39,40]. The maps were edited following the general principle of reducing the amount of distractions present, which can overload the user’s attention, while maintaining the maximum informative value of the map for the questions asked [41]. Similarly to popular online map portals (e.g., Google Maps, Bing Maps, Yandex Maps) and other experiments with navigational maps [33,36,37], we did not include legends to the maps, requiring users to intuitively understand the symbols. The geographic names on the maps were not the same for all linguistic variations, they were sometimes changed to have a more natural sounding for the native speakers.

The first map was a relief map with 5-m contour line intervals, a north arrow, and a scale bar (note that the map was not oriented to north). It also included points marked on the map and a path represented with a dashed line (Figure 1). Three questions were related to this map. Question number 1 (Q1) tested the interpretation of hypsography by defining the relative height of the marked points. Solving this task required spatial abilities and an understanding of the concept of contour lines. Question number 2 (Q2) tested the orientation skill and mental rotation by determining the direction between two points. The third question (Q3) measured the distance and travel time estimation skill by defining the distance between two points using the scale bar. Each response option included both distance and duration.

The concerning questions were the following (Correct answer is in bold):

• Which statement is correct?
  • (A) Point C is the lowest.
  • (B) Point D is lower, than point A.
  • (C) Point B and D area the same height.
  • (D) Point B is higher, then point A.
  • I don’t know.
• In what direction is point D from point B?
  • (A) Northwest
  • (B) Southwest
  • (C) Southeast
  • (D) Northeast
  • I don’t know.
• How long is the path between point A and B? How long is the travel time on foot?
  • (A) 4.5 km, 1 h and 20 min
  • (B) 2.6 km, 2 h
  • (C) 10 km, 4 h and 30 min
  • (D) 3 km, 1 h
  • I don’t know.

The second map was another relief map with 20 m interval contour lines, hydrography, and a north arrow (Figure 2). Like the previous map, this one was not oriented to the north either. The fourth question (Q4) was related to this map and tested the interpretation of hypsography and mental rotation skills in a complex way by defining the flow direction of the stream depicted on the map.

The concerning question was the following (Correct answer is in bold):

4.

In which direction does the Singer Beck flow?

• (A) Northwest
• (B) Southwest
• (C) Southeast
• (D) Northeast
• I don’t know.

The third map resembled a simplified topographic map with higher information density than the previous ones: along with the relief, it also contained hydrography, land cover and various geographic names (Figure 3). Question number 5 (Q5) tested the interpretation of map symbols by defining the coverage of a marked region in the map. The symbols applied here are commonly used in tourist maps, so ideally, experienced map users were familiar with them. The sixth question (Q6) tested the interpretation of geographic names by finding the name of a designated area on the map. For this, the participants had to recognize and categorize the geographic names. Unlike the others this one was an open-ended question meaning that the participants had to type in the answer.

The concerning questions were the following (Correct answer is in bold):

5.

What type of area does mark “A” show?

• (A) Forest
• (B) Rocky
• (C) Vineyard
• (D) Bush
• I don’t know.

6.

What is the name of the town indicated with dark grey color? (open-ended question: the answer should be typed)

• Weston

The fourth map, in contrast to the others, did not contain contour lines, but other topographic features, geographical names and a scale bar (Figure 4). The seventh question (Q7) tested the interpretation of topographic objects by defining the number of rail crossings between two points. This task required the categorization of linear objects and recognition of different road types. The final, eighth question (Q8) measured the use of a scale bar, and distance estimation by measuring the distance between two points.

The concerning questions were the following (Correct answer is in bold):

7.

How many railway crossings* can be found between Upper Ings and Felston railway station?

*Road and railway meeting at the same level (the explanation was provided within the test).

• • (A) 6
  • (B) 5
  • (C) 8
  • (D) 1
  • I don’t know.

8.

What is the distance between point A and point B?

• (A) 2 km
• (B) 5 km
• (C) 3.5 km
• (D) 6 km
• I don’t know.

For proper data evaluation we also asked some questions regarding demographic information from the participants: their age, education, sex, and map use frequency (Supplementary File S1). The full procedure of the presented study is depicted in Figure 5.

2.2. Test Sample

To ensure a large sample we included the collected data of a previous study done with the same questionnaire, the results of which have already been published [20]. This means that the data collection started in the autumn of 2015 and lasted until April 2022 (with the first study lasting from 2015 September to November). Although the test was not distributed as a campaign from the end of 2015, it remained available and was occasionally presented at promotional events related to university work. The average fill time was 8.5 min. A total of 805 people participated in the research, which is a 32% increase compared to the data analyzed in the previous work. To ensure the consistency of the data, we removed all unfinished tests and filtered the ones with fill time below 2.5 min and above 15 min. The time limits were defined according to the natural breaks in the dataset. After the filtering, the remaining 511 completions provided the data for the analysis. The participants were selected by voluntary sampling, as they were invited to the study via social media, email lists and the help of colleagues from our and other universities, which also helped us reach many people. Data collection is therefore considered non-probability sampling, which is why the study is considered exploratory in nature.

The research was available in six languages: 54% (276 participants) of the participants finished their test in Hungarian, 19% (96 participants) in Romanian, 8% (44 participants) in English, 8% (41 participants) in Spanish, 6% (30 participants) in Bulgarian and 5% (24 participants) in German. Almost half, 56% of the participants were female, and 44% were male. The test subjects were categorized into 4 age groups: below 20 (15%), 21 to 25 (37%), 26 to 30 (16%), above 31 (32%). The youngest participant was 16, while the oldest was 71 years old. The age groups were consistent with only a few gaps above 55 years. These age groups were defined parallel with the previous study to keep the results comparable.

As the test, launched in 2015, was part of a research targeting university students, the majority of the initial completions are from university students or graduates. The languages were chosen based on the research group’s connections with universities abroad to facilitate the collection of data from foreign participants. Subsequent completions have partly compensated for this, but the results are still dominated by graduates. Hence most of the participants (89%) have at least a bachelor’s degree, and the remaining participants finished secondary or primary education (7% and 4% respectively).

In addition to basic demographic data, map readers were categorized not only by language: we also grouped the participants by how often they used maps, as our previous studies have shown a direct link between correct answers and the frequency of map use [20,34]. Based on the results of these previous studies we categorized the participants into two map reading groups: those who use maps at least once a month (experts), and those who use maps less (beginners), 74% and 26% of the participants respectively.

In our research, we also investigated whether the success of reading large scale maps depends on the screen size of the map reading device, so we recorded the display size using client-side JavaScript and user-agent strings. These were used to log the time the page was loaded and the time the form was submitted in order to accurately evaluate the time it took to complete the questions. Cookies were also installed when the test was launched from a browser to prevent a typical user from completing the test repeatedly. It is important to note that the display size information was collected later during the research, so only 171 participants’ browser information was stored, with 158 of them finishing the test.

3. Results

The results were first evaluated from a general point of view and then we focused on the incorrect answers. In both aspects, it is possible to distinguish between navigation task-related and wayfinding-related question types. Questions Q1–4 tested the skills needed mainly during wayfinding tasks, while questions Q5–8 tested skills that can be important both during the navigation tasks and during wayfinding. For the analysis we used IBM SPSS Statistics version 26.0. For the analysis we used the chi square test to find connections between the results and demographic variables, one-way ANOVA for the relation between the demographic groups and fill time, Pearson’s correlation to see the relation between the fill time and score also between the screen size and score. Finally, we applied the binomial test to relatively see the frequency distribution of mistakes.

3.1. The Results from a General Aspect

The results in general show that participants reached the highest score at the interpretation of hypsography (76% correct answers). On the other hand, the worst performance was shown with the interpretation of topographic elements (38% correct answers).

During the general evaluation, the participants were compared based on their age group, gender, education, and map reading skill.

Table 2. Comparison of the results (percentages of correct answers) of various subcategories. The cells highlighted are significantly different from the cells indicated in the subscript at p ≤ 0.05 Chi-square test.

	All	Gender		Qualification			Map Reading Skill		Age Category
		Female (56%)	Male (44%)	Higher (89%)	Secondary (7%)	Primary (4%)	Beginner (26%)	Expert (74%)	0–20 (15%)	21–25 (37%)	26–30 (16%)	31– (32%)
		A	B	A	B	C	A	B	A	B	C	D
Q1	76.1	76.6	75.6	77.5	66.7	63.6	69.5	78.4 a	69.2	82.2	77.2	71.8
Q2	71.8	72.4	71.1	72.4	66.7	68.2	71.0	72.1	69.2	70.2	74.7	73.6
Q3	73.4	74.5	72.0	73.3	72.2	77.3	65.6	76.1 a	66.7	71.7	81.0	74.8
Q4	67.9	71.0	64.0	70.0 b,c	52.8	50.0	58.0	71.3 a	53.8	64.9	69.6	77.3 a
Q5	75.1	74.1	76.4	74.6	80.6	77.3	73.3	75.8	83.3	73.8	75.9	72.4
Q6	60.2	64.3	56.0	62.5 c	55.6	31.8	53.4	63.2	46.2	51.3	62.0	77.9 a,b,c
Q7	38.4	33.9	44.0 a	38.2	41.7	36.4	28.2	41.8 a	29.5	34.6	34.2	49.1 a,b
Q8	75.1	76.9	72.9	76.4	63.9	68.2	69.5	77.1	67.9	77.5	78.5	74.2
Total	67.3	68.0	66.5	66.5	62.5	59.1	61.1	69.5	60.7	65.8	69.1	71.4

shows the rate of correct answers based on these variables and highlights statistically significant differences between them.

The table shows that the most significant differences can be found when comparing the participants based on their map reading skills. For this we analyzed the proportion of correct answers by each of the demographic categories. Since we wanted to know the association between these two nominal variables, we utilized the chi square test. This was possible because the variables were independent as each entity only contributes to one cell in the contingency table (i.e., no participant is in multiple groups within a demographic category, and no participant made correct and incorrect answer at the same time), and the frequency of each group is larger than 5, it was possible to use the chi square test for the analysis. The expert map readers had significantly better performance in four tasks: the interpretation of hypsography (Q1) with X2 (1, N = 511) = 4.299, p = 0.038 V = 0.092 showing negligible association between the task and map reading skill [42]. Regarding the distance and travel time estimation, significance with a moderate association was found with X2 (1, N = 511) = 5.399, p = 0.020 V = 0.103. Similar results occurred with the complex hypsography and mental rotation task (Q4) with X2 (1, N = 511) = 7.908, p = 0.005 V = 0.124 showing significant differences with moderate association. Finally, regarding the interpretation of topographic objects, the experienced map users performed significantly better with X2 (1, N = 511) = 7.618, p = 0.006 V = 0.122 showing moderate association.

The participants’ gender had association with performance in case of only one question: males performed better during the interpretation of topographic objects (Q7) significantly with X2 (1, N = 511) = 5.416, p = 0.020 V = 0.103 indicating a moderate association. Other than that, no other significant difference was found.

When looking at education, participants with higher education reached significantly higher results at the complex question (Q4) testing both hypsography and mental rotation compared to the primary education group (p = 0.048) and to the secondary education group (p = 0.032) with X2 (2, N = 511) = 7.909, p = 0.019 V = 0.124 indicating a moderate association between the education level and the performance for this task. This group also performed better compared to the primary education group (p = 0.004) regarding geographic names (Q6) a significance of X2 (2, N = 511) = 8.686, p = 0.013 V = 0.130 again showing moderate association.

Regarding age, the 31+ group had the highest result at the question testing both the mental rotation and the interpretation of hypsography (Q4), with a detectable significance compared to the youngest group (p = 0.001) at the value of X2 (3, N = 511) = 14.563, p = 0.002 V = 0.169 indicating a moderate association. Significant differences were found between the oldest and the other age groups: p = 0.000 with the youngest and with the 21–25 groups, and p = 0.009 with the 26–30 age group when interpreting geographic names (Q6) at the value of X2 (3, N = 511) = 34.275, p = 0.000 V = 0.259 showing a medium association between age and the performance at this task. They also performed statistically better than the two youngest groups at the interpretation of topographic objects (Q7) at p = 0.024 compared to the youngest and p = 0.034 compared to the 21–25 age group with X2 (3, N = 511) = 12.273, p = 0.007 V = 0.155 indicating a moderate association.

Regarding response time, the time per question and total time were compared within each classification group using one-way ANOVA. This was possible because, as stated earlier, the variables are independent and most of the groups were larger than 30, so it can be assumed that they are normally distributed based on the central limit theorem [43], except for the primary education category, where the Shapiro-Wilk test was not significant (p = 0.064), proving the normal distribution of this group too. The variances were the same throughout the data and finally, the response time was a ratio variable. No significant difference was found for response time.

3.2. Looking at the Different Circumstances of Fillings

As mentioned earlier, the test was internationally available in multiple languages in order to inspect cultural variances in map reading, similarly to the study done by Ito and Sano [32]. Although we asked participants to fill the test with their native or best-spoken language, this may not guarantee their nationalities.

Table 3. Comparison of the results (percentages of correct answers) based on the participants’ language. The cells highlighted are significantly different from the cells indicated in the subscript at p ≤ 0.05 Chi-square test.

	Total	Language
		Hungarian (54%)	English (8%)	German (5%)	Romanian (19%)	Spanish (8%)	Bulgarian (6%)
		A	B	C	D	E	F
Q1	76.1	74.6	84.1	91.7	71.9	73.2	83.3
Q2	71.8	75.0 d	79.5	87.5	58.3	65.9	70.0
Q3	73.4	73.6	75.0	70.8	74.0	75.6	66.7
Q4	67.9	66.7	75.0	70.8	74.0	53.7	66.7
Q5	75.1	83.0 b,e	36.4	70.8	78.1 b	61.0	73.3 b
Q6	60.2	46.0	95.5 a,f	79.2 a	81.3 a,f	75.6 a	43.3
Q7	38.4	48.2 d,f	38.6	50.0 d,f	19.8	29.3	10.0
Q8	75.1	77.2	70.5	83.3	69.8	68.3	83.3

shows the significant differences according to the language participants used for the test. As mentioned in the previous section, the data met the prerequisites of the Chi2 test, thus it was used for the analysis. The results of orientation skill and mental rotation (Q2) showed that Hungarian speaking participants performed significantly better than the Romanian speaking ones at p = 0.030 with X2 (5, N = 511) = 14.99, p = 0.010 V = 0.171 indicating a moderate association. The interpretation of map symbols (Q5) showed a noteworthy result where the English-speaking participants performed statistically lower than the Hungarian (p = 0.000), Romanian (p = 0.000) and Bulgarian (p = 0.027) speaking participants with X2 (5, N = 511) = 49.639, p = 0.000 V = 0.312 indicating strong association. On the other hand, regarding the interpretation of geographic names (Q6) English speaking participants reached the highest score significantly outperforming the Hungarian (p = 0.000) and Bulgarian (p = 0.000) speaking participants with X2 (5, N = 511) = 75.247, p = 0.000 V = 0.384 also showing strong association. This task seemed to be especially difficult for Hungarian and Bulgarian speaking participants: English (p = 0.000 for both) and Romanian (p = 0.000 and p = 0.001 respectively) speaking participants performed significantly better than both, while German (p = 0.027) and Spanish (p = 0.006) speaking participants also had a statistically better performance than Hungarians. Finally, it’s also interesting to note the interpretation of topographic objects (Q7) where the Hungarian and German speaking participants performed the best, showing statistically better results than the Romanian (p = 0.000 and p = 0.037 respectively) and Bulgarian (p = 0.001 and p = 0.017 respectively) speaking counterparts with X2 (5, N = 511) = 38.290, p = 0.000 V = 0.274 indicating strong association.

As well as the choice of language, another important circumstance of the test was the screen size the participants used. Along with the results the test stored the clients’ screen size. The screen size was represented by the screen diameter, derived from its width and height. Then we calculated the Pearson correlation between the screen size and total score and total filling time to see if they associate to one another. This was possible because the variables were at least interval-type and normally distributed (based on more than 30 items and the central limit theorem). The results showed that there was no correlation between the score and the screen size at r(156) = 0.017, p < 0.835 nor between the fill time and the screen size with the result of r(156) = −0.015, p < 0.851. As mentioned in the previous section, we started collecting this information later during the research, so only 158 records were evaluated.

3.3. Focusing on the Errors

The proportion of wrong answers for each question can provide meaningful insights (Figure 6). For this we excluded the correct answers and only compared the ratio of false answers. Of the four possible answers to the questions, in theory, those who give the wrong answer—because they don’t know the correct answer—will click on the remaining three options in equal proportions. The proportion is different if, for some reason, the respondent prefers one of the wrong answers. Examining the incorrect answers can therefore provide information about respondents’ misconceptions. The interpretation of geographic names wasn’t included in this kind of analysis, because unlike the others this was an open question.

The results show that the questions regarding hypsography and mental rotation, and the interpretation of map symbols were the most balanced in terms of wrong answers. The other side of the spectrum was the distance estimation and utilization of scale bar, where more than 75% of wrong answers slightly overestimated the distance (C answer). Similar proportions show up regarding the interpretation of topographic elements, where 73% of wrong answers misinterpreted the railway crossings with the overpasses (D answer), resulting in many mistakes. The interpretation of hypsography is also worth mentioning, where 65% of wrong answers misinterpreted the contour lines. Regarding the question on mental rotation, most of the wrong answers indicated the opposite direction. This could be because the map was not oriented northwards, but it could also be because the question was misinterpreted. Finally, regarding travel time estimation, most of the wrong answers (C answer) assumed that the distance is more than double the correct one.

For greater insight, the results were grouped into seven classes according to the number of errors (1–7) depicting the relative frequency distribution of mistake types. This way the most common errors can be examined by the number of wrong answers. We also created an eighth category as a reference value, which depicts the assumption that the rate of mistake for each question has uniform distribution and has equal frequency (Figure 7). Since there were 8 questions in the test, if 12.5% of participants made mistakes per question, would achieve a uniform distribution. This helps us find certain mistake patterns between the groups. The higher the ratio, the more group members made a mistake at the specific question.

Because we assumed the uniform distribution of the data, we applied the binomial test to compare the reference value with each of the mistake categories (

Table 4. Proportion of incorrect responses per question, based on the number of mistakes. The cells highlighted are significantly different from the reference value (12.5) at p ≤ 0.05 binomial test.

	1 Mistake		2 Mistake		3 Mistake		4 Mistake		5 Mistake		6 Mistake		7 Mistake
	Prop.	p	Prop.	p	Prop.	p	Prop.	p	Prop.	p	Prop.	p	Prop.	p
Q1	8.8	0.547	8.0	0.067	6.6	0.001	9.1	0.074	9.8	0.245	13.1	0.796	12.3	1.00
Q2	5.3	0.110	8.5	0.107	10.6	0.383	10.1	0.234	11.9	0.847	15.3	0.302	12.3	1.00
Q3	8.8	0.547	5.5	0.002	8.0	0.018	11.0	0.497	13.1	0.771	12.4	1.00	14.8	0.502
Q4	7.0	0.313	6.5	0.010	11.0	0.485	15.5	0.126	15.2	0.208	12.4	1.00	13.6	0.737
Q5	8.8	0.547	10.6	0.454	9.0	0.067	9.8	0.149	9.8	0.245	7.3	0.070	11.1	0.867
Q6	31.6	0.001	19.6	0.005	16.6	0.036	13.9	0.445	11.9	0.847	8.8	0.243	11.1	0.867
Q7	12.3	1.00	34.2	0.001	29.2	0.001	21.1	0.001	20.5	0.001	18.2	0.251	12.3	1.00
Q8	17.5	0.232	7.0	0.018	9.0	0.067	9.5	0.107	7.8	0.025	12.4	1.00	12.3	1.00

). The question related to topographic elements (Q7) had a significantly higher rate of errors for the mistake groups 2, 3, 4 and 5. Interestingly the rate of error was the same for the best and worst performing mistake groups. On the other hand, the mistake group 1 and 2 had significantly more errors at the geographic names (Q6) compared to the reference value.

The data shows that the mistake group 3 had a significantly lower rate of errors than the reference value. Regarding the travel time estimation, it is noteworthy that the mistake group 2 had a significantly lower rate of errors than the reference value. Finally, it can be also noted that the mistake groups 6 and 7 had the same proportion of errors as the reference value for 3 and 4 questions respectively.

4. Discussion

The results can be discussed in more than one way: according to the research questions, on the one hand, and according to the primary map use, on the other. In addition to answering the primary research questions, the practical applicability of the results was addressed.

4.1. On the Demographic Variables Most Related to Map Reading Competencies

Regarding the first research question, the results showed that the participants’ map reading skill is related to all demographic variables. Gender had a moderate relation with one question: it showed a significant connection with the topographic object interpretation task. This is important to highlight as the gender gap is thought to be much more significant in terms of the number of studies on this issue. For example, in the review by Havelkova and Hanus, half of the 67 studies examined included gender as a factor [11]. This review study concluded that gender as a factor can only be considered as an influential one with a high degree of uncertainty, and, as mentioned earlier, the first studies decades ago emphasized that males perform significantly better than females in only a fraction of map reading competences [26,27], which findings are also consistent with our results.

Education and age of the participants had more associations with the map interpretation. Participants with higher education tended to perform better at the complex hypsography & mental rotation task and at interpreting geographic names. The results were similar for the age groups, where along with these two tasks, the oldest participants outperformed the younger in the topographic object interpretation task. Comparing the results of the age groups with the results of our previous study in 2016, it was found that the same pattern was observed there [20]. This is consistent with the results of other studies examining a similarly broad age range [18,28]. Not too surprisingly, map reading experience had the most connections with the test results, having a significant association on five tasks. The latter, therefore, confirmed our own [20] and other authors’ [12,19,21,29] previous results, which highlighted the frequency of map use as the most significant factor.

In map-reading studies, the interpretation of the more complex concepts of spatiality (such as relief and perspective) and mental rotation are most often associated with experience and gender differences [17,24,25,27]. Our results suggest that there is no significant relationship with these demographic variables when considering mental rotation only, but when considering spatiality and mental rotation as a complex issue, the relationship with experience is clear. In this case, age and education were found to have a significant association in addition to experience, and in all three cases those with more experience (i.e., older, or higher educated) performed significantly better. In the present study, no significant gender difference was found in this complex competence, which is interesting because in our previous study with the same questionnaire, this difference was still present [20]: males performed better than females among university students. The different results may be explained by the fact that the sample population was different in the two studies.

4.2. On the Map Reading Performance Related to the Test Subjects’ Language

Moving on to the second research question, the results showed significant relationships between some of the tasks and the participants’ language. It is important to note, however, that the difference is more likely to be caused by the varying cartographic traditions of different countries rather than explicitly the spoken language. The relatively few studies that have looked at the effect of linguistic (or cultural) background confirm this assumption [24,32].

For example, during the interpretation of map symbols a large percentage of wrong answers assumed “rocky” is the correct answer, which might be caused by the symbol style resembling a round boulder. Interestingly this symbol is widely used in Hungarian tourist maps for bushy open areas, but it could have caused confusion for participants from other countries where different symbols are used. This can be traced in the results of language groups, where participants who completed the English version performed the worst and Hungarian speaking participants outperformed all other groups. This issue could be tackled by applying a map legend depicting all navigation-related map objects, but we emphasize that this is a limited possibility for small-screen maps.

4.3. On Screen Size Affecting the Map-Reading Performance

The third research question is best answered by the use of scale bars on maps and the related results. The wrong answers at the distance estimation show that participants overestimated the distance between the two points by 75%. It is also noteworthy that no significant differences were found between the respondent groups for this task, meaning that this type of error could occur for all kinds of map users, although the probability of mistake increased in proportion to the number of wrong answers for this task. The estimation was made by the respondents using a short scale bar. Although it would be logical to assume that a larger screen would improve the performance compared to a mobile device, and the findings of an early study on small display maps also refer to this [33], the correlation between screen size and results showed no evidence of this. In fact, the lack of correlation between fill time and screen size suggests that the device size does not play a significant role in navigation-related tasks, answering our third research question.

4.4. On Specific Patterns in the Error Types

Concerning the fourth research question, the results for each of the test questions are discussed.

The question Q1 (“hypsography”) was not among the more difficult ones, as the relative proportion of incorrect answers remained below the reference value even for those who made the most mistakes. It is also remarkable that the mid-performing group with 3 mistakes had a significantly lower rate of error than the reference. Test subjects who made errors failed to notice a smaller height difference of two points, and in most cases estimated the two points marked in the answer option to be the same height. Knowledge of the use of a contour map was required to detect the difference. This type of task is most likely to be encountered by map readers on foot, e.g., when hiking. To tackle this issue, it may be wise to aid map readers with additional information such as indicating the exact height of points. This is particularly important for novice map users who are not sure about how to interpret contour lines. This has been demonstrated previously by Clark and colleagues [44].

Regarding mental rotation (Q2), no significant difference was found for the demographic groups. Interestingly though, the most common false answer was the opposite direction of the correct answer (Southwest vs. Northeast). This may be due to the lack of contour interpretation mentioned above, as the map was a contour map, although the question itself was not related to height. It is also noteworthy that the probability of errors in this task increases in parallel with the number of mistakes, which implies that this task is more difficult for novice map readers. Another possible explanation is related to the orientation based on the cardinal directions. Studies on the use of cardinal directions suggest that there is a general aversion to them among some people [45,46].

Regarding travel time estimation (Q3), most of the wrong answers assumed that the distance is more than double the correct one. Overestimation of travel time was a characteristic of participants in several studies [47,48,49]. It was also shown in some experiments that unfamiliar terrain and curvy roads increase the estimated travel time [49]. Since both conditions were true for the trail in our question, our results are consistent with these observations.

The proportion of wrong answers in Q4 shows that except for those, who made only one or two mistakes, participants did not always use the topographic information from the contour lines to determine the course of the stream. The question Q4 also assessed mental rotation in parallel with the hypsography. The misinterpretation of contours was already observed in Q1, however, in the latter case (i.e., Q4), there was no significant difference in the relative frequency distribution.

The typical mistakes made in the interpretation of map symbols (Q5) have been described earlier. The wrong answers were due to the fact that, because of the cultural background, the bush symbol was confused with a rocky area by non-Hungarian-speaking respondents.

The recognition of geographic names (Q6) was one of the most difficult ones for the participants. Focusing on the errors, we found that, proportionally, misinterpretation of the names was much more common among those who made fewer errors (experienced map readers), but since this was an open-ended question, statistical methods could not be used to find out more about the nature of the errors. Geographic name placement and proper label fonts are crucial during map creation [38]. In addition to the general principles of typographic element placement, we used a bold font and horizontal placement for the geographic name that represents the correct answer in the questionnaire, which has been shown to be the most effective in map-reading tests for geographic names [50,51]. Despite this, our results show that the correct identification of their reference object on the map seems to be a difficult task even for experienced map users, as this was a frequent mistake for them.

Concerning the topographic objects (Q7), all participants except the best and worst performing ones had significant difficulties. Regarding the topographic objects, the number of correct answers was even lower than the wrong ones. In case of the latter, an overwhelmingly high number of participants mistook the overpass symbol for a railway level crossing, even though the question explained that it was a level crossing. This implies that topographic symbols require detailed attention when creating navigational maps, because they can be easily misinterpreted.

It should be added, however, that the maps used for the questions were deliberately simple, grey-scale graphics without explanations. This reduced the complexity of the signs, but also left their recognition to the respondent, as would be the case in the interface of a navigation application. It was previously thought that this may play an important role in the case of question Q7 on the level crossing, where, in addition to the linear signs indicating the transport elements, the linear signs indicating the settlement boundary also had to be recognized to give the correct answer. However, based on the incorrect answers, the problem was clearly not that the settlement boundary was also seen as a road, but that the more prominent sign of the overpass was not identified as an overpass and that the railway crossings that were not prominent at all were ignored.

For distance estimation (Q8), wrong answers indicate that participants overestimated the distance between two points by 75% and that the probability of mistake increased in proportion to the number of wrong answers for this task. This was already discussed in relation to screen size.

4.5. On Primary Map Use Scenarios and Practical Aspects

The results can also be discussed in terms of which map use scenario (wayfinding or navigation) might be relevant to what they show. In particular, questions on the competences used in navigation are considered here, as they may occur in high-risk situations.

A navigation task involves finding the correct reference point for a geographical name, interpreting topographical signs (including symbols) and estimating distance. In our test, these tasks were not presented on an interface designed for car navigation, nor were the maps designed to simulate this situation. Nevertheless, the pattern of errors could, in our opinion, help research in this direction. Focusing on these errors, the role of voice-based information, for example, might also be worth exploring, as in navigation tasks, auditory information can improve participants’ performance [52].

The other scenario, wayfinding, was present in all but one of the test questions. This type of map use requires more complex skills, and the performance of map users is mostly associated with experience. This is confirmed by a series of previous studies [12,19,20,21,29]. Although the typical situation (hiking, finding your way on foot in a city) is less risky than navigation, the lessons that can be learned from incorrect answers (e.g., about symbols) have practical value.

Due to the unregulated style of online surveying, the size of demographic groups varied, and weren’t representative of any population. Because of this, our study is considered as an exploratory analysis of the discussed phenomena. However, as we do not have up-to-date information on the structure of the population using large-scale digital maps for orientation or navigation, statistical evaluation of data from this type of research could also provide valuable support for practical applications. It is suggested that in future studies, appropriate group sizes, determined by a preliminary survey, should be established to evaluate the proportion of wrong answers of these groups to get more granulated results.

5. Conclusions

Overall, regarding navigation related tasks the interpretation of map symbols and interpretation of linear objects seem to be the most difficult for map readers. The performance for these tasks is connected not only with map reading experience, but with gender, age, and language too.

When expert map readers make errors, it’s most likely to be related to the referencing of geographic names or recognizing topographic elements, while beginners tend to make more mistakes with wayfinding related tasks, especially when it requires complex problem solving (i.e., hypsography & mental rotation). This result is particularly noteworthy because it concerns a group of people who, because of their experience, may feel more confident in interpreting the map, but still make mistakes. In our opinion, this could be a risk factor in car navigation.

The present study also confirmed that map reading experience has the most association with map comprehension, although beyond that, all demographic groups showed some significant differences in map reading performance. On the other hand, educational background had the least number of relations with map reading.

Finally, cultural background (evaluated as the “language” in the present study) also seems to be largely associated to map comprehension. The most likely reason for this is that countries have diverse cartographic heritages and use different visualization for certain phenomena. This should also be considered for navigation applications, but to get an accurate picture of this, data collection at global level would be needed.

This research examined the navigational and wayfinding use cases in a broader sense and the population sample examined covered a wide age range, which is rare in studies of this kind. Future user tests could, however, model the conditions of the use cases (e.g., time constraints in responding to navigation-related questions) and could also investigate respondents’ willingness to use navigation tools to achieve more granular results.