The importance of web maps in the development of the so-called “Multimedia Cartography” was noticed by researchers several years ago [1
]. Also, the Commission on Maps and the Internet of the International Cartographic Association presented the purpose of investigating the role of efficient integration multimedia and maps on the Internet. Multimedia, such as graphics, photos, or video, plays an essential role in the transmission of spatial information on the web [2
]. In internet cartography, one of the fundamental factors is interactivity. Interaction is a human–map communication system [4
]. This is the way in which the user manipulates the map (by changing the scale or panning movement) [5
]. The interaction on the map mainly takes place using the graphical user interface (GUI). It consists of buttons that have specific functions and a symbolic icon [7
]. The most popular interactive buttons include geolocation, searching, changing layers, and routing [9
]. They are available on almost all global map services, such as Google Maps, Bing Maps, OpenStreetMap, Baidu Maps, or Yandex Maps. Kraak and Ormeling [10
], in their manual, described GUI as a ‘minimum requirement’ for designing an interactive map.
Interactivity is part of the users’ experience [11
]. Most web cartographic products are interactive. However, there is a difference between the user interface (UI) and the user experience (UX). User interface refers to tools (usually GUI buttons) that allow communication with a digital map and data manipulation [13
]. User experience is a broader term. This refers to the experience and user preferences of the two-way communication process request–result [14
]. According to Norman [15
], this experience is responsible for the success of a given product. Thus, UI design is a process aimed at implementing tools that allow interaction, e.g., using a Leaflet.js library or dedicated API [16
]. On the contrary, UX design involves developing the interaction result and the communication process itself so that the experience is satisfying for the user [18
From a pragmatic point of view, assessment of both UI and UX seems relevant [19
]. Objective evaluation of GUI tools, such as are currently used on online maps, is possible through efficiency parameters. Efficiency includes, among others, the time for proper competition and a spatial task using a given interactive tool [20
]. Therefore, the GUI buttons themselves have an essential impact on the efficiency of a web map. GUI research considers the placement, number, and graphics of buttons in the first view as influential factors [4
]. This is especially true for global online map services like Google Maps or OpenStreetMap that have the same interactive tools with various arrangements, numbers of buttons, and graphics [26
]. As a result, the use of various map services is associated with a slightly different experience of interactivity. As a consequence, it leads to the separation of users who prefer a given web map [27
Due to the diverse UX of interaction on different web mapping services, the need for measuring these differences arises. Eye-tracking is one of the methods that enable understanding the UX when working with multimedia and the interactive product [28
]. It provides many indicators related to the movement of the gaze in space and time, such as time to the first fixation, fixation count, fixation duration, saccadic amplitude [30
]. Fixations are places were gaze maintains relatively constant, and saccades are a quick movement of the gaze [32
]. This provides an objective way of measuring the users’ perceptual (visual) experience called the visual strategy [33
]. It provides mental attention data [34
]. This type of research is crucial, since some interactions, such as navigation, sometimes do not have a tool in the form of a button, and the user can still experience it.
The research topic presented in this way raises the question about differences in UX when interacting with a web map. Questions that arise include: does a different number of corners with buttons affect the user’s view path? Are there measurable differences in UX between two GUIs with the same buttons but arranged differently? In particular, we aimed to define the difference in the users’ visual experience using eye-movement metrics, such as the total number of fixations (FC), number of fixations on the specific GUI button (FCB), percentage of participants’ attention needed for button identification (ATT), saccadic amplitude (SA), and time to the first fixation (FFT) in three interactive tasks. Participants performed a task in two different GUIs based on Google Maps or OpenStreetMap. The tasks consisted of finding and using the appropriate button that caused the desired interactions (geolocation, searching, finding the route). Eye movement measurements defined users’ visual experience. An additional goal was to determine the efficiency of individual interfaces to compare UI/UX. This was measured with the use of time to the first click on the appropriate button in each task (TC). This study shows that different arrangement of the same interactive tools may have a different effect on the user experience and efficiency of web map.
2. Related Work
The number and placement of buttons play an important role in interface design. Wang [27
] drew attention to this, especially during the opening page. He noticed that if the start page appears to be disordered and does not support the user’s habits, it negatively affects its efficiency. Interface disorder is closely related to the location of the button because, as Wang noted, the search button plays an essential role in the interface layout. On the other hand, Nivala et al. [35
] noticed that interactive web maps are meant for the general public and may not always meet all users’ needs. Nivala, in conclusion, drew attention to the need to design the GUI in an orderly manner. Hegarty et al. [36
] also referred to the issue of the importance of the interface in experiencing interaction. Newman et al. [37
] came to similar conclusions about the simplicity of the user interface. Based on the respondents’ responses, they redesigned the interface to suit preferences (e.g., they changed buttons location, removed navigation arrows, and changed the way of choosing layers).
Some researchers [7
] have drawn attention to the GUI differences arising not only from the placement of individual buttons but also that each map provider has a different graphic style of buttons. Even the same interactive functions, such as wayfinding, may work differently, e.g., by adding waypoints manually or typing the next location. However, as noted by Horbiński et al. [9
], UX and preferences may be different from what map providers propose. Haklay [13
] explained this by the fact that user expectations may be due to the various devices on which they use web maps.
Assessment of the GUI and methods of assessing UX constitute a research gap in cartography. Although there are studies on the use of eye-tracking in interface studies, only some refer to cartography [38
]. Çöltekin et al. [22
] analysis is one of the few studies in the cartographic field. In their research, they focused on two differently designed interfaces. They measured effectiveness, efficiency, and overall satisfaction during three tasks. The experiment also used eye movement recording. The results gave interesting conclusions about user experience. It turned out that despite the more accurate execution of tasks with the Natlas interface, the study participants preferred the Carto.net interface. These types of conclusions also appear not only concerning for the interface but also for the methods of cartographic visualization [39
]. Eye-tracking results discovered usability problems associated with individual buttons on both GUIs. According to Çöltekin, data from eye-tracking also provide information on users’ expectations.
There are significant similarities in UX while using different interfaces of web map navigation in zooming tasks [40
]. The eye-tracking analysis helped to find differences and similarities in UX while using four methods of zooming interaction: pan zoom, rectangle zoom, double click, and wheel zoom. Manson’s research shows that most users performed better with rectangle zoom, even though they sometimes felt frustration using it. Pan zoom and click zoom were rarely preferred, and users were not satisfied using them.
According to research on the effectiveness of user interfaces, eye-tracking methodology combined with questionnaires provides a crucial setting for examining user experience. The presented studies motivated us to research the assessment of users’ visual experience while performing different interactive tasks based on Google Maps or OpenStreetMap interface.
The research methods used include web map and GUI design, efficiency and eyeball movement data acquisition, and data processing. We also presented the participants’ pool and experiment procedures.
3.1. Web Maps
]. Figure 1
presents the OpenStreetMap opening view developed in Leaflet.js.
3.2. Graphical User Interface
In GUI design, we used the placement of buttons according to Google Maps and OpenStreetMap, which are two of the most popular and recognizable web maps in the world. For the study, we adopted a simplified version of the interface, consisting of the six most significant buttons that enable interaction with the map: geolocation, search, route, change layers, and zoom buttons (plus and minus) [42
]. According to Horbiński and Cybulski [7
], these buttons are on every global web map. Figure 2
as both <body>
. Based on this assumption, we treated all buttons between the <div id="map"> </div>
Because each GUI has its unique graphic style, we decided to unify button graphics and dimensions. We proposed black and white 60 × 60 px buttons with a 2 px frame. Figure 3
shows all buttons along with the labels that refer to Figure 2
. First, we prepared the concept of the layout, and then we inserted button graphics along with interactive functions.
Buttons with specific graphics were placed in the GUI, as shown in Figure 4
, following the arrangements of layouts according to OpenStreetMap and Google Maps. We believe that naming GUIs based on the names of web mapping services is necessary. The point is to draw attention to the fact that it is a practical solution used by map providers.
) plugin is responsible for geolocation, but it does not work correctly on desktop computers. For this reason, we simplified the plugin in such a way that the press of the button caused the view change to a specific location. The leaflet-search.js (https://github.com/stefanocudini/leaflet-search
) plugin, which uses the OpenStreetMap spatial database, is responsible for the search. A circle with a diameter of 5 px marks the found point. We supplemented the plugin with the auto-complete function. We used the leaflet-routing-machine.js plugin (https://github.com/perliedman/leaflet-routing-machine
) so that we could change the position of the zoom out and zoom in buttons. The possibility of zooming and dragging was blocked due to the study of using the geolocation button. We used Boolean values available in the L.map function of Leaflet.js:
3.3. Participants and Experimental Process
In this study, we adopted two research groups of 20 participants each. One group performed survey tasks on a GUI based on OpenStreetMap and the other based on Google Maps. For both study scenarios, participants were randomly selected from a group of students from Adam Mickiewicz University in Poznan from a geodetic and cartographic course. Participants took part in the survey voluntarily and had the opportunity to opt-out of continuing at any time. In the participants’ group performing tasks based on the Google Maps GUI, there were 16 men and 4 women. Nine people were in the 18–21 age range, ten in the 22–25 age range, and one was in the age group of 26 and over. We determined the experience in using Google Maps based on the frequency of use. Five people said they use it every day, eleven people once a week, three people once a month, and one person does not use it at all. In the same way, we defined the experience in using OpenStreetMap. Two people use it once a week, eleven once a month, and seven people do not use it at all. Four people use other web mapping services once a month and the other sixteen do not use them at all. There were 12 men and 8 women in the OpenStreetMap-based GUI group. Fourteen people were in the 18–21 age range, five people in the 22–25 age range, and one person in the age group of 26 and over. One person used Google Maps every day, nine use it once a week, seven people once a month, and three people said they did not use it at all. As for OSM, one person used it every day, eight people used it once a month, and eleven did not use it at all. One person uses other web mapping services once a month, and rest of the participants group do not use at all.
We experimented on a desktop computer with Windows 10 with the Firefox browser. We displayed web maps on a 21.5-inch screen with a resolution of 1920 × 1080 px. To capture gaze, we used Tobii eye-tracker X2-60 with a sampling frequency of 60 Hz. We used the Tobii Fixation Filter with a velocity threshold of 35 pixels/windows and distance threshold 35 pixels [43
]. We used these parameters to classify raw eye movement data into fixations and saccades.
We asked participants in the questionnaire about the frequency of using Google Maps, OpenStreetMap, and other web maps. There were four possible answers for each question: I use daily (rank 4), once a week (rank 3), once a month or less (rank 2), I don’t use it (rank 1). We used ranks to categorize participants’ experience. Therefore, we were able to check if there were correlations between the frequency of using the specific GUI and the time of task completion.
Each group had three tasks to perform using GUI buttons. We implemented all web maps, instructions, tasks to solve, and questionnaires (with questions about age, gender, and frequency of using Google Maps, OpenStreetMap, and other web maps) in Tobii Studio 3.4. Before the participants started the task, it was necessary to calibrate the gaze with the eye-tracking device. The first task was to ‘find your location using geolocation button’. The participant had to point to and click on the geolocation button. This action took him to the current location. However, to complete the task, the participant had to press the F10 button. The second task was to ‘find the Żywiec city using the search button’. After clicking the appropriate search button, the user entered the particular city name and then confirmed it. The F10 button finished the task. The third task was to ‘find a route between Leżajsk and Jasło using the route button’. After clicking the route button, a window appeared in which the participant entered the start and destination and then confirmed his choice. As in previous tasks, the F10 button ended the task. Each participant performed the tasks in this order with no time limit.
3.4. Efficiency and Eye Movement Metrics
Based on the web maps and questionnaires, we obtained time data on the solution of each task and the eye movement recordings. Because all users performed the individual tasks correctly, we used parameters related to the time of task execution to assess efficiency. Therefore, the first time parameter was the time taken to solve the task, evaluated from displaying the map to the first click on the geolocation button (TC). It was measured for each task separately. For the second task, it was time to the first click on the search button, and for the third task, the time to the first click on the route button. We used the time to the first click because participants also had to enter the name of the place, and typing names is not directly related to GUI efficiency—it relates to participants’ ability to write speed on a computer. Mouse events, such as time to the first click, are often a part of web map assessment [44
An essential part of UX is eye movement assessment. Therefore, the first UX assessment parameter was the total number of fixations during the task (FC). This tells you how frequently the participant stopped his gaze on the map screen or any GUI element. A visualization that is visually more demanding often has more fixations in overall [45
]. The reason for the higher number of fixations may be users’ lower spatial abilities [46
]. However, in the study, we adopted homogeneous groups, so this factor does not influence the number of fixations. The next parameter is the number of fixations that appeared on the button used to complete the task (FCB). To obtain this parameter, it was necessary to specify the so-called areas of interest (AOI) [47
]. The ratio of FCB to FC allows for determining what percentage of attention the participant needed to finding and identifying the specific button, which is another parameter (ATT). Saccadic amplitude is another parameter that defines the user’s visual experience (SA). It is the angular distance of eye movement between fixations [48
]. Some studies claimed that the shorter the SA, the less effective the visual scanning [49
]. The last parameter defining visual UX is time to the first fixation (FFT) on the specific button.
4.1. GUI Efficiency
GUI efficiency was determined based on TC for two interfaces independently. The time needed to complete individual tasks was lower when using the OpenStreetMap-based GUI. For the first task, TC median was 8.5 s for the OpenStreetMap-based GUI and 11.7 s for the Google Maps-based GUI. However, the Mann–Whitney test did not show statistically significant differences (p > 0.05). A similar situation, but with a smaller difference, was observed with the second task, in which TC median resulted in 1.8 s for Google Maps-based GUI and 1.6 s for OpenStreetMap-based GUI (p > 0.05). The third task presents similar efficiency measured with TC—median 3.0 s for Google Maps-based GUI and 4.2 s for OpenStreetMap-based GUI (p > 0.05).
As the TC results of individual participants show, the first task was the most time-consuming in both interfaces (Figure 5
and Figure 6
). The Mann–Whitney test confirms the statistical significance of results in both GUIs (TC in Task 1 > TC in Task 2 p
< 0.05; TC in Task 1 > TC in Task 3 p
< 0.05). As for the differences in TC between tasks 2 and 3, only the OpenStreetMap-based GUI has a statistically significant difference (TC in Task 2 < TC in Task 3 p
We used the Spearman correlation test to study the relationship between TC and participants’ preferences. We found several statistically significant correlations (p < 0.05). As for the first task, which was to find your location using the geolocation button, there was a substantial relationship between the frequency of using Google Maps and TC on Google Maps-based GUI (r = −0.54). It means that the more often participants use Google Maps, the faster they perform the first task while using the Google Maps-based GUI. In the second task, the frequency of using OpenStreetMap correlated with TC while using the OpenStreetMap-based GUI (r = 0.51). This means that the more often participants used OpenStreetMap, the more time they needed to complete the task with the OpenStreetMap-based GUI. In the third task, there was a similar correlation as in the first task. The more often participants used Google Maps, the faster they performed tasks based on the Google Maps-based GUI (r = −0.56). It means that the more often participants use Google Maps, the faster they perform the third task using the Google Maps-based GUI. The correlation between the frequency of using OpenStreetMap and the time to complete the third task based on the Google Maps-based GUI is similar (r = −0.68).
4.2. Users’ Visual Experience
Despite the similarities in the effectiveness of both tested GUIs, the study participants showed differences in experiencing interactions. These differences were determined based on eye movement analysis. The first parameter is the number of fixations (FC). In the first task, the median was different for the Google Maps-based GUI—36 fixations—and for the OpenStreetMap-based GUI—38. Although the Mann–Whitney test did not show statistically significant differences (p
> 0.05), the spatial distribution of fixations indicates different places of the participants’ gaze concentration. This mainly applies to the first task (Figure 7
). Google Maps-based GUI users searched for the geolocation button in three corners of the screen, while OpenStreetMap-based GUI users only searched for it in two, as shown by heatmaps. In the second task, the median FC on the Google Maps-based GUI was 34, while on the OpenStreetMap-based GUI, it was only 28. Although the Mann–Whitney test did not show statistical significance, the recorded distribution of fixations shows that, as in the first task, Google Maps-based GUI users were looking for a ‘search’ button in three corners of the screen. In the third task, the median FC is very similar. The Google Maps-based GUI median was 51 fixations, and the OpenStreetMaps-based GUI median had 54 fixations. Here, too, Google Maps-based GUI users searched for a ‘route’ button in three corners. The main similarity in UX when working with the interface is that the participants visually analyze all the corners where the buttons are, regardless of the task performed. The Google Maps-based GUI requires participants to analyze one corner more, compared to OpenStreetMaps-based GUI.
The number of fixations on the searched button (FCB) is another visual UX assessment parameter along with the percentage of attention paid to it (ATT). In the first task, the average FCB for both GUIs was 3. However, concerning FC, it gives 4.8% attention to the geolocation button in the Google Maps-based GUI and 9.3% attention while using the OpenStreetMap-based GUI. In the second task, the average ATT is at a similar level—35.5% (13 FCB) when using the Google Maps-based GUI and 40.2% (11 FCB) while using the OpenStreetMaps-based GUI. The longer interaction process associated with typing names caused an increased ATT level. In the third task, FCB for two interfaces was 2, which translated into 4.5% ATT using the Google Maps-based GUI and 3.5% ATT using the OpenStreetMap-based GUI, respectively. This task also requires a more sustained interaction. While the route search extends the name entry field, we did not obtain fixations from the area in which the participant entered the name.
The median saccadic amplitude (SA) in the first task on the Google Maps-based GUI was 6.50°, while on the OpenStreetMap-based GUI, it was 6.16°. In the second task, SA on the Google Maps-based GUI was 5.73°, and that on the OpenStreetMap-based GUI median was 5.53°. In the third task, SA on the Google Maps-based GUI was 4.59°, and on the OpenStreetMap median it was 4.90° on average. We did not found statistically significant differences using the Mann–Whitney test. Spearman’s correlation showed that the more FC, the shorter SA (r = −0.47; p < 0.05) in the first task while using the Google Maps-based GUI. We found statistically significant differences (p < 0.05) with the Mann–Whitney test in FFT during the first task. For the Google Maps-based GUI, the FFT median was 7.8 s, and for OpenStreetMap-based GUI, FFT was 4.6 s. In the other two tasks, the Mann–Whitney test showed no statistical significance; however, the FFT on Google Maps-based GUI was slightly longer (for Task 2, the median was 0.5 s, and for Task 3 it was 1.2 s) comparing to OpenStreetMap-based GUI (Task 2—median was 0.4 s and Task 3—0.8 s).
Based on the test results presented, we confirm Wang’s [27
] conclusions that different GUIs may cause different user experience. This is especially evident in the visual experience called the user’s visual strategy described by eye movement metrics [30
]. Depending on the GUI, the spatial distribution of fixations confirms this conclusion. In each task, participants observed three corners in the Google Maps-based GUI. However, in the OpenStreetMap-based GUI, only two corners were observed. This agrees with the claim of some researchers about the crucial button placement in the GUI layout [9
]. More corners of the screen with buttons can translate into a longer cognitive process. However, as the correlation results show, the frequency of using a given map service has a high impact on the efficiency of the GUI. Wang [27
] claimed that the preference of individual map services leads to the separation of users. However, the frequency of use (preference) of a given web map does not translate into generally better performance while using any GUI. Users perform more efficiently if they work with their preferred GUI. Using a preferred GUI of a multimedia map reinforces user habits [7
]. However, participants who preferred OpenStreetMap were less efficient while using the OpenStreetMap-based GUI. It can also be a tip for designers of web multimedia maps to introduce GUI changes gradually, as Google does [50
]. As noted by Horbiński and Cybulski [7
], the GUI buttons of web mapping services are characterized only by pictograms. However, as noted by Muehlenhaus [51
], button symbols are built on certain conventions that influence UX and efficiency.
Using the GUI for specific tasks on the map generates specific visual experiences. As the results in Figure 7
show, the participants investigate all the corners where the buttons occur. In this context, Nivala et al. [35
] suggested that the interface on the map should be simple. Users examine all corners but not all buttons. According to this observation, we conclude that users analyze the GUI layout in search of the target button. They do not analyze all buttons in turn. On this basis, we can also recommend a guide for interactive map designers. The idea is to design groups of buttons with similar functions and place them together. In our opinion, this is a factor that contributes to increasing the efficiency of a web map, which is part of multimedia cartography [52
]. This is also consistent with the principle presented by Shneiderman and Plaisant [53
]—striving for consistency.
Map interactivity is a crucial element of multimedia cartography on the internet. The methodology of evaluating the visual user experience presented in the study has provided interesting conclusions about the performance of the graphical user interface. The results show the differences in experiencing interaction while using two different GUIs. The main difference in eye movement between Google Maps-based GUI and OpenStreetMap-based GUI is the visual analysis of a higher number of corners with buttons. There are also differences in time parameters such as the time to first fixation, fixation count, and saccadic amplitude. However, not all of the presented relationships showed statistical significance. This may be due to a small research sample. We think that increasing the number of participants could have resulted in more definite differences. This is true, especially since some test results showed p < 0.10.
Like any study, we rely on experimental simplifications. The two GUIs compared prove the importance of button placement in the UX. Of course, we see the possibility of developing our research on GUIs based on other web mapping services, e.g., Baidu Maps, Yandex Maps, Bing Maps, Map Quest, and HERE Maps. We can also include the GUI arrangement according to user preferences in future analysis. There are several significant lessons to be learned from extending this comparison. Firstly, if we examine the higher the number of GUIs based on different web maps, the more meaningful the results will be. Secondly, this can lead to more recommendations of proper GUI design that are desirable in GUI design. Third, it could help to exclude the least effective solutions. Fourthly, it would help in better understanding the UX when using the GUI. Among other things, one could answer the question of which solutions increase distraction and which are intuitive and supportive for users.
Eye-tracking combined, with a questionnaire, proved to be an effective method to obtain data on GUI efficiency and users’ UX. In future studies, we see the possibility of adaptation of this methodology to study the GUI of web maps on mobile devices such as smartphones. It is also possible to examine UX while performing tasks with an animated map interface. This type of interface has additional buttons (sliders) responsible for changing the time [54
]. It is also possible to use this methodology to study GUI on maps in augmented reality [56
The use of eye-tracking brought surprising conclusions regarding UX. The participants of the study observe all the corners with buttons in each of the tasks. More corners with buttons generally result in a longer and more complex view path. This translates to eye-tracking metrics that define differences in UX. A larger number of corners with buttons also slightly reduces the performance of the web map. Studies have shown that the arrangement of GUI buttons according to the OpenStreetMap layout was more efficient when performing tasks. We noticed that participants who preferred Google Maps were better when working on an interface based on Google Maps. This allows us to believe that users who work with the web map interface are largely guided by their habits. However, the most problematic task was to find the geolocation button in both GUIs.
Observing all the corners (not all buttons) to find the appropriate button prompts us to recommend arranging buttons with similar interactive functionality together (searching, routing). Another recommendation is to arrange the buttons in the lowest number of screen corners. Since the participant visually examines all of the screen corners, the more corners a display uses, the more complex the scanning path.