# Using Eye Tracking to Evaluate the Usability of Flow Maps

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

- To indicate flows, do straight lines and curves influence the usability of a flow map?
- To represent flow volume, do line thicknesses and color gradients have different impacts on the usability of a flow map?

## 2. Related Work

## 3. Methods

#### 3.1. Experimental Design

#### 3.2. Participants

#### 3.3. Apparatus

#### 3.4. Materials

#### 3.5. Procedure

- Q1: Among the outflows from city A, to which city does the flow have the largest (smallest) volume?
- Q2: Among the inflows to city A, from which city does the flow have the largest (smallest) volume?
- Q3: The largest (smallest) data flow occurs between which two cities?

#### 3.6. Analysis Indices

## 4. Results

#### 4.1. Straight Lines and Curves

#### 4.2. Line Thicknesses and Color Gradients

#### 4.3. Questionnaires

## 5. Discussion

#### 5.1. Comparison between Straight Lines and Curves

#### 5.2. Comparison between the Line Thickness and the Color Gradient

## 6. Conclusions and Future Work

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## Appendix A

**Figure A1.**Samples of maps with AOIs. SL = straight line; CV = curve; LT = line thickness; and CG = color gradient. (

**a**) SL + LT; (

**b**) CV + LT; (

**c**) SL + CG; and (

**d**) CV + CG.

## References

- Jenny, B.; Stephen, D.M.; Muehlenhaus, I.; Marston, B.E.; Sharma, R.; Zhang, E.; Jenny, H. Design principles for origin-destination flow maps. Cartogr. Geogr. Inf. Sci.
**2018**, 45, 62–75. [Google Scholar] [CrossRef] - Boyandin, I.; Bertini, E.; Lalanne, D. Using flow maps to explore migrations over time. In Proceedings of the Geospatial Visual Analytics Workshop in Conjunction with the 13th AGILE International Conference on Geographic Information Science, Guimaraes, Portugal, 11–14 May 2010. [Google Scholar]
- Grandjean, M. Connected World: Untangling the Air Traffic Network. Available online: http://www.martingrandjean.ch/connected-world-air-traffic-network/ (accessed on 20 July 2018).
- Balcan, D.; Colizza, V.; Gonçalves, B.; Hu, H.; Ramasco, J.J.; Vespignani, A. Multiscale mobility networks and the spatial spreading of infectious diseases. Proc. Nat. Acad. Sci. USA
**2009**, 106, 21484–21489. [Google Scholar] [CrossRef] [PubMed][Green Version] - Guo, D. Flow mapping and multivariate visualization of large spatial interaction data. IEEE Trans. Vis. Comput. Gr.
**2009**, 15. [Google Scholar] [CrossRef] - Padmanabhan, A.; Wang, S.; Cao, G.; Hwang, M.; Zhao, Y.; Zhang, Z.; Gao, Y. Flumapper: An interactive cybergis environment for massive location-based social media data analysis. In Proceedings of the Conference on Extreme Science and Engineering Discovery Environment: Gateway to Discovery, San Diego, CA, USA, 22–25 July 2013; p. 33. [Google Scholar]
- Jenny, B.; Stephen, D.M.; Muehlenhaus, I.; Marston, B.E.; Sharma, R.; Zhang, E.; Jenny, H. Force-directed layout of origin-destination flow maps. Int. J. Geogr. Inf. Sci.
**2017**, 31, 1521–1540. [Google Scholar] [CrossRef] - Zhou, H.; Xu, P.; Yuan, X.; Qu, H. Edge bundling in information visualization. Tsinghua Sci. Technol.
**2013**, 18, 145–156. [Google Scholar] [CrossRef] - Vrotsou, K.; Fuchs, G.; Andrienko, N.; Andrienko, G. An interactive approach for exploration of flows through direction-based filtering. J. Geovisual. Spat. Anal.
**2017**, 1. [Google Scholar] [CrossRef] - Rae, A. Flow-data analysis with geographical information systems: A visual approach. Environ. Plan. B Plan. Des.
**2011**, 38, 776–794. [Google Scholar] [CrossRef] - Koylu, C.; Guo, D. Design and evaluation of line symbolizations for origin—Destination flow maps. Inf. Vis.
**2017**, 16, 309–331. [Google Scholar] [CrossRef] - Bertin, J. Graphics and Graphic Information Processing; Walter de Gruyter: Berlin, Germany, 1981. [Google Scholar]
- Wolfe, J.M.; Horowitz, T.S. What attributes guide the deployment of visual attention and how do they do it? Nat. Rev. Neuros.
**2004**, 5, 495–501. [Google Scholar] [CrossRef] [PubMed] - Dong, W.; Ran, J.; Wang, J. Effectiveness and efficiency of map symbols for dynamic geographic information visualization. Cartogr. Geogr. Inf. Sci.
**2012**, 39, 98–106. [Google Scholar] [CrossRef] - Garlandini, S.; Fabrikant, S.I. Evaluating the effectiveness and efficiency of visual variables for geographic information visualization. In Proceedings of the International Conference on Spatial Information Theory, Aber Wrach, France, 21–25 September 2009; Springer: Berlin, Germany, 2009; pp. 195–211. [Google Scholar]
- Kubíček, P.; Šašinka, Č.; Stachoň, Z.; Štěrba, Z.; Apeltauer, J.; Urbánek, T. Cartographic design and usability of visual variables for linear features. Cartogr. J.
**2017**, 54, 91–102. [Google Scholar] [CrossRef] - Holten, D.; Isenberg, P.; Van Wijk, J.J.; Fekete, J.-D. An extended evaluation of the readability of tapered, animated, and textured directed-edge representations in node-link graphs. In Proceedings of the 2011 IEEE Pacific Visualization Symposium (PacificVis), Hong Kong, China, 1–4 March 2011; IEEE: Piscataway, NJ, USA, 2011; pp. 195–202. [Google Scholar]
- Holten, D.; van Wijk, J.J. A user study on visualizing directed edges in graphs. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Boston, MA, USA, 4–9 April 2009; pp. 2299–2308. [Google Scholar]
- Imhof, E. Thematische Kartographie; Walter de Gruyter: Berlin, Germany, 1972; Volume 10. [Google Scholar]
- Dent, B.D.; Torguson, J.S.; Hodler, T.W. Cartography: Thematic Map Design; WCB/McGraw-Hill Boston: Boston, MA, USA, 1999; Volume 5. [Google Scholar]
- Xu, K.; Rooney, C.; Passmore, P.; Ham, D.-H.; Nguyen, P.H. A user study on curved edges in graph visualization. IEEE Trans. Vis. Comput. Gr.
**2012**, 18, 2449–2456. [Google Scholar] [CrossRef] [PubMed] - Purchase, H.C.; Hamer, J.; Nöllenburg, M.; Kobourov, S.G. On the usability of lombardi graph drawings. In Proceedings of the International Symposium on Graph Drawing, Redmond, WA, USA, 19–21 September 2012; Springer: Berlin, Germany, 2012; pp. 451–462. [Google Scholar]
- Steinke, T.R. Eye movement studies in cartography and related fields. Cartogr. Int. J. Geogr. Inf. Geovisual.
**1987**, 24, 40–73. [Google Scholar] [CrossRef] - Reichenbacher, T.; Swienty, O. Attention-guiding geovisualization. In Proceedings of the 10th AGILE International Conference on Geographic Information Science, Aalborg, Denmark, 8–11 May 2007; Aalborg University: Aalborg, Denmark, 2007. [Google Scholar]
- Webb, N.; Renshaw, T. Eyetracking in hci. Res. Methods Hum. Comput. Interact.
**2008**, 35–69. [Google Scholar] - Çöltekin, A.; Fabrikant, S.I.; Lacayo, M. Exploring the efficiency of users’ visual analytics strategies based on sequence analysis of eye movement recordings. Int. J. Geogr. Inf. Sci.
**2010**, 24, 1559–1575. [Google Scholar] [CrossRef] - Çöltekin, A.; Heil, B.; Garlandini, S.; Fabrikant, S.I. Evaluating the effectiveness of interactive map interface designs: A case study integrating usability metrics with eye-movement analysis. Cartogr. Geogr. Inf. Sci.
**2009**, 36, 5–17. [Google Scholar] [CrossRef] - Ooms, K.; De Maeyer, P.; Fack, V.; Van Assche, E.; Witlox, F. Interpreting maps through the eyes of expert and novice users. Int. J. Geogr. Inf. Sci.
**2012**, 26, 1773–1788. [Google Scholar] [CrossRef][Green Version] - Ooms, K.; Maeyer, P.D.; Fack, V.; Assche, E.V.; Witlox, F. Investigating the effectiveness of an efficient label placement method using eye movement data. Cartogr. J.
**2012**, 49, 234–246. [Google Scholar] [CrossRef] - Dong, W.; Liao, H.; Roth, R.E.; Wang, S. Eye tracking to explore the potential of enhanced imagery basemaps in web mapping. Cartogr. J.
**2014**, 51, 313–329. [Google Scholar] [CrossRef] - Popelka, S.; Brychtova, A. Eye-tracking study on different perception of 2 d and 3 d terrain visualisation. Cartogr. J.
**2013**, 50, 240–246. [Google Scholar] [CrossRef] - Liao, H.; Wang, X.; Dong, W.; Meng, L. Measuring the influence of map label density on perceived complexity: A user study using eye tracking. Cartogr. Geogr. Inf. Sci.
**2018**. Available online: https://www.tandfonline.com/doi/full/10.1080/15230406.2018.1434016 (accessed on 20 July 2018). [CrossRef] - Alaçam, Ö.; Dalcı, M. A usability study of webmaps with eye tracking tool: The effects of iconic representation of information. In Proceedings of the International Conference on Human-Computer Interaction, San Diego, CA, USA, 19–24 July 2009; Springer: Berlin, Germany; pp. 12–21. [Google Scholar]
- Dong, W.; Liao, H.; Xu, F.; Liu, Z.; Zhang, S. Using eye tracking to evaluate the usability of animated maps. Sci. China Earth Sci.
**2014**, 57, 512–522. [Google Scholar] [CrossRef] - Ho, H.Y.; Yeh, I.C.; Lai, Y.C.; Lin, W.C.; Cherng, F.Y. Evaluating 2d flow visualization using eye tracking. Comput. Gr. Forum
**2015**, 34, 501–510. [Google Scholar] [CrossRef] - Blascheck, T.; Burch, M.; Meisel, T.; Schneider, T.; Mumin, S. Exploring eye movements with node-link graph layouts. In Eye Tracking for Spatial Research, Proceedings of the 3rd International Workshop, Zurich, Switzerland, 14 January 2018; ETH Zurich: Zurich, Switzerland, 2018. [Google Scholar]
- Netzel, R.; Burch, M.; Weiskopf, D. Comparative eye tracking study on node-link visualizations of trajectories. IEEE Trans. Vis. Comput. Gr.
**2014**, 20, 2221–2230. [Google Scholar] [CrossRef] [PubMed] - Goldberg, J.H.; Kotval, X.P. Computer interface evaluation using eye movements: Methods and constructs. Int. J. Ind. Ergon.
**1999**, 24, 631–645. [Google Scholar] [CrossRef] - Byrne, M.D.; Anderson, J.R.; Douglass, S.; Matessa, M. Eye tracking the visual search of click-down menus. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Pittsburgh, PA, USA, 15–20 May 1999; pp. 402–409. [Google Scholar]

**Figure 1.**Samples of the experimental materials generated with dataset 1 (Table 1). The cities shown in the maps include Hulunbuir, Qiqihar, Harbin, Mudanjiang, Beijing, Dalian, and Shanghai (not all cities that appear in the four datasets are displayed in the sample maps). SL = straight line; CV = curve; LT = line thickness; and CG = color gradient. (

**a**) SL + LT; (

**b**) CV + LT; (

**c**) SL + CG; and (

**d**) CV + CG.

**Figure 2.**Group setting and material quantities in the experiment. SL = straight line; CV = curve; LT = line thickness; and CG = color gradient; M is the number of material flow maps; N is the number of participants.

**Figure 3.**Statistics for the different line shapes and Mann-Whitney U test results (* p < 0.05, ** p < 0.01 and NS. = not significant). (

**a**) Accuracy; (

**b**) finish time; (

**c**) fixation count; (

**d**) percentage of fixations in AOIs; and (

**e**) time to first fixation.

**Figure 4.**Statistics for the two different representations of the flow volume and Mann-Whitney U test results (** p < 0.01 and NS = not significant). (

**a**) Accuracy; (

**b**) finish time; (

**c**) fixation count; (

**d**) percentage of fixations in AOIs; and (

**e**) time to first fixation.

**Figure 5.**Statistics for questionnaires (SL = straight line, CV = curve, LT = line thickness, and CG = color gradient). (

**a**) Line shape and (

**b**) flow volume.

**Figure 6.**Samples of gaze opacity maps (generated by the fixation count of all participants with a radius of 50 pixels in Tobii Studio). SL = straight line; CV = curve; LT = line thickness; and CG = color gradient. (

**a**) SL + LT; (

**b**) CV + LT; (

**c**) SL + CG; and (

**d**) CV + CG.

Line Shape | Material (Order) | Dataset | Question | Variable 1 | Variable 2 | |
---|---|---|---|---|---|---|

Group A & B | Straight line | Map 1 | D1 | Q2 | Dalian | Smallest |

Map 2 | D2 | Q1 | Harbin | Smallest | ||

Map 3 | D3 | Q2 | Harbin | Largest | ||

Map 4 | D4 | Q3 | - | Smallest | ||

Curve | Map 5 | D1 | Q2 | Beijing | Smallest | |

Map 6 | D2 | Q2 | Tianjin | Largest | ||

Map 7 | D3 | Q1 | Suihua | Largest | ||

Map 8 | D4 | Q1 | Dalian | Smallest |

Indices | Description | |
---|---|---|

Task indices | Accuracy (AC) | Number of task questions answered correctly |

Finish time (FT, s) | Average time used to complete each task (seconds) | |

Statistical eye tracking indices | Fixation count (FC) | Average fixation counts during each task |

Percentage of fixations in AOIs (PFA) | Number of fixations located within AOIs divided by all fixations | |

Time to first fixation (TtFF, s) | Duration from the beginning of the task to the first fixation located within the AOIs (seconds) |

Descriptive | Inferential | |||
---|---|---|---|---|

Straight Line | Curve | Mann-Whitney U Test | ||

M ± SD | M ± SD | U | p | |

Accuracy | 6.76 ± 0.62 | 7.43 ± 0.81 | −3.085 | 0.002 ** |

Finish time | 5.10 ± 2.68 | 7.58 ± 2.56 | −2.981 | 0.003 ** |

Fixation count | 25.99 ± 11.31 | 37.47 ± 9.36 | −3.283 | 0.001 ** |

Percentage of fixations in AOIs | 0.20 ± 0.07 | 0.26 ± 0.09 | −2.428 | 0.015 * |

Time to first fixation | 2.27 ± 0.69 | 2.07 ± 0.84 | 1.094 | 0.274 |

Descriptive | Inferential | |||
---|---|---|---|---|

Line Thickness | Color Gradient | Mann-Whitney U Test | ||

M ± SD | M ± SD | U | p | |

Accuracy | 6.75 ± 0.71 | 7.52 ± 0.60 | −3.341 | 0.001 ** |

Finish time | 6.50 ± 3.30 | 6.35 ± 3.46 | 0.183 | 0.855 |

Fixation count | 32.74 ± 12.89 | 31.28 ± 13.37 | 0.574 | 0.566 |

Percentage of fixations in AOIs | 0.23 ± 0.07 | 0.28 ± 0.08 | −1.956 | 0.050 |

Time to first fixation | 1.99 ± 0.56 | 1.96 ± 0.73 | 0.365 | 0.715 |

Comparison | Value | df | p |
---|---|---|---|

Straight line | 21.518 ^{a} | 3 | 0.000 ** |

Curve | |||

Line thickness | 93.845 ^{b} | 4 | 0.000 ** |

Color gradient |

^{a}0 cells (0.0%) have an expected count of less than 5. The minimum expected count is 35.50.

^{b}Two cells (20.0%, line thickness: very unclear and color gradient: very unclear) have an expected count of less than 5. The minimum expected count is 2.00. df = degree of freedom, ** p < 0.01.

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Dong, W.; Wang, S.; Chen, Y.; Meng, L. Using Eye Tracking to Evaluate the Usability of Flow Maps. *ISPRS Int. J. Geo-Inf.* **2018**, *7*, 281.
https://doi.org/10.3390/ijgi7070281

**AMA Style**

Dong W, Wang S, Chen Y, Meng L. Using Eye Tracking to Evaluate the Usability of Flow Maps. *ISPRS International Journal of Geo-Information*. 2018; 7(7):281.
https://doi.org/10.3390/ijgi7070281

**Chicago/Turabian Style**

Dong, Weihua, Shengkai Wang, Yizhou Chen, and Liqiu Meng. 2018. "Using Eye Tracking to Evaluate the Usability of Flow Maps" *ISPRS International Journal of Geo-Information* 7, no. 7: 281.
https://doi.org/10.3390/ijgi7070281