Evaluating Geovisualizations Based on Open Data: An Integrative Framework for Engagement, Openness, and Accessibility
Abstract
1. Introduction
2. Related Work: Evaluating Geovisualizations in the Open Data Context
2.1. Open Data Reuse and Public Engagement
2.2. Geovisualizations as Public-Facing Data Interfaces
2.3. Existing Approaches to Evaluating Geovisualizations
2.4. Towards Integrated Evaluation Frameworks
3. Dimensional Components of Geovisualizations Based on Open Data
3.1. Cartographic Representation
3.2. Interaction and Engagement Affordances
3.3. Openness
3.4. Accessibility
3.5. Contextual Characteristics
4. An Integrative Framework for Evaluating Geovisualizations
5. Materials and Methods
5.1. Research Design
5.2. Scope and Research Questions
5.3. Sampling Strategy and Case Selection
5.4. Operationalization and Coding Procedure
5.5. Methodological Scope and Limitations
6. Results and Discussion: Configurations of Geovisualizations Based on Open Data
6.1. Contextual Configurations: Who Produces What, and at Which Scale
6.2. Representation–Interaction Configurations: Conventional Cartography as the Exploratory Default
6.3. Engagement-Oriented Configurations: Exploration Without Analysis
6.4. Openness–Accessibility Imbalance: Transparency Without Inclusiveness
6.5. Cross-Dimensional Competing Priorities and Design Implications
6.6. Linking Identified Configurations to Analytical Outcomes
7. Conclusions
Policy and Design Recommendations
- Move beyond “moderate interactivity” as a default design choice. Current practice is dominated by exploratory interfaces with medium levels of interactivity that support basic data inspection but rarely support more advanced analytical reasoning or knowledge generation. Designers and public institutions should move beyond this “safe middle ground” by deliberately aligning interaction complexity with intended user groups. Geovisualizations that support guided, narrative-driven exploration and are associated with lower levels of interactivity may be more appropriate for non-expert users, where excessive interface complexity can increase cognitive load and reduce usability. In contrast, higher levels of analytical interactivity—including comparative tools, scenario simulations, querying functionalities, or temporal analysis—may be more suitable for data-literate users and experts. Such functionalities are particularly relevant in planning, monitoring, and decision-support contexts that require deeper exploratory and analytical capabilities.
- Embed openness within the design process, instead of limiting it to metadata statements or licensing notes. The frequent absence or ambiguity of licensing information substantially limits reuse, even when datasets are technically open. Open data policies should explicitly require that both datasets and resulting geovisualizations carry clear, standard open licenses and provide transparent documentation of data sources and update practices. From a design perspective, openness should be operationalized through visible download, sharing, and reuse affordances that actively encourage downstream use rather than merely permitting it.
- Make accessibility a fundamental design principle, not an afterthought. The low level of compliance with web accessibility standards reveals a critical misalignment between the inclusive ambitions of open data initiatives and actual geovisualization practices. Accessibility requirements—such as sufficient color contrast, color-blind-safe palettes, keyboard navigation, screen-reader compatibility, and multilingual support—should be integrated from the earliest design stages. Policymakers and funding bodies should treat accessibility compliance as a minimum requirement for publicly funded geovisualizations rather than an optional feature.
- Adopt configuration-aware evaluation and procurement practices. Rather than evaluating geovisualizations through isolated checklists of features, institutions should assess them as socio-technical configurations that balance engagement, reuse potential, and inclusiveness. The analytical framework proposed in this study offers a practical tool for such evaluation and can inform commissioning, benchmarking, and quality assurance processes. Applying configuration-aware assessment criteria can help prevent common competing priorities, such as highly engaging yet legally non-reusable or formally open but inaccessible visualizations.
Supplementary Materials
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
Abbreviations
| OGD | Open Government Data |
| NGO | Non-Governmental Organizations |
| GIS | Geographic Information System |
| ML | Machine Learning |
| API | Application Programming Interfaces |
| DOI | Digital Object Identifiers |
| HCI | Human Computer Interaction |
| FAQ | Frequently Asked Question |
| AI | Artificial intelligence |
| WCAG | Web Content Accessibility Guidelines |
| UN | United Nations |
| DHMZ | Croatian National Meteorological Service |
| EUMETSAT | European Organization for the Exploitation of Meteorological Satellites |
| GeoAI | Geospatial Artificial Intelligence |
Appendix A
| Name and Author | Data Source(s) | Description | Key Features |
|---|---|---|---|
| 1. Radar Composite (DHMZ) [75] | Open meteorological datasets and radar observations provided through DHMZ XML services under the Open Licence of the Republic of Croatia. | A national-scale meteorological radar visualization showing cloud cover and precipitation intensity across Croatia. | Dynamic radar imagery updated every 15 min, sequential precipitation color scale, animated weather monitoring, and zoom/pan navigation. |
| 2. Satellite Slider (RAMMB) [82] | The SLIDER primarily displays real-time and archived satellite imagery derived from NOAA’s GOES-R Series (and JMA’s Himawari-8/9 satellites, along with data from polar-orbiting satellites like SNPP | An interactive satellite visualization platform for exploring near real-time atmospheric and weather conditions. | Temporal sliders, multi-band spectral filtering, advanced spatial navigation, layer comparison, near real-time satellite imagery. |
| 3. EUMETView (EUMETSAT) [76] | Data from EUMETSAT’s satellites and Copernicus Sentinel missions. | An online map service providing near real-time and historical visualization of meteorological and Earth observation data | Multi-layer selection; interactive legend; animation control, temporal filtering, data download/API options. |
| 4. De Olho nas Metas (Rede Nossa São Paulo) [88] | São Paulo municipal open government and policy monitoring datasets. | The visualization tracks São Paulo’s municipal goals and projects, showing progress on public policies and initiatives to promote transparency and civic engagement. | Sectoral filtering, city-level spatial indicators, and integrated progress charts. |
| 5. Alberta Major Projects (Government of Alberta) [83] | Government of Alberta infrastructure and economic development open datasets. | A geovisualization that provides an interactive inventory of major infrastructure and development projects across the province of Alberta, Canada. | Interactive project exploration, searchable project database, project filtering, and metadata pop-ups. |
| 6. An Interactive Visualization of NYC Street Trees (Cloudred) [85] | Open meteorological datasets and radar observations provided through DHMZ XML services under the Open Licence of the Republic of Croatia. | A national-scale meteorological radar visualization showing cloud cover and precipitation intensity across Croatia. | Dynamic radar imagery updated every 15 min, sequential precipitation color scale, animated weather monitoring, and zoom/pan navigation. |
| 7. Find Local Cherry Trees (DDOT’s Urban Forestry Division) [89] | The primary data source is the District Department of Transportation (DDOT) Urban Forestry Division’s street tree database. | A geovisualization designed to help users locate cherry trees, particularly during the blooming season, in Washington, D.C. | Location-based search, interactive attribute pop-ups, and seasonal exploration. |
| 8. Building Ages in The Netherlands (Parallel) [80] | BAG (Basisregistratie Adressen en Gebouwen) building registry data combined with the Dutch National Height Model (AHN) and 3D building datasets developed by TU Delft. | An interactive map showing the construction year and urban characteristics of over 11 million buildings across the Netherlands. | Interactive 3D visualization, color-coded building ages, and temporal exploration of construction periods. |
| 9. How Old Are the Buildings in St. Petersburg? (Nikita Slavin) [81] | Multiple open geospatial and historical datasets, including Rosreestr open cadastral data, OpenStreetMap, Saint Petersburg Open Data Portal, Ministry of Culture open data, WikiMapia, and CityWalls.ru. | An interactive map visualizing the age of buildings in St. Petersburg. | Color-coded building ages, searchable addresses, and building information pop-ups. |
| 10. Traffic Simulation in Plzeň (City of Plzeň) [77] | Digitized traffic information from the National Traffic Information Centre (NDIC) combined with municipal traffic datasets used by the City of Plzeň | An interactive geovisualization platform for monitoring and simulating traffic conditions in the city of Plzeň. | Interactive traffic simulation, visualization of traffic intensity and average speeds, temporal navigation, and scenario-based traffic analysis. |
| 11. Swimplaces (Mapotic) [90] | A global community-oriented platform for discovering and reviewing swimming locations worldwide. | Crowdsourced swimming location data submitted through Swimplaces platform and bathing water quality data from the European Environment Agency (EEA). | User reviews, search functionality, mobile-responsive interface, and interactive exploration of swimming places |
| 12. Traffic Accidents in Flanders (Flemish Government) [91] | Flemish Government traffic accident open datasets | An interactive geovisualization of traffic accidents in the Flanders region of Belgium. | Interactive accident filtering, spatial and temporal exploration of accident patterns, summary statistics visualization. |
| 13. CityScale (CityScale) [92] | Ukrainian municipal and national open datasets, including cadastral, environmental, crime, health, education, transport, and statistical data provided by governmental institutions and public agencies. | A platform visualizing socio-economic, environmental, transport, and public service indicators across Ukrainian cities. | Interactive thematic map, comparative indicator visualization, and address-level search. |
| 14. City of Edinburgh Parking Areas and Bays [78] | Open datasets from The City of Edinburgh Council, including parking bay types and parking price data for different city zones. | An interactive map showing the location, type, and operational details of parking bays across the city of Edinburgh. | Information-focused, layer toggling for parking types, location search. |
| 15. Payroll Employment Locations in Boston [84] | It uses U.S. Census Bureau LEHD (Longitudinal Employer-Household Dynamics) Origin-Destination Employment Statistics (LODES) datasets. | A geovisualization of employment distribution and workforce statistics across Boston. | Interactive dashboard interface, spatial filtering, linked chart visualizations. |
| 16. Healthy throughout Life (Federal Government of Germany) [86] | National statistical and quality-of-life indicator datasets developed within the German federal government initiative “Gut leben in Deutschland”. | A narrative visualization exploring quality-of-life and health indicators across Germany. | Interactive thematic visualization, integrated explanatory text, and comparative exploration of regional quality-of-life indicators. |
| 17. Atlas of Economic Complexity (Harvard Growth Lab) [87] | It uses multiple international trade and economic datasets, including data from the United Nations Comtrade database, International Monetary Fund (IMF), and Federal Reserve Economic Data (FRED). | An interactive platform designed to help users explore international trade flows, export structures, productive capabilities, and the economic complexity of countries. | Thematic switching between trade indicators, attribute filtering by country, product, and year, and comparative exploration of global trade flows. |
| 18. eBird (Cornell Lab of Ornithology) [93] | The geovisualization uses the eBird Basic Dataset (EBD), a global crowdsourced bird observation dataset managed by the Cornell Lab of Ornithology, combined with environmental datasets from NASA, NOAA, and USGS. | A biodiversity platform that uses statistical models and machine learning to visualize bird migration, abundance patterns, and species distributions | Animated migration visualization, species filtering, and interactive exploration of abundance patterns. |
| 19. Global Forest Watch (World Resources Institute) [66] | It uses multiple forest-monitoring datasets, including Global Forest Watch data, University of Maryland GLAD lab datasets, Landsat satellite imagery, and data provided by Google, USGS, and NASA. | A global-scale web-based geovisualization platform designed to monitor forest change and support environmental governance and public awareness. | Interactive map with temporal sliders, data filtering, real-time updates, and analytical tools such as forest loss detection and anomaly alerts; supports data download and API access. |
| 20. Identify Your Watershed and Sewer System (District of Columbia Government) [94] | It uses open datasets from the District of Columbia Open Data Portal and USGS Watershed Boundary Dataset (WBD), including watershed boundaries, hydrologic unit codes, and sewer infrastructure data. | A municipal environmental map identifying watershed boundaries and sewer system types in Washington, D.C. | Address-based search, base map switching, bookmarks, infrastructure-related information pop-ups. |
| 21. Beescape (Penn State University) [70] | It uses various environmental, climatic, agricultural, and land-cover datasets, including the USDA-NASS Cropland Data Layer (CDL), PRISM Climate Group climate data, and U.S. Census of Agriculture datasets. | A web-based environmental geovisualization designed to assess landscape suitability and habitat quality for bees and other pollinators. | Layer toggling, thematic switching between pollinator habitat indicators, interactive legends, spatial queries, statistical summaries of habitat quality, and calculation of pollinator-related agricultural value. |
| 22. Coronavirus in the U.S.: Latest Map and Case Count (The New York Times) [67] | The geovisualization uses COVID-19 datasets collected from U.S. state and local health departments and public health reporting agencies. | A national public-health geovisualization tracking COVID-19 cases, deaths, and hospitalizations across the United States. | Interactive maps and charts, temporal trend visualization, county-level data exploration, and comparative analysis of COVID-19 indicators. |
| 23. European Air Quality Index (European Environment Agency) [68] | It uses near real-time air-quality data from European monitoring stations collected by the European Environment Agency (EEA), supplemented by Copernicus Atmosphere Monitoring Service (CAMS) data. | A near real-time interactive geovisualization of air quality conditions across Europe based on monitoring station data and pollutant measurements. | Thematic switching between air-quality indicators and pollutants, attribute filtering, multiple coordinated map and chart views, time-slider and animation controls for temporal exploration, attribute calculations, and comparative analytical tools. |
| 24. Open Data Impact Map (Center for Open Data Enterprise) [95] | It uses datasets from various open data research initiatives, crowdsourced surveys, regional supporter contributions, and publicly available organizational case studies collected by the Center for Open Data Enterprise and the Open Data for Development Network (OD4D). | A global scale map documenting how organizations use open data across sectors and regions. | Interactive pop-ups, legend interactivity, thematic filtering by country and sector and display of organization-related information. |
| 25. NYC Taxi Trips (Kepler.gl) [71] | The geovisualization uses New York City Taxi and Limousine Commission (TLC) open taxi trip datasets. | A geovisualization of taxi trip pick-ups and drop-offs in New York City designed for exploration and analysis of large-scale urban mobility data. | Coordinated map views, temporal filtering with time-slider controls, interactive legends, advanced trip-data queries, statistical summaries, and comparative analysis of taxi mobility patterns. |
| 26. Afghanistan: Visualising the Impact of 20 Years of War (Al Jazeera) [69] | It uses datasets from international organizations and projects, including Brown University’s Costs of War project, UN humanitarian data, and publicly available socio-economic and conflict datasets related to Afghanistan. | A narrative geovisualization examining the humanitarian, social, and economic impacts of prolonged conflict in Afghanistan. | Scrollytelling narrative structure, temporal exploration, and comparative visualization of conflict impacts. |
| Indicator | Description | Codes (Values) |
|---|---|---|
| Map Title | The map has a clear title | 0 = no, 1 = yes |
| Title | The primary, visible title of the map | Text (String) |
| Subtitle/Tagline | The secondary title or short descriptive text | Text (String)/N/A |
| Thematic Domain | The primary subject matter of the map | 1 = Environment, 2 = Transport, 3 = Health, 4 = Demographics, 5 = Economy,6 = Housing, 7 = Public Services |
| Map Theme | A summary of the map’s subject matter | Text (String) |
| Author | The primary organization or individual responsible | Text (String) |
| Author Type | The primary category of the creator | 1 = Government; 2 = Local Government; 3 = Agency/Public Inst; 4 = Academic; 5 = Media/Journalism; 6 = NGO/Civil Society; 7 = Commercial |
| Year of Launch | The year of the map’s first publication | Year (YYYY) |
| Intended Audience | The declared or inferred target audience | Multi-select 1 = General Public; 2 = Decision-makers 3 = Experts; 4 = Media 5 = Education; 6 = Other |
| Geographic Scope | The geographical area the map covers | 1 = Global; 2 = Continental 3 = National; 4 = Regional 5 = City/Local + Text (Name of area) |
| Geographic Scope | Name | Text (String) |
| About/Info page | Checks for a page/section with project details | 0 = no, 1 = yes |
| Indicator | Description | Codes (Values) |
|---|---|---|
| Visualization Type | Assesses if the map’s content is static or dynamic | 1 = Static; 2 = Dynamic |
| Mapping Technique | The primary cartographic method(s) used | Choropleth Map; Symbol Map; Proportional Symbol Map; Flow Map; Isoline Map; Cartogram; Dot Density Map; Heat Map |
| Multiple Techniques | Checks if multiple techniques are used simultaneously | 0 = no; 1 = yes |
| Thematic Category | Classifies the primary subject matter of the map into a broad thematic category. | 1 = Physical-geographical; 2 = Socio-economic; 3 = Technical |
| Classification Method | Identifies the data classification method used. | 1 = Natural Breaks (Jenks); 2 = Quantile; 3 = Equal Interval; 4 = Standard Deviation; 5 = Custom (Manual) |
| Legend Design | Assesses the presence and interactivity of the map legend. | 0 = no; 1 = yes (basic, static); 2 = yes (advanced/interactive) |
| Legend/Symbol Consistency | Assesses the consistency between map and legend symbols. | 0 = no; 1 = yes |
| Scale-Dependence | Assesses if the visual representation of data changes dynamically with the map scale | 0 = no (scale is fixed); 1 = yes |
| Labeling Strategy | Assesses whether a clear and systematic strategy is used for placing and displaying labels | 0 = no; 1 = yes |
| Projection Information | Assesses whether the cartographic projection used is explicitly stated to the user | 0 = no; 1 = yes |
| Interactivity Type | Indicator | Description | Scale/Codes |
|---|---|---|---|
| Spatial Navigation | Pan & zoom | Users can pan the map in all directions and zoom using scroll, +/− buttons, or pinch gestures. | 0 = no, 1 = yes |
| Rotation & Tilt | Map can be rotated around its Z-axis and perspective can be tilted. | 0 = no, 1 = yes | |
| Full 3D Environment | Maps enable a 3D scene including buildings, terrain, or other objects. | 0 = no, 1 = partial, 2 = full 3D | |
| Mini-map/Overview | Additional overview map or extent frame shows current view context. | 0 = no, 1 = yes | |
| Bookmarks/Presets | Predefined positions or bookmarks for quick navigation. | 0 = no, 1 = yes | |
| Thematic Navigation | Layer Toggling | User can turn thematic data layers on/off. | 0 = no, 1 = yes |
| Attribute Filtering | User can filter data based on attributes. | 0 = no, 1 = yes | |
| Thematic Switching | Switch between different thematic views (e.g., Indicator A vs. B). | 0 = no, 1 = yes | |
| Linked/Coordinated Views | Multiple panels (e.g., map + chart) update synchronously. | 0 = no, 1 = yes | |
| Symbology Selection | Choose between different symbols or layer styles. | 0 = no, 1 = yes | |
| Temporal Navigation | Time Slider | Slider control to browse data through time. | 0 = no, 1 = yes |
| Animation Controls | Play, pause, and replay temporal animations. | 0 = no, 1 = yes | |
| Temporal Filtering | Filter data by specific time range. | 0 = no, 1 = yes | |
| Real-time Data Stream | Connected to live external data sources. | 0 = no, 1 = yes | |
| Temporal Comparison | Compare two time periods (e.g., before/after). | 0 = no, 1 = yes | |
| Display & Classification Interactivity | Pop-ups/Hover Info | Clicking or hovering shows feature attributes. | 0 = no, 1 = yes |
| Legend Interactivity | Users interact with legends to filter or toggle data. | 0 = no, 1 = yes | |
| Symbology Adjustment | Change symbols (color, size, shape). | 0 = no, 1 = yes | |
| Classification Adjustment | Change data classification method or class breaks. | 0 = no, 1 = manual, 2 = advanced | |
| Basemap Switching | Change base map style (e.g., satellite, topographic). | 0 = no, 1 = yes | |
| Layer Transparency Control | Adjust transparency of layers. | 0 = no, 1 = yes | |
| Label Toggling | Turn labels or annotations on/off. | 0 = no, 1 = yes | |
| Custom Color Schemes | Select or define custom color palettes. | 0 = no, 1 = yes | |
| Analytical Interactivity | Data Queries | Filter or query data interactively. | 0 = no, 1 = basic, 2 = advanced |
| Statistical Summaries | Show basic stats (count, mean, distribution). | 0 = no, 1 = yes | |
| Interactive Highlighting | Selecting in one view highlights another. | 0 = no, 1 = yes | |
| Download Analysis Results | Export analysis results or filtered data. | 0 = no, 1 = yes | |
| Spatial Analysis Tools | Tools like buffer, hotspot, nearest neighbor. | 0 = no, 1 = yes | |
| Attribute Calculation | Generate new attributes (sums, ratios). | 0 = no, 1 = yes | |
| Comparative Analysis | Compare datasets, layers, or time periods. | 0 = no, 1 = yes | |
| Narrative Interactivity | Text & Multimedia Integration | Explanatory text, images, video, or media integrated. | 0 = no, 1 = partial, 2 = integrated |
| Step-by-step Guided Tour | Structured guide walks through visualization. | 0 = no, 1 = yes | |
| Scrollytelling | Maps react dynamically to user scrolling. | 0 = no, 1 = yes | |
| Annotation and Highlighting | Labels, arrows, drawings direct attention. | 0 = no, 1 = yes | |
| Narrative Sequencing | Sequential storytelling structure. | 0 = no, 1 = yes | |
| Interactive Storytelling Controls | User controls story flow (skip, choose scenarios). | 0 = no, 1 = yes | |
| Contextual Triggers | Text or buttons trigger map changes. | 0 = no, 1 = yes | |
| AI & Advanced Analytics | Natural Language Query | Interact using text or voice commands. | 0 = no, 1 = yes |
| Automated (AI) Map Generation | Map generated automatically from data. | 0 = no, 1 = yes | |
| Pattern Detection | Detects clusters, anomalies, spatial patterns. | 0 = no, 1 = yes | |
| Predictive Modeling | Includes future scenarios or simulations. | 0 = no, 1 = yes | |
| Multi-agent Simulation | Shows dynamic interactions of entities. | 0 = no, 1 = yes | |
| ML/Optimization | Uses ML or optimization for styling/insights. | 0 = no, 1 = yes | |
| Recommendation Systems | Suggests relevant layers or analyses. | 0 = no, 1 = yes | |
| Automated anomaly alerts | The system automatically alerts to significant changes or anomalies in real time | 0 = no, 1 = yes |
| Indicator | Description | Scale/Codes |
|---|---|---|
| Engagement Use | Defines whether the visualization is intended for private or public use. | 1 = Private; 2 = Public |
| Engagement Purpose | Indicates the main purpose of engagement with the visualization. | 1 = Presentation 2 = Exploration; 3 = Revelation/Knowledge Generation |
| Engagement Interactivity Level | Reflects the degree of interactivity provided to the user. | 1 = Low; 2 = Medium; 3 = High |
| Engagement Potential | Represents the overall engaging quality or ability of the visualization to attract and sustain user attention. | 0 = Not engaging; 1 = Engaging; 2 = Highly engaging |
| Map Functionality | Refers to the dominant functional type of the map. | 1 = Exploratory; 2 = Narrative; 3 = Analytical |
| Indicator | Description | Scale/Codes |
|---|---|---|
| Reuse License (Visualization) | Clearly stated license for the visualization itself | 0 = Unknown; 1 = Descriptive; 2 = Standard |
| Reuse License (Data) | Clearly stated license for the dataset(s) used. | 0 = Unknown; 1 = Mixed; 2 = Standard open license; 3 = Restrictive |
| Data Download | Ability to download the underlying data. | 0 = No; 1 = Yes (tabular); 2 = Yes (geospatial, e.g., GeoJSON/Shapefile); 3 = Yes (via API); 4 = Multiple formats/options |
| Data Source (Citation) | Transparent citation or reference to the source dataset(s). | 0 = No; 1 = Yes (in-text reference); 2 = Yes (hyperlink); 3 = Citable (DOI) |
| Share/Embed Options | Options for sharing or embedding the visualization. | 0 = No; 1 = Share link; 2 = Embed code; 3 = Social sharing options |
| DOI/Identifier | The existence of a DOI or internal version identifier | 0 = no; 1 = yes; +Tekst (ID) |
| Terms/Privacy Policy | Accessible page with terms of use and/or privacy information. | 0 = No; 1 = Yes |
| Contact/Maintainer | Way to contact the maintainer or author (email, form, GitHub issues, etc.). | 0 = No; 1 = Email; 2 = Web form; 3 = Repository issues; 4 = Multiple options |
| Open Repository | Source code or data hosted in an open repository (e.g., GitHub, GitLab, Zenodo). | 0 = No; 1 = Yes (link) |
| Versioning/Changelog | Presence of changelog or version history for the visualization/data. | 0 = No; 1 = Yes |
| Last Update Date | Date of most recent update (of visualization or dataset). | Text (YYYY-MM-DD or N/A) |
| Cookie/Tracking Transparency | Clear information about cookies or tracking use and purpose. | 0 = No; 1 = Yes |
| Methodology Details | Description of the data processing or analytical methods used. | 0 = No; 1 = Brief; 2 = Detailed |
| Update Frequency | Declared frequency of data or visualization updates. | 0 = Unknown; 1 = Real-time; 2 = Daily; 3 = Weekly; 4 = Monthly; 5 = Ad hoc |
| Indicator | Description | Scale/Codes |
|---|---|---|
| Language(s) | Primary interface language used | Text (ISO language code) |
| Multi-lingual Toggle | Ability to switch interface language. | 0 = No; 1 = Yes |
| Interface Languages | Multi-language availability and switching options | 0 = One language; 1 = ≥2 languages (manual toggle); 2 = ≥2 languages (systematic, localized) |
| Color Contrast | Text–background contrast meets the minimum WCAG 2.1 AA standard | 0 = Not assessed; 1 = Obvious problems; 2 = Passes |
| Color-blind Schemes | Use color blind friendly schemes | 0 = No; 1 = Yes |
| Text Size/Scaling | Text can be enlarged (up to 200%) without breaking layout or UI. | 0 = No; 1 = Yes |
| Responsive Design (Mobile) | Interface adapts to narrow screens (mobile-friendly layout). | 0 = No; 1 = Yes |
| Alternative Text (Alt) | Alt text for key images/icons and “empty alt” for decorative ones. | 0 = No; 1 = Partially; 2 = Systematically applied |
| Screen Reader Accessibility | Navigation and map control elements have accessible names/roles. | 0 = No; 1 = Partially; 2 = Systematically implemented |
| Keyboard Navigation | Tab/Shift + Tab can focus controls; interface usable without a mouse. | 0 = No; 1 = Partially; 2 = Fully accessible |
| Focus Visible | Clear “focus” indicators are visible on interactive elements. | 0 = No; 1 = Yes |
| Help/FAQ | Presence of a help or FAQ section to support users. | 0 = No; 1 = Yes |
References
- Purwanto, A.; Zuiderwijk, A.; Janssen, M. Citizen engagement with open government data: Lessons learned from Indonesia’s presidential election. Transform. Gov. People Process Policy 2020, 14, 1–30. [Google Scholar] [CrossRef]
- Attard, J.; Orlandi, F.; Scerri, S.; Auer, S. A systematic review of open government data initiatives. Gov. Inf. Q. 2015, 32, 399–418. [Google Scholar] [CrossRef]
- Cahlikova, T.; Mabillard, V. Open data and transparency: Opportunities and challenges in the Swiss context. Public Perform. Manag. Rev. 2020, 43, 662–686. [Google Scholar] [CrossRef]
- Nikiforova, A.; McBride, K. Open government data portal usability: A user-centred usability analysis of 41 open government data portals. Telemat. Inform. 2021, 58, 101539. [Google Scholar] [CrossRef]
- Natvig, M.K.; Jiang, S.; Stav, E. Using open data for digital innovation: Barriers for use and recommendations for publishers. JeDEM 2021, 13, 28–57. [Google Scholar] [CrossRef]
- Zuiderwijk, A.; Janssen, M. Open data policies, their implementation and impact: A framework for comparison. Gov. Inf. Q. 2014, 31, 17–29. [Google Scholar] [CrossRef]
- Zuiderwijk, A.; Shinde, R.; Janssen, M. Investigating the attainment of open government data objectives: Is there a mismatch between objectives and results? Int. Rev. Adm. Sci. 2019, 85, 645–672. [Google Scholar] [CrossRef]
- Dietrich, D. The role of civic tech communities in PSI reuse and open data policies. Eur. Public Sect. Inf. Platf. 2015, 5, 2015. [Google Scholar]
- Janssen, K. The influence of the PSI directive on open government data: An overview of recent developments. Gov. Inf. Q. 2011, 28, 446–456. [Google Scholar] [CrossRef]
- Gunther, W.A.; Mehrizi, M.H.R.; Huysman, M.; Feldberg, F. Debating big data: A literature review on realizing value from big data. J. Strat. Inf. Syst. 2017, 26, 191–209. [Google Scholar] [CrossRef]
- Mutambik, I.; Almuqrin, A.; Lee, J.; Zhang, J.Z.; Alomran, A.; Omar, T.; Floos, A.; Homadi, A. Usability of the G7 open government data portals and lessons learned. Sustainability 2021, 13, 13740. [Google Scholar] [CrossRef]
- Ansari, B.; Barati, M.; Martin, E.G. Enhancing the usability and usefulness of open government data: A comprehensive review of the state of open government data visualization research. Gov. Inf. Q. 2022, 39, 101657. [Google Scholar] [CrossRef]
- Safarov, I.; Meijer, A.; Grimmelikhuijsen, S. Utilization of open government data: A systematic literature review of types, conditions, effects and users. Inf. Polity 2017, 22, 1–24. [Google Scholar] [CrossRef]
- Chokki, A.P.; Simonofski, A.; Frénay, B.; Vanderose, B. Engaging citizens with open government data: The value of dashboards compared to individual visualizations. Digit. Gov. Res. Pract. 2022, 3, 21. [Google Scholar] [CrossRef]
- Bohman, S. Civic participation and empowerment through visualization. In Proceedings of the SIGRAD 2015, Stockholm, Sweden, 1–2 June 2015. [Google Scholar]
- Eberhardt, A.; Silveira, M.S. Show me the data! A systematic mapping on open government data visualization. In Proceedings of the 19th Annual International Conference on Digital Government Research: Governance in the Data Age, Delft, The Netherlands, 30 May–1 June 2018; pp. 1–10. [Google Scholar]
- Park, S.; Gil-Garcia, J.R. Open data innovation: Visualizations and process redesign as a way to bridge the transparency-accountability gap. Gov. Inf. Q. 2022, 39, 101456. [Google Scholar] [CrossRef]
- Quarati, A.; De Martino, M.; Rosim, S. Geospatial open data usage and metadata quality. ISPRS Int. J. Geo-Inf. 2021, 10, 30. [Google Scholar] [CrossRef]
- Koua, E.L.; Kraak, M.-J. Geovisualization to support the exploration of large health and demographic survey data. Int. J. Health Geogr. 2004, 3, 12. [Google Scholar] [CrossRef]
- Fleta-Asín, J.; Muñoz, F.; Sáenz-Royo, C. A methodological approach for enhancing visualization of country data representation in the presence of significant spatial disparity. MethodsX 2024, 13, 102833. [Google Scholar] [CrossRef]
- Dilk, G.; Behr, F.-J. Democratization of mapping: Conceptual design and implementation of a multi-channel, multi-purpose participatory mapping program. Int. J. Disaster Stud. Clim. Resil. 2025, 1, 55. [Google Scholar] [CrossRef]
- Korycka-Skorupa, J.; Nowacki, T. Cartographic presentation—From simple to complex map. Misc. Geogr. 2019, 23, 16–22. [Google Scholar] [CrossRef]
- Orozco Frutos, G.; Díaz-Cuevas, P. Developing an open-source tool for collaborative mapping: Process, challenges and solutions. GeoFocus Int. Rev. Geogr. Inf. Sci. Technol. 2025, 36, 1–24. [Google Scholar]
- Degbelo, A.; Kauppinen, T. Increasing transparency through web maps. In Proceedings of the Web Conference 2018 Companion, Lyon, France, 23–27 April 2018; pp. 899–904. [Google Scholar]
- Degbelo, A.; Kray, C. Intelligent geovisualizations for open government data (vision paper). In Proceedings of the ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, Seattle, WA, USA, 6–9 November 2018; pp. 77–80. [Google Scholar]
- Marzouki, A.; Lafrance, F.; Daniel, S.; Mellouli, S. The relevance of geovisualization in citizen participation processes. In Proceedings of the 18th Annual International Conference on Digital Government Research, Staten Island, NY, USA, 7–9 June 2017; pp. 397–406. [Google Scholar]
- Smith, D.A. Online interactive thematic mapping: Applications and techniques for socio-economic research. Comput. Environ. Urban Syst. 2016, 57, 106–117. [Google Scholar] [CrossRef]
- Cybulski, P.; Medyńska-Gulij, B.; Horbiński, T. Users’ visual experience during temporal navigation in forecast weather maps on mobile devices. J. Geovisualization Spat. Anal. 2023, 7, 32. [Google Scholar] [CrossRef]
- Roth, R.E.; Ross, K.S.; MacEachren, A.M. User-centered design for interactive maps: A case study in crime analysis. ISPRS Int. J. Geo-Inf. 2015, 4, 262–301. [Google Scholar] [CrossRef]
- Roth, R.E. An empirically-derived taxonomy of interaction primitives for interactive cartography and geovisualization. IEEE Trans. Vis. Comput. Graph. 2013, 19, 2356–2365. [Google Scholar] [CrossRef] [PubMed]
- Brewer, C.A.; Pickle, L. Evaluation of methods for classifying epidemiological data on choropleth maps in series. Ann. Assoc. Am. Geogr. 2002, 92, 662–681. [Google Scholar] [CrossRef]
- Fabrikant, S.I.; Hespanha, S.R.; Hegarty, M. Cognitively inspired and perceptually salient graphic displays for efficient spatial inference making. Ann. Assoc. Am. Geogr. 2010, 100, 13–29. [Google Scholar] [CrossRef]
- Zhu, X.; Thomas, C.; Moore, J.; Allen, S. Open government data licensing: An analysis of the U.S. state open government data portals. In Electronic Government and the Information Systems Perspective; Toeppe, K., Yan, H., Chu, S.K.W., Eds.; Springer: Cham, Switzerland, 2021; pp. 260–273. [Google Scholar]
- Chirumavilla, V. Digital citizenship for all: Accessibility requirements in public domain websites. J. Comput. Sci. Technol. Stud. 2025, 7, 273–282. [Google Scholar] [CrossRef]
- Mohamad, Y.; Paternò, F.; Pulina, F.; Sorrentino, C.; Hine, G.; Yesilada, Y. Requirements for large scale web accessibility evaluation. In Computers Helping People with Special Needs; Miesenberger, K., Manduchi, R., Penaz, P., Eds.; Springer: Cham, Switzerland, 2020; pp. 275–283. [Google Scholar]
- Janssen, M.; Charalabidis, Y.; Zuiderwijk, A. Benefits, adoption barriers and myths of open data and open government. Inf. Syst. Manag. 2012, 29, 258–268. [Google Scholar] [CrossRef]
- Weerakkody, V.; Irani, Z.; Kapoor, K.; Sivarajah, U.; Dwivedi, Y.K. Open data and its usability: An empirical view from the citizen’s perspective. Inf. Syst. Front. 2017, 19, 285–300. [Google Scholar] [CrossRef]
- Smith, G.; Sandberg, J. Barriers to innovating with open government data: Exploring experiences across service phases and user types. Inf. Polity 2018, 23, 249–265. [Google Scholar] [CrossRef]
- Perovich, L.; Wylie, S.; Bongiovanni, R. Chemicals in the creek: Designing a situated data physicalization of open government data with the community. IEEE Trans. Vis. Comput. Graph. 2021, 27, 913–923. [Google Scholar] [CrossRef]
- Hossain, M.A.; Dwivedi, Y.K.; Rana, N.P. State-of-the-art in open data research: Insights from existing literature and a research agenda. J. Organ. Comput. Electron. Commer. 2016, 26, 14–40. [Google Scholar] [CrossRef]
- Shaharudin, A.; Van Loenen, B.; Janssen, M. Towards a common definition of open data intermediaries. Digit. Gov. Res. Pract. 2023, 4, 6. [Google Scholar] [CrossRef]
- Graves, A.; Hendler, J. A study on the use of visualizations for open government data. Inf. Polity 2014, 19, 73–91. [Google Scholar] [CrossRef]
- Hagen, L.; Keller, T.E.; Yerden, X.; Luna-Reyes, L.F. Open data visualizations and analytics as tools for policy-making. Gov. Inf. Q. 2019, 36, 101387. [Google Scholar] [CrossRef]
- Baker, J.; Jones, D.; Burkman, J. Using visual representations of data to enhance sensemaking in data exploration tasks. J. Assoc. Inf. Syst. 2009, 10, 533–559. [Google Scholar] [CrossRef]
- Çöltekin, A.; Griffin, A.L.; Slingsby, A.; Robinson, A.C.; Christophe, S.; Rautenbach, V.; Chen, M.; Pettit, C.; Klippel, A. Geospatial information visualization and extended reality displays. In Manual of Digital Earth; Springer: Singapore, 2020; pp. 229–277. [Google Scholar]
- MacEachren, A.M. Visualization in modern cartography: Setting the agenda. In Visualization in Modern Cartography; MacEachren, A.M., Taylor, D.R.F., Eds.; Pergamon: Oxford, UK, 1994; pp. 1–12. [Google Scholar]
- Kraak, M.J. Geovisualization. In International Encyclopedia of Human Geography; Kitchin, R., Thrift, N., Eds.; Elsevier: Oxford, UK, 2009; pp. 468–480. [Google Scholar]
- Tobon, C. Usability Testing for Improving Interactive Geovisualization Techniques; Working Paper; Centre for Advanced Spatial Analysis, University College London: London, UK, 2002. [Google Scholar]
- Herman, L.; Popelka, S.; Hejlova, V. Eye-tracking analysis of interactive 3D geovisualization. J. Eye Mov. Res. 2017, 10, 15. [Google Scholar] [CrossRef] [PubMed]
- Harding, J. Usability of geographic information—Factors identified from qualitative analysis of task-focused user interviews. Appl. Ergon. 2013, 44, 940–947. [Google Scholar] [CrossRef] [PubMed]
- Fish, C.S. Cartographic content analysis of compelling climate change communication. Cartogr. Geogr. Inf. Sci. 2020, 47, 492–507. [Google Scholar] [CrossRef]
- Muehlenhaus, I. Another Goode method: How to use quantitative content analysis to study variation in thematic map design. Cartogr. Perspect. 2011, 69, 29–40. [Google Scholar] [CrossRef]
- Zuo, C.; Wei, M.; Zhang, P.; Chen, C.; Meng, L. Assessing maps for social topic representation: A qualitative content analysis of maps for sustainable mobility. Int. J. Cartogr. 2025, 11, 458–476. [Google Scholar] [CrossRef]
- Çöltekin, A.; Bleisch, S.; Andrienko, G.; Dykes, J. Persistent challenges in geovisualization—A community perspective. Int. J. Cartogr. 2017, 3, 115–139. [Google Scholar] [CrossRef]
- Engebretsen, M.; Kennedy, H. (Eds.) Data Visualization in Society; Amsterdam University Press: Amsterdam, The Netherlands, 2020. [Google Scholar]
- Slocum, T.; Blok, C.; Jiang, B.; Koussoulakou, A.; Montello, D.; Fuhrmann, S.; Hedley, N. Cognitive and usability issues in geovisualization. Cartogr. Geogr. Inf. Sci. 2001, 28, 61–75. [Google Scholar] [CrossRef]
- Roth, R.E. Interactive maps: What we know and what we need to know. J. Spat. Inf. Sci. 2013, 6, 59–115. [Google Scholar] [CrossRef]
- Fairbairn, D.; Andrienko, G.; Andrienko, N.; Buziek, G.; Dykes, J. Representation and its relationship with cartographic visualization. Cartogr. Geogr. Inf. Sci. 2001, 28, 13–28. [Google Scholar] [CrossRef]
- Monmonier, M. Lying with maps. Stat. Sci. 2005, 20, 215–222. [Google Scholar] [CrossRef]
- Garlandini, S.; Fabrikant, S.I. Evaluating the effectiveness and efficiency of visual variables for geographic information visualization. In Spatial Information Theory; Hornsby, K., Claramunt, C., Denis, M., Ligozat, G., Eds.; Springer: Berlin/Heidelberg, Germany, 2009; Volume 5756, pp. 195–211. [Google Scholar]
- Harrower, M.; Brewer, C.A. ColorBrewer.org: An online tool for selecting colour schemes for maps. Cartogr. J. 2003, 40, 27–37. [Google Scholar] [CrossRef]
- Neumann, A. Spatial, Temporal and Thematic Navigation: Visualizing Biographies of European Artists Using a Task Oriented User Interface Design Approach. Ph.D. Thesis, ETH Zurich, Zurich, Switzerland, 2010. [Google Scholar]
- Crampton, J.W. Interactivity types in geographic visualization. Cartogr. Geogr. Inf. Sci. 2002, 29, 85–98. [Google Scholar] [CrossRef]
- Roth, R.E. Cartographic design as visual storytelling: Synthesis and review of map-based narratives, genres, and tropes. Cartogr. J. 2021, 58, 83–114. [Google Scholar] [CrossRef]
- Mai, G.; Xie, Y.; Jia, X.; Lao, N.; Rao, J.; Zhu, Q.; Liu, Z.; Chiang, Y.-Y.; Jiao, J. Towards the next generation of geospatial artificial intelligence. Int. J. Appl. Earth Obs. Geoinf. 2025, 136, 104368. [Google Scholar] [CrossRef]
- Global Forest Watch. Available online: https://www.globalforestwatch.org/map/ (accessed on 1 December 2025).
- Coronavirus in the U.S.: Latest Map and Case Count. The New York Times. Available online: https://www.nytimes.com/interactive/2021/us/covid-cases.html (accessed on 1 December 2025).
- European Environment Agency. European Air Quality Index. Available online: https://airindex.eea.europa.eu/AQI/index.html (accessed on 1 December 2025).
- Al Jazeera. Afghanistan: Visualising the Impact of 20 Years of War. Available online: https://interactive.aljazeera.com/aje/2021/afghanistan-visualising-impact-of-war/index.html (accessed on 1 December 2025).
- Beescape. Available online: https://beescape.psu.edu/ (accessed on 1 December 2025).
- Kepler.gl. NYC Taxi Trips. Available online: https://kepler.gl/demo/nyctrips (accessed on 1 December 2025).
- Jansen, B.J.; Guan, K.W.; Salminen, J.; Aldous, K.K.; Jung, S.-G. What Is User Engagement? A Systematic Review of 241 Research Articles in Human–Computer Interaction and Beyond. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI’25), New York, NY, USA, 26 April–1 May 2025; pp. 1–19. [Google Scholar]
- Peters, C.; Castellano, G.; De Freitas, S. An Exploration of User Engagement in HCI. In Proceedings of the International Workshop on Affective-Aware Virtual Agents and Social Robots (AFFINE 2009), Boston, MA, USA, 6 November 2009; pp. 1–3. [Google Scholar]
- Coolors Contrast Checker. Available online: https://coolors.co/contrast-checker (accessed on 1 December 2025).
- Državni Hidrometeorološki Zavod. Radarska Slika Kompozit. Available online: https://meteo.hr/podaci.php?section=podaci_mjerenja¶m=radari (accessed on 1 December 2025).
- EUMETSAT. EUMETView. Available online: https://view.eumetsat.int/productviewer (accessed on 1 December 2025).
- RoadTwin. Traffic Simulation in Plzeň. Available online: https://plzen.roadtwin.com/scenarios?mapmode=modelSTA&mapitem=9 (accessed on 1 December 2025).
- The City of Edinburgh Council. Find Parking Bays. Available online: https://www.edinburgh.gov.uk/parking-spaces/find-parking-bays (accessed on 1 December 2025).
- Guo, Y.; Zhang, Z. How Open Government Data Enhances Public Service Delivery: A Quasi-Natural Experiment from Government Data Platforms. Systems 2026, 14, 408. [Google Scholar] [CrossRef]
- Parallel. Building Age Netherlands. Available online: https://netherlands.parallel.co.uk/ (accessed on 1 December 2025).
- Kontiki Maps. How Old Are the Buildings in St. Petersburg? Available online: https://kontikimaps.com/how-old/saint-p (accessed on 1 December 2025).
- RAMMB Slider Satellite Imagery Viewer. Available online: https://slider.cira.colostate.edu/ (accessed on 1 December 2025).
- Government of Alberta. Alberta Major Projects. Available online: https://majorprojects.alberta.ca/#/ (accessed on 1 December 2025).
- Payroll Employment Locations. City of Boston. Available online: https://boston.maps.arcgis.com/apps/dashboards/1af39de1500b4acfb2dd7bb32feb2dd1 (accessed on 1 December 2025).
- Cloudred. NYC Street Trees. Available online: https://www.cloudred.com/labprojects/nyctrees/index.php?year=2015 (accessed on 1 December 2025).
- Healthy Throughout Life. Available online: https://www.gut-leben-in-deutschland.de/report/health/ (accessed on 1 December 2025).
- Harvard Growth Lab. The Atlas of Economic Complexity. Available online: https://atlas.hks.harvard.edu/explore/geomap (accessed on 1 December 2025).
- Rede Nossa São Paulo. De Olho Nas Metas (Overview of São Paulo’s Goals). Available online: https://2017.deolhonasmetas.org.br/ (accessed on 1 December 2025).
- District of Columbia Geographic Information System (DCGIS). Find Local Cherry Trees. Available online: https://dcgis.maps.arcgis.com/apps/Nearby/index.html?appid=11cadab2140e475f89ddf5162f2d4108 (accessed on 1 December 2025).
- Swimplaces. Available online: https://www.swimplaces.com/ (accessed on 1 December 2025).
- Innoconnect. Accidents Flanders. Available online: https://accidentsflanders.innoconnect.net/ (accessed on 1 December 2025).
- CityScale. Available online: https://www.cityscale.com.ua/en/index.htm (accessed on 1 December 2025).
- Cornell Lab of Ornithology. eBird Status and Trends: Brant Abundance Map. Available online: https://science.ebird.org/en/status-and-trends/species/brant/abundance-map (accessed on 1 December 2025).
- Identify Your Watershed and Sewer System Area Web App. Available online: https://www.arcgis.com/home/item.html?id=57e0b80b693a4522a69d1397d392d7e8 (accessed on 1 December 2025).
- Open Data Impact Map—Europe and Central Asia Visualizations. Available online: https://opendataimpactmap.org/eca#visualizations (accessed on 1 December 2025).








| Dimension ↓/Outcome → | Engagement Potential | Reuse | Inclusiveness |
|---|---|---|---|
| Context & Scope * | ● | ● | ● |
| Cartographic Representation | ●● | ● | ●●● |
| Interaction & Engagement Affordances | ●●● | ● | ● |
| Openness | ● | ●●● | ● |
| Accessibility | ● | ● | ●●● |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Published by MDPI on behalf of the International Society for Photogrammetry and Remote Sensing. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.
Share and Cite
Miletić, A.; Divjak, A.K. Evaluating Geovisualizations Based on Open Data: An Integrative Framework for Engagement, Openness, and Accessibility. ISPRS Int. J. Geo-Inf. 2026, 15, 259. https://doi.org/10.3390/ijgi15060259
Miletić A, Divjak AK. Evaluating Geovisualizations Based on Open Data: An Integrative Framework for Engagement, Openness, and Accessibility. ISPRS International Journal of Geo-Information. 2026; 15(6):259. https://doi.org/10.3390/ijgi15060259
Chicago/Turabian StyleMiletić, Andrea, and Ana Kuveždić Divjak. 2026. "Evaluating Geovisualizations Based on Open Data: An Integrative Framework for Engagement, Openness, and Accessibility" ISPRS International Journal of Geo-Information 15, no. 6: 259. https://doi.org/10.3390/ijgi15060259
APA StyleMiletić, A., & Divjak, A. K. (2026). Evaluating Geovisualizations Based on Open Data: An Integrative Framework for Engagement, Openness, and Accessibility. ISPRS International Journal of Geo-Information, 15(6), 259. https://doi.org/10.3390/ijgi15060259

