Since the introduction of modern Web maps (https://gistbok.ucgis.org/bok-topics/web-mapping
) that lets users zoom and pan around, there has been a relentless popularization of Web mapping [1
]. Nowadays, Web maps are used in many software systems to provide features ranging from trivial ones, such as helping users to enter addresses in mobile applications (e.g., Uber and Booking), to complex and rich dynamic maps such as the Earth Wind Map (https://earth.nullschool.net
), an animated map with wind predictions, Nine Point Five (http://www.ninepointfive.org
), a 3D map of earthquakes over the past 30 years or the COVID-19 Dashboard (https://coronavirus.jhu.edu/map.html
), a map of reported cases of coronavirus disease 2019 in real time. The enabler to accomplish these Web maps is the idea of decomposing large and complex maps into many small tiles
] that are downloaded, seamlessly joined together, and displayed on demand by a Web browser such as Google Chrome or Mozilla Firefox. As a consequence, this has a very positive impact on performance and user experience, since only tiles covering areas of the map that are visited are downloaded [3
], as depicted in Figure 1
One of the first Web mapping applications that popularized the idea of using tiles is Google Maps. Later, as the demand for more rich and interactive maps increased, developers designed methods for overlaying data over Web maps. Typical ways for doing so was to use a base map of some map provider such as Google Maps (https://www.google.com/maps
), HERE (https://www.here.com/
) or OSM (https://www.openstreetmap.org
), and then draw over this map additional information as markers, lines, or polygons. The practical impact of these technologies has been important, since they provided a simple way to enrich maps with geographic information from third parties [5
], making Web mapping applications more flexible and even more widespread [7
]. Accompanying this evolution, standard formats for representing and transferring geographic data for the Web emerged. Notable examples are GeoJSON [8
], an open standard format designed for representing simple geographical features, along with their non-spatial attributes, TopoJSON (https://github.com/topojson/topojson-specification
), a variant of GeoJSON that encodes topology stitching together shared line segments, and the Geography Markup Language (GML), an XML grammar for expressing geographical features [9
These advances have fundamentally reshaped the environment in which maps are used and produced [2
]. From the rich, but static maps made popular by Google Maps in 2005, to the fully-fledged Web-based GIS [1
] such as ArcGIS Online or CARTO, Web mapping technologies have significantly increased their quality and scope [10
]. Web maps have also evolved from static raster tiles to highly dynamic, customizable, and open vector tiles enabling a plethora of applications [7
], where multiple sources, including real-time geospatial data sources provided by sensors and crowd-sourcing are combined, collaboratively processed and analyzed, by relying on technologies such as Web Services [12
] and Big Data infrastructures [14
] provides interactivity and processing capabilities directly in the Web browser, also called client
. These technologies are the cornerstone of modern vector based mapping solutions, which were later sped up by WebGL [17
], a standard for rendering graphics that exploits the Graphics Processing Unit (GPU) available on modern personal computers and mobile phones. Vector-based maps and WebGL have moved map styling from the server to the Web browser, not only decreasing costs in server hardware, but also allowing advanced functionalities such as geoprocessing and visualization that were reserved to GIS software executing on high-end computers [10
As both end-users and developers take advantage of Web maps, there is an increasing pressure for increased quality, speed, dynamicity, and information integration that is only possible through development tools that ease the ever-increasing complexity of Web and mobile mapping. Modern Web maps are now created on-the-fly from the underlying geographic databases and their style and layout depend on the user’s preferences.
Several libraries emerged in the last few years to simplify the development of interactive maps in Web browsers. Examples of such libraries are Leaflet (https://leafletjs.com/
), Mapbox GL JS (https://github.com/mapbox/mapbox-gl-js
), and OpenLayers (https://openlayers.org/
). They offer simple ways to visualize and stack semi-transparent map layers from heterogeneous sources and map providers, including static raster and vector layers with different styling methods. Despite these solutions relying on trustworthy and mature open Web standards, the complexity of the technological stack involving Web browsers, operating systems, GPUs, and internal design characteristics of each library make their performance and network usage highly variable. Furthermore, since smartphones replace personal computers as the most used medium to access Web maps, the scenario becomes much more challenging considering that the diversity and fragmentation of mobile devices is huge.
Nowadays, as the advantages of vector tiles [18
] are slowly making them more widespread, developers have a hard time figuring out what library to use given their requirements. This is particularly important in scenarios with heterogeneous devices, including mobile devices with slow WebGL implementations and personal computers with fast GPUs. Now, the question is no longer only whether to use raster or vector maps, but also which library to use for rendering Web maps. While many other factors may be important, such as licensing costs, support for third party add-ons/maps and integration with mobile operating systems, in the rest of the paper, we focus on providing an initial answer to this question by empirically assessing the combination of raster/vector map layers and different devices using either of the three above mentioned libraries. Therefore, we compared several implementations of a Web map for displaying a Life Quality Index (LQI) designed for Argentina [19
]. This index was mapped at the level of census radius
(a higher level of territorial disaggregation than census tracts), thus there are more than fifty thousand individual areas for the whole country [20
]. For the comparison, we automatically executed common pan/zoom actions a typical user would, using several mobile devices (tablets and smartphones) ranging from low end to high end, and a mid-end personal computer. In these tests, we measured network usage and execution time. We also analyzed a strategy for reducing network usage that relies on lossy data compression techniques. We aim to provide developers with valuable information to guide their choices when selecting popular Web mapping libraries, with either raster/vector maps when executing on a variety of computing devices.
The rest of the paper is organized as follows. In Section 2
, we briefly describe the Life Quality Index for Argentina and discuss the main alternatives for displaying it as a Web map. In Section 3
describes the experimental setting and and analyzes the empirical results. Section 5
describes and discusses related efforts. Finally, Section 6
concludes the paper and proposes some guidelines for future research and development.
2. The Life Quality Index for Argentina
The work in [20
] proposed a calibrated compound social indicator of human welfare, LQI from now on that takes into account groups of socioeconomic and environmental indicators. This LQI has been successfully applied in many social and geographic studies in Argentina [19
]. The data associated with the LQI have been processed, analyzed, and applied through traditional GIS software targeted to geographers and social scientists. The LQI has been calculated and mapped at the level of census radius for 52,408 units. Each unit has an associated LQI represented as a decimal number in the range [0, 10], where the higher the value, the better the quality of life in the respective unit.
In an effort to make wider use of the LQI and reach a broad audience, we developed a Web mapping application (https://icv.netlify.app/
) to visualize the LQI data. One of the main requirements of the application was to support a variety of smartphones, even with slow and unstable 3G/4G data services, which are typical in rural areas in Argentina [24
]. The goal was to draw over a base map of Argentina a representation of the LQI values using some color scale [25
]. Figure 2
shows a region of Argentina with the LQI data overlayed as a choropleth map using a single-hue progression, where white areas have low LQI values, light green represents mid values and dark green have high values.
To build a Web mapping application for displaying the LQI data as a layer over a base map, we considered three main technological alternatives, which are summarized in Figure 3
Raster tiles: this alternative displays static images or bitmaps that are provided by a server. These images have a fixed resolution and would not allow for much interactivity, such as changing the color scale without reloading the map, or showing the LQI value under the mouse pointer. The hardware requirements for displaying this type of maps on a client Web browser would be low because there is no need for fast CPUs/GPUs for displaying the tiles. This would enable low-end smartphones access to the maps. However, raster tiles are known for requiring more network bandwidth than vector tiles [18
]. With respect to the server software, there are many mature and tested alternatives for serving raster tiles, either pre-rendered or rendered on demand [26
]. However, we considered that, regardless of the server software, the best choice would be to pre-render all the LQI map tiles as semi-transparent bitmaps in order to avoid server processing when the maps are being navigated by the users.
]. While this solution would be ideal in terms of interactivity, visualization quality, and decreased server costs, network usage would be high, as the whole LQI data file of 200 MB would have to be transferred through the Internet, even when viewing a small area of the map. In addition, the requirements for storing and displaying 52,408 polygons on a Web browser would be high, requiring high-end smartphones with powerful GPUs.
Vector tiles: the server provides vector objects that are rendered by the Web browser. This is similar to transferring raw data, but vector maps are tiled so that only the data, in this case tiles, required for the displayed area at a specific zoom level are transferred. In addition, typical data representation formats for vector tiles such as MBTiles or PBF (Protocol buffers) are very compact, binary-based, and usually compressed, in opposition to the very verbose and text-based GeoJSON/TopoJSON formats used for raw data. This alternative would mix the best of raster tiles and raw data. However, the technological stack and production of vector tiles are quite complex both on the server and on the Web browser, as we will see in the next sections. Furthermore, while this solution seems less CPU/GPU hungry, it is worth noting that, for displaying the whole country, the Web browser would have to render 52,408 polygons, without counting the base map, which may be too much processing for low-end smartphones.
3. The LQI Web Mapping Application
This section describes the main design decisions and technical aspects of the LQI Web application. We first describe the server side, which mainly consists of the provision of the LQI map layer. We then provide details on the development of the client Web application.
3.1. The LQI Layer Server
As discussed in Section 2
, we considered three alternatives for building the LQI map layer. Figure 4
depicts the process followed to build the three map alternatives. The original LQI dataset is stored as an ESRI ShapeFile, which was then converted to GeoJSON format by using MapShaper [29
]. In this conversion, we simplified some shapes in order to reduce the resulting file size. We will analyze the impact of simplification on map quality in Section 4
. The resulting GeoJSON file is stored on a Web Server for providing raw map data.
The GeoJSON file is then further simplified and converted to vector tiles and stored as a MBTiles database. For this purpose, we used Tippecanoe (https://github.com/mapbox/tippecanoe
), a tool for building vector tilesets. As a result, we produced a tiled vector map with 12 zoom levels (0–12). The minimum zoom level covers the whole country with the 52,408 polygons, while the maximum zoom level is detailed at the level of city blocks. The simplification performed during this stage only includes removing redundant or non-visible lines at a given zoom level. While the typical practice is to reduce map complexity by merging adjacent polygons, we did not do so because this would imply removing census radius and potentially mixing adjacent polygons with different LQI, which is clearly unacceptable. The resulting vector tiles are stored in a server. We used Tileserver PHP (https://github.com/maptiler/tileserver-php
), an implementation of the Web Map Tile Service (WMTS) (https://www.ogc.org/standards/wmts
) standard for serving vector tiles stored as MBTiles or pre-rendered raster map tiles. Vector tiles are transferred as Google protocol buffers (https://developers.google.com/protocol-buffers
), a platform-neutral, efficient, and extensible mechanism for serializing structured data. In addition, the server supports styling
maps using the standard Mapbox format (https://docs.mapbox.com/mapbox-gl-js/style-spec/
). A style defines the visual appearance of a map, including what data to draw, the order to draw it in, and how to paint the data when drawing it.
For producing raster tiles, we used as a starting point the vector tiles stored in Tileserver PHP (https://icv.conicet.gov.ar/tileserver-php/icv/
) along with a style to paint the polygons of the vector LQI map layer depending on the LQI value of each census radius. As this is a very time-consuming process performed at the server, but we are focused on the Web client, we pre-rendered the raster tiles of the entire LQI layer at all zoom levels. To do so, we developed a crawler that requests to the server all the tiles covering Argentina at all zoom levels. Each tile is saved as a PNG file, resulting in 4 GB of images for the LQI layer. Then, these images are stored in a server following the standard XYZ tiled map filenames.
3.2. The LQI Web Mapping Application
The three libraries are based on the idea of overlaying semi-transparent map layers that are combined and displayed on a Web browser. Each layer can be a tiled map or a raw data source. In turn, layers can be stored locally as files on the client or on a server. Finally, tiles can be either raster or vector.
As described in Section 2
, we considered three alternatives for implementing the LQI map layer:
Raw data: similarly to raster tiles, raw data in GeoJSON format are well supported in all libraries. The LQI data represented as a GeoJSON file occupy almost 200 MB, which is too much data even for a typical 3G/4G connection. Therefore, we tried reducing the size of the data, which is discussed in the next section.
Vector tiles: both OpenLayers and Mapbox GL JS provide native support for vector tiles. Leaflet, on the other hand, does not support vector tiles natively, but through the Leaflet—VectorGrid (https://github.com/Leaflet/Leaflet.VectorGrid
) third party plugin. In all cases, the implementation of the LQI layer involves styling the polygons during rendering. Since a map area may contain thousands of polygons that have to be painted with a color according to its LQI value, we carefully optimized this. Mapbox GL JS and OpenLayers rely on WebGL for displaying vector tiles, while Leaflet uses SVG. As from a user’s perspective WebGL or SVG rendered layers look the same, there are significant differences. First, WebGL requires a computer with a supported GPU. Fortunately, while even low-end smartphones have that, their performance may be poor. On the other hand, SVG is very well supported and can be accelerated by the GPU, depending on the Web browser and GPU model, but not to the degree of WebGL. This suggests that, while vector tiles can be probably displayed even on old/low-end smartphones, operations such as pan and zoom may be sluggish.
We began implementing the Web map by using a raster base layer with overlaid LQI raw data because this seemed the most straightforward alternative that is well supported on all libraries. However, this showed severe performance problems because, as discussed before, the LQI data represented as a GeoJSON file stresses out any typical 3G/4G connection. While this might be a viable alternative for some applications, in this case, even a marginal reduction on data size caused significant artifacts on the resulting map, which were unacceptable. This, in addition, the fact that in informal tests drawing the full GeoJSON data was painfully slow forced us to discard this alternative.
Then, we implemented six variants of the LQI Web mapping application using as a base layer a raster map, with either a raster or vector LQI layer. In terms of functionality, both types of base layers are equivalent. However, for the LQI layer, the vector based solution enables interaction so that it is possible, for example, to show the LQI value under the mouse pointer or customize the color scales for the LQI values without reloading the map. In addition, for Mapbox GL JS, we also implemented two variants with a vector base layer. We did not implement these variants with Leaflet/OpenLayers, since we found out that they provide limited or preliminary support for the complex vector map styles used in typical vector basemaps. Table 1
summarizes the combinations of library, basemap, and LQI layer considered for the empirical evaluations.
In the next section, we describe the experiments to assess the performance of the eight variants.
This section analyzes the performance of different implementations of the LQI mapping application. In addition, as raw data may require significant network bandwidth, we analyzed a strategy that relies on lossy data compression techniques [30
] for studying the impact of line simplification on the resulting map quality. Despite it being possible to compress the LQI vector tiles, we did not do so because the impact of such compression would be marginal considering that vector tiles are already compressed in protocol buffers using the lossless GZIP algorithm. As a consequence, as we will see in the results, vector tiles are already very network efficient.
The next section focuses on analyzing the performance of the different variants, while Section 4.2
assesses the compression ratio versus quality trade-off.
We compared eight different variants of the LQI web mapping application that combine raster and vector layers for the basemap and LQI map, implemented with the three mentioned libraries. These variants are called, for example, LeafletRV
, which stands for the application implemented with Leaflet using a Raster basemap and a Vector LQI layer. We measured network usage and time required for performing typical activities such as panning and zooming. We executed the applications on a variety of devices, including a PC, tablets, and smartphones ranging from low-end to high-end. Table 2
lists the devices used for the experiments in inverse order of performance. As shown, it includes a wide range of the most popular chipsets present on the market. For executing the Web mapping application, we used Google Chrome (version 80), the most used Web browser (https://gs.statcounter.com/browser-market-share/mobile/worldwide
To perform a fair comparison, we built a benchmark by using Google Puppeteer (https://pptr.dev/
), a tool to automate most things that users can do manually in a Web browser. In particular, we automated the actions required for panning and zooming to a location of the map and then waiting there until the map is fully rendered. This is repeated for six different locations of the map that include large cities (Avellaneda, La Plata, Palermo, Posadas, Retiro, Capital-San Juan). This benchmark was executed 10 times on each device connected through WiFi to a 10 Mbit/s Internet connection. In all cases, we used the default Web browser configuration, which includes caching enabled. The cache was cleaned before executing the benchmark for each variant.
shows the resulting average execution time measured in Seconds. The graphics x
-axis are ordered according to the device PassMark score (https://www.passmark.com/
), increasing from left to right. Figure 6
depicts a less detailed view of the same data, where blueish colors represent fast execution times. The time required for executing the benchmark was similar for most devices and LQI application variants except LeafletRV, which lagged behind all the others. This difference may be caused by the third party plugin used for supporting vector layers on Leaflet. For the rest of the variants, an interesting pattern emerges from the results. On low-end devices, raster layers (RR) clearly outperformed vector based variants, especially VR and VV. This is caused by the slow CPUs/GPUs of these devices, which have a hard time rendering the complex base map and its associated styles. Note that this tendency gradually changes as we move to the bottom of the figure, where mid and high-end devices are, and both vector and raster variants behave similarly in terms of performance. The exception is Leaflet in its both variants, which did not speed up as the others do on faster devices. This might be caused by the simple architecture of Leaflet, which does not rely on cutting edge Web technologies such as WebGL and Canvas, which are accelerated by GPUs and thus take advantage of powerful hardware available on mid/high-end devices.
shows the average network usage of the variants measured in Megabytes, while Figure 8
depicts a less detailed view of the same data, where blueish colors represent less network usage. We can observe that vector based implementations were the winners in terms of network efficiency, regardless of the devices. In particular, MapboxVV required significantly less network bandwidth than all the variants. In addition, more powerful devices tend to have bigger screens, so that variants using raster layers transferred more data. This explains the lower network usage of raster variants in low-end devices at the top of the figure. Leaflet used significantly more network bandwidth than the others on all devices. Again, Leaflet’s simple architecture and compatibility with all Web browsers impact negatively on efficiency.
In the reported results, it is rather difficult to clearly distinguish the best combination of layers and libraries in terms of both execution time and network utilization. In fact, there are several variants that ran fast on some devices but show high network utilization. For example, LeafletRR achieved competitive execution time on low/mid-end devices such as the Moto G5, G6, or Xiaomi A2 Lite, but doubled the network usage on these devices compared to the other variants. In order to better analyze the empirical data, Figure 9
summarizes the results using scatter plots of the generated network traffic (y
-axis) versus benchmark execution time (x
-axis), where each marker denotes a single execution of a variant. In other words, markers near the y
-axis are faster, while those near the x
-axis use less network bandwidth. We removed the three slowest devices and the PC (outliers in performance and network) from the figure to better appreciate the differences.
The scatter plots show that, when aiming for network efficiency, vector maps are clearly the winners, with MapboxVV is followed closely by MapboxVR. In these cases, the savings in network bandwidth for using rich and complex but very compact vector based basemaps explain the results. Moreover, while these variants double the time required to finish the benchmark on low-end devices, on mid-end devices such as the Moto G6 or Xiaomi A2 Lite, they are just a bit slower than raster based variants. For high-end devices, both Mapbox GL JS and OpenLayers, either using raster or vector maps, have shown similar execution times, but the savings in network bandwidth make vector based variants are preferable. Finally, the two Leaflet variants have struggled to perform competitively, showing poor execution time and high network usage across all devices.
4.2. LQI Vector Layer Simplification
Considering the LQI layer based on raw data in GeoJSON format is too large, we tried using lossy compression to reduce its size. The goal was to achieve an acceptable balance between data size and map quality. We rely on the MapShaper tool [29
] with the well known Visvalingam line simplification algorithm [30
]. This algorithm removes points with the least-perceptible change, which in practice means removing the vertex which together with its neighboring vertices form a triangle with the smallest area. We tuned it for preventing polygon features from disappearing. Furthermore, we did not merge adjacent polygons with different LQI values. We generated nine different GeoJSON files, retaining from 90% to 10% of the points of the original file. The resulting files sizes were from 126 MB to 35 MB, which are rather large for a typical mobile data connection. Then, we rendered the GeoJSON files as a map for five regions that have varied levels of details. All these regions contain a great number of polygons to properly assess the quality loss of applying the line simplification method.
To measure the quality of the simplified LQI map versus the original one, we used two sets of metrics:
traditional signal fidelity measures [31
] that compare the differences between corresponding pixels in the original and the compressed layer:
Mean Absolute Error (MAE): lower values of MAE indicates high similarity.
Peak Signal to Noise Ratio (PSNR): the higher the PSNR, the better the quality of the compressed map.
Normalized cross-correlation (NCC): a value of 1 indicates maximum match between the maps, while 0 indicates no match.
perceptual visual quality metrics that predict quality from an end-user perspective [32
Structural Similarity Index (SSIM) [33
]: utilizes structure information to determine image distortion. SSIM is 1 when the images are identical, while 0 indicates no structural similarity.
Gradient Magnitude Similarity Deviation (GMSD) [34
]: measure the distortions of the image gradient. The higher the GMSD score, the lower the image perceptual quality.
The resulting metrics shown in Figure 10
, Figure 11
, Figure 12
, Figure 13
and Figure 14
suggest that discarding more than 40% of the points degrades the map quality significantly, as measured by all the metrics. In addition, keeping more than 70% of the points makes almost no difference in quality, but the savings in space are significant because the compressed GeoJSON file has 100 MB versus 137 MB of the original version. On the other hand, even with a compressed LQI data file, the worse tiled variant (LeafletRR) requires a third of the network bandwidth (33 MB).
5. Related Work
The inception of map tiles [3
] was a key enabler for the widespread use of Web maps. Many research efforts have studied how to cope with the increasing requirements these maps put on the whole infrastructure. In particular, intelligent caching of tiles on servers [35
] has been proposed as a way to reduce latency and server load when rendering raster Web maps. Similarly to other Web applications, mapping apps involve a series of client/server interactions. Ref. [28
] aimed at improving these considering the impact each interaction has on the server side.
As the evolution of Web technologies enabled client side processing, geoprocessing moved from the server to Web browsers capable of executing code and displaying rich visualizations [7
], but also creating choices for software developers that impact how well the resulting applications work. Ref. [11
] analyzed the impact of such technologies and their trade-offs for geoprocessing spatial data on the server or on the client side. The transmission of vector maps provides much flexibility compared to raster tiles, as the former are composed of multiple lines, and these lines can be simplified to save network bandwidth and improve performance. In this line, [36
] proposed a method for progressive transmission of vector maps that significantly reduces the required network bandwidth.
Recently, rendering vector maps with good performance made them a viable choice to raster tiles [16
]. Ref. [18
] compared the performance of raster and vector tiles using Mapbox GL JS and hosted in different servers. The testing was semi-automatic, as it involved a user performing the same pan and zoom operations over a map. They analyzed the time, number of requests and network traffic generated by these operations. In addition, the paper also reports a preliminary experiment involving a single scenario with four different Web browsers and three different devices (PC, tablet, and smartphone). In the experiments, the authors observed that vector tiles had a better loading time, but, on maps with only a few zoom levels, raster tiles appear to be better. The most valuable contribution of [18
] is a deep study of server load patterns on different server technologies (Mapbox Studio, TileServer PHP, TileServer GL, and plain files) when using raster and vector tiles.
Complementary to our work, [6
], lines, or polygons, over a basemap.
according to IGN (https://www.ign.gob.ar/NuestrasActividades/Geografia/DatosArgentina/LimitesSuperficiesyPuntosExtremos
The results presented in this work point out that vector-based representation of geographical information requires less bandwidth, while being as fast as raster-based representation when rendering on mid-end and high-end devices for all the libraries but Leaflet. This is a consequence of both Mapbox GL JS and OpenLayers reliance on WebGL for rendering, while Leaflet does not support WebGL. Since vector-based representations are well-suited for metered connections, such as 3G or 4G, and can be efficiently rendered by mid/high end mobile devices, these kinds of representations are well suited for Web mapping applications.
From the experimental results, we also conclude that Mapbox GL JS has the best overall performance on mid and high end devices for all map variants, including vector basemap and LQI layer. OpenLayers surpassed the others for displaying raster basemaps on all devices. From the perspective of devices, a mid-end device such as the Moto G6 achieved competitive performance rendering vector maps with Mapbox GL JS. This result is a clear indication that vector-based maps are a safe bet for new Web mapping applications, since performance is on par with raster maps, but consume much less network bandwidth.
A limitation of this work is that, although we have evaluated time and network usage for Web maps, power consumption has not been evaluated. Power consumption is related to both network usage and processing, but it cannot be directly estimated from these variables because it depends on several factors, such as hardware architectures, operating system, or wireless signal quality. Power consumption is a major issue as mobile devices rely on batteries, and overconsumption affects the user experience. In future work, power consumption would be analyzed. Finally, we will assess the behavior of Web mapping libraries in other widely used platforms and browsers, such as iOS, Safari, and Mozilla Firefox.