- Status: approved
- Deciders: Lucas Brown, Shelby Switzer, Nat Hillard
- Date: 2021-04-21
- Tags: frontend
Technical Story: #7
In order to visualize the metrics of the Justice 40 initiative, we need a client-side mapping solution. This solution should should reflect the values of our initiative, and it should reflect and respond to the diverse needs of our users.
Team Values: The solution should be open-source, modular, and adopt modern best practices.
User needs: The solution should be performant, accessible, internationalized/localized, and minimize data usage.
We provide more detail on these factors below.
- Open source : Our technology choices should "default to open" as detailed in the Digital Services Playbook, in order to encourage community contribution, minimize vendor lock-in, and lower the barrier to entry to participate in the development of the tool. Ideally such a solution should be be true Free and Open Source Software, licensed for use, copy, and modification as well as source code-visible.
- Ease of implementation : In the name of welcoming a large body of open-source contributors, our implementation should not require too significant insider knowledge of mapping or GIS systems. At the same time, more advanced functionality should be familiar to those who do have such backgrounds, to enable them to contribute expertise where warranted. Additionally, it should be possible to get an environment up and running locally without a necessary API key or other paid plan.
- Modular : we should minimize vendor lock-in and allow for mixing and matching solutions from different providers -- from server-vended tiling solutions to tile storage and basemaps -- in order to optimize user experience and find the best tool for the job at all layers of the stack.
- Modern : the library should allow for its work to be done entirely in the browser; it should provide the ability to load on demand (in binary, protobuf format) and visualize so-called vector tiles to make use of the rendering power of WebGL; and it should provide for client-side dynamic map styling according to relevant standards as opposed to just server-side pre-rendering of raster tiles.
- Performant : Library should be performant on a range of device types, enabling fast load times and ease of navigation in a highly-complex rendering environment where there may be hundreds of thousands of features.
- Accessible : the implementation we build with this library must be compliant according to section 508 of the Rehabilitation Act (29 U.S.C. § 794d), as well as Section 501 guidance regarding Reasonable Accommodation per US federal government legal requirements. Ideally it should adopt Web Content Accessibility Guidelines to at least the AA level. The library our solution builds upon should provide some of this out of the box.
- Internationalized/Localized : Nearly 22% of the US population speaks a language other than English at home. Our mapping solution should be familiar and easy to use by this population.
- Minimizes network usage : Where possible, we should seek to minimize network usage to ensure lower data usage costs and faster load times.
- Mapbox GL JS - v2.2.0 (3/25/21)
- MapLibre GL JS - v1.14.0 (3/24/21)
- Leaflet - v1.7.1 (9/4/20) + Leaflet.VectorGrid - v1.3.0 (2017)
- OpenLayers - v6.5.0 (12/27/20)
- ArcGIS API for Javascript - v4.18 (12/2020)
- Non-ArcGIS-JS-API Esri products - The focus of this investigation is web APIs that can be incorporated into a website and rendered as part of a cloud-based, modular web-development flow. Additionally, these are not open source.
- Google Maps Platform/Bing Maps APIs - These mapping are also closed source and proprietary, and do not allow for significant customization or visualization of the sort we would need for this product
- d3.js - though impressive in its offerings and performance, d3 appears more oriented to more general-purpose infographic use than a mapping-specific product requiring significant GIS-adjacent features. We may consider aspects of D3 for particular chart renderings.
OpenLayers (Source - v6.5.0 (12/27/20)). This library is completely free and open-source, modular, and modern. As a modern, well-typed and well-documented javascript library, it is approachable, yet offers a full-featured and familiar interface as well for those with a background in GIS systems. As it is free to use, anyone could clone our source and run our project locally without payment or registration. Additionally, it has relatively good performance for large feature sets, despite not using WebGL for rendering.
- Licensing : OpenLayers is BSD 2-Clause licensed
- More fully featured : OpenLayers has a number of GIS features that other web-based tools do not. A somewhat older analysis done in 2017 concluded that when compared to a set of considered features, OpenLayers 3 had a relatively large number of supported overall GIS features:
- Popularity : OpenLayers is second only to Leaflet in the number of Github stars it has received, close to 8000
- Performance :
- The below chart comes from an September 2020 study (www dot mdpi dot com/2220-9964/9/10/563). The purpose of this study was to compare OpenLayers to Mapbox-GL-JS and Leaflet (both raster and vector tile variants) as the potential basis for a Life Quality Index for 55,000+ census radius jurisdictions in Argentina.
(Source (www dot mdpi dot com/2220-9964/9/10/563))
In this chart, the two letters following the library name are for basemap layer and feature layer. Further, "R" is "raster" and "V" is vector, and lower numbers indicate faster load times. "OpenLayersRR" and "OpenLayersRV", (signifying a raster base layer and vector feature layer), performed quite well across all device types compared to other libraries. - We also performed local testing using puppeteer and web performance APIs, tested against a choropleth map of the cenus block groups, which represents a likely usecase for us. The results were as follows:
(Source)
Testing and results can be found on the client_perfbranch in this repository. - Finally, we tested these mapping solutions against a large dataset with many features, a map of national highways, at a lower zoom level. The tiles were still loaded on-demand, but the greater detail and number of features meant a more difficult render. Trends seemed similar to the above, with the exception of Leaflet:
(Source)
Notes:
- OL+MB did not file the
tiledidloadevent and thus there was not a separate measure for style loaded - Apparent performance was different from measured/reported overall, particularly when it comes to zoom performance. This is an area to dig into further and measured at a later time to understand better.
- OL+MB did not file the
- The below chart comes from an September 2020 study (www dot mdpi dot com/2220-9964/9/10/563). The purpose of this study was to compare OpenLayers to Mapbox-GL-JS and Leaflet (both raster and vector tile variants) as the potential basis for a Life Quality Index for 55,000+ census radius jurisdictions in Argentina.
- Data Usage : The same study above also analyzed the amount of data usage for each of the libraries under investigation, and the result was the below chart (Lower values are better). OpenLayers overall performed quite well
(source (www dot mdpi dot com/2220-9964/9/10/563))
- Learning Curve : A recent survey ranking various mapping libraries ranked OpenLayers as "Hard" in a scale from "Basic" to "Very Hard", and furthermore calculated "Approximate Learning Curve for Javascript", by which it was on the harder end as well (more detail in paper):
(Source)
This is part of the tradeoff in the library's large feature set. Anecdotally, however, for the purpose of a simple choropleth map, implementation felt similar to other libraries. - Modern Stack : OpenLayers requires a plugin for the combination of MVT vector tiles and Mapbox GL Styles, though it does support it.
- Rendering : According to geoapify, it has 'slightly less "polished" rendering quality and performance' compared to Mapbox/MapLibre GL. This is partly related to the fact that it does not yet support using WebGL for rendering anything more than points (source).
- Accessibility
- Accessibility for web mapping technology in general is not good. As of now, the default 2D visualizations in and of themselves do not provide explicit 508 compliance, and there is no official accessibilty statement for OpenLayers.
- According to a recent exhaustive WCAG 2.1 evaluation that compared several map providers, though almost all web maps performed poorly, OpenLayers came out in the middle of the pack:
(Source: original here, based on data here)
This chart counts the number of successful WCAG 2.1 criteria from the above-linked study. - Mitigations:
- According to at least one accessibility expert, it is sufficient for compliance reasons to provide an alternative means of viewing the same information provided by the map, perhaps in a table or other format. This is a common solution to this problem.
- Though the library itself does not provide an accessibility guarantee, many of the 508 and WCAG standards are a matter of how you use a particular library - e.g. ensuring proper color contrast, providing alternative markers not based on coloration, and providing text highlighting. We can guarantee that these are in place and ensure an accessibility audit of our featureset before release alongside user testing with low-vision and visually-impaired users directly.
Source - v2.2.0 (3/29/21)
- Performance : As can be seen in the above charts, Mapbox performed quite well in performance tests, in some instances performing the best of the examined options.
- Data Usage: In the data usage chart above, MapboxRV and MapboxVV also do quite well in the category of data usage.
- Modern stack: Mapbox supports WebGL rendering, vector tiles, and standardized tile styling
- Modularity: Mapbox interoperates with various basemap providers, and thanks to the open-sourcing of both its vector tile format and its styling language, there are a number of tools, libraries, and frameworks that easily interoperate with this solution.]
- (Relative) Accessibility : Mapbox-GL-JS came out 4th out of 12 in the above accessibility analysis. Mapbox also released several patches recently to address the issues that were found and are actively working on this.
- Localization : Through the mapbox-gl-language plugin, it is possible to dynamically change the language of mapbox-provided basemaps to other languages. Additionally, they have a plugin for displaying RTL languages, and the ability to format numbers according to locale conventions.
- Popularity : According to NPM Trends Mapbox-GL is the most downloaded package among those considered. More info here
- Licensing : Mapbox's December 2020 announcement of version 2.0 of their software changed ther license to proprietary and changed their pricing scheme to cover tiles loaded from outside of their service. This decision was met with some criticism in the open-source mapping community.
- Library Size : Mapbox-GL is one of the larger libraries considered as far as filesize, coming in at 214.1 KB (Source)
- Pricing : As of version 2.0.0, Mapbox charges per "map load", defined as "every time Mapbox GL JS initializes on a webpage or in a web app". Up to 50,000 loads per month are free, and beyond this pricing varies. Users are likely to do a number of loads per sitting, however, so this is something we will need to consider, even in a relatively low-traffic tool.
Source - v1.14.0 (3/24/21) MapLibre is a fork of Mapbox 1.13.0 that preserves the older BSD 3-clause license.
- Everything above : Being a fork of Mapbox GL JS (though an earlier version), MapLibre has all of the benefits of Mapbox called out above
- Licensing : Returns to 3-Clause BSD License means true FOSS licensing, as well as free operation
- Maturity : This library is still quite young, having been formed only mid-December last year
- Licensing Complications : There may be licensing or legal challenges from Mapbox around the use of this tool
- Component-based Wrappers : Though supported by react-map-gl, there is not yet as full-featured support for MapLibre-GL in other component-based wrapper libraries
- Backwards Compatibility : There is an issue open now discussing the possibility of breaking backwards compatibility as part of a MapLibre 2.0 release. In practice this would mean we would not be able to use layers created within Mapbox in MapLibre tooling.
- New Features : There is not yet clarity on how and when this library would be updated and what its upcoming feature roadmap looks like
Source - v1.7.1 (9/4/20) + Leaflet.VectorGrid - v1.3.0 (2017) Leaflet is a popular open-source mapping visualization library that has been around since 2010.
- Popularity : Leaflet has by far the largest number of stars on Gitbub (30,500 at the time of writing)
- Modularity : Leaflet has a number of plugins that augment its functionality, including Leaflet.VectorGrid which enables Vector Tiles
- License : Leaflet is BSD 2-Clause licensed
- Performance : As seen above, Leaflet performance is quite poor across device types, relative to other libraries, even with the vector plugin
- Modern Stack : The lack of out-of-the-box vector tile support and the necessity of another library opens the possibility for version requirement mismatches and security vulnerabilities. Additionally, the last update to the VectorGrid plugin was a year ago.
Source - v4.18 (12/2020)
- Accessibility : ArcGIS provides regularly-updated accessibility conformance reports detailing their products' conformance with 508 standards.
- Note: These audits are not all up to date - the ArcGIS online evaluation was last completed in September 2018
- Anecdotally, screen readers still struggle to read content without annotations - the onus is still on the map creator to ensure proper compliance.
- Internationalization/Localization : the API has its own
intlmodule, described here- Note: The default ES6
Intlmodule appears to do all of the things described above without need for further modificaiton
- Note: The default ES6
- Name recognition : Esri in general holds "approximately 43% of the global market share and estimated annual revenues of approximately $1.1 Billion, from roughly 300,000 customers" (source). Relatedly, Esri tools interoperate well with other Esri tools, and their ecosystem is large.
- Note: according to one analysis specifically the ArcGIS Web API holds only 0.48% of the marketshare overall.
- Modularity : As seen in ArcGIS documentation, Esri interoperates with Leaflet (through a custom fork), Mapbox GL JS, and OpenLayers.
- License : Most importantly, though Esri maintains a handful of open-source tools, ArcGIS API for JS itself, as well as many other tools within the Esri / ArcGIS portfolio, is closed source and proprietary, hosting only minified versions of their software and not encouraging community contribution or feedback beyond forum contributions.
- Complexity : The feature set comes with the tradeoff of greater complexity in implementation for ArcGIS as well. Additionally, many more advanced features of the ArcGIS API require you to utilize the Dojo framework and/or Dijit components (see examples here), a requirement not held by other frameworks.
- Modern : ArcGIS-JS-API only recently got support for ES Modules in version 4.19 (source). Additionally, many examples from earlier versions are still common, including in the ArcGIS API for JS documentation itself, which still uses HTML includes. There are recently some newer examples in github, but they also seem to have been recently added.
There are a handful of other similar comparisons out there, here are some references:
- Geoapify comparison (Source)
- Flatlogic Comparison
