DOI: 10.14714/CP97.1707

© by the author(s). This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/4.0.

A Comparison of Feature Density for Large Scale Online Maps

Michael P. Peterson (he/him), University of Nebraska at Omaha | mpeterson@unomaha.edu

INTRODUCTION

One of the primary benefits of using online map services like those available from Google, Bing, and OpenStreetMap, is that zooming-in allows access to large-scale maps. Maps at these large scales are not available to most (if any) individuals from any other source. The features and labels that are included on these large-scale maps are an important indicator of both the completeness of the underlying database and the conversion process from data to map.

Online mapping services all rely on vector databases composed of point, line, and area features, along with feature attributes. When using these services, we view a version of the vector data, rendered appropriately for the scale. The rendering process involves the scale-dependent generalization and symbolization of the spatial data, and subsequent tiling (Clouston and Peterson 2014). Rendered map tiles can then be zoomed and panned from side-to-side, producing what we refer to as multi-scale pannable (MSP) maps (Peterson 2015; 2019).

Since the introduction of the technique in 2005 by Google, all major online map providers have adopted the same underlying technology. Vector data is projected and divided into vector tiles at multiple scales. The tile boundaries are identical between all mapping services. The maps vary only as a result of differences in the underlying vector database, and the generalization / symbolization used in their rendering.

MSP maps from the major online providers receive considerable use, and it is therefore important to evaluate the quality of the maps provided by these services. In this paper, my evaluation method centers on examining the density of both features and labels at the 19^th zoom level for three online map providers: Google, Microsoft, and Mapbox. This zoom level was chosen because, when examining large-scale maps from these services, it can be observed that no new features are added above the 19^th zoom level. While features are made larger at larger scales, additional features are not being added to the display. This seems to be true even in more densely populated urban areas where competition for map space would normally result in a selection of displayable features.

By assessing the density of features and labels for a set of randomly chosen locations, the findings reveal which of these online map providers has made the greatest effort to offer detailed large-scale maps.

ORIGINS OF THE MAP DATA

As we are evaluating differences in the underlying vector databases between these online mapping services, it is important to understand their origins. There are essentially three different types of sources that MSP map providers can draw upon: (1) a governmental agency, such as a city, state, or federal entity; (2) a proprietary database, as created by Google, TomTom, HERE, and others; or (3) a public-domain database based on crowdsourcing, as created by OpenStreetMap (OSM) and Wikimapia. The distinctions between these are becoming increasingly fuzzy as local governmental agencies provide data at no cost to commercial entities such as Google. The same agencies may also contribute to OSM, a service that itself relied initially on data made freely available by some governments, notably the United States. Some commercial mapping entities, like Mapbox, rely solely on data from OSM.

There are only a few sources of global geospatial data. While governments generally restrict mapmaking to the area within their borders, some governments map foreign lands for military purposes. Not only are military maps kept secret but many governments also keep secret, or charge fees for, maps of their own territories, even in the more developed parts of the world. On the opposite end are crowdsourcing platforms like OSM and Wikimapia that make their data available to anyone (Hall et al. 2010). Both platforms produce a large proportion of their data using satellite imagery, often from Google Earth—a proprietary source. GPS devices are also used to trace roads and pathways. Of the two, OSM maintains a much larger crowdsourced database. This volunteered geographic information (VGI) is made available under the Open Database License (Feick and Roche 2013).

Only a handful of companies collect proprietary global geospatial data, including Google, HERE, and TomTom. Google is fairly new to the world of mapping, introducing Google Maps in 2005 with acquired technology. Initially, Google had spatial data only for the United States and the United Kingdom. It was 2009 before their maps included features for every country (Garfield 2012). Its spatial-data-capturing Street View vehicles have driven more than 10 million miles since 2007 (Mogg 2019), in countries where they have been allowed to drive. Maps from Google are now the most used maps in the world.

In contrast to Google, HERE had its origins in the mid-1980s as NavTeq, a pre-GPS car navigation company. It was subsequently acquired by the Finnish company Nokia in 2007 to provide navigation for its phones, and then by a consortium of German automobile companies, Audi, BMW and Daimler, in 2016 for autonomous car navigation. TomTom has been building a proprietary map database for navigation since 2001. Its Automotive and Licensing division, providing GPS-based navigation for car manufacturers, represents a major part of its business. Like HERE, the company is also based in the Netherlands.

After selling its mapmaking technology to Uber in 2015, Microsoft no longer collects geospatial data. It has instead built business relationships with both HERE and TomTom (Stevenson 2016), even contracting with HERE for map rendering services. Microsoft has been using the Bing moniker for many of its services, including maps, since 2009.

Mapbox has become a major player in online mapping. It classifies itself as a developer platform only and does not provide a publicly available MSP map. The only way to view Mapbox-rendered tiles is to use the Mapbox Application Programming Interface (API). It is completely reliant on OSM for its map data (Bliss 2015). Figure 1 shows two pairwise comparisons between OSM and Mapbox-generated maps. While the symbolization varies, the comparisons clearly demonstrate that they have identical features and labels and are based on the same vectors.

Figure 1. Two comparisons between OpenStreetMap (left) and Mapbox (right) demonstrating that Mapbox relies exclusively on data from OpenStreetMap. The maps are based on identical vector points. The symbolization and labeling vary only slightly.

COMPARING VECTOR DATABASES

While evaluations of the accuracy and completeness of vector databases are a major area of research, they have unfortunately been limited to assessments of VGI. Since it is not possible for researchers to obtain proprietary vector data, they have focused on comparing crowdsourced data with government sources, or have assessed the completeness and accuracy of OSM data using a variety of other indicators such as a ranking of its contributors. The main purpose of this research has been to demonstrate the basic utility of crowdsourced spatial data (Flanagin and Metzger 2008). This is analogous to previous research that examined the validity of Wikipedia pages as compared to published encyclopedias (Okoli et al. 2012). The primary comparisons that have been made are between OSM and so-called “authoritative” spatial data, usually government databases like Britain’s Ordinance Survey and Germany’s ATKIS (Haklay 2010; Fan et al. 2014; Zielstra and Zipf 2010; Wang et al. 2013). Jackson et al. (2013) propose a method for quantifying the completeness and accuracy of volunteered geographic point datasets using a national geospatial dataset as the reference.

A number of studies have examined OSM data quality for a particular country or region. Arsanjani and Vaz (2015) assess the accuracy of its land use classifications in seven large European metropolitan regions. Siebritz and Sithole (2014) assess the quality of OSM data in South Africa with reference to national mapping standards. Zhao et al. (2015) implement a statistical analysis of OSM data for Beijing, China. Corcoran, Mooney, and Bertolotto (2013) examine the growth of OSM street networks in Ireland and demonstrate that two elementary spatial processes of densification and exploration are responsible for increasing the density of information and expanding the network into new areas. Girres and Touya (2010) assess data for France based on geometric, attribute, semantic, and temporal accuracy; logical consistency, completeness, lineage, and usage; and different methods of quality control. Their study raises questions about heterogeneity of processes, scales of production, and contributors’ compliance to standardized and accepted specifications. They suggest that in order to improve data quality, there needs to be a balance between the contributors’ freedom and their respect of specifications.

Another area of research has examined approaches to defining and measuring spatial data quality. Ciepłuch, Mooney, and Winstanley (2011) suggest generic quality indicators for OSM. Barron, Neis, and Zipf (2014) present a framework containing more than 25 methods and indicators for OSM quality assessments based solely on the data’s history. Antoniou and Skopeliti (2015) contribute to the ongoing effort to create a practical method for evaluating data quality. Senaratne et al. (2017) review VGI quality assessment methods. Sehra, Singh, and Rai (2017) present an extension of the QGIS Processing toolbox to assess the completeness of spatial data using intrinsic indicators.

Muttaqien, Ostermann, and Lemmens (2018) propose an intrinsic measure of OSM data quality not based on the data itself but by a measure of aggregated expertise of the contributors. Similarly, Nasiri et al. (2018) suggest an improvement in the quality of contributed data by examining historical contributions of data providers.

Other researchers have compared building footprints, a common feature in large-scale maps. Hecht, Kunze, and Hahmann (2013) examine the spatial accuracy and completeness of OSM footprints using official data from national mapping and cadastral agencies for comparison. They found a completeness rate of about 25% in different German states by 2012. Brovelli and Zamboni (2018) also examine completeness, and Brovelli et al. (2016) implement a process of automatic homologous pairs detection for the same purpose. Törnros et al. (2015) apply two commonly used, unit-based methods to evaluate building completeness in OSM data and find strongly different results depending on the method used. They propose a simple pre-processing of the building footprint polygons that leads to a more accurate completeness estimation for one of the methods.

All of this research has essentially demonstrated the basic validity of OSM data, at least as compared to data gathered by some government entities. Heterogeneity of data collection is a recognized problem, as crowdsourcing favors the more densely-populated area and affluent countries (Bittner and Glasze 2018). As with anything crowdsourced, the quantity and quality of data is a function of the crowd. Where there are few people, there is less data—and fewer to check its quality. Put simply, crowdsourced data compares favorably to government data in spatial accuracy, but it is uneven in coverage.

COMPARING RENDERED MAPS

While a number of studies have examined the spatial accuracy and feature density of OSM vector data, rendered maps from online map providers have received little attention. One exception is Boottho and Goldin (2017), who implement an automated approach to assess the quality of rendered maps from different web mapping services. They use the APIs from Google, HERE, MapQuest, and Bing to perform repeatable queries and compare the results to reference data gathered by ground survey and external sources. In a study on a small area in Thailand, they found that HERE had a higher completeness score, MapQuest had the least discrepancy score, and Bing and Google tied for the highest richness score, a measure of feature density. The automated method used for analysis did not examine labeling.

It is important to examine the rendered maps that map users actually see. In order to compare feature and label density, I implement a pairwise comparison of large-scale maps of randomly selected locations in North America, Europe, and sub-Saharan Africa. The advantage of this approach, compared to more automated approaches, is that both feature and label density can be assessed. The pairwise comparisons are implemented through the use of the APIs for Google Maps, Bing Maps, and Mapbox. These three services represent a cross-section of online mapping technology. Google Maps, the most-commonly used online mapping service (Panko 2018), relies heavily on its Street View vehicles for collection of spatial data. Bing Maps is based on business agreements with both HERE and TomTom, two major global spatial data providers. Mapbox was chosen for its exclusive reliance on OSM data.

I have made the tools for pairwise comparison openly available on the web page for my Mapping in the Cloud (Peterson 2014) book at maps.unomaha.community/cloud/code/code10/Pairwise (see Figure 2). The pairwise comparison approach represents an easy-to-replicate method for comparing mapping services. Comparisons between other mapping services could be easily implemented by integrating their API code. Examining other areas of the world would only require a modification to the bounding box used for randomly selecting points.

Figure 2. Website for pairwise comparisons at maps.unomaha.community/cloud/code/code10/Pairwise.

My initial attempt at comparing large-scale maps involved the counting of features and labels. For example, Figure 3 shows a comparison between Google Maps and Mapbox for a randomly selected point in North America. Here, the number of labeled features (the road) would be identical, while the map from Google on the left depicts one more feature in total, as it shows a waterway paralleling the road.

Figure 3. A pairwise comparison between Google Maps (left) and Mapbox (right) for a randomly chosen location in North America. Google Maps displays a water feature that is missing in the OSM-based Mapbox map. Both services label the road.

However, a particular map service may present a whole layer of information that is not present on another, such as land ownership boundaries (cadastre), building footprints, business establishments, traffic directions, or alternative road names (see Figure 4). When this is the case, the counting of features and labels becomes problematic. For example, each land ownership boundary line segment could be counted as a separate feature.

Figure 4. A pairwise comparison between Google Maps (left) and Bing Maps (right) for a randomly chosen location in North America. While both maps include building footprints (extruded with Bing), the map from Google also depicts property boundaries (cadastre), the name of a business establishment, and an alternative name for the highway. None of these features and labels are visible on the map from Bing.

While a count of features or labels between the two maps cannot be easily done and may unfairly advantage one map service over another, it is a simple matter to judge that the Google map on the left of Figure 4 has a greater feature and label density. A similar assessment can be made for the two maps in Figure 3. This type of evaluation can be represented by a binary classification: the denser map is assigned a “1” and the other a “0.” Such a binary classification, based on a visual inspection, is the basis of the experiment. I use this method to answer the research questions of how the map services compare in judged feature and label density at the 19^th zoom level, and whether this varies by continent.

THE 19^th ZOOM LEVEL

The 19^th zoom level (also referred to as the 19^th Level of Detail) is a very large-scale representation. With a screen resolution of 96 dots per inch, it corresponds to a scale of 1:1128.50 (1 cm: 11.29 m) at the Equator (Microsoft 2018). As a result of the Web Mercator projection used in most MSP maps, the scales become even larger as one moves north and south of the Equator (Lapon, Ooms, and De Maeyer 2020). At 60°N (Shetland Islands in Scotland), the scale is exactly twice that at the Equator, 1:564.25. For a far northern part of mainland Norway, about 71°N, the scale is approximately 1:367.4. Variations in scale will affect the density of represented features, with the smaller scales showing more area and therefore having a greater possibility of features being present. It should be emphasized that these scales given above are only for purposes of comparison. The exact scale of any map presented through the internet varies based on monitor resolution, browser zoom settings, and other factors influencing the display size. Maps presented by MSP map services include only a bar scale, because representative fraction and verbal scales cannot be provided without knowing the final display size of the map on the monitor.

I am evaluating maps at the 19^th zoom level since it seems to represent the scale at which all features in the underlying vector database are displayed. It is a zoom level that is not normally accessible to most users of Google Maps, Bing Maps, or OpenStreetMap; the largest scale that is presented on their respective websites is the 18^th zoom level. But, when presenting maps through their API, zoom levels are extended up to at least 22, though the maximum zoom level that is available may vary for different parts of the world, and different map types. Because these services charge a cost to use their respective APIs (above a certain number of monthly map downloads), one could say that the value of the API is being evaluated as well as the density of features and labels.

Since the 19^th zoom level is so large-scale, a small map of a randomly selected location will most often include no features beyond land or water. Of a random selection of 100 North American locations on Google Maps, only 16 depicted any other features. For Europe, the number was 32. While Europe is more densely settled, it lies further north and therefore the maps would typically be at a larger scale, thus covering less area and correspondingly less possibility of features being present.

The possible features that can appear at the 19^th zoom level include roads, paths, railroads, forested areas, rivers, water bodies, political boundaries, building footprints, churches, commercial establishments such as stores or post offices, and land-ownership boundaries. Of these, roads, rivers, and building footprints are the most commonly found. Maps from Google often also include business locations and corresponding names.

METHODOLOGY

The pairwise comparison is implemented by simultaneously displaying maps from two services in a single web page, utilizing each service’s API. The two maps show the same exact (randomly chosen) point on the Earth’s surface at the same scale. Each map is 800 × 500 pixels, corresponding to what would be easily visible on most mobile phone displays. The different map services that are being compared each offer a variety of named map styles (emphasizing terrain, imagery, roads, etc.). The specific styles being compared are Roadmap from Google Maps, Road from Microsoft Bing, and Streets-v9 from Mapbox. Note that Mapbox labels its zoom levels in a non-standard way. While Google’s 19^th zoom level matches Bing’s 19^th zoom level, Mapbox’s corresponding zoom level is 18 (Figure 5).

Figure 5. A map from Google at the 19^th zoom level (left) compared to one from Mapbox at the 18^th (right). The maps are at the same scale. Mapbox uses a different numbering scheme for its zoom levels compared to other online map providers.

For each of the three regions chosen for the study (Europe, sub-Saharan Africa, and North America), a bounding box is first defined, covering the central regions while avoiding large bodies of water (Figure 6). To randomly choose a location for comparison within a region, a random point is determined using JavaScript’s random number generator, which returns a number between 0 and 1. This value is multiplied by the range of latitude or longitude covered by the bounding box, and then added to the minimum value. For example, the bounding box used for North America has a minimum latitude of 30° N and a maximum of 50° N, for a difference of 20°. If the random number function returns a value of exactly 0.5, the randomly determined latitude is 40° N (0.5 × 20 + 30) — half-way between the latitudes that define the bounding box. Part of the JavaScript code for the random definition of points is shown in Figure 7.

Figure 6. Bounding boxes used for the random selection of locations within North America, Europe, and sub-Saharan Africa.

Figure 7. JavaScript code for randomly determining a latitude and longitude within a bounding box. The first two lines determine the lngspan/latspan, or difference, between the bounding box’s latitude and longitude. The lat and lon values are then calculated by multiplying the latSpan and lngSpan by a random number (always a value between 0 and 1), and then adding this to the minimum latitude and longitude values. The last four lines interface with the Google Maps API, centering the map on the randomly chosen location.

When comparing maps, differences in symbolization were ignored. For example, a small road may be indicated by a dashed line on one map and a solid line on the other. While these differences were not assessed here, they could be an area for further research. Differences in generalization, however, were considered. For example, the two maps in Figure 8 feature different levels of detail in the coastline. This may be a result of either a less detailed database, or a line generalization process taking place during rendering. The coastline is clearly more detailed on the Bing map (on the right), and therefore it is chosen as the map with greater feature density. Feature density is interpreted as both the number of the features and the amount of detail within the features themselves.

Figure 8. Comparison of a coastline between Google Maps (left) and Bing Maps (right) at the 19^th zoom level. The two are either using a different underlying vector databases or different line generalization settings. Whatever the case, the Bing representation includes more detail.

During the experiment, a web page creates two maps from two different map services. A comparison can be made only if at least one of the two maps includes any features other than land or water, and the web page is refreshed (randomly choosing a new location) as many times as is needed to meet this condition. When features are found, an assessment is made about which of the two maps depicts more features and labels, and that map is given a value of 1. A tie is declared when features and labels are identical between the two maps. The process continues until 100 comparisons have been made, after which the results are compared by map provider and continent. At the end, conclusions are drawn about which of the three map services offers greater feature/label density at the 19^th zoom level, and how this varies by continent.

RESULTS OF PAIRWISE COMPARISONS

Figure 9 presents some of the pairwise comparisons between Google and Mapbox for the three continents. The relative ease of making a judgment on feature density between the two maps should be apparent in all of these illustrations.

Figure 9. Comparisons between two randomly-chosen locations in North America (a–b), Europe (c–d), and sub-Saharan Africa (e–f). With the exception of pair e, the Google Map on the left has more features and labels than the Mapbox map on the right.

RESULTS: NUMBER OF TRIES

The number of page refreshes needed to find maps with features varied by both continent and map provider. Figure 10 shows the combined number of tries needed for each continent. Similar numbers of tries were required for North America (1163) and Europe (1179) to reach the necessary 300 comparison pairs (100 comparisons each of Google/Bing, Bing/Mapbox, and Google/Mapbox). However, the maps for sub-Saharan Africa had far fewer features, and more than three times as many total attempts were needed, 3648, to find the required number of maps with features.

Figure 10. Number of tries needed by continent to find 100 comparison pairs in each of the three pairwise comparisons. The values for North America and Europe are remarkably similar, while more than three times more pairwise comparisons were needed to find the required number for Africa.

Figure 11 shows the number of tries needed by map service across all comparisons. Here, a lower number reflects better on the mapping service. Bing Maps fared the best, while comparisons involving Mapbox required the most tries to find a map with features.

Figure 11. Number of tries needed by map service over all comparisons.

RESULTS: NUMBER OF TIES

The number of ties in pairwise comparisons also varied by continent and map service. A tie was declared if no difference in feature/label density could be determined between two maps. Figure 12 shows the ties by continent. In this comparison, North America and Europe were again almost identical. The number of ties for Africa was far lower, indicating less congruity in features and labels between the three map services for this continent. When examining specific comparisons of services across all continents (Figure 13), the results show that the number of ties was lowest for the Google/Bing comparisons while almost identical for Google/Mapbox and Bing/Mapbox.

Figure 12. The number of ties for all pairwise comparisons by continent. The maps in the pairwise comparison were rarely identical but merely had a relatively equal number of features/labels.

Figure 13. Total number of ties for each pairwise comparison of services.

RESULTS BY CONTINENT

Figure 14 presents the results of the pairwise comparisons for North America: the number of times that one service had a greater feature density than the other. The results for each pair are remarkably similar, with Google having an advantage over both Bing and Mapbox, and Bing having an almost equal advantage over Mapbox.

Figure 14. Comparison of densities of features and labels for North America. Ties are not included, so pairings will not add to 100 here or in subsequent figures. Google Maps benefitted from the inclusion of land ownership boundaries in the United States. Building footprints were often missing on maps from Mapbox.

A big advantage for Google Maps in North America is the inclusion of cadastral information in urban areas. Land ownership boundaries were not present on maps from Bing. Mapbox is missing not only cadastral information, but also many building footprints, as can be seen in Figure 15. Another factor in favor of Google, particularly in more rural areas, was the inclusion of labels for water features.

Figure 15. A comparison between maps from Mapbox (left) and Google (right) for a part of Tulsa, Oklahoma. The Mapbox map is missing most of the building footprints, as well as property boundaries.

Figure 16 shows that the results for Europe are very similar to those for North America, with Google holding an edge over both Bing and Mapbox. The results for Bing and Mapbox are very similar. Bing’s advantage here was in having more labeled features.

Figure 16. Comparison of features/labels for Europe. The results are almost identical to those for North America. Bing Maps had a smaller advantage over Mapbox than in North America.

The results for Africa diverge sharply from those for North America and Europe (Figure 17). Here we find that Bing Maps holds an advantage over both Google and Mapbox. Bing’s main advantage was again in having more labels and building footprints. Google scored better than Mapbox.

Figure 17. Comparison of features/labels for Africa. Bing Maps benefitted from the inclusion of more labels and building footprints. Google Maps had a smaller advantage over Mapbox than on other continents.

Finally, Figure 18 shows a summary of the results by map service across all three continents. Google and Microsoft Bing have almost identical values. Maps from Mapbox, relying on data from OpenStreetMap, did not compare as favorably.

Figure 18. Overall results by map service for all three continents. The results for Google and Bing are almost identical. Mapbox, using data from OpenStreetMap, did not compare favorably in pairwise comparisons with Google and Bing at the 19^th zoom level.

REPEATABILITY

The question with any experiment-based research approach is whether or not the results can be repeated reliably. The use of the random approach to select locations will result in some variability between trials. To check repeatability, a second, smaller experiment was done between Google and Mapbox in Europe to determine if the results are relatively consistent between trials (Figure 19).

Figure 19. Results of two different trials between Google and Mapbox in Europe. The results are remarkably similar, considering the selection of random locations for map comparisons.

In the second trial, 494 tries were needed to find 100 maps with features, compared to 430 in the initial trial. The number of times that each map provider was found to have greater feature density is similar between the two trials. While the exact numbers vary, the ratio between the two map services is approximately the same.

REGIONAL REPEATABILITY

Map providers will sometimes display slightly different maps to different regions of the world as a result of local interests and different interpretations of country borders. It has also been observed that map tiles within a map come from multiple servers in different locations. This may indicate a difference in feature/label density for maps accessed in one part of the world compared to another.

The results reported thus far compared maps generated from the United Kingdom. A VPN (Virtual Private Network) was used to instead make the map servers view my browser as being located in the United States. A trial was then done once again between Google and Mapbox for Europe. The results were 41 for Google, 32 for Mapbox, and 27 ties in 506 attempts. This compares to 48 for Google, 27 for Mapbox, and 25 ties in 430 attempts when done from the UK. The results are sufficiently similar to discount any major differences in maps served up between the UK and the US.

SCALE COMPARISON

The question arises as to what effect the choice of zoom level has on the results. To investigate this, a preliminary trial was done on the 22^nd zoom level between Google and Mapbox for North America (Figure 20). With this much larger scale, many more attempts were needed to find maps with any features at all. The results showed that there were fewer ties and Google slightly expanded its advantage over Mapbox (its 51% to 21% lead became 61% to 23%) and the number of ties was markedly reduced. More research is needed to determine if whatever advantage one service has over the other at the 19^th zoom level is accentuated at larger scales.

Figure 20. Comparison in perceived feature density between the 22^nd and 19^th zoom levels. Google Maps expanded its edge over Mapbox, and there were far fewer ties.

DISCUSSION

The pairwise comparison of online maps at the 19^th zoom level showed some major differences in feature and label density. The differences occurred across map providers, and the area of the world that was being mapped.

Maps from Google had greater feature and label density for both North America and Europe, though their advantage was slightly less in Europe. A major factor in Google’s favor in North America was the inclusion of land ownership boundaries in the United States (these were not found in the maps of Canada). Bing Maps and Mapbox (displaying spatial data from OpenStreetMap) had no such cadastral information. None of the services included land ownership boundaries in Europe. Interestingly, they were found within the country of South Africa, but not the remainder of the continent.

Mapbox also scored much lower in feature density because it was disadvantaged by a relative lack of building footprints. While all three services had relatively similar building footprints for Europe, there were considerable differences for North America and Africa. While building footprints for Mapbox and Bing were essentially identical, those used by Google seem to be derived from a different source. Building footprints are primarily acquired in an automated fashion from remote sensing imagery.

In more rural or undeveloped areas, the presence of roads, rivers, and lakes differentiated the three map providers. In general, Google represented more roads and water features, along with their labels, for the United States and Europe. In urban areas, maps from Google included more labels for businesses.

Perhaps the most troubling aspect of the study is the general scarcity of features in sub-Saharan Africa for all three services. More than three times more attempts were needed to find the necessary number of maps with features. The poor performance of Google Maps in Africa is particularly disturbing given the prevalence of mobile phones utilizing Google’s Android operating system in Africa, where it has an 80% market share (vs. 69% in Europe and 44% in North America; StatCounter 2021). The Google Maps application that comes with Android would be a poor choice here for large-scale maps. Bing Maps performed much better. It appears that business alliances with HERE and TomTom have helped Microsoft provide more detailed maps compared to other services, at least for this part of the world.

Differences were also noted in spatial positioning, particularly between Google and the other two services. It was not the purpose of this study determine which representation is more spatially accurate—the best method for this would have been to conduct a GPS survey of the features in question. However, rectified remote sensing imagery could also be used in some cases where features are not obscured by vegetation or other features.

As most land area is more rural or undeveloped, the random point selection method employed here will favor the selection of maps in these less populated areas. These are precisely the areas where it is known that OSM has gaps in coverage, and this is likely why Mapbox performed so poorly. An alternative approach would be to randomly choose points only within more built-up areas. This may favor a crowdsourced spatial data source, but would reinforce the notion that OSM has uneven coverage.

The pairwise comparison method used here represents an alternative to prior research on OSM data that only examined the underlying vector data. As most vector data is located in more urbanized areas where features are located, prior research favored the comparison of spatial data in these areas. The advantage of the approach used here is that it appraises areas more evenly. In addition, both the underlying data and the rendering process are being evaluated. Most prior studies examined only the underlying vector data.

CONCLUSION

Making very large-scale maps of the world is not an easy task. Efforts have often focused on more developed parts of the world, where commercial interests lead to competition and some fairly detailed maps, including ground-level panoramic imagery as implemented by Google’s Street View. This imagery, a major source of spatial data and updates for Google, is not acquired in most of Africa, nor in some countries where governments have forbidden it, like Belarus and Germany. Less developed parts of the world will continue to be dependent on crowdsourcing services like OSM for their maps—resulting in data that will subsequently be utilized by commercial interests without much scrutiny.

Comparing large-scale maps from online map providers is an important way of evaluating these services. Map providers need to be subjected to this constant scrutiny of their feature/label density and spatial accuracy. When a map provider is shown to have a product that compares unfavorably to their competitors, it should provide an impetus to improve their map databases and large-scale representations.

When making maps using these services, or when using them for navigation, we should be aware that there are some significant differences between them. Not only is finding a particular location only possible when it is included on the map, having more features and labels on a map increases our connection with the part of the world being depicted. This research showed that Google Maps would be a better choice when seeking large-scale maps of North America and Europe, but not Africa, where Bing Maps provides a better option. Mapbox, using data from OpenStreetMap, would not be a good choice for large-scale maps in any of the three continents.

The pairwise comparison procedure used here can be applied in numerous other ways. For example, an assessment could be made as to which map has more legible text, better design elements, more appropriate generalization for the particular scale, or is even drawn more quickly. Other map providers could also be evaluated such as MapQuest, Here.com, Apple, and Esri. The comparisons could also be done for different parts of the world such as Asia, Australia, or South America. A more automated approach could also be implemented, perhaps utilizing optical character recognition (OCR) for the recognition of text. The methodology presented here opens many avenues for future research to evaluate the quality of MSP maps presented by online services.

REFERENCES