An Assessment of the Accuracy of Volunteered Road Map Production in Western Kenya

The introduction of web based mapping facilities that use satellite imagery, offers local people the possibility to map their environment. However, maps need to be accurate, which is the reason why map making is assigned to professionals. In this paper, we investigated the classification accuracy of road infrastructure from high resolution satellite imagery of an urban area in western Kenya achieved by surveyors and non-surveyors alike, with and without local knowledge. Those with local knowledge classified roads with over 92% accuracy on average, irrespective of surveying background. Professional surveyors and laymen without local knowledge achieved lower accuracies of 67.7% and 42.9% respectively. We argue that local knowledge is also likely to improve the classification accuracy of many other attributes featured in topographic maps and thus conclude that there is reason to consider engaging local expertise in the production and updating of topographic maps.


Introduction
Topographic maps are a critical source of information in rural and urban development, response to disasters and management of natural resources.Yet, their utility becomes limited when outdated [1], a limitation which is common in developing countries.Maps are outdated for many third world cities [2][3][4][5][6], let alone in rural areas; topographic maps were for example found to be outdated when required in support of flood relief in Yemen [7].
Three recent developments offer an opportunity to improve the currency of topographic information.The first is the availability of high resolution satellite imagery on the internet, allowing updating of maps [8].The second development is the emergence of participatory mapping [9] by people without any specific surveying background.Until now participatory mapping required technical expertise to support the mapping by local people, but this constraint disappeared with the introduction of user-friendly web based mapping software [10] that allows participatory mapping without significant outside support.
Tulloch [11] reviewed the potential of participatory mapping in view of the rapidly developing availability of the combination of satellite imagery and web based mapping software.Web based software, combining mapping facilities and access to high resolution photography or satellite imagery, includes OpenStreetMapper [12] and Google Map Maker [13][14][15].Both have user-friendly interfaces allowing people without a surveying background to delineate and classify infrastructure and land resources against a backdrop of aerial photography or satellite imagery.
These developments offer the possibility to produce and update maps locally.Yet, the decision to engage local people with the mapping of their environment should also consider the quality and accuracy of the maps produced.Prevalent questions are then how accurately can local people produce maps, and whether they can achieve the accuracy standards maintained by professional surveyors?These questions need also to be considered in view of the costs of engaging these different groups.
There is little doubt that professional surveyors will perform well when given the opportunity to validate the classification of all mapped features in the field.The costliness of this is however one of the major constraints as to why maps do not remain up-to-date.Without verification, it might well be that local people perform not only cheaper but also more accurately as, contrary to those without local knowledge, they may recall the identity of features in the area familiar to them.
In this study we demonstrate that road maps for medium and small sized urban centers in Kenya are severely outdated.We equally assess the ability of local people to classify the road infrastructure in their neighborhood using high resolution imagery and demonstrate that people with local expertise classify roads more accurately than surveyors and non-surveyors without such local knowledge.

Topographic Maps
We investigated the topographic maps for a sample of large and medium sized urban centers in Kenya.We first gathered population statistics for the urban centers from the Kenya National Bureau of Statistics [16][17][18].We then acquired, for every urban centre, information on the date and scale of the most recent topographic map in the archives of the Survey of Kenya.Finally, we plotted the year of production of the most recent topographic map against the urban centers' population size to visually examine the relation between urban population size and topographic maps.

Study Area
Homa Bay in West Kenya was chosen to compare the thematic map accuracy of roads achieved by image interpreters with and without local knowledge and with and without surveying experience.Homa Bay was chosen because it is a typical example of a city facing the problem described in the introduction.It is a small but rapidly growing urban centre in Western Kenya.The latest topographic map of Homa Bay dates back to 1977 and is thus unlikely to reflect the current situation.

Image Interpretation and Road Classification
The research focused on roads as a sub sample of topographic map features, and assessed how well people, with and without surveying background and with and without local knowledge, were able to classify these.A person was considered local when having spent the last five years in Homa Bay.The sample (Table 1) included four Nairobi based professional map makers without local knowledge and one professional map maker with local knowledge, the Homa Bay district surveyor.Taxi drivers, teachers and bicycle taxi drivers from Homa Bay formed the sub-sample of non-surveyors with local knowledge, while a similar group of professionals from Nairobi composed the sub-sample of non-surveyors without local knowledge.We selected taxi drivers in particular because they are known to have an exceptional memory for street names and topography [19,20].

No
Taxi driver Bicycle taxi Teacher Roads in the 1:50,000 Kenyan topographic maps are classified according to five categories (Table 2).These classes are not accommodated by web based mapping facilities such as Google Map Maker and Open Street Mapper.Given this, we proceeded as follows to assess the road classification accuracy of the 22 interpreters.We selected an area of Homa Bay and its surroundings in Google Map Maker, with the GeoEye Digital Globe high resolution satellite image of 20 July 2009 as a backdrop, and coded all road segments in the satellite image from 1 to 52.The satellite image and the coded road segments were downloaded from Google Map Maker and print outs were made (Figure 1) to assist the image interpreters.

Road ID and Category Description
A The 22 image interpreters, seated with these print outs in front of a computer showing the high resolution satellite image, were asked to classify all 52 road segments according to the above described Survey of Kenya road classification scheme.The image interpretation and road classification (Figure 2) was completed between 2 and 28 September 2009, within two months of the image being taken.

Ground Truth Observations and Accuracy
Next we visited the 52 road segments in October 2009 to visually verify true road category and consulted local people regarding the seasonality of the murram roads.The classifications by the 22 image interpreters were cross-tabulated against the observed ground truth.We used z-tests to test for differences between groups of interpreters based on calculated accuracy and standard errors [21].

Results
Of the 10 urban areas with more than 100,000 people, four, including Nairobi, had maps at 1:10,000, four had 1:50,000 maps, while other towns were mapped at a scale of 1:250,000 or above (Figure 3).Apart from Nairobi, mapped in 2005, other towns had maps older than 10 or up to 40 years old.

Classification accuracy
Figure 5(a) shows a general decline in mapping accuracy from tarmac roads (A) to paths (E).The effect of surveying experience was not consistent for all road classes; surveyors performed better than laymen for road classes A, B and D, but the reverse was true for road classes C and E. Figure 5(b) on the other hand reveals an effect of local knowledge which was consistent across all road categories, those with local knowledge performing significantly better than those without for all road classes.

Discussion
This research revealed that local knowledge and surveying expertise affected road classification accuracy.Image interpreters with local knowledge classified road infrastructure with 92% accuracy on average, which was 24% and 49% higher than the accuracy achieved by surveyors and non-surveyors without local knowledge respectively.In other words, our results indicate that interpreters with local knowledge do classify roads more accurately than those without local knowledge, including professional surveyors.These results challenge the practice to exclusively charge surveyors with the task to map our roads and demand that we reconsider whether we should not engage those knowing the road infrastructure, such as taxi drivers, in the mapping process.
Why does local knowledge make such a difference?Our results suggest that road mapping from satellite imagery is not straightforward, as those without local knowledge performed relatively poorly.The reason for this is that there are several road classes used in Kenyan topographic maps that cannot be exclusively classified from satellite imagery.Satellite imagery does not, for example, distinguish whether a murram road is all weather or seasonal; local experts do, however, know this.Also, those with local knowledge knew to differentiate between tracks and paths more accurately than those without local knowledge.It is this local knowledge which explains why local experts outperform those without local expertise.However, local experts did not always outperform those without local knowledge; there was little difference in the classification accuracy of tarmac roads.Their darker tone in the satellite imagery allowed all interpreters their accurate discrimination from other road categories.
The differences between people with and without local knowledge reflect differences between these groups in spatial memory of their environment.People form spatial memories through sensory perception (particularly vision) of their environment while traveling through an area.Spatial memories are thought to be summarized in cognitive maps in the hippocampus [22].Those intensively travelling, like taxi drivers, tend to have superior spatial cognition and a more than average developed hippocampus [19,20].Having a developed spatial memory of one's environment is most likely the basis of the reported superior classification by locals.
Are we allowed to extrapolate this finding to other urban areas?We expect local people to deliver superior road maps in other cities also, as long as they have local expertise.Local knowledge might have a similar classification accuracy enhancing effect for topographic map features other than roads, for example, built up features such as churches or schools, because the successful identification of their true class identity, which would be difficult from aerospace imagery alone, could be much improved by local cognition.We therefore suggest that local experts are likely to outperform surveyors without local knowledge in classifying other features displayed in topographic maps.
Our results suggest that there is reason to involve local experts in road mapping.The better accuracy achieved by those with local knowledge is one reason for this.A second argument to open up or delegate road mapping, is the possibility to reduce, or altogether avoid, the effort and associated cost of professionals executing field surveys to acquire the knowledge to produce accurate maps, because that knowledge is apparently already available among the local population.A final argument why local engagement requires serious consideration is that road maps, when mapped at lower costs, could be updated more frequently, thus ensuring they are maintained up-to-date.Taken together, these arguments lead us to conclude that engagement of local knowledge would offer the possibility to maintain the up to date and accurate road maps required to address the challenges of rapid urban and peri-urban change.
Does this imply delegating the entire road mapping process to locals?We don't think so.It is true that locals performed better in the road classification, but road mapping involves more than classification.It also involves the pre-processing and geometric registration of imagery, the definition of road categories and standards, the delineation of road segments and quality control of the end products.Local experts do not have the expertise to perform these other tasks.They perform, however, for reasons explained above, excellently in image classification and there is therefore justification to delegate this particular task to local experts, while leaving other tasks which require a more technical background, to more qualified parties.
Web based mapping techniques that combine the provision of geo-referenced imagery and a user friendly mapping environment are still in their infancy.The technology allowing lay men to volunteer in geo information production (VGIP) are rapidly expanding beyond active VGIP, where volunteers develop maps, to more reactive VGIP.Commercial firms like TeleAtlas, Navteq and TomTom each use web based customer input to locate and qualify mapping errors and feature updates required in their road network databases.There are several social networking and collaborative web based efforts like Open Street Map, Tagzania, Wayfaring.com,the People's Map, and Platial.The People's Atlas now enables experts and amateur enthusiasts alike to create and share limited, theme oriented geo-information [14].Improved access to such web based mapping initiatives will globalize the place where mapping can be performed and ultimately enable everyone wishing to produce maps to do so, including local authorities and communities in the developing world.
The development of web based volunteered mapping, challenges the primacy of national survey organizations in topographic map making.One possible reaction to this, which appears to prevail in some countries [23] would be to prohibit mapping by communities and limit mapping to geodetic engineers.Such policy would be difficult to implement however in an open society like Kenya, where maps can be produced by anyone with access to web based computing facilities.Because of this, we suggest that it is time for surveying organizations to review their role and that of the layman in topographic mapping and the provision of geo information to society.
Subsequent research should focus on three issues.The first is the analysis of governance conditions that would allow an administrative decentralization of the mapping process, based on the encouraging results of this research.The second is the analytical understanding of how geo-information is co-produced by alliances of mapping agencies and spatially cognitive lay volunteers.It would consider this in the context of voluntarism as part of broader public governance innovations, underpinning the co-production of social services by state and non-state actors [24].The third is the analysis of different forms that the governance of 'co-production' can take, in comparative, empirical, studies around the world.
Engagement of volunteers is core to co-production and of crucial importance therefore are the different motivations volunteers may have to engage in co-production of geo-information.According to cultural theorists [25,26], different motivations derive from a limited number of different ways-individualist, hierarchist and egalitarian-human actors have of perceiving reality.Each of the ways humans have of perceiving reality is aligned to different values and motivations.Human actors' values and motivations shape their preferred governance styles.Cultural theory has shown empirically that 'clumsy' but viable governance arrangements among individualist, hierarchist and egalitarian actors can emerge, when all voices are included in the struggle and the contestation is harnessed to constructive, if noisy, argumentation [27].The cultural theory lens holds promise for effective comparative empirical research on the governance of geo-information co-production.
Our results thus indicate that there is compelling reason to consider engaging volunteers in the mapping activities of surveying organizations.Engagement of volunteers will release a thus far untapped capacity to maintain highly accurate and current spatial information on road and other infrastructures and benefit society, since better mapping services can be achieved at reduced costs.We therefore suggest that, from a societal perspective, it is worthwhile to actively stimulate voluntarism in the co-production of geo-information.

Figure 1 .
Figure 1.Part of the GeoEye Digital Globe high resolution satellite image of 20 July 2009 used by image interpreters, with road segments coded by number.

Figure 2 .
Figure 2. The 52 road segments in Homa Bay used in this research: (A) Ground truth; (B) Classification by a local taxi driver; and (C) Classification by a teacher without local knowledge.Misclassified road segments in red.

Figure 3 .
Figure 3. Urban population size and year of the most recent topographic map for 70 Kenyan cities.

Figure 4 .
Figure 4. Classification accuracy (with standard error) for interpreters with and without local knowledge and with (grey) and without (white) surveying skills.

Figure 5 .
Figure 5. Road classification accuracy (proportion correctly classified and standard error) for: (a) interpreters with (grey bars) and without (white bars) surveying skills; and (b) interpreters with (grey bars) and without (white bars) local knowledge.See Table2for road category classification.