Share Our Cultural Heritage (SOCH): Worldwide 3D Heritage Reconstruction and Visualization via Web and Mobile GIS

Abstract

Despite being of paramount importance to humanity, tangible cultural heritage is often at risk from natural and anthropogenic threats worldwide. As a result, heritage discovery and conservation remain a huge challenge for both developed and developing countries, with heritage sites often inadequately cared for, be it due to a lack of resources, nonrecognition of the value by local people or authorities, human conflict, or some other reason. This paper presents an online geo-crowdsourcing system, termed Share Our Cultural Heritage (SOCH), which can be utilized for large-scale heritage documentation and sharing. Supported by web and mobile GIS, cultural heritage data such as textual stories, locations, and images can be acquired via portable devices. These data are georeferenced and presented to the public via web-mapping. Using photogrammetric modelling, acquired images are used to reconstruct heritage structures or artefacts into 3D digital models, which are then visualized on the SOCH web interface to enable public interaction. This end-to-end system incubates an online virtual community to encourage public engagement, raise awareness, and stimulate cultural heritage ownership. It also provides valuable resources for cultural heritage exploitation, management, education, and monitoring over time.

1. Introduction

Cultural heritage (CH) has always been a vital part of humanity. It reflects the development of human societies throughout history, providing humans with invaluable legacy and spiritual values from their ancestors. As declared by the United Nations Educational, Scientific, and Cultural Organization (UNESCO) in 2013, culture should be centered in sustainable development policies [1], and CH was thereafter introduced in the Sustainable Development Agenda. CH has provided a source of tourism for many countries, both developing and developed. According to a World Travel and Tourism Council report, the direct contribution of travel and tourism to global Gross Domestic Product (GDP) will grow at an average rate of 3.9% per year over the next ten years [2]. By 2027, the sector is expected to support more than 380 million jobs globally, implying one in nine jobs across the world. Travel and tourism will account for 11.4% of global GDP, outperforming the global economy. However, CH has not only great economic value, but also spiritual and educational functions. Physical heritage assets, such as monuments, architectural buildings, sculptures, paintings, and archaeological sites, known as tangible CH, serve human social activities. Even small heritage artefacts can be significant sources for education.

Despite its paramount importance, CH is often under both natural and anthropogenic threats from human conflicts, urbanization, deterioration, disaster, tourism, mismanagement, etc. [3]. Therefore, significant effort has been put on the preservation of CH, including documentation, restoration, and reconstruction of artefacts, monuments, and sites. However, there are still many challenges when it comes to heritage conservation, including technical, practical, and political issues. This is especially the case in developing countries where the value of heritage is often underestimated and resources to safeguard it are limited.

Among the various challenges in heritage documentation, one major difficulty is the allocation of limited resources for heritage at different levels of importance or urgency. Less than one thousand CH sites are listed as UNESCO world heritage sites, which are documented and preserved to a recognized standard. The vast majority are thus unlisted, and hence rely on the local operating authorities to determine the effort of preservation. Some important heritage structures, even those that have already been identified and recognized, are not well preserved. Moreover, not all heritage assets are located in easily accessible urban areas, and those located in remote areas might not have been recorded, even where they hold great value to local people [4]. Some heritage sites only consist of a single structure, such as a temple, monument, or statue, and thus can easily be missed out in any attempt to catalogue heritage. Huge resources are needed to identify, locate, and document such unattended heritage treasures.

Another challenge lies in the monitoring of heritage structures. Depending on the construction material used, some structures suffer from rapid natural decay, especially under a changing climate regime. Other major impacts can occur as a result of human misbehavior, e.g., destruction, overexploitation, or mismanagement. Therefore, heritage assets need to be regularly monitored and maintained. This means that a one-time documentation is unlikely to fulfil the preservation task. Instead, repeat observations are necessary in order to continually monitor the status of heritage.

Reality-based documentation is a common practice, where the physical objects are measured, mapped, modelled, and recorded in situ. Terrestrial laser scanning is one of the most popular technologies for such tasks since data acquisition is rapid and exempt from poor light conditions, and results in high data quality in terms of geometric accuracy [5]. Modern terrestrial laser scanners also incorporate cameras so that the visual appearance of a heritage object can be recorded. However, such scanners are normally relatively expensive and training is required to use such instruments to acquire and process data. Another popular approach to reality-based heritage documentation is photogrammetry [6], which has appealing advantages over laser scanning, such as portability, low cost, ease of use, photorealism, and ubiquity. With recent developments in photogrammetric computer vision, such as structure from motion and dense image matching, the whole reconstruction process can be fully automatic and the quality of 3D models suitable for documentation purposes. According to Altman et al. [7], the difference between models generated from a digital camera and a high-end laser scanner is around 2~5 cm. Indeed, even images from modern mobile phones can produce comparable models to laser scanners [8]. The recent advancements in low-cost photogrammetry is therefore potentially of great value for 3D heritage documentation. As both photogrammetry and laser scanning have their pros and cons in terms of 3D modelling, the fusion of those two technologies is sought wherever possible [9,10,11,12,13,14]. The fusion has been proved to improve model accuracy [11] as well as readability [13].

In order to effectively record and preserve heritage assets at a large scale on a regular basis, appropriate data acquisition techniques are required. In such a case, individual effort is not sufficient to survey the world’s scattered, hidden, or uncaptured heritage structures. However, community or citizen participation in heritage digitization and documentation can contribute significantly to heritage preservation [15]. In this context, open crowdsourcing is an ideal technique to enable the capture of CH data from all across the globe. Nowadays, mobile phones with inbuilt cameras have become ubiquitous. Mobile phone-enabled online crowdsourcing can therefore help to collate massive amounts of photos from the global population and, more importantly, from those with knowledge of local heritage, which will help both heritage protection and exploitation.

Crowdsourcing has become increasingly popular in recent years, with many crowdsourcing websites now available for different purposes. Goodchild [16] reviewed the phenomenon of web-based crowdsourcing of geospatial information, such as the well-known volunteered online map, OpenStreetMap. Citizens, empowered by webGIS, are considered as sensors to create global geographic information. Alt et al. [17] reviewed various characteristics of crowdsourcing techniques that are complex in nature and difficult to solve by computer. Heipke [18] reviewed developments in crowdsourcing of geospatial data and explained the basic technology required. Brovelli et al. [19] discussed public participation or volunteered geographic information via mobile applications. Most current geo-crowdsourcing projects focus explicitly on geospatial data that can be integrated into a 2D base map. However, not all crowdsourced data can be processed in such a way, for instance, textual data and images.

This paper reports the development of a web- and mobile-based geo-crowdsourcing platform, empowered by web-mapping for data visualization, aiming to document and share CH worldwide. The platform is known as Share Our Cultural Heritage (SOCH), a simple name adopted to encourage public participation. Heritage data, such as textual information, location, and imagery, can be acquired by portable devices, which are able to capture geotagged images. Those data are not strictly spatial data and hence need to be handled differently instead of projecting them onto a 2D basic map. In SOCH, images are categorized for documentation and further processed to generate 3D models, which are then visualized for dissemination and visual-inspection purposes. A preliminary web-based prototype [20] is presented to demonstrate the concept of a CH sharing system. Built upon the preliminary work, this paper presents an end-to-end system with the following main functions: (1) heritage data collection through a web interface that also presents uploaded data by web-mapping; (2) user database management and quality control via public voting; (3) server-side 3D heritage modelling from crowdsourced images using photogrammetry; (4) web-based interactive visualization of the reconstructed 3D models; (5) access to heritage locations and 3D models through open Application Programming Interfaces (APIs) to allow integrations into other web applications; and (6) an Android application that is compatible with portable devices to allow multiplatform crowdsourcing so that the system can benefit from citizen sensors.

2. Related Work

The concepts of crowdsourcing and citizen science have been used for many volunteered geographic information projects, especially in volunteered mapping supported by web and mobile GIS (Geographic Information System). OpenStreetMap [21,22] is a prominent example through which the public is encouraged to upload their traces to the web and to edit the public map to keep it up to date. Methods are developed to merge crowdsourced road segments from different users [23]. Another similar project is Wikimapia, where users can draw content and annotate on the base Google map. It is reminiscent of Wikipedia, which is similarly built upon the idea of crowdsourcing. Instead of focusing on vector maps, Geo-Wiki is an online crowdsourcing system to validate and improve global land cover maps [24]. There are discrepancies among three available global land-cover datasets, GLC-2000, MODIS, and GlobCover. The online system aims to increase the amount of ground measurements for validation so as to fuse those data into a more accurate map. The need for feedback and interaction for community building is highlighted.

Apart from spatial mapping, mobile and web-based crowdsourcing techniques have also been applied in disaster studies. Goodchild and Glennon [25] discussed the benefits of using crowdsourcing geographic information for disaster response. An example of four wildfires in Santa Barbara suggested that the local community could contribute effectively to the response, even though the Volunteered Geographic Information (VGI) contents needed further verification. Riccardi [26] suggested that all countries can benefit from crowdsourcing based on case studies of two Colorado wildfires and the 2010 Haiti earthquake. Crowdsourcing is a tool that is readily available to disaster mangers, and it does not require great amounts of investment in technology. People are now living in a connected world and the technology is already in the hands of people; however, it was argued that investment in trained volunteers who could provide valid data was an essential requirement. Olyazadeh et al. [27] developed a mobile GIS application for rapid landslide data collection, such as landside type, feature, volume, date, and patterns, so that landslide inventory maps can be created with the help of the general public.

There are also a few existing projects using the concept of crowdsourcing for heritage documentation. One example is Rekrei [28], formally known as Project Mosul because of the destruction of heritage objects in the Mosul Museum in Iraq. Here, imagery of heritage objects is uploaded through the website by volunteer participants before it is sorted and processed to generate 3D models using photogrammetric reconstruction methods. A 2D map is used to indicate the locations and number of mapped heritage sites, but no data or results are visualized or presented to the public. Another heritage-related crowdsourcing project is Curious Travellers [29], aiming at the documentation of archaeological sites, monuments, and heritage at risk. Further crowdsourcing projects can be found for heritage documentation and protection, such as the Million Image Database [30] and Heritage Together [31]. However, these projects have not yet harnessed mobile- or web-GIS techniques to enhance the geo-crowdsourcing process or to present the heritage data.

Mobile-based crowdsourcing has been used in many applications, mostly related to geographic information collection. Mahmud and Aris [32] reviewed the state of mobile crowdsourcing applications from both research communities and industry. Examples include indoor building mapping, hazard reporting, child abuse reporting, flood reporting, traffic condition sharing, sound recognition, etc. UN-ASIGN Crowd [33] is a mobile application that is developed to facilitate the collection of photos and damage assessment of CH. Indeed, through the ubiquity of smartphones and the internet, rich heritage data, e.g., image, location, history, status, and damage degree can be collated. However, no information was shared with the public in return by this mobile application. By the combination of web-mapping and crowdsourcing, a mobile application can be developed for not only heritage documentation, but also for heritage information dissemination [15,20,28,34].

Crowdsourced data can be directly shared with the public through web GIS. However, there will be added value if they can be further processed so that people can better interpret the information. Karl et al. [31] developed tools to crowdsource heritage imagery from communities and to create 3D models of the heritage sites and objects using photogrammetry. Both images and 3D models are documented in the local Historic Environment record, which can be used for heritage change monitoring and assessment, and provide local people and tourists alternative views of the CH. The availability of a large amount of online images has been harnessed to create 3D models [35]. Images were taken directly from the web and were not quality controlled, meaning they were of different resolutions, taken from various perspectives by different people using a variety of cameras. Nevertheless, visually appealing models were generated automatically using the structure from motion algorithm. The advancement in image-based 3D modelling provides the basis for worldwide 3D heritage documentation, for which an efficient image crowdsourcing mechanism is sought.

In summary, mobile- or web-based VGI systems have been developed to support various applications, including heritage documentation, and crowdsourced or online public data have been used to model heritage assets in 3D. However, there is currently no reported system that takes full advantage of mobile and web GIS in terms of data crowdsourcing, management, and visualization. In this paper, we propose the first end-to-end online sharing system that not only collects heritage data from users, but also processes collected images into 3D modelling and visualizes the data interactively on the web. Users are therefore both contributors and beneficiaries of this online sharing system.

3. Conceptual Design

3.1. System Conceptual Framework

SOCH is designed as an end-to-end framework that the public can both contribute to and benefit from. It crowdsources heritage data directly from the public and serves as an information hub, easily accessible by the general public for different purposes. To make the system useful for multipurpose users, multisource data are collected, some of which will be further processed, e.g., 2D images will be used to generate 3D models. Derived models can then be used for documentation, structural inspection, interactive visualization, education, etc.

Figure 1 shows the conceptual design of the SOCH system that creates a bridge between the public and heritage by providing a convenient platform that everyone can participate in, and a database that different users can take advantage of. Users who contribute to the SOCH system might be tourists, local residents, heritage asset managers, urban planners, public authorities, etc. Tourists will only visit limited numbers of selected heritage sites, which are normally well-known and well-preserved due to their perceived high value. Other heritage assets that are less well-recognized or only known to local people will need the effort of those people who value it. Heritage managers, who have daily access to heritage assets, may have more comprehensive and continuous recordings in terms of both textual information and physical representation. Heritage data include not only the textual metadata, such as the history and ownership, but also the background stories and its associations with people. Structural information, e.g., the degree of damage, signs of deterioration, and construction material, can also be collected for structural analysis or damage assessment. Moreover, digital recordings, e.g., imagery and video, can be used for 2D recording and for 3D reconstruction and documentation using photogrammetry. Since 3D photogrammetric reconstruction will need a relatively large number of photos taken from different perspectives, public participation is particularly important. 3D models provide added value to heritage documentation and restoration. The idea is to create a virtual data sharing community for heritage conservation and promotion, supporting 3D modelling and visualization.

3.2. Conceptual Data Flow

The SOCH system allows multilevel user interactions. A user can upload their images or other heritage data to the system, and manage their own datasets. In addition, a user can view and interact with other users’ databases, such as add further data to an existing heritage item, or report spam or invalid images. Moreover, the user can browse and search the existing heritage database by either searching for the heritage names or by browsing the web map to the respective geolocations. Given a heritage structure of interest, the user can retrieve the heritage information and view the 2D imagery and derived 3D models interactively. The system can also support free data downloads if permission is given by the data owner and authorized by the heritage manager.

The system is targeted at general crowdsource users with the aim of bringing them all together regardless of their background, skills, knowledge of heritage, or their location of interest and residence. It is anticipated that data will be gathered from users and information provided to the public anytime and anywhere.

Through this platform, a user can upload a series of captured photos of the heritage structure and create a unique photo album with textual information. Further information, such as degree of damage to a structure, can also be collected by the platform. The photos can be examined by the platform itself through image screening and by public users through a voting mechanism. Users are also able to review their own photo albums. Albums are constantly updated and photos are screened by their size and resolution. In the feedback process, the user is allowed to submit additional photos as required. This process of collecting feedback and improving photo quality is iterated. The whole concept and process flow is illustrated in Figure 2.

The platform prototype was developed considering various aspects such as user interactivity and user friendliness, helping define essential features for general crowdsource users. The whole system has been designed and developed under free and open source systems (FOSS).

4. SOCH Architecture Development

The SOCH system architecture, with the key building blocks of SOCH, is depicted in Figure 3. The SOCH platform is designed and developed using FOSS tools and techniques and consists of three main components: (1) Database management system server as the physical layer in which crowdsourced information and the heritage library are stored and managed; (2) geospatial and photo server as the application layer through which heritage data and geolocations are processed and served; and (3) web interface as the presentation layer where users interact with the system to upload photos, manage and view heritage data with the use of web mapping and web visualization techniques. This presentation layer is further split into web and mobile platforms. Each of these layers is discussed in the succeeding sections.

4.1. Physical Layer

This is the bottom layer of the system and is the foundation for the application and presentation layers. This layer provides data infrastructure for higher-level layers, and holds various databases such as administrative, heritage (e.g., photos and 3D models), and spatial data guided by data standards and sharing policies, and is treated as a central and integrated geospatial database.

The geodatabase server in this layer takes care of data storage and management. Two open-source spatial database servers, MySQL and PostgreSQL/PostGIS, are popular for database management systems (DBMS). Although both database systems are very powerful in spatial support systems, PostgreSQL/PostGIS has a larger range of spatial functionality and is used by many recent Web-GIS solutions [36,37,38,39]. This geodatabase server was therefore selected for system implementation.

4.2. Application Layer

The main purpose of this layer is to host the SOCH System. The system accepts a variety of requests and queries submitted by web- and mobile-client users, and manages the underlying geospatial database, metadata, and map servers. Here, the database server manages data, the metadata server manages data/information about data, and the map server renders data to create dynamic web maps, which are then published for web integration.

The map server bridges the database server and web browsers, making it possible to access and display spatially enabled data as a map layer and enable querying and the process of displayed data. MapServer and GeoServer are both similar types of open-source map servers. GeoServer was chosen due to its ease of administration to aid in testing and system development.

Moreover, this layer is setup with a photogrammetric processing module that is used to process 2D images to construct 3D models using photogrammetry. Heritage photos are stored as individual photo albums in the photo database on which the module is executed. The photogrammetric process can be fully automatic due to recent advancements in structure from motion and multiview stereo. The images ought to cover different parts of the structure whilst having certain degrees of overlaps. Common points in the overlapped areas will constraint the relative positions and orientations of overlapped images (i.e., extrinsic parameters). Then, all images can be relatively fixed with respect to an initial arbitrary image. Those relatively oriented images can then be used to reconstruct the 3D object by densely matching corresponding pixels in each image. There are open-source photogrammetric packages such as MicMac, Colmap, and OpenMVG. However, the system is currently using a proprietary software, Photoscan (Agisoft LLC; St. Petersburg, Russia), via Python scripting to ensure high quality reconstruction.

The SOCH web system is developed in a PHP-based platform, which is a server-side scripting language, and it provides several web services used by the web application and the mobile clients. All crowdsourced albums associated with geographical information are integrated with the web system for web mapping and visualization.

4.3. Presentation Layer: Web Platform

This layer disseminates the results of the application layer in various ways, such as album lists, photo viewers, album mapping, 3D viewers, etc., over a user friendly and interactive graphic user interface intuitively for different levels of users.

The geo-crowdsourcing system is an AJAX-based web platform that enables distribution of tasks to wider crowd users. The platform allows users to upload photos and share them with the wider community, or search for an existing heritage album from the search bar. Crowd users can also vote on other users' photos for quality control purposes. This allows, for example, the avoidance of duplication of photos for the same structure taken from the same location or the rejection of photos where unnecessary objects are blocking the heritage of interest. The prototype of the web platform is shown in Figure 4.

The overall web-platform interface consists of three main elements: (1) base map; (2) user album manager; and (3) photo viewer. By default, all crowdsourced albums are tagged onto the map and all registered users can visualize all submitted photo albums on the map. Whilst creating a photo album, the geolocation of the album can be set independently from the current location. Moreover, the system allows any user to search, view short details, and view photos in the photo viewer of each album. Furthermore, the owner of the album is allowed to manage albums, such as editing the album’s details, delete unused albums, and delete photos, if needed. Figure 5 shows the view of images in a photo album of the previous user (Figure 4) and the function of casting a vote on the images. The total number of votes is shown on top of each image, and it is used to eliminate spam or images that are not useful for 3D reconstruction.

The albums are stored in the photo database in the physical server. Images in the album are processed to reconstruct a 3D model using the 3D photogrammetric modelling module in the processing/application server. Then, an online 3D model viewer is integrated to publish and visualize 3D models through the web platform for public access.

4.4. Presentation Layer: Mobile Platform

The platform has been extended with similar web functionality to mobile clients that can cater to larger crowdsource users who prefer to use, or only have access to, mobile devices. This mobile client application will help to collect numerous photo albums across the world (Figure 6).

Currently, the mobile application is only available on the Android platform due to limited development resources. The development of applications for other mobile operation systems is planned as future work. The mobile application has most of the web functionality, such as heritage data uploading, album management, image viewing, and spam reporting. Only the visualization of 3D heritage models is not supported, because practically the download of the 3D models requires a large amount of cellular data and the visual quality is limited.

5. Prototype System Usage Examples

5.1. Kathmandu Valley, Nepal

Nepal is a country rich in CH. In Kathmandu Valley, a UNESCO listed heritage site, many heritage assets can be found, including royal palaces, Buddhist monuments, and Hindu temples. The tiered temples, for example, are usually made of bricks and timber beams, with rich ornaments. The windows and roof struts have delicate decorative carvings. The monuments and temples reflect the prosperity of Buddhist and Hinduist religious culture in the region, and demonstrate unique craftsmanship in construction and timber-carving.

During the 2015 Gorkha earthquake, many heritage assets were damaged, while others were destroyed. According to UNSECO [40], about 2900 heritage structures in the Kathmandu Valley and the northwestern region of Nepal were affected. About 700 were damaged, among which 131 were completely destroyed. Drones were flown above heritage sites to capture images to help estimate the damage for restoration planning. However, only a few sites were surveyed and the contribution of aerial images for damage assessment was limited. A number of issues were identified after the earthquake in terms of heritage protection and impact mitigation, including the lack of periodic maintenance, the importance of heritage documentation and community engagement, as well as the seismic strengthening of the structures.

In such cases of natural disaster, the SOCH system can be used to collect first-hand heritage information, such as structure properties, damage levels, and images. Firstly, the crowdsourced information can be used to provide a preliminary assessment of overall damage across the affected region. Secondly, it will enable visual inspections of heritage structures for structural analysis by experts to identify the degree of damage. Lastly, the imagery can be used to generate 3D models so that further structural assessment can be performed in 3D. Some examples are given in Figure 7.

This example demonstrates the usability of the SOCH system for large-scale heritage 2D and 3D information collection. It can be used for continuous heritage documentation and monitoring through time. In case of disasters, it can be harnessed for damage assessment as well as restoration progress monitoring.

5.2. Newcastle University Quadrangle Gateway, UK

The Newcastle University Arches, as a gateway from King’s Road into the Quadrangle yard, form part of a three-story building, which is an iconic symbol of the University (Figure 8). The building was built in 1911, with a pair of arches through which people can pass through. Above the Arches, a statue of King Edward VII was recessed in a niche. The building was listed as a Grade II building on March 30, 1987.

Most heritage structures, like the Arches, are rather complicated with many details. This means that the documentation has to be precise and fine. To evaluate the quality of crowdsourced image-based 3D documentation, the Arches building is taken as an example. The images are taken by a mobile-phone camera. The reconstructed model in the form of 3D structural points is compared with a 3D point cloud acquired by a high-precision laser scanner (Leica HDS P20). As the photogrammetric model is not scaled, meaning the model does not have the real dimensions of the structure, a one-dimensional scale factor would be necessary. The latest crowdsourced reconstruction takes the advantage of object semantics, e.g., height distribution of people in the scene [41]. For heritage structure documentation, if the dimensions are necessary, archival data including at least one dimension or a tape measurement are suggested to be crowdsourced.

The two different point clouds were approximately aligned by manual editing, then coregistered by minimizing the point-to-point distances using iterative closed point (ICP). Then, the point-to-point distance is considered as the difference between the two modelling techniques. The comparison result showed that more than half of the photogrammetric points were within 2 cm to the laser-scanning points, and the majority of them (83.6%) were under 5 cm. Readers are kindly referred to Reference [8] for full details of the comparison. This means the reconstructed model from images can be used for both 2D appearance and 3D geometry documentation at a reasonable precision. The geometric dimensions of documented structures can be used as a guidance for restoration or reconstruction purposes.

6. Discussion

The prototype SOCH system demonstrates the applicability of both mobile and web GIS in the field of heritage documentation and promotion. SOCH is an interactive system that retrieves data from the public and feeds the public back with rich, crowdsourced, and further derived information via mobile and web GIS. This concept has been used, and will continue to be used, in many other fields, such as environment mapping and natural hazard reporting. Data with geolocation information can be collected and managed by GIS, with location and geospatial information often being of great importance. In the case of natural hazards, such as flooding or landslides, timely crowdsourced data with location information can help support rapid response and allocation of resources [25]. As often happens after major natural disasters, volunteers often rush into the afflicted site with aid donations. Additional sources are needed to manage those volunteers and donated goods. A similar mobile- and web-GIS crowdsourcing system can help in such situations. All donation information, such as good types and locations, can be gathered and managed in an online information system. Optimized allocations can be computed based on the real shortage and distance to the site, for example.

The SOCH system crowdsources heritage data, and also constructs 3D models of heritage structures or objects from 2D images. Further analyses can be carried out based on those data, such as visual inspection of structural health for those that have suffered damage or decay. Moreover, multitemporal data can be investigated to analyze the evolution of heritage structures through time. The system can also serve as an online platform for community engagement. A local community is able to contribute to the data collection, and to benefit throughout the process of sharing, for instance, in discovering unknown recordings or histories. Crowdsourced information will facilitate heritage promotion and enrich education materials. The SOCH system can be used by local communities to promote local heritage to younger generations and people in the wider world. SOCH focuses on heritage information collection and dissemination. Once there is sufficient information, local authorities can integrate them into a more comprehensive spatial information system [42] for heritage management or research purposes.

The proposed SOCH system is an end-to-end crowdsourcing system that maximizes the interaction with the crowd. Nevertheless, there are still a few aspects that can be improved, such as privacy, data quality control, and hosting of other types of heritage data, such as videos, which can represent both tangible and intangible heritage. Currently, the crowdsourced images are screened by the system based on simple criteria. Security and abuse issues have to be considered, and users should be able to choose whether their photo albums are visible to the public or only to other registered users. Advanced image processing and retrieval methods are to be introduced to improve image filtering and categorization. Heritage images are visualized on the website based on their geolocations. Multiple users may present the same heritage structure but have slightly different geolocations due to imperfect geolocalization. A location buffer is therefore needed to merge albums that are located sufficiently close to each another. Another possible way to improve this is through image matching. For example, features may be extracted from each of the albums that are in close geospatial proximity to each other, and then they can be merged if there are sufficiently similar image features.

Currently, the platform only collects data from registered users in a passive way. There are actually masses of data available to the public already, such as from image hosting websites Flickr and Google Earth. Data from these sources can also be harnessed to enrich the heritage database, even if they are not heritage focused. When there are sufficient images collected for a heritage structure, they can be used as seeds to retrieve additional images, or even videos, from online databases using image retrieval technologies. Apart from 2D images, other types of heritage-related data, both tangible and intangible, could also be collected. Most of the time, there is intangible culture attached to tangible heritage, especially for spiritual structures. Video is often a better form to present intangible heritage and is easier to attract attention from a wider audience. Textual information, such as background stories linked to a specific heritage structure, will help the public to better understand the value of their heritage.

One functionality of the SOCH system is the interactive online visualization of 3D heritage models. Crowdsourced images are used to automatically reconstruct 3D models on the server. Serval open-source and free photogrammetry software has been tested, such as MicMac, Colmap, OpenMVG. The advantage is that those software packages can be fully integrated into the system for automatic 3D reconstruction using shell-controlled command lines. However, none of them performing sufficiently well compared to proprietary ones such as Contextcapture or Photoscan. Colmap would also require a high-quality Graphics Processing Unit (GPU) for dense reconstruction. Therefore, the system is currently using Photoscan for 3D modelling via Python scripting. The 3D models are in the form of point clouds and are visualized online using Potree. Further development will exploit online visualization in virtual reality mode, which will provide an immersive and appealing experience that can be used for enhanced virtual tourism.

In addition, 3D point cloud models obtained from the SOCH system could be used for the structural documentation, inspection, and analysis of historic structures. Their complex geometries inform that it is imperative to capture as accurately as possible their geometries and represent these in advanced numerical methods for their structural analysis. Over the last few years, significant research focus has been put on the development of semiautomated and fully automated procedures to transform three-dimensional point clouds of complex objects into a three-dimensional finite- and discrete-element geometric models for the structural analysis of historic structures [43,44,45]. By coupling the SOCH system with the current engineering approach for structural inspection and assessment using point clouds, a high degree of automation at each operational level can be obtained and outcomes of this approach could be beneficial for engineers and asset managers who want to classify and assess the structural condition of their historic assets and device action plans.

7. Conclusions

Existing crowdsourcing heritage conservation projects primarily focus on one particular aspect, such as online crowdsourcing, 3D reconstruction, or mobile application. The prototype SOCH system reported in this paper takes full advantages of mobile- and web-GIS technologies, such as geo-crowdsourcing, web-mapping, as well as photogrammetric modelling and online visualization technologies, to provide an end-to-end user experience. It enables public engagement for both heritage data collection and sharing, allowing participative mapping and interaction by adding and viewing heritage information. Moreover, a mobile application has been developed so that users will be able to interact with the platform separately from the web interface, which makes the platform ubiquitous and more accessible globally. Collated photos are used for both visual inspection and 3D reconstruction, and the resulting 3D models are visualized on the web. Users from around the world will be able to contribute to, and benefit from, the system that can be a web-based resource-sharing hub for tourism, conservation, education, and research purposes. In addition, 3D point cloud models obtained from the SOCH system can also be used for structural documentation, inspection, and analysis of historic structures.