Special Issue "Machine Learning and Deep Learning in Cultural Heritage"

A special issue of ISPRS International Journal of Geo-Information (ISSN 2220-9964).

Deadline for manuscript submissions: closed (1 December 2021) | Viewed by 12986

Special Issue Editors

Dr. Susana Del Pozo
E-Mail Website
Guest Editor
Cartographic and Land Engineering Department, Higher Polytechnic School of Avila, University of Salamanca, Hornos Caleros, 50, 05003 Avila, Spain
Interests: multisensor multi-source data analysis; satellite imagery; geographic information systems; crop mapping; irrigation activity detection; remote sensing
Special Issues, Collections and Topics in MDPI journals
Dr. Jan Dirk Wegner
E-Mail Website
Guest Editor
EcoVision Lab, ETH Zurich, Switzerland
Interests: deep learning for geospatial data analysis; large-scale machine learning; 3D computer vision
Special Issues, Collections and Topics in MDPI journals
Mr. Lloyd A. Courtenay
E-Mail Website
Guest Editor
Cartographic and Land Engineering Department, Higher Polytechnic School of Avila, University of Salamanca, Hornos Caleros, 50, 05003 Avila, Spain
Interests: data science; machine and deep learning; applied statistics; quaternary sciences; laser scanning; archaeology; taphonomy; human evolution; African heritage

Special Issue Information

Dear colleagues,

Digital and computer transformations not only lower costs for technologies and services, but also save time when improving final products and results. Specifically, machine and deep learning are two powerful tools that are transforming the face of many sectors, from medicine to physics, humanities, engineering, and many others. The components of machine learning prepare computers using a multitude of different algorithms to learn from large amounts of complex data to extract discriminative evidence for efficient decision-making. Algorithms currently excel in high-level feature extraction and pattern recognition tasks, such as image and natural language processing or classification. While they remain unknown to many, these algorithms now form part of our daily lives and are achieving revolutionary results in most fields of science.

In this context, it is essential to analyze the versatility and potential that these techniques have in the cultural heritage (CH) sector, in which the analysis of vast amounts of highly complex information is key. Diagnostics and preservation of CH are truly important to determine the state of conservation of historical monuments and buildings. This sector needs new solutions in order to objectively and efficiently manage the vast amount of information, usually in image or point cloud format, regarding the documentation and analysis of our cultural legacy. Efficient and accurate modern machine learning methods can be viewed as complementary to social sciences and humanities, providing powerful tools for analytical as well as didactical techniques. Machine learning excels in processing large, complex data, removing a significant degree of error which often the product of arguably subjective human input. In this regard, new challenges arise in order to apply computer technologies to the study and preservation of CH assets.

This Special Issue originates from the CIPA Symposium “CIPA 2019—Documenting the Past for a Better Future”, held in September 2019 in Avila, Spain. One of the main symposium’s scope is to bring together scientists, developers, and advanced users who apply sensors and methods in CH. Additionally, a special focus will be placed on the use of complex deep learning algorithms, capable of reaching the highest degrees of precision and resolution when processing both human-obtained data and images, which are typical of most CH projects. The most exciting and innovative papers related to machine and deep learning presented at the symposium will be selected to be extended and included in this Special Issue. In addition to this, we invite you to contribute to this Special Issue by submitting articles on your recent research, experimental work, reviews, and/or case studies related to the field of artificial intelligence applied to CH.

Relevant topics include, but are not limited to:

  • Robotic technologies applied to cultural heritage;
  • Monitoring heritage through time;
  • Cultural heritage diagnostics;
  • Impact of conservation tasks;
  • Virtual and augmented reality;
  • Automatic feature extraction in ancient buildings;
  • Image classification;
  • Improvements in artificial intelligence models and methods.

Dr. Susana Del Pozo
Dr. Jan Dirk Wegner
Mr. Lloyd A. Courtenay
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. ISPRS International Journal of Geo-Information is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1400 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • cultural heritage
  • computer vision
  • artificial intelligence
  • big data
  • machine and deep learning
  • neural networks
  • feature extraction and classification
  • monitoring
  • conservation
  • statistics

Published Papers (8 papers)

Order results
Result details
Select all
Export citation of selected articles as:

Research

Article
Extracting Spatio-Temporal Information from Chinese Archaeological Site Text
ISPRS Int. J. Geo-Inf. 2022, 11(3), 175; https://doi.org/10.3390/ijgi11030175 - 04 Mar 2022
Viewed by 797
Abstract
Archaeological site text is the main carrier of archaeological data at present, which contains rich information. How to efficiently extract useful knowledge from the massive unstructured archaeological site texts is of great significance for the mining and reuse of archaeological information. According to [...] Read more.
Archaeological site text is the main carrier of archaeological data at present, which contains rich information. How to efficiently extract useful knowledge from the massive unstructured archaeological site texts is of great significance for the mining and reuse of archaeological information. According to the site information (such as name, location, cultural type, dynasty, etc.) recorded in the Chinese archaeological site text, this paper combines deep learning and natural language processing techniques to study the information extraction method for automatically obtaining the spatio-temporal information of sites. The initial construction of the corpus of Chinese archaeological site text is completed for the first time, and the corpus is input into the Bidirectional Long Short-Term Memory with Conditional Random Fields (BiLSTM-CRF) entity recognition model and Bidirectional Gated Recurrent Units with Dual Attention (BiGRU-Dual Attention) relationship extraction model for training. The F1 values of BiLSTM-CRF model and BiGRU-Dual Attention model on the test set reach 87.87% and 88.05%, respectively. The study demonstrates that the information extraction method proposed in this paper is feasible for the Chinese archaeological site texts, which promotes the establishment of knowledge graphs in archaeology and provides new methods and ideas for the development of information mining technology in archaeology. Full article
(This article belongs to the Special Issue Machine Learning and Deep Learning in Cultural Heritage)
Show Figures

Figure 1

Article
Deep Descriptor Learning with Auxiliary Classification Loss for Retrieving Images of Silk Fabrics in the Context of Preserving European Silk Heritage
ISPRS Int. J. Geo-Inf. 2022, 11(2), 82; https://doi.org/10.3390/ijgi11020082 - 21 Jan 2022
Cited by 1 | Viewed by 847
Abstract
With the growing number of digitally available collections consisting of images depicting relevant objects from the past in relation with descriptive annotations, the need for suitable information retrieval techniques is becoming increasingly important to support historians in their work. In this context, we [...] Read more.
With the growing number of digitally available collections consisting of images depicting relevant objects from the past in relation with descriptive annotations, the need for suitable information retrieval techniques is becoming increasingly important to support historians in their work. In this context, we address the problem of image retrieval for searching records in a database of silk fabrics. The descriptors, used as an index to the database, are learned by a convolutional neural network, exploiting the available annotations to automatically generate training data. Descriptor learning is combined with auxiliary classification loss with the aim of supporting the clustering in the descriptor space with respect to the properties of the depicted silk objects, such as the place or time of origin. We evaluate our approach on a dataset of fabric images in a kNN-classification, showing promising results with respect to the ability of the descriptors to represent semantic properties of silk fabrics; integrating the auxiliary loss improves the overall accuracy by 2.7% and the average F1 score by 5.6%. It can be observed that the largest improvements can be obtained for variables with imbalanced class distributions. An evaluation on the WikiArt dataset demonstrates the transferability of our approach to other digital collections. Full article
(This article belongs to the Special Issue Machine Learning and Deep Learning in Cultural Heritage)
Show Figures

Figure 1

Article
Improvement of Oracle Bone Inscription Recognition Accuracy: A Deep Learning Perspective
ISPRS Int. J. Geo-Inf. 2022, 11(1), 45; https://doi.org/10.3390/ijgi11010045 - 09 Jan 2022
Viewed by 667
Abstract
Deep learning techniques have been successfully applied in handwriting recognition. Oracle bone inscriptions (OBI) are the earliest hieroglyphs in China and valuable resources for studying the etymology of Chinese characters. OBI are of important historical and cultural value in China; thus, textual research [...] Read more.
Deep learning techniques have been successfully applied in handwriting recognition. Oracle bone inscriptions (OBI) are the earliest hieroglyphs in China and valuable resources for studying the etymology of Chinese characters. OBI are of important historical and cultural value in China; thus, textual research surrounding the characters of OBI is a huge challenge for archaeologists. In this work, we built a dataset named OBI-100, which contains 100 classes of oracle bone inscriptions collected from two OBI dictionaries. The dataset includes more than 128,000 character samples related to the natural environment, humans, animals, plants, etc. In addition, we propose improved models based on three typical deep convolutional network structures to recognize the OBI-100 dataset. By modifying the parameters, adjusting the network structures, and adopting optimization strategies, we demonstrate experimentally that these models perform fairly well in OBI recognition. For the 100-category OBI classification task, the optimal model achieves an accuracy of 99.5%, which shows competitive performance compared with other state-of-the-art approaches. We hope that this work can provide a valuable tool for character recognition of OBI. Full article
(This article belongs to the Special Issue Machine Learning and Deep Learning in Cultural Heritage)
Show Figures

Figure 1

Article
SPPD: A Novel Reassembly Method for 3D Terracotta Warrior Fragments Based on Fracture Surface Information
ISPRS Int. J. Geo-Inf. 2021, 10(8), 525; https://doi.org/10.3390/ijgi10080525 - 05 Aug 2021
Viewed by 876
Abstract
As one of China′s most precious cultural relics, the excavation and protection of the Terracotta Warriors pose significant challenges to archaeologists. A fairly common situation in the excavation is that the Terracotta Warriors are mostly found in the form of fragments, and manual [...] Read more.
As one of China′s most precious cultural relics, the excavation and protection of the Terracotta Warriors pose significant challenges to archaeologists. A fairly common situation in the excavation is that the Terracotta Warriors are mostly found in the form of fragments, and manual reassembly among numerous fragments is laborious and time-consuming. This work presents a fracture-surface-based reassembling method, which is composed of SiamesePointNet, principal component analysis (PCA), and deep closest point (DCP), and is named SPPD. Firstly, SiamesePointNet is proposed to determine whether a pair of point clouds of 3D Terracotta Warrior fragments can be reassembled. Then, a coarse-to-fine registration method based on PCA and DCP is proposed to register the two fragments into a reassembled one. The above two steps iterate until the termination condition is met. A series of experiments on real-world examples are conducted, and the results demonstrate that the proposed method performs better than the conventional reassembling methods. We hope this work can provide a valuable tool for the virtual restoration of three-dimension cultural heritage artifacts. Full article
(This article belongs to the Special Issue Machine Learning and Deep Learning in Cultural Heritage)
Show Figures

Figure 1

Article
POSE-ID-on—A Novel Framework for Artwork Pose Clustering
ISPRS Int. J. Geo-Inf. 2021, 10(4), 257; https://doi.org/10.3390/ijgi10040257 - 11 Apr 2021
Viewed by 869
Abstract
In this work, we focus our attention on the similarity among works of art based on human poses and the actions they represent, moving from the concept of Pathosformel in Aby Warburg. This form of similarity is investigated by performing a pose clustering [...] Read more.
In this work, we focus our attention on the similarity among works of art based on human poses and the actions they represent, moving from the concept of Pathosformel in Aby Warburg. This form of similarity is investigated by performing a pose clustering of the human poses, which are modeled as 2D skeletons and are defined as sets of 14 points connected by limbs. To build a dataset of properly annotated artwork images (that is, including the 2D skeletons of the human figures represented), we relied on one of the most popular, recent, and accurate deep learning frameworks for pose tracking of human figures, namely OpenPose. To measure the similarity between human poses, two alternative distance functions are proposed. Moreover, we developed a modified version of the K-Medians algorithm to cluster similar poses and to find a limited number of poses that are representative of the whole dataset. The proposed approach was also compared to two popular clustering strategies, that is, K-Means and the Nearest Point Algorithm, showing higher robustness to outliers. Finally, we assessed the validity of the proposed framework, which we named POSE-ID-on, in both a qualitative and in a quantitative way by simulating a supervised setting, since we lacked a proper reference for comparison. Full article
(This article belongs to the Special Issue Machine Learning and Deep Learning in Cultural Heritage)
Show Figures

Figure 1

Article
Comparing Machine and Deep Learning Methods for Large 3D Heritage Semantic Segmentation
ISPRS Int. J. Geo-Inf. 2020, 9(9), 535; https://doi.org/10.3390/ijgi9090535 - 07 Sep 2020
Cited by 30 | Viewed by 3074
Abstract
In recent years semantic segmentation of 3D point clouds has been an argument that involves different fields of application. Cultural heritage scenarios have become the subject of this study mainly thanks to the development of photogrammetry and laser scanning techniques. Classification algorithms based [...] Read more.
In recent years semantic segmentation of 3D point clouds has been an argument that involves different fields of application. Cultural heritage scenarios have become the subject of this study mainly thanks to the development of photogrammetry and laser scanning techniques. Classification algorithms based on machine and deep learning methods allow to process huge amounts of data as 3D point clouds. In this context, the aim of this paper is to make a comparison between machine and deep learning methods for large 3D cultural heritage classification. Then, considering the best performances of both techniques, it proposes an architecture named DGCNN-Mod+3Dfeat that combines the positive aspects and advantages of these two methodologies for semantic segmentation of cultural heritage point clouds. To demonstrate the validity of our idea, several experiments from the ArCH benchmark are reported and commented. Full article
(This article belongs to the Special Issue Machine Learning and Deep Learning in Cultural Heritage)
Show Figures

Figure 1

Article
A Neural Networks Approach to Detecting Lost Heritage in Historical Video
ISPRS Int. J. Geo-Inf. 2020, 9(5), 297; https://doi.org/10.3390/ijgi9050297 - 05 May 2020
Cited by 7 | Viewed by 1605
Abstract
Documenting Cultural Heritage through the extraction of 3D measures with photogrammetry is fundamental for the conservation of the memory of the past. However, when the heritage has been lost the only way to recover this information is the use of historical images from [...] Read more.
Documenting Cultural Heritage through the extraction of 3D measures with photogrammetry is fundamental for the conservation of the memory of the past. However, when the heritage has been lost the only way to recover this information is the use of historical images from archives. The aim of this study is to experiment with new ways to search for architectural heritage in video material and to save the effort of the operator in the archive in terms of efficiency and time. A workflow is proposed to automatically detect lost heritage in film footage using Deep Learning to find suitable images to process with photogrammetry for its 3D virtual reconstruction. The performance of the network was tested on two case studies considering different architectural scenarios, the Tour Saint Jacques which still exists for the tuning of the networks, and Les Halles to test the algorithms on a real case of an architecture which has been destroyed. Despite the poor quantity and low quality of the historical images available for the training of the network, it has been demonstrated that, with few frames, it was possible to reach the same results in terms of performance of a network trained on a large dataset. Moreover, with the introduction of new metrics based on time intervals the measure of the real time saving in terms of human effort was achieved. These findings represent an important innovation in the documentation of destroyed monuments and open new ways to recover information about the past. Full article
(This article belongs to the Special Issue Machine Learning and Deep Learning in Cultural Heritage)
Show Figures

Figure 1

Article
Combining Deep Learning and Location-Based Ranking for Large-Scale Archaeological Prospection of LiDAR Data from The Netherlands
ISPRS Int. J. Geo-Inf. 2020, 9(5), 293; https://doi.org/10.3390/ijgi9050293 - 01 May 2020
Cited by 22 | Viewed by 2383
Abstract
This paper presents WODAN2.0, a workflow using Deep Learning for the automated detection of multiple archaeological object classes in LiDAR data from the Netherlands. WODAN2.0 is developed to rapidly and systematically map archaeology in large and complex datasets. To investigate its practical value, [...] Read more.
This paper presents WODAN2.0, a workflow using Deep Learning for the automated detection of multiple archaeological object classes in LiDAR data from the Netherlands. WODAN2.0 is developed to rapidly and systematically map archaeology in large and complex datasets. To investigate its practical value, a large, random test dataset—next to a small, non-random dataset—was developed, which better represents the real-world situation of scarce archaeological objects in different types of complex terrain. To reduce the number of false positives caused by specific regions in the research area, a novel approach has been developed and implemented called Location-Based Ranking. Experiments show that WODAN2.0 has a performance of circa 70% for barrows and Celtic fields on the small, non-random testing dataset, while the performance on the large, random testing dataset is lower: circa 50% for barrows, circa 46% for Celtic fields, and circa 18% for charcoal kilns. The results show that the introduction of Location-Based Ranking and bagging leads to an improvement in performance varying between 17% and 35%. However, WODAN2.0 does not reach or exceed general human performance, when compared to the results of a citizen science project conducted in the same research area. Full article
(This article belongs to the Special Issue Machine Learning and Deep Learning in Cultural Heritage)
Show Figures

Figure 1

Back to TopTop