Digital Twins and Extended Reality: Opportunities and Challenges of Integrated Applications

A special issue of Drones (ISSN 2504-446X).

Deadline for manuscript submissions: closed (20 August 2024) | Viewed by 20935

Printed Edition Available!
A printed edition of this Special Issue is available here.

Special Issue Editors


E-Mail Website
Guest Editor
Department of Architecture, Built Environment and Construction Engineering, Politecnico di Milano, Milan, Italy
Interests: CH documentation and drawings; 3D survey and digital representation; HBIM; historical constructive techniques; conservation and restoration of historical buildings
Special Issues, Collections and Topics in MDPI journals

E-Mail Website
Guest Editor
Department of Architecture, Built Environment and Construction Engineering, Politecnico di Milano, Milan, Italy
Interests: digital humanities; 3D survey; digital twin; H-BIM; Scan-to-BIM; digital representation; visual communication; 3D modelling; virtual reality (VR); augmented reality (AR); virtual museum; extended reality (XR); digital documentation workflows
Special Issues, Collections and Topics in MDPI journals

E-Mail Website
Guest Editor
Department of Humanities, IULM University, Milan, Italy
Interests: digital humanities; 3D survey; digital twin; H-BIM; Scan-to-BIM; digital representation; visual communication; 3D modelling; virtual reality (VR); augmented reality (AR); virtual museum; extended reality (XR); digital documentation workflows
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

The architecture, engineering and construction (AEC) industry and the digital heritage domain are both benefiting from the augmented reality (AR) and virtual reality (VR) revolutions. Architectural representation, along with advanced tools and techniques of digital visualization, enables users to increase the informative value of digital models.

Researchers and architects are increasingly capable of devising complex scenarios, characterised by buildings of high historical-cultural value, along with historical and modern infrastructure, archaeological sites, and museums, and have directed their resources towards innovative processes such as scan-to-BIM, BIM projects, HBIM projects, BIM-to-FEA, CoSim, BEM, InfraBIM, able to digitise and transform simple point clouds (3D survey data) into informative digital models.

Within this context, interoperability and data sharing become key factors for developing digital models oriented towards different purposes and uses, such as restoration, building site design, energy and structural analysis, cost evaluation, valorisation, knowledge sharing and virtual representation.

On the other hand, these innovative forms of representation present low levels of interactivity and immersion. The digitisation of built structures, whether modern or ancient, requires a specific cognitive approach that aims to create a faithful and accurate representation of geometries. The aim in so doing is to capture specific levels of detail and accuracy (LOD-LOA), as well as the morphological and constructive characteristics of each element, according to the final purpose of the digital twin (DT).

Therefore, acquiring quantitative and qualitative data relevant to the management of complex scenarios DT integrated with VR and AR can improve the use of 3D digital models. Users who may benefit from these applications range from professionals involved in the life cycle of the building to those imagining new forms of virtual tourism.

Within this framework, the use of drones (indoor and outdoor) can provide additional support in documentation activities and data interpretation. More specifically, drones can support the survey of those parts of buildings or sites that are not always easily accessible or inspectable (e.g., collapsed parts of a building due to earthquakes or simply roof or tall buildings challenging to access). Further, drones can provide solid support to developing a DT of the recorded objects or sites, allowing the development of 2D measured drawings and 3D models obtained either by integrating aerial data with those obtainable using ground survey instruments or simply by using UAV to survey and represent built environments.

Additionally, the resulting model can be implemented and updated during the various phases of the maintenance/restoration/reuse of a building or a site, becoming fundamental to the control and simulation of activities and acting as a reference for all the different actors involved in site conservation and management, ranging from surveyors and archaeologists to architects, conservators, engineers and plant engineers.

Accordingly, the digital representation—acquired either by photogrammetry or LiDAR—can be used for dissemination and communication. Within this context, VR and AR play a key role. Virtual and augmented experiences allow museums, cultural institutions, and archaeological sites to propose new forms of virtual–visual storytelling (VVS), allowing virtual tourists (who cannot access the site) to raise their awareness remotely.

Consequently, we invite the submission of articles dealing with innovative approaches and new perspectives, as well as practical applications, addressed to a broad range of professionals working in the fields of documentation, surveying, representation, digitisation, visualisation, interpretation and simulation of complex scenarios.

Dr. Daniela Oreni
Dr. Fabrizio Banfi
Dr. Davide Mezzino
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Drones is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • UAVs digital twin XR-MR-AR-VR scan-to-BIM—BIM projects
  • HBIM projects
  • BIM-to-FEA
  • CoSim
  • BEM
  • InfraBIM conservation
  • management and dissemination of cultural built heritage fault diagnosis energy and structural analysis and costs evaluation virtual museum
  • virtual-visual storytelling drone-based and drone-supported solutions

Benefits of Publishing in a Special Issue

  • Ease of navigation: Grouping papers by topic helps scholars navigate broad scope journals more efficiently.
  • Greater discoverability: Special Issues support the reach and impact of scientific research. Articles in Special Issues are more discoverable and cited more frequently.
  • Expansion of research network: Special Issues facilitate connections among authors, fostering scientific collaborations.
  • External promotion: Articles in Special Issues are often promoted through the journal's social media, increasing their visibility.
  • e-Book format: Special Issues with more than 10 articles can be published as dedicated e-books, ensuring wide and rapid dissemination.

Further information on MDPI's Special Issue polices can be found here.

Related Special Issue

Published Papers (10 papers)

Order results
Result details
Select all
Export citation of selected articles as:

Research

23 pages, 53795 KiB  
Article
Drones and Real-Time Kinematic Base Station Integration for Documenting Inaccessible Ruins: A Case Study Approach
by Daniele Treccani, Andrea Adami and Luigi Fregonese
Drones 2024, 8(6), 268; https://doi.org/10.3390/drones8060268 - 17 Jun 2024
Cited by 2 | Viewed by 1303
Abstract
Ruins, marked by decay and abandonment, present challenges for digital documentation due to their varied conditions and remote locations. Surveying inaccessible ruins demands innovative approaches for safety and accuracy. Drones with high-resolution cameras enable the detailed aerial inspection and imaging of these inaccessible [...] Read more.
Ruins, marked by decay and abandonment, present challenges for digital documentation due to their varied conditions and remote locations. Surveying inaccessible ruins demands innovative approaches for safety and accuracy. Drones with high-resolution cameras enable the detailed aerial inspection and imaging of these inaccessible areas. This study investigated how surveying technologies, particularly Unmanned Aerial Vehicles (UAVs), are used to document inaccessible ruins. Integration with Real-Time Kinematic (RTK) technologies allows direct georeferencing in photogrammetric processing. A case study of the Castle of Terracorpo in Italy was used to demonstrate UAV-only surveying feasibility in inaccessible environments, testing two different scenarios. The first scenario involved the use of a DJI Matrice 300 RTK coupled with the D-RTK2 base station to survey the Castle; both direct and indirect georeferencing were exploited and compared through the photogrammetric process. This first scenario confirmed that this approach can lead to a centimetre-level accuracy (about three times the GSD value for indirect georeferencing and seven times the GSD value for direct georeferencing exploting RTK). The second scenario testing the integration of data from drones at varying resolutions enabled the comprehensive coverage of ruinous structures. In this case, the photogrammetric survey performed with the dji Mavic 3 Cine drone (indirect georeferencing) was integrated with the photogrammetric survey performed with the dji Matrice 300 RTK drone (direct georeferencing). This scenario showed that GCPs extracted from a direct georeferencing photogrammetric survey could be successfully used to georeference and integrate other drone data. However, challenges persist in surveying underground or enclosed spaces that are inaccessible to UAVs. Future research will explore integrating robotic LiDAR survey systems and advanced data fusion techniques to enhance documentation. Full article
Show Figures

Figure 1

16 pages, 6441 KiB  
Article
Three-Dimensional Documentation and Reconversion of Architectural Heritage by UAV and HBIM: A Study of Santo Stefano Church in Italy
by Guiye Lin, Guokai Li, Andrea Giordano, Kun Sang, Luigi Stendardo and Xiaochun Yang
Drones 2024, 8(6), 250; https://doi.org/10.3390/drones8060250 - 6 Jun 2024
Cited by 3 | Viewed by 1268
Abstract
Historic buildings hold significant cultural value and their repair and protection require diverse approaches. With the advent of 3D digitalization, drones have gained significance in heritage studies. This research focuses on applying digital methods for restoring architectural heritage. It utilizes non-contact measurement technology, [...] Read more.
Historic buildings hold significant cultural value and their repair and protection require diverse approaches. With the advent of 3D digitalization, drones have gained significance in heritage studies. This research focuses on applying digital methods for restoring architectural heritage. It utilizes non-contact measurement technology, specifically unmanned aerial vehicles (UAVs), for data collection, creating 3D point cloud models using heritage building information modeling (HBIM), and employing virtual reality (VR) for architectural heritage restoration. Employing the “close + surround” oblique photography technique combined with image matching, computer vision, and other technologies, a detailed and comprehensive 3D model of the real scene can be constructed. It provides crucial data support for subsequent protection research and transformation efforts. Using the case of the Santo Stefano Church in Volterra, Italy, an idealized reconstructed 3D model database was established after data collection to preserve essential resources such as the original spatial data and relationships of architectural sites. Through the analysis of relevant historical data and the implementation of VR, the idealized and original appearance of the case was authentically restored. As a result, in the virtual simulation space, the building’s style was realistically displayed with an immersive experience. This approach not only safeguards cultural heritage but also enhances the city’s image and promotes tourism resources, catering to the diverse needs of tourists. Full article
Show Figures

Figure 1

29 pages, 35171 KiB  
Article
Machine Learning-Based Monitoring for Planning Climate-Resilient Conservation of Built Heritage
by Lidia Fiorini, Alessandro Conti, Eugenio Pellis, Valentina Bonora, Andrea Masiero and Grazia Tucci
Drones 2024, 8(6), 249; https://doi.org/10.3390/drones8060249 - 6 Jun 2024
Viewed by 1319
Abstract
The increasing frequency and intensity of extreme weather events are accelerating the mechanisms of surface degradation of heritage buildings, and it is therefore appropriate to find automatic techniques to reduce the time and cost of monitoring and to support their planned conservation. A [...] Read more.
The increasing frequency and intensity of extreme weather events are accelerating the mechanisms of surface degradation of heritage buildings, and it is therefore appropriate to find automatic techniques to reduce the time and cost of monitoring and to support their planned conservation. A fully automated approach is presented here for the segmentation and classification of the architectural elements that make up one of the façades of Palazzo Pitti. The aim of this analysis is to provide tools for a more detailed assessment of the risk of detachment of parts of the pietraforte sandstone elements. Machine learning techniques were applied for the segmentation and classification of information from a DEM obtained via a photogrammetric drone survey. An unsupervised geometry-based classification of the segmented objects was performed using K-means for identifying the most vulnerable elements according to their shapes. The results were validated through comparing them with those obtained via manual segmentation and classification, as well as with studies carried out by experts in the field. The initial results, which can be integrated with non-geometric information, show the usefulness of drone surveys in the context of automatic monitoring of heritage buildings. Full article
Show Figures

Figure 1

22 pages, 39277 KiB  
Article
Multi-Sensor 3D Survey: Aerial and Terrestrial Data Fusion and 3D Modeling Applied to a Complex Historic Architecture at Risk
by Marco Roggero and Filippo Diara
Drones 2024, 8(4), 162; https://doi.org/10.3390/drones8040162 - 19 Apr 2024
Cited by 2 | Viewed by 1507
Abstract
This work is inscribed into a more comprehensive project related to the architectural requalification and restoration of Frinco Castle, one of the most significant fortified medieval structures in the Monferrato area (province of Asti, Italy), that experienced a structural collapse. In particular, this [...] Read more.
This work is inscribed into a more comprehensive project related to the architectural requalification and restoration of Frinco Castle, one of the most significant fortified medieval structures in the Monferrato area (province of Asti, Italy), that experienced a structural collapse. In particular, this manuscript focuses on data fusion of multi-sensor acquisitions of metric surveys for 3D documenting this structural-risky building. The structural collapse made the entire south front fragile. The metric survey was performed by using terrestrial and aerial sensors to reach every area of the building. Topographically oriented Terrestrial Laser Scans (TLS) data were collected for the exterior and interior of the building, along with the DJI Zenmuse L1 Airborne Laser Scans (ALS) and Zenmuse P1 Photogrammetric Point Cloud (APC). First, the internal alignment in the TLS data set was verified, followed by the intra-technique alignments, choosing TLS as the reference data set. The point clouds from each sensor were analyzed by computing voxel-based point density and roughness, then segmented, aligned, and fused. 3D acquisitions and segmentation processes were fundamental for having a complete and structured dataset of almost every outdoor and indoor area of the castle. The collected metrics data was the starting point for the modeling phase to prepare 2D and 3D outputs fundamental for the restoration process. Full article
Show Figures

Figure 1

23 pages, 9950 KiB  
Article
Digital Strategies to Enhance Cultural Heritage Routes: From Integrated Survey to Digital Twins of Different European Architectural Scenarios
by Sandro Parrinello and Francesca Picchio
Drones 2023, 7(9), 576; https://doi.org/10.3390/drones7090576 - 12 Sep 2023
Cited by 8 | Viewed by 2688
Abstract
This paper focuses on a research project for the acquisition and post-production of digital data to create informative virtual representations and digital twins of different European Cultural Heritage sites. The goal was to establish a reliable database for a multi-scalar web platform, also [...] Read more.
This paper focuses on a research project for the acquisition and post-production of digital data to create informative virtual representations and digital twins of different European Cultural Heritage sites. The goal was to establish a reliable database for a multi-scalar web platform, also accessible through extended reality (XR) tools. This initiative aims to support the promotion and management of cultural and historical monuments within the context of European Cultural Routes supported by the Council of Europe. The project involves different case studies spanning European geographic regions, such as the Upper Kama in Russia, the Valencian Routes of Jaime I in Spain, and the Gdańsk fortresses in Poland. The methodology employed in this effort primarily relies on integrated rapid survey techniques. Unmanned aerial vehicles (UAVs) and simultaneous localization and mapping (SLAM) technologies were used for data collection. These methods contribute to the creation of accurate 3D databases and models that transform the cultural routes into a digital format accessible via an informative platform. The actions presented in this paper are part of the European project “PROMETHEUS”, which is funded by the Horizon 2020 program of the European Union. The project involves collaboration between universities and enterprises, fostering inter-sectoral cooperation. Various techniques such as photographic archives, census analysis, and scan-to-BIM (building information modeling) processes are employed to develop this method further. In fact, the ultimate goal of the project is to establish a framework that can be replicated in other cultural contexts, enhancing the digital documentation and valorization of heritage sites. Full article
Show Figures

Figure 1

28 pages, 50335 KiB  
Article
Integrated Surveying, from Laser Scanning to UAV Systems, for Detailed Documentation of Architectural and Archeological Heritage
by Daniele Calisi, Stefano Botta and Alessandro Cannata
Drones 2023, 7(9), 568; https://doi.org/10.3390/drones7090568 - 4 Sep 2023
Cited by 7 | Viewed by 2238 | Correction
Abstract
Nowadays, the study and digitization of historical, architectural, and archaeological heritage are extremely important, covering the creation of digital twins—virtual replicas of real spaces and environments. Such reconstructions can be achieved using technologies with passive or active light sensors: laser scanners as light [...] Read more.
Nowadays, the study and digitization of historical, architectural, and archaeological heritage are extremely important, covering the creation of digital twins—virtual replicas of real spaces and environments. Such reconstructions can be achieved using technologies with passive or active light sensors: laser scanners as light emitters, or photogrammetry through the creation of photographic images. As for the latter case, a distinction must be made between terrestrial and aerial shots, increasingly facilitated by the spread of UAV systems. Point clouds are aligned using georeferenced points measured with a total station. To create a faithful virtual model of the subjects, dense point clouds from a laser scanner are used to generate meshes, which are textured in high resolution from aerial and terrestrial photographs. All techniques can be integrated with each other, as demonstrated through the experiences of two case studies, each serving different purposes. The first is a detailed survey conducted for CAD representation of certain areas of Rocca Farnese in Capodimonte. The second is an instrumental survey for the creation of a realistic digital twin, aimed at providing an immersive VR experience of the archaeological area of Santa Croce in Gerusalemme in Rome. Full article
Show Figures

Figure 1

27 pages, 31771 KiB  
Article
Enhancing Building Archaeology: Drawing, UAV Photogrammetry and Scan-to-BIM-to-VR Process of Ancient Roman Ruins
by Chiara Stanga, Fabrizio Banfi and Stefano Roascio
Drones 2023, 7(8), 521; https://doi.org/10.3390/drones7080521 - 9 Aug 2023
Cited by 12 | Viewed by 3314
Abstract
This research investigates the utilisation of the scan-to-HBIM-to-XR process and unmanned aerial vehicle (UAV) photogrammetry to improve the depiction of archaeological ruins, specifically focusing on the Claudius Anio Novus aqueduct in Tor Fiscale Park, Rome. UAV photogrammetry is vital in capturing detailed aerial [...] Read more.
This research investigates the utilisation of the scan-to-HBIM-to-XR process and unmanned aerial vehicle (UAV) photogrammetry to improve the depiction of archaeological ruins, specifically focusing on the Claudius Anio Novus aqueduct in Tor Fiscale Park, Rome. UAV photogrammetry is vital in capturing detailed aerial imagery of the aqueduct and its surroundings. Drones with high-resolution cameras acquire precise and accurate data from multiple perspectives. Subsequently, the acquired data are processed to generate orthophotos, drawings and historic building information modelling (HBIM) of the aqueduct, contributing to the future development of a digital twin. Virtual and augmented reality (VR-AR) technology is then employed to create an immersive experience for users. By leveraging XR, individuals can virtually explore and interact with the aqueduct, providing realistic and captivating visualisation of the archaeological site. The successful application of the scan-to-HBIM-to-XR process and UAV photogrammetry demonstrates their potential to enhance the representation of building archaeology. This approach contributes to the conservation of cultural heritage, enables educational and tourism opportunities and fosters novel research avenues for the comprehension and experience of ancient structures. Full article
Show Figures

Figure 1

24 pages, 23578 KiB  
Article
Digital Recording of Historical Defensive Structures in Mountainous Areas Using Drones: Considerations and Comparisons
by Luigi Barazzetti, Mattia Previtali, Lorenzo Cantini and Annunziata Maria Oteri
Drones 2023, 7(8), 512; https://doi.org/10.3390/drones7080512 - 3 Aug 2023
Cited by 4 | Viewed by 1801
Abstract
Digital recording of historic buildings and sites in mountainous areas could be challenging. The paper considers and discusses the case of historical defensive structures in the Italian Alps, designed and built to be not accessible. Drone images and photogrammetric techniques for 3D modeling [...] Read more.
Digital recording of historic buildings and sites in mountainous areas could be challenging. The paper considers and discusses the case of historical defensive structures in the Italian Alps, designed and built to be not accessible. Drone images and photogrammetric techniques for 3D modeling play a fundamental role in the digital documentation of fortified constructions with non-contact techniques. This manuscript describes the use of drones for reconstructing the external surfaces of some fortified structures using traditional photogrammetric/SfM solutions and novel methods based on NeRFs. The case of direct orientation based on PPK and traditional GCPs placed on the ground is also discussed, considering the difficulties in placing and measuring control points in such environments. Full article
Show Figures

Figure 1

27 pages, 7209 KiB  
Article
Analysis and Precision of Light Detection and Ranging Sensors Integrated in Mobile Phones as a Framework for Registration of Ground Control Points for Unmanned Aerial Vehicles in the Scanning Technique for Building Information Modelling in Archaeological Sites
by Juan Moyano, Juan E. Nieto-Julián, María Fernández-Alconchel, Daniela Oreni and Rafael Estévez-Pardal
Drones 2023, 7(7), 477; https://doi.org/10.3390/drones7070477 - 20 Jul 2023
Cited by 2 | Viewed by 1850
Abstract
The protection of heritage sites is one of the keys that our civilisation presents. That is why great efforts have been invested in order to protect and preserve movable and immovable property with a certain historical value, as is the case of archaeological [...] Read more.
The protection of heritage sites is one of the keys that our civilisation presents. That is why great efforts have been invested in order to protect and preserve movable and immovable property with a certain historical value, as is the case of archaeological sites scattered throughout the territory of southern Iberia (Spain) in the form of dolmens and negative structures dug into the ground, constituting a good sample of the megalithic culture in southern Spain. To study, manage and preserve these archaeological monuments, considered a set of cultural assets, various techniques and methodologies are required to facilitate the acquisition of three-dimensional geometric information. The Scan-to-BIM approach has become one of the most up-to-date work exponents to carry out these objectives. The appearance of LiDAR techniques, and recently their incorporation into smartphones through integrated sensors, is revolutionising the world of 3D scanning. However, the precision of these techniques is an issue that has yet to be addressed in the scientific community. That is why this research proposes a framework, through experimental measurement, comparison and knowledge of the limitations of this technology, to know the precision of the use of these smartphones, specifically the iPhone 13 Pro, as a measurement element to establish points of control with the aid of photogrammetry by unmanned aerial vehicles (UAVs) in archaeological sites. The results demonstrate a residual uncertainty of ±5 mm in the capture of GCPs from the mobile phone’s LiDAR light detection and ranging sensor, and there was a deviation of the measurements in a range between 0 and 28 m of distance between the GCPs of (0.021, 0.069) m. Full article
Show Figures

Figure 1

14 pages, 4037 KiB  
Article
Scan-to-HBIM Reliability
by Stefano Brusaporci, Pamela Maiezza, Adriana Marra, Alessandra Tata and Luca Vespasiano
Drones 2023, 7(7), 426; https://doi.org/10.3390/drones7070426 - 27 Jun 2023
Cited by 4 | Viewed by 1259
Abstract
The modeling of the historical architecture can be characterized by different levels of geometric development, more or less advanced, which correspond to different values of deviation between the real object and its three-dimensional representation. The aim of the paper is a critical study [...] Read more.
The modeling of the historical architecture can be characterized by different levels of geometric development, more or less advanced, which correspond to different values of deviation between the real object and its three-dimensional representation. The aim of the paper is a critical study of the different levels of geometry (LoG) that the architectural elements of a model can have in relation to the efforts required to achieve them and the modeling objectives. More specifically, the contribution of this study proposes—on the basis of a survey campaign of a case study conducted by drone—the evaluation of the deviation between the point cloud and HBIM models with the different levels of development in order to assess their reliability according to specific objectives, such as the documentation of the architectural asset, feasibility study, restoration project, etc. Full article
Show Figures

Figure 1

Back to TopTop