Towards Digital Twinning on the Web: Heterogeneous 3D Data Fusion Based on Open-Source Structure

: Recent advances in Computer Science and the spread of internet connection have allowed specialists to virtualize complex environments on the web and offer further information with realistic exploration experiences. At the same time, the fruition of complex geospatial datasets (point clouds, Building Information Modelling (BIM) models, 2D and 3D models) on the web is still a challenge, because usually it involves the usage of different proprietary software solutions, and the input data need further simpliﬁcation for computational effort reduction. Moreover, integrating geospatial datasets acquired in different ways with various sensors remains a challenge. An interesting question, in that respect, is how to integrate 3D information in a 3D GIS (Geographic Information System) environment and manage different scales of information in the same application. Integrating a multiscale level of information is currently the ﬁrst step when it comes to digital twinning. It is needed to properly manage complex urban datasets in digital twins related to the management of the buildings (cadastral management, prevention of natural and anthropogenic hazards, structure monitoring, etc.). Therefore, the current research shows the development of a freely accessible 3D Web navigation model based on open-source technology that allows the visualization of heterogeneous complex geospatial datasets in the same virtual environment. This solution employs JavaScript libraries based on WebGL technology. The model is accessible through web browsers and does not need software installation from the user side. The case study is the new building of the University of Twente—Faculty of Geo-Information (ITC), located in Enschede (the Netherlands). The developed solution allows switching between heterogeneous datasets (point clouds, BIM, 2D and 3D models) at different scales and visualization (indoor ﬁrst-person navigation, outdoor navigation, urban navigation). This solution could be employed by governmental stakeholders or the private sector to remotely visualize complex datasets on the web in a unique visualization, and take decisions only based on open-source solutions. Furthermore, this system can incorporate underground data or real-time sensor data from the IoT (Internet of Things) for digital twinning tasks.


Introduction
In recent years, the fruition of urban datasets has become even more exhaustive but at the same time, more complex. New technologies in the field of Geomatics have allowed the fast acquisition of dense point clouds of indoor and outdoor environments with a high accuracy. Terrestrial Laser Scanner (TLS) static acquisitions allow achieving cm-level accuracies [1][2][3], and new smart technologies now allow for making acquisitions in movement, such as GeoSlam [4] and Bagpack [5] solutions. At the same time, the integration of Unmanned Aerial Vehicle (UAV) technology has opened new possibilities for survey operations, offering the opportunity to make digital reconstructions on the base of aerial point cloud acquisitions [6]. Parallel solutions based on digital photogrammetry allow the virtual 3D reconstruction of complex environments starting from image acquisitions taken from nadiral orientation or around the object [7,8]. These solutions are based on Structure from Motion (SfM) and Multi View Stereo (MVS) algorithms [9][10][11]. The integration of Ground Control Points is necessary to correctly scale and localize the survey with high levels of accuracy [12]. All these solutions are now commonly exploited in Geomatics; however, the management and accessibility of these huge datasets remain a challenge due to the complexity and the data volume [13].
The management of a huge dataset is also a challenge when it comes to the use of Virtual Reality (VR) and Augmented Reality (AR). Recent research in this domain is focused on testing possibilities of virtual navigation such as immersive 3D fruition based on 3D visors [14], development of AR applications for mobile devices connected with real 2D or 3D targets [15], or generation of virtual web navigation models for desktop and mobile devices [16]. The latest solution represents an interesting way to share 3D spatial information because it employs the features of web browsers and WebGL opensource JavaScript libraries to offer users a smart virtual navigation solution [17]. In any case, VR and AR solutions require working with 3D models with a limited number of polygons inside, because representing a 3D environment on the web remains a big challenge [18].
Recent advances in Geographic Information Systems (GIS) have allowed the visualization of 3D Globe-based environments on the web [19]. The idea of developing 3D GIS solutions was born in light of solving the problems related to the complexity of the integration between 3D spatial and non-spatial datasets [20]. Integrating the third dimension in WebGIS is useful to generate more realistic visualization and analyze and describe complex urban datasets that cannot be represented in 2D [21]. In fact, the evolution of faster internet connections allowed specialists to load even more detailed datasets on desktop and mobile devices [22]. These advances allowed the testing of the integration of multiscale information from the urban scale to the architectural one, passing from urban map information to building description in the same visualization model. Most of these solutions are based on a proprietary software platform that encapsulates a ready-to-use visualization on the web, employing WebGL technologies (the same kind of libraries initially developed for VR applications).
All the technologies described below relate to the currently very popular fields of research related to Smart Cities or even more so the concept of Digital Twins (DT). Firstly, DT was introduced in industrial engineering, and relates to the creation of a digital replica of the product, with a real-time connection between the virtual and physical model [23]. The concept was recently extended to architectural and urban planning domains aiming to analyze and manage living environments with the support of heterogenous data, integration data, and analytics techniques [24].
The integration of heterogeneous datasets to manage, such as point clouds, geospatial datasets, 3D models, Building Information Models (BIM) is useful today to manage the urban environment at multiscale levels. The innovation and challenges include searching for the most suitable approaches in fusing multiscale ad multiresolution datasets that allow for analyzing urban environments from the building architectural scale to the urban context. It is necessary to acquire and integrate information in such a way that experts from multiple domains can benefit and apply it in a variety of fields, such as urban planning, quality of life, public services, transport, pollution analysis, energy efficiency, etc. To perform multiscale analysis, visualization in the same context of different 2D and 3D information that compose 3D city models is needed. A recent challenge in this field is the development of 3D city models ( Figure 1) that integrate diverse datasets including BIM 3D models, GIS geospatial, text or other semantic information needed for detailed analysis, VR and AR visualization. Recent software solutions provided the first interesting results of integrated visualization for the management of multiscale levels of detail with BIM-GIS integration [25], and geomatic acquisition-VR integration [26]. Even though there have been many web visualization advances, there are still many challenges facing heterogeneous data integration based only on open-source solutions. In fact, the real-time digital visualization of the urban environment still represents a challenge due to the complexity of geometric information that is aimed to be displayed and to the cost of the property software solutions used for managing urban datasets.
In this context, the adoption of open-source strategies is useful for the development of solutions that can be widely adopted from specialists that want to make DT experimentations avoiding property software costs. At the same time, the choice to share the dataset on the web, employing WebGL technologies, represents opportunities towards digital twinning since it allows the real-time web visualization of the decision-making support for every kind of device. Therefore, this research proposes a solution based on WebGIS open-source platform visualizing urban datasets in a multiscale mode. We integrated heterogeneous datasets (point clouds, IFC model, 3D meshes, 2.5D and 2D geospatial models) in a geospatial environment. The case study is the new building of the ITC Faculty of Geoinformation-University of Twente, located in Enschede (the Netherlands), and its surrounding area. The next section will show the research carried out in this field, then the methods and datasets used for the development of the platform will be described, and, in the end, the results and new open scenarios will be explained, followed by conclusions and discussion.

3D WebGIS Solutions for the Navigation of Complex Urban Datasets
In the last decade, the use of 3D city models has been tested and developed in many applications all over the world [27,28]. The complex structure of the city and the heterogeneous dataset produced in the urban environment still leads researchers to consider the implementation of 3D city models an emerging challenge [29]. The development of 3D WebGIS solutions has opened new possibilities of different kinds of datasets integration, such as GeoBIM (3D city models-BIM integration) [30]. The complexity of the dataset involved in this kind of application needs to make BIM-GIS conversions with the risk of information losses [31]. The use of 3D city models represents a strategic solution in many fields of research, for example, to study and monitor the urban environment for flood disaster prevention, where the three-dimensional flood simulation plays a key role in the decision-making process for the flood risk analysis [32]. For instance, the design and the construction of semantic 3D city models have been employed to provide an open dataset of attributes, semantics, and geometries as a testbed for several energy analysis cases [33]. Some experimentations testing the CityGML Conceptual Model have been carried out, also in the field of digital Cultural Heritage (CH) documentation, working with the challenge regarding the level of details to assess the constructive and semantic characteristics of CH sites [34]. Integrating BIM/IFC models achieves a deep level of information useful for cadastral registration of the legal spaces, allowing a more accurate description of the spaces with subdivision procedures applied to buildings [35]. The development of these kinds of solutions could give useful response solutions in emergency scenarios inside buildings, allowing the real-time generation of navigation paths inside the 3D visualization of indoor environments [36]. Still in the field of CH valorization, the integration of 3D WebGIS solutions allows users to explore cultural sites that are not easily accessible [37]. Integrating detailed BIM models inside a GIS environment is still challenging due to the two different scales of these systems [38]. Recent studies in 3D City models tested the automatic 3D reconstruction of slum areas to automatically update the urban Digital Twins of complex urban environments [39]. The first experiments considered the development of Digital Twins to create living virtualization models of city environments [40]. Considering this scenario, recent experiments tested the application of open-source technologies to visualize complex web city environments and to real-time manage urban and architectural elements integrating different kinds of 2D and 3D datasets [41][42][43].

Materials and Methods
The 3D web platform proposed in this research has been developed for a case study located in Enschede, The Netherlands. The area of the campus and, more particularly, the new building of the ITC Faculty of Geoinformation-University of Twente (ITCnb) in Enschede were explored ( Figure 2). The building has two floors and internal open spaces meant to be for green areas. The web platform developed for this research is based on an open-source structure employing WebGL JavaScript libraries (Cesium.js and Three.js) and is stored in the Apache web server. The visualization platform allows common users to directly explore the building on the web at different scale levels, loading different datasets.
The workflow followed to produce the platform is composed of a chain of necessary operative stages (Figure 3), from the dataset retrieval to the final test of the platform on the web. The first part consists of acquiring the dataset that concerns the 3D navigation environment, composed of digital elaborations of urban survey datasets and remote WebGIS acquisition from open databases. The next phase consists of generating and customizing the 3D web visualization framework stored in a web server, based on WebGL open-source technologies and developed in an Html format. The third step includes the 3D models creation. In this step, the initial point cloud data acquired from the terrestrial laser scanning system must be simplified significantly in order to be smoothly visualized on the web. In this phase, the level of simplification should be balanced with the quality of the web browsing navigation. Once the 3D models have been loaded into the 3D web environment, it is necessary to develop a proper control system according to the visualization scale to obtain and properly link the different visualization scales. In the end, the 3D web navigation platform should be tested with different web browsers on desktop and mobile devices.

Used Dataset
The heterogeneous geospatial dataset visualized in the platform includes a variety of data captured with different acquisition techniques, scales, and qualities. The included dataset starts from the 3D volumes of the buildings and ends with the point cloud visualization of the indoor environments of the ITCnb.
The dataset was catalogued according to the corresponding Level of Detail (LoD) to visualize in the web navigation. The web visualization, as shown below, is divided in different webpages according to a proper scale of navigation.
Each scale of navigation contains the respective 3D models visualization in the platform, according to the appropriate quality of visualization. As shown below, every navigation scale could contain more LoD visualizations.
In detail, the geospatial dataset included in the web navigation models (Table 1) is composed by the following:

Data Processing
Considering the 3D models of the neighborhoods (Figure 4), the dataset is not modified but still maintains the LoD and the aspect of the original source. Instead, the datasets that are for the new ITC building (the 3D textured mesh, the point cloud, and the BIM model) needed further processing to be visualized on the web navigation platform.
The 3D textured mesh generated from a photogrammetric reconstruction needs to be a light model to be loaded quickly into the globe-based environment. For this reason, the number of polygons that compose the mesh must be reduced to 10-20 megabytes in this process, according to the WebGL loading capabilities of the most common web browsers [13].
Considering the point cloud, it represents the most complex dataset to manage and simplify to be loaded and visualized in the WebGL context due to the huge number of points that compose the entire survey of the building ( Figure 5). It was necessary to apply noise reduction algorithms [45] using Cloud Compare open-source software to decimate the dimension of the cloud and, simultaneously, guarantee a realistic visualization on web browsing. In particular, two LoDs of point cloud were developed: a general Point Cloud Building Model (gPCBM) that includes the overall visualization of the ITCnb, and a detailed Point Cloud Building Model (dPCBM) of every floor of the building.  The BIM model of the ITCnb, designed and kindly provided by the architects of the building, was converted into IFC format to be loaded into the web navigation platform.

Structure of the Platform
The navigation platform is composed of different parts in correspondence with different visualization scales (Table 2). For each scale, a proper navigation system has been developed with the other ones, as seen below. The choice of the dataset to be visualized for each navigation system depends on the different needs required in the web navigation of the urban environment. In detail, three scales of visualization ( Figure 6) have been developed in this research:  • A building outdoor-scale visualization using three.js libraries inside an Html template, allow orbiting around the ITCnb and exploring the environments from a bird's-eye view. In this 3D web navigation, users can navigate nearer the building or explore a further LoD of the ITC building. In fact, the dataset involved includes colored point clouds and the BIM model of the building. The dataset includes the low-detailed general point cloud of the entire ITC building and its surroundings; the mediumdetailed point cloud of the ground zero of the ITC building; the medium-detailed point cloud of the first floor of the ITCnb; and the BIM model of the ITC building loaded in IFC standard.
• A building indoor-scale visualization developed with three.js libraries inside and an Html template, where users can explore the indoor environment of the ITCnb. The dataset employed in this scale of navigation model consists of high-detailed point clouds of the ITCnb. A first-person navigation model has been developed to allow users to explore the indoor environment in a realistic way.

Results
The web navigation platform is composed of a system of Html pages connected by links that allow users to switch between different levels of detail and different navigation modes. Every Html page contains JavaScript strings that call WebGL opensource JavaScript libraries located in the Apache webserver (Figure 7). The system starts with an Html page that contains a globe-based WebGIS visualization of the ITCnb and its neighborhoods, with the LoD1, LoD2, and Image-Based Model visualization. This visualization is generated employing Cesium.js libraries, stored in the Apache webserver and linked in the Html page using JavaScript strings. The globe-based visualization contains a link in correspondence with the ITC new building that opens a popup window with a brief description of the structure, its sample visualization, and a connection with the Point Cloud Building Model visualization. The web navigation platform developed is stored in the webserver of the University of Twente. It is possible to freely access the web navigation platform using the VPN connection of the university. Web navigation is available with the most common web browsers (chrome, firefox, safari) and using desktop and mobile devices. A short preview of the system is available online [46].
The starting visualization shows the 3D globe navigation model at urban-scale visualization in Enschede in the zone close to the ITCnb. In this scale of visualization, it is possible to select LoD1, LoD2, and the IBM of the ITCnb. A drop-down menu allows for highlighting the buildings according to the type of building, switching between residential, office, commercial, and industrial ones. Activating the image-based model's visualization, a yellow popup appears in correspondence with the building. The popup activates a window that shows a preview visualization and a brief description of the building with a link connected with the outdoor scale visualization of the construction ( Figure 5).
The outdoor-scale visualization allows web navigation with a bird-flight view around the ITCnb, visualizing the general point cloud of the construction. In this scale of visualization, a drop-down menu allows for switching between the ground-floor and the first-floor point clouds and the BIM model of the building. This level of scale allows users to achieve a complex vision of the building with the possibility to explore the distribution of the spaces inside and outside. In this scale of visualization, there are sprite elements that link to the first-person navigation of each indoor environment.
The indoor-scale of visualization leads the user inside the building with the possibility to explore the environment with first-person navigation. In this visualization scale, the 3D model shown is a well-detailed point cloud limited to the boundaries of the shown environment.

Discussion
The research involved in this work offers a possible low-cost strategy to implement 3D city models integration on the web, with the possibility of adding further modules to be real-time visualized. In fact, the use of a simple Html structure allows for integrating coding strings that can activate real-time acquisitions and database connections and visualizing them in the 3D WebGIS structure. This solution is very useful to visualize the heterogeneous 3D models dataset online with no property software requirement. Unfortunately, some aspects, regarding the dimensional limitations of the dataset to load in the web navigation experience, remain a challenge. For instance, the point clouds need to be strongly simplified to be loaded in the platform, and it represents the kind of data that needs more elaborations to be integrated in the 3D visualization; however, at the same time, its integration allows the exploitation of TLS or GeoSlam datasets [47,48]. In the future, new solutions based on tiling operations can solve part of these limitations. At the same time, the BIM model integration inside a GIS environment is an operation that needs an accurate study on BIM-to-GIS data conversion [49]. Comparing our work with other similar recent interesting studies where software combinations allowed for exploration of GIS-BIM integrations on the web [50][51][52], the studied solution represents a good strategy to integrate different kinds of datasets in a 3D WebGIS urban environment, and because it is totally based on an open-source solution, could be implemented in the future with further modules and represents the environment in a multiscale approach. In fact, in recent times, national governments invested massive technological efforts in these kind of solutions [53,54].

Conclusions and Open Scenarios
The developed solution shows how WebGL technologies could be exploited to create 3D Digital Twins of complex urban environments navigable on the web. Connecting different visualization scales in the same structure, generating an indoor/outdoor navigation model, and integrating datasets with different LoD are possible. Furthermore, this solution, also employing Cesium.js libraries, locates the urban-scale model in a geospatial environment on a 3D WebGIS, allowing the possible integration of Web Map Services (WMS) and Web Feature Services (WFS) datasets for further vector and raster data integration. The employment of JavaScript libraries inside an Html structure offers the possibility to integrate JSON-based script in real-time, connected with sensor networks' real-time data acquisitions. The developed platform could be considered the reference structure for the construction of DT regarding urban and architectural datasets, avoiding property software implementations. Furthermore, the possibility of visualizing the DT on the web from any kind of device enhances the digital twinning capabilities.
The framework developed in this work could be exploited in future research for civil engineering digitalization, where DT can be applied to infrastructures, buildings, archaeological sites, facilities, etc. [55]. In fact, in the future, the integration of sensor network datasets (with real-time data acquisition) into the 3D web navigation platform will allow the digital twinning of urban and architectural datasets on the web. In this way, specialists and municipalities will be able to connect and visualize urban and architectural digital twins employing open-source solutions, while saving costs. The limited dimensions of the 3D dataset for visualization still remain a challenge, but there are early experimentations in this field that are starting to solve this problem [56].
Further implementations will be on the connection of the web digital twin visualization with a Relational Database Management System (RDBMS), offering the possibility to integrate Python calculation modules to visualize into the platform. These solutions will be the further step of this research, aimed at the development of open-source solutions for digital twinning on the web.