Towards the Use of 3D Thermal Models in Constructions

: The use of point clouds in architecture and civil engineering has, to date, been limited almost exclusively to functional geometric features. Nevertheless, hardly any works have attempted to process and explore 3D thermal models for buildings. This paper presents a method for the visualisation and exploration of 3D thermal models (3D-T) of building interiors. A 3D-T model consists of a thermal point cloud, which has been generated with a 3D thermal-scanner platform. Given a 3D-T of a building at a speciﬁc time, the user can visualise and navigate through di ﬀ erent room models and each room can, in turn, be segmented into its architectonic components (walls, ceilings and ﬂoors), from which thermal orthoimages can be generated. When the building is sensed at di ﬀ erent times, a 3D temporal-thermal (3D-TT) model is integrated. The temporal-thermal evolution of these structural components, along with selected zones of them, can then be analysed by performing a new type of thermal characterisation. This method has successfully been tested using real building-related data.


Introduction
The digitalisation of buildings with 3D data-generation devices and the use of the resulting point clouds is currently a very widespread practice in the field of Architecture, Engineering and Construction (AEC). However, its application has been limited almost exclusively to the evaluation of the geometric characteristics of the building [1][2][3].
Although the geometric information is of great importance as regards assessing the state of a building, other kinds of evaluations are also necessary. One of these is thermal evaluation, which provides data on the energy performance of a building and identifies zones with special thermal characteristics [4,5].
The thermal evaluation of a building is often carried out using thermal cameras. These cameras generate 2D images containing information on the temperatures of different zones of the building. The thermal characteristics of the different architectural components can also be explored. However, thermal cameras have a reduced field of vision, which complicates the task of correctly positioning the images in order to create a whole model of the building. [6] Most thermal camera manufacturers provide specific tools for the analysis of these images, as occurs with, for example, the Fluke ("Fluke SmartView") [7], the FLIR ("Advanced Thermal Analysis

Obtaining 3D Thermal Models of Buildings
The principal objective of the proposed method is to monitor 3D thermal models of building interiors. It provides Architecture, Engineering and Construction (AEC) professionals with the opportunity to visualise and explore the models without the need to be in situ in the building. This exploration will be of great use as regards detecting zones with thermal features, such as thermal bridges and dampness, while simultaneously positioning them with great precision within the building.
The acquisition of a 3D point cloud with thermal information must first be carried out, after which several algorithms will process this huge amount of information and provide users with valuable functionalities and results. The 3D thermal data-acquisition system is composed of a 3D RieglVz 400 scanner (RIEGL Laser Measurement Systems, Horn, Austria), an RGB Nikon D90 camera and a FLIR AX5 thermal camera, as shown in Figure 1a.

Obtaining 3D Thermal Models of Buildings
The principal objective of the proposed method is to monitor 3D thermal models of building interiors. It provides Architecture, Engineering and Construction (AEC) professionals with the opportunity to visualise and explore the models without the need to be in situ in the building. This exploration will be of great use as regards detecting zones with thermal features, such as thermal bridges and dampness, while simultaneously positioning them with great precision within the building.
The acquisition of a 3D point cloud with thermal information must first be carried out, after which several algorithms will process this huge amount of information and provide users with valuable functionalities and results. The 3D thermal data-acquisition system is composed of a 3D RieglVz 400 scanner (RIEGL Laser Measurement Systems, Horn, Austria), an RGB Nikon D90 camera and a FLIR AX5 thermal camera, as shown in Figure 1a. During the acquisition stage, the whole system rotates twice. First, the scanner captures the 3D coordinates of the visible scene in a 360° turn. Afterwards, the thermal cameras shot 12 overlapped photos in a second 360° turn. Points and temperatures are eventually matched, yielding a 360-thermal point cloud of the visible scene. More information on this technique can be found in [28].
The T-point cloud of a room is then obtained by aligning a series of consecutive T-scans from different positions of the scanner. When a new scan is taken, the thermal point cloud is first preprocessed and then aligned into a common reference coordinate system. With the help of an inclinometer the point cloud is aligned in such a way that the floor points lie on horizontal planes and the wall points lie on vertical ones. The registration of the current point cloud with the previous scan is primarily carried out by using the odometry of our scanning platform MoPAD [29], and is later refined by applying the well-known ICP (Iterative Closest Point) technique. The accumulated point cloud of the scene is eventually updated. More information on this stage can be found in [30] Figure 1b shows an example of a 3D thermal point cloud composed of three T-scans. The colour palette assigned to the points helps the user to understand the current scene in a 3D environment from a thermic point of view.
The T-scanning of a building follows the strategy shown in Figure 2. This is a sequential process in which the 3D-T models of different adjacent rooms are integrated into an accumulated model. In practice, we use MoPAD, which is able to autonomously navigate and collect 3D thermal data in the indoor environments of buildings, as shown in Figure 3a. A 3D thermal model of a floor plan of a building obtained with MoPAD is shown in Figure 3b,c. During the acquisition stage, the whole system rotates twice. First, the scanner captures the 3D coordinates of the visible scene in a 360 • turn. Afterwards, the thermal cameras shot 12 overlapped photos in a second 360 • turn. Points and temperatures are eventually matched, yielding a 360-thermal point cloud of the visible scene. More information on this technique can be found in [28].
The T-point cloud of a room is then obtained by aligning a series of consecutive T-scans from different positions of the scanner. When a new scan is taken, the thermal point cloud is first preprocessed and then aligned into a common reference coordinate system. With the help of an inclinometer the point cloud is aligned in such a way that the floor points lie on horizontal planes and the wall points lie on vertical ones. The registration of the current point cloud with the previous scan is primarily carried out by using the odometry of our scanning platform MoPAD [29], and is later refined by applying the well-known ICP (Iterative Closest Point) technique. The accumulated point cloud of the scene is eventually updated. More information on this stage can be found in [30] Figure 1b shows an example of a 3D thermal point cloud composed of three T-scans. The colour palette assigned to the points helps the user to understand the current scene in a 3D environment from a thermic point of view.
The T-scanning of a building follows the strategy shown in Figure 2. This is a sequential process in which the 3D-T models of different adjacent rooms are integrated into an accumulated model. In practice, we use MoPAD, which is able to autonomously navigate and collect 3D thermal data in the indoor environments of buildings, as shown in Figure 3a. A 3D thermal model of a floor plan of a building obtained with MoPAD is shown in Figure 3b,c.

Exploring 3D Thermal Models
The complete T-point cloud of a building can be explored by studying separate rooms, which can also be segmented into their different Architectonic Components (AC). We have established a chain of exploring functionalities that ranges from the generation and visualisation of a building floor plan to the exportation of orthoimages of the architectonic components of which it is composed. Figure 4 shows a schematic diagram of the different functions and steps.  Figure 2. T-scanning of a building: general strategy (the alignment algorithm can be found in [30]).
Sustainability 2020, 12, x FOR PEER REVIEW 4 of 13 Figure 2. T-scanning of a building: general strategy (the alignment algorithm can be found in [30]).

Exploring 3D Thermal Models
The complete T-point cloud of a building can be explored by studying separate rooms, which can also be segmented into their different Architectonic Components (AC). We have established a chain of exploring functionalities that ranges from the generation and visualisation of a building floor plan to the exportation of orthoimages of the architectonic components of which it is composed. Figure 4 shows a schematic diagram of the different functions and steps.

Exploring 3D Thermal Models
The complete T-point cloud of a building can be explored by studying separate rooms, which can also be segmented into their different Architectonic Components (AC). We have established a chain of exploring functionalities that ranges from the generation and visualisation of a building floor plan to the exportation of orthoimages of the architectonic components of which it is composed. Figure 4 shows a schematic diagram of the different functions and steps.  The thermal model of the building (such as that shown in Figure 3c) is initially a disorganised point cloud, consisting of millions of 3D coordinates with their respective temperature values. A 2D image of the floor plan of the building provides a general view of the structure of the building, in which the user can move around and select the room/s to be explored. This automatically activates the room exploration functionality. The observer can then move to different positions in the interior of the room, rotate the camera or zoom the view. This is very useful as regards approaching specific zones of the architectonic components and analysing temperature details in them.
Our system additionally provides other options, such as setting the temperature range or defining a specific colour code for a better visualisation. Apart from the visual exploration of the model, the system can yield quantitative thermal information on rectangular zones, which are selected beforehand, and provide the average, minimum and maximum temperature values.
As mentioned above, the thermal model can be segmented into its architectonic components, which are typically walls, ground, ceiling and columns. A RANSAC based algorithm [30] yields point cloud clusters belonging to different architectural components of the building. The input of the algorithm here is the unorganised point cloud of the scene together with certain parameters, such as the maximum permitted distance on the plane, the number of iterations and the stop threshold. These parameters greatly affect the segmentation results and must first be tested and then fitted. The raw 3D point cloud is eventually visualised as a set of segments in different colours, which are also freely hidden or displayed. Since segmentation failures might occur, the input parameters can be refitted in order to obtain better segmentation results.
Note that the information captured by our system corresponds to 3D thermal points in a 3D context, which is an unusual format when dealing with thermal data. A thermal reconstruction of a building using exclusively thermographic cameras is unpractical. Such a technique would need hundreds of photos to generate a 3D thermal model of the scene, as it happens with a conventional RGB-photogrammetric technique. Thus, performing a thermal evaluation of a building with such an enormous number of images, each corresponding to small areas of the building, is a tedious and impractical task. Additionally, the precise positioning of each image in the 3D geometric model of the building (if it is available) is also very difficult.
With a view of resolving the aforementioned problem, our system first obtains the whole 3D point cloud of the building, then segments it into a set of architectural components and, finally, calculates the corresponding thermal orthoimages. This last stage is carried out by projecting each segmented AC point cloud onto its corresponding fitted plane and discretising the projected points. We then generate a set of 2D images and fill the gaps in each one by using the Navier-Stokes filling algorithm [31]. The reason for these gaps comes from the fact that the number of points collected from the scanner with an assigned temperature is less than the resolution of the 2D orthoimage. In other words, not all pixels of the orthoimage have an assigned temperature. If the resolution is very high, there will be some pixels without assigned temperature. On the contrary, if it is very low, all pixels will have an assigned temperature (even several temperatures might fall on the same pixel and, in  The thermal model of the building (such as that shown in Figure 3c) is initially a disorganised point cloud, consisting of millions of 3D coordinates with their respective temperature values. A 2D image of the floor plan of the building provides a general view of the structure of the building, in which the user can move around and select the room/s to be explored. This automatically activates the room exploration functionality. The observer can then move to different positions in the interior of the room, rotate the camera or zoom the view. This is very useful as regards approaching specific zones of the architectonic components and analysing temperature details in them.
Our system additionally provides other options, such as setting the temperature range or defining a specific colour code for a better visualisation. Apart from the visual exploration of the model, the system can yield quantitative thermal information on rectangular zones, which are selected beforehand, and provide the average, minimum and maximum temperature values.
As mentioned above, the thermal model can be segmented into its architectonic components, which are typically walls, ground, ceiling and columns. A RANSAC based algorithm [30] yields point cloud clusters belonging to different architectural components of the building. The input of the algorithm here is the unorganised point cloud of the scene together with certain parameters, such as the maximum permitted distance on the plane, the number of iterations and the stop threshold. These parameters greatly affect the segmentation results and must first be tested and then fitted. The raw 3D point cloud is eventually visualised as a set of segments in different colours, which are also freely hidden or displayed. Since segmentation failures might occur, the input parameters can be refitted in order to obtain better segmentation results.
Note that the information captured by our system corresponds to 3D thermal points in a 3D context, which is an unusual format when dealing with thermal data. A thermal reconstruction of a building using exclusively thermographic cameras is unpractical. Such a technique would need hundreds of photos to generate a 3D thermal model of the scene, as it happens with a conventional RGB-photogrammetric technique. Thus, performing a thermal evaluation of a building with such an enormous number of images, each corresponding to small areas of the building, is a tedious and impractical task. Additionally, the precise positioning of each image in the 3D geometric model of the building (if it is available) is also very difficult.
With a view of resolving the aforementioned problem, our system first obtains the whole 3D point cloud of the building, then segments it into a set of architectural components and, finally, calculates the corresponding thermal orthoimages. This last stage is carried out by projecting each segmented AC point cloud onto its corresponding fitted plane and discretising the projected points. We then generate a set of 2D images and fill the gaps in each one by using the Navier-Stokes filling algorithm [31]. The reason for these gaps comes from the fact that the number of points collected from the scanner with an assigned temperature is less than the resolution of the 2D orthoimage. In other words, not all pixels of the orthoimage have an assigned temperature. If the resolution is very high, there will be some pixels without assigned temperature. On the contrary, if it is very low, all pixels will have Sustainability 2020, 12, 8521 6 of 13 an assigned temperature (even several temperatures might fall on the same pixel and, in this case, the average temperature would be assigned). In our case, the resolution of the orthoimage is one pixel per centimetre.

Temporal Models
This is a new way in which to manage the thermal information of a building. If several T-scan sessions are repeated at regular time intervals, a multi-temporal T-point cloud of the building is obtained. This is what was denominated as the 3D-TT model in the abstract. Our system has been designed to use this data structure in order to analyse the temporal evolution of each room, along with its corresponding ACs. The analysis is carried out by means of a visual exploration of the orthoimages of the ACs and several graphics of the mean temperatures of selected zones. Figure 5 shows a schematic diagram of the different steps required to deal with and compare 3D-TT models.
obtained. This is what was denominated as the 3D-TT model in the abstract. Our system has been designed to use this data structure in order to analyse the temporal evolution of each room, along with its corresponding ACs. The analysis is carried out by means of a visual exploration of the orthoimages of the ACs and several graphics of the mean temperatures of selected zones. Figure 5 shows a schematic diagram of the different steps required to deal with and compare 3D-TT models.
The original input of this process is, therefore, a set of 3D thermal models of the same scene but acquired at different times. These models are subsequently segmented into a set of 2D orthoimages corresponding to different ACs. It is, therefore, possible to attain a visual comparison of the temperature at a specific time, along with its evolution over time, for each AC.
The system additionally allows the selection and labelling of regions within an AC, denominated as regions of interest (RoI) in Figure 5. This is the thermal characterisation stage in Figure 5, in which the thermal evolution over time of a selected zone is evaluated by employing a series of basic statistical indicators, such as the average, the minimum and the maximum temperatures. This basic statistical analysis can be exported to a .csv file, which facilitates comparisons with data gathered by other sensors and data estimated using common energy simulation software.
Data acquisition and data processing stages have been programmed by the authors of this paper using free software libraries (with the exception of Matlab). Specifically, the data acquisition process has been developed under ROS framework and using C/C++ and Matlab. For data processing and monitoring, we have used C++, OpenCV and PCL libraries.

Case Study
In this section, we show the exploration and analysis results of a 3D thermal model obtained with the aforementioned MoPAD platform. The model is composed of 122 million points and corresponds to a 16-room building. Figure 6a shows the 2D floor plan (plan view) of the building, which was generated by projecting the raw 3D  The original input of this process is, therefore, a set of 3D thermal models of the same scene but acquired at different times. These models are subsequently segmented into a set of 2D orthoimages corresponding to different ACs. It is, therefore, possible to attain a visual comparison of the temperature at a specific time, along with its evolution over time, for each AC.
The system additionally allows the selection and labelling of regions within an AC, denominated as regions of interest (RoI) in Figure 5. This is the thermal characterisation stage in Figure 5, in which the thermal evolution over time of a selected zone is evaluated by employing a series of basic statistical indicators, such as the average, the minimum and the maximum temperatures. This basic statistical analysis can be exported to a .csv file, which facilitates comparisons with data gathered by other sensors and data estimated using common energy simulation software.
Data acquisition and data processing stages have been programmed by the authors of this paper using free software libraries (with the exception of Matlab). Specifically, the data acquisition process has been developed under ROS framework and using C/C++ and Matlab. For data processing and monitoring, we have used C++, OpenCV and PCL libraries.

Case Study
In this section, we show the exploration and analysis results of a 3D thermal model obtained with the aforementioned MoPAD platform. The model is composed of 122 million points and corresponds to a 16-room building.  Figure 7a illustrates the 3D point cloud model of a room that is segmented into different architectonic components, with the points belonging to each one in a different colour. Figure 7b presents the thermal point cloud of some of the segmented architectonic components in separate windows. Figure 7c shows an example of the original thermal orthoimage of a wall and the final result after filling the thermal gaps. The mean temperature of the selected zones is always available. Figure 7a illustrates the 3D point cloud model of a room that is segmented into different architectonic components, with the points belonging to each one in a different colour. Figure 7b presents the thermal point cloud of some of the segmented architectonic components in separate windows. Figure 7c shows an example of the original thermal orthoimage of a wall and the final result after filling the thermal gaps. Several views of different selected rooms are presented in Figure 6b-d. These images clearly show hot and cold areas of the thermal point cloud viewed. The possibility of personalising the properties of the scale and the colour-code map helps the users to visualise the temperature variations in the scene. The mean temperature of the selected zones is always available. Figure 7a illustrates the 3D point cloud model of a room that is segmented into different architectonic components, with the points belonging to each one in a different colour. Figure 7b presents the thermal point cloud of some of the segmented architectonic components in separate windows. Figure 7c shows an example of the original thermal orthoimage of a wall and the final result after filling the thermal gaps. In order to carry out the temporal thermal analysis, 48 3D thermal models of the same room were taken every 30 min over 24 h. The scenario is an inner courtyard illuminated by a high and large cover glass, which provides a natural source of heat. Figure 8 shows a panel with 12 representative orthoimages of a particular AC at times in which the temperature variation is evident.  In order to carry out the temporal thermal analysis, 48 3D thermal models of the same room were taken every 30 min over 24 h. The scenario is an inner courtyard illuminated by a high and large cover glass, which provides a natural source of heat. Figure 8 shows a panel with 12 representative orthoimages of a particular AC at times in which the temperature variation is evident. In order to carry out the temporal thermal analysis, 48 3D thermal models of the same room were taken every 30 min over 24 h. The scenario is an inner courtyard illuminated by a high and large cover glass, which provides a natural source of heat. Figure 8 shows a panel with 12 representative orthoimages of a particular AC at times in which the temperature variation is evident.   In a second step, a detailed study on this architectural component is performed by selecting and labelling certain zones of interest. Figure 9b presents the temporal evolution of the temperature in four selected zones. We have chosen four regions belonging to: (1) a picture on the wall, (2) the left-hand side of the wall, (3) the door on the right-hand side of the wall, and (4) the alarm device see Figure 9a. In a second step, a detailed study on this architectural component is performed by selecting and labelling certain zones of interest. Figure 9b presents the temporal evolution of the temperature in four selected zones. We have chosen four regions belonging to: (1) a picture on the wall, (2) the lefthand side of the wall, (3) the door on the right-hand side of the wall, and (4) the alarm device see Figure 9a.  It is clear that the temperatures of regions (1) and (2) follow similar evolutions during the day, slighting decreasing from 16:30 onwards. The temperature of region (4) is by far the highest, reaching 24 • C, and is maintained throughout almost the entire day. On the contrary, region (3), which corresponds to the door, undergoes the highest variation, rising to 19.6 • C at 16:30 and falling to 15.2 • C at 08:30. We assume that the door communicates with some sort of storeroom with deficient thermal insulation.
In order to compare the evolution of the temperature of regions (1) to (4) with respect to external and internal (ambient) temperatures, we include the respective graphics in Figure 9c. Note that, although the external temperature varies significantly from one day to the next, the internal temperature remains quite constant for 24 h, decreasing from 18 to 17 • C. Note also that regions (1), (2) and (4) follow the same decreasing pattern, which demonstrates the good thermal insulation of the courtyard. Nevertheless, the surface of the door (region (3)) follows the external temperature pattern and is, therefore, considered as the main thermal leakage area of that wall. This is just one example of how the use of 3D temporal thermal models can help engineers to understand the thermal behaviour of a building and locate different regions with different thermal inertias and transmittances. This process can be carried out for any part of the 3D thermal model, and is a new way in which to explore and analyse the thermal efficiency of buildings.
One of the aspects to be considered when dealing with thermal data regards the ground truth. Obviously, the ground truth in this kind of experiment is not available because there is not a system able to register the true temperature of such amount of points on walls, ceilings and floors. Therefore, in order to yield an approximate error measure, we conducted a set of local measurements in walls by using S-TMB onset temperature sensors with accuracy of 0.2 • C and compared temperatures at the corresponding positions. The mean error reported in this experiment was below 0.7 • C.
Finally, Table 1 provides interesting information of average times for different parts of the whole process. As regards the acquisition phase, the time of a single 360-thermal scan is separated into the time spent in the collection of 3D coordinates and the time devoted to obtaining thermal photos. As can be seen, each of these takes less than a minute in time. The point-temperature matching step lasts around one second. Times assigned to the segmentation and recognition processes greatly depend on the amount of points collected. For a point cloud of ten million points we have estimated an average time of around 250 s. Orthoimage generation and refining can take 10 to 12 s depending on the resolution of the image. Finally, times for experiments of temperature evolution are included in the last rows. Taking a single architectural component (e.g., a wall), the time is greatly reduced because the 3D coordinates are only collected in the first scan. Thus, in the following samples, the scanner does not take 3D data and only the set of 12 thermal photos are captured, which are afterwards matched to the already existing points. For sessions of 24 h, 48 samples at time intervals of 30 min are taken.

Discussion
This article presents the work developed in our lab in order to obtain and explore 3D thermal models of buildings and to perform analysis of the thermal evolution over time In the last ten years, several research groups have worked on the creation of the so-called geometric BIM models, including ours. However, there are very few groups in the world that obtain thermal BIM models, which include the dimension "temperature". Almost all of these groups generate only unstructured and partial point cloud models, with colour information and temperature.
The applicability of the methodology proposed in this article is principally intended for architects and civil engineers. Among other applications we firstly point out the visualisation and detection of zones with special thermal features in a 3D environment and the extraction of thermal images of architectural components separately. Secondly, a new application is the study of the evolution over time of the temperature of different architectural components and the subsequent comparison. This analysis leads the engineers to better understand the energetic behaviour of the building and the energy-modelling process.
As mentioned in Section 2, our hardware component is a mobile scanning platform that is able to collect 3D thermal data of interior of buildings without the human intervention. All this huge information is later organised and prepared in order to be easily explored and analysed. Several visualisation options have been programmed within our system by using low level programming languages and free software tools (with the exception of MATLAB). The objective here is to explore, at stated times, whatever zone of the building in a 3D framework. On the other hand, the study in time of thermal point clouds has entailed a great increase of data and new complex processing algorithms. With the help of these algorithms, the construction professionals have identified specific zones with different thermal inertia.
The current version of MoPAD has several limitations and disadvantages. The first limitation concerns the energetic autonomy of the system. The MoPAD's battery provides power supply to the robot, the sensors and the two on-board computers. Owing to the amount of data collected and the hard processing, the whole system can operate only for several hours. This implies to stop the scanning process every four hours in order to recharge MoPAD batteries. In the case of a large building this might break the scanning process in several sessions.
The second limitation regards the applicability outdoors. So far, our system is only applicable to building interiors. The navigation procedure and the data processing (including registration and segmentation of point clouds) is only designed for indoors of buildings. Of course, outdoors could be scanned using MoPAD, but most of the acquisition algorithms should be adapted or totally reprogrammed.
The third disadvantage is that the 3D data processing is focussed on dealing with plane architectural components. Therefore, data corresponding to curve and irregular ACs are neither segmented not recognised, and the final 3D thermal model in these regions would be wrong.
Finally, it is worth noting that, during the data acquisition stage, neither people nor moving objects are permitted. Note that the inclusion of people would even entail a temperature noise source in the scene.

Conclusions
The use of 3D data-acquisition devices to generate point clouds with geometric information on buildings has become a common practice in the field of architecture and civil engineering. Some researchers and companies provide developments with effective and practical functionalities that can be used to evaluate this geometric information. However, few specific works that explore and analyse 3D thermal models can be found to date.
This article presents a representative case study in which a thermal point cloud of the floor plan of a building has been generated and analysed. The results obtained demonstrate the potential utility of our system and its applicability to real scenarios. Nevertheless, this work needs to be improved and extended in the near future. Among other improvements, we aim to generate efficient algorithms that will segment an AC into its representative thermal zones. This could be solved by studying the temporal evolution of selected thermal regions. Furthermore, we feel encouraged to extend our research to multi-story buildings with new practical functionalities.