As a pilot project, indoor navigation based on 5G New Radio (5G NR) mobile network is being developed and tested in a public building. For the research project the interesting components of the work with the new standard are the improvement of the positioning protocols that were initially implemented in the 4G Long Term Evolution-Advanced standard (4G LTE-A) and were further developed for 5G NR with improved positioning protocols for an increased accuracy in positioning [
8,
9]. In addition, the project will focus on the usage of the enhanced mobile broadband user profile (eMBB) with the possibility to send real-time data and larger data files, because of the increased available data rates for up- and downlink [
10]. Furthermore, software solutions will be developed to facilitate the creation and integration of navigation maps. The building data will be collected using building information modelling and used for positioning as well as a basis for visualization. The complex building of the HafenCity University Hamburg will be used as a demonstration room.
The basic idea of 5G-based indoor navigation will be implemented with the existing radio cells of the providers (or their used standard technologies). It is expected that the complete coverage of building with the campus network’s radio signal will suffer from the existing building structure and therefore have an impact on the accuracy of the position estimation. Against this background, a technology partnership was planned within the framework of this project, in which an improvement in radio technology is to be achieved that will lead to a significantly higher quality of navigation (X < 2 m). However, the navigation architecture is designed for pedestrians who can still orient themselves visually even with 2 m accuracy.
In the following section, we present the included work packages of the project and the method in which we tried to achieve the goals for indoor positioning and navigation.
2.1. Generation of Building Plans and Models
A fast, safe, and reliable navigation is only possible if detailed knowledge about a location is available. While many official, commercial, and open source geodata are available worldwide for outdoor applications, hardly any plans are available for buildings or are not accessible to the user. Consequently, an effective process is required for the effective and efficient generation of building plans and models for indoor navigation applications.
In [
11], the authors presented a method for creating indoor maps from rescue plans. For this aim, the user takes pictures of the rescue plans, which are then processed with the help of a suitable image editing pipeline. The result is a rough floor plan, which consists of polygonal room outlines and identified staircases, if possible. The authors of [
12] integrated this approach into their investigation for deriving routing graphs from building plans. A photograph of the escape and rescue plan is first corrected by projective rectification. Then, all color elements are erased from the image to remove the supplementary routing symbols. Thresholds are used to generate a binary image that can then be used to calculate edges. However, this approach is not yet fully automatic, as some remnants of symbols and gaps in walls or black pixels require a final manual correction of the generated results.
In order to provide suitable map material for indoor navigation application in the L5IN project, different available 2D and 3D geodata for public buildings need to be evaluated regarding their usability. In doing so, which possibilities exist for further processing of building plans and how complex these are will need to be investigated. For this purpose, the data are to be digitized, generalized, and blended automatically. Existing approaches are to be implemented and an optimal strategy developed. In this way, it should be possible to create a map data set as easily and quickly as possible, e.g., with escape and rescue plans, which exist almost everywhere in public buildings such as universities, shopping centers, and airports.
For new buildings, there are often already digitized building plans available, e.g., in dwg format. These can be processed into a BIM. One advantage is the fast handling, because additional information such as room number and type are often included and can be used. In the absence of plans or if the available geodata is insufficient, current measurement systems and procedures for a complete 3D survey of the essential features of a building are to be investigated. In order to evaluate them with regard to the requirements of the overall project, the following criteria must be observed: required resolution of the 2D/3D geodata, target accuracy, object characteristics (measurability of building parts), building dimensions, coordinate system (geo-referencing to the exterior). Based on the most economical overall solution, comprising measurement technology, measurement procedures, and fusion algorithms, the survey of the building of HafenCity University is to be implemented together with a BIM. Furthermore, models for navigation and visualization are to be derived from the survey data. This modeling of geometric models is to be automated as much as possible using data-based and stochastic algorithms, so that efficient scalability of the usable data is possible. In the next step, an enrichment with case-specific semantic information can lead to a BIM that can be used as a data source for indoor navigation.
2.2. Building Information Model
A BIM is used to support indoor navigation within L5IN project. The underlying approaches for the decision are described in the following. The advantage of working with BIMs is that different information about a building can be collected centrally within a data set. Information that otherwise comes from different sources (such as drawings, tables, presentations, and charts) can be bundled within a BIM. The fact that working with BIMs allows the use of a combination of semantic, geometric, and topological attributes is one reason for their growing usage in different cases [
13,
14].
Wherever possible, open data formats will be used in the L5IN project. The data format of the Industry Foundation Classes (IFC) from buildingSmart is one of the most widely used formats. Even if not all software manufacturers have implemented the current IFC 4 schema yet, it will be used in the project due to its actuality. Further information on the schema can be found in the documentation provided by buildingSmart [
15]. The use of the IFC schema can also be found in various other projects for purposes of geometric representation, routing, etc. [
13,
16,
17]. Note that the IFC STEP Physical File (IFC SPF) is not the only way to export BIMs from the native data formats of the modeling software. Besides the IFC SPF, there are also export options in the form of data tree structures such as JSON or XML [
18]. In contrast to the IFC SPF, these offer a more user-friendly and human-readable form and are therefore used in various other projects [
13,
14] and will be in the L5IN project as well.
The BIM will be stored in a database for use in the L5IN project; therefore, it should be considered that BIMs, in their function as a single source of information, have several features that are not required for indoor positioning and navigation. Like other projects, the model will be decomposed into geometrical, semantic, and topological dimensions, whereby the range of features will be reduced accordingly before they can be mapped in a suitable database framework [
13,
19,
20,
21,
22].
The use of BIMs in the context of the presented project is shown below for subarea navigation and positioning separately. In the area of navigation, the project team expects added value from the use of a BIM for map creation and display, as well as for the generation of routing graphs. Due to the already mentioned role of the model as a single source of information for relevant building information, it makes sense to generate the required maps directly from the model. A distinction must be made whether the modeled 3D geometries of the BIM should be used directly for a navigation application [
17] or if they should be abstracted from the 3D geometries and used as a 2D map [
13,
19].
The existing geometry of the models are the basis for creating 2D maps. It will be investigated how a corresponding approach can be made using the data format IFC JSON, which is not yet officially published. Current algorithms for the conversion of IFC SPF into IFC JSON [
23] are available from members of the technical room [
24] of buildingSmart. This schema will be used to generate map data with the necessary details and information. Here, the model-based approach comes into play, so that both the semantic information (e.g., the occupancy, type of use, and accessibility of rooms) [
19] and the topological information (e.g., door belongs to wall and wall is connected to room) [
16] are used.
For positioning tasks, the project is currently looking at various approaches that are intended to support the determination of the user’s position, such as the provision of material information for the evaluation of the received signal strength, the provision of object information within the scope of map matching, and the modeling of landmarks to support positioning algorithms. To support the determination of the position based on the received 5G signals, information about the material properties of the surrounding components can be provided to the algorithm. The received signal should be better evaluated with regard to the change when hitting a building element (e.g., reinforced concrete wall and facade elements with coated glass surfaces).
With an eye to the transfer of information for map matching, relevant features can be added to the modeled building elements in order to use them accordingly (e.g., a wall represents an obstacle that is not passable or particles are located in rooms/zones that the user profile is not allowed/able to enter) [
22,
25]. To support the problem of orientation drift, it will be investigated to what extent the modeling and use of landmarks in BIMs should be considered. Landmarks can be extended as additional features with existing objects such as stairs, elevators, and doors with information about a change of direction or a change in altitude. Additional objects might have to be modeled to provide information about changes of direction in corridors or the change in altitude by using stairs or the elevator. It will be investigated whether the positioning algorithm can make a decision based on patterns of movement with respect to the path followed and how efficient the approach is [
16,
26].
2.3. Position Estimation
Micro-electro-mechanical systems (MEMS) are found in every smartphone. Reliable indoor positioning based on MEMS has been one of the goals of researchers in recent years. Among all of the available sensors in smartphones, inertial measurement units (IMUs) are those sensors that can be used at any time. Therefore, many positioning techniques rely on them as key sensors. Focusing only on the sensor-based methods with special attention on smartphones, it seems to provide a promising solution for people’s everyday lives. However, positioning techniques are still mostly limited to sensor drifts, placements, and defined labeled data. Therefore, wide ranges of different sensor technologies have been used to overcome the challenges in developing an indoor positioning system. Generally, inertial sensors can be used to provide information about the orientation and position of any object that they are attached to.
In 2010, smartphones with gyroscope sensors (such as the iPhone 4 [
27]) were used for the first time after decades of studying the methods of the strapdown inertial navigation system (INS) [
28]. Researchers were able to determine the 3D position, velocity, and attitude of sensors by integrating the raw data from the accelerometer and gyroscope. However, MEMS sensors are different from the expensive, heavy, and high precision IMUs used in ships, aircrafts, etc. Besides, in contrast to the traditional INS, mobile devices can be located in many different places, even without a unique pose in relation to the navigation body. Different scenarios of indoor positioning, such as on-shoe mounted and handheld techniques, began taking shape at this point in time.
The basis of pedestrian dead reckoning (PDR) is a combination of a pedometer, a known or estimated stride length, and orientation. The activity detection methods for indoor localization purposes, such as in [
29], and PDR methods have started to mature, hoping to address the problems with INS. Due to the popularity of smartphones, the previous experiences in both the INS and the PDR methods remained in the center of attention. Although there were different step counting algorithms in different time domains, frequency domains, and clustering moods, evaluation and comparison helped to clarify windowed peak detection as the optimal choice [
30] for the L5IN project, as it was shown to be work well in one of the most popular PDR benchmarks [
31].
End-to-end systems are more complete systems that use different sensor fusion approaches, as well as external supports such as maps and absolute positioning techniques. They became even more interesting when passing doors and not walls, combined with topological approaches using particle filtering [
32]. Due to the BIM generation process, geo-features and landmarks shape indoor positioning algorithms. For a robust real-time positioning solution, the challenges and open questions must be defined.
The first challenge in research regarding pose tracking is how to determine the initial pose (position and orientation) of the device. This seems to be a very important problem, because if it is possible to calculate the initial position, then it should also be possible to determine the next points. However, in order to provide a high-quality position, absolute methods using WLAN, Bluetooth, or UWB are among those individual solutions that either push users to register into the desired network or create more sensitivity regarding small environmental or anchor changes, as can be seen in fingerprinting approaches [
31,
32]. GPS initialization [
33] is an accepted standard initialization method, but it obviously results in a strong limit for starting points. Manual user input is the only remaining way to start a navigation journey, maybe by using a QR code as another specialized solution [
34]. While the real-time initial position needs to be solved by an absolute positioning technique, the initial orientation can be calculated differently for the pitch, roll and yaw. The pitch and roll are reliably calculatable based on the gravity estimation from accelerometer measurement [
35]. However, a calibrated geomagnetic sensor value needs to be followed by a local magnetic map. Such a radio map or a magnetic distortion map relies heavily on a labelled dataset [
36].
The next challenge addresses the different approaches of sensor tracking and pedestrian positioning. Research on orientation tracking in the context of indoor positioning tries to assess the heading of the device rather than the heading of the pedestrian. They therefore assume that the position of the mobile device is fixed and known or they neglect the misalignment between the smartphone heading and the user’s walking direction as an open problem [
37]. However, there have been some attempts to calculate the heading independent of orientation, based on acceleration values and the principal component analysis (PCA) calculation [
38] or similar approaches, such as frequency domain analysis, but there is potential for improvement to achieve the desired accuracy [
39].
To overcome the abovementioned challenges, map support is not enough. Therefore, 5G positioning, which is an infrastructure-based solution and provides absolute position information, will be used for the L5IN project. The potential of 5G technology is seen as the base requirement. Dense 5G networks with a range of a few meters to tens of metres are envisaged. For instance, one can assume several access nodes per room in indoor enviroments and one access node on each lamp in outdoor areas [
40]. In view of the popularity of smartphones for positioning, 5G-based methods are a perfect complement. It has been shown that an accuracy of 1 m and better is achieved in 70% of cases when different 5G positioning techniques are used [
40].
Proximity, time of arrival (ToA), and direction of arrival (DoA), in combination with Kalman filtering, are common in the 5G positioning domain. In the proximity method, the position of the mobile terminal is calculated relative to the antenna node positions, for which the cell ID (CID) or the received signal strength can be used. For the ToA method, the propagation delay from a mobile terminal to an antenna node is measured. The distance is the propagation delay multiplied by the propagation speed of the signal (speed of light). The position of a mobile terminal can then be calculated using trilateration from the distances to several antenna nodes. For the DoA, the direction of the signal propagation from a mobile terminal to a referenced antenna node is measured using antenna arrays. To determine the angle, the difference of the time of arrival (TDoA) at each element of the antenna array is measured. These methods will be discussed and developed in future works.
In summary, sensor-based positioning methods need to overcome the problem of inertial sensor drift with additional support, such as maps and accurate absolute positions, to keep error accumulation at bay. Regarding the process of the L5IN project, 5G constraints can not only determine the initial position and orientation of the device, but can also provide solutions about the misalignment between the user and the device. Step detection and heading estimation methods can be accurately calibrated using 5G signals. The 5G runtime data and their related methods can be used in order to reach the required accuracy and frequency in a positioning task.
2.4. Modeling and Digitalization
Based on the available data and the selected methods, the following structure was chosen for the navigation application (
Figure 1). The data basis of indoor navigation serves, on the one hand, to create a map data set. These data can be existing building plans, building plans derived from escape and rescue plans, or other geodata. Apart from geometry, topological and semantic information also play a role. If no data are available for a building, it will be surveyed. Models were derived from these survey data, and all these data were integrated into a BIM. On the other hand, a routing graph for route calculation can form the basis as well. If necessary, this routing graph can be derived from building plans or directly from the BIM. In order to transfer the information about the locations to the navigation system, the data were stored in a database and an effective transfer format via the 5G mobile phone network to the end devices was developed. These data were used together with smartphone sensors and 5G signals for position estimation, as well as for navigation and visualization in the app.
In the area of digitization of the L5IN project, a first prototype for a navigation application is to be developed that can demonstrate the functionality and performance. In the first phase, a first 2D navigation model was developed with the existing data and information, containing the basic functionalities (location-to-location navigation, real-time updates, etc.). The essential basis of this prototype was the available planning fundamentals (construction plans, escape plans, etc.). For a later extension by 3D navigation, 3D models of the environment were developed with the help of a game engine based on the images. Corresponding navigation functionalities were implemented. At the end of the L5IN project, it is planned that all available software tools and class libraries will be provided. For this purpose, a corresponding application documentation is to be developed, which will also allow third parties to use it.