Next Article in Journal
A Review of Oil Spill Remote Sensing
Next Article in Special Issue
Accurate 3D Positioning for a Mobile Platform in Non-Line-of-Sight Scenarios Based on IMU/Magnetometer Sensor Fusion
Previous Article in Journal
Wireless sEMG System with a Microneedle-Based High-Density Electrode Array on a Flexible Substrate
Previous Article in Special Issue
A Large-Scale Multi-Hop Localization Algorithm Based on Regularized Extreme Learning for Wireless Networks
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

3D Indoor Positioning of UAVs with Spread Spectrum Ultrasound and Time-of-Flight Cameras

by
José A. Paredes
1,*,
Fernando J. Álvarez
1,
Teodoro Aguilera
1 and
José M. Villadangos
2
1
Sensory System Research Group, University of Extremadura, 06006 Badajoz, Spain
2
Department of Electronics, University of Alcalá, Alcalá de Henares, 28805 Madrid, Spain
*
Author to whom correspondence should be addressed.
Sensors 2018, 18(1), 89; https://doi.org/10.3390/s18010089
Submission received: 2 November 2017 / Revised: 15 December 2017 / Accepted: 27 December 2017 / Published: 30 December 2017

Abstract

:
This work proposes the use of a hybrid acoustic and optical indoor positioning system for the accurate 3D positioning of Unmanned Aerial Vehicles (UAVs). The acoustic module of this system is based on a Time-Code Division Multiple Access (T-CDMA) scheme, where the sequential emission of five spread spectrum ultrasonic codes is performed to compute the horizontal vehicle position following a 2D multilateration procedure. The optical module is based on a Time-Of-Flight (TOF) camera that provides an initial estimation for the vehicle height. A recursive algorithm programmed on an external computer is then proposed to refine the estimated position. Experimental results show that the proposed system can increase the accuracy of a solely acoustic system by 70–80% in terms of positioning mean square error.

1. Introduction

The production of Unmanned Aerial Vehicles (UAVs), commonly known as drones, has experienced a notable increase in the last few years, with the development of new applications both in the civil and the military market. As is well known, the autonomous operation of these airships outdoors depends on their Global Navigation Satellite System (GNSS) receiver, which provides them with location coordinates in a global geographical system. However, the accuracy of GNSS is insufficient for many applications, and even worse, GNSS signals are severely attenuated indoors where they cannot be reliably used. For these reasons, many researchers are focusing their attention on the development of a solution to the accurate and reliable location of UAVs in indoor environments. This development includes the exploration of alternative technologies that must be compared in terms of coverage, accuracy, precision, cost and adaptability to existing infrastructures. One of these technologies is ultra-wideband (UWB) radar [1], which provides centimeter accuracy, strong multipath resistance and a good material penetration capability, which can be useful under Non-Line-of-Sight (NLoS) conditions, thanks to its very large bandwidth. However, this technology cannot be considered an ideal solution, since the transmission of information can be adversely affected under strong scattering conditions. Laser technology has also been proposed to conduct Simultaneous Localization and Mapping (SLAM) tasks in the design of a fully-autonomous UAV [2], but its financial costs remain relatively high. A different approach is based on the exploitation of existing infrastructures and the so-called Signals-Of-Opportunity (SOP) to conduct a scalable positioning at a reduced cost. Most of these systems rely on radio-frequency technology, as the one described in [3], which proposes the combined use of the GSM network with a Received Signal Strength (RSS) fingerprinting to improve the 50–200 m range accuracy provided by this mobile network. The RSS measurement is also employed with the WLAN networks extensively deployed in buildings, factories and malls. Most commercial UAVs incorporate a video camera that can be used for positioning and mapping purposes by performing a 3D scanning of the environment and a pattern recognition procedure. This use can be extended to unknown environments through image processing and the comparison with databases [4]. Furthermore, [5] proposes the use of a passive depth camera, mounted on a UAV with an ad hoc setup, specially designed for outdoor scene applications. According to the author’s experiments, this system can be adequate for mapping projects with lower resolution and accuracy requirements. A similar approach appears in [6], where the authors utilize a stereo vision sensor as an indoor positioning system for UAVs. The system makes use of two video cameras for stereo vision capture and a set of fast algorithms to provide position information with reliable accuracy in real time. The possibilities of this technology are wide, with admissible implementation costs both at the hardware and the software level, but its characteristic computation times are still long for real-time use.
All the technologies described above can be used in a complementary way to achieve a more robust design, taking advantage of the strongest points of each one depending on the task to be performed. They can also be combined with an Inertial Measurement Unit (IMU) to improve the accuracy of the positioning and state parameters. For instance, the authors in [7] use it with a monocamera and a laser range finder, and data from a structured light scanner, UWB and an inertial navigation system is integrated in [8]. Furthermore, a navigation system stabilizing the LiDAR measurement plane using an IMU sensor is proposed in [9]. Finally, a fusion of monocular vision with laser measurements, IMU and altimeter is presented in [10].
This work proposes the combination of spread spectrum ultrasound and Time-Of-Flight (TOF) cameras as new technologies in the field of indoor UAV positioning. In the first case, in spite of being considered a classical technology for the local positioning community, with well-recognized characteristics such as high precision and reduced cost, the use of ultrasound for the location and navigation of UAVs is still very early in development. In the second case, TOF cameras are a relatively new technology that has begun finding application in indoor positioning very recently.
This work is thus a first attempt to combine all the knowledge acquired by the authors in the field of general purpose Ultrasonic Local Positioning Systems (ULPS) and their most recent experiences in the use of TOF cameras, with the aim of designing an accurate 3D indoor positioning system for UAVs. To properly contextualize this effort, the next section first presents a review of general purpose ULPS, from the first narrowband proposals appearing in the late 1990s, to the most advanced spread spectrum systems developed recently. This section concludes with a review of recently-proposed range imaging positioning systems. Next, Section 3 presents the proposed system based on the combination of these two technologies. Some experimental results obtained with this system in a real scenario are provided in Section 4, and finally, the main conclusions of this work are drawn in Section 5.

2. Ultrasonic and Range Imaging Indoor Positioning

2.1. Ultrasonic Positioning

As stated before, active ultrasonics has been extensively used in the design of indoor positioning systems, and it is today considered a classical solution to this technological challenge [11,12]. First, general-purpose ULPS, dating from the late 1990s and early 2000s, made use of envelope detection techniques to estimate the Time-Of-Arrival (TOA) of constant-frequency narrowband ultrasonic pulses. This is, for example, the case of the Active Bat system [13,14], where wireless badges (bats) carried by personnel or attached to certain equipment emit 40-kHz ultrasonic pulses of a 50- μ s duration after being triggered over a wireless link. These pulses are received by a set of ceiling-mounted sensors that measure the pulse TOAs and compute the badge 3D position by spherical lateration. A different approach is proposed in the Constellation System [15], where a set of ultrasonic emitters is deployed at known locations in the environment. These beacons emit a 40-kHz ultrasonic pulse after receiving an infrared trigger code from the unit to be located, which communicates with the beacons one-at-a-time. The receiving unit needs to compute at least three TOAs from the emissions of different beacons to obtain its position by spherical lateration. Note that, in this case, the device to be located is in charge of computing its own position using the signals emitted from different beacons. This architecture has been defined by some authors as privacy-oriented, in contrast to the centralized architecture of the Active Bat system. A third example of a narrowband ULPS that cannot be considered centralized nor privacy-oriented is the Cricket system [16,17]. This system is based on a set of independent beacons that incorporate an RF transceiver, an ultrasonic emitter and an ultrasonic receiver. Each beacon can compute its own position by measuring the TOAs of the 40-kHz ultrasonic pulses of 125 μ s in duration emitted by nearby beacons, which also broadcast their own position through a 433-MHz RF signal. Again, spherical lateration is used to compute the desired location.
All the systems described above featured very simple emitter and receiver acoustic modules, but at the expense of providing a limited positioning accuracy of some decimeters with also high sensitivity to in-band noise. A solution to these limitations is provided by the pulse compression technique extensively used in radar systems [18]. As is well known from this theory, the precision of a range measurement can be improved if the effective bandwidth of the received signal is increased while keeping its energy constant. This bandwidth extension can be achieved by shortening the duration of a constant frequency pulse, but then, its amplitude should be increased above the limit imposed by the linear operation regime of real amplifiers and transducers. An alternative is to modulate the original waveform and use a matched filter to detect it at the receiver. This spread spectrum technique had been successfully incorporated in the development of high precision airborne sonars some years before [19,20,21], so its application in the field of ULPS was just a question of time. One of the first broadband ULPS is presented in [22,23], where the authors propose the use of 511-bit Gold codes to modulate a 50-kHz ultrasonic carrier with a bit period of 50 μ s, thus giving a total emission duration of 25.55 ms. In [22], eight receivers are installed in the ceiling of an office room to configure a centralized architecture that measures the TOAs of the signals emitted by a set of synchronized transmitters. The authors reported positioning accuracies slightly above 2 cm when using spherical lateration in a noisy environment. In [23], a privacy-oriented architecture is presented with positioning accuracies around 5 cm. A similar privacy-oriented architecture is presented in [24], based on the modulation of a 50-kHz carrier with 127-bit Gold codes and a bit period of 20 μ s, for a significantly shorter emission duration of 2.54 ms.
However, the notably improved precision and robustness to noise of broadband ULPS do not come for free. The use of longer and simultaneous emissions gives rise to new problems that may hinder the detection of these signals. Some of the most recent works in the field of broadband ULPS are focused on the proposal of solutions for some of these problems, such as Doppler shift [25], multipath propagation [26] or Multiple Access Interference (MAI) [27]. A parallel trend is the development of ULPS for commercial portable devices, whose limited hardware resources and computing capability require a complete redesign of the signal processing algorithms [28,29]. The current state of evolution of these systems suggests their use for the accurate positioning of UAVs, a technological challenge that must face new problems that were not present in the positioning of previous receivers, such as the effect of the rotors on the propagation channel.

2.2. Range Imaging Positioning

There are only a few works that propose the use of range imaging systems to perform positioning tasks, probably because of the need for an initial calibration step and the high computational cost of image processing. In [30,31], the authors present an algorithm to identify the room where the camera is installed, via object detection and its comparison with a specific building database. No precise location inside the room is provided, though. In [32,33], depth information is used to detect walls and other vertical plane furniture. Furthermore, in [34], depth information is combined with odometry and laser ranging to improve the autonomous navigation of a robot. All these works rely on a mobile positioning system to cover relatively large areas, and their use is limited by the battery life. In [35], the authors present a method for real-time people tracking and coarse pose recognition in a smart room, using ceiling-mounted and downward-pointed TOF cameras. The number of cameras needed is the main limitation of this proposal. A similar people detection and tracking system is presented in [36].
Although range imaging does not seem to be the most suitable technology to perform the real-time location of fast-moving vehicles, this work proposes the use of a TOF camera to complement the positioning information provided by a central ultrasonic system. The details of this complementary operation are described in the next section.

3. Proposed Indoor Positioning System

Figure 1 depicts a diagram of the proposed system, which is composed of three different elements. The central element is an ultrasonic emitter module installed in the ceiling of the room. The acoustic signals emitted by this system are acquired by a portable receiver module installed in the drone top cover. Below the emitter module and at the floor level, there is a TOF-camera pointing upwards. The third element is an external computer that collects both the acoustic signal acquired by the portable receiver and the range-image generated by the TOF camera, which are used to compute the UAV location. All these modules are described next.

3.1. Ultrasonic Module

The ultrasonic emitter module consists of five transducers or beacons, distributed around a 70.7 cm × 70.7 cm square structure. The coverage area is of 30 m 2 when it is installed at the ceiling at a height of 2.8 m. Five different 63-bit Kasami sequences [37] have been used to accurately identify every beacon of the ULPS. The codes have been Binary Phase Shift Keying (BPSK) modulated with two periods of a sinusoidal carrier at 41.67 kHz, in order to adapt their spectral features to the frequency response of the ultrasonic transducers (Prowave 328ST160 [38]). The modulated codes have been previously generated offline and sampled at 500 kHz, so 12 samples per carrier period are considered.
If K = k m l { 1 , 1 } ; 0 m M 1 ; 0 l L 1 is a family of Kasami codes of length L = 63 , in which the M = 5 codes with better correlation properties are selected for encoding the beacons of the ULPS and s [ n ] is the modulation symbol with N c = 2 carrier cycles and oversampling factor equal to O f = f s / f c = 500   kHz / 41.67   kHz = 12 , the modulated pattern to be emitted can be represented as (1):
sk m [ n ] = i = 0 L 1 k m [ i ] · s [ n i · N c · O f ]
The modulated codes are stored in the memory of the ULPS microcontroller that manages all of the emission process. All parameters regarding the code design (type of code, length and oversampling factor) can be easily configured according to the demands of the final application. The emitter module is based in a Cortex-M3 microcontroller. Since only one Digital to Analog Converter (DAC) is available in this microcontroller, a Time Division Multiple Access (TDMA) technique has been implemented, which also reduces multipath and MAI. Figure 2a shows the block diagram of this module, composed of an LPC1768 microcontroller, an amplifier to adapt the transducers’ voltage level and a multiplexer to select among the different Kasami-modulated sequences. Furthermore, Figure 2b shows the DAC output signal, where the BPSK modulation and the TDMA emission scheme can be observed. Note the 4-ms delay between consecutive emissions and the total emission cycle of 20 ms.
These signals are acquired by a specifically-designed receiver module. This portable module is based on low cost components and includes an MEMS SPU0414HR5H-SB microphone by Knowles Acoustic (Itasca, IL, USA) [39] with an adequate bandwidth at 41.67 kHz and a serial WiFi module for data wireless transmissions. This module, represented in Figure 3, has a total weight of about 6 g, which is far below the drone payload.
The signals captured by the microphone are high-pass filtered and amplified by a configurable gain amplifier. A STM32F103CBT6 microcontroller by STMicroelectronics (Geneva, Switzerland) [40] digitizes the incoming ultrasonic signals at a sampling rate of 100 kHz. Note that a decimation by five has been included to reduce the amount of data to be processed. The receiver buffer size ensures the reception of one complete emission from all five beacons. A buffer with 2000 samples (4 ms) of the ultrasonic signal is periodically acquired and sent via a UART interface to a serial WiFi module (DT-06 Wireless WiFi) [41] from whence these samples are transmitted to the external process unit.
In the external unit, the acquired signal is first cross-correlated with all Kasami patterns to obtain the set of correlation peaks that are represented in Figure 4. From these peaks and taking into account the 4-ms delay between consecutive emissions, the Time Differences Of Arrival (TDOAs) can be easily extracted to compute the receiver’s location by multilateration. This technique has been previously used in the literature, as can be seen in [28,42].
The ultrasonic system described in this section has been demonstrated to provide an accurate positioning of the receiver in a horizontal plane whose height is known a priori, for instance in [43]. However, the performance of this system is significantly poorer when used to provide a 3D positioning with a variable height, due to the particular distribution of beacons in the ceiling plane and the consequences of this distribution on the Vertical Dilution of Precision (VDOP). For this reason, this work proposes the use of a TOF camera as a means to provide an initial estimation of the receiver’s UAV height that will be later refined by means of an iterative geometrical correction based on the 2D coordinates obtained from a 2D multilateration procedure. The operation of this camera and the positioning algorithm are described in detail in Section 3.3.

3.2. ToF Camera

As mentioned above, the second element of the proposed system is a TOF camera installed at the origin of coordinates to get depth information of the scene. Every pixel, built in CCD/CMOS technology [44], works independently. The working principle (Figure 5) consists of emitting continuously an infrared wave, but modulated by a square signal. The lens receives the returning wave in a matrix of smart pixels. They are able to compute information about this optical signal: the constant DC offset, the modulation amplitude and the phase shift, from which the distance between the camera and the measured object is calculated as,
d = c 2 × Δ φ 2 π f
where c is the speed of light and f is the modulation signal frequency. It is important to take into account that the angle between incident and reflected waves on the target is near zero because the distance d is much greater than the separation between the LEDs (emitters) and the lens (receiver). A detailed mathematical explanation of this behavior can be found in [45], and a clarifying study from the point of view of the CCD/CMOS pixels is included in [46]. Hence, this kind of camera provides not only 2D information in a single image, but also information about the third dimension. Since they do not provide exact volume data, these devices are also known as 2.5D cameras.
Moreover, this device must be properly calibrated in order to get absolute information. Many works in the literature deal with this crucial topic. These works try to elaborate different algorithms and situations to obtain intrinsic and extrinsic parameters that correct certain errors, mainly due to the lenses [47,48,49]. This procedure is usually arduous and time-consuming. The idea in this paper is to avoid this initial calibration and consider only the radial distance to the drone with a slight correction due to radial distortion, since depths measured far from the vertical axis suffer from small deviations. This effect can be easily corrected by adding an offset from the axis to the last pixel, taking into account how far this point is.
There are nowadays different solutions in the TOF camera market to address this problem. In our case, a prototype board from Bluetechnix has been used to have as many different configurations available as possible. The kit—by Bluetechnix Lab GmbH (Wien, Austria)—is composed mainly of a PMD PhotonICs 19k-S3 TOF 3D chip and a Spartan 6 FPGA for real-time image processing [50]. The illumination module provides the scene with 850-nm wavelength light (12 LEDs), not visible to the human-eye. Related to the variable settings, f = 10 MHz has been chosen and an integration time (time during which the pixels are capturing light; the equivalent of exposure time in RGB cameras) of 2.5 ms. This configuration allows one to obtain a good enough image so as to process it. Figure 6 shows an example of images taken from the camera. Figure 6a shows the returning wave amplitude in each pixel, and in Figure 6b, different colors represent different depths, as can be read from the vertical color bar. In this snapshot, in which the drone was flying around pixels ( x p , y p ) = ( 40 , 30 ) , the measured range is about 1.9 m.
This depth image is sent to the external computer, where a simple image recognition procedure starts. Previously, a series of preceding images had been averaged in order to get the background B in the instant t:
B ( x , y , t ) = 1 N i = 1 N D ( x , y , t i )
where D is a depth image (different in each time) and (x, y) are the image pixels. N is the number of measurements (100 for this study). Now, the foreground F ( x , y , t ) can be achieved via subtracting the background from every instant frame:
F ( x , y , t ) = D ( x , y , t ) B ( x , y , t )
so that all elements greater than a threshold (maximum of 1 cm in this system) correspond to new objects appearing in the scene. Then, F ( x , y , t ) can be converted into a binary image (Figure 6c), where white pixels build new objects. After that, the connected elements are identified making use of MATLAB’s Image Processing Toolbox, separating them from each other. Thus, the largest connected region will be the drone. Additionally, the range value d is taken from the geometric centroid of this area.
This section is finally completed with a brief description of the selected drone, depicted in Figure 7. This is a low cost device, with a mass of 107 g and a size of 31 cm × 31 cm × 8 cm . The drone area visible from the camera is approximately 28 cm × 28 cm, since the propellers are not detected due to their fast rotation.

3.3. Positioning Strategy

The proposed positioning algorithm executed in the external computer appearing in Figure 1 can be better explained with the help of the flux diagram represented in Figure 8. As mentioned before, this computer receives the ultrasonic signal from the portable receiver installed in the drone and the depth image from the TOF camera. It then computes the TDOAs of the ultrasonic emissions and the range to the drone, respectively, following the procedures already described in the two previous sections. All this information is used as the input of a two-Dimensional Gauss–Newton (2D-GN) algorithm that iteratively solves the horizontal position of the drone using the range as an estimate for the drone’s height, i.e., updating the drone’s position r = ( x , y , z ) as:
r ( n ) = r ( n 1 ) + ( A T × A ) 1 A T × B
where r ( 0 ) = ( 0 , 0 , d ) , and matrices A and B are given by:
A = x 2 x ( n 1 ) d 2 ( n 1 ) x 1 x ( n 1 ) d 1 ( n 1 ) y 2 y ( n 1 ) d 2 ( n 1 ) y 1 y ( n 1 ) d 1 ( n 1 ) z 2 z d 2 ( n 1 ) z 1 z d 1 ( n 1 ) x 3 x ( n 1 ) d 3 ( n 1 ) x 1 x ( n 1 ) d 1 ( n 1 ) y 3 y ( n 1 ) d 3 ( n 1 ) y 1 y ( n 1 ) d 1 ( n 1 ) z 3 z d 3 ( n 1 ) z 1 z d 1 ( n 1 ) x 4 x ( n 1 ) d 4 ( n 1 ) x 1 x ( n 1 ) d 1 ( n 1 ) y 4 y ( n 1 ) d 4 ( n 1 ) y 1 y ( n 1 ) d 1 ( n 1 ) z 4 z d 4 ( n 1 ) z 1 z d 1 ( n 1 ) x 5 x ( n 1 ) d 5 ( n 1 ) x 1 x ( n 1 ) d 1 ( n 1 ) y 5 y ( n 1 ) d 5 ( n 1 ) y 1 y ( n 1 ) d 1 ( n 1 ) z 5 z d 5 ( n 1 ) z 1 z d 1 ( n 1 )
B = ( d 2 ( n 1 ) d 1 ( n 1 ) ) ( d 2 d 1 ) ( d 3 ( n 1 ) d 1 ( n 1 ) ) ( d 3 d 1 ) ( d 4 ( n 1 ) d 1 ( n 1 ) ) ( d 4 d 1 ) ( d 5 ( n 1 ) d 1 ( n 1 ) ) ( d 5 d 1 )
( x i , y i , z i ) being the position of the i-th beacon, d i ( n 1 ) the distance from beacon i to the drone’s ( n 1 )th position estimate and d i the experimental distance measured to beacon i. Note that in the two-dimensional algorithm, the z coordinate appearing in matrix A keeps its original value. Once the 2D-GN algorithm has provided an estimate for the drone’s horizontal position, this information is used to update the drone’s height z using trigonometry as follows:
z = d 2 x ( n ) 2 y ( n ) 2
Finally, if the updated position r = ( x ( n ) , y ( n ) , z ) changes above a certain margin of tolerance, the updated height is fed back to the 2D-GN algorithm to obtain a more accurate horizontal position. In other cases, the updated position is provided as the final estimate.

4. Experimental Results

The main objective of this work is to hybridize the ultrasonic and optical technologies to develop an accurate 3D positioning system for UAVs. Figure 9 shows a picture of the actual experimental site, where the performance of the proposed positioning system has been analyzed by defining three different test points: p 1 = ( 0 , 0 , 0.97 ) m, p 2 = ( 0.60 , 0.37 , 1.59 ) and p 3 = ( 0.60 , 0.90 , 1.41 ) and computing a total of 100 positions in each of these points.
First of all, a comparison between the proposed hybrid system and a solely ultrasonic system performing 3D multilateration has been conducted. Table 1 shows the results of this comparison in terms of the positioning Mean Squared Error (MSE) and its corresponding Standard Deviation (SD). As can be seen, the proposed system performs significantly better in all the test positions, with relative improvements between 70% and 80% in the MSE. An important decrease in the SD is also observed, which also indicates an improvement in the system precision. This table can be compared with the results provided in [1], where a UWB system is developed to localize a flying drone. In this work, errors less than 0.176 m for 50% of cases are reported.
Since the proposed positioning strategy is based on an iterative multilateration procedure, it is also important to study the rate of convergence of this process. Figure 10 represents the MSE against the number or iterations in the three tests points. As expected, the number of required iterations increases with the distance from the drone to the vertical z-axis, since the TOF-camera provides an initial estimation of this drone’s height as its range to the origin of coordinates. In any case, a maximum number of five iterations is required to obtain a stable estimation in the furthest point, with a refreshing time of less than one second.
Finally, it is important to remark that the system-measured update rate is about 0.42 s, which is good enough to allow real-time operation in a constrained environment where the drone operation velocity must be limited.

5. Conclusions

This work has proposed the use of a hybrid broadband ultrasonic and optical positioning system to perform the accurate positioning of UAV in indoor environments. The proposed acoustic module is based on a T-CDMA scheme, where the sequential emission of five spread spectrum ultrasonic codes is performed to estimate the vehicle horizontal position following a 2D multilateration procedure. The signals emitted by this module are acquired by a portable receiver attached to the top cover of the UAV. This receiver is based on a MEMS microphone, a microcontroller and a WiFi module for wireless data transmission to an external computer. The optical module is based on a TOF-camera that sends a range-image of the scene to the same computer. This computer first uses this image to obtain an initial estimation of the UAV height, which is combined with the ultrasonic TDOAs to refine the 3D position following an iterative procedure. Experimental results obtained in a real environment confirm a significant improvement in the performance of this system with respect to that of the system that only uses the acoustic module to perform 3D multilateration. These results also show that the proposed iterative algorithm needs a maximum of five iterations to converge to a stable solution.

Acknowledgments

This work was funded by the Spanish Government and the European Regional Development Fund (ERDF) under the projects TARSIUS (TIN2015-71564-C4-4-R), REPNIN (TEC2015-71426-REDT) and SOC-PLC (TEC2015-64835-C3-2-R); and the Regional Government of Extremadura and ERDF-ESF under Project GR15167 and through a pre-doctoral scholarship (45/2016 Exp. PD16030).

Author Contributions

José A. Paredes is the main responsible for the optical subsystem. He has programmed the image processing algorithms, has participated in the experimental testing and has performed the data analysis; Fernando J. Álvarez is the project coordinator and the main responsible for the positioning strategy. He has conceived the experimental implementation and testing of the system; Teodoro Aguilera is the main responsible for the acoustic subsystem. He has programmed the acoustic processing algorithms and has participated in the experimental testing of the system; José M. Villadangos is the main responsible for the hardware design. He has designed both the emitter and receiver ultrasonic modules.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Tiemann, J.; Schweikowski, F.; Wietfeld, C. Design of an UWB indoor-positioning system for UAV navigation in GNSS-denied environments. In Proceedings of the International Conference on Indoor Positioning and Indoor Navigation (IPIN), Banff, AB, Canada, 13–16 October 2015; pp. 1–7. [Google Scholar]
  2. Grzonka, S.; Grisetti, G.; Burgard, W. A Fully Autonomous Indoor Quadrotor. IEEE Trans. Robot. 2012, 28, 90–100. [Google Scholar] [CrossRef]
  3. Otsason, V.; Varshavsky, A.; LaMarca, A.; De Lara, E. Accurate GSM indoor localization. In Proceedings of the International Conference on Ubiquitous Computing (UbiComp), Tokyo, Japan, 11–14 September 2005; p. 903. [Google Scholar]
  4. Teulière, C.; Eck, L.; Marchand, E.; Guénard, N. 3D model-based tracking for UAV position control. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Taipei, Taiwan, 18–22 October 2010; pp. 1084–1089. [Google Scholar]
  5. Deris, A.; Trigonis, I.; Aravanis, A.; Stathopoulou, E.K. Depth Cameras on UAVs: A First Approach. In Proceedings of the ISPRS-International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Nafplio, Greece, 1–3 March 2017; Volume XLII-2/W3, pp. 231–236. [Google Scholar]
  6. Mustafah, Y.M.; Azman, A.W.; Akbar, F. Indoor UAV Positioning Using Stereo Vision Sensor. Procedia Eng. 2012, 41, 575–579. [Google Scholar] [CrossRef]
  7. Wang, F.; Cui, J.; Phang, S.K.; Chen, B.M.; Lee, T.H. A mono-camera and scanning laser range finder based UAV indoor navigation system. In Proceedings of the International Conference on Unmanned Aircraft Systems (ICUAS), Atlanta, GA, USA, 28–31 May 2013; pp. 694–701. [Google Scholar]
  8. Wang, C.; Li, K.; Liang, G.; Chen, H.; Huang, S.; Wu, X. A Heterogeneous Sensing System-Based Method for Unmanned Aerial Vehicle Indoor Positioning. Sensors 2017, 17. [Google Scholar] [CrossRef] [PubMed]
  9. Kumar, G.A.; Patil, A.K.; Patil, R.; Park, S.S.; Chai, Y.H. A LiDAR and IMU Integrated Indoor Navigation System for UAVs and Its Application in Real-Time Pipeline Classification. Sensors 2017, 17. [Google Scholar] [CrossRef] [PubMed]
  10. López, E.; García, S.; Barea, R.; Bergasa, L.M.; Molinos, E.J.; Arroyo, R.; Romera, E.; Pardo, S. A Multi-Sensorial Simultaneous Localization and Mapping (SLAM) System for Low-Cost Micro Aerial Vehicles in GPS-Denied Environments. Sensors 2017, 17. [Google Scholar] [CrossRef] [PubMed]
  11. Mautz, R. Indoor Positioning Technologies. Ph.D. Thesis, Institute of Geodesy and Photogrammetry, Department of Civil, Environmental and Geomatic Engineering, ETH Zurich, Zürich, Switzerland, 2012. [Google Scholar]
  12. Mainetti, L.; Patrono, L.; Sergi, I. A survey on indoor positioning systems. In Proceedings of the 22nd International Conference on Software, Telecommunications and Computer Networks (SoftCOM), Split, Croatia, 17–19 September 2014; pp. 111–120. [Google Scholar]
  13. Ward, A.; Jones, A.; Hopper, A. A new location technique for the active office. IEEE Pers. Commun. 1997, 4, 42–47. [Google Scholar] [CrossRef]
  14. Addlesee, M.; Curwen, R.; Hodges, S.; Newman, J.; Steggles, P.; Ward, A.; Hopper, A. Implementing a sentient computing system. Computer 2001, 34, 50–56. [Google Scholar] [CrossRef]
  15. Foxlin, E.; Harrington, M. Constellation: A wide-range wireless motion-tracking system for augmented reality and virtual set applications. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques, Orlando, FL, USA, 19–24 July 1998; pp. 371–378. [Google Scholar]
  16. Priyantha, N.B.; Chakraborty, A.; Balakrishnan, H. The Cricket Location-support System. In Proceedings of the 6th Annual International Conference on Mobile Computing and Networking (MobiCom’00), Boston, MA, USA, 6–11 August 2000; ACM: New York, NY, USA, 2000; pp. 32–43. [Google Scholar]
  17. Priyantha, N.B. The Cricket Indoor Location System. Ph.D. Thesis, Massachusetts Institute of Technology, Cambridge, MA, USA, 2005. [Google Scholar]
  18. Skolnik, M.I. Introduction to Radar Systems, 2nd ed.; McGraw Hill Book Co.: New York, NY, USA, 1980. [Google Scholar]
  19. Peremans, H.; Audenaert, K.; Campenhout, J.M.V. A high-resolution sensor based on tri-aural perception. IEEE Trans. Robot. Autom. 1993, 9, 36–48. [Google Scholar] [CrossRef]
  20. Jörg, K.W.; Berg, M. Sophisticated mobile robot sonar sensing with pseudo-random codes. Robot. Auton. Syst. 1998, 25, 241–251. [Google Scholar] [CrossRef]
  21. Ureña, J.; Mazo, M.; García, J.J.; Hernández, Á.; Bueno, E. Classification of reflectors with an ultrasonic sensor for mobile robot applications. Robot. Auton. Syst. 1999, 29, 269–279. [Google Scholar] [CrossRef]
  22. Hazas, M.; Ward, A. A Novel Broadband Ultrasonic Location System. In Proceedings of the 4th International Conference on Ubiquitous Computing, Göteborg, Sweden, 29 September–1 October 2002; Borriello, G., Holmquist, L.E., Eds.; Springer: Berlin/Heidelberg, Germany, 2002; pp. 264–280. [Google Scholar]
  23. Hazas, M.; Ward, A. A high performance privacy-oriented location system. In Proceedings of the First IEEE International Conference on Pervasive Computing and Communications (PerCom 2003), Fort Worth, TX, USA, 26 March 2003; pp. 216–223. [Google Scholar]
  24. Villadangos, J.M.; Urena, J.; Mazo, M.; Hernandez, A.; Alvarez, F.; Garcia, J.J.; Marziani, C.D.; Alonso, D. Improvement of ultrasonic beacon-based local position system using multi-access techniques. In Proceedings of the IEEE International Workshop on Intelligent Signal Processing, Faro, Portugal, 1–3 September 2005; pp. 352–357. [Google Scholar]
  25. Álvarez, F.J.; Hernández, Á.; Moreno, J.A.; Pérez, M.C.; Ureña, J.; Marziani, C.D. Doppler-tolerant receiver for an ultrasonic LPS based on Kasami sequences. Sens. Actuators A Phys. 2013, 189, 238–253. [Google Scholar] [CrossRef]
  26. Álvarez, F.J.; Aguilera, T.; López-Valcarce, R. CDMA-based acoustic local positioning system for portable devices with multipath cancellation. Digit. Signal Process. 2017, 62, 38–51. [Google Scholar] [CrossRef]
  27. Seco, F.; Prieto, J.C.; Ruiz, A.R.J.; Guevara, J. Compensation of Multiple Access Interference Effects in CDMA-Based Acoustic Positioning Systems. IEEE Trans. Instrum. Meas. 2014, 63, 2368–2378. [Google Scholar] [CrossRef]
  28. Aguilera, T.; Paredes, J.A.; Álvarez, F.J.; Suárez, J.I.; Hernández, A. Acoustic local positioning system using an iOS device. In Proceedings of the International Conference on Indoor Positioning and Indoor Navigation, Montbeliard-Belfort, France, 28–31 October 2013; pp. 1–8. [Google Scholar]
  29. Pérez, M.C.; Gualda, D.; Villadangos, J.M.; Ureña, J.; Pajuelo, P.; Díaz, E.; García, E. Android application for indoor positioning of mobile devices using ultrasonic signals. In Proceedings of the International Conference on Indoor Positioning and Indoor Navigation (IPIN), Alcala de Henares, Spain, 4–7 October 2016; pp. 1–7. [Google Scholar]
  30. Kohoutek, T.K.; Mautz, R.; Donaubauer, A. Real-time indoor positioning using range imaging sensors. In Real-Time Image and Video Processing; SPIE: Bellingham, WA, USA, 2010; Volume 7724. [Google Scholar]
  31. Kohoutek, T.K.; Droeschel, D.; Mautz, R.; Behnke, S. Indoor Positioning and Navigation Using Time-of-Flight Cameras. In TOF Range-Imaging Cameras; Remondino, F., Stoppa, D., Eds.; Springer: Berlin/Heidelberg, Germany, 2013; pp. 165–176. [Google Scholar]
  32. Cunha, J.; Pedrosa, E.; Cruz, C.; Neves, A.J.; Lau, N. Using a Depth Camera for Indoor Robot Localization and Navigation. In Proceedings of the Robotics Science and Systems Conference, Los Angeles, CA, USA, 27–30 June 2011. [Google Scholar]
  33. Biswas, J.; Veloso, M. Depth camera based indoor mobile robot localization and navigation. In Proceedings of the IEEE International Conference on Robotics and Automation, Saint Paul, MN, USA, 14–18 May 2012; pp. 1697–1702. [Google Scholar]
  34. Vaz, M.; Ventura, R. Real-time ground-plane based mobile localization using depth camera in real scenarios. In Proceedings of the IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), Espinho, Portugal, 14–15 May 2014; pp. 187–192. [Google Scholar]
  35. Jia, L.; Radke, R.J. Using Time-of-Flight Measurements for Privacy-Preserving Tracking in a Smart Room. IEEE Trans. Ind. Inform. 2014, 10, 689–696. [Google Scholar] [CrossRef]
  36. Stahlschmidt, C.; Gavriilidis, A.; Velten, J.; Kummert, A. People Detection and Tracking from a Top-View Position Using a Time-of-Flight Camera. In Proceedings of the 6th International Conference on Multimedia Communications, Services and Security (MCSS 2013), Krakow, Poland, 6–7 June 2013; Dziech, A., Czyżewski, A., Eds.; Springer: Berlin/Heidelberg, Germany, 2013; pp. 213–223. [Google Scholar]
  37. Kasami, T. Weigh Distribution Formula for Some Class of Cyclic Codes; Technical Report R-285; Coordinated Science Laboratory: Urbana, IL, USA, 1968. [Google Scholar]
  38. Pro-Wave Electronics Corporation. Air Ultrasonic Ceramic Transducers 328ST/R160, Product Specification; Pro-Wave Electronics Corporation: New Taipei City, Taiwan, 2014. [Google Scholar]
  39. Knowles Acoustics. SiSonic Design Guide, Application Note rev. 3.0; Knowles Acoustics: Itasca, IL, USA, 2011. [Google Scholar]
  40. STMicroelectronics 32-bit ARM Cortex M3 Microcontroller. 2017. Available online: http://www.st.com/en/microcontrollers/stm32f103cb.html (accessed on 29 December 2017).
  41. DT-06 Wireless WiFi Serial Port Transparent Transmission Module. Shenzhen Doctors of Intelligence & Technology Co., Ltd. 2017. Available online: ttl.doit.am (accessed on 29 December 2017). (In Chinese).
  42. Prieto, J.C.; Jimenez, A.R.; Guevara, J.; Ealo, J.L.; Seco, F.; Roa, J.O.; Ramos, F. Performance Evaluation of 3D-LOCUS Advanced Acoustic LPS. IEEE Trans. Instrum. Meas. 2009, 58, 2385–2395. [Google Scholar] [CrossRef]
  43. Álvarez, F.J.; Esteban, J.; Villadangos, J.M. High Accuracy APS for Unmanned Aerial Vehicles. In Proceedings of the 13th International Conference on Advanced Technologies, Systems and Services in Telecommunications (TELSIKS), Niš, Serbia, 18–20 October 2017; pp. 217–223. [Google Scholar]
  44. Lange, R. 3D Time-of-Flight Distance Measurement with Custom Solid-State Image Sensors in CMOS/CCD-Technology. Ph.D. Thesis, University of Siegen, Siegen, Germany, 2000. [Google Scholar]
  45. Langmann, B. Wide Area 2D/3D Imaging. Development, Analysis and Applications; Springer: Berlin, Germany, 2014. [Google Scholar]
  46. Hussmann, S.; Edeler, T.; Hermanski, A. Machine Vision-Applications and Systems; InTech: Rijeka, Croatia, 2012; Chapter 4: Real-Time Processing of 3D-ToF Data; pp. 1–284. [Google Scholar]
  47. Chiabrando, F.; Chiabrando, R.; Piatti, D.; Rinaudo, F. Sensors for 3D Imaging: Metric Evaluation and Calibration of a CCD/CMOS Time-of-Flight Camera. Sensors 2009, 9, 10080–10096. [Google Scholar] [CrossRef] [PubMed]
  48. Lindner, M.; Schiller, I.; Kolb, A.; Koch, R. Time-of-Flight Sensor Calibration for Accurate Range Sensing. Comput. Vis. Image Underst. 2010, 114, 1318–1328. [Google Scholar] [CrossRef]
  49. Matheis, F.; Brockherde, W.; Grabmaier, A.; Hosticka, B.J. Modeling and calibration of 3D-Time-of-Flight pulse-modulated image sensors. In Proceedings of the 20th European Conference on Circuit Theory and Design (ECCTD), Linkoping, Sweden, 29–31 August 2011; pp. 417–420. [Google Scholar]
  50. MTK USB Evaluation KIT for TIM-UP-19k-S3-Spartan6. Bluetechnix Lab GmbH. Bluetechnix (Wien, Austria) 2017. Available online: https://support.bluetechnix.at/wiki/MTK_USB_Evaluation_KIT_for_TIM-UP-19k-S3-Spartan6 (accessed on 29 December 2017).
Figure 1. General view of the proposed system. ULPS, Ultrasonic Local Positioning System.
Figure 1. General view of the proposed system. ULPS, Ultrasonic Local Positioning System.
Sensors 18 00089 g001
Figure 2. Ultrasonic emitter module, (a) Block diagram of the ultrasonic emitter module, (b) Beacons’ emission pattern.
Figure 2. Ultrasonic emitter module, (a) Block diagram of the ultrasonic emitter module, (b) Beacons’ emission pattern.
Sensors 18 00089 g002
Figure 3. Ultrasonic receiver module.
Figure 3. Ultrasonic receiver module.
Sensors 18 00089 g003
Figure 4. (a) Acquired signal for the receiver and (b) normalized cross-correlation with a set of five Kasami codes assigned to the beacons.
Figure 4. (a) Acquired signal for the receiver and (b) normalized cross-correlation with a set of five Kasami codes assigned to the beacons.
Sensors 18 00089 g004
Figure 5. TOF camera working principle. At the top, a temporal diagram representing the phase shift between the emitted and the received wave. At the bottom, a scene diagram: the angle between incident and reflected waves is near zero because the distance d is much greater than the separation between the LEDs (emitters) and the lens (receiver).
Figure 5. TOF camera working principle. At the top, a temporal diagram representing the phase shift between the emitted and the received wave. At the bottom, a scene diagram: the angle between incident and reflected waves is near zero because the distance d is much greater than the separation between the LEDs (emitters) and the lens (receiver).
Sensors 18 00089 g005
Figure 6. Images from the TOF camera. The configuration has been chosen so as to reach all possible distances in a proper way, as can be appreciated. In this case, the drone flew approximately 1.9 m from the lens. (a) Image of amplitudes, (b) Image of distances, (c) Drone recognition after subtracting the background.
Figure 6. Images from the TOF camera. The configuration has been chosen so as to reach all possible distances in a proper way, as can be appreciated. In this case, the drone flew approximately 1.9 m from the lens. (a) Image of amplitudes, (b) Image of distances, (c) Drone recognition after subtracting the background.
Sensors 18 00089 g006
Figure 7. Image of the SYMA X5C quadcopter drone.
Figure 7. Image of the SYMA X5C quadcopter drone.
Sensors 18 00089 g007
Figure 8. Flux diagram of the positioning algorithm.
Figure 8. Flux diagram of the positioning algorithm.
Sensors 18 00089 g008
Figure 9. Snapshot of the experimental site, with the ULPS installed in the ceiling of an office room and the TOF-camera placed at the origin of coordinates.
Figure 9. Snapshot of the experimental site, with the ULPS installed in the ceiling of an office room and the TOF-camera placed at the origin of coordinates.
Sensors 18 00089 g009
Figure 10. Number of necessary algorithm iterations to converge to a solution.
Figure 10. Number of necessary algorithm iterations to converge to a solution.
Sensors 18 00089 g010
Table 1. Mean Squared Error (MSE) and its Standard Deviation (SD) in each position under study.
Table 1. Mean Squared Error (MSE) and its Standard Deviation (SD) in each position under study.
(m)Position 1Position 2Position 3
Acoustic System0.140(0.089)0.350(0.032)0.303(0.257)
Hybrid System0.037(0.009)0.067(0.017)0.081(0.061)
MSESDMSESDMSESD

Share and Cite

MDPI and ACS Style

Paredes, J.A.; Álvarez, F.J.; Aguilera, T.; Villadangos, J.M. 3D Indoor Positioning of UAVs with Spread Spectrum Ultrasound and Time-of-Flight Cameras. Sensors 2018, 18, 89. https://doi.org/10.3390/s18010089

AMA Style

Paredes JA, Álvarez FJ, Aguilera T, Villadangos JM. 3D Indoor Positioning of UAVs with Spread Spectrum Ultrasound and Time-of-Flight Cameras. Sensors. 2018; 18(1):89. https://doi.org/10.3390/s18010089

Chicago/Turabian Style

Paredes, José A., Fernando J. Álvarez, Teodoro Aguilera, and José M. Villadangos. 2018. "3D Indoor Positioning of UAVs with Spread Spectrum Ultrasound and Time-of-Flight Cameras" Sensors 18, no. 1: 89. https://doi.org/10.3390/s18010089

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop