^{*}

This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).

The paper presents an algorithm for estimating a pedestrian location in an urban environment. The algorithm is based on the particle filter and uses different data sources: a GPS receiver, inertial sensors, probability maps and a stereo camera. Inertial sensors are used to estimate a relative displacement of a pedestrian. A gyroscope estimates a change in the heading direction. An accelerometer is used to count a pedestrian's steps and their lengths. The so-called probability maps help to limit GPS inaccuracy by imposing constraints on pedestrian kinematics, e.g., it is assumed that a pedestrian cannot cross buildings, fences

GPS-NAVSTAR (Global Positioning System-NAVigation Signal Timing And Ranging), popularly known as the GPS system, has considerably gained in civilian interests, since May 2000. Earlier, the system was practically reserved for military purposes. The positioning accuracy for the civilian sector was

There are many techniques to improve the location accuracy. Along coasts, for marine purposes, special ground DGPS (Differential GPS) reference stations broadcast differential corrections that allow a GPS receiver to eliminate tropospheric, ionospheric, ephemeris and clock errors. The overall error is reduced to 10 m with accuracy decreasing by 1 m with each 150 km increase in distance from the reference station. The corrections are transmitted on a

A-GPS (Assisted GPS) is a technique that downloads from the Internet the data concerning GPS satellite constellation [

Apart from L1 = 1,227 MHz, a second frequency called L2C = 1,575 MHz, has been made available to the civilian sector with the aim of reducing the ionospheric and tropospheric errors which are a well defined function of frequency. Also, the so-called augmentation systems effectively reduce tropospheric and ionospheric errors by sending differential corrections from geostationary satellites directly to GPS receivers. The geostationary satellites imitate also GPS satellites and improve mainly the vertical accuracy. EGNOS (European Geostationary Navigation Overlay Service) works in Europe, WAAS (Wide Area Augmentation System) in the US, MSAS (Multi-functional Satellite Augmentation System) in Japan and GAGAN (GPS aided Geo-Augmented Navigation) in India.

Real Time Kinematics technique achieves the accuracy of millimetres in an open space and is designated for cartographic measurements. However, two GPS receiver are necessary. The base receiver is placed in a known position and calculates tropospheric, ionospheric, ephemerids and clock errors of satellites. The corrections are sent via a radio link to a mobile GPS receiver. In the trials reported in [

All the above mentioned techniques are helpless against multipath propagation errors. Algorithms harnessed in car navigation alleviate these errors by taking advantage of car kinematics and a comparatively sparse network of roads, as described, e.g., in [

This work presents a navigation scheme using GPS readouts, digital maps, inertial sensors and stereovision images with an aim of navigating a blind pedestrian. An accelerometer is used to detect pedestrian's strides and estimate their length. A gyroscope serves for estimating the heading direction. Those data sources help eliminate gross positioning errors and outliers. The digital maps are used twofold. Firstly, the so-called probability map is built to eliminate improbable user transitions, like traversing water ponds, crossing walls and buildings, etc. Secondly, a 3D model of the environment is built. The model is compared to stereoscopic images recorded by a mobile stereo camera. Interestingly, this comparison of 3D geometry of the environment provides good positioning accuracy in the surrounding of buildings, where GPS readouts are compromised. The proposed scheme employs the particle filter, also known as a sequential Monte Carlo method.

The topic of pedestrian navigation including navigation aids for blind pedestrians has been described in many publications [

A travelled distance of a pedestrian can be estimated with surprisingly good accuracy by measuring the length of steps. This is done by analysing the acceleration in the gravity axis [

A heading direction can simply be read out from a magnetic compass, optionally supported by a gyroscope. A compass is sensitive to local distortions of a magnetic field due to cars, power lines

The combination of GPS and inertial sensors readouts provides continuous estimates during GPS outages in harsh environments like tunnels, underground passages, dense urban areas etc. Positioning data from these two sources are usually integrated by the Extended Kalman filter or particle filter, which perform well when errors can be modelled by white noise which has the property of being uncorrelated with itself. However, the GPS errors are characterized by coloured noise [

To improve the accuracy of dead-reckoning techniques, an additional source of the absolute pedestrian location should be introduced. Experiments in [

A further improvement of positioning accuracy requires positioning with respect to known landmarks,

On the other hand, precise maps of outdoor or indoor environments may already be available, e.g., a plan of a building or plans of a city which are accurate to single centimetres, accuracy hardly achievable by any SLAM techniques. There are different sensors used to compare the environmental map with a robot's or pedestrian's location. Ultrasound sensors measure distance with 1% accuracy, being at the same time very cheap and compact. The maximum achievable distance is

In [

Finally, stereovision is a passive imaging technique that can be used indoors and outdoors. The cameras can be compact and inexpensive. However, stereovision imaging is limited to the environments featuring good lighting conditions, requires calibration of the stereovision optics and offers worse depth estimation accuracy than the earlier outlined active methods [

The work presented here combines global and local positioning techniques. GPS location estimates are augmented by dead reckoning techniques derived from inertial sensors,

A block diagram of the proposed system is presented in

The built electronic module, see

The 6DOF sensor, ADIS16355 from Analog Devices, comprises a 3-axial accelerometer and 3-axial gyro. The data from the sensor is read out through a digital interface, SPI. The samples are of 14-bit resolution whereby practically the three least significant bits are random. The ADIS16355 has been superseded by much less noisy ADIS16375. The static parameters of the former were investigated by the Allan variance [

The gyroscope is used to estimate the pedestrian orientation. Angular velocity

_{w}_{f}_{w}_{ARW}(_{ARW}(_{ARW} grows at a rate of

Similarly, integrated flicker noise _{f}_{F}_{F}_{F}

Inertial sensors also suffer from other errors like non-linearity or temperature random bias. The former error is ±0.3°/s. The experienced random bias due to temperature was much higher than the manufacturer claimed and it reads 0.5°/s for ±3

The accelerometer is used in the project to estimate the user's steps and their lengths. This method is more precise than integrating twice the acceleration readouts to obtain the displacement. The accelerations readouts are also used to estimate the gravity direction which is necessary for correct calculation of the orientation change. By analogy, velocity random walk is a product of integration of white noise in acceleration. On average, the velocity random walk was

The PCB board houses a standard GPS receiver with an integrated ceramic antenna, measuring 15 mm by 15 mm. The receiver is built on the MTK MT3329 chipset and supports WAAS, EGNOS, MSAS, GAGAN corrections. GPS readouts are based on the WGS84 coordinate system aligned in the centre of the Earth, being represented by an ellipsoid. The conversion to a local coordinate system follows in two steps. Firstly, the polar coordinates, _{WGS84}, _{WGS84}, _{WGS84}). This step, along with associated errors and corrections, is described in detail in [

The rotation _{x} is directed along with longitude, _{y} with latitude (towards the pole), _{z} is determined by _{x} and _{y}. The conversion accuracy is in an order of 10 cm for 20 km from the centre of the local coordinate system.

A GPS receiver provides the HDOP parameter (Horizontal Dilution of Precision) informing about the accuracy of the estimated coordinates,

The standard deviations of estimating _{x}_{y}_{e}_{e}_{e}_{e}_{GPS}(_{e}, σ_{GPS}) is accordingly scaled by a factor of 2. The coefficient _{GPS} in

If a GPS receiver loses track of satellites, prediction of its position is based on the previous velocities and position. The first order autocorrelation coefficients for _{x}_{y}

_{x}_{y}

Another problem associated with GPS readouts is the correspondence of the HDOP parameter with the actual accuracy of latitude and longitude. _{e}

When a person walks, the body undulates in unison with the steps. The step-detection algorithm examines the acceleration of the body only in the vertical direction (parallel to the gravity axis). The gravity axis is obtained from the algorithm presented in Section 3.3. The acceleration signal also reflects knee movements, body swings, sensor jerks and so forth. A 3 Hz 4-order low-pass Butterworth filter was applied to eliminate undesired constituents. The step detection algorithm is presented in _{min} and maximum _{max} acceleration in the filtered signal are detected. The difference between the peaks is raised to the empirical power of 0.25. The result is multiplied by the scale factor

Tilt estimation,

Calculation of the relative change in the heading direction. When the sensor is not levelled, the gyroscope's axis does not coincide with the pivotal axis of the pedestrian. When the user turns, the relative change in the heading direction is underestimated. For example, when the gyroscope axis is tilted by 25° from the pivotal axis, the angular velocity is underestimated by ≈ 10%. Consequently, the relative direction change might be 81° instead of 90°—

Estimation of the camera orientation with respect to the ground. The stereo camera is used to measure distance to nearest objects like buildings. When the camera is not levelled the measured distance will be overestimated.

A human rotates along the gravity axis when changing the walking direction. Thus, the task of estimating the correct change in direction comes down to finding the gravity axis. The problem is known in aeronautics as Inertial Navigation Systems. The estimation of six coordinates of an aeroplane (three for orientation and three for position) is not a trivial task. A standard approach based on off-the-shelf sensors and cosine direction matrices is very inaccurate after few dozens of seconds [

_{x}, a_{y}_{z}^{2}. When a user walks, the accelerations will fluctuate by ±0.25 g, depending on the walking style. The acceleration vector due to walking is treated here as measurement noise. Hence, calculating the direction of the gravity axis from

This reduces the number of degrees of freedom by one. Gyroscopes can be used to improve the estimation of the sensor orientation. The transition equation of the Extended Kalman Filter (EKF) is given by _{y}_{y}

_{y}_{x}

The measurement equation of the EKF is given by

The accelerometers readouts are corrupted by noise v(_{x}_{y}_{x}_{y}

The angular velocity can be corrected by

As mentioned before, this approach is valid for limited range of pitch and roll, due to singularity in _{z}

A gyroscope provides an angular velocity

The digital map is divided into three types of area. The areas have their associated weights:

forbidden areas—buildings, ponds, walls, fences _{map}(

probable areas—lawns, fields _{map}(

preferred areas—pavements, streets, squares, alleys _{map}(

_{map}(

A stereovision camera [_{L}_{R}_{L}_{R}_{d}

For the legibility sake, the stereovision system can be treated as one camera, whose simplified coordinate system is shown in

The coordinates of a point (_{P}

The depth of a point (_{p}

The distance from the Y axis of the camera to a point with the (_{Y}

Let us define a function which returns the distance from the

The data from the stereovision camera is digitized. The (_{p}, y_{p}_{pix}, _{pix}). The conversion follows through

_{pix} is the focal length of the camera expressed in pixels and is provided by the manufacturer in the technical specification. The difference between the maximal and minimal value of _{pix} is called the horizontal resolution of the camera and is denoted by _{res}. The vertical resolution _{res} is defined by analogy. A depth map or a depth picture

The coordinates (_{pix}, _{pix}) through transformations _{pix}, _{pix}) is represented by a floating point-number. Using the identity

As a matter of fact, stereovision cameras first determine the disparity map _{d}

As the object recedes from the camera, the disparity _{d}_{d}_{d}_{L}, x_{R}_{d}_{d}_{d}_{d}

The error of determining the disparity does not only come from quantizing x_{d}, in which case, the disparity maps presented in Section 3.8 would be devoid of any artifacts or falsely calculated disparities—see [

The camera used in this work calculates disparity by correlating small blocks (e.g., 7 × 7 pixels) and looking for best match. The probability density function of the disparity error is disputable in this case. Let it be the Gaussian distribution, expressed by _{d}

The probability density function of Δ_{z}

_{d}

Thus, _{z}

The shape of the function _{z}

A precise 3D map of an urban terrain was provided by the local cartographic office of Lodz (MODGiK). The map is divided in layers containing: roads, buildings, entrances to buildings, steps, fences, lawns, trees, bushes, ponds, lamp posts and many other data not used in the project. The accuracy of building deployment is in an order of single centimetres and the buildings' heights were measured with a bit worse accuracy. The radius of trees' crowns was assumed 3 m and the radius of lamp posts 0.5 m.

Estimated location based on the stepmeter, gyroscope and digital maps is accurate to within 15 m (a street with pavements on both sides) [

Let us use the local coordinate system defined in Section 3.1.2. Let the vector [_{S}, y_{S}, z_{S}, φ_{S}^{T}_{S}_{V}, y_{V}, z_{V}_{V}_{S}_{V}_{L}_{S}, L_{V}

Formally, the problem comes down to finding the vector [_{V}, y_{V}, z_{V}, φ_{V}^{T}_{l}_{S}, L_{V}_{S}_{V}

Assuming _{S}_{V}_{L}_{S}_{L}_{S}_{V}_{S}_{S}_{S}

Reject the part of the depth map _{S}

Divide vertically the upper part of _{S}_{S}_{j}, j_{S}_{S}_{V}

for _{S}

From the area _{j}_{j}_{W}_{W}_{W}

Check if _{j}

Calculate the angle _{j}_{j}

Calculate the distance _{Sj}_{j}

Check what the virtual camera sees at the angle _{j}_{V}, y_{V}, z_{V}, φ_{V}_{Vj}

Compare the distances _{Sj}_{Vj}_{Lj}

Return the comparison error between _{S}_{V}_{L}_{S}, L_{V}_{L}_{1} · _{L}_{2} · _{L}_{3} · _{L}_{4} · f_{L}_{5}.

_{j}

_{j}_{j}

_{j}_{j}

Window _{j}_{j}_{thr}, where an empirical value of _{thr} ≈ 0.8.

_{j}_{j}^{(c)} denotes the _{j}

_{Sj}_{j}_{j}

_{Vj}_{j}

_{Sj}_{Vj}_{Lj}_{Lj}_{j}_{j}_{thr}. The function _{Lj}_{Sj}_{Vj}_{Sj}_{Lj}

_{L}

The 3D match algorithm is evoked 10 times a second.

The particle filter is a sequential version of the Monte Carlo method [_{i}

_{i}_{i}_{i}_{i}_{i}, y_{i}, φ_{i}^{T}, x_{i}, y_{i}_{i}

_{i}

The implementation of the particle filtering algorithm for processing the positioning data is as follows:

_{i}(1) are randomly initialized according to a given distribution. The weights _{i}

_{i}_{i}

_{i}_{i}_{i}_{i}

^{−1} then _{i}^{−1}.

Go to point 2.

_{i}_{i}

GPS. The measurement update for the GPS readouts are given by

_{GPS} is a coefficient which enables to weight GPS readouts with appropriate importance. Larger values of _{GPS} makes the _{GPS}() function more tolerant to GPS errors.

comparison between the 3D virtual environment model and depth picture retrieved from the stereo camera. The measurement update equation is given by

For each particle, a virtual environment is generated individually. The coordinates of the virtual camera are set to [_{i}, y_{i}, h, φ_{i}_{V,i}_{L}_{S}, L_{V,i}^{10/}^{s}^{·10}^{s}, c.f.

The trials were carried out at the campus of Lodz University of Technology, in an area of

The results are presented by plots of paths, a histogram and a cumulated histogram of errors and a table that summarizes the errors. An interesting question is how much a given data source improves the accuracy of the positioning. _{e}

Root-mean-square value, denoted by RMS_{e}

Mean error, denoted by _{e}

CEP50—Circle of Error Probability 50%—the radius of a circle encompassing 50% of errors,

CEP90—the radius of a circle encompassing 90% of errors,

CEP95—the radius of a circle encompassing 95% of errors,

CEP99—the radius of a circle encompassing 99% of errors, i.e., 99th percentile,

Maximal error—the maximum value of an error recorded during a trial, denoted by max(_{e}

The results were calculated based on

It is interesting to note that when the initial position is known, the location of a pedestrian can be estimated without having to use a GPS receiver. The setup number 5 (see

As mentioned before, on the bases of GPS readouts and the probability map, a user's location can be estimated within

The presented algorithm works off-line. It is quite computationally demanding. The particle filtering algorithm requires generation of

The project is financed by the National Centre of Research and Development of Poland in years 2010–2013 under the grant NR02-0083-10.

The block diagram of the system. The electronic module is a dedicated electronic circuit comprised of a 6DOF sensor, GPS receiver and microcontroller. The module is connected with the PC via USB interface. The stereovision block represents a stereo camera.

A picture of the built electronic module. The black box is a 6DOF sensor—3-axial gyroscope and 3-axial accelerometer. The orange cuboid is an antenna of a GPS receiver. The module is connected with the PC through a mini-USB socket.

The relation between the values of the HDOP parameter and the distance errors _{e}

The algorithm for estimating the number of steps and their lengths.

The local coordinate system of the 6DOF sensor. _{x}, ω_{y}_{z}_{x}, α_{y}_{z}

The relationship between the angles _{x}_{y}

An example of a probability map of the University Campus. Forbidden, probable and preferred areas are denoted by black, grey and white respectively. The streets at the campus are paved and have the same texture as pavements. A blind user would rather refrain from walking through lawns.

The local coordinate system of the stereovision camera. _{P}, y_{P}_{P}

An illustration of the quantization effect on estimating the _{d}_{d}_{d}

(_{z}_{z}_{d}

(

(_{j}_{1} through _{5}. One can see that for windows _{2} and _{3} the depth map was determined correctly and these windows should be compared with corresponding windows in the _{V}_{d}_{j} as a function of the _{pix} coordinate in the disparity picture. For _{1}, _{4} and _{5} windows, the disparity _{d}

(_{Lj}_{j}_{thr}. Example values of _{L}_{Lj}_{j}_{thr} and Δ_{Sj}_{Lj}

An explanation of the unperturbed state transition: _{z}_{z}

(

(

(

(

(

(

(

Results of the trial at the University Campus. Seven setups of data sources were simulated. The sign ‘+’ in a column denotes that a given data source was included in the simulation. Otherwise the data source is not used. Setup number 4 was carried out with all data sources on. The results for this setup are best.

Setup | 1 | 2 | 3 | 4 | 5 | 6 | 7 |

GPS | + | + | + | + | |||

Inertialsensors | + | + | + | + | + | + | |

Probability map | + | + | + | + | |||

3D matching algorithm | + | + | |||||

RMS_{e} |
5.23 | 4.96 | 2.10 | 1.26 | 1.65 | 4.39 | 107.5 |

_{e} |
3.27 | 3.45 | 1.56 | 0.87 | 0.95 | 2.94 | 86.4 |

CEP50 [m] | 2.19 | 2.47 | 1.24 | 0.57 | 0.55 | 1.89 | 76.4 |

CEP90 [m] | 6.54 | 6.79 | 3.39 | 2.03 | 2.13 | 6.92 | 182 |

CEP95 [m] | 10.7 | 12.0 | 4.28 | 2.90 | 3.19 | 9.09 | 210 |

CEP99 [m] | 23.3 | 18.2 | 6.41 | 4.50 | 7.02 | 16.0 | 261 |

max(_{e} |
30.6 | 20.6 | 8.64 | 5.24 | 12.8 | 25.5 | 265 |

Figure |