A Neural Network Approach for Building An Obstacle Detection Model by Fusion of Proximity Sensors Data

Proximity sensors are broadly used in mobile robots for obstacle detection. The traditional calibration process of this kind of sensor could be a time-consuming task because it is usually done by identification in a manual and repetitive way. The resulting obstacles detection models are usually nonlinear functions that can be different for each proximity sensor attached to the robot. In addition, the model is highly dependent on the type of sensor (e.g., ultrasonic or infrared), on changes in light intensity, and on the properties of the obstacle such as shape, colour, and surface texture, among others. That is why in some situations it could be useful to gather all the measurements provided by different kinds of sensor in order to build a unique model that estimates the distances to the obstacles around the robot. This paper presents a novel approach to get an obstacles detection model based on the fusion of sensors data and automatic calibration by using artificial neural networks.


Introduction
Obstacle avoidance is one of the main challenges for any practical design in robotics. Many approaches to face this problem can be found in the literature. Some of them use optical sensors and computer vision for general purpose object detection tasks [1][2][3]. In particular, proximity sensors are a widely-implemented solution for obstacle detection and collision avoidance in mobile robotics. These sensors are normally located around the robot in order to detect and avoid objects when it navigates in a dynamic environment [4][5][6].
The proximity sensors are able to detect the presence of nearby objects without physical contact, providing an estimation of the distance to the obstacles. Commonly, this kind of sensor emits electromagnetic fields (e.g., infrared light) or sound pulses (e.g., ultrasonic), in order to detect changes in the field or returned signal [7]. For this reason, the surface properties of the objects have an important influence on the estimation of the distance. The other aspect that has to be taken into account is the time response of the sensor because the distance is estimated based on this value.
The infrared (IR) and ultrasonic (US) sensors are the most extensively used in robotics due to their low cost and fast time response, which make them an attractive option for real-time object detection [8]. These sensors use different technologies to estimate the distance to an object. On the one hand, IR sensors use light of wavelength in the range of 760 nm (IR spectrum) for sensing the reflected light intensity. On the other hand, US sensors use a high-frequency sound wave (40 kHz) in order to detect the returning pulses (echo) [9]. IR sensors are used to measure short distances (around 0.1 to 1.5 m) with a narrow (line) beam shape, while US sensors are used to measure long distances (around 0.15 to 6.5 m) with a conical beam shape. The width of the beam is a function of the surface area, frequency, and type of transducers [9,10].
In both cases, the distance estimation depends greatly on what the object is made of. For example, since the IR sensors are a vision-based measurement, the colour of the object and the environment light condition could affect their outcome [11,12]. On the other hand, because of the US sensors are sound-based, they are useful for transparent objects and able to work under poor lighting conditions, but this kind of sensors is much more sensitive to mirror-like surfaces, and the measurement can be affected by environmental parameters such as temperature, humidity, ambient noise, among others [8]. A more detailed description of this issue can be found in [9], where the authors make an analysis of the influence on the behaviour of both sensors for different types of materials such as: cardboard, paper, sponge, wood, plastic, rubber and tile.
Despite the popularity of proximity sensors, they need to be carefully calibrated before using since the measured distance between the robot and the obstacle is relative. A deficient calibration can produce significant errors in the distance estimation, and therefore, the performance of the obstacle avoidance algorithm is poor. If the sensors are included with the mobile robot, the manufacturer usually gives a model to convert raw data into distance, however such model provides an average performance of the distance estimation since this built-in model is not tuned for each sensor. Thus, in order to increase the accuracy of the distance estimation, the traditional method of calibration for a suitable distance model, turns frequently into a manual and time-consuming process [13].
Fusion of different kind of sensors is a fast developing area of research that shows many examples in the literature through the integration of multiple sensors [14,15]. Thus, the integration of the information supplied by US and IR sensors can be a natural extension of previous works in order to provide reliable distance measurements for robot navigation by using these proximity sensors, because the advantages of one sensor compensate for the disadvantages of the other. The fusion of sensors can be implemented by using fuzzy logic [16], neural networks [17][18][19], Kalman filtering [20], or support vector machines among other solutions [14]. Although we can find examples of sensor fusion for robot navigation in the literature, many of the them show only theoretical results [20,21] or quite ad hoc experiments [16][17][18], or they mix the distance model with the collision avoidance algorithm [17,19,22], implying that work on scientific experimental validation of sensor fusion methods for robot navigation still needs further research. This paper presents a novel approach for modelling the distance to obstacles using the data provided by two types of proximity sensors. The proposed approach is applied on the fusion of IR and US sensor of the very popular Khepera IV robot [23]. Our approach also provides an automatic way to obtain and calibrate the model for distance estimation to obstacles by using artificial neural networks. After obtaining an accurate distance to the objects it is possible to use any kind of collision avoidance algorithm to perform a successful navigation of the robot in a dynamic environment. In order to validate the proposed approach, the results section of the paper provides real experiments of obstacle avoidance with a comparison between the traditional calibration method (provided by the manufacturer) versus the developed approach by using the same collision avoidance algorithm.
The remainder of the paper is organized as follows: Section 2 describes the traditional sensors calibration method and its drawbacks; Section 3 presents the novel method of automatic smart calibration of the sensors; Section 4 presents some results and experiments obtained with the developed method in comparison with the traditional calibration method; and finally, Section 5 shows the main conclusions and future work.

Traditional Calibration of Proximity Sensors
The traditional calibration method of proximity sensors is a manual repetitive process. It consists in placing some objects around the robot and measuring the raw values given by the sensors (usually voltages). With this values, a model of the sensors is built using a conventional identification method. The result of this procedure are the coefficients of the model (linear or nonlinear) that calculates the distances to the objects using the voltage of the sensors as inputs. Usually the same model is used for all the sensors of the same type, but in practice, not all the sensors work identically and this can introduce erroneous values.
To carry out the traditional calibration process in the laboratory, two Khepera IV robots [23] are used. Khepera IV is a mobile robot that has been designed for indoor pedagogical purposes and brings numerous features, for example, a colour camera, WI-FI and Bluetooth communications, an array of 8 infrared sensors (IR) for obstacle detection (2 mm to 20 cm) and 5 ultrasonic (US) sensors for long range object detection (20 cm to 40 cm), etc. Figure 1 represents the distribution and the ranges of the proximity sensors in the Khepera IV robot. The IR sensors (red colour) are distributed every 45 • around the robot (form 1 to 8). While the US sensors (green colour) are distributed every 45 • in the front half of the robot (1, 2, 3, 7 and 8). Figure 2 shows the set up configuration in the laboratory to carry out the calibration. To start the process, the robot is placed at the center of a "calibration carpet". Plastic obstacles of different colours conforming a wall concentric to the robot sensors are used to measure their values. The robot stores 20 values per sensor and then the obstacles are displaced 1 cm over its radius. The process is repeated from 1 cm to 40 cm for all the sensors. If the distance is less than 40 cm the process is repeated. Otherwise, the data stored in the robot is sent to the PC to calculate the mean value for each sensor/distance. Then, the values are linearly interpolated. Figure 3 shows the flow diagram with the steps to carry out this process. The blocks represented in green colour are related to the robot. The blocks represented in orange colour are related to the PC. And, the pink colour blocks are related to the manually action taken by the user.    Figure 5 shows the results of the previous process for the 5 Figure 4 shows the results of the previous process for the IR sensors: on the left side, the raw values from the sensors for the Robot 1; and on the right side, the raw values of the Robot 2. As can be seen, the sensors have a nonlinear behaviour for distance values less than 20 cm and the values are different for same sensor/distance (note that all these sensors are of the same type). For distance values greater than 20 cm the behaviour is almost the same. That is why it is important to uses the IR sensors only to measure distances less than 20 cm.  IR1  IR2  IR3  IR4  IR5  IR6  IR7  IR8   0  5  10  15  20  25  30 Figure 5 shows the results of the previous process for the 5 US sensors of Robots 1 and 2. As can be seen, in both cases for distance values less than 20 cm the results are not good. But, for values greater than 20 cm, the results are almost linear. Note that like in the previous case, the values from the sensors in both robots are different, despite the fact that they are the same type of sensor. Following the traditional method the next step is to calculate the model of both type of sensors with the obtained values. Note that the model for each kind of sensor is different. You can obtain a model that combine these two models into one, like in [24].
In the resulting model, for values less than 20 cm the model of the IR sensors is used and for values grater than 20 cm the US model is used. Note that if you need more accuracy, you can obtain a different model for each combination of US and IR on each robot. This is a heavy work because you need to do this process manually in a repetitive way for each robot/sensor if you want to obtain accurate models for all sensors.
As was mentioned before, the results of the manual calibration shows that the information of the sensor must be fused to detect obstacles in the entire range (from 0 cm to 40 cm) in front half of the robot. While in the half back of the robot the obstacles can only be detected from 0 cm to 20 cm. Figure 6 shows the mean value and the standard deviation of the sensors measurements for the Robot 1. On the left side are represented these values for the IR sensors. As can be seen, for short distances (between 2 cm and 15 cm) there are appreciable differences between the raw values of the sensors for the same distance. Which means that the use of the same model for each sensor may cause erroneous measurements in this range. For distances greater than 20 cm the differences are smaller. On the right side of Figure 6, the values of the US sensors are shown. As can be seen, for distances between 20 cm and 26 cm there are appreciable differences between the raw values of the sensors for the same distance. For distances greater than 27 cm the differences are smaller.
Next section presents the main contribution of this work: An automatic smart method to carry out the calibration of the sensors based on data and using an artificial neural network.

Automatic Smart Calibration Method
As was explained before, the traditional calibration is a manual and repetitive process that can be heavy depending of the accuracy that you need to obtain from the model of your sensors. In this section the novel automatic method based on data and using artificial neural networks is proposed.

Platform Used in the Laboratory
The method needs to obtain the absolute position of the robot each step execution. This task is carried out by the platform presented in [25], which is a previous work of the authors. This platform implements an IPS (Indoor Positioning System) to locate the robot. Figure 7 shows the architecture of the platform whose components are the following: (a) A personal computer (PC) with the Ubuntu Linux operating system that executes the software tools (Swistrack [26,27]

Automatic Smart Calibration Method
The method consists of constructing the model of the proximity sensors automatically. For this, some obstacles are placed in known locations in the workspace. Then the robot navigates through the workspace storing the values from the proximity sensors. With the known positions of the obstacles and the information provided by the sensors of the robot, a neural network is built offline based on this data. The resulting model gets the raw values from the sensors as input and provides as output the distance to the obstacles around the robot. Note that this method needs to know the absolute position of the robot during the experiment.
The method consist of the following steps: 1. Set up of the workspace: A square shaped wall (80 cm) is placed in the Arena. The robot is placed in the center of the square with a known orientation. Note the model will work until 40 cm. The robot starts to move implementing the Braitenberg algorithm [28] to avoid the obstacles. Figure 8 shows the aerial view of the set up. 2. Data acquisition: The robot stores the data of the sensors (S 1 , S 2 , ..., S n ) and the time during its movement. On the other hand, the PC stores the position of the robot (coordinates and orientation) and also the time during the experiment. At the beginning of the process the time in the PC and the robot starts in 0. But the sample time is different for both because they run independently. Which means that the acquired data must be synchronized in time by interpolation.
After the experiment finishes, the data of the robot is copied to the PC. Since the time stored in the robot and in the PC are not coincident this data needs to be adjusted and synchronized. This process is carried out of a code developed in MATLAB to do this task. Figure 9 shows an example of the data stored in the robot and in the PC. 3. Data conditioning and synchronization: The data is adjusted by liner interpolation of the time, using Equation (1): where T 0 and T 1 are the initial and final time of the position data where the interpolation is made. t is the time that is evaluated which corresponds with the data of the sensors. X (t), Y (t) and θ (t) are the pose of the robot in time t. Figure 10 shows the results of this process, where the data of the robot sensors and the robot positions are merged. X sn = radius robot * cos (angle sn + θ robot ) + X robot Y sn = radius robot * sin (angle sn + θ robot ) + Y robot (2) Figure 11 represents the distances from the sensors to the walls (yellow colour). The black solid lines are the walls. The blue circle is the robot, the small rectangles are the sensors and the blue arrow represents the orientation of the robot. The dashed lines represent the detection directions of the senors. The dashed circle (40 cm of radius) represents the maximum detection range of the sensors.
The distance from each sensor to the walls is calculated by the intersection points (red) between the detection direction lines and the walls. Equation (3) shows how the coordinates of the intersection point P(X m ;Y m ) are calculated for sensors 1, 2 and 3.
For sensors 7 and 8 is the same process only changing the axis because in this case the intersection is with the wall x = 40. Having these intersection points the distances and the angles are easily calculated by Equation (4). Finally the obtained data is filtered. Figure 11. Distance from the sensors to known obstacles (walls).

Artificial Neural Network (ANN):
With the calculated values and the raw data of the sensors, an ANN is built using the Neural Network Toolbox 9.0 of MATLAB [29], to obtain the model. Figure 12 shows a representation of the ANN implemented based on Levenberg-Marquardt method [30].
Version February 19, 2018 submitted to Sensors 10 of 21 Figure 11. Distance from the sensors to known obstacles (walls) For sensors 7 and 8 is the same process only changing the axis because in this case the intersection is with the wall x=40. Having these intersection points the distances and the angles are easily calculated by equations 4. Finally the obtained data is filtered.

Artificial Neural Network (ANN): With the calculated values and the raw data of the sensors, an
180 ANN is built using the Neural Network Toolbox 9.0 of MATLAB [29], to obtain the model. Figure   181 12 shows a representation of the ANN implemented based on Levenberg-Marquardt method [30].  The training was carried out with the 70% of the acquired data, the validation with another 15% and the testing with the remaining 15%. Note that the parameters of the ANN (number of neurons in the hidden layer and activation functions) have been determined empirically by trial and error after multiple tests. 5. Testing the resulting model: Figure 13 shows a test developed in the workspace where the model was obtained. The sequence of images shows the displacement of the robot through the scenario. The black lines are the walls of the square (80 cm by 80 cm). The red small line represents the position and orientation of the robot. The performance of the models is drawn around the robot: (a) Blue lines: are the real distances to the obstacles (less than 40 cm and calculated with the intersection process described before); (b) Green lines: are the results of the the ANN model, taking and fusing the raw inputs from the sensors and providing the 8 distances; and (c) Red lines: are the distances to the obstacles with the traditional calibration method. Figure 13. Images sequence of a test in the workspace.
As can be seen, all the models have different behaviors for different situations. The better results are obtained in general for distances less than 30 cm. But there are appreciable differences between the results of the half front part of the robot and the back half. This is due to the fact that the back part of the robots does not have US sensors. Even so, these results are more than acceptable compared with the majority of existing obstacle avoidance methods, and based on the maximum velocity of this robot and its steering characteristics. In all the cases, the smallest error values are represented in bold. As can be seen, the results for both robots are different. In the case of Robot 2 the traditional method shows worse results than the ANN method. In the case of Robot 1, the behavior is similar to the previous one. Only distances 1 and 8 present worse results. In general, for both cases the ANN model presents better results than the traditional model. Figure 14 shows the flow diagram of the developed method. The block related to the robot are represented in green colour. The orange colour represents the blocks related to the PC. Note that the ANN model is obtained offline with the acquired data by the robot and by the PC. After that the model is implemented in the robot. The model gets the raw values from the sensors and gives the distances to the objects in the 8 directions where the sensors are located in the robot. the sensors. The columns represent the results for both models (Artificial Neural Network (ANN) 206 and Traditional (Trad)) for robots (Robot 1 and 2  Note that this method has been developed for the Khepera IV robot. The properties of the robot and platform have a direct influence in the performance of the method. But it does not mean that the method cannot be implemented in other robots/platforms that use proximity sensors to estimate the distance to obstacles. To do that, one important element to be taken into account is the use of an indoor positioning system which gives the absolute position of the robot in running time. Other important aspect is the location of the sensors around the robot. This configuration has to be considered when the distances from each sensor to the walls (see Figure 11) are calculated by using Equations (2)-(4).

Experiment of Position Control with Obstacles Avoidance
This experiment is titled position control or point stabilization of a differential wheeled mobile robot. This problem has been widely studied mainly from an automatic control perspective [31]. The objective of the experiment is to drive the robot from the current position C(x c , y c ) and orientation (θ) to the target point T p (x p , y p ). Figure 15 shows a representation of the variables involved in this experiment.

Motors
Position Sensor The block OAA represents the Obstacle Avoidance Algorithm, in this case: Braitenberg algorithm the output velocities of the block OAA are the same as its inputs (ν, ω).

241
The Braitenberg algorithm [28] creates a weighted matrix that converts the sensor inputs into motor speeds. This matrix is a two-dimensional array with the number of columns corresponding to the number of obstacle sensors (8) and the number of rows corresponding to the number of motors (2). The weights of the matrix are determined empirically depending on the location of the sensors in the robot.The 8 sensors of the Khepera IV robot are numbered clockwise beginning with the front sensor. Equation 7 represents the mentioned matrix where, for example, the term W LS1 represents the weight of the sensor S 1 in the speed of the left motor. Equation 8 represents the raw data of the proximity sensors at each time. The block Compare calculates the distance (d) and the angle α to the target point (T p ) from the current position of the robot (C). The block Control Law tries to minimize the orientation error, θ e = α − θ, and at the same time, to reduce the distance to the target point (d = 0) by manipulating the control signals (linear velocity (ν) and angular velocity (ω) of the robot). Equations (5) and (6) show the implementation of this control law based on [32].
The block OAA represents the Obstacle Avoidance Algorithm, in this case: Braitenberg algorithm [28], in which the robot's sensors are tied directly to the motor controls and motor speeds respond to the sensor input directly. The Model block represents the developed algorithm which obtains the distances to the obstacles (D 1 -D 8 ) from the raw data of proximity sensors (S 1 -S 13 ). If no obstacle is detected, the output velocities of the block OAA are the same as its inputs (ν, ω).
The Braitenberg algorithm [28] creates a weighted matrix that converts the sensor inputs into motor speeds. This matrix is a two-dimensional array with the number of columns corresponding to the number of obstacle sensors (8) and the number of rows corresponding to the number of motors (2). The weights of the matrix are determined empirically depending on the location of the sensors in the robot.The 8 sensors of the Khepera IV robot are numbered clockwise beginning with the front sensor. Equation (7) represents the mentioned matrix where, for example, the term W LS 1 represents the weight of the sensor S 1 in the speed of the left motor. Equation (8) represents the raw data of the proximity sensors at each time.
With these matrices, the velocities for each motor are calculated as is shown in Equation (9). Where (S max ) represents the maximum value of the sensor output.   Figure 18 shows the results of both experiments with the ANN and Traditional models. The red lines and circles are the obstacles. In both cases, the robot begins the experiment starting at point 1 (cyan circle) and it must reach the target point 2 (green circle). The small blue points represent the position of the robot during the experiment for the traditional calibration model (Trad), while the small red points represent the position of the robot for the developed method (ANN).
In both cases the robot reaches the target point avoiding the obstacles that it finds on its way. As can be seen, the behavior of the robot with the ANN model has better performance than with the traditional model. These differences show that you can use the functions provided by the manufacturer or calibrate the sensors manually using the traditional method. In both cases you need to take into account that your obstacle avoidance algorithm can be working correctly for the values of distances that it is receiving. However, the real problem is that the sensors can be providing erroneous values due to bad calibration or the use of a poorly tuned model for all the sensors.  Table 2 shows the Mean Absolute Percentage Error (MAPE %) of these experiments presented in Figure 18. The columns represent the following: (Exp) the number of the experiment, (Method) the model, (MAPE) the mean values of the distance errors (from D 1 to D 8 ) and (Time) the time to reach the target point. The results represented in bold indicate the smallest values of the errors and time. As can be seen, for all experiments the ANN model presents less mean error than the traditional model. In addition, in both experiments the robot reaches the target point in less time, which means that the robot describes a smoother and more direct path to the target point due to better detection and avoidance of the obstacles.
Additionally, Figure 19 shows the angular velocity of the robot in the second experiment (right side of Figure 17) for both models: blue line represents the traditional model; and red line represents the ANN model. As can be seen, for the same experiment, with the ANN model the angular velocity has less oscillations than with the traditional model. Due to better detection and avoidance of the obstacles, the behavior of the robot is smoother because it can avoid the obstacles in a better way with less direction changes. Because it reaches the target point first, that means that the trajectory is more efficient. Figure 20 shows the linear velocities for the same experiments and models. In the case of the linear velocities, also the ANN model has less oscillations and it becomes zero first. This means that the robot reaches the target point faster than with the traditional model and with a more constant velocity.

Conclusions
The most common proximity sensors in mobile robotics are the US and IR sensors. They are widely used for obstacle detection without physical contact. But they introduce uncertainties in the measurements due to their mode of operation: they need a feedback signal that depends on the surface of the object and the ambient conditions. Another important issue for these sensors is the calibration of their outcomes. Usually one has to obtain a model that converts the raw measurements provided by the sensors in distances. The method for building this model is a repetitive and lengthy process that usually is done manually. Moreover, the obtained model cannot provide the same distance even for two sensors of the same type (due to the uncertainties introduced by the surfaces of the objects). That is why in some situations it could be useful to gather all measurements provided by different sensors and combine them in order to get an accurate model to estimate the distances to the obstacles around the robot.
This paper presents a novel approach to get an obstacle detection model based on the combination of sensor data. The method is automatic and based on data using machine learning. The method implements an artificial neural network to obtain the model of the sensors. The model receives the raw values from the 13 sensors (8 IR + 5 US) and provides the distances to the obstacles around the robot (8 values). This means that the method combines the information from both kinds of sensors and builds a unique model for obstacle distances.
The developed method has been designed for the Khepera IV robot in an indoor environment with an IPS to obtain its absolute position in running time. Although it has been designed for these characteristics, this method can be implemented to different platforms. The fundamental points that have to be taken into account are the indoor positioning system and the location of the sensors around the robot.
To show the performance of the developed model, some results are provided and discussed. The experiment of position control with obstacles avoidance based on the Braitenberg algorithm was selected. To test it, two different scenarios were used for this experiment. In addition, the velocity of the robots was analyzed in both cases. The comparison between the traditional and the developed method shows the improvement in the results.
This method is undoubtedly an advance in the calibration of proximity sensors because it eliminates the traditional laborious method and provides better results. Future works are related to the implementation of this method in other platforms with other kinds of robots and sensors.