A Doorway Detection and Direction (3Ds) System for Social Robots via a Monocular Camera

In this paper, we propose a novel algorithm to detect a door and its orientation in indoor settings from the view of a social robot equipped with only a monocular camera. The challenge is to achieve this goal with only a 2D image from a monocular camera. The proposed system is designed through the integration of several modules, each of which serves a special purpose. The detection of the door is addressed by training a convolutional neural network (CNN) model on a new dataset for Social Robot Indoor Navigation (SRIN). The direction of the door (from the robot’s observation) is achieved by three other modules: Depth module, Pixel-Selection module, and Pixel2Angle module, respectively. We include simulation results and real-time experiments to demonstrate the performance of the algorithm. The outcome of this study could be beneficial in any robotic navigation system for indoor environments.


Introduction
Navigating in indoor environments inevitably requires detection and crossing doors that are regarded as integral parts of any indoor setting, particularly in human habitats (homes). Whereas this task does not require much effort for humans and even their pets, it is a challenge for social and other autonomous robots. As such, it is desired that social robots have the same skill and are able to move around a house seamlessly and know their own whereabouts based on an ability to classify each room and its functionality [1,2]. Indoor navigation is inherently multifaceted and includes several tasks including but not limited to localization, mapping, Simultaneous Localization and Mapping (SLAM), path planning, object, and scene recognition. However, the capacity to detect doors and their orientation are critical for any navigation system and are the main subject of this paper; though the related problem of passing through a door is not within the scope of this study. This research question has attracted attention by many researchers on robotics and as we shall discuss in Section 2, the detection and navigation through a doorway are mostly addressed via sensor fusion techniques, deployment of rather expensive built-in sensor(s) on-board the robot, or augmenting the environment by appropriate and dedicated sensors or Quick Response (QR) Codes.
The motivation for this research is the following question: can this problem also be solved practically via a monocular camera? Therefore, the objective of this project is to design a system just for detecting and directing a social robot towards a doorway using only a monocular camera that captures only a 2D image. The proposed system is one of the components of an end-to-end navigation strategy inspired by Behavioristic Robotics, in particular the ubiquitous Brook's Subsumption architecture [3] for social robots with limited sensors. This methodology is based on the Sense-Perception-Act theme calculate the relative angle direction of the robot with respect to the doorway from a 2D image. The angle direction is important information for controlling the robot towards the target. Therefore, a global or explicit Cartesian position, as well as distance information towards the doorway are excluded; although they might provide crucial cues for other robotic applications. Additionally, our study focuses on the discrete decision of the sense-perception-act theme, which is unlike other visual servo techniques, such as [18], that address the navigation problem continuously with the integration of a conventional controller. The third goal is to compute the angle direction from only a still 2D image via a monocular camera, which can be inferred through estimating the depth information. Therefore, we adopted the model from [24], for estimating the depth values from a 2D still image with no need for additional image preprocessing, over other computer vision methods, such as machine learning methods that depend on careful engineered designs, and SfM that needs a sequence of 2D images. Besides, the work in [24] is considered as one of the state-of-the-art studies in the field of estimating depth information from 2D images as the author presented the success of their network compared with other studies in the research area. Accordingly, we propose a Doorway Detection and Direction 3Ds-system for social robots with limited sensors (monocular camera). This system can detect an open door and then can direct the robot toward the doorway based only on a 2D image that is captured by a monocular camera. The system combines several modules with different approaches: learning-based, pixel-based, and triangular-based methods.

Proposed System and Methodology
The key concept of the proposed system is based on the Sense-Perception-Action architecture (see Figure 1). Accordingly, the proposed 3Ds-system for detecting a doorway and directing a social robot through it is shown in Figure 2. It consists of several modules to enable a social robot equipped with only a monocular camera, i.e., Nao robot, to provide an appropriate angle toward the doorway from its current location. The algorithm is initiated by acquiring a 2D image using the top camera of Nao. This image is then passed to the CNN-SRIN Doorway module to classify the image as either an open door or a no-door scene. SRIN is a dataset for indoor settings specifically designed for short robots such as Nao. If the image is classified as an open-door scene, the depth module is triggered to construct a depth map using the captured 2D image. Next, the Pixel-Selection module is applied to the depth map to determine the best pixel that represents the doorway location. Finally, the selected pixel is passed to the Pixel2Angle module that converts the depth value of that pixel into an appropriate angle which will be used to guide the robot towards the door. The Pixel2Angle module is triggered only if there is no obstacle between the robot and the doorway, which can be detected via a vertical correlation in the Pixel-Selection module.  The following sub-sections will explain the function of each module in more detail. As the algorithm is meant for social robots, we present these modules for the Nao humanoid robot. The same algorithm can be readily applied to any (social) robot equipped only with a monocular camera.

2D Image from Nao Monocular Camera
The Nao humanoid robot has two monocular cameras that are mounted vertically on its face. Since there is no overlap between them, the system is not considered as a stereo camera set, i.e., there is no direct depth information or direct way to extract the depth values. For this project, we employ the top camera to extract a 2D image, which is set up with a size of 640 × 480. The specifications of the camera are crucial for achieving the purpose of this project successfully, specifically the horizontal field of view, = 60.9°, and the vertical field of view, ℎ = 47.6° as shown in Figure 3. As our goal is to control the direction of the Nao robot, then the will be used in the calculation of the Pixel2Angle module.

CNN-SRIN Doorway Module
The aim of this module is to detect whether or not the scene in front of the robot is a door. We achieve this goal by training a CNN model via the transfer learning process as shown in Figure 4 using our collected SRIN dataset (all samples and python codes can be downloaded from the author's GitHub page via this link: https://github.com/KamalOthman/SRIN-Dataset.git) [2]. Thus, we call this model CNN-SRIN throughout the paper. A detailed discussion of the CNN-SRIN architecture is further explained in Section 4.1. There are two classes of SRIN dataset for doorway used for training CNN model: no-door and open-door, in which this module will be useful for any indoor robotic The following sub-sections will explain the function of each module in more detail. As the algorithm is meant for social robots, we present these modules for the Nao humanoid robot. The same algorithm can be readily applied to any (social) robot equipped only with a monocular camera.

2D Image from Nao Monocular Camera
The Nao humanoid robot has two monocular cameras that are mounted vertically on its face. Since there is no overlap between them, the system is not considered as a stereo camera set, i.e., there is no direct depth information or direct way to extract the depth values. For this project, we employ the top camera to extract a 2D image, which is set up with a size of 640 × 480. The specifications of the camera are crucial for achieving the purpose of this project successfully, specifically the horizontal field of view, FoV w = 60.9 • , and the vertical field of view, FoV h = 47.6 • as shown in Figure 3. As our goal is to control the direction of the Nao robot, then the FoV w will be used in the calculation of the Pixel2Angle module.  The following sub-sections will explain the function of each module in more detail. As the algorithm is meant for social robots, we present these modules for the Nao humanoid robot. The same algorithm can be readily applied to any (social) robot equipped only with a monocular camera.

2D Image from Nao Monocular Camera
The Nao humanoid robot has two monocular cameras that are mounted vertically on its face. Since there is no overlap between them, the system is not considered as a stereo camera set, i.e., there is no direct depth information or direct way to extract the depth values. For this project, we employ the top camera to extract a 2D image, which is set up with a size of 640 × 480. The specifications of the camera are crucial for achieving the purpose of this project successfully, specifically the horizontal field of view, = 60.9°, and the vertical field of view, ℎ = 47.6° as shown in Figure 3. As our goal is to control the direction of the Nao robot, then the will be used in the calculation of the Pixel2Angle module.

CNN-SRIN Doorway Module
The aim of this module is to detect whether or not the scene in front of the robot is a door. We achieve this goal by training a CNN model via the transfer learning process as shown in Figure 4 using our collected SRIN dataset (all samples and python codes can be downloaded from the author's GitHub page via this link: https://github.com/KamalOthman/SRIN-Dataset.git) [2]. Thus, we call this model CNN-SRIN throughout the paper. A detailed discussion of the CNN-SRIN architecture is further explained in Section 4.1. There are two classes of SRIN dataset for doorway used for training CNN model: no-door and open-door, in which this module will be useful for any indoor robotic

CNN-SRIN Doorway Module
The aim of this module is to detect whether or not the scene in front of the robot is a door. We achieve this goal by training a CNN model via the transfer learning process as shown in Figure 4 using our collected SRIN dataset (all samples and python codes can be downloaded from the author's GitHub page via this link: https://github.com/KamalOthman/SRIN-Dataset.git) [2]. Thus, we call this model CNN-SRIN throughout the paper. A detailed discussion of the CNN-SRIN architecture is further explained in Section 4.1. There are two classes of SRIN dataset for doorway used for training CNN model: no-door and open-door, in which this module will be useful for any indoor robotic visual navigation system. Within the proposed 3Ds system, the following module will be triggered if the robot detects an open-door with CNN-SRIN.
Sensors 2020, 19, x FOR PEER REVIEW 6 of 17 visual navigation system. Within the proposed 3Ds system, the following module will be triggered if the robot detects an open-door with CNN-SRIN.

Depth Module
The objective of this module is to estimate a depth map from a 2D image extracted from Nao's monocular camera. Estimating depth information from 2D-colored images is among open research problems in computer vision. We adopted the trained Depth Dense network from [24], which is considered as state-of-the-art in this area. The Depth Dense network is designed based on the encoderdecoder architecture as shown in Figure 5. The encoder part is a pre-trained CNN architecture, specifically DenseNet 169, which has layers for extracting features through the down-sampling process. The decoder part has layers for constructing the estimated depth information through the up-sampling process. Every layer in the decoder was fed by the output of a specific layer in the encoder, i.e., this concept is referred to as skip connection. The network was trained while keeping the encoder part frozen, i.e., transfer learning process, using two different RGB-D datasets: NYU Depth-v2 [30] and KITTI [31]. Both datasets provide RGB images as inputs, whereas the depth map is the respective output. The authors [24] presented the success of their network compared with other work in the field of estimating depth information from 2D images. For that reason, this trained model was adopted to test and estimate 2D images from Nao within a robotic application. The Nao 2D image is fed to the Depth Dense Network in size of 640 × 480, where the network will estimate the depth information of size 320 × 240 . All depth map pixels carry a value from 0 to 1, in which the value 1 is the deepest distance.

Depth Module
The objective of this module is to estimate a depth map from a 2D image extracted from Nao's monocular camera. Estimating depth information from 2D-colored images is among open research problems in computer vision. We adopted the trained Depth Dense network from [24], which is considered as state-of-the-art in this area. The Depth Dense network is designed based on the encoder-decoder architecture as shown in Figure 5. The encoder part is a pre-trained CNN architecture, specifically DenseNet 169, which has layers for extracting features through the down-sampling process. The decoder part has layers for constructing the estimated depth information through the up-sampling process. Every layer in the decoder was fed by the output of a specific layer in the encoder, i.e., this concept is referred to as skip connection. The network was trained while keeping the encoder part frozen, i.e., transfer learning process, using two different RGB-D datasets: NYU Depth-v2 [30] and KITTI [31]. Both datasets provide RGB images as inputs, whereas the depth map is the respective output. The authors [24] presented the success of their network compared with other work in the field of estimating depth information from 2D images. For that reason, this trained model was adopted to test and estimate 2D images from Nao within a robotic application. The Nao 2D image is fed to the Depth Dense Network in size of 640 × 480, where the network will estimate the depth information of size 320 × 240. All depth map pixels carry a value from 0 to 1, in which the value 1 is the deepest distance.

Depth Module
The objective of this module is to estimate a depth map from a 2D image extracted from Nao's monocular camera. Estimating depth information from 2D-colored images is among open research problems in computer vision. We adopted the trained Depth Dense network from [24], which is considered as state-of-the-art in this area. The Depth Dense network is designed based on the encoderdecoder architecture as shown in Figure 5. The encoder part is a pre-trained CNN architecture, specifically DenseNet 169, which has layers for extracting features through the down-sampling process. The decoder part has layers for constructing the estimated depth information through the up-sampling process. Every layer in the decoder was fed by the output of a specific layer in the encoder, i.e., this concept is referred to as skip connection. The network was trained while keeping the encoder part frozen, i.e., transfer learning process, using two different RGB-D datasets: NYU Depth-v2 [30] and KITTI [31]. Both datasets provide RGB images as inputs, whereas the depth map is the respective output. The authors [24] presented the success of their network compared with other work in the field of estimating depth information from 2D images. For that reason, this trained model was adopted to test and estimate 2D images from Nao within a robotic application. The Nao 2D image is fed to the Depth Dense Network in size of 640 × 480, where the network will estimate the depth information of size 320 × 240 . All depth map pixels carry a value from 0 to 1, in which the value 1 is the deepest distance.

Pixel-Selection Module
This module is designed with a premise that the pixel with the deepest value is associated with the doorway. Therefore, the simple way to select a pixel related to the doorway is the maximum depth value from the depth map; let us call them Max-Pixel and Max-Depth. However, the Max-Pixel is not the best choice for the robot direction as it might be very close to the edge of the door, or it might be close to the top corners of the room. For this reason, we need to find the Best-Pixel for the robot direction based on the horizontal correlation in the lower half of the image. This can be obtained by comparing every two adjacent pixels starting from the Max-Pixel in both directions, i.e., right by incrementing the width and left by decrementing the width. If the difference of the depth values is less than a threshold = 0.01 unit, then we move to the next pixel that is next to the current pixel. The proposed algorithm keeps comparing every two adjacent pixels from the right and left until the difference of depth is greater than a threshold value from both directions. This implies that the most likely pixel related to the edges is the door. Then, the Best-Pixel is the mid pixel between the last right and left correlated pixels. This Best-Pixel will be passed on to the Pixel2Angle module.
In many cases (for instance, image 2 of Table 3), the robot can detect a door while there is an obstacle between the robot and the door. Therefore, we need to find a Trigger-Pixel to make sure that there is no obstacle in the way to the door, and then to trigger the next module. This can be performed by applying the idea of pixel correlation vertically to the depth map through the bottom direction only, i.e., incrementing the height value from the Max-Pixel, with a threshold = 0.045 unit. If the height of the last correlated bottom pixel is over 200, then this will trigger the next module to find the proper angle. Otherwise, it is implied that there is an obstacle in the way towards the door. In that scenario, there is no need to calculate the angle in the Pixel2Angle module. Figure 6 illustrates the concept of pixel correlation and selection from the 2D depth map. depth value from the depth map; let us call them Max-Pixel and Max-Depth. However, the Max-Pixel is not the best choice for the robot direction as it might be very close to the edge of the door, or it might be close to the top corners of the room. For this reason, we need to find the Best-Pixel for the robot direction based on the horizontal correlation in the lower half of the image. This can be obtained by comparing every two adjacent pixels starting from the Max-Pixel in both directions, i.e., right by incrementing the width and left by decrementing the width. If the difference of the depth values is less than a ℎ ℎ = 0.01 , then we move to the next pixel that is next to the current pixel. The proposed algorithm keeps comparing every two adjacent pixels from the right and left until the difference of depth is greater than a threshold value from both directions. This implies that the most likely pixel related to the edges is the door. Then, the Best-Pixel is the mid pixel between the last right and left correlated pixels. This Best-Pixel will be passed on to the Pixel2Angle module.
In many cases (for instance, image 2 of Table 3), the robot can detect a door while there is an obstacle between the robot and the door. Therefore, we need to find a Trigger-Pixel to make sure that there is no obstacle in the way to the door, and then to trigger the next module. This can be performed by applying the idea of pixel correlation vertically to the depth map through the bottom direction only, i.e., incrementing the height value from the Max-Pixel, with a ℎ ℎ = 0.045 . If the height of the last correlated bottom pixel is over 200, then this will trigger the next module to find the proper angle. Otherwise, it is implied that there is an obstacle in the way towards the door. In that scenario, there is no need to calculate the angle in the Pixel2Angle module. Figure 6 illustrates the concept of pixel correlation and selection from the 2D depth map.

Pixel2Angle Module
After selecting the Best-Pixel toward the doorway and making sure that the door's pixel is located on the trigger area of the depth map, i.e., no obstacle in the way to the door, then we apply the Pixel2Angle module for calculating the proper and approximate angle direction toward the door. It is a simple, but effective triangular algorithm applied to the selected pixel. As our goal is that Nao turns left (+ ) or right (− ), then the calculation will be focusing on the horizontal pixel values, although the vertical calculation can be processed similarly for other applications. Figure 7 presents an idea of how this module works and how the target's angle is calculated. As illustrated in Figure 7a, the robot center view is represented as the center pixel in the depth image, the depth value of the selected pixel is the perpendicular distance between the target and the robot location. Therefore, the real horizontal distance X between the robot and the target is represented as the number of pixels from

Pixel2Angle Module
After selecting the Best-Pixel toward the doorway and making sure that the door's pixel is located on the trigger area of the depth map, i.e., no obstacle in the way to the door, then we apply the Pixel2Angle module for calculating the proper and approximate angle direction toward the door. It is a simple, but effective triangular algorithm applied to the selected pixel. As our goal is that Nao turns left (+θ) or right (−θ), then the calculation will be focusing on the horizontal pixel values, although the vertical calculation can be processed similarly for other applications. Figure 7 presents an idea of how this module works and how the target's angle is calculated. As illustrated in Figure 7a, the robot center view is represented as the center pixel in the depth image, the depth value of the selected pixel is the perpendicular distance between the target and the robot location. Therefore, the real horizontal distance X between the robot and the target is represented as the number of pixels from the selected pixel Pixel best and the center pixel Pixel center in the depth map. First, we need to find the horizontal length x between the selected pixel Pixel best and the center pixel Pixel center from the depth map. Then, we need to calculate the angular size of each pixel α pixel in the depth map by dividing the field of view (FoV) by the size of the depth image. The horizontal field of view of Nao is FoV w = 60.9 • , whereas the width of the depth map from the Depth module is 320 pixels. Thus, each pixel in the depth image has 0.19 • angular size. After that, it is easy to calculate the desired angle θ w between Nao and the target direction toward the door by multiplying the angular size α pixel by the horizontal length x. This angle will be passed to Nao as a negative value if the Pixel best is in the right half of the depth map; otherwise, it is positive. For other applications that deal with distances, if the unit of the depth map is known, e.g., depth in meters, then it is worth calculating the distance to the target, i.e., the door in our application. First, we calculate the real horizontal distance X by multiplying the Tangent of the desired angle θ h by the depth value Z. Then, we can find the distance using the Pythagorean equation. Figure 7b gives the mathematical algorithm of this module.
dividing the field of view (FoV) by the size of the depth image. The horizontal field of view of Nao is = 60.9°, whereas the width of the depth map from the Depth module is 320 pixels. Thus, each pixel in the depth image has 0.19° angular size. After that, it is easy to calculate the desired angle between Nao and the target direction toward the door by multiplying the angular size by the horizontal length . This angle will be passed to Nao as a negative value if the is in the right half of the depth map; otherwise, it is positive. For other applications that deal with distances, if the unit of the depth map is known, e.g., depth in meters, then it is worth calculating the distance to the target, i.e., the door in our application. First, we calculate the real horizontal distance X by multiplying the Tangent of the desired angle ℎ by the depth value Z. Then, we can find the distance using the Pythagorean equation. Figure 7b gives the mathematical algorithm of this module.

Experiments and Results
The experiments and results of all modules of this project are presented into two stages: the doorway detection stage and the angle extraction based on depth and pixel selection. In the first stage, the system detects the door via CNN-SRIN [1]. The second stage presents the results of other modules on some selected images from the first stage. Afterward, we present real experiments with a Nao robot in a new environment in order to validate the overall performance of the 3Ds-system.

Stage 1: CNN-SRIN for Doorway Detection
The design of CNN-SRIN architecture consists of a features extractor via VGG16 and an image classifier via fully connected (FC) network using Keras API [32]. In this project, the first stage of transfer learning concept that was shown in Figure 4 was only applied to the CNN-SRIN architecture, for which VGG16 was frozen while FC was trainable. FC began with an average pooling layer, then a layer of 1024 neurons with the rectified linear unit (ReLU) activation function. The model was terminated with a logistic layer to predict one of the two classes: no-door vs. open-door. In this stage, the learning rate was 0.001 and Adam optimizer ran for 10 epochs. The no-door class consisted of 7062 images, whereas the open-door class included 7432 images. We trained the CNN-SRIN model for doorway detection on the Graham cluster provided by Compute Canada Database [33] for several epochs, 10, 20, and 30, respectively. The validation accuracy reached 95.79% after 36 min for the 10 epochs. Whereas it increased up to 97.96% after 1:10 h of training for 20 epochs, and 97.51% after 1:46 h of training for 30 epochs. Accordingly, the trained model with 20 epochs was adopted to be tested on

Experiments and Results
The experiments and results of all modules of this project are presented into two stages: the doorway detection stage and the angle extraction based on depth and pixel selection. In the first stage, the system detects the door via CNN-SRIN [1]. The second stage presents the results of other modules on some selected images from the first stage. Afterward, we present real experiments with a Nao robot in a new environment in order to validate the overall performance of the 3Ds-system.

Stage 1: CNN-SRIN for Doorway Detection
The design of CNN-SRIN architecture consists of a features extractor via VGG16 and an image classifier via fully connected (FC) network using Keras API [32]. In this project, the first stage of transfer learning concept that was shown in Figure 4 was only applied to the CNN-SRIN architecture, for which VGG16 was frozen while FC was trainable. FC began with an average pooling layer, then a layer of 1024 neurons with the rectified linear unit (ReLU) activation function. The model was terminated with a logistic layer to predict one of the two classes: no-door vs. open-door. In this stage, the learning rate was 0.001 and Adam optimizer ran for 10 epochs. The no-door class consisted of 7062 images, whereas the open-door class included 7432 images. We trained the CNN-SRIN model for doorway detection on the Graham cluster provided by Compute Canada Database [33] for several epochs, 10, 20, and 30, respectively. The validation accuracy reached 95.79% after 36 min for the 10 epochs. Whereas it increased up to 97.96% after 1:10 h of training for 20 epochs, and 97.51% after 1:46 h of training for 30 epochs. Accordingly, the trained model with 20 epochs was adopted to be tested on new images collected by Nao humanoid robot (all captured images by Nao can be found in this link: https://github.com/KamalOthman/SRIN-Dataset.git) since the model has the highest validation accuracy within a reasonable period of time on the Graham cluster. We randomly selected 12 related images, i.e., six images for each class. Table 1 shows all images with their predictions. The model successfully predicted five images out of six with the correct class for each category, i.e., a total of 10 correct predictions as shown in Table 2. These results validated that this module within the 3Ds-system will be a good trigger for the next module.

No-Door
Open-Door Nao image CNN-SRIN Prediction Nao images CNN-SRIN Prediction accuracy within a reasonable period of time on the Graham cluster. We randomly selected 12 related images, i.e., six images for each class. Table 1 shows all images with their predictions. The model successfully predicted five images out of six with the correct class for each category, i.e., a total of 10 correct predictions as shown in Table 2. These results validated that this module within the 3Ds-system will be a good trigger for the next module.

Stage 2: Angle Extraction from 2D Images Based on Depth Map and Pixel Selection
The next modules of the 3Ds-system were tested on several real-time images from the previous module in order to get a practical proof of the successful performance. The expected outputs of the angles are in the range of [−30°, 30°] as the Nao's horizontal = 60.9°. We selected the six open-door images of Nao robot as well as the image of no-door with the false-positive prediction. All results of these modules are presented in Table 3. This table presents every Nao's 2D image with its CNN-SRIN trigger status. If the status is "Yes", then the rest of the other modules' results are presented. The depth module provides a depth colored image, in which the yellowish pixels are considered as far No-door accuracy within a reasonable period of time on the Graham cluster. We randomly selected 12 related images, i.e., six images for each class. Table 1 shows all images with their predictions. The model successfully predicted five images out of six with the correct class for each category, i.e., a total of 10 correct predictions as shown in Table 2. These results validated that this module within the 3Ds-system will be a good trigger for the next module.

Stage 2: Angle Extraction from 2D Images Based on Depth Map and Pixel Selection
The next modules of the 3Ds-system were tested on several real-time images from the previous module in order to get a practical proof of the successful performance. The expected outputs of the angles are in the range of [−30°, 30°] as the Nao's horizontal = 60.9°. We selected the six open-door images of Nao robot as well as the image of no-door with the false-positive prediction. All results of these modules are presented in Table 3. This table presents every Nao's 2D image with its CNN-SRIN trigger status. If the status is "Yes", then the rest of the other modules' results are presented. The depth module provides a depth colored image, in which the yellowish pixels are considered as far Open-door accuracy within a reasonable period of time on the Graham cluster. We randomly selected 12 related images, i.e., six images for each class. Table 1 shows all images with their predictions. The model successfully predicted five images out of six with the correct class for each category, i.e., a total of 10 correct predictions as shown in Table 2. These results validated that this module within the 3Ds-system will be a good trigger for the next module.

Stage 2: Angle Extraction from 2D Images Based on Depth Map and Pixel Selection
The next modules of the 3Ds-system were tested on several real-time images from the previous module in order to get a practical proof of the successful performance. The expected outputs of the angles are in the range of [−30°, 30°] as the Nao's horizontal = 60.9°. We selected the six open-door images of Nao robot as well as the image of no-door with the false-positive prediction. All results of these modules are presented in Table 3. This table presents every Nao's 2D image with its CNN-SRIN trigger status. If the status is "Yes", then the rest of the other modules' results are presented. The depth module provides a depth colored image, in which the yellowish pixels are considered as far Open-door (false) accuracy within a reasonable period of time on the Graham cluster. We randomly selected 12 related images, i.e., six images for each class. Table 1 shows all images with their predictions. The model successfully predicted five images out of six with the correct class for each category, i.e., a total of 10 correct predictions as shown in Table 2. These results validated that this module within the 3Ds-system will be a good trigger for the next module.

Stage 2: Angle Extraction from 2D Images Based on Depth Map and Pixel Selection
The next modules of the 3Ds-system were tested on several real-time images from the previous module in order to get a practical proof of the successful performance. The expected outputs of the angles are in the range of [−30°, 30°] as the Nao's horizontal = 60.9°. We selected the six open-door images of Nao robot as well as the image of no-door with the false-positive prediction. All results of these modules are presented in Table 3. This table presents every Nao's 2D image with its CNN-SRIN trigger status. If the status is "Yes", then the rest of the other modules' results are presented. The depth module provides a depth colored image, in which the yellowish pixels are considered as far accuracy within a reasonable period of time on the Graham cluster. We randomly selected 12 related images, i.e., six images for each class. Table 1 shows all images with their predictions. The model successfully predicted five images out of six with the correct class for each category, i.e., a total of 10 correct predictions as shown in Table 2. These results validated that this module within the 3Ds-system will be a good trigger for the next module.

Stage 2: Angle Extraction from 2D Images Based on Depth Map and Pixel Selection
The next modules of the 3Ds-system were tested on several real-time images from the previous module in order to get a practical proof of the successful performance. The expected outputs of the angles are in the range of [−30°, 30°] as the Nao's horizontal = 60.9°. We selected the six open-door images of Nao robot as well as the image of no-door with the false-positive prediction. All results of these modules are presented in Table 3. This table presents every Nao's 2D image with its CNN-SRIN trigger status. If the status is "Yes", then the rest of the other modules' results are presented. The depth module provides a depth colored image, in which the yellowish pixels are considered as far No-door accuracy within a reasonable period of time on the Graham cluster. We randomly selected 12 related images, i.e., six images for each class. Table 1 shows all images with their predictions. The model successfully predicted five images out of six with the correct class for each category, i.e., a total of 10 correct predictions as shown in Table 2. These results validated that this module within the 3Ds-system will be a good trigger for the next module.

Stage 2: Angle Extraction from 2D Images Based on Depth Map and Pixel Selection
The next modules of the 3Ds-system were tested on several real-time images from the previous module in order to get a practical proof of the successful performance. The expected outputs of the angles are in the range of [−30°, 30°] as the Nao's horizontal = 60.9°. We selected the six open-door images of Nao robot as well as the image of no-door with the false-positive prediction. All results of these modules are presented in Table 3. This table presents every Nao's 2D image with its CNN-SRIN trigger status. If the status is "Yes", then the rest of the other modules' results are presented. The depth module provides a depth colored image, in which the yellowish pixels are considered as far Open-door accuracy within a reasonable period of time on the Graham cluster. We randomly selected 12 related images, i.e., six images for each class. Table 1 shows all images with their predictions. The model successfully predicted five images out of six with the correct class for each category, i.e., a total of 10 correct predictions as shown in Table 2. These results validated that this module within the 3Ds-system will be a good trigger for the next module.

Stage 2: Angle Extraction from 2D Images Based on Depth Map and Pixel Selection
The next modules of the 3Ds-system were tested on several real-time images from the previous module in order to get a practical proof of the successful performance. The expected outputs of the angles are in the range of [−30°, 30°] as the Nao's horizontal = 60.9°. We selected the six open-door images of Nao robot as well as the image of no-door with the false-positive prediction. All results of these modules are presented in Table 3. This table presents every Nao's 2D image with its CNN-SRIN trigger status. If the status is "Yes", then the rest of the other modules' results are presented. The depth module provides a depth colored image, in which the yellowish pixels are considered as far No-door accuracy within a reasonable period of time on the Graham cluster. We randomly selected 12 related images, i.e., six images for each class. Table 1 shows all images with their predictions. The model successfully predicted five images out of six with the correct class for each category, i.e., a total of 10 correct predictions as shown in Table 2. These results validated that this module within the 3Ds-system will be a good trigger for the next module.

Stage 2: Angle Extraction from 2D Images Based on Depth Map and Pixel Selection
The next modules of the 3Ds-system were tested on several real-time images from the previous module in order to get a practical proof of the successful performance. The expected outputs of the angles are in the range of [−30°, 30°] as the Nao's horizontal = 60.9°. We selected the six open-door images of Nao robot as well as the image of no-door with the false-positive prediction. All results of these modules are presented in Table 3. This table presents every Nao's 2D image with its CNN-SRIN trigger status. If the status is "Yes", then the rest of the other modules' results are presented. The depth module provides a depth colored image, in which the yellowish pixels are considered as far Open-door accuracy within a reasonable period of time on the Graham cluster. We randomly selected 12 related images, i.e., six images for each class. Table 1 shows all images with their predictions. The model successfully predicted five images out of six with the correct class for each category, i.e., a total of 10 correct predictions as shown in Table 2. These results validated that this module within the 3Ds-system will be a good trigger for the next module.

Stage 2: Angle Extraction from 2D Images Based on Depth Map and Pixel Selection
The next modules of the 3Ds-system were tested on several real-time images from the previous module in order to get a practical proof of the successful performance. The expected outputs of the angles are in the range of [−30°, 30°] as the Nao's horizontal = 60.9°. We selected the six open-door images of Nao robot as well as the image of no-door with the false-positive prediction. All results of these modules are presented in Table 3. This table presents every Nao's 2D image with its CNN-SRIN trigger status. If the status is "Yes", then the rest of the other modules' results are presented. The depth module provides a depth colored image, in which the yellowish pixels are considered as far No-door accuracy within a reasonable period of time on the Graham cluster. We randomly selected 12 related images, i.e., six images for each class. Table 1 shows all images with their predictions. The model successfully predicted five images out of six with the correct class for each category, i.e., a total of 10 correct predictions as shown in Table 2. These results validated that this module within the 3Ds-system will be a good trigger for the next module.

Stage 2: Angle Extraction from 2D Images Based on Depth Map and Pixel Selection
The next modules of the 3Ds-system were tested on several real-time images from the previous module in order to get a practical proof of the successful performance. The expected outputs of the angles are in the range of [−30°, 30°] as the Nao's horizontal = 60.9°. We selected the six open-door images of Nao robot as well as the image of no-door with the false-positive prediction. All results of these modules are presented in Table 3. This table presents every Nao's 2D image with its CNN-SRIN trigger status. If the status is "Yes", then the rest of the other modules' results are presented. The depth module provides a depth colored image, in which the yellowish pixels are considered as far Open-door accuracy within a reasonable period of time on the Graham cluster. We randomly selected 12 related images, i.e., six images for each class. Table 1 shows all images with their predictions. The model successfully predicted five images out of six with the correct class for each category, i.e., a total of 10 correct predictions as shown in Table 2. These results validated that this module within the 3Ds-system will be a good trigger for the next module.

Stage 2: Angle Extraction from 2D Images Based on Depth Map and Pixel Selection
The next modules of the 3Ds-system were tested on several real-time images from the previous module in order to get a practical proof of the successful performance. The expected outputs of the angles are in the range of [−30°, 30°] as the Nao's horizontal = 60.9°. We selected the six open-door images of Nao robot as well as the image of no-door with the false-positive prediction. All results of these modules are presented in Table 3. This table presents every Nao's 2D image with its CNN-SRIN trigger status. If the status is "Yes", then the rest of the other modules' results are presented. The depth module provides a depth colored image, in which the yellowish pixels are considered as far No-door accuracy within a reasonable period of time on the Graham cluster. We randomly selected 12 related images, i.e., six images for each class. Table 1 shows all images with their predictions. The model successfully predicted five images out of six with the correct class for each category, i.e., a total of 10 correct predictions as shown in Table 2. These results validated that this module within the 3Ds-system will be a good trigger for the next module.

Stage 2: Angle Extraction from 2D Images Based on Depth Map and Pixel Selection
The next modules of the 3Ds-system were tested on several real-time images from the previous module in order to get a practical proof of the successful performance. The expected outputs of the angles are in the range of [−30°, 30°] as the Nao's horizontal = 60.9°. We selected the six open-door images of Nao robot as well as the image of no-door with the false-positive prediction. All results of these modules are presented in Table 3. This table presents every Nao's 2D image with its CNN-SRIN trigger status. If the status is "Yes", then the rest of the other modules' results are presented. The depth module provides a depth colored image, in which the yellowish pixels are considered as far Open-door

Stage 2: Angle Extraction from 2D Images Based on Depth Map and Pixel Selection
The next modules of the 3Ds-system were tested on several real-time images from the previous module in order to get a practical proof of the successful performance. The expected outputs of the angles are in the range of [−30 • , 30 • ] as the Nao's horizontal FoV w = 60.9 • . We selected the six open-door images of Nao robot as well as the image of no-door with the false-positive prediction. All results of these modules are presented in Table 3. This table presents every Nao's 2D image with its CNN-SRIN trigger status. If the status is "Yes", then the rest of the other modules' results are presented. The depth module provides a depth colored image, in which the yellowish pixels are considered as far distances to a specific target, whereas the dark pixels represent very close objects. Then, the pixel selection module results are provided as follows: maximum depth value with its pixel, best-selected pixel with its depth value, and the vertical trigger status with its pixel. All depth values are rounded to two decimal points in this table for simplification. The last column of this table shows the calculated angle value from the last module if it is triggered by the previous module, otherwise, it gives a not applicable "n/a" value which means the robot does not receive any signal. The positive angle value means the robot turns left, whereas the negative value is for turning right.
The overall results show the success of the proposed system for detecting and navigating the robot toward a door in the indoor environments. It can be seen in images 1, 3-5, that the 3Ds-system successfully detects the doorway and estimates proper angle to direct Nao. The interesting results are shown in image 2, 6 and 7 that need to be discussed further. The system was able to detect an obstacle between the robot and the doorway when the pixel-based module was applied, as shown in image 2. Therefore, it did not send any angular value to Nao. Although the 3Ds-system could not detect a door in image 6, this does not affect the overall performance as there is an obstacle in front of the robot, which will be detected by pixel module and no angular value will be expected to be sent to Nao. The last tested image 7 is the false positive prediction of the CNN-SRIN trigger module. Since it predicted that there was a door, the other module was triggered and obtained its results. The angular value of image 7 leads the Nao robot to the free space direction, which is considered relatively as a good action within a navigation system that would lead to the doorway. This is certainly not conclusive evidence as it is possible to fail in other cases. distances to a specific target, whereas the dark pixels represent very close objects. Then, the pixel selection module results are provided as follows: maximum depth value with its pixel, best-selected pixel with its depth value, and the vertical trigger status with its pixel. All depth values are rounded to two decimal points in this table for simplification. The last column of this table shows the calculated angle value from the last module if it is triggered by the previous module, otherwise, it gives a not applicable "n/a" value which means the robot does not receive any signal. The positive angle value means the robot turns left, whereas the negative value is for turning right. The overall results show the success of the proposed system for detecting and navigating the robot toward a door in the indoor environments. It can be seen in images 1, 3-5, that the 3Ds-system successfully detects the doorway and estimates proper angle to direct Nao. The interesting results are shown in image 2, 6 and 7 that need to be discussed further. The system was able to detect an obstacle between the robot and the doorway when the pixel-based module was applied, as shown in image 2. Therefore, it did not send any angular value to Nao. Although the 3Ds-system could not detect a door in image 6, this does not affect the overall performance as there is an obstacle in front of the robot, which will be detected by pixel module and no angular value will be expected to be sent to Nao. The last tested image 7 is the false positive prediction of the CNN-SRIN trigger module. Since it predicted that there was a door, the other module was triggered and obtained its results. The angular value of image 7 leads the Nao robot to the free space direction, which is considered relatively as a good action within a navigation system that would lead to the doorway. This is certainly not conclusive evidence as it is possible to fail in other cases.

Validating the Overall Performance of 3Ds-System in Real-Time Experiments with Nao Humanoid Robot
As this work focuses on the door detection and direction, we evaluated the process by testing the 3Ds-system in real-time experiments with Nao in a new indoor environment. These experiments were carried out at Autonomous Intelligent System Laboratory (AISL) at Simon Fraser University (SFU). For practical purposes, it is important to mention that the Depth module is implemented on python 3 version, whereas Naoqi API works with python 2 version. Therefore, different modules in Yes Sensors 2020, 19, x FOR PEER REVIEW 10 of 17 distances to a specific target, whereas the dark pixels represent very close objects. Then, the pixel selection module results are provided as follows: maximum depth value with its pixel, best-selected pixel with its depth value, and the vertical trigger status with its pixel. All depth values are rounded to two decimal points in this table for simplification. The last column of this table shows the calculated angle value from the last module if it is triggered by the previous module, otherwise, it gives a not applicable "n/a" value which means the robot does not receive any signal. The positive angle value means the robot turns left, whereas the negative value is for turning right. The overall results show the success of the proposed system for detecting and navigating the robot toward a door in the indoor environments. It can be seen in images 1, 3-5, that the 3Ds-system successfully detects the doorway and estimates proper angle to direct Nao. The interesting results are shown in image 2, 6 and 7 that need to be discussed further. The system was able to detect an obstacle between the robot and the doorway when the pixel-based module was applied, as shown in image 2. Therefore, it did not send any angular value to Nao. Although the 3Ds-system could not detect a door in image 6, this does not affect the overall performance as there is an obstacle in front of the robot, which will be detected by pixel module and no angular value will be expected to be sent to Nao. The last tested image 7 is the false positive prediction of the CNN-SRIN trigger module. Since it predicted that there was a door, the other module was triggered and obtained its results. The angular value of image 7 leads the Nao robot to the free space direction, which is considered relatively as a good action within a navigation system that would lead to the doorway. This is certainly not conclusive evidence as it is possible to fail in other cases.

Validating the Overall Performance of 3Ds-System in Real-Time Experiments with Nao Humanoid Robot
As this work focuses on the door detection and direction, we evaluated the process by testing the 3Ds-system in real-time experiments with Nao in a new indoor environment. These experiments were carried out at Autonomous Intelligent System Laboratory (AISL) at Simon Fraser University (SFU). For practical purposes, it is important to mention that the Depth module is implemented on python 3 version, whereas Naoqi API works with python 2 version. Therefore, different modules in [185,194] 0 distances to a specific target, whereas the dark pixels represent very close objects. Then, the pixel selection module results are provided as follows: maximum depth value with its pixel, best-selected pixel with its depth value, and the vertical trigger status with its pixel. All depth values are rounded to two decimal points in this table for simplification. The last column of this table shows the calculated angle value from the last module if it is triggered by the previous module, otherwise, it gives a not applicable "n/a" value which means the robot does not receive any signal. The positive angle value means the robot turns left, whereas the negative value is for turning right. The overall results show the success of the proposed system for detecting and navigating the robot toward a door in the indoor environments. It can be seen in images 1, 3-5, that the 3Ds-system successfully detects the doorway and estimates proper angle to direct Nao. The interesting results are shown in image 2, 6 and 7 that need to be discussed further. The system was able to detect an obstacle between the robot and the doorway when the pixel-based module was applied, as shown in image 2. Therefore, it did not send any angular value to Nao. Although the 3Ds-system could not detect a door in image 6, this does not affect the overall performance as there is an obstacle in front of the robot, which will be detected by pixel module and no angular value will be expected to be sent to Nao. The last tested image 7 is the false positive prediction of the CNN-SRIN trigger module. Since it predicted that there was a door, the other module was triggered and obtained its results. The angular value of image 7 leads the Nao robot to the free space direction, which is considered relatively as a good action within a navigation system that would lead to the doorway. This is certainly not conclusive evidence as it is possible to fail in other cases.

Validating the Overall Performance of 3Ds-System in Real-Time Experiments with Nao Humanoid Robot
As this work focuses on the door detection and direction, we evaluated the process by testing the 3Ds-system in real-time experiments with Nao in a new indoor environment. These experiments were carried out at Autonomous Intelligent System Laboratory (AISL) at Simon Fraser University (SFU). For practical purposes, it is important to mention that the Depth module is implemented on python 3 version, whereas Naoqi API works with python 2 version. Therefore, different modules in Yes Sensors 2020, 19, x FOR PEER REVIEW 10 of 17 distances to a specific target, whereas the dark pixels represent very close objects. Then, the pixel selection module results are provided as follows: maximum depth value with its pixel, best-selected pixel with its depth value, and the vertical trigger status with its pixel. All depth values are rounded to two decimal points in this table for simplification. The last column of this table shows the calculated angle value from the last module if it is triggered by the previous module, otherwise, it gives a not applicable "n/a" value which means the robot does not receive any signal. The positive angle value means the robot turns left, whereas the negative value is for turning right. The overall results show the success of the proposed system for detecting and navigating the robot toward a door in the indoor environments. It can be seen in images 1, 3-5, that the 3Ds-system successfully detects the doorway and estimates proper angle to direct Nao. The interesting results are shown in image 2, 6 and 7 that need to be discussed further. The system was able to detect an obstacle between the robot and the doorway when the pixel-based module was applied, as shown in image 2. Therefore, it did not send any angular value to Nao. Although the 3Ds-system could not detect a door in image 6, this does not affect the overall performance as there is an obstacle in front of the robot, which will be detected by pixel module and no angular value will be expected to be sent to Nao. The last tested image 7 is the false positive prediction of the CNN-SRIN trigger module. Since it predicted that there was a door, the other module was triggered and obtained its results. The angular value of image 7 leads the Nao robot to the free space direction, which is considered relatively as a good action within a navigation system that would lead to the doorway. This is certainly not conclusive evidence as it is possible to fail in other cases.

Validating the Overall Performance of 3Ds-System in Real-Time Experiments with Nao Humanoid Robot
As this work focuses on the door detection and direction, we evaluated the process by testing the 3Ds-system in real-time experiments with Nao in a new indoor environment. These experiments were carried out at Autonomous Intelligent System Laboratory (AISL) at Simon Fraser University (SFU). For practical purposes, it is important to mention that the Depth module is implemented on python 3 version, whereas Naoqi API works with python 2 version. Therefore, different modules in distances to a specific target, whereas the dark pixels represent very close objects. Then, the pixel selection module results are provided as follows: maximum depth value with its pixel, best-selected pixel with its depth value, and the vertical trigger status with its pixel. All depth values are rounded to two decimal points in this table for simplification. The last column of this table shows the calculated angle value from the last module if it is triggered by the previous module, otherwise, it gives a not applicable "n/a" value which means the robot does not receive any signal. The positive angle value means the robot turns left, whereas the negative value is for turning right. The overall results show the success of the proposed system for detecting and navigating the robot toward a door in the indoor environments. It can be seen in images 1, 3-5, that the 3Ds-system successfully detects the doorway and estimates proper angle to direct Nao. The interesting results are shown in image 2, 6 and 7 that need to be discussed further. The system was able to detect an obstacle between the robot and the doorway when the pixel-based module was applied, as shown in image 2. Therefore, it did not send any angular value to Nao. Although the 3Ds-system could not detect a door in image 6, this does not affect the overall performance as there is an obstacle in front of the robot, which will be detected by pixel module and no angular value will be expected to be sent to Nao. The last tested image 7 is the false positive prediction of the CNN-SRIN trigger module. Since it predicted that there was a door, the other module was triggered and obtained its results. The angular value of image 7 leads the Nao robot to the free space direction, which is considered relatively as a good action within a navigation system that would lead to the doorway. This is certainly not conclusive evidence as it is possible to fail in other cases.

Validating the Overall Performance of 3Ds-System in Real-Time Experiments with Nao Humanoid Robot
As this work focuses on the door detection and direction, we evaluated the process by testing the 3Ds-system in real-time experiments with Nao in a new indoor environment. These experiments were carried out at Autonomous Intelligent System Laboratory (AISL) at Simon Fraser University (SFU). For practical purposes, it is important to mention that the Depth module is implemented on python 3 version, whereas Naoqi API works with python 2 version. Therefore, different modules in Yes Sensors 2020, 19, x FOR PEER REVIEW 10 of 17 distances to a specific target, whereas the dark pixels represent very close objects. Then, the pixel selection module results are provided as follows: maximum depth value with its pixel, best-selected pixel with its depth value, and the vertical trigger status with its pixel. All depth values are rounded to two decimal points in this table for simplification. The last column of this table shows the calculated angle value from the last module if it is triggered by the previous module, otherwise, it gives a not applicable "n/a" value which means the robot does not receive any signal. The positive angle value means the robot turns left, whereas the negative value is for turning right. The overall results show the success of the proposed system for detecting and navigating the robot toward a door in the indoor environments. It can be seen in images 1, 3-5, that the 3Ds-system successfully detects the doorway and estimates proper angle to direct Nao. The interesting results are shown in image 2, 6 and 7 that need to be discussed further. The system was able to detect an obstacle between the robot and the doorway when the pixel-based module was applied, as shown in image 2. Therefore, it did not send any angular value to Nao. Although the 3Ds-system could not detect a door in image 6, this does not affect the overall performance as there is an obstacle in front of the robot, which will be detected by pixel module and no angular value will be expected to be sent to Nao. The last tested image 7 is the false positive prediction of the CNN-SRIN trigger module. Since it predicted that there was a door, the other module was triggered and obtained its results. The angular value of image 7 leads the Nao robot to the free space direction, which is considered relatively as a good action within a navigation system that would lead to the doorway. This is certainly not conclusive evidence as it is possible to fail in other cases.

Validating the Overall Performance of 3Ds-System in Real-Time Experiments with Nao Humanoid Robot
As this work focuses on the door detection and direction, we evaluated the process by testing the 3Ds-system in real-time experiments with Nao in a new indoor environment. These experiments were carried out at Autonomous Intelligent System Laboratory (AISL) at Simon Fraser University (SFU). For practical purposes, it is important to mention that the Depth module is implemented on python 3 version, whereas Naoqi API works with python 2 version. Therefore, different modules in distances to a specific target, whereas the dark pixels represent very close objects. Then, the pixel selection module results are provided as follows: maximum depth value with its pixel, best-selected pixel with its depth value, and the vertical trigger status with its pixel. All depth values are rounded to two decimal points in this table for simplification. The last column of this table shows the calculated angle value from the last module if it is triggered by the previous module, otherwise, it gives a not applicable "n/a" value which means the robot does not receive any signal. The positive angle value means the robot turns left, whereas the negative value is for turning right. The overall results show the success of the proposed system for detecting and navigating the robot toward a door in the indoor environments. It can be seen in images 1, 3-5, that the 3Ds-system successfully detects the doorway and estimates proper angle to direct Nao. The interesting results are shown in image 2, 6 and 7 that need to be discussed further. The system was able to detect an obstacle between the robot and the doorway when the pixel-based module was applied, as shown in image 2. Therefore, it did not send any angular value to Nao. Although the 3Ds-system could not detect a door in image 6, this does not affect the overall performance as there is an obstacle in front of the robot, which will be detected by pixel module and no angular value will be expected to be sent to Nao. The last tested image 7 is the false positive prediction of the CNN-SRIN trigger module. Since it predicted that there was a door, the other module was triggered and obtained its results. The angular value of image 7 leads the Nao robot to the free space direction, which is considered relatively as a good action within a navigation system that would lead to the doorway. This is certainly not conclusive evidence as it is possible to fail in other cases.

Validating the Overall Performance of 3Ds-System in Real-Time Experiments with Nao Humanoid Robot
As this work focuses on the door detection and direction, we evaluated the process by testing the 3Ds-system in real-time experiments with Nao in a new indoor environment. These experiments were carried out at Autonomous Intelligent System Laboratory (AISL) at Simon Fraser University (SFU). For practical purposes, it is important to mention that the Depth module is implemented on python 3 version, whereas Naoqi API works with python 2 version. Therefore, different modules in Yes Sensors 2020, 19, x FOR PEER REVIEW 10 of 17 distances to a specific target, whereas the dark pixels represent very close objects. Then, the pixel selection module results are provided as follows: maximum depth value with its pixel, best-selected pixel with its depth value, and the vertical trigger status with its pixel. All depth values are rounded to two decimal points in this table for simplification. The last column of this table shows the calculated angle value from the last module if it is triggered by the previous module, otherwise, it gives a not applicable "n/a" value which means the robot does not receive any signal. The positive angle value means the robot turns left, whereas the negative value is for turning right. The overall results show the success of the proposed system for detecting and navigating the robot toward a door in the indoor environments. It can be seen in images 1, 3-5, that the 3Ds-system successfully detects the doorway and estimates proper angle to direct Nao. The interesting results are shown in image 2, 6 and 7 that need to be discussed further. The system was able to detect an obstacle between the robot and the doorway when the pixel-based module was applied, as shown in image 2. Therefore, it did not send any angular value to Nao. Although the 3Ds-system could not detect a door in image 6, this does not affect the overall performance as there is an obstacle in front of the robot, which will be detected by pixel module and no angular value will be expected to be sent to Nao. The last tested image 7 is the false positive prediction of the CNN-SRIN trigger module. Since it predicted that there was a door, the other module was triggered and obtained its results. The angular value of image 7 leads the Nao robot to the free space direction, which is considered relatively as a good action within a navigation system that would lead to the doorway. This is certainly not conclusive evidence as it is possible to fail in other cases.

Validating the Overall Performance of 3Ds-System in Real-Time Experiments with Nao Humanoid Robot
As this work focuses on the door detection and direction, we evaluated the process by testing the 3Ds-system in real-time experiments with Nao in a new indoor environment. These experiments were carried out at Autonomous Intelligent System Laboratory (AISL) at Simon Fraser University (SFU). For practical purposes, it is important to mention that the Depth module is implemented on python 3 version, whereas Naoqi API works with python 2 version. Therefore, different modules in [183,73] 0 distances to a specific target, whereas the dark pixels represent very close objects. Then, the pixel selection module results are provided as follows: maximum depth value with its pixel, best-selected pixel with its depth value, and the vertical trigger status with its pixel. All depth values are rounded to two decimal points in this table for simplification. The last column of this table shows the calculated angle value from the last module if it is triggered by the previous module, otherwise, it gives a not applicable "n/a" value which means the robot does not receive any signal. The positive angle value means the robot turns left, whereas the negative value is for turning right. The overall results show the success of the proposed system for detecting and navigating the robot toward a door in the indoor environments. It can be seen in images 1, 3-5, that the 3Ds-system successfully detects the doorway and estimates proper angle to direct Nao. The interesting results are shown in image 2, 6 and 7 that need to be discussed further. The system was able to detect an obstacle between the robot and the doorway when the pixel-based module was applied, as shown in image 2. Therefore, it did not send any angular value to Nao. Although the 3Ds-system could not detect a door in image 6, this does not affect the overall performance as there is an obstacle in front of the robot, which will be detected by pixel module and no angular value will be expected to be sent to Nao. The last tested image 7 is the false positive prediction of the CNN-SRIN trigger module. Since it predicted that there was a door, the other module was triggered and obtained its results. The angular value of image 7 leads the Nao robot to the free space direction, which is considered relatively as a good action within a navigation system that would lead to the doorway. This is certainly not conclusive evidence as it is possible to fail in other cases.

Validating the Overall Performance of 3Ds-System in Real-Time Experiments with Nao Humanoid Robot
As this work focuses on the door detection and direction, we evaluated the process by testing the 3Ds-system in real-time experiments with Nao in a new indoor environment. These experiments were carried out at Autonomous Intelligent System Laboratory (AISL) at Simon Fraser University (SFU). For practical purposes, it is important to mention that the Depth module is implemented on python 3 version, whereas Naoqi API works with python 2 version. Therefore, different modules in Yes Sensors 2020, 19, x FOR PEER REVIEW 10 of 17 distances to a specific target, whereas the dark pixels represent very close objects. Then, the pixel selection module results are provided as follows: maximum depth value with its pixel, best-selected pixel with its depth value, and the vertical trigger status with its pixel. All depth values are rounded to two decimal points in this table for simplification. The last column of this table shows the calculated angle value from the last module if it is triggered by the previous module, otherwise, it gives a not applicable "n/a" value which means the robot does not receive any signal. The positive angle value means the robot turns left, whereas the negative value is for turning right. The overall results show the success of the proposed system for detecting and navigating the robot toward a door in the indoor environments. It can be seen in images 1, 3-5, that the 3Ds-system successfully detects the doorway and estimates proper angle to direct Nao. The interesting results are shown in image 2, 6 and 7 that need to be discussed further. The system was able to detect an obstacle between the robot and the doorway when the pixel-based module was applied, as shown in image 2. Therefore, it did not send any angular value to Nao. Although the 3Ds-system could not detect a door in image 6, this does not affect the overall performance as there is an obstacle in front of the robot, which will be detected by pixel module and no angular value will be expected to be sent to Nao. The last tested image 7 is the false positive prediction of the CNN-SRIN trigger module. Since it predicted that there was a door, the other module was triggered and obtained its results. The angular value of image 7 leads the Nao robot to the free space direction, which is considered relatively as a good action within a navigation system that would lead to the doorway. This is certainly not conclusive evidence as it is possible to fail in other cases.

Validating the Overall Performance of 3Ds-System in Real-Time Experiments with Nao Humanoid Robot
As this work focuses on the door detection and direction, we evaluated the process by testing the 3Ds-system in real-time experiments with Nao in a new indoor environment. These experiments were carried out at Autonomous Intelligent System Laboratory (AISL) at Simon Fraser University (SFU). For practical purposes, it is important to mention that the Depth module is implemented on python 3 version, whereas Naoqi API works with python 2 version. Therefore, different modules in distances to a specific target, whereas the dark pixels represent very close objects. Then, the pixel selection module results are provided as follows: maximum depth value with its pixel, best-selected pixel with its depth value, and the vertical trigger status with its pixel. All depth values are rounded to two decimal points in this table for simplification. The last column of this table shows the calculated angle value from the last module if it is triggered by the previous module, otherwise, it gives a not applicable "n/a" value which means the robot does not receive any signal. The positive angle value means the robot turns left, whereas the negative value is for turning right. The overall results show the success of the proposed system for detecting and navigating the robot toward a door in the indoor environments. It can be seen in images 1, 3-5, that the 3Ds-system successfully detects the doorway and estimates proper angle to direct Nao. The interesting results are shown in image 2, 6 and 7 that need to be discussed further. The system was able to detect an obstacle between the robot and the doorway when the pixel-based module was applied, as shown in image 2. Therefore, it did not send any angular value to Nao. Although the 3Ds-system could not detect a door in image 6, this does not affect the overall performance as there is an obstacle in front of the robot, which will be detected by pixel module and no angular value will be expected to be sent to Nao. The last tested image 7 is the false positive prediction of the CNN-SRIN trigger module. Since it predicted that there was a door, the other module was triggered and obtained its results. The angular value of image 7 leads the Nao robot to the free space direction, which is considered relatively as a good action within a navigation system that would lead to the doorway. This is certainly not conclusive evidence as it is possible to fail in other cases.

Validating the Overall Performance of 3Ds-System in Real-Time Experiments with Nao Humanoid Robot
As this work focuses on the door detection and direction, we evaluated the process by testing the 3Ds-system in real-time experiments with Nao in a new indoor environment. These experiments were carried out at Autonomous Intelligent System Laboratory (AISL) at Simon Fraser University (SFU). For practical purposes, it is important to mention that the Depth module is implemented on python 3 version, whereas Naoqi API works with python 2 version. Therefore, different modules in Sensors 2020, 19, x FOR PEER REVIEW 10 of 17 distances to a specific target, whereas the dark pixels represent very close objects. Then, the pixel selection module results are provided as follows: maximum depth value with its pixel, best-selected pixel with its depth value, and the vertical trigger status with its pixel. All depth values are rounded to two decimal points in this table for simplification. The last column of this table shows the calculated angle value from the last module if it is triggered by the previous module, otherwise, it gives a not applicable "n/a" value which means the robot does not receive any signal. The positive angle value means the robot turns left, whereas the negative value is for turning right. The overall results show the success of the proposed system for detecting and navigating the robot toward a door in the indoor environments. It can be seen in images 1, 3-5, that the 3Ds-system successfully detects the doorway and estimates proper angle to direct Nao. The interesting results are shown in image 2, 6 and 7 that need to be discussed further. The system was able to detect an obstacle between the robot and the doorway when the pixel-based module was applied, as shown in image 2. Therefore, it did not send any angular value to Nao. Although the 3Ds-system could not detect a door in image 6, this does not affect the overall performance as there is an obstacle in front of the robot, which will be detected by pixel module and no angular value will be expected to be sent to Nao. The last tested image 7 is the false positive prediction of the CNN-SRIN trigger module. Since it predicted that there was a door, the other module was triggered and obtained its results. The angular value of image 7 leads the Nao robot to the free space direction, which is considered relatively as a good action within a navigation system that would lead to the doorway. This is certainly not conclusive evidence as it is possible to fail in other cases.

Validating the Overall Performance of 3Ds-System in Real-Time Experiments with Nao Humanoid Robot
As this work focuses on the door detection and direction, we evaluated the process by testing the 3Ds-system in real-time experiments with Nao in a new indoor environment. These experiments were carried out at Autonomous Intelligent System Laboratory (AISL) at Simon Fraser University (SFU). For practical purposes, it is important to mention that the Depth module is implemented on python 3 version, whereas Naoqi API works with python 2 version. Therefore, different modules in

Yes (False)
Sensors 2020, 19, x FOR PEER REVIEW 10 of 17 distances to a specific target, whereas the dark pixels represent very close objects. Then, the pixel selection module results are provided as follows: maximum depth value with its pixel, best-selected pixel with its depth value, and the vertical trigger status with its pixel. All depth values are rounded to two decimal points in this table for simplification. The last column of this table shows the calculated  angle value from the last module if it is triggered by the previous module, otherwise, it gives a not applicable "n/a" value which means the robot does not receive any signal. The positive angle value means the robot turns left, whereas the negative value is for turning right. The overall results show the success of the proposed system for detecting and navigating the robot toward a door in the indoor environments. It can be seen in images 1, 3-5, that the 3Ds-system successfully detects the doorway and estimates proper angle to direct Nao. The interesting results are shown in image 2, 6 and 7 that need to be discussed further. The system was able to detect an obstacle between the robot and the doorway when the pixel-based module was applied, as shown in image 2. Therefore, it did not send any angular value to Nao. Although the 3Ds-system could not detect a door in image 6, this does not affect the overall performance as there is an obstacle in front of the robot, which will be detected by pixel module and no angular value will be expected to be sent to Nao. The last tested image 7 is the false positive prediction of the CNN-SRIN trigger module. Since it predicted that there was a door, the other module was triggered and obtained its results. The angular value of image 7 leads the Nao robot to the free space direction, which is considered relatively as a good action within a navigation system that would lead to the doorway. This is certainly not conclusive evidence as it is possible to fail in other cases.

Validating the Overall Performance of 3Ds-System in Real-Time Experiments with Nao Humanoid Robot
As this work focuses on the door detection and direction, we evaluated the process by testing the 3Ds-system in real-time experiments with Nao in a new indoor environment. These experiments were carried out at Autonomous Intelligent System Laboratory (AISL) at Simon Fraser University (SFU). For practical purposes, it is important to mention that the Depth module is implemented on python 3 version, whereas Naoqi API works with python 2 version. Therefore, different modules in

Validating the Overall Performance of 3Ds-System in Real-Time Experiments with Nao Humanoid Robot
As this work focuses on the door detection and direction, we evaluated the process by testing the 3Ds-system in real-time experiments with Nao in a new indoor environment. These experiments were carried out at Autonomous Intelligent System Laboratory (AISL) at Simon Fraser University (SFU). For practical purposes, it is important to mention that the Depth module is implemented on python 3 version, whereas Naoqi API works with python 2 version. Therefore, different modules in the 3Ds-system should be managed and combined via a python module called subprocess that includes Popen constructor for executing a child program with its suitable python virtual environment in a new process. The goal of these experiments is to show that Nao is able to detect the doorway and direct itself towards the doorway properly with a correct angle value. Simultaneously, it is able to detect an obstacle in the way to the door and prohibit applying the angle direction. We considered three different scenarios for this evaluated experiment: Nao is in front of the doorway from different distances and angles, Nao is not in front of the door, and Nao is in front of the door while an obstacle is in the way to the door (see some examples in Figure 8). environment in a new process. The goal of these experiments is to show that Nao is able to detect the doorway and direct itself towards the doorway properly with a correct angle value. Simultaneously, it is able to detect an obstacle in the way to the door and prohibit applying the angle direction. We considered three different scenarios for this evaluated experiment: Nao is in front of the doorway from different distances and angles, Nao is not in front of the door, and Nao is in front of the door while an obstacle is in the way to the door (see some examples in Figure 8).  Table 4 provides the results of several real-time experiments with the Nao humanoid robot. It shows the validation results of the three aforementioned scenarios. As we can see, there are Nao's perceptions before and after the "Turn" behavior. Nao decides to turn based on the acquired output from each module in the 3Ds-system. These results show the success of our proposed system in practice.   Table 4 provides the results of several real-time experiments with the Nao humanoid robot. It shows the validation results of the three aforementioned scenarios. As we can see, there are Nao's perceptions before and after the "Turn" behavior. Nao decides to turn based on the acquired output from each module in the 3Ds-system. These results show the success of our proposed system in practice. the 3Ds-system should be managed and combined via a python module called subprocess that includes Popen constructor for executing a child program with its suitable python virtual environment in a new process. The goal of these experiments is to show that Nao is able to detect the doorway and direct itself towards the doorway properly with a correct angle value. Simultaneously, it is able to detect an obstacle in the way to the door and prohibit applying the angle direction. We considered three different scenarios for this evaluated experiment: Nao is in front of the doorway from different distances and angles, Nao is not in front of the door, and Nao is in front of the door while an obstacle is in the way to the door (see some examples in Figure 8).
(a) (b) (c)  Table 4 provides the results of several real-time experiments with the Nao humanoid robot. It shows the validation results of the three aforementioned scenarios. As we can see, there are Nao's perceptions before and after the "Turn" behavior. Nao decides to turn based on the acquired output from each module in the 3Ds-system. These results show the success of our proposed system in practice. the 3Ds-system should be managed and combined via a python module called subprocess that includes Popen constructor for executing a child program with its suitable python virtual environment in a new process. The goal of these experiments is to show that Nao is able to detect the doorway and direct itself towards the doorway properly with a correct angle value. Simultaneously, it is able to detect an obstacle in the way to the door and prohibit applying the angle direction. We considered three different scenarios for this evaluated experiment: Nao is in front of the doorway from different distances and angles, Nao is not in front of the door, and Nao is in front of the door while an obstacle is in the way to the door (see some examples in Figure 8).
(a) (b) (c)  Table 4 provides the results of several real-time experiments with the Nao humanoid robot. It shows the validation results of the three aforementioned scenarios. As we can see, there are Nao's perceptions before and after the "Turn" behavior. Nao decides to turn based on the acquired output from each module in the 3Ds-system. These results show the success of our proposed system in practice. the 3Ds-system should be managed and combined via a python module called subprocess that includes Popen constructor for executing a child program with its suitable python virtual environment in a new process. The goal of these experiments is to show that Nao is able to detect the doorway and direct itself towards the doorway properly with a correct angle value. Simultaneously, it is able to detect an obstacle in the way to the door and prohibit applying the angle direction. We considered three different scenarios for this evaluated experiment: Nao is in front of the doorway from different distances and angles, Nao is not in front of the door, and Nao is in front of the door while an obstacle is in the way to the door (see some examples in Figure 8).
(a) (b) (c)  Table 4 provides the results of several real-time experiments with the Nao humanoid robot. It shows the validation results of the three aforementioned scenarios. As we can see, there are Nao's perceptions before and after the "Turn" behavior. Nao decides to turn based on the acquired output from each module in the 3Ds-system. These results show the success of our proposed system in practice. the 3Ds-system should be managed and combined via a python module called subprocess that includes Popen constructor for executing a child program with its suitable python virtual environment in a new process. The goal of these experiments is to show that Nao is able to detect the doorway and direct itself towards the doorway properly with a correct angle value. Simultaneously, it is able to detect an obstacle in the way to the door and prohibit applying the angle direction. We considered three different scenarios for this evaluated experiment: Nao is in front of the doorway from different distances and angles, Nao is not in front of the door, and Nao is in front of the door while an obstacle is in the way to the door (see some examples in Figure 8).
(a) (b) (c)  Table 4 provides the results of several real-time experiments with the Nao humanoid robot. It shows the validation results of the three aforementioned scenarios. As we can see, there are Nao's perceptions before and after the "Turn" behavior. Nao decides to turn based on the acquired output from each module in the 3Ds-system. These results show the success of our proposed system in practice. the 3Ds-system should be managed and combined via a python module called subprocess that includes Popen constructor for executing a child program with its suitable python virtual environment in a new process. The goal of these experiments is to show that Nao is able to detect the doorway and direct itself towards the doorway properly with a correct angle value. Simultaneously, it is able to detect an obstacle in the way to the door and prohibit applying the angle direction. We considered three different scenarios for this evaluated experiment: Nao is in front of the doorway from different distances and angles, Nao is not in front of the door, and Nao is in front of the door while an obstacle is in the way to the door (see some examples in Figure 8).
(a) (b) (c)  Table 4 provides the results of several real-time experiments with the Nao humanoid robot. It shows the validation results of the three aforementioned scenarios. As we can see, there are Nao's perceptions before and after the "Turn" behavior. Nao decides to turn based on the acquired output from each module in the 3Ds-system. These results show the success of our proposed system in practice.

Turn Right
Sensors 2020, 19, x FOR PEER REVIEW 11 of 17 the 3Ds-system should be managed and combined via a python module called subprocess that includes Popen constructor for executing a child program with its suitable python virtual environment in a new process. The goal of these experiments is to show that Nao is able to detect the doorway and direct itself towards the doorway properly with a correct angle value. Simultaneously, it is able to detect an obstacle in the way to the door and prohibit applying the angle direction. We considered three different scenarios for this evaluated experiment: Nao is in front of the doorway from different distances and angles, Nao is not in front of the door, and Nao is in front of the door while an obstacle is in the way to the door (see some examples in Figure 8).
(a) (b) (c)  Table 4 provides the results of several real-time experiments with the Nao humanoid robot. It shows the validation results of the three aforementioned scenarios. As we can see, there are Nao's perceptions before and after the "Turn" behavior. Nao decides to turn based on the acquired output from each module in the 3Ds-system. These results show the success of our proposed system in practice. the 3Ds-system should be managed and combined via a python module called subprocess that includes Popen constructor for executing a child program with its suitable python virtual environment in a new process. The goal of these experiments is to show that Nao is able to detect the doorway and direct itself towards the doorway properly with a correct angle value. Simultaneously, it is able to detect an obstacle in the way to the door and prohibit applying the angle direction. We considered three different scenarios for this evaluated experiment: Nao is in front of the doorway from different distances and angles, Nao is not in front of the door, and Nao is in front of the door while an obstacle is in the way to the door (see some examples in Figure 8).
(a) (b) (c)  Table 4 provides the results of several real-time experiments with the Nao humanoid robot. It shows the validation results of the three aforementioned scenarios. As we can see, there are Nao's perceptions before and after the "Turn" behavior. Nao decides to turn based on the acquired output from each module in the 3Ds-system. These results show the success of our proposed system in practice.

Open door
Sensors 2020, 19, x FOR PEER REVIEW 11 of 17 the 3Ds-system should be managed and combined via a python module called subprocess that includes Popen constructor for executing a child program with its suitable python virtual environment in a new process. The goal of these experiments is to show that Nao is able to detect the doorway and direct itself towards the doorway properly with a correct angle value. Simultaneously, it is able to detect an obstacle in the way to the door and prohibit applying the angle direction. We considered three different scenarios for this evaluated experiment: Nao is in front of the doorway from different distances and angles, Nao is not in front of the door, and Nao is in front of the door while an obstacle is in the way to the door (see some examples in Figure 8).
(a) (b) (c)  Table 4 provides the results of several real-time experiments with the Nao humanoid robot. It shows the validation results of the three aforementioned scenarios. As we can see, there are Nao's perceptions before and after the "Turn" behavior. Nao decides to turn based on the acquired output from each module in the 3Ds-system. These results show the success of our proposed system in practice.

Turn Left
Sensors 2020, 19, x FOR PEER REVIEW 11 of 17 the 3Ds-system should be managed and combined via a python module called subprocess that includes Popen constructor for executing a child program with its suitable python virtual environment in a new process. The goal of these experiments is to show that Nao is able to detect the doorway and direct itself towards the doorway properly with a correct angle value. Simultaneously, it is able to detect an obstacle in the way to the door and prohibit applying the angle direction. We considered three different scenarios for this evaluated experiment: Nao is in front of the doorway from different distances and angles, Nao is not in front of the door, and Nao is in front of the door while an obstacle is in the way to the door (see some examples in Figure 8).
(a) (b) (c)  Table 4 provides the results of several real-time experiments with the Nao humanoid robot. It shows the validation results of the three aforementioned scenarios. As we can see, there are Nao's perceptions before and after the "Turn" behavior. Nao decides to turn based on the acquired output from each module in the 3Ds-system. These results show the success of our proposed system in practice. 4 the 3Ds-system should be managed and combined via a python module called subprocess that includes Popen constructor for executing a child program with its suitable python virtual environment in a new process. The goal of these experiments is to show that Nao is able to detect the doorway and direct itself towards the doorway properly with a correct angle value. Simultaneously, it is able to detect an obstacle in the way to the door and prohibit applying the angle direction. We considered three different scenarios for this evaluated experiment: Nao is in front of the doorway from different distances and angles, Nao is not in front of the door, and Nao is in front of the door while an obstacle is in the way to the door (see some examples in Figure 8).  Table 4 provides the results of several real-time experiments with the Nao humanoid robot. It shows the validation results of the three aforementioned scenarios. As we can see, there are Nao's perceptions before and after the "Turn" behavior. Nao decides to turn based on the acquired output from each module in the 3Ds-system. These results show the success of our proposed system in practice. Open door the 3Ds-system should be managed and combined via a python module called subprocess that includes Popen constructor for executing a child program with its suitable python virtual environment in a new process. The goal of these experiments is to show that Nao is able to detect the doorway and direct itself towards the doorway properly with a correct angle value. Simultaneously, it is able to detect an obstacle in the way to the door and prohibit applying the angle direction. We considered three different scenarios for this evaluated experiment: Nao is in front of the doorway from different distances and angles, Nao is not in front of the door, and Nao is in front of the door while an obstacle is in the way to the door (see some examples in Figure 8).  Table 4 provides the results of several real-time experiments with the Nao humanoid robot. It shows the validation results of the three aforementioned scenarios. As we can see, there are Nao's perceptions before and after the "Turn" behavior. Nao decides to turn based on the acquired output from each module in the 3Ds-system. These results show the success of our proposed system in practice.

Turn
Right the 3Ds-system should be managed and combined via a python module called subprocess that includes Popen constructor for executing a child program with its suitable python virtual environment in a new process. The goal of these experiments is to show that Nao is able to detect the doorway and direct itself towards the doorway properly with a correct angle value. Simultaneously, it is able to detect an obstacle in the way to the door and prohibit applying the angle direction. We considered three different scenarios for this evaluated experiment: Nao is in front of the doorway from different distances and angles, Nao is not in front of the door, and Nao is in front of the door while an obstacle is in the way to the door (see some examples in Figure 8).
(a) (b) (c)  Table 4 provides the results of several real-time experiments with the Nao humanoid robot. It shows the validation results of the three aforementioned scenarios. As we can see, there are Nao's perceptions before and after the "Turn" behavior. Nao decides to turn based on the acquired output from each module in the 3Ds-system. These results show the success of our proposed system in practice.

Discussion
The number of experiments in Section 4.3 may appear to be inconclusive. We included only six experiments for different scenarios as the other attempts within the accessed area were almost similar to what is reported. Besides, we can consider the experiments on Nao's images from Section 4.2 as an extra validation since the angle outputs are the values that are supposed to be passed to the robot to turn, which is similar to what was presented in Section 4.3. However, implementing further experiments in different environments, such as schools or community centers, in the future are potentially useful validation steps but not within the scope of this paper. In addition, more images will be useful to be collected in future by Nao, or any similar social robot. We encourage the Nao robot community to assist and provide more indoor environment images to improve the validation performance of the doorway detection in Section 4.1. Table 5 presents a comparative evaluation of the proposed algorithm versus related methods outlined in Section 2. We acknowledge that this comparison is subjective and inferred from the source papers. Nevertheless, the main features of each algorithm including respective computational resources as well as their relative robustness are listed.

Discussion
The number of experiments in Section 4.3 may appear to be inconclusive. We included only six experiments for different scenarios as the other attempts within the accessed area were almost similar to what is reported. Besides, we can consider the experiments on Nao's images from Section 4.2 as an extra validation since the angle outputs are the values that are supposed to be passed to the robot to turn, which is similar to what was presented in Section 4.3. However, implementing further experiments in different environments, such as schools or community centers, in the future are potentially useful validation steps but not within the scope of this paper. In addition, more images will be useful to be collected in future by Nao, or any similar social robot. We encourage the Nao robot community to assist and provide more indoor environment images to improve the validation

Discussion
The number of experiments in Section 4.3 may appear to be inconclusive. We included only six experiments for different scenarios as the other attempts within the accessed area were almost similar to what is reported. Besides, we can consider the experiments on Nao's images from Section 4.2 as an extra validation since the angle outputs are the values that are supposed to be passed to the robot to turn, which is similar to what was presented in Section 4.3. However, implementing further experiments in different environments, such as schools or community centers, in the future are potentially useful validation steps but not within the scope of this paper. In addition, more images will be useful to be collected in future by Nao, or any similar social robot. We encourage the Nao robot community to assist and provide more indoor environment images to improve the validation performance of the doorway detection in Section 4.1. Table 5 presents a comparative evaluation of the proposed algorithm versus related methods outlined in Section 2. We acknowledge that this comparison is subjective and inferred from the source papers. Nevertheless, the main features of each algorithm including respective computational resources as well as their relative robustness are listed.

Discussion
The number of experiments in Section 4.3 may appear to be inconclusive. We included only six experiments for different scenarios as the other attempts within the accessed area were almost similar to what is reported. Besides, we can consider the experiments on Nao's images from Section 4.2 as an extra validation since the angle outputs are the values that are supposed to be passed to the robot to turn, which is similar to what was presented in Section 4.3. However, implementing further experiments in different environments, such as schools or community centers, in the future are potentially useful validation steps but not within the scope of this paper. In addition, more images will be useful to be collected in future by Nao, or any similar social robot. We encourage the Nao robot community to assist and provide more indoor environment images to improve the validation performance of the doorway detection in Section 4.1. Table 5 presents a comparative evaluation of the proposed algorithm versus related methods outlined in Section 2. We acknowledge that this comparison is subjective and inferred from the source papers. Nevertheless, the main features of each algorithm including respective computational resources as well as their relative robustness are listed.

Discussion
The number of experiments in Section 4.3 may appear to be inconclusive. We included only six experiments for different scenarios as the other attempts within the accessed area were almost similar to what is reported. Besides, we can consider the experiments on Nao's images from Section 4.2 as an extra validation since the angle outputs are the values that are supposed to be passed to the robot to turn, which is similar to what was presented in Section 4.3. However, implementing further experiments in different environments, such as schools or community centers, in the future are potentially useful validation steps but not within the scope of this paper. In addition, more images will be useful to be collected in future by Nao, or any similar social robot. We encourage the Nao robot community to assist and provide more indoor environment images to improve the validation performance of the doorway detection in Section 4.1. Table 5 presents a comparative evaluation of the proposed algorithm versus related methods outlined in Section 2. We acknowledge that this comparison is subjective and inferred from the source papers. Nevertheless, the main features of each algorithm including respective computational resources as well as their relative robustness are listed.

Conclusions
In this paper, we focused on addressing a doorway detection algorithm that will ultimately be used in indoor navigation for social robots with limited sensors. We proposed a robotic system called