Next Article in Journal
Prediction Model of Shrinkage Crack Propagation in Cross-Section of Timber Components
Previous Article in Journal
Seismic Performance of Rectangular Hollow Section X-Joints Subjected to In-Plane Bending Moment
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Passive Auto-Tactile Heuristic (PATH) Tiles: Novel Robot-Inclusive Tactile Paving Hazard Alert System

by
Matthew S. K. Yeo
*,†,
Javier J. J. Pey
and
Mohan Rajesh Elara
Engineering Product Development (EPD), Singapore University of Technology and Design, 8 Somapah Rd, Singapore 478372, Singapore
*
Author to whom correspondence should be addressed.
These authors contributed equally to this work.
Buildings 2023, 13(10), 2504; https://doi.org/10.3390/buildings13102504
Submission received: 4 September 2023 / Revised: 26 September 2023 / Accepted: 28 September 2023 / Published: 2 October 2023
(This article belongs to the Section Architectural Design, Urban Science, and Real Estate)

Abstract

:
Mobile service robots often have to work in dynamic and cluttered environments. Multiple safety hazards exist for robots in such work environments, which visual sensors may not detect in time before collisions or robotic damage. An alternative hazard alert system using tactile methods is explored to pre-emptively convey surrounding spatial information to robots working in complex environments or under poor lighting conditions. The proposed method for robot-inclusive tactile paving is known as Passive Auto-Tactile Heuristic (PATH) tiles. These robot-inclusive tactile paving tiles are implemented in spatial infrastructure and are aimed to allow robots to pre-emptively recognize surrounding hazards even under poor lighting conditions and potentially provide improved hazard cues to visually impaired people. A corresponding Tactile Sensing Module (TSM) was used for the digital interpretation of the PATH tiles and was mounted onboard a mobile audit robot known as Meerkat. The experiment yielded a 71.6% improvement in pre-emptive hazard detection capabilities with the TSM using a customized Graph Neural Network (GNN) model.

1. Introduction

Robots are deployed in more public spaces, with the range of tasks performed by robots increasing over time [1,2,3], especially those tasks that are considered dull, dirty, and dangerous to humans [4]. In particular, mobile service robots are conducting more cleaning, cleanliness auditing [5] and inspection tasks in built-up transit zones like hotel lobbies, airports, and restaurants [6,7]. These mobile service robots compensate for the lack of manpower for mundane and tedious jobs [8,9]. Robots encounter operational inefficiency if the existing environment is not catered for their deployment, along with the spatial obstacles they can encounter. Robot stoppage, damage to property and robots through collisions, and reduced robotic productivity are some problems that occur if the situation is not addressed.
Existing robotic detection methods that rely solely on visual means are limited by occlusion, lighting conditions, and image quality [10,11]. The sensors installed on robots let them perceive their surroundings and the hazards in their workspace. Mobile robots utilize a variety of sensors for them to localize themselves within their deployed workspace [12] and move to the next point of interest based on their sensor inputs and programming. Some of the most common localization techniques involve Simultaneous Localization And Mapping (SLAM) [13], landmark recognition or beacon arrays [14], the use of fiducial markers [15,16], and the Global Positioning System (GPS) [17]. A summary of these methods and their limitations is seen in Table 1. Error information obtained from the sensors could lead to errors in the robot’s localization or false positives in detecting the surrounding hazards, depending on the robot’s software [18,19,20].
Robots inevitably run into obstacles in their environment during operation, be it objects in their path (spatial hazards) or items that cause detection problems in the robot’s input sensors (perceptional hazards). A non-exhaustive list of spatial hazard classes encountered by mobile service robots is categorized in Table 2. The examples listed are not exhaustive but contain the general types of hazards that mobile robots encounter during operation.
Robot designers should consider alternative methods for hazard detection for robots to sense their environment instead of solely looking upon visual means of spatial hazard detection. In light of the listed limitations in visual means, tactile means could be explored for object localization and detection [21] and even be combined with vision sensors [22,23,24,25] for more comprehensive hazard detection of obstacles in the environment. In [26], various modified tactile sensors were experimented upon in different simulations to show the viability of using tactile sensors in robotics for hazard perception.

1.1. Advantages of Tactile Sensing Usage by Robots

There are numerous benefits to using tactile sensing in hazard detection for robots. The main advantages that tactile sensing could provide are cheaper hardware implementation [27,28] and reduced computational load compared with visual methods [29,30].
The use of tactile methods as sensor inputs could reduce hardware implementation costs, with less reliance on expensive visual detection equipment such as LIDAR or depth cameras. The resources used for such robotic components, such as rare metals and plastics, would be freed for other uses. By bringing down the overall costs of building robots, more robots could then be built with the same amount of financial capital for sales or deployment, improving cost effectiveness and overall production output [30].
Tactile sensing could cut down the computational load that the robotic processors operate with due to simplified data types being parsed, oftentimes by having the system detect the tactile changes upon the surface themselves as Boolean states, instead of having to deal with visual clutter, sub-optimal lighting conditions, or visual inputs with grainy image resolutions.
Visual clutter or downgraded image resolution can increase the time or computational memory required for the robot to identify and detect hazards in its surroundings. Tactile sensing could also complement visual sensing to reduce error rates in the spatial hazard detection of the robot’s surroundings [31,32] as a backup when the visual input cannot be used effectively by the robot.
Input data gathered using tactile sensing would also be unaffected by the local lighting conditions of the area. This could enable robotic aid equipped with these tactile sensors to be deployed in regions with low lux or outdoor areas illuminated with bright lights without hampering the detection of hazards within its surroundings.
There is thus the potential and opportunity to adapt this tactile paving system to aid robotic deployment to both improve usage rates and refine the type of information conveyed by the updated tactile indicators. More information on the existing human tactile paving system is detailed in the next section.
Robots could thus benefit greatly from a tactile hazard alert system and its corresponding interpretation program to have a robust tactile hazard detection method. To achieve that, we can look into existing tactile systems used for conveying information on hazards present in the environment. One of the successful existing tactile paving systems effectively aids visually impaired or blind (VIB) users in navigating to specific amenities such as elevators and service counters in public settings. It also helps VIB people avoid obstacles in their path [33]. Existing tactile sensing infrastructure is underutilized by the general population but is still needed by VIB people for their daily commute. Then, tactile sensing could also enable robots to be deployed in areas with extreme lighting conditions or in restricted areas that prohibit visual mapping.

1.2. Existing Tactile Paving System for Visually Impaired/Blind People

The tactile paving system was created by Seiichi Miyake in 1965 and implemented in Okayama, Japan, in 1967 [34]. Standards for the design and implementation of tactile paving are referenced from ISO/FDIS 23599 [35] and CEN/TS 15209 [36]. Countries such as Singapore [37,38], New Zealand [39], the United Kingdom (UK) [40], the United States [41], and Japan [42] have their own customized national standards for tactile paving [43,44].
These tactile paving tiles are usually brightly colored in red or yellow for distinct color contrast for locations near level changes [45,46,47] or can be colored to blend into the surrounding floor material [46]. Tactile paving is commonly made from durable materials, such as steel, granite, or hard rubber. This allows it to withstand constant foot traffic; heavy rolling loads; extreme outdoor UV levels; and other weather elements, such as wind and rain.
The existing tactile paving tiles commonly use round bumps or long, slender directional bumps [48,49]. There are specific patterns of tactile paving that convey different information types. One design provides directional cues for VIB people to guide them towards amenities. Another design denotes warnings of nearby level changes. Other designs of tactile paving tiles serve to delineate the boundary between cycling tracks and pedestrian walkways located beside each other. The distance between tactile paving and the actual hazard is dependent on the country’s requirements [41,44]. The various designs are detailed below.
Blister tactile paving tiles are typical tactile ground surface indicators (TGSIs) that have rows of flat-topped blisters placed in a rectilinear grid pattern, as seen in Figure 1A. This type of tactile paving are usually located at pedestrian crossings. They alert VIB users about the interface between footpaths and roadways or about locations with crossing gaps. The blisters are raised, truncated cones. Each blister is about 5 ± 0.5 mm high and 35–50 mm away from the others. The blisters’ base diameter is about 30–35 mm, with a diameter of 25 mm at the top.
Offset blister tiles are a specific type of blister tactile paving that warns of platform edges or cliffs ahead. The blisters are laid out in a diamond grid pattern instead of the rectilinear grid of the normal blister tiles, as seen in Figure 1B. The tile’s blisters are spaced about 66.5 mm from each other. These tiles are usually set back about 50 cm from platform edges to inform VIB users about the hazard ahead to let them stop or slow down before the level’s edge.
Corduroy tactile paving tiles are made of flat-topped bars with rounded ends to convey directional cues. These bars are 6 ± 0.5 mm high and about 20 mm in width and are spaced 50 mm apart, as shown in Figure 1C. The length of the raised bars is dependent on the tile’s length and any local tactile paving standards. Corduroy paving aligned parallel to the direction of travel denotes a safe path for VIB users to follow along. Corduroy paving tiles that are laid perpendicularly to the direction of travel are often located near level changes (such as staircase landings, ramps, level crossings, and intersections of footpaths and roadways) to warn nearby users of the presence of hazards ahead.
Lozenge paving tiles are a subtype of corduroy tactile paving that functions as a platform-edge warning surface, with rows of 6 ± 0.5 mm rounded lozenge shapes, as seen in Figure 1D. These lozenge tiles are arranged parallel to the platform edge and set back from the platform edge to give VIB users time to stop and change direction once they come into contact with lozenge tactile paving.
Cycleway tactile paving is made of multiple, continuous, raised, flat-topped stripes. Each stripe is 5 ± 0.5 mm high, about 30 mm wide, and spaced 70 mm apart. The central delineator strip is 12–20 mm high, 150 mm wide, 5 cm in width with sloping sides. The delineator strip is usually colored in white for visual contrast for sighted cyclists. Cycleway tactile paving is located on any shared route where the pedestrian side is not physically separated from the cyclist side. They are often located at the end of a shared segregated route, at regular intervals along its length, and at junctions intersecting with other pedestrian or cyclist routes.
Directional or guidance tactile paving tiles are a series of raised, flat-topped bars running in the direction of pedestrian travel. The bars are 5–6 mm in height, 35 mm wide, and spaced 45 mm apart. They are implemented in zones as obstacle avoidance or directional guides to guide VIB people to amenities.
Directional and blister tiles are often used in combination, such as those seen in Figure 2, to communicate a safe path to VIB people in public areas with multiple spatial hazards. At junctions where there is a directional change, a 2 × 2 grid of four blister tiles is usually used to indicate that there is a fork in the tactile path for VIB users. Drop-off points and pedestrian crossings also require blister tactile paving that spans the pavement–road interface.
An overview of the various tactile paving types and functions is shown in Table 3, referenced from [43,48]. Most countries implement blister and directional tiles more for widespread usage compared with the other tile variants.
Current tactile paving only provides limited tactile information on the surrounding spatial hazards to VIB people. More could be added to such a system to enhance the information conveyed to both humans and robots on the types of spatial hazards in the vicinity. Information such as the type of hazard and how far it is from the current tactile paving could be encoded upon the modified tactile paving. Examples could be warning VIB users and robots of level changes like stairs and slopes, or zones with high traffic, and the corresponding distance from the tactile paving’s location. Some existing tactile paving structures have installation errors and differences in applications among countries [50,51], which leads to improper or unsafe paths [52] for VIB users. Consequences of such counterproductive tactile aid placement could be directing VIB people into obstacles planted in the way or guiding them to places with numerous fast-moving obstacles, leading to collisions and injuries. Having an updated tactile paving catalog and proper installation layouts could help provide a way of correcting these errors by implementing the updated system.

1.3. Modifying Ground Tactile Indicators as Hazard Warning System for Robots

Upgrading the tactile paving catalog would be beneficial for both VIB people and mobile robots to passively obtain data on the surrounding hazards and obstacles, and their corresponding distances, with minimal processing or the need for wireless network connectivity in the area. This improved tactile paving system would also require less maintenance than active hazard warning systems and could also be scaled to outdoor environments without being subject to extreme lighting conditions.
The working zones of mobile robots could be spatially denoted by placing boundary tiles at the perimeter of the workspace to alert humans and robots working simultaneously in the same zone. The tiles could be installed on floor surfaces instead of requiring the invasive embedding of materials used for detection within the floor slabs during the building stage, making them easier to install or replace. This novel, exploratory design for an updated tactile paving system is termed Passive Auto-Tactile Heuristic (PATH) tiles and will be discussed in its own section.
This paper proposes contributions in the following areas:
  • The usage of modified ground tactile markers used as an advanced hazard warning system for mobile robots, known as Passive Auto-Tactile Heuristic (PATH) tile system.
  • A designed low-cost Tactile Sensor Module (TSM) made to detect such tactile cues.
  • A customized graph neural network (GNN) framework made for robots to robustly interpret the tiles and for informing the robot about the hazards ahead.
Among the following sections of the paper, Section 2 delves into the novel modified tactile paving tiles, known as Passive Auto-Tactile Heuristic (PATH) tiles, and the designed TSM and expands on GNNs and dataset generation for allowing robots to interpret PATH tiles using the TSM. Section 3 discusses the results and validation of the GNN model on the PATH tiles inputted by the TSM. Section 4 discusses the accuracy of the GNN in real-life experiments and discusses future works. Section 5 then concludes the paper.

2. Materials and Methods

The upgraded tactile paving system, also known as Passive Auto-Tactile Heuristic (PATH) paving tiles, is a supplemental hazard warning system adapted to conveniently help mobile service robots move safely through their work environment. The modified tactile hazard detection system does not compromise on its existing function of providing hazard recognition or spatial guidance to VIB users but builds upon the existing system for VIB users and includes mobile service robots as users for them to discern and navigate their environment with a lower likelihood of accidents.
Customizing the existing tactile system, which is integrated worldwide in architectural settings, could help in applicability and ease of implementation to improve the productivity of robot deployment, akin to passive infrastructural elements such as integrated roofing for adjustable shading and ventilation [53,54] or passive building cooling systems [55,56] to improve working conditions for people. There are various obstacles faced by mobile robots during operation; hence, PATH tiles could help define the different types of spatial hazards and relay the hazard data to the robots’ sensors for differentiated interpretation and response. This tactile sensing system could thus enhance the robustness of robotic environmental hazard detection and act as a fail-safe system for robots’ operational safety by employing tactile infrastructure for robotic hazard detection.
With the modified tile designs, a proper interpreter system is also developed to allow robots to decipher the information from the PATH tiles installed in their environment. This is achieved by using a customized TSM and its related Graph Neural Network (GNN). The GNN enables the robot to parse multiple separate parameters and embed relationships, as well as catering to the need fir quick prediction through noise reduction during operation.

2.1. Passive Auto-Tactile Heuristic (PATH) Tiles

PATH tiles are a catalog of modified tactile paving tiles that convey distance and hazard data to robots working in the area. The implementation of PATH tiles thus aims to build upon a novel approach [57,58,59] by improving robotic productivity with the use of robot-inclusive architectural design changes, instead of solely improving the robots’ functionalities.
The proposed infrastructural system with PATH tiles is not adversarially created as another hazard warning system, such as those implemented in existing built-up areas, but is a novel, non-intrusive architectural integration. This system aims to improve mobile robots’ ability in detecting and reacting to environmental hazards in their deployment sites. Implementing this modified tactile system could also provide an impetus for correcting installation errors in existing tactile paving layouts. The PATH tiles are categorized into two types, one for the hazard type (hazard PATH tiles) and the other for conveying the distance from PATH tiles to an obstacle (distance PATH tiles), as seen in Figure 3 for differentiated responses to different hazard placement. When discussing the distance PATH tiles, it is assumed that the robot takes its current location upon coming into contact with the PATH tile as the corresponding distance from the hazard.
A typical PATH tile measures 300 mm by 300 mm and has reflectional symmetry along the axis of travel to allow the robot to decode the same information when the robot traverses the PATH tile. Counterparts to existing tactile indicators, as seen in the bump designs of the guidance tile, the moving obstacle tile, and the vertical level change PATH tile, were designed to be analogous to the existing guidance tile, blister tile, and offset blister tile designs, respectively. This consistency in the signals conveyed could aid VIB users in interpreting the updated tactile paving. This was reflected in the design choices, whereby truncated blisters are still used to mainly indicate hazards and raised bars or stripes are used to indicate paths of travel in the updated tactile paving system [41]. This maximizes the overlap with the functions of some of the existing tiles and the subsequent information conveyed.
The split between distance and hazard PATH tile designs would give robots the time to detect and process the information conveyed by the PATH tiles when they travel over the tiles at different speeds. It also allows customization between the various safety distances and hazard types based on different object layouts for different areas.
Distance PATH tiles are designed with horizontal bars or bumps, with the density of bars or bumps increasing when the tile denotes hazards at a closer distance ahead of the robot and the TSM. Hazard PATH tiles have a varied design in terms of bump patterns to allow the TSM to make a better distinction among the various hazard types ahead.
The distance and hazard PATH tiles are meant to be used in conjunction to provide a complete representation of the hazard type and its distance from the current position of the robot as the robot passes over the PATH tiles. For blister and guidance tiles, being analogs of existing tactile paving, they can be arranged in a similar manner to convey the same type of information on which path to follow. The PATH tile pair is arranged perpendicularly to the direction of travel, spanning the width of the path if needed, to mitigate cases where a robot does not detect the PATH information due to gaps in the PATH tile layout.
For our experiment, the tiles were fabricated through 3D printing, split into halves and quarters due to constraints in print bed size, using PLA material. Samples of the 3D-printed tiles that were used in multiples for our experiment are shown in Figure 4.

2.2. Design of a Tactile Sensor Module for PATH Tile Classification

As discussed in Section 1, the detection of PATH tiles using visual means is hindered by limitations such as lighting conditions, image quality, and occlusion. As such, the perception capabilities of the robot through visual equipment, like stereo cameras, may not be the most appropriate option, as the detection of hazards could be masked by environmental conditions. Therefore, given the benefits of including tactile methods for hazard detection discussed in Section 1.1, we propose the development of a novel TSM that collects information from the PATH tiles using non-visual means. The elegance of this tactile detection system is that the contact switches do not use light or sound as a form of input, rendering it insusceptible to changes in lighting or auditory conditions. However, being a tactile sensor that is dependent on ground contact, it is sensitive to ground surface undulations. This is mitigated using the Graph Neural Network to filter out irrelevant bump patterns and ensure that the tiles can be interpreted by the robot properly. Moreover, it depends on the robot’s pathing algorithm whether the robot traverses the tiles on its route for the sensor to detect the tiles in the first place. The TSM was mounted as a modular component onboard an in-house-developed autonomous mobile robot, Meerkat. Details of Meerkat are seen in Figure 5, while its system architecture is seen in Figure 6. This robot is capable of hazard detection and spatial auditing and is controlled using the Robot Operating System 2 (ROS2) software architecture.
Our proposed in-house-developed TSM aims to be modular for robots and easy to use. To satisfy these conditions, limit switches were used as the major components to detect the variation in PATH tile contours that the robot passes over. Limit switches can toggle operating states as a reaction upon crossing a threshold input value [60], making them suitable to detect changes in critical values such as pressure or voltage. The favorable advantages of reliability, precision, and ease of installation further provide easy monitoring and data collation when deployed in real-world environments. This module was further integrated with our GNN models, enabling the robot to classify PATH tiles based on readings received by the TSM.
The TSM consists of a linear array of 10 KH-9015-HRL limit switches that are evenly spaced and mounted onto the sensor frame that was designed and 3D-printed in our lab using PLA filament, as shown in Figure 7. The TSM was installed onto the front of the Meerkat, as shown in Figure 8.
Given that limit switches detect changes through Boolean logic conditions, two types of readings are produced. Each limit switch is connected through the Common and Normally Open (NO) ports. In this study, a reading of 0 indicates that the switch is depressed and an elevated contour is present at the position of that specific limit switch. In contrast, a reading of 1 indicates that the switch is not depressed and no contour is present at that specific position. Figure 9 depicts the limit switches being depressed and released as the robot traverses PATH tiles to identify the type of PATH tile pattern.
Figure 10 shows a close-up of a possible limit switch depression configuration when passing over a PATH tile from the front view. During deployment, the data collected from the engagement and disengagement of the limit switches within the TSM provide an approximate detection of the PATH tile that the robot is traversing.
To obtain the tactile data, the linear array of tactile sensors is made to traverse the PATH tile to a generate 2-dimensional (2D) tactile input array made by consecutive 1-dimensional (1D) inputs. The 2D array is then used as input for the GNN classification system to enable the robot to autonomously recognize the information provided by the PATH tiles with regards to the hazards nearby.

2.3. Graph Neural Network PATH Tile Classification Framework

GNNs have received significant coverage over recent years due to the ability to map relationships and connections among multiple complex data parameters into a consolidated graph network well, which has resulted in convincing performance. As data structures, graphs have great expressive abilities that can be used to model a multitude of applications, ranging from social networks [61] and knowledge relations [62] to even traffic forecasting [63], and were also explored for usage in structural crack detection in maintenance works [64,65,66]. Furthermore, additional features and attributes can also be integrated into the graph structure to supplement the network to provide more complexity and detailed information about the relationships.
The GNN classification framework is split into training pipeline and deployment phases, shown in Figure 11. In the training pipeline phase, the selected GNN model is trained in simulation based on a curated PATH tile dataset. The trained weights from the model are then tuned and optimized prior to deployment. During deployment, the optimized weights are loaded onto the GNN model using the same network architecture to classify PATH tiles in real-time. To model the states of the limit switches as a graph, displayed in the overview in Figure 12, the Boolean logic for each of the 10 limit switches is consolidated. A value of 0 indicates that the switch is engaged, which represents traversing an elevated contour. In contrast, a value of 1 represents that the switch is disengaged and no contour is detected. This forms an array of binary values at a particular time instance. In total, n instances of such an array are taken to form a data batch as the robot moves across the PATH tiles. This is to ensure that there are sufficient data to clearly distinguish among and classify the 8 tile patterns. A ground-truth representation of each of the PATH tiles is displayed in Figure 13, with black cells representing 0 values and white cells representing 1 values.
The sequence of the collected 10 × n data points forms a data batch representing a single node in the graph and is given a label tagged to the PATH tile pattern class. An edge connection is formed between two nodes if the sequence and the values are completely equivalent. To supplement graph modeling, two edge attributes were added. The first attribute is the total number of zeroes in each data batch, represented by A 1 . This shows the number of contour elevations detected at all limit switch positions over the course of n instances. The second attribute is the sum of the number of Boolean logic value changes from each limit switch position over n instances, represented by A 2 . For example, for each limit switch across 2 consecutive instances, a change is valid if the Boolean logic value switches between 2 states, either from 0 to 1 or from 1 to 0. In contrast, if the states remain the same between the 2 instances, no change is detected. This is repeated for all 10 limit switches on the TSM. A simplified weight score using the two attributes is also calculated. Given that both attributes are equally important in describing the characteristics of each data batch, the weight score, represented by W, is the average of the sum between the total number of zeroes and Boolean logic value changes, as shown in Equation (1). In the context of this experiment, the goal of the GNN is to predict the class of an unlabeled node when inputted into the graph using the learned embeddings and representations of labeled nodes.
W = A 1 + A 2 2
For dataset curation, each ground-truth representation is rotated in both directions, mirrored horizontally or vertically, or subject to random noise addition. In practical scenarios, the robot may not always be perfectly aligned with the tiles during the data collection process for pattern detection. Additionally, some of the limit switches may also not engage or disengage with their supposed respective Boolean logic values. As such, the generated dataset arrays can cater, simulate, and compensate for deviations by the robot during locomotion while still providing the avenue for unique pattern detection and classification. The artificially introduced variations were used in the training and validation steps of GNN model tuning. This allowed the model to learn and recognize patterns similar to the 8 listed and null patterns. With the tuned GNN model, the TSM can perform consistently regardless of environmental conditions. The validation dataset is for allowing the model to recognize what is to be considered part of the 8 tiles plus null detection instances. The best-performing trained GNN model is then further tuned to refine the trained weights to cater for any of the previously unknown patterns.
A total of 10,000 data batches were generated for each PATH tile pattern in reference to the respective ground-truth representations, with a 80–20% training–validation data split. Training and validation masks were generated to ensure that the training and validation data were kept separate. In total, 3 sets of test data, comprising 2250 data batches, were generated for further testing and evaluation.
In this paper, 3 baseline GNN models were considered based on their relevance to map networks and relationships for node classification tasks, such as predicting the research topic of different articles within a citation network or the gene type within protein–protein interactions. Zhou et al. [67] and Xiao et al. [68] documented the usage of the convolutional and attention mechanism classes commonly used for node classification. Convolutional mechanisms focus on conducting pooling or convolutional operations within the graph structure. This enables the extraction of features and higher-level representation for each specific node, which can then be inputted into a node classifier. Attention mechanisms rely on learning the different contributions towards the target node as opposed to relying on a single fixed weight. The mechanism also varies the number of neighbors that the target node can take, which can thus accommodate inputs of different sizes while placing attention on specific relevant and important sections.
From the existing literature, Graph Convolutional Network (GCN) [69], Graph Attention Network (GAT) [70], and Chebyshev Graph Convolutional Network (CHEB) [71] were selected, as these models are commonly used in the analysis and prediction of unseen and unlabeled nodes. GCN is spectral-based and expresses the essential hidden representation features of each target node. GAT has the ability to capture and identify the different importance of neighborhood information for a specific target node when generating higher hidden representations. CHEB is a spectral-based graph convolutional network that has a spectral filer derived from the Chebyshev polynomial. The pipeline for training and predicting the output classes of unlabeled nodes using each of the algorithms is documented in Figure 14, and the weights of each model are saved.
The training and validation of each model were conducted on the Nuvo-8108GC Series Rev. A1 Industrial PC by Neousys Technology Inc. with the NVIDIA GeForce RTX 3080 Lite Hash Rate GPU graphics card using the Pytorch Geometric library [72]. Each model was fitted with an early stopping callback, which allows the model to stop training and validating to save computation time and memory. A threshold of 500 epochs was implemented, which instructs the model to stop training and validating. This occurs if the validation loss does not improve for 500 epochs. An additional hyper-parameter tuning process was conducted to improve the performance of the selected model. The optimized weights were then tested on the 3 test sets to demonstrate the model’s performance on unseen data in simulation.
During actual deployment, the TSM is connected to an Arduino Uno micro-controller, which is connected to the Industrial PC. The ROS2 program calls a service to run a script that obtains real-time readings of the TSM from the Arduino Uno. Upon completing data collection, the loaded GNN model and the corresponding weights classify the obtained data, after which the ROS2 service is terminated.

2.4. Experimental Setup

The experiment is a proof of concept for utilizing automated tactile infrastructure and detection systems. One of the novel aspects is the use of a GNN as a model to cater for scenarios where the robot passes over the tiles at an angle or for delayed detection during mobile robot deployment. The Meerkat robot was made to move towards the chosen hazards of stairs, ramps, and glass doors 30 times for the blister and guidance tile patterns, for both setups before and after PATH tile installation. The robot was stopped if it was going to fall off the level change or collide into the hazards. The results were then compared if they outputted the correct predicted tile patterns using the customized GNN model with the corresponding PATH tile types in the environment. The experimental setup is seen in Figure 15.
When testing with the PATH tiles, the surrounding vicinity was laid with boards to make the surface flush with the tile base to prevent readings from the initial encounter with the tiles from affecting the readings. The robot was made to pass over the tiles 10 times for the blister and guidance tile patterns, with the customized GNN used in Section 2.3 being made to differentiate between the patterns.
The obtained outcomes of the experiment are tabulated in Section 3.

3. Results

The optimized weights from the training, validation, and testing of CHEB were loaded onto the robot. In real-world deployment, the robot encounters hazards in cases where the PATH tiles are absent, due to the lack of environmental cues in case of stairs or ramps. The robot was able to detect glass using its LIDAR sensors in some of the trial runs. Stairs, ramps, and glass doors were deemed the typical hazards that would cause the most damage to the robot, the environment, and human users in the vicinity if the robot were to collide, lose balance, or fall off the edge because of the inability to detect such edges or transparent objects in time.
For the guidance PATH tile pattern, the GNN provided correct recognition in 20 out of 30 trials, whereas for the blister PATH tile pattern, the correct pattern was recognized by the GNN in 23 out of 30 trials instead, as seen in Table 4. In the case of the blister tile pattern, incorrect predictions were made for the ’static obstacle’ or the ’2 m distance’ PATH tile patterns instead. This could have been due to the close similarities in the binary pattern created by the tiles themselves and the inherent variations in granularity of the input data in each run. This could have been due to having performed data collection at only 10 points (the number of limit switches for our current experiment) along the tile instead of a continuous line, which would have provided more distinct differences in the classification. Another possible reason for the error classification instances might be that the limit switches did not engage properly with the contours of the PATH tiles during the experiment.
From the results, we can see that installing the TSM onto the robot allowed it to better detect the upcoming hazards with the implementation of tactile aid. Given the correct detection rating of 43 out of 60 runs, there was a 71.6% success rate in pre-emptive hazard detection runs after the implementation of PATH tiles. Through this, it is seen that the tactile PATH tile patterns could be used for spatial hazard detection to improve robot safety during deployment in the future, using its corresponding GNN to utilize and differentiate the different tactile PATH tile patterns.

4. Discussion

In model training and validation, CHEB performed best in terms of the highest validation accuracy as well as the lowest validation loss, with average plateau validation accuracy and loss of 91.34% and 1.4235, respectively. Additionally, CHEB required the shortest inference time to complete training and validation before early stopping was engaged. GCN had validation accuracy of 83.86% and validation loss of 1.4456. In contrast, GAT obtained the lowest validation accuracy, 71.42%, and the highest validation loss, 1.4886, while requiring the longest training time. Table 5 collates the best loss and accuracy metrics during the training process. From our defined metrics, in the context of our task, CHEB was, therefore, selected as the most suitable model for node classification, as the model achieved the highest validation accuracy and the lowest validation loss. The loss and accuracy curves for each GNN model can be found in Appendix A.
To optimize the performance of the CHEB model, hyper-parameter tuning using the RAY library was used to provide the best configuration of hyper-parameters. The hyper-parameters used for tuning were (1) number of hidden channels, (2) dropout rate, (3) learning rate, and (4) weight decay, with the tuning range of each parameter being documented in Table 6. The metrics used for optimization maximized the validation accuracy, and this metric was reported back to the RAY tune reporter upon completion of every epoch. A total of 10 samples of different hyper-parameter combinations were used, and each combination was prematurely terminated if the performance of the model did not outperform the previous combinations. From the tuning process, the best values consisted of 128 hidden channels, a dropout rate of 2.4260 × 10 1 , a learning rate of 1.8055 × 10 2 , and weight decay of 3.2048 × 10 6 .
The weights of the optimized model, as well as the hyper-parameters, were saved before loading for usage during implementation onto the same training and validation dataset. Upon the completion of tuning, the validation accuracy of 95.12% and the validation loss of 1.4033 were obtained with the optimized model. This improved the validation accuracy by 3.78% and reduced the validation loss by 0.0202 compared with the results obtained prior to model tuning, as shown in Table 7.
During the GNN baseline model training and validation phase, CHEB outperformed the other baseline models in terms of both validation accuracy and validation loss. There are a few potential reasons for CHEB’s satisfactory performance. Firstly, CHEB uses Chebyshev polynomials, which have the ability to capture information from higher-order neighborhoods. This enables CHEB to learn long-range dependencies and embeddings from nodes within a large graph effectively. Secondly, CHEB has the ability to adaptively adjust the size of the target node’s neighborhood. This allows CHEB to select and include node neighbors based on the importance of their influence and relevance to the target node. Lastly, CHEB utilizes the spectral domain for computing Chebyshev polynomials, which can lead to improved scalability and efficiency. This was advantageous in our context, given that a large-scale graph was used within the experiment.
As a means of benchmark comparison, it was difficult to find similar previous works on GNNs related to our use case. Hence, comparisons were made in reference to Convolutional Neural Network (CNN) usage for safety or hazard detection scenarios. Comparing the results of the conducted experiment with relevant state-of-the-art classification methods, [73] used a CNN model to provide early safety warnings for intrusive behavior to mitigate accidents and casualties at construction sites using video image classification. Their proposed model obtained an accuracy rate of 87% using two classes. Ref. [74] also utilized a CNN model for reducing the repair cost of damaged concrete structures by detecting and classifying the presence of bacteria within the structure. The model achieved an average validation accuracy rate of 91.5%. Ref. [75] achieved variable classification accuracy ranging from 28% to 70% across three classes with single-shot detection using CNN to streamline job site management and decision making for machine safety. The GNN provided a better neural network alternative, as it could capture more abstract relationships among data entries for our use case. CHEB achieved higher accuracy, over 90%, in our case while also being able to accommodate classification of up to eight classes compared with three classes in the comparison literature.
The saved weights of the optimized model were then loaded onto a separate CHEB model with the same architecture documented in Section 2.3. Each test set was inputted into the optimized model for evaluation, and the results of the metric scores are documented in Table 8. From the results, an average test accuracy rate of 96.69% was obtained using the three test sets with our trained and tuned CHEB model. The obtained average test accuracy was 1.57% higher than the validation accuracy obtained after hyper-parameter tuning. The respective normalized confusion matrices for the eight different classes and the null pattern in the three test sets can be found in Appendix B.
The PATH tile system is slated to have more designs to enhance robotic wayfinding and navigation and also to demarcate different zones for the various extents of collaboration in human–robot collaborative spaces [76]. Furthermore, the TSM could be further improved with other types of tactile sensors instead of limit switches, as limit switches give inconsistent readings and could degrade over time with prolonged contact with the ground surface. The ideal case would be to use a sensor that tracks the whole span of the PATH tile to obtain an input closer to that of a line for pattern classification instead of point data. The TSM could perhaps be re-designed with adaptive sensor quantities for different robot sizes or corridor widths in different environments or by improving a support frame design for the sensor array to reduce variations in the results.
Another potential exploration direction is to improve the robustness of the PATH tile system and determine how to encode other types of environmental information onto this architectural infrastructure. Signals from the tactile detector could also be integrated into robotic systems to make use of the data gained from coming into contact with the PATH tiles in future iterations. Future ventures could also look into making the PATH tile system more conducive for usage by VIB people as well.

5. Conclusions

This paper presented the potential of using modified tactile paving known as Passive Auto-Tactile Heuristic (PATH) tiles in eight different designs as a complement in advance obstacle detection for VIB users and mobile robots. A customized TSM was developed and mounted onto a mobile robot platform for the validation of the tile-and-detector system. A validation experiment was conducted to determine the improvement in hazard detection through the implementation of spatial robot-inclusive tactile paving using PATH tiles and the corresponding TSM for enabling robots to interpret the tactile cues and showcased an improvement of 71.6% in terms of pre-emptive hazard detection by robots. This could further provide the basis for implementing infrastructural modifications to improve robotic safety in spatial environments.

Author Contributions

Conceptualization, M.S.K.Y. and M.R.E.; methodology, M.S.K.Y. and J.J.J.P.; software, J.J.J.P.; validation, M.S.K.Y., J.J.J.P. and M.R.E.; formal analysis, M.S.K.Y.; investigation, M.S.K.Y. and J.J.J.P.; resources, M.R.E.; data curation, M.S.K.Y. and J.J.J.P.; writing—original draft preparation, M.S.K.Y. and J.J.J.P.; writing—review and editing, M.S.K.Y. and J.J.J.P.; visualization, M.S.K.Y.; supervision, M.R.E.; project administration, M.R.E.; funding acquisition, M.R.E. All authors have read and agreed to the published version of the manuscript.

Funding

This research was supported by the National Robotics Programme under Robotics Enabling Capabilities and Technologies (Ermine III: Deployable Reconfigurable Robots, Funding Agency Project No. M22NBK0054) and was also supported by A*STAR under its “RIE2025 IAF-PP Advanced ROS2-native Platform Technologies for Cross sectorial Robotics Adoption (M21K1a0104)” program.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

  • The following abbreviations are used in this manuscript:
PATHPassive Auto-Tactile Heuristic
TSMTactile Sensing Module
GNNGraph Neural Network
SLAMSimultaneous Localization And Mapping
GPSGlobal Positioning System
VIBVisually impaired or blind
GCNGraph Convolutional Network
GATGraph Attention Network
CHEBChebyshev Graph Convolutional Network

Appendix A. Accuracy and Loss Curves for GNN Models

Figure A1, Figure A2 and Figure A3 document the accuracy and loss curves of the GCN, GAT, and CHEB models, respectively.
Figure A1. (a) Accuracy curves and (b) loss curves using Graph Convolutional Network (GCN).
Figure A1. (a) Accuracy curves and (b) loss curves using Graph Convolutional Network (GCN).
Buildings 13 02504 g0a1
Figure A2. (a) Accuracy curves and (b) loss curves using Graph Attention Network (GAT).
Figure A2. (a) Accuracy curves and (b) loss curves using Graph Attention Network (GAT).
Buildings 13 02504 g0a2
Figure A3. (a) Accuracy curves and (b) loss curves using Chebyshev Spectral Graph Convolution (CHEB).
Figure A3. (a) Accuracy curves and (b) loss curves using Chebyshev Spectral Graph Convolution (CHEB).
Buildings 13 02504 g0a3

Appendix B. Confusion Matrices for GNN Models

Figure A4 documents the confusion matrices for the three test sets.
Figure A4. Confusion matrices for the test sets.
Figure A4. Confusion matrices for the test sets.
Buildings 13 02504 g0a4

Appendix C. Detailed Results of PATH Tile Runs

Table A1. Detailed results of PATH tile runs.
Table A1. Detailed results of PATH tile runs.
Run No.Detection Results after Blister
PATH Tile Implementation
Run No.Detection Results after Guidance
PATH Tile Implementation
11
22Detected as ‘0.5 m distance’ tile
3Detected as ‘2 m distance’ tile3
44
5Detected as ‘level change’ tile5
66
77Detected as ‘level change’ tile
88
9Detected as ‘static obstacles’ tile9
10Detected as ‘2 m distance’ tile10
1111
1212
13Detected as ‘level change’ tile13
1414
1515
1616Detected as ‘2 m distance’ tile
17Detected as ‘1 m distance’ tile17
1818
1919
20Detected as ‘level change’ tile20
2121
2222Detected as ‘2 m distance’ tile
2323Detected as ‘null’ tile
2424
25Detected as ‘2 m distance’ tile25
2626
27Detected as ‘null’ tile27
2828Detected as ‘level change’ tile
29Detected as ‘level change’ tile29
3030Detected as ‘static obstacle’ tile

References

  1. Ciupe, V.; Maniu, I. New trends in service robotics. In New Trends in Medical and Service Robots: Theory and Integrated Applications; Springer: Berlin/Heidelberg, Germany, 2014; pp. 57–74. [Google Scholar]
  2. Hajduk, M.; Koukolová, L. Trends in industrial and service robot application. Appl. Mech. Mater. 2015, 791, 161–165. [Google Scholar] [CrossRef]
  3. Sun, Y.; Wang, R. The research framework and evolution of service robots. J. Comput. Inf. Syst. 2022, 62, 598–608. [Google Scholar] [CrossRef]
  4. Trevelyan, J.; Hamel, W.R.; Kang, S.C. Robotics in hazardous applications. In Springer Handbook of Robotics; Springer: Berlin/Heidelberg, Germany, 2016; pp. 1521–1548. [Google Scholar]
  5. Pey, J.J.J.; Povendhan, A.P.; Pathmakumar, T.; Elara, M.R. Robot-aided Microbial Density Estimation and Mapping. In Proceedings of the 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan, 23–27 October 2022; pp. 2265–2272. [Google Scholar]
  6. Wirtz, J.; Kunz, W.; Paluch, S. The service revolution, intelligent automation and service robots. Eur. Bus. Rev. 2021, 29, 909. [Google Scholar]
  7. Yörük, T.; Akar, N.; Özmen, N.V. Research trends on guest experience with service robots in the hospitality industry: A bibliometric analysis. Eur. J. Innov. Manag. 2023; ahead of print. [Google Scholar]
  8. Shukla, M.; Shukla, A.N. Growth of robotics industry early in 21st century. Int. J. Comput. Eng. Res. 2012, 2, 1554–1558. [Google Scholar]
  9. Qureshi, M.O.; Syed, R.S. The impact of robotics on employment and motivation of employees in the service sector, with special reference to health care. Saf. Health Work. 2014, 5, 198–202. [Google Scholar] [CrossRef]
  10. Yang, J.; Wang, C.; Jiang, B.; Song, H.; Meng, Q. Visual perception enabled industry intelligence: State of the art, challenges and prospects. IEEE Trans. Ind. Informatics 2020, 17, 2204–2219. [Google Scholar] [CrossRef]
  11. Hua, C. A Review of Visual Perception for Mobile Robot Navigation: Methods, Limitation, and Applications. In Proceedings of the 2022 2nd International Conference on Algorithms, High Performance Computing and Artificial Intelligence (AHPCAI), Guangzhou, China, 21–23 October 2022; pp. 729–737. [Google Scholar]
  12. Morar, A.; Moldoveanu, A.; Mocanu, I.; Moldoveanu, F.; Radoi, I.E.; Asavei, V.; Gradinaru, A.; Butean, A. A comprehensive survey of indoor localization methods based on computer vision. Sensors 2020, 20, 2641. [Google Scholar] [CrossRef]
  13. Dey, S.; Mukherjee, A. Robotic slam: A review from fog computing and mobile edge computing perspective. In Proceedings of the Adjunct Proceedings of the 13th International Conference on Mobile and Ubiquitous Systems: Computing Networking and Services, Hiroshima, Japan, 28 November–1 December 2016; pp. 153–158. [Google Scholar]
  14. Filliat, D.; Meyer, J.A. Map-based navigation in mobile robots:: I. a review of localization strategies. Cogn. Syst. Res. 2003, 4, 243–282. [Google Scholar] [CrossRef]
  15. Lim, H.; Lee, Y.S. Real-time single camera SLAM using fiducial markers. In Proceedings of the 2009 ICCAS-SICE, Fukuoka City, Japan, 18–21 August 2009; pp. 177–182. [Google Scholar]
  16. Figat, J.; Kasprzak, W. NAO-mark vs QR-code Recognition by NAO Robot Vision. In Progress in Automation, Robotics and Measuring Techniques; Springer: Berlin/Heidelberg, Germany, 2015; pp. 55–64. [Google Scholar]
  17. Shen, L.; Stopher, P.R. Review of GPS travel survey and GPS data-processing methods. Transp. Rev. 2014, 34, 316–334. [Google Scholar] [CrossRef]
  18. Panchpor, A.A.; Shue, S.; Conrad, J.M. A survey of methods for mobile robot localization and mapping in dynamic indoor environments. In Proceedings of the 2018 Conference on Signal Processing And Communication Engineering Systems (SPACES), Vijayawada, India, 4–5 January 2018; pp. 138–144. [Google Scholar]
  19. Skrzypczyński, P. Mobile robot localization: Where we are and what are the challenges? In Automation 2017: Innovations in Automation, Robotics and Measurement Techniques 1; Springer: Berlin/Heidelberg, Germany, 2017; pp. 249–267. [Google Scholar]
  20. Hernández, A.C.; Gómez, C.; Crespo, J.; Barber, R. Object detection applied to indoor environments for mobile robot navigation. Sensors 2016, 16, 1180. [Google Scholar] [CrossRef]
  21. Argall, B.D.; Billard, A.G. A survey of Tactile Human–Robot Interactions. Robot. Auton. Syst. 2010, 58, 1159–1176. [Google Scholar] [CrossRef]
  22. Luo, S.; Bimbo, J.; Dahiya, R.; Liu, H. Robotic tactile perception of object properties: A review. Mechatronics 2017, 48, 54–67. [Google Scholar] [CrossRef]
  23. Lee, M.H.; Nicholls, H.R. Review Article Tactile sensing for mechatronics—A state of the art survey. Mechatronics 1999, 9, 1–31. [Google Scholar] [CrossRef]
  24. Lee, J.T.; Bollegala, D.; Luo, S. “Touching to see” and “seeing to feel”: Robotic cross-modal sensory data generation for visual-tactile perception. In Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada, 20–24 May 2019; pp. 4276–4282. [Google Scholar]
  25. Alatise, M.B.; Hancke, G.P. A Review on Challenges of Autonomous Mobile Robot and Sensor Fusion Methods. IEEE Access 2020, 8, 39830–39846. [Google Scholar] [CrossRef]
  26. Taddeucci, D.; Laschi, C.; Lazzarini, R.; Magni, R.; Dario, P.; Starita, A. An approach to integrated tactile perception. In Proceedings of the International Conference on Robotics and Automation, Albuquerque, NM, USA, 20–25 April 1997; Volume 4, pp. 3100–3105. [Google Scholar]
  27. Nicholls, H.R.; Lee, M.H. A survey of robot tactile sensing technology. Int. J. Robot. Res. 1989, 8, 3–30. [Google Scholar] [CrossRef]
  28. Dargahi, J.; Najarian, S. Advances in tactile sensors design/manufacturing and its impact on robotics applications—A review. Ind. Robot. Int. J. 2005, 32, 268–281. [Google Scholar] [CrossRef]
  29. Lee, M.H. Tactile sensing: New directions, new challenges. Int. J. Robot. Res. 2000, 19, 636–643. [Google Scholar] [CrossRef]
  30. Dahiya, R.S.; Valle, M.; Postolache, O.; Pereira, J.M.D. Tactile sensing for robotic applications. Sens. Focus Tactile Force Stress Sens. 2008, 298–304. [Google Scholar]
  31. Chi, C.; Sun, X.; Xue, N.; Li, T.; Liu, C. Recent progress in technologies for tactile sensors. Sensors 2018, 18, 948. [Google Scholar] [CrossRef]
  32. Zou, L.; Ge, C.; Wang, Z.J.; Cretu, E.; Li, X. Novel tactile sensor technology and smart tactile sensing systems: A review. Sensors 2017, 17, 2653. [Google Scholar] [CrossRef]
  33. Rosa, M.P.; de Mello, G.S.; Morato, S. Tactile paving surfaces at bus stops. J. Access. Des. All 2021, 11, 259–294. [Google Scholar]
  34. Sakaguchi, R.; Takasu, S.; Akiyama, T. Study concerning the colours of tactile blocks for the visually handicapped-visibility for the visually handicapped and scenic congruence for those with ordinary sight and vision. In Proceedings of the JIPEA World Congress, Tohoku, Japan, 28 May–10 June 2000; pp. 453–462. [Google Scholar]
  35. ISO. Assistive Products for Blind and Vision-Impaired Persons; ISO: Geneva, Switzerland, 2012. [Google Scholar]
  36. CEN/TS 15209:2008; Tactile Paving Surface Indicators Produced from Concrete, Clay and Stone. European Committee for Standardization (CEN): Brussels, Belgium, 2013.
  37. Division, D.B.C. Tactile Poster v0.6—Land Transport Authority (LTA). 2015. Available online: https://www.lta.gov.sg/content/dam/ltagov/industry_innovations/industry_matters/development_construction_resources/quick_guide_series_for_development_related_proposals/TactileTilesPosterV0.6.pdf (accessed on 8 September 2023).
  38. Song, S.K. Singapore Government Circular. 2006. Available online: https://www.corenet.gov.sg/einfo/Uploads/Circulars/CLTA060921.pdf (accessed on 4 June 2023).
  39. Standards New Zealand, S.A. Australian/New Zealand Standard. 2002. Available online: https://www.saiglobal.com/pdftemp/previews/osh/as/as1000/1400/n14284.pdf (accessed on 7 August 2023).
  40. BSI. BS 7997:2003. 2003. Available online: https://www.roadstone.ie/wp-content/uploads/2014/03/BS-7997-tactile-std.pdf (accessed on 6 August 2023).
  41. Emerson, R.W. Tactile Walking Surface Indicators in the United States and Internationally. In Research, Standards, Guidance, and Practice; Western Michigan University: Kalamazoo, MI, USA, 2021; Available online: https://www.pedbikeinfo.org/cms/downloads/TWSIreview-Bentzen-NIDILRR.pdf (accessed on 6 August 2023).
  42. Yamauch, S. 2003. Available online: http://www.rehab.go.jp/english/whoclbc/pdf/E13.pdf (accessed on 23 July 2023).
  43. Lu, J.; Siu, K.W.M.; Xu, P. A comparative study of tactile paving design standards in different countries. In Proceedings of the 2008 9th International Conference on Computer-Aided Industrial Design and Conceptual Design, Kunming, China, 22–25 November 2008; pp. 753–758. [Google Scholar]
  44. Vstedal, L.R.; Lindland, T.; Lid, I.M. On our way establishing national guidelines on tactile surface indicators. In Proceedings of the International Congress Series; Elsevier: Amsterdam, The Netherlands, 2005; Volume 1282, pp. 1046–1050. [Google Scholar]
  45. Deichmann, J. Directional tactile pavings in a universal design perspective. In Universal Design 2016: Learning from the Past, Designing for the Future; IOS Press: Amsterdam, The Netherlands, 2016; pp. 594–600. [Google Scholar]
  46. Pavlos, E.; Sanford, J.A. Detectable Tactile Surface Treatments; Technical report; Georgia Institute of Technology: Atlanta, GA, USA, 1983. [Google Scholar]
  47. Heuten, W.; Henze, N.; Boll, S.; Pielot, M. Tactile wayfinder: A non-visual support system for wayfinding. In Proceedings of the 5th Nordic Conference on Human-Computer Interaction: Building Bridges, Lund, Sweden, 20–22 October 2008; pp. 172–181. [Google Scholar]
  48. Department of the Environment. The Regions. 2000. Available online: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/918353/tactile-paving-surfaces.pdf (accessed on 19 May 2023).
  49. Courtney, A.; Chow, H. A study of tile design for tactile guide pathways. Int. J. Ind. Ergon. 2000, 25, 693–698. [Google Scholar] [CrossRef]
  50. Mizuno, T.; Nishidate, A.; Tokuda, K.; Kunijiro, A. Installation errors and corrections in tactile ground surface indicators in Europe, America, Oceania and Asia. IATSS Res. 2008, 32, 68–80. [Google Scholar] [CrossRef]
  51. Kang, B.K.; Shin, D.H.; Park, K.J.; Kim, S.W. A Study on Improvement of Installation Standards for Tactile Walking Surface Indicators. J. Korea Inst. Healthc. Archit. 2015, 21, 57–66. [Google Scholar] [CrossRef]
  52. Padzi, F.A.; Ibrahim, F.; Karim, N.A. Incongruent installation of tactile ground surface indicator toward visual impaired people’s need: Masjid Jamek Station. Procedia-Soc. Behav. Sci. 2013, 101, 130–139. [Google Scholar] [CrossRef]
  53. Design, N.S. Integrated Building Systems. 2013. Available online: https://research.design.ncsu.edu/building-systems-integration/building-systems-integration/ (accessed on 19 May 2023).
  54. Dowson, A. The development of surface tactile indicators. In Proceedings of the 7th International Conference on Concrete Block Paving, Orlando, FL, USA, 9–13 September 2001; pp. 1–10. [Google Scholar]
  55. Samuel, D.L.; Nagendra, S.S.; Maiya, M. Passive alternatives to mechanical air conditioning of building: A review. Build. Environ. 2013, 66, 54–64. [Google Scholar] [CrossRef]
  56. Kamal, M.A. An overview of passive cooling techniques in buildings: Design concepts and architectural interventions. Acta Tech. Napoc. Civ. Eng. Archit. 2012, 55, 84–97. [Google Scholar]
  57. Elara, M.R.; Rojas, N.; Chua, A. Design principles for robot inclusive spaces: A case study with roomba. In Proceedings of the 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China, 31 May–7 June 2014; pp. 5593–5599. [Google Scholar]
  58. Tan, N.; Mohan, R.E.; Watanabe, A. Toward a framework for robot-inclusive environments. Autom. Constr. 2016, 69, 68–78. [Google Scholar] [CrossRef]
  59. Yeo, M.S.K.; Samarakoon, S.M.B.P.; Ng, Q.B.; Muthugala, M.A.V.J.; Elara, M.R. Design of Robot-Inclusive Vertical Green Landscape. Buildings 2021, 11, 203. [Google Scholar] [CrossRef]
  60. Wilcox, B.; Dankowicz, H. Limit-Switch Sensor Functionality Based on Discontinuity-Induced Nonlinearities. J. Comput. Nonlinear Dyn. 2011, 6, 031004. [Google Scholar] [CrossRef]
  61. Wu, Y.; Lian, D.; Xu, Y.; Wu, L.; Chen, E. Graph convolutional networks with markov random field reasoning for social spammer detection. In Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA, 7–12 February 2020; Volume 34, pp. 1054–1061. [Google Scholar]
  62. Hamaguchi, T.; Oiwa, H.; Shimbo, M.; Matsumoto, Y. Knowledge Transfer for Out-of-Knowledge-Base Entities: A Graph Neural Network Approach. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17, Melbourne, VIC, Australia, 19–25 August 2017; pp. 1802–1808. [Google Scholar] [CrossRef]
  63. Jiang, W.; Luo, J. Graph Neural Network for Traffic Forecasting: A Survey. Expert Syst. Appl. 2022, 207, 117921. [Google Scholar] [CrossRef]
  64. Wu, Z.; Tang, Y.; Hong, B.; Liang, B.; Liu, Y. Enhanced Precision in Dam Crack Width Measurement: Leveraging Advanced Lightweight Network Identification for Pixel-Level Accuracy. Int. J. Intell. Syst. 2023, 2023, 9940881. [Google Scholar] [CrossRef]
  65. Tang, Y.; Huang, Z.; Chen, Z.; Chen, M.; Zhou, H.; Zhang, H.; Sun, J. Novel visual crack width measurement based on backbone double-scale features for improved detection automation. Eng. Struct. 2023, 274, 115158. [Google Scholar] [CrossRef]
  66. Cao, W.; Liu, Q.; He, Z. Review of pavement defect detection methods. IEEE Access 2020, 8, 14531–14544. [Google Scholar] [CrossRef]
  67. Zhou, J.; Cui, G.; Hu, S.; Zhang, Z.; Yang, C.; Liu, Z.; Wang, L.; Li, C.; Sun, M. Graph neural networks: A review of methods and applications. AI Open 2020, 1, 57–81. [Google Scholar] [CrossRef]
  68. Xiao, S.; Wang, S.; Dai, Y.; Guo, W. Graph Neural Networks in Node Classification: Survey and Evaluation. Mach. Vis. Appl. 2022, 33, 1–19. [Google Scholar] [CrossRef]
  69. Kipf, T.N.; Welling, M. Semi-Supervised Classification with Graph Convolutional Networks. arXiv 2016, arXiv:1609.02907. [Google Scholar]
  70. Veličković, P.; Cucurull, G.; Casanova, A.; Romero, A.; Liò, P.; Bengio, Y. Graph Attention Networks. arXiv 2017, arXiv:1710.10903. [Google Scholar]
  71. Defferrard, M.; Bresson, X.; Vandergheynst, P. Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering. arXiv 2016, arXiv:1606.09375. [Google Scholar]
  72. Fey, M.; Lenssen, J.E. Fast Graph Representation Learning with PyTorch Geometric. In Proceedings of the ICLR Workshop on Representation Learning on Graphs and Manifolds, Room R7, Ernest N. Morial Convention Center, New Orleans, LA, USA, 6 May 2019; pp. 1–9. [Google Scholar]
  73. Zhao, J.; Xu, Y.; Zhu, W.; Liu, M.; Zhao, J. Real-Time Early Safety Warning for Personnel Intrusion Behavior on Construction Sites Using a CNN Model. Buildings 2023, 13, 2206. [Google Scholar] [CrossRef]
  74. Nešković, J.; Jovanović, I.; Markov, S.; Vučetić, S.; Ranogajec, J.; Trumić, M. Bio-Induced Healing of Cement Mortars in Demineralized and Danube Water: CNN Model for Image Classification. Buildings 2023, 13, 1751. [Google Scholar] [CrossRef]
  75. Lung, L.W.; Wang, Y.R. Applying Deep Learning and Single Shot Detection in Construction Site Image Recognition. Buildings 2023, 13, 1074. [Google Scholar] [CrossRef]
  76. Michalos, G.; Makris, S.; Tsarouchi, P.; Guasch, T.; Kontovrakis, D.; Chryssolouris, G. Design considerations for safe human-robot collaborative workplaces. Procedia CIrP 2015, 37, 248–253. [Google Scholar] [CrossRef]
Figure 1. Common types of tactile paving tiles in existence.
Figure 1. Common types of tactile paving tiles in existence.
Buildings 13 02504 g001
Figure 2. Examples of existing tactile paving used in combination to denote turns, path bifurcations, or areas of interest.
Figure 2. Examples of existing tactile paving used in combination to denote turns, path bifurcations, or areas of interest.
Buildings 13 02504 g002
Figure 3. Top: Guidance, distance PATH tiles indicating varying safety distances from hazards; Bottom: hazard PATH tiles indicating varying hazard types.
Figure 3. Top: Guidance, distance PATH tiles indicating varying safety distances from hazards; Bottom: hazard PATH tiles indicating varying hazard types.
Buildings 13 02504 g003
Figure 4. Sample of 3D-printed PATH tiles for testing. Left: blister PATH tile; Right: guidance PATH tile.
Figure 4. Sample of 3D-printed PATH tiles for testing. Left: blister PATH tile; Right: guidance PATH tile.
Buildings 13 02504 g004
Figure 5. Left: Meerkat, autonomous mobile robot developed in house for hazard auditing within built environments.
Figure 5. Left: Meerkat, autonomous mobile robot developed in house for hazard auditing within built environments.
Buildings 13 02504 g005
Figure 6. System architecture of Meerkat robot with the integration of the proposed novelties highlighted in red.
Figure 6. System architecture of Meerkat robot with the integration of the proposed novelties highlighted in red.
Buildings 13 02504 g006
Figure 7. In-house Tactile Sensor Module and its components using KH-9015-HRL limit switches.
Figure 7. In-house Tactile Sensor Module and its components using KH-9015-HRL limit switches.
Buildings 13 02504 g007
Figure 8. Tactile Sensor Module mounted onto the front of the Meerkat robot.
Figure 8. Tactile Sensor Module mounted onto the front of the Meerkat robot.
Buildings 13 02504 g008
Figure 9. Side view of limit switch changes when moving over PATH tile contours; the limit switch is being depressed in the center image upon encountering a tactile bump of a blister PATH tile.
Figure 9. Side view of limit switch changes when moving over PATH tile contours; the limit switch is being depressed in the center image upon encountering a tactile bump of a blister PATH tile.
Buildings 13 02504 g009
Figure 10. Zoomed-in front view of Tactile Sensor Module moving over a blister PATH tile, with the different limit switches’ states being dependent on the contour patterns of the blister PATH tile.
Figure 10. Zoomed-in front view of Tactile Sensor Module moving over a blister PATH tile, with the different limit switches’ states being dependent on the contour patterns of the blister PATH tile.
Buildings 13 02504 g010
Figure 11. Novel approach of using GNN in PATH tile classification.
Figure 11. Novel approach of using GNN in PATH tile classification.
Buildings 13 02504 g011
Figure 12. Overview of graph modeling process of PATH tile patterns using limit switches.
Figure 12. Overview of graph modeling process of PATH tile patterns using limit switches.
Buildings 13 02504 g012
Figure 13. Ground-truth representations of the 8 PATH tile patterns: (a) guidance, (b) 0.5 m from hazard, (c) 1 m from hazard, (d) 2 m from hazard, (e) blister, (f) ramp/slope, (g) offset blister, (h) static obstacle. (i) Example of a null result (patterns of 10 or fewer ON signals in a single input have been added to cater for inputs that do not fall within these 8 categories).
Figure 13. Ground-truth representations of the 8 PATH tile patterns: (a) guidance, (b) 0.5 m from hazard, (c) 1 m from hazard, (d) 2 m from hazard, (e) blister, (f) ramp/slope, (g) offset blister, (h) static obstacle. (i) Example of a null result (patterns of 10 or fewer ON signals in a single input have been added to cater for inputs that do not fall within these 8 categories).
Buildings 13 02504 g013
Figure 14. Pipeline to obtain trained weights for GCN, GAT, and CHEB during model training.
Figure 14. Pipeline to obtain trained weights for GCN, GAT, and CHEB during model training.
Buildings 13 02504 g014
Figure 15. Experimental setup. Left: blister tile; right: guidance tile. A metal plate was placed in front of the tiles to make the floor surface to be flush with the PATH tiles.
Figure 15. Experimental setup. Left: blister tile; right: guidance tile. A metal plate was placed in front of the tiles to make the floor surface to be flush with the PATH tiles.
Buildings 13 02504 g015
Table 1. Typical visual localization methods used by existing mobile robot platforms.
Table 1. Typical visual localization methods used by existing mobile robot platforms.
Localization MethodDescriptionLimitations
SLAM [13]Use of LIDAR sensors to generate
two-dimensional maps of robot and
its surroundings
-
Limited to extent of sensor operational capabilities;
-
Compounded sensor errors over time;
-
Computational resources increase drastically as area being mapped increases.
Robot beacon array [14]Usage of Wifi/RFID and other
wireless technologies to help robot
to triangulate its location within
a given space
-
Effective range limited to extent of beacon placement;
-
Computational resources increase drastically as more beacons are implemented in workspace.
Fiducial markers [15]Visual (usually monochromatic)
markers placed at strategic locations
for robot to detect and localize
-
Require line of sight;
-
Affected by lighting conditions.
Table 2. Categories of robotic indoor hazards and robots’ responses upon encounter.
Table 2. Categories of robotic indoor hazards and robots’ responses upon encounter.
Spatial Hazards to RobotsExamplesCommon Responses by Robots
Static obstacles
-
Doors;
-
Walls;
-
Interior furniture;
-
Dead-ends.
-
Slow down;
-
Reverse;
-
Pause to adjust.
Moving/transient obstacles
-
Human traffic;
-
Vehicle traffic.
-
Slow down;
-
Stop.
Level changes/inclines
-
Stairs;/escalators;
-
Ramps/slopes;
-
Elevator entrances.
-
Stop;
-
Reverse;
-
Turn left/right;
-
Pause to adjust.
Narrow passage
-
Gaps between furniture items;
-
Corridors;
-
Choke points;
-
Walkways.
-
Slow down;
-
Pause to adjust;
-
Reverse;
-
Turn left/right.
Sharp turns
-
Path junctions;
-
Corners;
-
Path bifurcations.
-
Slow down;
-
Pause to adjust;
-
Turn left/right.
Overhangs
-
Table edges;
-
Railings;
-
Spatial barriers.
-
Reverse;
-
Turn left/right;
-
Stop.
Table 3. Summary of existing tactile paving types.
Table 3. Summary of existing tactile paving types.
Tactile Indicator TypeTypical LocationsUsage/Function
Blister pavingBefore crossing interfacesAlerting VIB users of intersection
ahead and to proceed with care
Offset blister pavingNear level change of
railway platform edge
Warning VIB users of
platform edges/level drop
Corduroy hazard
warning paving
Near obstacles or level
changes
Warning VIB users of hazards
ahead, to proceed with care
Lozenge pavingOn-street platform edgesWarning VIB users of
platform edge of light
transport systems
Cycleway pavingBeginning and end of
cycleway and pedestrian
intersections
Alerting VIB users of
pedestrian pathways and
cycleway paths
Guidance/
directional paving
Safe route around obstaclesIdentifying safe route for VIB
users, providing directional
cues, and avoiding obstacles
Table 4. Results of PATH tile runs, details in Appendix C Table A1.
Table 4. Results of PATH tile runs, details in Appendix C Table A1.
Detection in 30 RunsCorrect DetectionError Detection
Blister PATH tile2010
Guidance PATH tile237
Table 5. Best training and validation accuracy and loss of GNN models, the total training epochs completed before early stopping, and the total time required.
Table 5. Best training and validation accuracy and loss of GNN models, the total training epochs completed before early stopping, and the total time required.
ModelTrain AccVal AccTrain LossVal LossEpochsTime/s
GCN84.20%83.86%1.54641.44561732236.44
GAT71.33%71.42%1.67251.48862156663.33
CHEB91.94%91.34%1.46821.4235239093.94
Table 6. Hyper-parameters used for tuning.
Table 6. Hyper-parameters used for tuning.
Hyper-ParameterValuesTune TypeBest Value
Hidden channels16, 32, 64, 128Choice128
Dropout rate0 to 0.5Uniform2.4260 × 10 1
Learning rate1 × 10 4 to 1 × 10 1 Uniform1.8055 × 10 2
Weight decay1 × 10 6 to 1 × 10 3 Uniform3.2048 × 10 6
Table 7. Training and validation accuracy and loss values of the CHEB model prior to and after tuning.
Table 7. Training and validation accuracy and loss values of the CHEB model prior to and after tuning.
Train AccVal AccTrain LossVal LossEpochsTime/s
Before91.94%91.34%1.46821.4235239093.94
After96.07%95.12%1.41201.4033149063.12
Table 8. Metrics of the normalized confusion matrix from the tuned CHEB model on the test sets.
Table 8. Metrics of the normalized confusion matrix from the tuned CHEB model on the test sets.
Test SetVal AccPrecisionRecallSpecificityF1 Score
196.58%0.96600.96580.96580.9659
296.72%0.96740.96720.96720.9671
396.76%0.96780.96760.96760.9676
Average96.69%0.96710.96690.96690.9669
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Yeo, M.S.K.; Pey, J.J.J.; Elara, M.R. Passive Auto-Tactile Heuristic (PATH) Tiles: Novel Robot-Inclusive Tactile Paving Hazard Alert System. Buildings 2023, 13, 2504. https://doi.org/10.3390/buildings13102504

AMA Style

Yeo MSK, Pey JJJ, Elara MR. Passive Auto-Tactile Heuristic (PATH) Tiles: Novel Robot-Inclusive Tactile Paving Hazard Alert System. Buildings. 2023; 13(10):2504. https://doi.org/10.3390/buildings13102504

Chicago/Turabian Style

Yeo, Matthew S. K., Javier J. J. Pey, and Mohan Rajesh Elara. 2023. "Passive Auto-Tactile Heuristic (PATH) Tiles: Novel Robot-Inclusive Tactile Paving Hazard Alert System" Buildings 13, no. 10: 2504. https://doi.org/10.3390/buildings13102504

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop