RoboEye, an Efficient, Reliable and Safe Semi-Autonomous Gaze Driven Wheelchair for Domestic Use

Abstract

Any severe motor disability is a condition that limits the ability to interact with the environment, even the domestic one, caused by the loss of control over one’s mobility. This work presents RoboEYE, a power wheelchair designed to allow users to move easily and autonomously within their homes. To achieve this goal, an innovative, cost-effective and user-friendly control system was designed, in which a non-invasive eye tracker, a monitor, and a 3D camera represent some of the core elements. RoboEYE integrates functionalities from the mobile robotics field into a standard power wheelchair, with the main advantage of providing the user with two driving options and comfortable navigation. The most intuitive and direct modality foresees the continuous control of frontal and angular wheelchair velocities by gazing at different areas of the monitor. The second, semi-autonomous modality allows navigation toward a selected point in the environment by just pointing and activating the wished destination while the system autonomously plans and follows the trajectory that brings the wheelchair to that point. The purpose of this work was to develop the control structure and driving interface designs of the aforementioned driving modalities taking into account also uncertainties in gaze detection and other sources of uncertainty related to the components to ensure user safety. Furthermore, the driving modalities, in particular the semi-autonomous one, were modeled and qualified through numerical simulations and experimental verification by testing volunteers, who are regular users of standard electric wheelchairs, to verify the efficiency, reliability and safety of the proposed system for domestic use. RoboEYE resulted suitable for environments with narrow passages wider than 1 m, which is comparable with a standard domestic door and due to its properties with large commercialization potential.

1. Introduction

The drive-by mean of eyes, gazing at a monitor or any equivalent interface, is an open research topic studied since the early 2000s. Nevertheless, there is limited technology transfer through commercial solutions for potential daily users—persons affected by severe motor disabilities. The literature also shows a similar trend, with a limited number of references on autonomous mobility topics through assistive solutions.

Mainly due to the few technological options available in the past, such limitations are recently being reduced thanks to innovative and more commercial hardware and software solutions developed by both established and new manufactures [1]. Contactless devices represent one of the most important examples of such improvements, allowing gaze tracking without any other body-mounted tools. During the last years were presented on the market a lot of new contactless eye tracking models [2], an index of how this technology is promisingly growing.

This work addresses the problem of implementing a reliable and efficient driving strategy based on eye-tracking starting from a standard power wheelchair. That involved the developed custom hardware and software solutions derived from the mobile robotic fields, resulting in a semi-autonomous system able to interact with the user smoothly and comfortably. The reconfiguration of power wheelchairs, already used and known to the final user offer a larger possibility of commercialization. For this advantage there are other works that aim to modify the existing structures of wheelchairs and in-sightseeing strollers, adapting them to the individual needs of the user and his individual degree of disability. For example, [3,4] worked on a hybrid assisted wheelchair for its propulsion and [5] about wheelchair parallel control trough electroencephalogram signals or [6] that investigated how the center of gravity of the human body affects the performance parameters of a wheelchair.

The aim of our work was to provide an additive module to the wheelchair in order to make users with severe motor disabilities autonomous in movement. We developed two driving modalities taking into account the uncertainties of the user’s gaze and the implemented sensors. In particular, the semi-autonomous modality was tested through numerical simulations and experimental verification for the passage of the wheelchair through a door. The results demonstrated that narrow passages can be easily and safely overcome, and which parameters are more effective than others in reaching a target point.

The paper was organized as follows:

• The first part reports the state-of-the-art references on the eye-driving topic, used in this work as initial guidelines for the system developed.
• Subsequently, the sections present the original design choices in integrating eye-tracking technology on a power wheelchair, the automation and functionalities included, and the driving strategies and the motion performance of the system, both evaluated through numerical simulations and experimental tests.
• The final part describes the modalities followed in the experimental campaign, the results obtained and the related operational outcomes.

RoboEYE combines two technologies: an eye tracker and a power wheelchair. Considered individually, each of these technologies was well researched in the past, but, in contrast, the integration of the two represents a topic with a limited number of references in the literature, [7,8,9] are examples. The results of these studies suggest that eye-gaze control may be a solution for wheelchair navigation for users with severe motor disabilities.

Other studies have integrated additional input mechanisms, such as Brain–Computer Interfaces [10,11] or soft-switch [12], to increase the sensitivity and accuracy of gaze control.

However, our tests prove that it may not be necessary to integrate other functional inputs, this aided by the semi-autonomous driving modality for more difficult operations.

1.1. Eye Tracking

Eye-tracking technology provides a measurement of the gazed point, or area, on a monitor. More than the medical and commercial applications [13,14], this technology has improved to a maturity suitable for its application as an effective human–machine interface (HMI), exploiting the fact that the eye has one of the fastest human movements, although mainly conceived for exploration and less for control.

As the main advantage, the usage of eyes as input [15] allows those users who, due to disease or physiological status, cannot use standard interfaces such as a joystick or a keyboard, to interact with other people or the environment quite efficiently. The main disadvantage affecting this technology is the sensibility to various factors such as light condition, the color of the iris, head movement [16,17]. In this context, [18,19,20] analyzed these elements and provided a more detailed characterization of the measurement process together with a method to assess and compensate [21,22] for uncertainty in eye tracking. The latter element was considered in the proposed solution, and it proved to be a key element for the smooth control of the wheelchair using a commercial, non-invasive, eye tracker.

1.2. Eye Driven Wheelchairs

As for wheelchair systems that exploited an eye-tracking technology as a driving solution, [23] represents the first real attempt: a glass-mounted camera was used to track eye movement from a short distance. This approach, like many others based on different technologies [24], foresaw the use of wearable technology, the only one available in the past. However, these can only provide a limited number of commands, can prove tiring [25], and therefore not suitable for prolonged use.

As for non-wearable (and less invasive) technology, [26] proposed a solution based on a webcam and gaze direction analysis. The solution is interesting but both the camera position (almost in front of the user’s face) and the lack of a video interface could lead to poor management of the system and to complex driving practice. Other works, such as [27], used a rather expensive eye-tracking system; definitely more promising and probably more efficient than the previous one, although the high cost considerably reduces its affordability for private users.

With regard to the use of these systems, [28] has indeed shown that drive-by mean of eyes achieves better performance than standard control techniques and interfaces. As well as [29,30], in which the interaction with the eye tracker was included in a simulated environment, achieving a safer training environment than a real space, but also useful for optimizing HMI design and navigation modalities.

A comparison [31] between a screen-based interface and an interface without the use of a digital screen [32,33] mainly highlighted the limitations and problems related to the feedback provided to users with the latter.

A work similar to ours is proposed in [34], the main contribution of our paper comes from the inclusion of the uncertainty information about the gaze position in the human–machine interaction and therefore the creation of a smart HMI able to adapt to this parameter. In addition, fatigue and safety issues during driving were considered, proposing as a solution a semi-autonomous navigation modality that allows the system to safely navigate through narrow passages. Last but not least, the design of the system considered the cost-effectiveness of the technology, its ability to be configured, customized and then transferred to the end-user.

2. Eye Tracking as Driving Interface

A good driving interface should achieve two operational conditions: to minimize user stress resulting from the intensive use of the system and, at the same time, maximize driving performances and comfort of motion. The interaction with the technology should result intuitive for the user and possibly take into account the metrological performances of the sensing interface, in this case, the eye tracker, and the physiological characteristics of the users.

These two points, in the form of uncertainties in the measurements and in the user’s model, are usually not considered in most common driving interfaces, i.e., a joystick, because the inputs provided to the system are very fast and accurate. In the case of eye-tracking, the uncertainty in the assessment of the gazed point is instead a critical element: its magnitude, which is not negligible, tends to decrease the accuracy of the interaction between the user and mechatronic system. This uncertainty, in the form of both random and systematic errors in the measurement of the gazed position, should be corrected before applying any control law for wheelchair navigation to improve the usability and intuitiveness of the system [22,29].

3. Materials and Methods

RoboEYE, shown in Figure 1a, was built from a GR558, a power wheelchair of Nuova Blandino Srl, reconfigured to include some advanced robotic functionalities. It included two driving modalities: direct and semi-autonomous. These were designed to provide to the user different driving experiences, resulting from the focus required, the feeling derived from the amount of control over motion, and the level of safety provided by the system about the environment. The user at any time can switch from one modality to the other from an idle state.

3.1. Direct Drive

The direct drive modality can be selected by gazing at the corresponding button for 2 s on the designed interface. It should provide a versatile maneuverability, with reduced jerks, and a comfortable overall experience [29]. The proposed solution tackles such assumption by foreseeing the continuous variation of frontal and angular velocities with two different control laws: a rational one for the forward direction and a linear one for the rotations. In particular, the speed values increase moving the gaze from the central bottom part of the screen to the external corners, where it reaches the maximum allowable speed values. For both, a rest zone is defined, in the central bottom part of the monitor, where the wheelchair does not move.

The width of the area is defined to be comparable to the maximum eye tracker uncertainty, 100 pixel in our setup [20].

3.2. Semi-Autonomous Drive

Although direct control is a very natural and effective way of controlling motion, it may cause fatigue if used for long periods due to the need to keep attention on the monitor. The drive modality used for the semi-autonomous navigation, from [35], solves the problem of fatigue by enabling the wheelchair to reach the selected target position autonomously. It foresees the use of visual markers and Augmented Reality (AR) to localize a specific Point of Interest (POI) in the domestic environment and then drive the wheelchair there. The POI detection is performed by searching in the RGB frame through the ArUco library, something similar to [36,37] who analyze and use the segmentation algorithms in video-surveillance systems and cluttered environments.

The semi-autonomous navigation technique can be described as follows:

• Detection of POIs in the Field of View (FoV) of the camera through ArUco markers (each POI is uniquely defined by a couple of markers).
• Localization of the wheelchair with respect to each single POI.
• Planning of the best path to reach each POI.
• Show to the user the result about the path related to the reachable POIs and wait the choice.
• Execution of the selected path.

A localization algorithm [35] returns the position and the attitude of all POIs detected in the frontal FoV and, the AR engine shows pins in correspondence of these plus a maneuver calculated by a path planning algorithm [38]. The user can select a POI by gazing at its pin for 2 s while a loading circle marks the selection in time. After that, the wheelchair performs the maneuver by exploiting the path-following task from [38]. At the end of the path, the wheelchair automatically stops allowing the user to select a new POI or switch back to direct ocular driving.

However, in the context of domestic navigation, a fully dynamic path generation and action could be dangerous from the presence of narrow passages or obstacles. For this reason, the planning tool foresaw the division of the maneuver into two sub-parts: (i) a path that brings the wheelchair nearby the POI, and (ii) a fixed conclusive maneuver, defined for each POI, thus ensuring a safe approach or passage toward the target position.

The first is determined dynamically by exploiting a clothoid [38,39], a curve characterized by a continuous third-order curvature that ensures continuity in the jerk and so an optimal comfort level in the navigation. On top of that, the curve can be expressed analytically [40,41], providing as advantages a simple data management for control, HMI and AR, but also it facilitates the identification of obstacles and thus their avoidance. In this way, it is possible to evaluate the feasibility of the planned path. As for the final maneuver, this was associated with a graph-based map of the environment containing all POIs’ information. Depending on the type of POI, passage (i.e., through a door) or approach point (i.e., for reaching a desk), a linear or curvilinear segment was defined considering the geometry of the enthronement, the position of the markers and POI.

Given a selected POI and the complete path, the HMI checks and notifies the presence of obstacles along it. If so, the HMI highlights the path in red, preventing the execution of the maneuver. In this case, it is asked to the user himself to perform a corrective movement and bring the wheelchair into a suitable condition for a safe maneuver. This strategy stems from the desire to emphasize the user’s decision rather than the development of a fully autonomous system.

3.3. Prototype Components

The default electronics of a GR558 power wheelchair were removed and replaced with a commercial driver to power the original motors and two encoders on the wheels for odometric localization, Figure 1.

A Time of Flight (ToF) Microsoft Kinect V2 camera was mounted on the front of the wheelchair, a few centimeters above the user’s legs. The position was identified by considering the advice from a pool of volunteer testers, regular users of standard and special powered wheelchairs. They highlighted the importance of seeing their knees inside the driving interface, resulting in an effective strategy for better depth perception from the video interface [35].

A monitor was mounted at the front to show the user the HMI and the information from the TOF about the environment. A Windows PC manages the system logic. It collects the data from the encoders, calculates the position of the wheelchair and the control parameters required to pilot the drivers. It also manages the eye tracker (fastened below the monitor), the HMI and the ToF.

The PC was configured with a real-time Windows Operating System (OS). Both the HMI and the hardware manager were developed in UNITY, and the communication between the OS and the driver was structured over RS232 serial communication.

3.4. Navigation

Advanced robotics functionalities, based on video and 3D data processing, represent a core element of the semi-autonomous navigation modality. These were derived from specific libraries, which could not be directly coded at the HMI level, and were therefore coded in a dedicated C++ DLL. It was organized in three levels with two parallel tasks performing absolute localization and driver communication. On top of this, a wrapper class connects the C++ level to the C# of the HMI. Figure 2 shows a general scheme.

Wheelchair localization is performed by merging two contributions: encoder odometry and the custom-designed absolute localization method based on TOF data.

As for the first, this was derived from the odometric recursion from [42]. As for the second, this was designed differently from the canonical ones, which usually rely on matching range data with a previously created map [43]. A vision-based solution foresees the use of AR functionalities [44,45] to locate the wheelchair with respect to the POIs without the use of a map. The main advantages are: (i) the simplification of the data structure, (ii) a lower computational cost and (iii) the possible use of low-cost sensors, this at the only cost of introducing visual tags, as spatial anchors, into the domestic environment.

RoboEYE was designed to work mainly in a domestic environment. In this context, the furniture is a reliable landmark, as it is unlikely to be moved frequently over time. The most interesting furniture from an operational point of view, such as doors, tables and so on, were then referenced using AR tags [46]. For each of them, a POI was defined as a target pose $[x,y,\theta ]$. An operational example is a television: the POI is defined with respect to the marker, placed nearby the device so that the user can reach with the semi-autonomous navigation modality a predefined position, and watch it most comfortably.

The steps involved in the localization process are shown in the Algorithm 1 and detailed here:

• Data acquisition: the information concerning the color image and the depth stream data are collected by the ToF camera and passed to the processing block.
• Kinect position assessment: the system determines the transformation, in terms of height and attitude, between the sensor and the ground using a RANdom SAmple Consensus plane fitting (RANSAC) [47]. This task is fundamental to compensate for the mobility of the camera on the chassis, especially in the attitude.
• Roto-translation of 3D points: depth frames (3D cloud points) are transformed from the ToF to the wheelchair reference system. This also allows the organization of a more versatile ad efficient AR framework.
• Target detection: an ArUco function analyzes the RGB frames searching for markers. If present, the algorithm evaluates their 3D positions by isolating the corresponding 3D points at the markers. This strategy, compared to the one based on standard vision, works without the knowledge of the camera’s intrinsic parameters. The main advantage is to avoid a calibration phase, otherwise necessary. The localized markers, and the corresponding POIs, are then passed to the interface as possible target and anchor positions for autonomous navigation.

Algorithm 1 Localization algorithm

whileKeepRunningdo     if CameraConnected then         Acquire RGB and depth frames         Calculate plane equation relative to the floor         Determine camera pose w.r.t. the wheelchair         Transform cloud points into the wheelchair reference system         Search ArUco marker in the RGB image         for EachMarkerDetected do            Determine pose of the POI related to each marker         end for         Send POIs detected information to the HMI     end if end while

The two concurrent localization modalities are merged to minimize both the influence of the slow ToF camera localization and the drift error of the odometric localization. A Bayesian-based approach was considered by [48] for that, resulting in very accurate navigation, when compared to the single localization options, and thus a safer system.

4. Sources of Uncertainty

The passage through a door, or narrow passage in general, is one of the most important tasks for the sake of autonomous domestic mobility. In the case of a semi-autonomous motion, the maneuver depends on the localization of the wheelchair with respect to the environment, which is a task mostly required to a single, properly organized device. In RoboEYE, the ToF fulfills such a role in combination with the POIs. Both ToF and POIs detection are subject to uncertainty.

4.1. Uncertainty of Human–Machine Interface

The front monitor displays the HMI, shown in Figure 3, and presents the navigation options to the user. They include a video stream from the camera in the background, buttons to start/stop navigation in the foreground, and, on top of all the detected POIs with identified the feasible paths.

The graphics engine considers the role of uncertainty from eye-tracking and its effect on human–machine interaction, proportionally adapting the size of buttons and pins according to the magnitude of such parameter. It was computed online, exploiting an exponential moving average of the gazed point on the monitor over time:

$${\overline{\mathsf{\Delta }p}}_{k+1}={\displaystyle \frac{(N-1)·{\overline{\mathsf{\Delta }p}}_k+\mathsf{\Delta }{p}_k}{N}}$$

(1)

${\overline{\mathsf{\Delta }p}}_k$ is the value of the exponential moving average at a step k, $\mathsf{\Delta }{P}_k$ is the difference of the gaze position in two consecutive steps and N is the number of samples considered in the moving average, set in the proposed application set to 10. A uniform weighting of $1/N$ was considered for all the samples in the average.

The presented organization achieved the main advantage of fast, reliable and structured access from high-level software interfaces to low-level variables, modules and functions, allowing the natural integration of kinematic information into the AR. The result was a natural, simple and efficient HMI resulting in a smooth and comfortable control of the motion of the wheelchair.

4.2. Uncertainty of ToF

The onboard position of the ToF device can be assessed with a limited accuracy since it has to be fixed at a critical point. From the specifications identified by the testers, to be effective in driving the ToF has to (i) be frontal to the system, (ii) point toward the floor to identify possible obstacles and (iii) allow easy repositioning to allow the user to sit in and get out of the wheelchair even with assistive equipment.

Such a setup implies possible small variations in the position of the device from the calibrated reference, caused by vibrations, slack in the mounting system, or mechanic tolerances. This can be modeled as uncertainty in such parameters, resulting in an error that propagates from the initial path planning, when the wheelchair is stationary, throughout the maneuver, where the path is automatically updated every time the target POI falls within the FoV of the ToF. The result is a non-negligible displacement of the wheelchair from the intended target position, which represents a dangerous condition for the user. For this reason, an error budget analysis was structured to highlight the influence of the uncertainty in the ToF position for the performance of the autonomous navigation toward a target POI.

4.3. Uncertainty of Marker-Based Localization

Any localization process exploited in (semi-)autonomous navigation should include, for safety reasons, an assessment of the uncertainty of the detected positions. In RoboEYE, this requirement was achieved through a repeatability analysis involving the measurement of static positions of multiple markers placed in the FoV of the ToF. The tests considered two conditions: distance and orientation. In the first, the marker and the wheelchair were aligned, moving the latter to different distances, Figure 4 reports the measurements as point-type markers. In the second, represented by the cross-type markers, the attitude of the wheelchair was changed with respect to the marker, keeping the position fixed.

The experimental evidence highlighted a correlation between the number of 3D points within the marker area and the covariance of the measure. Such observation was then exploited to determine the standard deviation of the $[x,y]$ position of the marker as a function of the number of points detected:

$$\sigma \left(n_p\right)={\displaystyle \frac{p_1·n_p+p_2}{n_p+q_1}}$$

(2)

$n_p$ is the number of 3D points detected within the marker, and $p_1$, $p_2$, $q_1$ are numerical coefficients resulting from fitting Equation (2) with the experimental data. Equation (2) was applied on the main decomposition of the 3D point cloud at the marker, thus determining two contributes, frontal and transverse, and subsequently combining them in a covariance matrix.

5. Maneuver Modeling with Monte Carlo Simulation

A Monte Carlo simulation [49] was structured to model the influence of the uncertainty in the position of the ToF along with the maneuver. Figure 5 shows the spatial configuration considered, which represents an ideal operational scenario: an area starting 2 $\mathrm{m}$ away from the wheelchair, 2 $\mathrm{m}$ long and 4 $\mathrm{m}$ wide. Within this area, a uniform grid of doors $d_{xy}$ was defined with fixed steps of $0.5$ m in the directions x and y. For each $d_{xy}$, 5 attitudes ranging from 0 to 45 ${}^{\circ }$ were defined. The simulation considered the wheelchair starting from the origin and the door defined at a position $\left[xy\theta \right]$. The target pose $P_t$ was defined as a point placed $0.5$ m in front of the door.

Only the upper part of the area (positive ys) was considered for symmetry. As a further simplification, only positive angles were considered for the attitude of $d_{xy}$s: negative angles simulate an approach from a high incidence direction, an unlikely operational condition given the vision-based structure of the navigation.

The simulation included the kinematic model of the wheelchair and the uncertainty in the kinematic parameters. The result is a set of covariance matrices along the executed path, which represent the possible true position reached by the wheelchair at a defined confidence level [42]. This was set to 99.7% ($3\sigma$s) to maximize the reliability of the results, and hence the safety of the system.

Given the nth of the N simulation runs and the dth door, the nominal onboard position of the ToF is modified by adding a systematic error randomly selected from a pool of possible uncertainty levels in displacement, named ${\sigma }_{xy}$, and attitude, ${\sigma }_{\theta }$. Such bias remains fixed for the entire duration of the nth run, and it causes an error in the generation of the virtual measurements linking the wheelchair to $P_t$. To reduce the number of factors in the proposed analysis, the values defined by ${\sigma }_{xy}$ represent radial displacements and not independent x and y components. These were instead computed starting from the distance definition and by randomly selecting a displacement angle (different from $\theta$).

The simulation starts with the initial planning and motion of the wheelchair from the origin towards $P_t$. After a defined latency, the simulation performs a data update as a new measurement of $P_t$ seen from the actual position of the wheelchair. A new path planning and path following operations are then run, structuring a loop that ends when the wheelchair is less than $0.3$ $\mathrm{m}$ away from $P_t$. After that, a conclusive planning step is performed and the final pose $P_{fp}$ and the covariance matrix $C_{fp}$ are recorded. As a final step, another 100 random seeds are taken from $C_{fp}$ and used to project the position of the wheelchair towards the door’s line (a segment of $0.5$ $\mathrm{m}$ length). Figure 6 shows an example of the simulation.

The kinematic parameters considered for the wheelchair model were a wheelbase of $0.650\pm 0.005$ m and a wheel radius of $0.155\pm 0.010$ m ($3\sigma$). Regarding the data acquisition process, an update in the planning at 10 Hz was considered, equivalent to a routine every 3 samples from the ToF (native operational frequency of 30 Hz). The wheelchair speed was set to $0.5$ m/s, the fastest speed suitable for safe domestic use. Any lower speed reduces the influence of the uncertainty in the final position as it implies a slower motion and therefore more updates and the application of more corrective actions during the approach to the target position.

The levels of uncertainty considered in the sensor position were chosen based on hardware configurations, mounting and fastening options available with standard (certified) mechanical mounts:

• ${\sigma }_{xy}=[0.001,0.005,0.01,0.02,0.05]$ m
• ${\sigma }_{\theta }=[0.1,1,2,5,10]\pi /180$ [rad]

For each ${\sigma }_{xy}$-${\sigma }_{\theta }$ configuration, 100 trials were run.

5.1. Minimum Door Clearance

The transverse displacement of the wheelchair along the door line was considered as a performance factor $P_f$ for each seed. From the distribution $P_f$, the minimum door clearance $d_{c_{min}}$ was assessed using the following expression:

$$d_{c_{min}}=2·\left({\mathsf{\Sigma }}_{P_f}+\frac{3·{\sigma }_{P_f}+W_w}{2}\right)·1.2$$

(3)

$W_w$ is the wheelchair width, while ${\mathsf{\Sigma }}_{P_f}$ and ${\sigma }_{P_f}$ represent the systematic and random errors in reaching the door: the first is computed as the mean of the distribution of $P_f$s, the latter as the standard deviation. The value of $1.2$ stands as an additional safety factor ($20\%$). Figure 7 shows a geometric representation of the formula.

Given the multidimensional structure of the data, the 2 $\sigma$s, and the 3 spatial coordinates of the door, the maximum value among the 5 available was used as a representation of the identified clearance. This condition is the most precautionary as the widest door clearance derives from it.

Figure 8 shows the resulting door clearance organized along the tested $xy$ doors’ position at different levels of ${\sigma }_{xy}$ and ${\sigma }_{\theta }$. Figure 9a instead reports the maximum clearance value among the whole tested area.

5.2. Success Rate

The success rate in navigating through a door for a given clearance $d_c$ was assessed by computing the percentage of seeds that resulted in a passage without impact, modeled as $P_f\left(i\right)<=0.5·(d_c-W_w)$. Figure 9 shows the percentages for a door of 1 $\mathrm{m}$ (a) and $1.2$ $\mathrm{m}$ (b), standard dimensions for domestic environments in which a disabled subject lives [50].

Such results can be used in several ways: (i) to determine the minimum door clearance given an uncertainty level in the position of the sensor, (ii) to identify the maximum allowable $\sigma$s given the doors within a home or (iii) to identify the best trade-off condition for the design of the wheelchair and the environment in which it will be used.

In this work, the focus was on the latter point, identifying in the $1.0$ m clearance the suitable condition given an assessment of the position of the ToF with ${\sigma }_{xy}$ and ${\sigma }_{\theta }$ lower than $0.05$ m and 2 deg, respectively. Between the two, the angular condition is the most restrictive one, and the RoboEYE design was modified accordingly by considering a vertical rather than horizontal rotating support.

The highlighted results are specific to the considered geometry and the implementation of the presented system. However, the definitions and methodology presented are general and aim to include in the design and modeling of the system all the relevant amount of interest for a better understanding of the autonomous system, its implications also to the operational environment. The method can be directly applied to different geometries and configurations of a wheelchair, or other similar applications, with only minor modifications in the parameter definition.

6. Experimental Testing

An experimental test campaign was also carried out. Similar to the previous analysis, the performance of semi-autonomous navigation was assessed through a repeatability analysis on the final positions reached by the real wheelchair given a set of initial, controlled states.

Two markers were set at a clearance of $1.2$ $\mathrm{m}$, Figure 10a shows the considered scheme and the semi-autonomous navigation activated by pointing as the associated POI. Figure 10 shows the sequence of action followed in the tests. The initial position [X′, Y′, θ] of the wheelchair was referenced to the reference frame $[X,Y]$, set at the center and aligned along with the door. The following test conditions were achieved:

• fixed initial position and attitude;
• shifted from the initial reference position along the Y′ direction, from $-0.90\pm 0.01$ m to $+0.90\pm 0.01$ m in steps of $0.30\pm 0.01$ m;
• all the tree initial conditions are changed randomly.

Table 1. Initial conditions of the repeatability analysis.

N. Test	x′ [m]	y′ [m]	$\mathit{\theta }$ [rad]
1	4.26 ± 0.01	0.61 ± 0.01	0.0 ± 0.1
2	4.26 ± 0.01	Variable	0.0 ± 0.1
3	Variable	Variable	Variable

shows the initial condition of the tests.

The outcomes of the test, reported in Figure 11, were:

• a correlation between the initial shift along the y′ direction and the final position reached by the wheelchair, visible in Figure 11b with samples grouped accordingly to their initial starting position;
• in the case of a variable starting positions, maximum displacements from the target position of 0.18 m along the transversal direction, and 0.08 m longitudinally, Figure 11c;
• a systematic displacement between the mean position of the sample set and the target, probably deriving from the limited knowledge of extrinsic parameters of the ToF with respect to the wheelchair.

Table 2. Results of repeatability analysis.

N. Test	$\mathbf{\Delta }\mathit{X}$ [m]	$\mathbf{\Delta }\mathit{Y}$ [m]	${\mathit{\sigma }}_{\mathbf{Max}}$ [m]	${\mathit{\sigma }}_{\mathbf{min}}$ [m]
1	−0.045	−0.003	0.024	0.014
2	0.034	0.028	0.049	0.013
3	−0.014	−0.054	0.064	0.026

reports the average offset between the target pose and the mean final wheelchair position, values that confirm the results achieved in simulation.

7. Conclusions

A domestic robotic wheelchair driven by the user’s gaze was designed in this work to allow users with severe motor disability to move easily and autonomously within their homes. The wheelchair, named RoboEYE, was developed considering commercial contactless technology to interface the user with its control. This non-invasive technology and other components are added to a reconfigured power wheelchair of Nuova Blandino Srl. Compared to other state-of-the-art solutions, RoboEYE offers (i) the minimum invasiveness to the user, (ii) it integrates functionalities derived from the mobile robotics field, (iii) it considers the role of the uncertainty in the human–machine interaction and (iv) it is based on low-cost hardware solutions. The result is an efficient and cost-effective system, suitable for the technological transfer to its potential users.

Two options are offered to the user: (a) direct eye control, in which the motion of the wheelchair is directly connected to the gazed point on the monitor, and (b) semi-autonomous, in which the user only has to select a visual target to allow autonomous maneuvering towards such point. The main investigation of this work regards the latter, potentially the most hazardous of the two. This driving modality allows the user to navigate safely also through narrow passages without fatigue. In particular, the performance of the system was evaluated in detail by performing both numerical simulations and experimental verification. The analysis showed a maximum transverse displacement from the target position of $0.18$ m, resulting from uncertainties that inevitably affect the knowledge of the parameters of the kinematics and the sensor position. For the considered wheelchair, of $0.62$ m width, the operational outcome is a minimum clearance of $0.98$ m for autonomous, reliable and safe navigation through a narrow passage. Such a requirement meets the standards for a domestic usage of the system, such as a house with doors wider than 1 m, which is a common dimension for a structured environment for a wheelchair user. Another evidence obtained from the results is that the angular condition of the ToF ${\sigma }_{\theta }$ has more impact during the motion than ${\sigma }_{xy}$, so more attention should be given to this parameter in wheelchair design.

The advantage of the methodology presented here is that with minor changes it can be applied to different geometries and configurations of a wheelchair, or other similar applications.

The limitations of the proposed approach are related to the fact that RoboEYE is designed to work mainly in a domestic environment or more generally in indoor environments. The technologies used for eye-tracking would be affected by sunlight if the system were used outdoors. Another issue not discussed here is related to the level of illumination that can negatively affect the performances of the POIs detection.