You are currently viewing a new version of our website. To view the old version click .
MDPIMDPI
Search all articles
Sensors
Download PDF
  • Article
  • Open Access

24 March 2023

Compensating for Sensing Failures via Delegation in Human–AI Hybrid Systems

,
and
1
Department of Computer Science, Universitá di Pisa, 56124 Pisa, Italy
2
Institute for Informatics and Telematics (IIT), National Research Council (CNR), 56124 Pisa, Italy
*
Author to whom correspondence should be addressed.
Sensors2023, 23(7), 3409;https://doi.org/10.3390/s23073409
This article belongs to the Section Sensor Networks
Version Notes
Order Reprints

Abstract

Given the increasing prevalence of intelligent systems capable of autonomous actions or augmenting human activities, it is important to consider scenarios in which the human, autonomous system, or both can exhibit failures as a result of one of several contributing factors (e.g., perception). Failures for either humans or autonomous agents can lead to simply a reduced performance level, or a failure can lead to something as severe as injury or death. For our topic, we consider the hybrid human–AI teaming case where a managing agent is tasked with identifying when to perform a delegated assignment and whether the human or autonomous system should gain control. In this context, the manager will estimate its best action based on the likelihood of either (human, autonomous) agent’s failure as a result of their sensing capabilities and possible deficiencies. We model how the environmental context can contribute to, or exacerbate, these sensing deficiencies. These contexts provide cases where the manager must learn to identify agents with capabilities that are suitable for decision-making. As such, we demonstrate how a reinforcement learning manager can correct the context–delegation association and assist the hybrid team of agents in outperforming the behavior of any agent working in isolation.

1. Introduction

The prevalence and growing capabilities of autonomous (or human augmented) systems indicate we can expect an increase in the use of hybrid teams of humans and autonomous systems. While in some cases AI systems can replace human activities, a likely scenario is that of a hybrid system where the human and AI collaborate to solve a certain task. The composition and expectations for these teams will of course depend on the use case and acceptance level for autonomous control. A key aspect of the acceptable level of autonomous control is how reliably the autonomous system can successfully complete the given task. At the very least, to reach acceptance, one would expect a system to meet the performance level of a human. Therefore, it is essential to recognize when an agent can meet this threshold of performance to enable high-performing teams.
In this context, we are not considering a human or AI system’s performance at the granular level of individual actions. We consider this a more general measure of desirable behavior, as it is less subject to minor details, so our focus can instead prioritize overall success or critical failure. Therefore, our focus is instead on identifying deficiencies that would lead to broader suboptimal performances for humans, AI systems, or both. Recognition of these deficiencies is a key aspect of our context of hybrid teaming. Given the opportunity to delegate tasks to a team member, we should prioritize delegations for members who exhibit fewer performance issues and higher overall performance. Of course, the specific use case will determine the constraints regarding delegations (e.g., frequency, maximum number of assigned tasks, safety requirements, etc.), but the primary component we will consider is recognition of factors contributing to reduced performance. With the ability to recognize these deficiencies leading to reduced performance, we aim to properly utilize individual team members to improve overall team outcomes.
Given a notion of individual performance—success rate, safety level, cost, etc.—a key step is to determine how best to assign control in each state or context. This applies more specifically to cases where control is not shared but instead is assigned to one of the entities in the hybrid team. In such a context, a primary aspect is determining a method of associating the measure of success with delegation decisions. In our context, we will train a delegation manager to find these associations. The delegation manager will be responsible for learning to recognize contexts where the human or AI system indicates potential deficiencies through the outcomes resulting from their delegated control. As the manager learns to recognize these cases, it will associate them with an appropriate delegation decision. In our scenario, we will not assume any preference between delegation to a human or AI system, just a prioritization based on perceived performance with respect to outcomes. The association between contexts and failures will be learned in a reinforcement learning framework to enable manager training through reward-based feedback. This ensures we will not need to generate a set of rules governing the manager’s behavior; instead, the manager will learn through exploring actions in states.
A further goal of ours is to generate a manager who can make these performance-related determinations and execute only a small number of delegations. In other words, we will attempt to generate a manager who not only identifies a desirable agent but is capable of making fewer delegation changes. The use of a penalty during training lets us discourage this behavior without requiring any domain-specific upper bound on these switches. Discouraging this behavior is obviously desirable in most cases, as frequent delegation changes would make the interactions between the human and the autonomous system cumbersome, mentally taxing, and potentially dangerous.
To demonstrate our approach, we train our manager to make delegation decisions based on representations of agent observations and contexts. The manager chooses, in each state, whether to assign control to the human or the autonomous agent. To demonstrate the importance of the delegation task, we provide scenarios in which one, both, or neither of the agents exhibit signs of sensing deficiencies. We discuss these deficiencies in later sections, but they are intended to approximate various conditions and errors related to perception and the resulting decisions. These can include issues such as: occlusions, obstructed/broken sensors, adverse lighting/weather conditions, etc. Each of these cases demonstrates a scenario in which human or autonomous sensing can encounter cataclysmic failures. In each context, the manager focuses on the potential existence of sensing deficiencies that could result in a failure case. The notion of failure depends on the use case, but we focus on safety, as it is of primary relevance.

1.1. Driving Safety

To test and demonstrate the performance of our approach, we consider delegation in the context of driving. We assume vehicle control can be delegated to either the human driver or an autonomous driving system. Given this control paradigm, we focus on safe navigation in a driving environment. For our scenario, the key aspect of the delegation task is recognition of either agent’s sensing deficiencies, which impact success likelihoods. In the case of driving, there are numerous aspects that contribute to passenger safety and that of other vehicles, pedestrians, etc., in the environment. These can range from driver attentiveness/awareness to adverse weather and lighting conditions, distractions, and more. Since there are numerous factors that contribute to the safe operation of a motor vehicle, we need to select our primary focus. In the work we present in this paper, we consider safety-related factors that can be attributed to an error in sensing. As an example, consider driving in adverse weather conditions, such as fog. Depending on the severity of the fog, the visible range for object detection can be either marginally or significantly impacted. It seems reasonable to assume that the severity of the adverse conditions and the resulting impact on sensing can be a significant contributing factor in a driving incident.
To justify our consideration of adverse sensing conditions as relevant in the context of vehicle crashes, we investigated the Fatality Analysis Reporting System (FARS) data available via [1]. These data are available from the National Center for Statistics and Analysis, an office of the National Highway Traffic Safety Administration. Their resources offer numerous tools and data repositories related to driving incidents, which are available for multiple use cases and approaches. These resources support querying and compiling statistical data from crash reports regarding numerous aspects pertaining to the driver, weather, road, etc. Therefore, we utilized these resources to identify several scenarios and conditions of interest. These illustrate common cases of incidents as well as likely contributors to adverse sensing outcomes.
As seen in Table 1, the FARS data indicate scenarios where accidents occurred in conditions capable of contributing to impaired sensing. Contexts such as fog, darkness, rain, etc. can all contribute to increased difficulties with accurate perception. As a clear example, heavy fog would easily impact the distance and clarity possible for perception. Therefore, we will consider several adverse conditions as the basis for our various scenarios.
Table 1. Sample crash (filtered by injury-only) data involving adverse conditions. In this case, fog and related weather.
As indicated by the data, crashes occur in various conditions, including cases of mixed conditions. For example, from Table 2, we see cases including both darkness and fog. Therefore, we can also consider combinations in our scenarios. Our assumption is that, in many cases, the combined effect of the conditions would be stronger than either would be individually.
Table 2. Sample crash (filtered by injury-only) data involving adverse conditions. In this case, a combination of foggy (and similar) weather with darkness and possible ambient lighting.
In the case of autonomous driving, we were unable to identify a similar resource for extensive historical data. We suspect this is due to several factors, such as protection of proprietary systems or data, lack of widespread use of autonomous driving systems, etc. Despite our inability to identify a specific data source, we were able to identify several current research topics relating to sensing in the context of autonomous driving. These topics demonstrate various aspects of the challenges regarding sensing in autonomous driving systems, and they indicate how sensing failures can occur. We discuss these topics further in the related work section (see Section 2). The key result is that we identified additional cases and relevant factors that could contribute to an undesirable outcome in an autonomous driving setting.

1.2. Key Results

We demonstrate the use of augmentation observations in a simulated environment to represent various sensing capabilities and inhibitors. These augmentations enable models of sensor output that link operating contexts and the subsequent impact on sensing. The impacts (e.g., reduced accuracy/range, occluded sensors, etc.) generate contexts with a high likelihood of agent failure for activities reliant on perception. The impact on perception is represented for both human and autonomous sensing. In some cases, both agent types would exhibit similar deficiencies for the same augmentation, so we can therefore test various conditions for our hybrid teams. As indicated in our results, the manager recognizes the impact deficiencies have on outcomes to improve delegation choices.
With our proposed manager model, we demonstrate a manager capable of learning delegation policies for various simulated contexts with hybrid teams. In each context, these hybrid teams demonstrate mixed cases where one, both, or neither agent is limited by perception error. In this teaming context, our manager demonstrates an ability to select the appropriate delegations to enable team success. We see our trained manager avoiding collisions whenever there is an agent available with the ability to succeed at the task. Further, we see our manager maintaining a consistent and strong performance across multiple domains. This strong performance is demonstrated despite our penalty for frequent delegations. These results indicate a manager who can learn key states for delegation changes and avoid burdensome delegation behavior.
We further demonstrate the performance of our manager against a random manager. In this context, we illustrate how a random manager could identify the correct delegations by chance, albeit with impractical behavior patterns. The random manager would require the ability to make an unacceptable number of delegation decisions at too high a frequency (e.g., one decision every second). Even with this unrealistic allowance, the random manager still fails to outperform our trained manager, whom we trained with a penalty for the same frequent delegation changes. We see an increase of up to 127 % and 390 % in mean episode reward for our environments and the elimination of up to 26 avoidable collisions when using the learning manager in the same scenario versus the random manager.

3. Delegation Manager

As noted previously, the delegation task can prioritize various aspects of the problem: agent performance, agent cost, etc. For our purposes, we focus strictly on performance and do not consider cost. Further, we assume no preference between a human or autonomous system having control, just that the delegated decision-maker is the higher-performing member of the team. This contrasts with cases of assistive technologies aiding drivers in maintaining safe operation [33]. The delegation decisions are intended for a scenario where the manager is tasked with selecting whether to delegate control to either a human driver or autonomous driving system (see Figure 1), which we will refer to as the driver(s), driving agent(s), or simply agent(s). The key factor is the manager’s ability to recognize which of the two driving agents would be least likely to cause an undesirable outcome (e.g., a collision). As such, we provide a reward signal and training samples to guide the manager’s learning. In this driving scenario, the reward function motivates safety from the manager’s perspective by penalizing collisions. Consequently, the primary signal indicating desirable delegations is whether the selected driver can safely navigate the given driving conditions. In our scenario, the delegation change should occur when the manager believes there are conditions likely to cause the currently selected driving agent to fail in detecting and responding to another entity in the environment. Our assumption is that such a failure would likely lead to a collision when the vehicle paths cross, necessitating a delegation change to avoid the collision.
Figure 1. Diagram of manager and agent observation, selection, action, and environment update.
Regarding the act of delegation, we make some simplifying assumptions. First, we assume that both the human and the autonomous system are maintaining awareness, to the best of their ability, and are therefore ready to take control when a delegation change is made. Next, we assume the manager can make a delegation decision at any given time step. In other words, there is no explicit restriction on the frequency of the delegations. This is not to say we do not consider how a high frequency of delegation changes would be impractical, but instead we simply do not enforce a threshold. In this regard, keeping safety as a key aspect, we prioritize manager behavior that would make the delegation change easier and safer [34]. Therefore, the manager is penalized when they make frequent or immediate delegation changes. In the same way that clutter in a heads-up display can reduce usability [35], we want to ensure our manager maintains usability from the perspective of delegation change frequency. Lastly, we do not assume the manager receives an immediate reward per time step, which would allow them to learn a more direct association between their immediate actions and the feedback received. We instead assume the manager will receive only a single reward at the end of an episode.
We use the single reward to reduce the manager’s access to domain-specific information. To define a state-action reward, we would need to use domain knowledge to label the utility of a particular delegation decision in a particular domain or context. By using a single reward, we can define a more general manager training paradigm in which the specific case is irrelevant. The single reward therefore denotes whether an entire sequence of choices results in a desirable outcome, regardless of use case. This will still implicitly enable the manager to generate a relationship between choices and feedback but will significantly reduce the amount of information we assume the manager has access to at training time. In other words, a delegation decision is not immediately rated for the manager but is instead learned through observing outcomes of trajectories (with potentially many delegation decisions). Again, this allows us to significantly reduce the amount of domain knowledge we assume the manager can access. Additionally, the use of a single reward value for all actions in a trajectory prevents training a manager who exploits particular utilities for specific states. Therefore, like the chess example in Section 3.2 of [36], the manager must learn to improve trajectories or outcomes rather than trying to reach a particular state or achieve a subgoal. Note that, in principle, this approach requires a combinatorial number of trajectories to train the manager in each specific action. However, the approach also allows the manager to learn combinations of actions and their dependencies more generally.
The aforementioned characteristics denote the key aspects of the manager’s goal and responsibility. As seen in Figure 1, the manager provides oversight to make performance-improving delegation decisions. This starts with the manager receiving state information from the human and the autonomous system as well as additional contextual details (e.g., weather, lighting, distance/direction to goal, etc.). Based on these states and contexts, the manager decides on the current delegation. In the first state of the episode, this is a selection of the agent who will perform the first action. In all subsequent states, the manager makes a similar determination but is instead deciding whether to keep the current agent or change the delegated agent. In either case, the manager is using an association between state or context information and corresponding driver performance to learn the most suitable delegations and when to make them. Each delegation decision gives authority to a particular driver, and the driver’s subsequent action generates an updated state for the hybrid team. This update is reflected in an environment update based on the vehicle moving along its path. The environment updates are then observed, with the imposed agent-specific perception constraints, by each agent, and the cycle repeats.

Manager Model

To define the manager, we utilize an architecture capable of integrating both visual and contextual input to learn a behavior policy. The manager is expected to observe representations of state for the human and autonomous system, along with their corresponding contextual information. As such, we utilize two convolutional neural network (CNN) heads to ingest the state representations (human and autonomous driver, respectively). The output values of the two CNN heads are then concatenated along with a vector representing additional contextual information (e.g., positional data). Finally, these values are passed to the remaining hidden layers forming our deep Q-network as seen in Figure 2. This architecture was chosen because it enables several aspects of the learning process. First, the use of CNN heads enables direct manager observation of (visual/sensor) perception estimates. Further, the use of two heads enables a separate association for human perception and AI perception. Lastly, the use of a deep Q-network architecture enables manager training through reward-based feedback during exploration of delegation decisions in the given state spaces (i.e., driving context). This allows the manager to learn an association between states or contexts and delegation actions.
Figure 2. Manager deep reinforcement learning architecture diagram.
Following the diagram of observation, delegation, and action flow seen in Figure 1 and the network diagram seen in Figure 2, the process of converting observations to manager actions is as follows. First, at time t, the human and AI agent provide their observations o t H and o t A I , respectively. In our scenario, these observations are RGB images with height and width assigned per agent based on sensing capabilities. The manager has CNN heads c H and c A I , which receive the corresponding human and AI observations as input. Each head c x , x ∈ { H , A I } is composed of several convolutional layers. The output of each layer undergoes batch normalization [37] and is passed to the layer’s activation function. In our model we use the rectified linear unit [38]
g ( z ) = max ( 0 , z )
for activation. The output of the final layer of a CNN head, ρ x = c x ( o t x ) , is then flattened to create a compatible input vector for our remaining hidden layers.
Prior to the hidden layers, the ρ x are concatenated to generate a single vector ρ t = [ ρ H T , ρ A I T ] T representing the processed visual model of the current state for the team. As additional input, the contextual features (e.g., weather, light, goal distance, etc.) are converted to vector form f t ∈ R k , where k depends on the context features supported. For our model, the elements of vector f t are the (normalized) scalar representation of the contextual features. The final combined vector i t = [ ρ t T , f t T ] T serves as input to the remaining hidden layers of our deep Q-network. As output, the deep Q-network provides a value estimation v ( · ) ∈ R | A | , where | A | denotes the size of the manager’s action space. Finally, our manager chooses the delegation δ t for the current state by selecting the argmax action:
δ t = argmax a ∈ A v ( a ) .
To motivate desirable manager delegation choices, we include several factors in the manager reward function. First, we want delegations to lead to efficient paths from the start to goal positions. Next, we want the manager to learn a strong motivation to avoid vehicle collisions, so we include a large penalty for such cases. Further, we want the manager to avoid making many immediate delegation switches. This is intended to improve the ease of use, as it should prevent the manager from frequently, and within short time windows, making multiple delegation changes. Such delegation changes would be challenging for a human user, as the manager would appear erratic and require increased attention. Additionally, a manager making frequent changes would increase the likelihood of encountering erroneous behavior while the control is transferred to or from the human driver. Lastly, we want to provide the manager with a positive reward for a successful navigation to a goal state, as this is the underlying behavior we are trying to accomplish with all drivers. Therefore, we define the manager reward as
R = 100 − s e − d e goal is reached − 100 − s e − d e otherwise
where s e denotes the number of steps to reach the goal, and d e denotes the number of immediate delegation reversions in an episode. In other words, d e signifies the number of times the manager made two delegation changes in a row. We use d e as a penalty to discourage these immediate delegation changes, as they would be inefficient and unwieldy in many scenarios. We include the additional penalty s e as further motivation for the manager. In this case, it is a motivation to identify the more efficient of the two driver behaviors with respect to time, so efficient driving is prioritized within the confines of safety considerations.

5. Simulation Modeling

For our simulation and experimentation, our environment is based on the CARLO simulator introduced in [40], where it is used as one of the reference simulation environments. We chose this environment because it allows us to test our approach in a driving scenario while allowing us to restrict details of the driving task. CARLO reduces the environment to a two-dimensional representation of basic geometric shapes. This allows us to ignore extraneous details such as surface textures, complexity of object detection and recognition, etc. We can instead focus on a simulation of sensing and detection in this environment, and investigate how changes in perception impact driver performance in our hybrid team case.
The CARLO simulation environment provides a two-dimensional representation of a driving environment (see Figure 6a), which enables research in topics such as automatic control [41]. The environment supports several entity types as well as graphical representation of scenes modeling an aerial view. The key supported entities include vehicles, pedestrians, and buildings. Additional entities can be used to define sidewalks and other basic shapes. In addition to the entities, CARLO enables basic control of movable objects (e.g., cars and pedestrians) via kinetic controls. Entities are initialized with positional and control parameters to define a starting state. Throughout the execution of an episode, the controls for acceleration and steering angle can be modified to generate turning behavior or changes in speed. With these controls, it is possible to simulate vehicles and pedestrians navigating the two-dimensional driving environment.
Figure 6. Sample driving environment including rectangular buildings, sidewalks, and cars (blue, cyan, red, and green rectangles) with shortest path trajectory. (a) Main environment and (b) Sample path (black and green lines representing road segments and shortest path in red).
In addition to entity generation and control, CARLO supports queries relating to inter-object distances and collision detection. The states of all entities in the environment are updated at a given discrete time step size (e.g., one update per 0.1 s). State updates generate updated positioning and kinematic values for each moving agent. Given these updated states, the queries provide a richer representation of the current world state. These are key to our purposes, as we are then able to utilize parameters defining acceptable safe distances and related features. Further, collision detection allows for immediate awareness of an undesired outcome. With these, we can generate test cases that provide a basic yet reasonable analog for simplified driving cases.
To demonstrate the effectiveness of our delegation manager, we model several scenarios designed to recreate, or serve as an analog for, the cases as described in Section 4.1. Based on our simulation environment, where we use rectangular buildings and two-lane roads, we separate the problem space into two primary domains. First is the case where two roads intersect and generate a four-way intersection (see Figure 7a). This is a commonly observed road intersection, so we view it as a key case when demonstrating our approach. The second is the case of a road end intersecting a continuing road, which generates a T-intersection (see Figure 7b). As with the four-way intersection, this is also a commonly encountered intersection type, so it is an additional and essential case for our scenarios. It is worth noting these two domains can be used to form the basis for interaction zones in a larger domain. As such, we train and test our delegation manager in these domains to demonstrate how our approach can learn a desirable delegation model. Additionally, the use of these domains lets us draw specific attention to key interaction zones. Further, we note that the use of a manager trained in these zones would support a more generalized use case (e.g., a Manhattan scenario or grid [42]).
Figure 7. Key interaction zones for driving scenario(s), with cars represented by blue, red, and green rectangles. (a) Four-way intersection and (b) T-intersection.

5.1. Scenarios

With our various supported contexts, to demonstrate the impact of context on performance, we generate scenarios, including the above-mentioned sensing cases. These scenarios are parameterized to create two outcomes for human and autonomous drivers. More specifically, we identify parameters for contexts that can result in either successful or unsuccessful navigation of an environment when a driver operates independently. As a result, each combination of contexts generates a set of case–outcome pairs possible for a manager. In all but one, the manager can select at least one successful agent. The unavoidable failure case is simply the one in which both the human and the autonomous drivers are unable to succeed in a context. In this case, the manager has no choice but to choose a failing driver for delegation. On the other hand, the remaining cases always include at least one driver to whom the manager can delegate to achieve successful navigation of the scenario.
To demonstrate the effectiveness of our delegation manager, we consider several measures of performance. First, we show how the manager can learn a preference for drivers who can successfully navigate a given context. In other words, we show that the manager learns to select an error-avoiding driver when possible. Next, we compare the performance of the manager to that of a manager who simply selects drivers at random at each time step. The use of a random manager could still allow for successful navigation by chance but, at a minimum, would demonstrate a highly undesirable manager. More specifically, the random manager would in most cases be undesirable, as it is likely to make too many frequent changes in the delegation. These rapid changes would make the driving task untenable, as the human and autonomous system would be expected to rapidly gain and relinquish control. This would surely be exhausting and unsettling for a human driver, and this dynamic could increase the chances of failure during a change in control. To compare these various aspects of performance, we utilized the following scenario configurations in our experiments.

5.1.1. Day Driving with Masks

The daytime driving scenario serves to introduce errors with only minor changes made to the observation. In our experiments, we utilize a single mask in the forward sensing region. This enables error-induced and error-free behavior based on the severity level of the mask. For instance, a small mask at the farthest reaches of the sensing region would often have little or no impact on the driver’s ability to detect other entities. However, increasing the mask’s size in the forward region would result in a reduced ability to perceive entities in the direction the vehicle is traveling. As the size of the mask increases, the time between detection and error shrinks. For our experiments, we generate sensor and mask configurations that enable both success and failure cases for both human and autonomous drivers, illustrated in Table 4. Consequently, in each episode, the manager is observing one of four cases.
Table 4. Human–AI pair cases.

5.1.2. Day Driving with Fog

For the fog scenario, errors are introduced through increasing severity of fog levels. In our experiments, we still utilize the sensing regions and masks, but only regions and masks that do not result in errors on their own. In other words, we only use sensing regions and mask parameters from successful weather-free cases. To introduce errors, we utilize fog severity levels that generate failures in detection. For the error-free cases, we still include fog, but only at levels that do not degrade detection to the point of failure. A further key detail in this scenario is that we are only assuming a weather-based impact on sensing for the human driver. For the autonomous system, we are instead reusing the sensor and mask cases from the autonomous driver seen in Section 5.1.1. This demonstrates a case where humans are impacted by the weather conditions, but where the autonomous system is instead only subject to their masks. This allows us to represent a disparity in the sensitivity to contexts between the human and autonomous system. As a result, the manager is learning to delegate in the combinations of scenarios found in Table 5.
Table 5. Human–AI pair cases.

5.1.3. Night Driving

Like the fog scenario, errors are introduced through increasing severity of lighting levels. As in the fog case, we are using differing levels of severity to either induce or omit erroneous behavior. These severity levels impact the darkness level, the width of the headlight region, and the distance covered by the headlight region. Further, the autonomous system is again not subject to the lighting conditions; instead, they will utilize the parameters seen in Section 5.1.1. Like the fog scenario, this demonstrates a case where the human is impacted by the lighting conditions, but the autonomous system is instead only subject to masks. This again allows us to represent a disparity in the sensitivity to contexts between the human and autonomous system. As a result, the manager is learning to delegate in the combinations of scenarios found in Table 6.
Table 6. Human–AI pair cases.

5.1.4. Failed Color Detection

To represent failed detection or segmentation due to erroneous interpretation of sensing, we generate cases with an autonomous agent unable to perceive another vehicle in the environment due to its color. These cases will result in the autonomous system struggling to clearly recognize an entity exists. As such, this would lead to a failure to react to a potential upcoming collision. To offer success cases, we assign error colors that are not used for potential collision entities (i.e., unobserved or non-interacting entity color). Further, the human driver is not subject to the color-sensitivity conditions; instead, they will again utilize the parameters seen in Section 5.1.1. As a result, the manager is learning to delegate in the combinations of scenarios found in Table 7.
Table 7. Human–AI pair cases.

5.1.5. Day Driving with Fog and AI Color Sensitivity

Extending on the previous fog case, the human driver’s success or failure is determined by the severity parameters. The distinction in this case is that we are no longer assuming the autonomous agent is unaffected by the fog. In this scenario, the assumption is that the autonomous system is sensitive to a particular color. This is further complicated by the fog levels, so we provide parameters that produce fog levels that modify the similarity of the observed color to the blindness color. This demonstrates how the contextually based color can shift the perception toward or away from an error case. As a result, the manager is learning to delegate in the combinations of scenarios found in Table 8.
Table 8. Human–AI pair cases.

5.1.6. Noisy Estimates

As an additional test case, we determine if adding noise to the generation of contexts impacts the manager’s performance. In this case, the observations provided to the manager are altered by some noise to introduce a divergence between the ground truth and the manager observation. The noise parameters affect the severity levels used to create the context-based observation. The degree to which the ground truth and observed state vary depend on the noise parameters. We sample new parameters via a truncated normal distribution centered at the current values. The truncation bounds and the standard deviation determine how widely the noisy estimate varies from the ground truth. To demonstrate these effects, we test the inclusion of noisy parameter estimates in the fog and night driving cases.

6. Results

In the following, we present and discuss the results for our trained delegation manager. The results represent manager performance in the various scenarios, including fog, night driving, masked sensors, etc. As noted in Section 5.1, these contexts demonstrate cases where a human or autonomous driver could enter an undesired state (e.g., collision with another vehicle) as a result of a sensing or detection failure. We consider several aspects with respect to manager performance. First, we track the number of collisions the manager could avoid given the team dynamics. Avoidable collisions are those that could have been avoided had the manager selected an error-free agent. For example, if the human’s perception is impacted severely enough by fog to inhibit prompt detection of other vehicles, but the autonomous system’s perception is not, then a collision encountered when the human is delegated control would be considered avoidable. Hence, the manager should have detected the potential issue and selected the autonomous driver instead.
The next aspect we consider is the number of delegation changes. For our scenarios, we consider basic delegation changes (i.e., a single and persistent change) and sudden delegation changes. A sudden delegation change is a case where the manager changes delegation and then reverses the delegation in a sequence of three consecutive time steps (e.g., Human → AI → Human ). As we noted previously, this is discouraged via a penalty in the manager’s reward function. For basic delegation changes, we do not penalize the manager. Our intention is to focus on the more severe and inefficient case of sudden changes. It could be worth considering a penalty for total delegation choices in a modified reward function, but this was beyond the scope of our current work.
Given the reward function in Equation (3), we can determine a sense of desirable manager behavior. In our scenarios, the episode lengths are larger than the positive reward received for a successful navigation outcome, so all episode rewards will be negative. The key distinction will be the magnitude of episode rewards. We expect a high-performing manager to identify successful driving agents and make few or no sudden delegation changes. Therefore, a high-performing manager should demonstrate similar performance values in the S/S, S/E, and E/S cases ( E → Error , S → Success ). Further, these episode rewards should be as close to zero as the episode lengths allow.

Delegation Manager

The results presented in this section demonstrate our manager’s ability to identify necessary delegation decisions. Further, in all cases, the results indicate the manager correctly identifies the suitable delegation(s) to avoid a collision with another vehicle. The performance of our manager is demonstrated in each of the scenarios we present. For each, the manager provides necessary delegations while still avoiding the discouraged cases of immediate delegation reversals.
As a general note regarding the following results, there are a few key aspects to remember. First, the episode lengths are used as a penalty, so the maximum achievable reward in our scenario is reduced. As noted in Equation (3), the maximum reward for successful navigation is 100, and the penalty for a collision is − 100 . After episode lengths, the remaining penalty is the number of immediate delegation reversals. With these factors, the expectation is that the manager performance across the two environments will be similar but will vary according to the episode length. In the case of the T-intersection environment, the shortest expected episode is approximately 165 time steps. For the four-way intersection environment, the shortest expected episode is slightly shorter at approximately 120 time steps. Therefore, the best reward the manager could receive would be approximately − 65 and − 25 for the T-intersection and four-way intersection environments, respectively.
Regarding the two delegation changes highlighted in the following results, we provide counts for two cases to illustrate related aspects of delegation. First, the basic changes reflect the manager’s propensity to make delegation changes over an episode, but not cases where these changes are immediately reversed. The desire would still be a manager with fewer of these basic changes, but we do not explicitly discourage this behavior. Instead, our reward function is focused on the case of sudden changes.This case would be far more challenging in a realistic driving setting, so we prioritized it in our training and testing. As is demonstrated in our results, our reward function successfully discourages these immediate changes in delegation, preventing untenable manager behavior. As will be demonstrated in Section 7, our use of the penalty significantly diminishes the occurrence of immediate delegation changes, especially when compared to a manager without this penalty.
Our manager’s behavior and performance indicate the ability to both identify desirable delegations and reduce immediate delegation switches. For example, the results in Table 9 and Table 10 illustrate the approximately optimal path found by the team. Further, we see that each case includes few or no immediate delegation changes. On the other hand, as expected, the scenarios where the manager has no choice but to select an erroneous agent show our manager is indeed encountering unavoidable failure cases. In both the T-intersection and four-way intersection cases, we see the E/E cases resulting in significantly worse performance overall, which indicates our team is encountering the forced failure case. Therefore, the delegation has proven capable of associating the failure cases with desirable delegation choices. As seen in the following tables, the manager’s performance is consistently strong across our various scenarios.
Table 9. Human: sensor and mask, AI: sensor and mask. Test results for manager performance and behavior.
Table 10. Human: weather, AI: sensor and mask. Test results for manager performance and behavior.
The results for our proposed manager are presented in Table 9, Table 10, Table 11, Table 12, Table 13, Table 14 and Table 15:
Table 11. Human: light, AI: sensor and mask. Test results for manager performance and behavior.
Table 12. Human: sensor and mask, AI: color. Test results for manager performance and behavior.
Table 13. Human: weather, AI: weather and color. Test results for manager performance and behavior.
Table 14. Human: noisy light, AI: sensor and mask. Test results for manager performance and behavior.
Table 15. Human: noisy weather, AI: sensor and mask. Test results for manager performance and behavior.
As the above results indicate, our delegation manager successfully identifies desirable agents for delegation while reducing the occurrence of undesirable immediate delegation changes. Further, we see consistently strong results across the various contexts presented. The primary exception would be the “Human: noisy weather, AI: sensor and mask” case. In this case, we see improved performance in the E/E case. This can likely be attributed to the fact that our error cases only occur if the agents reach the interaction point at the same time. If the agent observations are skewed enough by the context, it is feasible for the conflicting vehicles to reach the interaction point at an interval that enables accidental avoidance. This, combined with a manager operating on noisy observations, could alter the behavior of the managed driving agents, which could reduce the likelihood of reaching the given failure scenario. Therefore, we believe this occurred in this case, resulting in slightly improved (suboptimal) performance. It is still apparent that the results indicate a significant increase in failures for the E/E error case, so our manager’s impact regarding improvement in performance is still demonstrated.
As an additional side-effect of the noisy observations, the degradation of the observations based on the weather or lighting conditions obscures more of distinguishing features of a given state, which could lead to higher uncertainty in the manager’s decisions. Further, this could shift the manager’s preference between either driver toward uniform. In this case, the manager would have an approximately equal preference between the two agents, so we would expect an increased occurrence of basic changes as indicated in our results. In future approaches, we could investigate altered reward functions to discourage other frequencies of delegation changes. We could either penalize any delegation change or vary the magnitude of the penalty by type of delegation change. As indicated by the results, these changes would likely offer little change regarding the success of navigation but would reduce the likelihood of changes in manager delegations.
From the results demonstrated, we conclude that our manager has learned to both identify the agents likely to lead to successful navigation and avoid immediate delegation changes. This is accomplished with a reward function that offers little in the way of domain knowledge. The manager is not provided rewards specific to a driving task, only details regarding how successful the team performed and how well the manager avoided undesirable delegation behavior. Therefore, we conclude the proposed manager model can learn a suitable association among contexts, behavior, and desirable decisions.

7. Random Manager

We determined that a purely random manager with no limit on delegation interval would have an unfair advantage over the learned manager in our setting. This is because the random manager would not respect a penalty discouraging frequent, immediate delegation changes. As a result, the random manager can make delegation choices frequently enough that they could, by chance, perform a sequence of delegations and complete a driving episode. This would be beneficial if the ability to make multiple delegation changes in rapid fashion were not a problem for the user. To counteract this, we introduced an interval that blocks the delegation decision until the end of the interval (e.g., once every 10 time steps). This was intended to illustrate two key factors. First, how sensitive the scenario is to the frequency of delegation decisions. Second, how much of a gap in performance is seen between the interval-restricted manager and learning manager in the same contexts. When comparing our delegation manager to the random manager, the effect of numerous immediate switches would still be apparent in agent reward, but we believed this would be a less significant representation of performance overall. As a final note, we are not testing the noisy scenarios for the random manager, as the noise is unlikely to impact the manager’s behavior. Therefore, the results for the noisy case are already indicated by the noise-free cases.
The results for the random manager are presented in Table 16, Table 17, Table 18, Table 19 and Table 20.
Table 16. Mean rewards and avoidable collision counts (by human–AI configuration) for a random manager with given delegation interval lengths in the human: sensor and mask, AI: sensor and mask case.
Table 17. Mean rewards and avoidable collision counts (by human–AI configuration) for a random manager with given delegation interval lengths in the human: weather, AI: sensor and mask case.
Table 18. Mean rewards and avoidable collision counts (by human–AI configuration) for a random manager with given delegation interval lengths in the human: light, AI: sensor and mask case.
Table 19. Mean reward (by human–AI configuration) for a random manager with given delegation interval lengths in the human: sensor and mask, AI: color case.
Table 20. Mean rewards and avoidable collision counts (by human–AI configuration) for a random manager with given delegation interval lengths in the human: weather, AI: weather and color case.
As indicated in the results, in many cases, there is a bit of variance across the number of avoidable collisions as the interval length increases. For instance, Table 16 demonstrates possible periodicity in the performance. We see smaller numbers of avoidable collisions in some intervals (e.g., 4 / 9 for the T-intersection with 10-step interval) while seeing higher numbers in others. We attribute this to several aspects of the scenarios. First, the fact that the manager is behaving completely randomly can by chance still allow for randomly successful outcomes. Second, the time step where the detection is essential versus the interval length is important. If the interval divides almost equally (e.g., 85 steps in the T-intersection case), then the random manager has a better chance of randomly selecting the correct agent in time. The correct agent in the key time step triggers a response for collision avoidance, so an interval that aligns well with these key states could improve the team’s chance of success. Further, this key time step is different between the two environments, so we can see these values vary between the two environments. A final factor is the fact that the longer interval lengths reduce the number of overall delegation decisions made. Therefore, the likelihood of a correct choice at the right time can increase with this compatibility between interval lengths and key time steps.
Despite the observed variance in performance with respect to avoidable collisions, the results still clearly indicate the improvements made from a learned delegation manager policy over a random manager. Each case demonstrates how the random manager both chooses to make significantly more sudden delegation changes and makes far fewer desirable delegations. In all the cases where the manager can fail (i.e., S/E, E/S, and E/E), we see far more cases of avoidable collisions and significantly smaller rewards for the random manager. Further, we see a much more severe variance in random manager performance versus the learned manager. This again shows how the random manager can select correct delegations, but the rate of success and consistency of performance are much lower than the learned model. For example, with respect to the performance of the learned manager (see Table 9), we note a drop in performance up to 127 % and 390 % in rewards for the T-intersection and four-way intersection cases, respectively. Additionally, compared to zero for the learned manager, we see an increase for the random manager of up to 26 cases of avoidable collisions. Combined, these aspects indicate the delegation manager’s significance with respect to team performance. A manager failing to properly utilize the proper agent clearly fails to avoid collisions. This further indicates that the use of a trained manager enables improved performance beyond that of cases where either agent would be allowed to operate without intervention.

8. Conclusions

8.1. Final Remarks

As our results indicate, the inclusion of a learned delegation behavior policy improves team performance. The improvement in performance is a result of the manager identifying a desirable driver and corresponding delegation of control. We further emphasize the impact of our manager by comparison with a random manager. We acknowledge that a random manager with the ability to make frequent delegation changes could by chance select the appropriate driver. On the other hand, we also note how the random manager’s use of frequent delegation changes would make it incompatible with a real setting. A human user would most likely be unable or unwilling to use a delegation manager that is so unpredictable. The user would be burdened by the random manager’s unpredictable behavior and subsequently be burdened with a higher cognitive load. Therefore, we demonstrate an effective and delegation-efficient manager.

8.2. Future Work

As we acknowledged earlier, our simulator and test environments are a simplification of realistic driving cases. In one aspect, we perform a simplification through reduced fidelity in the simulation. Our use of basic geometric shapes and kinematics reduces the alignment to real-world driving. As a future approach, we could investigate extending this approach to simulators with higher fidelity in their models, such as the the CARLA simulation environment [43] or DeepDrive [44]. Another simplification we utilize in our scenarios is the reduced size of our driving scenarios. We utilized our environments to specifically highlight the consequences of sensing deficiencies in highly instructive cases. While this enables our demonstration of performance in key interaction cases, given the strong manager performance, we could investigate extended driving scenarios with more complex paths and more potential cases for interaction. These changes would demonstrate how well our method translates to a more true-to-life setting, but we cannot say with certainty whether it would provide an improved demonstration of the delegation learning itself.
An additional aspect we could investigate, to see how well our results translate to new cases, would be the inclusion of more managed vehicles in a single environment. With the additional vehicles and a larger map, the tested vehicles would demonstrate longer driving sequences and multiple interactions. Further, the inclusion of additional managed vehicles would indicate how well two or more managed vehicles can account for each other. On the other hand, it is our belief that this demonstrates less a case of compensating for sensing failures and more a case of improved driving capabilities. Similarly, within our CARLO scenario, we could further extend our environments to include more complexity in roads and related elements. For instance, we could add traffic signals at intersections to modify the failure scenarios to include cases involving missed traffic signals. In these cases, we could simulate an agent failing to detect a traffic signal, resulting in the agent entering the intersection at the wrong time. These scenarios would align well with the computer vision concepts we noted in Section 2. They could demonstrate cases where either sensors fail to detect an object or a perturbation is made to prevent detection.

Author Contributions

Conceptualization, A.F., A.P. and M.C.; Investigation, A.F.; Methodology, A.F., A.P. and M.C.; Software, A.F.; Supervision, A.P. and M.C.; Writing—original draft, A.F.; Writing—review & editing, A.P. and M.C. All authors have read and agreed to the published version of the manuscript.

Funding

This work was partly supported by the H2020 Humane-AI-Net project (grant No. 952026), CHIST-ERA (grant No. CHIST-ERA-19-XAI-010-MUR-484-22), and by the European Union under the Italian National Recovery and Resilience Plan (NRRP) of partnership on “Artificial Intelligence: Foundational Aspects” (PE0000013-program “FAIR”).

Data Availability Statement

The data presented in this study are openly available in the Fatality and Injury Reporting System Tool (FIRST) at https://cdan.dot.gov/query.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

References

  1. Fatality and Injury Reporting System Tool (FIRST). Available online: https://cdan.dot.gov/query (accessed on 6 February 2023).
  2. Fuchs, A.; Passarella, A.; Conti, M. Modeling, Replicating, and Predicting Human Behavior: A Survey. ACM Trans. Auton. Adapt. Syst. 2023. just accepted. [Google Scholar] [CrossRef]
  3. Urmson, C.; Anhalt, J.; Bagnell, D.; Baker, C.; Bittner, R.; Clark, M.; Dolan, J.; Duggins, D.; Galatali, T.; Geyer, C.; et al. Autonomous driving in urban environments: Boss and the urban challenge. J. Field Robot. 2008, 25, 425–466. [Google Scholar] [CrossRef]
  4. Chen, L.; Li, Y.; Huang, C.; Li, B.; Xing, Y.; Tian, D.; Li, L.; Hu, Z.; Na, X.; Li, Z.; et al. Milestones in autonomous driving and intelligent vehicles: Survey of surveys. IEEE Trans. Intell. Veh. 2022, 8, 1046–1056. [Google Scholar] [CrossRef]
  5. Ghorai, P.; Eskandarian, A.; Kim, Y.K.; Mehr, G. State estimation and motion prediction of vehicles and vulnerable road users for cooperative autonomous driving: A survey. IEEE Trans. Intell. Transp. Syst. 2022, 23, 16983–17002. [Google Scholar] [CrossRef]
  6. Badrloo, S.; Varshosaz, M.; Pirasteh, S.; Li, J. Image-Based Obstacle Detection Methods for the Safe Navigation of Unmanned Vehicles: A Review. Remote Sens. 2022, 14, 3824. [Google Scholar] [CrossRef]
  7. Zhang, Y.; Carballo, A.; Yang, H.; Takeda, K. Perception and sensing for autonomous vehicles under adverse weather conditions: A survey. ISPRS J. Photogramm. Remote Sens. 2023, 196, 146–177. [Google Scholar] [CrossRef]
  8. Yeong, D.J.; Velasco-Hernandez, G.; Barry, J.; Walsh, J. Sensor and sensor fusion technology in autonomous vehicles: A review. Sensors 2021, 21, 2140. [Google Scholar] [CrossRef]
  9. Kuhn, C.B.; Hofbauer, M.; Petrovic, G.; Steinbach, E. Introspective black box failure prediction for autonomous driving. In Proceedings of the 2020 IEEE Intelligent Vehicles Symposium (IV), Las Vegas, NV, USA, 19 October–13 November 2020; IEEE: Piscataway, NJ, USA, 2020; pp. 1907–1913. [Google Scholar]
  10. Hecker, S.; Dai, D.; Van Gool, L. Failure prediction for autonomous driving. In Proceedings of the 2018 IEEE Intelligent Vehicles Symposium (IV), Changshu, China, 26–30 June 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 1792–1799. [Google Scholar]
  11. Kuhn, C.B.; Hofbauer, M.; Petrovic, G.; Steinbach, E. Trajectory-based failure prediction for autonomous driving. In Proceedings of the 2021 IEEE Intelligent Vehicles Symposium (IV), Nagoya, Japan, 11–17 July 2021; IEEE: Piscataway, NJ, USA, 2021; pp. 980–986. [Google Scholar]
  12. Besnier, V.; Bursuc, A.; Picard, D.; Briot, A. Triggering Failures: Out-Of-Distribution detection by learning from local adversarial attacks in Semantic Segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Virtual, 11–17 October 2021; pp. 15701–15710. [Google Scholar]
  13. Kuhn, C.B.; Hofbauer, M.; Petrovic, G.; Steinbach, E. Better look twice-improving visual scene perception using a two-stage approach. In Proceedings of the 2020 IEEE International Symposium on Multimedia (ISM), Naples, Italy, 2–4 December 2020; IEEE: Piscataway, NJ, USA, 2020; pp. 33–40. [Google Scholar]
  14. Zimmermann, M.; Wotawa, F. An adaptive system for autonomous driving. Softw. Qual. J. 2020, 28, 1189–1212. [Google Scholar] [CrossRef]
  15. Guo, J.; Kurup, U.; Shah, M. Is it safe to drive? An overview of factors, metrics, and datasets for driveability assessment in autonomous driving. IEEE Trans. Intell. Transp. Syst. 2019, 21, 3135–3151. [Google Scholar] [CrossRef]
  16. Rosique, F.; Navarro, P.J.; Fernández, C.; Padilla, A. A systematic review of perception system and simulators for autonomous vehicles research. Sensors 2019, 19, 648. [Google Scholar] [CrossRef]
  17. Secci, F.; Ceccarelli, A. On failures of RGB cameras and their effects in autonomous driving applications. In Proceedings of the 2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE), Coimbra, Portugal, 12–15 October 2020; IEEE: Piscataway, NJ, USA, 2020; pp. 13–24. [Google Scholar]
  18. Eykholt, K.; Evtimov, I.; Fernandes, E.; Li, B.; Rahmati, A.; Xiao, C.; Prakash, A.; Kohno, T.; Song, D. Robust physical-world attacks on deep learning visual classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–22 June 2018; pp. 1625–1634. [Google Scholar]
  19. Elsayed, G.; Shankar, S.; Cheung, B.; Papernot, N.; Kurakin, A.; Goodfellow, I.; Sohl-Dickstein, J. Adversarial examples that fool both computer vision and time-limited humans. Adv. Neural Inf. Process. Syst. 2018, 32, 3914–3924. [Google Scholar]
  20. Akhtar, N.; Mian, A.; Kardan, N.; Shah, M. Advances in adversarial attacks and defenses in computer vision: A survey. IEEE Access 2021, 9, 155161–155196. [Google Scholar] [CrossRef]
  21. Deng, Y.; Zheng, X.; Zhang, T.; Chen, C.; Lou, G.; Kim, M. An analysis of adversarial attacks and defenses on autonomous driving models. In Proceedings of the 2020 IEEE International Conference on Pervasive Computing and Communications (PerCom), Austin, TX, USA, 23–27 March 2020; IEEE: Piscataway, NJ, USA, 2020; pp. 1–10. [Google Scholar]
  22. Cao, Y.; Xiao, C.; Cyr, B.; Zhou, Y.; Park, W.; Rampazzi, S.; Chen, Q.A.; Fu, K.; Mao, Z.M. Adversarial sensor attack on lidar-based perception in autonomous driving. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, London, UK, 11–15 November 2019; pp. 2267–2281. [Google Scholar]
  23. Cao, H.; Zou, W.; Wang, Y.; Song, T.; Liu, M. Emerging Threats in Deep Learning-Based Autonomous Driving: A Comprehensive Survey. arXiv 2022, arXiv:2210.11237. [Google Scholar]
  24. Zhou, W.; Berrio, J.S.; Worrall, S.; Nebot, E. Automated evaluation of semantic segmentation robustness for autonomous driving. IEEE Trans. Intell. Transp. Syst. 2019, 21, 1951–1963. [Google Scholar] [CrossRef]
  25. Rahman, Q.M.; Sünderhauf, N.; Dayoub, F. Did you miss the sign? A false negative alarm system for traffic sign detectors. In Proceedings of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China, 3–8 November 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 3748–3753. [Google Scholar]
  26. Meresht, V.B.; De, A.; Singla, A.; Gomez-Rodriguez, M. Learning to Switch Between Machines and Humans. arXiv 2020, arXiv:2002.04258. [Google Scholar]
  27. Straitouri, E.; Singla, A.; Meresht, V.B.; Gomez-Rodriguez, M. Reinforcement Learning Under Algorithmic Triage. arXiv 2021, arXiv:2109.11328. [Google Scholar]
  28. Jacq, A.; Ferret, J.; Pietquin, O.; Geist, M. Lazy-MDPs: Towards Interpretable RL by Learning When to Act. In Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems, Online, 9–13 May 2022; pp. 669–677. [Google Scholar]
  29. Richards, D.; Stedmon, A. To delegate or not to delegate: A review of control frameworks for autonomous cars. Appl. Ergon. 2016, 53, 383–388. [Google Scholar] [CrossRef] [PubMed]
  30. Palmer, S.; Richards, D.; Shelton-Rayner, G.; Izzetoglu, K.; Inch, D. Assessing Variable Levels of Delegated Control—A Novel Measure of Trust. In Proceedings of the HCI International 2020—Late Breaking Papers: Cognition, Learning and Games: 22nd HCI International Conference, HCII 2020, Copenhagen, Denmark, 19–24 July 2020; Springer: Cham, Switzerland, 2020; pp. 202–215. [Google Scholar]
  31. Candrian, C.; Scherer, A. Rise of the machines: Delegating decisions to autonomous AI. Comput. Hum. Behav. 2022, 134, 107308. [Google Scholar] [CrossRef]
  32. Fuchs, A.; Passarella, A.; Conti, M. A Cognitive Framework for Delegation Between Error-Prone AI and Human Agents. In Proceedings of the 2022 IEEE International Conference on Smart Computing (SMARTCOMP), Helsinki, Finland, 20–24 June 2022; pp. 317–322. [Google Scholar] [CrossRef]
  33. Razak, S.F.A.; Yogarayan, S.; Abdullah, M.F.A.; Azman, A. Modeling User Acceptance of In-Vehicle Applications for Safer Road Environment. Future Internet 2022, 14, 148. [Google Scholar] [CrossRef]
  34. Trösterer, S.; Meschtscherjakov, A.; Mirnig, A.G.; Lupp, A.; Gärtner, M.; McGee, F.; McCall, R.; Tscheligi, M.; Engel, T. What we can learn from pilots for handovers and (de) skilling in semi-autonomous driving: An interview study. In Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Oldenburg, Germany, 24–27 September 2017; pp. 173–182. [Google Scholar]
  35. Murali, P.K.; Kaboli, M.; Dahiya, R. Intelligent In-Vehicle Interaction Technologies. Adv. Intell. Syst. 2022, 4, 2100122. [Google Scholar] [CrossRef]
  36. Sutton, R.S.; Barto, A.G. Reinforcement Learning: An Introduction; MIT Press: Cambridge, MA, USA, 2018. [Google Scholar]
  37. Bjorck, N.; Gomes, C.P.; Selman, B.; Weinberger, K.Q. Understanding batch normalization. Adv. Neural Inf. Process. Syst. 2018, 32, 7705–7716. [Google Scholar]
  38. Nair, V.; Hinton, G.E. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th International Conference on Machine Learning (ICML-10), Haifa, Israel, 21–24 June 2010; pp. 807–814. [Google Scholar]
  39. Riemersma, T. Colour Metric. Available online: https://www.compuphase.com/cmetric.htm (accessed on 3 February 2023).
  40. Cao, Z.; Biyik, E.; Wang, W.Z.; Raventos, A.; Gaidon, A.; Rosman, G.; Sadigh, D. Reinforcement Learning based Control of Imitative Policies for Near-Accident Driving. In Proceedings of the Robotics: Science and Systems (RSS), Virtual, 12–16 July 2020. [Google Scholar]
  41. Golnaraghi, F.; Kuo, B.C. Automatic Control Systems; McGraw-Hill Education: New York, NY, USA, 2017. [Google Scholar]
  42. Korger, U.; Del Galdo, G.; Grosch, A.; Haardt, M. Quality of service oriented spatial processing in the Manhattan grid. In Proceedings of the 2008 International ITG Workshop on Smart Antennas, Darmstadt, Germany, 26–27 February 2008; IEEE: Piscataway, NJ, USA, 2008; pp. 362–369. [Google Scholar]
  43. Dosovitskiy, A.; Ros, G.; Codevilla, F.; Lopez, A.; Koltun, V. CARLA: An Open Urban Driving Simulator. In Proceedings of the 1st Annual Conference on Robot Learning, Mountain View, CA, USA, 13–15 November 2017; pp. 1–16. [Google Scholar]
  44. Quiter, C.; Ernst, M. Deepdrive/Deepdrive: 2.0. Available online: https://zenodo.org/record/1248998#.ZBrpsHaZOjg (accessed on 9 March 2023).
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Article Metrics

Citations

Article Access Statistics

Journal Statistics
Multiple requests from the same IP address are counted as one view.
Download PDF
Real-Time Recognition and Localization Based on Improved YOLOv5s for Robot’s Picking Clustered Fruits of Chilies
Previous
Multi-Scale Feature Interactive Fusion Network for RGBT Tracking
Next