What Is Worse than a Back-Seat Driver? A Remote One: Rethinking Teleoperation in Automated Vehicles

Highlights

What are the main findings?

• Remote driving is difficult and less safe than in-vehicle driving.
• Humans can solve novel and complex driving problems providing remote assistance to AVs.
• Passenger interactions will add a new level of complexity to problem scenarios.

What are the implications of the main findings?

• Remote driving should be avoided or kept to an absolute minimum.
• Humans should support AVs solving problems not just taking-over from AVs.
• Research is needed to understand future currently unexpected passenger needs.

Abstract

Much of the research and proposed industrial deployment of Remote Operations (ROs) in support of automated vehicles is founded on the optimistic premise that in-vehicle standby drivers and Safety Officers (SOs) can easily be replaced with ROs, with some commercial models proposing that a single RO supervise over 30 vehicles. However, emerging evidence suggests that the RO task is fundamentally different from the in-vehicle driving task. Furthermore, communications latency and reliability constraints, coupled with fragmented attention and altered task demands, introduce distinctive human factor challenges. These include degraded situational awareness, increased cognitive workload, and reduced capacity for timely intervention. The result is a widening gap between what is commercially desirable and what may be operationally appropriate. This paper argues that the central question for remote operation in support of automated vehicles is not one of technical feasibility but of human-centred appropriateness, and debates which RO roles should continue to be developed and which should be constrained or avoided. We present a synthesis of research on remote vehicle operations, identifying recurring human-factor limitations and mapping them to proposed remote tasks. The paper concludes with targeted recommendations for designers, operators, and regulators intended to question the scaling of teleoperation models and to reframe the debate from “Can we teleoperate?” to “Under what conditions should we?”

1. Introduction

Over the past few decades there has been a steady increase of interest and international research in the development and deployment of Automated Vehicles (AVs). In the USA and China much of the interest has been around smaller, individual and private transport such as the Tesla “robotaxis” [1]. However, in the UK and EU, AVs are more frequently being tested as a specialist extension of the public transport system, such as the SCALE project [2], which aims to operate an AV shuttle bus around the Birmingham National Exhibition Centre.

Whilst most of these trials are encouraging, unfortunately the evidence from AV and human safety officer disengagements [3,4,5] indicates that AVs remain error prone and continue to require humans to intervene and to solve driving or navigational problems. Consequently, there remains a prevalent expectation that, for the foreseeable future, human operator input in the form of supervision and intervention will still be required for all forms and types of AV (whether individual Robotaxi or large public transport systems), especially for situations involving critical decision making [6,7,8]. The currently widespread solution is to place a safety officer (SO) inside the vehicle, who acts as a standby driver, supervising the AV and taking over the driving task in situations of automation disengagement [9]. This requirement has been clearly identified by the Centre for Connected and Autonomous Vehicles in the UK (CCAV), who have produced a publicly available specification for the scope and training requirement for on-board SOs in the British Standards Institution (BSI) PAS 1884 [10].

However, whilst this model of employing a single human to provide on-board supervision of a single AV might be acceptable during initial capability tests and trials, it is neither scalable nor rational for implementation on any commercially focused AV service. The longer-term solution proposed by most mid-size AV manufacturers and test and trial projects is to physically remove the SO from the AV and place them in a remote operations centre (ROC) where they can then provide a centralised monitoring and teleoperations services to multiple AVs [11]. This will then allow an increase in the number of AVs a single human operator can manage and will facilitate the greater commercial efficiencies and flexibility in vehicle operations required for successful implementation of an AV service [12,13].

The conversion of the SO, located inside the vehicle using directly linked controls, into a remote operator (RO) using a teleoperation interface has been predicted and expected for some time (e.g., [14]), and there has certainly been plenty of research conducted on the technological requirements (e.g., [15,16]) and practical methods of interacting with and controlling a vehicle using telecommunications (e.g., [17,18,19]). A common observation within the research into implementing teleoperations is that remote operation of highly automated vehicles is not simply an extension of in-vehicle driving or supervising; it is a new field of profession with many new human factor challenges and user needs that will need a new design of workplace and human–automation interface [20].

There are a number of causes for this change. On-board an AV an individual is responsible for just a single vehicle using an interface and controls that are directly linked (wired) to the AV. When in a ROC, the RO will likely be assigned to support multiple vehicles, which will require the development of new methods of supervision and vehicle management [21]. When in a vehicle the individual is able to build their situation awareness of what is going on around them using their own senses as well as feeds and information from the vehicle, whereas when in a ROC they are likely to suffer from a degraded state of situation awareness as they only have access to transmitted feeds from the vehicle [22]. Furthermore, those information feeds have been sent using a radio link or mobile communications network, which can suffer from bandwidth limitations and data transfer latency [23], meaning that the remote operator is being provided with delayed or incomplete information, further decreasing their situation awareness.

The impact of the changes, especially those resulting from data communication latency and reduced situation awareness, is that remote operation can be highly stressful, exacting a high cognitive workload leading to decreased performance and mental fatigue [24]. This potentially significant negative impact on the performance and capabilities of the teleoperator is of concern for organisations researching the implementation of remote operation. This was certainly true for the SCALE consortium [2], who determined that a primary objective of the project was to evaluate how the potentially negative issues around remote operation technology and associated human factors might change the role and task of the operator as they transitioned from in-vehicle SO to an RO in a remote location. This evaluation would in turn be used to research the human–automation interface requirements for the newly created RO position. Key research (and commercial) questions raised were:

• What tasks conducted as a safety officer (SO) could and would be undertaken remotely; what are plausible, what are viable, and conversely what should be avoided due to safety and performance concerns?
• What unexpected, or previously unnoticed tasks emerged as the SO was removed from the vehicle that needed to be addressed?
• Just how many AVs could a remote operator (RO) supervise and manage at a time? Are there factors that affect this ratio and, if so, how could the human automation teaming structure and interface design address these effects?

2. Methodology

2.1. Initial Comparison with Existing Surveys

Given the extensive research conducted on remote operations over the last two decades, the initial expectation of the SCALE project [2] researchers was that many of these questions would have easily accessible and collatable answers. However, it very quickly became apparent that whilst there were a number of contributing reviews or articles discussing what remote operation might involve (e.g., [22,25,26]), they did not provide direct or complete answers to our research questions. Furthermore, they tended to provide discussions on hypothetical or proposed solutions to remote operations, rather than provide recommendations derived from past attempts at conducting RO. For example, Cummings et al. [9] did provide a short review of the scale and scope of RO, identifying the needs and issues associated with the many new tasks, such as monitoring, remote control and goal based supervisory control. However, when they then modelled the impact of these new tasks, their evaluation was largely based upon estimates of task durations and workload. Furthermore, the modelling was based upon the premise that the new RO functions would be added to the job description of an individual employed as a fleet manager (and already fully tasked).

It was also observed that whilst there was plenty of discussion on what RO could be attempted, there was a shortage of discussion on whether those RO solutions should be attempted for safety, legal or other objections. For example, many of the papers considered as reviews had discussed the technologies required for remote driving and methods of facilitating or entering into remote driving (e.g., [27]); yet, few had considered whether it would be commercially viable to implement a driving capability that research had indicated would be inherently less safe that the in-vehicle driving capability it replaced (imagine the hypothetical legal situation of a commercial public transport operator having to defend in a court their decision to deliberately implement a driving solution that tests and trials had indicated was more error prone).

Some researchers do consider the legal aspects of implementing RO functions; in fact, the primary focus of Goodall [28] is to consider the legal as well as technical viability of conducting RO. Much of the paper is dedicated to identifying conditions and models to estimate how many personnel would be required to achieve the requirements for RO. However, Goodall [28] identifies that the assumptions on scale and scope of RO are so significant that their results are not particularly conclusive (the author estimate ranges from a few as 24 to as many as 1.4 million operators would be needed in the continental USA).

A number of the reviews came very close to providing the full scope of information needed. In a 2020 summary review of remote operations for the Economic Commission for Europe, Carsten [21] considered topics and challenges pertinent to answering the research questions posed. However, the review is short and is limited in its references, suggesting that it could be expanded upon. Kalaiyarasan et al. [29] conducted a more extensive review of the literature, providing discussions on the human factors resulting from the use of remote operation technologies and the human performance likely to be impacted by remote operations. They conducted interview workshops with remote operator stakeholders to establish what current approaches and perceptions of remote operations were prevalent, and what research gaps there were that needed addressing. However, much of the discussion was scoping and hypothetical; they identify what might be done and what issues from human factors there might be rather than recommend what could (and could not) be done based upon results from tests and experimentation. In their defence, this is likely a consequence of when they conducted the review and the empirical data available. As they themselves observed, at the time of the review, there appeared to be a paucity of information on the testing and thus the plausibility of remote operation activities. Were they to do the review now, perhaps they would have more information from which to make more conclusions and recommendations on implementing remote operations.

In contrast, Amador, Aramrattana and Vinel [30] provided a detailed discussion of research into the scope, technologies and human factors found to affect the three RO roles of remote driving, remote assistance and remote monitoring. However, the scope of their review is quite broad spectrum, with them attempting to discuss issues and challenges from not only human factors but also the telecommunication technologies and human machine design. Furthermore, as they themselves identify, at the time of their survey there was a scarcity of work and thus conclusions available on remote assistance and monitoring. Thus, their work, whilst comprehensive, could benefit from a narrowing of focus and an update and extension of work on remote operations other than remote driving.

Skogsmo et al. [11] take a slightly different approach in their review, seeking to collate and identify legal considerations for implementing RO in Europe. They do discuss technical and human factors concerns; however, they appear to be primarily interested in how regulation should be prepared so as to ensure remote operator organisations address these technical and human factor issues. Thus, whilst close to providing answers to the research questions posed in this review, they tend to provide the legal perspective and counterpoint rather than the empirical performance evidence for the adoption or not of any specific RO task or function.

Other reviews, whilst not providing direct answers to the research questions, did provide observations and arguments that affect the discussion on viability. Mutzenich et al. [22] and then Parr et al. [26] provide detailed discussion and proposed definitions of sub-categories of RO but do not evaluate the plausibility of those sub-categories. Favarò, Eurich and Nader [31] provide insights into autonomous vehicle disengagements, Guanetti, Kim and Borrelli [15] provide discussion over the technologies used to conduct RO, and Kamtam et al. [16] discuss a review of the literature on RO network latency and its effects and possible solutions. A summary of all reviews considered is provided in

Table 1. Summary of review of existing surveys of remote operation.

Year	Author	Title	Summary/Scope	Gap or Issue
2010	Chen, Barnes and Harper-Sciarini	Supervisory Control of Multiple Robots: Human-Performance Issues and User-Interface Design [32]	Review of FO and HMI primarily for UAV but also for UGV, observing that that FO is significantly impacted by the control task	Assumes RO is in command as supervisor and falls back and attempts all facets of RO (RM, RD and RA)
2014	de Winter, Happee, Martens, and Stanton	Effects of adaptive cruise control and highly automated driving on workload and situation awareness: A review of the empirical evidence [33]	Review of Workload and SA in partially automated vehicles	Useful for background on SA but primarily only applicable to Remote Driving and then at a low LOA (3-)
2018	Endsley	Situation Awareness in Future Autonomous Vehicles Beware the Unexpected [34]	Review of issues and concerns over degradation of SA when operating highly automated technology leading to slow reaction and poor decision making	Written from perspective of human as fall back driver and recommends against intermediary LOA
2018	Favarò, Eurich, and Nader	Autonomous vehicles disengagements: trends, triggers, and regulatory limitations [31]	Attempt to catgorise and spot trends in causes of AV voluntary or SO-initiated disengagements	Focuses on problem but not RO solutions
2018	Guanetti, Kim, and Borrelli	Control of connected and automated vehicles: state of the art and future challenges [15]	Discussion over technologies for CAV rather than human factors	Little in the way of discussion of RO
2019	Carsten and Martens	How can humans understand their automated cars? HMI principles, problems and solutions [35]	Focuses on HMI design for AV functions when the driver is inside the vehicle, which could be laterally applied when SO becomes RO	Does not consider difference in SO to RO, nor multi vehicle RM
2020	Carsten	Human Factors Challenges of Remote Support and Control A Position Paper from HF-IRADS [21]	Discussion on how the change in environment between SO & RO leads to differences in tasks and then HMI	As the discussion is more of a literature review, it is a bit short and has limited references.
2020	Cummings, Li, Seth and Seong	Concepts of Operations for Autonomous Vehicle Dispatch Operations [9]	Discussion on how future dispatchers will need to have the new task of being the RO teleoperators included in the job description	Starts from assumption that RO is to be added to existing dispatcher workloads and does not consider RO as a separate position
2020	Goodall	Non-technological challenges for the remote operation of automated vehicles [28]	Legal, technical and manning factors affecting RO in the USA	Does not provide recommendations based upon empirical data
2020	Marcano, Díaz, Pérez, and Irigoyen	A Review of Shared Control for Automated Vehicles: Theory and Applications [36]	Comprehensive review of the literature on engineering formulae and solutions for implementing a model of human and AV sharing driving controls in-vehicle	Centred on in-vehicle sharing of driving controls rather than remote operation and limited to RD (no RA or RM application)
2020	Xing, Huang, and Lv	Driver-Automation Collaboration for Automated Vehicles: A Review of Human-Centered Shared Control [37]	Comprehensive review of the literature on engineering formulae and solutions for implementing a model of human and AV sharing driving controls in-vehicle	Centred on in-vehicle sharing of driving controls rather than remote operation and limited to RD (no RA or RM application)
2021	Kaliyarasan, Simpson, Jenkins, Mazzeo, Ye, Obazele, Kourantidis, Courtier, Wong, and Wilford	Remote Operation of Connected and Automated Vehicles [29]	Project Review of how to implement a remote Safety Driver and Test Assistance	Observations and discussion very hypothetical identifying gaps and possibilities for RO
2021	Mutzenich, Durant, Helman, and Dalton	Updating our understanding of situation awareness in relation to remote operators of autonomous vehicles [22]	Discussion on cases for human intervention, types of intervention possible (RA, RMgt, RD) and SA demands for those cases	Focuses on defining but does not get to examining whether the RO sub-categories are plausible
2022	Amador, Aramrattanan, Vinel	A Survey on Remote Operation of Road Vehicles [30]	A broad but by necessity shallow review of scope, technology and challenges for RO	Too broad in scope to answer research questions and needs updating
2022	Majstorovic, Hoffmann, Pfab, Schimpe, Wolf, and Diermeyer	Survey on Teleoperation Concepts for Automated Vehicles [25]	Attempt to categorise concepts and methods for human operators to team and interact with automation that helps separate RD from RA	Primarily concerned with attempting to create categories of RO rather than determine whether any specific method is more viable
2023	Skogsmo, Andersson, Jernberg, and Aramrattana	Remote Operation of Multiple Vehicles [11]	Reviews legal perspective on viability of implementing RO tasks and sub-groups	Tends to make recommendations purely from a legal rather than practical perspective
2024	Musicant, Botzer, and Richmond-Hacham	Safety, Efficiency, and Mental Workload in Simulated Teledriving of a Vehicle as Functions of Camera Viewpoint [38]	Review of effect of driving viewpoint on situation awareness, workload and general performance of remote drivers	Primarily aimed at preparing an experimental study into whether changing the viewpoint of a remote driver could improve their performance and workload
2024	Parr, Harvey, Burnett, and Sharples	Investigating levels of remote operation in high-level on-road autonomous vehicles using operator sequence diagrams [26]	Proposes sub-categories of RO with definitions and task scope and then scenario tests them using operator sequence diagrams	Very useful at providing RO sub-task work scope proposals but could do with expanding and experimental evaluation
2024	Zhao, Nybacka, Aramrattana, Rothhämel, Habibovic, Drugge, and Jiang	Remote Driving of Road Vehicles: A Survey of Driving Feedback, Latency, Support Control, and Real Applications [39]	Review of papers related to methods and technology for providing driving feedback through controls.	Focused on technological issues and solutions for remote driving
2025	Wolf, Krauss, Schmidt, and Diermeyer	Control Centre Framework for Teleoperation Support of Automated Vehicles on Public Roads [27]	Contains a review of remote operator literature to generate and define discrete roles and tasks for ROs	Does not address value, preference, or HF of each type of RO

below:

2.2. Search Process

The PRISMA statement [40] was used as guidance for the process of conducting the selection process. As the search was primarily to identify issues and concerns and collate information on human factors, the emphasis was on conducting a qualitative review rather than simply a quantitative analysis.

The searches were conducted primarily to seek academic or scientific journal papers on research being conducted on the RO of ground- or road-based AVs. However, as there is substantial commercial and governmental interest in the RO of AVs, and much of that interest can have a direct effect on the research conducted (e.g., this RO research was conducted as part of the SCALE [2] project, which is funded by the UK Department for Business and Trade), it was determined to make the selection more heterogeneous. As such, commercial standards, commercial web-sites and government consultancy work were also included for consideration within the scope of the initial search.

It had been intended to limit the search and review to literature covering the provision of RO to just ground-based AVs. Literature on autonomous and uncrewed vehicles operating in other environments was (initially) not to be considered. However, the review of the articles collected from the first few searches made it apparent that there was a shortfall in contemporary research on the number of AVs a single operator could manage (also known as the “Fan-Out” ratio of multiple vehicles to single operator).

It was known to the authors that there existed earlier (legacy) research on calculating and estimating Fan-Out requirements and numbers for generic autonomous “robots” and Uncrewed Aerial Vehicles (UAVs) that are known to have a high level of automation in the form of “auto-pilots” built in by design. Therefore, after the original planned search on literature pertinent to ground AVs was completed, a second search was conducted using the expanded criteria of including papers that discussed an individual supervising multiple generic “Robots” and/or Uncrewed Aerial Vehicles (UAVs). However, simply as a consequence of the iterative expansion of the scope of the search, the new search was not at the time extended to include research on maritime vehicles or other forms of non-autonomous remote operation. Thus, maritime vessels were not deliberately excluded by design, rather they were simply not included by default.

Thus, the process to search for, gather and select papers and articles for inclusion was conducted in two sets of back-to-back searches. The generic process for each search was identical; the difference between them was the expansion in the second search of the selection criteria to include older UAV/Robot research literature.

2.2.1. Stage One—Initial Searches

Two primary methods of search were used:

• Database search: a search of well-known research databases using meta-search engines such as “Locate”, the Coventry University Library search tool, and Google Scholar to search across core publishers (e.g., ACM, EBSCO, Gale, IEEE Xplore, MDPI, ProQuest, ScienceDirect Elsevier, Spinger, etc.) and publicly accessible research databases (e.g., BASE, CORE, Science Gov, refseek, ResearchGate) using keywords in

  Table 2. Keywords and keyword combinations used for searches.

  Lead	Secondary	Context	Optional
  Remote	Operation(s)	Automated Vehicle	Passenger
  Tele	Control	Connected and Automated Vehicle	Situation Awareness
  Takeover	Communication	AV	Workload
  	Supervision	CAV	Human Factors
  	Monitor/Monitoring	Autonomous Vehicle	Human Machine Interface
  	Driver/Driving	Autonomous Mobility	Fan-Out
  	Assistance/Assisting

  below, either initially identified by the authors or latterly observed as common or prevalent within the papers being viewed;
• Cross-reference search: a secondary cascade or “snowball” method of reviewing references and citations found in the peer review articles discovered in the database searches (above).

Initial Search Database Search

Multiple searches were conducted across all the databases and search engines using combinations of the keywords in Table 2. Many of the meta-search engines would generate thousands of results (e.g., “remote operation of automated vehicles” generated 72,457 results from CORE, and 529,000 results from Google Scholar). Therefore, it was determined for practical purposes to limit the initial review of those massive search returns to either the first 500 (the approximate average number of articles given for a search using the Coventry University library search engine Locate) or until the page/list being displayed no longer provided any new (not previously collected) and relevant suggestions.

It was observed that many of the articles reviewed would appear in many of the database searches, i.e., there was a substantial overlap in the sets of search results. It was also observed that many of the articles presented by the databases were not relevant to this review, often because the search engine had included them based upon just one of the keywords (e.g., automation), and as a consequence they were not directly relevant to the research question or topic of the paper (e.g., they covered remote operation of manufacturing, laboratory equipment or even medical operating theatre equipment).

In the database, search papers were to be positively selected for further detailed review primarily based upon their perceived relevance to the three research questions and overall remote operation topic. A two-stage selection process was followed. The first stage of the selection process took place during the implementation of the keyword search where a decision on inclusion for further review was made based upon the content of the title, abstract or summary as follows.

Initial Search Inclusion Criteria

The aim of the review was to identify literature that could provide novel observations, insights and results that would assist in answering the research questions posed:

• What tasks conducted as an SO could and would be undertaken remotely; what are plausible, what are viable, and conversely what should be avoided due to safety and performance concerns?
• What unexpected, or previously unnoticed tasks emerged as the SO was removed from the vehicle that needed to be addressed?
• Just how many AVs could an RO supervise and manage at a time? Are there factors that effect this ratio and, if so, how could the human automation teaming structure and interface design address these effects?

Thus, the primary criteria for inclusion was the question “does the article provide information on human factors of SO or RO that addresses the research question?” As the search was intended to be exploratory and evolutionary, no inclusion/exclusion criteria were set on pre-determined qualitative categorisation or data coding of articles (this could lead to a bias to select articles on known human factor issues or solutions). However, to assist in evaluating whether the articles answered the research questions, two screening criteria were used during the database search to positively identify papers to be selected for further detailed review as follows:

• The title, abstract or summary indicated the paper discussed human factors associated with the driver or safety officer (SO) of an AV for a task/activity that an RO might reasonably anticipate as being carried over to future remote operations. The primary activity looked for was the reactive takeover of the direct driving task (either forcefully taking control from the AV or taking over at the request of the AV), as it is envisaged that in some commercial models the RO could, at times, be expected to undertake the duties of a standby driver or safety officer. However, articles discussing human factors of other driving or SO-related tasks were also considered;
• The title, abstract or summary indicated the paper discussed human factors or human performance of remote operation of automated or autonomous ground (or road) vehicles.

Initial Search Exclusion Criteria

Papers where the title, abstract or summary indicated the article focused solely on AV technology solutions were excluded unless there was an indication that a human user trial of the technology had been conducted and had provided results, or there were reflections on the impact the technology might have on human performance.

As the review was aimed at identifying human factors that are not necessarily dependent upon the currency or level of technology (or IT), no specific exclusion criteria or limit was set on the date of publication of the papers in the searches conducted. However, initially preference was given to publications made within the last decade (2015–2025).

Initial Search Cross-Reference (Snowball) Search

In addition to the primary search using meta-search engines, the authors also consulted the reference lists of articles selected for inclusion (effectively taking advice and consultancy from our peers). Particular attention was given to the references found in the literature listed in Table 1.

Thus, the search process was effectively iterative, with the results of initial searches providing additional sources and lists of authors likely involved in research of primary interest.

2.2.2. Stage Two—Article Review and Sift

In total 257 articles were positively selected based upon the content of their title, abstract or summary. The full text of the collected articles was then subject to a detailed review and analysis. Two refinement selection sifts were conducted (see Figure 1) with articles positively eliminated in each (i.e., literature was excluded from the review during the two sifts).

Review Inclusion Criteria

As with the initial searches, the primary criteria for inclusion were the ability of the paper to provide novel observations, insights and results that would contribute towards answering the research questions posed. Therefore, the inclusion criteria from the initial searches were used. As before, the aim of the review was to proactively look for literature that could provide observations on human factors effecting the ability to conduct RO tasks. However, given that at present the research and implementation of RO is in its infancy, in addition, papers that provide information on in-vehicle safety officer (SO) activities that were considered relevant to future remote operations (ROs) (i.e., tasks that could be reasonably expected to be transferred from SO to RO) were also used as primary sources. Furthermore, given that human machine interfaces (HMIs) are generally designed to support or enable a human to carry out a task, papers that discussed practical evaluation of an RO HMI or the requirements for an RO HMI were also sought.

Review—Additional Exclusion Criteria

Whilst the same inclusion criteria from the original search of title, abstract or summary were used, two new exclusion criteria were used, one per sift.

As the aim of the article was to answer the human factor questions, in the primary sift, papers focused on only discussing remote operation technologies; technology solutions were excluded unless there was an indication that a human user trial of the technology had been conducted (and there were results or observations on human behaviour) or there were reflections on the impact the technology might have on human performance. In total 74 article were removed.

As the aim of the article was to determine reflective observations on practical viability over theoretical possibility, in the secondary sift, scientific papers that did not include results or reference results from an experimental study (qualitative or quantitative) were excluded (this criterion was not applied to the more heterogeneous papers such as standards and policy documents). Novel and original data was sought; therefore, articles in series that appeared to duplicate information rather than provide novel perspectives were removed. In total a further 33 articles were removed during the secondary sift, leaving 128 journal articles and 12 commercial documents.

2.3. Paper Structure

There are three primary sections to the Discussion section of the paper. The first section (1) prepares the reader for the latter discussions by reviewing and providing a reconciled view of the multitude of terms used to discuss remote operations. The aim is to establish, just for use within this paper, a consistent baseline set of definitions and context to the terms used to describe tasks and sub-categories of remote operation.

The paper is organised and structured as shown in Figure 2 below.

Section 2 provides the core discussion on the roles or types of RO that have been identified as necessary to support an AV and more frequently discussed in the standards and papers reviewed. The section is itself then sub-divided into four parts. The first part is an opening discussion on generic remote supervision that primarily leverages legacy research and provides both a discussion on the concept of “Fan-Out” and the background on the total scope and core human factors of remote operation. The remaining three sections then consider in turn each of the three core types of RO identified within commercial standards [41,42]: remote monitoring (RM); remote driving (RD); and remote assistance (RA). The Section 2 discussion attempts to answer the question: what practical gains and issues have been found when attempting to implement these roles and what does that mean for their commercial viability?

Finally in Section 3, the paper discusses concerns raised in reviewed papers and by the authors on the shortfalls of research and implications of the findings from Section 2 on future research, the aim being to identify research gaps and make recommendations on future research needs.

Throughout the paper core observations and recommendations are collated and provided in summative tabular form within each section and as an attachment after the conclusions. These observations and recommendations include propositional positions of the authors on the viability of the three sub-tasks of remote operation.

2.4. Contributions

To the best of our knowledge there did not appear to be a comprehensive study into the plausibility and viability of the many different roles and tasks being proposed as within the scope of remote operations. As a consequence, it was determined to carry out a comprehensive and systemic review of research and literature on practical attempts to simulate or conduct remote operations with an intent of identifying the key human factors affecting the plausibility and viability of implementing RO as a solution to supporting commercialization of AVs. The review would achieve this by examining available research that provides empirical data on attempts to conduct RO in support of AVs, alongside research either proposing hypothetical solutions or identifying research gaps.

This discussion would be used to either answer the three research questions posed earlier or identify the research gaps needed to provide those answers.

The primary novel contributions of this paper are:

• A discussion on terms and labels for RO in common usage, with the aim of assisting readers understand and identify potential discord and mis-use of terms;
• An exploration of the recent literature on RO used to update of the findings of prior peer reviews and provide an updated “current state of science” of research into RO;
• A position, derived from the review of existing literature, on the point of separation between remote driving and remote assistance based upon command and control of the direct driving task;
• A position, derived from the review of existing literature, on the plausibility and viability of implementing the primary or standard RO positions (remote monitoring, remote driving and remote assistance);
• A proposal of the primary knowledge and research gaps that, when addressed, are likely to have the most impact on future design and implementation of remote operator roles.

In addition, when conducting the review, it was identified that whilst some researchers were investigating the scenarios and situations that have and could lead to AVs requiring fall-back driver support (e.g., [5,26]) and possible support or technology solutions [43,44], there could be gap in research being conducted to evaluate and anticipate future disengagements and how they might be handled for handover to remote operators. Finally, it was observed that there was also an apparent shortfall in research into what new non-driving support requirements could emerge as the driver was removed from the vehicle, that is non-driving support to those on-board the vehicle and non-driving support to the vehicle itself.

This review paper will discuss, in detail, aspects of these knowledge challenges by providing an evaluation of the current status and findings of available research on remote operations with an intent to identify important research gaps that need addressing to support the commercial implementation of CAV systems. A summary of observed knowledge challenges is presented in

Table 3. Connected and automated vehicles—implementation knowledge challenges.

Challenge Category	Identified Issues	Impact/Implications
Operational Scope	Scalability: Lack of knowledge of how many vehicles one operator can manage RO Role Time and Workload impact on FO and practical viability RO Role Combination: Lack of knowledge on how to safely combine tasks and human factor effects of task combinations Task Transitions: Lack of framework for methods and limitations of switching between roles and for team structures and interactions (including Human Automation Teams)	Unclear workforce planning Workforce size and team structure Work allocation Unknown capacity limits Inefficient resource allocation Uncertain commercial viability
Limitations of Current Standards	Standards focus on vehicle operation technology Incomplete knowledge of disengagement scenarios and associated human support costs Insufficient guidance on safety and feasibility evaluation Lack of commercial frameworks No research roadmaps for public implementation	Gap between standards and practical deployment Limited support for commercial decision-making Unclear path to public space integration

below.

3. Discussion

3.1. Part 1—Remote Operation Terms and Titles

What Is Really Meant by Remote Operation?

Initially, researchers tended to discuss remote operations as a generic all-inclusive activity using simple terms such as supervision, supervisory control and teleoperation. These terms can be frequently found in older literature discussing research on human interaction with uncrewed aircraft or ground vehicles [45] or interaction with robots or robotic systems [46].

However, as studies and reviews exposed the considerable scope of activities that a remote operator could be asked to undertake, some authors attempted to produce taxonomies that grouped this slightly bewildering myriad of possible sub-tasks into discrete functions (e.g., [26,27,29]), or developed proposals for how individuals might be engaged to undertake these sub-tasks for specific fleets and types of operations [9]. Whilst much of the literature reviewed clearly identifies similar sub-tasks and uses similar terminology, there are discrepancies and thus disagreement between authors. For example, Parr et al. [26] and Wolf et al. [27] have very similar scopes and definitions for the terms Remote Monitoring (observation and/or surveillance of automated vehicles) but have quite different definitions of other common terms. They both use the terms Remote Assistance and Remote Driving. However, Parr et al. [26] identify Remote Assistance as a service provided to the vehicle user, whereas Wolf et al. [27] suggest that the remote operator would not interact with the user, but rather that this function would be carried out by a separate individual acting as Fleet Manager. Wolf et al. [27] identify Remote Assistance as only being provided to the vehicle, while Parr et al. [26] use the term Remote Management to describe operator assistance to the vehicle. Parr et al. [26] identify Remote Driving as including Remote Control, whereas Wolf et al. [27] identify Remote Driving as a sub-task of Remote Control. Wolf et al. [27] and Kalaiyarasan et al. [29] both identify a second remote control task of Remote Intervention (although Kalaiyarasan et al. [29] label it “remote emergency intervention”), where the remote operator can forcibly take over control of the AV. However, Parr et al. [26] do not appear to even consider a situation where an individual would forcibly take over control of the vehicle; in all their scenarios the AV initiates the disengagement and handover of control. Some researchers even further confuse the discussion by creating labels for remote operation that appear to merge functions together; for example, Vreeswijk et al. [47] (p. 149) identify a concept of remote support that “involves a remote (human) operator providing instructions, permissions or way points to the vehicle, or remotely driving”.

Recently, in what seems to be an attempt to provide some unity and standardisation over use of terms, two groups of commercial organisations associated with producing industry standards have established and published a baseline of definitions that group, name and classify the scope of tasks that can be considered remote operations. As well as providing a five-tier framework defining the Level of Automation (LOA) of an Automated Vehicle [41], the American Society of Automotive Engineers (SAE) and partners also produced recommendations for practice on driving automation systems that included definitions of sub-sets of remote operations (see [41,48]).

In the UK, the Centre for Connected and Autonomous Vehicles, a UK government agency established to lead the development of policy and regulation for self-driving vehicles (i.e., Avs), has sponsored the production of a range of standards or guidelines for industry derived from academic research and best practice experience. These are published by the British Standards Institute (BSI) as Publicly Available Specifications (PAS) or Flex Standards. The CCAV BSI standards are published as a suite of documents within the range BSI 1881 [49]–BSI 1891 [50] that include a vocabulary and series of definitions of terms.

From these standards it is possible to identify a single collective overarching term Remote Operation (conducted by a remote operator) and three common primary categories or types of remote operation:

• Remote Monitoring (RM). The observation and oversight of one or more AVs, their behaviour and performance, progress on a journey, safety and security of passengers, and any passengers, cargo or load.
• Remote Driving (RD). Real-time and direct control of the vehicle, extending to conducting part or all of the direct driving task.
• Remote Assistance (RA). The provision of information or advice to an automated driving system-equipped vehicle or the passengers of the vehicle to facilitate trip continuation.

The two terms RM and RD do generally appear to be used consistently; however, there is apparent disagreement within the standards over the full scope of the subject of remote assistance; they tend to disagree on who it is that receives the attention of the RA. The BSI 1890 [42] (p. 13, 3.1.89) recommends that remote assistance covers “the provision of information, advice or guidance to either a connected and automated vehicle, its occupants, other road users and other external agencies interacting or needing to interact with the vehicle”. However, the SAE identify assistance and guidance provided to the customer to be under the auspices of a Customer Support function, although they do note that “A RA(s) may also perform other fleet operations functions”. For this review the wider BSI version of remote assistance will be used instead of the more specific but open version proposed by the SAE. This is because some researchers have observed that there could be legal obligations for remote operators to interact with passengers to communicate safety-critical information in situations where the AV is conducting a minimum risk manoeuvre (MRM) [20].

Despite the publication of these definitions, academic and scientific literature continue to use the legacy terms such as remote supervision and teleoperation to discuss remote operations (e.g., [24,27,51]).

However, the scientific literature examined does not appear to present a singular consolidated definition for the terms supervision or teleoperation, with the consequence that it is difficult to determine the exact scope of tasks that a supervisor or teleoperator could be expected to undertake. From the literature examined, it would appear that:

• A Supervisor (or someone conducting Remote Supervision) is an individual in a responsible and commanding position who could have a wide range of loosely defined tasks, monitoring vehicles (remote monitoring) and taking actions (remote assistance or remote driving) to improve AV safety and efficiency [11]. The primary value of the term is that it implies that the individual, the Supervisor, is the team lead and has oversight or command responsibility (e.g., “supervisory control” by Schitz et al. [52] (p. 174), Georg et al. [23] (p. 915)).
• Teleoperation is technically used to identify that the human is using some form of telecommunication data link (e.g., radio or Wi-Fi) to send information and directions to the AV. However, within many research papers, the term teleoperation appears to be synonymous with remote driving in a much broader context; both the type of remote operation meant and the type of vehicle supported. It continues to be used by many contemporary authors to describe using telecommunications to directly control vehicles both with and without automation. Some authors also seem to include remote assistance within the scope of teleoperation (e.g., [53]). Thus, the expectation of what is meant by teleoperation is inconsistent across the literature could lead to significant misunderstanding of context. Parr et al. [26] note that the term teleoperation is not defined consistently within the literature and instead elect to use the more standardised term remote operation.

In this paper we will use one of these latter terms: Supervision as an overarching term to indicate a hierarchical position where the individual conducts all possible RO tasks, in particular monitoring for a control intervention. We will also use the four standardised terms, Remote Operation, Remote Monitoring, Remote Assistance and Remote Driving, as defined by the British Standards Institute in [54]. However, as the scope of the term teleoperation appears to be somewhat ubiquitous and is frequently used to discuss remote operation of non-automated land, sea and aviation vehicles, it can be both confusing and misleading. Therefore, as per the example of Parr et al. [26], for this review its use shall be avoided and the term remote operation used instead. That is not to say that we totally disagree with the term teleoperation; rather, we prefer to use the appropriate specific “remote” sub-task (Remote Monitoring, Remote Assistance or Remote Driving) mentioned in AV remote operation standards to make explicit the context of discussion.

3.2. Part 2—Remote Operation: An Analysis and Position

With the publication of the many RO standards, it appears that the requirements, concept of operations and thus scope for each sub-category of remote operations have already and quite firmly been established. However, whilst the standards are relatively consistent in the aim and definition of the three RO roles (Monitor, Assistant or Driver), they do not necessarily provide any indication of either the practical viability (i.e., are they practically possible), nor the total scale of service provided by each of the roles (i.e., how many vehicles could be Monitored, Assisted or Driven at a time). Neither do they assess whether and how roles might be combined for practical or commercial advantage (i.e., they do not indicate which combination of the three tasks an individual supervising or supporting AVs might be expected to undertake, and how they might transition between tasks or job share with other human or artificial team members).

Furthermore, the standards tend to be focused on providing practical advice on technologies directly associated with operating a vehicle. They do not necessarily move beyond this into providing advice on evaluating what remote operations and RO roles are safe and feasible (for example some researchers argue against implementing remote driving given practical and human factor limitations) and how one might utilise them to commercialise and use AVs to achieve goals and missions. They also do not provide guidance on what further research needs to be conducted to facilitate the implementation of AVs in the public space. Therefore, the remainder of this review will focus on looking beyond proposed hypothetical concepts and examine the findings of researchers directly or indirectly attempting to implement remote operator roles and tasks. The review will attempt to provide an answer to a core question: which of the proposed concepts of remote operation are not just possible, but also viable and practical?

3.2.1. Remote Supervision: How Many Vehicles Can We Manage?

The primary motivation for enterprises to transition from an onboard driver supporting an AV to a remote operator seems to be to allow them to leverage commercial economies of scale. Enabling a single operator to supervise multiple AVs simultaneously should reduce labour costs [12], providing AV commercial organisations utilising remote operations with a competitive advantage over traditional in-vehicle driver employment. However, before an AV commercial organisation can just jump in to remotely operating a sizable fleet of AVs, an immediate and practical question that they need to answer is “How many AVs can be supervised safely by an individual at the same time?”. And what exactly is meant by multiple?

This question appears to have been troubling researchers for quite some time. A substantial portion of the research into remote operations over the last three decades has been directed at determining what is the optimum number of vehicles that can be operated simultaneously (e.g., [55,56,57,58,59]). Whilst the older and more substantial portion of the research was conducted on Unmanned Aircraft Vehicles (UAVs) or more generic “robots” the results are still relevant and often transferable to the remote operation of AVs. Therefore, this review shall draw observations taken from the literature on generic “Robots” and UAVs, with the necessary caveat that the discussion and determinations drawn are based upon evidence from empirical studies into RO supervision of vehicles not necessarily on the ground or in road traffic conditions.

In their own review of literature on supervision of multiple robots, Chen, Barnes and Harper-Sciarini [32] identified 11 different studies in which researchers had drawn conclusions on an ideal number of vehicles or robots an individual could manage. They used the term Fan-Out (FO) to describe the number of vehicles or robots that an individual can simultaneously operate effectively (see [60] for a definition and discussion on Fan-Out). Of the 11 studies, seven tested FOs between one and four, and the remainder tested ranges that extended wider, often between four and 12. The results were equally diverse, with some researchers identifying optimum supervision occurring at FOs as low as just two [61,62], and others indicating that up to 12 could be managed without degradation in performance [56,63]. Chen, Barnes and Harper-Sciarini [32] (p. 442) summarise “that FO can be as high as around 8–9 robots in less demanding tasking environments; however, in more challenging conditions, human operators can only supervise around 4–5 robots effectively simultaneously”, observing that this latter figure is very close to the classically proposed “magical number of seven plus or minus two” [64].

The reason for this variance is that human supervision of a collection of moving vehicles in a dynamically changing environment is complex with high levels of uncertainty, highly variable operator workload, and often intense time pressures on all decisions and activities [65]. A consistent finding of researchers [56,59,66] is that as the size of the collection of vehicles increases so the workload also increases, and that in turn may lead to the human supervisors failing to maintain adequate situation awareness (SA) when their attention is constantly switching between the vehicles.

In 1908, Yerkes and Dodson [67] proposed that when undertaking a task, there was a relationship between the amount of stress an individual was under and the quality of their task performance; that relationship follows a curve shaped like an inverted U. At the beginning of the curve, when stress is increased, the quality of performance also increases. However, as stress continues to increase, there comes a time when the associated increase in performance first slows and then levels off. Finally, if stress is increased too much, the quality of the performance of the individual reverses and actually starts to decrease (the downside of the inverted U). Cummings and Guerlain [63] observed that when controlling multiple UAVs, when participant workload (measured by activity) exceeded 70% of available time, performance in supervision significantly degraded, and they equated this degradation to the performance drop-off expected when at the latter part of the Yerkes Dodson inverted U curve [67].

This latter observation would appear to provide the first indication of a measure and thus step towards determining the viable FO of supervision. Specifically, it appears that researchers (e.g., [60,63]) imagine that to calculate the approximate gross maximum possible FO, all one must do is identify the work demand as a % of available time per vehicle and multiple that figure by integers (vehicles) until workload reaches 70%.

However, this simplistic calculation fails to take into account the significant workload demands of cognitive and teaming activities associated with supervision. It also does not account for unpredictable time demands such as that required to build SA, react to situations, to form and take decisions, and to communicate with other members of the supervisory team. Nor does it account for factors likely to affect the subjective stresses and frustrations of the tasks, such as working in a safety-critical situation or working with non-transparent automation. Finally, and potentially most importantly, it fails to account for any surge in workload and time demand that would occur when the supervisor is required to concentrate and solve complex problems for unique and challenging edge cases for which there is no simple “off-the-shelf” solution.

In summary:

• Legacy research into optimal FO is largely inconclusive, offering findings of FO as low as two or as high as 12;
• When workload exceeds 70% of available time, performance in remote supervision significantly degrades;
• Simplistic FO calculations based upon time on task divided by time available fail to take into account the significant human factor effects of stress and teaming.

Note on Review Limitations: It is important to emphasise that the discussion and conclusions on FO are primarily drawn from research into the operation of generic robots and UAVs with incidents and scenarios appropriate to their environmental conditions.

3.2.2. Remote Supervision: The Effect of Level of Automation

A common theme across the older robot and UAV studies is that the FO is linked to the Level Of Automation (LOA) of the system that they are supervising. This observation is explained by Cummings and Mitchell [68] (p. 453): “One of the primary variables that will influence the operator capacity in the supervision of multiple vehicles is the level of system autonomy”. The linkage identified is that the higher the LOA of the system, the lower the workload, the more spare capacity the operator has, and thus the higher the FO possible. However, Ruff, Narayanan and Draper [55] also observed a concern that when using a higher LOA during normal operations Ros tended to be removed from the task, inactive and thus poorly practiced, leading to poor performance when required to intervene during periods of automation failure. Endsley [69] (p. 8) later succinctly labelled this inverse relationship between LOA and take-over performance “the automation conundrum”, explaining:

“The more automation is added to a system, and the more reliable and robust that automation is, the less likely that human operators overseeing the automation will be aware of critical information and able to take over manual control when needed.”

Another central concern on how LOA affects remote supervision is that expressed by Bainbridge [70], that in general when designing a system, it is the easier and more commonly used functions that are automated. This has two consequences: the operator now has fewer opportunities to practice the manual tasks they are expected to conduct; and the problems they will be expected to solve will be more infrequent, unusual and likely difficult edge cases. The impact of implementing higher LOA is thus to reduce the human task to handling problems that can only be solved by practiced and experienced experts, but that same higher LOA would result in the supervisor being less practiced and less experienced and therefore likely to have less capability to solve any problems. Ultimately, the concern is that increasing LOA could compromise safety-critical operator competency during remote operations, while simultaneously increasing the likelihood that the situation encountered by the operator is a hazardous, difficult to solve edge case.

Other robot and UAV researchers offer an additional concern about implementing a high LOA to increase supervision FO. When examining studies into human supervision of multiple vehicles (both those found for this review and those cited by Chen, Barnes and Harper-Sciarini [32]), it becomes evident that it is not necessarily the LOA in normal circumstances that has the determining effect on size of FO, but rather it is the LOA to which the system falls or degrades when an issue occurs that is the determining factor for FO. The lower the LOA, the more intervention is needed from the RO and the less capacity to Monitor, Assist or Drive other vehicles simultaneously.

Indeed, much of the robot, UAV and AV research into FO reviewed for this report appears to be examining methods by which to reduce the workload for the operator during periods of intervention or to reduce the effect of automation degradation so as to counter the issues raised by the automation conundrum and thereby increase the level of FO that an individual can manage.

Early UAV researchers (e.g., [55]) attempted to overcome the issue by making sure that even when degraded the automation remained at a high LOA, providing decision support advice and recommendations for action for the human operator to select from. In this proposed solution the human is never required or able to provide a direct manual control of the system or robot, they act as an RA not an RD.

Ruff, Narayanan and Draper [55] found that the optimum decision-support LOA for consistent performance between FO was the relatively high “Management-By-Consent”, where the automation presented options, made a recommendation, but then waited for operator selection. Parasuraman et al. [71] developed a similar but more operator-centric scheme with comparable success improving performance at higher FO. They identified their scheme as “Playbook”, where in an exceptional situation requiring human intervention, the human operator would select from a pre-determined series of “plays” or activity sequences and task the automation to carry out that play. The primary difference between the two models appears to be that in Management-By-Consent the automation generates the shortlist of options, whereas in the Playbook the human does it.

Later researchers set about determining whether varying the LOA of the robots or vehicles could help offset any likely reduction in performance as FO was varied [72]. They took a slightly different approach to the research, examining whether it was possible to maintain an optimal level of operator performance by dynamically adjusting the LOA or use of automation in response to the situation. However, Calhoun et al. [73] found that performance was generally better when LOA was globally consistent across functions rather than varying across functions. They found evidence that participants made more “mode errors” in the mixed LOA configurations, apparently forgetting which LOA the system was at for specific functions. Kidwell et al. [74] continued this evaluation of variable LOA using the same experimental apparatus, but this time examined whether dynamically varying the global LOA in time rather than by function would improve performance. They evaluated the difference between giving the operator control of LOA variance against the automation having control of LOA variance, finding that automation control of LOA variance lead to the higher operator performance.

However, many of these variable LOA schemes for increasing FO are generally founded upon generic robot or UAV scenarios where operator intervention is the exception rather than the norm. This does not necessarily account for all systems design, particularly ground-based systems in crowded environments and situations where a human has a specific and constant task, such as taking decisions for legal purposes. Lewis et al. [56] addressed this case in a study where an Uncrewed Ground Vehicle (UGV) was used to carry out search and rescue activities and the human remote operator was required to make a decision on whether an object found was an individual needing to be rescued. What they observed was the highest effective FO of eight occurred when a high LOA was implemented for much of the search task (navigation and search pattern), leaving the human to focus on deciding whether an object was an individual to be rescued or not. Thus, rather than setting a binary system where either the automation or human was in control, they created a human autonomy teaming (HAT) system where the automation and human were each assigned separate tasks but had to work together to complete the mission. However, even in this design, Lewis et al. [56] observed that operator workload continued to rise with FO, ultimately limiting the FO level.

In summary:

• The higher the LOA of the system, the lower the workload and more spare capacity the operator has and thus the higher the FO possible;
• The more consistent the LOA the better the human performance (and conversely, the more variable the LOA the more errors made from forgetting what LOA the system was at);
• Even with high LOA, as FO increases so does operator workload, ultimately limiting the FO.

3.2.3. Fragmenting Remote Supervision into Sub-Tasks: The Control Task Factor

In all robot, UAV and AV FO case studies reviewed, workload was consistently a limiting factor. This is likely because the one thing that is common across all of these studies and systems is that irrespective of architecture of the system there remains a central requirement for the RO to both monitor and control the path of the vehicle (equivalent of Driving). In all of the legacy FO research cases, the operator is either required by design to take part in the control task or required to intervene during periods of automation degradation. The issue with this requirement is that the controlling task (in AV driving or assisting) inevitably requires a momentary singular focus (i.e., the individual has to focus exclusively on just one vehicle) and takes a finite amount of time. During this period of direct control the operator’s cognitive and physical resources are challenged. This reduces or even removes their ability to monitor let alone control a second, third or more vehicles. Indeed, recent AV research [75] has shown that attempting to provide the control task to two or more vehicles simultaneously (even when the task is simplified from driving to assistance) will result in a degradation of performance. Colley et al. [75] showed that operators attempting to manage multiple simultaneous requests for control were more likely to miss AV demands for attention, and even when able to drive were more likely deviate from their lane.

Researchers have evidentially been aware of the impact that the control task has on FO, with various researchers (e.g., [60,68,72]) attempting to derive a formula to model the relationship between the Interaction Time (control) and Neglect Time (monitoring) and even the Wait Time (a combination of the time for decision-making, of waiting for a control task to succeed and of time spent re-building SA when returning to general monitoring). More recently, Goodall [28] suggested using the queuing theory Erlang C formula used by many call centres. Whilst constructing the formula can be useful as it forces researchers to quantitatively consider how human factors can affect the FO, the resulting equations derived by researchers were often complex and difficult to calculate [21] and only had limited success in making consistent and reliable FO predictions. Goodall [28] observes that by varying the assumptions used in the formula, the estimate of the number of remote operators required in the USA could range from as few as 24 to as many as 1.4 million individuals.

However, one common thread within the modelling research is the observation that there is an inverse relationship between the time it takes for the supervisor to interact with any given robot and the FO possible (see [60]). Also, they tend to show that it is the relationship between monitoring workload and driving workload that is the strongest factor in determining FO.

Furthermore, whilst the various LOA solutions and tools to reduce driving workload can help reduce the impact of the penalty, they cannot eliminate it. The issue, it appears, is that while a remote operator attempts to drive, they are immediately and significantly limited in their ability to supervise and even drive other vehicles.

In summary:

• There is a general inverse relationship between the FO and the time it takes for the supervisor to interact with any given vehicle;
• While a remote operator attempts to drive, they are immediately and significantly limited in their ability to supervise and even drive other vehicles.

3.2.4. Remote Monitoring: So What Is the Baseline Number of AVs to Be Monitored?

Thus, legacy research in remote operation of aviation vehicles indicates that the one-to-one tasks requiring the operator to provide a direct input to a vehicle via remote control devices is the dominant limiting factor when for determining FO. As identified by Chen, Barnes and Harper-Sciarini [32], the frequency and complexity of any task requiring an individual to focus on a single vehicle directly affects the FO possible; the more often an individual has to focus on one vehicle and the longer that interaction is, the fewer the number vehicles that can be successfully supervised simultaneously.

However, it is highly likely that this limitation could also come into effect when attempting remote assistance. Commercial reports [29], industry standards [41,54] and academic researchers [25,26] indicate is that the interactions of remote assistance could vary from simple object detection (e.g., confirming the presence or absence of a possible obstruction) to complex problem solving and interactions (e.g., assisting a passenger deal with an emergency or driving an AV around complex traffic). None of these tasks are of a simple, set or predicable duration. Thus, it is highly likely that the workload associated with any RA task as well as any remote driving tasks could be highly variable and the duration of interventions highly unpredictable. Given that unpredictability, any attempt to determine the optimum number of AVs an individual could remotely assist and drive would likely be largely inconclusive [76]. This deducted concern is very much supported by the fact that the legacy FO research has never reached consensus and has proposed FOs of between 1:2 [61,62] and 1:12 [63].

Thus, it appears that if any RM study includes the remote operator conducting a RA or RD task it would be unable to provide a clear indication of what the underpinning effective and safe FO could be. To address this, Bogg & Birrell [76] determined to completely remove any RA or RD task and thus establish what the baseline underpinning FO would be if the individual were limited to simply monitoring. They conducted an experimental study where participants were tasked with monitoring multiple ground-based Autonomous Vehicles (AVs) moving through multiple Operation Design Domains (ODDs), assessing the AVs’ operating status and determining if they required a human control intervention. When an intervention was required, instead of providing that control, the participants alerted a separate RD and passed that AV over to them to control. Thus, participants were tasked with monitoring for problem identification and solving without conducting any significant assistance or manual driving control. This accorded with the recommendation provided by Kalamkar et al. [77] that the remote operator organisation would employ two individuals to provide the remote operation; one to carry out remote monitoring and one to carry out remote driving.

From the study, Bogg & Birrell [76] identified that the optimum baseline FO for supervising AVs was five (5), with an ability to temporarily surge to seven (7). This finding is directly and centrally within the FO range of four to eight predicted by Chen, Barnes and Harper-Sciarini [32], suggesting that RM of an AV is a mid- to high-level challenging task.

In the study, participants subjectively reported the lowest workload and demand on building SA was at an FO of three. However, they also reported that they were under-stimulated, and an analysis of behaviour indicated that at the FO of three the participants tended to “play with the cars”, often unnecessarily handing over an AV that was simply stopped in traffic to a RD. At an FO of five, participants reacted the quickest, positively identified and reacted to nearly 90% of scripted incidents, and had a low workload and demand on SA. However, at an FO of nine, participants were on average missing a third of incidents. Ironically, a number of participants reported that they subjectively preferred the higher level of stimulation at the FO of nine and believed themselves to be at their most competent. This finding on an over-optimistic view of their capability at high FO is echoed by Andersson et al. [78], who observed that participants believed they could manage to monitor up to ten vehicles without problems.

Interestingly, this finding that the optimum range of FO is between five and seven appears to coincide with the number of bits of information an individual can hold in working memory [64,79], suggesting that our ability to actively monitor is likely limited simply by our innate working memory capacity. The concern of course is that any distraction that is likely to claim working memory will diminish the scope or quality of monitoring.

3.2.5. Remote Monitoring: But Is It Actually Needed?

It appears to be generally assumed in all Fan-Out (FO) research and almost all (post FO) remote operations research that the remote operator will always, as a default, carry out some form of remote monitoring, staying “in-the-loop” [80] by watching the AVs to assess the driving situation and determine if an intervention is needed. However, it is well established that maintaining a high level of attention and SA is stressful [70,81], and in industry many supervisory tasks are limited in duration, with individuals doing the tasks in set period shifts. For example, EU guidelines for Air Traffic Controller fatigue management [82] suggest shifts should be limited to 8 h long (to a maximum of 12 h) and controls should take 30 min rest every 2 h when working in a radar environment (sitting at a workstation in a closed room working multiple PC screens).

Kettwich, Schrank and Oehl [83] acknowledged that continuously monitoring a system to determine when to take over is mentally very taxing and can cause a subjective work overload that can lead to slower reaction times and decreased hazard recognition. They proposed an HMI designed to support AV monitoring and control that did not depend upon the remote operator being vigilant. They still required their RO to monitor the AV but instructed them that the AV would provide information of an incident (a “disturbance”) graphically and that they should then react to that attention getter. This allowed their remote operators to behave more passively. They tested their HMI and their results indicated that participants supported their design, but that participants’ workload was almost too low and they would occasionally miss important emergency calls, indicating that they could be underloaded.

Despite these concerns about underload, at least one commercial AV operator, “2GetThere” [84], has implemented a Control Centre in Rivium (Holland) where remote operators do not undertake any monitoring activities but instead wait for either the AV or an external agency to call them and ask for an intervention. The project that 2GetThere are involved in appears to be successful, which indicates that that implementing remote operation without monitoring could be viable. However, it is important to note that the AVs of the Rivium project are in a very limited and controlled ODD, with the vehicles in a Geofenced area where they are only subject to limited surrounding traffic–participant interaction.

Interestingly, researchers have observed that some of the legislation for implementing AVs in Europe does not appear to explicitly demand that a vehicle is actively monitored all the time that it is being remotely operated (e.g., [11]). In fact, the UK Automated Vehicle Act [85] (p. 2, 1(5)(b)) specifically cautions that a vehicle is only considered to be travelling autonomously if “neither the vehicle nor its surroundings are being monitored by an individual with a view to immediate intervention in the driving of the vehicle”. Thus, there could be a legal argument to back up the scientific suggestion that removing the monitoring task could reduce RO workload, improve safety and even potentially expand the Fan-Out (FO) ratio achievable.

Thus, there is some research and legal justification to simply not implementing the monitoring task and instead having the RO wait for an AV to call the RO with a request or even demand for support. However, it is possible to imagine that there could be negative effects on both reaction time and performance if the consequence of not monitoring is the RO is startled into reaction. Therefore, the option to not implement remote monitoring deserves further research. That research should investigate both the negative and positive effects on workload, reaction time and performance when the monitoring task is excluded and only the RA task is conducted. This would allow ROCs to determine when proactive remote monitoring is needed, with its potential limitation (as per [76]) on FO and when it could be possible to eliminate the requirement to monitor.

In summary:

• The effective operational range for Fan-Out is four to eight, optimized at five for proactive remote monitoring of an AV;
• Remote monitoring can induce high levels of subjective workload in remote operators, leading to degraded performance (reaction time and decision making);
• Research into how the presence or absence of remote monitoring can affect human workload, performance and decision making is needed.

3.2.6. Remote Driving: Should the Task Even Be Attempted?

Having established during the 2010s that RD (or remote flying) was the dominant factor in determining FO (above and beyond that established for just monitoring), researchers started to investigate methods and practices for altering the scope or complexity of the RD task to make the task less impactful on workload (e.g., [14,86]).

However, whilst research on implementing RD continued in the 2010s to the current date, and has been included and heavily discussed in the various standards [41,54], there has recently been “debate in the industry as to whether remote management or teleoperated driving is even a genuine possibility” [22] (p. 12). Cummings et al. [9] (p. 9) have raised similar concerns about the model of “remote drivers literally take over full control, including steering and speed and brake control”, observing that “there are critical issues that raise concern about the viability of such an approach”.

The concerns raised seem to fall into two general fields: (1) Technology, and (2) Human Factors, as discussed below.

Technology

At a base level there are simple concerns about the technology used to conduct RD. It has been observed that most remote operation systems have non-zero data processing, signal processing and data transmission time, meaning RDs suffer additional time delays or lags when moving controls and receiving situational data compared to an in-vehicle driver [87]. Research has shown that such time lags within control systems can increase the workload and reduce the speed and accuracy of driving actions of human operators [88], critically when those time lags are above 700 ms [89]. However, even at latencies as low as 300 ms, driving performance was significantly worse than no latency [87], with drivers in a simulator experiment more often making an error that led to them leaving their driving lane, or using up all the available “gap” between vehicles just to brake the vehicle and bring it to a stop [90]. As Neumeier et al. [87] observe, “Latency Matters”.

Furthermore, there is always the risk of a more serious break or interruption in the communication link between remote operator and vehicle. This risk means that even if lags could be removed, there is no guarantee that all control data packets will be received by the AV, and thus no guarantee that the operator will be able to continuously and thus safely control the vehicle [91]. Some researchers are adamant that “AVs must be able to guarantee consistent, real time streaming of all relevant data to the control centre to enable ROs to build and maintain SA” [22] (p. 12).

Attempts have been made to introduce mitigations that would overcome these issues of communication lags and potentially missing or delayed data (see [16] for a review of network latency issues and mitigations). A major issue is that an attempt to solve one of the two communication issues can often make the other factor worse. For example, Schrank, Wendorff and Oehl [92] observed that lowering the video resolution to increase speed of data transfer and reduce lag resulted in a poor picture quality that had a significant and negative effect on driver SA.

One solution proposed by Chucholowski et al. [86] to overcome issues with missing data was to attempt to estimate what the next data package should be (in case it did not arrive). They successfully developed a software tool that was able to predict the driver’s intent to a location accuracy of 2–3 cm. However, this software was predicated upon the AV buffering imagery and control inputs to purposefully create a constant 500 millisecond delay (thus allowing time to identify gaps and calculate estimates of data for those gaps). Thus, a mitigation for data loss factor actually led to the introduction of an unforced and constant time lag factor. Chucholowski et al. [86] (p. 8) admitted that whilst they were content with the software modelling of missing data “it will also be interesting as to how exact the operator will be able to actually steer the vehicle with a [constant] time delay”.

Other researchers have also attempted to develop methods to overcome the time lag using predictive techniques. In a pair of papers [93,94], Prakash et al. proposed calculating the time lag and using it to forecast where an AV vehicle would actually be, then transforming (or warping) the visual image presented to the driver so as to appear it had been generated from the predicted position. They reported that this aid helped significantly reduce the number of driving errors, but only worked well at low speeds (<10 km/h), as at higher speeds the forecast images became blurred and effectively unviewable. This method also relies on other traffic behaving as per the prediction, which is potentially a dangerous assumption. Should the other traffic not follow the path they were expected to follow (plausible), the driver will have an incorrect SA and could in fact drive the AV into a collision.

There is also concern that any deficit in tactile feedback and motion compared to that encountered by drivers in the vehicle could have a negative effect on driving competency. Hosseini, Wiedemann, and Lienkamp [95] (p. 400) observed that “remote controlling of a moving vehicle from a stationary location creates a special driving feeling for the human operator which makes highly dynamical manoeuvres difficult”. Individuals provided an experience of remote driving complained they had difficulty estimating distance, linear and angular acceleration, the speed of the vehicle and the inclination of the road [96]. They reported poor spatial awareness, high cognitive workloads and even nausea and dizziness.

Even when using remote controlling devices and systems leveraging off modern telecommunications (5G) to ensure low latency and fast speed, RDs have complained of the lack of depth perception and not being able to feel the movement of the vehicle when it is executing a manoeuvre [97]. This can lead to poor driving behaviour, with Papaioannou et al. [98] observing that RDs are much more likely to make their passengers motion sick.

Attempts to improve driver perception by sending additional environmental data have not necessarily been successful. Larsson, De Souza and Begnert [99] found that sending ambient auditory data led to participants subjectively believing themselves to have good situation awareness of their surroundings. However, the evidence was the that the sound information did not assist them in determining the location and direction of traffic.

Aviation has addressed some of the haptic issues through the use of controlling devices and systems that create or replicate haptic sensations. However, full motion simulators can be relatively expensive (see costs of implementing full motion simulators in aviation), which could reduce any commercial advantage gained from implementing RO. An alternative is to provide partial haptic systems such as just haptic steering wheels. Hosseini, Richthammer and Lienkamp [100] designed a system that predicted AV movement to provide haptic feedback through the steering wheel. They claimed that in a system with a lag of 500 ms the driving of the operator was better (in terms of the scale of steering inputs and the rate of steering reversal) with the predictive haptic feedback than without it. Hacinecipoglu, Konukseven and Koku [101] evaluated a system where if the system (AV) identified an obstacle it would give the steering wheel an artificial deflection force that turned the steering wheel away from the obstacle (in a manner similar to contemporary lane control systems). If the driver accepted the deflection the result would be the AV driving around the obstacle. They claimed an improvement in task completion time and reduction in errors (collisions) when provided haptic support over being provided no haptic feedback support.

However, despite these improvements, Zhao et al. [102] demonstrated that even in no latency situations with simulated tactile feedback, RDs are prone to giving larger steering inputs when manoeuvring with a consequential increase in risk of vehicle instability and passenger discomfort. Zhao et al. [102] (p. 3765) reported that RDs “tend to feel unsafe and think that remote driving is more difficult than real-life driving”.

Human Factors

In addition to the technical factors, there are also concerns about the significant human factors associated with the models of employment of RD. For an AV operating at SAE L4 [41] or above, RD is only anticipated for situations where the AV disengages [103] after encountering a road scenario it cannot handle autonomously [53] and has come to a halt. In these situations, the RO takes over the dynamic driving task (DDT) from the AV when it is stationary. However, in the immediate future of AV tests and trials, commercial entities could expected ROs to act as an SO carrying out a safety-critical intervention in the way the vehicle drives [54] (see definition of remote driving), taking over control when the vehicle is moving.

In both take-over cases above, the RD is normally required to transition from a “hands-off” to “hands-on” state, which will take a finite time. Whilst this time to take over is not necessarily safety-critical for the stationary L4 scenario, it most certainly will be when the vehicle is moving.

To evaluate the viability of the moving vehicle take-over task, literature on take-over as a standby driver and safety officer (SO) was also included within the scope of the review. This was considered appropriate as the response time demands would be similar, and much of the research is conducted in a simulator environment when the participants encounter many of the reduced visual and tactile perception issues of a RO workstation.

Gold et al. [104] reported that in a simulator, in-vehicle standby drivers, even given 5–7 s of warning of an impending incident, did not have sufficient time to take over and react (brake) safely. Gold et al. observed that participants exhibited an average gaze reaction time of approximately 0.5 s, an average hands-on-controls time of approximately 1.5 s and an average take-over time of approximately 2.5 s (i.e., a total response time of 4.5 s). Eriksson and Stanton [105] reported similar results, with average take-over times of approximately 4.5 s when fully engaged, rising to 6s when given a secondary task. Brecht et al. [103] also using a simulator and studying in-vehicle driver reactions reported similar results, with average take-over times of approximately 4.5 s when fully engaged, rising to 6 s when given a secondary task (and monitoring more than one vehicle would certainly be akin to undertaking a secondary task). These times appear to be considerably greater than the 2.5 s to react and start breaking than Shin et al. [106] expect of regular manual drivers of non-automated vehicles. With these likely delays “Any [concept of operations] CONOPs that requires any remote operator to take control of a vehicle on the scale of seconds would be extremely unsafe, especially at highway speeds” [9] (p. 9).

As identified earlier, this time to react might not be a particular problem when the AV is stationary or in slowly unfolding take-over scenarios where the standby driver has full SA of the driving environment and is thus prepared for the manoeuvre they were to conduct. However, there is no guarantee that standby drivers, and indeed ROs, will be able to pay attention to the road situation all the time, especially if supervising multiple vehicles. Liang et al. [107] found that the standby driver’s SA was significantly and negatively affected by the activities being conducted by the driver just before the request to take over, especially if the activity involved the individual looking away from the driving display (to, for example, monitor another AV).

To further complicate the moving take-over, it has been observed that the time to build SA of a situation can be greater than the reflexive hands-on-steering-wheel time to take control. Lu, Coster and De Winter [108] observed that individuals shown a video of a road condition take between 7 and 12 s to build complete SA of that condition. Samuel et al. [109] reported similar times, with participants not paying direct attention to an AV needing 8 to 12 s to be able to detect the same number of latent hazards as those constantly engaged in the driving task.

Thus, it is possible to anticipate the situation where a RD could move their hands and feet to take over within 4 to 4.5 s but not obtain full cognitive SA of the situation for at least another 4–8 s. Of course, an immediate counter point question is “so what? why does that matter?”. To answer the question, we need to consider what exactly is meant by SA and what is important about it. SA is generally identified as “knowing what is going on around you” or “having the big picture” [110] (p. 98) and is generally identified as a required precursor to decision-making. Endsley [81] identified three elements of SA: Perception (collection of data), Comprehension (merging of data to provide understanding) and Projection (anticipation of future events). Thus, in taking over and providing control inputs in under 8–12 s, there is a risk that the driver is doing so with an incomplete understanding of what is happening and what their actions could lead to.

Melnicuk et al. [111] conducted a simulator experiment where SOs in an AV were given a secondary task to distract them (which would result in a reduction of SA of the road ahead). They observed that the SOs’ driving performance after taking over was affected by the workload of the secondary task, with participants showing impaired driving performance (especially in lateral control) for up to 20 s after take-over. It is important to reflect that these durations to return to driving normally were obtained from a study of in-vehicle SOs where the driver is only monitoring one vehicle. It is plausible to expect that the take-over workload and duration of poor driving performance would only increase when the SO is an RO with reduced SA and has multiple AVs to monitor.

Thus, the literature reviewed does raise concerns for the specific scenarios of attempting a dynamic take-over of a moving vehicle. But what about a stationary vehicle? Would that be an issue? As Schitz et al. [52] (p. 175) observe in most contemporary and urban scenarios, “as the vehicle moves along the specified corridor the vehicle environment changes continuously”. Thus, whilst the RO may not have time pressure issues to consider for the original take-over, even when responding from a stationary position, the driver would likely find themselves under pressure to react quickly (under lag conditions) to changing traffic situations (e.g., pedestrians stepping onto the road to enter a stationary vehicle).

All these factors could have a greater impact if the RO is managing multiple vehicles or has other mission-based tasks to undertake that take their attention away from actively monitoring the vehicle. As Mutzenich et al. [22] observe, even when a remote operator is not previously distracted and knows why they have been asked to intervene, the time and effort to build up the necessary SA to make a decision is not trivial. In developing a taxonomy of SA for remote driving, Mutzenich et al. [112] identified 59 aspects of the driving situation that a driver has to regularly consider when making driving decisions. They cite references that indicate it could take on average up to 29 s to build SA and respond [113]. This time to build SA and respond can rise to as high as 162 s when the remote operator has to try and deduce the cause of the take-over request [113].

Thus, with remote driving, we can expect either rapid take-over reaction based upon poor decision-making, or, more likely, a take-over of a stationary vehicle followed by a sort of “stop-and-go” driving behaviour [52] with long pauses in AV motion as the remote operator takes time to (re)build SA and make decisions.

Some researchers have sought to address the factor by attempting to improve the ability of the RO to build SA with somewhat limited success. A common solution is to attempt to improve the telepresence or “experience of being there” [114] of the RD by designing the RD workstation to resemble an in-vehicle driving position [115].

A preventive solution (e.g., [116,117]) is to provide the RO with virtual reality (VR) goggles to provide a mixed or augmented reality view. Hosseini and Lienkamp [116] conducted research where their VR provided location and vehicle dimensions data over the video playback. They reported that participant’s driving accuracy improved at slow speeds but observed that at higher speeds of up to 50 km/h there was no significant improvement in directional stability or perceived workload. Georg et al. [117] continued the study, but observed the VR goggles did not provide significant advantage in perception of immersion, telepresence or workload. Mutzenich et al. [118] reported that participants using a Head Mounted Display (HMD) made faster decisions with an improved probability of accuracy without any significant variance in subjective workload. However, they observed that many participants complained that they found the HMD extremely uncomfortable to wear, suggesting that long-term use of a HMD could likely lead to an associated increase in stress and workload.

In an alternative to VR, Musicant, Botzer and Richmond-Hacham [38] evaluated whether changing the RO camera feed from inside the vehicle to above the vehicle (effectively giving a gaming 3rd person perspective) would improve driving performance. They reported that using the 3rd person perspective lowered workload; however, there was no significant improvement in driving safety, and in fact drivers using the 3rd person perspective tended to steer more vigorously.

Kallioniemi et al. [119] moved away from attempting to provide improved data and instead assessed whether providing warnings as visual, haptic or auditory cues would assist RDs build SA. They found that perception and actual driving performance improved with a combination of just visual and haptic warnings. However, they found that other combinations had no effects, and auditory warnings alone actually increased the subjective level of frustration and thus workload.

So, Should Remote Driving Be Attempted?

When considered collectively, much of the research discovered and discussed above does appear to provide evidence that there are issues and concerns about reaction times and general driving performance when attempting RD. This is particularly the case when the remote driving has to be carried out as a forceful take-over for a safety-critical intervention.

Researchers have observed that RD has proven to be exceedingly challenging [96]. Concerns expressed include the operator having to rely solely on video information [39] and losing tactile cues [8,95], leading to the operator workload becoming overwhelming [120], and the operator driving more slowly and carefully than normal and being prone to a frequent “stop-and-go” style of driving behaviour [52].

As an adjunct, it has been observed [30] that many of the major commercial entities that have and are testing and trialling the implementation of AVs (e.g., Nissan, Cruise, Waymo, Uber and Zoox) have not implemented RD as a remote operation activity, choosing instead to keep the scope to RM and RA. This may be due to the technical and human factor concerns over the safety or dependability of RD provided in the discussion above. Alternatively, there could simply be commercial and legal concerns, as explained by Cornet et al.’s [13] (p. 21) rather blunt commercial analysis that “remote driving is not desirable at the moment, considering the risks associated with shared liability between the [Public Transport Operator] PTO and the [Autonomous Driving System] ADS provider, and the confusion that arises when roles are mixed, potentially allowing a PTO to assume the responsibilities of an ADS”. Or it may simply be that commercially, as Shen et al. [17] (p. 1420) observe, “there is arguably little merit in teleoperating a single passenger car on typical roads”.

Overall, given the evidence on reduced reaction times and degraded SA it is hard for the authors to present a compelling case, based on the scientific literature found, for the implementation of RD as a sensible safety-critical take-over option. The authors agree with Tener and Lanir [8,96] that there are many technical and human factor challenges to overcome before being able to safely and satisfactorily implement RD. They also agree with the Human Factors in International Regulations for Automated Driving Systems (HF-IRADS) [21] (p. 7) statement that “It should not be assumed that remote handling constitutes a viable backup for problems encountered by vehicles under the control of an ADS, or that remotely controlled driving of a vehicle is feasible in busy environments or on high-speed roads”.

In fact, it is the opinion and position of the authors that the research and other heterogenous literature consulted in this review does provide sufficient and pervasive evidence to indicate that RD, even when there is highly reliable technology leading to no communications lag, is simply too challenging and fraught with human factors issues to be a viable RO task. This is particularly the case when the RD is established to provide support to an AV, but may even extend to the provision of RD as a manual service for non-autonomous vehicles. However, it is accepted that this latter conclusion may be challenged by the reader as this review, by being focused on RO in support of high levels of automated driving, did not extend to specifically investigating literature on the implementation of RD of non-automated vehicles as a chauffeur or hire car movement service. It is observed that there are certainly commercial entities reporting success at providing this sort of RO service [121,122].

In summary:

• Technical issues such as data latency, the potential for data loss during communication and reduced situational information increase the difficulty and reduce the reliability of remote driving;
• Human factors issues such as reaction time, take-over time and time to build situation awareness challenge the viability of RO as a take-over solution for a moving AV or an AV in a dynamic environment;
• Many commercial AV organisations are not attempting to implement remote driving solutions (although there are some organisations providing RD of non-automated vehicles as a service).

3.2.7. Remote Driving: Could Involving the AV in the Task Make It More Viable?

Whilst the literature reviewed may present evidence to suggest that RD is challenging and, certainly in the opinion of the authors, should be avoided, the current practical limitations of technology means that it is not yet possible to eliminate the need for human support to the AV. This is because many of the AV systems being built and tested are at SAE LOA4 [123,124,125] where they are able to execute most of, if not all, of a trip but will encounter situations outside of their programmed (or AI) capability when they will likely disengage and need a human to provide the fall-back driving solution. In fact, so commonplace is the expectation that there will need to be a fall-back driver that research has been conducted for some time to try and identify the situations and scenarios likely to lead to disengagement where a human remote operator will then need to provide an input (e.g., [31,126]).

This expectation of the need for a fall-back driver likely explains why, not long after Chen, Barnes and Harper-Sciarini [32] were concluding that the control task is the primary factor limiting supervision of multiple AVs, many researchers appeared to switch focus from investigating how to control multiple vehicles to investigating how to address and overcome latency and SA issues negatively affecting the ability to carry out the control task (e.g., [14,124]). As with the earlier LOA research (e.g., [56,74]), much of this research was on implementing RD where the driver is supported in their task by automation (e.g., [103]). However, rather than follow the systemic division of labour usually found in a traditional implementation of LOA, where tasks are assigned to either a human or automation, the researchers appeared to take a more Human Autonomy Teaming (HAT) approach to sharing work. In HAT, researchers attempt to identify how the autonomy and humans can cooperate and, through dynamic interaction, jointly solve a specific problem (see [127] for a comprehensive review of the original principles and literature of HAT).

One of the earliest researched HAT system designs to address latency issues was the “shared teleoperation” model. In this model, the RDs’ inputs are sent to the AV as desired driving actions or intentions. The vehicle receives the desired actions, accepting that there may have been a delay in the receipt of those inputs, and then calculates whether it can complete them safely. If it can, it uses them. If it cannot (e.g., it detects a possible collision) it overrides them or executes a minimum risk manoeuvre [25,125] and then informs the human operator. Prakash et al. [51] used a method identified as successive reference-pose-tracking (SRPT), whereby the AV did not receive the actual control inputs but rather received a reference pose (location and heading) of the desired future position of the AV calculated from the driver inputs. They ran a simulator experiment to compare driving using their “intent” system against driving using direct commands with and without a lag. They found that in the presence of a lag the participants drove better using their system than when driving normally, but were unable to navigate around a sudden obstacle without reducing speed.

Kim and Ryu [124] proposed a version of this “shared teleoperation” where the RO “drives” a vehicle through a 3D virtual map, that driving generating a path of waypoints that the real AV then attempts to follow using its own direct driving capabilities. They tested a process whereby the waypoints were generated automatically at set time intervals against one where the RO generated them irregularly by pressing a button on the steering wheel. They found that the optimal crash-free path following occurred when the RO generated the waypoints, as they tended to place waypoints to facilitate obstacle avoidance. Fennel, Zea and Hanebeck [128] took a slightly different approach, providing the moving RO who was plotting the path with “live” tactile feedback from the AV that indicated whether the AV calculated it could follow the path proposed (could get to the waypoint presented). They reported that ROs using this method proposed much more efficient pathways. However, their study suffered from a major limitation in that the ROs did not actually drive but instead walked through a physical mock-up of the problem environment. Thus, in practice their solution was slow to assess, slow to solve and could only address physically small (short) solutions. However, it is possible to anticipate time and performance improvements if the physical environment was exchanged for a virtual environment and was scaled.

Gnatzig, Schuller and Lienkamp [14] proposed what they called a “trajectory-based shared control”. In their solution, rather than just dropping waypoints, their driver created a path of trajectory curves, each curve ending in a waypoint. The AV would attempt to follow the trajectory curve and if unable to do so or not provided with a subsequent curve would stop at the next available waypoint. They tested the concept and reported a successful implementation. Hosseini, Wiedemann and Lienkamp [95] expanded upon the trajectory design by getting the AV to propose alternative trajectory curves that the operator could also select from, thus giving the operator multiple potential trajectories to select from. However, as Schitz et al. [129] identified, as the AV environment is dynamic, the curves and paths constantly change as the AV moves through its environment, picks up new information and adapts its paths [129]. This can lead to the operators having an overwhelming number of options to select from, which could result in long decision-making times and the original stop-and-go behaviour they wanted to avoid. To overcome this issue Schitz et al. [129] worked on algorithms that the AV (or unspecified third-party AI) could use to analyse the path options generated and reduce them down to a reasonable number that the human operator could manage (their example had three options). They claimed success, generating an algorithm that was able to analyse and optimise paths in 50–300 ms.

Hoffmann, Majstorović and Diermeyer [91] returned to researching improvements to the trajectory-based design concept. As in previous research, the RD provided a control input via a steering wheel and pedals, with the HMI converting these into a goal trajectory that the AV attempted to follow. However, they improved the RO station HMI by providing a scale graphical representation of the AV on the workstation visuals and also the corridor the AV would likely occupy as it manoeuvred along the trajectory. Hoffmann, Majstorović and Diermeyer [91] successfully conducted elementary proof of concept tests to confirm feasibility.

However, in their later work they returned to a waypoint-based path planning method [130]. They evaluated a method whereby the RO did not drive but instead dropped waypoints for a path using a mouse. The AV then calculated a path that was a best fit to cover those waypoints [130]. They then conducted a practical evaluation of using this waypoint trajectory-based planning to remote guide a vehicle with passengers on board, comparing the results with those obtained when remotely driving the vehicle using direct controls (steering wheel and pedals). They observed that the waypoint trajectory paths were more conservative, keeping a safer distance from obstacles, but observed that the time to complete the task (from human starting to plot to AV completing the route) took on average approximately 16 s longer to complete compared to the direct driving.

Wolf, Taupitz and Diermeyer [131] also evaluated waypoint planning, but this time compared using different input devices to position the waypoints. They tested using a keyboard and mouse, a steering wheel and peddles, and a touch screen. They observed that operators using the steering wheel and peddles were quicker at producing a route, but that the operators preferred the keyboard and mouse and found it to have the highest usability and lowest workload.

However, these exact path plotting methods have not met with universal accolade as feeds from the various types of AV sensors can have a limited range and do not always provide information on the depth or full shape of an object detected (for example it may not be able to detect whether there is another vehicle in front of the one obstructing it). Furthermore, the environment the AVs operate in is normally dynamic, constantly changing while the AV and operator are solving the path problems. This can result in the remote operator selecting a path that appears viable, but soon after the AV moves the environment changes and the path turns out to be unachievable. This can often lead to undesirable stop-and-go driving behaviours [129] as the AV attempts to drive a path, finds itself blocked and has to stop and again ask the remote operator for assistance selecting another alternative path. To address this issue Schitz et al. [52] proposed an alternative and looser form of path plotting they called corridor-based path planning. This method leveraged off the waypoint plotting method, but instead of the AV attempting to rigidly stick to the waypoint path, it used the waypoints to generate a wider corridor that it could then plot its own path through. The human could continue to assist the AV by adjusting the boundaries of the corridor and highlighting zones containing obstacles. Schitz et al. [52] tested the concept using simulation and found that it was easier and quicker to use than a waypoint- or trajectory-based path control, being less subject to “stop-and-go” driving and constant replanning.

In summary:

• Attempts have been made to provide AVs with software that gives them the capability to modify RD provided control inputs to allow the AV to “share” the driving task by modifying the path to be driven;
• Most shared driving solutions still result in “stop-and-go” style of driving as the AV attempts a path that, due to communication and processing delays, is quickly made obsolete;
• Shared driving solutions where the AV calculates and proposes multiple paths can overwhelm human decision-making. The optimum number of paths is three.

3.2.8. Remote Assistance: Should Humans Just Avoid Remote Driving?

In all the literature discussed above on how to address AV disengagement, the researchers tended to focus their attention on solutions that saw the human take over control and conduct some form of the driving task. This was understandable in the early research as the paradigm at the time was that the human operator had lead responsibility for the dynamic driving task (DDT) (the real-time lateral and longitudinal control of a vehicle) and as a driver had provided target paths, waypoints or trajectories using a control device such as a steering wheel.

However, it is noticeable that the solutions discussed above, which tend to focus on how to overcome error and uncertainty, have resulted in more and more automation being introduced between the RO and the AV. In fact, more recent research has resulted in the decision to add so much automation that the RO can no longer be considered to be attempting to carry out the DDT. Instead, the “remote driver” is reduced to using a mouse or touchscreen to either drop waypoints on a map or visual projection [128] or selecting or drawing from AV-generated waypoints [132] or trajectory paths (e.g., [126,129,131,133]). This type of intervention shifts the operator’s tasks and responsibility away from driving and more towards navigating. It is more closely aligned with the definition of remote assistance (see [41,54]) than that of remote driving; the transition to remote assistance appears to have been made.

This observation that many RD solutions being proposed appear to have shifted to actually being RA has been made by other researchers. Some have attempted to categorise the solutions proposed and broadly separate them into either RD or RA. Majstorović et al. [25] and latterly Brecht et al. [103] use the same set of six categories, three of which they considered RD (Direct Control, Shared Control and Trajectory Guidance) and three identified as RA (Waypoint Guidance, Interactive Path Planning and Perception Modification). See

Table 4. Categories of remote driving and remote assistance.

Scope	Category	Description
Remote Driving	Direct Control	Remote operator uses hand and feet controls to provide direct driving task inputs (vehicle wheel and engine) without any input from the AV.
	Shared Control	Remote operator uses hand and feet controls to provide direct driving task inputs (vehicle wheel and engine). The inputs that are interpreted, manipulated or adjusted by the AV to “live” situation and accuracy.
	Trajectory Control	Remote operator uses hand and feet controls to provide driving goals and/or routes. The AV then calculates vehicle wheel and engine outputs to achieve that route or conducts a minimum risk manoeuvre.
Remote Assistance	Waypoint Guidance	Remote operator uses IO devices to set point locations of a route. The AV connects the point location and calculates a “best fit” route then executes the direct driving task along that route.
	Interactive Path Planning	Remote operator and AV team to propose and then select possible routes or paths for the AV to drive. Paths are primarily generated by AV although there is scope for inclusion of operator-generated paths. Scope also extends to human being executive with selection authority, to AV being executive with selection authority.
	Perception Modification	Remote operator acts as observer and situation analyst. The operator actively or passively analyses the driving environment and assists the AV to identify potential driving obstructions, threats or other driving-related situations. The remote operator has no input to the driving task.

below.

Whilst this attempt to classify systems is useful, it is not totally without issues. Majstorović et al. [25] seem to base the separation between RD and RA as depending upon what input device is used by the remote operator: a RD uses a steering wheel and pedal; a RA uses a keyboard and mouse [25] (Table 2). However, this division could lead to confusion, as many of the authors of articles that are not in the “shared control” category appear to identify their solutions as shared control. For example, Gnatzig Schuller and Lienkamp [115] identify their solution as “trajectory-based shared autonomy control” (i.e., both Shared Control and Trajectory Based), and Schitz et al. [52] identify their solution as “Shared Autonomy”, whereas Majstorović et al. [25] categorises it as Interactive Path Planning. Interestingly, Majstorović et al. [25] identify waypoint guidance as a separate remote assistance solution. Yet, in the solution proposed by Kim and Ryu [124], a steering wheel is used to generate waypoints that are then passed to the AV to follow. Thus, the solution proposed by Kim and Ryu [124] appears to fall into the definition of both remote assistance (waypoint control) and also remote driving (steering wheel input). Prakash et al. [51], who are not included in the [25] review, appear to identify their video feed image warping solution as “shared-control” when it would seem more aligned with the definition of “direct control” (although in [94] p. 4637, they do identify their work on their SRPT prediction algorithm “strengthens the direct control concept”). Thus, these categories of terms do not appear to be consistently used and applied across all literature.

It could in fact be viewed that this attempt to apportion the solutions into six subcategories is not helpful nor necessary. The effort could be judged just as unproductive as legacy attempts to define and limit generic Levels of Autonomy (LOAs), an issue so divisive it eventually lead to the United States Department of Defence complaining that “The attempt to define autonomy has resulted in a waste of both time and money spent debating and reconciling different terms” [134] (p. 23).

The point that was being made is it appears that the only applicable and commercial reason to categorise a remote operation method is to determine whether it is remote driving and the remote operator is legally responsible for the DDT, or if it is remote assistance and the remote operator has to stay sufficiently “hands-off” to ensure that they cannot be viewed as legally responsible for the DDT [11]. Of interest, Skogsmo et al. [11] observe that at least two countries (Sweden and Germany) accept that the RA can either switch to becoming a RD or can pass responsibility for the DDT to another actor.

Taking these points into consideration, it is proposed here that the input device should not be used to determine whether a RO is remote driving or providing remote assistance. Instead, it is recommended that the ability of the AV (specifically the ADS) to retain control of the DDT is what is used to determine whether or not any human interaction is remote driving. It is suggested that if the human cannot override the AV, leaving the AV totally in control of the DDT, and the AV is able to reject or alter any human suggestion, or like in Schitz et al. [129], the AV calculates its own waypoints based upon a loose ODD passed by the operator and drives itself, it is remote assistance. If like Chucholowski et al. [86] the human is leading on manoeuvre control and the AV is simply filling in gaps in the remote operators’ control instructions that are being issued in real time, then it is remote driving [39].

Some researchers (e.g., [53]), whilst appearing to take this approach, still expect that the RA will be able to issue some task commands. However, they expect that those commands would be high-level or abstracted directives that allow the AV to calculate and control its own DDT. The input would be more like that of a navigator or air traffic controller, intent-based rather than physical control-based. The RA is effectively providing decision making support to the automation when it is stuck on a problem. This decision support advice could be relatively simple situation information (e.g., ref. [83] object identification) to tactical path planning (e.g., [126]) or even more operational scenario management such as dynamically managing the AV’s ODD limits and constraints so the AV can modify its own solutions to a task [133]. The important factor is that the RO does not get directly involved in the DDT.

In a pair of studies, Tener and Lanir [8,53] set out to identify and test a set of strategic commands that could be used to provide an AV with SA information or to direct an AV to carry out a task or manoeuvre. They determined that this use of commands would make the control task less physically challenging and would also be considerably faster than attempting to manually control an AV. To identify the list and structure of the likely commands, Tener and Lanir [53] conducted an experimental study in which experienced remote operators were asked to verbally state what high-level command they would provide to solve a selection of given problems. They then used the outcome of that study to design and test a graphical user interface (GUI) with text, control option buttons and menus overlaid on video feed in a style similar to that seen in contemporary game and desk-top simulators [8]. Of particular importance, the GUI was not limited to providing just one of the three categories of remote assistance identified by Brecht et al. [103]. Instead it provided options for the remote assistance to select any or all of the three categories; the GUI was designed to provide a “playbook” of options rather than a singular category of assistance.

It was observed that the use of this versatile GUI offered an additional advantage of command control as the GUI could be used for control of a wide range of vehicles of varying size, shapes and utility. They reported that most participants responded positively to the GUI and found it useful and efficient, and the concept of using high-level commands realistic and desirable. However, not every participant found every control useful or easy to use, and several users expressed a desire to move the AV camera used to provide a view of the scene. In addition, several participants raised concerns that the modern button and menu GUI may encourage remote operators to treat teleoperation as a game. Finally, the participants observed that the hoped for time saving of using command lines may not manifest as “after issuing a high-level command, the RO must also monitor its execution till the scenario is resolved” [8] (p. 19).

However, whilst there was definitely an observable shift in recent research away from remote driving towards decision-support, it was notable that much of the research found for the review was highly conceptual, and the decision-support requirements were predominately hypothetical. This is likely a consequence of researchers simply not having sufficient information on the types, forms and timings of decision-support advice needed due to a shortfall in knowledge of the situational edge cases when AVs are likely to need support [135]. Whilst the SAE [48] have been able to give some advice on scenarios and decision-support requirements, the results of this review indicate that there is a shortfall in research and knowledge to specify what decision-support is needed and, therefore, how the RA would need to work.

Returning to consider the method of providing decision-support via commands, it is of interest that whilst Tener and Lanir [53] collected verbal command statements from their participants, they then converted these into a graphics-only interface. There did not appear to be any attempt to implement a speech-based interface even though research has indicated that it is both viable and can provide an advantage [136], especially when attempting to communicate with more than one AV at a time [46]. As Bogg et al. [137] observe, this means that their interface potentially missed opportunities for greater cognitive time-sharing [138] that they could have gained by using the audio communication channel as an additional route for information exchange. As Bogg et al. [137] identify, it would not be necessarily to create an interface that could interpret general conversation; rather, lessons could be applied from the aviation industry requirements for voice communications to be constructed as “plain talk” (see CAP 719 [139]) with a limited dictionary (CAP 1430 [140]) following simple radiotelephony structures (CAP 413 [141]). In summary:

• Researchers have attempted to create categories or levels of remote driving and remote assistance; however, not all remote operator and AV teaming solutions fit neatly into the categories proposed;
• It is proposed that the point of difference between remote driving and remote assistance is the decision making capability of the AV;
• If the AV can modify or override the human input and the AV has the complete DDT, it is remote assistance;
• If the human has control over the DDT, it is remote driving;
• A GUI designed to provide multiple “categories” or methods of RA as “plays” is more useful than limiting RA to a singular method;
• A GUI designed to provide strategic goals and directives (e.g., route plans) is more adaptable to be used to support fleets of mixed vehicle types.

3.2.9. Remote Operation: Summary Observations for Viability of RO Implementation

In this section the three facets of remote operation (RO) have been considered in the light of a fundamental question: which of the three functions can (and should) be implemented and if so, when? The results are summarized and provided below in

Table 5. Concluding observations for viability of RO implementation.

	Role	Task	Task Scope	Recipient	Technology Issues	Human Factors Issues	Mitigation Issues	Authors’ Recommended Implementation	References
1	Remote Monitoring	Look-Out For Emergency Intervention	Observing sensor/video feeds from vehicles with an intent to proactively identify safety, emergency or other time-critical reactive interventions in AV DDT.	AV and/or Passenger	Latency negatively impacts time to react [87], reducing safety margins. The reduction in safety margins and braking distance is amplified almost linearly by vehicle speed [90].	Effort to maintain SA increases in proportion to number of AVs monitored. Optimum performance is affected by task complexity and varies between 4 and 8 [32,76]. Observational errors are more likely when monitoring 9+ [76].	Mitigations address and reduce latency but do not eliminate it and often introduce new human factors. Lowering video resolution to reduce streaming time leads to reduced SA [92]. Estimate location at t + time lag and warping incoming feed/display only works at low speeds and does not accurately predict other traffic, which can result in driver building wrong SA (e.g., thinking another vehicle is in a location it is not) [93,94].	Should only be implemented when: Latency is eliminated. Number of AVs monitored is <7 [76]. AV is moving at low speed (<4 ms−1) [90,94] or has large safety margins and braking distances (e.g., because it is geofenced from pedestrians or other vehicles).	[86,87,88,89,90,93,94]
2	Remote Monitoring	Actively Preparing To Provide On-Demand Support	Observing vehicle feeds (sensor/location) with an intent to build SA in preparation for providing support on demand.	AV and/or Passenger	Nil. On demand expected to be requested at LOA 4 [41] when AV has completed Minimum Risk Manoeuvre (MRM) and AV is stationary [7,142].	Effort to maintain SA increase in proportion to number of AVs monitored with optimum performance between 5 and 7. Observational errors/misses likely when monitoring 9+ [76].	Mitigations are often centred around designing Human Machine Interface (HMI) where operator gathers information on AV. Research tends to focus on conducting task for one vehicle, e.g., [8,53,83]. Further research is needed into HMI design requirements for monitoring multiple vehicles and operator performance when responding to multiple demands, e.g., [75]	Should only be implemented when: AVs provide early warning as they initiate MRM. Warning could include use of audio and haptic communication channels [35,53]. HMI must provide SA information to prepare for support task demanded [126].	[7,8,35,53,75,83,103,126]
3	Remote Monitoring	Passively Preparing To Provide On-Demand Support	Conducting secondary task while waiting for AV/passenger demand for support.	AV and/or Passenger	Nil. Demand expected to be requested when AV has completed Minimum Risk Manoeuvre (MRM) and AV is stationary.	No effort to maintain SA, but could be a sharp rise in workload (startle response) to demand to build SA. Absent SA needs to be completely built, which can take 8–12 s [108,109].	The majority of research was based upon operators actively monitoring AV. Only some involved evaluating operators looking away from the video feed (e.g., [143]. Secondary evidence is that no mitigation is required when the AV is geofenced [84]. More research is required on the human factors and viability of implementing passive remote monitoring.	AVs provide early warning as they initiate MRM. Warning should be provided by multiple communication channels (visual, auditory, haptic) as it is likely operator will not be looking at visual feed.	[83,103,108,109,111,126,143]
4	Remote Driving	Emergency Intervention	Task includes Role 1. Task is to carry out emergency manoeuvres, both longitudinal (braking) and even lateral (swerving), to avoid an incident such as collision or road departure.	AV	As for Role 1, plus: Lack of tactile and motion data lead to reduced driving competency [95]. Reduced driving competency can make passengers motion sick [98].	As for Role 1, plus: Time to build SA [108,109] can be longer than time to react [104,106], meaning fast reacting drivers could be making driving decisions based upon incomplete SA, or drivers choose to wait (likely under stress) to establish SA before reacting, increasing reaction time and reducing safety margins.	As for Role 1: Mitigations are primarily focused on: Give drivers pre-existing SA and early warning [104]. The warning should be sufficient to build SA (approx. 8–12 s) [108]. Reducing the number of vehicles being monitored [75] to reduce impact of “secondary” tasks [100].	Should only be implemented when: Latency is minimised or eliminated. Driver is monitoring a single or low number of AVs (<5) [32,76]. Driver is provided an early warning, ideally more than 7 s [104] to allow sufficient time to build SA [108]. Remove other tasks, e.g., monitoring other vehicles, on warning issue (as secondary tasks prior to intervention reduce effectiveness of intervention [100]).	[9,32,75,76,95,98,100,103,104,106,108]
5	Remote Driving	Manoeuvre Vehicle	Manoeuvre an AV past a situation that it cannot complete the DDT for. Reasons for inability to complete DDT could include inability to identify objects on or near road, situation complexity and dynamic nature, limitations of ODD.	AV	As for Role 1, plus: Interruptions in communication link reduce ability to continuously control AV [91] Lack of tactile and motion data lead to reduced driving competency [95] with drivers prone to hesitant “stop-and-go” driving behaviour [52]. Reduced driving competency can make passengers motion-sick [98].	As for Role 1, plus: Building SA is more difficult and drivers’ perception of safety and competency is negatively affected [22,102].	As for Role 1, plus: Mitigations address and reduce loss of tactile awareness but do not eliminate it and often introduce new human factors. Introducing sound from AV surroundings can create illusion of improved SA without actually improving driver SA and performance [99]. Haptic feedback through steering wheel can improve task time and reduce errors [100,101], and haptic warnings given with visual alone can improve driving performance [119]. Improving telepresence through devices such as VR goggles can positively improve driving competency at low speeds [116], but other researchers have observed that VR goggles can be uncomfortable [118]. Introducing additional automation between the driver and the vehicle, such as sending trajectory curves [51]. Waypoints [124] or movement vectors [124] not steering data can improve driving performance but can lead to errors and “stop-and-go” driving behaviour.	Should only be implemented when: Latency is minimised or eliminated. Driver is monitoring a single or low number of AVs (<5) [32,76]. Driver is provided an early warning, ideally more than 7 s [104] to allow sufficient time to build SA [108]. Remove other tasks, e.g., monitoring other vehicles on warning issue (as secondary tasks prior to intervention reduce effectiveness of intervention [100]).	[14,22,51,52,86,87,88,89,90,91,93,94,95,96,97,98,99,100,101,113,116,117,124,125,128]
6	Remote Assistance	Assist Vehicle Identify Object	Assist AV to identify and categorise an object and determine if current path can be achieved or if alternative solution is needed [120].	AV	Requires AV to have visual sensors for conditions it will operate in [92], e.g., optical in day light, infra-red/NV for low light.	Humans may have difficulty determining which object is to be identified and therefore can make mis-identification.	Research focused solely on this function appeared to be limited, indicating a requirement for further research. Research either identified this as a function [126] or simply assumed it was possible within the scope of preparing a solution [143].	Should only be implemented when: AV LOA is 4+. AV can conduct an MRM. AV can provide visual or other electronic imagery for all environmental conditions the AV will operate in [22,92]. AV can provide a clear indication of the object to be identified [12,126].	[8,22,53,92,120,126,143]
7	Remote Assistance	Assist Vehicle Manoeuvre	Assist AV formulate or identify a manoeuvre to get past a situation that it cannot currently complete the DDT for. Reasons for inability to complete DDT could include inability to identify objects on or near road, situation complexity and dynamic nature, limitations of ODD [41,42,54].	AV	Requires the AV to be more capable than for Task 4 as vehicle needs to either calculate alternative solutions or adapt new solutions into DDT.	Human provision of solutions takes a finite time; however, the AV environment is normally dynamic, meaning solutions presented can be obsolete before or during implementation, ref. [129] leading to “stop-and-go” driving and constant re-calculation.	Assistance is the mitigation to latency making conventional DDT challenging. It sees the human provide a solution to a driving problem that the AV accepts, modifies or rejects. Currently researched options include: Trajectory Planning. The human selects trajectories/paths the AV then drives [14,95,129]. Waypoint Planning. The human provides/selects/confirms waypoints or routes made of waypoints, and the AV then uses them to drive [124,130,131]. Corridor Planning. The human suggests a safe corridor wider than the vehicle, and the AV dynamically calculates a path as it moves within the bounds of the corridor [52,91].	Should only be implemented when: AV LOA is 4+. AV can conduct an MRM. RA cannot conduct DDT (to conduct DDT makes the RO a RD not RA). RA is providing command and mission oversight (supervision) to one or more AVs [8].	[8,14,25,52,91,95,103,124,129,130,131,132,133,142,143]
8	Remote Assistance	Assist Vehicle User (Passenger)	Respond to, diagnose and solve a passenger request or observed issue. Could extend to interactions with external agencies or interfacing between passenger and AV [138].	AV and Passenger	Communication with passenger [13].	Communication with passenger [13].	To the knowledge of the authors not a widely researched topic. Research is underway to identify user/passenger requirements [13,144] but none was found that discussed how a remote operator may team with a passenger and AV to formulate a mutually agreeable solution.	In the view of the authors, further research is needed on three-way interactions between operator, AV and customer.	[13,138,144]

. This table compromises a key contribution of this review.

The review of the literature indicated that much research had been conducted that was sufficient to answer some of the most pressing questions around viability of RD. However, as is so often the case, the review also indicated that further research would be beneficial, which will be discussed in the following section.

3.3. Part 3—Remote Operation: Research Gaps?

3.3.1. Accounting for Scenario Variability in Remote Operation

The review of the current literature on remote operation of AVs indicates that a critical yet largely underexplored dimension in remote operations research is the systematic consideration of scenario variability and its impact on operator workload, task requirements, and feasible supervision ratios. Some research (e.g., [126,145]) and commercial best practice data is available [48]; however, much of this is based upon early simulations and deployments or general hypothetical theory of driver-centric scenarios (e.g., [146]). It does not necessarily provide key insights into the likely problem-solving tasks an operator could encounter in future AV deployments. In defence of the researchers reviewed, this is likely not a consequence of lack of interest but rather due to a lack of available observations from complex scenarios and situations.

3.3.2. Scenario Variability: The Limitations of Current Workload Models

Previous approaches to determining workload and the number of AVs that can be supported often relied on relatively simplistic workload calculations. As discussed earlier, Cummings and Guerlain [63] proposed that one could calculate the maximum number of AVs a remote operator could supervise by identifying work demand as a percentage of available time per vehicle and multiplying that figure by integers until workload reaches 70%. However, this calculation methodology fails to account for the heterogeneous nature of scenarios that remote operators will encounter. The workload and time to confirm the presence of a static obstruction differs dramatically from that required to navigate an AV through complex urban traffic or to assist a passenger experiencing a medical emergency [25,26,29,54]. Furthermore, whilst a calculation can be made based on workload and response time for routine tasks, this is likely immediately redundant as the legacy observation of Bainbridge [70] still stands: routine tasks can and will be automated, leaving behind only those problems that have never previously been encountered or are so complex as to defy programming.

The central issue is that it is likely to be extremely difficult to estimate the surges in workload that occur when supervisors are required to concentrate on and solve problems associated with unique and challenging edge cases. Infrequent or novel edge cases can demand significantly elevated cognitive resources, extended decision-making time, and focused attention on a single vehicle. It is possible to imagine scenarios such as medical situations where the ratio of remote operators to vehicle could reverse with more than one remote operator providing support to the AV and the passengers (if as in [48] the Remote ADS Assistance is separated from Customer Support). This surge effect means that even if an operator’s average workload across routine scenarios suggests a viable FO of eight, the sudden emergence of a single complex edge case could overwhelm their capacity and compromise their ability to maintain adequate situation awareness across their fleet, such as the findings of [76], where participants reported higher workload, lower SA and missed more intervention opportunities at a FO of nine than at FO five and FO seven.

3.3.3. Scenario Variability: The Challenge for Automation Support Models

The importance of scenario consideration becomes particularly important when implementing remote assistance where the AV is expected to generate some or all solution options (e.g., [129,133]). However, this relies upon an implicit assumption: that the automation can anticipate and prepare appropriate responses for the scenarios that will be encountered. This assumption becomes increasingly tenuous as the range of possible disengagement scenarios expands from simple ODD issues to complex issues involving AVs, passengers and external agents. For such novel or unanticipated scenarios, the automation may simply fail to generate suitable options, leaving the remote operator to generate all facets of the problem solution.

Thus, there is a need to focus research not just onto identifying the more extreme but at least predictable (or at least plausible) edge case scenarios, but also on identifying the types of support and likely tasks that could be needed and the best method for apportioning those tasks across the larger human automation team of the remote operations centre. As per Clark et al. [147], it is vital to consider the type and methods of communication needed between operator and AV that will allowing greater discourse and teaming between the human and the AV and facilitate more open-ended problem solving. The research on scenarios and methods of problem-solving will allow for more informed decisions about task interaction dynamics; it will help answer the questions: Should a single operator handle remote monitoring, remote assistance and potentially remote driving for multiple vehicles? Or should these tasks be distributed among specialized operators?

3.3.4. Scenario Variability: Scenario Classification

Research has been conducted to identify situations and scenarios likely to lead to the AV disengaging from the dynamic driving task and thus requiring remote operator input [31,96,126]. The presence of this research indicates that the scientific and commercial community recognizes the importance of this disengagement scenario dimension. However, systematic classification schemes delving into more detailed scenario categorisation by type, complexity, required operator response, expected duration, and frequency remain underdeveloped. Without such classification, it remains difficult to compare findings across studies, to generalize results from one operational context to another, or to make informed predictions about the viability of proposed remote operation designs in specific deployment scenarios.

To address scenario variability systematically, it is essential to distinguish between Operational Design Domain (ODD) and scenarios. The ODD defines the environmental and operational boundaries within which an AV is designed to function, including factors such as geographic area, road types, speed ranges, weather conditions, and time of day [41]. Scenarios, in contrast, refer to the specific disengagement situations and traffic interactions that occur within that ODD and that may require remote operator intervention. These might include encounters with unexpected obstacles, unclear road markings, unusual traffic participant behaviour, ambiguous right-of-way situations, or system uncertainty about object classification.

The relationship between ODD and scenarios is not trivial. A simplified ODD, such as a dedicated shuttle route with minimal interaction with other traffic participants may naturally generate a more constrained set of scenarios, potentially with lower complexity and higher predictability. Conversely, a complex urban ODD with diverse road users, variable infrastructure quality, and high interaction density will produce a wider array of scenarios with greater variability in complexity and lower predictability in frequency. As Schitz et al. [52] observed, the dynamic nature of the AV environment means that paths and options constantly change as the AV moves through its environment and encounters new information. In complex ODDs, this dynamism can lead to operators facing an overwhelming number of options, resulting in extended decision-making times and potentially degraded performance. Perhaps, just as in dynamic path planning [129] reducing the number of path options to select from can reduce human workload, dynamically reducing the scope of the ODD to meet the specific time and location of an incident could also assist operator workload. However, just as Calhoun et al. [66] found that dynamically varying LOA of the system can lead to mode errors (the operator forgetting what specific LOA they were in), there is a risk that presenting the remote operator with multiple localised ODDs could result if a form of mode error as the operator struggles to recall exactly what variance of the ODD they are in at any specific time. Certainly more research is needed on methods for dynamically varying ODD to fit smaller localities and on how to then present these localised ODDs to the remote operator in such a way as to assist them build and maintain SA.

3.3.5. Scenario Variability: A Path Forward

It must be acknowledged that the scenarios requiring remote operator intervention are not yet fully identified, nor may it be possible to comprehensively classify and consider them in their entirety at this stage of AV development. As automation capabilities evolve and ODDs expand, new scenario types will inevitably emerge that were not anticipated during initial system design. The “long tail” of edge cases means that exhaustive scenario enumeration may remain perpetually incomplete. However, this uncertainty should not paralyze progress or justify ignoring scenario considerations altogether.

A pragmatic approach would be to adopt a methodology similar to scenario-based testing frameworks currently used for ADS validation (e.g., PEGASUS, SAKURA, ASAM OpenSCENARIO). Rather than attempting to create a complete and static scenario taxonomy before deploying remote operation systems, the field should develop living scenario catalogues that document disengagement scenarios as they are encountered in trials, deployments, and operations. Although much literature exists on ADS scenario catalogues, far less addresses RO-specific scenarios. Kettwich et al. [126] (p. 3) acknowledge that remote operation of AVs has not been widely rolled out so far, and that consequently “there is limited knowledge about concrete use cases and scenarios that are most relevant to it”. In their paper, the authors interviewed personnel at control centres, operator workplaces, and on-board operators to derive use cases and create scenario catalogues, capturing not only driving-related scenarios but also the broader operational challenges facing remote operators. Similarly, Tener and Lanir [53] conducted semi-structured interviews with 14 teleoperation experts, employing thematic analysis to identify intervention road scenarios. Their work classified disengagement scenarios into ten major categories (including road obstacles, infrastructure issues, technical problems, weather conditions, human communication needs, and rules/regulations) with 35 sub-categories, providing a comprehensive taxonomy of situations requiring remote intervention. Importantly, their findings highlighted that RO scenarios extend far beyond the driving task itself, encompassing passenger communication, legal compliance, and coordination challenges. These scenario types have no direct parallel in traditional ADS testing frameworks.

These catalogues should capture scenario characteristics, the remote operation tasks they required, operator workload profiles, resolution strategies, and outcomes. As new scenarios are identified through operational experience, they can be systematically added to the catalogue, progressively building a more comprehensive understanding of the scenario space.

This iterative, evidence-based approach offers several advantages. First, it allows deployment to proceed while systematically building knowledge rather than waiting for complete understanding. Second, it creates a feedback loop where operational experience directly informs system refinement and operator training. Third, it enables scenario catalogues to be tailored to specific ODDs and applications while still allowing cross-learning when similar scenarios appear in different contexts. Fourth, it provides an empirical foundation for progressively refining workload models, FO predictions, and task allocation strategies as the scenario database grows.

By keeping scenario variability in mind from the outset and establishing mechanisms to systematically capture and learn from encountered scenarios, the remote operations field can build toward more robust, context-appropriate system designs while maintaining the flexibility to adapt as both technology and operational understanding progress.

In summary:

• A significant limiting factor for the supervision of multiple AVs is the surge in workload the operator is subject to when conducting remote assistance or remote driving;
• The intensity of the subjected workload has been linked to the complexity of the scenario that led to human interaction and the AV ODD at the point of interaction;
• More research is needed on scoping and categorising the complex scenarios that would lead or need humans providing solution generation support.

3.3.6. What About the Users of Automated Vehicles?

There has certainly been plenty of research into the relationship between AVs and their users. Much of the early research sought to establish what is likely to be required of an AV to improve acceptance and uptake of their use. The research concentrated on answering two very primary question: “what do individuals think about travelling in AVs?” and “what would users expect of or in an AV?”.

However, a potential concern is that much of this type of research is hypothetical and generic in nature. Individuals are being asked to imagine what their perception of automation is [148,149,150] and what level of automation they would accept [151] without actually having any substantial experience of using the technology. Even when participants did experience a controlled AV trial, when asked how an AV should behave—either human-like stopping at junctions or “peeking” around corners to see if it were safe, or CAV-like as it was communicating with other vehicles and infrastructure and “knew” if the junction was clear to proceed without stopping or slowing down—there was no significant difference in user trust, with post AV interaction interviews indicating that there were pros and cons for both driving styles [152]. The concern here is that the user data and findings are being drawn from subjective supposition rather than reflected experience. Furthermore, there is always a risk that answers to perception questions could easily be provided out of context. For example, Liu Yang and Xu [153] reported that their participants wanted self-driving vehicles to be four to five times safer than human-driven cars; however, they also noted that similar studies on human-driven cars (with no automation) also indicated that participants in those studies would also prefer those cars to be five to seven times more safe. Thus, the research from Liu, Yang and Xu [153] leaves us with a finding but also a further question: do potential users specifically want AVs to be more safe, or do they generally want all cars to be more safe?

The more recent upsurge in practical implementations of AV systems has created the opportunity to collect data from primary users. Li et al. [123] conducted a study interviewing individuals with experience using a remote driving system. The study was aimed at determining system usability in order to elicit a design specification for a remote driving and control workstation. They then later extended the study [152] to attempt to identify if there could be issues with getting in-vehicle passengers to accept that the vehicle could be driven out of their control. However, for that latter study they again had to return to interviewing participants with limited or no experience with AVs, meaning that their results returned to being largely hypothetical. The participants were shown an example of a remote driving service and asked if they would use it. Most of the participants said they did not see themselves using the remote driving service, but if they were to, they would want to monitor the driving and remain in the driving loop, where “the majority of participants indicated that they would like to take over control by themselves if possible” [152] (p. 219). This was perhaps predictable from a hypothetical audience, as in earlier research Schaefer and Straub [154] had observed that participants stated they had more trust in an AV when traditional human controls (wheel and pedals) were present.

Research has shown that passengers generally value and prefer getting more information from the AV on what it is doing [155]. However, it seems that requirement extends to the remote operator too. In the Li et al. [155] study some participants expressed concern about trusting remote operators and stated that they would prefer having someone they knew or had been recommended as their RD. The participants also expected that any RD taking over or providing assistance would provide an explanation for what had happened and what they were planning to do. Thus, the participants seem to be indicating that they do not simply want a customer support service; they want to know when and how a human is interacting with the AV. Thus, it would seem from established research that the scope of the RD or RA should include the ability to interact with both the AV and the users, possibly providing a conduit for communication between the AV and user.

This user demand for interaction with the driver or assistant is potentially at odds with commercial standards (e.g., [41,54]), which tend to separate the interaction between a remote operator and the AV from the interaction between a remote operator and the AV customers and users. Often they identify them [48] as customer support with a completely separate scope, such as simply providing fare payment assistance. However, researchers consider this too simplistic a scope. Parr et al. [26] identify that the remote operator will be required to interact with the users and even external agents to solve problems. They envisage the remote operator communicating with the user to provide information about a given situation and the current state of the AV, for example, communication about issues with the AV that require the user to change from one (broken down) AV to another (on its way to pick up the user). This user requirement for a human remote operator to talk to the on-board user (passenger) is supported by research from the UK Department for Transport [144], who identified that with no driver present, passengers in an emergency situation could be under great pressure if they are expected to contact the emergency services and the transport operator. The requirement, it seems, is for the human remote operator to provide support to the users, reassuring them and through empathetic, reactive and complex communication help the user manage the situation.

This latter demand for the remote supervisor to interact with both the AV and the end-user has been introduced as a legal requirement in a number of European countries, including Sweden and Germany [11]. Interestingly, Parr et al. [26] do not limit the scope of a remote operator to just one of the sub-roles and functions (remote monitoring, remote assistance, remote driving etc). There is one limitation suggested in the BSI 1887 “Human factors for remote operations of vehicles—Guide” [54] (p. 10) where it is suggested that “an [remote operator] RO should carry out either single vehicle [remote driving] RD tasks or [remote monitoring] RM tasks, but not both simultaneously”. Research on remote operator interaction with AVs [8] and passengers [156,157] is ongoing, so there is an expectation that the need for remote assistance to cover assistance to users will emerge.

The research being conducted by the UK Department for Transport [144] is of particular value as it highlights the requirement for a new facet of user-orientated study; the exploration of user needs in specific situations such as emergencies or threats where in current conditions the user would turn to the driver for support or assistance. There is concern that should the RA be near-continuously engaged with monitoring the progress of the AV through external feeds showing the surrounding area that they could miss a passenger emergency and misinterpret any AV reaction to that passenger. For example, if the AV executed a minimum risk manoeuvre in response to a passenger emergency, how long would the remote operator spend scanning the external views and AV system data before considering the likelihood that the source of the problem was the passenger? Conversely, there is also concern that should the operator be engaged assisting a passenger they could encounter the limiting surge in workload discussed earlier and miss other external situations.

Another area of research that needs to be addressed is the post hoc impact of the use of AVs. Much of the sociological or ethical research (e.g., [158]) to date has been on whether or not AVs will be used, yet as Brooks [159] observes, there does not seem to be much consideration for what are the unexpected consequences of introducing the technology. Brooks [159] (p. 1) appears to have a jaundiced view, suggesting that “such cars will be pariahs and their owners will act obnoxiously”; he envisages owners setting AVs to constantly circle an inner city while the owner attends a social event, or the owner sending their AVs early to “claim” a spot at the front of the school run queue.

Whilst this latter ethical research is slightly outside of the scope of this review (as it covers use of AVs rather than the operation of AVs), it is still essential for the future of RO as it will help operators understand the likely situations that they could find themselves having to monitor and provide advice for. There is much to be considered on what support and interaction the future user might want with the AV and any remote operator, and how they might want the service they receive to be adjusted to meet their specific needs. This research is considered to be an essential element tied to the research needed on scenario development discussed above, as one can often drive the other.

In summary:

• Legacy research into user perception and potential uptake of use of AVs was largely hypothetical;
• Recent user interest and concerns have been less about how the AV might perform and more about the safety and security of users when no driver is present in the vehicle;
• There is a gap for research into unexpected consequences of AV use that will impact on the scope of the support task an RO is expected to provide.

4. Conclusions

The primary aim of this review was to attempt to address research questions that had emerged as the authors joined an Automated Vehicle (AV) commercialization project [2] where they were engaged to research future remote operation (RO) requirements and paths to implementation. However, the findings of the review extended much further than simply answering the questions “how many AVs can an RO manage” and “what tasks should an RO carry out”, with the result that the primary novel contributions of this paper are:

• A discussion on terms and labels for RO in common usage, with the aim of assisting readers understand and identify potential discord and misuse of terms;
• An exploration of the recent literature on RO used to update the findings of prior peer reviews and provide an updated “current state of science” of research into RO;
• A position, derived from the review of existing literature, on the point of separation between remote driving and remote assistance based upon command and control of the direct driving task;
• A position, derived from the review of existing literature, on the plausibility and viability of implementing the primary or standard RO positions (remote monitoring, remote driving and remote assistance);
• A proposal of the primary knowledge and research gaps that, when addressed, are likely to have the most impact on future design and implementation of remote operator roles.

The scope and definition of remote operations of automated vehicles has certainly advanced with research over the last 20 or so years. In early research simple generic terms like teleoperation and supervision were used. Over the years this has changed and been refined with the new term remote operation being introduced to provide a point of separation and new terms used that clearly identify specific sub-tasks such as remote monitoring, remote driving and remote assistance. The scope of the provision of remote operations has also been expanded from the original concept of only interacting with the AV to now also include interacting with the user and with external agencies. And the development of the terms is by no means complete. With the observation that there are still many unknown and unanticipated scenarios where an AV will require some form of remote support, we can expect the definitions to continue to expand and acquire more and more detail as more and more remote operator tasks are identified and introduced.

When conducting the review of literature, the authors observed that much of the original research into remote operations was aimed at finding answers to such questions as “how many vehicles can an individual supervise” and “how can a safe driving solution be implemented in the presence of communication lags and lapses”. As these questions have been answered (and not all with the expected or desired results), our understanding of what is possible, what is plausible and what is recommended (for safety and practicality) have changed. Original expectations that the primary role of the remote operator would be the standby driver or someone who could provide a remote chauffeur service seem, in the face of concerns over safety, to have largely abandoned. They have been overtaken by the expectation that the human will be more gainfully and usefully employed at a higher more strategic level.

Whilst some researchers continue to attempt to solve the problems and risks of carrying out the direct driving task (despite plenty of evidence to suggest it is not a safe, sensible or even a needed option), most modern research appears to have switched to addressing how a human might add value to the developing competencies of the AV. Instead of attempting to be the fall-back driver, more and more research is evaluating how the human and automation can team, with the human providing situation awareness context and recommended solutions for complex problems and the AV conducting all the driving. To borrow from aviation: the shift appears to be moving away from researching two geographically separated pilots into providing a pilot (the AV) and a navigator/observer (the remote human).

It is the position of this paper that the literature found and reviewed provides sufficient evidence to suggest that the technical and human factors issues associated with attempting remote driving as an AV intervention and take-over or even as a standby support function to AVs render it inherently more difficult, less reliable and thus generally less safe than in-vehicle driving. It is therefore the position of this review that remote driving in support of AVs should be avoided wherever possible. Alternatively, if it is not possible to completely eliminate the demand for the remote driving task, then remote driving should only be implemented when the AV ODD is appropriate to reduce the impact of human factors affecting reaction time and SA building time. Thus, it is recommended that the incidence of intervention remote driving should be limited to situations where the AV is moving slowly (and reaction time is less of a concern), where the number of vehicles being monitored is low (and SA of a single vehicle is high) or the AV is in a geo-fenced environment with no other objects or road users (and the RO does not have to react at speed to a rapidly changing scenario).

In the absence of remote driving, the focus of future research should be on continuing to develop and test concepts of how a remote operator can team with the automated vehicle, providing intelligent problem solving input for novel and usual situations that the AV cannot easily derive solutions for. It is the position of this paper that the remote operator team skills and competencies should complement not duplicate the AV capabilities. The human may occupy a managerial or supervisory position, but they should not fulfil a duplicate “fall-back” role. Research needs to continue investigating how that human autonomy teaming structure should be implemented, identifying the scope of tasks to be undertaken and the division of labour between human and AV. Only once this is completed is it possible to design and implement a truly user-centric HMI.

The research on remote monitoring has already provided an indication that the baseline of vehicles an individual can actively monitor is as low as just five, a number that could challenge the expectation that remote operation of automated vehicles offers a commercial advantage for future public transport. Counterintuitively, it may be preferable from a human factor workload and commercial perspective to simply not monitor but wait for the AV to demand support. Research now needs to turn its head towards addressing these monitoring limitations and propose and then test alternative remote operation solutions that can open the door to higher numbers of AVs that a single human can manage or supervise.

Finally, research also needs to look to address human factors beyond simple one to one interaction with the AV. As knowledge emerges from projects such as the SCALE [2], which feeds back into AV design and capability, the problems remaining to be solved by a remote operator are likely to be residual cases with a scope far beyond simple driving or manoeuvring. Furthermore, the number of novel as yet unknown problem situations involving remote operator interactions with the passengers of the AVs will increase as these interaction scenarios emerge during AV use. These case will not be easily solved through prepared (programmed) interactions, as they are likely to be highly infrequent and complex edge cases.

It is the scope of these residual interactions that the authors identify where the most substantial mass of the research gap resides; the investigation of the currently unknown or un-explored complex edge case scenarios where the remote operator is going to be asked to solve unusual and often unique problems, and the investigation of non-driving related support and problem solving tasks. It remains to be seen what human presence, visual, virtual or otherwise, the users will want for them to trust and use the AV services being made available. Perhaps unsurprising, it may yet continue to be the case that “it is good to talk”.

A summary of all the key points and finding from the review are provided in

Table 6. Key observations and recommendations.

Topic	Observations
Remote Supervision: Limits of monitoring	Legacy research into optimal FO is largely inconclusive, offering findings of FO as low as 2 or as high as 12; When workload exceeds 70% of available time, performance in supervision significantly degrades; Simplistic FO calculations based upon time on task divided by time available fail to take into account the significant human factors effects of stress and teaming.
Remote Supervision: The effect of LOA	The higher the LOA of the system, the lower the workload and more spare capacity the operator has and thus the higher the FO possible; The more consistent the LOA the better the human performance (and conversely, the more variable the LOA the more errors made from forgetting what LOA the system was at; Even with high LOA, as FO increases so does operator workload, ultimately limiting the FO;
Remote Supervision: The limitation of supervision because of the control task	There is a general inverse relationship between the FO and the time it takes for the supervisor to interact with any given vehicle; While a remote operator attempts to drive, they are immediately and significantly limited in their ability to supervise and even drive other vehicles
Remote Monitoring: Active with limits or passive?	The effective operational range for Fan-Out is 4 to 8, optimized at 5 for proactive remote monitoring of an AV; Remote monitoring can induce high levels of subjective workload in remote operators, leading to degraded performance (reaction time and decision making); Further research into how the presence or absence of remote monitoring can affect human workload, performance and decision-making is needed.
Remote Driving: Observations and concerns	Technical issues such as data latency and the potential for data loss during communication and reduced situational information increase the difficulty and reduce the reliability of remote driving; Human factors issues such as reaction time, take-over time and time to build situation awareness challenges the viability of RD as a take-over solution for a moving AV or an AV in a dynamic environment; Many commercial AV organisations are not attempting to implement remote driving solutions (although there are organisations that provide RD of a non-automated vehicle as a service).
Remote Driving: Attempts and issues to overcome remote driving limitations	Attempts have been made to provide AVs with software that gives them the capability to modify RD-provided control inputs to allow the AV to “share” the driving task by modifying the path to be driven; Most shared driving solutions still result in “stop-and-go” style of driving as the AV attempts a path that, due to communication and processing delays, is quickly made obsolete; Shared driving solutions where the AV calculates and proposes multiple paths can overwhelm human decision-making. The optimum number of paths is three.
Remote Assistance: Future of remote operation?	Researchers have attempted to create categories or levels of remote driving and remote assistance; however, not all remote operator and AV teaming solutions fit neatly into the categories proposed; It is proposed that the point of difference between remote driving and remote assistance is the decision-making capability of the AV: ○ If the AV can modify or override the human input and the AV has the complete DDT, it is remote assistance; ○ If the human has control over the DDT, it is remote driving. A GUI designed to provide strategic goals and directives (e.g., route plans) is more adaptable to be used to support fleets of mixed vehicle types.
The impact of scenarios and ODD	A significant limiting factor for the supervision of multiple AVs is the surge in workload the operator is subject to when conducting remote assistance or remote driving; The intensity of the subjected workload has been linked to the complexity of the scenario that led to human interaction and the AV ODD at the point of interaction; More research is needed on scoping and categorising the complex scenarios that would lead or need human’s providing solution generation support.
User perception	Legacy research into user perception and potential uptake of use of AVs was largely hypothetical; Recent user interest and concerns have been less about how the AV might perform and more about the safety and security of users when no driver is present in the vehicle; There is a gap for research into unexpected consequences of AV use that will impact on the scope of the support task an RO is expected to provide.
Conclusion	Remote driving in support of AVs should be avoided wherever possible, or should be limited to situations where the AV is moving slowly (<4 ms), where the number of vehicles being monitored is low or the AV is in a geo-fenced environment with no other objects or road users; Future research should focus on: ○ Continuing to develop and test concepts of how remote assistance can be provided; ○ Investigating the currently unknown or unexplored complex edge case scenarios where the remote operator is going to be asked to solve unusual and often unique problems. Investigation of non-driving related support and problem solving tasks.

below.

5. Limitations

The authors acknowledge the limitations of this review, specifically that the literature sources are heterogenous, covering not only scientific findings and results but also many commercial and even some legal perspectives on the viability of conducting remote operations to support automated vehicles. The heterogenous nature extends to the use of literature discussing the remote operation of UAVs and generic robots, particularly in the sections discussing the concept of Fan-Out.

Furthermore, whilst the authors and some readers may consider the position of this review, in particular that on remote driving, to be transferrable to remote teleoperation of non-autonomous vehicles (i.e., remote chauffeuring, or remote delivery of hire cars), it must be reiterated that this review does not attempt to directly consider that specific use case. The positions of the paper are based upon consideration of literature pertinent to and discussing the specific case of remote operation: support to automated vehicles where the human remote operator is providing a standby support role.