A Helping Human Hand: Relevant Scenarios for the Remote Operation of Highly Automated Vehicles in Public Transport
Abstract
:1. Introduction
1.1. Regulation, Standardization, and Conceptualization of Remote Operation of AVs
1.2. Real-World Tests of Remote Operation
1.3. Rationale and Objectives
2. Materials and Methods
2.1. Process of Collecting Scenarios
2.1.1. Control Centers
- Time sampling: Control center staff’s behaviors were observed during a fixed time interval by researchers.
- Unstructured observation: Observations were conducted in a holistic way. The researcher entered the field with some general ideas of what might be salient, but not of what specifically will be observed, i.e., without using any pre-determined objectives, schedules, or variables (cf. [24]).
- Naturalistic setting: The process involved observing and studying the spontaneous behavior of the control center employees in their natural environment.
2.1.2. Highly Automated Vehicles
2.2. System of Structuring Scenarios
- Remote Operator, who may be both Remote Driver of Remote Assistant in SAE’s [8] terminology,
- Highly Automated Vehicles at SAE Level 4,
- Passengers,
- Infrastructure, e.g., crosswalks, traffic lights, or intelligent road-side infrastructures (including road-site units, i.e., sensors along the road that scan the traffic in their immediate surroundings and submit this information to a traffic management center, cf. [28]), and
- Other Actors, e.g., emergency services, control center staff, and other road users.
3. Results
3.1. Structure and Catalogue of Scenarios
3.2. Example for Scenario “Vehicles Parked in Second Row”
“Due to vehicles parked in the second row, the AV’s lane is blocked. This leads to a delay in the further travel of the AV. The RO allows the AV to continue, sets a new trajectory, e.g., by setting waypoints or selecting a pathway, or steers the AV manually around the obstacle.”
4. Discussion
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Camacho, F.; Cárdenas, C.; Muñoz, D. Emerging technologies and research challenges for intelligent transportation systems: 5G, HetNets, and SDN. Int. J Interact. Des. Manuf. 2018, 12, 327–335. [Google Scholar] [CrossRef]
- Ullah, H.; Gopalakrishnan Nair, N.; Moore, A.; Nugent, C.; Muschamp, P.; Cuevas, M. 5G Communication: An Overview of Vehicle-to-Everything, Drones, and Healthcare Use-Cases. IEEE Access 2019, 7, 37251–37268. [Google Scholar] [CrossRef]
- Van Mierlo, J.; Berecibar, M.; El Baghdadi, M.; De Cauwer, C.; Messagie, M.; Coosemans, T.; Jacobs, V.A.; Hegazy, O. Beyond the State of the Art of Electric Vehicles: A Fact-Based Paper of the Current and Prospective Electric Vehicle Technologies. World Electr. Veh. J. 2021, 12, 20. [Google Scholar] [CrossRef]
- Meyer, G.; Beiker, S. (Eds.) Road Vehicle Automation; Springer International Publishing: Cham, Switzerland, 2014; ISBN 978-3-319-05989-1. [Google Scholar]
- Arena, F.; Pau, G.; Severino, A. An Overview on the Current Status and Future Perspectives of Smart Cars. Infrastructures 2020, 5, 53. [Google Scholar] [CrossRef]
- Pavone, M. Autonomous Mobility. In Autonomes Fahren; Maurer, M., Gerdes, J.C., Lenz, B., Winner, H., Eds.; Springer: Berlin/Heidelberg, Germany, 2015; pp. 399–416. ISBN 978-3-662-45853-2. [Google Scholar]
- Machado, C.; de Salles Hue, N.; Berssaneti, F.; Quintanilha, J. An Overview of Shared Mobility. Sustainability 2018, 10, 4342. [Google Scholar] [CrossRef] [Green Version]
- Society of Automotive Engineers. Taxonomy and Definitions for Terms Related to Driving Automation Systems for on-Road Motor Vehicles; SAE: Washington, DC, USA, 2021; Available online: https://www.sae.org/standards/content/j3016_202104 (accessed on 2 July 2021).
- Shi, E.; Frey, A.T. Theoretical Substitution Model for Teleoperation. In Automatisiertes Fahren 2021; Bertram, T., Ed.; Springer Fachmedien Wiesbaden: Wiesbaden, Germany, 2021; pp. 69–81. ISBN 978-3-658-34753-6. [Google Scholar]
- Sag, A. The State of 5G in Early 2022. Forbes [Online]. Available online: https://www.forbes.com/sites/moorinsights/2022/01/18/the-state-of-5g-in-early-2022/?sh=1247e9125652 (accessed on 20 January 2022).
- Gesetz zur Änderung des Straßenverkehrsgesetzes und des Pflichtversicherungsgesetzes—Gesetz zum Autonomen Fahren. 2021. Available online: https://www.bundesrat.de/SharedDocs/beratungsvorgaenge/2021/0401-0500/0430-21.html (accessed on 17 March 2022).
- Rosenzweig, A. Regulators Know Teleoperation Is Key for Self-Driving Vehicles to Succeed. Available online: https://venturebeat.com/2021/06/17/regulators-know-teleoperation-is-a-must-have-for-self-driving-vehicles-to-succeed/ (accessed on 20 January 2022).
- The Knowledge Base on Connected and Automated Driving. France Takes Lead on Allowing Automated Driving on Public Roads. Available online: https://www.connectedautomateddriving.eu/blog/france-takes-lead-on-allowing-automated-driving-on-public-roads/ (accessed on 20 January 2022).
- National Conference of State Legislatures. Autonomous Vehicles | Self-Driving Vehicles Enacted Legislation. Available online: https://www.ncsl.org/research/transportation/autonomous-vehicles-self-driving-vehicles-enacted-legislation.aspx (accessed on 20 January 2022).
- Weimer, J. U-Shift—A Novel on-the-Road Modular Vehicle Concept; German Aerospace Center: Stuttgart, Germany, 2019. [Google Scholar]
- Hamburger Hochbahn. The Future Is Driverless: Be Part of the Hochbahn Research and Development Project Heat. Available online: https://www.hochbahn.de/hochbahn/hamburg/en/home/projects/expansion_and_projects/project_heat (accessed on 29 June 2021).
- RealLab Hamburg. Autonomes Fahren. Available online: https://reallab-hamburg.de/projekte/autonomes-fahren/ (accessed on 25 March 2021).
- Freie Universität Berlin. KIS-M—KI-basiertes System für Vernetzte Mobilität. Available online: https://www.mi.fu-berlin.de/inf/groups/ag-ki/Projects/KIS-M/index.html (accessed on 21 January 2022).
- DLR. U-Shift: Information on the Vehicle Concept. Available online: https://verkehrsforschung.dlr.de/en/projects/u-shift (accessed on 17 March 2022).
- Kettwich, C.; Dreßler, A. Requirements of Future Control Centers in Public Transport. In Proceedings of the AutomotiveUI ’20: 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Washington, DC, USA, 21–22 September 2020; ACM: New York, NY, USA, 2020; pp. 69–73. [Google Scholar]
- Kettwich, C.; Schrank, A.; Oehl, M. Teleoperation of Highly Automated Vehicles in Public Transport: User-Centered Design of a Human-Machine Interface for Remote-Operation and Its Expert Usability Evaluation. MTI 2021, 5, 26. [Google Scholar] [CrossRef]
- Kettwich, C.; Schrank, A.; Oehl, M. Supervising the Automation: Tasks and Human-Machine Interfaces for on-Board Operators of Highly Automated Shuttles in Real-World Laboratories across Germany. In Proceedings of the Annual Meeting of HFES Europe Chapter, Torino, Italy, 20–22 April 2022. [Google Scholar]
- International Organization for Standardization. ISO Standard No. 9421-210:2019. Ergonomics of Human-System Interaction–Part 210: Human-Centered Design for Interactive Systems; International Organization for Standardization: Geneva, Switzerland, 2019. [Google Scholar]
- The Sage Encyclopedia of Qualitative Research Methods. Given, L., Given, L.M., Eds.; Sage: Los Angeles, CA, USA, 2008; ISBN 9781412941631.
- Dreßler, A. User-needs-based design of public transport with autonomous vehicles: User-centered research in the project HEAT. In Proceedings of the 2021 SIP-Adus Workshop, Human Factors Breakout Session, Online, 29 October 2021. [Google Scholar]
- Ulbrich, S.; Menzel, T.; Reschka, A.; Schuldt, F.; Maurer, M. Defining and Substantiating the Terms Scene, Situation and Scenario for Automated Driving. In Proceedings of the IEEE International Annual Conference on Intelligent Transportation Systems, Las Palmas, Spain, 15–18 September 2015; pp. 982–988. [Google Scholar]
- Wilbrink, M.; Schieben, A.; Markowski, R.; Weber, F.; Gehb, T.; Ruenz, J.; Tango, F.; Kaup, M.; Jan-Henning, W.; Portouli, V.; et al. interACT D1.1. Definition of interACT Use Cases and Scenarios. 2018. Available online: https://www.interact-roadautomation.eu/wp-content/uploads/interACT_WP1_D1.1_UseCases_Scenarios_1.1_approved_UploadWebsite.pdf (accessed on 19 January 2022).
- Geis, T.; Tesch, G. Basiswissen Usability und User Experience: Aus- und Weiterbildung zum UXQB® Certified Professional for Usability and User Experience (CPUX)—Foundation Level (CPUX-F); dpunkt.verlag: Heidelberg, Germany, 2019; ISBN 9783960886297. [Google Scholar]
- Dzida, W.; Freitag, R. Making use of scenarios for validating analysis and design. IIEEE Trans. Softw. Eng. 1998, 24, 1182–1196. [Google Scholar] [CrossRef]
- DLR. Anwendungsplattform Intelligente Mobilität. Available online: https://www.dlr.de/ts/aim#gallery/25291 (accessed on 22 March 2022).
- German Aerospace Center. AIM ve hi cle Fleet. Available online: https://www.dlr.de/content/en/research-facilities/aim-vehicle-fleet.html (accessed on 11 February 2022).
- Schrank, A.; Kettwich, C.; Heß, S.; Oehl, M. Highly automated yet highly controlled: A case study of HAVs’ on-board operators’ workplaces across three real-world laboratories. In Proceedings of the 64th Conference of Experimental Psychologists (TeaP), Online, 20–23 March 2022. [Google Scholar]
- Eclipse Foundation. Automated Driving Open Research (ADORe). Available online: https://projects.eclipse.org/proposals/eclipse-automated-driving-open-research-adore (accessed on 22 February 2022).
- Lapoehn, S.; Heß, D.; Böker, C.; Böhme, H.; Schindler, J. Infrastructure-aided Automated Driving in Highly Dynamic Urban Environments. In Proceedings of the ITS World Congress 2021, Hamburg, Germany, 11–15 September 2021. [Google Scholar]
- TransAID. Transition Areas for Infrastructure-Assisted Driving. Available online: https://www.transaid.eu/ (accessed on 22 February 2022).
- Weimer, J.; Ulrich, C.; Conzelmann, M.; Fleck, T.; Zofka, M.R.; Grünhäuser, M. Managed automated driving: A new way for safe and economic automation. In Proceedings of the 27th ITS World Congress, Hamburg, Germany, 11–15 October 2022. [Google Scholar]
- Hi-Drive. Hi-Drive: Addressing Challenges towards the Deployment of Higher Automation. Available online: https://www.hi-drive.eu/ (accessed on 11 February 2022).
- Verband deutscher Verkehrsunternehmen. Stellungnahme zum Entwurf Eines Gesetzes zur Änderung des Straßenverkehrsge- setz und des Pflichtversicherungsgesetz. Available online: https://www.bundestag.de/resource/blob/838594/305010163d3d1daf817adbdec8483507/19-15-489-C-data.pdf (accessed on 11 February 2022).
- Lenz, K. Data-Centric Solutions for Future Mobility: GAIA-X 4 Future Mobility—Five Projects Presented; DLR: Cologne, Germany, 2022; Available online: https://fgvt.htwsaar.de/site/en/gaia-x-4ams-2021-2024/ (accessed on 17 March 2022).
- Schieben, A.; Wilbrink, M.; Kettwich, C.; Dodiya, J.; Oehl, M.; Weber, F.; Sorokin, L.; Lee, Y.M.; Madigan, R.; Merat, N.; et al. Testing External HMI Designs for Automated Vehicles—An Overview on User Study Results from the EU Project interACT; Tagung Automatisiertes Fahren: Munich, Germany, 2019. [Google Scholar]
- Lau, M.; Le, D.H.; Oehl, M. Design of External Human-Machine Interfaces for Different Vehicle Types. In Contributions to the 63rd Tagung Experimentell arbeitender Psychologen; Huckauf, A., Baumann, M., Ernst, M., Herbert, C., Kiefer, M., Sauter, M., Eds.; Pabst Science: Lengerich, Germany, 2021. [Google Scholar]
Use Case Cluster | Use Case | Cause | Event | Consequence | Description 2 (Is Scenario) | Actors | Mode of ROn 1 | |||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
RO 1 | AV 1 | P 1 | I 1 | OA 1 | RD 1 | RA 1 | ||||||
1 Interaction RO with Passengers | 1.1 Conflicts | Vandalism | AV 1 is damaged (inside and/or outside) | AV cannot continue its ride. | 1.1.1 Due to vandalism, the AV is damaged (inside and/or outside). This results in the AV being unable to continue its ride. | X | X | X | X | X | ||
NA 1 | Passenger abuses intercom | Control center employee is unnecessarily distracted | 1.1.2 Due to an undefined cause, a passenger abuses the intercom. This leads to unnecessary distraction of the control center staff. | X | X | X | X | |||||
NA | Passenger does not want to get off | Delay | 1.1.3 Due to an undefined cause, a passenger does not want to get off the AV. This leads to a delay of the AV. | X | X | X | X | |||||
NA | Disputes between passengers | AV cannot continue its journey | 1.1.4 Due to an undefined cause, there are disputes between the passengers. This leads to the AV not being able to continue its journey. | X | X | X | X | X | ||||
1.2 Incidents requiring intervention by RO | Mobility-impaired passenger would like to use AV | Mobility-impaired passenger receives support from the control center for using the AV | Mobility-impaired passenger uses AV | 1.2.1 Due to a request of a mobility-impaired passenger, they receive support to use the AV. This leads to the mobility-impaired passenger being able to use the AV. | X | X | X | X | X | |||
Emergency | Control center contacts passengers | Passengers is able to appropriately react to emergency | 1.2.2 Due to an emergency, the control center contacts the passengers. This leads to the passengers becoming enabled to appropriately react to the emergency. | X | X | X | X | |||||
Emergency | RO opens the door between regular stops | Passengers can leave the AV between regular stops | 1.2.3 Due to an emergency, the doors of the AV are opened by the RO between regular stops. This leads to the fact that passengers can leave the AV between regular stops. | X | X | X | X | X | ||||
Medical emergency | Passenger needs immediate medical treatment | Onward journey might be delayed | 1.2.4 Due to a medical emergency in the AV, a passenger needs immediate medical treatment. This might delay the onward journey. | X | X | X | X | X | ||||
NA | Passenger/RO finds ownerless suitcase | Possible security risk | 1.2.5 Due to an undefined cause, a passenger or the RO finds an ownerless suitcase. This may lead to a security risk. | X | X | X | X | |||||
Technical malfunction | AV stops | Onward journey of the AV delayed, restricted or not possible | 1.2.6 Due to a technical malfunction, the AV stops. This leads to the fact that the onward journey of the AV is delayed, restricted or not possible. | X | X | X | X | |||||
NA | Passenger would like to exit between regular stops | Onward journey delayed | 1.2.7 Due to an undefined cause, a passenger wants to exit the AV between regular stops. This leads to a delay in the onward journey of the AV. | X | X | X | X | |||||
Bulky objects inside the AV | Doors blocked | Doors do not close, AV cannot continue its journey, onward journey delayed | 1.2.8 Due to bulky objects inside the AV, the doors are blocked. This leads to the doors not closing and the AV not being able to continue its journey. The onward journey of the AV is delayed. | X | X | X | X | X | ||||
AV crowded | Passengers block the door | Doors do not close, AV cannot continue its journey, or onward journey is delayed | 1.2.9 Due to the AV being crowded, passengers block the door. This leads to the doors not closing and the AV not being able to continue its journey or the onward journey being delayed. | X | X | X | X | |||||
Spilled liquids, scattered food | AV’s interior dirty | Passenger satisfaction declines | 1.2.10 Due to spilled liquids or scattered food in the interior of the AV, the AV’s interior is dirty. This leads to declining passenger satisfaction. | X | X | X | X | |||||
1.3 Incidents not requiring intervention by RO | Emergency | Passenger applies emergency brake | AV stops, door opens, passengers can get off | 1.3.1 Due to an emergency, a passenger applies the emergency brake. This leads to the doors opening and the passengers being able to get off. | X | X | X | X | ||||
2 Interaction RO with AV | 2.1 Emanating from AV | Sensors detect obstacle | AV stops | AV cannot continue its journey/onward journey is delayed, AV waits for clearance from RO until it can resume ride | 2.1.1 Due to obstacles detected by sensors of the AV, the AV stops. This leads to the fact that the AV cannot continue its journey or the journey being delayed. The AV waits for the RO to give clearance so it can resume its ride. | X | X | X | ||||
Sensors detect obstacle (e.g., other road users, parked vehicles, trees, foliage, animals), technical malfunction | AV stops | AV cannot continue its journey/onward journey is delayed, AV waits for the RO to give clearance, select a trajectory or set waypoints so that AV can resume ride | 2.1.2 Due to obstacles detected by sensors of the AV, the AV stops. This leads to the fact that continuing the journey is not possible or delayed. The AV waits for the RO to give clearance, select a trajectory or set waypoints so that it can resume its ride. | X | X | X | X | |||||
Technical malfunction | RO receives an error/malfunction message | Resolving the error/malfunction | 2.1.3 Due to a technical malfunction, the RO receives an error/malfunction message. This leads to the RO resolving the error/malfunction. | X | X | X | X | |||||
AV is partially defective but still operable | AV stops | AV cannot resume ride/is delayed. AV waits for the RO to give clearance, select a trajectory or set waypoints so that AV can resume ride. Resuming ride only possible for AV until a safe stopping point with restrictions | 2.1.4 Due to the AV being partially defective but still operable, the AV stops. This leads to the fact that resuming the ride is not possible for the time being and is delayed. The AV waits for the RO to give clearance, select a trajectory or set waypoints so that the AV can continue its ride to a safe stopping point. | X | X | X | ||||||
Objects or parts of the AV on fire | Fire alarm activated | AV stops, cannot continue its journey | 2.1.5 Due to objects or parts of the AV being on fire, the fire alarm is activated. This results in the AV stopping and not being able to continue. | X | X | X | X | |||||
Smoke emitted by passengers from cigarette, e-shisha, etc. | Fire alarm activated | AV cannot continue its journey or onward journey is delayed | 2.1.6 Due to passengers emitting smoke from cigarettes, e-shishas, etc. in the AV, the fire alarm is activated. This results in the AV stopping and not being able to continue or the onward journey being delayed. | X | X | X | X | |||||
Technical malfunction | RO brakes too hard | Wheels block, critical situation or accident may occur | 2.1.7 Due to a technical malfunction of the AV, the RO brakes too hard. This makes the wheels block, a critical situation or accident may occur. | X | X | X | ||||||
Unforeseen situation (traffic jam, route change, dispatching error, etc.) | AV battery’s state of charge low | AV cannot reach the planned charging station or AV cannot complete the roundtrip as planned | 2.1.8 Due to an unforeseen situation (traffic jam, route change, dispatching error, etc.), the battery has a low state of charge. This leads to AV not being able to reach the planned charging station or to complete the roundtrip as planned. | X | X | X | X | |||||
AV conducted roundtrip with high energy consumption, e.g., air conditioning, traffic jam | State of charge not sufficient for another roundtrip | AV needs to be charged earlier than planned | 2.1.9 Due to a roundtrip with high energy consumption, the state of charge is not sufficient for another roundtrip. This leads to the fact that AV needs to be charged earlier than planned. | X | X | X | X | |||||
Technical malfunction | Charging not possible | AV cannot continue its journey | 2.1.10 Due to a technical malfunction, charging the AV’s battery is not possible. This leads to the AV not being able to continue its journey. | X | X | X | X | X | ||||
Demand for charging stations exceeds supply | No free charging stations | Charging not possible | 2.1.11 Due to the demand for charging stations exceeding the supply, the AV cannot find a free charging station. This leads to the fact that the AV cannot be charged. | X | X | X | X | X | ||||
Allocated charging station is out of order | Charging not possible | AV cannot continue its journey | 2.1.12 Due to the allocated charging station being out of order, charging is not possible. This leads to the fact that the AV cannot continue its journey. | X | X | X | X | X | X | |||
2.2 Emanating from RO | RO’s misbehavior | RO is inattentive or distracted, does not recognize street signs | AV overrides RO’s input because it is no longer compliant with traffic rules | 2.2.1 Due to misbehavior of the RO during direct control, the inattentive or distracted RO does not recognize the street signs. This leads to the AV overriding the RO’s input because it is no longer compliant with the traffic rules. | X | X | X | |||||
Accident on driving route | Closure of parts of the route | AV cannot continue route as planned (with or without stopping), planned route is adjusted | 2.2.2 Due to an accident on the AV’s route, there is a closure of parts of the AV’s planned route. This means that the AV cannot continue its route as planned (with or without stopping). The route must be adjusted. | X | X | X | ||||||
Major event, protest, construction site, etc. | Change of route | Changed order of stops; stops may be omitted, RO contacts passengers | 2.2.3 Due to a major event, protest, construction site, etc., there is a change in the route. This leads to a change in the order of the stops and, if necessary, stops might be skipped. The RO contacts the passengers and lets them know about the changes. | X | X | X | X | X | ||||
RO’s misbehavior | RO brakes too hard | Wheels block, accident | 2.2.4 Due to the misbehavior of the RO, the AV brakes too hard. This leads to the wheels blocking and the risk of a critical situation or an accident. | X | X | X | ||||||
Distraction RO | RO accidentally goes off-track | Critical situation or accident | 2.2.5 Due to a distraction of the RO during remote driving, the AV goes off-track. This leads to a critical situation or an accident. | X | X | X | ||||||
Technical malfunction, latency too high | RO accidentally goes off-track | Critical situation or accident | 2.2.6 Due to a technical malfunction and a too-high latency in direct control, the AV accidentally goes off-track. This leads to a critical situation or an accident. | X | X | X | ||||||
AV is urgently needed | No complete charging possible | Range of the AV is restricted | 2.2.7 Due to an urgent need of the AV, it is not possible to fully charge the AV’s battery. This leads to a limited range of the AV. | X | X | X | X | X | ||||
2.3 Source unclear | Soiling of the AV’s interior camera | Camera images from the AV’s interior cannot be recognized by RO | RO may not notice passenger interactions (e.g., disputes) | 2.3.1 Due to a soiled camera inside the AV, the RO cannot see the camera images inside the AV. This leads to the RO not noticing passenger interactions (e.g., disputes). | X | X | X | X | X | |||
Soiling of the sensors | Sensors limited in their functionality | AV cannot continue its journey or onward journey delayed | 2.3.2 Due to soiled sensors on the AV, the functionality of the sensors is limited. This leads to the AV not being able to continue its journey or in a delayed onward journey. | X | X | X | X | X | ||||
Connectivity issues between control center and AV | RO loses connection to AV | RO cannot establish contact with AV | 2.3.3 Due to connectivity issues between the control center and the AV, the RO loses connection to the AV. This results in RO no longer being able to contact the AV. | X | X | X | X | X | ||||
Connectivity issues of audio stream from AV to control center | Strong noise interference | RO cannot understand passengers | 2.3.4 Due to connectivity issues of the audio stream between the control center and the AV, there is strong noise interference, resulting in the RO not being able to understand the passengers. | X | X | X | X | X | ||||
Connectivity issues of video stream from AV to control center | Desired camera image is not transmitted in sufficient quality | AV can only continue journey with restrictions | 2.3.5 Due to connectivity issues of the video stream between the control center and the AV, the desired camera image is not transmitted in sufficient quality. This leads to the fact that the AV can only continue its journey to a limited extent. | X | X | X | X | |||||
3 Interaction AV with infrastructure | 3.1 Road-site units | Unmapped construction site | Road-site units support AV while maneuvering through unmapped construction site | AV can continue its journey without stopping | 3.1.1 Due to an unmapped construction site, road-site units support the AV while maneuvering through the unmapped area. This leads to the AV being able to continue its journey without stopping. | X | X | |||||
4 Interaction RO with infrastructure | 4.1 Road-site units | Road infrastructure is disrupted | Unstable/disturbed connection from the AV to the road-site units | RO is informed and forwards error so that it can be fixed as quickly as possible | 4.1.1 Due to a disruption of the road infrastructure, the connection from the AV to the road-site units is unstable or disturbed. This leads to RO being informed about the unstable or disturbed connection. The RO forwards the fault to the relevant actors as quickly as possible so that the faults can be fixed as soon as possible. | X | X | X | X | X | ||
5 Interaction AV with other actors | 5.1 Other road users | Vehicle or VRU is occluded by object or outside the focus area | AV reacts abruptly to suddenly appearing vehicle or VRU | Critical situation or accident | 5.1.1 Due to the occlusion of a vehicle or VRU 1 by another object or because a vehicle or VRU is outside of the focus area, the AV reacts abruptly to the suddenly appearing vehicle or VRU. This leads to a critical situation or an accident. | X | X | |||||
Several vehicles reach an intersection without a right-of-way sign at the same time and congest it | Deadlock occurs at an intersection without a right-of-way sign (left-yields-to-right rule) | RO gives clearance | 5.1.2 Due to vehicles reaching the intersection from all directions at the same time, a deadlock occurs. This leads to the RO giving clearance and permitting the AV to proceed if the other road users do not resolve this situation beforehand. | X | X | X | X | |||||
VRU or animal suddenly or unexpectedly crosses the intersection | AV performs emergency braking maneuver | Critical situation or accident | 5.1.3 Due to a suddenly crossing or unexpected VRU or animal, the AV has to perform an emergency braking. This leads to a critical traffic situation or an accident. | X | X | X | X | |||||
VRU’s misbehavior or technical malfunction | AV collides with VRU | AV cannot continue its journey | 5.1.4 Due to a misbehavior of the VRU or a technical malfunction, the AV collides with a VRU. This leads to the AV being unable to continue its journey. | X | X | X | X | X | ||||
Vehicles parked in second row | Pathway blocked | Continuation of ride delayed; RO assesses situation and guides AV around the obstacles | 5.1.5 Due to vehicles parked in the second row, the AV’s pathway is blocked. This leads to a delay in the onward travel of the AV. The RO gives clearance to the AV to continue, sets a new pathway, e.g., by setting waypoints or selecting a pathway, or steers the AV manually around the obstacles. | X | X | X | X | X | ||||
Accident | AV stops | AV cannot continue its journey | 5.1.6 Due to an accident, the AV stops. This leads to the AV not being able to continue its journey. | X | X | X | X | |||||
Driver’s distraction, difficult weather conditions, etc. | Collision of another vehicle with AV | Damage to the AV by another vehicle, AV cannot continue its journey, damage must be fixed and passengers must be informed | 5.1.7 Due to the driver’s distraction, difficult weather conditions, etc., another vehicle collides with the AV. This leads to the AV being damaged by the other vehicle and being unable to continue its journey. The RO conducts procedure as with accidents in general (i.e., contacts passengers, informs blue light organization, organizes a substitute vehicle if necessary, documentation, etc.). | X | X | X | X | X | X | |||
Major event, protest, construction site, etc. | Traffic jam | Onward journey is delayed | 5.1.8 Due to a major event, protest, construction site, etc., a traffic jam occurs. This leads to a delay in the onward journey of the AV. | X | X | X | X | X | ||||
6 State RO | 6.1 Sickness | NA | RO sick | RO cannot fulfill supervisory duty | 6.1.1 Due to an undefined cause, the RO is sick. This leads to the RO being unable to fulfill their supervisory duties or to perform their tasks. | X | X | X | X | |||
6.2 Distraction | Fatigue, stress, distraction in general, etc. | RO distracted | RO cannot fulfill supervisory duty | 6.1.2 Due to fatigue, stress, distraction in general, etc., the RO is distracted. This leads to the RO being unable to fulfill their supervisory duties or to perform their tasks. | X | X | X | X | ||||
7 Contextual Factors | 7.1 Weather conditions | Difficult weather conditions | Overload or soiling of the sensors on the vehicle | AV cannot continue its journey | 7.1.1 Due to difficult weather conditions, the sensors on the AV are overloaded or soiled. This leads to the fact that they cannot detect the environment accurately so the AV cannot continue its journey. | X | ||||||
Difficult weather conditions, such as icy roads, fog, heavy rain, or snowfall | AV goes off-track, no other parties involved | Critical situation or accident, onward journey is delayed, restricted, or not possible | 7.1.2 Due to difficult weather conditions, such as icy roads, fog, heavy rain, or snowfall, the AV goes off-track. There are no other parties involved. This leads to a critical situation or accident and to the fact that the continuation of the AV’s journey is delayed, restricted, or not possible. | X | ||||||||
Difficult weather conditions, such as icy roads, fog, heavy rain, or snowfall | AV comes off the road, there are other parties involved | Accident with internal and external damage, onward journey is delayed, onward journey is delayed, restricted, or not possible | 7.1.3 Due to difficult weather conditions, such as icy roads, fog, heavy rain, or snowfall, the AV goes off-track. There are other people involved. This leads to an accident with internal and external damage and the fact that the continuation of the AV’s journey is delayed, restricted, or not possible. | X | X | |||||||
Flood, fire, etc. | Traffic routing changed, road closed, speed limit changed | Planned route changed, planned stops skipped | 7.1.4 Due to a flood, fire, etc., for example, the traffic routing was changed, a road was closed, or the speed limit was changed. This leads to the planned route being changed and planned stops being skipped. | X | X | X | ||||||
8 Technical malfunction | 8.1 Internal cause | Technical malfunction | Doors do not close | AV cannot continue its journey | 8.1.1 Due to a technical malfunction, the doors of the AV do not close. This leads to the AV not being able to continue its journey. | X | X | |||||
Technical malfunction | Doors do not open | Passengers can neither get on nor off | 8.1.2 Due to a technical malfunction, the doors of the AV do not open. This leads to passengers not being able to get on or off and the AV not being able to continue its journey. | X | X | |||||||
Leakage in tank or pipes | AV loses liquid | AV cannot continue its journey; may obstruct other road users | 8.1.3 Due to a leakage in the tank or pipes, the AV loses liquid. This leads to the fact that it is not possible for the AV to continue its journey and other road users being obstructed. | X | X | |||||||
GPS receiver malfunction or connectivity issues | AV cannot be localized | Monitoring and controlling of the AV by RO not possible | 8.1.4 Due to a defective GPS receiver or connectivity issues, the AV cannot be localized. This leads to the fact that monitoring and controlling the AV by the RO is not possible. | X | X | X | X | |||||
Technical malfunction | AV’s fire detector disturbed | Passengers’ safety cannot be guaranteed | 8.1.5 Due to a malfunction, the AV’s fire detector is disturbed. This leads to the fact that the safety of the passengers cannot be guaranteed. | X | X | X | X | |||||
Technical malfunction | Air conditioning out of order | Overheating of the passenger cabin; reduction of comfort or impairments of passengers’ health | 8.1.6 Due to a technical malfunction, the air conditioning in the AV is out of order. This leads to an overheating of the passenger cabin and to a reduction in comfort or impairments of passengers’ health. | X | X | |||||||
Technical malfunction | Signal and/or brake light fails | RO receives an error message | 8.1.7 Due to a technical malfunction, a signal and/or brake light of the AV fails. This leads to the RO receiving an error message. | X | X | X | X | |||||
Technical malfunction | Headlight cleaning system defective | Limited visibility of the AV | 8.1.8 Due to a technical malfunction, the headlight cleaning system of the AV is defect. This leads to limited visibility of the AV. | X | ||||||||
Technical malfunction | Problems with thermal management | Danger of overheating of the vehicle’s battery or its interior | 8.1.9 Due to a technical malfunction, the AV has problems with the thermal management. This leads to an overheating of the vehicle’s battery or its interior. | X | X | |||||||
Failure of parts of the sensors | MRM 1 | AV cannot continue its journey and may endanger passengers and other road users due to AV’s location on the road | 8.1.10 Due to a partial failure of the AV’s sensor system, the AV performs an MRM. This leads to the AV being unable to continue its journey and possibly endangering passengers and other road users due to AV’s location on the road. | X | X | |||||||
Interface between AV and control center disturbed | Target and actual direction of travel do not align | AV drives on the wrong pathway | 8.1.11 Due to a malfunction in the interface between the AV and the control center, the AV’s planned and actual driving directions do not align. This leads to the AV driving on the wrong pathway. | X | X | X | ||||||
Interface between AV and control center disturbed | Target and actual speed do not align | AV drives at the wrong speed | 8.1.12 Due to a malfunction in the interface between the AV and the control center, the desired and actual speed of the AV do not align. This leads to the AV driving at the wrong speed. The AV decides to adapt the speed according to the traffic regulations. | X | X | X | ||||||
8.2 External cause | Spiky objects on the roadway | Damaged tire | AV can continue to drive if necessary but with increased risk of accident; onward journey is delayed | 8.2.1 Due to spiky objects on the road, a tire of the AV is damaged. This leads to the AV being able to continue driving if necessary but with increased risk of accident. The onward journey is delayed. | X | |||||||
Collision with an obstacle: vehicle hits curb or boundary of construction site | Damage to the AV, possibly further damage | AV cannot continue its journey | 8.2.2 Due to a collision with an obstacle, such as a curb or a boundary of a construction site, the AV is damaged. Further damage may occur. This leads to the AV not being able to continue its journey. | X | ||||||||
8.3 Unknown cause | NA | AV stops | AV cannot continue its journey; onward journey is delayed | 8.3.1 Due to an undefined cause, the AV stops. This leads to the fact that the AV cannot continue its journey or the onward journey is delayed. | X | |||||||
Technical malfunction | AV cannot be controlled remotely | AV may endanger passengers and other road users | 8.3.2 Due to a technical malfunction, the AV cannot be controlled remotely. This leads to the AV not being able to continue driving or the onward journey being delayed. Passengers and other road users may be endangered. | X | X | X | X | |||||
Connectivity issues | RO cannot connect to AV | AV no longer complies with all safety regulations, may endanger passengers and other road users | 8.3.3 Due to connectivity issues, the RO cannot connect to the AV. This leads to the fact that the AV no longer complies with all safety regulations. Passengers and other road users may be endangered. | X | X | X | X | |||||
Technical malfunction or transmission error | RO fails to identify the road signs | Onward journey is delayed | 8.3.4 Due to a technical malfunction or transmission error, the RO does not recognize the road signs. This leads to the onward journey being delayed. | X | X | X | ||||||
Poor object identification | Unclear detection situation | AV cannot continue its journey, AV waits for the RO to give clearance so it can continue journey | 8.3.5 Due to poor object identification, the detection situation of the AV is unclear. This leads to the AV not being able to continue its journey. The AV waits for the RO to give clearance so it can continue its journey. | X | X |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Kettwich, C.; Schrank, A.; Avsar, H.; Oehl, M. A Helping Human Hand: Relevant Scenarios for the Remote Operation of Highly Automated Vehicles in Public Transport. Appl. Sci. 2022, 12, 4350. https://doi.org/10.3390/app12094350
Kettwich C, Schrank A, Avsar H, Oehl M. A Helping Human Hand: Relevant Scenarios for the Remote Operation of Highly Automated Vehicles in Public Transport. Applied Sciences. 2022; 12(9):4350. https://doi.org/10.3390/app12094350
Chicago/Turabian StyleKettwich, Carmen, Andreas Schrank, Hüseyin Avsar, and Michael Oehl. 2022. "A Helping Human Hand: Relevant Scenarios for the Remote Operation of Highly Automated Vehicles in Public Transport" Applied Sciences 12, no. 9: 4350. https://doi.org/10.3390/app12094350
APA StyleKettwich, C., Schrank, A., Avsar, H., & Oehl, M. (2022). A Helping Human Hand: Relevant Scenarios for the Remote Operation of Highly Automated Vehicles in Public Transport. Applied Sciences, 12(9), 4350. https://doi.org/10.3390/app12094350