The Need for Cooperative Automated Driving

Abstract

In this paper we describe cooperation and social dilemmas in multiagent systems, with an analogy applied to road traffic. Cooperative human drivers, based on their perception of trust and fairness, find efficient solutions for such dilemmas. In the development of automated vehicles (AVs) it is therefore important to ensure that this cooperative ability is maintained even without a human driver. Therefore, the topic of cooperative intelligent transport systems (C-ITSs) is discussed in detail and different characteristics of cooperation and their implementation are derived. Further, three planning levels with the corresponding communication techniques are discussed and several methods for maneuver planning are listed. All in all, we hope that this paper will allow us to better classify different cooperative scenarios, develop novel approaches for cooperative AVs (CAVs), and emphasize the need for cooperative driving.

1. Motivation

Interaction between vehicles in road traffic is inevitable and, to date, these interactions are based on the human driving style, which depends on factors such as the driver´s experience or current emotional situation. With higher automation levels however, the driver will become obsolete, with the advantage being that automated vehicles will drive more rationally and we will no longer see acts of spite in traffic. Yet, there is also a challenge in the replacement of the human driver with an automated system. Until now, the human has been the only cooperative module within the vehicle, and its removal it will also eliminate mutualism and altruism from our traffic, leaving selfish vehicles that strictly follow traffic rules and optimize only their own costs. The German road traffic regulations (StVO) state in the first paragraph that “Use of the road requires constant care and mutual respect” [1]. While this generic statement is understandable for a human driver, an automated system needs a clear formalization of this rule, and this so far does not exist. Without cooperation, traffic will lose efficiency, with conflicts and aggressive behavior [2,3].

Imagine a situation with heterogenous agents such as fast cars and slow heavy trucks, where the trucks want to drive from the on-ramp onto the freeway. Due to their mass, their acceleration and maximal velocity are limited. According to the StVO, the cars on the freeway have the right of way and with no large gap between the cars, the truck has to stop and wait for that gap, thereby shortening the acceleration lane. With more than one truck, this could lead to a full blockage of the acceleration line, not only for the slow trucks but also for fast cars. This problem is a social dilemma, where selfish interactions bring the most benefit for individual agents (cars), until all agents act selfishly and the whole group suffers from increased costs.

In Section 2 we explain the concepts of cooperation and social dilemmas and then reflect on this using several transport scenarios to identify real-life social dilemmas and problems that can arise due to lack of cooperation. In Section 3 we then discuss how cooperation in road transport, i.e., cooperative intelligent transport systems (C-ITS), can be characterized, and we give three examples of cooperative scenarios. In Section 4 we indicate how cooperation can be implemented in maneuver planning and describe how this actually happens in nature by animals and human drivers. Then, in Section 5 we present in detail how cooperation can be implemented for automated vehicles based on three previously presented features, and conclude the paper with a discussion of the upcoming problems and challenges in Section 6 and a conclusion in Section 7.

2. Cooperation and Social Dilemmas in Multi Agent Systems

Road traffic, with its various road users in the shared environment of the road, is a classic example of a multi-agent system (MAS) [4]. In the following we will discuss MASs in general, as well as cooperation among agents and the resulting social dilemmas.

2.1. Multi-Agent System

A multi-agent system consists of different agents which are located in the same environment and try to collectively reach a common goal. Using the example of the Sense–Plan–Act concept of an automated vehicle (AV) [5] and in accordance with the general MAS definition of Dorri et al. [6], we define an agent and its environment as an entity with the following properties:

Agent (e.g., Vehicle)

• State: Properties of an agent including the action history. Can be constant or dynamic (e.g., vehicle length = 3 m, velocity = 80 km/h).
• Perception (Sense): Ability to sense the surroundings or just parts of it. Might be noisy (e.g., a radar detecting an object or a communication module receiving a message).
• Strategy (Plan): Decides which action is executed to reach the goal (e.g., to reach the desired speed the first action could be to accelerate).
• Action (Act): The agent’s capability to influence its own or other states for a certain cost (e.g., accelerating and breaking or sending a message).
• Costs: The sum of all costs generated by the performed actions (e.g., fuel consumption and travel time).
• Private Goal: A certain amount of cost to the agent that should not be exceeded (e.g., arrival time within 2 h).

Environment (e.g., Road Network)

• State: Properties of the environment (e.g., the weather conditions).
• Road: The drivable area on which the agents can move (e.g., two lanes).
• Infrastructure: Elements to support the agents with information (e.g., traffic lights).
• Constraints: Limitation of the agent´s action scope (e.g., the road traffic regulations.

The core element of an agent is the strategy for the location of decision-making. With several agents this can be combined for distributed problem-solving (DPI) [6], i.e., reaching a greater common goal. Therefore, we define the MAS as the combination of agents working together in an environment towards a common goal, as shown in Figure 1.

Multi Agent System (e.g., Road Traffic)

• Agents: An arbitrary number of entities performing actions and creating costs (e.g., traffic participants such as cars, trucks, and pedestrians).
• Environment: One shared environment that allows the agents to interact (e.g., the road network in which the agents are located).
• Common Goal: The summed agent’s cost within the MAS that should be optimized. Might conflict with the private goals of the agents (e.g., zero fatalities and low CO₂ emissions).

2.2. Cooperation

With multiple agents in one environment, the action of one agent might also influence other agents. If an action intentionally influences at least one other agent it is called an interaction. There are four motivations for interactions which differ in the way the costs of the involved agents are influenced [5,7]:

• Mutualism: An agent interacts in such a way that the costs of all involved agents are reduced.
• Altruism: An agent interacts in such a way that his/her own costs increase while the costs of other agents are reduced.
• Selfishness: An agent interacts in a way that his/her own costs are reduced while the costs of other agents increase.
• Spite: An agent interacts such in a way that the costs of all involved agents increase.

Further, an action of an agent is stated to be cooperative if he/she reduces the costs for other or for all involved agents, i.e., through mutualism and altruism. Therefore, cooperation is not necessarily bidirectional, as an altruistic interaction requires a selfish interaction from the counterpart. Acting consequently selfish in a group of mainly cooperative agents is called free-riding and allows one individual agent to achieve very low private costs, while the global cost, i.e., the sum of all private costs, increases. The question of to what extent free-riding can be applied or tolerated by the agents leads us to social dilemmas.

2.3. Social Dilemma

Cooperation in general has the underlying intention of reducing costs; however, if the cooperation is not mutual, this may result in increased costs for some of the involved agents [7]. The question arises as to why an agent should act cooperatively if there is the risk that his/her action will only benefit other agents and increase the agent´s own costs. Such situations are widely discussed within the fields of psychology and game theory, and are called social dilemmas. They are defined by two properties [8]:

• Each individual is sometimes tempted to defect (e.g., for lower costs in the short-term).
• Collective defection leads to higher cost than collective cooperation.

One famous example for a so-called social dilemma is the prisoner’s dilemma, as illustrated in

Table 1. The prisoner’s dilemma with the actions of Agent A and B with the resulting time in prison.

	B	Cooperate		Betray
A
Cooperate			1		0
		1		3
Betray			3		2
		0		2

: Two prisoners can choose between betraying their partner or remaining silent (cooperating). If both remain silent, both have to serve one year; if they betray each other, both have to serve two years; and if only one betrays, the one who remained silent has to serve three years while the other one is set free. This example shows that there is no trivial solution to the question of whether an agent should rather cooperate or defect when the intentions of the other agents are not clear [8,9]. Yet, the prisoner’s dilemma illustrates only one of many possible social dilemmas. Further famous social dilemmas include the chicken or assurance dilemmas which are, among others, discussed in depth in “The psychology of social dilemmas: A review” [8].

The Nash equilibrium, i.e., when no agent can improve his/her situation by decision alone independently of the decisions of others, occurs for the prisoner’s dilemma when the agents betray each other [3]. This can lead to zero or two years in prison, while cooperating can lead to one or three years in prison. Simultaneously, we see that the globally optimal solution, i.e., a total prison time of two years, can only be achieved if both agents cooperate. Knowing from the start, however, that an agent is in a group of cooperating agents, he can minimize his private costs by betraying the other cooperating agents. Therefore, an important element in the decision-making process is trust and fairness. If the two agents were to be able to communicate prior to settlement, they could agree on the global optimum instead of the Nash equilibrium with an accumulated prison time of four years.

Imagine a variation of the prisoner’s dilemma where a selfish agent can optimize his/her local cost and fulfill its goal by defecting, while a global optimal cost can only be achieved with cooperative behavior. This is similar to the example of the freeway on-ramp with the fast cars and slow heavy trucks. Each agent (car) on the freeway has to decide: “Should I be cooperative and slow down, or should I defect and let the car following me handle the situation?” [10]. With a high amount of defection due to selfish agents, the trucks on the on-ramp cannot find a gap for a lane-change and must come to a full stop, resulting in large costs. If one car on the freeway wants to cooperate after the previous ones have defected, the costs for this altruistic cooperation are already much higher than they would have been for the previous agent (as the truck on the on-ramp has to accelerate from a full stop), leading to even more defection. However, if all agents cooperate and collectively reduce their velocity so that they form a zip-like system (alternating freeway and on-ramp), no further breaking or full stops are required, resulting in a globally optimal solution. A variation of this social dilemma with a fleet of miniature cars resulted in a 35% increased throughput when the cars acted cooperatively as compared to selfishly [11].

The choice for cooperation or defection is mainly based on the cost of interaction, but trust and fairness must also be taken into account [12]. A human agent is less likely to cooperate with an untrustworthy agent and is more likely to cooperate if he/she considers the interaction as fair. In a cooperative environment, cooperation can be exploited when the trustworthiness and fairness of the agents is not tracked. An agent (car) that often asks for cooperation (e.g., to drive onto the freeway) but never grants it is considered a free-rider [13]. As long as the environment is cooperative, the free-rider can achieve very low costs, while the individual costs of the remaining agents and the global costs are not significantly worsened.

Cooperation and social dilemma in road traffic are large research areas, not only for human drivers but also for the development of automated vehicles (AVs) [3,14]. When it comes to developing cooperative AVs, it is important to have a precise definition of the desired cooperation. In road traffic, there are a variety of scenarios in which humans cooperate, although the type of cooperation in each situation can be fundamentally different. It is therefore not enough to say that a cooperative system is being developed, and it is necessary to clearly define the scenarios for which this system is intended. In the following, we give an overview of how different types of cooperation can be described and then we assign three exemplary situations from road traffic to the types of cooperation.

3. Cooperation in Road Traffic

In road traffic, there are many agents (vehicles) in a common environment that follow a set of rules that lead to certain total costs, i.e., the sum of agent costs (local costs), which in most cases is equivalent to the traffic flow. Inefficient driving or accidents reduce the traffic flow; therefore, driving strategies for the vehicles should be found in accordance with the road traffic rules in order to increase efficiency and reduce accidents. A road traffic scenario (hereinafter only referred to as a scenario) is defined by a snapshot of the road traffic, the duration, and the resulting costs, and can either be taken from a real situation or constructed so that there is an unlimited number of scenarios. In each snapshot, the environment, agents, rules, and costs are defined and serve as a starting point. From there, the agent’s strategy selects actions over the duration of the scenario that lead to specific costs. The combined actions of all agents form a maneuver that can be described via the agents’ trajectories and has the same duration and cost as the scenario. The goal is to find strategies for the agents to produce minimal global costs during a scenario, and one approach is to implement cooperative strategies that lead to cooperative maneuvers, i.e., the agents collectively seek an interaction that lowers global costs and supports fairness. Since there are unlimited snapshots, each of which can lead to unlimited maneuvers, the number of cooperative scenarios is enormous [15,16].

3.1. Characteristics of Cooperation

We have developed characteristics of cooperation in order to work out differences and similarities from the variety of cooperative scenarios. The following eight features help to classify cooperative scenarios and maneuvers in road traffic and highlight various aspects to consider when developing cooperative driving systems:

3.1.1. Agents Involved

The number and homogeneity of the vehicles are of interest to the agents involved. Cooperation can take place between two vehicles of the same type, for example between two trucks, but also between a variety of different types of vehicles, such as trucks, cars, and ambulances. Cooperation requires at least two agents, but this number is not limited.

3.1.2. Location

The location can play an important role in the analysis of whether and which cooperative scenario is required. For example, there are static cooperation areas at on-ramps where the type of cooperation required (the creation of a gap for on-ramping vehicles) is already fixed. At these locations, maneuver coordination may also be supported by communication with the infrastructure (roadside units). A platoon, on the other hand, is not tied to a single location but is created at any point on the freeway and extends over long distances.

3.1.3. Urgency and Costs

In order to decide whether a cooperative scenario is necessary, urgency is a crucial factor. Urgency is not symmetrical for all actors involved and can be expressed by the costs that a single agent would avoid by collaborating. For example, if cooperation only brings comfort-related gains, the avoidable costs are very low (low urgency), as compared to cooperation to prevent an accident (high urgency).

3.1.4. Interaction Type

The interaction types can be divided into passive and active cooperation. Only the exchange of information, without adapting one’s own maneuver plan to create a mutual or altruistic benefit, is a passive form of cooperation and causes no direct costs. This includes, e.g., wireless communication of messages such as cooperation requests or acknowledgments. If, however, one’s own maneuver is adapted, e.g., a lane change is carried out to enable cooperation, we speak of active cooperation.

3.1.5. Duration

The duration of the cooperative scenario has a direct influence on the required planning horizon. For a short duration, such as a lane change on the freeway in order to open a gap for a vehicle on the on-ramp, a forecast of a few seconds is sufficient. However, a truck-overtaking maneuver for example can take up to 45 s, and platooning will require planning over several hours.

3.1.6. Mutuality

In mutual cooperation, all the agents involved benefit (mutuality), but cooperation sometimes requires a subset of the agents involved to accept higher costs in order to reduce global costs (altruism). Therefore, it is necessary to identify which agents are the profiteers and which grant cooperation. In order to increase the motivation for cooperation, an altruistic interaction can be transformed into a mutual interaction if the profiteers compensate the altruistic costs via another channel, e.g., a micropayment.

3.1.7. Preparation Time

The preparation time is the time from the first consideration of the cooperation to the execution of the first cooperative interaction and thus includes the available time for the planning of the cooperative maneuver. In the case of platooning, for example, it can occur that before the maneuver begins it has already been determined which vehicles are to form a platoon at which point. The cooperation is therefore prepared well in advance. In most cases, however, the cooperation takes place spontaneously with only a few seconds of preparation time, and in emergency situations with no preparation time at all.

3.1.8. Initiation

Since several agents are involved in the cooperation, there are also various possible initiators, namely a profiter, a cooperation granter, or a coordinating master. In addition, a distinction can be made between offering, requesting, and announcing a specific interaction for cooperation. For example, the cooperation granter could offer cooperation and the profiter could request cooperation, while a coordinating master could announce certain cooperative interactions and enforce them through sanctions.

3.2. Examples of Cooperative Situations

Three exemplary scenarios are presented in order to put the above-mentioned characteristics of cooperative maneuvers into context. These show that cooperation can take many different forms and, in addition, that there is not only one correct way to implement cooperation, but many different possibilities.

3.2.1. Platooning

The general idea of platooning as shown in Figure 2 is to drive with a reduced distance between vehicles in order to use the slipstream of the vehicle in front [17]. The reduced air resistance reduces fuel consumption and the carrier saves money [18]. Therefore, two or more agents are involved, following each other on the motorway. In the short term, the urgency and potential cost savings are very low, but with consistent operation over long distances, fuel savings of up to 10% can be achieved [19]. The type of interaction depends on the position in the platoon; the leading vehicle has a passive role and only exchanges information, especially about its braking maneuvers. This is necessary so that the following vehicles can drive at a reduced distance and react quickly to a breaking situation, thereby performing active maneuvers. This form of cooperation is mutual in the sense that it benefits all agents involved, although the fuel savings for the leading vehicle are significantly lower than for the vehicles in the slipstream. Therefore, although all agents benefit, the benefits are not fairly distributed. A platoon can be formed spontaneously on the motorway or planned well in advance to ensure that two suitable trucks actually meet on the freeway. In both cases, the time available to prepare the cooperation is long and there is no urgency. The biggest profiteer is a following vehicle that wants to enter the platoon, so one can assume that this vehicle should initiate the cooperation and request the transmission of the signals. However, it is also conceivable that a central authority, e.g., a fleet manager of the carrier, initiates the cooperation and forces the vehicles into a platoon.

3.2.2. Lane Merge

In the drive-on scenario or generally in lane-merging scenarios as depicted in Figure 3, the idea is to create fairness by alternately merging vehicles from both lanes, creating a zipper-like pattern [11,20]. Although there may be many vehicles in such a scenario, in most cases only two agents of any type will be involved in the actual cooperation. The location of the cooperation is limited to the drive-up area, and for some vehicles, especially slow trucks, the urgency of the cooperation may be high, as otherwise a braking maneuver must be initiated. When cooperation takes place, it is an active form of cooperation for all agents involved, since the cooperation partner on the freeway adjusts his trajectory so that the profiteer on the driveway can choose a better trajectory. Therefore, it is clear that the cooperation is not mutual, and the grantor accepts costs with an altruistic interaction to enable the profiteer to save (much higher) costs. Since both the duration and the preparation time are short and limited to the time in the drive-up area, all costs incurred can be regarded as one-time expenses. In the long run, however, further costs could arise, e.g., if a very slow vehicle is allowed in front of one’s own vehicle. The initialization depends on the vehicle strategy and two possible variants are that the granter proactively offers cooperation or that the profiteer sends a cooperation request.

3.2.3. Truck Overtaking

This overtaking scenario is explicitly about a truck overtaking another truck on the freeway while a car approaches on the left lane from behind, as Figure 4 shows. The challenge in this scenario is related to the relative speeds; these are low between the two trucks (1–20 km/h) while being high between the trucks and car (40–180 km/h). This is due to the fact that for example in Germany the speed limit for trucks is 80 km/h, while for passenger cars it is often 120 km/h or there is no limit at all [1]. The number of agents involved in the cooperation is thus three: two trucks and one car. Such a maneuver can take place on any freeway section with at least two lanes and without overtaking prohibition. Further, there is no high urgency for cooperation, because overtaking would only result in a time saving for the overtaking vehicle. For the car, the interaction is active in any case, as the speed must be reduced to allow the overtaking truck to change lanes. The interaction of the front truck can be active if it slows down the speed to allow the overtaking truck to pass faster, or passive if it leaves its speed unchanged and sends only information about its speed. The duration of the overtaking maneuver is regulated by law and may not exceed 45 s in Germany [21]. Since the overtaking truck first has to approach with a low relative speed from behind, there is a lot of preparation time for the cooperation between both trucks, while there is only little time for coordination with the fast-approaching car. The overtaking truck is clearly the profiteer, as cooperation allows it to drive at its desired higher speed, so the maneuver should also be initiated by the overtaking truck. A proactive offer from the other cooperation partners would be a possibility, but in contrast to the drive-on scenario, there is no indication as to when it would make sense here since it is not bound to one location.

4. Implementation of Cooperation

When it comes to implementing cooperation, this means choosing a strategy for the agents that trigger cooperative interactions. With regard to road traffic, strategies are called maneuver planners in the following, and while maneuver planners are not new in automated driving, cooperative maneuver planners are only at the beginning stages of research. However, before going into maneuver planners and automated systems in more detail, we first introduce three characteristics of the implementation of cooperation. Afterwards, we present how cooperative movements have evolved in nature, e.g., with animal swarms, and how cooperation in road traffic is currently applied by humans.

4.1. Characteristics of the Cooperation Implementation

Besides maneuver planning, it is necessary to consider two further fundamental characteristics of cooperation which are partly mutually dependent: the planning level and the communication.

4.1.1. Planning Level

Cooperation can be planned on three levels: Centralized, decentralized with coordination, and decentralized without coordination [22]. “Centralized” means that there is an “all-knowing” master that coordinates the maneuvers of all vehicles, while “decentralized” means that the maneuvers are determined on the vehicles themselves. At the decentralized level with coordination, the vehicles have a communication channel to communicate directly with the vehicles in the neighborhood to plan the maneuver, in contrast to the decentralized level without communication, where the vehicles can be observed in the neighborhood, but there is no possibility of explicit information exchange. In the case of the centralized planning level, the global knowledge is available, theoretically allowing for the global optimum to be found. In the case of the decentralized planning with coordination, only the local knowledge (limited by the communication range) can be accessed, eventually leading only to a local optimum. This is similar at the decentralized planning level without coordination, but further, each vehicle can only plan on its own and must anticipate the behavior of other road users instead of receiving their planed behavior. This means that no negotiation, confirmation, or denial of cooperation is possible.

4.1.2. Communication

A communication channel is required for the centralized and decentralized level with coordination, and we distinguish between direct and indirect communication. Direct communication implies that signals from one vehicle are sent directly to another vehicle, as in the case of the WLAN standard IEEE 802.11p [23] that has been specially developed for vehicle-to-anything communication (V2X). It establishes an ad hoc network between the vehicles and enables a range of approximately 400 m. Indirect communication, on the other hand, uses infrastructure to enable communication over an unlimited range, and a widely-used standard for this is Long-Term Evolution (LTE, c-V2X) [24]. Messages addressed to neighboring vehicles are sent via infrastructure to a backend server, which only forwards the messages to vehicles located in a specific area (geo-messaging). In addition to the communication channel, a standardized communication protocol and message type must also be used, and the development for V2V or V2I is still in its early stages.

4.1.3. Maneuver Planning

The task of maneuver planning is to determine which actions to perform; in the context of automated driving this means that drivable trajectories have to be created. The above-mentioned characteristics of the cooperation, such as the location, the involved vehicles, and the costs have to be considered. For maneuver planning itself, different approaches based on trajectories, atomic maneuvers, state machines, artificial intelligence, and others are available and in development. The special feature of cooperative maneuver planners is that vehicles in the environment are not regarded as un-influenceable obstacles, but as possible partners with whom maneuver can be jointly developed.

4.2. Cooperation in Nature

In nature there are many examples of cooperation, such as the symbiosis of different plants in the jungle or the cleaner fish and sharks. The most interesting example of cooperation in nature, in this context, comes from swarms, herds, and flocks and deals with the collective movement of a large number of individual animals. The biggest motivation behind this cooperation is safety and efficiency: the zebras in a herd are much better protected from the lion than if they are standing isolated and birds flying in a V-formation consume less energy on the long way to wintering near the equator. Cooperation within a swarm follows three basic rules [25]:

• Separation: “avoid crowding neighbors (short-range repulsion)”
• Alignment: “steer towards the average heading of neighbors”
• Cohesion: “steer towards the average position of neighbors (long-range attraction)”

With these elegant rules and without any further communication channels or centralized control, swarms are able to implement complex interactions involving a large number of individuals. A prerequisite for this is an appropriate environment awareness to perceive the distances and speeds of the surrounding animals and to react accordingly. However, the interests of each individual are limited with respect to the global interests of the swarm. A transfer of the swarm to the road traffic is therefore only possible to a very limited extent. While the rules could be applied in a crowd of people walking, a person in a vehicle, especially at high speeds, cannot estimate the distances of other road users and react to them with a necessary small delay [2]. While a small bump among pedestrians is not a problem, such an error can have serious consequences for vehicles at high velocities. Another reason why the swarm behavior is not directly transferable is that in road traffic there is no homogeneous swarm, but rather a combination of many different individual vehicles with different goals and capabilities [25].

4.3. Human Cooperation in Road Traffic

Cooperation in road traffic only occurs if the situation is not clearly regulated by the StVO or an agent voluntarily renounces the rights given by the StVO. At a right-before-left intersection in western countries, there is normally no cooperation, as the situation is clearly determined by the rules of the StVO for example. A driver does not voluntarily stop to give the other vehicle the chance to drive, but because he/she is punished otherwise by law enforcement. Cooperation between humans does not result from a fixed set of rules or an assistance system, but exclusively from the intuition of the human driver. In a situation that is unclear to him/her, the driver might act cautiously and leave others the right of way in order not to provoke an accident, or he/she recognizes that there is a need for cooperation with another vehicle and helps. In the following, the mentioned characteristics of cooperation and its implementation are discussed in order to understand to what kind of cooperation a human driver is capable of.

• Planning level: Cooperation is planned in a decentralized manner, partly with but mainly without coordination. This means that each driver executes what he/she considers as the appropriate action, but usually does not discuss this with the other vehicles.
• Communication: Communication can take place explicitly via horns, light signals, or gestures, but also implicitly via the driver’s own behavior. By proactively changing to the left lane in front of an on-ramp, for example, it can be signaled that the resulting gap can be used for the drive on. However, it is not possible to transmit complex plans.
• Maneuver planning: The maneuver is planned in each vehicle itself, based on the experience and intuition of the driver.
• Involved agents: As the complexity increases with the number of participants, usually only two vehicles are involved in the cooperation between human drivers.
• Location: Since no infrastructure is required, the cooperation is largely location-independent.
• Urgency and costs: The urgency and costs can only be roughly estimated based on the driver’s experience.
• Interaction type: Due to limited communication, little information can be exchanged and the majority of actions are active and have a direct influence on the traffic situation.
• Duration: Due to limited communication, an agreement for a long time is not possible and cooperative maneuvers are limited to short moments.
• Mutuality: This is strongly dependent on the attitude of the drivers, whereby there are drivers who act altruistically, mutually, or selfishly.
• Preparation time: In most cases, cooperation occurs spontaneously. Far-in-advance planned cooperation can only occur among drivers who have the possibility of prior agreement. An example would be a joint holiday trip involving two vehicles, one of which drives ahead to show the way and the other follows.
• Initiation: Both the profiteer and the grantor can initiate the cooperation. When driving on the freeway, the cooperation can be offered by the grantor on the freeway through a proactive lane change, and the profiteer can request cooperation through the turn signal or force the cooperation by changing lanes on the freeway himself.

This list shows in several points that the limited communication between human drivers is one of the main restricting factors for their cooperation, as was also shown by Fekete et al. [3]. The correct assessment of the situation, especially with respect to the urgency of the cooperation, is very difficult and since there is no suitable opportunity for coordination and knowledge exchange, the situation often remains unclear. Therefore, cooperation might not take place although it would make sense, or, in contrast, cooperation might get started although it is not needed. In these cases, since it is usually an active form of cooperation, costs are incurred from which nobody benefits. A classic example of this is a lane change on the freeway, as shown in Figure 3, where it is not clear for vehicle B whether vehicle A will allow him/her to drive in front, and for B whether A wants to drive in front or behind him/her. If A assumes that B will not let him/her in front but B does, there is a useless deceleration. Such misunderstandings could be avoided by a synchronized level of knowledge among the drivers, but this requires improved communication [14]. Furthermore, due to the limited ability of humans to plan maneuvers, they would only be left with simple cooperative maneuvers. However, due to the human’s concept of fairness and emotions, human drivers are often capable of avoiding bad Nash equilibria, leading to lowered common costs due to their cooperation. Although the costs generated by human drivers are not optimal, they are unmatched by cooperative automated vehicles [3].

5. Cooperation for Automated Vehicles

As soon as the human hands over the driving task, it is up to the automated system to cooperate. For this purpose, we discuss the three points on the implementation of cooperation in the context of automated driving.

5.1. Planning Level

The three planning levels of centralized planning (CP), decentralized planning with coordination (DP), and decentralized planning without coordination (DPWC) come with different advantages and disadvantages and are therefore suitable for different scenarios [22,26]. With central planning, all existing knowledge of the vehicles is collected in a central backend and a global optimum is then formed. The prerequisite is therefore a backend with the appropriate computing power and stable communication to the vehicles. As with all centralized systems, this results in a single point of failure, which means that the system loses robustness and further has a high demand for communication. In order to be able to react immediately in safety-critical moments, fast planning is necessary, but communication to the backend and back again can result in a greater delay depending on the communication medium. Furthermore, vehicle control must be transferred from the vehicle itself to the centralized system, which raises concerns about responsibility and legislation.

Communication is also required for decentralized planning with coordination, but since no backend is required, an ad hoc network such as with ITS-G5 can be used. This makes the system more robust, but it is limited to the range of the communication medium. Since only a subset of the knowledge of the central planning level is available for maneuver planning, it can only be optimized locally. The direct communication between the vehicles and the maneuver-planning running on the vehicles themselves allows a faster reaction.

Both of these planning levels require communication, but decentralized planning without coordination is not dependent on it. This makes it the only planning level that can also be used effectively in mixed traffic when both automated and manually (human-driven) vehicles drive together on roads. Due to the lack of knowledge exchange, however, the behavior of other road users must be approximated, which introduces an additional inaccuracy into the system. The decentralized planning level is therefore robust and independent of communication delays or failures, but cannot achieve the same quality of maneuver planning as decentralized planning with coordination due to the small knowledge base and lack of coordination options. In addition, extra computing effort is required to estimate the behavior of other road users.

With regard to the evaluation metrics in

Table 2. Evaluation of the three planning levels for cooperation (circle: full = positive, empty = negative). CP: centralized planning; DP: decentralized planning with coordination; DPWC: decentralized planning without coordination.

	Quality	Communication Needs	Mixed Traffic	Robustness	Computation	Infrastructure	Delay
CP
DP
DPWC

it is clear that the different planning levels complement each other. Starting from decentralized planning with coordination, the system can be extended with decentralized planning without coordination for cooperation with manually controlled vehicles. Further, this could also serve as a fallback level in case of communication failure. Further, relevant information can be exchanged via a central backend, e.g., traffic densities in order to cooperate not on the maneuver level, but on the routing level to avoid a jam.

5.2. Communication

Connected vehicles form a trending topic mentioned by almost every vehicle manufacture in their vision and current research priorities [27,28,29,30]. This can include a connection to the vehicle owner via an app, a connection to an entertainment platform and other cloud services, or a connection to the vehicle service station to transmit maintenance data. However, regarding safety and efficiency, the real-time communication between connected vehicles is the most promising aspect.

In communication, a distinction can be made between direct communication (V2X) and indirect communication via an additional infrastructure (cellular–V2X). For indirect communication, the Long-Term Evolution (LTE) standard, which is widely used in smartphones for mobile data, has established itself. It requires a connection to the LTE radio masts, which cannot be guaranteed especially in rural and highly isolated areas such as tunnels or underground car parks. If the radio mast or the backend were to fail, an entire area would be affected and unable to communicate. Furthermore, the large number of involved nodes in the system increases the potential for manipulation or fault. With the currently available LTE REL-14 (4G), bandwidths of up to 1000 Mbit/s are possible with a latency of 5 ms [31] over an unlimited range. Moreover, with the newest REL-15 to 17 known as 5G, bandwidths of 10 Gbit/s and latency of 1 ms [32] are achievable. However, when changing from one radio mast to the next, which can often occur when driving over the motorway, short delays occur.

The ITS-G5 (IEEE 802.11p) standard [23,33], also known as WAVE, has been explicitly developed for use in communication between vehicles. An ad hoc network will be set up between the vehicles, whereby no infrastructure elements are required, thus there are fewer failure points and attack points, and no running costs arise. The system is therefore more robust and location-independent in comparison to LTE. Bandwidth and latency, however, are strongly dependent on the distance between the vehicles. The maximum range in an open field is approximately 800 m [34], but is reduced to 300 m [35] in autobahn scenarios and less in interconnected cities. The bandwidth and latency lie between 6 and 54 Mbit/s [36] and 1 and 5 ms, respectively [37]. With ITS-G5 only information can be exchanged directly between vehicles and no additional connection to the Internet can be established to receive routing information or traffic jam messages.

Connected vehicles provide interfaces that could be used by adversaries to do damage. Therefore, for connected vehicles and especially their communication, not only safety but also security are important aspects that have to be considered. Examples for such cyber-attacks are false data injection [38], reply attacks [39], and denial-of-service attacks [40]. It is therefore crucial, independent of the communication interface, that such attacks can be detected, blocked, and prevented.

As far as the communication medium is concerned, there are therefore already two strong established standards, with 5G a promising option for the future (

Table 3. Evaluation of the vehicle-to-anything communication (V2X) technologies (circle: full = positive, empty = negative). LTE: Long-Term Evolution.

	Availability	Reliability	Latency	Bandwidth	Range	Local Independence	Costs	Security
ITS-G5
LTE 4G
LTE 5G

). If a centralized planning level is preferred, c-V2X has to be chosen for the communication with the backend. For decentralized planning with coordination IEEE802.11p is also sufficient. Of course, a combination of both approaches is again possible, whereby routing information is received via c-V2X from a central planning level but maneuver planning is coordinated locally via V2X.

Paul Watzlawick wrote in his first axiom “One cannot not communicate. Activity or inactivity, words or silence all have message value”. This is the case even when no dedicated communication medium is involved [41]. Decentralized planning without coordination, where the vehicles “communicate” their intentions through their behavior, uses this. However, since no actual communication via radio is involved, this concept is not discussed here.

Not only the medium is relevant for communication, but also the message protocol and the message type. This choice determines which information is available in which form for maneuver planning and which bandwidth is required for the transmission. Two message formats have already been standardized by the ETSI as Day 1 Messages: CAM and DENM. CAMs contain ego information such as position, speed, vehicle type, etc. and are periodically transmitted at 1–10 Hz via single-hop broadcasting (point-to-multipoint) [42].

DENMs contain information about events that could affect safety or traffic efficiency and include an event type, location, start time and duration. These can be caused by the sending vehicle itself, e.g., an emergency-stop, or they can be detected by sensors, e.g., flash ice. The transmission is event triggered and forwarded over several hops to warn vehicles in a larger environment [43].

Currently, the ETSI is working on the standard for Collective Perception Messages (CPMs) [44], based on the Environmental Perception Message (EPM) [45] as part of the Day 2 Messages. The purpose of the CPM is to share objects detected by the vehicle sensors, thereby augmenting the environmental perception.

These three message types are purely informational and thus allow to improve the perception but do not yet allow a cooperative maneuver coordination. If cooperation on a central or decentralized level with coordination is to be planned, new message standards must be developed and established across manufacturers.

Oliver Sawade et al. developed the Collaborative Maneuver Messages (CMMs) to synchronize several vehicles with a distributed state machine [46]. The CMMs distinguish between three message types: Inform, Request, and Response. While the Inform CMM is used to share information, the other two CMM types allow for starting a negotiation between the vehicles on how to transition to the next state of the distributed state machine.

Bernd Lehmann et al. designed the Maneuver Coordination Message (MCM) [47] that is currently standardized by the ETSI [48] in order to exchange trajectories between vehicles. Different attributes such as “planned” and “desired” allow then to express the need for cooperation, start a negotiation, and finally execute the maneuver.

5.3. Maneuver Planning

The goal of maneuver planning is to generate trajectories, which are then executed by the vehicle. Factors such as safety (no collisions), efficiency, feasibility, traffic rules, and comfort have to be considered. Different classical approaches have already proven themselves feasible and also artificial intelligence methods are advancing in the field [49].

In the case of non-cooperative maneuver planners, vehicles in the environment are regarded as obstacles that cannot be influenced. In order to drive cooperatively, it is therefore necessary to rethink and consider how one’s own trajectory affects other road users. Thus, it can be considered how the maneuver can be carried out in a jointly optimized way. One possibility of including the behavior of the surrounding vehicles in the maneuver planning of the ego vehicle is to coordinate the maneuvers of the involved vehicles by V2X communication. In literature, there are already different approaches on how the concept of such a coordination could look like.

Hyldmar et al. implemented cooperative maneuver planning in a miniature fleet by having the vehicles in a lane merge send their desired position for the future via V2X [11]. The receiving vehicles then accelerated or decelerated to allow the sending vehicle to reach the desired position. An experiment with 16 of the miniature cars on a round course showed a more than 30% improvement in throughput through the cooperation.

Sawade et al. developed the Collaborative Maneuver Protocol (CMP) [46], which uses a distributed state machine to synchronize all vehicles of a maneuver in one common state. In each state, there are different roles and tasks for the vehicles and a transition to the next state can only occur if all vehicles agree. The CMP is thus a first step towards cooperative automated vehicles and shows a way in which vehicles can synchronize their plans with each other and collectively decide on the next actions. However, the state machine must be defined in detail for each particular maneuver, e.g., for a cooperative lane merge.

Bernd Lehmann et al. presented a generic concept which creates two trajectories for the vehicle, a plan and a desire, and sends this as an MCM [47]. The planned trajectory conforms to traffic rules and therefore does not lead to collisions with the planned trajectories of other vehicles. The desired trajectory, on the other hand, shows the optimal trajectory for the ego vehicle, which, however, is not traffic rule-compliant as it conflicts with the planned trajectory of another vehicle. This other vehicle then has the possibility to cooperate and adapt its own planned trajectory to enable the desired trajectory of the other vehicle.

If there is no possibility to coordinate one’s own behavior with that of other traffic participants, e.g., because not every vehicle is equipped with V2X communication, the behavior of the other road users must be anticipated. In order to implement behavioral planning under these prerequisites, different methodologies are applied in literature.

In the early stage of cooperative driving algorithms, Wei et al. [50,51] modeled behavior planning as a combination of different driving functions covering modules for distance-keeping to the leading vehicle, selection of the desired lane, and a corresponding merge planner. The lane selector acts as a supervisor that determines the best lane to drive in and switches between lane keeping and merging behavior. The lane-keeping module generates a set of constant accelerations, predicts the reaction of the surrounding vehicles, and evaluates the resulting scenarios by a cost function to select the best acceleration strategy. In subsequent work [52], Wei et al. extended their approach by splitting the acceleration in two acceleration segments with equal lengths, and enhanced the behavior prediction of other vehicles by an intention estimation. Since these approaches evaluate all possible strategies, the complexity of the strategies is limited due to the limitation of available computing time. For more complex strategies, search algorithms must be used which do not evaluate all possible behavior plans.

In the field of tree searches, each vehicle has a set of discrete actions from which it can choose. Each action is valid for a specified time step. A maneuver plan is then created from a sequence of actions over the planning horizon. The purpose of the tree search algorithm is to find the best possible combination of actions as a solution of the planning problem. The behavior of surrounding vehicles can also be modelled with a discrete set of actions under the assumption that they make rational decisions. Lenz et al. used a Monte Carlo Tree Search (MCTS) algorithm to solve this search problem [53]. Kurzer et al. [54] extended this approach by integrating macroscopic actions into the MCTS in order to plan over longer time horizons. Furthermore, they used a continuous action space instead of a discrete action set, which is useful in urban scenarios and tight spaces [55].

Another approach is the representation of cooperative behavior planning from a game theoretic perspective. While tree search methods find the best solution according to a given cost function, game theory considers all possible combinations of available strategies of all vehicles (referred to as players or agents). The solution in the form of an equilibrium is the strategy that yields the best reward for the considered player, regardless of which strategy the other players choose. However, the application in the field of cooperative behavior planning [56,57] has been limited to the assessment of two vehicles. Moreover, the scenarios in which the functionality of the approach is demonstrated include only simple scenarios.

Besides classical methods, artificial intelligence is also applied in cooperative maneuver planning. Research focuses on reinforcement learning techniques [58,59], in which the vehicles learn their own behavior based on positive or negative reward. The behavior modeling of the surrounding traffic is done implicitly within the neural network, which can be advantageous because no separate prediction is required, but also has disadvantages because the basis of the decision-making cannot be validated.

6. Problems and Challenges

We have shown that there have already been many developments and achievements in the development of C-ITS, but there are still some challenges to be overcome. For communication, there is disagreement between V2X and C-V2X [60,61,62,63], and the international standards are also inadequate for the various message types. It will be necessary to find agreement across manufacturers so that all vehicles can communicate with each other. Only when the equipment rate exceeds a certain threshold will the positive effects of the cooperation, especially with CP and DP, become noticeable.

However, with an increasing number of equipped vehicles and more messages for the coordination of more complex maneuvers, the channel load also increases, resulting in reduced communication range and message failures. Therefore, it will become more and more important to implement optimized message generation rules and congestion control in order to transmit the relevant information in any situation.

In cooperation that is not based on communication, it is important to avoid misunderstandings and to correctly assess the costs and behavior of other road users. With human drivers this is a particular challenge and requires special trust between automated vehicles and humans.

Approaches for cooperative maneuver planners have already been tested in simulation or on a small scale, but the lack of agreement on standards for the messages also makes it unclear what information can be exchanged for maneuver planning purposes.

Various projects like IMAGinE [64], 5G-MOBIX [65], and ICT4CART [66] have already recognized the urge for cooperation and attempted to develop solutions for the challenges mentioned here.

7. Conclusions

Despite traffic rules, there are situations that are not clearly or inefficiently regulated and where cooperation is required to ensure efficient and safe traffic. While human drivers are intuitively able to cooperate and communicate through light signals, gestures, or proactive actions, this is not yet the case with automated vehicles (AVs). Without humans as a cooperative module, the social dilemmas that arise in daily traffic become a major challenge for AVs. While the first AVs can even gain advantages in certain situations as free-riders in the cooperative environment due to the presence of human drivers, with an increasing number of selfish AVs the point will come where the entire road traffic loses efficiency due to lack of cooperation. Therefore, our mission must be to develop not only automated vehicles, but cooperative automated vehicles.

While animals can cooperate with simple and elegant rules, these cannot be applied to our complex road traffic. New methods have to be developed to tackle the challenges of CAV, and here we have provided several characteristics for clustering different cooperative scenarios in order to identify the challenges in a first step. Afterwards we discussed how cooperation could be implemented, and determined that currently no agreement exists on this should be done. While some challenges can be solved at a centralized planning level, other challenges are best tackled with a decentralized approach. In the field of vehicle-to-vehicle communication especially there is a huge disagreement on whether to use ITS-G5 (WLAN) or LTE (cellular) technologies. When discussing cooperative vehicles, however, it is key to realize that cooperation has to start in development. There must be unity among the vehicle manufacturers and there must be open standards that are collaboratively developed in order to foster cooperation in a large scale on our roads. We further showed that there are great synergies between the existing and promising approaches for CAVs that might lead to a combined and unified implementation in the future.

All in all, we highlighted the need for cooperative ITSs, and introduced the characteristics that need to be considered in cooperation and implementation in order to provide an overview and a starting point for further collaborative developments.