Impact of External Human–Machine Interface Communication Strategies of Automated Vehicles on Pedestrians’ Crossing Decisions and Behaviors in an Urban Environment

Abstract

The development of automated vehicles (AVs) and their integration into traffic are seen by many vehicle manufacturers and stakeholders such as cities or transportation companies as a revolution in mobility. In future urban traffic, it is more likely that AVs will operate not in separated traffic spaces but in so-called mixed traffic environments where different types of traffic participants interact. Therefore, AVs must be able to communicate with other traffic participants, e.g., pedestrians as vulnerable road users (VRUs), to solve ambiguous traffic situations. To achieve well-working communication and thereby safe interaction between AVs and other traffic participants, the latest research discusses external human–machine interfaces (eHMIs) as promising communication tools. Therefore, this study examines the potential positive and negative effects of AVs equipped with static (only displaying the current vehicle automation status (VAS)) and dynamic (communicating an AV’s perception and intention) eHMIs on the interaction with pedestrians by taking subjective and objective measurements into account. In a Virtual Reality (VR) simulator study, 62 participants were instructed to cross a street while interacting with non-automated (without eHMI) and automated vehicles (equipped with static eHMI or dynamic eHMI). The results reveal that a static eHMI had no effect on pedestrians’ crossing decisions and behaviors compared to a non-automated vehicle without any eHMI. However, participants benefit from the additional information of a dynamic eHMI by making earlier decisions to cross the street and higher certainties regarding their decisions when interacting with an AV with a dynamic eHMI compared to an AV with a static eHMI or a non-automated vehicle. Implications for a holistic evaluation of eHMIs as AV communication tools and their safe introduction into traffic are discussed based on the results.

1. Introduction

With the introduction of automated vehicles (AVs), today’s mobility will undergo fundamental changes. Nowadays, the human driver still executes the driving task. In future highly and fully AVs (SAE level 4 and 5), the human driver will be more or less decoupled from the vehicle control [1]. Along with this shift of control, the communication with other traffic participants (TPs), previously performed by the human driver, needs to be substituted by AVs [2,3]. Schieben et al. [4] describe today’s dyad of interaction between human drivers and other TPs (e.g., cyclists and pedestrians) shifting to a triad of interaction. Within this triad, the main actors are represented by the onboard user (i.e., the former driver), the AV, and other TPs in the driving environment of the AV. Regardless of the existence of a human driver, AVs need to communicate with other TPs in their surroundings in ambiguous traffic situations, enabling a cooperative interaction among all TPs [4].

Taking current traffic communication patterns into account, a combination of implicit communication via the driving behavior (e.g., deceleration, position in lane) and explicit communication via an external Human–Machine Interface (eHMI) seems to be a promising approach. The eHMI consists of an external interface connected to the vehicle, which can transmit explicit signals enabling interaction between AVs and other TPs [2,4,5,6].

To develop eHMI interaction strategies, it is crucial to take both possible positive (e.g., faster reaction times, higher subjective trust) and negative effects (e.g., confusion, miscommunication) for the communication between AVs and other TPs at an early stage into account.

Hence, the presented study aims to contribute to this topic by investigating pedestrians’ crossing behaviors when interacting with AVs with different eHMI communication designs. Therefore, we investigate the crossing initiation time and gaze behavior before and during a pedestrian’s crossing when interacting with an AV. Further, we collect subjective data such as participants’ road crossing certainty, the perceived eHMI usability, and the affective reaction to the eHMI.

1.1. Interaction between AVs and Other Traffic Participants

Initially, AVs will be introduced into mixed traffic environments. This means that AVs will coexist and interact with other TPs, e.g., conventional manually driven vehicles, cyclists, and pedestrians, in the same traffic environment. Therefore, AVs need to communicate with other TPs to interact safely [2,4,7]. Moreover, Stanciu et al. [8] state that adequate communication is highly relevant, especially for VRUs, e.g., pedestrians, to maintain the traffic flow. Generally, humans use different cues for communication and interaction in traffic, which can be categorized into the following categories [4,9]: implicit cues are signals that are not directly sent to a receiver or are sent out unattended by the sender. This means that the content of the message may not be evident at first glance. For example, the braking process is an essential implicit signal that illustrates the intention to prioritize a pedestrian [10]. However, explicit communication is directly sent out to a particular receiver. It can be transmitted via different modalities reflecting the vehicle’s status, e.g., hand gestures or the use of turn indicators [9]. Both means of communication, explicit and implicit, can give information about future intentions and maneuvers of the AV, and therefore they are of high relevance for the understanding and coordination of other TPs [4]. Studies by Dey and Terken [11] and Lee et al. [12] investigated the behavior of pedestrians when interacting with a conventional manually driven vehicle while crossing a street. The results show that the pedestrians used mainly implicit signals to decide if they could cross the street or not. Explicit communication between vehicle drivers and pedestrians was rarely used. The authors concluded that pedestrians tend to call upon additional explicit communication, especially when the vehicle does not behave as expected.

However, Lundgren et al. [13] state that the communication requirements will differ between pedestrians and AVs. Current research often focuses only on the technical avoidance of collisions for surrounding traffic, e.g., unpredictable stopping maneuvers of an AV, rather than the interaction between AVs and other TPs. Besides the high relevance of collision avoidance to maintain a safe traffic flow, a lack of communication between AVs and other TPs can lead to frustration due to poor understanding or even more severe accidents [4,14]. According to the current number of traffic-related deaths and accidents, more than one-third can be referred to as the interaction between pedestrians and vehicles [12,15]. This number demonstrates the risk potential for (critical) interaction between pedestrians and vehicles. Having this in mind, an understandable and expectation-conforming interaction between the AV and other TPs becomes even more relevant to traffic safety. Therefore, AVs’ future intentions and behaviors need to be transmitted to the surrounding traffic to enable other TPs to derive their actions subsequently [2].

1.2. eHMI as a Communication Tool

AVs could use an eHMI as a communication tool addressing the requirements for smooth and safe interactions with other TPs. According to Schieben et al. [4], the communication content of the signals presented by an eHMI can be divided into four information categories. Firstly, information about the vehicle’s driving status allows other TPs to understand the current automation status of the AV, which is needed to ensure proper mode awareness. Secondly, information about future maneuvers enables TPs to anticipate the AV’s upcoming actions and plan theirs accordingly. The third main information category is the AV’s perception of its environment. This information allows other TPs to verify if the AV has perceived them. The fourth category includes the cooperation capabilities of the AV. This category contains information if an AV can communicate cooperatively.

Focusing on the interaction between AVs and pedestrians, Bartels and Liers [16] emphasize that pedestrians’ behavioral patterns can vary significantly between and within individuals depending on the situation. Therefore, an eHMI should not only display the vehicle automation level but also transmit information about the AV’s perception and intention [5,17]. Although it seems that implicit signals have a significant impact on pedestrians’ awareness, explicit signals still need to be considered in the design of an eHMI to resolve ambiguity during low speed, e.g., [11,17,18]. Additionally, an AV should communicate through an eHMI, at least in situations where pedestrians are not satisfied with implicit signals from an AV.

One major problem in the interpretation of existing eHMI research results is the large variety of different eHMI designs tested. Thus far, various eHMI designs have been developed, e.g., projections on the street, light bands, or displays [19,20,21]. Conclusions made in different research studies are always highly affected by the used eHMI design. It is sometimes difficult to form generalized statements regarding the pure effect of the eHMI itself [22]. In the present study, we distinguish between static eHMIs, which only display the current vehicle automation status (VAS), and dynamic eHMIs, which are capable of communicating an AV’s perception and intention additionally.

1.3. Effects of eHMIs on the Interaction with Other Traffic Participants

The use of an eHMI as a communication tool presents a promising approach to support a safe interaction between AVs and pedestrians, e.g., [2,4,23,24,25]. Besides the positive effects, possible negative effects, i.e., miscommunication, need to be considered as well [26,27]. Concerning a safe interaction and efficient traffic flow, it is of interest how the use of an eHMI influences the interaction between AVs and other TPs. Especially for vulnerable road users (VRUs), such as pedestrians and cyclists, an understandable and safe interaction would be essential for traffic safety.

Generally, when interacting with AVs, many pedestrians see the need for an AV to be equipped with an eHMI as a form of communication [5,17,25]. Habibovic et al. [2] argue that an eHMI is particularly beneficial in situations where negotiation is required, e.g., who moves first. Moreover, the use of an eHMI could improve safety, acceptance, and traffic flow even before it is actually necessary [4]. Previous findings support this assumption by showing a higher sense of safety for pedestrians when interacting with an AV equipped with an eHMI [5,26]. Furthermore, De Clercq et al. [5] state that a safe street-crossing can influence the overall acceptance of the eHMI.

Focusing on pedestrians’ crossing behaviors in interaction with AVs, the latest research shows controversial findings. Whereas Clamann et al. [17] found no significant differences for the crossing decision when interacting with a dynamic eHMI as opposed to a static eHMI, De Clercq et al. [5] describe that the earlier the dynamic eHMI indicates a deceleration of the AV, the sooner participants feel safe to cross the street. The authors conclude that an eHMI can increase the efficiency of the crossing process.

Another important factor regarding safe street-crossing for pedestrians is an adequate strategy for gaze checks. Especially in traffic scenarios with multiple TPs, VRUs need to ensure that they have been perceived by all TPs and that a street-crossing is safe. Mitman, Ragland and Zegeer [28] showed that pedestrians at unmarked crossings tend to check the traffic more often in both directions than at ordinary pedestrian crossings. This finding is supported by Knoblauch, Nitzburg and Seifert [29], who show that pedestrians tend to make more gaze checks while crossing the street as opposed to waiting at the curb. With regard to the effects of an eHMI, Kitazaki and Daimon [26] conducted a study in which they investigated the use of an eHMI on pedestrians’ street crossing behaviors by taking the frequency of gaze checks into account. The results indicate that participants who encounter AVs with a dynamic eHMI tend to check the oncoming traffic less than those with a static eHMI. The additional information of the dynamic eHMI regarding the perception and intention of the AV could create a feeling of overtrust regarding the future behavior of the AV and the behavior of other road users. This overtrust might lead to the impression that it is always safe to cross the street when an AV triggers an eHMI, assuming that other road users behave accordingly to the AV. As a result, reduced gaze checks toward the oncoming traffic could lead to critical situations and demonstrate that the presence of an eHMI can also have negative effects on a pedestrian’s street-crossing behavior.

To sum up, eHMIs can be described as a possible solution showing benevolent effects for communication, hence the interaction between AVs and other TPs [4]. Nevertheless, negative effects, e.g., reduced gaze checks to the traffic environment, can occur and need to be considered to prevent accidents, especially in the interaction with VRUs [12]. A poor eHMI design can lead to overtrust and confusion if a message is misinterpreted or different designs are used to transfer the same message. These ambiguous situations can be potentially dangerous for a safe traffic flow [5]. Therefore, an area of growing research interest is the evaluation of possible negative effects of eHMI designs on VRUs [2,5,17,26,29], which should be performed in an early stage of an AV’s interaction design development.

1.4. Study Purpose and Research Questions

Derived from the current research on eHMIs, the present experimental study focuses on a pedestrian street-crossing scenario in an urban context. While participants took over the role of a pedestrian, participants had to decide when to cross a street in front of an approaching vehicle (from the left). Participants encountered interactions with automated and non-automated vehicles in a VR pedestrian simulator in a mixed traffic scenario. Since the used mixed traffic environment should induce a situation with realistic uncertainties for the participants, there was no driver–pedestrian interaction when the participant encountered a non-automated vehicle. Further, the current study used a constant approaching behavior (in terms of speed and braking) for all vehicles. This was performed to standardize the effect of the implicit cues sent by the vehicle movement in all conditions.

The first research question was if participants behave differently while interacting with an automated compared to a non-automated vehicle in the chosen pedestrian street-crossing scenario. Therefore, we compared participants’ behaviors in scenarios with a non-automated vehicle (no additional driver–pedestrian interaction, no eHMI) to scenarios where participants faced an automated vehicle with a static eHMI. The static eHMI only displayed the current VAS.

The second research question was if participants would benefit from additional information regarding the automated vehicle’s perception or future intention transferred via the eHMI regarding their crossing behavior. Therefore, we compared participants’ crossing behaviors while interacting with a non-automated vehicle, an AV with a static eHMI, and an AV with different dynamic eHMI design variants (see

Table 1. Experimental design of the presented study.

Interacting Vehicle Type (Within)
Non-Automated Vehicle	N = 62
AV with Static eHMI	N = 62
AV with Dynamic eHMI	Dynamic eHMI interaction design (between)
	Intention-based eHMI	Perception-based eHMI	Combined eHMI
	N = 20	N = 22	N = 20

). Furthermore, we wanted to investigate which of the different dynamic eHMI design variants is the most supportive one in regard to participants’ crossing behaviors.

In addition to possible positive effects of the eHMI on participants’ crossing behaviors, we also analyzed possible negative effects, e.g., reduced gaze behavior of dynamic eHMIs in a critical scenario with additional traffic from the opposite side (opposite lane from the right side).

2. Methods

2.1. Study Sample

A total of 62 participants (29 female) aged between 19 and 61 years old (mean (M) = 33.19 years, standard deviation (SD) = 11.67 years) took part in the present study. All participants were recruited through a company internal participant pool. Participants were residents of the Brunswig area in northern Germany. The familiarity of the situation, i.e., crossing a street in urban traffic in front of a vehicle, was rated on average as very familiar to the participants (M = 4.69, SD = 0.64). The participants felt very experienced in dealing with such situations (M = 4.56, SD = 0.76) each on a 5-point Likert scale (from 1 = “do not agree” to 5 = “completely agree”). The results of the ATI (German version [30], 6-point Likert scale [31]) showed a rather strong average technology affinity of the participants (M = 4.43, SD = 0.93).

In regard to the chosen virtual reality (VR) study environment (presented via a head-mounted display (HMD)), participants were asked about their perception of objects in the VR and possible restrictions due to the HMD in regard to their field of vision. A total of 63% of the test persons already had experiences with virtual reality (VR), and 37% had already taken part in a previous VR experiment with a different research question. The experiment was conceptualized and realized in accordance with the Declaration of Helsinki. Informed consent was obtained from all participants before the experiment. The participants were allowed to stop the experiment at any point without justification or consequence. The participants volunteered, but they were financially compensated.

2.2. Research Design

This experimental study was based on a 3 × 3 mixed design with “interacting vehicle type” as within-factor and “dynamic eHMI interaction design” as between-factor (Table 1).

The factor “interacting vehicle type” consisted of three levels: non-automated vehicle, AV with static eHMI, and AV with dynamic eHMI. Using these vehicle types, the first research question regarding the participants’ behaviors while interacting with an automated compared to a non-automated vehicle was addressed. Further, the second research question towards the benefits of additional information regarding the automated vehicles’ perception or future intention transferred via the eHMI was investigated.

The factor “dynamic eHMI interaction design” included three different interaction designs for the dynamic eHMI. While this factor was a between-factor, participants only experienced one of the three dynamic eHMI interaction designs. Using this factor, the research questions regarding the most supportive dynamic eHMI interaction strategy and possible negative effects of the eHMIs in a critical scenario were addressed. Participants were assigned to the different conditions of this factor in a randomized order. Each condition of the dynamic eHMI interaction design contained approximately the same number of participants, and a balanced gender ratio between the factor levels was achieved.

2.3. Experimental Factors

The first factor was the “interacting vehicle type”. Since the “interacting vehicle type” was a within-factor, all participants encountered the three different vehicle types used in this study. The “non-automated vehicle” was a regular vehicle without an eHMI or any other additional information (Figure 1a). In the factor level “AV with static eHMI”, the AV was equipped with a 360° LED band, which was mounted on the chassis of the vehicle (Figure 1b). The information richness transmitted by this eHMI design variant was limited to only displaying the VAS by illuminating the LED band in a static cyan color [32]. The factor level “AV with dynamic eHMI” contained an AV equipped with an LED band transmitting a higher level of information richness to the pedestrian, i.e., in addition to the VAS, the dynamic eHMI also communicated the AV’s intention (braking) or perception (pedestrian) via the LED band (Figure 1c).

The between-factor “dynamic eHMI interaction design” contained three different dynamic eHMI variants: intention-based, perception-based, and a combination of both interaction designs (see

Table 2. Description of the dynamic eHMI interaction designs used in the present study.

Dynamic eHMI Interaction Design	Description and Illustration
VAS + intention-based eHMI	The static illumination of the cyan LED band represents the VAS at the beginning of the scenario. At a closer distance to the pedestrian, the LED band started pulsing (frequency of 1.5 Hz) to indicate that the AV was slowing down, i.e., the light intensity of the LED band initially decreased until it was no longer illuminated for a short time and then began to increase again. This animation was repeated constantly.
VAS + perception-based eHMI	The static illumination of the cyan LED band represents the VAS at the beginning of the scenario. At a closer distance to the pedestrian, a signal lamp mounted at the upper center of the windshield turned on and indicated that the AV had perceived the highlighted pedestrian. The color of the signal lamp matched with the LED band (cyan).
VAS + perception-based + intention-based eHMI	This eHMI interaction design used both the pulsing LED band and the signal lamp for the interaction. The combination of both strategies allowed perception- and intention-based communication at the same time.

). The eHMI interaction designs were added on top of the pure information regarding the VAS [33], i.e., on top of the static eHMI as additional information in the AV’s communication with the pedestrian. The dynamic eHMI interaction design communicated the intention and perception of the AV by displaying different light animations on the LED band. Due to the enhanced information richness of the signal, pedestrians should be able to predict the future behavior of the AV and plan their actions accordingly. All vehicles showed an identical approaching behavior regarding their velocity and deceleration.

2.4. Study Measures

Pedestrians’ crossing decision was analyzed regarding the timing as an objective measure, i.e., how fast a participant decided to cross the street. Therefore, a button press on the handheld controller was captured at the point in time when participants intended to cross the street. Further, participants’ subjective certainty regarding the crossing decision was measured after each trial by using a 5-point scale (from “very uncertain” to “very certain”). For participants’ gaze behaviors (representing gaze checks), their head rotation towards the upcoming traffic from the right at the curb and while crossing the street was evaluated. For the analysis, gaze data were coded as 0 and 1 (0 = no gaze check; 1 = gaze check). The Self-Assessment Manikin questionnaire (SAM) by Bradley and Lang [34] was used to obtain the non-verbal assessment of valence, arousal, and dominance, which is associated with the individuals’ affective reactions to certain stimuli, i.e., here the interaction with different vehicles and different eHMI communication strategies. The dimension valence ranges from “unpleasant” to “pleasant”, arousal is from ”aroused” to “calm” and for dominance from “no subjectively perceived control over the situation” to “full control”. All dimensions were rated on a 9-point Likert scale. The usability of each eHMI was assessed on 7-point scales by the participants’ ratings regarding the needed effort to perceive the communicated information by the eHMI (from “exhausting” to “not exhausting”), how well participants felt informed (from “bad” to “good”), and whether the information helped to anticipate the vehicle’s intention (from “little” to “strong”). Additionally, participants’ perceived safety while crossing the street was assessed by asking whether the presented information increased their subjective perceived safety on a 5-point scale (from “do not agree” to “completely agree”).

2.5. Experimental Setting

For the present study, an urban traffic scenario in VR was designed with “Unreal Engine” (Software version 4.21) and conducted with the “VIVE PRO” VR glasses and the associated handheld controller. The corresponding system was operated with the “Steam VR” software (version: 1.4.5). Enhancing the effect of emersion, participants’ movement in the real world was transferred into the VR. Therefore, participants were able to walk inside the VR environment physically. To ensure the participants’ safety at all times, the study took place in a laboratory with sufficient space and without real-world obstacles. Additionally, the walking participants were supervised by the experimenter.

At the beginning of the urban traffic scenario, the participant stood at the street curb facing the corresponding street (see Figure 2, a). In all trials, the vehicle approached from the left (perspective of the participant). A four-way intersection was located 16.5 m to the participant’s right (see Figure 2, b). In each run, the participant stood in the same position at the beginning of a trial. One lane in each direction was presented in the VR simulation. At the end of the street and to the left of the pedestrian, a busy street crossed at a distance of about 90 m. On the opposite sidewalk, pedestrians moved to the left in the direction of the busy street. AVs and non-automated vehicles were included in the scenario, and therefore, it can be described as a mixed traffic environment, ensuring external validity.

The presented vehicle corresponded to the BMW i3. The participants were instructed that a vehicle would approach from the left and they decided whether they could cross the street in front of the vehicle. Participants were asked to behave naturally, like in their everyday life, and start walking as if they would cross the street in real traffic. Further, participants were instructed that they can rely on a vehicle’s behavior, i.e., a vehicle that starts to brake or activates a dynamic eHMI will definitely stop. Each subject encountered eleven different scenarios. In these scenarios, four vehicles did not give priority to the pedestrian (one non-automated vehicle; one non-automated vehicle with oncoming traffic from the right; one AV with static eHMI; one AV with static eHMI and oncoming traffic from the right). These runs served as distractors and were used to increase the external validity of the study.

Moreover, in seven scenarios, the vehicles gave priority to the pedestrians (two times a non-automated vehicle; two times an AV with static eHMI; two times an AV with dynamic eHMI; once an AV with dynamic eHMI and suddenly oncoming traffic from the right). On the one hand, the additional oncoming traffic from the right (other lane) was used to avoid an artificial environment where vehicles would only travel in one direction. On the other hand, these vehicles should create a critical situation if a pedestrian decided to cross the street without checking for upcoming traffic from the right. Participants were instructed to press a button on the handheld controller in all scenarios if they intended to cross the street. Additionally, participants should start walking physically to change their position in the simulated environment. In one last scenario (no. 11) with a yielding AV with a dynamic eHMI, a non-automated vehicle was triggered by the pedestrian’s button press and encountered the pedestrian as suddenly oncoming traffic from the right. Therefore, in this last and critical scenario, as soon as a participant pressed the button, a non-automated vehicle turned onto the street that the participant intended to cross (see Figure 3). If the participant was not gaze checking to the right while crossing the street, the situation evolved into a near-miss with the upcoming vehicle. This situation represents a controllability-like scenario where we wanted to investigate if participants followed the signal of the dynamic eHMI blindly and tend to care less about ensuring a safe crossing by checking the traffic from the other side. This trial was carried out as the last of the eleven scenarios to avoid a substantial shift of attention to possible oncoming traffic. The different ten scenarios before were randomized.

As pedestrians rely to a great extent on a vehicle’s driving behavior to make their decisions to cross the street in a vehicle–pedestrian interaction [11], the vehicles’ driving behaviors in terms of speed and braking, if applicable, were standardized. Therefore, all vehicles started with a velocity of 30 km/h and showed identical braking behavior, if applicable, over the trials. If an AV was giving way, the presentation of a dynamic eHMI started at a distance of 26.3 m from the vehicle to the participant. At a distance of 21.0 m, all vehicles (automated and non-automated) that gave way started to decelerate from 30 km/h to 15 km/h. At a distance of 12.3 m, the vehicle reduced its speed further from 15 km/h to 5 km/h. Finally, the vehicle reduced its speed to 1 km/h at a distance of 6.2 m. The approaching behavior of the vehicle and start times of the eHMI were selected after a prior investigation by experts to reach the absolute threshold at approximately the same time for both presentations. The absolute threshold is a type of perception threshold and marks the necessary intensity of a stimulus so that it can be perceived [35]. In the VR simulation, no acoustic signals were presented.

2.6. Procedure

Following the welcoming procedure, the participants filled out a consent form, a confidentiality agreement, a demographic questionnaire, and the ATI questionnaire. This was followed by an instruction phase in which the participants were introduced to the VR glasses and the VR environment. After the instruction phase, participants familiarized themselves with the VR glasses and the VR environment by performing test trials in an urban scenario without any other traffic participants. Within the test trials, participants could act freely and experience locomotion in VR. Subsequently, the respective eHMI interaction design was presented and explained in VR. The central task was explained to the participant and then practiced in one to two training sessions. The participants were instructed to press a button of the handheld controller as soon as they wanted to cross the street. They were not explicitly instructed to pay attention to possible oncoming traffic to avoid a distortion of the results regarding the frequency of gaze control. However, it was pointed out that they should follow the standard German traffic rules and behave naturally like in their daily lives. Before the trials were conducted, the participants were familiarized with the AVs with static eHMIs and the non-automated vehicles. It was made clear that the AVs with static and dynamic eHMIs were fully automated vehicles and, therefore, capable of performing all driving tasks without the input of a human driver. Subsequently, the eleven experimental runs were carried out. After each trial, the SAM questionnaire, the usability ratings, the crossing initiation time (time to press the button), and rating on the certainty to cross the road were collected. Before the participants were informed about the actual goals of the experiment, they completed the final suspicion check and received an expense allowance of EUR 10 per hour. The study took about 45 min in total.

3. Results

To address the research questions, we investigate pedestrians’ crossing behaviors while interacting with different vehicle types (non-automated vehicle, AV with a static eHMI, AV with a dynamic eHMI). The analysis of crossing behavior was split into participants’ certainties and times of decision to cross the street, the number of gaze checks, and a self-assessment regarding the affective reaction while interacting with the different vehicle types. In addition, we analyzed the usability of the three dynamic eHMI design variants. Here, we wanted to assess any differences between the dynamic eHMI design variants to answer the research question of the most supportive one.

3.1. Pedestrians’ Crossing Behaviors

3.1.1. Decision Times and Certainty

Mixed ANOVAs were used to explore the participants’ certainties and times of decision to cross the street depending on the interacting vehicle type and the eHMI design. For the execution of mixed ANOVAs, the prerequisites were met, and the significance level was determined p < 0.05. All participants with a mean certainty above or below 2.5 SD of the average of all participants were excluded from further analysis since we considered them as outliers. Therefore, one participant for the certainty to cross the street from the factor level ”perception-based eHMI” and one participant for the timing from the factor level ”combination eHMI” were excluded.

Regarding the time of the decision to cross the street (in seconds), a significant main effect was found for “interacting vehicle type” (Huynh–Feldt F(2, 114) = 21.02, p < 0.001, partial η² = 0.27). The point in time participants made their decision to cross the street was significantly earlier when participants interacted with an AV with a dynamic eHMI (M = 10.26, SD = 0.86) than with non-automated vehicles (M = 10.89, SD = 1.25), and with AVs with only a static eHMI (M = 10.85, SD = 1.35). All other post hoc comparisons showed no significant differences (p > 0.05).

Regarding the certainty of the crossing decision, a significant main effect for “interacting vehicle type” was found (Huynh–Feldt F(2, 114) = 24.89, p < 0.001, partial η² = 0.30). The certainty to cross the street was significantly higher when participants interacted with an AV with a dynamic eHMI (M = 4.62, SD = 0.51) than with non-automated vehicles (M = 4.07, SD = 1.03) and AVs with a static eHMI (M = 4.05, SD = 1.05). All other post hoc comparisons showed no significant differences (p > 0.05).

3.1.2. Gaze Behavior

For the gaze behavior, we investigated participants’ additional gaze checks towards the right side (possible oncoming traffic) at the curb and while crossing. Ensuring that no negative effects were induced by the VR hardware, we asked the participants regarding any subjectively perceived restrictions in the field of view experienced by the VR headset. Twenty-three participants (37.01%) reported a restricted view due to the VR headset. These participants were evenly distributed across the factor levels of the independent variable “dynamic eHMI interaction design”. Seven out of the 23 participants could not compensate for the restricted view with their head rotation. These participants were excluded from the gaze behavior results.

In

Table 3. Relative frequencies (in percent) of gazes at the curb and while crossing dependent on the interacting vehicle type.

	Non-Automated Vehicle		AV with Static eHMI		AV with Dynamic eHMI
	At the curb	While crossing the street	At the curb	While crossing the street	At the curb	While crossing the street
Yes	27.8%	1.9%	30%	4.6%	30.3%	4.3%
No	72.2%	98.2%	70%	95.4%	69.8%	95.7%

, the relative frequencies of gazes depending on the interacting vehicle type are reported. The results reveal that when participants executed a gaze check to identify the oncoming traffic, it is performed only at the curb rather than while crossing the street. Consequently, the highest relative frequencies of participants who did not check the oncoming traffic could be identified while crossing the street. Mixed ANOVAs were conducted to investigate the gaze behavior at the curb and while crossing the street depending on the interacting vehicle type and the eHMI design. No significant effects were found regarding the gaze behavior at the curb and while crossing.

A particular analysis was performed for the critical scenario (last one no. 11). It was represented by one of the yielding AVs with a dynamic eHMI and with an additional suddenly upcoming vehicle from the right side. Using the gaze data of the participants, we investigated if participants followed the signal of the dynamic eHMI blindly and crossed the street carelessly without checking the traffic from the other side. As described in the previous section, the overall gaze checks to identify the oncoming traffic were highest while still standing at the curb (27.7–30.3% of all trials) and very low during the actual crossing of the street (1.9–4.6% of all trials). Since the critical scenario evolves after participants decided to cross the street (and pressed the controller button to confirm their decision), only gaze data while crossing the street become relevant. However, there were only two participants (3.2%) that looked at the right side and checked for a safe crossing in the critical scenario. In all other cases (96.8%), a critical situation (near-miss) arises due to the absence of participants’ gaze checks regarding the upcoming traffic.

3.1.3. Affective Reactions

In the following, the descriptive (

Table 4. Descriptive results (M; SD) per dynamic eHMI design as between-factor and interacting vehicle type as within-factor for valence (V), arousal (A), and dominance (D) by the SAM questionnaire [34].

	Intention-Based eHMI			Perception-Based eHMI			Combined eHMI
	Non-Automated Vehicle	AV with Static eHMI	AV with Dynamic eHMI	Non-Automated Vehicle	AV with Static eHMI	AV with Dynamic eHMI	Non-Automated Vehicle	AV with Static eHMI	AV with Dynamic eHMI
	M (SD)	M (SD)	M (SD)	M (SD)	M (SD)	M (SD)	M (SD)	M (SD)	M (SD)
V	7.78 (1.90)	7.68 (1.70)	8.10 (1.72)	6.73 (1.52)	6.98 (1.74)	7.82 (1.37)	7.50 (1.53)	7.45 (1.57)	8.03 (1.11)
A	7.63 (1.60)	7.60 (1.73)	8.08 (1.06)	7.11 (1.20)	6.96 (1.49)	7.71 (1.33)	7.28 (1.63)	7.30 (1.67)	7.71 (1.32)
D	7.38 (1.77)	7.35 (1.49)	8.08 (1.01)	6.73 (1.52)	6.73 (1.79)	7.64 (1.26)	6.83 (1.99)	6.88 (1.89)	7.28 (1.69)

Note: M = Mean; SD = Standard deviation. Scale for valence (V), arousal (A) and dominance (D) ranges from 0 to 9.

) and inferential (

Table 5. Results of the ANOVA regarding eHMI design and interacting vehicle type for valence, arousal, and dominance of the SAM questionnaire [34].

	df1, df2	F	p	Partial η2
Valence
eHMI design	2, 57	4.81	0.01 *	0.14
Interacting vehicle type	2, 114	8.84	0.00 **	0.13
eHMI design * Interacting vehicle type	4, 114	0.76	0.55	0.03
Arousal
eHMI design	2, 59	0.96	0.39	0.03
Interacting vehicle type	2, 118	5.61	0.01 *	0.09
eHMI design * Interacting vehicle type	4, 118	1.69	0.95	0.01
Dominance
eHMI design	2, 57	0.75	0.48	0.03
Interacting vehicle type	10.30, 114	10.52	0.00 **	0.16
eHMI design * Interacting vehicle type	4, 114	0.47	0.76	0.02

Note: * p < 0.05, ** p < 0.001.

) results of the SAM questionnaire [34] are presented. For each eHMI design, participants with a mean above or below 2.5 SD of the average of all participants were treated as outliers and were excluded from further analysis (two participants). Therefore, for valence, one participant was excluded from the factor levels “intention-based” and “combination”. For arousal, no participant had to be excluded, and for dominance, one subject from the factor levels “perception-based” and “combination” was excluded.

Regarding the participants’ ratings on all three dimensions, valence, arousal, and dominance, the results showed overall high ratings on a scale from 0 to 9. Moreover, whereas the highest ratings were given when interacting with an AV equipped with an intention-based eHMI design, the lowest ratings were given when interacting with a non-automated vehicle or an AV without a dynamic eHMI (see Table 4).

A mixed ANOVA was conducted to explore the reported valence, arousal, and dominance dependent on the interacting vehicle type and the eHMI design (Table 5). The assumption of a normal distribution was violated, which can be neglected due to the same size of the testing groups. The premises of sphericity and variance homogeneity were met. Due to the lack of robustness of the Mauchly test, the Huynh–Feldt adjustment was used. Mixed ANOVAs were tested with a significance level of 0.05.

For valence, there was a significant main effect for “dynamic eHMI design” (F(2, 57) = 4.81, p = 0.01, partial η² = 0.14). The reported valence was significantly more positive for the intention-based eHMI (M = 8.46, SD = 0.66) than for the perception-based eHMI (M = 7.82, SD = 1.37). The combined dynamic eHMI (M = 8.21, SD = 0.80) did not differ significantly (p > 0.05). Furthermore, valence differed significantly for “interacting vehicle type” (Huynh–Feldt F (2, 114) = 8.84, p < 0.001, partial η² = 0.13). Participants reported a significantly more positive valence for the AVs with a dynamic eHMI (M = 8.14, SD = 1.03) compared to the AVs with only a static eHMI (M = 7.51, SD = 1.47), and compared with non-automated vehicles (M = 7.46, SD = 1.52). No significant differences for the interaction between eHMI design and vehicle type were found (p < 0.05).

Regarding the arousal, the main effect for the “interacting vehicle type” was significant (Huynh–Feldt F (2, 118) = 5.61, p = 0.01, partial η² = 0.09). Participants felt significantly calmer when they interacted with an AV with a dynamic eHMI (M = 7.83, SD = 1.52) than with non-automated vehicles (M = 7.33, SD = 1.47), and compared with AVs with only a static eHMI (M = 7.27, SD = 1.62). All other comparisons showed no significant differences (p < 0.05).

For dominance, there was a significant main effect for the “interacting vehicle type” (Huynh–Feldt (F(10.30, 114) = 10.52, p < 0.001, partial η² = 0.16). The perceived own dominance was significantly higher when participants interacted with an AV with a dynamic eHMI (M = 7.82, SD = 1.06) than with AVs with only a static eHMI (M = 7.11, SD = 1.67), and compared with non-automated vehicles (M = 7.08, SD = 1.64). Moreover, no significant differences for “eHMI design” and the interaction between eHMI design and interacting vehicle type were found (p < 0.05).

3.2. Usability of Dynamic eHMIs

In

Table 6. Participants’ mean ratings (M; SD) on the perception of the information, feeling of being informed, experienced help to anticipate the vehicle’s intention, and the perceived safety (from 1 to 7) regarding the dynamic eHMI interaction designs, i.e., intention-based eHMI, perception-based eHMI, and the combination of both.

Items	Intention-Based eHMI	Perception-Based eHMI	Combined eHMI
	M (SD)	M (SD)	M (SD)
Was it easy to perceive the information?	5.60 (1.57)	5.95 (0.95)	5.85 (1.09)
How well did you feel informed?	5.80 (1.24)	5.45 (1.30)	5.75 (0.97)
Did the information help to forecast the vehicle’s intentions?	6.10 (0.85)	5.68 (1.09)	5.95 (0.89)
Did the information increase your perceived safety?	6.03 (0.88)	5.64 (1.03)	5.58 (1.14)

Note: M = Mean; SD = Standard deviation.

, the descriptive results of the usability items for each dynamic eHMI design are described. For all three designs, the mean ratings were relatively high, indicating that the presented information via a dynamic eHMI was accessible and well-perceived and that the participants felt well-informed. Moreover, the participants experienced the given information as supporting, and the eHMIs increased the individuals’ perceived feelings of safety. Non-parametric Kruskal–Wallis tests were used for analyzing the data due to violated normal distributions. The results indicated no significant differences between the three dynamic eHMI designs (p > 0.05).

4. Discussion

Regarding the first research question, we analyzed if participants behave differently while interacting with an automated compared to a non-automated vehicle. Therefore, we compared participants’ behaviors in scenarios with a non-automated vehicle (no additional driver–pedestrian interaction, no eHMI) with a scenario facing an automated vehicle with a static eHMI only displaying the current VAS. Regarding participants’ decision times to cross the street, the results indicated no significant difference between non-automated vehicles and automated vehicles with a static eHMI. Further, no significant differences regarding the subjective certainties of the crossing decisions were found between non-automated vehicles and automated vehicles with a static eHMI. Moreover, the gaze behavior regarding gaze checks towards the right side (possible oncoming traffic) at the curb and while crossing did not differ significantly between non-automated vehicles and automated vehicles with a static eHMI.

The second research question was if participants’ crossing decisions would benefit from additional information regarding the automated vehicle’s perception or future intention transferred via the eHMI. Therefore, we compared participants’ crossing behaviors while interacting with a non-automated vehicle and an automated vehicle with a static eHMI, with a dynamic eHMI design. The results showed a significant difference between the groups regarding the decision times and certainties to cross the street. Participants made their decisions to cross the street earlier when interacting with an AV with a dynamic eHMI compared to an AV with a static eHMI or a non-automated vehicle. These results show that pedestrians seem to trust the signals of the dynamic eHMI. Despite the short duration of the experiment, significant differences in the timing to cross the street could be observed, which underlines that the eHMI designs are recognizable and easy to understand. The positive impression of the dynamic eHMI was also reflected in the pedestrians’ certainties to cross the street. The results reveal that the participants felt significantly more certain to cross a street when the AV provided a dynamic eHMI than a static eHMI or a non-automated vehicle without eHMI. This finding is in line with the results of De Clercq et al. [5], demonstrating higher decision reliability when dynamic eHMIs are available. These findings are supported by the participants’ ratings on their self-assessment. The participants described a more positive valence, felt significantly calmer, and reported a higher feeling of control when interacting with an AV equipped with a dynamic eHMI compared to an AV with a static eHMI or a non-automated vehicle without presenting any form of external communication. De Clercq et al. [5] underline a positive effect on the pedestrian’s overall acceptance if the AV provides a safe crossing for the pedestrian. Therefore, it can be assumed that the presented eHMI can help to provide a safe crossing of the street and thus supports a well-working interaction. However, no significant differences were found regarding the participants’ gaze behaviors when interacting with a non-automated vehicle, an AV with a static eHMI, or an AV with a dynamic eHMI.

The third research question focused only on the three dynamic eHMI design variants. Here, the question was which of the dynamic eHMI design variants was the most supportive one in regard to participants’ crossing behaviors. Although the dynamic eHMI leads in general to earlier crossing decisions (compared to a non-automated vehicle and an AV with a static eHMI), there was no significant difference between the three chosen dynamic eHMI variants in regard to subjective certainties and times of decisions to cross the street. Overall, the three dynamic eHMI designs achieved high usability ratings and, therefore, can be identified as an adequate way of communication between AVs and pedestrians. However, no significant difference in the usability of the three eHMI designs can be found. The only significant differences between the dynamic eHMI design variants can be found in the participants’ ratings on their self-assessment. Participants reported a higher positive valence for the intention-based eHMI compared to the perception-based eHMI.

Finally, we analyzed possible negative effects of the dynamic eHMIs regarding participants’ gaze behaviors in a critical scenario with additional upcoming traffic from the opposite side (opposite lane from the right side). When crossing a street, it is particularly necessary to check the oncoming traffic from both sides of the road to cross the street safely. Referring to the frequency of gaze checks before and after entering the street, no significant differences could be found for the interacting vehicle type. Derived from this, we could not find any negative effects of the static eHMI or the dynamic eHMI variants compared to a non-automated vehicle without an eHMI.

4.1. Limitations

Although the pedestrians’ certainties to cross the street in front of an AV with a dynamic eHMI were significantly higher, it should be noted that the certainty to cross the street was generally high. A high level of confidence in decision-making, regardless of the type of interacting vehicle, could have been created due to the presentation of low frequency urban traffic and the instruction that a vehicle would stop as soon as it decelerates. Further, the onset of the dynamic eHMI started before the vehicles started to decelerate (5.3 m earlier). The different onset times could have created an earlier cue, leading to faster crossing decision-making. Additionally, the results regarding the frequencies of gaze controls by pedestrians before and after entering the street (27.7–30.3% of all trials) could be influenced by the scenario. Knoblauch et al. [29] point out that on highly frequented streets, considerably more visual checks are necessary for the safe crossing of a street. This could explain the participants’ fewer visual checks conducted in this study by the used (low frequency) traffic.

The lack of acoustic stimuli from approaching vehicles could also have had a negative influence on the frequencies of the gaze controls. Even if the results did not show any differences between the vehicle type regarding the frequencies of gaze control, in the critical scenario with upcoming traffic from the right, we found near misses in 96.8% of all trials. In general, future research on the frequency of gaze control is recommended to simulate the entire street crossing process preferably. Stanciu et al. [8] mentioned that the comprehensibility of information conveyed by an eHMI is influenced not only by context and culture but also by experience. Thus, if the experiment is conducted over a more extended period, the effects of different eHMI variants regarding the crossing initiation and the perceived safety while crossing could be even stronger due to increasing familiarity. Moreover, this study was conducted in a VR environment. Although care was taken to ensure that the use of an HMD did not influence the participants, it may still have led to a reduction in immersion and, therefore, to reduced external validity. Future studies need to consider this limitation and should be performed in a setting as close to reality as possible. Furthermore, results could also be influenced by cultural differences, manifested in traffic in many different ways through deviant behavior. Therefore, it is generally essential to examine the influence of eHMIs on the frequency of gaze control, considering cultural aspects as well as the type of eHMI design.

4.2. Further Research

AVs will interact in mixed traffic with pedestrians but also with other road users. An eHMI should contribute to appropriate interaction with pedestrians. The present study shows that an AV with a dynamic eHMI can contribute to the pedestrians’ certainty to cross the street. In mixed traffic, the certainty to cross the street with a dynamic eHMI seems to be higher than in non-automated vehicles and AVs with a static eHMI. Additionally, the time to cross the street may be reduced with a dynamic eHMI.

Furthermore, the present study could not support the assumption that a dynamic eHMI negatively influences the frequency of gaze control of oncoming traffic by pedestrians. This discrepancy to the current state of research underlines the necessity to further investigate the influence of a dynamic eHMI on the frequency of gaze control in the future. All in all, the research field needs a more transparent structure through additional findings. Therefore, it is generally essential to examine the influence of eHMIs on the frequency of gaze control, considering cultural aspects and the type of eHMI design. Additionally, the combination of eHMIs and other communication tools for AVs, e.g., dynamic HMIs, which transmit implicit information, i.e., via vehicle behavior, should be investigated in future research. Moreover, different user groups and more complex traffic scenarios need to be addressed in future studies to investigate possible positive and negative effects of eHMIs and to enhance the external validity and generalization of study results.

However, the present study already allows for initial progress in the understanding of mixed traffic consisting of non-automated vehicles and AVs with static and dynamic eHMIs.

5. Conclusions

This paper aimed to investigate the possible behavioral effects of different eHMIs on the interaction between AVs and pedestrians by using the example of a street-crossing pedestrian in an urban traffic environment. Subjective and objective measurements were considered to identify pedestrians’ assessments on different vehicle types and different eHMI designs and their crossing behaviors in the interaction with these vehicles. The present study looked at non-automated vehicles and AVs with different eHMI interaction designs, which exclusively used light signals, i.e., a LED light band and/or signal lamp. The results reveal that additional information regarding the VAS transferred by a static eHMI had no effect on pedestrians’ crossing decisions and behaviors compared to a non-automated vehicle without any eHMI. Further, no differences between non-automated vehicles and automated vehicles with a static eHMI regarding the decision time to cross the street, subjective certainties of the crossing decision, or gaze behaviors were found.

However, the results show that participants benefit from the additional information of a dynamic eHMI. Participants made their decisions to cross the street earlier. They felt more certain regarding their decision when interacting with an AV with a dynamic eHMI compared to an AV with a static eHMI or a non-automated vehicle. Additionally, participants described a more positive valence, felt significantly calmer, and reported higher control feelings when interacting with an AV equipped with a dynamic eHMI compared to an AV with a static eHMI or a non-automated vehicle. No significant differences in the usability of the three dynamic eHMI designs were found. Finally, we could not find any negative effects of the static eHMI or the dynamic eHMI variants compared to a non-automated vehicle without an eHMI.