Speculative Exploration on Future Sustainable Human-Machine Interface Design in Automated Shuttle Buses

Abstract

Automated Shuttle buses (ASB) are considered an essential and sustainable direction for the future application of autonomous driving technology in public transportation. As the driver’s role gradually decreases and disappears, the Human–Machine Interface (HMI) for information exchange and communication between users and ASB takes a more prominent role and progressively becomes a hotspot in research. However, the unpredictability and complexity of autonomous driving, an exceptionally fast-growing technology, have hindered its future study. This work first reviewed related literature in three categories: internal, external, and station of the ASB. Secondly, the importance of systemic and speculative design is affirmed by exploring existing HMI designs for ASB. Thirdly, the concepts for ASB resulting from three parallel workshops were analyzed. Finally, online questionnaires and interviews completed the critical reflection and discussion. The results show that the introduction of tools and methods related to systemic and speculative design into the design process of the HMI for ASB may help designers to think critically about the future uncertainty of ASB and to deal with the complexity of the system.

1. Introduction

With the advancement of autonomous driving technology, beyond the effort to increase automation in private vehicles, automated shuttles are progressively becoming the focus of research and the development direction. Many companies in various countries have put their automated shuttles into commercial use, whose primary functions are delivering goods and transporting human passengers [1,2]. Examples are the robotic delivery car from Nuro (Mountain View, CA, USA) and Udelv (Burlingame, CA, USA) in the United States, “Olli” from Local Motors Company (Phoenix, AZ, USA), Autonom^® Shuttle Evo from Navya Company (Villeurbanne, France), EasyMile EZ10 from EasyMile Company (Toulouse, France), driverless taxi “Apolong” from Baidu (Beijing, China) in China, and “e-Palette” from Toyota in Japan (see Figure 1).

The merging of shared public buses and logistics vehicles is called Shared Autonomous Vehicles (SAV), of which robot taxis and shuttle buses are two typical representatives. An SAV is a vehicle that transfers human passengers from point A to point B and provides shared service among multiple targeting users’ communities [3]. The new SAE International Standards J3016 (2019) [4] states that most current automated shuttles are at SAE Level 4 and above. Automated vehicles of levels 1–3 mandatorily require a human driver, while vehicles of levels 4 and 5 support crewless vehicle operation [5]. They are committed to providing users a demand-oriented transportation mode, especially for complementary services in specific areas and last-mile operations [6].

Automated Shuttle Buses (ASB) are the sub-category of SAV on which we focus our attention. The interaction between ASB and diversified user groups will undergo subversive changes with the support of technologies in the future. At the same time, communication channels and interfaces established between users and ASB will play a more prominent role compared with manual ones. Until now, the unpredictability of these interactions has hindered the establishment of public acceptance and trust in this technology [7,8]. As a technology with an unusually rapid development trend, future research on autonomous driving technology is essential. Automated shuttles are considered an essential preliminary step in realizing the adoption of autonomous driving technology in future public transportation.

This article aims to explore the future speculative scenario, which constructs numerous contact points that users and service providers may generate to understand their different needs more comprehensively. Consequently, the main output of this study is future scenarios for the early design stage and concepts designed with the tools and approaches of speculative design and systemic design. Such output can help researchers speculatively explore and improve Human–Machine Interface (HMI) design from an automated shuttles’ user experience (UX) perspective. The paper’s organization is as follows:

• Section 2 describes the research phases and methodology.
• Section 3 discusses findings about existing solutions, state-of-the-art research for the ASB interior, exterior, and station, as well as the HMI design exploration for ASB to explain the workshop’s usefulness.
• Section 4 describes the workshops’ processes and illustrates the “near future” and “far future” concepts for three generated scenarios.
• Section 5 discusses the workshops through an online questionnaire and interviews.
• Section 6 concludes the article, including limitations and recommendations for future work.

2. Materials and Methods

The city transport system is considered a socio-technical system [9,10,11], which involves a complex interaction between humans, machines, and the environmental aspects of the work system [12]. Scholars have employed a socio-technical transition perspective to study various facets of Automated Shuttle Buses [13]. Therefore, we acknowledge the importance of adopting a speculative and systemic approach when exploring the future of sustainable public transport systems. The design exploration process should incorporate systemic, multilevel, multi-actor elements.

The research follows a research-through-design (RtD) [14] process (see Figure 2). First, a literature review and an exploration of design tools were performed. The literature review was conducted on publications indexed in Google Scholar, Web of Science, and Scopus to draw on specific findings in the HMI of ASB for the interior, exterior, and station categories. In parallel, the most suitable design processes and tools for the HMI system were researched. The related work phase led to a conceptual reference for applying speculative exploration and the systemic design approach to the future scenario of ASB.

Then, three parallel creative workshops were conducted at the School of Mechanical Engineering of Shandong University in China to test the theoretical design process. The workshops were held as part of the Product Design Concept and Practice course, which aimed at generating HMI concepts speculating on the ASB, focusing on issues of sustainability and inclusiveness. In addition, the workshops were conceived as experimental processes incorporating speculative and systemic dimensions, involving online questionnaires and interviews to iterate and reflect the whole process.

3. Related Work Research

The study of ASB, as part of the future transportation system, involves many specific academic branches. Some contemporary scholars are devoted to investigating its impact on public social influence [15], urban planning [16], road management [17], policy system formulation and regulation [18,19], and so on. In addition, another group of researchers is interested in improving the functionality of electronic information technology [20] through communication between users and it. This work aims to innovate user research by designing sustainable interactions. This necessitates the development of new HMIs that meet the user’s needs while not negatively impacting the environment, the road, the driver, or the vehicle manufacturer [21].

3.1. Literature Review

3.1.1. Interior of the Automated Shuttle Bus (ASB)

The purpose of improving the HMI system in the interior scene of the ASB is to make it more attractive to passengers through more humanized service and demand-oriented interaction, allowing users to accept and trust the technology. Most recent research on this direction in the design field is devoted to the aspects and perspectives described hereafter:

• Transparency of communication: Includes the automated vehicle’s clear interaction of user input and system output.
• Satisfaction of user needs: Classifies groups according to the motivation of different users.
• Interior development: Enhances the atmosphere for different scenes of the interior environment.
• Remote Operation: Real-time vehicle monitoring and management conducted remotely.
• Control transfer: Grants passengers temporary management authority in an emergency.

In addition, there is much research on the technical standards and specifications of various communication means in the vehicle environment in the mechanical engineering and electronic information field. However, we are more inclined to explore the UX and interaction modes.

Transparency of Communication

A lack of human factors in ASB can lead to greater reliance for users on the communication means of HMI in the interior bus. More precisely, passengers can only obtain the communication information they need (e.g., route, timetable, operating status, delay situation) from the interactive system. As a result, interior HMI determines UX by delivering the information the system needs to transmit.

Due to the continuous enhancement of the functions and forms of onboard electronic equipment, the development of multimodal and multi-channel integration of the interaction interface in-vehicle has become a trend. According to its function, HMI in autonomous vehicles (AV) can be sorted according to the input and output types. The former includes the vehicle’s channels to receive the driver’s input information, such as buttons, steering wheels, pedals, and touch screens. The latter refers to the vehicle’s channels to provide system status information to the user by transmitting multi-channel signals mainly based on visual and auditory stimuli (i.e., displays, lights, and voice) [22]. According to Lundquist [23], there is still a lack of trust in ASB or private cars with the abundant HMI types above. The reason is that a comfortable ride experience should be built on safety and efficiency, with users having some control over their surrounding environment or information. Therefore, it is crucial to consider the user’s trust, control, and sense of security when designing the interior HMI. The user-centered design approach is now mainstream in communicating and delivering in-vehicle information, which means that intuitive user input interactions and valuable system outputs increase the overall acceptance of ASB [24]. In summary, we must design a clear human-centered HMI through multimodal information transfer in the future.

Satisfaction of Diverse Needs

ASB needs to meet the needs of various user groups as much as possible as a form of public transportation, which puts forward high requirements for inclusiveness. That is why many countries (e.g., France, Switzerland, USA) have considered the role of mobility in social inclusion by adopting legal provisions to ensure that people with disabilities use modified vehicles (e.g., a vehicle adapted to be driven with a joystick) [25]. It also shows that related fields have mainly focused on groups with travel barriers and made corresponding HMI adjustments to promote user understanding in recent years.

The use of universal design principles was advocated by Ferati et al. (2018) [26] and Sundararajan et al. (2019) [27] for the design of AVs’ in-vehicle interaction. Moreover, the importance of considering the needs of people living with a disability (PLWD) early in the design process was also highlighted by Tabatannon and D’ Souza (2019) [25]. The benefits of autonomous driving technology are more evident at level 4 and above because, at lower levels of automation, the driving capability is still needed, which is not people living with dementia (PLWD)-friendly. Compared with private vehicles, PLWDs need to rely on others to meet their mobility needs to a certain extent when using public transportation [28]. Therefore, avoiding the obstacles they encounter should be appreciated when accessing public transport in the current motorized system.

In contrast, public transport also has its advantages. With automatic driving support, drivers or service personnel can improve service quality due to time availability. Recent research has also concentrated on developing digital technologies with multiple functions for detecting the user’s health status in the shuttle, such as blood pressure, heart rate, and other physical data. This information is critical for PLWDs to receive more timely medical assistance if they experience unexpected physical conditions through these technologies. However, security and privacy concerns prevent the widespread implementation of these technologies. Data privacy is of the utmost importance in such a multidisciplinary field as healthcare, sensor technology, ASB, road traffic, and laws in different countries regulate data privacy. To address these privacy and security problems, scholars who contributed to this field, such as Kang, James Jin, et al. [29], have advocated that personal data must be encoded, authenticated, and only accessible to authorized personnel. The majority of other studies targeted at developing technologies capable of detecting biometrics are still in the conceptualization phase. Therefore, the protection of personal information is a neglected aspect. The author believes that a privacy policy can be communicated to the user before data collection to notify them of their rights and describe the purposes of data collection and processing, which would be more beneficial in preserving user rights. Furthermore, people with disabilities are diverse, and usable design solutions must be tailored to each group’s needs, circumstances, and preferences. Future research in various groups should include more participatory action design and engineering studies as well as higher-quality, prospective experimental studies to evaluate outcomes of accessible AV technology.

Interior Development

Interior development relates to the interior experience of diverse user groups engaging in different activities in ASB. A wide range of autonomous driving scenarios will produce different information expectations, and the needs of passengers should be explored in a specific public transportation environment. Thus, the discoveries of private cars cannot be simply transferable to ASB.

According to Lundquist (2018) [23], tourists want to probe for localized information about the surrounding environment to obtain a brilliant travel experience. Additionally, commuters focus on efficiency and reliability, such as traffic conditions, delays, or changes. Furthermore, with the disappearance of driving tasks, users’ demand for entertainment and motivation for specific activities such as office and rest gradually rose, requiring designing in-car mixed reality to enhance passenger experiences [30]. Finally, as indicated in the previous section, ASB’s inclusive travel requirements should meet the needs of a more generic user base, including the elderly, overweight people, people with dementia, people carrying large items, and so on. It requires a recalibration of ASB interior concepts (e.g., position, type, reach of seat, controls, and displays). Dynamic design aspects and elements such as lighting, materials, smell, sound, and seats should be adjusted compared to conventional buses.

Remote Operation

Remote operation can potentially roll out highly automated vehicles (SAE Level ≥ 4) more safely and quickly. Laws and regulations for remote supervision of AV without an onboard driver have been adopted in the UK, Sweden, Germany, Japan, and a few US states [31]. The remote operator could observe automated driving operations and intervene when the automation capabilities of the AV are exceeded. This approach is becoming increasingly feasible with the widespread introduction of high-bandwidth, low-latency communication technologies (such as 5G) with the option to prioritize certain data types [32]. Many laboratories have tested automated shuttles with onboard operators to expand public transport services. However, since remote operators, whose tasks are like those performed by onboard operators, are almost non-existent in urban road traffic, testing with onboard operators is often used instead [33].

HMIs for this purpose are created under the concept of human-centered design solutions. Research in this direction aims to improve the usability, UX, trust, and acceptance of SAVs through HMIs that address human factors issues such as distraction and fatigue. A common approach in these projects is to collect a variety of scenarios to improve the workplace for remote operations, including interviews and observations of control center staff, video analyses from naturalistic road events, and interviews with safety operators of AVs [34]. Participants in such studies are not limited to human users but also include roles played by external hardware (e.g., crosswalks, traffic signals, or smart roadside infrastructure) or other subjects (e.g., passengers, emergency services, control center staff, and other road users) [35]. Remote operation may be a long-term alternative to human operators on-site without compromising on the sometimes-irreplaceable skills of humans. It is, therefore, more than a preliminary technology to bridge the gap to a certain level of automated driving. It also has the potential to develop autonomous driving technology for public transportation applications.

Transformation of Control

The control transition of an automated driving vehicle is designed to perform the handover from the system to the driver and back, including information exchange through HMI solutions, strategies, and modalities, considering the criticality and complexity of the situation [36].

The role of the HMI is crucial to managing unexpected events and ensuring a safe and smooth control transition. The transition between the automated driving system and the human driver has extensive attention from scholars when discussing private vehicles for L3 automation. In most commercial cases of ASB, this function relies on the remote operating function mentioned before. It needs to be discussed more in the literature since the level of automation is generally at level 4 and above. Even though passengers are not as familiar with automated public transportation as with private cars, the transition of control rights in situations such as switching between different customization of personalized services is still essential to the user’s experience of this technology [37]. Therefore, the development of HMI hosting this functionality aspect is also a relevant future direction.

3.1.2. Exterior of the Automated Shuttle Bus (ASB)

Since no human is behind the wheel, the communication of ASB with their nonautomated environment (e.g., pedestrians, cyclists, motorized road users) should be a primary research focus. The most critical perspectives related to external communication with ASB are as follows:

• Communication and interaction: how other road users communicate and interact effectively with ASB.
• Visibility/recognizability: the design’s recognizability and comprehensibility to other road users.
• Anthropomorphism: aspects of human-like communication influence the users’ experience, focusing on trust, acceptance, and safety.

Communication and Interaction

Before the advent of AVs, other road users primarily accessed information through gestures and eye contact with human drivers. Several studies suggest that external human–machine interfaces (eHMI) for ASB can achieve those communication functionalities, which influence the confidence, trust, and perceived safety of crossing pedestrians. Scholars have focused on whether eHMI can support Vulnerable Road Users (VRUs) to make faster road-crossing decisions and prevent their misperceptions. Vulnerable Road Users are non-motorized road users, such as pedestrians, cyclists, motorcyclists, and persons with disabilities or reduced mobility and orientation [38]. The potential for unambiguous driving and communication behaviors when automated shuttles and other road users meet in traffic is a primary concern for scholars, specifically in the following application scenarios:

(1)

Crossing (C): The shuttle approaches a zebra crossing and informs that it has identified the pedestrians about to pass, leaving them a priority.

(2)

Unregulated junction (UJ): The shuttle and another road user (car, motorbike, bicycle) are approaching an unregulated T-junction from different directions. Other road users are informed of the shuttle’s intention via eHMI.

(3)

Regulated junction (RJ): Clear communication is necessary for VRUs, letting them know whether the shuttle will change its trajectory at a regulated intersection due to circumstances like a pedestrian crossing the road.

(4)

Boarding (B): The shuttle notifies passengers of the time to get on board and alerts passengers in the vicinity that is about to leave before closing the doors.

(5)

Passing (P): The shuttle passes an oncoming pedestrian on the side of a road without pavement and communicates that it has recognized them and is keeping enough distance.

In addition, eHMI in ASB can be divided into “explicit” and “implicit” communication [39]. Explicit communication refers to the information exchange between the vehicle and the user through external information, which manifests explicitly through multimodal stimuli (e.g., lights, gestures, voice, etc.). Implicit communication refers to judging the vehicle’s state through inherent hidden details, such as the vehicle’s trajectory.

Visibility/Recognizability

Most case studies have compared the effects of different forms (such as morphing arrows, icons, LED bars, etc.), presentation modality (information screen, animation, audio), and locations of eHMI (on the bus, at the station, on the wearable) on user behavior. The aim was to compare the impact of different eHMIs on VRUs and evaluate to which extent eHMIs attract their attention. The results differ due to the experimental and data analysis methods.

Given the challenging nature of assessing eHMIs for ASB in real traffic, most research has been conducted simulating driving scenarios, adopting methods such as Wizard of Oz, or combining virtual reality and natural venues [40]. The test procedure is mainly proposed to be carried out in a traffic environment without right-of-way rules since eHMI is the only standard. When there is a conflict between the bus’s driving direction and a vulnerable road user, the eHMI substantially impacts other users. The majority of simulated experiments were set up with three conflict scenarios proposed by Kaß, Christina, et al. (2020) [41], which were the following:

(1)

The vulnerable user approaches the automated vehicle frontally;

(2)

The vulnerable user approaches orthogonally from the side;

(3)

The vulnerable user merges in front of the automated vehicle with a lateral approach direction.

In summary, research studies by scholars have shown that visual communication, including texts and symbols, is the most suitable due to its ability to convey complex messages. Shorter and non-verbal cues are more appropriate, prioritizing visibility over the richness of communicated content [42].

Anthropomorphism

Anthropomorphic communication is prevalent in robotics [43,44], and some researchers have already proposed human-like eHMIs (e.g., eyes on the car) [45] to enhance user acceptance and safety. Automobile faces have been perceived the same way as human faces, with a higher face ratio (ratio of face width to face height) associated with a greater sense of dominance. The behavioral and physiological evaluation revealed that participants trusted the vehicle would perform more competently as it acquired anthropomorphic features, such as name, gender, and voice [46]. Meanwhile, the tolerance for malfunction was higher with anthropomorphic framing. Another significant advantage of anthropomorphism is that it could be instrumental in breaking down communication barriers due to cultural differences.

Researchers have gradually recognized the potential of anthropomorphism for the field of eHMI design. To be more precise, some researchers have explored forms of anthropomorphism (e.g., artificial eyes and smiles on AVs). The most debated topic is whether communication between AVs and VRUs should be egocentric or allocentric. Saleh et al. (2017) [47] and Dey et al. (2018) [48] suggested that the information provided by the vehicle should inform VRUs of the AV’s intentions (allocentric communication) rather than guide the user’s behavior outside the vehicle (egocentric communication).

In conclusion, humans need intuitive and effective signals from eHMI, so ASB must act as intelligent and thoughtful entities during the interaction [43]. If the shuttles present human-like characteristics or behaviors, they will be perceived as safer, smarter, and more trustworthy [49].

3.1.3. Station of the Automated Shuttle Bus (ASB)

The station is the third area of exploration in this study. ASBs are aimed at helping users achieve frequent demand-tailored service [50]. Therefore, there is a trend of transforming bus stations into transit-oriented development [51], which means there will be a virtual station whenever the user arrives at their destination and needs to get off the bus.

Virtual stops (vStop) are common speculation among scholars. When the physical station disappears, its functions will be displayed through the in-vehicle visual screen and the user’s mobile device, including communicating information about upcoming stops and stop requests to the passengers [52]. Improving ease of access, waiting time for all passengers, and loading and unloading time for those with special needs is essential [53]. Hub and Oehl [54] propose this viewpoint, considering that shared automated mobility-on-demand (SAMOD) is a promising mobility solution for the future. In 2022, they developed an Augmented reality (AR) vStop HMI for mobile devices. Users book trips on-demand via smartphones, and the service algorithm sets a vStop where users can board the ASB [54]. Their research proves that mobile AR is a very effective HMI model, which also provides another way of changing toward more sustainable modal choices for individuals.

Physical stations cannot be eliminated nowadays, and they are indispensable when providing tourists with micro-transportation commuting on looped closed roads. Such examples are deployed in Germany (Ioki project), Norway (Kolumbus project), and the USA (GoMentum station project) [55]. HMI design for these stations mainly focuses on the visual information typically displayed on a station monitor currently in use [52].

3.2. HMI Design Exploration for ASB

Designing HMI for automated driving remains an emerging area which user interface design principles may guide. User-Centered Design (UCD) is the most extensive and applied method in HCI methodologies for developing and establishing HMI design for AV perspectives [56]. According to ISO 9241-210 [57], UCD is defined as a development process that aims to make systems usable by focusing on users’ fundamental needs. The individual phases are executed in an iterative pattern to incrementally improve the design and develop a solution that meets the user requirements [58]. Many scholars and practitioners also used participatory design [59] to investigate the needs and challenges of a specific category of users (e.g., the elderly, visually impaired users, pedestrians, etc.) and co-creation.

Regardless of which design approach is used for designing the HMI for ASB, the design process can be summarized in a three-step Analysis, Synthesis, and Evaluation model based on the relevant interaction design principles.

Analysis refers to defining a design problem by gathering design requirements. The frequently discussed methods include carefully considering contextual factors, developing empathy for potential users, forming highly diverse research teams, etc. As a field that is still evolving, ASB has many uncertainties and complexities, so it is crucial to balance the interests of different stakeholders and summarize the requirements of various categories of users. However, it seems challenging to meet these requirements only from the perspective of interaction design. As an approach to integrating different thinking, systemic design [60] can help designers shift their attention from a single element to the big picture by constructing a holistic view of the system, focusing on the relevant actors inside and outside it. Systemic design integration in the HMI design analysis phase is a direction worth exploring.

Synthesis is the phase where designers generate ideas that aggregate all the user data to create multiple solutions. After these ideas are generated, reflecting on them needs to be improved in existing research. Related studies are eager to move to the evaluation stage [61,62], with the risk of limiting critical thinking and innovation.

In transitioning from traditional public transportation to ASB, it is necessary to apply more thoughtful and innovative thinking methods to facilitate such a radical transformation. Therefore speculative design, a future-oriented and critical design practice conceptualized by Dunne and Raby in 2013 [63] seems promising in helping designers conceive possible scenarios for emerging modes of transportation [64]. Nevertheless, the application of this approach in ASB is almost nonexistent.

Finally, Evaluation refers to assessing potential design solutions using the requirements from the analysis phase and generating a final design concept. The evaluation of HMI design is the focus of research attention in AVs, and the methods commonly used are divided into structured questionnaires [65], Wizard of Oz experiments [66], and VR experiments [67].

The three phases mentioned above are, in essence, connected. Moreover, systemic design and speculative design methods and tools have been widely applied in various design research fields, making remarkable contributions to sustainable development, urban planning, healthcare, public administration, etc. The background information and literature review in this section support the conclusion that systemic design aids in integrating systems thinking with design to attain environmental, social, and economic sustainability. While speculative design emphasizes the influence of technology on innovation, it can also identify and explore significant challenges that may arise in the future. In contrast, this work’s distinctiveness is investigating the research gap where the ASB domain has not included these two design approaches.

4. Creative Workshops

We explored the future HMI of ASB through systemic and speculative design. Indeed, these two approaches could help designers critically consider uncertainty and handle systemic complexity in the HMI design process [68,69]. We, therefore, extracted the “comprehensive understanding of the system” from systemic design and the “construction of speculative future scenarios” from speculative design. Then, through three design workshops, we combined these two components (including process and tools) in a two-step interaction design process.

The design workshops were part of the “Product Design Concept and Practice” course for undergraduate students at Shandong University in China. In the academic year 2022–2023, the theme of the class was to explore ASB and propose future scenarios. The three creative workshops were conducted with (n = 28) student participants (19 female and 9 male) aged between 18 and 22. The think-aloud method was used to encourage students to express their thoughts. The course was concluded with a semi-structured interview that examined the participants’ general experiences in public transportation and their expectations for an automated shuttle bus.

4.1. Workshops Design

The workshops were conducted for one month, from 21 September to 30 October 2022, and were structured in four steps: systemic exploration—critical future envision—iterative loop—critical reflection (see Figure 3).

• Before the workshops: We identified the design themes and divided the students into six teams, four groups of 5 people and two groups of 4 people. A total of 28 students took part. They were tasked with conducting background research to comprehend the main issues surrounding shared autonomous vehicles. Participants with prior course knowledge, such as autonomous driving and car design, were then given knowledge from relevant literature.
• During the workshops: Each team was asked to perform the following tasks: (1) Role playing; (2) Gigamapping [70], including system actors mapping and related technical mapping; (3) Brainstorming, creating possible futures according to the actor’s role and the relationship between roles; (4) Future scenarios, divided into foresight (short-term future) and speculation (long-term future); (5) Backcasting, defining a desirable future and identifying programs that will connect that future to the present; (6) Problem (re)definition; (7) Brainstorming, specifically the service ideation starting from the actors’ role and the relationship between them; (8) Ideas validation; (9) Idea iteration; and (10) Idea development.
• After the workshops: Each team was asked to perform critical reflection and discussion.

4.2. Process and Tools

Three workshops were conducted simultaneously for the interior, exterior, and station of the Automated Shuttle Bus. Each scenario was analyzed by groups A and B, exploring the short-term future (in 5 years) and speculating for the long-term future (50 years later). For any scene, each student was required to choose one of the following roles:

• For the interior scene: five roles of ordinary passengers, service providers, people with travel impairment and disabilities (the elderly, pregnant women, etc.), and pets.
• For the exterior scene: four roles of pedestrians, motor vehicle drivers, non-motor vehicle drivers, and pets.
• For the station scenario: five roles of people waiting for the bus, online service providers, offline service providers, passersby, and pets.

At the beginning of the workshop, the participants build empathy through role-playing, a tool to complete the gigamaps. Members of each team chose different roles and started preparation based on their roles. It helps them to see the world through the eyes of the people they were designing for, to understand their reality and uncover their latent needs, and temporarily withdraw from the designer’s role (see Figure 4 for examples). Every team prepared rough materials to facilitate them to act out their performance. For the interior scene, Group A chose people with travel impairments so that the audience could focus on the story and propose solutions. Through system actors mapping, different stakeholders were discovered and explored.

The transition to SAVs undoubtedly disrupts to current transport socio-technical systems [71]. Therefore, gigamapping, as an ultra-wide mapping across multiple layers and scales, was used to critique system concepts and frameworks, which helped the participants investigate relations between seemingly separate categories. The map represents all the stakeholders involved in a project, clarifying their roles and relationships. As a tool for reflecting on the different positions, it helps the designer to define how to deal with them.

The forms and trends of autonomous driving and human–computer interaction technology could be used to enrich the system diagram, as illustrated in Figure 5.

Based on the gigamaps, all groups were required to build their worldviews and brainstorm possible futures, including multiple directions and possibilities. They were also necessary to consider different personas and imagine usage scenarios. After that, each group was required to choose three scenarios and use the scenario creation tool to structure and detail a narrative, considering the trends, contexts, and implications to create future scenarios.

Opposite speculations for the long-term future emerged—precisely, perfect idealism (see Figure 6) and pessimistic realism. The three groups A showed a more positive vision of the short-term future based on the development of the current society.

After visually introducing the future scenarios, future backcasting is a suitable tool for group activities to reflect on how their choices affect the future and (re)define the problem. The technological development is still predictable for all groups A (+5 years). Different elements of the future concept must be distinguished, identifying which aspects seem too futuristic. Then, they redesigned these elements to make them appear believable in the short term. Ultimately, all groups were required to describe or illustrate the scenario with redesigned future elements and identify design challenges to implement the backcast scenario. As for all groups B, they needed to explore the value of the expectations since they had few restrictions. They had to answer questions, such as: What is provocative about this? What is interesting or unusual about this opportunity? How might this be meaningful to the scenario?

Finally, service ideation was completed by brainstorming. Every group formed its concept library (see Figure 7) consisting of preliminary design concepts to achieve a desired future or avoid an unwanted one. Group members use different colors to distinguish their proposed ideas.

The final phase was iteration. All groups had to vote on three ideas to extend further and visually present. Groups A and B shared their design ideas in the same scene, reflected on their schemes, and thought about how to iterate. There are two iterations in this phase. Finally, the design scheme was deepened according to the feedback. It could be presented in various ways, such as design prototypes and videos, the design background, the system diagram (including stakeholders), future scenarios, etc.

4.3. Results

4.3.1. Interior Workshop

When envisioning the distant future (+50 years), the external environment forecast for the shuttle bus was progressively worsening. Hence, interior design tends to be more intelligent in solving emerging challenges. Five insights were identified for the interior of the automated shuttle bus:

Inclusiveness of functionalities. To be more inclusive, potential users’ demands must be better classified according to different user groups and various internal activities. Concerning the target user groups, group A provided liftable and convenient access for people who need wheelchairs. Group B used an age-appropriate interface for elderly users with reduced cognitive function. Interior activities, including relaxation, entertainment, sociality, companionship, and official business, were considered valuable and desirable for the customers during the ride.

Communication transparency. A combination of passive (screen or announcement) and active (voice assistant) communication is best for important information. Due to technical limitations, group A installed a prompt screen on the seat handle and helped users browse the info needed with gesture operations. Group B planned to use front-projected holographic display and augmented reality technology to achieve the interaction, as illustrated in Figure 8 and Figure 9.

Interior layout. Group A preferred more functional and personal spaces in the +5 years concept, while Group B imagined the +50 years position as more social and relaxed. Figure 10 and Figure 11 present the comparison. Passengers have the opportunity to communicate with friends in the +5 years concept. Still, there is an isolated space for travelers who do not want to be disturbed or have social phobia. Additionally, Group A favored a minimalist and functional design language. As for the +50 years concept, Group B modularized the space’s design and expanded the entertainment activities by rotating and combining the seats according to different scenes.

Seat design. This included shape, arrangement, and functionality. The arrangement is divided into body position, seat placement, and direction. Compared to the +5 years concept, the seat in +50 years overcomes position limitations even further, allowing for 360-degree rotation and convenience for in-vehicle users who can face each other in a circle for group entertainment activities. On the other hand, this concept focuses more on comfort, with the overall shape of the seat more in line with ergonomics. Figure 12 depicts this.

Classification of the systemic mode. This was proposed by Group B. With increasing environmental concerns, the harsh environment outside the vehicle triggers the need for a safety mode to protect the occupants. Environmental conditions outside determine the internal operating way. Normal mode allows passengers to perform the following activities: entertainment, seat adjustment, and real-time 3D information browsing. When encountering slight intrusion (unexpected severe weather conditions, such as hail or snowstorm) or road congestion, the system switches to emergency mode, and the vehicle establishes a safe space for passengers in advance.

4.3.2. Exterior Workshop

The concepts developed for closer (+5 years) and distant (+50 years) future scenarios facilitate communication between VRUs and the ASB differently. Specifically, Group A focused on cyclist users and developed a set of hybrid shuttle bus and bicycle travel modes. In comparison, Group B took visual communication as a better way for other road users to experience, which is valuable to convey a sense of security. These two design schemes are described in detail below.

Group A served the cycling community that prefers the green mode of travel, and the increase of counter-urbanization is their speculation for the closer (+5 years) future. Although bicycle sharing has become a mature system in China, its physical exertion and speed limitations mean that it needs to be used in some cases with other modes of transportation to extend its freedom. A user journey (telling a story in Figure 13) with crucial situations was chosen to describe the design outputs. In the regular section of the road (as shown in the black numbers), the passenger swipes the card at the station to pay for the bike via Near Field Communication (NFC) technology and pushes the bike to the designated position, where it will be automatically suspended. When the user arrives at the destination, they get off the bus and retrieve the bike to continue riding.

Furthermore, the bike’s front wheel is fixed and attached to the bus when the shuttle bus enters an improved safety zone (as indicated by the blue number). The cyclist can continue riding with the bus pulling to save energy. Group A focused on the presenting the design solution’s mechanical structure and the bus–user interaction’s process and functionality. Figure 14 shows the rail device at the bus’s rear for securing and transporting bicycles, the boarding and alighting process for regular passengers and bicyclists, and the safety tips for VRUs.

In the distant (+50 years) future scenarios, Group B focused on implementing interaction with the bus for VRUs in different critical situations. The worldview they built is biased toward positive development, the population tending to decrease, and the city being divided according to functional areas. However, private cars still exist. Based on such a worldview, transportation is in a suspended mode of operation. They updated the appearance of the shuttle bus (Figure 15, the model references the shape of a yacht). The roof extends backward and provides space for other vehicles that the passengers carry at the rear.

Figure 16 shows the overall visual communication. In the security zone situation, when a passenger stands too close to the bus, the aura of the bus pulses red to signal to move away from it, and a green area appears on the ground to mark the waiting area. During the entry/exit process, the aura is red to show that passengers getting off the bus come first. Then the signal area on the ground turns green to inform that passengers are allowed to get on. When an obstacle is encountered, the shuttle bus makes an emergency stop. It uses a front-projected holographic display to project an indicator signal in the airspace to alert vehicles behind it. In addition, as the shuttle bus breaks down, holographic projection is used to display signs for warnings and to guide the vehicles behind it to pass.

4.3.3. Station Workshop

According to the prediction of scholars, the fixed stations of ASB will be eliminated in the future. Flexible virtual bus stops are requested by passengers so that they can board or get off wherever they want. Nevertheless, the participants gave the stations other functions.

In the near future (+5 years), a multi-themed immersive station is proposed by Group A, and their foresight is illustrated in Figure 17. The station is given the function of exhibition and dissemination with different themes, including history and culture, green travel, animal protection, etc. Due to the time constraint of the workshop, Group A deepened the station for the animal protection theme, taking penguins (the most affected animal by glacial melting) as the interactive character and dividing the whole station into two areas (see Figure 18). Area A contains traditional functions, such as providing a resting space with seats, tactile paving for visually impaired user groups, and a ground-embedded progress bar showing the arrival time, etc. Area B is a promotional and experiential area for the related theme with an interactive screen. Passengers can interact with the penguin images through the interface and purchase souvenirs and daily products through the vending machine; a portion of the proceeds will be donated to animal protection organizations. The three views of the station and the external rendering are shown in Figure 19.

When envisioning the distant future (+50 years), Group B’s speculations seem more science-fictional and catastrophic (see Figure 20). A new community model is applied to the crowded and populated environment where humans will experience temperature and sea level rise and a sudden decrease in areas suitable for survival. Due to the harsh external environment and the relatively closed community space of human beings, the shuttle bus is aimed at cross-community fixed-point transportation. The community environment is not limited to the ground; some communities will be built beneath the seabed to avoid the risk of tsunamis, for example. Furthermore, the area of the air is also regarded as a space where the community can be developed, which is on the futuristic side. At the same time, the transmission of information between humans and machines is de-realized based on a new interaction form of the Brain–Computer Interface (BCI).

The design output is divided into two aspects. Group B redesigned the appearance and layout of the station (see Figure 21). The concept they developed is significantly different from the current shuttle bus station and is more oriented toward a large train station, which is very large in scale and contains many other functions, such as shopping and business negotiation, as shown in the storyboard (Figure 22). It is more like a transportation hub that connects with about ten surrounding communities. An Iris HMI system is proposed to support humans to receive the end of information directly and to help the user process it (Figure 23). The images are projected on a person’s retina and converted into data by Electroencephalographic (EEG) technology, thus completing the response to interactive actions through BCI.

4.3.4. Discussion of the Workshop Results

The outcomes of these workshops suggest that when systemic design tools and speculative design strategies are combined in the investigation of ASB, designers will disrupt future urban transport patterns and open a challenging design space to address more systematically. Current existing design research or workshops in this field have been primarily guided by service design through subjective data collection methods (such as observation and interviews) [72], as well as on prototype outputs and iterations of HMIs in interaction design [73], which implies a lack of a systemic design perspective. Thus, the results of the workshops are entirely novel in the relevant literature, which indicates the viability of utilizing systems design to speculate the sociotechnical system innovations triggered by the future diffusion of ASB.

5. Reflection and Discussion

The results of the workshop yielded feedback on the design process we proposed. This work focuses on whether the proposed design process can stimulate designers’ creativity and assist them in better exploring future HMI in automated shuttles; thus, critical reflection on the design process proposed to participating students is critical. A mixed-methods approach was followed, collecting both qualitative (interviews) and quantitative (online questionnaire) data.

An online questionnaire was prepared via Tencent Questionnaire for the 28 undergraduate students participating in the workshop, and all of them answered it. Moreover, the questionnaire was not distributed to other individuals who had not attended the workshop since they lacked experience in this subject, rendering their comments irrelevant to the issue under investigation. Participation was voluntary, and there was no type of incentivization. The survey was designed in Chinese, and the questionnaire contained open text, multiple choice, and Likert scale questions. It was possible to gain qualitative insights and quantitative data in this way.

The questionnaire consists of a fe” par’s. The first part briefly introduces the objective of the survey and collects demographic information about the respondents, including grouping situation, primary responsibilities within the group, previous design methods used, etc.

The second part uses multiple choice and open text to collect respondents’ perceptions of the impact of tools of speculative design and systemic design used in the workshop. Figure 24 and Figure 25 show the respondents’ perceptions and preferences. Participants in this section are free to select any number of tools. If all 28 respondents had chosen the same tool, the statistics for that tool would indicate 100%; more specifically, the percentage in the graph shows the proportion of survey respondents who selected that tool to all survey respondents (28 individuals).

The three tools that participants found the most valuable for speculative exploration were constructing worldview, future scenarios, and brainstorming design ideas. They would also like to learn more about gigamapping, stakeholder maps, and system map construction.

The third part consists of aggregating the factors that respondents perceive to influence the design output and exploring the extent to which they pay attention using 5-point Likert scales (cf. Figure 26). The factors participants proposed include Perceived Usefulness (PU), Perceived Ease-of-Use (PEOU), Performance Expectancy (PE), Effort Expectancy (EE), Self-Efficacy, Perceived Safety, Perceived Benefits, Perceived Risks, and nine other characteristics of ASB.

The last part of the questionnaire was to determine how the participants felt about the whole workshop through open-ended questions and interviews. Several participants could hardly imagine new inventions for the +5 years configuration because it seems functionally/technically impossible. In contrast, some participants mentioned that the future (+50 years configuration) would be an electronically focused world where Artificial Intelligence will gradually replace human occupations and help them do jobs that may be difficult. In addition, a specific type of product may integrate the functions of various other products, thus presenting an integrated character and eventually eliminating products with only a particular function. For example, a smartphone combines the functions of a camera, a tape recorder, an MP3 player, and other products.

In addition, it is essential to consider the applicability and utility of the design process and tools utilized in this workshop in other study disciplines. The design tools used in the workshop (such as gigamapping, future scenario construction, etc.) have been utilized extensively in different domains (such as healthcare, sustainability transitions, public organization, urban area, etc.). Speculative design in healthcare services, for instance, has been proven to drive service designers to approach organizational and community problems from a future-oriented viewpoint [74]. Alternately, the potential of speculative design applied in practice as a tool for systems transformation [75] has been examined. Moreover, through a systems-oriented design (SOD) approach, an integrated research approach combining methods such as research by design, gigamapping, interviews, and literature mapping was utilized to gain new insights into the contemporary design practice of fostering and disseminating organizational co-design capacities [76]. Additionally, the combination of systemic design and speculative design methodologies might facilitate decision making for credible, desirable, and sustainable urban futures [77]. In conclusion, combining these design methods is also helpful for workshops on other subjects.

6. Conclusions

In exploring a future HMI for ASB, this paper provides insights to enrich and innovate the HMI design process based on methods in speculative design and systemic design. Expanding the HMI design dimension is achieved by critically considering the uncertainty of future ASB development and the complexity of public transportation systems.

Three parallel workshops were organized to test the proposed approach’s value. The entire workshop process was divided into four steps: (1) systemic exploration, (2) critical future foresight (divided into short-term future and long-term future), (3) three or more iterative design cycles, and finally, (4) critical reflection. The workshop outputs provide perspectives on the future HMI design for ASB for in-vehicle, out-vehicle, and station.

The proposed process proved valuable in the feedback. With the inclusion of the two design approaches, participants constructed their worldviews and future scenarios. They were able to produce design outputs to solve the design problems and challenges, delivering insights into the future HMI design for ASB.

The speculative exploration generated by our design process still needs to be evaluated, validated, and modified in the real world. Future iterations in real projects will help to consider other specificities of the usage environment and explore unforeseen opportunities and risks.

In conclusion, Systemic Design and Speculative Design are approaches that should be applied in exploring autonomous driving technology in the future public transportation field. The design process proposed in this paper focuses on the Analysis and Synthesis steps based on the relevant principles of interaction design. In the future, the action of Evaluation should also be considered, such as how to detect the impact of ASB HMI on user acceptance. Furthermore, understanding the personal preferences of different users’ categories is a worthy research direction for the future that, to the authors’ knowledge, has not been investigated yet.