A Sustainable Development Process for Visually Interactive Companions in Ubiquitous Passenger Information Systems

Abstract

In today’s increasingly complex and multimodal mobility environments, passengers are confronted with fragmented information, inconsistent user interfaces, and limited context-adaptivity across public transport systems and services. These challenges hinder a positive mobility experience, reduce trust, and limit the broader adoption of sustainable transport options. This paper addresses these gaps by introducing a structured, user-centered development methodology for Visually Interactive Companion Technologies in Ubiquitous Passenger Information Systems (VICUPISs). The approach incorporates system characteristics, contextual factors, and a comprehensive process framework. Drawing on applied research and development projects, the methodology defines a five-phase development cycle—from field to concept and back—combining expert insights and user participation across iterative development stages. A central contribution is the integration of a rich context model spanning eight dimensions, enabling adaptive, multimodal, and personalized interaction across mobile, embedded, and public displays. The methodology also incorporates AI-supported adaptivity and addresses the resulting challenges for usability evaluation. Sustainability is considered at three levels: resource-efficient system development, long-term extensibility and adaptability of digital systems, and support for a modal shift toward environmentally friendly public transport. The proposed methodology offers a replicable and transferable foundation for designing human-centered, future-ready information systems in public mobility, complemented by practical heuristics and insights from two case studies of sustainable transport ecosystems.

1. Introduction

The development and operation of sustainable mobility systems today faces change and challenges: On one hand, the increasing digitalization of public space today plays a major role for topics such as flexibility, optimized use of resources and improved usage options for passengers. On the other hand, public transport is increasingly focusing on the mobility experience [1] as the basis for growth in eco-mobility, but often with little success, as apps, services and systems are often created in processes characterized more by implementation, technology and island solutions than user centricity and sustainability.

In addition, the way in which passengers obtain information, which services and flexibility they expect and how they interact with mobility infrastructures as a whole is changing fundamentally. With the further development of mobile and ubiquitous technologies, a new generation of visually interactive systems is currently emerging that is intended to provide context-sensitive, adaptive and multimodal support for travelers and, as a new class of highly interdependent mobility systems, will only deliver sustainable solutions if they are developed in a user-centric and methodical manner, focused on public transport and context factors that impact use as well as development processes. However, despite numerous innovations and implementations at the level of individual devices or services, namely apps and web services, there is still a significant lack of exactly such a systematic and user-centric approach that uses the understanding of structures and characteristics of public mobility systems to develop sustainable systems that also keep pace with the short-cycle changes in technology, policy and user requirements within the complex and dynamic environment of public transport.

From a user-centric perspective, existing frameworks of human–computer interaction and user-centered design provide a solid foundation for the design of interactive systems, but are predominantly tailored to stable, single-user and device-centric contexts that are no longer sustainable in dynamic and adaptive environments. In addition, the mobility sector and especially public transport pose fundamentally different challenges for sustainable systems. These must function in heterogeneous physical spaces (vehicles, train stations, regions, devices), enable interactions between users in public environments and adapt dynamically to changing contexts such as changes in location, delays, crowds or user stress. Traditional methods of usability and interaction design often do not respect these circumstances, which leads to fragmented or suboptimal passenger information systems that are then often hardly used or are withdrawn. There is a particular lack of special methods for the user-centered development of sustainable systems geared towards public mobility systems as part of a Mobility Experience Design that enables the seamless integration of dynamic mobility, adaptive systems and visually interactive interfaces in ubiquitous mobility systems in a human-centered way. Unfortunately, standard design processes usually do not consider the ubiquity, adaptivity and context dependency that characterize these systems.

The aim of this work is therefore to develop a new approach that enables interactive passenger information systems in ubiquitous mobility systems and their user contexts to be systematically described, categorized and developed with a strong UX-focused engineering process that comprises the field characteristics as well as conceptual innovations in a wholistic process, compatible with current process frameworks. This approach must therefore bring together user-centered design principles and field-oriented development and evaluation methods typical for information systems in public transport to achieve sustainable results. It ultimately has to aim for supporting the creation of coherent, user-friendly and adaptive companion technologies in increasingly ubiquitous public transportation systems.

The following sections provide an overview of the current state of the art, identify gaps, classify contexts and system types and develop a development process for these systems—in the form of a structured, iterative and field-integrated framework for the development of sustainable, ubiquitous passenger information systems.

2. Materials and Methods

This paper is the result of a comprehensive analysis, abstraction, and synthesis of findings from over 15 years of national, regional and international applied research projects, development, and evaluation in the field of Ubiquitous Passenger Information Systems. It draws on a series of public transport research projects conducted primarily in Germany between 2008 and 2025, each addressing different aspects of digital mobility, adaptive systems, and multimodal passenger communication.

2.1. Project-Based Methodological Foundation

The methodology proposed in this paper builds on findings from research and development projects as well as conceptional scientific work, empirical insights, prototyping efforts, and user studies from the following major research and development initiatives:

• IP-KOM-ÖV: Focused on integrated passenger information services and early concepts of digital communication platforms and information services in public transport.
• DynAPSys: Developed an agenda-based dynamic travel companion capable of context-sensitive decision support and guidance throughout the mobility chain.
• DYNAMO: Created a door-to-door multimodal mobility assistant, integrating real-time data and adaptive routing through personal apps and public displays.
• SmartMMI: Introduced smart windows as in-vehicle public displays for ambient, location-aware passenger guidance; included lab and in-train field testing with real passengers, third-party data sources and novel interaction concepts in public transport vehicles.
• RegioKArgoTramTrain (ongoing): Focuses on hybrid human–robot interaction and information systems in tram-based cargo and passenger transport.
• IADAPT (ongoing): Investigates adaptive control center interfaces and integration of AI-driven context-adaptivity in control centers and public transport operations.
• A range of multimodal transport and pedestrian guidance projects (e.g., Real Lab GO Karlsruhe), exploring personalized and visual mobility assistance in urban contexts, including a real-lab for pedestrians and bikes, e-scooter, public transport smart data and visualizations.

These projects combined qualitative and quantitative methods, including user surveys, eye-tracking, lab and field experiments, participatory design workshops, and multi-modal data collection as well as computer science and engineering research and prototyping. Many of the core methodologies have been iteratively refined and applied across different technological contexts, contributing to the holistic development process presented in this paper.

From these domain-specific projects with their concepts, implementations and experimental evaluations, we derive a framework.

2.2. Data and Material

The paper builds on project datasets, design models, evaluation instruments, and prototypes developed during the aforementioned research projects. While some datasets are available through publicly funded data and project portals, others are subject to confidentiality agreements with transport operators. Many aggregated datasets, studies and results contained have already been published in the references we have included. No sensitive human or health-related data was collected; ethical approval was not required for the usability studies cited, as they were conducted in accordance with institutional guidelines for non-invasive, anonymized evaluation procedures.

2.3. Use of Artificial Intelligence Tools

To support the formulation and refinement of this publication, generative AI tools were employed: OpenAI’s ChatGPT GPT-4o was used mainly under the direction of the lead author for structured drafting, argument refinement, outline generation, and stylistic editing, particularly in the synthesis of project results into the methodological framework developed. DeepL Translator v25 was selectively used to improve linguistic quality and terminology consistency, especially in the conversion of draft materials between German and English. No AI tools were used editing this version.

AI was not used for data generation, analysis, or the creation of original scientific claims. All conceptual models, diagrams, and research results are derived from primary research and author expertise.

3. Related Work, State of the Art, Research Gap and Objectives

Public transport systems are continuously enhanced through digital services that provide passengers with interactive and personalized travel information. In practice, on one hand, necessary services are realized, based on standards like the TRIAS specification [2]; on the other hand, information apps are released by public transport authorities and companies as well as stop displays that are more and more technically advanced. These newer developments also comprise mobile travel companions with extended and individualized functionalities, smart displays integrated into the environment, and context-aware interfaces, which aim to improve passenger experience, reduce uncertainty while improving the perceived level of security, and support multimodal mobility decisions [3,4].

With the advent of ubiquitous computing, these systems are no longer bound to static environments but are embedded in mobile, pervasive, and context-rich infrastructures, thus becoming ubiquitous. Numerous studies have explored technologies such as smartphone-based trip tracking [5], context-aware public displays [6], and personalized routing applications [7]. Similarly, advances in User-Centered/Human-Centered Design (UCD/HCD) and Human–Computer Interaction (HCI) have introduced methods for tailoring interactive systems to user needs, leveraging techniques such as contextual inquiry, UI concepts and heuristics [8], and usability testing [9,10], as well as standardizing the User-Centered Design Processes [11], also tailored to Pervasive Computing environments, e.g., in public displays mainly deployed statically on stops and often based on mobile devices like [12] in Public Transport.

This leads to massive changes and challenges in the following five domains, targeted by our research and generally in public transport systems:

• Visually interactive systems from LED-displays to projections;
• User-Centered Design approaches for improving mobility experience;
• Context-awareness and adaptivity of passenger information systems;
• Ubiquitous and public system challenges;
• (User) evaluation of adaptive systems in dynamic mobility contexts.

In the following, we describe these challenges in more detail to derive insights and requirements for our contribution to defining, developing and evaluating these systems.

3.1. Visually Interactive Systems in Public Transport

In recent years, we have seen a growing interest in improving the mobility experience in public transport through interactive and more visual than textual or auditive communication with passengers. From mobile devices to public displays, research has shown that digital passenger information can significantly reduce perceived waiting times, increase satisfaction and improve the overall public transport experience [6].

Public displays, for example, have been extensively studied for their role in urban communication and passenger information [12]. However, evaluations often focus on static installations rather than adaptive, mobile and embedded solutions [13] provides an overview of public transport interfaces but highlights the lack of structured methodologies for developing adaptive, ubiquitous systems that are integrated across user devices and infrastructure. This also requires the development and establishment of visualizations and interactions that are appropriate to the public transport context, i.e., the public and public systems, usable for non-experts, as this has happened in domains such as weather simulation (forecasting), which was made consumable for the general public, e.g., web users.

3.2. User-Centered Design Approaches for Mobility Experience

User-centered design (UCD) approaches are well established in software development and Human–Computer Interaction (HCI) [10]. ISO 9241-210 [11] formalizes these principles, specifying four iteratively applied phases containing user and context analysis, requirement specification, solution development (creation) and evaluation. Also, psychological aspects like the impact of real-time information has been subject to research [14]. However, applying these models to public mobility systems raises significant challenges: real-world transport environments are highly dynamic, involving changing user contexts, variable service conditions, large user groups and socio-technical constraints [15]. New or adapted approaches are therefore needed to deal with situational variability, public system challenges and integration of field data into the design and development process.

Despite advances in UCD, significant challenges remain in the user-centered design of highly usable systems for the mobility experience. The complexity of combining different modes of transport into coherent mobility chains and the increasing functionality required of mobility companions with regard to context and user adaptability, as well as heterogeneity, complicate the creation of intuitive and efficient user interfaces [16] in traditional development processes. The requirements to meet the accessibility needs of diverse passenger groups also highlight the critical importance of inclusive design practices for future public transport systems in projects like [17] by UITP and show the heterogeneity of user groups in public transport.

3.3. Context-Aware and Adaptive Passenger Information Systems and Companions

The design and development of adaptive passenger information systems that dynamically respond to user context has become a prominent area of research in the field of ubiquitous computing and mobility systems. Recent approaches have leveraged multi-device environments to enhance personalization and responsiveness, often relying on passengers’ personal mobile devices as the primary source of contextual information and way of interaction [18]. In addition, the integration of wearables and public displays has been explored to provide passengers with seamless, context-aware information services that adapt to location, activity, and user preferences to enhance engagement and navigational support in transport environments [19].

However, with the increasing complexity of these systems, scientific challenges have emerged: The need for multidimensional, machine-interpretable context modeling to enable real-time adaptation to contextual factors—including spatial location, temporal constraints, socio-technical contexts, cognitive states, and physical conditions—has greatly increased. These context types need to be captured, interpreted and operationalized for interaction and communication in a way that ensures accuracy while still being computable and manageable [20]. Systems such as our DynAPSys have demonstrated the potential of adaptive companion technologies for public transport passengers, but a generalized and (re)usable modeling framework for context (adaptivity) in public transport is still lacking.

In addition, public transport environments and systems are characterized by heterogeneous interaction spaces, ranging from personal mobile companions to vehicle-embedded displays and large-scale public information systems such as smart windows. Users are informed across devices, modalities and usage situations, often simultaneously or synergistically. This requires selective, adaptive and multimodal communication strategies that can distribute content meaningfully across different interaction modalities and interactive systems without overwhelming users or fragmenting the experience [20].

Despite promising prototypes and partial solutions, current research often addresses individual aspects—such as context recognition technology, device orchestration, visual communication or personalization—in isolation, while a comprehensive methodological foundation for the systematic and user-centered development of context-aware and adaptive passenger information systems is still lacking.

3.4. Ubiquitous and Public System Challenges in Shared Spaces

Ubiquitous systems deployed in public transport environments must operate in shared physical spaces with multiple and simultaneous (e.g., smart windows) users, often strangers, leading to problems in privacy, e.g., shoulder surfing [21], even for personal mobile devices. This creates challenges not typically encountered in private, business or classic personal device contexts. Foremost among these is the tension between personalization and public visibility/public systems: these systems need to provide relevant, user-specific information while avoiding revealing sensitive or private data to nearby observers [19,22]. This trade-off is particularly salient in passenger information systems, where individual routing, preference or accessibility needs must be addressed without compromising social acceptability or usability in crowded, public environments. In addition, public systems must support universal accessibility for a broad user base—including people with disabilities, language barriers, or unfamiliarity with the system—often without knowing user attributes in advance and without collecting them as part of the design and development process. These conditions require new interaction paradigms and visualization strategies that are context-aware, non-intrusive, and capable of mediating between ambient public information and discrete personal assistance across mobile and embedded (ubiquitous) devices. Research into proxemic interactions, glanceable interfaces and cross-device user handovers has begun to address these challenges [6,23], but there remains a significant gap in existing design processes towards coherent design of adaptive visual companions into ubiquitous public (transport) infrastructures and in addressing these requirements within a user-centered development process over the whole range from field to concepts and back.

3.5. Evaluating Adaptive Systems in Dynamic and Public Mobility Contexts

The evaluation of adaptive systems in public and dynamic mobility environments poses significant methodological challenges that fundamentally contradict the assumptions embedded in classical usability evaluation approaches. Traditional user-centered design (UCD) processes, as defined by ISO 9241-210 and similar frameworks, typically rely on evaluation methods centered on specific user group and conducted in stable, controlled settings, with an emphasis on well-defined tasks and repeatable procedures [9,11]. These approaches assume relatively static usage contexts and do not take into account the fluid, unpredictable and situational interactions that are common in public transport environments.

In contrast, adaptive systems in dynamic and public mobility contexts operate in highly dynamic socio-technical environments characterized by movement, interruption, contextual variability and user diversity. Passengers interact with systems while in vehicles, transferring between modes, or reacting to disruptions—situations where attention is fragmented, tasks change rapidly, and environmental conditions (e.g., crowding, noise, infrastructure availability) influence interaction. In such situations, adaptive systems need at some point to be evaluated in situ, focusing not only on interface performance, but also on how system behavior adapts in real-time to individual needs and changing contexts under real-world conditions.

Applying static or summative lab-based evaluation methods to such systems often provides incomplete or misleading insights, as the core functionality and benefits of adaptivity only unfold under authentic, temporally and spatially embedded usage conditions. For example, interaction results changed dramatically when we gave luggage and an umbrella to mobile companion users walking to the stop while aiming to catch the right bus. Our own studies on adaptive mobile travel companions therefore confirm that meaningful evaluation requires contextual realism—including real traffic delays, weather conditions, station environments, wrong trains, boarding dynamics and passenger workflows [24,25,26].

These insights are echoed in the emerging literature on multimodal and context-aware interface evaluation, which advocates novel evaluation approaches that can capture fluid interaction patterns, support heterogeneous device eco systems, and reflect the multi-user dynamics of public environments [6,23]. There is a need to move beyond conventional UCD practices towards field-integrated, scenario-driven, time- and even context-responsive evaluation methods that can assess system and user behavior and adapt to the effects of contextual changes in real-world transportation and mobility situations in general.

3.6. Research Gaps and Objectives

Therefore, while the relevance of digital interaction in public transport is widely acknowledged, major research gaps remain:

• There is a lack of a systematic approach to understand and describe visual, mobile, and ubiquitous interactive systems, particularly those embedded in dynamic, public transport environments and mobility. Traditional HCI models fall short in capturing the interruptible, time-sensitive and multi-user nature of public transport scenarios [27].
• Also, there is a lack of a systematic description of contexts in public (transport) systems and models or heuristics to accomplish this. There is also no structured approach for evaluating contexts. Existing taxonomies and design frameworks often address either static public displays [6,19], mobile applications [28], or wearable systems [29] but rarely integrate all aspects within a cohesive context-aware framework.
• Ubiquitous Passenger Information Systems deviate strongly from classical desktop or web interaction paradigms, requiring user-centered design system and interaction processes that consider dynamics like transient attention, context shifts, physical movement, shared displays, and infrastructure constraints.
• Sustainable ubiquitous public systems also need to consider specific requirements, contexts and development paths in future and already in the development, thus requiring field insights as well as lab- and concept-based aspects to fulfill these requirements in the long term and for many and changing users.

These gaps hinder the structured development, evaluation, and comparison of such systems, leading to fragmented solutions that are difficult to scale, evaluate, transfer into operational practice and evolve—thus not being sustainable.

Therefore, this paper aims to address the identified gaps by proposing a user-centered, structured and field-oriented development approach for Visually Interactive Companions in Ubiquitous Passenger Information Systems (VICUPIS). The goal is to establish a methodology that

• Enables systematic description and classification of VICUPIS and adjacent systems to better understand their specialties;
• Includes a basic and general context model to be used for VICUPIS, containing multiple contextual dimensions;
• Supports the development of usable and adaptive user interfaces for public transport scenarios, informed by real-world user requirements and contextual data;
• Integrates field as well as conceptual aspects in a coherent, iterative design process;
• Contributes to the broader understanding of how user experience (UX) design and UCD processes must adapt to ubiquitous mobility environments that are dynamic, multi-device, and socio-technical by nature.

By combining practical evaluation methods with conceptual structuring, this work aims at contributing to establishing a replicable and extensible foundation for the design and evaluation of VICUPIS in both research and applied settings, covering the journey from field via lab to concept and back, see Figure 1:

4. Systematics

4.1. Systematizing Ubiquitous Passenger Information Systems Through Mobility and Embeddedness Dimensions

To establish a coherent understanding of and differentiate ubiquitous passenger information systems, we adopt and extend the classification by Lyytinen and Yoo [30], which organizes ubiquitous systems along two core dimensions:

• Mobility: Denotes the extent to which users can interact with a system while in motion, or conversely, the degree to which the system itself supports or facilitates mobile usage.
• Embeddedness: Refers to how deeply a system is physically and functionally integrated into its surrounding infrastructure or environment—ranging from solitary devices to fully embedded public installations.

This two-dimensional framework provides a structured basis for positioning and differentiating passenger information systems, their components, and facilities in terms of their spatial presence, infrastructure coupling, and (together with context modeling) situated user experience.

In the context of public transportation and passenger information systems, we propose to extend this model by introducing a third analytical perspective: Visual interaction including Visually interactive behavior. As visual communication has more and more become the dominant modality in both personal and public mobility systems, this dimension captures visual interfaces, adaptive visual information presentation, and visually oriented multimodal user interaction in public transportation. Visual interaction is not just a channel—it is a design driver that affects attention, perception, accessibility, and trust in real-world scenarios, while on the other hand introducing the risk of requiring perceptive focus and increasing cognitive load and distraction.

The resulting systematization serves as both an analytical tool and a practical framework: it supports the classification of existing and emerging public transport and passenger-centric technologies, guides the design and evaluation of new systems, and provides a conceptual foundation for the development process presented in the following sections.

Based on the dimensions of mobility and embeddedness, designers and researchers can better position and analyze such systems. Importantly, the visual interaction perspective adds a layer of understanding to how such systems operate in real-world mobility contexts, considering both technical capabilities and human factors. As Cyber–Physical Systems (CPSs), transportation systems experience real-world constraints, and adaptivity allows systems to dynamically respond to environmental stimuli or user behavior while coping with changes in the real world and limitations of physical environments that are not fully controllable.

While the presentation of public vs. private data is not used as a formal dimension, it remains an important contextual consideration. Systems such as companion apps work with personal user data and require privacy safeguards, while digital signage must generalize information for group consumption. Thoughtful design must balance visibility, relevance, and privacy across different types of systems.

4.2. Categories and Characteristics by Example

The classification of ubiquitous passenger information systems, especially VICUPISs, can be illustrated by existing representative system types, positioned along the axes of mobility and embeddedness, and extended by their Visually interactive qualities. While issues such as privacy, personalization, or the level of CPS integration of physical and virtual system components are relevant, they are treated as contextual design considerations and parameters rather than primary classification dimensions. Example system types include the following:

• Mobility apps (e.g., smartphone-based travel assistants): Highly mobile and not embedded. These systems provide graphical, touch, and visual user interfaces on a mobile device that support real-time travel information, routing, and user preferences.
• Smartwatch/wearable UIs are also highly mobile and lowly embedded, with minimalist, at-a-glance, touch-based interactions suitable for micro-interaction and notification support and (near)body sensory capabilities.
• In-Vehicle Displays are semi-embedded and semi-mobile systems within public transport vehicles. They provide low to no interactivity, shared visual communication for orientation, stops, advertising, news and announcements.
• Interactive windows (e.g., transparent in-vehicle displays) are only moderately mobile (used in transit, moving with the vehicle, sometimes used also from outside), but highly embedded in the vehicle infrastructure. Visual interaction is ambient, contextual, and often also passive and tied to location or motion (context).
• Digital signage at bus stops or train stations as embedded and static systems support bold, legible displays for real-time departure information and service changes. Interaction is often minimal or nonexistent.
• Ticket vending machines: Fixed or in-vehicle systems that provide more transactional, task/user-driven visual interfaces, mainly for booking and trip planning; typically touch-based and requiring focused attention.
• Wall/floor projections or screens may be embedded in stations or vehicles; these systems sometimes support ambient interaction and custom icons for users or user groups. Although not mobile, they may be proximity sensitive and provide orientation or dynamic notifications; others are mainly used for advertising, which is not in focus here.

Table 1. Overview of system examples in terms of mobility, embeddedness and visual interaction.

System Type	Mobility	Embeddedness	Visual Interaction
Mobility App	High	Low	Mobile, screen-based, graphical touch user interface plus device gestures
Smartwatch/ Wearables	High	Low	Glanceable, minimal interaction, small text and graphical user interface
Interactive Window	Medium	High	Ambient display, AR, graphic, touch
In-Vehicle Display	Medium	Medium	Shared, low interactivity, graphic UI
Digital Signage	Low	High	Bold, simple, public display, low/no interaction, often text, symbols
Ticket Machines	Low	Medium	Transactional, often graphic touch interfaces, more text and buttons
(Wall/Floor) Projection/Screen	Low/Medium	High	Ambient, gestural or passive, sometimes connected and providing individual symbols or only advertising

Summarizes the example system types in terms of mobility, embeddedness and visual interaction.

Using the classification of Section 4.1, the system-type examples are visually categorized in Figure 2:

4.3. Companion System Types in Public Transport

Starting from these example systems, we can upgrade them companion-wise and generalize them to types of VICUPIS (see

Table 2. Generalized system types.

System Type	Mobility	Embeddedness	Visual Interaction Focus
Mobile Personal Companions	High	Low to Medium	Personal, dynamic, mobile-friendly rather small touch displays, with graphic and text output
Public Contextual Displays	Low	High	Shared, robust, highly visible content often graphical (maps, etc.)
Vehicle-Embedded Ambient Companions	Medium	High	Ambient information surfaces, with symbolic, text or graphic UI, integrated in vehicle
Ubiquitous Companion Systems	High	High	Cross-device adaptive interaction and personalization, rich UI components

) already found in this form in public transport, or at least to their predecessors above.

Mobile Personal Companions operate on individual users’ mobile devices such as smartphones, tablets, or smartwatches. These systems provide personalized, (often time/location) context-sensitive travel information for individual users, including real-time navigation, connection alerts, (re-)planning and disruption notifications, typically on personal devices. They are minimally infrastructure dependent, but highly tailored to the user’s preferences, travel history, agenda (DynAPSys project), and situational factors (e.g., current mode of transportation). They allow continuous access to assistance services throughout the journey, across vehicles, stations and even transport providers. The visual interaction is optimized for small displays and mobile usage situations, often using eye-catching, non-intrusive designs.

Use cases include smartphone apps for commuters that provide location-aware, real-time information links with re-routing, or a smartwatch app with haptic notification for interchange and gesture-based stop requests.

Public Contextual Displays (e.g., digital signage, wall/floor projections, ticket vending machines, in-vehicle displays) are fixed installations within the transport infrastructure, such as at stations, stops or in vehicles. They are designed to present relevant service information, real-time updates, and navigation assistance to all users at a given location. These systems often support group-based or passive interaction, with a focus on clarity, accessibility, and immediate visibility under varying environmental conditions (e.g., sunlight, crowds). Some displays provide limited active interaction, such as touch input on ticket machines or wayfinding kiosks. Visual interaction must address a broad and often heterogeneous user group, focusing on robustness and clarity rather than deep personalization, also due to a lack of personal information or privacy issues. Use cases include arrival and departure information on a screen at a bus stop, including delay and detour information dynamically based on live data; wall projections in large stations that guide users through complex buildings with many platforms by highlighting paths; and low-fidelity color displays in infrastructure to announce vehicle status, such as color indications of crowding levels in Stuttgart, which already exist in practice.

Vehicle-Embedded Ambient Companions (e.g., Smart Windows) are integrated into the interior infrastructure of transportation vehicles such as trams, buses or trains. These systems present real-time ambient visual information and interaction directly in the passenger’s environment. The goal is to enhance situational awareness (e.g., next stops, route progress, service changes) using visual modalities that passengers can passively perceive while traveling and that augment the physical environment. The embedded nature ensures reliability and consistency across the transit fleet and integration into the mobility system. The visual interaction design emphasizes ambient information delivery: low cognitive load, high visibility, and subtle integration into the physical and experiential space of the vehicle, sometimes enhanced by AR, gesture, and similar concepts. Use cases include a transparent smart window that displays upcoming stops, maps, estimated transfer times, and landmark icons that augment the real view outside. Light strips embedded in the window frame or floor can additionally guide passengers.

Ubiquitous Companion Systems (e.g., Mobile-Device-Connected Smart Windows) connect personal mobile devices with embedded public infrastructure and vehicle systems as well as data from background systems to create an integrated, seamless experience and ubiquitous systems. These systems use cross-device communication—e.g., a smartphone interacting with a smart window or a station display—to dynamically extend, personalize, or adapt the public interface based on user-specific context with shared data and services. They combine the high embeddedness of fixed systems with the mobile continuity and personalization of and by personal devices, allowing information and services to “follow the user” across modes, stations, and vehicles. Visual interaction is hybrid and flexible: it can move between public and personal displays, maintain personalization without compromising privacy, and adapt to the infrastructure available at each location. Use cases include the user’s smartphone and travel companion synchronizing with a smart window in the train (SmartMMI project) to show only their personalized route and next transfer stop, including the path to the final destination and options to take travel information with them when they leave the train, or a station projection and speakers that adapt signage depending on the detected user profile (e.g., barrier-free paths and elevator information for accessibility needs) as an integration of the mobile device, network infrastructure and vehicle/public information system. The four system types discussed are shown in Figure 3 with their relative mobility and embeddedness values after [30].

4.4. Visually Interactive Companion Technologies in Ubiquitous Passenger Information Systems (VICUPISs)

The system types described above—Mobile Personal Companions, Public Contextual Displays, Vehicle-Embedded Ambient Companions, and Ubiquitous Companion Systems—all share essential characteristics that qualify them as Visually Interactive Companion Technologies in (Ubiquitous) Passenger Information Systems—VICUPISs. Together, they represent a distinct and evolving class of systems that support passenger interaction with mobility services in a visual, adaptive, and context-aware manner.

VICUPIS are embedded in complex socio-technical environments that span vehicles, stations, mobile devices, and digital infrastructure as well as passengers, personnel and other system stakeholders. Unlike traditional user interfaces, they are not limited to static screens or isolated applications. Instead, they operate across devices, phases of travel, and situational contexts, addressing dynamic passenger needs and environmental variability.

VICUPIS are digital systems designed to support and guide passengers throughout their travel chain by providing visual, interactive, and adaptive information across public and personal platforms. They should perform the following:

• Provide real-time, context-sensitive and correct information;
• Support multimodal interaction;
• Enhance situational awareness, orientation, and confidence;
• Adapt seamlessly across devices, locations and travel situations;
• Integrate different systems, data sources, technologies, and organizations,

• requiring sustainable, evolving systems in contrast to existing system monoliths and short-lived apps.

VICUPISs share the following characteristics—identified in the different system types above—to different degrees:

• Focus on Visual Interaction: Interfaces are designed for clarity, visibility, and immediate comprehension—accommodating mobile, noisy, and socially diverse public transport contexts. Visuals are often combined with haptic or auditory cues in multimodal ways and optimized for screen size, distance, and environment.
• Ubiquity and Seamlessness: VICUPIS operate across personal, system, and public contexts, connecting mobile devices, vehicle interfaces, and station infrastructure with systems in the background. They support continuous access and interaction across the entire mobility chain.
• Context-Awareness, Adaptivity, and Personalization: These systems actively integrate real-time contextual information—such as location, transport mode, crowding levels, weather, or passenger profiles—to adapt content, prioritization, and interaction modality accordingly. Interfaces respond to passenger roles (e.g., commuter, tourist), preferences (e.g., accessibility needs), as well as cognitive states and limitations (e.g., distraction, stress). Personalization ranges from discreet alerts to full route and transfer guidance.
• User-Centricity: Due to their deployment in heterogeneous, dynamic, and socially diverse environments, VICUPISs must be designed through user-centered processes that integrate real-world passenger needs, behaviors, and accessibility requirements as well as organizational and technical restrictions of CPSs. Their sustainability, acceptance and success depend on user-centered, iterative design cycles that incorporate contextual user studies, user-centered design practices, and evaluation under authentic conditions to ensure usability, trust, and adoption across all potential user groups and usage scenarios.

4.5. From Interfaces and Apps to Ubiquitous Companions: The VICUPIS Paradigm

The distinguishing characteristics of a VICUPIS that set it apart from conventional interfaces extend beyond its physical location and form factor. Rather, its distinguishing features lie in its role as a companion system that follows, supports, and adapts to passengers throughout their journey. Mobile Personal Companions, Public Contextual Displays, Vehicle-Embedded Ambient Companions, and Fully Ubiquitous Companion Systems already represent an ecosystem comprising interactions, information, mobility experience contexts and systems—typical for ubiquitous systems. For instance, a passenger may receive an alert on the smartwatch and a coherent message via loudspeaker announcement, subsequently confirming it via a smart window, carrying the plan on the smartphone when leaving the train (and following signage) [20].

This also demonstrates the high value of context in VICUPIS scenarios from design and development to operation.

5. Understanding and Modeling Context for Development of VICUPIS

As public, ubiquitous, and adaptive systems, Visually Interactive Companion Technologies in Ubiquitous Passenger Information Systems (VICUPIS) are inherently context-based. Their effectiveness relies not only on delivering relevant information but also on understanding the passengers’ current situation, environment, and interaction capabilities. Accordingly, the ability to identify, interpret, and respond to contextual conditions is central to the design and development of these systems.

Recognizing VICUPIS as a unified class of visually interactive, adaptive technologies provides a crucial foundation for the following:

• Developing systematic, context-driven design methods that reflect their multimodal and ubiquitous nature;
• Supporting modular and scalable architectures in which components interoperate across infrastructures and transport modes;
• Enabling seamless and inclusive user experiences that adapt to passengers’ needs in dynamic, real-world mobility contexts.

Thus, understanding, modeling and using context is not only a supporting activity—it is a prerequisite for expert- and user-centered development processes. The following context dimensions in

Table 3. Context dimensions and their role in VICUPIS design (summary).

Context Dimension	Key Role in VICUPIS Design
User	Personal preferences, accessibility aspects, emotional state, capabilities, personal agenda
Spatial	Location, relative location, in-vehicle, distance
Temporal	Urgent updates, schedule, plans, agenda
Task	Travel phase-dependent support (planning, boarding, waiting, alighting, change), user tasks
Physical	Environmental adaptation, lighting conditions, motion, weather, vehicle type and infrastructure, Wi-Fi access
Socio-Technical	Social appropriateness, operational status, privacy, security
Interaction	Preferred and available modalities, input/output channel selection, visualizations
Smart Mobility	Multimodal journey awareness, system integration along travel chain, mobility preferences and adaptation

, based on [31], should be used in the whole development process as well as in the VICUPIS developed alike.

5.1. User Context

The user context encompasses individual characteristics such as identity, role, behavioral preferences, and personal mobility patterns. The system should differentiate between frequent commuters, tourists, and passengers with specific accessibility needs, e.g., based on mobility personas like [32] or using automatic acquisition of user preferences [33]. The context model should also incorporate cognitive elements, including attention span, emotional state, and stress level, which have been demonstrated to significantly impact interaction timing and modality in mobility as seen for autonomous vehicles [34]. Adaptive systems are required to respond in a manner consistent with the user’s needs. For instance, during periods of elevated cognitive load, these systems should present simplified information and easily recognizable symbols.

5.2. Device Context

The concept of “device context” pertains to the physical and technical characteristics of the interaction platform, which may include a smartphone, an embedded display, or a wearable device. Factors such as screen size, sensor availability, battery state, orientation, and mobility (e.g., handheld vs. mounted) influence the type and complexity of visual interaction. To illustrate this point, consider the difference between a smart window in a tram, which can display more detail over time, and a smartwatch, which must present information in a concise and glanceable manner.

Device context facilitates the integration of personal and public devices in hybrid usage scenarios, including also the context of the device itself. This multidimensional perspective—also explored in research projects like DYNAMO and contextual storyboarding work at TU Dresden [35]—enables VICUPIS to tailor visual and interaction behavior to match the momentary and individual conditions of passengers and adjacent devices in complex transport environments.

5.3. Spatial Context

The spatial context encompasses both absolute and relative positioning, such as GPS coordinates or proximity to station infrastructure or vehicles. VICUPIS can leverage this context to trigger spatially relevant actions (e.g., geofencing, device near-by), such as displaying boarding instructions when approaching a platform. Interfaces have the capacity to modify visual density and size in accordance with the physical distance of the user from the display. This adaptability enhances accessibility and responsiveness, exemplified by the ability to display large symbols at a distance and detailed information at closer proximities. There is also a reason why early context-adaptive systems have often focused on location for context-adaptivity.

5.4. Temporal Context

The concept of time assumes a pivotal role in the context of public transportation. The temporal context encompasses the current time of day, scheduled itineraries, and real-time updates, including delays or early departures. Users’ personal temporal limitations, such as connection windows, are also part of this dimension. VICUPIS have the capacity to adapt to the urgency and visibility of content in real time. In high-pressure moments, it can highlight messages that are critical for establishing connections and reduce nonessential information.

5.5. Task Context

The task context is indicative of the user’s current phase in the mobility process, which may include trip planning, waiting, boarding, riding, transferring, alighting, or managing disruptions. Each stage of development exhibits distinct interaction requirements and attention capacities. For instance, in-trip guidance should be glanceable and minimal, while planning phases may support more detailed interfaces. It has been demonstrated that recognizing task context facilitates seamless transitions and meaningful information filtering.

5.6. Physical Context

The physical context dimension comprises physical environmental conditions that affect the use of VICUPISs, including but not limited to lighting, movement, vibration, sound, and weather. These factors impose limitations on both perception and interaction. VICUPISs have the capacity to, e.g., modify display brightness or transition between different modalities (e.g., from visual to audio output) as a reaction to conditions of bright sunlight or reduced visibility or perceptibility in general.

5.7. Socio-Technical Context

This dimension encompasses both social settings, such as crowding, the presence of others, and the need for privacy, and technical–operational conditions, including service status and device availability. To illustrate, shared displays may be configured to avoid displaying personalized data to prevent “shoulder surfing” [21] or unintended disclosure. From a technical perspective, information may be subject to adjustment in accordance with system status, such as vehicle occupancy, deviations or events. Including socio-technical context factors ensures, e.g., the relevance of content and the protection of privacy for users.

5.8. Interaction Context

Interaction context is defined as the momentary state of the interaction itself, which encompasses user attention to the output, input/output capabilities, and situational restrictions for interactions, e.g., offering interaction possibilities only in certain steps of the interaction process, like ticket machines accept money only in the payment step. For instance, cyclists may not engage with touch screens, necessitating alternative forms of feedback such as passive, voice-based, or simple haptic interfaces. Interaction strategies can be adapted based on a variety of factors, including engagement levels, hand availability, and even the user’s posture or gaze direction.

5.9. Smart Mobility Context for VICUPISs’ Domain Specificity and Contextual Design

This context dimension includes multimodal journey awareness, system integration along travel chain, as well as mobility preferences and adaptation.

Finally, the smart mobility context aims at the integration of real-time operational data and multimodal (transport) system knowledge. The latter encompasses transport modes, transfer points, delays, service types and service disruptions. VICUPISs employ this domain-specific layer to facilitate journey management, decision-making processes, and dynamic rerouting. In the event of delays, systems have the capacity to suggest alternatives, highlight intermodal opportunities, or provide continuity of guidance across different vehicles and providers. This is especially important in VICUPISs, because public transport comprises many unique systems and stakeholders that need to be taken into account and integrated to sustainably deliver transport services with a high-mobility experience.

5.10. Usage of Context Models

The proposed context dimensions and upper-model elements function not only as a theoretical foundation but also as a practical design and development instrument for the entire lifecycle of VICUPISs. It provides support to developers, designers, and transport stakeholders in the structuring and completion of requirements, the selection of appropriate modalities, and the definition of adaptive behaviors across a range of usage scenarios. It can be basically used in the early and late steps of the development process like an extended check list for analysis and specification but also as a machine-interpretable model for context-adaptivity in public transport systems like is currently realized in the IADAPT project as a semantic context model [36], as shown in the example Figure 4 from [37].

Within the context of deployed systems, VICUPISs can employ this model to dynamically recognize and interpret context, thereby enabling personalized assistance that is tailored to passenger roles, preferences, and situational constraints. For instance, by identifying preferences of passengers [31] and patterns in the mobility behavior of individual users, the system can prioritize and plan for elderly passengers, suggest barrier-free access options, or reduce complexity for infrequent travelers. In this manner, context not only influences interaction. It also serves as a fundamental catalyst for the development of adaptive, user-centered mobility support. This, in turn, enhances the accessibility, responsiveness, intuitiveness and overall sustainability of public transportation for all users.

6. Methodology for User-Centered Development of VICUPIS

Building upon the system classification presented and the learnings from multidimensional context modeling introduced, we now propose a methodology tailored to the sustainable, user-centered development of Visually Interactive Companions in Ubiquitous Passenger Information Systems (VICUPIS). The structured understanding of system types (Section 4)—ranging from mobile personal companions to ubiquitous systems comprising private and public displays—and their shared characteristics as adaptive, visually interactive, and context-aware technologies provides a solid foundation for design together with results from our research projects that executed all of the steps mentioned in various combinations and for all the different VICUPIS types. At the same time, the ability to identify and operationalize relevant context dimensions (see Section 5) is essential for ensuring that these systems behave appropriately and meaningfully across varying mobility phases, environments, and user needs. Combining these perspectives, the following development approach integrates context-driven design thinking, iterative user involvement and focus, as well as multi-stage real-world validation to support the creation of usable, scalable and sustainable VICUPIS solutions.

6.1. User-Centered Design for Adaptive Passenger Information Systems

Notwithstanding substantial progressions in mobile and ubiquitous computing, prevailing development methodologies for passenger information systems are missing a comprehensive, user-centered approach that acknowledges the particular challenges posed by Visually Interactive Companions in Ubiquitous Passenger Information Systems (VICUPIS). Although general process frameworks for human-centered design (HCD) are well established—most notably in ISO 9241-210 [11]—they are rarely applied in a form that fully captures the dynamic, situated, public and multimodal nature of public mobility contexts [15,38].

As digital companions in public transportation become increasingly interactive, adaptive, and AI-supported, the development process has to cope with increasing complexity of orchestrating and user experience across heterogeneous environments growing accordingly [30,39]. Closing the methodological gap of design processes relying on laboratory studies and controlled testing, ill-suited to capture the unpredictable nature of real-world public transportation scenarios [22], is the focus of the VICUPIS development process. It aims to overcome fragmented solutions that, while exhibiting functionality in test settings, often lack the capacity to scale or adapt effectively in operational environments of public transport. A field-aware, user-centered, iterative design process is therefore the core that also integrates context modeling, multi-phase multi-type prototyping, and systematic validation under real-world conditions.

The VICUPIS development process is a domain-specific, user-centered approach that extends user-centric approaches like ISO 9241-210 [11] into a mobility-oriented design methodology. It is tailored to the constraints and potential of visually interactive adaptive systems in public transportation.

6.2. Companion Adaptivity and the Role of Context in Field-Centered Development

Adaptivity and AI enable systems to move beyond static interfaces, offering personalized, context-aware, and predictive information tailored to individual users, their current situation, and preferences. These companion technologies can dynamically respond to environmental change, behavioral shifts, or transport disruptions—for instance, by adjusting visual layouts based on time pressure, cognitive load, or stress. Modern VICUPIS are thus no longer passive displays or fixed-function apps. They act as adaptive visual companions, embedded into vehicles (e.g., smart windows), located at stations (e.g., digital signage), or carried by users (e.g., smartphone apps), tailoring interaction to physical, social, and technical circumstances [6].

However, this shift toward adaptive, AI-enhanced behavior also introduces new challenges for system design and usability evaluation. As shown in our prior work on adaptive public transport systems [40], classic usability methods—built around static tasks, predefined interfaces, and reproducible states—struggle to cope with the fluid, non-deterministic nature of adaptive systems. The increasing variability in system responses, shaped by user behavior, transport events, and learned preferences, makes it impractical to anticipate and test all possible outputs in traditional lab settings. This creates uncertainty around predictability, consistency, and accessibility—central principles of usability that are now harder to ensure.

Therefore, AI fundamentally transforms both the development and the usage of VICUPIS. These systems must not only learn and adapt but also must remain transparent and trustworthy, even when their behavior changes. This demands novel evaluation strategies, including simulation-based prototyping, user-in-the-loop experiments, and real-time feedback integration to validate adaptive features under realistic and evolving conditions—from conceptual and lab situations to field settings. Crucially, adaptivity must be rooted in human-centered processes, grounded in field evidence, and validated through user experience, not just system intelligence [38,41]. In response, the VICUPIS development approach integrates the core activities from ISO 9241-210—understanding context, specifying requirements, designing solutions, and evaluating outcomes [11]—but adapts them to the unique demands of public transport environments:

• Users are engaged once per iteration, rather than at both the start and end, to accommodate time constraints and reduce fatigue and operational blindness in mobility, situationally diverse contexts [42] and iterative development.
• The focus of user involvement shifts from exploratory research and modeling in early stages to evaluation and iteration in later stages of the overall process not only per iteration.
• Context is core but not static—it is understood as dynamic and co-evolving, shaping not only design input but also system behavior and performance over time, addressing context iteratively and all context dimensions completely in the development process.

Through the field-integrated approach, VICUPISs are developed not as closed, monolithic applications, but as modular, context-aware companions—adaptive to changing usage conditions, scalable across transport infrastructures, and grounded in authentic, human-centered user-experience field and lab research as well as system development.

6.3. Overall VICUPIS Development Process Approach

In order to address the inherent complexity and contextual variability of Visually Interactive Companions in Ubiquitous Passenger Information Systems (VICUPISs), a structured, user-centered development methodology composed of three iterative and interlinked clusters is proposed: Field, Lab, and Concept. These clusters function as overarching environments that guide the design process from real-world observation to theoretical modeling and back—each supporting distinct but interconnected activities for understanding, designing, developing and validating adaptive companion systems in the context of public transportation—see Figure 5 below.

• The field cluster entails the utilization of authentic environments, including trams, stations, and control center systems [36]. It facilitates the acquisition and evaluation of authentic user behavior, domain-specific constraints, and operational semantics directly within the public mobility system.
• The laboratory cluster provides a controlled yet realistic testing environment, encompassing physical mock-ups and immersive XR setups, where prototypes can be developed, refined, and evaluated in conditions not restricted by operational limitations and uncontrolled variables.
• The concept cluster is home to abstraction, modeling and conceptualization, supporting the formulation of interface designs, atomic interaction concepts, use cases, and system architectures. It aims at transforming empirical insights into generalized models and preparing conceptual and basic scientific approaches for design, technology and as a basis for later scalable implementation. This part has proven to be especially strong—and in itself possibly iterative—in research projects compared to focused development projects aiming at fast development.

Together, these three clusters form a bidirectional and iterative development logic, allowing transitions from reality to theory and back. This ensures that conceptual innovations remain grounded in real-world constraints while being methodologically robust and scalable.

6.4. Process Phases and Activities

The development process comprises five key phases associated with the three clusters (see Figure 6) described above, each of which integrates expert and user activities, progressing in both directions from Field to Concept and back from Concept to Field. This integrates empirical field research, controlled (lab) prototyping and evaluation, conceptual modeling, pre-deployment (lab) testing and real-world (field).

This iterative and layered structure fosters both different embedded levels of abstraction and user-centric process steps. The five phases are as follows:

• Field-in: Captures expert knowledge and user requirements directly within public transport environments. This includes, e.g., stakeholder input, ethnographic observation, system research and domain-specific vocabulary that reflect the constraints and semantics of operational mobility systems, resulting in a field-oriented set of requirements and constraints, building also on requirements of engineering approaches.
• Lab-in: Translates real-world insights into first low-fidelity prototypes and evaluates them through expert walkthroughs and user-focused methods such as focus groups and role-based scenario testing for systematic gathering of system and interaction requirements, also using virtual and real simulation environments, including physical prototypes and XR testbeds to facilitate safe, controlled and iterative exploration under varying viewpoints.
• Concept Phase: Synthesizes findings from the field and lab into formalized design representations, such as use cases, UI models, cognitive task flows, and interaction principles. Methods include card sorting, creative ideation, cognitive walkthroughs, and early-stage interaction prototyping. This phase also anticipates technical, legal, and platform-specific constraints for scalable deployment. It also allows technical and interaction research to be carried out leading to new and adapted concepts, (meta-) models and atomic interactions as well as specifications grounded in theory and methods. It also includes software development core actions.
• Lab-out: From concepts back to the lab, this phase tests mid- to high-fidelity prototypes and system components in controlled yet lifelike environments. This phase focuses on usability, robustness, and interaction under near-real conditions while avoiding the risk of disruptions and uncontrollable environments or variables typical for public systems in the field. Iterative feedback from users and experts drives refinement and final (prototype) system before public deployment.
• Field-out: Focuses on integration, validation, and sustainable transfer of VICUPIS into real operational ecosystems. This includes evaluating full-system prototypes in situ, navigating regulatory or infrastructural dependencies, and preparing for long-term adoption. The field test has proven very valuable for public transport information systems, especially with in-vehicle (e.g., SmartMMI, https://smartmmi.iiius.de, accessed 1 July 2025) and personal companion systems (e.g., DynAPSys). Outputs include (re-)design heuristics, reusable components, and deployment guidelines for broader implementation for research prototypes and field feedback and product finalization for product development.

Each of the phases is described in detail in Section 7, Section 8, Section 9 and Section 10 below with the user-centered and expert steps taken to accomplish each of the five phases.

6.5. Phase Transitions

With the phases defined, the VICUPIS methodology intentionally separates development into five interlinked phases with distinct roles and progression criteria, ensuring rigorous, iterative advancement from real-world understanding to viable system deployment as well as defined transitions (shown in Figure 7) between the steps and phases. Within each phase, transitions between the expert and user steps (in-phase transitions) are typically triggered by reaching analytical/study or prototyping saturation that satisfies the quality and functional criteria for expert or user steps to be accomplished.

Moving between phases (between-phase transitions) is governed by more formal criteria tied to deliverables. For instance, the transition from Lab-in to Concept occurs once low-fidelity prototypes and systematic requirements (derived from expert methods and user focus groups) yield a stable foundation of validated needs and early design insights as well as a requirements specification. Key deliverables marking this phase completion include consolidated requirement documents, scenarios, and initial interaction mock-ups, which are then formalized into more detailed use cases, cognitive task flows, and user interface models during the Concept phase. Similarly, transitions to Lab-out depend on the availability of interactive conceptual prototypes robust enough for controlled evaluation and a specification of interactions to be implemented, while moving to Field-out requires a near-final and field-ready system prototype based on the validations under simulated conditions in the lab phase and ready for in situ deployment.

Across all phases, clear and up-to-date documentation such as requirements and UI design specifications ensures that each subsequent phase builds systematically on empirically grounded and stakeholder-approved results without losing findings and decisions. This structured approach balances exploratory agility with accountable decision points, supporting reproducibility and sustainable development outcomes as well as a manageable process.

Table 4. Phase transitions, criteria, and deliverables in the VICUPIS development process.

Phase Transition	Criteria for Transition
Field-in → Lab-in	Collected user requirements via surveys, observations, ethnographic studies, studies/field analysis finalized
Lab-in → Concept	Validated low-fidelity prototypes tested with users and experts, key interaction and usability issues identified, requirements specification finalized, lab studies finalized
Concept → Lab-out	Formal models developed, technical and conceptual specifications finalized (validating needs and constraints), implementation into high-fidelity prototypes/system implementation ready for lab evaluation
Lab-out → Field-out	Usability, robustness, and performance confirmed in realistic lab studies, studies finalized, (field-)prototype stable for real-world deployment and field test
Field-out → Project end	Successful real-world validation under authentic operational and socio-technical conditions, insights and generalization of results (research projects) finalized, product finalized and deployed (product development projects)

shows the between-phase transitions and criteria for transition.

6.6. Integration with Human-Centered Design Process (ISO 9241-210)

The VICUPIS development approach is conceptually grounded in the principles of Human-Centered Design (HCD) as defined by the international standard ISO 9241-210. The HCD process specifies four key activities that must be addressed in any user-centered development lifecycle [11]:

(a)

Understanding and specifying the users and the context of use;

(b)

Specifying the user requirements;

(c)

Producing design solutions to meet these requirements;

(d)

Evaluating the designs against the requirements in a user-centered way.

In the context of VICUPISs, these activities are consistently embedded within each phase of the development process—Field-In, Lab-In, Concept, Lab-Out, and Field-Out. Each phase includes both an Expert Step and a User Step, with the HCD activities distributed across these roles.

The Expert Step in each phase typically addresses (see Figure 8) the following:

• (b) Specifying the user requirements, based on previously gathered data, domain knowledge, and modeling techniques—also using expert knowledge in UCD and the domains associated with the project—here typically public transport and technology.
• (c) Producing design solutions, including low- or high-fidelity prototypes, interaction concepts, and system models.

The User Step in each phase typically focuses on (see Figure 8) the following:

• (a) Understanding and specifying the context of use, e.g., through observation, ethnography, contextual inquiry, questionnaires, iterative personas and scenarios.
• (d) Evaluating the design, using methods such as usability testing, walkthroughs, or field observations that fit user groups and (here: public transport) domain.

This structured alternation between expert activities and user engagement mirrors the iterative nature and user involvement of HCD, while being adapted to the specific constraints and dynamics of the VICUPIS domain of transport systems with expert phases b and c surrounded by user involvement in a and d. The concrete role and steps of HCD in VICUPIS development process can be found in

Table 5. Summary mapping: HCD and VICUPIS.

HCD Activity (ISO 9241-210)	Role in VICUPIS Development Process	Typical Step Focus
(a) Understand and specify context of use	User observations in field and lab, contextual inquiry, focus groups, requirements gathering	User Step (early)
(b) Specify the user requirements	Planning of activities, domain modeling, synthesis, modeling, requirements specification	Expert Step (domain)
(c) Produce design solutions to meet requirements	Definition of system parameters, concept creation, prototyping, product development	Expert Step (technology)
(d) Evaluate the designs against requirements	Formative and summative usability testing, feedback, apply user heuristics, field test	User Step (late)

.

Agile and iterative concepts are embedded as per phase as defined in the VICUPIS development process, but it also leaves space for an in-phase iteration if necessary.

This practical adaptation in the VICUPIS methodology introduces a further reduction in user tasks to one integrated step per inner development cycle (phase) for more practical and compressed involvement. In contrast to the ideal HCD model, where users are separately engaged both at the beginning (to understand context and requirements) and end (to evaluate solutions), the VICUPIS approach consolidates user involvement into a single focused phase per cycle. This choice is driven by experience in the field, where the following hold:

• User involvement is logistically and contextually more feasible in a single, concentrated session per phase;
• The focus of user contribution shifts over the course of the project—from exploratory insights in early phases to more targeted evaluation and feedback in later phases (e.g., during Lab-Out and Field-Out).

This phased focus allows for efficient progression and transfer from evaluation (d) in a preceding iteration to context understanding and analysis (a) in early user steps of a following iteration, while optimizing the traceability and user involvement central to HCD over the cycles. Moreover, the expert activities ensure continuity in requirement specification and solution evolution across iterations, which has proven vital for success in VICUPIS development project in the past.

This hybrid approach preserves the core principles of Human-Centered Design while acknowledging the real-world constraints and dynamics of ubiquitous public transport systems. It ensures that Visually Interactive Ubiquitous Companion Technologies (VICUPISs) are developed not only with users in mind—but with users involved at key stages where their input is most fruitful.

6.7. Comparison with and Extensions to Human-Centered Design Process (ISO 9241-210)

The specific extensions and changes to the HCD process developed for the VICUPIS development process are described in

Table 6. HCD and VICUPIS development process in comparison.

HCD Activity (ISO 9241-210)	VICUPIS Development Process	VICUPIS Development Process Offers …
Iterative process, standard iterations	Iterative process with customized iterations	… phases that resemble the iterations, however they also define scope and content.
Domain agnostic	Domain specific	… tailoring to match VICUPIS development.
Four steps with user activities	Four steps mirrored and compressed	… four steps compressed into two user and expert steps per phase, still containing the tasks.
General user involvement	Specific user steps and expert steps	~
Number of iterations not defined	Different phases instead of iterations	… specialized iterations regarding scope.
General rules	Specific methods	… specific focus and methods per phase.
User focus	Field focus	… user, field and stakeholder focus.
Same iterations	Specific iterations	… mandatory per-phase and additional in-phase iterations where required.
End by Evaluation	End by field test/phase	… clear end-phase and criterium.

.

While based on HCD, the VICUPIS development process is tailored to the public transport and passenger information systems development scope and addresses some shortcomings of general HCD, e.g., regarding loss of technology and public domain focus and specialties of ubiquitous and context-adaptive systems.

In the following we establish the process with its five phases.

7. Field-In Phase: Capturing Real-World Requirements and Context

The first phase of VICUPIS development process starts with field-in, i.e., gathering information and planning from field perspective in public transport.

7.1. Phase Focus, Considerations and Specialties

The Field-In phase initiates the VICUPIS development process by grounding design activities in the authentic conditions of public transport environments. This phase focuses on systematically capturing the domain-specific semantics, operational constraints, and user behaviors that influence the design and function of the visually interactive companion to be developed. Situated in real trams, buses, stations, or platforms as well as the infrastructure and systems, the Field-In phase emphasizes contextual observation and user involvement to establish a realistic and scalable foundation for VICUPIS development [40].

7.2. Expert Step: Domain Knowledge and Operational Semantics

In the expert step, field-based knowledge from stakeholders—such as transport operators, planners, and system integrators—is gathered to define the operational and infrastructural landscape in which VICUPIS will be deployed. Activities include the following:

• Eliciting expert knowledge on regulatory frameworks, system interfaces, and passenger communication workflows;
• Defining domain semantics and vocabulary, e.g., terminology used by transport operators for services, disruptions, and routing, also as a basis for context modeling;
• Identifying system boundaries, critical dependencies, and infrastructure constraints, such as data availability, device compatibility, or platform-specific limitations and existing systems—hardware and software.

This step ensures that VICUPIS development aligns with the technical, legal, and organizational realities of public transport systems.

7.3. User Step: Observing and Understanding Passenger Behavior

The user step focuses on gathering passenger needs, behaviors, and pain points through a range of context-oriented methods, such as the following:

• Ethnographic observation (“ethnography”);
• Contextual inquiry;
• Structured surveys;
• Participatory observation.

For example, in one of our studies, passenger interactions with public displays at tram stops were recorded and analyzed to identify recurring challenges in information access and comprehension as well as impact of context factors and common interaction issues [43]. Similarly, mobile user needs were captured through in situ app usage studies, offering insight into interface expectations, usability under real-world conditions and specific mobile user needs [44].

These observations inform user modeling and requirement analysis, helping to establish realistic design goals and system expectations early in the process.

7.4. Case-Study: SmartMMI Research Project and Use of MobiDiary Project

We introduce and use SmartMMI project as a case study for the development process, as it was the first project to follow the methodology. The SmartMMI project developed and evaluated a Smart Window for Tramtrains and illustrates the challenges and opportunities of the VICUPIS development process and its phases in the frame of a concrete project.

In the Field-In phase of SmartMMI we conducted field studies in real public transport vehicles, facing constraints typical for CPS: limited deployment flexibility, shared user interfaces, and regulatory barriers. Additional tools such as 360° cameras and mobile eye-tracking were used to collect attention and interaction data under real conditions [45] from the field.

As an example tool to be used in the “user” step, our MobiDiary app and system offers a context-aware mobile app replacing paper-based travel diaries with automated and user-augmented data collection [46]. Key contributions for use in Field-In phase included the following:

• Multimodal context capture: Automatic logging of mode, location, weather, and headphone use enhance user modeling and provided rich data for SmartMMI personas, scenarios and typical mobility settings for the users of the SmartWindow.
• User involvement in decision points: Users can annotate when and why they made decisions (e.g., route changes) on trip, enriching data with situational user context.
• Cross-validation: Combined manual and automated data (e.g., Google Awareness and Places APIs) improve reliability and provenance, i.e., better field data quality.
• Scalability: Enabled efficient, semi-automated large-scale data collection—even under pandemic conditions experienced—with early-stage feasibility without paper-based diaries and their adjacent ineffective and inefficient processing.

MobiDiary helped us in user research for the early-phase field requirement analysis. In our mobility behavior analysis, we compare collected mobility data from October 2018 being collected before the corona virus pandemic and October 2020, collected after the outbreak of SARSCoV-2. In both measuring periods, students were asked to record their daily mobility over a two-week period with our mobility evaluation tool “MobiDiary”.

The mobility dataset from 2018 comprises records from 38 students, capturing a total of 308 trips that together cover 4954 km and over 89 h of travel time. A high density of movement is evident in the areas around the campus and associated outposts. In 2020, the number of participants slightly increased to 41. However, the amount of data collected declined significantly, by more than 50%, to just 150 recorded trips. The total distance dropped by approximately 63%, amounting to 1760 km over 74 h of travel time. This substantial reduction can be attributed to the transition to online and remote teaching formats, which largely eliminated the need for students to physically attend lectures on campus.

The eye-tracking study was conducted with six media communication experts from the International University Karlshochschule (five female, one male). Participants were divided into pairs and calibrated with eye-tracking glasses. Each pair was then asked to sit in front of the SmartWindow prototype, describe their initial impressions, and subsequently perform a series of tasks while verbalizing their thoughts [25].

7.5. Findings, Practical Take-Away and Heuristics

The Field-In phase provides actionable insights for both expert-driven system framing and user-centered requirements gathering. Key findings and heuristics include:

• Personas as Bridging Tools: Based on prior work (e.g., [43,47]), public transport personas [48] proved essential and very helpful in aligning expert and user perspectives, and also as a vehicle for participation of users and domain experts. Personas helped clarify typical user types based on usage frequency, system familiarity, and smartphone behavior, as well as serving for scenarios and as a basis for multi-disciplinary discussion throughout the whole process, enabling targeted design decisions [40].
• Low barriers for user engagement: Low-effort, targeted feedback channels (e.g., info steles, simplified surveys) yielded better response rates than more complex participation apps—even when incentivized—unless embedded in motivated environments such as student coursework. User-centric Field-in approaches for gathering information often yielded very valuable feedback when barriers and efforts for participating were as low as possible, e.g., in real lab Go Karlsruhe we developed an app, a web tool and a physical stele—i.e., an interactive “poster” with a visually supported question and lo-fi interaction via buttons presses that were transmitted via mobile radio. The stele feedback outperformed all fully digital and more complex tools, where web was better than app for user engagement, even with prizes and cooperation with a broadcast station for app usage improvement.
• Hybrid methods are essential for data quality: The combination of automated tracking and qualitative annotation improves both data quality and insight depth. Error tolerance mechanisms must be considered early (e.g., forgotten trip ends, API misclassifications).
• Context in the field has great influence on user results: MobiDiary revealed that device context (smartphone vs. public screen) and socio-technical context (shared space, data privacy) critically influence interaction preferences [43] and information design.
• High potential impact on users improves engagement: When people were highly affected—e.g., in a study with disabled people in the city of Bogota [48], expecting improvement of their living conditions—user participation and feedback were much stronger than with typical academic user studies for requirements—with the exception of students running them as part of their studies with MobiDiary, which also led to more and better results, being part of their study work, but might have a bias.

These findings emphasize the central role of context and early, structured user engagement in sustainable design of VICUPIS. Field-In is not only a requirements-gathering phase—it establishes the semantic and mobility-experiential foundation for the entire development process and success in public transport.

8. Lab-In Phase

Entering the lab from the field, the steps of Lab-in phase connect the contextual grounding from the Field-in phase and the abstract modeling of the Concept phase with strengths and aspects of both. It focuses on transforming real-world observations and constraints into tangible, early-stage system requirements, concepts and low-fidelity prototypes, which can be explored, evaluated, and iterated in a controlled, yet realistic, environment. In the VICUPIS development process, this phase plays a crucial role in enabling safe experimentation, collaborative ideation, and early user involvement without the technical, logistical and legal limitations of public deployments in the field but providing an environment not as theoretical as in typical specification phases in classical software and systems development.

8.1. Phase Focus, Considerations and Specialties

The Lab-in phase enables the exploration and evaluation of advanced interaction and system concepts derived from field insights often based on lo-fi user interface prototypes, while still being abstract enough to allow flexible design variation. It allows developers and researchers to test new interface modalities (e.g., augmented reality, multimodal displays) in simulated or semi-physical setups—such as mock tram stops, XR environments and other laboratory prototypes (see example of in-train Human–Robot Interaction in Figure 9).

This phase is particularly well-suited for testing radically new or unconventional ideas that would be risky or impractical to trial in live transport settings but have to be assessed and decided early in the project.

For instance, in our research on AR passenger information on mobile, transparent public displays [43], lab simulations enabled us to assess visibility, interaction comfort, and contextual relevance of augmented information long before any real-world deployment and test took place.

Similarly, passenger information in disruption scenarios was explored in a lab-based user study [49], yielding valuable insights into user expectations during stressful and ambiguous travel moments.

Also, focus groups play a major role in this phase for integrated user and domain expert involvement (see example focus group setting in Figure 10).

8.2. Expert Step

The expert activities in the Lab-in phase involve translating the findings from Field-in into specifications, concepts and prototype artifacts in a structured way as well as creating new ideas and approaches for solutions. The expert activities include the following:

• Requirement specification based on field findings.
• Context modeling, also based on review of contextual factors.
• Preparation of user-centric activities like focus groups.
• Designing low-fidelity prototypes, such as wireframes, storyboards, cardboard models, or static screen flows, based on the usage patterns and requirements gathered in real mobility contexts, as well as further development of ideas stemming from these.
• Embedding of real-world constraints (e.g., limited space, noisy environments, shared usage, luggage, social restrictions) into lab setups to ensure validity.
• Creating testable design hypotheses, grounded in context models and personas for early tests in the user phase, targeting conceptual processes for the concept phase.

Public transport experts, system designers, and researchers use this phase to synthesize semantic knowledge (e.g., signage conventions, service logic, contextual restrictions) into study setups that can be understood, extended, iterated and evaluated with stakeholders and potential users.

8.3. User Step

The Lab-in phase’s user step introduces real users and stakeholders into the discussion, creation of interactions and evaluation of early concepts and system behaviors. Typical activities include the following:

• Engagement of diverse user groups, such as tourists, elderly passengers, visually impaired individuals, and transport staff, to validate inclusion aspects and contextual fit as well as focusing sustainability by design and longevity design of systems from user perspectives.
• Use of focus groups, scenario and cognitive walkthroughs, and Wizard-of-Oz simulations to explore VICUPIS behavior in real-world situations.
• Deployment of mock components and stations, tram stop and tram replicas, and extended reality (XR) environments to ensure spatial and experiential realism of the future VICUPISs and reality-oriented interaction for sustainable as well as conceptually and technically optimized system design with focus on developing and testing shall-requirements.

These lab-based activities allow for early detection of usability issues, detailed requirement analysis, uncovering of implicit user expectations/implicit requirements, and the refinement of design directions before high investment in full system development and deployment as a sustainable, long-term solution.

8.4. Case-Study: SmartMMI Research Project

The SmartMMI project case study exemplifies Lab-in practices through its systematic testing of visual user interface components and Smart Window interaction constraints such as arm reach and personal areas. Core activities included the following:

• Grip radius analysis for interactive screens embedded in smart windows, accounting for reachability and ergonomic limitations within vehicles as well as social restrictions for cooperative and parallel interaction, already in the Lab-in phase with first prototypes. All participants were asked to report their body height, which ranged from 159 cm to 186 cm. All participants were right-handed and between 20 and 34 years of age. The study was conducted in two iterations. In the first iteration (Part A), 18 participants (n = 18) took part without any prior knowledge of the SmartWindow system or its intended purpose. The mockup designs and interaction methods were unfamiliar to them. In the second iteration (Part B), 13 participants (n = 13) were provided with preliminary information about the system. They were shown conceptual sketches of the SmartWindow and were allowed to interact with an early-stage prototype in order to develop a basic understanding of its functionalities. Following this introduction, they performed the same set of tasks as the participants in Part A [25].
• Early design of graphic elements such as icon design and active (mainly touch) and passive (mobile, context-driven) interactions, also including student works and developments and evaluations of specific aspects executed by different lab groups. Approximately 250 individuals from diverse age groups, with varying levels of experience in using public transportation and differing degrees of affinity for new technologies, participated in the conducted online survey. The data collected provides valuable insights into the heterogeneous needs and preferences of different passenger groups [25].
• Lab usability testing, where alternative visual representations for service states, crowd levels, or upcoming stops were compared under lab conditions with user tests and using heuristic evaluation and structured user feedback During the evaluation, thirteen participants interacted with the SmartWindow prototype by completing predefined tasks. Following the task completion, they filled out a questionnaire evaluating the usability and perceived likelihood of using various interactive components and modalities, including some not implemented in the prototype. While modalities such as voice input, gesture control, and smartphone-based interaction were largely perceived as unintuitive or overly futuristic, multi-touch interaction was rated as intuitive and more likely to be used [25].

These experiments informed both the physical design of SmartMMI’s smart window display systems, and the visual/interaction grammar used across multiple display types, like the upper passive (far, standing) and lower active display (low, sitting close) areas. We also used several different evaluation methods. Public transport experts were involved in the development of our personas and requirements as well as regular passengers, e.g., from the passenger association, in lab-based evaluations [50] as can be seen from Figure 11, and groups discussions. Applying our user-centered VICUPIS design process, we also used several different evaluation methods [43].

8.5. Findings, Practical Take-Away and Heuristics

The Lab-in phase provides an essential space for risk-free exploration, participatory design, and practical validation of conceptual and field requirements. Key insights from our work include the following:

• Low-fidelity is powerful: Even simple mock-ups, when situated in realistic environments (e.g., a mock tram), are sufficient to provoke deep feedback from users and stakeholders—and offers fast evaluation of alternatives.
• Scenario-driven testing improves relevance: Simulated disruptions, transfer stress, or crowding scenarios are particularly useful in gathering context-specific interface and content needs.
• Stakeholder engagement ensures domain alignment: Involving transport experts in the prototyping process helps maintain consistency with infrastructure, regulations, and service logic—but requires artifacts and discussions to be less technical in terms of domain-specificity, like for wording or (corporate) design in visuals and prototypes.
• Visual interaction needs iterative validation: Icon choices, layout complexity, and multimodal combinations (e.g., visual and audio cues) require early and continuous formative user testing to avoid ambiguity and cognitive overload.

In summary, the Lab-in phase supports rapid iteration, structured exploration, and grounded design refinement, positioning VICUPIS prototypes for robust conceptual modeling and validation in the next development phase.

9. Concept Phase

The Concept phase represents the transition from empirical observation, lab and prototyping to formalization and abstract modeling focused on new and rather atomic concepts—and then backwards to lab and field. Based on the insights and validated ideas from the Field-in and Lab-in phases, this phase focuses on developing structured interaction concepts, user interface models, and adaptive behaviors within a context-rich but constraint-free design space. Especially in applied research projects, this is a core and often iterative phase. By combining expert design activities with cognitive and perceptual user studies, the Concept phase enables thorough exploration of interaction logic, user expectations, and visual communication strategies—setting the stage for advanced prototyping, product development and integration with strong concepts at hand.

9.1. Phase Focus, Considerations and Specialties

The Concept phase allows teams to investigate fundamental interaction principles, cross-device logic, and user experience questions beyond the physical limitations of the field or lab environment. Here, VICUPIS concepts can be freely modeled, visually designed, and user-tested in the form of paper prototypes, flow diagrams, and compact simulations for specific aspects. Studies such as [40] on adaptivity in public transport or iterative evaluations of AR-based passenger information concepts [43] are typical examples, where subtle interface and perception mechanisms are studied for cognitive compatibility and potential. This includes development of conceptual models for the UI, interaction design, and system specifications including more formal artifacts like use-case diagrams, activity diagrams, wireframes, semantic models, machine learning approaches and lo-fi prototypes for critical or innovative interactions.

9.2. Expert Step

In the (first) Expert step of the Concept phase, experts consolidate the findings from earlier phases into formal artifacts and structured design concepts. Activities include the following:

• Development of conceptual UI and interaction models like wireframes, InteractionCases [51], informed by personas, task analysis, and usage contexts;
• Software Engineering artifacts like the creation of system specifications, such as use-case diagrams and activity diagrams, architecture and class diagrams, communication models, or device roles and interface/API definitions;
• Development of semantic models like integration and extension of upper ontologies, OWL [52] machine-interpretable formalized domain and context models;
• Generative models and adaptation models;
• Algorithm design, interface and API definitions;
• Conceptual prototypes of critical interactions and interface elements, also for validation in the following User step.

This step ensures that all further development is anchored in a sound conceptual foundation, anticipating technological integration and platform constraints as well as new technology and research findings.

9.3. User Step

The user step in the Concept phase applies user-centered design techniques to evaluate, compare, and refine conceptual models. The aim is to explore cognitive usability, user understanding, and acceptance of interaction principles, even before building interactive prototypes. Common methods include the following:

• Cognitive walkthroughs for validating interaction process logic and predictability of interactions by users;
• Card Sorting for testing, e.g., naming conventions, icon sets, auditory icons, or functional mappings;
• Creativity workshops for participatory idea generation of atomic concepts where applicable, e.g., again, focus groups;
• A/B testing or preference studies to assess layout variations, visualizations, adaptivity or personalization strategies;
• In the multi-device environments [50], e.g., users interacted with concepts for cross-device interaction between SmartWindow and smartphone, revealing critical insights into pseudonymization, privacy control, and information handover strategies.

To maintain the more conceptional and basic focus of this phase, activities are focused more on atomic and generalized aspects and single aspects and components than in the phases before. The core goal of this phase is to create a sound technical, interaction and conceptual foundation for the rest of the project, based on the earlier phases and then implemented into the following ones. Typical Usability Evaluations focus rather on whole prototypes, full user interfaces and products, whereas the concept phase’s User Step offers a deep dive into basic concepts, UI innovations and more scientifically grounded research.

9.4. System Prototypes (Expert) Step

Once conceptual designs are validated, the expert team creates interactive, lab-executable prototypes that are still quite free from field constraints. These prototypes allow early exploration of, e.g., the following:

• Adaptive interface behavior across devices and situations;
• Complete User Interface design for a device or the whole application;
• Ubiquitous System designs;
• Personalized and context-sensitive content delivery;
• Transition scenarios between public and private spaces, including privacy designs;
• Full-system implementations in product development projects can already take place here.

Prototypes or core systems are built on accessible and standardized platforms with an eye on final product and field, allowing for rapid modification and exploration of interactions in simulated mobility contexts. Building an Evaluation Environment for Public Transport [26], we have designed a platform to host such prototypes in a modular and scenario-driven way.

These prototypes not only serve as proof of concept but as testbeds for design decisions, informing visual structure, timing, and feedback behavior as well as already implementing system designs, software architecture patterns and holistic UI design.

Still interactive, lab-executable prototypes can be designed here without the field’s hardware/legal constraints, in a controlled, adaptable environment without impact on production systems and public transport field restrictions. The focus lies on the concept and its realization in a physical but controlled environment allowing for a full-fledged prototype on a cheap and standardized target [26].

9.5. Case-Study: SmartMMI Research Project

In SmartMMI, the Concept phase was marked by the abstraction of findings from field and lab studies into generalized interaction patterns and display design. As the technical system design and hardware was completely new for public transport companies as well as technology providers and authorities, a hardware setup, basic system as well as huge and standardized tests, like vibrations and pressure on the window hardware while other trains passing by, had to be planned and executed, making it necessary to fix hardware concepts at this stage to have enough time for change, integration, tests and field preparation. Also, regarding different stakeholders and software/system development, we executed expert reviews and stakeholder co-design workshops to assess concept feasibility and iteratively create a system version for the following Lab-out phase’s User step. This also included re-iterating some of the Lab-in Tests with the new concepts.

These activities ensured that prototypes for SmartMMI’s Lab-out phase were not only technically feasible and based on the specified concepts like areas of interaction but aligned with cognitive usability, domain logic, and passenger needs, e.g., symbols, visualizations, screen contrast, gestures for interaction, and ease of physical interaction when sitting next to SmartMMI’s Smart Window, etc. This included pseudonymization concepts for privacy and combining Smart Window and smartphones conceptionally in multi-device interactions [50] and adjacent studies, but also conceptual clarification, e.g., on the reach of the users for touching the SmartWindow elements in the new position easily and how their personal space is affected by this new system concept, as can be seen in Figure 12.

An empirical user study with 16 participants implemented a multi-device evaluation methodology combining mobile devices (smartphones, smartwatches) and public displays to assess adaptive passenger information systems, with particular attention to maintaining privacy through visual pseudonym linking. Three pseudonym approaches were tested in a user study to evaluate how clearly and acceptably passengers could connect personal and public-device notifications. Results demonstrated that participants generally accepted the multi-device setup and intelligibility of the pseudonyms, offering valuable insights for designing privacy-aware, context-adaptive information systems [50].

The research employed a four-phase iterative evaluation methodology, beginning with two online studies to identify promising AR content for semi-transparent SmartWindow displays, followed by a lab-based user study to refine user scenarios and interaction designs. Finally, a video-based online survey with 109 participants evaluated specific AR design parameters—such as animation speed, color, fade effects, and information types—revealing user preferences and acceptance levels for different AR configurations. This iterative, multi-modal approach allowed the authors to systematically optimize AR content based on empirical feedback and usability insights [43].

The study evaluated a prototype system combining interactive smart displays in tram compartments and a smartphone app to deliver real-time disruption information to passengers. A user study with 12 participants was conducted across three groups, comparing different combinations of notification modalities and devices during simulated travel scenarios involving route disruptions. Results showed that participants preferred receiving disruption information via smartphones but often dismissed notifications quickly, highlighting important usability challenges for designing effective passenger information systems [53,54].

9.6. Findings, Practical Take-Away and Heuristics

Adaptive passenger information has the potential to significantly enhance the mobility experience and represents a promising step toward smarter public transport systems. In [40], we examined the concept of adaptive passenger information and explored strategies to improve the intelligibility of adaptive features. To assess user acceptance, we conducted an online survey. Over a period of three weeks, 133 questionnaires were fully completed. The results of the study emphasize the relevance of context adaptivity in passenger information systems. Based on data from an online survey with 133 participants, we found that the acceptance of adaptive features strongly depends on the context in which the information is presented. This highlights the importance of tailoring adaptive systems to situational factors in order to enhance user acceptance and overall usability.

Key insights from the Concept phase include the following:

• Conceptual clarity is essential: A strong conceptual model supports both development and communication. Diagrams, personas, use cases, and simplified wireframes help align technical and UX teams. Often projects go for implementation and product too fast.
• Cognitive compatibility and atomic core interactions as well as wording and mental models must be tested early: Many issues with visual encoding (e.g., icons, colors) and interaction patterns can be resolved with paper prototypes and card sorting as well as HCI and cognitive psychology competence long before technical implementation.
• Core technical concepts like cross-device logic and interoperability must be modeled, not assumed: Smooth transitions between public and private devices (e.g., SmartWindow to phone) require careful handling of privacy, persistence, and context continuity for the broad user groups in public transport. Often, architectural and technological questions are shifted to later phases or assumed to be solved easily, while they should be defined and evaluated in the Concept phase.
• Constraint-free does not mean artificial/unrealistic but abstract and atomic: Prototypes should still simulate critical conditions (e.g., urgency, cognitive load) to reflect the realities of public transport usage but concentrate on the core aspects.
• User engagement with abstract artifacts is feasible: Even low-tech tools such as sketches, scenario stories, and mock interfaces can generate valuable user feedback when embedded in well-structured workshops, but for good reasons, the Concept phase has two expert steps surrounding user involvement.

In summary, the Concept phase transforms fragmented observations, ideas and low-fidelity trials into cohesive, adaptable system structures, components, models, specifications, architectures and interactions, preparing VICUPIS for high-fidelity realization and contextual deployment in the next (development) phases.

10. Lab-Out Phase

The Lab-out phase marks a critical turning point in the VICUPIS development process. It transitions the system from conceptual design via lab towards field-readiness, focusing on the evaluation and refinement of high-fidelity prototypes or product increments in simulated but controlled (lab) environments. The goal is to test advanced VICUPIS solutions under conditions that closely resemble real-world public transport usage—while maintaining the control necessary for structured analysis, testing and iterative improvement.

This phase enables rigorous testing of usability, user experience, robustness, and system performance, bridging the gap between design/technological concepts and practical deployment. It also facilitates final adjustments in visual, interactive, and technical components before entering live operational contexts.

10.1. Phase Focus, Considerations and Specialties

The Lab-out phase emphasizes pre-deployment validation in realistic (still lab) test conditions. Unlike early lab prototyping (Lab-in), this phase involves fully functional and integrated systems, complete with CPS components and near-production visuals, logic, and responsiveness. These prototypes are tested in laboratory setups designed to emulate real environments—e.g., mock vehicles, dynamic displays, or AR-enhanced stations.

This phase’s lab evaluations also include study activities like pre-testing evaluation methods for a field test concept [50] as a basis for the next and last phase: Field-Out.

10.2. User Step

In the Lab-out phase, the user step focuses on structured usability and user experience studies, performed with high-fidelity VICUPIS prototypes or first product versions in lab settings that simulate real-world public transport scenarios—at least for specific scenarios, parameters and environments. The advantage of this still controlled setting lies in the ability to perform the following:

• Recreate/simulate specific use cases (e.g., disruptions, transfers, crowded conditions);
• Employ measurement tools like cameras, sensors, or eye-tracking glasses;
• Test with (still) unavailable elements or for potentially dangerous settings, like in our current Human–Robot-Interaction project on transport robots in trains;
• Enable repeatable testing across diverse user groups and interfaces and stable environmental conditions.

Instruments and methodologies typically include standardized questionnaires, sometimes adapted to VICUPIS and similar systems and settings (see Section 10.5).

By combining such instruments, the Lab-out phase supports holistic system validation, covering both usability fundamentals and domain-specific experience dimensions towards the field but still in the development phase.

The execution of lab-based usability studies in environments resembling real public transport scenarios helps judge the outcomes of the concept phase inspired by the lab-in and field-in phase, while yielding more focused but also repeatable as well as more formative usability assessments that still can impact the development process well. The advantage here lies in still controlled conditions allowing for focused evaluation (e.g., eye-tracking studies in real train seats with contextual mock-ups including VR as we are currently executing for Human–Robot-Interaction), cameras, sensors and integrated visuals and interactions, as we have described in [25].

In the Lab-Out phase, the primary focus of the user step lies in conducting structured user studies within a controlled lab environment using high-fidelity prototypes that closely resemble real-world public transport settings. The goal is to evaluate the usability, user experience, and perceived system qualities of VICUPIS prototypes under simulated and repeatable conditions but with a field “feeling” based on realistic elements and even environments. Here, a range of standardized and custom-developed questionnaires are employed to assess user responses to visually interactive companion technologies [53].

One of the central instruments used for such evaluations is the User Experience Questionnaire (UEQ), which enables a fast and reliable assessment of both pragmatic (e.g., efficiency, clarity) and hedonic (e.g., stimulation, novelty) aspects of user experience. The UEQ’s dimensions—Attractiveness, Perspicuity, Efficiency, Dependability, Stimulation, and Novelty—provide a holistic overview of how users perceive the system’s usability and emotional appeal. It is particularly well-suited for comparing design iterations and identifying strengths and weaknesses in interaction concepts at an early stage. The original German version of the UEQ was created 2005 by a data analytical approach to ensure a practical relevance of the constructed scales, which correspond to distinct quality aspects [55].

Complementing UEQ, the Questionnaire for User Interface Satisfaction (QUIS) is often applied in its short version (QUIS 5.5) [56] to obtain detailed feedback on interface-specific aspects. QUIS captures user satisfaction in dimensions such as screen layout, terminology and information clarity, learnability, and perceived system capabilities, thereby allowing for a differentiated analysis of the visual and interactive layers of VICUPIS prototypes. The full QUIS consists of 27 items in five sections, which makes it valuable for use in complex lab studies but often is too much for short user studies in addition to others like UEQ and SUS Questionnaire [57,58].

For VICUPISs, being domain-specific developments, customized questionnaires are also often used to assess aspects specific to the mobility domain, including navigation quality, map visualization and usage, perceived sustainability, technical sustainability factors like power consumption, the presence and perception of advertisements, and overall satisfaction with the systems’ answers and guidance. These items are crucial for tailoring the companion system to the specific needs and constraints of public transport contexts and passengers.

10.3. Expert/Technical Step

From the expert perspective, the Lab-out phase focuses on building and refining a field-ready VICUPIS prototype. It builds on the final user lab (Section 10.2) results and comprises the final steps towards a field-ready system or prototype, including the following:

• Hardware/software integration across heterogeneous platforms (e.g., smart windows, smartphones, displays);
• Continuous technical testing of context responsiveness, system latency, and real-time adaptability with the system under construction;
• Ensuring compliance with technical, regulatory, and infrastructural requirements for field operation;
• Close involvement of experts and customers mainly from public transport domain for preparation of the field test and product/prototype delivery.

Particular emphasis is placed on integration of physical phenomena and states with software in CPS: synchronizing sensors, real-time data streams, and visual interfaces into a robust, modular system for real-world physical conditions and systems, often including stress-testing to ensure durability, error tolerance, and legal operability before public trials focusing on latency issues, offline situations, multi-user and adaptivity issues. Iterative development of a field-ready research prototype or product helps focus on hardware/software integration, robustness, and real-time context responsiveness including heterogenous platforms/devices, technologies, norms/laws for production environments like tested and secured hardware and platforms as well as sustainability concerns and paving the ground for a sustainable and long running system allowing for system evolution.

10.4. Case-Study: SmartMMI Research Project

In SmartMMI, the Lab-out phase involved a physical simulation of vehicle environments, including seating, movement cues, and lighting conditions, especially focused on the smart window prototype within a tram lab environment resembling the final geometry of the vehicle, physical and social conditions in the four-seat environment with tram surrounding. This proved very valuable for this pre-field phase as a preparation for the field-out phase, which made us start better and faster also with physical and safety testing of the prototype before installation in the vehicle for regular operation.

We conducted an eye-tracking study to draw attention to an interactive window. In this study, we compared the visual attention of digital natives and digital immigrants—each group consisting of twelve participants—using eye tracking to evaluate four menu design variants displayed in a public transport context. Results showed that digital natives had faster and more erratic gaze patterns, while animated elements significantly increased attention across both groups, confirming earlier findings that static designs like our initial prototype fail to attract sufficient user focus [26].

A Usability study with 24 participants was conducted to identify potential usability problems of older persons (digital immigrants) versus digital natives in public transport passenger information systems. In this study, we used a clickable prototype for passenger information on interactive windows in a three-step evaluation [26].

In the user study, we developed and tested a prototype Android app and interactive tram display system to communicate disruption information in public transport, using different notification modalities such as visual alerts, sound, and vibration. Through a controlled user study with 12 participants split into three groups, we examined preferences for device type and modality in two travel scenarios involving route changes. Results showed that while participants preferred receiving disruption information via smartphone, many dismissed notifications too quickly to fully read them, revealing usability issues that will be addressed in future iterations of our design [26].

In the user study, using a mockup setup closely replicating the vehicle configuration helped us refine our evaluation methods and finalize the concept for in situ usability testing. In the study, key combinations were used to manually trigger different information displays, allowing asynchronous and participant-specific interactions, which proved useful given varying familiarity levels across age groups and supports future use in real-world settings. The study also revealed that initial interaction with the interactive window was not always intuitive, highlighting the need for clearer cues—an aspect better explored in upcoming field tests with unsuspecting passengers in real trams [26].

Evaluation of accessibility and multimodality, such as haptic and audio augmentation for public displays, was much easier in the lab than changes and variations in the field, which required a final system that could not be technically change after deployment anymore. In the frame of iterative user testing of AR passenger information on mobile public displays [43], with mockups projected in realistic dimensions and display positions in a physically simulated tram, as can be seen in Figure 13, SmartMMI’s Lab-out phase helped us finalize system and its parameters, including, e.g., positions of elements, graphical coding and items.

These Lab-out evaluations ensured that SmartMMI interfaces were perceivable, legible, and usable across a range of environmental and user-related constraints before final deployment [53], e.g., user size, crowds, interchange and mobile device coupling. In our usability evaluation of the interactive window prototype [25] participants interacted with prototypes in real train seats within a mock-up setting and could also evaluate AR functionality [43] through the SmartWindow in user evaluations with regular passengers to also test symbolic representation, travel phase orientation, and multimodal output combinations as well as in-process disturbance and restrictions of the semi-transparent screen.

Another advantage was the possibility of pre-testing evaluation methods for the field test [50], which provided valuable insights into how AR and multimodal feedback can be reliably tested before full public roll-out, including cross-device interaction in the field.

10.5. Findings, Practical Take-Away and Heuristics

The User Experience Questionnaire (UEQ) proved a good approach for gathering user feedback at this stage. It was developed and published in German and English from 2005 to 2008 [59], has a continuously updated handbook [60] and is especially useful for benchmarking iterations and emotional impact. As it measures pragmatic and hedonic UX qualities including Attractiveness, Perspicuity, Efficiency, Dependability, Stimulation and Novelty, it has proven a valuable tool for innovative systems like VICUPIS in our projects and also scientific UI prototypes. The same applies for the older Questionnaire for User Interface Satisfaction (QUIS 5.5) designed by Chin et al. [61]; QUIS provides detailed feedback on more Usability focused topics like screen layout, terminology, learnability and perceived system capabilities, which makes up a good combination.

In evaluation of Mobility Systems like VICUPISs, custom questionnaires with mobility-specific contents often use a domain-specific addition or as an extension of the more generalized but stronger used ones above. These evaluate domain-relevant criteria such as: navigation support, map visualizations, battery usage, sign and advertising perception.

Recommendations and insights from the Lab-out phase include the following:

• Combine UEQ and QUIS: Use UEQ for quick benchmarking and hedonic measures, complement with QUIS for detailed interface analysis.
• Customize for context: Tailor or add questionnaire items to features under test (e.g., map usage, visual load, wayfinding comprehension) rather than always applying the correct and clean version—at when focusing on the system development rather than scientific papers.
• Diverse user inclusion: Involve multiple “real” user groups (commuters, elderly, tourists) to validate universal design assumptions also in the lab-out phase, ideally the same in the field-out phase.
• Simulate realism: Lab environments should reflect transport conditions (e.g., noise, screen angle, space constraints) to ensure transferability like our tram environment.
• Integrate CPS early: Cyber–Physical System challenges such as data synchronization, fallback behavior, signal reliability and generally synchronous physical and virtual/data state should be addressed before entering the field-out phase that does not forgive bad preparation.
• The Lab-out phase ensures that VICUPIS prototypes are not only conceptually sound and usable—but also technically viable, ready for public transport deployment and system integration in a field more demanding than, e.g., office settings.

11. Field-Out Phase

The Field-out phase represents the culmination and final phase of the VICUPIS development process. It brings the system prototype into the real-world operational environment for final validation and evaluation. This phase is essential for understanding how visually interactive companion technologies perform under the complex, dynamic, and often unpredictable “real-world” conditions of public transport.

Through real-time deployment—on trams, buses, platforms, or stations—this phase reveals not only technical functionality issues that often arise in the lab-out phase but also authentic user behaviors, environmental influences, and socio-technical challenges that cannot be fully reproduced in laboratory settings. Furthermore, it lays the groundwork for transfer to practice, turning tested systems into reusable solutions ready for long-term deployment.

11.1. Phase Focus, Considerations and Specialties

The Field-out phase focuses on in situ validation of VICUPIS prototypes under authentic operating conditions and the final system development based on these findings. Special conditions here include the full range of contexts described in Section 4, mainly the following:

• Physical unpredictability (e.g., noise, vibration, light changes, weather);
• Social dynamics (e.g., crowding, competing attention, passenger flow);
• System-level dependencies (e.g., data flow interruptions, latency, but also transport delays, infrastructure behavior);
• Platform diversity (e.g., different vehicles, stations, display configurations, user devices, communication channels).

In contrast to lab studies, field testing requires high system readiness, robustness, and real-time responsiveness to be ready for evaluation, which typically is not possible iteratively. Systems must prove at this stage to be flexible and context-adaptive, supporting adaptive UI behavior on the one hand and consistent interaction across heterogeneous devices and user roles on the other.

11.2. Field Testing (User Test) Step

In the user step, the VICUPIS system is tested with real passengers and in live transit environments. This phase captures how the system behaves when exposed to the true complexity of public transport, including the following:

• Varying attention states, cognitive loads, and emotional conditions;
• Unexpected events like route changes, delays, or user interruptions;
• Diverse user groups with differing digital literacy, mobility needs, and privacy preferences.

As we have shown in [56], field studies highlight how well VICUPISs adapt to situational constraints and whether passengers actually trust, use, and benefit from them.

Besides user observation and task-based user tests, evaluation methods in field testing again comprise questionnaires—often as a final step of the test—like User Experience Questionnaire (UEQ), which remains a key instrument, especially to compare lab vs. field conditions as well as dimensions such as efficiency and dependability gain priority in field use, where response time and reliability directly affect passenger satisfaction. QUIS 5.5 (Short Version) is used selectively to evaluate core aspects like screen readability, learnability, and real-time interaction, where applicable. Due to time pressure, abbreviated formats or contextual prompts are preferred in the field phase. Custom questionnaires, observations and extensions are tailored to evaluate field-specific aspects, such as the following:

• Did the system support decision-making under pressure?
• Were maps and navigation cues effective in crowded or noisy environments?
• Was battery consumption tolerable during extended trips?
• How was the integration of advertising, situational information and information for others perceived?
• Did the companion help in multimodal or disruption scenarios?

In the Field-Out phase, the VICUPIS prototype is deployed in the real operational environment, allowing for in situ validation under authentic usage conditions. This user step focuses on understanding how the system performs amid the complex, unpredictable, and socio-technical realities of public transport. Field tests are essential not only for evaluating the technical robustness of the system but also for examining real-world user experience, behavioral reactions, and systemic integration.

As in the Lab-Out phase, the UEQ remains a valuable tool for capturing user perceptions, particularly when comparing lab and field results. In field settings, however, some dimensions such as efficiency and dependability gain more critical relevance, as they directly affect the perceived usefulness of the system during time-sensitive or cognitively demanding travel situations. The hedonic dimensions (e.g., novelty or stimulation) may also reflect differently under real travel stress or environmental distractions.

QUIS 5.5 can be applied selectively in the field, focusing especially on aspects related to screen readability, learnability, and real-time interaction. Due to the time constraints and attention demands on users in transit, shortened or adapted versions of the questionnaires may be necessary to maintain data quality without overburdening participants.

Generally, a good preparation of the field test together with other stakeholders, especially public transport companies and operators, is the key to success and valuable insights, as well as a stable and fully functional prototype or system resulting from the lab-out phase.

11.3. Transfer to Practice/Product Roll-Out and/or Expert Abstraction Step

Beyond testing, the Field-out phase enables knowledge abstraction and preparation for system scaling and roll-out. Expert activities—especially in research-oriented projects—mainly focus on the following:

• Documenting reusable interface components, patterns, and modules.
• Deriving heuristics and best practices for further public transport/VICUPIS design.
• Preparing technical documentation for partners, cities and transport operators.
• Identifying opportunities for standardization and integration into broader mobility platforms.

This supports long-term sustainability and adoption in real mobility systems beyond research prototypes.

In the frame of product development projects, focus lies more on the development of the final product and roll-out of the system, while the approach of both foci is similar due to the specific restrictions and requirements of public (transport) systems.

This step is crucial to ensure long-term sustainability, enabling the transition from research prototypes and beta system versions to deployable, maintainable, and extensible public transport technologies and sustainable further development and roll-out with product status.

11.4. Case-Study: SmartMMI Research Project

In SmartMMI, we had the opportunity to carry out a field test with our Smart Window system running for one year with different supervised and non-supervised tests for collecting data (see Figure 14). As often found with VICUPISs, the effort lies more in the preparation and first phase of the field test, e.g., obtaining approvals for real-world use, rather than letting the installed system run for a longer time, and also having classic public transport systems in the production state run for decades, which is a positive aspect and shows potential for sustainability of such systems, but requires the anticipation of older and heterogenous field systems.

SmartMMI Field testing was carried out under full real-world constraints, including the following:

• Deployment on an operational tram, running on the same line for one year.
• Access to real-time data for monitoring tram status such as position by AVG (transport company).
• Coordination with Karlsruhe’s transport providers for tests in standard operation and with real passengers.
• Full integration into the train system—power, onboard systems, window, seamless physical integration in the vehicle.
• Collection of full anonymous log data on a server.
• Typical technical context and limitations due to CPS integration and network dependencies like loss of signal, sun, rescheduling, passengers wearing gloves, etc.
• Use of camera recordings and mobile eye-tracking [6] in supervised tests to gather rich interaction data.
• External factors such as pandemic restrictions, which required remote user monitoring and adaptive procedures, including a reduced information mode deployed.

The latter also reveals how big unforeseen impacts can occur in a real-world field test: the COVID-19 pandemic hit us exactly when the core field-test was scheduled to happen. Public transport authorities were therefore quite restrictive about tests with users, which made users switch to info mode remotely in the first part of the pandemic. This also shows how important it is to plan for unforeseen events and restrictions for field-test scenarios.

All these efforts above revealed real-world challenges—such as display glare, standing vs. seated interaction patterns, the pandemic situation, and interruptions—that would have been missed in lab-only testing (see also Field-in phase, Section 7).

The paper reports on a field test of an innovative passenger information system installed in public transport vehicles, where user interactions were recorded and analyzed in a real-world environment. Data was collected through system logs, video observations, and passenger surveys to assess usability, engagement, and information accessibility. The evaluation revealed patterns in how passengers used the system, identified usability issues, and provided insights for improving interface design and information delivery in future iterations [56].

11.5. Findings, Practical Take-Away and Heuristics

Recommendation for use: In the field, a lightweight questionnaire set should be used immediately after the interaction experience to ensure high response rates in line with good memory and context. Mobile-friendly survey formats (e.g., to fill in on a tablet), short verbal feedback also as thinking aloud during the test or retrospectively, or contextual prompts (e.g., triggered on trip end) can facilitate effective data collection with rich results. Combining subjective measures with system usage logs and observational data enhances validity and helps triangulate findings. It helps planning to have an eye on general field-test goals:

• Real-world validation of the prototype in live environments (e.g., on tram, at stations).
• Involve actual passengers, infrastructure, regulatory constraints, and operational dynamics in various parameter sets (daytime vs. night, events, user groups/persona, …).
• Capture unpredictable socio-technical parameters (e.g., crowd behavior, interruptions, system latency, weather conditions, interchange, other passengers).
• Flexible and realistic testing of system, UI and user behavior across devices, contexts and heterogeneous platforms [56].

Some key insights and practical heuristics from the Field-out phase that we have learned during our projects in the past nearly two decades include the following:

• Field testing is non-negotiable for VICUPIS: Many real-world issues—especially environmental and behavioral factors—are invisible in the lab, while often stakeholders shy away from the effort of these.
• Keep instruments light and mobile or integrated: Short, well-timed questionnaires (e.g., at trip end), mobile devices, paper or integrated (e.g., in Smart Window UI) yield higher participation and better context.
• Ensure technical reliability and stability: Even minor malfunctions in the field undermine trust and overshadow otherwise positive experiences, especially in field tests, which is why they have to be scheduled in the last phase of the project.
• Support user diversity: Field trials should include a wide spectrum of users and be sensitive to different needs and behaviors rather than “casted” or personnel from the transport company.
• Design for interruptions: Interfaces must be resilient to distraction, multitasking, and interaction dropout—common in mobile and public settings.
• Plan enough time for the final (expert) development step for products. We often experienced planning that places the field test in the last days of a project with no chance for learning and improvements—in the sense of UCD process understood wrongly, putting evaluation as a last step of the project.
• For research projects a big potential for sustainability lies in this last phase of making results, validated concepts and technology available for others and further projects by consolidating and generalizing them, which often does not happen with the project’s end approaching fast and people leaving.

The Field-out phase completes the VICUPIS development cycle by validating concepts, interfaces, and systems in the environments where they will ultimately operate. It strengthens system reliability, usability, and user trust—while also enabling the abstraction of results into transferable design knowledge. This makes Field-out not only a final testbed but a launchpad for sustainable innovation and practical adoption within the public mobility ecosystem.

12. Case Study, Discussion and Conclusion

12.1. Application Case Study: RegioKArgoTramTrain

We applied the VICUPIS Development Process in the frame of RegioKArgoTramTrain Research and Development Project for developing a Parcel Robot and its information and interaction system in the frame of a case-study: The effectiveness and versatility of the proposed VICUPIS development methodology was demonstrated through its application in the interdisciplinary research project RegioKArgoTramTrain, conducted in Karlsruhe, Germany. This project explores innovative concepts for integrating automated parcel logistics with passenger transport in a tram-train system, aiming to enhance sustainable urban and regional mobility while adding new services and logistics functions by a parcel robot. The approach combined passenger and logistics services within existing public transport infrastructure, supported by a visually interactive companion system to mediate human–robot interactions (HRI) on-robot and app-based.

12.1.1. Field-In Phase: Capturing Real-World Requirements and Context

In the Field-in phase, the project engaged directly with operational stakeholders, most notably the Albtal-Verkehrs-Gesellschaft (AVG), Karlsruhe’s regional public transport operator, alongside other municipal, research and industrial partners. This phase involved field observations and structured surveys targeting both passengers and transport personnel as well as logistics partners to uncover specific expectations, concerns, and requirements surrounding the introduction of automated parcel logistics in a shared passenger environment. Key insights included preferences for discrete and low-barrier interaction points with the system on-train and outside, trust-building / acceptance needs for autonomous operations in mixed settings with passengers and robot on-board, and accessibility considerations for diverse passenger groups. This resulted in a comprehensive, context-rich requirement catalog grounded in the operational semantics and constraints of the existing mobility ecosystem in Karlsruhe.

12.1.2. Lab-In Phase: Developing and Evaluating Interaction Concepts Under Controlled Conditions

Building on the field-derived requirements, the Lab-in phase transformed these findings into tangible prototypes and exploratory setups with three steps of fidelity:

• A stationary cardboard model of the parcel lockers on the robot and its base.
• A manually moveable robot mockup made of clad aluminum profiles, which already have manual interactions and multimodal “player” capabilities for sounds, etc.
• (Specification of) a moving high-fidelity model of the robot (for the lab-out phase).

The lab-based prototype of the parcel robot was developed to simulate the combined passenger and logistics scenario, focusing particularly on the human–robot interaction modalities required for sending and receiving parcels directly within tram-trains and at stops. This prototype already incorporated multiple interactive technologies for the Lab-in phase—some mocked, some technically already functional, including the following:

• Touch display for direct input and transaction handling;
• Laser trajectory projection markers on the floor to visually guide users and indicate robot movement;
• E-ink displays for dynamic, low-power informational updates at the side and front (face) of the robot;
• Acoustic and speech interfaces to support accessibility, reduce visual overload and attention control;
• Light-based signals to convey system status and next actions (“car lights”, errors).

Focus groups and structured cognitive walkthroughs were conducted with stakeholders and representative passenger groups within the controlled lab environment, including physical mock-ups of tram interiors, stop settings and the robot prototype above. Especially the focus group followed by a cognitive walkthrough proved very successful and very well perceived by participants and partners. This phase enabled the iterative refinement of the interaction approaches and collected systematic feedback on usability, clarity, and passenger trust in autonomous functions—and especially the robot setup, including dimensions as well as inserting and taking out parcels.

12.1.3. Concept Phase: Formalizing Models and Specifications for System Development

The insights, requirements and validated early prototypes from the Lab-in phase provided the foundation for the Concept phase, where specifications of the interactions, dimensions of all robot assets, robot orientation in the train and interaction-concept basic sound types (here, e.g., with help of music design master’s students) and also technical specifications for the robot partner company were developed. These included detailed interaction scenarios use-case diagrams, activity models outlining user and robot decision points, and further cognitive task analyses to conceptually ensure seamless integration into the passenger environment, also by extensive literature research on HRI. Particular attention was given to balancing intuitive interaction across modalities (visual, tactile, auditory) and systems (robot, app, public) and ensuring robust processes for typical public transport situations. These specifications from concept directly inform the subsequent design and construction of the physical (field) robot by an industrial partner and laid the ground for future Lab-out and Field-out phases, ensuring that the VICUPIS components within the RegioKArgoTramTrain ecosystem are not only technically sound but also grounded in realistic, user-centered interaction needs based on the new concepts.

Through this multi-phase application, the VICUPIS methodology proved effective in aligning complex technological innovation with genuine passenger requirements and operational realities. It underscored how iterative, field-informed, and context-aware development processes are essential for creating sustainable, trust-enhancing visual-interactive systems in next-generation public transport. The approach was very positively received and commented on by the companies and public partners in the project, thus proving the practical viability and positive effects of VICUPIS development process.

12.2. The VICUPUS Development Process

The structured yet flexible VICUPIS development process presented allows for iterative development that is both grounded in real-world mobility scenarios and open to conceptual innovation. It ensures that adaptive and intelligent systems remain understandable, usable, and trusted by both operators and passengers. Building on many projects and years of experience in research and development of VICUPIS and similar systems, this work aims at making the findings, generalizations and experience available to others in the domain of public (transport information) systems.

Section 7, Section 8, Section 9, Section 10 and Section 11 have detailed a comprehensive, field-informed, and user-centered methodology for the development of Visually Interactive Companion Technologies in Ubiquitous Passenger Information Systems (VICUPIS). Through the five iterative and interlinked phases—Field-in, Lab-in, Concept, Lab-out, and Field-out—the process integrates real-world complexity, technical feasibility, and human-centered design into a cohesive framework tailored to the mobility domain.

The Field-in phase grounds the development in authentic public transport conditions, capturing domain semantics and contextual user needs. It highlights the importance of observing actual passenger behavior and leveraging tools such as MobiDiary or ethnographic field studies to gather context-rich data. The Lab-in phase then provides a space to safely explore early interaction concepts through low-fidelity prototyping, involving users and experts in participatory design and evaluations, e.g., in focus groups, which has always proven to be a valuable approach with good results.

Building on these insights, the Concept phase formalizes design intentions into structured models, cognitive walkthroughs, and abstract prototypes—clarifying interaction logic and visual principles before technical investment. It also offers space for new ideas, innovations and sustainable approaches such as low energy interactions with e-ink displays.

The Lab-out phase transitions these concepts into high-fidelity, testable systems, allowing rigorous usability evaluation under controlled, yet realistic conditions using instruments such as UEQ, QUIS, and domain-specific feedback. It also contains most of the (expert) system realization with software coding and final hardware.

Finally, the Field-out phase is a core concept and deploys VICUPIS prototypes in the operational environment, validating them under real-world socio-technical constraints. It serves both as a test of robustness and as a gateway to scaling, standardization, and long-term adoption—supporting the transfer of research results into applied public transport systems and realizing long-running, sustainable VICUPUS in a finale development step.

Key qualities and foci of the VICUPIS process presented include the following:

• Context-driven: Emphasizes dynamic, real-time adaptation to physical, social, and technical environments typical for public (transport) systems.
• User-centered: Builds on UCD concepts and involves passengers and stakeholders across all phases in meaningful, accessible and efficient ways.
• Controlled and customized iterations integrated in five phases for a more plannable and reliable process with different foci and instruments.
• Bidirectional and field-oriented: Supports movement from field to abstraction and back with a focus on the real field at the beginning and end of the project.
• Modular and scalable: Aims at facilitating the design of reusable concepts and components across heterogeneous platforms and mobility phases with a framework of phases that can still be executed with more than one iteration, if needed, but cover all phases of typical VICUPIS development projects.
• Sustainable: Aligns research outcomes, innovations and sustainability considerations with field and implementation realities for long-term viability in transport infrastructures and system evolution.

This structured yet flexible process enables researchers, designers, and public transport providers to collaboratively create trusted, inclusive, and adaptive companion technologies that respond to the realities of modern mobility. By embedding usability, robustness, and context-awareness into the development cycle from the very beginning, the VICUPIS process ensures that digital innovation in public transport remains grounded, usable, and human-centered building sustainable systems.

The system definition of VICUPIS helps envision the characteristics and challenges of this type of systems, while the definition of context factors in VICUPUS can serve as a kind of checklist or map for context analysis and definition but also as a basis for definition and implementation of parameters for context adaptivity in VICUPIS implementation.

12.3. Contributions to Research and Practice

The key contribution of this work is the formulation of a complete development cycle for VICUPIS, comprising five interconnected phases—Field-in, Lab-in, Concept, Lab-out, and Field-out—each embedded with specific expert and user activities. The approach integrates core principles of Human-Centered Design Process (ISO 9241-210) [11] while expanding, structuring and customizing them to address the unique demands of ubiquitous computing, CPS environments, public and shared spaces found in VICUPIS.

By building upon an extended context model (Section 4), the methodology enables systems to be informed by and responsive to a full spectrum of user, spatial, temporal, physical, interactional, and socio-technical factors. This expands traditional design paradigms to accommodate fluid, dynamic, and multi-user settings inherent in public mobility infrastructures. The paper thus contributes to both methodological innovation and theoretical understanding of adaptive and context-sensitive interaction systems in complex, real-world domains.

The system classification (Section 3) grounded in the Lyytinen and Yoo ubiquitous systems model [30] adds further analytical structure, enabling developers and researchers to reason about different VICUPIS types based on their mobility, embeddedness, and interaction roles. This classification not only facilitates better design decisions but also supports scalable system architecture and comparative evaluations across deployment contexts for a quite new type of public system.

12.4. Integration of Evaluation Across Phases

A distinctive feature of the VICUPIS development process is the systematic integration of evaluation methods throughout all phases. Rather than isolating evaluation at the end of development or repeated in similar form in each iteration, this approach embeds tailored feedback loops in each phase, ranging from early cognitive walkthroughs and low-fidelity tests to high-fidelity lab experiments and in situ field validation. The combined use of standardized instruments like UEQ and QUIS with domain-specific custom questionnaires ensures that both subjective user experience and context-specific functional outcomes are assessed in a focused way.

This holistic integration of evaluation helps mitigate one of the core challenges in designing adaptive systems: the fail of standard UI evaluation procedures and the difficulty of anticipating all potential system behaviors, especially when adaptivity is driven by context-awareness and AI mechanisms. By incorporating user-in-the-loop evaluation and simulation-based testing, the process addresses the opacity and unpredictability often associated with adaptive companion technologies.

12.5. Relevance for Sustainable, Inclusive Mobility and Sustainable Systems Development

From a practical perspective, the framework directly supports current goals in digital mobility innovation, such as sustainability, inclusivity, and seamless intermodality. VICUPIS, as defined in this work, are not mere apps or displays: They represent a new and rising category of digital companion systems that can span devices, locations, user types, and usage scenarios, helping passengers navigate increasingly complex mobility networks.

Importantly, the methodology also recognizes the social and ethical dimensions of deploying systems in public and semi-private spaces. Through emphasis on contextual sensitivity, user diversity, and accessibility, the process supports the creation of systems that are not only technically functional, but also socially acceptable, legible, and respectful of privacy concerns in public environments.

12.6. Limitations and Challenges

Despite its strengths, the VICUPIS development process is not without limitations. The field-based, iterative nature of the process can be resource-intensive, requiring collaboration with transport operators, technical infrastructures, and diverse user groups, which may not be feasible in all research and development projects for VICUPIS. While the systematization enables modularity and applicability, actual transfer and scaling across different transport systems may require adaptation due to varying regulatory, technical, and cultural contexts, which cannot be covered fully in such an approach. On the other side of the applicability dimension, many concepts can be applied to similar domains and projects (see Section 12.5).

Our approach has been derived from existing and well-used and proven approaches like the UCD process, and nearly 20 years of experience in researching and developing companion technologies in public transport systems. While to our knowledge there are no more specific approaches for VICUPIS based on studies, some of the heuristics and findings presented are more qualitative and episodic in nature and should be backed by deeper studies carried out in VICUPIS development and for different phases and tools. However, this paper also provides a framework and starting point for this kind of research work, which we will also use and recommend for further research.

Balancing experimental control with contextual realism remains a challenge—especially in field phases, where measurement accuracy and user compliance may suffer due to environmental unpredictability. For this reason, the lab and concept phases also contain user activities to reduce limitations by focusing on different types of evaluation fitted to different phases. But still limitations exist for evaluations in each phase.

While the approach addresses sustainability in three parts, specific measures for product sustainability have to be integrated into each project specifically and cannot be predefined in a general process. However, the literature and guidelines also exist in the frame of this journal for reference.

Rapid technology evolution, especially in information and AI technology, is also a challenge to this kind of project and system—with or without a VICUPIS development process. The long development cycles required for VICUPIS and present in public transport may at times lag behind the rapid evolution of display and interaction technologies, AI and systems, requiring continuous methodological updates, and may invalidate field and lab findings in the “in” phases, and make them obsolete or changed for the “out” phases due to the cycles and foci required. Projects affected could introduce agile cycles as mentioned to assess such topics in later phases again, when necessary.

12.7. Sustainability Dimensions in the VICUPIS Development Methodology

The VICUPIS development methodology presented in this work yields concrete advances in the important software requirements on sustainability [62] on multiple, interrelated levels. By embedding a user-centered and context-driven approach throughout all development phases, it systematically reduces wasted resources and unnecessary iterations, ensuring that prototypes and final systems closely align with real-world user needs and domain constraints. This resource-conscious process directly supports sustainable system engineering, minimizing redundant efforts and fostering evolvable, long-lived systems in the sense of sustainable Software Engineering [63] including software development [64] and software architecture [65]. Moreover, the methodology’s emphasis on modular, adaptable, and extensible system design promotes longevity and future-proof passenger information systems, counteracting the rapid obsolescence typically seen in isolated, fast, or monolithic solutions also in the senso of economic sustainability [66]. On a broader societal level, the visually interactive companion systems designed through this process actively encourage a modal shift towards eco-friendly public transport by improving user trust, transparency, and perceived quality of service. By reducing uncertainty, enhancing accessibility, and elevating the overall mobility experience, VICUPIS systems contribute to increasing the attractiveness and uptake of (automated [67]) public transport and adjacent agile software development (e.g. with SCRUM [68]). Thus, the methodology not only supports sustainable system creation and deployment but also helps achieve broader goals of environmental sustainability and social inclusion in public mobility ecosystems, including fast transfer of research results into practice by experts and through this process for sustainability in public transport [69].

The VICUPIS development methodology presented in this work yields concrete advances in sustainability on multiple, interrelated levels. By embedding a user-centered and context-driven approach throughout all development phases, it systematically reduces wasted resources and unnecessary iterations, ensuring that prototypes and final systems closely align with real-world user needs and domain constraints. This resource-conscious process directly supports sustainable system engineering, minimizing redundant efforts and fostering evolvable, long-lived systems. Moreover, the methodology’s emphasis on modular, adaptable, and extensible system design promotes longevity and future-proof passenger information systems, counteracting the rapid obsolescence typically seen in isolated, fast or monolithic solutions. On a broader societal level, the visually interactive companion systems designed through this process actively encourage a modal shift towards eco-friendly public transport by improving user trust, transparency, and perceived quality of service. By reducing uncertainty, enhancing accessibility, and elevating the overall mobility experience, VICUPISs contribute to increasing the attractiveness and uptake of public transport. Thus, the methodology not only supports sustainable system creation and deployment but also helps achieve broader goals of environmental sustainability and social inclusion in public mobility ecosystems, including fast transfer of research results into practice by experts and through this process.

The proposed VICUPIS development methodology systematically embeds sustainability on three interrelated levels: the development process, the resulting systems, and their broader societal impact on sustainable mobility.

12.7.1. Sustainable Development Process

The VICUPIS development methodology emphasizes sustainable development projects by combining user-centered design, context and domain-specific concepts for sustainable transport as well as resource efficiency without repetition and sustainable results in system development. Therefore, it combines expert-driven and user-centered steps in structured phases, targeting evolvable, flexible and extensible systems. Each phase enables targeted use of resources by aligning efforts with real user needs, typical required steps and results, domain constraints, and context factors. Rather than relying on trial-and-error or over-specification, the process ensures that prototypes and concepts evolve iteratively and systematically, informed by empirical validation, avoiding unnecessary development cycles and reducing the burden on personnel, users’ infrastructure and other resources. This contributes to sustainability in the sense of resource-conscious, goal-oriented software and system engineering for public transport projects. Specifically, the early integration of users in the field-in phase helps improve the viability of results as well as the integration of user actions in one step per phase/cycle help, keeping users motivated and having a high impact. It also helps new players such as startups to enter the field with a reliable approach for developing VICUPIS.

12.7.2. Sustainable Systems Design

The framework promotes the creation of extensible, adaptable systems that can evolve over time and foresee possible developments and challenges as far as possible. By embedding modularity, personalization capabilities, and context-awareness into the core design, systems can respond to changing requirements from users, transport providers, and society. The method supports long-term usability and system growth including addition of new services, and integration with evolving infrastructure, data sources and needs. This future-proofing orientation ensures that once-deployed systems remain relevant, extensible and operable over extended life cycles, avoiding premature obsolescence and promoting continuity in digital public services. Specifically, using this approach we were able to sustainably deploy even innovative research user interfaces on SmartWindows in production train systems and cope with new developments like Corona pandemic and its restrictions.

12.7.3. Sustainable Mobility Impact

At its foundation, the VICUPIS approach contributes to the broader goal of sustainable mobility by strengthening public transport and the sustainable transport network. Visually interactive companion technologies improve access to information and offer user guidance, improving public transport mobility experience, e.g., perceived security, flexibility and situation awareness, promoting a shift toward sustainable modes of transport, with public transport as the backbone of the eco-friendly transport network.

By making multimodal and sustainable mobility more visible, accessible, and user-friendly, VICUPIS technologies help shift mobility behavior from motorized individual mobility toward resource-efficient transport modes, such as trams, buses, and shared services. This not only reduces the environmental footprint but also supports the social mission of inclusive, equitable urban mobility tailored to different user groups with varying demands, like reliable guidance for disabled people. Improved passenger information, particularly when visual and adaptive, reduces uncertainty and perceived barriers to public transport use—thereby enhancing its attractiveness and uptake, contributing to the modal shift towards ecomobility.

In sum, sustainability in this paper is not an external goal but an integral design principle that is reflected in how systems are developed, how sustainable they become, and what societal functions they support. The VICUPIS development methodology thus aligns closely with contemporary goals of digital and ecological transformation in public transport. Figure 15 shows the three layers of sustainability targeted in the approach.

12.8. Transferability of the Approach

While the VICUPIS development methodology has been tailored for the complex requirements of public transport systems, its structured, user-centered, and context-driven approach is broadly transferable to other domains that share similar characteristics. Fields with strong field requirements such as healthcare, smart cities, construction, production, logistics, or digital public services and generally public systems—where systems must operate across heterogeneous platforms, adapt to dynamic contexts, and serve diverse user groups in public or semi-public environments–can benefit from the same structured multi-phase, field-informed development structure. The modularity of the process, emphasis on real-world requirements validation, and integration of human factors with CPS’ constraints make it particularly suited for designing sustainable, adaptive systems that support human interaction with distributed, intelligent infrastructures, such as Human-Centered CPS in general.

Thus, the VICUPIS methodology offers a robust processual framework for innovative public projects oriented in public systems and companion technologies as well as innovation across multiple domains seeking to align technical advancement with user relevance, domain-orientation, field integration and long-term usage for sustainable systems.

12.9. Conclusions

In this paper we have developed and presented a comprehensive, sustainability-oriented methodology for the development of Visually Interactive Companion Technologies in Ubiquitous Passenger Information Systems (VICUPISs). Building on over fifteen years of applied research in multimodal mobility, human–computer interaction, and adaptive public systems, especially companion systems in Public Transport, the proposed development approach integrates field-based realism, lab-driven exploration, and conceptual abstraction in a well-structured and guiding, iterative process. By embedding UCD principles and process elements, domain-specific contextual modeling from Field-In to Field-Out for a well-defined group of innovative systems—VICUPIS—the methodology ensures that resulting systems are not only technically robust and adaptable, but also usable, inclusive, and aligned with the evolving needs of passengers, transport providers, and society—also for a longer timeframe of use and evolution.

A key contribution of this work lies in its explicit alignment with sustainability on multiple levels: the development process is resource-efficient and guided by real needs, making the systems developed and project results extensible and future-proof thus more sustainable than typical monoliths, isolated short living apps and technocratic systems with high cognitive load and little UX for users—private or professional. The target domain—public transport—forms the backbone of sustainable mobility but has no processes and tools tailored to its domain-specific requirements despite its importance for sustainable mobility. Moreover, the framework embraces the complexities of contemporary system design, including the role of adaptive systems and artificial intelligence, the challenges of context and context-aware adaptivity, and the demand for long-term and evolutionary digital transformation in public infrastructures.

Ultimately, VICUPISs are more than digital tools—they are socio-technical companions embedded in the public system realm, shaping the way people navigate, experience, and trust in sustainable mobility and its systems—leading to better usage and modal split. The proposed methodology offers a replicable path, model and framework for researchers, developers, and public transport authorities to collaboratively create the next generation of intelligent passenger information systems.

13. Outlook and Future Work

Building upon the presented VICUPIS development methodology, future work will focus on one hand on expanding and transferring the approach and framework to other domains and applications—and on the other hand strengthening it by further research on methods, heuristics and sustainability approaches.

Advanced technologies such as the currently fast developments and applications of the AI field also in Public Transport [70] fit very well to the domain and approach for adaptive public systems. One key direction therefore lies in the integration of AI-based user models and adaptive system behavior and prediction, enabling even more personalized, anticipatory system responses that will in turn profit from the approach of the VICUPIS process for gaining maturity and sustainability.

Additionally, future research should explore how VICUPISs can better support inclusive digital mobility services [71], addressing the needs of users with disabilities, neurodiverse passengers, or those unfamiliar with digital systems, where we started projects also for ubiquitous companion systems not needing mobile devices or complex technology as well as a promising European funded project on Human–Robot-Interaction in trains. Another promising area is the deeper fusion of VICUPIS with smart city infrastructure, which is directly connected to public transport, leveraging real-time data from urban environments, user-centric [72] mobility-as-a-service (MaaS, [73]) platforms, and edge computing systems—but also for sustainable public transport in rural areas, where adaptive services are a key to maintain and enhance public transport services.

Finally, longitudinal studies and cross-regional implementations are needed to evaluate the long-term sustainability, scalability, and societal impact of VICUPISs in diverse transport networks. These future efforts should aim to further mature VICUPIS into a foundational layer of intelligent, user-centered public mobility ecosystems and sustainable socio-technical systems in general.

Future research should also technically explore and transfer toolkits and platform frameworks to support more efficient implementation of the VICUPIS methodology, including plug-and-play evaluation modules, reusable interface components, and cross-system context modeling libraries or even agents. Additional work is needed to better integrate machine learning into user-centered systems for passengers and personnel as well.

Furthermore, research and standardization for adaptive public systems could support broader adoption and policy alignment in the era of AI—benefitting people, society and the environment in a sustainable way.