Exploring Augmented Reality HMD Telemetry Data Visualization for Strategy Optimization in Student Solar-Powered Car Racing

Abstract

This article explores how different modalities of presenting telemetry data can support strategy management during solar-powered electric vehicle racing. Student team members using augmented reality head-mounted displays (AR HMD) have reported significant advantages for in-race strategy monitoring and execution, yet so far, there is no published evidence to support these claims. This study shows that there are specific situations in which various visualization modes, including AR HMDs, demonstrate improved performance for users with varying levels of experience. We analyzed racing team performance for specific in-race events extracted from free and circuit-based real race datasets. These findings were compared with results obtained in a controlled, task-based user study utilizing three visualization interface conditions. Our exploration focused on how telemetry data visualizations influenced user performance metrics such as event reaction time, decision adequacy, task load index, and usability outcomes across four event types, taking into account both the interface and participant experience level. The results reveal that while traditional web application-type visualizations work well in most cases, augmented reality has the potential to improve race performance in some of the examined free-race and circuit-race scenarios. A notable novelty and key finding of this study is that the use of augmented reality HMDs provided particularly significant advantages for less experienced participants in most of the tasks, underscoring the substantial benefits of this technology for the support of novice users.

1. Introduction

Electric Vehicles (EVs) are becoming increasingly popular, particularly in developed societies, and it is predicted that by 2040, EVs will account for approximately 35% of all new vehicle sales. With the European Union planning to prohibit the registration of new internal combustion engine cars after 2035, nearly all car manufacturers have recently added electric vehicles to their product portfolios [1]. The next milestone in the development and adoption of EVs is the integration of solar power [2]. Although the necessary technology is already available, solar-powered cars remain largely in the development and testing phase and are not commercially widespread. Just as Formula One serves as a testing ground for combustion engines, solar races provide a valuable platform for the development and evaluation of innovative solar-powered vehicle technologies [3]. Car racing is among the most demanding sports, pushing teams to develop cutting-edge technologies. Innovations initially designed for Formula One, such as active suspension systems [4], sequential gearboxes [5,6], and energy control and recovery systems [7], have subsequently been widely adopted in the automotive industry. Similarly, solar racing events contribute significantly to sustainable innovation, pushing teams to maximize energy efficiency and tackle unique engineering challenges in vehicle design.

Competitive races held on both circuit tracks and public roads are a crucial pathway for technological advancement. The most renowned solar competition, the Bridgestone World Solar Challenge, has been held biennially since 1987, covering a demanding 3000 km route across Australia. Fierce competition among solar teams drives continuous adoption of advanced technologies during vehicle design. The car body and chassis are engineered to be ultra-lightweight with superior aerodynamics [8], using carbon composites [9,10], state-of-the-art telemetry systems [11], and specialized lightweight suspension [12]. The streamlined shape results from CFD modeling and collaboration with designers, while interior materials prioritize low weight and high durability. With every second crucial to race outcomes, an optimized race strategy is essential [13,14,15]. Making split-second decisions requires robust, real-time telemetry systems and applications that clearly visualize critical information. Data-driven systems have proven essential in both Formula One [12,16] and solar racing [11,17], enabling real-time vehicle monitoring and rapid anomaly detection by technical teams. Using Virtual Reality (VR) to train people in the design and effective operation of solar energy systems can play a crucial role in the successful implementation of solar technologies. Mixed and Augmented Reality (AR) systems are particularly promising for visualizing strategically important information, with Head-Up Displays (HUDs) now becoming standard in modern vehicles [18]. According to the literature, integrating AR interfaces into automated driving systems (ADS) significantly enhances trust, acceptance, and overall safety [19,20]. Augmented Vehicle Reality (AVR) can further extend a vehicle’s visual awareness by wirelessly sharing visual data with nearby vehicles [21]. The growing interest in Human–Vehicle Interaction (HVI) has driven research into intelligent vehicle interiors aimed at improving user experience, acceptance, and trust [22].

The work presented in this paper was conducted as part of one of Poland’s largest student competition-based projects, which aims to develop a highly efficient solar-powered electric car optimized for minimal energy consumption per passenger and superior race-strategy performance [23]. The main focus is the empirical evaluation of different telemetry data visualization modalities, including classical web-based and augmented reality head-mounted displays, in the context of solar vehicle race-strategy management. We illustrate how these visualization modes support the decision-making process for free and circuit track races using real in-race data. Our work goes beyond traditional telemetry interfaces for solar racing vehicles [24], which typically rely on flat, web-like graphs, by also exploring augmented reality-based visualizations as an alternative modality [17]. The findings demonstrate that while more experienced participants perform well regardless of interface type, augmented reality head-mounted display telemetry data visualization can significantly enhance telemetry data monitoring and performance in specific scenarios, particularly for less experienced users, highlighting its value as an effective decision-support tool in high-pressure environments.

The paper is organized as follows. Section 2 provides an overview of visualization methods for telemetric systems and particular solutions employed in solar racing vehicles. Section 3 presents details of the visualization modes considered in our research. Section 4 introduces the research questions and outlines the methodology of the task-based user experiment employed to evaluate the proposed concepts. Section 5 presents the results, including overall performance analysis and a detailed breakdown of quantitative and qualitative data across different experimental conditions and user groups, as well as self-reported cognitive and physical workload. Section 6 summarizes the key findings and discusses the limitations of the study.

2. Telemetry-Related Work and System Overview

Telemetry is the process of taking measurements from a distance [25] employed within a wide range of applications, including wildlife monitoring [26], engineering [27] and healthcare [28] with variety of modalities from raw analog sensing signals, to audio, to video, and satellite monitoring [29,30]. A telemetry system includes modules for measurements, the transmission of the data stream from remote sensors (typically using wireless communication), visualization, analysis of data, and finally, monitoring of object states. The telemetry system provides direct insights into the data, enabling visualization of spacecraft imaging [31] and observatory operations [32,33] and analytics [34] while also enabling remote interaction with the object.

2.1. Telemetric Data Visualization

Data visualization [35,36] has proven to be a valuable tool for efficient problem analysis [37,38], facilitating the development of strategies for solving real-world tasks [39], including in applications related to telemetry [32,40] and wireless data-based transmission [41]. It facilitates exploration, monitoring, and diagnostics across various fields [27]. Visualization approaches often vary by domain, with researchers developing specialized systems for space exploration [42], oceanography [43], e-commerce and marketing [44,45], healthcare [28], and industrial applications [46,47], as well as for motorsport [48], including car racing [49], or even for autonomous vehicles [50]. The demand for effective data visualization continues to grow in parallel with the increasing volume of data collected by modern telemetry systems.

Effective real-time telemetry visualization and reliable data acquisition are essential for rapid decision-making during races. Several system architectures have been proposed. In [11], the authors described a CAN Bus-based system with four embedded devices monitoring the batteries, motors, photovoltaic panels, and driver inputs. A similar WiFi-based topology consisting of five embedded devices was presented in [15]. Another approach used ZigBee networks due to their significantly lower power consumption compared to WiFi [51]. Distributed architectures, in which multiple devices collect data and transmit it to a central data bus, are not the only possible solution. Some studies [24] propose a highly centralized architecture where a single embedded device is responsible for collecting all telemetry data. However, this approach may pose significant risks, as it lacks redundancy. If the central unit fails, the team could lose access to all vehicle data, making the system neither fail-safe nor reliable under critical conditions.

Once transmitted from the vehicle, the data must be properly received and analyzed by the team, including the remote strategy crew located outside the vehicle. Some studies [17] describe the use of established platforms such as LabView or DeltaV to build basic graphical user interfaces for data analysis. However, these often offer limited customization. Moreover, most existing visualization systems are typically limited to 2D representations [52]. In recent years, Virtual Reality (VR) [53,54] and augmented reality (AR) technologies have gained significant attention [55], opening new possibilities for researchers exploring innovative forms of data visualization, including head-mounted displays (HMD) in medical, training, and industrial applications [56,57]. However, despite growing interest in applying augmented and mixed reality (MR) [58] in various domains, such as sight therapy support [59] and technical tasks completion [60], as well as experiments with gaze- and gestural-mixed reality interaction input [61,62], much of the research has focused solely on static or offline visualization. Real-time telemetry-based AR visualization has remained largely unexplored. In [63], the authors analyzed mobile AR apps for data visualization, noting issues including limited screen space and visual clutter. They recommended combining data types carefully to avoid clutter and ensure user comprehension. However, using tablet-sized mobile applications offers more display space, which can help mitigate these limitations [64]. Another example, described in [65], demonstrated the potential of AR for supporting emergency service, presenting a mobile AR application for flood risk management. Other studies have also explored AR applications in industrial settings for planar and 3D data analysis [66] as well as for production support [67,68]. For instance, in [69], the authors proposed an AR system that facilitates dynamic data exchange between digital and physical production environments. The system features a user-friendly and intuitive interface designed for practical use. Similar approaches have been considered for industrial applications, where AR has been used to enhance system monitoring and operator interaction [61,68].

Taking into account current trends in information visualization and the growing demand for advanced data presentation techniques, a modular, web-based telemetry system has been developed [70]. The system enables both offline data analysis and real-time visualization during races. Additionally, it supports integration with an AR headset, providing an advanced, hands-free method for data presentation and analysis. Details of the system design and implementation are given in the next section [23].

2.2. Telemetric Data Visualization Module for Eagle Two

The Solar Race Car system is divided into the following key main sections, as schematically illustrated in Figure 1: the onboard data acquisition and monitoring module, the strategy crew with data visualization modules, and the transmission system connecting them. Efficient collaboration and communication between these components, along with accurate interpretation of telemetry data by both the crew and driver, are essential for achieving success in the race.

2.3. System Overview

The Eagle Two vehicle is equipped with an up-to-date dedicated carbon monocoque chassis, two independent in-wheel electric motors, a solar power array on the roof, a substantial Li-ion battery set, and a dedicated control system, all designed and implemented to withstand both circuit-type and open-terrain race conditions. Its unique construction results in a low vehicle mass, enabling it to travel up to 1500 km on a single charge, with up to five passengers on board. Figure 2 presents the version of the vehicle that participated in the World Solar Challenge (WSC) 2019 in Australia.

The system design encompasses the driver input interface, car body construction, and electric motor inverter controller. It is also important in such EVs to implement driver assist features, such as cruise control, to monitor and adjust each motor’s torque in real time and maintain the target velocity. The vehicle control systems are based on STM32 microcontrollers interconnected via a CAN data bus for robust operation. This allows both flawless vehicle control and remote telemetric data collection and aggregation. The data can then be used to both inform the vehicle control system and support live telemetry-driven race-strategy decision-making. As CAN is a multiplex, message-based standard that allows microcontrollers to communicate without any central host computer, it ensures real-time communication among all onboard devices, handling control signals and managing external subsystems such as brushless direct current electric motors, battery packs, lights, etc. Moreover, it makes it possible for the race team to access almost on the fly the most important data, including the battery condition, motor status, energy flow, and parameters set by the driver [23]. The telemetry data, including battery status, motor conditions, user inputs, vehicle lighting status, solar power inflow, and charging status, serve as the driving force behind the race-strategy application. The strategy application focuses on providing a comprehensive and efficient overview of vehicle conditions and dynamics. A schematic operation of the strategy application and its visualization engine is illustrated in Figure 3. The application’s core logic module processes and prepares data for visualization. Its architecture allows for direct deployment at the strategist’s workstation or, if necessary, cloud-based operation.

3. Telemetry Data Visualization Mode Investigation

As the Lodz Solar Vehicle project has evolved over the years, consecutive modules have been designed and implemented. Therefore, this investigation is based on previously adopted solutions rather than on new designs or experimental prototype-driven research. In other words, the approach we took was to analyze the existing solutions developed and successively utilized as new features and modalities were incorporated into the project over the years instead of designing different, novel interface visualization modalities to suit the currently proposed comparative experimental study. Three different existing data visualization interfaces were studied.

The first visualization (henceforth referred to as the WebBase App) is a simple, flat webpage that collates all gathered data in the form of an ordered table. It was designed to review and debug all subsystems. The second visualization mode (henceforth referred to as the WebAdv App) is more advanced in terms of graphical data representation. This version highlights the most important selected information in a more graphically appealing way than the WebBase App mode. The third data visualization mode (henceforth referred to as HMD AR) was deployed using augmented reality holographic technology and head-mounted display glasses.

3.1. Basic Web Visualization Mode: WebBase App

The WebBase App was developed using the Angular front-end framework, so as to preserve basic design flexibility. This version of the app was implemented to visualize data using simple tables, as simplicity has been a common standard in solar racing dashboards for years [24]. Such a tabular example view is presented in Figure 4. The table view is split into four main vertical sections: vehicle speed, errors, lights, and miscellaneous data. There are three main types of information displayed: numbers, text descriptions, and status info indicators. It is worth emphasizing that the overall appearance of the interface is structurally well organized but, at the same time, simple and visually dull due to its uniformity.

Status data are highlighted in green when they indicate a positive Boolean value. Engine and battery errors appear against a red background. This simple technique allows users to easily spot any dynamic data changes. The view design makes it possible to present a substantial amount of information on a single screen in a clear, organized manner. As a result, users can view all parameters and a large amount of data on a single screen without switching pages. However, each parameter is displayed in the same uniform manner, without clear distinctions.

3.2. Advanced Web Visualization Mode—WebAdv App

Since some parameters may change more frequently or be more important or more complicated, extra information, such as the physical location of the vehicle on the route, is sometimes needed. Therefore, another mode of visualization was proposed, as illustrated in Figure 5. This interface presents a richer, more diverse view, divided into four main sections arranged across two rows. The upper row consists of a top bar displaying essential information through basic color icons and alphanumeric parameter values. The lower row occupies most of the screen and shows more visually detailed information. There are three widgets presenting the general status of the car on the left part of the top bar. When the green widget is highlighted, the vehicle is ready to drive. Red lights indicate a general battery or motor error. The right part of the upper bar shows other important battery and motor parameters, such as the state of charge, cell temperature, total battery pack voltage, vehicle instantaneous velocity, etc.

A race map is located just below the right-hand section, allowing the strategist to monitor the car’s location on the track in real time. On the left-hand side, four gauges display essential race information, such as car speed and power consumption, in a clear, separate format. By combining different display types, all critical and high-priority information about the car’s current condition is shown. However, this solution does not provide detailed debug information for the car’s systems.

3.3. Augmented Reality Application

The rapid development of AR technology has opened new opportunities for data visualization. Therefore, a new solution featuring this technology was proposed. The team began experimenting with the HoloLens HMD AR device. This augmented reality headset enabled features such as spatial mapping, hand gestures, and voice commands. We implemented the data visualization on a 3D car model, as shown in Figure 6. The left image displays the car’s main components: motors, batteries, and photovoltaic panels. These are highlighted in different colors to indicate their status. For instance, green photovoltaic rooftop panels show that the solar energy collection is near maximum (as indicated with #2 in the left picture). Dark blue rooftop panels indicate lower energy production status (indicated with #2 in the right picture). The motors and battery are highlighted in red to signal any errors.

The right image in Figure 6 shows an alternative view: four gauges displaying car parameters in the foreground, with the control panel visible in the background. The information presented is aligned with the model’s position via the control panel.

4. Evaluation

Solar car races can be extremely demanding, both physically and mentally. Routes often stretch over several days and pass through challenging terrain. Teams face not only race-related risks (e.g., tire punctures) but also potential threats in the environment, such as bushfires or sandstorms. This places significant stress on the entire crew, but particularly on the strategists, who must focus on the team’s current standing while planning future actions. Consequently, continuous efforts are made to enhance the monitoring and decision-support tools used by race teams to improve the overall experience.

One such effort involved deploying augmented reality (AR) technology to provide a more intuitive method for interpreting real-time data. Augmented reality was first adopted by the Lodz Solar Team in 2018 and received positive feedback, although no structured evaluation was conducted at the time. Team members reported performance improvements, yet it was unclear which specific AR features accounted for these gains. To address this gap, the authors later conducted a user study in a controlled laboratory setting. This study used a task-based approach to evaluate the methodology. Participants performed four tasks based on real historical race data presented in three different ways. Because everyone worked with the same dataset, their decisions could be verified against actual outcomes. The participants analyzed data in three distinct visual interface setups and completed four assignments. Their performance was assessed in two ways: by simulating the impact of their decisions on overall race results and by examining specific task outcomes. The first, broader analysis focused on overall simulated race performance, while the second, more detailed assessment included quantitative measures (task completion time, reaction time, decision accuracy) and qualitative indicators (Task Load Index and System Usability Scale).

4.1. Research Questions

Our goal was to investigate how different data presentation methods can enhance the assimilation of information by human operators and improve decision-making during solar car races. The project was guided by assumptions aimed at improving strategic decision-making in solar races and possibly reducing the mental workload on users who rely on AR interfaces, as reported previously by the team members. In addition, we sought to explore more broadly how AR could be utilized in the context of solar car racing, particularly with respect to telemetry data readability and usability. Given that solar car racing is a relatively niche pursuit and that AR holographic glasses are still unfamiliar to most people, we also considered whether personal experience might affect our findings. Accordingly, we formulated the following main research questions:

RQ1:

Does augmented reality-based data visualization for on-the-fly strategy adjustments in solar car races affect overall performance?

RQ2:

Which aspects of AR employment influence individual standard tasks related to the effective execution of telemetry-based Solar Race Car data analysis?

RQ3:

How does the experience level of team members translate into the effectiveness of using information presented in different interfaces?

Building on these main areas, we conducted a user-based study to compare a series of previously developed solutions and draw conclusions about their potential effects on performance, cognitive and physical workload, and usability. We used real datasets from two historic races—one free road race and one circuit track race—and employed task-based emulation to explore users’ reactions and decision-making processes. To thoroughly assess this scenario, we devised a study plan that incorporated both quantitative and qualitative methods.

4.2. User Study: Task-Based Evaluation

To compare various approaches to data visualization, we designed a task-based study involving the analysis and interpretation of telemetric data reflective of real race scenarios. These scenarios mirror typical situations in which the race crew must decide on strategy and response tactics. Four tasks were selected by experienced senior solar team race members to represent the most typical, standard situations that occur during the races. Each task incorporated a short time series of telemetry data recorded during actual events: the 2019 World Solar Challenge in Australia (for tasks T1, T2, and T3) and the 2018 European Solar Challenge in Belgium (for tasks T1, T3, and T4). The results from each task can be analyzed either individually or in aggregate. In particular, the aggregated results from T1, T2, and T3 may provide insight into the prospective overall performance change in free road racing, while tasks T1, T3, and T4 can highlight performance variations relevant to circuit races.

To explore this issue in more detail, four task-based scenarios were introduced, each designed to simulate real-world conditions with increasing complexity. The first task (T1), concerning tire-puncture recognition, is relatively straightforward to perform. However, in an actual race, such an event can lead to catastrophic consequences, including severe accidents on roads or circuit tracks. Because a tire puncture can rapidly compromise the car’s stability and handling, it is essential to monitor all incoming data throughout the race to ensure the safety of both the driver and passengers. The other scenarios involved more complex situations, such as a battery malfunction (T2), deciding whether to increase the vehicle’s speed during the free race (T3), or whether to charge the car during the circuit race (T4). These scenarios require the analysis of multiple, often unconnected, signals. When adjusting speed, the user must also consider previous events and current conditions, while T4 requires determining optimal charging times based on lap counts. Given that these scenarios were derived from historical data, the correct decision for each situation could be determined through documented choices and expert analysis. This enabled the evaluation of participants’ decisions in terms of both accuracy and the time taken, which are crucial factors in this context. Consequently, this approach offered an additional layer of analysis beyond traditional user experience studies.

The evaluation involved four task-based scenarios analyzed under three distinct visualization modes:

• Baseline simple, web-based application with a table-like simple interface application (henceforth referred to as WebApp Base);
• More advanced, containing graphical elements, yet still a web-based window view interface application (henceforth referred to as WebApp Adv);
• A head-mounted display–based, augmented reality holographic interface viewed through HoloLens (henceforth referred to as HMD AR).

Each of the four scenarios was scheduled to last five minutes. During this time, a predefined event was designed to occur, prompting the participant to detect a system malfunction and/or decide on the appropriate team action. The participants were presented with the following tasks:

T1:

Tire puncture—A sudden drop in air pressure in one of the wheels.

T2:

Battery malfunction—An unexpected voltage drop in one of the battery pack cells.

T3:

Optimal speed decision—Determine the best vehicle speed for the remainder of the race based on the current car condition and race situation.

T4:

Charging strategy—Decide when to charge the vehicle during a circuit race, specifically how many laps should be completed before charging, taking into account the car’s status and race conditions.

4.3. Participants

The study employed a between-subjects design. A total of 60 participants were recruited, including 27 females and 33 males. The participants had different levels of expertise in the analysis of telemetric data from solar vehicles—starting from main strategists in three races with significant experience in this field $n_{str}$ = 9, to moderately experienced team members with experience performing other roles in the team $n_{mod}$ = 18, to those with no previous experience of solar car racing at all $n_{no}$ = 33. We attempted to recruit as many experienced solar car race contestants as possible and succeeded in recruiting $n_{exp}$ = 27 such experienced participants, including 20 males and 7 females, all of whom were members of the Lodz Solar Team.

We composed three groups of participants, each consisting of 20 people. Each group was assigned to a different experimental condition, as follows. First, we coded their personal details and divided the participants according to their declared gender. We then differentiated them into three classes of experience level. For each condition, we assigned 3 of the 9 highly experienced users, 6 of the 18 moderately experienced users, and 11 users with no prior experience, bringing the total number in each group to 20 people. On average, the participants had been members of the Solar Team for 21 months, with a standard deviation of 7 months.

4.4. Experimental Setup

The experimental evaluation was conducted in the controlled laboratory settings of the Institute of Applied Computer Science, Lodz University of Technology, Poland. The tasks were conducted using MS Hololens mixed reality smart glasses, which allowed for smooth operation within an augmented reality space with a hands-free head-mounted display. The web-based solutions (basic and advanced versions) were running on a standard PC workstation equipped with 32GB RAM, average i-5 CPU, and basic GPU graphics driving a 27-inch display to ensure seamless operation. We used the original body of the Lodz Solar Team Eagle Two vehicle for all user study tasks. The environment, consisting of HoloLens applications, was programmed in C#, with a web server implemented in C# using the .NET framework and a front-end developed with TypeScript and Angular.

4.5. Quantitative and Qualitative Assessment

At the start of the experimental session, each of the participants gave written consent to take part in the study. Each participant was presented with the functionalities of all the researched applications and given the chance to use each of the systems before the actual experiment took place. Next, the experimental scenario was described to allow participants to better understand the goal of their task. The answer sheets were distributed separately before the start of each task. To ensure counterbalancing and reduce order effects, participants were evenly assigned to starting conditions such that every third participant began the experiment with a different interface modality: one third with the WebBase App, one third with the WebAdv App, and one third with the HMD AR application. Within each group, the order of tasks was randomized to minimize potential learning or fatigue effects. The participants and their answer sheets were anonymized. Their attempts were recorded to allow further analysis. After the completion of the investigation, all the files were deleted.

To check the stated hypothesis, the experiments were designed in such a way that the results could be measured both quantitatively and qualitatively. From a quantitative perspective, we decided to use the popular measures of task completion time and accuracy. Accuracy was measured by comparing the answers given by the participants to the ground truth answer tables. The tables were prepared by former Lodz Solar Team members who had been the strategists in the races. The task completion times for all the tasks were created using the video material from participants’ attempts. From a qualitative perspective, the workload index assessments and participants’ opinions concerning all the systems were taken into consideration. After finishing the tasks, participants completed two surveys on their experience during the task session. To assess the usability of different versions of telemetric data visualization, the System Usability Scale (SUS) was used. According to Bangor et al., SUS results can be translated into adjectives characterizing the assessed system [71]. To measure the effort required to complete each of the tasks, we decided to use the NASA Task Load Index (TLX) [72]. The NASA TLX is one of the most popular methods of measuring the workload needed to perform a task. Finally, after collecting these two questionnaires, we conducted a debriefing interview. We asked the users about their impressions of the AR data visualization and potential contexts in which such presentation of information/data would be useful.

5. Results and Discussion

This section covers the analysis of the results obtained during the user study, compiled comparatively against the historical race data. First, we proposed a pseudo-baseline comparison estimation for the anticipated impact on the overall race performance parameters. The comparison is based on average user study results contrasted with real, historical results recorded for the free and circuit types of races. In this way, we aim to simplify the process of quantitatively assessing earlier reports from Lodz Solar Team members regarding the potential benefits of using augmented reality in-race strategy execution. Subsequently, the results of the user study are presented and discussed in a detailed breakdown of distinct tasks. Additionally, we present a breakdown of the results for different subgroups of participants divided according to level of experience. Although the participants were carefully assigned to the conditions (for each 20-person cohort, as described in Section 4.3), taking into account three levels of experience (experts, moderate experience, no race-experience), the final analysis was conducted for two groups: high-experience users and a group of low-experience users formed by joining the moderately experienced and novice users.

The rest of this section is organized as follows. First, the prospective performance impact of this research is presented in Section 5.1. Then, detailed quantitative and qualitative measures are demonstrated in subsections regarding users’ reaction times and adequacy Section 5.2, cognitive workload, and usability Section 5.3. Finally, performance variance depending on the users’ experience levels divided into visualization modalities is highlighted in Section 5.4.

5.1. Prospective Impact on the Overall Performance Variation

The aggregate results for the free and circuit races were estimated, before analyzing the results for individual tasks in depth. Hence, in order to assess prospective performance change, a comparative analysis of the raw, historical race data and the average performance results obtained from the user study was conducted. All the race events within the historical data were analyzed to designate those types that were reflected by the tasks nominated for the user study. To exclude potentially erroneous results from the baseline dataset, the data were curated by a team of three expert users, who reviewed all designated events to eliminate any questionable decisions made by the original team during the specific race. Then, the total numbers of occurrences of each type of event were counted in each of the race (free- and circuit-mode) data sets. The simplified overall performance was approximated by the sum of significant parameters (such as the reaction times and accuracy of the decisions) for all the events. Next, the corresponding sum of the parameter values was calculated for the results obtained by the user study participants. For simplification, average results obtained by the participants for distinct tasks were taken as an approximation. Later, these average parameter values were multiplied by the number of occurrences for each of the event types. Finally, a comparative analysis was conducted to assess the prospective change in overall performance based on the sum of occurrences for the analyzed events.

Figure 7 presents a performance comparison bar chart for the World Solar Challenge 2019 free-race dataset. The three groups of three bars show results corresponding to the tasks T1, T2, and T3 for both real WSC dataset records and the controlled user study. The left bar in each group corresponds to real data. The other two bars in each group show results for controlled experimental data (high-level experienced users (HLExp) and low-level experienced users (LLExp)). To provide deeper insight, user study results for each task are presented with two bars relevant to the two groups of study participants—i.e., HLExp and LLExp.

Figure 8 presents an estimated performance comparison bar chart for the circuit-race dataset. As with the previous figure, the three groups of three bars show results corresponding to the three tasks. However, in the case of the circuit race, the tasks T1, T3, and T4 were analyzed for both the real circuit dataset records and the controlled user study. In the circuit race, T2-type events occurred too seldomly to be included in the analysis. The left bar in each group corresponds to real data. The other two bars in each group show the results of the controlled experiments. Again, the results for each task are split into two bars corresponding to the two groups of study participants—i.e., HLExp and LLExp.

Figure 9 presents an aggregated performance improvement estimation bar chart. The two left bars show results for the free-race data. The two bars on the right show circuit-race data. In each group, the left bars show the calculated total estimated performance difference calculated for HLExp, while the right bars correspond to LLExp.

As can be seen in Figure 9, there is a significant difference in estimated performance improvement for HLExp participants, who scored approximately 9%, while for LLExp users, the difference is less impactful, amounting to only approx. 3%.

5.2. Detailed Quantitative Results

The first quantitative results gathered for the experimental study were the task reaction times and response adequacy, as these were more comprehensive than the usually reported task completion times and error rate. Task completion times were measured for all four tasks performed by the participants. However, the reaction times and response adequacy describe users’ performance more specifically. Figure 10 shows a plot of the average reaction for each task for each application. As can be seen, the measured values differ greatly depending on the task, from a few seconds for T1 and T2 to over a minute for T3 and T4. This difference was expected since T3 and T4 were more complicated than T1 and T2. Users using advanced, graphical, web-based visualization had significantly lower TCT for tasks T2, T3, and T4. For T1, the difference between each visualization mode is not particularly noticeable.

The next and possibly one of the most crucial parameters we examined was reaction adequacy, which corresponds directly to strategy accuracy. We defined the reaction adequacy as a correct response given a timeout set as a multiplied value of the experts’ reaction average time. Hence, on the one hand, it is a strictly defined quantity, and at the same time, setting the particular timeout threshold value provides a certain measure of insight into possible user performance. A timeout value set strictly equal to the experts’ average results would result in a strong demonstration of suboptimal outcomes by inexperienced users. Setting this value to a greater multiply would tend to illustrate less weakness by novice users in comparison to well-versed experts. Therefore, the reaction adequacy defined in these terms is an illustrative parameter of adjustable meaning.

We decided to show the results of this parameter for different data visualization modes set at double of average times measured for experts’ responses. Such a threshold does not reflect most clearly the superiority of individual records for respective conditions, yet it gives a good overview of fluctuations in results depending on conditions for different tasks. The response accuracy for three visualization modes is compared in Figure 11.

Notably, with the reaction adequacy threshold set to double the expert reaction time, the results for all tasks vary by condition and, at the same time, are far from baseline—except for two conditions in T1 (tire-puncture event). For T1, participants achieved as high as 98% adequacy for the HMD AR visualization condition and 94% for the WebAdv App condition. For the other three tasks, the results mostly fluctuated at around 70–80%. The WebAdv App results were most consistent. The WebBase App recorded the worst results, dropping to as low as 70%, 64%, and 53% for T1, T2, and T3, respectively, although T4 with 81% was the highest T4 value among all conditions. Overall, it can be concluded that the WebAdv App mode yields reasonable performance in all cases, similar to the HMD AR condition. The WebAdv App achieved the best results in two tasks, whereas HMD AR did so in only one.

5.3. Cognitive Workload Assessment

Cognitive and physical workloads during all tasks were measured using NASA TLX and SUS methods. The tasks were designed to represent different levels of complexity, to measure the impact of human perception. The results of T2 and T3 were used to compare phenomena occurring with rising levels of complexity. To fully compare the impact of different visualization methods, results should be cross-validated with task completion times and task accuracy scores. The first task, T1, is associated with an important and rapidly occurring event: a tire puncture that calls for immediate attention. Figure 12 presents the results of NASA TLX T1 for all conditions.

Notably, the task load index scores for T1 show relatively high mental demand and high frustration for the simple WebBase App interface. Moreover, the reported effort was the highest for this type of visualization, although it remained at a reasonable level, within 35 scores. Physical demand in using the system was the highest for the AR application interface, as was temporal demand (although the temporal demand level was the same as for the WebBase App modality). Users perceived a high level of performance for all the modalities, with the lowest value of 80 out of 100 for the WebBase App mode. Generally, temporal demand scores can be very low, with values between 10 and 15.

Figure 13 shows NASA TLX results for T2 (battery malfunction detection). The NASA TLX results for T2 show generally more demanding conditions than for T1. As with T1, the WebBase App mode received notable mental demand and frustration scores; however, these scores increased significantly for the other modalities as well. Performance scores were high, yet lower than in the case of T1. Again, the WebBase App received the lowest score of 70. The AR-based solution required noticeably higher effort in terms of both physical and temporal demand than the other two methods.

The mental, physical, and temporal demands, as well as the frustration and effort, were generally higher than for T1, especially in the case of the AR HMD app. Users still assessed AR more positively than the WebBase App in terms of performance. On the other hand, the WebAdv App modality received the best scores in all the categories for T2.

Task 3 was associated with strategic decision-making during an open road race: adjusting vehicle speed according to the current situation and optimizing the predicted remainder of the race. For T3, the NASA TLX scores illustrate even higher numbers in terms of demands, effort, and frustration, with much lower scores for performance assessment (Figure 14). For most categories, the WebAdv App receives the best scores, requiring the lowest mental and physical effort, causing the least frustration, and receiving the best performance score. Overall, users found T3 highly mentally demanding, with scores from 60 (for WebAdv App condition) to 80 and 85 (for WebBase App and AR modalities, respectively).

During Task 4, participants performed the even more difficult task of analyzing modeled dynamic race conditions. Figure 15 shows that this task was significantly more demanding, introduced more frustration, and yielded lower perceived outcomes than T3. From a direct comparison of both tasks, it can be observed that the overall demand increased—as was expected, as the task was, in general, more difficult. A general drop in the participants’ confidence can also be observed as a reported decrease in the subjective performance score. Again, the WebAdv App application scored better than the other two solutions.

As shown in Figure 16, all modalities exhibit comparable SUS results, with only minor variations, except for T1. A significant difference is observed in T2, where the HMD AR application receives the lowest score. This is an interesting finding, as it contrasts with the quantitative results, which indicate the best performance for this modality in terms of reaction times and task load index values. On the other hand, while the results are generally comparable, the WebAdv App consistently receives the highest ratings in the System Usability Scale analysis.

5.4. User’s Professional Experience Level vs. Mode of Telemetry Visualization

To gain a deeper understanding of how data interpretation can influence user performance, it is worth comparing response results obtained by HLExp and LLExp users. Figure 17 and Figure 18 compare the correct response reaction time performance (RTP) to each particular event for consecutive tasks (T1, T2, T3, and T4). Each graph for consecutive tasks shows the summarized results broken down per visualization mode (WebBase App, WebAdv App, and HMD AR).

As shown on the RTP graph for T1 on the left of Figure 17, LLExp users performed better than HLExp users in WebBase App visualization mode but achieved worse results for both WebAdv App and HMD AR modes. Notably, the HLExp users performed worst with the WebBase App, significantly better with the WebAdv App, and achieved the best results with HMD AR, whereas LLExp users performed comparably for all modalities, achieving the best results with the WebAdv App—although still considerably worse than HLExp users. The RTP graph for T2 shown on the right of Figure 17 demonstrates the slightly better performance of HLExp users over LLExp users. However, whereas the results for both Web App conditions show comparable values, HMD AR conditions show significantly worse results achieved by LLExp users.

The results for T3 are shown on the left-hand side of Figure 18. For this task, the differences for all three conditions are significantly better for HLExp users. The HLExp users performed well and similarly for all three modes, while the LLExp users performed best on HMD AR and worst with the WebBase App. However, the average RTP times for HMD AR were still almost 5 times worse than for HLExp users. The results for T4 are shown on the right-hand side of Figure 18. Both groups of users achieved the best results with HMD AR and the worst results with the WebBase App. Interestingly, LLExp users performed significantly better than HLExp users for each condition in T4.

5.5. Summary of Results

To determine whether any discernible patterns align with the racing team’s earlier reports of potential performance differences stemming from various data visualizations, we analyzed the results achieved by participants using three distinct visualization interfaces. The strongest evidence supporting the potential for improving race-strategy performance is presented in Figure 9. However, it is important to acknowledge that there is no definitive evidence that the new interfaces consistently outperform all others in every situation, which would be the ideal outcome in response to RQ1.

We also analyzed the results obtained under experimental conditions to examine whether, how, and to what extent different data visualization modes could enhance user performance while perhaps also reducing cognitive and physical workload (RQ2). Generally, the overall performance difference between the real results extracted from the races and the average results achieved by the participants using the emulated data reflecting those real results visualized using proposed modes showed possible progress, as shown in Figure 7 and Figure 8. The average possible increase in efficiency ranges from 1% to 9%, as can be observed in Figure 9. Interestingly, the reaction times varied and were not always superior over the baseline condition (WebBase App) for other experimental conditions (WebAdv App and HMD AR); yet the baseline never reached the best results, as can be seen in Figure 10. Reaction adequacy analysis revealed poor results for the baseline condition for most tasks, except T4, for which the results for all the conditions were comparable, as depicted in Figure 11. Remarkably, when analyzing the entire group of participants, the HMD AR condition proved superior only for a specific task (T1, tire puncture), achieving an almost perfect score of 98%.

On the other hand, task workload analysis revealed heterogeneous and diverse results across different tasks and conditions. Results for T1, in which the HMD AR condition proved highly efficient, showed significantly higher physical demand compared to the other conditions yet received low scores for mental workload and high scores for self-assessed performance (Figure 12). For the other three tasks (T2, T3, and T4), HMD AR appears to have been more demanding than in T1, and in most cases more demanding than the WebAdv App condition, as can be seen in Figure 13, Figure 14 and Figure 15. The WebAdv App condition was also rated as the least demanding and least frustrating during most tasks, and it was self-reported as providing the best performance, even though it did not actually achieve the top results (as illustrated by T4).

Regarding reaction adequacy, the WebAdv App condition was similar to HMD AR but lagged behind the WebBase App condition for T4. However, it was again the best rated in terms of NASA TLX parameters. System usability results revealed similar relationships as NASA TLX analysis. As illustrated in Figure 16, HMD AR achieved the best results for T1 and performed comparably well to the WebAdv App in T4. The WebAdv App achieved the best scores for T2 and T3, while for T1, the WebBase App achieved particularly low marks. At this point, one might conclude that the study’s outcomes are ambiguous since there is no clear evidence that any single visualization mode is superior to the others; therefore, no strict guidelines can be derived from this study.

However, analyzing the results by user experience level offers a different perspective. As shown in Figure 17 and Figure 18, for T1, T2, and T4 LLExp users achieved significantly better performance when using the HMD AR interface than for other conditions, while HLExp users outperformed them across all experimental conditions only in T3. Therefore, a key aspect of this work involves showing that emerging interfaces such as augmented reality in head-mounted displays can—in some cases and for some users—be highly effective in diverse decision-making scenarios.

Figure 19 presents a summative comparison of three types of results: reaction time (RT), result adequacy (RA), and System Usability Scale (SUS), across all three experimental conditions, divided between more experienced (HLExp) and less experienced (LLExp) participants. The best scores in each category and task are highlighted in bold in the table. As can be seen, these top scores are almost exclusively achieved by HLExp participants (except for the first two T1 measures, RT and RA), although they are unevenly distributed across the different visualization modalities. The fact that the best results are obtained by more experienced users is unsurprising. Moreover, the tendency for traditional interface types to be more effective for these users could be attributed to their longer familiarity with such systems.

However, a deeper analysis of the data reveals additional insights. In the table, cells with a vertically dashed background indicate the best scores achieved by LLExp participants. Strikingly, most of these top results fall under the HMD AR condition (except for one measure: RT in T2, T3, and T4). This clearly highlights the significant potential of augmented reality HMDs to support less experienced users.

6. Conclusions

This article has explored various modalities for displaying telemetry information to support strategy selection and execution during solar car races. The results demonstrate how different data visualization methods reflecting the vehicle’s current status can benefit different user groups to varying degrees. Intriguingly, while standard interfaces provided users with good command in most situations, non-standard interfaces such as augmented reality in a head-mounted display were found to successfully assist certain types of users with specialized tasks. Notably, response reaction times analysis showed that while participant experience is crucial for more challenging tasks, less experienced users can sometimes achieve significantly better results when using an AR HMD than their more experienced counterparts. Moreover, the results suggest the potential for improved strategy execution in at least half of the examined events—i.e., two out of three free-race event types and two out of three of the circuit-race events. While the NASA TLX results were, to some extent, ambiguous, users reported experiencing significant mental and physical load when using the AR HMD. In some cases, they also gave lower scores for self-assessed performance, even when this perception did not align with the actual measured response times.

Therefore, one of the most significant takeaways from this study is the need to pay close attention to users’ experience levels. Our findings show that emerging technologies, such as augmented reality, have notable potential, particularly for novice users. This suggests that well-structured training programs, specifically tailored to specialized tasks performed in AR-equipped head-mounted displays, can be highly effective. By focusing on the unique demands of these interfaces, novice users can quickly develop skills, potentially achieving performance on a par with, or even exceeding, that of more experienced users.

6.1. Limitations

Many limitations exist in the current study, beginning with the controlled environment used during the experiments. In real-life scenarios, the application would be deployed during an actual race, placing users under intense time pressure and responsibility. Moreover, members of solar car racing teams lack sufficient time to rest and recover throughout the event. Hence, the limited timing, employed experimental setup, and calm settings of this study may not fully capture the application’s real-world conditions. The study’s findings are also limited to the tested scenarios only.

6.2. Directions for Future Work

While the presented infovisual interface graphics are imperfect, we did not prototype any new designs for the sole purpose of this study, relying instead on previously deployed visuals to avoid introducing additional confounding variables into this exploratory study. However, for further exploration of the identified research directions, it would be worthwhile to propose specifically designed interfaces and select new experimental scenarios that incorporate these novel elements. Furthermore, as one of the key findings of this study highlights the strong potential of augmented reality as the most effective option for certain user groups, it is essential to further investigate the preferences of novice users in various contexts. This could serve as a foundation for designing and prototyping new interfaces from the ground up, specifically tailored for this group and leveraging the AR modality, namely augmented reality delivered through head-mounted displays. Such an approach would likely lead to significantly improved outcomes. It can also be assumed that there is substantial potential for applying this technology in other domains that require decision-making support for novice users, such as industry and healthcare.