Experimental Performance Assessment of an Automated Shuttle in a Complex, Public Road Environment

Abstract

Automated, electric shuttles are expected to be key for the future of public transportation, providing a safe, efficient, and robust operation with a minimum carbon footprint. However, in complex, urban environments, their reliable operation is particularly challenging and shows a lack of performance and comfort. This study presents a quantitative benchmark of an automated shuttle compared to a conventional, human-operated bus on the same route. Speed and acceleration across geofenced segments are systematically analyzed based on over 12 million GNSS and IMU data points. The results show that the automated shuttle operates at about half the average speed of the bus. Furthermore, frequent abrupt decelerations are reducing passenger comfort, while the main distributions and mean values of the measured acceleration indicate a smooth operation of the automated shuttle; outliers reveal critical braking events. The presented methodology enables objective performance tracking and supports the iterative improvement of autonomous shuttles through datadriven optimization.

1. Introduction

Public transportation (PT) is the backbone of public mobility in Germany and key to reducing greenhouse gas emissions and achieving sustainable mobility. The integration of electric drivetrains and the implementation of high levels of automation in PT systems offer a powerful pathway to further enhance efficiency, safety, and minimize environmental impact. Despite substantial progress in the electrification and automation of road transport, the reliable operation of automated shuttles in complex urban environments remains a significant challenge—particularly regarding speed, comfort, and consistency of performance. This paper addresses these limitations by presenting an objective, data-driven benchmark of an automated shuttle operated in a complex public road environment, directly compared to a conventional, human-driven bus on the same route. Unlike prior studies that rely primarily on subjective passenger feedback or broad performance indicators, this work employs a comprehensive and quantitative analysis with over 12 million data points based on measurements with the Global Navigation Satellite System (GNSS) and an inertial measurement unit (IMU) collected during several months of operation. The route is systematically segmented into geofenced sections, allowing for a detailed and statistically robust comparison of both vehicles’ performance. This study not only quantifies current performance gaps between automated and conventional operation but also establish a repeatable framework for performance benchmarking and continuous improvement of automated shuttles. This approach lays the foundation for a Plan–Do–Check–Act (PDCA) optimization cycle in public transportation automation, supporting evidence-based enhancements in speed, safety, and passenger comfort.

2. Related Work

In this section, related work is summarized within the context of this study, including documented implementations of automated shuttles for PT, the assessment of user acceptance, performance evaluations of PT, and the assessment of human performance compared to automated operation of a vehicle.

2.1. Automated Shuttles for PT

Automation and electrification are central to developing affordable and low-emission mobility. Automated shuttles are typically introduced into existing PT fleets to test new service models and gather operational experience toward full automation. Numerous pilot projects in Europe, the U.S., and Australia have explored different use cases (Table 1). All current systems operate under human supervision and correspond to SAE Level 2 [1], focusing on safety and early integration into existing services. The shuttle used in this study represents this class: an SAE Level 2 vehicle operating in mixed traffic on multiple road types, including main roads and narrow, historic city streets. Manual interventions were documented and excluded from the data set, ensuring that results reflect automated performance only. The applied geo-segmentation method enables correlating performance with road characteristics, supporting transferability to other routes. Previous studies lacked sufficiently segmented, large-scale, sensor-based datasets and did not directly compare automated and human-driven operation.

Table 1. Overview of international projects focusing on autonomous shuttles for PT and their characteristics.

Name	Project Goal	Vehicle Type	Road Type	Characteristics	Reference
AHOI (Hamburg, Germany)	integration	midsize bus	city streets	demand-controlled PT, mixed operation, control center	[2]
ALBUS (Hannover, Germany)	operation, integration	midsize bus (∼40 seats)	city streets	SAE level 4, rural areas	[3]
Testfield Friedrichshafen (Germany)	operation, integration	shuttle (10 seats)	city streets, country and back roads	V2x, urban development concept (ISEK)	[4,5,6]
FLASH (Nordsachsen, Germany)	operation	shuttle (20 seats)	city and back roads	speeds up to 70 km/h	[7]
A.R.T.Forum (14 partners in 6 countries)	operation	shuttle (11 seats)	city streets	impact of automated PT on the road transport system and city life	[8,9,10]
Sohjoa Baltic (Finland)	operation	shuttle (11 seats)	city streets	various scales, remote operation	[11,12,13]
Linie 23 (Bern, Switzerland)	operation, integration	shuttle (9 seats)	city streets	on-demand, control center	[14]
FABULOS (Helsinki, Talinn, Gjesdal, Helmond, Lamia)	operation	shuttle (10 seats)	city streets	segregated lanes, speeds up to 40 km/h, on-demand service, remote control	[15,16]
AVENUE (Geneva, Lyon, Copenhagen, Luxembourg)	operation, integration	shuttle (11 seats)	city streets	large areas with low demand	[17,18,19]
Automated MIDI-bus (Stavanger, Norway)	operation	midsize bus (∼40 seats)	city streets	goal: SAE level 4, speeds up to 40 km/h	[20]
Murray (Australia)	operation	shuttles (10–15 seats)	city streets	with weak PT and strong tourism	[21]
Fairfax, Virginia (USA)	operation	shuttle (6 seats)	city streets	level 3 with significant complexity and higher traffic volumes	[22]
Elizabeth Vale (Australia)	integration	shuttle (6 seats)	city streets	control center	[23]
Aggie Auto Shuttles (Grensboro, USA)	operation	shuttle (6 seats)	urban and suburban	handover study	[24]
this study	operation	shuttle (11 seats)	city streets, country roads, slopes	complex environment, V2x	[25]

2.2. User Acceptance

Passenger acceptance is essential for the success of automated shuttles. Most studies rely on interviews or questionnaires to assess perceived safety, comfort, speed, and convenience (Table 2). However, many parameters are not objectively defined, depend on the individual respondent, and vary across studies. Despite these differences, speed, braking behavior, and ride comfort consistently emerge as key factors influencing user perception. To date, few works have linked passenger acceptance to objective performance data [26]. The present study addresses this by comparing the automated shuttle and a conventional bus using quantitative, objective sensor data for speed and acceleration in identical operating conditions, enabling an objective view of the status and potential advantage of automation. Specifically, the assessment of the Total Dynamic Acceleration (TDA) introduced in Section 6.2 is presented as an objective proxy for perceived comfort within this document.

Table 2. Studies on user acceptance.

Reference	Year	Vehicle Type	Information Sources	Perception
[6]	2024	automated shuttle	questionnaires	reliability, speed, safety, security, cost
[27]	2024	automated shuttle	questionnaires	compatibility, relative advantage, safety, security, cost, time, frequency
[28]	2024	automated shuttle	questionnaires	accessiblity, time, cost, frequency
[29]	2023	automated shuttle	questionnaires	performance, effort, social, risk
[30]	2023	automated shuttle	questionnaires	usefulness, ease of use, trust, safety, security
[10]	2022	automated shuttle	field observations, interviews, speed, waiting time, safety, emergency stopping
[31]	2022	conventional bus	questionnaires, GNSS (position, speed, direction)	travel time, speed, turning, waiting time, stopping, dwell time
[19]	2021	automated shuttle	interviews, literature review, personas	safety, comfort, reliability, routes, timetables, speed
[25]	2021	automated shuttle	questionnaires, interviews	safety, traffic obstruction
[32]	2021	automated shuttle	questionnaires, interviews	safety, speed, (abrupt) braking, frequency
[14]	2021	automated shuttle	interviews, manual documentation of emergency stops	speed, predictability, usefulness, environmental friendliness
[12]	2020	automated shuttle	interviews, questionnaires	smoothness, speed, braking, security, safety, comfort, travel time, accessibility, service coverage, cost
[16]	2020	automated shuttle	interviews, questionnaires	braking, speed, (unplanned) stops, accessibility, comfort
[33]	2020	automated shuttle	questionnaires	frequency, safety, comfort, time, cost
[34]	2020	automated shuttle	questionnaires	(abrupt) stops, safety, efficiency
[35]	2020	automated shuttle	questionnaires	safety, trust, time, user experience
[36]	2019	automated shuttle	calculations, simulations (acceleration, jerk)	-
[26]	2018	automated vehicle	physical simulator (acceleration, jerk)	comfort, safety
[37]	2016	automated shuttle	questionnaires	performance, effort, social, time, efficiency, speed, convenience, cost
[38]	2008	subway vehicle	in-vehicle measurement (acceleration, jerk)	-

2.3. Experimental Performance Assessment and Benchmarking of PT

Several studies have evaluated PT performance using GNSS data or information based on setups for Automated Vehicle Location (AVL) and Automated Passenger Count (APC) (Table 3). Most focus on travel time, schedule adherence, or quality-of-service metrics for conventional buses [39,40]. While useful for identifying system-level trends, such analyses rarely reveal the causes of performance deviations. Some studies of automated shuttles rely on infrastructurebased event logging or simulation models [22,41], highlighting challenges such as low speed and infrastructure constraints but lacking detailed, measured data and comparisons to human operation. The present work fills this gap through geo-segmented, objective, sensor-based benchmarking of automated and conventional vehicles on the same route.

Table 3. Performance assessment of the buses in PT.

Reference	Year	Vehicle Type	Original Data	Spacial Analysis	Statistic Asessment
[39]	2023	conventional bus	GNSS	route matching, route completion, on-path, on-schedule	-
[40]	2023	conventional bus	GNSS	bus-stop-based	travel times
[22]	2023	automated shuttle	vehicle- and infrastructure-based	infrastructure-based	events, incidents
[41]	2019	automated shuttle	none, model-based assessment	-	-
[42]	2017	conventional bus	AVL, APC	route-based, bus-stop-based	transit performance indicators
[43]	2016	conventional bus	AVL, APC	localization-based visualization	-
[44]	2015	conventional bus	AVL	bus-stop-based	punctuality
this study	2025	automated shuttle compared to conventional bus	GNSS, IMU	track-structurebased, average segment length $76.8$ m	speed and acceleration distribution per segment

2.4. Benchmarking Automated Operation Against Human Performance

Achieving human-level driving performance remains a central goal in vehicle automation [45]. Comparisons have been made in racing contexts [46,47,48], revealing differences in risk-taking and control limits. However, no prior studies have benchmarked automated shuttles against conventional buses in complex urban environments—a primary contribution of this work.

3. Objectives, Questions of Research, and Structure

Automated shuttles in complex, urban environments do not operate at a level competitive with a conventional bus with respect to both speed and comfort. Existing research is limited either by its qualitative approach or its limited level of detail. To improve research in this field, we address the following questions of research:

• How can objective, sensor-based data be used to benchmark automated shuttles against conventional buses?
• How does an automated shuttle perform compared to a conventional bus on a complex, urban track?
• Which characteristics have the largest impact on speed and comfort of the automated shuttle?

A methodology and a set of relevant criteria is developed in Section 4. The experimental setup, extent of the measurement campaign and test environment is explained in Section 5. The results of the comparative, statistical speed, and the acceleration analysis is presented in Section 6. The authors conclude and present opportunities for future work in Section 7.

4. Performance Criteria

To assess the performance criteria, two quantities are identified, which can be automatically and objectively measured, statistically assessed, and are relevant for both users and operators. The identified parameters are intended to compare performance consistently and allow the definition and implementation of measures for improvement.

4.1. Speed

In [37], the authors describe that the most important factor for deciding on the way of transportation is the expected performance compared to other PT systems. The speed of transportation is one of the most relevant criteria to assess with direct impact on efficiency and cost for the operator and comfort for the customer. On the one hand, the time from start to destination is expected to be reasonably short. On the other hand, the variation in transportation times is to be minimized, which directly relates to the reliability and performance according to schedule. Thus, besides looking at the average travel time for a fixed distance, the distribution, including the minimum and maximum values, are relevant. In this study, the travel time from a start to an endpoint is assessed based on a predefined, detailed geo-segmentation. This allows to assess root-causes for differences in performance and to identify potential measures for improvement.

4.2. Acceleration

Another important quantity for a potential customer is the expected ride comfort which can be reduced significantly by the occurrence of sudden and surprising high values of acceleration or jerk as indicated in the literature review [26,36,38]. Within this study, the TDA, which is explained in Section 5, is analyzed statistically and compared between the automated shuttle and the conventional bus. Since the static part of the acceleration (gravitational acceleration and static sensor offsets) do not contribute to discomfort, these are removed for each axis of the sensor and each segment specifically.

5. Material and Methods

This section contains a brief description of the track under investigation, the vehicles used, and the data acquisition, including the analysis flow.

5.1. Track Under Investigation

The tracks of the automated shuttle and the conventional bus are included in Figure 1. They are located in the city of Lauenburg (Elbe), Germany, a small historical town of about 11,000 inhabitants, approximately 40 km southeast of Hamburg. An automated shuttle operates in a mixed traffic mode with human-operated vehicles [49]. The modal share is driven by commuter traffic with about 60% passenger vehicle use and a public-transportation share of close to 10% [50]. The total track length of the shuttle is approximately $2.45$ km. To perform a comparative analysis of both vehicles, the common part of the tracks has been broken down into 20 geo-segments as indicated in Figure 1 and Table 4, respectively. The segmentation has been setup in consistency with the programming of the automated shuttle, leading to an average segment length of $76.8$ m. All tracks included in the analysis are plotted Figure 2. The key data and specifics of every segment are summarized in Table 4. The conventional bus and the automated shuttle share three bus stops on the track. For each vehicle, a fourth stop has been installed in segment 13 for the bus or in segment 16 for the shuttle, as indicated in Table 4. Since the automated shuttle must operate and find its location also with reduced GNSS-performance due to trees blocking the sky, additional landmarks have been setup in the segments 8 to 10. These consist of concrete blocks shown in Figure 3. Segments with very narrow streets are marked accordingly in Table 4, which present a particular challenge for keeping a minimum clearance of 20 cm around the automated shuttle. Any obstacles, such as overhanging shrubbery and twigs, found in this area around the vehicle will lead to an immediate and sudden stop. Thus, this information is included in Table 4.

Figure 1. Track under investigation in Lauenburg (Elbe) [51].

Table 4. Programmed path length per segment of the track under analysis. Abbreviations: Shuttle Stop (S), Bus Stop (B), Traffic Light (TL), Speed Bump (SB).

Segment	Start Shuttle [m]	End Shuttle [m]	Length [m]	Flow Controls	Pavement	Slope	Remarks
1	0	53.8	53.8	TL	asphalt	-
2	53.8	120.8	67.0	-	asphalt	-	main road
3	120.8	193.6	72.8	-	asphalt	-	main road
4	193.6	286.0	92.4	TL	asphalt	-	main road
5	286.0	360.1	74.1	-	asphalt	-	main road, speed limit 30 km/h
6	360.1	427.3	67.2	-	cobblestone, asphalt	down	-
7	427.3	487.7	60.4	SB	cobblestone, asphalt	down	-
8	487.7	557.9	70.2	-	asphalt	down	dense treetops
9	557.9	611.8	53.9	SB	cobblestone, asphalt	down	dense treetops
10	611.8	681.2	69.4	SB	cobblestone, asphalt	down	dense treetops
11	681.2	754.6	73.4	S, B	asphalt	down	-
12	754.6	855.9	101.3	SB	cobblestone, asphalt	down	-
13	855.9	951.2	S: 95.3	B	cobblestone	-	sep. paths
			B: 125.3				shuttle/bus
14	951.2	1030.6	79.4	S, B	cobblestone	-	-
15	1030.6	1106.2	75.6	-	cobblestone	-	-
16	1106.2	1190.1	83.9	S	cobblestone	-	-
17	1190.1	1269.3	79.2	S, B	cobblestone	Up	-
18	1269.3	1332.0	62.7	bollard	cobblestone	-	two-way road, width ≤ 5 m
19	1332.0	1418.1	86.1	-	cobblestone	-	two-way road, width ≤ $3.5$ m, vegetation
20	1418.1	1503.6	85.5	-	cobblestone	-	two-way road, width ≤ 5 m

Figure 2. Segmentation of the route and experimental GNSS-tracks (a) of the automated shuttle (b) of the conventional bus in direct comparison.

Figure 3. Concrete landmarks added to ensure sufficient self-localization with GNSS signals blocked by treetops.

5.1.1. Segments 1 to 5

The segments 1 to 5 contain two traffic lights (TLs) which are equipped with ITS-G5-based Road-Side-Units (RSUs) to optimize the longitudinal guidance of the automated shuttle. The transmission of the current state and prognosis of the transition to a stopsignal allows the automated shuttle to either slow down or keep at its maximum speed with proper foresight in the segments 1 and 4. This feature was specifically included in segment 4 to minimize the impact of the automated shuttle operating at a maximum speed of 18 km/h (5 m/s) compared to the running traffic with a speed limit of 30 km/h ($8.3$ m/s) and minimize the potential for traffic jams as well as avoiding combinations of acceleration directly followed by a strong deceleration which would reduce passenger comfort.

5.1.2. Segments 6 to 11

These sections include a downhill slope between 6% and 16% as well as a speed limit of 20 km/h ($5.6$ m/s). The street surface consists of a continuous change between cobblestone and asphalt. Several speed bumps are included in segments 7, 9, and 10. The view to the sky is partially blocked by a dense canopy of treetops. Segment 11 contains a shared bus stop for both vehicles.

5.1.3. Segments 12 to 16

Starting in segment 12 the pavement consists of cobblestone only. Due to a very narrow one-way section in segment 13, the conventional bus follows a different route which includes a bus stop. Thus, segment 13 is the only part of the track under investigation which cannot be directly compared. The difference in lengths is included in Table 4. Both vehicles share a stop in segment 14.

5.1.4. Segments 17 to 20

The pavement in segments 17 to 20 consists of cobblestone, which is combined with an uphill slope of 10% over a distance of 25 m in segment 17. The route of the shuttle contains a stop in segment 16, while in segment 17 both vehicles share a stop. To enter Section 18, the vehicle must pass an automatically operated bollard, which keeps other vehicles without permission out of this section. The bollard is triggered automatically in advance with a cellphone, based on a defined geofence to minimize waiting times. The total width of the road in segment 19 is below $3.5$ m for two-way traffic, which is the lowest value of the full track. This is combined with overhanging shrubbery and twigs.

5.2. Vehicles

The automated, electric shuttle used in this study is a NAVYA evo [52]. The measurements for the conventional bus were performed in a MAN A66 midi-bus [53] with a Diesel engine. Both vehicles are compared in Table 5. A picture of both vehicles used in this study is included in Figure 4.

Table 5. Overview of vehicle parameters.

Parameter	Automated Shuttle	Conventional Bus
Type	NAVYA evo	MAN A66
Drive Type	4-wheel, electric	2-wheel, diesel
Max. Speed	25 km/h	96 km/h
Speed Limit	18 km/h	-
Peak Engine Power	34 kW	184 kW
Number of Passengers (seated + standing)	11 + 0	22 + 39

Figure 4. (a) Automated shuttle NAVYA evo and (b) conventional bus MAN A66.

5.3. Data Acquisition

A block diagram of the application-specific data logger, developed for this project, is included in Figure 5. The data logger allows for a fully independent operation of the vehicle under investigation and includes a single-board computer, an IMU, communication means based on wireless local access network (WLAN) and the cellular network, and a board containing a receiver for GNSS. An additional IMU and the GNSS-antenna are connected externally. The external IMU allowed flexible positioning and adjustment of the orientation to better coincide with the vehicle axes as indicated in Figure 6. The external IMU was used for the measurements analyzed in Section 6.2. The measures quantities and their characteristics are included in Table 6.

Figure 5. Application-specific data logger.

Figure 6. Schematic setup of the data logger, IMU and GNSS antenna in (a) the automated shuttle and (b) the conventional bus.

Table 6. Characteristics of the measured quantities.

Data Source	Quantity	Unit/Format	Measurement Frequency	Measurement Range	Measurement Resolution
GNSS	UTC Time	[HH:MM:SS]	1 Hz	24 h	1 s
GNSS	Latitude	deg	1 Hz	$\pm 90$ deg	0.000001 deg
GNSS	Longitude	deg	1 Hz	$\pm 180$ deg	0.000001 deg
GNSS	Number of Satellites in View	-	1 Hz	99	1
GNSS	Speed	km/h	1 Hz	-	$0.01$ km/h
IMU	Acceleration x	g	50 Hz	$\pm 8$ g	0.000244 g
IMU	Acceleration y	g	50 Hz	$\pm 8$ g	0.000244 g
IMU	Acceleration z	g	50 Hz	$\pm 8$ g	0.000244 g

The data logger was setup for four weeks in the automated shuttle. After this first part of the measurement campaign it was moved to a conventional public bus that was used on the same track for the following 13 weeks. All experimental data were collected in the time frame between November 2020 and April 2021. Table 7 includes an overview of the parameters of the data-acquisition campaign.

Table 7. Overview of the measurement campaigns.

	Automated Shuttle	Conventional Bus
Time Frame of Data Collection	November 2020–December 2020	December 2020–April 2021
Total Number of Data Points	5,639,742	6,999,327
Total Time analyzed	$31.33$ h	$38.89$ h
Total Number of Passages	123	347

5.4. Data Preparation and Analysis

In this study, we analyze the speed per segment based on the GNSS-signals and a spatial segmentation of the track. The data flow is visualized in Figure 7. After the automated data acquisition, data sets with known manual operation are automatically removed from the data under analysis. Thus, the data points contributing to the results are based on its automated operation and manual interference can be almost completely excluded. The remaining cycles are separated based on a geofence for each segment. The time per segment is calculated and based on the entry and exit time for a segment. Based on the programmed path length, the average speed is estimated, which is finally statistically analyzed.

Figure 7. Data flow for the segment-specific speed analysis.

The acceleration is statistically reviewed following a similar flow as indicated in Figure 8. Just as for the speed analysis, data subsets with manual intervention are automatically removed. However, due to the impact of gravity, intrinsic sensor offsets, and sensor orientation, special measures were implemented to minimize its impact on the results. Since the track under investigation includes uphill and downhill slopes (Table 4), the value of the gravitational acceleration will be distributed to the three axis of the acceleration sensor and the distribution will change over position of the vehicle. To remove this offset, the TDA is introduced. For each segment and each axis, the average value of the corresponding acceleration is calculated, which is subtracted from each single, measured value in the respective segment as indicated in Equations (1)–(3). For constant conditions in each segment, this approach removes all static residuals for each axis. Thus, only the dynamically changing part of the acceleration is used for the analysis.

$$a_{x,TDA}\left(t\right)=a_x\left(t\right)-{\displaystyle \frac{1}{n}}\sum _{i=1}^n{a}_x\left(t_i\right)$$

(1)

$$a_{y,TDA}\left(t\right)=a_y\left(t\right)-{\displaystyle \frac{1}{n}}\sum _{i=1}^n{a}_y\left(t_i\right)$$

(2)

$$a_{z,TDA}\left(t\right)=a_z\left(t\right)-{\displaystyle \frac{1}{n}}\sum _{i=1}^n{a}_z\left(t_i\right)$$

(3)

Finally, for each data point, the magnitude of the total acceleration is calculated based on Equation (4). This value is the base for the analyses presented in the following sections.

$$a_{TDA}\left(t\right)=\sqrt{a_{x,TDA}{\left(t\right)}^2+a_{y,TDA}{\left(t\right)}^2+a_{z,TDA}{\left(t\right)}^2}$$

(4)

Figure 8. Data flow for the segment-specific acceleration analysis.

This method successfully removes static offsets, which would also include constant lateral acceleration signals due to curves extended along a full segment. These can have an impact on perceived comfort and would be removed in the calculation. However, this effect was taken into account during geo-segmentation. For the track under investigation, the chosen single segments are almost linear.

6. Results

In this section, the measurement results with respect to speed and the TDA are presented, and statistically and comparatively analyzed.

6.1. Comparative Speed Analysis

The distributions of the speed per segment is included in Figure 9, Figure 10, Figure 11 and Figure 12. The results are summarized in Table 8 and Table 9. While the speed distributions of the conventional bus are mostly symmetrical, the automated shuttle most of the time operates at or close to its programmed maximum speed, which leads to asymmetric speed distributions. Prominent examples are shown in Figure 10a,b. The observed confidence intervals for a confidence of 95% (Table 8) with a typical total width of 0.2 m/s confirm a precise distribution with statistically valid differences between the segments.

Figure 9. Speed distributions for the segments 1 to 5, including the overall range of observed values for the conventional bus and the automated shuttle.

Figure 10. Speed distributions for the segments 6 to 10, including the overall range of observed values for the conventional bus and the automated shuttle.

Figure 11. Speed distributions for the segments 11 to 15, including the overall range of observed values for the conventional bus and the automated shuttle.

Figure 12. Speed distributions for the segments 16 to 20, including the overall range of observed values for the conventional bus and the automated shuttle.

Table 8. Mean speed per segment and the confidence interval for 95% confidence.

Segment	Automated Shuttle, 95% Confidence			Conventional Bus, 95% Confidence
	${\mathit{v}}_{\mathbf{lower}}$ [m/s]	${\mathit{v}}_{\mathbf{mean}}$ [m/s]	${\mathit{v}}_{\mathbf{higher}}$ [m/s]	${\mathit{v}}_{\mathbf{lower}}$ [m/s]	${\mathit{v}}_{\mathbf{mean}}$ [m/s]	${\mathit{v}}_{\mathbf{higher}}$ [m/s]
1	1.07	1.19	1.30	1.77	1.92	2.07
2	3.79	3.87	3.96	6.16	6.28	6.39
3	4.23	4.34	4.44	7.73	7.85	7.98
4	3.68	3.85	4.01	6.84	7.04	7.24
5	4.19	4.26	4.32	8.07	8.17	8.27
6	2.37	2.44	2.51	5.33	5.44	5.54
7	2.63	2.69	2.76	4.67	4.74	4.81
8	2.00	2.05	2.11	4.85	4.92	4.99
9	2.31	2.40	2.49	4.77	4.85	4.93
10	2.25	2.35	2.44	5.13	5.21	5.30
11	1.97	2.08	2.19	4.99	5.08	5.16
12	2.06	2.14	2.22	4.51	4.59	4.67
13	1.47	1.53	1.58	2.22	2.29	2.36
14	1.57	1.71	1.84	3.82	3.91	4.01
15	2.67	2.76	2.84	4.22	4.32	4.42
16	2.03	2.13	2.23	3.52	3.60	3.67
17	1.47	1.55	1.64	3.37	3.47	3.58
18	1.19	1.22	1.26	2.71	2.77	2.83
19	1.10	1.19	1.27	3.02	3.10	3.17
20	1.56	1.66	1.75	3.59	3.66	3.75

Table 9. Measurement results for the speed per segment.

Segment	Automated Shuttle					Conventional Bus				Ratio of Mean Speeds Bus/Shuttle
	${\mathit{v}}_{\mathbf{min}}$ [m/s]	${\mathit{v}}_{\mathbf{max}}$ [m/s]	${\mathit{v}}_{\mathbf{mean}}$ [m/s]	${\mathit{v}}_{\mathbf{prog}}^{\mathbf{mean}}$ [m/s]	${\mathit{\sigma }}_{\mathit{v}}$ [m/s]	${\mathit{v}}_{\mathbf{min}}$ [m/s]	${\mathit{v}}_{\mathbf{max}}$ [m/s]	${\mathit{v}}_{\mathbf{mean}}$ [m/s]	${\mathit{\sigma }}_{\mathit{v}}$ [m/s]
1	0.40	2.58	1.19	2.73	0.64	0.65	6.48	1.92	1.43	1.61
2	2.15	4.53	3.87	4.47	0.48	2.31	9.52	6.28	1.12	1.62
3	1.74	4.97	4.34	4.77	0.57	2.17	11.97	7.85	1.18	1.81
4	1.30	4.84	3.85	4.72	0.90	2.22	11.32	7.04	1.88	1.83
5	2.22	4.88	4.26	4.78	0.36	4.52	11.99	8.17	0.95	1.92
6	0.45	4.13	2.44	2.87	0.41	0.84	8.84	5.44	0.96	2.23
7	0.67	3.72	2.69	3.00	0.37	3.11	6.77	4.74	0.64	1.76
8	1.46	3.68	2.05	2.36	0.30	1.76	6.80	4.92	0.70	2.40
9	1.97	4.94	2.40	2.30	0.50	2.14	7.51	4.85	0.73	2.02
10	1.92	4.68	2.35	2.30	0.55	1.52	7.32	5.21	0.82	2.22
11	0.58	4.26	2.08	1.99	0.62	0.59	7.43	5.08	0.80	2.44
12	0.68	3.76	2.14	2.27	0.44	0.53	6.93	4.59	0.75	2.14
13	0.34	3.18	1.53	2.15	0.29	0.55	4.64	2.29	0.89	1.97
14	0.03	4.05	1.71	2.15	0.76	0.86	6.16	3.91	0.91	2.29
15	0.55	4.75	2.76	3.00	0.47	1.34	7.99	4.32	0.91	1.57
16	0.24	3.97	2.13	2.82	0.56	1.02	6.09	3.60	0.70	1.69
17	0.31	4.16	1.55	1.74	0.48	0.52	5.53	3.47	0.99	2.24
18	0.31	1.94	1.22	1.47	0.20	0.26	4.24	2.77	0.59	2.27
19	0.57	4.02	1.19	1.33	0.46	0.22	5.30	3.10	0.72	2.61
20	0.74	4.85	1.66	1.78	0.54	0.46	6.60	3.66	0.93	2.20

6.1.1. Segments 1 to 5

Segments 1 to 5 consist of main roads with an asphalt surface and TLs in segment 1 and segment 4. In segment 1, the distribution of speeds for the automated shuttle and the conventional bus show a significant overlap and similarity (Figure 9a). This indicates, that most of the time, the traffic flow control limits the performance of both vehicles to a similar level. However, when analyzing the spread of the observed values, in single instances, the conventional bus is running at speeds up to $6.48$ m/s, while the speed of the automated shuttle stays below $2.58$ m/s. The automated shuttle reaches its highest top ($4.97$ m/s) and mean speed ($4.34$ m/s) in segment 3, while the conventional bus reaches these in segment 5 with a maximum speed of $11.99$ m/s and a mean speed of $8.17$ m/s. Both segments are located on the main road (Figure 1) and offer very well controlled environments with almost no interruptions allowing the operation at the maximum allowed speed. For both segments and both vehicles, the form of the speed distributions is symmetrical and bell-shaped (Figure 9c,e). In contrast to segment 3, the measurement results for segment 4 indicate the largest standard deviation of all segments for the automated shuttle. The speeds histogram (Figure 9d) indicates a double distribution with one peak between $1.5$ m/s and 2 m/s and a second one around $4.1$ m/s. This can be explained by the traffic light at the end of segment 4. The automated shuttle has been programmed to either keep its speed if it can pass the traffic light without stopping or slow down if the received information indicates that a stop is unavoidable. Thus, these two modes of operation create a double distribution in the speed histogram. In segment 5, the conventional bus reaches its highest mean speed of $8.17$ m/s and its highest maximum speed of $11.99$ m/s. This segment begins shortly after a traffic light on the state road with a speed limit of 30 km/h. Obviously, the human driver manages to pass this traffic light swiftly. Similarly, the automated shuttle stays very close to its maximum speed with a very tight distribution as indicated in Figure 9.

6.1.2. Segments 6 to 11

In segments 6 to 11, the track is constantly running downhill with a mixed cobblestone and asphalt pavement and several speed bumps. The automated shuttle is operated at mean speeds between $2.05$ m/s and $2.69$ m/s. The conventional bus is typically around twice as fast in these segments. In segment 9, the measured maximum speed for the automated shuttle is $4.94$ m/s which is much higher than the programmed mean value of $2.30$ m/s. With respect to the distribution of speeds in Figure 10, this maximum is identified as an outlier, potentially caused by the human operator briefly taking over without documenting the intervention. The narrow distributions shown in Figure 10 and the small standard deviation indicate a relatively smooth ride in the automated shuttle. Both vehicles operate as expected in this downhill slope. Obviously, the dense treetops, potentially blocking GNSS-reception, do not have an impact on automation. The countermeasure implemented (Figure 3) is successfully providing reference marks for localization.

6.1.3. Segments 12 to 16

The pavement of segments 12 to 16 consists of cobblestone, which is mostly running horizontally with two planned stops for each vehicle. In Segment 13, the vehicles take a slightly different route leading to a difference in path lengths. In segment 14, the lowest minimum speed of $0.03$ m/s is observed for the automated shuttle, which was caused by extended operator breaks at the planned stop. In segment 15, the speed ratio of 1.57 indicates that the conventional bus is 1.57 faster than the automated shuttle, which is the minimum across all segments. Assessing the speed distributions in the segments 12 to 16 (Figure 11 and Figure 12), the automated shuttle regularly operates at its programmed maximum speed. Furthermore, the speed ratio between both vehicles indicate the lowest advantage for the conventional bus. Thus, these segments could be characterized as most suitable for automated operation.

6.1.4. Segments 17 to 20

Segment 17 is the only section that includes an uphill slope. However, the observed quantities are still very much in line with the results in earlier sections. The segments 18 to 20 can be characterized by their very low width, having to accommodate two-way traffic with a total width of less than 5 m and $3.5$ m. Thus, the programmed speeds are lowest for all segments under analysis to mitigate any risk of collision. In segment 18 the speed distributions of both vehicles show a minimum standard deviation of $0.20$ m/s for the automated shuttle and $0.59$ m/s for the conventional bus. Due to the small width of the street, as well as the bollard, both vehicles operate very consistently at reduced speeds. In contrast to the conventional bus, the distributions of speed for the automated shuttle in Figure 12c–e are strongly one-sided and show a maximum and cut-off at the programmed values, which limit its velocity. In segment 19 the speed ratio indicates that the conventional bus is 2.61 faster than the automated shuttle, which is the maximum ratio across all segments. The speed distribution of the automated shuttle indicates that its operation is limited by the programmed maximum speed and is eventually reduced by objects coming close to the automated shuttle leading to a slow-down. Obviously, the human driver is operating at significantly higher speeds in these narrow streets.

6.1.5. Conclusion—Comparative Speed Analysis

The speed analysis presented in this section leads to these key findings:

• The conventional bus is up to 2.61 times faster than the automated shuttle. The average ratio is 2.02. This is consistent with the total number of Passages (Table 7) for the shuttle and the bus, which indicates a ratio of 2.27.
• Speed histograms indicate a much bigger spread of velocities for the human driver. The standard deviation is typically 2 to 3 times larger compared to the automated shuttle. Furthermore, single-sided distributions rarely occur.
• Traffic flow controls like traffic lights can lead to double-distributions separating runs with and without halt (segment 4) or enforce a significant overlap limiting both vehicles (segment 1).
• Single-sided, asymmetric speed distributions are observed if the automated shuttle regularly reaches its programmed limit (segments 17–20).

6.2. Comparative Acceleration Analysis

In this section, the measurements performed with the external acceleration sensors are compared for the automated shuttle and the conventional bus. The analysis is expected to deliver quantitative, statistically valid, and objective results that allow an assessment of the perceived comfort.

6.2.1. Time-Series Analysis

In Figure 13, a time-based measurement of the TDA in segment 1 are included in (a) for the automated shuttle and in (b) for the conventional Diesel bus with the engine switched on throughout the measurement. The sensor axes were oriented as indicated in Figure 14. The x-axis is aligned to the driving direction (longitudinal acceleration), the y-axis represents the lateral acceleration and the z-axis the vertical acceleration. If the vehicle was aligned perfectly, the gravitational acceleration would be measured on the z-axis only. However, since the vehicleorientation is not controlled precisely, and it is moving on curved surfaces with uphill and downhill slopes, the gravitational acceleration vector can affect all axes of the sensor. To remove the impact of gravitation, imperfections in vehicle- and sensor-orientation, quasi-static sensor offsets, and the impact of the orientation of the street surface, the TDA was introduced as described in Section 5. Figure 13 includes an interval of several seconds with each vehicle standing still, which is marked accordingly. For the automated shuttle, the acceleration values are very close to zero. For the conventional bus; however, the Diesel engine produces a noise baseline, that is detected on all axes. The baseline noise for all axes of both vehicles is summarized in Table 10. The ratio of the total acceleration peak to peak baseline values for the Diesel bus compared to the electric shuttle during standstill is 2.66. An analysis of the time-based measurement in Figure 13b in the frequency domain leads to a fundamental frequency of around 20 Hz which correlates with an engine speed of about 1200 ${\min }^{-1}$. This noise background can be one of the reasons for passengers perceiving the conventional Diesel bus more comfortable since it covers part of the acceleration and vibration created by the moving vehicle. The influence of noise and vibration on comfort in Diesel engine cars running at idle has been confirmed previously in a controlled simulator setup [54]. The evaluation of the longitudinal and lateral accelerations in Figure 15 with respect to the limits given in Table 11 indicate that the automated shuttle is operating in an acceleration zone considered comfortable [36]. However, the acceleration measured for the conventional bus extends beyond what is considered typical for a normal driving style. However, Figure 13 includes only a very short time-frame of about 40 s and does not reflect the outliers identified in the measurements for the automated shuttle (Figure 16, Figure 17, Figure 18 and Figure 19). The histograms of the components of the TDA for the automated shuttle (Figure 15a) are typically much tighter compared to the distributions of the TDA of the conventional bus (Figure 15b) in segment 1.

Figure 13. Time-series of the TDA in the x-, y- and z-direction in segment one of (a) the shuttle (b) the bus (Diesel engine on). Marking lines are set according to Table 11 on the limit comfort-normal and normal-aggressive driving.

Figure 14. (a) Shuttle with data logger setup (b) application-specific data logger (c) installed IMU (d) close-up of the IMU.

Table 10. Typical peak-peak values for the baseline noise level during standstill of the vehicles.

	Automated Electric Shuttle	Conventional Diesel Bus
longitudinal ${\mathrm{a}}_x$ [g]	0.04	0.1
lateral ${\mathrm{a}}_y$ [g]	0.03	0.08
vertical ${\mathrm{a}}_z$ [g]	0.02	0.06

Figure 15. Histograms of the TDA in the x-, y- and z-direction in segment one for the measurement shown in Figure 13. (a) Shuttle (b) Bus (Diesel engine on). Marking lines are set according to Table 11 on the limit comfort-normal and normal-aggressive driving.

Table 11. Driving styles and typical acceleration ranges according to [36]. The gravitational acceleration has been approximated by 1 g = 10 m/${\mathrm{s}}^2$.

	Longitudinal ${\mathbf{a}}_{\mathit{x}}$ [g]	Lateral ${\mathbf{a}}_{\mathit{y}}$ [g]
comfort	−0.09 ... 0.09	−0.09 ... 0.09
normal	−0.2 ... 0.147	−0.4 ... 0.4
aggressive	−0.508 .... 0.307	−0.56 ... 0.56
extremely aggressive	−0.76 ... 0.76	−0.76 ... 0.76
airbag deployment	−0.76 ... −0.56	-

Figure 16. Distribution and range of the TDA of the bus (blue) and shuttle (orange) for the segments 1 to 5.

Figure 17. Distribution and range of the TDA of the bus (blue) and shuttle (orange) for the segments 6 to 10.

Figure 18. Distribution and range of the TDA of the bus (blue) and shuttle (orange) for the segments 11 to 15.

Figure 19. Distribution and range of the TDA of the bus (blue) and shuttle (orange) for the segments 16 to 20.

6.2.2. Statistical Analysis

Since the analysis of the time-series of the TDA does not allow an overall comparative assessment of both vehicles, the complete data set has been investigated based on histograms for each vehicle in each segment, as indicated in Figure 8. The distributions of the TDA per segment is included in Figure 16, Figure 17, Figure 18 and Figure 19 and the results are summarized in Table 12 and Table 13. The typical width of the confidence intervals for a confidence of 95% is approximately $0.004$ g, which indicates a precise and reproducible measurement.

Table 12. Mean acceleration per segment and the confidence interval.

Segment	Automated Shuttle, 95% Confidence			Conventional Bus, 95% Confidence
	${\mathit{TDA}}_{\mathbf{lower}}$ [g]	${\mathit{TDA}}_{\mathbf{mean}}$ [g]	${\mathit{TDA}}_{\mathbf{higher}}$ [g]	${\mathit{TDA}}_{\mathbf{lower}}$ [g]	${\mathit{TDA}}_{\mathbf{mean}}$ [g]	${\mathit{TDA}}_{\mathbf{higher}}$ [g]
1	0.037	0.040	0.042	0.088	0.091	0.093
2	0.044	0.045	0.046	0.068	0.069	0.070
3	0.042	0.044	0.045	0.056	0.057	0.058
4	0.047	0.050	0.051	0.077	0.079	0.080
5	0.044	0.046	0.047	0.057	0.059	0.059
6	0.077	0.077	0.081	0.144	0.149	0.151
7	0.071	0.073	0.074	0.153	0.156	0.158
8	0.039	0.041	0.042	0.092	0.094	0.095
9	0.078	0.082	0.084	0.194	0.198	0.201
10	0.041	0.044	0.045	0.107	0.109	0.111
11	0.044	0.047	0.048	0.103	0.105	0.107
12	0.084	0.088	0.090	0.183	0.187	0.190
13	0.088	0.092	0.095	0.177	0.182	0.186
14	0.080	0.087	0.092	0.211	0.217	0.221
15	0.136	0.141	0.145	0.306	0.317	0.326
16	0.120	0.126	0.130	0.280	0.289	0.296
17	0.075	0.078	0.081	0.174	0.179	0.183
18	0.062	0.064	0.066	0.149	0.151	0.154
19	0.059	0.063	0.065	0.157	0.161	0.165
20	0.068	0.071	0.074	0.159	0.164	0.168

Table 13. Measurement results for the mean, modus, and maximum of the TDA per segment.

Segment	Automated Shuttle			Conventional Bus			Ratio of ${\mathit{TDA}}_{\mathbf{mean}}$ Bus/Shuttle	Ratio of ${\mathit{TDA}}_{\mathbf{max}}$ Bus/Shuttle
	${\mathit{TDA}}_{\mathbf{mean}}$ [g]	${\mathit{TDA}}_{\mathbf{modus}}$ [g]	${\mathit{TDA}}_{\mathbf{max}}$ [g]	${\mathit{TDA}}_{\mathbf{mean}}$ [g]	${\mathit{TDA}}_{\mathbf{modus}}$ [g]	${\mathit{TDA}}_{\mathbf{max}}$ [g]
1	0.04	0.01	0.81	0.09	0.05	0.64	2.35	0.79
2	0.05	0.03	0.85	0.07	0.05	0.55	1.40	0.65
3	0.04	0.03	0.77	0.06	0.05	0.55	1.50	0.71
4	0.05	0.04	0.87	0.08	0.05	0.74	1.60	0.85
5	0.05	0.04	0.89	0.06	0.05	0.31	1.20	0.35
6	0.08	0.04	0.89	0.15	0.07	1.49	1.88	1.67
7	0.07	0.03	1.08	0.16	0.06	1.44	2.29	1.33
8	0.04	0.02	1.26	0.09	0.05	1.26	2.25	1.00
9	0.08	0.07	0.88	0.20	0.14	1.49	2.50	1.69
10	0.04	0.02	0.89	0.11	0.05	1.44	2.75	1.62
11	0.05	0.02	0.85	0.11	0.05	1.26	2.20	1.48
12	0.09	0.07	0.68	0.19	0.11	1.49	2.11	2.19
13	0.09	0.07	0.89	0.18	0.07	1.27	2.00	1.43
14	0.09	0.01	0.72	0.22	0.15	1.49	2.44	2.07
15	0.14	0.01	1.29	0.32	0.21	1.45	2.29	1.12
16	0.13	0.01	1.33	0.29	0.17	1.49	2.42	1.12
17	0.08	0.06	0.63	0.18	0.07	1.49	2.25	2.37
18	0.06	0.05	0.77	0.15	0.10	1.29	2.50	1.68
19	0.06	0.05	0.73	0.16	0.10	1.49	2.67	2.04
20	0.07	0.06	0.76	0.16	0.11	1.44	2.28	1.90

6.2.3. Segments 1 to 5

The distributions for the absolute value of the TDA are shown in Figure 16 and descriptive values are included in the first five lines of Table 12 and Table 13, respectively. Compared to the conventional bus, the main part of the histograms of the automated shuttle are much tighter and the mean and modus are located at lower values. However, the maximum values for the automated shuttle exceed those of the conventional bus significantly in these first segments. Thus, in certain situations, passengers of the automated shuttle are subject to these outliers, which can lead to a significant reduction in perceived comfort. This observation is confirmed, when comparing the ratio of the mean values to the ratio of the maximum values of the TDA, which are included in the last two columns of Table 13. For the segments 1 to 5, the mean TDA for the conventional bus is larger compared to the automated shuttle, while the maximum observed TDA is larger for the automated shuttle. The ratio of the mean values of 2.35 (Table 13) for segment 1 is significantly higher, than the values for segments 2 to 5, which indicates that the driver regularly strongly accelerates to reach a green traffic light. For the segments 2 to 5, the distributions of the TDA for bus and shuttle show a significant overlap. Specifically, for the segments 3 and 5, the main distributions cover an almost identical range of dynamic acceleration. Both segments are part of the asphalt-covered main road without any flow control. This observation coincides with the analysis presented in Table 9, which indicates a comparably smooth flow with the shuttle reaching its highest speed of $4.97$ m/s in segment 3, and the bus reaching a speed of $11.99$ m/s in segment 5. For both segments, this correlates with a lognormal distribution, indicating an undisturbed path of the vehicle through the segment. Furthermore, when comparing the mean, the modus, and the maximum of the TDA across Table 13 for both vehicles, the mean and modus show values between $0.01$ g and $0.09$ g, while the values for the maxima were measured in an interval between $0.31$ g and $0.89$ g. This confirms that the distribution of TDA-values consists of a main part, separated from outliers at much higher values. The segments 1 to 5 are the only segments distributed along a main road paved with asphalt, while the average speeds are highest (Table 9), the TDAs observed are low compared to other segments (Table 13).

6.2.4. Segments 6 to 11

In the segments 6 to 11, the highest modus value of the TDA for the shuttle of $0.07$ g was measured in segment 9. The histogram in Figure 17 shows indications of a double distribution with a shoulder at about $0.03$ g. This can be explained by the structure of segment 6, where the vehicle leaves the main road and enters an area with a cobblestone pavement. The road in the segents 6 to 11 consists of a downhill slope. However, its impact has been removed by calculating the TDA, which removes any constant offset in acceleration. For the conventional bus, the maximum TDA of $1.49$ g was found in the segments 6 and 9. However, a review of Figure 17a,d clearly indicate that these are outliers with values far above the main distribution. Specifically, in segment 9 these outliers can be caused by a speed bump (Table 4).

6.2.5. Segments 12 to 16

In the segments 12 to 16, the histograms of the TDA of the automated shuttle mostly consist of double distributions, including a broader part with a modus value around $0.07$ g and a peak around $0.1$ g. The lower peak is expected to result from a significant time of standstill of the automated shuttle bus. The exception is segment 14, with a peak containing almost 35% of all measured values at $0.01$ g. This is caused by extended periods of standstill at the bus stop in this segment. The lower peak of the TDA for the automated shuttle in Figure 19a is related to regular stops due to parking vehicles blocking the programmed path. The highest value of the TDA of $1.33$ g for the shuttle was measured in segment 16 while the highest mean value of $0.14$ g was found in segment 15. For the segments 11 and 14, this observation correlates with the existence of planned stops. In segments 12, 13, and 15, operation of the shuttle is interrupted by vehicles parking in the route of the shuttle, which typically causes breaking, standstill and potentially manual intervention. For the conventional bus; however, all distributions show one clear maximum. Maximum values of $1.49$ g were measured for the bus in the segments 12, 14 and 16 and the highest mean and modus values were found in segment 15. In summary, this group of segments shows the highest density of extreme values for the TDA due to the cobblestone pavement. Even though the values for the conventional bus are significantly higher than for the shuttle, the bus may still be perceived as more comfortable due to the coverage by Diesel engine baseline vibrations.

6.2.6. Segments 17 to 20

In segments 17 and 18, the measurements obtained for the automated shuttle indicate double distributions, potentially related to periods of standstill caused by the stop in segment 17 and the bollard in segment 18. Mean and modus values are in a similar range, maximum values are typically a factor of 10 higher, which is consistent with the other groups of segments. The lower shoulder observed in the TDA of the automated shuttle can be caused by regular stops due to operation in two-way traffic on a narrow road. For the conventional bus, values similar to segments 12 to 16 have been observed, including a maximum of $1.49$ g for segments 17 and 19.

6.2.7. Analysis of the Form of the Statistical Distributions

For both the automated shuttle and the conventional bus, the distributions in Figure 16, Figure 17, Figure 18 and Figure 19 have been reviewed for the form of their envelope. An example is given in Figure 20 for each vehicle. In both cases, an almost perfect fit could be achieved for a lognormal distribution [55]. The distributions have been reviewed for each vehicle on every segment and the result is summarized in Table 14. For the conventional bus, most distributions follow the form of a lognormal distribution. In rare cases (segments 1, 13, and 17) shoulders or double distributions indicate special operating conditions interrupting the normal flow of operation. For the automated shuttle, very steep contributions close to 0 g indicate significant times of standstill or operation with a very limited speed. The huge number of identified double distributions is an indicator of interruptions or switch-overs to separate modes of operation within a segment. They are expected to correlate with a reduced comfort compared the conventional bus. The results of this analysis are summarized in Table 14.

Figure 20. The distributions of the TDA for the (a) the automated shuttle in segment 11 and (b) the human-operated bus in segment 8. Both distributions have been approximated by a lognormal distribution and the parameters are included with the graph.

Table 14. Summary of the form of the distributions of the TDA for both vehicles (S: shuttle, B: bus).

Segment	Automated Shuttle	Conventional Bus	Comment
1	one-sided, several shoulders	lognormal, slight shoulder	TL
2	lognormal	lognormal	undisturbed operation
3	lognormal	lognormal	undisturbed operation
4	significant contribution at 0.0 g	lognormal	TL
5	lognormal	lognormal	undisturbed operation
6	lognormal, shoulder	lognormal	leaving main road, mixed pavement
7	double distribution	lognormal	speed-bump
8	steep lognormal	lognormal	undisturbed operation
9	double distributions	broad, lognormal	speed-bump
10	steep lognormal	lognormal	speed-bump
11	steep	lognormal	stop (S, B)
12	double distribution	lognormal	speed-bump, parking vehicles
13	double distribution	double distribution	stop (B), parking vehciles
14	very steep contribution at 0.0 g	very broad distribution	extended standstill (S), stop (S, B)
15	double distribution	very broad distribution	parking vehicles
16	double distribution	broad, lognormal	stop (S)
17	double distribution	double distribution	stop (S, B)
18	lognormal	lognormal	tight road, automated bollard
19	lognormal	lognormal	tight road
20	lognormal, shoulder	lognormal	tight road

6.2.8. Conclusion—Acceleration Analysis

The analysis of the TDA presented in this section leads to these key findings:

• The concept of the TDA allows the automated analysis of acceleration sensor data. Offsets, including the effect of gravitational acceleration for up- or downhill slopes are successfully removed without the need for manual data conditioning.
• For a sufficient analysis and to conclude on the quality of the ride, the main distribution and existing outliers must be considered. These outliers are related to the subjectively perceived comfort and offer an opportunity to create a quantitative and objective proxy for perceived comfort. Based on the data set for this study, the main distribution of the TDA indicates lower values for the automated shuttle compared to the conventional bus. However, the measurements for automated shuttle contain outliers significantly surpassing the bus. Interviews, which were conducted with passengers of the automated shuttle, indicate that comfort and braking should be improved [25].
• The form of the distribution of the histogram of the TDA is related to the quality of operation of the vehicle.
• Double distributions in the histograms of the TDA indicate multiple modes of operation, like standstill and driving, within one segment.
• A lognormal distribution of the TDA is an indicator for flawless operation without interruptions.
• The analysis of the form of the distribution of the TDA for every segment indicate a lognormal-behavior of the human driver, while a significant number of double distributions and shoulders in the probability density indicate an operation of the shuttle, including a switch-over from automated to manual operation or regular standstill. Thus, double distributions of the TDA can be a valid indicator for reduced performance and reduced comfort and a trigger point to define and initiate corrective actions.

6.3. Impact of Vehicle Type and Size

This analysis is based on two existing vehicles with given technical parameters. The conventional bus has a much higher maximum speed and a larger vessel size, as indicated in Table 5. However, the vehicle footprint is larger compared to the automated shuttle. While the maximum speed of the regular bus is never applied, it is still typically operated at higher speeds, than the automated shuttle. With a technically defined maximum speed of 25 km/h ($6.9$ m/s), the automated shuttle could actually reach the mean speeds of the conventional bus for all but segments 3, 4, and 5 (Table 9). Thus, the lower speeds of the automated shuttle are generally not limited by the maximum speed of the vehicle, but by the performance of the automation in contrast to the human driver. As indicated in Section 6.2, the main distributions for the TDA of the automated shuttle indicate lower values compared to the conventional bus. However, the outliers are significantly higher. Since the Diesel engine of the conventional bus provides a noise floor covering human perception, the outliers observed for the automated shuttle are potentially perceived as much more disruptive. Since the conventional bus offers room for up to 61 passengers, which is about 5 to 6 times more than the automated shuttle, the overall transportation, including the higher average speed performance is about 12 times as high.

6.4. Summary of Experimental Results

The experimental results presented in the previous sections were analyzed with a focus on a comparison between both vehicles. The transportation performance was assessed based on the mean vehicle speed for every segment. The results would have to be multiplied with the number of available (or taken) seats to generate a Key Performance Indicator (KPI) and compare final transportation performance. For this study, the vessel size was treated as a given characteristic. The width of the distribution of mean speeds per segment delivers an indicator for reproducible transportation leading to the punctuality as a KPI. Finally, the TDA was analyzed to assess perceived comfort in an objective and quantitative way. Another perspective on the results is included in Table 15. Based on the assessment, segment 18 shows optimal results in terms of comfort and punctuality, and segments 3 and 5 the best results in terms of transportation performance.

Table 15. Summary of experimental results.

Related Performance Metric	Measured Indicator	Observation Shuttle (Segment)		Observation Bus (Segment)
		Lowest Value	Highest Value	Lowest Value	Highest Value
Transportation Performance	Mean Speed (higher is better)	$1.19$ m/s (1, 19)	$4.34$ m/s (3)	$1.92$ m/s (1)	$8.17$ m/s (5)
Punctuality	${\sigma }_v$ (lower is better)	$0.20$ m/s (18)	$0.64$ m/s (1)	$0.59$ m/s (18)	$1.88$ m/s (4)
Comfort	$TD{A}_{\max }$ (lower is better)	$0.63$ g (18)	$1.33$ g (1)	$0.31$ g (18)	$1.49$ g (4)

7. Conclusions and Outlook

In this study, speed and acceleration measurements have been performed to benchmark an automated, electric shuttle to a conventional Diesel bus on the same track in a complex, urban environment. The track was geographically divided into segments which contained typical elements causing difficulties for both the automation and the human driver. The TDA was introduced to automatically remove all kinds of offsets and deliver valid information for the vehicle dynamic acceleration. On average, the human driver was 2.02 times as fast as the automated shuttle and applied a much bigger spread of speeds on the segments. The mean values of the TDA for the human driver showed average values which were higher by a factor of 2.38. However, the maximum values of the TDA of the automated shuttle exceeded those of the human driver significantly for certain segments, which is an indicator of sudden decelerations and interruption of the automated operation. The analysis of the envelope of the distribution of the TDA shows a strong correlation to obstacles in the path and a limited operational robustness of the automated shuttle. The questions of research have been addressed and answered:

• How can objective, sensor-based data be used to benchmark automated shuttles against conventional buses?
  The measurement of speed and acceleration with reference to each segment allows an assessment of performance and identification of causes. The results can be utilized as a key performance indicator within a Plan–Do–Check–Act cycle.
• How does an automated shuttle perform compared to a conventional bus on a complex, urban track?
  Based on the results obtained, the performance of a human driver is about a factor of 2 better than the automation in the shuttle bus under investigation. This value can be used as a starting point for planning the changeover from conventional buses to automated shuttles. The observed outliers in the TDA indicate a reduced comfort of the automated shuttle.
• Which characteristics have the largest impact on speed and comfort of the automated shuttle? The measurements of the TDA show a lognormal distribution for the automated shuttle whenever a continuous, smooth operation was achieved. Specifically in segment 3, the shuttle also reached its maximum mean and peak speed and the speed ratio bus/shuttle was 1.81. Thus, an environment with a minimum of random obstacles is best for automation in PT.

This objective and quantitative, comparative study was based on a rich experimental data set on a defined track with predominantly human traffic, and given vehicles. While methodology presented can applied to other tracks, including mixed traffic flows, the specific results are valid for this track only. Thus, next generation automated shuttles on a track with different geometries, flow controls and speed limits will lead to different overall results, while the focus of the underlying project Tabula [25] was a prototypical implementation, including all relevant players, it did not quantitatively address the passenger comfort. Thus, further studies should include a combination of measurements with correlated, quantitive customer feedback to confirm the correlation to outliers in the TDA. Furthermore, both methodology and experimental results can provide input for improving the performance of the driving system of automated vehicles with respect to safety, efficiency, and personal tolerance [56]. On top of that, the results obtained can provide a valuable add-on to existing methods for the assessment of safety of automated vehicles [57].

The results presented here are a base for an automated analysis of the performance of an automated shuttle. The human driver continues to be the benchmark for improving automation in PT.

This objective and quantitative, comparative study was based on a rich experimental data set on a defined track with predominantly human traffic, and available vehicles. While methodology presented can applied to other tracks, including mixed traffic flows, the specific results are valid for this track only. Thus, next generation automated shuttles on a track with different geometries, flow controls and speed limits will lead to different overall results, while the focus of the underlying project Tabula [25] was a prototypical implementation, including all relevant players, it did not quantitatively address the passenger comfort. Thus, further studies should include a combination of measurements with correlated, quantitive customer feedback to confirm the correlation to outliers in the TDA. Furthermore, both methodology and experimental results can provide input for improving the performance of the driving system of automated vehicles with respect to safety, efficiency, and personal tolerance [56]. On top of that, the results obtained can provide a valuable add-on to existing methods for the assessment of safety of automated vehicles [57].