1. Introduction
Given the high number of vehicle-crash victims, it has been established as a priority to reduce this figure in the transportation sector. For this reason, many of the recent research works are focused on including different control systems in commercial vehicles in order to improve their stability, comfort and handling [
1,
2,
3]. These systems need to know in every moment the dynamics of the vehicle through variables such as longitudinal and lateral accelerations, yaw rate, roll rate, roll angle and sideslip angle, among others, when different maneuvers are performed, in order to actuate by means of the systems in the vehicle (brakes, steering, suspension) and, in this way, achieving a good behavior.
IMU sensors directly provide measurements of angular rates (yaw, roll and pitch rates) and accelerations (longitudinal, lateral and vertical accelerations) via rate gyroscopes and accelerometers. However, there are some difficulties in obtaining the vehicle angles (sideslip, roll and pitch angles) directly from sensors. A GPS dual-antenna system provides measurements of these angles, but this technique is very expensive. For this reason, this system is not mounted on commercial vehicles. Nowadays, many researches are focused on the design of observers in order to estimate these variables using the data provided by sensors that are installed on current vehicles, or by low-cost devices in order to solve this problem. In some studies, the vehicle roll angle is estimated by fusing the data of longitudinal acceleration, lateral acceleration, yaw rate and roll rate [
4,
5], through the fusion of lateral accelerations and roll rate [
6], fusing the data obtained from an inertial angle sensor and a gyroscope [
7], fusing the information obtained from a six-dimensional IMU [
8] or fusing the information provided by a low-cost GPS with the data provided by a wheel speed sensor, steering angle sensor and a yaw rate sensor [
9]. Regarding these works, the most common sensors used to estimate the vehicle dynamics are accelerometers and gyroscopes, which are integrated in an IMU sensor.
Concerning the estimation of vehicle sideslip angle, in [
10], a sensor fusion of lateral acceleration, longitudinal velocity and yaw rate is carried out. In [
11], the sideslip angle is estimated through the yaw rate. In [
12,
13], the estimation of sideslip angle is obtained through the lateral acceleration and yaw rate.
In [
14], both the roll angle and sideslip angle are estimated using the measurements provide by sensors installed on commercial vehicles, such as the longitudinal velocity, the wheel speed, the steer angle, the lateral acceleration and yaw rate.
The observers’ design to estimate the vehicles angles is based on different techniques. These techniques are based on artificial intelligence [
4,
10], Kalman Filter [
4,
8,
10], Bayesian Filters [
9], sliding mode observer [
6] and robust observers [
5,
11,
13]. All these observers must work under real-time constraints.
Nowadays, some researches analyze and design embedded systems for vehicles’ applications [
15,
16,
17]. With the objective of developing vehicle on-board systems, it is necessary that they have a small size, a small response time and enough accuracy. Another characteristic is that they must be low-cost in order to maintain the price of production vehicles. The use of small computers with a high processing capacity and a number of input/output interfaces allows incorporating into vehicles both estimators and controllers, increasing their safety. Previous research [
18] analyzed the integration of low-cost sensors in vehicles. This research indicates that this kind of low-cost kit could present relevant bias and noise that need to be studied experimentally. Even more, this research provides recommendations for packaging these devices with plastic boxes. This research supports the need and opportunity for the research discussed in this article.
Small computers are used as data acquisition systems. Nowadays, the most popular small single-board computers are the Raspberry Pi and Intel Edison. They have a low price, flexibility and have high support from the creators and Internet community developing new libraries to give solutions for complex necessities. The use of these devices has been increasing in recent years.
The Intel Edison is meant to be a deeply-embedded IoT computing module. Although it is not a single-board computer, there are many studies that use it like an acquisition and processing system. Like Raspberry Pi, Intel Edison is very popular, and there are many resources to improve the capabilities of this processor. In [
19], a prototype implementation based on the Intel Edison was used to verify an identity-based data aggregation protocol. In [
20], real-time vital parameters on neonates were acquired using Intel Edison and IoT integration with biomedical devices. In [
21], Intel Edison was used to develop a neurofeedback system to enable anyone with an attention deficit to practice regulating their brain to reach an attentive state of mind. In [
22], Intel Edison was used to create a prototype to analyze and process geospatial data.
In [
23], a Raspberry Pi was used to acquire the riding dynamics of a human-powered vehicle. In [
24], a Raspberry Pi was used to develop an virtual laboratory, which allows the sensor and data fusion to be employed in autonomous, robotic and transportation systems. In [
25], a low-cost contact angle goniometer/tensiometer was constructed based on a Raspberry Pi. In [
26], a voltage regulation system was proposed for photovoltaic energy sources based on Raspberry Pi. The Raspberry Pi managed the data acquisition and controlled the system. In [
27], the Raspberry Pi was used as a gateway for an efficient integration of wireless sensor networks with the Internet through IoT.
As can be seen, in previous works, these small computers are used for very specific works. In [
28], the comparison of some of their inherent capabilities, such as connectivity and consumption, was performed under static conditions.
The novelty reflected in this work is the analysis of low-cost small computers and sensors under high dynamic conditions, by following specific best practices and considerations. The set of these devices is integrated with a specifically-designed IoT architecture oriented towards smart vehicles.
One of the main results of this research is the design of an architecture based on IoT, which integrates low-cost sensors and a small single-board computer to be mounted on a vehicle in order to carry out the estimation and control of vehicle dynamics, comparing the results with a high-end professional device. In this work, the low-cost sensor considered is an IMU due to it being the most common sensor used to estimate the vehicle dynamics. This research is focused on the comparison of two different low-cost systems (Raspberry Pi with IMU BNO0055 and Intel Edison with IMU LSM9DSO) in terms of accuracy, acquisition time and reliability. Both devices are going to be compared with the VBOX device from Racelogic, which will be used as the ground truth.
This article is organized as follows. In
Section 2, the methodology is presented, including the experimental testbed design, experiments’ definitions, and the data gathering and analysis, the experimental results and the calculation of the RMS error are presented in
Section 3. Finally, in
Section 4, the discussion and conclusion of the results and the method are exposed.
2. Methodology
In this section, firstly, the design of the experimental testbed defined to achieve the research goals is presented; then, the experiments to gather the data required are specified; and finally, the data gathered to analyze low-cost sensor kits’ performance and reliability are presented, indicating the data analysis strategies for identifying relevant results and conclusions.
2.1. Experimental Testbed Design
The design of the experimental testbed considered for this research work is composed of three perspectives: hardware, software and communications.
Hardware perspective:
The experimental testbed is an Internet of Things (IoT) architecture embedded in a vehicle. Although the complete architecture is packaged in a product that can be integrated in any vehicle, the Ford Fiesta was used.
To perform the comparative analysis properly, three kits of sensors were considered:
- –
Reference or ground truth kit. This is composed of a VBOX 3i dual antenna data logger [
29] having an IMU (Inertial Measurement Unit) connected and a dual antenna for incremental GPS from Racelogic. To accurately measure the roll rate, the two antennas form a 90-degree angle with respect to the traveling direction. For IoT kits’ synchronization and data gathering, the VBOX controller is connected to a laptop embedded in the vehicle. The actual specifications of the ground truth kit hardware elements are detailed in
Table 1.
- –
First low-cost sensor kit composed of a Raspberry Pi 3 Model B [
30,
31], low-cost Inertial Measurement Unit Shield [
32]. The actual specifications of the Raspberry kit hardware elements are detailed in
Table 2.
- –
Second low-cost sensor kit composed of an Intel Edison System on Chip [
33] linked to a SparkFun 9 Degrees of Freedommodule [
34]. The actual specifications of the Intel kit hardware elements are detailed in
Table 3.
The IMU and the low-cost sensor kits were located in the gravity center of the vehicle considered in the testbed as is represented in
Figure 1. These three kits were also interconnected using a WiFi router to manage the communications among them to synchronize the experiments and gather the information ready to be compared. According to [
18], the accurate determination of IMU and controller position is essential for enhancing the precision of low-cost kits.
Software perspective:
A software architecture was designed to gather the data provided by the sensor kits in a synchronized way and to provide the datasets necessary to analyze the precision and performance of each kit considered for this research work. The main components of this architecture are shown in
Figure 2).
The Experiments Manager Componenthas the responsibility to synchronize the data-gathering of the kits taking part in the experiments. The specific classes included in this component are:
Experiments Managerthat includes the user interface to let the researcher send a signal to the experimental kits for starting and finishing an experiment in a synchronized way.
Communications Client that has the responsibility of managing the communication between the Experiments Manager and the Communications Server running in each experimental kit. The possible signals that can be sent from the Communications Client in the Experiments Manager and the Communications Servers in experimental kits are: 0, shutdown experimental kit; 1, keep running the experiment; 2, start the experiment; and 3, end the experiment and save the data in a file.
Dataset Storage Manager that is in charge of taking the data coming from the kits and storing them in a CSV file. The information stored includes gyroscope and accelerator data gathered considering a sampling rate of 100 Hz.
The Intel Edison Software component is in charge of gathering the information provided by gyroscope and accelerator sensors included in its hardware architecture. This component was implemented in C++. The specific classes included in this component are:
The NTP Client is in charge of registering the actual date-time in the hardware controller of the experimental kit to ensure that all the kits in the testbed have the same date-time, supporting comparison during the data analysis stage in this research work.
Communications Server that is in charge of receiving the signals from the Experiment Manager and sends the Experiment Launcher the order to start a new experiment. Moreover, this class sends the experiments’ ending order to the Experiment Stopper class to process the information gathered during the experiment.
The Experiment Manager class is in charge of creating an empty data structure in RAM memory when it receives the start experiment signal from the communications server and sending the data structure having the data gathered during an specific experiment to the Experiments Manager for storage purposes when it receives the end experiment signal.
The Sensors Handler class has the responsibility of registering data items from sensors attending to the pre-configured sampling rate (100 Hz for this research work). The data are obtained from the gyroscope and accelerometer drivers that were obtained from GitHub [
35,
36].
The Raspberry Pi Software component has the same class structure as the Intel Edison Software Component, but it was developed in Python because the available drivers for low-cost IMU were only available in this language.
The VBOX Component is in charge of gathering the information provided from the Racelogic IMU sensor and GPS dual antenna data.
Communications perspective:
In order to provide homogeneity in communication among the different items that compose the solution and ensure synchronization when performing the different tests, a communications architecture has been defined.This architecture is shown in
Figure 3.
Given the nature of Racelogic VBOX devices, they need to be physically connected via a cable to the experiments managerand among themselves. However, both Raspberry Pi 3 and Intel Edison come with wireless communication interfaces that ease the connectivity among components and allow one to locate them in virtually any place of the vehicle without worrying about setting up specific data cables. Even more, the sensors used by these low-cost platforms are straightforwardly attached to the development boards by using the GPIO ports. By means of a wireless (802.11 g) access point, they can be connected to the experiments manager, which signals to them their operation mode via a TCP socket connection.
2.2. Experiments Definition
The hypotheses to evaluate during this research work are the following:
H1: The precision of low-cost sensor kits is similar to the precision provided by expensive experimental kits (i.e., VBOX-based kits).
H2: The performance and reliability of the low-cost sensor kits (i.e., Raspberry Pi and Edison Kits) is similar to the performance and reliability provided by expensive experimental kits (i.e., VBOX-based kits).
To evaluate the previous hypothesis, eight controlled experiments were executed (see
Table 4. The experiments considered typical maneuvers such as J-turn and lane change maneuvers. These maneuvers are the most common used to test the vehicle’s behavior. Furthermore, a long test simulating a general execution has been done. In the experiments carried out, the lateral acceleration and the roll rate are the variables that have suffered higher variation, and for this reason, these variables are considered in order to analyze the accuracy of different devices.
The experimental tests have been carried out in Leganes (Madrid, Spain) on a commercial vehicle and a Ford Fiesta, as can be seen in
Figure 4 in a period were the setting had no traffic restrictions interfering with the appropriate execution of the considered experiments.
2.3. Data Gathering and Analysis
The data obtained for the sensors considered for the previously-defined experiments were: For each experiment, the controller of each kit stored a CSV formatted file identifying the experiment and its execution date and time. The variables considered were the lateral acceleration measured by the accelerometer and the roll rate measured by the gyroscope included in each kit for the three axes. The measures were gathered according to the sampling rate stated for the experiments, which was 100 Hz.
Figure 5 presents an example of the data gathered in each experiment.
To determine the accuracy of the values obtained from low-cost kits, the root mean square (RMS) error was calculated. The maximum error comparing each point has also been calculated. This comparisons has been done using Racelogic VBOX as the ground truth.
Section 3 presents the results obtained in the different tests.
2.4. Threats to Validity
In order to analyze the validity of the results obtained in this experimental work, several threats were considered prior to the experiments’ execution:
Internal validity is the extent to which a causal conclusion based on the experiments defined is appropriate avoiding the introduction of systematic errors in the data used to determine the results and conclusions.
External validity is the extent to which the results of a study can be generalized to other situations.
(A) Internal validity: In this research, the only factor contributing to internal validity is related to the specific sensors used to configure each low-cost kit and the software components implemented to manage the required data:
The first threat was mitigated using two different kits for each type in order to prevent errors produced by sensors providing incorrect values. Even more, all the kits considered were tested in a static environment configuring the corresponding calibrations to assure that specific sensors included were providing appropriate data.
Regarding software components, the possible threats were mitigated implementing an exhaustive unit testing process to ensure that each functionality properly processes the received values and the synchronization among devices is correctly implemented.
To verify the validity of the results, three similar tests for J-turn and double lane change maneuvers have been carried out.
(B) External validity: In the scope of this research, the factors that influence the external validity are related to the replication of this experiment. These replications must consider several relevant factors: sensors and controllers included in the experimental kits, their location in the vehicle and road conditions:
Regarding sensors and controllers, the possible threats were mitigated using sensors and controllers available on the market having regular features [
37,
38,
39]. In this sense, the conclusions obtained are valid for the low-cost sensors currently available on the market, and as the technology is always improving the prior conditions, the conclusions can be used for future low-cost sensors.
Regarding vehicle conditions, the threats were related to the appropriate location of experimental kits in order to ensure equal conditions among them. This threat was mitigated creating a box to put the three sensor kits in a 3D printed box that was located in the vehicle gravity center. These issues are important to replicate this experiment properly in other vehicles. In this research, as the experiments have been carried out with a real vehicle, it has been difficult to maintain the same exact driving conditions related to steering wheel angle and velocity. In spite of this problem, similar tests for typical maneuvers have been carried out.
Regarding road conditions and experiments execution, the threats were related to the representativeness of the scenarios considered. This threat was mitigated considering a road without relevant slope variations and including different experiments with different types of directions, constant and variable speed.