Next Article in Journal
Radiometric Top-of-Atmosphere Reflectance Consistency Assessment for Landsat 8/OLI, Sentinel-2/MSI, PROBA-V, and DEIMOS-1 over Libya-4 and RadCalNet Calibration Sites
Previous Article in Journal
Synergistic Modern Global 1 Km Cropland Dataset Derived from Multi-Sets of Land Cover Products
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Remote Sensing
Volume 11
Issue 19
10.3390/rs11192252
remotesensing-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Sensor Reliability in Cyber-Physical Systems Using Internet-of-Things Data: A Review and Case Study

by
Fernando Castaño
1,*,
Stanisław Strzelczak
2,
Alberto Villalonga
1,
Rodolfo E. Haber
1 and
Joanna Kossakowska
2
1
Centre for Automation and Robotics (CSIC–UPM), Spanish National Research Council, Arganda del Rey, 28500 Madrid, Spain
2
Faculty of Production Engineering, Warsaw University of Technology, 02-524 Warsaw, Poland
*
Author to whom correspondence should be addressed.
Remote Sens. 2019, 11(19), 2252; https://doi.org/10.3390/rs11192252
Submission received: 26 July 2019 / Revised: 21 September 2019 / Accepted: 25 September 2019 / Published: 27 September 2019
(This article belongs to the Section Engineering Remote Sensing)
Browse Figures
Review Reports Versions Notes

Abstract

:
Nowadays, reliability of sensors is one of the most important challenges for widespread application of Internet-of-things data in key emerging fields such as the automotive and manufacturing sectors. This paper presents a brief review of the main research and innovation actions at the European level, as well as some on-going research related to sensor reliability in cyber-physical systems (CPS). The research reported in this paper is also focused on the design of a procedure for evaluating the reliability of Internet-of-Things sensors in a cyber-physical system. The results of a case study of sensor reliability assessment in an autonomous driving scenario for the automotive sector are also shown. A co-simulation framework is designed in order to enable real-time interaction between virtual and real sensors. The case study consists of an IoT LiDAR-based collaborative map in order to assess the CPS-based co-simulation framework. Specifically, the sensor chosen is the Ibeo Lux 4-layer LiDAR sensor with IoT added capabilities. The modeling library for predicting error with machine learning methods is implemented at a local level, and a self-learning-procedure for decision-making based on Q-learning runs at a global level. The study supporting the experimental evaluation of the co-simulation framework is presented using simulated and real data. The results demonstrate the effectiveness of the proposed method for increasing sensor reliability in cyber-physical systems using Internet-of-Things data.

Graphical Abstract

1. Introduction

Nowadays, the precise knowledge of the most appropriate sensor operating conditions and fault detection systems are among the cornerstones of scientific and technical studies for automated systems [1]. These are based upon on-line monitoring processes and additional comprehensive interpretation of sensor data by assessing sensor reliability. Sensors are driving the rapid growth of Cyber-Physical Systems (CPSs) and the Internet of Things (IoT) [2]. Both paradigms are pushing towards the next generation of sensor networks and unpredictable future applications, meaning that sensor reliability has become one of the most important and desirable performance indicators in the design, implementation, and deployment of future sensor networks [3,4].
An important reliability-related issue to be detected in autonomous systems, in order to self-correct problems such as lost data packages, and data collision, among others, is the failure of one network element [5,6,7]. One possible solution is to build real-time prediction models that maximize robustness and lifetime [8]. There are, in fact, several methods for the evaluation of sensor reliability. Each issue of reliability, or that might affect it, can be assessed individually and as a whole, through a total error band figure. There are important features to be considered, such as sensitivity, range, precision, resolution, accuracy, offset, linearity, dynamic linearity, hysteresis and response time [9,10]. Evaluating sensor reliability includes probabilistic and statistical data that increase estimation reliability [11]. Evidence theory can be used, such as the Dempster-Shafer theory of belief functions. Quantifying reliability implies predictions concerning sensor lifetime and failure probability. Reliability can therefore be based on both statistical and Artificial Intelligence (AI) models. Suitable probability functions must be defined, which will be used to calculate the future behavior of devices, based either on carefully controlled laboratory experiments or on thorough failure analysis while in use. A typical product will be liable to various failure modes that change over time in a characteristic manner, so that the probability functions are themselves time dependent.
Currently, two of the most widely applied sensors are the 3D stereoscopic camera and the LiDAR [12,13]. Dozens of new applications have recently been reported [14,15,16]. However, the computational load and the processing time for data fusion of 3D stereoscopic camera and the LiDAR sensors in automotive applications are very high [17]. Therefore, a possible solution is to use only the point cloud provided by the LiDAR with a model for predicting the reliability of the sensor data and only activate the support sensors when a possible failure is anticipated [18].
Indeed, it is very difficult to guarantee the real topology and distance of objects with less uncertainty due to dead zones, object transparency, light reflection, weather conditions, and sensor failures, among others [19]. It is mandatory to obtain data in parallel from other connected LiDAR sensors to guarantee reliability. Therefore, new IoT-based technologies open up a wide range of methodologies for estimating and ensuring reliability in key emerging applications [20].
The most widely used techniques for modelling predictions in product lifetime and failure probability are probabilistic methods. Probabilistic methods for uncertain reasoning represent another group of techniques. Probability theory predicts events from a state of partial knowledge, while possibilistic models, represented by fuzzy systems, are applied to situations with intrinsic vagueness and uncertainty.
However, the prediction techniques are hardly limited to those mentioned above. Several clustering techniques such as nearest neighbor methods have been explored, in order to enable self-detection and self-correction capabilities [21]. Other capabilities to be considered from the perspective of reliability are self-adaptation and self-organization by embedding artificial neural networks (ANNs) in CPSs [22]. Efficient performance of multiple sensors and their online monitoring and self-correction procedures, through the application of machine learning (ML), such as Support Vector Machines (SVM) and ANNs, are essential for reducing maintenance costs, risk associated with uncalibrated and faulty sensors, increasing sensor reliability and, consequently, extended equipment life [23,24]. With the aim of guaranteeing certain safety and security conditions in some critical applications, the verification of sensory data and subsequent data evaluation are described in this paper on the basis of simulation of virtual and real scenarios, as well as a framework that properly combines both scenarios.
A reliability assessment procedure is therefore described in this paper that is applicable to data captured by IoT LiDAR sensors in automotive applications: LiDAR self-testing methodology. The reliability analysis is based on the paradigm of cyber-physical systems (CPS) by distributing nodes locally and globally, as will be explained later on. Each computing node has data-processing methods and machine-learning models for reliability prediction. In addition, a run-time self-learning and decision-making model runs within a global node, in order to determine the best model and the model updating mechanism on request.
This paper will be organized into five sections. Following this introduction, the second section will present a review of the state-of-the-art of the CPS-based reliability concept for sensor system reliability using AI methods. Subsequently, the specifications and the requirements obtained from the review of CPS reliability frameworks will be summarized in Section 3. A particular implementation of a CPS-based co-simulation framework will also be proposed in this section. In addition, a case study for the evaluation of an IoT sensor network using a CPS-based co-simulation framework approach will be described in Section 4. In that section, the experimental results and a discussion relating to a comparative study will also be addressed. Finally, the conclusions and future research steps will be presented in Section 5.

2. Reliability of Sensors in Cyber-Physicals Systems. A Brief State-of-the-Art Review

Reliability of sensors has become not only the main focus of on-going worldwide research, but also the priority of research and innovation actions (RIA) supported by Horizon 2020 Programme. For the sake of giving only a brief overview, Table 1 summarizes some of the most challenging projects targeting sensor reliability, funded mainly by Electronic Components and Systems for European Leadership (ECSEL) Joint Undertaking [25] in the last five years. It is clearly shown that the reliability of sensors and remote sensing systems is a key enabling step towards massive utilization of sensor networks in all application fields from manufacturing up to maritime and aeronautic applications.
About twenty thousand scientific reports have considered reliability as the cornerstone of the main works [6,37]. The assessment of the sensor’s reliability in a classification problem based on the transferable belief model was presented in [38], whereas the Dempster-Shafer theory of belief functions was the foundation for a framework in another seminal paper [11]. The reliability of leap motion sensor in static and dynamic tracking was analyzed in [39], with not-very-promising expectations due to limited sensory space and inconsistent sampling frequency. Special attention has been received by wireless sensor networks, considering a measure of reliability using a probabilistic graph [40], packet delivery mechanism for guaranteeing quality of service [41] and adaptive and cross-layer framework for reliable and energy-efficient data collection [42]. Some methods for improving reliability in data transmission and localization by combining particle and finite impulse response filtering [43] and a new routing protocol [44] have recently been reported. The reader may find an interesting review of reliability assessment of wireless sensor network for industrial applications in [45].
Cyber-physical systems pose more scientific and technical challenges due to the need for conjunctive approaches for evaluating sensor reliability using Internet-of-Things data [46,47]. A pioneering work suggested a method called Loss Inference based on Passive Measurement (LIPM) to compute the link loss performance in digital ecosystems with plenty of distributed data sensors [48]. Scheduling policies formulated as a risk-sensitive Markov Decision Process were considered in an asymptotic approach to reliability [49]. Likewise, the Markov model for the reliability analysis of sensor nodes in wireless sensor networks was proposed in [3]. Similarly, the use of the stochastic optimization problem to maximize estimation reliability in water distribution networks was suggested in [50]. The quantification of the reliability of grid splitting under degraded communication conditions in the presence of cyber interdependencies was presented in [51]. Recently, a method based on the finite element model of a bridge integrated with structural vibration data for a self-governing structural health monitoring system was shown in [52]. A set of algorithms for estimating the reliability of the system under certain responsible time constraints was also recently suggested in [53]. Based on this review, it is evident that probabilistic-based approaches have been explored for addressing sensor reliability.
Certainly, the evaluation of sensor reliability is a highly important task, especially for Internet of Things (IoT) applications, and new performance indices need to be proposed [54]. Some recommendations for the standardized evaluation of sensor reliability are given in [55]. A recent review of analyses of the state of the art time delays, network size, energy efficiency, scalability, and reliability of mobile wireless sensor networks can be found in [56]. Attention has also been focused on lowering latency and improving protocols [57,58].
How can machine learning methods enable improvements in sensor reliability in cyber-physical systems?
The answer to this question is not easy, because the conditioning and re-elaboration of machine learning methods for assessing sensor reliability is not straightforward. This is indeed a new research trend with some pioneering reports in the early of 1990s [59]. Bayesian approaches [60,61], Fuzzy set theory [62,63], Dempster-Shafer (D-R) evidence theory [64,65,66], and Grey Group Decision-Making [67], among other methods, have recently been explored to address the reliability of sensors using Artificial Intelligence. However, many issues related with sensor reliability in cyber-physical systems using Internet-of-Things data remain unsolved, such as efficient conditioning and pre-filtering of big sensor data, high computational cost and limited parametrization of machine learning methods.

3. CPS-Based Reliability Approach

The truly challenging aspects of sensor network reliability and its evaluation have yet to prompt an exhaustive exploration and evaluation of sensory data under critical conditions. A gap is addressed in this study by sensors and sensor data in a CPS.

3.1. Sensor Reliability Assesment

One approach for assessing sensor reliability in key emerging applications, such self-driving in the automotive sector [5], is a model-based design for representing key sensor parameters. These parameters can be derived from monitoring processes where ‘functional parameters’ refer to the sensor characteristics and sensor lifetime, as well as to economic aspects (see Figure 1).
With the aim of increasing the reliability of data collected by LiDAR, metrological assessment procedures should also be applied. Linear interpolation of measurements from three detectors arranged in series is a time-saving procedure for processing and reducing LiDAR data [68].
All the major sources of potential error that influence point positioning accuracy have to be considered in the analytical derivations in order to determine the reliability of the achievable point positioning accuracy of LiDAR systems. Some of the random errors are reported in [69]. Some mathematical foundations are also provided for point positioning accuracy derived from the LiDAR equation, via error propagation:
r M = r M , I N S + R I N S M ( R L I N S · r L + b I N S )
where r M represents the 3D coordinates of an object point in the mapping frame; r M , I N S represents time-dependent 3D INS coordinates in the mapping frame, provided by GPS/INS; R I N S M is the time-dependent rotation matrix between the INS body and the mapping frame; R L I N S is the boresight matrix between the laser frame and the INS body frame; r L represents the 3D object coordinates in the laser frame; and, b I N S is the boresight offset vector.
In addition, the calculation of the accuracy of the estimated location of an object using the LiDAR sensor can be performed by other key performance indices. For example, the use of the Distance Root Mean Squared (DRMS) measure for the data that are tracked on the x-y plane (2D) and the Mean Radial Spherical Error (MRSE) measure for the data that are tracked in the x–y–z space (3D) were reported in [27,70]. Using derivable error formulas, any given random error and scan angle in the LiDAR range can be modelled and simulated. By doing so, the factors affecting LiDAR system accuracy can be analyzed [71].

3.2. Statistical and Artificial Intelligence-Based Methods

The Bayesian and Hidden Markov techniques are the most widely applied modeling techniques for reliability assessment under fuzzy environments [72,73]. A Bayesian network is a directed acyclic graph consisting of a set of nodes, representing random variables and a set of directed edges, representing their conditional dependencies. The dependencies in a Bayesian network can be adaptively determined from a dataset through a learning process. The objective of this training is to provide the best description of the probability distribution over the dataset because the attribute values are not supplied in the dataset [74].
In addition to those probabilistic methods, new tools have been reported in the literature, highlighting the use of Artificial Intelligence (AI) techniques and, in particular, Machine Learning (ML), to solve complex situations [34]. AI techniques also provide cognitive abilities, so that performance may be improved by increasing network lifetime and reliability [75]. Those techniques include ANN and fuzzy inference systems [76,77]. Zhang et al. proposed a soft-computing system based on Genetic Algorithm-Support Vector Regression (GA-SVR), in order to facilitate the reliability and survivability of the Structural Health Monitoring (SHM) system faced with, for example, an invalid fiber link in the sensor network [78].

4. CPS-Based Co-Simulation Framework

Some factors that can affect CPS reliability include component failure, environmental effects, task changes, and network update. A strategy for testing the reliability of CPSs and for their evaluation is proposed in [79] by analyzing both the internal and the external factors that influence their reliability. One solution could be to evaluate each element that constitutes the system: testing hardware, software, and architecture, as well as performance reliability including service reliability, cyber security reliability, resilience and elasticity reliability, and vulnerability reliability.
Behavioral simulations of CPS and IoT are increasing in their relevance with respect to analyzing reliability, because precise mathematical modeling is not straightforward [80]. These simulations are based on addressing four main topics: node localization, energy management, network multi-objective optimization, and self-capabilities approach [81,82].
While the reliability evaluation of physical systems is well understood and has been extensively studied, the reliability evaluation of a CPS is still under-studied, because CPS will not degrade the performance in the same way of physical systems and cannot be represented by a well-defined failure model. An evaluation framework is therefore required in order to assess the performance of CPS. A framework for CPS reliability analysis that includes reliability-based runtime reconfiguration is proposed in [83]. This framework is codified in a domain-specific modelling language that provides details on operational constraints and dependences.
However, domain-specific modelling-based analysis is, in many cases, unable to compute reliability functions efficiently (e.g., in terms of failure distributions) for complex systems. To do so, a frequency-domain reliability analysis framework of transportation CPSs was described in [84]. The advantage of that method is its capability to capture higher-order moments of the system characteristics, its scalability for the analysis of the reliability of complex systems, and efficient calculations.
In addition, it is important to consider the evaluation of other aspects of the CPS, such as safety and, particularly, security; different aspects still in the focus of research in the past few years. Therefore, the design of the simulation framework for CPS should consider these aspects at three levels: security objectives, security approaches and security in specific applications [85]. However, the cyber part not only should be secured, but also the physical part should take into account possible threats. A multi-cyber (computational unit) framework was compared with traditional models to improve the availability of the CPS based on the Markov model is reported in literature. The evaluation was carried out in terms of availability, downtime, downtime costs, and the reliability of the CPS framework [86]. Finally, another work considered an Internet-based computing platform in the form of a global computing node. In [87], a new cloud security management framework was introduced, based on improving collaboration between cloud providers, service providers, and service consumers for the management of cloud platform security and the hosting services. In addition, although in some applications this will not be possible, it is important to consider the possibility of introducing the human factor in the reliability analysis procedure. A human-interactive Hardware-In-the-Loop Simulation (HILS) framework for CPS was developed in [88] to support reliability and reusability in a fully distributed operating environment.

4.1. CPS-based Co-simulation Framework

Based on the above contributions, and considering the initial list of requirements for the deploying an IoT sensor network, a CPS-based co-simulation framework is proposed. The IoT sensor network will supply physical data having (local and global) computing nodes for processing the sensory data.

4.1.1. Conceptual Scheme

In addition, the IoT sensor network has a global or main node composed of a knowledge database, a Q-learning method for decision-making, and an AI-based model library. During the simulation and the real running, a decision-making module will decide which specific model is the best in the current instant, taking into account the data received by all nodes that make up the network.
The functionalities are distributed in different nodes, both virtual and real, according to their functions. The distributed virtual or real nodes manage the capture of sensor data and run the error prediction calculation with the required accuracy, while the global or main node incorporates the runtime model that is generated, the library, and the knowledge database (see Figure 2).
The IoT sensors should be able to establish reliable and accurate wireless communications, ensuring that all the intrinsic challenges in an IoT network and in the different CPSs can be solved. This is achieved through the implementation of the architecture that is represented in Figure 2 by a network of n nodes, each node having n IoT sensors. In addition, the computing nodes communicate with each other and with their corresponding global node.

4.1.2. Procedure Description

The framework is designed taking into account that both the real and the virtual (local) computing nodes operate in parallel with the global computing node [89]. Data exchange between the different nodes takes place in two different ways. On the one hand, data exchange between local nodes is produced in both the virtual (3D model simulation tool) and the real scenario. On the other hand, there is the data exchange between different local nodes and the global node using the 802.11p wireless communication protocol.
Therefore, there is an interaction between the simulated and the real environments and external applications that are running in the main node. Figure 3 shows the schematic diagram of the data exchange or messages exchange within the co-simulation framework.
In this specific implementation, which is described more accurately in the following section, there is a wireless exchange of messages between different nodes using the 802.11p communication protocol in the following way. First, the local nodes with their IoT sensors detect different objects and their respective properties. Secondly, this information is shared on the network through a broadcast process.

5. IoT LiDAR-based Collaborative Mapping – A Case Study

The IoT sensor network chosen to evaluate the CPS-based co-simulation framework is composed of virtual and real LiDAR sensors [28]. An Ibeo Lux LiDAR 4-layer sensor was used with the following specifications: horizontal field of 120 deg, horizontal step of 0.125 deg., vertical field of 3.2 deg., vertical step of 0.8 deg., range of 200 m, and an update frequency of 12.5 Hz. As previously mentioned, the sensor network to be evaluated is composed of IoT sensors. The sensor network therefore has IoT capabilities connected to its computing nodes. These nodes are on-board computers integrated in an autonomous vehicle with a wireless communication interface between them.
The particular implementation of the CPS-based co-simulation framework, the LiDAR-based collaborative map is based on a co-simulation framework between two different software systems, designed in [90]. However, the contribution of this study is to include the real part in the co-simulation framework. This framework consists mainly of a computer-aided system to enable efficient interaction between the virtual scenario with virtual nodes setting in the simulation tool of Webots for automobile 8.6 [91] and an external application development for the computing nodes in the real scenario. The scenario in this particular case, in which the vehicles are represented as nodes, is as follows. A real vehicle (in a real scenario) and three virtual vehicles in the simulated scenario are detecting obstacles. Both kinds of vehicles share the position, object type and size of the obstacles (e.g., pedestrian, trees on the road and another vehicle). This is possible thanks to the IoT LiDAR network using an IoT obstacle detection application (see Section 4.1), created in run time. Figure 4 shows the detailed diagram of the implementation of the LiDAR-based collaborative map using the CPS-based framework approach.
As mentioned above, the exchange of information packets between the local nodes with the main or global node is possible thanks to the use of a communication protocol using a UDP (User Datagram Protocol) as the transport layer and a Wi-Fi 802.11p as the physical layer. The visualization of the co-simulated vehicle (real node) in the 3D virtual scenario in the Webots simulation tool is also possible. An example of the execution of the co-simulation architecture can be seen in Figure 5.
In addition, another application implemented in the IoT LiDAR-based collaborative map is the LiDAR self-testing methodology incorporated in each local computing node (autonomous vehicle), in order to evaluate the reliability of each IoT sensor in the network that will be described in Section 5.2.

5.1. Obstacle Detection in the IoT Application

The framework is implemented in an external application using Qt 5.10, which is an open-source widget toolkit for creating graphical user interfaces. The framework consists of an illustrated map updated in run time (see Figure 6a) and a database with the information on both virtual and real objects that are detected (see Figure 6b). The information contains the position, object type, and size of the obstacles in order to guarantee security and safety of the object detection process with a single sensor.
Figure 6 depicts the visual interface developed for the case study. Specifically, the collaborative map is globally updated in the main computing node. A partial area of this updated map can also be sent at the request of a local node. A set of computational procedures is in charge of adapting and transferring sensory information from Webots, virtual nodes with the Ibeo Lux sensor model, and the real node, real vehicles with the real Ibeo Lux sensor; and vice versa.

5.2. LiDAR Selft-testing Application

The external application also includes a LiDAR data self-testing methodology using the AI-based error-prediction models. Figure 6c shows the graphical interface that represents the estimated error with regard to time on the left-hand side. However, on the right-hand side, the admissible error threshold is observed, whereby, if it is exceeded, it must be requested that a decision be made regarding the best performance of each model at any given time. Specifically, the results are focused on showing the improved performance of the IoT sensor network composed of each CPS element with each LiDAR sensor plus added IoT capabilities. To do so, a reliability prediction model dedicated to obtaining the accuracy error in obstacle detection is incorporated in each computing node.

5.2.1. Reliability Prediction Models

A reliability model is generated for both virtual and real IoT LiDAR sensors for predicting the accuracy error in obstacle detection. The procedure for developing these models is extracted from the methodology described in [27], with a different set of training data. In this study, a model-based procedure is used with a point-cloud clustering technique, in this case Density-Based Spatial Clustering of Applications with Noise (DBSCAN) [92]. In addition, an error-based prediction model library is implemented, highlighting AI-based model techniques, such as, Multilayer Perceptron Neural Network (MLP), k-Nearest Neighbors (k-NN), and Linear Regression (LR). A difference in the particular implementation described in this paper is that, while k-NN, MLP and LR were maintained, SVM is incorporate as a new technique to the modeling library [93,94,95].

5.2.2. Model Parametrization and Validation

With the aim of determining which model training strategy based on AI provides the best reliability prediction model, an experimental validation was performed. The training dataset was composed of 998 scenes for the model training and 250 scenes for the model validation. All of them were obtained from a simulation procedure. The data input consisted of geospatial statistics [70,96] that were extracted from the point cloud supplied by the LiDAR sensor, so that the models could generate the figures of merit in terms of accuracy error: DRMS and MRSE.
The four AI-based strategies considered can be summarized as follows. First, a multilayer perceptron neural network with backpropagation error (MLP) is selected with two hidden layers, each with five neurons and sigmoid activation functions, and an output layer with a lineal activation function, two neurons, and 5000 epochs. The initial value of the learning rate (μ) is 10−3 with a decrease factor ratio of 10−1, an increase factor ratio of 10, and a maximum μ value of 1010. The minimum performance gradient was 10−7. The training process stop criteria is as follows: the maximum number of epochs (repetitions); goal performance minimization; the performance gradient below a minimum gradient; or, a μ value in excess of the maximum value. The second modeling technique is k-nearest neighbors (k-NN), with k = 5 and using Euclidean distance as the distance function. The third method is a lineal regression that was also obtained by minimizing the sum of squared differences between the predicted and the observed values. Finally, a support vector machine model is also obtained by means of data standardization and the radial basis function kernel.

5.2.3. Self-Learning-Based Decision-Making. Q-Learning Algorithm

The global or main computing node executes several parallel procedures in a specific self-learning module that uses a Q-learning algorithm (see Figure 7). On the one hand, a dataset for training by default is done in the global node. On the other hand, a knowledge database (warehouse) is also included, which can be updated in run time with the data provided by each local node. The self-learning strategy also runs in the global node in order to determine the best model behavior, when new traffic situations are generated by providing new point clouds (environment information).
The local node simulates the reliability model. When the error surpasses the 20% threshold in one the figures of merit (DRMS or MRSE), it is inferred that the current model is failing. A request is therefore made to the global module to establish whether there is a model that is working better. The decision for selecting the best prediction model available in the library is taken during the decision-making stage at each instant by means of a self-learning module, on the basis of the generalization capability and the accuracy of the model. The corresponding performance metrics are R2 and RAE, respectively. In summary, based on this information flow and the previous prediction results (knowledge database), when a request from one of the local nodes is received and a new best model behavior is detected, the current error prediction model is then switched from MLP to k-NN, and vice versa.

6. Experimental Results

First, a comparative study was conducted on the basis of different AI techniques applied to model the prediction error obtained in objects detection. Spatial statistics, reported in [27], are the inputs for being aware of the condition of the point cloud in each scan provided by the LiDAR sensor. Subsequently, based on the results of this study, the best models are determined for the final evaluation. The final decision-making is done by means of a Q-learning strategy, which was selected for its good features among unsupervised learning methods [97]. This procedure predicts in advance which model has the best behavior on the basis of the available data information.

6.1. Reliability Model-based Validation

Table 2 shows the evaluation results obtained during the initial validation of each reliability model. Five error-based performance indices and two classification criteria were considered in the validation process: Mean Absolute Error (MAE); Root Mean Squared Error (RMSE); Relative Absolute Error (RAE); Root Relative Squared Error (RRSE); and, the coefficient of determination (R2). Only the models generated with k-NN and MLP returned R2 results higher than 90%. Key performance indices based on plane (DMRS) and space (MRSE) figures of merit.
Figure 8 illustrates the behavior of the LiDAR error on the plane (DRMS) and space (MRSE) for each model (MLP, LN, k-NN and SVM) with regard to the validation data. The modeling techniques that provide the best performance are MLP and k-NN, according to the comparative study of the four modelling strategies, improving about 30% the performance indices with regard to LR and SVM. These two type models are then chosen for validating the decision-making module.

6.2. Self-Learning for Decision-Making Evaluation

Based on the previous knowledge generated by the rewards obtained from similar situations studied in the simulations, the ranges of the values of each reward are defined as a function of the prediction error of the reliability model, calculated during the simulation. The reward function is chosen for setting the best possible Q-value in 100 different scenes. Therefore, the function for updating the Q-values is [98]:
Q ( s t + 1 , a t + 1 ) ← Q t ( s t , a t ) + α t ( R t + 1 + γ   max a ϵ A   Q ( S t + 1 , a t ) − Q ( S t ,   a t ) )
where st is the state in time t; at is the action taken in time t; R(t+1) is the reward received after performing action at; αt is the learning rate (0 ≤ αt ≤ 1); and γ is the discount factor which trades off the importance of sooner-versus-later rewards. Table 3 lists the error reward matrix based on knowledge of the behavior of those prediction models.
The decision-making is based on two of the main performance indices. First, the coefficient of determination (R2) is taken into account, because it provides a measure of the generalization capacity of the model. The Relative Absolute Error (RAE), which is a measure of model accuracy, is also considered.
Finally, another simulation is performed in order to determine the quality of the Q-learning method for automatically selecting the best prediction model. One hundred new scenes captured by the LiDAR sensor were analyzed. The Q-learning method runs online in the main node. When new data becomes available, models contained in the library are evaluated. Once each model has been assessed, the performance indices are calculated, and the Q-learning determines the best reliability model between MLP and k-NN. Figure 9a,b depicts the behavior of RAE and R2, respectively, for both reliability models.
Figure 10 shows the Q-learning classification error matrix. In all considered cases, the best model chosen was k-NN 76 times and MLP 24 times. Analyzing the results obtained, the best model selected had a RAE between 0 and 20% and a R2 above 80% in 61% of the scenarios. The system was able to guarantee models with a greater capability of generalization in 71% of the scenarios, based on a coefficient of determination higher than 80%. Overall, reliability can be predicted with an RAE of less than 40% and an R2 of about 70% in 90% of the scenarios, which demonstrates the quality of the learning process. The models presented a low generalization, with a coefficient of determination of less than 70% in only 9% of the scenarios and an RAE greater than 40% in only 1%. Therefore, the Q-learning method that evaluates reliability on the basis of the prediction error model at each instant worked appropriately for calculating the best model that represented the LiDAR performance to a high degree of accuracy and that guaranteed the required levels of safety and reliability for automotive applications.

7. Conclusions

This paper presents a review of the main research and innovation actions dealing with sensor reliability in cyber-physical systems supported by the European Commission in recent years. The review demonstrates that cyber-physical systems pose greater scientific and technical challenges due to the need for conjunctive approaches for evaluating sensor reliability using Internet-of-Things data. New research trends are focused on machine learning methods for improving metrics and sensor reliability in cyber-physical systems. However, the insufficient performance of big data conditioning and pre-filtering methods, the high computational cost and the ad-hoc parametrization of machine learning methods still limit generalization, real-time application and certification of solutions based on Internet-of-Things data for increasing sensor reliability.
This paper also presents a case study of sensor reliability assessment in autonomous driving scenario. The procedure for evaluating the reliability of IoT sensors in a cyber-physical system is proposed. A co-simulation framework is designed in order to enable real-time interaction between virtual and real sensors by running simulations in appropriate safety conditions. The case study consists of a IoT LiDAR-based collaborative map in order to validate the CPS-based co-simulation framework.
The assessment of the proposed method is divided into two parallel procedures. First, at a local level, each reliability model evaluates the condition of the IoT LiDAR sensor. The evaluation is carried out on the basis of AI-based models under 250 scenes. Based on the obtained results, only the multilayer perceptron and k-nearest neighbor methods were chosen for the validation of the decision-making module. Secondly, at a global level, a self-learning strategy for decision-making calculates the most appropriate behavior in the reliability model library, also in run time. Based on the results with 100 different scenarios, the Q-learning method improves the reliability of LiDAR sensor data with regard to using a single model at a local level.
Therefore, the proposed co-simulation framework serves to assess the performance of IoT LiDAR sensor data very accurately, guaranteeing safety and reliability in this autonomous driving scenario. These promising results serve as the basis for future work in validating the proposed method in real autonomous driving conditions.

Author Contributions

R.E.H., J.K. and S.S. reviewed all technical and scientific aspects of the article. A.V. and F.C. was in charge of the implementation of software application, the library models and the reinforcement learning algorithm. F.C. and A.V. designed and implemented the scenario, the external application and the LiDAR self-testing procedure, and drafted the paper.

Funding

This work was partially supported by the project Power2Power: Providing next-generation silicon-based power solutions in transport and machinery for significant decarbonisation in the next decade, funded by the Electronic Component Systems for European Leadership (ECSEL-JU) Joint Undertaking and the Ministry of Science, Innovation and Universities (MICINN), under grant agreement No 826417. In addition, this work was also funded by the Spanish Ministry of Science, Innovation and Universities through the project COGDRIVE (DPI2017-86915-C3-1-R). Preparation of this publication was also partially co-financed by the Polish National Agency for Academic Exchange (NAWA) through the project: “Industry 4.0 in Production and Aeronautical Engineering (IPAE)”.

Acknowledgments

The authors would like to thank the AUTOPIA group located at the Center for Automation and Robotics, jointly owned by the Spanish National Research Council and Technical University of Madrid.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Beruvides, G.; Quiza, R.; Del Toro, R.; Haber, R.E. Sensoring systems and signal analysis to monitor tool wear in microdrilling operations on a sintered tungsten-copper composite material. Sens. Actuators A Phys. 2013, 199, 165–175. [Google Scholar] [CrossRef]
  2. Iarovyi, S.; Mohammed, W.M.; Lobov, A.; Ferrer, B.R.; Lastra, J.L.M. Cyber-Physical Systems for Open-Knowledge-Driven Manufacturing Execution Systems. Proc. IEEE 2016, 104, 1142–1154. [Google Scholar] [CrossRef]
  3. Kabashkin, I.; Kundler, J. Reliability of Sensor Nodes in Wireless Sensor Networks of Cyber Physical Systems. Procedia Comput. Sci. 2017, 104, 380–384. [Google Scholar] [CrossRef]
  4. Jakovljevic, Z.; Petrovic, M.; Mitrovic, S.; Miljkovic, Z. Intelligent sensing systems—Status of research at KaProm. In Lecture Notes in Mechanical Engineering; Springer: Berlin/Heidelberg, Germany, 2018; pp. 18–36. [Google Scholar]
  5. Godoy, J.; Pérez, J.; Onieva, E.; Villagrá, J.; Milanés, V.; Haber, R. A driverless vehicle demonstration on motorways and in urban environments. Transport 2015, 30, 253–263. [Google Scholar] [CrossRef]
  6. Routh, D.; Seegmiller, L.; Bettigole, C.; Kuhn, C.; Oliver, C.D.; Glick, H.B. Improving the reliability of mixture tuned matched filtering remote sensing classification results using supervised learning algorithms and cross-validation. Remote Sens. 2018, 10, 1675. [Google Scholar] [CrossRef]
  7. Zhao, Y.; Feng, D.; Yu, L.; See, L.; Fritz, S.; Perger, C.; Gong, P. Assessing and improving the reliability of volunteered land cover reference data. Remote Sens. 2017, 9, 1034. [Google Scholar] [CrossRef]
  8. Hao, X.; Wang, L.; Yao, N.; Geng, D.; Chen, B. Topology control game algorithm based on Markov lifetime prediction model for wireless sensor network. Ad Hoc Netw. 2018, 78, 13–23. [Google Scholar] [CrossRef]
  9. Hessner, K.G.; El Naggar, S.; von Appen, W.J.; Strass, V.H. On the reliability of surface current measurements by X-Band marine radar. Remote Sens. 2019, 11, 1030. [Google Scholar] [CrossRef]
  10. Jia, F.; Lichti, D.D. A model-based design system for terrestrial laser scanning networks in complex sites. Remote Sens. 2019, 11, 1749. [Google Scholar] [CrossRef]
  11. Guo, H.; Shi, W.; Deng, Y. Evaluating Sensor Reliability in Classification Problems Based on Evidence Theory. IEEE Trans. Syst. Man Cybern. Part B 2006, 36, 970–981. [Google Scholar] [CrossRef]
  12. Lei, L.; Qiu, C.; Li, Z.; Han, D.; Han, L.; Zhu, Y.; Wu, J.; Xu, B.; Feng, H.; Yang, H.; et al. Effect of leaf occlusion on leaf area index inversion of maize using UAV-LiDAR data. Remote Sens. 2019, 11, 1067. [Google Scholar] [CrossRef]
  13. Okhrimenko, M.; Coburn, C.; Hopkinson, C. Multi-spectral lidar: Radiometric calibration, canopy spectral reflectance, and vegetation vertical SVI profiles. Remote Sens. 2019, 11, 1556. [Google Scholar] [CrossRef]
  14. Chen, P.; Pan, D. Ocean optical profiling in South China Sea Using Airborne LiDAR. Remote Sens. 2019, 11, 1826. [Google Scholar] [CrossRef]
  15. Dash, J.P.; Watt, M.S.; Paul, T.S.H.; Morgenroth, J.; Pearse, G.D. Early detection of invasive exotic trees using UAV and manned aircraft multispectral and LiDAR Data. Remote Sens. 2019, 11, 1812. [Google Scholar] [CrossRef]
  16. Kerfoot, W.C.; Hobmeier, M.M.; Green, S.A.; Yousef, F.; Brooks, C.N.; Shuchman, R.; Sayers, M.; Lin, L.; Luong, P.; Hayter, E.; et al. Coastal ecosystem investigations with LiDAR (Light Detection and Ranging) and bottom reflectance: Lake superior reef threatened by migrating tailings. Remote Sens. 2019, 11, 1076. [Google Scholar] [CrossRef]
  17. Xiao, L.; Wang, R.; Dai, B.; Fang, Y.; Liu, D.; Wu, T. Hybrid conditional random field based camera-LIDAR fusion for road detection. Inf. Sci. 2018, 432, 543–558. [Google Scholar] [CrossRef]
  18. Zeng, Y.; Yu, H.; Dai, H.; Song, S.; Lin, M.; Sun, B.; Jiang, W.; Meng, M. An Improved Calibration Method for a Rotating 2D LIDAR System. Sensors 2018, 18, 497. [Google Scholar] [CrossRef]
  19. Kempf, J.; Arkko, J.; Beheshti, N.; Yedavalli, K. Thoughts on reliability in the internet of things. In Proceedings of the Interconnecting Smart Objects with the Internet Workshop 2011, Prague, Czech, 25 March 2011; Volume 1, pp. 1–4. [Google Scholar]
  20. Ahmad, M. Reliability Models for the Internet of Things: A Paradigm Shift. In Proceedings of the 2014 IEEE International Symposium on Software Reliability Engineering Workshops, Naples, Italy, 3–6 November 2014; pp. 52–59. [Google Scholar]
  21. Zhang, H.; Liu, J.; Pang, A.-C. A Bayesian network model for data losses and faults in medical body sensor networks. Comput. Netw. 2018, 143, 166–175. [Google Scholar] [CrossRef]
  22. Serpen, G.; Li, J.; Liu, L. AI-WSN: Adaptive and Intelligent Wireless Sensor Network. Procedia Comput. Sci. 2013, 20, 406–413. [Google Scholar] [CrossRef] [Green Version]
  23. Nsabagwa, M.; Mugume, I.; Kasumba, R.; Muhumuza, J.; Byarugaba, S.; Tumwesigye, E.; Otim, J.S. Condition Monitoring and Reporting Framework for Wireless Sensor Network-based Automatic Weather Stations. In Proceedings of the 2018 IST-Africa Week Conference (IST-Africa), Gaborone, Botswana, 9–11 May 2018; pp. 1–8. [Google Scholar]
  24. Chuan, Y.; Chen, L. The Application of Support Vector Machine in the Hysteresis Modeling of Silicon Pressure Sensor. IEEE Sens. J. 2011, 11, 2022–2026. [Google Scholar] [CrossRef]
  25. Commission, E. Project impact articles. In Electronic Components and Systems for European Leadership (ECSEL) Joint Undertaking; 2019; Available online: https://www.ecsel.eu/project-impact (accessed on 20 September 2019).
  26. Mäyrä, A.; Hietala, E.; Kutila, M.; Pyykönen, P.; Tiihonen, M.; Jokela, T. Experimental study on spectral absorbance in fog as a function of temperature, liquid water content, and particle size. In Proceedings of the SPIE REMOTE SENSING, Optics in Atmospheric Propagation and Adaptive Systems, Warsaw, Poland, 11–14 September 2017. [Google Scholar]
  27. Castaño, F.; Beruvides, G.; Villalonga, A.; Haber, R.E. Self-tuning method for increased obstacle detection reliability based on internet of things LiDAR sensor models. Sensors 2018, 18, 1508. [Google Scholar] [CrossRef]
  28. Godoy, J.; Haber, R.; Muñoz, J.J.; Matía, F.; García, Á. Smart sensing of pavement temperature based on low-cost sensors and V2I communications. Sensors 2018, 18, 2092. [Google Scholar] [CrossRef]
  29. Le, V.H.; den Hartog, J.; Zannone, N. Security and privacy for innovative automotive applications: A survey. Comput. Commun. 2018, 132, 17–41. [Google Scholar] [CrossRef]
  30. Valencia, J.; Goswami, D.; Goossens, K. Comparing platform-aware control design flows for composable and predictable TDM-based execution platforms. ACM Trans. Des. Autom. Electron. Syst. 2019, 24, 32. [Google Scholar] [CrossRef]
  31. Druml, N.; Macher, G.; Stolz, M.; Armengaud, E.; Watzenig, D.; Steger, C.; Herndl, T.; Eckel, A.; Ryabokon, A.; Hoess, A.; et al. PRYSTINE—PRogrammable sYSTems for INtelligence in AutomobilEs. In Proceedings of the 2018 21st Euromicro Conference on Digital System Design (DSD), Prague, Czech Republic, 29–31 August 2018; pp. 618–626. [Google Scholar]
  32. Godoy, J.; Artuñedo, A.; Villagra, J. Self-Generated OSM-Based Driving Corridors. IEEE Access 2019, 7, 20113–20125. [Google Scholar] [CrossRef]
  33. Guerra, R.H.; Quiza, R.; Villalonga, A.; Arenas, J.; Castano, F. Digital Twin-Based Optimization for Ultraprecision Motion Systems with Backlash and Friction. IEEE Access 2019, 7, 93462–93472. [Google Scholar] [CrossRef]
  34. La Fe-Perdomo, I.; Beruvides, G.; Quiza, R.; Haber, R.; Rivas, M. Automatic Selection of Optimal Parameters Based on Simple Soft-Computing Methods: A Case Study of Micromilling Processes. IEEE Trans. Ind. Inform. 2019, 15, 800–811. [Google Scholar] [CrossRef]
  35. Toma, D.M.; Masmitja, I.; del Río, J.; Martinez, E.; Artero-Delgado, C.; Casale, A.; Figoli, A.; Pinzani, D.; Cervantes, P.; Ruiz, P.; et al. Smart embedded passive acoustic devices for real-time hydroacoustic surveys. Measurement 2018, 125, 592–605. [Google Scholar] [CrossRef] [Green Version]
  36. Lizé, E.; Rébillat, M.; Mechbal, N.; Bolzmacher, C. Optimal dual-PZT sizing and network design for baseline-free SHM of complex anisotropic composite structures. Smart Mater. Struct. 2018, 27, 115018. [Google Scholar] [CrossRef] [Green Version]
  37. Fratarcangeli, F.; Savastano, G.; D’Achille, M.C.; Mazzoni, A.; Crespi, M.; Riguzzi, F.; Devoti, R.; Pietrantonio, G. VADASE reliability and accuracy of real-time displacement estimation: Application to the Central Italy 2016 earthquakes. Remote Sens. 2018, 10, 1201. [Google Scholar] [CrossRef]
  38. Elouedi, Z.; Mellouli, K.; Smets, P. Assessing Sensor Reliability for Multisensor Data Fusion within the Transferable Belief Model. IEEE Trans. Syst. Man Cybern. Part B 2004, 34, 782–787. [Google Scholar] [CrossRef]
  39. Guna, J.; Jakus, G.; Pogačnik, M.; Tomažič, S.; Sodnik, J. An analysis of the precision and reliability of the leap motion sensor and its suitability for static and dynamic tracking. Sensors 2014, 14, 3702–3720. [Google Scholar] [CrossRef]
  40. AboElFotoh, H.M.F.; Iyengar, S.S.; Chakrabarty, K. Computing reliability and message delay for cooperative wireless distributed sensor networks subject to random failures. IEEE Trans. Reliab. 2005, 54, 145–155. [Google Scholar] [CrossRef]
  41. Felemban, E.; Lee, C.G.; Ekici, E. MMSPEED: Multipath Multi-SPEED Protocol for QoS guarantee of reliability and timeliness in wireless sensor networks. IEEE Trans. Mob. Comput. 2006, 5, 738–753. [Google Scholar] [CrossRef]
  42. Di Francesco, M.; Anastasi, G.; Conti, M.; Das, S.K.; Neri, V. Reliability and energy-efficiency in IEEE 802.15.4/ZigBee sensor networks: An adaptive and cross-layer approach. IEEE J. Sel. Areas Commun. 2011, 29, 1508–1524. [Google Scholar] [CrossRef]
  43. Pak, J.M.; Ahn, C.K.; Shmaliy, Y.S.; Lim, M.T. Improving Reliability of Particle Filter-Based Localization in Wireless Sensor Networks via Hybrid Particle/FIR Filtering. IEEE Trans. Ind. Inform. 2015, 11, 1089–1098. [Google Scholar] [CrossRef]
  44. Shen, J.; Tan, H.; Wang, J.; Wang, J.; Lee, S. A novel routing protocol providing good transmission reliability in underwater sensor networks. J. Internet Technol. 2015, 16, 171–178. [Google Scholar]
  45. Silva, I.; Guedes, L.A.; Portugal, P.; Vasques, F. Reliability and availability evaluation of wireless sensor networks for industrial applications. Sensors 2012, 12, 806–838. [Google Scholar] [CrossRef]
  46. Mitchell, R.; Chen, I.-R. Effect of Intrusion Detection and Response on Reliability of Cyber Physical Systems. IEEE Trans. Reliab. 2013, 62, 199–210. [Google Scholar] [CrossRef]
  47. Shiliang, L.; Lianglun, C.; Bin, R.; Quanmin, Z. An Improved Intelligent Ant Colony Algorithm for the Reliability Optimization Problem in Cyber-Physical Systems. J. Softw. 2014, 9, 20–25. [Google Scholar]
  48. Yang, Y.; Xu, Y.; Li, X.; Chen, C. A Loss Inference Algorithm for Wireless Sensor Networks to Improve Data Reliability of Digital Ecosystems. IEEE Trans. Ind. Electron. 2011, 58, 2126–2137. [Google Scholar] [CrossRef]
  49. Xueying, G.; Singh, R.; Kumar, P.R.; Zhisheng, N. A high reliability asymptotic approach for packet inter-delivery time optimization in cyber-physical systems. arXiv 2015, arXiv:1504.03076. [Google Scholar]
  50. Kartakis, S.; Yang, S.; McCann, J.A. Reliability or Sustainability: Optimal Data Stream Estimation and Scheduling in Smart Water Networks. ACM Trans. Sens. Netw. 2017, 13, 18. [Google Scholar] [CrossRef]
  51. Tian, D.-A.; Sansavini, G. Impact of cyber dependencies in critical infrastructures: The reliability of grid splitting in power systems. J. Risk Reliab. 2018, 232, 491–504. [Google Scholar] [CrossRef]
  52. Ozer, E.; Feng, M.Q. Structural Reliability Estimation with Participatory Sensing and Mobile Cyber-Physical Structural Health Monitoring Systems. Appl. Sci. 2019, 9, 2840. [Google Scholar] [CrossRef]
  53. Zhang, J.; Zhu, Y.; Xiao, F. Modelling and analysis of real-time and reliability for WSN-based CPS. Int. J. Internet Protoc. Technol. 2019, 12, 76–84. [Google Scholar] [CrossRef]
  54. Deif, D.; Gadallah, Y. A comprehensive wireless sensor network reliability metric for critical Internet of Things applications. EURASIP J. Wirel. Commun. Netw. 2017, 2017, 145. [Google Scholar] [CrossRef]
  55. Duking, P.; Fuss, F.K.; Holmberg, H.-C.; Sperlich, B. Recommendations for Assessment of the Reliability, Sensitivity, and Validity of Data Provided by Wearable Sensors Designed for Monitoring Physical Activity. JMIR mHealth uHealth 2018, 6, e102. [Google Scholar] [CrossRef]
  56. Yue, Y.-G.; He, P. A comprehensive survey on the reliability of mobile wireless sensor networks: Taxonomy, challenges, and future directions. Inf. Fusion 2018, 44, 188–204. [Google Scholar] [CrossRef]
  57. Beyranvand, H.; Levesque, M.; Maier, M.; Salehi, J.A.; Verikoukis, C.; Tipper, D. Toward 5G: FiWi Enhanced LTE-A HetNets with Reliable Low-Latency Fiber Backhaul Sharing and WiFi Offloading. IEEE ACM Trans. Netw. 2017, 25, 690–707. [Google Scholar] [CrossRef]
  58. Kartsakli, E.; Antonopoulos, A.; Lalos, A.; Tennina, S.; Renzo, M.; Alonso, L.; Verikoukis, C. Reliable MAC design for ambient assisted living: Moving the coordination to the cloud. IEEE Commun. Mag. 2015, 53, 78–86. [Google Scholar] [CrossRef]
  59. Wang, X.; Shi, T.; Yan, M.; Yang, S. The affirmation of sensor signal reliability for a machine monitoring and fault diagnosing system. J. Huazhong Univ. Sci. Technol. 1993, 21, 57–62. [Google Scholar]
  60. Pourali, M.; Mosleh, A. A Bayesian approach to sensor placement optimization and system reliability monitoring. J. Risk Reliab. 2013, 227, 327–347. [Google Scholar] [CrossRef]
  61. Pozna, C.; Precup, R.E.; Tar, J.K.; Škrjanc, I.; Preitl, S. New results in modelling derived from Bayesian filtering. Knowl. Based Syst. 2010, 23, 182–194. [Google Scholar] [CrossRef]
  62. Duan, R.; Fan, J. Dynamic Diagnosis Strategy for Redundant Systems Based on Reliability Analysis and Sensors under Epistemic Uncertainty. J. Sens. 2015, 2015, 592142. [Google Scholar] [CrossRef]
  63. Song, Y.; Wang, X.; Zhu, J.; Lei, L. Sensor dynamic reliability evaluation based on evidence theory and intuitionistic fuzzy sets. Appl. Intell. 2018, 48, 3950–3962. [Google Scholar] [CrossRef]
  64. Yuan, K.; Xiao, F.; Fei, L.; Kang, B.; Deng, Y. Modeling Sensor Reliability in Fault Diagnosis Based on Evidence Theory. Sensors 2016, 16, 113. [Google Scholar] [CrossRef]
  65. Jiang, W.; Zhuang, M.; Xie, C. A Reliability-Based Method to Sensor Data Fusion. Sensors 2017, 17, 1575. [Google Scholar] [CrossRef]
  66. Zichang, H.; Wen, J. Quantum Mechanical Approach to Modeling Reliability of Sensor Reports. IEEE Sens. Lett. 2017, 1, 7500304. [Google Scholar]
  67. Song, K.; Xu, P.; Wei, G.; Chen, Y.; Wang, Q. Health Management Decision of Sensor System Based on Health Reliability Degree and Grey Group Decision-Making. Sensors 2018, 18, 2316. [Google Scholar] [CrossRef]
  68. Rodrigues, C.E.D.; Da Silva, R.P.M.; Nunes, R.A. Metrological assessment of LIDAR signals in water. Measurement 2011, 44, 11–17. [Google Scholar] [CrossRef]
  69. Yang, X.; Quan, Z.; Dan, J. The effect of INS and GPS random error on positioning accuracy of airborne LIDAR system. J. Comput. Inf. Syst. 2011, 7, 3795–3802. [Google Scholar]
  70. Maalek, R.; Sadeghpour, F. Accuracy assessment of ultra-wide band technology in locating dynamic resources in indoor scenarios. Autom. Constr. 2016, 63, 12–26. [Google Scholar] [CrossRef]
  71. Dan, J.; Yang, X. Modeling of range and scan angle random errors from airborne LIDAR. J. Comput. Inf. Syst. 2013, 9, 2747–2753. [Google Scholar]
  72. Wu, H.C. Bayesian system reliability assessment under fuzzy environments. Reliab. Eng. Syst. Saf. 2004, 83, 277–286. [Google Scholar] [CrossRef]
  73. Wu, H.C. Fuzzy Bayesian system reliability assessment based on exponential distribution. Appl. Math. Model. 2006, 30, 509–530. [Google Scholar] [CrossRef]
  74. Du, K.L.; Swamy, M.N.S. Probabilistic and Bayesian Networks. In Neural Networks and Statistical Learning; Springer: Berlin/Heidelberg, Germany, 2014; pp. 563–619. [Google Scholar]
  75. Gheisari, S.; Meybodi, M.R. A new reasoning and learning model for Cognitive Wireless Sensor Networks based on Bayesian networks and learning automata cooperation. Comput. Netw. 2017, 124, 11–26. [Google Scholar] [CrossRef]
  76. Haber, R.E.; Alique, J.R. Nonlinear internal model control using neural networks: An application for machining processes. Neural Comput. Appl. 2004, 13, 47–55. [Google Scholar] [CrossRef]
  77. Ramírez, M.; Haber, R.; Peña, V.; Rodríguez, I. Fuzzy control of a multiple hearth furnace. Comput. Ind. 2004, 54, 105–113. [Google Scholar] [CrossRef]
  78. Zhang, X.; Liang, D.; Zeng, J.; Asundi, A. Genetic algorithm-support vector regression for high reliability SHM system based on FBG sensor network. Opt. Lasers Eng. 2012, 50, 148–153. [Google Scholar] [CrossRef]
  79. Li, Z.; Kang, R. Strategy for reliability testing and evaluation of cyber physical systems. In Proceedings of the IEEE International Conference on Industrial Engineering and Engineering Management, Singapore, 6–9 December 2015; pp. 1001–1006. [Google Scholar]
  80. Vo, M.-T.; Thanh Nghi, T.T.; Tran, V.-S.; Mai, L.; Le, C.-T. Wireless Sensor Network for Real Time Healthcare Monitoring: Network Design and Performance Evaluation Simulation; Springer International Publishing: Cham, Switzerland, 2015; pp. 87–91. [Google Scholar]
  81. Cacciagrano, D.; Culmone, R.; Micheletti, M.; Mostarda, L. Energy-Efficient Clustering for Wireless Sensor Devices in Internet of Things. In Performability in Internet of Things; Springer: Berlin/Heidelberg, Germany, 2019; pp. 59–80. [Google Scholar]
  82. Hossain, S.; Fayjie, A.R.; Doukhi, O.; Lee, D.-J. CAIAS Simulator: Self-driving Vehicle Simulator for AI Research. In Proceedings of the International Conference on Intelligent Computing & Optimization, Pattaya, Thailand, 4–5 October 2018; Springer: Berlin/Heidelberg, Germany, 2018; pp. 187–195. [Google Scholar]
  83. Nannapaneni, S.; Mahadevan, S.; Pradhan, S.; Dubey, A. Towards Reliability-Based Decision Making in Cyber-Physical Systems. In Proceedings of the 2016 IEEE International Conference on Smart Computing, SMARTCOMP 2016, St. Louis, MO, USA, 18–20 May 2016. [Google Scholar]
  84. Liu, X.; He, W.; Zheng, L. Transportation cyber-physical systems: Reliability modeling and analysis framework. In Proceedings of the National Workshop for Research on High-Confidence Transportation Cyber-Physical Systems: Automotive, Aviation and Rail, Seattle, WA, USA, 18–20 November 2008; pp. 18–20. [Google Scholar]
  85. Lu, T.; Zhao, J.; Zhao, L.; Li, Y.; Zhang, X. Towards a framework for assuring cyber physical system security. Int. J. Secur. Appl. 2015, 9, 25–40. [Google Scholar] [CrossRef]
  86. Parvin, S.; Hussain, F.K.; Hussain, O.K.; Thein, T.; Park, J.S. Multi-cyber framework for availability enhancement of cyber physical systems. Computing 2013, 95, 927–948. [Google Scholar] [CrossRef]
  87. Almorsy, M.; Grundy, J.; Ibrahim, A.S. Collaboration-based cloud computing security management framework. In Proceedings of the 2011 IEEE 4th International Conference on Cloud Computing, CLOUD 2011, Washington, DC, USA, 4–9 July 2011; pp. 364–371. [Google Scholar]
  88. Kim, M.J.; Kang, S.; Kim, W.T.; Chun, I.G. Human-interactive hardware-in-the-loop simulation framework for cyber-physical systems. In Proceedings of the 2013 2nd International Conference on Informatics and Applications, ICIA 2013, Lodz, Poland, 23–25 September 2013; pp. 198–202. [Google Scholar]
  89. Artuñedo, A.; Del Toro, R.M.; Haber, R.E. Consensus-based cooperative control based on pollution sensing and traffic information for urban traffic networks. Sensors 2017, 17, 953. [Google Scholar] [CrossRef]
  90. Castaño, F.; Beruvides, G.; Haber, R.E.; Artuñedo, A. Obstacle Recognition Based on Machine Learning for On-Chip LiDAR Sensors in a Cyber-Physical System. Sensors 2017, 17, 2109. [Google Scholar] [CrossRef]
  91. Michel, O. Cyberbotics Ltd. Webots™: Professional mobile robot simulation. Int. J. Adv. Robot. Syst. 2004, 1, 5. [Google Scholar] [CrossRef]
  92. Huang, F.; Zhu, Q.; Zhou, J.; Tao, J.; Zhou, X.; Jin, D.; Tan, X.; Wang, L. Research on the Parallelization of the DBSCAN Clustering Algorithm for Spatial Data Mining Based on the Spark Platform. Remote Sens. 2017, 9, 1301. [Google Scholar] [CrossRef]
  93. Banerjee, T.P.; Das, S. Multi-sensor data fusion using support vector machine for motor fault detection. Inf. Sci. 2012, 217, 96–107. [Google Scholar] [CrossRef]
  94. Niu, G.; Yang, B.-S.; Pecht, M. Development of an optimized condition-based maintenance system by data fusion and reliability-centered maintenance. Reliab. Eng. Syst. Saf. 2010, 95, 786–796. [Google Scholar] [CrossRef]
  95. Castaño, F.; Beruvides, G.; Villalonga, A.; Haber, R.E. Computational Intelligence for Simulating a LiDAR Sensor. In Sensor Systems Simulations; Springer: Berlin/Heidelberg, Germany, 2020; pp. 149–178. [Google Scholar]
  96. Premebida, C.; Ludwig, O.; Nunes, U. Exploiting LIDAR-based features on pedestrian detection in urban scenarios. In Proceedings of the 2009 12th International IEEE Conference on Intelligent Transportation Systems, St. Louis, MO, USA, 4–7 October 2009; pp. 1–6. [Google Scholar]
  97. Haber, R.E.; Juanes, C.; Del Toro, R.; Beruvides, G. Artificial cognitive control with self-x capabilities: A case study of a micro-manufacturing process. Comput. Ind. 2015, 74, 135–150. [Google Scholar] [CrossRef] [Green Version]
  98. Beruvides, G.; Juanes, C.; Castaño, F.; Haber, R.E. A self-learning strategy for artificial cognitive control systems. In Proceedings of the 2015 IEEE 13th International Conference on Industrial Informatics (INDIN), Cambridge, UK, 22–24 July 2015; pp. 1180–1185. [Google Scholar]
Remotesensing 11 02252 g001 550
Figure 1. Procedure for sensor reliability assessment using a model-based approach for sensor data and key performance indices.
Figure 1. Procedure for sensor reliability assessment using a model-based approach for sensor data and key performance indices.
Remotesensing 11 02252 g001
Remotesensing 11 02252 g002 550
Figure 2. General scheme of the CPS-based co-simulation framework with virtual and real computing nodes and IoT sensor network.
Figure 2. General scheme of the CPS-based co-simulation framework with virtual and real computing nodes and IoT sensor network.
Remotesensing 11 02252 g002
Remotesensing 11 02252 g003 550
Figure 3. Conceptual flowchart showing the operation of a reliability co-simulation framework with CPS computing nodes and IoT sensors.
Figure 3. Conceptual flowchart showing the operation of a reliability co-simulation framework with CPS computing nodes and IoT sensors.
Remotesensing 11 02252 g003
Remotesensing 11 02252 g004 550
Figure 4. Detailed diagram of the implementation of the LiDAR-based collaborative map approached through a CPS-based co-simulation framework.
Figure 4. Detailed diagram of the implementation of the LiDAR-based collaborative map approached through a CPS-based co-simulation framework.
Remotesensing 11 02252 g004
Remotesensing 11 02252 g005 550
Figure 5. Data interchange between LiDAR sensors in (virtual and real) driving assistance scenarios in Webots for automobiles.
Figure 5. Data interchange between LiDAR sensors in (virtual and real) driving assistance scenarios in Webots for automobiles.
Remotesensing 11 02252 g005
Remotesensing 11 02252 g006 550
Figure 6. (a) Collaborative mapping; (b) obstacles detected in the database; (c) LiDAR data for run-time accuracy error detection.
Figure 6. (a) Collaborative mapping; (b) obstacles detected in the database; (c) LiDAR data for run-time accuracy error detection.
Remotesensing 11 02252 g006
Remotesensing 11 02252 g007 550
Figure 7. Flow diagram between the global node (self-learning module), IoT network, and local nodes (actual failure detection model).
Figure 7. Flow diagram between the global node (self-learning module), IoT network, and local nodes (actual failure detection model).
Remotesensing 11 02252 g007
Remotesensing 11 02252 g008 550
Figure 8. Behavior of LiDAR error of each model with regard to the validation data, (a) on the plane (DRMS) and (b) space (MRSE).
Figure 8. Behavior of LiDAR error of each model with regard to the validation data, (a) on the plane (DRMS) and (b) space (MRSE).
Remotesensing 11 02252 g008
Remotesensing 11 02252 g009 550
Figure 9. Behavior representation of both reliability models. (a) Prediction error behavior, RAE, and (b) correlation index behavior, R2.
Figure 9. Behavior representation of both reliability models. (a) Prediction error behavior, RAE, and (b) correlation index behavior, R2.
Remotesensing 11 02252 g009
Remotesensing 11 02252 g010 550
Figure 10. Q-learning classification error matrix.
Figure 10. Q-learning classification error matrix.
Remotesensing 11 02252 g010
Table
Table 1. Research and innovation actions in sensor reliability supported by H2020 Programme in the last five years.
Table 1. Research and innovation actions in sensor reliability supported by H2020 Programme in the last five years.
Horizon 2020 InitiativeMain SensorsApplication FieldReference
RobustSENSE: Reliable, Secure, Trustable Sensors. For automated drivingLaser scanners, position, Virtual sensorsAutomotiveMäyrä et al. [26]
IoSense: Flexible FE/BE Sensor Pilot Line for the Internet of EverythingMicrophones, force, pressure, gas, camera, LiDAR, accelerometer, othersManufacturing, automotive, energy, environmentCastaño et al. [27]
Godoy et al. [28]
SECREDAS: Cyber Security for Cross Domain Reliable Dependable Automated SystemsCameras, position, ultrasounds, LiDAR, pressureAutomotiveLe et al. [29]
I-MECH: Intelligent Motion Control Platform for Smart Mechatronic SystemsPosition, vision, forceManufacturing, pharmaceutic, healthValencia et al. [30]
PRYSTINE: Programmable Systems for Intelligence in AutomobilesCameras, LiDAR, position, ultrasoundsAutomotiveDruml et al. [31]
Godoy et al. [32]
Power2Power: Providing next-generation silicon-based power solutions in transport and machinery for significant decarbonisation in the next decadeTemperature, PZT, radar, current, voltage, accelerometersManufacturing, energy, industrial machineryGuerra et al. [33]
La Fe et al. [34]
NeXOS: Next Generation Web-Enabled Sensors for the Monitoring of a Changing Oceanoptics, passive acoustics sensors, detectorsEnvironmentToma et al. [35]
ReMAP: Integrated Fleet Management solution aimed at replacing fixed-interval inspections with adaptive condition-based interventionsPiezo-electric, acoustic emission, optical-fibberAeronauticLizé et al. [36]
Table
Table 2. Key performance indices based on plane (DMRS) and space (MRSE) figures of merit.
Table 2. Key performance indices based on plane (DMRS) and space (MRSE) figures of merit.
ModelMAERMSERAERRSER2
DMRSMRSEDMRSMRSEDMRSMRSEDMRSMRSEDMRSMRSE
MLP0.00460.00351.2751.2700.1870.1880.3950.3920.9330.933
k-NN0.0020.00021.0141.0100.1140.1140.3710.3650.9630.961
LR0.67810.65302.3052.2850.7010.6950.7820.7880.4340.435
SVM0.47350.47402.0722.0650.4420.4470.7730.7730.6920.684
Table
Table 3. Q-learning reward matrix for error admissible threshold.
Table 3. Q-learning reward matrix for error admissible threshold.
R2RAE
0–10%10–20%20–40%40–70%>70%
90–100%10.90.80.50.2
80–90%0.850.80.650.40.15
70–80%0.70.60.50.30.1
30–60%0.50.40.30.20.05
0–30%0.30.20.150.10.01

Share and Cite

MDPI and ACS Style

Castaño, F.; Strzelczak, S.; Villalonga, A.; Haber, R.E.; Kossakowska, J. Sensor Reliability in Cyber-Physical Systems Using Internet-of-Things Data: A Review and Case Study. Remote Sens. 2019, 11, 2252. https://doi.org/10.3390/rs11192252

AMA Style

Castaño F, Strzelczak S, Villalonga A, Haber RE, Kossakowska J. Sensor Reliability in Cyber-Physical Systems Using Internet-of-Things Data: A Review and Case Study. Remote Sensing. 2019; 11(19):2252. https://doi.org/10.3390/rs11192252

Chicago/Turabian Style

Castaño, Fernando, Stanisław Strzelczak, Alberto Villalonga, Rodolfo E. Haber, and Joanna Kossakowska. 2019. "Sensor Reliability in Cyber-Physical Systems Using Internet-of-Things Data: A Review and Case Study" Remote Sensing 11, no. 19: 2252. https://doi.org/10.3390/rs11192252

APA Style

Castaño, F., Strzelczak, S., Villalonga, A., Haber, R. E., & Kossakowska, J. (2019). Sensor Reliability in Cyber-Physical Systems Using Internet-of-Things Data: A Review and Case Study. Remote Sensing, 11(19), 2252. https://doi.org/10.3390/rs11192252

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop