Next Article in Journal
Animal Movement Prediction Based on Predictive Recurrent Neural Network
Previous Article in Journal
Modeling and Optimal Design for a High Stability 2D Optoelectronic Angle Sensor
Open AccessArticle

Resource-Efficient Sensor Data Management for Autonomous Systems Using Deep Reinforcement Learning

1
Department of Software, Sungkyunkwan University, Suwon 16419, Korea
2
Department of Mathematics, Sungkyunkwan University, Suwon 16419, Korea
*
Author to whom correspondence should be addressed.
Sensors 2019, 19(20), 4410; https://doi.org/10.3390/s19204410
Received: 23 August 2019 / Revised: 6 October 2019 / Accepted: 9 October 2019 / Published: 11 October 2019
(This article belongs to the Section Internet of Things)
Hyperconnectivity via modern Internet of Things (IoT) technologies has recently driven us to envision “digital twin”, in which physical attributes are all embedded, and their latest updates are synchronized on digital spaces in a timely fashion. From the point of view of cyberphysical system (CPS) architectures, the goals of digital twin include providing common programming abstraction on the same level of databases, thereby facilitating seamless integration of real-world physical objects and digital assets at several different system layers. However, the inherent limitations of sampling and observing physical attributes often pose issues related to data uncertainty in practice. In this paper, we propose a learning-based data management scheme where the implementation is layered between sensors attached to physical attributes and domain-specific applications, thereby mitigating the data uncertainty between them. To do so, we present a sensor data management framework, namely D2WIN, which adopts reinforcement learning (RL) techniques to manage the data quality for CPS applications and autonomous systems. To deal with the scale issue incurred by many physical attributes and sensor streams when adopting RL, we propose an action embedding strategy that exploits their distance-based similarity in the physical space coordination. We introduce two embedding methods, i.e., a user-defined function and a generative model, for different conditions. Through experiments, we demonstrate that the D2WIN framework with the action embedding outperforms several known heuristics in terms of achievable data quality under certain resource restrictions. We also test the framework with an autonomous driving simulator, clearly showing its benefit. For example, with only 30% of updates selectively applied by the learned policy, the driving agent maintains its performance about 96.2%, as compared to the ideal condition with full updates.
Keywords: autonomous system; sensor network; real-time data; digital twin; reinforcement learning; action embedding autonomous system; sensor network; real-time data; digital twin; reinforcement learning; action embedding
MDPI and ACS Style

Jeong, S.; Yoo, G.; Yoo, M.; Yeom, I.; Woo, H. Resource-Efficient Sensor Data Management for Autonomous Systems Using Deep Reinforcement Learning. Sensors 2019, 19, 4410.

Show more citation formats Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop