Next Article in Journal
A Refrigerated Web Camera for Photogrammetric Video Measurement inside Biomass Boilers and Combustion Analysis
Previous Article in Journal
A New Tissue Resonator Indenter Device and Reliability Study
Sensors 2011, 11(1), 1229-1242; doi:10.3390/s110101229
Article

Intelligent Sensing in Dynamic Environments Using Markov Decision Process

1,* , 2
,
3
 and
4
1 Division of Engineering, King’s College, University of London, London, UK 2 Melbourne School of Engineering, The University of Melbourne, Melbourne, VIC, Australia 3 Microsoft Corp., Redmond, WA, USA 4 Crocker Capital, San Francisco, CA, USA
* Author to whom correspondence should be addressed.
Received: 25 November 2010 / Revised: 18 January 2011 / Accepted: 18 January 2011 / Published: 20 January 2011
(This article belongs to the Section Physical Sensors)
View Full-Text   |   Download PDF [1112 KB, 21 June 2014; original version 21 June 2014]   |   Browse Figures

Abstract

In a network of low-powered wireless sensors, it is essential to capture as many environmental events as possible while still preserving the battery life of the sensor node. This paper focuses on a real-time learning algorithm to extend the lifetime of a sensor node to sense and transmit environmental events. A common method that is generally adopted in ad-hoc sensor networks is to periodically put the sensor nodes to sleep. The purpose of the learning algorithm is to couple the sensor’s sleeping behavior to the natural statistics of the environment hence that it can be in optimal harmony with changes in the environment, the sensors can sleep when steady environment and stay awake when turbulent environment. This paper presents theoretical and experimental validation of a reward based learning algorithm that can be implemented on an embedded sensor. The key contribution of the proposed approach is the design and implementation of a reward function that satisfies a trade-off between the above two mutually contradicting objectives, and a linear critic function to approximate the discounted sum of future rewards in order to perform policy learning.
Keywords: sensor network; Markov decision process; sensing; reward shaping sensor network; Markov decision process; sensing; reward shaping
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Share & Cite This Article

Further Mendeley | CiteULike
Export to BibTeX |
EndNote
MDPI and ACS Style

Nanayakkara, T.; Halgamuge, M.N.; Sridhar, P.; Madni, A.M. Intelligent Sensing in Dynamic Environments Using Markov Decision Process. Sensors 2011, 11, 1229-1242.

View more citation formats

Related Articles

Article Metrics

For more information on the journal, click here

Comments

Cited By

[Return to top]
Sensors EISSN 1424-8220 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert