Optimizing Age Penalty in Time-Varying Networks with Markovian and Error-Prone Channel State
Abstract
:1. Introduction
- We study the scheduling strategy for age penalty minimization in multi-sensor bandwidth constrained networks through time-varying and error-prone channel links with power limited sensors. To study a practical network, we model the channel to be a finite-state ergodic Markov chain. The packet loss probability and power consumption depend on the current channel state. Unlike previous work, we model the effect of data staleness in different scenarios via a class of monotone increasing function related to AoI.
- Through relaxing the hard bandwidth constraint and Lagrange multipliers, we decouple the multi-sensor optimization problem into several single-sensor constrained Markov decision process (CMDP) problems. To deal with the potential infinite age penalty, we deduce the threshold structure of the optimal policy and then obtain the approximate optimal single-sensor scheduling decision by solving a truncated linear programming (LP). We prove the solution to the LP is asymptotic optimal when the truncated threshold is sufficiently large.
- The sub-gradient ascend method is applied to find the optimal Lagrange multiplier to satisfy the relaxed bandwidth constraint. Finally, we propose the truncated stationary policy to meet the hard bandwidth constraint. The average performance of the strategy is verified through theoretical analysis and numerical simulations.
2. System Model
2.1. Network Model
2.2. Age of Information and Age Penalty
3. Problem Formulation and Decomposition
3.1. Problem Formulation
3.2. Problem Decomposition
4. Single-Sensor Problem Resolution
4.1. Constrained Markov Decision Process Formulation
- State Space: The state of each sensor consists of two parts: the current AoI and channel state . Thus, is infinite but countable.
- Action Space: There are two possible actions in the action space for the scheduling policy, denoted by . Action means the sensor chooses to schedule while means idling. Notice that here does not need to satisfy the bandwidth constraint.
- One-Step Cost: The one-step cost consists of two parts: the age penalty growth and scheduling penalty, which can be computed byAnd the one-step power consumption is
4.2. Characterization of the Optimal Policy
4.3. Linear Programming Approximation
5. Multi-Sensor Problem Resolution
5.1. The Relaxed Problem Resolution
Algorithm 1 Construction of the optimal stationary policy |
Initialization: , , , , and |
for each do |
compute and |
end for |
if then |
else |
while or do |
if then |
else |
end if |
for each do |
compute and |
end for |
if and then |
end if |
if and then |
end if |
end while |
end if |
5.2. Truncation for the Hard Bandwidth Constraint
- In slot t, compute the scheduling set according to the optimal stationary policy .
- If , then schedules all these sensors as does.
- If , the hard bandwidth constraint is never satisfied. Therefore, randomly chooses M out of sensors to be scheduled in the current slot.
6. Simulation Results
6.1. Average Age Penalty Performance
6.2. Sensor Level Analysis and Threshold Structure
7. Conclusions
Author Contributions
Funding
Conflicts of Interest
Appendix A. Proof of Theorem 1
Appendix B. Proof of Lemma 1
- If , for any ;
- If , for any , ,where is any monotone increasing function;
- If and , then ;
- If and , then:
Appendix C. Proof of Lemma A1
Appendix D. Derivation of Problem 4
Appendix E. Proof of Theorem 3
Appendix F. Proof of Theorem 4
References
- Kaul, S.; Gruteser, M.; Rai, V.; Kenney, J. Minimizing age of information in vehicular networks. In Proceedings of the 2011 8th Annual IEEE Communications Society Conference on Sensor, Mesh and Ad Hoc Communications and Networks, Salt Lake City, UT, USA, 27–30 June 2011; pp. 350–358. [Google Scholar]
- Zhou, B.; Saad, W. Optimal Sampling and Updating for Minimizing Age of Information in the Internet of Things. In Proceedings of the 2018 IEEE Global Communications Conference (GLOBECOM), Abu Dhabi, UAE, 9–13 December 2018; pp. 1–6. [Google Scholar]
- Kaul, S.; Yates, R.; Gruteser, M. Real-time status: How often should one update? In Proceedings of the 2012 Proceedings IEEE INFOCOM, Orlando, FL, USA, 25–30 March 2012; pp. 2731–2735. [Google Scholar]
- Chen, K.; Huang, L. Age-of-information in the presence of error. In Proceedings of the 2016 IEEE International Symposium on Information Theory (ISIT), Barcelona, Spain, 10–15 July 2016; pp. 2579–2583. [Google Scholar]
- Costa, M.; Codreanu, M.; Ephremides, A. On the Age of Information in Status Update Systems with Packet Management. IEEE Trans. Inf. Theory 2016, 62, 1897–1910. [Google Scholar] [CrossRef]
- Najm, E.; Yates, R.; Soljanin, E. Status updates through M/G/1/1 queues with HARQ. In Proceedings of the 2017 IEEE International Symposium on Information Theory (ISIT), Aachen, Germany, 25–30 June 2017; pp. 131–135. [Google Scholar]
- Kosta, A.; Pappas, N.; Ephremides, A.; Angelakis, V. Age and value of information: Non-linear age case. In Proceedings of the 2017 IEEE International Symposium on Information Theory (ISIT), Aachen, Germany, 25–30 June 2017; pp. 326–330. [Google Scholar]
- Wang, X.; Duan, L. Dynamic Pricing for Controlling Age of Information. In Proceedings of the 2019 IEEE International Symposium on Information Theory (ISIT), Paris, France, 7–12 July 2019; pp. 962–966. [Google Scholar]
- Sun, Y.; Uysal-Biyikoglu, E.; Yates, R.; Koksal, C.E.; Shroff, N.B. Update or wait: How to keep your data fresh. IEEE Trans. Inf. Theory 2017, 63, 7492–7508. [Google Scholar] [CrossRef]
- Ceran, E.T.; Gündüz, D.; György, A. Average Age of Information With Hybrid ARQ Under a Resource Constraint. IEEE Trans. Wirel. Commun. 2019, 18, 1900–1913. [Google Scholar] [CrossRef] [Green Version]
- Bacinoglu, B.T.; Ceran, E.T.; Uysal-Biyikoglu, E. Age of information under energy replenishment constraints. In Proceedings of the 2015 Information Theory and Applications Workshop (ITA), San Diego, CA, USA, 1–6 February 2015; pp. 25–31. [Google Scholar]
- Kadota, I.; Uysal-Biyikoglu, E.; Singh, R.; Modiano, E. Minimizing the Age of Information in broadcast wireless networks. In Proceedings of the 2016 54th Annual Allerton Conference on Communication, Control, and Computing (Allerton), Monticello, IL, USA, 27–30 September 2016; pp. 844–851. [Google Scholar]
- Maatouk, A.; Kriouile, S.; Assaad, M.; Ephremides, A. On the Optimality of the Whittle’s Index Policy for Minimizing the Age of Information. arXiv 2020, arXiv:2001.03096. [Google Scholar] [CrossRef]
- Kadota, I.; Sinha, A.; Modiano, E. Scheduling Algorithms for Optimizing Age of Information in Wireless Networks with Throughput Constraints. IEEE/ACM Trans. Netw. 2019, 27, 1359–1372. [Google Scholar] [CrossRef]
- Kadota, I.; Sinha, A.; Uysal-Biyikoglu, E.; Singh, R.; Modiano, E. Scheduling Policies for Minimizing Age of Information in Broadcast Wireless Networks. IEEE/ACM Trans. Netw. 2018, 26, 2637–2650. [Google Scholar] [CrossRef] [Green Version]
- Bedewy, A.M.; Sun, Y.; Shroff, N.B. Age-optimal information updates in multihop networks. In Proceedings of the 2017 IEEE International Symposium on Information Theory (ISIT), Aachen, Germany, 25–30 June 2017; pp. 576–580. [Google Scholar]
- Chen, X.; Gatsis, K.; Hassani, H.; Bidokhti, S.S. Age of Information in Random Access Channels. In Proceedings of the 2020 IEEE International Symposium on Information Theory (ISIT), Los Angeles, CA, USA, 21–26 June 2020. [Google Scholar]
- Hsu, Y.; Modiano, E.; Duan, L. Age of information: Design and analysis of optimal scheduling algorithms. In Proceedings of the 2017 IEEE International Symposium on Information Theory (ISIT), Aachen, Germany, 25–30 June 2017; pp. 561–565. [Google Scholar]
- Talak, R.; Kadota, I.; Karaman, S.; Modiano, E. Scheduling Policies for Age Minimization in Wireless Networks with Unknown Channel State. In Proceedings of the 2018 IEEE International Symposium on Information Theory (ISIT), Vail, CO, USA, 17–22 June 2018; pp. 2564–2568. [Google Scholar]
- Sun, Y.; Polyanskiy, Y.; Uysal, E. Sampling of the Wiener Process for Remote Estimation over a Channel with Random Delay. IEEE Trans. Inf. Theory 2019, 66, 1118–1135. [Google Scholar] [CrossRef]
- Wang, J.; Ren, X.; Mo, Y.; Shi, L. Whittle Index Policy for Dynamic Multichannel Allocation in Remote State Estimation. IEEE Trans. Autom. Control 2020, 65, 591–603. [Google Scholar] [CrossRef]
- Tang, H.; Wang, J.; Song, L.; Song, J. Minimizing Age of Information With Power Constraints: Multi-User Opportunistic Scheduling in Multi-State Time-Varying Channels. IEEE J. Sel. Areas Commun. 2020, 38, 854–868. [Google Scholar] [CrossRef] [Green Version]
- Wu, S.; Han, D.; Cheng, P.; Shi, L. Optimal Scheduling of Multiple Sensors over Lossy and Bandwidth Limited Channels. arXiv 2018, arXiv:1804.05618. [Google Scholar] [CrossRef] [Green Version]
- Beutler, F.J.; Ross, K.W. Optimal policies for controlled Markov chains with a constraint. J. Math. Anal. Appl. 1985, 112, 236–252. [Google Scholar] [CrossRef] [Green Version]
- Tajan, R.; Ciblat, P. Information-theoretic multi-user power adaptation in retransmission schemes. In Proceedings of the 2016 IEEE 17th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC), Edinburgh, UK, 3–6 July 2016; pp. 1–5. [Google Scholar]
- Hernández-Lerma, O.; Lasserre, J.B. Discrete-Time Markov Control Processes: Basic Optimality Criteria; Springer Science & Business Media: Berlin, Germany, 2012; Volume 30. [Google Scholar]
- Diaconis, P.; Zabell, S. Closed form summation for classical distributions: Variations on a theme of de Moivre. Stat. Sci. 1991, 6, 284–302. [Google Scholar] [CrossRef]
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Chen, Y.; Tang, H.; Wang, J.; Song, J. Optimizing Age Penalty in Time-Varying Networks with Markovian and Error-Prone Channel State. Entropy 2021, 23, 91. https://doi.org/10.3390/e23010091
Chen Y, Tang H, Wang J, Song J. Optimizing Age Penalty in Time-Varying Networks with Markovian and Error-Prone Channel State. Entropy. 2021; 23(1):91. https://doi.org/10.3390/e23010091
Chicago/Turabian StyleChen, Yuchao, Haoyue Tang, Jintao Wang, and Jian Song. 2021. "Optimizing Age Penalty in Time-Varying Networks with Markovian and Error-Prone Channel State" Entropy 23, no. 1: 91. https://doi.org/10.3390/e23010091
APA StyleChen, Y., Tang, H., Wang, J., & Song, J. (2021). Optimizing Age Penalty in Time-Varying Networks with Markovian and Error-Prone Channel State. Entropy, 23(1), 91. https://doi.org/10.3390/e23010091