Resource Optimization for 3D Video SoftCast with Joint Texture/Depth Power Allocation
Abstract
:1. Introduction
- The optimal solutions for minimum and target distortion problems are obtained in terms of texture and depth using proposed resource allocation algorithms, and the trade-off relationship between power and bandwidth usage is analyzed.
- The proposed resource control scheme for 3D transmission integrates the optimal PAR with resource allocation algorithms, ensuring balanced power allocation between texture and depth to improve the transmission results.
2. Related Work
2.1. SoftCast Video Transmission
2.2. SoftCast Power Allocation
2.3. Resource Allocation Optimization Problem
3. Minimum Distortion Optimization
3.1. Investigation and Motivation
3.2. Problem Formulation
3.3. Problem Solution
Algorithm 1: Minimum Distortion Optimization. |
Input: [texure,depth] |
Output: |
initialization: , |
1: for to N |
2: Compute chunk variance and |
3: end |
4: for to 2 |
5: for to N |
6: ; |
7: Calculate optimal power allocation |
8: Calculate MSE via (12). |
9: Obtain PSNR = 10log |
10: end |
11: PSNR PSNR. |
12: ; |
13: for to N |
14: If PSNRPSNR |
15: ; |
16: break; |
17: end |
18: end |
19: Calculate using as in (11) and (17); |
20: end |
4. Target Distortion Optimization
4.1. Investigation and Motivation
4.2. Problem Formulation
4.3. Problem Solution
4.3.1. The Power Distortion Optimization
Algorithm 2: Power Distortion Optimization. |
Input: [texure,depth] |
Output: |
initialization: |
1: for to N |
2: Compute chunk variance and |
3: end |
4: for to 2 |
5: for to |
6: ; |
7: Calculate optimal power allocation via (26) |
8: Calculate via (21) . |
9: If ( and |
10: break; |
11: end |
12: end |
13: |
14: end |
4.3.2. The Bandwidth Distortion Optimization
Algorithm 3: Bandwidth Distortion Optimization. |
Input: [texure,depth] |
Output: |
initialization: |
1: for to N |
2: Compute chunk variance and |
3: end |
4: for to 2 |
5: while do |
6: |
7: Calculate optimal power allocation via (29) |
8: Calculate via (27) |
9: end |
10: for all i |
11: end |
4.3.3. Power and Bandwidth Trade-Off
5. Simulation Results
5.1. Performance Evaluation for PAR
5.2. Minimum Distortion Optimization Performance
5.3. Target Distortion Optimization Performance
5.3.1. Power Distortion Optimization Performance
5.3.2. Bandwidth Distortion Optimization Performance
5.3.3. Power and Bandwidth Trade-Off
5.4. The Resource Control Scheme Performance Comparison
5.5. Complexity Analysis
6. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Merkle, P.; Smolic, A.; Muller, K.; Wiegand, T. Multi-view video plus depth representation and coding. In Proceedings of the 2007 IEEE International Conference on Image Processing, San Antonio, TX, USA, 16 September–19 October 2007; Volume 1, pp. 201–204. [Google Scholar]
- Cao, B.; Xia, S.; Han, J.; Li, Y. A distributed game methodology for crowdsensing in uncertain wireless scenario. IEEE Trans. Mob. Comput. 2020, 19, 15–28. [Google Scholar] [CrossRef]
- Fehn, C. Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV. In Proceedings of the Stereoscopic Displays and Virtual Reality Systems XI, San Jose, CA, USA, 19–22 January 2004; Volume 5291, pp. 93–104. [Google Scholar]
- Zhu, C.; Li, S. Depth image based view synthesis: New insights and perspectives on hole generation and filling. IEEE Trans. Broadcast. 2016, 62, 82–93. [Google Scholar] [CrossRef]
- Li, S.; Zhu, C.; Sun, M.T. Hole filling with multiple reference views in DIBR view synthesis. IEEE Trans. Multimed. 2018, 20, 1948–1959. [Google Scholar] [CrossRef] [Green Version]
- Shannon, C.E. A mathematical theory of communication. Bell Syst. Tech. J. 1948, 27, 379–423. [Google Scholar] [CrossRef] [Green Version]
- Vembu, S.; Verdu, S.; Steinberg, Y. The source-channel separation theorem revisited. IEEE Trans. Inf. Theory 1995, 41, 44–54. [Google Scholar] [CrossRef] [Green Version]
- Yuan, H.; Fu, H.; Liu, J.; Xiao, J. End-to-end distortion-based multiuser bandwidth allocation for real-time video transmission over LTE network. IEEE Trans. Broadcast. 2017, 63, 338–349. [Google Scholar] [CrossRef]
- Kratochvíl, T.; Štukavec, R. DVB-T digital terrestrial television transmission over fading channels. Radioengineering 2008, 17, 96–102. [Google Scholar]
- Katabi, D.; Rahul, H.; Jakubczak, S. Softcast: One Video to Serve All Wireless Receivers; Tech. Rep. MIT-CSAIL-TR-2009-005; Massachusetts Institute of Technology: Cambridge, MA, USA, 2009. [Google Scholar]
- Jakubczak, S.; Katabi, D. SoftCast: One-size-fits-all wireless video. In Proceedings of the ACM SIGCOMM 2010 Conference, New Delhi, India, 30 August–3 September 2010; pp. 449–450. [Google Scholar]
- Jakubczak, S.; Katabi, D. A cross-layer design for scalable mobile video. In Proceedings of the 17th Annual International Conference on Mobile Computing and Networking, Las Vegas, NV, USA, 19–23 September 2011; pp. 289–300. [Google Scholar]
- Giannopoulos, A.; Spantideas, S.; Kapsalis, N.; Karkazis, P.; Trakadas, P. Deep reinforcement learning for energy-efficient multi-channel transmissions in 5G cognitive hetnets: Centralized, decentralized and transfer learning based solutions. IEEE Access 2021, 9, 129358–129374. [Google Scholar] [CrossRef]
- Cumino, P.; Lobato Junior, W.; Tavares, T.; Santos, H.; Rosário, D.; Cerqueira, E.; Villas, L.A.; Gerla, M. Cooperative UAV scheme for enhancing video transmission and global network energy efficiency. Sensors 2018, 18, 4155. [Google Scholar] [CrossRef] [Green Version]
- Fan, X.; Xiong, R.; Wu, F.; Zhao, D. WaveCast: Wavelet based wireless video broadcast using lossy transmission. In Proceedings of the 2012 Visual Communications and Image Processing, San Diego, CA, USA, 27–30 November 2012; pp. 1–6. [Google Scholar]
- Fan, X.; Wu, F.; Zhao, D.; Au, O.C. Distributed wireless visual communication with power distortion optimization. IEEE Trans. Circuits Syst. Video Technol. 2013, 23, 1040–1053. [Google Scholar] [CrossRef]
- Xiong, R.; Wu, F.; Fan, X.; Luo, C.; Ma, S.; Gao, W. Power-distortion optimization for wireless image/video SoftCast by transform coefficients energy modeling with adaptive chunk division. In Proceedings of the 2013 Visual Communications and Image Processing (VCIP), Kuching, Malaysia, 17–20 November 2013; pp. 1–6. [Google Scholar]
- Cui, H.; Xiong, R.; Luo, C.; Song, Z.; Wu, F. Denoising and resource allocation in uncoded video transmission. IEEE J. Sel. Top. Signal Process. 2015, 9, 102–112. [Google Scholar] [CrossRef]
- Balsa, J.; Fresnedo, Ó.; García-Naya, J.A.; Domínguez-Bolaño, T.; Castedo, L. JSCC-Cast: A Joint Source Channel Coding Video Encoding and Transmission System with Limited Digital Metadata. Sensors 2021, 21, 6208. [Google Scholar] [CrossRef]
- Cui, H.; Song, Z.; Yang, Z.; Luo, C.; Xiong, R.; Wu, F. Cactus: A hybrid digital-analog wireless video communication system. In Proceedings of the 16th ACM International Conference on Modeling, Analysis & Simulation of Wireless and Mobile Systems, Barcelona, Spain, 3–8 November 2013; pp. 273–278. [Google Scholar]
- Liu, X.L.; Hu, W.; Luo, C.; Pu, Q.; Wu, F.; Zhang, Y. ParCast+: Parallel video unicast in MIMO-OFDM WLANs. IEEE Trans. Multimed. 2014, 16, 2038–2051. [Google Scholar] [CrossRef]
- He, D.; Luo, C.; Lan, C.; Wu, F.; Zeng, W. Structure-preserving hybrid digital-analog video delivery in wireless networks. IEEE Trans. Multimed. 2015, 17, 1658–1670. [Google Scholar] [CrossRef]
- Yu, L.; Li, H.; Li, W. Wireless scalable video coding using a hybrid digital-analog scheme. IEEE Trans. Circuits Syst. Video Technol. 2014, 24, 331–345. [Google Scholar] [CrossRef]
- Yu, L.; Li, H.; Li, W. Wireless cooperative video coding using a hybrid digital–analog scheme. IEEE Trans. Circuits Syst. Video Technol. 2015, 25, 436–450. [Google Scholar] [CrossRef]
- Liang, F.; Luo, C.; Xiong, R.; Zeng, W.; Wu, F. Superimposed Modulation for Soft Video Delivery with Hidden Resources. IEEE Trans. Circuits Syst. Video Technol. 2018, 28, 2345–2358. [Google Scholar] [CrossRef]
- He, C.; Wang, H.; Hu, Y.; Chen, Y.; Fan, X.; Li, H.; Zeng, B. MCast: High-quality linear video transmission with time and frequency diversities. IEEE Trans. Image Process. 2018, 27, 3599–3610. [Google Scholar] [CrossRef]
- He, C.; Hu, Y.; Chen, Y.; Fan, X.; Li, H.; Zeng, B. Mucast: Linear uncoded multiuser video streaming with channel assignment and power allocation optimization. IEEE Trans. Circuits Syst. Video Technol. 2020, 30, 1136–1146. [Google Scholar] [CrossRef]
- Yang, T.; Luo, L.; Zhu, C.; Tang, S. Block DCT based optimization for wireless SoftCast of depth map. IEEE Access 2019, 7, 29484–29494. [Google Scholar] [CrossRef]
- Luo, L.; Yang, T.; Zhu, C.; Jin, Z.; Tang, S. Joint texture/depth power allocation for 3-D video SoftCast. IEEE Trans. Multimed. 2019, 21, 2973–2984. [Google Scholar] [CrossRef]
- Liu, D.; Wu, J.; Cui, H.; Zhang, D.; Luo, C.; Wu, F. Cost-distortion optimization and resource control in pseudo-analog visual communications. IEEE Trans. Multimed. 2018, 20, 3097–3110. [Google Scholar] [CrossRef]
- Zhang, T.; Mao, S. Joint power and channel resource optimization in soft multi-view video delivery. IEEE Access 2019, 7, 148084–148097. [Google Scholar] [CrossRef]
- Xiong, R.; Wu, F.; Xu, J.; Fan, X.; Luo, C.; Gao, W. Analysis of decorrelation transform gain for uncoded wireless image and video communication. IEEE Trans. Image Process. 2016, 25, 1820–1833. [Google Scholar]
- Cui, H.; Luo, C.; Chen, C.W.; Wu, F. Robust uncoded video transmission over wireless fast fading channel. In Proceedings of the IEEE INFOCOM 2014-IEEE Conference on Computer Communications, Toronto, ON, Canada, 27 April–2 May 2014; pp. 73–81. [Google Scholar]
- Saito, T. Nagoya University Multi-View Sequences Download List. Nagoya University, Fujii Laboratory. 2015. Available online: http://www.fujii.nuee.nagoya-u.ac.jp/multiview-data/ (accessed on 1 May 2015).
- Domañski, M.; Grajek, T.; Klimaszewski, K.; Kurc, M.; Stankiewicz, O.; Stankowski, J.; Wegner, K. Poznan multiview video test sequences and camera parameters. Paper presented at the MPEG/M17050, Xian, China, October 2009. ISO/IEC JTC1/SC29/WG11. [Google Scholar]
- Fraunhofer Heinrich Hertz Institute. 3D High Efficiency Video Coding (3D-HEVC)—JCT-VC. Available online: https://hevc.hhi.fraunhofer.de/3dhevc (accessed on 10 September 2021).
Sequence | 3D Scene Nature | Reference Viewpoints | Synthesized Viewpoints | Resolution |
---|---|---|---|---|
Balloons | real | 1 and 5 | 2, 3,4 | 1024 × 768 |
Kendo | real | 1 and 5 | 2, 3, 4 | 1024 × 768 |
Newspaper | real | 2 and 5 | 2.5, 3, 3.5 | 1024 × 768 |
Dancer | computer-generated | 1 and 5 | 2, 3, 4 | 1920 × 1088 |
gtFly | computer-generated | 1 and 5 | 2, 3, 4 | 1920 × 1088 |
PoznanHall2 | real | 5 and 7 | 5.5, 6, 6.5 | 1920 × 1088 |
PoznanStreet | real | 3 and 5 | 3.5, 4, 4.5 | 1920 × 1088 |
Shark | computer-generated | 1 and 5 | 2, 3, 4 | 1920 × 1088 |
Sequence | Estimated PAR | Global Optimal PAR |
---|---|---|
Balloons | 2.23 | 2.05 |
Kendo | 2.55 | 2.60 |
Newspaper | 3.32 | 2.85 |
Dancer | 3.08 | 3.65 |
gtFly | 5.78 | 7.00 |
PoznanHall2 | 2.84 | 3.45 |
PoznanStreet | 2.24 | 2.25 |
Shark | 4.50 | 4.25 |
(a) = 20 dB | |||
---|---|---|---|
3D Video Sequences | SoftCast | Distortion Resource Algorithm | Proposed Resource Control Scheme |
Balloons | 100% | 67.5292% | 51.1108% |
Kendo | 100% | 60.0586% | 45.8862% |
Newspaper | 100% | 65.1855% | 48.3398% |
Dancer | 100% | 89.3554% | 56.4208% |
gtFly | 100% | 91.5039% | 49.1577% |
PoznanHall2 | 100% | 56.9824% | 34.9609% |
PoznanStreet | 100% | 57.0801% | 49.9633% |
Shark | 100% | 95.0195% | 83.1665% |
(b) = 15 dB | |||
3D Video Sequences | SoftCast | Distortion Resource Algorithm | Proposed Resource Control Scheme |
Balloons | 100% | 36.9629% | 19.5922% |
Kendo | 100% | 30.6641% | 18.1274% |
Newspaper | 100% | 33.8379% | 17.6775% |
Dancer | 100% | 73.3398% | 40.8203% |
gtFly | 100% | 75.0977% | 41.6748% |
PoznanHall2 | 100% | 37.5000% | 21.6919% |
PoznanStreet | 100% | 34.6191% | 24.9146% |
Shark | 100% | 84.9609% | 59.1309% |
(c) = 10 dB | |||
3D Video Sequences | SoftCast | Distortion Resource Algorithm | Proposed Resource Control Scheme |
Balloons | 100% | 9.3262% | 4.5898% |
Kendo | 100% | 12.0117% | 7.2875% |
Newspaper | 100% | 10.4980% | 4.3212% |
Dancer | 100% | 40.3809% | 22.4365% |
gtFly | 100% | 39.2578% | 25.8056% |
PoznanHall2 | 100% | 19.5801% | 11.7187% |
PoznanStreet | 100% | 17.2852% | 10.3271% |
Shark | 100% | 49.7558% | 19.2138% |
(d) = 5 dB | |||
3D Video Sequences | SoftCast | Distortion Resource Algorithm | Proposed Resource Control Scheme |
Balloons | 100% | 2.7344% | 1.5380% |
Kendo | 100% | 4.7852% | 2.8930% |
Newspaper | 100% | 3.4180% | 1.5014% |
Dancer | 100% | 11.9141% | 6.8603% |
gtFly | 100% | 8.9356% | 8.0688% |
PoznanHall2 | 100% | 5.8594% | 3.4058% |
PoznanStreet | 100% | 6.3965% | 3.4790% |
Shark | 100% | 12.3046% | 7.0313% |
(e) = 0 dB | |||
3D Video Sequences | SoftCast | Distortion Resource Algorithm | Proposed Resource Control Scheme |
Balloons | 100% | 1.0254% | 0.6225% |
Kendo | 100% | 2.1484% | 1.1352% |
Newspaper | 100% | 1.1230% | 0.6225% |
Dancer | 100% | 2.5391% | 1.5136% |
gtFly | 100% | 1.2207% | 1.3672% |
PoznanHall2 | 100% | 2.2461% | 0.9521% |
PoznanStreet | 100% | 2.2539% | 1.2939% |
Shark | 100% | 3.4179% | 2.0019% |
(a) = 20 dB | ||||||
---|---|---|---|---|---|---|
3D Video Sequences | SoftCast | Distortion Resource Algorithm | Proposed Resource Control Scheme | |||
PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | |
Balloons | 49.3196 | 0.9964 | 46.1196 | 0.9942 | 46.2761 | 0.9943 |
Kendo | 49.7899 | 0.9957 | 47.6392 | 0.9936 | 48.0136 | 0.9939 |
Newspaper | 49.2141 | 0.9963 | 46.6435 | 0.9943 | 46.9753 | 0.9946 |
Dancer | 46.0253 | 0.9916 | 43.2292 | 0.9893 | 43.6010 | 0.9913 |
gtFly | 48.9609 | 0.9930 | 47.6194 | 0.9916 | 47.1173 | 0.9927 |
PoznanHall2 | 53.1829 | 0.9969 | 50.1385 | 0.9944 | 51.2116 | 0.9957 |
PoznanStreet | 48.2154 | 0.9952 | 45.1048 | 0.9912 | 45.9560 | 0.9925 |
Shark | 48.1204 | 0.9933 | 46.3205 | 0.9899 | 47.0311 | 0.9929 |
(b) = 15 dB | ||||||
3D Video Sequences | SoftCast | Distortion Resource Algorithm | Proposed Resource Control Scheme | |||
PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | |
Balloons | 45.4115 | 0.9928 | 42.5971 | 0.9878 | 42.5716 | 0.9878 |
Kendo | 45.8277 | 0.9911 | 43.9952 | 0.9862 | 44.3260 | 0.9875 |
Newspaper | 45.1731 | 0.9921 | 42.8919 | 0.9877 | 43.0334 | 0.9884 |
Dancer | 41.5428 | 0.9781 | 39.2760 | 0.9740 | 39.6065 | 0.9790 |
gtFly | 45.1369 | 0.9845 | 43.6927 | 0.9814 | 43.8269 | 0.9852 |
PoznanHall2 | 49.0760 | 0.9935 | 46.2265 | 0.9882 | 47.1799 | 0.9905 |
PoznanStreet | 43.9320 | 0.9888 | 41.5915 | 0.9812 | 42.2585 | 0.9837 |
Shark | 44.1887 | 0.9845 | 42.5072 | 0.9776 | 42.9480 | 0.9847 |
(c) = 10 dB | ||||||
3D Video Sequences | SoftCast | Distortion Resource Algorithm | Proposed Resource Control Scheme | |||
PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | |
Balloons | 41.3268 | 0.9830 | 39.2838 | 0.9743 | 39.3090 | 0.9769 |
Kendo | 41.5470 | 0.9780 | 39.8723 | 0.9696 | 40.6042 | 0.9760 |
Newspaper | 40.7441 | 0.9801 | 39.0226 | 0.9728 | 38.7993 | 0.9733 |
Dancer | 37.0933 | 0.9402 | 35.0093 | 0.9329 | 35.3295 | 0.9453 |
gtFly | 40.6633 | 0.9588 | 39.0618 | 0.9545 | 40.3215 | 0.9655 |
PoznanHall2 | 44.5969 | 0.9844 | 42.3917 | 0.9731 | 43.6213 | 0.9788 |
PoznanStreet | 39.5928 | 0.9713 | 37.7700 | 0.9557 | 38.5942 | 0.9634 |
Shark | 39.7686 | 0.9587 | 38.5368 | 0.9516 | 38.6494 | 0.9663 |
(d) = 5 dB | ||||||
3D Video Sequences | SoftCast | Distortion Resource Algorithm | Proposed Resource Control Scheme | |||
PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | |
Balloons | 36.8079 | 0.9526 | 35.8347 | 0.9512 | 36.2920 | 0.9613 |
Kendo | 36.8431 | 0.9389 | 35.5959 | 0.9398 | 36.6900 | 0.9568 |
Newspaper | 36.1632 | 0.9460 | 35.0736 | 0.9391 | 35.3860 | 0.9479 |
Dancer | 32.4094 | 0.8434 | 30.9076 | 0.8469 | 31.6366 | 0.8695 |
gtFly | 36.0398 | 0.8898 | 34.4277 | 0.9028 | 36.0597 | 0.9268 |
PoznanHall2 | 40.0081 | 0.9584 | 38.4855 | 0.9466 | 39.8148 | 0.9566 |
PoznanStreet | 34.8735 | 0.9192 | 33.8732 | 0.9069 | 34.8194 | 0.9270 |
Shark | 35.0505 | 0.8880 | 34.0345 | 0.9067 | 34.9220 | 0.9307 |
(e) = 0 dB | ||||||
3D Video Sequences | SoftCast | Distortion Resource Algorithm | Proposed Resource Control Scheme | |||
PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | |
Balloons | 31.9165 | 0.8578 | 32.0593 | 0.9015 | 32.9264 | 0.9328 |
Kendo | 31.6785 | 0.8227 | 31.3943 | 0.8887 | 32.7896 | 0.9280 |
Newspaper | 31.1809 | 0.8513 | 30.9788 | 0.8707 | 32.0076 | 0.9064 |
Dancer | 27.4676 | 0.6483 | 27.8362 | 0.7346 | 28.5260 | 0.7732 |
gtFly | 31.1336 | 0.7310 | 30.8460 | 0.8469 | 32.6345 | 0.8842 |
PoznanHall2 | 35.0796 | 0.8827 | 34.7832 | 0.9127 | 36.2844 | 0.9345 |
PoznanStreet | 29.7831 | 0.7757 | 30.2480 | 0.8329 | 31.2267 | 0.8715 |
Shark | 29.9380 | 0.7188 | 30.2311 | 0.8336 | 31.2955 | 0.8776 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Thabet, S.K.S.; Osei-Mensah, E.; Ahmed, O.; Seid, A.M.; Bamisile, O. Resource Optimization for 3D Video SoftCast with Joint Texture/Depth Power Allocation. Appl. Sci. 2022, 12, 5047. https://doi.org/10.3390/app12105047
Thabet SKS, Osei-Mensah E, Ahmed O, Seid AM, Bamisile O. Resource Optimization for 3D Video SoftCast with Joint Texture/Depth Power Allocation. Applied Sciences. 2022; 12(10):5047. https://doi.org/10.3390/app12105047
Chicago/Turabian StyleThabet, Saqr Khalil Saeed, Emmanuel Osei-Mensah, Omar Ahmed, Abegaz Mohammed Seid, and Olusola Bamisile. 2022. "Resource Optimization for 3D Video SoftCast with Joint Texture/Depth Power Allocation" Applied Sciences 12, no. 10: 5047. https://doi.org/10.3390/app12105047
APA StyleThabet, S. K. S., Osei-Mensah, E., Ahmed, O., Seid, A. M., & Bamisile, O. (2022). Resource Optimization for 3D Video SoftCast with Joint Texture/Depth Power Allocation. Applied Sciences, 12(10), 5047. https://doi.org/10.3390/app12105047