Reinforcement Learning in Blockchain-Enabled IIoT Networks: A Survey of Recent Advances and Open Challenges
Abstract
:1. Introduction
1.1. Characteristics and Overview of Reinforcement Learning Techniques
1.1.1. Model-Based RL
1.1.2. Model-Free RL
- Q-learning: Q-learning is the most studied and well-known RL technique that has been used for solving different sorts of problems. In Q-learning, an agent takes action based on the Q-values in the Q-table [20]. There are four major components of a Q-network, i.e., a stage set, an action set, a reward, and transition probabilities. For each state, the agent executes some action under its pre-defined policy. Subsequently, the agent adopts the policy such that it has maximum Q-value. After each sequence, the agent revises the Q-table for a more accurate estimation of the Q-values and updating the policy of the agent. Thus, in due course of time and after many steps, the policy converges to the optimal policy of the agent.
- Multi-armed Bandit Learning: The agent in the multi-armed bandit approach selects the action without having the state information of the environment. Relevant to the action performed by the action time step, the agent receives a reward that is maximized in the next iterations [14]. Since the agent is unaware of the environment and the associated reward, there always exists a tradeoff between exploitation and exploration. Due to this reason, the multi-armed bandit learning, although lower in complexity, is not efficient for highly dynamic environments.
- Actor-critic Learning: The actor-critic learning divides the agent into two roles, i.e., actor and critic [20]. The action selection policies over the action space are represented by the actor. In contrast, the critic is the observer which anticipates the expected reward received in the future by passing through the same state. Based on the observed reward, the state values are updated by the critic. Here, the critic can be considered as a trainer which trains the actor to improve its stability and select the optimal action in each state. Using the probability density function given by the actor, the size of the action space does not increase the complexity of the action selection. Therefore, it is much suitable for continuous and large action state spaces.
- Miscellaneous: Other techniques include policy optimization, soft actor-critic, and deep policy optimization techniques. These techniques are mostly variants of the above-mentioned RL techniques that use entropy regularization, and stochastic policies to stabilize RL. Sometimes the conventional RL techniques may not be feasible for large-scale networks. In other words, the dimensionality of state, action, and reward could render the problem difficult to solve. Thus, for high dimensional optimization, deep neural networks have been proposed to work along with conventional RL techniques to improve the network. These deep neural networks can be used to estimate the Q-values during each iteration, also known as deep Q-network [20]. The agent can select the maximum estimated value and store the experience again to train the neural network for estimation. Nevertheless, it is much more complex and computationally exhaustive than conventional RL techniques.
1.2. Motivation
1.3. Related Surveys and Our Contributions
- Identifying the integration challenges of blockchain with IIoT and formulating a problem statement for a case study.
- A thorough review of the current paradigms in blockchain and their associated consensus algorithms.
- A review of reinforcement learning, its characteristics, and how they can help address the integration problems of blockchain and IIoT.
- An abstract representation of a blockchain-IIoT network with three-layer network architecture, i.e., Physical Layer, Network Layer, and Application Layer.
- A concise review of how RL can be used to power different avenues in blockchain-enabled IIoT networks.
- A case study to demonstrate the feasibility and improvements introduced by using RL in a blockchain-IIoT network.
1.4. Organization
2. Blockchain-enabled IIoT Networks
2.1. Improved Security
2.2. Data Integrity
2.3. Cost Effectiveness
2.4. Trustless
2.5. Autonomy
2.6. Smart Contracts
3. Recent Studies and Gap Analysis
3.1. Physical Layer
3.2. Network Layer
3.3. Application Layer
4. Applications of RL in Blockchain-Enabled IIoT Networks
4.1. Minimizing Forking Events
4.2. Improving Energy Efficiency
4.3. Time to Finality Minimization
4.4. Enhancing Transaction Throughput
4.5. Improving Link Security
4.6. Average Block Time Reduction
5. Case Study: Minimization of Forking in Blockchain
5.1. Problem Formulation
5.2. Algorithm Design
5.2.1. States
5.2.2. Actions
5.2.3. Reward
5.3. Results and Discussion
6. Conclusions and Future Work
6.1. Energy Constrained IIoT Devices in Blockchain
6.2. Independence of RL Techniques and Blockchain
6.3. The Scalability Paradox
6.4. Selection of Appropriate RL Techniques
6.5. Financial Forecasting
6.6. Smart Agents
6.7. Anonymous Data Sharing
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
References
- Mohanta, B.K.; Jena, D.; Satapathy, U.; Patnaik, S. Survey on IoT Security: Challenges and Solution using Machine Learning, Artificial Intelligence and Blockchain Technology. Internet Things 2020, 11, 100227. [Google Scholar] [CrossRef]
- Javaid, U.; Siang, A.K.; Aman, M.N.; Sikdar, B. Mitigating loT Device Based DDoS Attacks Using Blockchain. In Proceedings of the 1st Workshop on Cryptocurrencies and Blockchains for Distributed Systems, Munich, Germany, 15 June 2018; pp. 71–76. [Google Scholar] [CrossRef]
- Wu, J.; Tran, N.K. Application of Blockchain Technology in Sustainable Energy Systems: An Overview. Sustainability 2018, 10, 3067. [Google Scholar] [CrossRef] [Green Version]
- Sun, Y.; Zhang, L.; Feng, G.; Yang, B.; Cao, B.; Imran, M.A. Blockchain-enabled wireless Internet of Things: Performance analysis and optimal communication node deployment. IEEE Internet Things J. 2019, 6, 5791–5802. [Google Scholar] [CrossRef] [Green Version]
- Alladi, T.; Chamola, V.; Parizi, R.M.; Choo, K.R. Blockchain Applications for Industry 4.0 and Industrial IoT: A Review. IEEE Access 2019, 7, 176935–176951. [Google Scholar] [CrossRef]
- Bansal, K.; Mittal, K.; Ahuja, G.; Singh, A.; Gill, S.S. DeepBus: Machine learning based real time pothole detection system for smart transportation using IoT. Internet Technol. Lett. 2020, 3, e156. [Google Scholar] [CrossRef]
- Jameel, F.; Wyne, S.; Jayakody, D.N.K.; Kaddoum, G.; O’Kennedy, R. Wireless Social Networks: A Survey of Recent Advances, Applications and Challenges. IEEE Access 2018, 6, 59589–59617. [Google Scholar] [CrossRef]
- Jameel, F.; Hamid, Z.; Jabeen, F.; Zeadally, S.; Javed, M.A. A survey of device-to-device communications: Research issues and challenges. IEEE Commun. Surv. Tutorials 2018, 20, 2133–2168. [Google Scholar] [CrossRef]
- Mao, D.; Hao, Z.; Wang, F.; Li, H. Innovative Blockchain-Based Approach for Sustainable and Credible Environment in Food Trade: A Case Study in Shandong Province, China. Sustainability 2018, 10. [Google Scholar] [CrossRef] [Green Version]
- Jameel, F.; Javaid, U.; Sikdar, B.; Khan, I.; Mastorakis, G.; Mavromoustakis, C.X. Optimizing Blockchain Networks with Artificial Intelligence: Towards Efficient and Reliable IoT Applications. In Convergence of Artificial Intelligence and the Internet of Things; Springer: Berlin, Germany, 2020; pp. 299–321. [Google Scholar]
- Kaur, P.; Singh, A.; Gill, S.S. RGIM: An Integrated Approach to Improve QoS in AODV, DSR and DSDV Routing Protocols for FANETS Using the Chain Mobility Model. Comput. J. 2020. [Google Scholar] [CrossRef]
- Jameel, F.; Javed, M.A.; Jayakody, D.N.; Hassan, S.A. On secrecy performance of industrial Internet of things. Internet Technol. Lett. 2018, 1, e32. [Google Scholar] [CrossRef]
- Jameel, F.; Khan, W.U.; Shah, S.T.; Ristaniemi, T. Towards intelligent IoT networks: Reinforcement learning for reliable backscatter communications. In Proceedings of the 2019 IEEE Globecom Workshops (GC Wkshps), Waikoloa, HI, USA, 13 December 2019; pp. 1–6. [Google Scholar]
- Alpaydin, E. Introduction to Machine Learning; MIT Press: Cambridge, MA, USA, 2009. [Google Scholar]
- Jameel, F.; Sharma, N.; Khan, M.A.; Khan, I.; Alam, M.M.; Mastorakis, G.; Mavromoustakis, C.X. Machine Learning Techniques for Wireless-Powered Ambient Backscatter Communications: Enabling Intelligent IoT Networks in 6G Era. In Convergence of Artificial Intelligence and the Internet of Things; Springer: Berlin, Germany, 2020; pp. 187–211. [Google Scholar]
- massoud Farahmand, A.; Shademan, A.; Jagersand, M.; Szepesvári, C. Model-based and model-free reinforcement learning for visual servoing. In Proceedings of the 2009 IEEE International Conference on Robotics and Automation, Kobe, Japan, 12 May 2009; pp. 2917–2924. [Google Scholar]
- Zou, X.; Yang, R.; Yin, C.; Nie, Z.; Wang, H. Deploying tactical communication node vehicles with AlphaZero algorithm. IET Commun. 2020, 14, 1392–1396. [Google Scholar] [CrossRef]
- Ernst, D.; Glavic, M.; Capitanescu, F.; Wehenkel, L. Reinforcement learning versus model predictive control: a comparison on a power system problem. IEEE Trans. Syst. Man Cybern. 2008, 39, 517–529. [Google Scholar] [CrossRef] [PubMed]
- Yang, D.; Roth, H.; Xu, Z.; Milletari, F.; Zhang, L.; Xu, D. Searching Learning Strategy with Reinforcement Learning for 3D Medical Image Segmentation. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Shenzhen, China, 17 October 2019; pp. 3–11. [Google Scholar]
- Arulkumaran, K.; Deisenroth, M.P.; Brundage, M.; Bharath, A.A. Deep reinforcement learning: A brief survey. IEEE Signal Process. Mag. 2017, 34, 26–38. [Google Scholar] [CrossRef] [Green Version]
- Brewer, E.A. Towards Robust Distributed Systems (Abstract). In Proceedings of the Nineteenth Annual ACM Symposium on Principles of Distributed Computing, Portland, OR, USA, 16 July 2000. [Google Scholar] [CrossRef]
- Sun, Q.; Li, H.; Ma, Z.; Wang, C.; Campillo, J.; Zhang, Q.; Wallin, F.; Guo, J. A Comprehensive Review of Smart Energy Meters in Intelligent Energy Networks. IEEE IoT J. 2016, 3, 464–479. [Google Scholar] [CrossRef]
- Lei, A.; Cao, Y.; Bao, S.; Li, D.; Asuquo, P.; Cruickshank, H.; Sun, Z. A blockchain based certificate revocation scheme for vehicular communication systems. Future Gener. Comput. Syst. 2019, 110, 892–903. [Google Scholar] [CrossRef]
- Ren, Y.; Leng, Y.; Zhu, F.; Wang, J.; Kim, H.J. Data Storage Mechanism Based on Blockchain with Privacy Protection in Wireless Body Area Network. Sensors 2019, 19, 2395. [Google Scholar] [CrossRef] [Green Version]
- Anderson, R.; Fuloria, S. Who Controls the off Switch? In Proceedings of the 2010 First IEEE Int. Conference on Smart Grid Communication, Gaithersburg, MD, USA, 4–6 October 2010; pp. 96–101. [Google Scholar] [CrossRef]
- Wang, X.; Zha, X.; Ni, W.; Liu, R.P.; Guo, Y.J.; Niu, X.; Zheng, K. Survey on blockchain for Internet of Things. Comput. Commun. 2019, 136, 10–29. [Google Scholar] [CrossRef]
- Duy, P.T.; Hien, D.T.T.; Hien, D.H.; Pham, V.H. A survey on opportunities and challenges of Blockchain technology adoption for revolutionary innovation. In Proceedings of the Ninth International Symposium on Information and Communication Technology, Lahore, Pakistan, 12 November 2018; pp. 200–207. [Google Scholar]
- Liu, J.; Liu, Z. A survey on security verification of blockchain smart contracts. IEEE Access 2019, 7, 77894–77904. [Google Scholar] [CrossRef]
- Joshi, A.P.; Han, M.; Wang, Y. A survey on security and privacy issues of blockchain technology. Math. Found. Comput. 2018, 1, 121–147. [Google Scholar] [CrossRef] [Green Version]
- Yang, R.; Yu, F.R.; Si, P.; Yang, Z.; Zhang, Y. Integrated blockchain and edge computing systems: A survey, some research issues and challenges. IEEE Commun. Surv. Tutorials 2019, 21, 1508–1532. [Google Scholar] [CrossRef]
- Xie, J.; Yu, F.R.; Huang, T.; Xie, R.; Liu, J.; Liu, Y. A Survey on the Scalability of Blockchain Systems. IEEE Network 2019, 33, 166–173. [Google Scholar] [CrossRef]
- Atzei, N.; Bartoletti, M.; Cimoli, T. A survey of attacks on ethereum smart contracts (sok). In Proceedings of the International conference on principles of security and trust, Prague, Czech Republic, 11 April 2017; pp. 164–186. [Google Scholar]
- Rouhani, S.; Deters, R. Security, performance, and applications of smart contracts: A systematic survey. IEEE Access 2019, 7, 50759–50779. [Google Scholar] [CrossRef]
- Li, X.; Jiang, P.; Chen, T.; Luo, X.; Wen, Q. A survey on the security of blockchain systems. Future Gener. Comput. Syst. 2020, 107, 841–853. [Google Scholar] [CrossRef] [Green Version]
- Danzi, P.; Kalor, A.E.; Stefanovic, C.; Popovski, P. Analysis of the communication traffic for blockchain synchronization of IoT devices. In Proceedings of the 2018 IEEE International Conference on Communications (ICC), Kansas City, MO, USA, 24 May 2018; pp. 1–7. [Google Scholar]
- Jabeen, T.; Ali, Z.; Khan, W.U.; Jameel, F.; Khan, I.; Sidhu, G.A.S.; Choi, B.J. Joint Power Allocation and Link Selection for Multi-Carrier Buffer Aided Relay Network. Electronics 2019, 8, 686. [Google Scholar] [CrossRef] [Green Version]
- Sun, Y.; Zhang, L.; Feng, G.; Yang, B.; Cao, B.; Imran, M. Performance Analysis for Blockchain Driven Wireless IoT Systems Based on Tempo-Spatial Model. In Proceedings of the 2019 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC), Guilin, China, 15 June 2019; pp. 348–353. [Google Scholar]
- Aitzhan, N.Z.; Svetinovic, D. Security and Privacy in Decentralized Energy Trading Through Multi-Signatures, Blockchain and Anonymous Messaging Streams. IEEE Trans. Dependable Secur. Comput. 2018, 15, 840–852. [Google Scholar] [CrossRef]
- Jameel, F.; Chang, Z.; Huang, J.; Ristaniemi, T. Internet of Autonomous Vehicles: Architecture, Features, and Socio-Technological Challenges. IEEE Wirel. Commun. 2019, 26, 21–29. [Google Scholar] [CrossRef] [Green Version]
- Jameel, F.; Hamid, Z.; Jabeen, F.; Javed, M.A. Impact of co-channel interference on the performance of VANETs under α-μ fading. AEU Int. J. Electron. Commun. 2018, 83, 263–269. [Google Scholar] [CrossRef]
- Jameel, F.; Duan, R.; Chang, Z.; Liljemark, A.; Ristaniemi, T.; Jantti, R. Applications of Backscatter Communications for Healthcare Networks. IEEE Netw. 2019, 33, 50–57. [Google Scholar] [CrossRef] [Green Version]
- Rifi, N.; Agoulmine, N.; Chendeb Taher, N.; Rachkidi, E. Blockchain technology: Is it a good candidate for securing iot sensitive medical data? Wirel. Commun. Mob. Comput. 2018, 2018, 9763937. [Google Scholar] [CrossRef] [Green Version]
- Zhang, R.; Xue, R.; Liu, L. Security and Privacy on Blockchain. ACM Comput. Surv. 2019, 52, 51:1–51:34. [Google Scholar] [CrossRef] [Green Version]
- Burhan, M.; Rehman, R.; Khan, B.; Kim, B.S. IoT elements, layered architectures and security issues: A comprehensive survey. Sensors 2018, 18, 2796. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Javaid, U.; Aman, M.N.; Sikdar, B. BlockPro: Blockchain Based Data Provenance and Integrity for Secure IoT Environments. In Proceedings of the 1st Workshop on Blockchain-enabled Networked Sensor Systems, Shenzhen, China, 10 November 2018; pp. 13–18. [Google Scholar] [CrossRef]
- Javaid, U.; Aman, M.N.; Sikdar, B. DrivMan: Driving Trust Management and Data Sharing in VANETs with Blockchain and Smart Contracts. In Proceedings of the 2019 IEEE 89th Vehicular Technology Conference (VTC2019-Spring), Kuala Lumpur, Malaysia, 1 May 2019; pp. 1–5. [Google Scholar]
- Jiang, L.; Xie, S.; Maharjan, S.; Zhang, Y. Blockchain Empowered Wireless Power Transfer for Green and Secure Internet of Things. IEEE Netw. 2019, 33, 164–171. [Google Scholar] [CrossRef]
- Watanabe, H.; Fan, H. A Novel Chip-Level Blockchain Security Solution for the Internet of Things Networks. Technologies 2019, 7. [Google Scholar] [CrossRef] [Green Version]
- Chaintoutis, C.; Akriotou, M.; Mesaritakis, C.; Komnios, I.; Karamitros, D.; Fragkos, A.; Syvridis, D. Optical PUFs as physical root of trust for blockchain-driven applications. IET Softw. 2019, 13, 182–186. [Google Scholar] [CrossRef]
- Gill, S.S.; Tuli, S.; Xu, M.; Singh, I.; Singh, K.V.; Lindsay, D.; Tuli, S.; Smirnova, D.; Singh, M.; Jain, U.; et al. Transformative effects of IoT, Blockchain and Artificial Intelligence on cloud computing: Evolution, vision, trends and open challenges. Internet Things 2019, 8, 100118. [Google Scholar] [CrossRef] [Green Version]
- Rawat, D.B. Fusion of Software Defined Networking, Edge Computing, and Blockchain Technology for Wireless Network Virtualization. IEEE Commun. Mag. 2019, 57, 50–55. [Google Scholar] [CrossRef]
- Ioini, N.E.; Pahl, C. A Review of Distributed Ledger Technologies. On the Move to Meaningful Internet Systems. In Proceedings of the OTM 2018 Conferences—Confederated International Conferences: CoopIS, C&TC, and ODBASE 2018, Valletta, Malta, 22–26 October 2018; pp. 277–288. [Google Scholar] [CrossRef]
- Hu, Y.; Manzoor, A.; Ekparinya, P.; Liyanage, M.; Thilakarathna, K.; Jourjon, G.; Seneviratne, A. A Delay-Tolerant Payment Scheme Based on the Ethereum Blockchain. IEEE Access 2019, 7, 33159–33172. [Google Scholar] [CrossRef]
- Li, A.; Wei, X.; He, Z. Robust Proof of Stake: A New Consensus Protocol for Sustainable Blockchain Systems. Sustainability 2020, 12, 2824. [Google Scholar] [CrossRef] [Green Version]
- Borkowski, M.; Sigwart, M.; Frauenthaler, P.; Hukkinen, T.; Schulte, S. Dextt: Deterministic Cross-Blockchain Token Transfers. IEEE Access 2019, 7, 111030–111042. [Google Scholar] [CrossRef]
- Kumar, N.M.; Mallick, P.K. Blockchain technology for security issues and challenges in IoT. Procedia Comput. Sci. 2018, 132, 1815–1823. [Google Scholar] [CrossRef]
- Li, J.; Ni, X.; Yuan, Y. The Reserve Price of Ad Impressions in Multi-Channel Real-Time Bidding Markets. IEEE Trans. Comput. Soc. Syst. 2018, 5, 583–592. [Google Scholar] [CrossRef]
- Qian, Y.; Jiang, Y.; Chen, J.; Zhang, Y.; Song, J.; Zhou, M.; Pustišek, M. Towards decentralized IoT security enhancement: A blockchain approach. Comput. Electr. Eng. 2018, 72, 266–273. [Google Scholar] [CrossRef]
- Griggs, K.N.; Ossipova, O.; Kohlios, C.P.; Baccarini, A.N.; Howson, E.A.; Hayajneh, T. Healthcare blockchain system using smart contracts for secure automated remote patient monitoring. J. Med. Syst. 2018, 42, 130. [Google Scholar] [CrossRef] [PubMed]
- Khan, M.A.; Salah, K. IoT security: Review, blockchain solutions, and open challenges. Future Gener. Comput. Syst. 2018, 82, 395–411. [Google Scholar] [CrossRef]
- Park, L.W.; Lee, S.; Chang, H. A sustainable home energy prosumer-chain methodology with energy tags over the blockchain. Sustainability 2018, 10, 658. [Google Scholar] [CrossRef] [Green Version]
- Liu, M.; Teng, Y.; Yu, F.R.; Leung, V.C.; Song, M. Deep Reinforcement Learning Based Performance Optimization in Blockchain-Enabled Internet of Vehicle. In Proceedings of the ICC 2019-2019 IEEE International Conference on Communications (ICC), Shanghai, China, 24 May 2019; pp. 1–6. [Google Scholar]
- Herrera-Joancomartí, J.; Pérez-Solà, C. Privacy in bitcoin transactions: new challenges from blockchain scalability solutions. In Proceedings of the International Conference on Modeling Decisions for Artificial Intelligence, Sant Julia de Loria, Andorra, 19 September 2016; pp. 26–44. [Google Scholar]
- Thakkar, P.; Nathan, S.; Viswanathan, B. Performance benchmarking and optimizing hyperledger fabric blockchain platform. In Proceedings of the 2018 IEEE 26th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS), Milwaukee, WI, USA, 28 Septemebr 2018; pp. 264–276. [Google Scholar]
- Yli-Huumo, J.; Ko, D.; Choi, S.; Park, S.; Smolander, K. Where is current research on blockchain technology? A systematiC review. PLoS ONE 2016, 11, e0163477. [Google Scholar] [CrossRef] [PubMed]
- Luong, N.C.; Anh, T.T.; Binh, H.T.T.; Niyato, D.; Kim, D.I.; Liang, Y.C. Joint transaction transmission and channel selection in cognitive radio based blockchain networks: A deep reinforcement learning approach. In Proceedings of the ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK, 17 May 2019; pp. 8409–8413. [Google Scholar]
- Xu, H.; Zhang, L.; Liu, Y.; Cao, B. Raft based wireless blockchain networks in the presence of malicious jamming. IEEE Wirel. Commun. Lett. 2020, 9, 817–821. [Google Scholar] [CrossRef] [Green Version]
- Jameel, F.; Wyne, S.; Kaddoum, G.; Duong, T.Q. A comprehensive survey on cooperative relaying and jamming strategies for physical layer security. IEEE Commun. Surv. Tutorials 2018, 21, 2734–2771. [Google Scholar] [CrossRef] [Green Version]
- Liu, M.; Yu, F.R.; Teng, Y.; Leung, V.C.; Song, M. Performance optimization for blockchain-enabled industrial internet of things (IIoT) systems: A deep reinforcement learning approach. IEEE Trans. Ind. Informatics 2019, 15, 3559–3570. [Google Scholar] [CrossRef]
- Danzi, P.; Kalør, A.E.; Stefanović, Č.; Popovski, P. Delay and communication tradeoffs for blockchain systems with lightweight IoT clients. IEEE Internet Things J. 2019, 6, 2354–2365. [Google Scholar] [CrossRef] [Green Version]
- Alrubei, S.; Ball, E.; Rigelsford, J.; Willis, C. Latency and Performance Analyses of Real-World Wireless IoT-Blockchain Application. IEEE Sensors J. 2020, 13, 7372–7383. [Google Scholar] [CrossRef]
- Novo, O. Blockchain meets IoT: An architecture for scalable access management in IoT. IEEE Internet Things J. 2018, 5, 1184–1195. [Google Scholar] [CrossRef]
Attributes | Blockchain | Tangle | Hashgraph | Sidechain |
---|---|---|---|---|
Data Structure | Linked list | Directed Acyclic Graph (DAG) | DAG | Linked lists |
Topologies | Public | Private | Private | Semi-public |
Consensus | Proof-of- Work-SHA256 Proof-of- Stake | Proof-of- Work-Hashcash | Virtual Voting | Proof-of- Work-Ethash |
Transaction Structure | Blocks | Sites | Gossip Event-based | Consortium Chain |
Transaction Fee | Yes | No | No | Yes |
Transactions Per second | 4–12 | 500–800 | >200,000 | 3000–20,000 |
Verification Time | Order(minutes) | Order(seconds) | Order(seconds) | Order(minutes) |
Privacy | Moderate | Low | Low | Moderate |
Security | Very High | High | High | High |
Maturity | Many Implementations | Experimental Phase | Experimental Phase | Experimental Phase |
Platforms | Bitcoin, Ethereum | IOTA | Hyperledger | Ethereum, Monax, Elements |
Copyright | Open Source | Open Source | Patented | Open Source |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Jameel, F.; Javaid, U.; Khan, W.U.; Aman, M.N.; Pervaiz, H.; Jäntti, R. Reinforcement Learning in Blockchain-Enabled IIoT Networks: A Survey of Recent Advances and Open Challenges. Sustainability 2020, 12, 5161. https://doi.org/10.3390/su12125161
Jameel F, Javaid U, Khan WU, Aman MN, Pervaiz H, Jäntti R. Reinforcement Learning in Blockchain-Enabled IIoT Networks: A Survey of Recent Advances and Open Challenges. Sustainability. 2020; 12(12):5161. https://doi.org/10.3390/su12125161
Chicago/Turabian StyleJameel, Furqan, Uzair Javaid, Wali Ullah Khan, Muhammad Naveed Aman, Haris Pervaiz, and Riku Jäntti. 2020. "Reinforcement Learning in Blockchain-Enabled IIoT Networks: A Survey of Recent Advances and Open Challenges" Sustainability 12, no. 12: 5161. https://doi.org/10.3390/su12125161
APA StyleJameel, F., Javaid, U., Khan, W. U., Aman, M. N., Pervaiz, H., & Jäntti, R. (2020). Reinforcement Learning in Blockchain-Enabled IIoT Networks: A Survey of Recent Advances and Open Challenges. Sustainability, 12(12), 5161. https://doi.org/10.3390/su12125161