Recent Advances in Deep Reinforcement Learning Applications for Solving Partially Observable Markov Decision Processes (POMDP) Problems Part 2—Applications in Transportation, Industries, Communications and Networking and More Topics

: The two-part series of papers provides a survey on recent advances in Deep Reinforcement Learning (DRL) for solving partially observable Markov decision processes (POMDP) problems. Reinforcement Learning (RL) is an approach to simulate the human’s natural learning process, whose key is to let the agent learn by interacting with the stochastic environment. The fact that the agent has limited access to the information of the environment enables AI to be applied efﬁciently in most ﬁelds that require self-learning. It’s essential to have an organized investigation—we can make good comparisons and choose the best structures or algorithms when applying DRL in various applications. The ﬁrst part of the overview introduces Markov Decision Processes (MDP) problems and Reinforcement Learning and applications of DRL for solving POMDP problems in games, robotics, and natural language processing. In part two, we continue to introduce applications in transportation, industries, communications and networking, etc. and discuss the limitations of DRL.


Transportation
An intelligent transportation system (ITS) [1] is an application that aims to provide safe, efficient, and innovative services to transport and traffic management and construct more intelligent transport networks. The technologies include car navigation, traffic signal control systems, container management systems, variable message signs, and more. Effective technologies like sensors, Bluetooth, radar, etc., have been applied in ITS and have been widely discussed. In recent years, with DRL steps into vision, the application of DRL in ITS has been researched. Haydari and Yilmaz [2] presented a comprehensive survey on DRL for ITS.

Traffic Signal Control (TSC)
An application of ML in transportation is the scheduling of traffic signals in multiintersection vehicular networks. Automatic signal control makes traffic flow more efficient and reasonable and alleviates traffic congestion.
Arel et al. [3] explained the configuration of RL where the states are based on traffic statistics, with each element being the traffic flow at each lane. The agent selects an action based on vehicle positions, according to its policy. Haydari and Yilmaz [2] reviewed two popular types of state representations in intersections. The first format is an image-like representation called Discrete Traffic State Encoding (DTSE). It acquires high resolution and practical information from the intersection. Four characteristics, including speed and position of vehicles, signal phases, and accelerations, are selected in different research, shown in separate arrays in DTSE. The second approach is forming a feature-based vector. The average or total value of information for each lane is represented on a vector instead of using vehicle-based state representation. The features include queue length, cumulative waiting time, the average speed on a lane, phase duration, and the number of vehicles in each lane. The two representations are shown in Figure 1. State and reward settings are also discussed in [2]. Li [4] discussed works regarding adaptive control for signal control: MARLIN-ATSC proposed by El-Tantawy et al. [5]; Van der Pol and Oliehoek [6] combined Deep Q Networks (DQN) [7] and coordination algorithm; Mannion et al. [8] provided an experimental review of DRL for adaptive traffic signal control. For more surveys, see [2,[9][10][11][12][13].
Genders and Razavi [14] proposed the discrete traffic state encoding, which is informationdense, as the input to the DQN networks for traffic signal control agent (DQTSCA) and evaluated state representations from low to high-resolution using Asynchronous Advantage Actor Critic (A3C) in [15]. Garg et al. [16] built a traffic simulator on a 3D virtual reality software, Unity3d, taking collision count, speed of vehicles across the intersections, dynamic generation, etc. into consideration, and created a simulation environment closely based on the real-world traffic specifications. Rodrigues and Azevedo [17] developed an open-source callback-based framework (CAREL) integrated with AIMSUN, for testing as a benchmark. Wei et al. [18] proposed a decentralized RL method for multi-intersection traffic signal control on arterial traffic, with each intersection with an individual control agent. Wang et al. [19] introduced a Double Dueling Deep Q Network (3DQN) with highresolution event-based data, which is collected directly from vehicle-actuated detectors.
Recently, Ma and Wu [20] proposed Feudal Multi-agent Advantage Actor-Critic (FMA2C), an extension of MA2C [21] with feudal hierarchy, with each split region controlled by an agent. Wu et al. [22] presented multi-agent recurrent deep deterministic policy gradient (MARDDPG), based on Deep Deterministic Policy Gradient (DDPG) [23]. Xu et al. [24] used a data-driven approach to find critical nodes, which can cause a reduction in traffic efficiency. They then introduced a policy gradient method on these nodes. This method can effectively lower the average delay and travel time.
In 2020, Haydari and Yilmaz [2] provided tables of outlines of single and multiple agent RL approaches for Traffic Signal Control (TSC), DRL methods for TSC, and DRL solutions for other ITS applications.

Autonomous Driving
Autonomous driving is an essential topic of ITS. TORCS is often used as the autodriving simulator for algorithms such as DDPG [23], Deep Deterministic Actor Critic Algorithm (DDAC) [25], Fine Grained Action Repetition (FiGAR) [26], Normalized Actor-Critic (NAC) [27], etc., as mentioned in [28]. Due to safety concerns, road test for algorithms on auto-vehicles hasn't been widely applied. But researchers built many open-source simulators, and most methods were evaluated in simulation. Kang et al. [29] provided an overview of driving datasets and virtual testing environments. There are several tasks involved in autonomous driving, including motion planning, overtaking, merging, lane change, auto-parking, etc., see [30,31] for surveys of DRL algorithms in auto driving.

Sim-to-Real
Osiński et al. [32] presented simulation-Based RL for real-world autonomous driving. They used RL in simulation to make the driving system control the real-world vehicle and achieved sim-to-real policy transfer. Navigation Navigation is a fundamental task in autonomous driving, and DRL has been proven to be effective in navigation problems: Fayjie et al. [33] presented a DQN-based approach for navigation in the urban environment, and Isele et al. [34] used a DQN-based method for navigating in occluded intersections.
Pusse and Klusch [35] introduced a hybrid solution, HyLEAP, for pedestrian collisionfree navigation. It combines selected POMDP planning methods and DRL prior to other individual methods regarding German In-Depth Accident Study (GIDAS) pedestrian safety.

Lane Change
Sharifzadeh et al. [36] proposed an inverse reinforcement learning (IRL) approach with DQN to extract the rewards for collision-free lane changing. The agent can perform human-like lane changing behavior; Hoel et al. [37] used a Double DQN agent for speed change and lane change, and overtaking cases, which outperforms the combination of the Intelligent Driver Model (IDM) and Minimizing Overall Braking Induced by Lane changes (MOBIL) model in highway driving; Shi et al. [38] applied Hierarchical DQN: Firstly, DQN is used to decide when to perform the lane change. Secondly, a Q-function in a quadratic form is designed for car-following and the gap in the target lane. Lastly, the execution step is to perform movement. Wang et al. [39] later presented a rule-based DQN, which outperforms individual DQN or rule-based methods. Regarding the driver assistant systems, Min et al. [40] proposed a supervisor agent using Quantile Regression Deep Q Network (QR-DQN) for lane changing and other control. Ye et al. [41] adopted DDPG for the training and high-fidelity virtual simulation environment VISSIM, getting better results than IDM and Constant Time Headway (CTH).

Decision Making (and Optimum Control)
Making safe and effective decisions in complex traffic environments is crucial in autodriving. The method must be general to handle the changing situations. The problems include when to change lanes, or whether or not to stop at an intersection, etc.
Due to the varying environments, the uncertainty should be considered when applying autonomous driving to the real world. Modeling the problem as a POMDP will be necessary. The requirement for storing observations can cause inefficiency. Qiao et al. [42] introduced Hierarchical Options MDP (HOMDP), which learns discrete options in the high-level process and low-level continuous actions simultaneously.
Combining the concepts of planning and learning, Hoel et al. [43] introduced Monte Carlo tree search (MCTS) and DRL framework for tactical decision making, based on the AlphaGo Zero algorithm with a continuous state space. The agents outperform baselines.
Autonomous driving is a typical multi-agent setting, Yu et al. [44] employed coordinationgraph-based multi-agent RL (MARL) approaches to achieve coordinated maneuvers for multiple vehicles. The method can achieve a high level of safety by properly coordinating vehicles' overtaking maneuvers.
With the concept of Connected and Automated Vehicles (CAVs), a vehicle's behaviors are based on shared information. Zhou et al. [45] proposed a DDPG-based car-following model and trained CAVs to obtain appropriate behaviors to improve travel efficiency, fuel consumption, and safety at signalized intersections in real-time.

Path Planning
Makantasis et al. [46] considered path planning for an autonomous vehicle that moves on a freeway. The experiments show that the DDQN-derived driving policy can achieve better performance comparing to DP (Dynamic Programming) or SUMO policies; Qian et al. [47] proposed a planning features-based deep behavior decision method (PFBD), trained with Twin Delayed DDPG (TD3), to select an optimal maneuver.

Pedestrian Detection
For pedestrian detection, Chae et al. [48] proposed an autonomous braking system using DQN.

Other Applications in ITS Ramp Metering
Belletti et al. [49] presented Multi-Task DRL for control of systems modeled by discretized non-linear Partial Differential Equations (PDEs) and achieved expert-level control of Ramp Metering. Chalaki et al. [50] developed a zero-shot transfer of a policy from simulation to the University of Delaware's Scaled Smart City (UDSSC) testbed. The adversarial multi-agent policy improves system efficiency even under stochastic. Based on this, Jang et al. [51] trained two policies, and the noised policy significantly outperformed the noise-free version.

Energy Management
Qi et al. [52] designed a DQN-based PHEV (plug-in hybrid electric vehicles) energy management system to autonomously splits fuel/electricity from interactions between the car and the environment, making the model capable of achieving energy savings.

Industrial Applications
Industry 4.0, which denotes The Fourth Industrial Revolution, uses modern innovative technology to automate traditional manufacturing and industrial practices. Artificial intelligence enables many applications in Industry 4.0, including predictive maintenance, diagnostics, and management of manufacturing activities and processes [4].
Robotics, including manipulation, locomotion, etc., will prevail in all aspects of industrial applications, which was mentioned in [28]. For example, Schoettler et al. [53] discussed insertion tasks, particularly in industrial applications; Li et al. [54] also discussed a skill-acquisition DRL method to make robots acquire assembly skills.

Inspection and Maintenance
Health Indicator Learning (HIL) is an aspect of maintenance that learns the health conditions of equipment over time. Zhang et al. [55] proposed a data-driven approach for solving HIL problem based on model-based and model-free RL methods; Holmgren [56] presented a general-purpose maintenance planner based on Monte-Carlo tree search (MCTS); Ong et al. [57] proposed a model-free DRL algorithm, Prioritized Double Deep Q-Learning with Parameter Noise (PDDQN-PN) for predictive equipment maintenance from an equipment-based sensor network context, which can rapidly learn an optimal maintenance policy; Huang et al. [58] proposed a DDQN-based algorithm to learn the predictive maintenance policy.

Management of Engineering Systems
Decision-making for engineering systems can be formulated as an MDP or a POMDP problem [59]. Andriotis and Papakonstantinou [60] developed Deep Centralized Multiagent Actor-Critic (DCMAC), which provides solutions for the sequential decision-making in multi-state, multi-component, partially, or fully observable stochastic engineering environments. Most studies on industrial energy management are working on modeling complex industrial processes. Huang et al. [61] developed a model-free demand response (DR) scheme for industrial facilities, with an actor-critic-based DRL algorithm to determine the optimal energy management policy.

Process Control
Automatic process control in engineering systems is to achieve a production level of consistency, economy, and safety. In contrast to the traditional design process, RL can learn appropriate closed-loop controllers by interacting with the process and incrementally improving control behavior.
Spielberg et al. [62] proposed a DRL method for process control with the controller interacting with a process through control actions. Deep neural networks serve as function approximators to learn the control policies. In 2019, Spielberg et al. [63] also developed an adaptive model-free DRL controller for set-point tracking problems in nonlinear processes, evaluated on Single-Input-Single-Output (SISO), Multi-Input-Multi-Output (MIMO), and a nonlinear system. The results show that it can be utilized as an alternative to traditional model-based controllers.

Smart Grid
Smart grids are the development trend of power systems. They've been researched for years. The rise of artificial intelligence enables more complex techniques in smart grids and their future development. Zhang et al. [64] provided a review on the research and practice on DRL in smart grids, including anomaly detection, prediction, decision-making support for control, etc.
Rocchetta et al. [65] developed a DQN-based method for the optimal management of the operation and maintenance of power grids, which can exploit the information gathered from Prognostic Health Management devices, thus selecting optimal Operation and Maintenance (O&M) actions.
State estimation is critical in monitoring and managing the operation of a smart grid. An et al. [66] proposed a DQN detection (DQND) scheme to defend against data integrity attacks in AC power systems, which applies the main network and a target network to learn the detection strategy.
Wei et al. [67] proposed a recovery strategy to reclose the tripped transmission lines at the optimal time. The DDPG-based method is applied to adapt to uncertain cyber-attack scenarios and to make decisions in real-time, shown in Figure 2. The action in the cycle is to to reclose the tripped lines at a proper time. The reward is the transient energy including potential energy and kinetic energy.
Mocanu et al. [68] utilized DRL in the smart grid to perform online optimization of schedules for electricity consuming devices in buildings and explored DQN and Deterministic Policy Gradient (DPG), both performing well for the minimization of the energy cost.

Communications and Networking
Modern networks, including the Internet of Things (IoT) and unmanned aerial vehicle (UAV) networks, need to make the decisions to maximize the performance under uncertainty. DRL has been applied to enable network entities to obtain optimal policies and deal with large and complex networks. Jang et al. [51] provided a survey on applications of DRL in communications and networking for traffic routing, resource sharing, and data collection. By integrating AI and blockchain, Dai et al. [69] proposed a secure and intelligent architecture for next-generation wireless networks to enable flexible and secure resource sharing and developed a caching scheme based on DRL. Also, Yang et al. [70] presented a brief review of ML applications in intelligent wireless networks.

Internet of Things (IoT)
The Internet of Things (IoT) connects a great amount of devices to the Internet, where the devices collect and share sensory data to reflect the status of the physical world. Autonomous IoT (AIoT) integrates IoT, ML, and autonomous control. AI is a promising method to achieve autonomy, for decision making. Lei et al. [71] proposed a general 3-layer model for the applications of RL/DRL in AIoT. For each layer, the state is the system state and the reward is the performance of the system. The action of the loop is the control to the layer systems, as shown in Figure 3.

Industrial Internet of Things (IIoT)
Blockchain is a promising solution for data storing/processing/sharing securely and efficiently in the industrial Internet of things (IIoT). Blockchain-enabled IIoT systems can utilize DRL techniques to improve the performance [72,73].

Mobile Edge Computing (MEC)
Mobile Edge Computing (MEC) is a promising technology to extend the services to the edge of the IoT system, and DRL has been successfully applied in the MEC networks in recent years [74][75][76]. Zhu et al. [77] discussed DRL in caching transient data. Chen et al. [78] proposed intelligent resource allocation framework (iRAF) to solve the resource allocation problem for collaborative mobile edge computing (CoMEC). The technology of fog computing is a promising paradigm for IoT to provide proximity services. There are research focusing on utilizing DRL in fog-enabled IoT [79][80][81].

Others
Zhu et al. [82] proposed a DQN-based transmission scheduling mechanism for the cognitive radio-based IoT (CIoT) to find the optimal strategy to transmit packets of different buffers among multiple channels to maximize the system utility. Ferdowsi and Saad [83] proposed a DRL-based watermarking algorithm for dynamic authentication of IoT signals to detect cyber-attacks. Jay et al. [84] presented Aurora, a congestion control protocol powered by DRL.

Connected Vehicles
Vehicle networks share information and entertainment to drivers and vehicles by connecting services, content, and application providers through wireless networks, including safety warnings, managing, playing audio, navigation, delivering entertainment, social networking, making phone calls, autonomous driving, etc. This section is similar to what's discussed in Section 1.1 but will focus more on communications between vehicles.
He et al. [85] proposed an integrated framework that enables dynamic orchestration of networking, caching, and computing, to improve the performance of next-generation vehicular networks. Doddalinganavar et al. [86] provided a survey on DRL protocols in Vehicular Adhoc Networks (VANETS).

Computing and Caching
DRL has also been utilized successfully in computing and caching in-vehicle networks. Tan and Hu [87] developed a DRL-based multi-timescale framework, with mobility-aware reward estimation. Ning et al. [88] and Liu et al. [89] provided DRL insights for vehicular edge computing and constructing an intelligent offloading system. Ning et al. [90] developed an intent-based traffic control system based on DRL for the 5G-envisioned Internet of Vehicles (IoCVs).

Resource Allocation
In Vehicle-to-Vehicle (V2V) networks, device-to-device (D2D) communications provide direct local message dissemination with low latency and energy consumption. Effec-tive resource allocation mechanisms are necessary to manage the interference between the D2D links and the cellular links. Ye et al. [91] presented a decentralized resource allocation mechanism for V2V communications based on DRL that can be employed in both unicast and broadcast scenarios.

Traffic Scheduling
Chinchali et al. [92] presented a DRL-based scheduler that can adapt to highly dynamic traffic and various reward functions set by network operators to optimally schedule traffic, named HVFT-RL (High Volume Flexible Time).

Others
The vehicle-to-infrastructure (V2I) communication via millimeter-wave (mmWave) base stations is crucial for the operation of 5G ITS, which offers high capacity channel resources toward connected vehicles. There exists the cell association and resource allocation problem called CARA. Kwon and Kim [93] proposed the 3-tier heterogeneous vehicular network (HetVNet) using a multi-agent deep deterministic policy gradient (MADDPG) approach to solve CARA problems.
With the vehicle sensors, it's still tricky to detect objects occluded by other moving/static obstacles. [94] presented a cooperative perception scheme with DRL to select data to transmit, thus enhancing the detection accuracy. The Cooperative & Intelligent Vehicle Simulation (CIVS) Platform was developed to evaluate the scheme, and the results show significant improvement accuracy compared to the baseline.

Resources Management
Resources management problems are vital in all applications, as discussed in previous sections. In systems and networking, they often appear as online decision-making tasks where solutions depend on understanding the workload and environment. In these situations, DRL is helpful to deal with resource management problems. Mao et al. [95] presented DeepRM in 2016, which translates the problem with multiple resource demands into a learning problem. Li et al. [96] considered resources management in network slicing, and Zhang et al. [97] introduced intelligent cloud resource management with DRL.

More Topics
There are many applications based on DRL in various domains. In this section, the applications in healthcare, education, finance and aerospace will be briefly discussed.

Healthcare
RL is a suitable method to develop robust solutions for many healthcare domains, where a long and sequential procedure usually characterizes diagnosing decisions or treatment regimes.
Esteva et al. [98] presented a guide to DL in healthcare, in which DRL is discussed in the context of robotic-assisted surgery. Liu et al. [99] presented a survey of DRL methods on clinical decision support.
Yu et al. [100] discussed the broad applications of RL in healthcare, including dynamic treatment regimes in chronic diseases and critical care, automated medical diagnosis, and other control or scheduling domains in many aspects of a healthcare system. As shown in Figure 4, RL is mainly used in dynamic treatment regimes (DTRs) [101], automated medical diagnosis and others.

Education
Reddy et al. [102] mentioned that guiding students with a sequence of lessons and helping them retain knowledge is one of the central challenges in education. They studied a DRL method to learn flexible and scalable teaching policies that select the following item to review.
Zheng et al. [103] proposed a DQN-based framework for online personalized news recommendation.

Finance
DRL can provide a long-term strategy to maximize cumulative reward, making it popular in trading, stocking, and marketing. Like AlphaZero and other games, there are similar rules in trading. The most widely used mechanism in financial markets is the "continuous double auction order book with time priority", based on which, Ritter [104] discussed RL algorithms mathematically. See [105] for a survey on RL in economics and finance.
Financial trading is crucial to investment companies, and DRL has developed as a good option to generalize trading strategies or to analyze [106,107]. Wang et al. [108] proposed AlphaStock for better quantitative trading (QT) strategies.
Financial portfolio management is the procedure of constant redistribution of a fund into different financial products. DRL has been proved to be effective for financial portfolio management [109,110].

Aerospace
Increasingly complex space missions have encouraged the development of autonomous command and control approaches for handling high-dimensional, continuous observations and action spaces with hard-to-analyze behavior. DRL techniques have been researched for providing safety and performance in aerospace. Harris and Schaub [111] and Harris et al. [112] have examined DRL methods in spacecraft command and control and decision making.

Deep Reinforcement Learning Limitations
DRL is the combination of Deep Learning and Reinforcement Learning, and it's more robust than Deep Learning or Reinforcement Learning. However, it inherits some drawbacks that DP and RL have.
Deep Learning extracts features and tasks from data. Generally, the more data provided in training, the better performance DL has. Deep Learning requires lots of data and high-performance GPUs to achieve specific functions. Due to the complex data models, it's costly to train the models. There's no standard rule for selecting DL tools or architectures, and tuning the hyperparameters could also be time-consuming. This makes DL unpractical in many domains.
Reinforcement Learning imitates the learning process of humans. It is trained by making and then avoiding mistakes. It can solve some problems that conventional methods can't solve. In some tasks, it also has the ability to surpass humans. However, RL also has some limitations. First of all, too much reinforcement might cause an overload of states, diminishing the results. Secondly, RL assumes the environment is a Markovian model, in which the probability of the event depends only on the previous state. Thirdly, it has the disadvantages of the curse of dimensionality and the curse of real-world samples. What's more, we have mentioned the challenges of setting up rewards, balancing exploration and exploitation, etc. [28]. Reinforcement Learning is an expensive and complex method, so it's not preferable for simple tasks.
Employing DRL in the real world is complex. Dulac-Arnold et al. [116] addressed nine significant challenges of practical RL in the real world. They presented examples for each challenge and provided some references for deploying RL: 1. Modeling the real world is complex. Many systems cannot be directly trained on.
An off-line off-policy approach [116] could be deployed to replace a previous control system. Logs from the policy are available, and the policy is trained with batches of data obtained from the control algorithm. 2. Practical systems do not have separate training and evaluation environments. The agent must explore and act reasonably and safely. Thus, a sample-efficient and performant algorithm is crucial. Finn et al. [117] proposed Model Agnostic Meta-Learning (MAML) to learn within a distribution with few shot learning. Osband et al. [118] used Bootstrapped DQN to learn an ensemble of Q-networks and Thompson Sampling to achieve deep efficient exploration. Using expert demonstrations to bootstrap the agent can also improve efficiency, which has been combined with DQN [7] and DDPG [23]. 3. Real-world environments usually have massive and continuous state and action spaces. Dulac-Arnold et al. [119] addressed the challenge for sizeable discrete action spaces. Action-Elimination Deep Q-Network (AE-DQN) [120] and Deep Reinforcement Relevance Network (DRRN) [121] also deals with the issue. 4. The learned policy might violate the safety constraints. Constrained MDP (CMDP) [116] and budgeted MDP [122] take the constraint components into consideration during training. 5. Considering POMDP problems, Dulac-Arnold et al. [116] presented Robust MDPs, where the learned policy maximizes the worst-case value function. 6. Formulating multi-dimensional reward functions is usually necessary and complicated. Distributional DQN Bellemare et al. [123] models the percentile distribution of the rewards. Dulac-Arnold et al. [116] presented multi-objective analysis and formulated the global reward function as a linear combination of sub-rewards. Abbeel and Ng [124] gave an algorithm is based on inverse RL to try to recover the unknown reward function. 7. Policy explainability is vital for real-world policies as humans operate the systems. 8. Policy inference should be made in real-time at the control frequency of the system.
Hester et al. [125] presented a parallel real-time architecture for model-based RL. AlphaGo [126] improves with more rollouts rather than running at a specific frequency. 9. Most natural systems have delays in the perception of the states, the actuators, or the return. Hung et al. [127] proposed a memory-based algorithm where agents use recall of memories to credit actions from the past. Arjona-Medina et al. [128] introduced RUDDER (Return Decomposition for Delayed Rewards) to learn long-term credit assignments for delayed rewards.

Summary
This is the second part of the two-part series of survey papers. In the first part [28], the fundamental concepts and some applications have been discussed. In this part, we continue presented applications in more domains, where DRL hasn't been as widely employed as in Gaming and Robotics.
In an intelligent transportation system, we discussed Traffic Signal Control (TSC), auto-driving, and more control applications. However, testing the policies in the real world is a huge barrier. In industries, communications and networking applications, DRL is proven to be an alternative to conventional methods. Note that there are some inevitable overlapping among those applications. For example, applying DRL in the game is also part of the application in transportation; robotics can also be in industrial or healthcare applications.
As we witnessed, many research groups pushed forward the fast development of DRL, including OpenAI, DeepMind, AI research office in Alberta, and research center led by Rich Sutton, and more. Many new algorithms are being developed and discussed. This overview provides an organized investigation to help us get more familiar with the usability of different methods and architectures. And it's always inspiring to optimize the strategies and to overcome the limitations.
Funding: This research received no external funding.

Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.