Smart cities are built on the foundation of information and communication technologies with the sole purpose of connecting citizens and technology for the overall improvement of the quality of lives. While quality of life includes ease of mobility and access to quality healthcare, amongst others, effective management and sustainability of resources, economic development and growth complement the fundamental requirements of smart cities. These goals are achieved by proper management and processing of the data acquired from dedicated or non-dedicated sensor networks. In most cases, the information gathered is refined continuously to produce other information for efficiency and effectiveness within the smart city.
In the Internet of Things (IoT) era, the interplay between mobile networks, wireless communications and artificial intelligence is transforming the way that humans live and survive via various forms of improvements in technological advancements, more specifically improved computing power, high performance processing and huge memory capacities. With the advent of cyber-physical systems, which comprise the seamless integration of physical systems with computing and communication resources, a paradigm shift from the conventional city concept towards a smart city design has been coined as the term smart city. Basically, a smart city is envisioned to be ICT driven and capable of offering various services such as smart driving, smart homes, smart living, smart governance and smart health, just to mention a few [1
]. ICT-driven management and control, as well as the overwhelming use of sensors in smart devices for the good and well-being of citizens yields the desired level of intelligence to these services. Besides continuously informing the citizens, being liveable and ensuring the well-being of the citizens are reported among the requirements of smart cities [2
]. Therefore, the smart city concept needs transformation of health services through sensor and IoT-enablement of medical devices, communications and decision support. This ensures availability, ubiquity and personal customization of services, as well as ease of access to these services. As stated in [3
], the criteria for smartness of an environment are various. As a corollary to this statement, the authors investigate a smart city in the following dimensions: (1) the technology angle (i.e., digital, intelligent, virtual, ubiquitous and information city), (2) the people angle (creative city, humane city, learning city and knowledge city) and (3) the community angle (smart community).
Furthermore, in the same vein, Anthopoulus (see [4
]) divides the smart city into the following eight components: (1) smart infrastructures where facilities utilize sensors and chips; (2) smart transportation where vehicular networks along with the communication infrastructure are deployed for monitoring purposes; (3) smart environments where ICTs are used in the monitoring of the environment to acquire useful information regarding environmental sustainability; (4) smart services where ICTs are used for the the provision of community health, tourism, education and safety; (5) smart governance, which aims at proper delivery of government services; (6) smart people that use ICTs to access and increase humans’ creativity; (7) smart living where technology is used for the improvement of the quality of life; and (8) smart economy, where businesses and organizations develop and grow through the use of technology. Given these components, a smart health system within a smart city appears to be one of the leading gateways to a more productive and liveable structure that ensures the well-being of the community.
Assurance of quality healthcare is a social sustainability concept in a smart city. Social sustainability denotes the liveability and wellbeing of communities in an urban setting [5
]. The large population in cities is actually a basis for improved healthcare services because one negligence or improper health service might lead to an outbreak of diseases and infections, which might become epidemic, thereby costing much more in curtailing them; however, for these health services to be somewhat beneficial, the methods and channels of delivery need to be top-notch. There have been various research studies into the adequate method required for effective delivery of healthcare. One of these methods is smart health. Smart health basically is the provision of health services using the sensing capabilities and infrastructures of smart cities. In recent years, smart health has gained wide recognition due to the increase of technological devices and the ability to process the data gathered from these devices with minimum error.
As recent research states, proper management and development of smart health is the key to success of the smart city ecosystem [6
]. Smart health involves the use of sensors in smart devices and specifically manufactured/prototyped wearable sensors/bio-patches for proper monitoring of the health status of individuals living within a smart city, as shown in Figure 1
. This example depicts a scenario for air quality monitoring to ensure healthier communities. Smart city infrastructure builds upon networked sensors that can be either dedicated or non-dedicated. As an integral component of the smart city, smart health systems utilize devices with embedded sensors (as non-dedicated sensing components) for environmental and ambient data collection such as temperature, air quality index (AQI) and humidity. Besides, wearables and carry-on sensors (as the dedicated sensing components) are also utilized to acquire medical data from individuals. Both dedicated and non-dedicated sensory data are transmitted to the data centres as the inputs of processing and further decision making processes. To achieve this goal, the already existing smart city framework coupled with IoT networking needs to be leveraged.
In other words, a smart city needs to provide the required framework for smart health to grow rapidly and achieve its aim. Ensuring the quality of big sensed data acquisition is one key aspect of smart health challenges, and the ability to leverage these data is an important aspect of smart health development, as well as building a sustainable smart city structure [7
]. Applications of smart health within smart cities are various. For example, Zulfiqar et al. [10
] proposed an intelligent system for detecting and monitoring patients that might have voice complication issues. This is necessary since a number of services within the smart cities are voice enabled. As such, any disorder with the voice might translate into everyday service problems within the smart city. The same problem has been studied by the researchers in [11
], where voice data and ECG signals were used as the inputs to the voice pathology detection system. Furthermore, with the aim of ensuring air quality within a smart city, the researchers in [12
] developed a cloud-based monitoring system for air quality to ensure environmental health monitoring. The motivation for environmental health is ensuring the wellness of communities in a smart city for sustainability. The body area network within a smart city can be used for ECG monitoring with the aim of warning an individual of any heart-related problem, especially cardiac arrest [13
], and also helps in determining the nature of human kinematic actions with the aim of ensuring improved quality of healthcare whenever needed [14
]. Data management of patients is also one of the applications of smart health in smart cities that is of paramount importance. As discussed in [15
], proper management of patient records both at the data entry and application levels ensures that patients get the required treatment when due, and this also helps in the development of personalized medicine applications [16
]. Furthermore, the scope of smart health in smart cities is not limited to the physiologic phenomena in human bodies; it also extends to the environment and physical building blocks of the smart infrastructure. Indeed, the consequences of mismanaged environment and/or physical structures are potentially unhealthy users and communities, which is not a sustainable case for a smart city. To address this problem, the researchers in [17
] created a system to monitor the structural health within a city using wireless sensor networks (WSN). It is worth mentioning that structural health monitoring also leads to inferential decision making services.
With these in mind, calls for new techniques that will ensure proper health service delivery have emerged. As an evolving concept, machine intelligence has attracted the healthcare sector by introducing effective decision support mechanisms applied to sensory data acquired through various media such as wearables or body area networks [18
]. Machine learning techniques have undergone substantial improvements during the evolution of artificial intelligence and its integration with sensor and actuator networks. Despite many incremental improvements, deep learning has arisen as the most powerful tool thanks to its high level abstraction of complex big data, particularly big multimedia data [19
Deep learning (DL) derives from conventional artificial neural networks (ANNs) with many hidden perceptron layers that can help in identifying hidden patterns [20
]. Although having many hidden perceptron layers in a deep neural network is promising, when the concept of deep learning was initially coined, it was limited mostly by computational power of the available computing systems. However, with the advent of the improved computational capability of computing systems, as well as the rise of cloudified distributed models, deep learning has become a strong tool for analysing sensory data (particularly multimedia sensory data) and assisting in long-term decisions. The basic idea of deep learning is trying to replicate what the human brain does in most cases. Thus, in a sensor and actuator network setting, the deep learning network receives sensory input and iteratively passes it to subsequent layers until a desirable output is met. With the iterative process, the weights of the network links are adapted so as to match the input with the desirable output during the training process. With the widespread use of heterogeneous sensors such as wearables, medical imaging sensors, invasive sensors or embedded sensors in smart devices to acquire medical data, the emergence and applicability of deep learning is quite visible in modern day healthcare, from diagnosis to prognosis to health management.
While shallow learning algorithms enforce shallow methods on sensor data for feature representation, deep learning seeks to extract hierarchical representations [21
] from large-scale data. Thus, the idea is using deep architectural models with multiple layers of non-linear transformations [23
]. For instance, the authors in [24
] use a shallow network with a covariance representation on the 3D sensor data in order to recognize human action from skeletal data. On the other hand, in the study in [25
], the AlexNet model and the histogram of oriented gradients features are used to obtain deep features from the data acquired through 3D depth sensors.
In this article, we provide a thorough review of the deep learning approaches that can be applied to sensor data in smart health applications in smart cities. The motivation behind this study is that deep learning techniques are among the key enablers of the digital health technology within a smart city framework. This is due to the performance and accuracy issues experienced by conventional machine learning techniques under high dimensional data. Thus, it is worth noting that deep learning is not a total replacement of machine learning, but an effective tool to cope with dimensionality issues in several applications such as smart health [26
]. To this end, this article aims to highlight the emergence of deep learning techniques in smart health within a smart city ecosystem and at the same time to give future directions by discussing the challenges and open issues that are still pertinent. In accordance with these, we provide a taxonomy of sensor data acquisition and processing techniques in smart health applications. Our taxonomy and review of deep learning approaches pave the way for providing insights for deep learning algorithms in particular smart health applications.
This work is organized as follows. In Section 2
, we briefly discuss the transition from conventional machine learning methodologies to the deep learning methods. Section 3
gives a brief overview of the use of deep learning techniques on sensor network applications and major deep learning techniques that are applied on sensory data, while Section 4
provides insights for the smart health applications where deep learning can be used to process and interpret sensed data. Section 5
presents outstanding challenges and opportunities for the researchers to address in big sensed health data by deep learning. Finally, Section 6
concludes the article by summarizing the reviewed methodologies and future directions.
5. Deep Learning Challenges in Big Sensed Data: Opportunities in Smart Health Applications
Deep learning can assist in the processing of sensory data by classification, as well as prediction via learning. Training a dataset by a deep learning algorithm involves prediction, detection of errors and improving prediction quality with time. Based on the review of the state of the art in the previous sections, it can be stated that integration of sensed data in smart health applications with deep learning yields promising outcomes. On the other hand, there are several challenges and open issues that need to be addressed prior to realization of such integration. As those challenges arise from the nature of deep learning, sensor deployment and sensory data acquisition, addressing those challenges paves the way towards robust smart health applications. As a corollary, in this section, we introduce challenges and open issues in the integration of deep learning with smart health sensory data; and pursuant to these, we present opportunities to cope with these challenges in various applications with the integration of deep learning and the sensory data provided.
5.1. Challenges and Open Issues
Deep learning techniques have attracted researchers from many fields recently for sustainable and efficient smart health delivery in smart environments. However, it is worth noting that the application of deep learning techniques on sensory data still experiences challenges. Indeed, in most smart health applications, CNNs have been introduced as revolutionized methodologies to cope with the challenges that deep learning networks suffer. Thanks to the improvements in CNNs, to date, CNN has been identified as the most useful tool in most cases when smart health is involved.
To be able to fully exploit deep learning techniques on medical sensory data, certain challenges need to be addressed by the researchers in this field. The challenges faced by deep learning techniques in smart health are mostly related to the acquisition, quality and dimensionality of data. This is due to the fact that inferences or decisions are made based on the output/outcome of processed data. As seen in the previous section, data acquisition takes place on heterogeneous settings, i.e., various devices with their own sampling frequency, operating system and data formats. The heterogeneity phenomenon generally results in a data plane with huge dimensions. The higher the dimension gets, the more difficult the training of the data, which ultimately leads to a longer time frame for result generation. Moreover, determining the depth of the network architecture in order to get a favourable result is another challenge since the depth of the network impacts the training time, which is an outstanding challenge to be addressed by the researchers in this field.
Value and trustworthiness of the data comprise another challenge that impacts the success of deep learning algorithms in a smart health setting. As any deep learning technique is supposed to be applied to big sensed health data, novel data acquisition techniques that ensure the highest level of trustworthiness for the acquired data are emergent.
Uncertainty in the acquired sensor data remains a grand challenge. A significant amount of the acquired data is partially labelled or unlabelled. Therefore, novel mechanisms to quantify and cope with the uncertainty phenomenon in the sensed data are emergent to improve the accuracy, as well as the efficiency of deep learning techniques in smart health applications.
Furthermore, data acquisition via non-dedicated sensors is also possible in smart cities sensing [85
]. In the presence of non-dedicated sensors for data acquisition in smart health, it is a big challenge to know how much data should be acquired prior to processing. Since dedicated and non-dedicated sensors are mostly coupled in smart health applications, determining the amount of data required from different wearables becomes a grand challenge, as well. Recent research proposes the use of compressing sensing methods in participatory or opportunistic sensing via mobile devices [86
]; hence, data explosion in non-dedicated acquisition can be prevented. However, in the presence of non-dedicated sensing system, dynamic determination of the number of wearables/sensors that can ensure the desired amount of data is another open issue to be addressed prior to analysing the big sensed data via deep learning networks.
In addition to all this, ensuring trustworthiness of the acquired sensory data prior to deep learning analysis remains an open issue, while auction and game theoretic solutions have been proposed to increase user involvement in the trustworthiness assurance stage of the data that are acquired via non-dedicated sensors [87
]. In the presence of a collaboration between dedicated and non-dedicated sensors in the data acquisition, coping with the reliability of the non-dedicated end still requires efficient solutions. It is worth noting that deep learning can also be used for behaviour analysis of non-dedicated sensors in such an environment with the aim of eliminating unreliable sensing sources in the data acquisition.
Last but not least, recent research points out the emergence of IoT-driven data acquisition systems [78
5.2. Opportunities in Smart Health Applications for Deep Learning
In this section, we discuss some of the applications of deep learning. To this end, we categorize these applications into three main groups for easy reference. Table 3
shows a summary of these applications together with the deep learning methods used under the three categories, namely medical imaging, bioinformatics and predictive analysis. The table is a useful reference to select the appropriate deep learning technique(s) while aiming to address the challenges and open issues in the previous subsection.
Medical imaging: Deep learning techniques have actually helped the improvement of healthcare through accurate disease detection and recognition. An example is the detection of melanoma. To do this, deep learning algorithms learn important features related to melanoma from a group of medical images and run their learning-based prediction algorithm to detect the presence or likelihood of the disease.
Furthermore, using images from MRI, fMRI and other sources, deep learning has been able to help 3D brain construction using autoencoders and deep CNN [90
], neural cell classification using CNN [65
], brain tissue classifications using DBN [67
], tumour detection using DNN [65
] and Alzheimer’s diagnosis using DNN [91
Bioinformatics: The applications of deep learning in bioinformatics have seen a resurgence in the diagnosis and treatment of most terminal diseases. Examples of these could be seen in cancer diagnosis where deep autoencoders are used using gene expression as the input data [92
]; gene selection/classification and gene variants using micro-array data sequencing with the aid of deep belief networks [93
]. Moreover, deep belief networks play a key role in protein slicing/sequencing [94
Predictive analysis: Disease predictions have gained momentum with the advent of learning-based systems. Therefore, with the capability of deep learning to predict the occurrence of diseases accurately, predictive analysis of the future likelihood of diseases has experienced significant progress. Particular techniques that are used for predictive analysis of diseases are autoencoders [96
], recurrent neural networks [97
] and CNNs [97
]. On the other hand, it is worth mentioning that in order to improve the accuracy of prediction, sensory data monitoring medical phenomena have to be coupled with sensory data monitoring human behaviour. Coupling of data acquired from medical and behavioural sensors helps in conducting effective analysis of human behaviour in order to find patterns that could help in disease predictions and preventions.
With the growing need and widespread use of sensor and actuator networks in smart cities, there is a growing demand for top-notch methods for the acquisition and processing of big sensed data. Among smart city services, smart healthcare applications are becoming a part of daily life to prolong the lifetime of members of society and improve quality of life. With the heterogeneous and various types of data that are being generated on a daily basis, the existence of sensor and actuator networks (i.e., wearables, carry-ons and other medical sensors) calls for effective acquisition of sensed data, as well as accurate and efficient processing to deduce conclusions, predictions and recommendations for the healthiness state of individuals. Deep learning is an effective tool that is used in the processing of big sensed data especially under these settings. Although deep learning has evolved from the traditional artificial neural networks concept, it has become an evolving field with the advent of improved computational power, as well as the convergence of wired/wireless communication systems. In this article, we have briefly discussed the growing concept of smart health within the smart city framework by highlighting its major benefits for the social sustainability of the smart city infrastructure. We have provided a comprehensive survey of the use of deep learning techniques to analyse sensory data in e-health and presented the major deep learning techniques, namely deep feed-forward networks, autoencoders, convolutional neural networks, deep belief networks, Boltzmann machine and restricted Boltzmann machine. Furthermore, we have introduced various data acquisition mechanisms, namely wearables, probes and crowd-sensing. Following these, we have also linked the surveyed deep learning techniques to existing use cases in the analysis of medical sensory data. In order to provide a thorough understanding of these linkages, we have categorized the sensory data acquisition techniques based on the available technology for data generation. In the last part of this review article, we have studied the smart health applications that involve sensors and actuators and visited specific use cases in those applications along with the existing deep learning solutions to effectively analyse sensory data. To facilitate a thorough understanding of these applications and their requirements, we have classified these applications under the following three categories: medical imaging, bioinformatics and predictive analysis. In the last part of this review article, we have studied smart health applications that involve sensors and actuators and visited specific problems in those applications along with the existing deep learning solutions to effectively address those problems. Furthermore, we have provided a thorough discussion of the open issues and challenges in big sensed data in smart health, mainly focusing on the data acquisition and processing aspects from the standpoint of deep learning techniques.