Machine Learning Applications: The Past and Current Research Trend in Diverse Industries

Abstract

Dramatic changes in the way we collect and process data has facilitated the emergence of a new era by providing customised services and products precisely based on the needs of clients according to processed big data. It is estimated that the number of connected devices to the internet will pass 35 billion by 2020. Further, there has also been a massive escalation in the amount of data collection tools as Internet of Things devices generate data which has big data characteristics known as five V (volume, velocity, variety, variability and value). This article reviews challenges, opportunities and research trends to address the issues related to the data era in three industries including smart cities, healthcare and transportation. All three of these industries could greatly benefit from machine learning and deep learning techniques on big data collected by the Internet of Things, which is named as the internet of everything to emphasise the role of connected devices for data collection. In the smart grid portion of this paper, the recently developed deep reinforcement learning techniques and their applications in Smart Cities are also presented and reviewed.

1. Introduction

To date, the economic, architectural, and fundamental changes in traditional internet have transformed not only the social environment but also the business ecosystem. However, the transformation of big, unstructured and stream data to meaningful knowledge is still a challenge ahead of researchers, even though the cyber-physical infrastructure, sensing technologies and telecommunication systems have accelerated the data collection and transition. Machine learning is the general technique of artificial intelligence that can learn relationships from data, which can be provided by information technology without the requirement of defining them in advance [1]. For example, in health, acquiring knowledge and actionable perceptions from complex, multi-dimensional and heterogeneous data is a primary challenge in transforming health care.

Several internet- and network-based paradigms are under the Internet of Everything (IoE) umbrella, such as the Internet of Things (IoT), Internet of People (IoP), and Industrial Internet (II) [2]. The IoE is a modern smart technology paradigm envisioned as a universal connection network of machines and smart devices capable of networking with each other, business processes, people and the social environment. The IoE is recognised as one of the most critical areas of future technology, and is gaining vast attention from an extensive range of industries [3]. IoE sensors and devices are generating massive amounts of high-dimensional and heterogeneous data that need to be stored and processed.

This paper will explore the challenges and issues of IoE and machine learning techniques in three different and diverse business domains, such as health management, supply chain management and the smart grid to demonstrate the applicability and functionality of the technology and the methodology.

The remainder of the paper is structured as follows. Section 2 explains the issues and challenges in healthcare, and Section 3 reviews the smart grid and the related machine learning research. The deep reinforcement learning for smart cities briefly describes the reinforcement learning topic, and Section 4 analyses the deep reinforcement learning applications for smart cities. Applications in supply chain management are discussed in Section 5, and Section 6 concludes the paper.

2. IoT and Machine Learning in Healthcare

2.1. Issues and Challenges in Healthcare

The ageing population is putting pressure on the healthcare budget, and new solutions need to be proposed to compensate for scarce healthcare resources. One promising technology to solve this problem is the IoT. With application to healthcare, IoT can be considered as a facilitator for monitoring, diagnostics, and even the possibility for telesurgery via the internet. The challenges regarding the implementation of smart and connected devices for healthcare purposes are discussed in this paper [4,5].

Considering the wide distribution of IoT devices in the healthcare sector, one of the main challenges will be related to the security and identification of the nodes. This is necessary to detect the received information and assign it to the correct node. The security of the nodes is particularly crucial, as the system can be affected by malicious activities. One of the solutions for this problem could be the use of trusted execution environments (TEEs) [6,7].

The other approach is to identify the nodes effectively through a unique identifier (UID) so that the information exchange can be completed without ambiguity. The future of IoT devices needs to be compatible with the following features:

(1)

A global ID method to locate the items effectively;

(2)

Identity management to be able to secure the full cycle of the encoding/encryption, authentication and repository management.

Another issue that needs to be addressed—particularly in the health sector—is telecommunication management. Selection of the most optimised combination for telecommunication technologies could significantly reduce the downtime of the device and the availability of patient monitoring systems [8].

A further challenge for the healthcare IoT is the development of location technology. The high penetration of IoT devices in the healthcare industry demands the requirement of real-time location systems. This technology can track the treatment process securely and help to reconfigure the healthcare system based on the availability of distributed resources. One of the effective techniques in this area could be a combination of global positioning systems (GPS) and the IoT. The addition of local positioning systems is necessary to enhance the location accuracy [9].

Another challenge is related to the continuous data acquisition from patients. The availability of continuous data will enable doctors to provide more accurate treatment. Some of the traditional data acquisition methods include heart and blood oxygen saturation detection. Some of the recent developments enable the massive use of instruments such as accelerometers, gyroscopes, and surface electrodes for recording the data [10].

In the implemented IoT-based healthcare, millions of nodes could be deployed. Service-oriented architecture is considered to be a promising solution for the connectivity of millions of data points. This system can support modular design and application integration. This architecture will support different programming environments and service implementation in various languages [11]. The successful implementation of IoT-based healthcare systems is based on the big data collected from different sources including but not limited to homes, hospitals, aged care centres and specialised clinics. Although the cost of storage is rapidly dropping, the collection of the tremendous amount of data still is a very costly exercise. The crucial missing point is the development of intelligent algorithms that can distinguish the redundant data and remove the unnecessary storage [12].

2.2. Research Trend in Health by Studying Bibliometric Networks

We referred to the Web of Science (WoS) database [13] to discover the research trends in health considering IoT and machine learning topics since the year 2000. In Figure 1, the research fields are categorized by different colours, and the size of the spheres shows the volume of papers published in each area. For example, the keywords such as “security”, “healthcare system”, “communication technology”, “IoT technology” and “radio-frequency identification (RFID)” are in a green colour. In this context, most of the research has focused on security. However, the second category refers to keywords such as “heart rate”, “wearable devices”, “detection”, “health monitoring”, “real-time” and “cloud computing”, which are categorised in another group with a red colour in which cloud computing and health monitoring have attracted more research. In general, the trend of research in IoT and health among 1000 published papers between 2000 and 2017 is towards the security, protocol, cloud computing, health monitoring, detection and wireless networks. We conducted the same research about the machine learning and health topics, and the result of the correlation between the research subjects is shown in Figure 2. The results indicate that 2190 papers were published between 2000 to 2017, and can be categorized into five sectors. Among various machine-learning methods, the support vector machine, random forest, and logistic regression were the primary methods studied.

Figure 2 reveals that there was a significant relationship between artificial intelligence, big data, expert systems, and ontology with health topics such as infection, genomics, intensive unit care, bioinformatics, and pathogens. In this figure knn, ann, and svr stand for k nearest neighbors, artificial neural network, and support vector machine methods.

However, the logistic regression technique has mainly been applied for health topics such as depression, stress and anxiety. We used the VOSviewer software to analyse the data, as this tool has been used extensively by scholars to investigate literature reviews in various domains [14].

3. Machine Learning Applications in Smart Grid

Machine learning has become an emerging technology in multiple fields, and has extensive use from commercial to non-commercial uses. In particular, the fast-growing applications of deep learning have increased the development of machine learning. Consequently, deep learning has become a hot research topic in research organisations and tech companies. In general, deep learning uses a multi-layer neural network model to extract high-level features into a combination of low-level abstractions in order to find distributed data features, and solve complex problems in machine learning. The smart grid in general, as well as clean-technology, are some of the early adopters of machine learning technologies. In the following, some of the practical applications of deep learning and machine learning are reviewed. Fault detection in solar photovoltaic (PV) arrays is a crucial task for increasing the reliability and safety of PV systems. Because of PV’s nonlinear characteristics, a variety of faults may be difficult to detect by conventional protection devices, which can potentially lead to safety issues and in some cases can lead to fire hazards in PV fields. Machine learning techniques can solve this problem by detecting the faults based on voltage, current, irradiance, and temperature. Graph-based, semi-supervised learning models using only a few labelled training data that are normalized for better visualisation can be deployed to detect faults in photovoltaic arrays [15]. Another application of machine learning in the smart grid is energy forecasting for renewable plants considering the uncertainties in renewable generation. Uncertainty quantification is a crucial element input to maintain acceptable levels of reliability and profitability in power system operation. The extreme learning machine (ELM) is a fast regression model, trained in varied ways to obtain both point and quantile forecasts of solar power generation [16].

The increasing penetration of solar generation systems in the main electricity grid demands more accurate forecasting of electricity generation. The central element for PV systems is the Sun and its intensity. The smart grid needs to be able to predict PV generation fluctuations and adapt the system accordingly. One of the new methods is to predict the PV generation based on various satellite images and a support vector machine (SVM) learning scheme. The motion vectors of clouds are forecasted by utilising satellite images of atmospheric motion vectors (AMVs) [17].

Further applications of machine learning in the smart electricity environment are related to the prediction of the electricity customer from the retailer’s point of view. The behaviour of a day-ahead (DA) retail electrical energy market, with price-based demand response (DR) through a hierarchical multi-agent framework, employs a machine learning approach. At the top level of the hierarchy, a retailer agent buys energy from the DA wholesale market and sells it to the consumers. The goal of the retailer agent is to maximise profit by setting the optimal retail prices. Since the retailer agent does not have direct access to the consumption pattern of the retail customer, the retailer’s decision creates a risk for the retail agent, and there is considerable uncertainty. Machine learning can help by developing a reliable model of the aggregate behaviour of price-sensitive customers to reduce the possibility of the decision-making process. The applied method leads to a reduction in overall power consumption cost [18]. The other applications of machine learning are related to micro-grids. The historical data which are collected by the energy management system can be used to predict the pattern of generation and consumption in the micro-grid. A combination of kernelised and regularised least squares, extreme learning machines, and random forests are used. In the machine learning field, these algorithms are the best choices in three different families of techniques: kernel methods, neural networks, and ensemble methods, respectively. The applied method can significantly improve the operation of the micro-grids and eventually lead to considerable savings [19].

4. Deep Reinforcement Learning Applications for Smart Cities

One of the significant contributors to boost the penetration level of green energy resources is the adaptation of the demand side to adjust consumption based on the signals from the electricity market. Primary consumers of electricity include residential and commercial buildings. Improving the energy efficiency and applying new approaches for energy management could result in a higher penetration of green energy, marking a shift towards a sustainable energy environment. The key factor of the practical demand side is understanding individual consumption patterns based on the information from the extensive data collected from advanced metering infrastructure (AMI) as well as new data sources from smart home IoT. The advanced analytics on available consumer data will lead to optimised building energy consumption and better insights about building energy management [20].

The traditional methods search the whole optimisation space and select the best possible solutions. The new machine learning methods can introduce more effective methods for building energy optimisation. The availability of the historical data made it possible to train powerful machine learning methods to extract insights from the electricity consumption pattern. After comparing different machine learning methods, reinforcement learning (RL) can provide promising results for the cost minimisation of the total energy cost of a building [21].

To achieve the energy optimisation goal, a building which is considered for RL energy optimisation needs to be modelled using a Markov decision process. In 2015, the successful applications of Q-learning (e.g., playing an Atari2600 game and defeating the world champion for the game called GO) demonstrated the promising horizon for new applications using deep learning. Deep reinforcement learning combines reinforcement learning and neural networks, and can deliver promising applications in the energy industry such as energy price predictions as well as energy optimisation, without noticeable compromise in the customer lifestyle.

The present research reviews the advantages of the deep policy gradient method, which is considered as a subgroup of deep reinforcement learning in the context of complex networks such as electricity networks. The proposed algorithm is fast enough to enable the successful applications of real-time energy and cost optimisation in the smart grid context. Two advanced methods of deep policy gradient (DPG) and deep Q-learning are compared for solving the real-time energy optimisation problem. A point of difference for deep reinforcement learning is that the developed method considers the electricity consumption pattern by the electricity customers in the optimisation procedure. The latter is very important to achieve an optimum solution with minimum impacts on customer lifestyle [22].

5. Applications in Supply Chain Management

Organisations cannot act in isolation, and they depend on the capabilities and resources embedded in their suppliers, customers and collaborators. Attention towards supply chain management (SCM) has been given since 1980, when organisations realised the benefits of collaborative relationships via SCM within their boundaries [23]. Sustainably managing supply chains has become an increasing concern for organisations of all sizes and across a wide range of industries. This more reactive approach of responding to external pressure from governments, consumers and non-governmental organisations (NGOs) and media can be complemented by the development and introduction of sustainable products [24]. In the current competitive environment, supply chain professionals are struggling to handle the large structured and unstructured data. They are surveying new techniques to investigate how data are produced, captured, organised and analysed to give valuable insights to industries. Big data analytics is one of the best methods to help them in overcoming their problem. IoT in the context of SCM is defined as “a network of physical objects that are digitally connected to sense, monitor and interact within a company and between the company and its supply chain enabling agility, visibility, tracking and information sharing to facilitate timely planning, control and coordination of the supply chain processes” [25]. Our search in the Web of Science database showed that 308 papers were published in the topics of “IoT” and “supply chain” between 1980 to 2018. Figure 3 shows the contribution of countries in the research of these topics, in which China has contributed most by producing 45% of the knowledge domain whereas the USA stood in the second rank with 20% of the overall published papers.

However, there was an equivalent segment among countries such as India, South Korea, Taiwan and Finland. One of the interesting points about Figure 3 is the contribution of India in the knowledge field, which is 3% in contrast to China, as these countries are the most populated states in the list.

Figure 4 shows the trend of publications from 2008 to February 2018 in which these topics accounted for 1% of overall papers in 2008, and 21% in 2016 and 2017. The trend validates how drastically the technology expanded in 2014 when the knowledge production was constant in 2011, 2012, and 2013.

Overall, the acceptance of sustainability by an organization can boost its competitiveness, as it can enhance the firm’s image in the eyes of the customers and can increase the firm’s economic performance [26]. In the stream of supply chain management, sustainability is mainly discussed regarding green supply chain management (GSCM), which is not a new concept as it was introduced in 1989 [26]. However, the literature on GSCM criteria is rare since the GSCM philosophy is in parallel with, or overlaps with, some other eminent management agendas (e.g., cleaner production).

The bibliometric networks study on GSCM and IoT topics shown in Figure 5 reveals that 634 papers fell under GSCM and IoT topics and were published between 2000 and Feb 2018.

Figure 5 shows that applications of the IoT in the green supply chain are divided into two categories: practice and process. The environment protection and sustainable development, green product and green supplier selection are mainly interrelated, whereas the subjects such as GSCM implementation, iso (international standard organization), green purchasing and economic performance are associated with green practices and drivers class.

The challenges and issues of IoT in SCM have been discussed broadly in the literature [27,28]. Research has been conducted in each chain of supply from supplier control and purchasing, to customer relationship management and recycling. Technologies such as RFID tags, wireless sensor networks (WSN), and cloud computing have been utilised to develop the SCM.

6. Conclusions

This paper demonstrates the diversity of applications of the IoE as a cyber-physical network in which data are transferred and stored, and machine learning as techniques to manipulate data and construct meaningful knowledge in three separate domains. We demonstrate these applications in the fields of health, smart electrical grids, and GSCM. This study can be expanded in more detailed domains, and the authors show the broad applications of the technology and methodology to demonstrate how they are supporting each other to enhance the knowledge domain.

Our research shows that even though there are many machine learning techniques, researchers have employed specific techniques for specific problems. For example, in health pattern recognition techniques are applied for health monitoring, or deep learning is used for health management. In smart grids, different methods have been employed in different sectors. For example, the SVM method has been used for forecasting the power generation by solar energy; or in energy consumption, kernel methods, neural networks, and ensemble methods are used to learn the aggregate behaviour of price-sensitive customers to reduce the uncertainty of the decision-making process. This study shows that technologies such as RFID tags, wireless sensor networks, and cloud computing have been utilised to develop the SCM.

Big data and cloud computing, which are the main parts of the IoE subject, were not discussed herein as they were beyond the scope of the paper and can be considered for future study and research.