Next Article in Journal
A BIM Platform for the Manufacture of Prefabricated Steel Structure
Next Article in Special Issue
The New York City COVID-19 Spread in the 2020 Spring: A Study on the Potential Role of Particulate Using Time Series Analysis and Machine Learning
Previous Article in Journal
MobiDiabet: Mobile Telemonitoring System for Patients with Type 2 Diabetes Mellitus (T2DM)
Previous Article in Special Issue
Development of a Mortality Risk Model in Elderly Hip Fracture Patients by Different Analytical Approaches
Article

Risk-Averse Food Recommendation Using Bayesian Feedforward Neural Networks for Patients with Type 1 Diabetes Doing Physical Activities

1
Norwegian Centre for E-health Research, University Hospital of Northern Norway, 9019 Tromsø, Norway
2
Department of Computer Science, Faculty of Science and Technology, UiT The Arctic University of Norway, 9019 Tromsø, Norway
3
Department of Mathematics and Statistics, Faculty of Science and Technology, UiT The Arctic University of Norway, 9019 Tromsø, Norway
*
Authors to whom correspondence should be addressed.
Appl. Sci. 2020, 10(22), 8037; https://doi.org/10.3390/app10228037
Received: 7 October 2020 / Revised: 6 November 2020 / Accepted: 9 November 2020 / Published: 12 November 2020
(This article belongs to the Special Issue Medical Artificial Intelligence)

Abstract

Background. Since physical activity has a high impact on patients with type 1 diabetes and the risk of hypoglycemia (low blood glucose levels) is significantly higher during and after physical activities, an automatic method to provide a personalized recommendation is needed to improve the blood glucose management and harness the benefits of physical activities. This paper aims to reduce the risk of hypoglycemia and hyperglycemia (high blood glucose levels), and empowers type 1 diabetes patients to make decisions regarding food choices connected with physical activities. Methods. Traditional and Bayesian feedforward neural network models are developed to provide accurate predictions of the blood glucose outcome and the risks of hyperglycemia and hypoglycemia with uncertainty information. Using the proposed models, safe actions that minimize the risk of both hypoglycemia and hyperglycemia are provided as food recommendations to the patient. Results. The predicted blood glucose responses to the optimal and safe food recommendations are significantly better and safer than by taking random food. Conclusions. Simulations conducted on the state-of-the-art UVA/Padova simulator combined with Brenton’s physical activity model show that the proposed methodology is safe and effective in managing blood glucose during and after physical activities.
Keywords: type-1 diabetes; machine learning; feedforward neural networks; Bayesian neural networks; physical activities type-1 diabetes; machine learning; feedforward neural networks; Bayesian neural networks; physical activities

1. Introduction

Type 1 diabetes (T1D) is a chronic metabolic disorder characterized by elevated blood glucose levels over a prolonged period, leading to long-term damage to the heart, kidneys, eyes, nerves, and blood vessels. High blood glucose (hyperglycemia) in T1D is a consequence of the lack of insulin produced by the pancreas so that T1D patients require external insulin administration to regulate their blood glucose concentrations [1], while too much insulin dangerously reduces blood glucose level (hypoglycemia).
Treatment of T1D mainly consists of either a basal-bolus insulin regimen, where patients take basal insulin dose to regulate fasting blood glucose levels and insulin boluses around mealtimes to quickly reduce the impact of carbohydrate intake, or through an insulin pump providing a continuous insulin infusion. The insulin pump infuses both regular basal insulin rate and meal boluses activated by the user during food intake to avoid hyperglycemia [2]. In both therapies, the insulin is administered subcutaneously in the fatty tissue just below the skin. In addition to the external insulin administration, monitoring blood glucose levels is required and can be done via a continuous glucose monitor (CGM) embedded in the subcutaneous tissue, or several times per day via manual finger-prick measurements [3]. Finally, T1D patients in collaboration with a physician will design a treatment based on individual patient needs. Insulin doses will be self-administered by the patients according to the treatment plan and self-measured blood sugar concentrations. External insulin therapy aims to keep the blood glucose concentrations within the normoglycemic range between 70 and 180 mg/dL [4,5].
Regular physical exercise has many proven health benefits and is therefore widely recommended as part of a healthy lifestyle. However, exercise significantly alters glucose homeostasis in patients with T1D. In addition, physical activities increase glucose uptake by muscles leading to a drop in blood glucose concentration, which can reach the hazardous hypoglycemic values. Increased insulin sensitivity, during and several hours or even days after the exercise [6], creates long-lasting effects on daily activities of patients.
The common method for preventing hypoglycemia is to reduce insulin doses. However, the slow absorption of insulin from the subcutaneous tissue and the physical limit of insulin reduction make this method insufficient to prevent hypoglycemia. Additional to the insulin treatment, many T1D related conditions can be mitigated by a nutrition therapy, which arises as an important solution to prevent, manage and control diabetes, as well as relieve complications associated with T1D by adjusting the quantity, quality and methods of nutrient intake [7]. Along with a healthy diet, physical activity plays a vital role in diabetes treatment, producing multiple general and diabetes-specific health benefits [8]. Despite the evidenced benefits, many people are physically inactive [9], since exercise is a major source of hypoglycemia in diabetic patients [10], and risk of hypoglycemia is a significant limiting factor of blood glucose regulation in T1D patients [11]. For most diabetic people, exercise has far less adverse health consequences than sedentary lifestyle [12].
Healthcare approaches change from the traditional relationship between providers and patients to a paradigm that gives patients a crucial role in guiding their care [13]. The change emphasizes the importance of self-management, which is considered a necessary part of chronic disease management and secondary prevention [14], especially for diabetes patients. Evidence shows that supporting patients to manage their health will improve clinical outcomes, reduce the economic burden, and improve quality of life [14]. Food recommendation systems emerge as a new self-management solution that can suggest the best diets according to patients’ health situation and preferences, solving the physical activity paradigm while following a nutrition therapy for diabetic patients [15]. Among others, Phanich et al. [16] used a food-clustering analysis to propose a food recommendation system for patients with diabetes, while Norouzi et al. [17] developed a smartphone application for managing diabetic patient nutrition using artificial intelligence techniques. In [18], Lee et al. develop a diabetic food recommendation agent that, according to a personal lifestyle and particular health needs, can create a meal plan. Mohammed and Hagras [19] present a type-2 fuzzy logic-based diet recommendation system to help achieve a healthy lifestyle to control diabetes. A complete systematic review of nutrition recommendation systems with a focus on technical aspects can be found in [20]. Previous work from the authors has shown promising results for using machine learning techniques in a food recommendation system, maintaining healthy blood glucose levels on a T1D simulator during exercise [21].
The term machine learning is considered a large family of mathematical and statistical methods that have historically focused on prediction [22]. With the development of new technology, a vast amount of health-related data is continuously generated. However, data availability is varied among various dimensions and quality. Machine learning and statistical techniques such as feedforward neural networks and the Bayesian inferencing mechanism become powerful tools to understand and quantify data quality into uncertainties, which is a crucial step to make use of the increasingly available data safely and effectively.
Due to the importance and clinical benefits of the diet in diabetes, different studies have been conducted to develop diet recommendation systems to diabetes patients. Unlike [16,23], our work focused on preventing the complications related to physical activity in type-1 diabetes patients, alleviating the risks associated with doing exercise while having diabetes. Xie and Wang [24] proposed a food recommendation system with a similar purpose using a Nonlinear Auto Regressive Moving Average. However, the method did not provide a measurement of uncertainties in the data and how to compensate for these uncertainties. In this work, we introduced a new technique for food recommendation using Bayesian feedforward neural networks that can minimize the risk of hypoglycemia and hyperglycemia during and after physical activities while improving blood glucose regulation. We performed in-silico experiments, including the exercise model described in [25] on the UVA/Padova simulator [26,27]. Our experiments demonstrate that the proposed food recommendation system is able to reduce the risks of hypoglycemia and hyperglycemia while maintaining the blood glucose levels in the healthy range during and after the exercise.

Structure of Paper

We describe the methods in Section 2, where we introduce the experimental setup, defined the outcome and risk functions, and describe the models used to predict those functions. Section 3 presents the results and discussion. Section 4 provides the clinical significance and limitations of the method. In Section 5, we present concluding remarks and directions of possible future work.

2. Methods

The risk-averse food recommendation system presented in this paper used three criteria: the blood glucose outcome, the risk of hypoglycemia, and the risk of hyperglycemia. Recommendations were derived such that the risk of taking an action must be lower than a specified level and within a measure of probabilistic confidence level, while maximizing the outcome. This section starts with a description of the in-silico simulations used in the paper, followed by a formulation of the outcome and risks. Finally, implementation of deterministic and Bayesian feedforward neural networks prediction models are described.

2.1. In-Silico Simulation

In the simulation scenario used in the paper, the food recommendation is given before the beginning of each exercise session for a virtual patient with no meal boluses associated with the recommended amount of carbohydrates. During the experiments, the basal insulin dose was constant and equal to the optimal value, which means the virtual patient stays at the healthy reference blood glucose concentration B G r e f = 108 mg/dL in steady-state. Training data was obtained by repeated simulations from the blood glucose simulator under scenarios where a patient with T1D performs physical exercises with the same intensity but consuming a different amount of food.
The Physical Activity Guidelines for Americans recommends adults do vigorous enough exercise to raise their heart rate to 50–85% of their maximum heart rate, defined as 220 beats per minute minus their age, during 75 to 150 min a week—values might vary for younger people [28]. The virtual patient used in our simulations is 24 years old, with a maximum heart rate of 196 beats per minute and recommended heart rate during exercise between 98 and 167 beats per minute, based on the guidelines from the American Heart Association. The duration and intensity of physical activities are set to be constant at 50 min and 157 beats per minute (80% of its maximum heart rate), respectively. In absence of carbohydrate intake, a hypoglycemic excursion is induced as a consequence of the exercise session setup. The virtual patient always eats at 15 min before the exercise starts to avoid the hypoglycemic event. The outcome of each exercise is evaluated by measuring the average scores of the blood glucose over the course of four hours starting at 15 min before the exercise (mealtime). The blood glucose is sampled every 5 min during the simulations, which is similar to the sampling time of common CGM devices.
There exist mainly three physiological models in the T1D research field, namely the Bergman (minimal) model [29], the Hovorka model [30] and the UVA/Padova model [26], see also [31]. The minimal model is a simplified model consisting of two equations describing the internal dynamics of glucose and insulin and does not account for the significant delay involved in subcutaneous insulin infusion. The Hovorka and the UVA/Padova models both include this delay as well as the delay in the subcutaneous glucose measurement. In this work, we used the UVA/Padova model, since this is the only computer simulator of the dynamics of the human metabolic glucose-insulin system which is FDA approved as a substitute for the pre-clinical testing of certain control strategies in T1D [27]. An extension of the UVA/Padova model has been used in this paper where the effect of physical activity is included [25].
The UVA/Padova model consists of seven internal compartments describing the dynamics of glucose kinetics, insulin kinetics, glucagon kinetics and secretion, glucose rate of appearance, endogenous glucose production, glucose utilization and renal excretion, while three external compartments describe subcutaneous glucose, insulin and glucagon kinetics [27]. In addition, a physical activity model was included in the glucose-utilization subsystem, modifying the insulin-dependent utilization component to simulate exercise sessions describing changes in glucose-insulin dynamics [25]. The original UVA/Padova simulator includes ten children patients, ten adolescents patients, and ten adults patients, as well as one average child patient, one average adolescent patient, and one average adult patient. In this work, we used the adult patient number four during our experiments, since this patient presents acute hypoglycemia as a consequence of our physical activity experimental setup. Finally, we use a CGM for glucose measurements during our simulations, where the CGM sensor noise is generated based on the model and the parameters determined by [32]. The non-Gaussian sensor noise is given by:
e n = 0.8 ( e n 1 + v n ) , n > 0
v n N i i d ( 0 , 1 )
ε n = ξ + λ sin h e n γ δ ,
with the initial condition e 0 N i i d ( 0 , 1 ) . Note that in the original model from [32], the error e n introduced in the sensor noise is multiplied by a factor of 0.7, while in this work we increased the CGM noise multiplied the error e n by a factor of 0.8, adding more uncertainty information to the Bayesian feedforward neural network. The CGM sensor noise ε n was added to the blood glucose values obtained from the UVA/Padova simulator with physical activity. The numerical values used in this paper for ξ , λ , γ and δ are shown in Table 1.

2.2. Outcome Function and Risk Definition

This subsection defines the outcome and risk functions for the designed food recommendation system before physical activities for T1D patients. The blood glucose outcome is represented by the average score assigned for the blood glucose levels during and after physical activities. The score was calculated based on the asymmetric reward function previously introduced in [33], designed to reduce hypoglycemia while rewarding time spent in normoglycemia. The reward function is designed as a piecewise smooth function and gives a strong negative reward for severe hypoglycemia, followed by an exponentially decreasing negative reward for hypoglycemic events starting at severe hypoglycemia, and negative reward when hyperglycemia occurs. Positive rewards from a symmetric linear function are given for glucose values in normoglycemic range. To be specific, the function is given as:
r ( B G ) = 10 : B G < B G h y p o exp ( log ( 19.157 ) B G h y p o B G ) 19.157 : B G [ B G h y p o , B G h y p o ] 1 36 B G 2 : B G [ B G h y p o , B G r e f ] 1 72 B G + 5 2 : B G [ B G r e f , B G h y p e r ] 5 : B G > B G h y p e r ,
where B G is the current blood glucose value. The parameters of the reward function were selected based on the guidelines as described in [33] and can be found in Table 2. A graphical representation of the reward function is shown in Figure 1.
In order to quantify the risks of hypo- and hyperglycemia, we used clinically defined low and high blood glucose indices (LBGI and HBGI, respectively) as described in Clarke and Kovatchev [34], which measure the frequency and extent of low and high blood glucose readings based on a symmetrization of the blood glucose measurements:
L B G I = 1 n i = 1 n r l ( B G i ) and H B G I = 1 n i = 1 n r h ( B G i ) ,
where B G i is the measurement i in mg/dL and n is the number of measurements during and after the physical activity. The risk functions r l ( B G ) and r h ( B G ) are defined as follows:
r l ( B G ) = 10 × f ( B G ) 2 if f ( B G ) < 0 and 0 otherwise
r h ( B G ) = 10 × f ( B G ) 2 if f ( B G ) > 0 and 0 otherwise
where
f ( B G ) = 1.509 × [ ( ln ( B G ) ) 1.084 5.381 ] ,
and B G is the current blood glucose value.

2.3. Modelling of the Outcome, Risk and Uncertainties

This subsection describes the methods to predict the blood glucose outcome, the hypoglycemia and the hyperglycemia risks of actions based on a predefined set of variables. Uncertainty estimation using Bayesian techniques will be described with these methods.
The blood glucose outcome, hyperglycemia and hypoglycemia risks were predicted in this paper by using a deterministic feedforward neural networks (FFNN) and two Bayesian feedforward neural nerworks (BFNN) with the same structure. FFNN and BFNN are types of artificial neural networks which are constructed by neurons organized into layers. The networks estimate the outcome and risks based on inputs such as the intensity of physical activity, historical blood glucose and dietary information (carbohydrate intake) before the physical activity. The structure of the deterministic FFNN and the two BFNN implemented in this paper are similar and can be found in Figure 2. For demonstration, we include only the food amount (carbohydrate intake) as the input for the networks. It is assumed that other conditions are the same every time physical activity is conducted.
Information from the input also flows through hidden nodes, which are nonlinear functions of the carbohydrate intake amount D:
h j = σ w j ( 1 ) D + b j ( 1 ) ,
where h j is the output value of the j th , j = 1 , . . , n hidden node, w j and b j are the weights and biases. The function σ is a rectified linear unit activation function (ReLu):
σ ( x ) = max ( 0 , x ) .
The model output y, (either predicted risks or blood glucose outcome), is calculated from the outputs of the hidden layers by a linear function:
y ^ = w ( 2 ) D + b ( 2 ) ,
where w ( 2 ) and b ( 2 ) are the weight and bias for the output layer. The task of training the outcome model is to find the optimal values of of w and b such that the following mean squared error (MSE) cost function is minimized:
M S E = 1 N i = 1 N ( y ^ i y i ) 2 ,
where N is the number of training samples, y ^ i is the predicted value and y i is the actual blood glucose score for each training sample i. Training of the outcome model was done using least square estimation and implemented through the Pytorch library [35].
Two models were built to predict the hypoglycemia and hyperglycemia risks based on a Bayesian feedforward neural network in order to capture the uncertainties in the risk prediction. The networks have similar structure like the outcome prediction model with six hidden nodes. However, the weight and bias of the hidden and output layers are represented by Gaussian distributions instead of discrete numbers. The prior distribution of all the parameters were chosen to be normal distributions with mean of zero and standard deviation of one. The task of training the hyperglycemia and hypoglycemia models is to determine the posterior distribution for the weights and biases using Bayesian inference. The posterior distribution was estimated by using the stochastic variational inference method [36] and implemented with Pyro [37]. For each weight and bias z, a family of distributions is generated with each distribution characterized by parameters θ . Then, an approximation of the posterior distribution p θ ( z ) is conducted by selecting from this family a distribution that is closest to the true posterior p ( z | y , x ) , where x and y are the training data set. The selection is conducted by finding the optimal set of parameters θ that minimize the Kullback–Leibler divergence, which is a measurement of how two probability distributions are different from each other [36,38]:
K L ( p θ ( z ) , p ( z | y , x ) ) = z p θ ( z ) l o g p θ ( z ) p ( z | y , x ) = z p θ ( z ) l o g p ( z ) p ( y | z , x ) p ( z ) + l o g ( p ( y | x ) ) = L ( θ ) + l o g ( p ( y | x ) ) ,
where L ( θ ) is defined as the evidence lower bound (ELBO) function [36]:
L ( θ ) = z p θ ( z ) l o g p ( y | z , x ) p ( z ) p ( z ) .
Since the Kullback–Leibler divergence is negative, the task of determining the posterior distributions of all the weights and biases can be obtained by maximizing the ELBO by using the stochastic gradient descend method, through variational inferencing [36].

2.4. Validation

In order to validate the performances of the neural networks, parts of the data (20% for the FFNN and 50% for the BFNN) were randomly selected for testing the model’s accuracy and performances. By propagating the carbohydrate information in each sample set through the neural networks, the prediction values are calculated and compared with the testing data. For the FFNN that is used to blood glucose outcome, the distributions of the estimated risks and the actual risks are compared by using the MSE (Equation (12)), which is also used as the validation criteria. For the BFNNs that are used to estimate the hyperglycemia and hypoglycemia risks, the distributions of the estimated risks and the actual risks are compared by using the ELBO criteria (Equation (14)). Since more data is need for the ELBO than for the MSE calculation, a higher validation portion is assigned for the BFNN than the FFNN.

3. Results and Discussion

By using data generated from the UVA/Padova simulator, the outcome and risk prediction models were built based on gradient descend and the stochastic variational inference. Figure 3 shows the MSE of the training and testing data during the training process of the feedforward neural networks. Table 3 shows the final ELBO (defined by Equation (14)) of the Bayesian neural networks under without and with noise in the CGM measurements. Figure 4 shows the training data and predictions made by the models for different carbohydrate values without and with CGM noise, respectively. The deterministic reward prediction is represented by the solid red line. The predicted risks with 90%-confidence intervals are represented by a green and a blue-shaded uncertainty bands in the figure, correspondingly to the hypoglycemia and hyperglycemia risks, respectively.
It can be seen from Figure 3 and Table 3 that the prediction errors are higher when measurement noise is introduced. Consequently, Figure 4 shows that the predicted risk uncertainty bands are wider when there is measurement noise, indicating that the predicted risk cannot be accurately predicted compared to nominal condition.
Based on the prediction models of the reward and risks, safe and optimal actions are derived as shown in Table 4. Safe actions are defined as actions that have a 90% probability of the hypoglycemia risk under 5 and the hyperglycemia risk under 7. The risk thresholds reflect that higher emphasis was put on hypoglycemia due to the higher adverse effects of getting hypoglycemia. It can be seen that the lower and upper bound for safe actions are tighter under noise conditions, since the risk models capture the uncertainties and provide more conservative recommendations under more uncertain condition. The optimal recommendation is chosen among the safe actions that maximize the blood glucose outcome predicted by the deterministic feedforward neural network. Hence the value is the same for both conditions when there is CGM noise and when there is no CGM noise.
Figure 5 and Figure 6 show different ranges of blood glucose when the patient follows the optimal recommendation, consumes food within the safe recommendation actions, or takes food with carbohydrate amounts between 0 and 200 g. The distribution of blood glucose measurement according to food intakes at certain time points can also be seen in the figures. The results show that the range of blood glucose responses to the optimal and safe actions are significantly better than the range of blood glucose when any action can be selected. The blood glucose range is slightly narrower in the case with CGM noise since the recommendation is more conservative in order to compensate for uncertainties in the responses.

4. Clinical Significance and Limitations

The methods presented in the paper demonstrate that a clinical decision support tool can be built using machine learning. Uncertainties in the data can be estimated and recommendations are provided with a statistically confident level. The tools have the potentials to be applicable in many safety-critical applications beyond type-1 diabetes and contribute towards the development of safe artificial intelligence. However, future efforts are needed to ensure the algorithms can be applied beyond in sillico simulations.
As a demonstration of how risk-averse machine learning methods can be developed and how the relationship between food intake and reward and risk can be visualized, only one variable (carbohydrate) was chosen as the input of the neural networks. Future studies should incorporate more variables that can affect blood glucose during and after physical activities such as previous food intakes, insulin doses, blood glucose levels before exercises and heart rate. The uncertainties can also be expanded to include other sources such as incorrect carbohydrate measurements, changes in intensity of physical activities and blood glucose kinetics. It is also noted that more complex neural networks require more data to obtain a satisfactory accuracy or performance, therefore, new methods that can effectively reduce the dependence on a large dataset is needed. Propagation of uncertainty analysis also contributes and make the methods more reliable. The current technique assumes that the outputs of the Bayesian neural networks are Gaussian-distributed. New methods that can accurately predict the output distribution will increase the accuracy of the prediction interval, and also the recommendations provided to the patients.

5. Conclusions

This paper provides a method to predict blood glucose outcome and risks associated with physical activities. The prediction models were used to select safe and optimal food amounts for patients to consume before physical activities. Simulation results show that the feedforward neural networks accurately predicted the blood glucose outcome while the Bayesian neural networks effectively capture the uncertainties due to measurement noise in the risk predictions. The blood glucose responses to the safe and optimal actions are significantly better than random actions within the input range. The results also present a potential direction for the future development of safe AI methods that are not only effective but also minimizing potential risks to the patients.

Author Contributions

P.N. developed the food recommendation algorithms, performed training and validating the deterministic and Bayesian feedforward neural networks for predicting blood glucose outcome, hyperglycemia and hypoglycemia risks, conducted numerical simulations and lead the writing process. M.T. (Miguel Tejedor) implemented the UVA/Padova simulator with physical activities, developing risk and reward functions, CGM noise model, performed numerical simulations, and was involved in the writing process. F.G. acquired funding and resource, managed and supervised the project leading to this publication. M.T. (Maryam Tayefi) and T.C. provided critical feedback, analysed results, read and approved the final manuscript. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by Tromso Research Foundation under project “A smart controller for T1D using RL and SS representation” (Grant/award number: A3327). The publication charges for this article have been funded by a grant from the publication fund of UiT The Arctic University of Norway.

Conflicts of Interest

The authors declare no conflict of interest

References

  1. World Health Organization. Diabetes. Available online: https://www.who.int/health-topics/diabetes (accessed on 2 September 2020).
  2. Misso, M.L.; Egberts, K.J.; Page, M.; O’Connor, D.; Shaw, J. Continuous subcutaneous insulin infusion (CSII) versus multiple insulin injections for type 1 diabetes mellitus. Cochrane Database Syst. Rev. 2010. [Google Scholar] [CrossRef] [PubMed]
  3. Juvenile Diabetes Research Foundation Continuous Glucose Monitoring Study Group. Continuous glucose monitoring and intensive treatment of type 1 diabetes. N. Engl. J. Med. 2008, 359, 1464–1476. [Google Scholar] [CrossRef] [PubMed]
  4. El Fathi, A.; Raef Smaoui, M.; Gingras, V.; Boulet, B.; Haidar, A. The Artificial Pancreas and Meal Control: An Overview of Postprandial Glucose Regulation in Type 1 Diabetes. IEEE Control Syst. Mag. 2018, 38, 67–85. [Google Scholar]
  5. Breton, M.; Farret, A.; Bruttomesso, D.; Anderson, S.; Magni, L.; Patek, S.; Man, C.D.; Place, J.; Demartini, S.; Toffanin, C.; et al. Fully Integrated Artificial Pancreas in Type 1 Diabetes. Diabetes 2012, 61, 2230–2237. [Google Scholar] [CrossRef] [PubMed]
  6. Yardley, J.E.; Brockman, N.K.; Bracken, R.M. Could Age, Sex and Physical Fitness Affect Blood Glucose Responses to Exercise in Type 1 Diabetes? Front. Endocrinol. 2018, 9, 1–11. [Google Scholar] [CrossRef] [PubMed]
  7. Bantle, J.P.; Wylie-Rosette, J.; Albright, A.L.; Apovian, C.M.; Clark, N.G.; Franz, M.J.; Hoogwerf, B.J.; Lichtenstein, A.H.; Mayer-Davis, E.; Mooradian, A.D.; et al. Nutrition recommendations and interventions for diabetes: A position statement of the american diabetes association. Am. Diabetes Assoc. 2008, 31, 61–78. [Google Scholar]
  8. Hayes, C.; Kriska, A. Role of Physical Activity in Diabetes Management and Prevention. J. Am. Diet. Assoc. 2008, 108, 19–23. [Google Scholar] [CrossRef]
  9. LaMonte, M.J.; Blair, S.N.; Church, T.S. Physical activity and diabetes prevention. J. Appl. Physiol. 2005, 99, 1205–1213. [Google Scholar] [CrossRef]
  10. Basu, R.; Johnson, M.L.; Kudva, Y.C.; Basu, A. Exercise, Hypoglycemia, and Type 1 Diabetes. J. Am. Diet. Assoc. 2014, 16, 331–337. [Google Scholar] [CrossRef]
  11. Diabetes Control; Complications Trial Research Group. The relationship of glycemic exposure (HbA1c) to the risk of development and progression of retinopathy in the diabetes control and complications trial. Diabetes 1995, 44, 968–983. [Google Scholar] [CrossRef]
  12. Sigal, R.J.; Armstrong, M.J.; Colby, P.; Kenny, G.P.; Plotnikoff, R.C.; Reichert, S.M.; Riddell, M.C. Physical Activity and Diabetes. Can. J. Diabetes 2018, 37, 54–63. [Google Scholar] [CrossRef] [PubMed]
  13. Gobeil-Lavoie, A.P.; Chouinard, M.C.; Danish, A.; Hudon, C. Characteristics of self-management among patients with complex health needs: A thematic analysis review. BMJ Open 2019, 9, e028344. [Google Scholar] [CrossRef] [PubMed]
  14. Grady, P.A.; Gough, L.L. Self-Management: A Comprehensive Approach to Management of Chronic Conditions. Am. J. Public Health 2014, 104, e25–e31. [Google Scholar] [CrossRef] [PubMed]
  15. Norouzi, S.; Nematy, M.; Zabolinezhad, H.; Sistani, S.; Etminani, K. Food recommender systems for diabetic patients: A narrative review. Rev. Clin. Med. 2017, 4, 128–130. [Google Scholar]
  16. Phanich, M.; Pholkul, P.; Phimoltares, S. Food Recommendation System Using Clustering Analysis for Diabetic Patients. In Proceedings of the 2010 International Conference on Information Science and Applications, Seoul, Korea, 21–23 April 2010; pp. 1–8. [Google Scholar]
  17. Norouzi, S.; Ghalibaf, A.K.; Sistani, S.; Banazadeh, V.; Keykhaei, F.; Zareishargh, P.; Amiri, F.; Nematy, M.; Etminani, K. A Mobile Application for Managing Diabetic Patients’ Nutrition: A Food Recommender System. Arch. Iran. Med. 2018, 21, 466–472. [Google Scholar] [PubMed]
  18. Lee, C.S.; Wang, M.H.; Li, H.C.; Chen, W.H. Intelligent ontological agent for diabetic food recommendation. In Proceedings of the 2008 IEEE International Conference on Fuzzy Systems (IEEE World Congress on Computational Intelligence), Hong Kong, China, 1–6 June 2008; pp. 1803–1810. [Google Scholar]
  19. Mohammed, H.A.; Hagras, H. Towards Developing Type 2 Fuzzy Logic Diet Recommendation System for Diabetes. In Proceedings of the 2018 10th Computer Science and Electronic Engineering (CEEC), Colchester, UK, 19–21 September 2018; pp. 56–59. [Google Scholar]
  20. Abhari, S.; Safdari, R.; Azadbakht, L.; Lankarani, K.B.; Niakan Kalhori, S.R.; Honarvar, B.; Abhari, K.; Ayyoubzadeh, S.M.; Karbasi, Z.; Zakerabasali, S.; et al. A Systematic Review of Nutrition Recommendation Systems: With Focus on Technical Aspects. J. Biomed. Phys. Eng. 2019, 9, 591–602. [Google Scholar] [CrossRef]
  21. Ngo, P.D.; Tayefi, M.; Nordsletta, A.T.; Godtliebsen, F. Food Recommendation Using Machine Learning for Physical Activities in Patients with Type 1 Diabetes. In Proceedings of the 2019 Scandinavian Conference on Health Informatics (EHiN), Oslo, Norway, 12–13 November 2019; pp. 45–49. [Google Scholar]
  22. Crown, W.H. Potential application of machine learning in health outcomes research and some statistical cautions. Value Health 2015, 18, 137–140. [Google Scholar] [CrossRef]
  23. Othman, M.; Zain, N.M.; Muhamad, U.K. e-Diet Meal Recommender System for Diabetic Patients. In Proceedings of the Second International Conference on the Future of ASEAN (ICoFA) 2017, Kangar, Malaysia, 15–16 August 2017; Saian, R., Abbas, M.A., Eds.; Springer: Berlin, Germany, 2017; Volume 2, pp. 155–164. [Google Scholar]
  24. Xie, J.; Wang, Q. A personalized diet and exercise recommender system for type 1 diabetes self-management: An in silico study. Smart Health 2019, 13, 100069. [Google Scholar] [CrossRef]
  25. Dalla Man, C.; Breton, M.; Cobelli, C. Physical Activity into the Meal Glucose—Insulin Model of Type 1 Diabetes: In Silico Studies. J. Diabetes Sci. Technol. 2009, 3, 56–67. [Google Scholar] [CrossRef]
  26. Dalla Man, C.; Rizza, R.A.; Cobelli, C. Meal Simulation Model of the Glucose-Insulin System. IEEE Trans. Biomed. Eng. 2007, 54, 1740–1749. [Google Scholar] [CrossRef]
  27. Dalla Man, C.; Micheletto, F.; Lv, D.; Breton, M.; Kovatchev, B.; Cobelli, C. The UVA/PADOVA Type 1 Diabetes Simulator: New Features. J. Diabetes Sci. Technol. 2014, 8, 26–34. [Google Scholar]
  28. Piercy, K.L.; Troiano, R.P.; Ballard, R.M. The Physical Activity Guidelines for Americans. J. Am. Med. Assoc. (JAMA) 2018, 320, 2020–2028. [Google Scholar] [CrossRef] [PubMed]
  29. Bergman, R.N. Toward physiological understanding of glucose tolerance: Minimal-model approach. Diabetes 1989, 38, 1512–1527. [Google Scholar] [CrossRef] [PubMed]
  30. Hovorka, R.; Canonico, V.; Chassin, L.J.; Haueter, U.; Massi-Benedetti, M.; Federici, M.O.; Pieber, T.R.; Schaller, H.C.; Schaupp, L.; Vering, T.; et al. Nonlinear model predictive control of glucose concentration in subjects with type 1 diabetes. Physiol. Meas. 2004, 25, 905. [Google Scholar] [CrossRef]
  31. Kanderian, S.S.; Weinzimer, S.A.; Steil, G.M. The identifiable virtual patient model: Comparison of simulation and clinical closed-loop study results. J. Diabetes Sci. Technol. 2012, 6, 371–379. [Google Scholar] [CrossRef]
  32. Breton, M.; Kovatchev, B. Analysis, Modeling, and Simulation of the Accuracy of Continuous Glucose Sensors. J. Diabetes Sci. Technol. 2008, 2, 853–862. [Google Scholar] [CrossRef]
  33. Tejedor, M.; Myhre, J.N. Controlling Blood Glucose for Patients with Type 1 Diabetes Using Deep Reinforcement Learning—The Influence Of Changing The Reward Function. In Proceedings of the Northern Lights Deep Learning Workshop 2020, Tromso, Norway, 10 January 2020; Volume 1, pp. 1–6. [Google Scholar]
  34. Clarke, W.; Kovatchev, B. Statistical tools to analyze continuous glucose monitor data. Diabetes Technol. Ther. 2009, 11, S45–S54. [Google Scholar] [CrossRef]
  35. Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L.; et al. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32; Wallach, H., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E., Garnett, R., Eds.; Curran Associates, Inc.: Vancouver, BC, Canada, 2019; pp. 8024–8035. [Google Scholar]
  36. Wingate, D.; Weber, T. Automated Variational Inference in Probabilistic Programming. arXiv 2013, arXiv:1301.1299. [Google Scholar]
  37. Bingham, E.; Chen, J.P.; Jankowiak, M.; Obermeyer, F.; Pradhan, N.; Karaletsos, T.; Singh, R.; Szerlip, P.; Horsfall, P.; Goodman, N.D. Pyro: Deep Universal Probabilistic Programming. J. Mach. Learn. Res. 2019, 20, 973–978. [Google Scholar]
  38. Kullback, S.; Leibler, R.A. On Information and Sufficiency. Ann. Math. Stat. 1951, 22, 79–86. [Google Scholar] [CrossRef]
Figure 1. The asymmetric reward function. Low blood glucose levels (a) are more penalized than (b) high blood glucose levels.
Figure 1. The asymmetric reward function. Low blood glucose levels (a) are more penalized than (b) high blood glucose levels.
Applsci 10 08037 g001
Figure 2. Structure of the feedforward neural networks for predicting the blood glucose outcome and risks (a) blood glucose outcome (b) risks.
Figure 2. Structure of the feedforward neural networks for predicting the blood glucose outcome and risks (a) blood glucose outcome (b) risks.
Applsci 10 08037 g002
Figure 3. Training (solid blue line) and testing (orange dashed line) error without CGM measurement noise (a) and with CGM measurement noise (b).
Figure 3. Training (solid blue line) and testing (orange dashed line) error without CGM measurement noise (a) and with CGM measurement noise (b).
Applsci 10 08037 g003
Figure 4. Training data (dots and cross), predicted outcome and risks under no CGM noise (a), and CGM noise (b). Green and blue shaded uncertainty bands represent the 90% confident intervals for hypoglycemia and hyperglycemia risks respectively.
Figure 4. Training data (dots and cross), predicted outcome and risks under no CGM noise (a), and CGM noise (b). Green and blue shaded uncertainty bands represent the 90% confident intervals for hypoglycemia and hyperglycemia risks respectively.
Applsci 10 08037 g004
Figure 5. Blood glucose responses with no CGM noise in training data: (a) overtime as the effects of all possible actions (blue shade), safe actions (green shade), and optimal action (red line) (b) over the amount of carbohydrates at certain time points (red solid lines indicate safe actions).
Figure 5. Blood glucose responses with no CGM noise in training data: (a) overtime as the effects of all possible actions (blue shade), safe actions (green shade), and optimal action (red line) (b) over the amount of carbohydrates at certain time points (red solid lines indicate safe actions).
Applsci 10 08037 g005
Figure 6. Blood glucose responses with CGM noise in training data: (a) overtime as the effects of all possible actions (blue shade), safe actions (green shade), and optimal action (red line) (b) over the amount of carbohydrates at certain time points (red solid lines indicate safe actions).
Figure 6. Blood glucose responses with CGM noise in training data: (a) overtime as the effects of all possible actions (blue shade), safe actions (green shade), and optimal action (red line) (b) over the amount of carbohydrates at certain time points (red solid lines indicate safe actions).
Applsci 10 08037 g006
Table 1. Parameters for continuous glucose monitor (CGM) sensor noise extracted from [32].
Table 1. Parameters for continuous glucose monitor (CGM) sensor noise extracted from [32].
ParameterValue
ξ −5.471
λ 15.96
γ −0.5444
δ 1.6898
Table 2. Parameters of the reward function [33].
Table 2. Parameters of the reward function [33].
ParameterDescriptionValue
B G r e f Reference blood glucose108 mg/dL
B G h y p e r Hyperglycemia blood glucose180 mg/dL
B G h y p o Hypoglycemia blood glucose72 mg/dL
B G h y p o Severe hypoglycemia blood glucose54 mg/dL
Table 3. Evidence lower bound (ELBO) of the risk models for training and validation data.
Table 3. Evidence lower bound (ELBO) of the risk models for training and validation data.
RiskTrainingValidation
without Noisewith Noisewithout Noisewith Noise
Hyperglycemia0.41980.06320.53400.0307
Hypoglycemia−0.4650−0.7172−0.3112−1.0458
Table 4. Recommended actions (grams of carbohydrates).
Table 4. Recommended actions (grams of carbohydrates).
Action TypeNo Noisewith Noise
Safe lower26.334.3
Safe upper74.760.6
Optimal50.250.2
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Back to TopTop