Next Article in Journal
Behavioral, Emotional and Social Apathy in Alcohol-Related Cognitive Disorders
Next Article in Special Issue
The Application of Exercise Training for Diabetic Peripheral Neuropathy
Previous Article in Journal
Reference Intervals of Thyroid Function Tests Assessed by Immunoassay and Mass Spectrometry in Healthy Pregnant Women Living in Catalonia
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
JCM
Volume 10
Issue 11
10.3390/jcm10112445
jcm-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Time in Range for Closed-Loop Systems versus Standard of Care during Physical Exercise in People with Type 1 Diabetes: A Systematic Review and Meta-Analysis

by
Max L. Eckstein
1,
Benjamin Weilguni
2,
Martin Tauschmann
3,
Rebecca T. Zimmer
1,
Faisal Aziz
2,
Harald Sourij
2 and
Othmar Moser
1,2,*
1
Division of Exercise Physiology and Metabolism, Department of Sport Science, University of Bayreuth, 95440 Bayreuth, Germany
2
Interdisciplinary Metabolic Medicine, Division of Endocrinology and Diabetology, Department of Internal Medicine, Medical University of Graz, 8036 Graz, Austria
3
Department of Pediatrics and Adolescent Medicine, Medical University of Vienna, 1090 Vienna, Austria
*
Author to whom correspondence should be addressed.
J. Clin. Med. 2021, 10(11), 2445; https://doi.org/10.3390/jcm10112445
Submission received: 23 April 2021 / Revised: 26 May 2021 / Accepted: 27 May 2021 / Published: 31 May 2021
(This article belongs to the Special Issue Diabetes, Physical Activity and Exercise)
Browse Figures
Review Reports Versions Notes

Abstract

:
The aim of this systematic review and meta-analysis was to compare time in range (TIR) (70–180 mg/dL (3.9–10.0 mmol/L)) between fully closed-loop systems (CLS) and standard of care (including hybrid systems) during physical exercise in people with type 1 diabetes (T1D). A systematic literature search was conducted in EMBASE, PubMed, Cochrane Central Register of Controlled Trials, and ISI Web of Science from January 1950 until January 2020. Randomized controlled trials including studies with different CLS were compared against standard of care in people with T1D. The meta-analysis was performed using the random effects model and restricted maximum likelihood estimation method. Six randomized controlled trials involving 153 participants with T1D of all age groups were included. Due to crossover test designs, studies were included repeatedly (a–d) if CLS or physical exercise interventions were different. Applying this methodology increased the comparisons to a total number of 266 participants. TIR was higher with an absolute mean difference (AMD) of 6.18%, 95% CI: 1.99 to 10.38% in favor of CLS. In a subgroup analysis, the AMD was 9.46%, 95% CI: 2.48% to 16.45% in children and adolescents while the AMD for adults was 1.07% 95% CI: −0.81% to 2.96% in favor of CLS. In this systematic review and meta-analysis CLS moderately improved TIR in comparison to standard of care during physical exercise in people with T1D. This effect was particularly pronounced for children and adolescents showing that the use of CLS improved TIR significantly compared to standard of care.

1. Introduction

Achieving glycemic targets remains a major hurdle in people with type 1 diabetes (T1D). Less than 40% of patients have a so called “optimal glycemic control” despite recent advancements in glucose sensing technologies, insulin pumps, and insulins [1]. A recently published position statement highlighted advancements in diabetes technology around physical exercise to support both health care professionals (HCP) and people living with T1D in their therapy management [2]. This is not only of interest for adults with T1D, but also for children and adolescents. Physical activity and exercise are also recommended to be performed as often as possible since the health benefits of being physically active are well established for this young population [3,4,5].
Closed-loop systems (CLS), also called “artificial pancreas” systems, may facilitate diabetes management around physical exercise. These systems are able to automatically adjust exogenous insulin administration based on sensor glucose levels in line with dosing algorithms [6,7]. Those systems have already proven to be safe and efficient in several randomized controlled trials in adults [8], children and adolescents [9], during day- and nighttime [10,11], and in inpatient [12] and outpatient settings [13]. Additionally, CLS performed well during glycemic challenges such as meals or unannounced physical exercise [14]. The safety and efficacy under environmental conditions have also been confirmed in a recent meta-analysis, comparing CLS against standard of care, leading to 15.15% (95% CI 12.21% to 18.09%) more time spent in range (70–180 mg/dL (3.9–10.0 mmol/L)) when using a CLS [15].
In general, physical exercise is one of the biggest challenges for glycemic management in people with T1D [16]. Glucose levels may increase, stay stable, or decrease depending upon exercise type, duration, intensity, and experience [2]. In particular, children and adolescents require support during physical exercise since they are more prone to dysglycemia resulting in elevated risk of hyper- and hypoglycemia, especially during post-exercise nocturnal periods [17].
CLS have the potency for participation in physical exercise by self-learning algorithms to withhold or deliver micro-doses of insulin to maintain euglycemia. Yet, those systems rely on continuous glucose monitoring (CGM) technology, which has been shown to be less accurate during physical exercise and rapid glucose changes, where mean absolute relative differences of up to 26.5% were observed [18,19]. The physiological lag-time between interstitial fluid, in which the glucose sensor is placed, and capillary or venous blood remains the biggest challenge for CLS algorithms [20]. Furthermore, since exogenous insulin kinetics are only mimicking endogenous glucose lowering effects, CLS may struggle with timely reactions to hypo- and hyperglycemia, especially during physical exercise [21]. Even though faster acting bolus insulins are available now, two main differences to endogenous insulin are present: no first-pass through portal circulation and their subcutaneous absorption is significantly slower in comparison to endogenous insulin [22].
Consequently, it is of interest to assess if CLS are superior or even inferior to standard of care during physical exercise in people with T1D. Therefore, this systematic review and meta-analysis aimed to investigate the percentage of time in range (TIR) of CLS in comparison to standard of care during physical exercise in people with T1D.

2. Materials and Methods

This systematic review and meta-analysis is reported in accordance with the PRISMA guidelines and was registered at the International Prospective Register of Systematic Reviews (http://www.crd.york.ac.uk/prospero; CRD42929136669, 24th May 2019) [23].

2.1. Search Strategy and Selection Criteria

We selected relevant studies published from inception until the 21st of January 2020 by searching EMBASE, PubMed, Cochrane Central Register of Controlled Trials and ISI Web of Science. We used combined terms medical subject headings (MeSH) and text words (tw) to search relevant studies. Search terminology and strategy can be found in the supplemental material (Supplemental Material, Tables S1–S4). Potentially eligible studies were considered for review if the articles were written in English. Studies were excluded if time in range (70–180 mg/dL (3.9–10.0 mmol/L)) in people with T1D during physical exercise while on CLS or standard of care was not reported. Likewise, we excluded the studies if neither type nor duration of physical activity was defined.

2.2. Data Extraction and Quality Assessment

Two independent investigators (M.L.E. and B.W.) reviewed the study titles and abstracts. Studies that satisfied the inclusion criteria were then retrieved for full-text evaluation (Supplemental Material, Tables S1–S4). Disagreements were resolved by two other investigators (O.M., and H.S.). The following data were extracted from each selected study: study characteristics (author, publication year, journal country, city, continent, type of study), population (pathology, number of participants, age range, participants completed the study, gender, age of participants that completed the study, anthropometrics), outcome measures (device (CGM, pump), type of drug (if applicable), algorithm, time in range, dysglycemic events, number of experiments), and performance parameters (exercise type, duration, intensity, frequency). If data were missing, authors were contacted to receive data. In case of data unavailability, e.g., missing TIR or missing comparator (standard of care) these studies were excluded.
Studies were independently assessed by two investigators (M.L.E. and B.W.) for methodological quality using the risk of bias assessment (ROB) tool from the Cochrane Collaboration [24] in its revised version [25]. The following sources of bias were detected: overall bias, selection of the reported result, bias of the measurement of the outcome, missing outcome data, deviations from the intended interventions and randomization process (Figure 1). We did not exclude any studies based on the risk of bias assessment since included trials were judged as low risk of bias following the assessment.

2.3. Data Synthesis and Analysis

A narrative descriptive analysis was performed to summarize the characteristics of studies such as year of publication, trial design, sample size, country where the study was conducted, age of the participants, types of interventions, and duration of interventions. TIR was defined as 70–180 mg/dL (3.9–10.0 mmol/L) measured by interstitial CGM devices as described by Battelino et al. [26]. CLS were defined as non-adjustable insulin pump systems (single-hormone) or insulin and glucagon pump systems (dual-hormone) with glucose sensing technology and algorithm-based insulin delivery, which are considered as “fully closed-loop systems” within this systematic review and meta-analysis [27]. Adjustable dual-hormone systems were considered as CLS since sensor-augmented pump dual-hormone systems were not commercially available when performing the search.
Standard of care is defined as any kind of standard care provided to people with T1D. This also includes predictive low glucose suspend systems (PLGS), open-loop and hybrid CLS, since those systems were made available by the US Food and Drug Administration (FDA) almost two years ago and demand action by the individual if deemed necessary [28]. Where required, means and standard deviations were calculated using appropriate equations (SE = SD√n) [29]. If studies included more than one appropriate data set, these data were extracted and analyzed separately.

2.4. Meta-Analysis

The meta-analysis was performed using the random effects model and restricted maximum likelihood estimation method. The absolute mean difference with corresponding 95% CI was used to summarize the overall and within study difference in TIR between intervention and control groups. The positive mean difference indicated a higher TIR in the intervention group compared with the control group, while the negative mean difference indicated a higher TIR in the control group than the intervention group. The results of the meta-analysis were presented as forest plots.

3. Results

A total of six clinical trials were extracted from 1474 articles that met the objectives [30,31,32,33,34,35]. The steps of the article selection process are described as a flow diagram in Figure 2. Studies published between 2013 and 2019 enrolling a total of 153 participants were included. Due to the crossover designs of four clinical trials, the results have been split for meta-analyses [30,31,32,33]. Therefore, the total population increased to 266 participants and 12 comparisons. Four studies showed a low risk of bias [32,33,34,35], while two studies showed some risk of bias [30,31].
One study was conducted in a crossover trial and compared a single hormone CLS with both predictive low-glucose suspend systems and regular diabetes treatment, both considered as standard of care for this analysis [33]. One study compared a dual-hormone system with and without exercise announcement [31] vs sensor-augmented pump systems, leading to two comparisons. Another study compared single-hormone CLS with standard of care with different exercise types, leading to two comparisons [30]. One study compared a single and a dual-hormone CLS with the standard of care and PLGS, resulting in four comparisons [32].
All other included studies compared single hormone CLS versus standard of care leading to six comparisons [30,33,34,35]. All included studies compared both treatment options during physical exercise at a moderate intensity [31,32,33,34,35], while Dovc et al. also investigated the effects of moderate intensity exercise and sprints [30]. Time under exercise conditions in the studies ranged from 40 min [30] to 330 min [34]. Four studies were conducted under laboratory conditions with an exercise duration of ≤60 min [30,31,32,33]. Two studies were conducted under cold outdoor conditions, which may explain the increased duration of the physical exercise period [34,35]. Two studies conducted the exercise test on a cycle ergometer [30,33], two studies on a treadmill ergometer [31,32], and two studies outside while skiing [34,35]. Four studies were conducted in adolescents of age <18 years [30,33,34,35], while two studies were conducted in adults [31,32]. General study characteristics are summarized in Table 1.
TIR was higher with an absolute mean difference (AMD) of 6.18%, 95% CI: 1.99 to 10.38% in favor of CLS (Figure 3).

3.1. Subgroup Analysis

Subgroup analysis based on age showed the following parameters for studies conducted in adolescents (<18 years): mean difference 9.46% (95% CI −2.48% to 16.45% six study arms, I2 = 65%, τ2 = 51.43, p = 0.01) and for studies conducted in adults (>18 years): mean difference 1.07% (95% CI −0.81% to 2.96%); six study arms, I2 = 0%, τ2 = 1.15, p = 0.44 (Supplemental Material, Figure S1). For the cycle ergometer the mean difference in time in range was 12.34% (95% CI 4.09 to 20.58%; four study arms, I2 = 70%, τ2 = 53.58, p = 0.02), for the treadmill ergometer the mean difference was 1.07% (95% CI −0.81% to 2.96%; six study arms, I2 = 0%, τ2 = 1.15, p = 0.44) and for skiing the mean difference was 1.35% (95% CI −7.95% to 10.65%; two study arms, I2 = 0%, τ2 = 0, p = 0.09) (Supplemental Material, Figure S2). In subgroup analysis for study type, a mean difference for crossover randomized trials of 6.87% (95% CI 2.17 to 11.58; I2 = 85%, τ2 = 39.06, p < 0.01, ten studies) and a mean difference of 1.35% (95% CI −7.95% to 10.65%; I2 = 0%, τ2 = 0.0, p = 0.88, two studies) for studies of parallel design was shown (Supplemental Material, Figure S3).

3.2. Secondary Outcomes

The included studies reported only few data regarding our predefined secondary outcomes. The reported data are again given as percentage of total time in the given time period. Due to missing data points and the high variability of the available data, no statistical analyses were performed as very heterogeneous results were to be expected. Detailed data are shown in Supplemental Material Tables S5 and S6.
Reported data regarding time spent in hypoglycemia (<70 mg/dL; <3.9 mmol/L) during exercise showed values between the lowest of 0.0 (0.0-0.0) (no hypoglycemic event) reported in the intervention group in [30] a, up to a Mean of 8.3 ± 12.6% time in hypoglycemia in the intervention group by Castle et al. [32]. However, due to the lack of data no conclusions can be made whether the results favor the intervention or comparison group.
For five studies data were reported regarding the time spent in hyperglycemia (>180 mg/dL; >10 mmol/L) during exercise, showing large interquartile ranges and variances for all data points. In the intervention group in the study by Dovc et al. [30] a; the time in hyperglycemia was lowest at 17.1% (7.2–33.0) while in the comparative group in the study from Ekhlaspour et al. [35], it was highest at 41.5 ± 30.3% (Supplemental Material Table S5). The number of hypoglycemic events (<70 mg/dL; <3.9 mmol/L) during physical exercise was reported for the study by Dovc et al. [30] b and by Breton et al. [34]. In the study from Dovc et al. [30] b, one hypoglycemic event occurred during physical exercise in the intervention and four in the comparison group. For the study by Breton et al. [34], overall 0.3 ± 0.4 events in the intervention and 0.3 ± 0.7 in the comparison group were reported.
Five included studies reported data for time in range during the post-exercise period. In these studies, the evening or night after exercise was considered as the post-exercise period. Data in all five study arms favored the intervention group, however there were large interquartile ranges and variances. In study [34], time in range during the post-exercise period was 79.3% ± 29.8 in the intervention and 68.8% ± 24.1 in the comparison group. Five included studies reported data for time spent in hypoglycemia (<3.9 mmol/L) during the post-exercise period. All data points are close to 0.0 with the highest values reported in study [34] with 2.2% ± 2.3 in the intervention group and 2.5% ± 6.5 in the comparison group for time in hypoglycemia.
Three included studies reported data for the time spent in hyperglycemia (>10 mmol/L) during the post-exercise period. In these studies the intervention group showed less time in hyperglycemia with the biggest difference in study [33] a, with 1.0% (0.0 to 13.9%) in the intervention group and 64.6% (1.6 to 97.5%) in the comparison group.
The number of hypoglycemic events during the post-exercise period was reported in two included studies. In study [30] a, three hypoglycemic events occurred in the intervention and in the comparison group. Breton et al. reported a number of hypoglycemic events in the intervention group with 0.1 ± 0.3 and 0.1 ± 0.4 in the comparison group during the post-exercise period [34].

3.3. Assessment of Publication Bias

The publication was assessed in terms of the meta-bias. Specifically, a funnel plot was generated to visualize the publication bias and Egger’s test was performed to assess the asymmetry of the data and the impact of publication bias. The corresponding p-values <0.05 would indicate a significant publication bias (Supplemental Material Figure S4). The funnel plot indicates moderate asymmetry (p = 0.13), suggesting that publication bias cannot be excluded as a factor of influence on the present meta-analysis (Supplemental Material Figure S4). It remains possible that due to the lack of studies in this specific research field, studies with smaller samples have not been published or failed to do so.

4. Discussion

Our meta-analysis indicated that CLS have superior effects on the TIR during physical exercise in people with T1D when compared against standard of care. Furthermore, in a subgroup analysis for age, CLS were especially favorable in children and adolescents, detailing 9.46% more time spent in range when compared to standard of care. This finding contrasted with the results found in adults where only a 1.07% improvement in TIR was observed. These results may potentially be due to specific alterations in the designs of the studies (e.g., type of exercise or set-up). Intriguingly, one might expect less beneficial effects of CLS in the younger population due to elevated glycemic variability [36] that may affect exogenous insulin requirements and deteriorate CGM accuracy, which is seen during moderate-intensity exercise [2,37,38]. Two recent position statements gave recommendations on how to manage glucose around physical exercise according to values given by intermittently scanned glucose monitoring systems (isCGM) and CGM sensors [2,3]. These statements have also shown a Mean Absolute Real Difference (MARD) of 13.6% over all systems that may deteriorate the performance of CLS. Independent of how good an algorithm is, sensor accuracy is reduced due to the physiological lag time between blood and interstitial glucose during acute physical exercise, and therefore remains the biggest issue in maintaining euglycemia [39]. This is partly supported by our analysis (Table S5) since both CLS and standard of care lead to comparably lower time in hypoglycemia, whereas time in hyperglycemia is higher during standard of care. However, these findings must be interpreted cautiously since most studies included in our analysis did not show these data.
Physical exercise performed on a cycle ergometer showed beneficial effects of CLS on TIR when compared against standard of care. In contrast, this was not seen when the exercise was performed on a treadmill ergometer or during a skiing session. The metabolic effects of exercise are often generated based on the absolute amount of muscle mass used [40]. Taking this into account and hypothesized in a previous letter [41], treadmill exercise and skiing may yield a greater rate of change in glucose, hence deteriorating CLS performance.
For most people with T1D, scheduled physical exercise is favorable to spontaneous exercise since insulin dosing can be adjusted prior to exercise and additional carbohydrates can be consumed if necessary. However, the glycemic management in the post-exercise phase has proven to be difficult for people with T1D. Due to increased glucose transporter type -4 activity, glucose uptake is increased for up to 24 h following exercise [16], which may induce hypoglycemia. On the other hand, fear of hypoglycemia and excessive carbohydrate consumption during exercise may lead to hyperglycemia, which deteriorates HbA1c over extended periods of time [42,43]. This may also explain our findings in Table S6. Time in range appears to be higher in the post-exercise period compared to standard of care; however, we have not conducted formal analysis given that this outcome is available in only a few studies. Notably, time in hypoglycemia appears similar with a maximum of 2.2 ± 2.3% in CLS and 2.5 ± 6.5% during standard of care [34]. Hypoglycemic events were also comparable, with a total of three in CLS and four in standard of care [30] (Table S6).
A recent study conducted by Viñals et al. investigated the effects of the performance of a multivariable CLS controller with automatic carbohydrate recommendation during and after announced and unannounced physical exercise in people with T1D [44]. It was shown that CLS in both conditions increased the time in range, avoided hypoglycemia and reduced time spent in hyperglycemia during post-exercise conditions [44], which is in line with the findings from our meta-analysis. A previous systematic review and meta-analysis investigated the efficacy and safety of CLS in 2017 [15]. This analysis concluded that the devices are safe and efficacious in treating outpatients with T1D. A more recent network meta-analysis had similar findings by underpinning that CLS led to a lower median glucose of 0.75 mmol/L compared with open-loop systems. They concluded that the DiA platform was superior to other platforms with an advantage for dual-hormone systems to lower mean glucose and increase time in range [45]. Due to the lack of studies applying dual-hormone systems during physical exercise, a comparison within our meta-analysis was not possible.
CLS have raised awareness in the population of people with T1D using continuous subcutaneous insulin infusion systems, which led to the popularity of do-it-yourself CLS that have become present within a small community of people with T1D. However, this still remains an ethical issue for health-care professionals since CLS, even though successfully tested for research purposes over decades, are not yet commercially available [46]. However, the complexity of implementing these devices in glycemic therapy and making them commercially available is reliant on research. The potential of CLS to ameliorate exercise management in T1D has previously been discussed in a narrative review by Bally and Thabit, which is in line with our findings [47] We suspect that due to the velocity at which technological innovations for CLS are released, running research is already outdated prior to publication, which in turn hinders the market release of those devices.
Our meta-analysis has the same limitations as other previously published meta-analyses as the inconsistency in the outcome reporting does not allow a clear conclusion. A further limitation in the interpretation of the results is the complexity of physical exercise since time, intensity, duration and study setting were influencing the results, which might have led to an increased reporting bias. Therefore, more studies in inpatient and outpatient settings with larger sample sizes are needed to overcome this bias in outcome reporting.

5. Conclusions

Our findings highlight that CLS lead to a greater time in range during physical exercise when compared against standard of care in particular in children and adolescents. CLS ameliorates glycemic control during exercise in this cohort, may reduce parental fear due to their unique properties and allow more spontaneous physical activity during childhood [48].
The future of glycemic management in people with T1D might be based upon the development of commercially available CLS. These systems should thus be regarded as a safe and efficacious method to maintain glucose control around physical exercise.

Supplementary Materials

The following are available online at https://www.mdpi.com/article/10.3390/jcm10112445/s1, Figure S1: Subgroup analysis on age; Figure S2: Subgroup analysis on type of exercise; Figure S3: Subgroup analysis on study type; Figure S4: Egger’s test: p-value = 0.13; Table S1: Search Pubmed, January 2020; Table S2: Search Embase, January 2020; Table S3: Search ISY Web of Science, January 2020; Table S4: Search Cochrane Controlled register of Trials (CENTRAL); Table S5: Secondary outcomes> time in percentage below and above target range during exercise and events <3.9 mmol/L; Table S6: Secondary outcomes: time in percentage during the post-exercise period including number of events <3.9 mmol/L.

Author Contributions

M.L.E., B.W. and O.M. conceived the systematic review and analysis plan. M.L.E. and B.W. selected studies for inclusion and abstracted data. F.A., O.M. performed the statistical analysis. M.L.E., B.W., M.T., R.T.Z. and H.S. interpreted the data. All authors wrote the first draft, critically revised the manuscript for important intellectual content and approved the final draft. O.M. is guarantor of this work. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Ethical review and approval were waived for this review, since it is an analysis of already published data.

Informed Consent Statement

Not applicable.

Data Availability Statement

Data may be obtained from the corresponding author upon reasonable request.

Conflicts of Interest

M.L.E. has received a KESS2/European Social Fund scholarship and travel grants from Novo Nordisk A/S and Sanofi-Aventis. O.M. has received lecture fees from Medtronic, travel grants from Novo Nordisk A/S, Novo Nordisk AT, Novo Nordisk UK, Medtronic AT, Sanofi-Aventis, research grants from Sêr Cymru II COFUND fellowship/European Union, Novo Nordisk A/S and Novo Nordisk AT as well as material funding from Abbott Diabetes Care. H.S. has received honoraria, travel support or unrestricted research grants by Amgen, Astra Zeneca, Boehringer-Ingelheim, Eli Lilly, MSD, Novo Nordisk and Sanofi-Aventis. All other authors have no conflict of interest relevant to the article to declare.

References

  1. Shah, V.N.; Grimsmann, J.M.; Foster, N.C.; Dost, A.; Miller, K.M.; Pavel, M.; Weinstock, R.S.; Karges, W.; Maahs, D.M.; Holl, R.W. Undertreatment of cardiovascular risk factors in the type 1 diabetes exchange clinic network (United States) and the prospective diabetes follow-up (Germany/Austria) registries. Diabetes Obes. Metab. 2020, 22, 1577–1585. [Google Scholar] [CrossRef]
  2. Moser, O.; Riddell, M.C.; Eckstein, M.L.; Adolfsson, P.; Rabasa-Lhoret, R.; van den Boom, L.; Gillard, P.; Nørgaard, K.; Oliver, N.S.; Zaharieva, D.P.; et al. Glucose management for exercise using continuous glucose monitoring (CGM) and intermittently scanned CGM (isCGM) systems in type 1 diabetes: Position statement of the European Association for the Study of Diabetes (EASD) and of the International Society for Pediatric and Adolescent Diabetes (ISPAD) endorsed by JDRF and supported by the American Diabetes Association (ADA). Diabetologia 2020, 63, 2501–2520. [Google Scholar] [CrossRef] [PubMed]
  3. Moser, O.; Riddell, M.C.; Eckstein, M.L.; Adolfsson, P.; Rabasa-Lhoret, R.; van den Boom, L.; Gillard, P.; Nørgaard, K.; Oliver, N.S.; Zaharieva, D.P.; et al. ISPAD GUIDELINES Glucose management for exercise using continuous glucose monitoring (CGM) and intermittently scanned CGM (isCGM) systems in type 1 diabetes: Position statement of the European Association for the Study of Diabetes (EASD) and of the International Society for Pediatric and Adolescent Diabetes (ISPAD) endorsed by JDRF and supported by the American Diabetes Association (ADA). Pediatr. Diabetes 2020, 21, 1375–1393. [Google Scholar] [CrossRef] [PubMed]
  4. Sherr, J.; Hermann, J.M.; Campbell, F.; Foster, N.C.; Hofer, S.E.; Allgrove, J.; Maahs, D.M.; Kapellen, T.M.; Holman, N.; Tamborlane, W.V.; et al. Use of insulin pump therapy in children and adolescents with type 1 diabetes and its impact on metabolic control: Comparison of results from three large, transatlantic paediatric registries. Diabetologia 2016, 59, 87–91. [Google Scholar] [CrossRef] [PubMed]
  5. Phillip, M.; Battelino, T.; Rodriguez, H.; Danne, T.; Kaufman, F.; for the Consensus Forum Participants. Use of Insulin Pump Therapy in the Pediatric Age-Group: Consensus statement from the European Society for Paediatric Endocrinology, the Lawson Wilkins Pediatric Endocrine Society, and the International Society for Pediatric and Adolescent Diabetes, endorsed by the American Diabetes Association and the European Association for the Study of Diabetes. Diabetes Care 2007, 30, 1653–1662. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  6. Peyser, T.; Dassau, E.; Breton, M.; Skyler, J.S. The artificial pancreas: Current status and future prospects in the management of diabetes. Ann. N. Y. Acad. Sci. 2014, 1311, 102–123. [Google Scholar] [CrossRef] [PubMed]
  7. Kovatchev, B. The year of transition from research to clinical practice. Nat. Rev. Endocrinol. 2017, 14, 74–76. [Google Scholar] [CrossRef]
  8. Breton, M.; Farret, A.; Bruttomesso, D.; Anderson, S.; Magni, L.; Patek, S.; Man, C.D.; Place, J.; DeMartini, S.; Del Favero, S.; et al. Fully Integrated Artificial Pancreas in Type 1 Diabetes: Modular Closed-Loop Glucose Control Maintains Near Normoglycemia. Diabetes 2012, 61, 2230–2237. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  9. Haidar, A.; Legault, L.; Matteau-Pelletier, L.; Messier, V.; Dallaire, M.; Ladouceur, M.; Rabasa-Lhoret, R. Outpatient overnight glucose control with dual-hormone artificial pancreas, single-hormone artificial pancreas, or conventional insulin pump therapy in children and adolescents with type 1 diabetes: An open-label, randomised controlled trial. Lancet Diabetes Endocrinol. 2015, 3, 595–604. [Google Scholar] [CrossRef]
  10. Tauschmann, M.; Allen, J.M.; Wilinska, M.E.; Thabit, H.; Stewart, Z.; Cheng, P.; Kollman, C.; Acerini, C.L.; Dunger, D.B.; Hovorka, R. Day-and-Night Hybrid Closed-Loop Insulin Delivery in Adolescents with Type 1 Diabetes: A Free-Living, Randomized Clinical Trial. Diabetes Care 2016, 39, 1168–1174. [Google Scholar] [CrossRef] [Green Version]
  11. Weinzimer, S.A.; Steil, G.M.; Swan, K.L.; Dziura, J.; Kurtz, N.; Tamborlane, W.V. Fully Automated Closed-Loop Insulin Delivery Versus Semiautomated Hybrid Control in Pediatric Patients with Type 1 Diabetes Using an Artificial Pancreas. Diabetes Care 2008, 31, 934–939. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  12. Forlenza, G.P.; Cameron, F.M.; Ly, T.T.; Lam, D.; Howsmon, D.P.; Baysal, N.; Kulina, G.; Messer, L.; Clinton, P.; Levister, C.; et al. Fully Closed-Loop Multiple Model Probabilistic Predictive Controller Artificial Pancreas Performance in Adolescents and Adults in a Supervised Hotel Setting. Diabetes Technol. Ther. 2018, 20, 335–343. [Google Scholar] [CrossRef] [PubMed]
  13. Renard, E.; Farret, A.; Kropff, J.; Bruttomesso, D.; Messori, M.; Place, J.; Visentin, R.; Calore, R.; Toffanin, C.; Di Palma, F.; et al. Day-and-Night Closed-Loop Glucose Control in Patients with Type 1 Diabetes Under Free-Living Conditions: Results of a Single-Arm 1-Month Experience Compared With a Previously Reported Feasibility Study of Evening and Night at Home. Diabetes Care 2016, 39, 1151–1160. [Google Scholar] [CrossRef] [Green Version]
  14. Huyett, L.M.; Ly, T.T.; Forlenza, G.P.; Reuschel-Di-Virgilio, S.; Messer, L.H.; Wadwa, R.P.; Gondhalekar, R.; Doyle, F.J.; Pinsker, J.; Maahs, D.M.; et al. Outpatient Closed-Loop Control with Unannounced Moderate Exercise in Adolescents Using Zone Model Predictive Control. Diabetes Technol. Ther. 2017, 19, 331–339. [Google Scholar] [CrossRef] [PubMed]
  15. Bekiari, E.; Kitsios, K.; Thabit, H.; Tauschmann, M.; Athanasiadou, E.; Karagiannis, T.; Haidich, A.-B.; Hovorka, R.; Tsapas, A. Artificial pancreas treatment for outpatients with type 1 diabetes: Systematic review and meta-analysis. BMJ 2018, 361, k1310. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  16. Moser, O.; Eckstein, M.L.; West, D.J.; Goswami, N.; Sourij, H.; Hofmann, P. Type 1 Diabetes and Physical Exercise: Moving (forward) as an Adjuvant Therapy. Curr. Pharm. Des. 2020, 26, 946–957. [Google Scholar] [CrossRef] [PubMed]
  17. Adolfsson, P.; Riddell, M.C.; Taplin, C.E.; Davis, E.A.; Fournier, P.A.; Annan, F.; Scaramuzza, A.E.; Hasnani, D.; Hofer, S.E. ISPAD Clinical Practice Consensus Guidelines 2018 Compendium Exercise in children and adolescents with diabetes ISPAD CLINICAL PRACTICE CONSENSUS GUIDELINES ISPAD Clinical Practice Consensus Guidelines 2018: Exercise in children and adolescents with Diabetes. Pediatr. Diabetes 2018, 19, 205–226. [Google Scholar] [CrossRef]
  18. Aberer, F.; Hajnsek, M.; Rumpler, M.; Zenz, S.; Baumann, P.M.; Elsayed, H.; Puffing, A.; Treiber, G.; Pieber, T.R.; Sourij, H.; et al. Evaluation of subcutaneous glucose monitoring systems under routine environmental conditions in patients with type 1 diabetes. Diabetes Obes. Metab. 2017, 19, 1051–1055. [Google Scholar] [CrossRef] [Green Version]
  19. Moser, O.; Mader, J.K.; Tschakert, G.; Mueller, A.; Groeschl, W.; Pieber, T.R.; Koehler, G.; Messerschmidt, J.; Hofmann, P. Accuracy of Continuous Glucose Monitoring (CGM) during Continuous and High-Intensity Interval Exercise in Patients with Type 1 Diabetes Mellitus. Nutrients 2016, 8, 489. [Google Scholar] [CrossRef] [Green Version]
  20. Majeed, W.; Thabit, H. Closed-loop insulin delivery: Current status of diabetes technologies and future prospects. Expert Rev. Med. Devices 2018, 15, 579–590. [Google Scholar] [CrossRef] [PubMed]
  21. Moser, O.; Yardley, J.E.; Bracken, R.M. Interstitial Glucose and Physical Exercise in Type 1 Diabetes: Integrative Physiology, Technology, and the Gap In-Between. Nutrients 2018, 10, 93. [Google Scholar] [CrossRef] [Green Version]
  22. Evans, M.; Wilkinson, M.; Giannpolou, A. Fast-Acting Insulin Aspart: The Rationale for a New Mealtime Insulin. Diabetes Ther. 2019, 10, 1793–1800. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  23. Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G.; The PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med. 2009, 6, e1000097. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  24. Higgins, J.P.T.; Altman, D.G.; Gøtzsche, P.C.; Jüni, P.; Moher, D.; Oxman, A.D.; Savović, J.; Schulz, K.F.; Weeks, L.; Sterne, J.A.C.; et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 2011, 343, d5928. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  25. Sterne, J.A.C.; Savović, J.; Page, M.J.; Elbers, R.G.; Blencowe, N.S.; Boutron, I.; Cates, C.J.; Cheng, H.-Y.; Corbett, M.S.; Eldridge, S.M.; et al. RoB 2: A revised tool for assessing risk of bias in randomised trials. BMJ 2019, 366, l4898. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  26. Battelino, T.; Danne, T.; Bergenstal, R.M.; Amiel, S.A.; Beck, R.; Biester, T.; Bosi, E.; Buckingham, B.A.; Cefalu, W.T.; Close, K.L.; et al. Clinical Targets for Continuous Glucose Monitoring Data Interpretation: Recommendations from the International Consensus on Time in Range. Diabetes Care 2019, 42, 1593–1603. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  27. Steil, G.M.; Rebrin, K.; Darwin, C.; Hariri, F.; Saad, M.F. Feasibility of Automating Insulin Delivery for the Treatment of Type 1 Diabetes. Diabetes 2006, 55, 3344–3350. [Google Scholar] [CrossRef] [Green Version]
  28. Minimed 670G System—P160017/S031 | FDA. Available online: https://www.fda.gov/medical-devices/recently-approved-devices/minimed-670g-system-p160017s031 (accessed on 19 March 2020).
  29. Hozo, S.P.; Djulbegovic, B.; Hozo, I. Estimating the mean and variance from the median, range, and the size of a sample. BMC Med. Res. Methodol. 2005, 5, 13. [Google Scholar] [CrossRef] [Green Version]
  30. Dovc, K.; MacEdoni, M.; Phillip, M.; Battelino, T.; Bratina, N.; Lepej, D.; Nimri, R.; Atlas, E.; Muller, I.; Kordonouri, O.; et al. Closed-loop glucose control in young people with type 1 diabetes during and after unannounced physical activity: A randomised controlled crossover trial. Diabetologia 2017, 60, 2157–2167. [Google Scholar] [CrossRef]
  31. Jacobs, P.G.; El Youssef, J.; Reddy, R.; Resalat, N.; Branigan, D.; Condon, J.; Preiser, N.; Ramsey, K.; Jones, M.; Edwards, C.; et al. Randomized trial of a dual-hormone artificial pancreas with dosing adjustment during exercise compared with no adjustment and sensor-augmented pump therapy. Diabetes Obes. Metab. 2016, 18, 1110–1119. [Google Scholar] [CrossRef]
  32. Castle, J.R.; El Youssef, J.; Wilson, L.M.; Reddy, R.; Resalat, N.; Branigan, D.; Ramsey, K.; Leitschuh, J.; Rajhbeharrysingh, U.; Senf, B.; et al. Randomized Outpatient Trial of Single- and Dual-Hormone Closed-Loop Systems That Adapt to Exercise Using Wearable Sensors. Diabetes Care 2018, 41, 1471–1477. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  33. Elleri, D.; Allen, J.M.; Kumareswaran, K.; Leelarathna, L.; Nodale, M.; Caldwell, K.; Cheng, P.; Kollman, C.; Haidar, A.; Murphy, H.R.; et al. Closed-loop basal insulin delivery over 36 hours in adolescents with type 1 diabetes: Randomized clinical trial. Diabetes Care 2013, 36, 838–844. [Google Scholar] [CrossRef] [Green Version]
  34. Breton, M.D.; Cherñavvsky, D.R.; Forlenza, G.P.; De Boer, M.D.; Robic, J.; Wadwa, R.P.; Messer, L.H.; Kovatchev, B.P.; Maahs, D.M. Closed-Loop Control During Intense Prolonged Outdoor Exercise in Adolescents with Type 1 Diabetes: The Artificial Pancreas Ski Study. Diabetes Care 2017, 40, 1644–1650. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  35. Ekhlaspour, L.; Forlenza, G.P.; Chernavvsky, D.; Maahs, D.M.; Wadwa, R.P.; DeBoer, M.D.; Messer, L.H.; Town, M.; Rn, J.P.; Kruse, G.; et al. Closed loop control in adolescents and children during winter sports: Use of the Tandem Control-IQ AP system. Pediatr. Diabetes 2019, 20, 759–768. [Google Scholar] [CrossRef]
  36. Schreiver, C.; Jacoby, U.; Watzer, B.; Thomas, A.; Haffner, D.; Fischer, D.-C. Glycaemic variability in paediatric patients with type 1 diabetes on continuous subcutaneous insulin infusion (CSII) or multiple daily injections (MDI): A cross-sectional cohort study. Clin. Endocrinol. 2013, 79, 641–647. [Google Scholar] [CrossRef]
  37. Moser, O.; Eckstein, M.L.; McCarthy, O.; Deere, R.; Pitt, J.; Williams, D.M.; Hayes, J.; Sourij, H.; Bain, S.C.; Bracken, R.M. Performance of the Freestyle Libre flash glucose monitoring (flash GM) system in individuals with type 1 diabetes: A secondary outcome analysis of a randomized crossover trial. Diabetes Obes. Metab. 2019, 21, 2505–2512. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  38. Moser, O.; Eckstein, M.L.; Mueller, A.; Birnbaumer, P.; Aberer, F.; Koehler, G.; Sourij, C.; Kojzar, H.; Holler, P.; Simi, H.; et al. Impact of physical exercise on sensor performance of the FreeStyle Libre intermittently viewed continuous glucose monitoring system in people with Type 1 diabetes: A randomized crossover trial. Diabet. Med. 2019, 36, 606–611. [Google Scholar] [CrossRef]
  39. Zaharieva, D.P.; Turksoy, K.; McGaugh, S.M.; Pooni, R.; Vienneau, T.; Ly, T.; Riddell, M.C. Lag Time Remains with Newer Real-Time Continuous Glucose Monitoring Technology During Aerobic Exercise in Adults Living with Type 1 Diabetes. Diabetes Technol. Ther. 2019, 21, 313–321. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  40. Spendier, F.; Müller, A.; Korinek, M.; Hofmann, P. Intensity Thresholds and Maximal Lactate Steady State in Small Muscle Group Exercise. Sports 2020, 8, 77. [Google Scholar] [CrossRef]
  41. Moser, O.; Riddell, M.C.; Eckstein, M.L.; Adolfsson, P.; Rabasa-Lhoret, R.; Boom, L.V.D.; Gillard, P.; Nørgaard, K.; Oliver, N.S.; Zaharieva, D.P.; et al. Glucose management for exercise using continuous glucose monitoring: Should sex and prandial state be additional considerations? Reply to Yardley JE and Sigal RJ [letter]. Diabetology 2021, 64, 935–938. [Google Scholar] [CrossRef]
  42. Fahey, A.J.; Paramalingam, N.; Davey, R.J.; Davis, E.A.; Jones, T.W.; Fournier, P.A. The Effect of a Short Sprint on Postexercise Whole-Body Glucose Production and Utilization Rates in Individuals with Type 1 Diabetes Mellitus. J. Clin. Endocrinol. Metab. 2012, 97, 4193–4200. [Google Scholar] [CrossRef] [Green Version]
  43. García-García, F.; Kumareswaran, K.; Hovorka, R.; Hernando, M.E. Quantifying the Acute Changes in Glucose with Exercise in Type 1 Diabetes: A Systematic Review and Meta-Analysis. Sports Med. 2015, 45, 587–599. [Google Scholar] [CrossRef] [Green Version]
  44. Viñals, C.; Beneyto, A.; Martín-SanJosé, J.-F.; Furió-Novejarque, C.; Bertachi, A.; Bondia, J.; Vehi, J.; Conget, I.; Giménez, M. Artificial Pancreas with Carbohydrate Suggestion Performance for Unannounced and Announced Exercise in Type 1 Diabetes. J. Clin. Endocrinol. Metab. 2021, 106, 55–63. [Google Scholar] [CrossRef] [PubMed]
  45. Karageorgiou, V.; Papaioannou, T.G.; Bellos, I.; Alexandraki, K.; Tentolouris, N.; Stefanadis, C.; Chrousos, G.P.; Tousoulis, D. Effectiveness of artificial pancreas in the non-adult population: A systematic review and network meta-analysis. Metabolism 2019, 90, 20–30. [Google Scholar] [CrossRef] [PubMed]
  46. Shaw, D.; Crabtree, T.S.J.; Hammond, P.; McLay, A.; Wilmot, E.G. The DIY artificial pancreas system: An ethical dilemma for doctors. Diabet. Med. 2020, 37, 1951–1953. [Google Scholar] [CrossRef] [PubMed]
  47. Bally, L.; Thabit, H. Closing the Loop on Exercise in Type 1 Diabetes. Curr. Diabetes Rev. 2018, 14, 257–265. [Google Scholar] [CrossRef]
  48. Quirk, H.; Blake, H.; Dee, B.; Glazebrook, C. “You can’t just jump on a bike and go”: A qualitative study exploring parents’ perceptions of physical activity in children with type 1 diabetes. BMC Pediatr. 2014, 14, 1–12. [Google Scholar] [CrossRef] [PubMed]
Jcm 10 02445 g001 550
Figure 1. Risk of bias assessment tool. Across trials (n = 6). Information is either from trials at low risk of bias (green), or from trials with some concerns of bias (yellow), or from trials at high risk of bias (red).
Figure 1. Risk of bias assessment tool. Across trials (n = 6). Information is either from trials at low risk of bias (green), or from trials with some concerns of bias (yellow), or from trials at high risk of bias (red).
Jcm 10 02445 g001
Jcm 10 02445 g002 550
Figure 2. Prisma flow diagram (accessed on, 27th of July 2020).
Figure 2. Prisma flow diagram (accessed on, 27th of July 2020).
Jcm 10 02445 g002
Jcm 10 02445 g003 550
Figure 3. Forest plot of meta-analysis showing the overall effect of control vs. CLS for time in range during physical exercise.
Figure 3. Forest plot of meta-analysis showing the overall effect of control vs. CLS for time in range during physical exercise.
Jcm 10 02445 g003
Table
Table 1. Generals study characteristics.
Table 1. Generals study characteristics.
YearStudy TypenAge (years)% TIR CLS % TIR Control Device InterventionAlgorithmDevice ControlExercise
Type		Exercise DurationExercise intensity
[33] (a)2013Crossover RCT1215.0 ± 1.479 (54–99)56 (24–70)Animas 2020 Dexcom Seven Plus CGMmodel predictive controlAnimas 2020 Pump, Dexcom Seven Plus CGMCycling60 minModerate-intensity
[33] (b)2013Crossover RCT1215.0 ± 1.460 (54–88)71 (18–95)Animas 2020 Dexcom Seven Plus CGMmodel predictive controlAnimas 2020 Pump, Dexcom Seven Plus CGMCycling60 minModerate-intensity
[31] (a)2016Crossover RCT2132 ± 767 (60–75)68 (60–76)Tandem t:slim Dexcom G4 CGM (dual hormone)Proportional-integral-derivativeSensor augmented pump;
Dexcom G4 CGM		Running45 minModerate-intensity
[31] (b)2016Crossover RCT2132 ± 772 (66–78)68 (60–76)Tandem t:slim Dexcom G4 CGM (dual hormone)Proportional-integral-derivativeSensor augmented pump;
Dexcom G4 CGM		Running45 minModerate-intensity
[30] (a)2017Crossover RCT2014.2 ± 2.080.9
(64.3–92.2)		68.1
(59.1–83.6)		Paradigm Veo Enlite II sensorfuzzy-logicMedtronic Paradigm Veo;
Enlite II sensor		Cycling40 minModerate-intensity
[30] (b)2017Crossover RCT2014.2 ± 2.075.5
(66.6–92.9)		68.4
(52.1–77.2)		Paradigm Veo Enlite II sensorfuzzy-logicMedtronic Paradigm Veo;
Enlite II sensor		Cycling40 minModerate-intensity + Sprint
[34]2017RCT3213.2 ± 1.763.2 ± 31.162.8 ± 31.4Tandem t:AP pump/ Roche-Accu-Check Spirit Combo pump Dexcom G4 CGMDiASensor-augmented pump;
Dexcom G4 CGM		Skiing330 minModerate-intensity
[32] (a)2018Crossover RCT2034.5 ± 4.784.3 ± 16.778.2 ± 26.2Tandem t:slim pump Dexcom G5 CGM (dual hormone)Fading memory proportional-derivative“standard of care”Running45 minModerate-intensity
[32] (b)2018Crossover RCT2034.5 ± 4.783.3 ± 16.778.2 ± 26.2Tandem t:slim pump Dexcom G5 CGM (single hormone)Fading memory proportional-derivative“standard of care”Running45 minModerate-intensity
[32] (c)2018Crossover RCT2034.5 ± 4.784.3 ± 16.778.3 ± 18.9Tandem t:slim pump Dexcom G5 CGM (dual hormone)Fading memory proportional-derivativePredictive low glucose suspend: t:slim pump,
Dexcom G5 CGM		Running45 minModerate-intensity
[32] (d)2018Crossover RCT2034.5 ± 4.783.3 ± 16.778.3 ± 18.9Tandem t:slim pump Dexcom G5 CGM (single hormone)Fading memory proportional-derivativePredictive low glucose suspend: t:slim pump,
Dexcom G5 CGM		Running45 minModerate-intensity
[35]2019RCT4812.3 ± 3.257.8 ± 27.355.9 ± 31.1Tandem t:slimX2 Dexcom G6 CGMDiASensor-augmented pump;
Dexcom G5 CGM		Skiing240 minModerate-intensity
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Eckstein, M.L.; Weilguni, B.; Tauschmann, M.; Zimmer, R.T.; Aziz, F.; Sourij, H.; Moser, O. Time in Range for Closed-Loop Systems versus Standard of Care during Physical Exercise in People with Type 1 Diabetes: A Systematic Review and Meta-Analysis. J. Clin. Med. 2021, 10, 2445. https://doi.org/10.3390/jcm10112445

AMA Style

Eckstein ML, Weilguni B, Tauschmann M, Zimmer RT, Aziz F, Sourij H, Moser O. Time in Range for Closed-Loop Systems versus Standard of Care during Physical Exercise in People with Type 1 Diabetes: A Systematic Review and Meta-Analysis. Journal of Clinical Medicine. 2021; 10(11):2445. https://doi.org/10.3390/jcm10112445

Chicago/Turabian Style

Eckstein, Max L., Benjamin Weilguni, Martin Tauschmann, Rebecca T. Zimmer, Faisal Aziz, Harald Sourij, and Othmar Moser. 2021. "Time in Range for Closed-Loop Systems versus Standard of Care during Physical Exercise in People with Type 1 Diabetes: A Systematic Review and Meta-Analysis" Journal of Clinical Medicine 10, no. 11: 2445. https://doi.org/10.3390/jcm10112445

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop