# EPO Dosage Optimization for Anemia Management: Stochastic Control under Uncertainty Using Conditional Value at Risk

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Hemoglobin Response Modeling

## 3. Deterministic Control Formulation: Setpoint MPC and Zone-MPC

## 4. Conditional Value at Risk

## 5. Stochastic Control Using CVaR Constraints

## 6. Stochastic Control Using a CVaR Cost Function

## 7. Computer Simulation Results

#### 7.1. Test under An ARX Model Based Simulator

#### 7.2. Test under PK/PD Model Based Simulator

## 8. Conclusions

## Author Contributions

## Acknowledgments

## Conflicts of Interest

## Abbreviations

ARX | Autoregressive with exogenous inputs |

CKD | Chronic Kidney Disease |

CVaR | Conditional Value at Risk |

DDE | Delayed Differential Equation |

EPO | Erythropoetin |

IOE | Integrated Output Error |

IU | International Units |

PIZ | Percent of Points in the Zone |

PK/PD | Pharmacokinetic and pharmacodynamic |

MPC | Model Predictive Control |

RBC | Red Blood Cells |

VaR | Value at Risk |

## References

- Hayat, A.; Haria, D.; Salifu, M. Erythropoetin stimulating agents in the management of anemia of chronic kidney disease. Patient Preference Adherence
**2008**, 2, 195–200. [Google Scholar] [PubMed] - Miskulin, D.; Weiner, D.; Tighiouart, H.; Ladik, V.; Servilla, K.; Zager, P.; Martin, A.; Johnson, H.; Meyer, K. Computerized decision support for EPO dosing in hemodialysis patients. Am. J. Kidney Dis.
**2009**, 54, 1081–1088. [Google Scholar] [CrossRef] [PubMed] - Grosman, B.; Dassau, E.; Zisser, H.; Jovanovic, L.; Doyle, F. Zone Model Predictive Control: A Strategy to Minimize Hyper- and Hypoglycemic Events. J. Diabetes Sci. Technol.
**2010**, 4, 961–975. [Google Scholar] [CrossRef] [PubMed] - Batora, V.; Tarnik, M.; Murgas, J.; Schmidt, S.; Nørgaard, K.; Poulsen, N.; Madsen, H.; Boiroux, D.; Jørgensen, J. The Contribution of Glucagon in an Artificial Pancreas for People with Type 1 Diabetes. In Proceedings of the 2015 American Control Conference (ACC), Chicago, IL, USA, 1–3 July 2015; pp. 5097–5102. [Google Scholar]
- Rivadeneira, P.; Ferramosca, A.; Gonzalez, A. Impulsive Zone Model Predictive Control with Application to Type I Diabetic Patients. In Proceedings of the 2016 IEEE Conference on Control Applications (CCA), Buenos Aires, Argentina, 19–22 September 2016; pp. 544–549. [Google Scholar]
- Lee, J.; Gondhalekar, R.; Doyle, F. Design of an Artificial Pancreas using Zone Model Predictive Control with a Moving Horizon State Estimator. In Proceedings of the 2014 IEEE 53rd Annual Conference on Decision and Control (CDC), Los Angeles, CA, USA, 15–17 December 2014; pp. 6975–6980. [Google Scholar]
- Bradbury, B.D.; Danese, M.D.; Gleeson, M.; Critchlow, C.W. Effect of Epoetin alfa dose changes on hemoglobin and mortality in hemodialysis patients with hemoglobin levels persistently below 11 g/dL. Clin. J. Am. Soc. Nephrol.
**2009**, 4, 630–637. [Google Scholar] [CrossRef] [PubMed] - Gaweda, A.; Jacobs, A.; Aronoff, G.; Brier, M. Model predictive control of erythropoietin administration in the anemia of ESRD. Am. J. Kidney Dis.
**2008**, 51, 71–79. [Google Scholar] [CrossRef] [PubMed] - Brier, M.; Gaweda, A.; Dailey, A.; Jacobs, A.; Aronoff, G. Randomized trial of model predictive control for improved anemia management. Clin. J. Am. Soc. Nephrol.
**2010**, 5, 814–820. [Google Scholar] [CrossRef] [PubMed] - Gaweda, A.; Jacobs, A.; Aronoff, G.; Rai, S.; Brier, M. Individualized anemia management reduces hemoglobin variability in hemodialysis patients. J. Am. Soc. Nephrol.
**2014**, 25, 159–166. [Google Scholar] [CrossRef] [PubMed] - Chait, Y.; Horowitz, J.; Nichols, B.; Shrestha, R.P.; Hollot, C.V.; Germain, M.J. Control-Relevant Erythropoiesis Modeling in End-Stage Renal Disease. IEEE Trans. Biomed. Eng.
**2014**, 61, 658–664. [Google Scholar] [CrossRef] [PubMed] - Akabua, E.; Inanc, T.; Gaweda, A.; Brier, M.; Kim, S.; Zurada, J. Individualized Model Discovery: The Case of Anemia Patients. J. Comput. Methods Programs Biomed.
**2015**, 118, 23–33. [Google Scholar] [CrossRef] [PubMed] - Alves, M.; Vilaca, S.; Carvalho, M.; Fernandes, A.; Dusse, L.; Gomes, K. Resistance of dialyzed patients to erythropoietin. Rev. Bras Hematol. Hemoter.
**2015**, 37, 190–197. [Google Scholar] [CrossRef] [PubMed] - Weis, L.; Metzger, M.; Haymann, J.; Thervet, E.; Flamant, M.; Vrtovsnik, F.; Gauci, C.; Houillier, P.; Froissart, M.; Letavernier, E.; et al. Renal Function Can Improve at Any Stage of Chronic Kidney Disease. PLoS ONE
**2013**, 8, e81835. [Google Scholar] [CrossRef] [PubMed] - Malhotra, V.; Beniwal, P.; Pursnani, L. Infections in Chronic Kidney Disease. Clin. Queries Nephrol.
**2012**, 1, 253–258. [Google Scholar] [CrossRef] - Saeed, F.; Agrawal, N.; Greenberg, E.; Holley, J. Lower Gastrointestinal Bleeding in Chronic Hemodialysis Patients. Int. J. Nephrol.
**2011**, 2011, 272535. [Google Scholar] [CrossRef] [PubMed] - Rockafellar, R.; Uryasev, S. Optimization of Conditional Value-at-Risk. J. Risk
**2000**, 2, 21–42. [Google Scholar] [CrossRef] - Parisio, A.; Molinari, M.; Varagnolo, D.; Johansson, K. A Scenario-based Predictive Control Approach to Building HVAC Management Systems. In Proceedings of the 2013 IEEE International Conference on Automation Science and Engineering (CASE), Madison, WI, USA, 17–20 August 2013; pp. 428–435. [Google Scholar]
- Bemporad, A.; Puglia, L.; Gabbriellini, T. A Stochastic Model Predictive Control Approach to Dynamic Option Hedging with Transaction Costs. In Proceedings of the 2011 American Control Conference (ACC), San Francisco, CA, USA, 29 June–1 July 2011; pp. 3862–3867. [Google Scholar]
- Zhang, X.; Liu, P.; Xu, C.; Ming, B.; Xie, A.; Feng, M. Conditional Value-at-Risk for Nonstationary Streamflow and Its Application for Derivation of the Adaptive Reservoir Flood Limited Water Level. J. Water Resour. Plan. Manag.
**2018**, 144, 04018005. [Google Scholar] [CrossRef] - Bemporad, A.; Morari, M. Robust model predictive control: A survey. In Robustness in Identification and Control; Springer: London, UK, 1999; pp. 207–226. [Google Scholar]
- Mesbah, A. Stochastic model predictive control: An overview and perspectives for future research. IEEE Control Syst.
**2016**, 36, 30–44. [Google Scholar] [CrossRef] - McAllister, J.; Li, Z.; Liu, J.; Simonsmeier, U. Erythropoietin Dose Optimization for Anemia in Chronic Kidney Disease Using Recursive Zone Model Predictive Control. IEEE Trans. Control Syst. Technol.
**2018**, in press. [Google Scholar] [CrossRef] - Thanakitcharuand, P.; Jirajan, B. Prevalence of hemoglobin cycling and its clinical impact on outcomes in Thai end-stage renal disease patients treated with hemodialysis and erythropoiesis-stimulating-agents. J. Med. Assoc. Thail.
**2016**, 99, 28–37. [Google Scholar] - Turksoy, K.; Bayrak, E.; Quinn, L.; Littlejohn, E.; Cinar, A. Multivariable Adaptive Closed-Loop Control of an Artificial Pancreas Without Meal and Activity Announcement. Diabetes Technol. Ther.
**2013**, 15, 386–400. [Google Scholar] [CrossRef] [PubMed] - Ren, J.; McAllister, J.; Li, Z.; Liu, J.; Simonsmeier, U. Modeling of Hemoglobin Response to Erythropoietin Therapy through Constrained Optimization. In Proceedings of the 6th International Symposium on Advanced Control of Industrial Processes, Taipei, Taiwan, 28–31 May 2017; pp. 245–250. [Google Scholar]

Controller Identifier | MPC | Zone-MPC | CVaR${}_{1}$ | CVaR${}_{2}$ |
---|---|---|---|---|

Q | 0.8 | 10 | 0.8 | 0.3 |

M | - | - | 500 | 500 |

$\Delta {u}_{H}$ (IU) | 20,000 | 20,000 | 20,000 | 20,000 |

$\beta $ | - | - | - | 0.95 |

$\u03f5$ | - | - | 0.1 | - |

${y}_{L}$ | 10.5 | 10 | 10.5 | 10 |

${y}_{H}$ | 10.5 | 11 | 10.5 | 11 |

Target | 10.5 | 10–11 | 10.5 | 10–11 |

Constraint Limits | - | - | 9.5–11.5 | - |

Performance Statistic | MPC | Zone-MPC | CVaR${}_{1}$ | CVaR${}_{2}$ |
---|---|---|---|---|

IOE ($\frac{g}{dL\phantom{\rule{4pt}{0ex}}week}$) | 112.1 | 174.3 | 113.2 | 108.0 |

PIZ (%) | 74.2 | 64.8 | 74.0 | 74.0 |

EPO/week (IU) | 5449 | 5450 | 5450 | 5448 |

Avg $\Delta EPO$ | 603 | 489 | 938 | 541 |

Time per Iteration (sec) | 0.005 | 0.005 | 0.091 | 0.588 |

Performance Statistic | MPC | CVaR${}_{1}$ |
---|---|---|

IOE ($\frac{g}{dL\phantom{\rule{4pt}{0ex}}week}$) | 146.7 | 115.5 |

PIZ (%) | 69.9 | 73.4 |

EPO/week (IU) | 5435 | 5438 |

Avg $\Delta EPO$ | 701 | 1413 |

Performance Statistic | Zone-MPC | CVaR${}_{2}$ |
---|---|---|

IOE ($\frac{g}{dL\phantom{\rule{4pt}{0ex}}week}$) | 189.2 | 140.8 |

PIZ (%) | 65.3 | 71.0 |

EPO/week (IU) | 5433 | 5435 |

Avg $\Delta EPO$ | 592 | 698 |

Parameter | Description |
---|---|

${H}_{en}$ | Hemoglobin Level due to Endogenous Erythropoietin |

$\mu $ | Mean RBC life span |

V | Maximal Exogenous Erythropoietin clearance rate |

${K}_{m}$ | Exogenous Erythropoietin level that produces half maximal clearance rate |

$\alpha $ | Linear clearance constant |

S | Maximal RBC production rate stimulated by ${E}_{P}$ |

C | Amount of EPO that produces half maximal RBC production rate |

D | Time required for EPO-stimulated RBCs to start forming |

Controller Identifier | MPC | Zone-MPC | CVaR${}_{1}$ | CVaR${}_{2}$ |
---|---|---|---|---|

Q | 3 | 10 | 3 | 1.5 |

M | - | - | 500 | 500 |

$\Delta {u}_{H}$ (IU) | 20,000 | 20,000 | 20,000 | 20,000 |

${u}_{H}$ (IU) | 30,000 | 30,000 | 30,000 | 30,000 |

$\beta $ | - | - | - | 0.95 |

$\u03f5$ | - | - | 0.3 | - |

${y}_{L}$ | 11 | 10 | 10 | 10 |

${y}_{H}$ | 11 | 12 | 12 | 12 |

Target | 11 | 10–12 | 10–12 | 10–12 |

Constraint Limits | - | - | 10–12 | - |

Performance Statistic | MPC | Zone-MPC | CVaR${}_{1}$ | CVaR${}_{2}$ |
---|---|---|---|---|

IOE ($\frac{g}{dL\phantom{\rule{4pt}{0ex}}week}$) | 25.99 | 30.47 | 25.04 | 39.05 |

PIZ (%) | 80.0 | 76.5 | 81.3 | 80.0 |

EPO/week (IU) | 20,044 | 20,059 | 20,399 | 19,795 |

Avg $\Delta EPO$ | 1333 | 805 | 1267 | 1343 |

Performance Statistic | MPC | Zone-MPC | CVaR${}_{1}$ | CVaR${}_{2}$ |
---|---|---|---|---|

IOE ($\frac{g}{dL\phantom{\rule{4pt}{0ex}}week}$) | 44.58 | 55.12 | 40.7 | 45.10 |

PIZ (%) | 70.4 | 65.2 | 77.8 | 71.7 |

EPO/week (IU) | 20,429 | 20,548 | 20,793 | 20,779 |

Avg $\Delta EPO$ | 1223 | 838 | 1236 | 1289 |

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

McAllister, J.; Li, Z.; Liu, J.; Simonsmeier, U.
EPO Dosage Optimization for Anemia Management: Stochastic Control under Uncertainty Using Conditional Value at Risk. *Processes* **2018**, *6*, 60.
https://doi.org/10.3390/pr6050060

**AMA Style**

McAllister J, Li Z, Liu J, Simonsmeier U.
EPO Dosage Optimization for Anemia Management: Stochastic Control under Uncertainty Using Conditional Value at Risk. *Processes*. 2018; 6(5):60.
https://doi.org/10.3390/pr6050060

**Chicago/Turabian Style**

McAllister, Jayson, Zukui Li, Jinfeng Liu, and Ulrich Simonsmeier.
2018. "EPO Dosage Optimization for Anemia Management: Stochastic Control under Uncertainty Using Conditional Value at Risk" *Processes* 6, no. 5: 60.
https://doi.org/10.3390/pr6050060