# Handling Uncertainty in Cost-Effectiveness Analysis: Budget Impact and Risk Aversion

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. A Hypothetical Example

- (i)
- Area A where the program is both affordable and cost-effective;
- (ii)
- Area B where the program is affordable but cost-ineffective;
- (iii)
- Area C where the program is not affordable but cost-effective;
- (iv)
- Area D where the new program is neither affordable nor cost-effective.

_{NMB}as previously defined can be written as

_{NMB}denotes the expected NMB of a program, μ

_{E}denotes mean effect, μ

_{C}mean cost of a program, and λ the ceiling ratio. DD

_{NMB}denotes the downside deviation, defined as

_{i}denotes a sample observation, which may be derived, for example, from bootstrapping mean costs and effects of a program [16]. The root-mean-square of all sample observations corresponds to the DD

_{NMB}. The S

_{NMB}Equation (1) penalises the expected NMB of a program μ

_{NMB}for its “bad” risk (i.e., its downside deviation DD

_{NMB}). Recalling Equation (3), DD

_{NMB}will be higher either if the number of observations n below μ

_{NMB}is higher and/or if the magnitude of deviations below μ

_{NMB}is higher. The method allows to include different levels of risk-aversion by defining a different minimally acceptable NMB for the downside deviation. For example, instead of penalising expected NMB for the downside deviation relative to the mean, a less risk-averse decision-maker may decide that any NMB below the 25% percentile of the NMB distribution denoted as ${\eta}_{{25}_{NMB}}$ would be considered as underperformance, and we would rewrite Equation (3) as

_{NMB}relative to a common yardstick. The concept of downside deviation is very versatile and powerful, and allows the decision-maker to define a constant or varying threshold level for a minimally acceptable NMB below which an intervention would be considered as providing insufficient economic value. For example, if there are three treatment options for the treatment of lung cancer, the analyst may want to define the 25% percentile of the NMB distribution for surgery/chemotherapy (intervention 1) as the minimally acceptable NMB, and use that same threshold level to also estimate DD

_{NMB}for radiotherapy/chemotherapy (intervention 2), and radiotherapy/chemotherapy/immunotherapy (intervention 3).

_{NMB}for each individual program and for all possible ceiling ratios λ by sampling 10,000 times from the distributions defined in Table 1. Figure 3 shows the CERAC for program F and program E. As can be seen from Figure 3, at a ceiling ratio of $9600 per QALY, program F becomes the preferred strategy and offers a higher net benefit to risk ratio. In other words, the threshold level where program F becomes preferable is different when comparing the CEAC ($13,333/QALY) with the CERAC ($9600/QALY).

## 3. The Example of Breast Cancer Treatment

_{NMB}relative to the mean NMB, which is of course different for each program for a given ceiling ratio. As also shown in Equation (4) for the example of the 25% percentile of the NMB distribution, the definition of downside deviation DD

_{NMB}offers much more flexibility, and a decision maker may want to use a common yardstick for both programs below which any NMB would be considered as providing insufficient economic value (i.e., underperformance). Let us assume that for a given ceiling ratio the decision-maker considers any NMB sample observation below the mean NMB of LET as underperformance for both PALLET and LET, then the DD

_{NMB}for PALLET would need to be modified accordingly, and the respective CERACs estimated via simulation are shown in Figure 8. As can be seen, when mean NMB of LET is used as a common yardstick to estimate downside deviation for both programs, then the CERAC for PALLET crosses the CERAC for LET at a ceiling ratio of CHF 363,000 ($US 390,051) per QALY (Figure 8) and becomes the preferred strategy, offering more expected return per unit of downside risk. As another example, let us assume a decision maker rather wants to define any NMB sample observation below the 25% percentile of the NMB distribution of PALLET as underperformance, and at the same time any NMB sample observation below the median of LET as underperformance. These two respective CERACs estimated using simulation are shown in Figure 9. In Figure 9, the CERACs for PALLET and LET cross at CHF 209,600 ($US 225,219) per QALY where PALLET becomes preferable. As shown by these examples, the CERAC is very versatile, and can accommodate a constant or a varying value for the minimally acceptable NMB below which one would consider a program’s return on investment as insufficient. A lower minimally acceptable NMB implicitly reflects a lower degree of risk-aversion.

## 4. Discussion

## 5. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- Van Hout, B.A.; Al, M.J.; Gordon, G.S.; Rutten, F.F. Costs, effects and C/E-ratios alongside a clinical trial. Health Econ.
**1994**, 3, 309–319. [Google Scholar] [CrossRef] [PubMed] - Stinnett, A.A.; Mullahy, J. Net health benefits: A new framework for the analysis of uncertainty in cost-effectiveness analysis. Med. Decis. Mak.
**1998**, 18, S68–S80. [Google Scholar] [CrossRef] [PubMed] - Zethraeus, N.; Johannesson, M.; Jönsson, B.; Löthgren, M.; Tambour, M. Advantages of using the net-benefit approach for analysing uncertainty in economic evaluation studies. Pharmacoeconomics
**2003**, 21, 39–48. [Google Scholar] [CrossRef] - Tambour, M.; Zethraeus, N.; Johannesson, M. A note on confidence intervals in cost-effectiveness analysis. Int. J. Technol. Assess. Health Care
**1998**, 14, 467–471. [Google Scholar] [CrossRef] - Briggs, A.; Fenn, P. Confidence intervals or surfaces? Uncertainty on the cost-effectiveness plane. Health Econ.
**1998**, 7, 723–740. [Google Scholar] [CrossRef] - Al, M.J. Cost-effectiveness acceptability curves revisited. Pharmacoeconomics
**2013**, 31, 93–100. [Google Scholar] [CrossRef] - Groot Koerkamp, B.; Hunink, M.G.M.; Stijnen, T.; Hammitt, J.K.; Kuntz, K.M.; Weinstein, M.C. Limitations of acceptability curves for presenting uncertainty in cost-effectiveness analysis. Med. Decis. Mak.
**2007**, 27, 101–111. [Google Scholar] [CrossRef] - Sendi, P.P.; Briggs, A.H. Affordability and cost-effectiveness: Decision-making on the cost-effectiveness plane. Health Econ.
**2001**, 10, 675–680. [Google Scholar] [CrossRef] [PubMed] - Fenwick, E.; Briggs, A. Cost-effectiveness acceptability curves in the dock: Case not proven? Med. Decis. Mak.
**2007**, 27, 93–95. [Google Scholar] [CrossRef][Green Version] - O’Brien, B.J.; Sculpher, M.J. Building uncertainty into cost-effectiveness rankings: Portfolio risk-return tradeoffs and implications for decision rules. Med. Care
**2000**, 38, 460–468. [Google Scholar] [CrossRef] - Al, M.J.; Feenstra, T.L.; Hout, B.A.V. Optimal allocation of resources over health care programmes: Dealing with decreasing marginal utility and uncertainty. Health Econ.
**2005**, 14, 655–667. [Google Scholar] [CrossRef] [PubMed] - Ben-Zion, U.; Gafni, A. Evaluation of public investment in health care. Is the risk irrelevant? J. Health Econ.
**1983**, 2, 161–165. [Google Scholar] [CrossRef] - Zivin, J.G.; Bridges, J.F. Addressing risk preferences in cost-effectiveness analyses. Appl. Health Econ. Health Policy
**2002**, 1, 135–139. [Google Scholar] - Zivin, J.G. Cost-effectiveness analysis with risk aversion. Health Econ.
**2001**, 10, 499–508. [Google Scholar] [CrossRef] - Elbasha, E.H. Risk aversion and uncertainty in cost-effectiveness analysis: The expected-utility, moment-generating function approach. Health Econ.
**2005**, 14, 457–470. [Google Scholar] [CrossRef] [PubMed] - Sendi, P. Dealing with Bad Risk in Cost-Effectiveness Analysis: The Cost-Effectiveness Risk-Aversion Curve. Pharmacoeconomics
**2021**, 39, 161–169. [Google Scholar] [CrossRef] [PubMed] - Matter-Walstra, K.; Ruhstaller, T.; Klingbiel, D.; Schwenkglenks, M.; Dedes, K.J. Palbociclib as a first-line treatment in oestrogen receptor-positive, HER2-negative, advanced breast cancer not cost-effective with current pricing: A health economic analysis of the Swiss Group for Clinical Cancer Research (SAKK). Breast Cancer Res. Treat.
**2016**, 158, 51–57. [Google Scholar] [CrossRef] - Sendi, P.; Al, M.J.; Zimmermann, H. A risk-adjusted approach to comparing the return on investment in health care programs. Int. J. Health Care Financ. Econ.
**2004**, 4, 199–210. [Google Scholar] [CrossRef] [PubMed] - Plantinga, A.; de Groot, J.S. Risk-adjusted performance measures and implied risk attitudes. J. Perform. Meas.
**2002**, 6, 9–20. [Google Scholar] [CrossRef][Green Version] - Sendi, P.; Gafni, A.; Birch, S. Opportunity costs and uncertainty in the economic evaluation of health care interventions. Health Econ.
**2002**, 11, 23–31. [Google Scholar] [CrossRef] [PubMed] - Sendi, P.; Gafni, A.; Birch, S.; Walter, S.D. Incorporating Portfolio Uncertainty in Decision Rules for Healthcare Resource Allocation. Healthcare
**2021**, 9, 325. [Google Scholar] [CrossRef] [PubMed] - O’Brien, B.J.; Gertsen, K.; Willan, A.R.; Faulkner, L.A. Is there a kink in consumers′ threshold value for cost-effectiveness in health care? Health Econ.
**2002**, 11, 175–180. [Google Scholar] [CrossRef] [PubMed] - Severens, J.L.; Brunenberg, D.E.M.; Fenwick, E.A.L.; O’Brien, B.; Joore, M.A. Cost-effectiveness acceptability curves and a reluctance to lose. Pharmacoeconomics
**2005**, 23, 1207–1214. [Google Scholar] [CrossRef] - Sendi, P.; Ramadani, A.; Zitzmann, N.U.; Bornstein, M.M. A Systematic Review of WTA-WTP Disparity for Dental Interventions and Implications for Cost-Effectiveness Analysis. Healthcare
**2020**, 8, 301. [Google Scholar] [CrossRef] [PubMed] - Tu, H.A.T.; de Vries, R.; Woerdenbag, H.J.; Li, S.C.; Le, H.H.; van Hulst, M.; Postma, M.J. Cost-Effectiveness Analysis of Hepatitis B Immunization in Vietnam: Application of Cost-Effectiveness Affordability Curves in Health Care Decision Making. Value Health Reg Issues
**2012**, 1, 7–14. [Google Scholar] [CrossRef] [PubMed] - Kim, S.-Y.; Salomon, J.A.; Goldie, S.J. Economic evaluation of hepatitis B vaccination in low-income countries: Using cost-effectiveness affordability curves. Bull. World Health Organ.
**2007**, 85, 833–842. [Google Scholar] [CrossRef] [PubMed] - Matthys, C.; De Vijlder, W.; Besseler, J.; Glibert, M.; De Bruyn, H. Cost-effectiveness analysis of two attachment systems for mandibular overdenture. Clin. Oral Implant. Res.
**2020**, 31, 615–624. [Google Scholar] [CrossRef][Green Version] - Yi, Y.; Lucherini, S. Use of cost-effectiveness affordability curves in evaluating cancer drugs in England. Value Health
**2019**, 22, S462. [Google Scholar] [CrossRef] - Yi, Y.; Lucherini, S.; Bellanca, L.; Heron, L. Assessing the Joint Probability of Cost-Effectiveness and Affordability in Decision Making. Value Health
**2017**, 20, A753. [Google Scholar] [CrossRef][Green Version] - Weinstein, M.; Zeckhauser, R. Critical ratios and efficient allocation. J. Public Econ.
**1973**, 2, 147–157. [Google Scholar] [CrossRef] - Stinnett, A.A.; Paltiel, A.D. Mathematical programming for the efficient allocation of health care resources. J. Health Econ.
**1996**, 15, 641–653. [Google Scholar] [CrossRef] - Sendi, P.; Al, M.J. Revisiting the decision rule of cost-effectiveness analysis under certainty and uncertainty. Soc. Sci. Med.
**2003**, 57, 969–974. [Google Scholar] [CrossRef] - Birch, S.; Gafni, A. Cost effectiveness/utility analyses. Do current decision rules lead us to where we want to be? J. Health Econ.
**1992**, 11, 279–296. [Google Scholar] [CrossRef] - Gafni, A.; Birch, S. Guidelines for the adoption of new technologies: A prescription for uncontrolled growth in expenditures and how to avoid the problem. CMAJ
**1993**, 148, 913–917. [Google Scholar] - Gafni, A.; Birch, S. Incremental cost-effectiveness ratios (ICERs): The silence of the lambda. Soc. Sci. Med.
**2006**, 62, 2091–2100. [Google Scholar] [CrossRef] - Sendi, P. Bridging the gap between health and non-health investments: Moving from cost-effectiveness analysis to a return on investment approach across sectors of economy. Int. J. Health Care Financ. Econ.
**2008**, 8, 113–121. [Google Scholar] [CrossRef][Green Version] - Lomas, J.R.S. Incorporating Affordability Concerns within Cost-Effectiveness Analysis for Health Technology Assessment. Value Health
**2019**, 22, 898–905. [Google Scholar] [CrossRef][Green Version] - Sortino, F.A.; Van Der Meer, R. Downside risk. J. Portf. Manag.
**1991**, 17, 27–31. [Google Scholar] [CrossRef]

**Figure 1.**Incremental costs and effects of program F versus program E on the cost-effectiveness plane. λ denotes the ceiling ratio, β denotes the budget constraint. A denotes the area where the intervention is both affordable and cost-effective, B denotes the area where the intervention is affordable but not cost-effective, C denotes the area where the intervention is cost-effective but not affordable, D denotes the area where the intervention is neither affordable nor cost-effective.

**Figure 2.**Cost-effectiveness affordability curves (CEAFCs) for different budget constraints comparing program F to program E. Without any budget constraint, the CEAFC corresponds to the CEAC.

**Figure 3.**Cost-effectiveness risk-aversion curve (CERAC). At a ceiling ratio of $9600 per QALY, program F becomes preferable to program E (Table 1) as it offers more expected return per unit of downside risk.

**Figure 4.**Joint distribution of total costs and effects of PALLET (palbociclib and letrozole) versus LET (letrozole) in patients with metastatic ER + HER2- breast cancer. One Swiss Franc (CHF) corresponds to $US 1.07.

**Figure 5.**Incremental costs and effects of PALLET (palbociclib and letrozole) versus LET (letrozole) in patients with metastatic ER + HER2- breast cancer. The red line represents the ceiling ratio. The proportion of samples below the ceiling ratio represents the probability that PALLET is cost-effective at CHF 200,000 per QALY, which is 11%. One CHF (Swiss Franc) corresponds to $US 1.07.

**Figure 6.**Cost-effectiveness affordability curves (CEAFC) of PALLET (palbociclib and letrozole) versus LET (letrozole) in patients with metastatic ER + HER2- breast cancer. In the absence of a budget constraint, the CEAFC corresponds to the CEAC. The horizontal line at probability 0.5 represents the anticipated minimal joint probability an intervention is both cost-effective and affordable.

**Figure 7.**Cost-effectiveness risk-aversion curve (CERAC) of PALLET (palbociclib and letrozole) versus LET (letrozole) in patients with metastatic ER + HER2- breast cancer. CERACs are estimated using Equation (3) as downside deviation.

**Figure 8.**Cost-effectiveness risk-aversion curve (CERAC) of PALLET (palbociclib and letrozole) versus LET (letrozole) in patients with metastatic ER + HER2- breast cancer. CERACs are estimated using the mean NMB of LET to estimate the downside deviation for both PALLET and LET.

**Figure 9.**Cost-effectiveness risk-aversion curve (CERAC) of PALLET (palbociclib and letrozole) versus LET (letrozole) in patients with metastatic ER + HER2- breast cancer. CERACs are estimated using the 25% percentile of the NMB distribution for PALLET and the median of the NMB distribution for LET to estimate downside deviation. The two CERACs cross at CHF 209,600 ($US 225,219) per QALY where PALLET becomes preferable.

Program | μ_{C} ($) | ơ_{C} ($) | μ_{E} (QALY) | ơ_{E} (QALY) | p |
---|---|---|---|---|---|

E | 50,000 | 5000 | 10 | 1.3 | 0.4 |

F | 90,000 | 15,000 | 13 | 1.1 | 0.8 |

_{C}denotes mean costs, ơ

_{C}denotes standard deviation of costs, μ

_{E}denotes mean effects, ơ

_{E}denotes standard deviation of effects; normal distributions for costs and effects are assumed; correlation between costs and effects of each program is denoted by p; QALY denotes quality-adjusted life-years.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Sendi, P.; Matter-Walstra, K.; Schwenkglenks, M. Handling Uncertainty in Cost-Effectiveness Analysis: Budget Impact and Risk Aversion. *Healthcare* **2021**, *9*, 1419.
https://doi.org/10.3390/healthcare9111419

**AMA Style**

Sendi P, Matter-Walstra K, Schwenkglenks M. Handling Uncertainty in Cost-Effectiveness Analysis: Budget Impact and Risk Aversion. *Healthcare*. 2021; 9(11):1419.
https://doi.org/10.3390/healthcare9111419

**Chicago/Turabian Style**

Sendi, Pedram, Klazien Matter-Walstra, and Matthias Schwenkglenks. 2021. "Handling Uncertainty in Cost-Effectiveness Analysis: Budget Impact and Risk Aversion" *Healthcare* 9, no. 11: 1419.
https://doi.org/10.3390/healthcare9111419