# Non-Parametric Generalized Additive Models as a Tool for Evaluating Policy Interventions

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Methods

#### 2.1. The Interrupted Time Series Design

#### 2.2. Segmented Linear Regression Models

#### 2.3. Generalized Additive Models

`mgcv`in R [16] because they are an optimal smoother given basis dimension/rank ([17]) and they are more flexible than the cubic smoothing splines [18]. Thin plate regression splines do not have knots but are more computationally costly to set up [16]. GAMs can be defined by the next equation:

## 3. Simulation Analysis

#### 3.1. Data Generation Process

#### 3.2. Results of the Simulation Analysis

## 4. Illustration with Real Data: Impact of the Cost-Sharing Reforms on Pharmaceutical Prescriptions Established in Spain

`gam.check`[16]). We also checked that the default maximum possible dimensions of the basis used for the trend spline ($k=9$) was enough.

## 5. Conclusions

## Supplementary Materials

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- Shadish, W.R.; Cook, T.D.; Campbell, D.T. Experimental and Quasiexperimental Designs for Generalized Causal Inference; Houghton Mifflin: Boston, MA, USA, 2002. [Google Scholar]
- Ho, A.K.F.; Wan, A.T.K. Testing for covariance stationarity of stock returns in the presence of structural breaks: An intervention analysis. Appl. Econ.
**2002**, 9, 441–447. [Google Scholar] [CrossRef] - Lagarde, M. How to do (or not to do)... Assessing the impact of a policty change with routine longitudinal data. Health Policy Plan.
**2012**, 27, 76–83. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Briesacher, B.A.; Soumerai, S.B.; Zhang, F.; Toh, S.; Andrade, E.; Wagner, J.L.; Shoaibi, A.; Gurwitz, J.H. A critical review of methods to evaluate the impact of FDA regulatory actions. Pharmacoepidemiol. Drug Saf.
**2013**, 22, 986–994. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Wagner, A.K.; Soumerai, S.B.; Zhang, F.; Ross-Degnan, D. Segmented regression analysis of interrupted time series studies in medication use research. J. Clin. Pharm. Ther.
**2002**, 27, 299–309. [Google Scholar] [CrossRef] [PubMed] - Taljaard, M.; McKenzie, J.E.; Ramsay, C.R.; Grimshaw, J.M. The use of segmented regression in analysing interrupted time series studies: An example in pre-hospital ambulance care. Implement. Sci.
**2014**, 9, 77. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Gillings, D.; Makuc, D.; Siegel, E. Analysis of interrupted time series mortality trends: An example to evaluate regionalized perinatal care. Am. J. Public Health
**1981**, 71, 38–46. [Google Scholar] [CrossRef] [PubMed] [Green Version] - McCleary, R.; McDowall, D.; Bartos, B. Design and Analysis of Time Series Experiments; Oxford University Press Inc.: New York, NY, USA, 2017. [Google Scholar]
- McDowall, D.; McCleary, R.; Meidinger, E.E.; Hay, R.A. Interrupted Time Series Anlaysis; Sage: Newbury Park, CA, USA, 1980. [Google Scholar]
- Hategeka, C.; Ruton, H.; Karamouzian, M.; Lynd, L.D.; Law, M.R. Use of interrupted time series methods in the evaluation of health system quality improvement interventions: A methodological systematic review. BMJ Glob. Health
**2020**, 5, e003567. [Google Scholar] [CrossRef] [PubMed] - Hastie, T.; Tibshirani, R. Generalized Additive Models; John Wiley & Sons, Inc.: Hoboken, NJ, USA, 1990. [Google Scholar]
- Sullivan, K.J.; Shadish, W.R.; Steiner, P.M. An introduction to modeling longitudinal data with generlaized additive models: Applications to single-case designs. Psychol. Methods
**2015**, 20, 26–42. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Puig-Junoy, J.; Rodríguez-Feijoó, S.; López-Valcárcel, B.G. Paying for formerly free medicines in Spain after 1 year of co-payment: Changes in the number of dispensed prescriptions. Appl. Health Econ. Health Policy
**2014**, 12, 279–287. [Google Scholar] [CrossRef] [PubMed] - Prais, S.J.; Winsten, C.B. Trend Estimators and Serial Correlation; Working paper 383; Cowles Commission: Chicago, IL, USA, 1954. [Google Scholar]
- Turner, S.; Karahalios, A.; Forbes, A.B.; Taljaard, M.; Grimshaw, J.M.; Cheng, A.C.; Bero, L.; McKenzie, J.E. Design characteristics and statistical methods used in interrupted time series studies evaluating public health interventions: A review. J. Clin. Epidemiol.
**2020**, 122, 1–11. [Google Scholar] [CrossRef] [PubMed] - Wood, S. Package “mgcv”. Mixed GAM Computation Vehicle with Automatic Smoothness Estimation. R package version 1.8-33. 2020. Available online: https://cran.r-project.org/web/packages/mgcv/mgcv.pdf (accessed on 26 January 2021).
- Wood, S.N. Thin plate regression splines. J. R. Stat. Soc. Ser. B
**2003**, 65, 95–114. [Google Scholar] [CrossRef] - Perperoglou, A.; Sauerbrei, W.; Abrahamowicz, M.; Schmid, M. A review of spline function procedures in R. BMC Med. Res. Methodol.
**2019**, 19, 46. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Wood, S. Low rank scale invariant tensor product smooths for generalized additive mixed models. Biometrics
**2006**, 62, 1025–1036. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Wood, S. Generalized Additive Models: An Introduction with R; Taylor and Francis: Boca Raton, FL, USA, 2006. [Google Scholar]
- Van Gellecom, F.S. Advances in non-linear economic modeling-theory and applications. In Dynamic Modeling and Econometrics in Economics and Finance; Springer: Berlin/Heidelberg, Germany, 2014; Volume 17. [Google Scholar]
- Official State Bulletin (BOE). Urgent Measures to Guarantee the Sustainability of the National Health System and Improve the Quality and Safety of Services; Royal Decree Law (RDL) 16/2012; BOE: Madrid, Spain, 2012. [Google Scholar]
- Hernandez-Izquierdo, C.; López-Valcárcel, B.G.; Morris, S.; Melnychuk, M.; Abásolo, I. The effect of a change in co-payment on prescription drug demand in a National Health System: The case of 15 drug families by price elasticity of demand. PLoS ONE
**2019**, 14, E0213403. [Google Scholar] - Hastie, T.; Tibshirani, R. Generalized Additive Models. Stat. Sci.
**1986**, 1, 297–318. [Google Scholar] [CrossRef] - Yee, T.W.; Mitchell, N.D. Generalized additive models in plant ecology. J. Veg. Sci.
**1991**, 2, 587–602. [Google Scholar] [CrossRef] - Ruppert, D.; Wand, M.P.; Carroll, R.J. Semiparametric Regression; Cambridge University Press: Cambridge, UK, 2003. [Google Scholar]
- Wood, S. Generalized Additive Models and Introduction with R, 2nd ed.; Chapman and Hall/CRC: London, UK, 2017. [Google Scholar]
- Wasserman, L. All of Nonparametric Statistics; (Springer Texts in Statistics); Casella, G., Fienberg, S., Olkin, I., Eds.; Springer: New York, NY, USA, 2006; Chapter 5; pp. 61–123. [Google Scholar]
- Hewamalage, H.; Bergmeir, C.; Bandara, K. Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directions. Int. J. Forecast.
**2020**, 37, 388–427. [Google Scholar] [CrossRef] - Levin, E. A recurrent neural network: Limitations and training. Neural Netw.
**1990**, 3, 641–650. [Google Scholar] [CrossRef] - Simpson, G.L. Modelling Palaeoecological Time Series Using Generalised Additive Models. Front. Ecol. Evol.
**2018**, 6, 149. [Google Scholar] [CrossRef] [Green Version]

**Figure 1.**Simulated data. The left upper plot shows the linear model, the right upper plot shows the quadratic model and the lower plot shows the polynomial model. In all pictures, the black line refers to the deterministic part of the simulated data, and the dashed line refers to the projection of the pre-intervention model in the post-intervention period.

**Figure 2.**Results for simulated data. The left plot shows the segmented linear regression analysis and the right plot shows the generalized additive analysis. In both pictures, the blue line refers to the simulated data, the black line refers to the fitted values and the dashed line refers to the projection of the pre-intervention model in the post-intervention period.

**Figure 3.**Histogram plots of the per-capita prescriptions for the pre-intervention period (left plot) and the post-intervention period (right plot).

**Figure 4.**Results for real data. The left plot shows the segmented linear regression analysis and the right plot shows the generalized additive analysis. In both pictures, the blue line refers to the simulated data, the black line refers to the fitted values and the dashed line refers to the projection of the pre-intervention model in the post-intervention period.

Model | Specification | E [Immediate Impact] | E [Total Impact] |
---|---|---|---|

Linear model | ${Y}_{t}=30-0.08\xb7{T}_{t}+5\xb7{I}_{t}+0.02\xb7T{I}_{t}+{u}_{t}$ | 5 | 50.9 |

Quadratic model | ${Y}_{t}=30-0.25\xb7{T}_{t}+0.002\xb7{T}_{t}^{2}-$ $-5\xb7{I}_{t}+0.1\xb7T{I}_{t}+0.005\xb7T{I}_{t}^{2}+{u}_{t}$ | $-5$ | $-44.075$ |

Polynomial model | ${Y}_{t}=30+0.2235\xb7{T}_{t}-0.008\xb7{T}_{t}^{2}+0.00006\xb7{T}_{t}^{3}-$ $-5\xb7{I}_{t}+0.1\xb7T{I}_{t}-0.05\xb7T{I}_{t}^{2}+0.0001\xb7T{I}_{t}^{3}+{u}_{t}$ | $-5$ | $-59.5475$ |

Simulation Model: | Linear Model | Quadratic Model | Polynomial Model | |||
---|---|---|---|---|---|---|

Level change | ||||||

Estimated model: | SLRM | GAM | SLRM | GAM | SLRM | GAM |

Mean (sd) | $5.0218\left(0.3935\right)$ | $5.0246\left(0.4411\right)$ | $-4.5442\left(0.6264\right)$ | $-4.8969\left(0.4787\right)$ | $-4.2842\left(0.6205\right)$ | $-4.5857\left(0.5535\right)$ |

MSE | 0.1550 | 0.1948 | 0.5993 | 0.2393 | 0.8966 | 0.4774 |

MPE | 0.0627 | 0.0704 | 0.1273 | 0.0776 | 0.1594 | 0.1133 |

Cumulative effect | ||||||

Estimated model: | SLRM | GAM | SLRM | GAM | SLRM | GAM |

Mean (sd) | 51.0629 (2.5283) | $51.1704\left(2.9021\right)$ | $-23.6094\left(5.4400\right)$ | $-40.5424\left(4.8718\right)$ | $-36.0033\left(5.0288\right)$ | $-49.8200\left(5.8647\right)$ |

MSE | 6.4061 | 8.4787 | 448.3769 | 36.1665 | 579.5690 | 128.9503 |

MPE | 0.0403 | 0.0449 | 0.4643 | 0.1119 | 0.3954 | 0.1698 |

**Table 3.**Descriptive statistics of the per-capita prescriptions dispensed in Spain during the period 2004–2015.

Pre-Intervention (2004–June 2012) | Post-Intervention (July 2012–2015) | 2004–2015 | |
---|---|---|---|

Mean | 1.6006 | 1.5429 | 1.5838 |

Median | 1.6025 | 1.5599 | 1.5841 |

Standard deviation | 0.0134 | 0.01222 | 0.0104 |

Coefficients | Estimate (Standard Error) | p-Value |
---|---|---|

Intercept | 1.4765 (0.0151) | <0.0001 |

${T}_{t}$ | 0.0039 (0.0001) | <0.0001 |

${I}_{t}$ | −0.3068 (0.0138) | <0.0001 |

$T{I}_{t}$ | −0.0012 (0.0005) | $0.0214$ |

Stockpiling | 0.1117 (0.0477) | 0.0207 |

January | reference | |

February | −0.0896 (0.0215) | <0.0001 |

March | −0.0324 (0.0188) | 0.0871 |

April | −0.0500 (0.0195) | 0.0115 |

May | −0.0373 (0.0194) | 0.0559 |

June | −0.0676 (0.0198) | 0.0009 |

July | −0.0978 (0.0194) | <0.0001 |

August | −0.2033 (0.0194) | <0.0001 |

September | −0.1207 (0.0194) | <0.0001 |

October | −0.0393 (0.0195) | 0.0461 |

November | −0.0886 (0.0189) | <0.0001 |

December | −0.0723 (0.0214) | 0.0010 |

M1 | M2 | |||
---|---|---|---|---|

Coefficients | Estimate (Standard Error) | p-value | Estimate (Standard Error) | p-value |

Intercept | 1.6615 (0.0065) | <0.0001 | 1.6491 (0.0146) | <0.0001 |

${I}_{t}$ | $-0.2704$ (0.0198) | <0.0001 | $-0.2704$ (0.0198) | <0.0001 |

Stockpiling | 0.1604 (0.0471) | 0.0009 | 0.1604 (0.0471) | 0.0009 |

$T{I}_{t}$ | 0.0021 (0.0021) | 0.3153 | ||

Smooth terms | EDF | p-value | EDF | p-value |

$s\left({T}_{t}\right)$ | 4.167 | <0.0001 | 4.167 | <0.0001 |

$s\left(T{I}_{t}\right)$ | 1.000 | $0.315$ | ||

$s\left({month}_{t}\right)$ | 9.169 | <0.0001 | 9.169 | <0.0001 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Pinilla, J.; Negrín, M.
Non-Parametric Generalized Additive Models as a Tool for Evaluating Policy Interventions. *Mathematics* **2021**, *9*, 299.
https://doi.org/10.3390/math9040299

**AMA Style**

Pinilla J, Negrín M.
Non-Parametric Generalized Additive Models as a Tool for Evaluating Policy Interventions. *Mathematics*. 2021; 9(4):299.
https://doi.org/10.3390/math9040299

**Chicago/Turabian Style**

Pinilla, Jaime, and Miguel Negrín.
2021. "Non-Parametric Generalized Additive Models as a Tool for Evaluating Policy Interventions" *Mathematics* 9, no. 4: 299.
https://doi.org/10.3390/math9040299