This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).

Women are advised to be vaccinated for influenza during pregnancy and may receive vaccine at any time during their pregnancy. In observational studies evaluating vaccine safety in pregnancy, to account for such time-varying vaccine exposure, a time-dependent predictor can be used in a proportional hazards model setting for outcomes such as spontaneous abortion or preterm delivery. Also, due to the observational nature of pregnancy exposure cohort studies and relatively low event rates, propensity score (PS) methods are often used to adjust for potential confounders. Using Monte Carlo simulation experiments, we compare two different ways to model the PS for vaccine exposure: (1) logistic regression treating the exposure status as binary yes or no; (2) Cox regression treating time to exposure as time-to-event. Coverage probability of the nominal 95% confidence interval for the exposure effect is used as the main measure of performance. The performance of the logistic regression PS depends largely on how the exposure data is generated. In contrast, the Cox regression PS consistently performs well across the different data generating mechanisms that we have considered. In addition, the Cox regression PS allows adjusting for potential time-varying confounders such as season of the year or exposure to additional vaccines. The application of the Cox regression PS is illustrated using data from a recent study of the safety of pandemic H1N1 influenza vaccine during pregnancy.

Routine annual seasonal influenza vaccination is recommended for all persons six months of age or older. In particular, women are advised to be vaccinated for influenza during pregnancy and may receive vaccine at any time during their pregnancy [

In addition, in studies on vaccine safety carried out using pregnancy exposure cohorts, women may also enter a study at any time during their pregnancy. An example of one ongoing prospective pregnancy exposure cohort study evaluating the safety of vaccines and medications is that conducted by the Organization of Teratology Information Specialists (OTIS) Collaborative Research group based at the University of California San Diego as part of the national Vaccines and Medication in Pregnancy Surveillance System (VAMPSS) [

As the pregnancy safety studies are typically observational in nature, it is inevitable that potential confounders need to be taken into consideration. The Cox proportional hazards model can be used to handle left truncated spontaneous abortion or preterm delivery data, as well as to adjust for potential confounders in a regression setting. In addition, the Cox model can properly handle the time-dependent vaccine exposure caused by women receiving vaccine at arbitrary times during pregnancy.

There can be many potential confounders for spontaneous abortion or preterm delivery in association with vaccine exposure. Some examples are: maternal age, race/ethnicity, socioeconomic status, tobacco or alcohol use, pre-pregnancy body mass index, use of vitamin supplements, pregnancy history, previous preterm delivery, infection, fever, maternal asthma, depression, autoimmune disease, hypertension, and additional vaccine exposures [

Traditionally, when the exposure status is binary yes or no, the propensity score can be calculated using a logistic regression model with the confounders as predictors. With time-dependent vaccine exposure, while one can still calculate the propensity score using logistic regression by ignoring the timing of exposure and treating it as binary yes or no, one has the additional option to calculate it using the Cox model with time to exposure as the dependent variable. In the following we compare the two approaches using Monte Carlo simulation, as well as apply the Cox regression propensity score approach to a recent study of pandemic H1N1 vaccine exposure in pregnancy with time-dependent confounders.

Here, we consider preterm delivery, while the data and model for spontaneous abortion are similar. The time scale is gestational age, with the first day of a woman's last menstrual period (LMP) as time zero, and the day of preterm delivery or lost to follow-up (_{0}(

We carry out Monte Carlo simulation studies to investigate the two different approaches to calculate the propensity score to be included in

In the first scenario, we have the following five steps:

_{1}, _{2} and _{3} as independent standard normal N(0, 1).

_{1}, _{2} and _{3} using a probit model:
_{0}, _{1}, _{2}, _{3}) = (1,1,1,1). Note that this is not the final exposure status; in fact _{0} = 1 tends to give more

_{1}, _{2} and _{3} from an exponential distribution with rate exp(_{0} + _{1}_{1} + _{2}_{2} + _{3}_{3}), where (_{0}, _{1}, _{2}, _{3}) = (1, 1, 1, 1). For those with

_{1}, _{2}, _{3} and _{1}(_{0}(_{1} takes on various values as given in the tables, and (_{2}, _{3}, _{4}) = (1, 1, 1). This is effectively a piecewise exponential distribution, with different rates on the two intervals (0,

The final data set consists of the potentially right censored event time _{1}, _{2}, _{3}, and time-dependent exposure _{1}(_{1}(

In the second scenario, we skip Step 2 above, and instead generate a potential exposure time

The difference between the two scenarios above might be interpreted as follows. In the first scenario, some individuals inherently do not get vaccinated during pregnancy, while some other individuals end up not getting vaccinated because they have waited too long and the pregnancy has ended. In the second scenario, everyone has the potential to get vaccinated, some end up getting it during pregnancy, some end up not. The first scenario attempts to at least partially mimic the logistic regression set up, while the second scenario follows the Cox regression model. In practice we may not be able to tell which is the true data generating mechanism. The purpose of this investigation is to see if either of the propensity score methods might be sensitive to the different underlying assumptions.

Once the data have been generated, we build the propensity score using two approaches. The first approach uses logistic regression by treating the final exposure status _{1}(_{1}, _{2} and _{3}. Denote the estimated coefficients from this logistic regression as _{0}, _{1}, _{2} and _{3}, then the linear combination PS_{1} = _{1}_{1} + _{2}_{2} + _{3} is a monotone function of the estimated probability of final exposure. We use PS_{1} as the propensity score, and fit the final adjusted Cox regression model for preterm delivery

In the second approach to build propensity score we use the Cox regression model by treating time to exposure as outcome, and with predictors _{1}, _{2} and _{3}. For those never exposed in the data, time to exposure is censored at the end of the observation time. Denote the estimated coefficients from this Cox regression as _{1}, _{2} and _{3}, then the linear combination PS_{2} = _{1}_{1} + _{2}_{2} + _{3}_{3} is a monotone function of the estimated risk of exposure under the Cox model. We use PS_{2} as the propensity score, and fit the final adjusted Cox regression model

In the following we carry out simulation with sample size 100, and about 25% right-censoring. This gives about 75 events which is comparable to the vaccine data below.

_{1} under the true Cox model (2), using the logistic propensity score approach (3), and using the Cox model propensity score approach (4). The table gives the average of the estimates over 2,000 simulation runs, its standard deviation (SD), the average of the estimated standard error (SE), the mean squared error (MSE), and the coverage probability (CP) of the nominal 95% confidence intervals. From the table we see that the true Cox model (2) does well for _{1} = 0.5,1 or 1.5, as expected. The logistic propensity score approach has some bias when _{1} = 0.5, with relatively low coverage probability of 89% and relatively high MSE compared to the other two approaches. But as _{1} increases, the Cox model propensity score approach has increasing downward bias, leading to increasing MSE and coverage probabilities of 95% confidence intervals as low as 91%.

First scenario of simulation: intermediate exposure status generated as binary, then timing of exposure is generated.

_{1} |
||||||
---|---|---|---|---|---|---|

0.5 | True | 0.511 | 0.269 | 0.267 | 0.073 | 95.3 |

Logistic PS | 0.700 | 0.253 | 0.262 | 0.104 | 89.2 | |

Cox PS | 0.440 | 0.258 | 0.264 | 0.070 | 95.2 | |

| ||||||

1 | True | 1.022 | 0.265 | 0.261 | 0.071 | 95.0 |

Logistic PS | 1.087 | 0.237 | 0.256 | 0.064 | 95.9 | |

Cox PS | 0.909 | 0.264 | 0.257 | 0.078 | 93.1 | |

| ||||||

1.5 | True | 1.535 | 0.265 | 0.261 | 0.071 | 95.2 |

Logistic PS | 1.480 | 0.237 | 0.253 | 0.057 | 96.5 | |

Cox PS | 1.382 | 0.275 | 0.254 | 0.089 | 90.7 |

Notes: Sample size 100, about 25% right-censored. Proportion of exposed: about 35%. 2,000 simulation runs. “SD” = standard deviation, “SE” = standard error, “MSE” = mean squared error.

Second scenario of simulation: every subject has a potential exposure time, some occurred after end of pregnancy.

_{1} |
||||||
---|---|---|---|---|---|---|

0.5 | True | 0.509 | 0.316 | 0.310 | 0.100 | 94.9 |

Logistic PS | 1.108 | 0.433 | 0.297 | 0.557 | 45.1 | |

Cox PS | 0.505 | 0.313 | 0.310 | 0.098 | 95.0 | |

| ||||||

1 | True | 1.022 | 0.311 | 0.301 | 0.097 | 94.5 |

Logistic PS | 1.434 | 0.376 | 0.295 | 0.329 | 63.9 | |

Cox PS | 1.005 | 0.308 | 0.301 | 0.095 | 94.8 | |

| ||||||

1.5 | True | 1.535 | 0.297 | 0.288 | 0.089 | 94.7 |

Logistic PS | 1.862 | 0.344 | 0.285 | 0.249 | 71.6 | |

Cox PS | 1.506 | 0.294 | 0.287 | 0.086 | 95.0 |

Notes: Sample size 100, about 25% right-censored. Proportion of exposed: about 63%. 2,000 simulation runs. “SD” = standard deviation, “SE” = standard error, “MSE” = mean squared error.

Overall the logistic and the Cox model propensity score approaches appear to have comparable performances under the first simulation scenario. Note that under the first simulation scenario the Cox model does not reflect the true mechanism for generating the time-dependent exposure, and neither does the logistic regression model. But under the second simulation scenario, the results from the logistic propensity score approach appear unreliable. This is perhaps due to the fact that the logistic propensity score model does not adequately capture the data generating mechanism, whereas the Cox propensity score model does. Such sensitivity of the logistic propensity score approach to the underlying data generating mechanism makes it not suitable for general use in the presence of time-dependent exposure.

A prospective cohort study of pandemic H1N1-vaccine (pH1N1)-exposed pregnancies and unexposed comparison pregnancies was carried out in order to assess the risks and relative safety of the pH1N1-containing vaccines during pregnancy [

There were 6 preterm deliveries among 160 unexposed women, and 69 preterm deliveries among 753 exposed women (

We build a propensity score for pH1N1 vaccine exposure consisting of maternal race/ethnicity, previous preterm delivery, influenza season and influenza infection, using the time-dependent covariate Cox model. Notice that in this case one cannot use the logistic regression to build a propensity score, since two of the confounders are time-dependent. From the Cox model fit we have

Left truncated Kaplan-Meier curves of one minus preterm delivery rates from the pH1N1 vaccine data.

Enrollment (

Notice that this is a time-dependent propensity score. Using it together with autoimmune disease and seasonal vaccine exposure as covariates, the adjusted hazards ratio (HR) for preterm delivery associated with pH1N1 vaccine exposure becomes 2.46 with 95% CI (1.02, 5.93). In comparison, directly adjusting for the six covariates (race/ethnicity, previous preterm delivery, influenza season, influenza infection, autoimmune disease and seasonal vaccine exposure) gives an adjusted HR of 2.47 with 95% CI (1.02, 5.98). In this second adjusted model, pH1N1 vaccine exposure, previous preterm delivery and autoimmune disease are significant predictors of preterm delivery (

Vaccination time in gestational age from the pH1N1 vaccine data.

We note that the analysis here is slightly different from the preterm delivery analysis of Chambers

In this paper we have compared the performance of two approaches to build propensity scores for time-dependent vaccine exposure during pregnancy: using logistic regression ignoring the timing of vaccine exposure, or using the Cox regression to model time to vaccine exposure. Our simulation results indicate that the Cox model approach should be preferred. In addition, the Cox model propensity score can also accommodate time-dependent confounders, such as the influenza season that we illustrated in the pH1N1 vaccine data analysis, to form a time-dependent propensity score. Several other time-dependent covariates may be relevant to vaccine safety in pregnancy studies, e.g., vaccines other than the one of primary interest, pregnancy induced hypertension/preeclampsia, time of entry into or access to prenatal care [

Time-dependent propensity score was considered in Li

Li

Some clinicians have raised concerns over potentially different effects of different timing of vaccine in pregnancy. Chambers

This project has been funded in part with Federal funds from the Office of the Assistant Secretary for Preparedness and Response, Biomedical Advanced Research and Development Authority, Department of Health and Human Services, under Contract No. HHS 10020100029C.

Ronghui Xu contributed to the design of the project and simulation studies, overseeing data analysis, and writing of the manuscript. Yunjun Luo carried out programming and data analysis. Robert Glynn contributed to the initial idea and was involved in the discussion of the manuscript. Diana Johnson, Kenneth L. Jones and Christina Chambers contributed to the collection of data used in the manuscript. Christina Chambers also contributed to the initiation and editing of the manuscript.

Ronghui Xu, Kenneth L. Jones and Christina Chambers received grant funding from GlaxoSmithKline, Novartis and CSL Limited.