Exploring Facilitators and Barriers to Initiation and Completion of the Human Papillomavirus (HPV) Vaccine Series among Parents of Girls in a Safety Net System

Objective: To assess, among parents of predominantly minority, low-income adolescent girls who had either not initiated (NI) or not completed (NC) the HPV vaccine series, attitudes and other factors important in promoting the series, and whether attitudes differed by language preference. Design/Methods: From August 2013–October 2013, we conducted a mail survey among parents of girls aged 12–15 years randomly selected from administrative data in a Denver safety net system; 400 parents from each group (NI and NC) were targeted. Surveys were in English or Spanish. Results: The response rate was 37% (244/660; 140 moved or gone elsewhere; 66% English-speaking, 34% Spanish-speaking). Safety attitudes of NIs and NCs differed, with 40% NIs vs. 14% NCs reporting they thought HPV vaccine was unsafe (p < 0.0001) and 43% NIs vs. 21% NCs that it may cause long-term health problems (p < 0.001). Among NCs, 42% reported they did not know their daughter needed more shots (English-speaking, 20%, Spanish-speaking 52%) and 39% reported that “I wasn’t worried about the safety of the HPV vaccine before, but now I am” (English-speaking, 23%, Spanish-speaking, 50%). Items rated as very important among NIs in the decision regarding vaccination included: more information about safety (74%), more information saying it prevents cancer (70%), and if they knew HPV was spread mainly by sexual contact (61%). Conclusions: Safety concerns, being unaware of the need for multiple doses, and low perceived risk of infection remain significant barriers to HPV vaccination for at-risk adolescents. Some parents’ safety concerns do not appear until initial vaccination.


Introduction
Attitudinal and logistical factors were identified early on as barriers to human papillomavirus (HPV) vaccination and, despite more than a decade of vaccine availability and recommendation, these barriers persist [1]. In studies done prior to and shortly after introduction of the vaccine in 2006, parents consistently were concerned about vaccine safety, and many expressed a low perceived risk of infection. In addition, in contrast to the other "new" adolescent vaccines at the time, tetanus-diphtheria-acellular pertussis vaccine (Tdap) and meningococcal conjugate vaccine (MCV), the need for three doses for HPV vaccine posed significant logistical challenges, as completion of the series required additional adolescent visits [2]. However, despite the promise of HPV vaccine to prevent cancer, rates of initiation and completion of the series remain low among adolescent girls nationally [3].
While barriers to HPV vaccine have been explored, there remain unanswered questions. There have been relatively few studies among parents who had actually declined or delayed vaccination. Also, no prior studies have specifically examined differences in the perspectives of those parents whose daughters had not initiated the series versus those who did not complete it. Moreover, there may be important differences in attitudes about the vaccine by ethnicity and language [4,5] that have not been fully explored. Also, black and Hispanic girls may be less likely to complete the HPV vaccine series than non-Hispanic white girls once they have received a dose [6,7], yet there is little understanding of reasons underlying this phenomenon. We undertook this study to address these gaps in the literature, and to explore and confirm findings from an earlier qualitative study by our group [8]. In that study, we found differences in the attitudes and beliefs of Spanish-speaking and English-speaking parents regarding HPV vaccine, with Spanish-speaking mothers more concerned that accepting the vaccine would encourage sex and English-speaking parents more concerned about vaccine safety. Our objectives were to assess, among parents of predominantly minority, low-income adolescent girls who had either not initiated or not completed the HPV vaccine series, attitudes and other factors important in promoting initiation versus completion of the series, and whether attitudes differed by language preference.

Study Design
We conducted a cross-sectional survey among parents of adolescent girls in a large safety net system in Denver, Colorado (Denver Health (DH)). The Colorado Multiple Institutional Review Board (COMIRB) approved this study.

Study Setting and Population
From August to October 2013, a mail survey was conducted among parents of girls aged 12-15 years randomly selected from administrative data in the DH system. DH serves a population of >17,000 adolescents annually that is predominantly Hispanic, and approximately 40% of the total patient population identify Spanish as their primary language. In 2013, at the time of the study, HPV coverage of ≥1 dose in 13-17 years old girls was 89.8% and coverage for ≥3 doses was 66.8%. The population of girls age 12-15 at the time of the study was 6848. From this population, random samples of 400 parents of girls who were identified as not having received any HPV vaccine doses (non-initiators) and 400 parents of girls who had started but not completed the HPV vaccine series (non-completers) were targeted based on information from the electronic health record. Non-initiators were adolescents with a well-child visit within the previous 2 years but no record of an HPV vaccine in DH administrative data or the state immunization registry. Non-completers were adolescents who had received 1-2 HPV vaccines within the previous 2 years with no record of a 2nd or 3rd dose within 6 or 12 months of the first dose, respectively (or record of subsequent completion of the series). These cutoffs were chosen based on discussions of the study team with CDC (Centers for Disease Control and Prevention) and Denver Health collaborators.

Data Sources
Denver Health has its own immunization registry, VaxTracks, that has been shown to be highly accurate [9]. There are daily uploads to the Colorado Immunization Information System (CIIS), Colorado's immunization registry. While reporting to CIIS is not mandated in Colorado, >90% of pediatric providers and >70% of family medicine providers report to CIIS. For this study, we first identified the eligible patient population using administrative data from Denver Health. To capture vaccinations received elsewhere, these data were then merged with CIIS data to determine the final study population. The data were extracted from DH administrative records two weeks prior to the launch of the survey and merged with data from CIIS.

Survey Design
The survey was in English or Spanish based on language of preference recorded in the medical record. The language of record is what is used for all DH communications. The survey was based on a combination of qualitative work performed by the study group within the same study population [8] and standardized survey instruments for assessment of parental HPV immunization attitudes and beliefs [10]. All parents were asked demographic questions and 28 questions regarding HPV-related attitudes using 4-point Likert scales (strongly agree to strongly disagree). Parents were also asked to report their daughter's HPV vaccination status. Parents who responded that their daughter had received 1-2 doses of HPV vaccine were asked a series of nine yes/no questions regarding reasons their daughter had not received a second or third dose followed by a modified four question decisional regret scale on a 4-point Likert scale [11]. All decisional regret questions assumed prior vaccination, such as "Vaccinating my daughter against HPV was the right decision". Parents who responded that their daughter had never received HPV vaccine were asked to rate the level of importance (very important to very unimportant) of seven factors in helping them decide whether or not to get their daughter vaccinated against HPV. Parents who reported their daughters had already completed the series did not answer questions related to non-initiation or non-completion. The survey was piloted among 6 English-and 6 Spanish-speaking parents and modified based on feedback.

Survey Administration
Parents of non-initiators (n = 400) and non-completers (n = 400) were mailed a pre-letter followed by the survey two weeks later. Non-respondents were sent a reminder postcard and up to two additional surveys over a two-week period. All surveys included cover letters describing that the survey was regarding HPV vaccine and sought to understand their thoughts about HPV vaccine to better understand how parents make decisions above HPV vaccination for their daughters. There were up to two reminder phone calls to non-respondents from a physician within DH recorded in English and Spanish. A $5 bill was included in the initial survey mailing. For surveys that were returned as undeliverable, the study team contacted staff at DH to identify those patients who had moved or gone elsewhere (MOGE) and if confirmed as unreachable, were then removed from the sample. For those undeliverable surveys not identified as MOGE, an Internet search was performed to determine new addresses if available (Intellius, Inc., Bellevue, WA, USA). Initially, 154 surveys were returned as undeliverable or identified by DH staff as MOGE and were searched using Intellius. This search yielded 33 new addresses, and of these, 19 surveys were again returned as undeliverable and 14 surveys were successfully delivered. These 14 were included in the final analysis and the remainder (n = 140) classified as MOGE.

Analytic Methods
Chi-squared test of proportions or Fisher's Exact test were used for comparisons of characteristics of respondents and non-respondents. When results regarding attitudes were stratified into non-initiators and non-completers, this was based on self-reported vaccination status rather than electronic data, as we hypothesized that parental perception of vaccination status was more relevant to attitudinal responses than actual vaccination status. Results stratified by electronic data are available in an online Appendix A. All statistical analyses were performed using SAS software (SAS 9.3, SAS Institute, Cary, NC, USA).

Response Rates and Study Sample
The overall response rate was 37% (244/660; 140 MOGE). Respondents were similar to non-respondents in terms of age of the child and language preference (the only variables available for comparison). Overall, 83% of respondents reported being the patient's mother, 7% the father, 7% the legal guardian, and 3% other. As shown in Table 1, non-initiators were more likely than non-completers to have a college degree, to be White, and to have an income >$50,000. Sixty-six percent of respondents took the survey in English and 34% in Spanish. Respondents to the Spanish-language survey were more likely than respondents to the English-language survey to have public insurance, have less than a high school education, be Hispanic and have income <$20,000/year.

Differences in Electronic Data and Self-Reported Vaccination Status
There were significant differences in self-reported vaccination status and vaccination status determined electronically (Figure 1). Based on electronic data, 50% (n = 122) of respondents were non-initiators and 50% (n = 122) were non-completers. Among those who self-reported as non-initiators (n = 131), 69% were non-initiators based on electronic data and 31% were non-completers. Among those who self-reported as non-completers (n = 66), 73% were non-completers based on electronic data and 27% were non-initiators. Six percent of electronically identified non-initiators (n = 7) and 25% of electronically identified non-completers (n = 31) reported they had completed the series.

Attitudes Regarding HPV Infection and HPV Vaccine
Overall, 62% of parents disagreed that their daughter was at risk for HPV infection. Regarding specific reasons, most parents agreed that their daughter was not at risk for HPV infection because she was not having sexual intercourse (71%). Twenty-four percent of parents agreed that their daughter was only at risk of HPV if she had premarital sex. Few parents (10%) agreed that it was better to get HPV infection than vaccination, or that HPV is not a problem because it can be treated (13%) or because few people have cervical cancer (9%).
There were some notable differences in attitudes regarding HPV infection and HPV vaccine between self-reported non-initiators and non-completers ( Figure 2). Non-initiators were more likely than non-completers to disagree with the statement "My daughter is at risk for HPV infection" and more likely to agree that "My daughter is not at risk, she is currently not having intercourse". Parents of non-initiators also had more safety concerns and were less likely to agree that the vaccine is good for protecting their daughter's health or at preventing cervical cancer. Non-initiators were also more likely to endorse that the HPV vaccine is too new and they wanted to wait before vaccinating their daughter.

Attitudes Regarding HPV Infection and HPV Vaccine
Overall, 62% of parents disagreed that their daughter was at risk for HPV infection. Regarding specific reasons, most parents agreed that their daughter was not at risk for HPV infection because she was not having sexual intercourse (71%). Twenty-four percent of parents agreed that their daughter was only at risk of HPV if she had premarital sex. Few parents (10%) agreed that it was better to get HPV infection than vaccination, or that HPV is not a problem because it can be treated (13%) or because few people have cervical cancer (9%).
There were some notable differences in attitudes regarding HPV infection and HPV vaccine between self-reported non-initiators and non-completers ( Figure 2). Non-initiators were more likely than non-completers to disagree with the statement "My daughter is at risk for HPV infection" and more likely to agree that "My daughter is not at risk, she is currently not having intercourse". Parents of non-initiators also had more safety concerns and were less likely to agree that the vaccine is good for protecting their daughter's health or at preventing cervical cancer. Non-initiators were also more likely to endorse that the HPV vaccine is too new and they wanted to wait before vaccinating their daughter.

Reasons for Non-Completion and Decisional Regret
Thirty-eight parents reported their daughters had completed the series and were skipped out of questions regarding reasons for non-completion and decisional regret. Among those parents who reported that their daughters had initiated but not completed the HPV vaccine series (n = 40 had started but not completed, n = 11 had started but uncertain of completion), the most commonly reported reasons for not having completed the series were reporting not being due for the next dose yet (52%), not knowing more doses were needed (42%), and reporting that they had not initially had safety concerns about HPV vaccine, but do now (39%). Few parents regretted the decision to vaccinate their daughter (21%) or thought that the HPV vaccine caused their daughter harm (8%). Among these parents who reported their daughter had started but not finished the series ("don't know/not sure" did not answer this question), 39 out of 40 (98%) reported they planned to complete the series.

Influences on the Decision to Vaccinate
Parents who reported that their daughter had not started the HPV series were asked to rate the importance of several possible influences on the decision of whether or not to vaccinate their daughter against HPV ( Table 2). The influences that the highest percentages of parents reported as very important included "If I knew that HPV vaccine can keep my daughter from getting cancer when she is older" (75%), "If there was more information about vaccine safety available" (74%), and "If there were more information saying that the vaccine works really well at preventing cancer" (70%). Of note, of the seven factors queried, "If my daughter's health care provider strongly recommended it" was the factor least often rated as very important (47%).

Reasons for Non-Completion and Decisional Regret
Thirty-eight parents reported their daughters had completed the series and were skipped out of questions regarding reasons for non-completion and decisional regret. Among those parents who reported that their daughters had initiated but not completed the HPV vaccine series (n = 40 had started but not completed, n = 11 had started but uncertain of completion), the most commonly reported reasons for not having completed the series were reporting not being due for the next dose yet (52%), not knowing more doses were needed (42%), and reporting that they had not initially had safety concerns about HPV vaccine, but do now (39%). Few parents regretted the decision to vaccinate their daughter (21%) or thought that the HPV vaccine caused their daughter harm (8%). Among these parents who reported their daughter had started but not finished the series ("don't know/not sure" did not answer this question), 39 out of 40 (98%) reported they planned to complete the series.

Influences on the Decision to Vaccinate
Parents who reported that their daughter had not started the HPV series were asked to rate the importance of several possible influences on the decision of whether or not to vaccinate their daughter against HPV ( Table 2). The influences that the highest percentages of parents reported as very important included "If I knew that HPV vaccine can keep my daughter from getting cancer when she is older" (75%), "If there was more information about vaccine safety available" (74%), and "If there were more information saying that the vaccine works really well at preventing cancer"

Differences in English-and Spanish-Speaking Parents
Items where there were statistically significant differences between English-(ES) and Spanish-speaking (SS) parents of non-initiators are shown in Table 3. Among non-initiators, there were a number of differences in attitudes regarding risk of HPV infection, most notably agreement that their daughter was at risk for HPV infection (ES, 24%, SS, 53%). Regarding attitudes about HPV vaccine, ES non-initiators were more likely than SS to agree that HPV is pushed by health care providers to make money (ES, 41%, SS, 11%) and that it might cause short term side effects (ES, 83%, SS, 56%) and less likely to agree that it is good for protecting their daughter's health (ES, 63%, SS, 82%). There were fewer differences between ES and SS non-completers, with SS more likely to agree that their daughter is at risk for HPV infection (ES, 46%, SS, 67%, p = 0.02). Among non-completers who answered that their daughter had received 1 or 2 doses of HPV vaccine, more SS parents reported that they did not think it was important to get all 3 HPV shots (ES, 20%, SS 52%, p = 0.01) and that "I wasn't worried about the safety of the HPV vaccine before, but now I am" (ES, 23%, SS, 50%, p = 0.04).

Discussion
We report the results of a survey regarding attitudes related to HPV vaccine conducted in an underserved population at high risk for HPV-related disease. We show that safety concerns continue to be a barrier to HPV vaccination, and that there are differences in attitudes of parents who report their daughters have started the series compared to those who have not. We also present new information showing that parents whose daughters have started but not finished the series may acquire safety concerns after initiating the series. Finally, our data show important differences in the reasons for non-initiation and non-completion between Spanish-and English-speaking parents.
Although concerns from parents about the safety of HPV vaccine are not new, two findings from our analysis are cause for concern. First, in 2013, at the time of this survey and a full seven years after licensure of HPV4, there was a great deal more safety data available than there was in the first few years [12][13][14], yet many parents still raised concerns about safety despite extensive efforts by CDC, AAP (American Academy of Pediatrics) and others to publicize the vaccine's safety record [15,16]. This implies that despite concerted efforts to inform the public about HPV vaccine safety, these messages are not reaching or are not effectively reassuring many parents. Secondly, many parents whose daughters had started the series reported that they had safety concerns only after starting the series, although it is reassuring that most still planned to complete the series. These parents theoretically had had an opportunity to be educated about the safety of the vaccine at the time of the initial vaccination. It is unknown how those concerns developed, but this finding deserves further exploration. In our prior qualitative study, some English-but not Spanish-speaking parents reported that their concerns about HPV vaccine safety developed after their daughters had received the first dose when they began to read or hear things that encourage doubt [8]. With this survey, we confirm that safety concerns developing after initiation is not uncommon, and that it is not limited to English-speakers. This finding is also counter to the supposition that reasons for non-completion of the series are only logistical, whereas reasons for non-initiation are more focused on safety. Our data suggest that efforts to get adolescents to complete the series will involve not only solving logistical issues but also the need for ongoing messaging about the vaccine's safety among initiators. Another recent study found that child's fear of needles was a significant reason for non-completion [17], and it may be that parents considered this fear of needles a safety concern since we did not ask about this explicitly.
Responses among parents who reported their daughters had not started the HPV series provide some focus for vaccine advocates, researchers, and health educators in the creation of future educational interventions to improve vaccination uptake. Similar to prior work, parents request more information about the vaccine's safety. In addition, it appears there should be more emphasis on cancer prevention. We were somewhat surprised that more parents did not endorse a strong provider recommendation as "very important", since much of the current messaging around increasing uptake of HPV vaccine is focused on the importance of the provider's recommendation. A strong provider recommendation was not even in the top five of the possible strategies considered "very important" in the decision to receive the HPV vaccine. While provider recommendation has consistently been shown to influence HPV vaccine uptake [1,18,19], our findings are consistent with a prior study among a predominantly publicly insured population in Florida which showed that parent perception of HPV vaccine safety was strongly influential of vaccine receipt [20]. This finding may also be related to our methods: by providing a list of possible reasons, we may have brought up reasons parents may not have otherwise thought about. For example, the National Immunization Survey-Teen (NIS-Teen) from the same year asked about the most important factors in determining when a teen will receive HPV vaccine in an open-ended fashion, and the most commonly reported reason was lack of provider recommendation [21].
An actionable finding from our study, and consistent with our recently published qualitative study [8], is the need to reinforce the message that multiple doses are needed to complete the HPV series among Spanish-speaking parents. In the NIS-Teen, which did not categorize the study population by race/ethnicity, 15% of parents of partially vaccinated girls reported "clinician did not recommend" or "did not know additional shots needed" as a reason for non-completion [21], which is similar to our finding of 20% among English-speakers (versus over half of Spanish speakers). It is also worth noting, though, that the vast majority of parents in the NIS-Teen were not aware that three doses were required at the time of this study. In our study, the reason why knowledge of the need for additional doses differed between Spanish and English-speaking parents is unclear. Most providers within Denver Health, where the study was conducted, speak Spanish, and patient handouts, including educational handouts and Vaccine Information Statements, are available in both English and Spanish. This finding could indicate providers are communicating the need for subsequent doses in English to the English-speaking adolescent, assuming either that the Spanish-speaking parent understands or that the adolescent will tell them if they do not. It is also possible that even though many of the providers at Denver Health speak Spanish, most are not native speakers and may not be comfortable discussing HPV vaccine with the nuance they feel is required. A recent qualitative study offers evidence that this problem is not unique to Denver Health. Investigators in New York found that the primary barrier among Spanish-speaking immigrants was lack of any provider recommendation for the vaccine [22]. Another recent qualitative study from Utah showed that there was confusion about the need for three doses among Spanish-speaking parents [23]. In any case, further work should explore if our finding regarding lack of knowledge regarding completion of the series is common in other settings. Providers should make clear to all parents, English-and Spanish-speaking, that three doses are required for completion of the series. Similarly, with a 2-dose schedule now recommended for those who begin the HPV vaccination series before age 15 [24], the importance of completing two doses will need to be emphasized. It should also be noted that with this study, we were able to examine differences between English-and Spanish-speaking parents, but did not have the power to examine differences by race/ethnicity. Future work should examine attitudes regarding HPV vaccine in specific populations, as there may other important unexplored differences.
Electronic data and self-report are often used to assess vaccination status in research studies. The differences we found between electronic data and self-report have several potential interpretations and implications. For those we identified electronically as non-initiators or non-completers but whose parents reported initiation or completion, respectively, there are at least three possibilities: that their daughters had received the vaccine after the data were pulled but before they answered the survey; that their daughters had received the HPV vaccines but these were not captured electronically; or that they were mistaken and their daughters had not received the vaccines. This last possibility seems less likely as prior work has shown that over-reporting of HPV vaccination is unusual [25]. For those parents who identified their daughters as non-initiators but we identified as having received one or two doses-about 1/3 of the self-reported non-initiators-the interpretation is perhaps more straightforward: it is likely these parents simply did not know their daughter had received the vaccine. It is also possible, however, that some of these parents perceived stigma from admitting to their daughters' receipt of a vaccine for a sexually-transmitted infection, even if in an anonymous survey. Our findings were similar to the aforementioned study comparing self-reported vaccination status to medical record review, where among 66 mothers whose daughters had received the HPV vaccine according to medical records, 16 reported that they had not [25]. While one could speculate on why this might be, the implication from these data is that researchers should be careful drawing conclusions based only on parental report of vaccination status.
This study had several strengths and limitations. It is complementary to our recently published qualitative study within the same population [8], confirming several findings that are actionable. This study was also among populations at high risk of not completing the series and of cervical cancer. However, the study focused on one safety net system, and therefore may not be generalizable. Also, survey respondents may have differed from non-respondents. Further, despite extensive efforts, our response rate was sub-optimal, although similar to other surveys in low SES populations. These results must be considered in that context, and further work may be necessary to confirm these findings. This survey was also administered in 2013, so some of these findings may have changed in the interim. In addition, although we asked about educational attainment, we did not measure the literacy rate in the study population, so it is possible that some respondents did not fully understand the questions. It is also unknown if the person answering the survey was the most knowledgeable about the adolescent's vaccination status, which may explain some of the differences we found between electronic data and self-report. Finally, we performed this study only among girls, and parents of boys may have different attitudes and practices.

Conclusions
Ongoing safety concerns and low perceived risk of HPV infection remain significant barriers to HPV vaccination among at risk adolescents. Spanish-speaking parents appear to have different barriers than English-speaking parents for initiation of the HPV vaccine, and a concerning proportion of Spanish-speaking parents whose daughters start the series are unaware of the need for subsequent doses. Many parents develop safety concerns about HPV vaccine after starting the series. Providers and health educators should emphasize cancer prevention and vaccine safety when counseling parents and developing educational materials.

Acknowledgments: This study was funded by the Centers for Disease Control and Prevention (Grant #3U01IP000310-02S1).
Author Contributions: Sean T. O'Leary conceptualized and designed the study, drafted the initial manuscript and subsequent revisions, and approved the final manuscript as submitted. Steven Lockhart coordinated and supervised data collection, critically reviewed the manuscript, and approved the final manuscript as submitted. Juliana Barnard assisted with study design and data collection, critically reviewed the manuscript, and approved the final manuscript as submitted. Anna Furniss carried out the initial analyses, reviewed and revised the manuscript, and approved the final manuscript as submitted. Miriam Dickinson conceptualized and designed the study, supervised the data analysts, reviewed and revised the manuscript, and approved the final manuscript as submitted. Amanda F. Dempsey conceptualized and designed the study, supervised the data analysts, reviewed and revised the manuscript, and approved the final manuscript as submitted. Shannon Stokley conceptualized and designed the study, critically reviewed the manuscript, and approved the final manuscript as submitted. Steven Federico conceptualized and designed the study, critically reviewed the manuscript, and approved the final manuscript as submitted. Michael Bronsert carried out the final analyses, reviewed and revised the manuscript, and approved the final manuscript as submitted. Allison Kempe conceptualized and designed the study, oversaw all aspects of the study, reviewed and revised the manuscript, and approved the final manuscript as submitted.

Conflicts of Interest:
The authors declare no conflict of interest. Amanda Dempsey serves on advisory boards for Merck, Pfizer and Sanofi Pasteur. She does not receive any research funding from these groups nor did they play a role in this study. None of the other authors have any financial disclosures.

CI
Confidence Interval AAP American Academy of Pediatrics CDC Centers for Disease Control and Prevention Appendix A Table A1. Attitudinal differences between electronically identified non-completers and non-initiators.