An Integrated Framework to Conceptualize and Develop the Vancouver Airways Health Literacy Tool (VAHLT)

There is currently no comprehensive tool to assess the functional health literacy (HL) skills of chronic airway disease (CAD) patients. The purpose of this article is to describe the development of a new HL measure, the Vancouver Airways Health Literacy Tool (VAHLT). The tool was developed through the following phases: (1) Tool conceptualization, consisting of: (A) a systematic review (SR), (B) focus group sessions with CAD patients to understand barriers and facilitators to CAD management, (C) a survey with key-informants to obtain strategies to mitigate self-management barriers and validate patient-derived topics, and (D) respiratory physicians’ review of the topics; (2) Scenario and item development; and (3) Tool testing and content validation. The SR identified the lack of a valid HL measurement tool for CAD patients. Patients provided an initial shortlist of disease-related self-care topics. Key-informants helped to finalize topics for inclusion. Respiratory physicians and patients contributed to the development of a scenario-based questionnaire, which was refined during three rounds of testing to develop a 44-item instrument comprising nine self-management passages. We highlight the holistic process of integrating information from the literature with knowledge gained from key stakeholders into our tool framework. Our approach to stakeholder engagement may be of interest to researchers developing similar tools, and could facilitate the development and testing of HL-based interventions to ultimately improve patient outcomes and reduce the burden on the healthcare system.


Introduction
Health literacy (HL) is defined by the Canadian Expert Panel on HL (CEPHL) [1] and Calgary Charter on HL (CCHL) [2] as having five core domains encompassing "an individual's capability to access, understand, communicate, evaluate, and use health information and care services to make informed decisions for one's health and well-being". Numeracy is also considered to be a HL skill [3]; however, it is typically assessed across the HL domains, rather than as an independent entity [4]. In Canada, over 60% of the adult population (≥16 years of age) and 88% of seniors (>65 years of age) have insufficient abilities to manage their health or make informed health-related decisions [5][6][7]. Low HL is associated with poor health outcomes and excess healthcare spending in a variety of chronic health conditions [8,9], including asthma and COPD, the two main chronic airway diseases (CAD) [10][11][12][13][14]. HL challenges in CAD management are associated with unnecessary hospitalizations and emergency department visits [15,16], reduced medication adherence [17,18],

Theoretical Grounding and Rationale
A validity-driven approach [48] was employed in the development of the VAHLT. We applied the five-domain HL definition suggested by CCHL [2], which is widely used in the HL field [38].
In 2008, the CEPHL recognized the need to improve HL measures, as well as empower patients to properly manage their disease [1,52]. In 2013, we convened a week-long international workshop to ensure that our research plan aligned with HL research priorities, while integrating patients', healthcare professionals', and researchers' perspectives [53,54]. The concept of the Chronic Care Model (CCM) [55] (Figure 1) was applied as a framework to facilitate participants' engagement in discussing HL and related disease-management topics. Attendees identified important aspects of social context and its influence on HL, including barriers to navigating the health system and actively engaging in self-management practices, challenges among ethno-cultural and marginalized communities, and the role of healthcare professionals and systems in addressing HL gaps between patient population groups. The CCM model was applied throughout our tool-development process. One critical outcome of this engagement was endorsement of the need for a performance-based HL tool to facilitate better CAD management. Our proposed initiative also aligned with the CEPHL's emphasis on conceptualizing HL in chronic disease management and the changes in communication strategies, clinical practice, research agendas, and policy procedures required to improve Canadians' HL [1,52]. Additionally, our initiative was enriched through knowledge gained during various qualitative and interventional HL and disease management studies with asthma and COPD patients [56][57][58][59][60]. This enabled us to identify patient challenges across HL domains, such as understanding verbal/written instructions and accessing information and services.
international workshop to ensure that our research plan aligned with HL research p ties, while integrating patients', healthcare professionals', and researchers' perspec [53,54]. The concept of the Chronic Care Model (CCM) [55] (Figure 1) was applied framework to facilitate participants' engagement in discussing HL and related dis management topics. Attendees identified important aspects of social context and its i ence on HL, including barriers to navigating the health system and actively engagi self-management practices, challenges among ethno-cultural and marginalized com nities, and the role of healthcare professionals and systems in addressing HL gap tween patient population groups. The CCM model was applied throughout our too velopment process. One critical outcome of this engagement was endorsement of the for a performance-based HL tool to facilitate better CAD management. Our propose itiative also aligned with the CEPHL's emphasis on conceptualizing HL in chronic di management and the changes in communication strategies, clinical practice, rese agendas, and policy procedures required to improve Canadians' HL [1,52]. Addition our initiative was enriched through knowledge gained during various qualitative an terventional HL and disease management studies with asthma and COPD patients 60]. This enabled us to identify patient challenges across HL domains, such as unders ing verbal/written instructions and accessing information and services.

Study Procedures
The tool framework development process comprised three interrelated phas consultation with key-stakeholders, item identification and refinement, and testing steps undertaken during the study period (2015-2020) are outlined in the following tions and summarized in Figure 2.

Study Procedures
The tool framework development process comprised three interrelated phases of consultation with key-stakeholders, item identification and refinement, and testing. The steps undertaken during the study period (2015-2020) are outlined in the following sections and summarized in Figure 2.

Phase I: Tool Conceptualization
The goal of this phase was to identify disease-management topics representing content areas for the tool. To ensure content relevance and comprehensiveness, we obtained input from the literature and engaged both patients and professionals (Figure 3).
We conducted an SR of measures used to evaluate the HL skills of patients with CAD. In parallel, we established an advisory panel ("knowledge hub") that included patient partners, national and international HL experts, and respiratory specialists. Patients were recruited through clinical/network referrals and experts via snowball sampling [61] initiated by a focused search for multi-disciplinary scientists with expertise in HL and chronic disease management [62].
Next, we hosted focus group sessions with English-and French-speaking CAD patients from six collaborating sites across Canada. The goal of the group sessions was to understand patients' perspectives on various aspects of CAD self-management. Participants were asked to talk about barriers and challenges pertaining to the five HL domains (including numeracy), as well as facilitators of optimal disease-management practices. They discussed disease-management topics, including their perceived importance, and provided questions for each topic that they would ask healthcare professionals. The study procedures, including focus-group structure and qualitative analysis, have been described in-depth in previous publications [43,45,47,49,50,62].

Phase I: Tool Conceptualization
The goal of this phase was to identify disease-management topics representing content areas for the tool. To ensure content relevance and comprehensiveness, we obtained input from the literature and engaged both patients and professionals (Figure 3).
We conducted an SR of measures used to evaluate the HL skills of patients with CAD. In parallel, we established an advisory panel ("knowledge hub") that included patient partners, national and international HL experts, and respiratory specialists. Patients were recruited through clinical/network referrals and experts via snowball sampling [61] initiated by a focused search for multi-disciplinary scientists with expertise in HL and chronic disease management [62].
Next, we hosted focus group sessions with English-and French-speaking CAD patients from six collaborating sites across Canada. The goal of the group sessions was to patients and professionals were analyzed using NVivo software (QSR International, version 12) with coding via thematic analysis [63]. We have reported the process and findings of the informant interviews and respiratory experts' input elsewhere [30,43,46,62]. Knowledge generated from the SR and consultations with patients and professionals allowed us to identify and verify important disease-management topics, and understand the internal stimuli (e.g., beliefs, worldviews) and external barriers (e.g., health system and socioeconomic factors) to engage in CAD self-management [48].

Phase II: Scenario and Item Development (Preliminary Version Development)
Initially, we analyzed a list of potential factors (e.g., barriers and facilitators across the five core HL domains and numeracy) that were suggested by participants in Phase I. We then developed a preliminary set of scenarios for our measurement tool. Passages were presented in different formats (e.g., writing, pictures, diagrams, maps, website Subsequently, we conducted in-person or telephone/Skype interviews with national and international key-informants to review our framework and document their perspectives on our proposed process. We aimed to obtain professional insights about the role of HL in CAD management, elicit strategies to mitigate the challenges/barriers expressed by the focus-group participants, and verify the disease-management topics proposed by the patients. Following interviews, expert respiratory clinician-scientists from across Canada reviewed the patient-identified topics and shared optimal disease-management skills that a CAD patient should have to properly manage their condition. The data collected from patients and professionals were analyzed using NVivo software (QSR International, version 12) with coding via thematic analysis [63]. We have reported the process and findings of the informant interviews and respiratory experts' input elsewhere [30,43,46,62]. Knowledge generated from the SR and consultations with patients and professionals allowed us to identify and verify important disease-management topics, and understand the internal stimuli (e.g., beliefs, worldviews) and external barriers (e.g., health system and socioeconomic factors) to engage in CAD self-management [48].

Phase II: Scenario and Item Development (Preliminary Version Development)
Initially, we analyzed a list of potential factors (e.g., barriers and facilitators across the five core HL domains and numeracy) that were suggested by participants in Phase I. We then developed a preliminary set of scenarios for our measurement tool. Passages were presented in different formats (e.g., writing, pictures, diagrams, maps, website screenshots, charts, forecasts, etc.) and took different approaches (e.g., role playing/story-related format, writing answers in sentences, check-in, or pictorial and educational informative statements), just as one would encounter in a real-world healthcare setting. We then developed questions (items) for each scenario. The item bank was refined by assessing and eliminating repetitions, and integrating patients' and professionals' recommendations from the previous phase. Each item/question was structured in a multiple-choice format (four option answers) with only one correct answer. Next, the scenarios and corresponding questions were presented to selected patients and key-informants from the advisory panel to review the content, including each scenario and item's perceived relevance to disease management (both patients and professionals) and HL domains (only HL researchers). We sought a consensus among patients and professionals on the confirmed topics and items to be included in the tool. Further revisions were applied to the tool's format to ensure the inclusion of important disease-management topics and all HL domains and numeracy, and further elimination of unnecessary or repeated items.

Phase III: Tool Testing and Content Validation
To confirm the content validity of the VAHLT, the revised tool was pre-tested in 2017 with a new cohort of CAD patients and selected key-informants to obtain a final consensus regarding the tool's overall layout in terms of the scenarios, the order of topics, and comprehensiveness of the item pool to measure the core HL domains [49]. Patients completed the questionnaire in person and were instructed to write their comments and suggestions about the tool, including any difficulties in understanding the scenarios, as well as its perceived relevance to CAD management. The key-informants received a digital copy of the tool and were informed about the procedures to provide feedback (e.g., rating of items in terms of relevance to CAD disease management and the HL domains, and the difficulty level of the questions).
Finally, a professional adult patient educator conducted a thorough evaluation of the tool to ensure that the content was user-friendly, improved the layout and orientation of certain items and diagrams, and refined the wording to an appropriate reading level. The aim was to ensure that our proposed tool was capable of distinguishing differences that were attributable to HL skills as opposed to other factors, including prior disease-related knowledge, or complex terms and medical jargon. The results of this phase have been reported in separate publications [43,49,50].

Subsequent Testing of VAHLT
Following Phases I, II, and III, we tested the VAHLT in two cohorts of CAD patients recruited from six collaborating sites across Canada to reduce the items and assess the measurement properties of the final tool. These studies will be reported in future publications.

Ethics
Study approval was obtained from the University of British Columbia Clinical Research Ethics Board (project approval number: H15-01954). Ethics approval was also received from the Institutional Review Boards at each of the participating centers prior to recruitment. Written informed consent was obtained from all subjects involved in the study.

Systematic Review
Our SR [33] identified major deficiencies in currently available screening-based HL instruments, and confirmed the need for a functional HL tool for CAD patients. The key findings pertaining to our tool development were: (1) the five core HL domains outlined by the CCHL [1,52] were underrepresented among the tools; (2) overreliance on the "understand" HL domain was common, indicating the potential misuse of word comprehension as a proxy for HL skills; and (3) use of a self-evaluative design as the dominant approach, meaning patients were assessed based on the indication of their perceived level of competence to act in a given health scenario, as opposed to actually completing the task whilst displaying their functional HL skills. Additionally, numeracy assessment was absent from most articles, and no HL measurement tool had been developed specifically to assess the functional HL skills of adult CAD patients. Our findings provided two main insights. First, the review identified the need to assess HL in a complete manner (accounting for all domains). Second, it highlighted the importance of having patients complete realistic health-related tasks to accurately gauge their HL capacities (rather than self-evaluate)-a fundamental aspect of our performance-based tool.

Patient Involvement
The SR identified a lack of patient engagement in the development of most existing HL measurement tools. This is a significant limitation, with potentially serious implications for the validity of the resultant instruments [33,38,43,64,65]. During 16 focus-group sessions with CAD patients, participants mentioned challenges in accessing relevant information, difficulty understanding treatment instructions, limited ability and inadequate knowledge to act on information, and the constraints of daily demands in managing their disease. These data are discussed in detail elsewhere [46,62]. Patients' responses were analyzed across the five core HL domains and included the following key insights into the design of our tool: (1) Access: participants suggested clarifying the different ways to access and obtain health information and services. They defined "passive access" as the availability of health information provided by healthcare professionals, while "active access" was identified as patients navigating and finding needed information independently. "Lack of access" was also identified as a major barrier to receiving needed health information and care services. (2) Understand: patients' perception of understanding is often related to their difficulty in comprehending the complexity of the information, particularly treatment instructions (i.e., how to adhere), triggers of disease worsening (i.e., how to manage), and health education materials (i.e., how to apply). Medical terms/jargon used by healthcare professionals or found in written material contributed to patients' inability to understand health information. Finally, the participants talked about challenges in grasping numerical aspects of their action plan, tests results, or instructions received from care providers that prevented them from fully comprehending and following the information. (3) Evaluate: healthcare professionals' credibility made them the most trustworthy information source for many participants. Several provider-related factors were cited when patients evaluated the trustworthiness of the information received from a care provider: communication skills, confidence level, professional reputation, and specialization/credentials. When evaluating the validity of sources, participants mentioned that they judged information based on the source and whether they deemed it to be reputable and correct. (4) Communicate: descriptions of communications were often centered on improper or inadequate patientprovider interactions. Patients mentioned a lack of proper information exchange between a care provider and a patient, including managing an exacerbation and treatment side-effects. Similarly, challenges were identified in treatment adherence, as many patients noted that they did not always adhere to their prescribed treatments, often due to miscommunications with care providers. (5) Use/Apply: participants mentioned two main factors that they believed promote the use of health information: (1) that it was from a credible source, and (2) receiving the information from a care provider when they are ready to accept it in a suitable setting.

Key-Informants Involvement
Forty-five key informants provided their perceived solutions to the barriers reported by patients. They noted that care providers should not only help patients to understand disease-and treatment-related information, but also to support their engagement in the self-management process by responding to their needs and concerns. They emphasized the provision of easy-to-follow instructions, increased information accessibility, and promotion of proper communication practices to enhance information uptake, all of which can enhance the patient's confidence to apply the received information in making informed health decisions [66].

Respiratory Clinician Validation
Seventeen respiratory physicians reviewed the disease-management topics and provided additional "actionable" items, which emphasized combining HL skills and selfmanagement competencies for CAD patients to properly manage their disease [67]. The physicians agreed that to accurately measure HL, a patient should be placed in a situation consistent with their daily routine that would require them to act upon self-management information in a realistic scenario. This approach would allow us to measure performancebased skills, as opposed to a self-evaluation, and was consistent with findings from the SR [30] and suggestions made by key-informants [40,45,46].

Phase II: Scenario and Item Development (Preliminary Version Development)
Data from patients and key-informants helped us to identify, verify, and prioritize the most important disease-management topics relevant to both asthma and COPD patients. Participants frequently emphasized the need to enhance some scenarios with graphical and pictorial features, and also to integrate numeracy testing into the tool. It was noted that numeracy is a critical part of a patient's decision-making process, as health and medical information is often presented as numbers, graphs, or tables [3, 4,42,50,68]. In developing the VAHLT, we considered 159 potential disease-management elements suggested by patients and experts to develop a short list of the items by eliminating irrelevant and duplicated items. We recruited a further five asthma and five COPD patients in Vancouver, who had not previously participated in the study, to help develop the preliminary content of scenarios for each selected disease-management topic, as well as to suggest items pertaining to each topic from the item bank. Eventually, a draft tool was developed, that included 12 topics specific to asthma and COPD (eight shared + two asthma + two COPD) and 66 corresponding items (four to six items for each scenario) ( Table 1). Each question tested one of the five core CCHL [2] domains and numeracy ( Table 2). Appendix A contains a sample measurement-tool passage.

Diet and Nutrition Both
You are invited to a restaurant for a dinner event. You noticed that the entire dinner menu is heavy and greasy. You want to enjoy the dinner with your friends but you also want to make the right decisions considering the tips you received from the dietitian.

Access
After visiting the lung doctor at the Lung Centre, you need to pick up your medications at the Pharmacy. Using the Map scenario, which of the following 2 places would you pass by if you took the shortest route from the Lung Centre to the Pharmacy? (Pick only one).

Phase III: Tool Testing and Content Validation
We pre-tested the tool with patients and professionals (recruited from our knowledge hub) in two rounds. First, input from 19 CAD patients and 26 key-informants resulted in refinement of the scenarios and confirmed the inclusion of the selected topics and corresponding multiple-choice questions in our tool. Second, the revised tool was pretested with a new cohort of 75 CAD patients and 39 key-informants, who provided a final round of feedback regarding the tool's relevance, difficulty, and practicability. The participants were positive about the relevance and appropriateness of the self-management topics (patients), layout of the scenarios (patients and key-informants), inclusion of all core HL domains and numeracy, and appropriateness of the multiple-choice scaling approach (patients and key-informants). Notably, participants deemed the scenarios to be informative and helpful in improving a patient's understanding of disease-management practices. Based on the initial results, a tool consisting of 44 items and nine topics (seven shared + two disease-specific for asthma and COPD) was approved for a further psychometric validation (to be published elsewhere).

Discussion
We aimed to develop the VAHLT by combining patients' perspectives with the viewpoints of professionals on key factors affecting CAD patients' HL and self-management practices. Their involvement also helped to improve the VAHLT during multiple rounds of testing and feedback in terms of layout, content relevance, and applicability to CAD management. This article describes the process and framework we applied in the development of our HL measurement tool for CAD management.
During the development process, we identified several deficiencies in currently available HL tools. One such limitation was overreliance on self-evaluation and a patient's perceptions of their HL skills, which are often inaccurate. In contrast, our performancebased tool was designed to measure HL abilities by asking participants to respond to questions based on information provided within each disease-management scenario. The aim of the VAHLT is to measure objective components of disease management, while minimizing potential biases associated with self-perceived or screening-based HL measures [38,42,52].
We found that patient involvement in prior HL tool-development processes was rare [33,48,49,51]. Our participants also mentioned, from their experience, that patients' voices were often missing when developing educational interventions, particularly as related to disease-management practices [45,60,69]. In contrast to previously reported HL measurement tools [64,65,70,71], our community-engagement model [49,50,72] drew extensively on patients' and health professionals' insights to inform the process throughout its development. Evaluation of our community-engagement model supported its effectiveness [55,72], with implications for future initiatives in functional HL tool development. Patient participants were generally happy that their viewpoints were appreciated, and provided suggestions that were later integrated with those of healthcare professionals and HL researchers [43,62].
During our study, we identified numerous internal and external factors that prevent proper access to and use of health information. Our study participants indicated that for some Canadian population groups, including those with limited English and/or French language proficiency, low levels of general literacy, and living in rural areas, obtaining adequate access to health information is often difficult. Considering that access is dependent not only on the availability of information, but also on the way it is presented, we based our scenarios on CAD patients' routine practices and applied plain language and pictorial components, when applicable. We considered the "Understand" skill as a function of cognitive and general literacy skills, as well as previous knowledge and experiences within the disease-management process, and drafted our scenarios and questions accordingly. For the "Evaluation" skill, we prepared two scenarios for each topic, which were similar in content but differed in format/layout, credibility of the sources, and relevance to general population vs. asthma/COPD. The aim was to assess patients' ability to judge the information in paired scenarios based on their relevance, trustfulness, comprehensiveness, and applicability to their disease management. Although some of the qualities are somewhat subjective, the two scenarios were kept similar, but had some significant differences that allowed the participants to respond to the question. For the "Communication" skill, we focused largely on patient and care provider interactions, and designed our scenarios and questions to assess the patient's ability to ask necessary questions and express their concerns and opinions. For the "Use" skill, we assessed patients' abilities and motivation to apply the information in self-management practices. Lastly, for the "Numeracy" skill, we included numbers in both scenarios and questions to assess participants' abilities to apply numerical information in their decision-making processes.
Patients' and health professionals' involvement in the conceptualization of our tool allowed us to confirm that a performance-based HL tool, based on the characteristics mentioned above, is needed. As a secondary benefit, such a tool could help to identify the patient's HL gaps to facilitate implementation of interventions to address such gaps [73][74][75][76][77].
Our study had limitations. The VAHLT was only evaluated with English-speaking patients, and therefore requires validation in other languages. In addition, we recruited participants from a specialty care setting, where patients may have possessed different disease-specific knowledge or perceptions than CAD patients from the general population. To address this issue, we have now collected additional data from patients recruited from the community, and expect to be able to confirm the generalizability of our instrument. Furthermore, we simplified the reading level of the VAHLT to a 5th-grade level to ensure that differences in score were attributable to actual CAD-related functional HL skills and not general literacy, prior disease-related knowledge, or knowledge of medical terminology. However, a reduction of the reading level could affect the assessment of real-world abilities, where confusing or complex terminology/language is commonly seen by patients in health-system encounters [76][77][78][79]. Finally, the length of our measurement tool (nine topics, 44 questions) with an average completion time of approximately 45 min decreased the feasibility for the current version of the tool to be used in clinical practice.

Conclusions
In this paper, we described the integrated approach taken to develop the VAHLT. We employed a collaborative research and exploratory process aimed at increasing community involvement in our study. A major contribution of this article is that it summarizes the comprehensive process of conceptualizing and developing a performance-based HL measurement tool and highlights the key elements of the development process. We hope our insights will help researchers and clinicians to not only develop measures to address HL gaps in a comprehensive way, but also identify areas in health organizations that must be improved to create HL-competent healthcare systems.
Detailed results on psychometric and clinical validation studies will be addressed in future publications. We also plan to assess the impact of interventions, such as asthma education or pulmonary rehabilitation, on HL scores using VALHT. Enhanced functional measurement of HL will facilitate the development and testing of HL-based interventions to ultimately improve patient outcomes and reduce the burden on the healthcare system. Author Contributions: Each author made substantial contributions to acquiring the data, and helped to write, edit, and prepare the manuscript. I.P. and J.M.F. conceived and designed the study. I.P. drafted the manuscript, and the rest of the co-authors (J.M.F., J.K., N.T., J.S., R.S. and R.H.) critically revised the manuscript and gave final approval for submission. All authors contributed to the manuscript development and revisions, read and approved the submitted version, and agreed to be accountable for their own contributions. All authors agreed to be personally accountable for their own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature. All authors have read and agreed to the published version of the manuscript. Informed Consent Statement: Informed consent was obtained from all subjects involved in the study.
Data Availability Statement: Data available on request due to privacy restrictions. The data presented in this study are available on request from the corresponding author. The data include interviews with patients and clinicians and are not publicly available due to a lack of explicit agreement from participants to share their data in this way.