Next Article in Journal
Two Faces of Vitamin C in Hemodialysis Patients: Relation to Oxidative Stress and Inflammation
Previous Article in Journal
Dietary Phosphorus as a Marker of Mineral Metabolism and Progression of Diabetic Kidney Disease
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Nutrients
Volume 13
Issue 3
10.3390/nu13030790
nutrients-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Functional Health Literacy: Psychometric Properties of the Newest Vital Sign for Portuguese Adolescents (NVS-PTeen)

by
Osvaldo Santos
1,2,3,*,
Miodraga Stefanovska-Petkovska
1,
Ana Virgolino
1,3,
Ana Cristina Miranda
1,
Joana Costa
1,
Elisabete Fernandes
3,
Susana Cardoso
4,5 and
António Vaz Carneiro
1,3,6,7
1
Instituto de Saúde Ambiental, Faculdade de Medicina, Universidade de Lisboa, 1649-028 Lisboa, Portugal
2
Unbreakable Idea Research, Lda., 2550-426 Painho, Portugal
3
Faculdade de Medicina, Universidade de Lisboa, 1649-028 Lisboa, Portugal
4
MARE, Escola Superior de Turismo e Tecnologia do Mar, Instituto Politécnico de Leiria, 2520-614 Peniche, Portugal
5
CiTechCare, Instituto Politécnico de Leiria, 2410-541 Leiria, Portugal
6
Institute for Evidence Based Healthcare, 1649-028 Lisboa, Portugal
7
Cochrane Portugal, 1649-028 Lisboa, Portugal
*
Author to whom correspondence should be addressed.
Nutrients 2021, 13(3), 790; https://doi.org/10.3390/nu13030790
Received: 13 January 2021 / Revised: 21 February 2021 / Accepted: 23 February 2021 / Published: 27 February 2021
(This article belongs to the Section Nutrition and Public Health)
Browse Figures
Versions Notes

Abstract

:
Self-management of health requires skills to obtain, process, understand, and use health-related information. Assessment of adolescents’ functional health literacy requires valid, reliable, and low-burden tools. The main objective of this study was to adapt and study the psychometric properties of the Newest Vital Sign for the Portuguese adolescents’ population (NVS-PTeen). Classic psychometric indicators of reliability and validity were combined with item response theory (IRT) analyses in a cross-sectional survey, complemented with a 3-month test-retest assessment. The NVS-PTeen was self-administered to students enrolled in grades 8 to 12 (12 to 17 years old) in a school setting. Overall, 386 students (191 girls) from 16 classes of the same school participated in the study (mean age = 14.5; SD = 1.5). Internal reliability of the NVS-PTeen was α = 0.60. The NVS-PTeen total score was positively and significantly correlated with Portuguese (r = 0.28) and mathematics scores (r = 0.31), school years (r = 0.31), and age (r = 0.19). Similar to the original scale (for the U.S.), the NVS-PTeen is composed of two dimensions, reading-related literacy and numeracy. Temporal reliability is adequate, though with a learning effect. IRT analyses revealed differences in difficulty and discriminative capacity among items, all with adequate outfit and infit values. Results showed that the NVS-PTeen is valid and reliable, sensible to inter-individual educational differences, and adequate for regular screening of functional health literacy in adolescents.

1. Introduction

Over the past decades, health literacy has become a flourishing field of research. This concept has evolved from a rather simple one, mainly focused on specific health- or disease-related contents, toward a complex construct [1]. Despite the variations concerning its definition, health literacy is generally understood as a set of individual capacities to obtain, process, and understand basic health information and services, which supports appropriate health- or disease-related decision making [2]. Health literacy relates to general literacy, as it involves the combination of verbal (reading and writing) and numerical skills with specific health literacy skills to access, understand, appraise, and apply the information gathered in order to make decisions and engage in health behaviors [1,3]. The extent to which an individual is autonomous and empowered to self-manage health, following the chain of access–understand–appraise–apply health-related information, is broadly indicated by his/her levels of functional, interactive, and critical literacy [4]. Individuals possessing basic reading, writing, and numeracy skills that are necessary for them to function effectively in everyday situations, including managing their health or disease more easily, show adequate levels of functional health literacy [1,4]. Increased autonomy and empowerment in self-management of health come from advanced cognitive, social, and literacy skills that allow individuals to gather, interpret, and apply health-related information to changing conditions (interactive literacy) [4] and, on top of this, critically appraise health-related information (critical literacy), promoting the individual’s control over his/her own health [4,5]. The ultimate consequences of adequate health literacy levels include better life course health outcomes and reduced burden of healthcare service utilization (e.g., costs, frequency, and length of hospitalizations) [1,6]. The potential positive effects of health literacy promotion have been increasingly debated and are increasingly influential as a policy priority by decision makers across countries (e.g., [7]).
A critical step, prior to the design and implementation of sustainable and (cost- and time-) effective health literacy policies, is to measure citizens’ health literacy. The European Health Literacy Survey (HLS-EU), an eight-country effort to measure health literacy, reported inadequate health literacy for 12.4% of the adult population from the eight European countries that took part in the survey (Austria, Bulgaria, Germany, Greece, Ireland, the Netherlands, Poland, and Spain) [8]. Furthermore, 35% of the participants in the HLS-EU had limited health literacy. In the particular case of Portugal, two nationwide studies using the same HLS-EU instrument estimated that limited functional health literacy ranges between 49% [9] and 61% [10] for the Portuguese adult population.
Efforts to estimate health literacy have been mainly focused on clinical and community samples of adults (≥18 years old), whereas health literacy among children and adolescents has been sparsely measured [11,12,13,14,15]. Despite this, increased attention has been given to the association between parents’ health literacy levels and health outcomes of their dependents (e.g., [16,17]). At this level, the existing evidence suggests that parents’ limited health literacy is associated with a higher number of non-urgent visits to the pediatric emergency department [17] and hinders both their engagement in shared-decision making and their children’s therapeutic adherence [16]. As such, empowering children and adolescents to actively participate in health decision making in a meaningful manner is fundamental to achieving better health outcomes from a life course perspective.
Health outcomes and behaviors in adolescence and adulthood are strongly associated. Thus, the lack of research in health literacy during adolescence is somehow striking. Adolescence is a life transition period characterized by numerous developmental changes (physical, cognitive, and emotional) (e.g., [18,19,20,21]), which are inherently linked to (health) behavioral change [20,22] and habit formation, as well as to an increasing level of individuals’ autonomy in many spheres of their lives, including health decision making. At this stage, several levels of influence—individual traits, peer and family influences, school and neighborhood environments—collectively determine individual development and the adoption of health behaviors [12], namely in relation to physical activity, healthy eating, substance abuse, and sexual risk-taking behaviors, these being major determinants of later life health outcomes and inequities [23,24,25]. Under this scenario, effective health literacy programs are expected to be implemented within the environments in which adolescents are embedded [12]. Schools are particularly relevant toward this end [26,27,28], as also emphasized by Manganello [12], because they provide the resources and supportive environment that facilitate the development of general literacy skills, i.e., reading, writing, and numeracy, which are also required for health literacy.
Given the undisputable role of health literacy during adolescence for health gains in a life span perspective, why are data scarce for this life stage? The lack of validated instruments for measuring health literacy, and, most specifically, functional health literacy, during adolescence is perhaps the main contributor to this knowledge gap (e.g., [14]). Much of the measurement instruments currently under use are of weak or moderate validity [29] and heterogeneous concerning their scope, which also shows the lack of consensus regarding the definition and conceptualization of health literacy, as previously mentioned [11,29,30]. Nevertheless, adaptations of health literacy measurement instruments developed for adults, such as the Test of Functional Health Literacy in Adults (TOFHLA) [31], the Rapid Estimate of Adult Literacy in Medicine (REALM) [32], and the Newest Vital Sign (NVS) [33] (just to name some of the most widely used tools in this area), have been used to assess functional health literacy in children and adolescents (e.g., [13,15,29]), although their psychometric properties have not always been studied. These three instruments measure different domains of functional health literacy: TOFHLA is a 50-item reading comprehension and numerical ability test that takes approximately 22 min to complete [31]; REALM is a reading recognition test that takes approximately 2–3 min to complete [32]; and, finally, the NVS is a 6-item functional health literacy instrument that assesses reading comprehension and numeracy [33].
The main objective of this study was to adapt the NVS instrument to the adolescent Portuguese population (NVS-PTeen) and examine its psychometric properties. The NVS instrument was selected because it is a brief, easy-to-administer scale [33,34,35] and has revealed adequate psychometric properties for assessing functional health literacy, including good internal consistency, for adult populations from different cultures, as well as for some adolescent populations (see Table S1, in Supplementary Material, for an extensive list of studies presenting the psychometric properties of the NVS). The original instrument consists of a food nutrition label with six associated questions scored on a dichotomous scale [33]. Although it has been previously adapted for the Portuguese adult population [36,37], its psychometric properties have not been evaluated yet for the adolescent Portuguese population (12 to 17 years old). As such, this is a pioneering, relevant, and timely contribution to the assessment of functional health literacy among Portuguese adolescents.

2. Materials and Methods

2.1. Study Design and Setting

A cross-sectional observational study was conducted to assess the internal reliability, as well as the construct and convergent validity, of the NVS-PTeen. Regarding temporal reliability, data collection took place at two different moments, with a 3-month interval. This rather long time interval was set up for minimizing the potential learning effect after applying the instrument to the baseline.
Self-administered pencil-and-paper questionnaires of the NVS-PTeen were applied in a school setting by a trained researcher. At the beginning of selected lectures, students were asked to fill in the questionnaires, i.e., collective application, with no time restriction. Questionnaires were collected by the same researcher immediately after completion. Due to the collective application of the instrument, the time required for the adolescents to complete the NVS-PTeen was not assessed.

2.2. Sampling and Participants

To evaluate the psychometric properties of the NVS-PTeen, adolescents aged 12 to 17 years and enrolled in grades 8 to 12 in a Portuguese public school were invited to participate in this study (census approach), in a total of 16 classes. Exclusion criteria were (a) being Portuguese native speakers and/or (b) having special educational needs (e.g., due to cognitive impairment).
There are no clear guidelines about the minimum sample size that is required for assessing psychometric properties of psychosocial scales. Anyway, taking into account the type of statistical analyses considered for this psychometric approach (Spearman’s rank and simple logistic regression, intra-class correlations, exploratory factorial analysis, and item response theory (IRT); further details in the Statistical Analysis section below), a minimum sample size of 250 adolescents was defined. This sample size also corresponds to the median sample size found in a systematic literature review for determining the sample size for validating patient-reported outcome measures [38].
A sub-sample of students participated in the test-retest component of the study. The minimum sample size was settled at 100 students (attending to the nature of the statistical test, without stratification for test-retest analysis). The selection of these participants was done through a random sampling of the classrooms, one class per grade. All students from each randomly selected class were invited to participate in the retest assessment.

2.3. Instruments of Data Collection

Table S1 in Supplementary Material provides the psychometric properties of various adapted and validated versions of the NVS questionnaire for use in different languages and countries. Contents of the NVS-PTeen and its scoring system are available in Table S2 (Supplementary Material).
The original version of the NVS instrument has been already cross-culturally adapted and validated for the Portuguese adult population [37,39,40], revealing adequate psychometric properties. The authors of the two already mentioned Portuguese versions of the NVS [37,40] followed the standard method for cross-cultural adaptation of instruments by the Scientific Advisory Committee of the Medical Outcomes Trust for linguistic adaptation [41]. In both cases, the authors assumed that cultural issues regarding food labels (typically available on the back of food packages) are not substantially different between the U.S. and Portugal, therefore ensuring conceptually equivalent versions. In the case of the NVS-PTeen, the Portuguese linguistic and cultural adaptation, directly from the original English version, was initially performed for the adult version (same instrument used by Paiva et al. [37]), as follows: (a) two bilingual native Portuguese speakers independently translated the original version of the NVS from English to Portuguese, (b) these two translated versions were merged into a single consensus Portuguese version, (c) the Portuguese consensus version was then back-translated by two bilingual native English speakers, and (d) the two independent English versions were merged into a single consensus version, which was compared to the original NVS instrument. The research team agreed that the two versions did not differ in any relevant way. The NVS-PTeen mainly differs from the adult version by adopting the second-person singular, thus promoting a rather colloquial style. A pre-test was conducted involving three women and three men of different ages (age range: 20–65 years) and with different levels of education (high school and university education).
In addition to the NVS-PTeen, adolescents were asked to provide sociodemographic information, i.e., sex, age, and school year, as well as their final classification results for mathematics and Portuguese disciplines from the previous year. Each questionnaire form included a pre-stamped random individual code, which guaranteed respondents’ anonymity, while ensuring the longitudinal component of the project (only for the test-retest sub-sample of students).

2.4. Statistical Analysis

Descriptive statistics (i.e., mean, median, standard deviation, and frequency) were calculated for sociodemographic indicators (sex, age, and education level). Data normality was assessed using the Kolmogorov–Smirnov non-parametric test (Lilliefors corrected K-S test), complemented with analyses of kurtosis and skewness of the distributions.
Since the dataset of the total NVS-PTeen score was not normally distributed, comparisons between sexes and among educational levels were performed using Mann–Whitney U and Kruskal–Wallis tests, respectively. The total score for the NVS-PTeen was recoded according to the cutoff points proposed by the authors of the original American English version of the instrument [33]: likelihood of inadequate health literacy (0 to 1 correct answers), limited health literacy (2 to 3 correct answers), and adequate health literacy (4 to 6 correct answers). The percentage of correct/non-correct answers to the NVS-PTeen items and health literacy levels were compared between sexes and among educational levels using chi-square tests.
Psychometric properties of the NVS-PTeen were evaluated using two different approaches, reliability and validity properties (classic psychometry) and item response theory (IRT). NVS-PTeen reliability was assessed by calculating its internal consistency reliability and its reproducibility (temporal/test-retest reliability). Due to the dichotomous nature of NVS-PTeen items, the internal consistency reliability of this instrument was measured through the Kuder–Richardson 20 (KR20) coefficient [42]. Spearman’s rank correlation and pairwise odds ratios were used for assessing inter-item and item-total associations. A reliability coefficient of 0.70 and a corrected item-total subscale correlation of 0.30 or higher were considered good cutoffs for internal reliability [43]. Test-retest reliability was conducted to assess reproducibility of the NVS-PTeen instrument; as such, the two-way mixed, single-measure intraclass correlation coefficient (ICC) was used.
Concerning validity of the NVS-PTeen, convergent validity and construct validity were assessed. Convergent validity was studied using bivariate correlation analysis (Spearman’s r correlation coefficient) between its global score and five theoretically related variables: age, school years, previous-year final classifications for Portuguese and mathematics, and the average final classification of Portuguese and mathematics. Construct validity was studied through exploratory factorial analysis (EFA) with direct oblimin rotation, following the same procedure as used in previous studies of the NVS (e.g., [24,44]). The Kaiser–Meyer–Olkin (KMO) test and Bartlett’s test of sphericity were performed to determine assumptions of EFA and sampling adequacy for principal component analysis. The correlation matrix of all six items and mean inter-item correlation were verified to evaluate the strength of association between the items. An eigenvalue higher than 1 and a screen plot were used to determine the number of factors. After oblimin rotation, items with a factor loading of 0.40 or greater were considered adequate for measuring a factor.
Finally, IRT was used for estimating item difficulty, discrimination, and fit. Item difficulty refers to the level of health literacy required to meet at least 50% chance of correctly answering an item; item discrimination refers to the capacity of an item to differentiate students with high health literacy from students with low health literacy (items with discrimination values below 1 indicate less discriminating efficacy); and item fit refers to the degree to which observed responses to an item correspond to expected responses, given the difficulty of the item and the respondent’s level of health literacy. Values above 0.8 indicate an adequate item fit [45].
Statistical analyses were performed with IBM Statistical Package for the Social Sciences (SPSS), version 24.0, and with jMetrik, version 4.0.6, for IRT analysis. Statistical significance was set to α = 0.05.

2.5. Ethical Considerations

Authorization for adapting and validating the NVS instrument for the Portuguese population was granted from Pfizer, Inc., the company that holds its copyright, and approved by the Ethical Committees of the Universidade do Porto and Centro Académico de Medicina de Lisboa.
The assessment of the psychometric properties of the Portuguese version of the NVS-PTeen followed the guidelines laid down in the Declaration of Helsinki, amended in Fortaleza [46]. Data collection was approved by the direction board of the school where data collection took place. Prior to study enrollment, the adolescents were informed of the study objectives, of its disassociation from the curricular activities, and that their participation was voluntary, with no impact on their academic activities and/or results. Furthermore, they were explained that the filling in of the questionnaires was part of a research study, not a school test/exam and also that teachers would not have access to the results of the NVS-PTeen. Only the adolescents whose parents signed a consent form (with detailed information about the goals and tasks of the project) and who confirmed their willingness to participate were involved in the study. Anonymity was not possible for the students enrolled in the longitudinal component of the study. However, it was explained that only members of the research team would have access to the data collected, which would be kept confidential, and that no personal data allowing their identification (i.e., name) would be recorded in the main database (a random code was attributed to each student for test-retest matching).

3. Results

3.1. Sample Characterization

Overall, 386 students (48.8% female) from 16 different classes participated in the study (Table 1). Regarding upper secondary education (grades 10 to 12), 28 students were not enrolled in mathematics courses and, therefore, only classification marks from Portuguese classes were used. Participants were aged, on average, 14.4 (SD = 1.4) years; no statistically significant differences between boys and girls were found (p = 0.79). In addition, no significant association was found between being a male or a female and the grade in which the students were enrolled; about 60% of the sample was enrolled in lower secondary education (grades 8 to 9; Table 1). About 10% of the students had negative scores in Portuguese in the previous academic year, whereas 20% of them scored negative in mathematics (normal distributions for students in grades 8 to 9; non-normal, left-skewed distribution for students in grades 10 to 12; data not provided).

3.2. Functional Health Literacy among Adolescents

Girls failed slightly more NVS-PTeen questions than boys, except for items 5 and 6 (Table 2). However, statistically significant differences between boys and girls were only found for item 1 (p = 0.014). The prevalence of adequate health literacy was high for both sexes (80.6% for girls and 86.7% for boys), with no statistically significant differences (p = 0.26). In addition, 44.0% of the total sample answered the six questions of the NVS-PTeen correctly, thus obtaining the maximum score (score = 6; 45.0% for girls, 43.1% for boys).
The educational level (i.e., number of school years) was significantly associated with all NVS-PTeen items (p < 0.05), except for items 1 and 5 (Table 2). The prevalence of compromised health literacy was higher among students in grade 8 (4.8% of the students had inadequate health literacy and 23.8% had limited health literacy) and grade 9 (3.8% of the students had inadequate health literacy and 16.3% had limited health literacy) than students in grades 10 to 12 (p < 0.001). Indeed, adequate health literacy was detected among more than 90% of the students in grades 10 to 12.

3.3. Internal Consistency and Test-Retest Reliability

The overall internal consistency reliability of the self-administered NVS-PTeen was KR20 = 0.61 (95% CI = 0.54–0.66).
The inter-item Spearman’s rank correlation coefficients and odds ratios are provided in Table 3. All inter-item correlation and odds ratio were statistically significant, except for the pairs of items 2 and 3, and 2 and 6. The highest inter-item Spearman’s rank correlation coefficients and odds ratios were obtained for the pair of items 5 and 6 (r = 0.59). Regarding the item-total correlation, which is an item discrimination indicator, Spearman’s rank correlation coefficients ranged from r = 0.49 (item 3) to r = 0.67 (item 4) (Table 3).
The test-retest reliability was acceptable (ICC = 0.605; 95% CI = 0.54–0.66). The majority of deviations were toward improved health literacy from the first to the second observation. Item 2 had low consistency (Table 4).

3.4. Convergent Validity of the NVS-PTeen

As presented in Table 5, a weak correlation was found between age and the NVS-PTeen total score. The educational level, given as the number of school years the student was enrolled for at the time of data collection, and final classifications of Portuguese and mathematics from the previous academic year were moderately correlated with the NVS-PTeen final score.

3.5. Construct Validity-Dimensionality

The Kaiser–Meyer–Olkin (KMO) test suggested an adequate fit (KMO = 0.64) of the dataset for factorial analysis. A similar result was obtained after Barlett’s test of sphericity, which indicated that the correlation matrix was significantly different from zero (p < 0.001) and, thus, suitable for factorial analysis. Two factors were obtained by means of factorial analysis with direct oblimin rotation, eigenvalues above 1 and factor loading above 0.4. The eigenvalues for these two factors were 2.09 and 1.13, with 34.82% and 18.89% of the explained variance, respectively. The two factors, comprising the six items, explained 53.71% of the total variance (Figure 1). Factor 1 was associated with reading-related literacy, while factor 2 was associated with numeric skills. Item 3 revealed to be weakly associated with both factors.

3.6. Item Response Theory (IRT): Item Difficulty, Discrimination, and Fit

After adjusting for spuriousness, i.e., after removing the variance attributable to the NVS-PTeen total score due to the item-specific variance, IRT curves (Figure 2) revealed that items 4, 5, and 6 were the most discriminative ones (a = 0.42), whereas items 2 and 3 were the least discriminative ones (a = 0.26). Concerning item difficulty, items 3 and 6 scored as the easiest ones (b = −0.37 and b = 0.24, respectively), whereas item 2 was the most difficult one (b = 0.82). Finally, the fit between observed and expected responses was adequate for all items (UMS, unweighted mean squares, and WMS, weighted mean squares, values ranged between 0.8 and 1.2), except for item 5.

4. Discussion

In this study, the psychometric properties of a self-administered version of the Newest Vital Sign for adolescents, the NVS-PTeen were assessed by combining classic and modern (IRT) psychometric tools. Although there are a few published studies that assess health literacy in Portuguese samples [37,39,40], none of them specifically targets adolescents. The main findings of this observational cross-sectional study (with a longitudinal component) are as follows: (a) functional health literacy levels for 83.4% of the participants were adequate; (b) upper secondary students had higher functional health literacy levels than lower secondary education students; (c) overall, the NVS-PTeen had acceptable psychometric properties measured using the classical methods (i.e., reliability and validity); and (d) the analysis of IRT curves allowed the identification of the most discriminative and easiest items in the instrument and revealed a good fit between observed and expected answers.

4.1. Prevalence of Adequate Health Literacy in the Adolescent Sample under Study

Health literacy assumes an indisputable role in supporting adequate and effective health decisions. Indeed, available evidence supports the association between inadequate health literacy, decreased health outcomes, increased healthcare use, and increased health expenditure (e.g., [47,48,49]). Since health and health behaviors during childhood and adolescence are strongly associated with health outcomes during adulthood [23,24,25], adequate health literacy assumes particular relevance during early life as a health promotion strategy across the life span. However, health literacy data during childhood and adolescence is scarce [12,15], which might be due to a combination of factors, namely no consensus regarding the definition of the health literacy construct [50] and a lack of adequate tools for measuring this indicator at this life stage [13,29]. In addition, there has been some debate on whether to measure health literacy among children. Indeed, as defended by Weiss, “Why would we expect children, particularly elementary school children (some of whom, such as the 7-year-olds, are still learning to read) to be able to interpret the complexities of a nutrition label, something that even many adults cannot do” [51] (p. e19). This argument is less tenable for adolescents because analytical thought, namely regarding text interpretation and numerical/arithmetic capabilities, is mostly developed at these ages [52]. Therefore, it is highly relevant to evaluate how school-related achievements are effectively contributing to the improvement of (applied) functional health literacy. In the particular case of Portugal, the few studies available have mainly focused on measuring youth mental (content-specific) health literacy (e.g., [53]), whereas data on functional health literacy for adolescents aged below 18 years have been generally disregarded.
The prevalence of adequate functional health literacy, measured with the NVS, among Portuguese adolescents participating in this study was higher than among US adolescents aged 12–19 years (51%) [54]. The lack of prevalence measures of functional health literacy for adolescents, with a particular focus on European adolescents, measured with the NVS instrument or even with other measuring tools, precludes the comparison of our data with results from elsewhere. As such, the results from this study are coarsely placed in a wider context. The prevalence of adequate functional health literacy reported here contrasts with the one by Paiva et al. [37] for a representative sample of the adult Portuguese population (aged 16 to 79 years), measured using the same NVS instrument. Paiva et al. [37] reported considerably lower levels of adequate health literacy (27.1%) than the ones reported here (83.4%). Results from the European Health Literacy Survey (HLS-EU), an eight-country survey of health literacy (European citizens aged ≥15 years), revealed highly variable levels of adequate health literacy, including the NVS instrument as a component of the assessment [8]. These ranged from 36.9%, in Spain, to 76.3%, in the Netherlands; in Portugal, the same HLS-EU methodological approach revealed high levels of inadequate health literacy, as previously mentioned in the Introduction section [9,10]. Multiple factors might explain the differences in the average prevalence values reported across studies, including sample heterogeneity in terms of sociodemographic characteristics. Indeed, individuals with lower socioeconomic status, lower educational level, and higher age are more vulnerable to low functional health literacy than their counterparts [1,8].
The educational level plays a key role in functional health literacy [1,8]. Paiva et al. [37] detected a significant association between educational level and functional health literacy in the adult Portuguese population. In their study, individuals who completed a university degree provided more correct answers than those with a lower education level. For example, 55.5% of those with a university degree had adequate literacy versus 25.9% of those who completed lower secondary education [37]. In the particular case of our study, the overall prevalence of adequate health literacy among adolescents was much higher. The most plausible explanation for such higher prevalence of functional health literacy when compared to results from other studies has to do with sampling—only one school was involved in this study. Despite the heterogeneity in terms of social and economic family backgrounds of the students enrolled in this school, teaching methods are somehow homogenous across classes (e.g., the same teacher teaches more than one class per grade and even classes from different grades) and the Portuguese and mathematics classification marks from the previous year were generally good. It is worth mentioning that in the year before data was collected, this school ranked 154 position (out of 593) in the national secondary education schools ranking [55]. Thus, the prevalence values of functional health literacy provided here cannot be generalized for the adolescent population in Portugal.

4.2. Psychometric Properties of the NVS-PTeen

The main endeavor of this study was to investigate the psychometric properties of the NVS instrument for the Portuguese population aged 12 to 17 years. The combination of classical and modern methods for assessing the psychometric properties of the NVS-PTeen instrument used here allowed a comprehensive understanding of the global instrument, but also of each item separately. Several previous studies that adapted, validated, and investigated the psychometric properties of the NVS instrument used a face-to-face (hetero-)administered questionnaire (see Table S1 in Supplementary Material for an extensive list of these studies). In this study and a few others [40,56,57,58,59], the questionnaire was self-administrated with a potential reduced burden of administration compared to its hetero-administration [60]. Indeed, available evidence indicates that self-administered versions of the NVS can take up to 6 min to complete [40,59], whereas hetero-administered versions of the questionnaire can take up to 8 min to complete [33]. Unfortunately, due to the collective self-administration of the NVS instrument in the school-class setting employed in this study, no data on the completion time were gathered.
The internal consistency of the NVS-PTeen was acceptable as estimated with KR20 (KR20 = 0.61), a special case of Cronbach’s α for dichotomous variables. This finding contrasts with good internal consistency reported for the original NVS instrument (Cronbach’s α = 0.76) by Weiss et al. [33], as well as for the adult Portuguese version of this instrument (Cronbach’s α = 0.85) by Paiva et al. [37] and others who assessed functional health literacy among adolescents [54,56,61,62]. Nevertheless, the coefficient α (i.e., Cronbach’s α and KR20) is sensitive to scale length, and it tends to be lower for shorter instruments [63]. As such, internal consistency of the NVS-PTeen is satisfactory, given that this is a short-length scale with six dichotomous (correct/incorrect) scoring-format items. In addition, as it is widely acknowledged, coefficient α should not be used as the only measure of internal consistency of an instrument; inter-item associations are also useful for a comprehensive assessment of its internal consistency [64]. In this study, inter-item correlation and odds ratio values were satisfactory (broadly within the interval 0.20–0.40, with a few exceptions) and similar to the ones previously obtained by Martins and Andrade [40] for a Portuguese clinical (adult) sample. Reproducibility of the instrument, as assessed by test-retest, was also acceptable (ICC = 0.605); similar findings were previously reported by Cruvinel et al. [65] (ICC = 0.57), Zotti et al. [66] (Spearman’s r = 0.65), and Kogure et al. [57] (Pearson’s r = 0.82).
Moderate convergent validity between the NVS-PTeen total score and education-related indicators (i.e., educational level, previous-year classifications of Portuguese and mathematics) was detected. This is particularly informative because the NVS-PTeen assesses functional health literacy, mainly requiring reading and numeracy skills [33]. The low correlation coefficient obtained between the NVS-PTeen total score and age adds support to this claim and suggests that the scoring of this instrument is not affected by potential developmental bias. Concerning construct validity, two factors, each reflecting a particular aspect of functional literacy—reading and numeracy skills—were identified and collectively explained 53.71% of the variance in the six-item questionnaire, a slightly lower value than the one found by Martins and Andrade [40] for their Portuguese version of the NVS instrument (60.97%). Finally, the overall analysis of the psychometric properties of the NVS-PTeen instrument using the classical test theory generally agreed with the findings from the analysis of the IRT curves.
A considerable ceiling effect was detected in this study: 44.0% of the respondents scored the maximum value for the NVS-PTeen. The same effect, although less pronounced, was also reported for versions of the NVS instrument administered to community dwellers in the UK [34] and in the Netherlands [67], as well as for English children [56] and US adolescents [54]. Interestingly, the opposite effect (i.e., floor effect) was reported by Fransen et al. [68] and Kogure et al. [57] for clinical samples of adults and also for Portuguese older community dwellers [39]. Scoring near the possible upper or lower limit of the NVS instrument seems to be a major limitation precluding the discrimination among individuals on the top or bottom ends of the scale. In the particular case of this study, the ceiling effect may be due to sampling, which involved students from only one school. This methodological approach guaranteed control over data collection, including data quality, but did not mirror the Portuguese context at all; this school was well ranked in terms of Portuguese and mathematics final scores, as previously addressed [55], and this may have had some effect on the results. Nevertheless, the main purpose of this study was to adapt and validate the NVS instrument for the adolescent Portuguese population and not to investigate Portuguese adolescents’ functional health literacy. As such, it can be concluded that the NVS-PTeen is an adequate, low-burden screening tool for functional health literacy in a very specific, relevant, and less studied age group.

4.3. Strengths and Limitations

For a comprehensive analysis of the results of this study, its strengths and limitations should be acknowledged. Concerning the strengths, this is the first adaptation and psychometric study of the NVS for the adolescent Portuguese population, in a self-administered format. Previous adaptation and validation studies of this instrument for the Portuguese population concerned only adult community dwellers [37,39,40]. Second, contrary to previous studies (e.g., [39,40,69]), a gender-balanced sample was enrolled; the sample size was also highly satisfactory (N = 386) [70]. A major limitation of this study was sampling. As mentioned above, all sampled students came from only one school, which precludes the generalization of the results on functional health literacy to the adolescent Portuguese population. Moreover, this non-probabilistic approach much probably contributed to the marked ceiling effect observed, as discussed above. Another limitation was with regard to the fact that the NVS-PTeen is, in essence, identical to the adult Portuguese version (only differing from the adult version by adopting the second-person singular, a more colloquial and adequate tone when addressing adolescents). This might be a limitation because no additional adolescent-cultural-adapted efforts have been made, specifically for the Portuguese population. Additional research, using cognitive interview processes, would be adequate to better understand if this instrument would benefit from changes, both in the food label that is presented to respondents and/or regarding the writing of the questions. Nevertheless, the NVS-PTeen revealed acceptable psychometric properties, which indicates that this instrument can confidently be administered to a larger and more heterogeneous sample of Portuguese adolescents.

5. Conclusions

Serious health issues in adulthood result from multiple risk behaviors established during adolescence, including alcohol and other substances abuse, sexual risk-taking behaviors, tobacco use, unhealthy eating habits, and little or no physical activity, among others [71]. Under this context, adolescents attaining higher levels of functional health literacy will potentially have improved health outcomes during the transition to adulthood and as adults [12,14]. Nevertheless, literacy among adolescents has been reported to differ between socioeconomic status and ethnicity (none of these predictors were formally assessed here) [12]. As such, actions addressing health literacy asymmetries between specific groups of adolescents assume a relevant role as cost-effective health promotion strategies in the life span and also as avenues toward reduction in health inequities.
Schools have been increasingly recognized as key settings for achieving health literacy (e.g., [26,27,28]). This is because the school environment potentially provides the resources and supportive environment that facilitate behavioral change and skill improvement, thus contributing to the prevention of multiple risk behaviors during adolescence and adulthood (e.g., [72]). The significant correlations reported here between NVS-PTeen scores and mathematics and Portuguese final classifications put into evidence the role of schools in active health promotion rather than simply providing the students with specific disease- or health-related information. Learning skills and analytical and objective thought are main developmental tasks that should be promoted at school as prerequisites for achieving functional health literacy. As argued by Winkelman et al. [73], there are four steps needed for improving health literacy at schools: (a) curricula covering health education topics, (b) funding, (c) partnerships between the healthcare and education sectors, and (d) the incorporation of health-literacy-screening systems, namely through the use of easy-to-administer, valid, and reliable tools. Results herein provide some support for the use of the NVS-PTeen to assess functional health literacy among Portuguese adolescents. The usage of adequate instruments to assess functional health literacy has the potential to promote valuable insight for both public health and education institutions concerning the relationship between health literacy, health behavior, and health outcomes in adolescents (and adulthood). We are confident that the NVS-PTeen is a useful health education monitoring tool if used regularly in the school context (e.g., at the end of each school cycle) in order to evaluate how official educational curricula may be translated into functional health literacy. Because health literacy is a pivotal variable for public health promotion through health education initiatives [4], this instrument may, therefore, constitute a relevant tool for assessing public health promotion among adolescents.

Supplementary Materials

The following are available online at https://www.mdpi.com/2072-6643/13/3/790/s1, Table S1: Selected published psychometric studies of the Newest Vital Sign. Table S2: Newest Vital Sign | Portuguese adolescent population version (NVS-PTeen).

Author Contributions

Conceptualization, O.S.; methodology, O.S.; coordination of data collection, S.C.; statistical analysis, O.S. and E.F.; writing of original draft, O.S., with contributions from M.S.-P., J.C., A.V., and A.C.M.; literature review and discussion of the results, O.S., with valuable contributions from M.S.-P., A.V., A.C.M., J.C., and A.V.C. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Fundação para a Ciência e a Tecnologia (FCT), under the project Health Information of Portuguese Population: Health Knowledge & Perceived Quality and Accessibility of Health Information Sources (Reference: HMSP-IISE/SAU-ICT/0004/2009). The writing of the paper also benefited from funding from the FCT under the grants UIDB/04295/2020 and UIDP/04295/2020. This funding entity had no role in the design or writing of this article.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Ethical Committee of the Universidade do Porto, the Ethical Committee of the Centro Académico de Medicina de Lisboa, and the direction board of the school where data collection took place.

Informed Consent Statement

Informed consent was obtained from parents of the adolescents involved in the study; adolescents gave also their agreement for their voluntary participation in this study.

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Acknowledgments

The authors thank the school board and teachers of the Instituto Inácio de Loyola for their collaboration in data collection. The students enrolled in the study and their parents are thanked for their valuable contributions.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Sørensen, K.; Van den Broucke, S.; Fullam, J.; Doyle, G.; Pelikan, J.; Slonska, Z.; Brand, H. Health literacy and public health: A systematic review and integration of definitions and models. BMC Public Health 2012, 12, 80. [Google Scholar] [CrossRef] [PubMed][Green Version]
  2. Ratzan, S.C.; Parker, R.M. Health literacy - identification and response. J. Health Commun. 2006, 11, 713–715. [Google Scholar] [CrossRef]
  3. Nutbeam, D.; McGill, B.; Premkumar, P. Improving health literacy in community populations: A review of progress. Health Promot. Int. 2018, 33, 901–911. [Google Scholar] [CrossRef] [PubMed]
  4. Nutbeam, D. Health literacy as a public health goal: A challenge for contemporary health education and communication strategies into the 21st century. Health Promot. Int. 2000, 15, 259–267. [Google Scholar] [CrossRef][Green Version]
  5. Chinn, D. Critical health literacy: A review and critical analysis. Soc. Sci. Med. 2011, 73, 60–67. [Google Scholar] [CrossRef]
  6. Eichler, K.; Wieser, S.; Brügger, U. The costs of limited health literacy: A systematic review. Int. J. Public Health 2009, 54, 313–324. [Google Scholar] [CrossRef][Green Version]
  7. Weishaar, H.; Hurrelmann, K.; Okan, O.; Horn, A.; Schaeffer, D. Framing health literacy: A comparative analysis of national action plans. Health Policy 2019, 123, 11–20. [Google Scholar] [CrossRef] [PubMed]
  8. Sørensen, K.; Pelikan, J.M.; Röthlin, F.; Ganahl, K.; Slonska, Z.; Doyle, G.; Fullam, J.; Kondilis, B.; Agrafiotis, D.; Uiters, E.; et al. Health literacy in Europe: Comparative results of the European Health Literacy Survey (HLS-EU). Eur. J. Public Health 2015, 25, 1053–1058. [Google Scholar] [CrossRef][Green Version]
  9. Espanha, R.; Ávila, P. Health Literacy Survey Portugal: A contribution for the knowledge on health and communications. Procedia Comput. Sci. 2016, 100, 1033–1041. [Google Scholar] [CrossRef][Green Version]
  10. Pedro, A.R.; Amaral, O.; Escoval, A. Literacia em saúde, dos dados à ação: Tradução, validação e aplicação do European Health Literacy Survey em Portugal. Rev. Port. Saude Publica 2016, 34, 259–275. [Google Scholar] [CrossRef]
  11. Guo, S.; Armstrong, R.; Waters, E.; Sathish, T.; Alif, S.M.; Browne, G.R.; Yu, X. Quality of health literacy instruments used in children and adolescents: A systematic review. BMJ Open 2018, 8, 1–18. [Google Scholar] [CrossRef] [PubMed]
  12. Manganello, J. Health literacy and adolescents: A framework and agenda for future research. Health Educ. Res. 2008, 23, 840–847. [Google Scholar] [CrossRef]
  13. Ormshaw, M.J.; Paakkari, L.T.; Kannas, L.K. Measuring child and adolescent health literacy: A systematic review of literature. Health Educ. 2013, 113, 433–455. [Google Scholar] [CrossRef]
  14. Perry, E.L. Health literacy in adolescents: An integrative review. J. Spec. Pediatr. Nurs. 2014, 19, 210–218. [Google Scholar] [CrossRef] [PubMed]
  15. Sansom-Daly, U.M.; Lin, M.; Robertson, E.G.; Wakefield, C.E.; McGill, B.C.; Girgis, A.; Cohn, R.J. Health literacy in adolescents and young adults: An updated review. J. Adolesc. Young Adult Oncol. 2016, 5, 106–118. [Google Scholar] [CrossRef] [PubMed]
  16. Chari, R.; Warsh, J.; Ketterer, T.; Hossain, J.; Sharif, I. Association between health literacy and child and adolescent obesity. Patient Educ. Couns. 2014, 94, 61–66. [Google Scholar] [CrossRef]
  17. Morrison, A.K.; Schapira, M.M.; Gorelick, M.H.; Hoffmann, R.G.; Brousseau, D.C. Low caregiver health literacy is associated with higher pediatric emergency department use and nonurgent visits. Acad. Pediatr. 2014, 14, 309–314. [Google Scholar] [CrossRef][Green Version]
  18. Feldman, S.S.; Elliott, G.R. At the Threshold: The Developing Adolescent; Harvard University Press: Cambridge, MA, USA, 1990. [Google Scholar]
  19. Yurgelun-Todd, D. Emotional and cognitive changes during adolescence. Curr. Opin. Neurobiol. 2007, 17, 251–257. [Google Scholar] [CrossRef] [PubMed]
  20. Sisk, C.L.; Zehr, J.L. Pubertal hormones organize the adolescent brain and behavior. Front. Neuroendocrinol. 2005, 26, 163–174. [Google Scholar] [CrossRef] [PubMed]
  21. Adams, G.R.; Berzonsky, M.D. Blackwell Handbook of Adolescence; Blackwell Publishing Ltd.: Malden, MA, USA, 2006. [Google Scholar]
  22. Dahl, R.E.; Forbes, E.E. Pubertal development and behavior: Hormonal activation of social and motivational tendencies. Brain Cogn. 2010, 72, 66–72. [Google Scholar] [CrossRef]
  23. Viner, R.M.; Ozer, E.M.; Denny, S.; Marmot, M.; Resnick, M.; Fatusi, A.; Currie, C. Adolescence and the social determinants of health. Lancet 2012, 379, 1641–1652. [Google Scholar] [CrossRef]
  24. Yang, T.-C.; Chen, I.-C.; Choi, S.-W.; Kurtulus, A. Linking perceived discrimination during adolescence to health during mid-adulthood: Self-esteem and risk-behavior mechanisms. Soc. Sci. Med. 2019, 232, 434–443. [Google Scholar] [CrossRef] [PubMed]
  25. Landstedt, E.; Hammarstrom, A.; Winefield, H. How well do parental and peer relationships in adolescence predict health in adulthood? Scand. J. Public Health 2015, 43, 460–468. [Google Scholar] [CrossRef]
  26. Begoray, D.L.; Wharf-Higgins, J.; MacDonald, M. High school health curriculum and health literacy: Canadian student voices. Glob. Health Promot. 2009, 16, 35–42. [Google Scholar] [CrossRef] [PubMed]
  27. Kilgour, L.; Matthews, N.; Christian, P.; Shire, J. Health literacy in schools: Prioritising health and well-being issues through the curriculum. Sport. Educ. Soc. 2015, 20, 485–500. [Google Scholar] [CrossRef]
  28. Leger, L.S. Schools, health literacy and public health: Possibilities and challenges. Health Promot. Int. 2001, 16, 197–205. [Google Scholar] [CrossRef]
  29. Okan, O.; Lopes, E.; Bollweg, T.M.; Bröder, J.; Messer, M.; Bruland, D.; Bond, E.; Carvalho, G.S.; Sørensen, K.; Saboga-Nunes, L.; et al. Generic health literacy measurement instruments for children and adolescents: A systematic review of the literature. BMC Public Health 2018, 18, 166. [Google Scholar] [CrossRef] [PubMed][Green Version]
  30. Bröder, J.; Okan, O.; Bauer, U.; Bruland, D.; Schlupp, S.; Bollweg, T.M.; Saboga-Nunes, L.; Bond, E.; Sørensen, K.; Bitzer, E.-M.; et al. Health literacy in childhood and youth: A systematic review of definitions and models. BMC Public Health 2017, 17, 361. [Google Scholar] [CrossRef]
  31. Parker, R.M.; Baker, D.W.; Willia, M.V.; Nurss, J.R. The Test of Functional Health Literacy in adults: A new instrument for measuring patients’ literacy skills. J. Gen. Intern. Med. 1995, 10, 537–541. [Google Scholar] [CrossRef]
  32. Murphy, P.W.; Davis, T.C.; Long, S.W.; Jackson, R.H.; Decker, B.C. Rapid Estimate of Adult Literacy in Medicine (REALM): A quick reading test for patients. J. Read. 1993, 37, 124–130. [Google Scholar]
  33. Weiss, B.D.; Mays, M.Z.; Martz, W.; Castro, K.M.; DeWalt, D.A.; Pignone, M.P.; Mockbee, J.; Hale, F.A. Quick assessment of literacy in primary care: The Newest Vital Sign. Ann. Fam. Med. 2005, 3, 514–522. [Google Scholar] [CrossRef] [PubMed][Green Version]
  34. Rowlands, G.; Khazaezadeh, N.; Oteng-Ntim, E.; Seed, P.; Barr, S.; Weiss, B.D. Development and validation of a measure of health literacy in the UK: The Newest Vital Sign. BMC Public Health 2013, 13, 116. [Google Scholar] [CrossRef] [PubMed][Green Version]
  35. Johnson, K.; Weiss, B.D. How long does it take to assess literacy skills in clinical practice? J. Am. Board Fam. Med. 2008, 21, 211–214. [Google Scholar] [CrossRef][Green Version]
  36. Santos, O.; Oliveira, A.; Carvalho, C.; Lunet, N.; Azevedo, A.; Paiva, D.; do Carmo, I. Self-administration of the Newest Vital Sign in Portuguese young adults: Study of its internal reliability. In Proceedings of the Abstracts from the ICBM 2012 Meeting, Budapeste, Hungary, 29 August–1 September 2021; p. S189. [Google Scholar]
  37. Paiva, D.; Silva, S.; Severo, M.; Moura-Ferreira, P.; Lunet, N.; Azevedo, A. Limited health literacy in Portugal assessed with the Newest Vital Sign. Acta Med. Port. 2017, 30, 861–869. [Google Scholar] [CrossRef] [PubMed][Green Version]
  38. Anthoine, E.; Moret, L.; Regnault, A.; Sbille, V.; Hardouin, J.B. Sample size used to validate a scale: A review of publications on newly-developed patient reported outcomes measures. Health Qual. Life Outcomes 2014, 12, 1–10. [Google Scholar] [CrossRef] [PubMed][Green Version]
  39. Salgado, T.M.; Ramos, S.B.; Sobreira, C.; Canas, R.; Cunha, I.; Benrimoj, S.I.; Fernandez-Llimos, F. Newest Vital Sign as a proxy for medication adherence in older adults. J. Am. Pharm. Assoc. 2013, 53, 611–617. [Google Scholar] [CrossRef]
  40. Martins, A.C.; Andrade, I.M. Cross-cultural adaptation and validation of the Portuguese version of the Newest Vital Sign. Rev. Enferm. Ref. 2014, IV, 75–83. [Google Scholar] [CrossRef]
  41. Aaronson, N.; Alonso, J.; Burnam, A.; Lohr, K.; Patrick, D.; Perrin, E.; Stein, R. Assessing health status and quality-of-life instruments: Attributes and review criteria. Qual. Life Res. 2002, 11, 193–205. [Google Scholar] [CrossRef]
  42. Kuder, G.F.; Richardson, M.W. The theory of the estimation of test reliability. Psychometrika 1937, 2, 151–160. [Google Scholar] [CrossRef]
  43. Nunnally, J.C.; Bernstein, I.H. Psychometric Theory; McGraw-Hill, Inc.: New York, NY, USA, 1994. [Google Scholar]
  44. Huang, Y.M.; Shiyanbola, O.O.; Smith, P.D.; Chan, H.Y. Quick screen of patients’ numeracy and document literacy skills: The factor structure of the Newest Vital Sign. Patient Prefer. Adherence 2018, 12, 853–859. [Google Scholar] [CrossRef][Green Version]
  45. Meyer, P. Applied measurement with Jmetrik; Rutledge: New York, NY, USA, 2014; ISBN 9780203115190. [Google Scholar]
  46. World Medical Association (WMA) World Medical Association Declaration of Helsinki: Ethical principles for medical research involving human subjects. JAMA 2013, 310, 2191–2194. [CrossRef] [PubMed][Green Version]
  47. Rasu, R.; Bawa, W.A.; Suminski, R.; Snella, K.; Warady, B. Health literacy impact on national healthcare utilization and expenditure. Int. J. Heal. Policy Manag. 2015, 4, 747–755. [Google Scholar] [CrossRef][Green Version]
  48. MacLeod, S.; Musich, S.; Gulyas, S.; Cheng, Y.; Tkatch, R.; Cempellin, D.; Bhattarai, G.R.; Hawkins, K.; Yeh, C.S. The impact of inadequate health literacy on patient satisfaction, healthcare utilization, and expenditures among older adults. Geriatr. Nurs. 2017, 38, 334–341. [Google Scholar] [CrossRef] [PubMed]
  49. Shieh, C.; Halstead, J.A. Understanding the impact of health literacy on women’s health. J. Obstet. Gynecol. Neonatal Nurs. 2009, 38, 601–612. [Google Scholar] [CrossRef]
  50. Baker, D.W. The meaning and the measure of health literacy. J. Gen. Intern. Med. 2006, 21, 878–883. [Google Scholar] [CrossRef] [PubMed][Green Version]
  51. Weiss, B.D. The use of the Newest Vital Sign in children. HLRP Heal. Lit. Res. Pract. 2019, 3, e19. [Google Scholar] [CrossRef] [PubMed][Green Version]
  52. Keating, D.P. Cognitive and brain development in adolescence. Enfance 2012, 3, 267–279. [Google Scholar] [CrossRef]
  53. Rosa, A.; Loureiro, L.; Sequeira, C. Literacia em saúde mental sobre depressão: Um estudo com adolescentes portugueses. Rev. Port. Enferm. Saúde Ment. 2019, 21, 40–46. [Google Scholar] [CrossRef]
  54. Manganello, J.A.; DeVellis, R.F.; Davis, T.C.; Schottler-Thal, C. Development of the Health Literacy Assessment Scale for Adolescents (HAS-A). J. Commun. Healthc. 2015, 8, 172–184. [Google Scholar] [CrossRef][Green Version]
  55. Público Comunicação Social SA Ranking das Escolas. 2017. Available online: https://www.publico.pt/ranking-das-escolas-2017/lista (accessed on 30 December 2020).
  56. Driessnack, M.; Chung, S.; Perkhounkova, E.; Hein, M. Using the “Newest Vital Sign” to assess health literacy in children. J. Pediatr. Heal. Care 2014, 28, 165–171. [Google Scholar] [CrossRef] [PubMed]
  57. Kogure, T.; Sumitani, M.; Suka, M.; Ishikawa, H.; Odajima, T.; Igarashi, A.; Kusama, M.; Okamoto, M.; Sugimori, H.; Kawahara, K. Validity and reliability of the Japanese version of the Newest Vital Sign: A preliminary study. PLoS One 2014, 9, e94582. [Google Scholar] [CrossRef] [PubMed]
  58. Mansfield, E.D.; Wahba, R.; Gillis, D.E.; Weiss, B.D.; L’Abbé, M. Canadian adaptation of the Newest Vital Sign©, a health literacy assessment tool. Public Health Nutr. 2018, 21, 2038–2045. [Google Scholar] [CrossRef][Green Version]
  59. Tseng, H.-M.; Liao, S.-F.; Wen, Y.-P.; Chuang, Y.-J. Adaptation and validation of a measure of health literacy in Taiwan: The Newest Vital Sign. Biomed. J. 2018, 41, 273–278. [Google Scholar] [CrossRef] [PubMed]
  60. Welch, V.L.; VanGeest, J.B.; Caskey, R. Time, costs, and clinical utilization of screening for health literacy: A case study using the Newest Vital Sign (NVS) instrument. J. Am. Board Fam. Med. 2011, 24, 281–289. [Google Scholar] [CrossRef][Green Version]
  61. Hoffman, S.; Marsiglia, F.F.; Lambert, M.C.; Porta, M. A psychometric assessment of the Newest Vital Sign among youth in Guatemala city. J. Child Adolesc. Behav. 2015, 3, 190. [Google Scholar] [CrossRef][Green Version]
  62. Warsh, J.; Chari, R.; Badaczewski, A.; Hossain, J.; Sharif, I. Can the Newest Vital Sign be used to assess health literacy in children and adolescents? Clin. Pediatr. 2014, 53, 141–144. [Google Scholar] [CrossRef] [PubMed]
  63. Tavakol, M.; Dennick, R. Making sense of Cronbach’s alpha. Int. J. Med. Educ. 2011, 2, 53–55. [Google Scholar] [CrossRef] [PubMed]
  64. Schmitt, N. Uses and abuses of coefficient alpha. Psychol. Assess. 1995, 8, 350–353. [Google Scholar] [CrossRef]
  65. Cruvinel, A.F.P.; Méndez, D.A.C.; Chaves, G.C.; Gutierres, E.; Lotto, M.; Oliveira, T.M.; Cruvinel, T. The Brazilian validation of a health literacy instrument: The Newest Vital Sign. Acta Odontol. Scand. 2018, 1–8. [Google Scholar] [CrossRef]
  66. Zotti, P.; Cocchi, S.; Polesel, J.; Mis, C.C.; Bragatto, D.; Cavuto, S.; Conficconi, A.; Costanzo, C.; De Giorgi, M.; Drace, C.A.; et al. Cross-cultural validation of health literacy measurement tools in Italian oncology patients. BMC Health Serv. Res. 2017, 17, 410. [Google Scholar] [CrossRef][Green Version]
  67. Fransen, M.P.; Leenaars, K.E.F.; Rowlands, G.; Weiss, B.D.; Maat, H.P.; Essink-Bot, M.L. International application of health literacy measures: Adaptation and validation of the Newest Vital Sign in The Netherlands. Patient Educ. Couns. 2014, 97, 403–409. [Google Scholar] [CrossRef] [PubMed]
  68. Fransen, M.P.; Van Schaik, T.M.; Twickler, T.B.; Essink-Bot, M.L. Applicability of internationally available health literacy measures in the Netherlands. J. Health Commun. 2011, 16, 134–149. [Google Scholar] [CrossRef] [PubMed]
  69. Rodrigues, R.; De Andrade, S.M.; González, A.D.; Birolim, M.M.; Mesas, A.E. Cross-cultural adaptation and validation of the Newest Vital Sign (NVS) health literacy instrument in general population and highly educated samples of Brazilian adults. Public Health Nutr. 2017, 20, 1907–1913. [Google Scholar] [CrossRef] [PubMed][Green Version]
  70. Frost, M.H.; Reeve, B.B.; Liepa, A.M.; Stauffer, J.W.; Hays, R.D.; Sloan, J.A. What is sufficient evidence for the reliability and validity of patient-reported outcome measures? Value Heal. 2007, 10, S94–S105. [Google Scholar] [CrossRef][Green Version]
  71. Kipping, R.R.; Campbell, R.M.; MacArthur, G.J.; Gunnell, D.J.; Hickman, M. Multiple risk behaviour in adolescence. J. Public Health 2012, 34, i1–i2. [Google Scholar] [CrossRef] [PubMed]
  72. Kupersmidt, J.B.; Scull, T.M.; Austin, E.W. Media literacy education for elementary school substance use prevention: Study of media detective. Pediatrics 2010, 126, 525–531. [Google Scholar] [CrossRef][Green Version]
  73. Winkelman, T.N.A.; Caldwell, M.T.; Bertram, B.; Davis, M.M. Promoting health literacy for children and adolescents. Pediatrics 2016, 138, e20161937. [Google Scholar] [CrossRef][Green Version]
Nutrients 13 00790 g001 550
Figure 1. Exploratory factor analysis after direct oblimin rotation (with the screen test plot of eigenvalues) for the NVS-PTeen (N = 386).
Figure 1. Exploratory factor analysis after direct oblimin rotation (with the screen test plot of eigenvalues) for the NVS-PTeen (N = 386).
Nutrients 13 00790 g001
Nutrients 13 00790 g002 550
Figure 2. NVS-PTeen items: characteristic and information curves (N = 386).
Figure 2. NVS-PTeen items: characteristic and information curves (N = 386).
Nutrients 13 00790 g002
Table
Table 1. Sample characterization: age and educational level of the participants by sex.
Table 1. Sample characterization: age and educational level of the participants by sex.
Female
(n = 191)		Male
(n = 195)		Total
(N = 386)		p-Value 1
Age (years)1214 (7.3%)14 (7.2%)28 (7.3%)0.672
1349 (25.7%)45 (23.1%)94 (24.4%)
1439 (20.4%)54 (27.7%)93 (24.1%)
1535 (18.3%)30 (15.4%)65 (16.8%)
1633 (17.3%)34 (17.4%)67 (17.4%)
1721 (11.0%)18 (9.2%)39 (10.1%)
Mean (±SD)14.46 (1.49)14.41 (1.43)14.43 (1.46)0.791
Median14.0014.0014.00
Educational levelGrade 866 (34.6%)60 (30.8%)126 (32.6%)
Grade 943 (22.5%)61 (31.3%)104 (26.9%)
Grade 1029 (15.2%)21 (10.8%)50 (13.0%)0.321
Grade 1135 (18.3%)36 (18.5%)71 (18.4%)
Grade 1218 (9.4%)17 (8.7%)35 (9.1%)
1p-Value calculated using the chi-square test for categorical variables and the Mann–Whitney U test for continuous variables.
Table
Table 2. The Newest Vital Sign for the Portuguese adolescents’ population (NVS-PTeen) score comparison for (a) sex and (b) educational level.
Table 2. The Newest Vital Sign for the Portuguese adolescents’ population (NVS-PTeen) score comparison for (a) sex and (b) educational level.
NVS-PTeen Items (I)NVS Total Score
(Range: 0–6;
Mean ± SD)		Health Literacy Level
I1 (%)I2 (%)I3 (%)I4 (%)I5 (%)I6 (%)Inadequate (Score 0–1; %)Limited
(Score 2–3; %)		Adequate
(Score 4–6; %)
SexFemale (n = 191)84.371.283.271.290.679.14.72 ± 1.413.715.780.6
Male (n = 195)92.371.388.775.488.279.04.86 ± 1.272.111.386.7
p-value 10.0140.9870.1210.3530.4500.9840.5260.259
Educational levelGrade 8 (n = 126)84.161.186.557.184.963.54.27 ± 1.474.823.872.2
Grade 9 (n = 104)85.673.176.964.486.578.84.50 ± 1.313.816.379.8
Grade 10 (n = 50)94.094.096.096.096.092.05.68 ± 0.892.02.096.0
Grade 11 (n = 71)90.173.288.788.795.891.05.28 ± 1.030.05.694.4
Grade 12 (n = 35)100.065.791.497.191.494.35.29 ± 0.830.02.997.1
p-value 20.0530.0010.013<0.0010.060<0.001<0.001 <0.001
Total (N = 386)88.371.286.073.389.479.04.79 ± 1.342.813.583.4
1p-Value calculated using the chi-square test for percentages and the Mann–Whitney U test for continuous variables; 2 p-value calculated with the chi-square test for percentages and the Kruskal–Wallis test for continuous variables. Statistically significant values are in bold.
Table
Table 3. Inter-item and item-total NVS-PTeen scores (Spearman’s rank correlation coefficients matrix and inter-item pairwise odds ratio (OR) with 95% confidence intervals (95% CI); N = 386).
Table 3. Inter-item and item-total NVS-PTeen scores (Spearman’s rank correlation coefficients matrix and inter-item pairwise odds ratio (OR) with 95% confidence intervals (95% CI); N = 386).
Item 2Item 3Item 4Item 5Item 6NVS-PTeen
Total Score
rhoOR
(95% CI)		rhoOR
(95% CI)		rhoOR
(95% CI)		rhoOR
(95% CI)		rhoOR
(95% CI)		rho
Item 1:If you eat the whole container of ice cream, how many calories are you going to consume?0.197 **3.33
(1.77–6.29)		0.179 **3.40
(1.67–6.93)		0.201 **3.41
(1.81–6.44)		0.163 **3.35
(1.54–7.29)		0.170 **2.94
(1.53–5.67)		0.445 **
Item 2:If you could eat 60 g of carbohydrates, what quantity of ice cream would you be able to eat? 0.0571.42
(0.77–2.60)		0.315 **4.37
(2.69–7.09)		0.115 *2.12
(1.10–4.11)		0.0941.63
(0.97–2.73)		0.596 **
Item 3:Your doctor has advised you to reduce the amount of saturated fat in your diet. You generally eat 42 g of saturated fat per day, which includes one portion of ice cream. If you were to stop eating ice cream, how many grams of saturated fat would you be consuming per day? 0.213 **3.37
(1.86–6.09)		0.176 **3.44
(1.65–7.17)		0.196 *3.18
(1.73–5.86)		0.447 **
Item 4:If you generally eat 2500 calories per day, what percentage of the daily value of calories would you be consuming if you ate one portion of ice cream? 0.210 **3.77
(1.95–7.32)		0.264 **3.75
(2.24–6.28)		0.662 **
Item 5:Is it safe for you to eat this ice cream? 0.586 *63.28
(21.51–186.15)		0.498 **
Item 6 (asked if the participant answered no to item 5): Why not? 0.599 **
* p < 0.05; ** p < 0.01.
Table
Table 4. Internal consistency (coefficient α) and reproducibility of the NVS-PTeen given as temporal reliability.
Table 4. Internal consistency (coefficient α) and reproducibility of the NVS-PTeen given as temporal reliability.
Coefficient α If Item Deleted
(N = 386)		Item Difficulty: % of Students Answering the Item Correctly
(N = 386)		Temporal Reliability
% of Test-Retest Accuracy
(n = 127)		% of Test-Retest Score Improvement
(n = 127)
Item 10.5888.3%96.9%0.8%
Item 20.6071.2%66.1%27.6%
Item 30.5986.0%78.7%7.1%
Item 40.5273.3%81.9%6.3%
Item 50.5389.4%92.9%3.1%
Item 60.5379.0%86.6%10.2%
Table
Table 5. Convergent validity of the NVS-PTeen assessed as the correlation between theoretically related variables and the NVS-PTeen total score.
Table 5. Convergent validity of the NVS-PTeen assessed as the correlation between theoretically related variables and the NVS-PTeen total score.
Pearson’s r
Age (N = 386)0.19 *
Educational level (N = 386)0.31 **
Final classification: Portuguese 1 (N = 386)0.28 **
Final classification: mathematics 1 (N = 358)0.31 **
Average final classification: Portuguese and mathematics 1 (N = 358)0.39 **
1 Final classifications from the previous academic year; * p < 0.05; ** p < 0.001.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Santos, O.; Stefanovska-Petkovska, M.; Virgolino, A.; Miranda, A.C.; Costa, J.; Fernandes, E.; Cardoso, S.; Vaz Carneiro, A. Functional Health Literacy: Psychometric Properties of the Newest Vital Sign for Portuguese Adolescents (NVS-PTeen). Nutrients 2021, 13, 790. https://doi.org/10.3390/nu13030790

AMA Style

Santos O, Stefanovska-Petkovska M, Virgolino A, Miranda AC, Costa J, Fernandes E, Cardoso S, Vaz Carneiro A. Functional Health Literacy: Psychometric Properties of the Newest Vital Sign for Portuguese Adolescents (NVS-PTeen). Nutrients. 2021; 13(3):790. https://doi.org/10.3390/nu13030790

Chicago/Turabian Style

Santos, Osvaldo, Miodraga Stefanovska-Petkovska, Ana Virgolino, Ana Cristina Miranda, Joana Costa, Elisabete Fernandes, Susana Cardoso, and António Vaz Carneiro. 2021. "Functional Health Literacy: Psychometric Properties of the Newest Vital Sign for Portuguese Adolescents (NVS-PTeen)" Nutrients 13, no. 3: 790. https://doi.org/10.3390/nu13030790

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop