Next Article in Journal
Impacts of Public Debates on Legalizing the Same-Sex Relationships on People’s Daily Lives and Their Related Factors in Taiwan
Next Article in Special Issue
Cross-Cultural Adaptation, Validation and Reliability of the Spanish Satisfaction with Daily Occupations-Occupational Balance (SDO-OB): An Evaluation Tool for People with Mental Disorders
Previous Article in Journal
The Association between Nursing Skill Mix and Mortality for Adult Medical and Surgical Patients: Protocol for a Systematic Review
Previous Article in Special Issue
Development and External Validity of a Short-Form Version of the INICIARE Scale to Classify Nursing Care Dependency Level in Acute Hospitals
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
IJERPH
Volume 17
Issue 22
10.3390/ijerph17228605
ijerph-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Content Validation of an Instrument for the Assessment of School Teachers’ Levels of Knowledge of Diabetes through Expert Judgment

by
Trinidad Luque-Vara
1,
Marta Linares-Manrique
1,
Elisabet Fernández-Gómez
1,*,
Adelina Martín-Salvador
1,
María Angustias Sánchez-Ojeda
1 and
Carmen Enrique-Mirón
2
1
Department of Nursing, Faculty of Health Sciences, Melilla Campus, University of Granada, Calle Santander s/n, 52001 Melilla, Spain
2
HUM-613 Research Group, Department of Inorganic Chemistry, Faculty of Health Sciences, Melilla Campus, University of Granada, Calle Santander s/n, 52001 Melilla, Spain
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2020, 17(22), 8605; https://doi.org/10.3390/ijerph17228605
Submission received: 7 October 2020 / Revised: 16 November 2020 / Accepted: 17 November 2020 / Published: 19 November 2020
(This article belongs to the Special Issue Instruments for Measuring Health)
Versions Notes

Abstract

The objective of this study was to describe the content validation, through expert judgment, of a questionnaire for determining the level of knowledge that school teachers have of diabetes in order to design relevant educational interventions to improve the health of school-aged children. This psychometric study involved 15 experts who assessed each of the items in the instrument. The results revealed that the strength of agreement shown by the questionnaire ranged from substantial to almost perfect in its four dimensions, with the characteristics of “sufficiency” and “relevance” having the highest scores (0.982 and 0.903, respectively) based on the judgments made by the participating experts. Regarding statistical significance, the characteristics “sufficiency”, with p < 0.001, and “relevance”, with p = 0.001, were particularly relevant. The overall degree of understandability for the new version of the instrument was high (91.54%). The psychometric results obtained from validation of the “grado de conocimientos sobre diabetes en docentes del ámbito escolar” (Spanish for “level of knowledge of diabetes in school teachers”)—GCDDaE questionnaire through expert judgment and the pre-test indicate that it is recommended for use as it is both relevant and quick and easy to administer.

1. Introduction

Diabetes is one of the most common metabolic disorders in the world. In 2019, there were 578 million people with diabetes globally, which is expected to grow to 700 million by 2045 [1]. Diabetes is associated with levels of morbidity and mortality that result in a major public health problem and significant economic expenditure for both families and national economies [2]. According to the International Diabetes Federation, approximately 1,110,100 children and adolescents have type 1 diabetes mellitus (DM1), with 98,200 newly diagnosed cases each year. This figure rises to 128,900 when the age range is under 20 years, with significant regional and national differences in the number of children and adolescents (aged 0–14) with prevalent (existing) and incident (newly diagnosed) DM1 [3]. In Spain, epidemiological studies on DM1 in children under 15 years of age have revealed a mean incidence of 17.69 cases per 100,000 inhabitants per year [4].
The criteria established in the “Standards of Medical Care in Diabetes” by the American Diabetes Association [5] mention that schools must ensure that children with DM1 have a safe environment that allows for proper control of the disease and thus facilitate their full participation in school and extracurricular activities. In this regard, it is essential that teachers have a minimum knowledge of the management of the disease, particularly in the event of an emergency, in order to improve the quality of life of children with diabetes and avoid long-term complications [6].
There is also a need for health measuring instruments that may be used in clinical and research practice. These instruments must undergo a validation process, which is a prerequisite to being able to ensure their quality [7,8].
Research on the validation process as well as on the construction and adaptation of measurement instruments is insufficient in the field of health, and information is often fragmented [9].
According to Galicia et al. [10], there are different types of validation methods, including construct validity, criterion validity, and content validity, with construct validity being the most widely used. In this study, content validity was the validation criterion used.
Content validity may be defined as “the degree to which elements of the measurement instrument are comprehensive, relevant, and representative of the construct for a particular assessment purpose” [11] (p.238). As a result, higher levels of content validity in an instrument are indicative of higher levels of precision in the measurement of the target construct [12]. For a measurement instrument to be considered to be valid, it must be simple and affordable, free of bias, appropriate for the problem under study, have clearly defined dimensions, and adequately reflect the concept that it is intended to measure [13].
In addition, instruments must have two essential characteristics of good quality, i.e., validity and reliability, so that they can be used by researchers in their studies [14,15] while ensuring the integrity of their results [12].
The usual procedure for assessing the content validity of the instrument is to consult experts [16]. According to Escobar-Pérez and Cuervo-Martínez [17], expert judgment may be defined as an informed opinion from individuals with a track record in the field, who are regarded by others as qualified experts in the field, and who can provide information, evidence, judgments, and assessments (p. 29). The way these judges are chosen is thus fundamental, since they must be experts in the field, either because of their academic training or because of their work experience. Once selected, the experts make an assessment of the different dimensions of the questionnaire using a numerical scale as part of the procedure [10]. According to Fernández-Gómez et al. [18], the said assessment should incorporate both qualitative and quantitative data. The process that judges undertake becomes a fundamental effort to eliminate unimportant aspects, modify aspects that need to be modified, and incorporate relevant aspects [15]. The researcher will then analyze this process thoroughly, and will determine what to modify, improve, or remove from the instrument to be validated [10].
With this technique, it is possible to calculate its content validity index, with the adoption of rigorous statistics and an appropriate methodology, so that the instrument assessed may be used for the purpose for which it was designed [19].
Today, an increasing number of school children have chronic health problems, diabetes is one of the most prevalent conditions affecting this group [20]. Given that diabetes at a young age requires constant care in both family and school settings [21] and that, according to a number of authors [22,23,24,25,26,27,28] teachers’ knowledge of diabetes is insufficient, this study aims to present a validated questionnaire for gathering information on teachers’ levels of knowledge of this disease.
Appropriate control of this condition can prevent the occurrence of complications related to elevated blood glucose. In particular, aspects such as the frequent monitoring of blood glucose that must be performed daily by the child, the administration of insulin either through multiple injections or with an infusion pump, the monitoring of physical activity and nutrition [29,30], and most importantly, hypoglycemia, which demands immediate care [31]. Such control should be addressed in the school environment, as children spend much of their day in school and the likelihood of experiencing complications from this disease is high [32].
It is essential that school staff is appropriately trained to provide safe and effective care to students with diabetes [33]. This is why new educational strategies must be developed to improve teachers’ knowledge of this disease, which is so widespread among children and adolescents.
The objective of this study was to present the content validation procedure of an instrument for determining the degree of knowledge of DM1 in school teachers through expert judgment, which will make it possible to design promotion, prevention, and health education plans for school teachers that will undoubtedly have an impact on the quality of life of schoolchildren with diabetes.

2. Materials and Methods

This study, which was conducted at the Melilla Campus of the University of Granada, Spain, used a descriptive, cross-sectional design to present the content validation procedure for a questionnaire to determine levels of knowledge of DM1 among school teachers using expert judgment and a pre-test.

2.1. Participants

The sample was selected using convenience (intentional) sampling, and consisted of 15 professionals from different areas of knowledge, with professional experience (in both teaching and conducting research) of between 5 and 30 years (SD: 19.4 years). They all voluntarily agreed to participate in the study by signing an informed consent form. Table 1 shows the areas of knowledge as well as the years of experience for each of the experts.

2.2. Instrument

The validation process was applied to a 33-item questionnaire built on the basis of the 24 items that make up the diabetes knowledge questionnaire (DKQ-24) [34], an instrument that was designed to assess general knowledge of diabetes in individuals in accordance with the content recommendations of the U.S. National Standards for Diabetes Patient Education Programs [35], plus 9 newly created items to complement the different aspects under study, such as knowledge of complications and knowledge of diabetic patient care (Appendix A). These 9 items were prepared based on the literature review conducted using the Web of Science database taking into account the classical test theory [36].
The questionnaire, “Grado de conocimientos sobre diabetes en docentes del ámbito escolar” (Spanish for “Level of knowledge of diabetes in school teachers”), henceforth GCDDaE, consists of 4 dimensions: general knowledge of the disease (14 items), knowledge of symptoms (5 items), knowledge of complications (5 items), and knowledge of diabetic patient care (9 items). The questionnaire was evaluated by assessing each item using the “expert judgment” template developed by Escobar-Pérez and Cuervo-Martínez [17]. These authors established four categories to be assessed (sufficiency, clarity, coherence, and relevance) using a scale with four options: does not meet the criterion, low level, moderate level, and high level (Table 2). In addition, also included was the possibility for experts to make qualitative observations on the accuracy of the language and content of each of the 33 items that made up the original instrument.

2.3. Procedure

After contacting the experts by telephone, the questionnaire to be validated was sent to them by e-mail together with the information necessary to make their evaluation as well as the informed consent form and our assurance that their data would remain confidential. The experts were given one month to assess and rate all the items. After this time, they returned their assessments and their signed informed consent form by e-mail. No reminders had to be sent to them.
Once the questionnaire had been evaluated by the experts, and taking into account the qualitative observations made, two items (belonging to the dimension “knowledge of diabetic patient care”) were removed from the original questionnaire because they failed to meet all the established requirements, resulting in a final instrument consisting of 31 items distributed in the same 4 aforementioned dimensions: general knowledge of the disease (14 items), knowledge of diabetic patient care (7 items), knowledge of symptoms (5 items), and knowledge of complications (5 items).
Subsequently, the instrument was pre-tested using dichotomous yes/no responses in order to measure its degree of real-life feasibility and/or applicability and to assess the level of understandability of each item [37]. The pre-test was conducted with a random sample of 114 education students from the Faculty of Education and Sport Sciences at the Melilla Campus of the University of Granada, Spain, who had previously submitted their signed informed consent form. The percentages and levels of comprehensibility obtained with the data collected were classified into the following categories: high (equal to or greater than 85%), medium (from 80 to 85%), and low (less than 80%).

2.4. Ethical Considerations

All research participants were duly informed and voluntarily accepted to take part in the study by signing the corresponding informed consent, referring to the ethical principles of the Helsinki Declaration. The approval of the Ministry of Education and Vocational Training was obtained with reference: EA0022075s18N0002381 on 11 December 2018.

2.5. Statistical Analysis

Data were processed using the statistical package SPSS, version 24.0, for Windows. As the instrument contained a number of categories of an ordinal nature, Fleiss’ κ coefficient was used to assess the degree of agreement between the judges [18,38,39].
Fleiss’ κ values, which can range from 0 to 1, were interpreted using the classification created by Landis & Koch [40], as shown in Table 3, which expresses the strength of agreement between the judges.
Basic statistics (frequencies and percentages) were used to analyze the data collected in the pre-test.

3. Results

3.1. Content Validation through Expert Judgment

Table 4 shows the Fleiss’ κ values for the proportion of possible agreements occurring in each dimension, as well as the strength of agreement estimated using the classification created by Landis and Koch [40]. In this case, the agreement is considered to be almost perfect for three dimensions (“general knowledge of the disease”, “knowledge of symptoms”, and “knowledge of complications”) and substantial for the remaining dimension (“knowledge of diabetic patient care”).
Fleiss’ κ values by pairs of experts were also determined, which yielded a greater variability, between moderate and almost perfect, revealing some discordance, especially in the dimension of “knowledge of diabetic patient care”, as shown in Table 5.
In order to assess the characteristics of the instrument as a whole, the categorical indicators of sufficiency, clarity, coherence, and relevance were rated using an ordinal scale. The results revealed a strength of agreement between substantial and almost perfect, with “sufficiency” and “relevance” having the highest Fleiss’ κ values (0.982 and 0.903, respectively). Statistical significance was below 0.05 in all cases, with a 95% confidence. As shown in Table 6, the characteristics “sufficiency” (p < 0.001) and “relevance” (p = 0.001) were especially relevant.
This statistical procedure and its results, together with the qualitative observations and recommendations made by the judges for each item, produced the final validated instrument (Appendix A).

3.2. Measurement of Applicability: Pre-Test

The final validated instrument (Appendix A) was completed by a total of 114 education students, and therefore future educators, to determine the degree of comprehensibility of the dimensions and their corresponding items. All of them were valid. The analysis of the data (percentages of yes/no responses given by the participants) allowed us to establish the degree of understandability of the instrument in the highest range, with percentages of understanding equal to or higher than 85% for all items, as shown in Table 7. The overall understandability of the final validated questionnaire was 91.54%.

4. Discussion

This study presents the content validation through expert judgment of an instrument for determining the degree of knowledge of diabetes in school teachers. Several studies highlight that these professionals’ knowledge of different aspects regarding emergencies in individuals with diabetes, such as hypoglycemic and hyperglycemic events, is quite limited, making it difficult to safely and fully integrate children with diabetes into schools [22,27,28,41,42].
The design and construction of a measurement instrument is a complex process in which the aim is to define the objective, the references, the construct, and the dimensions in which the items are proposed [43]. This is not a disadvantage when it comes to obtaining an easily and quickly administered instrument with acceptable psychometric results both at a global level and at the level of each dimension and, of course, an instrument that is welcomed by professionals [44].
This study is based mainly on the process of transcription of the instrument, its validation, and the analysis of its psychometric properties. This process was structured following methodological designs that indicate in a stepwise manner how to explore the different psychometric aspects of the instrument, which provide statistical consistency and support that should be taken into account. More psychometric properties being determined across different contexts and subjects will result in a more consistent instrument. Content validation by expert judgment is the key part of the present study [7].
The content validity of a questionnaire is the extent to which its measurement represents the greatest number of dimensions of the concept to be studied. The content of an instrument is thus considered to be valid if it represents all the aspects related to the concept under study [45]. To avoid creating an overly broad, wide-ranging instrument and facilitate completion, a questionnaire should only include the most relevant, informative range of questions regarding the topic it is intended to address [43]. The questionnaire of our study captures, in 31 items, the main aspects needed to determine the level of knowledge of diabetes in school teachers.
This study used the expert judgment technique because this technique is reliable and relatively simple, and because it is frequently used in the content validation of instruments before administering them to the population [46]. However, difficulties are inevitable, such as the number of experts required to participate. In fact, in the literature consulted on this subject, the number of experts required differs considerably, and no consensus has been reached [37]. Delgado-Rico [47] recommend consulting at least three experts; a study by [10] involved eight experts, whereas other authors point out that the number of experts will depend on the objectives of the study, with a range of between seven and thirty judges [48]. In the present study, 15 experts were selected, which is approximately in line with the findings of the different authors consulted. With regard to experience, Escobar-Pérez and Cuervo-Martínez [17] recommend that the expert group should include, among others, measurement and assessment professionals. As a result, in this study, two judges from the field of research and diagnostic methods in education were included, as reported in a study by Fernández-Gómez et al. [18].
In addition, since the aim of this study was to validate the items of a questionnaire on diabetes (the GCDDaE questionnaire), six of the experts were nurses, who were selected for their connection with diabetes education. The judges’ wide variety of areas of knowledge turned out to be very positive, as it brought together different forms of expertise on the subject matter [49]. The quality and diversity of professionals may be more meaningful than their number [50].
The selection of the experts is another important consideration. Cabero and Llorente [37] list a number of selection procedures, including structured selection procedures, such as the proximity or affinity of the researchers to the judges, and unstructured selection procedures, such as graphical biographies and the expert competence coefficient. An unstructured selection procedure has been used in this study.
This type of validation, however, may involve significant errors. If the judges know what is intended to be measured and these constructs are defined by the researchers themselves, there is a risk of directing the experts’ assessments [51].
Nevertheless, if we wish to obtain advantageous results to achieve our main objective, we must pay particular attention to other characteristics in the selection of the judges [37]. According to [17], these other characteristics in the selection of the judges are their experience, their academic and scientific reputation, their availability, and their motivation to participate, but most importantly, their honesty and commitment regarding these characteristics. Judges with the aforementioned criteria participated in this study in order to avoid introducing content bias in the data analysis [36].
In addition, to ensure validity, in addition to the set of experts, it is very useful to include non-specialists, mainly individuals from the reference population [50]. In the present study, 114 university students from the Faculty of Education and Sport Sciences at the Melilla Campus of the University of Granada, Spain, all of them future school teachers, voluntarily participated in the pre-test, in a similar way to what [36] described in their study. As in the case of their study, the degree of comprehensibility of the GCDDaE questionnaire was found to be high (91.54%).
Fleiss’ κ was suitable for identifying and assessing the magnitude of the strength of agreement, which was almost perfect for each dimension and between moderate and almost perfect for pairs of experts, in line with the results obtained by Bernal-García et al. [36]. This coefficient is based on the statistic known as Cohen’s coefficient (κ), which calculates the probability of agreement as long as it is between two raters or observers [52]. However, to calculate the degree of agreement between three or more raters, Fleiss’ κ is used [53], which is the aim of the present study. This procedure offers the possibility of eliminating random chance agreement and provides quantifiable methods to assess observations on content [52]. Evaluating these properties is an essential criterion for determining the quality of their measurements [7].
Following the procedure used by [17], the analysis of each item was performed on the basis of four criteria: sufficiency, clarity, coherence, and relevance, with Fleiss’ κ coefficient confirming the applicability of the test. In addition, the qualitative analysis of the items made by the experts regarding the transcription of the items and the coherence, accuracy, and comprehensiveness of the content allowed us to modify and eliminate items without compromising the validity of the instrument [36]. In the present study, seventeen items were modified and two items were removed after the qualitative analysis of the experts. With respect to the degree of comprehensibility of the dimensions and their items in the final version of the validated instrument, it should be noted that the dimensions that were given the highest scores were those relating to the general knowledge of the disease and its potential complications, as opposed to the dimensions that were given the lowest scores, those relating to the knowledge of symptoms and the knowledge of diabetic patient care.
With regard to the implications for health professionals, it is worth noting the unavoidable need to disseminate guidelines for the creation of questionnaires, their consolidation, and their adaptation using content validation through expert judgment. This could be serve to develop new instruments that contribute to the practice and research of safe and reliable methods that help to improve the quality of life and safety of schoolchildren, also serving as indicators to guide the implementation of strategies, programs, and interventions in relation to students with DM1.
In the process of validation through expert judgment, there are aspects that researchers cannot fully control. As a result, our study is not devoid of limitations. However, although the results of the tests conducted for the validation of this instrument were acceptable, it is advisable to review the content validity of the instrument periodically.
Possible new lines of research derived from this study could be aimed at developing health education training programs in schools and training guidelines on the management of childhood diabetes for the educational community.

5. Conclusions

The results obtained using the Fleiss’ κ coefficient increased the objectivity of the instrument by adding an interesting quantitative value for measuring the degree of agreement between experts and enabling the researchers to validate the questionnaire correctly. To conclude with validity, the pre-test procedure helps to determine the degree of understandability of the original instrument. It is important for both healthcare professionals and researchers to understand the procedure for conducting a study on the content validation of an instrument. We believe it is advisable to use the GCDDaE questionnaire, as it is feasible and quickly administered, it has acceptable psychometric characteristics and has been approved by the experts both at the level of each dimension and at a general level. The questionnaire has a considerable potential to elucidate the degree of knowledge that teachers have and contribute to their training by increasing their knowledge of diabetes and, as a result, increase the quality of life and safety of schoolchildren with DM1.

Author Contributions

Conceptualization, T.L.-V. and E.F.-G.; formal analysis, C.E.-M.; investigation, T.L.-V. and M.L.-M.; methodology, C.E.-M.; resources E.F.-G. and A.M.-S.; software, A.M.-S. and M.A.S.-O.; supervision, M.L.-M.; visualization, E.F.-G. and M.A.S.-O.; writing—original draft preparation, T.L.-V.; writing—review and editing, M.L.-M. and C.E.-M. All of the authors contributed to the conception and design of this study, critically revised the manuscript, gave their final approval, and agreed to be accountable for all aspects of work, ensuring integrity and accuracy. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Conflicts of Interest

The authors declare that there are no conflict of interest.

Appendix A

Table A1. Level of Knowledge of Diabetes.
Table A1. Level of Knowledge of Diabetes.
TrueFalseI Don’t Know
1Diabetes is caused by having a sufficient amount of a hormone called insulin.
2There are two main types of diabetes: type 1 (insulin-dependent) and type 2 (non-insulin-dependent).
3Increases in blood insulin are caused by the intake of large amounts of food.
4Pharmacological treatment is more important in controlling diabetes than diet and exercise.
5Insulin is produced in the kidneys.
6A person with diabetes is more likely to have children with diabetes.
7In untreated diabetes, the level of blood glucose usually increases.
8Diabetes can be cured.
9A fasting blood glucose level of 210 (mg/dL) is very high.
10The best way to control diabetes is through urine tests.
11Regular exercise will increase the need for insulin or other diabetes medication.
12Diabetes often causes poor blood circulation.
13How food is prepared (fried, roasted, etc.) is as important as the food that is consumed.
14Diabetes can cause damage to the kidneys.
15A diet for people with diabetes consists mainly of special foods.
16Eating too much sugar and other sweet foods is one of the causes of diabetes.
17The common cause of diabetes is a lack of effective insulin in the body.
18Diabetes is caused because the kidneys cannot filter the sugar from the blood.
19Frequent urination and thirst are signs of low blood glucose.
20Trembling and sweating are signs of low blood glucose.
21Wounds and scratches heal more slowly in people with diabetes.
22People with diabetes have to be extra careful when clipping their toenails.
23An individual with diabetes should clean the wound first with povidone-iodine and then with alcohol.
24Diabetes can cause a loss of sensation in the hands, fingers, and feet.
25The main symptoms that children with diabetes may have are polydipsia, polyphagia, polyuria, and anorexia, among others.
26In the case of hypoglycemia with loss of consciousness, juice, sweets, sugar, etc. should be administered.
27Capillary blood glucose monitoring is done with a finger prick, on the fingertip if possible, which is the least pain-sensitive area of the finger.
28The area and the injection site of the insulin must be changed weekly.
29Cold insulin hurts less and is better absorbed.
30The inclination of the needle on the skin when punctured should always be 90°.
31Odd behavior and slurred speech are symptoms of hyperglycemia.

References

  1. Saeedi, P.; Petersohn, I.; Salpea, P.; Malanda, B.; Karuranga, S.; Unwin, N.; Colagiuri, S.; Guariguata, L.; Motala, A.A.; Ogurtsova, K.; et al. IDF Diabetes Atlas Committee. Global and regional diabetes prevalence estimates for 2019 and projections for 2030 and 2045: Results from the International Diabetes Federation Diabetes Atlas, 9th edition. Diabetes Res. Clin. Pract. 2019, 157, 107843. [Google Scholar] [CrossRef]
  2. Baena-Díez, J.M.; Peñafiel, J.; Subirana, I.; Ramos, R.; Elosua, R.; Marín-Ibañez, A.; Guembe, M.J.; Rigo, F.; Tormo-Díaz, M.J.; Moreno-Iribas, C.; et al. Risk of cause-specific death in individuals with diabetes: A competing risks analysis. Diabetes Care 2016, 39, 1987–1995. [Google Scholar] [CrossRef]
  3. Atlas de la Diabetes de la FID. Novena Edición 2019. Available online: www.diabetesatlas.org (accessed on 4 November 2020).
  4. Conde Barreiro, S.; Rodríguez Rigual, M.; Bueno Lozano, G.; López Siguero, J.P.; González Pelegrín, B.; Rodrigo Val, M.P.; Compés Dea, M.L. Epidemiología de la diabetes mellitus tipo 1 en menores de 15 años en España. An. Pediatric 2014, 81, 189–e1. [Google Scholar] [CrossRef] [PubMed]
  5. Estándares Para la Atención Médica de la Diabetes. American Diabetes Association. 2019. Available online: https://www.redgdps.org/gestor/upload/2019/2019%20ADA%20ESPAN%CC%83OL.pdf (accessed on 9 May 2020).
  6. Boden, S.; Lloyd, C.E.; Gosden, C.; Macdougall, C.; Brown, N.; Matyka, K. The concerns of school staff in caring for children with diabetes in primary school. Pediatric Diabetes 2012, 13, e6–e13. [Google Scholar] [CrossRef] [PubMed]
  7. Carvajal, A.; Centeno, C.; Watson, R.; Martínez, M.; Sanz Rubiales, Á. ¿Cómo Validar un Instrumento de Medida de la Salud? An. Sist. Sanit. Navar. 2011, 34, 63–72. Available online: http://scielo.isciii.es/scielo.php?script=sci_arttext&pid=S1137-66272011000100007&lng=es&tlng=es (accessed on 7 May 2020). [CrossRef] [PubMed]
  8. Velarde-Jurado, E.; Avila-Figueroa, C. Consideraciones metodológicas para evaluar la calidad de vida [Methodological considerations for evaluating quality of life]. Salud Pública Méx. 2002, 44, 448–463. [Google Scholar] [CrossRef]
  9. Luján-Tangarife, J.A.; Cardona-Arias, J.A. Construcción y validación de escalas de medición en salud: Revisión de propiedades psicométricas. Imedpub J. 2015, 3, 1. [Google Scholar] [CrossRef]
  10. Galicia Alarcón, L.A.; Balderrama Trápaga, J.A.; Edel Navarro, R. Validez de contenido por juicio de expertos: Propuesta de una herramienta virtual. Apertura 2017, 9, 4253. [Google Scholar] [CrossRef]
  11. Haynes, S.N.; Richard, D.C.C.; Kubany, E.S. Content validity in psychological assessment: A functional approach to concepts and methods. Psychol. Assess. 1995, 7, 238–247. [Google Scholar] [CrossRef]
  12. DeVon, H.A.; Bock, M.E.; Moyle-Wright, P.; Ernst, D.M.; Hayden, S.J.; Lazzara, D.J.; Sayou, S.M.; Kostas-Polston, E. A psychometric toolbox for testing validity and reliability. J. Nurs. Scholarsh. 2007, 39, 155–164. [Google Scholar] [CrossRef]
  13. Prieto, L.; Badía, X. Cuestionario de salud: Concepto y metodología. Aten. Prim. 2001, 28, 201–209. [Google Scholar] [CrossRef]
  14. Lacave Rodero, C.; Molina Díaz, A.I.; Fernández Guerrero, M.; Redondo Duque, M.A. Análisis de la fiabilidad y validez de un cuestionario docente. In Proceedings of the Actas de las XXI Jornadas de la Enseñanza Universitaria de la Informática, Andorra La Vella, Andorra, 8–10 July 2015; ISBN 978-99920-70-10-9. [Google Scholar]
  15. Robles Garrote, P.; Rojas, M.D.C. La validación por juicio de expertos: Dos investigaciones cualitativas en Lingüística aplicada. Rev. Nebrija Lingüíst. Apl. Enseñ. Leng. 2015, 18. [Google Scholar] [CrossRef]
  16. Sireci, S.G. The construct of content validity. Soc. Indic. Res. 1998, 45, 83–117. [Google Scholar] [CrossRef]
  17. Escobar-Pérez, J.; Cuervo-Martínez, A. Validez de contenido y juicio de expertos: Una aproximación a su utilización. Av. Med. 2008, 6, 27–36. [Google Scholar]
  18. Fernández-Gómez, E.; Martín-Salvador, A.; Luque-Vara, T.; Sánchez-Ojeda, M.A.; Navarro-Prado, S.; Enrique-Mirón, C. Content validation through expert judgement of an instrument on the nutrional knowledge, beliefs, and habits of pregnant women. Nutrients 2020, 12, 1136. [Google Scholar] [CrossRef]
  19. García Perales, R. Diseño y construcción de un instrumento de evaluación de la competencia matemática: Aplicabilidad práctica de un juicio de expertos. Ensaio Aval. Pol. Públ. Educa. Rio Janeriro 2018, 26, 1–26. [Google Scholar] [CrossRef]
  20. Clay, D.; Farris, K.; McCarthy, A.M.; Kelly, M.W.; Howarth, R. Percepciones familiares de la administración de medicamentos en la escuela: Errores, factores de riesgo y consecuencias. J. Sch. Nurs. 2008, 24, 95–102. [Google Scholar] [CrossRef]
  21. Bobo, N.; Kaup, T.; McCarty, P.; Carlson, J.P. Diabetes management at school: Application of the healthy learner model. J. Sch. Nurs. 2011, 27, 171–184. [Google Scholar] [CrossRef]
  22. Aycan, Z.; Önder, A.; Çetinkaya, S.; Bilgili, H.; Yıldırım, N.; Baş, V.N.; Peltek Kendirci, H.A.; ğladıoğlu, S.Y. Assessment of the knowledge of diabetes mellitus among school teachers within the scope of the managing diabetes at school program. J. Clin. Res. Pediatric Endocrinol. 2012, 4, 199–203. [Google Scholar] [CrossRef]
  23. Kise, S.S.; Hopkins, A.; Burke, S. Improving school experiences for adolescents. J. Sch. Health 2017, 87, 363–375. [Google Scholar] [CrossRef]
  24. Marks, A.; Wilson, V.; Crisp, J. The management of type 1 diabetes in primary school: Review of the literature. Issues Compr. Pediatric Nurs. 2013, 36, 1–2. [Google Scholar] [CrossRef]
  25. Wodrich, D.L.; Hasan, K.; Parent, K.B. Type 1 diabetes mellitus and school: A review. Pediatric Diabetes 2011, 12, 63–70. [Google Scholar] [CrossRef] [PubMed]
  26. Wang, Y.L.; Brown, S.A.; Horner, S.D. The school-based lived experiences of adolescents with type 1 diabetes. J. Nurs. Res. 2013, 21, 235–243. [Google Scholar] [CrossRef] [PubMed]
  27. Chatzistougianni, P.; Tsotridou, E.; Dimitriadou, M.; Christoforidis, A. Level of knowledge and evaluation of perceptions regarding pediatric diabetes among Greek teachers. Diabetes Res. Clin. Pract. 2020, 159, 107952. [Google Scholar] [CrossRef] [PubMed]
  28. Gutiérrez-Manzanedo, J.V.; Carral-San Laureano, F.; Moreno-Vides, P.; de Castro-Maqueda, G.; Fernández-Santos, J.R.; Ponce-González, J.G. Teachers’ knowledge about type 1 diabetes in south of Spain public schools. Diabetes Res. Clin. Pract. 2018, 143, 140–145. [Google Scholar] [CrossRef] [PubMed]
  29. Cheung, R.; Cureton, V.Y.; Canham, D.L. Quality of life in adolescents with type 1 diabetes who participate in diabetes camp. J. Sch. Nurs. 2006, 22, 53–58. [Google Scholar] [CrossRef]
  30. Jackson, C.C.; Albanese-O’Neill, A.; Butler, K.L.; Chiang, J.L.; Deeb, L.C.; Hathaway, K.; Kraus, E.; Weissberg-Benchell, J.; Yatvin, A.L.; Siminerio, L.M. Diabetes care in the school setting: A position statement of the American Diabetes Association. Diabetes Care 2015, 38, 1958–1963. [Google Scholar] [CrossRef]
  31. Evert, A. Managing hypoglycemia in the school setting. Sch. Nurse News 2015, 11, 16–20. [Google Scholar]
  32. Tolbert, R. Managing type 1 diabetes at school: An integrative review. J. Sch. Nurs. 2009, 25, 55–61. [Google Scholar] [CrossRef]
  33. Wang, Y.L.; Volker, D.L. Caring for students with type 1 diabetes: School nurses’ experiences. J. Sch. Nurs. 2013, 29, 31–38. [Google Scholar] [CrossRef]
  34. Garcia, A.A.; Villagomez, E.T.; Brown, S.A.; Kouzekanani, K.; Hanis, C.L. The Starr County Diabetes Education Study: Development of the Spanish-language diabetes knowledge questionnaire. Diabetes Care 2001, 24, 16–21. [Google Scholar] [CrossRef]
  35. National standards for diabetes patient education programs: National diabetes advisory board. Diabetes Educ. 1984, 9, 11–13. [CrossRef] [PubMed]
  36. Bernal-Garcia, M.I.; Salamanca Jiménez, D.R.; Pérez Gutiérrez, N.; Quemba Mesa, M. Validez de contenido por juicio de expertos de un instrumento para medir percepciones físico-emocionales en la práctica de disección anatómica. Educ. Med. 2018. [Google Scholar] [CrossRef]
  37. Cabero Almenara, J.; Llorente Cejudo, M.C. La aplicación del juicio de expertos como técnica de evaluación de las tecnologías de la información y comunicación (TIC). Rev. Tecnol. Inf. Comun. Educ. 2013, 7, 11–22. [Google Scholar]
  38. Falotico, R.; Quatto, P. Fleiss’ kappa statistic without paradoxes. Qual. Quant. 2015, 49, 463–470. [Google Scholar] [CrossRef]
  39. McHugh, M.L. Interrater reliability: The kappa statistic. Biochem. Med. 2012, 22, 276–282. [Google Scholar] [CrossRef]
  40. Landis, J.; Koch, G. The measurement of observer agreement for categorical data. Biometrics 1977, 33, 159–174. [Google Scholar] [CrossRef]
  41. Gómez Manchón, M.; Gómez Carrasco, J.A.; Ramírez Fernández, J.; Díez Fernández, T.; García de Frías, E. Necesidades del niño escolarizado con diabetes mellitus. Visión de padres y profesores [Special needs of schoolchildren with diabetes mellitus. Point of view of parents and teachers]. An. Pediatr. 2009, 70, 45–52. [Google Scholar] [CrossRef]
  42. Rodriguez Dixe, M.D.A.C.R.; Gomes de Oliveira Gordo, C.M.G.; Pereira Catarino, H.B.; Kraus, T.; Guilherme Menino, E.P.D.S. Effects of an education program on knowledge and self-perception of school personnel in preparing to care for type 1 diabetes students. Einstein 2020, 18. [Google Scholar] [CrossRef]
  43. Alarcon, A.M.; Muñoz, S. Medición en salud: Algunas consideraciones metodológicas. Rev. Méd. Chile 2008, 136, 125–130. [Google Scholar] [CrossRef]
  44. Ramos-Morcillo, A.J.; Martínez-López, E.J.; Fernández-Salazar, S.; del-Pino-Casado, R. Diseño y validación de un cuestionario sobre las actitudes ante la prevención y promoción de la salud en atención primaria (CAPPAP). Aten. Primaria 2013, 45, 514–521. [Google Scholar] [CrossRef] [PubMed][Green Version]
  45. García de Yébenes Prous, M.J.; Rodríguez Salvanés, F.; Carmona Ortells, L. Validación de cuestionarios. Reumatol. Clín. 2009, 5, 171–177. [Google Scholar] [CrossRef] [PubMed]
  46. Zarco, V.A.; Cardoso, G.M.A.; Sánchez, R.J.G.; Arellano, C.J.F. Construcción y validación por juicio de expertos de un instrumento para medir atribuciones causales de las complicaciones crónicas de la diabetes mellitus. Rev. Salud Pública Nutr. 2017, 16, 35–44. [Google Scholar]
  47. Delgado-Rico, E.; Carretero-Dios, H.; Ruch, W. Content validity evidences n test development: An applied perspective. Int. J. Clin. Health Psychol. 2012, 12, 449–460. [Google Scholar]
  48. Urrutia Egaña, M.; Barrios Araya, S.; Gutiérrez Núñez, M.; Mayorga Camus, M. Métodos óptimos para determinar validez de contenido. Rev. Cuba. Educ. Médica Super. 2014, 28, 547–558. [Google Scholar]
  49. Moura, E.R.F.; Bezerra, C.G.; de Oliveira, M.S.; Damasceno, M.M.C. Validação de jogo educativo destinado à orientação dietética de portadores de diabetes mellitus. Rev. APS 2008, 11, 435–443. [Google Scholar]
  50. Koller, I.; Levenson, M.R.; Glük, J. What do you think you are measuring? A mixed-methods procedure for assessing the content validity of test items and theory-based scaling. Front. Psychol. 2017, 8, 126. [Google Scholar] [CrossRef]
  51. Pedrosa, I.; Suárez-Álvarez, J.; García-Cueto, E. Evidencia sobre la validez de contenido: Avances teóricos y métodos para su estimación. Acción Psicol. 2014, 40, 3–20. [Google Scholar] [CrossRef]
  52. Wynd, C.A.; Schmidt, B.; Schaefer, M.A. Two quantitative approaches for estimating content validity. West. J. Nurs. Res. 2003, 25, 508–518. [Google Scholar] [CrossRef]
  53. Cerda Lorca, J.; Villarroel del, P. L Evaluación de la concordancia inter-observador en investigación pediátrica: Coefeiciente de Kappa. Rev. Chil. Pediatric 2008, 79, 54–58. [Google Scholar] [CrossRef]
Table 1. Judges, areas of knowledge, and work experience.
Table 1. Judges, areas of knowledge, and work experience.
JudgeField of Expertise/Academic TrainingYears of Work Experience
1Language and Literature Didactics29
2Language and Literature Didactics35
3Health Sciences5
4Health Sciences33
5Health Sciences23
6Health Sciences21
7Health Sciences6
8Health Sciences7
9Research and Diagnosis Methods in Education21
10Research and Diagnosis Methods in Education18
11Personality, Evaluation, and Psychological Treatment34
12Personality, Evaluation, and Psychological Treatment30
13Didactics and School Organization14
14Nutrition and Food Science6
15Biochemistry and Molecular Biology II and Immunology10
Table 2. Categories and indicators used by the judges to validate the tool.
Table 2. Categories and indicators used by the judges to validate the tool.
CategoriesIndicators
Sufficiency—The items within the same dimension suffice to measure this dimensionThe items are sufficient to measure the dimension
The items measure some aspects of the dimension, but do not represent the full dimension
A few items must be added in order to fully assess the dimension
The items are insufficient
Clarity—The item can be understood easily, i.e., syntax and semantics are appropriateThe item is unclear
The wording of the item requires several modifications or a very large modification in terms of meaning or word order
Some of the terms in the item require very precise modifications
The item is clear, with appropriate semantics and syntax
Coherence—The item is logically related to the dimension or indicator
it is measuring		The item bears no logical relationship to the dimension
The item has a tangential relationship to the dimension
The item has a moderate relationship to the dimension
it is measuring
The item is completely related to the dimension
it is measuring
Relevance—The item is essential or important, i.e., it must be includedThe removal of the item would not affect the measurement of the dimension
The item is somewhat relevant, but another item may be covering what this item is measuring
The item is rather important
The item is very relevant and should be included
Source: adapted from Escobar-Pérez and Cuervo-Martínez (2008, p. 37).
Table 3. Fleiss’ κ values and strength of agreement (Landis and Koch, 1977).
Table 3. Fleiss’ κ values and strength of agreement (Landis and Koch, 1977).
Fleiss’ κStrength of Agreement
0Poor
0.1–0.20Slight
0.21–0.40Fair
0.41–0.60Moderate
0.61–0.80Substantial
0.81–1.00Almost perfect
Table 4. Strength of agreement among judges for the dimensions of the original instrument.
Table 4. Strength of agreement among judges for the dimensions of the original instrument.
DimensionsFleiss’ κStrength of Agreement (Landis & Koch, 1977)
General knowledge of the disease (D1)0.902Almost perfect
Knowledge of symptoms (D2)0.898Almost perfect
Knowledge of complications (D3)0.873Almost perfect
Knowledge of diabetic patient care (D4)0.791Substantial
Table 5. Agreement by pairs of experts.
Table 5. Agreement by pairs of experts.
DimensionsFleiss’ κ—Agreement by Pairs of Experts
1–152–143–134–125–116–107–98–79–610–511–412–313–214–115–8
D10.9290.8680.7630.8850.7170.92010.8300.8110.7850.9000.7350.88911
D210.9840.7110.7000.7320.8330.9310.8460.7440.7140.7060.7080.94911
D310.8440.8110.7900.8320.8930.9310.7460.9440.8140.9060.8500.84911
D40.8120.7840.8110.7320.7120.6330.7530.6460.6740.8140.6560.7180.74910.892
Table 6. Fleiss’ κ coefficient and statistical significance of the characteristics of the original instrument.
Table 6. Fleiss’ κ coefficient and statistical significance of the characteristics of the original instrument.
CharacteristicsFleiss’ κp
Sufficiency0.9820
Clarity0.7920.024
Coherence0.8120.016
Relevance0.9030.001
Table 7. Percentages of comprehensibility of the dimensions and their items in the final version of the validated instrument.
Table 7. Percentages of comprehensibility of the dimensions and their items in the final version of the validated instrument.
DimensionItemDegree of Comprehensibility (%)
YesNo
General knowledge of the disease (D1)Insulin95.53.5
Types of diabetes97.32.7
Relationship between food intake and insulin93.86.2
Medication versus diet92.97.1
Insulin production91.28.8
Heredity and diabetes96.53.5
Glycemia and treatment94.75.3
Cure for diabetes93.85.2
Blood sugar levels86.713.3
Diabetes control96.53.5
Physical activity and diabetes88.510.5
Cause of diabetes: kidney filtering failure95.64.4
Cause of diabetes: insulin96.53.5
Cause of diabetes: intake of sugar and sweets94.75.3
Knowledge of symptoms (D2)Frequent urination and thirst93.86.2
Trembling and sweating93.86.2
Hypoglycemia87.612.4
Hyperglycemia85.015.0
Diabetes in children 85.015.0
Knowledge of complications (D3)Kidney damage 96.53.5
Blood circulation97.32.7
Loss of sensation in extremities94.75.3
Wound healing93.86.2
Cuts89.410.6
Knowledge of diabetic patient care (D4)Food preparation85.015.0
Special foods90.39.7
Cleaning of wounds92.97.1
Capillary blood glucose monitoring87.612.4
Insulin injection site91.28.8
Insulin temperature88.511.5
Inclination of the needle87.612.4
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Luque-Vara, T.; Linares-Manrique, M.; Fernández-Gómez, E.; Martín-Salvador, A.; Sánchez-Ojeda, M.A.; Enrique-Mirón, C. Content Validation of an Instrument for the Assessment of School Teachers’ Levels of Knowledge of Diabetes through Expert Judgment. Int. J. Environ. Res. Public Health 2020, 17, 8605. https://doi.org/10.3390/ijerph17228605

AMA Style

Luque-Vara T, Linares-Manrique M, Fernández-Gómez E, Martín-Salvador A, Sánchez-Ojeda MA, Enrique-Mirón C. Content Validation of an Instrument for the Assessment of School Teachers’ Levels of Knowledge of Diabetes through Expert Judgment. International Journal of Environmental Research and Public Health. 2020; 17(22):8605. https://doi.org/10.3390/ijerph17228605

Chicago/Turabian Style

Luque-Vara, Trinidad, Marta Linares-Manrique, Elisabet Fernández-Gómez, Adelina Martín-Salvador, María Angustias Sánchez-Ojeda, and Carmen Enrique-Mirón. 2020. "Content Validation of an Instrument for the Assessment of School Teachers’ Levels of Knowledge of Diabetes through Expert Judgment" International Journal of Environmental Research and Public Health 17, no. 22: 8605. https://doi.org/10.3390/ijerph17228605

APA Style

Luque-Vara, T., Linares-Manrique, M., Fernández-Gómez, E., Martín-Salvador, A., Sánchez-Ojeda, M. A., & Enrique-Mirón, C. (2020). Content Validation of an Instrument for the Assessment of School Teachers’ Levels of Knowledge of Diabetes through Expert Judgment. International Journal of Environmental Research and Public Health, 17(22), 8605. https://doi.org/10.3390/ijerph17228605

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop