Next Article in Journal
The Effects of a Low Calorie Ketogenic Diet on Glycaemic Control Variables in Hyperinsulinemic Overweight/Obese Females
Next Article in Special Issue
Effect of the Flipped Classroom and Gamification Methods in the Development of a Didactic Unit on Healthy Habits and Diet in Primary Education
Previous Article in Journal
Vitamin K2 Needs an RDI Separate from Vitamin K1
Previous Article in Special Issue
Rapid Dissemination of College Food Insecurity Findings in A Multi-Institutional Study Using the eB4CAST Approach
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Development of the Home Cooking EnviRonment and Equipment Inventory Observation form (Home-CookERITM): An Assessment of Content Validity, Face Validity, and Inter-Rater Agreement

1
Department Nutrition and Dietetics, Faculty of Health Professions, Bern University of Applied Sciences, Murtenstrasse 10, 3012 Bern, Switzerland
2
Priority Research Centre for Physical Activity & Nutrition (PRCPAN), The University of Newcastle, 1 University Drive, Callaghan, NSW 2308, Australia
3
School of Health Sciences, Faculty of Health and Medicine, The University of Newcastle, 1 University Drive, Callaghan, NSW 2308, Australia
4
School of Environmental and Life Sciences, Faculty of Science, The University of Newcastle, 10 Chittaway Road, Ourimbah, NSW 2258, Australia
5
School of Medicine and Public Health, Faculty of Health and Medicine, The University of Newcastle, 1 University Drive, Callaghan, NSW 2308, Australia
6
Hunter Medical Research Institute (HMRI), New Lambton Heights, NSW 2305, Australia
*
Author to whom correspondence should be addressed.
Nutrients 2020, 12(6), 1853; https://doi.org/10.3390/nu12061853
Submission received: 30 April 2020 / Revised: 9 June 2020 / Accepted: 12 June 2020 / Published: 21 June 2020

Abstract

:
Introduction: Quantifying Home Cooking EnviRonments has applications in nutrition epidemiology, health promotion, and nutrition interventions. This study aimed to develop a tool to quantify household cooking environments and establish its content validity, face validity, and inter-rater agreement. Methods: The Home Cooking EnviRonment and equipment Inventory observation form (Home-CookERI™) was developed as a 24-question (91-item) online survey. Items included domestic spaces and resources for storage, disposal, preparation, and cooking of food or non-alcoholic beverages. Home-CookERITM was piloted to assess content validity, face validity, and usability with six Australian experts (i.e., dietitians, nutrition researchers, chefs, a food technology teacher, and a kitchen designer) and 13 laypersons. Pilot participants provided feedback in a 10 min telephone interview. Home-CookERI™ was modified to an 89-item survey in line with the pilot findings. Inter-rater agreement was examined between two trained raters in 33 unique Australian households. Raters were required to observe each item before recording a response. Home occupants were instructed to only assist with locating items if asked. Raters were blinded to each other’s responses. Inter-rater agreement was calculated by Cohen’s Kappa coefficient (κ) for each item. To optimize κ, similar items were grouped together reducing the number of items to 81. Results: Home-CookERITM had excellent content and face validity with responding participants; all 24 questions were both clear and relevant (X2 (1, n = 19; 19.0, p = 0.392)). Inter-rater agreement for the modified 81-item Home-CookERI™ was almost-perfect to perfect for 46% of kitchen items (n = 37 items, κ = 0.81–1), moderate to substantial for 28% (n = 23, κ = 0.51–0.8), slight to fair for 15% (n = 12, κ = 0.01–0.5), and chance or worse for 11% of items (n = 9, κ ≤ 0.0). Home-CookERITM was further optimized by reduction to a 77-item version, which is now available to researchers. Conclusion: Home-CookERI™ is a comprehensive tool for quantifying Australian household cooking environments. It has excellent face and content validity and moderate to perfect inter-rater agreement for almost three-quarters of included kitchen items. To expand Home-CookERI™ applications, a home occupant self-completion version is planned for validation.

1. Introduction

Food skills, cooking skills, and home cooking frequency are associated with higher diet quality [1,2,3,4,5,6] and improved health outcomes, including lower risk for obesity [5,7,8] and type 2 diabetes [8]. However, since the 1960s, involvement in food preparation and time spent cooking at home has decreased in many western countries [9,10,11]. Scientific understanding of what “home food preparation” or “home cooking” behaviors encompass, and how to consistently measure these behaviors is still developing [4,9,10,11,12,13,14,15]. Informed by the social ecological model [16], home cooking behavior can be influenced by a myriad of individual, interpersonal, and environmental factors which include but are not limited to food and cooking skills, values and enjoyment of cooking, education and nutrition knowledge, culture and ethnicity, and resources (i.e., time, money, cooking spaces, and equipment) [4]. To better understand the association between cooking, dietary intake, and health, robust scientific approaches and measurement tools specific to home cooking are required. High quality validated measures to assess food and cooking skills are available, [17] but in order to gain a more comprehensive understanding of home cooking behavior, these existing tools must be further complemented by specific instruments for assessing more facets of the socioecological influences on home cooking behavior.
One aspect that is beginning to be explored is the Home Cooking EnviRonment, which is defined as the spaces and equipment available for domestic food preparation and cooking, and its impact on diet quality. Exploration of the international literature has identified six questionnaires from seven papers that measure the Home Cooking EnviRonment in the United States of America (US) and France [6,18,19,20,21,22,23]. All tools vary markedly in length, containing between 4 and 44 different kitchen items, with most lacking detail of scientific rigor during development and varying in scientific quality. None of the tools identified could be considered a gold standard for measuring the Home Cooking EnviRonment and equipment.
One of the US tools assessed housing quality and requirements in low-income Latino farmworker families [18]. The tool was created from the housing quality standards promulgated by the US Department of Housing and Urban Development for the Housing Choice Voucher Program without any validation procedure reported.
Another US study used a different tool to investigate the availability of 19 different food preparation items, represented with pictures and text, to tailor nutrition education programs for low income clients applying for a Food Stamp Program [19]. The authors identified methodological shortcomings of the tool such as the absence of any validation procedures, the high likelihood of missing some basic kitchen equipment and not assessing functionality of electric appliances. They also highlighted a need to confirm construct validity of the tool.
Two other US studies [6,23] used a 41-item Food Preparation Checklist, developed specifically for those studies in consultation with a registered dietician and without scientific validation, to examine the association between household food preparation supplies and dietary behaviors in 6–13-year olds.
Another US intervention study in low income families created a 35-item inventory of basic cooking utensils (e.g., measuring cups), tableware and eating utensils (e.g., plates, silverware), cookware (e.g., pan, skillet), appliances (e.g., working oven), and other resources (e.g., table and chairs) to predict children’s adiposity indices [21]. Details regarding the development and validity testing of the inventory were not reported.
Two other tools—the comprehensive home environment survey (CHES) [20] from the US and a set of questions about home cooking resources in a supplemental component of the NutriNet-Santé Study [22] from France—had a more rigorous development process but had other limitations. The 181-item CHES had a structured development procedure, including face, content, predictive, and concurrent validity, as well as inter-rater agreement between two caregivers and test–retest agreement, but the comprehensiveness of the kitchen-subscale was limited to only seven items [20]. The tool used in the cross-sectional NutriNet-Santé Study was a web-based, self-reported questionnaire that was assessed for face and content validity. The food preparation behaviors section was also limited to seven utensils and appliances (pressure cooker, zester, baking pan, measuring cup, food processor, gas oven, or electric oven). The item list was derived from French statistics on income and living conditions. Among volunteers from the general adult population (n = 60,000) who completed the questionnaire, food preparation behaviors were not significantly different across socioeconomic groups. However, a larger range of kitchen equipment was observed in households with high income [22]. While both tools [20,22] were assessed for face and content validity, neither have been assessed for concurrent validity against a gold standard and both are relatively short kitchen sub-scales which limits their capacity to comprehensively measure Home Cooking EnviRonments and equipment [20,22].
In order to robustly test hypotheses regarding the impact of the Home Cooking EnviRonment and equipment on diet quality and/or health in future studies, a ‘gold standard’ survey tool to measure Home Cooking EnviRonments and equipment is needed. It is proposed that such a tool can be developed by applying a standard scientific framework [24,25] to establish face and content validation, and inter-rater agreement of the survey tool where the raters are content experts and independently perform direct observations of the same Home Cooking EnviRonments.
This study aimed to develop a gold standard tool, the Home Cooking EnviRonment and equipment Inventory observation form (Home-CookERI™) to identify the presence of spaces, surfaces, and equipment (e.g., heavy and small kitchen appliances, kitchenware, cleaning facilities, access to recipes) in Australian households that are used for the storage, preparation, or cooking of food or non-alcoholic beverages in the kitchen and elsewhere in a residence.

2. Materials and Methods

2.1. Study Design

A four-stage-mixed-method approach was used to develop Home-CookERITM using qualitative and quantitative measures with differing population samples, as summarized in Table 1, to establish the gold standard measure. First, our research team developed the initial draft of the online Home-CookERITM tool (Stage 1). Next qualitative procedures were used to establish face and content validation of Home-CookERITM with experts and laypersons to ensure its clarity, relevance, comprehensibility, and user-friendliness [25,26,27] (Stage 2). Subsequently, visits were made to households to assess inter-rater agreement of the Home-CookERITM tool when completed by trained members of the research team [25,27,28,29,30] (Stage 3). Ultimately, the final version of Home-CookERITM was produced including all adaptations from the previous stages (Stage 4).
Participants were recruited via networks of the research team using e-mail invitations and from previous research team contacts who had agreed to be re-contacted for future research participation.
The study was approved by the Human Research Ethics Committee of The University of Newcastle (UON), Australia, Approval No. H-2019-0189. Participants in Stages 2 and 3 provided informed consent to participate via an online form in Qualtrics®.
Each development stage is described in more detail in Table 1 and Table 2 and in Appendix B.

2.1.1. Stage 1: Home-CookERITM Initial Design

The initial version of Home-CookERITM was developed iteratively by the authors, i.e., eight of whom have nutrition and cooking expertise with one or more of the following qualifications/roles: dietitian (n = 5), chef (n = 1), nutrition researcher (n = 7), and home economist (n = 1). Statistical considerations for the development of Home-CookERITM was guided by two authors (i.e., C.O., M.F.).
The initial survey included 24 questions covering 91 items likely to be located in contemporary Australian Home Cooking EnviRonments (see Table 2). Home-CookERI™ was developed as an online survey using Qualtrics® allowing for completion on smart phones, tablets, or computers. Most questions grouped similar items (e.g., manual tools for food preparation tasks: slotted spoon, ladle, whisk, spatula) and each question was displayed on a single page to support logical completion. Home-CookERITM was divided into eight sections, as described in Table 2. The naming and definition of each section in the survey was determined within the research team. All questions were multiple choice items with an option to declare that the participant did not have the listed items. Questions were in text format, with pictures only used to assist in distinguishing different types of knives. Fridges could be distinguished by sizes, chopping boards by quantity and questions about pots, pans, frypans, and mixing bowls provided options to distinguish both quantity and size. The initial selection of items was based on team members’ observations of Australian kitchenware websites, department stores, cookbooks, personal recall of items previously observed in Australian Home Cooking EnviRonments, contents of The University of Newcastle teaching food laboratories, and reviewing existing kitchen checklists used by international researchers [6,18,19,20,21,22]. Inclusion criteria for kitchen items were to adequately represent the construct of interest, i.e., spaces, surfaces, and equipment in Australian households used for the storage, preparation, or cooking of food or non-alcoholic beverages in the kitchen and elsewhere in a residence. The authors conducted a qualitative pretest, where each survey item was discussed step-by-step to identify divergence based on deep knowledge of the research aims [31]. The initial 91 included items were thought to be an optimum number to allow collection of comprehensive data about the Home Cooking EnviRonment and equipment while minimizing response burden [32].

2.1.2. Stage 2: Face and Content Validity and Usability Testing

Face and content validity were assessed from August-September 2019. Seven authors and six nutrition and cooking experts from a convenience sample (i.e., dietitians, nutritionists, researchers in nutrition-related fields, chefs/cooks, home economists/food technology teachers, cooking program facilitators, and experts on kitchen architecture/cooking equipment design) qualitatively assessed the face and content validity of Home-CookERITM via a Qualtrics® survey. Other inclusion criteria for participants were as follows: age 18 years or older, access to the internet, and provision of informed consent to participate. Nutrition and cooking experts reviewed all 24 questions and 8 content definitions of the sections (Table 2) in Home-CookERITM, rating each question/section definition for clarity (conciseness, use of correct grammar and presentation in an appropriate format) and relevance to domestic cooking-environments. Further space was provided for the expert participants to provide additional comments on each question or to suggest specific revisions [33]. Further information on the survey review is provided in Appendix A and Appendix B. Experts were invited to participate in a subsequent “respondent debriefing”, i.e., short face-to-face or phone interview [30,32,34], lasting up to 15 min, with a member of the research team (SS). They were asked to ‘think aloud’ and to further discuss their ratings and additional comments about survey clarity, meaningfulness, purposefulness and completeness [25,28,29].
Home-CookERITM was further assessed for face and content validity by a convenience sample of 13 “laypersons” (i.e., not having any professional background in the field of the “nutrition and cooking experts”) from the general adult population. The other inclusion criteria and assessment procedures were the same as those for the expert group, except all participants needed to live in Australia.
Finally, based on the feedback from experts and laypersons, a revised version of Home-CookERITM was created for Stage 3, containing 38 questions and covering 89 items of the Home Cooking EnviRonment (see Table 2).

2.1.3. Stage 3: Inter-Rater Agreement

In September–October 2019 the revised version of Home-CookERITM was used to assess inter-rater agreement of the tool in a home visit study involving volunteer participants. Participants were a convenience sample of adults from 33 different households in Newcastle, Australia. This sample size was chosen prospectively as a compromise between practical limitations and operating characteristics, and would enable estimation of a Kappa statistic with 95% confidence interval having a 25% margin of error, assuming a true Kappa of 0.75 (estimated using the kappa size package in R) [35,36]. Raters who performed the home visits were dietician researchers (S.S., R.A., V.S.) and a student dietician (S.St.) from UON. All raters attended a 2-h training session where they were briefed about home visit safety and the inter-rater agreement research protocol. The UON health and safety policies and procedures were adhered to throughout. In order to test the research protocol, raters performed an assessment with Home-CookERITM in a training kitchen at UON.
Two raters were selected for each visit based on their availability at the time that was nominated by the participant. All combinations of raters were used for the 33 home visits. The two raters independently completed Home-CookERITM in each participant’s home. The first rater completed Home-CookERITM, while the second rater completed an audio activity on an electronic device wearing headphones and, when possible, situated themselves in an adjacent room. This protocol was used so that the second rater was audio and visually blinded to the first rater’s completion of Home-CookERITM. Then the second rater completed Home-CookERITM. Participants (i.e., the home occupant) were instructed to assist with locating items only if asked by the rater to prevent them from biasing a rater’s completion of Home-CookERITM. To allow rater’s responses for a specific participant’s household to be linked for analysis, each household was allocated an identification number. Raters were also provided with an individual identification number to enter on each completion of Home-CookERITM.

2.1.4. Stage 4: Optimization of Home-CookERITM

Home-CookERITM was revised again after the inter-rater agreement evaluation, where kitchen items were specified, generalized, collapsed, or removed according to the findings, to increase the validity and inter-rater agreement of the tool. For more details please refer to Table 2 and Table 3 and Appendix B. The final tool is available for researchers to use (see Appendix C) and can be shared as a Qualtrics® survey upon request and approval of the corresponding author.

2.2. Data Analysis

For Stage 2 descriptive statistics were calculated in Excel for the initial version of Home-CookERITM. A two-tailed t-test was carried out to determine consistency between experts and laypersons on their judgement of clarity and relevance of the 24 questions and 8 content definitions of the sections. In Stage 3, all 89 Home-CookERITM kitchen items produced categorical, binary data which were analyzed for inter-rater agreement using Cohen’s Kappa coefficients. These were calculated using Stata statistical software (StataCorp, 2017, Stata Statistical Software: Release 15. College Station, TX: StataCorp LLC.).
Kappa values (κ) can range from −1.0 (agreement worse than expected, or systematic disagreement) to 1.0 (perfect agreement between raters). Random chance agreement is represented by “0”. Due to diverging recommendations about reasonable Kappa levels [25,28,30,37], a subject-sensitive approach to define levels of inter-rater agreement across multiple data collectors was carried out where “almost-perfect” to “perfect” inter-rater agreement was defined as κ = 0.81 to κ = 1.0, respectively. “Moderate” to “substantial” inter-rater agreement for the identified home cooking items was determined as κ = 0.51 to 0.8, “slight” to “fair” inter-rater agreement as κ = 0.01 to 0.5 and “no” inter-rater agreement when items presented as κ ≤ 0.0.

3. Results

3.1. Results Stage 2: Content and Face Validity

The pilot test found Home-CookERITM had excellent content and face validity with participants responding that all 24 questions were both clear and relevant (X2 (1, n = 19; 19.0, p = 0.392)). No significant differences were found within the respective ratings for clarity and relevance from experts and laypersons. Findings from Stage 2 helped to determine which items needed reviewing or deletion and where ungrouping of some items into single questions and changing the question order were necessary to optimize the tool. More details of changes can be retraced in Table 2 and Appendix B. Other key adjustments included formatting Home-CookERITM for ease of use during home visits in Stage 3 of development, such as displaying all questions on one page to allow raters to easily navigate the tool if items were initially missed and viewed later during an observation. At the end of each section a space was provided for the rater to record notes during the inter-rater agreement procedure (Stage 3). Following these changes, the total assessable kitchen items were reduced to 89 at the commencement of Stage 3 (Table 2 and Appendix B).

3.2. Results Stage 3: Inter-Rater Agreement

Table 3 reports the Kappa values and inter-rater agreement across 81 assessable kitchen items at the end of Stage 3, which originally were 89 items at the beginning of Stage 3. This reduction was performed to optimize the tool’s inter-rater agreement. Specifically, seven items were collapsed to improve inter-rater agreement (e.g., different sizes of pots/saucepans, pans, mixing bowls, fridges without freezer) and one item was removed (“other storage spaces in the kitchen”). The latter item was originally included to ascertain if all kitchen storage spaces had been identified. All answering options “other–please specify” and “I have none of these items” are excluded from Table 3, leading to Kappa values for a total of 81 kitchen items. For those 81 assessable kitchen items almost perfect to perfect agreement between raters was found in 46% of the items (n = 37, raters = 2, κ = 0.81–1). However, for 7 of the 23 observable kitchen items scoring perfect agreement (see Table 3), no variance was found amongst rater responses (100% “yes” or 100% “no”) that yielded an incalculable Kappa value. Inter-rater agreement was moderate to substantial for 28% of items (n = 23, raters = 2, κ = 0.51–0.8) and slight to fair for 15% of items (n = 12, raters = 2, κ = 0.01–0.5). Agreement between raters could not be determined in 11% of the items (n = 9, raters = 2, κ ≤ 0.0) and is explained further in the discussion.

3.3. Stage 4

Based on the findings (Stage 3), 12 items were removed or combined to produce the final Home-CookERITM with 77 items (Appendix C). Due to poor inter-rater agreement for the paring knife (κ = 0.35), it was collapsed into one item with the utility knife. Kitchenware for storage of food and beverages was collapsed with kitchenware for transportation of food and beverages “to go”. Other items where inter-rater agreement could not be determined were removed: “Any storage space indoors and outside kitchen” (κ = −0.10) and “electric pressure cooker” (κ = −0.03). The final Home-CookERITM tool does not include non-assessable answering options such as “Other”.

4. Discussion

Home-CookERITM has high face and content validity (Stage 2), indicating the relevance of its contents based on lay and expert participants’ subjective judgements. Stage 2 participants reported Home-CookERITM questions as being clear and meaningful for the purpose of measuring Home Cooking EnviRonments. Inter-rater agreement for the majority of the assessable Home-CookERITM items was moderate to perfect.
Findings confirm Home-CookERITM is a suitable tool to identify the presence of spaces, surfaces, and equipment in households for food and non-alcoholic beverage storage, preparation or cooking. Cohen’s Kappa levels of κ ≥ 0.81 indicate good agreement in relevant scientific literature [38,39]. From a topic-specific point of view, in this study a more modest agreement coefficient for acceptable inter-rater agreement levels was set. In 74% of the cases, raters identified the kitchen items with a Kappa of κ ≥ 0.51, indicating “moderate” to “perfect” agreement. As Home-CookERITM assesses the presence of items for the purpose of the preparation of food and non-alcoholic beverages, this result is considered to be very good.
The best inter-rater agreement results were found for items that had high levels of agreement for both “yes (present)” and “no (absent)” and that allowed for higher discrimination as Kappa requires both agreement outcomes to calculate a value [38,39].
“Pseudo-perfect” agreements happened for broadly available items in this study. For example, cold/hot water tap and stovetop were present in all assessed homes and identified as such by all raters (100 % inter-rater agreement; κ = 1.0). However, the presence of these items was a reflection of the few assessed households rather than “true” perfect inter-rater agreement for both “yes” and “no” responses. By increasing the sample size to include more socio-economically diverse households, a better distribution of both “yes” and “no” responses could have been reached, which would have produced a balanced Kappa equation. Other “false perfect” agreements were found for example for the “electric food processor/blender of any kind” (κ = 1), where the description of the item was not specific enough to produce variance, i.e., it only produced “yes” answers. This machine was interpreted as anything from a stick blender, a hand-mixer or benchtop food processor and is similar to the “mixer question”. It was kept in the final tool however to account for the presence of any electric appliance which would fulfill the function of blending/mixing.
Limited response variability impacted the results even further, where a single discrepant “yes” and “no” amongst otherwise perfect agreement in all other answers occurred. These “rare cases” produced a heavily weighed Kappa denominator which skewed the value negatively and resulted in κ ≤ 0, despite the raw inter-rater agreement ratio being 99.8%. This coincidence is a well-known limitation of Cohen’s Kappa [39].
For Kappa values κ < 0.5, it was assumed that rater agreements were more likely to occur by chance, such as for the differentiation of fridges and freezers. When analyzing potential root causes for uncertainties, potential sources of confusion were identified, for example the differentiation between the definition and the functionality of kitchen items. This also happened for the slotted spoon (κ = 0.45). It was discovered that one rater confirmed its presence when an object looked similar to the way it would be illustrated in department store websites or in cookbooks, whereas another rater confirmed its presence if any type of utensil was found in the kitchen which could fulfill similar functions as the slotted spoon.
Discriminant sizing of similar items led to poorer inter-rater agreement across specific Home-CookERITM items (for example small and large saucepans). To create a more harmonious research tool it was agreed to condense such items, see Appendix B.
In contrast to the above-mentioned skewed Kappa values due to rare cases, “true” negative Kappa values are an important issue for inter-rater agreement, as this usually indicates a systematic disagreement among raters, and a significant design flaw. In line with literature recommendations, the interpretability of the respective items in Home-CookERITM and the understanding of the raters were analyzed [38,39]. Kappa values below zero were found for five “unclear items” in the list of 81 assessable items in Home-CookERITM at Stage 3. They were either not observable and therefore reliant on the answer of the participants, e.g., “any space…” (κ = −0.10) where food is stored or prepared or designations were confusing, e.g., “baking tray” (κ = −0.04) versus “cookie sheet” (κ = 0.14). Items with a negative κ were removed, adapted, or collapsed (Appendix B).
These examples demonstrate the complex challenge of the unambiguous identification of items related to Home Cooking EnviRonments, even despite direct observations from experts. Differences in culture, socioeconomic background, and cooking skills are just three of many aspects that influence if and how items are recognized. To address this, the use of pictures for each item was considered, but was decided that this may limit rater’s interpretation of the variation in appearance of a particular item. However, the use of pictures was tested for the “knives-question” in Home-CookERITM and could confirm good inter-rater agreement for three of the five knife types. The utility and paring knives were regularly used interchangeably, likely due to their similarity in appearance and were therefore collapsed for the final Home-CookERITM tool.
Previous studies either did not perform any structured validation procedures [6,18,19,21] or only consisted of a very small number of items [20,22]. One of the most important strengths of Home-CookERITM is that it does not rely on subjective means of assessment as currently available questionnaires do. Home-CookERITM is not only the first scientifically evaluated tool to comprehensively measure items for home cooking but it has novelty character being an objective gold standard measure to quantitatively assess domestic kitchens and which can be used by anyone with a background in nutrition research or people who are trained by this group. Findings confirm Home-CookERITM to be easy to understand and scientifically robust for identifying the presence of spaces, surfaces and equipment in households for food and non-alcoholic beverage storage, preparation, or cooking. Additionally, the final survey consisting of 77 items has low response burden with a predicted duration of 12 min for completion according to the “Expert Review” function from the Qualtrics® software.
Limitations of this study include the use of convenience samples from the general population. Study participants came from the researchers’ networks and that may have influenced face and content validity results for Home-CookERITM. Similarly, the convenience sample of experts chosen for the Home-CookERITM development might have impacted the agreement about the relevance of certain items. On the other hand, as Home-CookERITM is aimed at researchers in the field of nutrition, the constituted sample of dietitians, nutritionists, researchers in nutrition-related fields, chefs/cooks, home economists/food technology teachers, cooking program facilitators, and experts for kitchen architecture/cooking equipment design, can be considered a strength to judge items in the tool as they are the ones who are most likely using it in their professional work. As data saturation about the consensus of clarity and relevance of the items among all participants was achieved, the sample was further confirmed as purposeful and the relevance of their judgement unambiguous [40]. Blinding was used between raters when assessing home environments. This strengthens the validity of findings within the study via elimination of potential observer biases.
The development of the Home-CookERITM tool as a gold standard, through investigation of actual Home Cooking EnviRonments by trained raters, was the first logical and most essential step to establish a robust assessment tool in this field. This study provides a methodology for researchers in other countries to replicate to create a locally relevant gold standard tool.
Future research should investigate validating the tool for self-completion by laypersons as this would allow it to be used with confidence in population-based studies. Such a tool would enable the conduct of research to facilitate a greater understanding of the link between Home Cooking EnviRonments, cooking behaviors (including cooking/food skills and confidence), eating patterns, diet quality and health in people with diverse socioeconomic backgrounds.
A self-completion version of Home-CookERITM could also be used in clinical and public health settings to enable the personalization of recipe provision and cooking skill education to individual’s home cooking resources. Pictures could then be considered to clarify the items in question and to better suit specific groups of end-users. Development of context-specific versions of Home-CookERITM may be beneficial. Whilst for the design of a cooking class it might be reasonable to ask whether people have particular tools available to perform preparation tasks for specific dishes, epidemiological research could aim to build rankings of kitchen equipment quality and potentially include frequency of use. While a poor cooking environment might be expected to limit the ability to prepare healthy diets, some households might harbor tools which are never used. A cooking resource usage score could help distinguish between the presence of the items, versus their frequency of use when adding a weighted analytic component. Looking at correlations between resource usage, cooking skills, and diet quality could be a highly relevant field for future research.

5. Conclusions

Presently Home-CookERITM is the only domestic kitchen inventory tool with high-level face and content validity and inter-rater agreement confirmed via direct observation of Home Cooking EnviRonments. Home-CookERITM can be recommended for use in studies when Australian Home Cooking EnviRonments are observed by researchers or it could be adapted to reflect other eating cultures. Future research is needed to adapt the Home-CookERITM tool for self-completion by the general population.

Author Contributions

Conceptualization, S.S. (Sonja Schönberg), V.A.S., L.A., T.B., and K.V.d.H.; methodology, S.S. (Sonja Schönberg), V.A.S., C.O., K.V.d.H., and M.J.F.; validation, S.S. (Sonja Schönberg), V.A.S., and M.J.F.; formal analysis, S.S. (Sonja Schönberg), V.A.S., and K.V.d.H.; investigation, S.S. (Sonja Schönberg), V.A.S., R.A., and S.S. (Samantha Stewart); resources, K.V.d.H. and C.E.C.; software, S.S. (Sonja Schönberg) and V.A.S.; data curation, S.S. (Sonja Schönberg), M.J.F., and V.A.S.; writing—original draft preparation, S.S. (Sonja Schönberg); writing—review and editing, V.A.S., M.J.F., K.V.d.H., L.A., R.A., C.E.C., C.O., T.B., and S.S. (Samantha Stewart); visualization, S.S. (Sonja Schönberg) and M.J.F.; supervision, V.A.S. and K.V.d.H.; project administration, S.S. (Sonja Schönberg); funding acquisition, V.A.S. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the School of Health Sciences Strategic pilot grant scheme, University of Newcastle.

Acknowledgments

The authors would like to acknowledge the contribution made by Karin Zbinden Gysin and Christine Jurt Vicuña Muñoz from the School of Agricultural, Forest, and Food Sciences (HAFL) in Switzerland with regard to their support on the development of the qualitative survey components. Further acknowledgement is dedicated to Bern University of Applied Sciences in Switzerland to provide time and financial resources for Sonja Schönberg to travel to Australia for the purpose of this study.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analysis, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

Appendix A

Figure A1. Example of Feedback Stage 2: Face and Content Validity.
Figure A1. Example of Feedback Stage 2: Face and Content Validity.
Nutrients 12 01853 g0a1

Appendix B

Figure A2. Changelog Home-CookERITM.
Figure A2. Changelog Home-CookERITM.
Nutrients 12 01853 g0a2

Appendix C. Home-CookERITM–Final Questionnaire

Introduction and Eligibility
Consent has been confirmed within the pre-visit online survey and prior to entering the property of the home occupant. I am allowed to start the home visit.
☐ Yes, I hereby confirm.
Enter your researcher ID:_______________________
Enter the home visit ID:_________________________
Home cooking Assessment
HOME COOKING ENVIRONMENT-SPACES
Defined as spaces and surfaces for food storage, preparation, cleaning, and disposal.
Whenever you are not sure about the right answer, please provide your best guess/choose the most appropriate option provided.
At home, what spaces do you use to prepare food/meals?
Please tick all applicable options.
☐ Indoor Kitchen/Kitchenette
☐ Outdoor Kitchen/BBQ
☐ I do not have any space to prepare food
At home, what spaces do you use to store any resources/ingredients for food preparation?
Please tick all applicable options.
☐ Pantry/Cupboards
☐ Walk-In pantry
☐ Garage
☐ Cellar
☐ Pantry/Cupboards/Other storage spaces NOT in the kitchen
☐ I do not have any storage space
At home, what resources do you use to clean/dispose of waste when preparing food?
Please tick all applicable options.
☐ Sink
☐ Cold water tap
☐ Hot water tap
☐ Rubbish bin/Garbage disposal
☐ Recycling bin/Compost bin
☐ Have none of these resources
HOME COOKING ENVIRONMENT-HEAVY KITCHEN APPLIANCES
(i.e., major appliances, domestic appliances), defined as large machines used for routine house
keeping tasks around cooking, washing and food preservation, that use electricity or fuel.
Whenever you are not sure about the right answer, please provide your best guess/choose the most appropriate option provided.
Which kind of working fridge/freezer do you have access to?
Please tick all applicable options.
☐ Fridge, without freezer section
☐ Fridge, with freezer section
☐ Standalone upright freezer (several compartments)
☐ Chest freezer (large box or chest with a hinged lid)
☐ Portable fridge or freezer
☐ Have none of these items
At home, do you have access to a working stovetop/cooktop that is fixed in place (NOT portable)?
☐ Yes
☐ No
At home, do you have access to a working oven that is fixed in place (NOT portable)?
☐ Yes
☐ No
At home, do you have access to a working rangehood/exhaust fan?
☐ Yes
☐ No
At home, do you have access to a working dishwasher?
☐ Yes
☐ No
Please find some space provided below to comment or suggest any revisions upon home visit.
Also note here if you couldn’t sight any objects for safety reasons but the home occupant confirmed they had it.
__________________________________________________________________________________________________
HOME COOKING ENVIRONMENT-SMALL KITCHEN APPLIANCES FOR FOOD PREPARATION AND COOKING WITH HEAT
(i.e., small domestic appliances, small electrics) are defined as portable or semi-portable machines, used with electricity or other fuel to accomplish different food preparation tasks. They are often placed on benches or other household-platforms, in- or outside the house.
Whenever you are not sure about the right answer, please provide your best guess/most
appropriate option provided.
At home, do you have access to the following working, electric bench top appliances for wet cooking methods?
Please tick all applicable options.
☐ Electric Rice cooker
☐ Electric Steamer
☐ Electric Slow cooker
☐ Electric multi cooker
☐ Electric All-in-one appliance (chops, mixes, cooks), e.g., Thermomix
☐ Have none of these items
At home, do you have access to the following working, electric bench top appliances for dry cooking methods?
Please tick all applicable options.
☐ Deep fryer
☐ Air fryer
☐ Toaster
☐ Electric grill press/Sandwich maker
☐ Electric Frypan/Wok
☐ Have none of these items
At home, do you have access to a working microwave?
☐ Yes
☐ No
At home, do you have access to a working electric benchtop oven?
☐ Yes
☐ No
At home, do you have access to a working BBQ?
☐ Yes
☐ No
At home, do you have access to a working electric kettle?
☐ Yes
☐ No
At home, do you have access to a working electric coffee machine?
☐ Yes
☐ No
Please find some space provided below to comment or suggest any revisions upon home visit.
Also note here if you couldn’t sight any objects for safety reasons but the home occupant confirmed they had it.
__________________________________________________________________________________________________
HOME COOKING ENVIRONMENT-SMALL KITCHEN APPLIANCES FOR FOOD PROCESSING
Defined as portable or semi-portable machines, used with electricity or other fuel to accomplish different food preparation tasks, e.g., mixing, food processing, blending, juicing and water heating. They are often placed on benches or other household-platforms, inside the house.
Whenever you are not sure about the right answer, please provide your best guess/choose the most appropriate option provided.
At home, do you have access to a working electric food processor/blender of any kind?
☐ Yes
☐ No
At home, do you have access to a working electric mixer of any kind?
☐ Yes
☐ No
At home, do you have access to a working electric juicer?
☐ Yes
☐ No
Please find some space provided below to comment or suggest any revisions upon home visit.
Also note here if you couldn’t sight any objects for safety reasons but the home occupant confirmed they had it.
__________________________________________________________________________________________________
HOME COOKING ENVIRONMENT-KITCHENWARE FOR FOOD PREPARATION AND COOKING
Defined as the cookware to prepare food and drinks on a stove/cooktop.
Whenever you are not sure about the right answer, please provide your best guess/choose the most appropriate option provided.
At home do you have access to a pot/pan?
☐ Yes
☐ No
At home, do you have access to a steamer basket/insert for saucepans?
☐ Yes
☐ No
At home, do you have access to a stove top pressure cooker?
☐ Yes
☐ No
At home do you have access to a pan (frypan/saute pan/grill pan/wok)?
☐ Yes
☐ No
At home, do you have a Dutch oven/casserole (thick-walled cooking pot with lid) that can be used both on the stove top and in the oven?
☐ Yes
☐ No
Please find some space provided below to comment or suggest any revisions upon home visit.
Also note here if you couldn’t sight any objects for safety reasons but the home occupant confirmed they had it.
__________________________________________________________________________________________________
HOME COOKING ENVIRONMENT-UTENSILS FOR FOOD PREPARATION AND COOKING
Defined as small tools, used for different tasks in the food preparation process, e.g., cutting, grating, mixing, stirring, turning, frothing, storing and measuring.
Whenever you are not sure about the right answer, please provide your best guess/choose the most appropriate option provided.
At home, do you have access to manual tools for different tasks of food preparation?
Please tick all applicable options.
☐ Utensils to turn hot food, e.g., spoon/turner/spatula/tong
☐ Slotted spoon
☐ Ladle
☐ Potato masher
☐ Whisk
☐ Rolling pin
☐ Have none of these items
At home, do you have access to measuring tools?
Please tick all applicable options.
☐ Measuring spoons
☐ Measuring cups
☐ Measuring jugs
☐ Scales
☐ Have none of these items
At home, do you have access to a mixing bowl?
☐ Yes
☐ No
At home, do you have access to a colander/strainer?
☐ Yes
☐ No
At home, do you have access to a salad spinner?
☐ Yes
☐ No
At home, do you have access to recipes e.g., in a cook book or on internet?
☐ Yes
☐ No
At home, do you have access to a chopping board?
☐ Yes
☐ No
At home, which of the following knives for food preparation do you have access to?
Please tick all applicable options.
Cook’s/Chef’s knife: large, bladed, multi-purpose knife. Performs well at mincing, shredding, slicing and chopping vegetables or meat and disjointing large cuts. Nutrients 12 01853 i001
Serrated bread knife: large, strong, serrated, not flexible knife. Performs well to cut soft bread without crushing it. Nutrients 12 01853 i002
Carving knife: large, thin, bladed or serrated knife. Performs well to slice precise, thin cuts of poultry, roasts, hams, and other large cooked meats. Nutrients 12 01853 i003
Utility or paring knife: small-to-medium-sized, bladed, all-purpose-knife. Performs well at general cutting tasks, at peeling (or “paring”) fruits and vegetables and other small or intricate cutting tasks. Nutrients 12 01853 i004
☐ Don’t have any
At home, do you have access to other manual tools for chopping up food items?
Please tick all applicable options.
☐ Kitchen shears/scissors
☐ Manual peeler
☐ Manual grater
☐ Have none of these items
At home, do you have access to a manual tin/can opener?
☐ Yes
☐ No
Please find some space provided below to comment or suggest any revisions upon home visit.
Also note here if you couldn’t sight any objects for safety reasons but the home occupant confirmed they had it.
__________________________________________________________________________________________________
HOME COOKING ENVIRONMENT-KITCHENWARE FOR BAKING
Defined as the cooking vessels intended for use inside an oven.
Whenever you are not sure about the right answer, please provide your best guess/choose the most appropriate option provided.
At home, do you have access to the following bakeware?
Please tick all applicable options.
☐ Oven/baking tray
☐ Baking sheet, shallow, typically ~ 2cm deep
☐ Cake pan
☐ Bread/loaf pan
☐ Pie tin
☐ Deep dish
☐ Cooling rack
☐ Have none of these items
At home, do you have access to a roasting pan (often rectangular, open dish to use in oven)?
☐ Yes
☐ No
Please find some space provided below to comment or suggest any revisions upon home visit.
Also note here if you couldn’t sight any objects for safety reasons but the home occupant confirmed they had it.
__________________________________________________________________________________________________
HOME COOKING ENVIRONMENT-KITCHENWARE FOR STORAGE
Food and drink storage containers are vessels, that allow for food and drinks to be consumed at some later point in time. They are used to store preservable food and drinks or to take away.
Whenever you are not sure about the right answer, please provide your best guess/choose the most appropriate option provided.
At home, do you have access to sealable food containers/canisters to go or for storage in any dry storage space, fridge or freezer?
☐ Yes
☐ No
At home, do you have access to drinking bottles/mugs to go?
☐ Yes
☐ No
Please find some space provided below to comment or suggest any revisions upon home visit.
Also note here if you couldn’t sight any objects for safety reasons but the home occupant confirmed he/she had it.
__________________________________________________________________________________________________
This is the end of the survey. You can close your browser window now.

References

  1. Hasan, B.; Thompson, W.G.; Almasri, J.; Wang, Z.; Lakis, S.; Prokop, L.J.; Hensrud, D.D.; Frie, K.S.; Wirtz, M.J.; Murad, A.L.; et al. The effect of culinary interventions (cooking classes) on dietary intake and behavioral change: A systematic review and evidence map. BMC Nutr. 2019, 5, 1462. [Google Scholar] [CrossRef] [PubMed]
  2. McGowan, L.; Pot, G.K.; Stephen, A.M.; Lavelle, F.; Spence, M.; Raats, M.; Hollywood, L.; McDowell, D.; McCloat, A.; Mooney, E.; et al. The influence of socio-demographic, psychological and knowledge-related variables alongside perceived cooking and food skills abilities in the prediction of diet quality in adults: A nationally representative cross-sectional study. Int. J. Behav. Nutr. Phys. Act. 2016, 13, 111. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Tiwari, A.; Aggarwal, A.; Tang, W.; Drewnowski, A. Cooking at Home: A Strategy to Comply With U.S. Dietary Guidelines at No Extra Cost. Am. J. Prev. Med. 2017, 52, 616–624. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. Mills, S.; White, M.; Brown, H.; Wrieden, W.; Kwasnicka, D.; Halligan, J.; Robalino, S.; Adams, J. Health and social determinants and outcomes of home cooking: A systematic review of observational studies. Appetite 2017, 111, 116–134. [Google Scholar] [CrossRef] [PubMed]
  5. Wolfson, J.A.; Bleich, S.N. Is cooking at home associated with better diet quality or weight-loss intention? Public Health Nutr. 2015, 18, 1397–1406. [Google Scholar] [CrossRef] [Green Version]
  6. Appelhans, B.M.; Waring, M.E.; Schneider, K.L.; Pagoto, S.L. Food preparation supplies predict children’s family meal and home-prepared dinner consumption in low-income households. Appetite 2014, 76, 1–8. [Google Scholar] [CrossRef]
  7. Mills, S.; Brown, H.; Wrieden, W.; White, M.; Adams, J. Frequency of eating home cooked meals and potential benefits for diet and health: Cross-sectional analysis of a population-based cohort study. Int. J. Behav. Nutr. Phys. Act. 2017, 14, 109. [Google Scholar] [CrossRef]
  8. Zong, G.; Eisenberg, D.M.; Hu, F.B.; Sun, Q. Consumption of Meals Prepared at Home and Risk of Type 2 Diabetes: An Analysis of Two Prospective Cohort Studies. PLoS Med. 2016, 13, e1002052. [Google Scholar] [CrossRef] [Green Version]
  9. Möser, A. Food preparation patterns in German family households. An econometric approach with time budget data. Appetite 2010, 55, 99–107. [Google Scholar] [CrossRef]
  10. Smith, L.P.; Ng, S.W.; Popkin, B.M. Trends in US home food preparation andconsumption: Analysis of national nutritionsurveys and time use studies from 1965–1966 to 2007–2008. Nutr. J. 2013, 12, 45. [Google Scholar] [CrossRef] [Green Version]
  11. Zick, C.D.; Stevens, R.B. Trends in Americans’ food-related time use: 1975–2006. Public Health Nutr. 2010, 13, 1064–1072. [Google Scholar] [CrossRef] [Green Version]
  12. Monsivais, P.; Aggarwal, A.; Drewnowski, A. Time spent on home food preparation and indicators of healthy eating. Am. J. Prev. Med. 2014, 47, 796–802. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Kolodinsky, J.M.; Goldstein, A.B. Time use and food pattern influences on obesity. Obesity 2011, 19, 2327–2335. [Google Scholar] [CrossRef] [PubMed]
  14. Engler-Stringer, R. Food, cooking skills, and health: A literature review. Can. J. Diet. Pract. Res. 2010, 71, 141–145. [Google Scholar] [CrossRef] [PubMed]
  15. Mills, S.D.H.; Wolfson, J.A.; Wrieden, W.L.; Brown, H.; White, M.; Adams, J. Perceptions of ‘Home Cooking’: A Qualitative Analysis from the United Kingdom and United States. Nutrients 2020, 12, 198. [Google Scholar] [CrossRef] [Green Version]
  16. McLeroy, K.R.; Bibeau, D.; Steckler, A.; Glanz, K. An Ecological Perspective on Health Promotion Programs. Health Educ. Q. 1988, 15, 351–377. [Google Scholar] [CrossRef]
  17. Lavelle, F.; McGowan, L.; Hollywood, L.; Surgenor, D.; McCloat, A.; Mooney, E.; Caraher, M.; Raats, M.; Dean, M. The development and validation of measures to assess cooking skills and food skills. Int. J. Behav. Nutr. Phys. Act. 2017, 14. [Google Scholar] [CrossRef] [Green Version]
  18. Gentry, A.L.; Grzywacz, J.G.; Quandt, S.A.; Davis, S.W.; Arcury, T.A. Housing Quality Among North Carolina Farmworker Families. J. Agric. Safe 2007, 13, 323–337. [Google Scholar] [CrossRef]
  19. Landers, P.; Schults, C. Pots, Pans, and Kitchen Equipment: Do Low-Income Clients Have Adequate Tools for Cooking? J. Ext. 2008, 46, 1. [Google Scholar]
  20. Pinard, C.A.; Yaroch, A.L.; Hart, M.H.; Serrano, E.L.; McFerren, M.M.; Estabrooks, P.A. The Validity and reliability of the Comprehensive Home Environment Survey (CHES). Health Promot. Pract. 2014, 15, 109–117. [Google Scholar] [CrossRef]
  21. Brophy-Herb, H.E.; Horodynski, M.; Contreras, D.; Kerver, J.; Kaciroti, N.; Stein, M.; Lee, H.J.; Motz, B.; Hebert, S.; Prine, E.; et al. Effectiveness of differing levels of support for family meals on obesity prevention among head start preschoolers: The simply dinner study. BMC Public Health 2017, 17, 184. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  22. Méjean, C.; Si Hassen, W.; Gojard, S.; Ducrot, P.; Lampuré, A.; Brug, H.; Lien, N.; Nicolaou, M.; Holdsworth, M.; Terragni, L.; et al. Social disparities in food preparation behaviours: A DEDIPAC study. Nutr. J. 2017, 16, 62. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  23. Oakley, A.R.; Nikolaus, C.J.; Ellison, B.; Nickols-Richardson, S.M. Food insecurity and food preparation equipment in US households: Exploratory results from a cross-sectional questionnaire. J. Hum. Nutr. Diet. 2019, 32, 143–151. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  24. Anthoine, E.; Moret, L.; Regnault, A.; Sébille, V.; Hardouin, J.-B. Sample size used to validate a scale: A review of publications on newly-developed patient reported outcomes measures. Health Qual. Life Outcomes 2014, 12, 176. [Google Scholar] [CrossRef] [Green Version]
  25. Fayers, P.; Machin, D. Quality of Life: The Assessment, Analysis and Reporting of Patient-Reported Outcomes; John Wiley & Sons, Incorporated: Hoboken, UK, 2016; ISBN 9781118759011. [Google Scholar]
  26. Stone, A.; Bachrach, C.; Jobe, J.; Kurtznabb, H.; Cain, V. The Science of Self-Report: Implications for Research and Practice; Lawrence Erlbaum Associates: Mahwah, NJ, USA, 1999; ISBN 0805829911. [Google Scholar]
  27. Groves, R.M.; Fowler, F.J.; Couper, M.P.; Lepkowski, J.M.; Singer, E.; Tourangeau, R. Survey Methodology, 2nd ed.; Wiley: Hoboken, NJ, USA, 2009; ISBN 978-0-470-46546-2. [Google Scholar]
  28. Bajpai, R.; Bajpai, S. Goodness of Measurement: Reliability and Validity. Int. J. Med. Sci. Public Health 2014, 3, 112. [Google Scholar] [CrossRef] [Green Version]
  29. Rust, J.; Golombok, S. Modern Psychometrics. The Science of Psychological Assessment, 3rd ed.; Routledge: London, UK, 2009; ISBN 978-0-415-44216-9. [Google Scholar]
  30. Scholtes, V.A.; Terwee, C.B.; Poolman, R.W. What makes a measurement instrument valid and reliable? Injury 2011, 42, 236–240. [Google Scholar] [CrossRef]
  31. Döring, N.; Bortz, J. Forschungsmethoden und Evaluation (Research methods and evaluation). In Den Sozial- und Humanwissenschaften (Social and Human Sciences), 5th ed.; Springer: Berlin/Heidelberg, Germany, 2016; ISBN 978-3-642-41088-8. [Google Scholar]
  32. Tsang, S.; Royse, C.F.; Terkawi, A.S. Guidelines for developing, translating, and validating a questionnaire in perioperative and pain medicine. Saudi J. Anaesth. 2017, 11, S80–S89. [Google Scholar] [CrossRef]
  33. McGartland Rubio, D.; Berg-Weger, M.; Tebb, S.; Lee, E.; Rauch, S. Objectifying content validity: Conducting a content validity study in social work research. Soc. Work Res. 2003, 27, 94–104. [Google Scholar] [CrossRef]
  34. Goodwin, L.D.; Leech, N.L. The Meaning of Validity in the New Standards for Educational and Psychological Testing: Implications for Measurement Courses. Meas. Eval. Couns. Dev. 2003, 36, 181–191. [Google Scholar] [CrossRef]
  35. Rotondi, M.A. Sample Size Estimation Functions for Studies of Interobserver Agreement: Package ‘kappaSize’ 2018. Available online: https://rdrr.io/cran/kappaSize/ (accessed on 21 March 2020).
  36. R Core Team. R: A Language and Environment for Statistical Computing; R Foundation for Statistical Computing: Vienna, Austria, 2019; Available online: https://www.eea.europa.eu/data-and-maps/indicators/oxygen-consuming-substances-in-rivers/r-development-core-team-2006 (accessed on 21 March 2020).
  37. Landis, J.R.; Koch, G.G. The Measurement of Observer Agreement for Categorical Data. Biometrics 1977, 33, 159. [Google Scholar] [CrossRef] [Green Version]
  38. McHugh, M.L. Interrater agreement: The kappa statistic. Biochem. Med. 2012, 22, 276–282. [Google Scholar] [CrossRef]
  39. Viera, A.J.; Garrett, J.M. Understanding Interobserver Agreement: The Kappa Statistic. Fam. Med. 2005, 37, 360–363. [Google Scholar] [PubMed]
  40. Braun, V.; Clarke, V. Successful Qualitative Research. A Practical Guide for Beginners, 1st ed.; SAGE: Los Angeles, CA, USA; London, UK; New Delhi, India, 2013; ISBN 9781847875822. [Google Scholar]
Table 1. Home Cooking EnviRonment and equipment Inventory observation form (Home-CookERI™) four-stage development process.
Table 1. Home Cooking EnviRonment and equipment Inventory observation form (Home-CookERI™) four-stage development process.
StageName of StageN
1Home-CookERITM initial designAll authors
2Face and content validity, usability testing19 1
3Inter-rater agreement33
4Optimization of Home-CookERITMn.a.
1 Until data saturation achieved. Included 6 experts and 13 laypersons in addition to the authors.
Table 2. Home-CookERITM Structure, Stages 2–4.
Table 2. Home-CookERITM Structure, Stages 2–4.
Sections and Content DefinitionsNumber of Questions Stage 2Number of Kitchen Items Stage 2Number of Questions Stage 3Number of Kitchen Items Stage 3Number of Questions Stage 4Number of Kitchen Items Stage 4
Spaces/surfaces: food storage, preparation, cleaning, disposal313314312
Heavy kitchen appliances (large machines for routine housekeeping tasks around cooking, washing, food preservation, used with electricity or fuel) 51151059
Small kitchen appliances: food preparation/cooking with heat (portable/semi-portable, often placed on benches/other household-platforms, in or outside the house, used with electricity or fuel)317514715
Small kitchen appliances: mixing, processing, blending, juicing (portable/semi-portable, often placed on benches/other household-platforms, in or outside the house, used with electricity or fuel)135533
Kitchenware: food preparation/cooking (cookware to prepare food and drinks on a stove/cooktop)31061055
Utensils: food preparation/cooking (small tools, used for different tasks in the food preparation process, e.g., cutting, grating, mixing, stirring, turning, frothing, storing and measuring)72610261023
Kitchenware: baking (cooking vessels intended for use inside an oven)181728
Food/drink storage containers133322
Total249138893777
Table 3. Home-CookERITM Kappa values for inter-rater agreement at the end of Stage 3.
Table 3. Home-CookERITM Kappa values for inter-rater agreement at the end of Stage 3.
Items (n = 81)Kappa Value
κ
Inter-Rater Agreement Ratio (n%)
Kitchenette a1.00100.00
Sink a1.00100.00
Cold water tap a1.00100.00
Hot water tap a1.00100.00
Chest freezer (large box or chest with a hinged lid) a1.00100.00
Fixed Stovetop/Cooktop a1.00100.00
Oven that is fixed in place1.00100.00
Air fryer1.00100.00
Electric grill press/Sandwich maker1.00100.00
Electric food processor/Blender of any kind a1.00100.00
Electric kettle1.00100.00
Saucepans/Pots of different sizes: small, medium, large b1.00100.00
Pans (Frypan/Sauté pan/Grill pan/Wok): small, medium, large b1.00100.00
Whisk1.00100.00
Rolling pin1.00100.00
Mixing bowls of different sizes: small, medium, large b1.00100.00
Measuring jugs1.00100.00
Colander/Strainer1.00100.00
Cook’s/Chef’s knife1.00100.00
Kitchen shears/Scissors1.00100.00
Manual peeler1.00100.00
Bread/Loaf pan1.00100.00
Food canisters for storage1.00100.00
Electric coffee machine0.9396.97
Electric Frypan/Wok0.9396.97
Electric All-in-one appliance (chops, mixes, cooks)0.9296.97
Rangehood/Exhaust fan0.9196.97
Dishwasher0.9196.97
Chopping boards0.9196.97
Measuring cups0.8996.97
Cake pan0.8996.97
Steamer basket/Insert for saucepans0.8796.97
Salad spinner0.8793.94
Cooling rack0.8693.94
Fridge, without freezer (any size) b0.8496.97
Microwave0.8496.97
Electric slow cooker0.8290.91
Measuring spoons0.7990.91
BBQ0.7990.91
Electric multi cooker0.7693.94
Ladle0.7693.94
Electric rice cooker0.7490.91
Recycling bin/Compost bin0.7293.94
Garage0.7293.94
Potato masher0.7193.94
Outdoor kitchen/BBQ area0.6984.85
Deep fryer0.6596.97
Toaster0.6596.97
Manual grater0.6596.97
Drinking bottles/Mugs to go0.6596.97
Manual tin/can opener0.6493.94
Fridge, with freezer (any size)0.6493.90
Scales0.6384.85
Dutch oven/Casserole0.6281.82
Electric mixer of any kind0.6184.85
Electric juicer0.6087.88
Carving knife0.5578.79
Stove top pressure cooker0.5290.91
Serrated bread knife0.5290.91
Pantry/Cupboards/Other storage spaces NOT in the kitchen0.5278.79
Portable fridge or freezer0.4893.94
Standalone upright freezer (several compartments)0.4681.82
Slotted spoon0.4684.85
Pie tin0.3869.70
Roasting pan0.3790.91
Utility knife0.3790.91
Paring knife0.3590.91
Walk in pantry0.2284.85
Electric steamer0.2084.85
Benchtop oven0.2084.85
Deep dish0.1975.76
Cookie sheet0.1457.58
Cellar0.0096.97
Rubbish bin/Garbage disposal0.0093.94
Utensils to turn hot food, e.g., Spoon/Turner/Spatula/Tong0.0096.97
Cookbook/Internet0.0096.97
Electric pressure cooker−0.0393.94
Food canister to go−0.0393.94
Pantry/Cupboard in kitchen−0.0490.91
Oven/Baking tray−0.0490.91
Any storage space indoor, but outside kitchen−0.1075.76
a “False perfect” inter-rater agreement due to missing variance, i.e.,100% “yes” or 100% “no”. b Collapsed to compute inter-rater agreement statistics in Stage 3.

Share and Cite

MDPI and ACS Style

Schönberg, S.; Asher, R.; Stewart, S.; Fenwick, M.J.; Ashton, L.; Bucher, T.; Van der Horst, K.; Oldmeadow, C.; Collins, C.E.; Shrewsbury, V.A. Development of the Home Cooking EnviRonment and Equipment Inventory Observation form (Home-CookERITM): An Assessment of Content Validity, Face Validity, and Inter-Rater Agreement. Nutrients 2020, 12, 1853. https://doi.org/10.3390/nu12061853

AMA Style

Schönberg S, Asher R, Stewart S, Fenwick MJ, Ashton L, Bucher T, Van der Horst K, Oldmeadow C, Collins CE, Shrewsbury VA. Development of the Home Cooking EnviRonment and Equipment Inventory Observation form (Home-CookERITM): An Assessment of Content Validity, Face Validity, and Inter-Rater Agreement. Nutrients. 2020; 12(6):1853. https://doi.org/10.3390/nu12061853

Chicago/Turabian Style

Schönberg, Sonja, Roberta Asher, Samantha Stewart, Matthew J. Fenwick, Lee Ashton, Tamara Bucher, Klazine Van der Horst, Christopher Oldmeadow, Clare E. Collins, and Vanessa A. Shrewsbury. 2020. "Development of the Home Cooking EnviRonment and Equipment Inventory Observation form (Home-CookERITM): An Assessment of Content Validity, Face Validity, and Inter-Rater Agreement" Nutrients 12, no. 6: 1853. https://doi.org/10.3390/nu12061853

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop