A Comparison of the Percentage of “Yes” (Agree) Responses and Importance of Attributes (Constructs) Determined Using Check-All-That-Apply and Check-All-Statements (Yes/No) Question Formats in Five Countries

Check All That Apply (CATA) has become a popular type of questionnaire response in sensory/consumer research in recent years. However, some authors have pointed out potential problems with the method. An online survey using either a Check-All-That-Apply (CATA) or Check-All-Statements (CAS) format for questions was conducted to provide a deeper understanding of the response data using the two question formats. With CATA, respondents select all terms or statements that apply from a given list, while, with CAS, respondents must respond (e.g., yes/no or agree/disagree) to each term or statement to show that it applies or does not apply. Respondents from five countries (Brazil, China, India, Spain, and the USA) were randomly assigned one of the two question formats (N = 200 per country per method). Motivations for eating items that belong to five food groups (starchy, protein, dairy, fruits, and desserts) were assessed. Results showed that CAS had higher percentages of “agree” responses than CATA. Also, the response ratio of CAS and CATA data was different, suggesting that interpretations of the data from each response type would also be different. Respondents in the USA, China, and Spain took longer to complete the CAS questionnaire, while respondents in Brazil and India had similar time durations for the two question formats. Overall, the CATA format was liked slightly more than the CAS format and fewer respondents dropped out of the survey when using the CATA response type. These findings suggest that the CATA format is quick and relatively easy for consumers to complete. However, it provokes fewer “apply” responses, which some psychologists suggest underestimates applicable terms or statements and CATA provides a different interpretation of data than the CAS format that requires consumers to respond to each term or statement. Further, CAS may overestimate the applicable terms. Consumer insights collected using CATA and CAS can lead to different decisions due to differences in data interpretation by researchers (e.g., marketers, nutritionists, product developers, and sensory scientists). More investigation is needed for the CATA and CAS question formats.


Introduction
The use of the Check-All-That-Apply (CATA) or the Mark-All-That-Apply question format as reported by Sudman and Bradburn [1] has become popular in consumer research [2][3][4][5][6][7][8][9][10][11][12][13]. This question format asks respondents to check all items that apply from a list of options. For the Check-All-Statements (CAS) or the forced-choice question format, also known as the yes/no format or sometimes called "applicability scoring", a respondent is asked to check a "yes" or "no" option (or something similar such as agree or disagree) for each item [10,11,14]. Both the CATA and CAS question formats have been used extensively for questionnaires that are designed to be completed by respondents with little or no intervention. Although the research literature tends to be fairly recent, multiple industries have been using this procedure at least since the early 1980s [1,15].
CATA was developed to reduce the fatigue of respondents while they completed a self-administered questionnaire. This tool provides an easy and non-tedious way of collecting multiple responses that are reproducible [3,8,[16][17][18]. However, the CATA format has been criticized for ambiguity in interpreting the absence of a checkmark on listed options. The unchecked items can be interpreted as those that were not applicable or those for which the respondent was uncertain about the applicability. It is also possible that the respondent did not notice the item(s) as they hurriedly read the list of items [5,10] or that they only paid attention to the first items or a limited set of items to save time. Survey research theory [19] suggests that for self-administered surveys (e.g., online surveys), respondents may select the first acceptable response(s) and not pay attention to later responses because it takes too much effort. This may particularly be true based on the cognitive elaboration model [20] when the respondent is in visual mode (e.g., reading responses) because the respondent takes more time to consider the first options. Ares and Jaeger [21] showed that the order in which items in the CATA question were presented had an impact on the results. For instance, items that appeared in the top left corner of the ballot were checked more frequently as compared to items that were placed at the bottom of the ballot and this consequently affected the total number of responses provided. This suggests that the items seen early in the questionnaire are more likely to be rated as "apply" than those later in the questionnaire. Simply randomizing the terms can reduce the impact of order bias on specific terms, but does not eliminate the problem and exacerbates the impact that order bias has on differences in scoring frequency when attempting to compare consumers or cluster them based on their responses. In a recent study with children, different response patterns were found, suggesting that cognitive impacts are apparent even in CATA questionnaires [22].
Although some researchers have shown that both the CAS and CATA question formats provide similar results in terms of outcomes, time, and survey satisfaction, other researchers [12,23] disagree and suggest otherwise. Fundamentally, the CAS seeks a response (e.g., yes or no) for each item, while the CATA question format requires that respondents only check those that they believe apply (the "yes" response) [24]. Sudman [1] and Smyth et al. [10] suggest that respondents pay more attention, read all the items, and provide more thoughtful responses for CAS than CATA questions. CAS has also been shown to result in more detailed responses in terms of a mean number of affirmative checked ("agree" or "apply") responses per respondent as compared to the CATA format [10,17,24]. This finding is also consistent with behavior survey data conducted in different languages and countries of residence [25]. However, most of the CATA-CAS comparison studies are public opinion surveys, with only one studying perceptions of food or food behavior. A potential issue with this forced-choice kind of questioning has been associated with acquiescence bias [10]. Acquiescence bias is a type of response bias where respondents tend to mark (or agree with) with the positive connotation for all survey questions [26]. Further, Best and Krueger [27] suggested that requiring an answer for each item on a questionnaire could frustrate respondents and could lead to a high number of partial completes as respondents quit the survey before it is completed. Nicolaas et al. [12] and Smyth et al. [10] did not find such effects, but they had reasonably short questionnaires of 8-15 questions. Typically, studies in the sensory literature have longer questionnaires [28,29] than those in general survey studies that have been conducted, which could impact findings. Jaeger et al. [18] showed that respondents found that rating each attribute was "slightly more tedious" than using the CATA format. Perhaps because of that, CAS is popular with telephone surveys but appears to be rarely used with in-person or web surveys for sensory consumer research [11,14].
The overall objective of the current online survey was to compare the CAS and CATA question formats, which were used to collect consumers' motivations for eating five food items belonging to different food groups. Specific objectives were to collect and compare data in multiple (five) countries and a) compare the percentages of " apply" ("yes" or "agree") responses for CAS and CATA, b) establish the response ratios of CAS to CATA), c) identify the level of importance of the eating motivation constructs, and d) compare respondents' mean survey duration, survey liking, just about right (JAR) rating questions, and completion rates for the two question formats of an eating motivation survey (EMS).

Eating Motivation Survey (EMS)
An eating motivation survey (EMS) questionnaire [9,30,31] was modified to include questions on consumers' motivations for eating (in the original EMS) or not eating food items (added) that belong to five food groups [29]. The questionnaires were randomly assigned to respondents in either the CAS or CATA formats but not both (each respondent saw only one format). A total of 47 positive motivation items that could be categorized into 16 motivation constructs were assessed in each format of the EMS questionnaire (Table 1). Each eating motivation had 3 subscales or items except for the choice motivation that had only two subscales. The CATA and CAS questionnaires were designed so that respondents checked each of the motivation items that they agreed contributed to them eating that food (CATA) or checked either "yes" or "no" for each motivation (CAS) as to whether they believed it contributed to them eating that food. In the results and discussion sections of this paper, we use "agree" or "apply" to refer to responses for which the respondents selected motivation items (in CATA) or checked the "yes" option (in CAS). For CATA, all 47 items were presented on a single page, while, for the CAS question format, five questions were presented on each page. The number of CATA items (47) that were assessed in the current survey was not unusual. In fact, the literature shows several articles where a similar number of CATA items were evaluated [9,10,23,30]. The questionnaires focused on motivations for eating items from five food groups: foods rich in starch (e.g., potato and rice dishes), proteins (e.g., meat, beans), dairy, fruits, and sweet foods/desserts [30,32]. Food items fitting in these food groups and applicable to the particular country were used ( Table 2). For example, in the case of starchy foods, baked potatoes were used for the USA, while paella was used for Spain and white rice was used for Brazil, China, and India. These foods were chosen based on discussions with multiple sensory scientists in each country who reviewed and discussed all the foods chosen in all countries to ensure the products represented the "concept" of the food category as much as possible for that country. Where possible, similar foods were used (e.g., white rice in three countries for "starchy foods"), but where the product was not widely consumed in that form (e.g., Spain) or not consumed in a similar form by a large portion of the population (e.g., USA), alternative products were selected that were more commonly eaten. In addition to EMS, the online survey questionnaire also included other questions that were included in the survey timing. For example, two questions that investigated the respondents' survey experience in terms of liking (a hedonic question) and a rating based on the length of the survey (a just about right or JAR question) were included near the end of the survey. The survey liking question and the JAR question were each placed on separate pages. The survey was written in English for the respondents in the USA and the survey was translated into Simplified Mandarin, Hindi, Spanish, and Portuguese for respondents in China, India (English also provided as an option), Spain, and Brazil, respectively. The survey translation process used a variation of the translation, review, adjudication, retesting, and documentation (TRAPD) approach [33,34]. The full procedure for the survey methods, including translation, and the surveys in all five languages have been published previously [29].
This online survey was designed following a protocol for research with human subjects (IRB #7297.2) that was approved by the designated review board at Kansas State University, Manhattan, KS, USA.

Respondents and Recruitment
Respondents in five countries were recruited by Qualtrics, Provo, UT, USA using its or its partners' existing databases. Using the Qualtrics survey software, one format of the survey questionnaire with CAS questions and another with CATA questions were assigned randomly to 400+ respondents per country (N~200 per questionnaire per country) [29]. Respondents were required to be 18 or older and then were recruited to fill demographic quotas of age and gender for each questionnaire format (CAS and CATA). Four age groups (n = 50+ per age group) were used in this study: Generation Z (born in the years 1995 to 2001), Millennials (born in the years 1980 to 1994), Generation X (born in the years 1965 to 1979) and Baby Boomers or Boomers (born in the years 1944 to 1964). For each age group, 50% were female and 50% were male. Once the required number of completed responses for a particular quota was filled, newly qualified respondents (for the filled quotas) were discontinued from completing the EMS. Other demographic data that were collected for informational purposes included respondents' level of education, number of adults, and number of children in the households (Table 3). The percentages of CAS and CATA "agree" responses for each food group in each of the five countries was calculated. Percentages were used because the possible number of ticks/checks varied depending on how many people ate that particular food in a particular country and the number of subscales in the eating motivation category. Chi-square tests were used to compare significant differences in proportions between CATA and CAS.

Establishment of Ratios of CAS to CATA and Standard Indices for CAS and CATA
The ratios of percentages of "agree" responses for CAS to CATA were calculated to determine if the ratio of responses varied or remained the same between the two methods. Similarly, a "standard index of importance" (SII) of responses was determined for all motivation categories versus liking within CATA or CAS. This index value shows the proportion of the number of "agree" responses for any motivation to "agree" responses for the liking motivation, which has been shown in prior studies to be the highest motivation on average [9,30]. Using liking as the comparison index factor (the denominator in the proportion calculation) within each food group, country, and consumer demographic segment allows a within-sample "variable" to be used to adjust all comparisons and put them on a similar "scale" (typically 0-1.0). Note, it is possible to exceed 1.0 when a motivation exceeds liking in importance for a group of people, although this rarely happened. Put simply, the SII is 1.0 when the "agree" responses for any motivation are equal to the "agree" responses for liking within that method for that group of respondents. Similarly, the SII would be 0.5 when a motivation response is half the number compared to liking and so forth. The index was created using the principles espoused by creators of other indices for psychological phenomena that must be compared across various segments but can vary in response behavior across segments [35,36]. If the CAS and CATA formats were assessing the same behavioral patterns of consumers, then the SII values for CAS and CATA for the different motivation constructs would be similar or relatively close. However, if the SII values for the two formats were different, this would indicate that the questions from the two formats were interpreted, processed, and answered differently by the respondents. Major differences in standard index values for motivations within CATA or CAS would suggest that the results of the two methods likely would provide different information to the researchers. Such findings would suggest that CATA and CAS, for various reasons, do not measure the same psychological phenomena or, at a minimum, the results would be interpreted differently.

Identification of the Level of Importance for Motivation Constructs
Based on the percentages of "agree" responses for the CAS and CATA question formats of the online survey, ranking was used to identify the top five motivation constructs for each food group in each country.

Comparison of Question Format Completion Rates, Mean Survey Duration, Survey Liking, and JAR Rating
Analysis of variance (ANOVA) was conducted to assess the effect of survey format on survey liking, mean duration, and respondent JAR for EMS in each location. Percentages of completion rates for each of the two question formats were also calculated.

Percentages of "Agree" Responses for CAS and CATA
The CAS and the CATA question formats collected significantly different (p ≤ 0.01) percentages of "agree" responses for each of the 16 positive motivation constructs in all five countries for starchy foods (Table 4). Data for proteins, dairy, fruit, and sweet/dessert foods are presented in the (Tables A1-A4). The CAS format amassed a higher percentage of "agree" responses than the CATA question format, which is similar to that mentioned or found by other authors [1, [10][11][12]23,24]. For example, baked potatoes (a starchy food) in the USA (Table 4) received more "agree" responses for all of the eating motivation constructs when CAS was used as compared to when the CATA question format was used. A case in point, for the USA, 5% and 44% of respondents who completed the CATA question format identified choice limitation and liking, respectively, as important motivations for eating baked potatoes. Conversely, the percentage of responses that identified choice limitation and liking as important eating motivations was more than twice (25% and 92%, respectively) for people who answered the CAS format of the survey in the same country. Similar cases were observed for the other four countries (Brazil, China, India, and Spain) and for the other four food groups (protein-rich foods, dairy foods, fruit, and desserts). The large difference in the percentage of respondents who found a motivation "important" in CAS or CATA could be the result of several factors. Based on various survey theories, the higher proportion of unchecked items for the CATA format could be explained by respondents who tend to want to hurry through questionnaires and, thus, only focus on checking a few items, perhaps only the attributes or statements with the highest priority to them. Similarly, the lack of attention could also lead to checking only the first few items and not spending time reading the rest of the items but moving on to the next question after they felt they had satisfied the question requirement. Because the statement items were randomized for each respondent, the bias would be randomized throughout the test, but still influences the number of responses over all consumers. According to Jaeger et al. [21] and Smyth et al. [12], the smaller percentage of "agree" responses for CATA could partly be explained by respondents who missed them as they read the list of 47 items and it is also possible that respondents who were unsure whether an item was applicable did not mark that statement.
Conversely, the higher percentage of responses for CAS could be ascribed to the fact that the CAS question format required a response for each of the 47 eating motivations items, which was not the case for the CATA question format. Some authors suggest that the percentage of forced-choice scores (e.g., CAS) could be high if respondents choose the "yes" or "agree" category to avoid actively saying they "disagree" with any item or statement. This reasoning suggests that requiring respondents to provide either a "yes" or a "no" for each item could cause some respondents to lean towards giving a "yes" or "agree" response. This potential acquiescence bias has been noted by sensory researchers for some populations and is sometimes referred to as a "politeness" bias [37][38][39]. However, more recent studies have not shown such "politeness" effects [40,41]. Further, if the bias were true for this set of consumers and questions, we would expect to see consistently high percentages for CAS across all products and motivations within a country. We did not. The range of percentages for CAS "agree" responses is quite large, e.g., for the starch-rich food group, the CAS percentages range from 4% to 98% for Spain and 20% to 80% for India. For CATA, the range is smaller, from 0% to 63% for Spain and 2% to 33% for China. For all food categories, there are CAS "agree" responses greater than 90% and less than 10%, suggesting that respondents were not simply checking "agree" to be polite or checking in some random way.
The lower percentage of "agree" responses for the CATA format appears much more likely to be explained by a portion of respondents who paid lower attention to each statement and simply selected the "agree" option more randomly and chose fewer options because of that. This is supported by the fact that although less-used motivations tended to be used less in both methods, they were used so infrequently in CATA that they did not differentiate among themselves. This was not true for CAS data. For example, the least-used constructs for dairy foods generally were affect regulation, social norms, social image, visual appeal, and sociability. Within a country, the frequency of the use of and range for these attributes using the CATA format was less than 10% and 5%, respectively (except India, where it was 12% and 8% percentage, respectively). These attributes were neither discriminating nor helping to understand differences among motivations for eating by individuals using the CATA format. In comparison, these same motivations for dairy foods using the CAS format received a low of 10% to a high of 60% use (depending on the country), with a range of scores 24% points within any single country. Clearly, they were not only used more frequently in the CAS format but were also not contributing the same degree of explanation and discrimination of eating behaviors.

Response Ratios (CAS: CATA, Standard Index of Importance (SII))
Similar to prior studies [9,42], liking was almost always found to be peoples' greatest motivation (higher percentage of "agree" responses overall) for eating items from the five food groups in the five countries (Tables 4 and A1, Tables A2-A4). This did not change regardless of whether CAS or CATA was used for measurement in the survey. However, the importance of other constructs did change depending on the method of measurement used, CAS or CATA. This indicated that the detail of the data collected using CAS and that collected using CATA was different. In addition to the higher percentages of "agree" responses for CAS as compared to CATA, authors noticed that the ratio of CAS to CATA "agree" responses for each of the five food groups was not only greater than one but that it also varied largely depending on the particular motivation construct that was being assessed (Tables 5 and A5-A8). For example, the construct "health" was chosen at a higher frequency in CAS than CATA, suggesting that it may be more important than the CATA data imply. It is possible that either the CAS questions overestimated the level of importance of the health construct or that the CATA questions underestimated the level of importance of the same construct to the respondents. Further, and perhaps even more pertinent, the importance of the health construct changed (depending on the country) among the various food types when using CAS compared to CATA data. In the US, the importance of health in the dairy category (Table 5) increased 6-fold using CAS, whereas it only increased 2-fold for the fruit category (Table A5). These ratios were much more similar, approximately a 2-fold increase for both dairy and fruits, for the other countries in the study, but varied for other categories. In another example, the construct "traditional eating" in China showed large differences in importance for all food groups. However, the effect was much less for the starchy foods category (Table A6) than the protein food category (Table A7), a 3.4-fold vs. 6.2-fold increase, respectively. These changes were also noted for all other countries, but the differences among food groups were less with Brazil showing almost no variation among food groups. Quite large differences among CAS and CATA were noted for some other motivations, but those typically involved motivations where consumers actually chose that motivation infrequently in CATA. These findings point out a research gap for the validation of the CAS and CATA results using qualitative studies such as focus groups with respective populations to determine the level of accuracy that each question format provides. Another way to look at the importance of each construct is to compare its SII. This index, which shows how each construct or attribute compares in importance to the most important construct or attribute (in this case liking), is also shown for each of the five food groups for the five countries in Table 5, and Tables A5-A8. If the relative importance of each attribute is the same for the two methods, then the SII should be the same between the methods for each construct in each country. This was not the case in this study. Although we might expect some random variation among these values, the difference (SII:CAS minus SII:CATA) in the SII index ranges from 0.16 for the USA starch foods category, to 0.82 for that same category in China. This low SII index (0.16) shows some differences among methods but might be reasonably consistent, but the larger one clearly shows a large difference in information provided by the two methods may not be the same. Overall, the difference in the range of SII values within a country and food category is approximately 0.3, which would seem to indicate that the variation is enough to potentially impact subsequent data analysis and interpretation.
The variation in both percentage and index data is critically important because it shows that CAS and CATA are not necessarily providing the same information to researchers. If that is the case, researchers could infer different conclusions using the two methods. It is not a simple matter of getting more "agree" responses using the CAS format, it is a matter of getting both more and different percentages of "agree" responses depending (in this study) on the food category and the motivation. Thus, the suggestion that the primary difference in CAS and CATA is that people simply choose fewer statements or attributes when using CATA as compared to when using CAS is incorrect. This difference in scoring behavior between the two methods is a major issue that requires further investigation.

Level of Importance of Consumer Eating Motivations Based on CAS vs. CATA
To determine whether information from the two methods provides different interpretations would require a complete analysis of each country's data and publication of multiple maps and interpretations for each country. However, an examination of the major themes presented by each set of data can be gained by examining the relative positioning of the constructs consumers chose in the CAS and CATA studies (Table 6). Overall, the top five eating motivations were reasonably similar within a country within a food between CAS and CATA. The top motivation, which was liking for almost all foods and countries, was common for both CAS and CATA. Similarly, habits, need/hunger, pleasure, and health typically make up the top five constructs for eating behavior across most foods and countries regardless of whether CAS or CATA is used. Convenience, traditional eating, natural concerns, weight control, visual appeal, and sociability also appeared in various lists depending on the country, the food, and the method used for testing. In general, for so-called Western cultures (USA, Spain, and Brazil), the similarities between CAS and CATA data for the main constructs were similar, with some slight variation in their rankings based on percentage agreement with the statements. This was less true for India and China, where differences in the order of importance of the constructs and differences in which constructs were chosen were more frequent. This suggests that the "big picture" information may change, but not drastically, depending on the method and, in some cases, the cultural use of the two methods.   However, moderately used motivations (e.g., weight control, visual appeal, sociability, choice limitation, and social image) were much more likely to change in their ranked importance between CAS and CATA data within a food category within a country. For example, weight control was more likely to have a higher SII index for CATA than CAS data, pushing it into the "top 5" motivations, suggesting that it had greater importance when using that method.
Another potential way to look at the data is to compare it to prior studies using the same basic eating motivation survey. The most similar direct comparison (same CATA questionnaire used in the USA) [30] investigated food groups among people living in the USA. These findings suggested that all foods were eaten primarily because of liking. Need and hunger factored highly into meats, starchy foods, and dairy, with health important to dairy and fruit consumption. Habits for protein foods, convenience for starchy foods, and weight control for fruits were the other highest-ranking motivations. In addition to liking, the sweets category in that study was most influenced by pleasure and choice limitation. In the current survey, CATA findings for the USA showed some similarities and several differences in how respondents perceived the level of importance of each motivation construct for the five food groups that were tested. In both studies, liking was predominant and need and hunger often was key. Other constructs such as convenience, weight control, or health moved around somewhat or disappeared from the top set of motivations in one study or the other for some food groups. Other research in the USA [43] using the same set of constructs (except for choice limitation) and using a scale instead of a CATA measure found that liking, need and hunger, and habit were the three most important constructs with health, convenience, and pleasure next. These authors did not separate foods by groups. In that same study, results from India showed that liking, pleasure, need and hunger, and health were most important. This is somewhat different than the CATA findings from this study, which never showed pleasure in the top five of importance for India except for desserts. In Brazil [44], a study similar to the one from India showed that liking, habits, and need and hunger were the three primary constructs used by consumers to make food choices. Liking and habits were common findings in both that and our study. However, the current study with CATA data for India does not show need and hunger as one of the top three factors for any food, although it does tie for fourth place with fruit and dairy foods. In the CATA portion of this study, factors such as health, traditional eating, and natural concerns are more important. These differences to other studies show that CATA data find similar, but not the same information. These differences could be the result of the question format(s), e.g., the CATA format, or other issues including differences in the populations tested within each country, and differences in the recency of testing (although they were all completed within 5 years of each other).
The differences in response ratio for CAS and CATA in the current study suggest that motivations for eating foods could be shown to be more or less important depending on whether the CAS or CATA question formats were used in online consumer surveys [11]. This would be critical in how consumer data is interpreted and what decisions would be made thereafter. For example, product developers and sensory scientists usually use such data to identify product sensory attributes that drive consumer acceptance to guide product reformulations and product-line extensions. Sensory scientists may apply the CATA and CAS question formats in studies for product claim substantiation and studies that seek to understand more product consumer psychographics. Nutritionists and dietitians use these two question formats in baseline surveys when designing health and nutrition interventions and services for communities. However, without knowing which of the CATA or CAS question formats collects more accurate data, it may be difficult to recommend the use of one question format over the other in future online surveys. Information and interpretation based on these online surveys must be accurate if the consumer researchers are to attain their desired objective. This calls for careful consideration by consumer researchers when choosing to use either CATA or CAS in their online consumer surveys [11].

Mean Duration, Just About Right (JAR) Rating, Consumer Liking, and Completion Rates for CAS and CATA
The average mean duration to complete the CAS and CATA questions were found to be significantly different for countries such as the USA, China, and Spain. For these three countries, it took the respondents significantly longer to complete the CAS questionnaire than the CATA questionnaire ( Table 7). Smyth et al. [10] attributed the longer times taken to complete the CAS questions to the demand for deeper thought processes, as stated by Sudman and bardburn [1], and also the time consumed when selecting the disagree or "no" options, which was not the case for the CATA format of the survey. There was, however, no difference in the mean time required to complete both the CAS and CATA questions for respondents in Brazil and India. Both of these countries took longer to complete the questionnaire than respondents in other countries.  (Table 8). This partly explained the significantly higher scores for survey liking that the respondents in these countries expressed for the CATA format when they were asked about how they felt about the survey experience (Table 9). Although respondents in India also liked the CATA format more than the CAS format of the survey, they did not have a significant difference in the length of the two question formats of the survey.  The CAS version of the survey had a higher percentage of incomplete responses as compared to the CATA questionnaire (Figure 1). This was highlighted in India where the rate of survey completion for CAS was less than 50% as compared to the 95% completion rate for the CATA questionnaire in the same country. Such information must be considered as part of the overall planning for studies. However, completion rates and consumer liking of questionnaires should never be used as a reason to change a questionnaire format to one that collects data if the format will provide less or lower quality data and information.

Survey Limitations
This study has several limitations that must be considered when comparing the methods, food groups, and countries. First, except for the fruits and vegetables food group, where the banana was tested in all five countries, some different food items were used to test the other food groups in the countries. Differences in products chosen, such as potato for the USA and rice for all other countries, could have affected the reliability of the findings between countries in the online survey. However, specific differences among countries were not the focus of this study and the use of commonly eaten foods in each country likely was more critical to this study than a consistent food for each country that few people in a country might eat. Consumer researchers designing future similar studies may choose to have half of the products the same for all locations and the other half custom picked to match the individual locations that are being investigated. Such information might increase the cross-country comparison reliability, but would also double the size of the sample needed.
A more meaningful limitation is that all respondents who completed this survey in the five countries were literate and had access to the internet. This indicates that parts of the population(s) who were illiterate or had no access to the internet at the time of fielding were not included. Access is a common problem with surveys and each survey type has limitations related to its method of fielding. In this study, although the population was technically "national", most databases for online surveys have a preponderance of consumers with higher than average access to the internet. Survey respondents are also literate, have time and are willing to take an online survey, and can be distracted by other influences during the testing, which can slow them down or result in them becoming more frustrated with surveys that are more time consuming or more difficult to complete. Similar future international research surveys could be extended to include the responses of people who may not have access to the internet or be conducted in areas with more limited access to testing. However, survey location is a limitation of in-person surveys that usually are conducted in only one or a few locations, which provides a limited sample. For people who are unable to read, the survey can be read to them, either by telephone (if available) or in person. Such in-person, one-on-one testing has been reported, particularly in countries where literacy rates are low, but typically is not used for most survey research because of the large increase in resources required for such testing.

Conclusions
This online survey confirmed that the CAS question format provided more "apply" responses per attribute construct as compared to the CATA questionnaire format. Further, the response ratio of CATA to CAS responses was found to be different for each motivation construct, each food group, and for each country. This suggested that the level of importance that was accorded by the respondents to each motivation construct though similar in a few cases differed largely depending on whether the CAS or CATA question format was used in the online survey. The SII varied greatly within the CAS format and varied less for the CATA format, implying that the CAS format was much more discriminating among the motivation constructs than the CATA format. Although the overall "big picture" of the main constructs may appear similar when examining the ratios, the constructs that were not part of the top five appear to vary more, which alludes to differences in interpretation that may occur when the two methods are used and detailed information is needed.