Comparing Four Question Formats in Five Languages for On-Line Consumer Surveys
Abstract
:1. Introduction
1.1. Check-All-That-Apply (CATA)
1.2. Check-All-Statements (CAS)
1.3. Rate-All-That-Apply (RATA)
1.4. Rate-All-Statements (RAS)
2. Methods and Design
2.1. Questionnaire Development
2.2. Questionnaire Translation
2.3. Respondents and Recruitment
3. Procedure
3.1. Data Collection
3.1.1. Panel Screening Process
3.1.2. Survey Testing Design
3.2. Survey Timeline
3.3. Data Analysis
4. Expected Results
5. Discussion
5.1. Survey Timeline
5.1.1. Development of Survey Questionnaire (s)—Time for Completion: 3 Weeks in this Study
5.1.2. Test Design, Questionnaire, and Survey Flow Verification—Time for Completion: 1 Week in this Study
5.1.3. Respondents Recruitment and Survey Fielding—Time for Completion: 4–6 Weeks Depending on the Availability of Target Population
5.1.4. Data Analysis and Reporting—Time for Completion: On-Going
6. Limitations
Supplementary Materials
Author Contributions
Funding
Conflicts of Interest
References
- Antúnez, L.; Vidal, L.; De Saldamando, L.; Giménez, A.; Ares, G. Comparison of consumer-based methodologies for sensory characterization: Case study with four sample sets of powdered drinks. Food Qual. Prefer. 2017, 56, 149–163. [Google Scholar] [CrossRef]
- Jaeger, S.R.; Cadena, R.S.; Torres-Moreno, M.; Antúnez, L.; Vidal, L.; Giménez, A.; Hunter, D.C.; Beresford, M.K.; Kam, K.; Yin, D.; et al. Comparison of check-all-that-apply and forced-choice Yes/No question formats for sensory characterisation. Food Qual. Prefer. 2014, 35, 32–40. [Google Scholar] [CrossRef]
- Bruzzone, F.; Vidal, L.; Antúnez, L.; Giménez, A.; Deliza, R.; Ares, G. Comparison of intensity scales and CATA questions in new product development: Sensory characterisation and directions for product reformulation of milk desserts. Food Qual. Prefer. 2015, 44, 183–193. [Google Scholar] [CrossRef]
- Ares, G.; Etchemendy, E.; Antúnez, L.; Vidal, L.; Giménez, A.; Jaeger, S.R. Visual attention by consumers to check-all-that-apply questions: Insights to support methodological development. Food Qual. Prefer. 2013, 32, 210–220. [Google Scholar] [CrossRef]
- Ares, G.; Jaeger, S.R. Check-all-that-apply questions: Influence of attribute order on sensory product characterization. Food Qual. Prefer. 2013, 28, 141–153. [Google Scholar] [CrossRef]
- Ares, G.; Tárrega, A.; Izquierdo, L.; Jaeger, S.R. Investigation of the number of consumers necessary to obtain stable sample and descriptor configurations from check-all-that-apply (CATA) questions. Food Qual. Prefer. 2014, 31, 135–141. [Google Scholar] [CrossRef]
- Jaeger, S.R.; Beresford, M.K.; Paisley, A.G.; Antúnez, L.; Vidal, L.; Cadena, R.S.; Giménez, A.; Ares, G. Check-all-that-apply (CATA) questions for sensory product characterization by consumers: Investigations into the number of terms used in CATA questions. Food Qual. Prefer. 2015, 42, 154–164. [Google Scholar] [CrossRef]
- Rasinski, K.A.; Mingay, D.; Bradburn, N.M. Do Respondents Really “Mark All That Apply” On Self-Administered Questions? Public Opin. Q. 1994, 58, 400–408. [Google Scholar] [CrossRef]
- Jaeger, S.R.; Fiszman, S.; Reis, F.; Chheang, S.L.; Kam, K.; Pineau, B.; Deliza, R.; Ares, G. Influence of evoked contexts on hedonic product discrimination and sensory characterizations using CATA questions. Food Qual. Prefer. 2017, 56, 138–148. [Google Scholar] [CrossRef]
- Ares, G.; de Andrade, J.C.; Antúnez, L.; Alcaire, F.; Swaney-Stueve, M.; Gordon, S.; Jaeger, S.R. Hedonic product optimisation: CATA questions as alternatives to JAR scales. Food Qual. Prefer. 2017, 55, 67–78. [Google Scholar] [CrossRef]
- Alexi, N.; Nanou, E.; Lazo, O.; Guerrero, L.; Grigorakis, K.; Byrne, D.V. Check-All-That-Apply (CATA) with semi-trained assessors: Sensory profiles closer to descriptive analysis or consumer elicited data? Food Qual. Prefer. 2018, 64, 11–20. [Google Scholar] [CrossRef] [Green Version]
- Dos Santos, B.A.; Bastianello Campagnol, P.C.; da Cruz, A.G.; Galvão, M.T.E.L.; Monteiro, R.A.; Wagner, R.; Pollonio, M.A.R. Check all that apply and free listing to describe the sensory characteristics of low sodium dry fermented sausages: Comparison with trained panel. Food Res. Int. 2015, 76, 725–734. [Google Scholar] [CrossRef] [PubMed]
- Smyth, J.D.; Christian, L.M.; Dillman, D.A. Does “yes or no” on the telephone mean the same as “check-all-that-apply” on the web? Public Opin. Q. 2008, 72, 103–113. [Google Scholar] [CrossRef] [Green Version]
- Nicolaas, G.; Campanelli, P.; Hope, S.; Jäckle, A.; Lynn, P. Revisiting “yes/no” versus “check all that apply”: Results from a mixed modes experiment. Surv. Res. Methods 2015, 9, 189–204. [Google Scholar] [CrossRef]
- King, S.C.; Meiselman, H.L. Development of a method to measure consumer emotions associated with foods. Food Qual. Prefer. 2010, 21, 168–177. [Google Scholar] [CrossRef]
- Jaeger, S.R.; Swaney-Stueve, M.; Chheang, S.L.; Hunter, D.C.; Pineau, B.; Ares, G. An assessment of the CATA-variant of the EsSense Profile®. Food Qual. Prefer. 2018, 68, 360–370. [Google Scholar] [CrossRef]
- Phan, U.T.X.; Chambers, E. Motivations for choosing various food groups based on individual foods. Appetite 2016, 105, 204–211. [Google Scholar] [CrossRef] [PubMed]
- Smyth, J.D.; Dillman, D.A.; Christian, L.M.; Stern, M.J. Comparing check-all and forced-choice question formats in Web surveys. Public Opin. Q. 2006, 70, 66–77. [Google Scholar] [CrossRef] [Green Version]
- Sudman, S.; Bradburn, N.M. Asking Questions: A Practical Guide to Questionnaire Design, 1st ed.; Jossey-Bass: San Francisco, CA, USA, 1982. [Google Scholar]
- Phan, U.T.X.; Chambers, E. Application of An Eating Motivation Survey to Study Eating Occasions. J. Sens. Stud. 2016, 31, 114–123. [Google Scholar] [CrossRef]
- Krosnick, J.A. Response strategies for coping with the cognitive demands of attitude measures in surveys. Appl. Cogn. Psychol. 1991, 5, 213–236. [Google Scholar] [CrossRef]
- Jaeger, S.R.; Lee, S.M.; Kim, K.O.; Chheang, S.L.; Roigard, C.M.; Ares, G. CATA and RATA questions for product-focused emotion research: Five case studies using emoji questionnaires. Food Qual. Prefer. 2018, 68, 342–348. [Google Scholar] [CrossRef]
- Popper, R.; Kroll, D.R. Just-About-Right Scales in Consumer Research. Chemo Sense 2005, 7, 1–6. [Google Scholar]
- Ares, G.; Jaeger, S.R. Check-all-that-apply (CATA) questions with consumers in practice: Experimental considerations and impact on outcome. In Rapid Sensory Profiling Techniques and Related Methods: Applications in New Product Development and Consumer Research; Delarue, J., Lawlor, J., Ben Rogeaux, M., Eds.; Woodhead Publishing: Cambridge, UK, 2015; pp. 227–245. ISBN 9781782422587. [Google Scholar]
- Krosnick, J.; Licensors, I.T.S.; Disclaim, S.; Express, A.L.L.; Implied, O.R.; Limitation, I.W.; Warranties, A.N.Y.; Availability, F.O.R.; Or, M.; For, F. Survey research. Annu. Rev. Psychol. 1999, 50, 537–567. [Google Scholar] [CrossRef]
- Chang, L.; Krosnick, J.A. National surveys via RDD telephone interviewing versus the internet: Comparing sample representativeness and response quality. Public Opin. Q. 2009, 73, 641–678. [Google Scholar] [CrossRef] [Green Version]
- Thomas, R.; Klein, J. Merely Incidental?: Effects of Response Format on Self-reported Behavior. J. Off. Stat. 2006, 22, 221. [Google Scholar]
- Best, S.J.; Krueger, B.S. Internet Data Collection, Issue 141; SAGE: Los Angeles, CA, USA, 2004; ISBN 0761927107. [Google Scholar]
- Jaeger, S.R.; Ares, G. RATA questions are not likely to bias hedonic scores. Food Qual. Prefer. 2015, 44, 157–161. [Google Scholar] [CrossRef]
- Vidal, L.; Ares, G.; Hedderley, D.I.; Meyners, M.; Jaeger, S.R. Comparison of rate-all-that-apply (RATA) and check-all-that-apply (CATA) questions across seven consumer studies. Food Qual. Prefer. 2018, 67, 49–58. [Google Scholar] [CrossRef]
- National Health and Medical Research Council The Five Food Groups|Eat For Health. Available online: https://www.eatforhealth.gov.au/food-essentials/five-food-groups (accessed on 14 July 2020).
- Renner, B.; Sproesser, G.; Strohbach, S.; Schupp, H.T. Why we eat what we eat. The Eating Motivation Survey (TEMS). Appetite 2012, 59, 117–128. [Google Scholar] [CrossRef] [Green Version]
- Bell, R.; Marshall, D.W. The construct of food involvement in behavioral research: Scale development and validation ☆. Appetite 2003, 40, 235–244. [Google Scholar] [CrossRef]
- Roininen, K.; Tuorila, H. Health and taste attitudes in the prediction of use frequency and choice between less healthy and more healthy snacks. Food Qual. Prefer. 1999, 10, 357–365. [Google Scholar] [CrossRef]
- Pliner, P.; Hobden, K. Development of a scale to measure the trait of food neophobia in humans. Appetite 1992, 19, 105–120. [Google Scholar] [CrossRef]
- Castro, M.; Chambers, E. Willingness to eat an insect based product and impact on brand equity: A global perspective. J. Sens. Stud. 2019, 34, 1–10. [Google Scholar] [CrossRef] [Green Version]
- Curtarelli, M.; van Houten, G. Questionnaire translation in the European company survey: Conditions conducive to the effective implementation of a TRAPD-based approach. Transl. Interpret. 2018, 10, 34–54. [Google Scholar] [CrossRef] [Green Version]
- Harkness, J.A. Questionnaire Translation. In Cross-Cultural Survey Methods; Harkness, J.A., Van de Vijver, F.J.R., Mohler, P.P., Eds.; John Wiley & Sons, Ltd.: New York, NY, USA, 2003; pp. 35–56. [Google Scholar]
- Behr, D. Assessing the use of back translation: The shortcomings of back translation as a quality testing method. Int. J. Soc. Res. Methodol. 2017, 20, 573–584. [Google Scholar] [CrossRef]
- Harkness, J.; Pennell, B.-E.; Schoua-Glusberg, A. Survey Questionnaire Translation and Assessment. In Methods for Testing and Evaluating Survey Questionnaires; Presser, S., Rothgeb, J.M., Couper, M.P., Lessler, J.T., Martin, E., Martin, J., Singer, E., Eds.; John Wiley & Sons, Ltd.: New York, NY, USA, 2004; pp. 453–473. [Google Scholar]
- Koppel, K.; Suwonsichon, S.; Chitra, U.; Lee, J.; Chambers, E. Eggs and Poultry Purchase, Storage, and Preparation Practices of Consumers in Selected Asian Countries. Foods 2014, 3, 110–127. [Google Scholar] [CrossRef]
- Koppel, K.; Chambers, E.; Vázquez-Araújo, L.; Timberg, L.; Carbonell-Barrachina, T.A.; Suwonsichon, S. Cross-country comparison of pomegranate juice acceptance in Estonia, Spain, Thailand, and United States. Food Qual. Prefer. 2014, 31. [Google Scholar] [CrossRef] [Green Version]
- Castro, M.; Chambers, E. Consumer Avoidance of Insect Containing Foods: Primary Emotions, Perceptions and Sensory Characteristics Driving Consumers Considerations. Foods 2019, 8, 351. [Google Scholar] [CrossRef] [Green Version]
- Roser, M.; Ritchie, H.; Ortiz-Ospina, E. Internet. Our World Data. Available online: https://ourworldindata.org/internet (accessed on 5 June 2020).
- Chambers, E.I.; Smith, E.A. Use of qualitative research in product research and development. In Sensory Science Theory and Applications in Food; Lawless, H., Klein, B., Eds.; Marcel Dekker: New York, NY, USA, 1993; pp. 395–412. [Google Scholar]
- Muñoz, A.M.; King, S.C. International Consumer Product Testing Across Cultures and Countries; ASTM International: West Conshohocken, PA, USA, 2007; ISBN 978-0-8031-5690-6. [Google Scholar]
- Liu, M.; Wronski, L. Trap questions in online surveys: Results from three web survey experiments. Int. J. Mark. Res. 2018, 60, 32–49. [Google Scholar] [CrossRef] [Green Version]
Countries | Starchy Foods (Carbohydrates) | Proteins Foods (Meat, Fish, Eggs) | Milk and Dairy Foods | Fruit and Vegetables | Desserts(Fats and Sugars) |
---|---|---|---|---|---|
Brazil | White Rice | Feijao | Milk | Bananas | Brigadeiro |
China | White rice | Red braised pork belly | Soy Milk | Bananas | Pan-fried red bean paste cakes |
India | White rice | Toor Dal | Milk | Bananas | Gulab Jamun |
Spain | Paella | Jamón Serrano | Milk | Bananas | Turrón |
USA | Baked Potato | Hamburger | Cheese | Bananas | Chocolate cake with frosting |
Liking | Sociability |
---|---|
Because it tastes good | Because it is social |
Because I like it | So that I can spend time with other people |
Because I have an appetite for it | Because it makes social gatherings more comfortable |
Habits | Price |
Because I usually eat it | Because it is inexpensive |
Because I am familiar with it | Because it is on sale |
Because I’m accustomed to eating it | Because I don’t want to spend any more money |
Need and Hunger | Visual Appeal |
Because I’m hungry | Because it spontaneously appeals to me |
Because it is pleasantly filling | Because the presentation is appealing (e.g., packaging) |
Because I need energy | Because I recognize it from advertisements or have seen it on TV |
Health | Weight Control |
Because it is healthy | Because it is low in calories |
To maintain a balanced diet | Because it is low in fat |
Because it keeps me in shape (e.g., energetic, motivated) | Because I watch my weight |
Convenience | Affect Regulation |
Because it is quick to prepare | Because I am sad |
Because it is the most convenient | Because I feel lonely |
Because it is easy to prepare | Because I am frustrated |
Pleasure | Social Norms |
Because I enjoy it | Because I am supposed to eat it |
In order to indulge myself | To avoid disappointing someone who is trying to make me happy |
In order to reward myself | Because it would be impolite not to eat it |
Traditional Eating | Social Image |
Because I grew up with it | Because others like it |
Because it belongs to certain situations | Because it is trendy |
Out of traditions (e.g., family traditions, special occasions) | Because it makes me look good in front of others |
Natural Concerns | Choice |
Because it is natural (e.g., not genetically modified) | I want to eat it every day |
Because it contains no harmful substances | Because it is the only choice |
Because it is organic |
Liking | Sociability |
---|---|
I don’t like it | Price |
It is not something I have the desire to eat at this time | The price was too high |
Habits | Visual Appeal |
I don’t usually eat it | I don’t like the way it looked |
Need and Hunger | Weight Control |
It is not filling enough | It is too high in calories |
The portion size was not suitable | Affect Regulation |
Health | This food makes me feel sad, lonely, or frustrated |
It is not healthy | Social Norms |
Convenience | I am not supposed to eat it |
It is not convenient | Social Image |
Pleasure | It is not a food I eat around other people |
I do not want to indulge myself | Eating it makes me seem “behind the times” |
Traditional Eating | Choice |
I don’t think it is a snack | I had it recently and I don’t want to eat the same food too often |
It is not appropriate for the situation | I would never choose this because I like to eat the same food every day |
Natural Concerns | |
It is not organic |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Seninde, D.R.; Chambers, E., IV. Comparing Four Question Formats in Five Languages for On-Line Consumer Surveys. Methods Protoc. 2020, 3, 49. https://doi.org/10.3390/mps3030049
Seninde DR, Chambers E IV. Comparing Four Question Formats in Five Languages for On-Line Consumer Surveys. Methods and Protocols. 2020; 3(3):49. https://doi.org/10.3390/mps3030049
Chicago/Turabian StyleSeninde, Denis Richard, and Edgar Chambers, IV. 2020. "Comparing Four Question Formats in Five Languages for On-Line Consumer Surveys" Methods and Protocols 3, no. 3: 49. https://doi.org/10.3390/mps3030049