The State of Automated Facial Expression Analysis (AFEA) in Evaluating Consumer Packaged Beverages
Abstract
:1. Introduction
1.1. Neuromarketing and Its Benefits
1.2. Expression and Emotion Analysis History
1.3. Relationship between Emotion and Facial Expression
1.4. Targeting Specific Source of Emotion
2. Materials and Methods
2.1. Research Goals
- What software is currently being used?
- What population sizes are researchers studying?
- How are researchers developing experimental designs and what are their trends?
- Which methodologies have proven effective?
- Are there any general trends in results?
- How can AFEA be employed in product development settings?
2.2. PRISMA Systematic Literature Review
3. Results and Discussion
3.1. Systematic Literature Review
3.2. Study Objectives
3.3. Participants
3.4. Country of Origin
3.5. Implicit vs. Explicit Methodology
3.6. Stimuli Presentation
3.7. Sensory Environment Setup
3.8. Procedural Nature
3.9. Suggestions for Successful Implementation
- Crist et al. recommends between 10 and 50 participants for a trial [48]; however, Kostrya et al. refers to a sample size of 30 as small [46]. A group of ten participants does not have much statistical power. The interquartile range for the number of participants in study was 30–75 participants (Figure 3), which seems to be an appropriate participant range in studies.
- Before participants begin, the study protocol should be thoroughly reviewed [48].
- Studies should occur in an isolated sensory booth to minimize distractions [48].
- Only one participant should be in the sensory booth during experimentation [48].
- The camera and a monitor presenting the instructions should be at face level to keep participants’ focus on the camera [48].
- The height, distance, and angle of the camera should be adjustable to keep the participant’s face and head in the middle of the screen [57].
- Lighting is essential—100% overhead daylight lighting with a diffuse frontal light helps remove shadows, which can obscure the image [48].
- If liquid samples are used, the participant should drink the entire sample, quickly dropping the cup below their face. Video analysis should begin post-consumption, which is the moment the cup is not obscuring the face [48].
- Participants should avoid sudden movements and should not look away from the camera during the post-consumption period [48].
- Continuous calibration should be used during analysis [48].
3.10. Data Management and Analysis
3.11. Differentiation of Stimuli by Expressed Emotion
3.12. Population Influence
3.13. Expression Bias, Interpretation, and Calibration
3.14. Data Exclusion
3.15. Software Influence
4. Conclusions
Author Contributions
Funding
Conflicts of Interest
Appendix A
Author | Mean Age | Pop. | Sample | Process | AFEA Software | Other Analyses | Country |
Arnade [49] | n/a | 52 | Milk | Implicit | Face Reader 4.0 | EsSense, 9-point hedonic scale | USA |
Bartkiene [66] | 18–85 | 505 | Solid food | Implicit | Face Reader 6.0 | Questionnaire, 10-point hedonic scale | Lithuania |
Crist [60] | 18–70 | 65 | Caffeine solutions | Implicit | Face Reader 6.0 | 9-point hedonic scale | USA |
Danner [33] | 23 | 75 and 78 | Orange juice | Implicit and Explicit | Face Reader 4.0 | 9-point hedonic scale | Austria |
Danner [34] | 22.9 | 99 | Juices | Implicit and Explicit | Face Reader 5.0 | SCL, ST, HR, PVA, 9-point hedonic scale | Austria |
de Wijk [41] | 15.5 | 16 | Various foods | Implicit | Face Reader | SCR, HR, ST | The Netherlands |
de Wijk [53] | 18–65 | 32 | Cooked food (chicken, tofu, veggie chicken) | Implicit | Face Reader 7.0 | VAS, HR | The Netherlands |
de Wijk [55] | 30.1 (female), 36.2 (male) | 19 | Breakfast drinks | Implicit | Face Reader 4.0 | HR, SCL, ST, VAS | The Netherlands |
Fuentes [62] | n/a | 30 and 59 | Beers; Images | Implicit | Face Reader 7.1 | Questionnaire, VAS, HR, ST | Australia |
Garcia-Burgos [40] | 24.2 | 40 | Bitter solutions | Implicit | Face Reader 4.0 | Eating behavior questionnaire (FPQ, FFQ, DHQ, FCQ-SP), caloric intake, 9-point hedonic scale | Spain |
Garcia-Burgos [67] | 24.8 | 59 | Bitter foods | Implicit | Face Reader 4.0 | 9-point scale, questionnaire | Spain |
Gunaratne [65] | 25–55 | 60 | Chocolate packaging | Implicit | Face Reader 6.0 | Questionnaire, eye tracking | Australia |
He [50] | 20–25 | 26 | Food odors | Implicit | Face Reader 4.0 | VAS, PrEmo | The Netherlands |
He [51] | 18–30 | 28 | Food odors | Implicit | Face Reader 5.0 | VAS, PrEmo, HR, SCL, ST | The Netherlands |
He [52] | 23 | 24 | Semi-liquid foods | Implicit | Face Reader 4.0 | Questionnaire, SC, ST, HR | The Netherlands |
Horska [56] | n/a | 22 | Wine | Implicit | Face Reader | 9-point hedonic scale, EEG | Slovak Republic |
Juodeikiene [30] | 22 | 12–20 | Confectionary products | Explicit | Face Reader 5.0 | 7-point hedonic scale | Lithuania |
Juodeikiene [76] | 21–24 | 109 | Bread and chocolate | Explicit | Face Reader 5.0 | Questionnaire, 9-point hedonic scale | Lithuania |
Kerrihard [69] | 23.8 | 100 | Salt levels in mashed potato | Implicit | Face Reader | Questionnaire, | USA |
Kostyra [42] | 23 | 30 | Smoked ham | Implicit | Face Reader 4.0 | n/a | Poland |
Kostyra [46] | 23 | 30 | Smoked ham | Explicit | Face Reader 4.0 | QDA, eye tracking, 9-point hedonic scale | Poland |
Leitch [39] | 20–60 | 31 | Tea | Implicit | Face Reader 5.0 | 9-point hedonic scale, modified EsSense | USA |
Mahieu [47] | 18–65 | 44 | Chocolates, perfume, video ads | Implicit | Project Oxford | n/a | France |
Pentus [35] | n/a | 107 | Juice packaging | Implicit | Realeyes | none | Estonia |
Ploom [32] | n/a | 57 and 258 | Biscuit packaging | Implicit | Face Reader 7.0 | Eye tracking, online survey | Estonia |
Rocha [77] | 18–58 | 50 | Herbal infusions | Implicit | Face Reader 6.0 | EsSense, 9-point hedonic scale | Portugal |
Rocha-Parra [38] | 22.3 | 73 | Wine extract | Implicit | FACET SDK | 10-point rating scale | Argentina |
Samant [27] | 39 | 102 | Basic solutions | Implicit | iMotions 6.1 | EsSense, SCR, HR, ST, VAS, 9-point hedonic scale | USA |
Samant [63] | 41.13 | 100 | Vegetable juices | Implicit | iMotions | 9-point scale, VAS, EsSense | USA |
Torrico [70] | 18–60 | 60 | Images and chocolate | Implicit | Face Reader 6.1 | ST, BP, 3-point face scale | Australia |
Vergura [26] | 24; 26 | 40 and 60 | Package images | Implicit | Face Reader 5.0 | 9-point hedonic scale and questionnaire | Italy |
Walsh [36] | 18–25 | 27 | Milk | Implicit | Face Reader 5.0 | EsSense, 9-point hedonic scale | USA |
Walsh [54] | 18–29 | 40 | Videos showing food quality and safety concerns | Implicit | Face Reader 6.0 | EEG, ECG (HR), 7-point hedonic scale, EsSense | USA |
Walsh [61] | 18–29 | 40 | Videos of breakfast meals | Implicit | Face Reader 6.0 | EEG, HR, 7-point hedonic scale, EsSense | USA |
Yu [64] | 22.6 | 120 | Graphics | Implicit | Face Reader | Verbal self-reports | Taiwan |
Zeinstra [43] | 8.5 | 6 | Juices | Implicit | Observer TM | 3-point face scale, ranking | The Netherlands |
Zhi [57] | 24.1 | 46 | Fruit juices | Implicit | Face Reader 5.0 | 9-point hedonic scale | China |
Zhi [78] | 18–39 | 50 | Basic solutions | Implicit | Face Reader 5.0 | 9-point hedonic scale, VAS | China |
References
- Stasi, A.; Songa, G.; Mauri, M.; Ciceri, A.; Diotallevi, F.; Nardone, G.; Russo, V. Neuromarketing Empirical Approaches and Food Choice: A Systematic Review. Food Res. Int. 2017, 108, 650–664. [Google Scholar] [CrossRef] [PubMed]
- Köster, E.P.; Mojet, J. From Mood to Food and from Food to Mood: A Psychological Perspective on the Measurement of Food-Related Emotions in Consumer Research. Food Res. Int. 2015, 76, 180–191. [Google Scholar] [CrossRef]
- Ekman, P.; Friesen, W.V. Manual of the Facial Action Coding System (FACS); Consulting Psychologists Press: Palo Alto, CA, USA, 1978. [Google Scholar]
- Kring, A.M.; Kerr, S.L.; Smith, D.A.; Neale, J.M. Flat Affect in Schizophrenia does not Reflect Diminished Subjective Experience of Emotion. J. Abnorm. Psychol. 1993, 102, 507. [Google Scholar] [CrossRef]
- Izard, C.E. The Maximally Discriminative Facial Coding System; University of Delaware: Newark, NJ, USA, 1979. [Google Scholar]
- Tronick, E.; Als, H.; Brazelton, T.B. Monadic Phases: A Structural Descriptive Analysis of Infant-Mother Face to Face Interaction. Merrill-Palmer Q. Behav. Dev. 1980, 26, 3–24. [Google Scholar]
- Kohler, C.G.; Martin, E.A.; Stolar, N.; Barrett, F.S.; Verma, R.; Brensinger, C.; Gur, R.C. Static Posed and Evoked Facial Expressions of Emotions in Schizophrenia. Schizophr. Res. 2008, 105, 49–60. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Sayette, M.A.; Smith, D.W.; Breiner, M.J.; Wilson, G.T. The Effect of Alcohol on Emotional Response to a Social Stressor. J. Stud. Alcohol Drugs 1992, 53, 541–545. [Google Scholar] [CrossRef] [PubMed]
- Affectiva. Available online: https://www.affectiva.com (accessed on 27 January 2020).
- CrowdEmotion. Available online: https://www.crowdemotion.co.uk/ (accessed on 27 January 2020).
- EmoVu. Available online: https://www.eyeris.ai/ (accessed on 27 January 2020).
- Face++. Available online: https://www.faceplusplus.com/ (accessed on 27 January 2020).
- FaceReader. Available online: https://www.noldus.com/facereader (accessed on 27 January 2020).
- FacioMetrics. Available online: https://research.fb.com/category/augmented-reality-virtual-reality/ (accessed on 27 January 2020).
- Findface. Available online: https://findface.pro/en/ (accessed on 27 January 2020).
- iMotions. Available online: https://imotions.com/ (accessed on 27 January 2020).
- Insight SDK. Available online: https://sightcorp.com/insight/ (accessed on 27 January 2020).
- Kairos. Available online: https://www.kairos.com/ (accessed on 27 January 2020).
- Nviso. Available online: https://www.nviso.ai/ (accessed on 27 January 2020).
- Observer. Available online: https://www.noldus.com/observer-xt (accessed on 27 January 2020).
- Project Oxford. Available online: https://azure.microsoft.com/en-us/services/cognitive-services/face/ (accessed on 27 January 2020).
- Realeyes. Available online: https://www.realeyesit.com/ (accessed on 27 January 2020).
- SkyBiometry. Available online: https://skybiometry.com/ (accessed on 27 January 2020).
- Meiselman, H.L. A Review of the Current State of Emotion Research in Product Development. Food Res. Int. 2015, 76, 192–199. [Google Scholar] [CrossRef]
- Ekmann, P. Universal Facial Expressions in Emotion. Studia Psychol. 1973, 15, 140–146. [Google Scholar]
- Vergura, D.T.; Luceri, B. Product Packaging and Consumers’ Emotional Response. Does Spatial Representation Influence Product Evaluation and Choice? J. Consum. Mark. 2018, 35, 218–227. [Google Scholar] [CrossRef]
- Samant, S.S.; Chapko, M.J.; Seo, H.S. Predicting Consumer Liking and Preference Based on Emotional Responses and Sensory Perception: A Study with Basic Taste Solutions. Food Res. Int. 2017, 100, 325–334. [Google Scholar] [CrossRef]
- Erickson, K.; Schulkin, J. Facial Expressions of Emotion: A Cognitive Neuroscience Perspective. Brain Cogn. 2003, 52, 52–60. [Google Scholar] [CrossRef]
- Desmet, P.M.; Schifferstein, H.N. Sources of Positive and Negative Emotions in Food Experience. Appetite 2008, 50, 290–301. [Google Scholar] [CrossRef] [PubMed]
- Juodeikiene, G.; Basinskiene, L.; Vidmantiene, D.; Klupsaite, D.; Bartkiene, E. The Use of Face Reading Technology to Predict Consumer Acceptance of Confectionery Products. In Proceedings of the Food for Consumer Well-Being, 9th Baltic Conference on Food Science and Technology, Jelgava, Latvia, 8–9 May 2014; pp. 276–279. [Google Scholar]
- Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med. 2009, 151, 264–269. [Google Scholar] [CrossRef] [Green Version]
- Ploom, K.; Pentus, K.; Kuusik, A.; Varblane, U. The Effect of Culture on the Perception of Product Packaging: A Multimethod Cross-Cultural Study. J. Int. Consum. Mark. 2019, 1–15. [Google Scholar] [CrossRef]
- Danner, L.; Sidorkina, L.; Joechl, M.; Duerrschmid, K. Make a Face! Implicit and Explicit Measurement of Facial Expressions Elicited by Orange Juices Using Face Reading Technology. Food Qual. Prefer. 2014, 32, 167–172. [Google Scholar] [CrossRef]
- Danner, L.; Haindl, S.; Joechl, M.; Duerrschmid, K. Facial Expressions and Autonomous Nervous System Responses Elicited by Tasting Different Juices. Food Res. Int. 2014, 64, 81–90. [Google Scholar] [CrossRef]
- Pentus, K.; Mehine, T.; Kuusik, A. Considering Emotions in Product Package Design through Combining Conjoint Analysis with Psycho Physiological Measurements. Procedia-Soc. Behav. Sci. 2014, 148, 280–290. [Google Scholar] [CrossRef] [Green Version]
- Walsh, A.M.; Duncan, S.E.; Potts, H.; Gallagher, D.L. Comparing Quality and Emotional Responses as Related to Acceptability of Light-Induced Oxidation Flavor in Milk. Food Res. Int. 2015, 76, 293–300. [Google Scholar] [CrossRef]
- Walsh, J.A.; Creighton, S.E.; Rutherford, M.D. Emotion Perception or Social Cognitive Complexity: What Drives Face Processing Deficits in Autism Spectrum Disorder? J. Autism Dev. Disord. 2016, 46, 615–623. [Google Scholar] [CrossRef]
- Rocha-Parra, D.; García-Burgos, D.; Munsch, S.; Chirife, J.; Zamora, M.C. Application of Hedonic Dynamics Using Multiple-Sip Temporal-Liking and Facial Expression for Evaluation of a New Beverage. Food Qual. Prefer. 2016, 52, 153–159. [Google Scholar] [CrossRef] [Green Version]
- Leitch, K.A.; Duncan, S.E.; O’Keefe, S.; Rudd, R.; Gallagher, D.L. Characterizing Consumer Emotional Response to Sweeteners Using an Emotion Terminology Questionnaire and Facial Expression Analysis. Food Res. Int. 2015, 76, 283–292. [Google Scholar] [CrossRef] [Green Version]
- García-Burgos, D.; Zamora, M.C. Facial Affective Reactions to Bitter-Tasting Foods and Body Mass Index in Adults. Appetite 2013, 71, 178–186. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- De Wijk, R.A.; Kooijman, V.; Verhoeven, R.H.G.; Holthuysen, N.T.E.; de Graaf, C. Autonomic Nervous System Responses on and Facial Expressions to the Sight, Smell, and Taste of Liked and Disliked Foods. Food Qual. Prefer. 2012, 26, 196–203. [Google Scholar] [CrossRef]
- Kostyra, E.; Rambuszek, M.; Waszkiewicz-Robak, B.; Laskowski, W.; Blicharski, T.; Poławska, E. Consumer Facial Expression in Relation to Smoked Ham with the Use of Face Reading Technology. The Methodological Aspects and Informative Value of Research Results. Meat Sci. 2016, 119, 22–31. [Google Scholar] [CrossRef]
- Zeinstra, G.G.; Koelen, M.A.; Colindres, D.; Kok, F.J.; De Graaf, C. Facial Expressions in School-Aged Children Are a Good Indicator of ‘Dislikes’, but Not of ‘Likes’. Food Qual. Prefer. 2009, 20, 620–624. [Google Scholar] [CrossRef]
- Lagast, S.; Gellynck, X.; Schouteten, J.J.; De Herdt, V.; De Steur, H. Consumers’ Emotions Elicited by Food: A Systematic Review of Explicit and Implicit Methods. Trends Food Sci. Technol. 2017, 69, 172–189. [Google Scholar] [CrossRef]
- He, W.; Boesveldt, S.; De Graaf, C.; De Wijk, R.A. Dynamics of Autonomic Nervous System Responses and Facial Expressions to Odors. Front. Psychol. 2014, 5, 110. [Google Scholar] [CrossRef] [Green Version]
- Kostyra, E.; Wasiak-Zys, G.; Rambuszek, M.; Waszkiewicz-Robak, B. Determining the Sensory Characteristics, Associated Emotions and Degree of Liking of the Visual Attributes of Smoked Ham. LWT-Food Sci. Technol. 2016, 65, 246–253. [Google Scholar] [CrossRef]
- Mahieu, B.; Visalli, M.; Schlich, P.; Thomas, A. Eating Chocolate, Smelling Perfume or Watching Video Advertisement: Does It Make Any Difference on Emotional States Measured at Home Using Facial Expressions? Food Qual. Prefer. 2019, 77, 102–108. [Google Scholar] [CrossRef]
- Crist, C.A.; Duncan, S.E.; Gallagher, D.L. Protocol for Data Collection and Analysis Applied to Automated Facial Expression Analysis Technology and Temporal Analysis for Sensory Evaluation. J. Vis. Exp. 2016, 114. [Google Scholar] [CrossRef]
- Arnade, E.A. Measuring Consumer Emotional Response to Tastes and Foods through Facial Expression Analysis. Master’s Thesis, Virginia Tech, Blacksburg, VA, USA, 2014. [Google Scholar]
- He, W.; Boesveldt, S.; de Graaf, C.; de Wijk, R.A. The Relation between Continuous and Discrete Emotional Responses to Food Odors with Facial Expressions and Non-Verbal Reports. Food Qual. Prefer. 2016, 48, 130–137. [Google Scholar] [CrossRef]
- He, W.; de Wijk, R.A.; de Graaf, C.; Boesveldt, S. Implicit and Explicit Measurements of Affective Responses to Food Odors. Chem. Senses 2016, 41, 661–668. [Google Scholar] [CrossRef] [PubMed]
- He, W.; Boesveldt, S.; Delplanque, S.; de Graaf, C.; de Wijk, R.A. Sensory-Specific Satiety: Added Insights from Autonomic Nervous System Responses and Facial Expressions. Physiol. Behav. 2017, 170, 12–18. [Google Scholar] [CrossRef] [Green Version]
- De Wijk, R.A.; Kaneko, D.; Dijksterhuis, G.B.; van Zoggel, M.; Schiona, I.; Visalli, M.; Zandstra, E.H. Food Perception and Emotion Measured over Time In-Lab and in-Home. Food Qual. Prefer. 2019, 75, 170–178. [Google Scholar] [CrossRef]
- Walsh, A.M.; Duncan, S.E.; Bell, M.A.; O’Keefe, S.F.; Gallagher, D.L. Integrating Implicit and Explicit Emotional Assessment of Food Quality and Safety Concerns. Food Qual. Prefer. 2017, 56, 212–224. [Google Scholar] [CrossRef] [Green Version]
- De Wijk, R.A.; Mensink, M.G.; Verhoeven, R.H.; De Graaf, C. ANS Responses and Facial Expressions Differentiate between the Taste of Commercial Breakfast Drinks. PLoS ONE 2014, 9, e93823. [Google Scholar] [CrossRef]
- Horska, E.; Bercik, J.; Krasnodebski, A.; Matysik-Pejas, R.; Bakayova, H. Innovative Approaches to Examining Consumer Preferences When Choosing Wines. Agric. Econ. (Czech) 2016, 62, 124–133. [Google Scholar] [CrossRef] [Green Version]
- Zhi, R.; Wan, J.; Zhang, D.; Li, W. Correlation between Hedonic Liking and Facial Expression Measurement Using Dynamic Affective Response Representation. Food Res. Int. 2018, 108, 237–245. [Google Scholar] [CrossRef]
- FaceReader Reference Manual Version 6.1. Noldus Information Technology 2015. Available online: https://student.hva.nl/binaries/content/assets/serviceplein-a-z-lemmas/media-creatie-en-informatie/media--communicatie/observatorium/factsheet-facial-coding-reference-manual.pdf?2900513938585/ (accessed on 27 January 2020).
- Beringer, M.; Spohn, F.; Hildebrandt, A.; Wacker, J.; Recio, G. Reliability and Validity of Machine Vision for the Assessment of Facial Expressions. Cogn. Syst. Res. 2019, 56. [Google Scholar] [CrossRef]
- Crist, C.A.; Duncan, S.E.; Arnade, E.A.; Leitch, K.A.; O’Keefe, S.F.; Gallagher, D.L. Automated Facial Expression Analysis for Emotional Responsivity Using an Aqueous Bitter Model. Food Qual. Prefer. 2018, 68, 349–359. [Google Scholar] [CrossRef]
- Walsh, A.M.; Duncan, S.E.; Bell, M.A.; O’Keefe, S.F.; Gallagher, D.L. Breakfast Meals and Emotions: Implicit and Explicit Assessment of the Visual Experience. J. Sens. Stud. 2017, 32. [Google Scholar] [CrossRef]
- Fuentes, S.; Gonzalez Viejo, C.; Torrico, D.D.; Dunshea, F.R. Development of a Biosensory Computer Application to Assess Physiological and Emotional Responses from Sensory Panelists. Sensors 2018, 18, 2958. [Google Scholar] [CrossRef] [Green Version]
- Samant, S.S.; Seo, H. Using Both Emotional Responses and Sensory Attribute Intensities to Predict Consumer Liking and Preference toward Vegetable Juice Products. Food Qual. Prefer. 2019, 73, 75–85. [Google Scholar] [CrossRef]
- Yu, C.; Ko, C. Applying FaceReader to Recognize Consumer Emotions in Graphic Styles. Procedia CIRP 2017, 60, 104–109. [Google Scholar] [CrossRef]
- Gunaratne, N.M.; Fuentes, S.; Gunaratne, T.M.; Torrico, D.D.; Ashman, H.; Francis, C.; Gonzalez Viejo, C.; Dunshea, F.R. Consumer Acceptability, Eye Fixation, and Physiological Responses: A Study of Novel and Familiar Chocolate Packaging Designs Using Eye-Tracking Devices. Foods 2019, 8, 253. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Bartkiene, E.; Steibliene, V.; Adomaitiene, V.; Juodeikiene, G.; Cernauskas, D.; Lele, V.; Klupsaite, D.; Zadeike, D.; Jarutiene, L.; Guiné, R.P.F. Factors Affecting Consumer Food Preferences: Food Taste and Depression-Based Evoked Emotional Expressions with the Use of Face Reading Technology. BioMed Res. Int. 2019, 2019. [Google Scholar] [CrossRef] [PubMed]
- Garcia-Burgos, D.; Zamora, M.C. Exploring the Hedonic and Incentive Properties in Preferences for Bitter Foods via Self-Reports, Facial Expressions and Instrumental Behaviours. Food Qual. Prefer. 2015, 39, 73–81. [Google Scholar] [CrossRef] [Green Version]
- Prescott, J. Comparisons of Taste Perceptions and Preferences of Japanese and Australian Consumers: Overview and Implications for Cross-Cultural Sensory Research. Food Qual. Prefer. 1998, 9, 393–402. [Google Scholar] [CrossRef]
- Kerrihard, A.L.; Khair, M.B.; Blumberg, R.; Feldman, C.H.; Wunderlich, S.M. The Effects of Acclimation to the United States and Other Demographic Factors on Responses to Salt Levels in Foods: An Examination Utilizing Face Reader Technology. Appetite 2017, 116, 315–322. [Google Scholar] [CrossRef]
- Torrico, D.D.; Fuentes, S.; Gonzalez Viejo, C.; Ashman, H.; Gunaratne, N.M.; Gunaratne, T.M.; Dunshea, F.R. Images and Chocolate Stimuli Affect Physiological and Affective Responses of Consumers: A Cross-Cultural Study. Food Qual. Prefer. 2018, 65, 60–71. [Google Scholar] [CrossRef]
- Skiendziel, T.; Rösch, A.G.; Schultheiss, O.C. Assessing the Convergent Validity between the Automated Emotion Recognition Software Noldus FaceReader 7 and Facial Action Coding System Scoring. PLoS ONE 2019, 14. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- FaceReader Output Visualization. Available online: http://www.vicarvision.nl/products/facereader/#outputvisualization (accessed on 5 February 2020).
- Noldus: FaceReader Benefits. Available online: https://www.noldus.com/facereader/benefits (accessed on 5 February 2020).
- Lewinski, P.; den Uyl, T.M.; Butler, C. Automated Facial Coding: Validation of Basic Emotions and FACS AUs in FaceReader. J. Neurosci. Psychol. Econ. 2014, 7, 227–236. [Google Scholar] [CrossRef] [Green Version]
- Stöckli, S.; Schulte-Mecklenbeck, M.; Borer, S.; Samson, A.C. Facial Expression Analysis with AFFDEX and FACET: A Validation Study. Behav. Res. Methods 2018, 50, 1446–1460. [Google Scholar] [CrossRef] [PubMed]
- Juodeikiene, G.; Zadeike, D.; Klupsaite, D.; Cernauskas, D.; Bartkiene, E.; Lele, V.; Adomaitiene, V. Effects of Emotional Responses to Certain Foods on the Prediction of Consumer Acceptance. Food Res. Int. 2018, 112, 361–368. [Google Scholar] [CrossRef]
- Rocha, C.; Lima, R.C.; Moura, A.P.; Costa, T.; Cunha, L.M. Implicit Evaluation of the Emotional Response to Premium Organic Herbal Infusions through a Temporal Dominance Approach: Development of the Temporal Dominance of Facial Emotions (TDFE). Food Qual. Prefer. 2019, 76, 71–80. [Google Scholar] [CrossRef]
- Zhi, R.; Cao, L.; Cao, G. Asians’ Facial Responsiveness to Basic Tastes by Automated Facial Expression Analysis System. J. Food Sci. 2017, 82, 794–806. [Google Scholar] [CrossRef]
A Sampling of Commercially Available AFEA Software | ||
---|---|---|
Affectiva [9] | CrowdEmotion [10] | EmoVu [11] |
Face++ [12] | FaceReader [13] | FacioMetrics [14] |
Findface [15] | iMotions [16] | Insight SDK [17] |
Kairos [18] | Nviso [19] | Observer [20] |
Project Oxford [21] | Realeyes [22] | SkyBiometry [23] |
Source of Food Emotion | Example |
---|---|
Sensory attributes | Sweetness of beverage |
Experienced consequences | Relief of thirst |
Anticipated consequences | Health effects associated with soda |
Personal or cultural meanings | Root beer reminds me of childhood |
Actions of associated agents | Contempt towards those that consume water from disposable plastic bottles |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Kessler, S.J.; Jiang, F.; Hurley, R.A. The State of Automated Facial Expression Analysis (AFEA) in Evaluating Consumer Packaged Beverages. Beverages 2020, 6, 27. https://doi.org/10.3390/beverages6020027
Kessler SJ, Jiang F, Hurley RA. The State of Automated Facial Expression Analysis (AFEA) in Evaluating Consumer Packaged Beverages. Beverages. 2020; 6(2):27. https://doi.org/10.3390/beverages6020027
Chicago/Turabian StyleKessler, Samuel J., Funan Jiang, and R. Andrew Hurley. 2020. "The State of Automated Facial Expression Analysis (AFEA) in Evaluating Consumer Packaged Beverages" Beverages 6, no. 2: 27. https://doi.org/10.3390/beverages6020027