Review Reports - Neuromarketing in the Digital Age: The Direct Relation between Facial Expressions and Website Design

Round 1

Reviewer 1 Report

Please see the attached.

Comments for author File: Comments.pdf

Author Response

Thank you very much.

Reviewer 2 Report

Brief summary:
The authors used two methods (eye tracking and facial expression analysis) to investigate the effects of webpage design (dynamic versus static design) on people’s visual attention and facial expressions.

General comments:
Although I think the study is interesting and has a potential for a relevant scientific contribution in the area of marketing, I noticed some serious problems with the description of the experiment and also with the statistical significance of the interpreted results. Moreover, it seems that the authors did not perform a final check of the manuscript before the submission (missing in-text references, text duplication etc.)
In order to make the manuscript easier to read, I suggest using a standard structure of the research paper (as described in Instructions for Authors - Research Manuscript Sections, https://www.mdpi.com/journal/applsci/instructions).

Specific comments:
The main concern:
The methodology has to be described with sufficient detail to allow others to replicate it. What questions were used in the survey? What was the participants’ task? Duchowski (cited by the authors) emphases that in eye tracking experiments the task need to be defined.

Title:
Title “Neuromarketing in the Digital Age: the Direct Relation between Emotions and Website Design” is misleading. In order to reflect the real contribution of the study, I recommend deleting the word “direct” or changing “emotions” to “facial expressions”.

Keywords:
Due to the serious problems with the statistical analysis of the eye tracking data, “statistics” is not an appropriate keyword for this article.

Introduction:
Introduction is well-written. However, the first paragraph should be shorter and more concise.

After mentioning the abbreviation (UX) for the first time in Introduction, it should be used consistently throughout the manuscript.

LINE 114: It is not clear what the authors mean by “to recover their emotions”.

LINES 121-122: The authors say “We analyze the perception, appreciation, and emotions about the coffee brand.”, but they analyzed only visual attention and facial expressions while viewing the webpage designs. Emotions regarding a brand are not the same thing as facial expressions recorded during the webpage presentation.

Section 1.1. Related Work
This section contains many vague parts. The whole section should be written more concise and all the mentioned works (and concepts) should be clearly related to the research problem in this study.
LINES 137-139: Without an explanation of how the reference is related to this study, it seems like an unnecessary text.
LINES 141-142: The statement that people pay more attention to text than picture is still debatable. If the authors consider this to be undoubtedly true fact, then they should support it with more research-based references.
LINES 142-144: The sentence “The work of Wall et al. deals with…” is unclear. Consider rewriting it.
LINE 150: It is not clear who are the authors that the beginning of the sentence refers to.
LINES 162-167: It is not clear why mentioning color is relevant for this study. Please explain.
LINES 192-193: The sentence “It is here that in our work we corroborate the outcomes obtained by the software with respect to a survey elaborated under the same parameters of observation. “ should be written more clearly, and contain a comprehensible description of the survey.

Section 1.2. Study
LINES 224-229: The authors say “the experiment was designed to accomplish 3 goals”. However, developing marketing material (actually, it was stimuli design), recruiting participants and acquiring the measuring device – these are not research goals. These are an essential research plan and necessary methodology.

Table 1. It is not mentioned in the text of the manuscript.

LINE 249: The authors should justify why the participants’ age range was so small, and does this limitation have an impact on the interpretation of the results.

2.1. Hypotheses development and motivation
The hypotheses seem to be unappropriated. They are not fully in accordance with the first sentence in the Discussion (LINES 543-544).
The authors use “quality of interaction” as independent variable in H1, but it was not measured (or described as such) in other parts of the paper.

2.2. Web Design and 2.3. Eye tracking
Many references are missing.
Both sections should be shorter and more concise.

3. Results
The authors report different viewing distances; 60 cm (LINE 364) and 50 cm (LINE 372).

Figures 4, 5, and 6 can be presented as one figure which shows three frames with exact time line. This could make the experimental procedure repeatable.

LINE 392: The authors say that the participants answered the survey, but there is no description of the survey.
LINE 396: The description of the post-survey is missing.

LINES 411-413: This is an important part of the paper. The explanation of the hotspots order should be written more clearly.

LINES 473-501:
Statistical tests are missing. In order to draw meaningful interpretations of research findings, the results have to be analyzed with standard statistical tests. Reports such as “it seems to be somewhat related”, “no notable difference in terms of metrics”, “a clear difference (...) with 4.6 against 3” etc. cannot be considered as valid scientific conclusions.

LINE 534 and 540: The authors mention hypothesis H3. This is confusing, since only H1 and H2 are defined in the previous section.

4. Discussion:
LINES 547-560: The text is duplicated later (LINES 578-591).
LINES 565-570: The authors repeated the exactly the same text which was already written in other parts of the paper.
LINES 591-594: The conclusion is too broad. The experiment was not sufficiently described to report on direct impact of the webpage design and participants’ emotional state.
LINES 599-602: The authors say that the objective of their experiment was “to understand why Augmented Reality seems to be attractive at first instance, but then people seem to get tired of it.” This description of the research objective is not in line with the conducted experiment and the hypotheses.
LINES 603-604: The statement “The correct use of technology-based techniques has been shown in this study to overcome certain gaps in research findings when it comes to understanding human behavior.” has to be supported by arguments (or explained better) to be meaningful.
LINE 628: The authors base their description of the study’s aim on “experience with the brand”, but only experience with the webpage of the brand was investigated.

Other minor changes:
The reference list should be written more carefully.

Author Response

Reviewer 2 comments

Comment 1: The methodology has to be described with sufficient detail to allow others to replicate it. What questions were used in the survey? What was the participants’ task? Duchowski (cited by the authors) emphases that in eye tracking experiments the task need to be defined.

Response:

Thank you very much for your comment, it is true that more information regarding the questions was needed. To follow up this comment we have added two new table to the paper in which we establish the questions of the surveys and a brief description as follows:

“To accompany the objective approach of the study—data collected through software—we conducted a survey on the users at the end of the interaction with each scenario of the web page with the questions listed in Table 2. Which means first one webpage was presented and when it was finished the post-survey was answered, then the second webpage was presented and when it was finished the post-survey was answered. All the questions were presented on a single page on the screen and each one had a slider with the range listed in Table 2. There was no time limit or fixed order to answer them, all the answers were sent at once when the user clicked the Continue button. This survey reflects the main usability metrics for the web page, giving insight of how the user perceives the design and interacts with it. It is a qualitative measure of the UX that can be used as feedback of how good or bad was the perception of the user. With this activity, we obtain advantages such as describing if a design helps or hinders the UX, checking if the designs improve over time, objectively comparing the UX concerning the competition, comparing the UX to industry standards, and getting an idea about what could happen if we implement the new design. Then after interacting with both designs and answering both post-surveys a similar final survey was presented with the questions listed in Table 3. The direct intention of said survey was to obtain the opinion of the user after comparing both website designs and report if there was any improvement between designs.”

Along with that we added the next text to better define the participants task in terms of its interaction with the website menu design that was analyzed with the eye-tracking technique:

“Two homepages designs were evaluated. The participants could view each homepage for 2 minutes, surf through said page by scrolling up and down. In the case of the preview location element inside the new design the user could click and drag the interactive image to visualize different points of view for said location of the company branch. Others homepage elements in both designs were not interactive, all buttons that redirect to any other page were disable preventing from leaving the homepage for complete interaction with the design. This means that users could only interact within the presented home or new homepage design. In order to align the starting point of visual attention in both websites, respondents were asked to fixate on a fixation- cross which is presented for two seconds before the start of each homepage. At the end of the interaction with the homepages users answered the survey using an ordinary computer mouse. Each experiment took about 15 minutes on average to complete.”

Comment 2: Title “Neuromarketing in the Digital Age: the Direct Relation between Emotions and Website Design” is misleading. In order to reflect the real contribution of the study, I recommend deleting the word “direct” or changing “emotions” to “facial expressions”.

Response:

Thanks to the Reviewer's comment for pointing out the possible misleading factor in the title therefore, as recommended, we have changed the word “emotions” to “facial expressions” of the original title. Our new title now is “Neuromarketing in the Digital Age: the Direct Relation between Facial Expressions and Website Design”.

Comment 3: Due to the serious problems with the statistical analysis of the eye tracking data, “statistics” is not an appropriate keyword for this article.

Response:

We appreciate the Reviewer comment on what keywords should be included in the paper thus we have removed the statistics keyword of the listing.

Comment 4: Introduction is well-written. However, the first paragraph should be shorter and more concise.

Response:

We thank the Reviewer for the comment and, considering the advice, the first paragraph of the introduction has been revised and reduced to the following text:

“Throughout human history we have tried to answer every question we have encountered but key questions that are so ingrained in human nature like why we are here, what drives our actions and how to interpret human behavior are too complex to be answered in a lifetime of study. The complexity of these questions lies in the axiom that each human being is a different world and in the interaction of the human spectrum: actions, cognition and emotions. Despite substantial recent progress, our understanding of the principles and mechanisms underlying complex brain function and cognition remains incomplete. That is where neuroscience comes in with a very direct focus on answering these complex questions through the stimulation and observation of the human brain “

Comment 5: After mentioning the abbreviation (UX) for the first time in Introduction, it should be used consistently throughout the manuscript.

Response:

Thanks to the reviewer’s comments for pointing out that the abbreviation usage should be consistent. Therefore all further mentions, here listed, of User Experience were changed to the correspondent abbreviation UX:

LINE 155: “In [ 19 ] they describe the importance of the UX and how to optimize their perception against”

LINE 207: “UX is a key basis for achieving products that satisfy the wishes and needs of the user that is”

LINE 281: “The UX covers all aspects of the interaction of the end user with the company, its service, and”

Comment 6: LINE 114: It is not clear what the authors mean by “to recover their emotions”.

Response:

Thanks to the reviewer for making us aware of the uncertainty with the sentence: “More specifically, we present the design of a website to a consumer to recover their emotions and we will analyze the reactions of said consumer to try to model a stimulus-reaction relationship”. The latter was edited to this new sentence we consider more appropriate: “More specifically, we present the design of a website to a consumer, we measure the emotions provoked by this stimulus to analyze these consumer reactions to try to model a stimulus-reaction relationship”

Comment 7: LINES 121-122: The authors say “We analyze the perception, appreciation, and emotions about the coffee brand.”, but they analyzed only visual attention and facial expressions while viewing the webpage designs. Emotions regarding a brand are not the same thing as facial expressions recorded during the webpage presentation.

Response: We thank the reviewer for the comment. The mentioned text was edited to clarify the study’s analysis:

“We analyze the perception, attention levels, and emotions about the coffee brand through two approaches: an eye-tracking technique that directly reports what the user is seeing and a survey that reports what the user perceives and how it is perceived.”

Comment 8: Section 1.1. Related Work This section contains many vague parts. The whole section should be written more concise and all the mentioned works (and concepts) should be clearly related to the research problem in this study.

Response: Thank you very much. We have rigorously corrected the Related Work section and have placed the paragraphs consistently.

Comment 9: LINES 137-139: Without an explanation of how the reference is related to this study, it seems like an unnecessary text.

Response: We thank the Reviewer for pointing out the need for relevance to the sentence, we have reviewed and decided to remove the text as it was in fact unnecessary.

Comment 10: LINES 141-142: The statement that people pay more attention to text than picture is still debatable. If the authors consider this to be undoubtedly true fact, then they should support it with more research-based references.

Response: We thank the Reviewer for the observation and we added the following sentence to the text:

“which is still debatable and in fact was one of the motivations for this research, since we compared two webpages: one with text about the brand and the second with images that refer to the brand”

Comment 11: LINES 142-144: The sentence “The work of Wall et al. deals with…” is unclear. Consider rewriting it.

Response: We thank the Reviewer for the comment. The sentence referenced in the comment was changed to the following:

"The work of Wall et al. shows that online platforms and e-commerce sites guiding the customer in their purchase helps close the sale. The authors study how people form first impressions online; furthermore, this perspective matches the interest of our paper on examining the factors that influence online platforms. Here a question arises: are users of a platform correctly receiving the message that the system is trying to convey? In other words, there are theoretical implications of how we behave online and how others use this behavior to form judgments of perception, attention, and information gathering, which can boost or reduce sales [17]."

Comment 12: LINE 150: It is not clear who are the authors that the beginning of the sentence refers to.

Response: We thank the Reviewer for the observation. To make clear the reference we decide to change the beginning of the sentence with the following: “In [18] the authors”

Comment 13: LINES 162-167: It is not clear why mentioning color is relevant for this study. Please explain.

Response: We thank the Reviewer for the comment. Color is important in the study because on the second version of the website it is one of the parameters we took into account. On the "Scenario Design" section we explain a little further "For the old website the color scheme is strictly composed by the company colors: red, black and white. As Gelasca et al [42] states, the color red has a high index of prominence and this, together with the size/quantity, aims to capture the user's attention."

Comment 14: LINES 192-193: The sentence “It is here that in our work we corroborate the outcomes obtained by the software with respect to a survey elaborated under the same parameters of observation. “ should be written more clearly, and contain a comprehensible description of the survey.

Response: We thank the Reviewer for the observation. In order to solve the comment we changed the text with the following:

“That is why in our work we corroborate the metrics obtained by the software with a subsequent survey that collects the user's opinion regarding the website, thus allowing a more complete analysis of the interaction considering both factors: objective and subjective.“

And added the surveys description along with two tables with the questions:

Comment 15: LINES 224-229: The authors say “the experiment was designed to accomplish 3 goals”. However, developing marketing material (actually, it was stimuli design), recruiting participants and acquiring the measuring device – these are not research goals. These are an essential research plan and necessary methodology.

Response: We thank the Reviewer for the observation. We updated the text to:

“Based on that, the experiment was designed with 3 main steps: the first one was to develop some marketing material that could be shown to a panel of participants. The second step was to recruit people ready to be part of the experiment. And finally, the third step was to acquire a device that helped to measure emotions of the participants on different instances of the experiment. It is important to mention that the following materials developed were based on a Mexican coffee brand called “La Borra del Café” so this project specifically is about a coffee brand menu.”

Comment 16: Table 1. It is not mentioned in the text of the manuscript.

Response: Thank you for letting us see this mistake.

The Table 1 comprises several elements that are part of the appreciation of an interface [29]. We describe the main elements that directly impact the emotions of users. Color parameters, shapes, intensities, contrasts, movements, and repetitions influence us. These elements make up (some to a greater extent than others) virtual and augmented reality interfaces, which we want to implement in this study.

We have added the reference:

Kalkova, N.; Reutov, V.; Mitin, E.; Velgosh, N. Neuromarketing study of con cognitive perception of labeling Information on a prod package. In Proceedings of the International Scientific Conference" Far East Con"(ISCFEC 2020). Atlantis Press, 2020, pp. 3029–3036.

Comment 17: LINE 249: The authors should justify why the participants’ age range was so small, and does this limitation have an impact on the interpretation of the results.

Response: We thank the Reviewer for the observation. We updated section “1.2 Study” to include:

“Mainly focused on college students to understand better how to improve for that market.”

Taking into account that familiarity with the product can influence the intensity of facial microexpressions, it was considered pertinent to study the generation of young coffee consumers, who have shown an increase in the consumption of the category. According to a survey published in January 2022 by the National Coffee Association (NCA), consumption among 18-24 years has risen 14% in the last year (National Coffee Association of U.S.A, 2022) [29].

National Coffee Association of U.S.A. (2022). National Coffee Data Trends 2022. https://www.ncausa.org/Research-Trends/Market-Research/NCDT

Comment 18: The hypotheses seem to be unappropriated. They are not fully in accordance with the first sentence in the Discussion (LINES 543-544). The authors use “quality of interaction” as independent variable in H1, but it was not measured (or described as such) in other parts of the paper.

Response: Many thanks to the Reviewer for pointing this out. The Reviewer is correct, and the hypotheses were not well aligned with the results. We have changed the two hypotheses and adapted them to the discussion.

H1. The positive appreciation of a user or potential client depends on the interactivity of the web page's interface using different methods of observation and collection of comments.

H2. There are specific facial expressions when a user manipulates websites impacting in his/her emotions.

Comment 19: 2.2. Web Design and 2.3. Eye tracking: Many references are missing. Both sections should be shorter and more concise.

Response: We thank the Reviewer for the observation. We have corrected the references and reduce both sections:

“Web design is a cornerstone of the digital presence for a company. Depending on how the site is planned and executed the user can benefit from responsive items and easy to use solutions. Therefore, the quality of a company’s website is important to attract customers and improve the perceived quality of the company itself as well to boost the trust on the brand [29]. Unfortunately, web design is cumbersome, and the lack of a unified standard can create different interactions and experiences. One possible challenge comes from the diversity of devices that can browse the Internet with different aspect ratios and visualization performance. Therefore, the design has to carefully take into account and prioritize which aspect ratios are most likely to amount the majority of access to the site. Then the consideration of different strategies can be used based on the contents that a website allows, for instance, the combination of text, pictures, graphics, animations and interactive elements. The right combination of the before mentioned tools can boost the attractiveness and retain the attention of users for more time [30]. On the last decade, there has been an enormous interest in allowing more kinds of content to be available online and to improve the general experience of a user. Among the additions are video codecs like h.264 and hevc, that allow high quality videos to be streamed directly to the browser so the customer can enjoy the content as it is intended [31]. Also, the improving of stitching algorithms [32] have allowed the creation of panorama images, which are based on taking a series of photos and then putting them together to form a spherical image. Ultimately, a website or app can apply all the technologies mentioned before and still not capture the attention of the user but if all stimuli are correctly set and placed then the user could have an improved experience like TikTok [33]”

“Eye tracking technology uses high-speed cameras to track the movement of eyeballs, pupil dilation (pupilometry), and subject flicker, among other factors. It is a technique by which eye movements of an individual are measured. By using eye-tracking technology, a researcher can know what a person is looking at each moment and the sequence in which his/her eyes move from one place to another one. Tracking users’ eye movements can help to understand the processing of visual information and factors that may have repercussions on the usability of an interface. Therefore, eye movements recordings can provide an objective data source for the evaluation of interfaces, which provide information to improve the design. Eye tracking has taken tremendous steps to overcome many of the original issues it had. Eye tracking is based on recognizing the orientation of the head and finding the position of the eyes to determine the direction of the vision. This algorithm has to be done in milliseconds to get enough information about how the eyes rapidly change focus and to understand the reactions of the customer. This kind of tools allows the user to avoid touching a device, which can be helpful when the user does not have a limb. Eye tracking has been used to help from medicine [34] to marketing [35]. Regrettably, every technology has advantages and disadvantages, [36] has found that when a user is unrestrained, and it is not trained on how to use an eye tracker, then it is highly possible to get inaccurate data. Specifically when only one eye is visible but the system still believes that it is correctly tracking two eyes. In other words, researchers have to put extra attention on how they explained the correct posture during a session to analyze data that is more likely to be accurate. One important thing to mention is this kind of device purpose is to gather information. Afterwards, the researcher can use other software to analyze the data and reach conclusions.”

Comment 20: The authors report different viewing distances; 60 cm (LINE 364) and 50 cm (LINE 372).

Response: We have corrected the distance measurement by adding the following paragraph:

For optimal eye tracking the distance from the participant's eyes to the eye tracker should be approximately 60–65 cm (23–26″) for most screen based eye trackers from Tobii Pro.

Comment 21: Figures 4, 5, and 6 can be presented as one figure which shows three frames with exact time line. This could make the experimental procedure repeatable.

Response: We thank the Reviewer for comment, following the advice a new figure was presented as shown below:

Comment 22: LINE 392: The authors say that the participants answered the survey, but there is no description of the survey.

Response: We thank the Reviewer for pointing out this aspect. We have added Table 2, related to the survey carried out on users at the end of viewing the two scenarios of the web page. This survey reflects the main usability metrics for a web page. We qualitatively measure the user experience. With this activity, we obtain advantages such as describing if a design helps or hinders the user experience, checking if the designs improve over time, objectively comparing the UX concerning the competition, comparing the UX to industry standards, and getting an idea about what could happen if we implement a design.

Comment 23: LINE 396: The description of the post-survey is missing.

Response: We thank the Reviewer for the comment and considering it we have added the post-survey description:

Comment 24: LINES 411-413: This is an important part of the paper. The explanation of the hotspots order should be written more clearly.

Response: We thank the Reviewer for the comment and following the advice the mentioned text was replaced with the next:
“These hotspots were ordered by the Ratio metric—count of users who fixed on that area over the total number of users—of the corresponding AOI in Figure 1 to give an importance index based on user attention levels. Note that the Menu button was not an AOI set within the studio; however, it is a hotspot within the heat map so to diminish its importance it was decided to place it in the last position. For all other hotspots the AOI description was listed for better reference.”

Along with this change, the counterpart explanation for on-stimulus key fixation points had the next change:

LINES 454-458: “These hotspots were ordered in the same way as their off-stimulus analysis counterpart, but according to their own corresponding AOI in Figure 2 Note that the Liquid coffee within animation was not an AOI set within the studio; however, it is a hotspot within the heat map so to diminish its importance it was decided to place it in the last position. For all other hotspots the AOI description was listed for better reference.”

Comment 25: LINES 473-501:

Statistical tests are missing. In order to draw meaningful interpretations of research findings, the results have to be analyzed with standard statistical tests. Reports such as “it seems to be somewhat related”, “no notable difference in terms of metrics”, “a clear difference (...) with 4.6 against 3” etc. cannot be considered as valid scientific conclusions.

Response: We thank the Reviewer for the comment that it is entirely correct, one of the purposes of the study is to provide evidence of a quantitative difference that is why the text was edited and multiple tables and figures were added as it follows:

“The latest analysis is a more visual and intuitive approach that needs to be accompanied by quantitative analysis to provide hard facts. Table A3 is sorted ascending by Time To First Fixation (TTFF) and as shown in Figure 7 it is correlated by a -0.93 factor to Ratio, as for faster TTFF time, higher Ratio. As seen in the heatmap analysis, the most visited AOI is the Brand slogan in white font with a Ratio of 31 users out of 39 total users and it is also the most visited by 25 users out of 31 users who fixated. This particular AOI stands apart from the others by far as shown in Table 5, with at least double in all metrics except Revisits to the closest, indicating high attention dominance.”

“Table A4 is ordered as the counterpart analysis of the off-stimulus and similar to its analysis in this case there is a direct relation between TTFF and Ratio metrics with the same behavior—inversely proportional—but with a correlation factor of -0.97 as shown in Figure 8, indicating a higher correlation than before. Analyzing the percentage difference between the first and second AOI in Table 6 we can see that for the Ratio, Revisits and Fixations metrics the change between the two is very low which could indicate a good attention intensity result for both components. The domain of this pair of components seen in 6 and 7 is greater than that seen in the other design, except TTFF and Revisits, which could be interpreted as a higher level of attention for both AOIs.”

“As stated before, the closeness between the components could result in more attention as seen for those components closest to the first AOI in both designs, yet there is still a higher percentage difference for the second AOI of the off-stimulus against on-stimulus that can be seen comparing Table 5 and Table 6. This can be interpreted as a higher prominence for the second AOI in the new design versus the second AOI in the first design. Now, if we compare the first AOI in the new design to the first AOI in the old design as seen in Table 8, the percentage difference for the metrics Ratio, Revisitors and Fixations is lower than 10% which indicates minimum change, but when comparing the second AOI in the old design to the new design, the percentage difference comparing Time Spent and Fixations passes the 100%. Now comparing the components that remained between designs for a percentage decrease in Table 9 we can see for the AOI Brand name in white font there is a decrease in the metrics Ratio, Revisitors and Fixations, and for Brand name in white font in company sign there is a decrease in the metrics Ratio, Revisits and Fixations. This impact on the AOIs can be interpreted as a lack of prominence or user appeal that is overshadowed by the high salience of the animation and location preview.”

Comment 26: LINE 534 and 540: The authors mention hypothesis H3. This is confusing, since only H1 and H2 are defined in the previous section.

Response: We thank the Reviewer for the comment. We have changed the name for the alternative hypotheses in the Results section, in text and Figure 12, to avoid confusion as follows:

H₁ -> H_a1

H₂ -> H_a2

Comment 27: LINES 547-560: The text is duplicated later (LINES 578-591)..

Response: We thank the Reviewer for pointing out the duplicated text. Changes were made to have only one occurrence in LINE 571-573

Comment 28: LINES 565-570: The authors repeated the exactly the same text which was already written in other parts of the paper.

Response: We thank the Reviewer for the observation, said lines were removed.

Comment 29: LINES 591-594: The conclusion is too broad. The experiment was not sufficiently described to report on direct impact of the webpage design and participants’ emotional state.

Response: We thank the Reviewer for the observation and following the advice the conclusion was edited into the next text to better describe the experiment results:

“In the second hypothesis we established that psychological factors play an important role in the interplay between website and user and even though the psychological spectrum is very broad the results of the comparison between the interaction off-stimulus vs on-stimulus demonstrated that the website design can indeed directly influence in a very concrete area: emotions. This can be seen in the following basic emotions: joy, anger, surprise, fear and disgust, that is to say that in five of the seven basic emotions there was a significant difference, more specifically a higher percentage of observation time for these emotions indicating that for most emotions the difference is statistically proved.”

Comment 30: LINES 599-602: The authors say that the objective of their experiment was “to understand why Augmented Reality seems to be attractive at first instance, but then people seem to get tired of it.” This description of the research objective is not in line with the conducted experiment and the hypotheses.

Response: We thank the Reviewer for the observation. We realized that the text was not descriptive enough to understand where AR was involved within the study and therefore hypotheses, the following text was added to complement what was intended with the AR element:

“This was applied directly in the study through the animated image which was part of the second design of the website thus allowing a comparison between designs, one with an AR element and the other without AR. The animated image consisted of capturing an AR interaction between a cup of coffee and virtual coffee.”

Comment 31: LINES 603-604: The statement “The correct use of technology-based techniques has been shown in this study to overcome certain gaps in research findings when it comes to understanding human behavior.” has to be supported by arguments (or explained better) to be meaningful..

Response: We thank the Reviewer for the comment. A better explanation was added to make the statement more meaningful:

“This can be seen directly in the results of heatmaps and fixation metrics, returned by the eye-tracking device, which provide concise and time-based results on what grabs the user's attention and how they interact with the website compared to a human observation approach which relies on the ability of the observer to obtain the results which could lead to possible bias or incomplete metrics.”

Comment 32: LINE 628: The authors base their description of the study’s aim on “experience with the brand”, but only experience with the webpage of the brand was investigated.

Response: We thank the Reviewer for the observation and following the advice the next text was added at the end of sentence to specify the scope of the study:

“by observing and measuring one of the brand's key components: the website”

Comment 33: The reference list should be written more carefully.

Response: We thank the Reviewer for the observation. We have rigorously corrected the Related Work section and this has improved the organization of the references.

We added 2 more references in the Related Work section:

Filipovic, F.; Baljak, L.; Naumovi ´ c, T.; Labus, A.; Bogdanovi ´ c, Z. ´ Developing a web application for recognizing emotions in neuromarketing. In Marketing and Smart Technologies; Springer, 2020; pp. 297–308.
Burgos-Campero, A.A.; Vargas-Hernández, J.G. Analitical approach to neuromarketing as a business strategy. Procedia-Social and Behavioral Sciences 2013, 99, 517–525.

Author Response File: Author Response.pdf

Reviewer 3 Report

l It seems structurally correct to include the description of the experiment in Section 2, not in Section 1.2.

l Because the research hypothesis is abstract, it is not directly mapped with the actual experimental content, and the experimental results seem difficult to test this hypothesis accurately.

l Does Section 2.2 need to be included in this manuscript?

l In Section 2.3, describe the equipment used in this study. If the authors used Tobii, only need to explain it. And clarify whether or not AR equipment was used in this study. Also, delete the description from 1.2 if not used.

l Explain how the scenarios described in Section 3.1 relate to the hypotheses.

l For statistical results, attach p-values and other important figures, not test results.

Author Response

Dear

Editor

MDPI Applied Sciences

We are submitting the paper:

“Neuromarketing in the Digital Age: the Direct Relation between Facial Expressions and Website Design”

Authored by: Guillermo González-Mena, Carolina Del-Valle-Soto, Violeta Corona and Jafet Rodríguez.

We would like to thank the reviewers and editors for their detailed analysis of the manuscript; the comments are very valuable to us. In the revised version of the paper, we have incorporated all changes recommended by the reviewers.

Comments to all observations and suggestions including point-by-point responses are addressed in the following text.