Decision-Making for Product Form Image Based on ET-EEG Technology

Abstract

The use of neurophysiological data to acquire product image avoids the inherent subjectivity in the empirical design process. In this paper, we use eye tracking–electroencephalography (ET-EEG) to study mapping among user behaviour, eye movements, EEG features and image decisions in the process of product form cognition. First, we designed an ET-EEG experiment using hair dryer stimuli, with morphology rated on a Likert scale. Then, ET-EEG data were categorized according to image category: “unambiguous (matching)”, “unambiguous (mismatching)”, “ambiguous (positive)”, or “ambiguous (negative)”. Finally, the behavioural data, eye movement indicators, and event-related potentials (ERPs) were analysed to parse the cognitive features. The behavioural, ET and ERP data were highly consistent, and their values increased as the cognition resources devoted to decision-making increased. ET-EEG physiological data thus objectively and effectively reflected users’ image cognition of products, providing theoretical support for design research on the origin of cognition.

1. Introduction

Kansei engineering involves the mapping of human perceptions of objects [1], constructed by feedback regarding thoughts and emotions related to the product [2]. Visual aspects of products (e.g, colour, morphology and other visual elements) are first processed in perceptual regions of the brain during image acquisition, which is followed by image cognition and image evaluation of the product. The user’s Kansei values serve as key data that determine subsequent design work [3]. To improve design accuracy, it is necessary to ensure the validity and accuracy of product image evaluation data as much as possible in the image acquisition stage. Currently, user image studies mainly utilize the explicit measurement method (EMM) and implicit measurement method (IMM) to obtain product image values [4]. The former consists of subjective psychometric methods used to obtain the user’s image cognition of the product [5], while the latter obtains image values indirectly through objective physiological data such as electroencephalography (EEG) signals [6], eye movements [7], and blood oxygen content [8].

Psychometric methods mainly involve situational interviews [9], questionnaires [10], and semantic differences used to obtain user image evaluations of products through their subjective and controllable conscious behaviours [11], such as language, actions, and facial expressions. For example, Zhang et al. [12] applied the expert interview method and KJ method to obtain a representative product and adjectives related to Audi A4L car headlights in a multicriteria decision-making study of product form. Cheng et al. [13] used the semantic difference method to obtain a representative product and adjectives related to electric vehicles. Zeng et al. [14] used a subjective evaluation method to obtain a representative product and adjectives related to vases. However, experts are not the end users; therefore, their evaluation of the product cannot accurately reflect the real needs of users. Additionally, explicit behaviours observed after subjective processing are easily influenced by factors such as life experience and educational background, resulting in the inability to represent users’ potential assessments and affecting the accuracy of subjective psychometric results truly and accurately.

In recent years, research on the use of physiological features for design decision-making has been fruitful. Physiological data reflect users’ unconscious and involuntary behaviours [15], and researchers can assess product image acquisition by measuring and analysing subjects’ physiological data [16].

Of external information available to people, 80% is visual input [17]; visual stimuli undergo substantial processing in the brain. Therefore, eye tracking data are an effective reflection of user cognition. Ho et al. [18] assessed the emotional valence of products according to changes in pupil diameter, with positive and neutral product images evoking greater pupil dilation than negative product images. Li et al. [19] applied a psychological perspective, collecting data on four eye movement indicators, including the latency and duration of the first fixation, total fixation count, and average pupil diameter; these indicators were weighted with the comprehensive weighting method to obtain user evaluations. Qu et al. [20] found that subjects had an increased number of fixations, longer fixation durations and larger pupil diameters for pleasant and neutral products than for unpleasant products. Compared to men, women had larger pupil diameters when viewing pleasant and neutral products and smaller pupil diameters when viewing unpleasant product images. Wang et al. [21] eye tracking was used to collect eye movement data from designers reading task-related text and sketching; designer experience was directly proportional to the number, characteristics, and durations of regressions (looking backwards in the text) as well as the quality of the sketch and inversely proportional to the working time and evaluation accuracy.

EEG reflects the neural activity of the cerebral cortex, and its high temporal resolution has ensured its widespread use in the field of cognitive science. Zhou et al. [22] proposed an emotion analysis model based on long short-term memory (LSTM) and EEG and applied it to parametric design. Marina et al. [23] to evaluate user food preference, a product perception method based on EEG was proposed in the transmodal taste–vision task. Zhang et al. [24] objectively evaluated the appearance design of two charging piles from the user’s point of view using EEG with the help of multiple fractal detrended fluctuation analysis (MF-DFA) method. Yang et al. [25] behavioural and EEG data were used to examine the influence of product appearance features and familiarity on product recognition. Product colour features with low and high familiarity, shape features with low familiarity and material features with high familiarity improved product recognition.

There are drawbacks to using ET or EEG techniques individually to study a user’s underlying cognitive behavior. Although the eye movement index can speculate the cognitive process of users to a certain extent, there is still a certain degree of ambiguity that cannot truly reflect the cognitive processing of users. Although EEG technology can be a good solution to this problem, it is rarely so applied. Since ET and EEG data are individually unable to detect abnormalities, a combination of the two (ET-EEG data) can be used to explore the implicit processing of Kansei cognition [26]. For example, Zhu et al. [27] proposed a product evaluation method based on the fusion of ET and EEG to reduce the subjectivity of product evaluation and improve the reliability of product evaluation using a spatiotemporal neural network and fuzzy system. Wang et al. [28] design decisions were predicted by studying the relationships of design decisions with subject eye movement and EEG data. The results showed that the integration of eye movement and EEG data better fit the expert decision results. Guo et al. [29] the visual aesthetics of the table lamp were quantified by combining EEG and ET data. Yang et al. [30] analysis of the product form design elements and users’ Kansei cognition led to the proposal of a product Kansei image prediction model based on EEG and ET physiological indices.

Based on these findings, the combined application of eye tracking and EEG provides an effective paradigm for objectively capturing users’ perceptual imagery of products through multimodal data fusion, significantly reducing subjective and empirical biases in design evaluation [31]. These studies robustly confirm the predictive validity of ET-EEG physiological features for outcome variables such as design decision prediction, aesthetic quantification, and image modeling. However, current research focus remains predominantly on correlating physiological signals with design outcomes, while systematic empirical investigation into the underlying cognitive mechanisms of Kansei imagery formation remains scarce. In this paper, we used the ET-EEG technique to investigate the relationships among subject behaviours, eye movement features, EEG features and product image decisions when viewing product images. The aims were as follows: to explore user behaviour, visual processing and processing rules during image decision-making; to provide scientific support for developing products that meet user image needs; and to obtain evaluations of product images through objective physiological data.

The outline of this study is as follows:

(1)

Four conditions were generated for product image decision-making: unambiguous stimuli (a matching/mismatching for the Kansei adjective) [32] and ambiguous stimuli (associated with the positive/negative word in an adjective pair) [33]. The stimuli were classified by a subjective evaluation method.

(2)

The ET-EEG technique was used to collect behavioural, eye movement, and ERP data in the four conditions and to quantify the relationships among behavioural data, eye movement data, ERP data and image cognition.

(3)

The processing rules of user behaviour, visual perception and cognition during image decision-making were discussed in detail.

2. Product Image Evaluation Experiment

The hairdryer was selected as the research vehicle in this experiment based on the following considerations. First, as a small household appliance frequently used in daily life, participants possess high familiarity with hairdryers, which facilitates rapid and accurate form evaluations during the experiment. Second, hairdryers exhibit distinct form features and high recognizability, and their design language can convey clear form imagery, thereby supporting effective collection of imagery perceptual data. Furthermore, hairdryer casings are typically made of a single material, which minimizes interference from material factors in morphological imagery judgments. Additionally, their functional structure is relatively simple, helping to avoid complications in imagery perception caused by functional complexity. Finally, as a representative small household appliance, hairdryers share many common characteristics with products in this category, enhancing the representativeness and generalizability of the study’s findings.

In this study, a hair dryer was used as the stimulus to construct the image space according to the Kansei evaluation of the product. Based on the behavioural data, eye movement data and EEG data of the subjects in the ET-EEG experiment, the decision-making behaviour of subjects in the image matching process was analysed. The relationships of objective data (such as decision latency, fixation count, pupil diameter, and P300 and N400 amplitudes) with product image decisions were studied under different conditions.

2.1. Construction of Kansei Image Space

Product imagery is the user’s direct association of product features, is a reproduction of the user’s cognitive experience, the product form imagery originates from the human cognition of the product form. Different form of hair dryers brings different emotional experiences to users, we collect adjectives describing the form imagery of hair dryers through multiple channels such as the Internet, advertisements, magazines, etc., to build a vocabulary database of the form imagery of the products, and then statistically screen out the representative image words after user evaluation and scoring.

We searched for adjectives to describe hair dryers and obtained 68 adjectives. To reduce the spatial dimension of the imagery and screen out the imagery words that are not related to the description of the hairdryer’s form, two doctors in industrial design and four masters in product design were invited to screen the 68 adjectives twice; First, adjectives unrelated to form description (e.g., those describing brand, material, or function) were removed, followed by the elimination of semantically redundant or ambiguous adjectives; 20 adjectives with strong correlations to the shape of a hair dryer were obtained. The similarity of these 20 adjectives was investigated with a 5-point Likert scale. Twenty similarity questionnaires were collected, and SPSS19 was used for K-means clustering. Avoiding the extremes of a single group or an excessive number of groups, the classification achieved better results with 4–8 categories of adjectives. The number of categories was set at 4 because high dimensionality of adjectives would lead to overly complex subsequent physiological experiments [34]. Four representative terms were obtained using this K-means clustering as follows: “cute”, “fashionable”, “comfortable”, and “simple”. The antonym pairs of the representative words (“dull–cute”, “traditional–fashionable”, “stiff–comfortable”, and “complicated–simple”) were used as the target word pairs in the image space. The clustering results are shown in

Table 1. Adjective clustering.

Cluster	Adjectives
1	cute	slick	fresh
2	fashionable	technological	art	cool	exquisite	strong	textured	fluent
3	comfortable	sober	elegant	traditional	secure	environmental	handsome
4	simple	convenient

.

2.2. Experimental Stimulus Collection and Image Semantic Evaluation

From the internet, e-commerce platforms and other locations, 198 pictures of hair dryers were obtained. In the initial screening process, which involved the removal of similar shapes of hair dryers and blurred images, 100 stimuli were obtained. To ensure the accuracy of the experiment and remove the influence of irrelevant factors, the logo was removed from all stimuli, images were converted to greyscale, and the size, background and resolution of all stimuli were standardized. Two individuals with industrial design doctorates and three individuals with a Master of Arts were invited to screen the stimuli again, and 50 representative stimuli were identified based on form similarity, as shown in Figure 1. An image survey using a 5-level Likert scale (−2, −1, 0, 1, and 2) to evaluate each hair dryer stimulus was distributed online. A total of 40 subjects participated in the survey, all of whom were graduate students in product design. The survey data provided the image evaluation value for each hair dryer stimulus (see

Table 2. Product image evaluation scores. (Complete data is available in Appendix A).

Stimulus Number	Dull–Cute	Traditional–Fashionable	Stiff–Comfortable	Complicated–Simple
1	−0.23	−0.15	−0.1	0.13
2	0.15	0.8	−0.05	0.83
3	0.33	−0.18	0.68	0.58
4	0.5	0.25	0.63	0.53
5	0.5	−0.1	0.63	0.8
6	0.33	1.28	−0.08	−0.08
…	…	…	…	…
43	0.95	0.55	0.4	0.6
44	0.05	0.4	0	−0.15
45	0.38	0.45	0.28	0.58
46	0.28	0.45	0.28	0.55
47	0	0.65	−0.08	0.58
48	−0.38	−0.53	0.05	0.03
49	0.68	0.43	0.2	0.53
50	0.75	0.93	0.7	0.85

).

2.3. Experimental Design

In this experiment, “dull–cute”, “traditional–fashionable”, “stiff–comfortable”, and “complicated–simple” were used as the four dimensions of the product image space, and subjects evaluated each image of a hair dryer. The match between the stimulus and the given adjective was used to divide stimuli into four conditions: unambiguous images (matching or mismatching with the adjective) and ambiguous images (aligning with the positive or negative words in an adjective pair). By analysing the behavioural data, eye movement data and EEG data of subjects in different conditions, we explored the behavioural, perceptual and cognitive processing patterns of users during decision-making processes.

2.3.1. Experimental Materials

In the ET-EEG experiment, each pair of adjectives was matched to 15 hair dryer stimuli. For example, in the “dull–cute” dimension, based on the Kansei image evaluation of the 50 hairdryer stimuli in Table 2, five stimuli each representing extreme dullness, neutral, and extreme cuteness values were selected. The experiments were conducted using a “priming stimulus” and “target stimulus” paradigm [35]. Four pairs of adjectives were used as the priming stimuli, and a hair dryer stimulus was used as the target stimuli. To reduce individual subjective differences and increase experimental accuracy, product image boards [36] were designed to guide subjects. A representative image board is shown in Figure 2.

2.3.2. Experimental Subjects

A total of 30 design graduate students and 2 design undergraduates were recruited to participate in this experiment. There were equal numbers of males and females; subjects were 19–28 years old, and all were right-handed. All subjects were in good health, had no history of mental or neurological disorders and were able to clearly see the experimental stimuli presented on a screen. The experimental procedure was explained to subjects before the experiment, and all subjects signed an informed consent form before participating.

2.3.3. Experimental Equipment

The experiment utilized the ErgoLAB V3.0 human-machine environment test platform (developed by the Kingfar technology company, Beijing, China), and the Tobii (Stockholm, Sweeden) Pro Fusion 250 Hz eye-tracking device. EEG data were collected by a 32-channel BitBrain (Zaragoza, Spain) semi-dry EEG system, with a highest sampling rate of 1000 Hz per lead, and electrodes were arranged according to the international 10–20 system. The electrode arrangement is shown in Figure 3. A CPU (E5-2678V3) with a graphics card (RTX3090 24G) was used to collect and process the data. Stimuli were presented on a PHL242E2F (1920 × 1080) display. The eye tracker was mounted on the lower edge of the monitor frame, with subjects’ eyes positioned at an average distance of 70 cm from the device. The experiment was conducted in a quiet laboratory featuring soft lighting, comfortable temperature, and no noise interference. The experimental scenario is shown in Figure 4.

2.3.4. Experimental Process

Prior to the experiment, subjects donned EEG caps, and the eye-tracking equipment was calibrated. If the experiment is interrupted, recalibration is required. To minimize experimental errors caused by subject movement, subjects should minimize body movement during the experiment. To reduce experimental error due to individual subjective perceptions of the adjectives, subjects viewed the image boards before the experiment. Before the pre-experiment, the subjects were instructed to sit and meditate with their eyes closed for 5 min. After this meditation, the screen introduced the experiment, including the experimental paradigm and precautions. Subjects started the experiment after fully understanding the experimental content. In the pre-experiment, subjects were familiarized with the experimental process to ensure the smooth operation of the formal experiment. The formal experimental procedure was the same as the pre-experiment. The specific procedure was as follows: first, a fixation cross was presented in the centre of the screen for 1000 milliseconds. Subsequently, priming adjectives were presented for 2000 milliseconds. Finally, the target stimulus was presented (Randomly presented). The time elapsed between presentation and input of the subject decision was defined as the decision latency. Subjects indicated their image judgment as follows: 1, extreme mismatching; 2, mismatching; 3, neutral; 4, matching; and 5, extreme matching. Subjects pressed the corresponding number on the keyboard. After their decision was entered, the next trial began. Each stimulus in the experiment was assessed once for each of the adjective pairs for a total of 5 × 3 × 2 × 4 = 120 trials. To ensure the accuracy of the experiment and prevent errors caused by subject fatigue, a 3-min break was provided after 60 trials were performed. The entire experiment lasted 23 min. The experimental procedure is shown in Figure 5.

3. Statistical Analysis

3.1. Behavioural Analysis

In the experiment, ErgoLAB will record the time required for subjects to make decisions. The decision latency indicated the time between the appearance of the target stimulus and the keyboard response of subjects regarding their decision. It represents the time required for cognitive processing of the subject’s form imagery. One-way ANOVA was conducted on decision latency according to product image evaluation. Decision latency was significantly correlated with product image evaluation F(34, 83) = 1.725, p = 0.023 < 0.05. The decision latency was longer when the stimulus was unambiguous than when the stimulus was ambiguous. Additionally, unambiguous stimuli had longer decision latency when the stimulus was a mismatching compared to when the stimulus was a match. Ambiguous stimuli had a shorter decision latency for positive adjectives than negative adjectives, as shown in Figure 6.

3.2. Eye Movement Feature Extraction

ET technology records eye movements during cognitive processing and thus reflects the subject’s cognitive process. In this paper, a Tobii Pro Fusion 250 Hz eye tracker was used to collect eye movement indexes of subjects during the cognitive decision-making process of product imagery as an index of subjects’ emotional experience induced by product imagery decision-making behaviors. In this study, the ErgoLAB human-machine environment synchronization test cloud platform was used to extract 14 eye movement features. To reduce the effect of gross errors on the experimental data, the Pauta criterion (1) and the discriminant (2) were used to exclude gross errors for all eye movement data [37] as follows:

$$\sigma =\sqrt{{\displaystyle \frac{{\displaystyle {\sum }_{i=1}^n(Z(k)-\overline{Z})}}{n-1}}}$$

(1)

$$R_n(k)=\left|Z(k)-\widehat{Z}\right|$$

(2)

where n is the number of observations, $\overline{Z}$ is the arithmetic mean of the observations, σ is the standard deviation of the given parameter, R_n is the residual, and $\widehat{Z}$ is the estimated value of the observations.

To better reflect user cognition, the eye movement indexes most closely related to product image evaluation were determined. One-way ANOVAs were conducted on the eye movement data according to product image evaluations. Four eye movement indexes (average pupil diameter, maximum pupil diameter, fixation count, and total saccade time) were associated with product image evaluation during the decision latency, as shown in

Table 3. Correlations of eye movement indices with product image evaluations.

Eye Movement Index	F	p
Average pupil diameter (mm)	2.449	0
Maximum pupil diameter (mm)	1.813	0.015
Total saccade time (s)	1.548	0.056
Fixation count (N)	1.598	0.044

. There was edge correlation between total saccade time and product image evaluation, p = 0.056. Average pupil diameter, maximum pupil diameter and fixation count were all significantly correlated with product image ratings and thus significantly influenced by affective changes in the subjects.

The eye movement index values in the different conditions are shown in Figure 7. Statistical analysis showed that the values of eye movement indexes were greater when the stimulus was ambiguous than when the stimulus was unambiguous. When the stimulus was ambiguous, associations with the positive adjective led to a lower eye movement index value than associations with the negative adjective. When the stimulus was unambiguous, the fixation count and total saccade time of matching stimuli were greater than those of mismatching stimuli. Although there was no significant difference in average pupil diameter between the matching and mismatching stimuli, the maximum pupil diameter value was greater for the mismatching stimuli.

3.3. EEG Data Extraction

3.3.1. EEG Data Acquisition and Preprocessing

EEG can record scalp potential changes at higher frequencies, thus investigating the features of scalp potentials in different cognitive matching states. EEG data can respond to real-time changes in the EEG signals of a subject after receiving a stimulus, thus characterizing cognitive processing in brain regions. In this paper, BitBrain semi-dry electrode EEG system was used to collect EEG data, and EEGLAB (MATLAB-Based EEG Data Processing Toolbox) was used to pre-process the EEG data. While EEG data were collected from 32 subjects, equipment failure led to lack of acquisition of data from sub001 and sub002. Therefore, EEG data from 30 subjects were preprocessed. The main steps were as follows: selection of the EEG channel, loading of channel locations, application of a bandpass filter, switching of reference electrodes, performance of principal component analysis, removal of artefacts, data segmentation, and baseline correction. The average reference electrode was used in this study, and a bandpass filter of 0.1–30 Hz was applied to remove the influence of main power. Employing ICA to remove artifacts such as eye movements, heartbeat, and eye drift. The time window analysed lasted from 200 ms before target stimulus presentation (defined as 0 ms) to 1000 ms after the appearance of the target stimulus. Thus, 1200 ms of EEG data was subjected to time frequency analysis. Baseline correction involved the selection of data during the period [−200 ms, 0 ms]. The preprocessed EEG data of all subjects were overlaid to obtain the overlay data of individual trials for ERP analysis.

3.3.2. EEG Feature Analysis

Previous studies have found that the P1, N1 and P2 components reflect early cognitive processing and that the N2, P300 and N400 reflect later processing related to judgements and cognitive evaluations of events; therefore, the latter components were selected and analysed in this study to determine product image cognition.

• N2 component analysis

The N2 component, a negative peak that occurs between 200 ms and 300 ms, under different conditions was analysed by repeated-measures ANOVAs. There was a significant main effect of condition on N2 amplitude [F = 4.356, p = 0.006 < 0.01]; thus, the N2 amplitude was significantly influenced by condition and can be used as an indicator to explore the relationships between EEG responses and product image decisions. Pairwise comparisons of different conditions showed significant differences in N2 amplitude between unambiguous (matching) and ambiguous (positive) stimuli (p = 0<0.05), between unambiguous (matching) and ambiguous (negative) stimuli (p = 0<0.01), and between unambiguous (mismatching) and ambiguous (positive) stimuli (p = 0.01 < 0.05). There was also a significant difference in N2 amplitude between unambiguous (mismatching) stimuli and ambiguous (negative) stimuli (p = 0.013 < 0.05). The difference in N2 amplitude between unambiguous (matching) and unambiguous (mismatching) stimuli (p = 0.912 > 0.05) was not statistically significant. The difference in N2 amplitude between ambiguous (positive) and ambiguous (negative) stimuli (p = 0.071 > 0.05) was not statistically significant.

• P300 component analysis

Repeated-measures ANOVAs were performed for the P300 component, a positive peaks between 300 ms and 400 ms, according to condition. There was a significant main effect of condition on P300 amplitude [F = 11.393, p = 0 < 0.01]. Therefore, the P300 component can be used as a reference for studying the relationships between EEG data and product image decisions. Pairwise comparisons of conditions revealed that the mean P300 amplitude was significantly different between unambiguous (matching) and ambiguous (positive) stimuli (p = 0 < 0.01), between unambiguous (matching) and ambiguous (negative) stimuli (p = 0.005 < 0.01), and between unambiguous (mismatching) and ambiguous (positive) stimuli (p = 0.001 < 0.01). The mean P300 amplitude did not statistically significantly differ between matching and mismatching conditions for unambiguous stimuli (p = 0.32 > 0.05). When the stimuli were ambiguous, the mean P300 amplitude did not statistically significantly differ between positive and negative stimuli (p = 0.916 > 0.05). The mean P300 amplitude did not statistically significantly differ between unambiguous (mismatching) and ambiguous (negative) stimuli (p = 0.062 > 0.05).

• N400 component analysis

Repeated-measures ANOVAs were performed for the N400 component, a positive peak between 400 ms and 500 ms, according to condition. There was a significant main effect of condition on N400 amplitude [F = 11.393, p = 0<0.01], indicating that the N400 component can be used as a reference indicator for exploring judgements in each condition. The mean N400 amplitude was significantly different between ambiguous (positive) and unambiguous (matching) stimuli (p = 0<0.01), between ambiguous (positive) and unambiguous (mismatching) stimuli (p = 0<0.01), between ambiguous (negative) and unambiguous (matching) stimuli (p = 0.007 < 0.01), between ambiguous (negative) and unambiguous (mismatching) stimuli (p = 0.018 < 0.05), and between ambiguous (positive) and ambiguous (negative) stimuli (p = 0.019 < 0.05). However, for unambiguous stimuli, there was no statistically significant difference in N400 amplitude between matching and mismatching stimuli (p = 0.734 > 0.05).

To further explore differences in ERPs induced by different conditions, electrodes strongly correlated with the product image were screened by one-way ANOVA, as shown in

Table 4. One-way ANOVA results of product image evaluations and electrode measurements of different ERPs.

ERP Component	Time Window (ms)	Electrode	F	p
N2	200–300	F3	3.931	0.022
		FC5	3.587	0.031
		FC1	6.058	0.003
P300	300–400	CP5	8.401	0
		P7	3.317	0.04
		POZ	3.094	0.049
N400	400–500	FP2	3.481	0.034
		AF4	4.406	0.014
		F7	3.845	0.024
		FC1	4.287	0.016
		FC2	5.587	0.005
		C4	4.088	0.019
		T8	6.728	0.002
		CP2	5.149	0.007
		P4	6.42	0.002

. In the table, odd numbers indicate left electrodes, and even numbers indicate right electrodes. The analysis examined ERP amplitudes for unambiguous (matching/mismatching) and ambiguous (positive/negative) stimuli. The ERP waveforms under the two conditions are shown in Figure 8 and Figure 9.

The N2 amplitude in the frontal lobe was significantly correlated with that at the F3 [F(2, 177) = 3.931, p = 0.022 < 0.05], FC5 [F(2, 177) = 3.587, p = 0.031 < 0.05], and FC1 electrodes [F(2, 177) = 6.058, p = 0.003 < 0.01]. For unambiguous stimuli, the N2 amplitude in the matching condition was significantly greater than that in the mismatching condition. For ambiguous stimuli, the N2 amplitude was larger for positive stimuli than for negative stimuli. The P300 amplitude was significantly correlated among electrodes CP5 [F(2, 177) = 8.401, p = 0<0.01], parietal P7 [F(2, 177) = 3.317, p = 0.04 < 0.05] and POZ [F(2, 177) = 3.094, p = 0.049 < 0.05] in the central region. For unambiguous stimuli, the P300 amplitude induced by mismatching was larger than that induced by matching. For ambiguous stimuli, the P300 amplitude induced by positive words was larger than that induced by negative words. The N400 amplitudes in the frontal FP2 [F(2, 177) = 3.481, p = 0.034 < 0.05], AF4 [F(2, 177) = 4.406, p = 0.014 < 0.05], F7 [F(2, 177) = 3.845, p = 0.024 > 0.05], FC1 [F(2, 177) = 4.287, p = 0.016 < 0.05], FC2 [F(2, 177) = 5.587, p = 0.005 < 0.01], central C4 [F(2, 177) = 4.088, p = 0.019 < 0.05], CP2 [F(2, 177) = 5.149, p = 0.007 < 0.01], temporal T8 [F(2, 177) = 6.728, p = 0.002 < 0.01], and parietal P4 electrodes [F(2, 177) = 6.42, p = 0.002 < 0.01] were significantly correlated. For unambiguous stimuli, the N400 amplitude of mismatching stimuli was greater than that of matching stimuli; for ambiguous stimuli, positive word matching elicited a greater amplitude than negative word matching.

4. Discussion

4.1. Relationship Between Behavioural Data and Image Evaluation

For unambiguous stimuli, subjects had shorter decision latency, whereas for ambiguous stimuli, the decision latency was longer due to greater cognitive difficulty. Further study of unambiguous stimuli revealed that the decision latency in the matching condition took longer than that in the mismatching condition, a finding supported by Yang et al. [34]. The results suggest that when stimuli match with adjectives, subjects’ decisions consume less memory resources, product information is transmitted more efficiently, and cognitive processing is smoother. The analysis of ambiguous stimuli revealed that the decision latency was longer for the negative words than the positive words in the adjective pair, which indicates that negative words require more cognitive resources during decision-making than positive words, imposing a greater cognitive load and thereby increasing the difficulty of decision-making.

4.2. Relationship Between Physiological Features and Image Evaluation

4.2.1. Relationship Between Eye Movement Indexes and Image Cognition

The differences in the attraction potency and cognitive load of the stimulus form resulted in significant differences in attention allocation. According to previous studies, the fixation count and total saccade time are correlated with subjects’ information search and cognitive processes [38]. The analysis of mean values of the eye movement indexes in the four conditions showed that when subjects made decisions regarding unambiguous stimuli, the fixation count and total saccade time values were smaller, indicating that the unambiguous stimuli were more easily processed by users. For unambiguous stimuli (matching/mismatching), the fixation count and total saccades time were larger, which indicates that matching of adjectives with stimuli induced greater fixation. For ambiguous stimuli (positive/negative), the fixation count and total saccade time values were smaller when the stimulus was associated with positive words. Associations of positive words with the stimulus could attract subject attention and lead to more efficient information search, and it is consistent with the findings of Li et al. [39].

Pupil diameter is related to the cognitive demands of tasks and is one of the important measures of cognitive load; pupil diameter increases with increasing task demand and resource input [40]. Analysis of pupil diameters showed that pupil diameter was larger for ambiguous stimuli than unambiguous stimuli, and this is consistent with the findings of Henderson et al. in emotional imagery and pupil diameter [41]. When stimuli are ambiguous, the subjects need to allocate more attention to searching for relevant information, which hampers information processing and increases cognitive load, leading to increased pupil diameter. When stimuli were unambiguous, there were no significant differences in average pupil diameter between matching and mismatching conditions; however, the maximum pupil diameter in the mismatching condition was significantly larger than that in the matching condition. This suggests cognitive load during decision-making regarding mismatching stimuli. For ambiguous stimuli, stimuli associated with negative words induced a larger pupil diameter. The association of negative words with the stimulus may have consumed more attentional resources and imposed a greater cognitive load, and the study by Buter et al. supports this result [42]. Therefore, in the design process, we should avoid products whose form is inconsistent with the image.

4.2.2. Relationship Between EEG Features and Image Cognition

The N2 reflects an individual’s early processing of stimuli, is related to visual input, reflects the allocation of attentional resources, and increases in amplitude with increasing difficulty of recognition [43]. By comparing the N2 amplitude induced by unambiguous and ambiguous stimuli, we found that the N2 amplitude increased with increasing decision difficulty. The processing of this information mainly occurred in the left frontal area, with obvious differences between the hemispheres. The results showed that the N2 component reflects the recognition of stimulus information. Ambiguous stimuli lack unambiguous image feature information; therefore, decision-making requires a substantial investment of cognitive resources, which leads to a larger N2 amplitude. Additionally, when the stimulus and associated semantics are contradictory, this generates cognitive conflict reflected by a larger N2 amplitude [44]. Although the N2 component did not differ significantly between unambiguous (matching/mismatching) stimuli in this study, ERP waveforms showed that unambiguous (mismatching) stimuli evoked greater N2 amplitudes than unambiguous (matching) stimuli. There was no significant difference in N2 component between ambiguous (positive/negative) stimuli, but ERP waveforms were stronger for ambiguous (positive) stimuli.

The P300 component is related to psychological factors, and its amplitude is proportional to the cognitive load. The more difficult the decision is and the more complex the stimulus is, the greater the P300 amplitude. This ERP is associated with activities such as decision selection, stimulus discrimination, and object classification [45]. The decision-making process is realized by comparing the stimulus with knowledge stored in the brain. The greater the similarity between the stimulus and memory information, the higher the efficiency of processing. Compared with unambiguous stimuli, ambiguous stimuli required greater memory resources for evaluation and induced greater P300 amplitudes. For ambiguous stimuli, although stimuli associated with positive and negative words induced the P300 in this study, there was no significant difference in the P300 amplitudes between the two conditions. However, the positive condition was more strongly related to the P300. Because the brain is more inclined to process positive emotional stimuli during late evaluation, with negative stimuli, positive stimuli can induce a greater P300 component [46].

The N400 component is associated with semantic violations and is mainly observed in the frontal and central regions. Kansei adjectives not related to the stimuli and ambiguous stimuli can elicit a substantial N400 amplitude; both are observed in similar areas. It has been shown that the N400 amplitude is correlated with similarity; lower similarity results in larger N400 amplitudes [47]. Although there was no significant difference between N400 amplitudes evoked by matching and mismatching conditions of unambiguous stimuli in this study, the N400 amplitude was larger for the mismatching condition. Psychological studies have found that negative emotions elicit activity from more neural structures for information processing, occupy more cognitive resources, and increase the difficulty of the task, whereas positive emotions promote cognitive processing [48]. Inconsistent with these findings, in the present study, ambiguous stimuli associated with positive words induced a larger N400 amplitude.

4.3. Comprehensive Analysis

Integrated analysis of behavioral, eye tracking, and EEG data revealed significant differences in the cognitive process of product form imagery cognition based on varying levels of imagery unambiguous. When product form imagery is clearly defined, subjects’ cognitive processing of imagery proceeds more smoothly due to distinct product features, resulting in higher efficiency in information retrieval and matching. This manifests as shorter decision-making times and smaller values for eye-tracking metrics such as fixation counts, total saccade duration, and pupil diameter. When product form imagery is ambiguous, cognitive processing uncertainty increases, requiring greater cognitive resources for analysis and decision-making. This manifests as significant increases in all eye tracking metrics and decision-making time.

The event-related potential (ERP) results were consistent with the aforementioned findings. The N2, P300, and N400 components all exhibited distinct activation patterns under different product form imagery conditions. It is particularly noteworthy that this study observed significantly larger N400 amplitudes when samples were paired with positive words under conditions of low perceptual image clarity. While this phenomenon may appear inconsistent with the widely held view in psychology that “negative emotions mobilize more neural resources” [48], it actually reveals the specificity of cognitive processing under conditions of perceptual ambiguity. We propose that perceptual ambiguity itself creates an uncertain cognitive context, which may lead participants to form a relatively neutral cognitive expectation. In such a context, the presentation of an explicitly positive word may generate significant conflict or violation against this background expectation, thereby increasing semantic integration difficulty and manifesting as enhanced N400 amplitude. Therefore, the observed N400 enhancement likely stems from integration difficulties caused by the cognitive conflict between “unambiguous and ambiguity,” rather than being solely attributable to the emotional valence of the words.

In summary, the behavioral data, eye tracking features, and EEG features demonstrated strong correlation and consistency, collectively indicating that participants’ cognitive load followed an increasing gradient of unambiguous (matching) < unambiguous (mismatching) < ambiguous (negative) < ambiguous (positive). This validates multimodal data fusion as an effective and objective approach for capturing users’ unconscious cognitive preferences regarding product imagery.

5. Conclusions

The present ET-EEG product image cognition experiments were conducted to analyses behavioral, eye movement, and EEG data of users during the Kansei cognition of the shape of hair dryers from the perspective of visual and cognitive processing. Cognitive processing was facilitated, and decision latencies were shorter for unambiguous product images. Unambiguous stimuli that did not match the Kansei adjectives required greater cognitive resources compared to those that matched the Kansei adjectives. Ambiguous stimuli associated with negative words consumed more memory resources and led to longer decision latencies than those associated with positive words. The study showed higher consistency between subject behavioral, eye movement, and ERP data. At the same time, subject decision latency, fixation counts, pupil diameter and other eye movement indicators as well as the amplitude of the N2, P300 and N400 components served as objective data regarding users’ cognitive preferences for product images. These objective data can inform designers regarding the multidimensional evaluation of product images by users. ET-EET technology, as an objective basis for capturing user preferences, can provide new pathways for design decisions.

6. Prospects

This paper analyses the relationship between the four states of users’ product imagery decision-making and behavioral, eye movement, and EEG data. However, according to previous studies, it was found that there are differences in Kansei cognition between males and females, and that there is an obvious gender bias in cognitive preference for products. At the same time, the use of the dominant hand leads to a hemispheric effect in the cognitive processing of the brain, and most of the current products are designed with the right-handed as the target user group, causing some inconvenience to the life of the left-handed group. In the follow-up study, the effects of gender, handedness and sample greyness on users’ imagery cognition can be explored to enrich the quantitative criteria of Kansei cognition.