Evaluation of Rural Visual Landscape Quality Based on Multi-Source Affective Computing

Abstract

Assessing the visual quality of rural landscapes is pivotal for quantifying ecological services and preserving cultural heritage; however, conventional ecological indicators neglect emotional and cognitive dimensions. To address this gap, the present study proposes a novel visual quality assessment method for rural landscapes that integrates multimodal sentiment classification models to strengthen sustainability metrics. Four landscape types were selected from three representative villages in Dalian City, China, and the physiological signals (EEG, EOG) and subjective evaluations (Beauty Assessment and SAM Scales) of students and teachers were recorded. Binary, ternary, and five-category emotion classification models were then developed. Results indicate that the binary and ternary models achieve superior accuracy in emotional valence and arousal, whereas the five-category model performs least effectively. Furthermore, an ensemble learning approach outperforms individual classifiers in both binary and ternary tasks, yielding a 16.54% increase in mean accuracy. Integrating subjective and objective data further enhances ternary classification accuracy by 7.7% compared to existing studies, confirming the value of multi-source features. These findings demonstrate that a multi-source sentiment computing framework can serve as a robust quantitative tool for evaluating emotional quality in rural landscapes and promoting their sustainable development.

1. Introduction

Amid the dual forces of accelerating global urbanization and the concurrent antiurbanization trend, the rural landscape is undergoing an unprecedented transformation in value. In the postindustrial era, reflections on the “over-technicalization” of urban spaces have led to the social practice of “re-localization”, prompting a reassessment of the strategic importance of rural areas as the foundation of ecological resilience and cultural heritage [1]. While the implementation of China’s rural revitalization strategy has progressively elevated rural environments by “underdeveloped areas” to central fields for “ecological-cultural-economic collaborative revitalization”, the assessment of rural landscape sustainability remains mired in significant quantitative challenges. Although traditional ecological indicators (such as soil and water conservation rates and biodiversity indices) can depict the physical attributes of natural resources, they fail to capture the underlying forces of emotional carrying capacity in human settlements and the visual cognitive experience influencing community identity and ecological behavior [2]. This issue arises from the methodological limitations of a one-dimensional approach, where the “hard-data” of ecosystems and the “soft-value” of human emotions have long existed in a disciplinary divide [3]. As an emerging interdisciplinary innovation, emotional computing technology, through its (1) theoretical evolution, (2) technological breakthroughs, and (3) paradigm shifts, constructs a quantitative sustainability framework that integrates multi-source emotional computing with rural visual landscapes, unveiling the synergistic potential between the two.

1.1. Affective Computing

Emotion, as a complex psychophysiological phenomenon, plays a pivotal role in daily life [4]. A positive emotional state is essential for maintaining both physical and mental wellbeing, while prolonged negative emotions can profoundly affect an individual’s health [5]. Consequently, the study of emotional states has expanded across multiple disciplines, including neuroscience, psychology, medicine, biology, computer science, engineering, and the humanities, gradually fostering an interdisciplinary development trend [6,7,8,9,10]. For instance, in neuroscience, emotional research focuses on the neural mechanisms underpinning emotions; in psychology, it explores how emotions influence human behavior, resulting in physical and psychological changes; and in computer science, it encompasses the study and analysis of emotional responses, commonly referred to as affective computing, which has garnered increasing attention in recent years. Affective computing serves as an umbrella term for human emotion recognition and analysis [6]. As a cornerstone in the advancement of human-centered artificial intelligence and human–computer interaction [11], emotional computing has enabled computers to recognize, express, and respond to both their own emotions and those of humans, a concept introduced by Professor Picard in 1997 [12,13]. In practical applications, emotional computing has been employed across diverse fields such as healthcare, education, business services, intelligent driving, social media, and the integration of science and art [14,15,16,17,18,19], where it is used to identify users’ emotional states and provide appropriate feedback and adjustments.

1.2. Multi-Source Affective Computing

Multi-source emotion computing refers to the application of experiments utilizing multi-source data in emotion computing, aiming to transition from single-mode to multi-source data [20], transcending modal boundaries to achieve a higher emotion recognition rate [21,22]. Its research framework encompasses two core themes and five key aspects (as depicted in Figure 1). The two themes are emotion recognition and emotion analysis, while the five aspects include the foundational theory of emotion, signal collection, algorithm modeling, modal fusion, and output presentation, which may overlap and interrelate. In the domain of emotion recognition, the focus is on detecting human emotional states (i.e., discrete or dimensional emotions) through visual, auditory/speech, and physiological modalities. Sentiment analysis, on the other hand, primarily centers on evaluating and extracting preferences for objects or events [23], with results typically categorized as positive, negative, or neutral [24,25]. Among the five aspects, the first is the foundational theory of emotions. Psychologists have proposed two primary models—the discrete emotion model and the dimensional emotion model—to simulate human emotions from basic to complex states [26,27]. The second aspect concerns the collection of emotional signals. Recent advancements have seen the use of various physiological signals, such as electroencephalogram (EEG), electromyogram, electrocardiogram, and eye movement, alongside non-physiological signals, including text, speech, facial expressions, and body movements, all of which contribute to emotion recognition, supported by corresponding datasets [28,29,30]. Notably, EEG signals have been shown to outperform other physiological signals in emotion recognition tasks [31,32], while eye movement signals complement EEG data in multimodal emotion recognition scenarios [33,34], enhancing the accuracy of multimodal emotion classification systems. The third aspect, algorithm modeling of emotion, involves leveraging machine learning, ensemble learning, and deep learning techniques to model and identify emotional signals. Common classifiers for environmental emotion recognition include Logistic Regression, Support Vector Machines (SVMs), Decision Trees, ensemble models, and neural networks [35,36,37]. The fourth aspect, modal fusion, integrates emotional features and fusion algorithms derived from multimodal recognition to enhance emotion classification accuracy [38]. The fifth aspect is the output presentation of emotion, which enables machines to express emotions through facial expressions, vocal intonation, body movements, and visualization platforms, following the learning of emotional signals [39], thereby advancing human–computer interaction. Therefore, sentiment analysis utilizing multi-source data plays a pivotal role [40] and has been increasingly validated in sentiment classification and event detection applications [30,41].

1.3. Rural Visual Landscape Quality Assessment

As a spatial–cultural composite that embodies the evolving human–land relationship, the assessment of rural landscape visual quality plays a pivotal role in quantifying ecological service functions and preserving cultural memory [42,43]. In contrast to the homogenizing trends observed in urban landscapes, the regional heterogeneity of rural visual landscapes, coupled with the interplay between natural and human elements, facilitates a paradigm shift in evaluation—from traditional aesthetic criteria to interdisciplinary collaborative analysis [44]. The theoretical foundation for visual landscape quality assessment can be traced to Laurie’s concept of “the comparative relationship between the perception and evaluation of two or more landscapes” [43]. In the rural context, this concept is further extended to encompass “the systematic decoding of the visual characteristics and emotional value of local spaces”. Methods for assessing the quality of rural visual landscapes can be categorized into two distinct approaches [45]. The first, “externalist” evaluation, treats space as an object of observation. Common methods, such as GIS [46] and surveys [47], are employed to analyze stimuli including remote sensing images, classified maps, photographs, and actual landscapes. The second approach is “egocentric” assessment, which typically begins with direct human experience and employs physiological sensors—such as eye trackers [48] and EEG [47]—along with evaluative tools such as the Scenic Beauty Estimation (SBE) method [49], Analytic Hierarchy Process (AHP) [50], and Semantic Differential (SD) method [51]. This approach combines intuitive perception and assessment of photos, simulations, and real landscapes, incorporating both objective and subjective evaluations (through questionnaires). In addition to these two evaluation methods, the interdisciplinary integration of neuroscience, psychology, computer science, and visual landscape research has led to the development of techniques for random visual landscape perception, evaluation, and emotion classification prediction. For example, Ningning Ding et al. [52] analyzed the preferences of villagers and university students using eye tracking and EEG to explore the appeal of plant organ structures. They found that plants with distinct features and vibrant colors were preferred in rural landscapes, while simpler structures were favored in campus settings. Feng Ye et al. [53] developed a predictive model based on the correlation between eye-tracking metrics and 19 different emotional responses to rural landscapes, highlighting the effectiveness of eye-tracking technology in capturing and predicting emotional reactions to various landscape types. Wang Yuting et al. [54] innovatively employed aerial video and EEG technologies to assess rural landscapes, selecting seven representative landscape types, extracting EEG features, and classifying them with four classifiers. They demonstrated that SVM and Random Forest (RF) classifiers exhibited high accuracy, achieving 98.24% and 96.72%, respectively, and identified distinct classification patterns across different features and bands, thereby advancing novel methodologies for quantifying human perception. Thus, in the assessment of rural visual landscape quality, perception refers to the behavioral patterns of visual or brain activity, while evaluation pertains to preferences and ratings of the landscape. Both processes are intrinsically linked to the emotional experience of the visual landscape.

1.4. Summary of Relevant Research

In summary, the primary challenges in rural visual landscape affective computing research are as follows: (1) Insufficient interdisciplinary integration in rural landscape affective computing. Although affective computing technology has been extensively applied in fields such as healthcare, education, and urban design, its comprehensive integration into rural landscape visual quality assessment remains underdeveloped. (2) Limited collaborative classification of multimodal data and subjective questionnaires. While the fusion of objective modes, such as eye tracking and EEG, has been explored in multi-source affective computing, few datasets have been developed that use subjective questionnaire data as independent variables. (3) Existing studies lack systematic verification of the adaptation rules for classifier performance. Most research defaults to using a single classifier (e.g., SVM or Random Forest) to process emotion classification tasks across various categories, overlooking the boundary effects of task complexity on model performance. This leads to challenges such as difficulties in adaptive multi-source feature fusion, weak scene generalization, and small sample overfitting. In response, this study selected twelve rural landscape images, representing four distinct types from Dalian, Liaoning, China, as stimuli. It collected emotional signals using a combination of subjective and objective methods (“eye movement + EEG + questionnaire score”) from participants, and applied machine learning and ensemble learning techniques to develop an emotion classification model and spatial emotion quality assessment process. This approach is applicable to a variety of rural visual landscape quality assessments, with the final evaluation results providing valuable insights for rural landscape design and decision-making in rural revitalization.

This paper is structured into five sections: Section 1 provides an overview of the prior research. Section 2 introduces the study area, data, and methodologies employed. The key findings are presented and analyzed in Section 3. Section 4 discusses the results, along with the study’s limitations. Finally, Section 5 presents the conclusions.

2. Materials and Methods

2.1. Research Area

Dalian, situated in the southern part of Liaoning Province (120.58° E to 123.31° E, 38.43° N to 40.10° N), as depicted in Figure 2a, is renowned as the “Pearl of the North” and is a popular tourist destination. In addition to its rich marine culture, Dalian’s villages exhibit a diverse range of forms and regional characteristics, making it an ideal setting for rural visual landscape quality assessments. Therefore, three representative villages within Dalian were selected for investigation in this study: (1) Xutun Village, a traditional settlement located in Xutun Town, Wafangdian City, known for its long history and distinctive architectural style, which primarily focuses on restoration and conservation; (2) Yangquan Circle Village, a coastal village situated in Gezhenbao Street, Ganjingzi District, where the village’s development is influenced by a “medium intervention” approach from architects and urban planners; and (3) Shabao Village, a rural settlement in Shabao Street, Pulandian District, which has undergone spontaneous, top-down construction and renovation by the villagers themselves (as shown in Figure 2c,d). Field investigations were conducted, and key landscape features of the villages were photographed to serve as a foundation for the multi-source emotion computing experiments.

2.2. Experimental Elements

2.2.1. Element Presentation

Given the constraints of human and material resources for onsite landscape evaluation, landscape photographs were selected as substitutes for direct environmental investigations [55]. To ensure the subjects’ perceptions closely align with real-world experiences, virtual scenes, which could distort visual perception, were avoided [56,57,58]. Consequently, original images were utilized in this study, with all images sized at 1920 × 1080 pixels, rather than edited landscape photographs. A SONY A6000 SLR (SONY, Tokyo, Japan) camera was employed to capture landscape samples from a distance of 8 m and at a height of 1.6 m from the edge of the object space. The photos were taken from November 2023 to May 2024, resulting in a total of 360 images.

During the photo-selection process, five experts classified the images according to the following procedure: (1) Feature Annotation: Reviewed 360 photographs, extracted salient keywords for landscape elements (such as buildings, water bodies, vegetation, roads, etc.), and annotated each image’s core attributes. (2) Unrestricted Classification: Grouped images sharing similar annotated features. (3) Consolidation: Considering overlap or redundancy among the 360 photographs, provisionally sorted them into four categories—architecture, water, vegetation, and roads—with ten images per group. (4) Validation and Refinement: Experts scored and cross-referenced each category, reducing each group to five or six images. (5) Representativeness Assessment: Based on each village’s characteristics, retained the three most representative images per category. This process yielded a structured library of 12 landscape images from three villages to support subsequent experimental research. To mitigate potential bias from priming effects, four additional images were chosen as warm-up stimuli for the experiment (as shown in Figure 3).

2.2.2. Experimental Subjects

To facilitate the experiment, 35 students and 3 teachers (17 males, 21 females; average age: 25.63 years) participated. Of these, 23 were aged between 20 and 25 years, 11 between 25 and 30 years, 1 between 30 and 39 years, and 3 between 40 and 45 years. All participants were right-handed, with no history of mental illness or brain trauma, and had normal or corrected vision, except in the campus environment. None of the participants had previously visited the experimental site. Four participants were excluded from the final analysis due to signal artifacts, leaving data from 34 participants (17 males and 17 females) for analysis. All participants voluntarily consented to the study, having received full information regarding the research objectives, experimental procedures, and potential risks. They signed written informed consent prior to participation and were compensated upon completion, with the option to withdraw at any time without penalty.

2.2.3. Experimental Equipment and Questionnaire

The experimental setup comprised an Eyeso Glasses head-mounted eye tracker (Braincraft, Beijing, China) and a Waveguard™ 8-lead electrode cap (ANT Neuro, Berlin, Germany). The eye movement sampling rate was 380 fps, and the EEG was recorded via a 500 Hz micro-EEG amplifier. Supporting equipment included a 24-inch display (resolution 1920 × 1080), a laptop (running Windows 10), a dongle, and an adapter for convenient mobile portability. The laptop also served as the medium for displaying the landscape images (as shown in Figure 4). The experimental questionnaires were categorized into two sections: (1) The Beauty Assessment Scale and (2) The SAM Scale. (1) The Beauty Assessment Scale is a subjective tool for quantifying individuals’ visual experiences of landscapes. It integrates the SBE (Scenic Beauty Estimation) scale with the SD (Semantic Differential) method to create a streamlined evaluation framework, enabling participants to intuitively rate landscape aesthetics—higher scores correspond to greater perceived beauty. The SBE method quantifies the landscape’s aesthetic appeal based on evaluators’ personal standards [49], while the SD method employs a verbal scale to conduct psychological measurements of individuals’ intuitive responses [51], providing quantitative data for landscape assessment. Based on the works of Liu Binyi et al. [59] and Xie Hualin et al. [60], and considering the specific visual characteristics of rural landscapes in Dalian, eight semantic variables were chosen as evaluation criteria: naturalness (N), diversity (D), harmony (H), singularity (S), orderliness (O), vividness (V), culture (C), and agreeableness (A). (2) The SAM Scale (as shown in Figure 5) [61] is a versatile tool designed to track emotional responses to stimuli across various settings, enabling quick assessment of emotional reactions. During the experiment, participants’ emotional states were recorded through the SAM questionnaire, capturing both emotional valence (positive or negative) and arousal (high or low). Both questionnaires were rated using a 5-point Likert scale, with scores ranging from −2 (lowest) to +2 (highest) [62]. By synthesizing subjective impressions with objective measurements, the evaluation of landscape aesthetics can more fully elucidate visual allure and underpin a rigorous scientific basis for the sustainable design of rural landscapes.

2.2.4. Experimental Procedures

The experiments were conducted individually in a controlled laboratory setting, adhering to a standardized procedure, with participants instructed to wash their scalp prior to the experiment to minimize impedance [63]. Initially, participants, equipped with the necessary sensors, were asked to complete a pretest involving eye movement and electrical brain stimulation by remaining seated and viewing four warm-up photos as directed by the experimenter. Upon meeting the pre-experiment standards, participants, under the experimenter’s guidance, proceeded to view a series of 12 stimulus photos (A1–A12) sequentially, following the stimulus presentation protocol of 20 s of viewing, followed by a 10 s rest period, to ensure the consistency of experimental variables. The collected EOG and EEG signals were transmitted and stored in a laptop via signal amplifiers. Throughout the experiment, efforts were made to maintain a constant environment, ensuring that only the stimuli presented on the monitor influenced the participant’s responses, thereby ensuring a high correlation between the results and the stimulus conditions. Upon completing the physiological portion of the experiment, participants rested briefly, removed the physiological sensors, and then filled out both the Beauty Assessment Scale and the SAM Scale questionnaires. After questionnaire completion, participants were thanked, and the experiment concluded (as shown in Figure 6). The total duration of the experiment was approximately 23–28 min.

2.3. Data Processing

2.3.1. Eye Movement Data Preprocessing

Following the experiment, raw eye-tracking data were processed using Pupil Player (Braincraft, Beijing, China) and eyeeso Studio 6.23 (Braincraft, Beijing, China) to ensure data integrity and consistency. Data segmentation, primarily conducted in Pupil Player, involved multiple steps, including blink removal, target area identification, and video cropping. The segmented data were then imported into eyeeso Studio for further analysis. Subsequent procedures included gaze statistics, definition and editing of Areas of Interest (AOIs), generation of attention heatmaps (as illustrated in Figure 7), and visualization of eye movement trajectories. The software extracted 31 specified signal parameters and exported them as CSV files. The hotspot map analysis algorithm calculates the aggregated Gaussian distribution of each fixation point on a given visual stimulus. Assuming a mean µ = 0, the standard deviation σ and the co-directional distribution are computed using the following Formulas (1):

$$f\left(x,y\right)={\displaystyle \frac{e^{\frac{x^2+y^2}{2{\sigma }^2}}}{2\pi {\sigma }^2}},x,y\in [-s,s]$$

(1)

The standard deviation σ is internally defined as σ = s/5. For each fixation, every element of the kernel template is scaled by a weight proportional to that fixation’s duration. These weighted kernels—anchored at their respective fixation coordinates—are then summed into an array representing stimulus intensity. Finally, the array is normalized, yielding stimulus magnitude and height maps (topographic maps).

Attention heatmaps employ a two-dimensional Gaussian kernel convolution to visually depict subjects’ attention distribution across the stimulus. Red denotes areas of concentrated focus, while yellow and green indicate lower attentiveness. From these maps, we observe the following:

(1) Architectural landscapes: All three villages exhibit focal fixation on discrete features—wall details in Xutun Village, landmark edifices in Yangquan Circle Village, and dispersed points of interest in Shabao Village—underscoring the visual salience of unique or intricate architectural elements.

(2) Water landscapes: Hotspots arise at feature boundaries or protrusions—the stream banks in Xutun Village, cliff faces along Yangquan Circle Village’s shore, and open water in Shabao Village—suggesting that contrasts between dynamic/static and smooth/abrupt forms guide gaze behavior.

(3) Vegetation landscapes: Attention clusters correspond to variations in color, stratification, and morphology—layered vegetation in Xutun Village, distinctive species in Yangquan Circle Village, and density fluctuations in Shabao Village—demonstrating that biodiversity and structural diversity capture visual interest.

(4) Road landscapes: Both linear pathways and discrete roadside elements shape eye trajectories—unique road alignments in Xutun Village, signage in Yangquan Circle Village, and surrounding objects in Shabao Village—highlighting the combined influence of spatial guidance and salient features on gaze patterns.

2.3.2. EEG Data Preprocessing

The EEG signals were preprocessed using the data analysis software Asalab 4.10.2 (ANT Neuro, Berlin, Germany), which included steps such as downsampling, re-referencing, and artifact removal. Asalab software decodes and converts the received electrical signals into visual representations, while simultaneously storing and analyzing them in the background for detailed monitoring and assessment of EEG signals. In clinical settings, the frequency-band energy ratio (FBER) is commonly utilized as a characteristic parameter to quantitatively assess changes in the basic rhythm of EEG signals [64]. In this study, the FBER value is referred to as the R-value. The R-value reflects the proportion of different waveforms across the EEG electrodes, which include eight points: Fz, Cz, Pz, F3, F4, Fpz, C3, and C4. As a crucial EEG characteristic parameter, the R-value can be used to evaluate the subject’s cerebral cortex preference and excitability in response to the rural visual landscape, based on its magnitude and variation. A total of eight EEG characteristics were derived. The frequency-band energy ratio (R-value) is calculated as follows in (2) and (3):

$$E_{all}\left(k\right)=\sum E^{\left(j\right)}{\left(k\right)}_j$$

(2)

$$R={\displaystyle \frac{E^{\left(j\right)}\left(k\right)}{E_{all}\left(k\right)}}$$

(3)

where j represents any electrode in Fz, Cz, Pz, F3, F4, Fpz, C3, and C4, E^(j)(k) represents the power value of the electrode point, and E_all(k) represents the total power value of the eight electrode points.

2.3.3. Preprocessing of Scenic View Evaluation Data

The questionnaire, utilizing a Likert scale, explicitly instructs participants to interpret adjacent scores as representing equal psychological distances [65]. To facilitate the subsequent comprehensive analysis of physiological continuous variables (eye movement and EEG) within a unified dimension, statistical verification of isometric continuous variables was conducted. Initially, a data distribution test was performed using SPSS 24.0 (IBM, Armonk, NY, USA) on eight indicators, as shown in

Table 1. Beauty assessment data symmetric distribution test table.

Variable	Skewness	Kurtosis	Minimum Value	Maximum Value	Average Value	Standard Deviation
Naturalness	−0.752	−0.306	1	5	3.80	1.156
Diversity	−0.198	−0.684	1	5	3.14	1.113
Harmony	−0.244	−0.761	1	5	3.32	1.135
Singularity	0.092	−0.722	1	5	2.85	1.142
Orderliness	−0.201	−0.752	1	5	3.20	1.130
Vividness	−0.202	−0.487	1	5	3.24	1.064
Culture	−0.185	−0.636	1	5	3.09	1.104
Agreeableness	−0.185	−0.692	1	5	3.32	1.170

, revealing that the absolute values of skewness and kurtosis for all variables were less than 1, indicating an approximately symmetric distribution [66]. Subsequently, unidimensional verification through factor analysis yielded a Kaiser–Meyer–Olkin (KMO) value of 0.916 and a factor loading of 60.14%, confirming the suitability of the data for factor analysis and demonstrating strong unidimensional properties. This aligns with established practices in multimodal data fusion within the engineering domain [67].

2.3.4. SAM Scale Data Preprocessing

The SAM questionnaire responses were classified at three granularity levels—binary, ternary, and five-category. In the binary schema, valence and arousal are dichotomized into two poles (“positive”/“negative” or “high arousal“/“low arousal”); for example, a participant’s sense of pleasure toward a rural architectural scene is tagged “1”, whereas perceptions of monotony are tagged “–1”, facilitating rapid screening of landscapes in need of renovation. The ternary schema introduces a neutral category “0” to capture ambivalent states; for instance, when participants neither favor nor dislike a given rural road scene, it is assigned “0”, supporting refined assessments that distinguish landscapes requiring optimization from those warranting preservation. The five-category schema further subdivides emotional intensity into “−2, −1, 0, 1, 2” (ranging from “extremely negative” to “extremely positive”), such as rating a sea of flowers as “2” and a dilapidated farmhouse as “−2”. Although theoretically more granular, this approach may suffer from sample-imbalance, degrading model performance. Accordingly, for the binary model, samples with valence/arousal “0” were excluded; values “−2” and “−1” were consolidated as negative/low “−1” and “1” and “2” as positive/high “1”. In the ternary model, “0” samples are retained as neutral for both valence and arousal.

After data processing, four incomplete entries were removed, resulting in a final valid dataset of 34 participants. This dataset comprises eye movement data (12,648 entries), EEG data (3264 entries), and beauty evaluation data (3264 entries), yielding a total of 19,176 data points.

2.3.5. Feature Extraction and Reduction

To determine the number and validity of features, various software packages were employed to extract features from the physiological signals and Beauty Assessment Scale. This process yielded 31 eye movement signal features, 8 EEG features, and 8 beauty assessment features, for a total of 47 features. During the preprocessing stage, it was confirmed that the beauty assessment data could be comprehensively analyzed alongside the physiological continuous variables (eye movement and EEG). Consequently, data normalization was applied separately to both the beauty assessment and physiological data, standardizing dimensions and mitigating specific effects [68]. The calculation method is outlined in Formula (4) below:

$$z_i={\displaystyle \frac{x_i-\mu }{\sigma }}$$

(4)

where μ represents the mean and σ denotes the standard deviation, both calculated from the entire signal X, with x_i ∈ X being the individual data points collected from a subject.

The standardization process was carried out independently for three distinct data types (eye movement, EEG, and beauty assessment). Subsequently, principal component analysis (PCA) was conducted on 47 signal features using SPSS 24.0 (IBM, Armonk, NY, USA) [69,70,71]. The results indicate that the Bartley sphere test is statistically significant (p < 0.01), with a Kaiser–Meyer–Olkin (KMO) value of 0.772, confirming the efficacy of PCA. The cumulative contribution rate of the extracted eigenvalues was found to be 90.775%; for details, please refer to

Table A1. Principal component contribution rate and cumulative contribution rate.

Component	Eigenvalue	Contribution Rate *	Cumulative Contribution Rate *
F1	8.235	17.522	17.522
F2	6.317	13.441	30.963
F3	4.726	10.056	41.019
F4	4.281	9.109	50.128
F5	4.009	8.529	58.657
F6	2.057	4.376	63.033
F7	1.841	3.918	66.951
F8	1.254	2.668	69.618
F9	1.138	2.422	72.040
F10	1.080	2.297	74.338
F11	0.990	2.106	76.444
F12	0.953	2.028	78.472
F13	0.827	1.760	80.231
F14	0.780	1.659	81.890
F15	0.759	1.616	83.506
F16	0.672	1.429	84.935
F17	0.648	1.378	86.313
F18	0.602	1.281	87.594
F19	0.533	1.133	88.727
F20	0.518	1.102	89.829
F21	0.444	0.945	90.775

* The unit of these data is %.

in the Appendix A. After calculating and comparing the weights of each feature, 36 features (depicted in Figure 8), which exhibited a strong correlation with mood, were selected.

2.4. Model Construction and Evaluation Methods

During data collection and processing (Section 2.1, Section 2.2 and Section 2.3), four visual landscape datasets from three villages were assembled, encompassing physiological signals (EEG, EOG) and subjective evaluations (Beauty Assessment and SAM Scales). By synchronizing data acquisition, standardizing preprocessing, and applying PCA for feature extraction, we harmonized physiological and subjective features within a unified dimensional space, ensuring data coherence and optimizing inputs for the subsequent sentiment classification model. Thereafter, SMOTE and Bayesian optimization were employed to enhance binary, ternary, and quinary classification performance—elevating accuracy, precision, recall, and F1 score—and thereby validating the coordinated integration of physiological data with subjective questionnaire metrics.

2.4.1. Build the Model

The rural visual landscape serves as a setting for the daily recreation of villagers, with spatial stimuli predominantly evoking positive or tranquil emotional responses. We observed that the samples labeled “−2” and “−1” in the valence and arousal datasets were significantly smaller than other samples, which resulted in poor recognition of negative emotions in the training model. To achieve a balanced class distribution, we employed the Synthetic Minority Over-sampling Technique (SMOTE) [72], utilizing k-nearest neighbor interpolation (k = 5, determined via grid search) to augment the number of samples in each minority class to match those of the majority class. Additionally, we implemented Bayesian optimization for hyperparameter tuning. Hyperparameter optimization plays a pivotal role in enhancing the performance of machine learning models by intelligently searching the parameter space to maximize the model’s generalization capacity. In this study, the HyperOpt Python 3.9 (Python Software Foundation, Wilmington, DE, USA) library [73,74] was used for hyperparameter tuning within the classifier element pool. Compared to traditional grid search and random search methods, this optimization strategy is particularly well suited for the multi-level classification system (binary, ternary, five-element classification) developed in this research. By defining the objective function as the weighted accuracy index of the test set (independently verified by villages), SMOTE effectively mitigated the potential risk of data leakage post-cross-validation [75]. Consequently, the classification robustness of the model was significantly enhanced in a few categories.

In classifier selection, we observed that single classifiers such as Logistic Regression (LR), Support Vector Machine (SVM), Decision Tree (DT), Artificial Neural Network (ANN), and Random Forest (RF) are commonly employed [76,77,78,79]. However, ensemble learning has been shown to yield superior predictive performance by aggregating the predictions from multiple models. Consequently, we utilized two single classifiers and two ensemble classifiers for model training. The single classifiers were LR-GD and DT, while the ensemble classifiers consisted of RF (Python 3.9) and XGBoost (Python 3.9).

2.4.2. Evaluation Methods

The confusion matrix, also referred to as the error matrix, is a visual tool primarily employed to compare the classification outcomes with the actual observed values, thereby providing a clear representation of the classification accuracy. The performance metrics of the classification model include accuracy, precision, recall, and the F1 score. By utilizing the confusion matrix, one can assess the misclassifications made by the model, facilitating subsequent adjustments to model parameters or data augmentation. The calculation methods are as follows (5)–(8):

$$Accuracy={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}$$

(5)

$$Precision={\displaystyle \frac{TP}{TP+FP}}$$

(6)

$$Recall={\displaystyle \frac{TP}{TP+FN}}$$

(7)

$$F1-score={\displaystyle \frac{2\times (Precision\times Recall)}{Precision+Recall}}$$

(8)

where TP denotes the number of true positives in the positive category, FN represents the number of false negatives in the positive category, FP indicates the number of false positives in the negative category, and TN refers to the number of true negatives in the negative category.

3. Results

3.1. Impact of Feature Reduction on the Model

The PCA algorithm was employed to reduce the 47 extracted features to 36. However, while PCA effectively reduces the dimensionality of the independent variables, it does not clarify the significance of these variables in relation to the target variables. To assess whether the reduction in feature count positively impacts valence classification, we constructed binary and ternary classification models using both 47 and 36 signal features, respectively, with XGBoost and Random Forest as classifiers. The model performance results, before and after the reduction in valence and arousal features, are presented in

Table 2. Comparison of model valence performance based on 47 features and 36 features.

Class	Classifier	47 Features				36 Features
		Accuracy	Recall	Precision	F1-Score	Accuracy	Recall	Precision	F1-Score
Binary	XGBoost	79.3%	0.918	0.839	0.876	82.1%	0.918	0.867	0.891
	RF	79.3%	0.918	0.839	0.876	84.0%	0.941	0.870	0.904
Ternary	XGBoost	65.4%	0.654	0.654	0.630	77.2%	0.772	0.765	0.766
	RF	62.5%	0.625	0.606	0.611	64.0%	0.640	0.646	0.642

“%” indicates accuracy.

and

Table 3. Comparison of model arousal performance based on 47 features and 36 features.

Class	Classifier	47 Features				36 Features
		Accuracy	Recall	Precision	F1-Score	Accuracy	Recall	Precision	F1-Score
Binary	XGBoost	72.9%	0.804	0.816	0.810	77.9%	0.847	0.859	0.853
	RF	70.8%	0.750	0.811	0.780	75.8%	0.875	0.818	0.846
Ternary	XGBoost	59.6%	0.596	0.591	0.582	74.3%	0.743	0.740	0.738
	RF	57.4%	0.574	0.556	0.558	58.1%	0.581	0.578	0.579

“%” indicates accuracy.

.

3.2. Model Classification Results and Performance Comparison

3.2.1. Binary Classification

Villages 1 and 2 were utilized as the training models, while Village 3 served as the test model. The training and test sets were randomly selected, and SMOTE along with Bayesian optimization were applied to enhance the model’s performance. In binary classification, the target variable values were “−1” and “1”, with 36 signal features as independent variables, and valence and arousal as the dependent variables. The model performance results are presented in

Table 4. Performance comparison of binary classification models based on different classifiers for valence and arousal.

Emotional Category	Classifier	Accuracy	Recall	Precision	F1-Score
Valence	XGBoost	82.1%	0.918	0.867	0.891
	RF	84.0%	0.941	0.870	0.904
	DT	67.9%	0.718	0.859	0.782
	LR-GD	69.8%	0.694	0.908	0.787
Arousal	XGBoost	77.9%	0.847	0.859	0.853
	RF	75.8%	0.875	0.818	0.846
	DT	69.5%	0.792	0.803	0.797
	LR-GD	64.2%	0.653	0.839	0.734

“%” indicates accuracy.

and Figure 9.

The binary classification results indicate that the model’s recognition accuracy for emotional pleasure, based on XGBoost and RF, surpasses 80%. For emotional arousal, both the XGBoost- and RF-based models achieve recognition accuracy rates exceeding 75%, demonstrating robust classification performance. These findings further suggest that both models are effective in assessing the emotional quality of rural visual landscapes.

3.2.2. Ternary Classification

The target variable values for triadic classification are “−1”, “0”, and “1”, with all valid sample data utilized for model training and testing, incorporating SMOTE and Bayesian optimization. Following model evaluation, we obtained the classification accuracy for each performance indicator, as presented in

Table 5. Performance comparison of ternary classification models based on different classifiers for valence and arousal.

Emotional Category	Classifier	Accuracy	Recall	Precision	F1-Score
Valence	XGBoost	77.2%	0.772	0.765	0.766
	RF	64.0%	0.640	0.646	0.642
	DT	46.3%	0.463	0.549	0.490
	LR-GD	44.1%	0.441	0.574	0.468
Arousal	XGBoost	74.3%	0.743	0.740	0.738
	RF	58.1%	0.581	0.578	0.579
	DT	56.6%	0.566	0.604	0.572
	LR-GD	42.7%	0.427	0.485	0.445

“%” indicates accuracy.

and Figure 10.

Through the observation of emotional pleasure and arousal, the XGBoost-based model demonstrated higher performance index values, achieving recognition accuracies of 77.2% and 74.3%, respectively. In comparison, the RF model’s recognition accuracies were 64.0% and 58.1%, respectively. These results indicate that the XGBoost model is more effective in evaluating the emotional quality of rural visual landscapes.

3.2.3. Five-Element Classification

The target variable values for the five classifications are “−2, −1, 0, 1, 2”, with all valid sample data utilized for model construction, incorporating SMOTE and Bayesian optimization. After evaluating these models, we obtained the classification accuracy, precision, recall, and F1 scores for each category, as presented in

Table 6. Performance comparison of valence and arousal five-element classification models based on different classifiers.

Emotional Category	Classifier	Accuracy	Recall	Precision	F1-Score
Valence	XGBoost	64.0%	0.640	0.643	0.634
	RF	47.8%	0.478	0.487	0.476
	DT	26.5%	0.265	0.301	0.268
	LR-GD	26.5%	0.265	0.335	0.278
Arousal	XGBoost	59.6%	0.596	0.598	0.590
	RF	44.1%	0.441	0.475	0.452
	DT	30.9%	0.309	0.329	0.313
	LR-GD	24.3%	0.243	0.349	0.265

“%” indicates accuracy.

and Figure 11.

The results of the five classifications indicate that the XGBoost model exhibits the best classification performance in emotional pleasure, although its accuracy is only 64.0%. In terms of emotional arousal, the XGBoost model demonstrates superior classification performance; however, the accuracy of all four models remains below 60%. Therefore, in practical terms, none of these four models can adequately address the emotional quality assessment of rural visual landscapes in five-element classification.

3.2.4. Comparison of Optimal Classification Performance of Models

Figure 12 and Figure 13 present a comparison of the four indices for the binary, ternary, and quintuple classification models of emotional valence and optimal arousal performance (XGBoost), respectively. The results reveal a progressive decline in classification ability, with a significant decrease observed in the five-element classification. The binary and ternary classification models, however, are shown to meet the practical requirements.

3.2.5. External Validation

In addition to internal testing and performance benchmarking, the model underwent external validation. A new dataset was constructed using the four preheating images presented prior to the formal experiment and was input into the XGBoost classification model, which demonstrated relatively high accuracy. This process aimed to assess the model’s effectiveness in predicting the emotional quality of novel spatial stimuli. The model generated outputs in binary, ternary, and five-class classifications. By comparing these predicted classifications with the ground truth labels, we derived classification accuracy and confusion matrices, as presented in

Table 7. Comparison of valency and arousal accuracy of binary, ternary, and quinary classification models of XGBoost model.

Emotional Category	Accuracy	Recall	Precision
Valence	78.50%	70.23%	61.83%
Arousal	75.47%	67.94%	58.46%

“%” indicates accuracy.

and Figure 14.

The results indicate that the highest external validation accuracy was achieved in the binary classification, reaching 78.50%, followed by the ternary and five-class models with accuracies of 70.23% and 61.83%, respectively. Additionally, the classification accuracy for valence consistently outperformed that for arousal. The ternary confusion matrix reveals that samples labeled with a valence of −1 were more frequently misclassified than those in other categories. In the five-class system, due to the limited number of samples labeled as −2, the classification performance for labels 0 and 1 was comparatively more accurate.

3.3. Model Usage Process

This training model is designed to assess the quality of rural visual landscapes in practical settings. Consequently, we sought to establish a process for evaluating the emotional quality of rural visual landscapes using multi-source data (as illustrated in Figure 15). The process encompasses the following steps: First, we defined the experimental route and divided it into several segments. Next, we invited participants to engage in the study and sign a consent form. During the signal collection phase, we gathered both physiological and subjective data from participants while they viewed the images. After feature extraction, fusion, and dimensionality reduction, the processed data were input into the classification model. Based on the emotion score derived from the model, areas with a positive emotional value will be preserved, while spaces with a negative emotional value will undergo renovation.

4. Discussion

This study integrates eye movement, EEG, and a subjective beauty questionnaire to construct and evaluate a multi-source affective computing model tailored for assessing rural visual landscapes. Model performance was optimized through feature selection, SMOTE, Bayesian optimization, and ensemble classification. Generalizability was evaluated via external validation, and the normality assumption underlying the subjective aesthetic questionnaire was empirically examined.

4.1. Collaborative Classification of Multimodal and Subjective Data

As a crucial method for assessing visual quality in human–environment interactions, emotion calculation is primarily conducted through single modalities, such as GIS, questionnaires, and physiological sensors. Although multimodal sensors, such as eye movement + EEG and skin electrocardiogram + ECG, have been employed, subjective questionnaires are rarely used as independent variables. Consequently, this study integrates subjective questionnaires (beauty assessment) with objective physiological signals (eye movement, EEG) to create a multi-source dataset encompassing eight semantic variables (e.g., naturalness, diversity, coordination). The symmetric distribution test (with absolute values of skewness and kurtosis both < 1) and unidimensional test (KMO = 0.916, factor load 60.14%), as described in Shing-On Leung’s study [66], confirm that the Li Keert scale satisfies the conditions for continuous variable analysis in emotion questionnaires, thus mitigating the potential confounding effects of multi-dimensionality on model fusion. In model validation, robust fusion of multi-source features is achieved through ensemble learning. For instance, in the triadic classification task, the model combining subjective questionnaire and eye movement data achieved accuracy rates of 77.2% and 74.3% for valence and arousal emotion classification, respectively. This performance exceeds the accuracy of valence (76.1%) and arousal (67.7%) in triadic classification using the fusion of eye movement and EEG by Mohammad Soleymani et al. Additionally, the accuracy surpasses that of Wei-Bang Jiang et al. [30] and Kazuhiko Takahashi [79], who utilized eye movement and EEG as standalone modalities for emotion calculation, thereby confirming the efficacy of multimodal and subjective data collaboration in emotion classification.

4.2. Classifier Generalization Verification

In this study, Village 1 and Village 2 were used for training, while Village 3 served as the validation set. The performance of classifiers across binary, ternary, and quintuple tasks (XGBoost, RF, DT, LR-GD) was systematically compared. To enhance the comparability and practical applicability of model validation, PCA feature extraction was applied consistently across classifiers. After reducing the number of features from 47 to 36, the model’s recognition accuracy increased by an average of 5.2% for valence and 6.4% for arousal, with other performance metrics also improving. These findings demonstrate that the PCA algorithm effectively reduces data redundancy and noise, thereby enhancing model classification capabilities. However, obtaining a sufficient number of meaningful features remains a challenge, one that requires further academic consensus through extensive experimentation. Furthermore, the integration of SMOTE and Bayesian optimization techniques was employed to compare the performance of integrated versus single classifiers. The results showed that, for valence, the highest binary classification accuracy was 84%, ternary classification accuracy was 77.2%, and quintuple classification accuracy was 64%. For arousal, the highest accuracy was 77.9% for binary classification, 74.3% for ternary classification, and 59.6% for quintuple classification. Notably, the integrated classifiers exhibited superior performance in binary and ternary classification, with higher performance metrics, while the quintuple classification accuracy was notably lower—64% for both valence and arousal—compared to 79% achieved by Kalimeri and Saitis [80]. This discrepancy is attributed to the varied emotional responses elicited by image-based experiments versus real-world scenarios [81], an issue that will be further investigated through virtual scene construction or field experiments in the future. Furthermore, the proposed model underwent external validation to address the limitations associated with deriving both training and validation data from the same image set. When new data were introduced into the model, a decline in performance was observed, with average decreases of 4.25% in valence classification accuracy and 3.31% in arousal. These findings underscore the necessity of incorporating external validation, as relying solely on a unified dataset is insufficient for robust classification research. Additionally, it was observed that the average accuracy for valence exceeded that of arousal by 7.4% across binary, ternary, and quintuple classifications, supporting the weak V-shaped relationship and individual differences identified by Kuppens et al. [82] in their analysis of average relationships.

4.3. Enrich Sustainable Assessment Methods

Building on the analysis above, this study has been validated by constructing a multi-source emotion model that integrates eye movement, EEG, and landscape beauty questionnaires into a unified framework. This approach enriches the interdisciplinary methodology for assessing rural landscape sustainability. In practical terms, the model provides innovative solutions to address the three major challenges in rural revitalization, as outlined in Section 3.3 First, by establishing the spatial coupling relationship between eye movement, EEG, beauty assessment questionnaires, and the SAM emotion scale, we can accurately identify traditional villages that require protection, thus preventing the destruction of cultural heritage caused by the “largescale demolition and construction” transformation. Second, dynamic monitoring, based on an emotional feedback database, enables the establishment of a tiered update mechanism—comprising “negative spatial early warning, moderate intervention, and positive maintenance”—that overcomes time-domain constraints and facilitates real-time monitoring of emotional responses to rural landscape designs. Third, the model can be seamlessly integrated with GIS systems to transcend the limitations of point-based analysis, enhancing the ecological perception of tourists’ experiences. Finally, the visual emotional responses are translated into quantifiable landscape indicators through the affective computing model, enabling rural landscape designers to better contribute to SDG 11.7 (providing inclusive green public spaces) through the quantitative insights offered by the multi-source affective model.

4.4. Limitations and Challenges

Although this study has established a foundational framework for multi-source sentiment computing in rural visual landscapes, several notable limitations remain. First, the geographic concentration on three villages in Dalian constrains the generalizability of cultural and ecological insights to other regions. Future research should therefore focus on diversifying application contexts and expanding the geographical scope. Second, the participant pool—comprising exclusively students and teachers—exhibited a degree of homogeneity. Subsequent studies should include a broader demographic spectrum, such as rural residents and tourists, to more accurately evaluate the model’s applicability across different user groups and to strengthen the generalizability of the findings. Finally, the current experiment employed two-dimensional images as stimuli, limiting emotional extraction to flat visual environments. To improve the ecological validity of emotional assessments, future investigations should consider developing a multimodal virtual or augmented reality platform that incorporates environmental variables such as seasonal transitions and dynamic lighting conditions. This would foster more immersive experimental settings and enable dynamic, adaptable evaluations. Collectively, these advancements will offer critical technical support for refining the emotional assessment framework and enhancing its applicability in complex real-world environments.

5. Conclusions

Despite the aforementioned limitations, the rural visual landscape evaluation based on multi-source emotion computing effectively meets the decision-making requirements for rural landscape renewal through binary and ternary emotional assessments. Whether utilizing multimodal physiological sensors or various questionnaire evaluations, assessing rural visual landscapes across different regions, styles, and functions has been a contentious issue in rural revitalization. This study primarily focuses on enhancing the adaptability and classification performance of the proposed model. To achieve a more versatile model, data were collected from four distinct landscape types across three villages, ensuring the diversity of spatial data. Furthermore, efficient feature reduction, the SMOTE algorithm, Bayesian optimization, and ensemble learning techniques were employed to enhance the classification accuracy of the model. This paper also compares the performance of binary, ternary, and five-element classification models. Ultimately, accuracy comparisons reveal that the binary and ternary classification models outperform the five-element model in fulfilling practical requirements.

In future research, we aim to explore long-term emotional assessment, dynamic evaluation, and the integration of virtual and real-world emotional quality evaluations. By leveraging multi-source signal extraction and advanced machine learning technologies, we seek to continuously enhance the performance of the multi-source emotion computing model, thereby providing technical support for the development of rural visual landscapes.