Do Spanish Affective Picture Norms Transfer to Russian Speakers? Valence and Arousal Ratings of a Low-Level Matched Subset of the EmoMadrid Database

Featured Application

The EMselec subset of the EmoMadrid database can be used with Russian-speaking participants in protocols that contrast negative, neutral and positive pictures. When pictures are selected by their absolute valence or arousal values, the Spanish norms should be recalibrated with the category-specific differences reported here. Negative and positive pictures should not be assumed to be matched in arousal, and neutral and positive pictures should not be ranked by their Spanish valence norms.

Abstract

Norms of standardised affective picture sets obtained in one sample may not transfer to another. We examined whether the EMselec subset of the EmoMadrid database (40 negative, 40 neutral and 40 positive pictures matched for luminance, spatial frequency and chromatic complexity) retains its normative properties in a Russian-speaking sample. Sixty-three participants (M = 25 years, SD = 6; 43 women) rated the pictures online on five-point valence and arousal scales under the exposure parameters of the Spanish norms. Rank-order agreement was high across all pictures (valence ρ = 0.91; arousal ρ = 0.86) and lower within categories (arousal ρ = 0.40 to 0.73; neutral and positive valence ρ = 0.27 and 0.07). Absolute agreement was good for valence (ICC = 0.89) and imprecisely estimated for arousal (ICC = 0.70, 95% CI [−0.02, 0.89]). Arousal ratings were lower in all categories, by 0.24 to 0.59 points. Valence ratings were compressed towards the middle of the scale (calibration slope 0.68), and the overall valence difference was not significant. Unlike in the Spanish norms, negative pictures were rated as more arousing than positive ones (difference 0.35), so their arousal matching did not hold. EMselec keeps its category structure, but its absolute norms require local recalibration.

1. Introduction

Emotions organise much of human behaviour, and the induction and measurement of affective states are standard methods in affective science. Stimuli used for this purpose have to meet the technical requirements of the recording method and should produce comparable effects in different populations. Emotions are commonly induced with pictures of objects, scenes and faces [1,2,3], video clips [4,5,6], sounds and music [7,8,9], emotional words and phrases [10,11], and virtual reality and video games [12,13,14]. Static emotional scenes are widely used in laboratory research because they are relatively easy to standardise and can be used in many classical paradigms, such as oddball and affective priming tasks. Which stimuli are suitable depends on the method and the paradigm. Studies that record brain activity with functional magnetic resonance imaging (fMRI), electroencephalography (EEG), magnetoencephalography (MEG) or functional near-infrared spectroscopy (fNIRS) also have to control stimulus properties that affect the recorded signal [15].

Visual affective stimuli range from symbolic (words, sentences, pictograms) to non-symbolic (faces and scenes), and non-symbolic stimuli can be static or dynamic. Static scenes are the most widely used, because they are easy to standardise, present and replicate, and their low-level properties (brightness, contrast, spatial frequency) can be controlled, which makes valence and arousal ratings more comparable across studies [1,2,16,17,18]. In neurophysiological experiments this control is essential, because uncontrolled differences in visual parameters can affect the amplitudes and latencies of event-related potentials (ERPs) [19]. Faces are processed by specialised neural networks, so responses to them are difficult to compare with responses to other stimuli [20,21]. Video clips have higher ecological validity, but their perceptual characteristics are much harder to standardise and control than those of static scenes [22,23]. Static scenes are also more effective in inducing specifically negative affect, whereas videos evoke a broader and less specific range of experiences [24]. Static pictures are therefore often chosen as a compromise between ecological and affective validity, provided that they are carefully selected and standardised.

Most visual stimulus databases, however, do not control the low-level visual parameters of their images (brightness, spatial frequency and colour complexity), although these parameters matter in neurophysiological experiments such as ERP studies [25,26]. Uncontrolled visual differences between stimuli can confound measures of neural activity, especially at early processing stages that depend mainly on sensory information. To address this problem, the Brain, Affect and Cognition research group (Cerebro, Afecto y Cognición, CEACO) of the Universidad Autónoma de Madrid created the EmoMadrid database of emotional scenes [15]. The database contains 813 pictures with normative valence and arousal ratings. For each picture it also reports brightness, spatial frequency and colour complexity, so that stimuli can be selected for methods that are sensitive to these parameters, including EEG. The database is freely available for research, which makes results easier to reproduce and compare between laboratories.

Emotional experience is commonly described within a dimensional framework defined by valence (unpleasant–pleasant) and arousal (calming–activating) [27]. Some models add further dimensions such as dominance [28], but most affective picture sets are normed on valence and arousal. Cross-cultural studies show that the relative evaluations of stimuli often agree between samples, whereas the absolute levels of valence and arousal can differ [29]. This combination of similar ordering and different absolute values is the main reason why affective picture sets need to be validated when they are used in a new population.

The EmoMadrid norms were also collected with short exposures, which are typical of neurophysiological studies [30]. Each picture was shown for 1 s and rated during the following 4 s [15]. In the normative procedures of the International Affective Picture System (IAPS) and the Nencki Affective Picture System (NAPS), pictures were shown for 6 s [31] and 3 s, respectively, and in NAPS a reduced copy of the picture remained on the screen during the ratings [18]. The EmoMadrid presentation conditions are therefore closer to those of typical event-related potential studies. The database also codes the social content of the scenes, for example the presence of people or animals, so that the content of the stimuli can be varied systematically, and its pictures were chosen to elicit emotion without relying on clichéd or outdated scenes. These properties make EmoMadrid suitable for studies that need to separate the sensory and emotional contributions to stimulus processing [15,19,32].

Standardised picture databases such as the IAPS, the Geneva Affective Picture Database (GAPED), NAPS and the Open Affective Standardized Image Set (OASIS) have been widely used in research on affective processing. However, when their pictures were rated in other countries, the ratings often differed from the original norms. Adaptations of the IAPS outside the United States, reviewed by Branco et al. [33], show that the valence and arousal values of the original norms shift when the pictures are rated in other countries, for example in Brazil, Chile and China. These studies are directly relevant here, because they show that the norms of a widely used picture set do not transfer automatically to a new national sample, which is the question addressed for EMselec in the present study. Emotional expressiveness, the tendency to give extreme ratings and norms of emotional expression also vary across cultures [34,35]. In a 23-country study of emotion regulation, Matsumoto et al. [36] found that suppression of emotional expression was more common in cultures that emphasise social order and hierarchy and less common in cultures that value affective autonomy and egalitarianism. Russian participants, in particular, have been found to inhibit the expression of positive emotions towards strangers [37]. Differences in response style between the Russian-speaking and the Spanish samples can therefore be expected a priori.

Along with behavioural differences, the linguistic representation of emotions also varies: the same emotional states can be described with different words or may not have exact equivalents [38,39]. For example, Vietnamese speakers distinguish between ‘shame’ and ‘suffering’ as separate categories, whereas in English, they often overlap [40]. These factors can also influence the neurophysiological processing of stimuli. Perception of consonance and dissonance, for example, depends on enculturation as well as on the physical properties of sound [41,42,43]. Even formally standardised stimulus sets such as EmoMadrid therefore need to be validated when they are used in a new linguistic and cultural environment.

The present study used EMselec, a subset of EmoMadrid pictures selected by the database authors so that the negative, neutral and positive categories do not differ in spatial frequency, brightness (luminance) or chromatic complexity [15]. Differences in these parameters could otherwise affect both the affective ratings and the neuroimaging data. The pictures were downloaded from the EmoMadrid website [44] on 4 April 2025.

Three considerations frame the present comparison. The first concerns the balance between universal and culture-specific components of affective responses. The valence and arousal dimensions describe affective experience across cultures [34], whereas emotional expressiveness, response styles and norms of emotional expression differ between them [34,35,36]. The second concerns the level at which normative ratings are compared. A stimulus set may keep the relative ordering of its pictures in a new sample while its absolute values shift or are compressed [29], and the two properties matter for different designs: contrasts between affective categories depend on the ordering, whereas the selection or matching of pictures by their normative values depends on the absolute ratings. The third concerns recalibration. When a normative set is used in a new linguistic and cultural setting, its absolute values, and the matching criteria on which subsets such as EMselec are built, have to be checked in the new population before they can be relied upon [33].

This study had two aims: (1) to compare valence and arousal ratings between a Spanish reference sample and a Russian-speaking sample, and (2) to determine which properties of the Spanish EMselec norms can be used with Russian-speaking participants without recalibration [15,33]. The study concerns the applicability of EMselec in a new linguistic and cultural setting, since the capacity of the pictures to elicit emotions was established in the original normative study [15].

2. Materials and Methods

Figure 1 summarises the study. Of the 70 volunteers who took part, 63 were included in the analyses (Section 2.1). Each participant rated the EMselec pictures (Section 2.2) online in two blocks, one for valence and one for arousal, presented in counterbalanced order (Section 2.3). The ratings were compared with the Spanish norms at the picture level and analysed at the participant level within the Russian sample (Section 2.4).

Figure 1. Overview of the study. (a) Workflow: recruitment and exclusions, the online rating session and the picture-level and participant-level analyses. (b) Trial structure. Each picture was shown for 1000 ms and followed by a 4000 ms response screen showing the picture number and the rating scale of the current block (valence or arousal; the valence block is shown). Participants rated each picture by pressing keys 1 to 5. The pictures shown are examples of positive, neutral and negative EMselec pictures; the pictures were presented in random order within each block.

2.1. Participants

The final sample consisted of 63 participants aged 18 to 45 years (M = 25, SD = 6), of whom 43 were women. Of the 70 volunteers who took part, 7 were excluded because they did not complete either block, owing to technical problems or because they stopped participating. Valence ratings were available from 59 participants and arousal ratings from 62, because 4 participants completed only the arousal block and 1 only the valence block. Participation was voluntary. The sample size follows the recommendation of the database authors for contributed norms [45].

The Russian ratings were compared with the normative values of the EmoMadrid database [15]. These norms were collected between 2006 and 2018 from 768 undergraduate psychology students at three Spanish universities, aged 17 to 50 years (M = 20.1, SD = 3.4), of whom 593 were women. Each participant rated only part of the database (on average 155 pictures), so the number of ratings varies across pictures [15]. For the EMselec pictures, the Spanish norms are based on 104 to 319 ratings per picture for valence (median 178) and 103 to 322 for arousal (median 179). The demographic characteristics above refer to the normative sample of the database as a whole and are not reported separately for the raters of individual pictures. Participants rated the pictures in group sessions of 17 to 44 people in a quiet, dimly lit room, with the pictures projected onto a screen, using paper forms until 2008 and individual computers afterwards [15]. The rating scales, exposure parameters and block structure of the present study followed this procedure. The two samples differed in testing environment, display equipment and period of data collection and, judging by the database sample as a whole, probably also in age, sex composition and student status (Table 1).

Table 1. Characteristics of the EmoMadrid normative sample and the Russian-speaking sample.

Characteristic	EmoMadrid Normative Sample, Whole Database [15]	Russian-Speaking Sample (Present Study)
Participants	768 in total; each rated a part of the database (on average 155 pictures)	63 (valence 59, arousal 62); each rated all 120 pictures
Ratings per EMselec picture	104–319 (valence), 103–322 (arousal)	53–59 (valence), 56–62 (arousal)
Age, years	17–50 (M = 20.1, SD = 3.4) a	18–45 (M = 25, SD = 6)
Sex	593 women, 175 men a	43 women, 20 men
Population	Undergraduate psychology students, three universities a	Adult volunteers recruited online
Period of data collection	2006–2018	2025
Setting	Group sessions of 17–44 people, quiet dimly lit room	Individual, online (Pavlovia)
Display	Projection screen, computer screen	Participants’ own computers, not controlled
Response recording	Paper forms until 2008, then individual computers	Keyboard keys 1–5
Exposure/response window	1 s/4 s	1 s/4 s
Scales	Valence and arousal, 5 points, −2 to +2	Same, labels translated into Russian
Dimensions	Separate runs, order counterbalanced	Separate blocks, order counterbalanced
Picture order	Randomized per participant and block	Randomized per participant and block

Note. Characteristics of the Spanish sample are taken from the original database publication [15]. ^a Values refer to the normative sample of the whole database; the composition of the participants who rated the EMselec pictures is not reported.

2.2. Materials

The study used the EMselec subset of the EmoMadrid database [15], which comprises 120 pictures, 40 in each of the negative, neutral and positive categories. The subset was designed to minimise differences in physical image properties between the affective categories. According to the validation data reported by Carretié et al. [15], the categories do not differ significantly in spatial frequency, luminance, chromatic complexity or contrast (one-way ANOVA with Tukey HSD post hoc comparisons). In the Spanish norms, valence ratings differ significantly between all three categories, whereas arousal ratings do not differ between negative and positive pictures and are higher for both than for neutral pictures.

2.3. Procedure

The experiment was programmed in PsychoPy 2024.2.4 (Open Science Tools Ltd., Nottingham, UK) and ran online on the Pavlovia platform (Open Science Tools Ltd., Nottingham, UK; https://pavlovia.org). Participants took part from their own desktop or laptop computers. A physical keyboard was required because responses were given with the number keys 1 to 5, so the task could not be completed on smartphones or tablets. Screen size, resolution, brightness and colour rendering were not controlled. The physical matching of EMselec refers to the image files, and each participant saw all pictures on the same display. Because the categories are matched in luminance and colour, uncontrolled display properties are unlikely to have produced systematic differences between the categories within the Russian sample. Display conditions did, however, differ between the Russian and Spanish samples (Table 1), so part of the differences in absolute ratings between the samples may reflect display conditions rather than language or culture.

The experiment consisted of two blocks. In one block participants rated all pictures for valence and in the other for arousal. The order of the blocks was counterbalanced, so that half of the participants started with the valence block and half with the arousal block. Separate blocks were used to reduce a halo effect between the two dimensions, which can arise when they are rated together [46].

Each picture (1024 × 768 pixels) was presented for 1 s, followed by a 4 s response interval (Figure 1b). During the interval, a blank background was presented with the number of the previous image in the centre and the rating scale. Participants pressed a key (1, 2, 3, 4 or 5) corresponding to their rating within the 4 s interval. The valence scale was labelled as 1 = very negative, 2 = negative, 3 = neutral, 4 = positive and 5 = very positive, and the arousal scale as 1 = very calming, 2 = calming, 3 = neutral, 4 = arousing and 5 = very arousing. The Russian instructions and labels are given in Supplementary Table S2. Key presses were recoded as the key value minus 3, so that both scales ranged from −2 to +2 with 0 as the neutral point, as in the EmoMadrid norms. The Spanish normative values are reported on the same scale [15] and were used without transformation. The order of pictures was randomized independently for each participant and each block. A short break of about 15 s was provided after every 25 images. Before the main procedure, participants completed a practice block with several trial images. The instructions were translated into Russian by the authors and checked by a second author. They were presented as text in the PsychoPy interface before the experiment began.

2.4. Statistical Analysis

The main analyses were conducted at the picture level. Each picture contributed one mean rating per sample and dimension, which gives 240 observations per dimension. Because the Spanish norms are available only as picture-level means and standard deviations, these analyses treat pictures, not participants, as the unit of replication. Their p values and effect sizes therefore describe differences between the two sets of ratings of these pictures, not differences between the populations from which the samples were drawn. The Russian data were additionally analysed at the level of individual ratings (see the participant-level analysis below). Analyses were performed in Positron 1.103.0 (Posit Software, PBC, Boston, MA, USA) with R 4.3.0 (R Foundation for Statistical Computing, Vienna, Austria), using the packages lme4 1.1-34, lmerTest 3.1-3, emmeans 1.8.7, effectsize 1.0.1, performance 0.14.0, irr 0.84.1, car 3.1-2, nlme 3.1-163, ordinal 2026.7-26 and tidyverse 2.0.0.

Rank-order correspondence between the Russian-speaking sample and the Spanish normative values was assessed with Spearman’s rank correlation coefficient (ρ); because of tied ranks, p-values are based on the normal approximation. Correlations were computed across all pictures and separately within each affective category. To remove the contribution of the separation between categories, we also computed a partial rank correlation, the Pearson correlation between the ranks of the Russian and Spanish values after the category means of the ranks had been removed. Confidence intervals (95%) were obtained with the Fisher z transformation, using the standard error of Bonett and Wright for Spearman coefficients [47]. Absolute agreement was quantified with intraclass correlation coefficients estimated in a two-way random-effects model for single measurements with absolute agreement, ICC(A,1) in the notation of McGraw and Wong [48], reported with 95% confidence intervals and interpreted as poor below 0.50, moderate from 0.50 to 0.75, good from 75 to 0.90 and excellent above 0.90 [49].

Systematic shift was examined with the method of Bland and Altman. For each picture, the Russian minus Spanish difference was computed, and the mean difference was tested against zero with a one-sample t test. Limits of agreement are reported as the mean difference ± 1.96 SD, together with 95% confidence intervals for the mean difference and for each limit [50]. Category-dependent differences were modelled with linear mixed-effects models fitted separately for valence and arousal, with random intercepts for stimuli (EM_CODE), accounting for the non-independence of observations within images. The following model was applied:

$$Y_{sk} ={ \beta }_0 + {\beta }_1{Sample}_s + {\beta }_2{Affect}_k + {\beta }_3{\left(Sample\times Affect\right)}_{sk} +{ u}_k +{ \epsilon }_{sk},$$

(1)

where $Y_{sk}$ is the mean rating of picture $k$ in sample $s$, ${Affect}_k$ is the affective category of picture $k$, $u_k$ is the random intercept for picture $k$ (EM_CODE) and ${\epsilon }_{sk}$ is the residual.

Sample (Russian-speaking vs. Spanish normative) and Affect (Positive, Neutral, Negative) were specified as fixed effects. Models were estimated by restricted maximum likelihood in the lme4 package, with tests of fixed effects performed using the lmerTest package. Type III sums of squares were used, with the Kenward–Roger approximation for degrees of freedom.

Effect sizes (partial η²) were computed from F-statistics using the effectsize package, are reported with 90% confidence intervals, and were interpreted according to conventional benchmarks (0.01 small, 0.06 medium, ≥0.14 large).

To assess differences between samples within each affect category, pairwise contrasts were computed from estimated marginal means and are expressed throughout as Russian minus Spanish values. The three between-sample contrasts of each dimension were treated as one family, and their p values and confidence intervals were Bonferroni-adjusted for three comparisons. The effect size for each contrast was expressed as a correlation coefficient,

$$r=\sqrt{{\displaystyle \frac{t^2}{t^2+df}}},$$

(2)

which provides direct interpretability on a scale from 0 to 1 [51]. Degrees of freedom for the contrasts and marginal means were obtained with the Kenward–Roger method. The same model structure was additionally applied to the within-picture standard deviation of ratings as an index of rater dispersion, with the same adjustment of the contrasts.

Model fit was evaluated using marginal ($R_m^2$) and conditional ($R_c^2$) coefficients of determination following Nakagawa and Schielzeth [52], reflecting variance explained by fixed effects alone and by the full model, respectively.

Residual normality was assessed with the Shapiro–Wilk test. Constancy of error variance was assessed with the Breusch–Pagan score test [53], which relates the residual variance to the fitted values (check_heteroscedasticity in the performance package), and with Levene’s test across affective categories. Because the residuals of mixed models are not independent, the Breusch–Pagan test is only approximate in this setting. Both models were therefore also refitted with a separate residual variance for each Sample × Affect cell (varIdent variance structure, nlme package) and compared with the common-variance models by likelihood ratio tests.

To distinguish a shift in the rating scale from a compression of its range, the Russian picture means were regressed on the Spanish norms separately for each dimension:

$${RU}_k = a+b{ES}_k + e_k,$$

(3)

where $k$ indexes pictures, ${RU}_k$ and ${ES}_k$ are the Russian and Spanish means of picture $k$, and $e_k$ is the residual. Three models were compared: a shift-only model with $b$ fixed at 1, a compression-only model with $a$ fixed at 0, the neutral point of the scale, and a model with both parameters free. A slope below 1 indicates a compression of the range, and a non-zero intercept a displacement of the scale. The models were compared with AIC and BIC and with F tests of the nested models, and an interaction between the Spanish norm and Affect was added to test whether the slope differed between categories.

For the Russian sample, each individual rating was also expressed as its deviation from the Spanish norm of the same picture and analysed with a linear mixed model,

$$D_{pk}=R_{pk}-{ES}_k={\gamma }_0+{\gamma }_1{Affect}_k+{ v}_p+u_k+{\epsilon }_{pk},$$

(4)

where $R_{pk}$ is the rating of picture $k$ by participant $p$, ${ES}_k$ is the Spanish normative mean of picture $k$, and $v_p$ and $u_k$ are random intercepts for participants and pictures. In this model the Spanish norms act as a fixed reference, and the category means of $D$ estimate the between-sample differences while taking into account the variability among Russian participants and among pictures. Because of the number of observations, confidence intervals for these means are asymptotic (Wald) intervals. As a sensitivity analysis for the ordinal nature of the ratings, a cumulative link (proportional odds) mixed model with Affect as the predictor and random intercepts for participants and pictures was fitted to the raw Russian ratings (ordinal package). The Spanish sample could not be included in these models, because its data are available only as picture-level summaries.

3. Results

3.1. Agreement Between Valence and Arousal Ratings

Figure 2 shows the distributions of the picture means in the two samples. Each picture mean is an average of 53 to 62 discrete ratings in the Russian sample and 103 to 322 in the Spanish norms and can therefore take intermediate values on the −2 to +2 scale. The distributions have one peak for each affective category and deviate from normality, so rank-order agreement between the Russian (RU) and Spanish (ES) values was assessed with Spearman’s correlation coefficient and absolute agreement with intraclass correlation coefficients. The relation between the two sets of values is shown in Figure 3.

Figure 2. Distributions of picture mean ratings for (a) arousal and (b) valence in the Spanish normative sample (upper row) and the Russian-speaking sample (lower row). Each bar shows the number of pictures whose mean rating falls in an interval of 0.1 scale points; colours indicate the affective category. Russian arousal means lie below the Spanish norms in all categories, and Russian valence means cover a narrower range of the scale.

Figure 3. Russian picture means plotted against the Spanish normative values for (a) arousal and (b) valence. Each point is one picture, coloured by affective category (negative, neutral, positive). Each panel reports Spearman’s ρ and ICC(A,1). The diagonal line indicates equal ratings in both samples.

Rank-order agreement between samples was high for valence (ρ = 0.91, 95% CI [0.86, 0.94], S = 26,082, p < 0.001) and arousal (ρ = 0.86, 95% CI [0.80, 0.91], S = 39,009, p < 0.001). Within categories, the correlations were lower (Table 2). For arousal they ranged from ρ = 0.40 for neutral to ρ = 0.73 for negative pictures, and the partial correlation controlling for category was ρ = 0.62 (95% CI [0.49, 0.72]). For valence, the within-category correlation was ρ = 0.67 for negative pictures but 0.27 for neutral and 0.07 for positive pictures, with confidence intervals that include zero, and the partial correlation was ρ = 0.34 (95% CI [0.17, 0.49]). The high overall valence correlation therefore mainly reflects the separation between categories. Part of this attenuation is expected, because the Spanish valence norms vary little within the neutral and positive categories (SD of picture means 0.16 and 0.17; Supplementary Table S4). Absolute agreement, estimated as ICC(A,1), was good for valence (ICC = 0.89, F(119, 115) = 16.7, p < 0.001, 95% CI [0.84, 0.92]) and imprecisely estimated for arousal (ICC = 0.70, 95% CI [−0.02, 0.89], F(119, 2.96) = 13.2, p = 0.028) (Figure 3). The confidence interval for arousal is wide and extends below zero. This instability follows from the systematic shift reported below. Absolute-agreement coefficients penalise a constant difference between raters, and with a shift of this size relative to the variance between pictures, the approximate denominator degrees of freedom of the F test become small. The arousal coefficient should therefore be read as an approximate value.

Table 2. Spearman correlations between the Russian picture means and the Spanish normative values, across all pictures and within affective categories.

Pictures	n	Valence ρ [95% CI]	Arousal ρ [95% CI]
All	120	0.91 [0.86, 0.94]	0.86 [0.80, 0.91]
Negative	40	0.67 [0.42, 0.82]	0.73 [0.51, 0.86]
Neutral	40	0.27 [−0.05, 0.54]	0.40 [0.09, 0.64]
Positive	40	0.07 [−0.25, 0.37]	0.64 [0.38, 0.80]
Partial, category removed	120	0.34 [0.17, 0.49]	0.62 [0.49, 0.72]

Note. Confidence intervals were obtained with the Fisher z transformation (standard error of Bonett and Wright [47] for Spearman coefficients). The partial coefficient is the correlation between the ranks after removal of the category means of the ranks. p < 0.001 for all coefficients except neutral valence (p = 0.090), positive valence (p = 0.67) and neutral arousal (p = 0.011).

Agreement between the arousal ratings of the two samples was examined with a Bland–Altman plot (Figure 4a). The mean difference was −0.40 (95% CI [−0.46, −0.35]), SD = 0.31, and the 95% limits of agreement ranged from −1.01 (95% CI [−1.11, −0.92]) to 0.21 (95% CI [0.11, 0.31]). Negative and positive pictures were more widely scattered than neutral pictures.

Figure 4. Bland–Altman plots for (a) arousal and (b) valence. Each point is one picture, coloured by affective category. The x-axis shows the mean of the Russian and Spanish values, and the y-axis shows their difference (Russian–Spanish). The solid line indicates the mean difference, the dashed lines indicate the 95% limits of agreement (mean difference ± 1.96 SD), and the grey line marks zero.

For valence (Figure 4b), the mean difference was +0.08 (95% CI [−0.004, 0.16]; t(119) = 1.88, p = 0.063), SD = 0.45, and the 95% limits of agreement ranged from −0.81 (95% CI [−0.95, −0.67]) to 0.97 (95% CI [0.82, 1.11]). The differences formed three clusters corresponding to the affective categories.

3.2. Linear Mixed-Effects Regression Analysis

The effects of Sample (Russian-speaking vs. Spanish normative), affective category (positive, neutral, negative) and their interaction on the mean arousal ratings of the pictures were examined with a linear mixed-effects model with a random intercept for each picture (Equation (1); Table 3).

Table 3. Results of the two-way analysis of variance (Type III) for arousal: effects of Sample, Affect, and their interaction.

Effect	F(df1, df2)	p	η2p
Sample	250.96 (1, 117)	<0.001	0.68 [0.63, 1.00]
Affect	179.70 (2, 117)	<0.001	0.75 [0.71, 1.00]
Sample × Affect	15.98 (2, 117)	<0.001	0.21 [0.13, 1.00]

Note. η²_p = partial eta squared, with 90% confidence intervals in brackets; df = degrees of freedom. Upper bounds are fixed at 1.00 because the intervals are one-sided. No correction for multiple comparisons applies to omnibus effects.

The main effect of Sample was significant—F(1, 117) = 250.96, p < 0.001, η²_p = 0.68, 90% CI [0.63, 1.00]—with lower arousal ratings in the Russian-speaking sample. The effect of Affect was also significant—F(2, 117) = 179.70, p < 0.001, η²_p = 0.75, 90% CI [0.71, 1.00]. A significant Sample × Affect interaction (F(2, 117) = 15.98, p < 0.001, η²_p = 0.21, 90% CI [0.13, 1.00]) indicated that the magnitude of between-sample differences varied depending on the affect category.

Bonferroni-adjusted contrasts (Table 4; Figure 5a) showed that arousal ratings were lower in the Russian-speaking sample in all three categories, although the size of the difference depended on the category. The difference was largest for positive pictures (Mdiff = −0.59, t(117) = −13.37, padj < 0.001, r = 0.78), intermediate for negative pictures (Mdiff = −0.38, t(117) = −8.66, padj < 0.001, r = 0.62) and smallest for neutral pictures (Mdiff = −0.24, t(117) = −5.42, padj < 0.001, r = 0.45).

Table 4. Estimated marginal means and between-sample contrasts (RU–ES) of arousal within each affective category (Bonferroni adjustment for three contrasts).

Affect	M RU [95% CI]	M ES [95% CI]	Mdiff	95% CI adj.	t(117)	padj	r
Positive	0.44 [0.35, 0.54]	1.03 [0.93, 1.13]	−0.59	[−0.70, −0.48]	−13.37	<0.001	0.78
Neutral	−0.27 [−0.37, −0.17]	−0.03 [−0.13, 0.06]	−0.24	[−0.35, −0.13]	−5.42	<0.001	0.45
Negative	0.79 [0.70, 0.89]	1.18 [1.08, 1.27]	−0.38	[−0.49, −0.27]	−8.66	<0.001	0.62

Note. M RU and M ES = estimated marginal means with unadjusted 95% confidence intervals. Mdiff = contrast RU–ES; positive values indicate higher arousal ratings in the Russian sample. Degrees of freedom were obtained with the Kenward–Roger method (df = 117 for all contrasts). The confidence intervals of Mdiff and padj are Bonferroni-adjusted for the three between-sample contrasts of this dimension. Effect size r was computed with Equation (2).

Figure 5. Mean ratings with 95% confidence intervals for each affective category in the Russian-speaking (RU, solid line) and Spanish normative (ES, dashed line) samples for (a) arousal and (b) valence.

The marginal and conditional coefficients of determination were $R_m^2 = 0.742$ and $R_c^2 = 0.900$. Residuals were normally distributed (Shapiro–Wilk W = 0.995, p = 0.54), and the Breusch–Pagan test did not indicate non-constant error variance (p = $0$.94). A model with separate residual variances for the six Sample × Affect cells did not improve the fit (likelihood ratio = 4.83, df = 5, p = $0$.44), and all effects and contrasts were unchanged (Supplementary Table S4).

The same model was fitted to the valence ratings (Table 5).

Table 5. Results of the two-way analysis of variance (Type III) for valence: effects of Sample, Affect, and their interaction.

Effect	F(df1, df2)	p	η2p
Sample	9.07 (1, 117)	0.003	0.07 [0.02, 1.00]
Affect	888.03 (2, 117)	<0.001	0.94 [0.93, 1.00]
Sample × Affect	94.16 (2, 117)	<0.001	0.62 [0.55, 1.00]

Note. η²_p = partial eta squared, with 90% confidence intervals in brackets; df = degrees of freedom. Upper bounds are fixed at 1.00 because the intervals are one-sided. No correction for multiple comparisons applies to omnibus effects.

Bonferroni-adjusted contrasts (Table 6; Figure 5b) showed that the direction of the between-sample difference reversed across categories. Russian valence ratings were lower than the Spanish norms for positive pictures (Mdiff = −0.40, t(117) = −8.90, padj < 0.001, r = 0.64) and higher, that is, less negative, for negative pictures (Mdiff = +0.45, t(117) = 10.11, padj < 0.001, r = 0.68), with a small upward difference for neutral pictures (Mdiff = +0.18, t(117) = 4.01, padj < 0.001, r = 0.35). The Russian sample therefore used a narrower portion of the valence scale at both poles, which is tested directly in Section 3.3. Averaged over all pictures, the difference was small and not significant (+0.08; Section 3.1), because effects of opposite sign cancel at the aggregate level.

Table 6. Estimated marginal means and between-sample contrasts (RU–ES) of valence within each affective category (Bonferroni adjustment for three contrasts).

Affect	M RU [95% CI]	M ES [95% CI]	Mdiff	95% CI adj.	t(117)	padj	r
Positive	0.86 [0.77, 0.94]	1.26 [1.17, 1.34]	−0.40	[−0.51, −0.29]	−8.90	<0.001	0.64
Neutral	0.22 [0.14, 0.31]	0.04 [−0.04, 0.13]	+0.18	[0.07, 0.29]	4.01	<0.001	0.35
Negative	−0.90 [−0.98, −0.81]	−1.35 [−1.43, −1.26]	+0.45	[0.34, 0.56]	10.11	<0.001	0.68

Note. M RU and M ES = estimated marginal means with unadjusted 95% confidence intervals. Mdiff = contrast RU–ES; positive values indicate higher valence ratings in the Russian sample. Degrees of freedom were obtained with the Kenward–Roger method (df = 117 for all contrasts). The confidence intervals of Mdiff and padj are Bonferroni-adjusted for the three between-sample contrasts of this dimension. Effect size r was computed with Equation (2).

The marginal and conditional coefficients of determination were $R_m^2 = 0.919$ and $R_c^2 = 0.956$. Residual normality was violated (Shapiro–Wilk W = 0.982, p = 0.004), whereas the Breusch–Pagan test did not indicate non-constant error variance (p = 0.46). Levene’s test showed unequal variances across categories in both samples (Russian F(2, 117) = 9.28, p < 0.001; Spanish F(2, 117) = 11.28, p < 0.001). The standard deviations of picture means in each cell (Supplementary Table S4) show that this reflects the larger spread of negative pictures in both samples (Russian SD = 0.45, compared with 0.22 for neutral and 0.27 for positive pictures; Spanish 0.26, 0.16 and 0.17). A model with a separate residual variance for each Sample × Affect cell fitted better than the common-variance model (likelihood ratio = 45.88, df = 5, p < 0.001), and all effects kept their direction and significance (Sample × Affect F(2, 117) = 79.16, p < 0.001; all Bonferroni-adjusted between-sample contrasts p < 0.001; Supplementary Table S4).

3.3. Calibration of the Russian Ratings Against the Spanish Norms

Table 7 compares the three calibration models. For valence, the model with both parameters free fitted clearly better than the shift-only model (F(1, 118) = 164.87, p < 0.001) and somewhat better than the compression-only model (F(1, 118) = 7.27, p = 0.008; ΔAIC = 5.2). Its slope was 0.68 (95% CI [0.63, 0.73]) and its intercept 0.07 (95% CI [0.02, 0.13]). Russian valence ratings were thus compressed to about two thirds of the Spanish range, with a small upward displacement. The slope differed between categories (F(4, 114) = 5.33, p < 0.001), so the compression was not uniform across the scale. For arousal, the model with both parameters free also fitted best (shift-only model F(1, 118) = 43.22, p < 0.001; compression-only model F(1, 118) = 33.91, p < 0.001), with a slope of 0.74 (95% CI [0.67, 0.82]) and an intercept of −0.22 (95% CI [−0.29, −0.14]), and the slope again differed between categories (F(4, 114) = 7.45, p < 0.001). The arousal difference therefore combined a downward shift, a compression and a category-dependent component.

Table 7. Calibration models relating the Russian picture means to the Spanish norms (Equation (3)).

Dimension	Model	a [95% CI]	b [95% CI]	AIC	BIC
Valence	Shift only (b = 1)	0.08	1 (fixed)	153.53	159.11
	Compression only (a = 0)	0 (fixed)	0.68	55.79	61.36
	Shift and compression	0.07 [0.02, 0.13]	0.68 [0.63, 0.73]	50.62	58.98
Arousal	Shift only (b = 1)	−0.40	1 (fixed)	63.57	69.15
	Compression only (a = 0)	0 (fixed)	0.57	56.44	62.02
	Shift and compression	−0.22 [−0.29, −0.14]	0.74 [0.67, 0.82]	28.13	36.49

Note. a = intercept, b = slope; n = 120 pictures per dimension. Lower AIC and BIC indicate better fit. Test of b = 1: F(1, 118) = 164.87 (valence) and 43.22 (arousal), both p < 0.001; test of a = 0: F(1, 118) = 7.27, p = 0.008 (valence) and 33.91, p < 0.001 (arousal). Slope × Affect: F(4, 114) = 5.33, p < 0.001 (valence); F(4, 114) = 7.45, p < 0.001 (arousal).

A parallel analysis of the within-picture standard deviation of ratings showed that Russian ratings were, on average, more dispersed than the Spanish ratings, for valence (main effect of Sample F(1, 117) = 19.95, p < 0.001, η²_p = 0.15) and for arousal (F(1, 117) = 27.40, p < 0.001, η²_p = 0.19). For valence, the Sample × Affect interaction was significant (F(2, 117) = 4.81, p = 0.010, η²_p = 0.08), and the difference was reliable only for positive pictures (Mdiff = +0.15, 95% CI [0.08, 0.22], padj < 0.001, r = 0.43; negative +0.04, padj = 0.46; neutral +0.03, padj = 0.71). For arousal, the interaction was not significant (F(2, 117) = 0.72, p = 0.49), and the contrasts were +0.07 (padj = 0.073), +0.08 (padj = 0.016) and +0.12 (padj < 0.001) for negative, neutral and positive pictures. The Breusch–Pagan test indicated non-constant error variance in both dispersion models (p < 0.001), so these tests are approximate. Each picture received 53 to 59 valid valence ratings (median 56) and 56 to 62 valid arousal ratings (median 61) in the Russian sample (Supplementary Table S1).

3.4. Participant-Level Analysis of the Russian Ratings

The participant-level models included 6749 valence ratings from 59 participants and 7244 arousal ratings from 62 participants. The deviation of the Russian ratings from the Spanish norms depended on the affective category for both valence (F(2, 117) = 93.75, p < 0.001) and arousal (F(2, 117) = 16.02, p < 0.001). The estimated deviations were +0.45 (95% CI [0.35, 0.55]) for negative, +0.18 (95% CI [0.07, 0.28]) for neutral and −0.40 (95% CI [−0.50, −0.30]) for positive pictures for valence, and −0.38 (95% CI [−0.49, −0.28]), −0.24 (95% CI [−0.35, −0.14]) and −0.59 (95% CI [−0.70, −0.49]) for arousal (Supplementary Table S5). These estimates agree with the picture-level contrasts in Table 4 and Table 6, with somewhat wider confidence intervals. The cumulative link mixed models fitted to the raw Russian ratings gave the same pattern of category differences as the linear models. Valence ratings increased from negative to neutral to positive pictures, and arousal ratings were highest for negative, intermediate for positive and lowest for neutral pictures. All three pairwise differences had the same direction as in the linear models and were significant on both dimensions (all Tukey-adjusted p < 0.001), including the higher arousal of negative than positive pictures (Supplementary Table S5).

3.5. Affective Structure Within the Russian-Speaking Sample

Within the Russian sample, valence ratings followed the expected order (Supplementary Table S3). Negative pictures were rated lower than neutral pictures (difference −1.12, 95% CI [−1.29, −0.95]) and neutral pictures lower than positive pictures (−0.64, 95% CI [−0.81, −0.46]; negative vs. positive −1.76, 95% CI [−1.93, −1.58]; all Tukey-adjusted p < 0.001). The corresponding differences in the Spanish norms were −1.39, −1.21 and −2.61. For arousal, negative (+1.07, 95% CI [0.91, 1.23]) and positive (+0.72, 95% CI [0.56, 0.88]) pictures were rated as more arousing than neutral pictures, which reproduces the V-shaped profile of the Spanish norms. Unlike in the Spanish norms, where negative and positive pictures did not differ in arousal (0.14, 95% CI [−0.03, 0.31], p = 0.12), negative pictures were rated as more arousing than positive pictures in the Russian sample (0.35, 95% CI [0.19, 0.51], t(117) = 5.12, p < 0.001). The same difference was obtained in the participant-level model (0.35, 95% CI [0.19, 0.51]). The arousal matching of negative and positive pictures, one of the selection criteria of EMselec, therefore did not hold in the Russian-speaking sample.

3.6. Summary of Findings

Three results summarise the comparison. First, the category structure of EMselec was preserved in the Russian-speaking sample. The three categories were ordered in the same way as in the Spanish norms on both dimensions, and the ordering of individual pictures within categories was preserved for arousal (partial ρ = 0.62) but only weakly for valence (partial ρ = 0.34). Second, absolute ratings differed systematically. Arousal ratings were lower in all categories, by 0.24 to 0.59 points, and valence ratings were compressed towards the middle of the scale (calibration slope 0.68) with a small upward displacement. Third, negative pictures were rated as more arousing than positive pictures in the Russian sample (difference 0.35), so the arousal matching of these two categories did not hold. For the practical use of EMselec, these results mean that designs contrasting the three categories can use the set with Russian-speaking participants, whereas designs that select or match pictures by their absolute valence or arousal values, or that assume equal arousal of negative and positive pictures, should rely on the Russian values reported in Supplementary Table S1.

4. Discussion

The present study compared valence and arousal ratings of the EMselec subset of the EmoMadrid database in a Russian-speaking sample with the Spanish normative values. Across all pictures, rank-order correspondence was high on both dimensions (ρ = 0.91 and 0.86). Within categories it remained moderate to high for arousal, whereas for valence it was clear only for negative pictures, so the high overall valence correlation mainly reflects the separation of the three categories. Absolute ratings differed in dimension-specific ways. Arousal ratings were lower in the Russian sample in all three categories, and the size of the difference depended on the category. Valence ratings were less positive for positive pictures and less negative for negative ones, and the calibration analysis described this pattern as a compression of the range with a small upward displacement. High rank-order agreement together with lower absolute agreement is compatible with a systematic offset, a compression of the scale, a category-dependent bias or a combination of these, and the calibration models indicate a combination for both dimensions. The norms therefore need local recalibration, but the stimulus set itself remains usable.

The combination of a largely stable ordering of pictures and different absolute rating levels is compatible with a distinction between invariant and context-dependent components of affective evaluation. The high correlations between the samples are consistent with dimensional models of emotion [2], whereas the systematic differences in mean ratings may reflect how the two samples used the rating scales. Culturally embedded evaluative frameworks are one possible source of such differences, but the present design cannot separate them from the other differences between the samples listed in Table 1.

This pattern is also compatible with hierarchical models of affective processing, according to which the emotional significance of a stimulus is encoded early in relatively universal terms, whereas later evaluative judgements are shaped by higher-order cognitive and sociocultural factors [30,54]. From this perspective, the preserved ordering would reflect shared mechanisms of rapid affective evaluation, and the differences in absolute values would arise at later stages. Because the present study measured ratings only, it cannot identify the processing stage at which the differences arise. This question could be addressed with event-related potentials recorded to the same pictures in both samples.

From a constructivist perspective, these findings may also be understood in terms of the diversity of conceptualisations of affective experience, based on the arbitrary assignment of meanings. Whereas basic affective signals provide a common basis for experience (as discussed earlier), their classification and quantification depend on culture-specific semantic and interpretative systems [55,56]. Identical stimuli could then be mapped onto partially different rating scales in different communities, which would produce systematic differences in ratings.

The between-sample differences were more dispersed for valence than for arousal (SD of differences 0.45 vs. 0.31). For valence, the difference changed in size and sign across categories, whereas for arousal it had the same sign in all categories and differed only in size. Arousal is less clearly defined than valence, which is closely tied to approach and avoidance tendencies, and may therefore be construed differently across linguistic and cultural settings [57,58], which makes it sensitive to expressive norms and response calibration strategies [59]. Differences in the interpretation of the arousal scale have also been reported between normative samples of affective pictures [18]. The translation of the scale is a concrete instance of this problem. In the Russian instructions the arousal dimension was introduced as «вoзбуждение» (vozbuzhdenie), a word that denotes excitement and agitation and can carry connotations that the Spanish pole labels, from very calming to very activating, do not have. Part of the lower Russian arousal ratings may therefore reflect how this term was understood, and future Russian norming studies should use labels closer to the activation dimension.

The lower arousal ratings are also compatible with the theory of ideal affect, which holds that cultures differ in the levels of activation they consider desirable [60,61], and the reduced polarisation of valence ratings is compatible with the finding that Russians are less expressive towards strangers than towards friends and family [37]. Expression towards strangers and anonymous ratings of pictures are, however, different phenomena, and rating behaviour, subjective experience and overt expression are related but distinct constructs. These accounts are therefore possible interpretations that the present data cannot test. They could be examined in studies that measure ideal affect, display rules and picture ratings in the same participants.

For studies with Russian-speaking participants, three practical consequences follow. First, EMselec can be used without change in designs that contrast the three affective categories. Second, when pictures are selected or thresholded by their absolute values, the Spanish norms should be corrected with the category-specific differences in Table 4 and Table 6. Because the calibration slope differed between categories on both dimensions, these category-specific corrections are likely to be more accurate than the single calibration line in Table 7. Individual neutral or positive pictures should not be ranked, or selected within their category, on the basis of the Spanish valence norms, because the within-category correlations for these categories were low (ρ = 0.27 and 0.07). Third, negative and positive pictures should not be assumed to be matched in arousal, so studies comparing these two conditions should control arousal statistically or select pictures on the basis of local ratings.

The study has several limitations. First, the Russian-speaking sample and the Spanish normative sample differed in language and culture, in testing environment, display equipment and period of data collection, and probably in age, sex composition and student status (Table 1). The demographic characteristics of the Spanish raters are known only for the database as a whole, not for the raters of the EMselec pictures. The present design cannot separate these factors, so the observed differences describe the two samples and should not be read as a general effect of Russian versus Spanish culture. Second, women predominated in both samples (68% in the Russian sample and 77% in the normative sample of the whole database), and the Russian participants were older on average (M = 25 vs. 20.1 years). Sex-disaggregated analyses were not conducted because the Russian sample was not powered for subgroup comparisons. Third, the ratings were collected online, and display parameters such as screen size, brightness and colour calibration were not controlled. Because the categories are matched in luminance and colour and each participant saw all pictures on the same display, this is unlikely to have affected the differences between categories, but part of the differences in absolute ratings between the samples may reflect these display conditions, because the Russian participants used their own screens and the Spanish participants viewed projected pictures. Fourth, the Spanish normative values were available only as picture-level means and standard deviations. The between-sample comparisons therefore treat pictures, not participants, as the unit of replication, and the variability of the Spanish sample is not represented in any model; the participant-level analysis addresses this problem only for the Russian data. Fifth, the translation of the arousal scale may have contributed to the arousal difference.

Future studies could extend the validation of EMselec to samples from other countries. Such data would show which properties of the norms are stable across populations and which depend on the population in which they were obtained.

5. Conclusions

Across all pictures, the Russian-speaking sample and the Spanish norms showed high rank-order correspondence (ρ = 0.91 for valence and 0.86 for arousal). Within categories, the ordering of pictures was moderately preserved for arousal and for negative valence but not for neutral and positive valence. Absolute agreement was good for valence (ICC = 0.89) and imprecisely estimated for arousal (ICC = 0.70, 95% CI [−0.02, 0.89]). Arousal ratings were lower in the Russian sample in all three categories, by an amount that depended on the category, and valence ratings were compressed towards the middle of the scale with a small upward displacement. Negative pictures were rated as more arousing than positive pictures, so the arousal matching of the EMselec categories did not hold in the Russian-speaking sample. Ideal affect and display rules offer possible explanations of these differences that the present data cannot test.

The results also indicate that the EMselec subset is a potentially useful standardised stimulus set for studies with Russian-speaking participants. Its matched physical characteristics support comparisons between groups, provided that absolute norms are recalibrated and the arousal difference between negative and positive pictures is taken into account.

More generally, the results show that the normative values of an affective picture set should be checked when the pictures are used in a new population, even when their ordering is preserved.