EEG-Based Neural Synchrony Predicts Evaluative Engagement with Music Videos ^†

Abstract

The use of neuroimaging to predict individual and population-wide behaviors, also known as neuroforecasting, was long applied to estimate movie popularity. Only recently, EEG-based neural synchronization, which is indicative of engagement, was found as a valid predictor of the listening behavior of the population. However, the population’s evaluative responses to the songs were not incorporated. To fill this void, this study explored whether neural synchrony can also be related to likes, dislikes and comments for the same songs on YouTube more than two years after their release. In this way, we aimed to separate passive engagement (i.e., listening) from active engagement (evaluating). The results showed that neural synchrony was a significant predictor of the likes and comments on YouTube, even after controlling for explicit liking ratings from the lab study. In contrast, frontal alpha asymmetry did not predict YouTube likes. Thus, engagement as represented by neural synchronization could be a valuable tool for predicting active as well as passive engagement with entertainment products. This underlines the value of neural similarity in predicting the impact of music and videos before their true effect in the crowd can be known.

1. Introduction

Traditionally, when investigating human decision-making, an individual’s previous choices were the best indicator of future ones. However, more recently, it was supposed that the choices humans make can be linked to changes in brain activity. Due to the continuous advances in brain-imaging design and analysis, the use of brain activity to predict individual and population-wide choices emerged in recent years. This prediction is also called neuroforecasting [1] and can be applied in a variety of domains including consumer behavior, neuromarketing, health-related choices and financial decision-making [2,3,4,5]. Researchers tried to pinpoint neural dynamics that predict individual and population-wide behaviors, but with mixed results. In a study by Genevsky et al. [6], the neural activity in the reward center in the brain of only 30 human participants was predictive of the market-level crowdfunding outcomes weeks later, while the behavior of those participants did not provide any predictive value for market-level behavior.

Within this line of research, Berns and Moore [7] examined the brain activity in the reward center of their subjects during listening to music, and they were able to correlate these neural dynamics to the sales data of the music albums. Later studies [8] again found that music popularity could be forecasted with brain activity, but these implemented a metric of brain activity called neural synchrony.

The first notice of neural synchronization in the context of predicting behavior was in 2004, when Hasson et al. [9] identified similar brain responses across the subject pool while they were watching a movie. This similarity was not only found in visual and auditory cortices (where it could be expected due to the audiovisual input of the movie), but frontal and parietal regions also showed synchronized responses. This indicates that brain regions that are associated with narrative and emotional processing were engaged while watching the movie in a similar way across participants. Engagement with the narrative is a common definition of the neural synchronization during natural stimuli [10]. Several studies showed that scrambling a story, movie or music part significantly decreased the similarity in brain patterns between individuals [11,12]. This metric of synchronized brain activity between individuals was shown to be indicative of popularity of movies [13,14], social buzz [15] and recall [13,16]. Other studies, such as Chan et al. [17], showed that this metric is not only related to passive engagement but also evaluative responses.

Next to neural synchrony, another metric related to personal preferences should be considered as well; frontal alpha asymmetry (FAA) is a popular metric in neuromarketing and was linked to personal liking in several studies [18,19,20]. Moreover, FAA was a significant predictor in the study of Leeuwis et al. [8]; on an individual level, the FAA during a song was predictive of the individual ratings provided afterwards.

Thus, to capture the predictive value of neural measurements of evaluative responses on a population-level, this study presents an accompanying analysis to Leeuwis et al. [8], who showed that the number of plays on Spotify could be predicted by this neural marker. To elaborate the finding that passive engagement in the form of consuming the content could be predicted, we added the predictive value of neural synchrony for active engagement in the form of evaluative responses to the songs; we inferred the number of likes, dislikes and comments on YouTube and investigated whether neural markers of synchronization and frontal alpha asymmetry within a small group of individuals could be related to this evaluative engagement of the crowd as well.

2. Materials and Methods

Data for the experiment were obtained from Leeuwis et al. [8], where 30 participants (23 female, 7 male; Age M ± SD = 26.87 ± 10.80) listened to fragments of music tracks on two albums that were just released a few days prior to the experiment. During listening to the 24 s excerpt of each track, brain activity was recorded on nine EEG channels. After listening to a track, participants rated the track on a 1–5 Likert scale before continuing to the next one. The order of the albums was counterbalanced, and the series of tracks were randomized between subjects too.

Passive engagement was quantified by the number of views on YouTube. Four dependent variables (DV) of active engagement on YouTube were evaluated 2 years and 10 months after the release of the albums; likes and comments were directly assessed from YouTube.com. Dislikes were obtained with the Google Chrome extension Return to Dislikes (https://returnyoutubedislike.com/ accessed on 23 February 2023). Additionally, likes of the song YouTube were corrected by the times the song was played on YouTube, such that this dependent variable was reflecting the number of likes per play of the video.

The EEG data were processed by Leeuwis et al. [8]. After pre-processing, power spectral density was calculated in the alpha frequency range (8–12 Hz) and this data in the central electrodes (C3, Cz, and C4) were pairwisely correlated between all participants for each track separately to calculate neural synchrony for each track. Moreover, frontal alpha asymmetry (FAA) was calculated from the F3 and F4 electrodes. More details about the processing can be found in Leeuwis et al. [8].

Outliers were removed when their score on the dependent variable exceeded the boundary of 2.5 SD above or below the mean and by Mahalanobis distance on both the explicit ratings by participants in the lab and neural measurements.

Linear regression models were fitted to assess the predictability of the neural measurements. As noted by Boksem and Smidts [5], when assessing the predictability of neural measures, their value should be above and beyond the traditional methods (e.g., stated preferences) to be relevant. Therefore, for each DV, a baseline model was created as H0 to be compared to models H1 and H2 incorporating brain activity measures:

H0: 

DV~Likes Lab + Music Video (y/n)

H1: 

DV~Likes Lab + Music Video (y/n) + Neural Synchrony

H2: 

DV~Likes Lab + Music Video (y/n) + Frontal Alpha Asymmetry

This meant that the basic (H0) model incorporates the average explicit liking of the respondents in the lab and the fact that a track was accompanied by a music video as a dichotomous factor. It is important to note here that the participants in the lab did not see the music video, they only listened to musical excerpts of the songs while the desktop screen was black. The DV was one of the five YouTube variables mentioned earlier: the likes, dislikes, number of comments, the likes-per-view ratio, or likes-per-dislike ratio.

Consequently, the H0 model was then elaborated by adding neural synchrony or frontal aslpha asymmetry to the model and comparing the explained variance with an ANOVA test between the models. For the DV number of views, only H1 was compared to H0, as the previous research already showed that frontal asymmetry was not predictive of passive engagement in this context [8]. The assumptions underlying linear regression were tested with a Shapiro–Wilk test for normality of residuals and Breusch–Pagan test checking the assumption of constant variance [21]. Linearity assumptions were checked visually. Numerical variables were scaled before inputted to the model.

Investigating four dependent variables, and for each comparing two models to H0, resulted in 8 statistical tests. Moreover, the model for views was compared only on neural synchrony, making the total of 9 tests. Following Bonferroni correction, significance levels for these tests were set at 0.006.

Data acquisition, pre-processing, PSD and FAA calculation was carried out using iMotions (2019). The neural synchrony was calculated using R (R Core Team, 2019), and statistical analysis was also performed in R (R Core Team, 2022).

3. Results

One track exceeded the 2.5 SD boundary on likes and comments and was also identified as an outlier in Mahalanobis distance. This track was removed from analysis, which left 23 tracks in the data for analysis (liking in lab before removal: M = 2.67, SD = 0.28; liking in lab after removal: M = 2.66, SD = 0.28; likes YouTube before removal: M = 64.73, SD = 63.18; likes YouTube after removal: M = 56.58, SD = 50.09). Models without outlier removal did not meet the assumption of normality of errors; thus, all models reported here had the outlier removed.

3.1. YouTube Views

To start with active engagement, we applied the model of Leeuwis et al. [8] on the passive engagement on YouTube. To correct for normality of model errors, the YouTube views were log transformed before they were put into the model. Here, as well, neural synchrony added predictive value to the baseline model (F(20) = 8.57, p = 0.009). The H0 model predicted 29.05% of the views (${\mathrm{R}}_{\mathrm{a}\mathrm{d}\mathrm{j}}^2=$ 0.291, F(2,20) = 5.50, p = 0.012), while the model incorporating neural measures predicted 48.34% of the views (${\mathrm{R}}_{\mathrm{a}\mathrm{d}\mathrm{j}}^2=$ 0.483, F(2,20) = 7.86, p = 0.001).

3.2. YouTube Likes

The H0 model significantly predicted the number of likes on YouTube (${\mathrm{R}}_{\mathrm{a}\mathrm{d}\mathrm{j}}^2=$ 0.573, F(2,20) = 15.76, p < 0.001), where the fact that the track was released with an accompanying music video was significantly predictive (beta = 1.59, t(20) = 5.38, p < 0.001) but the average subjective liking of the track by the participants in the lab was not (beta = 0.26, t(20) = 1.65, p = 0.11), although an increasing line was observed (Figure 1).

Adding neural synchrony to this model of YouTube likes increased the predictive value (F(20) = 9.68, p = 0.006). The predicted variance was improved to 70.21% (${\mathrm{R}}_{\mathrm{a}\mathrm{d}\mathrm{j}}^2=$ 0.7038, F(3,19) = 18.28, p < 0.001). The predictive value of the stated preferences in the lab remained insignificant (beta = 0.13, t(19) = 1.09, p = 0.29), while the fact that the track was released with an accompanying music video was significantly predictive (beta = 1.46, t(19) = 5.80, p < 0.001), as well as neural synchrony (beta = 0.38, t(19) = 3.11, p = 0.006) (Figure 2).

Predicting the evaluative responses on YouTube by the stated preferences in the lab and the accompanying frontal alpha asymmetry measures, provided a significant model (${\mathrm{R}}_{\mathrm{a}\mathrm{d}\mathrm{j}}^2=$ 0.553, F(3,19) = 10.08, p < 0.001), although no improvement in predictive value was achieved beyond the model incorporating only the lab-based liking (F(20) = 0.120, p = 0.73).

3.3. YouTube Dislikes

The same comparisons were also performed for dislikes on YouTube. To meet the assumptions of homogeneity of variance, the dislikes were log transformed. The H0 model significantly predicted the number of dislikes on YouTube (${\mathrm{R}}_{\mathrm{a}\mathrm{d}\mathrm{j}}^2=$ 0.553, F(2,20) = 14.60, p < 0.001), where the fact that the track was released with an accompanying music video was significantly predictive (beta = 1.53, t(20) = 5.04, p < 0.001) but the average liking of the track by the participants in the lab was not (beta = 0.28, t(20) = 2.00, p = 0.06). Adding neural synchrony to this model increased the predictive value (F(20) = 7.39, p = 0.014), although not significantly when considering Bonferroni correction. The predicted variance was improved to 66.11% (${\mathrm{R}}_{\mathrm{a}\mathrm{d}\mathrm{j}}^2=$ 0.661, F(3,19) = 15.30, p < 0.001). The predictive value of the stated preferences in the lab remained insignificant (beta = 0.19, t(19) = 1.50, p = 0.15), while the fact that the track was released with an accompanying music video was significantly predictive (beta = 1.40, t(19) = 5.22, p < 0.001), as well as neural synchrony (beta = 0.35, t(19) = 2.72, p = 0.014). Interestingly, the positive value points out that neural synchrony was positively correlated to likes as well as dislikes. This indicates that it was a predictor of engagement rather than the direction of this engagement. Frontal alpha asymmetry did not add any predictive value to the H0 model (F(20) = 0.43, p = 0.52; ${\mathrm{R}}_{\mathrm{a}\mathrm{d}\mathrm{j}}^2=$0.540, F(3,19) = 9.60, p < 0.001).

3.4. YouTube Comments

Similarly, we further examined the predictability of YouTube comments on each video. To meet the assumption of normality of model errors, the comments were log transformed before putting into the model. The H0 model significantly predicted the number of comments on YouTube (${\mathrm{R}}_{\mathrm{a}\mathrm{d}\mathrm{j}}^2=$ 0.616, F(2,20) = 18.7, p < 0.001), where the fact that the track was released with an accompanying music video was significantly predictive (beta = 1.71, t(20) = 6.09, p < 0.001) but the average liking of the track by the participants in the lab was not (beta = 0.07, t(20) = 0.50, p = 0.62).

Adding neural synchrony to this model increased the predictive value (F(20) = 11.23, p = 0.003), where the predicted variance was improved to 74.61% (${\mathrm{R}}_{\mathrm{a}\mathrm{d}\mathrm{j}}^2=$ 0.746, F(3,19) = 22.55, p < 0.001). The predictive value of the stated preferences in the lab remained insignificant (beta = −0.03, t(19) = −0.28, p = 0.78), while the fact that the track was released with an accompanying music video was significantly predictive (beta = 1.58, t(19) = 6.79, p < 0.001), as well as neural synchrony (beta = 0.38, t(19) = 3.35, p = 0.003).

Adding frontal alpha asymmetry to this model did not increase the predictive value (F(20) = 0.20, p = 0.659), the model only explained 60.0% of the variance (${\mathrm{R}}_{\mathrm{a}\mathrm{d}\mathrm{j}}^2=$ 0.600, F(3,19) = 12.01, p < 0.001).

3.5. Likes Per View Ratio

The H0 model predicting likes-per-view ratio explained only 20.02% of the variance (${\mathrm{R}}_{\mathrm{a}\mathrm{d}\mathrm{j}}^2=$ 0.202, F(2,20) = 3.79, p = 0.04). In this case, adding neural synchrony did not add predictive power (F(20) = 0.20, p = 0.66), as the model explained 16.88% of the variance (${\mathrm{R}}_{\mathrm{a}\mathrm{d}\mathrm{j}}^2=$ 0.169, F(3,19) = 2.49, p = 0.09). FAA did also not improve the model (F(20) = 0.36, p = 0.56), as the model explained 17.57% of the variance (${\mathrm{R}}_{\mathrm{a}\mathrm{d}\mathrm{j}}^2=$ 0.176, F(3,19) = 2.56, p = 0.09).

4. Discussion

In investigating the use of neuroimaging techniques to predict population-wide evaluative responses to music on YouTube, we explored the predictive value of neural synchrony and frontal alpha asymmetry. The study presented an analysis of data from a previous experiment by Leeuwis et al. [8], where the neural markers of synchronization and frontal alpha asymmetry in response to music tracks of two new music albums were recorded. The results showed that neural synchrony had predictive value above and beyond stated preferences for evaluative responses by the population. The neural similarity was a positive factor in predicting likes as well as dislikes and comments. This indicates the potential usefulness of neural synchrony not only in predicting passive engagement (consuming the content) but also active engagement (evaluating the content) with music tracks, regardless of the direction of the evaluation. No such results were found for frontal alpha asymmetry. To further examine the active engagement measurement of likes, we corrected those by the number of times the videos were played and found that the predictive value was not present anymore.

While one could argue that the latter could be expected since the previous analysis by Leeuwis et al. [8] showed that the number of times the song was played was significantly predicted by neural synchrony, we believe this analysis of evaluative responses shows a new aspect of engagement that was not covered in previous explorations of neural synchrony. The previous analyses by Berns and Moore [7] and Leeuwis et al. [8], where the popularity was derived from album sales and online streams, respectively, were relating more to passive engagement. In this dataset, we validated once again that passive engagement was better predicted when neural synchrony was included in the model, although the explained variance was lower than the results observed for Spotify plays [8]. This could have to do with a difference platform engagement or the fact that these data were gathered two years later compared to the Spotify data.

Our study shows that the number of likes was not significantly predicted when corrected for the total number of views, which indicates that the number of views could be the most important contributor to the number of likes on the platform. However, the number of views could not be known beforehand, making it not feasible for popularity prediction. The other factors explored in the data, namely the neural correlates and the fact that the song is released with music video can be established almost immediately after the release of the song, enabling prediction of the evaluative responses straight away.

Previous studies that found that similarity in brain responses was indicative of evaluative responses [17] were more focused on market review panelists. This is different from YouTube video (dis)likes in the sense that for the latter, the likes are provided by individuals that consumed the content on a voluntary basis while the panelists are intentionally exposed to the stimuli. Ideally, in future research, we would correct the likes not for the total number of plays but by the number of individuals that played the video. This may present a more realistic scenario as it may be that a small group of individuals (assuming this will be the likers) is responsible for a great lot of the plays [22].

The fact that frontal alpha asymmetry did not add extra explanatory value could be explained by the fact that this metric is mostly found to be indicative of individual liking [8,18,19]. Moreover, the catchiness of music is not necessarily depending on positive emotions elicited; irritating tunes stick particularly well [23], explaining why approach may not be sufficiently predictive of passive or active engagement with the content. However, some studies also found generalizations out-of-sample to be correlated to FAA measurements; Shestyuk et al. [24] showed that frontal asymmetry in a combination of alpha and beta bands was predictive of TV viewership and social media engagement.

In discussing these measures, it should be taken into account that we discussed the brain activity measures of unimodal stimuli; only sound. For FAA, this was shown to elicit weaker activity patterns than multimodal stimuli that most of the previous studies employed [25]. Neural synchronization may also be diminished due to this unimodal approach, since engagement in general was higher with multimodal design [26].

Another limitation of the study was the fact that only two music albums were incorporated as stimuli, which may not be representative of the wide variety of entertainment media available. Future research could investigate the predictive value of neural metrics in larger and more diverse stimulus sets. A replication with new music could provide a valuable addition to this dataset. Moreover, the mediation variable of music videos, social media campaigns or marketing budget were not accounted for but could have an interesting effect on the predicted values of likes on YouTube.

Future research in this area could focus on tackling these limitations. By creating a full image of all variables impacting entertainment popularity, the undisguised effect of neural measure beyond all other could be explored. Moreover, replication of studies employing neural synchrony is needed to strengthen the validity of this emerging metric. Especially since various calculation methods exist, including different locations, frequency bands, neuroimaging techniques and algorithms in general. Moment-to-moment fluctuations of neural synchrony may also provide an interesting metric for examination of entertainment [11,27]. The field of neuroforecasting may also be expanded by the implementation of more sophisticated prediction algorithms, machine learning or deep learning, which generally improves prediction accuracies [28].

Our results underline that neuroforecasting has an additional value to surveys when determining the success of entertainment types. Applications could include ranking songs on an album to determine the marketing budget to put in each of them. Moreover, the field can be broadened to other branches such as marketing, health and financial decision-making where this metric could be used to indicate the engagement and predict the impact of communication statements.

5. Conclusions

This work provided valuable insights into the potential usefulness of neuroforecasting in predicting human behavior. The research findings suggest that neural synchrony may have predictive value above and beyond stated preferences, indicating that brain activity can be a useful tool in predicting not only population-wide passive engagement behaviors but also active engagements; neural synchrony added value for predicting YouTube likes, dislikes and comments almost three years after the release of the music tracks. Our results showed a distinction between active and passive engagement and the potential of neural synchrony to provide an indication of both types of interaction with the content. Thereby, this work further contributes to the body of literature of neuroforecasting. Further research in this field could have important implications for a variety of domains, including consumer behavior, neuromarketing, health-related choices, and financial decision making.