Applying Acoustical and Musicological Analysis to Detect Brain Responses to Realistic Music : A Case Study

Music information retrieval (MIR) methods offer interesting possibilities for automatically identifying time points in music recordings that relate to specific brain responses. However, how the acoustical features and the novelty of the music structure affect the brain response is not yet clear. In the present study, we tested a new method for automatically identifying time points of brain responses based on MIR analysis. We utilized an existing database including brain recordings of 48 healthy listeners measured with electroencephalography (EEG) and magnetoencephalography (MEG). While we succeeded in capturing brain responses related to acoustical changes in the modern tango piece Adios Nonino, we obtained less reliable brain responses with a metal rock piece and a modern symphony orchestra musical composition. However, brain responses might also relate to the novelty of the music structure. Hence, we added a manual musicological analysis of novelty in the musical structure to the computational acoustic analysis, obtaining strong brain responses even to the rock and modern pieces. Although no standardized method yet exists, these preliminary results suggest that analysis of novelty in music is an important aid to MIR analysis for investigating brain responses to realistic music.


Introduction
Measuring brain responses to music provides important insights into the auditory, non-verbal brain functions of functional hearing and hearing disabilities.The brain response is measured by identifying each time point in the music that evokes the brain response.Typically, the experimenter uses a set of predefined time points and designs artificial music stimuli adapted to the time points.However, this approach limits the opportunities for measuring brain responses to realistic music and restricts the generalization of the findings to ecological listening conditions.Since the artificially designed music differs from real music heard outside the laboratory, there is a growing interest in developing music information retrieval (MIR) methods [1,2] to identify the time points for measuring brain responses to real music [3][4][5][6][7].We have tested a new method, which combines acoustical MIR analysis with modeling of brain mechanisms to automatically extract time points when the brain responds to real music pieces [8].
Brain responses to acoustical changes in music measured with neurophysiological methods, such as electroencephalography (EEG) and magnetoencephalography (MEG), originate from synchronous post-synaptic electric potentials in coherently oriented groups of pyramidal neurons in the cortical layers of the brain [9].These are measured with EEG electrodes placed on the scalp, while the magnetic counterpart of the potentials is measured with MEG sensors positioned around the head.While basic brain responses originating from the auditory cortex in reaction to acoustical changes in music can be measured, novel events in music also evoke novelty-related brain responses originating from higher-order cortical structures such as the inferior frontal cortex [10][11][12][13][14].
A relevant source of novel events in music is the change of level in acoustical feature parameters.With respect to loudness, the metal subgenre of rock music is in particular attempting to maximize perceived loudness [15].This is achieved by applying loud bass drums, distortion and brightness increase filters on the bass and guitar, as well as dynamic range compression [15].The perceived loudness can be further increased by habituating the listener to low levels of loudness and brightness, while subsequently increasing the level for these parameters (e.g., see Figure 1,top).A novelty-related brain response is typically evoked after briefly learning to expect a certain parametric level in acoustical features followed by a change in the acoustical parameter [16].In addition, the novelty-related brain response can be evoked by novel events, which are infrequent in relation to the listener's prior developed understanding of which events are frequent and infrequent in the music style before hearing the current music piece [13,14,[17][18][19][20][21][22][23].To measure such novelty-related brain responses, it would be relevant to analyze the novelty of the musical events.
In the present study, we investigate whether automatic extraction of acoustical feature increases, based on MIR analysis, can be applied to isolate brain responses to real pieces of different musical genres, such as metal rock music, tango music and modern symphony orchestra music.We also consider whether it is relevant to include an analysis of the novelty in the musical structure to extract brain responses to novel events in the music.

EEG and MEG Dataset
The study did not involve any new data collection but instead utilized a subset of an existing dataset of combined electroencephalography (EEG) and magnetoencephalography (MEG) recordings called Tunteet, collected at the Biomag laboratory at Helsinki University Hospital (Vectorview TM 306-channel MEG scanner (Elekta Neuromag ® , Elekta Oy, Helsinki, Finland; sample rate: 600 Hz)) in the years 2012-2013 and already extensively published or under publication [19,[24][25][26][27][28][29][30][31][32][33][34][35][36] (ethical approval by the Coordinating Ethics Committee of the Hospital District of Helsinki and Uusimaa; approval number: 315/13/03/00/11, obtained on March the 11th, 2012).The subset here included EEG and MEG data from 48 healthy participants with self-reported normal hearing and no past cases of neurological or psychiatric disorder (28.3 years; st.dev.8.6 years; 23 females; 2 left-handed).During the measurement subjects were instructed to remain still, listen to the music through pneumatic headphones, and keep their eyes open.Further details about stimulus presentation and data acquisition are found e.g., in Reference [29].Preprocessing of MEG data was done applying Elekta NeuromagTM MaxFilter 2.2 Temporal Signal Space Separation (tSSS), and the EEG and MEG data was further processed with ICA decomposition using FieldTrip version r9093 [37] and Matlab R2013b (MathWorks, Natick, MA, USA).Further details are found in Reference [8].Since the planar gradiometer sensors of the MEG measure the difference in the magnetic field across two orthogonal directions, the measures from each couple of longitudinal and latitudinal gradiometer sensor were combined by applying the Pythagorean distance formula, as implemented in FieldTrip, d = longitudinal 2 + latitudinal 2 .

Stimuli
Three real music pieces were applied as stimuli, which were a subset of the larger "Tunteet" experiment.The song presentation order was counterbalanced between subjects.Participants heard a shortened version of the progressive metal/rock piece Stream of Consciousness by Dream Theater

Feature Extraction with MIR Toolbox
The musical features were extracted with the MIR Toolbox (version 1.6.1)for Matlab [1].A measurement time frame size of 25 ms and a hop size of 50% were applied, resulting in a sample rate of 80 Hz (1/(0.025s. × 0.5)) [8].Five acoustical features related to loudness and timbre were extracted [8].The Loudness was measured in root-mean-squared (RMS) values, approximating the perceived loudness across time.The Brightness was estimated by calculating the proportion of energy above and below 1500 Hz [1].Spectral flux was derived as the difference in spectral power between each successive time frame.Noisiness, or zero-crossing rate, measured the number of times the audio signal changed per second causing a shift in sign.Finally, a measure of Roughness was derived to describe the sensory dissonance experienced when two or more tones with narrow pitch intervals occur, e.g., in dissonant chords or inharmonic musical scales [38,39].

Automatic Capture of Acoustical Changes Evoking Cortical Responses
We applied an automatic method for extracting the time points with MIR feature increases assumed to evoke brain responses, which is described in further detail in Reference [8], and is developed based on a similar semi-automatic method [5,6].For extracting the time points with MIR feature increases assumed to evoke brain responses the following three criteria should be satisfied (for further details see Reference [8]): 1.
The time points exceed a perceptual threshold by which their magnitude of rapid increase (MoRI) [5,6] is among the 30% highest MoRI values.

2.
The time points show minimal effects of neural habituation [5,6].This is implemented by only including time points with at least 1 s preceding low increase phase (PLIP) with MoRI below the 30% highest threshold.

3.
In order to keep the number of extracted time points constant for comparable signal-to-noise ratios when averaging across more brain responses, no more than 10 time points in the music piece satisfying the above criteria and which are maximally distributed across time are extracted.
The number of time points for the five MIR features that meet these criteria is shown in Table 1, which includes statistics on the number of total MoRI peaks satisfying criteria 1, and the PLIP and time distribution after criteria 1-3 have been applied to extract assumed relevant time points.As visible from the Table 1, only few events were meeting the defined criteria, especially for the metal rock piece SC compared with the tango piece AN.

Musicological Analysis of Novel Events
Due to relatively high MIR feature levels and change rates in the metal rock and modern symphony orchestra music pieces, the automatic MIR approach identified too few acoustical feature-peak events satisfying the second criteria (the PLIP criteria), which might compromise the possibility of obtaining neural correlates of the acoustical feature processing.Hence, we also decided to introduce a musicological analysis of the pieces, which was conducted by the first author, to investigate the possible effects of novelty in the music structure.
The metal rock piece SC begins with a repetitive and quiet guitar solo consisting of an 8-note figure repeated 8 times.Every second 8-note figure is played on a C-minor-6 and Bb-major-7 chord.A quiet C pitch is played on the bas and fades out during the first 4 repetitions of the guitar figure.This is followed by drums, bas, guitar and keyboard playing an expected C-minor chord, but with a novel high loudness level and distortion effects added (Figure 1, top).
The excerpt of the modern symphony orchestra piece RS starts with an introduction part for woodwind instruments, consisting of phrases alternating between major-minor mode tonalities and harmonic progressions of unclear tonality (music score measures 1-75).The following 'Augurs of Spring' part begins with the novel 'Augurs' chord, which is commonly defined as atonal or as a polychord consisting of an E-major superposed on an Eb-major seventh chord (music score measure 76, at rehearsal number 13) [40].Although the chord is consistent with the chromatic tuning system of contemporary Western music, it is novel and presumably evokes a measurable novelty related brain response [17,20], since it neither belongs to a typical harmonic progression in major or minor mode, nor adheres to a common major or minor mode pitch class [40] (e.g., see Figure 1, bottom).
The excerpt of the modern symphony orchestra piece RS starts with an introduction part for woodwind instruments, consisting of phrases alternating between major-minor mode tonalities and harmonic progressions of unclear tonality (music score measures 1-75).The following 'Augurs of Spring' part begins with the novel 'Augurs' chord, which is commonly defined as atonal or as a polychord consisting of an E-major superposed on an Eb-major seventh chord (music score measure 76, at rehearsal number 13) [40].Although the chord is consistent with the chromatic tuning system of contemporary Western music, it is novel and presumably evokes a measurable novelty related brain response [17,20], since it neither belongs to a typical harmonic progression in major or minor mode, nor adheres to a common major or minor mode pitch class [40] (e.g., see Figure 1, bottom).These two cases of novel events in the music structure were found among the full set of automatically detected MoRI peaks for both the loudness (RMS) and the spectral flux increases (see These two cases of novel events in the music structure were found among the full set of automatically detected MoRI peaks for both the loudness (RMS) and the spectral flux increases (see Section 2.4 and Table 1).After applying processing steps 2 and 3 (see Section 2.4), the highly novel loudness increase in the metal rock piece SC was coincidentally among the final subset of maximally distant time points with increases in the spectral flux feature, even though the novelty of the music structure was not taken into consideration.However, the novel chord in the symphonic piece RS was not present in the final subset of maximally distant time points for either feature.Thus, the novel chord was identified only when the automatic detection of MoRI peaks was combined with the manual analysis of novelty in the music structure.

Statistical Analysis of Brain Responses
Since EEG is more commonly available than MEG systems, we focused here on the EEG data, while we apply the MEG for showing projections of the obtained brain response on the scalp with higher spatial accuracy than achievable with the EEG.The study aimed at identifying whether brain responses could be obtained by automatic MIR extraction or aided by musicological identification.Brain responses measured to automatically detected MIR time points were averaged for each song and listener, whereas the response to a novel event was identified with music structure analysis as a single event.The EEG baseline was identified as the mean EEG signal from −100 to 0 ms before the time point where the MIR feature increase begins.For testing the significance of the obtained brain responses for the two approaches, a one-sample t-test across participants was used to compare the brain responses and the signal measured at baseline.

Results
The automatic MIR analysis procedure resulted in successful identification of brain responses peaking ~200 ms after acoustical increases in the tango piece AN by Astor Piazzolla (more details are provided in Reference [8]) (Figure 2).Section 2.4 and Table 1).After applying processing steps 2 and 3 (see Section 2.4), the highly novel loudness increase in the metal rock piece SC was coincidentally among the final subset of maximally distant time points with increases in the spectral flux feature, even though the novelty of the music structure was not taken into consideration.However, the novel chord in the symphonic piece RS was not present in the final subset of maximally distant time points for either feature.Thus, the novel chord was identified only when the automatic detection of MoRI peaks was combined with the manual analysis of novelty in the music structure.

Statistical Analysis of Brain Responses
Since EEG is more commonly available than MEG systems, we focused here on the EEG data, while we apply the MEG for showing projections of the obtained brain response on the scalp with higher spatial accuracy than achievable with the EEG.The study aimed at identifying whether brain responses could be obtained by automatic MIR extraction or aided by musicological identification.Brain responses measured to automatically detected MIR time points were averaged for each song and listener, whereas the response to a novel event was identified with music structure analysis as a single event.The EEG baseline was identified as the mean EEG signal from −100 to 0 ms before the time point where the MIR feature increase begins.For testing the significance of the obtained brain responses for the two approaches, a one-sample t-test across participants was used to compare the brain responses and the signal measured at baseline.

Results
The automatic MIR analysis procedure resulted in successful identification of brain responses peaking ~200 ms after acoustical increases in the tango piece AN by Astor Piazzolla (more details are provided in Reference [8]) (Figure 2).The one-sample t-test across listeners shows that the mean brain response after the onset of the novel loudness increase, measured around the peak amplitude at 137 ± 15 ms, differs significantly from the baseline, t(43) = −9.2,p < 10 −6 , mean = -13.6 µV (Figure 6, left).Though brain responses to the maximally distant loudness (RMS) increases do not exceed the baseline, t(43) = −1.6,p = 0.125, mean = −1.2µV, while responses to the maximally distant spectral flux increases deviated slightly from the baseline, t(43) = −3.0,p = 0.005, mean = −1.8µV, suggesting an effect of the time point successfully correlating with the novel loudness increase (for comparison, see Figure 4, left, and Figure 5).The one-sample t-test across listeners shows that the mean brain response after the onset of the novel loudness increase, measured around the peak amplitude at 137 ± 15 ms, differs significantly from the baseline, t(43) = −9.2,p < 10 −6 , mean = -13.6 μV (Figure 6, left).Though brain responses to the maximally distant loudness (RMS) increases do not exceed the baseline, t(43) = −1.6,p = 0.125, mean = −1.2μV, while responses to the maximally distant spectral flux increases deviated slightly from the baseline, t(43) = −3.0,p = 0.005, mean = −1.8μV, suggesting an effect of the time point successfully correlating with the novel loudness increase (for comparison, see Figure 4, left, and Figure 5).The average brain response across listeners to the excerpt of Igor Stravinsky's Rite of Spring is shown in Figure 3 (bottom).Similarly, no clear evoked responses are observed for the maximally distant time points extracted randomly in relation to the predictability of the music structure (Figure 4, right).Despite this, the onset of the novel chord in the string group after 198.646 s introduction is identified among the loudness (RMS) and spectral flux increases detected by the automatic method.This chord appears to evoke a novelty related frontal-central negative brain response across the listeners, peaking 150 ms after the stimulus onset (Figure 3, bottom, and Figure 7).

Waveforms Topographies
Across listeners, the mean brain response after the onset of the novel chord, measured around the peak amplitude at 150 ± 15 ms, differs significantly from the baseline, t(47) = 2.9, p = 0.006, mean = 7.8 μV (Figure 6, right).However, no responses are significantly exceeding the baseline for the maximally distant loudness (RMS) increases, t( 47  The average brain response across listeners to the excerpt of Igor Stravinsky's Rite of Spring is shown in Figure 3 (bottom).Similarly, no clear evoked responses are observed for the maximally distant time points extracted randomly in relation to the predictability of the music structure (Figure 4, right).Despite this, the onset of the novel chord in the string group after 198.646 s introduction is identified among the loudness (RMS) and spectral flux increases detected by the automatic method.This chord appears to evoke a novelty related frontal-central negative brain response across the listeners, peaking 150 ms after the stimulus onset (Figure 3, bottom, and Figure 7).
Across listeners, the mean brain response after the onset of the novel chord, measured around the peak amplitude at 150 ± 15 ms, differs significantly from the baseline, t(47) = 2.9, p = 0.006, mean = 7.8 µV (Figure 6

Discussion
We have recently shown that time points in a tango music piece with increases in loudness and timbre-related features eliciting brain responses can be extracted automatically using MIR analysis when the rate of these feature changes is relatively slow [8].The results currently presented here suggest that for music with relatively fast rates of acoustical feature change (i.e., faster than the 1 s PLIP criteria defined above) it is important to combine the MIR estimates with analysis of the novelty of the music structure to identify measurable brain responses.Two types of brain responses to novel events in music are here investigated.First, it is shown that no clear brain responses are observed for the acoustical increases in the metal rock piece Stream of Consciousness by Dream Theater when the novelty of the musical structure is disregarded.However, a central N1 brain response is elicited for a novel increase in loudness and timbre features.Second, the time points not selected according to the novelty of the music structure in the modern ballet symphony orchestra

Discussion
We have recently shown that time points in a tango music piece with increases in loudness and timbre-related features eliciting brain responses can be extracted automatically using MIR analysis when the rate of these feature changes is relatively slow [8].The results currently presented here suggest that for music with relatively fast rates of acoustical feature change (i.e., faster than the 1 s PLIP criteria defined above) it is important to combine the MIR estimates with analysis of the novelty of the music structure to identify measurable brain responses.Two types of brain responses to novel events in music are here investigated.First, it is shown that no clear brain responses are observed for the acoustical increases in the metal rock piece Stream of Consciousness by Dream Theater when the novelty of the musical structure is disregarded.However, a central N1 brain response is elicited for a novel increase in loudness and timbre features.Second, the time points not selected according to the novelty of the music structure in the modern ballet symphony orchestra music Rite of Spring by Igor Stravinsky did not lead to successful extraction of brain responses.However, a frontal-central brain response is observed following the time point of the novel event of the 'Augurs' chord.Due to the few cases investigated here, it remains to be concluded in future studies whether the brain responses as those observed here are in fact related to the novelty or to other aspects of the music structure.
It is likely that the relatively fast rate of feature change in the metal rock and modern symphony orchestra pieces investigated in the current study has resulted in the lack of clearly observable brain responses to these music pieces [41][42][43].Previously applied MIR methods have been able to identify brain responses to slow rates of feature change for musical sounds preceded by silent breaks by combining semi-automatic and manual adjustments [5,6].Moreover, here we apply a new development of this MIR method, which in addition allows for automatically identify time points with acoustical feature increases evoking brain responses in music, and for slow as well as fast rates of acoustical feature change (i.e., also for music with shorter durations between the acoustical feature increases) (cf.Reference [8]).While the preliminary cases observed in the current study suggest that to identify brain responses in the context of fast rates of acoustical feature changes, it is additionally necessary to identify novel events in the music structure.Earlier studies have attempted to combine analysis of acoustical features in recorded music with analysis of the novelty of music structure based on self-similarity matrices [44][45][46][47].Though, recent perceptual validation suggests that the events found with the kernel-based self-similarity matrix approach relates mainly to the perception of musical phrase boundaries, and not necessarily to the novelty of the music structure [48].Therefore, it is relevant in future studies to investigate more possibilities for relating acoustical change detection with analysis of novel events.
A basic property of the brain responses to novel events is the creation of expectations based on a memory model for previously heard acoustical and musical structure [10,11,13,22].Here we investigated the novel events by means of manual musicological analysis of the music structure.For automatic music analysis, Kohonen neural network models are popular for recognizing chords and tonality in music recordings [1,[49][50][51][52], however they have been lacking the temporal information necessary for modeling novelty in music.Therefore, the recent applications of nearby contextual information in time [53], Hidden Markov Models (HMM) and the Viterbi algorithms [54] seem relevant, as these models are able to integrate the transitions from preceding musical chords in the chord recognition.Also, algorithms including memory in modeling of dissonance perception have been developed, showing improved performance in predicting dissonance perceived by listeners when including a memory for the preceding sound material [55].Such approaches could be useful for automatically identifying time points with novel events in music.Future studies might in addition continue investigating the usage of biologically-inspired neural network models with time integration and memory mechanisms, such as simple recurrent short-term memory models [56], hippocampus-inspired memory models facilitating temporary storage and retrieval for new stimulus patterns [57], or frontal cortex inspired working-memory neural network models enabling selective update-maintenance-retrieval procedures [58] (e.g., which are all freely available in the open source software Emergent [59]).
An additional topic worth to consider with respect to the measurement of brain responses is the attack of the acoustical events in the musical pieces, i.e., the rise in sound amplitude after the initial onset of a sound.The attack is dependent on the instruments involved; for example, a bass drum has a rapid attack shorter than 10 ms, while violins and flutes have relatively slower attacks in the range of 100 ms [60], whereas the MoRI measure was here optimized to measure magnitudes of increases fixed in relation to 50 ms attack curves.The attack can also be modified with music production, altering the acceleration of the rapid increase in acoustical features without modifying the composition.A complication resulting from this issue is that the perception of the rhythmic onset of a sound event in relation to other sound events (called the 'perceived attack time') is modulated by the acoustical characteristics of the attack [60].For example, the slow attack of string instruments, and the initial inharmonic noise and delayed high spectral energy in certain wind or brass instruments, causes a delayed perception of the rhythmic onset in relation to the initial rise in sound amplitude (e.g., strings or saxophones are perceived as rhythmically delayed in relation to a bass drum, even though their attack curves show identical onset times) [60].For instance, the Spectral Flux measure applied in the current study, which in the MIR Toolbox measures changes in loudness, timbre and pitch simultaneously (by summarizing the level of energy change across all spectral frequency bands), might be relevant for capturing the perceived rhythmic onsets in recordings with low dynamic range or containing wind instruments with delayed high spectral energy [2].As an example of this, the bass drum, keyboard, guitar and bass with distortion at the novel loudness increase at 20.316 s. in the metal rock piece appears to have a faster attack compared to the strings playing the novel chord at 198.646 s. in the Rite of Spring piece.However, it remains to be investigated in future studies whether the latencies of early cortical responses are related to the perceived sound onset, or they relate to a delayed perception of the rhythmic onset influenced by the attack characteristics of recorded music (e.g., see Reference [16]).

Conclusions
We tested a new method, which uses MIR to automatically and accurately extract time points with acoustic-feature changes in music and from those time points obtain measurable brain responses.Preliminary results of the present study however suggest that the method is less effective for music with fast rates of acoustical feature change, unless analysis of novel events in the music structure is also taken into consideration.

( 36 ©2001
abbreviation: SC) (7 min and 51 s) (Train of thought album, Elektra Records, 2003).Another piece was an excerpt of the first part of the modern orchestral ballet music Rite of Spring by Igor Stravinsky (abbreviation: RS) (7 min and 47 s) (Kirov Orchestra, St. Petersburg, conducted by Valery Gergiev, Decca Music Group Ltd., Universal International Music B.V).In addition, the participants heard a tango piece Adios Nonino by Astor Piazzolla (abbreviation: AN) (The Lausanne Concert album, BMG Music, 1993).

Figure 1 .
Figure 1.Sound spectrograms.Above is shown sound spectrograms for the metal/rock piece Stream of Consciousness (SC) (top) and the modern orchestral ballet music Rite of Spring (RS) (bottom).The novel loudness and brightness increase can be seen at 20.316 s (top), and the inharmonic frequency distribution of the unconventional polychord can be seen at 198.646 s (bottom).

Figure 1 .
Figure 1.Sound spectrograms.Above is shown sound spectrograms for the metal/rock piece Stream of Consciousness (SC) (top) and the modern orchestral ballet music Rite of Spring (RS) (bottom).The novel loudness and brightness increase can be seen at 20.316 s (top), and the inharmonic frequency distribution of the unconventional polychord can be seen at 198.646 s (bottom).

Figure 2 .
Figure 2. Brain responses to the tango piece measured with electroencephalography (EEG).Showing average brain responses to 10 loudness (RMS) and 10 spectral flux increases using the same method as applied in the current study.The time scale begins 100 ms before the feature increases (at −100 ms) and the feature increases start at 0 ms.

Figure 2 .
Figure 2. Brain responses to the tango piece measured with electroencephalography (EEG).Showing average brain responses to 10 loudness (RMS) and 10 spectral flux increases using the same method as applied in the current study.The time scale begins 100 ms before the feature increases (at −100 ms) and the feature increases start at 0 ms.

Figure 3 .
Figure 3. EEG waveforms.Showing grand average EEG across listeners measured at the Cz electrode.Responses to the novel loudness increase at 20.316 s (top) and the novel chord at 198.646 s (bottom) are visible.

Figure 4 .
Figure 4. EEG responses to maximally distant acoustical feature increases.Average EEG responses are shown for Dream Theater's Stream of Consciousness (left) and Igor Stravinsky's Rite of Spring (right).

Figure 3 .
Figure 3. EEG waveforms.Showing grand average EEG across listeners measured at the Cz electrode.Responses to the novel loudness increase at 20.316 s (top) and the novel chord at 198.646 s (bottom) are visible.

Figure 3 .
Figure 3. EEG waveforms.Showing grand average EEG across listeners measured at the Cz electrode.Responses to the novel loudness increase at 20.316 s (top) and the novel chord at 198.646 s (bottom) are visible.

Figure 4 .
Figure 4. EEG responses to maximally distant acoustical feature increases.Average EEG responses are shown for Dream Theater's Stream of Consciousness (left) and Igor Stravinsky's Rite of Spring (right).

Figure 4 .
Figure 4. EEG responses to maximally distant acoustical feature increases.Average EEG responses are shown for Dream Theater's Stream of Consciousness (left) and Igor Stravinsky's Rite of Spring (right).

Figure 5 .
Figure 5. Response to novel loudness increase in Dream Theater's Stream of Consciousness.Average MIR feature values per standard deviation, EEG, MEG axial magnetometer (MAG) and MEG planar gradiometer (GRAD) waveforms shown 100 ms before and 400 ms after the time point at 20.316 s.Topographies in relation to the head are shown for the negative N1-like brain response measured in a time window of ±15 ms around the peak.In the MEG topographies it can be seen that the brain response is strongest in sensors located approximately above brain regions involved in sound processing.

Figure 5 .
Figure 5. Response to novel loudness increase in Dream Theater's Stream of Consciousness.Average MIR feature values per standard deviation, EEG, MEG axial magnetometer (MAG) and MEG planar gradiometer (GRAD) waveforms shown 100 ms before and 400 ms after the time point at 20.316 s.Topographies in relation to the head are shown for the negative N1-like brain response measured in a time window of ±15 ms around the peak.In the MEG topographies it can be seen that the brain response is strongest in sensors located approximately above brain regions involved in sound processing.

Figure 6 .
Figure 6.Brain responses to a novel event compared to maximally distant feature increases.Histograms showing amplitudes of brain response in μV and number of participants responding within each voltage bin differing by steps of 5 μV.The brain responses are shown for the maximally distant loudness (RMS) (blue) and spectral flux increases (green) and for the novel loudness increase (red, left) and novel chord (red, right).The fit of normal distributions is indicated by solid lines.

Figure 6 .
Figure 6.Brain responses to a novel event compared to maximally distant feature increases.Histograms showing amplitudes of brain response in µV and number of participants responding within each voltage bin differing by steps of 5 µV.The brain responses are shown for the maximally distant loudness (RMS) (blue) and spectral flux increases (green) and for the novel loudness increase (red, left) and novel chord (red, right).The fit of normal distributions is indicated by solid lines.

Figure 7 .
Figure 7. Response to the 'Augurs' chord in Stravinsky's Rite of Spring.Average MIR feature values per standard deviation, EEG, MEG axial magnetometer (MAG) and MEG planar gradiometer (GRAD) waveforms shown 100 ms before and 400 ms after the time point at 198.646 s.Topographies in relation to the head are shown for the anterior MMN/ERAN-like brain response measured in a time window of ±15 ms around the peak.The MEG topographies suggest that frontal-central brain regions are involved in the response.

Figure 7 .
Figure 7. Response to the 'Augurs' chord in Stravinsky's Rite of Spring.Average MIR feature values per standard deviation, EEG, MEG axial magnetometer (MAG) and MEG planar gradiometer (GRAD) waveforms shown 100 ms before and 400 ms after the time point at 198.646 s.Topographies in relation to the head are shown for the anterior MMN/ERAN-like brain response measured in a time window of ±15 ms around the peak.The MEG topographies suggest that frontal-central brain regions are involved in the response.

Table 1 .
Automatically extracted time points.The number of peaks in the music information retrieval (MIR) feature increases, n magnitude of rapid increase (MoRI) peaks, corresponds to the MIR feature increase peaks among the 30% highest of the MoRI values.The assumed best time points, n time points, are the number of automatically extracted time points.For the assumed best time points with feature increase evoking brain responses the median PLIP in seconds and distance to previous time points in seconds are shown with ranges in parenthesis.