# Predicting Premature Video Skipping and Viewer Interest from EEG Recordings

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Materials and Methods

#### 2.1. Experimental Setup

#### 2.2. Data Preprocessing

#### 2.3. Analysis

#### 2.3.1. MSE

^{m}the probability that two sequences with length m are similar within r, and A

^{m}

^{+1}the probability that these two sequences remain similar within r when extending their length to m+1. The tolerance level was set to 0.15*std of the 1 s partition’s amplitude.

#### 2.3.2. Engagement, Arousal, and Valence Indices

#### 2.3.3. Features

#### 2.3.4. Classification

## 3. Results

#### 3.1. Skipping and Interest Prediction Accuracies

#### 3.2. Feature Relevance

## 4. Discussion

## Supplementary Materials

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- Abdulkader, N.S.; Atia, A.; Mostafa, M.M. Brain computer interfacing: Applications and challenges. Egypt. Inform. J.
**2015**, 16, 213–230. [Google Scholar] [CrossRef] [Green Version] - Wittevrongel, B.; van Hulle, M.M. Frequency- and Phase Encoded SSVEP Using Spatiotemporal Beamforming. PLoS ONE
**2016**, 11, 159988. [Google Scholar] [CrossRef] [PubMed] - Combaz, A.; Chatelle, C.; Robben, A.; Vanhoof, G.; Goeleven, A.; Thijs, V.; Van Hulle, M.M.; Laureys, S. A Comparison of Two Spelling Brain-Computer Interfaces Based on Visual P3 and SSVEP in Locked-In Syndrome. PLoS ONE
**2013**, 8, 73691. [Google Scholar] [CrossRef] [PubMed] - Wittevrongel, B.; Van Hulle, M.M. Faster P300 Classifier Training Using Spatiotemporal Beamforming. Int. J. Neural Syst.
**2016**, 26, 1650014. [Google Scholar] [CrossRef] [PubMed] - Jeon, Y.; Nam, C.S.; Kim, Y.; Cheol, M. Event-related (De) synchronization (ERD/ERS) during motor imagery tasks: Implications for brain e computer interfaces. Int. J. Ind. Ergon.
**2011**, 41, 428–436. [Google Scholar] [CrossRef] - Thakor, N.V.; Sherman, D.L. EEG Signal Processing: Theory and Applications BT. In Neural Engineering; He, B., Ed.; Springer: Boston, MA, USA, 2013; pp. 259–303. [Google Scholar]
- Mcmahan, T.; Parberry, I.; Parsons, T.D. Evaluating player task engagement and arousal using electroencephalography. Procedia Manuf.
**2015**, 3, 2303–2310. [Google Scholar] [CrossRef] - Freeman, F.G.; Mikulka, P.J.; Prinzel, L.J.; Scerbo, M.W. Evaluation of an adaptive automation system using three EEG indices with a visual tracking task. Biol. Psychol.
**1999**, 50, 61–76. [Google Scholar] [CrossRef] - Pope, A.T.; Bogartb, E.H.; Bartolomeb, D.S. Biocybernetic system evaluates indices of operator engagement in automated task. Biol. Psychol.
**1995**, 40, 187–195. [Google Scholar] [CrossRef] - Berka, C.; Levendowski, D.J.; Lumicao, M.N.; Yau, A.; Davis, G.; Zivkovic, V.T.; Olmstead, R.E.; Tremoulet, P.D.; Craven, P.L. EEG Correlates of Task Engagement and Mental Workload in Vigilance, Learning, and Memory Tasks. Aviat. Space Environ. Med.
**2007**, 78, 231–244. [Google Scholar] - Zhuang, N.; Zeng, Y.; Tong, L.; Zhang, C.; Zhang, H.; Yan, B. Emotion Recognition from EEG Signals Using Multidimensional Information in EMD Domain. BioMed Res. Int.
**2017**. [Google Scholar] [CrossRef] [PubMed] - Giraldo, S.; Ramirez, R. Brain-activity-driven real-time music emotive control. In Proceedings of the 3rd International Conference on Music & Emotion, Jyväskylä, Finland, 11–15 June 2013. [Google Scholar]
- van Camp, M.; de Boeck, M.; de Bruyne, G. EEG Technology for UX Evaluation: A Multisensory Perspective EEG Technology for UX Evaluation: A Multisensory Perspective; Springer: Cham, Switzerland, 2019. [Google Scholar]
- Lee, N.; Broderick, A.; Chamberlain, L. What is ‘Neuromarketing’? A Discussion and Agenda for Future Research. Int. J. Psychophysiol.
**2007**, 63, 199–204. [Google Scholar] [CrossRef] [PubMed] - Tonoyan, Y.; Looney, D.; Mandic, D.P.; van Hulle, M.M. Discriminating Multiple Emotional States from EEG Using a Data-Adaptive, Multiscale Information-Theoretic Approach. Int. J. Neural Syst.
**2016**, 26, 165005. [Google Scholar] [CrossRef] [PubMed] - Tonoyan, Y.; Chanwimalueang, T.; Mandic, D.P.; van Hulle, M.M. Discrimination of emotional states from scalp- and intracranial EEG using multiscale Rényi entropy. PLoS ONE
**2017**, 12, 186916. [Google Scholar] [CrossRef] [PubMed] - Wei, Q.; Liu, Q.; Fan, S.Z.; Lu, C.W.; Lin, T.Y.; Abbod, M.; Shieh, J.S. Analysis of EEG via multivariate empirical mode decomposition for depth of anesthesia based on sample entropy. Entropy
**2013**, 15, 3458–3470. [Google Scholar] [CrossRef] - Rehman, N.; Mandic, D.P. Multivariate empirical mode decomposition. Proc. R. Soc. A
**2009**, 466, 1291–1302. [Google Scholar] [CrossRef] - Wang, H.; Bi, L.; Teng, T. EEG-based emergency braking intention prediction for brain-controlled driving considering one electrode falling-off. In Proceedings of the 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Seogwipo, Korea, 11–15 July 2017. [Google Scholar]
- Azami, H.; Escudero, J. Coarse-Graining Approaches in Univariate Multiscale Sample and Dispersion Entropy. Entropy
**2018**, 20, 138. [Google Scholar] [CrossRef] - Costa, P.C.S.; de Melo, F. Coarse Graining of Partitioned Cellular Automata. arXiv
**2019**, arXiv:preprint/1905.10391. [Google Scholar] - Costa, M.; §Goldberger, A.L.; Peng, C. Multiscale Entropy Analysis of Complex Physiologic Time Series. Phys. Rev. Lett.
**2002**, 89, 68102. [Google Scholar] [CrossRef] [PubMed] - Mathworks. Matlab 2018b. Available online: http:// mathworks.com (accessed on 18 October 2019).
- Richman, J.S.; Moorman, J.R. Physiological time-series analysis using approximate entropy and sample entropy maturity in premature infants Physiological time-series analysis using approximate entropy and sample entropy. Am. J. Physiol. Hear. Circ. Physiol.
**2000**, 278, 2039–2049. [Google Scholar] [CrossRef] [PubMed] - Cristianini, N.; Shawe-Taylor, J. The Naive Solution: Gradient Ascent. In An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods; Cambridge University Press: Cambridge, UK, 2014; Volume 111, pp. 129–135. [Google Scholar]

**Figure 1.**Overview of the processing pipeline. Note that each step within the black outline applies to each frequency band. Abbreviations: MSE, multiscale version of sample entropy; SVM, support vector machine; kNN, k-nearest neighbor; EEG, electroencephalography.

**Figure 2.**Boxplots summarizing accuracies of skipping prediction using individual features for the SVM (red), kNN (blue), and RF (green) classifiers. Note: Outliers correspond to the distribution in its column.

**Figure 3.**As in Figure 3, but for interest prediction.

**Figure 4.**Difference between features in skipped and non-skipped cases. (

**a**) RCVB, (

**b**) RCVM, (

**c**) RCVT, (

**d**) REEI, (

**e**) REAI, and (

**f**) REVI.

**Figure 5.**Idem to Figure 5 but for interest and not interest cases. (

**a**) RCVB, (

**b**) RCVM, (

**c**) RCVT, (

**d**) REEI, (

**e**) REAI, and (

**f**) REVI.

**Table 1.**Accuracies for skipping and interest for the SVM, kNN, and random forest (RF) classifiers when using all features. Each column corresponds to the respective classifier, whilst each row corresponds to which subject was left out.

Skipped | SVM | kNN | RF | Interest | SVM | kNN | RF |
---|---|---|---|---|---|---|---|

s1 | 76.60 | 80.00 | 86.67 | s1 | 85.00 | 82.22 | 84.44 |

s2 | 96.97 | 84.44 | 77.78 | s2 | 64.52 | 80.00 | 75.56 |

s3 | 65.12 | 88.89 | 77.78 | s3 | 61.54 | 97.78 | 80.00 |

s4 | 64.52 | 75.56 | 77.78 | s4 | 82.14 | 53.33 | 62.22 |

Average | 75.8025 | 82.2225 | 80.0025 | Average | 73.3 | 78.3325 | 75.555 |

**Table 2.**Average accuracies and standard deviations for skipping (

**a**) and for interest (

**b**) when using individual features.

(a) | ||||||

Skipped | SVM | sd | kNN | sd | RF | sd |

RCVB | 0.734 | 0.113 | 0.712 | 0.099 | 0.654 | 0.105 |

RCVM | 0.705 | 0.118 | 0.647 | 0.098 | 0.621 | 0.105 |

RCVT | 0.759 | 0.109 | 0.684 | 0.099 | 0.615 | 0.105 |

REEI | 0.585 | 0.156 | 0.618 | 0.099 | 0.618 | 0.106 |

REAI | 0.578 | 0.144 | 0.608 | 0.108 | 0.583 | 0.106 |

REVI | 0.756 | 0.117 | 0.775 | 0.088 | 0.735 | 0.103 |

CV2EI | 0.518 | 0.131 | 0.516 | 0.107 | 0.544 | 0.115 |

CV2AI | 0.611 | 0.133 | 0.643 | 0.099 | 0.665 | 0.100 |

CV2VI | 0.676 | 0.113 | 0.723 | 0.099 | 0.602 | 0.104 |

(b) | ||||||

Interest | SVM | sd | kNN | sd | RF | sd |

RCVB | 0.676 | 0.131 | 0.740 | 0.096 | 0.640 | 0.105 |

RCVM | 0.670 | 0.137 | 0.719 | 0.102 | 0.591 | 0.113 |

RCVT | 0.693 | 0.139 | 0.728 | 0.095 | 0.614 | 0.107 |

REEI | 0.641 | 0.116 | 0.736 | 0.106 | 0.645 | 0.106 |

REAI | 0.654 | 0.122 | 0.696 | 0.099 | 0.592 | 0.108 |

REVI | 0.725 | 0.121 | 0.766 | 0.091 | 0.694 | 0.098 |

CV2EI | 0.465 | 0.138 | 0.607 | 0.107 | 0.577 | 0.107 |

CV2AI | 0.548 | 0.150 | 0.683 | 0.098 | 0.629 | 0.106 |

CV2VI | 0.396 | 0.177 | 0.526 | 0.106 | 0.463 | 0.108 |

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Libert, A.; Van Hulle, M.M.
Predicting Premature Video Skipping and Viewer Interest from EEG Recordings. *Entropy* **2019**, *21*, 1014.
https://doi.org/10.3390/e21101014

**AMA Style**

Libert A, Van Hulle MM.
Predicting Premature Video Skipping and Viewer Interest from EEG Recordings. *Entropy*. 2019; 21(10):1014.
https://doi.org/10.3390/e21101014

**Chicago/Turabian Style**

Libert, Arno, and Marc M. Van Hulle.
2019. "Predicting Premature Video Skipping and Viewer Interest from EEG Recordings" *Entropy* 21, no. 10: 1014.
https://doi.org/10.3390/e21101014