Adaptive Data Boosting Technique for Robust Personalized Speech Emotion in Emotionally-Imbalanced Small-Sample Environments

Bang, Jaehun; Hur, Taeho; Kim, Dohyeong; Huynh-The, Thien; Lee, Jongwon; Han, Yongkoo; Banos, Oresti; Kim, Jee-In; Lee, Sungyoung

doi:10.3390/s18113744

Submit to this Journal Review for this Journal Edit a Special Issue

Article Menu

...

Open AccessArticle

Adaptive Data Boosting Technique for Robust Personalized Speech Emotion in Emotionally-Imbalanced Small-Sample Environments

¹,

¹,

¹,

¹,

²,

³ and

^1,*

Department of Computer Science and Engineering, Kyung Hee University, (Global Campus), 1732, Deogyeong-daero, Giheung-gu, Yongin-si, Gyeonggi-do 17104, Korea

Department of Computer Architecture and Computer Technology, University of Granada, C/Periodista Daniel Saucedo Aranda s/n, E-18071 Granada, Spain

Department of Smart ICT Convergence, Konkuk University, 120 Neungdong-ro, Gwangjin-gu, Seoul 05029, Korea

Author to whom correspondence should be addressed.

Sensors 2018, 18(11), 3744; https://doi.org/10.3390/s18113744

Received: 26 June 2018 / Revised: 25 October 2018 / Accepted: 29 October 2018 / Published: 2 November 2018

(This article belongs to the Section Physical Sensors)

View Full-Text Download PDF

Browse Figures

Citation Export

Abstract

Personalized emotion recognition provides an individual training model for each target user in order to mitigate the accuracy problem when using general training models collected from multiple users. Existing personalized speech emotion recognition research has a cold-start problem that requires a large amount of emotionally-balanced data samples from the target user when creating the personalized training model. Such research is difficult to apply in real environments due to the difficulty of collecting numerous target user speech data with emotionally-balanced label samples. Therefore, we propose the Robust Personalized Emotion Recognition Framework with the Adaptive Data Boosting Algorithm to solve the cold-start problem. The proposed framework incrementally provides a customized training model for the target user by reinforcing the dataset by combining the acquired target user speech with speech from other users, followed by applying SMOTE (Synthetic Minority Over-sampling Technique)-based data augmentation. The proposed method proved to be adaptive across a small number of target user datasets and emotionally-imbalanced data environments through iterative experiments using the IEMOCAP (Interactive Emotional Dyadic Motion Capture) database. View Full-Text

Keywords: speech emotion recognition; personalization; machine learning; data selection; data augmentation

▼ Show Figures

Figure 1

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited

Share and Cite

MDPI and ACS Style

Bang, J.; Hur, T.; Kim, D.; Huynh-The, T.; Lee, J.; Han, Y.; Banos, O.; Kim, J.-I.; Lee, S. Adaptive Data Boosting Technique for Robust Personalized Speech Emotion in Emotionally-Imbalanced Small-Sample Environments. Sensors 2018, 18, 3744. https://doi.org/10.3390/s18113744

AMA Style

Bang J, Hur T, Kim D, Huynh-The T, Lee J, Han Y, Banos O, Kim J-I, Lee S. Adaptive Data Boosting Technique for Robust Personalized Speech Emotion in Emotionally-Imbalanced Small-Sample Environments. Sensors. 2018; 18(11):3744. https://doi.org/10.3390/s18113744

Chicago/Turabian Style

Bang, Jaehun, Taeho Hur, Dohyeong Kim, Thien Huynh-The, Jongwon Lee, Yongkoo Han, Oresti Banos, Jee-In Kim, and Sungyoung Lee. 2018. "Adaptive Data Boosting Technique for Robust Personalized Speech Emotion in Emotionally-Imbalanced Small-Sample Environments" Sensors 18, no. 11: 3744. https://doi.org/10.3390/s18113744

Find Other Styles

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Metrics

Abstract views Pdf views Html views

Article Menu

Adaptive Data Boosting Technique for Robust Personalized Speech Emotion in Emotionally-Imbalanced Small-Sample Environments

Abstract

Share and Cite

Article Metrics

Article Access Statistics

Article Access Map by Country/Region

Further Information

Guidelines

MDPI Initiatives

Follow MDPI