Phonetic Variation Modeling and a Language Model Adaptation for Korean English Code-Switching Speech Recognition
1
Software Development Department, IIR TECH Inc., Daejeon 34134, Korea
2
Artificial Intelligence Research Laboratory, Electronics and Telecommunications Research Institute, Daejeon 34129, Korea
*
Author to whom correspondence should be addressed.
Academic Editor: José A. González-López
Appl. Sci. 2021, 11(6), 2866; https://doi.org/10.3390/app11062866
Received: 16 February 2021
/
Revised: 16 March 2021
/
Accepted: 17 March 2021
/
Published: 23 March 2021
(This article belongs to the Section Computing and Artificial Intelligence)
In this paper, we propose a new method for code-switching (CS) automatic speech recognition (ASR) in Korean. First, the phonetic variations in English pronunciation spoken by Korean speakers should be considered. Thus, we tried to find a unified pronunciation model based on phonetic knowledge and deep learning. Second, we extracted the CS sentences semantically similar to the target domain and then applied the language model (LM) adaptation to solve the biased modeling toward Korean due to the imbalanced training data. In this experiment, training data were AI Hub (1033 h) in Korean and Librispeech (960 h) in English. As a result, when compared to the baseline, the proposed method improved the error reduction rate (ERR) by up to 11.6% with phonetic variant modeling and by 17.3% when semantically similar sentences were applied to the LM adaptation. If we considered only English words, the word correction rate improved up to 24.2% compared to that of the baseline. The proposed method seems to be very effective in CS speech recognition.
View Full-Text
Keywords:
speech recognition; code-switching; language model; domain adaptation; acoustic model; shallow fusion
▼
Show Figures
Figure 1
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited
MDPI and ACS Style
Lee, D.; Kim, D.; Yun, S.; Kim, S. Phonetic Variation Modeling and a Language Model Adaptation for Korean English Code-Switching Speech Recognition. Appl. Sci. 2021, 11, 2866. https://doi.org/10.3390/app11062866
AMA Style
Lee D, Kim D, Yun S, Kim S. Phonetic Variation Modeling and a Language Model Adaptation for Korean English Code-Switching Speech Recognition. Applied Sciences. 2021; 11(6):2866. https://doi.org/10.3390/app11062866
Chicago/Turabian StyleLee, Damheo, Donghyun Kim, Seung Yun, and Sanghun Kim. 2021. "Phonetic Variation Modeling and a Language Model Adaptation for Korean English Code-Switching Speech Recognition" Applied Sciences 11, no. 6: 2866. https://doi.org/10.3390/app11062866
Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.
