Low-Resourced Alphabet-Level Pivot-Based Neural Machine Translation for Translating Korean Dialects

Abstract

Developing a machine translator from a Korean dialect to a foreign language presents significant challenges due to a lack of a parallel corpus for direct dialect translation. To solve this issue, this paper proposes a pivot-based machine translation model that consists of two sub-translators. The first sub-translator is a sequence-to-sequence model with minGRU as an encoder and GRU as a decoder. It normalizes a dialect sentence into a standard sentence, and it employs alphabet-level tokenization. The other type of sub-translator is a legacy translator, such as off-the-shelf neural machine translators or LLMs, which translates the normalized standard sentence to a foreign sentence. The effectiveness of the alphabet-level tokenization and the minGRU encoder for the normalization model is demonstrated through empirical analysis. Alphabet-level tokenization is proven to be more effective for Korean dialect normalization than other widely used sub-word tokenizations. The minGRU encoder exhibits comparable performance to GRU as an encoder, and it is faster and more effective in managing longer token sequences. The pivot-based translation method is also validated through a broad range of experiments, and its effectiveness in translating Korean dialects to English, Chinese, and Japanese is demonstrated empirically.

1. Introduction

Most languages have their own dialects derived from social, ethnic, and regional causes. The Korean language also has eight major dialects according to the work of Choi [1]. They are Gyeongi, Gangwon, Chungcheong, Jeolla, Gyeongsang, Jeju, Pyeongan, and Hamgyeong, where the Gyeongi dialect is regarded as standard Korean. The main characteristic of Korean dialects is that they are specialized geographically, but each region in which a dialect is spoken is not large. Thus, all the dialects share the same grammar, while every dialect has its own specialized words and some morphological transitions.

The specialized words in a dialect make it difficult to directly translate the dialect to a foreign language. Figure 1 shows such an example. The standard sentence “이 나이에도 여전히 걱정만 끼쳐 죄송해요.” is translated to“I’m sorry I still worry you even at this age.” by Google translator. However, its corresponding sentence in Gangwon dialect, “이 나으에도 여전히 극정만 끼쳐 죄송해요,” is wrongly translated to “I’m sorry that I still cause extreme pain even in this situation.” This poor translation is caused by two dialect words, ‘나으 (age)’ and ‘극정 (worry)’. Even if they look similar on the surface to their standard forms of ‘나이’ and ‘걱정’, they are considered as out-of-vocabulary (OOV) words by legacy Korean–English machine translators and result in an incorrect translation.

The aforementioned OOV issue in managing Korean dialects is mainly caused by modern sub-word tokenizers which are naively applied to Korean at the syllable level. This is the point where the first research question arises: ‘Are there any better options for normalizing Korean dialects than simply applying syllable level sub-word tokenizers and a transformer architecture?’ We found some fundamental insight that Korean dialects exhibit unique phonetic and morphological characteristics that are not adequately captured by syllable-level tokenization methods: for instance, the verb ‘카다 (do)’ in the Gyeongsang dialect, which is pronounced as ka-da. The only difference in this verb from its standard form ‘하다’, pronounced as ha-da, is the first syllable ‘ka’ instead of ‘ha’. We suggested such examples in

Table 1. Example specialized words in Korean dialects.

Dialect	Standard	Dialect	Meaning
Gyeongsang	하다 [hada]	카다 [kada]	do
Jeolla	버르장머리 [pʌrɯdzaŋmʌri]	버르쟁이 [pʌrɯdzɛŋi]	courtesy
Jeju	있었어? [iśʌśʌ]	있언? [iśʌn]	was it?
Gangwon	고등학교 [kodɯŋhakkyo]	고등핵교 [kodɯŋhɛkkyo]	high school
Chungcheong	어떻게 [ʌt́ʌkhe]	어트케 [ʌdhɯkhe]	how

. Nevertheless, both words are represented differently by a sub-word tokenizer. According to Li et al. [2], character-level modeling is better than sub-word modeling in agglutinative languages especially when a translator is trained with a small dataset. Since Korean is an agglutinative language and its alphabet shares many features with English characters, the proposed normalization model performs at the Korean alphabet—Jamo—level. This is the answer to our first research question.

Then, the second research question of this paper arises: ‘Is there any way to apply a novel dialect normalization method to more practical fields?’ We found the answer: ‘by translating Korean dialects to foreign languages’. Korean dialects are spoken rather than written, but the needs for translating written dialects are increasing. Nowadays, commercial speech-to-text and speech-to-speech translators are emerging in translation fields. In such cases, spoken dialects are transcribed into text before translation. The current challenge in the field is a lack of available data to train a machine translator from a Korean dialect to a foreign language. That is, there is no publicly available parallel corpus for Korean dialect translation. Thus, the proposed model employs a pivot-based approach which adopts standard Korean as a pivot language. The pivot-based approach is a common choice in many low-resourced languages [3,4,5]. The normalization from a dialect to a standard sentence can be achieved by the proposed alphabet-level translation model. Once a dialect sentence is normalized to a standard sentence, the standard sentence can be translated to a foreign sentence by a legacy translator such as off-the-shelf neural machine translators or large language models.

The contributions of this paper can be summarized as follows:

• This is the first work, to the best of our knowledge, to translate a Korean dialect to a foreign language with a pivot-based translation model. According to the experimental results, the proposed pivot-based translation outperforms the direct translation.
• Alphabet-level tokenization is used to normalize dialect sentences, and its superiority to other sub-word tokenizations is shown empirically.
• The proposed sequence-to-sequence model for dialect normalization adopts minGRU as an encoder and GRU as a decoder. This paper proves that minGRU as an encoder can be an alternative option to GRU as an encoder, since minGRU is faster and more effective in managing longer token sequences than GRU.

The rest of this paper is organized as follows. Section 2 surveys previous works on dialects and their translations. Section 3 explains how the proposed pivot-based Korean dialect translator is structured, and Section 4 describes normalizing a dialect sentence to a standard form in detail. Section 5 presents the evaluation results, and finally Section 6 draws conclusions of this work.

2. Related Work

Dialect translations are clustered into two categories: inner dialect translation and dialect foreign translation. Inner dialect translation is targeting the translation between dialects of a language. It includes the translation of a non-standard dialect into a standard one, which is known as dialect normalization [6]. On the other hand, dialect foreign translation focuses on translation between a dialect and a foreign language.

One critical issue in both dialect translations is to cope with lexical variations. Tan et al. [7] proposed Base-Inflectional Encoding (BITE), which can be applied to any pre-trained language model with ease. It leverages inflectional features of English and thus is robust even in non-standard English. Abe et al. [8] tried to capture consistent phonetic transformation rules shared by various Japanese dialects. Thus, they used a multilingual NMT [9] to translate a dialect into standard Japanese. On the other hand, Honnet et al. [10] and Sajjad et al. [11] empirically proved that a character-level processing is effective in managing variations of Swiss German and Egyptian Arabic, respectively.

Another critical issue in dialect translation is the lack of resources for training translation models. Faheem et al. [12] applied a semi-supervised approach to normalize Egyptian Arabic to Standard Arabic in order to overcome the lack of training data. On the other hand, Liu et al. [13] prepared a dataset for direct dialect translation. They created a parallel corpus from Singlish to Standard English. This stresses on lexical-level normalization, syntactic-level editing, and semantic-level rewriting. When a machine translator is trained with limited data, it is prone to be excessively affected by noise or superficial lexical features. Therefore, input perturbations were adopted at both the word and sentence levels.

In dialect foreign translation, a pivot-based translation is a common approach to circumvent the low-resource problem [14,15]. In this approach, a dialect is first translated into a standard form, and then the standard form is translated again into a foreign language. In addition to this approach, back translation is often adopted to solve the issue of limited dialect data [16,17]. For instance, Tahssin et al. [18] applied back translation to overcome data imbalance. At the same time, there have been some efforts to construct data for the direct translation of dialects. Riley et al. [19] presented a benchmark for few-shot region-aware machine translation. This benchmark includes language pairs of English and two regional dialects of Portuguese and Mandarin Chinese. On the other hand, Sun et al. [20] proposed a translation evaluation method which is robust to dialects.

Recent studies on the translation of Korean regional dialects have mainly focused on exploring and improving existing neural machine translation methods. Lim et al. [21] adopted a transformer-based architecture and a syllable-level SentencePiece tokenizer for Korean dialect translation. They also confirmed the effectiveness of the copy mechanism and the many-to-one translation approach. Hwang and Yang [22] took a pre-training and fine-tuning approach in Korean normalization. They fine-tuned a BART variant using standard BPE tokenization and regional information tokens. Similarly, Lee et al. [23] demonstrated the potential of large language models (LLMs) as a translator for the Jeju dialect.

Korean dialects are used more frequently in speech than in writing. Therefore, the research on speech dialect recognition is essential, yet this topic has only been addressed by a few studies. Roh and Lee [24] showed an early exploration of this topic. The experiments were performed to investigate how commercial APIs could be used to recognize Korean dialects. According to their study, the Google Speech Recognition API is more accurate than other APIs. However, despite its high accuracy, challenges remain in recognizing dialects due to the unique phonetic and lexical characteristics of the Korean language. Na et al. [25] suggested an insight regarding how off-the-shelf ASR systems can be adapted for dialect recognition. Their experimental results showed that the modern ASR systems such as Whisper and wav2vec 2.0 perform well in recognizing Korean dialects. More recently, Bak et al. [26] improved the performance of Whisper’s dialect recognition by refining its results using GPT-4o-mini. In this study, transcription errors in Whisper were corrected by applying RAG to the GPT-4o-mini language model.

3. Pivot-Based Translation for Korean Dialects

Due to a lack of parallel corpus between Korean dialects and foreign languages, it is extremely difficult to construct a direct machine translator for Korean dialects. Thus, this paper adopts standard Korean as a pivot language between a dialect and a foreign language. That is, the proposed translator first translates a dialect sentence into a standard Korean sentence and then translates that into a foreign language.

Figure 2 depicts the overall structure of the proposed pivot-based translator for Korean dialects. It consists of two sub-translators. A dialect sentence is first normalized to a standard sentence by a GRU-based sequence-to-sequence model explained below. For instance, legacy machine translators do not understand a Jeju dialect sentence “목사님 그 앞에 모니터 좀 있으면 좋지 않안허쿠가 영 하니까,” where this dialect sentence is translated incorrectly into “The pastor, it would be nice to have a monitor in front of him because he’s so young.” However, they accept its standard sentence “목사님 그 앞에 모니터 좀 있으면 좋지 않겠어요 이렇게 하니까,” of which the meaning is “Pastor, wouldn’t it better if there is a monitor in front of the stuff like this?”

Once a standard sentence has been obtained, it can be translated again into a foreign language by a legacy machine translator. Due to the large volume of parallel corpora between standard Korean and major foreign languages, a number of machine translators including LLMs show high and reliable performance. This paper leverages neural machine translators from easyNMT (https://github.com/UKPLab/EasyNMT (accessed on 28 July 2025)) and Exaone [27], a Korean LLM, to translate standard Korean to a foreign language. Then, finally, the Jeju dialect sentence is translated into “Wouldn’t it be better if there is a monitor in front of the pastor like this?” if a target language is English.

4. Normalization Model from Dialect to Standard

4.1. Tokenization

There are several options for tokenizing a Korean text. Four widely used tokenizations among them are syllable-level SentencePiece, byte-level BPE, morpheme-level, and alphabet-level. The tokens by these tokenizations for the example dialect sentence used in Figure 2 are shown in

Table 2. An example of tokenization for a Korean dialect sentence. ‘##’ in morpheme-level tokens and ‘▁’ in SentencePiece tokens represent white space information.

Tokenization Method	Tokens
dialect sentence	목사님   그   앞에   모니터   좀   있으면   좋지   않안허쿠가   영   하니까
standard sentence	목사님   그   앞에   모니터   좀   있으면   좋지   않겠어요   이렇게   하니까
SentencePiece	▁목사님  ▁그  ▁앞에  ▁모니터  ▁좀  ▁있으면  ▁좋지  ▁않안  허  쿠가
	▁영  ▁하니까
byte-level BPE	목 ìĤ¬ëĭĺ Ġê·¸ ĠìķŀìĹIJ Ġ모ëĭĪ íĦ° Ġì¢Ģ ĠìŀĪìľ¼ë©´ Ġì¢ĭì§Ģ ĠìķĬìķĪ íĹĪ
	ì¿łê°Ģ Ġìĺģ ĠíķĺëĭĪê¹Į
morpheme-level	목사  ##님  그  앞  ##에  모니터  좀  있  ##으면  좋  ##지  않  ##안  ##허  ##쿠
	##가  영  하  ##니까
	ㅁ  ㅗ  ㄱ  <SEP>  ㅅ  ㅏ  <SEP>  ㄴ  ㅣ  ㅁ  <SEP>  <SPC>  ㄱ  ㅡ  <SEP>  <SPC>
	ㅇ  ㅏ  ㅍ  <SEP>  ㅇ  ㅔ  <SEP>  <SPC>  ㅁ  ㅗ  <SEP>  ㄴ  ㅣ  <SEP>  ㅌ  ㅓ
	<SEP>  <SPC>  ㅈ  ㅗ  ㅁ  <SEP>  <SPC>  ㅇ  ㅣ  ㅆ  <SEP>  ㅇ  ㅡ  <SEP>
alphabet-level	ㅁ  ㅕ  ㄴ  <SEP>  <SPC>  ㅈ  ㅗ  ㅎ  <SEP>  ㅈ  ㅣ  <SEP>  <SPC>  ㅇ   ㅏ  ㄶ
	<SEP>  ㅇ  ㅏ  ㄴ  <SEP>  ㅎ  ㅓ  <SEP>  ㅋ  ㅜ  <SEP>  ㄱ  ㅏ  <SEP>  <SPC>
	ㅇ  ㅕ  ㅇ  <SEP>  <SPC>  ㅎ  ㅏ  <SEP>  ㄴ  ㅣ  <SEP>  ㄲ  ㅏ  <SEP>

. Korean is a syllabary language. Thus, a syllable can be a natural tokenization unit. However, since a syllable is a combination of several base alphabets in Korean, the number of possible syllables is extremely large. As a result, syllable-level tokens such as syllable-level SentencePiece trained with a insufficient corpus can result in a poor performance. Byte-level BPE is effective because it does not suffer from the OOV problem, as it processes Korean texts at the byte level. However, it generates illegible output which is far from the original sentence as shown in this table. Therefore, this method is not intuitive.

Another option is to leverage morphemes as the unit of tokenization. This is standard practice in both rule-based and statistics-based machine translation. Since a morpheme is the smallest unit of meaning, this tokenization can preserve the meaning of each word. However, it depends on the performance of a morphological analyzer and also suffers from the OOV problem.

The last option for Korean tokenization is to use the Korean alphabet, Jamo. This paper proposes alphabet-level tokenization for dialect normalization. The proposed tokenizer uses Korean alphabets, alpha-numeric symbols, and two special tokens of ‘<SEP>’ and ‘<SPC>’ as tokenizing units. ‘<SEP>’ indicates the end of a syllable and ‘<SPC>’ represents a white space. For instance, a word “목사님” is represented as “ㅁ, ㅗ, ㄱ, <SEP>, ㅅ, ㅏ, <SEP>, ㄴ, ㅣ, ㅁ, <SEP> <SPC>”. Since there are only 24 letters in the Korean alphabet, Korean alphabet-level tokenization enjoys the same benefits as English character-level tokenization.

4.2. Normalization Model

A Korean dialect sentence and its standard sentence share most alphabet sequences. Therefore, the Gated Recurrent Unit (GRU) model proposed by Cho et al. [28] can be used for dialect normalization. It has shown reasonable performances in many NLP tasks and is more efficient than LSTM since it has fewer parameters.

The proposed model for dialect normalization is depicted in Figure 3. It is a GRU-based sequence-to-sequence model. However, its encoder is minGRU [29] rather than GRU since GRU suffers from slow convergence by back propagation-through-time, and alphabet-level token sequences are quite long in general, as shown in Table 2. On the other hand, its decoder is an original GRU. This is because the decoder aims at the precise generation of a target sentence in an auto-regressive way.

Assume that a natural language dialect sentence $\mathbf{x}=\left\{x_1,\dots ,x_n\right\}$ is given. If the t-th alphabet-level token is expressed as a vector embedding ${\mathbf{x}}_t\in {\mathbb{R}}^b$, then the sentence is represented as a matrix $\mathbf{X}\in {\mathbb{R}}^{n\times b}$ by concatenating ${\mathbf{x}}_t$ values. The encoder is a bi-directional multi-layer minGRU. That is, it consists of L minGRU layers. In the l-th layer ($1\le l\le L$), a minGRU transforms ${\mathbf{x}}_t$ into a hidden state vector $h_t^l\in {\mathbb{R}}^b$. In order to speed up training, minGRU removes hidden state dependencies of GRU and reduces hyperbolic functions. That is, when ⊙ is a point-wise multiplication, $h_t^l$ is obtained by

$$\begin{array}{ccc}\hfill h_t^l& =& (1-z_t)\odot h_{t-1}^l+z_t\odot {\tilde{h}}_t,\hfill \end{array}$$

(1)

where

$$\begin{array}{ccc}\hfill z_t& =& \sigma \left({Linear}_d\left(h_t^{l-1}\right)\right),\hfill \end{array}$$

(2)

$$\begin{array}{ccc}\hfill {\tilde{h}}_t& =& {Linear}_d\left(h_t^{l-1}\right).\hfill \end{array}$$

(3)

Here, $\sigma$ and ${Linear}_d$ denote a sigmoid activation function and a d-dimensional linear transformation, respectively. Note that there is no reset gate $r_t$ in these equations. In addition, the forget gate of GRU represented as $z_t^{GRU}=\sigma \left({Linear}_d(x_t\odot h_{t-1}^l)\right)$ is replaced with $z_t=\sigma \left({Linear}_d\left(x_t\right)\right)$. Compared with $z_t^{GRU}$, $z_t$ has no dependency on the previous hidden state $h_{t-1}^l$. Similarly, ${\tilde{h}}_t$ also does not depend on $h_{t-1}^l$. As a result, $z_t$ and ${\tilde{h}}_t$ can be processed in parallel for all t values. After $1-z_t$ and $z_t\odot {\tilde{h}}_t$ are computed for all t values, the hidden state $h_t^l$ in Equation (1) is obtained in parallel using the Parallel Scan algorithm [30,31]. The final output of the l-th layer becomes ${\mathbf{H}}_e^l=\left\{h_1^l,\dots ,h_n^l\right\}$. Equations (2) and () depend on $h_t^{l-1}$, which is the hidden state of the $(l-1)$-th layer. Thus, computing ${\mathbf{H}}_e^l$ can be understood as

$$\begin{array}{c}\hfill {\mathbf{H}}_e^l=minGRU\left({\mathbf{H}}_e^{l-1}\right),\end{array}$$

where ${\mathbf{H}}_e^0=\mathbf{X}$.

In order to reflect bi-directional contexts to the hidden state, both ${\overrightarrow{\mathbf{H}}}_e^l=\left\{{\overrightarrow{h}}_1^l,\dots ,{\overrightarrow{h}}_n^l\right\}$ and ${\overleftarrow{\mathbf{H}}}_e^l=\left\{{\overleftarrow{h}}_1^l,\dots ,{\overleftarrow{h}}_n^l\right\}$ are used. That is, ${\overline{\mathbf{H}}}_e^l\in {\mathbb{R}}^{n\times 2b}$ is used as a final hidden state matrix, where ${\overline{\mathbf{H}}}_e^l$ is $\left\{{\overline{h}}_1^l,\dots ,{\overline{h}}_n^l\right\}$ and ${\overline{h}}_t^l=[{\overrightarrow{h}}_t^l,{\overleftarrow{h}}_t^l]$. Here, $[·,·]$ is a concatenation of two vectors. Then, the final output of the encoder is ${\overline{\mathbf{H}}}_e^L$, which is the hidden state matrix of the L-th bi-directional minGRU layer.

The decoder of the proposed translator is a GRU which generates $y_1,\dots ,y_m$ auto-regressively. It applies the dot-product attention [32] between $h_t^d\in {\mathbb{R}}^{2b}$, the hidden state of the decoder, and ${\overline{\mathbf{H}}}_e^L$, the last hidden state of the encoder, to generate an output. That is, in generating $y_t$, the attention score $a_t$ is first calculated by

$$\begin{array}{c}\hfill a_t=softmax({\overline{\mathbf{H}}}_e^{L⊺}·h_t^d).\end{array}$$

Then, the context vector $c_t$ becomes

$$\begin{array}{c}\hfill c_t={\overline{\mathbf{H}}}_e^L·a_t.\end{array}$$

Since the decoder is a GRU, the hidden state vector of the decoder becomes

$$\begin{array}{cc}\hfill h_t^d=& GRU([{\mathbf{y}}_{t-1},c_{t-1}],h_{t-1}^d),\end{array}$$

(4)

where ${\mathbf{y}}_{t-1}\in {\mathbb{R}}^b$ is an embedding of $y_{t-1}$. Finally, when V is a vocabulary, $y_t$, the t-th output, is generated by

$$\begin{array}{ccc}\hfill {\tilde{\mathbf{y}}}_t& =& {Linear}_{\left|V\right|}\left([h_t^d,c_t]\right),\hfill \\ \hfill {\tilde{\mathbf{y}}}_t^{one}& =& \mathrm{one}\text{-}\mathrm{hot}\left({\tilde{\mathbf{y}}}_t\right),\hfill \\ \hfill y_t& =& Lookup\left({\tilde{\mathbf{y}}}_t^{one}\right).\hfill \end{array}$$

In these equations, ${\tilde{\mathbf{y}}}_t$ is a vector whose dimension is $\left|V\right|$, and it is converted to the output token $y_t$ through one−hot operation and vocabulary lookup. In Figure 3, ‘ㅗ’ is generated through this process and its embedding, ${\mathbf{y}}_t$, is fed to the generation of $y_{t+1}$.

5. Experiments

5.1. Experimental Settings

The Korean Dialect Speech Dataset released at the Korea AI Hub (https://aihub.or.kr (accessed on 28 July 2025)) is used for the training and evaluation of Korean dialect normalization. This dataset contains five South Korean dialects of Gyeongsang, Jeolla, Jeju, Gangwon, and Chungcheong. Two dialects of Pyeongan and Hamgyeong are not included in this dataset. This dataset was collected and published in South Korea, where it is impossible to collect data on these two dialects since they are mainly spoken in North Korea. Further details and linguistic features of these dialects can be found in the work of Choi [1]. The dataset consists of audio dialogue and its corresponding labels. The labels include various information about the dialect speakers, such as their age and place of residence. The labeled sentences are presented in two forms: a dialect sentence and its standard equivalent, with the two forms aligned and provided for every speech. Thus, the pairs of a dialect sentence and its standard sentence extracted from this dataset are regarded as a parallel corpus for normalizing Korean dialects.

Since the sentences in the dataset are spoken dialects, they are pre-processed to remove their noise such as stutters and laughter. In addition, sentences that are too short, with fewer than four eojeols, are also removed. An eojeol is a spacing unit in Korean. The original dataset contains a large portion of pairs in which the standard and dialect sentences are exactly the same. This is because the dataset comprises spoken dialogues. If speakers conversed in standard Korean, the standard and dialect forms are labeled as the same. Thus, all such cases are filtered out of the dataset. A simple statistic on the final pre-processed dataset is provided in

Table 3. A simple statistic of the dataset for Korean dialect normalization.

Dialect	# of Training Pairs	# of Validation Pairs	# of Test Pairs
Gyeongsang	260,494 (2,088,717)	14,210 (89,512)	14,181 (89,511)
Jeolla	254,207 (1,992,101)	25,922 (110,458)	25,727 (110,459)
Jeju	758,384 (2,774,257)	42,938 (80,062)	42,669 (80,061)
Gangwon	557,969 (1,573,237)	24,203 (91,346)	24,084 (91,345)
Chungcheong	260,494 (1,848,455)	15,434 (95,000)	15,601 (95,000)

. The number in the parenthesis indicates the number of original pairs. The ratio of dialect pairs in the training set varies considerably from 12% to 35%. Nevertheless, there are enough pairs to train and test a model for normalizing Korean dialects.

Four kinds of tokenizations are evaluated for normalizing Korean dialects: syllable-level SentencePiece, byte-level BPE, morpheme level, and alphabet level. SentencePiece tokenization is implemented with the SentencePiece module from GitHub (v. 0.2.0) and BPE is implemented with the ByteLevelBPETokenizer class from the tokenizers module of HuggingFace. Korean morphemes are analyzed by the MeCab-ko (https://github.com/hephaex/mecab-ko (accessed on 28 July 2025)) analyzer, and the decomposition of a syllable to an alphabet sequence is accomplished with the jamo module from GitHub.

The normalization models are trained to optimize the cross-entropy loss with the Adam optimizer and the ReduceLROnPlateau scheduler. The batch size was set to 200 for all tokenization and dialects except for morpheme-level tokens. As the vocabulary size of morpheme-level tokens is larger than others, the batch size for these was set to 64. The learning rate was initialized at 5 × ${10}^{-4}$. The weight decay is set to 1 × ${10}^{-5}$. Additionally, b in Equation (1), the dimension of the hidden state vectors of an encoder, is set as 128. Thus, that of a decoder is 256. The number of layers, L, is set to three.

The translation results are evaluated with ChrF++, BLEU, and BERT score. Among the metrics, BLEU and ChrF++ are used to assess the normalization results. Note that phonemic transitions are predominantly observed in Korean dialects. Thus, the primary work of the normalization models is to reconstruct phonemic varieties to restore the standard form. This is why n-gram-based metrics are useful for evaluating dialect normalization models. In particular, ChrF++ is designed for character-level evaluation, while BLEU focuses at the word level. On the other hand, the results of pivot translation are evaluated with ChrF++ and BERT score. Since this task is an ordinary translation task, the BERT score is adopted to evaluate the semantic similarity between original and translated sentences.

The translation models used to translate normalized dialects into foreign languages are (i) Opus-MT [33], (ii) m2m_100_1.2B [34], and (iii) EXAONE-3.0-7.8B-Instruct [27]. Llama-3.1-8B-Instruct is used to generate reference translations for pivot translation experiments. The Llama model is adopted since it is one of the most popular publicly available LLMs. Opus-MT and m2m_100_1.2B are neural machine translators. Opus-MT is designed based on Marinan NMT [35] and is trained using OPUS datasets [36]. On the other hand, m2m_100_1.2B is a many-to-many translation model. It can translate any pair of the one hundred languages it has been trained on. Their trained checkpoints are loaded and used by easyNMT, where easyNMT is a Python library (v. 2.0.2) for neural machine translation which provides a unified interface to various translation models. Two neural translation models have been selected as a baseline because they are effective and can be easily adapted for practical use through easyNMT. Exaone is a large language model (LLM) developed by LG AI Research. According to Sim et al. [37], it is specialized in understanding Korean culture, which implies that it is better at processing Korean dialects. This is the core reason why Exaone was adopted for this experiment. The LGAI-EXAONE/EXAONE-3.0-7.8B-Instruct checkpoint from HuggingFace is used in the experiments. Translations into English, Japanese, and Chinese are evaluated for m2m_100_1.2B and Exaone. However, only the English translation is reported for Opus-MT because the only available Opus-MT checkpoint is from Korean to English.

Two LLMs, Llama-3.1-8B-Instruct and Exaone-3.0-7.8B-Instruct, are used in the experiments. The prompts for each model are designed to elicit the desired translation outcomes. The prompt examples for each model are shown in Figure 4. The prompts are given in a zero-shot setting with no additional fine tuning. The prompts commonly include “다음 문장을 영어로 번역해줘 (Please translate the following sentence into English)”. For other languages, the term ‘영어 (English)’ is replaced with ‘일본어 (Japanese)’ or ‘중국어 (Chinese)’. The prompts are written in Korean for both models, forcing the language model to focus more on the Korean translation task. On the other hand, instruction for their output is in two forms: “You should generate only the translated text” for Llama-3.1-8B-Instruct and “번역한 문장만 출력하도록 해 (Please output only the translated sentence)” for Exaone-3.0-7.8B-Instruct. This is because Llama-3.1-8B-Instruct was primarily trained on English data, whereas Exaone-3.0-7.8B-Instruct was specialized for Korean.

Normalization experiments were performed three times with randomly initialized weights. The table states the average and standard deviation of the evaluation metrics across these three runs. This is crucial for evaluating the efficiency of each tokenization methods. Conversely, pivot translation experiments are conducted once for each translation model. This is because the translation models are pre-trained and their weights are fixed. The normalization model used in the pivot translation experiments is the model that performed best in the normalization experiments.

5.2. Evaluations on Normalization from Dialect to Standard

Table 4. Korean dialect normalization results evaluated on BLEU. * Means that the difference is statistically significant (p < 0.05) compared to the alphabet-level tokenization. The bold values indicate the best performance in each column.

Tokenizations	Gyeongsang	Jeolla	Jeju	Gangwon	Chungcheong	Overall
SentencePiece	$76.3\pm 4.0$ *	$28.6\pm 14$ *	$79.2\pm 4.8$	$51.7\pm 12$ *	$33.3\pm 13$ *	53.82
byte-level BPE	$48.5\pm 9.9$ *	$24.0\pm 7.0$ *	$81.2\pm 4.1$	$28.8\pm 12$ *	$21.7\pm 6.8$ *	40.84
morpheme-level	$93.5\pm 0.8$ *	$85.8\pm 2.2$ *	$81.4\pm 2.1$ *	$83.1\pm 2.4$ *	$83.1\pm 3.1$ *	85.38
alphabset-level	$\mathbf{97}.\mathbf{9}\pm \mathbf{0}.\mathbf{4}$	$\mathbf{96}.\mathbf{5}\pm \mathbf{0}.\mathbf{4}$	$\mathbf{90}.\mathbf{0}\pm \mathbf{0}.\mathbf{4}$	$\mathbf{93}.\mathbf{7}\pm \mathbf{0}.\mathbf{8}$	$\mathbf{96}.\mathbf{3}\pm \mathbf{0}.\mathbf{8}$	94.88

and

Table 5. Korean dialect normalization results evaluated on chrF++. * Implies statistical significance (p < 0.05) over the alphabet-level tokenization. The bold values indicate the best performance in each column.

Tokenizations	Gyeongsang	Jeolla	Jeju	Gangwon	Chungcheong	Overall
SentencePiece	$66.6\pm 5.1$ *	$20.3\pm 11$ *	$69.4\pm 6.0$	$39.2\pm 12$ *	$22.7\pm 11$ *	43.64
byte-level BPE	$36.3\pm 9.7$ *	$15.1\pm 4.9$ *	$71.9\pm 5.2$	$19.7\pm 9.6$ *	$13.5\pm 4.5$ *	31.3
morpheme-level	$90.3\pm 0.9$ *	$79.0\pm 3.2$ *	$72.5\pm 2.8$ *	$75.1\pm 3.2$ *	$75.2\pm 4.4$ *	78.42
alphabset-level	$\mathbf{96}.\mathbf{4}\pm \mathbf{0}.\mathbf{7}$	$\mathbf{94}.\mathbf{0}\pm \mathbf{0}.\mathbf{5}$	$\mathbf{84}.\mathbf{0}\pm \mathbf{0}.\mathbf{6}$	$\mathbf{89}.\mathbf{8}\pm \mathbf{1}.\mathbf{2}$	$\mathbf{93}.\mathbf{6}\pm \mathbf{1}.\mathbf{3}$	91.56

compare the performance of tokenization methods when normalizing Korean dialects. According to these tables, alphabet-level tokenization outperforms all other methods. Its average chrF++ score is over 90, implying that it restores almost perfect standard sentences from dialect sentences. Alphabet-level tokenization has a statistically significant advantage over other tokenization methods except for the Jeju dialect. A similar tendency is observed when BLEU is used as an evaluation metric. This is because the Korean dialects share their grammar and most words with standard Korean, as shown in Table 2.

One thing to note about these tables is that sub-words are ineffective for this task. They are heavily influenced by the initial weight of the model. The standard deviations of the sub-word tokenizers are higher than those of alphabet-level tokenization. SentencePiece and BPE are more complex and restrictive than the others. They model the natural language within a pre-defined vocabulary size and require a larger corpus to capture all the necessary patterns for the task. The vocabulary size in sub-word tokenization is set to 30,000, which is smaller than that in morpheme-level tokenization. The difference in vocabulary size is clearly evident in the performance gap between sub-word tokenization and morpheme-level tokenization.

Another thing to note is that the performance of the Jeju dialect is consistently lower than that of other dialects, whichever tokenization is used. That is, even though the Jeju dialect has the largest number of training instances, its performance is the worst. This is due to the geographical characteristics of the Jeju area. Jeju is an isolated island located far south of Seoul. Thus, the Jeju dialect differs significantly from standard Korean primarily at its surface form, resulting in poor normalization performance. This is also the reason why the sub-word tokenizers perform better with the Jeju dialect than with other dialects. No statistical significance was observed between sub-word tokenization and alphabet-level tokenization in the Jeju dialect.

The size of the proposed normalization model with alphabet-level tokenization is much smaller than the model with sub-word tokenizers. The proposed model with alphabet-level tokenization has 1.2M trainable parameters for 156 Korean alphabets including numbers and symbols. In constrast, the model with syllable-level SentencePiece has 21M parameters. That is, the alphabet-level tokenization achieves a higher performance with much fewer parameters.

The proposed model adopts minGRU as its encoder because the length of an input sequence becomes longer due to its alphabet-level tokenization. However, GRU can also be used as an encoder. According to

Table 6. Performance comparison according to encoder types: GRU vs. minGRU.

Dialect	BLEU		chrF++
	minGRU	GRU	minGRU	GRU
Gyeongsang	$97.9\pm 0.4$	$98.2\pm 0.1$	$96.4\pm 0.7$	$96.8\pm 0.2$
Jeolla	$96.5\pm 0.4$	$97.2\pm 0.1$	$94.0\pm 0.5$	$95.2\pm 0.1$
Jeju	$90.0\pm 0.4$	$90.7\pm 0.1$	$84.0\pm 0.6$	$85.1\pm 0.2$
Gangwon	$93.7\pm 0.8$	$94.7\pm 0.5$	$89.8\pm 1.2$	$91.4\pm 0.6$
Chungcheong	$96.3\pm 0.8$	$97.4\pm 0.1$	$93.6\pm 1.3$	$95.5\pm 0.3$
Overall	94.88	95.44	91.56	92.50

, the normalizing performance of using a bidirectional GRU is slightly better than that of using a bidirectional minGRU. However, this difference is not statistically significant with $p<0.05$. The true advantage of minGRU lies in its efficiency during its training and inference. That is, its execution time is faster than that of GRU. Figure 5 depicts how much faster minGRU is than GRU during the training. MinGRU takes less time to execute each epoch than GRU for all types of dialect. Overall, using a minGRU encoder saves about 15% of epoch time, even though the minGRU encoder consists of three minGRU layers and the GRU encoder has only one GRU layer. The three-layered minGRU and the one-layered GRU are compared because they demonstrate similar performance in character-level tokenization.

Figure 6 compares minGRU and GRU regarding the GPU processing time used for normalizing dialects to the standard. In this figure, X-axis is the number of tokens in a dialect sentence, and Y-axis represents the GPU time (msec) consumed to normalize a dialect sentence. This figure proves that minGRU always consumes less GPU time than GRU. The difference between minGRU and GRU is not significant when the sentence length is less than 400. However, the longer a dialect sentence is, the larger time gain minGRU has.

5.3. Evaluations on Pivot Translation

There is no parallel corpus for Korean dialects and foreign languages. Thus, a parallel corpus has been constructed from the normalization dataset. Note that the normalization dataset contains a standard Korean sentence for each dialect sentence. The standard sentences of the test set are first translated into foreign languages by an LLM, Llama-3.1-8B-Instruct, under the assumption that the translated sentences in this way are correct. Although the model has limited language modeling capability, it was chosen due to resource constraints. Then, three translation models—m2m_100_1.2B, Opus-MT and Exaone—are used to prepare pairs of a dialect sentence and its translated foreign equivalent as well as pairs of a normalized sentence and its translated foreign equivalent.

Table 7. Evaluations on Korean dialect translation to English. In this table, chr./B. means chrF++/BERT score.

Dialect	Direct Translation			Proposed Model
	Opus-MT	m2m_100_1.2B	Exaone	Opus-MT	m2m_100_1.2B	Exaone
	chr./B.	chr./B.	chr./B.	chr./B.	chr./B.	chr./B.
Gyeongsang	27.82/89.41	27.64/89.62	39.80/91.23	29.19/89.65	29.17/89.90	40.45/91.26
Jeolla	27.86/89.21	27.66/89.37	39.81/90.95	28.96/89.48	29.27/89.75	40.37/91.05
Jeju	23.30/88.15	22.97/88.18	33.86/89.85	26.06/88.90	25.94/89.20	36.75/90.51
Gangwon	25.95/88.92	25.61/89.15	37.18/90.56	27.83/89.45	27.43/89.62	38.36/90.77
Chungcheong	28.06/89.22	28.12/89.40	39.75/90.93	29.10/89.45	29.54/89.70	40.22/90.98
Overall	26.60/88.98	26.40/89.16	38.08/90.70	28.23/89.39	28.27/89.63	39.05/90.91

shows the evaluation results of the proposed pivot-based translation model for English. Here, ‘Direct Translation’ means that the dialects are translated directly into English without using a pivot standard. This table reports the ChrF++ and BERT score for pivot-based dialect translations. The proposed model achieves better performances than direct translation for all dialects, proving the effectiveness of using standard Korean as a pivot language. Although the improvement in performance is modest for both ChrF++ and BERT score, it is still significant. The ChrF++ score indicates the surface form distance, but this is not the only factor that determines the translation quality. The BERT score measures the semantic similarity between two sentences. However, according to Hanna and Bojar [38], the BERT score often assigns a high score even to incorrect translations. In the experiment of Hanna and Bojar [38], sentences containing several grammatical errors achieved a BERT score of around 82, while grammatically correct sentences achieved a score of around 83. The BERT score assigns a high score to incorrect translations, but it definitely penalizes defective translations. This shows that an improvement in the BERT score, even by less than 1.0 points, still implies a certain amount of improvement in the semantic level. In summary, the table shows that the proposed pivot-based translation is better in terms of both surface form and semantics.

Table 8. Evaluations on translation from Korean dialects to Chinese. In this table, BERT means BERT score.

Dialect	Direct Translation		Proposed Model
	m2m_100_1.2B	Exaone	m2m_100_1.2B	Exaone
	chrF/BERT	chrF/BERT	chrF/BERT	chrF/BERT
Gyeongsang	9.03/69.28	13.16/73.92	9.69/70.37	13.31/74.05
Jeolla	9.01/69.35	12.71/73.65	9.67/70.55	12.94/73.90
Jeju	7.62/67.19	10.82/71.02	9.02/69.92	12.09/73.10
Gangwon	8.18/68.47	12.01/72.86	9.12/70.16	12.50/73.58
Chungcheong	8.74/69.44	12.33/73.66	9.37/70.40	12.60/73.90
Overall	8.52/68.75	12.21/73.02	9.37/70.28	12.69/73.71

and

Table 9. Evaluations on translation from Korean dialects to Japanese. In this table, BERT means BERT score.

Dialect	Direct Translation		Proposed Model
	m2m_100_1.2B	Exaone	m2m_100_1.2B	Exaone
	chrF/BERT	chrF/BERT	chrF/BERT	chrF/BERT
Gyeongsang	12.94/74.18	19.69/78.59	13.83/75.01	20.67/79.07
Jeolla	13.03/74.18	19.44/78.48	13.78/75.00	19.92/78.81
Jeju	9.38/71.71	15.42/76.20	11.02/73.35	17.05/77.47
Gangwon	10.99/73.12	17.92/77.79	12.09/74.26	18.60/78.31
Chungcheong	12.71/73.90	19.08/78.21	13.51/74.60	19.70/78.51
Overall	11.81/73.42	18.31/77.85	12.84/74.44	19.19/78.39

demonstrate performance when translating dialects into Chinese and Japanese, respectively. Since neither language has word spacing, ChrF is used instead of ChrF++. Unlike Table 7, these tables do not include the Opus-MT’s performance, as there is no checkpoint for translating Korean to Chinese and Japanese. A similar phenomenon to that observed for English is seen for Chinese and Japanese, too. The proposed pivot-based translation consistently outperforms direct translation in these languages.

Exaone is a general-purpose LLM, while m2m_100_1.2B is a specialized neural machine translation model. It is important to note that both models outperform direct translation when the proposed pivot-based approach is used. This tendency is observed in all three foreign languages. This demonstrates the robustness and effectiveness of the proposed pivot-based translation approach with character-level tokenization for dialect normalization.

6. Conclusions

In this paper, we propose a pivot-based translation model for translating Korean dialects into foreign languages. To overcome a lack of a parallel corpus for direct translation from a dialect to a foreign language, our model first normalizes the dialect sentence to a standard sentence and then translates the standard sentence into a foreign language. The dialect normalization model is a standard GRU-based sequence-to-sequence model, using minGRU as an encoder and GRU as a decoder. As the model adopts alphabet-level tokenization, the input sentence tends to be a long sequence of tokens. To address this issue, a multi-layer minGRU is adopted as the encoder instead of a GRU. A legacy translator is then used for the translation between standard and foreign sentences. In this paper, two neural translators (Opus-MT and m2m_100_1.2B) and an LLM, Exaone, have been tested for this purpose.

Experiments on the Korean Dialect Speech Dataset demonstrate that alphabet-level tokenization achieves higher performance than sub-word tokenization and morpheme-level tokenization. Furthermore, minGRU is shown to be a more effective encoder than GRU for dialect normalization. In addition, it was also shown that the proposed pivot-based translation is superior to the direct translation when translating Korean dialects to English, Chinese, and Japanese.