Translation Alignment with Ugarit

Abstract

Ugarit is a public web-based tool for manual annotation of parallel texts for generating word-level translation alignment. We aimed to develop a user-friendly interactive interface to visualize aligned texts and collect training data in the form of translation pairs to be used later, (i) for training an automatic translation alignment system for historical languages at the word/phrase level, (ii) as a gold standard to evaluate automatic alignment and machine translation systems. Ugarit is now widely used for learning new languages, especially historical languages, and as a reading environment for parallel texts. In the following sections, we present the related works and similar projects; then, we give an overview of the visualization techniques used to present the alignment results. Further, we explain how we could derive the translation graph from the aligned translation pairs. Finally, we discuss the usage limitations of Ugarit, possible improvements, and future development plans.

1. Introduction

Translation alignment is a major task in Digital Humanities and Natural Language Processing. It is the process of comparing two texts in different languages to find translation correspondences among the textual units in the source and translation texts [1]. It can be performed at various granularity levels according to the project’s context or the research purpose.

Translation alignment is essential in neural and statistical machine translation [2], cross-lingual annotation projection [3,4], and translation lexica induction [5,6,7]. Several automatic approaches have been developed to perform the alignment at different levels [1,8,9,10]. However, most proposed models employ unsupervised statistical methods to generate alignment probabilities distribution between the source and target textual units. In general, The accuracy of the automatic alignment varies according to multiple factors, such as text type and length, size of the corpus, and translation quality and consistency. Recently, with the advances in transformers and contextualized language models [11,12], researchers have developed many approaches that can exploit contextualized multilingual word embeddings to generate word alignments from parallel texts [13,14].

In turn, manual alignment is still essential despite the advances in automatic translation alignments models, especially for creating alignment gold standards [15,16,17,18,19] and automatic alignment post-editing. However, manual alignment is expensive in terms of time and resources and requires annotation tools tailored for this purpose. Most word alignment annotation tools are either crafted for a specific project [20,21], specific language pairs, or specific texts [20,22].

The related works can be grouped into two main groups; the first one includes tools that offer an annotation interface to generate translation equivalents to be used in further development without any interest in visualizing the alignment. The second group includes tools that provide both the annotation and the visualization interfaces.

From the first group, we can mention the Blinker Project [20], which developed the first annotation tool for manual text alignment to align different versions of the Bible in French and English at the word level. The Linguistic Data Consortium (LDC) has also developed the LDC Word Aligner to perform the manual word alignment for Arabic–English and Chinese–English parallel texts from newswire and broadcast mainly [21]. Furthermore, TagAlign [23] allows users to annotate bilingual texts with a pre-defined tagset and create manual alignments at the sentence level. The tool was initially developed to align Brazilian, Portuguese, and English parallel texts. D. Benner [22] developed a manual word alignment tool to create gold standard for Hebrew–English by aligning the Hebrew bible with its English translation.

On the other hand, Yawat [24], Alpheios [25], SWIFT Aligner [26], and CLUE-Aligner [27], that fall into the second group, enable users to create their alignments manually at the word- and phrase-level and offer various possibilities for visualizing the aligned texts, such as side-by-side view, the interlinear text view [28], and alignment matrices [24,27]. Ugarit falls in the second group, it provides a manual annotation service and a reading environment for parallel texts.

The tools mentioned above use different approaches to link words in the source text with their correspondents in the translation. We can distinguish three main methods:

(i)

The two texts are placed on two parallel columns. Annotators can draw lines between corresponding words of the source and translation texts [22,26];

(ii)

The two texts are represented as a two-dimensional alignment matrix. Annotators can select a cell in the grid with a mouse click to create an alignment between the corresponding row- and column-words [27,29];

(iii)

The texts are placed side-by-side. Annotators can select the source words and their translation correspondents with mouse clicks.

The first two approaches are suitable for short text units (sentences, short paragraphs), where annotators can still capture the context while reading vertical texts, whereas the third approach is suitable for both short and long texts. Most tools mentioned above are limited to one alignment class between tokens in the original and its translation, CLUE-Aligner [27] distinguishes between two main classes, possible and sure alignments and Yawat [24] supports manual labelling of the alignment relations. SWIFT Aligner [26] provides support for Part-Of-Speech and syntactic dependency manual annotation.

Visualization of aligned texts was the subject of interest and research in recent years; many tools have been developed for this purpose. Various approaches have been utilized to visualize the text alignment at different levels [30,31], for instance, side-by-side views, parallel views, and text heat maps.

Ugarit is a crowd-sourcing project that enables users to create translation alignments at the word or phrase level; the resulting translation pairs can be used as gold standards to evaluate machine translation systems or create dynamic lexica and translation memories. Ugarit was initially designed to visualize the automatically aligned texts available at Perseus Digital Library [32] and collect training data in the form of translation pairs to implement a statistical translation alignment system for historical languages, mainly Ancient Greek, Latin, and Persian, for which few to none aligned datasets exist. Ideally, historical languages are closed systems with a finite number of words and minimal change in the foreseeable future. Therefore, it should be possible to create adequately efficient automated alignment methods based on a relatively small training dataset. Ugarit is not only an annotation tool, but it also offers a visually powerful reading environment, where the reader can analytically compare texts token by token and at the same time observe the results through interactive visualization and statistics. Unlike other manual alignment tools, Ugarit collects the translation equivalents created by annotators to construct the translations graph, which can be used for dynamic dictionaries induction, even for languages that do not have direct parallel texts, by applying triangulation and using other languages as a bridge. The translations graph contains over 500 k translation pairs in 45 languages. Further, Ugarit allows users to inspect how other users aligned a specific token using the translation pairs search function, which provides detailed results about the different alignment possibilities with a comprehensive visualization. Trilingual alignment, a unique feature of Ugarit, can be used in a variety of ways, such as comparing and visualizing competing translations, evaluating indirect translation by adding the mediating text or even supplementing another translation to a less common language pair. Additionally, Ugarit has been used as a pedagogical tool for vocabulary learning, morpho-syntactic comparisons and learning assessment both in the classroom and in self-study [33]. Finally, Ugarit is designed as a public web tool that does not require any installation, technical expertise, or hardware prerequisites.

In the following sections, we describe the development process and show how manual alignment can be performed in Ugarit. Next, we describe the different visualization approaches used to visualize the translation alignments, the dynamic lexicon search results, and the translation clouds. Finally, we discuss the limitations, possible improvements, and new features we intend to integrate into the next release of Ugarit.

2. Development Process

The development of annotation tools is a challenging task. It requires a deep understanding of the underlying task and the user’s needs and an experience in human–computer interaction methods and approaches. Before working on Ugarit, we studied the related tools and defined their limitations, we also consulted numerous research papers and surveys [34,35,36,37,38] that reviewed and analyzed the existing annotation tools and defined design principles and usability recommendations, which helped us to build a primary vision of the tool. The development of Ugarit has been achieved through the close collaboration of researchers from Computer Science, Digital Humanities, Classical Philology, and Translation Studies, aiming to gain a better understanding of users’ needs. The development started in 2017 at the Alexander von Humboldt-Chair for Digital Humanities at Leipzig University. We followed the user-centred design principles during the iterative development process (Figure 1) and the usability recommendations for annotation tools [37], leading to a user-friendly, intuitive, easy-to-use tool.

The starting point was to define the usage context and analyze the user tasks and requirements; this has been done through interviews and discussions with domain experts who explained their needs and defined the different usage scenarios. Domain experts defined a list of main features that must be provided in the tool: (1) Users can annotate texts in any language, alphabet, or writing system; (2) Users can annotate long text passages; (3) Users can create different alignment types, one-to-one, one-to-many, many-to-one, and many-to-many; (4) Time and clicks required to create the alignment should be minimized to the extent possible; (5) Users can analyze and review their alignments via interactive visualization; (6) Users can download and share their alignments in different formats.

Next, the development team prepared a set of mockups and prototypes that were discussed and approved with the domain experts. Since most of our audience has no technical expertise, we decided to create a public web-based platform-independent tool that is easily accessible and does not require any prior technical knowledge for installation and use.

Later, the first implementation cycle started and resulted in the first product, which has been tested and evaluated by domain experts. The improvement suggestions, problems, and difficulties faced by the domain experts were reported to the development team, who worked to solve these issues and enhance the tool’s usability. After multiple development iterations, we could reach a stable and reliable version of that tool.

Ugarit is implemented in Php on the server-side and Javascript on the client-side; MySql and Neo4J are used to store the data. Ugarit supports UTF-8 text format, which allows users to align texts in many languages and different alphabets, such as Aramic, Ancient Greek, and Coptic. It also shows Right-to-Left texts in the correct direction and provides automatic transliteration for non-Latin alphabets languages. The tool is not designed to be used on mobile devices; however, users can still create alignment for short passages with their mobile devices. For best use, it is recommended to use a computer.

Figure 2 summarizes the features and functionalities provided by Ugarit, users can align bi-and trilingual texts; combine their aligned passages in one big text or split a long aligned text into smaller units. Moreover, users can download their alignments, share them, and integrate them into their websites and blogs. Further, Ugarit provides a reading environment for aligned parallel texts, where users can read, search, and compare the aligned passages.

3. Alignment Workflow

Annotation is the process of adding information to a text at some level [39], and we call it Manual Annotation when humans perform it. Manual Annotation is a nontrivial task, and it requires intense focus and attention. Like any chore, it becomes boring after a short while. Manual translation alignment, in particular, is a daunting task; it requires moving eyes between two texts in different languages and finding translation correspondences and linking them.

When we designed the Ugarit Interface, we tried to make it easy to use, even for users with no experience with annotation tools. The first step is to create an account on Ugarit or log in if the user has already created an account and verified it. Users have to choose between bi- or trilingual text alignments to start a new alignment. Then, users can upload their parallel texts in plain text format or use the canonical text service (CTS) URNs to import texts from the Perseus Digital Library CTS (cts.perseids.org, accessed on 20 October 2021) repository [40], and the languages of the texts must be selected. Next, texts will be tokenized and prepared for alignment.

The alignment process is designed to be as simple as possible; we tried to minimize the mouse clicks needed to create and save the aligned units. In the case of bilingual text alignment, the panel is split into three main columns, two columns for the parallel texts and the right-side column for displaying the aligned translation pairs. A progress bar is located on the top of each language panel to show the alignment coverage so far. Ugarit also offers trilingual alignment, where users can align three parallel texts in three different languages (Figure 3). To align a translation pair, users must select a word/phrase from the original language and then select the corresponding word/phrase in the translated text. To select a token, the user needs to click the token, and then it will be highlighted with green. Clicking a selected token will deselect it and remove the highlighting. The paired tokens will be automatically saved when the user starts to align a new pair or clicks on the save icon. On the right side of the editor, users can see all aligned pairs and have the option to edit or delete any pair. The Ugarit editor allows users to create all types of alignment word-to-word (1-1), word-to-phrase (1-N), phrase-to-word (N-1), and phrase-to-phrase (N-N) alignments. The translation pairs list can be exported in XML or tabular format. The resulting translation pairs are automatically stored in the database and then can be exported in XML or tabular format. Furthermore, users can also decide whether the alignment can be publicly visible on the website or keep it private. Finally, users have to provide some information about the texts such as title, translator, and a short description. Once the user saves the alignment, it will appear on the home page in the New Alignments panel.

Figure 4 shows the different alignment types of the translation alignments of an ancient Greek text with its translations in Italian, German, English, and French (The alignments of Homer, Iliad, 21.1-53 Ancient Greek source text and its different translations, alignments are created by Chiara Palladino). The distribution of the alignment types differs according to various factors, including text languages, text type, text genre, translation type, annotator guidelines.

Alignment Guidelines

Annotation guidelines are a key component of any collaborative annotation work; they ensure annotation consistency and reduce annotation errors. The same applies to translation alignment. However, defining alignment guidelines is not a trivial task; it requires a deep understanding of the linguistic structure of both languages and the relations between them. In general, Ugarit does not impose any alignment guidelines, it gives users the freedom to tailor their own guidelines according to their projects or alignment purpose. However, we share some recommendations and refer to common guidelines used for the creation of some alignment gold standards [17,41,42].

We conducted experiments on guidelines for Ancient Greek–English and Persian–English text to measure the impact of the alignment guidelines on the alignment process. The results showed that defining and following alignment guidelines would increase the inter-annotators agreement by at least 10%. However, it is not possible to create guidelines for all language pairs in Ugarit, since an alignment guideline needs to consider multiple aspects of both the target and the source language at once, and should be tailored individually for each language pair by experts.

4. Visualization Techniques

Visualization of translation alignment plays an important role in understanding and interpreting the relation between texts and their translations [30]. Ugarit offers various approaches to visualize aligned texts and the derived Dynamic Lexicon.

4.1. Languages Graph

The graph is placed on the tool’s home page (http://ugarit.ialigner.com, accessed on 20 October 2021) to give users a quick overview of the languages currently hosted and how they are related to each other, as shown in Figure 5.

Each vertex represents a language. The vertex’s size reflects the number of texts in this language in the database. Each language is assigned a different color, and vertices are labeled with the language names in the original form. The connection between two vertices means that there are translation alignments between texts in these two languages, and the thickness of the line reflects the number of aligned translation pairs. In Figure 5, clicking a vertex will show all texts in the language it represents, whereas clicking a link between two vertices will load all aligned texts in these two languages.

4.2. Aligned Texts

The side-by-side view is the most intuitive and straightforward approach to display parallel text alignment at different levels. It is also widely used to visualize collation and mono-language alignments.

Texts are placed alongside each other, as shown in Figure 6. A coloring schema is used to distinguish between aligned and unaligned tokens. Since most tokens are aligned, we used black for aligned tokens and red for the unaligned to draw user attention. The alignment between corresponding tokens on the parallel sides is visualized via highlighting. When the user hovers an aligned word, the hovered word/phrase and the paired tokens will be simultaneously highlighted with red color.

A progress bar is located under each text to give the user an overview of how many tokens are aligned and their percentage; the aligned part is colored with green, whereas the unaligned part is colored with red, as we can see in Figure 6.

-

Embed: This option allows users to generate a link that can be used to share and embed the visualization of their alignments on any blog or website. Furthermore, and users can select which components can be included in the embedded text (progress bar, title, annotator info, text info) (Figure 6(1)).

-

Translation Pairs: This option shows an aggregated list of the translation pairs extracted from the aligned text, users can download them in JSON format (Figure 6(2)).

-

Alignment Statistics: This option shows statistics of the different alignment categories. This chart provides valuable information on the alignment quality, user’s language knowledge, and the relation between the aligned languages (Figure 6(3)).

-

Alignment Matrix: with this option, users can view the alignment in the form of a grid: the source text tokens are located on the horizontal axis, and the translation tokens are located on the vertical axis. The blue dots represent alignment between the corresponding column and row tokens. The diagonal dots indicate one-to-one alignments, whereas the vertical ones indicate the one-to-many (Figure 6(4)).

-

Transliteration: Ugarit contains texts in various languages with different alphabets. For better readability, especially for new language learners, Ugarit offers an automatic transliteration for non-Latin alphabets languages, which is visible when the pointer hovers the aligned word. This feature is currently available for Greek, Arabic, Persian, Armenian, and Georgian (Figure 6(5)).

-

Combine/Split Aligned Texts: Ugarit enables users to merge multiple aligned texts into one bigger text. It also lets users split a long aligned text into smaller units (sentences/paragraphs). This feature is beneficial since annotators prefer to align long paragraphs over short ones to avoid copying and pasting them multiple times. On the other hand, splitting long aligned text is useful when users want to create shorter aligned units that can be used later to train or evaluate machine translation or translation alignments models.

4.3. Translation Pairs & the Dynamic Lexicon

Ugarit provides a search function to enable users to look for a word or phrase and get detailed information about how this word/phrase occurs in the Ugarit texts collection and how it is translated in different languages. Moreover, Ugarit visualizes the dynamic lexicon search results in two approaches:

Tree View is a classic branching view that enables users to navigate the hierarchy to filter the results set until they reach the desired subset. Figure 7a shows how Ugarit visualizes the search results as three levels tree view. The query word is located at the first level as the root word, corresponding translations are grouped by languages, and languages are placed as nodes at the second level. The aligned translations are placed at the third level. The language nodes are initially collapsed for clarity, and users can expand them to explore the translations by clicking the language label.

Radial Cluster Dendrogram is similar to the tree view mentioned above; it shows the translation equivalents in the form of a rooted tree and all leaf nodes are placed at the same depth (Figure 7).

These two views illustrate the idea of the Dynamic Lexicon [7], which is an automatically derived lexicon from parallel aligned texts at the word/phrase level. Dynamic Lexicon uses triangulation to create new dynamic lexica using pivot languages based on the assumption that two words/phrases are likely to be translations if they are translations of the same word in a third language [43]. In the two examples shown in Figure 7, triangulation considers all leaf nodes synonyms (if they are in the same language) or translation equivalents since they share the exact English translation.

5. Translations Graph

Ugarit is characterized by its simplicity and ease of use, and for this reason, it has been used by many users and in several projects, which led to continuous growth in the number of aligned texts and translation pairs. Though the traditional relational database architecture implemented in Ugarit is no longer satisfactory, we realized that we could achieve more benefits if we modelled our data more sophisticatedly.

After experimenting with different models, we came to the conclusion that the graph data structure fits our data best. The proposed model can be described as follows:

Let $G=(V,E)$ be a translations-graph. where V is a set of all words/phrases in the the Ugarit aligned texts, and E is a set of all translation pairs. Let $v_1\in V$ be a word/phrase in language $l_1$, and $v_2$ be a word/phrase in language $l_2$. {$v_1$, $v_2$} be a translation pair that occurs n times in the dataset.

Each node $v\in V$ has four attributes, an internal id $id$, label t, language l, and n the frequency of this word/phrase in the whole dataset $v\{id,t,l,n\}$.

The translation relation between $v_1\{i{d}_1,t_1,l_1,n_1\}$ and $v_2\{i{d}_2,t_2,l_2,n_2\}$ can be represented as an edge $e\in E$, e encapsulates a single value n which indicates the frequency of this translation pair in the dataset. We have used the Neo4j database [44] to model the translations graph.

Translation Clouds

Visualizing the translation graph as a connected graph was not possible due to the large number of nodes and crossing edges, which would affect the translations’ clarity and readability. The solution was to keep the labels and remove the lines that connect them, resulting in a word cloud representation.

Translation cloud shows the translation equivalents to a given word, these equivalents are extracted from the translation graph up to the fifth level. Figure 8 shows the translations cloud of the root word city.

The view consists of three components, the translation cloud, the languages bar above it, and the concrete translation path on the right side. The search word is located at the cloud center, surrounded by its equivalents in other languages. Further, translation equivalents are grouped by language, and every language is assigned a unique color. However, languages are listed in the languages bar; clicking on a language label will hide all translation equivalents in other languages and keep the ones in the selected language. Labels are displayed in different font sizes according to the distance to the root word. Since the translations cloud visualizes connected translation equivalents up to the fifth level, the distance varies between 1 to 5; the lower the distance value, the higher the font size. Moreover, clicking a translation label will show the path between the selected word and the root. Additionally, a tooltip is used to show information about the label, such as language and frequency.

Using the Neo4j graph database at the back-end to store the translation pairs has shown many advantages; it reduced the response time of search queries, enabled us to perform complex queries, and facilitated the dynamic lexicon production.

6. Ugarit in Research and Pedagogy

The complexities involved in the operation of translation alignment are often a reflection of the dynamic relationship between original and translation, and between different languages. Ugarit provides an environment where the user can engage with those complexities in a very analytical way, on both small and big scale.

On a small scale, Ugarit offers an immersive and visually powerful environment, where the reader can analytically compare texts token by token, and at the same time observe the results through an interactive visualization. The comparison of parallel texts becomes a systematic operation, which encourages reflection regarding the interplay between two languages, the meaning of specific words, and overall the (im)perfect matching of words and expressions. This is also an exercise in cultural dialogue and reflection, not only upon the language(s) but upon the civilization that used it to reflect its values. Moreover, the opportunity to publish the results online provides a way to be part of a broader conversation on the reception and significance of a text over time. For these reasons, Ugarit is currently used in studies on translation and reception and language teaching, particularly for Ancient Greek and Latin, across the world. In pedagogy, translation alignment is often integrated with grammatical and syntactical observations to emphasize the complex interplay between the language of the translation and the target language, and students are assigned various alignment tasks and exercises to empower the analytical approach to the text [33,45]. On a bigger scale, Ugarit also provides manually aligned parallel corpora across languages that had never been compared before, such as Coptic, Ancient Greek, Arabic, Persian, Latin, Egyptian, Georgian, etc. The analysis of these aligned datasets and recurrent patterns in word matching can provide insights into how cross-linguistic and cross-cultural dynamics are affected by different language structures, cultural differences, text genres, and even language proficiency [46].

Since the tool was made public, the number and variety of languages included by the users has steadily increased and has gone far beyond the original intent: at the moment this paper is being written, 36 languages are included in Ugarit, and there are 295 active users, and about 23,500 parallel texts.

Ugarit has been used as a pedagogical tool for vocabulary learning, morpho-syntactic comparisons and learning assessment both in the classroom and in self-study. One case in point is an online course for teaching Ancient Greek in Persian speakers [47], where Ugarit was used extensively. After a five-minute tutorial, all participants of the course were able to register and use Ugarit for aligning translation successfully. During this course, Ugarit was used as a reading environment for parallel texts by the educator during the class, as a vocabulary learning tool by the students, and as a assessment/practice environment where the educator could assess and examine the students’ understanding of the text. Furthermore, the trilingual alignment provided a suitable environment for using a third language as a bridge between the source language and the target language, which in case of the Ancient Greek course would be English.

In addition to the pedagogical aspect, the trilingual alignment is a practical tool for research purposes, providing valuable information on complex texts, particularly through terminology extraction. In Figure 9, an obscure term from the inscription of Shapur I at Ka’ba-i Zardušt (ŠKZ) is shown along with its English and Greek equivalents throughout the text, presenting both morphological and semantic variations in the Greek version of the inscription.

7. Future Work

When we created Ugarit, we aimed to visualize the automatically aligned texts and collect training data for automatic alignment systems. Still, Ugarit has also been used in many research projects [48,49] and for different purposes, especially teaching and learning historical languages [50]. Experts, who have been using Ugarit regularly, have provided us with some ideas and improvement suggestions to enhance the usability of Ugarit. Therefore, new features and functionality should be implemented and developed to keep pace with users’ needs. We can sum them up as follows:

-

User roles: the next version of Ugarit will offer different user roles such as expert, instructor, student, which would help create accurate training data by considering the alignments created by experts and instructors, since they are supposed to produce correct and precise alignments. In contrast, students in the learning phase could make some alignment mistakes, and these mistakes should not affect the accuracy of the dynamic lexicon and training datasets;

-

Teaching: Further, experts and instructors will be able to create groups, add students to the groups, and create assignments. Instructors can upload these assignments in the form of plain parallel texts; students will be asked to align them with deadlines, with the possibility of uploading the correct alignment to allow the system to evaluate the assignments automatically and give notes to every student;

-

Alignments sharing and exporting: in the current version of Ugarit, users can export their results in XML format only. The next version will offer other formats such as JSON and CSV to facilitate the reuse of the alignment in other applications or for other purposes;

-

Automatic alignment: we are currently developing an automatic alignment system and planning to offer an automatic alignment option for texts in specific languages, such as Ancient Greek–English and Latin–English, or at least supporting the users with alignment suggestions to reduce the time required to align long texts;

-

Collaborative alignment: in the current version, users can only align their texts; however, the next version will provide an option for the collaborative alignment of long texts where multiple users can work on the same text.