How Machine Translation Works - Literal Translation, Free Translation, and Neural Translation?

2. The History of Machine Translation

The first machine translation appeared in the 1950s, initially using rule-based technology. This transitioned to statistical machine translation in the late 1980s, and further evolved with the introduction of neural machine translation in the 2010s.

In rule-based translation methods, humans need to create translation rules based on dictionaries and grammar, and this process is complex and difficult to update. Additionally, this method had low translation accuracy and could only translate formulaic sentences.

On the other hand, in statistical machine translation, computers learn rules. By reading a large number of pairs of original texts and their translations (e.g., 1 million sentences), they learn the correspondence between words and phrases in the original text and the translation from the data (corpus), making it relatively easy to handle new phrases. However, translation between languages with different grammar, such as English and Japanese, is difficult, and the translation accuracy was not practical.

In addition, there are hybrid translation methods that combine rule-based and statistical machine translation, as well as example-based translation techniques that extract similar parts from existing source and translated text pairs to utilize in translation. These methods have improved translation accuracy compared to traditional rule-based machine translation.

Neural translation, like statistical machine translation, learns by ingesting a large number of pairs of original texts and their translations. However, by using neural networks and deep learning, which are types of machine learning, it can utilize more information for translation. As a result, translation accuracy has significantly improved, allowing for more natural and fluent translations. The emergence of neural machine translation has brought attention to machine translation, which is now widely used in daily life and work.

3. What is the mechanism of machine translation?

We will explain the methods and technologies of machine translation in detail.

Rule-Based Machine Translation

Rule-Based Machine Translation (RBMT) requires expertise in the linguistics and grammar of both the source and target languages. The translation process mainly consists of the following three stages.

1. Morphological Analysis:
In this stage, the input sentence is divided into morphemes (the smallest units of meaning), and information such as part of speech and inflection forms is extracted. For example, the English sentence "I am eating a cake." is divided into the morphemes "I", "am", "eat+ing", "a", and "cake".

2. Syntactic and Semantic Analysis:
In syntactic analysis, morphemes are further analyzed according to language structure and transformed into a hierarchical structure called a syntax tree. A syntax tree is a tree structure that represents grammatical structure in natural language processing (NLP), showing how words and phrases combine to form the entire sentence. The syntax tree represents components of the sentence such as the subject, verb, and object as hierarchical nodes, explicitly indicating their relationships and functions, enabling more accurate translations. In semantic analysis, the meanings of words and sentences contained in the original text are extracted. The goal of semantic analysis is to understand the meaning of the source language text and accurately convey it in the target language. Even for polysemous words, appropriate translations are selected based on context.

3. Generation:
At this stage, sentences in the target language are generated based on the syntax tree and semantic information. Grammar rules and dictionaries are applied, and appropriate word order and morphemes are combined to construct sentences in the target language.

The advantages of rule-based machine translation are that the translation process is clear, making it easier to identify the causes when errors or issues occur. Additionally, it generally provides grammatically accurate translations by utilizing high-quality dictionaries and grammar rules created by experts.

The drawbacks are that a large amount of manpower is required to accommodate new language pairs and domains, and that it often struggles to handle diverse expressions and slang effectively. Furthermore, due to grammatical and lexical differences between languages, the translation results can often lack naturalness and fluency. For these reasons, data-driven approaches such as Neural Machine Translation (NMT) have become mainstream in modern translation systems.

Statistical Machine Translation

Statistical Machine Translation (SMT) is a method of machine translation that translates based on statistical patterns extracted by computers from large amounts of bilingual text (parallel corpora consisting of original texts and their translations). Unlike rule-based machine translation, which relies on linguistic knowledge and grammatical rules, statistical machine translation uses machine learning algorithms and probabilistic models to learn the transformations between language pairs.

The main approaches to statistical machine translation include the following.

1. Word-Based Translation:
The word-based approach uses the probability of one word being translated into another. This allows for the selection of the most likely combinations of words. However, this approach struggles to address issues of word order and structure.

2. Phrase-Based Translation:
The phrase-based approach deals with longer units (phrases consisting of multiple words) rather than individual words. This allows for a better capture of grammatical relationships and structures. In phrase-based translation, pairs of corresponding phrases are extracted from parallel corpora, and it determines how these pairs combine in the translated text.

3. Syntax-Based Translation:
The syntax-based approach captures the structure of sentences by utilizing the syntax trees of the source and target languages. This allows for the high-precision representation of grammatical relationships and semantic information. In syntax-based translation, the syntax tree of the source language is converted into the syntax tree of the target language based on syntactic rules extracted from parallel corpora.

The advantage of statistical machine translation is that it can automatically acquire translation knowledge from large amounts of data. Therefore, it is easier to accommodate diverse languages and domains, and it is effective for new expressions and slang. However, its performance may decline in cases where there is insufficient parallel corpus or when dealing with language pairs that have significantly different grammatical structures.

In recent years, neural machine translation approaches have become mainstream, and statistical machine translation is gradually giving way to them.

Neural Translation

Neural Machine Translation (NMT) is a state-of-the-art machine translation method that uses deep learning technology to translate from one language to another. This type of machine translation is also referred to as AI translation. Neural Machine Translation leverages neural network architectures such as Recurrent Neural Networks (RNN), Convolutional Neural Networks (CNN), and Transformers to automatically learn the knowledge necessary for translation from parallel corpora. Unlike statistical machine translation and rule-based machine translation, Neural Machine Translation can comprehensively capture the meaning of sentences and grammatical relationships through vector representations.

A typical neural machine translation system consists of the following two main components.

1. Encoder:
The encoder converts the input sentences in the source language into continuous vector representations. Each word or character is transformed into a pre-trained word embedding vector, and the structure and semantic information of the sentence are appropriately encoded. Recurrent Neural Networks (RNNs) and Convolutional Neural Networks (CNNs) are commonly used.

2. Decoder:
The decoder generates the translated text in the target language using the continuous vector representation obtained from the encoder. This is typically done word by word, where the most probable word is selected at each step, and the translated text is constructed. Similar to the encoder, RNNs and CNNs are also used in the decoder.

In recent years, the transformer architecture has been very successful, surpassing traditional approaches in terms of translation quality and speed. The transformer efficiently captures the relationships between words in the source and target sentences through the attention mechanism.

The advantages of neural machine translation include high accuracy and naturalness of translations, as well as the ability to automatically acquire grammar and expressions from training data. However, the need for large amounts of training data and computational resources, along with the complexity of the training process, pose challenges. Additionally, because the internal state of the model is often a black box, it can be difficult to identify and correct the causes of errors.

Even when using the same neural machine translation methods, the quality of translations varies depending on the machine translation service. Notable machine translation services include Google Translate and DeepL, with Google Translate generally being more literal (word-for-word) and DeepL being more interpretive. Google tends to translate literally, which can sometimes result in unnatural translations. On the other hand, DeepL often produces more natural translations due to its interpretive approach. However, a common issue with DeepL is the omission of original words or sentences from the translation, known as translation omissions. It is important to note that the smooth and natural expression of the translation can sometimes hinder the detection of these omissions when reading only the translated text. For more information on the translation accuracy of DeepL and Google Translate, please see the article below.

Latest Trends in Machine Translation and Comparison of DeepL and Google Translate

What is Deep Learning?

Deep learning is a type of artificial intelligence (AI) technology that specifically refers to machine learning algorithms using neural networks. Deep learning utilizes "deep neural networks" that have connections and structures mimicking the workings of the human brain to learn from large amounts of data, enabling advanced recognition and judgment. Its applications are not limited to machine translation.

Deep learning has the following advantages.

1. The unnaturalness of translations is being resolved:
Deep learning-based translation systems can provide more natural and accurate translations compared to traditional translation methods. This is because neural networks have the ability to understand the structure and context of sentences and translate them appropriately.

2. Familiarity with Technical Terms:
Systems using deep learning can correctly understand and translate words related to specific fields, such as technical terms and proper nouns. This is because translation services from companies like DeepL and Google primarily learn from vast amounts of data available on the internet, which includes various technical terms and proper nouns from different fields.