[Verification] What is the actual quality of machine translation?

"How good is the translation quality with machine translation?"
This is one of the frequently asked questions from customers considering machine translation.
No matter how much cost can be reduced with machine translation,
it is meaningless if the quality is not usable.

So, this time we will introduce the quality evaluation results of machine translation in a sample project actually conducted by our company.

Overview of Sample Project

• Period: October 2013
• Translation Target: End User Manual for Office Equipment
• Translation languages: English to French, Italian, German, Spanish
• Engine used: Moses (commercial statistical-based engine from overseas)
• Corpus loaded into a statistical-based engine: approximately 500,000 words

※Use past translation memory from the same series

Translation Evaluation Criteria

First, we trained the statistical-based engine with approximately 500,000 words of past translation memory for the target product group.
Training is the process of loading past translation data into the engine and teaching it grammar rules and terminology.
Then, we randomly selected 300 sentences from the machine-translated results using the trained engine and had them evaluated by two native translators for each language.

Here is the graph summarizing the results.

The blue portion of the graph represents the percentage of "good quality (Excellent/Good)" sentences,
while the gray portion represents the percentage of "poor quality (Medium/Poor)" sentences.

How is it? You can see that the blue part occupies a large percentage in all languages,
and I think you can understand that it is of very good quality.

The text in blue can be used as is, or with minimal modifications, as the result of machine translation. This means that it can be translated more efficiently than having a translator translate everything from scratch.

By choosing the appropriate engine for the project and building a workflow, it is possible to achieve high quality even with machine translation.

Please feel free to contact us.

 
 

 

Blog Writing Team

Tokuda Ai

・As a machine translation consultant, I provide consulting services for Japanese companies on machine translation implementation and process building.
・I place importance on the quality of the source text, which affects multilingual translation, and also provide consulting services for manual creation suitable for machine translation in the Japanese writing process.
・I also give presentations on the following topics related to machine translation.
- Presentation at the 23rd JTF (Japan Translation Federation) Translation Festival in 2013
"Approaches to Machine Translation in Multilingual Environments
- From the Perspectives of Evaluation and Process"
- Presentation at the 2014 AAMT (Asia-Pacific Association for Machine Translation) Machine Translation Fair
"Mastering Machine Translation - Improving Quality and Productivity"