[Verification] What is the actual quality of machine translation?

“What level of quality can machine translation achieve?”
This is one of the frequently asked questions
we receive from customers considering machine translation.
No matter how much cost can be reduced with machine translation,
if the quality is unusable, it’s meaningless, isn’t it?

Therefore, this time, we will introduce the quality evaluation results of machine translation from a sample project that our company actually conducted.

Overview of the Sample Project

• Period: October 2013
• Translation Target: User manuals for office equipment aimed at end users
• Translation languages: From English to French, Italian, German, and Spanish
• Engine used: Moses (commercial statistical-based engine from overseas)
• Corpus loaded into the statistics-based engine: Approximately 500,000 words

* Used past translation memory from the same series

Translation Evaluation Criteria

First, we trained the statistics-based engine with approximately 500,000 words of past translation memory for the target translations and the same product group.
Training involves loading past translation data into the engine to teach it grammar rules and terminology.
Then, using the trained engine, we randomly extracted 300 sentences from the results of the machine translation and had them evaluated by two native translators for each language.

Here is the graph summarizing the results.

The blue part of the graph represents the proportion of texts classified as "Excellent/Good",
while the gray part represents the proportion of texts classified as "Medium/Poor".

How is it? The blue portion occupies a large percentage in all languages,
and I believe you can see that the quality is quite good.

The text in the blue section is of a quality that can be used as is from machine translation results,
or with minimal corrections,
which means that translation can be done more efficiently than having a translator translate everything from scratch.

By selecting an engine suitable for the project
and building an appropriate workflow,
it is possible to achieve high quality even with machine translation.

Please feel free to contact us.