What Is ISO 5060? An Approach to Objectively Evaluate Translation Quality in Manufacturing

1. What Is ISO 5060

ISO 5060 is a general guideline for classifying issues contained in translation results by error type and evaluating them using severity and penalty points.

The scope includes human translation, post-edited machine translation, and raw machine translation.

The key point is to consider not only the number of errors but also the impact those errors have on the reader and the product.

2. Differences from ISO 17100, ISO 18587, and MQM

Standards and evaluation methods related to translation quality differ depending on their target and purpose. To understand the positioning of ISO 5060, it is important to distinguish between the translation service process, post-editing of machine translation, and the framework for quality evaluation.

While ISO 17100 and ISO 18587 mainly address the processes and systems that provide translation, ISO 5060 focuses on the evaluation of completed translations. MQM can be utilized as a flexible classification system and also serves as a reference when designing in-house evaluation criteria aligned with ISO 5060.

>Related service: Post-editing (MTPE)

3. Why the Number of Errors Alone Is Insufficient

For example, suppose the following two issues were found in an instruction manual.

Both have one reported case each, but the impact on product users is not the same. By adopting the ISO 5060 approach, you can separate and record the types and severity of errors, clarifying which issues should be prioritized for response.

4. Expected Benefits for Translation Departments in Manufacturing

• Even if the translation company or language differs, quality can be checked using a common approach
• Priority can be given to issues with significant impact, such as warning messages, operating procedures, numbers, and units
• It becomes easier to reduce variability in judgments among reviewers
• Human translation and machine translation can be compared under conditions suited to their intended use
• It becomes easier to explain the points raised and the risks to the quality assurance department and related departments

However, producing a score itself must not become the goal. The evaluation results need to be used as information to understand the causes of quality issues and to lead to the next improvements.

>Related Service: Industrial and Manufacturing Translation – Translated by Engineers with Extensive On-site Knowledge

5. Translation Quality Issues Are Not Always Caused Solely by Translation

If the same error is repeated, the cause may not only be how the translation is created but also the information and processes surrounding the translation.

• The original text's expressions are ambiguous and can be interpreted in multiple ways
• Different terminology is used by the design department and for each product
• The glossary and style guide have not been updated
• Old translations remain in the translation memory
• The target audience and document purpose have not been communicated to the translation company
• Review results have not been reflected in the glossary or translation memory

To continuously improve translation quality, it is important not only to score the translated text but also to isolate the causes by including the source text, glossary, style guide, translation memory, order conditions, and review process.

6. How Far Can AI-Based Evaluation Be Used?

By using generative AI and automated checks, candidates such as discrepancies in meaning between the source and translated texts, omissions, terminology violations, inconsistencies in numbers and units, and variations in notation can be efficiently extracted.

On the other hand, aspects such as the impact on safety, the appropriateness of technical terms, and clarity for the target audience require judgment based on the document’s purpose and product knowledge.

AI narrows down the candidates, and translation quality experts verify the content and severity. This division of roles makes it easier to balance evaluation efficiency and reliability.

>Related Blog: Evaluating the Performance of Generative AI Translation in 6 Languages [ChatGPT vs Human Translation Comparison Report]

7. Three Key Points When Adopting ISO 5060

1. Decide the Purpose of Evaluation

The evaluation targets and passing criteria vary depending on the purpose, such as vendor comparison, adoption decisions for machine translation, and verification of safety information.

2. Design criteria that fit your company's documents

The impact of the same error differs between general explanations and warning statements. Define severity according to the product, audience, and document purpose.

3. Decide on improvement targets after evaluation

Do not end with just pointing out issues to the translation company; reflect improvements in the source text, glossary, translation memory, and review methods.