Utilizing LLM Proofreading Tools in the Japanese-English Translation Review Process — Efforts for QA and Quality Stabilization Using In-House Developed Tools —

Table of Contents

• 1. What is an LLM Proofreading Tool｜Its Role in the Japanese-English Translation Review Process
• 2. Differences Between Conventional Translation QA Tools and LLM Proofreading Tools
• 3. Adjusting LLM Prompts According to Translation Review Requirements
• 4. Challenges and Operational Improvements When Introducing LLM Proofreading Tools
• 5. Future Use of LLM in the Translation Review Process

1. What is an LLM Proofreading Tool｜Its Role in the Japanese-English Translation Review Process

The LLM proofreading tool is our in-house developed QA support tool that utilizes large language models (LLM) and is designed to perform quality checks on deliverables after translation review. It supports multiple languages and features both monolingual checks, which verify without referring to the source text, and bilingual checks, which compare the source and target texts. Additionally, it includes a translation evaluation function that assesses translation quality from specific perspectives.

This article introduces how to utilize the LLM proofreading tool and future policies from the perspective of someone responsible for reviewing Japanese-English translation projects, including practical examples from actual work.

Related Blogs
>>Verification Report on Translation Proofreading Accuracy Using LLM Proofreading Tool | Generative AI × Translation
>>What is Translation Proofreading? A Thorough Explanation of How to Achieve High-Quality Translations

2. Differences Between Conventional Translation QA Tools and LLM Proofreading Tools

In the QA process conducted after the review stage of Japanese-English translation, multiple QA tools such as Trados QA Checker, QA Distiller, and XBench are used in combination. These tools are highly effective for rule-based detectable items such as inconsistent notation, terminology mismatches, omissions in translation, and spelling errors.

On the other hand, traditional QA tools may not fully cover aspects such as readability, naturalness, and awkwardness in sentence structure. A major feature of the LLM proofreading tool, which leverages AI, is its ability to complement these language quality-related perspectives.

Additionally, LLM proofreading tools have the strength of detecting expressions that are not grammatically incorrect but are unnatural as technical documents or are generally rarely used. They are especially effective for checking whether there are any unnatural or awkward expressions in existing translations registered in the translation memory. Below is an example.

As these examples show, LLM proofreading tools have the strength of detecting issues related to naturalness of expression and context—problems that conventional QA tools based on programmed rules and profiles find difficult to handle—and suggesting improvements.

3. Adjusting LLM Prompts According to Translation Review Requirements

The LLM proofreading tool is based on the latest version of the ChatGPT engine, and one of its strengths is the ability to flexibly customize prompts according to the characteristics of projects and assignments.

The prompts used by the Japanese-English translation review team as basic QA checks explicitly focus on verifying grammar, spelling, and unnatural sentence structures, while excluding checks on notations that do not pose issues for the project, such as the presence or absence of a period at the end of a sentence. Furthermore, the prompts specify not only the type of English used (such as American English) but also the target countries and regions.

By organizing the instructions in this way, we suppress feedback that does not meet the project requirements and obtain QA results that are easier to handle in practical work.

4. Challenges and Operational Improvements When Introducing LLM Proofreading Tools

When this tool was first introduced (about two years ago), processing speed was an issue, and there were cases where it took several hours to complete processing even for projects ranging from several thousand to 10,000 words. Since then, through continuous updates and improvements, there are now more cases where processing is completed within an hour even for projects of tens of thousands of words, significantly improving the issue of processing time.
Additionally, since QA checks can be performed even while the translation and review processes are ongoing, the overall impact on the workflow is limited even if some projects require longer processing times.

On the other hand, there are challenges unique to AI-powered tools. While AI has a vast amount of knowledge, it may sometimes make excessive corrections on expressions that are not problematic for the project, client-specific styles, or intentionally inconsistent expressions.
To address these points, as mentioned earlier, we explicitly specify expression rules to be ignored within the prompts and set correction categories to exclude unnecessary ones. By operating with an understanding of the tool’s characteristics, we adjust the accuracy of the corrections.

5. Future Use of LLM in the Translation Review Process

The LLM proofreading tool is currently in a stage of ongoing verification and improvement. Insights gained from actual usage results and operations are recorded in a management sheet and fed back to the development team as needed.

Since formally integrating this tool into the QA process, we have noticed a reduction not only in careless mistakes and oversights but also in the variability of review quality.

Going forward, we plan to apply monolingual and bilingual checks using the LLM proofreading tool to all translation and post-editing projects as a general rule. Since the types of errors detected tend to differ between monolingual and bilingual checks, using both in combination is expected to significantly reduce the risk of mistakes and oversights in translation deliverables.

In addition to manual reviews, by combining rule-based checks using conventional QA tools and monolingual and bilingual checks using AI-powered LLM proofreading tools, we are establishing a system that allows quality to be verified from different perspectives. By building such a multilayered QA process, we believe we can stabilize the quality of translation deliverables and keep the occurrence of errors and oversights at an extremely low level.

We plan to continue improving operations and thereby enhance the overall quality of the review process.

Human Science offers manual translation services and post-editing services. We are a translation company handling a wide range of fields including software, manufacturing, IT, automotive, and distribution. Since 1994, we have been assisting many companies with their translation needs over a long period. If you have any of the following concerns, please feel free to contact us.
・Translation takes too much time!
・Poor quality leads to many complaints!
・Translation costs are too high!
・Many problems caused by Japanese manuals!
・Unfamiliar with overseas laws and standards!