ChatGPT and DeepL ①: Comparison of Japanese-English Translation Accuracy in Manufacturing and IT Fields

1. Verification ①: Simple Structure Text

First, I will try with a simple structured sentence.
By the way, in all of this verification, I translated simply with the basic instruction of "Translate this sentence into English" without adding any special instructions or prompts to ChatGPT.
The result of the first example sentence is as follows.

Based on the above results, the points to consider when performing post-editing are as follows. (Note: The level of post-editing generally adheres to the standards of full editing.)

I feel that DeepL's translations excel in having a lower post-editing burden. On the other hand, ChatGPT's translations seem to contain many unnecessary expressions such as "purposes" and "please." Additionally, ChatGPT's translations tend to use plural forms, like "ports" and "purposes," which may require adjustments depending on the context.

In the case of human translation, it will be as follows. For comparison, I have also included the translation results from ChatGPT and DeepL before the above post-editing.

By the way, adding "any device" after "connect" is a unique touch that only human translation can provide. If you simply translate the sentence "Do not connect to this connector" as "Do not connect to this connector," it becomes unnatural because "connect" is a transitive verb and lacks an object.

It is almost impossible for machine translation or AI translation to interpret and accurately translate information that has been omitted in the original text. In Japanese technical documents, subjects and objects are often omitted, so it is important to keep this in mind when using machine translation or AI translation for Japanese-English translation.

2. Verification ②: Simple Structure Text

I will try another piece of writing with a simple structure.

There are two points of concern regarding the translation results of ChatGPT and DeepL. First, the original text uses "please," yet it is not translated in the imperative form (of course, it is not always necessary to translate "please" in the imperative form). The second point is that both translations use "please note." Indeed, it indirectly includes the meaning of "please be aware," but I found it interesting that both reached the same result.

Neither configuration meets the standards regarding quality, and a complete rewrite is necessary. (However, in the case of light editing, a complete rewrite is not required.)

If a person translates, it will look like the following. This time, for comparison, I have also included the results before the post-editing by ChatGPT and DeepL.

3. Verification ③: Complex Structured Text

Next, we will conduct verification with a slightly more complex structure of text.

In the case of DeepL, there are mistranslations and ambiguous expressions, but there are no significant issues with the overall structure, so the post-editing burden is not that great. On the other hand, in the case of ChatGPT, the structure of the text is indeed advanced, but since the original content is not accurately conveyed, a complete rewrite is necessary.

If translated by a person, it would be as follows. I believe this is a response unique to human translation, as I have made the original text, which is a single sentence, more readable by splitting it into two parts.

4. Results of Verification & Summary

Below are the results of this verification.

According to this comparative verification, it has been shown that, from the perspective of load, DeepL tends to be slightly lighter.

However, there are points to consider regarding this result. In this verification, we only provided a simple instruction to ChatGPT: "Please translate the target text into English." In reality, ChatGPT's strength lies in translating when given specific instructions regarding grammar, terminology, and sentence structure. By providing more detailed instructions, more accurate translations can be obtained.

However, there are some drawbacks to this strength of ChatGPT. First, if you are not familiar with the grammar, structure, style guides, formats, and basic rules of technical writing for the target language, it can be difficult to provide accurate instructions. Additionally, giving specific instructions each time can be cumbersome.

Ultimately, what is important is to understand the strengths of each engine and to utilize them appropriately according to needs.

For example, DeepL is useful for processing large amounts of text at once and when post-editing is desired. On the other hand, ChatGPT is suitable when you want to provide detailed instructions regarding terminology, grammar, and structure at the sentence or paragraph level.

Of course, there are many regulations and reference materials regarding formats and terminology, and in translation tasks that require detailed cross-references, human translation can sometimes be more efficient and cost-effective.

We will continue to verify the translation accuracy of technical documents using ChatGPT and DeepL, and we will publish the results, so please look forward to it.

Human Science offers human translation services and post-editing services. We are a translation company that handles translations in a wide range of fields including software, manufacturing, IT, automotive, and distribution. Since 1994, we have been assisting many companies with their translation needs. If you have any concerns like the ones below, please feel free to consult us.

・Translation takes too long!
・Quality is poor and there are many complaints!
・Translation costs are too high!
・Many issues are caused by the Japanese manual!
・Unfamiliar with overseas laws and standards!

Free Download

What are the five key points to success in Japanese-English translation projects?