Is ChatGPT a Good Translator? Thorough Verification at Each Stage of Translation

Table of Contents

• 1. Translate
• 2. Translate by specifying terms
• 3. Translate with specified style
• 4. Detect mistranslations and omissions
• 5. Proofread the translation
• 6. Create a glossary
• 7. About the security of ChatGPT
• 8. Summary

1. Translate

I compared the translation accuracy of popular automatic translation services DeepL and ChatGPT in English to Japanese translation.

 

Source	Leading the pack was AAA Media Group, followed by BBB Video, and CCC Technologies. DDD Brothers and EEE Wire came in fourth and fifth, respectively.
DeepL	トップはAAAメディア・グループ、2位はBBBビデオ、3位はCCCテクノロジーズでした。4位は「DDD Brothers」、5位は「EEE Wire」となっています。
ChatGPT	先頭を率いていたのはAAAメディアグループで、次いでBBBビデオ、そしてCCCテクノロジーでした。DDDブラザーズとEEEワイヤーは第4位と第5位になりました。

Both DeepL and ChatGPT translate accurately, but DeepL translates the phrase "followed by" in the first sentence as "2nd" and "3rd" in response to the phrase "fourth and fifth" in the second sentence, providing a more nuanced translation. ChatGPT translates faithfully to the original text.

 

Source	AAA is a highly successful fully remote company with hundreds of employees, so we know a thing or two about building a remote work environment.
DeepL	AAAは、数百人の従業員を抱えるフルリモートカンパニーとして大きな成功を収めており、リモートワークの環境構築について熟知しているのです。
ChatGPT	AAAは数百人の従業員を擁する非常に成功したフルリモートの会社であり、リモートワーク環境を構築することについて何か知っています。

Both DeepL and ChatGPT translate accurately, but DeepL rearranges the order of adjectives to create a more natural expression in Japanese. ChatGPT translates faithfully to the original text, but it may sound slightly unnatural.

 

The translation of ChatGPT is more literal compared to DeepL, but despite being a general-purpose chatbot service, it can achieve a translation accuracy similar to specialized automatic translation services. However, like automatic translation services, it is necessary to check the translated text by humans.

 

You can view the article comparing ChatGPT and DeepL in detail here.
Related Blog:What is DeepL? A comparison with ChatGPT

 

Free Download

ChatGPT Translation Accuracy Verification Report

2. Translate by specifying terms

In ChatGPT, you can also specify the translation of terms to be translated. In the following example, we specified the translation of the company name (tentative) and tried translating it. The expression has changed slightly compared to not specifying the terms, but the meaning of the translation remains correct.

 

3. Translate with specified style

I tried translating by specifying the style of the translation. In the following example, I translated using the polite form. It was translated correctly.

 

 

In the following example, we instructed to use full-width characters for alphabets and numbers, but while "AAA", "BBB", "CCC", "DDD", and "EEE" became full-width, other parts remained half-width. It seems that ChatGPT is not good at specifying character types.

 

4. Detect mistranslations and omissions

ChatGPT can point out mistranslations and omissions. The following example is a check of the translation of the first DeepL above. DeepL has determined that the parts where it paraphrased are mistranslations. This is a place where even humans struggle to determine whether it is a paraphrase or a mistranslation depending on the context, but the pointing out is not incorrect.

 

 

In the following example, we removed "3rd place is CCC Technologies" from the translation and checked it with ChatGPT. We correctly identified and fixed translation omissions. Additionally, we appropriately supplemented the missing "3rd place" in the original text.

 

 

In the following example, when checked without removing anything from the translation, it was pointed out that "highly" was missing, although it correctly grasped the correspondence between "highly successful" and "大きな成功を収めており". It seems that DeepL did not understand the natural expression. Note that the proposed correction by ChatGPT is the correct translation.

 

5. Proofread the translation

ChatGPT also includes a Japanese proofreading feature. In the example below, all instances of duplicated periods ("でした。。") and missing small tsu ("なています"), as well as inconsistencies in the use of parentheses around the company name, have been corrected.

 

6. Create a glossary

You can provide the original text and extract terms that can be candidates for a glossary, and translate their translations.

 

 

You can also extract term pairs from translated source texts and translations.

 

 

Can ChatGPT handle specialized translations, including medical translations?
For more details, please see here.

Related Blogs:ChatGPT and Medical Translation

7. About the security of ChatGPT

By default, any data input into ChatGPT will be used to improve the model. To prevent this, access https://chat.openai.com/#settings/DataControls in your web browser and turn off the toggle switch for "Improve the model for everyone."

 

 

If you use ChatGPT via an API, the data will not be used for training, as described below. In other words, if a tool or software uses API integration to connect with ChatGPT, confidentiality of the input data will be maintained when you use the translation function of ChatGPT via that software or tool.

 

(Source: https://openai.com/policies/api-data-usage-policies)

 

Please refer to the following blog article for information on the security of DeepL, which we have taken as a comparison target.

 

"Is confidentiality maintained with DeepL Translation? What about security?"

8. Summary

ChatGPT, an AI chatbot, can also be used for translation work. The translated text may sometimes sound slightly unnatural due to literal translation, but it can achieve a similar level of accuracy as dedicated automatic translation services like DeepL. You can use it to check translations and create glossaries. By using ChatGPT, you can streamline various tasks in the translation industry.

 

Our company offers an automatic translation product MTrans for Office that is integrated with OpenAI API. By using the API, you can utilize the translation function of ChatGPT without the risk of information leakage. With MTrans for Office, you can call OpenAI with just one click from Word, Excel, PowerPoint, and Outlook to translate, proofread, or transcribe English emails. You can also create your own prompts. Try it out for free for 14 days to experience the quality and usability.

Related Services

Translate Office products with the easy translation software MTrans for Office