Is ChatGPT a Good Translator? Thorough Verification at Each Stage of Translation

Table of Contents

• 1. Translate
• 2. Specify Terms for Translation
• 3. Specify the style for translation
• 4. Detecting mistranslations and omissions
• 5. Proofread the translation
• 6. Create a Glossary
• 7. Security of ChatGPT
• 8. Summary

1. Translate

We compared the translation accuracy of popular automatic translation services DeepL and ChatGPT in English to Japanese translation.

 

Source	Leading the pack was AAA Media Group, followed by BBB Video, and CCC Technologies. DDD Brothers and EEE Wire came in fourth and fifth, respectively.
DeepL	トップはAAAメディア・グループ、2位はBBBビデオ、3位はCCCテクノロジーズでした。4位は「DDD Brothers」、5位は「EEE Wire」となっています。
ChatGPT	先頭を率いていたのはAAAメディアグループで、次いでBBBビデオ、そしてCCCテクノロジーでした。DDDブラザーズとEEEワイヤーは第4位と第5位になりました。

Both DeepL and ChatGPT translate correctly, but DeepL interprets the phrase "followed by" in the first sentence as "second and third" in response to "fourth and fifth" in the second sentence, providing a more nuanced translation. ChatGPT translates more faithfully to the original text.

 

Source	AAA is a highly successful fully remote company with hundreds of employees, so we know a thing or two about building a remote work environment.
DeepL	AAAは、数百人の従業員を抱えるフルリモートカンパニーとして大きな成功を収めており、リモートワークの環境構築について熟知しているのです。
ChatGPT	AAAは数百人の従業員を擁する非常に成功したフルリモートの会社であり、リモートワーク環境を構築することについて何か知っています。

Both DeepL and ChatGPT translate correctly, but DeepL rearranges the order of adjectives to create a more natural expression in Japanese. ChatGPT translates faithfully to the original text, but it is somewhat unnatural.

 

ChatGPT's translations tend to be more literal compared to DeepL, but despite being a general-purpose chatbot service, it can achieve translation accuracy comparable to specialized automatic translation services. Additionally, like automatic translation services, human review of the translations is necessary.

 

You can view the article comparing ChatGPT and DeepL in detail here.
Related Blog:What is DeepL? A comparison with ChatGPT

 

Free Download

ChatGPT Translation Accuracy Verification Report

2. Specify Terms for Translation

With ChatGPT, you can specify translations for terms. In the example below, I specified the translation for a company name (pseudonym). Compared to when no terms are specified, the expression has changed slightly, but the meaning of the translation remains correct.

 

3. Specify the style for translation

I tried translating by specifying the style of the translation. In the example below, I specified the declarative form. It was translated correctly.

 

 

In the following example, I instructed to use full-width characters for letters and numbers, but while "AAA", "BBB", "CCC", "DDD", and "EEE" were converted to full-width, other parts remained half-width. It seems that ChatGPT struggles with specifying character types.

 

4. Detecting mistranslations and omissions

ChatGPT can point out mistranslations and omissions. The following example checks the first translation from DeepL mentioned above. It identifies the parts where DeepL has taken liberties with the translation as mistranslations. This is a point where even humans may struggle to determine whether it is a free translation or a mistranslation depending on the context, but the indication is not incorrect.

 

 

In the following example, I removed "3rd place is CCC Technologies" from the translation and checked it with ChatGPT. It correctly pointed out the missing translation and made corrections. Additionally, it appropriately supplemented the "3rd place" that was not in the original text.

 

 

In the following example, it was checked without removing anything from the translation, and while it correctly identifies that "highly successful" corresponds to "a great success", it points out that "highly" is missing in the translation. It seems that the natural expression by DeepL was not understood. Additionally, the correction proposal by ChatGPT is the correct translation.

 

5. Proofread the translation

ChatGPT also has a Japanese proofreading feature. In the following example, issues such as the duplication of punctuation ("deshita..") and the omission of a sokuon ("natteimasu"), as well as inconsistencies in the use of parentheses around company names, have all been corrected.

 

6. Create a Glossary

You can extract terms that will serve as candidates for the glossary from the original text and translate those terms.

 

 

You can also extract term pairs from the translated original text and the translated text.

 

 

Can ChatGPT handle specialized translations, including medical translations?
For more details, please see here.

Related Blogs：ChatGPT and Medical Translation

7. Security of ChatGPT

By default, any data input into ChatGPT will be used to improve the model. To prevent this, access https://chat.openai.com/#settings/DataControls in your web browser and turn off the toggle switch for "Improve the model for everyone."

 

 

If you use ChatGPT via an API, the data will not be used for training, as described below. In other words, if a tool or software uses API integration to connect with ChatGPT, confidentiality of the input data will be maintained when you use the translation function of ChatGPT via that software or tool.

 

(Source: https://openai.com/policies/api-data-usage-policies)

 

For information on the security of DeepL, which was used as a comparison, please refer to the blog article below.

 

Confidentiality with DeepL Translations. Is It Secure?

8. Summary

The AI chatbot ChatGPT can also be utilized for translation tasks. The translated text may sometimes be somewhat literal and may sound unnatural, but it can achieve a level of accuracy comparable to dedicated automatic translation services like DeepL. You can also check the translations or create glossaries. By using ChatGPT, you can streamline various tasks in the translation business.

 

Our company offers an automatic translation product, MTrans for Office, with OpenAI's API integration. By using the API, you can utilize the translation function of ChatGPT without the risk of information leaks. With MTrans for Office, you can call OpenAI from Word, Excel, PowerPoint, and Outlook with just one click, to translate, proofread, or transcribe English emails. You can also create your own prompts. Try the quality and usability of MTrans for yourself with our 14-day free trial offer.

Related Services

Translate Office products with the easy translation software MTrans for Office