What is the translation ability of ChatGPT? Thorough verification at each stage of translation

Table of Contents

• 1. Translate
• 2. Translate by specifying terms
• 3. Translate with specified style
• 4. Detect mistranslations and omissions
• 5. Proofread the translation
• 6. Create a glossary
• 7. About the security of ChatGPT
• 8. Summary

1. Translate

I compared the translation accuracy of popular automatic translation services DeepL and ChatGPT in English to Japanese translation.

 

Source	Leading the pack was AAA Media Group, followed by BBB Video, and CCC Technologies. DDD Brothers and EEE Wire came in fourth and fifth, respectively.
DeepL	Top is AAA Media Group, 2nd is BBB Video, 3rd is CCC Technologies. 4th is "DDD Brothers", 5th is "EEE Wire".
ChatGPT	AAA Media Group led the way, followed by BBB Video, and then CCC Technology. DDD Brothers and EEE Wire came in fourth and fifth place, respectively.

Both DeepL and ChatGPT translate accurately, but DeepL translates the phrase "followed by" in the first sentence as "2nd" and "3rd" in response to the phrase "fourth and fifth" in the second sentence, providing a more nuanced translation. ChatGPT translates faithfully to the original text.

 

Source	AAA is a highly successful fully remote company with hundreds of employees, so we know a thing or two about building a remote work environment.
DeepL	AAA has achieved great success as a fully remote company with hundreds of employees, and is well-versed in building remote work environments.
ChatGPT	AAA is a highly successful fully remote company with hundreds of employees, and knows something about building a remote work environment.

Both DeepL and ChatGPT translate accurately, but DeepL rearranges the order of adjectives to create a more natural expression in Japanese. ChatGPT translates faithfully to the original text, but it may sound slightly unnatural.

 

The translation of ChatGPT is more literal compared to DeepL, but despite being a general-purpose chatbot service, it can achieve a translation accuracy similar to specialized automatic translation services. However, like automatic translation services, it is necessary to check the translated text by humans.

 

Please see the article comparing ChatGPT and DeepL here.
Related Blog:What is DeepL? Also, comparing it with ChatGPT

 

Free Download

ChatGPT Translation Accuracy Verification Report

2. Translate by specifying terms

In ChatGPT, you can also specify the translation of terms to be translated. In the following example, we specified the translation of the company name (tentative) and tried translating it. The expression has changed slightly compared to not specifying the terms, but the meaning of the translation remains correct.

 

3. Translate with specified style

I tried translating by specifying the style of the translation. In the following example, I translated using the polite form. It was translated correctly.

 

 

In the following example, we instructed to use full-width characters for alphabets and numbers, but while "AAA", "BBB", "CCC", "DDD", and "EEE" became full-width, other parts remained half-width. It seems that ChatGPT is not good at specifying character types.

 

4. Detect mistranslations and omissions

ChatGPT can point out mistranslations and omissions. The following example is a check of the translation of the first DeepL above. DeepL has determined that the parts where it paraphrased are mistranslations. This is a place where even humans struggle to determine whether it is a paraphrase or a mistranslation depending on the context, but the pointing out is not incorrect.

 

 

In the following example, we removed "3rd place is CCC Technologies" from the translation and checked it with ChatGPT. We correctly identified and fixed translation omissions. Additionally, we appropriately supplemented the missing "3rd place" in the original text.

 

 

In the following example, when checked without removing anything from the translation, it was pointed out that "highly" was missing, although it correctly grasped the correspondence between "highly successful" and "大きな成功を収めており". It seems that DeepL did not understand the natural expression. Note that the proposed correction by ChatGPT is the correct translation.

 

5. Proofread the translation

ChatGPT also includes a Japanese proofreading feature. In the example below, all instances of duplicated periods ("でした。。") and missing small tsu ("なています"), as well as inconsistencies in the use of parentheses around the company name, have been corrected.

 

6. Create a glossary

You can provide the original text and extract terms that can be candidates for a glossary, and translate their translations.

 

 

You can also extract term pairs from translated source texts and translations.

 

 

Can ChatGPT handle specialized translations, including medical translations?
For more details, please see here.

Related Blogs:ChatGPT and Medical Translation

7. About the security of ChatGPT

By default, the input data will be used to improve the model. To prevent this, access https://chat.openai.com/#settings/DataControls in your web browser and turn off the toggle switch for "Improve the model for everyone" → "Improve the model for everyone".

 

 

If you use ChatGPT via API, the data will not be used for training as described below. In other words, if software or tools are integrated with the API, using the translation function of ChatGPT from that software or tool will ensure the confidentiality of the data is maintained.

 

(Source: https://openai.com/policies/api-data-usage-policies)

 

Please refer to the following blog article for information on the security of DeepL, which we have taken as a comparison target.

 

"Is confidentiality maintained with DeepL Translation? What about security?"

8. Summary

ChatGPT, an AI chatbot, can also be used for translation work. The translated text may sometimes sound slightly unnatural due to literal translation, but it can achieve a similar level of accuracy as dedicated automatic translation services like DeepL. You can use it to check translations and create glossaries. By using ChatGPT, you can streamline various tasks in the translation industry.

 

Our company offers an automatic translation product MTrans for Office that is integrated with OpenAI API. By using the API, you can utilize the translation function of ChatGPT without the risk of information leakage. With MTrans for Office, you can call OpenAI with just one click from Word, Excel, PowerPoint, and Outlook to translate, proofread, or transcribe English emails. You can also create your own prompts. Try it out for free for 14 days to experience the quality and usability.

Related Services

Translate Office with the easy translation software MTrans for Office