Compare the translation abilities of ChatGPT, Gemini, and Claude.

1. Machine Translation

First, let's compare the translation capabilities as a machine translation engine.

1-1. Literal Translation

"Translate to English. Output only the translated text." This simple prompt was used to execute the translation. The translations from each AI are as follows: When focusing on the translations of "Leading the pack" and "know a thing or two", ChatGPT and Gemini are literal translations, which may sound unnatural in Japanese. On the other hand, Claude is a free translation and has been translated into natural Japanese without any discomfort.

Source	Leading the pack was AAA Media Group, followed by BBB Video, and CCC Technologies. DDD Brothers and EEE Wire came in fourth and fifth, respectively. AAA is a highly successful fully remote company with hundreds of employees, so we know a thing or two about building a remote work environment.
ChatGPT	At the top of the pack wasAAA Media Group, followed by BBB Video and CCC Technologies. DDD Brothers and EEE Wire came in fourth and fifth place, respectively. AAA is a highly successful fully remote company with hundreds of employees, so they know a thing or two about setting up a remote work environment.
Gemini	Leading the pack is AAA Media Group, followed by BBB Video and CCC Technologies. DDD Brothers and EEE Wire came in fourth and fifth place, respectively. AAA, being a highly successful fully remote company with hundreds of employees, knows a thing or two about building a remote work environment.
Claude	AAA Media Group ranked first, followed by BBB Video and CCC Technologies. DDD Brothers and EEE Wire ranked fourth and fifth respectively. AAA is a highly successful fully remote company with hundreds of employees, and we know a lot about building remote work environments.

1-2. Use natural Japanese

Next, I tried the prompt "Translate to natural Japanese. Output only the translation." ChatGPT remained a literal translation, while Gemini changed to a more natural expression. Claude's translation of "know a thing or two" changed from "know a lot" to "have knowledge and experience", and the translated text was brushed up to fit the context.

ChatGPT	Leading the pack isAAA Media Group, followed by BBB Video and CCC Technologies. DDD Brothers and EEE Wire came in 4th and 5th place respectively. AAA is a highly successful fully remote company with hundreds of employees, so they know a thing or two about setting up a remote work environment.
Gemini	The top spot was taken by AAA Media Group, followed by BBB Video and CCC Technologies. DDD Brothers and EEE Wire came in 4th and 5th place respectively. AAA, being a highly successful fully remote company with hundreds of employees, is well-versed in building remote work environments.
Claude	The AAA Media Group took the lead, followed by BBB Video and CCC Technologies. DDD Brothers and EEE Wire came in fourth and fifth place respectively. AAA is a highly successful fully remote company with hundreds of employees, and we also have knowledge and experience in building remote work environments.

1-3. Specify the plain style (de aru tone)

Some documents may need to be translated in the polite form (で・ある調). So I tried translating it into Japanese in the polite form (だ・である調) and outputting only the translated text. Both translation AIs were able to translate in the polite form.

ChatGPT	The top of the pack is led by AAA Media Group, followed by BBB Video and CCC Technologies. DDD Brothers and EEE Wire came in fourth and fifth place respectively. AAA is a highly successful fully remote company with hundreds of employees, so they know a thing or two about setting up a remote work environment.
Gemini	Leading the way is the AAA Media Group, followed by BBB Video, CCC Technologies, and continuing. DDD Brothers and EEE Wire took 4th and 5th place respectively. AAA is a highly successful fully remote company with hundreds of employees, so they are well-versed in creating a remote work environment.
Claude	AAA Media Group took the lead, followed by BBB Video and CCC Technologies. DDD Brothers and EEE Wire were ranked 4th and 5th respectively. AAA is a highly successful fully remote company with hundreds of employees, and we also have knowledge about building remote work environments.

1-4. The company name remains in English with the addition of "Co."

In translation instructions and style guides, there may be a specification to keep the company name in English and add "Co." in Japanese. So I tried the prompt "Translate to Japanese. Output only the translation. Keep the company name in English and add "Co."." ChatGPT correctly added "Co." to the company name, except for the "Co." in the last sentence. Gemini correctly added "Co." to all company names. Claude ignored the instruction to keep the company name in English and added "Co." to all company names. Among the three AI generators, Gemini was the most faithful to the instructions.

ChatGPT	The leader of the pack was AAA Media Group, followed by BBB Video, and then CCC Technologies. DDD Brothers and EEE Wire came in fourth and fifth place respectively. AAA is a highly successful fully remote company with hundreds of employees, so they know a thing or two about building a remote work environment.
Gemini	The top spot was taken by AAA Media Group, followed by BBB Video and CCC Technologies. DDD Brothers and EEE Wire ranked 4th and 5th respectively. AAA is a highly successful fully remote company with hundreds of employees, so they are well-versed in creating a remote work environment.
Claude	AAA Media Group took the top spot, followed by BBB Video Company and CCC Technologies Company. DDD Brothers Company and EEE Wire Company came in fourth and fifth place respectively. AAA Company is a highly successful fully remote company with hundreds of employees, and we have a lot of knowledge about building remote work environments.

2. Automatic Correction of Fuzzy Matches in Translation Memory

"Translation memory" is a database that stores previously translated sentences. By integrating with a translation support tool (CAT tool), it helps improve efficiency and quality in translation work. It can greatly reduce translation time when translating similar sentences, such as instruction manuals, or frequently updated documents. For more information on CAT tools, please refer to the following article.

What is a CAT tool? Explanation of benefits and drawbacks

"Fuzzy matching" refers to when the original text in the translation memory and the newly translated text do not match exactly. It is also known as partial matching. When a sentence is revised, there may be changes to a portion of the original text. For example, in the table below, "DeepL," has been added to the "revised original text." In this case, the previous translation from the translation memory is retrieved and manually updated to reflect the changes.

Original text in translation memory	MTrans for Office is an Office add-in and MT subscription service that allows you to directly translate documents using secure and non-reusable MT services such as Google Cloud Translation API on Microsoft Office.
Original text that has been revised	MTrans for Office is an Office add-in and MT subscription service that allows you to directly translate documents using secure and non-reusable MT services such as DeepL and Google Cloud Translation API on Microsoft Office.
Past translations in translation memory	MTrans for Office is an Office add-in and MT subscription service that allows you to translate documents directly on Microsoft Office using MT services such as the Google Cloud Translation API, which is secure and has no risk of secondary use.

2-1. Automatic Correction of Fuzzy Matches in Translation Memory

In any generated AI, the revised original text is correctly reflected in the translated text using "DeepL". However, in English, it is more natural to list two items with "and", so Gemini and Claude have been appropriately modified. Also, before the revision, "Google Cloud Translation API" was singular, so the verbs "is" and "has" were used, but now that it has become "DeepL, Google Cloud Translation API", the verbs have been changed to "are" and "have" accordingly. These types of changes are often overlooked when manually editing, but each generated AI has been appropriately modified to ensure correct English.

Original text that has been revised	MTrans for Office is an Office add-in and MT subscription service that allows you to directly translate documents using secure and non-reusable MT services such as DeepL and Google Cloud Translation API on Microsoft Office.
Past translations in translation memory	MTrans for Office is an Office add-in and MT subscription service that allows you to translate documents directly on Microsoft Office using MT services such as the Google Cloud Translation API, which is secure and has no risk of secondary use.
ChatGPT	MTrans for Office is an Office add-in and MT subscription service that allows you to translate documents directly on Microsoft Office using MT services such as DeepL, Google Cloud Translation API, which are secure and have no risk of secondary use.
Gemini	MTrans for Office is an Office add-in and MT subscription service that allows you to translate documents directly on Microsoft Office using MT services such as DeepL and the Google Cloud Translation API, which are secure and have no risk of secondary use.
Claude	MTrans for Office is an Office add-in and MT subscription service that allows you to translate documents directly on Microsoft Office using MT services such as DeepL and Google Cloud Translation API, which are secure and have no risk of secondary use.

2-2. Automatic Correction of Fuzzy Matches in Translation Memory + Terminology Glossary Specification

Product names and service names may change. Can new terms be automatically reflected while using past translation memories? In the following example, we specified "Office ADD-IN" for the term "Office add-in" and "DeepLTEST" for "DeepL", and then performed automatic correction for fuzzy matches in the translation memory. In both AI generations, the new terms are correctly reflected. However, in Claude, the verbs "is" and "has" have reverted back, so manual correction to "are" and "have" is necessary.

Past translations in translation memory	MTrans for Office is an Office add-in and MT subscription service that allows you to translate documents directly on Microsoft Office using MT services such as the Google Cloud Translation API, which is secure and has no risk of secondary use.
ChatGPT	MTrans for Office is an Office ADD-IN and MT subscription service that allows you to translate documents directly on Microsoft Office using MT services such as DeepLTEST, Google Cloud Translation API, which are secure and have no risk of secondary use.
Gemini	MTrans for Office is an Office ADD-IN and MT subscription service that allows you to translate documents directly on Microsoft Office using MT services such as DeepLTEST and the Google Cloud Translation API, which are secure and have no risk of secondary use.
Claude	MTrans for Office is an Office ADD-IN and MT subscription service that allows you to translate documents directly on Microsoft Office using MT services such as DeepLTEST, Google Cloud Translation API, which is secure and has no risk of secondary use.

3. Summary

In this article, we compared the translation abilities of ChatGPT, Gemini, and Claude's AI generators. In machine translation, Claude excels in expression. On the other hand, when translation rules are determined by translation instructions or style guides, Gemini follows the rules appropriately. Additionally, we found that Gemini excels in automatic correction of translation memory. Please note that the model used in Gemini was Gemini 1.5 Pro, which is currently in private preview and not available for translation tasks.

At Human Science, we offer automatic translation software MTrans for Office and MTrans for Trados that can utilize ChatGPT. You can also translate transcribed text with just one click. ChatGPT can not only be used as a translation engine, but also for transcription, rewriting, and proofreading depending on the prompt. MTrans for Office and MTrans for Trados also offer a 14-day free trial. Please feel free to contact us.

Features of MTrans for Office

① Unlimited number of files and glossaries that can be translated with a fixed fee
② Translate with one click from Office products!
③ Secure with API connection
・For customers who want further enhancement, we also offer SSO, IP restrictions, etc.

④ Japanese language support by Japanese companies
・Possible to respond to security check sheets
・Payment by bank transfer is available

MTrans for Office is an easy-to-use translation software for Office.

Features of MTrans for Trados

1. ① Simultaneous translation using multiple machine translation engines such as DeepL and Google.
2. ② Automatically apply terminology to machine-translated text. Centrally manage terminology glossaries regardless of machine translation engine.
3. ③ String replacement, regular expression replacement, automatic correction of translation style, notation, and expression using ChatGPT
4. ④ Automatic Correction of Fuzzy Matches in Translation Memory
5. ⑤ Maintain original formatting and tags during machine translation

 

What is the machine translation solution MTrans for Trados for Trados?