Request from Cross Language Inc.
Cross Language is a software company that provides translation software and system solutions. They have been developing and selling machine translation software for nearly 30 years and are one of the leading machine translation engine manufacturers in Japan, offering various solutions while incorporating the latest technologies.
Challenges in the Development and Sales of Machine Translation Engines
・Since we conduct quality evaluations in-house while developing products, it requires a lot of man-hours.
To make the evaluation persuasive, I want to know the objective evaluation results from a neutral standpoint, rather than "according to our research."
We would like to conduct an objective evaluation of the selection of source texts for evaluation and the evaluation of translations by outsourcing it to external parties, rather than relying on our own standards.
In the past, the quality evaluation of machine translation engines was conducted in-house, but it was quite labor-intensive to evaluate while simultaneously developing. Additionally, there was a desire to conduct a complete blind test, as the evaluation results from our own research lacked credibility, leading to the request to outsource the selection of source texts for evaluation to an external vendor.
Therefore, as a neutral party that does not develop its own engine, our company has evaluated the translation quality of Cross Language's products and compiled the analysis results into a quality assessment report for your review.
Solutions of Human Science
1. Output translations for 8 language pairs (Japanese-English, English-Japanese, Japanese-Chinese, Chinese-Japanese, Japanese-Korean, Korean-Japanese, Japanese-Thai, Thai-Japanese)
and evaluate quality based on our unique standards.
2. Submit a detailed report summarizing the quality evaluation results
We output translations for 8 language pairs (Japanese-English, English-Japanese, Japanese-Chinese, Chinese-Japanese, Japanese-Korean, Korean-Japanese, Japanese-Thai, Thai-Japanese) and conducted quality assessments of the machine translation results.
Regarding the evaluation, we have adjusted it based on our unique evaluation criteria, taking into account the specific applications of customers who implement Cross Language's engine and the key points they prioritize regarding quality.
In addition, we conducted an evaluation of the output results of machine translation by two native translators of each language suitable for that field.
One of the points we have innovated is the extraction of the original text to be evaluated.
Due to the differences in strengths, sentence structures, and other factors of machine translation engines, evaluating arbitrarily selected original texts cannot be considered a completely objective assessment.
Therefore, to ensure fairness, we randomly extracted from the original data consisting of thousands to tens of thousands of sentences using the tool developed by our company, and proceeded to output the translated text.
The evaluation results have been compiled into a report that includes a detailed assessment from multiple perspectives such as accuracy, fluency, grammar, and terminology, analyzing the characteristics of each company with specific examples of errors and sample sentences.
Customer Feedback
This was the first time we requested a quality assessment from an external vendor. The evaluation was generally within our expectations, but it was very helpful to objectively understand the strengths, areas for improvement, and specialties of our own engine.
The evaluation results from third parties have a significant impact internally as well, and it seems that they can be useful in considering the development policy.
We aim to further leverage our strengths and improve on areas for enhancement, working towards increasing the accuracy of machine translation quality. Additionally, we are considering introducing this report as a reference material for users in the future.
Without getting too excited or discouraged by the evaluation results, we plan to use this evaluation as a starting point for improvement and to conduct regular quality assessments at the timing of updates. In the future, we hope to request evaluations that analyze trends in the number of errors in more detail.
From Human Science representatives
This is an example of our assistance in quality evaluation of engine manufacturers' products from the position of a neutral machine translation consultant that does not develop its own engine.
Outsourcing the quality evaluation that has been conducted in-house until now is a first attempt for Cross Language, and it is a project that holds significant importance in considering future development policies. We have approached this with a great sense of responsibility.
Many companies considering the introduction of machine translation utilize our machine translation quality evaluation services, but there are also cases like this one where we conduct quality evaluations at the request of engine manufacturers. We feel a great sense of fulfillment in being able to contribute the quality evaluation results we conducted to the future development of machine translation engines.
Cross Language will regularly engage in evaluations, such as when products are updated in the future.
We strongly hope to assist our clients in product development and sales by providing information exchange and quality evaluation from a neutral standpoint as machine translation consultants in the future.
+81 35-321-3111[Tokyo Headquarters] 
Contact Us
Request for Information
TOP
SITE MENU