Outsource high-speed and accurate annotation work
to ensure the accuracy and reliability of machine learning systems

Interview cooperation: Sumitomo Heavy Industries, Ltd. Technical Research Institute, Mr. Masato Inoue

Sumitomo Heavy Industries, Ltd. Overview

・Established: November 1, 1934

・Capital: 30.87 billion yen (as of December 31, 2022)

・Number of Employees: Consolidated: 25,211 (as of December 31, 2022)

・Business Content: As a comprehensive machinery manufacturer, we provide comprehensive solutions from general industrial machinery to cutting-edge precision machinery, as well as construction machinery, ships, and environmental and plant equipment.

Services

・Data Annotation Service

Sumitomo Heavy Industries, Ltd. Overview

HS: First, please tell us about your company's overall business.

Mr. Inoue: Yes. Our company is a comprehensive machinery manufacturer that covers a wide range of industries, from general industrial machinery to cutting-edge precision machinery, as well as construction machinery, ships, and environmental and plant equipment. We utilize our "moving and controlling" technology in various fields, from nanotechnology to large structures, and contribute to various locations in social infrastructure.

HS: What role does the technical research institute where Mr. Inoue belongs to play among comprehensive machinery manufacturers? Is there anything in particular that they are focusing on?

Mr. Inudo: We are focusing on research and development based on future prospects by monitoring the market, customers, and megatrends. Our areas of focus are "environment and energy" and "automation and digitization", which are global trends, and we are working on these areas with a strong emphasis.

HS: What kind of mission does Mr. Inudo have in that?

Mr. Yorifuji: At our company, we have developed a common platform for function development that can be used across our product range, called "SHICuTe" (※1), in order to accelerate our efforts in automation and digitization. By analyzing and utilizing various operational data gathered by this system, our mission is to develop functions that can meet our customers' various needs and support their business.

Challenges before implementation

HS: Please tell us the reason why you decided to outsource data annotation.

Mr. Inoue: As I explained earlier, our company is currently collecting various data and starting to consider value creation. As the amount of operational data continues to increase, efficient processing using machine learning is essential. To create this machine learning system, data annotation is necessary.
The accuracy and reliability of the machine learning system are greatly influenced by the accuracy of data annotation and the amount of data. Therefore, the challenge was how to perform accurate annotation quickly. That's why we asked your company, which has experience in annotating data used in machine learning.

HS: What specific challenges did you face before requesting our services?

Mr. Yorifuji: As the system is still in the development phase, I have been doing the data annotation myself, but I cannot keep up with the speed alone. In addition, data annotation may seem like a simple task, but as mentioned earlier, its accuracy directly affects the accuracy of the machine learning model. Therefore, it was difficult to establish an internal system for checking and training.

HS: There was an intention to use external services to speed up the work volume and expertise of the business.

Reasons for Choosing Human Science

HS: What was the process for selecting the client and what were the reasons for choosing our company?

Mr. Yorifuji: First, we listed candidates based on information from the internet and exhibitions, and then approached multiple companies based on their track record. When I received an explanation from your company, the content regarding concerns about workload, quality, cost, and structure related to the annotation work was the same as what I was feeling, which became the deciding factor.

HS: I see, the importance of data annotation in our company's materials was the same as what Mr. Inudo felt, so you wanted to request it from our company. Thank you very much.

HS: Did you have any concerns or worries when requesting our services?

Mr. Inudo: As I mentioned earlier, I wasn't worried about the quality or process because there was an explanation, but I had concerns about the cost because it was our first time outsourcing data annotation.

HS: Is it a matter of not knowing how much budget to prepare?

Mr. Inudo: Yes, that's right. I wasn't sure how much budget to prepare for the required amount of data, but your company provided multiple proposals based on factors such as the education and management costs of data annotators, the unit price per data implementation, and the number of checks. This was very helpful as we were able to consider within our set budget. Additionally, the issue of data confidentiality was also one of the selection criteria as it could be addressed domestically.

HS: Thank you very much.

Introduction Effect and Evaluation of Human Science

HS: Please tell us about the process from start to finish. If you have any thoughts or feedback during the process, please let us know.

Mr. Yorifuji: We were able to consider communication plans and methods during the kickoff, so we were able to start in a way that was feasible for both parties. Along the way, when there were ambiguous data that did not follow the annotation definition document, or when there were items that were prone to variation among annotators, we were able to review the definitions and proceed smoothly thanks to the frank feedback we received.

HS: Thank you very much. In point annotation (indicating the defined location) of images that project three-dimensional space onto two dimensions, depth must be taken into consideration, so it is often difficult to align perceptions between individuals.

Mr. Indou: We were able to entrust the work with peace of mind because you were strongly aware that even small discrepancies and differences in definitions can affect the accuracy of machine learning.

HS: Thank you very much.

HS: Can you please let us know if there are any areas for improvement in our service level?

Mr. Inudo: Yes, that's right. For general cases such as image segmentation and classification, it would be greatly appreciated if you could introduce us to annotation tools and suggest ways to convert data into a format suitable for annotation, utilizing your expertise to propose cost reduction and speed improvement.

 

 

※1 Sumitomo Heavy Industries Group Common Infrastructure Platform "SHICuTe" Development and Utilization Case Introduction (Press Release)
https://www.shi.co.jp/info/2021/6kgpsq000000li2h.html