Three perspectives for automating annotations. Is automation of work with annotation tools realistic?

1. Position and Types of Data Annotation in AI Development

1-1. Data annotation

To improve the accuracy of AI, machine learning using training data is necessary. The process of creating this training data is called data annotation. Each piece of data that serves as material is annotated with tags, labels, and metadata. By incorporating this training data, AI can recognize patterns and improve its accuracy.
This is the role of annotation in the process of AI development.

For more information on the meaning of data annotation, please refer to the following article.
>>What is Data Annotation? Explanation from its meaning to its relationship with AI and machine learning.

1-2. What kind of data can be annotated?

Data annotation can be divided into several types. Here, we introduce the currently common annotation.

1-3. Data Annotation for Images

Image annotation can be broadly classified into three categories.

・Object Detection
Identifies objects from images and adds meaningful tags depending on the target, such as "human" or "car".

・Region Extraction (Semantic Segmentation)
Select regions within an image and annotate them with tags. This identifies the meaning of the selected regions, such as "clothing" or "door".

・Image Classification
Tags are added to images to classify them. Additional information such as "polka dot or stripe" and "open or closed" is attached.

1-4. Video Data Annotation

This is a data annotation that has many similarities with images. Since videos can also be viewed frame by frame like images, it can be said that it is a difference in data format. Some differences from image annotation include tagging and labeling at the segment level (from which second to which second) in videos.

1-5. Voice Data Annotation

There are two types of tagging for volume and type of sound: one for tagging based on the sound itself, and one for tagging based on the meaning of human speech. In the latter case, the speech is transcribed into text and each word is tagged, which is a common procedure. This is used in various fields such as voice recognition and user intent extraction for smartphones and smart speakers, as well as customer service voice calls.

1-6. Text Data Annotation

Data annotation allows you to extract specific text from a large amount of documents, and aggregate necessary text and phrases from scattered data. It is also possible to tag and analyze documents according to pre-set rules, and to remove inappropriate content. It is used in various situations such as business documents, manuals, invoices, and contracts.

2. Challenges of Data Annotation

2-1. Takes a lot of time anyway

For any type of data annotation, it is necessary for the assigned worker to manually add information to each piece of data. This task requires a great deal of attention, patience, as well as a deep understanding and insight into the rules and the data being annotated. Some projects may also include a training period for workers to become proficient before starting the actual annotation process. In this case, it may also be necessary to assign a trainer. This is a project that requires a significant amount of time and the right resources.

2-2. Difficulty of Project Management

In data annotation projects, there may be dozens to over 100 workers. The role of the project manager is crucial in ensuring the quality of the training data. This includes creating work guidelines, handling questions and specification changes during the project, managing productivity, and being proficient in the work themselves.

3. Is it possible to automate work using data annotation tools?

 

3-1. Current Difficulties in Automation

The evolution of data annotation in its current state is more appropriately called efficiency rather than automation. There have been improvements and innovations in tools that have made tasks that previously required manual input now available in a selection format. However, ultimately it is the worker who makes the judgement. In conclusion, complete automation of data annotation is difficult at this point in time. While research is progressing, it is not realistic to expect the same level of work quality as a human data annotator.

3-2. Three Perspectives for Automating Data Annotation

When considering automating data annotation in the future, it is necessary to carefully consider the following perspectives.

・Is the automatic data annotation technology in that field at a practical stage?
・How much efficiency can be expected through automation?
・Is the risk of redoing or correcting acceptable?

In the current situation, attempts to automate data annotation have not been successful and may require manual intervention for corrections. In some cases, it may be more efficient to have the work done manually from the beginning. It is important to consider multiple perspectives to avoid this outcome.

3-3. Reasons for Choosing Data Annotation Services

As mentioned earlier, it currently takes time and manpower to perform high-quality data annotation. In addition to the data annotators who are responsible for the work, the roles of trainers, checkers, and project managers who manage the projects are also necessary. It may be difficult to achieve all of this with internal resources alone. Therefore, it is common for many companies to outsource data annotation projects to external service providers. However, the project management structure varies greatly depending on the outsourcing company. This difference in structure directly affects the quality of the training data, so careful selection of the outsourcing company is necessary.

4. Human Science's Data Annotation Outsourcing Service

4-1. Utilize the latest data annotation tools

At Human Science, we are constantly introducing the latest data annotation tools to pursue further improvements in quality and work efficiency. One of the tools we have introduced, AnnoFab, allows for progress checks and reviews of deliverables during project progress. It is also possible to use the check function to detect work omissions and common mistakes mechanically. Real-time communication with data annotators allows for immediate notification of changes and additions to standards and rules. We are also flexible in adapting to new methods such as 3D annotation. To ensure security, work data cannot be saved on local machines.

4-2. Resource Management without Using Crowdsourcing

Human Science's efforts for efficiency do not only involve reviewing work processes and selecting annotation tools. It also includes assigning tasks to resources that are suitable for the project's nature. Human Science does not use crowdsourcing, but instead directly contracts resources to carry out the project. Each member's work experience and evaluations from previous projects are thoroughly assessed to form a team that can perform at their maximum potential.

4-3. Equipped with a security room within the company

At Human Science, we have a security room that meets the ISMS standards in our Shinjuku office. We can handle highly confidential projects on-site. We consider ensuring confidentiality to be extremely important for all projects. We continuously provide security education to our staff and pay close attention to handling information and data, even for remote projects.

4-4. 48 million records of teacher data creation

If your company is interested in implementing AI but unsure of where to start, please consult with Human Science. At Human Science, we have participated in AI development projects in various industries such as natural language processing, medical support, automotive, IT, manufacturing, and construction. Through direct transactions with numerous companies including GAFAM, we have provided over 48 million high-quality annotated data. We handle various annotation projects regardless of industry, from small-scale projects to large-scale projects with 150 annotators.