Recommended 6 Text Annotation Tools: A Comparison ~ 3 Points to Consider When Choosing a Tool ~

1. 3 Points to Consider When Choosing a Data Annotation Tool

1-1. Purpose

Text annotation tools need to be selected according to the AI model that the company is building. Representative types of text annotation include "named entity extraction", "sentiment analysis", and "class classification", but the optimal annotation tool will vary for each. For example, for "named entity extraction", a function to surround specific words in the sentence with span tags is necessary. For "sentiment analysis" using dialogue, it would be useful to be able to tag each sentence. For "class classification" which classifies the entire text, a function to tag the entire text is necessary. The types of annotation that can be done vary depending on the tool, so choose the one that suits your purpose.

1-2. Functionality and Ease of Use

In data annotation work that handles a vast amount of data, the functionality and user-friendliness (usability) of the tools are important. In terms of usability, it is important for the productivity to have an intuitive UI (button arrangement and screen configuration) that can be operated without a manual, whether there are sufficient shortcut keys, and whether the operations such as data loading are smooth. In terms of functionality, it is good to consider whether it is possible to create the necessary data for AI to learn, such as the function to associate span tags with each other.

 

In addition, data annotation tools are divided into two main types: cloud-based and locally installed. With the cloud-based type, no installation is required and you can start using it immediately by creating an account and logging in.

 

On the other hand, local types are reassuring in terms of data security management as they allow you to work without taking data out to external cloud servers. Some tools require you to download them from version control systems such as GitHub or execute commands to install them, making the implementation process more difficult. Additionally, many tools do not have the ability to centrally manage data, making data management more complicated and not suitable for collaborative work.

 

In addition, the data format that can be output by each tool may differ. Whether the desired output format is supported or not is also an important factor to consider when choosing a tool.

1-3. Administration

When working with multiple annotators on a project, managing the annotators and tasks (the smallest unit of annotation work) is also an important point to consider. For example, being able to check the daily progress of annotators (number of annotations, completed tasks, number of revisions, etc.) and the status of each task (annotated, reviewed, in revision, on hold, etc.) can help with smooth management and ensure quality.

In most local tools, these management functions are not included, but many cloud tools have management functions and are effective for projects that involve large amounts of data and are carried out by multiple people over a long period of time.

2. Data Annotation Tool Comparison 6 Selections

This time, we will introduce six representative annotation tools in the field of text annotation.

2-1. FastLabel

FastLabel is a cloud-based data annotation tool that supports images, videos, text, audio, 3D, and automatic annotation.

 

 

FastLabel's text annotation supports "named entity extraction", "classification", and "pair classification".
"Named entity extraction" is an annotation that extracts specified words or sentences from the text. "Classification" allows you to categorize the entire text into specified types. In addition, with "pair classification", you can compare and classify two texts side by side.

 

In addition, FastLabel is fast and responsive, always displaying smoothly when loading pages or navigating between menus. It also supports auto annotation, allowing for a reduction in manual labor costs. Furthermore, it comes equipped with project management functionality, allowing for work progress and data review to be completed within the tool.

 

For more information about FastLabel, click here.

2-2. brat

brat is an open-source, locally installed tool that can be used in a browser. It stands for "BRAT Rapid Annotation Tool" and allows for named entity extraction and linking. It also allows for normalizing nouns by linking them to sources such as Wikipedia. Multiple people can access and work on annotation data simultaneously.

 

 

To use, you need python2 and install it by entering commands in the terminal, etc. Classification label settings cannot be done on the tool, so you need to directly write them in the label setting file prepared in the installed brat directory. Also, you need to create a file to export the annotation data in advance. Information about these installations and necessary settings is only explained in overview on the homepage, so the hurdle from installation to starting annotation work can be considered somewhat high. Also, there is no project management function such as review function or progress/status, so when working on a project with multiple people, it is necessary to create an appropriate management plan to supplement it.

 

There are many external forums for projects using this tool, and you can refer to various projects. It can be said that data annotation work is ideal for academic research.

 

For more information about brat, please click here.

2-3. LabelBox

LabelBox is a cloud-based data annotation tool. It supports various types of annotations such as images, videos, text, medical data compatible with DICOM, and map data such as COG. The paid version offers more features, while the free version is limited and serves as a trial version. Text annotation in the free version is limited to classifying sentences. It can be used for sentiment analysis in conversations, etc.

 

 

The paid version supports various text annotations such as named entity extraction and text classification. In addition, with already annotated data, automatic annotation is also possible. It also has management functions such as reviews and progress status, making it useful for large-scale projects and projects with continuity.

 

For more information about LabelBox, click here.

2-4. CVAT

CVAT (Computer Vision Annotation Tool) is an installation-based open-source annotation tool developed and released by Intel.

Supports image and video data annotation, with support for rectangles, polygons, lines, points, circles, and cubes, as well as automatic annotation functionality. Automatic annotation can be performed on over 80 pre-specified objects (such as cars, people, airplanes, bicycles, dogs, etc.).
While CVAT does not have a feature to directly return images with errors to annotators during checks, it does have an "Issue Tracker" input field where the URL of the image with errors can be recorded. This allows annotators to easily access and fix the images with errors.
Additionally, CVAT is known for its wide range of exportable data formats, including CVAT, COCO, Datumaro, CamVid, Cityscapes, and more.
Learn more about CVAT here.

2-5. VoTT

VoTT (Visual Object Tagging Tool) is an installation-based open source annotation tool developed by Microsoft.

Supports image and video data annotation, with smooth operation and an intuitive UI that even those without annotation experience can use.
VoTT has installers available for Windows, Mac, and Linux, making it easy for anyone to install.
However, it does not have features for managing annotators, task progress, or checking, so for projects with multiple people, it is necessary to manage them in a different way.
Output formats supported include Azure Custom Vision service, Microsoft Cognitive Toolkit (CNTK), PascalVOC, TensorFlow records, VoTT JSON, and CSV.

2-6. labelimg

Labelimg is an open source tool for image data annotation, supporting bounding box annotation.

 

 

When installing, you will need to use a terminal or similar tool to enter commands, but compared to other tools, the steps are simple.
To start the annotation process, simply place the file defining the class names as classes.txt in the specified folder, prepare the image folder and the folder for annotation output, and specify the paths for each folder in the tool. This tool can be used locally, so it can also handle annotation tasks that cannot be done with cloud tools.
Like VoTT, there is no management function for task assignment, progress management, or task feedback, so if multiple people are working on the task, these management skills will be necessary.
The output formats supported are PascalVOC and YOLO.

3. Summary

This time, we explained three points to consider when choosing an annotation tool, and introduced three recommended text annotation tools.

 

In recent years, the number of data annotation tools has increased, so it is important to choose and utilize the most suitable data annotation tool for your company's purposes in order to streamline the time-consuming and labor-intensive data annotation process as much as possible.

 

In addition, if you want to reduce the cost of introducing annotation tools, considering outsourcing the annotation itself is also one effective option. We offer a wide range of services from consultation on annotation tools to outsourcing of annotation, so please feel free to contact us.

4. Human Science's Data Annotation Outsourcing Service

4-1. Rich track record of creating 48 million pieces of teacher data

At Human Science, we are involved in AI model development projects in various industries such as natural language processing, medical support, automotive, IT, manufacturing, and construction. Through direct transactions with numerous companies including GAFAM, we have provided over 48 million high-quality training data. We handle various annotation projects regardless of industry, from small-scale projects to large-scale projects with 150 annotators. If your company is interested in implementing AI models but unsure of where to start, please consult with us. 

4-2. Resource Management without Using Crowdsourcing

At Human Science, we do not use crowdsourcing and instead directly contract with workers to manage projects. We carefully assess each member's practical experience and evaluations from previous projects to form a team that can perform to the best of their abilities.

4-3. Utilizing the Latest Data Annotation Tools

One of the annotation tools introduced by Human Science, AnnoFab, allows customers to check progress and provide feedback on the cloud even during project execution. By not allowing work data to be saved on local machines, we also consider security.  

4-4. Equipped with a security room within the company

At Human Science, we have a security room that meets the ISMS standards in our Shinjuku office. This allows us to provide on-site support for highly confidential projects and ensure security. We consider confidentiality to be extremely important for all projects at our company. We continuously provide security education to our staff and pay close attention to the handling of information and data, even for remote projects.