Comparison of recommended 5 annotation tools ~ 3 points to consider when choosing a tool ~

1. 3 Points to Consider When Choosing a Data Annotation Tool

1-1. Purpose

Data annotation tools need to be selected according to the AI model to be built in-house. The optimal annotation tool varies depending on the purpose, such as "image recognition", "speech recognition", and "natural language processing". For example, in the field of "object detection" in "image recognition", the function of tagging by enclosing specific objects in the image with a bounding box is necessary, and in the field of "natural language processing", the function of text annotation to extract specific named entities in the document is necessary.

Choose the appropriate tool for your purpose, as the types of data annotation that can be done vary depending on the tool.

1-2. Functionality and Ease of Use

In data annotation work that handles a vast amount of data, the functionality and user-friendliness (usability) of the tools are important. It is good to consider whether the UI (button arrangement and screen configuration) can be operated intuitively without a manual, whether the operation is smooth, such as loading images, and whether comments can be added to tagged images.

 

In addition, data annotation tools are broadly divided into cloud-based and installation-based types, with cloud-based tools allowing immediate use by creating an account and logging in without the need for installation. On the other hand, installation-based tools allow for local work without uploading data to external servers, providing a sense of security in data management. However, some tools may have a high barrier to entry, such as downloading from version control systems like GitHub or executing commands for installation.

 

In addition, the data format that can be output by each tool is different. JSON, PascalVOC, YOLO, COCO, etc. are all important points to consider when choosing a tool, as they support different output formats.

1-3. Administration

When working with multiple annotators on a project, managing the annotators and tasks (the smallest unit of annotation work) is also an important point to consider. For example, being able to check the daily progress of annotators (number of annotations, completed tasks, number of revisions, etc.) and the status of each task (annotated, reviewed, in revision, on hold, etc.) can help with smooth management and ensure quality.

2. 5 Data Annotation Tool Comparisons

This time, we will introduce five representative annotation tools in the field of image annotation.

2-1. Annofab

Annofab is a cloud-based data annotation tool that supports images and videos.

 

Region extraction that surrounds a specific area in an image (segmentation), object detection that surrounds a specific object in an image with a bounding box, and polyline (continuous line) extraction for objects that do not require size, such as road vehicle lanes and center lines.

 

In the class settings screen, you can set the class value, type, and color, as well as minimum settings (e.g. setting the minimum width and height of a rectangle) and error tolerance range (e.g. specifying the distance of allowable deviation for a rectangle in pixels).

 

One of the great features of Annofab is its strong support for shortcut keys and other functions in terms of operation. For example, you can assign shortcut keys for different types of classes such as "q" for cars, "w" for license plates, "e" for motorcycles, and "r" for people. This allows you to select classes with your left hand on the keyboard and perform annotation operations with your right hand on the mouse, making your work more efficient.

 

In addition, because the management function is well-equipped, you can search and list annotated data, check the progress of the entire project, and view statistical data that allows you to understand the number of times tasks (= the smallest unit of annotation work) have been returned for each data annotator.

 

Output format supports JSON.

 

For more detailed information on Annofab's basic information, advantages, and disadvantages, please see another article on this blog.

>>What is Annofab, the data annotation tool?

 

For information about Annofab (official website), click here.

2-2. FastLabel

FastLabel is a cloud-based data annotation tool that supports images, videos, text, audio, 3D, automatic annotation, and more.

 

The strength of FastLabel lies in its ability to handle a wide range of annotation types and output formats. As mentioned above, it covers all the necessary functions for annotations, from images and videos to text, audio, and even 3D. In terms of image annotation, it supports various types such as rectangles, circles, polygons, keypoints, lines, segmentation, and skeleton estimation. It also supports formats required in many projects, such as JSON, COCO, PascalVOC, YOLO, and CSV.

 

In addition, FastLabel is fast and responsive, always displaying pages and menus smoothly when loading or navigating between them.

 

In addition, the paid version has the strength of having functions to streamline tasks such as automatic annotation and smart annotation. Automatic annotation uses pre-built AI models to automatically detect and assign bounding boxes for objects such as cars, pedestrians, and license plates.

 

In addition, with Smart Annotation (only for image segmentation), it automatically detects regions based on the color values of the image, making it faster to annotate than manually detecting regions from scratch.

 

For more information about FastLabel, click here.

2-3. Labelme

Labelme is an installation-based image and video data annotation tool.

 

If it is image annotation, it corresponds to rectangles, polygons, circles, lines, points, etc. There is also a menu to adjust the brightness of the image, which can be used to increase the brightness of dark and difficult-to-see areas to help with the identification of annotation areas.

 

In addition, each individual action such as loading images and adding/editing annotations is quick and efficient, as the necessary minimum functions are focused on for data annotation.

 

Installation is convenient as it only requires downloading the executable file locally, without the need to create an account or log in, making it easy to start using immediately.

 

Output format supports JSON.

2-4. CVAT

CVAT (Computer Vision Annotation Tool) is an installation-based open-source annotation tool developed and released by Intel.

 

Supports image and video data annotation, with support for rectangles, polygons, lines, points, circles, and cubes. It also includes automatic annotation functionality. Automatic annotation can be performed on over 80 pre-defined objects such as cars, people, airplanes, bicycles, dogs, etc.

 

CVAT does not have a function to directly send back images with errors to the data annotator during checks. However, by recording the URL of the image with errors in an input field called "Issue Tracker", the data annotator can move to the indicated image from that link and make corrections.

 

In addition, it is also a feature that there are a wide variety of data formats that can be exported (CVAT, COCO, Datumaro, CamVid, Cityscapes, etc.).

 

For more information about CVAT, please click here.

2-5. VoTT

VoTT (Visual Object Tagging Tool) is an installation-based open source annotation tool developed by Microsoft.

 

Supports image and video data annotation, with smooth performance and an intuitive UI that even those without annotation experience can operate.

 

VoTT has installers available for Windows, Mac, and Linux, making it easy for anyone to install.

 

However, for projects with multiple members, it is necessary to manage them in a different way as it does not have functions such as data annotators, task progress management, and checking.

 

Output formats supported include Azure Custom Vision service, Microsoft Cognitive Toolkit (CNTK), PascalVOC, TensorFlow Records, VoTT JSON, and CSV.

3. Summary

This time, we explained three points to consider when choosing an annotation tool, and introduced five recommended image annotation tools.

 

In recent years, the number of data annotation tools has increased, so it is important to choose and utilize the most suitable data annotation tool for your company's purposes in order to streamline the time-consuming and labor-intensive data annotation process as much as possible.

 

In addition, if you want to reduce the cost of introducing annotation tools, considering outsourcing the annotation itself is also one effective option. We offer a wide range of services from consultation on annotation tools to outsourcing of annotation, so please feel free to contact us.

4. For inquiries about utilizing AI, please contact Human Science Co., Ltd.

4-1. Utilize the latest data annotation tools

One of the annotation tools introduced by Human Science, Annofab, allows customers to check progress and provide feedback on the cloud even during project execution. By not allowing work data to be saved on local machines, we also consider security.

4-2. 48 million records of teacher data creation

"I want to introduce AI, but I don't know where to start."

"I don't know what to ask for when outsourcing."

Please consult Human Science Co., Ltd. in such cases.

At Human Science, we are involved in AI development projects in various industries such as natural language processing, medical support, automotive, IT, manufacturing, and construction. Through direct transactions with many companies including GAFAM, we have provided over 48 million high-quality training data. We can handle various annotation projects regardless of industry, from small-scale projects to large-scale projects with 150 annotators.

>>Data Annotation Service by Human Science Co., Ltd.

4-3. Resource Management without Using Crowdsourcing

At Human Science, we do not use crowdsourcing and instead directly contract with workers to progress projects. We carefully assess each member's practical experience and evaluations from previous projects to form a team that can perform to the best of their abilities.

4-4. Equipped with a security room within the company

At Human Science Co., Ltd., we have a security room that meets the ISMS standards in our Shinjuku office. Even for highly confidential projects, we can provide on-site support. We consider ensuring confidentiality to be extremely important for all of our projects. We continuously provide security education to our staff and pay close attention to handling information and data, even for remote projects.