Comparison of 5 Recommended Annotation Tools - What are the 3 Points to Consider When Choosing a Tool?

1. Three Points to Choose an Annotation Tool

1-1. Purpose

Annotation tools need to be selected based on the type of AI model you want to build in-house. The optimal annotation tool varies depending on the purpose, such as "image recognition," "speech recognition," or "natural language processing." For example, in the area of "image recognition" for "object detection," a feature that allows you to enclose specific objects in the image with bounding boxes and tag them is necessary. In the field of "natural language processing," a text annotation feature that extracts specific named entities within a document is required.

In this way, the types of annotations that can be made by tools vary, so let's choose the tool that suits our purpose.

1-2. Features and Usability

In annotation work that processes vast amounts of data, the functionality and usability (operability) of the tools are crucial. It is advisable to consider whether the UI (button layout and screen configuration) is intuitive enough to operate without a manual, whether actions such as image loading are smooth, and whether comments can be added to tagged images.

 

Additionally, annotation tools are broadly divided into cloud-based and installed types. Cloud-based tools can be used immediately after creating an account and logging in, without the need for installation. On the other hand, installed tools allow you to work locally without uploading data to external servers, providing peace of mind regarding data security management. However, some tools may have a high barrier to entry, requiring you to download the tool from version control systems like GitHub or execute commands for installation.

 

Furthermore, the data formats that can be output by each tool vary. Whether the tool supports the desired output format, such as JSON, PascalVOC, YOLO, or COCO, is one of the important points to consider when choosing a tool.

1-3. Management

When working with many annotators on a single project, the management functions for annotators and tasks (the smallest unit of annotation work) are also crucial points not to be overlooked. For example, being able to check the daily progress of annotators (number of annotations, number of completed tasks, number of rejections, etc.) and the status of each task (annotated, reviewed, under review, on hold, etc.) can facilitate smooth management operations and also help ensure quality.

5 Selected Annotation Tool Comparisons

This time, we will introduce five representative annotation tools in the field of image annotation.

2-1. Annofab

Annofab is a cloud-based annotation tool that supports images and videos.

 

It supports area extraction (segmentation) that surrounds specific regions within an image, object detection that surrounds specific objects with bounding boxes, and polyline extraction for linear objects (such as vehicle lanes and center lines) that do not require size.

 

In the class settings screen, you can set not only the class values, types, and colors, but also minimum settings (e.g., setting the minimum width and height of a rectangle, etc.) and tolerance ranges (e.g., specifying the allowable distance (in pixels) for the displacement of a rectangle, etc.).

 

One of the strengths of Annofab is its comprehensive functional support in terms of operations, such as shortcut keys. For example, by assigning shortcut keys for each class type, such as 'q' for cars, 'w' for license plates, 'e' for motorcycles, and 'r' for people, users can efficiently work by selecting classes with their left hand on the keyboard and performing annotation operations with their right hand using the mouse.

 

In addition, due to the robust management features, you can search and display annotated data, as well as check statistical data for each annotator, such as the overall progress of the project and the number of times tasks (the smallest unit of annotation work) have been returned.

 

The output format supports JSON.

 

We provide more detailed information about Annofab's basic information, advantages, and disadvantages in another article on our blog, so please take a look.

>> What is the Annotation Tool Annofab?

 

For information about Annofab (official site), click here.

2-2. FastLabel

FastLabel is a cloud-based annotation tool that supports images, videos, text, audio, 3D, automatic annotation, and more.

 

FastLabel's strength lies in its support for a wide range of annotation types and output formats. As mentioned above, it covers all the necessary features for modern annotation, from images and videos to text, audio, and even three-dimensional (3D) data. Additionally, for image annotation, it supports various types such as rectangles, circles, polygons, key points, lines, segmentation, and skeleton estimation, and it supports many formats required for projects, including JSON, COCO, PascalVOC, YOLO, and CSV.

 

In addition, FastLabel operates smoothly, always displaying quickly when loading pages or navigating between menus.

 

Additionally, while it is a paid version, it is a strength that it includes features that streamline tasks such as automatic annotation and smart annotation. Automatic annotation utilizes pre-built AI models to automatically detect cars, pedestrians, license plates, and more, and assign bounding boxes.

 

Additionally, Smart Annotation (only for image segmentation) automatically detects areas based on the color values of the image, allowing for faster annotation in some cases compared to manually detecting areas from scratch.

 

For information about FastLabel, click here.

2-3. Labelme

Labelme is an installed image and video annotation tool.

 

For image annotation, we support rectangles, polygons, circles, lines, and points. There is also a menu to adjust the brightness of the image, which can help in distinguishing annotation areas by increasing the brightness of darker, less clear parts.

 

Additionally, because it is focused on the minimum necessary functions for annotation, each action such as loading images and adding or editing annotations is smooth.

 

One of the advantages is that installation can be started immediately without creating an account or logging in, as it only requires downloading the executable file locally.

 

The output format supports JSON.

2-4. CVAT

CVAT (Computer Vision Annotation Tool) is an installed open-source annotation tool developed and released by Intel Corporation.

 

Supports image and video annotation, with images accommodating rectangles, polygons, lines, points, circles, and cubes, and also features an automatic annotation function. The automatic annotation can be performed on over 80 predefined objects (such as cars, people, airplanes, bicycles, dogs, etc.).

 

Although CVAT does not have a feature to directly return images with deficiencies to the annotator during checks, by recording the URLs of the deficient images in an input field called "Issue Tracker," the annotator can move to the images that have been pointed out via that link and make corrections.

 

Additionally, it is characterized by a wide variety of exportable data formats (such as CVAT, COCO, Datumaro, CamVid, Cityscapes, etc.).

 

For information about CVAT, click here.

2-5. VoTT

VoTT (Visual Object Tagging Tool) is an open-source annotation tool developed by Microsoft that can be installed.

 

Supports image and video annotation, operates smoothly, and features a user interface that is intuitive enough for those without annotation experience to use.

 

VoTT provides installers for Windows, Mac, and Linux, making it easy for anyone to install.

 

However, since it does not have features such as annotator and task progress management or checking functions, it is necessary to manage projects with multiple members using a different method.

 

The output formats supported are Azure Custom Vision service, Microsoft Cognitive Toolkit (CNTK), PascalVOC, TensorFlow records, VoTT JSON, and CSV.

3. Summary

This time, we explained three points to consider when choosing an annotation tool, and introduced five recommended image annotation tools.

 

As the number of annotation tools has increased recently, it is important to choose and utilize the most suitable annotation tool for your company's purposes in order to streamline the time-consuming annotation tasks as much as possible.

 

If you want to reduce the cost of implementing annotation tools, considering outsourcing the annotation itself is also an effective option. Our company offers a wide range of services from consultation on annotation tools to the outsourcing of annotation, so please feel free to reach out to us.

4. Consultations on AI Utilization with Human Science

4-1. Utilizing the latest data annotation tools

One of the annotation tools introduced by Human Science, Annofab, allows customers to receive progress checks and feedback in the cloud even during the project's progress. By ensuring that work data cannot be saved on local machines, we also take security into consideration.

4-2. Achievements in Creating 48 Million Teacher Data Entries

I want to implement AI, but I don't know where to start.

I don't know what to request even if I want to outsource.

In such cases, please feel free to consult Human Science.

At Human Science, we participate in AI development projects across various industries, including natural language processing, medical support, automotive, IT, manufacturing, and construction. To date, we have provided over 48 million high-quality training data through direct transactions with many companies, including GAFAM. We handle a wide range of annotation projects, from small-scale projects to long-term large-scale projects with 150 annotators, regardless of the industry.

>>Human Science Annotation Services

4-3. Resource Management Without Using Crowdsourcing

At Human Science, we do not use crowdsourcing; instead, we proceed with projects using personnel directly contracted by our company. We form teams that can deliver maximum performance based on a solid understanding of each member's practical experience and their evaluations from previous projects.

4-4. Fully equipped security room within the company

At Human Science, we have a security room that meets ISMS standards within our Shinjuku office. Even for highly confidential projects, we provide on-site support. We consider the assurance of confidentiality to be extremely important for any project. Our staff undergoes continuous security training, and we exercise utmost caution in handling information and data, even for remote projects.