What is segmentation? What can be achieved by utilizing AI segmentation?

1. What is segmentation?

Segmentation refers to the meaning of "division or classification" and involves identifying the subjects captured in an image and determining which class they are divided into at the pixel level.

 

 

As shown in the figure above, semantic segmentation allows for a more visual understanding and detailed grasp of the shapes of objects compared to image classification and object detection. Additionally, since labeling is done at the pixel level of the image, it can be said to be optimal for AI models that require higher density and more detailed predictions than image classification and object detection.

However, unlike the rectangles (bounding boxes) used for training data in object detection, segmentation annotation tasks are generally more difficult and are characterized by taking more time to complete.

There are three types of segmentation methods: semantic segmentation, instance segmentation, and panoptic segmentation.

1-1. Semantic Segmentation

Semantic segmentation classifies all objects in an image at the pixel level. For example, classes such as "road," "car," "person," "sky," and "building" are defined and classified at the pixel level. Even if multiple "cars" are present, they are all classified as "car" without distinguishing them as "car1," "car2," etc. Since semantic segmentation classifies every pixel in the image, it is also suitable for recognizing spaces such as the background "sky" or "floor."

 

*A class is the name given to each target that you want to detect using AI.

1-2. Instance Segmentation

Instance segmentation is a method that classifies subjects of the same class within an image, in addition to the techniques of semantic segmentation. In annotation, when multiple objects of the same class are present in an image, they are classified separately for each object. For example, if multiple cars are present, they are assigned IDs such as "Car 1" and "Car 2" to enable the AI to identify each one. Typically, instance segmentation only classifies unique objects that need to be identified, and does not classify unnecessary areas such as the background "sky" or "floor."

1-3. Panoptic Segmentation

Panoptic segmentation is a method that combines semantic segmentation and instance segmentation. It classifies every pixel in the image into class categories (semantic) and also classifies all classes individually (instance). In other words, panoptic segmentation allows for the individual classification of objects within the same class, as well as pixel-level classification of the entire image, including backgrounds. The figure below compares the differences among these three types of segmentation.

 

 

(a) Original image photo
(b) Image with semantic segmentation applied. The entire image is classified into "vehicle group", "road surface", "sidewalk", "building", and "background".
(c) Image with instance segmentation applied. Cars are targeted and classified individually.
(d) Panoptic segmentation. All subjects are classified, and individual entities are also classified.

2. Segmentation Use Cases

As mentioned in the previous section, segmentation involves labeling objects within an image at the pixel level. Therefore, it is generally used in AI models that require more accurate and detailed detection than image classification or object detection. In other words, while object detection can identify the target object, it cannot accurately capture its shape or outline. This is why AI models and products that require contour and shape judgment utilize segmentation.

Here, I would like to introduce examples of the application of AI models using deep learning for segmentation services.

● Case 1: Inspection of Infrastructure Equipment

This is an example of the use of electric infrastructure and the telecommunications industry. AI can extract areas from images taken by drones of equipment such as transmission towers, identifying issues like paint peeling and rust, as well as the degree of rust progression. Tasks that previously required humans to climb the towers and perform visual inspections can now be carried out more safely, reducing labor time and enabling efficient maintenance inspections.
>>Inspection of transmission towers with drones and AI

● Case 2: Medical Support (Detection of Lesions)

AI is being utilized to improve the efficiency of diagnostic operations. Based on images from X-rays, ultrasounds, CT scans, and MRIs, AI detects areas that appear to have lesions or tumors. This technology supports diagnoses that previously relied solely on the experience of physicians, significantly reducing the medical burden caused by physician shortages, time taken for diagnoses, and diagnostic errors. Currently, it is applied to the detection of various human organs and lesions, enabling high-precision real-time diagnostics.
>> Artificial Intelligence (AI) supporting image diagnostics reduces the burden on physicians

● Case 3: Autonomous Driving

In autonomous driving, it is necessary for AI to accurately identify elements related to driving, such as traffic lights, oncoming vehicles, pedestrians, and lanes, from surrounding objects. Segmentation allows for the division of surrounding images on a pixel-by-pixel basis, enabling precise processing. To achieve this, annotations for processing objects and road signs around the vehicle must be conducted based on appropriate rules. As a result, the recognition accuracy of AI improves, making safe driving possible.
>>Driverless autonomous driving "Level 4" runs through the city

3. Summary

This time, we have discussed segmentation, its types, and use cases. These segmentations are currently applied in various fields of AI, and in the future, the areas of application will further expand, becoming more integrated into human life. To prove this, the AI development requests we receive for our annotation services come from a truly diverse range of companies.

This is just one example, but in addition to the previously mentioned use case, we would like to introduce the AI image recognition technology for which our company provided segmentation annotation services below.

Case Studies

https://www.science.co.jp/annotation/experience/index.html

Industry-Specific Examples

● Medical Industry: Surgical Assistance, Diagnostic Support (Object Detection)

　 https://www.science.co.jp/annotation/industry/medical.html

● Automotive Industry: Autonomous Driving Project 2D/3D (Object Detection)

　 https://www.science.co.jp/annotation/industry/automobile.html

● IT Industry: Automatic Invoice Recognition (Optical Character Recognition)

　 https://www.science.co.jp/annotation/industry/it.html

 

As explained in the previous section, various types of segmentation are optimal for AI models that require high accuracy and detailed predictions. However, the difficulty of annotation work is also high, and it generally takes a considerable amount of time, leading to increased costs. If you cannot prepare personnel in-house or want to keep costs down, considering outsourcing or delegating the segmentation annotation work to experts is an effective option. Our company offers a wide range of support, from consultation on annotation to assistance in formulating annotation specifications, creating specification documents, and proposing annotation tools, so please feel free to reach out to us.

4. For inquiries about AI utilization, contact Human Science

4-1. Utilizing the latest data annotation tools

One of the annotation tools introduced by Human Science, Annofab, allows customers to receive progress checks and feedback in the cloud even during the project's progress. By ensuring that work data cannot be saved on local machines, we also take security into consideration.

4-2. Achievements in Creating 48 Million Teacher Data Entries

I want to implement AI, but I don't know where to start.

I don't know what to request even if I want to outsource.

In such cases, please feel free to consult Human Science.

Human Science participates in AI development projects across various industries, including natural language processing, medical support, automotive, IT, manufacturing, and construction. To date, we have provided over 48 million high-quality training data through direct transactions with many companies, including GAFAM. We handle a wide range of annotation projects, from small-scale projects to long-term large-scale projects with 150 annotators, regardless of the industry.
>>Human Science Annotation Services

4-3. Resource Management Without Using Crowdsourcing

At Human Science, we do not use crowdsourcing; instead, we proceed with projects using personnel directly contracted by our company. We form teams that can deliver maximum performance based on a solid understanding of each member's practical experience and their evaluations from previous projects.

4-4. Fully equipped security room within the company

At Human Science, we have a security room that meets ISMS standards within our Shinjuku office. This allows us to handle even highly confidential projects on-site while ensuring security. We consider the protection of confidentiality to be extremely important for all projects. Our staff undergoes continuous security training, and we exercise the utmost caution in handling information and data, even for remote projects.