[Spin-off] Visual Inspection on Manufacturing Sites Shifts to AI Inspection ~ Annotation Manager from Manufacturing Industry Talks about "Challenges and Solutions in AI Implementation" ~ | AI & Annotation Blog

Spin-off Blog Project
— The Annotation Supporting AI in the DX Era. The Realities of That Analog Workplace
From Visual Inspection on the Manufacturing Floor to AI Inspection
— An Annotation Manager from Manufacturing Talks About "Challenges and Solutions in AI Implementation" —

Table of Contents

1. Japan's Manufacturing Industry and DX at a Major Turning Point
2. The Struggles and Thoughts When We Had No Choice but to Rely on Human Visual Inspection (Author's Experience)
3. If Only We Had AI Like Today Back Then...
4. The Challenges of AI-Based Visual Inspection
5. Three Essential Elements for On-Site AI Implementation (On-Site Understanding × AI × Data)
6. Providing "Manufacturing Site-Oriented Services" That Only Manufacturing Companies Can Offer
7. Features of Human Science Annotation Services

Until now, our company has been publishing various blogs related to annotation and AI. In those, we have mainly conveyed general information, knowledge, and know-how.

At first glance, annotation may seem simple when put into words, but because it is a task that inevitably involves humans and contains a lot of "ambiguity," it inevitably requires significant human interaction. For this reason, in a sense, it is a gritty process that often cannot be resolved by the neat logic commonly found elsewhere. To ensure quality and productivity, a variety of experience and know-how, as well as personnel capable of executing them, are actually necessary.

What specific responses and measures are being taken at our site? Unlike typical blogs, we want to convey the real situation on the ground, including our unique characteristics, thoughts, and commitments.

1. Manufacturing Industry and DX in a Major Turning Point

Products produced by Japanese manufacturing have been praised as Japanese quality and have dominated the world for over 30 years. Currently, every industry faces various challenges. These include labor shortages, skill transfer issues due to aging, intensified global competition, supply chain vulnerabilities, adapting to diverse and changing customer needs, rising raw material costs, and delays in DX. The list of challenges is so extensive that it could fill an entire column just by enumerating and explaining them. However, in the DX of manufacturing, advances in AI technology have shown great potential for solutions, and steady progress is being made step by step.

In particular, the automation of visual inspections and reduction of man-hours through advances in AI image processing technology have been realized relatively early in the recent AI boom. Initially centered around large companies, this trend has now begun to penetrate small and medium-sized enterprises, and the momentum is accelerating.

This time, I would like to explain the difficulties of manual visual inspection and the key points in the development and implementation of AI visual inspection, incorporating my own real-life experiences.

2. The Struggles and Thoughts When We Had No Choice but to Rely on Human Visual Inspection (Author's Experience)

This goes back about 20 years, but I once worked in the quality assurance department of a Tier 1 automotive parts manufacturer, where I was responsible for quality guidance and improvements on the production line. One production line I took over had various problems and challenges from the start. It was manufacturing parts for a core vehicle model of a certain automaker, and the quality required for those products was at a level distinctly higher than other products, with a target annual defect rate for deliveries to the manufacturer of just a few ppm or less.*

Not only for dimensions and specifications but also for appearance defects such as scratches and dents, the required quality level was no exception. Although the site took the lead in repeatedly making improvements to suppress the occurrence of defects, a certain number of defects were still unavoidable. In conclusion, to create a manufacturing process less prone to defects, it was necessary to completely replace equipment and review the processes and methods from scratch. At that time, not only was there no budget for this, but even if there had been, other methods would have resulted in poor productivity and would not have been profitable. Therefore, we had no choice but to rely on appearance inspections to suppress the "outflow of defects".

At that time, the visual inspection also had to rely solely on the inspectors' eyesight and skills. To minimize oversights, we implemented every possible measure such as "clarifying and simplifying inspection points," "improving inspection environments like lighting," "setting inspection times that maintain concentration," "rotating inspectors' shifts," and "conducting regular detection ability tests" to ensure inspectors could maintain focus and detection capability. Even inspectors capable of detecting scratches and dents that I myself could not find through careful examination still experienced several customer complaints per year due to oversights.

Once a complaint occurs, countermeasures must be implemented. To do so, interviews and investigations with inspectors are conducted to understand the current situation, but inspectors inevitably feel a sense of responsibility. Furthermore, in order to improve detection capability beyond the current level, measures such as “division of inspection points” and triple checks were implemented in addition to double checks, resulting in the number of inspectors swelling from one initially to three or four at peak times. This not only greatly deviated from the fundamental principle of quality control to eliminate defects at the source, but also led to a significant increase in inspection costs and the psychological burden on inspectors. I vividly recall my feelings of helplessness and frustration at having fallen into such a situation without any means to resolve it.
*ppm: A unit representing “one part per million.” It is used to indicate very small concentrations or proportions.

3. If only we had AI like we do now back then...

At that time, the person in overall charge of the site was also quite aware of the problems with relying on manual visual inspections, and we decided to introduce and consider equipment that would utilize the image processing and sensor technologies available then to reduce human burden and labor.

I consulted equipment manufacturers, inspection device manufacturers, and trading companies handling these products, but at every consultation, the consensus was that with the technology at the time, including changes to line equipment and processes, "even if we invested hundreds of millions, it was doubtful whether we could achieve the expected effects," so I had no choice but to give up helplessly. I remember that, in the end, with no effective means available, time just passed as we continued relying on manual visual inspections.

After various twists and turns, I came to be in charge of promoting the AI annotation business at our company. Shortly after starting the business, I came across an article titled "Automotive parts manufacturer succeeds in automating scratch and dent visual inspection using AI image recognition technology!" This brought back vivid memories from that time and stirred my emotions. At the same time, I was overwhelmed by an impulsive and fanciful desire to go back to that site—whose current existence I don’t even know—and tell them about the AI technology we have now, thinking, "If only we had AI like this back then..." However, recently, many commercially available inspection devices and applications equipped with AI have become widespread, and visual inspection using AI image recognition technology is gradually becoming common in the manufacturing industry.

4. Challenges in Introducing AI Visual Inspection

For the benefits and specific examples of AI-based visual inspection, please refer to our previous blogs and other materials. However, simply introducing AI does not automatically lead to automation, labor reduction, or solving various inspection-related issues. Even with advances in AI technology, there are many hurdles in manufacturing sites. Looking back calmly at my past experience, it is not a simple matter, and specific challenges become apparent.
Next, I will pick up and explain one by one the commonly mentioned “challenges in introducing AI visual inspection.”

・Difficulty in Collecting NG Sample Data
In many manufacturing lines, there are often very few NG defective product samples to begin with, or it is difficult to reproduce them. One study pointed out that "it is necessary to first collect a large amount of defect data and labeled images," but this is identified as a hurdle regardless of whether the manufacturer is large or small to medium-sized. Many customers who have consulted us about annotation have also faced the challenge of spending a lot of time collecting NG product samples and NG image data.

・Quality Standards for Annotation and Labeling
The accuracy of AI heavily depends on the quality of the "training data" and the "annotation work" that creates it. In most manufacturing sites, there are documents that serve as the "standards" for what constitutes an NG (defective) product, and inspections are conducted based on these. However, it is necessary to properly pass on and communicate these standards to the AI implementation and design personnel as well as the annotation workers.
Also, even if standards exist, there may be some tacit knowledge related to NG/OK judgments that cannot be verbalized. It is important to clarify these and reflect them in the annotation work during handover. In addition to properly carrying out these steps, if a process to suppress variability among annotation workers is not designed, inconsistencies in annotation accuracy and quality will occur, which will ultimately cause variability in AI accuracy as well.

・Consideration and Understanding of On-site Conditions
If AI design and implementation are carried out without taking into account the various on-site conditions that occur in manufacturing lines and products—such as material, glossiness, shape, and process conditions unique to the product—as well as ease of use in operation, the AI will not function as intended. Unexpected issues such as "poor visibility, reflections, and shadow effects" will cause false detections and misses. This results in a system that differs from what the site required, becoming "useless" or "something made by people who ultimately do not understand the site." Not only does this fail to achieve the purpose of AI implementation, but it also hinders the site's digital transformation (DX) and future AI adoption efforts, causing a loss of trust and cohesion from the site.

・Challenges of In-House Operational Resources
Many of the customers who consult with us are hardware engineers related to the product or staff from manufacturing support departments, and they are often not engineers with specialized knowledge in AI development. In addition, manpower is often insufficient, so when they try to proceed with "data collection, annotation, model building, and operation" in-house, it is often observed that development does not progress smoothly due to the following challenges and issues.
• There is little specialized knowledge about AI, and although they somehow managed to conduct a PoC on their own, they cannot proceed to subsequent phases.
• There is no established system to acquire diverse data, and cooperation from the field side is weak.
• They tried annotation with available internal personnel, but label errors and variations in accuracy occur, resulting in unstable quality; productivity is also poor, and they cannot secure the necessary amount of training data.
• There is not enough manpower to improve the model after operation, so the feedback loop cannot be maintained, causing accuracy to deteriorate.

・Cost and Cost-Effectiveness (ROI)
Not only the initial implementation costs (cameras, lighting, image processing equipment, AI development) but also costs can significantly increase when modifications or renovations to the production line are involved. Therefore, if cost-effectiveness and ROI are not clearly defined, not only will project approval be difficult to obtain, but the project itself tends to stall.
On the other hand, the cost of manual inspection is often buried within the total labor costs of other manufacturing staff and thus hard to see, so it is often neglected. When calculated, these costs can be considerably high. It may seem obvious when pointed out, but one article notes that "manual inspection has relatively low initial costs and can be easily started, making it easy to tackle, but in the long term, it is very costly."

Considering the hurdles mentioned above and recalling my experience with the manufacturing line at that time, it was necessary to have conveyor equipment to align the products and jigs for positioning so that the AI camera could properly capture scratches and dents on the products and workpieces. Also, since the products were plate-shaped and often had scratches and dents on both the front and back sides, it was necessary to decide whether to photograph both sides simultaneously with cameras or to flip the product with equipment and photograph the front and back separately. This required auxiliary equipment for camera shooting and line modifications. Furthermore, if the product is flipped, various considerations are needed, such as whether the slow cycle time of that equipment would become a bottleneck in the manufacturing line.

In addition to positioning these products, there are other issues as well. Since the products have a polished, glossy surface, the brightness and reflectivity of the lighting affect how scratches appear, making it necessary to consider the lighting equipment. Furthermore, considering the diversity of scratches and dents that occur, it is essential to prepare NG samples and image data under various conditions, and cooperation and understanding from the field are indispensable for data collection.
As a result, it is easy to imagine that most of the previously mentioned "hurdles to AI visual inspection implementation" apply here, and that AI introduction is not so simple.

5. Three Essential Elements for On-Site Implementation of AI Visual Inspection (On-Site Understanding × Data × Human Resources)

To summarize, simply automating inspection tasks with AI is not enough to increase the success rate of AI visual inspection implementation in manufacturing. By combining the following three elements, you can reduce the likelihood of failure during implementation and subsequent operation and maintenance.

•Understanding the Site (Grasping the Purpose, Constraints, and Scope of AI Visual Inspection):
To successfully implement AI and ensure smooth on-site deployment, it is necessary to set objectives and goals as well as understand and grasp the situation at the site as follows.
・ Setting the purpose of AI visual inspection such as labor saving and substitution, and target accuracy
・ Defining the scope of application including product types and inspection items subject to AI visual inspection
・ Understanding the processes/methods of the products and manufacturing line, and the mechanisms of occurrence of visual defects
・ Grasping the conditions of inspection equipment/lighting and their impact on visual inspection
・ Considering the site line takt and cycle time*
・ Understanding the actual conditions and status of the on-site manufacturing line

*Line takt (takt time): The allowed time to produce one product
The time during which each work process operates in sync, the time interval at which processed products are sent out from the line
*Cycle time: The actual time taken for one cycle from the start to the completion of a product’s process

• Securing Data (Data Collection and Ensuring Data Accuracy):
It is necessary to acquire NG and OK image data considering the various conditions of the manufacturing and inspection sites, establish inspection criteria such as NG judgment standards required to ensure annotation work accuracy, and to transfer, document, and organize the know-how.

• Human Resources and Structure (Securing personnel for AI development and annotation, cooperation system on-site, driving force, and continuous improvement):
To promote AI introduction and ensure continuous improvement after implementation, it is necessary to have "dedicated AI personnel assigned from development and introduction through maintenance and operation," "a personnel system that can obtain NG samples and stably produce high-quality training data," and "collection of data and on-site feedback." However, all of these require understanding and cooperation from the field.

Only when these three elements come together does the success rate of AI visual inspection implementation increase. For example, from the perspective of "on-site cooperation," it becomes possible to capture image data of defects that occur only once a month and tendencies such as "AI's tendency to produce false detections," enabling the formulation of future AI additional learning directions and appropriate maintenance plans. Conversely, if even one of these is missing, despite implementing AI visual inspection, it may become "useless on-site," ultimately leading to the failure of AI development and implementation.

6. Providing "Manufacturing Site-Oriented Services" That Only Manufacturing Companies Can Offer

We have discussed various points so far, but our company is not part of the manufacturing industry. However, as mentioned in a previous blog, our company started with manual creation and has a manufacturing mindset. We take pride in distinguishing ourselves from other annotation vendors in this regard, and even from my perspective as someone with a manufacturing background, although the industry-specific technologies differ, the manufacturing mindset and fundamentals are not significantly different. The basic approach to activities aimed at satisfying our customers' QCD is also the same. Above all, we deeply understand the struggles and hardships of the manufacturing floor firsthand.

I have had the opportunity to receive consultations regarding annotation and AI from customers across various industries and sectors, but when consulting with customers from the manufacturing industry, I inevitably feel a sense of closeness and find myself becoming somewhat passionate. There is no doubt that my experience in the manufacturing industry, especially in the automotive sector, which nurtured me, has shaped my attitude and the core of my work, and that has become my asset today.

Although it has long been said that Japan’s manufacturing industry, once globally dominant, has lost its momentum, I dare say without fear of misunderstanding that the excellence of the manufacturing industry and, more importantly, the dedication of the people working there to craftsmanship still rival the world. I firmly believe that those supporting manufacturing on the shop floor remain at an advantage.
Our company not only contributes to the DX and improvement of manufacturing clients through the power of data supporting AI, known as "annotation outsourcing," but also has a system in place to support everything from AI development and machine learning model creation to in-house annotation assistance. We aim to be a company that goes beyond mere annotation and continues to solve challenges by closely aligning with "manufacturing, and ultimately the manufacturing floor."

7. Human Science Teacher Data Creation, LLM RAG Data Structuring Outsourcing Service

Over 48 million pieces of training data created

At Human Science, we are involved in AI model development projects across various industries, starting with natural language processing and extending to medical support, automotive, IT, manufacturing, and construction, just to name a few. Through direct business with many companies, including GAFAM, we have provided over 48 million pieces of high-quality training data. No matter the industry, our team of 150 annotators is prepared to accommodate various types of annotation, data labeling, and data structuring, from small-scale projects to big long-term projects.

Resource management without crowdsourcing

At Human Science, we do not use crowdsourcing. Instead, projects are handled by personnel who are contracted with us directly. Based on a solid understanding of each member's practical experience and their evaluations from previous projects, we form teams that can deliver maximum performance.

Support for Generative AI LLM Data Structuring and "Manual Creation and Maintenance Support Optimized for AI"

Since our founding, our main business and service has been manual production. Currently, we also support the creation of "AI-recognizable documents" to facilitate the introduction of generative AI for corporate knowledge utilization. Even when performing RAG and other processes for sharing and utilizing corporate knowledge and documents using generative AI, current technology still cannot achieve 100% accuracy with tools alone. For customers who want to make the most of their past document assets, we also provide document data structuring services. We offer optimal solutions leveraging our unique know-how, gained from extensive familiarity with various types of documents.

Secure room available on-site

Within our Shinjuku office at Human Science, we have secure rooms that meet ISMS standards. Therefore, we can guarantee security, even for projects that include highly confidential data. We consider the preservation of confidentiality to be extremely important for all projects. When working remotely as well, our information security management system has received high praise from clients, because not only do we implement hardware measures, we continuously provide security training to our personnel.

In-house Support

We provide staffing services for annotation-experienced personnel and project managers tailored to your tasks and situation. It is also possible to organize a team stationed at your site. Additionally, we support the training of your operators and project managers, assist in selecting tools suited to your circumstances, and help build optimal processes such as automation and work methods to improve quality and productivity. We are here to support your challenges related to annotation and data labeling.