Helpful for Generative AI and LLM Development! Data Curation Case Studies and Keys to Success

1. What is Curation

Curation refers to the process of collecting and selecting data or content that fits a specific purpose from a vast amount of information, then organizing and providing it in an easy-to-understand manner. It is characterized not merely by the aggregation of information but by offering information that has been carefully chosen based on reliability and usefulness, delivering value to the user.

For example, in the construction of generative AI/LLM models, curation helps contribute to improving the quality of AI/LLM by selecting highly reliable information and ensuring data consistency and accuracy.

The blog article below provides an easy-to-understand explanation of the curation tasks and their importance in generative AI/LLM model construction, so please take a look.
What is curation in generative AI/LLM model construction? An easy-to-understand explanation of tasks and importance

2. Main Fields Where Curation is Utilized

Curation has become an indispensable process in various advanced fields, including AI and machine learning. Especially in development environments handling large volumes of data, the removal of unnecessary noise and the extraction and organization of high-quality data directly impact outcomes, thereby increasing the importance of curation.

Here, we introduce examples of representative business fields where curation is utilized.

2-1. AI Image Generation / Image Classification Model Development

In the development of AI models using images, a large amount of image data is required; however, this data may include low-quality or contextually inappropriate images. Curation work involves removing such noise and assigning accurate label information, thereby improving the model's reliability and accuracy.

2-2. Recommendation and Search Engines Using AI

The quality of the original information is extremely important to present appropriate content to users. Through curation, selecting information that matches the user's intent and excluding unreliable content can improve recommendation accuracy and the quality of the search experience.

2-3. Voice AI

In AI development for speech recognition and synthesis, selecting and adjusting high-quality voice data is indispensable. Curation plays a role in stabilizing AI model performance by balancing the content of utterances and removing samples with noisy recordings or recognition errors.

2-4. LLM Model Development

In the development of large language models (LLMs), it is required to extract and organize high-quality linguistic expressions from vast amounts of text data. Curation is utilized in tasks such as correcting mistranslations and unnatural expressions, as well as filtering out unnecessary content, targeting documents in Japanese and English.

In this way, curation is being introduced in many fields as an important process that supports the quality of various AI developments.

3. Representative Case Studies at Human Science

Human Science handles numerous curation tasks across various industries and development phases. Here, we introduce an example of an LLM development project we have actually worked on.

[Case Study] Parallel Data Evaluation Project for Improving LLM Accuracy
We received parallel data consisting of mechanically collected English texts and Japanese translations generated by an LLM from a client, and conducted curation work to determine and organize whether the data met a certain quality level.

■ Issues and Needs
The provided English source text was mechanically collected, and the Japanese translation was created by an LLM. The content included highly specialized fields such as medical, scientific, and financial areas. Therefore, it was necessary not only to verify the accuracy of the English text and the accuracy and naturalness of the Japanese translation but also to check the consistency of the Q&A content with the main text, requiring specialized background knowledge and expertise.

■ Tasks
・Evaluation of bilingual data (classification into usable/unusable)
・Assessment of the accuracy, naturalness, and clarity of the translations
・Verification of whether the question and answer pairs correspond to the content of the text

■ Points of ingenuity
・English-Japanese translators well-versed in each field handled content requiring high English and Japanese proficiency as well as expertise
・Secured abundant resources capable of processing large volumes of data in a short period
・Experienced project managers flexibly proposed and handled everything from system setup to work design
・Collaborated with the in-house technical team to develop tools for efficiency

In this way, Human Science leverages its expertise and abundant resources to flexibly respond to curation needs in LLM development.

4. Common Points Seen from Curation Case Studies

As introduced so far, curation plays an important role in AI/LLM development and large-scale data processing, and its results greatly affect the accuracy and quality of projects. Based on the extensive experience Human Science has accumulated through many projects, we will introduce three key points that we consider especially important for successful data curation.

1. Designing Evaluation Criteria According to the Purpose
The purpose of curation varies depending on the project. For example, whether the focus is on translation quality, the consistency of QA data, or giving equal importance to both, the evaluation criteria will differ significantly. Defining clear evaluation axes that match the purpose before starting the work leads to efficient and effective curation.

2. Securing Personnel with Expertise
In fields with particularly high specialization (such as medical, scientific, and financial sectors), there are many cases where general standards alone are insufficient for judgment. Therefore, involving experts with knowledge in the relevant field in the curation process enables more accurate and reliable data preparation.

3. Flexible Team Structure and Progress Management
To process large volumes of data in a short period, flexible resource adjustment and careful progress management are essential. Maintaining the schedule and quality of deliverables requires the presence of an experienced project manager and a team structure capable of responding smoothly, which are the keys to success.

At Human Science, based on these points, we propose the optimal curation system tailored to your challenges and objectives.

If you face any challenges related to AI/LLM development or data preparation in the future, please feel free to consult us. With our expertise and flexible system, we will firmly support your projects.

5. Summary: Request for Curation to Human Science

Over 48 million pieces of training data created

At Human Science, we are involved in AI model development projects across various industries, starting with natural language processing, including medical support, automotive, IT, manufacturing, and construction. Through direct transactions with many companies, including GAFAM, we have provided over 48 million high-quality training data. We handle a wide range of training data creation, data labeling, and data structuring, from small-scale projects to long-term large projects with a team of 150 annotators, regardless of the industry.

Resource management without crowdsourcing

At Human Science, we do not use crowdsourcing. Instead, projects are handled by personnel who are contracted with us directly. Based on a solid understanding of each member's practical experience and their evaluations from previous projects, we form teams that can deliver maximum performance.

Supports not only curation and annotation but also the creation and structuring of generative AI LLM datasets

In addition to labeling for data organization and annotation for identification-based AI systems, Human Science also supports the structuring of document data for generative AI and LLM RAG construction. Since our founding, our primary business has been in manual production, and we can leverage our deep knowledge of various document structures to provide you with optimal solutions.

Secure room available on-site

Within our Shinjuku office at Human Science, we have secure rooms that meet ISMS standards. Therefore, we can guarantee security, even for projects that include highly confidential data. We consider the preservation of confidentiality to be extremely important for all projects. When working remotely as well, our information security management system has received high praise from clients, because not only do we implement hardware measures, we continuously provide security training to our personnel.