The Growing Demand for Domain-Specific LLMs and Their Background

Table of Contents

1. Why Use Domain-Specific LLMs Now?
2. What Are General-Purpose LLMs and Domain-Specific LLMs?
3. Why Is the Demand for Domain-Specific LLMs Increasing Now?
4. Use Cases of Domain-Specific LLMs in Japanese Companies
5. Challenges Companies Face When Building Domain-Specific LLMs
6. The Key to Success: "Data Creation and Evaluation System"
7. Future Outlook: The Evolution of LLM Utilization in Japanese Companies
8. Features of Human Science's Annotation Services

1. Why Now "Domain-Specific LLM"?

The use of general-purpose LLMs and generative AI, including ChatGPT, is rapidly advancing. On the other hand, there are increasing voices from companies using them: "Shallow business knowledge," "Lack of understanding of industry customs and technical terms," "Insufficient accuracy and reproducibility for on-site use," "Answers seem plausible but are unusable in business," and so on. These issues are natural because they are general-purpose generative AI and LLMs. Having made a sensational debut by answering anything asked in chat, expectations that they could do everything were high, so this situation can also be seen as a backlash against those expectations. In response to such expectations and needs, domain-specific LLMs optimized for particular industries, specialized fields, or in-house operations are attracting attention. This time, rather than technical details such as specific construction methods of domain-specific LLMs, I would like to focus on use cases and challenges in implementation.

2. What Are General-Purpose LLMs and Domain-Specific LLMs?

So, what exactly are general-purpose LLMs and domain-specific LLMs? General-purpose LLMs each have their own characteristics, strengths, and weaknesses, but they are large-scale language models trained on a wide range of knowledge, such as ChatGPT, Gemini, Claude, and Meta's Llama. They are generally effective for broad uses like brainstorming, summarization, and translation, but they are not optimized for specific tasks or industries (domains). On the other hand, domain-specific LLMs refer to LLMs optimized for particular domains (industries, tasks, internal operations, etc.), such as healthcare, legal, finance, manufacturing, and customer support. In business settings, general-purpose LLMs like ChatGPT or Gemini excel at acquiring common knowledge that applies across industries, but they tend to be weak in the specialized fields of the industries to which individual companies belong. Using them as-is may not be sufficient for certain purposes within companies. Additionally, their understanding of Japanese business documents is still not fully adequate.

We will not delve much into technical details here, but to build a domain-specific LLM, there are methods such as integrating documents filled with industry or company-specific knowledge in a searchable format and linking them with a general-purpose LLM through a RAG (Retrieval-Augmented Generation) architecture, or fine-tuning a general-purpose LLM by further training it with industry or company-specific knowledge.

3. Why Are Domain-Specific LLMs Attracting Attention?

With the emergence of generative AI/LLMs represented by ChatGPT, which are seen as tools to improve work efficiency and address labor shortages, some have even said, "Companies that cannot master generative AI/LLMs will not survive." Since their debut, adoption by companies has rapidly progressed. How to effectively utilize generative AI/LLMs is an unavoidable challenge for most companies, but it is also true that simply introducing LLMs often results in their effectiveness being more limited than expected. This is especially true in specialized fields, where issues such as distrust and anxiety arise when encountering inaccuracies or hallucinations in responses, as well as serious concerns about accountability when such problems occur.

Recently, it has become a common understanding that simply introducing an LLM results in only limited usability. As a result, so-called "domain-specific LLMs," which are specialized based on unique industries, expert fields, and a company's own business knowledge, have begun to attract attention as a means to solve and realize the originally expected challenges of generative AI/LLMs, such as improving operational efficiency, addressing labor shortages, and resolving tacit knowledge (knowledge management).

4. Use Cases of Domain-Specific LLMs

Although many are still at the experimental stage, cases of building and operating domain-specific LLMs that reflect in-house business and industry knowledge have begun to appear, moving beyond the stage of simply using general-purpose LLMs as they are. Below, we will look at actual domestic examples that have been made public, examining their characteristics and the expected effects.

4-1. Manufacturing Industry: Utilization of LLMs Specialized in Technical Documents and Quality Information

In Japan's manufacturing industry, long-standing issues such as dependence on the tacit knowledge of veteran engineers and a shortage of young talent have been pointed out. Additionally, there is an enormous volume of documents containing corporate know-how, such as technical documents, procedure manuals, and defect reports; however, to varying degrees, no company can be said to fully leverage these documents. To address these challenges, there is a growing movement to utilize LLMs specialized in the company's own technical documents, quality documents, and data for purposes such as "searching past trouble cases on the manufacturing floor," "natural language search of work procedures and design information," "FAQ creation for technical information," and "equipment maintenance." These efforts are expected to support the early resolution of problems faced by less experienced engineers with limited know-how.

▼NEC Develops a Knowledge Transfer Support System for Manufacturing Sites Utilizing LLM (External Link)
NEC Develops a Knowledge Transfer Support System for Manufacturing Sites Utilizing LLM

4-2. Finance and Insurance: LLM Specialized in Internal Regulations and Policy Terms

In the finance and insurance industries, regulations, policy terms, and manuals are frequently updated, and issues such as the personalization of inquiry responses are common. Additionally, the nature of the work does not allow for incorrect answers. Against this backdrop, a particularly cautious approach is required for the use of generative AI. Among these, domain-specific LLMs targeting internal regulations, policy terms, and FAQs—such as inquiry response (operator support), regulation search and summarization, and administrative procedures—are attracting attention. The use of these is expected to reduce "variability in responses," "standardize response quality," and "improve operational speed."

▼Development of a Business-Specialized LLM for Advanced Inquiry Response Functions (NEC x Mitsui Sumitomo Insurance)
▼Joint Development of AI to Promote Efficiency in Insurance Contract Operations (Ricoh x Sompo Japan Insurance)
▼Launch of a Service to Build Company-Specific Models Using a Finance-Specialized PLaMo Trained on Japanese Financial Knowledge (Preferred Networks)

4-3. Back Office (Legal): Japanese Document-Specialized LLM

In the back office (legal) operations of Japanese companies, issues such as "a large volume of contract review work," "dependence on the experience level of personnel," and "increased workload for reviews and inquiry responses" have become apparent. In response, LLMs specialized in Japanese contracts, such as "contract clause search and key point summarization," "regulation check support," and "presentation of contract templates," have begun to be utilized, showing effectiveness in reducing review and inquiry times and alleviating the burden on personnel.

▼Revolutionizing Legal Work with AI! Aoba-BBT’s "AI Legal Assistant Legal"

5. Challenges Companies Face When Building Domain-Specific LLMs

When introducing a domain-specific LLM in-house, various challenges arise such as "no LLM specialists within the company," "inability to create annotation standards like data structuring," "lack of established evaluation methods," "even if attempting in-house development, data exists but documents are in PDF/Excel or paper formats and cannot be used as-is," and "inconsistent formats." Therefore, although companies may proceed tentatively with a PoC to reach a level of "somewhat usable," there are often many documents and data that the LLM cannot properly recognize, resulting in insufficient accuracy and failure to advance to actual implementation.

After all, to make the accumulated internal document data suitable for utilization by LLMs, it is necessary, to varying degrees, to structure the data. Additionally, to shape the document data containing knowledge that will continue to accumulate in a way that LLMs can easily recognize, it is essential to establish rules regarding how documents and texts are written. In either case, specialized expertise, experience, and knowledge are required.

In particular, structuring data requires time and cost proportional to the amount of information assets accumulated so far. Allocating internal personnel to this not only results in enormous costs but also hinders the progress of core business operations, slows down data structuring, and consequently delays the introduction of domain-specific LLMs. Neglecting data will inevitably prevent improvements in the accuracy of FAQ responses using LLMs.

6. The Key to Success is "Building a Collaborative Framework"

As mentioned above, to successfully implement these domain-specific LLMs, specialized knowledge about LLMs, an understanding of industry-specific operations, experience and knowledge in data preparation and structuring, and the ability to address issues unique to the Japanese language are required. Additionally, since internal documents and knowledge continue to grow, it is necessary to consider ongoing operations rather than temporary tasks at the time of implementation, including establishing rules for documenting knowledge and conducting regular evaluations.

If your company can cover that knowledge internally, all the better; however, if not, it is better to collaborate with specialized companies that support the introduction of generative AI and LLMs. While domain-specific expertise can only be handled by each respective company, support for system construction, data structuring, and how to write documents in a way that LLMs can easily recognize are areas that are difficult to address without companies experienced in LLM implementation. Therefore, dividing tasks according to each party’s strengths—between your company and specialized firms—and building a seamless system from system construction to data preparation to evaluation and operation is the shortcut to success.

7. Future Outlook: The Evolution of LLM Utilization in Japanese Companies

Going forward, the demand for domain-specific LLMs will continue to grow in addressing challenges such as labor shortages, the transfer of skills and know-how, and knowledge management, which form the foundation of these issues faced by Japanese companies and industries. It is also anticipated that domain-specific LLMs will become further subdivided to specialize in even more specific fields and industries. However, in building LLMs to solve problems like labor shortages and skill and know-how transfer, the fundamental source is the data fed into the LLM and its quality, which will increasingly become a critical source of competitive advantage for companies.

It may seem obvious, but the products and services created by each company only gain added value when the know-how and insights unique to that company are incorporated. This is what enables uniqueness and differentiation compared to other companies. Therefore, the presence of LLMs as a means to effectively leverage these assets will become increasingly important in the future.

8. Human Science Teacher Data Creation, LLM RAG Data Structuring Outsourcing Service

Over 48 million pieces of training data created

At Human Science, we are involved in AI model development projects across various industries, starting with natural language processing and extending to medical support, automotive, IT, manufacturing, and construction, just to name a few. Through direct business with many companies, including GAFAM, we have provided over 48 million pieces of high-quality training data. No matter the industry, our team of 150 annotators is prepared to accommodate various types of annotation, data labeling, and data structuring, from small-scale projects to big long-term projects.

Resource management without crowdsourcing

At Human Science, we do not use crowdsourcing. Instead, projects are handled by personnel who are contracted with us directly. Based on a solid understanding of each member's practical experience and their evaluations from previous projects, we form teams that can deliver maximum performance.

Generative AI LLM Dataset Creation and Structuring, Also Supporting "Manual Creation and Maintenance Optimized for AI"

In addition to labeling for data organization and creating training data for identification-based AI, we also support structuring document data for generative AI and LLM RAG construction. Since our founding, manual production has been our main business and service, and we now also assist with organizing business knowledge and manual creation aimed at future generative AI and RAG implementation and utilization. We provide optimal solutions leveraging our unique expertise in the structure of various documents.

Secure room available on-site

Within our Shinjuku office at Human Science, we have secure rooms that meet ISMS standards. Therefore, we can guarantee security, even for projects that include highly confidential data. We consider the preservation of confidentiality to be extremely important for all projects. When working remotely as well, our information security management system has received high praise from clients, because not only do we implement hardware measures, we continuously provide security training to our personnel.

In-house Support

We provide staffing services for annotation-experienced personnel and project managers tailored to your tasks and situation. It is also possible to organize a team stationed at your site. Additionally, we support the training of your operators and project managers, assist in selecting tools suited to your circumstances, and help build optimal processes such as automation and work methods to improve quality and productivity. We are here to support your challenges related to annotation and data labeling.