Comprehensive Comparison of Open Source Generative AI: Optimal Solutions and Implementation Points for Different Business Scenes

1. Why Open Source LLMs Now?

1-1. Differences from Commercial APIs

Currently, many companies are attempting to utilize generative AI by using commercial APIs such as OpenAI's ChatGPT, Anthropic's Claude, and Google's Gemini. The strength of commercial APIs lies in the ability to instantly access cutting-edge, high-precision models via the cloud. The fact that you can start immediately even with limited development resources is a major attraction.

However, on the other hand, costs tend to escalate due to monthly subscription fees and pay-as-you-go charges accumulating, and there are persistent concerns regarding the handling of input data. Additionally, the high black-box nature of the models makes customization and tuning difficult, which is also a challenge.

Open-source LLMs are attracting attention as an option capable of addressing these challenges. Since they can be operated in a company's local environment or closed network, they offer a high level of security assurance, and there is the freedom to fine-tune and extend the model based on an understanding of its structure. Of course, compared to commercial APIs, there may be differences in the generation quality of pre-trained models, but by specializing for business use, sufficiently practical results can be expected.

Reference Blog:
The Importance of Security in Medical AI Development

1-2. The Rise of Open Source LLMs

In the past 1 to 2 years, a variety of high-performance open source LLMs have emerged one after another, led by Meta's LLaMA series, as well as Mistral, Google's Gemma, Nous Research's OpenHermes, and others. These models have dramatically improved the balance of inference speed, accuracy, and lightweight design, making integration into corporate in-house systems a realistic option.

Additionally, the development of open-source toolsets such as LangChain and LLM-Toolkit has greatly facilitated not only standalone models but also the construction of chatbots specialized for specific tasks and document generation pipelines. This has created a strong tailwind, enabling non-technical on-site personnel to easily integrate generative AI into their workflows.

2. Comparison of Representative Open Source LLMs

Here, we will cover representative open-source LLMs expected to be used in practical work, explaining their features and applicable scopes.

Model Name	Developer	Parameter Scale	License	Features / Use Cases
LLaMA 3	Meta	8B / 70B	Commercial Use Allowed	High accuracy and large scale, highly customizable. Strong in handling long texts.
Mistral / Mixtral	Mistral AI	7B / MoE Architecture	Apache 2.0	Fast inference and lightweight. Suitable for small-scale resource environments.
Gemma	Google	2B / 7B	Commercial Use Allowed	Lightweight model. Suitable for experimental use and local operation.
OpenHermes	Nous Research	7B	Commercial Use Allowed	Strengths in script processing and multi-agent support.
Japanese-LLaMA	Such as rinna	Such as 13B	Many are not available for commercial use	Japanese-specialized model. High accuracy for Japanese business use.

2-1. Features of each model and key points for introduction

・LLaMA 3 (Meta)
A large-scale, high-precision model, particularly suited for long text processing and complex tasks. Highly customizable, it can also be fine-tuned with business-specific data. However, it requires high computational resources, so preparing the environment for using generative AI is also important when introducing it.

・Mistral / Mixtra
*Featuring a lightweight and fast inference due to the MoE (Mixture of Experts) architecture. It can be operated on small-scale hardware and is suitable for marketing areas and customer support where speed is prioritized. Commercial use is also secure under the Apache 2.0 license.

・Google Gemma
This model is built using the same technology as Google's Gemini and can be considered a lightweight version of Gemini. It is a small to medium-scale model that can operate in local environments, making it suitable for research and development.

・OpenHermes (Nous Research)
It excels in script processing and multi-agent implementation, making it suitable for business automation that divides and processes complex instructions. It is well-suited for legal affairs and complex document processing, and commercial use is also anticipated.

・Japanese-LLaMA (rinna, etc.)
As a Japanese-specialized model, it is well-versed in Japanese grammar and expressions. Its use is increasing in fields requiring Japanese language expertise, such as medical and financial sectors, and high-quality Japanese text generation can be expected. However, some parts have restrictions on commercial use, so license verification is necessary.

*MoE (Mixture of Experts) is a mechanism that selects and uses only the necessary sub-models from multiple ones, enabling efficient use of computational resources.

2-2. Precautions When Selecting

When selecting open-source LLMs, it is important to consider not only simple performance comparisons but also the following perspectives.

・Supported Languages and Domain Specialization
The capability to handle Japanese and specific industries (such as medical and legal) varies depending on the model. Choose one that fits your company's use case.

・Model Size and Required Resources
Large-scale models offer high accuracy but require significant GPU and memory resources to operate. If you have local deployment or cost constraints, lightweight models are also an option.

・License Terms
There may be restrictions on commercial use and redistribution. Be sure to check the license before implementation.

・Customizability
Fine-tuning and prompt adjustments are necessary to optimize for business use. Models that can be easily customized are convenient.

・Community and Documentation
The presence of an active development community and the richness of official documentation provide significant support during implementation and operation. This is especially important when handling it for the first time.

In this way, the key is to select the model that best fits your company's needs from among the diverse open-source LLMs and proceed with its utilization.

3. Recommended Generative AI and Use Cases by Business Scene

3-1. Marketing and Public Relations

In the field of marketing, using generative AI dramatically improves the speed and quality from campaign planning to execution. It also enables the rapid creation of multiple personalized versions of SNS posts and email newsletters, allowing for effective appeals tailored to different customer targets. It can generate a large number of ideas in a short time, such as drafts for advertising copy and SEO articles, making it useful for time-consuming brainstorming sessions.

Furthermore, the automation of draft generation for multilingual translation tasks enables global companies to disseminate information overseas more swiftly than ever before. For such applications, the lightweight and fast inference capabilities of Mistral-based models are especially suitable.

3-2. Manufacturing and Technical Development

In the fields of manufacturing and technical development, generative AI supports document creation that was traditionally done manually by engineers, enabling greater efficiency. For example, by automatically generating base texts for troubleshooting procedures and work reports, engineers can focus on reviewing and adjusting the content. Additionally, from vast amounts of data such as manufacturing logs and sensor data, AI automatically creates summaries and explanatory texts, making it easier to report abnormal trends and operational status clearly.

Furthermore, based on inquiries from employees and knowledge information, AI can automatically create and update internal FAQs, thereby improving the quality of knowledge sharing. In this area, the accuracy-focused LLaMA 3 is highly recommended as it is also strong in handling long texts.

3-3. Legal Affairs and Contract Management

In the legal department, generative AI can be utilized for drafting contracts, extracting summaries of key points, and reporting patent investigation results. In particular, support for reviewing complex legal documents helps alleviate labor shortages and shorten review times. While the generated results require expert verification to ensure reliability, they contribute to streamlining the initial stages of legal review. Operating open-source models internally also helps reduce the risk of information leaks. OpenHermes' ability to organize logical structures is well suited to legal documents.

3-4. Accounting and Corporate Planning

Based on financial statements and management indicators, generative AI automatically generates summary comments and explanatory texts. This allows personnel to significantly reduce the time required to draft reporting materials. Operations that structure data from financial statements and accounting documents to assist in analysis and the creation of audit comments are also gaining attention.

Efficiency improvements through tool integration are effective, especially centered on the Mistral model.

3-5. Medical and Healthcare

In the medical industry, efforts are underway to utilize AI for generating summaries of medical records and clinical notes, as well as creating easy-to-understand explanations of papers and guidelines. Including the automatic creation of FAQs for pharmaceutical information, high expertise is required, so it is desirable to use Japanese-specialized models or generative AI trained for the medical field.

LLaMA 3 is versatile but high-performance, so by enhancing its specialization through fine-tuning and RAG, and conducting expert reviews, it can also be utilized to support medical operations.

4. Operational Design for Business Implementation

The introduction of open-source LLMs into companies is not an end in itself; rather, operational design to continuously maximize business outcomes is crucial. Below, we detail the key points from the start of operation through stable functioning and improvement.

4-1. Organizing the Implementation Steps

Business Selection and Use Case Definition
Identify tasks that are highly likely to benefit from AI utilization, focusing on personalized tasks and repetitive routine tasks. Utilize on-site interviews and business analysis to formulate concrete application scenarios.

Model Selection and PoC
Evaluate candidate models using actual data such as internal documents, assessing generation accuracy, responsiveness, and operational costs. Incorporate feedback from the field to decide on the appropriate model and operational policy.

Establishing the Operational Framework
We will develop usage rules, review systems, and trouble response flows in collaboration with the business, IT, and security departments. It is also important to implement mechanisms that minimize the risks of misinformation and information leakage.

Gradual Scale-Up
Start by limiting the operation to a specific group within the organization, then expand while analyzing results and challenges. Promote adoption and acceleration by providing training to employees who will use the system and sharing success stories.

4-2. Evaluation and Improvement During Operation

Regular Performance and Quality Evaluation
Continuously check the output accuracy and quality of the AI model, and grasp the occurrence of errors and misinterpretations. This enables early detection of issues arising during operation and the implementation of corrective measures.

Strengthening the Human-in-the-Loop System
Establish a system where humans review and correct AI-generated content to improve quality and reduce risks. By enabling collaboration between humans and AI, safe and highly reliable operations are achieved.

Enhanced User Training and Support
We strengthen support systems such as user education, preparation of operation manuals, and inquiry response to promote the effective use of AI tools.

Compliance with Laws and Regulations and Compliance Management
We ensure thorough operation based on internal rules and ethical standards while complying with laws and regulations, striving to reduce legal risks and reputational risks.

Security and Privacy Management
When handling data that includes personal or confidential information, security measures to prevent information leaks and unauthorized access are essential. We establish a secure operational foundation for generative AI through encryption, access restrictions, and auditing of usage logs.

4-3. Evaluation and Improvement During Operation

Infrastructure Configuration Selection (On-Premises/Cloud/Hybrid)
Select either an on-premises environment, a cloud environment, or a hybrid configuration combining both, according to your company's security requirements and operational policies.

Optimization of Inference Environment (Utilizing GPUs and Introducing Model Compression Techniques)
We utilize hardware suitable for model inference (such as GPUs) to enhance computational efficiency, and introduce model compression techniques aiming for fast and cost-effective operation.

Continuous Management of Operating Costs and ROI Evaluation
Regularly review the running costs after implementation and evaluate the return on investment (ROI) to suppress unnecessary expenditures and maintain a sustainable operational system.

Strengthening Security Measures (Access Control, Log Monitoring, etc.)
Implement access permission management and usage log monitoring for the system to reduce the risks of information leakage and unauthorized use.

4-4. Best Practices for Maximizing Outcomes

Clarifying Objectives and Setting KPIs
By clearly defining the purpose of implementing generative AI and setting specific KPIs (Key Performance Indicators) to measure outcomes, effective operation and improvement cycles can be achieved.

Agile Phased Implementation and Feedback Loop Construction
Starting with a small-scale pilot implementation, we aim to gradually expand while incorporating feedback on usage conditions and issues, striving to build an optimal system that meets on-site needs.

Establishing an Internal Knowledge Sharing System
By sharing insights and success stories of AI utilization within the organization and creating a system that promotes horizontal deployment and skill enhancement, we aim to raise the overall level of AI adoption across the company.

5. Summary

In the business implementation of generative AI, what truly matters is "after the introduction." As we have seen so far, the operational phase requires continuous efforts from various perspectives, including performance evaluation, user support, risk management, cost optimization, and the establishment of organizational improvement loops.

Moreover, generative AI is not a static system that ends once constructed. The model's behavior and output continuously change according to usage conditions such as the frequency and trends of data input. Therefore, it is important to view AI operation not merely as "system maintenance" but rather as an "evolutionary process of learning and improving while in use" that the entire company should engage in.

We are entering an era where every decision and design in the introduction and operation of generative AI impacts user experience, operational efficiency, and even brand value. Conversely, by designing and operating these correctly, it becomes possible to create value that goes beyond traditional business processes.

Generative AI is not merely a tool for improving operational efficiency; it holds the potential to fundamentally transform the very nature of business. To fully unlock this potential, it is important to approach it from a comprehensive perspective that includes not only the "ways of using" it in daily operations but also the "ways of nurturing" it, encompassing mechanisms for long-term growth and preparations for risks.

6. Human Science Teacher Data Creation, LLM RAG Data Structuring Agency Service

Extensive Track Record of Creating 48 Million Pieces of Training Data
At Human Science, we participate in AI model development projects across a wide range of industries, starting with natural language processing and extending to medical support, automotive, IT, manufacturing, and construction. Through direct business with many companies, including GAFAM, we have provided over 48 million pieces of high-quality training data. From small-scale projects with just a few members to large, long-term projects with a team of 150 annotators, we handle various types of training data creation, data labeling, and data structuring regardless of industry.

Resource Management Without Using Crowdsourcing
At Human Science, we do not utilize crowdsourcing; instead, we advance projects with personnel directly contracted by our company. We carefully assess each member's practical experience and their evaluations from previous projects to form a team that can deliver maximum performance.

Not Only Creating Training Data but Also Supporting Generative AI LLM Dataset Creation and Structuring
We support not only labeling for data organization and the creation of training data for identification-based AI but also the structuring of document data for generative AI and LLM RAG construction. Since our founding, we have been engaged in manual production as a primary business and service, and we provide optimal solutions leveraging our unique expertise and deep knowledge of various document structures.

Equipped with an In-House Security Room
At Human Science, we have a security room within our Shinjuku office that meets ISMS standards. Therefore, we can guarantee security even for projects handling highly confidential data. We consider ensuring confidentiality to be extremely important in every project. Even for remote projects, our information security management system has received high praise from clients, as we not only implement hardware measures but also continuously provide security training to our personnel.

In-House Support
Our company also provides personnel dispatch services for annotation-experienced staff and project managers who match the customer's tasks and situation. It is also possible to organize a team under the customer's on-site supervision. Additionally, we support the training of your workers and project managers, selection of tools tailored to your situation, automation, work methods, and the construction of optimal processes to improve quality and productivity, assisting with any challenges you face related to annotation and data labeling.