Can annotations be made with ChatGPT?

1. What is ChatGPT?

Writing is difficult, so I asked ChatGPT for an explanation. When I asked ChatGPT, "What is ChatGPT?" I received the following response.

ChatGPT is one of the language models developed by OpenAI. GPT stands for "Generative Pre-trained Transformer" and is a deep learning model used in the field of natural language processing for tasks such as text generation, text classification, text summarization, and dialogue systems. ChatGPT is built using GPT to create a natural language dialogue system, enabling it to engage in natural conversations with users. ChatGPT is one of the technologies gaining attention in the field of artificial intelligence, with various applications expected.

 

That's amazing... It was explained so simply.

So, what can ChatGPT do? I asked ChatGPT about this as well.

ChatGPT is one of the language models used in the field of natural language processing, and it can do various things. Below are some of the representative features of ChatGPT.

Text Generation: ChatGPT can generate natural sentences based on the given text. For example, it can create short articles on specified themes or long stories like novels.

 

Text Classification: ChatGPT can classify the given text into categories. For example, it can determine whether an article is about sports or politics, or whether a movie review is positive or negative.

 

Text Summary: ChatGPT can summarize the given text. It can condense long articles or reports and provide an overview of their content.

 

Dialogue System: ChatGPT is also used as a dialogue system utilizing artificial intelligence. By creating a chatbot using ChatGPT, you can engage in natural conversations with users.

 

As mentioned above, ChatGPT is widely utilized in the field of natural language processing, and various applications are anticipated.

 

It seems that I searched the above text on Google, but it did not yield any results. Therefore, it appears that the text is not copied from somewhere else but is generated properly. ChatGPT can do many other things, but this is sufficient as an overview explanation.

2. Can annotations be made with ChatGPT?

So, can we use ChatGPT for annotation, particularly in text annotation? In addition to what ChatGPT itself has answered about its capabilities, it can actually write program code as well. The ability to write program code means it can also understand tags. If it can understand text and also understand tags, it might be able to perform text annotation as well.

 

Once again, I asked ChatGPT, "Can you do annotations with ChatGPT?"

The answer was as follows.

In a typical implementation of ChatGPT, annotation cannot be performed. ChatGPT can execute natural language processing tasks by learning from a large amount of text data. Annotation is a task performed by humans who label data, and such tasks cannot be carried out by ChatGPT.

 

However, by creating applications using ChatGPT, humans can perform annotations. For example, when developing a dialogue system using ChatGPT, humans can annotate the dialogue data collected in advance and use it as training data. Additionally, in tasks involving text classification using ChatGPT, labeled data collected in advance can also be used as training data.

 

It means that it cannot be done. If you clearly describe the instructions for labeling and text processing and request it as a task, it might be possible. I would like to try it as a last resort.

2-1. Tagging

As a test, I will request to tag the sections related to "organization," "name," and "date" using a part of Shohei Ohtani's Wikipedia. To make it easier for the AI to understand, I have written the sentences short and one meaning per sentence. I will copy and paste the relevant text below the command and execute it. I made the request as follows.

 

 

And the response from ChatGPT was as follows. It took about 1-2 minutes to complete the processing. What do you think of the result?

 

 

You are tagging quite properly and correctly. MLB and NLB are also recognized as "organizations". However, unfortunately, there are some parts that have been overlooked. Perhaps it was difficult to recognize them as organizations in context. But first, it seems fine to let ChatGPT handle it, and then make corrections while checking afterwards.

2-2. Analyze Emotions

How about reading the text and classifying emotions this time? I would like to analyze emotions using the text from chat interactions. The analysis will be based on the eight basic emotions defined by Plutchik's wheel of emotions. You will determine which of these emotions are relevant. The eight basic emotions are: "joy, trust, fear, surprise, sadness, disgust, anger, anticipation."

 

 

I thought I would choose one from the eight options, but I ended up selecting two out of the eight. The analysis is also thorough. Initially, I thought it might be better to reduce the options a bit since having eight choices seemed a bit overwhelming, but it was not a problem at all. I am impressed that I could make a judgment and articulate everything in less than 10 seconds.

2-3. Create Similar Sentences

To train AI models, we sometimes create multiple expressions with the same meaning. I would like to see if that is possible as well.

Based on the sentence 'Please print this document in A4 size, 10 pages', I have come up with 5 alternative expressions.

 

 

They created the document easily. They include sentence structures and keywords that I wouldn't think of myself, so even if I can't use it as is, it seems like I can refer to it.

 

ChatGPT is also capable of translating foreign languages.
You can view the results of the verification of translation accuracy and various features here.
Related Blog: How good is ChatGPT's translation ability? Thorough verification of each step of the translation process

3. About Security

As of February 17, 2023, ChatGPT is available as a Research Preview. The data inputted into and outputted from ChatGPT may be reviewed by OpenAI. After this release, it will be used for research unless you opt out. Please apply to opt out or be careful not to input confidential or personal information.

4. Summary

I think it's really convenient that you can give instructions in Japanese without using complex programming languages. When writing command statements, it seems necessary to make them clear. Even if you think you've written them clearly, there may be times when you don't get the desired results, so it seems necessary to investigate the parts that didn't work well and tune them by changing the expression or the words used.

 

While I am surprised by ChatGPT's responses, there are indeed times when they are incorrect. This is an unavoidable fact, but on the other hand, it is not true that humans will never make mistakes in their work and judgments. Therefore, I feel that if we create a workflow where humans check and correct the work done by AI, it can be used effectively in actual operations. Ultimately, I believe whether it can be utilized depends on the accuracy rate of GPT's responses. If the accuracy rate of GPT's answers is high, productivity will increase, but if the accuracy rate is too low, the time and effort spent correcting many mistakes in the overall work may exceed the time it would take to do the work from scratch. It is important to test this in a pilot project and make a judgment based on the results.

 

In any case, I believe there is potential to utilize GPT to improve efficiency in the creation of training data in the future. I would like to continue experimenting with various approaches.

 

By the way, while trying various things with ChatGPT, I received the message, "Too many requests in 1 hour. Try again later." If you intend to use it for business purposes, it seems better to contract for the API. Please refer to the page below for information about the API.

https://openai.com/api/