Can annotation be done with ChatGPT?

1. What is ChatGPT?

Since writing articles is difficult, I asked ChatGPT for an explanation. When I asked ChatGPT, "What is ChatGPT?", I received the following response:

ChatGPT is one of the language models developed by OpenAI. GPT stands for "Generative Pre-trained Transformer" and is a deep learning model used in the field of natural language processing for tasks such as text generation, text classification, text summarization, and dialogue systems. ChatGPT is a natural language dialogue system built using GPT, allowing for natural conversations with users. ChatGPT is a highly anticipated technology in the field of artificial intelligence, with various applications expected.

 

Wow, that's amazing... I was able to get a clear explanation.

So, what can CahtGPT do? I asked ChatGPT about this as well.

ChatGPT is one of the language models used in the field of natural language processing and can do various things. Below are some of the representative features of ChatGPT.

Text Generation: ChatGPT can generate natural sentences for given text. For example, it can generate short articles on specified topics or longer stories like novels.

 

Article Classification: ChatGPT can classify given articles into categories. For example, it can classify whether an article is about sports or politics, or if a movie review is positive or negative.

 

Summary: ChatGPT can summarize given text. It can shorten long articles and reports, allowing you to grasp the main points.

 

Dialogue System: ChatGPT is also used as an AI-based dialogue system. By creating a chatbot using ChatGPT, you can have natural conversations with users.

 

As mentioned above, ChatGPT is widely used in the field of natural language processing and is expected to have various applications.

 

It seems that the above sentence was generated rather than copied from somewhere, as I tried searching for it on Google but it did not come up in the results. With ChatGPT, there are many other things that can be done, but this overview explanation should suffice.

Can annotation be done with ChatGPT?

So, can we use ChatGPT for annotation, especially for text annotation? In addition to what ChatGPT can answer, it can also write program code. Being able to write program code means understanding tags. If it can understand sentences and tags, it may be able to do text annotation as well.

 

Once again, I asked ChatGPT, "Can data annotation be done with ChatGPT?"

The answer was as follows.

In general implementations of ChatGPT, it is not possible to perform data annotation. ChatGPT is able to perform natural language processing tasks by learning from large amounts of text data. Annotation is a task where humans label data, and ChatGPT is not able to perform such tasks.

 

However, by creating applications using ChatGPT, humans can perform data annotation. For example, when developing a dialogue system using ChatGPT, humans can annotate previously collected dialogue data and use it as training data. In addition, for tasks such as text classification using ChatGPT, previously collected labeled data can also be used as training data.

 

It means that it cannot be done. If you describe labeling and text processing commands clearly and make a request as a task, it may be possible to do it. I would like to try it as a last resort.

2-1. Tagging

As an experiment, we will request tagging for the "organization", "name", and "date" sections using a portion of Shohei Ohtani's Wikipedia page. We have shortened the sentences to make them easier for AI to understand. Then, below the command, we will paste the target sentences and execute. We made the request as shown below.

 

 

And the response from ChatGPT was as follows. It took about 1-2 minutes for the processing to finish. What is the result?

 

 

Thank you for properly tagging this. It seems that MLB and NLB are also recognized as "organizations". However, unfortunately, there are some parts that have been overlooked. Was it difficult to recognize them as organizations in context? But first, let's have ChatGPT do it and then check and make corrections, which seems to be a problem-free level.

2-2. Analyzing Emotions

Next, let's try to read the text and classify emotions. I would like to analyze emotions using the text exchanged in chats. The basis of the analysis will be the eight basic emotions defined in Plutchik's wheel of emotions. You will be asked to determine which emotions are involved. The eight basic emotions are joy, trust, fear, surprise, sadness, disgust, anger, and anticipation.

 

 

I thought I had to choose one out of eight, but I ended up choosing two out of the eight. The analysis was also thorough. At first, I thought it would be better to reduce the number of options since eight seemed like a lot, but there was no problem at all. I was impressed that I was able to make a decision and put it into words in less than 10 seconds for all of them.

2-3. Create similar sentences

In order to train AI models, we may also create multiple expressions with the same meaning. I would like to try if that is possible.

"We asked for 5 different ways to phrase the sentence 'Please print this document on 10 sheets of A4 paper'."

 

 

They quickly created the text for me. They included sentence structures and keywords that I wouldn't have thought of myself, so even if I can't use it as is, I can still use it as a reference.

 

ChatGPT also offers foreign language translation.
For results of verification on translation and accuracy of each function, please see here.
Related Blog：How good is ChatGPT's translation? Thorough verification at each stage of translation

3. About Security

As of February 17, 2023, ChatGPT is publicly available as a Research Preview. Data input and output from ChatGPT may be investigated by OpenAI. Even after this release, it may be used for research unless you opt out. Please apply for opt-out or be careful not to enter confidential or personal information.

4. Summary

I think it's really convenient to be able to give instructions in Japanese without having to use complex programming languages. When writing command sentences, it seems necessary to make them easy to understand. Even if you think you have written them clearly, there may be cases where you cannot get the desired results, so it seems necessary to investigate the parts that did not work well and tune them by changing the expression or using different words.

 

While it is surprising to see the responses from ChatGPT, there are still times when they are incorrect. This is an unavoidable fact, but it is also not true that humans will always make the right decisions and judgments. Therefore, I felt that if we create a flow where humans check and correct the work after it has been done by AI, it can be used in actual operation. The key factor will be whether the accuracy of GPT's responses is high enough to be utilized. If the accuracy is high, it will increase productivity, but if it is too low, there is a possibility that it will take more time and effort to check and correct the mistakes compared to starting from scratch. It is important to try and judge these aspects in a pilot project.

 

In any case, I thought there is a possibility to use GPT for efficient creation of teacher data in the future. I would like to continue trying various things in the future.

 

Also, while experimenting with ChatGPT, I received the message "Too many requests in 1 hour. Try again later." If you plan to use it for business purposes, it may be better to contract the API. Please refer to the following page for more information about the API.

https://openai.com/api/