OpenAI Can’t Detect Its Own ChatGPT-Generated Text Most of the Time

In response to the growing concern from educators over ChatGPT’s ability to help students cheat, OpenAI released a tool on Tuesday that can detect AI-written text. However, the company said, “Our classifier is not fully reliable.”

“In our evaluations on a ‘challenge set’ of English texts, our classifier correctly identifies 26% of AI-written text (true positives) as ‘likely AI-written,’ while incorrectly labeling human-written text as AI-written 9% of the time (false positives),” the company, which developed ChatGPT, wrote in a blog post.

The classifier, which was trained on a dataset of human-written and AI-written text on the same topic, is not yet dependable and is only supposed to complement other ways of determining who a text is written by. The limitations that the classifier has, according to OpenAI, include being unreliable on short texts and performing worse in other languages.

OpenAI said that it has developed preliminary resources on the impact of ChatGPT for educators, acknowledging that it is important to recognize “the limits and impacts of AI generated text classifiers in the classroom.”

Recently, NYU professors told students on the first day of classes that they were not allowed to use ChatGPT without explicit permission, saying that any usage of the tool would be considered plagiarism. Professors have been coming up with their own ways to detect AI writing in order to prevent any sort of cheating, such as running their essay prompts through ChatGPT to have a benchmark of what an AI-generated essay would look like.

A 22-year-old student named Edward Tian developed his own ChatGPT detector, which was launched in beta earlier this year and released in full as GPTZeroX on January 29. In this app, you can insert text or upload one or more documents at once and the app will generate a score for how much of the text was written by AI and highlight the sentences that were written by AI. According to Tian, the app was wrong less than 2 percent of the time when tested on a dataset of BBC news articles and machine-generated articles with the same prompt, which would make GPTZeroX seemingly more reliable than OpenAI’s own detector.

OpenAI includes a call-to-action in its blog post, offering people directly impacted by the language bot, “including but not limited to teachers, administrators, parents, students, and education service providers,” to provide feedback through a Google Form survey. As ChatGPT becomes capable of writing everything from college essays to code, teachers around the country are attempting to adjust their classrooms around the new technology and discussing how the tool can be used in an ethical way.

“We’re not educators ourselves—we’re very aware of that—and so our goals are really to help equip teachers to deploy these models effectively in and out of the classroom,” Open AI policy research director Lana Ahmad told CNN. “That means giving them the language to speak about it, help them understand the capabilities and the limitations, and then secondarily through them, equip students to navigate the complexities that AI is already introducing in the world.”