AI Detection: How to Pinpoint AI Generated Text and Imagery [+ Detection Tools]

AI-generated content is a fascinating development, and we’re seeing more and more articles, stories, and images created by AI tools. (Thanks, AI, for the intro sentence.)

But, the rise of advanced AI generation tools has exposed potential issues, from people being unable to detect the difference between AI and human generations to AI predictions and analysis being flat-out wrong.

This is where AI detection comes in, as it’s a way for people to uncover when text, images, and even videos are machine-generated, so they can make informed decisions on the content they consume. In this post, we’ll cover:

• What is AI detection?
• How to Detect AI-Generated Text
• How to Detect AI-Generated Images and Videos
• AI Detection Tools
• What’s the best AI detection tool?

AI detection tools don’t know the meaning of words and use context to analyze text. To get more technical, tools use the context of what’s to the left of the following word to predict the likelihood of the word to the right.

The more predictable the word to the right is, the more likely the text is AI-generated. On the other hand, human-written sentences vary from predictable patterns and are more creative.

If you’re anything like me, a basic example might be helpful to understand this. Let’s break it down.

Say someone inputs the sentence, “Bunnies are so fluffy.”

The tool uses learned data and context of words to the left of “fluffy” to predict that “fluffy” is more likely to come next, more so than words like “cute” or “soft.”

Since the sentence follows a highly predictable pattern, the tool will likely classify the text as AI-generated.

AI detection tools work at a much larger scale with more complex sentences and paragraphs than “Bunnies are so fluffy” to make predictions and classifications, but this is a basic example and shows how the process works.

Some detection tools analyze images and videos and use pixel anomalies to determine if something is AI-generated.

How to Detect AI-Generated Text

There are no set rules or guidelines for identifying AI-generated text, but here are some things to look out for:

• Repetition of words and phrases: AI knows what it’s talking about, but not to the extent human experts do. Its outputs might repeat the same keywords and phrases with little variation when discussing a topic.

• Lack of depth: Generation tools lack depth and can’t go beyond basic facts to truly analyze a topic and develop unique insight. AI-generated text might read more robotic and prescriptive than creative and have a generic tone.

• Inaccurate and outdated information: The facts that content generation tools have are typically correct, but since the tools make predictions, outputs can be incorrect or unrelated to true facts. In addition, information can be outdated, like how ChatGPT is limited to information pre-September of 2021.

• Format and structure: Generation tools follow the same sentence structure as humans, but sentences can be shorter and lack the complexity, creativity, and varied sentence structure humans produce. Content can be streamlined and uniform with little variation.

Human-written text is also more likely to have typos and use informal and casual language and slag.

Roft.io is a fun game to test your detection skills and see how good you are at predicting when text is AI-generated.

How to Detect AI-Generated Images and Videos

Identifying AI generated images and videos can be a bit more challenging than detecting text. Some commonly discussed tells are:

• Textured backgrounds, images that look airbrushed, random brush strokes throughout images
• Overall image sharpness, or parts of images that are blurry while others are more clear
• Noticeable text in the background of images
• Asymmetry in human faces, teeth, and hands
• Signs of artist watermarks or signatures (AI tools are trained from existing artwork)

Tools like DALL-E 2 place a watermark on image outputs, but they might not be easy to spot. OpenAI also allows people to remove a watermark. You can also reverse image search to see if there are any traces of an image on the web.

The challenge of detecting AI images and videos is why deepfakes are so dangerous, as videos and images that seem lifelike enough can rapidly spread misinformation.

AI Detection Tools

At the moment, it might be easier to tell if something is AI generated because it sounds robotic, or someone’s hand is missing two fingers in an image. If generation tools become more sophisticated, it might be harder for humans to find the key discrepancies.

Regardless of future progressions, detection tools can be more helpful than our own deduction abilities in classifying AI-generated content, and there are various options available.

Below we’ll go over some of them and rate their effectiveness using an AI-generated paragraph from HubSpot’s Content Assistant (which uses GPT). Here’s what it gave me when I asked it to write a paragraph about dogs:

“Dogs are simply amazing creatures. They are loyal, loving, and endlessly entertaining. Whether you need a furry friend to cuddle with on the couch or a loyal companion to explore the great outdoors with, dogs are always up for the task. They come in all shapes and sizes, from tiny teacup Chihuahuas to majestic Great Danes, but all dogs share one thing in common: a boundless capacity for love and affection. Whether you’re a lifelong dog lover or a newcomer to the world of canine companionship, there’s never been a better time to discover the joys of life with a furry friend by your side.”

Note that human writing can still trigger a tool if it follows a predictable pattern.

1. ZeroGPT

• Price: Free or contact for custom API
• Tests for: ChatGPT and Google Bard

ZeroGPT’s algorithm is trained on 10M+ articles and text to have a detection accuracy rate of 98%. It supports multilingual text and detects popular language generators like Chat GPT, GPT-4, and Google Bard. Outputs highlight sentences most likely to be written by AI.

I entered the AI-generated paragraph about dogs, and it predicted the text is 88.57% AI/GPT generated.

Best for: ZeroGPT was built for educators to test for AI-generated content, but it works for anyone looking to detect AI content.

2. Giant Language model Test Room

• Price: Free
• Tests for: Developed in 2019 for GPT-2 text, might be unreliable on other generators

MIT-IBM Watson AI lab and the Harvard NLP group created the Giant Language model Test Room to detect AI-generated text. It analyzes inputs based on how likely each word is to appear based on the word immediately to the left. The more predictable the word is, the more likely the text is written by AI.

This tool doesn’t give a percentage but color codes words based on their predictability, with green meaning the word is part of the top 10 most predictable words.

Most of my paragraph is highlighted green, so the words are part of the top 10 most predictable (based on context) and more likely to be AI-generated.

Best for: Testing GPT-2 and learning more about predictable writing through an in-depth probability analysis.

3. Originality.AI

• Price: Free 50 credit trial, then $0.01/100 words (1 credit scans 100 words)
• Tests for: ChatGPT, GPT-3, GPT-3.5, GPT-NEO, GPT-J

Originality.AI Chrome Extension, built by content marketing experts, detects multiple versions of GPT with 94% accuracy. It scores text on a scale of 0-100, with a higher score being a higher likelihood of being produced by AI. You can also use the tool to scan for plagiarism (beneficial for educators). It’s the most accurate with more than 50 words.

With my test, it said that the paragraph was 99% likely to have been written by AI.

Best for: The Chrome extension makes it perfect for anyone looking for a seamless and immediate detection process when writing and reading online. Writers, content marketers, and web publishers alike can leverage this tool; not for academics.

4. Content at Scale

• Price: Free version, or contact for API pricing
• Tests for: GPT

Content at Scale’s AI Detector uses 3 AI engines and natural language processing to detect ChatGPT, all versions of GPT, and other generators. You can use it to test SEO, educational, and marketing content. The tool needs at least 25 words for reliable results, and you can enter up to 25,000 characters.

My test results were inconclusive because the tool couldn’t say with certainty if the paragraph was AI-generated. It gave a human content score of 51% with 17% predictability. 

It did say with certainty that the last sentence is AI-generated.

Best for: SEO and marketing-focused content creators to get line-by-line text breakdowns and analyze longer pieces of content (up to 25,000 characters).

5. Writer AI

• Price: Free version or contact for API pricing
• Tests for: ChatGPT and other generators

Writer AI’s content detector estimates how much text is AI-generated. The free and paid versions have a 300-word limit (1,500 characters), and results give a prediction percentage for how much of the text is human-generated content.

It scored my paragraph as 87% human-generated, with a recommendation to edit the text until there’s less detectable AI content. 

Best for: B2B and enterprise and agencies looking to analyze and edit content before publishing.

6. Hive’s AI Detection Tools

• Price: Free demo, contact sales for API pricing
• Tests for: ChatGPT, GPT-3, DALL-E, Midjourney, Stable Diffusion

Hive offers a suite of AI detection tools for images, text, and deepfakes.

The text detection tool gives a confidence score for how likely something is AI-generated, and estimates which sections are most predictable. It also estimates which sections of text are more likely to be AI-generated. It works starting at 750 characters with a recommended length of 1500 characters.

I had to input extra words to reach the character limit, and it predicted the paragraph was 99.99% likely to contain AI-generated content.

The media recognition tool identifies AI-generated media, gives a classification (AI-generated or not), confidence score (≤ 1), and image generation source (like DALL-E). (Documentation, tool page)

The deepfake detection tool tests if images or videos are deepfakes through facial classification. (Documentation)

Best for: Screening work to detect AI content or for websites to detect and moderate AI-generated images and text.

7. Bonus: OpenAI’s Text Classifier

• Price: Free (requires account)
• Tests for: All versions of GPT

OpenAI’s Text Classifier can distinguish between AI-generated text and human-written text. It works best with more than 1,000 characters and English text.

OpenAI does note that it is not entirely reliable and only correctly identifies 26% of AI text and incorrectly labels human-written text as AI 9% of the time, but reliability increases for longer text. It recommends using the classifier as a complement to other testing methods.

Best for: Detecting GPT

What’s the best AI detection tool?

I outlined each tool’s individual test score above, but here’s a table comparing scores.

Based on these rankings, 

• First place is a tie between Originality.AI, GLTR, and Hive AI
• Second place is ZeroGPT
• Third place is Writer AI
• Fourth place is Content at Scale

Over to You

AI detection makes it a lot easier to distinguish between machine and human-generated text. As AI tools become more and more accurate, AI detection will remain important in helping people determine the legitimacy of the content they consume. 

Source link