AI21 Labs’ “Human or Not,” The Largest Turing Test to Date Finds That 32% of People Can’t Tell the Difference Between a Human and AI

AI21 Labs, a leader in generative AI and Large Language Models, revolutionizing the way people read and write, released the results of its social Turing game “Human or Not.” The game paired up players  for two minute conversations, after which users were asked to guess who they had been speaking with – a human or a bot. After over a month of play and millions of conversations, the results have shown that 32% of people can’t tell the difference between a human and AI.

“Human or Not?” used an AI bot based on leading LLM’s such as GPT-4 and Jurassic-2 and became a viral hit with users discussing their strategy across social media platforms.

AiThority: The 3 Building Blocks to Make AI Accessible

After analyzing the first two million conversations and guesses, here are some of the main insights from the experiment:

• 68% of people guess right when asked to determine whether they talked to a fellow human or an AI bot
• People found it easier to identify a fellow human: when talking to humans, participants guessed right 73% of the time. When talking to bots, participants guessed right just 60% of the time
• France has the highest percentage of correct guesses out of the top playing countries at 71.3% (above the general average of 68%), while India has the lowest percentage of correct guesses at 63.5%
• People used a few popular approaches and strategies to figure out if they were talking to a human: people tend to identify spelling and grammatical errors, as well as the usage of slang, with humans, and politeness was thought to be AI, the rest of the strategies can be found in the full report

“We created ‘Human or Not’ with the goal of enabling the general public, researchers, and policymakers to further understand the state of AI in early 2023,” said Amos Meron, Creative Product Lead at AI21 labs, the creator of the game. “Not looking at AI just as a productivity tool, but as future members of our online world, in a time when people are questioning how AI should be implemented in our futures.”

Read More: How ChatGPT Will Transform Customer Service

[To share your insights with us, please write to sghosh@martechseries.com]