New Study: People Prefer AI-Written Stories, But Can't Tell Them Apart From Human Ones

People are unable to distinguish between stories written by humans and those generated by artificial intelligence, and in many cases, they prefer the AI-authored content, according to a new study.

Researchers from Villanova University conducted an experiment involving over 2,500 adults aged 18 to 81. Participants were presented with a series of short stories, some written by humans and others by AI, such as ChatGPT. When participants were told that an AI-written story was actually authored by a human, they consistently rated it higher in terms of quality and engagement than any other combination.

In one part of the experiment, participants could only guess whether a story was AI-generated about 39.9% of the time, a rate worse than random chance. In another test, their accuracy was 52%, which is also not significantly better than guessing.

These findings suggest that public perception of AI capabilities may be outdated. While many people assume creative writing requires uniquely human qualities like emotional understanding and lived experience, the study indicates that AI is increasingly capable of producing work that readers find more appealing, particularly when its origin is misattributed.

Dr. Deena Weisberg, a senior author of the study, noted that familiarity with AI systems can help some individuals recognize patterns common in AI-generated writing, such as repetitive sentence structures or the use of em dashes. She suggested that improving AI literacy could help people better navigate an increasingly AI-influenced world.

The study, published in the journal Judgment and Decision Making, aligns with previous research showing a preference for AI-generated writing in various forms, including poetry and essays. Researchers theorize that AI writing is often clearer, more direct, and easier to process than human writing, which can be more subtle and complex. AI stories, for instance, tend to state themes explicitly, whereas human authors often rely on inference from characters' actions and words. This predictability might contribute to reader preference, as it requires less cognitive effort.

Tips for identifying AI-generated text have emerged, including looking for inconsistencies, repetition, abrupt shifts in tone, formulaic language, or excessive use of buzzwords. Sometimes AI might offer specific details without adequate context or maintain a consistently neutral tone. Quick response times, compared to the more considered replies of humans, can also be an indicator. Developers have also created AI detection tools to help identify AI-generated content in applications like student essays or job applications.