AI writing is preferred by most readers, according to a new study that suggests people may not favour human authorship as much as they assume. Instead, readers appear to like content that simply sounds human.
The research was led by Cambridge University Press together with researchers at Villanova University. It asked 1,682 participants aged 18 to 81 to read several fictional stories, determine whether they could tell if the texts were AI-generated, and then rate the quality of each story.
How the study was structured
The study comprised three separate experiments. In the first, all participants were given a story, with some told it was AI-generated and others told it was human-written, before rating its quality and engagement. In the second and third experiments, participants were divided into two groups, one with 424 people and the other with 481. Each person read one human-written and one ChatGPT-generated story, without being told which was which, and was asked to identify the story they believed had been authored by ChatGPT.
Researchers found that participants rated stories generated by ChatGPT most highly, especially when they were intentionally and incorrectly told a story had been written by a human. Many participants could not correctly identify the AI-generated content, with more than 60% unable to do so in the second experiment and more than 48% in the third.
Deena Weisberg, senior author of the study and professor in the Department of Psychological and Brain Sciences at Villanova University, said the results reveal a bias towards narratives written by real people. She said people assume creative writing requires uniquely human qualities, such as emotional understanding and lived experience, which leads them to underestimate AI’s capabilities. She added that public assumptions about AI’s capabilities are increasingly out of date.
Why readers favour AI-generated text
A growing body of evidence indicates that people like AI-generated writing, including short stories, essays and poetry. A 2023 study led by Renee Richardson Gosline of MIT reached similar findings, concluding that AI-generated content and augmented AI, where the AI made the final decision with human output as a reference, were seen as higher quality than work produced by human experts and augmented human experts, where the human made the final decision with AI output as a reference.
Weisberg suggests this is likely because AI-written stories are more direct, explicit and simple, whereas human-written stories tend to be more subtle or complex and leave more room for interpretation.
One AI-generated passage given to participants described sitting by a pond as autumn leaves fell, reflecting on a mother’s stories and wisdom, with koi continuing their movement beneath the water’s surface as a reminder that life goes on. A human-written passage, by contrast, described a sense of birth, with one figure labouring, another acting as midwife and a husband playing the part of a nervous father.
The study found that more than 60% of participants in the second experiment could not correctly identify which content was produced by AI.
