Disney neural network performs basic literary criticism
Disney researchers have created a neural network capable of judging the quality of short stories by analysing the interaction of story elements and events within them.
It may be widely accepted that many low-skilled jobs – such as in manufacturing, admin and construction – are likely to be taken over by robots and forms of artificial intelligence (AI), although jobs in the creative sector are generally considered safer in the short- and medium-term.
It may come as an unpleasant surprise to some that researchers based at Disney Research and the University of Massachusetts have demonstrated that it is possible for an AI to take on – to an extent – the role of a literary critic.
Previously, computers have proved capable of some “creative” acts based on masses of human artistic data, although examples such as Google’s unusual Deep Dream art tool, or Sunspring, a clunky film script written by an AI called Benjamin, leave much to be desired.
The US researchers, attempting to create an AI that could recognise structure and content of a good short story, developed an artificial neural network for this purpose.
As there are no large enough databases of short stories that have been evaluated by critics, the neural network was given 55,000 pieces of prose from Q&A website Quora, where many responses are in the form of short stories, and public “upvotes” are a basic indicator of story quality.
The researchers developed an algorithm to sort prose into stories and non-stories, resulting in a dataset of more than 28,000 stories with an average of 369 words per story.
“The ability to predict narrative quality impacts on both story creation and story understanding,” said Markus Gross, Disney Research vice president. “To evaluate quality, the AI needs some level of understanding of the text.
“If AIs are to create narratives, they need to be able to judge the quality of what they are producing.”
The researchers represented different story structures within the neural network by evaluating separate “regions” of each story. This allowed for a crude analysis of how the meaning of the interacting events – and the story as a whole – emerges from its narrative structure.
On the basis of story structure, the neural network was able to predict which short stories would prove most popular with readers.
“Our neural networks had some success in predicting the popularity of stories,” said Boyang Li, a research scientist at Disney. “You can’t yet use them to pick out wines for your local writing competition, but they can be used to guide future research.”