First, a computer painted a masterpiece. Then a computer wrote a book. Now Sondheim and Rogers and Hammerstein are in danger of being ousted by the clever use of computational creativity and algorithms.
The world’s first musical created by a machine premieres at the Arts Theatre in London’s West End this month. ‘Beyond the Fence’ is conceived and crafted by computer and modelled on a statistical study of the recipe for a hit musical success. Given that it has been authored by carefully-honed technology rather than an imperfect and flawed human, it is predicted to sell out.
‘Beyond the Fence’ has its origins in an experiment that sought to adapt computer programs already used in Artificial Intelligence (AI) and apply them to searching for the formula for a brand-new musical theatre sensation. Dr Catherine Gale, a researcher in computational biomedicine and executive producer at production company Wingspan, designed and coordinated the experiment. “Musical theatre is quite conservative in its construction,” says Dr Gale. “If you look, you can find elements which are more prevalent in the successful ones. Are some of these elements predictive of success? How big is the cast? How long is the show? How many songs? How long are they? Does this matter? What makes a sad song? These questions have been asked for a long time. But when these technologies become available, it gives us a new way of looking at old questions. That’s what we set out to do.”
This path of discovery began with a predictive big-data analysis of successful musicals conducted by the machine learning group at Cambridge University. “We included any data about musical theatre we could get our hands on,” says researcher Dr James Robert Lloyd, now head of machine learning at Qlearsite Organisational Science. “We downloaded information from the Internet Broadway Database, Wikipedia and any other free online resources. We also conducted some of our own data-gathering exercises such as asking people to fill in surveys about the emotions that particular musicals made them feel.” There were, however, areas they couldn’t reach.
Finding the formula
They interrogated cast size, backdrop, emotional structure, the importance of someone falling in love and dying, thus creating a set of constraints to which the musical had to conform in order to theoretically optimise chances of success. Patterns and causalities emerged. “The average success had some recognisable features, such as the peak in positive emotion around halfway through the second act,” says Professor Sir David Spiegelhalter, a statistician in the Cambridge team.
Theatre producer Neil Laidlaw, who worked on the enormously successful recent run of ‘Gypsy’ in London’s Savoy Theatre and is producing ‘Beyond the Fence’, outlines the main findings: “The investigation determined the show should open with a big energetic number that hooks the audience in. The placing of the songs was shown to be key. There should be a comedy song - such as in ‘Billy Elliot’ where there’s ‘Express Yourself’ and the boys dress up. The protagonist should be female and there must be a happy resolution.” Two possible periods for the setting came to the fore - the 1930s and the 1980s. The creative team, the ‘curators’, web-searched the decades and came up with a 1980s scenario - the women’s anti-nuclear protests at Greenham Common. “There’s still the need for this human input,” says Laidlaw. “What’s really interesting is how computers can be launch pads for human creativity.”
Yet some theatre producers are more sceptical of technology’s contribution to culture. According to Julius Green, Fellow of Birkbeck Centre For Contemporary Theatre at the University of London and author of ‘How To Produce a West End Show’, all musicals are instinctively derived from a successful formula which has already been tried and tested, whether or not these formulas are computer-generated. “For instance, ‘West Side Story’ derives its characters and plot from a Shakespeare play and ‘Les Misérables’ takes its story from a Victor Hugo novel,” says Green.
Green points to an example of human endeavour in a musical creation which echoes some of the scientists’ work. “One of the shows I help to produce is ‘Showstopper! The Improvised Musical’, in which a team of actors and musicians create a new musical based entirely on suggestions from the audience, including the title, plot and musical styles. Over 700 completely new musicals have been constructed in this way, and it gives a fascinating insight into what audiences themselves want to see in a show. For instance, the Vatican and tube train carriages crop up fairly regularly. Giving audiences what they want is, after all, a significant component in the mix when it comes to successful theatre of any sort.”
However, Dr Gale claims the computer wins at this creativity game. “The advantage of a computer is that it can crunch data far more quickly than a human brain,” she says. “When you have a computer with enough power to get through a lot of data, you can go looking for patterns that you would never be able to do within your own lifetime.” Gale also points out another advantage: “A machine doesn’t have biases about what it likes, or taste, or things it learns at school. It can just look at stories and look at what pattern works within them,” she says.
The musical has come late in the artistic canon of computational creativity. Since the 1970s, painter Harold Cohen has collaborated with a computer to make visual art using creative AI which he affectionately called Aaron. Aaron is a piece of software that creates artworks based on Cohen’s instructions when connected to a large-scale inkjet printer. Aaron’s work is regularly exhibited in the US and Europe.
There have also been attempts to produce computer-generated fiction. In 1983, an experimental book of poetry called ‘The Policeman’s Beard is Half Constructed’ was written using the Racter program. In 2009, a novel claiming to be the first written by a computer was published; ‘True Love’ was created by a group of software developers and philologists working for St Petersburg publisher Astrel-SPb using the program PC Writer. The philologists compiled dossiers on each of the novel’s main protagonists - their appearance, vocabulary, psychological profile and other key characteristics - and the novel was written by computer over three days. ‘True Love’ did not receive a good review in this magazine, which described it as “mongrel of a modern novel”, “clichéd” and “definitely not Tolstoy”.
The ‘Beyond the Fence’ team has consulted leading AI and statistics experts to ensure the very best chance of creating a classic. Once Cambridge had analysed the big data, further development was handed over to the team behind the What-If Machine at Goldsmiths, University of London. The What-If Machine uses ConceptNet, a semantic network based on Open Mind Common Sense (OMCS) data which utilises a large ‘Commonsense’ knowledge base from the contributions of many thousands of people across the web. OMCS has accumulated more than a million facts from over 15,000 contributors.
Dr Rose Hepworth, a member of the Goldsmiths team, explains: “For example, a very simple ‘what if’ style idea could be produced by taking a ‘fact’ from ConceptNet such as ‘Cat - desires - milk’ and inverting it so that you have X is afraid of Y rather than X desires Y. This gives you a very simple what-if idea: what if there was a little cat who was afraid of milk? Inverting facts was one of our starting points for the What If Machine.” The machine generated multiple central premises featuring key characters for the new show. The resulting premise was: what if a wounded soldier had to learn how to understand a child in order to find true love? This became the starting point for the new musical.
A plot structure was then generated computationally, led by Dr Pablo Gervás from Complutense University of Madrid. He adapted an existing story-telling computer system, PropperWryter, to musicals and built the core narrative arc of the new show. Gervás explains: “PropperWryter builds sequences of plot elements, which are representations of character behaviour at a certain level of abstraction. These are things like ‘boy meets girl’, ‘boy loses girl’, ‘villain kidnaps victim’, ‘hero rescues victim’. The system knows how these plot elements are likely to chain up. Characters paired into a couple may break up; a victim that has been kidnapped may be rescued. It also knows that if it mentions one of these in a plot, this introduces a tension, and that it should follow it up with a plot element that resolves it. It knows as well which plot elements are sad, which are happy, which are violent - a number of features considered relevant to predict the impact of the plot on the reader. It also knows which roles each character involved plays in these plot elements. Finally, it knows that stories cannot end until the tensions have all been resolved. The system takes all this information into account.”
Yet humans still play a role. “The data that the system handles are the set of plot elements, the dependencies between them, the narrative roles that the characters play in them, and the emotional features that correlate with the plot elements. These data had to be learnt by the system from examples of plot stories annotated by human volunteers with the relevant information. Those human judgements are essential as they provide the system with the knowledge it needs to operate,” Gervás says.
The story needed music. Dr Nick Collins of Durham University has created a computer composition system he’s dubbed Android Lloyd Webber (ALW). “ALW generates new songs in a musical theatre style based on statistics of existing songs of the genre and hard-coded rules. Machine learning acts over a corpus of musical theatre songs to determine such properties as the nature of pitch interval leaps in a melody, and a corpus of music theatre cast recordings with extracted chord sequences to prepare a model of harmony,” says Collins.
Isn’t there a risk that data crunching just comes up with more of the same old songs, and innovation is lost? “There is as much creativity in programming music software as traditional composing, and enough engineering choices to mean that it will necessarily engage with the new, even if there seem to be databases of existing music involved,” says Collins.
Yet Julius Green believes leaving composition to a computer doesn’t guarantee more bums on theatre seats. “The two key elements of success in musical theatre are the originality of the concept and the creative passion of the human beings behind it. Be it dancing cats, an epic French novel or transvestites from Transylvania, a project that has a truly unique selling point is more likely to be successful than one that sets out to recreate the formula of previous shows.”
Dr Collins admits that there may be hurdles to ‘Beyond the Fence’ winning an Olivier Award. “Most musicals are financial and critical flops and there are results showing that audiences can be biased against computers, often failing to see the human programmers behind. But the audience for ‘Beyond the Fence’ is forewarned, so may be more invested in it being successful.”