Interview: Beth Holmes, principal knowledge engineer, Amazon Alexa

In Recent years, voice activation has become ubiquitous, with Amazon’s cloud-based voice service Alexa a main player featuring on more than 100 million devices worldwide. Beth Holmes, who is principal knowledge engineer on Amazon Alexa, says there are billions of interactions with these devices per week.

Holmes has been with the product since November 2014, when “a small multi­disciplinary team launched Amazon Echo, with the aspiration of revolutionising daily convenience using artificial intelligence”. Before that, she was part of the technology team at start-up Evi Technologies, whose acquisition by Amazon meant its natural language AI program became a key component underpinning Alexa.

The statistics are mind-boggling. Holmes provides a screed of numbers. There are more than 200 million smart home devices ranging from lights and plugs to robotic vacuums and grills connected to Alexa. Holmes reckons that “customers are using Alexa every couple of milliseconds with their smart home devices to simplify their morning routines, take chores off their to-do lists and keep their homes safe”. There are over 140,000 Alexa-compatible products. Thanks to predictive and proactive features like Alexa Guard, Routines and Hunches, one-in-four smart home interactions are initiated by Alexa without customers saying anything. There are more than 130,000 Alexa skills (the equivalent of apps), while the volume of users engaging with skills is growing by 40 per cent a year, with strong growth in music, audio and games.

Just as apps are the lifeblood of the smartphone experience, ‘skills’ are the stimulus for what Holmes calls a “human-machine interaction paradigm shift” in voice-enabled ambient computing. Before Echo – the range of Amazon devices that deliver the Alexa voice platform – “customers were used to searches on desktops and mobile phones, where the onus was entirely on them to sift through blue links to find answers to their questions or connect to services. While app stores on phones offered ‘there’s an app for that’ convenience, the cognitive load on customers continued to increase. Alexa-powered Echo broke these human-machine interaction paradigms, shifting the cognitive load from customers to AI, causing a tectonic shift in how customers interact with a myriad of services, find information on the web, control smart appliances and connect with other people.”

In other words these skills are the little apps that help you get stuff done using just your voice. As Holmes explains: “Customers can use Alexa for many things: to play music or videos, answer questions, set timers and reminders, notify you of package deliveries or sports games updates, control your smart-home devices, call your friends and family.”

Holmes confirms that she uses her own product in everyday life: “I like using Alexa because spending all day typing on a computer is not so healthy. It’s nice to get some tasks done via conversation instead. The voice interface is particularly important for people who have problems interacting with computers, such as the elderly with arthritic hands or other mobility problems, the visually impaired, or kids who can’t read yet.”

Cambridge-based Holmes became principal knowledge engineer on Alexa at the start of 2020, in which her role is technical oversight of the knowledge engineering team. She describes knowledge engineering as a branch of ontology, or the study of how concepts exist and relate to one another. She explains that a simple ontology (or model) for a movie could contain multiple concepts such as casting, shooting schedules, release dates and so on. “For example, Prime Video would need to model whether the Italian dub of a director’s cut is available in Canada yet.

“Currently, I’m doing projects to drive consistency across the ontologies that Alexa teams expose to each other. I’m also leading an ontology health drive: when large pools of staff contribute to ontologies under time pressure, hacks and technical debt start to build up, so we do clean-up projects as we go.”

These ontologies can cross departmental boundaries, which means that Holmes will find herself communicating with ontologists in other departments such as the Amazon retail site. “I work with them on cross-ontologist community initiatives. I led our annual conference ‘KnowledgeCon’ for the past couple of years, and I get involved in analysing their staff satisfaction results.”

In response to general enquiries about her background, the 45-year-old forwarded me an article entitled ‘Lesbian Motherhood and the Artificial Insemination by Donor Scandal of 1978’, explaining that “my mother was gay and back then people were quite shocked by our family structure. So all my upbringing was by women who’d chosen their own path, which probably set me up with less stereotyped expectations from the start.”

At school, Holmes liked academic subjects but not sport. During this time she had a parallel interest in art, thought about becoming a taxi driver, an Olympic gymnast or even “a great writer like Enid Blyton. I don’t think any of my support system had any idea that I would end up in knowledge engineering. My mother assumed that I’d be a teacher because she’d been a teacher. My school were aware I’d get good grades, but they didn’t seem to have an opinion on what I should do with that.”

Holmes particularly liked maths because “it was easy and because I really liked it”, paving the way for her undergraduate degree in mathematics at what’s now called Aberystwyth University in Wales, followed by her doctorate (also in mathematics) at the University of Birmingham, where her thesis in the field of computational group theory was entitled ‘Computing in the Monster’ (the ‘monster’ being the largest of the sporadic simple groups and “too big for standard computational methods to work”).

“Its smallest representation was as a set of ~200,000 dimensional matrices. My PhD supervisor Rob Wilson and his colleagues had invented some custom techniques to work with those matrices. My thesis built on these techniques, which enabled me to clear up some open questions about the Monster’s structure.” After Birmingham, Holmes did “a few fellowships”, mostly continuations of her doctorate, but unable to relocate due to family commitments, she decided to leave academia.

With the help of a friend, the highly qualified mathematician got a job “to do with knowledge engineering at a start-up. I had no previous knowledge or experience of it, as it is such a niche area. But they weren’t looking for anyone with previous knowledge. They just wanted somebody new to come in, and hopefully learn. The start-up, Evi Technologies, worked by taking a question as natural language, transforming it into a machine-readable query, running the query to generate the result as data, and then transforming it back into a natural language answer.” Holmes explains how this works: “The customer types: ‘when is the next bank holiday?’ Evi translates it to meaning: ‘find the customer’s country, look up the list of public holidays and their dates, select the earliest upcoming one’. Evi’s query engine returns: England, Good Friday, 14/4/2022. Evi then builds it into a sentence: ‘The next public holiday in England is Good Friday on 14 April 2022’.”

Holmes says that this natural language processing was ‘a good fit’ for Alexa because its developers “already had voice technology for going from audio-to-text and then text-to-audio. So our text-to-text question-answering fitted nicely between those two steps. Before the start-up I worked for got acquired by Amazon, I had switched roles from being an individual contributor to becoming a manager. The acquisition led to a reasonable allowance to hire more people. So as a manager, I was able to build out my team quite a lot, which helped me progress in that role.” A bigger team meant a bigger management role and greater opportunities to meet and work with other people in the Alexa orbit. But management wasn’t the right path for Holmes: “I decided it was not working out for me, so I switched back to individual contributor at the start of 2020.”

It was around this time that Holmes was diagnosed with autism. And while her formal diagnosis at the age of 44 “has not made much difference, the one thing it has enabled me to do is be a lot more open about it”. She says she was only certain that she was autistic for a few years before her diagnosis. “I have always felt very different to other people, and acknowledging why that was, what was different about me and in what sort of ways, made it easier for me to understand why I find things harder or easier. And that’s OK for me. I now have a much better chance of being successful if I can angle towards those rather than trying to be good at the same things that everybody else is good at.”

Holmes says that autism brings with it “hindrances. But they were there regardless of whether I got a diagnosis or not.” But the diagnosis has provided some explanations.

She says that she used to feel alienated, “especially by the ‘women in careers’ messaging. There seemed to be lots of beliefs which I could not identify with.” She didn’t align with positive statements such as ‘women have better social skills and are good at building relationships’, “because I have hopeless social skills”. Neither could she empathise with some negatives: “We’re told ‘women find it hard to speak out in meetings’, but I am very outspoken.” Another idea that doesn’t ring bells of authenticity for Holmes is that technology could attract more women “by getting away from the image of software workers as being nerdy, scruffy, socially awkward”. These are in fact, she says, all common Asperger’s traits “that I relate to. So I felt like the women-focused efforts were aimed at recruiting ‘real women’ instead of people like myself.”

Holmes thinks that there are potential opportunities for people with autism in the STEM space. “Many autistic people become good at software development. Based on what I heard in community forums, it sounds like many work in other sciences, as lab technicians, for example. It is true that some companies need you to also have good people skills, particularly in big corporations. But many autistic people (particularly women) have learned those skills, and for others who haven’t there are still plenty of employers who just need the tasks done.” She adds that Amazon actively recruits from diverse backgrounds.

Reflecting on how she might influence others, Holmes categorically denies being a role model, preferring to reserve the label for “people who’ve had massive achievements, such as Greta Thunberg”. She goes on to say that while it can often seem that a lot of autistic people “suffer from depression and difficult life circumstances”, any problems she has had to deal with are largely behind her. “These days I feel good about most aspects of my life. In my career, the abilities that got me here aren’t anything exceptional. Instead they are common autistic traits that many others share.”

These characteristics, she says, include “going super-deep into topics, retaining information about them, forming independent opinions without getting too swayed by groupthink”. She finishes by commenting that it is important for her to use opportunities such as press interviews so that she can “show autistic people with similar traits that ‘things can be OK’, and to show neurotypical people another datapoint of what autism can look like”.