What if the Internet could tickle yet another of your senses, not just eyes and ears but also touch? With the data rates of mobile broadband soaring, cutting-edge scientists are starting to build the tactile Internet.
It’s just after 3pm, and Indonesian tycoon Anthony Salim turns brusquely to his energetic companion with black ruffled hair. “Can you believe it – every time my private jet needs servicing, I have to fly it to Texas,” he moans, rolling his eyes. “There’s no private jet servicing in Asia. None. Mischa, can you do something about it?”
Mischa is Mischa Dohler, who at 38 had just become the head of the Centre for Telecommunications Research at King’s College London – and one of the youngest professors in the UK. But when his Indonesian friend mentions the private jet dilemma of Asia’s rich and powerful, he certainly doesn’t expect Dohler to jump up and say: “Yes, actually, I think I can.”
This conversation, back in October 2013, provided a spark: What if we could build technology that helps service private jets on location in Asia, without a full-blown service facility? What if we could teach a child to play the piano when she’s in Norway, but you stay in Singapore? What if a skilled surgeon could carry out a delicate procedure on a patient, but she’s in a makeshift hospital halfway around the world? Or more mundanely, what if a shopaholic could feel the texture of that blue (or is it golden?) dress before buying it online, or a grandmother could tenderly touch her granddaughter’s blonde curls, while talking to her on Skype?
It’s called the ‘Tactile Internet’, a term first coined by Gerhard Fettweis, a professor of mobile telecom systems at the Technical University of Dresden in Germany, and former colleague of Dohler. Professor Fettweis and his team had started thinking about using touch to control robotic systems remotely and in real time as early as 2012.
However, Dohler wanted to go further than remote control. “My vision was to ‘close the data cycle’ – the need to actuate on the data you get,” he says. In other words: we should be able not just to see and hear things far away, but also touch and feel them; to transmit accurately the equivalent of human touch using the bits and bytes of data networks.
Mischa Dohler hopes that advances in robotics and 5G will lead to not only to transferring data but also manual skills through the Internet
To make it work, data rates have to get faster overall – much faster. Current research into 5G – the successor to today’s 4G mobile communications standard – aims to reach download speeds that are thousands of times faster than today’s average 4G, although the technology may only materialise in five years time, or longer.
Other missing elements are the devices needed to encode and decode touch, one of the senses least understood. Once that’s in place, next-generation autonomous robots could be guided from afar, such as “drones doing Internet-controlled agricultural harvesting, a robot in a car factory steered by an average Joe to custom-make a cool and affordable convertible, and bots in a burning building giving firefighters not only visual but also tactile feedback about the situation inside,” says Fettweis.
The challenges ahead are huge, but researchers and companies have started taking a keen interest in the idea. After Dohler spoke at this year’s Mobile World Congress in Barcelona, ‘tactile Internet’ has quickly become one of the event’s buzz words. Swedish telecoms giant Ericsson just signed a deal with King’s to investigate the potential of the technology, and the South Korean electronics firm Samsung believes that 5G can indeed make Dohler’s futuristic ideas come to life.
Curious pianist
For Mischa Dohler, bouncing through the hallways of King’s College in his trademark red shoes, with a neatly groomed stubble and stylishly messy hair, it could have been a very different career, of course. “I nearly enrolled into the Moscow conservatory to become a professional pianist,” he chuckles. “But my mum asked whether I’d be happy playing in bars if I don’t become the best. I said I wouldn’t.”
Making a complete U-turn, he majored in physics at Moscow State University instead. When he moved to Germany, his physics degree wasn’t recognised by TU Dresden’s physics department – but the engineering faculty accepted his papers on the spot. “I became an engineer in a day,” he remembers.
After working at the university for a while, Dohler embarked on a career-driven tour of Europe, swapping Dresden for London, London for Grenoble, and eventually moving to Barcelona, where in 2008 he co-founded Worldsensing, a smart city start-up trying to use sensors to make traffic – and parking – cleverer.
That was also the year he started to think beyond 4G. While chatting to Ze’ev Roth, the then-CTO of global telecoms provider Alvarion, “we suddenly realised that already then, in 2008, 4G was not going to be enough, not for smart cities, not for the overall smart world,” Dohler says. So they went to the European Commission and applied for funding to show that 5G was the future – and that it could be achieved.
In April 2012, Alvarion and partners, among them Dohler, set a world record, reaching a tenfold increase in mobile broadband infrastructure capacity. Their $6 million Beyond Next Generation (BuNGee) project achieved a ‘pre-5G’ throughput of 1Gbit/s per square kilometre at Alvarion’s premises in Tel Aviv. (A speed of 1Gbit/s allows you to download a standard high-definition movie in about a minute.)
5G mobile network will replace today’s 4G standards – but exactly what the infrastructure will look like is still unclear
The demonstration confirmed that high-speed mobile Internet solutions were commercially viable. “This was a huge deal, and I was ecstatic,” recalls Dohler. Not long after, in January 2013, came a phone call from King’s College in London – and he was offered the post of Chair Professor in Wireless Communications, the perfect place to expand on his visions for 5G.
The next frontier
Techies know that if we want to make the world smarter and more connected, we first need to speed up today’s data networks and make them all-encompassing. And the benefit, of course, will be more significant than that often-cited fridge telling you that it’s about to run out of milk.
The much-hyped Internet of Things, where more and more everyday objects have energy-efficient sensors, smart components and connectivity to talk to us and each other, has indeed a huge potential: It’s about smart traffic lights that instantaneously switch to green to make way for fire engines rushing to a blazing fire. It’s about driverless cars that communicate with each other to avoid crashes and congestion. And, further down the line, it’s about a doctor pairing with a soft robot miles away to perform an abdominal palpation during an Ebola crisis.
However, all these scenarios depend on ultrafast connectivity; there simply can’t be any perceptible delay in relaying the data – not more than one millisecond (ms). The few seconds of waiting for your Skype image to unfreeze is annoying but tolerable; for a surgeon and her patient, it could be a catastrophe. And if we relay audio, video and touch together, any time-lag between the virtual picture and human movement that’s more than one millisecond could lead to ‘cybersickness,’ says Professor Fettweis; a user would become disoriented, a bit like feeling sea-sick.
Researchers are working on creating artificial touch by embedding sensors into soft robotic structures
Promising to minimise this lag are fifth-generation mobile networks, which are set to be first trialled in 2020, at the Tokyo Olympic Games. 5G is obviously the successor of 4G and 3G networks, although it will be “very, very different – but at the moment no one has a clear idea how exactly it will work,” says John Cunliffe, CTO of Ericsson UK and one of the leaders in 5G development. That’s not least because there are no agreed standards for future 5G networks yet.
What we do know is that 5G networks should be able to provide wireless data connection with an astonishing 10Gbit/s peak data rate. Samsung believes it can push this to up to 50Gbit/s. The South Korean telecoms firm is, alongside Huawei and Ericsson, one of the industry leaders in developing 5G technology, as well as numerous research labs like the University of Surrey’s 5G Innovation Centre (5GIC). So far, Samsung has achieved 7.5Gbps while stationary on a 5G network and 1.2Gbit/s in a car moving at more than 100km/h. More recently, researchers at 5GIC have reported a breakthrough of 1 terabit per second (Tbit/s) – but this claim has yet to be peer reviewed.
Transmitting touch
So let’s assume we get 5G going and you’re happily downloading the latest blockbusters. For the tactile Internet to work, there are still pieces missing.
We also have to find a way to encode touch, transmit the data, and then enable the user to receive the sensation. “5G can facilitate the 1kHz response needed for tactile sensing, but the human sense of touch is extremely sensitive and it has been extremely difficult to build accurate touch sensors,” says Ken Goldberg, a roboticist at the University of California, Berkeley.
A team led by Calogero Maria Oddo, a bio-roboticist at Sant’Anna School of Advanced Studies in Pisa, Italy, experiments with sensors built using tiny micro-electromechanical systems, or MEMS – powerful machines that are just microns in size. The sensors are integrated into soft artificial tissue mimicking a fingertip. “When you touch something, your hand ‘encodes’ the strength of the touch and the qualities of the touch, or the texture. Our sensors can do the same, to a degree, understanding many different materials and differentiating between things like cotton, wood or marble,” says Oddo.
In the future, Dohler would like to see such sensor data being uploaded to the cloud, and from there transmitted to a user with a haptic display made of tiny actuators – effectively pins and needles pressing into the fingertip to reproduce the original sensation. “That could be a glove, or a second skin-like flexible exoskeleton, or some other kind of user interface like a joystick integrated with mechanical actuators,” says Kaspar Althoefer, a roboticist at King’s and close collaborator of Dohler.
Lunches at King’s are often spent brainstorming possible applications for the tactile Internet, for example the above-mentioned doctor in London palpating the stomach of that far-away patient. The doctor would touch a dummy with rubber organs, and her hand movements would be passed on to a robot on the other end to palpate the patient. In turn the doctor would wear a special glove with actuators and ‘feel’ the patient, thanks to the robot pressing on the real organs.
The problem is that such advanced robots don’t exist yet. Robotic touch is simply not soft enough, and their sensor can’t yet differentiate between the different haptic sensations given by various organs. “A robot may find it difficult, because in a body there are multiple layers of organs and muscles that contract and then stiffen. The human brain is capable of filtering all that out and just understanding where, say, the kidney is. So we have to solve that problem first,” says Thrish Nanayakkara, another roboticist at King’s.
Kaspar Althoefer at King’s College works on robots that in future will be better at replicating human touch and dexterity
The doctor’s receiving glove with actuators is the other side of the story. The leader in developing such ‘smart skins’ is the Harvard University team of roboticist Conor Walsh. Their biologically inspired Soft Exosuit is made of flexible functional textiles that mimic the action of the leg muscles and tendons. The suit contains a low-power microprocessor and a web of sensors that continuously monitor the suit tension, the position of the wearer, and so on – and, says Nanayakkara, could be perfectly adapted for someone to receive a handshake from the other side of the globe.
Touching a mummy
While we’re waiting for truly autonomous soft robots to hatch, such smart suits or at least smart gloves could help the tactile Internet to take baby steps in areas such as gaming. Say you’re wearing a headset such as Oculus Rift that shows you a 360-degree virtual environment as it follows your head movements. A smart suit could provide tactile information – so if you hit a virtual wall, you’ll feel it for real, “experiencing tactile information of ‘force’ as you move around in this space, virtual or created from the other side,” says Althoefer. “If in a video game someone shoots you, you’d feel it.”
The key will be ensuring that all of the stimuli are perfectly in sync and that there’s no time-lag. “Once your senses lose alignment, the experience can quickly become disorienting,” says Grant Martin, head of marketing at Avegant. Based in Ann Arbor, Michigan, this start-up has developed a headset called Glyph that projects images on to the retina using two million tiny mirrors in a narrow reflective band. Combined with tactile Internet, virtual reality could go way beyond gaming – into educational and social experiences, remote sensing and physical therapy, adds Martin.
The e-commerce applications of tactile Internet are also obvious: before buying something, you’d be able to touch it, perhaps even put on a virtual t-shirt and see in the headset what it looks and feels like on you.
And how about the sensation of touching or even moving around an Egyptian mummy? London’s Victoria and Albert museum has just submitted a funding bid to explore the use of ‘haptic glove’ tactile Internet technology in a museum setting. “The idea that people may soon be able to take more of our collections into their own hands – even if an object is in deep storage and they are living in another country – is truly thrilling,” says Bill Sherman, the museum’s head of research.
But it will still be years before all the pieces come together that will turn Mischa Dohler’s vision into reality – or, as he puts it, will shift “from an information delivery Internet to a skillset delivery Internet.” 5G could be with us in five or eight years. The right sensors? A decade, give or take, says Oddo. The haptic feedback smart gloves or even suits? We could be looking at even longer than that, says Nanayakkara.
One thing is certain, though. Sooner or later, Dohler and his fellow researchers around the world will be closing the data cycle and creating the tactile Internet. And both patients in remote locations and the planes of Asian billionaires will get the care they need.
Sign up to the E&T News e-mail to get great stories like this delivered to your inbox every day.
