'Minority Report' promotional rendering

Touchscreens: future technology goes beyond the glass

Designers are stretching the capabilities of capacitive touch as they inch their way towards fully gesture-based interfaces.

When it hit movie theatres in 2002, ‘Minority Report’ tried to convey a world 40 years into our future with billboards that respond to people as they walk by and a much-copied giant transparent computer display that responds to touches and mid-air gestures.

We are, luckily, a little way from billboards that know our names and shout them out. But the big panel that will understand the gestures we make when we wag our fingers at it? That is not far away at all.

The impetus is not coming from movies and TV shows. Technology is more a follower of real-world fashion. The driving force is, right now, the consumer electronics industry and the current fashion leader in that environment: Apple. It does not take long for success in the consumer field to translate into demand in other sectors. Car makers have begun to ship high-end models that incorporate touchscreens into their dashboards. And industrial control manufacturers have embraced increasingly complex forms of touch, such as the multiple-touch and gesture support found in the iPhone.

Ian Crosby, sales and marketing director at Zytronic, says: “Six years ago multitouch didn’t really exist in commercial and industrial markets. It was more just about buttons – pressing them to switch on or off. Then, after Apple introduced gestures for pitches and zooms, that sparked a rapid adoption in industrial markets.”

The mainstay of the industrial touchscreen and for many smartphones up to then was the resistive touchscreen. But it has drawbacks when it comes to creating user interfaces.

“The idea of multitouch with gestures was something that was never realised with resistive technology,” says TouchNetix managing director Chris Ard.

After the iPhone launch the consumer electronics industry jumped onto projected capacitive and it became the main technology for touch. “That didn’t affect us directly but what we did see was that more of our traditional customers also started to become interested in the benefits of projected capacitive touch,” Crosby says.

Simon Duggleby, product marketing manager for semiconductors at RS Components, says industrial designs have seen a migration away from traditional industrial user interfaces. “There was a time when everything used a seven-segment display with lots of buttons and knobs. But many have shifted to a display. People have gone from appreciating tactile buttons to almost not wanting them. They want a more seamless product.

“One of the other benefits for the professional, industrial customer is that it makes it easier to achieve a given IP rating for a sealed unit that needs to go into noisy and dirty environments, as well as medical situations,” Duggleby explains. “On the user side, people liked the feedback from screens. But they were not comfortable interacting directly with them. That has changed. People go up to a screen and expect to interact with the display directly. When they can’t, they start looking for a mouse and keyboard. It’s strange how expectations have moved so quickly.”

Traditionally, industrial user interfaces that use touch have favoured resistive or infrared technologies because they are mostly unaffected by moisture and still work when users are wearing thick gloves. Even here, the capacitive technology has encroached. “RS sells a range of gloves that work with capacitive,” Duggleby says. “And there is even paint to put on gloves to make them work, too.”

Fine tuning

One problem that has faced anyone who has splashed water onto their smartphone is that the touchscreen controller can become confused – registering phantom touches or not picking up any.

“When they first came out there were issues,” says Mike Logan, displays and input technologies manager at AndersDX. “But there are ICs that can overcome those problems.”

Logan points to a design put together for a coffee machine, which places a colour touchscreen above the nozzles that dispense the drinks. “You are going to get steam, and condensation forming on the display. At an early stage of development, it was upsetting the user interface. The good thing with this type of technology is that a lot of it is down to the software. The touch-controller supplier was able to write code that could recognise that it was just water. It all comes down to the fine-tuning. You will take each case as it comes.”

For systems that cannot tolerate false touches, TouchNetix developed its own auxiliary sensor interface that detects the tiny movements of a cover screen as the finger pushes down on the screen. The company has built up a business selling into agriculture machinery and marine designs.

“Our touchscreens behave flawlessly and they’re expected to because people die on a piece of agricultural equipment if it has a huge saw on it and it malfunctions. On a mobile phone, you lose an address and that’s all,” says Ard. “We built this ground-up with industrial in mind and developed our own chip. Our design has 100 per cent dirt and liquid tolerance: we can seal this thing completely.”

Although capacitive has become the mainstay of the touchscreen business, it has not replaced all other types. Logan says: “We still see projects that see resistive as the touch interface of choice. A lot of companies still use it because they are familiar with the features. But it’s very much for small monochrome or low-resolution displays. A lot ask whether it’s worth moving across to projected capacitive, as it may mean changing the software they have in place. When people start on new projects that require a touch interface, as nearly all of them do, they prefer capacitive.”

The capacitive touchscreens do not necessarily have to be for animated graphic displays. Logan says a number of the designs that his company works on involve custom printing and discrete touch keys that may surround a smaller display, much like older keypad-based designs.

Duggleby says some customers want to upgrade their traditional keypad and seven-segment LCD interfaces to graphical touch but keep their existing software. “The internal architecture of a product might be fine. But the expectation is that systems now have a touchscreen,” he says. This has led to the development of specialised touch controllers, such as the Eve series developed by FTDI Chip.

Gordon Lunn, FTDI’s engineering support manager, says: “We have been having a lot of success with companies who have existing seven-segment display solutions. The core functionality of the product they have runs on an 8-bit microcontroller but they want to get up to a colour TFT. They are worried that they are going to have to throw away their code with a move to a 32-bit processor with graphics. With this you just create a display list and Eve will handle all the graphics.”

Manufacturers of industrial systems expect Apple’s latest attempts to push capacitive touch further to have knock-on effects on their designs.

“The one we are starting to keep an eye on is force touch,” says Crosby. “It’s not really pressure sensing. It’s more that the screen detects the surface area of the fingertip. We can do that because it is not so much an electronics change as a firmware development. I can anticipate customers starting to ask for it once it gets mainstream in the consumer field.”

One application for Zytronic is in automated teller machines (ATMs) where the manufacturer wants to replace mechanical buttons with just a touchscreen. A light touch might act to trigger a beep to confirm that the user’s finger is on a virtual button, or magnify the text on the button, with a harder press activating it.

Ard says the TouchNetix design naturally lends itself to force touch. In this case, the force indication comes from the additional sensor in the rim of the cover glass. “When it comes to force sensing, Apple has been doing the marketing for us.”

Not everything the consumer industry introduces is an instant hit. Ard points to the ‘hover’ feature in some designs that detect when the user’s finger is close to the screen but not actually touching it. The idea is that the screen would light up as the finger approaches or zoom in on a group of icons. “In reality, nobody seems to use it. Most people turn it off as soon as possible,” he claims.

A possible application for hover sensing is in automotive dashboards as it could detect gestures such as a swipe or a circular wave of the finger to alter the music volume or skip to the next track without forcing the driver to look away from the road and land their finger on a particular part of the screen.

BMW has incorporated the idea into the AirTouch interface of the concept car it showed at January’s Consumer Electronics Show (CES) in Las Vegas. With this system, the driver uses gestures to move through menus, but confirms each action using a button on the steering wheel.

Free-space gestures

Another use is in appliances such as cooking hobs. “Do I really need to press buttons while I’m baking something and my hands are wet or covered in flour?” asks Andreas Guete, global marketing manager for human-interface devices at Microchip.

Microchip has developed a sensor technology based on the changes in the electric field around copper tracks on a PCB as objects move in and out of range. One implementation, tuned for low power, provides 2D gesture control for gadgets such as wireless headphones. The user changes volume, for example, by touching a side of one of the headphones and then sliding a finger up and down. The 3D version of the GestIC technology extends the sensing into free space: up to 20cm away.

“The customer either wants to touch the interface or not have to stretch out at all. Hovering at close range needs a lot of hand-eye coordination, which could be a nightmare in automotive systems. How do you know your finger is hovering just above the screen without looking?” Guete points out.

The combination of free-space gestures, multitouch and force touch point to a situation where the user-interface style presented as a future vision in ‘Minority Report’ is moving towards becoming a near-term reality.

The issue of screen size remains. Even there, capacitive is making inroads despite initially looking unsuitable for the job. Crosby points to the indium tin oxide (ITO) conductor usually printed onto the front of the touchscreen as the main problem. “ITO is pretty transparent. And you can [refractive] index-match it so you make it effectively transparent. The difficulty is that electrically speaking it has high resistance, which means you need to drive a high charge on large screens. Most touchscreen manufacturers using ITO top out at around 25in, although some have taken that up to 46in by pumping a lot of power into the matrix and doing things like tiling,” he explains.

Swedish company FlatFrog decided to move to a different technology that uses the way light reflects back inside the glass beyond a critical angle if not intercepted by something touching the surface. But, again, capacitive has been tuned to deal with the ITO problem.

Zytronic uses an array of very fine copper wire – just 10µm thick – to relay charge across the touchscreen. The much lower resistance of copper lets the screens scale up to more than 100in, Crosby says.

Such large screens today go into touch-tables for use in banks and stores so that customers can sit down and flick through a catalogue or salespeople can work through options for a loan or a new car. Museums have been among the earliest users of this type of display.

The integration of different touch technologies is part of Microchip’s plan for its gestural interfaces and Zytronic is pulling in other functions, such as adding cryptographic controllers through a partnership with Cryptera that will let it make payment terminals that comply with the banking industry’s security standards.

Duggleby says some customers have asked about the potential for touch on large 4K display, an indication of how much some want to push the technology. “The technology in Minority Report seemed almost miraculous at the time. A little over a decade later we’re almost there.”

Recent articles

Info Message

Our sites use cookies to support some functionality, and to collect anonymous user data.

Learn more about IET cookies and how to control them