VR and AR in factories and manufacturing

Read more about all aspects of Virtual Reality technology

Working and collaborating in virtual and augmented realities can boost productivity in every phase of the industrial development process - from initial design to assembly optimisation. Despite the set-up cost, more and more manufacturers are realising that these mind-altering environments provide a huge opportunity to save time and money. Designers, engineers, executive decision-makers and even clients can iterate on designs in real time. Plus, having a virtual, 3D prototype can cut the time, money and materials needed to create a series of physical prototypes.

Virtual reality (VR) is the main approach currently being used. It sees the user completely immersed in a computer-generated reality, or virtual world. Yet if manufacturers want to visualise and manipulate computer-generated content in the real world, it’s augmented reality (AR) that’s needed. This emerging technology offers the user digital information overlaid onto their immediate surroundings.

While this data can be contextualised to the user’s location, or to where the device’s camera is pointed, it’s not the same as being able to see holographic objects pinned, or anchored, to specific physical locations or objects in the real world. This is where future mixed reality technologies, such as Microsoft HoloLens and Magic Leap, are likely to revolutionise the industry.

Getting it right first time

As it stands, most industries that have already taken the step into using VR and AR have implemented the technologies in the field of product design. Balfour Beatty Rail, for example, uses VR engineering for planning and prototyping. Electric car start-up Faraday Future has revealed that its first concept car, the FFZERO1, was created from scratch in just 18 months with the help of time-saving VR and AR design processes.

In response to demand, world-leading 3D design software company Autodesk has a number of AR and VR initiatives under way. Its LIVE Design package combines the company’s existing 3D modelling tools with its games engine, Stingray, to create virtual experiences of buildings, interiors and walkthrough designs. The firm showcased this approach in December 2015 at its annual conference, with users accessing the virtual experience by wearing a HTC Vive virtual reality headset.

“Mixed reality changes every aspect of how you work with your data, environment, peers and customers,” says Garin Gardiner, senior business development manager in manufacturing at Autodesk. “Similar to how we saw the shift to the cloud give us increased collaboration, mixed reality takes this one step further by allowing us to interact directly with the environment in which we’re creating - in 3D.”

Autodesk has recently started working with Microsoft HoloLens - a mixed reality and holographic system - to create a transformative design environment. The hope is that the new Fusion 360 platform will enable collaboration, rapid iteration, digital prototyping, design reviews and client presentations to be improved during product development.

“It’s more intuitive to do this in 3D space,” says Gardiner, “and Fusion 360 will give us a hyper-collaborative environment that can be beneficial for any manufacturer or any enterprise that needs to facilitate collaboration between an industrial designer and mechanical engineer in real time - for example, enlarging the casing on a game controller to accommodate a larger battery.”

The HoloLens team is also working with Nasa, Dassault Systèmes, Trimble and others to develop industrial uses for holographic computing experiences.

Over at car manufacturer Ford, virtual manufacturing efforts are focused on the early design phase of a vehicle, by performing digital analysis before any physical builds are done. The most recent launches to benefit from the technology include the 2015 Ford Mustang, F-150, Edge and 2016 Explorer.

On average, more than 900 virtual assembly task assessments are made two or three years in advance of each new vehicle launch. By virtually simulating the build process, using both human and virtual test subjects, Ford can assess the physical labour that’s needed to build the vehicle, ensure the manufacturing process is feasible, and identify areas which may cause fatigue, strain and injury. The data collected guides engineers towards minimising these stresses.

“The virtual process has been deployed in all regions for vehicle operations manufacturing engineering and powertrain manufacturing engineering. And we’ve seen a 90 per cent decline in manufacturing-driven issues found during prototype build and launch,” says Gene Coffman, technical leader for virtual manufacturing at Ford.

The company uses full-body motion capture with 52 sensors to provide complex data on how one of its so-called ‘industrial athletes’ would move on the assembly line. Then, by putting on a head-mounted display, that person can immerse themself in a virtual world to test processes and movements on life-sized, 3D designs of the vehicle.

“We’re continuously looking for additional opportunities for using VR,” says Coffman. Our current focus is more around identifying gaps in our current virtual capabilities and developing solutions to fill those gaps.”

Jaguar Land Rover uses similar processes to test out manufacturing processes before production begins. The company also has a Multi-Seat Rig - cutting-edge visualisation technology that enables engineers and designers to see and touch new products long before prototypes exist. Sitting inside a physical cabin wearing an Oculus Rift VR headset “bridges the gap between the virtual and physical world” and allows engineers and designers to collaborate from day one.

Virtalis - a long-established British supplier of advanced visualisations - agrees that it has seen clients use virtual technologies to improve communication between departments and personnel with very different backgrounds.

“Say you’re building a submarine,” says company spokesperson Sarah Cockburn-Price. “You could have somebody high up in the Navy who doesn’t know much about engineering but knows about nuclear submarines, you could have somebody who knows everything to do with manufacturing engineering, you could have somebody that knows everything to do with design engineering, you can have somebody else that really knows about materials and FEA analysis, and you could have an ergonomics expert. Put them all together in a room over CAD drawings and only perhaps the design engineers would be happy. But put them all together in a virtual world and you haven’t got the barrier of the specialist language or the specialist skills. And it’s all one-to-one scale so everything is the size that you think it’s going to be.”

Putting the pieces together

Augmented reality is also rapidly making inroads into production lines and day-to-day operations, as it becomes more accessible. The ability to superimpose contextual digital content onto the real world is incredibly valuable as a source of real-time information.

Boeing is embarking on pilot projects that might eventually see augmented reality as an integral part of its aircraft assembly process, for example, by overlaying real components with digital ones.

Elsewhere in the aerospace industry, Lockheed Martin is experimenting with AR ‘smart glasses’ that act as real-time manuals to instruct engineers how to build and repair F-35 fighter jets.

Similarly, at the end of 2015, Volkswagen’s Wolfsburg plant saw the roll-out of Google Glass as a standard piece of equipment. Employees currently use this basic form of augmented reality for order-picking, receiving information such as storage locations and part numbers directly in their field of vision. A barcode reader camera in the glasses then lets the worker know they’ve selected the right item by superimposing a green or red colour over the item’s barcode. The main benefit is that instructions and communication are visual and acoustic, and the employee keeps both hands free.

“The problem with a classic barcode scanner is that you have to hold the scanner in one hand,” explains Leslie Bothge from the Wolfsburg plant. “However, within the picking process you need both hands to pick up one part, so the employee has to put the barcode scanner somewhere. With the smart glass, he’s hands-free and automatically gets the information on where to go.”

Having received positive feedback from staff, Volkswagen is planning to use the smart glasses in other areas of the production process, as well as in sales and marketing.

Virtalis has seen many of its clients starting to expand their use of VR and AR throughout the lifecycle of their products and into other business operations. Many companies that initially buy a virtual solution to solve shortfalls in the design process find that they then use the technology to do ergonomic analysis, FEA stress analysis and to look at manufacturability, decommissioning and even market research and sales.

“They’re taking the kit out to trade shows, with all the same data, and there’s no need to spend more money,” says Cockburn-Price. “Once you’ve invested in the technology for one thing, you might as well leverage that and use it throughout the lifecycle.”

Virtalis has supplied hundreds of its ACTIVEWALL interactive 3D visualisation systems to clients around the world. One is being used by BAE Systems Marine, which has put the power of the technology into the hands of the workers themselves.

Cockburn-Price explains: “When they’re building their submarines, they don’t make people schlep off to the IT suite. Instead, they actually strap a portacabin to the side of the submarine. It’s the most ramshackle thing - just a portacabin stuck to the side of this nuclear submarine! If the welders want a bit of help with knowing what to do next, they just break off from their work, go into the portacabin, have a look at the requisite part, see how it’s done and then go off and do it. They tell me that so many of the things in a nuclear submarine are symmetrical that they could easily get them the wrong way around and it’s so obvious in 3D in a way it’s just not in 2D CAD.”

Learning the virtual ropes

It’s a natural extension to move from systems like this - which optimise worker safety and ergonomics - to workplace training using VR and AR technologies. Replicating work environments in virtual environments allows companies to train their staff to deal with situations they wouldn’t normally be able to train for safely and cost-effectively, for example, in hazardous environments, or areas that are difficult to access.

“We do virtual helicopter crew training for the Royal Air Force,” says Cockburn-Price. “All the crew train together, wearing headsets, inside a virtual world within a wooden mock-up of a helicopter. In the real world, it costs £18,000 per half hour to go up in a Chinook but the four of them can train virtually and rehearse and rehearse and rehearse, and they’re using the landscape that they would do on an actual mission.”

Specialised venues are also centralising cutting-edge virtual technologies to make it easier for engineers to gain access to advanced training. At the newly opened National Training Academy for Rail in Northampton, engineers are learning new skills in the UK’s first dedicated immersive training suite. By viewing detailed 3D models through Oculus VR headsets, trainees will be able to familiarise themselves with engineering processes and safety systems before even setting foot on a real train.

The future of mixed reality

Yet challenges to expanding the use of VR and AR remain. Until it’s seen in action, it is impossible to know whether mixed reality frontrunners such as Microsoft HoloLens live up to the hype.

Every industry will be demanding headsets and hardware that are more robust, more reliable and more flexible than what’s currently available. Ever-improving gesture tracking will help users interact more naturally with virtual and augmented imagery. Haptics - if well-integrated into these technologies - will let users feel a greater sense of physical interaction by receiving feedback as they touch and manipulate virtual objects and holograms. In industry, a culture shift is often needed to give employees the mindset to explore and learn these incipient technologies.

The opportunities are endless. “There’s this great saying I’ve heard one of our clients use,” says Cockburn-Price, “that in the virtual world they can see what can’t be seen and they can do what can’t be done.” What could be better than that?