Beyond the green screen

For years, movie magic has been bought to life thanks to the power of the green screen. Whether it’s the final battle in a fantasy film or a single actor playing identical twins in a comedy, this technique suspends our disbelief and immerses us in a vivid and brilliant make-believe world.

Used as a tool for chroma key compositing, a technique for layering two images or video streams, the green screen is often used to reduce the cost of building sets and to avoid sending crew to faraway locations. Technologists, however, are now turning to large-screen LED displays and augmented reality (AR), which is likely to replace, or at least alter, the use of the green screen.

With the Covid-19 outbreak, however, our everyday lives have come to a halt. Could these immersive environments help us stay informed about the world, while also catering to broadcasters and those working remotely?

SmartStage, a system built by audio-visual company White Light, is an immersive video environment which allows presenters and viewers to see and interact with the content around them.

“This technology was conceived to facilitate creative minds in the live broadcast industry, specifically sports analysis,” says Andy Hook, White Light’s technical solutions director, “but it quickly became apparent that it could be used as a viable alternative to green screen presentation, especially for those outside the broadcast industry.”

While in lockdown, the company has been focusing on ways to help its clients improve on working at home, with its installations and systems design teams deploying remote-working solutions. “As a lot of companies are now doing meetings in Zoom, people are starting to realise how difficult the technology can be,” Hook explains. “They need good audio and video quality, good lighting, particularly at the corporate end.”

Camera tracking systems feed the positional data and field of view of the camera into the disguise media servers, allowing content to be generated in real-time to the perspective of the camera. Content for the LED walls, 360° set extension and any AR foreground objects are created and tracked as one synchronised, gen-locked system.

A physical LED video environment is created to display the content for the presenter and audience, using specialist LED panels designed as the result of extensive R&D. This environment can be standalone or integrated as part of a studio set.

SmartStage is rendering-engine agnostic, which means it can bring in content from popular gaming engines such as Unity and Unreal, while also harnessing the power and flexibility of real-time graphic tool Notch, which runs natively on the disguise servers. According to White Light, by using these real-time rendering engines, content and resolution are “completely scalable” as nothing needs pre-rendering.

Studio lighting is closely integrated and can be included as an extension of the digital canvas for a fully immersive environment. As well as naturally lighting anyone in the space, the studio lighting can also be replicated in the virtual world and controlled by the studio lighting desk, blending the real and virtual environments.

In autumn 2019, White Light collaborated with the University of Michigan’s business school (Michigan Ross) where they used SmartStage to create an immersive virtual classroom solution for a new online Masters in Business Administration programme. From this ongoing project, in which the lecturers and seminars are conducted over Zoom, it became apparent to the White Light team how remote communication would work in this space. “One thing that became quite clear was what the remote experience would look like, and what those remote participants would look like when they’re brought into the call,” Hook explains.

‘This is more about replacing the green screen and being able to create a virtual world.’

White Light is currently working with Michigan Ross to further improve on this virtual classroom experience. Hook says, however: “Those conversations we’ve been having with Michigan Ross are the daily conversations we’ve been having with broadcasters and various other people who are bringing all of these remote participants into calls, and trying to make them look better on camera.”

The firm has also deployed lighting kits to clients – these kits are controlled remotely by experts at White Light to ensure that participants are lit properly. “We have a flex system, which people can just plug into a Zoom call and get much better audio quality, much better video quality to the conferencing platform,” Hook adds.

Although the focus is on remote working, Hook says that the SmartStage set up – based in Wimbledon, London and built for R&D and demos – has been used by clients who want to come in and do broadcasts. In fact, at the end of March, Methodist Central Hall Westminster (MCHW) delivered its first live church service, which was streamed via SmartStage, enabling MCHW’s deputy superintendent minister, Reverend Tony Miles, to engage virtually with an international congregation of over 3,000 people. “The reverend walked into SmartStage and people from his congregation appeared around him in Zoom and he could talk to them, present the service and then leave,” Hook explains. “He didn’t have to interact with anybody or see anyone – and that worked really well.”

Hook says that this was a “perfect example of where you see better connection and engagement with the audience and the content”, adding that the company has also had conversations with companies in the leisure and gym space, where fitness instructors are doing virtual workouts.

With remote working now a part of our everyday lives, Hook believes that, once the pandemic phases out, various industries may begin to question their way of working. “I don’t think the world will ever be the same again after this outbreak,” he says, highlighting the events industry in particular.

“Where the entire event industry has been shut down overnight, when it comes back organisers are going to question whether they can do this virtually instead of doing every single event physically.”