Screen shot from Gravity

Digital visual effects: shaping the next cinematic experience

As the high-end computing resources needed to generate eye-popping on-screen effects become more available the VFX industry is playing an ever bigger part in shaping the cinematic experience.

This year's 86th Academy Awards honoured some of the strongest contenders in the Best Visual Effects category, including 'Iron Man 3', 'The Lone Ranger', 'Star Trek Into Darkness', and the winner, science fiction thriller 'Gravity', directed by Alfonso Cuarón. The film also went on to win six accolades from the Visual Effects Society. The award was a major accolade for the film's visual effects supervisor, Tim Webber, but also for the creative studio that generated the film's visual effects (VFX), UK company Framestore. It also highlighted the leading position UK VFX studios now occupy, and how advances in information and communications technology have facilitated many of these achievements.

VFX has been a crucial part of movie magic since cinema's earliest days and has always played a part in pushing the capabilities of the technologies that underpin it. VFX is often confused with other aspects of onscreen image processing, such as digital animation and computer-generated imagery (CGI), and the techniques overlap and complement each other to a great extent.

According to the Visual Effects Society, standard definition of VFX would cover the processes by which imagery is created and/or manipulated outside the context of a live action shot. VFX can involve the integration of live-action visuals and generated imagery to create convincingly realistic-looking environments that would be expensive, hazardous, impractical or impossible to otherwise commit to film or video. It also minimises the risk to actors performing a dangerous stunt.

Filmmakers may also turn to VFX to create virtual locations and backdrops: instead of trekking to real places or waiting for specific climatic phenomena to occur, visuals effects can be created relatively simply with the aid of computers. Ambiance can be added to actors performing in front of a green screen studio. The cost savings from avoiding location shooting can prove considerable, and has been another boost to the VFX industry in the straitened economies since 2008. This investment has enabled VFX specialists to invest in the ICT technology required which, in turn, has had a knock-on effect for the ICT industry.

VFX is also a creative enabler. A director's vision may not match the physical tools available to create the film – for instance, the materials to make extravagant costumes may be costly or impossible to make. This was the case in comic-strip-inspired film 'X-Men' where the character Mystique's transformation into Logan was only possible because of computer-aided effects.

Market analyst PWC reckons the value of the VFX industry (worldwide) will grow from $90.9bn in 2014 to $110.1bn in 2018, as revealed in its 'Global entertainment and media outlook' report. Plus, according to movie reporting service Box Office Mojo, in 2013 alone blockbuster movies renown for using visual effects generated millions in profit. 'Frozen' and 'Iron Man 3', for instance, have both grossed over $1.2bn to date.

With the development of high-definition and 3D home cinema, VFX can also play an important part in sales of the DVD and Blu-ray releases of a VFX-heavy title. To date, DVD and Blu-ray sales of 'Gravity' have reportedly reached a combined total value of more than $45m (according to research website

Another driving factor is that VFX technology has become much more accessible to filmmakers with the availability of inexpensive software that runs on non-specialist computer operating systems and computer hardware. Although impressive results can be achieved by using modest desktop devices, for state-of-the-art VFX typical to major movies very powerful workstation and graphics processing compute power is required, and these processes generate massive data sets requiring terabytes of storage capacity.

The availability of high-performance computing (HPC) resources and big data storage resources – both of which can now more-affordably be accessed through the cloud – have also played a part in making VFX more viable for a range of cinematic and televisual productions.

Alongside this are advances being driven by audience demand. With mainstream movies, it's long been argued that VFX can make or break a blockbuster, especially where films are targeting the more socially-active 16-24 age range. VFX is often the compelling feature that audience members will be most thrilled by, and have come to play a key part in word-of-mouth marketing.

The movie business is not the only sector dependent on visual effects, of course: television shows, advertisements, music videos and even low-budget documentaries are regular users of VFX. Much of this work is outsourced to specialist third-party service providers.

However, it's mainstream movies that make the biggest demands on the technology – and spend the biggest budgets – to exploit the use of visual effects. From Hollywood to Borehamwood, the entertainment movie industry is in a perpetual state of innovation in regard to VFXs, with competing productions seeking to out-do each other in terms of their visual achievements. Its range of genres, from fantasy adventures like the 'Hobbit' and the 'Harry Potter' series, through to come-alive comic hero outings like 'Iron Man' and 'Spiderman', also provide new scope for over-the-top VFX that make a tangible contribution to box-office revenues.

Visual effects have always been used in films, but filmmakers now have more technology to play around with. It is impossible to imagine how a Hollywood blockbuster would look without its visual effects – films with iconic moments and surroundings such as 'Avatar' and its fantasy world, 'Harry Potter' and his magic, 'Star Wars' set in space and 'Titanic' with its sinking ship were all completed with the use of VFX.

VFX award-winners

'Gravity''s Oscar success has generated huge interest in VFX, and its potential to help guarantee healthy box office takings – the film has generated a worldwide gross of $716,392,705 (at time of writing) on a $100m budget. The use of VFX has also attracted much media coverage, which helped the production marketing and advertising. 'Gravity' was unusual in that for much of the action, its stars – George Clooney and Sandra Bullock – at times appeared in face only, with their spacesuit-enclosed bodies being computer-generated.

London-based visual effects company, Framestore used a team of 400 VFX experts to create the vastness of space, the Earth, 30 million stars, the space shuttles, Hubble Telescope, the International Space Station, fragments of debris and the space suits.

"There was a stage initially where it was going to be made with actors in real space suits," recalls Framestore's visual effects supervisor Tim Webber. "They would have been hung-up on wires on partial sets, and we would have extended it and put it in the background. In the end considerably more of [the end result] is computer-generated than real."

Visual effects are about recreating reality, and another reality is that this aspect of movie-making has become a vital part in the production process to the point where it is unusual for a film not to employ VFX. While they have been embedded in films, the effects themselves have become increasingly sophisticated and more realistic, to the point that even industry insiders cannot always easily distinguish between the real and the computer-generated.

"The industry has changed out of all recognition," says Alex Williams, co-leader of BA in Animation at Bucks New University's Design, Media and Management faculty. "When I started we were still animating with pencil and paper. Back in 1987, my first job was on 'Who Framed Roger Rabbit?' working as an in-betweener, which involved putting the in-between drawings between the animator's main drawings.

"Nowadays computers give you this for free. The process has been automated, meaning there are huge efficiencies with digital animation, and almost all major commercial work done now is created digitally, one way or another. Saying this, computers give you nothing for free – you will start with a blank canvas or a blank screen."

Behind the scenes

Imagineer Systems develops software specifically designed for visual effects for post-production video and film. Its tracking software, Mocha, follows pixel patterns through difficult conditions, such as objects which go off screen, objects which are partially obscured, and out of focus footage. Mocha has been employed to create both movies and television series, such as 'The Wolf of Wall Street', 'The Hobbit' and 'The Walking Dead'.

"The box office suggests that VFX-heavy films are increasingly popular and they all require state-of-the-art disciplines, such as VFX, 3D animation, and motion capture," says John-Paul Smith, CEO at Imagineer Systems. "The trend has also continued into television, with episodic shows like 'Game of Thrones' or 'Walking Dead' producing high-quality visual effects content."

He adds: "Our software was originally developed for Microsoft Windows, Linux and Silicon Graphics' Irix operating system, which required enterprise-standard computers. Also at this time, emulsion film was being used for [origination] production, and then scanned to digital files for visual effects. Nowadays computers are obviously faster, to the point where our software can perform motion-tracking tasks in high-resolution on off-the-shelf laptops, running on Apple Mac OSX- and Linux-based platforms.

"There are huge budget constraints forcing our clients to deliver high-quality visual effects at an aggressive budget," reports Smith. "This, and the competitive nature of the software industry, have brought a trend in much lower prices for actual software."

The company's core technology is its 'Planar Motion Tracking', an algorithm which tracks patterns of pixels over moving sequences. This data is used by visual effect artists to insert elements into a scene – for instance, the 'moving paintings' seen in the 'Harry Potter' films – or replace a background to follow any camera motion.

Each new production calls for a new twist on established VFX, and pushes its capabilities further, Smith adds: "This trend will continue as we see next-generation filmmakers [showcasing] incredibly high-quality animation and VFX using software such as Mocha Pro, Adobe After Effects or HitFilm. Where 3D animation, effects and cameras were once expensive tools, the availability combined with the free learning tutorials now enables young students the ability to get started in the industry."

Major mainstream movie productions are increasingly the result of global collaborations featuring teams of contributing specialist companies. In terms of VFX collaboration, very large data sets such as a typically VFX-heavy production would be a whopping Petabyte. By way of comparison, according to Data Centre Knowledge, for its last month of 2009 movie 'Avatar''s production, some 40,000 processors were handling data at 7.5Gb/s running 24-hours a day. Each minute of a final copy of James Cameron's movie accounts for 17.28GB of stored data.

"Many of the facilities we now work with have locations around the world and have built efficient digital communications 'pipelines' for the sharing of large files," says Smith. "Shared storage is certainly used, and security is a huge concern for film studios."

Into the cloud

Cloud computing technology is also providing an alternative way for disparate production teams to work on a common project. UK-based VFX company Jellyfish Pictures – behind productions such as 'Doctor Who', 'The Whale', 'Jonah', 'Line of Duty' and 'The Secret Service', has begun to use the cloud to enable its staff to produce animation and motion graphics from any location.

The company recently partnered with cloud company Exponential-e to launch desktop-as-a-service for graphic processing units (DaaS-GPU). Built using NVIDIA's GRID Technology for high-performance graphics-accelerated computing power and VMware Horizon DaaS Platform for Service Providers, Exponential-e's DaaS-GPU is delivered on its secure, cloud-ready 100Gi.

The huge amounts of processing power previously needed to render animated and motion content meant that artists, designers and developers were tied to a physical desktop, says Jellyfish Pictures CTO Jeremy Smith.

"We are fundamentally reliant on creative and innovative technology delivery," Smith says. "Scaling in the picture industry, at speed with security and privacy centre stage, is inherently difficult. The DaaS-GPU solution means that we can be responsive and need not worry about scaling-up compute resources to meet the requirements of new projects."

Mainstream movie audiences are now apt to rate a feature film's visual content as highly as its other attributes, such as its storyline, emotional engagement, acting quality and cinematography. Of course, depending on the genre, for instance action-packed, horror or science-fiction, the expectations for convincing-looking visual effects will be higher. CGI no longer has to look realistic – it has to look real. 

Further information

Sign up to the E&T News e-mail to get great stories like this delivered to your inbox every day.

Recent articles