The first part of director Peter Jackson’s new Tolkien film trilogy is also the first attempt to double an 80-year-old film industry standard: the rate at which images pass before your eyes.
HFR sounds more like a vaccine than a form of entertainment. Trying to woo audiences with an acronym most don’t yet recognise probably won’t prove Hollywood’s smartest piece of marketing - even though there's a lot riding on it. We are, of course, talking about High Frame Rate digital cinema, which arrives on big screens this month with the 3D release of 'The Hobbit: An Unexpected Journey'.
'Lord of the Rings' director Peter Jackson is shooting his latest Tolkien trilogy at 48 frames-per-second (fps). The current filmmaking standard is 24fps for both 3D and 2D. So, if you see 'The Hobbit' as its makers intend, your eyes will be presented with twice as much information. The idea is that, coupled with enhancements to digital cinematography, this will provide an experience that is smoother and more ‘real’.
Jackson explained his rationale recently via Facebook: “In the digital age, there’s no reason whatsoever to stick to 24fps. We didn’t get it perfect [when the standard was set] in 1927,” he said.
“Science tells us that the human eye stops seeing individual pictures at about 55fps. Therefore, shooting at 48fps gives you much more of an illusion of real life. The reduced motion blur on each frame increases sharpness and gives the movie the look of having been shot in 65mm [film] or [the] IMAX [megascreen format].
“One of the biggest advantages is the fact that your eye is seeing twice the number of images each second, giving the movie a wonderful immersive quality. It makes the 3D experience much more gentle and hugely reduces eyestrain. Much of what makes 3D viewing uncomfortable for some people is the fact that each eye is processing a lot of strobing, blur and flicker. This all but disappears in HFR 3D.”
Sounds great. But not everyone is convinced. Criticism first emerged when test footage was shown earlier this year at a cinema trade show in Las Vegas.
The images were, some claimed, more like a brightly-lit soap opera than real life. Others said that the film was so immersive it made them queasy, more theme park than fleapit. More to the point, there were concerns that 48fps is so different that it breaks the suspension of disbelief: particularly troublesome on a tour of Middle Earth.
“A couple of issues with that,” says Don Shaw, senior director of product management for Christie, the company that makes many of the projectors that are 48fps capable. “First, that began with test footage - it hadn’t been through full post-production. But a second big factor was the tough venue, the Colosseum at Caesar’s Palace.”
As a live venue rather than a cinema, the Colosseum has more of a bowl layout.
“So, as is Hollywood, you had far less of a sweet spot. And in that, in the middle, you had all the executives, while on the sides where all the exhibitors who were being asked to look at the format and all the journalists who were going to write about it,” Shaw says.
However, as the first press screenings, premieres and full exhibitor previews now take place, reactions remain mixed. Rival director Bryan Singer ('X-Men') quipped that he was having some “serious frame rate envy”, although the general consensus is that it takes a little while to get used to HFR.
Either way, 'The Hobbit' is a seriously engineered experience. This is the technological state-of-the-art in digital cinema. But what will it do for the art of the movies - and the box office?
Filming 'An Unexpected Journey'
Given that 'The Hobbit' breaks new ground in film production, the degree to which the production team had to compromise is not especially surprising.
The film was shot with 48 of the latest RED Epic digital cameras at 5k (5,120x2,700 pixels) resolution - that’s more than five times the detail from a 1080p HDTV camera. The Epics were configured in 17 different combinations, each with two cameras to represent the right and left eyes.
Because of the size of the lenses, the cameras could not simply be set side-by-side because that would exceed the maximum interocular distance (that between human eyes is roughly an inch). Instead, Jackson worked with 3D camera specialist 3ality to create a system where the lenses were configured to point through or shoot the reflection off mirrors.
Which led to the first compromise. The 17 rigs were needed to provide the best camera configurations for different uses - shooting handheld, from a crane, on a dolly truck and so on. There was no 'one-size-fits-all'.
Then the cameras themselves turned out to have some issues with how they captured images. According to the production’s video blog, they tended to “eat colour”, so costumes, sets and make-up had to be made more garish on the sound stages than will appear on screen (incomplete ‘grading’, which makes colours more natural in the release prints, may explain some of the initially negative reactions in Las Vegas).
At the same time, though, the extra information captured was “unforgiving”.
“Obviously, this [shooting at 48fps] creates twice as much data,” explains Matt Cowan, chief scientific officer at 3D specialist RealD. “Effects shots have twice as many frames, and therefore need more rendering time. Additionally, because there is more clarity in the high frame-rate, fast-moving objects in effects such as explosions are required to have more detail, as there is less blurring. It therefore requires more care in design and execution.”
This raises some big cost issues for any move from HFR blockbusters to mainstream production. Extra effects time, more detailed sets and more realistic stunts all cost a lot more money. Each of 'The Hobbit’s three parts has been estimated at $150m. That's okay when its predecessor took $2.9bn; less enticing if you’re backing a less mainstream movie.
Yet, in some respects, 'The Hobbit' could even be seen as a very expensive laboratory experiment; the film is itself on an uncharted journey.
“Because of the additional clarity available in high frame-rate, lots of testing is being done to determine the best way to shoot HFR images. It covers things like: How fast can the camera pan? How fast can the action be without introducing unreasonable strobing? And so on.”
Beyond all that, the filmmakers also had to make sure that they could still show the film at good old 24fps. The distributor, Warner Bros, is releasing the HFR 3D version in only about 650 cinemas worldwide, mostly in major cities. The full global release will stretch to many thousands. As big as the film is itself, the HFR edition is very much a pathfinder.
“In the case of 'The Hobbit', there were a number of tests done, in particular considering the exposure time - or in film terminology, the ‘shutter angle’ - of the camera,” explains Cowan.
“In normal 24fps-captured content, the exposure time is approximately 1/48th second, which will create a certain amount of blur in motion. Translated to 48fps, this would be 1/96th second which would be a lot sharper. But if the content is too sharp, it will appear to strobe or judder at a lower frame rate. The production team on 'The Hobbit' chose to use a shutter speed of approximately 1/72 for most shots, a 270 degree angle.” To clarify shutter angles, these refer to the rotating film shutter which, for 'The Hobbit', had 270 degrees of the shutter open, or three-quarters of a complete circle. This kind of compromise is likely to be needed for quite some time.
Screening 'An Unexpected Journey'
Making sure that what was captured on set would still look good in 24fps screenings was only part of the challenge presented by exhibiting 'The Hobbit'.
Peter Jackson has worked with RealD, Christie and Dolby, among others, to get enough theatres ready to show his film the way he wants. Even given that he was recreating Middle Earth, that must have been a slog in its own right. So, you can see his point.
Christie’s Don Shaw explains the software-hardware combination that makes 48fps possible using existing digital projectors.
“The Series 2 projectors we’ve been rolling out for about three years can support HFR, although the Series 1 that came before cannot,” he says. “What those Series 2 systems need are, first, one of the software upgrades we roll out every two or three months; and, then, a piece of hardware called an Intermediate Media Block (IMB).”
The IMB is interesting; almost the projector equivalent of the specialist processors and on-chip memory cache found in today’s advanced semiconductor system designs.
“Before, the server outside the projector provided two functions in addition to storage: it decompressed the JPEG 2000 images and decrypted the file and then transferred it to the projector over an HD-SDI [serial digital interface] cable. The problem with 48fps is that there is two times the data. That means you have a bandwidth issue,” says Shaw.
“The IMB now takes the decryption and decompression inside the projector, and avoids that bottleneck.”
Architecture, though technologically a challenge, was still a comparatively simple problem. Welcome again to uncharted territory.
“There are no standards yet,” says Shaw. Indeed, while Jackson has shot his film at 48fps, 'Avatar’s director James Cameron is looking at 60fps for that blockbuster’s sequel. “So we have been working with Cameron since he started on HFR, and we engaged with Park Road Post, Jackson’s post-production company, from the beginning there too.
“The first thing was that they needed a good system for dailies as they were shooting.”
The one certainty, then, is that things will change.
Continuing 'An Unexpected Journey'
The frame rate debate goes beyond just Cameron and Jackson, undoubted pioneers that they are, although 60fps may well be the viable ceiling.
“Other organisations have shown tests at high frame-rates. Fraunhofer in Germany has demonstrated 120fps, [special effects pioneer] Doug Trumbull is pushing for 120fps and the BBC did tests a few years ago at 300fps,” says Matt Cowan.
“But from a fundamental human perception point of view, there is a phenomenon called the ‘critical flicker fusion frequency’. This identifies the repetition rate that appears to be continuous and is dependent on a number of factors, such as image brightness, and the individual. It tends to be between 55 and 60 frames per second for film brightness.
“So, the bottom line is that higher frame rates are a matter of diminishing returns. While it would be possible to produce a piece of test footage that would show better at 300fps than 60fps, the difference would be marginal in overall performance. The implications for shooting 300fps - covering processing through editing, effects, and so on - would be five times those for 60fps and 12 times those for 24fps. And the installed population of projectors capable of 60fps is not capable of 300fps. There are practical hardware limits.”
So, in addressing many of the fundamentals, Peter Jackson does take his industry a long way down the viable HFR path, albeit with one exception.
“There is one thing. HFR does help 3D. It’s smoother. It’s more realistic. But with 3D, you are still throwing away 85% of the light, and audiences perceiving 3D as ‘dark’ is one of the main complaints,” says Don Shaw.
“So, if you have a projector that’s rated at 14 foot lamberts (the exhibition industry standard for luminescence), with 3D what you’re getting is just around three.”
The next big step, assuming HFR does coalesce in the 48-60fps region, is laser projection, Shaw argues.
“We’ve done some spectacular test devices there,” he says. “We got to 60,000 lumens at IBC this past September, and more recently to 72,000 at a show in Beijing.”
In this regard, he worries that the film industry, for all the advances 'The Hobbit' offers, may “have cashed in on 3D a bit early”.
“Perhaps this film will change things, but if you look at the actual box office post-'Avatar', the number of 3D movies going into cinemas went up considerably, but the actual number of people going, paying the extra two, three or four bucks, went down. The novelty has worn off.
“Of course, if 'The Hobbit' is a massive success in its HFR screens that will change. By early 2013, you’ll either have a lot of directors talking about using HFR and 3D or you won’t - and it’ll all be about the numbers.”