Written By: Debra Kaufman, Creative Cow
Re-teaming with director David Fincher after their successful collaboration on The Social Network, Michael Cioni and the team at Light Iron built 5K workflows for real time, full resolution post for Fincher’s The Girl With The Dragon Tattoo. The running time of the 4K print is 2:38, with a data size larger than six 2K features combined!
Michael Cioni, CEO of Light Iron, is a champion of 4K data-based workflows. Prior to starting Light Iron, he co-founded and built PlasterCITY Digital Post, a desktop-based post production facility in 2003. Michael has served as a Digital Intermediate supervisor on hundreds of feature films, and provided 2D and 3D data-centric post services and support for many film and TV projects.
As a founding member and instructor at REDucation, RED Digital Cinema’s training program, Michael is a strong proponent of empowering clients through education. He sits on the Board of Directors of the Hollywood Post Alliance and Filmmakers Alliance and was an adjunct faculty member at USC’s Annenberg School of Journalism.
Ian Vertovec, co-founder of Light Iron and PlasterCITY Digital Post, is a supervising colorist. In addition to many music video and commercial credits, Ian has also been colorist on numerous feature films including The Social Network, Goats, and Street Kings 2.
Cioni and Vertovec spoke to Creative COW’s Debra Kaufman about what it takes to work with full-res files larger than 4K in real time — on multiple workstations no less – working with David Fincher, and what frame sizes larger than 4K mean for all of us.
MICHAEL CIONI: To describe what Light Iron did on The Girl with the Dragon Tattoo, I think the best place to start is to talk about our new facility in Hollywood. When we mastered The Social Network last year, we didn’t have a facility, so we worked at RED Studios.
When planning the layout of our new facility, we wanted to build it so that individuals like David Fincher would feel creatively and technically comfortable. Looking towards the technological future as David does, and embracing the demands he places on facilities, we had to get down to the tiniest details to build this facility to be the highest fidelity. We wanted to build a future-proof facility with, for example, a network capable of moving multiple gigabytes of data per second, 4K projection and non-perf projection screens, which look a lot better than the more typical perforated screens.
Most importantly, our facility offers a fully viable 4K pipeline and the ability to master in 4K and beyond. Dragon Tattoo was shot on location in Sweden over 167 days, using the RED Epic MX and Epic, which shot 4.5K and 5K resolutions respectively. The shoot produced 483 hours of footage; they printed 443 hours of footage, which translates to over 1.9 million feet of film in 3-perf. This is among the largest 4K movies ever delivered, if not the largest. At 2 hours and 38 minutes, it consists of almost a quarter of a million frames at 45 megabytes each.
The post team that worked together on Fincher’s The Social Network reunited on this picture: editors Angus Wall and Kirk Baxter, assistant editor Tyler Nelson and colorist Ian Vertovec.
Ian Vertovec: The Girl with the Dragon Tattoo is the type of film we built the facility for. It’s a purely data-centric movie with a very progressive workflow. A lot of people who design 4K equipment benchmark it at 4096; once it gets larger — Dragon Tattoo was done at 4352×2176 — things tend to slow down. This is why we initially designed Light Iron for a larger-than-4K pipeline.
The facility was also designed to be totally data-centric. We’re working very traditionally in the DI theatre with the colorist and the cinematographer and director, all looking at the output of our 2K Christie DLP projector. We did all the color correction off that 2K Christie and then viewed the DCP on a 4K Barco, which is in the same projection booth.
CIONI: A common question I hear is, are we a 2K or 4K industry? Most people say we’re 2K but we’re going towards 4K. That is to say, the technology is in transition. To do a 4K movie now we temporarily need both projectors. The 4K technology hasn’t matured enough to use it exclusively.
VERTOVEC: There are subtle differences between how each projector projects the images, like the subtle differences between Plasma and LCD displays. Even when the content has been captured at 2K resolution, it looks sharper in the 4K projection. We find the Christie 2K projector still has the deepest blacks and the best contrast. I haven’t seen any digital projection that can beat the Christie 2K projector in that category.
But the 4K projector is the only way we can look at every pixel. We also want to know if there is some noisy shot if we’re pushing the limits of the exposure how it’ll look at 4K. It’s diversifying how we view the material and better informing us overall.
CIONI: It’s also becoming very popular for audiences to see 4K projectors even though they’re not seeing 4K content. Sony has sold 17,000 4K projectors, and several theatre chains have stated their intent to switch to these projectors. 2K content looks better on the 4K projector because the distance between pixels is reduced, so the perception of higher resolution goes up because there’s less negative space. Dragon Tattoo was shot one-third with the RED Epic and the rest with the RED MX; these are essentially extremely low signal-to-noise-ratio cameras, very quiet, so they scale really well. So, although Dragon Tattoo will be released in 4K, it’s worth noting that 4K sourced projects that master in 2K scale up well to 4K.
We’re also doing tests for another film we’re starting soon which was shot 3K RAW with the ARRI Alexa. We did the blow-ups to 4K for a 4K DCP output and it looks amazing. New content can handle the blow-up better because today’s cameras start at greater pixel counts and are much quieter now.
VERTOVEC: David Fincher is a very post production-conscious director so he has a very strong post production team that manages the dailies all internally. Because they’re so post conscious, they don’t rely on us for front-end services. They only relied on us for color correction and finishing.
I think one of the most powerful techniques done was an intentional center extraction from the RED footage. The actual frame was probably 75 to 80 percent cut out of the center of the whole image they captured. We color corrected the full 4.5K plate, but only 3600×1500 made up the actual frame. We have almost 1,000 pixels horizontally to do repositions, stabilizations and blow-ups. David was able to come into the DI suite, look at a shot and then say, “…zoom in a little bit” or “pan left” without any resolution penalty.
CIONI: This is a good way for people to think about shooting with high resolution data. With tape it was typical to shoot the full aperture or almost the full aperture and go to post from there. With high resolution cameras, people capture the full resolution. From David Fincher’s point of view, he had enough resolution to spare and used that as a creative tool to adjust the framing with more precision in post rather than when it was shot.
I think there’ll be a trend that people want to follow in that you shoot high resolution full aperture, but only intend to use 75 or 80 percent for finishing. Some people think that makes sense for 3D, to compensate for convergence. But David is saying why doesn’t that make sense for 2D as well? There was no resolution penalty and we didn’t scale down as much as you normally would, so you won’t feel like the film is blown up.
VERTOVEC: There is also another benefit. Editorial did a large number of split screens; Angus and Kirk pick the takes they want for the best performance and Tyler builds a split screen. Sometimes there would be three- or four-way splits. One of the reasons you need that center extraction is to match all the plates together.
The bird’s eye view of the workflow is that I talked to Tyler and we planned to use what they shot on set, unless we need to go back and re-bake it at different ISO settings. We get full reels at this 4.5 K resolution from Tyler, and sub-clip it out into shorter DPX sequences so what the conform in our Quantel Pablo refers to the original camera source time code.
There really are no more standards in terms of frame rates or frame sizes. For the longest time, the DI was only 2040×1586 and people designed tools specifically for that, but with data you can have any frame size and any frame rate. I think our non-standard resolution/extraction combination is the wave of the future. Post people and manufacturers have to be thinking in those terms.
CIONI: All those split screens created an issue, however, that we had to deal with. The connection to the original information for each shot gets lost because, when you do all those split screens and then conform and render it out, they don’t have original file names. What’s the timecode of a shot if it comes from 4 different takes? That level of metadata all goes away.
To solve this problem, Light Iron’s Stevo Brock built an in-house custom app we call Sub-Clipper that allows the Pablo to treat the 4K DPX files as “camera original” footage. Because of the amount of visual effects, Red RAW files cannot be used in the DI. However, the cut may change well after VFX are already processed. With Sub-Clipper, all editing changes and VFX shots can smoothly ripple through to the composited 4K DPX files on the Pablo, even though the offline editorial list refers to an R3D source. It allows the final conform to automatically be in perfect sync with editorial, up to and including the latest revision.
Without Sub-Clipper, it’s like having a picture of North America but not knowing where all the state lines are. With Sub Clipper, we see where all the edits are and the names of the visual effects. Now when we load it into Pablo, it looks like the original files. We see it organized as if in an offline.
VERTOVEC: It’s analogous to a standard tape-to-tape correction suite where you have the timecode of the long-play tape but not the source code or original footage. Sub Clipper reverts the whole DI to a simpler form of itself.
If you do a standard DI, the conform system looks for the camera rolls and loads the shots in. The problem is that we’re destroying the relationship of the clips to the original camera reels. If you move the shot, you’re changing its position in the reel and negating any relationship it originally had in the sequence.
Sub-Clipper re-establishes that, so we can leapfrog the metadata over the assistant editors. We can now spit out all those frames into smaller sequences and then re-stripe the timecode of every single one of those sequences. So we have actual VFX shot names and the actual timecode of the original camera timecode. The advantage is if there were a re-conform or editorial changes – which there always are — we reload the new edit and it’ll just move the shot to the new place in the timeline. With the long play timecode, and
Color correcting The Girl with the Dragon Tattoo in such high resolution was a big deal. Being at 4.5K is about five times the file size of 2K and five times the processing power needed to calculate the color correction. Even if we had tripled our infrastructure and tripled our throughput, it would still be twice as slow as a regular 2K DI.
We’re working at full resolution in the Pablo at all times. The way I work with David, which we did before and was very successful, is that he’ll come in and set key frames with me. So we won’t work through the entire scene. He’ll show me the shots, we work on them together, and then he’ll ask me to match the whole scene to one shot.
This method saves David’s time and allows me to finish a reel unsupervised. Then when he comes in to review the reel that I’ve worked on, I’ll record our thoughts on a Flip camera and use this as my director’s commentary. That gives me a day or two’s worth of notes to address without monopolizing David’s time.
CIONI: The Girl with the Dragon Tattoo is 9 reels long, which is long for a movie. The fact that it’s 4K means the files are 45 megabytes per frame. If you think about that, the original source files from the RED camera are about 40 megabytes per second. So this is more than 25 times larger than the original source file, plus it’s 4K, plus it’s nine reels long. The data footprint of this DI, when you add it all together, is the equivalent of about six 2K 120-minute movies.
It’s really important to understand that when you’re engaging in 4K at this level, the data footprint is huge. It’s not twice or even quadruple 2K. On a linear scale, it’s five times the render, transfer, drives, waiting…there are so many levels that that can bite you. There were areas where it nearly did bite us – and areas where we were totally prepared.
VERTOVEC: We had a traditional structure with an assist station loading files and out-loading files and doing conform operations while I was coloring in another room. Monique Eissing was responsible for loading and prepping all the material, utilizing the Sub-Clipper application to carry over color corrections to newly revised VFX or stabilized sequences.
One pleasant surprise was how well Pablo with Gene Pool, Quantel’s shared storage solution, worked. Gene Pool allows our two Pablos sharing the same media to have 4K or greater playback at all times. We never reviewed anything at less than 24 fps. Most systems struggle to play back even a single stream of uncompressed 4K at 24 fps.
CIONI: With Gene Pool and Monique, we could multi-task. It’s like having two colorists working at the same time. We also enlisted the help of two other additional components; we have multiple DVS Clipsters equipped with 4K acceleration boards that allowed us to encode different types of files in 4K in virtually real time. That helped us tremendously. Also, we had a 10-gigabit Ethernet link between the Clipsters and Pablos.
We also used Shoeboxes, which is Light Iron’s version of a shuttle drive, but on steroids. We could push files around via “sneaker-net” or we could move them at greater than 500 megabytes per second. When we delivered the digital master — all 230,000 frames, properly organized — to Deluxe for film-out, the only way to move that number of terabytes and check it was to use Shoeboxes and a very fat pipe, a SAS connection. The solution is never just one component — it’s a series of steps that need to be planned.
On a purely technical level, the color correction was the easiest part of working on Dragon Tattoo. Delivering this film — which was invisible to Fincher — was the most difficult thing we’ve ever done as a facility: getting the footage into the facility and delivering it out of the facility.
When both Pablos and Clipsters were working — non-stop 24 hours — the facility was playing back 4 gigabytes per second. Our network operated at that level for days and days, and we’re impressed with that. It’s all due to SAS, fiber and 10-gigabit in unique configurations for each step, harmoniously working together
SAS (Serial Attached SCSI) is a protocol based off of E-SATA, but it moves the data 12 times faster than Firewire. It’s like taking four E-SATA cables and threading them together. It’s a very small connector that can push data almost up to 1.5 gigabytes per second on its own (provided the attached storage can support the bandwidth). That’s important for us with the Sony F65 because its files are enormous. And SAS is a connection that a lot of people need to take a serious look at. We’re still surprised that most facilities use antiquated protocols to move data around.
VERTOVEC: People are used to film and videotape, which only runs at 24 fps. If you had to transfer footage from one place to another, it always transferred at real time. With 4K data, and files over 40 megabytes a frame — that’s 800 megabytes a second, and that’s the challenge. Few people have technology that runs at 800 megabytes per second.
CIONI: In a data-centric world, clients have looked into our machine room, saw a couple of Mac towers and thought they could do it on their own. But the complexity of technology has increased. In actuality, their ability to do it themselves is as out of touch as when we printed film. Working in 4K is like 2K was several years ago, only four times bigger and six times more dynamic.
Over the course of working on The Girl with the Dragon Tattoo, I re-learned that every time technology comes out that advances something, an artist will use it. It’s our job as a facility to make that new technology as transparent as possible to the client. Directors like David Fincher will never stop pushing the boundaries, and companies like ours should always operate outside of our comfort zone. We need to keep inventing ways to make this technology as transparent — and available — and empowering as possible.