Thursday, 25 November 2010

My Welcome Intruders

I have recently finished the live action shoot for DJCAD animation undergraduates Daniel Borg and Michael Wilbourn's animation short 'My Welcome Intruders'. After seeing their great character designs and previous cg and match moving work I jumped at the chance to collaborate and came on board as live action director, editor and post production consultant.

By the time I joined pre-production they already had a script and storyboards but I was able to suggest changes to make the action more fluid and stage the shots in a visually interesting way. I brought on Ian Forbes as director of photography, our established working relationship gave us the short hand to work quickly and deal with the framing of characters that weren't yet in the shot. Fortunately Dan and Mike had made scale cardboard cut outs of the characters which gave us reference for framing and focus in rehearsal, we would rehearse several times before takes to make sure we had the reference we needed when the camera was rolling. Unfortunately we didn't have a monitor so having Dan and Mike sign off on the shots just from viewing the back of the camera was less than ideal, especially with our small location.

We agreed to use a canon dslr to shoot the piece with, this method comes with distinct advantages and disadvantages. The benefit of using a dslr is the cinematic look that can be achieved with using a large sensor for shallow depth of field and the small form factor which allowed us to make the most of the small location we were filming in. We also used smoke to give atmospheric depth to the shots while making sure that the smoke was even as moving smoke would look out of place and be tough to composite over the cg characters. Shallow depth of focus and atmospheric haze are great depth cues which can be matched for the cg characters in Nuke by exporting a camera depth pass from Maya for the characters.

The problems with using a dslr are twofold. Firstly the camera uses a cmos chip which exhibits rolling shutter artifacts, this causes the image to skew horizontally when the camera is panning or objects are moving through frame at a fast rate. If rolling shutter is not corrected then 3D camera solves can be broken by unreliable world space. Fortunately the Foundry make a plug in for Nuke which can correct this artifact. The second issue is lens breathing, this happens when using stills lenses for video. When pulling focus on most stills lenses there is a small change in the focal length which creates a subtle zoom, this is not usually a problem for live action however if it is not matched in a cg composite it can cause drift. Fortunately match moving software can account for this artifact by allowing the camera to be solved as a variable focal length.

In order to make post production easier we took reference stills, marked the focal length for each shot and occasionally placed tracking markers when Dan felt any shot might be particularly tricky to solve. I took a single 360 spherical panorama for the main room of the shoot but this was not a HDRI and was for reflections and lighting reference rather than a final gather lighting pass. Unfortunately, as I was acting as live action director, focus puller and VFX supervisor the third duty suffered a little causing me to forget to obtain a lighting reference for each shot. However I do feel we have enough reference in the footage and in additional stills to allow Dan and Mike to match the lighting well. I will also be encouraging the guys to render a normal pass and a point position pass which will allow us to add additional lighting tweaks in Nuke without resorting to rerendering.

Now that we have a near lock the live action edit, the guys can begin to 3d track the shots and begin the layout process, blocking out the characters within the scene before moving on to the articulate animation. Once layout is complete these shots can be reassembled in the edit to be sure that we have continuity of motion throughout the film.

Once animation is complete we plan to create multiple render passes to allow more flexibility in matching the characters to the background plates in the composite. Maya 2010 introduced the render passes function for Mental Ray so the guys should be able to automate this process somewhat. In addition to the regular passes such as beauty, colour, diffuse, specular and ambient occlusion etc. I would like to use information passes such as a z-depth pass, a 2D motion vector pass, a normal map pass and a point position pass. These will allow many rendering intensive tasks such as motion blur, depth of field defocus, and lighting tweaks to be achieved in the composite using Nuke rather than Maya. Nuke is faster at rendering these things as well as having the ability to render changes on the fly. The compositors working on the film District 9 were early adopters of a technique which can be used in Nuke to add new lights and basic shaders as well as creating depth based mattes by generating a point cloud recreation of 3D geometry
(see video). This technique requires a point position pass and a normal map pass to generate a cloud of faces which mimic the original geometry and can have lights and shaders applied to them.

I plan to composite a shot or two myself to give myself, it will be really good to see the process through from production to final comp. I already feel like I've learned a great deal from Dan and Mike and I can only see that continuing as the post production gets into full swing.

Final shout out goes to Mark Grossi for his excellent acting work, and an apology for the lack of direction he was given while we concentrated our attention on the illusive 'Intruders'.