Anna Post Nightmare

Attendees of my screening in 2009 were treated to an apparently once-in-a-lifetime event: the only screening of my 20 minute short, “Anna.”  This is due to the film having originated on 16mm, having been transferred to DV, and screened in standard definition.  Who would possibly screen standard def in 2009 much less today?  It’s a rough time looking at 720 x 480 these days.

Fortunately, the negative still exists.  But the Great Hard Drive Crash of 2009 destroyed everything used to make “Anna.”  All that was left was the SD MPEG-2 version screened on DVD.  Quite a few projects were lost that day, including “Big Dong” and “Cat’s Head Theatre.”

Recently I thought I might try to save poor Anna, as no one has really seen her since that night in 2009.  We spent so much time on the film, and I spent so much time editing, coloring, and doing effects on it – not to mention the lovely 5.1 mix I had.  It seemed sad to leave it fallow, especially when it never had a chance to get off the ground.

The problem was trying to get the negative transferred.  I thought maybe there was a way to do it without having to pay for all of it.  We shot 5000 feet, about a quarter of which is used to make the movie.  That’s actually a really good ratio – laughably low in some circles.  4:1 is pretty cheap.  At the current scanning rates, why would I want to transfer all those takes we never used?  I called around for prices, and a few places quoted me at 40¢ a foot.  That would cost $2500, more than half of which I simply wouldn’t use.

I developed an idea  by which I could get the transfer house to copy as little as possible without also starting and stopping the machines too many times.  Keep in mind that a film is shot out of order, so a given roll of camera negative may have shots from scenes 11, 13, and 7 next to scene 3.  I would have to generate a list of start and stop frames that would get all of the film without skipping any important bits. And I’d have to be generous and copy lots of extra frames in case I made some kind of addition or subtraction mistake.

Start with the End

First I ripped my own DVD copy to provide a starting point.  The film would have to be recut from existing dailies rolls.  The original video transfer to DV had not been lost.  Within hours of the 2009 hard drive crash I had gone right back to all my DV tapes and re-ingested them as soon as possible.  That old DV tape is getting kind of powdery and crappy now, so I’m glad I did.  Thus, the dailies transfers with their original time code numbers were intact.

But I had to re-edit the entire film by placing the DVD copy on a timeline in editing software (I switched to Premiere some time ago) and matching the frames of every edit, essentially recutting the film on top of itself. This was so that I could understand which pieces of film I had used and which ones I had discarded.

From that recut, I ended up with a list of time codes that corresponded to the DV film transfer.

Ugly Math

I had a list of time code numbers, but that was not the best.  I could give those time code numbers to the lab and hope that if they set the TC number at the film punch point it would all work out.  Or I could give them a list of key code numbers.  For those who have not worked with film, key code was a small number the film manufacturer put on every foot of film. It was on the edge of the strip, right outside of the sprocket.  These unique identifying numbers helped keep straight what piece of negative you were using.  In ye olden days when you transferred a piece of film, you asked for a “flex file” (.FLX) which was a concordance between the time code numbers used in the transfer to video and the key code numbers on the film.

This correspondence was famously confusing.  The numbers were not so regular.  Time code is 30 or 24 frame based; that’s one rate for film (24) and one for video (30).  Because color video was developed with various compromises, Standard Definition TV was actually running at 29.97.  In order to transfer 24 frames into 29.97 frames, the 24 had to be slowed down a bit to 23.98 fps.  No one keeps track of .98 a frame, it’s just a fancy way of saying 24 frames per 1.1 second.

On the other hand, film key codes are based on film having so many frames to the foot and running at so many feet per minute.  In the case of 16mm film, which is what I shot, this would be 40 frames to the foot and 36 feet per minute, running at 24 fps.  So you see, there has to be a way of figuring out how to connect the 40 frames per foot idea with the 29.97 fps idea.  The math is kind of ugly, so there were software tools designed to help with this process.

I had the foresight to get flex files for “Anna,” and had stored them on different media – in those days they gave you a floppy disk.  But it’s also the case that film would be stored in the box with key codes written on a piece of paper.  Final Cut Pro 7 used to come with software called “Cinema Tools,” a program specifically designed to deal with the film and video transfer issues.  Cinema Tools kept the key codes in a database, starting with the 1:00:00:00 time code number and the first key code number, and served to connect the time code of the transfer and the key codes together.  One should only need to export an EDL from the editing software, import it into Cinema Tools, and get a list of key codes back out.

Ideally.

Reassembly

Sadly the video transfer was just a bit weird compared to the film elements.  The film was boxed with rolls 1,3, and 2 together, rolls 4-10 in a second box, and rolls Z1-Z5 in a third.  The “Z” rolls were pickups.  But the DV transfer, which I do not remember doing separately, had rolls 1,3,2,4 and 5 on one transfer, 6-8 on a second, 9 and 10 on a third, and the Z rolls on a fourth.  Each new transfer session started at hour 1:00:00:00.  Thus roll 4 from box 1 was at hour one time code, as was Roll Z1.  I was already worried about keeping this all straight.

I made a list of each film negative roll and its starting time code from the original DV transfer.  Then I exported video files to match them, each roll as a separate clip, each beginning with the proper time code.  So I now had a digital version of exactly what I had in the film negative boxes.  These I connected to the database in Cinema Tools thereby linking negative film key codes to the time codes.  I should have been done with this, right?

“Uncutting” the Film

Next, I took the newly edited version and reassembled it.  First I “uncut” each edit by rolling back to the beginning of the take where the slate was. Then I assembled them so that all the shots were in the same order they were on the negative.  So if the film showed you scenes 4,5, and  6 in order, but I shot them in the order 6,13,10,5,1,and 4, then I had to gather all shots from scenes 13,10, and 1 to make sure I put them all together as they were shot.  This was because I could not tell the telecine operator to go jumping around the negative looking for shots – it’s not good for the film, and it’s a pain to the operator.  It makes things cost more.

Once these were reassembled into select reels, I had the time code numbers of all the scenes that were used in the film, omitting all the scenes not used, in order that they appear on the camera roll.

I expected Cinema Tools to open this list and spit out the key codes, but no amount of coaxing would provide me with that list.

I then realized that if I used the clip viewer in cinema tools, I could just roll forward to the beginning of each clip and read the key code number right out of Cinema tools, and make the list by hand.  First I had to reverse telecine the DV transfer.  I mentioned above that film was at 24fps, and video at 30.  And that the 30 was really 29.97 and the film was slowed to 23.98.

But more than that, how do you get 24 into 30?  The answer was that standard definition video used two “fields” of alternating interlaced lines instead of a single frame.  Thus SD video would show you lines 2,4,6, and 8 of a frame first, then lines 1,3,5,7, etc.  This was also an early TV engineering compromise. The result was that we actually had 60 fields of video for every 24 frames, a lower field of lines 2,4,6, etc. and an upper field of lines 1,3,5, etc.

The math was still not good.  Because the common denominator between 30 and 24 is 6, the problem was really about getting 4 frames onto 5, or, more properly, 4 frames onto 10 fields.  The process was called telecine, and involved splitting frames so that some video frames would be comprised of an interlace of two separate film frames.

That process had to be undone in order to work with computer files that had the same number of frames as my original negative.

I’m Brilliant, but It Comes to Nought

I went through each camera roll, going to each edit points and writing down the key code numbers.  Voila!  I seemed to have a set of negative pulls for the telecine operator.  He or she could set the film up, reset the counter at the lunch point, roll through to my first number, transfer from one key to the next, and keep doing so, skipping the sections in-between.

Now the problem was that I was only going to this post house to have the work done there because my friend worked there.  They laid her off badly, so there’s no way I’m taking my business to them.  My new telecine operator at a Point 360 digital suggested it would take twice as long to transfer pulls as to just transfer everything, and I’m inclined to trust him on that.  So all that work for nothing.

Transfer

The good news is that this new transfer is much better.  I did 10-bit 2K DPX files for everything.  The grain is super annoying, so I may soften it in post.  But I have transferred it totally flat, with no color correction at all, so that’s probably why it looks crap for now.  I did apply a pretty basic LUT to the image so that I could make proxies.  To maintain some consistency, I made 24fps DV sized proxies, so that I could RECUT the whole damn thing against the MPEG-2 version  yet another time because gawd, don’t I love recutting my same damn film over and over again?  It’s the best.

DaVinci Resolve actually turned out to be a bit tough on making these proxies.  The 2K scan was at full aperture, meaning 2048 by 1556.  None of that goes into DaVinci’s containers so easily, and my one attempt to give it a custom size resulted in a tiny image in the center of both black columns and letterboxes on top and bottom.  So no go.

After Effects, my old friend and universal can opener, linked to the DPX files like a charm, plus applied the necessary LUTs plus an additional level control for brightness, and crammed them all in a 24fps DV file like a charm.

Imported into Premiere, I put the old 29.97 DV MPEG-2 file on a 24fps timeline, which seems to resolve the 3:2 telecine problem outlined above.  The old DVD is now playing as if it were 23.98fps material, and the proxies are cutting against it just fine.

When I complete this, I’ll export an EDL and roundtrip it back to daVinci.  But there still remains the tons of After Effects work that I did back in the day. None of those project files survive, including the raw materials I used to make certain effects.  I even shot two or three rolls of Super 8 for the film, and those remain untransferred.  The copy I made was by shooting a projection off the wall and then increasing the contrast.  I feel like I should transfer these rolls and do it properly before it becomes impossible to do so.

ETA on a DCP of Anna?  Maybe in the next century?