Digitisation’s Last Hurdle, or a Bridge Too Far?
NFSA Development Manager Executive Dominic Case made the following presentation about archiving film at the 2011 SMPTE Conference in Sydney.
Four years ago I wrote: “most cinema-goers would be astonished to learn that the majority of movies they see are shot and projected, not digitally, but on 35mm film – yes, the same technology that’s been in use for over a century. The truth is that digital projectors capable of filling a screen as big as a bus are seriously expensive, and there’s been little reason for cinema owners to change”.
That was four years ago. How much has changed! In Australia, nearly all cinemas will be digital within a couple of years, and few of them will still be able to project film. We aren’t alone. I had an email from a colleague in Belgium last year, who runs an archive and a cinémathèque. He summed up his predicament, and I quote:
All screens in Belgium will be digital by the end of the month. No more film prints. They have dismantled the film projectors from the booths. They plan to keep perhaps one per multiplex, just in case for one year or so. And suddenly we don’t have any useful collection anymore, and no screens to show our films unless they are digital.
Film archives around the world have developed expertise in preserving film: many plan for their collections to last for 400 years, stored in carefully-controlled conditions. The old problems of inflammable nitrate film, of shrinking and decomposing acetate film, of fading colour dyes, are all understood and controlled. But how will we screen these prints in the future? And when will that future arrive? Why don’t we simply digitise everything? There are so many answers to this question. But first, I’d like to outline the task of a film archive – in particular, the National Film and Sound Archive of Australia.
In Australia, although the first film was shot as early as 1896, it was not until 1935 that any form of an Archive was established, as a section of the National Library, with a budget in its first year of sixty pounds. Today, our collection encompasses film, video and digital moving images, broadcast media and recorded sound, associated documents and artefacts, and new media. We have 1.6 million items in the collection, including 127 million feet of film. It includes a lot of nitrate film from before 1950, but we also collect and preserve new productions. We acquired three million feet of film – new and old – last year. Importantly, we don’t control the rights of most of the material we hold, although we have just taken responsibility for the substantial Film Australia Collection, in which the rights are held.
Preservation vs access
Our program is to develop, preserve and present Australia’s National Collection, and make it available to all Australians. Not just now, but forever, so we must ensure that the collection is accessible in the future. Archivists are not the dusty historians that you might think – we must be futurists too.
It is important to distinguish between preservation and access. We aim always to have several copies of a film, which we will categorise – simply – as for:
• preservation (not to be used for anything),
• duping (used to make new prints or transfers as required)
• access copies (distribution prints)
This gives us the benefit of redundancy in case of disasters, and also allows for the wear and tear on access copies without affecting the original or preservation copies.
We run touring cinema programs, reaching small country towns from Mildura to Mallacoota, screening films ranging from Storm Boy to South Solitary. Up to now, we have screened 35mm prints from our collection, (including the 75 features from recent years that we have printed and preserved with the sponsorship help of Deluxe and Kodak). But this year, many cinemas are telling us of their plans to convert to digital projection.
There really isn’t any alternative but to start acquiring or making Digital Cinema Prints for our screening collection. It’s a not insignificant task, and of course there are issues specific to this application. For current material we need to resolve protocols about encryption. For older material, it’s a big cost to digitise for just one or two copies.
There are a few common preservation principles to all archives. In summary they include:
• Never destroy the original copy, even after you have duplicated it. The dupe may last longer, but the original is the best source for any future re-copying as long as it lasts.
• Preserve the reproduction equipment along with the content.
• Always preserve and present in the original medium and format. Of course, not everyone who wants newsreel footage of the Snowy River Project needs the ambience of a 1950s newsreel theatrette, so modern access versions, from DVDs to clips on websites like our own Australianscreen Online, must be part of the mix.
Traditionally, we make preservation copies – now on polyester film stock – and store them in the right conditions for longevity, where they are left alone, apart from a cycle of inspection. But we can’t keep up with all the material we have to preserve. It requires fierce curatorial selection, picking what’s most important to copy. At least though, once a film is copied, we can store it, and move on.
On the face of it, photochemical preservation hasn’t been wonderful, although the issues haven’t just been technical; they’ve been to do with collection management, the learning curve, and plain old-fashioned mistakes. The great majority of our silent cinema is lost forever. Not because the film has rotted, but because it was thrown away. Or worse, burnt. In 1927, for a burning ship scene in the epic For the Term of his Natural Life, an old hulk was laden with reels of nitrate film and set afire. Archivists will forever wonder what earlier classics were lost.
The flammable nitrate film base was phased out, and completely replaced with acetate base film by 1951. Now we have polyester based film stocks, which have a much better prospect for long term survival.
All these materials are subject to gradual decay. Fortunately, black and white silver images are relatively stable, but the acetate film base hydrolyses to acetic acid – known as Vinegar Syndrome. Nitrate turns to gun-cotton. Colour dyes fade.
Fortunately, we understand these processes well. A nineteenth century physical chemist called Svante Aarhenius is famous for two things. One was his work on the rate of reactions. Given that dye fade and shrinkage depend both on temperature and time, he devised an experimental way of measuring the effect over short periods of time at a very high temperature and extrapolating that to find the effects over long periods of time at low temperatures. Using the right scales, we can thus very reliably predict the life expectancy of film stored at any given temperature.
Relative humidity is a factor as well, and on that basis, we can see that – for the type of stock in this example – at 30 percent RH, and 13 degrees C, the image has 200 years’ life.
The question of “life” is interesting. Analogue materials fade gradually, so it’s hard to be clear about a “use-by” date. It is possible to restore even severely faded colour images via a digital intermediate stage. Digital loss is much more abrupt, unpredictable, and devastating when it happens. Of course you cover that by having redundant copies of the data – just as you do with film, to guard against all forms of accidental damage.
So, we have a lot of knowledge of how to make film last a long time. At NFSA’s vaults in Mitchell, we preserve Nitrate film at 4C and 35%RH, Colour acetate at 4 to 6C and B&W Acetate 16C. That’s 100 years minimum. If we kept it colder, it would certainly last longer, but 129 million feet of film takes up a lot of expensive space, and it take a lot of energy to keep the chillers running.
The second thing Dr Aarhenius was known for was that he was the first person to predict that emissions of carbon dioxide from the burning of fossil fuels would cause global warming. That was in 1896. In terms of climate change, it does give us cause to wonder whether we are aiming to preserve our film beyond the end of civilisation as we know it, and if that is a good thing!
That aside, the scene looks good for film preservation. The problem is that we have a lot of film that, as yet, has no back-up preservation copy. And a lot of our colour collection from before 1980 is on the fading colour stocks of the time. Much of it was already faded before it reached us. It’s all kept in the best possible conditions, but there are many years – even decades – of work just preserving all that, even if we didn’t have new acquisitions coming in all the time.
And we don’t have “many years”. Film laboratories are closing or reducing their operations. NFSA operates a black and white laboratory focused on old and fragile material, and not a colour laboratory: only a handful in the world do. In a very few years’ time we must expect to see lab services become very hard to find. And if no-one is processing film, how long can we expect the manufacturers to continue to make it – just for archives?
So we must consider digital preservation. Archiving ethics say we must preserve at the highest possible quality. No compression. The quality of the cinema film image comes at a cost: the cost of massive amounts of data. The general public might believe we just need a DVD – 4.7Gigabytes – to preserve a film, but we all know it ain’t so. To store all the data in an original 35mm camera negative, a digital version takes up over 50 Megabytes for each frame. One single copy of a complete feature film might need between 10 and 20 Terabytes.
If we were to digitise all the preservation copies in our collection – just the pres. copies, leaving out the access copies which would be digitised at a much lower bit-rate, we would need somewhere between 80 and 120 Petabytes.
It’s true that compression technologies can reduce that data dramatically, but for archivists, the overriding rule for preserving any sort of image is that none of the original image detail should be lost.
Next, no-one knows how long digital data can be preserved. If you follow a store and ignore approach as we do for film, then typical estimates for most digital media to be totally reliable are around ten to twenty years. And while analogue materials – by their very nature – decay gradually and progressively, and retrievably, digital data is either recoverable or it isn’t.
Then there is the format issue. There are always new versions: LTO 1 to 6: MPEG 1 to 4 and so on.
Now we know that there are well-established methods of getting over these sorts of problem, such as the process of regularly refreshing data – copying it from one drive or tape to another. But the prospect of ongoing active preservation – continually refreshing – a digital film collection is a quite new way of working for a film archive that struggles to copy everything even once.
In 2007 the Academy of Motion Picture Arts and Sciences published a seminal report The Digital Dilemma. The standout feature of this report was the comparison it presented between the costs of storing a film on film, versus storing a film digitally. It’s a complex comparison, with many assumptions. For example, I found no discussion of the cost of temperature control for a film vault. As for digital storage, figures were based on quotes from commercial data farms. In both instances, the final cost represented the annual cost of fully managed preservation – of tri-sep film masters, and of 4k digital data stored on-line on hard disks, plus the initial cost of making the preservation elements – printing or scanning – amortised over 100 years.
The Academy found that the cost of preserving an average length feature film was just over $1,000 per year. The cost of preserving the same film in digital format was just over $12,000. Since 2007 the cost of digital storage might have halved twice, thanks to Moore’s Law, but one thing to bear in mind is that the cost of electricity needed to run and cool large data banks will far outweigh the cost of the memory itself.
Do we trust technology?
We can summarise other preservation issues under a few general headings:
• The issues of recovery. We need to preserve the hardware and software necessary to recover data, many years from now. For obvious reasons, we need to use open formats, not relying on any commercial ownership which may not last forever.
• The issues of translation from one format to another. In the past we’ve seen aspect ratio butchery – many films now are only seen cropped from whatever their intended shape was. Running speed is another change: 24 to 25 in the PAL world; sometimes 16 to 24 for silent film. Colour gamut, contrast, and the soundtrack are all format dependent. These are all real and unacceptable changes to the original intentions of the film.
• Then there is the matter of rights. This actually goes to the heart of the problem. We are charged with caring for the nation’s audiovisual memory for as long as possible. That’s a cultural matter not a commercial one. So who pays the up-front cost to preserve material for use in a time when it is long out of copyright?
• Finally, the most important, provenance. Many important films have survived only by chance, rather than design. Scraps of film, re-cut versions and odd reels turn up in sheds, in rubbish skips, and even, mis-labelled, in film archives. Often they can only be pieced together by physical inspection: the type of splices, the handwriting on labels, or the type of stock. Neglected and abused digital data will have far less chance of recovery.
NFSA Senior Curator Meg Labrum and Film and Video Specialist Susanne Haydon said it best at a lecture in 2010:
“It is a popular assumption that audiovisual archives of the future will be entirely virtual – a vast amount of digital files, containing the world’s recorded moving image and sound experiences. However, this quite misses one of the fundamental principles of such archives, which is that they are preserving not just the content of their collection, but also, to every possible extent, the complete viewing experience. And so that means that the content, the carrier, and the means of viewing and reproduction are inseparable from each other.”
Of all the audiovisual media, photographic film fits this argument much more completely than any other. Alone among video, audio tape, vinyl records, CDs and DVDs and even files, film is the one format that requires no – or virtually no – replay equipment. The image that is stored is exactly the image that you see: it’s one of light and shade and colour. It doesn’t need translating from an electrical signal or a wiggly groove or a stream of 0s and 1s. And so there is some defining nature about the artifact itself that makes the idea of preserving the original material so much more meaningful.
This really asks: do we trust the technology? All you need to see a film image is a light source. Once the equipment to retrieve a digital image is lost, the image itself is lost too.
It’s a changing game. We need to scan film for digital cinema access copies, as well as for digital colour fade correction, now. But on the best information we have today, preserving on film is the most consistent with archival philosophies, and appears to be the best prospect for long-term preservation. In a very few years, digital will certainly become cheaper, probably more secure in the long term, and we may resolve the philosophical issues. Film, on the other hand, will become harder to get and to use.
So the best plan seems to be to keep the chillers going, and continue making new preservation copies on film, until those things happen. But there is many years’ work, whichever technology is used. Even if we could switch to a low-cost, low maintenance, secure digital solution tomorrow, we’d still need our film collection to last for 100 years, because it would take that long to transfer everything onto it.
I’d like to leave you with a seditious thought:
“Keep your analogue copies – they may be all that’s left after the digital era is over.”