Managing a Preservation and Digitisation Plan of Audio-visual Archives Processes and technology Jean VARRA_ INA 2008
1
INA at a glance... •• INA INA(Institut (InstitutNational Nationalde del’Audiovisuel l’Audiovisuel- -France) France)isisin in charge chargeof ofthe thepreservation preservationand andexploitation exploitationof ofone oneof ofthe the richest and oldest radio and television archive holdings. richest and oldest radio and television archive holdings. •• These Theseholdings holdingsare aremainly mainlycomposed composedof ofprograms programsfrom from public national and regional TV and radio broadcasters public national and regional TV and radio broadcasters since sincethe thebeginning beginninguntil untilnow. now. •• They Theyinclude includecinematic cinematicnews newsitems itemsand andan animportant important amount of photographs. amount of photographs. •• Otherwise, Otherwise,INA INA has hasbeen beenin incharge chargeof ofthe thelegal legaldeposit depositof of the French radio and TV programs since 1995. the French radio and TV programs since 1995. •• The Theother othermissions missionsof ofINA INAare areProduction, Production,Research Researchand and Training. Training. •• In In1999, 1999,INA INAdecided decidedto tolaunch launchaamassive massiveand andsystematic systematic digitisation plan of its audio-visual archives. digitisation plan of its audio-visual archives.
Jean VARRA_ INA 2008
2
INA holdings TV holdings: 535 000 hours Film Film 35 35 and and 16mm 16mm 143000 143000 hours hours
Video Video 1p 1p and and 2p 2p 70 70 000 000 hours hours
Video Video 3/4 3/4 inch inch
BETACAM TAPES 192000 hours
135000 135000 hours hours
Radio holdings: 497 500 hours 78RPM 78RPM disks disks
1/4 1/4 »» audio audio tapes tapes
27000hours 27000hours
476000 476000 hours hours
DAT
Photo holdings: 1 200 000 photos
Jean VARRA_ INA 2008
3
Why a digitisation plan? All Allarchivists archivistsmust mustface face two twoimportant importantand andunavoidable unavoidableproblems: problems:
Old Oldmedia media inevitably inevitably degrade degrade
AND
Jean VARRA_ INA 2008
Obsolescence Obsolescence of ofplayback playback means means
4
3 main questions to answer... questions questions
Answers Answers
Degradation Degradationofofthe thecarriers carriers and obsolescence of techniques and obsolescence of techniques
Definition Definition and andsetting setting ofofpreservation preservation and and safeguarding safeguarding policies policies
Archive Archivevolumes volumes increase exponentially increase exponentially
Decision Decisionofof exhaustiveness exhaustiveness ofofpreservation preservation oror definition definitionofofselection selectioncriteria criteria
Archives Archiveshave havecommercial commercial and cultural and culturalvalues values
Definition Definitionofof an anaccess accesspolicy policy
Jean VARRA_ INA 2008
5
Typology of degradations 33types types of ofdecay decay
Biological Biological
(mould, (mould,bacteria….) bacteria….)
Chemical Chemical (spontaneous (spontaneous chemical chemical changes: changes: Acid…) Acid…)
Jean VARRA_ INA 2008
Mechanical Mechanical
( (changes changesininsize size and shape) and shape)
6
Some major degradations... ••
••
••
••
Film Film –– Flammability Flammabilityofofdecomposing decomposingnitrate nitratefilms films –– Vinegar syndrome of acetate films causing Vinegar syndrome of acetate films causingshrinkage shrinkageand anddecomposition. decomposition. –– Splice degradation( dry or greasy splices) and damaged perforations Splice degradation( dry or greasy splices) and damaged perforationsrequiring requiringheavy heavymechanical mechanicalrepair repair –– image degradation like scratches, dirt, colour fading…. image degradation like scratches, dirt, colour fading…. Tapes (video Tapes (videoand andaudio) audio) –– Stickiness Stickiness –– head headclogging cloggingand anddropout dropout –– mechanical deformation mechanical deformation(shrinkage,wavering…. (shrinkage,wavering…. Mechanical Mechanical disks disks( (78 78RPM, RPM,LP) LP) –– (grooves disappear producing (grooves disappear producingaafin finwhite whitepowder) powder) –– sensitivity to fungus, ultra violets and heat sensitivity to fungus, ultra violets and heat –– crackling cracklingand andshrinkage shrinkage Optical disks Optical disks –– layer layerseparation,lack separation,lackofofplanarity,crackling planarity,cracklingand andpinholes. pinholes. –– sensitive to pollutants , corrosion of the metallic layer sensitive to pollutants , corrosion of the metallic layerand andlong longlight lightexposure exposure
Jean VARRA_ INA 2008
7
The importance of good storage conditions •• “Old “Oldmedia mediaare areoften often archived archived ininstores stores with withbad badclimate climateconditions”. conditions”. •• the theeffects effectsof of climate climateconditions conditions((temperature temperature, ,humidity,dust, humidity,dust,air air quality…)on quality…)ondegradation degradationand andtherefore thereforeon onthe thelife lifeexpectancy expectancyof of carriers carriersare arenow nowwell wellknown. known. •• Preserving Preservingoriginal originalmedia mediainingood goodclimate climateconditions conditions(controlled (controlledT°, T°, Relative humidity and dust) and in suitable containers is always Relative humidity and dust) and in suitable containers is always far far less lessexpensive expensivethan than being beingobliged obligedto torestore restoredegraded degradedmedia. media.
But Butrelying relyingonly onlyon ongood goodclimate climateconditions conditions to topreserve preservecontent contentin inthe thelong longterm,is term,isnot notenough…. enough….
Vented cans Stil Design
•Archives films
Jean VARRA_ INA 2008
8
Recommended climate conditions Film Film
Magnetic Magnetictapes tapes
•• Main Mainvault: vault:Cold Cold
Acetate Acetatetapes: tapes:
–– T= T=5°C 5°C –– RH:25% RH:25% –– Light Lightover-pressure. over-pressure. –– air filtering. air filtering.
–– –– –– ––
•• transit transitstorage: storage:Cool Cool
•• •• ••
–– T= T=15°C 15°C –– RH:30% RH:30% –– dew dewpoint pointcontrolled controlledtotoavoid avoid condensation condensation Temperature Temperatureand andRH RHstability stability – – – –
Cool Cool12 12°C °Cmax max for for50% 50% max maxRH RH 17°C max for 30% max RH 17°C max for 30% max RH 23°C 23°Cmax maxfor for20% 20%RH RH frozen forbidden unless frozen forbidden unlesshigh highvinegar vinegar syndrome syndromelevel level (IPI (IPI22or orhigher). higher).
Polyester Polyester tapes: tapes:
T°= +/- 1°C over 1 hour T°= +/- 1°C over 1 hour Relative humidity : +/- 5% RH over 1 day Relative humidity : +/- 5% RH over 1 day
Frozen Frozenstorage storage vault vault0°C( 0°C(for for degrading degrading acetate films with vinegar syndrome acetate films with vinegar syndromeand andoror degrading degradingnitrate nitrate Isolate degraded Isolate degradednitrate nitrateand andacetate acetatefilm filmtoto avoid avoidcontamination. contamination.
–– Cool Cool12 12°C °Cmax max for for50% 50% max maxRH RH –– 17°C max for 30% max RH 17°C max for 30% max RH –– 23°C 23°Cmax maxfor for20% 20%RH RH –– Cold to avoid Cold to avoid –– frozen frozenforbidden forbidden
Disks Disks –– Store StoreVertically Vertically –– Frozen Frozenisisforbidden forbidden
Jean VARRA_ INA 2008
9
Why digitise our archives? Today Todayall all archive archiveowners owners have have the thesame same dilemma dilemmato tosolve: solve: •How could this analogue heritage be preserved? •How could this heritage be everlasting? •How could this heritage be more valuable?
Only Only one one way: way: Digitisation Digitisation. .
Digitisation Jean VARRA_ INA 2008
10
Setting a preservation and digitisation plan A methodological approach
Jean VARRA_ INA 2008
11
Some important preliminary considerations Why Whyaamethodological methodologicalapproach? approach?
•Because the holdings are often huge. •Because the holdings are often poorly identified •Because budgets for preservation are poor compared to the mass of material to be treated. •Because priorities must then be defined. •Because the materials to be treated are often damaged and their life expectancy is limited.
Jean VARRA_ INA 2008
12
A 3 entries problem‌
Volumes
Time
Means Technical, Human, Money
Jean VARRA_ INA 2008
13
A seven steps preliminary process...
11- -Define Definethe themeasurement measurementunit unitofofyour yourholdings holdings(nbr (nbrhours) hours) 22- -Map Mapyour yourholdings: holdings:(indexed) (indexed)contents contents&&physical physicalstorage storagemedia media 33- -Assess Assess the the condition conditionofofthe thephysical physicalstorage storagemedia media 44––Define Defineaapreservation preservationpolicy policy&&processing processingpriorities priorities(+(+preservation preservationcommittee) committee) 55––Define Definethe thetechnical technicalsolutions solutions(formats, (formats,storage storagemedia, media,workflows) workflows) 66- -Estimate Estimate your yourhuman, human,technical, technical,financial financialresources resources 77––Draw Drawup upannual annualbudgets budgetsand andprocessed processedvolumes volumes(over (overthe thepluriannual pluriannualplan) plan) Jean VARRA_ INA 2008
14
Some important preliminary considerations Why Whyisisititimportant importantto todefine defineand andset setaa massive, massive,industrial industrial and andpluri-annual pluri-annual digitisation digitisationplan? plan? • • •
•
Bring an answer to the urgency to face to degradation and obsolescence of analogue technology. “Massive and industrial” in order to reduce costs and go fast. Get a reference tool for negotiating funds and all other negotiations : human resources, technical resources and budgets. Provide a tool to select and define priorities in terms of materials to be processed. Jean VARRA_ INA 2008
15
Step 1 Measure Measureyour yourholdings holdings(volumes) (volumes)
•Number of materials •Number of catalogued documents •Number of programmes •Number of hours of programmes •Evaluate “redundancy”
Jean VARRA_ INA 2008
16
Step 2 Map Mapyour yourholdings holdings(volumes) (volumes)
•Map your contents: •using cataloguing, indexing, searching tools •Map your media: •make inventory of physical media •sort volumes according to the genre and type of media
Jean VARRA_ INA 2008
17
Examples of mapping
1500
2800
2500
2000
9800
200
Collections of news magazines
500
1000
1000
500
1622
4622
2 499
Total newsreels 1500
2500
3800
3000
3622
14422
300 300 200 200 300
Cinema,TV, theater programs 400 Tourism programs Total doc and mag 1700 ENTERTAINMENTS Pop, rock programs 200
2005
2006
2007
2008
Amount to 2009-2015 be treated
1000
DOCUMENTARIES AND MAGAZINES Documentaries Litterature programs Art programs Cultural programs Music programs
2004
TOTAL 2008
CONTENTS NEWSREELS News
2699
NOT REAL FIGURES
10000 7121 17121
350 400 400 100 300
300 400 400 100 300
200 200 200 100 300
100 100 100 100 200
1250 1400 1300 600 1400
1316 1007 730 2166 713
2566 2407 2030 2766 2113
300
300
200
200
1400
1850
1800
1200
800
7350
2660 1560 10152
4060 1560 17502
200
175
100
100
775
6925
7700
Total entertainments
200
200
175
100
100
775
6925
7700
SPORT Sports magazines Total sports
300 300
300 300
300 300
200 200
100 100
1200 1200
2556 2556
3756 3756
23747
4296 3000 3207 7162 39997
4296 3000 3207 7162 63744
YOUTH PROGRAMS ANIMALS PROGRAMS RELIGIONS GAMES TOTAL FILM
3700
4850
6075
4500
4622
FILM ACTUALITE - NEWSREELS News Regional news TOTAL NEWSREELS FICTIONS Fictions TOTAL FICTIONS DOCUMENTAIRES DOCUMENTARIES literature programs art programs cultural programs music programs science programs cinema, theater programs tourism programs society programs TOTAL DOCUMENTARIES ENTERTAINMENTS rock, pop programs TOTAL ENTERTAINMENTS SPORT sport programs TOTAL SPORTS YOUTH PROGRAMS ANIMALS PROGRAMS RELIGIONS GAMES
Jean VARRA_ INA 2008
1 and 2 inches Umatic/BVU
10 828 35 500 46 328
TOTAL
55 800 55 800
102 128
5 000 5 000
650 650
770 770
6 420 6 420
2 407 2 030 2 766 2 113 2 790 4 060 1 560 8 710 26 436
959 850 951 2 085 1 024 1 534 1 163 2 810 11 376
749 650 1 771 1 727 718 2 072 823 2 500 11 010
4 115 3 530 5 488 5 925 4 532 7 666 3 546 14 020 48 822
7 700 7 700
2 900 2 900
3 968 3 968
14 568 14 568
3 756 3 756 4 296 3 200 3 207 7 162
1 900 1 900 3 540 1 695 1 490 4 813
2 429 2 429 1 897 802 720 2 477
8 085 8 085 9 733 5 697 5 417 14 452
NOT REAL FIGURES
18
Step 3 Know Know the the condition conditionof ofphysical physicalmedia media
•Evaluate the degradation level of each type of media and the corresponding volumes. •Vinegar syndrome •Splice condition •Image degradation. •Sticky shed syndrome. •Mechanical deformations •…….
Define and estimate the level of difficulty to process them. Jean VARRA_ INA 2008
19
Example: items for film condition mapping News NITRATE BASE FILM
Black and white
KINESCOPE
YEAR
NEGATIVE 16 mm Acetate base film
small Rollls
COMMAG ACETATE
Edited on one reel
production
Colour
Manufacturer and type
type of copy
SEPMAG POLYESTER
VINEGAR SYNDROME
SPLICED WITH ADHESIVE TAPE
SPLICED WITH GLUE
LEVEL OF DEGRADATION IPI levels
STATE OF THE ADHESIVE TAPE
STATE OF THE GLUE
PHYSICAL STATE
PERFOR ATIONS
SCRATCHES
DAMAGED ?
Yes
no
base
severity?
specific telecine and or sepmag player film processing?
RETRAIN
gelatine
GRADING NOTCHES
SHRINKAGE
Value
0
Yes
1
2
no
3
GOOD
DAMAGED
DRY
Reconditionning? separation of film and sepmag ? storage conditions/isolation decisions to transfer?
GOOD
GREASY
simple check up or repair?
BREAKS
PICTURE
FADED COLOR
GOOD
colour graded ?
Yes
no
Color correction ?
Making statistics /decisions/processes...
Jean VARRA_ INA 2008
20
Step 4 Define Definepriorities priorities combining Technical reasons
Content reasons
heritage value, commercial value, scientific value, shortage...
Uniqueness of the carrier, decay level, exploitation difficulties...
With Withthe thehelp helpof ofaapreservation preservationcommission commission Jean VARRA_ INA 2008
21
Step 5 Choose Choosedigital digital target targetmedia mediaand andfiles files
•Prefer open media and standardised files. •Specify precisely the encoding schemes and the various quality levels to get. (see next chapter)
Jean VARRA_ INA 2008
22
Step 6 Measure Measureone’s one’sforces: forces: human humanand andtechnical technical
•
• •
Define mean processing times for each type of media according to its physical condition. Take an inventory of the available work forces and skills, outsourcing (service providers) and indoor and evaluate them. Specify the processing costs for each media type. Define the best cost effective technical solutions.
•
Define the best cost effective technical solutions
•
Jean VARRA_ INA 2008
23
Step 5 Specifications Specificationsof ofthe thetechnical technicaljob job
•The objectives of technical specifications is to define precisely the technical expected results. •Describe precisely all the steps of each process.
Jean VARRA_ INA 2008
24
Step 7 Set Setbudgets budgetsand andvolumes volumes (over (overthe theduration durationof ofthe theplan) plan) •Set precisely the dispatch of: • annual investment and operating budgets •volumes according to: internal /external processing capabilities.
Jean VARRA_ INA 2008
25
Volumes Pattern (INA) Digital preservation Plan ( volumes - hours)
Total
remaining
Achievement
hours
repartition over 2005 - 2015
to process achieved achieved Years
99 / 02
vinegar syndrome
64500
FILM telecine
143 500
mechanical repair
132 400
VIDEO 3/4 inch
122 200
VIDEO 1 et 2inch
70 000
Total TV
335 700
DISQ 78 RPM
20 700
preservation restoration
1/4 ich tapes
465 000
DAT
11 800
Total RADIO
497 500
TOTAL
833 200
%
03
Achieved 04
05 / 15 05
06
07
08
09
10
11
12
13
14
15
EXAMPLE PATTERN Jean VARRA_ INA 2008
26
Budget pattern (INA) Tableau 2
Preservation Plan Budgets ( M€
Total hours to process achieved Années vinegar preservation vinegar mesasurement FILM TC mechanical repair VIDEO 3/4i VIDEO 1 et 2i Total TV
78 RPM
99
00
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
Total
Total PSN
05/15
99/15
143 500 122 200 70 000 335 700
20 700
preservation
EXAMPLE
restoration
14 Inch tapes DAT Total RADIO Photo Storage storage TOTAL INVEST Inventory containers magnetic media tapes CD/DVD others management labour forces TOTAL FONCT
465 000 11 800 497 500
PATTERN
TOTAL
Jean VARRA_ INA 2008
27
Overall technical process Digitisation and new media creation new digital master low data rate versions create new archive item
Composition Identify and assemble items
Update archive quality control
Reception of sets
Reception of sets
packing original media
performing expected job
performing quality control
job orders
work progress
accept or reject work
Preparation of the sets
planning Organisation of transports
Organisation of transports
organise corrective actions
Jean VARRA_ INA 2008
replace old item with new update metadata Reception of sets Statistics about job achievement : budget and volumes
Organisation of transports
28
Digital solutions for preservation
Jean VARRA_ INA 2008
29
Digitisation is the key… But what digital? Keys Keysfor foreverlastingness everlastingnessor or““permanence” permanence”of ofdigital digital audio audioor orvideo videodata dataare: are:
The Theindependence independenceof ofdata data regard regardto to the the physical physicalmedia media
The Theuse useof ofstandardised standardised and and“non “nonproprietary” proprietary”:: ••Essence Essencedata data ••Metadata Metadataformats formats ••Storage Storagefile fileformats formats ••Exchange Exchangefile fileformats formats
Jean VARRA_ INA 2008
30
A few figures... Digitising Digitisingfilm, film,video videoor, or,ininaasmaller smallerscale scale audio, audio,requires requireslarge largestorage storagecapacities. capacities.
–
Film digitisation: keeping the original definition of a 1 hour film. • • • • • •
–
Video SD digitisation: •
–
1 hour of 16mm film requires 595 GB ( Gigabyte) 1 hour of film 35mm requires from 1008 GB to 3715GB according on the 35 mm formats a 4K (4096x3112) scan delivers a file of 4,13 TB (Terabyte) a 2K (2048x1556) scan delivers 1 TB a HD (1920x1080) telecine delivers a file of 560 GB One hour of uncompressed video
(720 x 576) requires 72 GB
Audio digitisation: one hour of stereophonic audio •
(16 bits, 48KHz) requires 700 MB
Jean VARRA_ INA 2008
31
Current digital media for archiving
Jean VARRA_ INA 2008
32
Two ways to store digital audio and video... There Thereare are22main mainapproaches approachesto tostore storeand andarchive archive digital digitalaudio audioand andvideo videoaccording accordingto to expected expected uses usesand andstorage storage hierarchy hierarchy..
Proprietary Proprietary solutions: solutions:
Open Opensolutions: solutions: File Fileformats formats (data) (data) on on
Hard Harddisk diskservers servers
Digital Digital SD SDor orHD HDvideo video
oror data datatapes tapesoror optical opticaldisks disks
audio audiodigital digitalformats. formats.
or orwithin within
oror
on onthe the“shelves” “shelves”
data datatape tapelibraries libraries or disc jukeboxes. or disc jukeboxes. Jean VARRA_ INA 2008
33
Possible current media for digital archiving Proprietary Proprietarysolutions solutions
Open OpenSolutions Solutions
SD SDdigital digitalvideo video tapes tapes
Hard Harddrives drives (RAID (RAIDarrays) arrays)
D1,D2,D3,D5,DCT D1,D2,D3,D5,DCT Digital DigitalBETACAM BETACAM BETACAM BETACAMSX SX IMX IMX DV, DVCAM DV, DVCAM, , DVC DVCPro Pro2525-50 50
Data Datatapes tapes SDLT SDLT LTO LTO
HD HDformats formats
Optical Opticalmedia media CD/DVD CD/DVD Blu Bluray ray Holographic Holographicdisk disk??
VOODOO, VOODOO,D5 D5HD, HD, DVC pro 100 DVC pro 100 HDCAM-SR HDCAM-SR
Audio Audiotapes tapes DAT DAT
Jean VARRA_ INA 2008
34
Open media •• Magnetic Magnetichard harddrive drivedisks disks capacity capacityisisstill stillgrowing: growing:itit doubles doublesevery every18 18months months (currently 1 TB/disk) (currently 1 TB/disk) •• Used Usedininservers serversas asRAID RAIDarrays arrays ((Redundant Redundantarray arrayof of inexpensive inexpensivedisks) disks)Solves Solves22 main main needs needs: :ititincreases increasesdata data bit bitrates ratesfor forvideo videoand and reliability reliability and andtolerance toleranceto todisk diskfailures failures •• Cost Costper perGB GBisisdramatically dramatically decreasing decreasing ((Currently Currently11to to2€/GB) 2€/GB)
••
Current Currentdata datatape tapesolutions solutionsare are –– SDLT: 1/2inch inchlinear linear SDLT:1/2
recording recordingby byquantum quantum
–– LTO LTO(LTO4) (LTO4)1/2 1/2linear linear
inch inchtape tapeby byIBM, IBM, Seagate, Seagate,Hewlett HewlettPackard Packard
•• Still Stillbest bestcost costper perGB GB(0,3 (0,3€/GB) €/GB) •• LTO LTOisisthe theleader leaderon onthe themarket market
Jean VARRA_ INA 2008
35
Open media •• Blu-ray Blu-rayDisk Disk –– –– –– ––
12 12cm cmand and88cm cm Capacity: Capacity: 23 23/25/ /25/27 27GB GB data databit bitrate: rate: 36,/ 36,/72 72/144 /144Mb/s Mb/s Suitable Suitablefor forHD HDand and compressed compressed video video –– Used UsedininXDCAM XDCAM( (SONY) SONY) –– Currently Currently1,2€ 1,2€/GB /GB
•• Holographic Holographicdisk disk –– Format FormatHDV HDVAlliance Alliance/ /InPhase InPhase technology technology –– Storage Storagecapacity: capacity:200 200GB GBtoto1,6 1,6TB TB –– Data bit rate: 20 MB/s to 1 GB/s Data bit rate: 20 MB/s to 1 GB/s
Jean VARRA_ INA 2008
36
Evolution of digital media technology ••
In Inthe thedigital digitaldomain, domain,media mediatechnology technologyevolves evolvesvery veryfast: fast: –– Road Roadmaps mapsof ofthe themanufactured manufacturedproducts productsshow showthat: that: •• capacity capacity((GB) GB)doubles doublesevery every22years years •• recording recordingand andplay playback backspeed speedalso. also. –– Backward Backwardcompatibility compatibilityof ofdata datatapes tapesisis22generations generationsat atthe theutmost. utmost. –– Obsolescence Obsolescenceof of digital digitalmedia mediacomes comesvery veryfast. fast.
To Toprevent preventfrom fromthat, that,digital digitalto todigital digitalmigration migration must mustbe beconsidered consideredas assoon soonas aspossible. possible.
Jean VARRA_ INA 2008
37
Essence data Formats
Jean VARRA_ INA 2008
38
Preserving quality The use of compression allows the user to match quality with different uses, while reducing storage needs and data transfer rates.
Archive Archiveowners ownersmust mustlearn learnto todeal dealwith with several severalquality qualitylevels: levels: Archiving Archiving quality quality
Exploitation Exploitation quality quality
local localviewing viewing quality quality
Jean VARRA_ INA 2008
distant distantviewing viewing quality quality 39
Preserving quality
Uncompressed Uncompressed
?
Lossless Lossless compression compression
Lossy Lossy compression compression Jean VARRA_ INA 2008
40
The central role of a preservation master file Exploitation Exploitationfile file
Original Originalformat format Film SD Analogue video
Preservation Preservation Master Masterfile file Uncompressed Loss less compression
SD Digital video HD Digital video
“Light” compression for current use
Viewing Viewingfile file
“Visually loss less” compression Jean VARRA_ INA 2008
Compression rate adapted to networks and Internet
41
Current file formats for archiving quality data datafiles filesfor forfilm filmand andvideo video Uncompressed Uncompressed
DPX DPX
AVI AVI
Loss Lossless less compression compression
JPEG2000 JPEG2000
Lossy Lossy compression compression
MPEG2 MPEG2 422 422 (I(I frame frame only only until until 100 100 Mb/s) Mb/s)
DV DV 25 25 /50/100 /50/100 Mb/s Mb/s
data datafiles filesfor foraudio audio Uncompressed Uncompressed
AES/EBU AES/EBU
Lossy Lossy compressed compressed
MPEG MPEG L2 L2
Wav Wav MPEG MPEG 33 AIFF AIFF
H264? H264? Jean VARRA_ INA 2008
42
Metadata and exchange format standardisation
Jean VARRA_ INA 2008
43
Metadata •• Metadata Metadataisisdata dataabout aboutdata data •• Information Informationthat thatmakes makesdata data useful: useful: –– –– ––
Technical Technicalmetadata metadata Descriptive Descriptivemetadata metadata Rights Rights
•• Several Severalinitiatives initiativesrelated relatedto to different differentdomains domainsand and environment environment(( libraries,broadcast, libraries,broadcast, multimedia,consumer…) multimedia,consumer…) •• Trying Tryingto toconverge…. converge….
Metadata Metadatainitiatives initiatives Dublin Dublin core core
SMPTE SMPTE
MPEG7 MPEG7
SMEF SMEFand and P/META P/META
AAF AAF
Jean VARRA_ INA 2008
TV TVanytime anytime 44
Exchange file formats (wrappers) A Wrapper gathers in one file format the content (essence) and its related metadata for data exchange between two different systems.
Main Mainstandardised standardisedwrappers wrappers
AAF AAF
Advance AdvanceAuthoring AuthoringFormat Format
MXF MXF
Media Mediaexchange exchangeformat format
Jean VARRA_ INA 2008
BWF BWF
Broadcast BroadcastWave WaveFormat Format
45
Current digital archive system at INA Digitisation Film Film (telecine) (telecine) 2”/1” 2”/1” tapes tapes BETACAM BETACAM tapes tapes 3/4 inch 3/4 inch tapes tapes 1/4 inch 1/4 inch audio tapes audio tapes 78 RPM 78 RPM discs discs
transfer to transfer to digital betacam digital betacam Automated Automated MPEG2 MPEG2 and/MPEG1 and/MPEG1 encoding encoding (flexicarts) (flexicarts)
transfer to transfer to WAV files WAV files BWF BWF
Storage ONLINE ONLINE MPEG1 MPEG1 storage storage (NAS servers) (NAS servers) NEAR ON LINE NEAR ON LINE MPEG2 MPEG2 storage storage (LTO 4 in (LTO 4 in ADIC library) ADIC library) distant distant BACK UP BACK UP STORAGE STORAGE HARD DRIVE HARD DRIVE SERVERS SERVERS
Delivery Browsing Browsing and viewing and viewing intranet intranet Internet (Ina media) Internet (Ina media) delivery of viewing copies delivery of viewing copies (DVD,VHS, DV….) (DVD,VHS, DV….) Delivery Delivery delivery of exploitation copies delivery of exploitation copies (Digital BETA, IMX, DVCPro….) (Digital BETA, IMX, DVCPro….) Delivery through dedicated links Delivery through dedicated links (TF1,FTV) (TF1,FTV) delivery over networks FTP delivery over networks FTP (all customers) (all customers)
78 RPM 78 RPM discs discs
INA Information system ( Databases,workflows, networks,commercial and rights management,... Jean VARRA_ INA 2008
46
Near future evolutions...
Digital DigitalBETACAM BETACAM
File File lossless losslesscompression compression JPEG2000 JPEG2000(SD (SDand andHD) HD) other? other?
MPEG2 MPEG288Mb/s Mb/s
MPEG2 MPEG2 higher higherbit bitrate rate or orH264 H264HD HDand andSD? SD?
MPEG1 MPEG11Mb/s 1Mb/s
H264 H264400Kb/s 400Kb/s
Jean VARRA_ INA 2008
47
A temporary conclusion about digital technology... •• Digital Digital media mediaand andtechnologies technologiesare areevolving evolvingand and renewing renewingextremely extremelyfast. fast. •• In Inthe thedigital digitaldomain, domain,we wemust mustlearn learnto tothink think of“everlasting of“everlasting file” file” and and forget forget the the idea idea to tofind findan an “everlasting “everlasting media” media” for for aa while. while. •• Once Onceanalogue analoguemedia media has hasbeen beendigitised, digitised,the thejob job is isnot notfinished, finished,we wemust mustforesee foreseeand andprepare preparethe the regular regular“migration” “migration”of of data datafrom fromcurrent currentdigital digital media mediato tonew newones. ones.
Jean VARRA_ INA 2008
48
A temporary conclusion about digital technology... •• The Thechoice choiceof ofaa suitable suitablepreservation preservationmaster masterfile file is isfundamental. fundamental. •• IfIf standardised standardisedessence essencedata dataformats formatsare aremore more and andmore moreused usedin intelevisions televisions(MPEG2, (MPEG2,DV, DV, …),there …),thereis isstill stillaalong longway wayfrom fromtheory theoryto to “real” “real” use use of ofstandardised standardisedmetadata; metadata; •• For Forstandardised standardisedexchange exchange formats, formats, MXF MXFand and BWF BWFseems seemsto totake takethe theleadership. leadership.
Jean VARRA_ INA 2008
49
Thank you for your attention and don’t let your archives become like that!
Jean VARRA_ INA 2008
50
Jean VARRA_ INA 2008
51