«This is a slightly modified version of the narrative submitted to NEH as part of the Sound Directions proposal. It has been updated in a few places ...»
This is a slightly modified version of the narrative submitted to NEH as part of the Sound Directions
proposal. It has been updated in a few places to reflect both our increasing understanding of the
worldwide context in which our project will take place and the recent publication of IASA TC-04,
Guidelines on the Production and Preservation of Digital Audio Objects.
I. Significance of the Project
Sound archives have reached a critical point in their history marked by the simultaneous rapid deterioration of unique original materials, the development of expensive and powerful new digital technologies, and the consequent decline of analog formats and media. It is clear to most sound archivists that our old analog-based preservation methods are no longer viable and that new strategies must be developed in the digital domain. The Indiana University Archives of Traditional Music (ATM) and the Archive of World Music (AWM) at Harvard University propose a joint technical archiving project—a collaborative research and development initiative with tangible end results—that will create best practices and test both emerging standards and existing practices for digital preservation.
The Sound Directions project focuses on field recordings—carriers of unique, irreplaceable and historically significant cultural heritage. As caretakers of these collections we must solve the problem of preserving audio resources accurately, reliably, and for the very long term; at the same time we must make our resources readily accessible to those who most need them. These issues have been the subject of work, discussion and study at a number of national agencies and institutional archives, including the Council on Library and Information Resources, the American Folklife Center, the Library of Congress Audio-Visual Prototyping Project, the Archive of World Music at Harvard University and the Archives of Traditional Music at Indiana University. Most of us are now approaching audio digitization in similar, deliberately cooperative ways. There are few published standards or best practices for audio preservation.
Committees of the Audio Engineering Society and the International Association of Sound Archives (IASA) have written best practices for some parts of the audio digitization process. However, the analog to digital conversion process is not complete until safe and secure storage is attained and a way to insure readability over time is developed. In addition to developing best practices in a number of areas, the work proposed by the ATM and the AWM builds on collective experience and recent work on audio digitization in an important new way: it contributes the final step to the process--the creation of interoperable digital audio preservation packages, containing audio essence and metadata, following the OAIS model.. This is a step that has never been taken before for archival audio. Only when we can feel assured that we have new programs in place that ensure the survival of our threatened cultural heritage, can we reliably take advantage of the dramatic expansions of access that digitization and the Internet afford.
The development of best practices and standards in many areas, especially the production of interoperable audio preservation packages, is the essential and exciting next step to insure the preservation of our national heritage of fragile and deteriorating recordings. The ATM and the AWM are poised to lead the way forward into this new frontier of digital preservation and access by initiating a highly collaborative and consultative research and development process, the results of which will be widely disseminated.
With the Sound Directions project we will:
a) Develop best practices and test both emerging standards and existing practices for archival audio preservation and storage in the digital domain and report our findings back to the field;
b) Establish, at each university, programs for digital audio preservation that will enable us to continue this work into the future, and which will produce interoperable results. This is groundbreaking work that is considered to be the next necessary step for true digital audio preservation and access;
c) In the process, preserve critically endangered, highly valuable, unique field recordings of extraordinary national interest.
While best practices have been and are being developed for the initial digitization process, they do not exist in many areas of the preservation chain. We will develop best practices by testing different procedures and techniques in selected areas including: specifications for master preservation files from different analog sources, management of digital files including names, announcements and embedded material identifiers, down-sampling and creating derivatives, ingestion and storage of digital audio objects in digital library repositories, implementation of preservation services (including data integrity checking) for digital audio in digital library repositories, quality control and checking procedures, and the interchange and reading of preservation files constructed using METS in an Archival Information Package (an AIP, following the Open Archival Information System model) between institutions. In each of these areas, and others, we will report on procedures and techniques that produced the outcomes we were seeking to the quality desired, as well as procedures that did not work within both the ATM and AWM preservation systems and workflows. This will be the most comprehensive and detailed development of best practices to date, covering many critical areas in the preservation chain. It will also, as noted above, be the first time that interoperability has been achieved for digital audio preservation packages from two archival institutions. This will yield valuable data from two different operations and perspectives for use by other institutions designing audio preservation projects.
B. Interoperability Simply put, if every institution’s buckets of bits are different in character they are idiosyncratic--not interoperable--and true preservation has not occurred. Real preservation depends on the usability and readability of files over an extended period of time. In addition, should one institution fail, this type of interchange guarantees preservation by enabling any engineer to access preserved content.
Interoperable files depend on appropriate metadata to insure readability over time, and the development of best practices for the collection of metadata will be a critical part of this project. To digitize and store a recording so that it can be migrated and preserved, descriptive, administrative and technical metadata are essential in order to understand and interpret the digital object. Opaque digital objects are difficult if not impossible to preserve. The development of compatible Submission Information Packages (SIPs), as proposed in this project, lays the groundwork for defining what constitutes a preservation object. The standards for the SIP already developed at Harvard offer a good place to start. Developing these standards further at two different institutions is critical, and the process of submitting them to the scrutiny of other professional engineers and digital library experts will enable further refinements. The lead engineer at the AWM, David Ackerman, has not only fostered the development of Harvard’s audio preservation efforts in this area, but is guiding the creation of technical metadata standards for audio internationally by leading the Audio Engineering Society’s SC0306, the Working Group on Digital Library and Archive Systems.
Further, Sound Directions will demonstrate that it is possible for different institutions to work within their differing workflows and physical settings and still attain preservation through the production of interoperable results. Thus the information generated through this work will generalize to other institutions who want to use the project’s innovations but cannot redesign their audio studios nor completely alter their staffing situations in order to do so. Working together, the ATM at Indiana and the AWM at Harvard will develop methods and best practices that are largely system-independent, that can be adopted by other institutions without overhauling their existing operations.
One important byproduct of our project will be the creation by grant programmers of tools for technical metadata capture, workflow management, ingestion of preservation files and the dissemination of
C. Content and Access The recordings chosen as test cases for Sound Directions will be drawn from the rich, outstanding and unique ethnographic field collections of the Archives of Traditional Music at Indiana University and the Archive of World Music at Harvard University. A complete list of these materials appears in Appendix E. Field collections have been selected based on the following criteria: a) research and cultural value; b) preservation needs; and c) recording format (in order to test the transfer of a range of formats for this research and development project.) At AWM, selected collections include historic field recordings from Egypt, Iraq, Iran, Afghanistan, Pakistan and India, unique documents of cultural history from regions of tremendous interest to Americans today. At Indiana, selected collections include critically important cultural materials such as music of Iraqi Jews in Israel, music from pre-Taliban Afghanistan, music related to the world’s longest-running civil war in Sudan, and African-American protest songs from the 1920s through the 1940s.
Sound Directions is conceived in two phases, the first of which is the subject of this proposal. While the focus of Phase 1 is research and development in areas critical to audio preservation, the project will also result in the preservation of the above collections along with the creation of basic access to these materials. Phase 2 of the project, which will require a follow-on grant, will emphasize access. In Phase 2 each institution will create on-line digital audio archives beginning with the collections selected for Phase
1. The ATM will build the Cultures in Conflict Digital Archive (CCDA), creating on-line access to the recorded heritage of peoples around the world whose cultural practices have been threatened or abolished as a result of conflict. The AWM at Harvard will create a digital archive using its rich historical collections of classical and folk music from Iran, Iraq, Pakistan and India. Both institutions will also pursue a program of “digital repatriation,” making access copies available to nations and communities whose recordings we house. Critical preservation problems, however, must be solved before we can move to providing this extended access.
D. Why this Project Now?
Recent years have brought forth significant public concern about the value of unique audio collections and the pressing need to reformat them to insure their survival. In the United States, the Council on Library and Information Resources (CLIR) wrote, in its proposal to survey audio collections that “collections of recorded sound are an irreplaceable record of the history and creativity of the twentieth century.”1 The proposal notes that “awareness that our audio heritage is in peril has reached the highest levels of government, but the needs remain great.”2 Efforts to make progress addressing the problem include the summit-type meetings such as the federally funded conference Folk Heritage Collections in Crisis in December of 2000 http://www.clir.org/pubs/reports/pub96/contents.html, the Save Our Sounds project at the Library of Congress that ensued http://www.loc.gov/folklife/sos/, the CLIR survey of unique audio collections held in academic libraries http://www.clir.org/pubs/abstract/pub128abst.html and the Sound Savings symposium at the University of Texas in 2003 http://www.arl.org/preserv/sound_savings_proceedings/introduction.html. Numerous workshops flowing from these efforts were held at professional societies to address the inextricably linked issues of preservation and access to these recordings, making the point that there effectively is no access without preservation. Indiana and Harvard have been integrally involved in these conversations.
At this time, the International Association of Sound Archives, along with many sound archives around the world, have come to the conclusion that long-term preservation of information contained on analog media requires transfer to the digital domain.3 Sweeping endeavors such as the Library of Congress’ AudioVisual Prototyping Project, and local ones including Harvard’s Music from the Archive project, Indiana’s
What is needed at this time is for these leaders in the field to move forward with the knowledge we have to develop more detailed and comprehensive best practices, test emerging standards and engage in the production of interoperable digital preservation packages.
E. Sound Directions and Other Projects in the United States We have researched a number of audio digitization projects and published recommended practices, looking for efforts similar to ours. Although we found no working projects that were as comprehensive or detailed as what we are proposing, we did find three projects with which we share certain commonalities or to which we have looked for insight.
1. Audio-Visual Prototyping Project at the Library of Congress http://www.loc.gov/rr/mopic/avprot/avprhome.html The audio preservation staff at the AWM has been keenly aware of the development of the Culpepper facility at the Library of Congress. AWM has created preservation procedures that are in step with this planned facility. Carl Fleischhauer advised us initially4, and both David Ackerman and Robin Wendler (from AWM and the Harvard University Library Office for Information Systems) were then invited to offer extensive comments on the metadata issues in the Culpepper plans. Sound Directions will make use of many ideas generated in the planning for Culpepper, instantiating them into an actual project while extending them into the domain of interoperable files, an area that LC has not yet addressed.
2. Digital Audio Archives Project at Johns Hopkins and Indiana Universities