Commentary on the Pittsburgh University Recordkeeping Requirements Project: A Progress Report (Delivery Draft).
Seamus Ross, Assistant Secretary (Information Technology), The British Academy (London) & Co-Director of the Centre for Information Management and Advanced Technology (CIMATS), London Guildhall University.
Society of American Archivists, 59th Annual Meeting, Washington DC [Thursday 31 August 1995, Session 20].
(An electronic version of these comments is available at http://britac3.britac.ac.uk/seamus/saacom).
I am delighted to be here and to act as a commentator on this Project. Sometimes I chair conference sessions and it can be a tough job to find something constructive to say about each paper or just that perfect question to bring out the best in a speaker. Other times I have presented papers myself and hoped that the chair was as generous to me as I hoped I had been to presenters in sessions I had chaired. But I have always avoided invitations to act as a commentator because a commentator must be insightful, critical but encouraging, diplomatic, promote discussion, and sometimes, as on this occasion, have the oratory and stylistic skills of Shakespeare's Mark Anthony. So last year when I was invited to act as a commentator on a project defining the functional requirements of electronic records my first inclination was to say 'no'.
The documentation in the form of Reports and Working Papers arrived in two great packets several months apart and my comments are primarily based on this material. I have talked with friends and colleagues and learned much from them, but in case I have misunderstood their views I shall not mention them by name. It is obvious from the presentations of Professor Richard Cox and Wendy Duff (University of Pittsburgh) that the project has moved on in the past six months, but this work does not substantially alter the conclusions I had reached. The project is exciting, could potentially fill a void in the field of electronic records management, and has a valuable role to play in the construction of products that incorporate record management capabilities. If the project is to develop a coherent `product', it will, to my mind, take some effort during the next six to nine months to pull all of its fascinating strands together. My experience of project management suggests that this is often required during the final phase of a project.
At the heart of this project is a reconceptualization of the principles of records management and archiving in the face of the new complexities posed by electronic records. A central hypothesis is that organizations will wish to maintain access to the records they create where a continued administrative need whether this is legal, evidential, or historical, for these records can be demonstrated. Ideally the records will not be documented by record managers or archivists, but at creation stage they will be encapsulated by metadata. This metadata will be sufficient to support the reconstruction of the decision-making process or provide audit trails throughout the records lifecycle. The project is developing several of the fundamental building blocks necessary for this reconceptualization.
II. The Problem of Metadata
The first problem about 'the metadata' is that while they are impressive in abstract terms, they are not derived from what organizations actually want, but they are abstracted from the published literature. The test sequence, as described in the working papers, applied at the five study organizations pre-disposes these organizations to focus on the functional requirements as identified by the project and not on attempting to derive them from the business processes of the organizations themselves. A more objective test would have involved independently deriving the functional requirements and metadata from an examination of the corporate needs and matching the results of this study against the data collected from a study of the published literature. Since the metadata categories were not derived from the analysis of the business process there is no way of verifying that they reflect the needs of business communication. Organizations and software developers may not, therefore, be prepared to risk implementing them, or perceive commercial value in their implementation.
The concentration on the definition of metadata apart from the processes which need to be operated on the metadata means that the project is only partially examining the issues associated with metadata. A view of processes would help us to understand how metadata is used and in turn help us to define more accurately the elements of metadata which need to be captured. As Avra Michelson pointed out 'metadata in and of itself is static and does nothing' (Michelson, 10 Jul 1995 11:07:25, BA/SR/I10079563). Only by manipulating the metadata does it have any value in support of business acceptable communications. The project must examine what 'processes need to be operated on the metadata in specific application environments' to give it value (ibid.). It would be possible to argue, however, that this project aims only to identify the underlying concepts and determine their validity, accuracy, and applicability. Processes could therefore be ignored at this stage and only examined at product design or implementation stage. A compelling counter argument holds that the functional requirements can only be verified within the context of processes: that is by evaluating the uses to which metadata will be put in the business process. How will metadata be manipulated to improve, for instance, management of electronic mail records.
III. Tactics and Organizations
This brings me to the issue of tactics. The project team has theorized that the way the functional requirements and metadata might be introduced into an organization would be through tactics--policy, design, implementation, or standards. But first there needs to be some evidence that the creation of metadata encapsulated records has corporate value. Organizations only preserve data if they have a proven need, legal or otherwise for it, or they have found value in historic data (e.g. marketing campaigns). Here is where the problem of interfacing with organizations becomes most crucial. The project has attempted to establish a typology of organizations which might help to inform strategies for introducing the concept of functional requirements into recordkeeping. There do not appear to have been investigations as to how this categorization of organizations would assist in practice implementation. It is assume that the introduction of new systems will be facilitated by being tailored to an organizations special needs, but the whole point of introducing new technology may well be to re-engineer corporate attitudes and processes. Since the work on organizational typologies is only based on a sample of five organizations, even this engagement is suspiciously thin. There are obvious omissions from the sample: financial institutions, central government, voluntary organizations to mention but three. We were not told how the organizations in question were selected but if the sample is of organizations which were interested and willing to participate, then they are already a biased sample. A full study of record retention and data warehousing strategies across the spectrum of organizations is an important next step. It might be useful to get information from organizations hostile to the whole concept of records management. Because the actual studies are limited and do not cut across the range of organizational infrastructures further work will be necessary if a methodology for implementation is to be developed. Such a method or model will be essential.
There has not been much discussion about organizational complexity, structure and environment as motivating or controlling factors in the use of or creation of recordkeeping systems which include these functional requirements. It is likely that functional requirements can be introduced successfully in centrally managed systems, but not in the world of the devolved budget. This contrast would be best borne out by comparing a financial institution with a local government in the UK. It was hardly surprising to hear that there was 'no relationship between individual corporate culture profiles and the choice by organizations of tactics to be employed' (Cox, Working Paper Introduction, 1995). Of course one of the problems with the organizational profiles is that they fail to give us a sense of the role the informants had within their respective organizations. Also if they do not examine the issue of the value of record access to the organization the project will not be in a position to design models which improve awareness of the business value of records. Awareness of the commercial value of records in increasing productivity, (and profits) is probably one of the most significant factors in promoting record preservation.
Furthermore the organizations introducing these systems will not be technology free environments. The project included among its objectives:
'the identification of variables in organizations that affect the way in which both software and hardware are utilized and which may affect the degree to which recordkeeping functional requirements can be adopted.'
In many ways the organizational culture studies does not take the variables to a low enough level. The huge investments in systems, and staff training which had already been made by organizations as they built up their IT infrastructure will have a remarkable impact on the take up of these concepts and on systems which implement them. Indeed the pervasiveness of IT infrastructures and complexity as well as the role of technology in the activities of the organization will probably have a great influence on the implementation of the functional requirements. These variables need to be defined.
From an organizational point of view I suspect that a business case would need to be built if organizations are to be encouraged to adopt strategies supporting proper archival management of electronic records. This case could form a part of the management training module. Indeed information about the archival value of data needs to be driven home to students on business courses. While numerous companies are just beginning to discover its value, many others have not yet identified the value of data. In addition to producing articles about the project in archival journals the team need to discuss their work in places such as the Harvard Business Review. Eventually a culture change could be brought about which made record documentation a component of the business process.
IV. Problem of what is compliance?
One of the major reasons for defining the functional requirements is to encourage organizations to determine whether or not their systems are compliant and if not where the areas of deficiency lie and how severe they are. Actually measuring compliance is difficult because there are no benchmarks against which it can be done and no guidelines for how to do it. This means that evaluation programs tend to be, like the paper by Heo and Murray in volume 2 of the Working Papers, very subjective and ad hoc. The sample evaluation which the project team has done based on product reviews in magazines is a case study of how not to evaluate software (and systems) for compliance, but this seems to have been a conclusion which the project itself has now reached (Cox, 1995: 29). Last Friday (25.08.95) morning I spent three hours attempting to apply these functional requirements to evaluate the British Academy's accounting systems. Even if it had been possible to complete this audit, it could not have been considered an unbiased evaluation of this system, however, because I was responsible several years ago for managing the project which implemented it and of course hoped it would comply with the functional requirements as identified by the project. What I can say is that asking the questions of the system in an attempt to find ways to measure compliance was a rewarding way to develop an understanding of the role of functional requirements. This little exercise reinforced my belief that in order to deliver successfully the functional requirements the project needs to develop objective measures of compliance and guidelines for compliance auditing
The attempt by David Bearman and Ken Sochats to formalize the functional requirements has not reached a testable stage because it has not been fully implemented in software. In conceptual terms production rules provide an excellent medium for implementing the functional requirements, but they can only be tested by prototype implementation in an expert system. This is because currently there is no proven strategy for verifying objects, attributes, values, and rules for deficiencies such as circularity, unresolved interdependencies, or coverage. Such an expert system would support the fuller testing of the practical relevance of the functional requirements. Had this expert system been prototyped earlier in the project it could have been used both for the software evaluations and organizational studies. As the project accumulated a more and more detailed and refined understanding of the functional requirements the expert system could have been incrementally enhanced to reflect this. This system would of course itself be self-documenting so that the records produced by its audits would form a corporate resource. The tests for compliance should also include the fourteen questions proposed by Sochats and included in the Introductory Paper of the second set of working papers by Richard Cox. These form a very good mechanism to contextualize compliance data.
V. Cultural Warrant and Software
I found the cultural warrant an ingenious idea. It is though based on a very small number of sources and this needs to be expanded dramatically. It is also very much an American cultural warrant, but the problem is an international one. The warrant needs to identify a demand for culture change much wider than 'need for'. What is really needed is evidence of a need to use culture change.
Software developers must be brought on board and convinced that their systems should support metadata. To do this the project must rework its metadata so that they form guidelines for implementation into software. Currently they are a very useful checklist. In the end it will be important to ask how many of the features could be implemented into software. In this regard the work of the National Historical Publications and Records Commission (NHPRC) funded Indiana University Electronic Records Project will be an important test case.
VI. Knowledge and Historians
The quantity and richness of knowledge of the cultural framework necessary to understand information or records increases as the time distance from their creation increases: so what is suitable contextual information for contemporary data reuse may not be suitable for reuse of those data 100 years from now, say climatic, pollution, or nuclear waste disposal records. In other words overtime increasing amounts of cultural, social, and linguistic information becomes essential to understand the context of a record. Metadata reflect current understandings of evidentiality and recordness and change overtime. The effect of these factors on metadata encapsulation strategies cannot be ignored. This means that the question of 'what is the period of historical/record relevance?' must be asked. And we must also ask at what stages the contextual metadata is to be enriched to reflect this increasing distance, and who should add this new metadata.
My interest in electronic records and their preservation reflects my early training as a historian and archaeologist. For me the period of historical relevance is measured in 100s, if not 1000s of years. It also pre-disposes me to the great variety of stories that these records and artifacts allow us to tell about the past. In our multi-cultural society this richness is crucial. As important as all the work being carried out by the project will prove to the preservation of records for the business process one issue which concerns me is the lack of consideration which has been given to the preservation of records for cultural and historical purposes. Historians always rely on the residue of past; information derived from a small number of surviving records. When Le Roy Ladurie was writing Montaillu he based his ethnohistorical study of this Cathar village in Southwestern France on the one surviving volume of the inquisition record for the village. (It is known that originally at least two other volumes existed.) But few categories of records have so contained the seeds of their own destruction as electronic records do. These new self-documenting and self-managing record retention systems may result in our leaving to future generations an inaccessible but sanitized record of our world. In this I think of Dr M. Brosius' groundbreaking study of women in ancient Persia from Cyrus to the sack of Persepolis by Alexander the Great which was based on an analysis of thousands of clay tablets, shaped like soap bars, which recorded economic transactions. Women in Ancient Persia has changed our perspective of women in the ancient near east: some owned large estates and presided over great wealth, some were leaders of large workgroups, and all were remunerated equally with men for the same work. While the informational content of these records has served one purpose, a second study is revealing how the archive worked by studying the form and format of the tablets themselves. These understandings of the past bring life to the present and help us to contextualize who we are and where we have come from.
VII. Repackaging and Conclusions
The message of this material needs to be repackaged if it is to reach a wider audience. These papers are intended for those already conversant with electronic records issues. A software developer identifying strategies to incorporate the functional requirements into, for example, document management software might find this material difficult to use. In order to reach these diverse audiences, much work needs to be done to address the issues of terminological clarity (e.g. what is a functional requirement, strategies, metadata), and the stylistic and linguistic quality of some papers needs to be pulled together. Quite possibly when completing the final product the project might wish to engage the services of a professional editor. But even if the papers are made clearer and supported by a richer array of examples there is no getting around the need for the project to develop a suite of training modules. Training will be necessary in at least four areas and I would suggest that as an adjunct to the project these modules be developed. They should result in a freely distributed training courses--possibly a web-based interactive courses would be the way forward. These courses should be in management awareness, functional requirements of archival and records management, software implementation and use. The later course is unlikely to come from the project because it will depend upon individual software implementation.
The work of the project team is laudable and I suspect that if it is successfully completed it will have a very positive long-term influence on electronic records. To bring the Project to conclusion:
(1) the limitations the project imposed by working practices (e.g. process by which the functional requirements were defined, omission of study of processes, ad hoc manner for selecting organizations for study, limited number organizations reviewed) must be clearly outlined;
(2) the material must be repackaged to give it coherence and to make it more accessible to the variety of constituencies which have an interest in these issues (e.g. business and government organizations, software developers, record managers);
(3) marketing needs to be carried out to encourage system developers to incorporate metadata elements into systems (either as add-on modules or as product features); and,
(4) methods and software for evaluating systems for compliance must be written.
Because the objectives of the project were not defined in a way that makes it possible to measure how well they were met, the NHPRC may have difficulty measuring the success of the project and whether it has given value for money. The hard work of Bearman, and Cox and his students will have lasting value if it is successfully completed and, to my mind, successful completion of this ambitious and critical project would more than justify NHPRC investment. Thank you.
 Copies of these documents can be obtained by contacting the Project (email@example.com). Some are available from http://www2.lis.pitt.edu/~sochats/nhprc.html
 The project began by defining a series of functional requirements for recordkeeping. P.C.Hariharan has focused his discussion on one of the fundamental limitations of the project caused by the project having used inadequate methods to define these functional requirements.
 Eddy Higgs of the Wellcome Unit for the History of Medicine at the University of Oxford and a Visiting Research Fellow at CIMATS and I have discussed at length the issue of process re-engineering and the role it plays in the deployment of systems using functional requirements.
 For more information about this project Philip Bantin at the University Archives (email: firstname.lastname@example.org).
 To be fair Wendy Duff's explanation of the functional requirements in the context of the cultural warrant delivered at the 1995 SAA Conference is the clearest I have heard or seen to date.
 As Richard Barry pointed out during the discussion which followed the formal papers in Session 20 the tactics should be extended to include training.
 The comments of P.C. Hariharan provide a crucial window on this issue and these must be addressed.