The UC Berkeley Digital Environmental Library:
Needs Assessment and Evaluation

Nancy Van House
Mark Butler
Lisa Schiff
School of Information Management and Systems
University of California, Berkeley
Submitted to
ISIC 96
Information Seeking in Context
Tampere, Finland, August 14-16, 1996
Correspondence:
Prof. Nancy Van House
School of Information Management and Systems
102 South Hall #4600
University of California
Berkeley, CA 94720-4600
USA
email: nav-lis@cmsa.berkeley.edu
phone: 510-642-0855
fax: 510-642-5814

The UC Berkeley Electronic Environmental Library Project is a multi-disciplinary project funded under the NSF/NASA/ARPA Digital Libraries Initiative. The goal is to develop a massive, distributed, electronic, work-centered library of environmental information containing text, images, maps, numeric datasets, and hypertextual multimedia composite documents to support environmental planning decisions. By work-centered, we mean designed to support the work of the users, in this case, environmental planning.

The goals of the user needs assessment and evaluation component of the project are to improve our understanding of the use of information for complex cognitive work; to develop conceptual bases and methods for evaluating digital libraries; and to support the development of the UC Berkeley Digital Environmental Library by providing the other components of the project with an understanding of the users, their context, and their work; to advise on content, functionality, and interface design, at the design stage, based on our knowledge of users and uses; and to provide feedback on designs and prototypes, by means of heuristic evaluation, small-scale user feedback, and larger-scale user testing.

The fundamental premises of the user needs assessment and evaluation efforts are that design should be user-centered, situated, iterative, and adaptive. We are using a variety of qualitative methods. We have conducted extensive interviews, learning about people's present work processes and tools and their perceived needs for a digital library. We are experimenting with a variation on thinking-aloud protocols that consists of videotaping users interacting with the system and then asking them to review and comment on their tapes.

The emerging model of information needs and uses assumes that information needs arise and information is used in the context of particular, concrete circumstances: the user's own history, the environment, and the task and situation at hand. In other words, information needs are uses are situated. Three research literatures address sense-making: cognitive psychology is concerned with individuals' knowledge representation and processing. The communications/information studies literature is concerned with the role of individual sense-making in information needs and uses. The organizational literature is concerned with how organizations make collective sense, which facilitates information processing and decision-making; and how they instill this sense in their members.

Information tools are themselves both products and determinants of sense-making. "Every human tool relies upon, and reifies, some underlying conception of the activity that it is designed to support" (Suchman, 1987, p. 3).

Several features of this project are particularly notable in the overall context of digital libraries and in considering the role of sense-making in digital library design, use, and evaluation. First, we are addressing several levels of users' work simultaneously: the overall context of their task, in this case, the environmental planning in California; the individual's work (e.g., water engineering) and tasks (e.g., designing a specific engineering project); his or her information acts (including information searching, analyzing), and finally his or her digital library use.

A second critical feature of this project is that we are working with actual users doing real work. The content of the digital library is that needed for this work.

A third significant feature is that our users are varied, ranging from scientists and engineers to public information specialists and the public. We are studying multiple, overlapping, and sometimes antagonistic communities of practice united by their common concern for an issue area but differing on their scientific expertise, familiarity with environmental planning, the frequency with which they use the Digital Library, and their access to and skills with information technology. A key question addressed by this project is, Can we design an effective digital library to serve multiple communities of practice, some of which are undefined and unstructured? What adaptations may we need to make to different communities of practice? This is an unusual opportunity to test the prevailing hypotheses that different communities and organizations will have different information needs and behaviors; and to look in detail at these groups' responses to a work-based information system.

Another critical feature of this project is that ours is a developing, research-based system. We are at the leading edge of design in several areas, adapting our research to users' reactions to our experimental tools.

This is the second year of a projected four-year project. We currently have preliminary findings about the differences in needs, uses, and preferences among our differing but interrelated communities of practice. By the time of the conference we will have progressed yet further in understanding different communities of practices' needs for and responses to this digital library.