The Power of Aggregation

By John Rees

We all appreciate the convenience of the modern shopping experience. Who doesn’t love visiting a local farmer’s market on a Saturday morning to browse all the variety of local produce and meats from nearby farms, or logging in to your favorite online shopping site to find that perfect shoe for a fancy night on the town or gift for a loved one. These are just a few examples of the power of aggregation—places that offer one-stop exploration of goods and services available from a myriad of sources.

NLM’s History of Medicine Finding Aids Consortium service strives to provide a similar environment for customers who are interested in the rich primary source information found in historical documents, personal papers, business records, and other resources maintained in archival collections related to the history of medicine and its allied sciences. Through agreements with archives in the U.S. and Canada, NLM harvests from the open web and indexes the finding aids describing these collections. Finding aids are the tool archivists create that provide contextual information about archival collections, often with detailed inventories, to help researchers locate relevant materials.

For example, someone using the Consortium to search about the life of renowned surgeon Michael DeBakey will find finding aids for 26 collections in 16 repositories spreading from Maryland, New York, Texas, Utah and places in between. A historian planning their summer research travels would benefit greatly with this knowledge.

The Consortium is a discovery tool providing keyword search services across a union catalog of finding aids describing archival collections. Currently it indexes over 8,000 finding aids from 48 special collections and archival repositories throughout the U.S., and one Canadian friend. We even harvest finding aids from another consortium—a consortium of consortiums! The service leverages NLM’s enterprise search engine IBM Watson Explorer. Using a variety of crawl protocols that are target-site specific, we are able to crawl, index, and provide access to finding aids that exist in a variety of data formats such as xml, html, or pdf. By crawling and indexing content locally and linking back to the owning repository, NLM can offer a multi-institutional discovery service, without the burden of managing external data. Crawls are currently performed on a monthly basis to capture changes over time.

Links to each institution’s finding aid not only provide information about the collection, but often also contact information to inquire about hours of operation, access requirements, collection availability, online contents, and reproduction services. Mining the search results traffic also helps NLM understand the current topics of interest to the research community and locate gaps in our collections that we and our partners might fill.

Each year NLM archivists seek out new repositories to add to the Consortium. Several institutions recently added their content to the consortium including Orbis Cascade Alliance, Archives West, University of Chicago Special Collections, Minnesota Historical Society and the University of Wisconsin Digital Collections, Archival Resources in Wisconsin. If you are from a repository and have at least 10 history of medicine-related archival collections, or know of one not already on our list, we’d love to hear from you.

Editor’s Note: On April 15, 2022, the National Library of Medicine (NLM) History of Medicine (HMD) Finding Aids Consortium was retired due to declining use and the technology advancements of other common web indexing services. Resources from most HMD Finding Aids Consortium members can be found in OCLC’s ArchiveGrid. If you have questions or suggestions, please contact NLM Customer Support.

John Rees is Archivist and Digital Resources Manager for the Archives and Modern Manuscripts Program in the History of Medicine Division at the National Library of Medicine.