(This article was first published in Against the Grain – June 2017, v.29 #3)
by Roger Schonfeld (Director, Library and Scholarly Communication Program, Ithaka S+R)
The need for new metrics in research libraries is well established. Some have described this need as being a matter of switching our thinking away from inputs towards outcomes, or away from how much we spend to how much value we create. These are absolutely important ways of understanding why universities should invest in their libraries and a positive direction for metrics. But in parallel, academic research libraries are making a strategic pivot, from an emphasis on general collections to an emphasis on more distinctive collections, partnerships, and services. As the contributions of a library shift, so should the metrics for evaluating its success. We need to shift not only away from an undue attention to inputs, which is complicated enough, but I am kept awake wondering how we move to ways of defining and measuring success that are appropriate to our strategic directions. Here is some preliminary in-process thinking on these topics.
A New Strategy
Demographic, fiscal, technological, and other types of change are today impacting every type of higher education institution. As higher education institutions look to differentiate themselves, their libraries are equally pursuing distinctive strategies. No longer is it the case (if indeed it ever was) that every library simply wishes to build the largest collection it can afford. Instead, libraries are looking to distinguish themselves for the services that they can provide in support of their parent institution’s research and/or educational mission.1
Broadly speaking, research libraries are pursuing a wide-ranging transition. Ultimately, they will provide less value by offering general collections of published materials, duplicated at other institutions, which are increasingly selected through bundled content, vendor profiles, or through an on-demand basis. Even if they spend a substantial amount of resources on these general collections, they recognize that their source of differentiation and value-add will be through distinctive collections and partnerships and services in direct support of research, teaching, and learning. The arc of these transitions is outlined in Figure 1.
Defining Desired Outcomes
The purpose of this strategic repositioning is to realign the library with the larger objectives of the parent university. The larger objectives of the university can also be seen as desired outcomes for the library:
- Maximize the productivity of the research enterprise
- Ensure student success, including retention, progression, completion, as well as learning and later-life outcomes
- Enhance the university’s reputation and its ability to attract students and scholars
- Increase grant support, public funding, and other sources of revenue
- Engage and include communities
Such outcomes are the work of the university as a whole. For almost any desired outcome, the library will be but one factor among many that makes a contribution. The library can make an important and meaningful contribution to each of these potential outcomes but it cannot single-handedly ensure them and therefore cannot be solely responsible for them. While desired outcomes are helpful in positioning the library’s strategy, it is exceedingly difficult to measure the library’s (or almost any other university unit’s) contributions to these types of outcomes.
While there has been a substantial move to correlate library investments and usage with these types of outcomes, perhaps outputs — the services that libraries are providing — constitute a cleaner way to describe the library’s value. Defining success at a service level may not suffice in every case, but it has the benefit of statistical validity.
To take just a few examples, the library might commit to:
- Provide collections and services (in person and virtually) to 10% more visiting researchers/students than it did last year, as one vehicle for enhancing the university’s reputation;
- Engage every faculty member in a one-on-one setting with a librarian during the course of the academic year, to stay up to date on their needs and ultimately maximize the productivity of their scholarship;
- Interact with every first-year student during their first semester on campus, serving proactively to support student success; or
- Increase by ten percentage points (compared with last year) the share of faculty publications that comply with open access and other appropriate policies, to increase and maintain grant and other forms of support.
Each of these proposed outputs is ambitious in itself. Each has the benefit of defining success in areas where the library can exercise agency, if not alone then in a clear partnership. Drawing together a strong narrative about why each of these output measures is an appropriate mechanism to bolster the university’s desired outcomes is a key part of this exercise.
Don’t Abandon Inputs
Some observers have expressed their concerns that research libraries devote too much attention to inputs, such as number of staff and size of budget. As raw figures, these measure little more than the amount of resources given to the library by its parent university and provide no indication of strategic direction or success. But when organized and analyzed, inputs can help leaders to track the implementation of their strategy, both within individual large and complex organizations and also across the community more broadly.
Although all libraries understand their top-line spending on employee compensation and on materials, few if any can associate this unambiguously with their strategic direction. As I have argued elsewhere, many library leaders do not have a clear understanding of how employee time–the most precious of all resources–is allocated. As a result, they find it difficult to track internally whether their organizations are allocating resources appropriately in response to a stated, or even implicit, strategic direction.2
Every library has its own strategy, and I make no claim that identical metrics are appropriate across all libraries, even those apparently similar to one another. To provide concrete examples of how input metrics can be used effectively by leaders, here are examples of some of the analysis that a research library might want to undertake:
- If it is trying to reduce its emphasis on local tangible collections as a measure of success and focus instead on providing access to resources,
° Instead of measuring the size of a local collection, in terms of volumes, measure the amount of materials that your library makes readily available (including shared print resources, licensed digital collections, facilitated open access materials, and even DDA options)
° And, beyond just measuring the absolute amount of materials made readily available, measure its growth curve and/or its ratio against local collection volume count
° Finally, instead of measuring the amount of money spent on materials, measure the ratio of the amount spent on building collections locally against that spent providing access to materials from elsewhere or on a shared basis
- If a library is trying to transition to a greater emphasis on rare and distinctive materials,
° Instead of measuring the amount of money spent on materials broadly, measure the amount of money spent on rare and distinctive collections, and also the ratio of money spent on rare and distinctive collections against all collections and access
° More ambitiously, measure not only the acquisitions costs but include also employee time and direct expenses, including processing, description, access provision, storage, and preservation. Consider calculating storage fees so that moving collections offsite reduces their cost. Using this approach, calculate the amount spent on rare and distinctive collections vs all collections and access.
Similar examples can be offered for the transition away from general collections towards partnerships and services, and for other strategic priorities.
Inputs are sometimes used too simplistically, not least because the kinds of measures proposed above are difficult to calculate in the way that many budgets are constructed and many employees report their time (if indeed they do so). Some organizations use mechanisms like project codes to assign expenses to product lines, which in the case of the library could be categories such as rare and distinctive collections, general collections, and partnerships and services. While the cultural transformation needed to think in these terms might be substantial for many organizations, without these kind of input metrics it is virtually impossible to be accountable for executing on the strategy established for the library.
In the future to which we are transitioning, alignment with a parent university strongly suggests that value at the vast majority of libraries will not be measured principally by the size of the collection that is made available locally. Given the importance of this alignment, it is appealing to try to tie the library to university outcomes. While establishing this alignment is vital, university outcomes may not in every case offer the best definition of success against which the library should measure itself. Library outputs, and even the dreaded inputs, may in fact offer smart insight for library leaders to measure their organizational success, strategic development, and university alignment.
- Constance Malpas and Roger Schonfeld, “University Futures; Library Futures,” http://www.sr.ithaka.org/blog/university-futures-library-futures/.