v32#1 Using Data to Rightsize Approval Spending

by | Apr 1, 2020 | 0 comments

by Jennifer A. Mezick  (Collection Strategist, University of Tennessee, Knoxville Libraries) 

and Louis T. Becker  (Assessment Programs Librarian, University of Tennessee, Knoxville Libraries) 

Book approval plans are intended to save the time of selectors by providing titles that meet pre-defined criteria.  Unintentionally, approval plans can be a source of seemingly uncontrollable overspending. This was the case at the University of Tennessee, Knoxville (UTK) Libraries.  What follows is a summary of how the UTK Libraries applied data towards rightsizing approval plan expenditures while still leveraging the ability to acquire books appropriate for our users. 

UTK is Tennessee’s flagship research institution.  GOBI (EBSCO) is the vendor for the Libraries’ approval plan.  Approval plan spending accounts for about 73% of books purchased.  Subject librarians maintain the approval plan criteria for call number ranges related to their subject areas.  Decades ago, approval plans were mediated: books sent on approval were held for review by librarians who had the opportunity to return books they did not want for the collection.  Today, UTK’s subject librarians lack the time to review titles and the Libraries lacks the space to hold books for review.  Additionally, to save staff time, titles sent on approval to UTK have barcodes and other physical processing applied before shipment, which makes them nonreturnable.  eBooks acquired on approval present an additional upset to the traditional review in that they cannot sit on a physical shelf.

Since the early 1990’s, approval plan allocations at UTK were sometimes targeted to help pay for other bills.  Even when money was not taken away from approval plan allocations, the titles received would sometimes total more than the amount budgeted.  Overspending the approvals budget became a consistent problem while the UTK Libraries experienced less flexibility in spending overall due to annual increases for subscription resources.  In reality, funds are actual money and not abstract numbers, spending more on approvals than is intended means spending less on other things.  To temporarily stop overspending, the Libraries would put the approval plan on virtual or on hold. Putting the plan on virtual means that the subject librarians receive slips (or electronic notifications) for all titles that would have been sent on approval.  Librarians would review these slips and place orders for the titles they wanted in the new fiscal year. Putting the plan on hold means that all titles are held until the new fiscal year when they are then shipped and billed. Neither of these alternatives resolved the ongoing issue that the Libraries received more content than was budgeted for.  The former depends upon librarians dedicating more time to reviewing slips and the latter perpetuates overspending in the next fiscal year. 

In August 2018, the UTK Libraries underwent a reorganization that resulted in assessment and collections initiatives coming together as one department named Assessment Programs and Collection Strategy (APCS).  This change placed an emphasis on making evidence-based decisions for collection management. At the same time, management of the Libraries’ collection budget moved to the Libraries’ Business Services Office.  This change unburdened subject librarians from managing funds and shifted the large, complex budget to the staff with the experience and knowledge needed to manage it well. Together with other organizational changes within the subject librarian department, the Libraries was set up to make collections decisions in a more collaborative manner. 

In project management there is the concept called triple constraint, in which three variables (cost, time, and scope) all affect quality with the understanding that it is possible to satisfy two of those variables but never all three because of conflicting demands (Wyngaard, Pretorius, & Pretorius, 2012).  Applying this theory to purchasing monographs, libraries can incur a lower cost while meeting the needs of users but at the expense of employee time; meet all the needs of our users while devoting little employee time but at a high cost for materials; at a low cost and while devoting little employee time but not necessarily meeting all the needs of our users.  GOBI employs staff to profile books and determine if titles meet approval plan criteria.  In theory, this helps UTK Libraries develop a collection that will meet our users’ needs efficiently by combining criteria decided upon by subject librarians with the security of knowing that a person, and not a computer, is profiling books.  At the UTK Libraries, it was apparent that something needed to shift to make the approval plan work better.  We wondered if we truly needed to increase the amount allocated for approvals or if we needed to tighten the criteria.  As the flagship university in the state, tightening the criteria implies that we will purchase fewer books, which may lead to questions about just-in-case versus just-in-time collection development and thus our role in the state higher education system.  Luckily, APCS could provide data showing how books acquired on approval are used as a measure to help subject librarians understand what is being used and adjust the criteria to, hopefully, provide us with more of the titles our users want and less of the titles our users will not use.  We could call this just-in-case informed by data.

Towards the end of fiscal year 2019, APCS looked at books purchased as part of the approval plan for four fiscal years, from July 2015 through May 2019.  (June 2019 could not be incorporated because the plan was on virtual.) Approval plan purchases during the period covered 38,291 titles, 25% (9,636) of which were in electronic format.  The Libraries has generally been shifting toward e-preferred purchasing, and in fiscal year 2019, eBooks accounted for 31% of approval plan titles (2,924 of 9,209 total). As part of this review, more approval plan criteria shifted to e-preferred from print and we expect the number of eBooks received on approval to increase in future years.  The print monograph collection included 716,159 titles at the conclusion of the study period, and total external print circulation was 193,407 loans over the four fiscal years. The approval plan purchases accounted for 9,271 of those loans (4.8%). The use of circulation data to evaluate approval plans is not new; Marcie Kingsley surveyed the basic technique in this magazine in 1996.  By fitting our data tightly to the form of our particular plan, however, we provided subject librarians information in a decision-ready form.

We used Microsoft’s PowerBI software to combine data from our integrated library system, Ex Libris’ Alma, with COUNTER reports of eBook usage and GOBI’s tables of Library of Congress (LC) classifications and imprints.  Rather than a single spreadsheet, the resulting interactive report allowed APCS and subject librarians to visualize the usage of approval plan purchases according to the categories GOBI considers.  The 4,099 LC class ranges and sub-ranges in our GOBI plan were used to group the purchased titles.  Summary information about the print monograph collection and its circulation over the study period were divided into the same ranges.  Because GOBI accepts instructions regarding specific publishers within a subject range, we wanted to review our purchases by imprint (Figure 1).  This required significant data-cleaning work. 2,159 individual text strings from the publisher fields (MARC fields 260b or 264b) of the bibliographic records were manually mapped to the GOBI publisher list.  After this cleanup we could see that purchases came from 442 GOBI publishers, plus 40 imprints that could not be decisively mapped to the GOBI list.  

Figure 1.  Example image from PowerBI visualization showing the overall number of Routledge titles purchased during the study period and organized by LC classification, year, and format.  Graphs are highlighted to show the proportion of Routledge titles in each category.

To allow comparison between electronic and paper usage, we focused on the percentage of titles ever used; for this metric a chapter use was equally valid as a title download or a print loan.  Because of changes in workflows for counting in-house use in Alma across the period under study, we considered only external loans as uses for most calculations. However, in-house use information remained viewable in the report.  While circulation data did not extend far past the date of the most recent purchases, a “sliding scale” filter was provided to optionally exclude approval books by purchase date. Viewers could set a minimum age that would be appropriate for their subject area or ignore older approval purchases if they had reconfigured their approval plan more recently.  While newer e-titles were more likely to have been used than equivalently recently purchased print titles, overall ever-used titles made up similar percentages of both the electronic and print approval purchases: 25.3% of approval eBooks vs. 21.3% of approval print titles had been used.

To correct for variations in the circulation patterns of different subjects, a use factor (Bonn, 1974) was calculated for each classification with approval titles.  Because of technical difficulties in linking eBook usage reports with classification data at the collection scale, this was done only for print books. A use factor larger than 1 indicates that the approval set accounted for a higher percentage of circulations in the class than would be expected for its share of the collection, while a use factor lower than 1 indicated a lower-than-expected share of loans.  Overall, the use factor for approval books was 1.4, indicating that recent approval purchases were circulating slightly more than the collection as a whole. For individual GOBI ranges, the use factor for approval books ranged from a high of 131, for one circulated approval book in a Forestry subsection with very low circulation overall, down to zero for sections where approval books did not circulate.

In calculating circulation totals for the print collection as a whole, the collection was divided only by classification.  Future analyses should also provide comparisons between books purchased by the approval plan and recent individual selections, to correct for variations in the circulation of books of various ages.  As the collection becomes more electronic, additional work will be needed to consider the eBook collection as a whole and categorize its usage — usually reported in the context of a particular vendor platform — in ways more directly comparable to the print collection. 

The PowerBI interactive report allows subject librarians to easily view these metrics for approval plan purchases from any subject classification or GOBI-assigned fund code, and then view the individual titles purchased from that segment (Figure 2).  Data on the approval plan purchases can also be viewed and filtered by publisher name. Due to the wide variations in imprint cataloging, data on the collection as a whole cannot be filtered by publisher.  Collection evaluation projects would be easier and more comprehensive if imprint data were assigned according to a controlled vocabulary like those used for author name headings.  

Figure 2.  Image from PowerBI visualization showing use of titles classified in Library of Congress’s GT area limited to purchase date between 10/9/2015 and 9/27/2018.

We decided that narrowing the criteria to stay within our current allocated amount for approvals was the appropriate goal.  While the overall use factor demonstrates reasonable circulation of approval books compared to the overall collection, the trend in overall circulation of the collection shows an annual decline, informing us that we should use our limited funds for other types of resources.  APCS arranged for GOBI’s Collection Development Manager to review approval plan areas with subject librarians.  In preparation, we provided a hands-on tutorial for using the data presented in PowerBI.  

Subject librarians held a roundtable discussion to share information about how to adjust approval plan criteria to influence what is received within the individual GOBI class ranges.  Beyond publishers, the plan can exclude or include only certain geographic areas, topics, content levels, languages, and formats.  While these categories were not built into the PowerBI report, librarians used the title-level list of purchases within the report to consider these changes.  APCS annotated approval plan parameters for subject librarians that provided brief information pulled from the PowerBI data about the use of items by area (Figure 3).  During review meetings, at least one collections librarian participated to provide supporting data from the PowerBI report. Some subject librarians changed little in their areas, but many made significant changes.  The data provided was an important factor in initiating those changes but other factors, such as new subject librarians and evolving research agendas in academic departments, were also considered.

Figure 3.  Example of annotated approval plan illustrating summarized use information from PowerBI report.

After final changes to the plan were submitted, GOBI provided a retroactive report that listed the titles that would have arrived on approval in the previous year, if the updated criteria were in place then.  The total cost for those titles is just below the amount allocated for approvals during the current fiscal year. 

APCS does not wish to inflict an intensive review of the approval plan on subject librarians annually, but we do plan to monitor spending and refresh our data over the next few years in hopes of seeing an increase in circulation due to more appropriate titles being received.  We plan to perform a thorough review of the approval plan again in about three years unless overspending becomes an issue before then. We also hope that continuing to monitor the approval plan using the metrics described above will, over time, provide a better understanding of the usage trends in each subject area.  Making decisions about acquiring and withdrawing monographs is much more difficult in subjects where books may not be used until five or more years after they have been published. Perhaps, in future years, our refreshed data report and other historical circulation data can be used to guide decisions with an increased level of comfort. 

References

Bonn, G. S.  1974.  Evaluation of the collection.  Library Trends, 22: 265-304.

Kingsley, M.  (1996, September).  Circulation statistics for measuring approval plan effectiveness.  Against the Grain, 8(4), 2, 16-17.  doi: https://doi.org/10.7771/2380-176X.2054

Van Wyngaard, C. J., Pretorius, J. H. C., Pretorius L.  (2012).  Theory of the triple constrain — A conceptual review.  2012 IEEE International Conference on Industrial Engineering and Engineering Managementdoi: https://doi.org/10.1109/IEEM.2012.6838095  

Sign-up Today!

Join our mailing list to receive free daily updates.

You have Successfully Subscribed!