by Shannon L. Farrell (Natural Resources Librarian, University of Minnesota)
and Kristen L. Mastel (Outreach and Instruction Librarian, University of Minnesota)
This issue of Against the Grain focuses on the various kinds of assessment that occur in libraries. Assessment is incorporated across all aspects of library programming and services, from instruction to outreach to space planning to collections to user experience. When we started soliciting articles, we did it with the intent of covering a wide variety of types of assessment in libraries.
Our experience and expertise with assessment has focused on our activities and programs around outreach. Over the past few years, we have undertaken numerous projects on the topic of outreach assessment. We became interested in this work as outreach activities have traditionally not been formally assessed, or if they were, they focused on simple quantitative measures like head counts. In our work, we hope to persuade librarians to learn more about various assessment measures and to establish goals that are tied to assessment for every outreach activity that they engage in.
Over the past year, we worked with our colleagues Stephanie Graves and Sarah LeMire from Texas A&M on the Association of Research Libraries (ARL) Spec Kit 361.1 Through surveying a range of ARL institutions, we gathered information on the current landscape of outreach staffing, responsibilities, funding, and methods of assessment. One of the core findings was that there is no agreed upon definition of outreach; as a result, we suggest that each institution needs to define what outreach encompasses for their community. This study also reinforced what we found in the literature,2 that outreach is often not planned programmatically or incorporated into institutional assessment efforts.
A resource that we have recently worked on to assist in developing institutional outreach and engagement assessment for academic libraries is the ACRL Libraries Transform Toolkit.3 In this Toolkit, there is helpful documentation to match outreach programming to institutional goals, assessment techniques, and institutional reporting. Example “Because” statements highlight unique services and resources that libraries provide and can aid libraries in their promotional activities by providing simple, eye-catching designs for ready distribution. The Toolkit is recruiting for additional content and case studies, so please submit relevant institutional examples that you may have.
As noted, libraries are conducting all kinds of assessment around different programs and services. Many assessments that libraries collect are driven by reporting to larger institutional bodies, such as ARL, state library services, and/or accreditation bodies. However, the data solicited is frequently just quantitative and does not tell the full story of what libraries are doing. There have been recent efforts to aid libraries in demonstrating their impact, such as Project Outcome,4 developed by the Public Library Association. Tools such as this provide some questions to use as a starting point for assessment, which can be especially useful when many staff do not have formal assessment training or expertise. We solicited articles to provide a balance of what assessment can look like in various roles and services within libraries.
As libraries push for more rigorous assessment beyond just quantitative measures, we are seeing a surge in job announcements for dedicated assessment positions and conferences focused on improving the library’s impact and story. Craig Smith, Assessment Specialist at the University of Michigan, is an example of how libraries are recruiting staff with functional expertise to fill specific roles. For instance, Smith’s background in psychology and institutional research equipped them with a unique perspective that allowed them to tackle institutional assessment challenges. Smith suggests several areas of improvement for libraries when developing assessments: perspective taking, asking questions with the right focus, avoiding common problems with questions, getting meaningful responses, seeking diverse informants, being careful with claims, using assessment strategically, and seeking and providing critical feedback. Through these suggestions, libraries are able to improve their assessment programs and services.
The importance of asking the right questions ties directly into the quality of the feedback we receive, and if we can measure the impact outlined in our goals. As the Head of Outreach and Communications, John Jackson, at Loyola Marymount University, shares assessment strategies to use in the areas of programming and communications. Jackson provides a simple and streamlined three-question feedback form that assesses not only why people came to the program and what they learned, but provides room for their thoughts and reflections. Assessment is taken one step further, by also using a rubric to evaluate the feedback and measure if the program “transformed” the attendees’ perceptions. While measuring engagement of communications can be more straight-forward through clicks and URL tracking, it can be time-consuming. However, tracking can be used to justify spending more time and energy on social media and outreach efforts. Since libraries are multifaceted, Jackson states that the ultimate assessment will be combining “multiple data points into a single indicator of success, similar to the Happiness Index or a Klout score.”
To understand the impact of information literacy instruction, the Project Information Literacy Research Team (PIL) of Alison J. Head (visiting scholar at Harvard University), Aliana C. Bill (University of Washington Tacoma), and Margy MacMillian (Mount Royal University) classified library instruction assessment strategies into four levels (e.g., micro, meso, macro, and mega) based on scope and level of collaboration across institutions. Using this classification, the PIL team has conducted ten major research projects that involve surveys, interviews, and observations, in an effort to “examine what students actually do, rather than what we think they should do.” These assessments go beyond simple measurements of acquired skills. By gathering more data and asking well-crafted questions about student learning, we can better understand the impact of library instruction.
Libraries have been collecting simple statistics on their collections for many years (e.g., how many times books have been checked out), but collections assessment can be more effective and can address large-scale issues when it is aligned with library initiatives. Julie Rashid, Manager of Acquisitions and Rapid Cataloging at the University of Minnesota (UMN), describes the initiatives that UMN Libraries undertook to assess their print acquisitions. These initiatives were established to navigate staff through a period of uncertainty, including shifting priorities and rapid advances in technology. The assessment served to address not only external drivers, such as budget issues, but also internal processes, such as the acquisitions department’s productivity and efficiency in offering services. Sunshine Carter, Electronic Resources Librarian, and Yumiko Toyota-Kindler, Library Program Specialist 1 in the E-Resource Management Unit, also at UMN, describe the process they undertook to assess post-cancellation access rights for the UMN Libraries e-journal collection, with a goal to identify which print serials could be withdrawn. Identifying precisely what an institution owns or has access to with regard to electronic purchases or subscriptions is paramount when deciding what print journals to discard. In their article, they provide recommendations on how other institutions could enact a similar process. Both of these articles highlight the importance of assessing both print and electronic collections as libraries face challenges in terms of space constraints, limited budgets, and ever-changing title ownership and publishing terms.
Emma Molls, Publishing Services Librarian at University of Minnesota, discusses how to assess a brand new library-based publishing program at UMN. The goal of the program is to provide an affordable alternative for publishing open access journals. However, the typical metrics used by publishers for assessing journals, such as capturing sales figures, are not appropriate in this case, as UMN Publishing Services are not a for-profit publisher. Further, they are utilizing a variety of platforms, rather than a single one, which is atypical of journal publishers. Molls’ article discusses how to utilize an assortment of assessment measurements to address a wide variety of situations and how to connect the assessment back to unique goals and objectives by gathering feedback from content providers.
While we have looked at how assessment can impact libraries’ programs and services for improvement and patron learning, we have yet to touch on how assessment can move our profession forward. Elliot Felix and the LibGoal project team created a tool to aid library staff in first identifying and assessing their skills, and then aligning them to both their personal priorities and the goals of their institution. Through a card sorting exercise, participants are asked to reflect on the soft and hard skills needed in various library roles. This type of activity can help libraries create “T-shaped library professionals with vertical depth of expertise as well as horizontal skills to enable collaboration and impact.” In order to stay relevant and nimble to our communities’ needs, we need staff that can easily adapt across roles and responsibilities.
As you will read in this special issue around assessment in libraries, there are efforts at the individual, institutional and professional levels to improve the demonstration of our impact and telling of our stories. Purposeful assessment, while it takes time and thoughtfulness, can help us address long-standing issues, answer difficult questions, and promote change in meaningful ways. Do not be intimidated; the only way we all get better is by learning through action! Over time, purposeful evaluation will allow us to investigate even more complex questions and push ourselves organizationally to confront areas we have yet to begin addressing in assessment, such as power, privilege, and equity.
We would like to thank Katina Strauch and Tom Gilson for the invitation to guest edit this issue on assessment and their support in revising solicited articles. In addition, we are grateful to our many contributors. We are elated that our paths have crossed with so many talented people willing to share their stories and we hope they inspire you to expand and evolve your assessment practices.
1. LeMire, Sarah, Stephanie J. Graves, Shannon L. Farrell, and Kristen L. Mastel. Outreach and Engagement. SPEC Kit 361. Washington, DC: Association of Research Libraries, November 2018.
2. Diaz, Stephanie A. “Outreach in academic librarianship: A concept analysis and definition.” The Journal of Academic Librarianship 45, no. 3 (2019): 184-194.
3. Libguides: ACRL Libraries Transform Toolkit: Introduction. Accessed June 4, 2019. https://acrl.libguides.com/transform.
4. Public Library Association. Project Outcome. Accessed June 4, 2019. https://www.projectoutcome.org.