Column Editors: John McDonald (EBSCO Information Services)
and Kathleen McEvoy (EBSCO Information Services)
When we began this column just over a year ago, we were interested in looking at analytics and how they were impacting, or would impact, the future of libraries. We looked at the information we had about user behavior and how people searched, the results of our user research studies and the information we gleaned from our search systems. We endeavored to be mindful of the issues inherent in analytics, including privacy.
In the past year we have showcased colleges and universities doing research related to their own users as they introduce new technology, publishers providing information to help libraries make decisions, how libraries and service providers balance the need for information with the need for privacy, consortial considerations around usage data for shared collections, how libraries use data and analytics to influence institutional decision makers and how libraries can build an analytics culture. In the future, we plan to address the ways additional librarians and providers approach analytics and the issues surrounding them with the goal of continuing to answer the questions, and highlight the issues surrounding the future of libraries and analytics.
We initially interviewed Rob O’Connell from Smith College Libraries. Smith College uses EBSCO Discovery Service and did extensive user testing to determine how to best present the now-un-siloed information the library provides to students and researchers through its discovery service. Common issues surrounding privacy prevailed, but librarians spun up a cross-departmental research team to bring together librarians and programmers to address assumptions and the status quo.
Using a Bento Box approach, the research team experimented with guerilla testing to reach students outside the library space and played with the language and terminology being displayed. Researchers found some surprises that challenged their assumptions about the information that should be presented, but with analytics to back up those findings, decisions were easier to make. Iteration was a common theme as the Smith College Library research team determined that whether addressing the language used, the databases being recognized (or not being recognized), or the optimal amount of content to be presented on the homepage, an iterative approach was determined to be better than continuing to force redesigns en masse. Another common theme was understanding the user. By introducing new ideas (e.g., searching by subject area rather than by database name) and by observing what did and didn’t work (e.g., drop down menus that users ignored), the research team was able to challenge its own assumptions, better understand user behavior and make decisions that were easy to test because they never represented a wholescale change but rather an iteration of the Bento Box presentation.
In April, Kristen Tepfer, Senior Marketing Manager from SAGE Publishing, wrote about the Customer Value Reports that SAGE creates for customers. These CVR’s present multiple metrics to customers so they can make decisions based on data including usage, turnaways and faculty reactions. The goal is to help libraries show the value of their investments with the publisher. One of the benefits to libraries is that these reports can be prepared for them and do not draw from library staff resources while providing a useful tool that librarians can use to indicate how users are engaging with their content and by showcasing areas of high usage and areas where usage may need encouragement. The reports also provide librarians with insight into their faculty members by showing faculty contributions to journals in library collections. While the CVR’s are an important offering to librarians looking to make collection decisions, SAGE also leverages the information for its own product management decision making as it prioritizes the enhancements the company makes to its platform and offerings.
In June, Neil Scully, the IT Director at Open Athens, wrote about Data, Privacy and the User Experience. This important topic hits a variety of concerns around privacy. Open Athens addressed these needs by introducing the “digital identity” — an “opaque ID tied to the user data in the institutional user directory.” The digital identity serves both the library and publishers. Libraries benefit by being able to authenticate users and show the number of students that are accessing course materials. The goal is to allow faculty members to make decisions about their teaching materials to increase usage of specialized content. Scully also says this information can help faculty to gain more insight into the use of library resources and incorporate them into course plans with the goal of improving student grades by providing more access to information aligned with course work — content from the library which does not require additional expenses. The usage information could also inform collection development decisions so libraries are investing in areas of greatest impact to faculty and students. The information gleaned from the digital identity could also help publishers better understand where users are and how they are using content. Since Open Athens is geared around authentication and single sign on, they are at a crossroads in terms of helping libraries make access to information more seamless to faculty and students, while also being able to provide anonymized information to librarians, faculty and publishers so they can leverage the data that usage creates to make decisions about collections, courses and content.
The September issue addressed Consortial Usage Statistics Analytics and the CC-Plus Project. Anne Osterman from VIVA, Jill Morris from PALCI and Jason Price from SCELC weighed in on the challenges consortia face when assessing the metrics and analytics for shared collections. Consortia members face the unique challenge of needing data relative to their shared collections as well as how to leverage that information to build their individual collections. The variety of consortial arrangements complicates these needs but Osterman, Morris and Price break it down into two areas: “the collection, storage and retrieval of usage statistics and resource evaluation and negotiation support.” These needs were addressed through the CC-Plus Project (Consortia Collaborating on a Platform for Library Usage Statistics). They also highlighted a data visualization project being developed alongside CC-Plus by the Virtual Library of Virginia or VIVA which aims to create an open source front end to the data from CC-Plus.
A survey that came out of a 2014 meeting of ICOLC (International Coalition of Library Consortia) surfaced areas that needed to be addressed including how to manage and use e-resources usage data with most respondents admitting to using manual processes and with only 20 percent of respondents making use of automated services such as SUSHI. Survey results also highlighted functionalities that were needed, including “streamline processing capabilities, the ability to combine key data points for improved contextual analysis, streamlined vendor password management, and visualization tools that would facilitate analysis.” CC-Plus was born out of these results to address the needs of consortia such as, “enabling high level, broad product comparison, while also being able to isolate and contextualize varied consortial collections.” With CC-Plus focusing on how data is harvested, processed and stored, the ability to analyze the data and visualize it was left to the open source front end from VIVA. The project addresses the needs of consortia while also allowing for the “individual institution data and the on-demand benchmarking” required of consortia members. The authors pared consortial usage data needs down to two goals: simplifying and automating the collection, storage and retrieval of usage statistics and evaluating ongoing subscriptions and supporting renewal negotiating.
In November, the column featured an article by the former Library Director at Chalmers University of Technology in Sweden, Daniel Forsman. Daniel discussed using data analytics for “strategic library transformation.” As the library moved toward digital resources, including the implementation of a discovery service, librarians began to realize that there were consequences inherent in the move to digital including changing user behaviors. “The value of library services eroded with our stakeholders and the library being marginalized by the decision makers at the University. The library had lost its place at the table.” A new mission was born, and data was at the center of this new electronic environment.
The data drawn from the library catalog, the institutional repository and the resources the library licenses, were leveraged to create new services and develop new competencies. Librarians joined with developers, designers and users to use the data as a tool to reinvent the library. By 2010, “the demand for scholarly metrics and analytics at the campus level became our first method of refining these data tools. The rise of bibliometrics as a measure of academic success and viability in the surrounding society was a result of non-academics or academic management attempting to simplify a complex system.” Chalmers librarians “embraced bibliometrics” in order to “safeguard academia from unfair methodologies and to make sure that bibliometrics were not used in an inappropriate way.” Regaining that seat at the table meant extracting “high-quality, timely, actionable insights for researchers, scholarly communicators, research funders” while leveraging publication patterns and usage patterns. Librarians pitched themselves as experts “on how to increase the visibility and the impact of research. By working with the data, understanding the data, and having the confidence to analyze and present conclusions of the data, the library becomes a strategic partner for stakeholders to make better, informed decisions.”
The final 2018 column focused on how to build a culture where analytics can thrive. Tabatha Farney, the Director of Web Services and Emerging Technology at the Kraemer Family Library at the University of Colorado, Colorado Springs wrote about the culture of analytics as “an organizational commitment to collect and analyze data to make decisions.” Breaking down the silos where data lives is an essential part of creating a system that can leverage data easily. Communications between teams is also essential to break down existing silos that can prevent the conclusions that can be drawn when there is a level playing field and insight into all the data that can impact an institution. “An analytics culture breaks down those data silos or, at least, enables the data to flow out of those silos to the individuals that need to analyze the data.” Farney says library administrators are essential to a successful analytics culture but the entire organization needs to support this new culture as well. One important part is not to overlook how data can be used by different parts of the organization. “Desk transactions can be used to measure the activity of the reference desk, but that same information can be used by administration to determine staffing based on the amount of interactions at specific time periods. That data can also be used to identify problems or gaps in services.”
Creating an analytics culture can start small. By connecting people with the data and by understanding who needs access to the data for analysis and who would benefit from the reports that analysis can generate, that culture grows. As Tabetha says, “You will know you are in an analytics environment when it feels uncomfortable making decisions without data.”
Common themes throughout the 2018 columns included privacy, understanding the user experiences and using an iterative approach; but, the most prevalent theme that emerged is that analytics have become essential and can change the perception of librarians within the library and the library within the institution.
Some issues that remain important to libraries include collection usage, space utilization, information literacy, research indicators and ROI. Librarians remain interested in the information being collected and analyzed so libraries can quantify the value of libraries and to showcase the value they bring and the contributions they make to the institution’s mission. Issues that remain top of mind include student retention and learning outcomes, faculty research productivity and the connection between data, usage, analysis and assessment.
Looking forward to 2019, our goal is to focus on analytics related to existing areas that continue to dominate discussions as well as new and emerging needs. Some of these topics fall into the categories of collection development and how analytics can help with decision making as institutions expand into new program areas, the analytics that surround ILS migration and implementation, open access and the findability of content and standards development and privacy. Please let us know if you have ideas surrounding data analytics and want to contribute to the discussion.