by Nancy K. Herther (writer, consultant and Sociology/Anthropology Librarian, University of Minnesota Libraries)
Today, Scopus is 15 years old and growing. However, the database remains a source of frustration to many, especially information professionals needing research-level citation analysis. 1Five years ago Against the Grain reported to readers on the newly announced archival expansion program that Elsevier was about to undertake to massively extend and expand coverage of this key database. As we noted at the time, “Many critics have questioned the lack of depth to the Scopus database, which has only gone back to 1996 with any certainty. Journal coverage has long been an issue for researchers, leading to a strong perception that the social sciences and arts and humanities are more ‘fringe’ elements of Scopus.”
Elsevier describes the now completed journal, book and conference backfile projects as a massive effort to deepen and expand Scopus coverage. Senior Product Manager for Content, Dr. Wim Meester, explained at that time that “We estimate that we will need to re-index 8 million documents in Scopus—that is on top of the 2.5 million documents that we add to Scopus each year. Some journals will be back filed even further than 1970; however, just going to 1970 for such a huge core of journals is a major advance.” The project was widely praised since Scopus was being used so broadly as a metric tool for researcher evaluations, computing H-index and other aspects of research productivity and value.
The book indexing project itself was expected to include “whatever metadata is available, including author information, affiliation and cited references. In order to do this, we need the full-text of the books in order to do the cited references, footnotes, or notes, embedded in the text,” Meester reported at the time.
The project was clearly a major program, made more complex by the need to re-index so many documents in Scopus in addition to all of the documents normally added to Scopus each year.
RESULTS OF THE PROJECT
The archival projects were completed in 2017 (details here). As Elsevier reports to Against the Grain: “The book expansion program was completed in 2015 (details here). We also conducted an additional expansion to cover more conference proceedings, which was also completed in 2015.
In sum, Elsevier cites the following data on the process:
- Total items indexed in Scopus = over 66 million,
- Number of serial titles (journals, trade journals and book series): 22,748,
- Number of books indexed: 140,000,
- Oldest item in Scopus dates back to 1823, and
- Cited references go back to 1970.
Elsevier describes the project’s impact in this way: “Over the past three years, Scopus has added over 195 million more cited references, dating back to 1970, to complement the database’s existing records that date back to 1788 and further increase the depth of content. Additionally, over 4.5 million articles have been reprocessed to index more cited references on Scopus, adding 7.5 million pre-1996 article abstracts for users to discover.”
In another company report Elsevier noted that “Over the past three years, Scopus content has significantly increased in depth. In addition to already including content records back to 1823, Scopus has added over 160 million cited references to its database, dating back to 1970. This allows you to construct long-term, extensive bibliometric and historic trend analyses, and has resulted in more complete author profiles and h-index measurements for individuals who began publishing prior to 1996.”
As Meester told Against the Grain in 2014, “Currently in Scopus there are no cited references for articles before 1996, and we will re-process everything in the database back to 1970 in order to add all of those cited references to the content in Scopus. By doing this, we want to be sure we have accurate citation information for all the content in Scopus and that all author profiles in Scopus have accurate information, that the H-index gives us what we would expect, given this enhanced database.” However, information professionals contacting me over the past five years have found the database still problematic in terms of comprehension.
THE DEVIL’S IN THE DETAILS
Information professionals are known for being concerned about quality control of their research products and Scopus is no exception. As I noted in the 2014 Against the Grain article: “Having this larger, more robust data set to work with will allow for more accurate assessments and trends over time. This will also add more comprehensiveness to the author profiles and H-index data.”
In the past few months, I’ve been asked for systematic review information or publishing options by our campus researchers. I always check the database journal lists to get a sense of depth of coverage and find glaring gaps, which are still common in Scopus and deserve recognition and attention:
- Annual Review of Anthropology (1972-) 1996-ongoing, 1994, 1990-1991, 1979-1988
- Annual Review of Applied Linguistics (1980-) 2005-ongoing
- Annual Review of Earth and Planetary Sciences (1973-) 1978-ongoing, 1976
- American Historical Review (1895-) 1990-ongoing, 1984-1987, 1981, 1977-1979, 1970-1974
- Journal of Global History (2006-) 2006-
- Journal of Global Marketing (1987-) 1987-
- Journal of Political Economy (1892-) 1987-ongoing, 1979-1985, 1973-1974, 1969
- Review of Economic Studies (1933) 1996-ongoing, 1983-1985, 1979, 1977, 1969
- American Journal of Political Science (1973), 1996-ongoing, 1986, 1982
Research databases are critical to all types of research institutions. Research itself is intended to verify or build on the existing research base. Whether for having the ability to quickly get a reliable picture of the existing research, data, assumptions and trends, or looking to evaluate the performance and strength of particular researchers or institutions, libraries and organizations rely on these databases. The cost of these products can only be supported by the depth and breadth of these databases. I have not tried to do an in depth analysis of the contents of the “new” Scopus; however, the continuing comments and questions that I have received over the past five years from Against the Grain readers led me to question the depth of the Scopus update. It has taken months to receive written responses from Elsevier and no interviews were ever arranged.
Elsevier continues to promote the value of Scopus as a key resource. “Today, Scopus now has over 5,000 customers globally, and our clients remain primarily academic and government institutions alongside a significant number of research-intensive corporations. Over the past years, Scopus has also increasingly been recognized for its data quality by government organizations, demonstrated by the fact that Scopus data feeds into 150+ national and regional agencies and institutes to set overall strategic direction, assess research productivity relative to other institutions, identify funding resources, enable collaboration and measure researcher performance. Elsevier is also a key partner for university ranking organizations, like Times Higher Education (THE), Quacquarelli Symonds (QS) and SCImago Journal Rank (SJR), who trust Scopus to provide reliable and comprehensive research performance data and analytics.
Further,” Elsevier continues in a written statement, “The company takes pride in their expanded business model to include more rights to data integration with university systems, particularly with enterprise university information systems, like CRIS (Current Research Information System) or RIM (Research Information Management), via our API’s. For subscribers, this integration is available at no additional cost. In the past three years Scopus has launched interoperable functionality with many new tools, both inside the Elsevier family and with other partners. Since acquiring tools like Mendeley and Plum Analytics, Elsevier has invested in integration across tools like Scopus, Pure, and more. With external partners, Scopus has launched integrations with discovery services, institutional repositories and enterprise RIM (Research Information Management) tools to support institutional needs. These integrations are often driven by user and customer demand and by our own research into ways to more effectively support the research workflow.
In the space of metrics, journal level metrics continue to be an important part of the basket of metrics, complementing new and alternative metrics to provide a multi-faceted view of a journal’s impact. On Scopus, you will find an evolving and expanding suite of journal metrics that go beyond just journals to include most serial titles, including supplements, special issues and conference proceedings. Freely available on Scopus you will find CiteScore metrics, SCImago Journal Rank (SJR) and Source Normalized Impact per Paper (SNIP).
In 2016 Elsevier launched CiteScore to help librarians and researchers track journal performance and make decisions. CiteScore metrics are a new standard to measure serial citation impact. Comprehensive, transparent, current and free, CiteScore metrics help you to analyze the impact of all serial titles – including journals – in Scopus (more info here).
Article-level metrics quantify the reach and impact of published research. Scopus now integrates data from PlumX Metrics as the primary source of its article-level metrics, along with traditional measures (such as citations), to present a richer and more comprehensive picture of an individual article’s impact.
Over the past five-years, Scopus has invested in increasing the discoverability of open access content. In 2018, Scopus partnered with CrossRef to retrieve open access information for ~2M records in Scopus. This currently includes ‘Gold’ OA, either in OA journals or hybrid journals. Users can now view open access articles which were previously indicated at the journal level (more information here). We then quickly also moved on to partner with ImpactStory which means that now Scopus users will be able to search over 7 million peer-reviewed articles tagged as OA in Scopus (more info here).”
Scopus has obvious size advantages; however, research and critics point out the duplication, data errors, missing or incomplete information and other issues with the system. Against the Grain asked some of the researchers who use and have studied Scopus for their perspectives on key issues Elsevier needs to address.
ELSEVIER RESPONDS DIRECTLY TO AGAINST THE GRAIN CONCERNS
Daniel Calto, Director of Product Management, Performance and Planning for Elsevier’s Academic & Government Products, responds to concerns brought by readers. “Scopus has very recently doubled its journal coverage of the arts and humanities, by adding more than 1,500 A&H (Arts & Humanities) journals. We worked in close cooperation with the European Science Foundation’s ERIH (European Reference Index for Humanities) on this project. As compared with years past, we have also added a substantial number of social science journals to Scopus content, and now cover over 18,000 journals globally.
Another significant effort that we have undertaken,” he continues, “is to better cover local language journals globally. This effort is important for the arts and humanities, the social sciences, and STM publications. Traditionally, indices have been slanted towards English-language journals published in established research countries, but over the past ten years, with the growth of emerging research powers like China, South Korea, and Brazil, this is something that is important to address for any reference A&I database.”
Calto recognizes concerns about the database and explains that “we do believe that the database is being continually improved, and are committed to doing so and continuing to invest in this important information resource. Ensuring that each title is covered comprehensively is an important part of this effort. The initial coverage of these subjects was based on the criteria of information that was readily available to us and relevance as determined by the Scopus Content Selection and Advisory Board (CSAB) which includes researchers from both social sciences and A&H. We have, however, been working to close any gaps and have made considerable progress on this front with the recent additions to the content.
The dynamic developments of today’s research funding climate are unprecedented in their scope and reach. The drivers for research performance assessment and funding include political, economic, and scientific factors. We think it is essential that researchers have access to practical tools like SciVal Funding (now integrated with Funding Institutional). As compared with existing products, we feel that SciVal Funding has two unique benefits. First, it has integrated award data that is linked to individual funding opportunities. This data allows researchers to better assess their chances and target proposals based on this additional information. Research administrators can use this data for a variety of analytical and assessment tasks as well. Second, SciVal Funding provides over five million U.S. research profiles that are pre-populated with Scopus publication information. By matching the profile data of available funding opportunities, SciVal Funding provides targeted recommendations on relevant grant opportunities.”
OPERATING IN A HIGHLY COMPETITIVE MARKETPLACE
“We believe that the marketplace is an open one for bringing out new and innovative solutions for academic and government research organizations,” Calto asserts. “It is a highly competitive marketplace, but that is advantageous for us as it is a driver of innovation across the board, and we welcome competition in this area.
Our product SciVal Spotlight does not use traditional citation analysis to evaluate researchers. It uses co-citation analysis, which allows the product to highlight an institution’s distinctive competencies in research. This method is well-established and can be reviewed in the scholarly literature for specifics on the methodology used. We feel that researcher and institutional evaluation using both qualitative and quantitative metrics is a very strong global trend, driven by limited research budgets as well as government and other funding agencies’ demands for transparency, and that this trend is unlikely to subside, but if anything will only intensify.
We did do beta testing with both of the products prior to the first release, and will continue to refine and further enhance the product with periodic product releases,” Calto reminds readers. “Institutions are free to use both products on a trial basis with no obligation to purchase them; we only ask that the institutions give us candid feedback about their experiences with the products. We envision that the primary customers for the products will be research-oriented or education-and-research-oriented institutions. The SciVal Funding product is being launched initially in the U.S., SciVal Spotlight globally. Cost for all Elsevier products is dependent on the institutional size and other market factors.”
In Part Two of this review of Elsevier’s efforts to enhance Scopus in the past five years, we look at some of the larger issues impacting the company – including perspectives from Kumsal Bayazit and talk with researchers and information professionals who deal with these databases daily and have worked out their own balance of systems to provide the best possible citation research today.
Nancy K. Herther is Sociology/Anthropology Librarian at the University of Minnesota Libraries, Twin Cities campus. [email protected]