Can Google Scholar survive the AI ​​revolution?

Can Google Scholar survive the AI ​​revolution?

Google Scholar, the largest and most comprehensive academic search engine, celebrates its 20th birthday this week. Over the past two decades, some researchers have noted that this tool has become one of the most important tools in science. However, in recent years, competitors have emerged that use artificial intelligence (AI) to improve the search experience, as well as others that allow users to download their data.

The impact of Google Scholar, run by Internet giant Google in Mountain View, California, is remarkable, says Jevin West, a computational social scientist at the University of Washington in Seattle who uses the database daily. But "if there was ever a moment where Google Scholar could be replaced as the primary search engine, it might be now, because of some of these new tools and the innovations that are happening elsewhere," West said.

Many of Google Scholar's advantages—the free access, the breadth of information, and the sophisticated search options—"are now shared by other platforms," ​​says Alberto Martín Martín, a bibliometrician at the University of Granada in Spain.

AI-powered chatbots such as ChatGPT and other tools that use large language models have become preferred applications for some researchers when searching, reviewing, and summarizing the literature. Some researchers have traded Google Scholar for these tools. “Until recently, Google Scholar was my default search engine,” says Aaron Tay, an academic librarian at Singapore Management University. It's still top of his list, but "lately I've started using other AI tools."

Still, given Google Scholar's size and how deeply entrenched it is in the scientific community, "it would take a lot of effort to dethrone it," West adds.

Anurag Acharya, co-founder of Google Scholar, welcomes all efforts to make scholarly information easier to find, understand, and build upon. “The more we can all do, the better it is for the advancement of science.”

The largest and most comprehensive

Google Scholar kicked in 2004 appear on the scene of literature research and changed everything. Back then, researchers used libraries to find information or searched for academic papers through paid online services such as the Web of Science citation database. The same month that Google Scholar launched, Elsevier also launched its paid service Scopus, a comprehensive database of scholarly references and abstracts.

Google Scholar searched the web for scholarly works of all kinds, such as book chapters, reports, preprints, and web documents—including those in languages ​​other than English. The goal was to “make the world’s researchers more effective and enable everyone to stand on a common frontier of science,” says Acharya.

Google Scholar's agreements with publishers give it unparalleled access to the full text of articles behind paywalls - not just the titles and abstracts that most search engines offer. The articles are ranked according to their relevance to a search query - usually the most cited articles are brought to the top - and further search queries are suggested. The depth of coverage allows for highly specific searches.

Google didn't disclose usage data for the service, but according to web traffic meter Similarweb, Google Scholar receives over 100 million visits per month.

The database is also very good at pointing users to free versions of an article, says Martín Martín. This encourages the open access movement, adds José Luis Ortega, a bibliometrician at the Institute for Advanced Social Studies of the Spanish National Research Council in Córdoba.

However, Google Scholar is opaque in other respects. A key concern is the lack of visibility into what content, including which journals, is being searched and what algorithm is being used to recommend articles. It also restricts mass downloads of its search results, which could be used for bibliometric analysis, among other things. “We don’t have much insight into one of the most valuable tools we have in science,” West says.

Acharya explains that Google Scholar is primarily a search tool and its main goal is to help scholars find the most useful research.

Updated search engines

In recent years, competitors offering such bibliometric data have emerged, although none can beat Google Scholar's size and access to full-text articles behind paywalls. A notable example is OpenAlex, launched in 2022. The year before, the Microsoft Academic Graph, which searched the Web for academic information, had been shut down and its entire dataset published. OpenAlex builds on this and other open sources of scientific data. Users can search the content that is cataloged by author, institution, and citation, and can also download the entire record for free. “They do what we hoped Google Scholar would do,” says Martín-Martín.

Another popular research tool, Semantic Scholar, launched in 2015 and uses AI to create readable summaries of papers and identify the most relevant citations. Another tool, Consensus, launched in 2022, uses Semantic Scholar's database to find answers to research-informed questions (West is a consultant for Consensus). One of Tay's favorites is Undermind, which uses sophisticated agent-based search in which an autonomous entity scans the scientific literature like a human and adjusts the search based on the content found. It takes a few minutes – compared to seconds on Google Scholar – to produce results, but Tay explains that it's worth the wait. “I think the quality of the results that come back is better than Google Scholar.”

Acharya says Google Scholar also uses AI to rank articles, suggest additional searches, and recommend related articles. And earlier this month, the company introduced AI-generated article summaries for its PDF reader. Acharya adds that the search tool attempts to understand the intent and context behind a query. This semantic search approach is based on language models and has been used for about two years, he says.

One thing Google Scholar isn't doing yet is incorporating AI-generated overviews of answers to a searched query, similar to those now found at the top of a typical Google search. Acharya says that it is challenging to summarize conclusions from multiple papers in a concise and context-rich way. “We have not yet seen an effective solution to this challenge,” he adds.