ChatGPT goes two: How the AI ​​chatbot changed scientists' lives

ChatGPT goes two: How the AI ​​chatbot changed scientists' lives

In the two years since Sharing ChatGPT to the public, researchers use it to communicate their improve scientific texts, review the scientific literature and Write code to analyze data. While some believe that the chatbot, which became popular on November 30, 2022, increases the productivity of scientists, others fear that he Plagiarism made easier, introduces inaccuracies in research articles and consumes large amounts of energy.

Hoboken, New Jersey-based publishing house Wiley surveyed 1,043 researchers in March and April about their use of generative AI tools like ChatGPT and shared the preliminary results with Nature. Eighty percent of respondents reported using ChatGPT either personally or professionally, making it the most widely used tool among academics. Three quarters of respondents believed that in the next 5 years it will be important for researchers to develop AI capabilities to do their work.

“AI word processors existed before, but a significant shift occurred with the release of these very powerful large language models,” explains James Zou, an AI researcher at Stanford University in California. The chatbot ChatGPT, developed by San Francisco-based technology firm OpenAI, was the catalyst for this change.

To mark ChatGPT's second birthday, Nature compiled usage data and spoke to scientists about how ChatGPT has changed the research landscape.

ChatGPT in numbers

• 60.000: Die Mindestanzahl an wissenschaftlichen Arbeiten, die 2023 veröffentlicht wurden und voraussichtlich mit Hilfe eines großen Sprachmodells (LLM) verfasst wurden 1. Dies entspricht etwas mehr als 1 % aller Artikel in der von dem Forschungsteam untersuchten Dimensions-Datenbank akademischer Veröffentlichungen.
• 10 %: Der Mindestanteil der Forschungspapiere, die von Mitgliedern der biomedizinischen Gemeinschaft in der ersten Jahreshälfte 2024 veröffentlicht wurden und voraussichtlich ihre Abstracts mit Unterstützung eines LLM verfasst haben 2. Eine andere Studie schätzte diesen Anteil für die Informatikgemeinde im Februar sogar auf höhere 17.5 % 3.
• 6.5–16.9 %: Der geschätzte Anteil der Peer-Reviews, die 2023 und 2024 an einer Auswahl von Top-AI-Konferenzen eingereicht wurden und vermutlich erheblich von LLMs generiert wurden 4. Diese Bewertungen beurteilen Forschungspapiere oder Präsentationen, die für die Konferenzen vorgeschlagen werden.

Writing assistant

All of these numbers, determined by evaluating patterns and keywords in texts that are characteristic of LLMs, are probably conservative estimates, says Debora Weber-Wulff, a computer scientist and plagiarism researcher at HTW Berlin. Their work shows that detection tools often fail when it comes to determining whether a paper was written with the assistance of AI 5.

Over the past two years, researchers have found that using ChatGPT to create abstracts, as well as grant applications and letters of support for students, allows them to focus on complex tasks. “The things that are worth our time are the hard questions and the creative hypotheses,” says Milton Pividori, a medical informaticist at the University of Colorado School of Medicine in Aurora.

Researchers report that LLMs are particularly helpful in overcoming language barriers. “It democratizes writing and helps people for whom English is a second language,” explains Gabe Gomes, a chemist at Carnegie Mellon University in Pittsburgh, Pennsylvania. An analysis published on the preprint server SSRN ahead of peer review in November found that the quality of writing in papers by authors whose first language is not English improved after ChatGPT's publication, more so than the writing of authors fluent in English 6.

Since its release in 2022, ChatGPT has undergone several upgrades. GPT-4, published in March 2023, impressed users with its ability to generate human-like texts. The latest model, o1, which was announced in September and available to some paying customers as well as certain developers in testing, OpenAI says it can “analyze complex tasks and solve more difficult problems than previous models in science, programming and mathematics.” Kyle Kabasares, a data scientist at the Bay Area Environmental Research Institute in Moffett Field, California, used o1 to to reproduce some code from his doctoral project. When he entered the information from the methods section of his research paper, the AI ​​system wrote code in just an hour that had taken him nearly a year of his graduate studies to create.

Limitations and potential

One area where ChatGPT and similar AI systems have been less successful is in conducting literature reviews, says Pividori. “They don't really help us be more productive,” he explains, because a researcher has to read and understand the relevant articles in full. “If the paper is not central to your research, you may be able to use AI tools to summarize it,” he adds. But LLMs have been shown to hallucinate 7 – that is, they make up information. For example, they could talk about numbers that don't even exist in an article.

Another concern for researchers when using LLMs is data protection. For example, when scientists input unpublished original data into one of these AI tools to write a paper, there is a risk that the content will be used to train updated versions of these models. “These are black boxes,” explains Weber-Wulff. “You have no idea what happens to the data you upload there.”

To avoid this risk, some researchers choose smaller, local models instead of ChatGPT. “You run it on your computer and nothing is shared externally,” Pividori says. He adds that certain ChatGPT subscription plans ensure that your data is not used to train the model.

A big question that researchers have been pursuing over the past year is whether ChatGPT will go beyond the role of a virtual assistant and an AI scientist can be. Some early efforts suggest that this is possible. Zou is leading the development of a virtual lab in which various LLMs take on the role of scientists on an interdisciplinary team while a human scientist provides high-level feedback. “They work together to formulate new research projects,” he says. Last month, Zou and his colleagues published the results of one of these projects on the preprint server bioRxiv ahead of peer review 8. The virtual lab designed nanobodies - a type of small antibody - capable of binding to variants of the SARS-CoV-2 coronavirus that caused the COVID-19 pandemic. Human researchers validated the work through experiments and identified two promising candidates for further study.

Gomes and his colleagues are also excited about the possibility of using ChatGPT in the lab. She use the tool to perform multiple chemical reactions using a robotic system they deployed late last year. “The expectation is that these models will be able to discover new science,” says Gomes.

1. Gray, A. Preprint on arXiv https://doi.org/10.48550/arXiv.2403.16887 (2024).

2. Kobak, D., González-Márquez, R., Horvát, E.-Á. & Lause, J. Preprint on arXiv https://doi.org/10.48550/arXiv.2406.07016 (2024).

3. Liang, W. et al. Preprint on arXiv https://doi.org/10.48550/arXiv.2404.01268 (2024).

4. Liang, W. et al. Preprint on arXiv https://doi.org/10.48550/arXiv.2403.07183 (2024).

5. Weber-Wulff, D. et al. Int. J. Educ. Integr. 19, 26 (2023).

   Article
   PubMed
   
   Google Scholar

6. Liang, Y., Yang, T. & Zhu, F. Preprint on SSRN https://doi.org/10.2139/ssrn.4992755 (2024).

7. Farquhar, S. et al. Nature 630, 625–630 (2024).

   Article
   PubMed
   
   Google Scholar

8. Swanson, K., Wu, W., Bulaong, N. L., Pak, J. E. & Zou, J. Preprint on bioRxiv https://doi.org/10.1101/2024.11.11.623004 (2024).

Download references