Exclusive: The studies that most often reference retracted research papers

A review article was published in January ¹on possibilities for detecting human diseases by examining the eye in a conference proceedings of the Institute of Electrical and Electronics Engineers (IEEE) in New York City. But neither the authors nor the editors realized that 60% of the paper he cited had already been retracted.

The case is one of the most extreme to date a huge project to identify papers whose results may be questionable, were discovered because they cite retracted or problematic research. The creator of the project, the computer scientist Guillaume Cabanac from the University of Toulouse in France, shared his details with the news team atNature, which she analyzed to find the papers that cite the most retracted papers but were not themselves retracted (see “Retracted References”).

"We don't accuse anyone of doing anything wrong. We just observe that in some bibliographies the references have been retracted or retracted, which means the paper could be dubious," says Cabanac. He calls his tool a Feet of Clay Detector, based on an analogy originally from the Bible that refers to statues or buildings that collapse due to their weak clay foundations.

The IEEE paper is ranked second in the byNaturecompiled list, with 18 of the 30 cited studies withdrawn. The authors did not respond to requests for comment, but IEEE integrity director Luigi Longobardi says the publisher was unaware of the problem untilNatureasked and that he was investigating it.

Cabanac, a research integrity detective, has already developed software to scan thousands of problematic papers in the literature for issues like computer-written texts or hidden plagiarism to point out. He hopes that his latest detector, which he has been developing over the past two years and which he released this week in a Comment inNature describes, offers another way to prevent bad research from spreading into the scientific literature - including some fake works from “papermill” companies.

Further investigations

Cabanac lists the detector's results on its website, but elsewhere online – on the paper review website PubPeer and on social media – he explicitly flagged over 1,700 papers that caught the eye because they relied heavily on retracted work. Some authors have thanked Cabanac for alerting them to problems in their references. Others argue that it is unfair to effectively cast suspicion on their work simply because of post-publication retractions that they believe do not affect their papers.

Retracted references do not clearly indicate that a paper is problematic, notes Tamara Welschot, part of the research integrity team at Springer Nature in Dordrecht, the Netherlands, but they are a useful sign that a paper could benefit from further review. (Nature's news team is independent of its publisher, Springer Nature.)

Some researchers believe that retracting references in a narrative review – which describes the state of research in a field – does not necessarily invalidate the original paper. But when studies assessed by a systematic review or meta-analysis are retracted, the result of that review should always be recalculated to bring the literature up to date, explains epidemiologist Isabelle Boutron of Paris City University.

Detecting fraudsters

Some of the papers that cite a high proportion of retracted papers are from well-known academic fraudsters who have retracted many of their own papers. These include engineer Ali Nazari, who was fired from Swinburne University of Technology in Melbourne, Australia, in 2019 after a university fraud investigation examined his activities. He previously worked at the Islamic Azad University in Saveh, Iran and his current whereabouts are unknown. AfterNaturehad told publishers about his remaining papers ^{2, 3}, including Elsevier and Fap-Unifesp, a nonprofit foundation that supports the Federal University of São Paulo in Brazil, said they were reviewing the articles. One of the relevant journals was discontinued in 2013, Elsevier noted.

Cabanac's detector also marks papers ⁴by Chen-Yuan Chen, a computer scientist who worked at the National University of Education in Pingtung, Taiwan, until 2014. He was behind a syndicate running fake peer reviews and increasing citations that came to light in 2014 after an investigation by publisher SAGE. A