Sleuth’s Work Has Led to the ID of Hundreds of Fraudulent Papers
Last month, Retraction Watch reported on the case of Hironobo Ueshima, an anesthesiology researcher found guilty of misconduct in more than 140 papers. A journal editor, John Loadsman, was the first to suspect there were issues in Ueshima’s work. But this was hardly the first time Loadsman had been the canary in the coal mine of the anesthesiology literature, placing him squarely on our ever-growing list of scientific sleuths that includes Elisabeth Bik, who has been in the news recently because of threats. In this Q&A, we ask Loadsman what happened in the Ueshima case, and for his sense of how big a problem fraud in the literature is.
Retraction Watch (RW): You were the editor who first spotted problems with Ueshima’s work. Walk us through how you cracked the case. What made you suspect that his data were not kosher?
John Loadsman (JL): The mechanism was briefly mentioned in your first post about this case a year ago, and also discussed in the introductory matter of the investigation report. Summary data in the paper submitted to Anaesthesia and Intensive Care had features very inconsistent with having been drawn from a random sample. The raw data were requested and promptly provided by the author, and those data files displayed incontrovertible evidence of a problem with veracity. The submission to our journal was therefore rejected with a detailed explanation and with the relevant aspects of the data files highlighted.
The rejection email was copied to Showa University, the Japanese Society of Anesthesiologists and the UMIN (Japanese clinical trial registry) and it included a request for an institutional investigation. Dr Ueshima replied the next day to inform me his research was indeed going to be investigated and this was subsequently confirmed by the Japanese Society. In the meantime I also contacted the editors of several other anaesthesia journals that had published Ueshima’s work. Two responded to say they concurrently had submissions either under consideration or accepted but not yet published.
One of those was to the journal Anaesthesia for which John Carlisle is an editor, and it had been submitted only days after the one to our journal. The raw data for that submission had been requested as part of their routine policy, I think, and I have little doubt John Carlisle would have independently drawn the same conclusions as I did within a few days. John’s further analysis of both papers confirmed what we already knew. As you have already reported, Dr Ueshima publicly apologised and several of his papers were soon retracted.
RW: Did you face any personal risks, or threats?
JL: No, not that I am aware of, not so far anyway. On the contrary, in fact. Dr Ueshima himself was very polite in our brief correspondence, despite the gravity of the situation. The various Japanese authorities, while obviously troubled by it all, have nevertheless been grateful and responsive in our communication over the course of the last year or so.
RW: What’s your opinion of how the journals have handled the affair?
JL: The journals (and I guess by that you mean the editors and the publishers) haven’t really had time to handle the affair yet, at least in terms of responding to the outcome of the investigation. I guess we will get a better idea about this over the coming weeks to months (hopefully not years).
A more pertinent question would perhaps have been about “how the institutions have handled the affair” and in this case the answer is superbly. In what is probably a record of some sort, it was a week between the submission of the index paper to our journal and being informed that Showa University and the Japanese Society of Anaesthesiologists had commenced an investigation.
Contrast that to other cases. That investigation has taken a year, but on the background of COVID and considering the thoroughness of their enquiry and the openness of their report I think that’s very understandable and actually quite remarkable. They are to be congratulated for their exemplary response, I reckon.
RW: Several of the biggest scandals — by retractions, at least — in science have involved anesthesiologists: Scott Reuben, Ueshima, Joachim Boldt and Yoshitaka Fujii. Together, these four researchers account for hundreds of retracted papers. That’s a lot! Do these bad apples reflect a rot in the field of anesthesiology generally? Is anesthesiology breeding fraudsters? Or, are journal editors in anesthesiology doing a better job than their peers in other fields of policing the literature?
JL: You’ve forgotten about Yuhji Saitoh who has another 50-something retractions. Yes, it’s a lot. And no, this does not “reflect a rot [specifically] in the field of anaesthesiology.” Any answer to your question would amount to speculation.
Anaesthesiology could have a particular problem with fraud, as you imply and as I have seen many suggest on social media and elsewhere recently, or maybe our specialty is better at finding it as you also wonder. It could be a combination of those, and there are other possible factors you haven’t mentioned. Fraudsters in our field may be particularly bad at covering their tracks for some reason. Researchers in anaesthesiology may be more willing to step forward with concerns about their peers, and our readers may be more observant and critical than those in other spheres.
I have a personal bias, of course, but I am inclined to think some of our editors do have a particularly strong awareness about and approach to this problem, and there is a lot of cooperation between the editors of many of the journals in our field which I think is rather unique and which has proved to be critical in some cases. There is also some statistical evidence thanks to John Carlisle’s mega-study, about which you reported and which resulted in the retraction-replacement of the PREDIMED trial, that problematic research may be as prevalent in general medicine as it is in anaesthesiology.
For the moment at least, it is not possible to reliably answer your question, but I would urge the relevant people in other biomedical specialties particularly to look a lot harder because I suspect similarly large cases exist outside our field.
RW: Given that the misconduct cases we do see represent the tip of an iceberg, how big is that berg?
JL: Rather than ice, which can be treacherous but is otherwise largely pure and useful, perhaps a better metaphor would be a fatberg floating in a sewer. The size of the problem, the rotting fat under the surface so to speak, would, again, be a matter of speculation. You only have to follow the work of people like Elisabeth Bik to know it’s big, much bigger than most people think.
The problem is clearly worse in places where people’s livelihoods and academic careers depend on sustained “research output” as measured by publication. And in addition to the so-far-unidentified published material that is affected by misconduct, very few people have any idea of the much, much larger subset of papers that have never seen the light of day because journal editors and peer reviewers have stopped them. Retraction Watch has, on several occasions including very recently, shared commentary that has been harshly critical of the editorial and peer review process, but much of the relevant “defensive” work is invisible and conducted by editors and peer reviewers, or at least some of them.
Another related issue now, however, is the ease with which such problematic papers are published as preprints, and some preprint servers are now indexed. Though there are those who claim the fraud problem is being solved by preprint publication, the opposite is probably much closer to the truth. 10,000 watts of light might be getting shone on a few high-profile preprints, but orders of magnitude more of them sit in a dark corner on those servers, occasionally to be included in meta-analyses. And, of course, all this is yet again eclipsed by what’s going on with predatory journals.
RW: What can editors, peer reviewers and others involved in scientific publishing be doing better to keep tainted papers out of print in the first place?
JL: I think it mainly amounts to having a very high index of suspicion, and that is not just for the people involved in the assessment and dissemination of research but also the recipients of it. This is obviously not what science should be about, and it’s a great pity for so many reasons, but my experience implies the problem is now endemic and we all need to be more watchful. Inadequate editorial scrutiny and peer review may have contributed to the problem, but a relatively complete lack of scrutiny, as occurs with the majority of preprints for example, is not a simple solution I suggest.
A potentially helpful step would be to insist all researchers make their raw data available alongside any form of publication, including preprint, before it is allowed. There are arguments for and against this concept, but I think on balance it’s where we might have to go sooner or later as it would be one way to set legitimate research apart from the nonsense. In the meantime I think it should become more or less routine for journals to request the raw data for editorial inspection.
RW: Are you working on any other big investigations now that rival Ueshima? Do you think anyone — anesthesiologist or otherwise — will topple Yoshitaka Fujii at the top of the Retraction Watch leaderboard?
JL: I am always investigating something, unfortunately, and I’m not the only one. Other editors I know are in the same boat. It never stops and it takes up a huge chunk of my life. At a guess I’d say we get one or two new cases involving fraud submitted to our journal every week or so, I very much doubt this is any different to other journals whether they realise it or not, and I think it makes up more than a few percent of our submissions.
To give you some idea, I directly caused around 30 retractions from other journals last year, including the few Ueshima papers already retracted, and this was mostly as a result of fraudulent material submitted to our journal by the authors concerned. Assuming the majority of the Ueshima papers identified by that investigation eventually get retracted, in the last five years or so I will have been directly involved in the discovery of two of the top ten cases on your leaderboard. At that rate one has to wonder what’s next and how soon.
Who knows if another case will be as extensive as these, and it’ll take a lot to topple Fujii, but the sad truth of it is that even he is likely to be pushed into second place one day, and I think the top end of your leaderboard will almost certainly become populated by more people from outside the field of anaesthesiology.
Source: Read Full Article
