Three "classic" US cases: 1979 and Vijay Soman; 1981 and John Darsee; 1985 and Robert Slutsky

1. Vijay Soman

Soman was an assistant professor of medicine at the Yale School of Medicine, who plagiarized parts of a manuscript sent in 1978 by the New England Journal of Medicine for peer review to his boss, Philip Felig, who passed the job on to him. Subsequently Soman and Felig published an article on the same topic, insulin binding in anorexia nervosa, in the American Journal of Medicine. Accused of plagiarism and conflict of interest, Felig seemed to settle the difficulties by stating that the work had been completed before they had received the paper for review. But its author, Dr Helena Wachslicht-Rodbard, a young researcher at the National Institutes of Health (NIH), who during this episode was to switch to hospital practice, persisted with her complaints. An inquiry by Dr Jeffrey Flier, a Boston diabetologist, in February 1980 showed that these were justified. Not only had Soman copied her manuscript, but most of the data in his own joint study had been faked. A subsequent investigation soon after found that, of 14 articles, only two could be approved, and the data were either missing or fraudulent in the remaining 12 (10 of them with Felig as co-author). All these articles were retracted by Felig.

Click here to read review of this book by Gyan Fernando

This case is important as it highlighted several themes that were to feature strongly in subsequent cases. Firstly, there was an abuse of editorial peer review. This was something that had often featured as anecdotal accusations by angry authors in the past but had rarely been documented (though it has occurred again since this episode). Secondly, there was the involvement of a distinguished figure as a "gift" author, who had put his name to work with which he had not been associated - indeed, he could not have been because the work had not been done and the results were invented. Thus the episode was a personal tragedy for Felig, who resigned from a prestigious post at Columbia University, to which he had been appointed while the episode was unfolding. Thirdly, in these early days the authorities did not know how to react. They were hesitant about instigating a full and proper inquiry into an accusation made against a senior figure, and only Wachslicht-Roadbard's persistence brought about the disclosures.

2. John Darsee

Darsee was a Harvard research worker in cardiology, who was seen to falsify data during a laboratory study. His overall head of department, the distinguished cardiologist Eugene Braunwald, decided that this was a single bizarre act and allowed him to continue to work under close supervision, but terminated his NIH fellowship. Six months later, however, it became clear that Darsee's data in a multicentre study of treatments to protect the ischaemic myocardium were different from those at the three other units taking part. Harvard Medical School set up a committee of investigation, as did the NIH and Emery University, where Darsee had also worked. It emerged that Darsee had committed an extensive series of frauds, originating in his undergraduate days at Notre Dame University and continuing at Emery and Harvard. These included non-existent patients or collaborators, and invented data, as in the multicentre trial. There were also procedures and results that on reflection were virtually impossible: drawing blood from the tail veins of 200 rats weekly for all of their 90-week lifespan, and obtaining venous blood specimens (including from a 2-year-old child). In all, during his career Darsee published over 100 papers and abstracts, many of them in prestigious journals and with distinguished co-authors; many of them had to be retracted.

...Possibly until recently, more ink has been shed on Darsee's case than on any other. In part, this was because it was the first major publicised case that was not an isolated blemish on the face of science (not mad - rather, bad); in part, because it concerned prestigious institutions, co-authors, and journals; in part, because of the charismatic personality of one of the central figures; in part, because it started the whole debate about the rights and wrongs of authorship (particularly gift authorship), data retention, the supervision of juniors, and the management of suspected cases of fraud....

Possibly until recently, more ink has been shed on Darsee's case than on any other. In part, this was because it was the first major publicised case that was not an isolated blemish on the face of science (not mad - rather, bad); in part, because it concerned prestigious institutions, co-authors, and journals; in part, because of the charismatic personality of one of the central figures; in part, because it started the whole debate about the rights and wrongs of authorship (particularly gift authorship), data retention, the supervision of juniors, and the management of suspected cases of fraud. There was also the realisation of the pressure to publish - and not merely important work but anything that showed a department's activity, though the results should somehow be positive. Finally, the case also shifted the whole climate of feeling of trust to thinking the unthinkable - the possibility that things might not be as they seemed. There was also the new concept: once a crook, often always a crook - Darsee was found to have had a long history of faking his results in different projects and in different settings.

All these ramifications are ably explored in Marcel LaFollette's book, and also in her revised article in this volume (Chapter 3). A particularly cogent account of why Darsee was trusted in the conditions prevailing at the time was provided by his former mentor, Eugene Braunwald. This episode also shows the inadequacy of editorial peer review for detecting fraud (although two workers at the NIH were subsequently to demonstrate some egregious errors in Darsee's papers), as well as the role of attempts by other workers to replicate results in revealing its existence.

Unlike some other disciplines, such as chemistry or physics, a lack of replication is a feature of research in medicine, given that it is often complex and expensive, and there is little enthusiasm by peers for merely confirmatory work. Nevertheless, failure to replicate results has occasionally brought other medical research frauds to light, as in the case of Claudio Milanese, an Italian immunologist who had claimed that IL-4A, a lymphokine, induced interleukin-2 receptors. Later research showed that IL-4A did not exist. Finally, subsequent research showed the depressing result that, although Darsee's articles had been retracted from the literature - and that this fact was reflected in the electronic databases - these were still being cited in the literature, not in articles on research fraud but as references in papers on the very topic with which Darsee's fraudulent research had allegedly been concerned....

...To question the veracity of data is then to look the gift horse in the mouth. It may be wise, however, to have at least the level of suspicion that the Trojans were advised to have when confronted with their gift horse. ..... ......It has been suggested that statisticians should not publish too much on the methods to detect fraud, since then those wishing to pervert science without being caught will learn how to avoid detection.....

The very use of the word "data" (from the Latin "given") suggests that those who receive data for analysis do so in the spirit of a gift. To question the veracity of data is then to look the gift horse in the mouth. It may be wise, however, to have at least the level of suspicion that the Trojans were advised to have when confronted with their gift horse.

The emphasis in statistical textbooks and training is on analysing the data, assuming they are genuine. Checking the data is sometimes mentioned, although this is directed mainly at the possibility of accidental errors rather than at deliberate falsification. Altman notes that, "It is the large errors that can influence statistical analyses." Accidental errors that are large clearly affect the analysis, but alteration or invention of data will be done in a way to attempt to conceal their false nature. The features of these false data will not be the same as ordinary errors. In spite of this, careful use of the best procedures for data checking with some simple extensions will go a long way to detect many instances of fraud. This chapter will outline routine methods for checking data that will help to correct accidental errors as well as the problems that are the target of this book. The accidental errors in data are (one hopes) more frequent than deliberate ones, so that effort expended in checking data will result in better quality reports of medical science even in the absence of attempts to cheat.

The power of modern computer programs for statistical data analysis is a great help with this type of data checking, but also facilitates fabrication of data. More emphasis in the training of statisticians needs to be given to this aspect of data analysis, since a perfect analysis on the wrong data can be much more dangerous than an imperfect analysis of correct data. It has been suggested that statisticians should not publish too much on the methods to detect fraud, since then those wishing to pervert science without being caught will learn how to avoid detection.

Examining data: two or more variables at a time

1. Statistical methods

When data are invented to manipulate or show an effect that is not present or not present so clearly in the genuine data, then a skilled manipulator will perhaps be able to produce convincing data when viewed in one dimension. It is very much more difficult to retain the nature of real data when viewed in two dimensions. The relationship between variables tends to disappear. In a well-documented example (Bailey KR. Detecting fabrication of data in a multicenter collaborative animal study. Controlled Clinical Trials 1991;12:741-52), a laboratory study on animal models of myocardial infarction involved a number of variables. The simplest example of this problem was the data relating weight of the dogs versus the weight of the left ventricle. In this example of very elaborate forgery, the range and variability of left ventricle weight was high, in fact higher than in the genuine data, with a similar range for the weights of the dogs. The correlation between these two measurements was very much weaker. The situation with infarct size versus collateral blood flow was even worse, where the variability in collateral blood flow was very much less than expected and the relationship that should have existed was absent.

...In a well-documented example, a laboratory study on animal models of myocardial infarction involved a number of variables. The simplest example of this problem was the data relating weight of the dogs versus the weight of the left ventricle. In this example of very elaborate forgery, the range and variability of left ventricle weight was high, in fact higher than in the genuine data, with a similar range for the weights of the dogs...

This type of problem is not easy to detect by simply reading a paper, but ought to be detected by a statistician with access to the raw data and familiar with the science of the study. In some cases, a correlation matrix may be presented, and careful examination of this may show unexpected findings that raise the index of suspicion.

In the example quoted, the study was being carried out in several laboratories simultaneously so that the differences between the laboratories could be studied very easily. In fact, the study itself was set up because of previous inconsistencies in the results from different laboratories.

In many situations, there are no data available from multicentre studies and considerable experience in the field may be necessary to detect the problem.

The situation with regard to several variables is an extension of that seen with two. The variables on their own tend to show reduced variability, but even when this is not so, the relationships among many variables become much weaker than they should be.

As has been noted above, the examination of the correlation matrix may also show where relationships are too weak (or, on occasions, too strong) for genuine data. This approach essentially examines the relationships between pairs of variables. True multivariate methods are able to look at the effect of many variables simultaneously. These can be of use in sophisticated data checking.

The first, well-known, multivariable method examines the "influence" of individual observations. It is of most help where data errors have been made and for ensuring that single observations do not distort the results of an analysis too much.

The basic idea is to have a single outcome variable that is the measurement of greatest importance. This is used as the response (dependent) variable in a multiple regression analysis, with a number of possible explanatory (independent) variables, including one for the treatment group if a comparative study is being analysed. The first step is to use standard methods of multiple regression. This entails obtaining as good a fit to the data as possible, which also makes biological sense. For these purposes, it may be reasonable to obtain the best fitting equation (also called a "model"), regardless of how sensible it is in biological terms. The inclusion of variables that are not thought to be medically relevant may indicate that there are problems with the data. The relationships with such variables may merit further investigation.

The book is amply illustrated with charts, graphs and diagrams all over (which should be a statistician's delight). This is how Evans illustrates the highly interesting concept of Cook's distance (This figure appears on page 197) [Click to enlarge]

There are several measures of "influence" available, probably the best of them is called "Cook's distance". This is like a residual in multiple regression: the distance between an observed point and the value predicted for that point by the regression equation. It measures how far off a point is In both the X and Y directions. An ordinary residual may not be very informative, since outliers may have small residuals in that they "attract" the regression line towards them. An alternative is a "deleted" residual, which involves calculating the equation for the regression line excluding that point, and obtaining the residual from the predicted value with this regression equation. This will be very effective when a single outlying point is present in the data. An outlier can influence the regression equation in two ways. It can influence the "height" when it is in the centre of the data, but it influences the slope when it is also an outlier in at least one explanatory variable. This effect is known as "leverage", and is illustrated in Figure 14.3, where only two dimensions are shown. The usual measures of leverage effectively relate to how far points are away from the centre in the X direction. Cook's distance for the outlier in Figure 14.3b is very large because it has leverage as well as being an outlier. In Figure 14.3a it is an outlier but does not have leverage and has a smaller Cook's distance. The slope of the line in Figure 14.3b has been notably altered by a single observation. Single outliers are probably more likely to be data errors rather than invented data, but investigation of the reasons for the outlier may be important in the data analysis for genuine as well as fraudulent data. There do exist statistical tests for multiple outliers, but these are beyond the scope of this introduction but the Hadi statistic is implemented in the statistical packages Stata (College Station, Texas) and DataDesk (Ithaca, NewYork). Statistical mathematics will not usually be as helpful as graphics in detection of problems.

The problem with an invented data point is that it is unlikely to be an outlier in any dimension; in fact, the exact opposite is true. Invented data are likely to have values that lie close to the mean for each variable that has been measured. In one instance of invented data of which I had experience, the perpetrator used the results of an interim analysis of means of all the measured variables to generate four extra cases. For these cases, either the original data were lost, or the results did not fit the desired pattern, and the records were destroyed by the perpetrator.

Role of editors and peer review

The ability of editors and the peer review process to uncover research misconduct is controversial. Riis (1994) states that, "Editors have few opportunities both in theory and practice to detect and to prevent fraud." However, Farthing writing six years later in the year 2000, has a more optimistic view, which may reflect the much higher profile of the problem and therefore the will to find solutions.

Whilst the peer review process is deemed highly desirable and is meant to detect weaknesses in design and execution, most research misconduct has only come to light through the offices of "whistleblowers". However, the infamous Pearce paper on embryo implantation was not peer reviewed and the editor of the journal was himself a gift author; the second paper deemed fraudulent was peer reviewed, and also reviewed by a statistician and a statistical report received, and it had also been through the LREC (Local Research Ethics Committee). With the gift of hindsight it was obvious that the number of patients reported to have been studied with this particular condition was highly unlikely given the catchment area of the investigator. However, although stringent peer review may detect plagiarism and redundancy of publications, the downside of the peer review process is that the reviewers may themselves be guilty of misconduct by plagiarism of ideas or data or failing to declare conflicts of interest, including competitive grant funding in the area of research in question, as well as financial interests such as external consultancy remuneration.

...the downside of the peer review process is that the reviewers may themselves be guilty of misconduct by plagiarism of ideas or data or failing to declare conflicts of interest, including competitive grant funding in the area of research in question, as well as financial interests such as external consultancy remuneration. Indeed, editors themselves are not immune from research misconduct with records of fabricated papers, introducing review bias, and a range of inbuilt prejudices. However, the relatively recent establishment of the Committee on Publication Ethics (COPE) has been a force for good, where discussion of the issues of prevention are regularly debated....

Indeed, editors themselves are not immune from research misconduct with records of fabricated papers, introducing review bias, and a range of inbuilt prejudices. However, the relatively recent establishment of the Committee on Publication Ethics (COPE) has been a force for good, where discussion of the issues of prevention are regularly debated.

In the 1995 report from the independent committee of inquiry into the circumstances surrounding the publication of two articles in the British Journal of Obstetrics and Gynaecology, a series of "editorial" recommendations were made to aid in the of prevention of future fraudulent research papers. Amongst many recommendations were the following.

Firstly no article should be accepted without a letter of submission signed by all authors, including confirmation that they understood the requirements for authorship and that journals must make these criteria absolutely transparent, in line with the International Committee of Medical Journal Editors (ICMJE) guidelines. Furthermore, they felt that consideration should be given to requesting that each author's role and area of responsibility is clearly specified in the letter of submission. This did not occur with the submission of the embryo implant case report. As discussed earlier, editors should feel free to request sight of raw data, and submission of a paper would include agreement that, if required such raw data would be provided. Indeed, it might be desirable to undertake random requests for raw data so that authors were aware that such verification was taking place. Thus, the ability to request raw data might detect fraud, whilst random requests might act as a deterrent.

The issue of "hawks" and "doves" as referees could partly be dealt with by the authors themselves suggesting appropriate referees, to be used at the discretion of editors, as well as authors indicating referees that they would not wish their paper to be sent to. It is also paramount that the referee be allocated by an editor who has no connection with the submitted article, either as an author or through the institution in which he or she works. If this had happened the fraudulent Pearce articles might never have seen the light of day. All papers being considered for publication with any statistical content should be reviewed by a statistician, and journals must allocate adequate resources for this to happen; if statistical queries are raised, the statistician must review the revisions prior to acceptance. Statisticians must also have a place at the editorial board table.

The embryo transplant paper was never refereed and two of the authors worked in an editorial capacity for the British Journal of Obstetrics and Gynaecology. The enquiry concluded that case reports should be dealt with using the standard operating system in a similar way to full papers and short reports, and must be refereed.

The enquiry then considered the system that should be in place when an editor is an author, or has an interest, or is associated with the authors, and concluded that a clear system must be in place with written documentation indicating the processes whereby such papers are dealt with. Clearly the process has to be entirely independent of the editor concerned. Therefore an independent editor must select the appropriate referees and deal with the manuscript, and the interested editor should never be present during any discussion at editorial meetings.

Many of these good practices are of course in place with many journals (but by no means all), and some of the enquiry's recommendations were peculiar to the British Journal of Obstetrics and Gynaecology. Of course there is no "evidence" that good editorial practices and good peer review processes will indeed be effective. This evidence needs to be acquired. The lessons to be learnt from the Pearce affair were also discussed in an editorial in the BMJ by Lock.

Whistleblowers

A whistleblower is a person who is aware that misconduct is occurring and responds accordingly. Data concerning the prevalence of whistleblowers around the globe are difficult to obtain. Wilmshurst in 1977 (Wilmhurst P. The code of silence. Lancet 1977;349:567-9) listed a series of instances when the UK employers (usually academics within universities) either failed to take seriously, or to make public, actions regarding allegations of or the uncovering of research misconduct. It makes depressing reading.

...A whistleblower is a person who is aware that misconduct is occurring and responds accordingly. Data concerning the prevalence of whistleblowers around the globe are difficult to obtain. Wilmshurst in 1977 listed a series of instances when the UK employers (usually academics within universities) either failed to take seriously, or to make public, actions regarding allegations of or the uncovering of research misconduct. It makes depressing reading...

Frank Wells (Wells FO. Clinical research fraud and misconduct: how is it diagnosed? Proc R Coll Physicians Edinb 2000;30:13-17) discusses the role of whistleblowers citing the Pearce case as well as that of Fairhurst (Chapter 5), but in that article does not discuss his/her protection. Usually whistleblowers in any arena have a bad time, often being either disregarded, dismissed, suspended, pilloried, shamed, or threatened by peers or managers. The most recent example (although not in the research arena) was the anaesthetist who blew the whistle on the results of paediatric heart surgery at the Bristol Royal Infirmary; he subsequently resigned and went to Australia to get another job. It is to be hoped that this will not happen in the future in the UK since, as of July 1999, the new 1998 Public Interest Disclosure Act provides some protection for whistleblowers.

However, more importantly, as Frank Wells points out, the hostile environment that whistleblowers find themselves in will only be improved when a stronger attitude of intolerance is shown towards research misconduct. This has also recently been enforced by the GMC in various publications including "Good Medical Practice" outlining the professional duties of doctors to uphold the integrity of the medical profession. Of course the GMC has no jurisdiction over non-medically qualified scientists. Both the GMC and the Royal Colleges have a duty of care towards patients and, in the interest of patient safety and to maintain confidence in scientific research, whistleblowers must be given support and protection where appropriate. Donald Irvine, President of the GMC has stated, "Where doctors are implicated in research misconduct, decisive GMC action will follow. Doctors found guilty of serious professional misconduct in relation to research can expect to be erased from the medical register unless there are compelling mitigating circumstances."

Click here to read review of this book by Gyan Fernando

Allocation of resources for research

There are several question marks over the legal validity of the screening and investigation procedures designed to investigate research misconduct as discussed elsewhere in this volume. Thus many questions remain unresolved. Does a researcher suspected of misconduct have a contractual obligation to participate in the investigating procedures! If not they should. Whilst employees of the pharmaceutical industry, device manufacturers, and the UK's MRC do, in the main this is not usually in place within the university system or the NHS.

However, all grant-giving organisations will in future expect transparent procedures to be in place in the UK. The Association of Medical Research Charities (AMRC) has stated that their member charities should include as part of their guidance to research applicants the following: