Professor of Anthropology

Jonathan Marks

A DNA Hybridization Bibliography

Primary Literature

Sibley, Charles G. and Ahlquist, Jon E. (1984) The phylogeny of the hominoid primates, as indicated by DNA-DNA hybridization. Journal of Molecular Evolution, 20:2-15.

This is the original paper claiming to have resolved the trichotomy into human-chimp, with a large battery of highly replicable measurements of genome difference, ΔT₅₀H. It does not mention any numerical corrections, simply the calculation of the heteroduplex melting temperature and subtraction from the homoduplex. A simple statistical test showed that the results were significant.

Sibley, Charles G. and Ahlquist, Jon E. (1987) DNA hybridization evidence of hominoid phylogeny: Evidence from an expanded data set. Journal of Molecular Evolution, 26:99-121.

This is the second paper, containing a larger array of data; it gives specific values for each experiment. It also doesn't mention the existence of numerical corrections. (Our inability to match the values given for 40% of their listed experiments is what led to the discovery of the omitted part of the data analysis.)

Marks, Jon, Schmid, Carl W., and Sarich, Vincent M. (1988) DNA hybridization as a guide to phylogeny: Relations of the Hominoidea. Journal of Human Evolution, 17:769-786.

Here we outlined the potential problems with DNA hybridization, showed that the data do not in fact seem to resolve the trichotomy for they subsume a great deal more variability in measurement than Sibley and Ahlquist had reported, and inferred the existence of unreported data alterations.

Caccone, Adalgisa and Powell, Jeffrey R. (1989) DNA divergence among hominoids. Evolution, 43:925-942.

A second set of DNA hybridization experiments, also from Yale's biology department, claims to have resolved the trichotomy into human-chimp again, and to have matched the altered Sibley-Ahlquist numbers (in spite of having measured a different variable, ΔT_m). The results are attributable to their DNA fragment-length correction, which is impossibly precise.

Sibley, Charles G., Comstock, John A., and Ahlquist, Jon E. (1990) DNA hybridization evidence of hominoid phylogeny: A reanalysis of the data. Journal of Molecular Evolution, 30:202-236.

The authors acknowledge the previously undisclosed data alterations and admit that the resolution of the trichotomy was predicated upon them. They also give their ΔT_ms, which is the variable measured by Caccone-Powell. Unsurprisingly, these neither resolve the trichotomy, nor match the Caccone-Powell numbers. This obviously raises questions about those numbers.

Hoyer, Bill H., N. W. van de Velde, Morris Goodman, and R. B. Roberts (1972) Estimation of hominid evolution by DNA sequence homology. Journal of Human Evolution,1:645-649.

This study was the first to apply DNA hybridization to the problem of human phylogeny. They got a low value for human-chimp, but acknowledged poor reciprocity and concluded that they could not resolve the trichotomy.

Benveniste, Raoul E. and Todaro, George J. (1976) Evolution of type C viral genes: Evidence for an Asian origin of man. Nature, 261:101-107.

This study also applied DNA hybridization to the apes. They found a 3-way split.

O'Brien, Stephen J., Nash. William G., Wildt, David E., Bush, Mitchell E., and Benveniste, Raoul E. (1985) A molecular solution to the riddle of the giant panda's phylogeny. Nature, 317:140-144.

I'm not sure if this dataset is independent of the one just mentioned, but they also found an unresolvable trichotomy, although their tree actually links human-gorilla.

Marks, J. (1992) What's old and new in molecular phylogenetics. American Journal of Physical Anthropology, 85:207-219.

Here I explain just what Sibley-Ahlquist and Caccone-Powell did to their data. You can decide for your self whether it's legitimate. Since the best arguments they can come up with to validate the "corrections" are that they got the same answers, it is important to know whether the answers are in fact the same or not.

Relevant derivative literature

Lewin, Roger (1984) DNA reveals surprises in human family tree. Science, 226:1179-1182.

Science journalist sings DNA hybridization's praises.

Lewin, Roger (1988) Conflict over DNA clock results. Science, 241:1598-1600.

Lewin, Roger (1988) DNA clock conflict continues. Science, 241:1756-1759.

While our two manuscripts were out for review, Lewin wrote up the controversy as news. He reports that a frightened post-doc named Fred Sheldon removed his name from the 1987 paper, and that although Sibley minimized the number of experiments altered, Sheldon verified it is about 40%. Lewin used ad hominem counter-arguments, such as a callout from Roy Britten: "Those manuscripts are not scientific articles, they are weapons with political purposes." He illustrated the articles with figures from our paper without permission. Then he cited Caccone-Powell's unpublished work in support of Sibley-Ahlquist's undocumented work. Lewin provided the first forum for Sibley's defense: It was error, not fraud; the accusers are jerks; and it doesn't matter whether they falsified their data, because they got the right answer. Lewin's conclusion is a classic: "There is no doubt that Sibley and Ahlquist were seriously in error in making substantial, unreported alterations to their data.... But the very combative and partisan tone with which the challenges have been made has not advanced Sarich and his colleagues' stated concern with scientific integrity." In other words, you should only accuse people of fraud nicely. (But we still didn't know exactly what they had done; we tried to discourage him from writing the story until the papers were out; and we tried to keep the focus exclusively on the data.)

 

Horgan, J. (1989) Time bomb: War breaks out in the field of evolutionary biology. Scientific American, (March 1989): 24-25.

Even Scientific American wrote it up.  More competently than Science, appreciating that the Caccone-Powell work was a red herring in discussing the honesty of the Sibley work.

 

Marks, Jonathan (1993) Scientific misconduct: Where "Just Say No" fails. American Scientist, 81:380-382.

In this featured book review on scientific fraud, I related my experiences as a whistle-blower to the books. (Of course, I'm a molecular anthropologist, not an expert on fraud, presumably they asked me to write the review based on my experience...). When Sibley threatened to sue, the editor agreed to solicit and print six letters criticizing me in the next issue. (You might think this would have stimulated an investigation, but it didn't.)

What I wrote was:

The interesting issues in scientific fraud are thus less psychological than sociological. It is not the mind of the perpetrator that requires exploration, it is the ways in which the structure of the scientific community can be exploited to cover one's tracks after the dishonest deed has been committed. The interesting structures are those that exist to prevent fraud from being detected and prosecuted and the knowledge of it from being disseminated once discovered.

There are consequently two processes here: first, the exposure of the misconduct; second, the reaction to it. The exposure itself may be an insufficient action if the social, hierarchical and bureaucratic nature of the scientific community can be exploited in the perpetrator's defense.

Indeed, the next generation of books on this topic will probably feature the story of Charles Sibley, a researcher in molecular evolution who was arrested in the 1970s for smuggling the eggs of endangered bird species out of England and into his starch-gel apparatus. In the 1980s he resurfaced with a technique called DNA hybridization, which solved the problems of avian and primate phylogeny with fanfare enough for election into the National Academy. In 1990, however, he admitted that the phylogenetic resolution had been achieved through the use of data manipulations that had not been mentioned in any of his papers. These included substituting controls across experiments and moving correlated points on a scatter plot into the regression line describing them (Journal of Molecular Evolution, 30:202) - protocols that might well be sufficient to fail an undergraduate.

In spite of the fact that the language used in the published papers wildly misled reviewers and readers as to the nature of the data analysis and the robustness of the conclusions, we have not seen a retraction of the work, an inquiry by the National Academy into the nature of the research of its member, nor a public repudiation of the work by the senior community of molecular evolutionists. Rather, those who have spoken out have taken the position that the data alterations were a bad idea (although not bad enough to jeopardize his membership), their absence from the previously published work was a sad coincidence, the subsequent withholding of documentation was a crying shame, and the serendipitous discovery of the data alterations by others in 1988 was a thorough embarrassment. In other words, "questionable research practices."

Right.

The lesson is not of widespread corruption but simply of bureaucracy in action. Although we may lose a bit of our idealism by facing the reality of scientific misconduct, we can help the next generation not to lose theirs. It is not a matter of telling graduate students not to cheat (the equivalent of instructing inner-city youth to "Just Say No"); it is a matter of subjecting the scientific establishment to examination - showing our students how science works and how it doesn't - so that when they enter the community, they can improve it from within.

A couple of letters just sort of said I was a big jerk, and that I couldn't accept that so much genetic data linked human-chimp, just like Sibley said -- which was neither relevant or true. A few others made more fascinating points, to wit.

Sibley, Charles G., and Ahlquist J. (1993) Reviewing misconduct? American Scientist, 81:407.

Sibley and Ahlquist's letter indignantly demands that (as I had suggested in my review) the National Academy of Sciences investigate the matter, knowing they wouldn't. Shortly thereafter Vince Sarich and I separately wrote to the NAS with charges and documentation. You might figure that if both sides clamor for an investigation, they might undertake it. But they didn’t.

Britten, Roy J. (1993) Reviewing misconduct? American Scientist, 81:408.

Britten's critical letter attempts to defend the data alterations. He tries to make it sound like they're okay in the passive voice: "Dr. Marks alleges that Dr. Sibley substituted controls across experiments; in fact, a correction was made by using a well-known value for the melting temperature of hybridized pieces of DNA when the individual runs showed erronious results." And, "Finally, Dr. Marks charges that Dr. Sibley moved 'correlated points into the regression line describing them.' This is untrue. The fact is that regression-line plots were used to detect outliers and make corrections on runs with large or small extents of hybridization." Sure glad he cleared that up... I wonder if he treats his own data that way?

Powell, Jeffrey R. (1993) Reviewing misconduct? American Scientist, 81:408.

According to Powell, "The Sibley work is good science inasmuch as it is repeatable and independently corroborated." Actually, good science is simply carried out competently and honestly. The fact that two people make the same claim doesn't mean the claim is right. In this case, Powell argues that it doesn't matter whether they falsified their data, because they got the right answer. A frightening thought, especially from someone charged with educating the next generation of biology majors at Yale.

Kirsch, John A. W. and Krajewski, Carey (1993) Reviewing misconduct? American Scientist, 81:410.

This argued that "their methods were logical, and made very little difference to inferences from the complete data." Well, I suppose their was logic to the thought, "Let's take the experiments that gave us numbers we don't like, and pretend they gave us numbers we do like." But the second clause is simply utterly false, by Sibley's own admission in their 1990 paper. 

Perhaps an investigation could have resolved these issues.

My response to these letters was edited beyond what I considered to be acceptable, and the editor published her version anyway. American Scientist accused me of making a "personal attack" on Sibley, when again, the issue is the work, not the person. Since my original review was vetted by two levels of editors to begin with, you might think they would have noticed any "personal attacks". There weren't any. Everything I said was accurate and relevant to evaluating scientific research.

In a letter to Charles Sibley dated August 31, 1993, the editor informed him that "Jonathan Marks, by the way, will not be reviewing books for American Scientist". I might mention that American Scientist is published by the scientific society called Sigma Xi, who also publish tracts on scientific integrity. Blacklisting is not discussed in any of their tracts.

Here is my full, unpublished response to the letters:

Readers of American Scientist may note that the current issue contains six letters critical of points I raised in a book review on scientific fraud in the last issue. The abbreviated response that appears after them was published without my approval, and against my wishes. I provide a full response below, for any interested readers.

Sibley and Ahlquist claimed in 1984 and 1987 to have proved genetically and statistically that humans, chimps, and gorillas, widely regarded as genetically equidistant from one another, were actually not. Rather, they asserted that humans and chimps were genetically more similar, and were each others' closest relatives. They were under the same obligations as any scientist: to prove that their conclusions were sustained by their data. In their 1990 paper (J. Mol. Evol., 30:202-236), Sibley et al. acknowledged that the conclusions of their earlier papers had been derived with the use of unreported analytic procedures: had it not been for these data alterations (in their own words) "... it is virtually certain that Sibley and Ahlquist would have concluded that Homo, Pan and Gorilla form a trichotomy" (p. 225).

The alterations were made without consistent objective criteria, as I have exhaustively detailed (Am. J. Phys. Anth., 85:207-219, 1991). The one they used most extensively involved constructing a bivariate plot of their data points, deciding (based on unclear criteria) that 40% of their data constituted aberrant outliers, and (in their words) "moving the aberrant point to the linear regression ... and calculating the new ... value." (1990, p. 232). This is fundamentally illegitimate, especially since they treated their data points as statistically independent in their papers, and neglected to inform readers that they had performed these data transformations. It is difficult to imagine that the papers would have been published has these data analysis procedures been specified.

Roy Britten's circumlocutions do not make the alterations any more legitimate. Indeed, the reason the alterations came to light at all was that Britten, in possession of the only data provided to anyone by Sibley (who had steadfastly refused many requests by many scholars), passed them on without having noticed their problematic nature. The alterations and their illegitimacy, however, were recognized by the fearful post-doc who removed his name from the work (Science, 241:1758, 1988); and by Vincent Sarich, Carl Schmid, and myself (J. Hum. Evol., 17:769, 1988; Cladistics, 5:3, 1989). This leads to the conclusion that the Sibley-Ahlquist data did not actually resolve the trichotomy at all, and were misrepresented as having done so.

Now, was the misrepresentation deliberate? In the best case, the research would be characterized methodological poverty; the omission of the vital information would be coincidental; and the refusal of the authors to allow others to see their data would also be coincidental. In the worst case, they knew exactly what they were doing, the omission was required to conceal the illegitimate analysis; and the sequestering of information was required to conceal the omission of the alteration procedures from the published papers.

The best light in which their work could be cast would be that they "got the right answer", and their conclusions were simply a matter of overzealous prophecy, rather than simple charlatainerie. Even if this were true, it would be irrelevant to evaluating the nature of the data alterations. The larger problem is that this argument places a premium on the conclusion, rather than on the rigor of the data collection and analysis. It reaches its most extreme form in Jeffrey Powell's "replication" of their altered numbers, measuring a non-comparable variable.

One reason to be skeptical of this is Powell's assertion that the "particulars" of the data analysis by Sibley and Ahlquist are of little concern, since they were used in obtaining the correct conclusion. The nature of their data, the methods of analysis, and their subsequent representations in the literature are the central issues; and I am hopeful that few readers will concur that the end ever justifies the means in scientific research. Valid scientific work cannot be conclusion-driven, contrary to Powell's assertion. It carries obvious consequences for the quality of both research and mentorship. It stands to reason that if we need not worry about Sibley's methodology, we need not worry about Powell's methodology either. Unfortunately, some of us do. Quite possibly the decline in popularity of DNA hybridization among mainstream scientists, noted by Kirsch and Krajewski, reflects an appreciation that in this field, "anything goes" methodologically, in generating the "right" answer.

Ultimately the issues center around data falsification, its exposure, and its concealment. Sibley and Ahlquist used covert techniques to alter their data, which they and their supporters can only justify by recourse to the conclusions of other researchers. That is not, and cannot be, a validation of their methodology; it is a non-sequitur.

But is there indeed a genetic concordance favoring human-chimp, as Sibley and Ahlquist concluded, thought their data didn't actually show it? Obviously, if the split among humans, chimps, and gorillas is so close as to be effectively a three-way split, any specific genetic study may randomly appear to link any two of the three genera. There are two classes of data bearing on this problem: (1) those that do not permit any clear bifurcation to be drawn; and (2) those that clearly link two of the three as closest relatives, and contrast with other studies clearly linking a different two of the three. I recently reviewed this literature (Curr. Opin. Genet. Dev., 2:883-889, 1992), and interested readers can find comprehensive references therein. Czelusniak and Goodman cite only their own data and ignore the rest. This is not scholarly, but coming as it does in defense of the work of Sibley and Ahlquist, it sadly does not seem inappropriate.

The scientific issue is that reviewers and readers of Sibley and Ahlquist's papers were misled about the analysis of the data and consequent robustness of the conclusions. That has been public knowledge for several years. The social issues are less clear cut: what to do, who should do it, and to whom it should be done -- which is why I raised it in my review of the books on misconduct. For Sibley's "illegally importing bird parts taken abroad in violation of foreign wildlife laws", I refer readers to the published account (Sports Illustrated, 24 June 1974), which appears to have been of greater concern to falconers than to molecular evolutionists. If I overreached, I extend my apologies to the American Scientist and stand duly chastised.

As for motives and self-interests, I choose not to speculate on those of the correspondents, and I hope that unless The Amazing Kreskin writes in, no one will be distracted by their speculations about mine.

 — continue —

Asimov, Isaac (1991) The narrow gap between chimps, humans. The Herald-Sun, 24 March 1991.

In his syndicated column, even Isaac Asimov is aware that there are suspicions of dishonesty about this work. It wasn't much of a secret.

Highley, Aaron M. (1991) DNA hybridization. Yale Scientific, 65:6-10.

A student-run science journal at Yale wrote up the controversy -- if you can call data falsification controversial.

Ruvolo M, Disotell TR, Allard MW, Brown WM, Honeycutt RL. 1991. Resolution of the African hominoid trichotomy by use of a mitochondrial gene sequence. Proceedings of the National Academy of Sciences 88(4):1570.

 

This is technically primary literature, but derivative with respect to DNA hybridization, and especially for stimulating the next piece cited, in Science.  A short stretch of mtDNA, skipping the orangutan and using a gibbon for the nearest outgroup, seemed to link human-chimp, as per Sibley.  Under ordinary circumstances the paper would probably have been unpublishable, but the circumstances were not ordinary, and it was published in the prestigious Proceedings of the National Academy of Sciences.  But it pushes the gorillas far away from human-chimp, which is contrary to the bulk of the data, including Sibley’s, but represents itself as concordant with Sibley’s phylogeny.  Oh, and it was communicated to the PNAS by Sibley himself.

 

Gibbons A. 1990. Our chimp cousins get that much closer. Science 250:376.

 

           Promoting the paper just mentioned, although it hadn’t been published yet.  Actually says in black and white, in the leading science journal in the US, that we accused Sibley of data falsification.  Also casually describes DNA hybridization as “tricky” - far less elegant than earlier.

 

 

 

DNA hybridization never did work, except crudely, and nobody does it any more.   As far as I know, the last active defenders of the DNA hybridization work in the anthropological community were David Pilbeam and Maryellen Ruvolo at Harvard, who were both heavily invested in it. Pilbeam wrote in a journal called Molecular Phylogenetics and Evolution: "I do not discuss the Sibley et al. (1990) data set here, although I will do so elsewhere, except to note that it is of poorer quality than that of Caccone and Powell." Pilbeam, sadly, was never able to evaluate this work critically, and of course was its earliest promoter in anthropology. I, for one, hope he doesn't make good on his threat to discuss it again elsewhere. Mercifully, nearly twenty years later, he hasn't.

Home