Thesis, antithesis, synthesis – is a primary pathway of science. Researchers trot out a hypothesis (hopefully backed up with evidence). Other researchers challenge the hypothesis, often through their own research results. Eventually the original hypothesis is confirmed, rejected, or in some way modified. Science moves on. The process may start with scientists presenting a research paper and the results of experimentation – in this case a paper published in the top journal Science that identified a specific set of gene variants indicating with a reported 77% accuracy the potential for people to live to 90 and beyond [SciTechStory: Gene variants for living to 100 identified]. As a rule, the research is peer reviewed before publication, which means it should have gone through one or more rounds of questioning and challenge, especially for a blockbuster result. Sometimes the challenge intensifies after the paper goes public, which is the case here. Almost immediately the results of the study were questioned by a variety of genetic scientists and on several grounds.

1. The study used two different DNA genome sequencing chips (analytical electronics), one for the control group and another for the study group. Although the chips were from the same company (Illumina), they are known to have different methodologies. This can lead to bias in the data.
2. The extraordinarily high prediction rate of 77% raises warning flags on general principles.
3. The relatively small number of people tested (sample size of about 1000) in the study.
4. The unusually large size of the SNPs involved. A ‘snip’ (SNP=single-nucleotide polymorphism) is a DNA sequence variation. The longer the SNP, the more likely for variation.
5. Many of the ‘new’ variants found in the study have never been associated with disease (presumably a connection needed for them to increase longevity).
6. None of the SNPs correlated to old age were verified with a different analytical system (from another company).

It is not necessary to grasp all the technical details (and there are many) to get the idea these collectively represent a serious challenge. In scientific parlance…this is a furor. It reaches this intensity when the subject matter is important. Predicting for old age certainly qualifies – knowing whether you have the genes to live to a ripe old age, while not everything, is certainly good (or bad) news. Perhaps more importantly, if these gene variations are really strongly correlated with old age, it means they are also strongly correlated with having good or bad health. That makes these genes potentially a rather large window into the relationships between genes, health, and living a long time. For scientists, it’s like having a new roadmap for ten or twenty years of research.

Naturally the scientists involved with the original study have their own opinions. Although somewhat taken aback by the immediate and often vociferous attack on their research, they were most surprised by the possibility of technical difficulty introduced by using two different generations of the Illumina analytical chip. Here are a few quotes from an interview with the principal authors of the study, Paola Sebastiani and Thomas Perls, both of Boston University:

Q: What do you make of the controversy? Did you expect the paper to be controversial?
P.S.: To be honest, I expected this to be controversial for the type of analysis, and not for some issues with the data. There are very innovative ideas in the paper, … the idea of not really looking at the effect of individual SNPs, but the global effects of several SNPs, … the idea of looking at patterns rather than individual variants. I expected that to generate discussion.
…
P.S.: I think there are errors in every paper. This is part of the scientific debate–from errors, sometimes you can come up with very good ideas.
Q: Do you feel that the results in the paper are generally correct, that they’ll hold up?
P.S.: We want to wait until all the analyses are done.
T.P.: That’s one of the problems that we do not want to contribute to. There have been very premature and harmful proclamations about what the data show or don’t show. I think people need to know that good science takes time, and while people are anxious to know and we completely understand that, we want to be able to do a thorough … analysis.

[Source: Science Insider]

So it’s back to the data for the authors, who promise to have their response(s) in relatively short order. This is called ‘debate’ – with the evidence as the focal point for the discussion. It is not a demonstration of the weakness of scientific research – it is one of the strengths. On the other hand, perhaps the criticism may hold up that this study was pushed forward more by a desire to capitalize on its more sensational aspects than to do a lot of painstaking fact checking. That too is valid scientific criticism as the attitude can sometimes be more powerful than what the evidence will bear.