As I write this there's a diary on the Rec list arguing that a recent paper in The Organic Systems Journal has published a "significant finding" to the effect that consumption of GMs did greater harm to the test subjects - here, a group of pigs - than consumption of organics. This is in fact not true, and one doesn't have to be a GM advocate to see, very clearly, what this study got wrong. I'll start with the most obvious point, and work my way to some of the broader criticisms, but this is yet another case of advocacy getting far in front of the science, to the detriment of the advocacy.
Without getting into the larger GM debate, I can promise those of you who oppose GM that pushing this study as evidence will do you more harm than good, for reasons that come from the study itself:
(Note: as I am not an expert on these issues, most of my criticisms are drawn from two sources, James Andrews of Food Safety News and Mark Hoofnagle of Scienceblogs. I'd recommend the Andrews article first, as it's a less strident and more balanced take.)
Here's the most damning point. We'll start with the claim:
In pigs eating genetically modified crops, the average rate of severe stomach inflammation was nearly three times as high as that for other pigs (32 percent vs. 12 percent). Among male pigs eating a GM diet, the rate of severe stomach inflammation was four times higher.
Here's the actual data from the study (
pdf):
Inflammation |
% non-GMO |
% GMO-fed |
none |
5.4% |
11.1% |
mild |
42.5% |
31.9% |
moderate |
39.7% |
25.0% |
severe |
12.3% |
31.9% |
One shouldn't need a degree in statistics to see what the problem is here. It'd be equally true (and just as meaningless - see below) to say that pigs eating genetically modified crops were
twice as likely not to have any stomach inflammation at all. Male pigs, which the authors say were even more susceptible to stomach inflammation than females, were also
four times as likely not to have any inflammation (11.1% versus 2.8%). The most one can say with confidence is that the numbers are a bit scattered and don't suggest any clear hypotheses, but in the interest of advocacy the authors have literally ignored half of their own data to give you only the conclusions that you want to hear.
The problem with using this data in the way that it has been used should be obvious, no matter what side of this issue you fall on. For those of you who advocate against GMs, studies like this undermine both your credibility and the evidence-based foundation of your arguments. This is not what you want.
++++
That's really all that needs to be said, but for those who are interested and want to keep going with this issue, we can break this down further. First is the problem of the table itself, which is a qualitative study of inflammation based on scoring by veterinarians, which... again, is not really the best way to approach this kind of thing:
Inflammation was classified as nil, mild, moderate, or severe based on a combination of the area of current inflammation and level of redness and swelling.
Oof. I assume they mean
percentage of area inflamed, and I have no idea how one scores "level of redness" in coherent way: why no study of the actual tissue pathology? (Answer: because there were no pathologists involved in the study. Which purports to be a study of pathology. Draw your own conclusions.) Furthermore, as Hoofnagle points out, if we divide the groups into two - little or no inflammation versus inflamed - the distinction between the GM-fed and non-GM groups disappears entirely. This should be another big alarm for readers.
update: see Mindful Nature's comment (and alain2112's follow-up) for a more sophisticated take on the study's problem with these numbers.
It gets worse for those who want to use the study for anti-GM advocacy: this was the most compelling evidence of anything like a health distinction between the two groups of pigs. Their chemical analysis of the difference in levels of glucose, AST, cholesterol, protein (total), albumin, nitrogen in the urea, creatinine, phosphorus, calcium, sodium, potassium, chloride, bicarbonate, and creatine kinase all came up statistically insignificant (with only one enzyme, gamma-glutamyl transferase, approaching a statistical difference, and even then inconclusively so.) Their search for abnormalities in the kidneys, hearts, livers, spleens, lungs, intestines, and ovaries all came up empty. The most that could be said about the uterus is that there were some size differences and fluid levels that may be worth pursuing.
We don't even have to go into the political objections - the team leader is an anti-GM advocate; the journal is by its nature pro-organics; the study's design was a (to use Hofnager's term) a "fishing expedition" - because these are going to be subjective points of conflict between advocates on both sides. And we can, following Andrews' lead, admit to the limitations that current GM research is faced with due to the problem of corporate (read: Monsanto) control of their product, to the point that researchers do not have access to the purest form of the product to test in a fully-controlled way. These are reasonable angles for critique that reflect the broader world of GM debate. But not this study. This is bad science, and at the very best, we can argue for a more specifically targeted look at stomach inflammation, preferably with someone on board who can assess the actual tissue pathology. It may be that some types of GM grain can cause higher incidence of stomach inflammation, but this study doesn't show that, full stop.
In conclusion:
If I were an advocate against GMs focusing on their potential health dangers, this would be the last study I'd promote. It was a wide-ranging expedition that found, on almost every axis, that there were no health distinctions between GM-fed and non-GM test subjects, and in a desperate bid for something, misrepresented the data to give the impression of one genuine threat.