Photo by Pascal
Have you ever seen a news report about something you already know about? If you have, I can pretty much guarantee you thought, “Wow, that reporter doesn’t know anything about this.” Think about it a bit and you realize that’s probably true of all the stories you don’t already know about, too.
For most reporters, the qualifications have more to do with speaking and writing well than knowing anything about a specific topic. (Having good hair helps, too.) This makes sense when you realize the reporter was covering a chemical spill yesterday and a lawsuit over cell phone towers today. What are the odds that reporter is an expert in chemistry, telecommunications and constitutional law?
That’s what I say when I’m being nice to them. The other problem is that news organizations care more about ratings than accuracy. A flashy, controversial story that takes 30 minutes to write, with a juicy headline, always beats a well-researched analysis of a complex subject that takes 2 weeks to write and most people won’t read anyway.
Put it all together and what do you get?
Good science is just about the most boring stuff in the world to read about. Even the people doing the work will admit how tedious it is. I know a guy who was on his way to a Ph.D. in biology when he started having panic attacks from the sheer mindless boredom of it.
But reading the abstract from a study, ignoring the paragraph about limitations in the design, skimming the statistical analysis (because really, who takes math when you’re majoring in communications?), misinterpreting the conclusions, and wrapping it up with a doom-and-gloom headline? Well hell, anyone can do that!
Here’s an example.
CNN published an article based on a study analyzing coronary plaque buildup and egg consumption.
Study title: Egg yolk consumption and carotid plaque
Story title: Is eating egg yolks as bad as smoking?
And we’re off to a good start. Shouldn’t the news be answering the question instead of asking it? If the reporter — or editor — believes the answer is settled, then the headline should report that answer. Unless … this is really just a press release. How can we tell?
The first sign is that nearly the entire “article” is quoting from the study or its author. There is a single quote near the end from an expert, but the quote refers to basic dietary guidelines and has nothing at all to do with the study.
Ignoring the fact that this is basically a press release, there’s also the problem of reporters not understanding statistics. Most important — because this is the one they virtually always ignore when reporting on nutrition — is the difference between causation and association.
- causation
- Matches cause fires.
- association
- Fire alarms are associated with fires.
In other words, even though you frequently hear alarms when there are fires, it would be wrong to say the alarms cause the fires. But that’s exactly what CNN does when they say in the photo caption:
Eating egg yolks regularly increases plaque buildup about two-thirds as much as smoking does, according to a new study.
No. What the study said was that carotid plaque area is higher in people who report eating the most eggs. That’s association. The study says clearly in the interpretation section:
This hypothesis should be tested in a prospective study …
That’s the study’s authors saying that this type of study, where they asked people after-the-fact what they had done and try find associations, is only good for identifying interesting questions. To show that doing this causes that, you have to start with at least two groups of test subjects, and make one of them change something — eat more eggs, eat fewer eggs — and see what happens. This is called a randomized controlled trial or RCT.
The problem with RCTs is that they take years to conduct, they’re incredibly expensive, and unless someone has thought of a way to make a ton of money off the results, no one has any incentive to spend that time and money.
So you can never trust what you read online, or see on TV?
Let’s see, how can I put this … Where have you been for the last 40 years?!
Photo by Matthias Weinberger
When I was 16 my father taught me to ride a motorcycle. He told me, “Always act as though half the drivers out there don’t even see you, and the rest are trying to run you down.”
It works with nutrition reporting, too. Half the time the reporter doesn’t know the first thing about what he’s reporting on, and the rest of the time someone’s probably trying to sell you something.