My kids still believe everything I say just because I’m the one saying it. That’s pretty cool. It’s also cool that they both understood sarcasm by the time they were five, although that says more about me than about them.
But nobody else believes me just because I’m so awesome. I have to actually sound like I know what I’m talking about. You know, explain myself, make sense, stuff like that. Scientists, on the other hand, seem to get a free ride from most people. I think it has something to do with the white coats … people really seem to like those.
However …
Scientists can be wrong [Gasp!]
I’m a fan of science. After all, scientists brought us computers, air conditioning and hot air popcorn poppers. But they also brought us bloodletting, phrenology and eugenics, so maybe not all scientists are worth listening to.
The question is, how do we tell the difference?
It takes some practice, but you can usually get a feel for junk science by looking at three factors:
- Assumptions
- Methodology, and
- Agenda
Here’s an example from a CNN article discussing a comparison of various diets:
Researchers at the Harvard School of Public Health and the Pennington Biomedical Research Center put four popular diets — high carb, high fat, low-fat and high protein — to the test to see which of the regimens resulted in more weight-loss success. After two years of monitoring the participants, “all the diets were winners,†said study co-author Dr. Frank Sacks, a professor of cardiovascular disease prevention at the Harvard School of Public Health. “All produced weight loss and improvements in lipids, reduction in insulin.
“The key really is that it’s calories. It’s not the content of fat or carbohydrates, it’s just calories,†said Sacks. The findings are published in the latest edition of the New England Journal of Medicine.
Let’s start with the …
Faulty assumptions
Their definition of a “high fat†diet was one where 40% of all calories from from fat. Is that a fair definition?
There was a study in 1960s performed by John Yudkin’s group at the University of London (get your copy here) where fat accounted for nearly 61% of calories consumed. Leaving aside the results of the Yudkin study — which I’ll come back to — it’s clear that the recent Harvard study was really comparing low-fat to very-low-fat diets. A genuinely high fat diet was never even considered.
And remember that conclusion — “It’s not the content of fat or carbohydrates, it’s just calories” — I’ll be coming back to that, too.
Flawed methodology
If the goal is to compare different diets, you need some way to control what people are actually eating. But according to the CNN article:
Participants could attend individual sessions where dieticians educated them and group sessions where they discussed their experiences with one another. Those who had better attendance in the sessions had stronger weight-loss results. “These findings together point to behavioral factors rather than macronutrient metabolism as the main influences on weight loss,†according to the study.
That means study participants were assigned a diet to follow, but their compliance was a matter of individual effort. And diet subjects are notoriously bad at self-reporting what they ate.
Assuming this flawed methodology works anyway, the scientists properly concluded that behavior — how well did they stick to their diet — was a better predictor of results than which diet a participant was assigned. This study was unable to conclude anything about which diet was better.
A conclusion that the co-author completely contradicts in his summary … quoted in the beginning of the article … the part most likely to be read by most people.
Different agenda
Note who did the study: a professor of cardiovascular disease prevention. Weight loss was only one of several factors used to judge success of the diets: “All [diets] produced weight loss and improvements in lipids, reduction in insulin.â€
These three factors are believed by some researchers to affect heart health. But the study didn’t actually measure cardiac outcomes. To conclude anything about heart health from this study, you would have to accept the lipid hypothesis.
(By the way, I realize I’m linking to a ton of extra reading here. But there’s a lot of research that suggests the last several decades of official recommendations have been wrong. There’s a lot more that I could be linking if I wanted to.)
It all adds up to: just plain wrong
So if this study wasn’t really comparing high fat to low fat diets … if they didn’t control what people actually ate … if they weren’t even looking primarily for effects on weight … are there any studies that didn’t have these flaws? As a matter of fact, there are.
The Yudkin study, which I mentioned earlier, placed subjects on a high fat diet — 61% of calorie intake — and found that people spontaneously limited themselves to less than 1600 calories per day. They felt full and lost weight.
Decades earlier, in 1944, Ancel Keys performed a starvation study where subjects were placed on an extremely low fat diet — just over 17% of calories. The total calorie intake was nearly identical to the Yudkin study but in the starvation diet people became emaciated, obsessed with food, and in several cases developed significant psychological problems.
There’s a really good analysis of these studies at The Four Hour Workweek in The Science of Fat-Loss: Why a Calorie Isn’t Always a Calorie.
If you find this conclusion a little hard to believe, think about this: One rice cake is about 35 calories. One cubic inch of beef jerky, three or four strips, is 49 calories. Which snack would satisfy your hunger longer, two rice cakes or a small bag of beef jerky? The rice might fill your belly, but you’ll still be hungry.
Enough crying wolf
Food scientists, politicians and food industry spokesmen have told us fat is bad, vegetable shortening is good, cholesterol is bad, carbs are good, carbs are bad, some cholesterol is good, vegetable shortening is bad, some fat is good … And this time they’re sure.
I haven’t seen any study that makes a definitive conclusion about a single nutrient. I haven’t seen any study showing that a radical new diet is better for the general population than any of the traditional diets. Until I do, I’m going to keep eating beef, bacon fat, butter, whole milk … and fresh fruits and vegetables, and home-made bread and pasta. And writing about it right here.
[private guest 2|guest 3|guest 4|guest 5|guest 6|guest 7|guest 8|guest 9|guest 10]
Part 4: Are you steam powered?
[/private]