How NOT to read a scientific report

This gem of a headline turned up in my social media feeds today: Clinical trial reaffirms diet beverages play positive role in weight loss! 

OOOOK. GREAT example of a sensationalist headline telling people what they want to hear. Don’t get me wrong— I am not suggesting that the science wasn’t done properly. I’m talking about the way the study is presented in the lay press and the misinterpretation that could follow. Let’s unpack this a little, shall we?  

 

a). How long was this study? 12 weeks. 12 weeks! That’s 3 months! In studying the effects of an intervention on weight, is 3 months a clinically relavent end-point? I’m pretty sure that LONG-TERM weight loss is both harder to make happen and more important.

b). Who funded the study? The American Beverage Association. Gotcha! While it may be properly designed to show what it purports to show, the entire setup was designed to produce a specific answer. And the participants got coupons from Coke, Pepsi, and Snapple.

c). The headline specifies the role in weight loss— not other health benefits— because that was what the study measured. It’s likely that some readers will interpret this as “diet soda is good for you!”. 

d). In terms of the design. . . This is an equivilance trial, trying to show that one is as good as another— but due to the nature of the subject (what kind of beverage you are allowed to drink), there’s no way to blind the participants. They knew. It’s not a fatal flaw, but it doesn’t help things. It’s also part of a bigger study looking at a weight-loss program, so we can’t take the results out of that context. If people aren’t participating in a structured weight-loss program, they might not have the same  effect.

e). The sample was about 300 people, 82% female and 68% white. So white ladies, sure. Latino gentlemen, who knows??

 

Ok. Get the picture? It’s just not as simple as the headline makes it sound, and as we all know, people (myself included) tend to just read the headlines. It’s a facebook, buzzfeed world.

 

 

2 thoughts on “How NOT to read a scientific report

What's cookin, good lookin?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s