It seems like every other day there is a headline about our diet or health.
“Saturated fats linked to heart disease”, “High sugar diet linked to heart disease”, “Coffee cuts liver cancer risk”, “Coffee is linked to high blood pressure” and on it goes. One seems to contradict the next.
Anybody else out there confused and wondering what the heck to believe or not believe?
When you read these two words do you instantly believe what follows? “Studies show…”
Well, I’ve got news for you—even if it comes straight out of your physician’s mouth or right out of a news article, it isn’t always true. How do you separate fact from fiction?
There is a difference between ‘good’ studies and ‘bad’ studies.
When I was in nursing school at a major medical research university, one of my professors told us that only a very small percentage of scientific studies (less than 10%) actually made it to mainstream media and were published.
Many of the ‘studies’ we read about have no basis in fact or scientifically gathered data—Really!
The majority of the so-called studies that make it to the media are actually flat out wrong, misinterpreted or totally skewed.
I know, you’re probably thinking, “how can that be?” It is SCIENCE, after all.
Keep this in mind as I go on…
‘Studies’ that we read about in the news can be one of two things:
1. Observational studies–which is just that–purely an observation of a particular situation, and some sort of conclusion is drawn based on that observation. There is no intervention or treatment, just observations and gathering of data.
2. A true clinical or scientific study—which follows these specific steps:
• An observation is made
• A hypothesis is formed as a possible explanation
• Conduct experiments
• Collect data
• Reach a conclusion
• The results must be CONSISTENT and REPEATABLE
In both types of studies—non-scientific observational studies, or true scientific studies, we look for results that show associations or correlations. As in ‘A causes B’, or ‘smoking is associated with lung cancer’.
Since our early days as cavemen, our brains have been hardwired to seek patterns. If A occurs, then B results. It was how we survived.
However, some of those ‘A causes B’ were actually just coincidence. For example, after a long drought, the dehydrated and desperate natives dance around wildly, and it starts raining. Did the dance cause the rain, or could it be a coincidence?
Our brains are still looking for the connection. And we tend to generate beliefs consistent to what we already believe.
The problem with purely observational studies, though, is that ‘A’ does not always cause ‘B’ to happen.
These observations can be incredibly inaccurate, and many other factors that may affect the outcome.
Other factors are called ‘confounding’ variables. For instance: It was thought that reading to young children helps them become better readers and get good grades in school. ‘A causes B’, right? Hmmm…
Consider this–parents who read to their children generally have higher IQ’s and better educations, so their children get higher grades in school anyway. There's the confounding variable.
Observational studies only make observations and form an untested guess as to any cause and effect. Not rocket science by any means.
Did you realize that most of the dietary and health headlines we read in the paper are based on observational studies?
Not very accurate, as we now know.
Did you know most of the health and dietary information our PHYSICIANS are giving us are based on purely observational studies?
The gold standard for forming any real scientific conclusions is a true scientific clinical study.
The following is a great example of an observational study versus a clinical study:
In an observational study on middle-aged Harvard nurses taking supplemental estrogen, it was observed that these nurses had 40% less heart disease. So of course, every physician was ready to start prescribing estrogen!
A clinical (real scientific) study on estrogen and middle-aged women was conducted. 16,000 women were studied for 5 years, and results were women taking estrogen had a 30% HIGHER risk of heart disease than the control group.
How did one study show the opposite effect of the other study?
Confounding variables in the observational study, that’s how. Turns out, in the first observational study on the nurses, the nurses had a healthier lifestyle than the general public, so they had a lower risk of heart disease in spite of taking estrogen.
But, even clinical scientific studies can be highly inaccurate.
Scientists skew data and the conclusions depending on the outcome THEY are looking for.
John Ioannidis is one of the world’s foremost experts who investigates the credibility of medical research. He has shown over and over again that the majority of what biomedical scientists conclude in published studies is not only misleading, but exaggerated, and often totally wrong.
These are the studies that doctors use to prescribe antibiotics, blood pressure medication or advise us on diet or lifestyle.
This expert who has made it his life’s work to follow up on medical studies, states that…
“as much as 90% of the published medical information that doctors rely on (to pass onto us) is flawed…"
"...when it comes to cancer, heart disease, and other common ailments, there is plenty of published research but much of it is remarkably unscientific based largely on observation.”
And, 20-25% of conclusions from actual scientific, clinical studies are wrong as well.
While we would all like to think of the scientific process as objective and unbiased, it’s very easy to manipulate results—intentionally, unintentionally or even unconsciously.
So next time you read the a ‘scientific’ study, ask yourself these questions:
1. Is it an observational study or a scientific or clinical study?
Remember, an observational study is only an observation between two things but may have other factors that affect the outcome. If A and B are associated, could it actually be because of C? Or C, D and E?
2. Did A cause B or did B cause A?
Consider this simple observation: Running makes you slender. Ok, then does basketball make you taller? Or, is it that good runners are more slender to begin with, like good basketball players are generally taller than average?
Or how about this: Low carb diets can make you fatter. Can it be because most of the people on a low carb diet are trying to lose weight?
3. Are results consistent every time?
Remember a true scientific conclusion must have results that are consistent and repeatable.
Like this: Saturated fat has been linked to heart disease.
Except for these groups: The Swiss, the French, the Inuits, the Masai, the Spanish and other ‘paradox’ populations.
Or: Red meat is linked to colon cancer–except for another ‘study’ showing vegetarians more likely to get colon cancer.
Neither of these are consistent or repeatable results.
4. Ask yourself who/what are the real subjects of the study? Do they translate to the general population?
How then can you translate any of the study results from say, mice to humans? We have to realize that many tests are performed on animals being fed a very unnatural diet. Although it seems obvious that mice don’t eat dairy products or saturated fats in their natural diet, the results are still translated to people.
5. What is a scientific ‘Difference’?
In science, absolute change or results come from subtraction. But, if scientists don’t see a big difference this way, they take the results and use division to come up with a more impressive number.
Here is a real example: Lipitor drug trials used two groups of men who were at high risk for heart disease. Lipitor was given to one of these groups for ten years. At the end of the study, 2 men out of the 100 in the Lipitor group had heart attacks. And, 3.05 out of the 100 men in the control group had heart attacks.
Does 1 out of 100 sound significant? NO!
The difference is 1 less heart attack out of 100 men in 10 years. Obviously to most that’s not an impressive number.
But, when you look for relative change, (2.0 divided by 3.05=0.64 and 1.0 minus 0.64=0.36); you get 36%—a far more impressive looking number! So guess what is plastered all over the ads? Lipitor reduces the risk of a heart attack by 36%! Sounds very impressive doesn't it?
What about ‘significant’? To most of us, significant means impressive, major, meaningful, or important. Significant to a scientist means, ‘not due to chance.’ That’s it.
As in this study: Two groups of people were put on the Mediterranean diet with one group adding large amounts of salt to their food. The other group ate the same diet, but low sodium. What was the difference in blood pressure of the two groups? Scientists called it ‘significant’. I would not.
BP in the high salt diet group was 126/81. BP in the low sodium group was 123/79.
6. Did scientists control the variables?
Remember this recent headline? “Red meat causes cancer”. Really? Did you know that red meat was grouped in with processed lunchmeats, pizzas, and hotdogs? There was no connection found with cancer and red meat by itself.
7. Compared to what?
“Whole grain foods prevent diabetes”. Really? Compared to what? Compared to white processed flour, yes.
Compared to a diet with no grains? No. Whole grains do not prevent diabetes.
Another example: Filtered cigarette smokers have a lower rate of cancer. Compared to unfiltered cigarettes, true. Compared to no cigarettes, false!
8. Do the real results really support the conclusion?
In another (real) study of low carb/high fat diets and high carb/low fat diets, the low carb group lost more body fat and had the biggest improvements in all cardiovascular markers including cholesterol, triglycerides, HDL, and LDL.
The conclusion that scientists published was, “Moderate approaches such as a moderate carbohydrate, low fat diet may be prudent.” Could it be because this study was actually funded by Kellogg’s?
We cannot trust the media, the medical community, and scientists to give us the unbiased truth unfortunately. We just cannot faithfully believe or every so-called ‘study’ we read about, until we dig a little deeper, even if it comes from our own doctors. Apply critical thinking the next time you hear about a 'study' in the news or read one in the headlines. It just may not be true.
We must apply critical thinking to what we read and hear, and go after the real truth ourselves.
Till next time, stay healthy and lean!
“Science for Smart People”, Tom Naughton, YouTube video.
David Freedman, Lies, Damned Lies, and Medical Science, Atlantic Monthly, November 2010.
Catherine (Cat) Ebeling RN BSN,is a back to basics diet and nutrition specialist. In addition to her advanced degree in nursing from a major medical school, she has spent the last 30 years intensely studying diet, health and nutrition. She also has a book titled “The Fat Burning Kitchen, Your 24 Hour Diet Transformation” that has sold over 60,000 copies worldwide, and has helped thousands of people transform their lives, lose weight and improve their health.
1 thought on “Scientific ‘Studies’ or Scientific Lies?”
Preach on, sister!! It seems like every other day, one thing is healthy for us and the next day it's being declared cancerous… I am extremely skeptical of many scientific studies. You're right–you have to consider all the variables and make an educated decision on how true it really might be.