clever_plant
New member
I just had to chime in here since this is right up my alley.
In general, if the study is published in a reputable journal, the study itself should not be doubted. I mean, I guess some people just pull numbers out of their ass but most studies come from universities or labs and has researchers names attached to them and looking up the credentials, if you are in doubt, should be easy peasy. The best way to judge a study, beyond actually understanding research methodology, statistical sciences etc. is to look at where the study is published. The more reputable the peer reviewed journal the better.
When you then have a study in your hands there are a couple of basic things to look for that can immediately tell you whether or not the study is worth paying attention to.
1: Is it a real study or a pilot study. Pilot studies are very small scale studies done to small test groups to sort of "test the waters" and try to gather semi anecdotal evidence in support of doing a full scale study, they can provide interesting info but I'd be hesitant to base actual advice on them
2: Does the study have a control group. This means that if you want to test if eating bananas is healthy, you don't just measure 10 peoples vitals while feeding them bananas, you measure 20 people, 10 of which are fed banans and 10 of which arent, and try to keep the rest of their lives the same, and see if the two groups express any significant differences in vitals.
3: Is the study groups large enough to actually display statistical significance. This is about avoiding the classic grade school "statistical research" where you ask two people if they like chocolate ice cream, they both answer no and you then declare in your paper "100% of people dislike chocolate ice cream". The group of people, animals or whatever studied needs to be big enough to be statistically significant. What "statistically significant" is, is a science unto itself but in general the bigger the groups the better the data.
4: Is the study double blinded. Now with regard to fitness and diet this is sometimes impossible to achieve but it should, whenever possible, be adhered to. A double blinded study means that neither the researcher, nor the test subjects, know which group they are in, the control group or the test group. I.E. Neither the doctor knows whether you are given the sugar pill or the pill with the medicine. If it is at all possible this is ESSENTIAL for the quality of a study. It has been proven over and over that even the most stringent, neutral, professional etc. researchers are affected by selection bias when they know which group test subject belongs to.
5: Does the conclusion / extrapolation in the study go beyond what the data actually suggests. For instance you might see a LOT of studies about homeopathy that goes something like: a non-controlled unblinded study of 5 people shows a 1% decrease in blood pressure after taking a homeopathic drug. And the conclusion is "ALL HOMEOPATHIC MEDICINE WORKS!" This is concluding / extrapolating beyond the merits of the study. a more reasonable conclusion to the study would be "This might be an indication that this specific homeopathic remedy could have a small effect on blood pressure, larger scale double blinded testing might be worthwhile"
In general, actual scientists usually qualify their statements a lot with "Might, suggests, could" etc. and people trying to sell you crap make a lot of ultimate statements like "definately, 100% guaranteed" etc.
When it comes to taking advice from people, ask to see sources, I have read through steves advice, pondered it, then I read lyle's stuff, went through the sources, looked up studies myself on pubmed and ended up thinking "ok, steves advice is generally well founded in actual research" but check for yourself when people give you advice
Granted I am able to understand ebil chemistry and nasty science due to my education but I'd say that even most lay people can read a lot of the studies on pubmed and pubcrawler and get a sense of whether or not it is utter crap. If not, you have to make some desicion on who to trust, I'd say go with people that qualify their advice with "to the best of my knowledge", "current science indicates" and such things. And avoid people that are trying to sell you something, and people that make ultimate statements, like the plague
edit: a little spelling.
