How to determine the credibility of a hair supplement study?

The hype around the hair loss industry

There is an insane amount of hype in many health product industries, whether it be weight loss aids, acne removal and of course hair loss. All the time, we encounter clickbait headlines like "Oranges hold the cure to hair loss" in the media. Most of these articles are based around conclusions that the journalists have drawn from latest scientific research. There are several problems with this.

• The latest research is not always correct. 
  
  
  • New research means treading into new territory. Scientists make educated guesses, which the research is consistent with. It doesn't mean the guess is always correct.
  • Some science is badly done.
• Many journalists are not scientists
  
  
  • When the read scientific literature, they don't always accurately assess the credibility of the article.
  • Journalists very often blow things out of proportion in order to attract readers.

In this article, I plan to point out three things you can look out for when reading a scientific study about hair loss.

Things to look for in hair loss articles

in vitro and in vivo

In vitro and in vivo are two commonly used scientific terms, and it is one of the crucial factors in determining how reliable the results of an experiment is. In vitro are so called "test tube experiments". It is where cells and molecules are studied outside of their natural context. The main weakness of in vitro experiments is that they don't precisely replicate the cellular conditions of the organism. Consequently, results of in vitro studies may not apply to the living organism.

For example, consider the published scientific paper “Procyanidin B-2, extracted from apples, promotes hairgrowth: a laboratory study.” This paper was published in 2002 in the British Journal of Dermatology. In this paper, it was shown that adding certain inhibitors that stop an enzyme called protein kinase A to the a mouse hair epithelial cell culture change the activity of some of its genes. The gene changes were associates with hair growth.

The main limitation of this study is that it was that these compounds were directly added to cells in culture. The compounds may not work the same way when applied directly to a human scalp or when taken orally. As a result, the result of this study by itself is not enough to show that procyanidin B2 can treat hair loss.

In vivo (a latin phrase for “within the living”) refers to experiments done using a complete, living organism. Animal studies and clinical trials are two forms of in vivo research. In vivo experiments is often preferred to in vitro as the overall effects/results of an experiment tends to be more applicable to the living subject.

Consider the scientific study “Investigation oftopical application of procyanidin B-2 from apple to identify itspotential use as a hair growing agent.”, published in the Journal Phytomedicine in the year 2000. In this experiment, procyanidin B2 was applied to the heads of 19 men. A corresponding placebo group had 10 men. After treatment, the men treated with procyanidin B2 had increased hair growth and hair diameter.

This is an example of an in vivo study. It should be easy to see that it provides much stronger evidence that procyanidin B2 can be used to treat hair loss. In particular, the study demonstrates the the compound can work in humans.

Animal studies vs human trials

The next major guideline I would suggest is to consider whether the study uses animals or human. Returning to our previous example - "Procyanidin B-2, extracted from apples, promotes hair growth: a laboratory study". This study uses hair epithelial cells from mice. Results obtained using hair epithelial cells from humans would be more compelling.

Even after animal trials suggest that a drug or compound will be safe and effective in humans, greater than 80% of potential therapeutics fail in clinical trials. Part of the problem lies in that any symptoms and conditions that scientists treat in mice is simply just a model for a human disease. Thus, not all treatments translate across from animals to humans.

Reproducibility

Reproducible science is reliable science, in most cases. This is particularly true if the results can be reproduced by independent laboratories. Science that is reproducible means that if the experiment is repeated as is describe, the results that result will be consistent. Recently, there was the STAP cell scandal, where Japanese researcher Haruko Obokata from RIKEN claimed to have devised a very simple method of generating stem cells. Armed with the instructions, several labs around the world set out to reproduce this miracle but failed. It later turned out that there were many errors in the manuscript and some of the images even turned out to be doctored.

Sample size 

There is a rule of thumb "The larger the sample, the more reliable the conclusion". This only holds true when statistical analysis is conducted in a rigorous manner. But assuming the statistical calculations are done correctly, what would be a sufficient sample size? This is one of those "how long is a piece of string" questions. But, as a rule of thumb for human trials, definitely be wary of sample sizes under 10.