Why Most Supplement Studies Are Too Small to Trust

A trial of 20 people per arm has roughly a 16% chance of detecting a real, modest effect. Not a 16% chance of being wrong. A 16% chance of finding something that is genuinely there.

That number is statistical power, and it is the single most useful thing you can work out about a supplement study. Almost nobody reports it. Almost every supplement trial you have read about fails it.

The arithmetic is unforgiving

Sample size requirements are not a matter of opinion. They fall out of the effect you are looking for.

Effects are usually expressed in standard deviations. A 0.8 SD difference is large — you would notice it without a study. A 0.5 SD difference is medium. Around 0.2 to 0.3 SD is where nutrition and supplement effects actually live, and it is small enough that individual variation swamps it.

Here is what each of those costs you in participants.

So a 350-person trial can reliably detect a 0.3 SD change. A 40-person trial cannot. Run that 40-person trial anyway and one of two things happens: you find nothing and call it inconclusive, or you find something and it is far bigger than the truth.

The second outcome is the dangerous one.

Half of biomedical studies run below 20% power

Button and colleagues estimated the median statistical power across 49 neuroscience meta-analyses at 21%. A later reanalysis by Nord and colleagues showed that figure hides real variation — some subfields do fine, candidate gene studies are catastrophic — but nobody disputed the direction.

Dumas-Mallet's team then reviewed meta-analyses across neurological, psychiatric and somatic research. About half of the individual studies had power between 0% and 20%. The conventional minimum is 80%.

The closest evidence to your training life is more recent and no better. Bleakley and colleagues examined 266 randomised trials of conservative treatment for musculoskeletal injury. Median power was 42%. Not one of the 266 had 80% power to detect a small effect, and only 7.9% could detect a medium one.

One detail from that review is worth keeping. Of the quarter of trials that reported a sample size calculation, 80% had expected a larger effect than they went on to observe.

Small trials do not just miss effects — they inflate them

This is the part that is genuinely counterintuitive, and it is why sample size matters more than the p-value beside it.

A small trial only crosses the significance threshold when random noise happens to push its result high. Weak-but-real results stay invisible. So the significant findings that survive from small trials are systematically the exaggerated ones. Ioannidis called this the winner's curse: newly discovered true effects are usually inflated at the moment of discovery.

You can measure the inflation. Dechartres and colleagues took 93 meta-analyses containing 735 randomised trials and sorted the trials inside each one by size.

Nüesch's group found the same pattern in osteoarthritis, where supplements are heavily marketed. Across 13 meta-analyses covering 153 trials and 41,605 patients, small trials reported effects 0.21 SD more favourable than large ones. In six of those 13 analyses, a pooled result that looked like a clinically meaningful benefit shrank to a non-significant one once the analysis was restricted to trials of at least 100 patients per arm.

Same molecules. Same outcome measures. Different answer, purely as a function of who counted.

What happens when someone finally runs the big trial

The supplement literature has a recognisable shape. Promising small studies, a decade of confident marketing, then one properly sized trial that ends the argument.

Glucosamine and chondroitin are the cleanest example. Wandel and colleagues restricted their network meta-analysis to trials of more than 200 patients — 10 trials, 3,803 people. The pooled difference in pain was 0.4 cm on a 10 cm scale for glucosamine and 0.5 cm for the combination. They had set 0.9 cm as the smallest difference a patient would notice. Nothing came close.

Vitamin E went the same way, only worse. Miller's meta-analysis pooled 19 trials and 135,967 participants and found that doses of 400 IU a day or more were associated with 39 extra deaths per 10,000 people. SELECT then randomised 35,533 men to selenium, vitamin E, both or neither, and found no reduction in prostate cancer.

Omega-3 is the one closest to home for us. The 2020 Cochrane review pooled 86 randomised trials and 162,796 participants. Long-chain omega-3 made little or no difference to all-cause mortality (RR 0.97, 95% CI 0.93 to 1.01). It may slightly reduce coronary heart disease events, with 167 people needing to take it for one to benefit.

None of that stopped the small positive studies from being published first, or from being quoted for years afterwards.

Most meta-analyses are built entirely out of small trials

The usual defence is that pooling fixes it. Sometimes it does. Often there is nothing solid to pool.

Turner and colleagues went through 14,886 meta-analyses drawn from 1,991 Cochrane reviews and calculated the power of every study inside them. In 70% of those meta-analyses, every single included trial was underpowered. Only 17% contained at least two adequately powered trials.

A meta-analysis of twelve 30-person trials is not equivalent to one 360-person trial. It inherits every bias in its inputs, including the publication bias that kept the null 30-person trials in a drawer.

Read the forest plot, not the summary line. If the biggest trial in it has 60 people, the pooled estimate is a tidier version of the same guess.

Four checks, thirty seconds

You do not need to run the power calculation yourself. Four questions get you most of the way.

How many per arm, not in total? A "60-participant study" is 30 per group. Against a subjective outcome, that is noise.

What was the pre-specified primary outcome? One outcome, named before the trial started. Everything else is exploratory, whatever the abstract implies.

How wide is the confidence interval? A small trial reporting a 25% improvement with an interval running from 2% to 51% has not measured 25%. It has measured "somewhere between trivial and enormous".

Who paid, and how big were the others? Industry-funded trials in the Wandel analysis reported larger effects than independent ones (p=0.02 for the interaction).

One caveat, because the rule is not universal. Small trials are fine for outcomes that move a long way and vary little between people — blood NAD+ concentration, plasma DHA, a pharmacokinetic curve. Twenty people are plenty to show that an oral dose raises a blood marker. Twenty people are useless for showing it improves your sleep.

Match the sample size to the noisiness of the outcome, and most of the confusion clears.

Why supplement trials stay small

Nobody is hiding this. The economics simply do not support anything bigger.

A supplement ingredient is rarely patentable, so a company that spends £5 million on a definitive trial hands the result to every competitor. Drug companies recoup that cost through exclusivity. Supplement companies cannot, so trials get funded at the scale a marketing budget will bear: 30 people, 12 weeks, one biomarker.

That is a structural problem, not a conspiracy. It does mean the evidence base under most of the category is thin, and it will stay thin.

The honest response is to say so, and to price your confidence accordingly.

What this means for NMN, and for us

We sell NMN. So apply the rule here first.

The pooled human evidence for NMN is eight randomised trials in 342 middle-aged and older adults, at 250–2000 mg a day for 14 days to 12 weeks. Chen and colleagues found no significant effect on fasting glucose, fasting insulin, HbA1c, insulin resistance or lipids. Eight small trials, one null answer.

That is the current state of it. Individual trials look better — some clearly do — but the ones that look best are also the smallest, which is exactly the pattern this article has been describing.

What the small trials establish well is the pharmacokinetic part: oral NMN raises blood NAD+, reliably, at doses people actually take. That is a low-noise outcome measured in a low-noise way. What they do not establish is that raising it changes anything you would notice.

If you want the same treatment applied to the statistics rather than the sample size, read what a p-value actually tells you. For a worked example of an entire literature reassessed once the large trials arrived, see our review of omega-3 and mood. And if you want to check what we put on a label against what is in the capsule, the NMN + Resveratrol page carries the batch certificate.

Ask any seller how many people were in the trial. Then ask how many were in the biggest one. The gap between those two numbers tells you most of what you need.

References

1. Dechartres A, Trinquart L, Boutron I, Ravaud P (2013). "Influence of trial sample size on treatment effect estimates: meta-epidemiological study." BMJ, 346, f2304. PMID 23616031. doi:10.1136/bmj.f2304 PubMed ↗
2. Turner RM, Bird SM, Higgins JP (2013). "The impact of study size on meta-analyses: examination of underpowered studies in Cochrane reviews." PLoS ONE, 8(3), e59202. PMID 23544056. doi:10.1371/journal.pone.0059202 PubMed ↗
3. Nüesch E, Trelle S, Reichenbach S, et al. (2010). "Small study effects in meta-analyses of osteoarthritis trials: meta-epidemiological study." BMJ, 341, c3515. PMID 20639294. doi:10.1136/bmj.c3515 PubMed ↗
4. Button KS, Ioannidis JP, Mokrysz C, et al. (2013). "Power failure: why small sample size undermines the reliability of neuroscience." Nature Reviews Neuroscience, 14(5), 365-376. PMID 23571845. doi:10.1038/nrn3475 PubMed ↗
5. Nord CL, Valton V, Wood J, Roiser JP (2017). "Power-up: A Reanalysis of 'Power Failure' in Neuroscience Using Mixture Modeling." Journal of Neuroscience, 37(34), 8051-8061. PMID 28706080. doi:10.1523/JNEUROSCI.3592-16.2017 PubMed ↗
6. Dumas-Mallet E, Button KS, Boraud T, Gonon F, Munafò MR (2017). "Low statistical power in biomedical science: a review of three human research domains." Royal Society Open Science, 4(2), 160254. PMID 28386409. doi:10.1098/rsos.160254 PubMed ↗
7. Bleakley C, Klempel N, Wagemans J, Netterström-Wedin F, Smoliga J (2025). "Statistical Power in Musculoskeletal Research: A Meta-Review of 266 Randomised Controlled Trials." Sports Medicine - Open, 11(1), 134. PMID 41269466. doi:10.1186/s40798-025-00908-8 PubMed ↗
8. Ioannidis JP (2008). "Why most discovered true associations are inflated." Epidemiology, 19(5), 640-648. PMID 18633328. doi:10.1097/EDE.0b013e31818131e7 PubMed ↗
9. Wandel S, Jüni P, Tendal B, et al. (2010). "Effects of glucosamine, chondroitin, or placebo in patients with osteoarthritis of hip or knee: network meta-analysis." BMJ, 341, c4675. PMID 20847017. doi:10.1136/bmj.c4675 PubMed ↗
10. Miller ER 3rd, Pastor-Barriuso R, Dalal D, Riemersma RA, Appel LJ, Guallar E (2005). "Meta-analysis: high-dosage vitamin E supplementation may increase all-cause mortality." Annals of Internal Medicine, 142(1), 37-46. PMID 15537682. doi:10.7326/0003-4819-142-1-200501040-00110 PubMed ↗
11. Lippman SM, Klein EA, Goodman PJ, et al. (2009). "Effect of selenium and vitamin E on risk of prostate cancer and other cancers: the Selenium and Vitamin E Cancer Prevention Trial (SELECT)." JAMA, 301(1), 39-51. PMID 19066370. doi:10.1001/jama.2008.864 PubMed ↗
12. Abdelhamid AS, Brown TJ, Brainard JS, et al. (2020). "Omega-3 fatty acids for the primary and secondary prevention of cardiovascular disease." Cochrane Database of Systematic Reviews, 3, CD003177. PMID 32114706. doi:10.1002/14651858.CD003177.pub5 PubMed ↗
13. Chen F, Zhou D, Kong APS, et al. (2025). "Effects of Nicotinamide Mononucleotide on Glucose and Lipid Metabolism in Adults: A Systematic Review and Meta-analysis of Randomised Controlled Trials." Current Diabetes Reports, 25, article 4. PMID 39531138. doi:10.1007/s11892-024-01557-z PubMed ↗

Judge Our Evidence by the Same Rule

The largest pooled analysis of NMN in humans covers eight trials and 342 adults, and it found nothing on glucose, insulin or lipids. We publish that alongside the trials that looked better. Our NMN + Resveratrol lists 500 mg of NMN and 600 mg of trans-resveratrol per serving, with a batch certificate of analysis you can read before you buy.

See the Label and the CoA →