Every other week, it seems, another study is published touting the benefits of mindfulness. The power of learning to be here now has been linked to mental-health improvements for just about everyone, with specific studies finding evidence for benefits in preschoolers, breast-cancer survivors, and parents of children with autism, to name just a few. In 2016 alone, 330 peer-reviewed studies on mindfulness have so far been published, and the National Institute of Mental Health has in recent years invested in research projects aimed at identifying ways of pairing the practice with treatment for mental illness.
And so it seems about time to ask — how much do we really know about mindfulness, anyway? Recently, a team of McGill University scientists attempted to investigate this question, with an analysis of 124 studies on mindfulness as a means of improving mental health. Their results show that many of these studies contained sample sizes that are too small to provide meaningful results — and they suggest that studies on mindfulness that have turned out negative results may have not been published.
Reporting in Nature, Anna Nowogrodzki explains the analysis, which was published earlier this month in PLOS ONE:
For the 124 trials, the researchers calculated the probability that a trial with that sample size could detect the result reported. Experiments with smaller sample sizes are more affected by chance and thus worse at detecting statistically significant positive results. The scientists’ calculations suggested that 66 of 124 trials would have positive results. Instead, 108 trials had positive results.
She notes that in addition to these 124 studies, the researchers also eyed another 21 registered trials, 62 percent of which were never published more than a year after they ended, which “hint[s] that negative results are going unpublished.”
They’re not saying, by the way, that mindfulness should be dismissed, or that it isn’t a potentially powerful way of helping people improve their mental health. In a way, they’re arguing for scientists to take these experiments more seriously, so that high hopes or expectations about the practice do not get in the way of rigorous scientific work. The researchers call for bigger, better studies of mindfulness, and argue that the negative results need to be published, too. As McGill psychologist Brett Thombs told Nowogrodzki, “For the health-care system, it’s just as important to know what doesn’t work.”
