Why the Biggest Mindfulness Study Got it Wrong

The Problem

Teens have developed an unusual habit lately of killing themselves and each other. A CDC study of grades 9-12 in 2019 showed that more than 15% of the students had made a suicide plan. Horrifying. And terribly embarrassing. If we can’t stop our own children from killing themselves, is there anything we can do correctly? An increasingly dangerous technological future is going to require a greater level of capability as a species than this, so it’s critical we learn to stop making these kinds of mistakes.

The main issue for the recent spike in teenage mental health issues is essentially known: smartphones and the internet.1 Screen time for the average American teenager is now over four hours per day. But taking phones away from teenagers is hard, just as taking crack pipes away from junkies is hard. Even if we’re the ones responsible for their addictions in the first place, it’s hard to unscramble an egg.

Even if phones are obviously the main problem, people have started investigating other solutions to the teen mental he‘alth problem that don’t involve taking away that good-good. One is meditation. Specifically mindfulness meditation, though I expect the scientists doing the research don’t really understand that there are other kinds of meditation besides mindfulness, and most think “mindfulness” is just another word for meditation, in the same way the word Sahara means desert, so Sahara Desert means Desert Desert. These kinds of misunderstandings happen when someone comes into an area they don’t understand and immediately start naming things.

There are a bunch of studies showing meditation is a very effective treatment for depression and anxiety, and that meditation causes all these brain changes that lead to greater wellbeing. Why not throw that at teenagers and see what happens? Maybe it’ll loosen the grip of that magic internet sauce.

The MYRIAD Project

That’s exactly what the scientists behind the MYRIAD trial were thinking. Let’s round up tens of thousands of students from 100 schools and teach them to meditate. Then, we’ll see what happens. They packaged the meditation instruction with psychoeducation in a package called SBMT (school-based mindfulness training). This was perhaps the largest meditation study ever.2

The study showed that, one year later, the students who took the SBMT program were no better off than their peers. The study also found that the students were actually higher in panic disorder and OCD scores and worse-off in other ways. The scientists explain this as being a result of the fact that “mindfulness training involves asking people to become more aware of thoughts and feelings, including unpleasant ones. It is possible in this setting, with this curriculum and these teachers, this can exacerbate difficulties, at least for some students.” (I don’t think I’ve ever read a more hedged, qualified statement, by the way.) They also say that “at the student level variation in mental health status and age/developmental stage, and trajectories of change need consideration”. The media reported on this study, talking about how “minfulness in schools does not improve mental health”, and how it can actually be harmful for some students. So maybe this meditation thing isn’t so good after all?

The MYRIAD Project Was a Waste of Money

Personally, I can’t help but feel like the media and these scientists are simply full of shit. Now, I’m no experienced meditator. I’ve been doing it regularly for less than two months. Enough to see very minor changes, but not nearly as long as most meditators will tell you is necessary to see results. So, I’m no expert. But I do know that in the Eastern tradition, when you learn meditation, it takes many months or even years to see real changes in your mind. You’re also supposed to learn it from someone who is an expert on the subject. Someone who’s been doing it for several hours a day for decades and has presumably reached enlightenment, whatever that means. Do I believe in meditation? I’m not sure yet. I’m open-minded. But I can see plain as day that what the scientists of this study investigated was not real meditation.

Firstly, they only gave the students ten lessons total. And the lessons were mostly psychoeducation PowerPoints, with brief mindfulness meditations sprinkled in. The students were supposed to do at-home meditation as well, but they didn’t really. Students self-reported an average of 1.16 out of 5 for how much they did the home practice, i.e. they didn’t do it. So it was mostly just the ten lessons.

Ten lessons. I’ve done dozens of half-hour meditation sessions and I’m still firmly in stage 2 of the 10 stages outlined in the book The Mind Illuminated. Stage 2 just means I still get lost in thought and forget I’m supposed to be meditating. And stage 1 is just “establish a regular practice”. So yeah, I’m a newbie. Maybe if meditation is bullshit, I’ll be a newbie forever, because the stages 3-10 don’t even exist. Or I won’t. Either way, ten sessions is not long enough to find out. The authors of the study do admit their study “may simply not be intensive enough to create” the changes they desired, which I think is putting it mildly.

Worse, even the teachers had only 8 lessons in mindfulness and then a 4-day training course. So even the teachers were still firmly in the early stages of meditation practice. Yet they’re supposed to be teaching it? How can you teach what you don’t understand? Even I have had more meditation experience than these teachers. Should I open a temple and start teaching people meditation?

What Wouldn’t Have Been a Waste of Money

In the Eastern traditions you’re supposed to be learning from a lifelong meditator – one of the people science has now shown have dramatic changes to their brains. Studies using MRI and fMRI have now shown experienced meditators have reduced activity in the default mode network, lower amygdala reactivity, greater cortical folding, increased thickness of the cortex, and increased volume in the hippocampus and prefrontal cortex. All from meditating for so long.

And in Buddhism, mindfulness isn’t taught by itself. It is taught alongside a second kind of meditation, concentration meditation, and there’s a whole ethical and philosophical framework that’s supposed to go along with it. Any of the “harms” discovered by this study — if they exist at all — could easily be the result of trying to teach meditation outside of this protective framework.

I’m not saying Buddhism is right about how to do meditation. Who knows. I’m saying that if it was your job to answer the question of whether meditation worked for teenagers, how would you do it?

Probably you would find out how meditation is supposed to work, according to the people who have been doing it for literally thousands of years. You would have teenagers do that, and see what happens. Maybe you’d find a meditation master and have them run an after-school program for teenagers based on what they thought would work best. Then you’d measure the teenagers’ mental health and see if it had improved.

Why didn’t that study happen? Why did we instead blow hundreds of thousands or millions of dollars on this huge-scale study that has almost no contact with the actual question of meditation and its efficacy? How can you do this bad a job at investigating something? Are we willing to accept any study that’s written up in sciencey language and follows proper sciencey procedures, even if it’s not really investigating anything?3

The Arrogance of Science

To be fair to the scientists, the study is actually totally valid. It does prove that this specific kind of mindfulness program is not helpful. If your school district is thinking about implementing a useless program like this, then this study can be used to shut that down. My problem is that they could have also included a condition in the study that tested real meditation alongside the bullshit program. But they chose not to, because they don’t know that the meditation they’re studying is not real meditation.

I think scientists just believe meditation is some kind of special breathing exercise or something. As a result, they end up studying what they call meditation without studying the real thing. So the study doesn’t match the actual thing we want to know about. It’s as if you wanted to prove that acupuncture doesn’t improve teenage mental health, so you took a photo of a needle and rubbed it on a teenager’s back. Of course that won’t work! But that’s also not what the acupuncture people are even saying might work! Real science must have more humility than this.

Of course, it might be the case that meditation needs to be adapted for modernity, and that scientists are right to be experimenting with alternative methods. But that adaptation can only happen after meditation as it already exists is understood in the first place. Otherwise you throw the baby out with the bathwater.

Why did they only teach students meditation for 10 days? It seems like they wanted to improve teenage mental health, but somehow decided in advance they were only allowed to make tiny modifications without actually changing anything about the way school or society works. We can’t reduce academic pressure. We can’t reduce class sizes. We can’t design a curriculum that students actually find interesting. We can’t take away their phones. We can’t give them less homework. We can’t have them spend a couple years actually learning to meditate. But a 10 session course? We can probably swing that.

If we want to actually fix teenage mental health, maybe we need to acknowledge that the current system as it exists is giving teenagers bad mental health, and actually be willing to perhaps maybe make some changes to that system. There is also something deeply offensive to me about harming a population, and then telling them they just need to learn to regulate their emotions better. The problem isn’t a lack of emotional regulation skills. It’s the fact that we’re harming them, and we don’t know how to stop harming them. We are powerless, which is unacceptable. Technology is going to keep getting stronger, and we must learn how to achieve and maintain human wellbeing in light of that fact.

Footnotes

  1. I’m not saying social media specifically, but “screens” generally. People point to other explanations like genetics, family dynamics, academic pressure, financial stress, suicide contagion, and so on. None of these are correct when it comes specifically to the recent spike in suicides in the 2000s. The timing is way off for all these other answers. Has the genetics of teenagers substantially changed since 2007? Because the number of iPhones sure has, so I’d probably put my money on iPhones being the problem. Same logic applies to the other explanations. Let’s take suicide contagion: If it was always true that suicide spread from one teen to the next (which is almost certainly true), then that would always have been happening, and an endemic equilibrium would have been reached a long time ago. A teenager whose friend committed suicide in the 1800s would have also been more likely to commit suicide themselves. This cannot explain the recent spike in suicides. If you want to explain something that started around 20 years ago, you’d better posit a main cause that hasn’t been around for much longer than that. All that said, while the recent spike is clearly the result of smartphones and the internet, that smaller spike is part of a larger trend toward suicidality that has been going on for much longer in the US and Canada, since at least the 50s, and seems to track with financial stress etc.

  2. Aside from online app-based studies like this one: https://pmc.ncbi.nlm.nih.gov/articles/PMC10548318/

  3. https://www.lesswrong.com/w/the-map-is-not-the-territory and https://www.lesswrong.com/posts/HxXb8ogQSLDwnL7xh/notes-on-actually-trying both came to mind here. There is a difference between Actually Trying and pretending to try, or just following some kind of procedure related to trying.