80 Comments
Comment removed
Expand full comment

I think that you mentioning Dozy is totally appropriate. In comparison to that Mitochondria startup shill article:

This wasn't the focus of an article, rather part of community feedback and follow-up to a topically related but unaffiliated article.

You didn't know the guy previously, rather than knowing the Mitochondria team before writing the article.

Compared to the previous investment opportunity of an early-stage biotech startup, I think that asking for advice and presumably much smaller funding for an app that is more likely to potentially turn a profit is much more appropriate.

Expand full comment

> (I know some of you don’t like when I advertise things on here; feel free to tell me in the comments whether or not you think this example was appropriate)

IMO, you should make it clear whether you have any personal stake in the things you're advertising. If you don't, IMO this is unambiguously fine; if you do, it's a bit more muddied (and the fact that you didn't make it clear leans against it).

Expand full comment

As someone who did the Say Goodnight To Insomnia program many years ago and found it effective, I want to highlight something to those who are skeptical about the efficacy of "therapy": the CBT part of it may not have been doing most of the lifting. The program also includes a stringent sleep hygiene protocol.

In a nutshell:

* Give up caffeine and alcohol.

* Keep a sleep log to figure out how many hours of actual sleep you get each night.

* Add two hours to that number and then keep to a strict bed schedule. If you are getting three hours per night, say, then you get five hours in bed. In bed at 10pm, up at 3am, etc.

* As your sleep quality improves, expand the time bounds until you are getting a full night's sleep.

There's more, but that's the gist.

There is also a CBT component that involves self-talk to help with rumination and other impediments to sleep. And I'm in no way critical of that aspect of the program. But if you can actually gut out the sleep hygiene part, it's a pretty hard reset to the system.

Caveat: I have no idea whether things have changed since I did this over a decade ago.

Expand full comment

> But a part of me worries that "FDA-approved" has a certain cultural meaning, and if the FDA approves one category of thing with lower proof than usual, people are going to get confused and think it's on a firmer evidence base than it is.

With a software background and skimming that material, the impression I get from "FDA-approved, but with a lower burden for software certification" is "ordinary clinical trials have shown clinical efficacy of the app, but when it comes to making sure the app won't crash or tell the user to sleep for 4294967295 hours tonight or whatever, we aren't certifying that type of correct operation in the same way that we would for a pacemaker."

If I'm correct, it doesn't seem like they're sacrificing the "evidence base" in the clinical trial sense that the FDA culturally means to me. Software certification has nothing to do with evidence of clinical efficacy, but in general means evidence that you're following some specific formal software engineering process, and have the paperwork to prove it.

One particular issue that comes to mind for apps is that the phone itself lacks any FDA software certification (and we don't want a stupid situation where the app can only be prescribed for specific approved phone models), so certification of the app has inherent limits. If the phone crashes, the app crashes, etc.

So I don't really see any problem here.

Expand full comment

"I agree that apps are inherently low-risk"

The risk to any given individual may be lower than ingesting a physical substance, but apps *scale*.

Expand full comment

I hope a bit of statistical nitpicking is in order:

"that in-person cognitive behavioral therapy for insomnia has an effect size of 0.98, and the same therapy delivered over the Internet has an effect size of 0.51 (both numbers significantly different from control, not significantly different from each other). Somryst itself has significantly outperformed placebo in several studies. A meta-analysis finds..."

These effect sizes (ES) don't have real units. 0.98 what? kg? m? The reason is that the difference between the groups is divided by the standard deviation, and both have the same unit, so we end up with a dimensionless number.

Some people might argue that ES have a unit, namely, the standard deviation---of what? Of the control group? Of the baseline measurement before randomization? The latter depends on the inclusion criteria, because strict inclusion criteria reduce the variance of the measurement.

Especially in sleep studies, it would be easy to report the effect of a therapy in hours, so using dimensionless "measures" of an effect does not seem necessary. Within a study, it does not cause too much harm (the 0.98 and 0.51 seem to describe the same dependent variable), but social scentists and psychologists and psychiatrist use these numbers to compare effects across studies or variables.

Example 1: A therapy increased sleep quality, as measured by a questionnaire, by 2 units (kg, say). Sleep duration was increased by 1 h. So, on which variable do we see a larger effect? Obviously, we cannot tell if 2 kg > 1 h. But psychologists can! Just convert both results to ES, say, 0.6 for quality versus 0.4 for duration, and then move on.

Example 2: Same as Example 1, but with two studies investigating two new therapies. For Therapy A, Study 1 finds improved sleep quality by 2 kg, ES = 0.6; for Therapy B, Study 2 finds increased duration of 1 h, ES = 0.4. Conclusion: Therapy A is better than B, because, well 2 kg are more than 1 h.

Example 3: Meta-Analysis. Same as Example 1, but with two studies both investigating the benefits of Therapy A. Study 1 uses sleep quality as the outcome variable and finds an improved qualty by 2 kg, ES = 0.6; Study 2 finds increased duration of 1 h, ES = 0.4. Meta-analyst says, the "average" effect is (0.6 + 0.4)/2 = 0.5, somehow averaging 2 kg and 1 h.

I concede it is a bit off-topic, but there are quite a few statistically literate people in this forum, and the use of ES is disturbingly high even in the statistically literate population.

Expand full comment

Headspace is mostly a meditation/mindfulness app but includes a sleep section. No idea how it compares to the cbt-I apps. You can get it for a small monthly subscription fee, and I know some companies have been buying bulk memberships for staff, don't know if doctors are prescribing it

Expand full comment

Insomnia Coach, by the US Deptartment of Veteran Affairs, is a free CBT-i app. They also have one called CBT-i Coach; I'm not sure the difference between the two. I've only used the former, and the latter looks less polished from screenshots. They also have several other free mental health/therapy/coaching apps.

Expand full comment

It’s wildly untrue to say it’s hard to charge more than $5 or $10 for an app!

Just in sleeping and wellness, Calm is $70/year, Headspace is £49.99/year, Sleep Cycle is £25/year, Pillow is £40/year – I could go on!

It is fair to say that pricing has gone from up-front to “free trial then hefty annual subscription”, but people are clearly paying – Calm generated $100 million+ in revenue last year!

Expand full comment

I, for one, think this advertising is okay, since he's a fellow EA.

Expand full comment

For what it's worth, I came across sleep.io, which I have no idea if it also qualifies - I never got into it except to take their onboarding quiz, after which it turned out they really are just an app (I don't have a smartphone) and there's no website equivalent. But they talk a lot about CBT in their science section, which suggests they might be in the same niche, too.

Expand full comment

Note that uptodate isn't always uptodate, of course. I suppose they too suffer from the tragedy of legible expertise.

COMISA (comorbid insomnia and sleep apnea) is one of the most common forms of chronic insomnia. It can be treated with just CBT-i, but there is reason why you wouldn't want to do that! Sleep apnea leads to a host of other issues in the short and long term and should be treated along with the insomnia.

All should now, especially if they have insomnia, that sleep apnea is much more common than people usually presume, even in the young, thin crowd. If you have any sleep issues and some of the common comorbidities that come with sleep apnea (ADHD, GERD, bruxism, for example) then you really ought to get a sleep study done, and especially if you are younger, thinner, or more fit, you'll need a test done that will pick up on arousal based sleep disturbances along with the oxygen desaturation based disturbances that are more commonly done.

Expand full comment

As someone who therapizes people, often with CBT/ACT strategies involved, I just want to say, the effectiveness of any of this type of strategy depends primarily on whether our frigging patients DO the frigging work! This is something that drives every therapist I know BONKERS. I do know that some therapists are better at this, since they have a kind of gentle persistence that I struggle to carry out, but even those ones have just a somewhat higher rate of compliance, it still ain't high.

Behaviour change is MUCH MUCH harder than most people realize, even for pretty small and simple stuff (5 frigging minutes of frigging slow deep breathing once a day, for frigs sake! FIVE MINUTES!).

Therapists dream and fantasize about all sorts of things that might work to increase the uptake on the strategies; follow our pts around nagging them, electric shock collars, charge them outrageous fees that only go down if they can show they DID THE STRATEGIES .....

I recently had a super interesting experience around this, because I DO have leverage over some people (unfortunately not my pts); my STUDENTS. In a Psych Disorders class, I offered people a range of strategies to try out, focussing on wellness and coping with the stress of COVID isolation and all-online learning. I told them to choose a strategy that was relevant to them. They then planned how to apply it 5 days a week for three weeks, and did so, with accountability. They knew it was WORTH GRADES. And they were curious to see what this was like, and super stressed, many of them. (And it's not hard to tell who faked their work on this assignment!)

Most struggled during the first week, forgetting to do the strategies, feeling they were doing them clumsily, that this was ridiculous etc. They only continued because it was a class assignment. It got easier in the 2nd week. By the third, applying the strategies was going quite smoothly, and many started experiencing good effects. SO FAST! They were astonished especially because I had framed this JUST as an experiential exercise around behaviour change, and told them it probably wouldn't actually start making a difference for them in that short time frame. I wanted to reduce placebo effects and disappointment, 'cause 3 weeks really is a short time. And many were astonished that it worked even though they didn't believe it would.

So I got to explain how, if something is effective, it WILL help most people who use it, even when they don't believe it will help. That if you have to believe in it for it to work, then that's not treatment, it's magic. AKA, the placebo effect. Excellent learning experience all around!

So yeah, if you want to try an app or a book or a therapist, the big secret is to DO IT.

Now if I could just figure out how to apply some leverage to my patients .....

Expand full comment

I'm an unreasonable type of person. Reading the Sleepedy thing annoyed me because of George and Susan. Perfect Susan, who gets perfect sleep every night because she follows the perfect routine of not thinking, just get into bed and fall asleep.

I wanted to punch her in her perfect face. Meanwhile poor George gets blamed for being anxious and causing his own problems. So it looked like your employer was going to go bust and you'd be laid off and you were anxious about this and that caused lack of sleep? Silly George, nothing bad happened! But now you *made* yourself unable to sleep!

How about if George really did get laid off and is now struggling to find a new job and in the meantime he can't pay his rent and now he's facing the prospect of eviction - would they be less finger-wagging smug then about his insomnia?

It just reinforces for me that CBT works for certain people, when the underlying problem doesn't actually exist, it's just baseless worry. I get that changing your thoughts and cognition works for that. But if you have real worries, plus insomnia, I don't see how this will help much - the advice as part of it that "stay out of bed until you are falling down with tiredness and then you'll sleep" will work (in the short term) and then you allow yourself extra time in bed until you build up gradually to eight hours a night by which time you allegedly will have learned to fall asleep without thinking and thus keeping yourself awake, but do you need an app to tell you that?

Expand full comment

The comp.risks take on this (or: what the Black Hat character from XKCD would do) is to create an interactive therapy app which seems helpful and gets approved, then turns dark and creepy on you when you use it long enough - longer than the FDA eval period. Imagine what a malicious therapy app which you had come to trust could do to your head.

Expand full comment

RE: (I know some of you don’t like when I advertise things on here; feel free to tell me in the comments whether or not you think this example was appropriate)

This one feel fine to me. Now that I have a "YES" example and an "Ehhhhhh" example, the difference!

One is just a dude who is 80% probability: a cool dude! While the other is a business doin business things. It's probably fine to mention a buisness's buiness things in the context of "this is what they are doing and I think it is cool", but anything beyond that leaves endorsment and enters advocacy, is the feeling I guess.

Fuzzy as shit.

Expand full comment

FWIW I thought your mention of the Dozy founder was extremely acceptable!

Expand full comment

Maybe FDA approved needs to be replaced with some kind of efficacy rating system. People are used to the AAA all the way to "D"

Expand full comment

ionamind.com is an excellent i-cbt app that simulates what a low intensity cbt therapist would do

Expand full comment

I googled Night Owl and found it also in Google Play, what am I missing?

https://play.google.com/store/apps/details?id=mindware.nightowl

Expand full comment

Re: advertising, I have had no problem up until (and including) this point.

I think the key is moderation. If he was advertising it every post, or every other post, it would become distracting, and I'd start fixating on it/being annoyed myself. If it's one post a week or two weeks, then only the (small) tail of the distribution will care.

Expand full comment

I’m an MFT and came across a very simple method of going to sleep or returning sleep when sleep is interrupted. It is likely that there’s some science out there on why this method works, though I can’t say I’ve looked into it...well, because on some intuitive level, it made sense to me (having once been a baby rocked inside a womb and once outside the womb, rocked as well), I tried it and it has worked for me, I have told others about it and it’s worked for them. So there’s about zero scientific engagement from my end and only a little qualitative date. So what is it? It’s rocking, it’s that simple. I’m a side sleeper so if I’m having trouble going to sleep, I lay on my side, my natural position, and I rock 15 to 30 times side to side. I don’t have to rock completely, just a gentle rock and boom, I’m asleep and can’t remember falling asleep or back asleep if I’ve woken during the night. When I first did it, my mind said to myself, “this won’t work” and it still worked. I don’t think there’s any potential harm in trying, it costs nothing and takes a minute, actually less than one minute. If you are so inclined, then try it.

Expand full comment

Something I would like to see mentioned in the quizzes is "are you a parent of a newborn/toddler?". I technically have insomnia according to the criteria, but I have a newborn and I think that should be an exception. I also suspect that the techniques described in these apps won't help and aren't tailored for new parents.

Expand full comment