510 Comments
Comment deleted
Expand full comment
Comment deleted
Expand full comment
Comment removed
Expand full comment

Thanks for the (second) shout-out. The thinking behind omura's wager led me to post this thread, which attempted to redesign a large chunk of medicine around it: https://twitter.com/alexandrosM/status/1435565853466972164?t=lOVddQ3v0IxEAv81Rub39g&s=19

Expand full comment

I've been struggling with this Pascalian *everything* for years for the exact reasons you articulate.

"Struggling" is probably not the right word, but it's always in the back of my mind.

I most don't take 250 supplements or do all the One-Weird-Trick-To-Enhance-Your-Love-Life things because...who has the time to suss out all this shit?

Just doing what Scott tells me is a lot easier.

Expand full comment

I always felt the ‘pascals’s wager’ part of this was obvious and a particularly important component in stopping COVID-19 in a world where some people will not take the new vaccines no matter what. Even if these treatments only have a 1% chance of making things better (and it seems higher than that to me), it’s better than the 0% we get in the current hyper-partisan world we inhabit. Not to mention the meta benefit of tamping down the divisive rhetoric. I’ve long felt the ‘vaccine only’ crowd is living in the world they wish existed and not the one that we’re actually inhabiting. Anyway, that’s my obscene, unenlightened $.02.

Expand full comment
founding

> From the Inside View, I have a lot of trouble looking at a bunch of studies apparently supporting a thing, and no contrary evidence against the thing besides my own skepticism, and saying there’s a less than 1% chance that thing is true.

I guess I don't understand why this is the case. In particular, what propertion of things that have at least that much evidence behind them are actually true? In this domain, where low-quality studies are omnipresent, it seems like it should indeed be less than 1%, so there's no problem.

I suppose that it feels intuitively weird to look at a not-exceptionally-bad scientific study and say that there's a >99% chance that it's wrong, but in fact we deal with things like that all the time in all domains. https://www.lesswrong.com/posts/Ap4KfkHyxjYPDiqh2/pascal-s-muggle-infinitesimal-priors-and-strong-evidence seems relevant (not the part about double exponentials, that doesn't apply here, but the part about how decillion-to-one chances crop up all the time).

Maybe the problem is just that a good study and an average study intuitively look similar-ish and like they should have similar-ish effects on our beliefs, even though in fact their Bayes factors differ by orders of magnitude? Maybe a person could cure themself of this intuition by reading enough average studies whose findings were later disconfirmed by good studies?

Expand full comment

It seems to me that Algernon's law would not be violated in cases where a person is taking in less of the particular substance than was normal in the evolutionary environment. In other words, in a world where many people are low in Vitamin D, topping up would seem to be a good strategy with an even smaller chance of side effects.

In that case ... which of Kurzweil's 100 supplements do I have a reasonable chance of being deficient in?

Expand full comment

There's a strong argument that Algernon's Law does not apply to Vitamin D. The maintenance dose for the Spanish RCT studying Vitamin D for COVID is around 8000 IU/day. This is about what someone would get from 2 hours in peak direct sunlight shirtless. Hunter-gatherers probably got this or equivalent, but we clearly don't.

(State nutritional guidelines are calibrated for the amount needed to avoid rickets, i.e. severe deficiency, but D does more than just prevent rickets.)

Similar considerations favor vegetarians taking creatine, but not randos taking Ivermectin or SSRIs.

Expand full comment

Just jumping in here to say that melatonin definitely, 100 percent works for sleep in at least some people and the fact that studies can't seem to prove it must be down to some weird confounder we haven't figured out yet. I went through a period of severe insomnia that was unresponsive to any of the "proven" drugs and melatonin cured it immediately and I can use it as necessary and get the same overwhelming therapeutic effect, in a way that no other pill even resembles. So there is certainly room in my ontology for effective therapies with idosyncrasies that defy straightforward investigation in RCTs. But there had better be very large signal buried in the overall noise.

Expand full comment

Aren’t *multivitamins* arguably a form of Pascalian medicine familiar from many people’s experience? I used to take them every day. I give them to my kids. For any individual vitamin in them, I’d assign a relatively small probability that supplementing it does anything good — but a higher probability that at least one does. So maybe these 30 Pascalian covid treatments could just be bundled into an “anti-covid multivitamin”?

Expand full comment

>if I “know” that onion farmers doing studies will convince me that onions have a 5% chance of curing cancer, I should just believe there’s a 5% chance onions cure cancer now.

That doesn't follow, because the fact that people are doing studies about something is itself evidence that should raise your confidence that there's substance to it. It's weak evidence, but it might not be so weak that it can't get you to 5%.

Furthermore, this should depend on the motivation for the studies. Your hypothetical assumes, as a premise, that the studies are motivated purely by a desire to sell onions. But in the scenario "what you think when you see the studies?", you don't know motivation. It may be that if you're fully informed about everything, including the motivation for the studies, you won't think there's even a 5% chance, but being partly informed may falsely lead you to think there's a 5% chance.

Expand full comment

Who’s gonna tell this guy that more people died in the treatment arm of Pfizer’s vaccine trial with significantly more frequent cardiovascular events? Or that only one died of Covid in the control arm vs zero in the vaxx? Or that a Pfizer whistleblower presented very serious accusations of data falsification?

I would love nothing more than to be so innocently credulous towards companies that have paid tens of billions in fines for outright lying in recent years, in an industry whose marketing budget dwarfs any industry in human history, and who are fighting tooth and nail to hide the primary data on which these trials are based from public scrutiny for 55 years.

Expand full comment

Not sure, whether this analogy is valid or not, but let’s apply this logic to various types of food.

Suppose you don’t know anything about vitamins and minerals, but you know e.g. that eating lemons helps agains scurvy and some other types of food help with other things. Applying the logic from the post you’ll end up with a “Pascalian diet”: eating as diverse food as possible. And with a hindsight, it doesn’t seem to be such a bad idea.

Expand full comment

> There’s a potential compromise solution, where smart doctors come up with Pascalian medicine protocols for the few patients who would actually want them.

Do what I do, and keep a stable of general practitioners (who do not have hippa release to each other) and rotate through them, presenting slightly different versions of your current health situation in order to have a doctor sign off on your plans

Expand full comment

Well, if it’s true that out of 20 most promising things (by the view of a year ago) 1 really works, then 5% chance of working does seem like pretty reasonable estimate, doesn’t it?

And if that is true, then in this instance, taking 20 most promising things really is a good strategy.

Expand full comment

I think you should just bite this bullet. I think it's OK to just say the Outside view is correct here.

"From the Inside View, I have a lot of trouble looking at a bunch of studies apparently supporting a thing, and no contrary evidence against the thing besides my own skepticism, and saying there’s a less than 1% chance that thing is true."

When low-quality studies keep piling up, eventually it should start to count against the conclusion. If nobody gets around to doing a very high-quality study despite all the low-quality ones, that starts to indicate that one can't do a high-quality study because it won't look good.

Expand full comment

The problem is another aspect of risk that you don't discuss. It's a kind of opportunity cost, but with a multiplier effect. Taking ivermectin *as an alternative to vaccination* presents a real cost. The ivermectin advocacy set significantly overlaps with the big pharma/medical researchers/public health officials and big tech and Dems conspiracy set. That set interacts with political sets in all sorts of realms.

Nonchalance towards ivermectin because there's a 10% chance it works with little downside risk is, in isolation, one calculation.

In the real world it interacts in complicated ways with many high damage function risks in a number of different domains. That isn't to say, don't consider ivermectin. It's only to say that you need to examine the related risks *in the full context*

Expand full comment

Is fluvoxamine really so innocuous that you would take it if it had 5% chance of reducing your COVID mortality 30%? Would it be a good idea to take 20 drugs with fluvoxamine's side effect profile? These are genuine non-rhetorical questions. The list of side effects on Wikipedia looks like something I wouldn't eat on a whim, but side effect lists always look scary.

I also have to assume there is probably some grand correlation between "likelihood of having an unanticipated positive clinical effect" and "severity of side effects". As in, if something is bioactive, it is more likely to have both good and bad effects. Obviously there are a huge number of exceptions in either direct (penicillin, sodium cyanide) and I'm supposing this is less true for antivirals where magic bullets that affect virus but not human are more likely to exist. But for non-pathogens, Algernon's law seems like a good guide. "X is a chemical that affects your metabolism. Is X more likely to be good or bad?". Probably bad. And if you know X is completely innocuous, it almost certainly doesn't help you either. I realize this paragraph is very handwavy.

Expand full comment

From a personal perspective yes, go all insanity wolf and the drugs, I think the bulk of your arguments make sense.

What you ignore is the potential social consequences of hastily incorporated drugs. Any side effects not mentioned (and deliberately avoided based on your "utilitarian calculus") will meet with such damning criticism that it might jeopardise the entire medical establishment and cause people to lose faith in medicine, given how risk averse we are.

And if people start losing faith en masse, we're done more harm overall then the benefit we gain by adding a bunch of game-theoretically (for the individual) better drugs.

Expand full comment

The biggest reason not to take any of this junk is that there are vaccines of proven efficacy and safety right now. Yet the word “vaccine” does not appear once in this essay. What is going on here?

Expand full comment

I definitely struggle with this as well. I take a multivitamin but no supplements. I'm sure that there is stuff in my multivitamins I don't need and supplements that I do 'need' but I can't tell which, so I stay stuck at this mostly arbitrary equilibrium.

Expand full comment

Tl;dr I would probably take the wager if (# of drugs for which i am ~X% confident will work) * X% * (treatments that might theoretically work) / (treatments that we reviewed to see if they work) = (the # of treatments our prior says we would expect to work). If this is true, then X is probably pretty well calibrated.

One argument against Pascallian medicine might be: I believe that thee is a 5% chance of this medicine working, but suppose i am miscalibrated by an average of 10% probability each time, which seems reasonable. Then (assuming my errors are normally distributed, of which I have no guarantee) I can be 95% confident that the probability a perfect Bayesian would hold of a drug working is… between -15% and 25%, not exactly confidence inspiring, especially when you remember Kightian uncertainty probably outweighs these numbers. So is there a way we narrow down the range of the probability of working?

… on the other hand, it seems to me that an onion farmer couldn’t get me to believe in a ~5% chance of onions curing cancer just by p hacking, so maybe, since the expected benefit of the drug is still positive, albeit barely, I should take it.

One way of resolving this problem might be this: suppose there are 10,000 chemicals that might theoretically cure cancer, but we only expect a few to work (say, 5 have really strong effects and 5 more have weak effects that are still worth having in some circumstance.) and suppose the lobbies of each drug hire some scientist to get positive results for their drugs, such that it seems to us there is a 5% chance of a given drug working. 5% of 10,000 is 500 drugs working against cancer, which is orders of magnitude than we would expect, so we should default to the outside view here. On the other hand, suppose only 400 chemicals seem to us to have a 5% chance of working, and we reject the other 9,600 out of hand after reviewing them. We find that this suggests that about 20 of the 10,000 work, which isn’t so far from our original estimate. Here, since the inside view and outside view agree, maybe we should take the 400 drugs. If this were to happen, I would think long and hard before probably taking the drugs.

Expand full comment

I struggle to understand where 'Pascalian Medicine' would substantially differ from the alternative / complementary medicine industry, which is almost a $100b sector of the economy. The underlying ethos of both seems to be "Uncontrolled, iterative n=1 trials of pharmacologically active substances and pseudo-behavioral interventions with the primary outcome of improved subjective sense of well-being." I have no problem with this, any more than I have a problem with people finding the right exercise and diet regimen for themselves. But there's a difference between 'improving wellness' and 'treating disease'. n=1 experimentation can be great at the former, and usually pretty poor at the latter.

I think there are likely specific clinical scenarios where a Pascalian approach makes more sense - poorly understood chronic diseases, like post-infectious syndromes, or diseases where there is no good a priori evidence of effective treatment. Many of the rare disease and chronic disease communities and their treating physicians effectively take a Pascalian / Omuran approach in settings where no clinical trials are available.

Expand full comment

expected-value-of-x = value-of-x * P(x-works) + bad-from-x * P(x-is-bad)

The value of not dying of covid is very large. Severe side-effects, including death, have a large negative value. P(x-works) and P(x-is-bad) are both small.

expected-value-of-x = infinity * 0 - infinity * 0

You can replace whatever large/small values you want to get whatever specific result you want for your expectation value.

To put it a different way, the expected value from taking x might be good, but you can't just consider the expected value. You have to consider the whole probability distribution, which is wide and flat.

Expand full comment

Another great post. I wish I could think like this. This methodically, this thoroughly. Why is this so rare? And then express those thoughts precisely and with humor. This is clearly even rarer.

Expand full comment

What do you think of the heuristic that "The body can deal with surplus better than scarcity" for certain types of substances. If too much of supplement A is not a big deal but not enough of supplement A is a big deal, then we can err on the side of too much of supplement A. I think that might be the case for a lot of supplements, so taking a bit too much is reasonable. For other stuff, not so sure.

Expand full comment

I think it's a mistake to group vitamin D in with things like Ivermectin. While the evidence that either therapy cures Covid is pretty thin the decision is not at all the same.

Vitamin D advocates are asking people to follow existing medical guidance to avoid deficiency. To maintain serum levels within 30-60 ng/ml most people will need to take a supplement in the winter, and almost everyone at risk for Covid (people with melanated skin, obese people, the elderly) will need to take one. This isn't taking some random therapy on the off chance it's helpful, it's following medical guidance that existed prior to Covid. Recommending vitamin D for these groups is a lot like recommending that they lose weight or quit smoking. It's possible that those things help with Covid, but even if they don't it's what your friendly neighbourhood endocrinologist would recommend anyway.

By contrast Ivermectin really isn't something that most people need. There's no such thing as "Ivermectin deficiency" and there aren't any medical associations who recommend that people get tested and see if they need more ivermectin in their diet.

Expand full comment

My concern with a lot of alternative/non-standard medication is that not that it, in itself, is dangerous, but that it acts a gateway drug (pun intended) into a world that really can be harmful. E.g., I'm skeptical that acupuncture has any real value, but I also do not expect it will cause any direct harm either (other than the time and money wasted). However, I am concerned that, while an acupuncturist is treating you, he will tell you about all the dangers of modern medicine and then give you a referral to a chiropractor, who will give you a referral to a naturopath, who will give you a referral to one of those people who want to you treat your cancer with peach pits or autism with bleach.

Modern medicine isn't perfect, but it's pretty clearly better than alternative medicine (as the joke goes, if alternative medicine works, we would just call it medicine). Encouraging people to start down the alternative medicine path itself is dangerous.

Also, taking any given supplement may only cost $20/month, but that means that taking 20 supplements would cost you $400/month, which starts getting into real money.

Expand full comment

This all brings to mind that old post about Epistemic Learned Helplessness: https://slatestarcodex.com/2019/06/03/repost-epistemic-learned-helplessness/. All these arguments in favor of Ivermectin and curcumin and all the rest, they prove too much: https://slatestarcodex.com/2013/04/13/proving-too-much/.

Expand full comment

what prevents algernon's law from accidentally proving stimulants don't work, when it seems like everyone who takes them loves them?

Expand full comment

My prior on circumin being better-than-nothing for covid was >50% because it's a weak anti-inflammatory and several other anti-inflammatories (budesonide, dexamethasone) were extremely beneficial. In some third world country that can't afford steroids, where dirt-cheap circumin is being produced locally, it might make sense. But here in the US where you can just walk into Walgreens and get a much better anti-inflammatory for $20, it's not worth trying.

Expand full comment

On PAINS: Curcumin isn't just a PAIN, it's the worst of them all. See: https://www.science.org/content/blog-post/curcumin-waste-your-time (by Derek Lowe).

Quercetin is also pretty bad.

Expand full comment

Great post. Hilarious, too, and that's coming from someone who's been in the natural supplement and nootropic manufacturing world for decades. I don't blame anyone for getting burnt out from superfood and supplement consumption, and the Everest size mountains of related research. It's eye-rolling and existentially exhausting, even for those of us who are deep believers. Thanks for the perspective, Scott.

Expand full comment

You're forgetting the placebo effect, which cuts both ways: it makes Pascalian medicine more effective, in that it actually does help whether any active ingredient actually works, and it argues for reducing the number of pills down toward an ideal of 1 magic pill.

Expand full comment

I think the principle of outsourcing decisions about what drugs are cost effective if prescribed by doctors using them only if they reasonably believe them to be cost effective for the individual patient is in principle a good system. I'd just like to see reforms in incentives that made those suppostions the case.

In the system we have FDA seems overly concerned by "side effect" costs, not concerned enough about delay of benefit costs, and not enough about resource costs. Doctors seem too much concerned with avoiding malpractice suits and not enough with cost of prescribed treatments.

Expand full comment

The argument here seems like it can be condensed down to "I trust science, but science is contradictory, help!".

Seems like the obvious way out is to trust science less. But that feels bad, because psychologically losing trust hurts, and people will mock you for it, etc...

Expand full comment

There's a significant difference between:

1) what one should do;

2) what protocols doctors should prescribe.

The second, as Scott notes, involves various second-order social factors. But I, as an individual, only care about the first.

Expand full comment

Most of the arguments that Scott puts forth against Pascalian medicine could equally be deployed against responses to most X-Risks. The logic is quite parallel.

Expand full comment

...Isn't the problem here that if there's a 0.25% chance that any given medication kills or seriously harms you, a 5% chance they cure survivable ailments, and you take 200 there's a near 100% chance you cure the ailment but a 41% chance you get killed or seriously harmed? Is that a good tradeoff?

A "bad outcome" might be much much less likely than a good one, but probably we want to avoid it much much worse.

Expand full comment

> But I have to admit that given everything I know about ivermectin - and Vitamin D, melatonin, etc - I still think on net the very small chance that this stuff helps you is higher than the extremely small chance it kills you. Even if the latter isn’t quite zero.

These two are not unrelated: normally the stronger intended effects anything has on our biochemistry, the stronger the unintended ones. So the expectation of how much any specific one _might_ help should go hand in hand with the expectation of the harm if _might_ produce.

I think we are betrayed by our intuition, because we judge the low-expected-value of the negative outcomes as "come on, whatever this does, it can't be that bad" and the low-expected-value of the positives as "I don't think it'll happen, but if it does it's amazing". Something something Taleb invest some money you can live without in buying some crazy stock.

But this is not the same as going long on stock: without good evidence that any of these cures actually help, and without any reasonable model of why they may, it's basically betting on a black swan that can go _either way_. Betting on more than one (the Insanity Wolf position) doesn't improve the basically-zero expected value, just the spread.

Which is why the intuition is: it's weird to tell any single specific individual "you shouldn't do this", but it sounds like evidently bad policy to recommend it at society level. Everyone gets to decide what's their personal risk/reward balance, but as a society we'd rather avoid large stupid catastrophes.

A recommendation to single individuals could be: if you want, take more or less a couple of whatever "no proven cure but no proven side effects" that seem reasonable to you, but not too many - you may still catch the black swan, while as a society we're hedged from everyone taking everything and one of those chemicals turning out to make us sterile or whatever.

Expand full comment

I'm not so sure about the hand waving away of side effects. Is it really so likely that a random drug is more likely to have a previously-unknown good effect than a previously-unknown bad effect?

You could respond that we have many decades of data about people taking ivermectin safely, and only a year or so of data about people with covid taking ivermectin, so we've had ample opportunity to discover ill effects and limited opportunity to discover bad side effects. On the other hand, do we have much data about people who don't already have worms taking ivermectin?

Expand full comment

Given only 50% of studies in medicine can't be replicated, that should immediately halve your probabilities. We need some analysis of what proportion of studies with small effects fail replication, but I bet that too would scale down the final probabilities significantly.

Expand full comment

Too much of the counter evidence against ivermectin is predicated on over-interpreting non-significant p-values. Alpha of 0.05 is a high bar to clear to begin with. Throw in all the complexities of treating patients for a novel disease in hospital settings while trying to run an RCT for a highly controversial repurposed drug, and it's an insanely high bar to clear. Factor in variations in dosages, days since onset of symptoms (which patients are notoriously bad at reporting accurately), severity of symptoms, and any number of factors which influence patient outcomes for Covid that can't be controlled for because they are unknowable, and you're basically set up to fail.

Keep in mind any p-value less than 0.5 means there's a greater chance of committing a Type II error than a Type I error.

When you have dozens of studies, no matter how hard it might be to compare them, consistently concluding there is a signal of efficacy for ivermectin, you *have* to change your burden of proof heuristics. Simply running a meta-analysis and interpreting the p-value @ alpha=0.05 is deeply reductionist.

This is why I am utterly unconvinced by the "debunkings" of ivermectin data. There is more than enough signal in the data, and when you have highly proficient clinicians around the world reporting success with a drug, you cannot dismiss that. You just can't.

All of this is besides the point of Pascalian Medicine, but is meant to address the probabilistic fallacy you are making in the foundation for how you've framed this argument. From my perspective, there's a 95% chance ivermectin is doing something to help treat Covid. The effect size and potential confounding variables may never be known, but it is quite clearly doing *something*.

Expand full comment

Among weightlifters/bodybuilders, the Pascalian Medication strategy is fairly popular. It's pretty common for serious weightlifters to take oodles of different supplements and medications that are suspected with varying degrees of confidence to improve various aspects of performance. Some of it's got pretty firm scientific backing (e.g. caffeine and creatine), and some of it is off-label or illegal use of well-understood prescription meds (e.g. anabolic steroids, metformin), but a ton of it is "broscience" (a sort of oral tradition build on various combinations of highly speculative preliminary study results, laymen's extrapolations from scientific principles, and personal anecdotes). Most weightlifters will tell you that a lot of the broscience is probably nonsense (the term itself is a self-deprecating joke within the community), but they'll nevertheless take it based on an expectation that the combined effect is more likely to help than to hinder their training.

Expand full comment

What a great time to work through my intuition's solution to pascal's wager, and see if it actually works.

Basically, my argument is that your prior for X should always be lower or equal to 1/|U(X)|, where U(X) is the expected utility of X (and |x| is the absolute value of |x|). (Yes, this means your prior is dependent on your utility function.)

We'll do our investigation with P(X) ~ 1/(2^|U(X)|). Why not P(X) ~ 1/|U(X)| directly? Because the integral doesn't converge over the reals, and while some improper priors are okay, I don't want to spend time proving that this one is. This is basically the simplest converging function that I can think of that satisfies the criteria. (The ~ symbol is pronounced "is proportional to", and means that we're eliding some possibly complicated constant that makes our priors actually proper probability functions)

Now, let's compare the hypothesis that taking some drug gives you some amount of positive utility `u`, which we'll call A_u, to the null hypothesis that it does nothing, A_0. We do a study, and find that P(study data | A_0) = 0.05, and also that the hypothesis H that best explains P(study data | H) is A_u. What are out posteriors P(A_0 | study data) and P(A_u | study data)? Well, standard frequentist methodologies don't report a P(study data | H), so this is usually impossible to compute, but if we make the VERY EXTREME assumption that the ONLY POSSIBLE hypotheses are A_0 and A_u, we can say that P(study data | A_u) = 0.95

P(A_0 | study data) ~ P(A_0) * P(study data | A_0) = 1/(2^0) * 0.05 = 1 * 0.05 = 0.05

P(A_u | study data) ~ P(A_u) * P(study data | A_u) = 1/(2^u) * 0.95

Well, what is u? Some random study online gives 1.51 QALYs (Quality Adjusted Life Years) for avoiding an infection entirely, and let's say our drug has a 10% chance of preventing any symptoms. But wait; why use QALYs and not QALDs (Quality Adjusted Life Days) or expected future earnings? This is the last part of our model that needs to be parameterized, and I have no idea how to do it, and it matters a lot: using QALYs means that we should, in fact, be ~95% sure that this thing works; using QALDs means that we require particle-physics levels of certainty before we even consider the fact that it might work.

Please comment if you have a solution to this problem!

Expand full comment

The crux of the Pascalian medicine argument is the assumption that the probabilities are independent. If they are, you can sum them up and get the expected benefit. But Scott has previously argued that he thinks there's a systemic issue with the scientific process that generates this evidence. If the systemic assumption holds (and it sounds like that argument is part of the process of estimating his probability estimates), it's not valid to sum up the probabilities since a systemic failure affects all the probabilities simultaneously.

A corollary to this would be the housing market crash in '08. The assumption that killed the mortgage-backed securities was that each home foreclosure was independent. Once foreclosures became interdependent, the underlying assumption of risk was no longer valid.

Expand full comment

Scott seems to say that there are only two possible ways of conducting Pascalian medicine, both of them problematic:

(1) Doctors try to make all patients take all drugs that might work, which is problematic because (a) they won't, and (b) it erodes trust when doctors give patients a bunch of drugs they know are very unlikely to help.

(2) Individuals try to design their own research-based courses of action, which is problematic because (a) lay individuals don't know how to do medical research (and shouldn't have to), so (b) this path is very likely to end up with some dead or severely injured lay individuals.

Door #2 is too risky for individuals, and Door #1 is too risky for society as a whole, so we're stuck, right? I take this to be Scott's position.

Fair enough. But isn't there a middle ground?

In Door #3, doctors initially prescribe only the drugs that actually work. But if a patient *asks* (unprompted) to be prescribed curcumin or onion paste or Ivermectin or whatever, the doctor shrugs and says, "Yeah, okay, probably won't help, but sure won't hurt, either," and writes a prescription. (A doctor speaking more precisely might say, "I believe this substance is unlikely to help you, p=0.05, but I also believe it's unlikely to harm you, p<0.05.")

Society is protected because doctors only go down this path with patients who ask for it. Individuals are protected because doctors are still using their medical knowledge to protect them from their own layperson research skills. The patient is unlikely to go full Pascalian and try EVERY drug, because the great majority of these patients are only interested in one drug they heard about from some trusted source.

...and social cohesion is strengthened because we stop treating all the Ivermectin people (and all similar groups in the future) as insane pariahs obsessed with lethal horse paste. If we instead start treating them as reasonable people who made a different judgment call, not only will we have fewer chemical accidents as they try to get around the restrictions; we MIGHT even build mutual trust and respect across cultures (leading to higher vaccination rates and lower risk of civil war).

Expand full comment

At one point I was taking about 12 pills a day, and ran up against the limit of what would fit in one chamber of my pill organizer. Lately I've cut down to just 3 pills (all taken with breakfast):

otc lithium aspartate: This one is labeled as 5mg of elemental lithium, which I calculate is equivalent to 150mg of lithium citrate, which is about 1/6 of the lowest dose that doctors prescribe for long-term control of bipolar disorder. In spite of the low dose, it has an obvious effect on my mood. I consider it a performance-enhancing nootropic drug that helps me focus on tasks and stay calm under pressure. As of this writing it was still available on amazon, but I'm 25% confident that it gets taken off of there within 5 years.

fish oil: Mixing this into my oatmeal has induced remission of ulcerative colitis many times. Neglecting to take it for a few months has made me sick many times. There also seems to be some minor antidepressant benefit.

multivitamin: Seems to benefit my energy levels and overall well-being.

Things I used to take, which actually did something:

NAC is an interesting one. While I was on it I got rid of a few bad habits that I've had for almost my entire life, and after I got off it I resumed the same bad habits. But also while I was on it I developed some serious joint problems throughout my body, which got better when I went off of it. This one is no longer available on amazon, because the FDA acts as if noticing that someone investigated it as a pharmaceutical in the 1950s should make us update towards it being less safe than other random things people take as dietary supplements.

Creatine basically just gave me 20% more reps in the gym, with no side effects except water retention, but the dose is a few gigantic pills every day if you take it in pill form.

Probiotics seemed to help with remission of ulcerative colitis, but did nothing else. I used to play a game of "how many different species of bacteria can I find on labels in the yogurt section of the grocery store?" and buy one of each. It seemed to work.

Supplements I used to take, which had no noticeable effect on me:

Vitamin D+K, Zinc, Green Tea extract, Glycine, Spirulina, Propionyl-L-Carnitine, Theanine, Magnesium, Calcium, Circumin

Expand full comment

My calculator says that taking 250 independent risks, each of 0.5%, results in a 71% risk of at least one hit. Liver damage, cardiac arrest, whatever. And how to tell which one? You have to either stop them all and hope that you recover from the damage, then re-introduce the (likely fake) drugs in sets while hoping you don't further damage your body

Gotta listen to William of Occam, too, and NOT multiply things unnecessarily

But you DIDN'T mention my greater concern re side-effects: a Gresham's Law where once you go down the rabbit hole of throwing stuff against the wall, you've likely lost all ability to make smart estimations of probabilities.

My two greatest sorrows were from the suicide of a family member, who had “recovered memories” of sexual assault by a man my mom was otherwise angry about

And another family member who died an agonizing death from metastasized breast cancer; when we cleaned out her apartment I found a large jar of apricot pits labeled as Laetrile.

Both women were victims of quacks; neither was able to protect herself.

Pascal would not roll dice in such crooked games. Possibly, ONE HALF MILLION Americans will die from COVID, and millions more will have suffered needlessly, due to the political assertions in favor of unlikely-at-best therapies instead of the best-available alternatives

I have yet to hear of better epistemological heuristics re self-treatment, other than the one I myself employ: “cui bono?” This doesn't override the sense that most published research is false, but it's a strong signal when a political party is using miracle cures as an adjunct to seizing power from existing medical & scientific institutions, or when somebody peddling pills on their website.

Expand full comment

The problem is your decision-procedure is subject to exploitation.

Currently, most drugs that show this 5% chance of curing cancer (or whatever) are selected by some process that is sometimes reasonable. But if people broadly change their decision-criteria toward taking everything that shows this 5% chance of curing cancer, then the number of things that people try this with will go up significantly due to stronger adversarial pressures, because there's a lot of benefit from showing that your onions (or whatever) can do this.

...this argument suggests that the current epistemic situation is the ideal one for taking lots of these such drugs.

Expand full comment

Doesn't stuff like kidney/liver load and combinatorial increase in interactions give an automatic decay function on how safe is a certain combination of drugs? If you're taking 10 drugs, there are 45 possible interactions (and even more tertiary interactions) and X grams of total crap that needs to be metabolized. If you're taking 100 drugs, there's 5000 interactions and 10X grams of total crap (since you can't just take less of each drug. You may need to actually take more to keep the same bioavailability).

Also, load on off-targets: each drug alone may have too weak off-target effects to register as "bad side effect", but if they have the same off-target (more likely than not, I think - some processes are especially prone) that can accumulate.

So the solution seems to be "I'm physically limited to only this many drug candidates before the risks start outweighing the benefits (since those grow quadratically or at least supralinearly), so I'll take at most the 2-3 most effective ones"

Expand full comment

> Should we worry that even if each drug individually is net positive, giving someone twenty medications will lead to some crazy interaction? I’m not too concerned about this; clinically significant drug interactions are rarer than most laypeople think

Seems like Birthday Paradox might apply here: chance of crazy interaction increases super-linearly with each additional drug

Expand full comment

1) It seems to me that effects of drugs are starting typically at lower doses than its side-effects. Therefore it may be a good strategy to take a lot of small doses of drugs with similar effects (eg nootropics), so that similar effects will add up together, and side-effects are different and small, and therefore not bothersome.

2) It seems that nootropics crowd is doing something like this, when a lot of people are taking stacks of ten different things at once. It seems that they mostly do not approach it with pascalian mindset. But some (inc. myself) do.

3) I heard that Eric Drexler is also taking tens of pills a day because of similar reasons.

Expand full comment

Why is the downside risk of taking a supplement greater than that of taking some new food, especially a food that didn't exist in our ancestral environment?

Expand full comment

I think the only place where I shake my head in dismay is when the same person (1) thinks tiny changes in his diet will have profound effects, like eating heirloom tomatoes instead of french fries will make him live to be 120 -- because lycopenes! -- but at the same time (2) doesn't fear unknown low-level side effects from ivermectin or HCQ because, hey, these are generally safe, right? Lots of people have taken them and I've heard of anything bad happening.

I mean, it's one or the other. Either the entire history of people eating tomatoes might have missed some subtle but important long-term benefit *and* the entire history of people eating ivermectin might have missed some subtle but important long-term harm, or in *neither* case is it likely that some subtle long-term effect has been missed. Believing

Expand full comment

I think the Law Of Conservation Of Expected Evidence is vulnerable to the same problem Pascal's Wager is: our tendency to overestimate small probabilities. There is a vast sea of potential hypotheses (even pairwise-incompatible hypotheses, if you will) that you have never thought of; if you understood how vast it is, you'd probably have an appropriately-low probability estimate for a randomly selected one, but by definition you don't. Just by someone stating a hypothesis to you, it gets elevated from an unknown unknown to a known unknown, and that messes with our perception of probability.

Expand full comment

Here's my naive attempt to steelman the "take a bunch of supplements" position:

Suppose there's something like a general factor of how hard your body is trying to fight illness. Think of it as an immune DEFCON. As DEFCON rises the body's effectiveness fighting infection increases. But so do things like energy use, physical weakness, and risk of long-term damage to other body systems (because the immune system is increasingly likely to attack them by mistake). This is kind of a stupid model but as far as I can tell it vaguely resembles how the immune system actually behaves, at a high level.

The Algernon principle says that raising the DEFCON isn't a free lunch in the ancient evolutionary environment. But it might be a nearly free lunch _for us_, at least under certain conditions. First, the downsides-- needing increased nutrients, decreasing out capability for strenuous physical activity-- might be things we care about less relative to a risk of e.g. serious COVID than our ancestors did. Second, modern medicine might be able to measure the risk of serious COVID better than the body (which has never seen COVID before) can.

So it could be true that supplements act-- assuming they act at all-- only by generically exciting the immune system and raising the DEFCON. And it could be that that's a net negative when we're healthy (or uncertain whether we're ill), per the Algernon principle. But it could still be a good deal to take a bunch of supplements whenever one's feeling under the weather (or better yet, isn't feeling under the weather yet but has a positive COVID test).

Expand full comment

This post doesn't quite get all the way to a cost-benefit analysis (even a quick back-of-the-envelope version). Let's try one:

How bad is getting covid? I've seen estimates of 1 cent per microcovid, which means that getting covid involves $10,000 of badness.

If there's a medicine that has a 5% chance of working, and cuts the expected total badness in half if it works, then: if it works then it alleviates $5,000 of badness, so its 5% chance of working gives it a $250 expected benefit.

That's... not very much.

What are the costs of taking this medicine once you get covid?

Money: let's say it costs $20

Time: maybe 30-60 minutes to figure out it's the thing to get, get it, and take it

Side effects: probably smallish, I'll guess on the order of $10

To stick with round numbers, let's say the costs add up to $50. That gives a net benefit of $200 for taking the medicine if you have covid.

But you don't have covid yet, and maybe right now you're thinking of doing some reading & thinking to figure out what you want to do if you do get covid. How likely is it that you'll get covid and be in a position to take the medicine? Maybe 5% for the next year (and if it happens more than a year from now, your research will be obsolete by then). So an expected benefit to you right now of 5% of $200, or $10, from figuring out now (when you're healthy) that a particular medicine has these expected effects on covid. And the costs of doing the research now to figure this out are more than that; even if it's just 30-60 minutes of research that time is still worth more than $10.

If you figure that out and share it widely, then it's a $10 benefit for each person who learns of it while healthy. If it just takes 2 minutes for each of those people to hear you then that's a net benefit for them of ~$9, but if they each need to spend their own 30-60 minutes of research to verify what you have to say then it's back into the negative. (Plus there's a $200 benefit for each person who learns of it after they get covid and are searching then for what to do. So there is a net benefit to creating an info hub that people can access once they get covid.)

Expand full comment

>Should we worry that even if each drug individually is net positive, giving someone twenty medications will lead to some crazy interaction? I’m not too concerned about this; clinically significant drug interactions are rarer than most laypeople think, and usually pretty predictable. Still, giving twenty different medications at once is almost unexplored territory, and something like this might be true.

I think the risk in this sort of situation is greater than "some drugs among the twenty may have harmful interactions with each other." I think when you start taking effective doses of so many different medications at once, you'd have to start worrying whether you're going to overtax your body's mechanisms for processing drugs at all. Maybe most of the drugs have non-overlapping pathways for removal from the body, but some of them will have overlapping pathways, and you'll start seeing the sort of harmful side effects that you would if you'd greatly upped the dosage of a single drug.

Expand full comment

Does Algernon's law apply when you're deathly ill?

Expand full comment

Joe Rogan is a proponent of Pascalian medicine for covid

https://www.instagram.com/tv/CTSsA8wAR2-/

Expand full comment

I guess you could call "eat a varied diet" a Pascal's Wager but that isn't an argument against it.

Expand full comment

There's a type of drug-drug interaction you haven't considered: that the drugs could prevent each other from working. This is probably more common than you immediately think: for example, a Pascalian approach to depression treatment might naively include a psychedelic, an atypical antipsychotic, and a dopamine agonist, all of which would be fighting each other. Similarly, a Pascalian approach to smoking cessation might involve nicotine replacement therapy, varenicline, and bupropion - all of which would be fighting each other. At the beginning of the pandemic, haloperidol was actually considered as a potential COVID treatment (I can probably find a reference for this if people want me to), because of its antagonism at the sigma-1 receptor - which would, in a Pascalian approach to COVID treatment, be fighting fluvoxamine (which is an agonist at the sigma-1 receptor). Perhaps it would even be worse than nothing. Then again, a Pascalian approach to HIV treatment might lead you to taking three or more antivirals with multiple different mechanisms of action - which is in fact the standard of care. So maybe it's different in infectious disease.

Dangerous drug-drug interactions are also probably more common than you appreciate, _when the drugs actually work_. For example, the above Pascalian depression treatment would probably include both amitriptyline and an MAOI, which is either really dangerous or a great idea [1]. Even on the more benign end, a Pascalian smoking cessation treatment would probably include both bupropion and naltrexone, which together cause a lot of weight loss - that usually won't be a problem, but sometimes it will be. If you're thinking of more benign drugs, like melatonin or curcumin in the case of COVID, then this is less likely, but more benign drugs are also less likely to work, so it takes more of them to reach an expected effect similar to drugs that have evidence. That is, I would expect the probability of drug-drug interactions to be at least proportional to the expected efficacy of all the drugs taken together, perhaps even more.

Separately, starting 20 drugs at once would make it impossible to know which one is causing a side effect. If you even just tried them all separately, you would be able to discontinue the ones you don't tolerate. But if you take them all at once, if even one of them isn't tolerable, you have to stop all of them. (Maybe you could do bisection here? Probably in practice that would be too difficult to carry out.)

So why not try them all, but one after another? The core thesis of Pascalian medicine is that there's little cost to trying one more drug. But if trying one drug means you can't try another until you're done, then that doesn't apply. (In the case of an acute condition like COVID, I suppose you also won't know if it works until your symptoms are gone and you're not dead, so this also doesn't apply.)

What's the lesson here? One of the core principles of pharmacotherapy is that less is more. Pascalian medicine throws that out the window, but it's probably best we don't abandon it.

[1] https://jamanetwork.com/journals/jamapsychiatry/article-abstract/491926

Expand full comment

I think that you are downplaying the magnitude of algernons law. The probability that a random chemical will harm you is vastly greater than it will help. People have weird intuitions about their bodies, so let’s think about another high performing complex machine like an engine. What percentage of random chemicals could you pour in your gas tank that would make your car run better vs make it worse? In covid your chance of not dying is actually quite high, thanks to an incredibly sophisticated and complex machine in a near perfect state of balance. Your prior should be that anything you add to that is very unlikely to help more than it hurts. So anything other than a large well run RCT, no matter how positive, shouldn’t make you think benefit:harm ratio is >1. In fact probably a single RCT shouldn’t be enough to overcome what should be very pessimistic priors.

For all comers, covid ICF is about 0.6%, so even in your fluvoxamine example a 5% chance of a 30% (relative risk) reduction is 0.01%. You don’t need many unexpected harms to wipe out that benefit.

Expand full comment

> We don’t fret over the unknown unknowns of Benadryl or Tylenol or whatever

We absolutely fretted over ibuprofen and Tylenol possibly *increasing* covid mortality early in the pandemic. I suppose you could call it a "known unknown" as soon as people start fretting, but what about before then?

Expand full comment

Looking from outside as a leyperson I think this discussion is often too narrow on biochemistry and statistics. These are imortant fields of sience, but talking about how to deal with problems as a society and expecially when there is no clear scientific answer (yet?), there are a few fore aspects that come to my mind:

1) people are different. I know so many stories about people reacting different to the same medication or diet. Of course with something acute like COVID there is no time to test several drugs once you are ill to see which helps you best. But with such mixed evidence like with Ivermectin it looks childish to me just to discuss who is wrong with the only options being 'it works so we recomend it to everyone' or 'its just a false positive and it harms if people have a false hope'. I think the focus should be to work together and gather as much data as possible to find out who profits from a specific treatment and who doesn't.

2) We need more public funding for independent medical research. This will not only generate more knowledge, but also improve trust in science. Many people are seeing the that the objects of research are not optimized for public good but for potential profits. So 'this is not scientifically aproved' or 'there are no good studies about it' does say more about the economic value than the medical value. This hurts trust expecially as we are often told not to use anything that has not been scientifically approved, but we know by experience that it helped i.e. traditional medicines.

If there are the voices now saying that this is the state is always inefficient in spending, we could decide publicly on the topics to be researched involving polls to see public interest or prediction markets to get the most promising candidates. We could also filter for drugs and treatments that are relatively cheap and can't be monopolized.

3) As there are so many candidates that could have a net positive effect, too many to take them all, but not a clear favorite to recomend to everyone, just let the people decide on their own. Just take care that information on risks and side-effects are easily available. This may go wrong sometimes, but i think the effect is net positive:

- if you want to stop somebody taking something, you have to show respect for opinion and his choice so he is much more likely to listen to you and your warnings.

- this builds kind experience what works and what doesn't. Of cause scientific knowledge is always better, but if there is none or not available experience is better than nothing.

- If you put force against something without good arguments, people often start to question your motives. This breaks trust and they stop believing you when even when you have good arguments next time.

- freedom always includes the freedom to take bad choices. Many people prefer freedom over safety so they will oppose anyone who wants to force them for the good. This is especially about the public pressure to get everyone vaxxed, but it also applies if you want to stop somebody from taking experimental medications with (soft) power instead of arguments.

- better people get the drugs they want to try from someone with medical knowledge knowing and telling the risks, than if they get veterinary versions without information what doses are usually save for humans.

Expand full comment

This reminds me of Ben Williams' "Surviving Terminal Cancer:" https://www.survivingterminalcancer.com/

I haven't watched the film but as it was told to me, Prof. Williams was a psychologist (I think?) that was diagnosed with glioblastoma and a one year survival timeline. He started researching anything that could possibly be effective and took them (metformin and cloroquine were two that I think ended up in his cocktail)... and 21 years later he was still around.

Expand full comment

I don't know why everybody is so critical of curcumin. Within living memory it has cured the British people of their worst affliction: bland food.

Expand full comment

As a critical care physician who has cared for dozens of Covid patients in the ICU, I saw a number of people who delayed seeking evidence based therapies such as monoclonal antibody because they were taking Ivermectin. By the time they decided to try the Monoclonal antibody—after Ivwrmectin failed—they had already developed respiratory failure and ultimately died.

Expand full comment

I wonder if Scott could find some interesting and potentially useful things by looking into things people take to enhance performance - including doping (ie illegal).

Apparently a quite common result is that things that enhance performance do not sum if taken together. Eg caffeine improves performance, as do nitrates, but taken together you don't get both performance boots.

Expand full comment

"And if I’m not 95% sure this doesn’t work, doesn’t that mean there’s a 5% chance it does work?" I understand what you're trying to say, but currently the sentence is logically false. Potentially adding "at least" to both parts would do what you want.

Expand full comment

"Conservation of Expected Evidence"

Sigh. The mental gymnastics some people will go through in order to try (and fail) to overcome "absence of evidence is not evidence of absence" is pretty sad, especially when cloaked in a façade of rationality.

Expand full comment

Exactly. It's a problem with all people who provide a service while dealing with complex systems: investment managers, meteorologists--not to mention the more scummy professionals like economists and social scientists in general. They're incentivized to pretend to know way more than they do. Probably the right approach--more accurate, at any rate-- in all these fields is much more epistemic humility, and openness to new approaches and experimentation at the margin.

Expand full comment

I haven't nailed down a precise scenario, but I suspect if you actually went all "insanity wolf" and really took "All medication all the time" you would, in fact, die.

Expand full comment

One mercy with diabetes is that you can take blood sugar measurements, which at least give you some short term information about whether something is helping. You can't test for COVID risk.

Expand full comment

A missing dimension in this discussion is the ability to get the dose right. Even if we think about drugs in binary terms (works versus doesn't; toxicity versus not), the dose is a key determinant. It's not easy to get dosing right! Most antimalarial drugs have been introduced at the wrong dose for some key subgroups (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3837842/). Chloroquine is an extremely safe drug (at the right dose) and an extremely dangerous drug (at the wrong dose - or right dose if suicide is the objective). Doctors can get the dosing wrong even for well known drugs.

An interesting example is chloroquine for COVID-19: in a clinical trial in Brazil in early 2020, the investigators got mixed up between salt and base weights (salt is the weight of the tablet; base is the weight of the active ingredient). There was a recommendation from China to use chloroquine 500 mg twice daily for 10 days, and they thought that was a base dose (chloroquine 250 salt = 150 mg base). They only had 150 mg base tablets so gave 600 mg base (4 tablets) twice daily for ten days. Turns out this gets you close to lethal concentrations and they probably killed a few people in the trial (https://elifesciences.org/articles/58631)

Expand full comment
founding

One obvious complication is that you can't treat 20 or 100 or 250 different drugs/supplements as statistically independent, where if there's a 5% chance of each one working then there's a 1-(1-0.05)^20 = 64% chance that at least one out of twenty will work. If the first drug fails, it's possible that it failed because the postulated underlying mechanism was wrong, and three of the other drugs had similar postulated mechanisms so they're now out. Or it didn't work because the test methodology was faulty in a way that exaggerates effectiveness by 5%, and six of the other drugs were tested using the same methodology.

And on the flip side, side effects are rare and multidrug interactions are rarer, but twenty drugs gives you 380 possible multidrug interactions.

Doing a proper quantitative analysis of the linked probabilities would be almost ineffably hard. But to the extent that there's an argument for Pascalian medicine (and I think there is), it's for taking a few of the most promising and most *differently* promising drugs, not for taking everything that ever showed an effect.

Expand full comment
founding

What do you think of traditional Chinese medicine in this context? I had the impression that TCM had a lot of similar characteristics with this approach: the treatments are based on a combination of plausible explanations for why things might work and anecdotal evidence; patients typically take a bunch of things at the same time all of which are generally safe and might or might not help; TCM treatments sometimes turn out to do well in western-style clinical trials and sometimes they don't and also sound ridiculous. TCM doctors have some way of choosing the "right" combination of things to give each patient...

Expand full comment

"There’s a potential compromise solution, where smart doctors come up with Pascalian medicine protocols for the few patients who would actually want them."

Congratulations. You just invented naturopathic medicine.

Expand full comment

My mum has been recommending for years that we take Armaforce when we feel fluey. I would always refuse her offer of, as I called it, "snake oil".

Armaforce is a combination pill of andrographis, echinacea, olive leaf, vitamin C and zinc. As far as I know, none of these chemicals are really proven to help with the common cold all that much. There's a little evidence for each of them, but not a slam dunk for any.

Anyway, I finally listened to her and took the damned pill, and what do you know: it works wonders. Once I discovered how useful it was, I constructed a justification for it that approximates your description of Pascalian medicine. I now take it whenever I get that familiar scratchiness in my throat, or when I feel that a lack of sleep may have left my immune system a little off its game. And I get sick far less than I used to.

100 long-shot supplements might be too many; but for me, at least, five has hit the sweet spot.

(I still refer to it disparagingly as "snake oil", but now with a hint of irony and self-deprecation.)

Expand full comment

I take an approximately Pascalian approach to depression. Not to the extent that I’m taking twenty things, but to the extent that I looked over the preliminary-noisy-low-level-promising-finding tier of treatments, picked the half-dozen or so that seemed to me to be lowest risk due to already being in *very* wide use for other purposes (creatine, fish oil, vitamin D, magnesium, melatonin, saffron), or an extremely slight chemical tweak on same (l-methylfolate). In addition to conventional antidepressants, ketamine, talk therapy, and other well-studied lifestyle stuff like intensive regular exercise and mindfulness meditation.

This was less an Insanity Wolf-style ‘REVIEW THE ENTIRE SUPPLEMENT LITERATURE AT ONCE’ thing and more a matter of a decade and a half of reading a dozen journal articles or meta-analyses a month and coming to the one-by-one decision that adding One More Thing to the regimen looked good from a risk/reward perspective. I might be a little more aggressive after reading this, actually- the ‘substantive and meaningful drug-drug interactions are rarer than you’d think’ bit is very interesting, and fear of interactions was one of the main things that kept my list of stuff to try from expanding faster (the others being the physical size of the box I use to organize my weekly medications, budget, and the desire to try only one new thing at a time to keep myself from being in the position of having significant side-effects and no idea of what to blame it on).

Certainly I’ll be looking for good reading material on polypharmacy.

Expand full comment

>When I reviewed ivermectin, I said I was about 90% sure.

When you make statements about ivermectin are you careful to separate its use as a prophylactic from its use as a cure?

Or not?

Expand full comment

Give the grant money to ivmmeta.com to research 10000 substances' effects on COVID. Overwhelm the system. DOS the science. Get a value for that prior.

Expand full comment

I'm not really understanding the application of Algernon's Law and anyone turning people away from taking vitamin D. From what I can tell there have been a large number of very well done studies which show there is widespread vitamin D deficiency.

It just seems like the idea of warning people away from vitamin D is a pretty poor example of something to warn people away from considering it is a widely accepted issue that people don't get or have enough of it.

I think a LOT of these studies fixate too much on the already sick as well. The arguments of 'being healthy' and 'not getting sick in the first place' through things like 'having enough vitamin D'....don't really get measured in hospital admission rates or negative PCR test for people who already fell into sickness.

I don't see how we can ignore that 90% plus of people in hospitals with covid also happen to have low vitamin D which is much higher than the 60-70% background rate of people who have low vitamin D. What possible cost benefit analysis would see anyone be discouraged from the general idea of supplementing with it?

Perhaps it isn't a perfect covid cure, but so what? There's no pascalian wager associated with supplementing vitamin D when you're deficient in it...and it might also help with many other things.

What kind of Pascalaian wager of harm are we making by exposing ourselves to all the pesticides and herbicides and such. Where is Algernon's law for all the crap in our environment which has been repeatedly shown to be harming us and killing people? And still none of it is regulated or tested or examined. I'm far more concerned about the hundreds or thousands of things thrown at us from our paint to known neurotoxins in our pillows in fire retardants, to asbestos and DDT, etc.

Only a few of those are household names, but where is ANY of this analysis for those things? If I take 1,000 industrial chemicals and throw them at people, all exogenous things we never evolved around....where's Algernon?

How can Algernon's law apply to sedentary indoor modern humans whose necks are bent at odd angles looking down at their black mirrors all the time with the idea....evolution has sorted everything out for us and everything is optimised.

It just seems like such an obviously preposterous idea with no nuance. Perhaps a human 100,000 years ago living outdoors and hunting and gathering across the African savannah had no problems with Vitamin D. Perhaps they also had far far fewer problems with their jaws being under developed, or scoliosis in their spines from sitting and inactivity, not being exposed to sundry toxic substances, chemicals, and endocrine/hormonal/etc. disruptors of our various bodily systems.

There is so much that is different and has changed very rapidly in our environment.

Any reasonable evolutionary look at humans would conclude that our environment has changed far far faster than even fast evolutionary ideas would project to account for. How could we possibly have genetically adapted to these moderate and soft forces in our newly toxic environments?

Are we adapted for phthalates and our shrinking taints and plummeting sperm counts? Are all these plastics and other endless novel chemicals all utterly harmless?

If we apply the methodology of skewing towards harms being more likely than benefits, then certainly the tens of thousands of untested chemicals and the ways they degrade and interact with each other which humanity just dumped everywhere...and every so often when people start dying fast enough and in concentrated enough numbers we notice and do things like ban asbestos or any number of other banned lesser known chemicals which gave everyone cancer downstream from the factory....and the probable hundreds of other chemicals which are hurting people in such extreme degrees and in more moderate ways.

All this science and knowledge and deep long analysis of things stops at the things which get studied. The endless exemptions and lack of testing for safety for so many things we're exposed to just goes....unnoticed. Hand waive away that problem...what?

Certainly the idea that there 'is no metabolic free lunch' from taking loads of exogenous drugs and supplements is an idea. But then taking that idea to a stupid degree one wouldn't take antibiotics or anything from outside your body. And certainly looking at endogenous compounds like vitamin D or eating certain foods would make sense.

But I think there is a layering issue of complexity where no metabolic free lunch applies from one perspective, but makes no sense in another...especially when one is talking about things that are missing from your lunch, such as eating it while outside or other nutrient deficiencies in the modern western diet which has proven itself to be a huge scourge in medicine with heart disease, diabetes, and probably certain forms of depression and anxiety coming from dietary issues for some people.

Smoking cigarettes has got to be bad for you if you have covid. Whatever else is going on with covid, it does impact the lungs. Does the concept of 'patients don't do what we tell them anyway' apply here? Does it change the truth of the matter if patient compliance is high or low?

But does this mean we shouldn't ask? Or if you do smoke you shouldn't stop for at least the period while you're sick?

Do we really need a study on this when we already know how bad smoking is for you? Or how bad vitamin D deficiency is bad for you?

Expand full comment

Fascinating post, BUT I'll be a non-technical pedant and note that a Pascalian approach to medicine might also imply rejecting medical assistance out of adherence to extremely ascetic practices.

Expand full comment

The thing I worry about with this whole poly-polypharmacy concept is EDCs - a dozen compounds that are more or less perfectly innocuous at the amounts we're routinely exposed to them (as far as the limits of their individual studies can tell), but taken together are responsible for something like a 50% decline in male fertility over the last 50 years.

What happens if you start taking 100 pills a day and then some weird additive effect like this crops up? What if you're perfectly healthy but your kids end up with some sort of weird issue due to the additive effects?

Expand full comment

Interesting view. Fact is in the beginning, people died without any treatment that can prevent them to become intubated. millions.

Its one to say, oh, we dont know if ivermectine works or not, but we dont have anything else, so why not lets just try it to save the patients.

and other to deny treatment, demonize/eliminate those doctors who were willing to treat patients and give life saving early treatment, and spread misinformation in the media about "horse dewormer"

That is criminal intent.

Especially since now we see Merck and Pfizer coming out with antiviral pills, that are supposed to tackle the same problem of early treatment, kill the virus before it becomes dangerous...

Expand full comment

So ironically people have actually run studies that ended up claiming positive anti-cancer properties of both onions and pumpkins…

https://www.sciencedirect.com/science/article/abs/pii/S0963996917301060

https://pubmed.ncbi.nlm.nih.gov/33978471/

I rank these both as providing low to no evidence though, since it is well known that "everything kills cancer cells in a petri dish".

Given the incredible variety of plant extracts available with touted health benefits for various things, Scott’s scenario is not hypothetical. People will and do market the most ridiculous things as having (very dubious) health benefits.

This also makes me sceptical that e.g. Kurzweil’s use of so many supplements should be described as rational - his behaviour is indistinguishable from that of someone (like my grandmother) who has decided for some quasi-superstitious reason to take a bunch of supplements.

Expand full comment

I practice a form of this on myself to treat depression and general fatigue. I take about 8-15 different supplements daily. I'm pretty sure half of them don't help at all, but I'm not sure which half. I can't really make myself get more than 15 pills down in a day, so if I find something else that looks promising I bump off the treatment I have the least faith in. Sometimes things immediately get worse, which provides evidence that whatever I discontinued was helping. Through this process I've found fish oil, magnesium, vitamin C, and lion's mane certainly help. No matter how much people swear up and down about it, vitamin D has no noticeable impact on my mood or health.

Expand full comment

A question I have is: What percentage of COVID patients are involved in a trial? If we have twenty to hundreds (from ivermectin and Vitamin D to onions and egg plants) of potential low risk chemicals to try out, are we? Given the number of hospitalizations in the US alone, the error bars on the effectiveness of anything that's already known to be generally safe for human consumption should be much lower than this post implies. It seems that every patient should be asked "Would you be willing to take pills from this bottle, which potentially will improve your situation, but will probably do nothing?" Each bottle would have some drug or control (eg, sugar pills), an identifier (check for allergies by entering them into a form which knows but doesn't tell which pill type it is), and after the patients illness is over one way or another, the data is recorded. This also may be the most ethical way to provide the placebo effect. Seeking better data to get the answers we want seems like a more productive line of action than choosing Omura's Wager.

Expand full comment

An interesting question is whether crank cures tend to be wholly fallacious or if they become well known because they work for a few persuasive people (but not for most people). As I've often burbled on about, I've found the Chinese traditional remedy of echinacea tea to be helpful. But in all the years I've enthusiastically blogged about it, only about four commenters have chimed in to say it works for them too.

Now it could well be that me and my four pro-echinacea supporters are deluded about echinacea's efficacy. Or it could be that it really does work for a vanishingly small percentage of the population, such as the Chinese sage who originally promoted it, me, and four people who read me, but not for most folks.

I've long been interested in figuring out a general theory of crank cures, but haven't made much progress on the question.

If any of you want to be the Darwin or Foucault of crank cures, please have at it.

Expand full comment

One issue is the slavish adherence to the dogma that all rigourous quantification of uncertainty has to be probabilistic. So saying 75% Vitamin D doesn't work automatically implies that 25% it does work.

No. You have a lower probability bound on the chance of failure of Vitamin D, not a complete probability distribution. We simply know that at best there is a 25% percent chance that Vitamin D might work.

Expand full comment

If "Insanity Wolf" really does have a valid argument for taking every medication all the time, shouldn't you call him(?) "Cooly Rational Wolf"?

Expand full comment

I read the linked article on LessWrong and this paragraph made me wonder if it is logical:

‚For a true Bayesian, it is impossible to seek evidence that confirms a theory. There is no possible plan you can devise, no clever strategy, no cunning device, by which you can legitimately expect your confidence in a fixed proposition to be higher (on average) than before. You can only ever seek evidence to test a theory, not to confirm it.‘ https://www.lesswrong.com/posts/jiBFC7DcCrZjGmZnJ/conservation-of-expected-evidence

I do not agree. Arguably, my priors are also based on a sample size. For example if my priors are based on N=1000, then one additional datapoint should adjust my believe by 1/1001 * effect size.

No?

Expand full comment

i am partial to the explanation that one should put lower probabilities on things with shody studies working.

to a bayesian the details matter a lot.

what is the probability that an egyptian study with n=22 finds an effect size of X of dryg z on sickness y, GIVEN the true effect size is T?

the above is the question one should truly ask

Expand full comment

Oh, got it. Many thanks for the clarification!

Expand full comment

"When I reviewed Vitamin D, I said I was about 75% sure it didn’t work against COVID. When I reviewed ivermectin, I said I was about 90% sure.

Another way of looking at this is that I must think there’s a 25% chance Vitamin D works, and a 10% chance ivermectin does. Both substances are generally safe with few side effects. So (as many commenters brought up) there’s a Pascal’s Wager like argument that someone with COVID should take both. The downside is some mild inconvenience and cost (both drugs together probably cost $20 for a week-long course)."

Yes, so the good thing is, if you don't bother with Vit D & Ivermectin, even if you die, you've saved 20 bucks.

Expand full comment

Thanks so much for all of this. I'm one who doesn't like taking drugs. But I will admit that when I feel a sniffle or there is covid peaking nearby, I'll pop a multi-vitamin, D and zinc. I'd do a prophylactic does of ivermectin if I had one. No harm, no foul. I just wanted to observe that the placebo effect is big (and useful?). By your pointing out that I'm deceiving myself abut D and zinc, I wonder if you are robbing me of the potential placebo effect. Do you ever think about it this way?

Expand full comment

Corrollary of Algernon Argument: Sure, you started out with a pretty convoluted balance of elements that left you in a more of less optimal state. But you've been messing with that state since the time you were born. With diet, with exercise and with drugs, usually in non-optimal ways. So it is entirely likely that most people have at least 1 or 2 imbalances that they could fix with supplements and thereby gain improvements.

Of course, if your balances are already skewed, dumping 20 odd supplements into your system is unlikely to restore it, and most side-effect statistics assume 'normal' health, and those would be amplified as well.

Also, this modelling assumes that the effects of each supplement are independant, right? Since supplements don't usually have contraindications, this might not matter at most scales, but at 20 (or 250?!), don't those interplaying effects also need to be accounted for? Is that even something we can model with accuracy?

Expand full comment

Non-sequiturs:

Current hot hypothesis in autism research is that short interpregnancy intervals lead to autism via folate deficiency which vaguely is upheld by subsequent trends of osteoporosis in those mothers

https://pubmed.ncbi.nlm.nih.gov/29164825/

(idk, but since Scott is really into birth order) Also over supplementation can be detrimental to neurodevelopment (murine models) and can allow growth of nascent tumors - it's actively discouraged for cancer patients and the elderly

Also fun fact, one of three cadavers has thyroid cancer. I think all MDs know.

Expand full comment

This reminds me of a report I heard on NPR about a clinic in California that treats people who think they might have CTE from repeated brain injuries, but we can't prove that without an autopsy and have no treatments anyway, by basically giving them a whole bunch of random treatments that almost sound like they might work but there's no real evidence for but probably aren't harmful, from trans-cranial magnetic stimulation through dubious herbal supplements through "diet and exercise". It's probably a scam the way it's done, and you could do many of them yourself and save several thousand dollars on a consultation, but I had trouble saying confidently that it wouldn't *work*.

https://www.nprillinois.org/2021-11-22/a-market-of-dubious-remedies-has-sprung-up-as-more-everyday-people-fear-they-have-cte

Expand full comment

This feels like a cop-out to me. By conflating all early treatment medicine together and pushing it down a slippery slope you can then stand on the other side of medicine guzzling junkies and appear the more reasoned one. Though I got to admit it's funny and enjoyed reading it. Still, if we know Covid mostly strikes those with immune system deficiencies then it stands to reason all the minor things a person can do to boot their immune system could have an effect. If being outside can reduce transmission to zero, if wearing a mask or running fan can help, then having healthy levels of vitamin D can help as well. And a drug that may slow down the replication of the virus by let's say 15 to 25%, could help, which could be the difference for a given patient. Should such immune optimizations be held up as a substitute for a vaccine...no. But to dismiss them is irrational from my view.

Expand full comment

A couple years ago I went full-scale Pascalian for my currently untreatable conditions of PSC + UC. I am taking about 10 supplements every day including berberine, SAMe, ALA, curcumin (yes) and others. On the balance, I can report it works for me so far, I am flare-free for a year now and generally feel better than at any point in the last 10 years at least. I ran my list of supplements by my doctor who recommended against one of them; a couple others I had cancelled myself after they seemed to make my liver worse.

Expand full comment

So what are the downsides?

Oh, there's lots!

The most obvious is kidney and liver damage. A lot of drugs are well tolerated as long as you take them in reasonable quantities, but can destroy your liver/kidneys if you take a lot of them. Taking ten drugs that all are individually well tolerated, but which can damage these organs if you OD on them, could easily basically be an OD that does the same thing - it's death by a million cuts.

The entire realm of drug interactions is also basically impossible to know, because the sheer combinatric matrix there is untestable. Take twenty meds at the same time, and you have 19+18+...+2+1 = 190 potential two-drug interactions. The higher order drug combinations are even more complicated, and may not be predictable just from the two-drug interactions. Anything which is biologically active could well have interactions, either counteracting the other things, or even counteracting something ELSE random (like, for instance, if you end up getting prescribed an antibiotic, it might have some weird interaction with some medication you are taking).

A third pontential issue is screwing up your gut microbiome. We don't exactly know what all your gut microbiome does, but throwing twenty random drugs at it at the same time seems like the sort of thing that might screw it up.

A fourth issue - especially for things like antiparasitic drugs - is that taking them all the time causes everything else to get resistant to these things. So giving these drugs all the time could easily lead to the pathogens we are trying to protect against becoming resistant or immune. This is why we don't take antibiotics 24/7 (well, one of the reasons) - it puts a kind of evolutionary pressure on pathogens we *really* don't want.

Things that you ingest normally - like, say, Vitamin D and Zinc - are PROBABLY okay if you stay relatively close to the DRA.

Taking well above the DRA is a bad idea, as some of these chemicals ARE possible to OD on - for instance, Vitamin D has both acute and chronic effects if you OD on it, and it isn't actually that hard to OD on Vitamin D - standard blood concentrations of Vitamin D are 20-50 ng/mL, and we start seeing toxic effects at 100 ng/mL. There's actually a lot of things we take where the toxic dose is only a low multiple of what the theraputic dose is.

On top of this, some micronutrients are much easier to get rid of than others, so bioaccumulation can become an issue when you oversupplement even when the acute toxicity is low.

And animal models won't always save you on this stuff.

For instance, Llamas seem to be vastly more tolerant of vitamin D overdoses than humans are. Why? Who knows. I only know this because I know one of the researchers who worked on trying to find the LD50 of Vitamin D in llamas and gave up after repeatedly doubling the dose; IIRC the final dose they gave them was completely ridiculous (IIRC 64,000 IU per kg of body mass) injected directly into their bloodstream.

For reference, the human RDA of Vitamin D is 600 or 800 IU per day. And no, that's *not* per kg for humans.

This applies to everything. There are artificial sweeteners that will cause organ damage in rats that humans are completely unaffected by. There are surely things that work in the opposite direction.

Expand full comment

I think it's worth pointing out that fluvoxamine (your example of a success) has quite a few significant side effects. I find it much less well tolerated than other SSRIs I prescribe.

Expand full comment

A lot of this comes down to risk aversion, both at the personal and professional levels. I don't think you'd find many doctors who want to put their career at risk by prescribing experimental remedies, even if the regulatory framework was more open to this. On the patient side, who wants to take the risk that their doctor is a quack giving them something where the risks very much outweigh the potential benefit? Even if there are benefits to society from loosening up on both these fronts, it's very difficult to chip away at these biases.

You see the same thing in other fields; finance especially comes to mind. I'm a bit of an investing geek, and do some unconventional things with my portfolio. I like the risk/reward tradeoff of the unusual choices I've made, but in managing my partner's portfolio I go with something very standard, because what if I'm wrong? Professional investment managers deal with this all the time - it's called "career risk" in the field. It's very hard for them to stick to contrarian strategies when they have a benchmark to track. There's a tendency to herd in the conventional path even against one's better judgement, because performing poorly alongside everyone else looks a lot better than performing poorly on one's own.

Expand full comment

Even if you take all twenty medicines, and doing so "works" (people who take the dosages don't get the COVID, even after multiple high-intensity exposures), what's to say that it's not the combination that "works"?

Expand full comment

I'm a big believer in science, rationality, common sense (no question - my own type of common sense ;-) )

That said: I learned over the years that in the end its all about trust into certain people and or systems systems and distrust others. All our high IQs, information, research and much much more counts not much. In the end its about trusting or distrusting.

Thats kind of good and bad at the same time for me.

Expand full comment

"Futurist Ray Kurzweil used to take 250 different supplements every day - but after realizing this was excessive, cut it down to only 100." was wondering the same - "maybe he really has found a secret or even the holy grail of liefe and death and I should do like he does" - some years later seeing an image of Kurzweil and thinking "looks pretty old this guy for his age - ok, let's forget that stupid 100+pills idea"

Expand full comment

The Placebo Effect and the affects of prolonged anxiety/fear are both well established, aren't they?

Combining the beliefs that 'they put it on the test swabs' and 'God decides when its your time' would seem to be more statistically effective than fear and supplements.

Expand full comment

"We don’t fret over the unknown unknowns of Benadryl or Tylenol or whatever, even though we know their benefits are minor."

Speak for yourself... I take Algernon's law seriously, and tend to avoid low benefit medications because the expected value doesn't favor them. I also do my best to approximate an evolutionarily informed diet (i.e. paleo, but making allowances for neolithic foods my ancestors ate for many many generations).

Expand full comment

I have long Covid, doctors are useless, and I have had far more luck with folk/quack remedies off of the internet than I have with anything doctors have recommended. If you actually have a medical condition which doctors have not and are unlikely to help you with, Pascal's Wagers are totally rational. The way to avoid being mugged by Pascal's quacks and avoiding tail risks is to do enough research to create a hierarchic list based on a cost/plausibility ratio and go down the list.

Expand full comment

Commenting a bit late, but this is basically my approach to supplements, with a focus on longevity and general health. I am nowhere near 100 myself (currently at around 10, peak was around 20-22), and if anyone is curious I wrote a brief (albeit slightly outdated post) outlining all of them with some reasoning as per why I take them(with the more interesting things on the bottom of the post. also keep in mind this is personal to myself!): https://nearcyan.com/supplements/

It does get interesting when you run cost/benefit analysis on things like this though, and I do have some uncertainty about the best ways to meta-manage this myself as well.

Expand full comment

scott mentions negative interactions, but what about positive interactions? If we see a number of proxies improving in response to the drugs, maybe some of them are hitting real parts of the problem, and in combination they'll have a real effect?

Expand full comment

That's a perfectly reasonable argument for why doctors shouldn't prescribe IVM / HCG + Zinc by default.

It's not the least justification for doctors refusing to give the prescriptions to people who want them, or for pharmacies to refuse to fill those prescriptions

Expand full comment

Its possible for Kurzweil to be entirely rational in taking 100 supplements to slow down ageing, and for you to be entirely rational in ignoring the weak evidence for ivermectin, under certain priors. Specifically, given that you know that vaccines work generally and for covid, it is much less likely that ivermectin works than would be the case if vaccines didn't work. Vaccination validates a far from complete, but still substantial causal map of the mechanism whereby covid operates, and we can map this to facts about infectious agents that have broad support. What we know about ivermectin has no causal link to this picture, and would imply a potential large set of unknown mechanisms at work. We have a deep prior that reality displays a causal parsimoniousness (even in biology). I don't have a fully developed argument as to why, and how justified this is, but I suspect that its quite hard to do science without it, and that the success of science operates as a macro confirmation of this prior.

Expand full comment

Normally wouldn't mind philosophical debate on fringe virus treatment, but every ounce of validation for these kinds of treatments means more folks won't get vaccinated, and more of either them or someone they contact will die. How about an essay on anticipatory regret - in effect taking every precaution so you don't unwittingly hurt someone?

Expand full comment

Wrote a short response to this arguing in favor of this approach for personal health/longevity if anyone is interested: https://nearcyan.com/pascalian-longevity-why-not/

I'm not sure at which point calling the approaches 'pascllian' becomes unfair (because the probabilities are much larger than that in Pascal's wager), but we still underexplore this search space in many regardless, so I think it's a good thing to encourage more usage of expected value and probabilistic approaches to these things.

Expand full comment

> I know of only one person who takes the Pascalian argument completely seriously. Futurist Ray Kurzweil used to take 250 different supplements every day

Well, my chronically ill mother took nearly a hundred different supplements a day for 30 years. She stayed more or less sick (it's been somewhat of a roller coaster). Recently she's started to have mini-strokes and had to be placed in a care home.

Expand full comment

some intuition for why maybe the 20-drug insanity wolf position isn't quite so crazy (aside from the efficacy claims in the meme):

We already consume tens of thousands of different chemicals every day in our food. If you plug in a prior of 0.01% that two random chemicals will have a harmful interaction, then you get a ~100% chance that our food will kill us. That prior must be wrong when applied to low doses, or when applied to chemicals that our ancestors have consumed for millennia.

Eating 20 more chemicals doesn't seem so significant in that context, as long as the dose is lower than what was well tolerated in studies. Also many/most of the supplements exist in food (like cicrumin in your curry). A thousand year history of consumption as food counts as much more evidence of safety than a few crappy studies that were probably funded by whoever is selling the supplement.

Expand full comment

Well, Ivermectin apparently does work if you have worms and need it to fight off the worms when the corticosteroids are used to dampen the cytocine storm. So there is a possible causal effect, under which it does work.

And vitamin D does work if you lack vitamin D — just like zinc and vitamin K2 and C, it is part of what the immune system needs to work. If you are a healthy adult during summer time and take enough sunlight, you don't need vitamin D. Nor do you need K2 (produced by bacteria in the intestines) or C (because you eat enough fruits), nor zinc, when you eat enough of it. But in winter and as old aged person, you have very likely already such a severe lack of vitamin D that your bones start to crumble, and probably your eating habits are not healthy, so you lack other things, too.

And this is a condition that is far more likely than the worm infection.

So, no, you should not use this mental “chance” model. You should try to understand why a drug works, and if it only works under some conditions, use the drug only under these conditions.

Studies in worm-infected countries closer to the equator probably find more likely that Ivermectin works, while studies in northern countries have a higher chance to find vitamin D works — if the supposed effect is real. So you not only have to look if the study is performed with good standards by trustworthy people, but also where it was done, and if the underlying conditions for the drug to work are likely. Actually, the people doing the study should do that analysis in the first place.

In the end, the human body is complicated, and therefore, treatment is not always simple and unconditional. You have treats with a high success rate (like vaccination), but the one immune evading mutant during your phase III test in exactly the country where you did test it can ruin your results. You have treats with a low success rate (like Ivermectin), because the conditions under which that works is rare, at least in western countries.

Expand full comment

1 Million supplements with1 Gram each makes you take in 100 Kg of supplements every day. Killing you with high probability.

Expand full comment

Isn't the leading recommendation in medicine for a Pascalian Diet?

Your safest bet is to each as wide a variety of foods as possible(excluding those that are harmful).

Maybe I'm mistaken that's the scientific consensus, but I've never seen any argue against it or recommend a very narrow diet of a few specific things that proven to be very healthy (except as extreme weightloss diets).

So aren't the onion farmers already money pumping us with the Pascalian Diet?

Why not let them money pump us with Pascalian Medicine too?

Expand full comment
Jan 3, 2022·edited Jan 3, 2022

Re. "I’m nervous about this scenario because it violates the Law Of Conservation Of Expected Evidence - if I “know” that onion farmers doing studies will convince me that onions have a 5% chance of curing cancer, I should just believe there’s a 5% chance onions cure cancer now."

There is no violation of the Law Of Conservation Of Expected Evidence. I assume you mean that the studies will conclude onions cure cancer if p < .05, so that 5% of the studies will conclude onions cure cancer if they don't, and that will convince you there's a 5% chance onions cure cancer.

No. Just do the math. A meta-review of these studies should look for the risk ratio multiplier that treatment of cancer with onions has for, say, death as outcome; then conclude that the best estimate of that risk ratio is R such that P(outcome | R) is maximum. If the studies pass a p < .05 test 5% of the time, I'm pretty sure that will work out to R = 1, meaning our best estimate is that onions have no effect on the risk. If they pass it less than 5% of the time, then R > 1, meaning our best estimate is that onions increase the risk of death from cancer.

When you phrase the question in terms of "probability onions cure cancer", you're making an asymmetric test, ignoring the possibility that onions make cancer worse. The case where R = 1 is presumably where the expected risk from onions cancels out the expect benefit from onions.

Expand full comment