327 Comments
Comment deleted
Expand full comment

1. There seems to be a confusion between "prediction markets will be accurate" and "prediction markets will be accurate after the fact."

2. The whole point of insider trading is that no one else knows about it. To argue that some insider is trading would be valuable information for prediction markets is both accurate and incoherent.

Expand full comment

I don't know about 'canonical', they're subject to manipulation too. I think they're useful as an additional source of information where people involved have real 'skin in the game'.

Expand full comment

I'm pretty sure I lifted the hierarchical court system, where each court is theoretically a prediction market about what the next higher-up court will say, up until you reach the Court of Final Settlement (Supreme Court), from somebody else. Possibly even Robin Hanson.

Expand full comment

I have a strong intuition that "people who have money to spend on prediction markets" and even more "who also have an interest or knowledge of prediction markets" is an incredibly biased sample of people.

I'm just very pessimistic on the ideal vs the practice. The same kinds of negatives I consider for the stock market. Or markets in general. Prediction markets depend very strongly on right wing economics being accurate imo.

Expand full comment

> how likely to the claim is to be true.

Typo, that first "to" shouldn't be there.

Expand full comment

My main question whenever I read about prediction markets is: why are they necessary? The best answer I can surmise from *light* googling is that it can serve as the most accurate representation of the wisdom of the crowd. Is that a good one-sentence answer to the question: "Why prediction markets?"

Expand full comment

>As a test, I tried to manipulate the market on whether Austin Chen, founder of Manifold Markets, would be charged with a felony.

If I recall correctly, you did this with his explicit permission? It would be worth having that fact displayed next to that sentence if so, since otherwise this reads as a minor alarm bell.

Expand full comment

I just wanted to pay a compliment. This is so beautifully written. A complex concept, simplified perfectly, without it being dumbed down.

Expand full comment

Thanks for writing this! If only we could tag Rostin Behnam :P

In addition to betting on Manifold, ordinary folks in the US and abroad should also participate in the markets on Kalshi, Polymarket, Insight Prediction and PredictIt. Real money is the ultimate play money, as a reputation system that most freely transcends the prediction platform. You don't need to be a hotshot prediction expert - skin in the game allows you to learn quickly about your interest.

To promote prediction markets from a regulatory standpoint, the CFTC cares about the economic purpose of these prediction markets, namely their use to hedge risk and serve as a price-basing mechanism. Directly using these markets for information or to hedge risks and demonstrating a positive economic effect to your life and business is a central legal argument to getting these markets approved!

Expand full comment

I had a concern with prediction markets that I think answered myself. Still posting it here for edification and potential correction if my answer was wrong.

> One concern I've had with prediction markets that this post doesn't address: is it good to expand the set of things that companies can hedge against?

> Let's say that I'm Satya Nadella and I'm worried that a startup might disrupt and take over Microsoft Office. Today, I can action that worry by investing into Microsoft Office, investment that presumably makes Microsoft Office better, increases the productivity of its users, and therefore raises the GDP of the human race.

> Let's say we live in Scott-topia and that I'm Satya Nadella and I'm worried that a startup might disrupt and take over Microsoft Office. I might decide that instead of investing in Microsoft Office and reducing the dividends paid to the shareholders that employ me, I'll buy a large stake in "yes" for the prediction market "[w]ill Microsoft Office make at least x bajillion dollars in fiscal year 20XX?" so I'll be set no matter what.

> To phrase my objection more cogently: what propels our society forward are the holes in the efficient market hypothesis; the innovations that are not priced into asset prices. Will large companies ability to hedge against those reduce the incentive to create and fund those innovations?

I think the answer to this objection is that the startups/VCs will take the opposite end of the trade with Microsoft and it'll end up being a massive waste of time for everyone involved (since they could just invest the money in their products) so they won't do it.

Expand full comment

This isn't central to your argument, but:

"Finally, I envision that someday people who want to know the answer to specific questions can subsidize prediction markets on them. For example, the Democratic Party might subsidize a market about which Democratic primary candidate is most likely to win the general election."

-- how would this market resolve? Only one person is going to get a chance to win the general election.

Expand full comment

I'm not crazy about the first sentence "Prediction markets are like stock markets, but for beliefs about future events." There are two crucial differences: most prediction markets have binary outcomes and they resolve at some point. This is not the case for the stock market. While most Americans are not familiar with binary options, I don't think using the stock market as an analogy is accurate enough. I'd just compare it to sports betting, even though if it may come with a slightly negative connotation.

Expand full comment
Dec 20, 2022·edited Dec 20, 2022

Couple things...

- Much of this just seems like "efficient markets hypothesis" repeated in different ways. Is that not right? Is there a reason to think it'll be more accurate than the stock market's predictions? The only one I saw is the point about how prediction markets can't have a ponzi scheme go on forever, but that applies to certain stocks as well, and doesn't seem like a super broad objection in any event.

And the thing is that (1) I don't think most people would call the stock market "accurate" or "unbiased" except perhaps in a hypertechnical sense, and (2) the stock market needs a lot of TLC to make sure it works properly. There's insider trading, but that's hardly the only example. In fact, to go back to the crypto thing, part of the objection and the reason (allegedly) that it seems like such a shitshow is that they DON'T have the regulations the stock market has.

More generally I bet a bunch of economist PhDs have written a lot of useful shit about the efficient markets hypothesis, that this could probably benefit from some engagement with, if I'm right about this being similar.

- Is the idea that prediction transactions will be public? There's all this stuff about how you can follow Nate Silver's lead, but does Nate Silver have to reveal who his bets? What if he's just quietly sitting there, anonymously betting, not making enough money to have anyone notice?

- I think "in the long run" is perhaps a larger qualifier than people are assuming. To take presidential elections again, they are every 4 years, and the political landscape changes over time such that there's never all that big a sample size of analogous presidential elections to the current one. If Nate Silver is the best predictor (to go with the analogy, not saying it's true), maybe his biases simply match the current landscape and it'll be different in 8 years, or maybe given the small sample size (4 elections?) and the inherent uncertainty of judging percent predictions for binary outcomes it's simply hard to say who is best.

- As a test, I propose something other than big picture geopolitical developments that people normally associate with prediction markets. Sports! Much higher sample size (2430 regular season baseball games per year, and that's just one sport, and the outcome is numerical and unambiguous!), and there's already a lot of predictions out there you can compare to. Compare the sports prediction market to Vegas money lines/odds, pundit predictions, etc.

Prediction markets AFAICT have (in theory) an advantage over Vegas odds, which is that the house edge probably negates the positive EV from certain arbitrage strategies, which probably makes them less efficient in theory.

- Regarding the "zero sum" point, can't you take the money from the primary (i.e. not secondary) purchasers of prediction "shares" (or whatever you call them) and invest them in US government bonds, and the interest is paid out proportionally to the winners?

Expand full comment

"... suppose that an import-export business spends millions of dollars betting that Trump will lose in order to hedge against his protectionist policies ..."

Wouldn't that business want to bet he would *win*, not lose? (That is, if the policies of Trump's main competitor would be more beneficial, on balance, for that import-export business.)

If Trump wins, the business would presumably lose some revenues and profits due to his policies. But winnings from their prediction markets bets on his election victory could at least partly offset that.

(If I'm confused and have misunderstood this, welcoming corrections!)

Expand full comment

I think that, in the ideal case, prediction markets will work as you say, and that would be a good thing. Likewise, in the ideal case, cryptocurrency would also usher in an era of decentralized prosperity, and that would be a good thing too. But in practice, as soon as large sums of money (even virtual money) is in play, the entire utopian landscape shatters into a hodgepodge of Ponzi schemes and general fraud. This is the case with cryptocurrency; this was the case with the stock market; and this will be the case with prediction markets if they ever gain enough popularity to be useful.

In the case of the stock market, the problem was somewhat improved by regulation. There are still scams and clever get-rich-quick schemes, only now they have to be a lot more clever, and perhaps valuable enough to buy off a regulator or two. The stock market sort of works, but it would be foolish to claim that the value of every stock accurately reflects the true value of the company it represents.

Sadly, I don't think prediction markets will ever get even to that level of accuracy, because the incentives are much more entangled. Arguably, a share of "Joe Biden will win the election" is, indirectly, a lever that someone can pull to influence the election; thus, regulators with their thumb on the scale would wield considerable power. Selling the election on the conventional stock market is much more difficult (if not outright impossible).

Expand full comment
Dec 20, 2022·edited Dec 20, 2022

This is incredibly and informative and well written. However, to the extent that I find prediction markets objectionable, it has very little to do with their predictive power, and more to do with the aesthetics of how the analysis is presented. I frequently, both here and in other rationalist spaces, see potential human disasters referenced in terms of prediction market odds. Whether it be numbers of future Covid deaths, mass shootings or the likelihood of Putin launching a nuclear weapon, someone is bound to bring it up in the context of prediction market odds, and those odds, definitionally, are defined by people who are framing the possibility of vast human suffering preliminarily in the context of whether they can make any money out of it.

I understand that the predictive power of the markets still have value in these cases, but even so, there's an "ick" factor I can't get past. It feels like reducing potentially devastating events to data analysis/investment opportunities without acknowledging the human toll plays into the worst stereotypes about the rationalist community.

Expand full comment

You draw comparisons between prediction markets and stock markets, but I’m curious if you think sports betting is another useful analogy. The global market for betting on, say, the World Cup outcome is probably orders of magnitude larger than any of the relatively niche prediction markets you highlight. Based on these answers, I would expect one of four things to be true:

-The betting market is an accurate, unbiased representation of which team is likely to win.

-The betting market has a consistent, material bias, so Goldman Sachs hires quants who are experts at soccer to take smarter bets, make tons of money, and correct the market.

-The betting market has a consistent bias, but Goldman Sachs isn’t exploiting it, so there’s a ton of money on the table that someone could pick up tomorrow (or at least in 2026)

-The betting market has a consistent bias, but it’s not viable or worth it for anyone to correct it (like the prediction markets you mentioned today)

As far as I know, option #2 isn’t happening. I could be wrong, but I’ve never read outrage bait stories about how Wall Street won big by betting against America or something. And if this is just because of regulations, I’d still expect someone to try it by setting up shop in the Bahamas.

If option #1 is the case, do we see the benefits you predict? I’m not well versed in the world of sports, so this is an honest question. Do team owners look at Draftkings picks to decide which players they should keep? Do coaches see terrible odds against their team and decide to change strategies before they even play? If so, this seems like a strong argument in favor of prediction markets, and one you might want to highlight in another revision. But if not, I’m curious why you think these markets aren’t living up to their potential, and whether you think this is a counterpoint to your optimistic take on what markets can do.

If option #3 is the case, why aren’t you making tons of money on Draftkings? And if it’s option #4, then what problems do you think are holding back sports betting markets, and why don’t you think other prediction markets will suffer the same fate?

(I suppose option #5 is that this is a flawed analogy, or that I’m missing another obvious takeaway.)

Expand full comment

I want to preface this by saying I like prediction markets and think they are a net good. But I think there are some clear problems with them that you've missed or hand-waved away in a fashion I don't find convincing.

For example: "4.2: Would prediction markets encourage harmful or illegal activities? ...I think the strongest evidence against is that this basically never happens in stock markets."

I think this is an awfully rosy image of stock markets. Because of many traumas over many decades we've built up complex systems of financial regulation that make stock markets less prone to exploitative harmful schemes than e.g. crypto is - but the natural state of a financial marketplace is to be scam-infested.

So it's illegal to run a "pump and dump" scheme in the stock market, but there's no such prohibition on doing it in a prediction market. Once real money starts going into these markets, you'll definitely see people doing things like buying up a bunch of shares in an outcome, creating fake hype to get people to believe that it's a real market trend, and then dumping their shares at an inflated price before reality sets in.

And of course there's any number of other market manipulation tactics that are known. You can set up a government agency to police these, and it might even work some of the time, but do bear in mind that the Securities and Exchanges Commission has a $2.5 billion annual budget. It's not enough to say "pump and dumps are banned", you need to have people with the job of prosecuting them, and that cost is not negligible. It might be worth it! But it's not negligible.

I also think that in "4.8: “Meme stocks” like Gamestop and AMC sometimes remain mispriced indefinitely. How do we know this won’t happen with prediction markets?" you don't adequately account for the "Market can stay irrational longer than you can stay solvent" effect on markets with resolution dates long in the future, or conditional resolutions that are likely to not be met. You mention these sorts of concerns elsewhere in the FAQ, but don't apply them to the failure mode where they are relevant!

Personally, I think it's absurd that Tesla is valued higher than all other automakers combined. But I don't short Tesla because I don't have any way to prick the bubble and force the stock price to reflect Tesla's real value as a company. There's no moment of truth - I could think the market is deluded, be correct that the market is deluded, and lose all my money betting on the correct value because it takes too long for the market to realise I'm right.

Now, betting on an election outcome in a couple of months doesn't have that problem - we know that the tide is going to go out and we'll get to see who's been swimming naked. But betting on, say, which candidate is more likely to win if nominated DOES have that problem.

Suppose the market is convinced that Clinton will be a stronger nominee than O'Malley. I'm Matt Yglesias, I'm sure that O'Malley would win, and I also happen to be right. But O'Malley never gets close to the nomination, and my correct belief never has the opportunity to be proven true. The market stays mispriced until it expires. My correct bet lost me money, and the market never accurately reflected the reality that O'Malley Would Have Won.

Now, these are sort of edge-case scenarios and I do agree that prediction markets are very good in modal cases. But I also think the edge-case problems are real.

Expand full comment

4.6.1.1 - This claim that hedging won't bias the markets seems way too rosy. There's not a bunch of people who all know the exact probability and one person using the market for hedging. There are a bunch of people all with different guesses as to the probability and different budgets/amounts of acceptable risks about how much they are willing to invest to correct errors. The (perhaps money weighted) average of these people is the "correct" value that the market should be trying to converge to. But if someone is investing a bunch of money as a hedge (rather than an honest prediction), this will tip the scales somewhat one direction or the other.

Expand full comment

4.8: Perhaps the better argument here is that it is *much* easier to bet against a meme prediction market than against a meme stock, making it a lot easier for the market to correct itself.

Expand full comment

Predictions markets that only evaluate if the chance is high enough (such as in the case of your comment policy) have some weird effects that I think could invalidate the results, and might not be fixable with the usual "free money" argument. If I wanted to make sure that a comment never reaches your attention, I can vote it down below the threshold for it to evaluate, and even though I am totally wrong on my "prediction", I'll never lose any money. There is still a chance at free money, but now the consequences to being wrong are removed (as long as you are rich enough to overwhelm the market). Perhaps this works better than I am thinking in practice, but especially for low traffic votes like this I think it would be easy to pull off.

On the opposite side, if I predict that you **won't** think that a comment was worth your time upon review, I have no incentive to vote negative below the threshold, since I won't get any of the money I am expecting to make. If I have sunk enough money in, it might even be worth my while to vote positive (against my prediction) in order to get it back above the threshold so that my bets get evaluated. I think this is less easily resolved than the first issue, and you should basically never see prediction far below the threshold because all that does is tie up your money while you wait for the market to close.

Expand full comment

Prediction markets also monetise disinformation. Instead of looking for mispricing and then getting rich until the market corrects, you can spend money on a disinformation campaign to induce mispricing and get rich until it corrects.

This is not very different from pump-and-dump schemes in penny stocks, but providing a continuous, liquid, anonymous incentive to spread disinformation on *every* topic seems like it should be a concern. Not because the market won’t correct - because of the second order effects of increased information pollution everywhere else in our society.

In the optimal case this provides financial incentives for very robust anti-disinformation detection. However, solving loss of trust by drawing all of society and our information and decision-making platforms into a high-stakes HFT battle for truth might turn out to be like throwing water on an oil fire.

Expand full comment

I think I'm not understanding why 4.6.1.1 is free money. If I buy a share of Biden at 10 cents, I have a 50% chance of making a dollar, so my EV is greater, but I still have a 50% chance of losing money. Is the free money part:

(a) The EV on that trade being >0

(b) I can at some point sell that share to another trader for 50 cents

(c) I can take a strategy of buying shares for the less-protectionist candidate every 4 years and make money in the long term if hedging makes those candidates are consistently undervalued

(d) The import/export company is also hedging in other markets, so I can bet against them in every market and make an overall profit on those trades

(e) Various groups hedge various markets and I can take a strategy of betting against them in general and make an overall profit

Expand full comment
Dec 20, 2022·edited Dec 20, 2022

I think the real issue with making a better prediction market is that it's such a niche product. Looking at the top eight most liquid predictions on polymarket they currently have a total liquidity of about $136,000. And there's only about 60 questions total. Let's be generous and roughly double that for $250k total liquidity on site. That's tiny. That's smaller than most minor league baseball games. Let's say there are five equally sized markets. That's still only about a million dollars. Let's round to a million (fair since one of them doesn't use money). Given a casino's 8% take (which would actually probably be lower since this isn't a casino) that's $80,000 in revenue. And that has to cover everything for the entire industry.

I can think of ways to build better versions. But if almost no one's going to use it then what's the point? At those rates I'm not even sure the revenues can cover regulatory costs. Regardless of whether it's a good idea I'd need to be convinced it could survive and hopefully even thrive.

Has anyone considered getting all that rationalist money behind a prediction market in an already high demand area like sports? That's a multibillion dollar industry with heavy existing demand and high growth. Yes, it's not as EA-sexy as stuff about politics or whatever. But fundamentally you'd be developing the same type of skillset and organization and technology. All that which you could eventually pivot into broader markets. Part of the point is such markets are useful for predictions in arbitrary fields. So why not arbitrarily choose the large, fast growing market?

Expand full comment

Firstly, great post!

Many of these examples are political questions, which you would always expect the masses to be right about in aggregate, since the democratic process is about aggregating opinions. Some of them are less political. But what about a question like "will it rain tomorrow in New York?" To make a bet on this question, the only sensible strategy would be to check what weather modellers are saying and bet with them. But because this is the only winning strategy, would anyone actually make a bet here? I guess the thrust of my question is - shouldn't there be very little money to be made on problems that are genuinely hard?

Expand full comment

I know very little about prediction markets, but one big obstacle to putting any serious money into them seems to be the timeframes involved. Who wants to tie up their money into predictions that would resolve in ten years? Or even five years?

If I was *very* confident in a prediction that would be resolved next year, then yeah, maybe I'd accept freezing my money for that year. But there's no way I'd ever try to freeze up any significant amount of money for an absolutely whopping *five years*, even if the market seemed comically mispriced. And in practice, any mispricing you'd otherwise want to correct would probably be in the single digit percentages anyway, so freezing money for that long simply can't be reasonable. The current inflation rate is what, 6% per year?

So I'd expect prediction markets to be fine at short-term questions, but pretty bad at long-term questions, because inflation makes sure that nobody actually has a real incentive to try to correct a 10% mispricing five years out from the resolution time. Am I missing something?

Expand full comment

Is there a risk of rich people using prediction markets not just to make money but to influence real-world events?

Suppose the tobacco lobby wants the tobacco-friendly candidate to win the election. They bet lots of money on the other candidate and make the market imbalanced. Lots of ordinary people notice the imbalance and buy shares on the tobacco-friendly candidate. Come election day, the vote is looking close-run, but there are lots of ordinary people who own shares on the tobacco guy, which they probably paid between 10 and 50 cents for, so if he wins, they'll get a 2x to 10x payout, and if he loses, they'll lose their investment. That would be enough to make several people who were undecided about who to vote for, or who might not have bothered to vote at all, vote for the tobacco guy - which may well swing the election.

This is more than just the tobacco lobby hedging against an outcome they wouldn't like; this is them paying to influence the outcome of an election, effectively buying votes.

Expand full comment
Dec 20, 2022·edited Dec 20, 2022

"For whatever reason, most people interested in prediction markets are American, so Polymarket has a limited userbase."

This is by far the weakest part of an otherwise great post. If prediction markets are legal elsewhere and are in theory so great, then why aren't they big there already? Is Polymarket really freely available in the rest of the first world?

Expand full comment

I think your argument that prediction markets are hard to influence needs some more qualification. In particular, it's not that special interests can't manipulate the market. It's that they can't manipulate it in a way that causes it to diverge from the actual probability of that outcome.

For instance, suppose prediction markets get widly adopted and trusted and there is a prediction market for whether there will be a run on the banks in X country this year. If someone wants the banks to fail they could start buying up contracts that say the banks will fail. As that changes the price in the market which in turn makes it more likely the banks will fail it's not true that the market resists manipulation (even if you know what's going on and have a ton of capital it's not clear you don't also bet on failure).

This is important as it creates important limitations for the use of prediction markets in making decisions. If you do things like couple voting or government policy to prediction markets it may be that the markets accurately reflect probabilities but they may bring about undesirable outcomes in doing so (eg bank run example).

Expand full comment

When I hear about stuff like this I always get hung up on the biases of the arbiters. Obviously you can add more and more specific resolution criteria, but I would worry about a right wing market deciding that the answer to "will trump win in 2020" was yes, because the person in charge of the market believes the election was manipulated (or even just refunding it all rather than call a no). Or something like "will we be in a recession Dec 2022" being refunded only by some markets because one market determiner beleives the government has vested interests in not announcing one even though there's obviously one happening right now. Just disagreements on judgment calls, when to refund, and eventually you could get two markets, but left-market says candidate has 30% chance to win, right-market says candidate has 40% chance, and there's no free money to be made because there's a 10% chance that one market or the other disagrees on the state of reality by the election. At that point they lose the benefit of being authoritative.

And again, I know you can narrow it down much further than I did here in my examples but especially in cases of when to refund, it seems like there are too many outside influences to properly phrase all of them. Who are "experts"? Who are "developers"? what is "commonly understood"? What does "interference" mean? Does a tweet count as an announcement or does it need to be at a press conference? Does covid count as a "disaster" causing a market to refund and if so, at what point does it stop counting as a disaster?

That being said, a bunch of question styles dodge this problem pretty easily, the "what would I decide/what would justice decide" style questions would be safe.

Expand full comment
Dec 20, 2022·edited Dec 20, 2022

"As a test, I tried to manipulate the market on whether Austin Chen, founder of Manifold Markets, would be charged with a felony. There’s no reason to think he should be, so the price started at 5%. I spent $200 in Manifold’s play money bidding it up to 95%. Within an hour, other investors noticed the mispricing and corrected it back down to 5% again."

How did the other traders know that you were manipulating the market, instead of trading on inside information? Realistically they didn't, they were just trading on their priors. That worked out in this case, but if Austen really had been about to be charged, there wouldn't have been any way to find out about it from the prediction market faster than from the news reports.

In general there's some tension between the market's ability to resist manipulation and the market's ability to respond to inside information, since they look the same to other traders. I'd expect this to equilibriate in different ways on different questions, depending on how plausible inside information was: you probably can't manipulate the "climate change in 2030?" question, but probably can manipulate the "Biden to resign in 2023?" question.

EDIT: this affects the hedging analysis too, come to think. We should expect a position taken as a hedge to move a functional prediction market, because the other participants can't know that it's a hedge and not the product of private research.

Expand full comment
Dec 20, 2022·edited Dec 20, 2022

An important feature of prediction markets is that their prices ARE NOT probabilities. I think you are mostly careful to use words like "likelihood" and I appreciate the need to keep it simple, but there are several real world effects that distort market prices relatively to probabilities quite a bit and it is useful to keep them in mind.

1. Time value of money - one would rather have money now rather than later. This is measured by the risk-free rate of interest, which is now around 5% in USD. Rather than bet $1 to get $1 in 1 year time on a prediction market, one could invest $1 in a portfolio of very safe 1 year bonds and get $1.05 in one year.

2. Credit risk of the exchange. Betting exchanges go bankrupt, are hacked, etc. This is not a theoretical possibility - 10-15 years ago there used to be a decent prediction market called intrade.com. I made there roughly x8 of my deposit on a series of sports and political bets, but this amount was not high enough to bother moving it in and out of exchange when I was not trading, so I lost 100% of it when intrade.com folded. Rather than invest $1 in a portfolio of high quality bonds, one could lend money to the betting market, or a bunch of equally dodgy blockchain companies. I do not think that the interest rate on such loans (illiquid, unsecured, to non-transparent companies of unclear regulatory status) would be lower than 15% in the current market.

3. Transaction costs. There is a non-trivial transaction cost, both mental, time spent, and commission based required to deal with many of these markets. I think putting it at 2% would be conservative for relatively small amounts bet.

These three effects create no-trade boundaries for a rational bettor. I can think that my predicted probability of an event Q is higher than market price P but I would not buy unless the difference is high enough to compensate me for the effects of 1-3. And I would not sell unless Q was sufficiently below P. These boundaries become quite wide for large time to resolution, but they are non trivial even for relatively short times. For example, using the parameters as above for a two year market, one gets no-trade boundaries of (-0.28, 0.09) for market price of 0.05, and (-0.02, 0.36) for market price of 0.25. Basically, it means that for market price of an event up to 0.25 no rational player would be selling at that price even if their subjective probability assessment was exactly 0%. So any market longer than a few months with rational traders is pretty useless for assessing low/high probability events. Even for a toss-up market price of 0.50, the no-trade region is pretty wide at (0.31,0.69).

Expand full comment

> 4.6.1: Then why should anyone play prediction markets, when on average they’ll only break even? It seems like this is a worse deal than stocks, which tend to go up over time.

You suggest subsidies, but do we even need that? Can't the market makers just put the money in the bank and return the money along with interest?

> 5.2: Politician pledge markets

Not a bad idea, but I don't think this will actually happen. Politicians can already do this with regular market. For example, a politician who wants to build solar energy system can promise to push up the price of some solar energy index fund, but I don't remember anyone doing that.

Expand full comment

Also, regarding the "meme stocks", another point to be made is that betting against the bubble is much easier since the downside is limited, unlike shorting stocks (as long as we're talking about simple binary outcome prediction markets).

Expand full comment

The stock market in general can't be manipulated, but then you have "madness of crowds" behavior like speculative bubbles that eventually crash etc. In principle you can "get rich quick" by shorting the bubble, but if it's you against the majority of investors, you probably don't have enough capital to budge the price. Or, in other words - your idealized argument would imply that the crypto bubble or the 2008 housing bubble shouldn't have existed, which is clearly wrong.

Expand full comment

So I'm a huge fan of legalizing prediction markets but I think it's worth considering a serious downside: it effectively requires legalizing manipulation (ie letting people with positions in the market move it via their pronouncements).

We *want* people like Nate Silver to publish his model and generally for experts to offer their expertise to the public. But if we force them to choose between this and investing in the market we create a strong incentive not to share information publicly which could result in less good predictions. Besides, such limits would likely be unconstitutional unlike as to stocks.

I don't really think this is such a bad thing. Norms about conflicts of interest seem like they can handle it and it's no more of a danger than scakmy reverse mortgages. And I'm not convinced it gives experts any more conflicts than they have now (indeed it let's them bet on what they really believe).

But it's worth pointing out.

Expand full comment

i like prediction markets, but i'm a lot less bullish on their potential to make broad social impact

prediction markets are nerd shit, and a lot of normies hate nerd shit

OR, you have a lot of normies who like nerd shit precisely when it happens to tell them what they want to hear, and say "Well the nerds can't know everything!" or "Remember that time it said 85% chance and the thing didn't happen? That just proves how inaccurate they are!" People will do this roughly to their hearts' content. Large portions of the population believe in homeopathy, intelligent design, fat burners, Qanon, etc, and many of those people aren't even dumb or uneducated. It costs them to believe in wrong things (in not so different a way it costs you to bet wrong in a market), but their need/desire to believe what they want to believe outweighs the costs they suffer, apparently.

Altogether, normies will sometimes listen to nerd shit, like weather probabilities or an economic analysis of the likely cost of some proposed bill or the stock market or the scientific consensus on global warming, and they will sometimes adjust their beliefs or behaviors accordingly, but they often won't. They may continue voting, acting, spending, talking in ways that are wholly at odds with the realities that prediction markets reveal, because they either can't wrap their heads around what this nerd shit actually means (esp if they have poor understanding of stats/probability, or distrust prediction markets because they don't have enough understanding of economics to see why they work), or because they'd rather spite the obnoxious nerds than make sensible decisions in their own lives, or because they would rather hold on to some comforting belief than make the ostensibly rational decision (i mean, the goal of life is to have fun anyways, not to make decisions that can be justified as "rational", so in some sense it's rational to do things that irritate rationalists because it's funny)

So not to say it won't make any dent on how normies think talk and act, but I think it might be a pretty modest dent. Prediction markets could be gods in the nerd world, but amongst those with actual power, they may be chihauhaus

Expand full comment
Dec 20, 2022·edited Dec 20, 2022

While the theory sounds nice, I fear that once translated into real world and widespread use, then when presented with "either the market is.... or get rich", people will go with 'get rich'.

Nobody buys Amazon shares because they think it's a great company and is properly priced, they buy shares because they hope the price will go up, and they will get a return on them. (It seems that "shares pay a steady dividend and prices don't go up or down very much" is no longer the way people make money).

Why expect a prediction market to be different? "Yes, we all agree that the Democrats will win the next election, the shares are priced correctly at 90 cents" and then nobody buys or sells shares and your market is stuck.

'Prediction markets all speak with one voice' - then why have three or six or ten markets, why not let one giant market eat them all up?

When it's a question of "did the Democrats win the election, yes or no", that can be resolved (unless somebody takes a court case about no they didn't). But when it's something more fuzzy, who makes the final resolution that yes, Messi is the greatest player of all time? You need somebody to do it, and that may well be an expert. Or there will be arguments over resolving the question because one set of investors will say you should measure it this way, another set say no, measure it that way.

If prediction markets take off, I don't expect them to be much different from the stock market or online bookies. Fine if anyone wants a bit of fun, or to try and make money. But God preserve us from mayors to presidents deciding to set public policy on the results of a market buying shares on "should we drain the Mississippi?"

"7.6: …if I’m the head of the Commodity Futures Trading Commission?

Please legalize real-money prediction markets in the United States. If it helps, here are a bunch of really famous economists including a Nobel laureate explaining why you should do this."

The CFTC might just be the least bit busy with something else going on right now about another new, ground-breaking, future of finance project:

https://www.cftc.gov/PressRoom/PressReleases/8638-22

(If that seems a little unfair, remember that FTX et al. came out of the rationalist/EA aligned communities, not just some common-or-garden Ponzi scheme. "But all the rationalists think this would work beautifully" is not necessarily going to endear prediction markets to the CFTC).

Expand full comment

> If you really wanted, you might be able to make it work in theory through a mechanism sort of like this one.

That post ends by noting that Robin Hanson explained why it wouldn't work.

https://www.lesswrong.com/posts/dLzZWNGD23zqNLvt3/the-apocalypse-bet?commentId=y9hmdcJQYzPCSbmJM

Expand full comment

I don't see one particular issue addressed: prediction markets can't make accurate predictions, but only aggregate the (market's) population's opinions.

When the Brexit vote was to be decided on Thursday, I discovered on Wednesday that, if the vote was to leave instead of remain, the stock markets would fall. I had decided the vote would be to leave, for non-financial reasons, so decided to bet the market would fall. That bet made me a small amount of money, but not as much as I lost elsewhere in the market. Of course, back then, I had never heard of prediction markets at all, anyway.

But prediction markets disagreed with polls. This site, from 6/16/2016 (one week before the referendum), says the markets were 60/40 to remain, but polls said 57/43 to leave. https://www.theparliamentmagazine.eu/news/article/brexit-betting-market-predicts-clear-win-for-remain-camp

But THAT market was two days before the vote. Another site, with no date I could find, says the prediction market turned accurate only hours before the vote. https://www.cam.ac.uk/research/news/gamblers-predicted-brexit-before-financial-traders-study-finds

And this site confirms that the betting markets changed wildly on the day OF the referendum: https://www.vox.com/2016/6/23/12022436/brexit-odds-of-a-british-exit-are-surging-on-betting-markets

The markets were different from the population because, well, the pool of people betting was different from the pool of people voting.

The market can only make accurate predictions if the participants have full knowledge of all relevant facts relating to a particular bet. So the predictions only produce aggregate opinions.

I'm grateful for the article, and now think prediction markets are a good tool for assessing broad opinions: people put their money where their mouth is.

Expand full comment
Dec 20, 2022·edited Dec 20, 2022

I don't think that the answer for 4.6 quite addresses that concern. This question is really "What happens if people don't care about getting rich?"

While "you" could get rich correcting a mispriced market with too many hedgers, there may not be enough market participants who care about getting rich. A market participant who is hedging is buying insurance, which is inherently about reducing the variance of outcomes at the explicit cost of worsening the average or expected outcome. They are risk minimizers not reward maximizers. If a market is full of hedgers, there is no "you" (or not enough "you"s) interested in getting rich. In that case, the entire mechanism behind the accuracy of prediction markets unravels; too many of the participants are unmoved by the incentive of getting rich by correcting mispricings.

This is not a theoretical problem. There have been many instances in commodity future markets where prices have gotten extremely far from plausible future probabilities because of this dynamic. However, in most cases, it usually doesn't take much profit maximizing money (aka those dastardly "speculators" and "hedge funds") to correct market mispricings. It just shouldn't be taken for granted; it doesn't happen 100% of the time.

Expand full comment

The caveat that always comes to mind for me when thinking about prediction markets is Goodhart's Law: https://en.wikipedia.org/wiki/Goodhart%27s_law . I didn't see this objection in Section 4.

As prediction markets become bigger, is there a risk that they go from neutral observers to an active incentive to bring about change in the outcome they're trying to predict? Is that a problem for the market concept? Does it change the conditions under which it's "ethical" to bet on one side or another?

Expand full comment
Dec 20, 2022·edited Dec 20, 2022

A lot of this sounds like the Efficient Market Hypothesis, which is probably true in its weak form and blatantly untrue in its strong form. Essentially, the only way for someone to make money is by research, so someone actually has to *make* said research, but the hypothesis assumes that this has *already* happened.

Expand full comment

Some additional questions to consider, or objections to think about.

1) Some questions have very little real information available to the general public. Sometimes the information available is wrong - maybe nobody knows the real information, or maybe the information is only available to certain insiders. We would expect all markets on this question to be wrong in the first case, or weirdly distorted in the second. For instance, if the publicly available but wrong information leads the general public to say 20%, but insiders think 80%, the market may converge on any number between them (assuming the insiders don't have infinite money to bet against the public, while the public thinks they are taking free money from the insiders, so they will want to keep pushing it to 20%).

2) We should expect every mature market to be distorted by an amount exactly equal to a combination of any costs (fees and taxes) of making the bet *and* the opportunity cost of not making another investment. You already noted that long term bets have this problem, but I think undersold the fees and even shorter term opportunity costs. At the very least, a badly mispriced market may stay moderately mispriced even after it resolves. I seem to recall some election markets resolving with lots of "free money" sitting around, because it was too expensive to pick up the amounts remaining.

3) The only people who buy stocks are people with disposable income. This problem will persist with prediction markets. You seem to be under the impression that poor people will bet in smaller amounts but in larger numbers. The reality is probably much closer to no bets at all for the bottom XX% of the population, and very minimal and haphazard bets for the next XX%. This will price in an "elite consensus" factor into all markets. It's like getting your political news from Twitter (at least before Musk, I don't know if it's different now) - it may be accurate often, but it's also heavily biased towards certain views. In a toy example with 10 possible predictors, if only three of them are able to invest and they all agree with each other, that's missing something big. This would be important for a whole series of questions related to the lives of poorer people, such as "the effect of UBI will be positive" or similar.

4) Even a 90% prediction based on good metrics will fail 10% of the time (and that assumes the underlying understanding/assumptions were correct). This is a bad failure mode for someone trying to make money and ties in heavily with #3 above. If I don't have $1,000 to lose "correcting" a market, then I don't bet. Also, see the next item.

5) One-off events are very hard to evaluate, even after the fact. The markets were wrong about various aspects of the Russia-Ukraine war. Other predictors were also wrong (which also ties in with #3 and #1 above - limited correct knowledge available generally, not enough liquidity among those who might predict better). If we were trying to use a prediction market to determine if/when various events would occur in that war, we would be wrong quite often.

6) Predictions about the actions of individuals or small groups (will Putin invade Ukraine?) is very different from predictions about aggregations of large groups (will the Democrats win?). We should expect markets about aggregates to do much better than markets about individuals.

Expand full comment

Scott you are actually wrong about if hedging distorts markets. I work in finance. Let me explain. Why does stock market go up on average? Because people don't want the stock market to go down. The economy being bad is correlated to all kinds of other bad things in their lives they would hedge if they could. There are plenty of natural shorters of the market in this world. Very few natural buyers. The expected return of the market is above 0 precisely because there are many people who would be happy to hedge against financial downturns if it was 0 by shorting. Very similarly, if there was a prediction market 'the economy will be worse than expected according to a prediction market' there will be more natural sellers willing to hedge and the market will remain systematically pessimistic being worse than expected. Because if it wasn't, and my cost of hedging is 0%, everyone who currently loses say 7% on average shorting the stock market would short this question instead, until it's expected loss equaled expected loss of shorting the market.

Expand full comment

> "Either prediction markets will be accurate, or you can get rich quick”. Unfortunately, it’s almost always the former.

This is a bit of a bait-and-switch. I think prediction markets will very often be inaccurate, but in ways where no one knows how exactly they're inaccurate. You can *technically* get rich quick in that it's theoretically possible to randomly choose one option and get lucky, but in practice you won't know to.

In this way, prediction markets aren't necessarily right at all, only not wrong in exploitable ways, so I have to disagree with referencing them as answers. They're just a sort of cumulative guess. For me, they shouldn't be taken as 'canonical' given their unreliability.

Expand full comment
founding

This part got me thinking, in light of the recent AI post: for "When you’re done, tell people whether you thought it was a good use of your time, so the market can resolve" you are training the system to provide content that you will retroactively rate highly, rather than content was in fact a good use of your time, which seems like it could be subtly different -- for example, maybe you have some hot button, and the system will quickly learn to press it. Other systems for allocating your attention almost certainly have this problem too, though, (or other problems) so maybe this is not specific to prediction markets?

Expand full comment

There is one thing I still don't understand:

Let's say, everything except for the prediction markets looks like event X (Biden resigning this year) is not going to happen.

Then, suddenly, the prediction markets say X is going to happen. What could possibly have happened?

A it was insider trading, and this made the markets more accurate

Or

B someone was very, very sure while being very, very wrong. Surely someone will fix the misspricing.

These seem to be the same situation from the outside — how are the prediction markets supposed to be able to resolve different?

Or is the only way to get everybody bankrupt who is capable of causing B?

Expand full comment

It's worth mentioning that sports betting, especially on the smaller scale (like some local lower-division soccer club), is *massively* tainted by corruption. Since it's so easy for someone on the team (or even more, in a sport like boxing) to either outright lose of degrade the performance significantly, and since there's a lot of betting, there's a lot of money to be made this way. So in this case, betting a) doesn't make the best estimate, and b) outright taints the sport.

I see no reason why big prediction markets wouldn't replicate this.

Expand full comment

This is pretty cool... in fact, you've inspired me to start paying. I tried my hands one year at the GJP, which was an interesting experience, but the time sink was too much for me.

One issue you dodge is the problem of thin trading. e.g., "will Prof. Dunderhead's new theorem in algebraic topology be proved correct?" Maybe ten people in the world can actually have an informed opinion, and it seems unlikely that as many as one of these 10 would choose to spend time playing prediction markets.

Expand full comment

For insider trading and illegal activities, I think the worry is more about money laundering than about encouraging illegal activity directly.

That is, if you make a million dollars selling drugs or whatever, you can then take that money, use it to place bets on innocuous things, then cash out your bet as "nothing to see here, officer, just my winnings from being a brilliant predictor." Gambling is heavily regulated to prevent that sort of thing.

I do think you're right that it could be at least as legal as, say, sports betting, but maybe there's some sort of clever money laundering scheme I'm not aware of which works better with arbitrary bets instead of sports gambling.

Expand full comment
Dec 20, 2022·edited Dec 20, 2022

Prediction markets are just a generalisation of stock markets. Stock markets answer whether a specific stock value rises. Prediction markets answer all kind of questions. That's all. No more, no less.

I'm skeptical on the utility of conditional prediction markets and prediction markets predicting the outcome of other prediction markets - I expect them to be much more vulnerable to manipulation and goodharting. But in general more acceptance and wider spread of prediction markets would be great and usefull.

Expand full comment

I have a question. What prevents a person with financial or political power from gaming a prediction market? For example, Elon Musk plans to sell a large number of shares of Tesla, and he hasn't told anyone. He can have someone he trusts put up a prediction that he will do so, and since no one expects this, there will be a large payout. He can then put a lot of money on it and make a big profit when he sells the shares.

Expand full comment

Conditional prediction markets seem like a context where the defenses against manipulation can be especially weak. In the vaccine choice example, there's presumably a major financial decision riding on which vaccine is chosen - let's say a $1bn contract that goes to pharma company A or B. We can also posit that the consensus on efficacy of these vaccines is pretty close - otherwise why bother seeking the wisdom of a prediction market?

So all pharma company A has to do to win the contract is make their vaccine's market trade slightly more favorably than B's, which should only require a marginal nudge. Meanwhile, it's far from obvious that there would be much neutral money in such a niche market. Not many people are vaccine experts, and in a situation like this where there's no glaring arbitrage opportunity, who's going to come in to correct the market by a few percentage points? Goldman Sachs might identify a slight mispricing and trade on it, but prudent risk management would keep the size of the trade fairly small, whereas pharma company A is incentivized to throw up to $1bn into the market. I would have said it's hard to imagine an army of Redditors descending on the market - these days I'm not so sure! - but to be clear there doesn't seem to be a "get rich quick" motive in correcting what might probablistically be a very slight mispricing in a niche contract.

One could argue that insider trading bans solve this, but there are situations much less blatant than the vaccine example where the party has a less direct financial interest. Also, a lot of corporate behavior in today's world is not totally above board, so it's not a huge stretch to imagine the laws being broken or bent.

I think conditional prediction markets will be most useful when the market has a large natural audience and attracts a lot of capital relative to any decisions informed by it.

Expand full comment

Prediction markets are great, but it's not rational to expect prediction market probabilities to accurately reflect risk-neutral real world probabilities, particularly when the event in question has economic significance. There will be distortions and risk premia like there are elsewhere in finance.

In section 4.6, you ask "If people use prediction markets to hedge risk, won’t that distort them?" and then say no because other traders will swoop in to correct the price. This is not at all what we see in traditional finance, nor does it really make sense from first principles. Rational traders (who all else being equal, prefer less risk to more) will only trade to either hedge a risk that they already have, or if they believe the expected profit on the trade justifies the volatility. If the risk neutral value of a contract is 25%, there is no incentive to trade it at 25% except to hedge.

Let's suppose there is a contract that we think should trade at 25% if everyone were perfectly risk neutral. Let's say there is hedging demand to sell and less or no hedging demand to buy.

If hedgers want to sell at 25%, and arbitrage traders don't want to buy at 25% (because they don't make any money in expectation), then the market price of that contract must be less than 25% in equilibrium. This sort of thing is one possible cause of "risk premia" in finance. The size of the premium here should mostly depend on the following factors:

1) How badly the hedgers want to hedge, in the aggregate - is the risk they are hedging existential to their business? Is it merely slightly annoying?

2) Amongst the rational traders, how correlated is this risk with other risks the rational traders tend to have on their balance sheets? (The more correlated, the more premium they demand).

3) How much pricing uncertainty is there? Can the rational traders be *really* confident the risk neutral value is 25%? How much potential is there for the hedgers (or really anyone selling the contract) to have inside information?

Prediction markets are still great. But in general, markets price things not only on expected value but also on distributional properties/correlations/intrinsic supply and demand imbalances. This his how proprietary traders and hedge funds and the like make money.

(Source: Was a prop trader, am now at a hedge fund)

Expand full comment
Dec 20, 2022·edited Dec 20, 2022

Isn't "the prediction market is as good as it gets when it comes to predictions" and "you should participate in the prediction market with a decent chance of making money on average" contradictory? *If* the prediction market is efficient, you won't make any money in it (barring insider trading, but then, only insiders should ever put in money), and if it isn't efficient, then it's not as good as it gets when it comes to predictions (after all, you can guess better)?

Without this, it's just sports betting or gambling - done for entertainment, while a reasonable actor with merely public information should expect to lose money. The way to make money is to either cheat (manipulating the market, insider trading) or to be the one running the market.

Expand full comment

I wonder if anyone has seen private prediction markets work well inside a large company? I was at a very large, multi-national Fortune 100 company that briefly experimented with prediction markets in its technology division a little over a decade ago. We ran prediction markets around things like, "Software release <X> will result in fewer than <Y> [really bad things]". (This was an easy statement to resolve unambiguously, I'm deliberately avoiding describing the really bad thing we were predicting.)

Pretty soon, the prediction markets revealed where there was a divergence from the politically expedient view. e.g. Prediction market says software release X will be a disaster, but executive in charge of it says it will be fine. Not long after that, the prediction market got shut down.

Was this a unique situation at this particular company? Is anyone seeing prediction markets used successfully inside large corporations?

Expand full comment

I'm, uh, not as sold on prediction markets. I do see a few major problems.

1. Too many markets/volume too low. I recall a market you wish you had: whether a certain outdoor event in a city in California would get rained out on a specific date. It's hard to imagine a) too many people caring about that and b) anyone having any better information on weather patterns than the meteorologists themselves, even if you don't trust them. With potentially thousands of markets, we'll probably see a power law where a handful of questions get all the attention and bets and many smaller markets where only a few people bet at all.

2. What's stopping the oil magnate CEO from putting hundreds of millions into YES on the "Will global temperatures rise 3 degrees by 2050?" market and then polluting so much that we hit the target and he makes a ton of money? Would people betting NO have to put in, collectively, just as much as he put in on YES to get the price back down?

Expand full comment

I notice a point that's come up a bunch in the comments is "hm, but prediction markets can only aggregate existing opinion, so they won't truly be *accurate* if *everybody* is wrong". Of course, the response to this is already implicit in the original piece -- no, they won't be truly accurate if everybody is wrong, but they will still be the *most* accurate available predictor, which is all you can hope for. Still, this might be worth making more explicit.

Expand full comment

We should probably look at this side: https://arxiv.org/abs/1907.11162

I'm not per se against prediction markets, but I think there is an over estimation of utility and a question of over estimation of precision. (I have participated in GJP many years ago and something that maybe was a precursor in 90s.)

That Yale/Harvard expert who says 70%. Is not really saying 70%, she is saying "about" 70%. Maybe what she is saying is there is between 50% and 90% change. There is a lot of difference between 50% and 90%.

Market manipulation: ask Nate Silver, a known gambler, to publicly state whether he privately bets on the question that he and his site make public predictions about. Predictions which affect the markets. He ostensibly is a data journalist but his ombudsman refuses to respond to inquiries about his private gambling.

Framing - almost all of these prediction market questions can be framed alternatively. Kahneman and Tversky

Is it really like the stock market or is it sports betting?

Expand full comment

I think instead of conditional prediction markets for P(A|B), markets for P(B) and P(A AND B) would be more flexible.

Suppose you want to bet on "on the condition that the next US president is a member of an ethnic minority (B), they identify as female (A)".

Betting a dollar on P(A|B) is equivalent to bet P(NOT B) dollar on NOT B and then betting the P(B) dollars on P(A AND B):

* If the outcome is any NOT B, P(A|B) will not resolve but return your bet of 1$. Equivalently, your NOT B market will return you the dollar.

* If the outcome is B AND NOT A, you lose everything

* If the outcome is B AND A, your return is P(B)/P(A AND B). This is the same return you would get from betting a dollar on P(A|B) (assuming effective market conditions.)

The advantage of AND-bets over are:

* Markets are always resolved

* It is a simpler basis which covers more ground. With a market for P(A), this would also cover the probability P(B|A).

Disadvantages:

* You need to track changes in P(B) and adjust your investment accordingly. (Unless you assume that P(B) is priced correctly and are risk-neutral or risk-friendly, in which case you would not need to bet on P(NOT B) to cover that possibility).

* Calculating the conditional probability is more complicated.

More generally, if we had N different observables A,B,C, ..., one might set up 2^N different markets predicting the probability of any specific outcome. If you want to bet on B, you would just invest in all the 2^(N-1) markets favoring B and weight each investment by that markets probability.

Of course, nobody could ever agree on the N interesting observables beforehand, and I am unsure if there is anything to be gained by converting bets on B to equivalent bets on A AND B and NOT A and B on the fly.

Expand full comment

I greatly appreciate your continuing efforts to be evenhanded and ethical.

Expand full comment

(I don't actively trade in prediction markets, but my background is finance.)

1. I was a little surprised that reversed trades didn't get mentioned as a criticism. In the vaccine example, if I take time to investigate vaccine A and trade accordingly, I lose all that effort if the vaccine isn't chosen. One might object that my knowledge could partially map over to vaccine B too, but in general there could be more than just two vaccines and there could be markets where none of the options get selected.

Furthermore, the potential tax implications just rub salt in the wound, as the reversed trades might not be in the same calendar year. Even if you're in the same tax situation both years, that's two sets of trades to report even though they ended up at a zero net payoff for you. (If nothing else, the sheer annoyance is a frictional cost.)

I'm not an absolutist by any means. If there's fraud or a major market malfunction, unwinding trades could be the best remedy. Markets where it is a routine occurrence, though, seem like they could have been designed better from the outset.

2. Regarding #4.6.1.1, there are still certain markets where this is unavoidable. Pretty much anything that correlates to general prosperity for the user base would potentially end up with distorted probabilities.

Expand full comment

It's debatable whether the stock market is efficient, and certainly there's no reason to think that a prediction market would be.

Markets are legal while gambling isn't generally because we want to allocate investment capital to enterprises, not just get a price for something. Having more money in the market means that companies have more paper value and thus can raise more money for actual operations - that is to say that the total amount of money in the market is meaningful, it creates value for the thing you invest in. In a prediction market there is no reason to want people to make larger bets, you just want enough to get accurate odds. Increasing the pot is likely harmful to society.

Having people make third-party bets with significant money is a bad use of money and regulators shouldn't allow it. Rather than betting on outcomes (zero-sum) people should invest in making outcomes happen (positive-sum).

Expand full comment

Another quick thought...

"Insider trading" doesn't quite equate to "I have knowledge that others don't have and I use it in my trading" as folks sometimes assume. If you're relying on the existing legal framework to solve that problem, you might be surprised by some of the outcomes. As a first approximation, if someone along the way didn't use for personal benefit information that "really" belongs to their employer, it probably isn't insider trading. (The personal benefit doesn't have to be financial, and the person who misuses the information isn't necessarily the one who trades.)

Expand full comment

Unless I'm misunderstanding, it seems like the use-case of prediction markets where only a minority are resolved by some event and for the majority the market's verdict is final is problematic, since in that case the response to the meme-stocks question 4.8 doesn't really apply.

E.g.

> A few randomly selected questions went before the Supreme Court each year, and the corresponding markets were resolved normally. In every other case, the market’s verdict was final, and bettors got their money refunded.

It feels like in this scenario it would be easy and fruitful for a rich person interested in a certain case outcome to artificially inflate or deflate the price of the market, knowing that there's only a 1/n chance that the supreme court is actually going to rule on it, and otherwise they are just buying their desired outcome. It feels like there should be a way to resolve this but I can't think of any off the top of my head.

Expand full comment

For far-future events, would recursive prediction markets work?

Given whether an event would occur within a 100 years, create a prediction market N about what a prediction market next year N+1 would resolve to be, where prediction market N+1 is about what prediction market N+2 will resolve to be, until prediction market N+99 which resolves to whether the event actually happens.

Prediction market N would close when the prediction market N+1 would start, and it pays out when prediction market N+1 closes.

Of course, if the event happens earlier you short-circuit the recursion.

Expand full comment

Conditional prediction markets for things like replication studies, where the market outcome determines which of many conditional prediction markets gets settled, seem to have two problems.

First of all, if you're redoing one of 100 studies, then the potential return on each market is only 1% of what it would otherwise be. Suppose the market says that Study #17 has a 10% chance of replicating, but I think it has a 20% chance. If I buy a bunch of 10-cent shares in this market, then I expect a 99% chance of getting my money back, an 0.8% chance of losing my money, and an 0.2% chance of getting 10x my money. This has an expected return of 1%, which is not worth betting on compared to our hypothetical 5% index fund alternative.

Second, there's a bias in how much this affects different opinions. In the calculation above, I assume that Study #17 has a 1% chance of being redone. But actually, if I think it has a high chance of replicating, then I think the prediction market has a high chance of agreeing with me - and since this is how we select which study to redo, maybe I actually estimate a 10% chance that Study #17 will be selected. This improves my expected return from 1% to 10% and now I have an incentive to bet.

On the other hand, my pessimistic friend who thinks Study #17 is total trash and has no chance of replicating also thinks it's very unlikely that Study #17 will be selected by the prediction markets! So for my friend, even though the conditional prediction market on Study #17 would have an 11% return if the study did get redone (my friend would buy WILL NOT REPLICATE shares for 90 cents and be certain to get a dollar back at the end), there is only a tiny chance that Study #17 will be redone in the first place, and the expected return is correspondingly tiny.

Doesn't this have an effect where all 100 conditional prediction markets will be biased upward, because correcting that bias has a lower rate of return?

Expand full comment

That is unrelated. People can make an unbiased estimate of how much companies will make in the future, discount that to present at the risk free rate, value the company at that amount, and still have a 0% expected return. The reason the stock market goes up over time faster than the risk free rate is because of risk premia, and the same dynamic would exist for prediction markets also used for hedging purposes.

Expand full comment

Well, your talent for writing persuasively is on full display here, very impressive. Personally I think it's all bollocks, however, for the simple reason that a prediction market is only better than an expert when the experts aren't very good.

Nobody imagines that a prediction market would correctly predict my eldest son's name better than either he or I. If my drains back up, I am clearly much better soliciting the advice of a plumber on how to clear them than going up and down the street and establishing a prediction market among my neighbors. If I want to build a functioning nuclear weapon, the way to do so is clearly to ask an expert nuclear weapon physicist, and asking a prediction market is silly.

And so on. For almost all of the basic factual matters of life, it's obviously more accurate and faster to ask someone who knows than to sample public opinion randomly, and so of course that's what we do.

Hence the questions to which the prediction market would apply are ipso facto those in which we already know (from bitter experience) that expert opinion does a poor job, presumably because the outcomes are very difficult to predict -- data is lacking, there is some sociological issue with making the accurate prediction, there is some inherent source of randomness, et cetera. Predicting the winners of the next election, predicting the outcome of a war, predicting the course of pandemic disease, predicting future major technological or social changes, predicting Black Swan events -- all of these fall into this category.

But in these categories, expert opinion is already known to be poor. Proving that prediction markets are at least as good as expert opinion doesn't say anything that's broadly and usefully actionable, because proving the prediction markets do better than known poor predictions is a long way from saying prediction markets make good predictions.

A prediction market can certainly provide exciting betting opportunities[1] for those who are into that -- and plenty of people are, hence the popularity of gambling -- but it can't make better predictions in a way that solves broad social or economic problems. I mean, to think otherwise, you need to think you can get something from nothing, that from a sufficiently clever summation of ignorance you can extract brilliance. This is like thinking that if there's a mathematical problem that no mathematician has been able to solve, I should be able to ask a whole lot of non-mathematicians and somehow from that huge mass of ignorant opinion extract the correct answer. This just isn't possible. Frankenstein is a myth, you can't create life from non-life, there is no algorithm to create meaning from ever so much nonsense.

--------------------

[1] It doesn't provide a way to steadily make money, like the stock market, because as you point out there is no separate value of investing: investing in the stock market as a means of betting on the future values of things is being a "growth" investor, but I hazard most investors are actually "value" investors. They're not buying a share of XOM because they're betting it will rise in the future, they're buying it to own a tiny piece of a profitable company that pledges to return some share of its future profits as dividends. Because the stock market is a collective investment vehicle as well as a betting market, you can make steady money in it, regardless of how well or poorly it predicts the future. There is no such collective investment aspect to the prediction market, it's just a betting pool.

Your argument that a prediction market would allow hedging is interesting, but you've just reinvented the insurance market, which is already healthy, and you're not going to gut insurance company profits until prediction markets prove *far more accurate* than any other source of prediction, which brings us right back to the main point above.

Expand full comment

I can't help but feel that all of your listed objections to prediction markets are downstream of something more fundamental:

Aren't markets cringe?

Depending on your outlook, markets lead directly to neo-feudalism or rainbows on everything in June. Either way, they're associated with those guys in the Econ department who wear bow ties and are politically unreliable at best. Markets allow the people you don't like to coordinate efficiently, and this is becoming increasingly intolerable. As more and more of society becomes politicized, both parties are becoming increasingly anti-market, and they have no incentive to overcome status quo bias and unban a new source of pluralism.

THIS is what you're up against. Change this attitude, THEN the point-by-point refutations will become relevant.

Expand full comment

I think it would be useful/helpful to briefly discuss the fee structure of these markets, which render them a negative-sum game. I love prediction markets but every time I feel interested in getting involved I see the fee structure and what it implies for the % I need to get right (weighted by price) and lose interest. Still think they have tremendous value though!

Expand full comment

The repeated "or someone can get rich quick" stories started to remind me very much of Anselm's argument. That is, there is a sense of "the true forecaster who can correct a mispriced market in reality is greater than all the ones who are only doing so in their imagination, therefore that true forecaster must exist."

Expand full comment

4.2.1. There are ppl killing for a killing on the share-market: https://en.wikipedia.org/wiki/Borussia_Dortmund_team_bus_bombing

Sergey Wenergold, a GTA V player (buy Air EMU, burn FlyUS) - maybe Osama had some put-options at 9/11. But: CEOs not killed - lol, John Kenneth Galbraith famously noticed how replaceable most of those are. Politicians are, too - I guess. But the chance of Biden/DeSantis/Trump to "be president in 2026" is zero on death. There is no option on Wall-Street "Tim Cook apple-CEO in 2024" - else I would worry for his life (though he and Musk spend on security.) Enough blood even without incentives:

https://en.wikipedia.org/wiki/List_of_incidents_of_political_violence_in_Washington,_D.C.#21st_century

Let prediction markets stay. Small.

Expand full comment

I love prediction markets and your enthusiasm for them in general (which is what originally made me aware of them), but you seem to be playing games with the "on average" thing a bit.

Like, in your Harvard v. Yale v. TinFoil thought experiment, suppose the TinFoil underdog turned out to actually be correct. It just so happens that the government did in fact make up the whole thing, and Harvard, Yale, and all other "respectable" institutions had the fatal blindspot of being funded by the government so <insert handwavium about incentives> they couldn't see it coming.

Your usual argument after this goes something like "This makes people more willing to trust TinFoil in the future, increasing TinFoil's weight in the market and thus making the market at least as accurate as TinFoil", but suppose TinFoil's talent is ONLY in play in case of global pandemics. Then people should only trust them in the *next* global pandemic, but how many global pandemics happen each century ? Probably 0 or 1 during TinFoil's lifetime, and they will probably die uncompensated for the truth they can help guide the market for, their talent wasted.

To tldr : averages only make sense on a big enough sample, and some events are one-shot (they don't even have to be true Black Swans, that's a ridiculously high bar, just as one-shot as a global pandemic or a nuclear war). Prediction Markets can theoretically always give the wrong answer and never correct. It's like a Prisoner's Dilemma where you only play once, Tit-For-Tat no longer works, you can Defect and get away with it.

Expand full comment

As others have noted, there are huge prediction market for sporting events. Usually there's a "house" involved and the house gets a cut, but the situation is only slightly different from a pure prediction market. Interestingly, it's known that the odds offered are not always accurate relative to the available information. I've seen an interview with someone who made a good living at sports betting by consuming all available information and learning the patterns of the irrationalities of the betting public. He said it took him about nine months of full-time work to get to that point, though no doubt he was very interested in sports and betting before that. OTOH, it required that he ingest all of the available news about sports, so it's likely that he could have earned more money on Wall Street.

My suspicion is that there are few consistently profitable ways of utilizing a prediction market, that there are few large categories of marketable predictions for which it is profitable to have accurate predictions. And that most of those are covered by existing financial markets. (E.g. the airline that wanted to financially insulate itself from a possible general downturn in the industry, so it placed a mass bet against the stock prices of all the other airlines. They did this shortly before 9/11 and so had to explain to the FBI what was going on!) The "clever uses" listed above seem to be between once every few years (which new vaccine to prioritize) and never (a CEO wants the public to be able to raise questions they care about).

Sure bet: a certain segment of the high-IQ blogosphere will continue to obsess about prediction markets.

Expand full comment

Odd thought maybe but I thought Id throw it out

Every bet is even money: a $ it will happen or a $ it won’t. The trading volume is analysed in real time to provide the prediction component.

Expand full comment

A masterful post, very well presented. However, I question the value proposition behind prediction markets. I consider three basic background variables:

a. The topic is "important" - meaning that an accurate assessment can guide current significant decisions. If not, it is merely "interesting". I'm talking here about the prediction market, not the outcome itself. An election outcome might be very important to many people, but an accurate assessment of the likely outcome won't actually change the outcome. Whereas an accurate assessment of the outcome of policy choices can be used to make those policy choices.

b. The topic is "timely" - meaning the final result will be knowable over a short enough timeframe to give predictors a real interest in seeing the outcome.

c. The topic is "independent" - meaning that an accurate prediction will not influence the result itself. Predicting the likelihood of a Yellowstone mega-eruption might be very "important", and possibly "timely", but Yellowstone won't decide on an eruption based on the prediction.

I see five basic situations:

1. The prediction deals with a topic that is "untimely", whether or not "important" or "independent". In this case, why would anyone participate in a market? We won't know an answer in a meaningful timeframe, so the "winners" won't collect. We could have a market on the likelihood of the Yellowstone eruption in the next 1000 years, but who will be around to collect in the year 3022?

1a. The prediction deals with a topic that is "important" but "untimely". In this case, partisans have a large incentive to tilt the prediction to favor their already-preferred outcome. For a current example, there are currently proposals to radically restructure the world's economy in order to prevent climate change and its associated predicted problems. Most of the predictions deal with potential effects by the year 2100. Most people alive today will not live to see the final result, but many have strong preferences on the policy question. If the prediction market is used to determine the policy choice, partisans have a strong incentive to try to tilt the prediction market to favor their preferred outcome. In such a case, the prediction market is really a vote by people willing to commit the resources to influence the policy outcome, but in a way less transparent and accountable than democratic governance, or even conventional lobbying and propaganda.

2. The prediction deals with a topic that is "interesting," "timely", and "independent". I consider election predictions in this category. We can certainly see election results in a reasonable timeframe. I doubt that voters will be much influenced by predictions stating the likelihood of "their" candidate winning. I doubt that anyone will make meaningful decisions based on current predictions of the next presidential election. If the election will have a material impact on the decision, it is almost certainly better to wait to see the actual election results. For instance, a business might consider a large investment in solar energy; and it might assume that a Democratic administration will offer large subsidies for such a project, while a Republican administration would not. In such a case, it would probably be a mistake to make significant resource commitments

In such a case, prediction markets are a useful counterweight to self-appointed experts, and can help people get a better read on the state of knowledge. But the markets don't serve an obvious social purpose other than perhaps balancing the shouting partisans.

3. The prediction deals with a topic that is "interesting" and 'timely", but not "independent". That is, we'll know the outcome in a reasonable timeframe, and it will be influenced by the prediction market, but the outcome isn't particularly important to most people. For example, "Who will be chosen to host the Oscars ceremony in 2025?" We'll be able to see the result, and it won't be of great importance to more than a few dozen people, but the decision is likely to be influenced by the prediction market. In such a case, I don't see a great social utility, and I don't see why anyone would participate, since future manipulation could tilt the market against current bets.

4. The prediction deals with a topic that is "important" and "timely", with strong partisan interest. For an example from 12 years ago, "Will a reform of the healthcare insurance market improve things for most Americans?" Setting up a prediction market to address this question gives partisans an incentive to tilt the prediction to favor their preferred policy, in order to enact their preferred policy or prevent a policy they don't like. In order to have a meaningful answer, the question would have to be much more sharply focused. For instance, "Will personal bankruptcies decrease by more than 50% within 5 years if the PPACA is enacted?" Or "Will deaths decrease by 40,000 per year from current trend within 5 years if the PPACA is enacted?" In such a situation, committed partisans will be willing to bet money to tilt the outcome to favor their preferences. Unless potential gains and losses in the prediction market are very large (hundreds of millions if not billions of dollars), I expect rational people with a strong policy preference would vote their policy preferences, rather than offering useful outcome prediction.

5. The prediction deals with a topic that is "important" and "timely", with no strong partisan interest. For example, "What is the likelihood of a Category 5 hurricane hitting New York City in the next 20 years?" This is "timely" because it's bound to a reasonable timeframe. It's "important", because an accurate assessment can be used to evaluate the desirability of measures to protect New York and nearby cities against the effects of hurricane. And it's independent because the Earth certainly isn't going to pay attention to the prediction market.

So, I think the only really useful applications of prediction markets are for issues that are "important" and "timely", but without strong partisan interests. Unless we can somehow exclude the partisans from the market, or attract so much capital to the market that even committed ideologues won't be willing to risk losing on the market.

Expand full comment

I'm still seeing a problem which I don't think was addressed in the post: certain/many controversial topics won't be able to be resolved fairly.

For example, let's say you wanted to assess climate migration. You could create a market like: "100 million people will move to a different country between now and 2040 because of climate change." No problem with the question, but how would you resolve it? You could link to a tracker from the UN or some other "authority" as the way to resolve it, but then the market really becomes a test of whether 100 million people will change countries at all, because said authority _will_ say it's because of climate change, whether that case is strong or weak. Or perhaps there would be another "authority" that would always resolve it the other way, no matter how many people move.

I guess I'm asking: what if there is no fair authority to appeal to on a complicated, controversial question? My intuition says that most squishy sociological questions would fail on prediction markets due to this.

Is this fixable?

(First Substack comment by the way; hi everyone!)

Expand full comment

Re 7.2: Using VPNs to access Polymarket is very safe and easy. It is not even possible for them to ban you as the smart contracts will always be accessible

Re 7.6: The head of the CFTC can't really do much. The commissioners have the final say and they vote

The main problems with Kalshi and Polymarket is that neither are sustainable. Both lose money by subsidizing liquidity. Polymarket does this directly and Kalshi does it via its affiliate, Kalshi Trading.

Without this, neither of the two platforms would have much if any liquidity on the long-tail markets that are probably most interesting to the readers of ACX

Expand full comment

What a great Xmas surprise, thanks for this post, I've been looking for something that breaks this down better than the wiki.

Expand full comment

Prediction markets don't work nearly as well as you make them sound. They have many problems you barely address:

1. The biggest problem is that if everyone was rational, nobody would bet in prediction markets (except for a few use cases like hedging by big firms). You briefly mention this and say "but in practice people do bet", yet I'm not sure that's true; in practice, people bet on the large, flashy items like presidential elections, but not on other things. There is no reason to believe this will scale.

2. You spend a lot of time focusing on the limit as the money in the prediction market goes to infinity. It's true that in the limit, prediction markets work. But see (1); the limit will never be reached, and the more rational society is, the less likely prediction markets are to have a lot of money in them.

3. Instead of focusing on the limit, consider the *rate at which that limit is approached*. Just how well do small prediction markets perform? What about medium ones? In general, they do pretty badly. There are a few reasons for this, including:

3.1. Goldman Sachs won't bet on them unless they can make more money than their quants' salaries cost. Even a $1000 edge is not worth people's time.

3.2. If the market has a lot of money in it, it basically implies that lots of stupid people have bet (see point 1), and

3.3. Smart people are less efficient at correcting markets than dumb people are at distorting them, due to the Kelly criterion; that is to say, if you believe the market prediction is off by 10%, it may only make sense for you to put in something like 1% of your money into the market due to Kelly (this depends on several factors), yet a dumb person may put in 100% of their money when distorting the market. You would have to have 100x more money than the dumb person to offset them.

4. The payoffs in conditional prediction markets are conditional, which amplifies all the problems in (3) and makes it a lot harder for them to work.

5. You spend a long time talking about whether prediction markets are effective, but not about whether they are *cost effective*. If I have $X and want a good prediction, should I (a) set up a prediction market, or (b) hire an expert or small group of experts? (I could average the group of experts myself later)

Almost certainly (b) is better for most things. This has been repeatedly observed empirically. It also makes sense theoretically: if predictions are expensive (e.g. they cost some fraction of $X for setting up a fancy model), then having a market fundamentally means *duplicating this expensive work*. Like, instead of one fancy model being set up, in a prediction market you're likely to end up with a larger number of smaller, cheaper models as people scramble to eat up the returns; this wastes effort as many players duplicate the same predictive work. There won't be enough edge left for Goldman Sachs to invest in the fancy model.

6. It just feels weird to say "in the glorious future, we'll all have perfect predictions funded by gambling addicts' losses". This is the same point as (1), but this time as a moral criticism. You are basically saying you want Goldman Sachs to set up fancy models, and those fancy models will be funded by the losses of gambling addicts, and you want it to be a stable, permanent equilibrium.

Expand full comment

> if the stock market grows 4% per year, I should expect any money invested in the stock market to multiply by 20x in 2100. So just doubling it in a prediction market is a bad option.

Normal finance has a good solution to this: futures markets. You can get very large leverage at very little upfront cost, and small liquidity loss by just agreeing with someone that you will pay them a particular price in exchange for a specified asset on a particular date.

Expand full comment

I disagree with a lot of this, especially the parts that argue for prediction markets being highly efficient. There's many reasons that markets don't incorporate all existing information, some of which you touch on - opportunity costs of capital, opportunity costs of traders - case in point, I made around 300k on polymarket in 2021 and then quit in favor of trading NFTs and later crypto more broadly. With interest rates close to 5% it makes even less sense to trade anything longer term without a massive edge.

But even the extent that current markets are fairly priced is sort of free loading off their current low profile. If prediction markets got actually relied on in the way you propose, the returns to manipulation would go up. In your example of the government choosing between two vaccine candidates, the value of that choice to the vaccine company will be valued in the billions of dollars, and they can easily ensure the market price moves to where it needs to be with a small fraction of that. Investors will push back some, but there will be a wide range of uncertainty and so they won't want to risk too much. This generalizes; any time you want a prediction market to impact $X of real resources, you create an incentive for rent seeking on behalf of the beneficiaries. There's always a bound for how much capital is willing to correct an apparent mispricing, with unknown risks, possible adverse selection, and opportunity costs.

I'm even worried that this kind of Goodharting would apply to existing financial markets if they were explicitly relied upon for policy. Scott Sumner has suggested using NGDP futures to determine Fed interest rate changes, but it's not clear to me how you'd prevent people from moving those markets in order to profit on bond positions.

Expand full comment

I think I could have a good crack at the time value of money issue. This wouldn't be starting a new prediction market, but selling a backend currency service to existing markets.

If anyone is based in the UK and would be interested in helping me start this, let me know below. Experience in payment operations or treasury tech stack would be particularly complementary to my own skills and experience (which are capital markets and regulatory structure across financials and infrastructure).

Expand full comment

Scott, you say with regard to very long term predictions "But if you really wanted to use a prediction market, you could theoretically solve this by putting investors’ money in index funds while they waited. Then the winner would get their (and the losers’) original deposits and investment profits, and it would go back to being a better option than investing in index funds directly. In practice this seems complicated and I wouldn’t expect it to work." This kind of paired structure is extremely common in financial markets (e.g. convertible bonds) and I would see no problem at all doing this with an index fund and a prediction position, provided the prediction market venue itself was stable. Again, the derivatives markets clearing infrastructure shows how this can be done.

Expand full comment

If I trust the prediction market to usually represent the best odds of any given scenario occuring, why would I ever bet against the odds currently presented by the prediction markets? Even if I was an expert in Event A, why would I trust myself to know the odds of event A occuring better than the collective knowledge of the market? It would not make sense to participate unless I didn't trust the market's accuracy to begin with. In which case, I would not consider the market's ability to make accurate predictive odds inherently more trustworthy than any other source. If the market is trustworthy, I would not participate. If it was untrustworthy, what's the point of making one in the first place? Other than

trying to make money gambling against people I assume are stupider than me.

Expand full comment
Dec 20, 2022·edited Dec 20, 2022

This post is extremely frustrating because you acknowledge all the counterarguments, you just refuse to connect the dots and follow them to the logical conclusion. It's basically this in essay form: https://i.kym-cdn.com/photos/images/newsfeed/001/286/448/863.png

For example, you have a section where you acknowledge the various reasons why real world prediction markets might be biased or inaccurate, and then spend the rest of the essay arguing based on the premise that prediction markets magically have infinite liquidity, zero transaction costs, etc.

Instead of responding to every question by saying "in the frictionless spherical cow world, you could make money by arbitraging this", the intellectually honest approach would be to try to quantify the bias in your hypothetical scenarios and see how much of a problem it would or wouldn't be. Even you yourself acknowledge at the end that real life prediction markets are highly biased and illiquid, and that while you might hope to improve that a bit, you also acknowledge that the real world stock markets, which function near-infinitely better than prediction markets could ever hope to, *still* have problems.

Another amusing bit was where you concede that "we shouldn't allow markets in dangerous things" and then later on complain about regulators restricting what you are allowed to have markets in. Obviously, you could have some principled position that the optimal level of regulation is nonzero but less than the status quo, but you never actually *say* that, so it just makes you look confused and dishonest, and it hides your true position.

Another huge weakness is that you completely dodge the "prediction markets will solve societal breakdown and lack of trust" thesis, despite that being *your leading argument*. You do finally get to the obvious question later about how the many people don't even trust wall street could possibly be expected to put their faith into a weird inferior version of the stock markets, but you just pathetically retreat to "well, the kind of person who trusts prediction markets will trust them and who cares about the rest?".

Lastly, you also don't have a good answer about the "why will people play a zero sum game"? problem. Obviously, people like gambling, but the demand for gambling is not infinite, and it is not evenly distributed. People like gambling on sports, but you're probably going to have a lot less luck getting people to gamble on say, vaccine stats (as you observe, this is borne out in the real world where prediction markets are tumbleweeds next to DraftKings et all). Also, gambling is probably a net negative to society anyway.

One other thing I should emphasize is that **the assumption of infinite liquidity and zero transaction costs is a load bearing element of your arguments**, despite you realizing that this is clearly not actually true. For example, consider proposals like "only decide a few of the markets randomly". If you have any actual finite amount of liquidity, risk tolerance, etc. then this is a huge problem because it cuts the effective liquidity down proportionally. This works in the spherical cow model because zero times any number is zero, but as soon as you plug in any actual nonzero number, you'll find that amplifying the bias this way is actually a problem.

Expand full comment

So what about Grossman-Stiglitz paradox? (On the Impossibility of Informationally Efficient Markets, 1980).

The crude argument goes like this. Suppose

(1) prediction market equilibrium prices reflect the current information completely ie arbitrage is impossible

(2) gathering information to update the market prices costs money.

Then, consequently, nobody has financial incentive to produce new, correct information and update the markets because there are no profits to made Thus we have proved all efficient markets are impossible with Logic™ ...which would be silly, so we probably didn't. But as a less silly consequence, Grossman & Stiglitz present a model and argue that a market can function only if the prices are imperfect enough to make information arbitrage possible.

Naively, I think this can be understood as a statement to the effect, an efficient market can be boundedly efficient, in general and over time, but not guaranteed at any particular given time point.

Here is also a neat conjecture: any person who considers themselves informed (say, they invest constant effort to remain informed) should try to get more investors, preferably less informed and overconfident investors to join prediction market, as many as possible. The informed person gets higher expected returns from increased opportunities, but the price system will be less informative.

Expand full comment

All of this seems premised on the idea that different models disagree about the future, and not the present. I don't see that as being true for most questions, and we already have proxies for most objectively verifiable questions in stocks/bonds/etc.

Prediction markets can only ever be as reliable as the resolution criteria. Faking the winner of a presidential election is hard. It's a question where social reality is physical reality, but where the market implications thereof are vague at best.

The same is not true for most other questions. If we grant one year ago, that ivermectin is safe and effective and covid vaccines are not, knowing that the FDA is going to say the opposite in a year, is still not surprising, and is not a meaningful evidence in favour of getting the vaccine, lockdowning harder, not abolishing the FDA, etc.

Also, you don't really need a prediction market to know what the FDA is going to say about the vaccine, because vaccines are produced by publicly traded corporations, and so have stock prices.

I'm sure that legalizing prediction markets would be great and all, but I don't expect it to have the policy/political ramifications that proponents desire of it. The many downstream effects are premised on prediction market prices being useful arguments that will convince people, and for the vast majority of issues, they just aren't.

Expand full comment

How is this different from the wisdom of the crowd, other than adding stakes for the crowd?

If you can't get rich, why are people betting?

Expand full comment

One thing I'm having trouble finding out with casual Googling: what is the standard way to bet *against* a prediction? If you already have a position in the market you can of course sell your position for a sufficiently low price, but if you don't already have a position can you "short" the prediction, and if so, what are the mechanics of that?

Or do markets just naturally come in pairs where that for every "the sun will rise tomorrow" market there's a complementary "the sun *won't* rise tomorrow" market?

Expand full comment

I've shorted hundreds of individual stocks over the years, and a crucial addendum to 4.8 is that the most-overpriced meme stocks become hard to borrow, meaning nobody could borrow shares to go short even if they wanted to. Even when shares are available to short, there is a risk that if you go short the price will spike and it will become hard to borrow and your shares will be recalled by the lender and you will be forced to cover your short at a very bad time, regardless of how much collateral you have. These structural barriers to short-selling make mispricings persist longer in the stock market. None of that is a problem on prediction markets. There is no limit to how many "no" contracts I can buy, and there is no risk of a temporarily overpriced market forcing me to close my "no" position prematurely. This will make prediction markets much more resilient than stocks against upward manipulation / memes / ponzi dynamics.

Expand full comment

The time value of money presents serious issues that hamstring current prediction markets as well. An event a year away needs to be mispriced by like 10% for it to be worth intervening. I think it would be good to concede that this is a huge problem with existing markets in the FAQ. This is trivially fixable by simply wagering shares of SPY or something but we need regulators to let this happen. I would be elated to have a decent prediction market but regulators need to let it happen.

Expand full comment

Oh no it’s Nash equilibria again! “If nate silver was consistently better on average you could make money by betting on his predictions. Do this until you’re rich and the market has corrected.” “If there’s an advantage to deviating from a strategy in a game, people will do it until the game equilibrates to a state where there’s no advantage to be gained by deviating.” I’m the man with a Nash equilibrium hammer and everything looks like a nail!

Expand full comment

"Prediction markets say there is only a 12% chance Ukraine will take Donetsk by winter, which is down by 9% since the logistical problems started”

I don't know if this an actual quote, but it either is, or it is an example that Scott created to demonstrate the usefulness of prediction markets, and it suits my needs either way.

It seems to me prediction markets should not shift like this (from one stable number to another). The whole point of them is to forsee unforseen events. So what's going on? We know two things for sure:

1. The price wasn't bid down before the logistics problems (from the Ukraine example), and 2. The price WAS bid down after the fact, so we know that experts considered the logistics problems significant to the outcome of the original question.

This leads to one of two conclusions. 1. The best experts either aren't participating in the markets or can't participate in the markets. This contradicts one of Scott's claims about prediction markets. Or 2. There are questions that markets are fundamentally unable to answer. The question "will there be an event that will lower expert's outlook of Ukraine taking Donetsk by winter by more than 8 points?" could not be correctly answered by the markets, since those who could answer that question would also bid on the original question in a way we know didn't happen.

So either markets don't represent the best expert opinion, or markets can't answer questions that the best experts consider important. Either one would be a huge flaw. So where am I going wrong?

Expand full comment

Good post. I would add one thing about why we might expect prediction markets to be accurate: Over time, people who are good at predicting will make money, and so they will keep predicting, and they will have more cash to do so with. And people who are bad at predicting will lose their money, and will probably go away, and if they don't go away they'll have less money to bet with. The net effect is to concentrate money in the hands of people who are better at predicting, and the more money you are working with, the more you sway the market. So the market is moved strongest by the best predictors.

Expand full comment
Dec 21, 2022·edited Dec 21, 2022

Great overview, thanks Scott!

Regarding Futuur, a few points of clarification:

• Futuur is a hybrid real-money / play-money prediction market platform. Real-money betting is not available in some jurisdictions, including unfortunately the US (for now). However US residents can still bet with play-money, and have full access to the forecasts generated from real-money wagering.

• Real-money and play-money markets operate seperately and generate independent forecasts. On the Mars prediction, it looks like you got your ~75% "no" forecast from the play-money version of this market, but the real-money version is currently showing 97% for "no". To access the real-money forecasts, you need to click on your account balance and switch to real money. (I know, this could and should be much more intuitive... we are working on fixing that!)

• When it comes to longer-term bets like the Mars one, another point to remember is that you can sell your position at any time. So if you bet early and are followed by others betting on the same outcome, you can exit your position at a profit well before the resolution date.

• Futuur (https://futuur.com) has more than 1000 markets, across all categories, and we're always looking for suggestions for more! You can make a suggestion here: https://futuur.com/i/feedback/suggestion

Expand full comment

> 4.6.1.1: If people use prediction markets to hedge risk, won’t that distort them?

That is, suppose that an import-export business spends millions of dollars betting that Trump will win in order to hedge against his protectionist policies. Since their bets aren’t based on the real chance of Trump winning, won’t that distort the market?

No. Suppose that everyone knows Trump has a 50-50 chance of winning. And suppose the import-export business, in the process of hedging risk, bids it up to 90-10. Since you know Trump has a 50-50 chance of winning, you can get rich quick by bidding it back down to 50-50. From your point of view, the import-export business is (in expectation) giving you free money. But they’re still happy to do it, because they’re hedging their risk successfully.

___

I think this misunderstands the main hedging argument that I'm familiar with, which is namely that some candidates may be predictably bad for the entire economy, so everybody's risks are directionally correlated.

Suppose you have two candidates, Joe Jughashvili and Sam Clemens. Sam is a generic centrist, Joe says the right words but you worry that he wants to create a communist revolution. The true probability is 50-50.

You think there's a small but nonzero chance S&P 500 will go down by >80% if Joe wins. So you bet on Joe winning, on the grounds that if he wins you can hedge your risks considerably. Other people hedge in a correlated way to you and bid the numbers to 60-40.

Now suppose an independent hedge fund manager looks at this market and polling data. She realizes that the true probability is 50-50. So she wants to bet against Joe. But then she checks her own exposure and (correctly) realizes that her wealth is also tied to this market. So she wisely refrains from betting.

This is also covered in Lizka's post in 4.3: https://forum.effectivealtruism.org/posts/E4QnGsXLEEcNysADT/issues-with-futarchy

Expand full comment

Why is Scott wasting his time asking people to try different things to help promote prediction markets? Surely the correct strategy would be to devise a bunch of conditional markets (if we try promotion strategy X, prediction markets will expand Y amount) and then simply do whichever strategy the markets claim will work?

Maybe also use this to test strategies to get the restrictions on U.S. use lifted?

Also, I'm curious about whether prediction markets will get big. Has anyone asked a prediction market this question? Maybe we can, in fact, narrow down precisely how big they'll get, and how long they'll stay that way.

For that matter, what will *replace* prediction markets, and can we get there now?

Maybe we can get still more meta. Can we use prediction markets to try and find what area is most likely to have wrong predictions, and use this to make money?

I honest-to-FSM have no idea if I am being snarky or serious here. (But maybe there's a way to find out!)

Expand full comment

Decision markets worry me, mostly because of the loss of "slack" it may cause in certain economic actors, much like may happen in similar-ish things that Robin Hanson advocates (eg CEO-firing-conditional stocks). The current level of imprecision in decision-making allows for more "human in the loop" situations, where people can make decisions taking into account many things other than their bottom line.

In a situation between two choices, a human decision-maker can assess things like ethics and external impacts. A decision market really can't take into account anything that isn't both easily measurable and explicitly spelled out for it. A human CEO is actually able leave the hundred-dollar bill on the ground if it's good for it to stay there. Even publicly-traded corporations are scarily uncontrollable since due to shareholders, and they do take actions with negative effects that most individual humans probably wouldn't. Surely, a market for individual choices would be even worse in that area?

Purely informational markets (as opposed to explicit decision markets) sound relatively safe, but even they can potentially limit freedom of action in decision-makers. This can be both good and bad: the things to be weighed are, on one hand, increased efficiency, better information, and probably lessened corruption, and on the other hand, serious tightening of misaligned incentives and negative externalities.

I don't doubt that prediction markets would work as described, but I really don't know whether this would be worth it. I remain undecided, with a moderately-sized part of the "in favor" side being based on "these smart people think using prediction markets would be smart."

Expand full comment

I think one major problem with current prediction markets is that good resolution criteria are hard to make. Resolution is ideally done based on clear objective outcomes, but what you care about is usually abstract general variables.

I think this problem can be resolved using reflective latent variables: https://www.lesswrong.com/posts/HygyWpDcwsekqmfnS/will-manifold-markets-metaculus-have-built-in-support-for

They are a mathematical technique that allows making correlated probability distributions over many variables.

Imagine a "how will the Ukraine war go?" market, which allowed people to make correlated bets on questions like who will be in control of various cities at various times, whether leaders will continue to be in power, who will be supported how much by their allies, etc.. Rather than getting difficult-to-interpret markets on narrow questions, this could give you probabilities for broader issues and complex scenarios.

Expand full comment

You left out a very important part in your article, which is what source of truth to check to resolve a prediction.

Iny option this is the reason why Prediction Markets won't replace pundits: they could always say The New York Times got their COVID stats wrong and dispute whether a vaccine worked at all.

I am forever hopeful this will stimulate the creation of a fact-checking economy: newspapers that only publish thoroughly reviewed facts and abstain from opinions, that all parties can 100% agree on, but I'm very skeptical of it (I will give this a 20% chance by 2030)

Expand full comment

Why shouldn't prediction markets invest the money waiting in the betting pools in index funds or in a randomized selection of stocks, to capture that 5% economy growth profit?

Expand full comment

Are the people putting down bets on a roulette wheel spin, a prediction market?

Is the guy with a computer in his shoe detecting the slight bias in the wheel an expert?

Expand full comment

What is the counter to comparing prediction markets to hedge funds but for events and claiming they have the same weaknesses? Some hedge funds do well in predicting winning stocks, but there's no way of telling which in advance, and YOY some winners become losers and vice versa. If you believe events are independent and predicting one gives no significant skill in predicting the next, they would fail on average in the same way as hedge funds predicting the stock market.

Expand full comment
Dec 21, 2022·edited Dec 21, 2022

> Someone who has resolved their past 100 prediction markets honestly will probably resolve this one honestly too, especially if they get paid to do so and will never get customers again if they lie.

So I should expect to see a lot of rugpull scams on Prediction markets, because this sort of reputational system doesn't work.

Expand full comment

Prediction markets aren't like stock markets, because stock markets.trade items of some.intrinisic value...not opinions. Prediction markets are unrestricted betting.

Expand full comment

What would be the legality of a company like Google giving each of its employees a free hundred dollars on an internal real money prediction market, which they can withdraw at the end of a year if they bet on enough markets?

They wouldn't be allowed to deposit their own money.

Would that still fall foul of US gambling laws?

Expand full comment

Back when people cared about movies, Cantor Fitzgerald started a prediction market called Hollywood Stock Exchange to encourage betting on the first four week box office totals of new movies. But because Cantor Fitzgerald leased the top floors of the World Trade Center, a huge number of Cantor Fitzgerald employees were murdered on 9/11.

Eventually, a reconstituted Cantor Fitzgerald applied to create a futures market in movies. But the feds couldn't figure out what to do about insider trading -- e.g., if you saw a preview screening of "Amsterdam" or "Babylon" were you entitled to point out they weren't very good movies? So the market wasn't allowed to get off the ground.

Expand full comment

One issue that is alluded to and other commenters point out is the 'oracle' or 'arbiter' problem, wherein the outcome of a given market depends on some measure that we are relying on some sensor or trusted reporter to accurately detect and convey.

If your prediction is "Will the maximum temperature in Boston on December 25, 2022 rise above 50 degrees Fahrenheit" there is the risk that the thermometer being used to measure the temp could be off by +/- 2 degrees for some reason, but maybe predictors are aware of this and adjust their bets accordingly. Still,

This could obviously be the problem for a market like "Will Bubba Joe have a Good Day on January 1st, 2023" because the only way to get the answer is to ask Bubba "did you have a good day" and maybe he reports that he sure did, even though his internal state is that his day was actually quite unpleasant.

In this sense, the contract is less betting on the actual underlying reality and more on what the reporter is going to say, which can add more and more uncertainty the less reliable the reporter is.

Hence, Prediction markets probably work best for events where many people can measure an outcome for themselves and confirm the validity of the report, rather than situations where we have to rely on a single 'authoritative' source which *could* be compromised.

On the positive side, we can also consider that if prediction markets gain widespread use, additional financial instruments can be built on top of them that abstract away the actual probability side of it and allow people to take positions on outcomes without having to be fully aware of all the information that might influence said outcome.

Expand full comment

Re 5.6 (Education). When I was a kid I developed a habit that I now realize serves a similar fog-clearing function to the “what odds would you give this in a prediction market” check. When I want to check my confidence in something, I wonder, “How scared would I feel if I found out I had to get a lash across the back for each instance I was wrong?”

E.g., if I start by thinking “Everyone seems to like me, I don’t think anyone I know has maligned me behind my back,” then ask the whipping question, I realize I’m not so certain - because I have some nonzero fear about how much pain I’m in for.

It’s helpful for checking whether you have grounded confidence in something or whether you’re just hyping yourself up.

Expand full comment

Your proof that prediction markets are canonical is wrong. You need to replace "get rich quick" with "get rich in expectation". I learnt recently about risk-neutral probabilities and how S&P futures are a counter-example to your claim. The historical average return from the S&P is something like 10% (maybe something like 7% after inflation). The average historical price of 1-year-out S&P futures is something like 2% to 3% above whatever the current price of the S&P was. (I may be slightly misremembering the numbers from when I was shown this example a couple weeks ago.) So you can make pretty much arbitrary amounts of free money in expectation by correcting this mismatch.

So here are some actual limitations on prediction markets that have good theoretical backing:

1. The accuracy of a prediction market is bounded by (at least) the highest savings account interest rate. If the resolution time is a year out and you could make 4% return on your money by putting it in a savings account or a CD, you shouldn't expect anyone to be correcting the market if their return from doing so is less than 4 cents on their dollar

2. Long-term prediction markets also need to correct for inflation

3. If the outcomes are not independent of the value of money (if you're betting in USD on the collapse of the dollar, or if your betting on an AI apocalypse in any currency you don't expect the AI to care about), you shouldn't expect pricing to reflect probability. I'd be willing to bet basically my entire fortune that humans will still be alive tomorrow (as long as the return is higher than the interest rate on savings accounts), because this increases my expected utility in worlds where humans are alive without changing my utility in the other worlds.

Expand full comment
Dec 31, 2022·edited Dec 31, 2022

"And although the contributions of a hundred people like me would add up to $85,000 and correct the mispricing, I guess there aren’t a hundred people who know about this strategy and care enough about making a few hundred dollars to pursue it. So the site continues to lean slightly conservative."

...he said, correcting the next election's mispricing.

Expand full comment

Although it wouldn't solve everything and would be hard to implement at scale, one path to addressing the issue of betting on things like the world ending by some date is to have the party that believes the world will continue pay out immediately so that the party who thinks the world will end can enjoy the funds while the world is still here. If the world doesn't end by the specified date, the party correctly believing the world would continue would receive a large payment from the counterparty to compensate for the opportunity cost. Garett Jones proposed this idea in more detail here: https://www.econlib.org/archives/2012/09/how_to_bet_on_b.html

Expand full comment

You mention a couple times that random people trying to participate in prediction markets should expect to lose money, but on Manifold you can see people's profits, and a strong majority of accounts I've looked at have positive profits. I was confused by this for a while but I think the reason is that Manifold markets are subsidized by market creators and by Manifold itself (through bots). (Weirdly, those bots also have positive profits! That I don't get.) This probably makes the site more fun for everyone, but also makes it harder to tell if you are doing a decent job. It would be cool to know what the average user profit on Manifold is (both in absolute numbers and as a return rate); that could maybe be treated as the zero point for individuals like me trying to have a sense of how we're doing.

Expand full comment

What about this failure mode?

"I envision that someday people who want to know the answer to specific questions can subsidize prediction markets on them. For example, the Democratic Party might subsidize a conditional market (see 5.1) about which Democratic primary candidate is most likely to win the general election."

Ok, so say the Democratic Party sets up this market, and plans on taking action based on that information. And let's say the underlying reality is that Biden is a bit more likely to win the 2024 general election than Newsom.

However, the Republican Party and its supporters then pour a lot of money into the market to make it look like Newsom would be a bit more likely to win than Biden. Sure, other bettors are busy making bets to bring it back in line with polls and experts, but the Republican Party keeps pouring money in.

At the deadline by which the Democratic Party needs to take action, the market says Newsom stands a better chance of winning the general election than Biden. Due to that misinformation and the actions the Democratic Party takes as a result, Newsom wins the primaries. Then Newsom loses the general election, a more likely outcome than if Biden had been nominated.

Expand full comment

Regarding doing destructive things to manipulate markets, a more complete answer is that having made those market transactions is a big piece of evidence of who did the destructive thing that tends to get you imprisoned for doing it. I remember reading about just such a case in (of course) Florida a few years ago: a guy was arrested for trying to bomb Target stores to make Target's stock price go down.

Expand full comment