Who needs journalists? Experts should start their own blogs.
I assume there's some sort of barrier there. Though Andrew Gelman and Derek Lowe have been my most trusted sources throughout, so it's not impermeable.
You'd think people who do epidemiology as a day job are even more frustrated than the rest of us at how bad mass-media coverage has been.
"Sources don't usually get to approve the way they're quoted in the article, or to see the article before it gets published."
Changing that norm right there, for some varieties of journalism, particularly those that rely on expert sources on nuanced topics, might help a lot. For example, I can imagine a journalist saying "here's a draft of the part of my article where I quote you, with some of the context... did I get that mostly right?"
I know much journalism is time- and resource-strapped, and a lot of sources don't want to go to extra effort to do someone else's proofreading for them, but... if I knew that sort of thing were the norm at a journalistic outfit, I'd both be more likely to read their output and more likely to cooperate as a source.
This is an interesting problem. I wonder if this problem has also gotten harder (or changed) over time.
Or to put it another way: a lot of our society's epistemic processes have come under assault all at once. From my perspective (and I wish I had data in front of me backing it up), many media sources have come under a lot of pressure to generate clicks quickly, and journalists have less time to dedicate to the quality of claims.
Additionally, the development of "alternative" medias and academies, have in many cases acted to push their more legitimate counterparts to try to circle the wagons.
To be clear: these gaps have likely always existed, and they likely always will exist to some extent. (Look inside a modern corporation, and you'll find that there are gaps between the analysts and top executives in terms of how the system works)
My guess though is that the "circling the wagons" effect is likely the dominant one though. Which is to say that if there was a presumption all speakers were honest & reliable, then the overall risks would be lower.
You do sometimes get mainstream journalism that involves thorough research, lots of interviews with experts, and ultimately a journalist giving a synthesis of their own opinion. It's really great when it works.
For example, I thought this piece in New York Magazine about the SARS-CoV-2 lab leak origin was good. I hope it came as a massive validation to whatever illegible "smart randos" had previously felt like they were shouting into the void. (And I hope no one replies saying this article is a terrible rehashing of some brilliant blogger I should have been following all along... but say if it's so.)
FWIW, two people I think of as striking an interesting balance between "legibly expert on COVID" and "contrarian enough to be willing to state underreported opinions and probably grant interviews if asked" are Bob Wachter (doc at UCSF) and Jeremy Faust (ER doc in Cambridge, brief19.com cofounder). Disclaimer of bias: Wachter is a hometown hero, thus someone I'm inclined to think of positively for San-Francisco-patriotic reasons, and Jeremy is a friend from non-COVID-related endeavors.
There's also a filtering problem inherent in bourgeois media. Chomsky and Herman cover this in "Manufacturing Consent" and Parenti also in "Inventing Reality". I suggest you read these works because they point out problems which are unlikely to be rectified by this blog. I mean, your posts only appeal to a very specific subset of "experts" so already you are going to get emails from a very specific kind of person.
The Israeli non-profit Midaat ("From Knowledge") is a group of experts, including epidemiologists, who dedicate a lot of time to proliferating reliable medical knowledge in the Israeli public. They know their shit and I guess would love for any extension of their reach via regular media.
They also have several years of experience - from the polio and measles outbreaks in Israel in the past decade - in crafting messages for the public re: vaccines.
Their website is unfortunately only in Hebrew for now, but I guess they can be reached at press@midaat.org.il or I can try to connect directly to some people I know.
This is interesting: I've recently been thinking about how wikipedia has a 'sources problem' in the sense that there are some articles which are, in my uninformed opinion, left-leaning (e.g. Parler's page is part of a series on Antisemitism: https://en.wikipedia.org/wiki/Parler), and in the talk section of the page the people opposed to the more left-biased stuff really have no recourse because all the sources are themselves left-leaning, and it's hard to find a right-wing source that repudiates the left wing claims. In the world of Wikipedia, it's hard to fight sources with anything but other sources, which makes for strange situations when sources are biased and certain topics are under/over-reported by different sides.
A big part of the vision for Vox, when it was founded, was to have 'explainer' articles that could just explain what was going on without needing to stick to restrictive journalistic conventions like needing to have all claims come from the mouth of a source.
The point about your comments being stripped of context ring true to me.
I'm a political scientist and I gave a 20 minute interview to a newspaper reporter on a topic about 5 months ago. Since then, that reporter has used one sentence quotes from that interview in over a dozen articles they have written about every political event since as if I had just given the interview in response to the latest news event. I am pretty careful about what I say in interviews, but these quotes still made it sound like I had a strong political agenda (supporting the political points the journalist wanted to make) because my comments were combined with a completely different context to the one they were made in.
I would say that about 5% of interactions with the media are like this, 75% just reword your press release, and 20% engage with it well. The 5% are the danger zone, but are common enough that you can't ignore the possibility, so you end up having to be pretty careful all the time.
Maybe another way to phrase what you're getting at is that the "illegible world"—blogs, substacks, people you trust on Twitter—is based on assuming good faith, while the legible world has to be robust to people acting in bad faith.
Newspaper journalism is a machine for converting mostly-generalist writers and sources with axes to grind (or worse) into a product that often represents a pretty good first draft of the truth. Scientific journals are a machine for weeding out bad faith research; they do a terrible job of this but doing any better would probably require putting a lot of trust in some kind of dictatorial editor position, and then how would you know that they're not acting in bad faith.
And I think it's basically the same kind of people and processes (mostly, people who really like process and processes that involve a lot of people) that have driven the failures in COVID response and the successes in election administration, just to focus on the two most consequential things that happened in the US last year. I...also think you can probably have the one without the other, but what do I know.
I think the thing that finally flipped the switch on journalism for me was seeing generic headlines long enough after turning off personalization on my phone. Now all I see when I look at journalism is the gossipy kids from school gossiping. Which, if you're feeling charitable, is an important function in society: the priest class, the meaning-makers, the people who give us a shared narrative and define the in/outgroups. The meme about the Bhagavad Gita as a model for how the Priest caste relates to the Warrior caste. Journalists are part of our Priest caste.
The first headline on my phone right now is how <actor> felt about being in <movie>, and the second is about <actor>'s upcoming TV show about how they felt about being in hollywood. The covid-related headlines lower down are similar: [high-status] airline pilots looking for [low-status] side gigs, plucky citizens doing adorable things to get by.
So expecting journalists to educate us on technical subjects is a type error, like expecting a monastery to act like a vocational school.
Your journalist's problem is appropriate: their problem isn't finding true facts, it's finding someone with high enough status willing to trade exposure for the status the journalist can confer.
As an aside, I'm less confident, but I suspect, that journalism is in that summer-after-graduation phase where people are getting real jobs and going away to college, and so they're much less interested in the high school popularity contest. The status of the status-makers is becoming shakier. That's definitely bad for them, and considering the ubiquity of Priests in all of civilization, probably also bad for us.
But the type error is the main problem in this case. The disturbing thing is why the type error is happening at all: why are we expecting the Priest to give a sermon on epidemiology? The masses shouldn't have to worry about epidemiology at all! The health-smiths are supposed to do their job and then the priests tell us when to line up for our cures.
The thing I fear is that the health-smiths had to move out of the forges because the clergy (speaking now more broadly than just journalists, to include the Faucis) needed overflow dormitory space. So we're starting to realize that we all need to reinvent health-smithing from first principles and brew our own cures. So from that perspective, our journalist/Priest friend is genuinely trying to help find a real health-smith to tell us how to cobble things together, which is good in itself but a symptom of a much more disturbing failure.
How about a reputation system that tracks whether sources feel they've been misrepresented?
It could either be a public reputation system (to shame journalists into doing a better job) or a private reputation system (a database where someone who's thinking of acting as a source could look up whether the journalist who contacted them has represented sources accurately in the past).
Shouldn't be that hard to build. All you've got to do is read news articles and email every source that was quoted in the article and ask whether the article represented them fairly. You might even be able to train an AI to do this.
I think decentralized information sources (blogs, social media, readers opinions in mainstream media) already are legit and maybe as important (in the sense they have as much influence on decision makers) than centralized sources (the mainstream news, the press and brodcast tv).
Not because the general public reads more decentralized sources, but because decision makers do (probably indirectly, i guess they have teams doing it for them) as a kind of opinion poll and early warning system, to identify public reactions and serious threats to their position. I suspect this has become standard political procedure in the western world after the arab springs, well before Trump, but there is a very distinct feeling of public action being a reaction to online buzz and broad tendencies starting then... While traditional media are late and follow instead then lead public action. There is also a feeling of gouvernement reacting early to stop contestation as soon as it emerge online before it reach a critical mass, especially since early covid.
It makes sense, it would completely stupid for any entity that use opinion pooling not to also use and act in function of decentralized news sources, especially after arab springs has shown what can happen once a buzz snowball...
The same issue happens also when organisations get large, and in business generally. There's a thirst for legibility that makes decision making prone to what can be proven and what can be defended, which is a very high bar. One of the problems is the very issue of legibility makes the search for proof (in this case a source) central to its identity and burdens everything. Actions, especially innovative or timely actions, are rarely those that can be easily proven to be right. It's been a thesis of mine on Combating Organisational Inertia ..
Also a note - you make it sound like it's an issue with primary sources (Zvi reading papers) vs secondary sources (journalist looking for sources). Part of it is that the level of trust of the latter comes from "here's what I can prove" instead of "trust me, I've read this. I'm right". And the latter is only ever right eventually, seldom immediately. The journalist has to be right *within* the article, whereas Zvi has to be right *across* time. .
As an anecdote when I was at McKinsey we used to laugh once in a while when clients wanted us to give creative recommendations for the project (best ways to expand our customer base) but we'd be hampered because solutions that can't be footnoted heavily would never pass muster.
I've spent a lot of my life writing things for media use (if they want it) preparing people (usually political figures) to talk to the media, and more recently being directly interviewed, either for attribution or on background. I have also had the chance to talk to a number of journalists about their profession.
The first thing to say is that virtually any subject that attracts the interest of the media is going to be complex. If it were simple there would be nothing to be explained. So whatever your field, if you're an expert, you are almost certainly going to begin by saying "it's complicated..." What's going on in Yemen now, asks a journalist who's just found it on the map. Well, it's complicated ... And it is.
The second thing though is that journalists want a "story." They want to attract readers and they want the first paragraph to give them a reason to keep reading. So, "Shocking new evidence of Chinese military involvement in Yemen emerged yesterday", rather than "it's all very complicated and it depends who you believe." But the news story is a narrative form, and it obeys narrative laws. It's been argued that in all literature there are only five (or six or seven) basic plots. It's a bit more complicated than that in journalism, but with the "expert" stories there are really only three. The mundane one is "experts disagree" but you can only run that a few times. The more interesting ones are "heroic lone expert speaks out against corrupt and bumbling consensus" and "dangerous lunatic with influence could kill millions." In most cases, journalists have to decide which story to write about an "expert", even if they may try to nuance the actual presentation a bit. Sometimes, as more information comes out, the story swerves violently from one extreme to the other.
Consider Dr Andrew Wakefield, the MMR man, for example. At first, the UK media treated him as a crusading whistleblower against corrupt drug companies, then, as more evidence became available, they turned on him as the devil incarnate, though the power of the "lone truth teller" story is so strong that some newspapers were reluctant to abandon the stereotype. The current example in France is Prof Didier Raoult, who has been pushing Hydroxychloroquine very hard in the media as an early stage treatment. With his flowing long hair and his very minority views, he's a perfect symbol of something, but the French media can't decide of what. Is he going to save the country or is he a dangerous lunatic? Which story do I write?
So the best thing an "expert" in any field can do is to help the journalist to tell a story. That means capturing the initiative at the beginning, and giving the journalist a framework in which to write. So rather than "it's complicated" you say, "well, the thing you really need to understand is this." What you need to do is to help a journalist write a story that begins
"X is a very complicated subject, but the key to understanding it is really Y" said Dr Expert, as he offered me a cup of coffee in his book-strewn office in This University. He seemed remarkably calm and composed in spite of the endless stream of emails and text messages, callers whom he politely turned away and endless requests for interviews. "And there are two things to take away from Y. This is the first one, and this is the second." In the end, if you don't impose a narrative on the media they'll impose one on you.
Furthermore Max Kennerly seemed quite reasonable about everything else. Then I realized that it is exactly the impression that the media was actively trying to deliver to us – that the AstraZeneca trial was a garbage and a failure. The result was that otherwise smart people got unnecessary negative impression.
This is in a complete opposite of what most people involved in clinical research think. There were some issues with AstraZeneca trial but it was only a small part of a total data collected and overall they did a decent job all things considered.
I don't think that I can be that charitable with the media as Scott. If they reported expert's opinion on this they failed monumentally. Experts tend to be very critical about certain details, it's their job to tear these things apart and yet they may still think that the study was a success.
This is one example where we would be much better informed by reading from Zvi than from professional journalists.
I once, with great trepidation, spoke to a reporter about my area of expertise. I took great care with explaining the topic in the hope that the article would make me sound intelligent, but I also insisted before I spoke to her that I must see the article before it was published. That was a good thing because I would have been horribly embarrassed if anyone I knew saw that article in its original form. I ended up rewriting it myself, and I was never willing to speak to a reporter again.
I see this as a recurring problem when reading articles about subjects that I am familiar with. Many articles never get rewritten and make the reporters and the people they interview look like fools. I don’t have a solution.
A sort of similar phenomenon: development vs. sales and marketing.
I was part of a team developing [some whizzy tech thing]. We were all excited about what we were building, and posted several technical blog posts to our company's website. They were, to my mind, really good! They explained how the [whizzy tech thing] worked and didn't work, what the tradeoffs were when using it, etc.
The company grew from "a room with some engineers" to "a place with a sales and marketing department." And one day I noticed that my blog posts had been edited. They now sported buzzword-laden introductions and conclusions (that totally misrepresented the tech), and all of "cons" and "negative tradeoffs" were gone. The "how it works" sections had been dumbed down into meaningless gibberish.
I complained about this, and got them restored. But it kept happening, and eventually the engineers quit writing blog posts. Sometimes one of the marketing people would ask me to educate them about some current buzzword, and the next day I'd find weird misunderstandings posted on the Internet.
One time I went to a trade show where the sales and marketing people were pitching [whizzy tech thing]. Looking through the literature in the booth, I found several glossy printouts touting the product. They were standard trade show fare, i.e. meaningless strings of buzzwords. But *I* was listed as the author!
What if the journalist and the interviewee signed a standard contract giving the interviewee a veto right on the final article? That seems like a good way to ensure they aren't grossly misinterpreted by the journalist.
Another problem is the issue of probability. Experts rarely think in terms of 100% or 0%, but degrees of probability. Journalism does not often publish probabilities. Journalism also creates a record that can be judged in hindsight.
Suppose than an expert thinks that there is a 25% chance that large outdoor gatherings with masks are still very dangerous. They think that these gatherings should still be avoided because it's better to err on the side of caution.
They are being interviewed by a journalist about an upcoming music festival's response to COVID. How should they respond, given that they want to be a respected public health intellectual for the rest of their career?
Sorry if the numbers are wrong for the example, I am not an expert.
One very minor related issue is that there are some conventions in journalism that are both unknown to the average reader and seem really stupid. For instance, that in almost all newspapers and magazines, the reporters are not allowed the write the headlines of their articles. An average person reads a provocative headline, then reads the article, and thinks "that headline really missed the point" or "that headline seems unnecessarily glib or provocative", and blame the reporter.
It's just one more obstacle to clarity for the people trying to tell a coherent story.
I am not sure I buy the bottleneck is getting sources. The issue with journalism in general is that you just need a few sources which do not have to be representative of the consensus opinion in the field. Articles with stupid opinions that misrepresent not only the general scientific consensus but even the quoted experts get published all the time, e.g. Fox News has no problem running a story about how masks or lockdowns do not work. But maybe all the epidemiologists have already wised up to the media because of the attention their field gets now?
A possible way to address this may be for the journalist/publication to commit to clearly link to the unedited original interview transcript in any article that quotes from it (and host the transcript alongside the article).
This does not presuppose a positive coverage - but should calm most concerns about the interview being "creatively" edited .
(1) Journalists are not experts on the subjects on which they are writing. The economic problems affecting print media means that a lot of reporters who had been assigned to particular beats and built up expertise on a particular topic were laid off, transferred to something else, or resigned. Even when a reporter is specialising in a particular subject, they can't be an expert on everything in that field - a science reporter is not likely, unless they're a rare polymath, to be up on cutting-edge in biology AND chemistry AND physics, not to say the sub-specialities in each field. (2) Journalists have to translate between the experts and a lay audience. That means that the nuances get smoothed out to a simpler message of "X good, Y bad". (3) Whoever writes the headlines needs and wants attention-grabbing, so "Some experts consider maybe X on the whole not very good" isn't going anywhere. "BROCCOLI WILL KILL YOU!!! EXPERTS SAY!!!!" is what they're going for, and many people don't read past/will only remember the big headline
At some point I was giving an interview about Solstice, and asked to have control over how I was quoted, and they said no, and then I went "ah geeze" and then just spoke really carefully for the rest of the time.
It might be that newspapers just need to give sources a lot more power. But, I for articles with multiple sources I can see that being quite a bit problem, if all the sources have veto power over their part of the story.
The word "legible" feels wrong in this and the last few posts. Let me try to illustrate why.
There is a structure of power, with its tools for verifying information and conjuring action. When this structure sanctions an action (whether the rationale for that action makes sense or not), it becomes The Right Way To Do Things. When that structure doesn't sanction an action, it becomes basically nonsense. Maybe the way it sanctions things is by doing whatever the Emperor likes, or perhaps it's something like representative democracy. Either way, there are people in this structure who get to approve or condemn things, according to whatever rule of conduct, and that's the dynamic that determines if a piece of knowledge or action is sanctioned or not. A blogger may as well create something completely legible (in the sense that it's scholarly, well-reasoned, easy to understand, etc) that can't get officially sanctioned because the way the power structure verifies things isn't likely to verify the blogger.
I might pitch it as something like "Vox meets 538, run on rationalist principles." The focus would be making the knowledge acquired and generated by rationalists available to the public, backed by some kind of legible expertise, and then doing analysis and making explicit forecasts which would then carry with them the weight of a publication with a growing and eventually established public reputation.
I mean in many cases the difference between a blog and an online news magazine is that some bloggers decided to take what they were writing and call it an online news magazine. Current Affairs comes to mind. Journalistic legitimacy isn't actually that hard to come by - you can generally just claim it, and if you have enough of an audience to acknowledge the claim, it's yours. Rationalism already has that audience.
I did some web development for a journalism startup a few years back so I have some loose knowledge of how it works and a few contacts I could tap for advice and assistance. I also have some editorial experience. I'd definitely be willing to contribute whatever I can to such an effort.
> But if a reporter wants to write "X Is Dumb And All Epidemiologists Are Idiots For Believing It" they can slice and dice your interview
I think that puts the wrong part of the worry at the forefront. It's not just that you're afraid a reporter may have an axe to grind. It's that you're pretty sure that MOST reporters will not UNDERSTAND you well enough to be able to paraphrase you or summarize your position.
I haven't talked to the press much, but even I have had reporters paraphrase things that I said in ways that I'm genuinely sure they thought didn't alter the meaning in any way... but did in fact change the meaning and make the statement outright wrong. Not wrong in a trivial way. Wrong with a capital W. Because they didn't UNDERSTAND the statement to begin with; they had no context and maybe no knowledge even of the language of the fopic. And there's usually an editor who gets a SECOND try to get it wrong. And you rarely get a chance to correct that before they publish it.
And THEN the readers get a chance to misinterpret it.
I feel like the norm should be that experts can revoke their quotes after seeing the article written before publication (like co-authoring for peer review).
I understand why a journalist would reserve the right to curate their quotes when doing a story about a person. However any case in which reporting doe not rely on the journalist's own authority, I don't see why how it make sense for a journalist to maintain unilateral authority over the content of the story.
This suggests there's a market opening for a news organization that had the ability to make a credible commitment to publishing *exact* quotes of experts of agreed-upon length.
As in, "if you give us 100 words of what you think about topic Y, we guarantee if we will either print none of it or print all of it once as an uninterrupted block of text of uniform size"
I hadn’t heard about your Substack as I’m on the outer fringe of the “rationalist community” such as it is. That is, until the NYT finally shipped the hit piece. So, uh, thanks, NYT. I look forward to the horrors of thinking unfiltered through cancel-culture-stricken editors at the Grey Lady.
I don't think there's a typology of media stories as such, though it would be nice to have one. Most of what I said I heard from journalists, or was based on my experience of seeing things I wrote deformed and fitted into a narrative that was already established. The point I've often heard journalists make is they need to be able to answer the question "where's the story?" The answer they often have to give hands to fit in one sentence. "It's complicated" isn't enough.
Who needs journalists? Experts should start their own blogs.
I assume there's some sort of barrier there. Though Andrew Gelman and Derek Lowe have been my most trusted sources throughout, so it's not impermeable.
You'd think people who do epidemiology as a day job are even more frustrated than the rest of us at how bad mass-media coverage has been.
"Sources don't usually get to approve the way they're quoted in the article, or to see the article before it gets published."
Changing that norm right there, for some varieties of journalism, particularly those that rely on expert sources on nuanced topics, might help a lot. For example, I can imagine a journalist saying "here's a draft of the part of my article where I quote you, with some of the context... did I get that mostly right?"
I know much journalism is time- and resource-strapped, and a lot of sources don't want to go to extra effort to do someone else's proofreading for them, but... if I knew that sort of thing were the norm at a journalistic outfit, I'd both be more likely to read their output and more likely to cooperate as a source.
This is an interesting problem. I wonder if this problem has also gotten harder (or changed) over time.
Or to put it another way: a lot of our society's epistemic processes have come under assault all at once. From my perspective (and I wish I had data in front of me backing it up), many media sources have come under a lot of pressure to generate clicks quickly, and journalists have less time to dedicate to the quality of claims.
Additionally, the development of "alternative" medias and academies, have in many cases acted to push their more legitimate counterparts to try to circle the wagons.
To be clear: these gaps have likely always existed, and they likely always will exist to some extent. (Look inside a modern corporation, and you'll find that there are gaps between the analysts and top executives in terms of how the system works)
My guess though is that the "circling the wagons" effect is likely the dominant one though. Which is to say that if there was a presumption all speakers were honest & reliable, then the overall risks would be lower.
You do sometimes get mainstream journalism that involves thorough research, lots of interviews with experts, and ultimately a journalist giving a synthesis of their own opinion. It's really great when it works.
For example, I thought this piece in New York Magazine about the SARS-CoV-2 lab leak origin was good. I hope it came as a massive validation to whatever illegible "smart randos" had previously felt like they were shouting into the void. (And I hope no one replies saying this article is a terrible rehashing of some brilliant blogger I should have been following all along... but say if it's so.)
https://nymag.com/intelligencer/article/coronavirus-lab-escape-theory.html
FWIW, two people I think of as striking an interesting balance between "legibly expert on COVID" and "contrarian enough to be willing to state underreported opinions and probably grant interviews if asked" are Bob Wachter (doc at UCSF) and Jeremy Faust (ER doc in Cambridge, brief19.com cofounder). Disclaimer of bias: Wachter is a hometown hero, thus someone I'm inclined to think of positively for San-Francisco-patriotic reasons, and Jeremy is a friend from non-COVID-related endeavors.
There's also a filtering problem inherent in bourgeois media. Chomsky and Herman cover this in "Manufacturing Consent" and Parenti also in "Inventing Reality". I suggest you read these works because they point out problems which are unlikely to be rectified by this blog. I mean, your posts only appeal to a very specific subset of "experts" so already you are going to get emails from a very specific kind of person.
The Israeli non-profit Midaat ("From Knowledge") is a group of experts, including epidemiologists, who dedicate a lot of time to proliferating reliable medical knowledge in the Israeli public. They know their shit and I guess would love for any extension of their reach via regular media.
They also have several years of experience - from the polio and measles outbreaks in Israel in the past decade - in crafting messages for the public re: vaccines.
Their website is unfortunately only in Hebrew for now, but I guess they can be reached at press@midaat.org.il or I can try to connect directly to some people I know.
This is interesting: I've recently been thinking about how wikipedia has a 'sources problem' in the sense that there are some articles which are, in my uninformed opinion, left-leaning (e.g. Parler's page is part of a series on Antisemitism: https://en.wikipedia.org/wiki/Parler), and in the talk section of the page the people opposed to the more left-biased stuff really have no recourse because all the sources are themselves left-leaning, and it's hard to find a right-wing source that repudiates the left wing claims. In the world of Wikipedia, it's hard to fight sources with anything but other sources, which makes for strange situations when sources are biased and certain topics are under/over-reported by different sides.
A big part of the vision for Vox, when it was founded, was to have 'explainer' articles that could just explain what was going on without needing to stick to restrictive journalistic conventions like needing to have all claims come from the mouth of a source.
They've had... mixed success.
The point about your comments being stripped of context ring true to me.
I'm a political scientist and I gave a 20 minute interview to a newspaper reporter on a topic about 5 months ago. Since then, that reporter has used one sentence quotes from that interview in over a dozen articles they have written about every political event since as if I had just given the interview in response to the latest news event. I am pretty careful about what I say in interviews, but these quotes still made it sound like I had a strong political agenda (supporting the political points the journalist wanted to make) because my comments were combined with a completely different context to the one they were made in.
I would say that about 5% of interactions with the media are like this, 75% just reword your press release, and 20% engage with it well. The 5% are the danger zone, but are common enough that you can't ignore the possibility, so you end up having to be pretty careful all the time.
Maybe another way to phrase what you're getting at is that the "illegible world"—blogs, substacks, people you trust on Twitter—is based on assuming good faith, while the legible world has to be robust to people acting in bad faith.
Newspaper journalism is a machine for converting mostly-generalist writers and sources with axes to grind (or worse) into a product that often represents a pretty good first draft of the truth. Scientific journals are a machine for weeding out bad faith research; they do a terrible job of this but doing any better would probably require putting a lot of trust in some kind of dictatorial editor position, and then how would you know that they're not acting in bad faith.
And I think it's basically the same kind of people and processes (mostly, people who really like process and processes that involve a lot of people) that have driven the failures in COVID response and the successes in election administration, just to focus on the two most consequential things that happened in the US last year. I...also think you can probably have the one without the other, but what do I know.
I think the thing that finally flipped the switch on journalism for me was seeing generic headlines long enough after turning off personalization on my phone. Now all I see when I look at journalism is the gossipy kids from school gossiping. Which, if you're feeling charitable, is an important function in society: the priest class, the meaning-makers, the people who give us a shared narrative and define the in/outgroups. The meme about the Bhagavad Gita as a model for how the Priest caste relates to the Warrior caste. Journalists are part of our Priest caste.
The first headline on my phone right now is how <actor> felt about being in <movie>, and the second is about <actor>'s upcoming TV show about how they felt about being in hollywood. The covid-related headlines lower down are similar: [high-status] airline pilots looking for [low-status] side gigs, plucky citizens doing adorable things to get by.
So expecting journalists to educate us on technical subjects is a type error, like expecting a monastery to act like a vocational school.
Your journalist's problem is appropriate: their problem isn't finding true facts, it's finding someone with high enough status willing to trade exposure for the status the journalist can confer.
As an aside, I'm less confident, but I suspect, that journalism is in that summer-after-graduation phase where people are getting real jobs and going away to college, and so they're much less interested in the high school popularity contest. The status of the status-makers is becoming shakier. That's definitely bad for them, and considering the ubiquity of Priests in all of civilization, probably also bad for us.
But the type error is the main problem in this case. The disturbing thing is why the type error is happening at all: why are we expecting the Priest to give a sermon on epidemiology? The masses shouldn't have to worry about epidemiology at all! The health-smiths are supposed to do their job and then the priests tell us when to line up for our cures.
The thing I fear is that the health-smiths had to move out of the forges because the clergy (speaking now more broadly than just journalists, to include the Faucis) needed overflow dormitory space. So we're starting to realize that we all need to reinvent health-smithing from first principles and brew our own cures. So from that perspective, our journalist/Priest friend is genuinely trying to help find a real health-smith to tell us how to cobble things together, which is good in itself but a symptom of a much more disturbing failure.
How about a reputation system that tracks whether sources feel they've been misrepresented?
It could either be a public reputation system (to shame journalists into doing a better job) or a private reputation system (a database where someone who's thinking of acting as a source could look up whether the journalist who contacted them has represented sources accurately in the past).
Shouldn't be that hard to build. All you've got to do is read news articles and email every source that was quoted in the article and ask whether the article represented them fairly. You might even be able to train an AI to do this.
I think it is all about picking your sources.
These folks warned about COVID before it was a thing:
https://www.endcoronavirus.org/
I think decentralized information sources (blogs, social media, readers opinions in mainstream media) already are legit and maybe as important (in the sense they have as much influence on decision makers) than centralized sources (the mainstream news, the press and brodcast tv).
Not because the general public reads more decentralized sources, but because decision makers do (probably indirectly, i guess they have teams doing it for them) as a kind of opinion poll and early warning system, to identify public reactions and serious threats to their position. I suspect this has become standard political procedure in the western world after the arab springs, well before Trump, but there is a very distinct feeling of public action being a reaction to online buzz and broad tendencies starting then... While traditional media are late and follow instead then lead public action. There is also a feeling of gouvernement reacting early to stop contestation as soon as it emerge online before it reach a critical mass, especially since early covid.
It makes sense, it would completely stupid for any entity that use opinion pooling not to also use and act in function of decentralized news sources, especially after arab springs has shown what can happen once a buzz snowball...
The same issue happens also when organisations get large, and in business generally. There's a thirst for legibility that makes decision making prone to what can be proven and what can be defended, which is a very high bar. One of the problems is the very issue of legibility makes the search for proof (in this case a source) central to its identity and burdens everything. Actions, especially innovative or timely actions, are rarely those that can be easily proven to be right. It's been a thesis of mine on Combating Organisational Inertia ..
Also a note - you make it sound like it's an issue with primary sources (Zvi reading papers) vs secondary sources (journalist looking for sources). Part of it is that the level of trust of the latter comes from "here's what I can prove" instead of "trust me, I've read this. I'm right". And the latter is only ever right eventually, seldom immediately. The journalist has to be right *within* the article, whereas Zvi has to be right *across* time. .
As an anecdote when I was at McKinsey we used to laugh once in a while when clients wanted us to give creative recommendations for the project (best ways to expand our customer base) but we'd be hampered because solutions that can't be footnoted heavily would never pass muster.
I've spent a lot of my life writing things for media use (if they want it) preparing people (usually political figures) to talk to the media, and more recently being directly interviewed, either for attribution or on background. I have also had the chance to talk to a number of journalists about their profession.
The first thing to say is that virtually any subject that attracts the interest of the media is going to be complex. If it were simple there would be nothing to be explained. So whatever your field, if you're an expert, you are almost certainly going to begin by saying "it's complicated..." What's going on in Yemen now, asks a journalist who's just found it on the map. Well, it's complicated ... And it is.
The second thing though is that journalists want a "story." They want to attract readers and they want the first paragraph to give them a reason to keep reading. So, "Shocking new evidence of Chinese military involvement in Yemen emerged yesterday", rather than "it's all very complicated and it depends who you believe." But the news story is a narrative form, and it obeys narrative laws. It's been argued that in all literature there are only five (or six or seven) basic plots. It's a bit more complicated than that in journalism, but with the "expert" stories there are really only three. The mundane one is "experts disagree" but you can only run that a few times. The more interesting ones are "heroic lone expert speaks out against corrupt and bumbling consensus" and "dangerous lunatic with influence could kill millions." In most cases, journalists have to decide which story to write about an "expert", even if they may try to nuance the actual presentation a bit. Sometimes, as more information comes out, the story swerves violently from one extreme to the other.
Consider Dr Andrew Wakefield, the MMR man, for example. At first, the UK media treated him as a crusading whistleblower against corrupt drug companies, then, as more evidence became available, they turned on him as the devil incarnate, though the power of the "lone truth teller" story is so strong that some newspapers were reluctant to abandon the stereotype. The current example in France is Prof Didier Raoult, who has been pushing Hydroxychloroquine very hard in the media as an early stage treatment. With his flowing long hair and his very minority views, he's a perfect symbol of something, but the French media can't decide of what. Is he going to save the country or is he a dangerous lunatic? Which story do I write?
So the best thing an "expert" in any field can do is to help the journalist to tell a story. That means capturing the initiative at the beginning, and giving the journalist a framework in which to write. So rather than "it's complicated" you say, "well, the thing you really need to understand is this." What you need to do is to help a journalist write a story that begins
"X is a very complicated subject, but the key to understanding it is really Y" said Dr Expert, as he offered me a cup of coffee in his book-strewn office in This University. He seemed remarkably calm and composed in spite of the endless stream of emails and text messages, callers whom he politely turned away and endless requests for interviews. "And there are two things to take away from Y. This is the first one, and this is the second." In the end, if you don't impose a narrative on the media they'll impose one on you.
When Tyler Cowen reported that one these people is a bad reasoner, many were confused, which one?
https://twitter.com/mattyglesias/status/1357064029332795392
Furthermore Max Kennerly seemed quite reasonable about everything else. Then I realized that it is exactly the impression that the media was actively trying to deliver to us – that the AstraZeneca trial was a garbage and a failure. The result was that otherwise smart people got unnecessary negative impression.
This is in a complete opposite of what most people involved in clinical research think. There were some issues with AstraZeneca trial but it was only a small part of a total data collected and overall they did a decent job all things considered.
I don't think that I can be that charitable with the media as Scott. If they reported expert's opinion on this they failed monumentally. Experts tend to be very critical about certain details, it's their job to tear these things apart and yet they may still think that the study was a success.
This is one example where we would be much better informed by reading from Zvi than from professional journalists.
It wouldn't be a complete solution, but would be a good thing for journalists to include a link to their subjects' blog or whatever?
It would both be a venue for correction and expansion and a source for extensive quarrelling.
Alternatively, should savvy readers make a habit of checking on whether a source has more to say about an article?
I once, with great trepidation, spoke to a reporter about my area of expertise. I took great care with explaining the topic in the hope that the article would make me sound intelligent, but I also insisted before I spoke to her that I must see the article before it was published. That was a good thing because I would have been horribly embarrassed if anyone I knew saw that article in its original form. I ended up rewriting it myself, and I was never willing to speak to a reporter again.
I see this as a recurring problem when reading articles about subjects that I am familiar with. Many articles never get rewritten and make the reporters and the people they interview look like fools. I don’t have a solution.
A sort of similar phenomenon: development vs. sales and marketing.
I was part of a team developing [some whizzy tech thing]. We were all excited about what we were building, and posted several technical blog posts to our company's website. They were, to my mind, really good! They explained how the [whizzy tech thing] worked and didn't work, what the tradeoffs were when using it, etc.
The company grew from "a room with some engineers" to "a place with a sales and marketing department." And one day I noticed that my blog posts had been edited. They now sported buzzword-laden introductions and conclusions (that totally misrepresented the tech), and all of "cons" and "negative tradeoffs" were gone. The "how it works" sections had been dumbed down into meaningless gibberish.
I complained about this, and got them restored. But it kept happening, and eventually the engineers quit writing blog posts. Sometimes one of the marketing people would ask me to educate them about some current buzzword, and the next day I'd find weird misunderstandings posted on the Internet.
One time I went to a trade show where the sales and marketing people were pitching [whizzy tech thing]. Looking through the literature in the booth, I found several glossy printouts touting the product. They were standard trade show fare, i.e. meaningless strings of buzzwords. But *I* was listed as the author!
Why can’t a source just go back and forth with the journalist until they’re quoted in a way that makes sense and doesn’t mangle what they were saying?
Do journalists typically refuse to do this? Is there some reason for it if so?
What if the journalist and the interviewee signed a standard contract giving the interviewee a veto right on the final article? That seems like a good way to ensure they aren't grossly misinterpreted by the journalist.
Another problem is the issue of probability. Experts rarely think in terms of 100% or 0%, but degrees of probability. Journalism does not often publish probabilities. Journalism also creates a record that can be judged in hindsight.
Suppose than an expert thinks that there is a 25% chance that large outdoor gatherings with masks are still very dangerous. They think that these gatherings should still be avoided because it's better to err on the side of caution.
They are being interviewed by a journalist about an upcoming music festival's response to COVID. How should they respond, given that they want to be a respected public health intellectual for the rest of their career?
Sorry if the numbers are wrong for the example, I am not an expert.
One very minor related issue is that there are some conventions in journalism that are both unknown to the average reader and seem really stupid. For instance, that in almost all newspapers and magazines, the reporters are not allowed the write the headlines of their articles. An average person reads a provocative headline, then reads the article, and thinks "that headline really missed the point" or "that headline seems unnecessarily glib or provocative", and blame the reporter.
It's just one more obstacle to clarity for the people trying to tell a coherent story.
I am not sure I buy the bottleneck is getting sources. The issue with journalism in general is that you just need a few sources which do not have to be representative of the consensus opinion in the field. Articles with stupid opinions that misrepresent not only the general scientific consensus but even the quoted experts get published all the time, e.g. Fox News has no problem running a story about how masks or lockdowns do not work. But maybe all the epidemiologists have already wised up to the media because of the attention their field gets now?
A possible way to address this may be for the journalist/publication to commit to clearly link to the unedited original interview transcript in any article that quotes from it (and host the transcript alongside the article).
This does not presuppose a positive coverage - but should calm most concerns about the interview being "creatively" edited .
Well there are two obvious solutions, both of which are unlikely to be implemented, for reasons of organizational inertia
. 1: let the interiviewee approve the quotations of them contained in the interview before publication.
2: Let the expert write the damn thing themself.
I am still in this mode where every time I finish the newest ACT post I just think, "goddammit, I am glad Scott is back."
(1) Journalists are not experts on the subjects on which they are writing. The economic problems affecting print media means that a lot of reporters who had been assigned to particular beats and built up expertise on a particular topic were laid off, transferred to something else, or resigned. Even when a reporter is specialising in a particular subject, they can't be an expert on everything in that field - a science reporter is not likely, unless they're a rare polymath, to be up on cutting-edge in biology AND chemistry AND physics, not to say the sub-specialities in each field. (2) Journalists have to translate between the experts and a lay audience. That means that the nuances get smoothed out to a simpler message of "X good, Y bad". (3) Whoever writes the headlines needs and wants attention-grabbing, so "Some experts consider maybe X on the whole not very good" isn't going anywhere. "BROCCOLI WILL KILL YOU!!! EXPERTS SAY!!!!" is what they're going for, and many people don't read past/will only remember the big headline
At some point I was giving an interview about Solstice, and asked to have control over how I was quoted, and they said no, and then I went "ah geeze" and then just spoke really carefully for the rest of the time.
It might be that newspapers just need to give sources a lot more power. But, I for articles with multiple sources I can see that being quite a bit problem, if all the sources have veto power over their part of the story.
The word "legible" feels wrong in this and the last few posts. Let me try to illustrate why.
There is a structure of power, with its tools for verifying information and conjuring action. When this structure sanctions an action (whether the rationale for that action makes sense or not), it becomes The Right Way To Do Things. When that structure doesn't sanction an action, it becomes basically nonsense. Maybe the way it sanctions things is by doing whatever the Emperor likes, or perhaps it's something like representative democracy. Either way, there are people in this structure who get to approve or condemn things, according to whatever rule of conduct, and that's the dynamic that determines if a piece of knowledge or action is sanctioned or not. A blogger may as well create something completely legible (in the sense that it's scholarly, well-reasoned, easy to understand, etc) that can't get officially sanctioned because the way the power structure verifies things isn't likely to verify the blogger.
What about starting a rationalist news outlet?
I might pitch it as something like "Vox meets 538, run on rationalist principles." The focus would be making the knowledge acquired and generated by rationalists available to the public, backed by some kind of legible expertise, and then doing analysis and making explicit forecasts which would then carry with them the weight of a publication with a growing and eventually established public reputation.
I mean in many cases the difference between a blog and an online news magazine is that some bloggers decided to take what they were writing and call it an online news magazine. Current Affairs comes to mind. Journalistic legitimacy isn't actually that hard to come by - you can generally just claim it, and if you have enough of an audience to acknowledge the claim, it's yours. Rationalism already has that audience.
I did some web development for a journalism startup a few years back so I have some loose knowledge of how it works and a few contacts I could tap for advice and assistance. I also have some editorial experience. I'd definitely be willing to contribute whatever I can to such an effort.
If you or zvi are trying to be legible, you should start by hiring an editor to cut your article length in half. Not being facetious.
> But if a reporter wants to write "X Is Dumb And All Epidemiologists Are Idiots For Believing It" they can slice and dice your interview
I think that puts the wrong part of the worry at the forefront. It's not just that you're afraid a reporter may have an axe to grind. It's that you're pretty sure that MOST reporters will not UNDERSTAND you well enough to be able to paraphrase you or summarize your position.
I haven't talked to the press much, but even I have had reporters paraphrase things that I said in ways that I'm genuinely sure they thought didn't alter the meaning in any way... but did in fact change the meaning and make the statement outright wrong. Not wrong in a trivial way. Wrong with a capital W. Because they didn't UNDERSTAND the statement to begin with; they had no context and maybe no knowledge even of the language of the fopic. And there's usually an editor who gets a SECOND try to get it wrong. And you rarely get a chance to correct that before they publish it.
And THEN the readers get a chance to misinterpret it.
I feel like the norm should be that experts can revoke their quotes after seeing the article written before publication (like co-authoring for peer review).
I understand why a journalist would reserve the right to curate their quotes when doing a story about a person. However any case in which reporting doe not rely on the journalist's own authority, I don't see why how it make sense for a journalist to maintain unilateral authority over the content of the story.
This suggests there's a market opening for a news organization that had the ability to make a credible commitment to publishing *exact* quotes of experts of agreed-upon length.
As in, "if you give us 100 words of what you think about topic Y, we guarantee if we will either print none of it or print all of it once as an uninterrupted block of text of uniform size"
I hadn’t heard about your Substack as I’m on the outer fringe of the “rationalist community” such as it is. That is, until the NYT finally shipped the hit piece. So, uh, thanks, NYT. I look forward to the horrors of thinking unfiltered through cancel-culture-stricken editors at the Grey Lady.
I don't think there's a typology of media stories as such, though it would be nice to have one. Most of what I said I heard from journalists, or was based on my experience of seeing things I wrote deformed and fitted into a narrative that was already established. The point I've often heard journalists make is they need to be able to answer the question "where's the story?" The answer they often have to give hands to fit in one sentence. "It's complicated" isn't enough.