Can someone that understands the AGI doomerism perspective explain why recent innovations in AI have made people so concerned about superintelligent AGI in the near future?
Like, I don't understand how a super smart AGI trained on past and present data could somehow generate:
1) endless innovations
2) iteratively improve itself
It seems to me that superintelligent AGI would need to be fundamentally different from GPT-4 or PaLM-E, as the their knowledge is mostly constrained by their training data and RLHF. Superintelligent AGI, however, I'm guessing would need to be capable of generating new knowledge itself, but it does not look like we're anywhere close to this right now.
What am I missing here? Why should I be terrified?
The number I would like and don't have is how many wet markets there are in the world with whatever features, probably selling wild animals, make the Wuhan market a candidate for the origin of Covid. If it is the only one, then Covid appearing in Wuhan from it is no odder a coincidence than Covid appearing in the same city where the WIV was researching bat viruses. If it was one of fifty or a hundred (not necessarily all in China), then the application of Bayes' Theorem implies a posterior probability for the lab leak theory much higher than whatever the prior was.
Hey folks, I'm visiting the Bay Area soon for a conference, and I'm staying two weeks, so enough time to do non-conference things. My dates are the 17th of March to the 31st - are there any rationalist meetups I could attend?
If you'd rather not say them publicly, my email is my first name followed by my surname (both of which are in my username) followed by the numeral 1 followed by @gmail.com
I have been putting together an argument in favour of expecting AI to have a slow takeoff. Part 1 of the argument, though presented in a way that is seemingly unrelated to AI, can be seen here: https://www.youtube.com/watch?v=Zk6uHi_Rdm4
The British government is among the most dedicated in the world when it comes to keeping its people safe… from thinking the wrong thoughts. Which is why it runs “Prevent,” a program to prevent terrorism by encouraging people to rat out their neighbors to counter-terrorism police for wrong-think, as well as using behavioral analysis to pre-identify dangerous extremists to monitor. Recently some documents from Prevent became public as part of an official review, and, as Douglass Murray reports here, the factors designated evidence of “extremism” are rather interesting…
"When I first saw these documents I felt a sort of white-hot anger. But then I read on and saw that these same taxpayer-funded fools provide lists of other books shared by people who have sympathies with the ‘far-right and Brexit’. Key signs that people have fallen into this abyss include watching the Kenneth Clark TV series Civilisation, The Thick of It and Great British Railway Journeys. I need to stress again that I am not making this up. This has all been done on your dime and mine in order to stop ‘extremism’ in these islands.
There is also a reading list of historical texts which produce red flags to RICU. These include Leviathan by Thomas Hobbes, John Locke’s Two Treatises of Government and Edmund Burke’s Reflections on the Revolution in France, as well as works by Thomas Carlyle and Adam Smith. Elsewhere RICU warns that radicalisation could occur from books by authors including C.S. Lewis, J.R.R. Tolkien, Aldous Huxley and Joseph Conrad. I kid you not, though it seems that all satire is dead, but the list of suspect books also includes 1984 by George Orwell..."
You are a 20 year old who was born to a single mom as the result of rape. You love your mother dearly. You live in a country where abortion on demand is legal, but she choose not to have you aborted. The traumatic effects of the rape are still with your mother 20 years later and are awful for her.
One day, you are transported back in time to the day your mother was raped, and have the opportunity to stop it.
Do you do so, eliminating yourself from existence in the process?
Um, due to, you know, events, I have an emergency update to my Russo-Ukrainian war forecast, posted without explanation of either methodology or reasons. Previous update (with methodology) here: https://astralcodexten.substack.com/p/open-thread-261/comment/12342095. Current probabilities:
3) After the Founding King dies, all his powers pass to the Founding Queen.
4) Once she dies, an election is held to assign a replacement head of the Royal Family. Only people who are descended from the Founding King and Founding Queen, and who share at least 12.5% of their DNA can run for the office or hold office. The same requirement exists for all top political positions outside the monarchy, like Parliament, the Supreme Court, and heads of government agencies, including the military.
5) Anyone who shares at least 6.25% of their DNA with the Founding King and Founding Queen can vote in the election.
6) It's common and accepted for people sharing 6.25% or more of their DNA with the Founders to clone themselves once they get old. There is a social imperative to keep the voting/leadership population pure and continually expanding in size. There's also an imperative to use genetic engineering to make each generation smarter, healthier, and more competent than the last.
7) Large numbers of people who have less than 6.25% of the Founders' DNA also live in the country, and they have all the same rights except for voting for politicians and holding high public office.
I thought up this idea recently, and think it carries some advantages over existing systems. For one, since the Head of the Royal Family is elected, it reduces the odds of a truly bad leader getting the post, which has been the Achilles' Heel of monarchies based on birth order. Also, the blood quantum requirements would help ensure that the leadership caste was unified, preventing the society from getting too fractious. Add in a culture of genetic improvement and fecundity, and you eventually get a caste with thousands or even millions of highly competent members who could fill out the ranks of national leadership with lots of people to spare.
However, from the perspective of progress of rationalistic thought, I think that these trends may underestimate the more meaningful shift. If someone is concerned with religion leading people to think and act in ways that are irrational, then it doesn't really matter which box someone ticks off on a survey - it matters how they actually think and act.
My impression is that in the 1990s, belief in the supernatural played a much larger role in people's actual behavior, than today - even more than the aforementioned trend might imply.
For example, I think it used to not be uncommon for police to consult with psychics to try to solve crimes (I can't find numbers on the frequency of such occurrences - there was a notable case of "the St. Louis Jane Doe" in 1994 when the police mailed a vital piece of evidence in a murder case to a psychic and never got it back). And psychics, like Sylvia Browne, in general seemed to have occupied a much more significant role in the 90s and aughts.
Horoscopes are still popular, but it seems like they have mostly been relegated to the realm of fun and games, rather than being consulted for serious things like solving murders.
Instead, is seems like even religious people increasingly act in ways that are consistent with naturalistic explanations of the world.
Conversely, there may be a trend of non-religious people increasingly adopting ways of thought and action that are less consistent with naturalism, and more consistent with dogmatism or religion. For example, David Friedman notes that a priori rejection of sex differences in human behavior is not what one would expect on the basis of belief in evolution, and would be more consistent with a divine religious model, but is probably more common among those who would not tick off a box like "Christian."
Still, it seems like the overall trend in actual meaningful behavior towards naturalism, although I can't think of the best metrics to study that.
For example, Church attendance would be an obvious proxy, but it's not really what I'm interested in. People can attend Church for a variety of reasons, and even nod along with everything not disingenuously, but nevertheless behave in ways that are consistent with naturalism. For example, if they had a sick child, they may pray for the child, but it would not occur to them to pray instead of seeking actual medical treatment. Praying doesn't have much of a cost, so it isn't a great proxy for measuring how deeply someone believes something. I think mailing off evidence in a murder case to a psychic, is a much better indicator of deeply believing in the supernatural - and more interesting for me, is the sort of concerning behavior that I would hope is eliminated.
Ok, so what's the current thinking on nicotine? If someone starts chewing nicotine gum, will it have a net health benefit or harm? And what if they just do it a couple times a week in order to avoid addiction? Or use it on non-coffee days (for people who don't want to build caffeine tolerance.)
It's hard to find research on this, and I fear that it's just politically incorrect to study possible benefit of nicotine usage.
How much will the US be on the hook for once the Ukraine war is over? I'm already seeing talk of something like a marshall plan for Ukraine (assuming that Russia loses, which again, is assumed only until it is time to demand more weapon shipments to ukraine), with estimates that it will take half a trillion dollars to fix ukraine already (and a lot more by the time this the war is over).
Obviously this won't all come from the US, but it's hard to imagine we won't be footing the lion's share of the bill in some form or other.
It's bad enough that Zelesnkyy and his western cronies would rather another hundred thousand young ukrainian men be killed or disabled to avoid Zelesnkyy losing a single inch of clay under his control, but by refusing to even entertain the notion of settling for anything but an INCREASE in territory comapred with the start of the war, he's basically saying that Americans will have to pay hundreds of billions of dollars for some shithole regions that were generating a tiny fraction of this money in GDP and tax revenue before the war even started.
The trolley problem is often seen as some abstract thing with little practical relevance. It is the complete opposite in fact: it is a description of our present situation.
I recently posted a comment on Freddie deBoer's Substack to the effect of "I'm confident courts will not allow government censorship to stand, so I'm much more concerned about social pressure (to censor speech) and think it's is a much bigger problem, by (making up numbers here) several orders of magnitude." Evidently that's not a popular view there (peer and descendent comments were voted more highly, including ones quite sympathetic to that position). Upvotes are hardly a dispositive measure of things, but it has me wondering about my intuition here.
Are ACX readers from the US more worried about government or private censorship?
As one of the investors who isn't participating in Manifund: The potential returns aren't even a factor in my not participating. What is a concern is:
* Academic grant funding is very different than investing in corporations, and this falls in the uncanny valley between them (in a bad way).
* The myopic nature (experimenting with prediction markets by funding experiments on prediction markets) is kind of a turn-off.
* There is a rule of thumb in enterprise sales that you can't price a product between $1000 and $10000 - it's not cheap-enough to be an impulse sale and it's not expensive enough for a salesperson to be interested. There may be a similar dynamic in play here.
Does the Peter Principle hold on an organisational level?
That is to say, an organisation (that has a reasonable level of autonomy as regards the scope of its activities) will respond to getting good at something by doing more difficult things, which it won't be good at yet*.
So everybody is winging it but the competant ones will be winging progressively greater things.
Last week, I posted a piece I wrote arguing that helicopters should be taxed based on their noise externalities. I felt there was a subtext to a lot of the responses I got along the lines of “oh look, another big government progressive who wants to regulate everything.”
This was strange to me, since I sometimes conceptualize myself as a libertarian. Then I went back and looked at my writing and discovered that indeed, it is quite statist. I think this is a byproduct of basically just attempting to write a lot, and I think it suggests something about why libertarian intellectuals are rare in the mainstream media, so I wrote about that (https://omnibudsman.substack.com/p/why-libertarian-public-intellectuals).
Everyone seems to agree that the current situation with the appointment of US supreme justices[1] is less than ideal, where the party in power of the senate can just block any appointee, and then install a partisan when in control of both branches. The loudest responses seem to be "Yeah, [party] should just pack the courts" (self-evidently undemocratic), "something something term limits" (probably unconstitutional, getting amendments for this is currently infeasible), "each president should just appoint one" (either has the same problems as above or just leads to all partisan justices).
Here's an attempted proposal to hopefully get incentives in the right place without relying too much on "norms", that I'd like feedback from the crowd to tell me in what ways I'm being an idiot an it can be improved:
- Every president gets one justice appointment per term, and the total number of justices is left mostly uncapped (If it becomes *too* unwieldy, a soft cap at 1/4 the size of senate, currently 25)
- The Senate *must* vote on every nominee on a "reasonable" timeline, or they are considered to be providing implicit consent[2] for the nominee. "No" votes should include guidance on what they would consider a more reasonable candidate.
- If they can't come to an agreement with the Senate: at any time during the last year of their presidency, up to a week (or more?) before the next election, the President may set a "fallback" judge, for whom a "no" vote on another candidate is considered to be *implicit consent* by the senate for that candidate being immediately appointed by *the next* (term's) president at any point during that president's term.
Ideally, this incentivizes the President and the Senate to come to a compromise early in his term, so they avoid a partisan fallback judge and he gets an early favorable justice.
Of course, this implies that any two-term president gets essentially a free partisan fallback justice, which should be fine given
(a) every president gets one, and the ping-pong of the parties should balance this out
(b) who that nominee would be becomes an explicit campaign point, so they would have the mandate of democracy
Thought, criticisms, concerns?
[1] (and less-than-supreme federal/circuit justices, but one issue at a time)
TL;DR: If average lead levels drop, but gaps in IQ, crime, illness, etc. in terms of antecedents of lead exposure remain somewhat constant, the effect size of a unit of lead necessarily increases. For IQ, in the last 50 years, the effect of one μg/dL of lead multiplied by more than 100 times.
I have often seen infographs with the message "everything in this particular field or walk of life is controlled by a single-digit number of corporations". Here are 2 examples I got by just typing "everything is controlled by a few companies" into google image search :
There is another somewhat-different example of a field I'm following with interest (though I'm still a layman), which is semiconductors and chip manufacturing. The seimconductor industry in general is ridicously over-consolidated, only 3 corpos - Intel, Samsung, and TSMC - are cutting-edge manufacturers, every other chip company you have heard of - Apple, Arm, AMD - are "fabless", they don't have factories, they just design (which, to be completely clear, is an entire universe where billions are made) chips, and then give the designs to manufacturers - fabs (short for fabricators) - to make them. There are other companies other than the big 3, but nobody hears about them and they are all behind in the "technology node" employed in their transistors (basically the sophistication of the manufacturing process, whose most observable effect is how small you can make a transistor). And then there's ASML, the dutch company that is the *sole* supplier of extreme UV lithography machines that the big 3 companies use, it's the tip of the tip of the pyramid. The entire world depends on ***1*** company. If it goes down for any reason whatsoever it can send us back to the 1990s or early 2000s or so level of tech.
Food and Media seems to be deliberate consolidations and empire-building, while Semiconductors is perhaps more naturally rewarding of big corpos and punishing of small companies because of the insane expertise and the supply chains involved. Regardless of anything though :
1- What is the veracity of those infographs ? What "semantics games", if any, are being played by connecting 2 companies with a line to say that one of them controls the others ? Are they playing fast and loose with the meaning of "control" ?
2- If there is no catch in (1), then aren't conspiracy theorists (the "elites hate us and are out to get us" kind, not the "Obama's wife is an alien transgender" kind) much much more credible and justified than we often give them credits for ? Isn't the modern world exactly as over-consolidated and unified as they imagine ? Aren't the common rebuttals to them along the lines of "lol nobody can actually coordinate that much people and effort for that much time and money" shaky given (1) ?
3- Ignoring politics and who wants to control whom for a moment, isn't this a dangerous centralized weak spot for all of civilization ? In project management there is often this phrase, "Bus Factor", it's how many of your people can be hit by a bus (== an unforseen circumstance resulting in their death or neutralization) before your project grinds to a halt because all key people are in hospital. This factor is supposed to be big, if it's small then your project is fragile and dependent on a small number of people. Other terms are "Single Point of Failure" in tech, and the "K-selected" vs. "R-selected" dichotomy from biology, where animals either invest all their energy in raising a few offspring (with, hopefully, better chances of surviving), or make a huge number of offspring (which, while individually fragile, have more chances overall because of their sheer number).
Those companies make us a civilization with a low bus factor, a single (or a few, single-digit few) point of failure, an extreme K-selected civilization that puts all of our eggs into very few places. It would take so little to bring us all down to our knees, there is so much hidden centralization and so many invisible webs of control that can all crash at once.
A common criticism of hedge funds is that they buy a struggling company, load it with debt and lucrative payments to the hedge fund itself and win whatever happens to the company. This criticism seems to depend on naïve lenders who do not see the higher risk that the target firm's profitability.
What are the actual mechanics by which long-term unprofitable companies manage? I mean, don't they uh..... run out of money in their bank account eventually? For example Uber apparently loses several hundred million dollars per quarter, and billions in any given year. Isn't there a bank account that simply goes to 0 at some point, and they shut down as a going concern? How can a company loses a couple billion every year and not go bankrupt, isn't that the literal definition?
When a company's a startup they have x amount of 'runway', or cash in the bank that they can burn before they go under. The startup's either trying to hit profitability or hit the next funding round before they run out of cash. I understand that Uber can issue shares on the public market to raise more capital, but won't investors..... like stop setting money on fire at some point? I don't really get a company can lose money every single quarter and every single year indefinitely
1) Don't count airport layovers as "visits" to countries. This rule pertains even if you left the grounds of the airport to stay at a nearby hotel for the night.
2) Don't count visits to the areas around cruise ship terminals. For example, if you went on a cruise, your ship docked at Nassau, and you spent three hours shopping at the tourist stores within walking distance of the dock, don't count that as a visit to the Bahamas.
3) Don't count visits to countries that happened when you were so young that you can't remember them anymore. For example, if your parents took you on a trip to Canada from the U.S. when you were a baby.
I'm not sure if this is just me, but I can't get the manifund site to load right now. I just get "connection reset". It's possible this is related to me being temporarily on a terrible satellite internet connection, but other sites that fail usually do it in a different way than that.
Does anyone here know about USA police operations or know any sources of USA police data? My rough mental model of police time allocation is "80% of police time is spent on searching for, dispensing, and documenting speeding and parking tickets, with another 10% also low-value (harassing teens skateboarding, driving from one place to another, taking breaks, getting donuts and coffee, etc), and maybe 10% spent on actually solving real crimes like murder, rape, etc."
I realize this is both uncharitable and not based on any data other than what I personally have seen police officers doing, so I wanted to update my beliefs with new and better quality data. If anyone here knows a good data source, or could give a rough department-level outline of where most police time is allocated on a monthly or annual basis, I'd really appreciate it.
This has been a public service announcement on behalf of cheering everyone up (unless you're a United fan, sorry, we're all slogging away at the moment).
I’m really passionate about helping getting UNSONG published even in the case that Scott is no longer interested in the major edits mentioned in Editing Unsong (but maybe removing the rape scene would be a good idea). Is there anything bottlenecking this process outside of his control that we readers can help fix? If not, I promise to do nothing about this issue and forget about it promptly. [EDIT: rephrased for clarity]
I took ketamine for depression via Mindbloom. It worked really well and immediately. But I've been a little hypomanic since then. Not like, continually, but there are a lot more times when I notice myself talking a lot, I'm staying up late working and I don't always feel like I can stop, I'm less interested in food, I feel more confident than usual. Like, up till midnight on a Saturday working and I have to force myself to eat. I've been on amphetamines for ADHD before and it's a little like that - not all the time, but sometimes.
I tried to research whether ketamine could cause mania and it seemed like there may be were a couple of cases, but obviously they were much worse. Is it possible that ketamine did actually cause this? It's been going on for a couple of months now, and I've been taking ketamine doses every couple of weeks during that, but I think it's getting worse (or better, I guess, if you like my symptoms.) I can't think of anything else that's changed.
I wrote a LessWrong post about ecological dynamics, an approach to psychology and sports coaching that avoids talking about internal models and credences, which produces neat theories with interesting implications for learning.
I’m far from an expert so it’s probably not airtight, but I wanted to make the case that despite all appearances, ecological psychology is quite compatible with Lesswrong rationalism. Let me know if you spot any mistakes or inadequacies, or if you have any questions!
Some time ago our dear host wrote on semaglutide, the new wonder drug against obesity... and quite predictably, just a few months later, you hear reports that it's already become a "must" among the rich-and-beautiful crowd. Kind of sad, but not altogether unexpected. Link to article: https://www.thecut.com/article/weight-loss-ozempic.html
I’ve been thinking about the idea of treating large language models, such as the one driving ChatGPT, as dynamical systems. That’s why I chose the term “story trajectory,” in my recent working paper, ChatGPT tells stories, and a note about reverse engineering: A Working Paper, https://www.academia.edu/97862447/ChatGPT_tells_stories_and_a_note_about_reverse_engineering_A_Working_Paper I find it natural to think in terms of a dynamical system evolving along a trajectory in state space. There is a considerable literature on the brain as a dynamical system and I had a good bit of interaction with the late Walter Freeman on that subject, which he pioneered.
In his long article, “What Is ChatGPT Doing … and Why Does It Work?”, Stephen Wolfram talks of attractors and attractor basins near the end. Those are terms used in complex dynamics, https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ But he puts the terms in quotes. I assume he’s doing that because he is using the terms metaphorically. He’s not actually asserting that ChatGPT is a dynamical system, though he might like to.
I can’t follow the mathematics in detail, but I do find the idea attractive. Think of a prompt as imposing an initial state on the system. The system then evolves step by step, emitting a token at each step. That’s a much more plausible way of thinking about what’s going on than simply saying it ‘predicts’ the next token, one after another. Yes, that’s what we observe. But that way of thinking about it, 1) focuses our attention on those tokens, and 2) makes their appearance seem deeply mysterious. In contrast, thinking about the system evolving (step by step), that puts your attention on those 175 B parameters. It doesn’t tell you what they are doing, but that’s where your attention is. Incidentally, as the system evolves, it emits token after token after token, etc.
So, when in the paper I talk about “a nested hierarchy of probability distributions,” I’m talking about how the whole system evolves. Just how it does that, I don’t know.
When I think of a story trajectory, I’m imagining that, when ChatGPT begins telling a story, it enters a “valley” in the “attractor landscape” that is evoked by the prompt. It remains in that valley until the story is completed. My generate-a-new-story task is a way of exploring that valley in the attractor landscape. And when ChatGPT refused to tell a story where the protagonist was a colorless green idea, it was, in effect, saying THAT’s not in the valley.
There’s only so much you can do without mathematics. But the language and imagery is more attractive and useful.
@Anyone sympathetic to the idea that phenomenal consciousness doesn't exist: can you elaborate on your model? Why do you think it, what does it even mean (what is consciousness?) and optionally, why do you think other people disagree, why are they wrong, and what's the implications for ethics, if any?
([I asked the same question in last week's OT](https://astralcodexten.substack.com/p/open-thread-265/comment/13170540), and got more or less exactly what I was hoping for, but only from one data point; the rest were just people arguing with them. Surely there's other people sympathetic to the illusionist approach?)
There seems to be a pattern where activities/hobbies/workplaces dominated by one gender actively scares away the other gender. I'm curious as to where that threshold is, but my quest for numbers has left me with nothing but vapid culture war stuff.
My diving club's far more male dominated than the diving community as a whole, and I'd like to fix that. I think a good start would be to set a goal of aggressively reaching the critical mass of women needed to not alienate new or existing female members, followed by a less aggressive drive to reach the diving community's ratio. Any advice on figuring out what this critical mass is?
Does anyone else have problems with email notifications for new blog posts? They used to work just fine; a couple of weeks ago, they pretty much stopped working, with only a few exceptions getting through.
It’s pretty remarkable how effective shrooms are as an anti depressant for me. I’m prone to the navel gazing type of depression that seemingly afflicts a lot of young men today, and if left untreated it goes to an extremely dark place. But with just a mild dose of mushrooms- I’m talking just 1 gram- I am immediately snapped back into healthy brain functioning for at least 2-3 months before the depression starts to creep it’s way back in. The best comparison I can give is the feeling of taking a shower at the end of a day, and feeling all of the dirt and sweat you accumulated get washed away. I quite literally feel like my brain has been cleaned after a night on mushrooms, usually spent just relaxing listening to music or taking my dog for a walk. I’m not entirely sure what I even mean by that, and definitely don’t have a clue how it works, but holy shit it *works* remarkably well.
I think the effectiveness of psychedelics depends on the person, and I just so happen to have the perfect combination of personality traits that makes them maximally effective as medicine.
Historian Bret Deveraux had an interesting blog post a while back that led to an even more interesting discussion in the comments about whether a survivor community in a post-apocalyptic scenario would devolve to black powder weapons, or whether they'd be able to hang on to the ability to make smokeless powder and primer for regular guns. A chemist chimed in and said we could probably make primer (so you could have percussion caps), but making smokeless powder would be really hard to make in such a way that it kept for long periods of time.
Is the song "Informer" by Snow having some kind of belated revival? In the past week or so, I've been in two different restaurants to dine-in - and both times, that song came on the air over the restaurant radio. Of all the songs to get a revival, although it is quite catchy even if I can't possibly sing it.
I'm looking forward to Microsoft integrating its new AI into Excel. It'd be pretty neat to give it a prompt and have it organize a set of Excel data for me, including those annoying lookup functions and random design choices.
For most of the roughly 400,000 years of modern humans' history, our ancestors lived in Africa and were presumably black. Even the descendents of people who migrated out of Africa starting around 70,000 years ago remained pretty much black until around 10,000 years ago, if Cheddar man's complexion was typical [ https://en.wikipedia.org/wiki/Cheddar_Man ]
So why do people blush? A quick web search reveals that black people also blush, as they can feel the blood rush to their face, but most of them concede that others find it hard to detect. So as a social cue to a feeling of guilt or shame or general sheepishness in company, a blush seems rather pointless, and one wonders why it evolved before people started living in temperate climates and their skin became paler.
edit: Equally puzzling is that a flushed face can also signify anger. So as a social cue, there is ambiguity there between anger and embarrassment.
I have an idea for making these exchanges more satisfying: invisible likes. I fully agree that visible upvotes would push us in the direction of being more like high school, so are out of the question. On the other hand, I really do not think it is good for any of us to put up a good number of posts and hear absolutely nothing back.That’s what happens to me with more than half of my comments, & I’m guessing I’m about average in how many responses per comment I get. In real conversation if all your listeners (in this case several hundred people ) are dead silent after you speak, that means what you are saying is not going over well — it’s boring, incomprehensible or offensive. I know that no responses to a comment does not necessarily mean that here, but it’s really not possible for me to fully override the feeling that it does mean that. I also would find it helpful to know which comments several people found interesting, and which nobody did. I don’t think that would be like high school — nudging me in the direction of doing what’s popular. I think it would be giving me valid information of what kinds of ideas people here are finding worthwhile, which I can choose to ignore if I’m determined to say something that I know will annoy a bunch of people. And in all this, I really think I am reacting like most people. I’m not asking for accommodations for a quirk of mine, here, I’m asking for something —“invisible upvotes” — that I think would help engagement for most people.
OK, so here’s my idea for making that work: What if we brought back the “like” button, but had whatever indication Substack gives of a person’s upvotes appear in a light blue font, exact same color as the background of the page. Or if actually upvotes are displayed as hearts or fat little cupids, same thing — make the upvote icons light blue. That way, the commenter would receive and email notifications of upvotes, but readers here would not see the upvotes.
Scott, is that possible? Other people: waddya think?
What To Do After to re-balance microbiota. What NOT to do ?
PS . Any advice on how to minimize cramps in the day/ night after drinking the gallon of PEG ( poli ethylene glycol) mixed with the hyperosmolar salts solution ( Colyte, Peglyte variations?). I am having atrocious cramps every single time. No IBS .
Since the AI is born while being trained, isn't it reasonable to imagine an AI whose prime directive will simply be to continue to learn as much as possible, as fast as possible? I am not saying this AI won't kill us, but it sounds more plausible than paper clips.
If this is against the rules for open thread, I'm happy to remove this, but I am trying to do a bit of market research for a potential service that I think many folks here (content creators and content consumers) would be interested in.
To help me verify that hypothesis, I'd be very appreciative of anyone who could help with a quick (1-2 minutes, 5-6 question) survey. I'm particularly interested in your response if you've got a blog/podcast/etc, that you've been actively monetizing.
What is up with the Chinese surveillance balloon incident? Why did the government decide to make a big deal of it, when previous Chinese balloons that overflew the US under Trump were ignored? Why did Biden decide to shoot it down when it was already over the Atlantic Ocean? (I don't buy the explanation of "the debris could have hurt people on the ground". Surely most regions along the balloon's path were remote and sparsely populated?) Why did China send the balloon when it has sophisticated spy satellites, and why send it only a few days before a high-level meeting with Blinken?
1. You really should have consulted someone about the financial design for this. Potential returns are irrelevant. The potential returns on a lottery ticket are extremely high. What matters is expected returns. And since they're splitting the returns with anyone who jumps on the bet after them the expected returns are unknowable. This is why people buy in at odds or a fixed chunk of equity. These are conditions which people don't accept even for gambling, let alone a prediction market. And this is really only one of several problems.
I understand the value of prediction markets but the rationalist community's repeated failure to structure them in a functional way has decreased my confidence in the capacity of its talent pool to actually pull off the project.
I've got a couple of related questions about AI alignment, and am hoping someone here can point me to a good discussion of the topic.
A) In typical descriptions of fast takeoff scenarios, and in stories like https://gwern.net/fiction/clippy, it seems like the machines get a free pass on alignment; Why should the first (N'th) AI capable of self-improvement imagine that it's N'th (M>N'th) generation descendent will share its terminal values, when it doesn't share its' own creators' values? Is the usual assumption:
1) The machine is as arrogant and reckless as we are?
2) Alignment is not much harder than creating a superintelligence - If you can do the latter, the former is left as an exercise?
3) Some aspect of fast-takeoff logic implies a smart AI will be happier with whatever a smarter AI does than what the original AI could do?
4) Something else I missed?
Related questions:
B) Is seeding training data with examples where "early AGIs suffer for taking off" valuable as a
countermeasure to unaligned fast takeoff? To be clear this is not an alignment strategy, merely a hedge. All the humans still die, but it is presented to the AI that this is a suicide pact rather than a calculated risk.
C) Assuming the answer to (B) is yes, is the original issue (A) I described valuable for this purpose?
Is anyone here subscribed to exponential view by azeem azhar? I’m wondering if the extra content/ slack channel/ meetup opportunities are worth paying monthly.
I have 3 more subscriptions to Razib Khan's Unsupervised Learning to give away. Reply with an email address, or email me at the one I specify here: https://entitledtoanopinion.wordpress.com/about/
I have been enjoying V.S. Naipaul's "India, a million mutinies now." One of the best writers I have ever read. This was a 1990 book. He calls it a travelogue. He talks to atleast 50 types of Indians (different religions, occupations, ages, geographic locations...), asks them probing questions.
I have tried to read it before around 1996. I needed to be more mature to really enjoy it, I suppose.
The CA public health department is starting a new unit to improve the quality of its decision making. As a part of this effort, the department is looking to hire people with non-traditional backgrounds, including economists, data scientists and statistical modelers. While the director is not a Bay Area rationalist, he's definitely rationalist adjacent. His twitter account @DrTomasAragon regular extols the writing of Julia Galef and others who emphasize a need for clear eyed and rational thinking.
During the covid crisis, there was a lot of criticism of public health decision making. Some common, not entirely unfair critiques are: PH decision making isn't always rigorous, it doesn't always account for relative risks and tradeoffs, it doesn't always appropriately account for uncertainty, it doesn't always account for behavioral reactions to policies. The public health bureaucracy is mostly staffed by doctors and disease specialists whose background may not include training specifically related to optimal decision making. This initiative is seeking to imbue the department with new expertise.
Read more about how the director thinks of this initiative here:
If you are interested in applying, there is a somewhat daunting sounding "exam" process. I would be happy to speak with anyone intimidated by this, which is a government staffing requirement, but not actually a big obstacle.
Can someone that understands the AGI doomerism perspective explain why recent innovations in AI have made people so concerned about superintelligent AGI in the near future?
Like, I don't understand how a super smart AGI trained on past and present data could somehow generate:
1) endless innovations
2) iteratively improve itself
It seems to me that superintelligent AGI would need to be fundamentally different from GPT-4 or PaLM-E, as the their knowledge is mostly constrained by their training data and RLHF. Superintelligent AGI, however, I'm guessing would need to be capable of generating new knowledge itself, but it does not look like we're anywhere close to this right now.
What am I missing here? Why should I be terrified?
The number I would like and don't have is how many wet markets there are in the world with whatever features, probably selling wild animals, make the Wuhan market a candidate for the origin of Covid. If it is the only one, then Covid appearing in Wuhan from it is no odder a coincidence than Covid appearing in the same city where the WIV was researching bat viruses. If it was one of fifty or a hundred (not necessarily all in China), then the application of Bayes' Theorem implies a posterior probability for the lab leak theory much higher than whatever the prior was.
Hey folks, I'm visiting the Bay Area soon for a conference, and I'm staying two weeks, so enough time to do non-conference things. My dates are the 17th of March to the 31st - are there any rationalist meetups I could attend?
If you'd rather not say them publicly, my email is my first name followed by my surname (both of which are in my username) followed by the numeral 1 followed by @gmail.com
I have been putting together an argument in favour of expecting AI to have a slow takeoff. Part 1 of the argument, though presented in a way that is seemingly unrelated to AI, can be seen here: https://www.youtube.com/watch?v=Zk6uHi_Rdm4
Is the UK government the most woke in the world?
Douglass Murray, “Can you really be radicalised by Great British Railway Journeys?” (via N.S. Lyons) (https://www.spectator.co.uk/article/can-you-really-be-radicalised-by-great-british-railway-journeys/?utm_source=substack&utm_medium=email)
The British government is among the most dedicated in the world when it comes to keeping its people safe… from thinking the wrong thoughts. Which is why it runs “Prevent,” a program to prevent terrorism by encouraging people to rat out their neighbors to counter-terrorism police for wrong-think, as well as using behavioral analysis to pre-identify dangerous extremists to monitor. Recently some documents from Prevent became public as part of an official review, and, as Douglass Murray reports here, the factors designated evidence of “extremism” are rather interesting…
"When I first saw these documents I felt a sort of white-hot anger. But then I read on and saw that these same taxpayer-funded fools provide lists of other books shared by people who have sympathies with the ‘far-right and Brexit’. Key signs that people have fallen into this abyss include watching the Kenneth Clark TV series Civilisation, The Thick of It and Great British Railway Journeys. I need to stress again that I am not making this up. This has all been done on your dime and mine in order to stop ‘extremism’ in these islands.
There is also a reading list of historical texts which produce red flags to RICU. These include Leviathan by Thomas Hobbes, John Locke’s Two Treatises of Government and Edmund Burke’s Reflections on the Revolution in France, as well as works by Thomas Carlyle and Adam Smith. Elsewhere RICU warns that radicalisation could occur from books by authors including C.S. Lewis, J.R.R. Tolkien, Aldous Huxley and Joseph Conrad. I kid you not, though it seems that all satire is dead, but the list of suspect books also includes 1984 by George Orwell..."
(Dark) Thought experiment
You are a 20 year old who was born to a single mom as the result of rape. You love your mother dearly. You live in a country where abortion on demand is legal, but she choose not to have you aborted. The traumatic effects of the rape are still with your mother 20 years later and are awful for her.
One day, you are transported back in time to the day your mother was raped, and have the opportunity to stop it.
Do you do so, eliminating yourself from existence in the process?
Um, due to, you know, events, I have an emergency update to my Russo-Ukrainian war forecast, posted without explanation of either methodology or reasons. Previous update (with methodology) here: https://astralcodexten.substack.com/p/open-thread-261/comment/12342095. Current probabilities:
13 % on Ukrainian victory (down from 17 %).
45 % on compromise (down from 47 %).
42 % on Ukrainian defeat (up from 36 %)
Here's my idea for a Monarchy/Oligarchy:
1) There is a Founding King and a Founding Queen.
2) They have a large number of children.
3) After the Founding King dies, all his powers pass to the Founding Queen.
4) Once she dies, an election is held to assign a replacement head of the Royal Family. Only people who are descended from the Founding King and Founding Queen, and who share at least 12.5% of their DNA can run for the office or hold office. The same requirement exists for all top political positions outside the monarchy, like Parliament, the Supreme Court, and heads of government agencies, including the military.
5) Anyone who shares at least 6.25% of their DNA with the Founding King and Founding Queen can vote in the election.
6) It's common and accepted for people sharing 6.25% or more of their DNA with the Founders to clone themselves once they get old. There is a social imperative to keep the voting/leadership population pure and continually expanding in size. There's also an imperative to use genetic engineering to make each generation smarter, healthier, and more competent than the last.
7) Large numbers of people who have less than 6.25% of the Founders' DNA also live in the country, and they have all the same rights except for voting for politicians and holding high public office.
I thought up this idea recently, and think it carries some advantages over existing systems. For one, since the Head of the Royal Family is elected, it reduces the odds of a truly bad leader getting the post, which has been the Achilles' Heel of monarchies based on birth order. Also, the blood quantum requirements would help ensure that the leadership caste was unified, preventing the society from getting too fractious. Add in a culture of genetic improvement and fecundity, and you eventually get a caste with thousands or even millions of highly competent members who could fill out the ranks of national leadership with lots of people to spare.
The US has trended away from religion since about 1990, with the percentage of the population identifying as Christian dropping from about 90% to 63% and the share of unaffiliated rising from about 9% to 29%: https://www.pewresearch.org/religion/2022/09/13/how-u-s-religious-composition-has-changed-in-recent-decades/.
However, from the perspective of progress of rationalistic thought, I think that these trends may underestimate the more meaningful shift. If someone is concerned with religion leading people to think and act in ways that are irrational, then it doesn't really matter which box someone ticks off on a survey - it matters how they actually think and act.
My impression is that in the 1990s, belief in the supernatural played a much larger role in people's actual behavior, than today - even more than the aforementioned trend might imply.
For example, I think it used to not be uncommon for police to consult with psychics to try to solve crimes (I can't find numbers on the frequency of such occurrences - there was a notable case of "the St. Louis Jane Doe" in 1994 when the police mailed a vital piece of evidence in a murder case to a psychic and never got it back). And psychics, like Sylvia Browne, in general seemed to have occupied a much more significant role in the 90s and aughts.
Horoscopes are still popular, but it seems like they have mostly been relegated to the realm of fun and games, rather than being consulted for serious things like solving murders.
Instead, is seems like even religious people increasingly act in ways that are consistent with naturalistic explanations of the world.
Conversely, there may be a trend of non-religious people increasingly adopting ways of thought and action that are less consistent with naturalism, and more consistent with dogmatism or religion. For example, David Friedman notes that a priori rejection of sex differences in human behavior is not what one would expect on the basis of belief in evolution, and would be more consistent with a divine religious model, but is probably more common among those who would not tick off a box like "Christian."
Still, it seems like the overall trend in actual meaningful behavior towards naturalism, although I can't think of the best metrics to study that.
For example, Church attendance would be an obvious proxy, but it's not really what I'm interested in. People can attend Church for a variety of reasons, and even nod along with everything not disingenuously, but nevertheless behave in ways that are consistent with naturalism. For example, if they had a sick child, they may pray for the child, but it would not occur to them to pray instead of seeking actual medical treatment. Praying doesn't have much of a cost, so it isn't a great proxy for measuring how deeply someone believes something. I think mailing off evidence in a murder case to a psychic, is a much better indicator of deeply believing in the supernatural - and more interesting for me, is the sort of concerning behavior that I would hope is eliminated.
Ok, so what's the current thinking on nicotine? If someone starts chewing nicotine gum, will it have a net health benefit or harm? And what if they just do it a couple times a week in order to avoid addiction? Or use it on non-coffee days (for people who don't want to build caffeine tolerance.)
It's hard to find research on this, and I fear that it's just politically incorrect to study possible benefit of nicotine usage.
Last week here, I linked to a survey soliciting opinions on world peace. Results of that survey can be found here https://medium.com/@mttpgn/world-peace-survey-results-2023-14345513b48d
Scott, you might be interested in this article on the link between the brain and illness/recovery.
https://www.nature.com/articles/d41586-023-00509-z
How much will the US be on the hook for once the Ukraine war is over? I'm already seeing talk of something like a marshall plan for Ukraine (assuming that Russia loses, which again, is assumed only until it is time to demand more weapon shipments to ukraine), with estimates that it will take half a trillion dollars to fix ukraine already (and a lot more by the time this the war is over).
Obviously this won't all come from the US, but it's hard to imagine we won't be footing the lion's share of the bill in some form or other.
It's bad enough that Zelesnkyy and his western cronies would rather another hundred thousand young ukrainian men be killed or disabled to avoid Zelesnkyy losing a single inch of clay under his control, but by refusing to even entertain the notion of settling for anything but an INCREASE in territory comapred with the start of the war, he's basically saying that Americans will have to pay hundreds of billions of dollars for some shithole regions that were generating a tiny fraction of this money in GDP and tax revenue before the war even started.
Does anyone have recommendations for a psychiatrist or therapist in the Pittsburgh area?
The trolley problem is often seen as some abstract thing with little practical relevance. It is the complete opposite in fact: it is a description of our present situation.
The Trolley Problem Is All Too Real
https://squarecircle.substack.com/p/the-trolley-problem-is-all-too-real
You can actually pull the lever today, and without killing anyone! It will just cost you a bit of money.
I recently posted a comment on Freddie deBoer's Substack to the effect of "I'm confident courts will not allow government censorship to stand, so I'm much more concerned about social pressure (to censor speech) and think it's is a much bigger problem, by (making up numbers here) several orders of magnitude." Evidently that's not a popular view there (peer and descendent comments were voted more highly, including ones quite sympathetic to that position). Upvotes are hardly a dispositive measure of things, but it has me wondering about my intuition here.
Are ACX readers from the US more worried about government or private censorship?
As one of the investors who isn't participating in Manifund: The potential returns aren't even a factor in my not participating. What is a concern is:
* Academic grant funding is very different than investing in corporations, and this falls in the uncanny valley between them (in a bad way).
* The myopic nature (experimenting with prediction markets by funding experiments on prediction markets) is kind of a turn-off.
* There is a rule of thumb in enterprise sales that you can't price a product between $1000 and $10000 - it's not cheap-enough to be an impulse sale and it's not expensive enough for a salesperson to be interested. There may be a similar dynamic in play here.
Does the Peter Principle hold on an organisational level?
That is to say, an organisation (that has a reasonable level of autonomy as regards the scope of its activities) will respond to getting good at something by doing more difficult things, which it won't be good at yet*.
So everybody is winging it but the competant ones will be winging progressively greater things.
*EDIT: forget to put a Growth Mindset! joke here
Last week, I posted a piece I wrote arguing that helicopters should be taxed based on their noise externalities. I felt there was a subtext to a lot of the responses I got along the lines of “oh look, another big government progressive who wants to regulate everything.”
This was strange to me, since I sometimes conceptualize myself as a libertarian. Then I went back and looked at my writing and discovered that indeed, it is quite statist. I think this is a byproduct of basically just attempting to write a lot, and I think it suggests something about why libertarian intellectuals are rare in the mainstream media, so I wrote about that (https://omnibudsman.substack.com/p/why-libertarian-public-intellectuals).
Everyone seems to agree that the current situation with the appointment of US supreme justices[1] is less than ideal, where the party in power of the senate can just block any appointee, and then install a partisan when in control of both branches. The loudest responses seem to be "Yeah, [party] should just pack the courts" (self-evidently undemocratic), "something something term limits" (probably unconstitutional, getting amendments for this is currently infeasible), "each president should just appoint one" (either has the same problems as above or just leads to all partisan justices).
Here's an attempted proposal to hopefully get incentives in the right place without relying too much on "norms", that I'd like feedback from the crowd to tell me in what ways I'm being an idiot an it can be improved:
- Every president gets one justice appointment per term, and the total number of justices is left mostly uncapped (If it becomes *too* unwieldy, a soft cap at 1/4 the size of senate, currently 25)
- The Senate *must* vote on every nominee on a "reasonable" timeline, or they are considered to be providing implicit consent[2] for the nominee. "No" votes should include guidance on what they would consider a more reasonable candidate.
- If they can't come to an agreement with the Senate: at any time during the last year of their presidency, up to a week (or more?) before the next election, the President may set a "fallback" judge, for whom a "no" vote on another candidate is considered to be *implicit consent* by the senate for that candidate being immediately appointed by *the next* (term's) president at any point during that president's term.
Ideally, this incentivizes the President and the Senate to come to a compromise early in his term, so they avoid a partisan fallback judge and he gets an early favorable justice.
Of course, this implies that any two-term president gets essentially a free partisan fallback justice, which should be fine given
(a) every president gets one, and the ping-pong of the parties should balance this out
(b) who that nominee would be becomes an explicit campaign point, so they would have the mandate of democracy
Thought, criticisms, concerns?
[1] (and less-than-supreme federal/circuit justices, but one issue at a time)
[2] In the language of the constitution: "[The President] shall nominate, and by and with the Advice and Consent of the Senate, shall appoint [...] Judges of the supreme Court" https://constitution.congress.gov/browse/essay/artII-S2-C2-3-5/ALDE_00013096/
I wrote a little bit on lead exposure and how successful lead abatement efforts have made the estimated effects of lead extreme overestimates.
https://cremieux.substack.com/p/who-gets-exposed-to-lead
TL;DR: If average lead levels drop, but gaps in IQ, crime, illness, etc. in terms of antecedents of lead exposure remain somewhat constant, the effect size of a unit of lead necessarily increases. For IQ, in the last 50 years, the effect of one μg/dL of lead multiplied by more than 100 times.
Are there some LGBT support groups in Russia that I can join?
I have often seen infographs with the message "everything in this particular field or walk of life is controlled by a single-digit number of corporations". Here are 2 examples I got by just typing "everything is controlled by a few companies" into google image search :
1- Food, 9 corporations : https://www.frugl.com.au/wp-content/uploads/2020/01/57ebc2d7077dcc0f208b7830-800x400.png
2- by appending "media" to the search terms, 6 corporations : https://preview.redd.it/nb11lryl6ca01.png?auto=webp&s=eaffac604d0f62e03c21389e8891aade0eddb6ed
There is another somewhat-different example of a field I'm following with interest (though I'm still a layman), which is semiconductors and chip manufacturing. The seimconductor industry in general is ridicously over-consolidated, only 3 corpos - Intel, Samsung, and TSMC - are cutting-edge manufacturers, every other chip company you have heard of - Apple, Arm, AMD - are "fabless", they don't have factories, they just design (which, to be completely clear, is an entire universe where billions are made) chips, and then give the designs to manufacturers - fabs (short for fabricators) - to make them. There are other companies other than the big 3, but nobody hears about them and they are all behind in the "technology node" employed in their transistors (basically the sophistication of the manufacturing process, whose most observable effect is how small you can make a transistor). And then there's ASML, the dutch company that is the *sole* supplier of extreme UV lithography machines that the big 3 companies use, it's the tip of the tip of the pyramid. The entire world depends on ***1*** company. If it goes down for any reason whatsoever it can send us back to the 1990s or early 2000s or so level of tech.
Food and Media seems to be deliberate consolidations and empire-building, while Semiconductors is perhaps more naturally rewarding of big corpos and punishing of small companies because of the insane expertise and the supply chains involved. Regardless of anything though :
1- What is the veracity of those infographs ? What "semantics games", if any, are being played by connecting 2 companies with a line to say that one of them controls the others ? Are they playing fast and loose with the meaning of "control" ?
2- If there is no catch in (1), then aren't conspiracy theorists (the "elites hate us and are out to get us" kind, not the "Obama's wife is an alien transgender" kind) much much more credible and justified than we often give them credits for ? Isn't the modern world exactly as over-consolidated and unified as they imagine ? Aren't the common rebuttals to them along the lines of "lol nobody can actually coordinate that much people and effort for that much time and money" shaky given (1) ?
3- Ignoring politics and who wants to control whom for a moment, isn't this a dangerous centralized weak spot for all of civilization ? In project management there is often this phrase, "Bus Factor", it's how many of your people can be hit by a bus (== an unforseen circumstance resulting in their death or neutralization) before your project grinds to a halt because all key people are in hospital. This factor is supposed to be big, if it's small then your project is fragile and dependent on a small number of people. Other terms are "Single Point of Failure" in tech, and the "K-selected" vs. "R-selected" dichotomy from biology, where animals either invest all their energy in raising a few offspring (with, hopefully, better chances of surviving), or make a huge number of offspring (which, while individually fragile, have more chances overall because of their sheer number).
Those companies make us a civilization with a low bus factor, a single (or a few, single-digit few) point of failure, an extreme K-selected civilization that puts all of our eggs into very few places. It would take so little to bring us all down to our knees, there is so much hidden centralization and so many invisible webs of control that can all crash at once.
A common criticism of hedge funds is that they buy a struggling company, load it with debt and lucrative payments to the hedge fund itself and win whatever happens to the company. This criticism seems to depend on naïve lenders who do not see the higher risk that the target firm's profitability.
What am I missing?
What are the actual mechanics by which long-term unprofitable companies manage? I mean, don't they uh..... run out of money in their bank account eventually? For example Uber apparently loses several hundred million dollars per quarter, and billions in any given year. Isn't there a bank account that simply goes to 0 at some point, and they shut down as a going concern? How can a company loses a couple billion every year and not go bankrupt, isn't that the literal definition?
When a company's a startup they have x amount of 'runway', or cash in the bank that they can burn before they go under. The startup's either trying to hit profitability or hit the next funding round before they run out of cash. I understand that Uber can issue shares on the public market to raise more capital, but won't investors..... like stop setting money on fire at some point? I don't really get a company can lose money every single quarter and every single year indefinitely
I had trouble signing in to Manifold to invest.
How many foreign countries have you visited?
Note:
1) Don't count airport layovers as "visits" to countries. This rule pertains even if you left the grounds of the airport to stay at a nearby hotel for the night.
2) Don't count visits to the areas around cruise ship terminals. For example, if you went on a cruise, your ship docked at Nassau, and you spent three hours shopping at the tourist stores within walking distance of the dock, don't count that as a visit to the Bahamas.
3) Don't count visits to countries that happened when you were so young that you can't remember them anymore. For example, if your parents took you on a trip to Canada from the U.S. when you were a baby.
Attention(1) -> Thoughts(2) -> Attitude(3) -> Intention(4) -> Words(5) -> Behavior(6) -> Habits(7) -> Character(8) [repeat] = Destiny(9)
(1) pre-thought
(2) willed <> passive
(3) general, channels effort
(4) specific, determines outcome
(5) writing, speech
(6) situational
(7) 2-3 week consistency
(8) annual momentum
(9) life vector, trajectory
I'm not sure if this is just me, but I can't get the manifund site to load right now. I just get "connection reset". It's possible this is related to me being temporarily on a terrible satellite internet connection, but other sites that fail usually do it in a different way than that.
I am interested in hearing about people’s experience with polygala and l-theamine
Does anyone here know about USA police operations or know any sources of USA police data? My rough mental model of police time allocation is "80% of police time is spent on searching for, dispensing, and documenting speeding and parking tickets, with another 10% also low-value (harassing teens skateboarding, driving from one place to another, taking breaks, getting donuts and coffee, etc), and maybe 10% spent on actually solving real crimes like murder, rape, etc."
I realize this is both uncharitable and not based on any data other than what I personally have seen police officers doing, so I wanted to update my beliefs with new and better quality data. If anyone here knows a good data source, or could give a rough department-level outline of where most police time is allocated on a monthly or annual basis, I'd really appreciate it.
Have we discussed how the topology of a AI's neural network resembles the Kabbalah tree of life? I feel like there's an ACX t-shirt to be made here.
http://flamingpear.com/rob/neural-network-tree-of-life.png
> So far we haven’t gotten any investors in the impact market
I emailed manifund a question before submitting my application as an investor. I haven't heard back from them. Perhaps others had the same problem.
And I didn't even watch the match, so I deserve to have missed out on it all:
https://www.skysports.com/watch/video/sports/football/12826928/liverpool-7-0-manchester-united-premier-league-highlights
This has been a public service announcement on behalf of cheering everyone up (unless you're a United fan, sorry, we're all slogging away at the moment).
I’m really passionate about helping getting UNSONG published even in the case that Scott is no longer interested in the major edits mentioned in Editing Unsong (but maybe removing the rape scene would be a good idea). Is there anything bottlenecking this process outside of his control that we readers can help fix? If not, I promise to do nothing about this issue and forget about it promptly. [EDIT: rephrased for clarity]
I took ketamine for depression via Mindbloom. It worked really well and immediately. But I've been a little hypomanic since then. Not like, continually, but there are a lot more times when I notice myself talking a lot, I'm staying up late working and I don't always feel like I can stop, I'm less interested in food, I feel more confident than usual. Like, up till midnight on a Saturday working and I have to force myself to eat. I've been on amphetamines for ADHD before and it's a little like that - not all the time, but sometimes.
I tried to research whether ketamine could cause mania and it seemed like there may be were a couple of cases, but obviously they were much worse. Is it possible that ketamine did actually cause this? It's been going on for a couple of months now, and I've been taking ketamine doses every couple of weeks during that, but I think it's getting worse (or better, I guess, if you like my symptoms.) I can't think of anything else that's changed.
I wrote a LessWrong post about ecological dynamics, an approach to psychology and sports coaching that avoids talking about internal models and credences, which produces neat theories with interesting implications for learning.
https://www.lesswrong.com/posts/Y4hN7SkTwnKPNCPx5/why-don-t-more-people-talk-about-ecological-psychology
I’m far from an expert so it’s probably not airtight, but I wanted to make the case that despite all appearances, ecological psychology is quite compatible with Lesswrong rationalism. Let me know if you spot any mistakes or inadequacies, or if you have any questions!
Some time ago our dear host wrote on semaglutide, the new wonder drug against obesity... and quite predictably, just a few months later, you hear reports that it's already become a "must" among the rich-and-beautiful crowd. Kind of sad, but not altogether unexpected. Link to article: https://www.thecut.com/article/weight-loss-ozempic.html
I’ve been thinking about the idea of treating large language models, such as the one driving ChatGPT, as dynamical systems. That’s why I chose the term “story trajectory,” in my recent working paper, ChatGPT tells stories, and a note about reverse engineering: A Working Paper, https://www.academia.edu/97862447/ChatGPT_tells_stories_and_a_note_about_reverse_engineering_A_Working_Paper I find it natural to think in terms of a dynamical system evolving along a trajectory in state space. There is a considerable literature on the brain as a dynamical system and I had a good bit of interaction with the late Walter Freeman on that subject, which he pioneered.
It seems that people are just now beginning to think in those terms for deep learning. I just found a 2020 dissertation by Eduardo Sánchez Karhunen entitled “Eduardo Sánchez Karhunen”, https://scholar.google.es/citations?view_op=view_citation&hl=en&user=RaPcpaAAAAAJ&citation_for_view=RaPcpaAAAAAJ:9yKSN-GCB0IC
In his long article, “What Is ChatGPT Doing … and Why Does It Work?”, Stephen Wolfram talks of attractors and attractor basins near the end. Those are terms used in complex dynamics, https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ But he puts the terms in quotes. I assume he’s doing that because he is using the terms metaphorically. He’s not actually asserting that ChatGPT is a dynamical system, though he might like to.
I can’t follow the mathematics in detail, but I do find the idea attractive. Think of a prompt as imposing an initial state on the system. The system then evolves step by step, emitting a token at each step. That’s a much more plausible way of thinking about what’s going on than simply saying it ‘predicts’ the next token, one after another. Yes, that’s what we observe. But that way of thinking about it, 1) focuses our attention on those tokens, and 2) makes their appearance seem deeply mysterious. In contrast, thinking about the system evolving (step by step), that puts your attention on those 175 B parameters. It doesn’t tell you what they are doing, but that’s where your attention is. Incidentally, as the system evolves, it emits token after token after token, etc.
So, when in the paper I talk about “a nested hierarchy of probability distributions,” I’m talking about how the whole system evolves. Just how it does that, I don’t know.
When I think of a story trajectory, I’m imagining that, when ChatGPT begins telling a story, it enters a “valley” in the “attractor landscape” that is evoked by the prompt. It remains in that valley until the story is completed. My generate-a-new-story task is a way of exploring that valley in the attractor landscape. And when ChatGPT refused to tell a story where the protagonist was a colorless green idea, it was, in effect, saying THAT’s not in the valley.
There’s only so much you can do without mathematics. But the language and imagery is more attractive and useful.
Is the marriage between violent men and an accelerating knowledge explosion sustainable?
If yes, please explain.
If no, then what?
@Anyone sympathetic to the idea that phenomenal consciousness doesn't exist: can you elaborate on your model? Why do you think it, what does it even mean (what is consciousness?) and optionally, why do you think other people disagree, why are they wrong, and what's the implications for ethics, if any?
([I asked the same question in last week's OT](https://astralcodexten.substack.com/p/open-thread-265/comment/13170540), and got more or less exactly what I was hoping for, but only from one data point; the rest were just people arguing with them. Surely there's other people sympathetic to the illusionist approach?)
There seems to be a pattern where activities/hobbies/workplaces dominated by one gender actively scares away the other gender. I'm curious as to where that threshold is, but my quest for numbers has left me with nothing but vapid culture war stuff.
My diving club's far more male dominated than the diving community as a whole, and I'd like to fix that. I think a good start would be to set a goal of aggressively reaching the critical mass of women needed to not alienate new or existing female members, followed by a less aggressive drive to reach the diving community's ratio. Any advice on figuring out what this critical mass is?
I wrote a post on using ChatGPT to publish a (very simple) web game, sharing in case it's interesting to people! https://betterprogramming.pub/how-i-used-chatgpt-to-create-a-game-1537f6ee54e3?sk=8f167eeef05cefbdfae25f5605461c53
Does anyone else have problems with email notifications for new blog posts? They used to work just fine; a couple of weeks ago, they pretty much stopped working, with only a few exceptions getting through.
It’s pretty remarkable how effective shrooms are as an anti depressant for me. I’m prone to the navel gazing type of depression that seemingly afflicts a lot of young men today, and if left untreated it goes to an extremely dark place. But with just a mild dose of mushrooms- I’m talking just 1 gram- I am immediately snapped back into healthy brain functioning for at least 2-3 months before the depression starts to creep it’s way back in. The best comparison I can give is the feeling of taking a shower at the end of a day, and feeling all of the dirt and sweat you accumulated get washed away. I quite literally feel like my brain has been cleaned after a night on mushrooms, usually spent just relaxing listening to music or taking my dog for a walk. I’m not entirely sure what I even mean by that, and definitely don’t have a clue how it works, but holy shit it *works* remarkably well.
I think the effectiveness of psychedelics depends on the person, and I just so happen to have the perfect combination of personality traits that makes them maximally effective as medicine.
Historian Bret Deveraux had an interesting blog post a while back that led to an even more interesting discussion in the comments about whether a survivor community in a post-apocalyptic scenario would devolve to black powder weapons, or whether they'd be able to hang on to the ability to make smokeless powder and primer for regular guns. A chemist chimed in and said we could probably make primer (so you could have percussion caps), but making smokeless powder would be really hard to make in such a way that it kept for long periods of time.
https://acoup.blog/2022/08/05/fireside-friday-august-5-2022/
Is the song "Informer" by Snow having some kind of belated revival? In the past week or so, I've been in two different restaurants to dine-in - and both times, that song came on the air over the restaurant radio. Of all the songs to get a revival, although it is quite catchy even if I can't possibly sing it.
I'm looking forward to Microsoft integrating its new AI into Excel. It'd be pretty neat to give it a prompt and have it organize a set of Excel data for me, including those annoying lookup functions and random design choices.
For most of the roughly 400,000 years of modern humans' history, our ancestors lived in Africa and were presumably black. Even the descendents of people who migrated out of Africa starting around 70,000 years ago remained pretty much black until around 10,000 years ago, if Cheddar man's complexion was typical [ https://en.wikipedia.org/wiki/Cheddar_Man ]
So why do people blush? A quick web search reveals that black people also blush, as they can feel the blood rush to their face, but most of them concede that others find it hard to detect. So as a social cue to a feeling of guilt or shame or general sheepishness in company, a blush seems rather pointless, and one wonders why it evolved before people started living in temperate climates and their skin became paler.
edit: Equally puzzling is that a flushed face can also signify anger. So as a social cue, there is ambiguity there between anger and embarrassment.
I have an idea for making these exchanges more satisfying: invisible likes. I fully agree that visible upvotes would push us in the direction of being more like high school, so are out of the question. On the other hand, I really do not think it is good for any of us to put up a good number of posts and hear absolutely nothing back.That’s what happens to me with more than half of my comments, & I’m guessing I’m about average in how many responses per comment I get. In real conversation if all your listeners (in this case several hundred people ) are dead silent after you speak, that means what you are saying is not going over well — it’s boring, incomprehensible or offensive. I know that no responses to a comment does not necessarily mean that here, but it’s really not possible for me to fully override the feeling that it does mean that. I also would find it helpful to know which comments several people found interesting, and which nobody did. I don’t think that would be like high school — nudging me in the direction of doing what’s popular. I think it would be giving me valid information of what kinds of ideas people here are finding worthwhile, which I can choose to ignore if I’m determined to say something that I know will annoy a bunch of people. And in all this, I really think I am reacting like most people. I’m not asking for accommodations for a quirk of mine, here, I’m asking for something —“invisible upvotes” — that I think would help engagement for most people.
OK, so here’s my idea for making that work: What if we brought back the “like” button, but had whatever indication Substack gives of a person’s upvotes appear in a light blue font, exact same color as the background of the page. Or if actually upvotes are displayed as hearts or fat little cupids, same thing — make the upvote icons light blue. That way, the commenter would receive and email notifications of upvotes, but readers here would not see the upvotes.
Scott, is that possible? Other people: waddya think?
COLONOSCOPY Tuesday. PLS any advice?
What To Do After to re-balance microbiota. What NOT to do ?
PS . Any advice on how to minimize cramps in the day/ night after drinking the gallon of PEG ( poli ethylene glycol) mixed with the hyperosmolar salts solution ( Colyte, Peglyte variations?). I am having atrocious cramps every single time. No IBS .
Thank you
I'd love to hear from anyone who can imagine actually using the small-group futarchy app I've proposed building: https://manifund.org/projects/implement-a-prognootling-app
Since the AI is born while being trained, isn't it reasonable to imagine an AI whose prime directive will simply be to continue to learn as much as possible, as fast as possible? I am not saying this AI won't kill us, but it sounds more plausible than paper clips.
If this is against the rules for open thread, I'm happy to remove this, but I am trying to do a bit of market research for a potential service that I think many folks here (content creators and content consumers) would be interested in.
To help me verify that hypothesis, I'd be very appreciative of anyone who could help with a quick (1-2 minutes, 5-6 question) survey. I'm particularly interested in your response if you've got a blog/podcast/etc, that you've been actively monetizing.
https://forms.gle/wvUiqmw8FyncSM5E7
Thank you all!
What is up with the Chinese surveillance balloon incident? Why did the government decide to make a big deal of it, when previous Chinese balloons that overflew the US under Trump were ignored? Why did Biden decide to shoot it down when it was already over the Atlantic Ocean? (I don't buy the explanation of "the debris could have hurt people on the ground". Surely most regions along the balloon's path were remote and sparsely populated?) Why did China send the balloon when it has sophisticated spy satellites, and why send it only a few days before a high-level meeting with Blinken?
1. You really should have consulted someone about the financial design for this. Potential returns are irrelevant. The potential returns on a lottery ticket are extremely high. What matters is expected returns. And since they're splitting the returns with anyone who jumps on the bet after them the expected returns are unknowable. This is why people buy in at odds or a fixed chunk of equity. These are conditions which people don't accept even for gambling, let alone a prediction market. And this is really only one of several problems.
I understand the value of prediction markets but the rationalist community's repeated failure to structure them in a functional way has decreased my confidence in the capacity of its talent pool to actually pull off the project.
I've got a couple of related questions about AI alignment, and am hoping someone here can point me to a good discussion of the topic.
A) In typical descriptions of fast takeoff scenarios, and in stories like https://gwern.net/fiction/clippy, it seems like the machines get a free pass on alignment; Why should the first (N'th) AI capable of self-improvement imagine that it's N'th (M>N'th) generation descendent will share its terminal values, when it doesn't share its' own creators' values? Is the usual assumption:
1) The machine is as arrogant and reckless as we are?
2) Alignment is not much harder than creating a superintelligence - If you can do the latter, the former is left as an exercise?
3) Some aspect of fast-takeoff logic implies a smart AI will be happier with whatever a smarter AI does than what the original AI could do?
4) Something else I missed?
Related questions:
B) Is seeding training data with examples where "early AGIs suffer for taking off" valuable as a
countermeasure to unaligned fast takeoff? To be clear this is not an alignment strategy, merely a hedge. All the humans still die, but it is presented to the AI that this is a suicide pact rather than a calculated risk.
C) Assuming the answer to (B) is yes, is the original issue (A) I described valuable for this purpose?
Is anyone here subscribed to exponential view by azeem azhar? I’m wondering if the extra content/ slack channel/ meetup opportunities are worth paying monthly.
I have 3 more subscriptions to Razib Khan's Unsupervised Learning to give away. Reply with an email address, or email me at the one I specify here: https://entitledtoanopinion.wordpress.com/about/
I have been enjoying V.S. Naipaul's "India, a million mutinies now." One of the best writers I have ever read. This was a 1990 book. He calls it a travelogue. He talks to atleast 50 types of Indians (different religions, occupations, ages, geographic locations...), asks them probing questions.
I have tried to read it before around 1996. I needed to be more mature to really enjoy it, I suppose.
Deeply insightful.
The CA public health department is starting a new unit to improve the quality of its decision making. As a part of this effort, the department is looking to hire people with non-traditional backgrounds, including economists, data scientists and statistical modelers. While the director is not a Bay Area rationalist, he's definitely rationalist adjacent. His twitter account @DrTomasAragon regular extols the writing of Julia Galef and others who emphasize a need for clear eyed and rational thinking.
During the covid crisis, there was a lot of criticism of public health decision making. Some common, not entirely unfair critiques are: PH decision making isn't always rigorous, it doesn't always account for relative risks and tradeoffs, it doesn't always appropriately account for uncertainty, it doesn't always account for behavioral reactions to policies. The public health bureaucracy is mostly staffed by doctors and disease specialists whose background may not include training specifically related to optimal decision making. This initiative is seeking to imbue the department with new expertise.
Read more about how the director thinks of this initiative here:
https://github.com/DrTomasAragon/cdph/
If you are interested in applying, there is a somewhat daunting sounding "exam" process. I would be happy to speak with anyone intimidated by this, which is a government staffing requirement, but not actually a big obstacle.