30 Comments

How exciting to have a highlighted comment!! I’m metaphorically jumping up and down with glee!

Now I have to write an article about my experience with the California DV program.

Expand full comment

Great comments! I made it to "We still enjoy expected rewards; we just don't much *care* about our enjoyment of them", before looking down at the granola I was mindlessly and unnecessarily munching on, and sealing the top.

Expand full comment

The Bouarab, Thompson, and Poulter paper (BTP hereafter) is certainly interesting, but I think the claim, from Eshel, et. al., 2015, that VTA GABA interneurons are reward predictors ought to be taken with a pretty big grain of salt, as nearly every brain area involved in reward learning has been touted as *the* site of RPE computation in recent years (including, for what it's worth, PFC and pedunculopontine nucleus, both drivers of phasic DA neuron activity independent of GABAergic dis/inhibition). The lack of anything close to consensus as to where and how "expected rewards" are represented and how deviation from them is computed is perhaps cause for healthy skepticism about the RPE model being the best way to think about what the reward system does.

To better explain my rationale for mapping the saliency and hedonic aspects of reward onto excitatory VTA afferents and inhibition of inhibitory VTA afferents, respectively, I'll need to complicate my previous simple sketch a little bit. Tonic and phasic dopamine activity leads to different patterns of dopamine release; the former to non-selective, extrasynaptic release and the latter to release primarily at synapses.

This paper offers a nice illustration: http://behavioralhealth2000.com/wp-content/uploads/2017/11/Afferent-modulation-of-dopamine-neuron-firing-differentially-regulates-tonic-and-phasic-dopamine-transmission.pdf. Researchers independently manipulated excitatory inputs from pedunculopontine nucleus and inhibitory inputs from ventral pallidum to VTA DA neurons and observed the effects on dopamine release in NAc. Initially, it seemed like inhibiting VP (i.e., lifting the GABAergic brake) had a greater effect on dopamine release (seemingly consistent with Eshel and BTP), but the only way to measure dopamine release here was extrasynaptically.

Because dopamine released in synapses is voraciously gobbled up by the dopamine transporter, it won't be picked up by extrasynaptic measurements. So they inhibited DAT and performed the same interventions, and this time they found that the phasic bursting (driven by PPN activation) increased NAc dopamine levels much more than VP inhibition. And this dopamine release, being primarily synaptic, is much more targeted, and so likely has much greater effects on the strengthening and weakening of connections between NAc and its afferents, which to me suggests that the excitatory "surprise" or saliency input is more important for reward learning than the lifting of the GABAergic brake. In vivo, the latter mechanism seems to be heavily mediated by endogenous opioid signaling, and I think this explains why we can like (i.e., get pleasure from) something without necessarily being positively motivated by it (e.g., when it is expected; when the surprise signal is absent and there is only tonic firing of VTA DA neurons, leading to diffuse, extrasynaptic accumbral DA release).

Of course, the truth is undoubtedly more complicated still. Perhaps hedonic and saliency information is present in both excitatory and inhibitory inputs of VTA DA neuron activity, just in different ways. Microscopic reality is notoriously flippant about our pre-theoretical categories. I also think it very likely there is more than one GABAergic brake. We know different populations of VTA DA neurons project to different regions of NAc and likely contribute differentially (by targeting different DA receptor subtypes) to appetitive and aversive conditioning. So there may brakes on some DA neurons that are lifted (say, by beta-endorphins or enkephalins) by pleasant stimuli and brakes on others that are lifted (say, by dynorphins) by unpleasant stimuli. Time, grant money, and human ingenuity will tell.

Expand full comment

*Well.* I can see when I'm not wanted! It was the logorrhea, wasn't it? I knew the amphetamine would betray me. [slams door and starts car violently, sobbing; or, maybe, violently sobbing]

Expand full comment

#2 I have a lot of experience with stimulants... maybe that’s why I don’t recognize stages 3 and 4. I guess I see what they’re saying...

“Dopaminergics also distort the geometry expressed in the motor-planning space: it tenses and adds curvature to it so that there is generically more coordinated action. But it also, as a consequence, might restrict the diversity of action-space.”

Um, what? This is either willfully obtuse pomo-style writing, or jargon that I have somehow never encountered on topic that I feel I should know about. Maybe a mix of both. I’m kind of both annoyed and intrigued.

Expand full comment

> But once the brain learns that a reward is reliably predicted by a cue, the reward ceases to elicit a surprise signal. This means it no longer increases VTA DA neuron firing rate. It may still cause endorphin release and thus keep the GABAergic brake off, but if there's no surprise signal driving phasic firing, dopamine release will be minimal.

... or, in the immortal words of the Notorius B.I.G.,

"Number four, I know you heard this before: / Never get high on your own surprise."

Expand full comment

I'm glad a posted citation made "you feel less bad."

How about this one:

https://www.biorxiv.org/content/10.1101/2021.01.16.426967v2.full.pdf

1. Not directly related but in the vein of exploring "reward seeking" and "punishment avoidance": when does punishment avoidance become a reward to be sought and when does failure to obtain reward become a punishment to be avoided.

2. I will confess that I tend to think neuroscience has become neo-Skinnerian behaviorism as if describing and finding all the levers and nobs will validate behaviorism. And I'm with Chomsky and many others on Skinner.

3. I wonder if trying to understand what's actually happening in a black box neural network ostensible AI starts looking like neuroscience?

Expand full comment

> Being poor is a state where you are open to unpleasant surprises. If something breaks, it might mean hours spent trying to repair it, or months spent without a necessity, whereas a rich person would just shrug and get a new one.

Strong agreement - IME, the most effective way to put money to use in one's life is first to eliminate active sources of dissatisfaction (that can be addressed by money, ofc), *then* as a distinct change in strategy start looking for new positive experiences. The first stage is an exercise in triage and has strong returns that are nonetheless characterized by what *didn't* happen, the second is speculative and less efficient but easy to point to.

Expand full comment

Unpleasant surprises are the worst. Even if it's a fairly minor unpleasant surprise - for example, if I drop and break something made of glass and now have to clean up the mess - it can put me in a bad mood for several hours and basically ruin my whole day. :/

Expand full comment
Sep 30, 2022·edited Sep 30, 2022

"Maybe a side of “be careful before moving across the country to be with this person, or getting married”, but I feel like this aspect is well-known and common sense."

You would think, and yet! So many people muck up their lives because they are blinded by infatuation and won't listen to family, friends, or advice about "take it slowly, this might not last, and that is not a good idea".

Then one fine day the pink fuzzy cloud finally dissipates and they look around and go "how the hell did I end up here with this jerk?" Partly because of all the emphasis our culture puts on romantic love as the be-all and end-all of life, and all the encouragement around "try it or you'll regret it for the rest of your life, this is your one chance, seize it!"

"I feel worse when it happens to polyamorous people who were trying to follow the rules and do things right."

'Scuse me while I wipe this smirk off my face. Yeah, imagine that: breaking rules can go wrong! This is why us mean ol' social/religious conservatives have rules in the first place. Because human nature and human appetites are powerful, and human reason is not seated as unshakeably on the throne as you might wish. Kipling talked about the gods of the copybook headings, but the basic principle is the same: breaking the rules won't set you free from all the drama and strife and trouble and things going wrong. The rules are there to be a guide and a support: when all the tingles are pulling you towards 'dump everyone else in my life and move halfway across the country with this person I only met yesterday', the dull boring restrictive old rules are 'don't do that because you shouldn't do that; you must fulfil your responsibilities; just because it feels good is no reason to do it'.

To quote Lewis and Tolkien:

(1) Lewis, from "The Four Loves"

"Everyone knows that it is useless to try to separate lovers by proving to them that their marriage will be an unhappy one. This is not only because they will disbelieve you. They usually will, no doubt. But even if they believed, they would not be dissuaded. For it is the very mark of Eros that when he is in us we had rather share unhappiness with the Beloved than be happy on any other terms. Even if the two lovers are mature and experienced people who know that broken hearts heal in the end and can clearly foresee that, if they once steeled themselves to go through the present agony of parting, they would almost certainly be happier ten years hence than marriage is at all likely to make them - even then, they would not part.

…But these lapses will not destroy a marriage between two "decent and sensible" people. The couple whose marriage will certainly be endangered by them, and possibly ruined, are those who have idolised Eros. They thought he had the power and truthfulness of a god. They expected that mere feeling would do for them, and permanently, all that was necessary. When this expectation is disappointed they throw the blame on Eros or, more usually, on their partners. In reality, however, Eros, having made his gigantic promise and shown you in glimpses what its performance would be like, has "done his stuff." He, like a godparent, makes the vows; it is we who must keep them. It is we who must labour to bring our daily life into even closer accordance with what the glimpses have revealed. We must do the works of Eros when Eros is not present. "

(2) "The Screwtape Letters"

"We have done this through the poets and novelists by persuading the humans that a curious, and usually short-lived, experience which they call “being in love” is the only respectable ground for marriage; that marriage can, and ought to, render this excitement permanent; and that a marriage which does not do so is no longer binding. This idea is our parody of an idea that came from the Enemy.

…From the true statement that this transcendental relation was intended to produce, and, if obediently entered into, too often will produce, affection and the family, humans can be made to infer the false belief that the blend of affection, fear, and desire which they call “being in love” is the only thing that makes marriage either happy or holy. The error is easy to produce because “being in love” does very often, in Western Europe, precede marriages which are made in obedience to the Enemy’s designs, that is, with the intention of fidelity, fertility and good will; just as religious emotion very often, but not always, attends conversion. In other words, the humans are to be encouraged to regard as the basis for marriage a highly-coloured and distorted version of something the Enemy really promises as its result. Two advantages follow. In the first place, humans who have not the gift of continence can be deterred from seeking marriage as a solution because they do not find themselves “in love”, and, thanks to us, the idea of marrying with any other motive seems to them low and cynical. Yes, they think that. They regard the intention of loyalty to a partnership for mutual help, for the preservation of chastity, and for the transmission of life, as something lower than a storm of emotion. (Don’t neglect to make your man think the marriage-service very offensive.) In the second place any sexual infatuation whatever, so long as it intends marriage, will be regarded as “love”, and “love” will be held to excuse a man from all the guilt, and to protect him from all the consequences, of marrying a heathen, a fool, or a wanton.

…If, on the other hand, he is an emotional, gullible man, feed him on minor poets and fifth-rate novelists of the old school until you have made him believe that “Love” is both irresistible and somehow intrinsically meritorious. This belief is not much help, I grant you, in producing casual unchastity; but it is an incomparable recipe for prolonged, “noble”, romantic, tragic adulteries, ending, if all goes well, in murders and suicides.

(3) Tolkien, from a letter to one of his sons:

"There is in our Western culture the romantic chivalric tradition still strong, though as a product of Christendom (yet by no means the same as Christian ethics) the times are inimical to it. It idealizes 'love' — and as far as it goes can be very good, since it takes in far more than physical pleasure, and enjoins if not purity, at least fidelity, and so self-denial, 'service', courtesy, honour, and courage. Its weakness is, of course, that it began as an artificial courtly game, a way of enjoying love for its own sake without reference to (and indeed contrary to) matrimony. Its centre was not God, but imaginary Deities, Love and the Lady. It still tends to make the Lady a kind of guiding star or divinity – of the old-fashioned 'his divinity' = the woman he loves – the object or reason of noble conduct. This is, of course, false and at best make-believe. The woman is another fallen human-being with a soul in peril. …Then it produces what I suppose is still felt, among those who retain even vestigiary Christianity, to be the highest ideal of love between man and woman. Yet I still think it has dangers. It is not wholly true, and it is not perfectly 'theocentric'. It takes, or at any rate has in the past taken, the young man's eye off women as they are, as companions in shipwreck not guiding stars. (One result is for observation of the actual to make the young man turn cynical.) To forget their desires, needs and temptations. It inculcates exaggerated notions of 'true love', as a fire from without, a permanent exaltation, unrelated to age, childbearing, and plain life, and unrelated to will and purpose. (One result of that is to make young folk look for a 'love' that will keep them always nice and warm in a cold world, without any effort of theirs; and the incurably romantic go on looking even in the squalor of the divorce courts)."

Expand full comment

"This is why I can’t entirely get behind critiques of 'scientism'. Not every deep philosophical riddle has a simple scientific answer - but when one does, it’s pretty great. ... Let me see if I understand this well enough to summarize: ... But this still confuses me a little: naively I would have expected the opposite ... I looked into a paper on the subject recommended by commenter JDK. ... The paper seems to be arguing something different, ... Is the paper making the case for the opposite of Grognoscente’s comment, ...? I’m not sure."

Wait, so what's the "simple" scientific answer, and how the heck does this in any way add up to an affirmation of scientism?

Expand full comment
Oct 1, 2022·edited Oct 1, 2022

I just read the linked LW article about the Sinclair method. Remember the Shangri-la diet? I did it for a while and I'm reasonably sure that's how it worked. The taste part of eating kicks off all kinds of rewards and compensations in the brain, the brain associates these with calories, and you get into a cycle where your brain wants those phenomena. If you eat calories without taste that at least temporarily breaks the association between taste and calories.

There's no need for a set point theory or anything like it. There's probably a way to test this?

Expand full comment

Re: #5 "And yes, because those patterns are so difficult to dislodge, he is more or less doomed to a miserable life. Although, like you, I never say stuff like that to them."

--Highlights From The Comments On Unpredictable Reward

I applaud your work with these men in that you showed them the score. The reasons why and the difficulty in trying to change. My add would be that human life is defined by struggle which can build character at whatever starting point we are placed. 'Life is Suffering' after all.

-In support of my point:

"The serious problems of life, however, are never fully solved. If it should for once appear that they are, this is the sign that something has been lost. The meaning and design of a problem seem not to lie in its solution, but in our working at it incessantly. This alone preserves us from stultification and petrifaction."

--Carl Jung, Modern Man in Search of a Soul, 1933

Expand full comment