Extremely cool. One possible way to salvage the original result is if there is a cumulative threshold effect: maybe you need to be in a low-carbon environment for a long time to see effects and if you regularly breathe fresh air your mental function is not affected. This wouldn't explain the submarine data point though
I recently (June) started monitoring AQ in my condo after reading similar studies, watching a presentation by DHH from Basecamp (MRqh8oLY7Ik) and trying to find ways to help my wife's migraines. I'm using the Awair AQM8002A.
Our overnight CO2 in our bedroom (shared with 3 children) went up as high as 1300ppm and rarely went under 900ppm any time during the day. I tried opening windows and I could push it down to 700 but as soon as I closed the window (since it's July in Houston) it shot right back up. The humidity however took much longer to recover.
We left for a month and it went down to the 400s (set the temperature to 78 instead of 71).
I just used Condensate Pan Treatment Tablets (Uric acid) two days ago and our overnight reading have gone down to high 700s (from 1300ppm). I've had a lot of trouble figuring out how such a large change in CO2 could have been caused by micro-organisms in the condensate pan... My first thought is faulty meter, which I haven't tested.
No way to know the effect on migraines as it's a long term occasional problem. My personal waking fatigue is much improved though.
Is urban dictionary a valid source of words for the game? Narc…
Could there be factors other than CO2 that correspond to ventilation? Air flow, humidity, level of VOC, sound levels, etc.
I sleep much better with a fan. A study might show that people sleep better with fans running. Headline, “Studies show high air flow helps sleep.” Later we find out it’s the white noise that’s helping.
My anecdotal evidence is that I feel better and more alert when I ventilate the car on long drives. But in car CO2 levels be much higher than a house or apartment. I can’t think of a good way to test this.
I had wanted to measure this exact same thing - ever since I moved into my current flat I feel that I can work so much better with the windows open. And the price did stop me - I thought these monitors were around $10 when I got the idea (why do they cost so much?). So... if logistics of me being in the UK is not an issue, I'd be happy to test?
If you're actually willing to buy the monitor for someone(s), I suspect you'll find plenty of video game streamers who would be willing to do this as a gimmick on their stream for some weeks, so then you'd just have to decide which game(s) you think are more "cognitition" focuses.
Personally, I feel like there *might* be an effect for tasks that depend heavily on what people call "emotional intelligence" (scare quotes), but not for anything else (unless you want to talk about long-term carbon dioxide overexposure or something, which is a different topic).
Nice self-replication! And I'm pretty convinced by your successive ruling-out of explanations as to why the effect is real but you failed to detect it (e.g., controlling for temperature).
Sorry if you mentioned this and I missed it, but is there any chance that time (like "index" of the trial/game) was a factor? Like, maybe you got better at the game over each successive play, but CO2 levels were also positively correlated with time? In which case you might mask a "real" effect of CO2 if performance is driven by Time (+) and CO2 (–). As you say, to salvage the effect you'd need for these effects to be of roughly the same size, which is a stretch. But might be worth checking.
The lesswrong post linked about chess is nearly a good measure but is made much noisier by the confounding factor of "how good is the random anonymous person I happen to be playing today?". Happily chess furnishes a much better measure of day-to-day fluctuations in cognition, which would be to solve chess problems: positions (either taken from real games or artificially composed) in which there is a single, objectively correct answer to the question "What is the winning move here"? This also seems more like "decision-making" than finding as many words as you can in a word-search puzzle in a set period of time.
Any chance you can test the correlation against different CO2 sensor noise/bias models? I believe you that low readings happen when you'd expect and high readings happen when you'd expect, but I can imagine observing that even if (CO2 reading) = m*(True CO2 level) + b or even something higher-order.
I did something similar playing Go against the computer when I worked long overnight shifts in an intellectually focus demanding job. Every few hours I'd play a few quick 9x9 games against the computer and decide based on my resulting rating how well I was mentally handling whatever level of fatigue I was in.
Scott: instead of doing univariate p-tests, why not fit a linear model where you add an extra term that depends on days since you started playing to model your skill increasing over time? I’m happy to fit the model for you if you feel comfortable sharing/uploading the data somewhere.
Really interesting to see this. I started a company to commercialize a CO2 capture material back in 2014. At the time, a lot of people I talked with were excited about this study, but I was always really suspicious of the effect size.
I was planning to kick of some self-experiments on cognition in a few weeks using chess puzzles, working memory, and math tests. I ordered a CO2 monitor and will add that in to the mix.
Anyone have any other interventions or supplements they think would be interesting to test?
I think that CO2 has a big impact on my productivity, because I seem to get distracted much easier and am less willing to do real work when I spend a longer time in a less ventilated room. (In fact, I am likely to play a game like WordTwist in that scenario :D) But I never measured this with a monitor, so I would be interested how well my notion of distracted-ness really correlates with CO2 levels. I just can't think of a good, unbiased, way of measuring it...
I put significant probability on the hypothesis that cognition responds to fast changes in CO2 concentration, but not to absolute CO2 concentration; ie, if you go from outside to a stuffy room, you'll be mildly impaired for some duration, and if you go from a stuffy room to outside you'll maybe get a boost for some duration (or perhaps an opposite-direction effect which is also some sort of impairment), but if you wait awhile in either environment you'll return to baseline. This is what you'd expect given a homeostatic mechanism that isn't instant. This would reconcile the original results (where people go straight into a high-CO2 room, and do worse) and your results (where you let the CO2 concentration rise at a slow, natural rate).
This makes sense. Blood co2 levels are not primarily driven by air co2, and the health effects of low ventilation that building codes are concerned about use co2 as a proxy for accumulation of volatile organic compounds and other gases, not co2 itself. At least climate change isn’t making us co2 stupid!
I will consider testing it, mostly because the game is fun. But also does anyone have any other circumstances to test this with that could affect intelligence? “Before vs after strenuous exercise”, “while fasting vs while not”, “before vs after big meal”
Oooh what about high vs low air pollution days? Although id expect that to fail too, the negative effects of partial combustion. product pollution is probably longer term. Many possibilities
Did you happen to experience the other negative symptoms associated with high carbon dioxide at the times in which your monitor recorded very high levels?
I don't think I have a very low credence on substantial individual differences on this. If nothing else, I get the impression that there's a lot of individual variation in how unpleasant it is to be in a high-CO2 environment. Still, for some people to have a large effect and some people to have zero effect would be somewhat surprising.
As a former submariner, I don't recall noticing any effects from varying CO2 levels. I do remember noticing huge effects from varying Oxygen levels as the concentration was adjusted. Lower levels and everyone was tired all the time. Higher levels it was harder to sleep and when I did, I would have bizarre dreams. Maybe there was an effect from CO2 but it wasn't noticeable to the larger effect of oxygen.
Is it consistent with your data that while CO2 has no effect on your performance *given that you decided to play the game*, it may affect whether you're up to the task of doing something vaguely cognitively taxing? In submarines and spaceships, you're in an environment that forces you to do things or else you die/are Severely Talked To by your superiors in an extremely strict social hierarchy; in an experimental room, it doesn't really matter whether you focus or not.
The game is called Boggle, tho that usually refers to the 4x4 version. I was addicted to it in grad school, but frustrated that the boards sometimes had only 3- and 4-letter words, hardly worth the effort of finding. So I wrote a program to generate Boggle boards with more and longer words in them.
First I tried making the probability of a letter showing up proportional to its frequency in English, but that resulted in boards where you could find TENT 3 different ways. Then I tried using a genetic algorithm that scored boards according to the frequency of the trigrams found in them, but that was even worse--it could fill the board with regular tilings of T ,H, and E.
Eventually I had the program count the dictionary words found in each board, and use that as the GA score. That worked well enough to run smack into premature convergence, which turned out to be so disturbingly powerful that it radically changed my views on the impact of technology on society.
(Now I think that rapid communications with high connectivity has taken us into a regime where we're no longer capable of evolving quality cultural artifacts / designs. Modern art was perhaps the first case of cultural premature convergence due to tech--an artistic ideology that spread like fire (their analogy, not mine) along the new rail networks linking Europe, and completely displaced all others in institutional art when it was still half-raw. I think premature convergence also contributed to the decline in innovation in pop music since 1990.)
For your purposes, you'd like to reduce the variance of the max possible score on each board. You could try giving yourself an adjusted score of (your score) / (max score). But I don't trust the scoring system to scale the right way with word length so that this adjusted score would have the same expected value on words of all lengths. Words like "intrapsychically" could mess with your adjusted score. It might be better to write a wrapper around the board-generating program that rejected boards with an unusual distribution of word lengths.
My purely anecdotal self-observation is that my cognition powers drop dramatically when I'm tired, whether or not I realize that I'm tired. In fact, noticing that my cognition is impaired is a clue that I must be tired, even though I don't feel tired, so I lie down and am usually out like a light. This must far outweigh any effects of CO2 - unless the CO2 is making me sleepy, which it could be.
Show your data. Show *all* your data. This would all fit so, so nicely into a single scatter plot, and I'd be about 5x more convinced if I could see it (especially if you can work in time data in a way that lets us know how your learning the game looks compared to other effects).
I’m not a statistician, but isn’t p=0.97 a really high p value? My understanding is that if you have a completely uncorrelated thing your testing, your p value should be a uniformly distributed RV. So there’s like a 5% chance of p<0.05 and same for p>0.95. I guess this isn’t a real critique beyond “if I’m understanding the stats correctly this seems so high it makes me wonder if something’s up”
I'm an environmental engineer, but I don't specialize in indoor air quality. It is not a climatic impossibility to have indoor CO2 levels < outside air CO2 levels. A local sink (most likely a houseplant) will easily accomplish that. Whether it's reasonable for your word gaming ecosystem is for you to determine.
How does that CO2 monitor compare with the "smart" one from Awair? Await keeps a record of the CO2 levels over time, so it might be easier to use in a study. I also don't know how accurate it is, but can tell that opening the window and the like affect the CO2 quality.
Andrew Gelman has a great term: “trying to weigh a feather in the pouch of a jumping kangaroo”. Sure there can be a ton of confounding variables, but if this were a meaningful effect (for certain definitions of “meaningful”) it wouldn’t require a research study that takes those into account to assess it.
Secondary problem is even if you did find an effect it doesn't account for adaptation effects. My prediction would be your body optimizes for a given CO2 level over some period. Consider climbers adjusting to higher elevation--takes a day or two.
It might apply more to motivation than calculation.
It's totally plausible that CO2 homeostasis mechanisms include throttling back the production of it, rather than just breathing more to get rid of it and excreting it in urine.
Throttling back production might involve the brain trying to avoid work when blood CO2 levels are elevated. But I don't know if 1000ppm is enough to have a measurable effect on blood CO2 levels.
"Generally, under normal physiologic conditions, the value of PCO2 ranges between 35 to 45 mmHg, or 4.7 to 6.0 kPa"
At atmospheric pressure slightly above sea level, 100kPa, this is equivalent to 47,000-60,000 PPM. So an extra 1000 PPM in the air you're breathing is not likely to matter. It'll slow CO2 clearance by 2%
It's likely other factors that correlate with having windows open are causing the inconsistent effects in some of the studies effects: both lighting and odors can have emotional effects which might affect performance.
Also as an Ashkenazi Jew your ancestors may have spent a 99th percentile amount of time indoors and picked up some adaptation, if there ever was any effect of CO2 per se. In hunter-gatherer and farming times maybe high blood CO2 sort of signaled "I've been exercising too much and need to rest" but after urbanization it meant nothing.
As a practical matter going for a walk outside in the fresh air feels good and elevates my mood. I don't know what the active ingredient is.
When you said high CO2 levels could affect focus and decision making, I thought that made a lot of sense - in the context of continuous high CO2 levels. e.g. if you work in a closed-up office but go out at intervals for 'fresh air', you might feel refreshed and able to continue working, but if you stay indoors all day you might feel much worse.
Your experiment seemed closer to the scenario of sometimes going out for fresh air - I'd be much more curious to see an experiment with one group working in the same high-CO2 environment every day, and another in a better-ventilated one.
Watch out for CO2 meters that "calibrate" their sensor by assuming that the lowest level in any 7 day period is outdoor air. This is a terrible assumption, and gives entirely bogus readings that change over time, but that's how a lot of them work.
Instead, you want to get one with dual IR sensors, one of which is used infrequently in order to calibrate the other. (The theory here is that the main source of decalibration is the IR sensor's LED dimming with repeated use.)
One way the competing factors could cancel despite large effect sizes is if your inclination to play is or is affected by a threshold effect. Like a more complicated and convoluted version of measuring the temperature in a room when the thermostat of a heater kicks in, and then concluding that opening the window doesn't have an effect on the air temperature.
You best hope there is no cognitive problems caused by high ambient CO2 levels. The US Navy has several Boomers on station, underwater, with very high CO2 levels at all times. Any one of them could kill a couple of hundred million people, and trigger Armageddon.
I firmly hope, and fervently pray that our brave sailors manning those vessels are not cognitively impaired.
Indoor air quality is typically significantly worse than outdoor air (fire season aside). Seems like a plausible explanation here is that CO2 is a proxy for how well ventilated a space is, but that it doesn't have a direct effect on cognition by itself. The original study may have been picking up some other pollutant where Scott's didn't.
Personal anecdote - I was taking a 3 hour exam in a lecture hall with ~100 other people with no circulating air. I felt like falling asleep due to how stuffy the air was, but others seemed like they were doing fine.
My family used to play this all the time as the game Boggle. Fantastic game, I crush all my friends mercilessly if they are foolish enough to challenge me, who has been forged in the fires of family game night for years.
So, as a submariner, I think I should point out that CO2 levels are very tightly controlled with atmosphere control equipment on board. It’s something that the OOD reports to the Captain every watch, and there is monitoring equipment in every space. Nonetheless, there are certainly higher levels of CO2 and lower levels of O2 on board than there are outside the boat - and every sailor on board can tell that it effects their mental abilities. It’s hard to describe, but the feeling when you come up and equalize atmospheres with the outside world and how that effects your mind is very noticeable. Now we certainly do very hard work very well, like you said, but mental effects from lower CO2 are real. Nonetheless, that’s obviously both anecdotal and referring to much greater changes in CO2 level than you are talking about, so it may not be a useful data point.
This just seems a priori kind of unlikely. 445ppm CO2 is 0.34 mmHg, and even 3000ppm is only 2.3 mmHg. Meanwhile, as far as I know pCO2 in tissues is around 40 mmHg, so even the highest concentration of CO2 you tried is 1/20 the normal concentration in tissues.
Amazon not only sells CO2 monitors, but also CO2 generators. They seem to work on the principle of adding citric acid to sodium bicarbonate. Of course, any acid, like the acetic acid in vinegar, should work. There must be some ppm of CO2 which has an effect on cognition. This would be approaching the problem from the other end. With appropriate caution, of course.
Also, increased partial pressure of CO2 should drive an increased respiratory rate to clear the CO2 by the usual physiological mechanisms of negative feedback. Measurements should show that respiratory rate correlates well ppm CO2. And at some ppm of CO2 the compensatory respiratory rate should be overcome.
Additionally, CO2 accumulation in the blood lowers the pH, (acidosis), which probably mediates the effects on concentration, headaches, etc.,
Nice post! Does anything else about air quality affect cognition? I would imagine at the very least it's harder to get work done if it's 50 or 90 degrees Fahrenheit in a room, and probably hard if the air is forest fire level. I often do feel that I think better outside, but it may just be that I think better standing up or walking. Indeed, at least stereotypically pacing back and forth is supposed to be good for pondering things and I feel there is some truth in this; it matches my own experience (not sure of the mechanism). Likewise changing environments may lead to coming up with new ideas. I've had experiences where I was stuck on something and then as soon as I've walked away a new idea comes up (lots of mathematicians do too). Still I don't know if this is about cognition rather than some sort of change in something. Of course, my feelings also may not reflect reality, but I think there's some truth to the idea that changing the environment leads to new ideas.
This might be harder to test. Especially since coming up with new ideas is very different from finding words. But I would imagine most SSC posts don't come to you while you're sitting down at your desks and while you are walking around. And also you find it hard to focus when there is someone else in the room (you said this in a previous SSC post at some point).
Sorry if this became a little incoherent but the general point is that at least my feeling is that there are some environmental factors that do not really affect cognition, and I'm curious which one it is. My suspicion is that there is something to the fact that a change of environment can be good for cognition, which is why so many scientists like walking. I also would guess that this also lends plausibility to the CO2 thing; it's a proposed mechanism for this because they get out of the high CO2-office. It's worth noting whether this effect is real in the first place, and what causes it anyway, if not the CO2.
Well think about where our ancestors spent most of their nights. Tiny tent/igloo/hut with all family members present. Longhouse with an entire tribe. Most of the year, all of the above had a fire lit and minimal ventilation in order to preserve heat. Imagine how high the CO2 levels were, and how low the O2 levels. Yet the ancestors of whoever is alive today thrived in that environment.
High-CO2-impares-cognition hypothesis makes a lot of sense to me, so here are two ways to save it:
1) Your performance in WordTwist is bounded in the type of intelligence CO2 is affecting: say, you play as good as you possibly can at 105 IQ, and CO2 causes your IQ to fluctuate 130 to 110 IQ. Variance in score is explained by other factors - experience, motivation, attention, and so on. Seems very plausible - the game is not that conceptually hard.
2) CO2 affects intelligence in part through motivation (I know I don't want to do anything hard if I can barely breathe!). So then you were feeling very bad (from high CO2 and other factors), you wouldn't want to play at all, and when you were feeling smart, energized and motivated (from low CO2 and other factors), you would do something more productive. If motivation and ability is correlated, and you only want to play at certain motivation, your score stays constant even if your ability is changing.
Hey Scott, could you post scatter plots of all your data here, correlation numbers are really not very informative at all - you can have low correlation despite a very obvious effect, and high correlation when the data clearly looks like a random cloud.
I bought a MH-Z19B co2 sensor off ebay a while back to wire up to a pi/arduino and save some $ from buying a whole meter, but not too shockingly never got to wiring it up. This might be the inspiration I needed for it!
If I do that and figure out a testing scheme, I'll make sure to come back and post a prereg of some sort.
Scott, you posted this essay within three hours of someone mentioning the Healthygamer Youtube / Twitch channel (on video game addiction) in the current Open Thread. I think you might be kabbalistically required to give it a look.
5-10 times 3 minutes might not be long enough duration. Do you have data available?
In all levels of school and university, the high CO2 levels causing impaired capability to participate was considered absolutely obvious consequence of spending prolonged time in poorly ventilated classrooms and lecture halls, which are many and numerous in Finland. I don't know if we ever become stupider, but after 1.5 hours, bad quality of air is obvious and people would start feeling sleepy. This was commonly attributed to CO2 from bad ventilation.
Abstract of the article by Zhang et al 2016 you linked to suggests that the effects are not due to CO2 but bioeffluents.
I can absolutely confirm KPier's observations about chess (I play blitz games at chess.com) - there are clearly good days and bad days, they are mostly unpredictable (ie. unrelated to sleep deprivation etc.) and the difference between them is unexpectedly large
Registering a prediction before I read your results: it'll be one of those maddening things where there's an effect but it's not significant and it's much smaller than the previous studies, so you decline to draw a conclusion other than "so, maybe?". I'm four paragraphs in and the admission that you're playing the game because you wanna rather than for the experiment, while reasonable as a life choice, smells like a potentially strong confounder and I had to pause to think about why.
I believe what I'm anticipating based on own gameplay habits is a self-selection effect where the more up to cognitive tasks you're feeling, the more you play the game. Then if CO2 is only one is several factors in your congnitive state, even if it's the largest one, I would expect no correlation at all - your high-CO2 gaming will occur when other factors combine to partly, if not totally, cancel it out.
One thing to check is if you're actually measuring your CO2 level at all. Most of these cheap meters just auto-calibrate such that whatever the lowest physical concentration they see over a 24 hour period is set to a given internal concentration, usually around 300 ppm.
That said, you'd probably still see large swings. Probably.
I just wanted to express my admiration for the fact that you wanted to replicate this study on your own, and went to such lengths to do it. This is a good win for the scientific method.
I think for certain intellectual activities, there's an optimal level of intelligence. If you're more intelligent then that, you'll want to do something else. Magnus Carlsen made this point about chess[1]. I assume it applies to the wordgame you play as well. This would mean that when you had a low CO2 concentration in your room, but still wanted to play the game, there must have been something else negatively affecting your intelligence at that moment, otherwise you wouldn't have wanted to play at all.
Could level scaling in the game confound any attempt to find (any) correlation (at all)?
I don't know this game, so this might not be applicable at all. Just throwing it out there.
But I'm guessing you would play many rounds in a single sitting, presumably at the same level of CO2. If you do well/not-well in one round, will the game compensate by giving you a harder/easier match next round? I'm sure there are game designers in the audience who can explain better than me why this is good/bad game design, but either way, many games do something to this.
One way to try to see this could be to use only the first game in a sitting as data. I don't have the skills or time to do this properly from here unfortunately.
Hypothesis for confounder: your brain switches signalling strategies between ones that are more or less CO2-tolerant in a predictive way. It uses your circadian rhythm, metabolic rate, etc. as independent variables, and observed CO2 as the dependent variable for this prediction. Your brain manages to function fine (at least in the short term) in both “modes.”
Then, it would only be when your brain mispredicts CO2 and stays in the CO2-intolerant mode while actually experiencing high CO2 load, that you’d see the conditions from these studies.
Under this hypothesis, your brain would probably re-predict easily enough when going from outside to inside, or when opening/shutting windows; so I’d guess you’d have to do something more like “releasing a bunch of CO2 into the room without otherwise changing AQ” to trigger the effect. (Which, if I understand correctly, is similar to what the studies do.)
This makes way too much of a very flimsy and poorly conceived informal experiment, if I understand it correctly. Start with an environment where CO2 levels fluctuate quite a bit (from about 500 ppm to about 3200 ppm), measure CO2 level just after playing WordTwist. Calculate whether there’s any correlation between CO2 level at end of game and score.
In other words, the only thing that was tested is, correlation between momentary exposure to CO2 to game score. No attempt to measure effects of continuous prolonged exposure is mentioned. Yet results are used to challenge more formal experiments that are measuring prolonged CO2 exposure, such as exposure of 8 hours.
This WordTwist experiment is of no significance, unless you hypothesize that the significant effects of exposure to high levels of CO2 is immediate and very transitory. I don’t think there’s any good reason to think that, and it’s certainly not what the studies that are being challenged presume.
It’s an important health and social issue as governments have been supporting or requiring, as a conservation measure, tighter construction to seal in heat, as well as indoor pollutants like CO2, which can easily build up, for example, overnight in a tightly sealed bedroom. The concern is, effects after such a prolonged exposure. If you want to do a realistic informal experiment, try, spending 8 hours in a room where CO2 builds up to say 2000 ppm or higher the whole time. (Well, don’t actually try this by hanging out in a poorly ventilated room full of people until after the pandemic, please.) That would be equivalent to the studies that are in question here. I’d even be willing to bet, your WordTwist scores would show a downward trend after several hours in this scenario.
The posting also asserts that cognitive impairment due to CO2 buildup in the ranges we’re discussion (like, 1500 - 3000 ppm) isn’t a concern on submarines, for example. There’s a comment below from a veteran Navy submariner who reports cognitive issues from CO2 buildup was indeed a big concern in practice. Though, as you note, there’s a Navy study claiming to have found no cognitive effect from even much higher levels of CO2 exposure — but the Navy only tested cognitive effects after exposure to high CO2 levels for 45 minutes, that is, intermittent exposure. Might make sense to test that if as the comment below suggests, steps are already in place to prevent longer term crew exposure to excessive CO2.
If you’re worried about whether a momentary variation in CO2 exposure is going to fry your brain, this WordTwist test should be reassuring, though I’d wonder why you’re worried about that. If you’re worried about a real problem — how conservation efforts may create indoor buildup of CO2 resulting in longer term exposure, this WordTwist test is a straw man.
I moved to a new house and bought 5 CO2 and particle sensors (nerd fact: mh-z19 and sds11 with esphome, 25$/each) to track different rooms. All are recording readings every 10 mins (nerd fact: into my homeassistant and then influx for longterm storage). I also got many cheap (10$/pc) humidity/pressure/temp sensors doing the same and I plan to monitor this for years for the whole 6 people household. Can't wait to see how it (doesn't) correlate with e.g. number of tasks etc.
I did something like this once. I wanted to try clonazepam to see if it would help with some work issues I was having, but I was worried about the cognitive side effects. I coded up the reverse-digit-span test from the Wechsler test as a quick Python program and took it regularly at various times for a week (it takes about two minutes), then started the clonazepam.
What I found was that clonazepam at the dose I was taking had no detectable effect, but there was a huge, massive, consistent improvement in my reverse digit span from taking a half-hour nap in the afternoon.
The thing I have the biggest trouble with is not short-term memory but executive function. I keep meaning to code up the trail-making A and B tests (the reverse digit span thing is a couple hundred lines of code) but I haven't gotten around to it yet.
isn't the way you happen to be breathing during each game a significant confounder here?
I recommend reading "Breath" by James Nestor for some insight into the variability and health effects of different breathing patterns. maybe using something like The Breathing App before each game in the experiment would help?
I am willing to do this and put in a minor amount of effort into a cognitive test (<2 minutes) once when I wake up and once when I fall asleep for about a month if someone is willing to pay for the monitor. Someone will also need to tell me which cognitive test to take.
By coincidence, I recently did something very similar (but did not record my results, so everything that follows is impression based.) I played several hundred games of a very similar word game on a 4x4 grid [Boggle with Friends], and found that I performed markedly worse under some pretty predictable circumstances:
- Before coffee
- Sleep deprived
- Drunk
- Drowsy antihistamines (cetirazine)
The interesting part is that board composition (availability of high-scoring words and the supply of T's, E's, S's, R's etc.) had a much higher impact on the score than any of those factors, and that makes me think that boggle-like word games might not be a good metric for measuring variations in cognition.
Also interesting was the fact that the following conditions didn't have a pronounced enough impact for me to remark on them at the time:
- Low to moderate alcohol intake
- Environmental distractions (not touch based)
- Non-drowsy antihistamines (fexofenadine)
- Hunger
- Time of day
And that the following DID have a pronounced enough impact to note at the time:
- Sitting position
- Touch-based environmental distractions
- Attractiveness of opponent (?!)
- Perception of time pressure*
*Notable because the game clock is always 90 seconds, no matter whether or not you feel like you're jamming the session in between two other tasks.
Don't know if this is a "really good experiment" but I'm interested in testing memory, logic, and creative thinking in the form of solving sunday nyt crossword puzzles (i recently bought a book of 500 of them) in a number of fixed environments (eg desk at home, sofa at home, outdoor patio, coffeeshop, park)
I am interested in tracking co2, particulate matter, temperature, humidity, light (brightness, natural vs artificial), visual stimuli (eg plants vs people vs nothing biological, clean vs cluttered, movement vs still), sleep quality/quantity, diet/weight, and time of day (in addition to solving "performance" obviously).
If the meters in the amazon list are purchased for me, I commit to solving at least 1 puzzle a day until the book is completed, share the results with Scott (and anyone who's interested), and to ship the equipment to whomever Scott directs me to upon completion of the study (I'm a minimalist and object more to there being more "junk" in my life/the world than the actual cost of the equipment).
If 3200 ppm causes headaches and nausea, consider me surprised that it has no effect on cognition. Did you plot your data to see if there's an effect in the highest range?
Quite simply the reason you failed to notice any difference in performance related to CO2 level is because you are not a plant.
The experts say the Earth is warming due to increased CO2 levels caused by humans and the graphs show a bump in the recent past although, nevertheless, we are technically still in an ice age, which has been going on for 2.6 million years, with 100,000 year ups and downs.
Currently there's a human caused CO2 increase. Sixty years ago CO2 was only 380 ppm, whereas now it is 413.93 ppm. When CO2 increases, plants thrive and the Earth greens.
Plants inside a greenhouse prefer a ppm of 900, which boosts plant growth and health from 25% to 100% depending on the plant. As for humans the current 413 ppm is no problem. The CDC says you need 100,000 ppm of CO2 to die and other sources say 40,000 ppm is unhealthy for 8 hours. A submarine's scrubber turns on at 8,000 ppm.
Earth has had much higher CO2 levels than the current 413 ppm and our ancestors apparently thrived. 150 million years ago it was at 3,000 ppm and 6,000 ppm farther back. The cyclical CO2 increase is caused by the oceans' warming which releases a bit of the 93% they contain.
Current climate science may be putting the cart before the horse and one might question the causal direction that 97% of professional climate scientists agree on. But experts are purveyors of groupthink - it's how they make their money and continue their funding; they should not be trusted to give a definitive answer.
In the 1980's and in 2005 Phillip Tetlock studied experts' predictions and found they were no better connecting with reality than anyone else. One might spin this result by saying that we live in a random universe and the experts correctly identified and properly proved that fact.
Thus, regarding CO2, there's a 50% chance we really need to produce more of it. Carbon credits should be carbon debits and we must increase our individual carbon footprint to save the planet. But we could be wrong. To best determine the correct answer what's needed is a monkey throwing darts.
I am less impressed with your study than I am with you finding intrapsychically.
Extremely cool. One possible way to salvage the original result is if there is a cumulative threshold effect: maybe you need to be in a low-carbon environment for a long time to see effects and if you regularly breathe fresh air your mental function is not affected. This wouldn't explain the submarine data point though
I recently (June) started monitoring AQ in my condo after reading similar studies, watching a presentation by DHH from Basecamp (MRqh8oLY7Ik) and trying to find ways to help my wife's migraines. I'm using the Awair AQM8002A.
Our overnight CO2 in our bedroom (shared with 3 children) went up as high as 1300ppm and rarely went under 900ppm any time during the day. I tried opening windows and I could push it down to 700 but as soon as I closed the window (since it's July in Houston) it shot right back up. The humidity however took much longer to recover.
We left for a month and it went down to the 400s (set the temperature to 78 instead of 71).
I just used Condensate Pan Treatment Tablets (Uric acid) two days ago and our overnight reading have gone down to high 700s (from 1300ppm). I've had a lot of trouble figuring out how such a large change in CO2 could have been caused by micro-organisms in the condensate pan... My first thought is faulty meter, which I haven't tested.
No way to know the effect on migraines as it's a long term occasional problem. My personal waking fatigue is much improved though.
Is urban dictionary a valid source of words for the game? Narc…
Could there be factors other than CO2 that correspond to ventilation? Air flow, humidity, level of VOC, sound levels, etc.
I sleep much better with a fan. A study might show that people sleep better with fans running. Headline, “Studies show high air flow helps sleep.” Later we find out it’s the white noise that’s helping.
My anecdotal evidence is that I feel better and more alert when I ventilate the car on long drives. But in car CO2 levels be much higher than a house or apartment. I can’t think of a good way to test this.
I had wanted to measure this exact same thing - ever since I moved into my current flat I feel that I can work so much better with the windows open. And the price did stop me - I thought these monitors were around $10 when I got the idea (why do they cost so much?). So... if logistics of me being in the UK is not an issue, I'd be happy to test?
If you're actually willing to buy the monitor for someone(s), I suspect you'll find plenty of video game streamers who would be willing to do this as a gimmick on their stream for some weeks, so then you'd just have to decide which game(s) you think are more "cognitition" focuses.
Personally, I feel like there *might* be an effect for tasks that depend heavily on what people call "emotional intelligence" (scare quotes), but not for anything else (unless you want to talk about long-term carbon dioxide overexposure or something, which is a different topic).
Nice self-replication! And I'm pretty convinced by your successive ruling-out of explanations as to why the effect is real but you failed to detect it (e.g., controlling for temperature).
Sorry if you mentioned this and I missed it, but is there any chance that time (like "index" of the trial/game) was a factor? Like, maybe you got better at the game over each successive play, but CO2 levels were also positively correlated with time? In which case you might mask a "real" effect of CO2 if performance is driven by Time (+) and CO2 (–). As you say, to salvage the effect you'd need for these effects to be of roughly the same size, which is a stretch. But might be worth checking.
The lesswrong post linked about chess is nearly a good measure but is made much noisier by the confounding factor of "how good is the random anonymous person I happen to be playing today?". Happily chess furnishes a much better measure of day-to-day fluctuations in cognition, which would be to solve chess problems: positions (either taken from real games or artificially composed) in which there is a single, objectively correct answer to the question "What is the winning move here"? This also seems more like "decision-making" than finding as many words as you can in a word-search puzzle in a set period of time.
Any chance you can test the correlation against different CO2 sensor noise/bias models? I believe you that low readings happen when you'd expect and high readings happen when you'd expect, but I can imagine observing that even if (CO2 reading) = m*(True CO2 level) + b or even something higher-order.
I did something similar playing Go against the computer when I worked long overnight shifts in an intellectually focus demanding job. Every few hours I'd play a few quick 9x9 games against the computer and decide based on my resulting rating how well I was mentally handling whatever level of fatigue I was in.
Scott: instead of doing univariate p-tests, why not fit a linear model where you add an extra term that depends on days since you started playing to model your skill increasing over time? I’m happy to fit the model for you if you feel comfortable sharing/uploading the data somewhere.
Really interesting to see this. I started a company to commercialize a CO2 capture material back in 2014. At the time, a lot of people I talked with were excited about this study, but I was always really suspicious of the effect size.
I was planning to kick of some self-experiments on cognition in a few weeks using chess puzzles, working memory, and math tests. I ordered a CO2 monitor and will add that in to the mix.
Anyone have any other interventions or supplements they think would be interesting to test?
I think that CO2 has a big impact on my productivity, because I seem to get distracted much easier and am less willing to do real work when I spend a longer time in a less ventilated room. (In fact, I am likely to play a game like WordTwist in that scenario :D) But I never measured this with a monitor, so I would be interested how well my notion of distracted-ness really correlates with CO2 levels. I just can't think of a good, unbiased, way of measuring it...
Whelp, found out that I suck at word games today
I put significant probability on the hypothesis that cognition responds to fast changes in CO2 concentration, but not to absolute CO2 concentration; ie, if you go from outside to a stuffy room, you'll be mildly impaired for some duration, and if you go from a stuffy room to outside you'll maybe get a boost for some duration (or perhaps an opposite-direction effect which is also some sort of impairment), but if you wait awhile in either environment you'll return to baseline. This is what you'd expect given a homeostatic mechanism that isn't instant. This would reconcile the original results (where people go straight into a high-CO2 room, and do worse) and your results (where you let the CO2 concentration rise at a slow, natural rate).
This makes sense. Blood co2 levels are not primarily driven by air co2, and the health effects of low ventilation that building codes are concerned about use co2 as a proxy for accumulation of volatile organic compounds and other gases, not co2 itself. At least climate change isn’t making us co2 stupid!
I will consider testing it, mostly because the game is fun. But also does anyone have any other circumstances to test this with that could affect intelligence? “Before vs after strenuous exercise”, “while fasting vs while not”, “before vs after big meal”
Oooh what about high vs low air pollution days? Although id expect that to fail too, the negative effects of partial combustion. product pollution is probably longer term. Many possibilities
Did you happen to experience the other negative symptoms associated with high carbon dioxide at the times in which your monitor recorded very high levels?
This post needs more scatter plot!
For the love of Christ, Scott, the game is called Boggle. "WordTwist" isn't the name of a game, it's just an end-run around a potential lawsuit.
I don't think I have a very low credence on substantial individual differences on this. If nothing else, I get the impression that there's a lot of individual variation in how unpleasant it is to be in a high-CO2 environment. Still, for some people to have a large effect and some people to have zero effect would be somewhat surprising.
As a former submariner, I don't recall noticing any effects from varying CO2 levels. I do remember noticing huge effects from varying Oxygen levels as the concentration was adjusted. Lower levels and everyone was tired all the time. Higher levels it was harder to sleep and when I did, I would have bizarre dreams. Maybe there was an effect from CO2 but it wasn't noticeable to the larger effect of oxygen.
Is it consistent with your data that while CO2 has no effect on your performance *given that you decided to play the game*, it may affect whether you're up to the task of doing something vaguely cognitively taxing? In submarines and spaceships, you're in an environment that forces you to do things or else you die/are Severely Talked To by your superiors in an extremely strict social hierarchy; in an experimental room, it doesn't really matter whether you focus or not.
The game is called Boggle, tho that usually refers to the 4x4 version. I was addicted to it in grad school, but frustrated that the boards sometimes had only 3- and 4-letter words, hardly worth the effort of finding. So I wrote a program to generate Boggle boards with more and longer words in them.
First I tried making the probability of a letter showing up proportional to its frequency in English, but that resulted in boards where you could find TENT 3 different ways. Then I tried using a genetic algorithm that scored boards according to the frequency of the trigrams found in them, but that was even worse--it could fill the board with regular tilings of T ,H, and E.
Eventually I had the program count the dictionary words found in each board, and use that as the GA score. That worked well enough to run smack into premature convergence, which turned out to be so disturbingly powerful that it radically changed my views on the impact of technology on society.
(Now I think that rapid communications with high connectivity has taken us into a regime where we're no longer capable of evolving quality cultural artifacts / designs. Modern art was perhaps the first case of cultural premature convergence due to tech--an artistic ideology that spread like fire (their analogy, not mine) along the new rail networks linking Europe, and completely displaced all others in institutional art when it was still half-raw. I think premature convergence also contributed to the decline in innovation in pop music since 1990.)
For your purposes, you'd like to reduce the variance of the max possible score on each board. You could try giving yourself an adjusted score of (your score) / (max score). But I don't trust the scoring system to scale the right way with word length so that this adjusted score would have the same expected value on words of all lengths. Words like "intrapsychically" could mess with your adjusted score. It might be better to write a wrapper around the board-generating program that rejected boards with an unusual distribution of word lengths.
My purely anecdotal self-observation is that my cognition powers drop dramatically when I'm tired, whether or not I realize that I'm tired. In fact, noticing that my cognition is impaired is a clue that I must be tired, even though I don't feel tired, so I lie down and am usually out like a light. This must far outweigh any effects of CO2 - unless the CO2 is making me sleepy, which it could be.
I am surprised by this (I was on team "CO2 adversely affects cognition" for a few years).
Interestingly there's a 2019 study (after my last round of CO2/cognition reading) that shows near zero effect of CO2 on cognition.
If anything, your results are more concordant with this study than any other study I know of about CO2/cognition.
https://www.nature.com/articles/s41526-019-0071-6
Show your data. Show *all* your data. This would all fit so, so nicely into a single scatter plot, and I'd be about 5x more convinced if I could see it (especially if you can work in time data in a way that lets us know how your learning the game looks compared to other effects).
I’m not a statistician, but isn’t p=0.97 a really high p value? My understanding is that if you have a completely uncorrelated thing your testing, your p value should be a uniformly distributed RV. So there’s like a 5% chance of p<0.05 and same for p>0.95. I guess this isn’t a real critique beyond “if I’m understanding the stats correctly this seems so high it makes me wonder if something’s up”
I'm an environmental engineer, but I don't specialize in indoor air quality. It is not a climatic impossibility to have indoor CO2 levels < outside air CO2 levels. A local sink (most likely a houseplant) will easily accomplish that. Whether it's reasonable for your word gaming ecosystem is for you to determine.
How does that CO2 monitor compare with the "smart" one from Awair? Await keeps a record of the CO2 levels over time, so it might be easier to use in a study. I also don't know how accurate it is, but can tell that opening the window and the like affect the CO2 quality.
Andrew Gelman has a great term: “trying to weigh a feather in the pouch of a jumping kangaroo”. Sure there can be a ton of confounding variables, but if this were a meaningful effect (for certain definitions of “meaningful”) it wouldn’t require a research study that takes those into account to assess it.
Secondary problem is even if you did find an effect it doesn't account for adaptation effects. My prediction would be your body optimizes for a given CO2 level over some period. Consider climbers adjusting to higher elevation--takes a day or two.
It might apply more to motivation than calculation.
It's totally plausible that CO2 homeostasis mechanisms include throttling back the production of it, rather than just breathing more to get rid of it and excreting it in urine.
Throttling back production might involve the brain trying to avoid work when blood CO2 levels are elevated. But I don't know if 1000ppm is enough to have a measurable effect on blood CO2 levels.
https://pubmed.ncbi.nlm.nih.gov/31869112/
"Generally, under normal physiologic conditions, the value of PCO2 ranges between 35 to 45 mmHg, or 4.7 to 6.0 kPa"
At atmospheric pressure slightly above sea level, 100kPa, this is equivalent to 47,000-60,000 PPM. So an extra 1000 PPM in the air you're breathing is not likely to matter. It'll slow CO2 clearance by 2%
It's likely other factors that correlate with having windows open are causing the inconsistent effects in some of the studies effects: both lighting and odors can have emotional effects which might affect performance.
Also as an Ashkenazi Jew your ancestors may have spent a 99th percentile amount of time indoors and picked up some adaptation, if there ever was any effect of CO2 per se. In hunter-gatherer and farming times maybe high blood CO2 sort of signaled "I've been exercising too much and need to rest" but after urbanization it meant nothing.
As a practical matter going for a walk outside in the fresh air feels good and elevates my mood. I don't know what the active ingredient is.
When you said high CO2 levels could affect focus and decision making, I thought that made a lot of sense - in the context of continuous high CO2 levels. e.g. if you work in a closed-up office but go out at intervals for 'fresh air', you might feel refreshed and able to continue working, but if you stay indoors all day you might feel much worse.
Your experiment seemed closer to the scenario of sometimes going out for fresh air - I'd be much more curious to see an experiment with one group working in the same high-CO2 environment every day, and another in a better-ventilated one.
Watch out for CO2 meters that "calibrate" their sensor by assuming that the lowest level in any 7 day period is outdoor air. This is a terrible assumption, and gives entirely bogus readings that change over time, but that's how a lot of them work.
Instead, you want to get one with dual IR sensors, one of which is used infrequently in order to calibrate the other. (The theory here is that the main source of decalibration is the IR sensor's LED dimming with repeated use.)
One way the competing factors could cancel despite large effect sizes is if your inclination to play is or is affected by a threshold effect. Like a more complicated and convoluted version of measuring the temperature in a room when the thermostat of a heater kicks in, and then concluding that opening the window doesn't have an effect on the air temperature.
Whether
You best hope there is no cognitive problems caused by high ambient CO2 levels. The US Navy has several Boomers on station, underwater, with very high CO2 levels at all times. Any one of them could kill a couple of hundred million people, and trigger Armageddon.
I firmly hope, and fervently pray that our brave sailors manning those vessels are not cognitively impaired.
Please Lord, protect them and us.
Indoor air quality is typically significantly worse than outdoor air (fire season aside). Seems like a plausible explanation here is that CO2 is a proxy for how well ventilated a space is, but that it doesn't have a direct effect on cognition by itself. The original study may have been picking up some other pollutant where Scott's didn't.
Nice self-experimentation. In my personal experience, air and weather changes (like changes in
pressure) affect some more than others. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3996041/
Personal anecdote - I was taking a 3 hour exam in a lecture hall with ~100 other people with no circulating air. I felt like falling asleep due to how stuffy the air was, but others seemed like they were doing fine.
I didn't collect a lot of data, but I used to gauge how well I was feeling mentally by how long it took to complete an easy websudoku.
Might I recommend Tetris or 1010! for visual/spatial reasoning?
My family used to play this all the time as the game Boggle. Fantastic game, I crush all my friends mercilessly if they are foolish enough to challenge me, who has been forged in the fires of family game night for years.
So, as a submariner, I think I should point out that CO2 levels are very tightly controlled with atmosphere control equipment on board. It’s something that the OOD reports to the Captain every watch, and there is monitoring equipment in every space. Nonetheless, there are certainly higher levels of CO2 and lower levels of O2 on board than there are outside the boat - and every sailor on board can tell that it effects their mental abilities. It’s hard to describe, but the feeling when you come up and equalize atmospheres with the outside world and how that effects your mind is very noticeable. Now we certainly do very hard work very well, like you said, but mental effects from lower CO2 are real. Nonetheless, that’s obviously both anecdotal and referring to much greater changes in CO2 level than you are talking about, so it may not be a useful data point.
I found a LUL, and I must tell the world about it.
Scott, I pay you too much to be banned for a innocuous meme. :|
_______________
I hope more people did these kind of highly controlled sniff tests. I love it.
This just seems a priori kind of unlikely. 445ppm CO2 is 0.34 mmHg, and even 3000ppm is only 2.3 mmHg. Meanwhile, as far as I know pCO2 in tissues is around 40 mmHg, so even the highest concentration of CO2 you tried is 1/20 the normal concentration in tissues.
Amazon not only sells CO2 monitors, but also CO2 generators. They seem to work on the principle of adding citric acid to sodium bicarbonate. Of course, any acid, like the acetic acid in vinegar, should work. There must be some ppm of CO2 which has an effect on cognition. This would be approaching the problem from the other end. With appropriate caution, of course.
Also, increased partial pressure of CO2 should drive an increased respiratory rate to clear the CO2 by the usual physiological mechanisms of negative feedback. Measurements should show that respiratory rate correlates well ppm CO2. And at some ppm of CO2 the compensatory respiratory rate should be overcome.
Additionally, CO2 accumulation in the blood lowers the pH, (acidosis), which probably mediates the effects on concentration, headaches, etc.,
Nice post! Does anything else about air quality affect cognition? I would imagine at the very least it's harder to get work done if it's 50 or 90 degrees Fahrenheit in a room, and probably hard if the air is forest fire level. I often do feel that I think better outside, but it may just be that I think better standing up or walking. Indeed, at least stereotypically pacing back and forth is supposed to be good for pondering things and I feel there is some truth in this; it matches my own experience (not sure of the mechanism). Likewise changing environments may lead to coming up with new ideas. I've had experiences where I was stuck on something and then as soon as I've walked away a new idea comes up (lots of mathematicians do too). Still I don't know if this is about cognition rather than some sort of change in something. Of course, my feelings also may not reflect reality, but I think there's some truth to the idea that changing the environment leads to new ideas.
This might be harder to test. Especially since coming up with new ideas is very different from finding words. But I would imagine most SSC posts don't come to you while you're sitting down at your desks and while you are walking around. And also you find it hard to focus when there is someone else in the room (you said this in a previous SSC post at some point).
Sorry if this became a little incoherent but the general point is that at least my feeling is that there are some environmental factors that do not really affect cognition, and I'm curious which one it is. My suspicion is that there is something to the fact that a change of environment can be good for cognition, which is why so many scientists like walking. I also would guess that this also lends plausibility to the CO2 thing; it's a proposed mechanism for this because they get out of the high CO2-office. It's worth noting whether this effect is real in the first place, and what causes it anyway, if not the CO2.
Well think about where our ancestors spent most of their nights. Tiny tent/igloo/hut with all family members present. Longhouse with an entire tribe. Most of the year, all of the above had a fire lit and minimal ventilation in order to preserve heat. Imagine how high the CO2 levels were, and how low the O2 levels. Yet the ancestors of whoever is alive today thrived in that environment.
High-CO2-impares-cognition hypothesis makes a lot of sense to me, so here are two ways to save it:
1) Your performance in WordTwist is bounded in the type of intelligence CO2 is affecting: say, you play as good as you possibly can at 105 IQ, and CO2 causes your IQ to fluctuate 130 to 110 IQ. Variance in score is explained by other factors - experience, motivation, attention, and so on. Seems very plausible - the game is not that conceptually hard.
2) CO2 affects intelligence in part through motivation (I know I don't want to do anything hard if I can barely breathe!). So then you were feeling very bad (from high CO2 and other factors), you wouldn't want to play at all, and when you were feeling smart, energized and motivated (from low CO2 and other factors), you would do something more productive. If motivation and ability is correlated, and you only want to play at certain motivation, your score stays constant even if your ability is changing.
Hey Scott, could you post scatter plots of all your data here, correlation numbers are really not very informative at all - you can have low correlation despite a very obvious effect, and high correlation when the data clearly looks like a random cloud.
I bought a MH-Z19B co2 sensor off ebay a while back to wire up to a pi/arduino and save some $ from buying a whole meter, but not too shockingly never got to wiring it up. This might be the inspiration I needed for it!
If I do that and figure out a testing scheme, I'll make sure to come back and post a prereg of some sort.
Scott, you posted this essay within three hours of someone mentioning the Healthygamer Youtube / Twitch channel (on video game addiction) in the current Open Thread. I think you might be kabbalistically required to give it a look.
Link to the comment: https://astralcodexten.substack.com/p/open-thread-184/comments#comment-2572226
I will replicate this for 1 min chess games if somebody pays for the CO2 meter.
5-10 times 3 minutes might not be long enough duration. Do you have data available?
In all levels of school and university, the high CO2 levels causing impaired capability to participate was considered absolutely obvious consequence of spending prolonged time in poorly ventilated classrooms and lecture halls, which are many and numerous in Finland. I don't know if we ever become stupider, but after 1.5 hours, bad quality of air is obvious and people would start feeling sleepy. This was commonly attributed to CO2 from bad ventilation.
Abstract of the article by Zhang et al 2016 you linked to suggests that the effects are not due to CO2 but bioeffluents.
I can absolutely confirm KPier's observations about chess (I play blitz games at chess.com) - there are clearly good days and bad days, they are mostly unpredictable (ie. unrelated to sleep deprivation etc.) and the difference between them is unexpectedly large
Registering a prediction before I read your results: it'll be one of those maddening things where there's an effect but it's not significant and it's much smaller than the previous studies, so you decline to draw a conclusion other than "so, maybe?". I'm four paragraphs in and the admission that you're playing the game because you wanna rather than for the experiment, while reasonable as a life choice, smells like a potentially strong confounder and I had to pause to think about why.
I believe what I'm anticipating based on own gameplay habits is a self-selection effect where the more up to cognitive tasks you're feeling, the more you play the game. Then if CO2 is only one is several factors in your congnitive state, even if it's the largest one, I would expect no correlation at all - your high-CO2 gaming will occur when other factors combine to partly, if not totally, cancel it out.
One thing to check is if you're actually measuring your CO2 level at all. Most of these cheap meters just auto-calibrate such that whatever the lowest physical concentration they see over a 24 hour period is set to a given internal concentration, usually around 300 ppm.
That said, you'd probably still see large swings. Probably.
I just wanted to express my admiration for the fact that you wanted to replicate this study on your own, and went to such lengths to do it. This is a good win for the scientific method.
I think for certain intellectual activities, there's an optimal level of intelligence. If you're more intelligent then that, you'll want to do something else. Magnus Carlsen made this point about chess[1]. I assume it applies to the wordgame you play as well. This would mean that when you had a low CO2 concentration in your room, but still wanted to play the game, there must have been something else negatively affecting your intelligence at that moment, otherwise you wouldn't have wanted to play at all.
[1]: https://en.chessbase.com/post/magnus-carlsen-on-his-che-career
Could level scaling in the game confound any attempt to find (any) correlation (at all)?
I don't know this game, so this might not be applicable at all. Just throwing it out there.
But I'm guessing you would play many rounds in a single sitting, presumably at the same level of CO2. If you do well/not-well in one round, will the game compensate by giving you a harder/easier match next round? I'm sure there are game designers in the audience who can explain better than me why this is good/bad game design, but either way, many games do something to this.
One way to try to see this could be to use only the first game in a sitting as data. I don't have the skills or time to do this properly from here unfortunately.
Two related pet peeves of mine:
1) This worked for me so it is universally applicable.
2) Studies say this doesn't work for most people so it won't work for me.
Thank you for not falling for either of them.
Hypothesis for confounder: your brain switches signalling strategies between ones that are more or less CO2-tolerant in a predictive way. It uses your circadian rhythm, metabolic rate, etc. as independent variables, and observed CO2 as the dependent variable for this prediction. Your brain manages to function fine (at least in the short term) in both “modes.”
Then, it would only be when your brain mispredicts CO2 and stays in the CO2-intolerant mode while actually experiencing high CO2 load, that you’d see the conditions from these studies.
Under this hypothesis, your brain would probably re-predict easily enough when going from outside to inside, or when opening/shutting windows; so I’d guess you’d have to do something more like “releasing a bunch of CO2 into the room without otherwise changing AQ” to trigger the effect. (Which, if I understand correctly, is similar to what the studies do.)
Geez! I would have blown off studies like this as "possibly bullshit, but do I care enough to try to confirm it? Nah."
But you go out and science the heck out of it! I'm awed.
This makes way too much of a very flimsy and poorly conceived informal experiment, if I understand it correctly. Start with an environment where CO2 levels fluctuate quite a bit (from about 500 ppm to about 3200 ppm), measure CO2 level just after playing WordTwist. Calculate whether there’s any correlation between CO2 level at end of game and score.
In other words, the only thing that was tested is, correlation between momentary exposure to CO2 to game score. No attempt to measure effects of continuous prolonged exposure is mentioned. Yet results are used to challenge more formal experiments that are measuring prolonged CO2 exposure, such as exposure of 8 hours.
This WordTwist experiment is of no significance, unless you hypothesize that the significant effects of exposure to high levels of CO2 is immediate and very transitory. I don’t think there’s any good reason to think that, and it’s certainly not what the studies that are being challenged presume.
It’s an important health and social issue as governments have been supporting or requiring, as a conservation measure, tighter construction to seal in heat, as well as indoor pollutants like CO2, which can easily build up, for example, overnight in a tightly sealed bedroom. The concern is, effects after such a prolonged exposure. If you want to do a realistic informal experiment, try, spending 8 hours in a room where CO2 builds up to say 2000 ppm or higher the whole time. (Well, don’t actually try this by hanging out in a poorly ventilated room full of people until after the pandemic, please.) That would be equivalent to the studies that are in question here. I’d even be willing to bet, your WordTwist scores would show a downward trend after several hours in this scenario.
The posting also asserts that cognitive impairment due to CO2 buildup in the ranges we’re discussion (like, 1500 - 3000 ppm) isn’t a concern on submarines, for example. There’s a comment below from a veteran Navy submariner who reports cognitive issues from CO2 buildup was indeed a big concern in practice. Though, as you note, there’s a Navy study claiming to have found no cognitive effect from even much higher levels of CO2 exposure — but the Navy only tested cognitive effects after exposure to high CO2 levels for 45 minutes, that is, intermittent exposure. Might make sense to test that if as the comment below suggests, steps are already in place to prevent longer term crew exposure to excessive CO2.
If you’re worried about whether a momentary variation in CO2 exposure is going to fry your brain, this WordTwist test should be reassuring, though I’d wonder why you’re worried about that. If you’re worried about a real problem — how conservation efforts may create indoor buildup of CO2 resulting in longer term exposure, this WordTwist test is a straw man.
I moved to a new house and bought 5 CO2 and particle sensors (nerd fact: mh-z19 and sds11 with esphome, 25$/each) to track different rooms. All are recording readings every 10 mins (nerd fact: into my homeassistant and then influx for longterm storage). I also got many cheap (10$/pc) humidity/pressure/temp sensors doing the same and I plan to monitor this for years for the whole 6 people household. Can't wait to see how it (doesn't) correlate with e.g. number of tasks etc.
I did something like this once. I wanted to try clonazepam to see if it would help with some work issues I was having, but I was worried about the cognitive side effects. I coded up the reverse-digit-span test from the Wechsler test as a quick Python program and took it regularly at various times for a week (it takes about two minutes), then started the clonazepam.
What I found was that clonazepam at the dose I was taking had no detectable effect, but there was a huge, massive, consistent improvement in my reverse digit span from taking a half-hour nap in the afternoon.
The thing I have the biggest trouble with is not short-term memory but executive function. I keep meaning to code up the trail-making A and B tests (the reverse digit span thing is a couple hundred lines of code) but I haven't gotten around to it yet.
I've started playing some chess and I also log the CO2 and other stuff, so I can look at the data in some weeks/months.
isn't the way you happen to be breathing during each game a significant confounder here?
I recommend reading "Breath" by James Nestor for some insight into the variability and health effects of different breathing patterns. maybe using something like The Breathing App before each game in the experiment would help?
I am willing to do this and put in a minor amount of effort into a cognitive test (<2 minutes) once when I wake up and once when I fall asleep for about a month if someone is willing to pay for the monitor. Someone will also need to tell me which cognitive test to take.
poast time series graphs
By coincidence, I recently did something very similar (but did not record my results, so everything that follows is impression based.) I played several hundred games of a very similar word game on a 4x4 grid [Boggle with Friends], and found that I performed markedly worse under some pretty predictable circumstances:
- Before coffee
- Sleep deprived
- Drunk
- Drowsy antihistamines (cetirazine)
The interesting part is that board composition (availability of high-scoring words and the supply of T's, E's, S's, R's etc.) had a much higher impact on the score than any of those factors, and that makes me think that boggle-like word games might not be a good metric for measuring variations in cognition.
Also interesting was the fact that the following conditions didn't have a pronounced enough impact for me to remark on them at the time:
- Low to moderate alcohol intake
- Environmental distractions (not touch based)
- Non-drowsy antihistamines (fexofenadine)
- Hunger
- Time of day
And that the following DID have a pronounced enough impact to note at the time:
- Sitting position
- Touch-based environmental distractions
- Attractiveness of opponent (?!)
- Perception of time pressure*
*Notable because the game clock is always 90 seconds, no matter whether or not you feel like you're jamming the session in between two other tasks.
I'd be curious to also see the effect of other indoor pollutants like PM25 and Formaldehyde.
Don't know if this is a "really good experiment" but I'm interested in testing memory, logic, and creative thinking in the form of solving sunday nyt crossword puzzles (i recently bought a book of 500 of them) in a number of fixed environments (eg desk at home, sofa at home, outdoor patio, coffeeshop, park)
I am interested in tracking co2, particulate matter, temperature, humidity, light (brightness, natural vs artificial), visual stimuli (eg plants vs people vs nothing biological, clean vs cluttered, movement vs still), sleep quality/quantity, diet/weight, and time of day (in addition to solving "performance" obviously).
The meters required for this experiment are listed at: https://www.amazon.com/hz/wishlist/ls/1P972IIAIEZZI?ref_=wl_share
If the meters in the amazon list are purchased for me, I commit to solving at least 1 puzzle a day until the book is completed, share the results with Scott (and anyone who's interested), and to ship the equipment to whomever Scott directs me to upon completion of the study (I'm a minimalist and object more to there being more "junk" in my life/the world than the actual cost of the equipment).
If 3200 ppm causes headaches and nausea, consider me surprised that it has no effect on cognition. Did you plot your data to see if there's an effect in the highest range?
Quite simply the reason you failed to notice any difference in performance related to CO2 level is because you are not a plant.
The experts say the Earth is warming due to increased CO2 levels caused by humans and the graphs show a bump in the recent past although, nevertheless, we are technically still in an ice age, which has been going on for 2.6 million years, with 100,000 year ups and downs.
Currently there's a human caused CO2 increase. Sixty years ago CO2 was only 380 ppm, whereas now it is 413.93 ppm. When CO2 increases, plants thrive and the Earth greens.
Plants inside a greenhouse prefer a ppm of 900, which boosts plant growth and health from 25% to 100% depending on the plant. As for humans the current 413 ppm is no problem. The CDC says you need 100,000 ppm of CO2 to die and other sources say 40,000 ppm is unhealthy for 8 hours. A submarine's scrubber turns on at 8,000 ppm.
Earth has had much higher CO2 levels than the current 413 ppm and our ancestors apparently thrived. 150 million years ago it was at 3,000 ppm and 6,000 ppm farther back. The cyclical CO2 increase is caused by the oceans' warming which releases a bit of the 93% they contain.
Current climate science may be putting the cart before the horse and one might question the causal direction that 97% of professional climate scientists agree on. But experts are purveyors of groupthink - it's how they make their money and continue their funding; they should not be trusted to give a definitive answer.
In the 1980's and in 2005 Phillip Tetlock studied experts' predictions and found they were no better connecting with reality than anyone else. One might spin this result by saying that we live in a random universe and the experts correctly identified and properly proved that fact.
Thus, regarding CO2, there's a 50% chance we really need to produce more of it. Carbon credits should be carbon debits and we must increase our individual carbon footprint to save the planet. But we could be wrong. To best determine the correct answer what's needed is a monkey throwing darts.