461 Comments

My family moved in the middle of the school year when I was in elementary school, and because of that I never actually learned long division or long multiplication (it was later in the year in the first school and earlier in the year in the second school). I also missed a whole host of lessons about things like animal classification, American folk tales, and who knows what else. I never managed to pick up either long division or long multiplication, but it didn't seem to have any impact on my future academic career (I was the top of my class for math in high school and got good grades on my math classes in college).

Potentially missing something in Kindergarten could have hurt more, but those are also the skills that will be obviously missing and that the teacher would know to help with. If the whole class next year can't do long division, the teacher can throw in an extra lesson or two on long division and even everyone out.

Expand full comment

I think that test scores are the wrong thing to look at. I suspect the main benefits of grade school come from getting improved socialization and developing better strategies for general learning/problem solving. Like in your Spanish example, I also don't remember the majority of my second-language education from grade school, but I feel like the experience alone let me explore a lot of new ways of thinking.

Expand full comment

As I've written at considerable length, generally children sort themselves into ability bands very early in life and then stay in those same bands, relative to peers, with remarkable fidelity. (See https://freddiedeboer.substack.com/p/education-doesnt-work) Presumably this is due to some kind of intrinsic ability asserting itself consistently throughout academic life, probably genetic in origin. My guess is that once students get back into school they will fall back into their old ability bands in dominant majorities and the hierarchy will have reasserted itself. What's less certain is how the schooling pause will influence how any given age cohort performs relative to another. Since the pause generally seems to be happening fairly uniformly across cohorts it's hard to see that it will make much difference, although I suppose people who miss their last year of formal schooling and never make it up could perform worse compared to those who finished right before the pauses. But in general I suspect that you're correct and that this just won't matter much, for the reasons you've laid out, save for those rare kids who just literally never return to formal schooling at all.

Expand full comment
founding

I feel like we have not properly grokked the fundamentally flawed nature of every social science study and far too many people treat them as statements of simple fact about reality instead of generally a mixture of fraud, other kinds of fraud, and luck.

Expand full comment

I wonder if quality of memory significantly influences one's take on this question. I was a top student, but I remember even less than Scott purports to. So to me, the idea of worrying about having missed 5% of which I will forget 95% is quite absurd. But perhaps if I remembered all the classes and the content, then the value of that time would be much more real, even years later.

Expand full comment

A serious issue is that kids in much of the US didn't miss a year of school, they had a year of sad-ass Zoom school during which an abnormally large number of kids checked-out and/or flunked everything. I seriously wonder if that is ultimately worse for them than a year of no school.

Expand full comment

Someone said the New Orleans results after Katrina were due to the new charter schools being able to better screen the new teacher cohort.

Expand full comment

> I’m inherently skeptical of these, because I’m suspicious that education researchers love finding that education has huge effects, that any disruption to education is a disaster, and that kids should be in school much more.

We should also expect education researchers to find that strikes are bad for students, since education researchers are incentivized to find results that school administrations approve of, and school administrations want reasons to shut down teacher strikes.

Expand full comment

Based on conversations with K-12 teachers, I've come to think:

- there are roughly two skills taught, "number sense" and "reading comprehension"

- good curriculum + teaching can improve them, though practice and exposure is a big part of it

- it's possible to catch up quickly, though gets harder later b/c you're missing foundations

I think many schools are not good enough at teaching these for absence to matter. But especially for kids who don't get much exposure elsewhere, absence is indeed a missed opportunity. Just, one that would only show up in the stats if the schools were better.

Expand full comment

If missing school affects graduation rates but not learning, maybe kids who miss a lot of school notice how they didn't miss much actual learning, so they realize school matters less than they thought it did, which makes them more likely to choose to drop out.

Expand full comment

One note I'd like to observe: When it comes to "excused" vs "unexcused" absences, what we are generally looking at is, "Kids who have parents who care and are with-it enough to communicate with the school when the child doesn't attend vs children who don't." So it's still very much a correlational relationship that "unexcused" absences predict learning deficits.

Expand full comment

Something that was missing from almost all of those studies is a measurement of resources required for catching the students back up. Almost all systems have some stabilizing mechanisms to help bring up students who e.g. got cancer back up to grade level (extra attention by classroom teachers, special ed, social workers, etc). Even disasters come with extra resources in the US to help support the students. Just because a system kept results within it's normal parameters during normal times doesn't imply it can do so during abnormal times. Looked at from this lens the Pakistan study looks much worse.

Expand full comment

While I think school is definitely overrated, and missing school underrated, a few comments:

- Presumably not only the amount of knoweldge retained, but also the "damage" from missing school varies quite heavily from child to child? Some children have positive effects from missing school, some children have negative effects from missing school. If "missing school" means "missing the science fair project you were going to win" then that's different to "missing maths class you were kind of average at".

- School is more than learning, missing school is probably less bad than missing "time to meet with friends".

- If everyone misses school, the social effects of a single child "missing out" can't be compared to the social effects of a snow day. If you used to be cool but your parents then stopped you from joining meetings with your friends maybe you become less cool? idk, I never understood how being cool worked in school, and the little I knew I have forgot.

- Lots of things vary from school to school, maybe in Argentina they have school figured out fully but not in the US (I kid, but this kind of thing makes it hard to run good studies).

Expand full comment

I suspect that you are entirely correct about grades and test scores as such, but what about harms from prolonged social isolation? I know of no studies on that, but my prior is that it will be a significant factor, especially for teenagers, and especially if they have less than ideal family support.

Expand full comment

I was unschooled until I was 15, I'm pursuing a PhD now. Catching up on the basics wasn't easy but only took a few months. There are still a bunch of random general knowledge things I don't know, but at most it's caused a moment of embarrassment in social situations (e.g., when I genuinely thought dinosaurs were mythical creatures).

BUT I was motivated to catch up, which I think makes a big difference. I'd say most kids probably don't care too much about their education, so for them, missing school might matter more

Expand full comment

Okay, this is tangential. Skip if you don’t want to read someone trumpet their liberal arts and science education.

You may be right about deleterious effects of missing school. But as a case study of the value of school (college), I got a liberal arts education with a double major in chemistry and philosophy. The chem helped me do my environmental chem work right out of school but the classes that I FEEL were most valuable include a course on Kant, a course on modernist epistemology, a course where we read Conjectures and Refutation, Against Method, and Lakstos’s Methodology of Scientific Research Programs. For whatever reason I also feel very influenced and enriched by a history and aesthetics of film class and a film comedy class. I later went to grad school in a different scientific field and then a career in science and was often told that my strength was critical thinking. Of course, we can’t draw causal conclusions. Maybe a talent for critical thinking drew me to my experiences. One thing I feel confident in is I encounter a lot of younger colleagues who can’t think. I cherish those four years. I know I am off point, but you mentioned not being a fan of school and I feel very different. Don’t know how it paid off economically (not too bad it seems) but in terms of what it feels like to be me, it was worth a lot to me.

Expand full comment

I've experienced a lot of different kinds of school. Public school, private schools, charter schools, home schools, self-education, distance learning, family trade education. And none of them for more than a few years. I'm the product of a bunch of incomplete educations.

It definitely handicapped me academically in some ways. I don't feel it's handicapped me all that much in life other than getting through credentialed gates. I feel that the education I had in common family knowledge/trade was the most valuable, for what it's worth. Self-education probably second place. Third was on the job training and vocational stuff.

This makes me think the old apprentice and community school model might be better than what we currently have. Though I'm not particularly certain. Especially in a generalizable sense: it might just be it would have been better for me.

Expand full comment

Oh, and it's probably worth mentioning that New Orleans improved relative to a Louisiana median that was itself a basket case, and so it's hard to draw meaningful conclusions from what happened there even independent of the large demographic changes in the city post-Katrina. Half of New Orleans's charter schools got the worst possible grades in the state's grading system in 2019, which again is relative to a terrible state baseline. Whatever "miracle" occurred post-Katrina it still left a huge portion of the system's students unable to meet basic proficiency standards.

Expand full comment

One problem with verifying your predictions is that standardized test standards keep getting lowered, or even outright eliminated (in favor of e.g. more diverse and inclusive metrics). Thus, comparing test scores today vs. five years from now might be comparing apples and oranges.

Expand full comment
founding

You wrote: "I think some of these points are stronger for poor/minority/at-risk kids, and weaker for comfortable/Eurasian/not-at-risk kids."

For the at risk kids, school closing is a disaster. Imagine a kid living in a homeless shelter or a kid whose parents are terrible role models or, worse, abusive, or a kid who doesn't have enough to eat. For those kids, school may be their only oasis of normalcy. Without school, they will at minimum suffer and might permanently lose a chance at a decent life. This has little to do with test scores and everything to do with socialization and having adults in their lives who care.

You probably leaned the 80/20 rule AFTER you left school, but I would say that 80% of the damage from school closings will be felt by 20% of the kids.

Expand full comment

A lot of kids in DC missed nearly a year and a half of in-person school, and we don't know how many of them also had very little involvement with virtual school. We have some data on learning loss already, and the real story is probably worse because we're not tracking anything for the kids who weren't present enough to take tests. I agree with the conclusion of this, which is that if you're the kind of parent who is an SSC reader, your young kids are probably not going to permanently academically affected in any way by not having school for awhile. But from the perspective of 'should we keep schools closed this year' - which is the perspective I am personally terrified of right now - this is a disaster on a number of levels. We see the kids who were already the worst-off losing the most learning. We don't know how many kids just aren't going to come back at all. (Chronic truancy already having been a major issue, it's not like we're great at making teenagers go to school if they don't want to.) Our teen (and tween) carjacking sprees certainly aren't being helped by not having kids in school. And the message that our city government has sent parents who do think it's important kids be in school is so negative for trust in institutions.

Expand full comment

Hear, hear. I had serious medical problems in grade 5, needed a major surgery in grade 6, and was told I'd have to miss a year. My parents tried homeschooling, rigorously followed a bunch of curricula, and discovered I could finish *all* the assigned coursework in 2 hours/day and spend the rest of the time reading my favorite books. We were so unimpressed by the time wasted in "regular school" that we kept homeschooling another 2 years. I now have a PhD, but those were among the best days of my life.

Expand full comment

I really like the section flow here, especially the straightforward statements of confidence levels at the end. Falsification might be the ideal and all, but once you've demonstrated calibration even the harder-to-check propositions go a long way towards clarifying what points you're trying to get across.

Expand full comment

This article cites a number of studies about various incidents that disrupted education, including Hurricane Katrina, the bombing of German cities during WWII, the Blitz, and the closing of the public school system in Prince Edward County, Virginia after Brown v. Board, that claim to have found long-lasting effects though I’m sure some of them suffer from the problems you mentioned: https://www.propublica.org/article/the-students-left-behind-by-remote-learning

Expand full comment

Kind of unimpressed with this analysis. You carefully look for confounders when you contemplate studies that show ill effects from missing school -- but don't do the same when you contemplate studies that show no difference. That's illogically asymmetric, but consistent with the normal human tendency to have a hard time seeing and testing hypotheses other than the preferred.

For example, if almost everyone right now *assumes* missing a year of school is A Very Bad Thing it follows all the parents/students/teachers/schools that can do so will work extra hard (like your burned-out friend) to make up for it, which would certainly do much to compensate for any intrinsic harm. But if people did *not* assume missing a year of school is bad, which is what you would like them to conclude, they might no longer do so, and hey presto ill effects will emerge as if by magic.

The inherent problem with this kind of study is that the real human beings underlying it change their behavior in response to the experimental conditions, so it is exceedingly hard to have genuinely comparable experiment and control groups -- the people in each group will *not* respond the same, even if they were hypothetically identical, meaning even if you could clone Parent A and Child A1, and put one copy of each in your experimental and control group, they would not respond the same, that being human nature. That's one good reason social science research is infinity times harder than doing research on cells or drugs.

Expand full comment

My daughter would have missed first grade last year. Instead we hired a teacher and gave her in-person lessons with a few other kids. I think the effect of missing that year would have been potentially very bad for her social development, *given her personality and how far along she was socially at the time the pandemic hit.* She of course would have made up some of this gap in the future, but I also suspect there would be some lasting consequences.

I of course can't be sure about any of this. And I'm not making the same claim for all kids, some (likely minority) of whom benefited socially from the school closure. Point is: school is more than just memorizing a bunch of facts.

Also, maybe worth point out that Scott doesn't just have an "anti-school bias." He is extreme on this issue ("SCHOOL IS JAIL") based on his own own idiosyncratic personal history, and I think we should massively discount claims that kids will just go learn what they need to anyway. *Scott* would do this, but he isn't the typical human.

I will also mention that I also harbor a fair degree of skepticism about the value of school, so I'm not coming at this as a diehard defender of education. But I think that a lot of critiques of the form that Scott makes are extremely glib, in the form of, "La di da, the effects might be worse for the socioeconomically disadvantaged." Yeah, no shit. It's easy for me to be skeptical of the value of school -- I went to prep school and Ivy League and have a masters. It's "obvious" to me that this was a form of credentialism that may or may not justify the tremendous expenditure of resources. But this is a very rarified perspective.

Expand full comment

Homeschooling is living proof that missing even significant amounts of school has relative modest effects to most of the population.

Note the word “most”. I’d argue it’s brutal to those from dysfunctional homes and actually net negative for bright kids with neuroatypical backgrounds.

Certainly the slicing of homeschooled kids’ standardized test results prove that they generally excel vs public schools even if their parents are poorer, completely uneducated and motivated by very different things (religion vs anti-bullying vs educational quality)

Expand full comment
founding

> What I would really like is some consistent absolute test (ie your score is how many questions you get right, not how well you do relative to others) that all schoolchildren take yearly.

This sounds like it would be really useful. Would also be nice to use that give that test at the beginning of the year to decide what to teach kids. One of the elementary schools I went to did this, and I was able to skip over the topics I already knew. Other kids benefitted from covering topics they still hadn't mastered or had forgotten over the summer. Seems like the kind of thing you would want to do if you thought what you were teaching in classes was important enough for students to actually know.

Expand full comment

The unschooling study probably doesn’t prove anything since it self-selects for parents of above average intelligence/conscientiousness.

Expand full comment

The MLA citation format stuff still makes me want to smash things. I remember getting crap from teachers and/or professors about incorrect citations and having . "Oh, you italicized the publication's name, instead of the article's title, like you're supposed to? Haha, dummy. Here's a half letter grade off your ten page paper about ancient Persopolis. Better score well on the GMAT son, because your grades aren't getting you anywhere in life."

They turn you into a monster, then they call you one.

Expand full comment

I think we're experiencing cognitive dissonance over this because we expect US public schools to do what they say they're supposed to do (i.e. provide the most education to eligible students within the allocated budget).

When I look at the institution of US public schools, they do not seem well-tailored to this problem. I'd be hard pressed to find a single problem they *are* suited for. As best I can tell, it's a mix between providing free daycare (so parents can work) keeping kids busy (off the streets and out of the labor market), inoculating kids with desirable values, and teaching them a bunch of facts. I think, despite the best intentions of their constituent personnel, the institution is pretty terrible at raising people and preparing them for life as adults.

So I can't be surprised that a year (or five) of missed school is easy to make up, under the current system. Would it be the same if schools were designed to engage students and teach them useful and relevant things for their lives? I hope it wouldn't be.

Expand full comment

In my childhood, the comic strip "Peanuts" would caricature schooling as memorizing long lists of Presidents and Civil War battles, of course portraying it as dreary and pointless. But I would have loved that as a kid! I had all the Presidents memorized by the time I was 12, and I taught them to my brother who was only FIVE. (We're both still big fans of Presidential trivia to this day, and have taken several trips together to visit Presidential historical sites.)

But I learned them all on my own, for fun. Never got anything like that in school.

Expand full comment

In regards to Katrina: while some kids may have missed a year or two of school, the vast majority were back in school somewhere not too long after the storm. A study from RAND says that 31% of displaced public school students didn't return to Louisiana public schools during the 2005-06 school year, but that doesn't take into account out-of-state schools (or private schools). Out of the people who did return to school, the median time before re-enrolling was five weeks (so early October 2005, as many of the school districts in the metro New Orleans area started reopening). I was a college student in New Orleans at the time and I transferred elsewhere for the semester; I have a sibling who was in school for a few weeks in Texas during September, then at a different school in the New Orleans area for the rest of the fall, then her original (private) school in January 2006. That experience was pretty common for middle- and upper-class people in the New Orleans area.

Responding more directly to the point of the graph: the population changes caused by Katrina are likely responsible for a lot of the increase. From 2000-2010 the population of the city itself (coterminous with the Orleans Parish school district) shrank by 140,000, mostly due to Katrina. The drop of the African-American population was 120,000 of that. I'm not sure if year-over-year ACT scores by race are available for New Orleans public schools, but I'm guessing that if you control for race, it wasn't as much of an improvement as it seems. I also suspect that among black New Orleanians, the ones who didn't come back were much more likely to be poor and have kids who performed poorly in school. If you were a homeowner during Katrina, flood insurance payments and Road Home money helped. If you were a renter, your place may very well have been completely destroyed, your job was probably gone, and you might as well start over in Houston or Atlanta or wherever else you ended up.

On a broader note, I think it's pretty clear that the various disruptions of COVID have negatively impacted at least some measurements of student learning. The 2021 AP exams returned to a more typical format (they were shortened and changed in 2020), and scores were way down across almost all subject areas. The AP tests are designed to be absolute and not relative. True, people probably end up forgetting about Teapot Dome or King Lear or Aeneas' trip to the underworld; but insofar as those tests do accurately depict what students are learning in class, the 20-21 school year was not good for those students.

Expand full comment

At the 50th percentile, 5 percentile points is 0.133 standard deviations. At the 98th percentile, that translates into less than 1 percentile point, so it would push you down to the 97th percentile.

Expand full comment

One thing covid has convinced me of is that the most important thing about much of the school system is to give young people an organized place to be and to interact with each other. For high school and university, there's some amount that involves actual learning of significant skills that can be done via online interaction, but for younger children, the online interaction is the worst of both.

I expect that things like pre-school, summer school, and other increased time in school are very helpful ways to give families more support in childcare and socialization, and that these likely have all sorts of positive effects on the future success of these children (and their parents - has anyone tested future earnings for *parents* of Head Start children, or just the children themselves?) But not too much of it is through the "education" aspect of school.

Expand full comment

Also, what about skipping grades, where kids are deliberately caused to miss a year's worth of material (without any gap in schooling) and that's seen as a good thing? I skipped 3rd and 7th grades. I know I missed out on some biology and European history due to skipping 7th grade; biology has remained one of my weaker points, but I think I'm fine at European history, because it comes up a lot and I'm interested in it. In 3rd grade I missed...... cursive? Honestly I have no idea what I would have learned that year; IME before 6th grade they barely even attempt to pretend that they were teaching anything in particular. (I guess I should note that even though I skipped grades I didn't skip any math, and maybe that's important.)

Expand full comment

Unclear whether prediction #4 is intended to be for (math AND reading) or just (math). Suggest editing wording from "...and math" to either "...in math" or "...in both reading and math".

Expand full comment

I'm torn on the value of schooling:

On the one hand, I think it's important to expose children to big topics that they'd otherwise have no exposure to. I'd have no interest in mathematics, computer science, or literature if somebody hadn't showed me why those things are interesting.

On the other hand, that didn't happen in school - I spent 4 years on the quadratic formula, a minute part of mathematics. I read about a dozen novels about the black American experience, some of which were quite good, and most of which were basically trying to copy those good ones, but none of which I personally related to. I discovered the cool stuff in college and through the internet.

Ironically, the one benefit of schooling was the one they were trying to de-emphasize while I was there - the vital skill of shutting up, sitting down, sitting still, and concentrating on boring crap for hour on end. That's what really prepares you for the working world.

Expand full comment

>This one is the classic, but its pattern of effects seems kind of suspicious - a teacher strike in fifth grade is bad for boys’ (but not girls’) sixth grade math scores, but a teacher strike in sixth grade is bad for girls’ (but not boys’) sixth grade reading scores? A strike is four times worse in fifth grade than in sixth grade for reading, but about the same for writing? Possibly I’m misunderstanding this methodology, but otherwise I’m not sure what to think.

This just looks like run of the mill p-hacking.

______________

If there is some impact from teacher strikes ... couldn't a negative impact on teacher quality explain worse student outcomes better than the small period of absence? A bunch of teachers striking and getting some permanent concession about retaining bad teachers could plausibly impact every generation that goes to that school. And you've already conveniently done the research on teacher quality: https://slatestarcodex.com/2016/05/19/teachers-much-more-than-you-wanted-to-know/

A future earnings drop sounds suspicious, but a permanent drop in teacher quality would show up decades later.

Expand full comment

"And they made good laws and kept the peace and saved good trees from being unnecessarily cut down, and liberated young dwarfs and young satyrs from being sent to school..."

~ the laws of Narnia under the kings and queens in C. S. Lewis's "The Lion, the Witch, and the Wardrobe"

Expand full comment

Reminds me a bit of Arthur Jensen's summary of "Isabelle":

"The effects of maturational readiness were seen most dramatically in the famous case of Isabelle, who, from birth to age 6.5, was reared in a semidarkened attic by her deaf-mute mother, without any other social contacts (Davis 1947). When she was finally discovered by the authorities (age 6.5), she was incapable of speech, acted much like an infant, and had a Stanford-Binet mental age of 1 year 7 months. Once she was placed in a normal social environment, however, her rate of learning was far in excess of that of the average child of the same mental age. She quickly learned to talk and acquired vocabulary with phenomenal speed, in fact, at about the rate that would be expected for a child with an IQ of 300! But this incredible rate of learning lasted only until her mental age caught up with her chronological age, or maturational level, at about age 8. Within two years she had advanced from a mental age of 1 year 7 months to a mental age and level of scholastic performance on par with her 8-year-old classmates."

Source: https://sci-hub.do/https://doi.org/10.1016/1041-6080(89)90009-5, page 45

Expand full comment

This is a discussion I have with my wife a lot. We have some minor disagreements, but our general feeling is that the educational value of school, meaning the facts you learn, is less important than learning to jump through the right hoops and get along with people. I guess it's a discouragingly Prussian way of thinking, but learning to be a good little citizen helps your prospects in life. It's nice if you can learn some basics along the way. I think in the higher grades it's critical to learn how to interact with your peers, as you form and break friendships and start working through romantic relationships a little bit. Those interactions happen at school, for the most part, and they're a big part of why we send the kids.

My kids had a rough year last year, including with their grades, but the worrying thing was more that they got more isolated and depressed. Maybe in the absence of formal, mandated education there would be another way for kids to really stretch their social skills. But in our system, it seems to happen at school or not really at all.

Expand full comment

The oddest aspect of the debate around covid and schooling is that the people who most strongly defend the importance of public schooling are the ones who most want to close/limit schooling.

I would estimate that my kids (4 ranging from 4-10) have had about 3 months of schooling since March 2020 (we’ve supplemented). We live in one of the best public school districts around. The exodus to private schools has been massive. If schooling matters the oncoming inequality will be massive.

We see kids at activities outside of school regularly. The parents who are most vocal about restrictions in school don’t seem to demand the same restrictions anywhere else. Such an odd dynamic.

Expand full comment

As a teacher, I agree with this. Perhaps I'm burned out and cynical, but it seems to me that school is mostly a holding pen for children while parents go to work. Certainly if there was one theme from how parents reacted to covid, online schooling, and everything else, it was that the primary tension came from how disruptive it was to the parents' lives to have no place to send the kids during the day.

I think there's a performative need to pretend to really care about what actually goes on at school, because 1) parents treat it as a proxy measure for their validity as parents, 2) as a society and sometimes personally we spend a bunch of money on it, and 3) politically we throw around the idea that education is the solution to all of our social problems. Oh, we could have beat covid if only the population were more educated! And global warming, and poverty, and racism, and and and... As if.

(aside: I imagine Biden's Child Tax Credit thing where he sends every parent ~$3000 per year per kid no questions asked will do more to solve social problems - including educational achievement - than spending double that amount on schooling would have done. Turns out the way to solve problems is to do something about them, rather than pretend to teach the next generation to solve them and then wait 20 years to see if they do.)

But anyway, when it comes down to it, I think it's pretty obvious just from interacting with people in society that the vast majority of people aren't really learning what school is teaching, and the world seems to sort of plod along anyway. Sure, there are some edge cases of kids like me who learned a lot in school, but I feel like we would have learned a lot anyway. But for most people it won't matter much if you've "fallen behind" on the stuff you're going to forget anyway because it didn't matter in the first place whether or not you learned it - we just pretended it mattered in order to defend our individual and collective egos from having to take a hard look at what it would actually mean to be a decent parent or a decent society.

And yeah, closing any social service will hit the socially vulnerable the hardest - but again, schools were never the best way to solve social problems. If we wanted to, we could still help the socially vulnerable without giving their kids covid.

Expand full comment

AP tests happen in early May nationwide. There's significant variance in when American high schools start (early August - early September), so some students get a full extra month of class time before the AP test (and those of us who started school in September get a full month with basically no instruction—my AP calculus class started a DnD group). If seat-hours matter, one would expect matched groups from school districts that start early to get meaningfully better AP scores than school districts that start late. I don't know whether anyone's looked into this, but it could be an interesting source of data.

The only thing I found with a very cursory google was this story, which suggests that a later start (how much later?) didn't change this school district's AP scores: https://www.argusleader.com/story/news/education/2017/08/15/late-start-date-didnt-stop-ap-students-scoring-well-exams/567997001/

Expand full comment

I think earnings is the wrong measure for educational value. It's pretty close to zero-sum - there are only so many positions, and either one person or a different person will get each one. We could chase after education systems that lead to higher earnings for their students, but that just means all the schools not implementing those systems as successfully will produce more students who wind up in *lower*-paying jobs than otherwise, as they're being outcompeted. I mean, yeah, I'd prefer my children be educated in the system that is most likely to guarantee them financial security. But as for how society at large should improve education, I'd rather focus on producing students who are better able to participate in society more generally: by being more informed voters, having better ability to discern which forms of evidence are meaningful, being more capable of expressing their views to others and understanding the views of others, etc. There is already an excess of qualified applicants in virtually every job requiring education.

Expand full comment

When I first looked at the post I missed the word "Missing," which made the subject even more interesting.

A couple of observations from an unschooling parent:

1. Our kids were initially in a very small private school run on unschooling lines. One year some of the kids wanted to learn math. They started at the beginning of arithmetic, ended in algebra. It was a small mixed group of kids of varying ages and talents.

2. In college, our unschooled daughter got along better with faculty and staff than with fellow students. She was struck by the fact that when a class she liked was canceled for a day, the other students were happy. She was bothered by the fact that she was being asked to write papers which would ever be read by only one person, and only because it was his job to read and grade them.

Oberlin, where she spent her first two years, has a one month winter term during which you can do any educational project a professor will sign off on, not necessarily on campus. Her second year she came home and translated an Italian recipe collection from the early 15th century. It's now webbed. That was the sort of thing she thought she ought to be doing, given her educational background.[http://www.daviddfriedman.com/Medieval/Cookbooks/Due_Libre_B/Due_Libre_B.html]

Expand full comment

> my own experience with these standardized tests is that eg my school’s social studies courses would move leisurely through the Native American and colonial era, get distracted talking about How Bad Slavery Was, and then a week before the standardized test go into “ohmigod we’re supposed to be at the 1950s now fuck fuck fuck Grant Hayes Garfield Arthur Cleveland Harrison Cleveland McKinley World War One World War Two labor unions Japanese internment ok good luck!” mode

This is hilariously accurate to my experience as well.

Expand full comment

Why is it considered so very terrible if a kid has to repeat a year of school? Okay, so they'll be 19 instead of 18 when they go to university- so what? Over here kids will repeat the Leaving Cert if they didn't get enough points for the course they wanted.

I can see the problem would be if an entire class needs to repeat, because now you have two classes of the same year. But if everyone is repeating a year, that will sort itself out, surely? This kind of hair-tearing panic about "their life will be ruined!!!" doesn't seem justified, since it's an unusual event due to the pandemic and every child who was kept out of school, not just your kid, is in the same boat.

And frankly, if your kid is going up against the kids of people who can easily afford cram schools and grinds and six extracurriculars calculated to get them into The Right College, they're not competing on a level playing field anyway so a year more or less is not that much of a difference.

Expand full comment

My son went to a serious Montessori school from age 3 to 11. He finished all their math and science material early, went through their grammar and history and then Latin. Maybe 3 hours a day is the longest work day they had.

He socialized with the other kids, and that's all the school really provided. Some of the peers were real bullies, but he seemed happy. He was not a troublemaker so the teachers completely ignored him.

When he turned 8, we suddenly thought he ought to do something more structured. There was a class Stanford University offered online, for gifted kids. He did their set theory class with great relish...maybe 20 minutes daily.

We also took him to weekend (once a month) outreach programs at our local university - specially in math, a little in computer science (they met rarely) and he went through the Cambridge Latin book on his own...

All that matters is motivation, was our vague philosophy. That can come from not forcing kids to do anything. Get poor quality stuff out of their environment. Maybe no screens. (The school seemed pretty zealous about that). With screens available, kids seek those bells and whistles all the time. We were pretty serious about ourselves having little to zero screens.

He refused to work on his handwriting. We did not see a way to fix that. Maybe a traditional school would have forced him to.

His elementary years spent in this fashion were actually pretty amazing intellectually. And laid a great foundation. He had fun. He made up his own set of meanings for various arithmetic operators and we had to help him unlearn (happened quite easily). He also designed menus for an imaginary hotel business for months.

I think our biggest issue at this non-school, was bullying.

Because they didn't force him to do anything, he retained his curiosity and motivation.

He lost interest in Latin though! But found other healthy passions like math, physics, computer science...

This is a data point of just one, but I think benign neglect of actual "work" atleast in the early years, can be very healthy.

It can go wrong too, I guess.

We were pretty confident this was better than a traditional system with grades and awards and homework, for these early years.

A pandemic would be a fun opportunity to implement something like this within a social bubble.

Expand full comment

We should find out what successful middle age adults remember from school. Anything most such adults don't remember should be considered non-vital to teach kids.

Expand full comment

Maybe school just teaches and reinforces important life skills:

1. Schedules, deadlines and organization.

2. Socialization.

3. Discipline.

The actual subjects and knowledge is maybe just window dressing, since the stuff that's relevant you'll learn anyway in due course. The above skills drive the economic engine of the world. The kids who drop out fail and succeed less perhaps fail to appreciate some of these skills, and so don't end up finding a good spot in the economic engine.

Expand full comment

I believe that schooling is used primarily for signaling purposes (~80%). The best book on this that I know of is Bryan Caplan's The Case Against Education. He believes education is a signal of conscientiousness, intelligence and conformity.

A major issue with education is the effect of the problem of forgetting or information retention. Students retain very little information. It is not surprising that students can actually catch up quickly because all the other students forget most of the old information anyway. What matters is how you stand in relation to other students for signaling purposes because you will forget a great deal of what you learned in school. Even if you remember it for the rest of your life, very little of that information is of use for the students in their personal or professional lives.

The entire process is wasteful and harmful because it is not quite zero sum but there is not a ton to be gained. We should legalize child labor and allow students and parents to make the decision. Here are the top careers in America according to indeed: Cashier, Food preparation worker, Janitor, Bartender, Server, Retail Sales Associate, Stocking Associate, Laborer, Customer Service representative, Office Clerk. A great deal of students will grow up to be in these jobs and not use history, biology, chemistry or even basic math. If you are going to force children to do something for their own best interest, you need to demonstrate that it's in their best interest beyond a reasonable doubt. Using coercion against someone in their so-called best interest when it is dubious or clearly not in their best interest is very bad. Furthermore, the cost of keeping children out of the labor force is ~$10,000. So it costs a great deal to do something that is not in many students best interest.

There is a straw man that child labor means children would be in coal mines but in reality children could probably work simple tasks that are not cognitively demanding. The really smart kids can stay in school because they will eventually have jobs that require technical knowledge. Those children could be compensated the cost of educating them and have that money invested for them. After years of experience, you could have 18 year olds who have developed a great deal of specialized skills and have demonstrated competence in an alternative way.

Expand full comment

Here is a viewpoint.

It is possible that some (maybe majority of) people who need to pick up something they didn't previously learn can pick it up when it becomes important to them. It can also make a mess of any planned curricula that builds on previously learned material, which results in repeating things some of the students know it for the benefit of those who don't know, for one reason or another. Naturally if most of everything gets repeated at some point, altogether missing a year or two does not hurt much. But wouldn't it be nice and efficient if everyone who comes to class would know the prerequisites, and on each class one would keep moving to new, exciting, more advanced things?

Expand full comment

Could anyone here recommend a good source on how much young children benefit from time spent by parents developing early academic/school readiness skills, skills such as letters and the names of letters and numbers, counting, reading out loud, holding a pencil, etc.? Everything I've seen confounds kid interest/IQ/socio-economic status with parental effort, but parent effort is the variable that I'm interested in because it is the variable that I control. I was hoping that Emily Oster's new book would cover this, but she pretty much entirely skips the preschool years.

Expand full comment

So schools are really bad at teaching.

What would an effective schooling look like for kids who actually really want to learn?

Expand full comment

A lot of comments are suggesting that schooling is important for socialization. For those who feel this way, what do you think will happen without this type of socialization?

Homeschooled kids socialize with friends or when they go to church or play little league sports. If children worked, they would socialize with their coworkers. Is this not enough or the wrong type of socialization?

Many people who read this blog are introverted or a little odd socially. Would it be beneficial to use taxpayer money to help them socialize better? I think people would probably say no to this. But it seems reasonable that if it is a good thing to do at age 10, 12, 14 and 18, it should be a good thing at 20, 25, 30, etc.

Expand full comment

I missed most of school, for various reasons, and never caught up on any of it. Now I have a son who has different problems, having developed speech very late, causing him to be held back a couple of years, and ultimately attending a private school where he is given much more attention by the teachers. My experiences with school systems in different countries agree with the article, but I would add that they are also intrinsically unfair. Where you live determines the school you go to, the college you go to, the job you wind up doing - or where you live means you don't get to go to college at all. Schools are a system of networks which unfairly sort you based upon arbitrary factors. The ones at the top of the pile will make sure that you get the grades you need to continue, barring an utter lack of participation in your studies.

Expand full comment

Speaking of which, I would like to use this article as a reading comprehension exercise for my 13 year old kid. Do you have a reading comprehension question appendix?

Expand full comment

Math & language instruction differ from each other, and over time, in systematic ways that I think are relevant here.

From a US perspective, grade-level math (e.g., 4th grade math) is much more repetitive than subject-level math (e.g., trigonometry). In my personal case, I was moving 2-3 "grades" of math per year of elementary, but once I got into the subject sequence (algebra 1 -> geometry -> algebra 2 -> precalc -> calc) I moved at the same pace as everybody else, just years younger. Missing a substantial fraction (no pun intended) of grade-level math is more readily recoverable because what follows doesn't depend on what came before to anything like the same degree. I missed one day of calc, when Reimann sums were covered, and struggled for weeks after (I've still never really grokked them, despite advancing much further in math).

For language arts, there's no apparent dependency at all, just repeated practice. At no level does it matter whether or not you read a particular work in a previous class, and teachers seemed to start from scratch on parts of speech &c every year regardless. I would expect missing this instruction at any point to be as recoverable as missing grade-level math.

Expand full comment

If you are interested in an anecdote:

I did not go to high school (well, attended for two or three months) and now I have a PhD from a very good university. Not receiving any formal education between the ages of 16 and 23 does not seem to have affected my ability to do college (and later grad school) level work.

Expand full comment

I wonder what effect school has on the ability to write Chinese characters. It seems to have been large in the Sinosphere. The papers on the matter are all probably in Chinese, though, and I don't read Chinese.

Expand full comment

https://www.harvardmagazine.com/2010/11/kindergarten-matters

This study seems to suggest that even though the effects in terms of school performance may not last beyond a few years, there is an effect that returns in adulthood, translating directly to increased income.

There might be something special about kindergarten age, or class size, and this effect may not apply to older students, but it does suggest that the quality of schooling may affect more than just school performance, and thus looking at school performance is insufficient.

Expand full comment

Many interviews with homeschoolers/unschoolers/altschoolers and their parents here for those who want to hear from people who skipped most or all of conventional schooling and have had great lives (including an interview with John Deming, father of Laura Deming, who never attended school but was admitted to MIT at 14, dropped out for a Thiel Fellowship at 16, and is now an anti-aging VC),

https://whenschoolsnotworking.com/episodes/

Expand full comment
founding

I fear that this may be a case of looking for one's lost keys under the streetlight. The aspects of education and human development that are easiest to measure and do statistics on, like reading and math and ability to name presidents in order, may be the part where formal education has the least marginal relevance.

You mention the possibility, and one study in support of same, that gaps in formal education might result in increased dropout rate. I'd generalize that to executive function, conscientiousness, and diligence generally. Probably someone like John Stuart Mill is going to grow up to be a quite diligent chap, but not everybody gets to be the son of James Mill. Averaged across the range of human possibility, there may be a substantial impact from formal education here - particularly when we're dealing with the offspring of lliM semaJ. But we're not talking about that here, because we don't have the data readily available.

And then there's the bit about how it is So Very Important that children go to a Proper School so that they can learn social skills; the homeschooled kids will all be doomed to a life of nerdish dorkitude. Which I've always been skeptical of, because of the obvious counterexamples and because every school I've ever been to "taught" social skills the same way being thrown in a lake teaches swimming.

Except, people who are thrown in a lake are *somewhat* more likely to learn to swim, than people who live in a desert. And we're (hopefully!) coming out of a year and a half when not just the schools were closed, but most of the other venues for childhood social interaction. Also, unlike e.g. math or reading, in childhood social interaction, being left behind in the development of social skills results in being excluded from opportunities to develop those skills, so less likely to be a rapid catch-up period at the end.

But, again, we're not talking about that because where's the data? Look, here's some shiny standardized test scores in math and reading.

Expand full comment

Thesis: "adolescence", the cultural phenomenon, is a pure artifact of the school system, in which young people with no responsibilities are forcibly confined with each other, prevented from interacting with the broader society or their own families, and given nothing to do. This alienates them from their families and causes them to develop "strange" (accurate, but only for them) ideas about how society functions.

My second-hand read of other people's impression of homeschooling is that the academic effects are nothing to remark on one way or the other (as noted in this essay), but homeschooled children display a striking tendency to fit into their parents' world without problems that ordinarily-schooled children do not.

Expand full comment

I'm homeschooling 6 kids, with the oldest two in college now, having started when they were13-14. So far they're doing well, better GPA than school educated me at an equally good college. This is in spite of a total of 6-10 hours per week of instruction before college. They all proceed at their own pace, but I predict the rest of them will follow their elder siblings in starting college early and finishing in 4-5 years.

This tells me that smart kids don't get much at all from school. I remember it mostly as busywork and a waste of time, and I'm happy that I get a chance to spare my children that misery.

Expand full comment

The year I started high school, the school I went to (GWHS in Denver) instituted a policy that any student with 11+ unexcused absences in a class would automatically receive an F in that class. I'm sure they had a mandate to fight truancy or whatever but... the sheer self-fulfilling prophecy of it all was obvious to us even as 14-year-olds.

Have I mentioned lately that school is a prison for children, and that education is not its purpose nor has it ever been?

Expand full comment

I really need to precommit to arguing against several views expressed here. My main points will be that dropping 5% percentile from missing a year of school is actually a big deal, that lower grade levels are more important for attendance than upper grade levels, and that measuring the benefits of school for different classes of people reveals how much of a cost absence is, and that metrics by category are the ones one should use for determining whether to go to school or not, as opposed to this extremely broad brush that Scott is using.

Expand full comment

"They should do whatever makes them, their child, and their family happiest and safest, without losing too much sleep over the educational consequences."

I think it would be great to see your take on the risk of COVID for young kids and their parents. If you ask your average Democrat voter, they'd probably tell you COVID is going to kill your child and getting infected is the worst thing that could happen to them. If you ask a Republican voter, they'd say they don't give a shit about COVID. Who's right and who's wrong?

Expand full comment

I question whether learning is relevant to your original question.

Going to an ivy league school isn't important because they teach better--they don't. I've taken classes at possibly a dozen different colleges, from a local community college to Johns Hopkins (and an online course at Stanford); and there is a strong negative correlation between university reputation and the quality of teaching. (For instance, the very best teacher was at the local community college, and two of the worst were at Stanford and Johns Hopkins). The higher reputation a school has, the less importance the professors attach to their classes (and the less importance their teaching proficiency was in their being hired as professors). (It's something like the phenomenon by which, if you go to see the most-famous surgeon in a specialty, you may get the worst care, because you'll sit in the waiting room for 3 hours before the famous surgeon's physician's assistant interviews you, and then the famous surgeon will spend 1 to 2 minutes reading the PA's notes before deciding what to do. Again, personal experience.)

Going to an ivy league school is important because every job search committee at every academic and research institution takes the stack of resumes and pulls out all the Harvards, Yales, Stanfords, and MITs, and only looks at the rest of the stack if none of those people accept an offer. Judging by the results, every grant committee, every venture capitalist, and every Nobel prize committee considers the college someone attended as literally the most-important thing about them.

Similarly, the purpose of going to a private school is probably not to learn, but to put that school's name onto an application to a "good" university. Not one of the "best" universities considers objective measures of learning such as SAT scores, so how could learning help in any case? The only question to ask is whether having attended that good private school /in and of itself/ increases their chances of getting into a "good" university.

Expand full comment

I learned English as a second language. I don't remember much about tenses or prepositional clauses, and I definitely don't remember all of the millions of words I must have heard or read while practicing. Nevertheless, I can now read, write, speak, and understand English mostly correctly most of the time. I only vaguely remember the plot of Lord of the Flies, but if someone alludes to it in an article, I'd know what they were referring to and understand the argument they were trying to make. As a teenager, I learned how to ride a bike. I don't remember any of the hundreds of motions I did while learning to ride a bike, but I can now ride bikes without thinking about it.

It takes practice to be good at anything. Just because you don't remember every detail about the practice, doesn't mean the practice was useless, or that there's a way to get the same result without practice. School drills into you a framework for understanding the world by forcing you to practice, in the form of understanding the details. Over time, the details fade from memory, but if the schooling was any good, the framework remains.

Expand full comment

I've commented below (in reply to an earlier comment) but that was specifically to do with the schooling.

Here, I want to take just a second to comment on the very last sentence of Scott's essay on earnings being affected by COVID.

I obviously don't know how 2019 graduates will fare against 2022 graduates 10 years from now. However I do believe that this data is available for the GFC and other previous economic crises and the evidence is fairly compelling : graduating during a recession really screw your earnings up for a long while (this is why, iirc, your birth year has a huge impact on your earnings... if 21-23 years after you're born, you're in the midst of an economic downturn, you're going to feel it).

https://siepr.stanford.edu/research/publications/recession-graduates-effects-unlucky

https://www.nber.org/digest/nov06/career-effects-graduating-recession

Expand full comment

Stuart Ritchie has a paper from 2018 that has this to say:

We meta-analyzed three categories of quasiexperimental studies of educational effects on intelligence: those estimating education-intelligence associations after controlling for earlier intelligence, those using compulsory schooling policy changes as instrumental variables, and those using regression-discontinuity designs on school-entry age cutoffs. Across 142 effect sizes from 42 data sets involving over 600,000 participants, we found consistent evidence for beneficial effects of education on cognitive abilities of approximately 1 to 5 IQ points for an additional year of education.

Sounds similar to the quasi-experimental results you cited with immigrants and Germany’s new schedule. Not sure how consistent 1-5 IQ points is with what you’ve been saying. It certainly conflicts with what we know about people like unschoolers who’ve missed a whole decade of school.

Expand full comment

An interesting question that you seem to have ignored is the effect of a student's missing out on many day at a *good* school. Or on education from a good teacher in a crap school. Most schools don't teach much so the results you cite are predictable and entirely in line with what we know about the ineffectiveness of the average school and teacher, but they don't really say anything about this other question.

There is some evidence that most of the actual teaching is done by a rather small proportion of teachers. Do keep that in mind.

Expand full comment

There will be somebody who says better, but I think you should split 5. in your list of predictions. There a two assumptions in there.

1) Low performing kids profit from learning "expected values and behaviors".

2) High performing kids suffer from the schol environment.

Expand full comment

One thing I’d consider regarding the primary school versus secondary school aspect is whether the subject matter difference matters. Like, if you miss a year of primary school where they teach multiplication, you can compensate easily. Multiplication comes up a lot in normal life, your parents are probably proficient multipliers, etc. Whereas if you miss a year of secondary school where they teach trigonometry, the rest of your life isn’t going to give you nearly as many opportunities to learn it anyway. That could contribute to primary school absences causing less of a long term effect than secondary school absences.

Expand full comment

There's a recent paper using data from the Netherlands that seems to be of pretty high quality compared to some of the others mentioned above:

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8092566/

I'm still trying to decide what to make of their results. The authors appear to favor the "learning loss is a big problem" explanation, but a "teaching to the test works well" explanation also seems to fit, at least at first glance.

Expand full comment

Hello, English policy expert specialising in education here. Fascinating blog as always. Some comments!

In England we have a National Reference Test - a fixed set of questions sat by a representative sample of 15/16 year olds each year. This will enable us to measure lost learning and test your hypotheses, and is probably as close as we'll get to the idea in your penultimate paragraph.

When the body politic discussed the response to "lost learning", our education experts wanted to lengthen the school day by half an hour but our Treasury wasn't happy with the evidence from it. Our designated "what works centre" for education has an evidence review (including links to various analyses in the full study). Their view is that a longer school day can see pupils make up to an additional two months progress, but very dependent on how the time is used. This is a finding in a lot of education research that will probably not surprise you - how you use time/spend money/implementing proven programmes with fidelity turns out to be very very important.

https://educationendowmentfoundation.org.uk/evidence-summaries/teaching-learning-toolkit/extending-school-time/

Expand full comment

https://astralcodexten.substack.com/p/kids-can-recover-from-missing-even

I have many things to say, about standardized testing and scientificity, about what school is about, and about privilege.

While I was reading, the tidbits about Gaussian elimination and dates of Civil War events stuck me: “when was this? the 1960s? how old are you?!?” Learning math techniques without the context of problem-solving and, even worse, lists of dates and events instead of sociological trends, was already long deprecated when I was a student myself in the late 1980s and the 1990s.

Then I realized it is probably a US thing, and I strongly think it must be related to standardized testing.

The idea, I think, is that standardized tests are best because they are more scientific. But are they, really?

There is more to science than using jargon to disguise everything as math. We always say that science is about making predictions that can be tested by experiment. This is true, but it is not the whole picture. Testable predictions are necessary for good science, it will never be good science if it is contradicted by experiment or untestable in the first place, but it is not sufficient.

For good science, we need impressive predictions framed in a convincing explanation.

This is where I make the case for a somewhat controversial position: science is not just a compendium of techniques and methods, it is also a form of art: being impressed by predictions, being convinced by an explanation, they are emotions, and triggering emotions is the realm of art.

And this is something that is missing in many recent scientific disciplines. Take economy, one of my pet peeves. Economists will talk about utility functions, saying they are complex objects meant to encode all the richness of people's feeling about the resources available to them. But when it comes down to doing the math, it is too complicated, so they just take utility = price and run the models. The models are not wrong, but they fail at convincing that they match reality, that they are relevant.

A similar phenomenon exists in computer science when it comes to proving that programs are valid. They can do very strict semi-automated proofs that the navigation system of a plane, for example, correctly implements a specification. But it only moves the problem: does the specification really mean I am safe in that plane? In principle, the specification should be simpler than the implementation, and therefore more understandable, more convincing. But in the end, there has to be a human doing the understanding, being convinced.

And the same goes in social and cognitive sciences. There are many experiments in social and cognitive sciences, each proving — assuming they can be replicated — a tidbit about human behavior. If we look at many of them, we certainly see patterns emerge: these results are not completely random, they are the manifestations of deeper phenomenons. But what these phenomenons are, there is still no theory, no explanation. And therefore, social and cognitive are not convincing, not yet.

That brings me back to standardized tests. By eliminating subjectivity in the evaluation, they have the veneer of science, they pose as objective measurements. But the subjectivity is not gone, it never can be. With standardized tests, all the subjectivity has been pushed into that simple question: what are these tests measuring?

I am talking, in essence, of Goodhart's law, but it is more fundamental than that. Until we have the frame of a theory, until we have a scientific language to express our ideas in a rigorous way and act as a bridge between our intuition and objective observations, we cannot design experiments and measurements capable of guiding us towards understanding and conviction.

Which is why I believe, as of now, standardized testing as practiced in the US is inferior to what other countries do. I know especially well what we do here in France (full disclosure: I teach math in a public high-school in one of the poorest suburbs of Paris): the examination exercises are much more subject to the subjectivity of teachers, both when it comes to designing the exercises and evaluating their difficulty, and when it comes to correcting them and grading students, even in math, but I believe they are much closer to the informal ideal of testing what we would like to test.

What I am saying is that the nascent sciences of education are still very far from being able to compete with the intuition of experienced teachers.

And that brings me to another side of the issue: what is school really for? I have skimmed the comments, and I have seen quite a few takes on this. I will give mine, focussing mostly on the latest part of school, from 12 to 18 years old approximately.

The obvious answer: school is about the knowledge and skills that kids gain, is clearly wrong. Acing the test about Gaussian elimination does not mean you will be able to do it until the end of your life, or even two years from now unless you keep practicing, and it is not why the test is administered.

There is an example I use on the first day of every year with my students: considering there is no jumping rope in a boxing match, why are boxers endlessly training with one?

To that question, the students always know the answer: to build the muscles they need.

And this is, in my opinion, the greatest role of school: to build the muscles of the mind at a time where it is most plastic.

By learning Gaussian elimination, you did not just learn a math technique, you trained to follow a set of rules with utmost application, and that will help you in life when it comes to, for example, filling your taxes. Well, maybe not that much, since taxes are usually less logical than they should.

(This is why Gaussian elimination is not that good an exercise, and we do not do much of it here. I teach it, but not as a core topic. Usually, it goes “now that you've toiled moving x, y and z around so many fractions and found the correct coordinates for the intersection of these geometric figures, let me show you how you could have done it twice as fast and without fractions.” (Thanks to the metric system, students here do not need to be as fluent with fractions as Americans.) Also, we certainly do not call it “Gaussian elimination” to the students' face.)

I think the kind of math we do in France in high school is a better example, or at lease one I can explain better. It is mostly centered around demonstration: we have a situation, maybe a function, possibly modeling some kind of real-world problem, and students have to prove properties of this function: is it increasing? is it always positive? What it trains is the ability to think of every tiny detail that can ruin the proof: can I divide by x, or may x be 0? can I use the intersection of these lines, or may they be parallel? It may help them become good mathematicians later, but it may also help become better detectives or lawyers or security experts, because a hole in a mathematical demonstration is the same thing as a loophole in a law or a flaw in a security system.

In the end, to use words that are trending right now, tests about a specific topic are only a proof-of-work for this training.

But I think there is another very important side to school, and I have seen it mentioned a few times in the comments.

I have a friend who toiled into getting their car driving license for utilitarian reasons. And then, on a whip, they started on the motorbike license, and surprisingly they liked it, a lot; now they are making many rides just for the fun of it.

You cannot find your life passion if you never try it.

And this is why school, with its wide array of mandatory topics, is so important.

I suspect many people who end up commenting on a rationalist blog come from a background, starting with family, where knowledge was valued, where trying new things for the fun of experience was part of life, and possibly where parents and other mentors could introduce these things to them. It is not wealth, it is not a network, but it is a kind of privilege too.

And I can assure you, it is not true for everybody.

My students literally live a stone's throw from one of the greatest cultural center of the world, but most of them, left to their own devices, would never go to a museum or an exposition.

School is very important to broaden the horizons of kids, and it is very easy, from the top of the hill that is the life of an adult reading a rationalist blog, to forget that so many kids start with very narrow horizons.

Expand full comment

Perhaps we should remember the very large number of kids around the world whose schooling was severely interrupted 1939-1945? My father was too old and was off fighting, but my mother, and both my in-laws had their schooling disrupted to varying degrees with bombings and evacuations. No one regards the kids who should have been at school in those years as 'damaged' and my relatives all turned out well career-wise.

Expand full comment

I taught physics and chemistry in UK secondary schools for 15 years, and every year the majority of students entered Year 11 (of 11) knowing next to nothing of either subject. Many would only start cramming for exams at Christmas, or later. Most achieved respectable grades, albeit from teachers frantically force-feeding them with facts and coaching the language of exams. If schooling worked properly, there would be no need for this annual pantomime. This year would have been little different than any previous year, except it didn't happen. Exams were abandoned in favour of ongoing 'teacher assessment'. Hence, little last minute learning and the best results ever.

Expand full comment

A bit of personal experience:

We live in China, where learning to write is a much bigger part of the curriculum, because it's much harder. My 10 year old did grades 1&2 in an international school, where Chinese teaching was minimal, then switched to a Chinese public school. He's starting grade 5 now and still struggling to keep up, because his slow writing slows him down in every class.

I should point out that this is not an endorsement of the teaching that Chinese schools do. They mostly teach kids by being very negative to the kids who do badly to incentivize parents to provide/get whatever help the kids need. But that function - the enforcing very clear standards function - seemed valuable enough to us that we made the transfer to Chinese schools when the international school was not providing it well enough.

My story is complicated by a bunch of individual factors, of course - my kid is particularly bad at writing (the fine motor side of it), so where another kid might have caught up by now, he hasn't. But it did make me wonder if there are certain crucial skills that schools might be delivering. Because I agree with Scott's thesis for the most part, but this writing thing is really kicking our ass.

Expand full comment

During the pandemic a lot of kids sat at home bored out of their minds, wasting away on their laptops, melting into twitter and facebook, eating bonbons, and getting fat. They missed badmouthing their teachers and playing jokes on their friends, having tween relationships, and playing sports. No chess club, robotics, jazz band, or lacrosse. Many parents saw their kids wasting away so they ponied up the big bucks for private schools that stayed open. One of my elementary-age daughters got into Finnish hobbyhorsing through school. The school does expose kids to things they would not see at home. None of this is measured on tests...

Expand full comment

I'm surprised nobody brought up schools as civic religion. That is, societies used to have something like the catechism and mass as a default canon that everyone was expected to know and share in. We don't do that anymore (I mean, many people do, but we don't define our society by that anymore), and I think school tries to fill in that gap, so that instead of not knowing your Old Testament prophets you would not know e.g. your US presidents. (My apologies if I misrepresented what is involved in the catechism.) It's not objectively essential for most people to know the list of US presidents, or the list of British kings, or the list of Chinese emperors; but knowing or at least pretending to know it is a marker of belonging to that culture.

To offer a silly US example, it is surely not imperative for anything to have played Oregon Trail as a kid, but most of my generation did, and not having done so marks me as an outsider. Obviously I can fix this particular deficit, but it's one of rather many; I come from a different school canon. Modern, liberal, nerd, US, society is very permissive of it, but I imagine that's less true in other social circles.

----

I think a common thread in many objections to Scott's post is "yes you can fill in any one thing you missed, but it doesn't follow that you can miss all the things simultaneously and expect to fill them all in". Stable systems tend to have some redundancy by design; the question would seem to be exactly how much of it you need in education, and whether we have too much of it.

Expand full comment

There are scenarios where schooling would make a big difference, especially in 12th grade, but even before that. If a student wants to get into a difficult field, like advanced math, they will need to build each year on their previous knowledge. If they lack the previous knowledge of, let's say, 9th grade math, then they may struggle more with 10th, 11th, 12th, and then college. Additionally, as you mentioned, if you are trying to use the [at that point fresh] knowledge in order to accomplish a task such as college admissions, then getting that information is vital during a time sensitive period.

I think this is directly supported by your first section. What information do we hold onto? The parts that we use. If we have to artificially "use" that information to keep it relevant (as with foreign exchange programs to learn a language or mid level math to learn advanced), then we depend on those programs. There were probably only a tiny fraction of foreign exchange students learning languages over the last 18+ months. That likely will have a huge impact on their ability with that language. Maybe they would have lost that ability over time anyway, but if they wanted to go to college to study Spanish, that immediate loss would be huge in terms of college applications and ability in the program.

Expand full comment

1) This post is very interesting and important (and I mostly agree), but I must note that "Steelmanning The Case Against Missing School" has to include a thorough critique of the methodologies of the studies you started with (Benezen, New Orleans, Potonchik, cancer etc.), as critical as you were with the following studies. Else, why should we believe you that one set of studies is less problematic than the other?

You seem to treat the first set as truth, and then interpret the other studies starting from that assumption.

2) Re: homework and other pedagogical tools. I'm ~familiar with studies regarding their efficacy, but also with personal experiences and those of teachers who insist that these tools are useful and are there for a reason (and these are intelligent teachers who actually actively optimize for their students' learning success, and remove things they deem unnecessary). While the discrepancy may be mostly (or even all) teacher's cognitive bias, I think there may be an effect similar to the discrepancy between global clinical trials of psychiatric drugs and psychiatrists' personal experience prescribing them - i.e. the human interaction provides subtle information and differences between people that global experiments miss.

Specifically about homework, though, it seems to me that class work is better than or equal to home work, but some work is better than no work at all, and teachers are logistically more limited in how much class work they can assign (limited time that has to be dedicated to frontal teaching). Idealistic experiments often compare homework to equivalent classwork but teacher's day-to-day is choosing between homework and no work.

Expand full comment

Education differs dramatically in the US between elementary school and middle school. In elementary school, all the students tend to be lumped together in a single class with a single teacher and you're taught all the subjects together.

Once you get out of elementary school, you start getting specialized classes that teach particular topics.

What I think is really going on:

1) Most of elementary school is really about making sure all the kids have the basics and like, know how to read and write, add and subtract and multiply and divide. Everything else is setting the groundwork for further repetition and reinforcement in later schooling, but you can pick this up elsewhere. Your parents can teach you all of these things, so the number of students in first world countries who get screwed here is basically the children whose families are totally illiterate and who can't pick it up otherwise, which is heavily skewed towards disadvantaged minorities.

2) Middle school and high school have you learn in tracks, where you build on previous knowledge. In reality, though, liberal arts mostly don't actually build up in this way; they're mostly about reinforcement. As such, what actually matters is learning math and science. If your parents can teach you this stuff, you're fine; if they can't, though, you're probably screwed if you miss out, because you end up permanently behind.

3) Schools generally teach as fast as the bottom few students learn, rather than as fast as the top few students learn. As such, it will mainly impact the bottom students because the rate of learning is so slow, and they can speed it up and just screw over a few students at the bottom.

As such, I'm not sure if elementary school studies are even useful, and it's possible that a lot of these effects are just "people catch up students who are behind, who can catch up because the speed of teaching is so slow".

There's some other issues with your analysis as well.

For instance, you're assuming that all kinds of absences are created equal. But what if they're not?

In the case of a medical absence, there was some big issue and the teachers know about it and they go out of their way for Little Timmy the Cancer Patient to catch up. In the case of snow days, everyone was gone, so time to catch everyone up! Disaster? Everyone was gone, catch them up.

The only way you lose out here is if you are graduating close enough to the disaster that it is a problem. So people who graduated from 12th grade last year will lose out more because they never had the chance to get caught up, while people who missed, say, 7th grade, they have a lot of years to catch up in.

But if you're just randomly gone a bunch of days, they don't catch you up. You're just missing out on that content entirely. So this hurts you a lot more, because the teachers don't fix things for you.

As such, I'd expect that random absences will matter a lot more than big ones like medical issues or disasters, so these probably aren't actually directly comparable. And I'd expect that later gaps matter more than earlier ones - missing two years of high school will matter a lot more than missing first and second grade.

If students actually were taught stuff as fast as they could learn it, I suspect that absences would screw students a lot more. But they don't.

Expand full comment

Someone may have said this elsewhere in the comments, but "missing school" for extended periods isn't really an option. You're not allowed to not school your kids. You can homeschool or unschool, but that's also currently a worse-than-usual option with stuff like libraries and homeschooling cooperatives not operating normally.

Younger kids are lonely and bored after a year of playing with Legos while their parents tried to work. You could take time away from work to homeschool (and some families do manage to homeschool while working full time). You could hire a nanny or tutor to do stuff with them. But you can't just do nothing.

But yes, during the extended freakout among parents in my child's first grade, I kept chanting "In Finland they don't start academics until age 7."

Expand full comment

A poster told me that all I really need to know I learned in kindergarten. Sucks for anyone who was in Kindergarten this last year, otherwise chill out.

Expand full comment

Kind of feels like all those "we should have scientific cost benefit analysis of shutdown" types spanked pretty hard.

I approach this from a slightly different angle. When the UK shutdown, there was a brief scare that domestic violence would spike and initially there was some surge in calls to the hotlines, but pretty fast calls returned to normal levels even though people were stuck at home with each other more. What happened?

Pretty simple, IMO, if you are the type of man who can't avoid beating his wife unless you have some 'out' to go 8 hours most weekdays, what will happen if all in the sudden you have to be home almost all the time because of a shutdown? Well what happens is I'm sure intutition is right, in some cases violence will happen when it wouldn't have happened before. Other cases, though, people said something like "since going out half the day is no longer an option, I either have to fix the situation or exit it now". Families that would have remained in a borderline situation either broke up or did what they needed to do to resolve their issues. The hotline reverted to baseline because violence that would have otherwise not happened was offset by violence that did not happen but otherwise would if the status quo had continued.

In other words, cost-benefit does not correctly account for compensation.

Expand full comment

So we have the conclusion for parents, which seems to be "don't worry too much".

What about the conclusion for teachers?

If you were a middle school teacher (in a very good school), what would you do differently having that information?

Expand full comment

A note on the hours of schooling vs. PISA math scores: I've read that in certain Asian countries that do well on the PISA (i.e. Korea), students don't spend a lot of hours in school, but they do spend a lot of hours in after school tutoring. Not sure how much that confounds this metric, but I'd suspect in places where students spend more hours in school (one-on-20/30) parents will be less inclined to send their kids to additional tutoring (often 1:1). Not that class size reductions have a strong argument for improving performance, but at a certain point along that curve you should expect some benefit.

Expand full comment

>And I think that’s the whole point. We learn lots of things in school. Then we forget everything except the things that our interests, jobs, and society give us constant exposure/practice to. If I lived in Spain, I would remember Spanish; if I worked in math, I would remember what Gaussian Elimination was. I think a lot of the stuff you’re exposed to and interested in, a sufficiently curious child would learn anyway; the stuff you’re not goes in one ear and out the other, hopefully spending just enough time in between to let you pass the standardized test.

The argument "because we forget much of what we learn in school, school isn't important" proves too much. I forget the content of most books I read years later. Is it not important to read books? No, I say -- those books affected me and had some small impact on my character and thinking. I have forgotten a lot of math since college, but learning the mathematics was still valuable because it taught me how to think mathematically.

Same applies to sports. Most kids won't end up playing professional sports, so should they not bother playing while young? I actually think that competition is important -- kids learn what they can and cannot do.

I think kids need to have some kind of structured learning. Something where they get to compete with themselves at learning something, something that ideally challenges them. Even if they forget all of it, they will hopefully remember the process of learning. Supposedly Yeats said "Education is not the filling of a pail, but the lighting of a fire." At its best, school inspires curiosity and a desire to learn more.

Expand full comment

I have long maintained that schools have a clear favoritism to girls, who make the honor roll twice as often and high honors three times as often. Boys drop out are far higher rates. That's some patriarchy you got there. Schools are designed by women for girls. (As a side note, I think much early feminist anger sprang from being told that absolutely killing it at school was the key, doing that, and then finding that the out-of-school world played by different rules. Betrayal. Unfair.) As conscientiousness is the Big Five trait where girls have a demonstrably greater strength, it is therefore likely that conscientiousness is one of the main things being taught. I was intrigued that you mentioned it in passing.

That is a very good thing to teach, and if schools taught nothing else they would still have some value. But it's not the only thing. I have five sons, three adopted, two of those foreign, and our experience with public school, private school and home school is varied. I am now convinced, a couple of hundred thousand dollars later, that it's mostly genetics for the academic side, but school has value for the "adapting to life with all its unfairness and randomness" aspect.

Expand full comment

Sorry if someone already sent this, haven't time to read all comments, but I'd be optimistic about enough between country variability in shool closures for comparisons vs. that country's results in previous years to be made. See https://twitter.com/SilverVVulpes/status/1401866064691183616

Expand full comment

I feel like it's possible the teacher strikes and Bad Education outcomes are both caused by the same thing rather than one causing the other

Expand full comment

So do I take

"A kid who misses all of 9th grade will ... be less than 5 percentile points behind in reading (60%)"

to be the same as

"40% of kids who miss all of 9th grade will ... be at least 5 percentile points behind in reading"

or am I misinterpreting you? Phrasing it my way strikes me as somewhat more concerning.

Expand full comment

THANK YOU!! I keep telling my libertarian friends that you cannot for decades complain that the public educational system does a LOUSY job educating children (25% functional illiteracy at high school graduation, for example) and SIMULTANEOUSLY cry out that if we can't get the kids into these same schools their education will be horribly and negatively affected.

Expand full comment

For the last several years, we've been tutoring a boy who missed several years of regular schooling because his alcoholic mother melted down and he was caught in the middle of a cross-state custody battle. He has made up a lot of ground, but he has also been attending a fairly good school and had two well educated tutors helping him.

His biggest problem was in mathematics. The problem with math is that so much of it is cumulative. Even when you are taking advanced analysis in graduate school, assuming you bother to do so, you will still need first grade addition and subtraction. He was in middle school and taking algebra, but we spent a lot of time going over stuff he had missed like the multiplication table, prime numbers and fractions. It is easy to see how someone could miss a chunk of the math curriculum and never recover, not because the material was impossible to learn, but because there was mechanism to teach it.

When I was a kid, I somehow missed whatever my school taught about multiplication and division. My parents, both college educated, made me flash cards, taught me the various algorithms and got me up to speed in a few weeks. If I had not had educated parents or if no one had caught the gap, I could have foundered.

More recently, my niece had a similar problem. She had never learned how to read critically, that is, analyzing what was written and making appropriate inferences. She did terribly on her SATs, so we wound up tutoring her. We gave her a big dose of 19th and early 20th century literature and got her up to speed.

So, a lot depends on what happens when school starts up again. Is the kid simply flunked out, dumped into some awful remedial course, taught at an appropriate level, or given personalized tutoring? Is the problem that led to school absence remedied or merely ameliorated? I think the people most worried about their children missing school due to COVID are the ones whose children are least likely to be affected. Odds are their parents will do the necessary or see it done. It's the children whose parents don't have the resources, the background, the culture or the priorities that are likely to have problems.

Expand full comment

When I first moved to the Dominican Republic I imagined myself holding classes for these poor less educated kids here. Well, that never happened.

What did happen is that I allowed them to do chores and bought them smart phones. Excellent budget phones like the Moto G and Redmi series. Not all at once. I'm certainly not rich. But I've purchased phones for about six kids now.

They are quite familiar with these phones and navigate faster than I do.

It dawned on me that being completely at ease with their smartphones probably has more to do with their future employment than having perfect grammar. The girl who didn't read and write well is quite adept with speech to text.

And "my" kids are not the only kids with phones. In fact they all seem to have them!

Expand full comment

Well, "Kids Can Recover From Missing Even Quite A Lot Of School", so, OK.

I'd agree with that, however, (at least) two questions arise; "Why?" And "What exactly is a lot?"

The "kids can recover" is based on future grades/marks, or subsequent earnings, and is presented as "catching up".

Here's three things; chronological age, developmental age and school age.

Chronological age is determined by your birth date (a statement of the bleeding obvious if ever there was one). Developmental age is a sequence of milestones passed at a chronological age.

So, crawling, walking, talking, not wetting the bed. That sort of stuff, for infants.

Chronological age is not necessarily the greatest specific predictor of developmental age. It roughly works, but can be really rather vague - "by about 18 or 24 months".

Note that the difference there is six months, being a third of 18 months, a quarter of 24.

School age (or school year, grade for the Yanks) is an artifact. The underlying assumption being that chronological age is a good predictor for developmental age, continuously, for the entire duration of formal education, for some large majority of children.

Testing, exams, take place at specific times during the school year(s). Notionally, they're measures of information, knowledge, facts received and retained, and abstractions successfully manipulated. Scott says this - that test scores, marks, are partially dependent on when the test is taken, and differences in ranking between children is due to the mis-match between the three ages.

So, just wait. For some children, delay delivering the information, for some period, and they magically catch-up with others as they get older and reach an equivalent developmental age.

And the system does, kind of, have at least one way of handling this, within each specific school year, or measurement period. Revision.

Finish elements of the curriculum early enough, to spend time going over the information again, ahead of the test. Some children who were struggling earlier, suddenly get it. Some of those, simply got older.

Where subjects might be interdependent, say maths and physics, you can get the same effect. Being taught something in Maths, and then having to use it in Physics two or three weeks later. Where subjects are not, English (other languages may be available) and, I dunno, Geography, you might have a bit of a problem, and rely on repetition alone during the teaching phase.

And there are birth date effects; "When you are born matters: the impact of date of birth on educational outcomes in England" (https://ifs.org.uk/wps/wp1006.pdf) or "Season of Birth Effects and the Role of Emotional Intelligence: A Review of the Literature" (https://www.cambridgeassessment.org.uk/Images/109784-birthdate-effects-a-review-of-the-literature-from-1990-on.pdf).

So, in the UK, the school year runs from first week of September into about the second week on July the following year. The cut-off for birth dates are 1st September to 31 August.

This means that a child born in early August can be in the same school year, attempting to learn the same things, as a child born in early September of the previous year - with the older child having an apparent developmental advantage of nearly twelve sodding months.

At age 6, that's really quite a large number of months, between the relative chronological ages. But as they age, that relative gap shrinks, but the apparent disadvantage persists.

Best guess; the current definition school age is too long compared to chronological age, being a vague proxy for developmental age. So developmental differences persist throughout the education system.

One solution - increase the granularity of school age, by dividing into quarters. Pupil intakes take place four times per (current) school year. 1st September, plus December, February, and May.

Exams take place four times per school year, once per quarterly cohort.

If the idea that the mis-match of school age and developmental age causes a significant difference between test rankings is correct, then the range of grades awarded, results achieved, should narrow when measured at chronological age, and could be very narrow indeed at school age.

At this point, I wouldn't like to speculate about what the wider effects might be, 'cos that seems like far too much work.

As to "what's quite a lot?" - I haven't a scooby. But altering the calendar, would most likely change the answer.

Expand full comment