Ep 45. A decade of decline: Examining falling test scores with Mike Petrilli

This transcript was created with speech-to-text software. It was reviewed before posting but may contain errors.

You can listen to the episode here: Chalk & Talk Podcast.

Timestamps:

[00:00:00] Introduction

[00:03:18] Discussion on standardized testing

[00:06:14] Criticisms of standardized testing

[00:12:25] Frequency and levels of testing

[00:20:04] International perspectives on testing

[00:23:11] Understanding NAEP

[00:28:07] Recent trends in NAEP scores

[00:28:41] Impact of the pandemic

[00:31:16] Pre-pandemic declines

[00:32:26] Comparing international trends: PISA scores

[00:33:08] The role of smartphones

[00:35:26] TIMSS scores

[00:38:16] Long-term impact of declining academic achievement

[00:39:20] Efforts to address learning loss

[00:42:12] States that bucked the trend: the “Mississippi miracle”
[00:46:44] Low-performing states

[00:49:54] Future strategies for education systems

[00:53:22] Influencing educational policy

[00:55:49] Conclusion and final thoughts

[00:00:00] Anna Stokke: Welcome to Chalk & Talk, a podcast about education and math. I'm Anna Stokke, a math professor and your host.

Welcome back to another episode of Chalk & Talk. In this episode, I talk with Mike Petrilli, President of the Thomas B Fordham Institute, which is known for its focus on research analysis and advocacy related to K to 12 education. Mike is an expert on education trends, particularly test scores, which is our main focus today.

We discuss the role of standardized testing, exploring both its benefits and common criticisms. We have a detailed discussion about trends in NAEP scores. That's the National Assessment of Educational Progress, which is a test in the United States. We talk about how it's conducted and what the latest declines in scores indicate about the state of education.

We also talk about international scores on PISA and TIMSS. We highlight states that have defied the downward trend like Mississippi and discuss what can be learned from their success. To wrap up, Mike shares some valuable insights on how to influence education policy. This was a fascinating conversation—I learned a lot, and I hope you do too. Now, without further ado, let's get started.

I am pleased to be joined today by Mike Petrilli. He is president of the Thomas B Fordham Institute. For listeners, Fordham Institute is a nonprofit education policy think tank that focuses on research, analysis and advocacy related to K to 12 education in the United States. He is also a research fellow at Stanford University's Hoover Institution and Executive Editor of Education. He's an award-winning writer and an expert on evidence-based practices and trends and test scores and other student outcomes, which we're going to talk a lot about today.

His writing has been published in the New York Times, Washington Post and Wall Street Journal. He helped to create the US Department of Education's Office of Innovation and Improvement and the Policy Innovators in Education Network. And he is the host of the Education Gadfly Show podcast.

Welcome, Mike. Welcome to the podcast.

[00:03:15] Mike Petrilli: Oh, Anna, it's great to be on. I really appreciate the invitation.

[00:03:18] Anna Stokke: You write a lot about standardized tests and trends in test scores, so I thought before we got into specifics that we'd have a broad discussion on standardized testing. Now, I gather you think standardized testing is important.

Can you elaborate on that, and can you explain for listeners why standardized testing is important?

[00:03:38] Mike Petrilli: Sure, sure. It’s true. I am probably one of the world's biggest fans of standardized testing. I have gone as a number two pencil, for Halloween a few times just to embarrass my children. I also understand why, why so many people, and especially educators are not big fans of standardized testing.

There's no doubt that the way that it has been used in some places, we have gone overboard with it. I know that there's a lot of concern about teaching to tests and especially if those tests are not very well designed, and don't encourage, you know, high quality teaching. That said, it's important to know how well schools are doing in helping kids master the important knowledge and skills that we know they need to master.

And when you ask yourself, well, how can we do that? How can we get good measures? You end up with something that looks like a standardized test? It's something that is valid and reliable at scale, unlike say, teacher grades, which also are super important and can tell you a lot of information about how kids are doing, but they're not standardized and we know there's all kinds of other issues with them.

So the bigger question before you get to testing is really just why does it matter that kids master reading and writing and math and science and history. And here, I think we know from decades of research that what you know, is going to give you a, a big edge up in terms of how far you're going to be able to go.

So you learn and eventually, someday you learn to earn. We all know that, that some of this is nature. Some of this is nurture. You know, schools don't have control over everything, but it is certainly the case that schools can make a big difference in how much kids learn and whether they end up at the, let's say, at the 20th percentile, or the 50th percentile, or the 70th percentile in any domain, and where they end up and whether they achieve their full potential is going to have a huge impact on the choices they're going to have as adults in terms of employment, in terms of participating fully in our democracies, you know, we know that there's a relationship between student achievement learning, and other important outcomes like getting married and staying out of the criminal justice system and on and on.

So it's super important. Now it's a measure, and we have to keep that in mind. And sometimes I know in our enthusiasm we forget that, but they're measuring skills that really matter, and they're measuring skills that are malleable. This isn't just looking at, “Hey, was this child lucky to be born brilliant or not?” This is the sort of thing that schools do have not full control over, but some control over how much kids are learning.

[00:06:14] Anna Stokke: So you mentioned a few things there: how much you know is important and we want to make sure that kids are actually learning stuff. And I see that, you know, as really critical in a subject like math because it's so cumulative and if you miss out on stuff, it's really going to impact how you do later on.

And also just as, as you mentioned later, income and, and your quality of life, et cetera. So let's talk about some of the criticisms of standardized tests, because there are a lot of them. You mentioned some of them already. So a big one is that standardized tests encourage teaching to the test. So how do you respond to that concern?

[00:06:56] Mike Petrilli: It's a real concern. And I remember when I first got into this, back in the No Child Left Behind days here in the United States, you know, there was this wave of policies at the state and then federal level to have annual testing. And those first waves of tests, they were not very good in reading and math.

They tended to be very low level, kind of multiple choice, fill-in-the-blank. Uh, just pitched at a, you know, the whole system was set up to make sure that every kid learned at a very basic level. But, you know, quickly, educators figured out that they could teach to those tests pretty effectively.

A lot of, you know, what people would say of drill and kill and not necessarily inspired teaching, and you could move the needle. And so that was a problem, like that was a legitimate concern.

Now, the good news is in the United States at least we moved away from those kinds of tests. For the most part, the current crop of testing is, is much better. It tends to be pitched at a higher level. It's richer. It's more like the, some of the kinds of exams that we see on the international stage or our national assessment of educational progress. Now, I think you can say to educators with a straight face that, “Hey, if you're teaching to this math test. You know, the only way you can do that effectively is just to teach.”

Well, you know, the tricks and the gaming and the drills, uh, that's not going to get you very far. I mean, sure, a little bit of that. So kids are familiar with the format might help a little bit, but really you got to teach the content and you know, in, in our systems we have, advanced placement courses in high schools that have been well respected for a long time.

Or you think about the, I don't know, the A levels overseas or if you think about some assessments that people respect, they tend to be the kinds of things where you're only going to do well on them if you've actually mastered the material. That's what we should be aiming for.

[00:08:43] Anna Stokke: I would add that a lot of times teaching to the test could be a good thing.

So for example, I go on a lot about things like basic skills like times tables that you know, it's really important to have your times tables memorized. That should be an easy thing to check with a test. So in that case, teaching to the test is probably a good idea. The main point I think you're making is that it has to be a good test, right?

You can have bad tests, in which case teaching to the test is just not a good thing to do. But if, if you have good tests, then teaching to the test might be a good thing. It ensures that everybody's kind of on the same page.

[00:09:19] Mike Petrilli: That's exactly right. And to be clear, and I don't want to denigrate the basic skills in math for sure.

[00:09:25] Anna Stokke: I wouldn't let you.

[00:09:26] Mike Petrilli: Yeah. Look, I think there's a big problem here in America, and I'm not sure how it's playing out in Canada, where, you know, we came out with this new set of math standards now 15 years ago, and there was a perception both from supporters and opponents that was meant to move away from a lot of the basic skills.

Well, it wasn't true. They were in there. Kids were expected to know their addition and subtraction facts and their times tables. Now, they also were asked to then be able to think conceptually. So at a, you know, at the right point as they got older, to be able to explain why is it that five times seven equals 35?

You know? But they also needed to know those numbers with automaticity. And we did a terrible job explaining that. And so there were a bunch of places out there that thought, oh, we're not supposed to teach times tables anymore. We're supposed to downplay the basic skills stuff, which was a disaster. So that piece is, is super important.

And it's just that you don't want it to only be that, especially as kids get older and even the basic skills stuff, even the times tables, there's, there's good challenging ways to be able to test that. And there's other ways. So the, the quality of the tests really do matter.

[00:010:30] Anna Stokke: Okay, so let's talk about a couple other criticisms, and this is a big one.

So this is the one that I hear most frequently hear where I live, by the way. So a lot of people do argue that standardized tests, disadvantaged low income students and students of color. So what are your thoughts on that?

[00:10:48] Mike Petrilli: Again, I think that this is, one of these classic areas where we're mistaking correlation for causation.

It is true that these big achievement gaps show up on tests. Racial achievement gaps, socioeconomic achievement gaps, and it's painful. I mean, this is, what a lot of us have been spending decades trying to do is to try to work on narrowing those achievement gaps by helping kids of color, low income kids do much better on these assessments and to learn more. But again, we don't want to shoot the messenger and the test is the messenger. I think there was a time when you can make a case that there were some tests that had some questions on them that were unfair to kids who maybe were growing up in poverty.

You know, the classic examples, if you're asking questions about sailing or playing polo or something that, you know, some Richie Rich kids would be the only ones who would understand, uh, what that was about. Of course, that doesn't make sense, but at this point, look, tests are, you know, scrubbed every which way to Sunday to make sure that there's no bias.

Then they have other ways of testing to make sure that different groups of kids, you know, don't perform systematically different on those different questions. So I think this is, again, one of those issues that maybe was a problem once upon a time. I don't think it's a problem anymore. I think when we see those achievement gaps, it's real.

Unfortunately, it means that we are not getting the job done on helping in America, especially black students, Hispanic students, learn as much as their peers, and that's where the action is. We've got to now really address the problem, not try to just sweep it under the rug.

[00:12:25] Anna Stokke: You can't fix what you can't see, is what I always say.

So you have to see where the problems are in order to address them.

I have a question about how they should be conducted. So do you think standardized tests should occur at the national level, the state, or in the case of Canada, the provincial level, or a combination of both?

[00:12:51] Mike Petrilli: I'll speak about the system I know best, which is here in the United States. We do have testing at all levels, and that's part of the problem is people feel like there's too much testing at the national level. We have this national assessment of educational progress. It's a sample, so it works kind of like you would for say, the TIMSS and some of the other international assessments, so a sample of kids in every state. In some of the larger districts that tells us trends on how kids are doing.

We can look over time, how different cohorts are doing, how different states are doing, break it down by different racial groups. That is super important to be able to get the big picture, though it leaves out a lot. You know, because it's a sample, we don't have individual-level data. We're not following individual kids over time.

And so, a lot of the more sophisticated studies, you can't really use those data for. So then we have, at the state level, the state tests. Where everybody is tested, everyone in public and charter schools, grades three through eight. Once in high school reading and math, and science, a little bit less frequently.

And because of all of that testing, you really can start to do some more sophisticated things like ask: Okay. In a given elementary school, which schools are helping, uh, their kids make a lot of progress from one year to the next. And let's then measure the effectiveness of the schools based on, on that computation.

And I think that's super important as well. And then a lot of local school districts, or schools then choose on their own to buy their own tests from some on the national market. It's sometimes because they want to know throughout the year if their kids are on track to do okay on the state test or because they want to give information back to teachers that's more timely and, and that they might actually use as instruction maybe to say, put a reading or math groups together, or to remix them or to, to have another check.

It's a lot. And I do think, you know, if we were starting from scratch, we could find a way, I think, to make it less burdensome. But there's also just the fact that they serve different purposes, these different tests. You know, the, the national and state tests are mostly to give policymakers on behalf of the public a sense of what the trends are and which schools are getting it done, and which are not, you know, which schools need more help and which do not, what decisions should they make in policy and, and around spending very different needs than a teacher who wants to have data that they can get back on a timely basis to adjust their instruction, adjust their strategies, mix up student groups, you know, and, and the like.

So there are some people, some of my friends who have this dream of being able to somehow merge all of these different tests together into one test. That would serve all these needs and, and have, uh, you know, less testing. I get it. I just think it's really hard because I think there's such different purposes.

[00:15:47] Anna Stokke: We'll get into this a little more and we'll talk about the NAEP in in a little bit. I just have a couple other questions just in the broad sense. If you were advising, what do you think would be the ideal frequency for standardized tests? Like would it be every year or would it be like 3, 6, 9, 12? What would it be?

[00:16:07] Mike Petrilli: I will say that in, in America, it has been a hugely positive step to be able to measure school quality based on how much students are progressing from year to year. So having that annual testing is critical to that. Now, probably smart people could figure out how to still make that work with maybe a little bit less testing, you know, so maybe if you have the right grade levels, for example, maybe you can only test in reading and kindergarten, second, fourth, and sixth. Then you test in math and you know, first and third and fifth and seventh, and then maybe that gives you enough, you can still get a pretty good read on the progress kids are making over time. It's just maybe two years instead of one year.

So I'm getting in the weeds here, but so you know, if you can, if you can figure out a lighter touch with that sort of approach, I think there's something to it. But I think less and more than two years, you're just not going to be able to use the data to look at progress. Over time and progress over time is where the name of the game is. If you, if, again, if you want to measure the effectiveness of the school or if you want to measure the effectiveness of any programs or that you've put in new strategies, new curriculum, you know, you’ve got to see where Tommy was in the third grade, and we want to know where Tommy was by the fifth grade and what did that, progress look like?

And was that, stronger or weaker than we would've seen without this new. Whatever. And so if you get to a point where it's very infrequent testing, you know, you kind of have a little dipstick into the school, you'll, you'll know something. But there's a lot you won't know 'cause you can't track individual kids.

[00:17:42] Anna Stokke: I mean, this is interesting to listen to you talk about that. So I'm going to just tell you a little bit about what it's like here. I don't know how familiar you are with the Canadian system, but it's, it varies across provinces. Education is provincially controlled in Canada, and so for instance, in my province there's one standardized test and it's at grade 12 and that is it.

There are sort of these like teacher professional judgment tests closer to the beginning of the year at grades, I think three and seven, but those don't really tell you much, right? They're very subjective. So we just have the one test at grade 12. A lot of people would like to get rid of that. I think it's just crazy to just have one test at the end of a student's K to 12 career, like sure, maybe it helps to determine whether they're ready for post-secondary, but you need to see what's going on before that to sort of catch the problems in the system.

[00:18:46] Mike Petrilli: Absolutely. And tell me, is that test, do the students have to pass that test in order to graduate or what?

Are there consequences for the kids?

[00:18:54] Anna Stokke: No. And so that test is worth 20% of their Grade 12 grade. That's it.

[00:19:01] Mike Petrilli: Gotcha. Okay.

[00:19:02] Anna Stokke: And I'll tell you more about how the results are made available to the public. They're just published as general averages. That's it. They're not published by school. They're not published even by school district.

[00:19:16] Mike Petrilli: Interesting. No, I mean, look, that's, I can imagine some listeners in America would say, wow, that sounds great. Look, I'll get rid of all this testing, but it also sounds like you're really flying blind, right? Like how do leaders in Winnipeg or taxpayers know if the schools are doing a good job or not?

At least here on this side of the border, everybody thinks their own school is doing great. Maybe we think there's a problem with the system as a whole, but we love our own kids' school. But you know, unless you have objective data. To look at and to say, okay, how are we doing? Especially compared to kids, other kids, schools serving kids similar to ours.

And how are we doing over time? You just don't know. You don't know that you have a problem, and you could have enormous problems and pockets where kids are learning very little and, and you just wouldn't have no idea.

[00:20:04] Anna Stokke: Yeah, exactly. That's what it's like. And then when the scores do get released at grade 12, you know, people do tend to get kind of upset because they're not that good.

And when they did try to get rid of the grade 12 test, I will say that parents actually really advocated to put it back and they, they did put it back, but it's just been so long since we've had any more testing than that. People are just used to it. Actually, I should mention, we do have a national test called PCAP, and I can't remember what that stands for off the top of my head.

I think it's every four or five years. So similar frequency to the PISA test, so I should mention that. So anyway, it's interesting to talk about what's going on in in other places, but in general, like, do you know, do successful systems tend to implement some forms of standardized testing like around the world?

[00:20:57] Mike Petrilli: It's a mix. I certainly think that America is an outlier on the high end in terms of the amount of testing we do, and also an outlier in saying that it hasn't turned us into the best system in the world by a long shot. You make a case that especially in the, in the nineties and the 2000s, we did make significant progress and that the testing was a part of that less so, more frequently.

Look, testing obviously has been a big part of the cultures of many of the East Asian countries, many of whom perform quite well on international assessments, right? South Korea and Japan, Hong Kong, when it's been tested as, as its own country, likewise with Taiwan, and those are places where, again, in the culture, there have been high stakes tests for years, decades, centuries, I think in some cases.

So it is a part of the culture and the pressure in those places, my understanding is, is really on the student in those places. The tests matter a lot for what your chances are in terms of where you get to go to college and what kind of career you're going to have, you know, in the United States, again, most of this tests, testing is really meant to measure the quality of schools and put pressure on the educators.

There's still some testing in the high school level that matters for kids, but much less so. I think though there's, you know, anywhere around the, the world where you want to see strong results. I do think systems that are clear on what they expect students to know and be able to do and have some mechanism for measuring that, you know, get better results.

You know, the United Kingdom has a great system of inspections, the school inspectorate, where they're able to send people in and look around at schools on a regular basis. And so you're not just using test scores, but to try to evaluate the quality of schools, know where there needs to be an intervention.

It's expensive. It's more hands-on, but it's another great way to think about measuring the quality of schools. And I think where we don't see progress, it tends to be places where there's just not a lot of measuring going on of school quality. What get measured gets done.

[00:23:11] Anna Stokke: So let's talk about NAEP. You mentioned it already, so just remind me again, what does NAEP stand for?

[00:23:17] Mike Petrilli: Yes, the National Assessment of Educational Progress.

[00:23:20] Anna Stokke: NAEP is administered yearly, right?

[00:23:24] Mike Petrilli: Every, every two years. Every two years.

[00:23:26] Anna Stokke: Oh, every two years. Okay. And it's grades four and eight, is that correct?

[00:23:32] Mike Petrilli: Yes, grades four and eight, reading and math. Occasionally–and it's kind of on its own schedule–grade 12 and occasionally other subjects as well.

It kind of depends on the budget and what, what they can afford.

[00:23:45] Anna Stokke: And you mentioned that it's actually just a sample of students, is that correct?

[00:23:49] Mike Petrilli: Yeah, that's right. And this is a design that goes way back and has been respected for a long time. For example, I think the national sample, it's something like 2000 students.

So every, every time they do this, they, they randomly select a number of schools from around the country, where they ask permission to test the kids. And yeah, those kids are tested, but it doesn't count for anything. Likely is if, if you've been chosen one year, your, your school may not get chosen again for another 10 years.

Now, I should say, it is also in every state of all the 50 states. So it is the case that there are some of the, the very low population states, such as near you, you know, up in our Great Plains where probably they do actually have to test quite a few of the kids every year or every, every time because they have to get a big enough sample.

If you're talking about Wyoming, there's just not that many kids to begin with, so they, they probably do experience this NAEP test more than in say, California.

[00:24:44] Anna Stokke: And then the individual states tend to have their own standardized tests as well. Do they all have standardized tests?

[00:24:52] Mike Petrilli: They do, and they, they all have them because they are required to by federal law. So this goes back to the federal, well, basically the No Child Left Behind Act, which was a big law in the George W. Bush era that was then replaced by the Every Student Succeeds Act, but but still has that same requirement. They have to have standardized tests. Grades three through eight, reading and math once in high school. So they've all got them.

[00:25:17] Anna Stokke: About the NAEP. Who writes it? Who designs it?

[00:25:22] Mike Petrilli: Right. So there's a group called the National Assessment Governing Board, or NAG-B. We do love our acronym acronyms. Uh, and it's basically a blue ribbon panel of people from around the country. I think about 25 folks that serve on that board, a mix of educators, uh, content.

Experts, uh, different policy makers. So they've got some state legislators, they've got some state superintendents, you know, there's, requirements to make sure there's some Democrats and some Republicans and, otherwise a, a mix of people. So it's meant to be a real blue ribbon honor to be on this committee and to, to represent this sort of broad diversity of America and look unlike some other boards this one actually has, has worked pretty well over the years. You know, they've taken their work seriously and, and they go through processes to look at the test design, to make updates, you know, to try to keep it current if there's new approaches that are common out there, while also trying to keep some consistency so you can, you know, compare from time to time.

[00:26:26] Anna Stokke: Okay, interesting. And who marks it?

[00:26:31] Mike Petrilli: And then it's a contractor. Yeah. That they, the government hires out this to a contractor. I think this one goes to the American, American Institutes for Research, but I'm not totally sure. One of the big testing companies. So yeah, there's, you know, and some of this is now can be done by AI and some of it's, you know, 'cause increasingly they have moved these tests, to be online. So kids are taking them on Chromebooks and the like.

[00:26:54] Anna Stokke: It’s about as objective as you could get. And then in terms of the questions, are they based on the common core curriculum or, or how do they decide what the questions are going to be like?

[00:26:65] Mike Petrilli: Yeah, no, it's a good question. So, you know, this is where they have this process where every decade or so, they do go through and look at their frameworks and say, okay, do we need to update them at all? After the Common Core came about in 2010, there was some discussion of that Common Core moved some math stuff around, you know, and do we need to align as well?

And I think there were some tweaks that were made. In some respects the Common Core was inspired by what had already been in place with the NAEO, which was pretty well respected. So it, it is meant to be a, a curriculum-based test. So they are trying to pay some attention to what is getting taught in schools.

And as you can imagine, there's always big debates about whether they're getting it right for a while after the common core. For example, they hadn't made some adjustments and there was worries that they were testing fourth graders on material that the common core didn't get to till fifth grade, and maybe that's why kids weren't doing well.

So those sorts of issues come up and, and they have to wrestle with them.

[00:28:07] Anna Stokke: Alright. So we've set this up. We know what NAEP is and, and how it's administered, who writes it, who marks it, all of that. So let's talk about what's going on with NAEP scores, because I read a lot about this sort of thing. So let's talk about trends over time and, and as you mentioned, and I agree, that's at the important thing to think about is what's happening over time.

So what do the most recent NAEP scores look like and how do they compare to the last results? So my understanding that the recent results came out end of January, is that right?

[00:28:39] Mike Petrilli: Mm-hmm.

[00:28:40] Anna Stokke: Tell me what you think.

[00:28:41] Mike Petrilli: Well, it was almost entirely bad news. This was tests that was taken last spring. So fourth and eighth graders in the spring of 2024. And what I think a lot of people were hoping was to see at least some improvement since the last time where the scores were terrible, and the scores were terrible last time. We assume because of the pandemic, of course, right? In 2022. First time since that pandemic, since in America, we had these super long school closures in lots of places in the country.

And so we expected bad, bad news and it was down dramatically wiped out 20 years of progress, because of those pandemic school closures. Unfortunately, the declines continued, especially in reading, especially at the fourth grade level. And that really surprised a lot of people and was disappointing.

Now, I wasn't super surprised, and that's partly because when you think about who those kids were that got tested, they were fourth graders in 2024. That meant they were kindergartners in 2020 when the pandemic struck. So these were five and six-year-olds who were, you know, sitting in their kindergarten class still learning how to do school, just starting to learn how to read, and then boom, the pandemic hits.

March, you know, and exactly now, right? Five years ago, they end up at home for the rest of that school year, and most of them, quote, come back in the fall, are moved along to first grade, have a new teacher they've never met, and these are six-year-olds, right? And are taught over Zoom for a year. For most of that school year, in many cases.

And so if you had to pick a group of kids who probably were most harmed by the pandemic, in terms of their learning, I mean, this would be the group. And lo and behold, it's true that they, they did even worse than kids who were two years older than them, who at least had gotten maybe a little bit more along the way on learning to read before the pandemic struck. So, and I think that just goes to show you that when we look at these cohorts particularly, you know, because of this huge disruption with the pandemic, you've really gotta think about, it's not just what happened this year, it's not what happened last year.

It's like, what have these kids experienced for their entire educational careers? And that's going to, you know, we're going to see them scarred by this as they make their way through school. Next time around it'll be, uh, in 2026, those will be kids, you know, who were toddlers when the, when the pandemic struck.

So, you know, maybe we're going to finally be getting to the point where we have a cohort of kids who were not harmed by all of that, but we're not there yet. That's the big picture. Now the other big picture, uh, Anna, is that it wasn't just the pandemic that you look at NAEP scores going back further and something was starting to happen in the mid to late 2010s where we started to see some declines, and that was super disappointing because we had made so much progress for so long in the 1990s into the two thousands through about 2010, huge progress, especially for the lowest performing kids who were disproportionately poor, disproportionately black, disproportionately Hispanic, disproportionately immigrant.

Those kids had made huge progress, and we think in part because of all the education reforms that had happened, and then they just kind of hit a wall in around 2010, 2013, 2015, we start seeing declines, especially for the lower performing kids.

And so, you know, we've all been trying to make sense of those, those trends as well, and, and figure out what might be going on. And these tests, they don't tell you for sure, but they can give you some hypotheses. You can try to look at the timing. You can try to look at the different trends for different groups of kids and start to pit, you know, stitch together a story about what might have happened.

[00:32:26] Anna Stokke: I think this sounds similar to what I've seen in Canadian PISA scores over time, which I've looked at quite a bit. And so the decline started really well before the pandemic. I think here it was around maybe 2006, 2012, that we started to get sort of steep declines. And a big thing too, with those PISA scores, those Canadian PISA scores, is that we're seeing way more students who struggle, but also fewer students who are the high flyers, like in on PISA, those are the students in levels five and six. So I'm wondering if you see something similar in the NAEP scores out of the US?

[00:33:08] Mike Petrilli: Yeah, I mean, what we do see is at the high level, the higher achieving kids seem to be kind of holding their ground. It's pretty flat or in a few subject areas, maybe up a little bit.

So that's better news. It's not great news. It's better news. The bad news about the high achievers is that that group is not as diverse as it used to be. So it tends to be the White and Asian and upper middle class kids who are still hitting those high marks, not as many Black, Hispanic, lower income kids.

That's a huge problem. But really the dynamic we're seeing is the bottom falling out. Those kids at the 10th and 25th percentile, significant declines. And you know, a lot of us trying to figure why that may be happening. You know, it is certainly the case that around the time that those declines started, we did back away from the No Child Left Behind Act, which was this act that had all this testing, but also that had a, you know, it was really meant to try to pressure schools to improve the performance for these low performing kids.

Threats of school closures and things like that. Well, we moved away from that around the same time. So you think, okay, maybe that's 'cause we eased up on accountability. We also moved to the common core standards around that same time. So maybe it's that we started actually aiming higher in terms of the level of instruction, and maybe some teachers struggled with that and the kids who were one or two or three grade levels behind those teachers didn't know how to help them get to those higher standards.

You know, that might have been part of it. And then, look, the other big thing that, that I think is, is something we gotta talk about is, is the phones. That that's when all of a sudden American childhood changed dramatically when every kid suddenly walked around with a supercomputer in their pocket. And you can just see this switch, almost overnight, that with test scores going down.

This is, you know, we see some of this evidence from the international tests as well. Of course, the dopamine hits they're getting from watching whatever YouTube shorts and TikTok and. And then take them into a school, even if we don't let them on their phones in school, which is a big debate here right now.

You know, still they're going to have a harder time focusing. And so I, I think it's some mix of all of this stuff that's causing the trouble. And then, you know, the pandemic, we just went over a cliff. It just was a complete free fall.

[00:35:26] Anna Stokke: I think there are a lot of things going on too. Definitely the phones are a big one.

I personally also think that educational philosophies could be a big one, that there's not as much focus on knowledge, academics, teaching to mastery or even using good instructional approaches, because we don't use common Core in Canada and certainly students in Canada are also struggling. So we're kind of seeing this really across North America, really.

What about the TIMSS scores? So I'm curious about that because, so I like TIMSS. I think it's a better test than PISA. That's my professional opinion. I can say TIMSS is, is the International Mathematics and Science Study and just for the listeners, that's an international assessment. It's focused on math and science.

It's administered every four years, I think, to fourth and eighth grade students. So US writes TIMSS and several Canadian provinces write TIMSS, and people might find it interesting that Canada performs worse than the US on TIMSS. That was on the most recent TIMSS and the one before that too. And you know, I read lots of articles about TIMSS scores in the US and people are upset, but nobody says anything about Canada and I can't figure it out. But in any case, tell me about the TIMSS scores, the US TIMSS scores, which you probably know a lot about.

[00:36:51] Mike Petrilli: Well, and I'm a big fan of that test as well. And like you and, and partly 'cause it really does try to test what kids are learning in school. PISA is trying to get at these other soft kinds of nebulous skills.

And so then you could get some strange results on PISA. You know, for example, where systems that don't seem to be very high-performing can pop up with these high PISA scores. So anyway, TIMSS is a great measure. And look, what was very upsetting is that for the United States, we saw the same pattern on TIMSS that we just saw with NAEP, which is the bottom falling out.

We see that our, the American students at the lowest levels, 10th percentile, 25th percentile going down. And I, I'm not a hundred percent sure about Canada, but I can say that in most of the world you did not see that same trend, that there's something, it, it seems to be a, pretty much a unique American phenomenon right now to have this bottom falling out for the low performing kids. And again, that's another piece of this puzzle. I mean, obviously phones are everywhere, so that's kind of weird. And so what is it, what is it? What happened here around 2012, 13, 15, that might've started something bad for the neediest kids. So that's what we're seeing on TIMSS too, is very much a similar story as NAEP.

[00:38:16] Anna Stokke: What do you see as the long-term impact?

[00:38:18] Mike Petrilli: Oh, I mean, this is huge. I mean, when you're talking about kids at the 10th percentile and whether they're making progress or not, I mean, these are the kids that are most at risk of dropping outta high school. The kids, the most at risk of not learning to read most at risk of ending up in the criminal justice system.

So, you know, for a lot of really important parts of our society, you know, it matters if we help these kids succeed. And they may never get to the point where they are prepared to go to college or, you know, they're going to have a white collar job necessarily. But you know, their life's going to be a lot different if they're literate versus illiterate, or numerate versus innumerate.

This matters, this is high stakes. There's been efforts, you know, we spent hundreds of billions of dollars in the last couple of years trying to help kids recover from the pandemic. Unfortunately, it looks like that money, you know, had pretty modest impact. It did some good, but pretty modest impact. So, you know, there's just still a lot of kids out there in America who are way behind.

And you know what we do every year? We just keep passing 'em on to the next grade.

[00:39:20] Anna Stokke: And I can only really speak to math, just to be clear on that, because that's the subject I really know. If you keep passing people along and they don't have the prerequisite skills, it's not going to get better. It's only going to get worse.

Right? Like those gaps are going to widen. I'm curious though, you mentioned a lot of money spent on trying to fix these problems that, some of it due to the pandemic, no doubt. But I think there's a lot more than the pandemic going on. So how did they use the money? What did they do with it?

[00:39:50] Mike Petrilli: Yeah, so you know, first of all, when Congress passed this money, the pandemic was still raging. So they needed to make it very flexible. So, 'cause they didn't know how much money would be need to be used just to help schools reopen during a pandemic and to be able to buy the PPP and to do the, all the social distancing. So it wasn't even clear that this money was going to be mostly spent for recovery.

So then they said you can use it just to get your schools open or for helping kids recover once things are kind of quote back to normal. But they left it almost wide open. And so we see a wide range of activities, and I mean a wide range, like there were certainly a lot of places that spent the money on new HVAC systems, you know, heating and air conditioning systems for their schools.

Now look, it was a pandemic. It had to do with the viruses in the air. And so, you know, air quality, you make a case for that. But boom, that's a huge expenditure. Other places did try to spend the money more on academics and the number one choice there was around some kind of tutoring program. And, so there was a lot of efforts to set up tutoring.

The more successful ones happened during the school day. Usually figuring out a way to bring in college students or somebody, sometimes bring people in online to tutor kids, but again, at school during the school day, and to try to help give them what they needed to catch up. And the places that put more of the money into that sort of academic intervention did see better results.

Not surprisingly so, you know, but it's still, it's not enough. I mean, if a kid missed a whole year or a year and a half of school basically, which did happen for a lot of these kids, 30 hours of tutoring. This, it's just not enough. The math doesn't work. What we probably should have done, and some of us called for and didn't happen anywhere as far as I know, is that we should have just said, “Hey, these kids missed a year of school. Let's just not pretend that they were in second grade last year at home. Let's just have 'em do second grade over again.”

It's a redo and there's no stigma. It's everybody instead of passing along. But no, we refused to do that. Instead, the system just said, push ahead. You did zoom school for second grade.

Now you're back. You're doing regular school for third grade.

[00:42:06] Anna Stokke: I can imagine that wasn't a popular proposal.

[00:42:10] Mike Petrilli: No, it was not a popular, yes, you're right.

[00:42:12] Anna Stokke: And I, I probably would've been one of the parents who's like, “wait a minute, you know, like we spent all this time helping our kids with math. No, no. They're not going to be held back.” But I can see that that wouldn't have been popular. Okay. Now back to the NAEP scores. I'm just curious, are there any states that kind of bucked the trend in math or reading?

[00:42:34] Mike Petrilli: Great question. Yes, a handful. So the big talk in the United States is Mississippi—happens to be our poorest state, uh, but they have been celebrating the Mississippi Miracle. That in the 2010s when everybody else was going backwards, they were making huge progress, especially in reading, especially in the early grades. Uh, but now you see it also in math and also in eighth grade. And, and they, you know, they had a setback with the pandemic, but then they have bounced back stronger than anybody else.

So if you adjust now for some of the demographic differences between the states in some categories, Mississippi comes out number one, and it really is remarkable. And here it's a story of a leader, Kerry Wright, who was the state superintendent. Who just kind of rolled up her sleeves and took real leadership of the state's schools and said, we are going to change teaching and learning in every classroom in Mississippi.

Our states aren't used to doing that because we have all these local school districts that are the ones that actually are responsible for running the schools and hiring the teachers and picking the curriculum. But she found a way to work with those local leaders and to really have an impact on what was going on in their classrooms.

Tons of money spent, especially on early reading, on the science of reading, less focus on math, and yet they've made huge progress in math too. Probably in part because the kids can read better and it's a lot easier to teach kids math who can read. And so that, that is a rising tide that has lifted all boats.

So people are definitely looking at Mississippi. Next door Louisiana had a good showing, this time around Alabama. It's interesting, these deep southern states that all are struggling with a lot of poverty, but also, have been willing to have a pretty tough love kind of approach and, and they're seeing some good results.

[00:44:20] Anna Stokke: That is really exciting. And okay, so about Mississippi, they kind of got into the science of reading you say, and they also saw progress in math. So it could be for sure if the students can read it might make it easier to do some math. It could also be that if the teachers were kind of trained in science of reading techniques and they saw those working for reading, they might have tried to use some similar techniques in math. It could be something like that.

[00:44:47] Mike Petrilli: Yeah. And there's a big discussion right now about, you know, what can we take from the science of reading and apply to math? And I think there's discussion, but also worry. I mean, I think there's a lot of math people that are worried that it's going to get too simplified.

Like In reading, the phonics we now know is essential. You've gotta do the sounding it out. You can't skip that like some advocates had believed. And so maybe the equivalent is that, you know, you just gotta know your math facts and you gotta know your basic algorithm. I mean, maybe that is a good analogy and a same kind of situation, but there's a big debate about what would it really mean to apply the science of math next.

[00:45:27] Anna Stokke: Yeah, I know all about that debate, Mike.

[00:45:29] Mike Petrilli: Yeah, I guess you're in the middle of it, right? You can tell me.

[00:45:31] Anna Stokke: Oh yeah, you betcha. But yeah, so it's interesting like have other states tried to copy what Mississippi's doing?

[00:45:40] Mike Petrilli: Yes, absolutely. And it looks like there may be some success. I mean, I was on with this Indiana State superintendent earlier and you know, they very much tried to copy and they've seen. They saw a nice, a nice bounce for fourth grade reading in the most recent scores. So that's encouraging. We'll see, you know, you want to see several years of trend and, and see what happens.

Yeah. I mean, all over the country states are saying they're now doing the science of reading and trying to follow Mississippi's lead. But yeah, like everything the details matter, implementation matters. Uh, in some cases, you know, they're not coming anywhere near to spending the kind of money that Mississippi spent, or exhibiting, you know, kind of making sure there's the same amount of control, quality control that Mississippi had over what was happening in the classrooms.

You know, so it's likely, seems likely that in those places that aren't spending as much or that are doing more, just happy talk about it or encouragement, you're not going to see the same results.

[00:46:44] Anna Stokke: On the other side of things, are there some states that dropped more than others or that are doing quite a bit worse than others?

[00:46:51] Mike Petrilli: You know, there are, I mean, you can definitely see that there are some states, especially once again, when you apply these controls for demographics, you know, and, and we gotta do that because we know, as we discussed earlier, that kids from more affluence tend to do better in tests.

And so, you know, some of our states just have a lot of affluent kids and if that's all you looked at, but yeah, no, there's some and that's interesting. I'd say for example, Maine is one of those places that just has really been struggling for a long time in terms of student achievement, but is not known as a state that's taken a lot of responsibility for trying to improve teaching and learning.

You know, that's been a place where they kind of just leave it to local school districts. And I think it shows, you know, I think that you can say that it shows. Massachusetts has for a long time been on top in the United States and even in some of the international assessments and they're hanging in there, but they're starting their ranking and, and in terms of their being way out front, they have lost some ground as well.

And I think the Massachusetts story is one of saying, look, they got crushed by the pandemic. You know, that many, many school districts in Massachusetts were closed for most, if not all, of the 2020, 2021 school years. And it shows, you know, those kids missed a ton of school and it's a long, long way back.

[00:48:04] Anna Stokke: What about California?

[00:48:09] Mike Petrilli: Yeah. You know, California is interesting. They actually did show some progress back in the 2010s. Again, keep in mind, most of the country was starting to kind of sputter and, and move backwards. That was a time when California put in a lot of new resources into their state.

They had a big school funding reform in 2013 and just spent billions and billions and billions of dollars on their schools and really tried to target it to the neediest school districts. And I think, you know, there's been some studies showing that they, the targeting worked and that you can look and see some strong achievement results.

So they, they were starting to show again, some progress as well. And then Bam! Hit the pandemic, like Massachusetts—Deep blue state–you know, long, long school closures, especially in the big cities for basically that whole 2020, 2021 school year. Huge, huge learning loss and now struggling to come back. I will say one more thing about a lot of these places, including Massachusetts, including California, is that they're dealing with learning loss.

They're dealing with chronic absenteeism, really struggling with getting kids back in school after the pandemic, and they are struggling with low enrollment. They have lost a ton of population. Families have moved out of California because it's so, you know, unaffordable. The birth rate, of course, in America is way down from where it used to be.

We used to be this exception on birth rate no more. And so schools are emptying out. So the next wave is that there's going to be hundreds, probably thousands of schools around the country that are going to have to close 'cause there's not enough kids to go around. So that's something and it's, you know, it does seem to be hitting the expensive blue areas the most, and so that's something they're struggling with also.

[00:49:54] Anna Stokke: How should education systems prepare for potential challenges in the coming years considering the current student performance? Like, what do you think they should do?

[00:50:03] Mike Petrilli: I do think it is not too late to be talking about the strategies I mentioned before about having a whole lot of kids repeating a grade.

As unpopular as it may be, we still see from the data that at least the United States, the kids coming into school, they are still behind where their counterparts used to be. Whether that's because they were babies during the pandemic, or is it 'cause their parents are giving them iPads to stare at too often or whatever it is they're coming in behind, and we need to get away from this situation where we just keep pushing kids through the system without mastering the material.

So I would love to see, and I keep floating it, and you say, I know it's unpopular, but maybe someday I'll convince somebody is in some schools, let's say, where you've got almost everybody is coming into kindergarten, way behind. Just add an extra grade to the early elementary years, do a transitional first grade or a grade one and a half, you know, just to give them a little more time.

We did this for one of our sons. We redshirted one of my sons, who we got just wasn't quite ready for kindergarten. Give them a little more time, spread it out. Help them get off to a strong start in those basics on reading and math, you know, before they start marching through school. I think that can make a huge difference.

I think otherwise we've really got to get serious about helping teachers take the materials they've got, but really say, “okay, if you're teaching third grade, but you've got kids at a first grade level. Here are some really good practical strategies to help those kids catch up as quickly as possible.”

I think we just tend to use magic words. We tell these teachers, you gotta scaffold instruction and you gotta, you gotta accelerate and you gotta, but don't remediate and you gotta, I don’t know, I'm not an instructional person, so to my, ears, some of that just sounds like magic talk.

And I understand from some teachers too, they feel like they're just not given, they're not actually being given the kind of help and instruction to say, “what do you do?” They got a whole lot of kids who are way behind and they're not ready for the fourth grade material they're supposed to be teaching them. What do you do tomorrow to help that? And I, I think that that kind of very precise support and instructional design kind of work could go a long way.

[00:52:15] Anna Stokke: Yeah. So you're talking about practical strategies, right?

[00:52:18] Mike Petrilli: Yeah.

[00:52:19] Anna Stokke: That's what you think would help. Just an off topic question, 'cause I have you here, and this is something that I'm really interested in. Do many states require content licensure exams, say in math or other subjects for teachers?

[00:52:33] Mike Petrilli: It depends. So yes, for older kids, say middle school and high school, if you want to be a math teacher, yes, you have to have some either major in math or you at least take a test.

In math for elementary school where the teachers teach everything, I think the answer is no. Now there, there may be a teacher certification exam that has some math on it or some math strategies on it. But, but my understanding is the way those tests are designed, you could have some pretty weak skills in math or math practices and you could still pass.

And there's been some efforts to try to beef up those tests when it comes to reading, when it comes to the science of reading and making sure teachers really understand that. And I think there's an understanding that we need to do something similar now on the math side.

[00:53:22] Anna Stokke: And then I have an advice question. So, because you're president of the Fordham Institute and you know, you know a lot about educational policy and I bet you've influenced policy over the years, so do you have any advice for that? How can a person go about influencing educational policy?

[00:53:39] Mike Petrilli: Oh, no thanks, Anna. Well, look, again, I'll speak from my experience here in the United States. You know, most policy and education really is made at the state level here. The federal government's a junior partner, and so the actions at the state and what you find is that in every state, there's probably 10 or 15 people that in any given year, that are really the ones engaged in the big debates, you know, and it tends to be chair, the chairs of the education committees and the legislature, the state superintendent, maybe an aide in the governor's office, the teacher's unions people, a couple of the other folks on the periphery. And so, you know, for us as a think tank, we try to make sure that our stuff is reaching those folks.

You know, we're not writing with the hope that the governor of Ohio reads our stuff, though, that would be nice, right? Or even in most cases that some legislature, it's more that some staff member or a lobbyist, or especially an advocate and, and we have a growing number in the United States of these education reform advocacy groups in a lot of state capitals that share our values, that care also care about student achievement.

Accountability and charter schools and all these, you know, improving teacher quality and they are our primary audience. We're trying to arm them with good information, ideas, evidence, and, you know, and then they can come to us for more if they need help with an op-ed or if they say, “Hey, can you help us write a bill?”

So it's really working with those people. And I think that to me is the biggest advice is that, you want to influence people who themselves have influence with the policymakers. Those are going to be people with boots on the ground in a given place, you know, that live in the state capital, in your case, in the provincial capital, who have deep relationships with the people that they're lobbying, that they're trying to influence.

You know, I'm some guy from DC running a think tank. If I try to parachute in and persuade a governor to do X, Y, or Z, I'm just some other guy. They're not going to listen to me. But they're going to listen to somebody they've known for 20 years, their kids go to school together, you know, they live in the same town. And so it's getting to those people that that's really important.

[00:55:49] Anna Stokke: Okay, that's super helpful. And I should mention, by the way, that I came to know about Fordham Institute is because I've read a lot of the articles on advanced education, which is something I'm also interested in. In fact, it's how I came across Jonathan Plucker, who I had on the podcast to talk about advanced education.

I like a lot of your work, actually, so I recommend that people. Check it out. This is my last question and, and maybe we've covered everything 'cause we've talked about a lot, but what do you see as the biggest challenges in US educational policy today?

[00:56:35] Mike Petrilli: Gosh, I mean, well this could give me a chance to describe my, my frustration with our presidential leadership right now, so, no, I know, but I feel terrible about all these crazy debates around tariffs and everything else and, and, and how he has treated Canada.

I was telling Anna earlier I got to go ski in Tremblant just a few months ago, and it was a wonderful trip. I loved lovely time and Quebec.

Everybody was so, so kind and welcoming. No, but we do miss having a president who made education a priority, and it's been a while now and I'll make it bipartisan. You know, Joe Biden did not make this a priority either. These two have been the first time in decades that we have missed having education presidential leadership. You know, we had George H W Bush back in the late eighties who promised to be the education president.

And then Bill Clinton, who had a huge focus on this from the center left. And then George W. Bush from the center, right, with the No Child Left Behind Act. And then Barack Obama embracing a lot of this education reform work and it mattered. It really mattered to get the country focused on, on improvement, and it showed up in these test scores.

It showed up in a sense that, hey, yeah, we can make our schools better. And it's now been about a decade since we had that kind of leadership and we really, really miss it. And I guess another presidential campaign is always around the corner. So, we can hope that maybe next time we'll get someone who makes this a priority.

[00:57:58] Anna Stokke: And I'll stay quiet on the politics, except for one thing–Canada's not going to become the 51st state.

[00:58:04] Mike Petrilli: Good.

[00:58:05] Anna Stokke: But yeah, so I mean, I, I want to echo what you said. Education is so important. It's, you know, the great equalizer. It changes lives. It really should be a priority for any politician. I really enjoyed our conversation today, and thank you so much for the work you're doing and for coming on and, and talking to me about it.

I learned a lot. And I really enjoyed the conversation.

[00:58:28] Mike Petrilli: Great. Thanks so much for having me. It's been my pleasure.

[00:58:40] Anna Stokke: As always, we've included a resource page that has links to articles and books mentioned in the episode. If you enjoy this podcast, please consider showing your support by leaving a five-star review on. Spotify or Apple Podcasts. Chalk and Talk is produced by me, Anna Stokke. Transcript and resource page by Jazmin Boisclair and Deepika Tung.

Subscribe on your favourite podcast app to get new episodes delivered as they become available. You can follow me on X, Blue Sky, or LinkedIn for notifications. Or check out my website, annastokke.com for more information. This podcast received funding through a University of Winnipeg Knowledge Mobilization and Community Impact grant funded through the Anthony Swaity Knowledge Impact Fund.