Ep 15. Modern relevance in the math curriculum with Brian Conrad

This transcript was created with speech-to-text software. It was reviewed before posting, but may contain errors. Credit to Jazmin Boisclair.

You can listen to the episode here: Ep 15 Modern relevance in the math curriculum with Brian Conrad

[00:00:00] Anna Stokke: Welcome to Chalk and Talk, a podcast about education and math. I'm Anna Stokke, a math professor and your host.

[00:00:15] Brian Conrad: I've heard one of the CMF writers say, in a radio interview that, you know, “Calculus, oh, this won't be important going forward,” and nothing could be further from the truth.

Data science, all the STEM fields, economics, marketing, right, with its reliance on data science. I mean, all this AI, machine learning, all of these things rely crucially on multi-variable calculus and a lot of math.

[00:00:42] Anna Stokke: You are listening to episode 15 of Chalk and Talk. That was Dr. Brian Conrad who is a math professor and director of undergraduate studies in math at Stanford. We discussed some modern-day applications of math, and he gives some advice for parents who wonder what type of math their kids should learn to be ready for a four-year college degree in STEM or other quantitative fields.

He also updates us on what happened with the California Math Framework (CMF) since my interview with Jelani Nelson in early July, and he takes us through examples of the many false or misleading citations he found permeating a draft copy of that 1000-page document.

The discussion of those findings illustrates how citation misrepresentation can lead to misunderstandings about math and data science among the general public. As an interesting side note, Brian Conrad's Ph.D. supervisor was Andrew Wiles, who proved the famous Fermat’s Last Theorem, and Brian appeared as an extra in the NOVA film about it called The Proof.

It's a fascinating story, and I will provide a link to the film on the resource page, and I invite you to check that out. I was deeply moved by Brian's passion and commitment to uncovering the inconsistencies in the CMF to inform the public. His logic is flawless, and his attention to detail is impeccable. I really learned a lot from our conversation.

Now, without further ado, let's get started.

I am really excited to have Dr. Brian Conrad joining me today, and he is joining me from Stanford in California. He is a mathematician and a professor in the Department of Mathematics at Stanford University. He has been the director of undergraduate studies at Stanford for 10 years. In that role, he initiated significant overhauls of large introductory math courses that have been very well received by students.

He has a Ph.D. in mathematics from Princeton. He's an award-winning researcher. He was awarded the Presidential Early Career Award for Scientists and Engineers. That's the highest honour awarded by the US Government for outstanding early career scientists and engineers, and he's also an award-winning teacher.

He was twice awarded the Dean's Award for distinguished teaching in recognition of exceptional teaching at Stanford, and he's here today to tell us more about the California Math Framework, which he knows quite a lot about. He single-handedly performed an in-depth analysis of the 1000-page draft of the CMF, and he found a shocking number of citation misrepresentations, misinformation about data science and other problems, totalling over 170 pages of corrections, which he documented and made publicly available on a website.

I'll link to that on the resource page for this episode, but you can also easily find it by Googling “Conrad CMF.” And this was a volunteer public service that he took on himself. He wasn't contracted or paid to do this. So we'll hear a lot about this today.

Welcome to my podcast.

[00:04:01] Brian Conrad: Thanks for having me. I'm glad to be here.

[00:04:03] Anna Stokke: The California Math Framework has been quite controversial, as we've heard about in a previous interview with Jelani Nelson, and the process spans several years. Can you update us on this? Was the CMF adopted by the State Board of Education?

[00:04:21] Brian Conrad: Yeah, so it was adopted. So what happened was, the extensive comments that I gave were on, in this spring of 2022, were on a version number two. That revision took basically a year, a little over a year to do because there were so many comments from the public on things to change.

The third version appeared very recently, at the end of June, June 26th. It was still around a thousand pages. This time around, the public was only given around two weeks rather than two months to give public comment. And then, the state board voted one week after that deadline. So that was on July 12th.

And the board unanimously voted to approve the document. They did say while they were doing that, “Oh, there are still some problems, and we'll fix those.” Of course, I think they should have fixed the problems first and then held the vote. But that's me.

[00:05:13] Anna Stokke: You are a university professor and a mathematician and the CMF concerns K to 12 education, and people might wonder why you got interested in this to the point where you read this huge document and, and documented the issues. So can you tell us why you got interested in the CMF and what prompted you to read through the drafts and document the problems?

[00:05:37] Brian Conrad: Sure. My father is, is now retired. He was a high school math teacher for many years. So I think I've had a bit more interest in paying attention to what's going on at the pre-college level for math than maybe most other mathematicians. I've been the director of undergrad studies in math at Stanford for many years now.

And in that role, I've heard both from colleagues here and people I know at other universities in many quantitative fields expressing alarm about the weakening math skills among evermore incoming students. And, and so combined with the fact that I was seeing a lot of false promises spreading all over the country about data science in the summer of 2021 and that this was appearing in the CMF, that got me quite concerned.

And so I felt obligated to look into this because I figured that since I'm at Stanford, that probably my position would make my voice carry some weight if in fact the concerns that I was hearing were born out by the details. Since the document was over a thousand pages long, I figured probably nobody's actually read this whole thing.

And, you know, when I was in college, I read this book of stories by Richard Feynman, in which he talked about his crazy experience on the California Math Textbook Commission in the 1960s. And it was like corruption and like nobody else was reading the books, but he read them all and found lots of nonsense.

And so I thought, “Okay, maybe I should just read all of this.” And so then I spent a very, very long, boring spring break just reading this document. And yeah, I documented everything that I found in it, and I was just shocked.

[00:07:11] Anna Stokke: So, how long is the spring break?

[00:07:14] Brian Conrad: Well, the spring break was one week. There were 14 chapters. So essentially, I read two chapters a day. But this also included downloading and reading a lot of papers and education that were being cited in very strange ways. So it was an extremely tiring week. And then, of course, it took much more time to write up.

I printed the whole thing and made a lot of comments in the margins. It had a huge stack of papers related that were being cited. And then it took much longer to write up everything in a typed form.

[00:07:43] Anna Stokke: I'm seeing a lot of statements about data science and what I think are maybe some misunderstandings about what it involves. Do you think that there are misunderstandings or a lack of awareness about the connection between high school math and university math, or the relevance of math for modern professions?

Can you talk a bit about that?

[00:08:05] Brian Conrad: Sure. There's no question that there's a lot of room for improvement in probably both the motivation and the examples that are given to kids. There's actually an organization called Skew the Script that produces a lot of really good material to help teachers on this. I mean, I think in general, the experience we had here at Stanford overhauling some of our courses, we had the benefit of talking with expert colleagues in other fields to hear about really cool examples in other fields that use a wide array of foundational math.

And you know, a teacher in a typical school district, they don't have anybody to talk to, and Googling doesn't help to find, you know, interesting examples cause you don't know what to look for. So I think it it's really necessary that publishers step up and provide teachers with a lot of better material so they can convey to students the contemporary relevance, and the fact is that the math curriculum, as it's existed for a long time, remains the foundation for all of the amazing technological developments that we've been seeing in recent years.

And, you know, for example, video game design uses trigonometry, microeconomics you need a solid fluency with functions, QR codes rest on polynomial division, machine learning uses calculus in billions of variables, recommender systems use matrix algebra. I mean, the list is endless. The fact is that there are these vast connections between math and many contemporary things, and frankly, even from data science as well.

And if these could just be infused into the curriculum as well chosen motivation, I think it could go a long way towards helping kids to see that the math that they're being taught really is useful for many things, and that can maybe give them more motivation to persevere and learn it.

I mean, I think a lot of times, parents may only dimly remember what was going on in high school math and may think because of computers that a lot of the topics are now somehow obsolete. And this is certainly been stated in public by, you know, some of the CMF writers and, and, and other people.

But this is completely false the role of quantitative skills is only becoming more important for secure jobs in the future. And you have to keep in mind that kids in early high school now, I mean, they're really going to be entering the job market, you know, close to 10 years. So the information about like what kind of quantitative skills are, you know, are needed for how many types of jobs now, it's only going to grow in the future.

Marketing is much more quantitative now than it used to be, and even fields like, you know, computer science and, and now data science. I mean, these rest on almost the full spectrum of the high school math curriculum. The college degrees in those fields require learning calculus and often multi-variable calculus fairly early in college.

I know somebody who works for Art of Problem Solving who recently mentioned that a parent asked him like, “Well, what, kind of math should my kid be learning in high school to prepare for a career in data science?” And the answer is math. Like, like math as it's always been.

As I say, we could definitely improve the motivation, but the, of course, the core content remains essential for any serious quantitative work as a college degree or as a career.

[00:10:58] Anna Stokke: Algebra and, you know, exponential functions and logarithmic functions, trigonometry, all that stuff, that's the very foundation for these careers.

[00:11:07] Brian Conrad: Yeah, and, I mean an example of how things can go awry. So, for example, in the state of Ohio, they now have rolled out this, you know, with a mission towards trying to make modern relevance come across better to students, they created what they called “alternative math pathways,” but these are actually dead ends in the sense that they have a data science pathway and a computer science pathway.

These are for the last two years of high school. And a typical student and parent might think, “Oh, if I want to go into, you know, computer science or data science, then I should choose that.” But if they choose that, they're going to be screwed when they get to college because those pathways as they're set up specifically warns, these are not for people who will need calculus in their career.

College degrees in those fields require calculus in an essential way. So if you're on a path that is not for learning calculus very early in college, then you're off the ramp. And this is a real problem. And I think that there needs to be much more involvement with people from industry and perhaps higher education to help to inform the process about the connection between content, motivation, and future relevance with jobs.

[00:12:17] Anna Stokke: Absolutely, and it sounds like even the CMF writers themselves maybe were unaware of some of these connections.

Let's talk a bit about those misrepresentations that you found of cited work in this CMF. So you found misrepresentations in several categories. So neuroscience, acceleration, tracking, assessment, devaluing advanced math, data science.

Let's talk about some of those. Let's start with neuroscience. So there were a lot of claims made about neuroscience, and so here's an example. The CMF claimed that a particular paper found that when students worked with numbers and also saw the numbers as visual objects, brain communication was enhanced and student achievement increased.

And so, as an example of sort of the issues that you found, can you explain how that particular paper was misrepresented?

[00:13:12] Brian Conrad: Sure. Yeah, that one was a real doozy. In that one sentence from the CMF, there are actually three lies. First of all, even though it talks about students, the paper was actually about a study of adults. It was not about a study of students. Secondly, that sentence from the CMF talks about brain communications, but the paper didn't involve any brain imaging at all.

There's nothing about brain communications in the paper. And the third lie is that the CMF statement talks about seeing numbers as visual objects. and this suggests something about the, you know, maybe teaching math in a visual way. But the study was just a study of people looking at large arrays of dots and trying to estimate the number of dots and maybe then doing some arithmetic with that.

I mean, this has nothing to do with the impression created by the sentence in the CMF. And this happens again and again, this is not atypical for the misrepresentations I found. Like you would have papers that say “x,” CMF says “it's about not x” or about something totally unrelated.

It's just completely bizarre. I have no idea. How this level of dishonesty is just flagrant, and there's nothing subtle about it. This flagrant dishonesty in these citations was just unbelievable.

[00:14:24] Anna Stokke: It is unbelievable. In general, what were the main issues with claims based on neuroscience?

[00:14:31] Brian Conrad: So I would say there are really two issues. So one is I think neuroscience was invoked, how to say, maybe to give a scientific aura to what really, at the end of the day, was some kind of ideology. So that was kind of one problem. An example of this is it would talk about, “Oh, we recommend a certain practice because this will increase brain connections,” whatever that means and that this should therefore improve math learning.

But this is misleading. So for example, the mathematical analog of dyslexia called dyscalculia, that also is associated with hyperconnectivity between certain regions of the brain. But the fact is, nobody on the CMF, either the writing team or the oversight team has any professional expertise in neuroscience.

So they had no business invoking this stuff, creating a kind of pseudo-scientific impression of, depth or, you know, scientific guidance on, on what's really, as I said, just basically ideology. I consulted with four professional neuroscientists to back up the things that I was claiming about what was wrong with how neuroscience was being invoked.

And in fact, there was a, a panel formed by the National Academies of Science some years ago related to learning mathematics. And it warned that attempts to make connections between neuroscience research and classroom practice are premature. Now, fortunately, the third and final version of the CMF did substantially dial down the hype and misrepresentation about neuroscience.

So that problem is largely fixed. But, but again, like, again, I can't understand whoever was behind putting this stuff in there in the first place why that was allowed to, to go through in the second version doesn't make any sense.

[00:16:08] Anna Stokke: So what about tracking assessment or devaluing advanced high school math? Do you mind talking about some of the citation misrepresentations in one of those areas?

[00:16:21] Brian Conrad: Yeah. So maybe to convey the range of the dishonesty, maybe I'll, I'll pick one from each of those areas. So first of all, I would say in each of these cases, there's a, there was a kind of ideological narrative that the CMF was trying to push, and then they would cite papers to promote that. And, and as I say, often the papers had nothing to do or little to do with what was being claimed.

So in the case of tracking, so in the context of the CMF, this refers to separating students by perceived ability and mathematics at a certain stage in education. And so the CMF preferred narrative is there should be no separation by ability before Grade 11. And this is, of course, this topic has a lot of nuances, which the CMF never acknowledges.

And so for example, there was one paper which it claimed demonstrated the harmful effects of tracking. But the paper used the word tracking in a completely different, with a completely different meaning that didn't even apply in the context of kids attending their local public schools. For assessment, the CMFs preferred narrative was to avoid giving grades and particularly no grading on homework.

It even contained the phrase, “Homework is one of the most inequitable practices in education.” And this devaluing, I mean, of course, kids may be assigned too much homework, and that should be advocated against, but this sort of denigration of homework is so completely absurd. I mean, you ask anybody who's ever attained any expertise in music, sports or anything, The idea that you shouldn't do some practice, you know, once you're at a, a suitable age, is so divorced from reality that it makes absolutely no sense.

And what was cited for the fact that homework is bad was some book that had no research basis at all for those claims. That also was removed from the final version. The devaluing of advanced math comes in two forms. There was, in general, a kind of advocacy against the value of learning calculus in high school, and there was advocacy against learning Algebra I in eighth grade.

These were both based on the fact that there are problematic demographics in the outcomes of success for different demographic groups in, let's say, calculus in high school or advanced math in high school.

But, you know, the way you address a problem is go to the root cause. Like why don't, why doesn't the CMF advocate more resources into the elementary grade so more kids can be prepared for eighth-grade Algebra I and to reach calculus in high school rather than this ideological nonsense that this is bad?

And the fact is, the way that you're going to diversify the industries and the STEM fields and the quantitative fields is by giving kids more access to advanced math in high school, not cutting everybody down. I mean, if you want to push it to its logical extreme, the people who advocate that the, you know, disparity gaps mean that we can't allow anybody to advance, I mean, let's just stop teaching math that'll get rid of the gaps.

I mean, it's completely absurd. And so, for example, the CMF cited a paper that it claimed gave evidence in favour of delaying calculus to college. But the paper was about the exact opposite. It, it said that kids who take calculus in high school and then wind up taking it again in college do better than those who only took it for the first time in college.

You know, it is just completely bizarre. As I say, it's like there was some preset narrative, and then they would just pick papers, and the paper didn't have to have anything to do with the actual claims being made. It was very strange. I found this over and over and over again every time. I mean, the thing is that you know, maybe it's partly based upon either knowing how math works at a more advanced level or just more experienced, but over and over I would just see some claim made that seemed to me completely false.

And so then, if it was based on a citation, then I would look up the citation. And in I think pretty much every single case, the paper being cited simply did not demonstrate in any way what was being claimed. And it was just shameless lying. I couldn't believe it.

[00:20:10] Anna Stokke: It is unbelievable. Maybe they were thinking that no one would check?

[00:20:15] Brian Conrad: It's surreal. And I also cannot understand why the State Board of Education did not do an investigation to determine who is responsible for this and make sure that the people responsible will never again have any role in public policy in the state of California for education. It's unbelievable.

[00:20:32] Anna Stokke: It might be worth noting, I mean, this was a team of five university professors, right?

[00:20:37] Brian Conrad: Yes, that's correct. So the writing team consisted of one mathematician and four professors of math education. And there was an oversight team of 20 people who generally were from K-12, either teachers or district staff.

[00:20:52] Anna Stokke: If you were a student in a class at a university and you submitted a paper where you were doing this, I think you would probably get an F. You should get an F.

[00:21:02] Brian Conrad: You would be in big trouble, yes. Now, of course, they might claim, “Oh, this was only a draft.” Right? But these dishonesties or inconsistencies, they have real consequences, you know.

So Bob Moses, I mean, he passed away a couple of years ago, but he launched this thing called the Algebra Project in the late 1980s, and this was aimed at giving more kids from under-resourced communities preparation so that they could be ready to do Algebra I in eighth grade and be on the kind of college prep pathway in math.

Right? Bob Moses said, you know, “Math is the next civil right.” Yep, and he began this in Cambridge, Massachusetts. And so it's really sad that very recently, Cambridge, Massachusetts announced that they will no longer offer eighth-grade Algebra I.

[00:21:44] Anna Stokke: Oh, that's terrible.

[00:21:46] Brian Conrad: Yes, it's completely absurd. And the thing is, this is, this is the essence of fake equity because of course we know what's going to happen. As we saw in the case of, San Francisco implemented this exact same ridiculous policy. They had no real mechanism to explain why it would lead to improved outcomes. It did not lead to any improvement in equity at all.

And all that happens is, the parents, the kids from families with resources go to external places, whether it's the Russian School of Math or other things, and they will keep learning what they would usually learn. And so when the public education is decimated in this way, the kids who suffer are exactly the ones who these so-called equity advocates are claiming to help.

[00:22:24] Anna Stokke: Those who know that they need to get their kids the extra help and can afford it, they self-insure. That will always happen. That's why it's really important to make sure that we do actually have an equitable public education system.

[00:22:37] Brian Conrad: So one interesting example that came, that came to my attention recently is in the beginning in 2019, the city of Dallas rolled out a system for honours or maybe accelerated math, which was you had to opt-out rather than opt-in. So basically by default, everybody was put on the more advanced math track unless they had a signed letter from their parents saying that they didn't want that.

And this has led to tremendous improvements in the demographic outcomes of those who succeed and pass and continue on in the advanced math classes. Now, why is it that the CMF, instead of focusing on cities and states in this country within the US education system that have managed to find better success, they instead point to other countries where the population demographics are nothing like the US, the design of the education system, the training of teachers is nothing like in the US, and there's no prognosis for making it look anything like those in the future.

So we should really have been looking closer to home and emulating policies in cities and states that have demonstrated better success records in math than California has done.

[00:23:44] Anna Stokke: There's been no accountability for the authors, right?

[00:23:46] Brian Conrad: None.

[00:23:47] Anna Stokke: Have they responded?

[00:23:48] Brian Conrad: Not to my knowledge. I mean, of course, to its credit, the State Board of Education has fixed most of the citation misrepresentations. I'll, maybe I'll can tell you about some of the others that occurred. There are still other problems as we can discuss, but in particular, I remain flabbergasted that nobody has been held accountable for the vast citation misrepresentation that was going on in the document, somebody should be held accountable for that.

[00:24:14] Anna Stokke: So let's address a few of the other things that you mentioned. You have a concern regarding a myth from 1892 that keeps getting repeated again and again and again. Can you talk a bit about that?

[00:24:28] Brian Conrad: In 1892, there was an organization put together called The Committee of Ten. It has a Wikipedia page, you can look it up, Committee of Ten on Wikipedia. And this was a committee of people who were appointed to try to propose general guidelines for content of the curriculum in public education.

So the late 19th century was a time, more people were coming into public education, of course, it wasn't remotely at the level that it became from the mid-20th century onwards. And it, it addressed topics, of course, many disciplines, so science, math, languages, history, and so on. And what was basically this myth that keeps being propagated in various places is that this Committee of Ten, and it's always pointed out that they were white men, which was completely, you know, to be expected in the late 19th century.

But in any event that this Committee of Ten set the high school curriculum on a pathway to calculus, and this is meant to somehow create the impression that calculus, I guess, is either somehow associated with white supremacy perhaps and more generally, that there's something very obsolete about it.

And this is completely absurd in multiple ways, but I should say that when I first heard this, I couldn't believe there was any truth to what was being said simply because I knew that calculus was not taught in the high schools until after Sputnik.

And so I was like, this makes absolutely no sense. So I just put into Google Committee of Ten, I found the Wikipedia page, and at the end of the Wikipedia page is the actual report from 1892 that was written and I just read the part of the report about math, and there is no mention of calculus in there anywhere.

But not only is there no mention of calculus, that document actually is anti-tracking on math, which is exactly what the CMF wants to advocate. Moreover, it proposes two separate math pathways. One for the technical colleges, which is where after geometry you do more algebra and maybe trigonometry, and one for, you know, what are called bookkeeping and, you know, and commercial arithmetic, which you might say was like the data science of the 19th century.

So paradoxically, given that the CMF was like pushing for a data science pathway and was pushing for anti-tracking, they actually should have been all in favour of this Committee of Ten document. It goes to exactly the 19th-century analog of what they were trying to push. But as is demonstrated by the citation misrepresentation, it seems that some of the people involved in this simply do not read the original sources.

So that myth is not only is the myth completely bogus, but I heard it, I've heard it repeated over and over again, and it seems that almost nobody who says this actually goes back to check, I guess, because they think the message fits their preferred narrative, that somehow calculus is obsolete. And for example, I've heard one of the CMF writers say in a radio interview that, you know, “Calculus, oh, this won't be important going forward,” and nothing could be further from the truth.

Data science, all the STEM fields, economics. marketing, right, with its reliance on data science, I mean all this AI, machine learning, all of these things rely crucially on multi-variable calculus and a lot of math.

[00:27:34] Anna Stokke: Did you write to any of them and tell them that this myth was, that this was incorrect, what they were saying?

[00:27:40] Brian Conrad: So this was among the many comments I gave. And again, to their credit, the revision team did in fact remove all mention of this. But unfortunately, I've heard it repeated again by, you know, some people associated with the board and I just wish that they would please stop saying it because it's completely false.

Even if it were the case, which it's not, that the goal of the 19th-century math curriculum was to prepare everybody for calculus. Like, so what? I mean, Newtonian mechanics was invented, you know, centuries ago.

It's not that, therefore, physics is obsolete. I mean, math is a very cumulative subject and our judgment about what's important or not should be based upon what math is relevant to contemporary things. And calculus, of course, as you know, is absolutely essential. All the optimization that underlies all of the modern technology relies crucially on calculus.

I can drive a car without knowing how it works. And so it's not the case that, “Oh, you know, in order to use an AI system, you should know the math behind it.” But in fact, to the contrary, we should use those modern applications to help kids to see the value in this material, and hopefully can motivate more people to get the preparation they need to maybe pursue those careers.

Because the careers with the more quantitative skills are going to be the ones that are more secure into the future. The reason that people get the high-paying jobs is because they have training that requires learning non-trivial skills, and that often involves math.

[00:29:04] Anna Stokke: And you have to do the hard work to get good at math. You can't just sort of avoid the math, right? There are no magical roads that can get you out of algebra and land you a career in data science.

[00:29:17] Brian Conrad: Right, and the fact is, if your skillset amounts to pushing buttons on a computer, then you're probably going to be replaced by a computer in due time. And, the people who actually understand math better are the ones who can actually analyze things when the numerical data that's fed into the computer has problems.

The computer often generally cannot figure out the mistakes in the input. And so you need people who understand, have good number sense, a good understanding of algebra and functions, and so on to diagnose the problem.

[00:29:45] Anna Stokke: Can you talk a bit about the Area C loophole? When I had Jelani Nelson on, we talked a bit about that. And just to recap a bit, and people can also go back to those episodes with Jelani and listen to more about that. The University of California system, those universities, have uniform high school requirements for admission for each of the subject areas and for math requirements that's referred to as the Area C requirement.

And you can say more about that if you want, but can you just tell us a little bit about that loophole that you discovered?

[00:30:20] Brian Conrad: Sure. So when I was reading the CMF, I noticed in, you know, in various places it was, you know, recommending taking Data Science instead of Algebra II. Of course, this was one of many things that set off my B.S. meter. I mean, I knew that was absurd advice. But then I saw somewhere where it said, “Oh, this has been, you know, approved by the University of California.”

And I couldn't believe that, like, that was just so completely ridiculous. And so I went into Google, and I looked up like, “What is going on?” Like how could the University of California have possibly allow Data Science a substitute for Algebra II? Because if somebody does that and does not learn Algebra II in high school, they are in a deep hole if they want to get a four-year quantitative degree at the University of California, which doesn't even teach Algebra II, even setting aside, like trying to get a college degree in four years, because you know, time is money.

And so what I discovered was that there was a committee that had set up a definition of advanced math and that had proposed the idea that some data science courses could perhaps satisfy the property, what they call validation. So for example, if a kid learns some math on their own, let's say Algebra II, and then they go off and take a calculus course in high school, then passing that calculus course can be used to validate, or substitute for Algebra II.

And this validation process was originally created for foreign languages, but it was also applied to math, simply so that, for example, a kid who was quite advanced in math or did things on their own could still fulfill the three-year math requirement, even if they didn't take the first three conventional years of high school math.

This idea that data science courses should also be allowed to validate Algebra II was very perplexing to me because I knew from looking at the curricula that the most popular high school data science courses have very little math in them. In particular, nothing from Algebra II that is not of the statistical topics.

And so this made no sense. So I referred to the following as the loophole. So I came up with this terminology. I, I realized that AP Statistics has an Algebra II prerequisite. And so I guessed that, gee, therefore, probably somebody figured that statistics should always validate Algebra II, even if it's not AP and well, data science is kind of like statistics so perhaps all data science courses should too.

Now this was just a conjecture of mine that because data science is kind of like statistics and because AP Statistics has this requirement that somehow it led to this loophole where just somebody who was rubber stamping, largely math-less, or math-light data science courses to validate Algebra II under this bogus, analogy.

But I figured I would never find any proof of my conjecture. But then the proof emerged that there were these emails that were some other people obtained through a California Public Records Act request between one of the CMF writers and this guy named Robert Gould at the university at UCLA, who's a statistics lecturer and who was the creator and lead developer of one of the two most popular data science courses.

And in an email from Rob Gould, he actually spelled out that this was exactly what happened. He was writing about getting certain data science courses approved, and he was expressing his concern that courses like his might not get approved, but he said, “Ah, it worked.” And then he spells out the back history, which was exactly what I had guessed.

And this was completely bizarre because the people who were advocating for this, like Gould himself, has to know perfectly well that a kid who follows this advice and does not learn Algebra II in high school will be totally off-ramped from any realistic chance at a four-year quantitative degree at UCLA or any other UC campus and at most colleges.

And you know, and just to be clear, I don't have a problem with high school data science courses and the people I know who share my concerns, nobody has a problem with the existence of these courses. They can be electives, just like high school economics. And alternatively, if a kid's going to take that and get re-motivated, then go back and take Algebra II, great.

But it's the advice that these courses are alternatives and they’re replacements. That's the huge problem. And the reason it's a problem isn't because, well, sure, there are some kids who will never pursue quantitative work, and for them, it may not matter so much, but you don't know who those kids are. And some kids' interests evolve early in college.

And the purpose of a well-rounded high school education is to prepare kids so that they can be ready to pursue any possible major when they get to college. And so giving people guidance that they can skip certain things in favour of other things without warning them at the very least about the very real long-term consequences of this, I think is, is thoroughly dishonest and is outrageous that it was included in the CMF.

And I'm very delighted that because of the raised awareness about this, that both the UC and CSU systems have recently revoked the approvals that have been rubber-stamped on these things, and they'll be looking more closely into clearing up what is the real meaning of advanced math to make sure that this kind of nonsense doesn't happen in the future. What was revoked was that high school data science courses can't de facto substitute for Algebra II.

[00:35:32] Anna Stokke: Okay. That's good.

[00:35:34] Brian Conrad: And in particular, like a faculty letter from the California State University system, was warning that a specific course, in fact, the UCLA course, you know, does not adequately prepare students for what they need for college and career readiness in quantitative directions. so those courses are now no longer allowed to validate for Algebra II.

I mean, of course, there's the coming year things will still be allowed because kids have already made their course plans. And as I say, both university systems, the University of California and Cal State University, will be looking more closely into their policies, but the faculty are now much more engaged in the process.

My impression is that the validations that were going on before were being done by people who were deeply ignorant about mathematics and did not understand the consequences of what they were doing. But that has now been stopped and hopefully it will stay stopped. And as I say, I mean, look, a good outcome could be that some people who create high school courses, maybe they're motivated to create an Algebra II course, which uses data science examples.

That's fine, content first, motivation and sources of examples can then support that. But the primary thing is making sure that kids learn the actual math and they don't get told these false tales that, you know, oh, “Data science is like math 2.0, a new kind of math, and it doesn't need any of that musty old stuff.”

This is completely wrong. And as I said before, there's ample room for improving the motivation, improving the applications that are at least indicated for the material that's being taught. But you know, if parents want to know what math is most important, what track through high school is most valuable to prepare for career in data science? And the answer is, it's the same math that you would take for any STEM degree.

[00:37:19] Anna Stokke: In general, people have to be really wary about things that sound really good. Where it sounds as though maybe you don't have to do that difficult math, you know, and you can do this instead, and you'll still be able to get like a really good stem career. Generally, these, things are too good to be true. I think that's sort of what's been happening with the data science classes.

They're being sold as it’s really trendy and hip to take these data science courses. They'll be fun, and you won't have to do that really tough algebra and you can still get a great career. So I think people have to be really aware that this isn't correct.

[00:37:58] Brian Conrad: And as I say, like it's certainly the case that there could be kids who get more motivated by such a class to then go back and persist and go through an Algebra II class because they see more value in it for their future interests. And so that's great to do. And as I say, frankly, the course materials that are given to teachers those should definitely be improved a lot.

Incorporate data science examples and other contemporary examples so kids do see the value. But this narrative going around, as I say, it's already implanted in Ohio, it was implanted in Virginia until that got stopped. And I'm sure it's spreading. I know that it's been popping up in Connecticut and other places.

And it, it, as I say, it almost happened in California, but we kind of put a stop to it. This idea that somehow there's this new kind of math is just completely a hundred percent false. The mathematical basis for all of the modern technology and these very quantitative careers is the same math that's always been taught.

And all that's really needed is an upgrade in the motivation, the upgrade in the examples, an upgrade in the range of applications. And of course, you can't do the full force applications of things, but I think like if kids just see a glimpse that, you know, “Oh, polynomial algebra is at work in QR codes,” you know, or, or how trigonometry is related to, you know, video games, like, at least it'll motivate them.

Be like, “Okay, this is valuable. I learned this, and maybe I can do other things.” Later they'll see a connection.

[00:39:27] Anna Stokke: You submitted your findings after the second draft, right? Were your recommendations adopted in the third draft?

[00:39:35] Brian Conrad: So most of them, many of them were, I should say. So most of the citation misrepresentations are fixed. But for example, you know, I had urged that they need to check because there was so much dishonesty, I said, “If you're gonna have all these citations, you gotta check all of them because nothing is reliable.” And I checked some things in the final version and that looked fishy and those were also misrepresentations.

So the problem is still there. Hopefully, they'll fix it somehow, even though the document has been approved anyway. So that was largely fixed. Neuroscience, in particular, it was largely gone. There was this other terrible thing in the early prior draft called the, well, I'll call the data science pathway or the MIC Pathway, MIC which stands for I think Mathematics: Investigating and Connecting. And this was a proposal that after Algebra I and Geometry, that kids would, you know, veer off and take two other courses that were kind of only nebulously described.

And it was all quite vague and it was, but it was obviously an off-ramp from any chance at a future in data science. And as I said, kind of in the spirit of what unfortunately has happened in Ohio. And, but fortunately, that has been removed. So, so anyway, those two fixes, the most of the misrepresentations in getting rid of this data science pathway, were very good improvements to the document.

Unfortunately, some other things remain. It is inconsistent about the value of eighth-grade Algebra I. So, for example, even though we had, San Francisco basically has run a 10-year experiment claiming magical equity will emerge from doing, you know, blocking eighth-grade Algebra I and this did not happen.

Despite the lesson of that, the document still, when it discusses eighth-grade Algebra I in one place, it talks about that this is valuable and it discusses a reference on how to try to implement it. But most of the, all the other places where this is brought up, it tends to advocate against it.

Although I've been told that the state board is committed to removing the opposition to this. And so I have sent them what I think is a comprehensive list of all the places where that's still there.

It still has an entire chapter on data science, which is full of hype and confusion. Basically because nobody involved in writing this has any expertise in data science. I had suggested they should actually appoint a panel of real experts in data science to give reliable guidance to parents and districts and teachers, but this wasn't done.

And in fact, the chapter five still communicates this bogus message that data science is somehow better suited than traditional math for girls and minorities, which is absolutely ridiculous.

[00:42:09] Anna Stokke: And insulting.

[00:42:10] Brian Conrad: And insulting. Oh, it's completely absurd. And it talks about how the data science field provides opportunities for equitable practice, to show the reality that all students can excel in data science fields. Well, sure, but you can say that about any, I mean, math also, like any field, just teach it well, give good motivation.

There is nothing special about data science in this. It's just, it is unbelievable. And I should say the document also has it's full of bloated writing, which makes it very hard to navigate. So parents who are trying to figure out like, “What exactly is my fourth grader supposed to learn?”

They're not going to find a list of that in this document. Publishers who are supposed to consult this document as guidance for their textbooks to be approved by the state in grades K through eight, they're also going to have to kind of navigate a zoo, just because the organizational structure is sort of so mixed up.

Prior California Math Frameworks and those of other states are so well organized. You can just, you want to know what's in Grade 2? You just look at the table of contents, and there it is. This thing largely shunts the discussion of content standards to somewhere else. And then it spends all of its time on some potpourri list of pedagogy topics.

And again, some of the suggestions are perfectly fine, but it's not clear what value this document is supposed to have. It's largely lacking, it's lacking in many actionable details. And as I said, there's this preference for certain ideological narratives and a lack of guidance from content experts on the overall process.

It's unfortunate that after four years and all this time and money spent on this whole thing, that we don't have a much better document. So to me, it's a big lost opportunity.

[00:43:53] Anna Stokke: I agree. And something you mentioned was that there were no experts in data science on the writing team, and yet they were really pushing data science. And, and actually, it seems that some of the writers didn't really know anything about data science, like Jelani, when he was on the podcast, he mentioned that one of the writers had written and said, “Are inequalities important at all?”

[00:44:15] Brian Conrad: Right.

[00:44:16] Anna Stokke: Which is just absurd!

[00:44:18] Brian Conrad: Yes, completely absurd.

[00:44:20] Anna Stokke: Really displaying ignorance.

[00:44:23] Brian Conrad: Yes.

[00:44:23] Anna Stokke: This is a big problem. And so do you think then that going forward maybe it should be required that there should be end users of math, people who, you know, teach at math at the university level, or use math in their profession or, data scientists, maybe people like that should be on the CMF writing team?

[00:44:45] Brian Conrad: Yeah, so in general, I think, you know, I'm not sure exactly what form it should take, but there has to be more involvement from content experts, right? From both higher education and industry in the future. I think to avoid both the scope of misinformation and the misleading guidance that permeated this process.

And, you know, in fact, there was an open letter signed by nearly 450 California university and college faculty across all quantitative fields, calling out this fact that, you know, data science is not a replacement for Algebra II for the readiness to pursue quantitative degrees and careers.

There was no point in this process where there was ever a panel of actual content experts who were brought in to inform the writing. More specifically, as I said before, there was one mathematician on the five-person writing team, and there were some parts of the document, I should say, that were extremely well-written about the relationship between different areas of math.

I'm pretty sure those were actually written by that guy. But anyway, that's kind of beside the point. But I think, for example, the previous framework document had an appendix about financial algebra. They acknowledged the fact that the grade-level content standards don't specifically cover financial math.

But nonetheless, some teachers and parents might want some general guidance about what are the important things to be learning in math related to, finance and business during high school. And so the prior 2013 framework had an appendix, which spelled out here are some authoritative references and here are some, you know, general list of content standards that do relate to financial math and that these are the things you should focus on to that end.

And I think that, yeah, this document really should have had an appendix written by actual data scientists, people with expertise in this field to provide guidance to districts, teachers and parents about the real connections between the math curriculum and preparation for these careers. And that would've been a huge public service because at the present time, there are no generally recognized standards for what constitutes a good high school data science class in terms of math.

And now I've heard some people say that, you know, this is kind of impossible because the data science involves too much of an interaction of statistics, computer science and math. So it really should always be like an elective course, like high school economics, not a math course. And maybe that's the case.

Maybe people will come up with something better. But regardless, I think it was very unfortunate that the document, rather than having this chapter on data science written without any guidance from real experts, the chapter should have been thrown out and replaced with an appendix which had much more reliability by being written by people who have deep knowledge of data science and its connection to math and where the subject is going in the future.

Because again, the high school education is preparing kids for the jobs of the future.

[00:47:42] Anna Stokke: So what were some lost opportunities with the CMF?

[00:47:47] Brian Conrad: So first of all, well, as I was saying, I think providing some kind of reliable and informed guidance about the interaction of math with, you know, data science, machine learning and so on. That, that's something that could have been done, let's say as an appendix. I think putting more pressure on publishers to modernize motivation and examples.

So there's one chapter in the document, I think it's chapter 13, which is specifically the guidance to publishers. You know, if you want your course material to be adopted with state approval, which is a requirement for grades eight and below, then, you know, these are the things you must do. And there's one sentence in there which speaks about, you know, modernizing the motivation.

I think there should have been far more discussion about this point. Also making it much clearer that the usual content really does still remain the essential foundation for all quantitative college degrees and the many of these secure jobs of the future. Precisely because it's very, it is very easy for people to be misled due to the rise of computers, for example, into thinking that maybe a lot of that math that's currently or has traditionally been taught in high school is somehow obsolete now. You know and I know that this is completely false.

That the people who are getting those high-paying jobs are getting their good salaries because they have the strong knowledge of math that allows them to use computers in a reliable way and to design new quantitative models and to analyze and troubleshoot them.

You know, as I was saying before, there's a lot of confusion out there about the interaction of the math curriculum with many of these things. And so I think that's, again, something that this document really could have stressed much more clearly. And again, in a few places, like in one sentence it says, “Oh yes, if you want a data science career, you need to be on the calculus pathway.”

This is one sentence in this thousand-page document, districts aren't going to see it. So that information is lost amidst the sea of other things. And I guess the other thing, which again, I think I may have mentioned earlier, is when looking for guidance on how districts can improve their demographic outcomes, rather than appealing to what's done in other countries, which often have an educational system totally unlike the one in the US in terms of training, funding and so forth, that maybe to have looked closer to home, other states or cities which are having much more success than California and point to those as things to look to.

[00:50:15] Anna Stokke: What is an example of a state that's having more success?

[00:50:20] Brian Conrad: Well, first of all, I think most states have more success than California. I don't remember the exact numbers, but California, I think is, is below the median, far below the median, in its success in math, which is of course really bizarre because there's all this high tech industry around here. For example, as I think I may have mentioned before, like Dallas is a place that has rolled out an approach to improving equitable outcomes without kneecapping anybody.

And in general, I think the, you know, the goal should be to enable, and as I've heard the State Board of Education President insist upon the real goal is to enable more kids to be better prepared to access the advanced math in high school so they can be ready to pursue those more lucrative and secure jobs and college degrees. But that involves, you know, the root cause of the disparities goes back to the elementary schools. And so I don't understand why I, you know, to me, there's so much of this, I don't want to quite say it's optics, but so much of this window dressing on the high school end and not acknowledging that actually the root causes come earlier.

Adrian Mims has, has spoken well to this that, you know, it's hard work, really hard work to fix these demographic outcomes, but you have to go to the earlier grades. There's a program out of Purdue University School of Engineering, which has had tremendous success in getting more kids from under-resourced communities ready to be prepared for Algebra I in eighth grade.

We should be looking to places that have been having that kind of success rather than this fake equity of San Francisco. And what happened in the first version, the CMF was they touted San Francisco as this wonderful model to follow. Then when it became too embarrassing, then they simply removed the mention of San Francisco, but they still keep advocating this policy.

Don't let people take eighth-grade Algebra I, it's somehow bad. And as I say, the final version is kind of on both sides of the fence. They claim it will, in the true final version, be fully supportive of eighth-grade Algebra I and helping more kids to access it.

I hope that happens. the kids whose parents have resources will not be held back. They will keep marching ahead.

[00:52:30] Anna Stokke: There are a lot of issues here and a lot of problems with the CMF, the way it was written, maybe the process, the citation misrepresentations. What are some lessons learned here?

[00:52:46] Brian Conrad: So I think, first of all, I think if there was more involvement at suitable stages of the process, more involvement from people with deep content expertise that could have flagged a lot of the misguided recommendations. And as I say, many of the misguided recommendations were fixed, but again, the process should not have taken as long as it had.

So anyway, greater involvement from content experts along the way. The conflict of interest laws need to be strengthened. You know, I mentioned before that, you know, Feynman in the sixties, was on the California Textbook Commission for Math. And at that time, I mean, he encountered massive corruption, in the process.

And, you know, the current laws prevent that from happening, but there are other ways that have become apparent in which significant conflict of interest can permeate the process. And, so I think the laws need to be tightened up to make sure that that aspect, to the extent it occurred this time around, never happens again.

And similarly, and this is in some ways partly repeating what I was saying before with the role of content experts, the definition of expert in the California Education Code is not really defined well enough. So, for example, there's this Instructional Quality Commission, which is supposed to, in some sense, look over some of these draft documents before they get sent out for public comments.

But here was nobody on that who had a math degree. Now, to be clear, I understand that it can be very difficult for the State Board of Ed, to get people with very advanced knowledge. Like they're not going to find a typical person who works at Google to, you know, step away from their job there and do this work for the State Board of Ed.

But nonetheless, there are a lot of university faculty out there, for example, some of whom do spend an enormous amount of time working with kids from under-resourced backgrounds and so on, and they can appreciate the difficulties that those kids may face, and yet also have the expert knowledge as to what these kids need to be able to access future college degrees. And again, to be clear, experts in pedagogy are essential to this process as well. K-12, experienced teachers are very essential.

They have the real kind of on-the-ground knowledge about what's realistic for kids to learn in a given year. To give another example to partly come back to the lost opportunity aspect, the current high school curriculum is too packed. There's too much material per year. The teachers all know this. Why were the content standards not revisited and trimmed in suitable ways to make it more feasible to fit into a year? I don't know why. So that should be done too.

[00:55:25] Anna Stokke: I think that the California Board of Education should call you the next time. I think you're the only person that read that, read over that document carefully.

[00:55:35] Brian Conrad: Yeah.

[00:55:36] Anna Stokke: So coming back to something you said, I don't know if you can be a bit more specific, but you mentioned the conflict of interest laws. So the implication is that there were some conflicts of interest involved here.

[00:55:48] Brian Conrad: Technically, there was not, for the following reason. So when Feynman was on the math textbook commission, that was for grades K through eight.

So in California, the schools must, can only use state approved materials for grades K through eight. And that is the part, that is the only part of the process that the education code covers for conflict of interest, nothing at the high school level. High schools do not have to get their course materials approved and therefore that falls out of the realm of the conflict of interest policy in the state education code.

And very important, the part of the process prior to the textbook adoption phase, such as the writing of the framework, which comes before the adoption of textbooks. This is not covered by the conflict of interest law at all. So, for example, if you look at section 9514 of the California Education Code, there is nothing in that code that prohibits an employee of McGraw Hill from serving as a CMF writer.

Nothing. So the definition of a conflict of interest really needs to be broadened to include the entire process. It needs to include people who may not, maybe they're not paid money by a publisher, but they have some kind of financial incentive behind promoting certain narratives.

To put this in a broader perspective, people who have any role in the development of professional development for teachers or in the development of materials that will be used in the public schools should not be serving in any part of this process.

This state has thousands and thousands and thousands of teachers and people who are experts in education and pedagogy, and content. There is an ample supply of people who are involved in that, who are not involved in the creation of course materials for the public schools or in the running of professional development procedures and so on.

And the state board should really be applying much greater scrutiny to the incentives that may lurk behind some of the people who are appointed to the process.

[00:58:05] Anna Stokke: So what should parents and students be aware of going forward?

[00:58:09] Brian Conrad: So, as I mentioned a couple of times, you know, there is ample room for modernizing the motivation of the usual curriculum. But they should be aware that even if in their district the material is perhaps not being taught in the most engaging way, and again, hopefully over time this will be improved, that the best mathematical preparation for readiness to pursue four-year college degrees in not just STEM fields, but in data science, economics, other quantitative fields, remains the usual math curriculum.

This is what's assumed as the background for students coming to Stanford, going to Berkeley, for the UCs, and the Cal State Universities, that there's no new math 2.0. That no matter what hype is being propagated around the country. The foundational math that you need to be prepared for quantitative coursework in college, what will give that college degree value is the conventional math content.

And the fact that content might not be presented in the most engaging way is something that needs to be improved, but that does not mean the content is obsolete. Computers have not rendered algebra and functions an obsolete topic to understand, to be able to be a confident user and creatively apply the technology in the future.

[00:59:29] Anna Stokke: So, is there anything else you want to add today?

[00:59:32] Brian Conrad: I hope the next time around, the process is a lot simpler than it was this time. Yeah, and I should say that I do hope that other states take the right lessons from this experience. That they don't make the mistake that Cambridge, Massachusetts has now made, and hopefully, the balance between data science and math can be more broadly understood, and these don't need to be in conflict.

And I hope that going into the future, a better array of courses can be offered that will help students to appreciate the, you know, the beauty and power of mathematics and seeing its relevance both in data science and in many other contemporary contexts.

[01:00:11] Anna Stokke: I want to thank you so much for going through that document and checking the references. I think you did everyone not just in California, a service. I would say all of North America because these things do trickle into other places and we'll be seeing this in Canada before we know it.

And I think, we should all be careful when, you know, we're looking at a curriculum document or an education document and something doesn't seem right. We should really check the references and check that things are being cited properly. And like you said, it wasn't just one case, it was -

[01:00:48] Brian Conrad: Pervasive.

[01:00:48] Anna Stokke: Pervasive. Throughout the document. So I think we will all have learned a lesson from that. And hopefully the CMF writers have learned a lesson too.

[01:00:57] Brian Conrad: Yes, yes.

[01:00:58] Anna Stokke: So thank you. Thank you so much for coming on my podcast today. It was such a pleasure to talk to you, and thanks for taking the time to talk to me today.

[01:01:06] Brian Conrad: Thanks a lot. Yeah, I enjoyed it.

[01:01:09] Anna Stokke: I hope you enjoyed today's episode of Chalk and Talk. Please go ahead and follow on your favourite podcast app so you can get new episodes delivered as they become available. You can follow me on Twitter for notifications or check out my website annastokke.com for more information. This podcast received funding through a University of Winnipeg Knowledge Mobilization and Community Impact grant funded through the Anthony Swaity Knowledge Impact Fund.