Ep 18. Education myth-busting with Daisy Christodoulou

This transcript was created with speech-to-text software.  It was reviewed before posting, but may contain errors. Credit to Jazmin Boisclair.

​

You can listen to the episode here: Chalk & Talk Podcast.

​​

Ep 18. Education myth busting with Daisy Christodoulou

 

[00:00:00] Anna Stokke: Welcome to Chalk and Talk, a podcast about education and math. I'm Anna Stokke, a math professor and your host.

 

You are listening to episode 18 of Chalk and Talk. My podcast recently surpassed 20,000 downloads. I want to thank my loyal listeners and to encourage you to please continue to share the podcast. I've got some phenomenal upcoming episodes planned with some amazing guests. The next episode will be published on November 3rd.

 

In this episode, I had the pleasure of talking with Daisy Christodoulou. I have been following her work for several years, so I was thrilled when she agreed to come on the podcast. Daisy is a teacher and an author and currently the Director of Education at No More Marking. This episode is all about debunking education myths, which is something Daisy is pretty skilled at.

 

We discussed the myth that facts prevent understanding and the importance of content knowledge. We discussed some common myths about exams and whether exams actually help students learn. I asked Daisy for her thoughts on a practice called ungrading, which I've been seeing crop up in some school districts.

 

We talked about how to acquire knowledge and remember it. We discussed the myth that the 21st century changes everything. And I asked Daisy why education myths persist. Daisy is extremely interesting and knowledgeable, and she's really mastered the skill of critical thinking. I had a great time in this interview, and I hope you enjoy it.

 

Now, without further ado, let's get started.

 

I am delighted to have Daisy Christodoulou joining me today, and she is joining me from London, England. She is a teacher, and she is the Director of Education at No More Marking, which is a provider of online comparative judgment software for schools. Before that, she was Head of Assessment at Ark Schools.

 

She has worked as a secondary English teacher in London. She is the author of three books about education: Seven Myths About Education, Teachers Vs Tech?, and Making Good Progress?, which is a book on assessment. She has been described as one of the 20 most influential figures in British education. She writes on Substack at No More Marking, and she hosts a podcast on the history of education called Lessons from History.

 

Welcome, Daisy, welcome to my podcast.

 

[00:02:50] Daisy Christodoulou: Brilliant. Thanks, Anna. Delighted to be here talking to you.

 

[00:02:54] Anna Stokke: So I have been an admirer of yours for quite some time now. I read your book, Seven Myths About Education, probably shortly after it first came out, and it really impacted me. I don't think there were a lot of books like that at the time, and it really resonated with me given some of the things that I was hearing in education.

 

You were a teacher when you wrote that book, and I'm just curious, what led up to that? Why did you decide to write the book?

 

[00:03:23] Daisy Christodoulou: Well, yeah. Well, thanks. Thanks. First of all, for saying you know, you find it useful. It's always lovely to hear. So yeah, I decided to write Seven Myths About Education because I was constantly hearing things about education from some quite big authorities in the UK.

 

They just didn't stack up and, to be honest, just didn't make logical sense in a lot of places and, you know, contradicted also a lot of evidence out there. And I started to read more and more about the evidence, and I was just astonished, yeah, just astonished at how so many of these big ideas that I heard all the time as a teacher just had so little evidence backing them up.

 

That was really astonishing to me, and still is, if I'm being honest. 

 

[00:04:05] Anna Stokke: I wonder if things have changed much, but probably not, right? 

 

[00:04:08] Daisy Christodoulou: I wrote the book ten years ago, and I think in the UK, definitely things have changed. So I don't want to mean, things have changed. I think things have changed for the better. But I still, I still think a lot of the ideas, the myths I talk about, they are quite persistent. 

 

They are still out there, and what's frustrating me at the moment is they seem to be getting a new lease of life with new technologies. So one of the myths I talk about, for example, is you can't just Google it. I feel like, you know, maybe there's been a little bit of success in people starting to think, “Oh, maybe actually you can't just Google it.”

 

And kind of AI and large language models and chatGPT came along. And now everyone's saying, “Well, maybe you can't Google it, but maybe we can just get our kids to look it up and chatGPT instead.” I'm like, “No, no, no. It's the same, you know, the same issue.” The reason why you can't Google it is the same reason you can't just look something up on chatGPT. So yeah, I think the, the last ten years, things have got a little bit better, but yeah, there's, there could be a lot better still.

 

[00:05:07] Anna Stokke: There's a lot of things I want to talk to you about. You have a lot of expertise in a lot of areas. But I think we better start with content knowledge. One of the myths that you mentioned in the book is the myth that facts prevent understanding, and content knowledge is often just devalued in favour of concepts.

 

And I'm going to read you a quote, actually, this is a quote from a recent description of British Columbia's new curriculum. “The redesigned curricula are described as concept-based and competency-driven. It places more emphasis on the deeper understanding of concepts and the application of processes than on the memorization of isolated facts and information.” 

 

And we also often hear things like memorizing math facts is harmful and unnecessary. So, can you comment on that? Why is content knowledge actually important?

 

[00:06:02] Daisy Christodoulou: Yeah, so that's what you've read out there is a kind of classic example of the genre. I quote a lot of similar examples in Seven Myths. And the problem with this idea, what you just read out, it's setting up an opposition: conceptual understanding and memorizing facts. And the reality is that we need facts to have conceptual understanding.

 

So it is kind of setting up a false dichotomy and what it's doing. The analogy I like to use here it is, it's like setting up an opposition between ingredients and cake. If you want a cake, you need ingredients. If you want conceptual thinking, you need facts, so that they're not in opposition, one makes up the other. There's a few ways in which facts really matter for conceptual understanding.

 

So, I'll give you a few examples. So, one really obvious one, like why does it still matter to memorize math facts? The reason why it still matters to memorize math facts, even though you can look them up, use your pocket calculator, is because we have very limited working memory. So our working memory is very limited.

 

So, the working memory is anything you can hold in information at any one time. And, you know, it's pretty accepted, maybe four, four to seven new items of information at one time. So when it comes to something like maths, and you'll, you'll see this yourself as an adult, if you're trying to do a maths problem with many stages in it, it quickly uses up that space in working memory. 

 

So the way you can get around this, there's no known way that you can expand working memory, but you can cheat its limitations by memorizing maths facts. So if you can fluently, you just know, so for example, you know, you're doing some kind of maths problem, and you end up with, I don't know, 9 over 18, if you just know there is a relationship between 9 and 18, that's freed up a really valuable chunk, one of these precious chunks in working memory. 

 

If you don't know that relationship, that's another thing you have to think about, another thing you have to work out. So the more maths facts you have, and the more fluency you have manipulating them, the more space you have, precious, precious space to deal with the things you don't know. 

 

So if you're in a shop and you're trying to work out if, I don't know, one particular three for two offer is better than a, I don't know, a buy one get one free offer, the, if you have those basic math facts, it's a much easier problem than if you don't, and if you don't have those math facts, you often just, you get stuck, you forget the beginning of the problem by the time you've got to the, the end of it.

 

So that's one example of why we need facts for conceptual understanding. I'll give you another one, and this is, people often think, “Oh, maybe it applies to maths. Does it apply to anything else?” I didn't teach maths; I taught literature. I’m a literature graduate. So I remember teaching Midsummer Night's Dream, the Shakespeare play. It's got quite a tricky setup of characters. It's got two women called Hermia and Helena, and their names sound very similar, they are quite similar, and if they fall in and out of love with two men all the time, and it's really confusing to keep track of what's going on. 

 

There's a really interesting, but yeah, then there's all this sort of, you know, it's a Shakespearean comedy so there's all this madness going on with love potions and people falling in and out of love and at the end of the play, one of the male characters, Demetrius, he is not in love with the woman he was in love with at the start.

 

He's had a love potion put on him, and he's now in love with a woman he was not in love with, and the love potions affect that haven't worn off. And so what is really interesting about this is when you get to the end of the play, and this becomes apparent, you can have a really interesting discussion with a class about, “Is it better to live in reality and be unhappy, or to live in a delusion and be happy?”

 

So Demetrius is like, he's living in a delusion now like he's got this potion on him, he's in love with someone who he wasn't kind of really in love with, but he's really happy. Whereas when he was not under the love potion, he was unhappy because the woman he loved didn't love him back. So if you get to that point at the end, you can have a great discussion, and kids really get engaged with it and love it.

 

But, if they don't get the kind of interplay of all these characters, and their names and who they're in love with at the start and how it changes, they can never get to that point. I taught Midsummer Night's Dream in a couple of different ways, and I taught it once where it just, you know, was a bit laissez-faire about understanding who the characters were.

 

And we got to the end, and there wasn't that payoff of having those rich discussions. And when I taught it with, basically at the start, drilling the characters, drilling their names, and drilling the kind of relationships between them at the start, by the time we got to the end, the payoffs were so rich, the discussions were so, like, deep.

 

And that cannot happen without some really, honestly, in some ways, maybe quite tedious basic facts that at the time feel a bit trivial, but when you see how they build together, you realize why you need them. So just giving you two examples there, you know, you can give hundreds, you can give hundreds. The analogy I like to use, I use my cake ingredients example, the other one I like to use is people often think about knowledge and skills as the pendulum. 

 

And they'll say, “Oh, you know, we mustn't tip the pendulum too far towards knowledge. Maybe it's gone too far towards skills in the past. We need a happy balance.” That is totally the wrong way of thinking about it.

 

It's the wrong metaphor. The better metaphor is that knowledge is the pathway to skill. Okay, so you have more knowledge and the knowledge builds on knowledge, and it deepens your conceptual understanding. So this idea that we're going to have conceptual understanding with no knowledge just doesn't hold true.

 

[00:11:50] Anna Stokke: Yeah, absolutely. And the way I like to think about it is like building a house, you need a really good foundation, and you kind of have to work your way up. And we've heard this sort of thing so much in math, for instance, like for years. I mean, you mentioned chatGPT, and I think this is probably causing a lot of trouble for English teachers, you know, we've had WolframAlpha and Symbolab and all these things have been causing trouble for math teachers for many, many years, and we'll often hear things like, “Computers can take care of the routine problems. So we don't have to spend time teaching students how to solve those routine problems or how to do calculations or anything like that. Instead, we can focus on problem-solving.” 

 

You know, it's really hard to get to a point where you can do problem-solving when you have nothing in your head to use.

 

[00:12:40] Daisy Christodoulou: So this really frustrates me, and you're right and in a sense you math teachers, you guys have been struggling with this for years and we English teachers, we're only just starting to get to grips with it. So this idea that we've got a machine that can, you know, do a lot of the basic stuff and maybe in some of the advanced stuff, so the kids don't have to bother with the basics.

 

They just need to know like how to operate the machine. Or like, you know, how to enter the right prompt into chatGPT or what to put into the calculator. But this is totally flawed because those advanced skills, so I agree those advanced skills are important, but those advanced skills depend on the more fundamental skills and there is no way you can leapfrog.

 

There is no way that you can jump ahead. Like if there was a way to say, “Well, I can just jump to this more advanced thing without going through the intermediate steps on the ladder,” then great, but that isn't possible. So, in order for students to be able to successfully grapple with the problems that the computers can't do, they're going to have to work through the problems that computers can do.

 

So, they are going to have to do stuff like times tables and sentence exercises that, yeah, WolframAlpha and ChatGPT can ace. That doesn't matter. Like, that's not the point of why we set tasks in school. The reason why we set them is to develop our students' thinking. And so you can't kind of skip or, or outsource these basics.

 

And the other analogy I really like to use here is chess because people seem to understand this intuitively with chess in a way they don't with maths and English. The chess computers started beating the best humans, you know, I think over 25 years ago now. I think Kasparov was defeated by Deep Blue back in 1998.

 

So you had a quarter century of chess computers being better than the best human. And they're now just way better than humans. Like, you know, Stockfish, AlphaGo, these are really powerful computers. Okay, so when a child wants to learn how to play chess, do we say to them, “Hey, you know, don't bother learning how the knight moves, you know, the computer can do that for you. What you need to do is focus on, you know, the advanced strategy the computers can't solve.”

 

Nobody says that. Everyone says if you want to learn chess, you need to learn how the pieces move, right? Like, that's the first thing you do when you teach a kid chess. You go, right, “These are the, you know, these are the rules, this is what happens.” And then they build up from that, and then they do like these little drills and they do some practice and then they can play games with other people at the same level. 

 

Nobody says, “Oh no, because AlphaGo can solve all these problems, don't worry your head with them.” Why do people understand that with chess? It makes much sense. It's exactly the same for English and Maths, right? If you want to get good at something, you've got to start at the beginning and work your way through it to be good at it.

 

[00:15:28] Anna Stokke: Yeah, and it's interesting because during the pandemic, we had to shift a lot of things online, including exams. And students would cheat, but we could often tell that they were cheating because they would copy something off of Symbolab and there would be a mistake in it, and they couldn't tell that there was a mistake. So if you don't know what you're doing, how would you recognize these errors, right? It just doesn't make any sense.

 

[00:15:55] Daisy Christodoulou: Yeah, exactly that. Yeah, and that's obviously the reason why, if you want to get good at something and you really want to understand it, you have to kind of work for it yourself. Otherwise, as you say, you won't be able to spot those errors or mistakes. You're just blindly copying something out.

 

[00:16:10] Anna Stokke: So let's shift a bit to assessment because that's obviously something you're really knowledgeable about. So I wanted to ask if you've heard of ungrading or sometimes called degrading. This is starting to gain traction in some Canadian schools. And so, for example, B.C. recently decided to remove grades altogether for grades K-9. And there's talk about expanding this up to Grade 12. 

 

So they're replacing letter grades on report cards with descriptors like emerging, developing, proficient, and extending, as determined by the teachers. And so some of the arguments we hear is that grades are dehumanizing the fixation on numbers, distracts from learning, tests cause rote learning since students just memorize the material, and of course exams cause too much stress and anxiety for students.

 

And so sometimes tests and exams are being replaced with things like self-reflections and projects or narrative evaluations where the teacher and student collaborate to come up with a grade together. So do you have any thoughts on this?

 

[00:17:21] Daisy Christodoulou: I have extensive thoughts, so this is my day job now. I work for a small assessment organization called No More Marking. We provide schools with online comparative judgment assessments of mostly writing, but we do a bit of maths and some other subjects as well. We work with a couple of thousand schools in the UK.

 

We work with schools in the USA. We work with schools in Australia. I don't think we yet work with any schools in Canada. So if there's any schools who would like to come and work with us, let us know. You know, we do, as I say, work with a lot of schools in a lot of other English-speaking countries. You know, this is, as I say, my day job now, like thinking about assessment, thinking about how we report assessments.

 

Thinking about what it is that an assessment means, and thinking about the best kind of conditions we can have assessments in. I would say first of all, I'm pretty keen on exams as opposed to teacher judgment. The reason I think that is that teacher judgment can mean a lot of different things.

 

Teacher judgment in the sense, you know, I think we had in England a few years ago, is we had kind of a large part of the national exam system was made up of coursework. And that coursework was kind of, it was almost sort of big homework pieces that a teacher would then assess and give the student a mark for, you know, it got abolished in the end, most of it because it's not a reliable way to assess.

 

And teachers are subject to all kinds of biases. There's all kinds of issues when you have teachers judging their own students. And it's not, this is not me having a go at teachers. This is an aspect of human judgment. There are lots of things about human judgment that is flawed, that is subject to bias, that's inconsistent.

 

So what exams try and do is strip out some of those issues. Particularly what's good about exams, you know, I'd say that the thing I really think is powerful about them is the way they have all the conditions standardized. So people talk about standardized tests, and they don't like the word standardized and they think it's soulless and dehumanizing.

 

For me, standardization, you know, another way of thinking about it, it's a form of equality. You know, standardization is equal conditions. what I think is really important about exams is the students doing the same task in the same conditions. And the problem you had particularly with coursework in England, was that it was completed in different conditions, there were a lot of gray areas, there were a lot of rumours that maybe parents were helping. 

 

Now you've got the chatGPT issue. So I'm generally in favour when I talk about exams, I think that the really big, you know, good thing about them is the standardized conditions, I think the other important thing about them is anonymous marking.

 

I think that's important too. I think that just is something that's, you know, fairer for everyone. It's often not nice for teachers to know that students who they like and trust and, you know, get on with that they've then got to make judgments about that are going to affect their future career. That's often not a great position for teachers to be in.

 

So I would say standardized conditions in anonymous marking are really good things and they're fairer, And then this vexed issue of grades, like what do we do with grades? So this might surprise you, Anna, but I'm actually not a fan of grades. I'm also not a fan of degrading from how you've described it.

 

But I'm not a fan of grades because grades, actually, it’s just arbitrary lines on a continuum. So I prefer to report what sits underneath the grade, and what sits underneath the grade typically is a scaled score. So with a lot of the work we do at no more marking, we like to report that underlying scaled score.

 

[00:20:59] Anna Stokke: A scaled score. So like a standardized score?

 

[00:21:02] Daisy Christodoulou: Yeah, yeah, yeah, so a standardized score, a standardized score. And the reason I prefer that is the problem with grades is they give you the assumption that students come in these neatly pre-packaged categories, and the worst kind of grades are three-part grading systems. And this is something that we have at primary level in England.

 

So up to age 11, our national system, there's a lot that involves three-part grading systems. And the problem with three-part grading systems is, student attainment is actually on a continuum. It's much more like age than it is, you know, something just coming in these neat categories, you know, so it's a smooth continuum. So putting kids into three categories, it's kind of like putting all humans into three categories, you know, young, middle-aged, old.

 

Like that's clearly quite distorting and where do you draw your cutoff? With age we understand just intuitively, don't we, that age is a continuum, you know, a difference of a day in age is not really that big a deal, but a difference of like 20 years, probably, you know, a bit more significant. Obviously, there are times in life where we do have to draw cutoffs on an age continuum.

 

You can't avoid it. So, you know, there's a certain age you can get a driving license, or you can drink, or you can get married. Yeah, you do have to sometimes draw cutoffs, but we understand that they're fairly arbitrary and they're just like a necessity. They don't really correspond to anything, you know, any massive, deeper meaning underneath them. 

 

Student attainment is the same. Okay, it's on a continuum. But by putting kids into grades, it's quite distorting. Because the kid at the top of one grade actually has more in common with the kid at the bottom of the next grade than they do with a kid in, you know, the other end of their grade. And you might think, “Oh, isn't that quite trivial, it's quite misleading.” Well, go back to my age example. If you just put people into three categories of young, middle-aged, old, And then you said, like, well, the cutoff's 18, so the minute you're 19, you're middle-aged. That would be quite distorting, right? It would be misleading.

 

So as I say, where possible,  I like to report the underlying scout score. 

 

[00:23:14] Anna Stokke: So just to back up on that. Would you prefer to report, say, the percentage?

 

[00:23:20] Daisy Christodoulou: So a percentage is also, I'm not keen on percentages either because the problem with percentages, they're not standardized. And so the problem of a percentage is you can get 60 percent on an easy test, and you can get 20 percent on a hard test, and 20 percent on a hard test might be better than the 60 percent on an easy test. So, I'm not that keen on percentages either. I'm keener on a way that takes a percentage and does something to it, transforms it, to standardize it.

 

That's hard to do at a class level. I understand that. It's hard for an individual teacher to do. So, you know, obviously I'm talking about some of the bigger assessments you get here.  But, you know, even when you're at a class level, I still think it's helpful to think about that because ultimately, that is useful information, you know, that some kind of standardized score like that is, is useful and is helpful.

 

Now, the traditional opposition to scaled scores is that they're very hard to understand. The way we get around this, and not just us, but lots of other organizations, is reading ages. So reading ages have been around for years.

 

And what I love about reading ages is that they take a continuous scale that we're all very familiar with, and apply it to one that we're less familiar with. Now, obviously, that as well will have its distortions. You know, any kind of analogy is not perfect. You know, all models are wrong, but some are useful.

 

But I think that comparison is quite helpful. And what we've done at No More Marking is we've applied the same principle to writing. So we think we've come up with the first writing age anywhere in the world, but it's very, very similar principles to a reading age. This idea that there is a continuum of performance. 

 

If we express it in terms of age, people will intuitively understand it more than if we express it as a scaled score. So then your point about grading and your point about these schools that are replacing, is it A, B, C, D, E, with emerging, expected, whatever. Okay, so what I would say is I don't love grades, but if you're going to have grades, I prefer to have letter grades.

 

And I prefer to have about 8 or 9 of them. So if they've got to exist, I would say maybe an eight to ten-part letter grading system is the least worst option. When people start replacing letter grades with emerging, expected, exceeding, which people have done in the UK, and it's amazing how Canada and the UK different countries, but I sometimes feel, you know, so many different countries, different policies, completely different kind of systems and yet people come up with the same ideas. It's like, there's a hive mind. 

 

So there's loads of people in the UK will say, oh, exactly what you said, “Let's replace the A, B, C, D, E, F with emerging, expected, exceeding.” I'm like, let's just stop and think about this. All you're doing, emerging, expected, exceeding is just grading by another name. You know, you're still putting kids into three or four categories. You're just making it harder to understand what those categories mean. So you're making a flawed system worse. And on top of that, it's really interesting where you say, “Oh, labelling kids with numbers and letters is really dehumanizing.”

 

I actually think labelling them with emerging, expected, exceeding, proficient is far worse. And I'm not alone in this. So the assessment writer, the assessment academic Daniel Koretz, he's a professor of education assessment at Harvard University. He writes about this extensively from a kind of, you know, real assessment of science perspective.

 

And his point is, you know, you get these terms like proficient, emerging, whatever. He's like, when you, when you call something proficient, proficient has an everyday meaning that people kind of understand. When it's applied to assessment, it's just an arbitrary line on a scale. So, you know, it's really, I'll just quote from him actually, he says this, “ Even if one were interested only in the binary distinction between proficient and not proficient—which in my experience few people actually are—the complication is that ‘proficient’ is merely an arbitrary point on a continuum of performance; it does not indicate mastery of all of a discrete set of skills.”

 

Yeah, it's just, it's just a line. So imagine a distribution and a line on the distribution, and we're saying, well, the kids on one side are proficient and the kids on one side aren't.

 

It's, you know, it's incredibly misleading. To go back to what I said before, two kids either side of the line have got more in common with each other than they have with kids who have the same label as them. So I would say if you have to have grades, it's much fairer and much like less dehumanizing to just give them quite neutral numbers or letters.

 

Once you start using words that have an everyday meaning, like emerging, proficient and developing, I think that's, that's worse. And I also think it's really misleading for parents and for the wider public. You know, it doesn't correspond to any kind of underlying reality. So I've gone on about this at length because I feel quite strongly about it, because I've also seen, this isn't just an academic point of me trying to be pedantic.

 

I have seen this problem cause real issues. You know, I have seen, like, real children end up getting kind of the wrong advice and getting the wrong kind of instruction and getting, you know, wrong assumptions made about them because of what is a really clunky and misleading assessment system.

 

[00:28:48] Anna Stokke: Also, you mentioned that having exams, like it's a standard thing, everybody gets the same conditions, it's better for equality, I'll hear the opposite. So I'll hear things like people saying, “Well, all you can tell by a test score or a standardized test score is where the students with low socioeconomic status live.”

 

People will say things like that. And that actually that grades are unfair to disadvantaged students. What do you think about that?

 

[00:29:23] Daisy Christodoulou: I think it's like, you know, what Winston Churchill said about democracy. Yeah, it's the worst system apart from all the others. So yeah, sure. Exams are the worst system apart from all the others. So if you care about fairness, you'll, of course, there's a link between exams and, yeah, socioeconomic status.

 

I'll tell you what, there's an even bigger link between teacher assessment or non-exam assessment and socioeconomic status. Right? So look, I don't know enough about the context of Canada yet, but there's, there's a global research literature on this and there's been a few, there's, there's a huge amount happening in America about this at the minute.

 

I find it astonishing the extent to which–so some fascinating stuff about American Ivy League university entrants is that the differences between the high-income earners and the lower-income earners on their SAT score is not that huge. The difference on the extracurricular activities and the personal statements and all the things that, you know, parents can pay for the difference between those for the high, the wealthier pupils and the less wealthy is much bigger. 

 

So the New York Times had a huge, huge set of graphs about this the other day, right? What we saw in the UK a couple of years ago in COVID, we couldn't run our national exams as normal. So it went to a system of teacher assessment and Independent schools, where wealthier children go to, they all had these teacher assessment scores way in excess of what they were achieving before and way in excess of what was being achieved in the poorer schools.

 

So this is one of the most pervasive myths about exams. People think exams are unfair on poor kids. Exams are the fairest system of all for poor kids. As I say, a huge, huge literature on this in many different countries. I don't know enough about the Canadian system, but you know, this is not just the UK I'm talking about, this is lots of countries. 

 

And it makes sense when you stop to think about it. You know, we hear so much nowadays about kind of unconscious bias and just biases in general. Well, the reason why exams were invented are as a way to try and get around some of the unfortunate, innate biases that we all have.

 

And, as I say, they do that by standardizing the conditions and by anonymizing the assessment. And those are two really important principles of equality. Another really fascinating study, really big study done in England a few years ago, there was a particular assessment point in England where children were being assessed both by exam and by their teacher, and it found that poor students did better on the exam than they did on the teacher assessment.

 

So again, and again, you see this pervasive idea. Everyone, even teachers, will think the teacher assessment is fairer on the poor kids. It’s not. And I've written, you know, quite a bit about this. You can, you can look on my blog and in my book, you know, there's, there's lots on this.

 

[00:32:10] Anna Stokke: And I also take your point about, you know, they're replacing letter grades with emerging, developing, proficient and extending. And so if I were a teacher in that situation, what I would be thinking is, “Okay, what percentage, what's going to be the cutoff for emerging? What's going to be the cutoff for developing?”

 

It's just exactly the same as assigning grades, And the parents are going to come and say, “You need to explain to me why my kid didn't get extending.” You're going to have to explain that, right? And how are you going to explain it?

 

You're going to explain it based on some percentage grade. 

 

[00:32:45] Daisy Christodoulou: Exactly. You, you have to always, whatever you're doing with grades, you always want to have some understanding of what sits underneath them, which is, as I say, a continuous scale you know, it's a continuous distribution.

 

[00:32:59] Anna Stokke: And the other thing we'll hear are things like, “Well, when students have tests and exams, we're promoting rote learning, they're just studying to memorize the stuff for the exam, and they're not actually learning.” Do you think students learn when they're preparing for tests?

 

[00:33:17] Daisy Christodoulou: Yeah, I do. So there's a lot of research on this as well. The testing effect shows that the act of retrieving something from memory actually helps to build the memory. So, testing is not as divorced from learning as people think. Now, I do want to be cautious here because there are ways that tests go wrong.

And I've seen those ways in the UK, I read a lot about the No Child Left Behind in America, and I think there were big problems with that. As I say, this professor of assessment I talked about before, Daniel Koretz, he is very good on a lot of these things. So, I do think tests can get misused, and I do think there are ways of doing test preparation, which maybe don't lead to learning and there are poorly designed tests and there are issues with that. 

 

But the fundamental principle that the act of retrieving something from memory helps to build it in memory, it just shows you that there is something about testing which can help you learn. And if we can make you know, make sure our tests are designed well and our instructions designed well to take advantage of that, that's really powerful.

 

And most students don't know that or don't understand it. So, there's all these surveys done of “how do students typically revise?” They typically revise by, or, you know, revising, you know, it's a sort of English term, isn't it? You know, when you're studying for an exam, when you're prepping for an exam how do you study? Most students, they re-read their notes and they highlight them. And that's the least effective study strategy. The most effective study strategy is to test yourself. 

 

So, you know, cover up the book, cover up the notes, and quiz yourself on what you've, what you've read. As a pure principle, testing can help with learning. I don't want that to, you know, read as I'm completely in favour of any kind of test strategy because I think there are issues with the way tests can get implemented. But the basic principle is tests can help you learn. And the reason they help you learn is, the thing I was talking about at the start, is that they help you get things into long-term memory.

 

And storing things in long-term memory is really powerful and helps you cheat these limitations of working memory. Tests, tests are not inimical to learning. I sometimes like to call myself a pro-exam romantic. And what I mean by that is I am very pro-exams for all the reasons I've talked about. I'm not someone who's like only pro-exams because I think, “Oh, they'll give you a bit of paper that will get you to the next level in life and get you the job you need.” The reason I say I'm a pro-exam romantic is I think they really can help you learn. They really can help you understand things better.

 

They can really, you know, just, just even if you were learning something for its own sake and you didn't care about the next stage or the job or the salary, an exam would still be useful. You can put this into practice in your own life. Like flashcards. There's this flashcard app called Anki, which has got a bit of a cult following. 

 

And a lot of, you know, full-time students use it to try and, when they have to remember large bodies of fact. A lot of medical grads, a lot of medical students use it, a lot of language students, you know, you've got to learn a lot of vocabulary. So, people who are doing studies where you have to learn a hell of a lot of stuff will often use flashcard apps like Anki.

 

But you can use it for things in your own life where you're not bothered about necessarily the career or the job you're going to get. You're just interested in something and you want to understand it better. This is what I say, the phrase I like to use pro-exam romantic.

 

[00:36:42] Anna Stokke: Yes. I'm also pro-exam. So I think, you know, a lot, I think you have a lot of knowledge in your own head because I saw that you were the captain of the University Challenge team, right? And you won in 2007, you were, you were actually called Britain's smartest student. 

 

[00:37:00] Daisy Christodoulou: Do you have that in Canada? 

 

[00:37:03] Anna Stokke: No, no, I watched it actually, I was super impressed with that. So that's amazing. Can you explain it? University, what University Challenge is? 

 

[00:37:13] Daisy Christodoulou: It's a very niche British thing. It's a TV quiz show for university students. It's been running for like a long time now. It's one of the longest-running shows on British television. And basically you have teams of university students who, a team of four, and they have this kind of knockout quiz format.

 

It plays for half an hour on a Monday night on the BBC, and yeah, it's a knockout between all these different universities. And then, you know, yeah, the team at the end who wins all the knockouts kind of wins the trophy. So yeah, that was years ago now. But I won it with, my university, which was fun.

 

[00:37:49] Anna Stokke: Yeah. And there was a question about Euclid's postulates, right? 

 

[00:37:53] Daisy Christodoulou: Which I didn't get right. 

 

[00:37:54] Anna Stokke: So that's one that I would have gotten right. 

 

[00:37:56] Daisy Christodoulou: Right? So the, the debate about University Challenge, there's a constant debate. As I said to you, I'm not a maths person, I'm a literature person. I'm the humanities. There's a big debate with University Challenge about are the questions biased towards the humanities or the sciences?

 

And, like, traditionally people would say, “Oh, they were really biased towards the humanities, and the mathematicians and the scientists don't get a fair deal.” So you'll have to watch it to decide your own opinion on that, but we definitely had a maths and science specialist on our team because, yeah, I wasn't getting all of those questions.

 

[00:38:27] Anna Stokke: So can you give us some tips? So what are the best ways to acquire knowledge and remember it?

 

[00:38:34] Daisy Christodoulou: So I would say when I was on University Challenge, I was, it was a long time ago now, and I was much younger. I didn't kind of have a flashcard system in place then, but I do think this flashcard, a lot of the stuff I read about flashcards, I think, I do think they're tremendous and I know they can be a little bit divisive, and even some people who are really into learning are like, “Oh, they're a bit too simplistic.”

 

But flashcards combined with space repetition are just amazing. And space repetition, honestly, even if you're not like that into it or convinced by it, just give it a go. So what is space repetition? So space repetition is this idea, you know, backed up by a lot of evidence that if you want to remember something, if you want to memorize something, the best way to do it is to set up a schedule where you don't cram, so you're not cramming all your study of it into like one night or a week.

 

You want to space your practice out over time. Yeah, if you have three hours to learn something, you are better off instead of having, you know, three one-hour sessions on a Monday, Tuesday, Wednesday, you're better off having like, I don't know, nine 20-minute sessions kind of over time.

 

And not only that, so that's one part that you want to space it out, but then you can get more sophisticated with these space repetition algorithms. What they're trying to do is to gradually kind of increase the distances between exposures of the same material and the idea is you almost the best time to study something. the most you know, the most useful time is just as you're about to forget it. That's when you'll get maximum benefit from restudying it. 

 

So there's a lot of apps out there which have some element of a space repetition algorithm built in. As I say, the one I really like is Anki, and Anki works on this principle. It's a flashcard app, but it will disrupt the flashcards to you on this space repetition schedule. Put really simply, say you want to learn some, you know, a hundred foreign language vocab words, you would put the hundred in and you would learn all 100 like on day one.

 

And then, it would get you to revise them all again, maybe on like day three and then say you'd get some wrong. So then the ones you get wrong, you would see again like the next day, but the ones you get right, you won't see again for another week. And then the ones you get right out of that, you won't see again for like maybe another two weeks.

 

So if you keep getting something right, the spaces get bigger. If you get it wrong, you have to revise it more frequently. And all I would say is, you know, just try it out because the people who I know have tried it out. It's like magic. It's really weird in that things will pop up in a month's time and you will know what they are, and you have no idea how you know it will just pop into your head.

 

It is quite freaky how that aspect of it works. And so you can end up learning quite a lot of stuff in a really quite efficient manner. Obviously, people will say, “Well, you know, okay, it's fine for like, yeah, you know, maybe vote foreign language, vocab and, you know, med school stuff, whatever. But what about, you know, the more sophisticated concepts?” 

 

And I think the point is, as I said before, it's the smaller things that builds the bigger things. And then also, if you get to be innovative and interesting with your questions, you can also come up with questions that are, you know, testing some sort of different things as, as well.

 

So, I do think flashcards combined with space repetition are very, very powerful.

 

[00:41:46] Anna Stokke: I'm going to put a link to that on the resource page for this episode, and that sounds amazing. And someone needs to do that for math. If anybody has any ideas out there, they should make an app like this for math because we have the same issue with math, right? So we're teaching a concept, and the students will forget what they learn, you know, a month ago. But the thing with that too, is as the instructor, you're surprised that students forget it.

 

I know this happens, but I always forget that and I'm really surprised because they knew it before and it's gone. But it isn't for me, right? So that's why I think we have trouble relating sometimes.

 

[00:42:27] Daisy Christodoulou: So you've hit on a couple of really important things there. I mean, so first of all, there's this whole, like the understanding versus memorization point, which we touched on right at the start. But, you know, what you said there is really relevant to it as well. Because you see with maths a lot, and you get it with English too, but I think maths is a really classic, canonical example of this.

 

You have a student and at the start you teach, I don't know, something quite tricky like simultaneous equations, right? And by the end of the lesson, they really get it. You know, so at the start of the lesson, they didn't get it, and by the end of the lesson, they can solve a simultaneous equation on their own.

 

And you're like, “Yes, you know,” nowadays, like “the kid gets it,” and then they come back the next lesson, and it's like they've never heard of a simultaneous equation. 

 

[00:43:08] Anna Stokke: Yes, exactly. 

 

[00:43:09] Daisy Christodoulou: What can we say? Have they really truly learned it or understood it? Now, look, you know, in one way you can say that lesson wasn't a hallucination, like they did work through it and get to the end and get to a point where they understood simultaneous equations.

 

But this is why understanding is so fragile and needs to be reinforced with memorization is because well, but that's all just drained away, right? So this is it. If you don't like reinforce and have the repetition of memorization, it's like, you know, it's like running the bath with a plug out. You have to, you can't just rely on like having taught it once and then having got it once.

 

The researchers Kirschner, Sweller and Clark they say if nothing has changed in long-term memory, nothing has been learned. You know, they're quite hard line about this. You know, if you understand simultaneous equations in lesson one, but then you forget them the next day, well, to what extent have you learned them? 

 

Is there any useful way you can say you have? And then the other thing I thought was really interesting about what you just said was when you said, you know, you're always surprised by it. And that's another really common kind of cognitive bias that all teachers have, which is that we are the experts in a subject.

 

We know it really well. Often we've taught it a hundred times. It's really hard for us with the best will in the world to put ourselves in the shoes of a student who is hearing this for the first time. It's really difficult. Because you, you know, it's so well. And so it is always constantly a surprise often when a student comes back and has completely forgotten it because you know it all, and that is a noted cognitive bias and it's you can call it the curse of knowledge.

 

You can call it expertise-induced blindness. The things we know, we know that we're expert and we know them so well, we forget what it was like not to know them.

 

[00:44:57] Anna Stokke: And I also think it feeds into this other myth that you see it's proof that the way that you're teaching isn't working, that if students just discovered this for themselves, they would never forget it. Which is also nonsense when the problem actually is that that is just normal, it is normal for students to forget things quickly. You know, and if you keep reinforcing it, it comes back, Then they remember it.

 

[00:45:25] Daisy Christodoulou: So you learn something by discovery, you learn something like, cause you're told it, you're likely to forget either because we forget most of what we're told and what we discover because we're, we're forgetting machines, you know, where humans are kind of intrinsically like quite forgetful. And you can sort of see why if you remembered everything that happens to you all the time it would be completely overwhelming. 

 

Part of actually being able to think and analysis is forgetting stuff. Actually, you know, forgetting the right things and remembering the right thing. So you can't expect that the first time you hear something you're going to just remember everything, you know, straight away, that's not how it works.

 

And, so how do we remember things? Well, there's that lovely line from Dan Willingham that you've had, had on your show, “Memory is the residue of thought.” You know, we remember what we think about. So if we think about something a lot, we remember it. Now, obviously, there's lots of ways to think about something a lot, and it is true.

 

That, for example, things that are emotionally charged we're more likely to remember. It's true that stories seem to be psychologically privileged, so we're more likely to remember them. And those are all useful things to know if you're a teacher, that emotion and stories are privileged and that they can help you remember things.

 

But I think they can still only take you so far, you know, particularly with something like maths. You know, you can come up with a bit of a story for some of it, and that can help. There are limits to what you can do there. And even with emotion, you know, again, it's good to, you know, try and have things that will have the emotion and have that charge that you might remember.

 

But again, there's a limit to how can you do that sign of every lesson week in week out. The thing I like to, the sort of analogy I like to use here is, what are things we remember? You we remember things, yes, if they're explained well and if they're communicated, you know, you've got that emotion aspect, then you've got the story aspect, those are all important. But we also need to have the repetition.

 

And the analogy I like to make is like brushing your teeth, right? You can have the best technique for brushing your teeth in the world. If you only brush your teeth once a year, that's not enough So you have the best toothbrush and the best tooth-brushing technique and you can like, you know, get everything right and if you only do it once it's not enough. And that's the problem and that's when it goes back to then the blame that people start to get.

 

You can teach the most incredible, memorable lesson that has all kinds of bells and whistles and is absolutely fascinating and brilliant. And if that's the only time you teach that topic, yeah, it's like brushing your teeth once a year. 

 

[00:47:53] Anna Stokke: Yeah. You're going to forget it. 

 

[00:47:54] Daisy Christodoulou: Obviously, it is great to have engaging and fun lessons that catch students' attention and are interesting and, you know, make it more likely they'll remember, but on its own, that is not enough.

 

And then I think that what's so awful about that is, as you say, it then sets teachers up to fail. Because then it's like, “Oh, well, the reason they didn't understand it is your lesson wasn't memorable enough. It wasn't exciting enough.” And it's like, well, it could have been really exciting and memorable, but it, you know, it needs to be followed up.

 

[00:48:20] Anna Stokke: Students just need a lot of practice and they need to be reminded of things. It's good for us to remember that. Let's talk about the myth that the 21st century fundamentally changes everything. Here's a quote from an article I read just last week. “Why are we not teaching kids the math they will need in the world they're moving into?” 

 

I also attended a presentation just last week where someone said that “Gen Z students think differently and they need to be taught differently. They'll disengage if they're not given assignments that incorporate social media. For instance, they should be writing blog posts instead of essays,” et cetera. What do you think about that?

 

[00:49:04] Daisy Christodoulou: Yeah, so this is another one. Obviously, as you say, I've got a whole chapter on this in my book. You know, you're being great here. You're picking out all these people who are just repeating all these ideas. So yeah, one of the myths in my book, I think myth three is this idea that The 21st century fundamentally changed everything, and we have to prepare students for 21st-century skills.

 

So what is the problem with this? Yeah, where do I start? Several problems with this. What do people mean when they talk about 21st-century skills? They'll often define them as things like problem-solving, creativity, critical thinking. I do think all those skills are important. I'm less convinced when people say they are uniquely 21st century. Are we saying that before the 21st century, nobody needed to problem-solve or think critically or be creative? 

 

So, you know, one of my issues is like, what do you mean by 21st century skills? What exactly are these skills of the 21st century, you know, you have to have in the 21st century, but have never been needed for any other aspect of human history? And you said there, you know, one of the things was, you know, writing a blog post or whatever, and you're like, well, the fundamental aspects of that are basically, you know, there's a lot of that, the fundamental aspects of that, which are to do with writing, which are not new. So, you know, one of my issues is, like, what do we mean here?

 

Define your terms. What do you mean by a 21st-century skill? Then you get this idea. We can't predict what's going to be known in the future. So we can't predict exactly what's going to be needed in the 21st century. So, you know, we can't kind of weigh our students down with knowledge that might become outdated or obsolete in like a year or two's time. So a number of problems with that. One problem with that is there is an extent to which that might be true. 

 

In some kind of, you know, apprenticeships or industrial employment, that if you're being taught to use a particular machine or a particular, you know, coding language that it might be out of date within a few years. At the job level, there might be some truth to that. And at the job level, I can kind of see that. But in schools, that is just not the case. Okay, because for in schools, you're mostly concerned with teaching the fundamental principles, and the fundamental foundations of most disciplines are not getting disproved every five minutes.

 

Right? So we don't have to worry that if we're, you know, teaching our students basic maths, that that's all going to get overturned in a couple of years time. And that leads me on to like the further point of this, which is that weirdly, the more concerned you are about the speed of change in the economy and technology, the more concerned you are about that and the more you think that is a problem, the more you should be concerned with teaching traditional knowledge.

 

And the reason for that is that the traditional kind of knowledge and technologies are ones that have stood the test of time for so long. They've got the best chance of carrying on standing the test of time, and so many other technologies and inventions have been built on top of those.

 

And my favourite example here is the alphabet and the numbering system. Now, we don't think of those as technologies, but they are. They are two of the most important technologies we have. They are two technologies that ultimately, a lot of schools, you know, historically, they're set up to teach because they are kind of artificial.

 

They are inventions of late civilization. They are extremely powerful, they're not natural, you're not just going to naturally pick them up. And that's why so much of education is organized around literacy and numeracy. I am going to make a bet now, whenever anyone says you can't predict what skills are going to be needed in the 21st century, I like to say, well, yeah, I can, I'm going to predict literacy and numeracy. 

 

The great irony is if there's anything we should be skeptical about teaching, it's the cutting edge. It's the cutting-edge technologies because the newer the idea, the more likely is to become obsolete. So microfiche readers and mini disc players and fax machines they have more chance of becoming obsolete. So as I say, you know, the specific like industry and job related knowledge, yeah, you know, that does get outdated over short periods of time. So if you're working in industry and you're thinking about how you're bringing on like, you know, new grads or new apprentices and you're thinking about, you know, what they need to be taught.

 

Yeah, it is fair enough to be worried. If you're a teacher and you're thinking, “What is the curriculum I'm planning for my seven-year-olds or my 11-year-olds?” Then, actually, we do have a pretty good idea of what needs to be taught and what is valuable. So, this whole 21st-century idea, 21st-century skills, I think it gets things back to front.

 

You know, the older the idea, the more likely it stood the test of time, the more likely it's going to be useful in the future.

 

[00:53:43] Anna Stokke: Absolutely. And just to emphasize that, literacy and numeracy, and I agree with you 100 percent there. Those are the main things that we need to be focusing on.