Ep 17. Do timed tests cause math anxiety? with Robin Codding

This transcript was created with speech-to-text software. It was reviewed before posting, but may contain errors. Credit to Jazmin Boisclair.

You can listen to the episode here: Chalk & Talk Podcast.

[00:00:00] Anna Stokke: Welcome to Chalk and Talk, a podcast about education and math. I'm Anna Stokke, a math professor, and your host. You are listening to episode 17 of Chalk and Talk. Before we get started, I want to mention that I may not be publishing episodes as frequently now that the regular university term is underway, but not to worry. I plan to continue publishing high-quality episodes, and I have some amazing guests lined up to discuss a variety of topics.

My next episode will be published on October 13th. My guest in this episode is Dr. Robin Codding. She is a psychology professor who researches math intervention and assessment tools and math anxiety. She's also one of the founding members of the group "The Science of Math." I have frequently heard some math education, thought leaders state that timed tests cause math anxiety.

I have been skeptical about that claim, so I wanted to ask an expert whether there is research to support the claim. I asked Robin to unpack this for us. I asked her to explain the relationship between math achievement and math anxiety, whether it's important to include time practice in math class, how much practice is needed to become fluent with math skills, at what point students should be engaging in time practice, what causes math anxiety, and best ways to mitigate it.

And, of course, I asked her whether the claim that timed tests cause math anxiety is backed by research evidence. Robin is extremely knowledgeable and very careful to cite research studies that back up her statements. I will include a link to a resource page in the show notes, which will list some of the research studies mentioned in the interview.

I hope this episode helps to clear up some misunderstandings about time tests and math anxiety. Now, without further ado, let's get started.

I am delighted to have Dr. Robin Codding with me today, and she is joining me from Boston. She has a Ph.D. in School Psychology, and she is a licensed psychologist. She is a professor in the Department of Applied Psychology at Northeastern University. She researches math interventions and assessment tools as well as math anxiety.

She has authored more than 100 articles and book chapters. She is one of the founding members of "The Science of Math," which is a group of psychologists, cognitive scientists, and math educators who are advocating for evidence-based math instruction. And we will talk about some evidence-based methods today. Welcome, Robin. Welcome to my podcast.

[00:03:05] Robin Codding: Well, I'm so delighted to be here. Thank you so much for having me.

[00:03:10] Anna Stokke: Can you tell us how you got interested in math interventions and math anxiety?

[00:03:15] Robin Codding: Sure. The faculty in my doctoral program really focused on database decision-making in schools and linking high-quality, reliable, valid assessment with instruction and intervention. But a lot of the work in the program and in the department was on reading assessment and intervention. And I thought, “Hey, what about math?”

Both my parents were teachers. One was a reading teacher, one was a math teacher. So it kind of made sense to me to evolve in that way. Math anxiety, however, was a very recent focus of my work that began in about 2016, and it really stemmed from my doctoral students. Doctoral students in school psychology have to have practicum fieldwork assignments and placements in schools.

And they were having difficulty completing those assignments, especially those that were related to universal screening, progress monitoring, and even intervention delivery when the focus area was math. And so they came to me saying, “Hey, we need to look into why this is happening.” So, I was also curious, why is this happening, that all of a sudden these things are not permitted in the school?

And they came back to me and said things like, “Conceptual-only math,” “timed tests cause math anxiety,” and in the response to those ideas, they decided that universal screening, progress monitoring, and even some forms of intervention delivery were not permitted. So a lot of these things are what myself, Amanda VanDerHeyden, Sarah Powell, Corey Peltier, Elizabeth Hughes have all referred to individually and collectively as math myths.

So that's what really drove my recent interest in math anxiety.

[00:04:52] Anna Stokke: Okay, so the doctoral students were going into the schools, and they were trying to do screening, and they weren't permitted to give timed tests. Is that what was going on?

[00:05:02] Robin Codding: Yeah, and it's interesting, right? Because these aren't even, like, I do not even consider these in the realm of a timed test per se. They are time-limited. And there's lots of data on these tests that have been produced over years, decades to illustrate why the time sampling is how it is, right?

It's enough of a sample. to obtain the data that's necessary in order to make good predictions about the data. So instead, schools were opting for much longer assessments, and so all of this didn't make much sense to me per se, but that was what was lumped all together. So there was a lot of lumping of anything that had a time limit into this idea of time.

[00:05:42] Anna Stokke: And people were saying that timed tests cause math anxiety. But let's start with, you're a school psychologist, you're a licensed psychologist. So what is math anxiety?

[00:05:53] Robin Codding: Officially, if we want to think about it, math anxiety refers to the feelings of apprehension, tension, fear that interfere with performance on math-related tasks. And they affect one's capacity to solve math problems, of course, in a classroom kind of situation, but also in a real-world situation, so things like calculating a tip, working on your taxes, it could be anything, both classroom or real-world experiences.

And we usually think of two kinds of dimensions. There is a somatic dimension and a cognitive dimension. So we often see that individuals with math anxiety experience sweaty palms or rapid heartbeats, but they also could experience intrusive thoughts that lead to escape or avoidance behaviours. So the physiological or somatic piece is that dread and unpleasant feeling associated with being in a math situation.

And then the cognitive dimension is reflecting the worry, negative expectations, self self-deprecating thoughts about the situation.

[00:06:56] Anna Stokke: So it's much more serious than being kind of stressed. Where is the line there? How do we know the difference between regular stress and math anxiety?

[00:07:09] Robin Codding: I think that's a really good question, and I think one of the things we have to remember is that a little bit of stress is actually a motivator for performance. That's like the Yerkes-Dodson Law that you might have learned in Psychology 101. And that's actually been demonstrated with sixth-grade students.

Michele Mazzocco and a colleague illustrated that as well. That a little bit of math anxiety is actually productive. So you need to have some level of anticipation of the performance to perform well. It's when that performance is interfering with the execution of being able to produce the math skills that it becomes a problem.

Typically there's really only one tool that's utilized, it's a self-report tool. There's many self-report tools. There was a meta-analysis by Barroso and colleagues in 2021 that analyzed kind of all the different ways that math anxiety was assessed. And it is, you know, there's multiple self-report tools that are out there.

So that's how it's assessed. One of the most common ones is abbreviated as the M.A.R.S. (Math Anxiety Rating Scale), and it has degradations of high, low and medium math anxiety. So I think a lot of the studies that have attempted to look at what math anxiety looks like will look at these different levels of math anxiety.

[00:08:28] Anna Stokke: Now, if you have math anxiety, maybe that causes you to be less successful in math, but maybe it could go the other way around, right? Like maybe low achievement in math could cause math anxiety. So what is the relationship between math achievement and math anxiety? Which comes first?

[00:08:46] Robin Codding: There's three theories. The two main theories are that it could be unidirectional, right? That math anxiety is going to lead to poor math performance. The other theory is that poor math performance leads to poor math anxiety, but the theory that's getting the most empirical support at this point in time is that there is a bi-directional relationship. And so we seem to see from these meta-analyses that have been published, there are five different meta-analyses that have been published between 1990 and 2021, and they have illustrated that there's a small to moderate negative bi-directional relationship between math anxiety and math performance.

So, it means that higher math anxiety is associated with lower math achievement and vice versa. And it could be either way that we are seeing this relationship occur. However, there is a caveat here. There is some suggestion that the reciprocal relationship is not fully uniform. So, there's evidence suggesting that early poor math performance may affect later math anxiety more so than the other way around. And there are four or five studies that have illustrated that relationship.

[00:09:54] Anna Stokke: Just to follow up on that a bit, so early math performance could impact math anxiety later on. So if you're performing poorly in mathematics at a young age, you're more likely to be math-anxious later on. Is that what that means?

[00:10:10] Robin Codding: Yeah, yep.

[00:10:11] Anna Stokke: Now to the tests. So we hear a lot of claims and from, you know, some very popular prominent math educators that timed tests cause math anxiety. What does the evidence actually say about this? Do timed tests cause math anxiety?

[00:10:29] Robin Codding: Yeah, I think this is a really good one to unpack. There have been very few studies in the K-12 space that have actually explicitly tested timed and untimed conditions to see whether math anxiety increases or decreases in those spaces. So there's very limited data to illustrate that timed math tests cause math anxiety.

We do have a couple of studies that have been done. I referred to one earlier, Tsui and Mazzocco. That study looked at timed and untimed conditions with sixth graders who were gifted, and they showed poor math performance during timed conditions, but only in a particular order. So they only found that when the timed condition proceeded the untimed condition. They also found that the timed and untimed conditions were not significantly different when the untimed condition was presented first.

And they explained this as indicating that these skills, so there was a lot of complex skills on these math assessment tools that they were using, and they explained it as the students did better when they had opportunities to practice with the material before they were tested on the material. There's a lot of nuances to these particular studies as well, but that's one main finding of that study.

And then there was a second study. That was at the elementary level as well, and they looked at both accuracy and fluency outcomes, and they demonstrated that students actually did better when given explicit timing conditions, regardless of whether their math anxiety was low, medium, or high. So again, they used that math anxiety tool that I was talking about earlier.

They divided their group of students into low, medium, and high, and showed that all students did better under explicit timing condition in terms of improving their fluency. And they also showed that students that had that higher math anxiety overall had lower performance.

[00:12:30] Anna Stokke: Okay, so what I'm hearing is that there aren't really any studies that have shown that timed tests cause math anxiety and actually timed tests likely improve math achievement. So where do you think this claim is coming from? This seems to be pervasive. A lot of educators do think that you shouldn't give timed tests because they cause math anxiety.

Where do you think that's coming from?

[00:12:56] Robin Codding: I think it's probably coming from anecdotal evidence. It might be coming from thought leaders who are generating these ideas. And I think it's really the application of instructional tactics at the wrong time that might actually be leading to what teachers are observing in the classroom. So it really might not be a matter of the timed task per se, but when they're providing that time task and the instructional sequence according to what the learner's needs actually are.

[00:13:27] Anna Stokke: And when you talked about how your doctoral students were going into schools and they weren't even allowed to give the screening measures, I always say you can't fix what you can't see. I think you need to be able to see where students are struggling and what they're struggling with in order to actually do something about it. One of those things does have to be, in my opinion, fluency and this is just absolutely common sense to me, knowing a lot of math.

So all very surprising, but should we assess students with timed tests and why should we assess students with timed tests?

[00:14:04] Robin Codding: Yeah, I agree with you. I think it's surprising. And I had the same exact reaction and concern that you just expressed when my students came to me. I'm like, “Oh my goodness, how are we going to know what students need and how to direct their instruction? How do we know if there are classroom and school-level and grade-level gaps in their learning that need to be addressed, knowing that math is so hierarchical?

So I had all of these same concerns as you did in terms of “Should we assess students with timed tests?” I think I alluded to this a little bit earlier when I said there's lots of data to illustrate, you know, how much of time sample is needed in order to get reliable information on students' performance.

So we have a lot of those data, but we also have data that illustrates that timed tests actually provide critical information on whether students have mastered key skills and concepts because the metric includes rate of performance as well as accuracy. Once a child has reached 100 percent accuracy, The metric cannot capture the additional learning that occurs, and there's lots of learning that occurs beyond accuracy.

So, rate-based metrics tend to be more reliable and better indicate students' actual instructional levels, and Matt Burns and Amanda VanDerHeyden have done a lot of work in this area, and Amanda talks a lot about, you know, accuracy is only the tip of the iceberg in terms of reflecting what students have learned.

So, we need to capture that fluency component. We do it in every other area, right? We do it in non-academic areas, we do it in reading, we're really comfortable with reading fluency at this point in time. Whenever you're talking about music or sports. We think about fluent skill, foundational skills, right?

And we know that they, that those fluent foundational skills generalize to that performance that you need to engage in, whether you're participating in a game or a concert, et cetera. We really do need to have those timed tests. There's also some data from Amanda VanDerHeyden in Matt Burns that showed that rate-based metrics correlate more strongly with high achievement tests than accuracy scores, and they are more efficient as well and waste less instructional time.

[00:16:06] Anna Stokke: So I think it's becoming clear that fluency building and measuring student speed with math skills is an important part of math instruction, and that would include timed tests.

When my kids were in school, in elementary school, I noticed that they weren't doing enough practice and they weren't doing times tables tests. I wrote this op-ed piece for the local paper about how it's really important to practice in math.

It's like with sports or playing a musical instrument that you have to practice, learn your scales and get good at it. And I started getting these messages and even physical letters from teachers saying, you have no idea what's going on in the schools.

They are telling us that we're not allowed to give worksheets, we're not allowed to give timed tests. if I want my students to practice times tables, I've literally got to close my door and pretend I'm not doing that. I just couldn't believe it. Like as someone who's learned a lot of math, you know, the way you get good at math is through practice and you do have to be able to do things efficiently and, quickly in a lot of situations.

So that brings me to my next question. You couldn't actually do the screening measures. What about teachers every day in the classroom? And should they be including timed practice as part of their routine?

[00:17:30] Robin Codding: Yeah, this is actually my favourite part because the timed test is one element, and we don't assess students more than we need to, right? We are assessing them to identify risk status, and we're assessing them to monitor progress. But the most of the time we need to be focusing on their instruction. And so this is actually, I think, the most critical reason or component of this discussion is that we do need to be including timed to practice as part of the routine.

And it's been stated actually in documents and practice guides from the Institute for Education Sciences (IES), which is an arm of the U.S. Department of Education. So they've had a lot of experts come together over many years and suggest this very thing. So I'll just give a little history. The IES practice guide on RtI (Response to Intervention) in math in 2009 suggested that 10 minutes of fluency building for especially foundational and simple computation skills should be embedded in the intervention routines for students that need additional support.

And then that was updated in March 2021, the lead author on that is Lynn Fuchs. And they actually analyzed the existing data to illustrate that timed activities have strong evidence for building fluency in mathematics, and they based it on 27 studies that were characterized as having strong internal validity.

These studies all yielded positive effect sizes with magnitudes between medium and large. They were with students in grades kindergarten through grade six. They included class-wide supplemental interventions as well as interventions for students that are at risk or struggling with math difficulties, and they covered a wide range of content areas.

So counting cardinality, algebraic reasoning, whole number knowledge, computation, whole number magnitude, as well as rational number knowledge, computation and magnitude understanding.

[00:19:27] Anna Stokke: It almost seems like there's a disconnect between the research evidence and what's happening in the schools or what teachers are maybe being told to do by some leading math educators. On that note, why is it important to have fluency with some of these skills? I watched a webinar where you gave a fantastic presentation about this, and you mentioned that accuracy and fluency contribute to word problem-solving, pre-algebra skills, and fraction knowledge.

Can you elaborate a bit on that? What things should students be fluent with to set them up for success with word problems and algebra?

[00:20:12] Robin Codding: This is really exciting research, I think because we have replication in this area. Lynn Fuchs, who I mentioned earlier, replicated a previous study from 2011 by Carr and Alexeev, which I may be mispronouncing that second name.

And then there is a whole group of researchers from the University of Delaware who engaged in work on fraction learning and demonstrated very consistent findings about the importance of both computation accuracy and fluency and their unique contributions to both word problem solving, pre-algebraic skills, and also fraction understanding and learning, and that fluency actually contributed to, when it comes to the fraction studies, conceptual understanding as well as procedural understanding.

And sometimes, there's this idea that fluency only contributes to procedural understanding, but that's not the case in terms of what these studies have found. I love these studies because they are looking at similar things and then replicating what they were finding. And again, this was done with students in grades two through grades six across all of these various studies.

I think one easy guideline or way to look at this is by thinking about the National Mathematics Advisory Panel, which suggested that there are some key areas that are necessary to be algebra-ready. And that's the goal, right, of kindergarten to grade eight. And so those things include being fluent with whole numbers and being fluent with fractions.

Those are two key areas in order to be ready for algebra. And then there are also areas that they discussed in their report relating to geometry and measurement. Of course, geometry and measurement requires students to be accurate and fluent with their whole numbers as well as their rational numbers.

[00:21:59] Anna Stokke: And when we say being fluent with fractions and whole numbers, we don't just mean conceptual understanding. You need to know the computational piece, because sometimes I think too much emphasis is placed on the conceptual piece, and not enough time is spent on the computational piece, and what I would say is that if you can't work with fractions, if you can't add fractions quickly and multiply and divide fractions quickly, algebra is going to be impossible for you.

I don't care about the pictures when it comes to that because it's just going to bog you down. You cannot forget that fluency piece because it's really important to be able to do math later. And you mentioned another thing that's interesting because I think a lot of times people think that you have to do the conceptual piece before the fluency piece.

Anecdotally, I've learned a lot of math and sometimes I didn't have the understanding piece until after I built the fluency for myself. And I'm glad you pointed that out that the evidence actually doesn't say that you have to have the conceptual piece before the fluency piece.

Is that right?

[00:23:11] Robin Codding: Yeah, I mean, that's right. I think we see it in a lot of different ways. We did see it in these studies that were done, this was by the research group out of the University of Delaware, that's Nancy Jordan and her colleagues. But we also see that Rittle-Johnson and her colleagues have a lot of really good studies that are explicitly looking at this relationship between concepts and procedures.

I had a doctoral student, Kristin Running, who did a very similar study. We tried to rule out the role of the curriculum, so we used a direct instruction curriculum, which we know is a more effective curriculum, in order to conduct this study. And she certainly found that the iterative approach seems to be the best approach if we're looking at both conceptual and procedural outcomes.

So I think we do have enough data leading in that direction that we can, that we can agree that that's the case.

[00:24:03] Anna Stokke: Interesting. So we just have to get that message out there. Now about practice, how important is practice in building fluency with math skills? And what happens when students don't get enough practice?

[00:24:17] Robin Codding: Yeah, it really resonated with me earlier, your prior observations that there wasn't enough practice happening in schools. We are definitely seeing that. I call it an instructional casualty when I work with students that struggle in math. And a lot of times, it is not the case that these students actually have a disability in math or dyscalculia, but that they are casually at their instruction.

In other words, they haven't had enough opportunities to practice in these foundational skill areas. This message has been started to be disseminated in popular press, which I think is great.

Where you have other researchers just simply saying, listen, the lack of foundational skills is why students are not progressing into higher-level content areas in math. These data are across domains. It's cognitive science, it's neuroscience, it's behavioural science. So we know that this generally is true if we think about how students learn.

When students are fluent, they can retain and transfer their skills, they demonstrate flexibility, and that's where they can get into novel problem-solving. So, it's critically important. my work is rooted in this notion of the instructional hierarchy. Sometimes it's referred to as the learning hierarchy by Haring and Eaton.

It was a framework or a heuristic described in 1978, and it suggests that learning kind of happens in stages, right? Students first have to acquire a skill, then they become fluent with the skill, then they have to transfer that skill and maintain that skill over time, and then they can adapt that skill to novel problem-solving.

And so that's how I think of it. So, if you don't have students engaging in practice in order to build fluency, you never really get to what ultimately we want from students, which is for them to be able to demonstrate flexibility in their thinking and to solve novel problems. It's hard to get there if you're not accurate and fluent, and the way to, for that to happen is with practice.

[00:26:14] Anna Stokke: It kind of breaks your heart in a way when you think about it because I think because sometimes students aren't getting enough practice, they really think they'll never be able to do math.

They think in their minds that they're just not the type of student that's going to be able to do math when in reality, they just need that practice. And some people do need more practice than others. I think it's really important that teachers spend a lot of time making sure that students practice. Should some of that practice be timed?

[00:26:46] Robin Codding: A couple things on your last point, which I think is important. One is we do vastly underestimate how much math practices need for students to master math skills and concepts. Matthew Burns has a really cool study in multiplication, illustrating this fact. He and his colleagues demonstrated that the average student in Grade 3 requires about eight and a half opportunities to practice a single math fact before mastering it. And that's more for students performing below average, it requires 11 opportunities to practice and less for students performing above average, about six. So that's just a single math fact.

So I think, you know, we really do, we are underestimating that opportunity to engage in those math facts. “How much is needed?” I think is is a question that we certainly need to be able to answer. And I think we can do practice in a couple of different ways.

I think of it as practice, guided practice, so, when students are first learning a skill, they are receiving opportunities for modelling, demonstration, guided practice. I think of it in terms of practice in isolation, so being able to practice a skill, you know, alone in terms of not being embedded in other word problems or not being embedded in multi-step problems.

Even a skill in isolation, like, when you practice with a fact card, I think then of skills also being combined or interleaved together as part of a cumulative practice opportunity. And then there's practice opportunities that can happen in the context of games. So I think we actually have to think about what our practice looks like.

And I just mentioned four different ways that we can construct our practice. And this practice is constructed according to where students are in their learning process. So we want more guided practice when students are acquiring skills. More timed practice when students are building fluency because we know timed practice produces fluency, right?

We already covered that and then it's those games and challenge problems that can be done to build generalization. And adaptation of skills so that cumulative review, and when it comes to cumulative review, I really would love to see more worked problems. So we actually present students with a worked problem, a partially worked problem, and then an unworked problem and have students work through problems that way.

I think the way in which we have previously conceptualized practice is maybe the problem. So it should not be the case that students are given multitudes of practice problems on a sheet. It just has to be well-constructed practice opportunities in line where students learning is. And for example, with timed practice, which we should cover more fully, when can you, when do you do timed practice, right?

You have to do timed practice, and this is clear in the IAS practice guide. When you are accurate. So if you are less than 90 percent accurate, you should not be engaging in timed practice activities. You should already be really, really accurate. It's not about building accuracy, it's about building fluency.

Those are really, I think, some key pieces on how we re-conceptualize what practice constitutes.

[00:29:49] Anna Stokke: So if a student is still struggling to even figure out what three times four is using repeated addition, that's not the time to be engaging in the time practice, right?

[00:30:01] Robin Codding: Exactly! And, you know, 90 percent or more of the time, when they're presented with a problem, they're able to produce the answer, that's not accounting for how long it takes them necessarily to produce the answer, right? But once they're accurate, we want them to be working on that speed as well.

[00:30:17] Anna Stokke: So, to recap, we tend to underestimate the amount of practice that students need. They need a lot of practice, and some of that practice needs to be timed once students are at that stage where they're around 90 percent accurate to build fluency. So timed tests really do need to be part of good math instruction.

The tricky part of that may be that the teacher has a lot of students in the class, and they're all at different points. So some of them would be at that point where they are at the 90 percent accuracy stage, and those students could move on to the timed part. But then there are other students who aren't at that stage.

So how would a teacher manage that?

[00:31:04] Robin Codding: Yeah, I think there's a lot of great instructional tactics that can be utilized that can be differentiated in that way. So, for example, if you are working on timed practice, you can do that with explicit timing worksheets or with flashcards, and while students are working independently or together on that task, other students that need to continue to build accuracy can use a cover copy compare sheet where a model is presented and then the student has to replicate the model.

So, you know, it's the same skill that they could be working on, but they're working on it in multiple ways.

[00:31:40] Anna Stokke: Let's go back to math anxiety a bit. What are some causes, particularly in terms of the type of instruction that's used, what is the best kind of instruction to use to mitigate math anxiety?

[00:31:56] Robin Codding: Yeah, so this is a great question, and those meta-analyses that I had mentioned previously, talking about the relationship between math anxiety and math performance, were really helpful in kind of illustrating the types of situations that, magnitude of that relationship was weaker or stronger.

And so, really, what they found is that the relationship between math skill and math anxiety is stronger when working on complex math tasks and when math performance measures impacted students' grades. So that gives us some indication on what might be related to the actual cause of math anxiety.

There have been really very few studies that have looked at those specific causes and I will mention two that I've been involved in in a minute, but I just want to give a little bit of background first, so we definitely need more empirical evidence. There's also been studies that have just looked at general factors that could be involved with the relationship between math anxiety and frameworks that are associated with thinking about what creates math anxiety.

Luttenberger and Lau, Lau et al. 2021 and Luttenberger in 2018 suggested that there's individual factors that include things like genetics, self-efficacy and concept, motivation, work and memory capacity, environmental factors like school culture, classroom environment, instructional tasks, which you're asking about, and interpersonal factors, they could be all in combination in contributing to math anxiety.

When it comes to instructional factors, my colleagues and I have looked at this in two different ways. So I was part of a study led by Kathrin Maki and Anne Zaslofsky where we actually looked at two conditions, test timing, covert and overt timing, and task difficulty.

So we gave a simple multiplication task and a complex three-by-three-digit addition task to fourth graders. And then we had, as I mentioned, that clear timing condition and a cut in an unclear timing condition. So the students were timed, but they were unaware. We found no differences in math performance according to math anxiety level with overtly timed tasks.

That is when we made timing very clear, but students reported higher math anxiety when completing complex computation problems. And students with medium or high math anxiety reported higher math anxiety when completing that was complex problems under a covert timing condition. So in our study, it actually looked like it was, reflecting what that meta-analysis had said, which is it's complex, not simple tasks that students experience higher levels of math anxiety, but it was also under a covert timing condition.

In other words, students didn't know how long they needed to participate in this task that was perceived by them to be challenging, and that created more math anxiety. So that was the case for our sample.

[00:34:50] Anna Stokke: In other words, when they actually weren't told how long they had to complete the task, that made them more anxious.

[00:34:58] Robin Codding: Correct.

[00:34:59] Anna Stokke: So that kind of reminds me of a conversation I had on a previous episode with Dan Rosen, and he mentioned that one of the things that causes students a lot of stress is if they don't have control over their situation.

So this sort of makes sense to me because you need to be clear with students what you're doing and what you expect of them, and they feel more secure when you are clear about these things.

[00:35:24] Robin Codding: Yeah, I think that, that's a great explanation. And we did look at, we did ask students about their perceived task difficulty and did see a positive correlation that was moderate to high between math anxiety and their perceived task difficulty as well. So that task difficulty piece is another layer that is really, really relevant to instructional tasks.

Amanda VanDerHayden and I, and another one of my doctoral students also tried to test that challenge, math challenge part, and this idea that I mentioned earlier about instructional match. So I had said sometimes we give a good instructional tactic at the wrong time, and I was referring to timed practice opportunities, and so we actually were able to test that in a single case design which is a form of a study that's done across individual students and students serve as their own control.

And so we looked at that functional relationship between whether we were giving a student the opportunity to build fluency, so that was the instructional tactic, or build acquisition. And then we aligned it or misaligned it with whether this skill was a challenging skill or a just right skill. And what we found is that when we aligned the appropriate tactic with their skill level that students did better.

So if we gave them a fluency-building tactic for a challenging task, they did not do well in terms of their math outcomes. We also saw, in some cases, that students had higher levels of anxiety under those conditions or rated those intervention strategies as less acceptable. But even with the challenging task, when we gave them an aligned condition, which would be to build acquisition, right, to build basic knowledge and understanding of that challenging skill, their math skills improved, and those strategies were more acceptable to them.

So, we were able to illustrate in this latter part that, you know, students who are in the process of building fluency, we could give them that timed activity, right? But yeah, not when it's a challenging skills. It's really the same thing that we've been describing earlier, we were just able to test that in a more controlled setting.

[00:37:36] Anna Stokke: So again, if there's a timed element, but the task is too challenging for the students, they don't do well. And generally, if the math task is too challenging, students may have higher levels of anxiety. But if you add timed tests, when students are at a point where they're more accurate, students do better.

So that means something like when they're at the point where they can figure out the answer to something like, say, five times four most of the time, but they may be slow, that's when to add the timed element so they get faster. So that timed element is really important to build fluency, but add the timed element at the right time.

It's similar to what you were talking about earlier, I think, when you indicated that students should be at 90 percent accuracy or more before using timed tasks to build fluency.

What are some effective measures for mitigating math anxiety? So what interventions work best and what does the research say? So you could do things like therapeutic interventions, which I guess would mean things like breathing exercises or something like that, or you could just work on building math skills.

What sort of things work well?

[00:38:54] Robin Codding: We want to start where you had mentioned earlier. We need to understand where students' skill strengths are and where their areas of skill weaknesses are so that we can address those skill weaknesses. Just can't get away from the fact that math is hierarchical, and those foundational skills, right, really are going to impact students' access and ability to solve higher-level problems.

We talked already about how whole number knowledge is really predictive of later content areas. So if you see that you have a student that's struggling with rational numbers, but they don't know their whole numbers, well, then we need to go back and work on the gaps in their whole numbers. So I think that's one basic way to think about it.

In our meta-analysis, when we actually looked at this very question that you asked, which was, “When we look at therapeutic versus skill building, what happens?” In these meta-analyses, which, you know, again we had 17 studies, so there needs to be a lot more direct evidence on this, but what we do know from the 17 studies that exist is that those therapeutic interventions, which included relaxation, breathing, journaling some brief CBT (Cognitive Behavioral Therapy) module kind of things or ideas about positive self-talk, that that mitigated math anxiety.

But of course, it doesn't address math achievement issues. So the students in these studies had both of those challenges. But when we looked at the math skill building interventions, and they, these are just like standard interventions.

They were tutoring, they were computer-assisted interventions, they were standard intervention protocols. They illustrated that they had a small, insignificant effect on math anxiety, and of course, they had a moderate to large effect on math skills. So if you just think of it from that perspective, the place to begin is with those math skill-building interventions.

And again, doing so in the way that I described a moment ago, where you're really kind of understanding where students’ strengths are in what areas they need to build on.

[00:40:52] Anna Stokke: And so just to emphasize that, so if we're looking at ways to mitigate math anxiety, the best place to start may well be to figure out the math skills that students are weak on and work on getting them better at those skills. And not only does that improve students' math skills, it also helps to alleviate the math anxiety.

[00:41:14] Robin Codding: Yeah, I mean, well, I'm always looking at parsimony in terms of intervention delivery, right? And so, the most parsimonious answer to this problem may be to remediate math skills first. And then, of course, if you see continuous persistence of math anxiety symptoms, then you can layer on some of those therapeutic approaches.

[00:41:33] Anna Stokke: Okay, got it, but avoidance probably isn't a good way to deal with it, right? Like just eliminating timed tests altogether because the students may be anxious, that probably isn't a good approach to dealing with math anxiety and certainly not dealing with math achievement either, right?

[00:41:53] Robin Codding: No, I mean, we, we end up in this pathway right with math avoidance where students then don't have those opportunities to practice or engage in math, and their performance becomes poorer, and they don't grow, and again, they continue to have larger gaps as they proceed through schooling. And we also know, you know, basic, Psychology 101, is that somebody has a fear of, or is anxious about something, we don't remove that stimulus, we introduce them to that stimulus in controlled ways.

[00:42:24] Anna Stokke: All right. So let's talk about some popular instructional techniques and whether they're effective and whether they might actually contribute to math anxiety. So I'd like to ask about productive struggle.

And so this is kind of a popular technique right now. The idea would be that instead of providing explicit instruction to students, you would let students struggle, and this is supposed to help with gaining a deeper understanding than if they were explicitly taught the skill.

So what do you think about that method? Would that cause anxiety? is it even an effective method? What do you think?

[00:43:08] Robin Codding: I don't know of any data that supports the use of productive struggle. On face validity alone, this is one of those where I'm like, “Wait a minute.” We don't toss a child into the deep end of the pool and then tell them to swim. So why would we do that in math? Where, you know, in math, it's predictable, which is wonderful and there are ways to solve the problem, and there are processes and procedures that we can use to approach the problem.

We know that when we don't provide those things to students that they struggle to begin. So beginning math work is always and persisting on math work is a challenge. And so I would think that that would be more difficult when it comes to this notion of productive struggle.

We certainly, I've mentioned a few times today that there is data to illustrate that challenging tasks actually cause math anxiety, or we seem to think they cause math anxiety based on the evidence we have. So this would be the ultimate challenging task, right? Provide an experience where we are not familiar with the component steps and don't have those prerequisite skills to maybe solve it.

So I would actually think it would lead to students actually avoiding the task.

[00:44:20] Anna Stokke: You know, I think it's obviously going to widen achievement gaps. You have the students that want to show off and they already know how to do this stuff, maybe because they caught on really quickly or maybe they're going to afterschool Kumon, and they excel in that situation.

And then, then there's all these other children who can't even begin. But certainly, we see this. I mean, it's this sort of thing is even being promoted. Personally, I would really recommend against it. Like what you said to me, it just, it doesn't make any sense at all.

And you just kind of think, “Really?” You know, you talked about worked examples earlier. That makes a lot of sense, showing them how to do a problem, having worked examples, and then maybe even fading, taking steps out of the problem and seeing if the students can fill in those steps and doing it sort of gently, right? And scaffolding.

[00:45:12] Robin Codding: Exactly.

[00:45:13] Anna Stokke: And with complex problems, cause you mentioned the complex problems. So I find that really interesting because of course, at some point we want students to be at a point where they can solve complex problems, but to solve the complex problems, you don't want to start with the complex problem, right? You want to build the skills. So, I think scaffolding is is really important also in effective instruction.

[00:45:41] Robin Codding: Scaffolding, explicit systematic instruction. Exactly. And the teachers that I've worked with who started with the productive struggle problem, I said, “Well, you don't have to throw out the idea of a challenge problem, you just put it in a different place in the instructional learning sequence,” right?

You put it in the place that in the adaptation or transfer of learning place. Students already have all the component skills. They're accurate and fluent with them. They now need this challenge problem to use those skills in a new and novel way.

[00:46:08] Anna Stokke: I noticed that you have a paper about philosophy-based instruction versus evidence-based instruction. What's the difference between philosophy-based instruction and evidence-based instruction, and what are some signs that an instructional method that maybe a teacher is being advised to use is philosophy-based and not evidence-based?

[00:46:33] Robin Codding: A lot of these writings just really came out of that experience that I had with my doctoral students back in 2016, where I was thinking to myself, “Man, so what's happening here?” I mean, systematic explicit instruction, for example, has been considered to be an evidence-based practice for a very long time across subject areas, it doesn't matter what subject you're talking about, but it's in math.

We've seen it in multiple meta-analyses again in math that have existed for a long time. And that's when we started to think about this idea of philosophy-based instruction, or sometimes I refer to that as pseudoscientific practices compared to empirical support.

And I think that pseudoscientific practices refer to those really that lack adequate empirical support with carefully controlled studies to support their claims. And so that's sometimes why you hear me make sure that I sort of back up what I'm saying with the evidence or I'm careful on how I'm stating things because I want it to be accurate.

And these practices that are pseudoscientific can be appealing because they superficially seem to be based in science, so they are tricky. They can be intuitively meaningful, but the assertions that are made about the benefits of the practice extend far beyond the actual evidence for their use.

And because they have common sense features and really enthusiastic claims about their benefit, they can be hard to identify, and they can be persuasive. I think they can be spotted by identifying the lack or limited peer review literature on the practice. So instead, the evidence is anecdotal or testimonial and over-emphasizing the positive results when de-emphasizing negative or disconfirming information or explaining away disconfirming information or minimizing it instead of contextualizing it or using it.

In scientific practices, right, if something we think is working, all of a sudden there's cumulative data to show that it doesn't work anymore, it doesn't work in a certain circumstances. Then in the scientific community, we'll acknowledge that and make an adjustment. And I think you don't see those things along these philosophically or pseudoscientifically.

What's striking to me is, like, how they get traction and, how to pause that and sometimes I wonder if it is because of this age of information that we're in and how we access information.

[00:48:59] Anna Stokke: I find that really interesting too. I think it's partially just the culture and education. It isn't the same as the culture in a scientific field. I think people are encouraged to be innovative and to do things differently, and innovative isn't always a good thing.

Sometimes what we've known worked best for a really long time is what worked best and like you say, maybe we have to make some tweaks to it. Like you were saying earlier, you were talking about the timed tests and how maybe that people thought negatively about this method because of the way the timed tests were being administered.

And sometimes it's just changing that. What about if someone said, “Well, the evidence I have for my program working is that students are more engaged.” What do you think about that?

[00:49:51] Robin Codding: Well, how do you know students are more engaged? Because I love that one because one of the things that I find fascinating is this idea of using tablets, right, or computers in school, where there's far more use than there is data on how and when to use it, and there is a claim that students are actually more engaged.

But when we have taken data, they're actually less engaged in the math task often on the tablet because they are roaming the tablet or because, just as if it's a paper type of task, the instructional task is too hard or not really meeting the students at their learning needs, and so therefore they are not engaging in the practice. So, I mean, we would really need to measure what engagement is and then also see if you're getting performance improvements out of that.

And I think we can do that, you know, if we are using so here's going to be my multi-tiered systems of service delivery plug here, but if we are using universal screening and if we are monitoring student progress, well, no, right? I'm all for if there's an innovation that ends in schools, there are situations and circumstances that, or problems that have to be solved and we don't have the evidence for solving that problem.

And so if you're taking good reliable, valid data and monitoring progress on that problem, then you have the data to support the strategy that you're using. But we still need to use data in order to make that claim.

[00:51:21] Anna Stokke: But I've also heard people say, “Well, you can't test critical thinking. You can't test creativity and the method that I'm telling you to use is developing good problem solvers, good critical thinkers.” So what do you say to that?

[00:51:38] Robin Codding: Yeah! I mean, Lilienfeld et al. 2012, how do you spot pseudoscientific practices? That would be right up there! That is a pseudoscientific practice. If you are saying that it can't be studied or it can't be measured, or it can't be tested, well, probably a pseudoscientific or philosophy-based approach.

[00:51:56] Anna Stokke: Yeah, I think you're likely right. So are there places that teachers can go where they can find out if a method is backed by research?

[00:52:07] Robin Codding: My colleague Corey Peltier has done this work looking at the math education space. Because I'm not in the math education space, I don't like to make claims about what happens in there. But he has certainly illustrated that there's a problem potentially right in math education university programs themselves in terms of how scientific data is described and talked about and used.

There's also data when we look just simply at implementation or that research-to-practice gap that there are challenges right with time for implementation and planning access, like you said, to the materials that do have evidence-based practices. There's certainly been a lot of systematic reviews of curricula and illustration at least in the elementary curricula that they don't actually include a lot of evidence-based practices.

So how are teachers getting exposed to those practices? There often is intervention compatibility gaps. So it's hard for teachers to figure out, “Okay, well, here's this evidence-based practice, but how do I put that into my regular classroom routine?” So I think there's a lot that needs to be supported at that institutional level, that school institutional level in terms of go to places.

I mean, it's always surprising to me that many educators that I experience don't even know about the National Mathematics Advisory Panel Report from 2008. And that is, you know, you can just Google that report. So that gives background information on what solid instructional practices are. The National Center for Intensive Intervention is a terrific website that has lots of information that's digestible.

The IRIS Center, is another great resource that has tons of videos, materials, you can even get, I think there's a whole module sequence you can get training on. So there are places to go that have these data. Of course, I mentioned earlier the Institute for Education Sciences in the What Works Clearinghouse from the U.S. DOE (Department of Education).

They have a practice guides, there's actually seven practice guides for math. So it covers lots of different content areas and it covers preschool all the way through high school. So those again gives summaries that are really digestible and easy to read on what to do.

[00:54:21] Anna Stokke: And I'll include some links to those things that you just mentioned on the resource page. What would you say to parents who are worried about their kids developing math anxiety?

[00:54:32] Robin Codding: I think there is data to suggest that if we have created an environment where we allow math anxiety to be part of our culture or comments, like “I don't like math. I can't do math. Math isn't for me.” That's a regular part of our culture.

We have to get rid of that. I think that is certainly not going to contribute to students’ math learning. It's making them assume that they can't approach this problem. So we're creating a culture where students don't believe that they can approach the problem, and then we're often not giving them scaffolding or explicit systematic instruction either.

So now, you know, we really have just thrown them into the deep end of it. At the end of the day, math is predictable. I actually love math because it's predictable. I specifically love stats. It requires understanding, but you can put the foundational skills together, it builds upon itself. So reminding parents about that.

I also think we just remove some of the unknowns by using checklists that help students approach problems. We know what steps students need to take to solve different problems. Start with that checklist. It eliminates the fear of having to start a problem, shows you what to do in the middle so you can persist through the problem and then at the end, you can go back and check your work because you have all the steps.

Did I do all the things? I think that really helps to mitigate that situation.

[00:55:56] Anna Stokke: And what advice would you give to a new teacher about teaching, say, elementary school math?

[00:56:04] Robin Codding: Use explicit systematic instruction at least part of the day and practice.

[00:56:11] Anna Stokke: Okay. That's great. Thank you so much for joining me today and sharing your expertise with my listeners. It was an absolute pleasure and I learned a lot.

[00:56:19] Robin Codding: Thank you so much. It was really fun. Thank you, Anna.

[00:56:22] Anna Stokke: I hope you enjoyed today's episode of Chalk and Talk. Please go ahead and follow on your favourite podcast app so you can get new episodes delivered as they become available. You can follow me on Twitter for notifications or check out my website annastokke.com for more information. This podcast received funding through a University of Winnipeg Knowledge Mobilization and Community Impact grant funded through the Anthony Swaity Knowledge Impact Fund.