Ep 39. Project Follow Through: Direct Instruction's overlooked success with Marcy Stein 

This transcript was created with speech-to-text software.  It was reviewed before posting but may contain errors. Credit to Jazmin Boisclair.

   

You can listen to the episode here: Chalk & Talk Podcast.

   

Ep 39. Project Follow Through: Direct Instruction’s overlooked success with Marcy Stein

 

[00:00:05] Anna Stokke: Welcome to Chalk and Talk, a podcast about education and math. I'm Anna Stokke, a math professor, and your host. You are listening to episode 39 of Chalk and Talk. My guest in this episode is Dr. Marcy Stein, professor emeritus in the Education Program at the University of Washington, Tacoma. Marcy has extensive experience with Direct Instruction. That's with a capital D and a capital I, or DI, and its implementation. 

 

What's fascinating is that she had a front-row seat during Project Follow Through, the world's largest longitudinal educational experiment. If you haven't heard of Project Follow Through, you're not alone. Despite being funded by the U.S. government to identify the most effective instructional methods for lifting children out of poverty, its results, which showed direct instruction outperformed all other methods, were largely ignored. 

 

I find this perplexing, and it's one of the key questions I wanted to explore with Marcy. Why were the results overlooked? And why is Project Follow Through so rarely discussed in education circles? In this episode, Marcy details her involvement with Project Follow Through. We talk about its design and the results.

. 

We discuss the origins, instructional design, and delivery of Engelmann's Direct Instruction. And its impact on at-risk students. Marcy also dispels common misconceptions about DI, emphasizing that it's effective for all students, not just those struggling academically. She even walks us through a typical DI math lesson. 

 

We also tackle some listener questions, including how to address varying skill levels in the classroom and what to do when older students lack prerequisite knowledge. Additionally, Marcy shares insights from her experience navigating educational politics as a DI facilitator and ensuring that the pre-service teacher she worked with understood the importance of effective instruction.  

 

This episode highlights the overlooked success of DI and underscores the importance of learning from past mistakes, specifically the failure to acknowledge the results of Project Follow Through. We need to better advocate for evidence-based practices in education. Throughout the episode, several articles and resources are mentioned and you will find those on the resource page for the episode. Now, without further ado, let's get started. 

  

I am honoured to be joined by Dr. Marcy Stein today, and she is joining me from Seattle. She has a Ph.D. in Special Education. She's a professor emeritus at the University of Washington, Tacoma, where she is one of the founding faculty members of the education program. She has published extensively on reading and math instruction, curriculum analysis, and textbook adoption. 

 

She has considerable experience consulting with teachers and administrators on improving the performance of students who are at risk for academic failure. She also has consulted with state departments of education on how to use research to improve instructional practice, and with schools, districts, and states on the systematic evaluation of reading and math curriculum materials. 

 

She's the lead author of Direct Instruction Mathematics and Designing Effective Mathematics Instruction, and I'm really excited to talk to her today. Welcome, Marcy. Welcome to my podcast. 

 

[00:03:49] Marcy Stein: Thanks, Anna. I'm happy to be here. 

 

[00:03:52] Anna Stokke: I'm really excited to talk to you because I understand that you had something to do with the implementation of Project Follow Through. So I've talked a little bit about this on the podcast in the past and we're going to get into it in detail here. So before we start with that, what exactly is your relationship to Direct Instruction? That's with a capital D and a capital I, and Project Follow Through?

 

[00:04:17] Marcy Stein: Well, I'm happy to share that with you. I was really lucky to happen upon Direct Instruction when I went to University of Oregon to get a master's degree. So, I initially went there because a very prominent special educator by the name of Barbara Bateman was there. 

 

She was prominent in the, in the area of learning disabilities. that was one of the schools I wanted to attend, and then actually, they gave me money. And I was from the Midwest, so that was a big deal to go to University of Oregon. 

 

I had read an article about Engelmann in that little learning magazine about how he critiqued teachers' instruction and a little bit about Follow Through and what was going on, who he was. And I thought to myself, “Well, I'm going to stay away from him” because he was, he watches teachers teach and he, and I didn't know what I was doing. 

 

And it was like, no way was I going to get involved until I got there and saw that there were courses in the learning disabilities sort of area on Direct Instruction, Direct Instruction reading language, math. And Englemann's name wasn't on it. It was Doug Carnine who became a mentor and close friend. 

 

I sat in Doug Carney's first class, I guess maybe you'd use the word gobsmacked. I was just like, “Oh my god, this makes so much sense.” I had three reading courses. I knew I didn't know what I was doing. I just got excited, and I went to him after the first year of my program and said, I want to work for you. 

 

It so happened that a good friend of mine, another student had gone to him too. And had she not gotten another job, it wouldn't have been me. I got involved. I became a consultant, then you moved from consultant to project manager over a couple Follow Through sites, then later on when there was additional funding. I became director of a kind of a spin-off of Follow Through. 

 

So I was involved in Follow Through from the start, from, I mean, they basically, they trained me and they prepared me and they taught me about what teaching is, and I was very lucky to have that just direct close contact with the Direct Instruction people.

 

[00:06:14] Anna Stokke: And we'll get into the details about Project Follow Through today. And I'm just going to tell you a story about how I first heard about it and about Siegfried Engelmann. So, I've advocated for better math instruction for Canadian kids for many years. And, I don't know, maybe about eight years ago, I was on a radio show, it's called Cross Country Checkup, it's a national radio show, and I was talking about some of the problems in how math is taught.

 

And this woman phoned in because it's a phone-in show, and she mentioned Project Follow Through and she said, you know, “We've known for a long time what works best for kids. And what's happening in the schools is academic child abuse.”

 

And I was really struck by that. I don't know who this woman was, but that was the first I ever heard of it. 

 

[00:07:05] Marcy Stein: Englemann has written a book I think that one of his books is titled Academic Child Abuse or a subtitle of it.  

 

[00:07:11] Anna Stokke: Maybe that's where it came from, but yeah, so let's talk a bit about it and let's get into the details. So could you tell us a bit about the origins of DI, that's Direct Instruction with a capital D and a capital I, and what inspired Engelmann to develop that approach? And it was developed in the 1960s, right? 

 

[00:07:30] Marcy Stein: This is my understanding because I wasn't quite sure Quite there then, but this Engelmann was at University of Illinois, well, Engelmann, and this has been written about and I have some resources for you that I can share about sort of his history that he did all these different things. And he had a BA in Philosophy from University of Illinois. and he wasn't an educator. 

 

The most direct relationship is that he was working in advertising and his boss said, “I want to know how to sell these candy bars,” and “What do kids pay attention to?” He started really being interested in, well, what do kids pay attention to and how would you influence kids, and how do you teach them? 

 

You'll see stories about him working with his twin boys, who are now, you know, grown adults. Kurt Engelmann is the head of the National Institute for Direct Instruction, and Owen Engelmann is part of the Engelmann Becker Corporation, and there are videos on YouTube with him working with the twins. 

 

So he got very excited about what he was kind of figuring out how to best teach kids and I think he kind of shopped it around and it was s Carl Bereiter at the University of Illinois who said, “All right, I'll give you a chance.” So Engelmann and Jean Osborn and a woman named Cookie Bruner, Elaine C Bruner, worked together at Colonel Wolfe School and they had a little preschool program. 

 

And they taught kids, interested listeners, to go, again, go on YouTube and take a look at Engelmann's video teaching. He's not teaching, let's be very clear. He's demonstrating what these little kids can do in math after some instruction in the 60s. 

 

That was Historic Colonel Wolfe School, they were designing the instruction, they were trying it out. Then he was invited to do Follow Through, University of Illinois wasn't crazy about him, and he needed to find a home because of Barbara Bateman, oddly enough, she had gotten her Ph.D. at Illinois, she was the catalyst that brought Engelmann and the Direct Instruction people out to Oregon and they found a home there for many years. 

 

And Jean Osborn stayed in Illinois, so did Cookie, and so did Elaine Bruner. They stayed in Illinois. Engelmann went out, and Follow Through kind of happened, you know, that became the home. 

 

One story about that. I met a woman out here who was this amazing teacher who had been in Illinois, who had gotten her master's degree, and she remembers that there were things going on there, they were developing Direct Instruction, and she said to me, “Marcy I, they told us that that stuff going on down the hall there, well kind of ignore that because that's something that's here today and gone tomorrow.”

 

And then she told me that she didn't realize that the kids they were teaching in Colonel Wolfe, they were kids from economically, a little challenged community, Champaign-Urbana community and that she was teaching their brothers and sisters in special ed. And these kids were like, you know, doing this amazing performance. 

 

So that's kind of, it started in Illinois. They were invited to join follow through because of the results they demonstrated with the little kids in Colonel Wolfe school, and then that led to the rest. 

 

[00:10:27] Anna Stokke: Yeah, and actually, I watched a YouTube video where he was working with kids, and I mean, it's very grainy, right? Because it's a long time ago. But I was kind of amazed because they were just little kids, and he starts off with like times tables and standard algorithms, adding, columns and that sort of thing and it ends with them solving systems of linear equations, and reasoning through these little systems of linear equations—two equations and two unknowns.

  

[00:10:55] Marcy Stein: I've never taught a math class that I haven't introduced the class with that video because one of the major criticisms of Direct Instruction is that it's rote memorization. If you watch that video, you know that there's no way that those little kids could have memorized all that, you know, within a year or even two years of 20 minutes a day. I would have my students count, count all the things that the kids are doing, and then we can talk about how that was able to happen with the brilliant design that he was testing out at the time. 

 

[00:11:26] Anna Stokke: Yeah, it's really, it's a really great video and I'm going to post a link on the resource page. So, let's talk about evidence supporting Direct Instruction. So first of all, can you give us an overview of Project Follow Through?

 

[00:11:40] Marcy Stein: Yes, I can. Project Follow Through, people have said and written about it being the largest educational experiment sponsored by the U.S. government. My understanding was that during Lyndon Johnson's Great Society, when he was, you know, trying to build up the society and address issues of poverty there was Head Start. Most communities who qualified for Head Start dollars got them, and they didn't have that same kind of capital for elementary schools. 

 

So they decided, let's do a study. Let's gather together the major models of the time. Let's do an experiment, let's see what happens, let's see how far we can get. So communities got to choose their own model— it was never forced on them. So when Follow Through funds became available and an economically challenged community decided “We're going to go for this. We want to participate.” 

 

They interviewed people from the different models and they chose the instructional model that they wanted. There were a range of instructional models that finally at the end of Follow Through, nine models were evaluated. They were kind of grouped as a basic skills model, as a cognitive model, as a kind of a more affective model just for the sake of talking about them. 

 

And then they implemented, then it was an implementation in 10 years. It was an amazing implementation. It was amazing to be involved in this. Every Follow Through school that signed up had a non-Follow Through control school in the very same community. That was amazing. So that constituted one of the analyses of the Follow Through data. They also then agreed upon measures. All of the directors of all of the models who are being evaluated agreed that these are the measures that we would use to measure student outcomes. 

 

And then you implement it for 10 years. The sites that the Direct Instruction model was implemented in were like amazing. They were just incredibly diverse across the United States. I was the consultant and then project manager for Flint, Michigan. I also went to Williamsburg County, South Carolina. That was a rural community with African American students and probably it, I think at one time it was the lowest income, the poorest community in the United States by one census.

 

There were five schools there, I think, in which DI was being implemented. Flint had two schools, there was East Las Vegas, New Mexico, there was Uvalde, Texas, and everyone's heard of Uvalde, Texas now. There was, I went to a site in Arkansas called Flippin, Arkansas, it was a little rural site.  

 

So, it was implemented for ten years, and then, at the end of ten years, there was an evaluation. and it really kind of wasn't close. If you look at the Follow Through data if you, if you don't already know or your listeners don't already know Jean Stockard and her colleagues wrote a book called All Students Can Succeed, and she basically analyzes 50 years of data on Direct Instruction, including, you know, starting with Follow Through. 

 

[00:14:40] Anna Stokke: Okay, so a couple follow-up questions. So there were nine types of instruction being evaluated, and there was a control group school for each school that had one of the methods. What were some of the other methods like? Was there like a constructivist, an inquiry-based type method being used? 

 

[00:15:00] Marcy Stein: Yeah, I have the models in front of me. There was a behaviour analysis model that was grouped with the Direct Instruction as a basic skills model. So it was different from DI, a lot of people think DI is just kind of purely behaviour analysis—it's not, it's more than that. 

 

But it, you know, shares some features. So it's Direct Instruction, behaviour analysis, there was something called TEEM [Tucson Early Education Model], and that was a a model that I think was a precursor to whole language. It was a language, kind of a language-based, language experience model. 

 

There was an open education model that was based on the British Infant Schools, so it was kind of a lot of kid choice on that. There was a model called the Cognitive Curriculum and it grew out of a preschool that was in Ypsilanti, Michigan the Perry Preschool. That had been a very popular preschool, and they then implemented in K-3. And that was the assumption was that they were teaching sort of more sophisticated cognitive skills than sort of what Direct Instruction was doing. There was the Bank Street model of education, still exists. It was a Follow Through model. 

 

Did I mention a Parent Education model that kind of, I was always told parent education could sort of bypass the schools and work with the parents. There were a couple others that I am not as clear on. There was something called Responsive Education, which I don't really know too much about and Southwest Lab there was a model out of Southwest Lab, but again, I wish, I don't I can't tell you too much about those models. 

 

But the range, right, the range of student-directed, teacher-directed, kind of that continuum was there. 

 

[00:16:35] Anna Stokke: And so the cognitive models, are those kind of like what we maybe call today the problem-solving approach?

 

[00:16:42] Marcy Stein: Yes.

 

[00:16:43] Anna Stokke: You start sort of with complex problems, and it's student-centered and the children are encouraged to sort of invent their own algorithms kind of thing.  

 

[00:16:52] Marcy Stein: I don't know the details of that model, but yes, I think it would fall under that category, of a kind of a constructivist model. Less detailed, less teacher-directed, but yes, a lot of problem, what they would have called problem-solving. I can't speak to the details of that model. 

 

[00:17:08] Anna Stokke: Just to make sure that we understand that these different types were evaluated, and what age, what grades were the kids in?  

 

[00:17:15] Marcy Stein: They were, there were kindergarten sites, starting sites and first grade starting sites. I think the kindergarten sites came a little bit later in some of the models but all the kids were evaluated at the end of third grade. I mean, we, you know, there were data annually and assessments to annually, but the big Follow Through evaluation was after you've implemented from K through three or first through three, the end of third grade came to the evaluation, the major evaluation  

 

[00:17:40] Anna Stokke: And then what were the control groups, which was a business-as-usual group, what will be happening in a business-as-usual group?  

 

[00:17:48] Marcy Stein: It was whatever was in the school system at the time. Remember, this is the 60s and 70s, so probably a lot of what then, you know, now are called core programs, but then were called basal programs. I don't think there was much teacher-designed programs at that point. 

 

I think that was too early for that. So we have a program in the school and we implement the program. 

 

[00:18:09] Anna Stokke: Did any of the models do worse than the business-as-usual groups. 

 

[00:18:15] Marcy Stein: Oh, many of the models did worse than business-as-usual groups. There's a very stunning graph that is that's published, and when you read about Follow Through, what they did is calculate an index of significant outcomes and, to kind of simplify it, this is the way I understand it, they kind of went, “All right. Well, how many times did the Follow Through group beat their control.” 

 

And then they calculated kind of that index and they, you know, where zero is, they're the same, you know, no differences. And there are only a few. The Direct Instruction model, I'm looking at it now, the Direct Instruction model in basic skills, well, these are how the measures were kind of grouped as well, basic skills, cognitive skills, affective skills, kind of outperformed their control groups consistently. Whereas the Parent Education model outperformed their control group in the affective measures, so did Behaviour Analysis. 

 

There was a little gain in basic skills for both Parent Education and Behaviour Analysis. Southwest Lab did something with the affective measures, but everybody else you can see that the business-as-usual groups were outperforming the instructional models, which had a lot of money. 

 

There were a lot of dollars going into those classrooms, and business-as-usual groups were outperforming them. 

 

[00:19:36] Anna Stokke: And the measures that they tested them on, so you said basic skills. So, that would be what we would expect like, can you do basic arithmetic and things like that, right? 

 

[00:19:46] Marcy Stein: And they were standardized tests. So it was, they used them at that time, Again, we're talking the 60s, they used the Metropolitan Achievement Test. And the subtests of the Metropolitan Achievement Test were like word knowledge, and spelling, and math computations. Those were considered to be the basic skills. 

 

The cognitive skills that they, you know, put under that cognitive label, they were still the subtests of the Metropolitan Achievement Test, but they were the subtests of reading comprehension, math concepts, math problem solving, and then there were affective measures. So they used the Coopersmith Self-Esteem Inventory and something called the Intellectual Achievement Responsibility score, and those were the, those are the measures that were used in Follow Through at the time, and agreed upon. 

 

Note that they were agreed upon by everybody. 

 

[00:20:35] Anna Stokke: And the affective measures, those were like self-confidence and things like that?  

 

[00:20:39] Marcy Stein: Yeah, I have a story about that, that was, is very telling about DI. I was coming back from Flint, Michigan to Oregon and Wes Becker, who was the director of Follow Through, was coming back from a meeting of directors when the preliminary results from the evaluation were being shared. 

 

And so we met up in Chicago, both going, you know, going back to Eugene, Oregon. And he said one of the interesting things that happened was the Direct Instruction kids outperformed everybody on these affective measures. And everybody said, “That can't be,” you know, “These kids can't like being in, you know, involved in this systematic instruction,” and that there must be something—I remember him telling me this, “there must be something wrong with the measures.”

 

Well, of course, we know that it's a really simple concept that when kids are successful, they feel really good about themselves. So, we know that that was part of our teaching. That was part of the delivery of instruction is pointing out to kids how smart they are, and what they did and what they accomplished and what they can do. 

 

You could see that in the in the videotape when you see Engelmann interacting with the kids. “Are you smart? Are you working? Who's thinking?” and all that kind of stuff. And yeah, our kids felt great. 

 

[00:21:51] Anna Stokke: Yeah. And I mean, Englemann, when you watch him, he's a very sort of dynamic character, right? This can go a long way with students as well, when the teacher is very charming and dynamic. But we're talking in this study, these were just regular teachers, right? And they'd been trained to use DI.

 

[00:22:11] Marcy Stein: Not only regular teachers, and I've been really kind of harping on this for a while, especially with the math data. The math programs were taught by paraeducators. There were three—Follow Through in the days of big money, you had a teacher and two paraeducators. And the typical Follow Through classroom, it didn't always happen, I only experienced it once where the teacher decided she wanted to teach the math and not the reading. 

 

Typically, the teachers, the classroom teachers taught the reading program. One paraeducator taught the language program and one paraeducator taught the math program. So we got unbelievable math results and these programs were taught by paraeducators. 

 

And that is something that should be paid attention to. 

 

[00:22:57] Anna Stokke: Okay, so just a point of clarification. What's the definition of a paraeducator? 

 

[00:23:01] Marcy Stein: They were the teaching assistants. They were not certified teachers. They were, in Follow Through, all the paraeducators, at least in Direct Instruction Follow Through, I can't speak to the others, all the paraeducators were hired from the community. I can't guarantee this, but I think they all had high school diplomas. 

 

It was a teaching assistant job that they had. You had a teacher and paraeducators in the classroom and the paraeducators taught. They taught kids math and language, and the language data are incredible also. 

 

[00:23:30] Anna Stokke: What do you think it was about DI that made it particularly effective? 

 

[00:23:36] Marcy Stein: Now, this is my opinion of this. You know, Direct Instruction is complex. People want to kind of, I think, think that it's a simple matter of a few details here and there. It's really a very kind of sophisticated kind of approach that involves how you organize your instruction, how you design the instruction, how you deliver it. 

 

What most people don't understand, and what I, and I am not an instructional designer, I think what makes direct instruction really stand out is the design of the content. I think people try to add some of the delivery kinds of aspects of Direct Instruction onto content that's not well-designed. I don't think you can get very far with that. 

 

You can learn how to engage kids, you can learn how to monitor, you can do kinds of things, you can allocate enough time, but I've always said, you know, the scripts were known for scripts. You can script anything. You can script instruction that makes no sense at all.

 

And you can put it in a script, and I've seen it, if you start evaluating programs and you see not as intensive scripts as the ones that we have, but you see scripting happening in more and more commercial programs, and you say, just because it's scripted doesn't mean that it's well-designed. 

 

So my feeling, even though the analysis hasn't, I don't think they pulled it apart, I don't think you can do Direct Instruction or you can be successful or say you're doing Direct Instruction if you don't understand the design that underlies all of the instruction.  

 

[00:25:12] Anna Stokke: And we'll get into that a little more in just a bit. So, okay, we have this large study; it was the largest educational study ever done. Clearly, Direct Instruction outperformed all the other models, and there's other evidence because Stockard has done some sort of a meta-analysis on it, right? But why weren't the results of Project Follow Through widely adopted?

 

Like, that was the entire point of the study, was it not? the entire point of this study was to figure out the best ways to teach children to lift them out of poverty, why weren't the results adopted?  

 

[00:25:50] Marcy Stein: Well, that's the million-dollar question now, isn't it? That's the question that's been haunting Direct Instruction people for 50 years. And don't know. I mean, when I explain Direct Instruction, when I, you know, do an overview of it or am asked to give a talk or even the introduction to, you know, my teacher preparation program like I mentioned earlier, I'll say “these are organization and management details, how you set up a classroom or a school,” or you need to do an implementation, I talk about design and I talk about delivery, and I've often added another section called politics, and it's the politics of education. 

 

And it just seemed that something this systematic and this scripted, and it was not very well understood, and it certainly was threatening to the mainstream, you know, what's been going on in education. And it just, it just didn't fly. And it's the same, I mean, you can see it happening throughout all of education where how many years have we been talking about the science of reading? 

 

You know, we've had different names for it. We've had, you know, research-based and evidence-based and scientifically-based, and now the science of reading, and it's been the same stuff in terms of beginning reading. Why is that, why isn’t that in it? Why have people not figured out that, yeah, you can teach this little piece of decoding really easily and without a lot of hassle if you just do a few things? 

 

I don't know. I mean, I wish I knew the answer to it. It seems people perceive it as being offensive to teachers. People see it as being not allowing teachers to be creative. They see it as being too rigid and too structured. They see it as not being fun. They see it as being rote. Because if you just see the delivery of Direct Instruction, you make that assumption that it's memorization when it's not. 

 

I don't know. I mean, there doesn't seem to be that kind of respect for the science that underlies this, and I don't know how to how else to answer it. 

 

[00:27:53] Anna Stokke: Yeah, I find it all rather unethical, I have to say. 

 

[00:27:57] Marcy Stein: Yes. I agree with you a hundred percent. In fact, when I'm teaching like, you know, when I had my little teacher prep students, and it's like, “you don't get to decide how to do this. You need to look at your kid data.” 

 

If I walked into a classroom as a consultant, and kids were performing using whatever it is they were using, I don't say a word. But all these years of talking about kids, who are from economically challenged communities across—quite diverse, but this, right, not performing. How do you, deal with that?  

 

And that's one of the things that Engelmann did, he kind of inspired you. Well, he would give talks to the National Institute for Direct Instruction as a conference every year, but every year Engelmann would do the closing. Even though you've you've heard it, it would be very moving because it was like, that's somebody's kid. 

 

That kid that's in your classroom, that's somebody's kid in that classroom, and you cannot take that lightly. Here's why you could get behind it too. I feel incredibly grateful that I can be as passionate about it now as I was when I was starting out because I know kind of some things to do that would get you there. If I was lost and I didn't know and I was grasping at straws, but I do know some things you can do to get there. 

 

Not that it's easy. And it's like, and we know some things. So yeah, I agree with you. I think it's unethical to continue to do things that don't work for kids. 

 

[00:29:28] Anna Stokke: Let's get into it. Let's talk a bit about what Direct Instruction is so that everybody knows what we're talking about. The first place to start, I think, is what's the difference between Direct Instruction, capital D, capital I, and just lowercase direct instruction or explicit instruction as it, as it is often called?  

 

[00:29:48] Marcy Stein: I will give you a reference that you can post. It was published in 2017, and the senior author is Charlie Hughes from Penn State University, , he wrote an article with several colleagues, and it's called “Explicit Instruction: Historical and Contemporary Contexts.” 

 

And it's in the journal, it’s Learning Disabilities Research and Practice, and it's a lovely historical piece about what does explicit mean, what is little di, what is big DI. What separates out little di from big DI? It's the curriculum. It's the design of the curriculum. It's what it is that you, you know, the content that you're teaching the kids. That's very simple. Little di to us is, kind of constitutes the delivery aspects of Direct Instruction, stuff that you would do.

 

I remember when I was a doctoral student, and it came out of this research area that was looking at correlation research between, you know, what teachers did in classrooms, observational research, and what seemed to be related to kid performance. And it was like engagement. It was like, you know, that's the old term, academic-engaged time. 

 

What were kids doing when they were sitting in the classroom? That's kind of all delivery stuff. So in my mind, little di, it's delivery, it's kind of important monitoring and, and checking for understanding and engaging and reinforcing and doing all that kind of, you know, good stuff with kids and interacting with kids. 

 

But it matters what it is you're, what you're teaching. So that's where DI, that's where big DI comes. 

 

[00:31:19] Anna Stokke: So they intersect at the delivery piece, the delivery is essentially, you know, very similar, but in the capital D capital I, there's a content piece. 

  

[00:31:30] Marcy Stein: Delivery necessary, but not sufficient. 

 

[00:31:33] Anna Stokke: Got it. 

 

[00:31:34] Marcy Stein: One of the questions I think we talked about at one point was, can you do DI with programs that are already in the schools? Yeah, no. You can get close if it's a phonics program, you can do some stuff with delivery, you know, another phonics program, not a DI, but, yeah, the answer's mostly, it's mostly no.  

  

[00:31:52] Anna Stokke: The two main features really then are instructional design and delivery of the instruction. So let's talk about those. So, what are some of the main features of instructional design in DI?  

 

[00:32:05] Marcy Stein: Here's what, again, how I think about it. As like we've mentioned, I've been working with Direct Instruction since I was a kid, for 50 years now. I'm not an instructional designer. I really can't do what they did because what people don't see is a phase that, that is the very beginning of all instructional design, which is an analysis of content. And it's this very sophisticated sameness analysis, what's the same about topics in, say, in math? What's the same about addition and subtraction and, you know, missing addend addition? 

 

What's the same about, you know, all the fraction skills that you have? What's the same about, you know, how would you teach fraction skills that would lead you to teaching for kids to understand decimals? So this analysis that gets done prior to designing teaching procedures, and that's the brilliance of Direct Instruction. 

 

I could write it an okay, decent program. I know enough about basics, things about programs, but the kind of analysis that they did in all of the content areas, I mean, they've done it over and over and over again, is really quite striking. And in my mind, there are actually only a few people who know how to, know how to, unfortunately, there are only a few people who know how to do that. 

 

So once they, once they do this analysis, then you get to break it down. Here's something that confuses people. Everybody thinks DI is just a task analysis. Oh, no, no, no, no. You don't, you don't say, “Well, this is how I multiply. So I'm going to break down the steps.” Everyone thinks it's a simple matter of breaking down the steps and then teaching the steps and scaffolding the steps. 

 

And it's what comes before the task analysis. It's the analysis of the strategies that you're going to, that you decide you're going to teach your students. It's how efficient those strategies are, how generalizable those strategies are, what mileage can you get from them. Once you get those strategies, then you break it down into a task analysis. 

 

[00:34:07] Anna Stokke: So you want to sort of pick the most generalizable strategy. Okay, so you mean something that would work best in most situations. Probably also, like, when I'm teaching, I try to teach the strategy that's likely to work the best for most students based on experience. So is that kind of what you mean?  

 

[00:34:26] Marcy Stein: Well, not necessarily for students. All right, let me give you an example of a generalizable strategy. I'll give you a little basic one. I like this one, it was in the very first math program. So they decided they're going to teach kids how to, add, right? 

 

Not math facts. Let's keep math facts separate, this is all the conceptual stuff. They're not using activities, they're using line drawing. So, eventually they're going to have the kids read a statement, four plus six equals how many? The very first step is circle the side you start counting on. And the kids learn a simple rule. 

 

You can't start counting on the side with a box. So they read the statement and they circle the side with four plus two, not equals how many. There's a box, and they teach the kids to say “how many” when they see a box. And then there are a set of steps that they do for teaching kids how to solve that problem. 

 

“So, how many do you have in the first group?” “Four.” “Write four lines.” And then they use that, and we'll get into this too because this is very, an important part of design: vocabulary. And they say, “Okay, what, what's next?” “What does plus tell you to do?” “It tells you to draw more lines.” “So, what's the next group? How many of the next group?” “Two.” “Four plus two equals how many?” You know, “Draw two lines.” 

 

Very important, now they say, “Let's figure out how many you, how many you end up on this side” because they learn a rule: you must end up with the same number on this side and on the other side of the equals, right? Language is very carefully controlled. The kids add the lines, later they learn how to do it the fast way—four, five, six. “How many did you end up with on this side?” “Six.” “How many do you have to end up on the other side?” “Six,” and they draw lines. They draw, okay, “Draw lines under the box because the lines under the box tell what goes in the box.” 

 

Okay, they do that. It took about 67 lessons to get there. They're drawing lines for numerals, they're doing simple identification, they're learning the rule about equals without computing “There's a six on this, there's a box on this.” 

 

You know, just sort of what the idea is, the application of the rule, they're doing all of those prerequisite skills, and then they're going to put them together in this strategy, right? This is this procedure that they go through and they should have been, the kids should have mastered each one of these individual little prerequisite skills before they ever put them together. 

 

The reason they do that is that's the same strategy that they can do for missing addend addition and for subtraction. And you look at the sign, you circle the side, you start counting on. So let's do four plus how many equals six. Every low-performing kid is typically, in a typical program, is going to add that four and add that six at the end and put them in the box, and it's going to be wrong, right?

 

And they've learned the rule without knowing anything about equality, you take two numbers, you put them in the box. They haven't learned how to discriminate when you do that, how you do that. Well, it's the steps, the application of the steps. Circle the side you start counting on. “Oh, you can't start counting on the side with a box,” so now you're starting counting on the side with six. 

 

“How many did you end up with on that side?” “Six.” “How many do you have to end up with on the other side?” “Six.” “Do you have six in the first group?” “Nope, you only have four.” “Start with four and end up with six. Draw the lines under the box.” That's why they even have that set. The lines under the box tell what goes in the box. 

 

And then the same thing with subtraction, you could do subtraction a couple different ways, but it's again, what side do you start counting on? How many do you have in the first group? What does the minus tell you to do? It tells you to either cross out or it tells you to count backwards. There are two different ways. 

 

Make the sides equal. One strategy that applies to three operations.  

 

[00:37:50] Anna Stokke: You want a strategy that's going to apply to a lot of examples. Okay, that's interesting because that was, one thing I noticed when my kids were young that I thought was kind of crazy is they were encouraged, in fact, it's written into our what you'd call standards, we actually call that a curriculum here in Canada, that students should learn multiple strategies for everything. 

 

My observation is that the students never really become good at anything and they get confused because they don't know what strategy to apply in what situation. I mean, this is even done with math facts, right? Like count up from a double. Oh, that doesn't work in this situation, so you do the something else.  

 

[00:38:36] Marcy Stein: It makes me crazy. Yes, I agree with it 100% or create your own strategies, not only try multiple strategies, create your own strategies. So, even if you got the answer to this problem with your little peers that you're working with, there is no guarantee that what you did is going to help you answer the next problem. 

 

And the big classic example of that is I remember seeing somebody had a slide, and it might have been Bernie Solano's slide, you want to find one-third of 12. Oh, well, you divide 12 by 3, and you can, you know, find one-third of 12. You can't do that if you want to find two-thirds of 12. You can only do that It's the numerator’s one. And then you have to go through all this teaching to show kids, “Oh, no, no, here's when you can do it or why you would do it or how you would do it.” 

 

It's a waste of time, and I will tell you one of the tenets of Direct Instruction is if you want to catch kids up, kids who are walking in the door because they’re, you know, a little bit behind because of their situation, then you have to teach more in less time, and it has to be efficient. 

 

And having kids explore or apply multiple strategies that don't always apply to certain examples and when did it apply? You can do it later. You can teach kids that there are multiple ways to solve problems, but if you can get them good at solving one way that is much more applicable and gets you mileage, why wouldn't you do it? 

 

[00:39:56] Anna Stokke: And that, that's a big one for me. The multiple strategies because I think, again, it's just, they're not getting good at using one strategy.