Journey to Interleaved Practice #3
“I realized that I had to study some material less because I knew how to tackle these problems. It’s kinda fun.” -Student
In my first post in this series I shared my thoughts on my motivation for the design the observational study, noting that discrimination was a key idea I wanted to explore. In my second post in this series I shared some of the tools and my thought process in designing the structure of the interleaved homework assignments. In my final post on my journey (for the season, anyway), I will share some preliminary results, some student solutions that I found interesting, and my overall thoughts on what I learned.
Preliminary Results
First, here are the overall trends in the assessments from this term.
Each solid line represents one of the 14 students who were involved in the observational study. The dashed black line represents the average progress of the class. A few things should be immediately apparent:
- The black line shows a general decline over the semester of about 20% if one observes Quiz #1 first and the Final Exam last. However, if one were to remove the quizzes, one would see a decrease of 15% from Test #1 to Test #2, followed by a slight increase of about 5% from Test #2 to the Final Exam. More discussion on this below.
- What the heck happened to that poor blue student? It might be that H found the interleaved structure of the course and homework overwhelming and needed more time for comprehension compared to the other students. Is it possible that students with special considerations benefit more from the structure of a blocked approach? I haven’t read much on this, but please feel free to share some research if you know about it.
- Aside from a few students who remained close to the top for the assessments, many students saw a drastic decrease around Test #2. Why is this? Test #2 contained 86 points dedicated to all the various integration techniques (substitution, integration by parts, strategies for trigonometric integrals, trigonometric substitution, partial fractions) and I told my students to do whatever questions they wanted to in order to obtain 50 marks. Perhaps this choice was too much, and a more structured test would have been better-suited.
If there are other items that are particularly noticeable, let me know and I will reflect a bit more on why that might be the case.
I also compared the scores of the 14 students on Test #1, Test #2 and the final exam from differential calculus to integral calculus. Since Test #2 was so varied from the structure of differential calculus, I decided to exclude it here (although there was a 10% decrease). Test #1 saw a change in scores of about 10% and the final exam also showed a slight increase in score of about 2%.
Student Solutions
A estimates the area under the upper half-circle using a parabola.
B shares how we know the integral is bounded above by 1 using properties of odd functions.
M shows the correct (and challenging) solution to two difficult integrations on Test #1.
What I find odd about P’s solution is that she gets the substitution in part (a), yet tries to use the power of -1 in part (b). This was a common misconception in the course that persisted all semester.
Trigonometric integrals offer interesting flexibility in answers. Here, I would have tried a substitution, but A decided to make everything cosines and use cosine reduction.
Here, a u-substitution would work, but N decides to use trigonometric substitution.
Discussion
First and foremost, while I did select an interleaved approach due to the hopes that it would make integral calculus a bit easier in the long run by allowing students to discriminate between integral techniques, I also noticed that students’ mindsets changed a bit this semester. In differential calculus, where they might not venture an answer, in integral calculus they would try substitution or integration by parts, even if it led them down a dangerous path. There was a difference in both effort and execution. They persisted and often came up with insightful solutions. It was also true that there was less cramming for tests and the exam. In fact, N came up to me and said “I realized that I had to study some material less because I knew how to tackle these problems. It’s kinda fun.” It would be interesting to follow-up with them over the summer months to see how much of this knowledge they retained.
From my perspective, I know that any fluctuations in grades are highly likely due to random chance factors, and not necessarily due to the interleaved practice. This said, it was an interesting first-go at something this big and I definitely want to try it again. The main difficulties I had were:
- Time. It took a lot of time to work through the homework solutions in class. Due to the time I lost, I had to teach differential equations in the lab portion of the course, and lost time discussing some aspects of power series. I’m not sure I would have necessarily changed this, as many students appreciated the extra time spent on solving questions and being able to ask specific questions.
- How do I measure whether or not the interleaved practice actually helped? I’m not sure that I effectively can do this based on the way the study is designed, but here is a thought. When a student tackles a question, either they use the correct technique or they don’t. What if I looked at the proportion of times a correct technique was used on Test #2 and compare it to the proportion of times a correct technique was used on the final exam? Maybe this would be helpful.
For the Love of Maths 
Pingback: Reflections on OAME #1 – For the Love of Maths
Pingback: How do we know what to put on the quiz? - ollie lovell
Hi Brian. Thanks so much for taking the time to put together this series of posts, it offers a lot of food for thought. Here’s some of the food regurgitated (epistemic fidelity of that metaphor…? I digress).
Considering the first post, I recall you talking about how one of your main goals was to help students to better be able to choose solution approaches under test conditions (when the ‘we’re in a Python lesson, thus, this must be a Pythag question’ logic no longer applies). I recognise the value of interleaving to this end, but I’d like to pose a different question, what about more practice on exactly what you’d like the students to improve upon? You could do the following, at the end of every lesson (or whenever you have time to do it) you could flash up a series of questions on the board, one at a time, and each time you show a question you could get students to volunteer solution approaches, you could table the on the board then do a heads down thumbs up vote on which approach. Then you could give the answer and move on to the next question. Thus, you’re not actually solving the problems (nor are the students), you’re simply isolating the skill that you’re trying to hone, i.e., selecting solution approaches. This is a point that I haven’t seen the learning scientists talk about, and I’d be interested in their reflections as well as yours. This is actually how I studied at Uni. Once I knew I could do processes, I’d get a big stack of past exams (with solutions) then I’d read a question, try to identify the appropriate solution strategy then (without actually doing the problem) I’d check my answer. Much more time efficient. Note: This will obviously only work once the processes are down pat, such an approach is very susceptible to the Dunning Kruger effect if a student isn’t honest with themselves/able to be honest with themselves due to a lack of understanding.
A second thought is from student N’s comment that you included, along with your own, “It was also true that there was less cramming for tests and the exam. In fact, N came up to me and said “I realized that I had to study some material less because I knew how to tackle these problems. It’s kinda fun.”’. I think that this effect, students simply seeming to know the content better, is probably more to do with your distributed practice of the concepts rather than your interleaving. Do you think that this could be the case?
Finally, I agree with you. I don’t think that it’s really possible to measure the efficacy of this approach given the current ‘experimental design’. And I don’t think that comparing correct approaches in Test 2 vs. final exam would shed much light (not to mention the time it would take you to code this!). That said, it’s been great to follow this series of yours and has sparked lots of ideas and questions. Looking forward to hearing about your next experiment!
Ollie.
LikeLike