“I realized that I had to study some material less because I knew how to tackle these problems. It’s kinda fun.” -Student
In my first post in this series I shared my thoughts on my motivation for the design the observational study, noting that discrimination was a key idea I wanted to explore. In my second post in this series I shared some of the tools and my thought process in designing the structure of the interleaved homework assignments. In my final post on my journey (for the season, anyway), I will share some preliminary results, some student solutions that I found interesting, and my overall thoughts on what I learned.
First, here are the overall trends in the assessments from this term.
Each solid line represents one of the 14 students who were involved in the observational study. The dashed black line represents the average progress of the class. A few things should be immediately apparent:
- The black line shows a general decline over the semester of about 20% if one observes Quiz #1 first and the Final Exam last. However, if one were to remove the quizzes, one would see a decrease of 15% from Test #1 to Test #2, followed by a slight increase of about 5% from Test #2 to the Final Exam. More discussion on this below.
- What the heck happened to that poor blue student? It might be that H found the interleaved structure of the course and homework overwhelming and needed more time for comprehension compared to the other students. Is it possible that students with special considerations benefit more from the structure of a blocked approach? I haven’t read much on this, but please feel free to share some research if you know about it.
- Aside from a few students who remained close to the top for the assessments, many students saw a drastic decrease around Test #2. Why is this? Test #2 contained 86 points dedicated to all the various integration techniques (substitution, integration by parts, strategies for trigonometric integrals, trigonometric substitution, partial fractions) and I told my students to do whatever questions they wanted to in order to obtain 50 marks. Perhaps this choice was too much, and a more structured test would have been better-suited.
If there are other items that are particularly noticeable, let me know and I will reflect a bit more on why that might be the case.
I also compared the scores of the 14 students on Test #1, Test #2 and the final exam from differential calculus to integral calculus. Since Test #2 was so varied from the structure of differential calculus, I decided to exclude it here (although there was a 10% decrease). Test #1 saw a change in scores of about 10% and the final exam also showed a slight increase in score of about 2%.
First and foremost, while I did select an interleaved approach due to the hopes that it would make integral calculus a bit easier in the long run by allowing students to discriminate between integral techniques, I also noticed that students’ mindsets changed a bit this semester. In differential calculus, where they might not venture an answer, in integral calculus they would try substitution or integration by parts, even if it led them down a dangerous path. There was a difference in both effort and execution. They persisted and often came up with insightful solutions. It was also true that there was less cramming for tests and the exam. In fact, N came up to me and said “I realized that I had to study some material less because I knew how to tackle these problems. It’s kinda fun.” It would be interesting to follow-up with them over the summer months to see how much of this knowledge they retained.
From my perspective, I know that any fluctuations in grades are highly likely due to random chance factors, and not necessarily due to the interleaved practice. This said, it was an interesting first-go at something this big and I definitely want to try it again. The main difficulties I had were:
- Time. It took a lot of time to work through the homework solutions in class. Due to the time I lost, I had to teach differential equations in the lab portion of the course, and lost time discussing some aspects of power series. I’m not sure I would have necessarily changed this, as many students appreciated the extra time spent on solving questions and being able to ask specific questions.
- How do I measure whether or not the interleaved practice actually helped? I’m not sure that I effectively can do this based on the way the study is designed, but here is a thought. When a student tackles a question, either they use the correct technique or they don’t. What if I looked at the proportion of times a correct technique was used on Test #2 and compare it to the proportion of times a correct technique was used on the final exam? Maybe this would be helpful.
“By shortening the labours [he has] doubled the life of the astronomer.” -Pierre-Simon Laplace
I had such an interesting conversation in my pre-service math class the other day. We were solving the equation
My goal here was to get them thinking about how they could us the power laws to help them. We worked our way down to
And someone offered the suggestions that the 3^8 and the 3^4 x 3^4 were the same, so all we really needed to determine was
and we eventually settled down on x = 2, since 4^2 = 16. Then I did something weird. I told them to pull out their calculator and evaluate
to which they found the answer to be 2. Now I had them intrigued. How were logarithms connected to this question?!
They knew that logarithms were a pre-calculus operation, but hadn’t made that connection between logarithms and exponentiation. It is likely that logarithms had been taught as a series of rules to follow, without a real explicit connection to how they are the inverse operation of exponentiation – or, more importantly (in my opinion) how they solve one piece of the “triple puzzle.” You see, the process of exponentiation involves three values: the base (a), the power (p), and the evaluation (b).
We could cover any one of these numbers up, so we have three different but related problems. (1) We could cover up b; this problem can be solved by the process of exponentiation.
(2) We could cover up a; this problem can be solved by applying a radical.
(3) We could cover up p; this problem can be solved by applying a logarithm.
I am not convinced that students get enough time exploring and developing their sense of logarithms, so I suggest utilising a structure that was brought up by David Butler today (the triangle typically used in science courses to remember arrangements of formulas). I do think in our case, the structure of the triangle works a bit better than it does for science formulas. Here, the triangle works nicely for a^p = b:
What if we fill in two values:
Is it possible that we can create meaning about logarithms by using these diagrams to introduce the three similar, but related, problems? I think so. We know 2^3 = 8, so how might we reason through this?
Well, we know ? must be close to 3 since if ? = 3, we would get 8. We also know ? must be larger than 2 since 2^2 = 4. Hmm… here we might begin to introduce the clunky notation of logarithms. Perhaps log(7)/log(2), or log 2(7). Aha! 2.807 seems reasonable based on what we have thought about. And we can now flush out the problem of non-integer powers.
Anyway, I don’t think we will ever be able to get rid of the unfortunate notational issues with logarithms, but I do think we can do better making the connections back to exponentiation. Maybe there is some space in the progression of learning about exponentiation for triforce notation? As always, I welcome your thoughts.
“Acquire new knowledge whilst thinking over the old.” -Confucius
In my last post, I gave some background to the study that I am undergoing with my calculus students this term. In this post, I want to share some of the tools and methods I used to make the path towards interleaving clearer to me.
I have been a fan of interleaved practice for some time since it is well-known in the scientific community to be a successful strategy for learning (here I am thinking about learning as a flexible and long-term change in long-term memory that can be measured through test performance). However, when thinking about how to successfully implement interleaved practice it feels like a very daunting task and there are a lot more questions compared to answers:
How many questions do I assign at each step?
How do I best mix-up all the questions?
Should some topics be more weighted compared to others?
In what order to I teach the topics? Should I also interleave the way I teach the topics?
So what I did was draw some inspiration from a Slack work-group where Yana‘s husband Fabian (congrats on the recent wedding!) put together an Excel worksheet that gave a potential teaching and quizzing structure using an interleaved approach. If you open this link, you can see space to enter the topics, as well as the number of classes you have, and finally the number of questions you want per quiz. Hitting the “Do Quiz” button will create two lists: one that suggests topics to teach during any particular class, and one that suggests the topics for each quiz (which I assume happens at the beginning or end of each class).
I took this basic structure and decided to create a list of potential topics for my integral calculus course. I divided this list into six “strands” each with a certain number of “lessons” (note: I am not done finalizing this list yet – it is a work in progress). Basically I sat down, went through each chapter of the textbook and made a map of how the topics were interconnected. For example, the Sequences and Series section of the textbook discussed geometric series. Well, I could easily do this in Strand One so that students have an introduction to sigma notation before working with sigma notation with approximations. Then I could circle back to sigma notation later in Strand Six when working with Taylor Series, effectively spacing out our work with sigma notation throughout the semester. Each placement of a lesson within a strand was a calculated choice to try to space out the important ideas as best I could.
Now that I had a list of topics, I could input this information into Fabian’s worksheet and get an idea of how to interleave topics. I decided that I had already interleaved teaching topics as best I could, so I ignored the top output. I chose 4 questions per quiz and focused my attention on the bottom output. Using the output at the bottom as a model, I created a new page that listed the four questions I wanted to include on each quiz. See Sheet 3 of this workbook for that page (again a work in progress).
My final decision was not to use quizzes, but homework assignments instead. That is, at the end of each lesson, I give a PDF handout like this one to each student that is due at the beginning of the next class. This particular PDF came after the lesson on the Fundamental Theorem of Calculus Part II (FTC II). Notice that there are questions about the FTC II, but there are also questions on the topics of geometric series and the definition of the definite integral as well (the limit of the Riemann sum question).
To ensure that students complete each 4-question homework assignment to the best of their ability, I check for completion only at the beginning of class. We then take the questions up as a class – focusing on the “hard” questions that students are having trouble with. So far things have been going very well. The first test is coming up next week, and I will definitely try to blog about any interesting information I gather from looking at their responses.
“Learning to pair problem types and procedures is especially challenging in mathematics because different problem types are often superficially similar.” -Doug Rohrer
This semester I decided to create a study on interleaved practice with my second-semester calculus class. By no means is the study empirical in nature – I am not using any controls, and haven’t thought much about confounding variables. The study is more observational in nature, with the goal of collecting student solutions to analyze how students are answering specific questions.
The idea came about through an email discussion with Yana Weinstein of the Learning Scientists (and University of Massachusetts, Lowell), and Doug Rohrer of the University of South Florida. I had been interested in using some of the interleaved practice tools that Yana had helped develop in our Slack team, and she thought it would be nice to touch-base with Doug, as he thinks a lot about how interleaved practice affects students’ learning in mathematics.
There were two specific papers that I remembered reading, this being one of them. I thought the discussion on discrimination rather appealing, and something that I tended to see each semester. Roughly speaking, we teach mathematics in a particular way, scaffolding from one idea to the next, with practice questions always coming from specific chapters. As students practice, they always know the strategy that they need to use in order to solve the question (ie. they think “the questions are at the end of the lesson on the Pythagorean Theorem, so I probably have to use the Pythagorean Theorem to answer the question”). They don’t, however, get much practice mixing the different strategies that they learn. Unfortunately, this means when they come to a summative test, extra effort has to be initially put in to determine what strategy to use to solve a question.
My main goal is to do a bit of observational research around discrimination on summative tests. I have developed a schedule of interleaved homework and interleaved lessons for my integral calculus class, and we are currently off to the races. When I check back in next time, I will share some of the tools I am using to create the interleaved homework.
“The world is waiting for you. Good luck. Travel safe. Go!” – Amazing Race
On Monday this week, I ran an Amazing Race event on my campus where my calculus students moved from location to location solving review questions and racing against the clock for extra credit.
The hardest part for me was thinking through the original logistics of the event. Luckily with the help of our campus recreation coordinator, Jo Ann, we developed a preliminary schedule with colour groups. The premise was the we would divide students into groups of three, assign a colour group, then have a series of four clues at various locations around campus. At some locations, the clues would be hung on the wall, and at other locations the group would have to interact with someone (eg. our receptionist) to receive their next question.
Now that the schedule was created, it was up to me to design questions that gave an appropriate clue letting the team know where the next question was. This was a bit tricky, but after a few hours of hard work, I had a series of questions that gave clues:
- Room PE 10, clue: p'(e) = 10
- Room PE C14, clue: C = 14
- Room PL 117, clue L = 117
- Room PC 133, clue C = 133
- Room PC 146, clue C = 146
- Room PC 108, clue C = 108
So you can see for the most part, the parameter C was involved and students were somehow solving for C to receive their hint as to what room to go to. My favourite was designing a slope question in which when x = e was substituted, the answer was 10. This not only gave the number of the room, but the building too (really proud of that one)!
When the day finally arrived, I got to school early to distribute the envelopes containing the clues in their various locations. Each envelope was colour coordinated, and contained a coloured paper. This way students & staff helping would know which was the correct envelope.
Overall, I believe the students had quite a bit of fun with the activity (I heard rumor that they were telling other students about how much fun this math class was as they ran around campus). Also, the questions were challenging, but not overly taxing, which allowed a bit of group work along the way (not one student could brute-force through all of the questions). Most teams completed in about 50-60 minutes, and I ended up giving out extra credit to all who participated (tiered so that the “winners” received more). I will absolutely use this activity again – and may put in some extra locations and questions to make it a bit longer. Thank you to all the staff who helped out!
The PPT Template I used for locations where I hung envelopes on the door.
My List of Questions for each colour group.
The introductory PPT Template stating ground-rules and prizes for winners.
(of importance here is that I required full-solutions and all group members to be back in order to get their final question)