The Value of SMALL Data and Microassignments

I'm taking a few minutes this morning to write a post inspired by a Twitter convo yesterday with Yang Fan.

Thread abt 20th century LMSs want to sell the buzz words of data mining, machine learning, etc... This 👇 comment resonates with me the most. https://t.co/StvqasOrKl

— Yang Fan, instructional design (@yangfan2learn) March 26, 2019

What I want to do here is report on my use of small data to help me in my teaching. I don't have any big data, and I don't want any big data.

(the meme starts here! really!)

Instead, I design my classes so that I get the good, small data I need to help my students make sure they are making progress so that they can pass the course.

Small Data

It's not rocket science; in fact, it's very simple. That's the point! Here's how it works:

Microassignments. My system depends on using microassignments in my classes so that I can see how things are going from week to week and help students who are not making good progress. Everything in the Canvas LMS works against using microassignments (that's a topic for a separate post), but I am committed to microassignments; this approach is good for the students, and it's good for me too. I've written about this in my Canvas Community blog, and I've copied over two of those blog posts here:
Microassignments and Completion-Based Grading

Microassignments and Data Analytics

Weekly Data. So here's what my data analysis looks like right now, based on manually transferring my students' points week by week into a spreadsheet (because god forbid that the Canvas Gradebook, a faux spreadsheet, actually let me conduct my own analysis of my own data); this is at the end of Week 9, sorting from high to low for the number of weeks a student has been failing the class:

That little snapshot of data tells me a lot! It tells me that right now, out of 90 students in the class, there are 9 students who are not passing, and that has been steady most of the semester, with around 7, 8, or 9 students in trouble in any given week. For me, that's important; I know there are always going to be students who are struggling in a class. If I were seeing a lot more students struggling, I would need to think about some major class redesign, but at this level, I am more focused on working with the individual students rather than some larger course redesign effort.

Something else you can see there is that the cohort of struggling students has changed over the semester. There is one group of students who have been struggling from the very start, another set of students who have just recently fallen behind, along with a set of students who found a good routine for the class and are no longer in trouble. That's what I learn from sorting on the number of weeks students have been in the danger zone.

Data for communication. But the way I usually look at the chart is what you see below, focusing on the students who are currently having trouble by sorting on the current week's column:

Each week I communicate with the students who are currently in danger of not passing the class, but how I do that varies from student to student. I can eyeball the chart to see if they made good progress from the previous week, and if so I can send them an email of encouragement to keep on doing more of the same. By this point in the semester, I also know each student's story, so I can also take that into account when I write to them (e.g., overwhelmed by school, by work, by health problems, by life problems; every student has their own story). I have other columns in the spreadsheet that make it easy to get a fuller understanding of what is going on with each student because I have links to their class blog, their course project, along with their email addresses (I cannot stand the Canvas Messaging system and never use it unless I have to; that is also a subject for a separate post).

Data. Real Data. For me, this system works great. Every piece of work the students do for class is reflected here: their reading, their writing, their class project, their participation at other students' blogs and websites, all of their work for the course leaves a digital trail that is reflected here in those numbers. The numbers are not the work itself, but they are a good enough proxy, and I can then use the links right here in the spreadsheet to access the students' blogs and websites, seeing the work itself with a single click. For example, here is a link to my blog for the class as a student; the different types of assignments have their own labels so it's easy to browse the blog by type of assignments and/or by week of the semester. (I need to write up a post here sometime about my #TotalCoLearner experiment where I take one of my classes each semester as a student, doing all the work just like a regular student; it's so helpful!)

By contrast, the Canvas Gradebook is useless to me, so in another post I will chronicle in more detail the failure of Canvas to help me track the data I need, along with the uselessness of the data that Canvas provides me with instead. Sure, Canvas has data about my students (all those clicks! all those eyeballs!), but the data that Canvas collects is meaningless because it understands absolutely nothing about my course design. Not to mention that it knows nothing about my students and their stories.

But more on that later. For now, I just wanted to sing the praises of small data and how it makes it possible for me to keep up with my 90 students and their progress at a cost of just 10 minutes or so of my time each week.

Microassignments and Data Analytics

---

I've copied and pasted this old post from my Canvas Community blog in order to reference it here.

---

Microassignments. Each week of my class students have a set of 6 core assignments plus up to 8 different kinds of extra credit assignments they can complete (here's what a week looks like). Each assignment is worth 2 or 4 points depending on whether it is something quick (maybe 10-15 minutes) or something that takes more time (maybe 30-60 minutes). I ask students to spend appx. 5-6 hours per week on the class, which is the equivalent of 3 classroom hours plus 2-3 hours outside the classroom; the difference is that we have no classroom time since I teach fully online, so all the time that the students spend doing work for the class is active work: read, writing, interacting with each other. There are 30 points every week over a semester of 15 weeks. Students can decide whether they want to complete points for an A, B, or a C (here's how I explain that to them), but I don't get into any of that; I have literally no idea how many students in my class right now are headed for an A or B or C; the students record their points in the Gradebook themselves (here's how that works). My only goal is that everybody should pass the class with at least a grade of C, so that is the only thing I track, and in this blog post I want to show how easy it is for me to do that using a simple spreadsheet.

Analytics. So, on Monday afternoon, after each week is over, I go into the Canvas Gradebook and sort the total points from low to high. I manually transcribe the names and points of any student who has less than a passing grade (70%) for each of the three classes that I teach based on the total points so far. So, for example, last Monday was the end of Week 13. There were 390 points possible thus far, so I recorded the name and points of any student with fewer than 273 points. It takes literally just a couple of minutes to copy out the names and points, and then I paste that into my spreadsheet. You can see the result here; this shows all the students who were failing the class at any point during the semester, with their points in the weeks column, plus one column that tracks automatically how many weeks they were failing. That's literally all the data I need in a single screenshot to show how I measure course progress. The completely steady schedule plus the fine-grained microassignments make this a reliable set of measures. As you can see, some very stable patterns emerge here:

Here's what I see in that data:

Number of students failing. This is the total number of students who, in that given week (column) did not have a passing grade. As you can see, there is a large group of students there at the end of Week 2; a total of 16 students. These are the students who didn't understand at first that they really have to do work for the class every week. They slacked off in Week 2, and they could see immediately the results: not good! Of those students who were failing at the end of Week 2, 6 of them got on track and basically did not have any more serious trouble. Then the total number of students who were failing was pretty steady (between 8 and 11 students) every week up until Week 12. At that point, when they could see the end of the semester approaching, a few students really got their act together, and now I am down to just 5 students who are in real danger of not passing. That's out of a total of 90 students.

Failing weeks per student. This is another way to look at that same data in terms of the number of weeks in which students were failing the class. There is a group of 9 students that just spent 1 or 2 weeks in the danger zone, and of that group only 1 of them is in real danger now (they only recently started having trouble with the class). There are 2 students who spent 5 or 6 weeks in the danger zone, but they both pulled themselves out of trouble by Week 8 and did not have any trouble for the rest of the semester. Then there is the group of 8 students who have been in trouble for more than 10 weeks, and 4 of those students find themselves still in danger of failing now in the last few days of the semester.

These are the only students that I communicate with about their class progress. I send various kinds of assignment-based reminders to the class, especially at the start of the semester when people are developing their class routines, but the only students I communicate with about their overall course progress are the students who show up here on this spreadsheet, the students who are not currently passing. Every Monday after I transcribe the points to the spreadsheet, I send an email to the students who is failing. Sometimes it's a generic email that I send to the students BCC, but sometimes it's an email to the individual student. As I get to know more about them and the problems they are facing (lack of time, procrastination, personal troubles, etc. etc. -- each student's story is different), I can try to use what I know to write useful and encouraging emails that are forward-focused on what they can do to get on track for the class.

Note that this is just a small percentage of my students overall in the class. There are 90 students total, so that is fewer than 20% of the students who even appear here on the spreadsheet at any time, and fewer than 10% who have been seriously struggling. I'm still optimistic that all of them will pass, which is my goal for the semester. Last semester everybody passed, so that was a good semester for me. I'll post an update here on Friday when I see how this semester turns out.

UPDATE: WHOO-HOO! EVERYBODY PASSED! That makes me very happy. I also wrote up a post about end-of-semester evaluation comments from the students, including comments re: grading, here:
End-of-Semester Evals: Grading and Creativity