Using Checklists in Online Teaching

Context

The Fall 2020 semester turns out to be one of the stranger semesters that I have experienced, and Spring 2021 looks like it will be much the same. Many faculty members (myself included) have opted to teach online this semester. We reached this decision after much discussion and debate. In the end, being able to avoid exposing students to the COVID-19 virus, but also to limit the chance of catching/spreading it ourselves was our main driver.

Initially, I was a little reluctant to do so since it is harder to interact with the class and watch their comprehension of the subject material. However, I have to say that my students this semester are doing at least as well as they have other years, and maybe might be doing a little better.

I did not expect that.

I teach four classes this semester: one first-year class, two sophomore classes, and one second-year graduate class. The first one, our CSC190 Computer Science Orientation Seminar, is a 1-credit class that meets only once a week. I opted to teach it synchronously. The course introduces new students to computational thinking and it coaches them in the logistics of being a college student.

The other three courses are also fully online but are delivered as a mix of synchronous teaching and asynchronous teaching. In synchronous instruction, everyone is on a video call simultaneously. There is real-time interaction among the students and between students and faculty. In asynchronous education, students are responsible for watching videos that I post to YouTube and for doing readings on their own time.

To verify that they actually watch the lessons and complete the readings, I ask students to submit their notes and take a short quiz (8-10 questions) at the end of each week. One the one hand, it keeps them engaged with the materials. On the other hand, it allows me to check comprehension.

I use the synchronous classes to review the lecture materials in about 20 minutes. I use the remainder of the time for q&a, discussions, and in-class activities.

Checklists

Why this elaborate intro? I felt that it was needed to illustrate that, while nothing in this is hard, there are many moving parts that I, as the instructor, need to monitor. For each class, I want to make sure that it opens on time, that I have an opening slide that is displayed before the start of the class, that classes are recorded, and that recordings are posted. For each class, I must make sure that all activities are posted, with deadlines and appropriate exceptions, etc. Nothing is hard. But, with four courses, it is easy to miss a step.

The title of this post is Using Checklists in Online Teaching. I realized that I was making mistakes in presenting the materials to the students almost from the start of the semester. They were often minimal issues that didn't impact the content of the lesson. Still, they do have an effect on the experience. Errors like setting appropriate quiz deadlines, making sure that students with accommodations have the extra time, ensuring that I record each session and making it available, etc. are easily made.

Years ago, I read the Checklist Manifesto by Atul Gawande. In it, the author makes a compelling argument that checklists, in all their simplicity, are excellent tools to ensure that steps aren't skipped, and avoidable mistakes aren't made. Checklists are not meant to be step-by-step instructions; instead, they provide a quick, lightweight mechanism to ensure that nothing was missed.

Many industries --- notably aviation and medicine --- use checklists. This semester, I decided that I would give them a go too in my teaching. Overall, the outcomes have been very positive. I catch mistakes easier and earlier, which means that I get fewer emails from students with avoidable questions. Because of that, my courses are now more predictable, allowing students to prepare better, and get better outcomes.

My checklists are available here. Feel free to use them for your own courses, or develop new ones based on them. If they help you out, please let me know! I would love to hear about your experiences and see what improvements you made!

Coronavirus and Fall 2020 preparation

The Coronavirus has thrown us all for a loop. Most retail is closed, supermarkets are structurally running low on supplies, and selection is limited. Employees are required to work from home if they can. Those who cannot work from home, and who are not deemed essential, are either fired or furloughed. The government's recommendations ask people to limit interaction with others, and use protections such as social distancing, face masks and gloves when direct contact is needed.

We (higher education) are fortunate that we are both considered essential, and can do most of our work remotely. Since Governor Cuomo has just announced it today, we now know for sure that the remainder of this academic year will remain online. While he did not come out and say it, it is clear that this includes the summer sessions.

I believe it to be unlikely that the Fall 2020 semester will revert to business-as-usual. Even if we reopen, it just won't be possible to have 30 students in a classroom and keep all protections in place. The same is true for office hours, tutoring sessions, etc. Until antibody testing is cheap, readily available and reliable, AND we know that having antibodies prevents illness, this will be the new normal.

I am impressed by my students and their willingness to stick with me. All my undergraduate sections are full, and I have waiting lists going for students who want to over-enroll. The picture for incoming first-year students isn't as good. We believe that a good number of high school graduates will delay going to college by at least one semester.

So, that brings the thought: how should I prepare for our next academic session? Let's analyze that rationally and ask what it means to teach and learn in a college course? Many professors typically structure their courses as a sequence of topics, and students must complete specific activities for each.

  1. Preparation (reading)
  2. Instruction (lecture)
  3. Classroom discussion and questions/answers
  4. supervised lab work to develop hands-on skills
  5. independent work and/or group assignments to develop those skills when no direct supervision is around
  6. assessment of outcomes

Preparation

The general idea is that students prepare readings before they come to class. That, of course, rarely happens. However, both topic preparation and instruction are easily moved online, through the dissemination of video recordings and assigned readings. That will only be effective if the videos useful, and the instructor is concise and to-the-point.

Unlike lecture-style delivery, students can watch videos at any time and as often as they need. Active learners can benefit from this method of asynchronous delivery, and for my CSC380 Computer and Network Security, I have produced videos and posted them to YouTube. Feedback is overall positive.

Interaction

While classroom discussion and q&a sessions can be done asynchronously as well through the use of email, blogs, discussion forums, etc., these technological options are no substitute for real-time direct communication. Videoconferencing can be a solution, but it has to be a complete buy-in by students and faculty alike.

However, students (and instructors) must have a technological infrastructure in place, have a quiet place to work, and should actively participate at a given time. Those are a lot of conditions, and it isn't always feasible to do so. However, if in-person instruction in the classroom is not an option, this is the next-best option.

Adelphi University supports both Zoom and Google Hangouts. My preference is for Zoom.

Labs

I teach computer science and information systems classes, and labs are an essential part of our method of instruction. Students work in small groups on guided activities, and can ask questions of peers, teaching assistants, and professors in real-time, and as they are working on problems. I have not found a good alternative for labs yet. Videoconferencing is clumsy; discussion forums remain unused, and Slack doesn't seem to appeal to students.

Independent work

Independent work (homework) is much the same. For years, I have been assigning work on our LMS (Moodle). Since I tend to lose assignments handed in on paper, and because we all like trees, for as long as I can remember, I have required that students hand in all assignments electronically.

Assessment

Assessment is important. For me, as an instructor, it helps me understand how well my students are mastering the materials that I am teaching them. Also, grading rubrics provide clear expectations for students. Assessment typically takes place in the form of quizzes and exams.

When dealing with remote education, that kind of assessment suffers from two main problems.

First: if I administer a written exam in a traditional on-ground classroom, all my students are in the same room, at the same time, for the same duration, and everyone has access to the same resources. In other words, there is a level playing field. Such is not the case when we are dealing with remote education. Everybody's circumstances are different. It might not be possible to find a quiet place for two hours straight without interruption. Students may have to share technology with family members. Internet connections may be unstable, etc.

Second: honesty is an issue. If nobody is watching, people may be tempted to cheat, and that means that the assessment is ineffective and unreliable. While there are all kinds of browser plugins and a wide range of technological solutions, none of those methods is effective without being overly invasive.

So, as a rule, I do NOT require exams when teaching remotely.

So, what do I do? What I have always done. I give my students weekly assignments, which is the basis for a continuous assessment strategy. There are so many benefits for the students that it is hard not to go this way.

I have added two steps: each week, I expect students to submit a "summary of learning," which includes an abstract of their readings, and notes of my videos. I also administer a short multiple-choice quiz. Students get three attempts to pass the quiz; the highest grade counts, and after each attempt, they can see which answers were correct and which were not. The quiz is also not locked down, as the main goal is to see if students watched and understood the videos.

Next semester

For now, I am going to prepare my Fall classes as if I have to teach them online. That means that each week will consist of

  1. Assigned readings (asynchronous)
  2. Video instruction (asynchronous)
  3. Classroom discussion and q&a (synchronous)
  4. Independent work:
    • summary of learning
    • short quiz
    • additional work, depending on the class.

I have not yet figured out how to deal with labs, but I'd love to hear suggestions.

CODASPY Canceled, Short paper published

ACM CODASPY '20 is the 10th ACM Conference on Data and Application Security and Privacy.

Sung Kim and I have been working on research involving creating models that can be used in proactive cyber defense practices. Our short paper and poster presentation were accepted for publication.

Unfortuantely, the conference was canceled as a result of the COVID-19 outbreak. The article can be found in the ACM Digital Library.

Service-Oriented Modeling for Cyber Threat Analysis

ACM CODASPY '20 is the 10th ACM Conference on Data and Application Security and Privacy. Sung Kim and I have been working on some research involving creating models that can be used in proactive cyber defense practices. We will be presenting at the conference on March.

The Computing Orientation Seminar

ACM Inroads, a peer-reviewed magazine serving professionals interested in advancing computing education on a global scale, published an article I wrote together with Sal Petrilli.

The title of the article is The computing orientation seminar and it is available in the ACM Digital Library.