Just-in-Time Teaching

(Usage hints for this presentation)

Computer Structures and Operating Systems 2023
Dr. Jens Lechtenbörger (License Information)

Data Science: Machine Learning and Data Engineering (Prof. Gieseke)
Dept. of Information Systems
WWU Münster, Germany

1. Usage hints

2. Motivation

2.1. Initial Problem and Improvement

2016: Classroom response system revealed lack of student understanding
- Yet, no in-class discussions, leaving me frustrated
  - Waste of our time
After introduction of JiTT: Situation improved

Improved Java MX understanding

I started with Just-in-Time Teaching (JiTT) in 2017, after I realized in the previous year that my mode of lecturing was a waste of our time (students’s time as well as mine). In particular, a classroom response system revealed that students did not understand what I wanted them to understand.

Here, you see results concerning a question that I have been asking since 2016. You will learn about its topic, mutual exclusion (MX), in this course, and I expect you to be able to explain that concept. In 2016, with traditional lecturing, I succeeded for about a third of students, while the majority was confused but unwilling to ask questions in class. This in itself is bad.

Even worse, let me point out that students were quite happy with the course, as indicated by the average evaluation grade of 1.8 (on a German scale from 1.0 as best grade to 5.0). This outcome raises deep questions about learning and teaching.

As shown in the figure, results improved considerably with JiTT.

Note that I did not just switch to JiTT. I learnt a lot about learning, for which you can find some pointers in this presentation; important key words are deliberate practice, active learning, and self-regulation.

In this regard, note that actual learning involves effort, while evaluation results might reflect a feeling of learning. This is explored in some detail in [D+19], about which the lead author Louis Deslauriers says this: “The effort involved in active learning can be misinterpreted as a sign of poor learning. On the other hand, a superstar lecturer can explain things in such a way as to make students feel like they are learning more than they actually are.”

2.2. General Improvements

Improved understanding compared with traditional lecturing

3. Just-In-Time Teaching (JiTT)

3.1. Overview

Teaching and learning strategy based on web-based study assignments (self-learning) and active learner classroom
- See JiTT on Wikipedia
- [MSN16] demonstrates improved learning for statistics courses
JiTT is an instance of active learning, which leads to improved learning in general [FEM+14]

3.2. Feedback Cycles with JiTT

Feedback cycles with JiTT

“Feedback cycles with JiTT” by Jens Lechtenbörger under CC BY-SA 4.0; from GitLab. Includes icons by The Noun Project under CC BY 3.0 US: Meeting by Ainsley Wagoner; knowledge sharing, professor, student, audio lesson, online task, online communication by ProSymbols.

The essence of my interpretation of JiTT is captured in this figure, which features three feedback loops:

Online discussions, e.g., in the Learnweb forum as in every other course
My feedback to your answers to JiTT questions
Focused discussions, shaped by what I learnt in JiTT answers

In pre-corona times, not many students attended class meetings. Evaluation results show two major reasons. First, students reported that their questions were answered ahead of time, leaving little value to class meetings. Indeed, some students even suggested that I would need to decrease the quality of provided material if I wanted more students to attend. I won’t go there. Second, some students decided to learn independently of our class rhythm.

In view of those results, I suggest not to rely on synchronous class discussions in 2020 too much. Instead, I suggest that you formulate your questions online when they come up.

On a personal note, I believe that synchronous meetings are most effective for small groups. As a class, we are no small group. In contrast, your group work will certainly benefit from synchronous meetings (with discussions of topics that are relevant to you, supported by your preferred tools at your preferred points in time).

I’m here to help.

3.3. Benefits

Feedback loop: Your out-of-class preparations are visible to me, allow me to offer feedback
More structure for out-of-class learning
- Content and questions, to be tackled at individual learning pace
More efficient interaction
- Identification and correction of misconceptions/misunder-standings/incorrect prior beliefs
- Valuable shared time is not used for one-way lecturing but for student-student and student-instructor interactions
  - Traditionally, you figure out what’s complicated when you are on your own
  - Now, we discuss once you found out what’s complicated

4. JiTT Organization

4.1. JiTT Assignments

Upcoming presentations will contain assignments to be submitted by you via Learnweb
- Exercises
  - Usual deadlines on Thursdays as in CS part
- But also voluntary assignments with earlier deadlines
  - Tuesdays at 8am
  - Thursdays at 10am
  - Giving me some time to check solutions ahead of scheduled meetings
  - Voluntary assignments serve as formative assessment; points without impact on grading
  - Voluntary assignments include one task to solicit questions/comments
    - Next slide

4.2. How to obtain Feedback?

Each JiTT assignment ends with this task:
- This slide serves as reminder that I am happy to obtain and provide feedback for course topics and organization. If contents of presentations are confusing, you could describe your current understanding (which might allow us to identify misunderstandings), ask questions that allow us to help you, or suggest improvements (maybe on GitLab). Please use the session’s shared document or MoodleOverflow. Most questions turn out to be of general interest; please do not hesitate to ask and answer where others can benefit. If you created additional original content that might help others (e.g., a new exercise, an experiment, explanations concerning relationships with different courses, …), please share.

4.3. Asking for Help

Some students struggle on their own for hours, getting frustrated
- Asking on so-called social networks where I am not around
  - Why do they do that? Seems inefficient to me.
- Asking search engines
- Consulting YouTube (sometimes with faulty explanations)

I suggest to ask (and answer) earlier and elsewhere
- In Learnweb forum
- A collaboratively edited document, URL in Learnweb

4.4. Sample Student Feedback

4.4.1. Negative Feedback and my Responses

“JiTT destroys our freedom!”
- Two meetings per week are given, define rhythm
  - You may adopt that rhythm, benefit from my help
  - Or explore alone at your own pace

“JiTT tasks are too difficult/open!”
- CSOS is worth 9CP, almost a third of term’s workload
- I do not just want you to remember my steps
  - I hope to instruct for independent movement in unfamiliar terrain
  - With challenging (I hope) hurdles and individual feedback
    - Missteps are part of learning
    - I’m here to help
    - If you ask early, you may receive help before deadlines are due

4.4.2. Positive Feedback

“JiTT is/was a very good idea and was very helpful to understand the course’s content”
“The JiTT-Assignment in combination with the lecture helped to understand the topics a lot!”
“Please continue with this type of lecture!”

Bibliography

[D+19] Deslauriers, McCarty, Miller, Callaghan & Kestin, Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom, Proceedings of the National Academy of Sciences 116(39), 19251-19257 (2019). https://www.pnas.org/content/116/39/19251
[FEM+14] Freeman, Eddy, McDonough, Smith, Okoroafor, Jordt & Wenderoth, Active learning increases student performance in science, engineering, and mathematics, Proceedings of the National Academy of Sciences 111(23), 8410-8415 (2014). https://www.pnas.org/content/111/23/8410
[MSN16] McGee, Stokes & Nadolsky, Just-in-Time Teaching in Statistics Classrooms, Journal of Statistics Education 24(1), 16-26 (2016). https://amstat.tandfonline.com/doi/abs/10.1080/10691898.2016.1158023

License Information

This document is part of an Open Educational Resource (OER) course on Operating Systems. Source code and source files are available on GitLab under free licenses.

No warranties are given. The license may not give you all of the permissions necessary for your intended use.

In particular, trademark rights are not licensed under this license. Thus, rights concerning third party logos (e.g., on the title slide) and other (trade-) marks (e.g., “Creative Commons” itself) remain with their respective holders.