Good Intentions & my “Epic Fail”

I recently wrote about a cheat sheet and quiz wrapper strategy. Students are allowed to prepare one side of a 3×5 index card to be used during a quiz that’s worth 10% of the course grade.

Good Intentions

I intentionally refer to the assessment as a quiz, not an exam. I stopped calling it a test when I sensed the names “exam” and “test” increase anxiety in unproductive ways. I also wanted to make a clear distinction on the syllabus. Learning in this course is assessed in a variety of ways: homework, classwork, essay exams, multiple-choice quiz, paper, and project.

The multiple-choice format is also an intentional decision. The quiz assesses students’ ability to perform computations, interpret them, interpret related graphs, and understand the implications of a specific set of characteristics. The quiz assesses lower-level Bloom’s taxonomy learning. Understanding the concepts and preparing computations is the foundation for more advanced thinking and analysis in the second part of the unit. Unlike the essay exams, where I want students drawing graphs and explaining in writing, the mechanical basics addressed in this part of the course can be assessed conveniently with bubble sheets.

I introduced the cheat-sheet index card policy a couple of years ago, as a way to reduce stress during the quiz and promote active study strategies. The goal was to get students thinking about the material earlier and differently, prepare more effectively, and perform better on the quiz.

Unfortunately the strategy is,

as my kids would say,

an Epic Fail.

Failed Stamp Showing Reject Crisis Or Failure

Calling it a quiz has not reduced anxiety. Because this is the first and only multiple-choice assessment, students don’t know what to expect. Lack of familiarity increases apprehension, regardless of what I call the assessment.

To give students an idea of what to expect, I provide a “practice quiz.” Perhaps it should be called “sample questions.” The practice quiz has the unintended consequence of limiting the scope of material studied. At least one student noted they didn’t study anything that wasn’t in the practice quiz.

Calling this assessment a quiz may actually produce more harm than good. “Quiz” may be less stressful than “test” or “exam.” But some stress is good. An unintended consequence of the name change: students may study less when it’s “only” a quiz.

Because quizzes may be seen as less important than exams or tests, some students may conclude the “cheat sheet” notecard is unnecessary. Thus, some students were insufficiently motivated to prepare one. In one case, a student noted they forgot about the quiz. Another prepared a card but neglected to bring it. Overall, about 10% of the class didn’t have one.

To be effective in promoting learning and improving scores, the card needs to be prepared in advance. Unfortunately, I noticed some students writing on theirs in the few minutes before the quiz. Others turned in cards that were incomplete or disorganized. I haven’t been able to analyze the data yet, but early findings are clear: many students aren’t preparing the cards or studying for the quiz as I intended.

From the student perspective, the purpose of a cheat sheet card is to improve their quiz score. Unfortunately, anecdotal evidence (which is all I have at this point) doesn’t bear that out. Grades on the quizzes aren’t much, if at all higher than before I allowed the cards.

That’s at least partly due to the observations above. I tried to convince students that the act of preparing the card promotes learning. The value of the card during the quiz is directly related to the quality of time and effort that went into preparing it. The message didn’t get through. Worse, feedback suggests students may have shifted from thinking about concept interrelationships toward putting basic definitions on the card.

Readers of this blog may be surprised (disappointed?) by this post’s focus on an ineffective strategy. But there is much you and I can learn from epic fails. Here are two quick takeaways:

  1. Wrappers or other mid-semester feedback is vital to understanding our students. I’m gaining valuable insight from the honest admissions about study time and strategies used. A lot of it depressed me (more about that soon). But I can’t improve instruction or enhance learning if I’m unaware of where my students are as learners.
  2. We learn a lot from mistakes.  Be brave. Pick one instructional strategy and critically examine the intended and unintended consequences. What are your assumptions? What are your intentions? What evidence can you gather to test how well they are or aren’t being met?

I’d appreciate learning from your “epic fail.” Please share what you learned from a strategy or policy that didn’t work. Let’s learn from and with each other.

Advertisements

Cheat Sheets & Wrappers

Crystal candies

For the past year or so I’ve been allowing students to use a note card, essentially a “cheat sheet,” when completing the single in-class “exam” in my microeconomics course. I put exam in quotes because I refer to the assessment as a quiz. It’s essentially an exam; it carries the same weight as a test. But, like calling an assignment a warm-up instead of homework, naming it a quiz makes it somewhat less scary.

Students appreciate having their card during the quiz. From the learning standpoint, I see it as an incentivized study activity. To be most helpful during the exam, students have to really think about the content. What do they understand best? What areas are most confusing? How can they use the card to clarify understanding?  How should the content be organized? What are the priorities? What are the interrelationships between and among concepts?

Song, Guo & Thuente (2016) report a positive correlation between the quality of the card, in terms of organization and content presentation, and exam performance.  My students’ quiz track similarly to their card quality.  The key questions for me are: Do better students already know how to study and prepare the cards? Or, can weaker students be coached on card preparation as a study strategy, to improve learning and academic performance?

Focusing on the students who have been less academically successful, I’ve become very intentional about discussing study strategies during class this semester. I handed out the cards this week, describing their preparation as a learning activity, not just an aid during the quiz.

I set aside just a few minutes of class time to talk about how and when to prepare them. Preparing the card is best done after the student has invested some time in the material, discovering potential problem spots.  We discussed a timeline for gapped study.  I reminded them of practice questions, resources and active learning strategies.

Of course, I also reviewed the “rules” for the cards.

  • The card must only have information on one side.
  • Their name goes on the other side.
  • Cards are turned in with their exam.
  • Content on the card must be handwritten. No multiple reductions of cut-and-pasted content.

Another twist I’m adding this term is a post-quiz wrapper. Here are the details.

This activity is designed to give you a chance to reflect on your quiz performance and, more importantly, on the effectiveness of your preparation. Please answer the questions sincerely. Your responses will be collected to inform instruction; they have no impact on your grade.

  1. Approximately how much time did you spend preparing for the quiz? _______
  2. What percentage (%) of your time was spent
  • Reading or re-reading the textbook
  • Self-testing / reciting
  • Reviewing homework & classwork solutions
  • Reworking problems & in-class practice
  • Watching the screencasts
  • Reviewing your own notes
  • Preparing the note card

Other? Please explain.

3. What areas do you think were most challenging on the quiz?

  • Trouble with the computations
  • Unclear about vocabulary
  • Confusion about the graphs
  • Lack of understanding of concepts
  • Careless mistakes

Other? Please explain.

  1. Did you feel prepared for the quiz? Are you surprised by your grade? Please explain.
  2. What advice would you give to future students preparing for the quiz?

I’ll share the takeaways in a future post.  What are your thoughts or experiences with cheat sheets? Have you asked students to reflect on how they used them or what they discovered about themselves as learners from preparing them? Please share!

Reference: Song, Y., Guo, Y., Thuente, D. (2016). A Quantitative Case Study on Students’ Strategy for Using Authorized Cheat-sheets. IEEE Frontiers in Education Conference Paper. 10.1109/FIE.2016.7757656, 1-9.

 

Discourse in the Classroom- Now more than ever

I’ve often wished I taught in the humanities. I imagine lively discussions in classes dealing with art, poetry, history, and philosophy. Contrast this with my actual teaching: debits and credits, marginal cost computations, and graphical analyses. Integrating classroom discussion in technical contexts is difficult.

But working with faculty across disciplines reveals my imagination is vastly different than reality. Literature, history, philosophy and art teachers find discussion-teaching and promoting student interaction challenging too.

The Foundation for Individual Rights in Education (FIRE) survey findings on student attitudes toward free speech reminds me of the value of classroom discussion in learning. (Reported by Inside HigherEd here) Two points struck me:

  • About 92% of students surveyed believe it is important to be part of a campus where they are exposed to ideas other than their own.
  • When students hear an opinion they disagree with in class, about 60 percent said they would attempt to understand their classmates’ views — 28 percent said they might avoid future interactions with the person with a dissenting opinion, but only 5 percent thought the person shouldn’t have expressed it.

What are the implications for teaching and learning? More than half the students surveyed say they are willing to hear each other out, to consider alternate viewpoints. Are we taking full advantage of the opportunities to get students listening and learning from each other? What insights and perspectives might be gained as students, from a range of backgrounds and life experiences, share their understandings of content and issues?

Of course, getting students to talk, particularly about sensitive topics isn’t easy. Many students don’t want to participate. Others do so reluctantly, offering short, tentative comments. When only a few students participate, the rest of the class gets tired of hearing them. The class may stop listening. If only a few students answer, rather than thinking about the question, the rest of the class may wonder when the over-participator will speak up. If the students who are not speaking are waiting for the over-participators to speak, they aren’t developing critical thinking skills, crafting a response, or defending their position. Class discussions can become narrow in focus, representing only the views and perspectives of the small percentage of students who choose to speak.

It’s not just students who hesitate to engage in classroom dialogue. Discussion teaching presents challenges for the teacher too. Discussions are unpredictable. Outcomes are messier to obtain and define. Teachers need to shift from expositor of knowledge to discussion facilitator. That requires a different set of skills, not typically part of graduate training and different from the ways in which many of us were taught.

But teachers are in a unique position to provide opportunities for students to practice discourse, as part of discipline-based learning. Now, more than ever, students need opportunities to learn how to discuss topics with reasoned statements, supported by evidence, without resorting to name calling and hostility.

Here are a few resources on facilitating classroom discussion, to help you get started or reinvigorate your discussion-teaching repertoire.

Brookfield, S. & Preskill, S. (2005). Discussion as a Way of Teaching. San Francisco, Jossey-Bass.

Jones, R. C. (2008). The “Why” of Class Participation, A Question Worth Asking. College Teaching, 56(1), 59-63.

Petress, K. (2006). An Operational Definition of Class Participation. College Student Journal, 40(4), 821-823.

Welty, W. M. (1989). Discussion Method Teaching: How to Make it Work. Change, July-August, 40-49.