Goal Shift: Starting with the End in Mind

Over the years I’ve unsystematically made changes to the items on the end of course evaluation form. Typically, it was hastily considered, with changes requested at the deadline. My motivation was a desire to capture feedback related to instructional strategies I used that term. Note the key words here- “hasty” and “unsystematic.”

I needed to be more intentional. Thus began an experiment last semester. Instead of thinking about the evaluation criteria near the end of the course, I chose questions before the semester started. As I prepared to teach and throughout the term, pedagogical decisions were made with these course evaluation criteria in mind:

• Relating course material to real life situations
• Making class sessions intellectually stimulating
• Helping students answer their own questions
• Encouraging students to apply concepts to demonstrate understanding
• Emphasizing learning rather than tests and grades
• Guiding students to be more self-directed in their learning

See Feedback Questions as Course Scaffold for additional background.

Student-created Practice Problems: I created several blank templates that allow students to set variable amounts and prepare the related analyses and entries. Sometimes the template was an in-class learning activity. In other cases, I offered a little extra credit. While learning accounting is the official goal of these templates (accounting colleagues, contact me if you’d like to discuss or obtain copies), they accomplish so much more. Students engaged in an active study strategy, instead of “looking over” notes. Peer evaluation fostered collaboration and community. Prepare-pair-share led to discipline-based interaction as students discussed the variables, solved each other’s problems and corrected mistakes.

Suggested Study Timeline: For the first exam, we mapped out a suggested timeline for study. This was done during class. Then, over the following five days, I posted encouraging (and humorous) reminders ad announcements in the LMS. Later in the term, students remarked how helpful that was. They also asked if I would map that out for them again. My response- Now that we’ve developed a study plan together in class, will you create a plan of your own? Will you use this strategy in other classes? While it’s tempting to “just do it” with or for them before each test, students sometimes need the teacher to step back and not fill the gap.

Textbook Reading Notes: One of the homework assignments (toward the middle of the term) asked students to take notes on the chapter reading. This led them to a wondrous discovery: Class time makes so much more sense when you read the chapter beforehand! Students got so much from this homework they suggested it be routinely assigned. My response- Now that you’ve seen how beneficial it is to read before class, will you continue to do it, even if it’s not for credit? Will you do it for the sake of learning?

During mid-semester informal feedback, some students acknowledged they could be doing more to own their learning. That is a distinct shift from prior semesters. Instead of primarily looking to the teacher to teach, there was a clear recognition by students regarding their contributions to promoting learning collectively during class as well as the kinds of activities and effort learning requires. This closely aligns with USC’s recent initiatives in the area of student evaluations of teaching:

Umbrella questions such as, “How would you rate your professor?” and “How would you rate this course?” — which Clark called “popularity contest” questions — are now out. In are questions on course design, course impact and instructional, inclusive and assessment practices. Did the assignments make sense? Do students feel they learned something? Students also are now asked about what they brought to a course. How many hours did they spend on coursework outside of class? How many times did they contact the professor? What study strategies did they use? While such questions help professors gauge how their students learn, Clark said, they also signal to students that “your learning in this class depends as much as your input as your professor’s work.” [emphasis added] [Source: Teaching Eval Shake-Up, InsideHigherEd, May 22, 2018]

When asked to describe what the instructor did to facilitate learning, one student put it this way: “Dr. Paff had a great method of having students read the chapter and use the screencasts that she prepared to grasp some fundamental concept before class…. This style of class is effective because it drives students learning by themselves and rewards students for being good students.”

With this framework, I spent less time picking out homework problems from the textbook (a standard practice in accounting) and more time devising strategies that help students become answerers of their questions, promote self-direct learning, and make class time intellectually stimulating. Students still learned accounting, but this time they learned about themselves as learners too.

Feedback Questions as Course Scaffold

The semester is winding down.  Soon we’ll face the sometimes anticipated but often dreaded task of reviewing students’ course feedback. While many schools refer to them as student evaluations of teaching (SET) or SRTEs (student ratings of teaching effectiveness), they aren’t really meant to be evaluations and shouldn’t be viewed as ratings. The instruments are only one source of feedback. The unfortunate truth is they’re often assigned undeserved weight and meaning.

I’ve spent a few months reading the course feedback literature.  Much of it is conflicting and I’m not confident enough in my understanding to write about best practices yet. So for now, let’s put aside the hot button issues related to ratings differences and biases across major, gender, student-level, discipline and other concerns.

Instead, I’d like to focus on the questions. Do you have the authority to set or change some or all of the questions on your feedback instrument? If so, when was the last time you did? Do you ask the same questions in different courses? Why or why not?  I recently examined my forms, reviewed our university’s question pool, and made significant changes. It was interesting and eye opening. I highly recommend going through this exercise during a break, not mid-semester (speaking from experience).

Like many schools, our form specifies some questions.  Beyond those mandated by the university, college, and program I have a lot of choices.  Here are a few that capture my instructional goals. Rate the instructor’s skill/effectiveness in

• Relating course material to real life situations
• Making class sessions intellectually stimulating
• Helping students answer their own questions
• Encouraging students to apply concepts to demonstrate understanding
• Emphasizing learning rather than tests and grades
• Guiding students to be more self-directed in their learning

These items reflect instruction involving more than content coverage. I want to develop students as learners while they master accounting or economics.

The questions asked in prior semesters primarily focused on the administrative aspects of instruction such as my availability during office hours, preparation for class, skill in stressing important material, and presentation clarity. These are important, but they present a very narrow interpretation of teaching.  So much more should be happening in vibrant, engaging learning spaces; course feedback should reflect that.

Unfortunately, I cannot predict how students will react. I may, as a dear friend suggested, need to sip a good glass of wine when reviewing them. Students, unfamiliar with these questions, may not know how to respond. They may have never thought about the educational process in these terms.  This made me realize how the questions can and should be used for more than obtaining feedback at the end of the semester.

Teachers can use them as a framework for scaffolding instruction. Here’s an example to illustrate. On the wall opposite my desk at home is a sheet of paper titled The Far Goal. It lists four questions that drove my scholarly activity over the past several years.  This was a period where I shifted away from economics scholarship toward faculty development, mentoring and consulting.

• What publishing, writing, and thinking positions me for the far goal?
• What publishing projects will best support writing a book?
• What publishing projects will open more consulting opportunities?
• What publishing projects will continue the quest to be the best teacher I can become?

Each time I sat at my desk, the Far Goals were there, providing a visible reminder of what I want to achieve. The Far Goals provide a framework from which consulting and research opportunities and projects were evaluated.

Similarly, course feedback questions can scaffold instructional decisions in many ways: choosing how and when to cede control; selecting  formative and summative assessments; balancing student- v. teacher-selected topics; and instructional practice decisions like developing a pool of response strategies aimed at helping students answer questions for themselves.

Teachers can also share their questions with students at the start of the course.  They can be part of the syllabus and class discussion.  Building on this process may lead to asking students to select items from a pool of choices, thereby sharing ownership, responsibility and control of learning even more.  If you haven’t thought about course evaluation questions or the syllabus in this way before, please considering reading Learner Centered Syllabi for details about my syllabi, and beliefs about sharing control with students.  Reservations & Setting the Table for Learning considers how tone and underlying messages can detract from learning even before the course starts.

I also plan to reframe the questions as Instructional Goals to be prominently posted in my school office. The Instructional Goals will orient my choices and remind me of my teaching/learning priorities before I head to the classroom each day.

Does your course evaluation incorporate particularly good questions? Are you using them in unexpected ways? Please share!

We’ll examine the results and their implications in January.   Until then, thank you for sharing your ideas and feedback in the comments and through email this year. I appreciate learning from and with you.  Happy Holidays!

Beating the Doldrums

I’m currently reading Duffy & Jones (1995) Teaching Within the Rhythms of the Semester. I just finished the chapter titled, The Interim Weeks: Beating the Doldrums. Quite a coincidence! The reading matches my reality. Dictionary.com defines doldrums as “a dull, listless, depressed mood; low spirits.” It’s an apt description of post-Thanksgiving break letdown. Are you experiencing it too?  Are your students?

Duffy & Jones describe the doldrums’ implications for learning this way:

“During a class period, professors and student interact intellectually and emotionally; they rely on each other to move the course forward. From an emotional point of view, the response of students can influence a professor both positively or negatively. Students who are attentive and focused in class provide the necessary spark for a professor who is feeling tired or discouraged, and a student with a good sense of humor can shift the mood of a class, redirecting potentially negative feelings into positive interactions… In contrast, the apathy of inattentive students is one of the greatest challenges. The ways in which professors acknowledge the doldrums and the techniques that they use to cope with them are critical, for it is the professors’ responses that will determine whether the semester ends with a bang or a whimper” (pp.162-163).

What happens when teachers overlook or disregard the significance of the emotional ebbs during the semester? Student energy wanes and class preparation falls. Unprepared students can’t engage effectively, resulting in weak class discussions. Flipped instruction becomes less effective when students aren’t motivated or prepared. A common instructional response is to resort to lecturing, which may further decrease students’ energy and motivation.

Some teachers may feel powerless to reenergize the class. Others may not believe it’s their responsibility. A few may blame “students these days.” Absences increase and for some teachers that can lead to resentment and frustration.

What can teachers do to minimize the doldrums?

Reconsider WHAT & WHEN. Some faculty teach material in the order it’s presented in the textbook. That may not be the most pedagogically effective in light of the doldrums. If the most challenging course content coincides with a period of listlessness, learning will be negatively impacted. To address systemic doldrums, like the period after a break, course planning should be designed with a focus on these questions:

• What topics are typically most interesting, from the students’ perspective?
• When should the most and least interesting material be taught and learned?

Save fascinating topics for the emotional low points of the semester. Front load challenging material whenever possible, to take advantage of the enthusiasm that exists at the start of a new term.

Reconsider HOW. If altering the content or sequence is impractical or pedagogically unsound, consider introducing a fresh approach when the doldrums occur. Novelty is a powerful force in regaining students’ interest, attention and focus. Do something unexpected. Introduce alternative media, allow different assignment formats or integrate some student choice. Bring in props. Incorporate humor. Integrate activities that encourage interaction and collaboration. Anything that breaks the routine can be the spark that moves learning forward.

The National Oceanic and Atmospheric Administration (NOAA) provides a slightly different and visually apt definition of the doldrums: a nautical term referring to a belt around the Earth, near the equator, where sailing ships sometimes get stuck on windless waters (this can endure for weeks!). Without action, the doldrums can persist in the classroom as well. Thus, teachers bear some responsibility for restoring progress. To do so, teachers should acknowledge the doldrums explicitly with their classes. It’s important to explain the reasoning behind the strategies we use to reinvigorate learning. And we should ask students about their effectiveness so that the doldrums are temporary lulls, not permanent fixtures along the learning journey.

What strategies do you employ to lift your students’ sails during the doldrums?  Please share in the comments.

Reference:

Duffy, D.K. & Jones, J.W. (1995). Teaching Within the Rhythms of the Semester. San Francisco: Jossey-Bass.