Abstract

Educators are familiar with evidence of the benefit that frequent and immediate feedback provides to our students, regardless of class size. We also often find ourselves in a situation where we do not have enough time planned or TA support available to give enough feedback to enable learners to monitor their progress. Here we describe an approach in which students take a low-stakes quiz about the assigned material at the beginning of each class period. The quiz is then peer graded as part of the class and thus provides another chance for all students to think about the problems in context.

Although we have applied this format in our smaller class setting, it is feasible and scalable for larger courses that want to provide additional or alternative options for formative assessment and opportunities to ask clarifying questions. As a strategy, it may be particularly fruitful for students challenged by gaps in learning skills or in classes in which faculty-written formative assessment feedback options are limited.

Citation

Lesson Learning Objectives

INTRODUCTION

PRE-CLASS PREPARATION

Student-centered teaching strategies have gained momentum in recent years as evidence continues to accumulate about the benefits (1). Institutions and programs took notice and instructors are replacing traditional lecture with more interactive higher-level cognitive tasks during class time or variations of blended learning (2-5). Many of these strategies maximize learning outcomes by requiring student preparation before class. Non-compliance and resistance to this requirement leads to frustration for students and instructors alike (6). While gaining popularity, the 'flipped classroom' model is sometimes misinterpreted to mean 'covering' more content, by offloading additional subject material to outside of class. However, implemented as intended, flipped classrooms enable lower level concepts to be first attempted by the student prior to the class meeting. Then, in class, the level of student understanding is checked for accuracy. When necessary, the instructor provides guidance before moving on to more complex or related topics, thus making the most of instructor-facilitated group work and class time. Holding students accountable for acquiring the foundational information beforehand is a crucial element of the flipped class format. We have used in-class quizzes, which are then peer graded for this purpose. We find that, aside from encouraging preparation for in-class activities, frequent assessments guide the learner toward higher level outcomes within a topic area.

ALTERNATIVES FOR ASSURING STUDENT PRE-CLASS READING PREPARATION

Several methods for encouraging student readiness prior to class have been described previously. Table 1 is not an attempt to comprehensively review these methods, but to provide an overview of commonly used options. These methods are not ranked in any particular order regarding usefulness or pedagogical merit. In all cases, though, it is necessary to make the expectation of pre-class preparation explicit to your students, outline the format of assessment, explain how much the assessment will count towards the grade (participation or accuracy), and describe how feedback will be given.

WHY WE USE THE PEER GRADING FORMAT IN OUR CLASS

Peer grading of in-class quizzes supports the structured student-centered format of our classes. Frequent low-stakes quizzing improves student achievement on summative exams (19,20). On the other hand, the turn-around time for returning quizzes can be longer than ideal, especially in larger classes and in those without TA support. Using the students themselves as "graders" benefits the instructor who wants to provide as many opportunities as possible for meaningful feedback.

Feedback is recognized as a key element of learning and retention (21-24). Good feedback is frequent and timely, specific to the task, and related to learning goals and objectives (22). Good feedback, assists both students and instructors in identifying knowledge gaps, and provides guidance on how to progress towards the final learning goal ("how am I doing, where am I going?"). Feedback and formative assessments are coupled and one without the other is of limited utility for self-regulated learning (21).

Additional motivation to use peer grading as a feedback opportunity includes the enhancement of some non-content aspects of learning. In our format, students have the opportunity to check their quiz answers against their peers, and access other resources (books, personal notes, instructor's narrative) for multiple explanations. This variety benefits students with different learning preferences and adds an element of collaboration that emphasizes conceptual learning (25). Peer grading sessions provide practice in scientific communication skills, as the students ask each other and the instructor clarifying questions. As many as 40-50% of our students have a primary language background other than English. These students, as well as those challenged with some form of attention or learning disorders, particularly seem to appreciate the extra interaction. Having the opportunity to be in the less-practiced role of grader, teacher, and supplier of feedback can help quieter students build confidence (26).

Learners are more likely to come prepared if there is an expectation that they will need to give and receive feedback in class (27-29). Frequent use of these and other group activities may also promote trust in peers as a learning resource. Instructors can point out that peer assessment is an important skill in professional practice. And finally, feedback is also a rich source for the instructor's reflective practice, which can inform future class sessions.

Through end-of-course evaluations, we have learned that students prefer having a hard copy of their reading quizzes to keep, and they report using the quizzes as guidance on content and question format in preparation for exams. In combination with other course materials, the quizzes remind them of the learning objectives that are emphasized in class. Student also report that seeing their incorrect answers on the quizzes reminds them of the misconceptions that led to those mistakes.

We were unable to locate published examples of the peer-grading approach that we use. A related approach, known as Calibrated Peer Review (CPR), uses peer grading of writing assignments. CPR requires more extensive preparation, including training the peers to properly evaluate the assignment (30). There have been discussions about the benefits of peer grading vs. self-grading (31), and there seems to be agreement that peer grading is accurate enough for low risk assessments (32).

TEACHING STRATEGY DESIGN

TESTED AUDIENCE

We used daily quizzes and in class peer grading of those quizzes in several semesters of a one-semester, "Introduction to Biochemistry" class. This class has an organic chemistry pre-requisite, and is itself a required course for admission to our medical school. It enrolls 3rd and 4th year undergraduates and post-baccalaureate students, most of whom plan to attend medical or other professional schools in the health sciences. The class is relatively small (~30 students). However, the peer grading approach is likely to be adaptable for any course level and can be adjusted to the needs of instructors in other disciplines.

CONTEXT

In our course, instruction time is used for regular group discussions and assignments. These groups consist of 3-4 students that stay together for the entire semester. The teams are put together as diverse as possible, with regards of demographics (such as age, gender, ethnicity, major) and if known other personal considerations. For example we avoid assigning the same group to people that are dating, or are all extroverts. We are are able to get much of this information from the previous instructors of a cohort. Another strategy, to have a small welcome survey filled out before the first class was used to help us decide.

Students are given a 10-minute quiz on the assigned pre-class material at the beginning of each class period. Peer grading, group discussion, and instructor clarification, take about 15 minutes. This sequence is typically followed by a 50-minute segment of guided inquiry for a total class time of 75 minutes. During this time the students are working in groups on worksheets or participate in an interactive lecture and graded group activity, which are designed to apply the pre-class knowledge.

IMPLEMENTATION

GENERAL PATTERN

1. Before each class, students read/review assigned textbook pages.  They use a set of learning objectives and a core slide presentation, which outlines the main concepts of the upcoming class for guidance. For example, for a session on enzyme kinetics, preceding one on enzyme regulation / inhibition, these objectives are known to the students:

1. Define and explain each of the terms in the Michaelis-Menten equation
2. Know the Michaelis-Menten equation and when it is appropriate to use (what type of enzymes are described and which assumptions are made?)
3. Solve the Michaelis-Menten equation for terms
4. Articulate and apply what the parameters Km, Vmax, kcat, and kcat/Km tell us about properties of different enzymes.
5. Describe an experimental setup that will allow you to measure kinetic parameters of an enzyme (Km and Vmax).
6. Draw and interpret Michaelis-Menten and Lineweaver-Burke graphs
7. Explain what information thermodynamics vs. kinetics data provide with regard to catalysis of enzymatic reactions.  

Each of these objectives are illustrated with a succinct slide presentation or other supplemental material available ahead of class time through the course administration system. Learners found it particularly helpful to have video clips that remind them of mathematical manipulations available (in the above example objective 3).

Students understand that foundational objectives tend to be the focus of the quiz (objectives 1-4) and that others will be given more time to work on together in class (objectives 5-7), but I don’t specify this exactly to reduce temptation that ‘gamers’ take a shortcut that would impact their group work negatively later on. However, the assignment for a focused graded group activity is posted as well, so it is clear what we are working towards; if desired individuals could prepare ahead of the class. 

Throughout the semester, they are reminded to refine their learning skills, the importance of monitoring progress and to test their understanding of the material by using one or a combination of strategies of their choice, such as end of chapter questions, creating their own questions relating to the objectives, explaining main points to a study buddy, or drawing a concept map. 

2. At the beginning of the class session, hand out paper quizzes to students. The format of the questions is flexible and may include selecting multiple choice options (single or multiple choices), writing short essay response or sketching graphical solutions when appropriate. The hard-copy quiz is typically a single page, closed book and completed individually (10 min or less).

3. Then prompt to stop and swap quizzes with students in other groups. We rotate the group-swaps, so there are different pairings each time. Encourage peer graders to work together in their small group, as they coordinate grading their assigned quizzes, and instruct them to identify and write corrections to misconceptions evident in other students’ answers, consulting the textbook or the instructor as necessary (5min).

If there are any concerns about confidentiality between students, you may consider to have students write their ID numbers instead of a name on their copy of the quiz. In our case, there did not seem to be a point, since they are a close cohort and the number of students is too small to pretend to provide anonymity.

4a. At this point solicit answers, one question at a time. Prompt to state if they changed their own ideas based on the other students’ answers, and to explain why they changed their mind. Add additional explanations or ask follow up questions (5 minutes).

4b. State the correct answers. This step provides closure and assures that uncertainties are resolved. Simultaneously, students grade the quiz in front of them. Encourage them to get feedback within their group and to leave additional comments, as well as the score, on the copy of the quiz they are working on. This step also helps the grader to resolve any of their own errors. We have seen comments like these: 

“Yeah, I got confused as well, but really C cannot be the answer, because …” or

“Our group is going to review Friday at lunch– join us?”  or simply

"you got it!” 

5. If necessary, give a mini-lecture to clarify deep misconceptions (optional).

6. At the conclusion of this section, collect all quizzes.

7. After class results are recorded, spot-check to assure that peer graders have done their job correctly and honestly. Teaching assistants may be able to accomplish this step during class so that the quizzes may be returned the same day.

COMMENTS ON IMPLEMENTATION 

The peer grading approach works best if there is a norming activity at the beginning of the semester. For this purpose, we do a ‘practice’ run with the first quiz, to clarify the expectations for graders in terms of the kind of feedback they should give and how to give it in a constructive, professional way. We suggest that the student graders use a different colored pen to make it easier to distinguish their comments from the quiz takers’ answers. We clearly state that there can be an instructor spot-check of their answers/grades, and that in the case of any inconsistencies (e.g. marking an incorrect answer as correct so that the student’s score is higher than it should be) both grader and graded may get point deductions. Thus far, this type of academic misconduct has never been observed in our classes.

Self-determination theory hypothesis that the learning progress itself can be experienced as reward, perpetuating self-motivation and devaluing learning for a grade. Yet, it is possible for the desired effect to turn sour. The result is a counterproductive classroom milieu, where learners feel monitored, perceive feedback as a control mechanism, or sense a detrimental need to compete. Fully explaining the goal of the peer grading activity ensures that learners understand how feedback is meant to help them progress. Shifting the focus deliberately to striving for personal best rather than outcompeting each other at all costs, can counteract negative perceptions and prevent disengagement (6, 33).

The instructor needs to find the balance of challenge and “friendly competition” while maintaining a safe learning environment in which all students are invited to contribute ideas and opinions. From exit surveys and verbal comments, students have mixed opinions on whether they ‘like’ the peer grading format, but they do recognize and appreciate its value.

Of course, reading quizzes need not be limited to textbook assignments, since any pre-class reading material can be used to create a quiz for the beginning of the class. Figures from primary literature, as used in many seminar style courses, can be employed as the foundation for process-oriented or analytical questions, or to test quantitative literacy. Videos on techniques or concepts can replace the required reading as pre-class assignment. News excerpts can be used to ask about arguments to promote reasoning skills.  Thus, the peer-grading of quizzes approach may be used very creatively, while still providing an informative assessment. Please, see supporting file S1(Example Quiz Questions) for inspiration.

SCIENTIFIC TEACHING THEMES

ACTIVE LEARNING

As outlined in the general pattern for implementation section, students individually review information from texts or other sources before class, using clear learning objectives as guides. During class, the students take a quiz individually. Then, in small groups at tables supplied with whiteboards, they engage in both offering and accepting peer evaluation of their own ideas. The strategy promotes student buy-in for active-learning by making the benefit of self-study, group discussion, and responsible grading procedures more transparent, while limiting the time spent on transmission-style teaching of basic content.

ASSESSMENT

Students' individual (peer-graded) quiz scores are recorded. The individual quizzes are low stakes (8 points out of possible 1000 points for course work), with the best 15 of 20 quizzes counting towards their final grade. Quality of student peer feedback on the quizzes itself is not graded, but participation is required. Summative exam assessments build on the concepts of the quizzes, and students who do well with their reading tend to score high in the higher stakes exams as well.

INCLUSIVE TEACHING

In our model, peer grading supports diverse ways for students to engage with content. It encourages use of both student- and discipline-appropriate language, promoting diverse "ways of knowing". Learners gain access to immediate feedback on misconceptions and additional opportunities to ask "stupid" questions. Many students are more inclined to ask such questions of peers, than of the instructor. Individual pre-class preparation for each class session promotes cognitive effort of all students, while allowing them to participate in ways that they feel comfortable with during the discussions.

SUPPORTING MATERIALS

S1. In-class Peer Grading-Example Quiz Questions

References

1. Freeman S, Eddy SL, McDonough M, Smith MK, Okoroafor N, Jordt H, Wenderoth MP. 2014. Active learning increases student performance in science, engineering, and mathematics. Proc Natl Acad Sci.111.23:8410-8415
2. Lund TJ, Pilarz M, Velasco JB, Chakraverty D, Rosploch K, Undersander M, Stains M. 2015. The Best of Both Worlds: Building on the COPUS and RTOP Observation Protocols to Easily and Reliably Measure Various Levels of Reformed Instructional Practice. CBE Life Sci Educ 14.
3. Wieman CE. 2014. Large-scale comparison of science teaching methods sends clear message. Proc Natl Acad Sci 111:8319-8320.
4. Drinkwater MJ, Matthews KE, Seiler J. 2017. How Is Science Being Taught? Measuring Evidence-Based Teaching Practices across Undergraduate Science Departments. CBE-Life Sci Educ 16:ar18.
5. Drake RL, McBride JM, Pawlina W. 2014. An update on the status of anatomical sciences education in United States medical schools. Anat Sci Educ 7:321-325.
6. Siedel SB, Tanner KD. 2013. " What If Students Revolt?"-Considering Student Resistance: Origins, Options, and Opportunities for Investigation. CBE-Life Sci Educ 12:586-595.
7. Jones JW, Dages KD. 2003. Technology trends in staffing and assessment: A practice note. Int J Sel Assess 11:247-252.
8. Duderstadt JJ, Atkins DE, Van Houweling DE. 2002. Higher education in the digital age: Technology issues and strategies for American colleges and universities. Greenwood Publishing Group.
9. Means B, Toyama Y, Murphy R, Bakia M, Jones K. 2009. Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies. US Dep Educ.
10. Marks BP. 2002. Web-based readiness assessment quizzes. J Eng Educ 91:97.
11. Michaelsen LK, Sweet M. 2008. The essential elements of team-based learning. New Dir Teach Learn 2008:7-27.
12. Parmelee D, Michaelsen LK, Cook S, Hudes PD. 2012. Team-based learning: A practical guide: AMEE Guide No. 65. Med Teach 34:e275-e287.
13. Kellum KK, Carr JE, Dozier CL. 2001. Response-card instruction and student learning in a college classroom. Teach Psychol 28:101-104.
14. Marmolejo EK, Wilder DA, Bradley L. 2004. A preliminary analysis of the effects of response cards on student performance and participation in an upper division university course. J Appl Behav Anal 37:405-410.
15. Herreid CF. 2006. " Clicker" Cases. J Coll Sci Teach 36:43.
16. Kay RH, LeSage A. 2009. Examining the benefits and challenges of using audience response systems: A review of the literature. Comput Educ 53:819-827.
17. Shirvani H. 2009. Examining an assessment strategy on high school mathematics achievement: Daily quizzes vs. weekly tests. Am Second Educ 34-45.
18. Weinstein SE, Wu S-W. 2009. Readiness Assessment Tests versus Frequent Quizzes: Student Preferences. Int J Teach Learn High Educ 21:181-186.
19. Barbarick KA. 1998. Exam frequency comparison in introductory soil science. J Nat Resour Life Sci Educ 27:55-58.
20. Klionsky DJ. 2001. Constructing knowledge in the lecture hall. J Coll Sci Teach 31:246.
21. Nicol DJ, Macfarlane-Dick D. 2006. Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Stud High Educ 31:199-218.
22. Hattie J, Timperley H. 2007. The power of feedback. Rev Educ Res 77:81-112.
23. Norcini J. 2010. The power of feedback. Med Educ 44:16-17.
24. Butler AC, Karpicke JD, Roediger III HL. 2008. Correcting a metacognitive error: feedback increases retention of low-confidence correct responses. J Exp Psychol Learn Mem Cogn 34:918.
25. Wiliam D, Black P. 1996. Meanings and consequences: a basis for distinguishing formative and summative functions of assessment? Br Educ Res J 22:537-548.
26. Park J, Williams K. 2016. The Effects of Peer-and Self-assessment on the Assessors, p. 249-254. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education. ACM.
27. Topping K. 1998. Peer assessment between students in colleges and universities. Rev Educ Res 68:249-276.
28. Sluijsmans D, Dochy F, Moerkerke G. 1998. Creating a learning environment by using self-, peer-and co-assessment. Learn Environ Res 1:293-319.
29. Falchikov N, Goldfinch J. 2000. Student peer assessment in higher education: A meta-analysis comparing peer and teacher marks. Rev Educ Res 70:287-322.
30. Gunersel AB, Simpson NJ, Aufderheide KJ, Wang L. 2008. Effectiveness of Calibrated Peer Review for Improving Writing and Critical Thinking Skills in Biology Undergraduate Students. J Scholarsh Teach Learn 8:25-37.
31. Sadler PM, Good E. 2006. The impact of self-and peer-grading on student learning. Educ Assess 11:1-31.
32. Freeman S, Parks JW. 2010. How accurate is peer grading? CBE-Life Sci Educ 9:482-488.
33. Deci EL, Ryan RM. 2012. Motivation, personality, and development within embedded social contexts: An overview of self-determination theory. Oxf Handb Hum Motiv 85-107.
34. Sommer F, Stahlman M, Ilkayeva O, Arnemo JM, Kindberg J, Josefsson J, Newgard CB, Fr?bert O, B?ckhed F. 2016. The gut microbiota modulates energy metabolism in the hibernating brown bear Ursus arctos. Cell Rep 14:1655-1661.
35. http://www.jove.com/video/3998/polymerase-chain-reaction-basic-protocol-plus-troubleshooting, Accessed September 29, 2016.
36. http://www.cnn.com/2015/08/09/us/colorado-epa-mine-river-spill/, Accessed September 29, 2016.