Abstract

Undergraduates who learn to evaluate primary literature demonstrate an enhanced ability to understand the process of science, weigh scientific evidence, and think critically (e.g. 6,8). Studies show that students who learn how to dissect primary literature through active learning practices, demonstrate mastery of the deeper levels of cognitive processing (9-11). Inspired by that evidence, we designed peer-led active learning sessions called "Reading Groups" (RGs) to supplement in-class learning in an advanced biology course. RGs are moderated by peers who are not content experts. They support novice learners in organizing new information and provide focusing questions to structure discussions. Thus, RGs allow students to discuss research articles in the absence of a content expert, helping them develop the skills and confidence to pose questions during subsequent in-class discussions. This paper describes the design of RGs and reports responses from student surveys. Findings indicate self-reported gains in both evaluating primary literature and self-efficacy. Specifically, students report development of critical thinking, data interpretation, communication skills, and greater confidence in their ability to critique papers. These data are encouraging and we hope that other instructors will consider implementing RGs as a tool to assist students in reading and evaluating primary literature.

Citation

Society Learning Goals

Science Process Skills

• Process of Science
• Locate, interpret, and evaluate scientific information and primary literature
• Pose testable questions and hypotheses to address gaps in knowledge
• Interpret, evaluate, and draw conclusions from data
• Construct explanations and make evidence-based arguments about the natural world
• Modeling/ Developing and Using Models
• Build and evaluate models of biological systems
• Quantitative Reasoning/ Using Mathematics and Computational Thinking
• Apply the tools of graphing, statistics, and data science to analyze biological data
• Communication and Collaboration
• Share ideas, data, and findings with others clearly and accurately

INTRODUCTION

LEARNING FROM PRIMARY LITERATURE

The current learning expectations for undergraduate biology degree programs in Ontario, Canada, include equipping students with the skills necessary to synthesize and apply their knowledge to address global problems (1-2). One prominent feature of undergraduate biology curricula (more commonly in the 3^rd and 4^th years) is the use of primary literature as a conduit for students to attain these higher levels of cognitive operations (e.g. analysis, evaluation, and synthesis) (3-4). Numerous studies have documented the benefits undergraduates gain by learning from primary literature (e.g., 5-7). Specifically, these studies report gains in students' ability to understand the process of science, evaluate evidence, and think critically (e.g., 6,8). Moreover, cognitive science research suggests that active learning strategies are essential for effective learning (9-11). Thus, students who learn how to successfully dissect and critique primary literature through active learning practices develop deeper levels of cognitive processing.

While the case is clear for the use of primary literature in improving undergraduate learning outcomes, some impediments remain. In the experience of the corresponding author (A.A.), students are highly reluctant to question the authors' interpretation of specific experimental data; instead, they "trust" the authors to present them with facts that they should accept at face value. Other instructors have published similar observations (e.g., #12#12). Some approaches have used structured formats that encourage students to interpret the experimental data presented in papers. Of note amongst these approaches is the CREATE method (5,8) in which students scrutinize figures from multiple papers authored by the same research group and the "Figure Facts" method that emphasizes the interpretation of experimental evidence rather than simply following the accompanying text of the paper (13). These methods provide clear evidence that data-focused approaches to evaluating primary literature provide students with an opportunity to learn or improve critical thinking. However, these methods demand significant instructor time and are often used in courses focused on skill development, rather than as a component of learning in a discipline-centered course.

PEER-BASED LEARNING PRACTICES

We propose that learning in the life sciences can be aided through peer-discussion practices, which can provide students with an authentic experience in how ideas and findings are debated within the scientific community. Student journal clubs in which a subset of students take the lead in discussing a paper have been shown to be an effective strategy to encourage learning through asking and answering questions (6,15-17). Peer-based learning strategies have also been employed to scaffold learning outside the classroom. Many peer-based learning strategies stem from the Supplemental Instruction (SI) model in which peer-led sessions are organized to supplement student learning in challenging foundational undergraduate courses; such supplemental instruction can result in important learning gains for the students (18; reviewed in 19). The SI philosophy has influenced the design of peer-led team learning (PLTL) models in which small groups of students (> eight students as described in 20) are guided by peer leaders to examine challenging course material through discussion (21-24). The learning environment in SI and PLTL models allow students to learn collaboratively with peers, while being guided by a peer leader, who is not a content expert, but who guides discussion surrounding challenging concepts (19,21,25).

Inspired by the examples reviewed here and other evidence, we designed peer-led discussion sessions called Reading groups (RGs), for a fourth year advanced biology course, Pathobiology of Human Disease. The course uses primary literature extensively to discuss the cellular and molecular basis of genetic and infectious human diseases. Here, we describe the design and value of of RGs to student learning. RGs supplement an existing course and do not place large demands on instructor time or resources. We also report student feedback for consideration by other instructors interested in peer-based learning strategies.

TEACHING STRATEGY DESIGN

DESIGN OF "READING GROUPS" TO SUPPORT LEARNING IN A FOURTH YEAR COURSE

Pathobiology of Human Disease is a fourth year, seminar-style, elective course with a class size of approximately 30 students. The course is designed to stimulate critical analysis of primary literature through extensive class discussions in weekly two-hour sessions. Student groups (of ~five students) are responsible for selecting research articles that become the weekly assigned readings. Each presenting group takes the lead in the discussion, much like a structured journal club, while also soliciting and addressing questions from peers. Students as a class summarize the impact of the article along with a candid assessment of its strengths and limitations. The instructor grades the quality of the student presentations and the questions posed by the non-presenting peers. The instructor also grades students' answers to weekly-questions that evaluate the paper that was discussed in-class (Please see Supporting File S1 for grading rubrics for grading class participation and weekly questions submitted by the non-presenting peers).

This seminar format requires students to understand dense scientific literature from high impact journals, weigh the quality of the evidence presented, and come up with questions that critically evaluate the appropriateness of the methodology used and the validity of the authors' claims. These critical thinking skills are important for future scientists and are eminently transferable to new situations. Developing these skills is a time-intensive and challenging goal for undergraduates, especially for novice users of primary literature. To support this development, we created a course component that allows students to set aside time to discuss the paper with peers in the absence of an evaluator (such as a TA or instructor) and to develop both the skills and confidence in their ability (self-efficacy) to pose questions. To be most effective, this course component should provide structure so that the conversations stay focused, offer students support in organizing new information, and ensure that everyone has the opportunity to weigh in or ask questions. That is the rationale behind developing student RGs in our pathobiology course. RGs provide opportunities for moderated, peer-based active learning structured around primary literature that does not pose additional demands on instructor time and resources. The major anticipated outcome from RGs is that students come to the class well prepared to discuss and critically evaluate primary literature.

THE ROLE OF THE RG PEER MODERATOR

The peer moderator is a student volunteer who has previously excelled in this seminar course, or similar upper level courses. Given that the papers discussed in the course each year are different, we do not expect that RG moderators are content experts. Instead, they have expertise in how to approach and organize new information, as well as in managing group dynamics. Indeed, to make these roles abundantly clear to the students, we decided to use the title of "peer moderator" rather than peer leader. The peer moderator (J.D., co-author) remained the same in both the 2013 and 2014 offerings of the course; thus, the effectiveness of the peer moderator was not a variable between the two years. The peer moderator attended specialized training sessions at the Teaching and Learning unit at the authors' institution in order to hone skills in communication, stimulating group discussion, understanding the pedagogical value of peer-based learning, and encouraging student metacognition. A description of the moderator's role and moderator training sessions is presented in Supporting File S2.

STRUCTURE OF RGS

RGs met weekly for an hour in both years that we offered them. In 2013 (year 1), RGs were a weekly, optional component of the course that was not formally scheduled; students were invited to attend, if they were interested. The students and peer moderator chose a time to meet based on the availability of the majority of students enrolled in the course. Based on overwhelmingly positive course evaluation comments, we integrated RGs into the course as mandatory sessions in 2014 (year 2). It is important to note that RGs in 2014 were mandatory only in so much as they were formally scheduled course components and the instructor greatly encouraged students to attend due to her belief that they augmented student learning. While student attendance at RGs was recorded by the peer moderator, this information was only used for us track the level of student attendance and interest. However, in keeping with the SI principles of voluntary attendance, students knew that no grades were associated with their attendance, that participation in the RGs was voluntary, and that they were individually responsible for their own learning. In year 1, about 15 students (50%) of the class attended the sessions consistently throughout the year. Interestingly, there were no students who attended only an initial few sessions and decided against the idea. In year 2, when the sessions were "mandatory", all 38 students enrolled in the course attended the majority of sessions (with only absences being illness related). It should be noted that the presenting group for each week were considered the "experts" and hence were not allowed to attend the RG session for that week.

RGs include two levels of discussion: one mini-group discussion and a larger discussion that encompasses all students attending the session in an adapted jigsaw model (Figure 1).

At the beginning of the RG meeting, the moderator creates groups of five to six students; the number of mini-groups is roughly determined by the number of key results presented in paper. The mini-groups are formed by the moderator, ad hoc, based on where students were seated in that day's RG session. This strategy made group formation logistically easier. In general, most RG sessions included 3-5 mini-groups. The moderator then assigns each group a small section of the paper to prepare for discussion. On occasion, based on the nature of the research paper, the moderator might work with the students at the beginning of the RG session to determine the focus of that week's discussion (e.g. the mini-groups might focus on different methods or different figures). Each mini-group is given 20 minutes to discuss its assigned section and then the class reconvenes to discuss the paper in its entirety. In this second stage, each mini-group leads the discussion around their assigned section of the paper. The RG discussions were typically focused on thoroughly understanding the data presented in the assigned paper as well as the techniques employed.

IN-CLASS DISCUSSION FOLLOWING RGS

Following the RG discussion, students work independently to further dissect the paper in their preparation for the class discussion sessions. Specifically, they coin questions that critically evaluated the methodology, results and conclusions of the paper and are asked to consult other published work to determine the significance of the conclusions presented in the assigned paper. During class discussions, the presenting group was charged with providing an overview to the field, in addition to critiquing the data presented in the paper. The non-presenting peers are asked to pose questions about the paper and its conclusions. Students pose questions individually and their participation is recorded and evaluated for a grade (Rubric in Supporting File S1).

Our goal was to train students in RGs to pose such critical questions in these class discussion sessions. In addition, the RG fostered interactions with diverse peers: while the in-class groups were pre-determined at the start of the semester, RG groups differed every week, allowing students to engage with different peers.

STUDENT FEEDBACK

Through the use of student response surveys (Supporting File S3), we evaluated students' overall perception and satisfaction with RGs. Specifically we wanted to know how RGs might facilitate learning from primary literature and contribute to self-efficacy beliefs. In 2013 (optional RGs), 15 students out of 30 enrolled in the class completed the survey. All 30 students attended the majority of RGs for that semester. In 2014 (mandatory RGs), 28 students out of 37 enrolled in the class completed the survey. Of the 37 enrolled students, 28 students attended the majority of RGs. The survey was provided to students who actually attended the sessions, and the survey required students to identify how many sessions they attended. No marks were assigned for completion of surveys, participation was entirely voluntary, and all responses were anonymous. Presented below is a brief summary of the survey results, which serve as the preliminary indicator of whether the RGs were valuable to student learning. Detailed analyses of the student survey data is presented in Supporting file S4. We hope that these data will inform the implementation of RGs by other instructors across disciplines.

The majority of students found RGs helpful in developing skills to read and interpret research papers, regardless of whether RGs were mandatory or optional (73% in 2013 and 50% in 2014 gave it a ranking of 4 on a scale of 1-5, where 5 is "very helpful"). These results support the utility of peer-based learning practices in helping students dissect primary literature. The majority of students (60.7% and 80% in 2013 and 2014 respectively) rated themselves as "4" on a scale of 1-5, with 5 representing "very confident" when asked to report self-efficacy beliefs after they attended RGs. This observation is consistent with other studies that have shown that academic mentorship has the potential to increase student self-esteem, academic self-efficacy, and overall satisfaction with their academic program (26-27).

When asked to describe how students specifically used RGs to help them read and interpret the assigned papers, the majority reported that they used RGs to aid in the interpretation of data (55%) or experimental procedures (24%) in both mandatory and optional RGs. Notably, 11% of students reported using RGs to think critically about the data in the paper. (e.g. "...WHY certain cells are used...", "...problems with methodology...", "Discussing figures led to understand problems with methodology and data reported."). Interestingly, we were able to assess development of cooperative learning skills and interpersonal skills as students reported that RGs helped them "gain input" from peers and "see how others ...think about the paper" or "Provided an opportunity to practice explaining concepts in front of an audience." These observations are in line with the findings of others that suggest that interpersonal skills develop substantially through cooperative learning (29).

BENEFITS OF RGS TO THE PEER MODERATOR

Benefits of peer mentoring are well documented (28-31) and include development of leadership, interpersonal, communication, and teaching skills (32-35). The RG peer moderator (J.D., co-author) reports being able to better connect with her peers, take on a leadership role, and develop her own critical thinking, group moderation, and communication skills. Importantly, J.D. reports that these skills have been highly transferable to contexts outside of RGs (e.g., clinical education). The RG moderator reported particularly enjoying the experience of breaking down a challenging problem into sub-problems that groups could then tackle. Such modeling is valuable to students, given that findings from primary literature evolve rapidly. Thus, formal documentation of the learning gains of peer moderators would be a worthy endeavor for us and other instructors incorporating RGs in their courses in the future.

RECOMMENDATIONS FOR FUTURE SESSIONS

Through our surveys, we have solicited feedback to improve RGs. Currently RGs are 60 minutes in length. However many students have suggested an hour is insufficient (e.g.,"Extend the time; so far - every paper felt rushed to analyze in just one hour"; "Longer sessions to dissect the paper more deeply..."). Extending the length of the sessions may promote more thorough discussions and bring the RG structure closer in line with the 90-120 minute session recommendations of PLTL models (22). The two-stage discussion structure of RGs (Figure 1) allows students to grapple in detail with smaller components of the paper. This intense analysis preserves some of the cognitive development supported by the SI model, in which resolving differences of opinion with peers, stimulates deep learning (19). To move the RG structure into a truly collaborative learning context, we and others could incorporate an additional step in RGs. In such a model, students could return to a "home group" following a jigsaw discussion (in which each group of students discusses a specific figure or sub-component of the paper). Within the home group, each member would lead the discussion on a specific component (e.g. Figure 1) of the paper. This additional step would then be followed by a discussion that spans the whole class (Figure 2). This modification would allow students to each become a resource to their peers and their engagement in discussions would directly influence the overall learning of individual members of their "home" group. This type of collaborative learning context is predicted to enhance student learning by the social interdependence theory (36).

EDUCATIONAL IMPLICATIONS

Overall, student feedback suggests that RGs (as optional or mandatory course components) helped students understand and evaluate articles and enhanced student confidence in successfully approaching primary literature adequately for meeting the expectations of a seminar-style course. Congruent with this result, the instructor (A.A.) felt that the class discussions were more engaging, with contributions from the majority of students. The quality of the questions formulated by students at the beginning of the term improved for almost every student in the course in both years. Figure 3 shows the evolution of one student's questions from simple at the beginning to complex at the end of the term.

Given these observations, we are hopeful that RGs could be easily implemented or adapted to courses across disciplines that are based on primary literature and represent a worthwhile investment for volunteer peer moderators. The structure of RGs is also designed to fit with current educational recommendations for student-centered classrooms, which encourage an interactive, inquiry-driven, cooperative, and collaborative learning environments that nurture development of (amongst others) critical thinking, effective communication and problem solving skills (1,37-38).

SCIENTIFIC TEACHING THEMES

Active Learning

The activities in RGs required student to engage in small and large group discussions (jigsaws), interpret and evaluate data from research papers, communicate their opinions clearly to peers, and both offer and accept peer criticism of their own ideas. Outside of RGs, students were required to engage in extensive class discussions about assigned scientific articles and pose questions to the presenting group of peers.

Assessment

The instructor graded a list of questions about an assigned paper that is turned in by individual students each week at the beginning of the class discussion session (following participation in RGs). Students also provided feedback via anonymous, optional surveys.

Inclusive Teaching

RGs encourage individual preparation for each discussion class session, but allow students to participate as they feel comfortable during the RG discussions. One of the roles of the peer moderator to ensure that there is a safe learning environment in which all students are invited to contribute ideas and opinions. As described in our recommendations for future sessions and in Fig. 3, the RG structure can be further adapted such that each student's contribution to a "home group" discussion is both essential and valued.

SUPPORTING MATERIALS

• Supporting File S1. Reading Groups design - Grading Rubrics.
• Supporting File S2. Reading Groups design - Peer moderator role and training.
• Supporting File S3. Reading Groups design - Student survey instruments.
• Supporting File S4. Reading Groups design - Student survey data and analyses.

ACKNOWLEDGEMENTS

We would like to gratefully acknowledge the help provided by Christine Fornal in coordinating the RG group discussions in Winter 2014 and in providing valuable suggestions for improvement of the student surveys. We would also like to thank Dr. Mark Bayfield, Dr. Carol Rolheiser and Dr. Robin Wright, for their insightful feedback and recommendations for improvement of this manuscript.

References

1. Lennon MC, Frank B, Humphreys J, Lenton R, Madsen K, Omri A, Turner R. 2014. Tuning: Identifying and Measuring Sector-Based Learning Outcomes in Postsecondary Education. Toronto: Higher Education Quality Council of Ontario.
2. OUCQA (Ontario Universities Council on Quality Assurance). 2012. OCAV's undergraduate and graduate degree level expectations. In Quality Assurance Framework (Appendix 1). Retrieved from http://oucqa.ca/framework/appendix-1/
3. Bloom BS, Engelhart MD, Furst EJ, Hill WH, Krathwohl DR. 1956. Taxonomy of educational objectives: The classifcaiton of educational goals. Handbook 1: Cognitive domain. New York: Longmans, Green.
4. Airasian PW, Cruikshank KA, Mayer RE, Pintrich PR, Raths J, Wittrock MC. 2001. A taxonomy for learning, teaching, and assessing: A revision of Bloom's Taxonomy of Educational Objectives, edited by Anderson, L.W. and D.R. Krathwohl. New York: Longmann.
5. Hoskins SG, Lopatto D, Stevens LM. 2011. The C.R.E.A.T.E. Approach to Primary Literature Shifts Undergraduates' Self-assessed Ability to Read and Analyze Journal Articles, Attitudes About Science, and Epistemological Beliefs. CBE Life Sci Educ 10(4): 368-378.
6. Kozeracki CA. Carey MF, Colicelli J, Levis-Fitzgerald M, Grossel M. 2006. An intensive primary-literature-based teaching program directly benefits undergraduate science majors and facilitates their transition to doctoral programs. CBE Life Sci Educ 5(4): 340-347.
7. Yeong FM. 2015. Using primary literature in an undergraduate assignment: Demonstrating connections among cellular processes. J Biol Educ 49(1): 73-90.
8. Gottesman AJ, Hoskin SG. 2013. CREATE cornerstone: introduction to scientific thinking, a new course for STEM-interested freshmen, demystifies scientific thinking through analysis of scientific literature. CBE Life Sci Educ. 12(1): 59-72.
9. Dolan EL, Collins JP. 2015. We must teach more effectively: Here are four ways to get started. Mol Biol Cell. 26(12): E13-11-0675.
10. NRC (National Research Council). 2000. How people learn: Brain, mind, experience, and school: Expanded ed., Washington, DC: National Academies Press.
11. Singer S, Smith KA. 2013. Discipline-based education research: understanding and improving learning in undergraduate science and engineering. J Eng Educ. 102(4): 468-471.
12. Gillen CM. 2006. Criticism and interpretation: teaching the persuasive aspects of research articles. CBE Life Sci Educ. 5(1): 34-38.
13. Round JE, Campbell AM. 2013. Figure facts: Encouraging undergraduates to take a data-centered approach to reading primary literature. CBE Life Sci Educ 12(1): 39-46.
14. Smith GR. 2001. Guided literature explorations: Introducing students to the primary literature. J Coll Sci Teach. 30(7): 465-469.
15. Colthorpe K, Chen X, Zimbardi K. 2014. Peer feedback enhances a 'journal club' for undergraduate science students that develops oral communication and critical evaluation skills. J Learn Des. 7(2): 105-119.
16. Glazer FC. 2000. Journal clubs--a successful vehicle to science literacy. J Coll Sci Teach. 29(5): 320-324.
17. Edwards R, White M, Gray J, Fischbacher C. 2001. Use of a journal club and letter-writing exercise to teach critical appraisal to medical undergraduates. Med Educ. 35(7): 691-694.
18. Martin DC, Arendale DA. 1992. Supplemental instruction: Improving first-year student success in high-risk courses (2(nd) edition, Monograph series No. 7). Columbia, SC: University of South Carolina, National Resource Center for the Freshman Year Experience.
19. Hurley M, Jacobs G, Gilbert M. 2006. The basic SI model. New Dir Teach Learn. 106: 11-22.
20. Gafney L, Varma-Nelson P. 2008. Peer-led team learning: evaluation, dissemination, and institutionalization of a college level initiative. In Innovations in Science Education and Technology Vol. 16, edited by Cohen KC. New York: Springer.
21. Woodward A, Weiner M, Gosser D. 1993. Problem solving workshops in general chemistry. J Chem Educ. 70(8): 651-652.
22. Varma-Nelson P. 2004. The Peer-Led Team Learning workshop model. What Works, What Matters, What Lasts, 4. Retrieved from http://cpltl.iupui.edu/media/3c7af56f-5645-4fc6-b062-0a8d751ff1f3/KwD7nw/cPLTLContent/2013/PLTL%20Literature%20PDFs/Varma-Nelson_2004.pdf
23. Cracolice MS, Deming JC. 2001. Peer-led team learning. Sci Teach. 68(1): 20-24.
24. Quitadamo IJ, Brahler CJ, Crouch GJ. 2009. Peer-led team learning: A prospective method for increasing critical thinking in undergraduate science courses. Sci Educ. 18(1): 29-39.
25. Springer L, Stanne ME, Donovan SS. 1999. Effects of small-group learning on undergraduates in science, mathematics, engineering, and technology: a meta-analysis. Rev Educ Res. 69(1): 21-51.
26. Boyd PJ. 1996. Mentoring by college faculty: Perception by students in evaluation of their satisfaction with college. PhD diss., University of Southern Mississippi.
27. Fogg RJ. 2002. The mentor-protege relationship in an academic setting: Contributions and outcomes. PhD diss., Kansas State University.
28. Beltman S, Schaeben M. 2012. "Institution-wide peer mentoring: Benefits for mentors." Int J FYHE. 3(2): 33-44.
29. Dutton, C. 2003. Mentoring: The contextualization of learning- mentor, prot?g? and organizational gain in higher education. Educ Train. 45(1): 22-29.
30. Shrestha C, May S, Edirisingha P, Burke L, Linsey T. 2009. From face-to-face to e-mentoring: Does the "e" add any value for mentors? Int J Teach Learn high Educ. 20(2): 116-124.
31. Terrion JL, Philion R, Leonard D. 2007. An evaluation of a university peer-mentoring training programme. Int J Evi Coach Ment. 5(1): 42- 57.
32. Budge S. 2006. Peer mentoring in post-secondary education: implications for research and practice. J Coll Read Learn. 37(1): 73-87.
33. Colvin JW, Ashman M. 2010. Roles, risks and benefits of peer mentoring relationships in higher education. Ment Tut: Part Learn. 18(2): 121-134.
34. Dennison, S. 2010. Peer mentoring: Untapped potential. J Nurs Educ. 49(6): 340-342.
35. Vaidya SR. 1994. Improving teaching and learning through peer coaching. Educ. 115(2): 241-246.
36. Johnson DW, Johnson F. 2009. Joining together: Group theory and group skills (10th ed.). Boston: Allyn & Bacon.
37. AAAS (American Association for the Advancement of Science). 2011. Vision and Change in Undergraduate Biology Education: A Call to Action, Washington, DC.
38. UOT (University of Toronto). 2014. Teaching and Learning Symposium Keynote. Retrieved from: http://www.president.utoronto.ca/speeches/teaching-and-learning-symposium-keynote000