Abstract

A successful scientist combines skill, creativity, and the art of persuasion to excel in his or her field of study. Critically important for the professional scientist is the ability to not only conduct research, but also to conceive a project and obtain funding to support research goals. As students grow in their scientific process skills over the course of their training, they are often surprised by the structures and processes used by professional scientists to procure funds for their work. We describe a semester-long experience in which students engage the process of science to design an innovative research plan on a topic that is relevant to the scientific community and society. Research teams seek funding to pursue a novel, high-impact research question, and submit proposals for peer review in a mock NSF-style study section. The module design uses a scaffolded series of writing and peer review activities, and culminates in a Pecha Kucha event in which students orally pitch and defend their proposal and vote for the best in session. The module may be scaled and adapted to suit a wide range of contexts where proposal writing and peer review is emphasized.

Citation

Society Learning Goals

Science Process Skills

• Process of Science
• Locate, interpret, and evaluate scientific information and primary literature
• Pose testable questions and hypotheses to address gaps in knowledge
• Plan, evaluate, and implement scientific investigations
• Interpret, evaluate, and draw conclusions from data
• Construct explanations and make evidence-based arguments about the natural world
• Address novel questions through authentic research experiences
• Modeling/ Developing and Using Models
• Recognize the important roles that scientific models, of many different types (conceptual, mathematical, physical, etc.), play in predicting and communicating biological phenomena
• Make inferences and solve problems using models and simulations
• Quantitative Reasoning/ Using Mathematics and Computational Thinking
• Apply the tools of graphing, statistics, and data science to analyze biological data

Lesson Learning Goals

Lesson Learning Objectives

• Propose a testable, novel question contributing to a biological field of study.
• Formulate a study rationale.
• Describe relevant background information on a topic using the primary literature.
• Choose appropriate scientific, mathematical, and statistical methods to analyze a research question.
• Determine the financial costs of a research project.
• Present a proposal for peer review and compose a constructive peer review.
• Collaborate as a member of a scientific team.
• Articulate the review criteria and process used in NSF-style proposal review.

INTRODUCTION

Exposure to undergraduate research experiences is vital to helping students understand and apply the process of science. However, students are often daunted by the prospect of conceiving an original research plan, and have little understanding for how research projects are funded, vetted, and prioritized by funding agencies. While undergraduate research experiences have grown in popularity and implementation styles (1), students typically are first exposed to the task of writing a full-blown research proposal in graduate school. Regardless of whether or not they pursue research careers, they frequently have little exposure as citizens to how their tax dollars are utilized to advance discoveries in science.

To address these challenges and help students appreciate the processes and institutions that support scientific research, we designed a course that mimicked the National Science Foundation (NSF) grant proposal and peer-review process. The target audience for this learning experience is biology majors with junior standing. Our goal was to reinforce basic scientific process skills (formulating hypotheses, experimental design, reading literature, scientific argumentation), while revealing the processes and infrastructure that support the rigors of our discipline (peer-review, funding agencies). Students work in teams to create a hypothetical research proposal on any topic, including a full budget. There is no budget limit for their ideas.

The learning experiences unfold over the course of a 15-week semester, with pre-class activities taking approximately 1-2 hours of time outside of class and in-class collaborative activities requiring 50 minutes of time. Students should have completed an introductory biology course sequence across scales of biological organization (cellular/molecular, organismic, ecology), which provides the background knowledge needed for students to develop a research topic related to these sub-disciplines. In addition, students should have exposure to scientific process skills through the laboratory curricula associated with an introductory course sequence.

A variety of grant writing experiences have been employed in undergraduate curricula (2-7). For example, in an apprenticeship model, undergraduate teams were paired with a faculty mentor who guided their production of a theoretical proposal over the course of an academic year (2). In other examples, smaller-scale proposal-writing exercises were used to facilitate scientific process skills in lower-division courses and were particularly useful in courses that lack a laboratory component (3-4). The NSF-style mock study section and blind peer review process that we describe in this lesson has been previously used as a capstone experience in upper-level courses to train students in grant writing (5-7). Our semester-long grant-writing experience synthesizes several features of previous approaches (e.g., cooperative writing groups; a highly scaffolded course structure; blind mock NSF-style peer review) and adds unique elements such as Pecha Kucha-style oral defense and a "flipped" pedagogy. Our course can be readily adapted for other contexts. Alternatively, instructors could use individual course elements to suit various learning goals that use writing as a tool for development of scientific process skills.

SCIENTIFIC TEACHING THEMES

ACTIVE LEARNING

The course uses a flipped, cooperative learning pedagogy. Students individually engage in targeted, online, pre-class activities, ranging from textbook reading to brainstorming activities to reflective writing and peer review. The pre-class activities are designed to help students acquire the essential skills and background knowledge needed to effectively interact with their research team in class. Class time is devoted to working collaboratively in teams on the research proposal, presentations, and peer review.

ASSESSMENT

Pre-class assignments generally target low-level Blooms domains and are assessed using short response or multiple-choice questions that each student completes independently. Comprehensive rubrics are used to evaluate the proposal drafts, the final proposal, and the oral Pecha Kucha presentation. The peer review and mock study section report are also evaluated using a simple checklist and rubric. All rubrics are included as Supporting materials (Supporting Files 7, 10, 14).

INCLUSIVE TEACHING

We leverage the unique background and perspective of each student in the classroom to create diverse and complimentary student teams (8-10). Students discuss and collaboratively choose a research topic that is relevant to and captures the interest of all members of the team.

LESSON PLAN

The course spreads the writing and review process over a 16-week semester, roughly divided into six modules. The assessment structure of the learning experience is tiered such that the weighting for each draft increases over time (Figure 1, Supporting File S1). The pre-class assignments (done individually by each student) generally take about one hour of student time outside of class, and collaborative in-class activities are designed for a 50-minute class session that meets once per week. Students spend additional time outside of class gathering in groups or collaborating electronically to pull together the proposal drafts and oral presentation. The role of the instructor is one of coach: helping students to make decisions, reason through the experimental design, and provide formative feedback on drafts. Table 1 illustrates the weekly sequencing of activities over the course of the semester.

MODULE 1 (WEEKS 1 - 2): CHOOSING A RESEARCH QUESTION (WHAT BIOLOGICAL QUESTION AM I INTERESTED IN? IS THIS QUESTION RELEVANT AND TESTABLE?)

The major outcome for Module 1 is for students to choose a research question that is real, relevant, and novel. Students will articulate the research question, variables to be tested, and the hypothesis.

WEEK 1 - CLASSROOM COMMUNITY AND COOPERATIVE RESEARCH GROUPS

PRE-CLASS ACTIVITIES

Students are asked to review the course syllabus (Supporting File S1) and understand the course goals and expectations before and during class. In addition, they must post responses to comprehension questions online prior to class (see Supporting File S2, which contains the pre-class assignments for the entire course).

IN-CLASS ACTIVITIES

Prior to the first class session, the instructor will establish working teams of four students. Since we are familiar with the students from teaching 100-level courses, we use our knowledge of the student to facilitate the process of building effective teams. We usually strive to balance gender, GPA, learning styles, and personality. However, the criteria for forming teams may be modified using a variety of alternative methods (8-10). Since our classes enroll 16-20 students, each instructor manages between four and five teams per semester, a "load" that may be modified based on enrollment size and other considerations, such as teaching assistant support.

At the beginning of the first class, students engage in a "Class Bingo" activity designed to help them build classroom community, since the course depends strongly on trust and rapport of classmates (7-10 minutes; see Supporting File S3). After this icebreaker, students spend time reflecting individually for 3 minutes about the attributes of a successful team. Students are placed into their cooperative learning teams of up to four students, and together the team engages in an exercise designed to encapsulate the theme of the course: to sell your work to an outside audience. The team spends approximately ten minutes in the "Tinker Team" activity (see Supporting File S3), wherein they are given a set of magnetic building parts and are asked to build the "best" structure they can in 5 minutes (8,9). When they are finished, a team spokesperson must try to convince the rest of the class that his or her team's structure is the best, practicing the art of persuasion. Together, we reflect on the experience to jointly construct the most common "mistakes" teams make and to compose a series of attributes for excellent teamwork. Students then construct a team contract that describes how they plan to operate, including expectations about attendance, preparation, division of the work, decision-making processes, and what the group plans to do about leadership and other roles. Students must also discuss how disagreements will be handled, how group members will treat each other, and what actions the group will take if the terms of the contract are violated. The contract serves as their first team writing assignment of the semester (Supporting File S3) and is submitted at the end of class.

WEEK 2 - ATTRIBUTES OF A RESEARCH PROPOSAL AND SELECTING A RESEARCH TOPIC

PRE-CLASS ACTIVITIES

Students view a 5-minute instructional video describing the elements of an effective research proposal and answer a series of comprehension questions. (See Supporting File S4 "Elements of a research proposal" for a scripted presentation file from which you can create a video.) Students are provided with an example research proposal (Supporting File S5) written by a student alumnus of the course and a professional proposal written by a scientist. They first compare and contrast the elements of a research proposal to a research paper, then look for similarities and differences between the student and professional proposals, especially focusing on target audience. The primary difference between these proposals is that the students are proposing a single experiment, whereas the professional proposal is a full research plan. A willing faculty colleague or instructor of the course should provide the sample professional proposal.

Students watch a second 5-minute instructional video (see Supporting File S6, "X-factor Exercise") that provides a framework for choosing a relevant and high-impact research topic. Students are not required to use the framework, but it is helpful to students who struggle with identifying a starting point. Briefly, students are instructed to go to a science news website, such as Science Daily and peruse articles that capture their personal interest. Then they are instructed to obtain the primary reference to a single study that interests them and read through the discussion section for the next research question that they would like to pursue on the topic. Each student then must come to class prepared to pitch their idea to the rest of the team. Students answer comprehension questions individually before class (see pre-class Supporting File S2 for question set).

IN-CLASS ACTIVITIES

Students come together in their teams and spend the class session discussing the ideas they individually collected for a research topic. Each student takes a turn sharing his or her idea pitch. By the conclusion of class, the team must agree on a single topic and attempt to identify a research question. The instructor visits with each team to facilitate the process of articulating a research question, particularly identifying the dependent and independent variables, and formulating a null hypothesis. Groups who are prepared to move forward can make progress toward the submission of a written research question and hypothesis (see Supporting File S7), their second group writing assignment for the course. Teams that require more assistance have the opportunity to brainstorm with the instructor. We use an active learning classroom and plenty of whiteboard space to diagram and process ideas in this session.

SUMMATIVE ASSESSMENT FOR MODULE 1

Students submit a summary of their research question and hypothesis (see Supporting File S7 for template and grading rubric). Each team submits only one document. Individually, students complete a Group Effort Analysis (GEA) form (see Supporting File S8) that enables them to reflect on their teams' work and their personal contributions in the first module. Both assignments are due four to five days after the class session, by 5pm, through an upload to the course management system.

MODULE 2 (WEEKS 3 - 5): SETTING UP THE BACKGROUND AND SIGNIFICANCE FOR THE PROJECT (HOW IS THE BIOLOGICAL QUESTION RELEVANT TO SCIENCE AND SOCIETY? WHY IS THE PROPOSAL WORTHY OF FUNDING?)

A major outcome for the second module is for students to appropriately use and document the scientific literature, and to set up the relevance and context for their research question.

WEEK 3 - USING AND DOCUMENTING THE SCIENTIFIC LITERATURE

PRE-CLASS ACTIVITIES

Students watch a 5-minute instructional video offering background information, tips, and advice for conducting a literature review and constructing the background and significance portion of a research proposal (see Supporting File S9 "Tips for writing the B&S" for scripted presentation). Students are provided with links to library resources and services through our course management software. They also do a short reading from McMillan (11) on CBE/LSE citation formatting and the differences between primary, secondary, tertiary sources. Students take an online quiz (see pre-class Supporting File S2) to gauge their learning. We use the results of the quiz to determine what elements to focus on during the class session, i.e. just-in-time teaching (12).

IN-CLASS ACTIVITIES

Depending on student questions and the results of their pre-class activities, class time is spent on reviewing citation style and/or accessing the primary literature. We provide individualized support to students and/or teams with unique questions. Teams that are ready to move forward make progress on generating a collaborative outline of the Background and Significance (Introduction) of their proposals, making joint decisions about the content needed for their target audience to understand and appreciate their proposed experiment. Students also make decisions about how they will divide and conquer the workload for this section of writing. At the end of the class session, teams upload an outline (it can be very rough) of their background and significance section to the course management system. The instructor provides feedback prior to the next class session.

WEEK 4 - USING AND DOCUMENTING THE SCIENTIFIC LITERATURE

PRE-CLASS ACTIVITIES

All members of the team work on their assigned portion of the background and significance outline, finding papers in the primary literature that will flesh out the details for their assigned section. Each student comes to class with an annotated bibliography of three to five references to share with the team (see pre-class Supporting File S2) that is also uploaded to the course management system before class.

IN-CLASS ACTIVITIES

Class time is spent on team discussion of what each individual learned from reading the primary literature. Students pool their bibliographies and submit a comprehensive annotated bibliography (minimum of ten to fifteen references) either by the end of the class session, or up to three days later as determined by the instructor and the pace of the student groups. In addition, they refine and revise their outlines, incorporating new knowledge from literature readings and instructor feedback. We suggest using Google Docs as the platform for all collaborative writing and instructor review.

WEEK 5 - BACKGROUND AND SIGNIFICANCE DRAFTS

PRE-CLASS ACTIVITIES

Individual students prepare a draft of their portion of the background and significance and submit it to the course management system prior to coming to class.

IN-CLASS ACTIVITIES

Students work in teams to synthesize the individual contributions of each member into a comprehensive draft that follows their outline. Teams are provided with the checklist and rubric (see Supporting File S10) that will be used to evaluate this section of their proposal and are given tips about how to use it (e.g., use it while re-writing; use it in the final edits). Teams are encouraged to choose one student who will take on the final responsibility of smoothing over the writing and completing a final round of editing before submitting the draft to the course management system. Since teams will not have sufficient time during class to finish the background and significance section, we negotiate the submission deadline to be up to five days later.

SUMMATIVE ASSESSMENT FOR MODULE 2

Students submit a draft of their Background and Significance (i.e. Introduction) for instructor review. This document is uploaded (one per group) to the course management system. See Supporting File S10 for the Introduction checklist and grading rubric. Note that this rubric is partitioned during the draft-writing phase to include only the relevant sections. Students also complete a second GEA reflecting on their own contributions as well as their team members' contributions to the background and significance section.

MODULE 3 (WEEKS 6 - 8): DESIGNING THE SCIENTIFIC, MATHEMATICAL, AND STATISTICAL METHODS FOR THE PROJECT (HOW WILL WE COLLECT, ORGANIZE, AND ANALYZE THE DATA? IS THIS RESEARCH PLAN REALISTIC AND FEASIBLE?)

WEEK 6 - HOW TO COLLECT AND ANALYZE DATA FOR YOUR PROJECT

PRE-CLASS ACTIVITIES 

Students individually reflect on their central question and hypothesis and begin thinking about how they will design a controlled experiment to test their hypothesis (see pre-class Supporting File S2). Students answer questions about their research question, their null and alternative hypotheses, controls, methods/experimental design, visual displays of data and statistical tests.

IN-CLASS ACTIVITIES 

Students spend the class session designing their experiment. This work includes creating a flow chart of the experimental design and making decisions about sampling, statistical testing, control and experimental variables. The instructor's role is to provide consultation and feedback that is individualized for a team's needs. The instructor also uses this time to help students who are struggling with control variables, hypothesis testing, and choosing a statistical test. To assist students with selecting tests, they refer to a decision tree (11). Students submit their flowchart to the instructor at the end of the session for review. The team also assigns roles/parts to team members for writing the experimental design section.

WEEK 7 AND 8 - WORK ON EXPERIMENTAL DESIGN

PRE-CLASS ACTIVITIES (WEEK 7)

Students work collaboratively and independently outside of class on their assigned portion of the experimental design section (see pre-class Supporting File S2). Before coming to class this week, each student constructs a one-paragraph reflection on his/her progress on writing in this section. Students also identify what work they must do to complete their portion of the experimental design writing.

IN-CLASS ACTIVITIES (WEEK 7)

Class time is dedicated as a team work period, with the instructor available to provide individualized feedback to teams. Teams receive a copy of the checklist and rubric for the experimental design section (see Supporting File S8). We request that students submit an ungraded work-in-progress draft at the conclusion of class in week 7 to hold them accountable for the writing tasks. In addition, we request that each student in the team defines and documents the individual action-steps to be completed before week 8 (see pre-class Supporting File S2).

PRE-CLASS ACTIVITIES (WEEK 8)

Students work collaboratively and independently outside of class on their experimental design section. Before coming to class this week, students use the individual action-steps submitted at the end of week 7 for a self-assessment. Students identify what they have accomplished and what remains to be completed.

IN-CLASS ACTIVITIES (WEEK 8)

Class time is again dedicated as a work period for teams, with the instructor visiting teams and providing individualized feedback as needed.

SUMMATIVE ASSESSMENT FOR MODULE 3

Each team submits a draft of the experimental design portion of their paper (Supporting File S10). In addition, all team members individually reflect on the quality of their teamwork during Module 3 (see Supporting File S8 for Group Effort Analysis).

MODULE 4 (WEEKS 9 - 10): BRINGING THE PIECES TOGETHER: ASSEMBLING THE FULL PROPOSAL DRAFT (HOW MUCH WILL THIS RESEARCH PLAN COST?)

WEEK 9 - BUDGETING THE COST

PRE-CLASS ACTIVITIES

Using their experimental design section, each student generates an itemized list of items they will need to place in their budget. Each student submits this list via the course management system before class and brings a copy of it with them to class.

IN-CLASS ACTIVITIES 

Teams generate one comprehensive itemized budget list of equipment and supplies. Then, students spend time researching the cost of their proposed research activities. The budget is not limited because we want students to get a real sense for the cost of scientific research. If they are doing field-based projects, it might be helpful to remind them that transportation, housing and food should be part of their budget. In addition, students forget to write in their "summer salary," so you may also wish to remind them about this budget item. By the end of class, the team submits a one or two sentence "Budget Update," describing what they have accomplished so far and what they need to do before next week.

WEEK 10 - WORK ON FULL PROPOSAL DRAFT

PRE-CLASS ACTIVITIES 

Each team member reviews the rubric/submission criteria and independently applies it to the team's proposal. They reflect on how well their drafts match the rubric and NSF's Intellectual Merit and Broader Impacts statements in a written paragraph (see pre-class Supporting File S2).

IN-CLASS ACTIVITIES 

Students use class time as a work period to incorporate all suggestions from instructor review and perform a self-assessment of their full proposal using the rubric (Supporting File S10). Students submit an annotated rubric highlighting what they perceive as the strengths and weaknesses of their proposal. Instructors use this class session to answer questions about the feedback and to help teams make progress on their revisions.

SUMMATIVE ASSESSMENT FOR MODULE 4

Each team submits a full draft of their proposal for instructor review (see Supporting File S10 for rubric) and assesses their teamwork using the GEA (Supporting File S8).

MODULE 5 (WEEK 11-12): MOCK NSF STUDY SECTION (WILL THIS RESEARCH PLAN BE FUNDED?)

WEEK 11 - GIVING AN EFFECTIVE PEER REVIEW

PRE-CLASS ACTIVITIES 

Students view a short video that explains NSF's Merit Review process (http://www.nsf.gov/news/mmg/mmg_disp.jsp?med_id=76467) so they have an idea of what kind of peer review happens to research proposals. Students respond to three reflection questions following the video. Then, students reflect on a time in their lives when they particularly benefited from the feedback of others. Students post these thoughts to an online forum.

IN-CLASS ACTIVITIES 

Working in their teams, students review the forum posts about effective feedback. Then they contribute to a class Google Doc (see Supporting File S11) that identifies characteristics of effective feedback. The second part of the in-class activity uses the same Google Doc, but each team is assigned examples of statements that are very typical from actual peer reviewers (e.g., "this is disorganized, more sources are needed, etc.). Students evaluate the statements against the criteria for effective peer review the class generated earlier. Then, they improve the statements based on the criteria for effective feedback generated at the beginning of class. Each team revises two to three feedback statements.

Alternatively, this class session could take place through a mini-lecture and active learning activity (see Supporting File S12 "Speaking Truth Effectively"). After a short lecture on peer review, students define the attributes of an effective peer review, analyze the effectiveness of example reviewer comments, and then apply the principles of giving effective feedback through a series of questions embedded in the presentation.

At this point, teams are encouraged to prepare a revised copy of their proposal for peer review in the mock study section. After class, the instructor de-identifies the proposals and uploads them to the course management system. Teams receive instructions about which proposal to review and may begin the peer review. (All members of the team will independently evaluate the same proposal.) If two sections of the course are being taught simultaneously, arranging a peer review across sections is preferred.

WEEK 12 - GIVING AN EFFECTIVE PEER REVIEW

PRE-CLASS ACTIVITY 

Students individually read the assigned proposal and submit online a 1-paragraph summary of the proposal, along with two strengths and two areas for improvement (see pre-class Supporting File S2).

IN-CLASS ACTIVITY

Teams break out into separate rooms and discuss the assigned proposal as a committee. To guide their discussion, they use a template for "Meeting Minutes" which includes a series of prompts. The review is itself evaluated based on the student's ability to support their claims and to apply the principles from week 11 (see Supporting File S13).

SUMMATIVE ASSESSMENT FOR MODULE 5

Students submit the "Meeting Minutes" for instructor review. The instructor provides commentary on the peer feedback, especially to affirm useful suggestions, prior to returning the peer review to the authors. The committee summary is de-identified prior to returning the proposal to the authors. The instructor then serves as the program officer, providing a cover letter to the authors indicating whether their proposal was recommended for funding by the committee. The committee review and cover letter are returned to the students. Students are told at the start of this project that the NSF has funds for only one proposal. Thus, the program officer (instructor) approves just one proposal for funding, making the final decision after consideration of the peer reviews. The authors then revise and resubmit their proposal, incorporating peer and instructor suggestions, for a final grade in Week 16.

MODULE 6 (WEEKS 13 - 15): ORAL DEFENSE (CAN I PERSUADE OTHERS?)

WEEK 13 - 14 - DEVELOPING THE ORAL PRESENTATION

PRE-CLASS (WEEK 13)

Students visit the Pecha Kucha website (http://www.pechakucha.org/) and review the Pecha Kucha style presentation by reading the "About" tab and watching at least one presentation (see pre-class Supporting File S2). Students also familiarize themselves with the presentation rubric and reflect on a few questions about oral presentation defenses.

IN-CLASS ACTIVITIES (WEEK 13)

Students spend time preparing a six to seven minute oral presentation that hooks the audience and sells their proposal. Students are required to use minimal text and mostly visuals in their presentation, using Pecha Kucha style (13), a six-minute presentation using 20 slides and 20 seconds per slide. Students work from a checklist and rubric to meet the essential requirements of the presentation (Supporting File S14).

PRE-CLASS ACTIVITIES (WEEK 14)

There are no pre-class activities this week, as students are busy preparing their presentations.

IN-CLASS ACTIVITIES (WEEK 14)

Student teams continue their presentation development and get feedback from the instructor as needed.

WEEK 15- PECHA KUCHA EVENT

PRE-CLASS ACTIVITIES

There are no pre-class activities this week, as students are busy preparing their presentations.

IN-CLASS ACTIVITIES 

Students unveil their work at a fun Pecha Kucha event, which includes snacks and (root) beer. At the conclusion of the event, students vote for a best-in-session pitch. Students score each presentation using the rubric (Supporting File S14) to justify their conclusions.

FINALS WEEK

Teams upload the final version of their proposals, as well as a final GEA form, both due at the final exam timeslot for the course.

TEACHING DISCUSSION

STUDENT AND INSTRUCTOR REACTIONS

One of the most rewarding aspects of teaching this semester-long experience is to witness the progression in the quality of the proposals from conception to final product. The structure of the experience incorporates a high level of both peer and instructor support to help students produce a high quality product by semester end. When we ask students to reflect on the learning objectives in which they progressed the most, they cite all of the objectives fairly equally. Perhaps this result reflects the value of the flipped, just-in-time course structure, which allows for more individualized feedback for certain parts of the process of science as students need it. Overwhelmingly, students comment that a collaborative and iterative writing experience helped them gain skills in working as a member of a team, especially in the area of communication. Even students who prefer to work alone found value in sharing responsibilities and ideas with others. In addition, students also commented that their final product was better than what they could have individually created. Students find that the overall course structure and the iterative drafts that allow for multiple rounds of feedback were effective in helping them manage this large project.

SUGGESTIONS FOR IMPLEMENTATION

We have found that the greatest investment of instructor time is in the experimental design portion of the course. Once students are grounded in how they will reach their proposed objectives, they can build the rest of the proposal autonomously. The section drafts are constructed with a high level of instructor support and feedback during the first three modules. The amount of feedback from the instructor tapers toward the end of the semester as students rely on peer review for improvements. At this point in the semester, we generally look for how well the teams incorporate suggestions. It is also important to ensure that students receive timely feedback to help them develop and sustain flow in their work.

We offer this experience as part of a 15-week, one-credit, 300-level Introduction to Research course at our institution. All of the activities we describe here were designed to use about two hours of student time outside of class each week. However, this approach can be easily adapted for smaller proposals or similar peer review activities could be embedded in other biology courses. For example, a "scale-down" version of this experience can be implemented over 6 weeks by using the most salient activities and assignments from each of the six modules (see Table 2). To simulate the mock study section in this scaled-down experience, we recommend converting the "Speaking truth effectively" mini-lecture into a screencast and providing it as a pre-class assignment. Then, students can use class time to incorporate the tips in their peer review. In addition, we recommend combining the Pecha Kucha event with the final proposal submission at the conclusion of week 6.

STUDENT ACCOUNTABILITY

To hold students accountable for individual and team contributions, it is important to make a concerted effort to establish the expectations for classroom community in the first week of the course. Without this investment in communicating expectations and consequences, we have found that individual accountability is more difficult to achieve. The team contracts composed in week one can be used as a framework whenever it is necessary for the instructor to intervene and assist teams in their functioning. Often reminding students of their commitments is sufficient for correcting challenges that arise.

To foster the process of evaluating and revising individual contributions and team dynamics, we use frequent group effort analyses and reflections at the conclusion of each module (see Supporting File S8). We find that requiring this frequent evaluation allows students and groups to self-correct and make modifications as needed. However, in cases where an individual student's contribution was detrimental to the group, his/her grade will be adjusted as a percentage of the contributions reported on the group effort analyses (see Supporting File S1 for syllabus). In our experience, the students in our courses view this strategy as fair.

Finally, to encourage accountability for incorporating feedback from both instructors and peers, we award points in the grading rubric for making the suggested changes to sectional drafts (see Supporting File S10). We have seen great improvements in student drafts since adding this accountability element to the rubric, compared to iterations of the course where this was not used.

SUPPORTING MATERIALS

• S1. Mock Grant-BIO399 course syllabus
• S2. Mock Grant-Pre-class assignments  
• S3. Mock Grant-Classroom community lesson plan  
• S4. Mock Grant-Elements of a research proposal presentation  
• S5. Mock Grant-Sample student proposal from BIO399  
• S6. Mock Grant-X-factor: selecting a novel and relevant research topic presentation  
• S7. Mock Grant-Research question and hypothesis assignment with rubric  
• S8. Mock Grant-Group Effort Analysis  
• S9. Mock Grant-Tips for writing the background and significance presentation  
• S10. Mock Grant-Comprehensive Rubric  
• S11. Mock Grant-Sample google doc template for characteristics of effective feedback  
• S12. Mock Grant-Speaking Truth Effectively presentation  
• S13. Mock Grant-Mock Study “Meeting Minutes” template  
• S14. Mock Grant-Pecha Kucha oral presentation rubric

ACKNOWLEDGEMENTS

We thank our colleagues in the Biology Department for their many contributions to BIO399 over the years. We are especially grateful to Teresa DeGolier for sharing content and question prompts included in Supporting File S4 and in the week three pre-class activities. We thank Tammy Long and Diane-Ebert May for inspiring the GEA forms.

References

1. Auchincloss LC, Laursen SL, Branchaw JL, Eagan K, Graham M, Hanauer D, Lawrie G, McLinn CM, Pelaez N, Rowland S, Towns M, Trautmann NM, Varma-Nelson P, Weston TJ, Dolan EL. 2014. Assessment of course-based undergraduate research experiences: A meeting report. CBE-Life Sci Educ. 13(1):29-40.
2. Stanford JS and Duwel LE. 2013. Engaging biology undergraduates in the scientific process through writing a theoretical research proposal. J Coll Biol Teach 39(2):17-24
3. Reynolds JA, Thaiss C, Katkin W and Thompson RJ. 2012. Writing-to-learn in undergraduate science education: a community-based, conceptually driven approach. CBE-Life Sci Educ 11:17-25.
4. Kover H, Wirt SE, Owens MT and Dosmann. 2014. "Thinking like a neuroscientist": Using scaffolded grant proposals to foster scientific thinking in a freshman neuroscience course. J Undergrad Neurosci Educ. 13(1):A29-A40.
5. Itagaki H. 2013. The use of mock NSF-type grant proposals and blind peer review as the capstone assignment in upper-level neurobiology and cell biology courses. J Undergrad Neurosci Educ. 12(1):A75-84.
6. Blair BG, Cline GR, Bowen WR. 2007. NSF-style peer review for teaching undergraduate grant-writing. Am Biol Teach. 69(1):34-37.
7. Tang BL, Gan YH. 2005. Preparing the senior or graduating student for graduate research. Biochem Mol Bio Educ. 33(4): 277-280.
8. Smith, KA 2003. Teamwork and Project Management. New York: McGraw-Hill
9. Detter-Schmelz DR, Kennedy, KN, and Ramsey, RP. 2002. Enriching our understanding of student team effectiveness. J Market Educ. 24(2):114-124. 10. McKendall M. 2000. Teaching groups to become teams. J Educ Bus. May/June, 277-282
10. McKendall M. 2000. Teaching groups to become teams. J Educ Bus. May/June, 277-282
11. McMillan, VE Writing Papers in the Biological Sciences. Bedford/St. Martin’s Press
12. Marrs KA and Novak G. 2004. Just-in-Time Teaching in Biology: Creating an active learner classroom using the internet. Cell Biol Educ 3:49-61.
13. Klentzin JC, Paladino EB, Johnston B, Devine C. 2010. Pecha Kucha: using “lightning talk” in university instruction. Ref Serv Rev. 38(1):158-167