Abstract

Biology students at the undergraduate level usually excel in knowing biological facts; however, they often struggle with connecting these facts to specific biological principles. In parallel, undergraduate students often struggle to read primary scientific literature (PSL), possibly for the same reason: they struggle to integrate the biological facts they know into the larger, and often complicated, biological principles presented in PSL. Our lesson bridges the gap between student understanding of content knowledge and their ability to connect this knowledge to larger biological principles through the integration of PSL and the 5 Core Concepts of Biology (5CCs) identified in the Vision and Change report. We begin by introducing students to PSL using a modified C.R.E.A.T.E. method and continue by walking students through Vision and Change as a way to introduce the 5CCs. Through the use of a matrix table detailing each one of the 5CCs and their related organizational levels, students learn how to integrate PSL and the 5CCs by connecting biological facts contained within PSL to a related biological core concept. Because students have to provide reasoning for why they connected a biological fact to a specific core concept, they begin to see biology as a larger entity, i.e., they begin to see the "big picture" of biology. Our lesson provides a novel strategy for introducing students to PSL.

Citation

Chatzikyriakidou K, Manrique C, Janelle Tacloban M, McCartney M. 2021. Exploring primary scientific literature through the lens of the 5 Core Concepts of Biology. CourseSource. https://doi.org/10.24918/cs.2021.5

Lesson Learning Goals

Students should be able to interact differently with PSL after learning how to use the 5CCs matrix table.

Lesson Learning Objectives

Students should be able to find examples of the 5CCs within the text of a piece of PSL.

INTRODUCTION

A successful science education includes teaching students science process skills such as critical analysis of primary scientific literature (PSL) (1,2). This is a fundamental skill for practicing scientists; however most college science departments lack a formalized curriculum for teaching undergraduates how to read, interpret, and discuss PSL (3-5). Instead, the use of PSL in the classroom remains limited due to several barriers, including students struggling with the actual practice of science (as opposed to the purely linear scientific method presented in textbooks), scientific jargon, and an inability to connect PSL to the broader context of the discipline (6-8). Additionally, most novice students are still developing the critical thinking skills needed to interpret the results and conclusions found in PSL. To complicate matters, some educators may be uncomfortable using PSL themselves, due to concerns about student pushback or their sense that the material is too complex for their students (9,10).

Despite these barriers, a growing body of literature shows that PSL is a valuable and useful tool for STEM education. For example, closely analyzing PSL in a classroom setting engages students in discussion and debate around interpretations of experimental data while building their insight into scientific practices (11). Teaching with scientific research papers has been shown to promote critical thinking, experimental design ability, and epistemological maturation as well as to improve students' positive attitudes about science and scientists in both two-year and four-year settings with first-year as well as upper-level students (12-17). PSL can also show students that even after a study is completed and published, there are still many unanswered questions for future scientists, promoting the development of creativity through study design assignments (3,11).

Published examples of PSL-based research show PSL being used by educators at all levels, highlighting the diversity, scalability, and flexibility of PSL as an educational tool. Programs include journal clubs, data and figure exploration, tutorials on how to read PSL, annotated literature, and full courses being taught only with primary literature (5,11,14,18,19). Assessment tools used to evaluate these programs are equally as diverse, ranging from rubric to validated survey (20,21).

We are interested in introducing Introductory Biology students to PSL. While undergraduate Biology students usually excel in knowing facts about biology, they struggle with connecting these facts to specific biological principles (4,22). We hypothesized that we would be able to integrate PSL and biological principles together, as a way to both introduce students to PSL and to encourage students to start thinking about biology less as a set of facts and more as a set of principles.

To do this, we developed an interactive classroom activity (summarized in Table 1) designed to introduce students to PSL using the Five Core Concepts of Biology (5CCs) described in Vision and Change (1,23). The 5CCs provide a detailed and content-based description of the knowledge of biology, summarized in five core biology concepts that dictate natural biological phenomena or processes (evolution; structure and function; information flow, exchange, and storage; pathways and transformations of energy and matter; and systems) connecting to five main biological scales (molecular, cellular, organismal, populational, and ecological) (Table 2). Understanding that every biological process or phenomenon can be analyzed into five different perspectives, as suggested by the 5CCs, is fundamental for understanding that biological facts link to concepts and that concepts link to other biological concepts (1). Using the 5CCs, we developed a matrix table with five rows, each containing a core concept and three columns, each containing a biological scale (for simplicity, we only use three biological scale levels, merging molecular/cellular to one scale, using organismal as the second scale, and merging population/ecology for the third scale) (an example 5CCs matrix table is shown in Supporting File S1. Exploring Primary Scientific Literature – Blank 5CCs Matrix table). Students were asked to fill in the matrix table as they read a selected piece of PSL as a way to encourage them to begin to think of scientific facts contained within PSL as part of larger biological concepts.

Intended Audience

This activity was used with students enrolled in "Essentials of Biology," an introductory biology course centered on reading PSL. The course was designed to introduce introductory students to the Five Core Concepts of Biology, along with critical thinking, analysis, and experimental design, through reading PSL. Over the course of the semester, students read 14 pieces of PSL accompanied by class discussions, active learning modules, accompanying worksheets, and homework assignments designed to show students that PSL is the global language of scientists, and that being able to deconstruct PSL is a valuable skill to have as a future scientist. The 5CCs matrix chart described here (Supporting File S1. Exploring Primary Scientific Literature – Blank 5CCs Matrix table) was implemented 14 times within this course.

Three offerings of "Essentials of Biology" had class sizes of n=32 (Fall 2018), n=11 (Spring 2019), and n=25 (Fall 2019, described in this manuscript). Collectively, the student demographics for these three offerings were 63% female; 74% Hispanic/Latino, 14% Black or African American, 7% White, and 4% two or more races; 45% freshman, 23% sophomores, 16% juniors, and 10% seniors; and 63% biology majors. While we have not yet implemented the 5CCs matrix charts into upper-level lecture or lab courses, we have no reason to believe they couldn't be implemented with more advanced students.

We implemented the 5CCs matrix charts at Florida International University, a public Carnegie R1-ranked urban university and a Hispanic Serving Institution, enrolling 41,794 undergraduates in Fall 2019, of which 67% were Hispanic/Latino, 12% were African American/Black, and 57% were women. The biology major had 4,159 students enrolled, composed mostly of Hispanic students and large numbers of First Generation (~20%) and Pell-eligible students (~51%).

Required Learning Time

The 5CCs matrix chart activity described here is designed to fit into a 50-minute class period. However, we implemented this activity several times over a semester long course, essentially using this 50 minute stand-alone activity repeatedly with each piece of PSL read in class (Supporting File S2. Exploring Primary Scientific Literature – example syllabus). While we used the 5CCs matrix chart 14 times over the course of the semester, we imagine different student populations and classroom environments would still benefit using this activity as a stand-alone or in a different variation of repetition. The amount of time needed to introduce students to both the 5CCs and PSL will vary, and there is more than one way for this to occur. We detail what has worked for us below.

Prerequisite Student Knowledge

This 5CCs matrix chart is intended for use at any point in the semester after students have received a basic introduction to the 5CCs. The 5CCs are described in detail in Vision and Change (Table 2) and in the BioCore Guide (1,23). In our experience, students should be able to define the 5CCs in their own words and understand the different scales of the 5CCs. We spend two class sessions on this prior to using the 5CCs matrix charts (Supporting File S2. Exploring Primary Scientific Literature – example syllabus, Supporting File S3. Exploring Primary Scientific Literature – Slides for introducing the 5CCs), however there is more than one way to introduce students to the 5CCs and instructors are encouraged to find a method that works for them. Frameworks, rubrics, assessments, and additional resources for implementing the 5CCs have been summarized in a recent essay which will be helpful for instructors less familiar with the 5CCs (24).

To introduce the 5CCs to our students, we start by explaining the Vision and Change meeting itself, describing how 500 biology educators sat in the same room for two days discussing what is truly essential in biology education. We ask them to envision just the students in the room distilling biology down to only 5 concepts, extrapolating to what it must have been like to go through this with 500 biology experts. Anecdotally, this discussion instills in the students a sense that what they are about to learn was debated and decided upon democratically by experts, and therefore it must be very important.

Instead of simply showing the students a list of the 5CCs, we provide an image representing each of the 5CCs and encourage a class discussion on what it could be (Supporting File S3. Exploring Primary Scientific Literature – Slides for introducing the 5CCs). Again, anecdotally, we have found that the students become more invested in learning about the 5CCs when they are allowed to "discover" them, rather than simply being told what they are.

Once each of the 5CCs has been identified, we lead a discussion around additional examples for each of the 5CCs that the students come up with by themselves. We place a strong emphasis here on how there can be more than one "correct" answer, which will come up in all future think-pair-share and group discussions relating to the matrix charts.

Students should also be comfortable reading a piece of PSL. In our experience, introductory students have a general idea of what PSL is, and some have prior experience with reading PSL, however they do not reflect positively on these experiences. We spend two class periods discussing PSL in general (Supporting File S2. Exploring Primary Scientific Literature – example syllabus). We bring old copies of Science and pass them out to students, each student with their own copy. Together, we flip through each page of Science and have a general discussion on the goals of each section, the audience for each section, and who develops content for each section (example discussion prompts include: What is the purpose of this section? Who writes this section? Are they a journalist or a scientist? What is the audience for this section? Do you find this section interesting? Do you find this section easy to read? Do you trust the person who wrote this?). These discussions are different with each group of students and therefore we are unable to provide detailed instructions on how to replicate this. In our experience, very few students have ever sat and looked through a scientific journal page by page, and this exercise provides an overview of PSL without actually reading any PSL, ultimately helping students to be more open to reading PSL.

Next, we use a modified version of C.R.E.A.T.E. (Consider, Read, Elucidate hypotheses, Analyze and interpret data, Think of the next Experiment) to introduce students to PSL (11,12; https://teachcreate.org/). C.R.E.A.T.E. is an evidence-based strategy that can be tailored to individual instructors and student populations. Instructors wanting to implement C.R.E.A.T.E. are encouraged to visit the "literature section" of the C.R.E.A.T.E. website to learn more about the research behind the C.R.E.A.T.E method (https://teachcreate.org/literature/). Examples of how other instructors have implemented C.R.E.A.T.E. can be found in the searchable database of "road maps" available (https://teachcreate.org/roadmaps/). Our modified implementation consists of using several pieces of stand-alone PSL instead of the four connected papers suggested by the C.R.E.A.T.E. method. Additionally, we limit our C.R.E.A.T.E. activities to cartooning the abstract, re-writing the title, defining unfamiliar words, and identifying the hypothesis being tested or question being addressed (12). While this adapted C.R.E.A.T.E method works for our course and our student population, it is possible for instructors to introduce students to PSL using other methods.

We use "Empathy and Pro-Social Behavior in Rats" as our introduction to reading PSL and work through the C.R.E.A.T.E. exercises of breaking down the title, cartooning the abstract, and predicting the next experiment (Table 3, Supporting File S2. Exploring Primary Scientific Literature – example syllabus, Supporting File S4. Exploring Primary Scientific Literature – Slides for introducing PSL). "Empathy and Pro-Social Behavior in Rats" does not contain large amounts of jargon and the experimental design is beautifully simple, allowing for students to really get into how and why the authors designed these experiments and the larger biological concepts involved. Depending on the response of the students, we work through 2-3 more pieces of PSL using modified C.R.E.A.T.E. (Table 3).

Prerequisite Teacher Knowledge

The instructor should have some familiarity with the 5CCs described in the Vision and Change report (1,23). For instructors wanting to learn more about the 5CCs, additional frameworks, rubrics, assessments, and other resources for implementing the 5CCs have been summarized in a recent essay (24). If an instructor is unfamiliar with the 5CCs we do not recommend implementing this activity.

The instructor should be comfortable reading PSL themselves and should be able to identify pieces of PSL that are at the appropriate level for their students. There is no easy method for selecting appropriate PSL for a given population of students. Finding appropriate PSL takes time and effort and, based on content to be covered and student knowledge levels, may need to be adapted each time the course is implemented. When considering a new piece of PSL, we first ask our Learning Assistants and undergraduate research assistants, who are closer to our student population with regard to content knowledge and vocabulary level, to read the PSL and give us their feedback. This has been a successful "first round" screening for us. We have listed the PSL we used in the fall semester of 2019, and given a brief reason why we used each piece, in Table 3. The majority of PSL used in our class is also available as annotated versions on the Science in the Classroom website (https://www.scienceintheclassroom.org/), a collection of annotated PSL that has been carefully selected to 1) be accessible to an introductory-level undergraduate and 2) be cutting edge, novel science. This resource is easily searchable and may be useful for instructors as they decide which PSL to select for their class. Additionally, annotated PSL can be used as an alternative method to C.R.E.A.T.E. for introducing PSL to students (19). If instructors are not comfortable reading PSL, or are unable to find PSL at the appropriate level for their students, we do not recommend implementing this activity.

SCIENTIFIC TEACHING THEMES

Active Learning

The activity includes a 5CCs matrix chart for students to complete on their own as they read a piece of PSL (Supporting File S1. Exploring Primary Scientific Literature – Blank 5CCs Matrix table). Recent research has shown that the addition of worksheets (5CCs matrix charts) as an active learning tool for in-class group activities promotes student collaboration and develops problem solving skills (25). We complement our matrix charts with think-pair-share (26) and whole class discussions. We start with think-pair-share as a way for students to "vet" their answers within a small group and to alleviate possible anxiety of speaking in front of the whole class. During this activity, we ask students to notice that their matrix chart might look completely different than their partners, yet both matrix charts can be correct. We ask students to share their reasoning for placing an answer in a certain box with their partners. While we have only anecdotal data, we do hear argumentation taking place during this activity as students explain their reasoning to their partner.

In whole class discussions, volunteers are encouraged to share their matrix charts with the whole class while the instructor facilitates moving the conversation from student to student. Because there is often more than one "correct" answer per matrix chart box, other students may support or disagree, generating an active discourse across the class. This is the same discussion that takes place during the one-on-one sharing (i.e., students are presenting their reason for putting an answer in a certain box to the rest of the class). More than one "correct" answer is often a new concept for our students, making instructor facilitation of these conversations a critical part of their success. Anecdotally, while we do not often have students disagreeing with the presented answer, we do have students acknowledging that their classmates' reasoning helped them to see either a 5CC or content contained within PSL in a new light.

The first implementation of this course took place in an active learning classroom where each student desk was on wheels and easily movable. In this setting, we were able to easily assign students to groups and have students move around the room, ensuring that they were often working with different partners. In the second implementation students sat in rows of tables, which made switching of groups difficult and students most often just worked with their neighbor. The third implementation used stationary desks that were somewhat moveable, allowing for more interaction among students than the rows of tables, but much less than the active learning classroom. While we did our best to regularly change student groups, we were not always successful. Most often, students started an activity alone, then paired with a partner for a first round of discussion, and then joined a larger group of four students for a second discussion.

Assessment

Assessment of student understanding comes from scoring the matrix tables. While we looked at matrix tables throughout the semester, by discussing them in class and having students work with a partner to compare/contrast their own answers, we only graded matrix tables completed as part of an exam. Our exams consisted of the full piece of PSL printed out and given to each student so they could write and take notes as they read. Students were given a printed exam containing a blank matrix chart (Supporting File S5. Exploring Primary Scientific Literature – example exam) and a printed copy of the piece of PSL (Table 3). We developed a rubric for each piece of PSL used for an exam, trained Learning Assistants and undergraduate research assistants on how to use the rubric, and worked as a team to grade matrix charts (Supporting File S6. Exploring Primary Scientific Literature – exam grading rubric). A sentence describing the connection between a piece of information from the given PSL and a core concept (connected to the correct biological scale), and the reasoning students used to get there, would typically receive full credit. Figure 1 shows four examples of completed matrix charts from an exam. We provide this figure as an example of 1) how students complete the 5CCs matrix chart and 2) and example of the variation among student answers for the same piece of PSL.

We provide student feedback in Table 4 to highlight student responses to our activity. Specifically, we show that our Learning Goal of "Students should be able to interact differently with PSL after learning how to use the 5CCs matrix table" has been achieved with the following student responses:

• "In future science courses, I will use the skill of breaking down the core concepts of biology...."
• "In all my future biology classes or careers, I think the thing I will remember most is to break down the article using the 5 cc's..."

We also see evidence that our hypothesis of being able to integrate PSL and biological principles together, as a way to both introduce students to PSL and to encourage students to start thinking about biology less as a set of facts and more of a set of principles, i.e., the big picture, is plausible, as shown in the following student responses:

• "Thinking about the bigger picture in the topics I learn."
• "Understanding the bigger picture of why biological experiments are being conducted."
• "...this class really helped with the understanding of biological concepts."

Additional student responses also suggest that students are becoming more comfortable reading PSL ("That there is no paper that I can NOT understand..."), are interacting differently with PSL after learning how to use the 5CCs matrix table ("...this class has opened my eyes to a more interactive and simpler way of braking down harsh topics..."), and are able to connect the 5CCs and PSL ("...I will use the skill of breaking down the core concepts of biology, and using it to see how it'll apply to specific research study..."). We have bolded, italicized, and underlined specific student examples showing this (Table 4).

Inclusive Teaching

Reading carefully selected PSL has the potential to engage many individuals, since questions about authentic scientific studies are typically very interesting to most students. This lesson seeks to create a learning environment where students' academic, social, and cultural backgrounds can be an asset to their learning. Because there can be a variety of responses in each matrix box, all student viewpoints and conclusions can be included in class discussions.

The majority of class discussions occurred in small groups, often with the same students, resulting in students becoming comfortable enough to share their answers freely. Additionally, because there was more than one "correct" answer for each matrix table, it became easier for students to share their answers without the fear of being wrong. However, for the larger group discussions, we asked for volunteers and did not force students to address the class if they did not volunteer.

LESSON PLAN

At this point, students should be comfortable with both the 5CCs and PSL. This activity, which is specifically using a 5CCs matrix chart to guide students reading a piece of PSL, is designed for a 50-minute class period after students have been introduced to the 5CCs and to reading a piece of PSL. In our course, we implement this 50- minute activity 14 times over one semester. Table 1 shows the teaching timeline for the activity.

To introduce the matrix charts to the students, show example matrix charts (one PowerPoint slide of 4 different matrix charts all filled in differently and all receiving full credit, similar to what is shown in Figure 1). Stress that students need to be able to explain why they chose a certain concept and organizational level and show a successful argument (e.g., for transformation of energy: energy from the sun allows for photosynthesis) versus a generic answer (e.g., energy moves through the system). During class discussions, also strive for this kind of detailed explanation: the student reasoning for why the answer goes in the box must be there. Discussion prompts for class discussion include: What is the structure? What is the function? What is the information? Where is the information stored? Where is the information flowing? How is the information flowing? What is the energy? What is the matter?

It is important to note that students are only required to fill in 3 boxes of the matrix chart. Do not specify that there must be one box per 5CC, or one box per organizational level; simply ask for 3 boxes in total. It is unrealistic for PSL to connect to each 5CC at three different levels, for a total of 15 connections. We settled on three as a compromise between 1) content contained in PSL, 2) student level of understanding of both 5CCs and PSL, and 3) time available for the activity. In our experience, students excel in filling in a wide array of boxes across the matrix chart; we rarely see three answers in the same concept or organizational level.

Start this activity with a paper you used in the introduction to PSL section (e.g., return to "Empathy and Pro-Social Behavior in Rats"), as students are already familiar with this piece of PSL, and they can just focus on learning how to connect the biological facts found within the paper to the larger biological concepts of the matrix table. Print out copies of the 5CCs matrix table to hand out to each student (each student has their own copy) so students are able to take notes and write answers directly onto the chart itself. Work through this first chart together, first with think-pair-share and then a larger group discussion. During both types of discussions, continuously stress that there is more than one "correct" answer, but students need to be able to explain why they put a certain fact from the PSL into a specific matrix box. Ask students "why that box? why that level?" whenever possible. Next, assign each small group another piece of PSL that they have already worked through using C.R.E.A.T.E. Task them with completing a 5CCs matrix chart that they will then present to the larger group. Continue in each subsequent class period to assign additional pieces of PSL and 5CCs matrix charts as homework (we work through one piece of PSL at a time) and continue to read additional PSL, with accompanying 5CCs matrix charts, together in class (Supporting File S2. Exploring Primary Scientific Literature – example syllabus). Generally, we recommend reading one piece of PSL per class; however, some pieces of PSL that you wish to spend more time on can be spread out over two class meetings. In our experience, students have given feedback on what papers they read through quickly and lose interest in (Table 3, PSL #2 and 4) and what papers catch their attention and keep them wanting more discussion (Table 3, PSL # 7, 10, and 11). We anticipate that any instructors implementing this activity will receive similar feedback from their own students.

Example Matrix Templates

Develop a rubric for each PSL (Supporting File S6. Exploring Primary Scientific Literature – exam grading rubric).Learning Assistants, Teaching Assistants, and Undergrad Research Assistants can be trained to help assess 5CCs matrix charts. It is helpful to assess the matrix charts together so questions can be asked about individual responses and any rubric clarifications can be addressed. Example matrix charts are shown in Figure 1 as example of how an average student in our course completes the chart.

TEACHING DISCUSSION

Reactions from Students

We collected qualitative data from students on their perceptions of the course as part of the final exam. Representative answers are shown in Table 4. In general, students indicate that they have made the connection between using the 5CCs to make PSL easier to understand and to see the "big picture" of biology. We are currently working on developing a more quantitative way to measure student gains using the 5CCs matrix charts.

Persistent Conceptual Difficulties

Some common issues we have seen across all three semesters include:

• Structure and function is traditionally the hardest one of the 5CCs for students to grasp. We have found that asking the students to explain "what is the structure and what is the function," and encouraging them to be as specific as they can, helps with student understanding.
• Evolution and reproduction are not the same thing. Students often confuse these concepts when arguing for placing content in the Evolution boxes.
• We receive a lot of "generic" reasoning for picking Pathways of Transformation of Energy and Matter and Information Flow, Exchange, and Storage. Our students struggle to identify the type of energy and matter, or the type of information, involved. We have started to require students to identify the type of energy and matter, or the type of information, in these matrix boxes to receive full credit.

Extensions to the Lesson

It is possible to add a backside to the 5CCs matrix chart consisting of the 6 core competencies described in Vision and Change (Supporting File S1. Exploring Primary Scientific Literature – Blank 5CCs Matrix table). This backside of the chart can be used in the same way as the 5CCs side, asking students to connect parts of the PSL they are reading to each competency. In our experience, including these competencies adds more of a "human face" to PSL. Additionally, this version of the matrix chart can encourage career exploration with discussions of "the ability to tap into the interdisciplinary nature of science" and "the ability to communicate and collaborate with other disciplines."

Passages from textbooks can also be used with the 5CCs matrix. During our third implementation of this course, as part of an exam, we provided sample text from an introductory biology textbook used at our university. We conferred with introductory biology instructors at our institution and asked them which topics students seemed to struggle with the most. Based on their replies, we identified sections of the introductory biology textbook (usually 2-3 paragraphs) on these topics and included this text on the third exam. We asked students to evaluate this text using the same matrix chart that they were using for the PSL. The idea was to show students that if/when they go on to introductory biology and are confused with the dense readings in the textbook (or a likely topic they might struggle with), they can apply the 5CCs to help with understanding.

Conclusions

As discussed in the introduction, there are many different ways to use PSL as a teaching and learning tool and combining the 5CCs with PSL can lead to multiple different teaching approaches. Our activity is novel and at the same time very basic, so it could be easily adapted for a plethora of class sizes and course levels. According to our knowledge, this is the first PSL activity designed to encourage students to connect the biological facts they read about to the larger concepts of biology. Our initial data collection suggests that students are beginning to see biology as a larger entity (i.e., they begin to see the "big picture" of biology). We are just beginning to explore how to use the 5CCs as a learning framework, but we believe that our combined 5CCs/PSL activity is an important step in uncovering best practices on how to do this.

SUPPORTING MATERIALS

• Supporting File S1. Exploring Primary Scientific Literature – Blank 5CCs Matrix table.
• Supporting File S2. Exploring Primary Scientific Literature – example syllabus.
• Supporting File S3. Exploring Primary Scientific Literature – Slides for introducing the 5CCs.
• Supporting File S4. Exploring Primary Scientific Literature – Slides for introducing PSL.
• Supporting File S5. Exploring Primary Scientific Literature – example exam.
• Supporting File S6. Exploring Primary Scientific Literature – exam grading rubric.

ACKNOWLEDGMENTS

We thank Dr. Sally Hoskins for providing mentoring and inspiration through C.R.E.A.T.E. We thank our Learning Assistant Nicholas Hernandez for help with assessing matrix charts. We also thank our three semesters of students for their participation and enthusiasm for both our class and for reading PSL.

References

1. American Association for the Advancement of Science. 2011. Vision and Change in Undergraduate Biology Education: A Call to Action. Washington, DC. https://live-visionandchange.pantheonsite.io/wp-content/uploads/2011/03/Revised-Vision-and-Change-Final-Report.pdf
2. Coil D, Wenderoth MP, Cunningham M, Dirks C. 2010. Teaching the process of science: faculty perceptions and an effective methodology. CBE–Life Sciences Education 9(4):524-35. https://doi.org/10.1187/cbe.10-01-0005
3. Kozeracki CA, Carey MF, Colicelli J, Levis-Fitzgerald M, Grossel M. 2006. An intensive primary-literature-based teaching program directly benefits undergraduate science majors and facilitates their transition to doctoral programs. CBE–Life Sciences Education 5(4):340-7. https://doi.org/10.1187/cbe.06-02-0144
4. Momsen JL, Long TM, Wyse SA, Ebert-May D. 2010. Just the facts? Introductory undergraduate biology courses focus on low-level cognitive skills. CBE–Life Sciences Education 9(4):435-440. https://doi.org/10.1187/cbe.10-01-0001
5. Sato BK, Kadandale P, He W, Murata PMN, Latif Y, Warschauer M. 2014. Practice makes pretty good: Assessment of primary literature reading abilities across multiple large-enrollment biology laboratory courses. CBE–Life Sciences Education 13(4):677-686. https://doi.org/10.1187/cbe.14-02-0025
6. Snow CE. 2010. Academic language and the challenge of reading for learning about science. Science 328:450-452. http://science.sciencemag.org/content/328/5977/450
7. Krontiris-Litowitz J. 2013. Using primary literature to teach science literacy to introductory biology students. Journal of Microbiology & Biology Education 14(1):66-77. https://doi.org/10.1128/jmbe.v14i1.538
8. Plavén-Sigray P, Matheson GJ, Schiffler BC, Thompson WH. 2017. The readability of scientific texts is decreasing over time bioRxiv 119370; https://doi.org/10.1101/119370
9. Dembo MH, Gibson S. 1985. Teachers' Sense of Efficacy: An Important Factor in School Improvement. The Elementary School Journal. The University of Chicago Press.
10. Seidel SB, Tanner KD. 2013. “What if students revolt?” - Considering student resistance: origins, options, and opportunities for investigation. CBE–Life Sciences Education 12(4):586-95. https://doi.org/10.1187/cbe-13-09-0190
11. Hoskins SG, Stevens LM, Nehm RH. 2007. Selective use of the primary literature transforms the classroom into a virtual laboratory. Genetics 176:1381-1389. https://doi.org/10.1534/genetics.107.071183
12. Hoskins SG, Lopatto D, Stevens LM. 2011. The C.R.E.A.T.E. approach to primary literature shifts undergraduates' self-assessed ability to read and analyze journal articles, attitudes about science, and epistemological beliefs. CBE–Life Sciences Education 10:329-435. https://doi.org/10.1187/cbe.11-03-0027
13. Gottesman AJ, Hoskins SG. 2013. CREATE Cornerstone: Introduction to scientific thinking, a new course for STEM-interested freshmen, demystifies scientific thinking through analysis of scientific literature. CBE–Life Sciences Education 12:59-72. https://doi.org/10.1187/cbe.12-11-0201
14. Round JE, Campbell AM. 2013. Figure facts: Encouraging undergraduates to take a data-centered approach to reading primary literature. CBE–Life Sciences Education 12:39-46. https://doi.org/10.1187/cbe.11-07-0057
15. Murray TA. 2014. Teaching students to read the primary literature using pogil activities. Biochemistry and Molecular Biology Education 42:165-173. https://doi.org/10.1002/bmb.20765
16. Stevens LM, Hoskins SG. 2014. The CREATE strategy for intensive analysis of primary literature can be used effectively by newly trained faculty to produce multiple gains in diverse students. CBE–Life Sciences Education 13(2):224-42. https://doi.org/10.1187/cbe.13-12-0239
17. Kenyon KL, Onorato ME, Gottesman AJ, Hoque J, Hoskins SG. 2016. Testing CREATE at community colleges: An examination of faculty perspectives and diverse student gains. CBE–Life Sciences Education  15(1):ar8. https://doi.org/10.1187/cbe.15-07-0146
18. Sandefur CI, Gordy C. 2016. Undergraduate journal club as an intervention to improve student development in applying the scientific process. Journal of College Science Teaching 45(4):52-58. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/27212737
19. Kararo M, McCartney M. 2019. Annotated primary scientific literature: A pedagogical tool for undergraduate courses. PLOS Biology 7(1):e3000103. https://doi.org/10.1371/journal.pbio.3000103
20. Stein B, Haynes A, Redding M, Ennis T, Cecil M. 2007. Assessing critical thinking in STEM and beyond. In Innovations in E-learning, Instruction Technology, Assessment, and Engineering Education (pp. 79-82). https://doi.org/10.1007/978-1-4020-6262-9_14
21. Sirum K, Humburg J. 2011. The experimental design ability test (EDAT). Bioscene: Journal of College Biology Teaching 37(1):8-16. https://files.eric.ed.gov/fulltext/EJ943887.pdf
22. Wood W. 2009. Innovations in teaching undergraduate biology and why we need them. The Annual Review of Cell and Developmental Biology 25(5):93-112. https://doi.org/10.1146/annurev.cellbio.24.110707.175306
23. Brownell, S.E., Freeman, S., Wenderoth, M.P., Crowe, A.J. (2014). BioCore guide: a tool for interpreting the core concepts of vision and change for biology majors. CBE–Life Sciences Education 13(2), 200-211.
24. Branchaw JL, Pape-Lindstrom PA, Tanner KD, Bissonnette SA, Cary TL, Couch BA, Crowe AJ, Knight JK, Semsar K, Smith, JI. Smith MK, Summers MM, Wienhold CJ, Wright CD, and Brownell SE. 2020. Resources for teaching and assessing the vision and change biology core concepts. CBE–Life Sciences Education (19):es1. https://doi.org/10.1187/cbe.19-11-0243
25. Weir LK, Barker MK, McDonnell LM, Schimpf NG, Rodela TR, Schulte PM. 2019. Small changes, big gains: A curriculum-wide study of teaching practices and student learning in undergraduate biology. PLOS One 14(8):e0220900. https://doi.org/10.1371/journal.pone.0220900
26. Lyman F. 1987. Think-Pair-Share: An expanding teaching technique: MAA-CIE Cooperative News 1:1-2.