Abstract

Success in biochemistry and other life science courses requires that students understand important biological concepts. One of these concepts is how gene regulation ensures that homeostasis is maintained in an organism. The lac operon is a well-studied and historically important example system that displays the key features of a “repressible” operon. Unfortunately, students still hold many misconceptions after repeated formal instruction, and they may benefit from a more immersive approach when learning about this system. Mechanistic computational models and simulations are an important pedagogical tool that can help students learn about the systems-level and dynamic aspects of biological processes such as gene regulation. We developed a modeling and simulation lesson called Advanced Concepts in Regulation of the Lac Operon that uses a guided approach to teach biochemistry or other upper-level undergraduate students. In this lesson, students simulate the lac operon model’s behavior to instantly observe the effects of changes such as altering the components, their regulation, or adjusting the cellular environment. Students predict the simulation output, observe and record the simulation results, and explain the model’s behavior. Students are also encouraged to think critically about the system by answering conceptual questions. The lesson was designed to be self-directed and can be implemented in a variety of educational settings, including in-person, online, and laboratory courses.

Primary image: Regulatory mechanisms and components of the lac operon. (All images used in the lesson, background reading, and slides were created by Christine S. Booth, using Inkscape 0.92; www.inkscape.org.)

Citation

Society Learning Goals

Biochemistry and Molecular Biology

• Discovery requires objective measurement, quantitative analysis and clear communication
• What is the scientific process?
• What skills are needed to access, comprehend and communicate science?
• Homeostasis
• What is the biological need for homeostasis?
• How is homeostasis controlled?

Lesson Learning Goals

1. Know how to quantify and describe homeostatic control mechanisms.*^, **
2. Understand the effect of perturbations on the homeostatic state.*^, **
3. Know that computational models and simulations can be used to visualize and study cellular phenomena.**

Lesson Learning Objectives

1. Mechanistically explain: 
   • Why and how different carbon sources alter the levels of allolactose and cAMP to affect the activity of regulatory proteins that determine lac operon expression.
   • The role and impact of β-galactosidase and promoter leakiness on lac operon expression.
   • The impact of extracellular conditions on the phosphoenolpyruvate transferase (PTS) system and its effect on lac operon expression.
   • The effect of familiar regulatory gene and structural mutations on lac operon expression. Note: familiar mutations are mutations that were explicitly addressed in the lesson.
   • The effect of unfamiliar regulatory gene mutations on lac operon expression. Note: unfamiliar mutations are mutations that were not explicitly addressed in the lesson.
2. Demonstrate transfer of knowledge by predicting the effect of changes in extracellular and intracellular conditions and mutations on the cellular outcomes of an unknown system.
3. Demonstrate transfer of knowledge by interpreting mutations in regulated cellular network diagrams as dynamic cellular events in an abstracted system.

Introduction

Deep understanding of biological systems represents a key requisite to solving global challenges, such as disease and food insecurity. Educators recognize the need to improve their teaching about biological systems (1–5) and have developed activities and approaches to help students analyze trends, make predictions, and overcome misconceptions related to biological systems (6–12). Among the many important concepts in biology and biochemistry courses, students must understand gene regulation and know how gene regulatory principles ensure the maintenance of homeostasis within a biological system (4, 5).

The lac operon is a well-studied and historically salient example of gene regulation, and it is frequently used as an example system to discuss the key features of “repressible” gene regulation in a variety of life science courses (13, 14). However, even after formal instruction, students often continue to have several misconceptions about the details of lac operon regulation (15–17). For example, students may have trouble understanding that biological systems are stochastic (i.e., governed by random probability as opposed to being controlled or directed; 15) or that mutations can manifest in different ways (e.g., mutations in the coding sequence of a gene may not affect the protein’s function while mutations in non-coding DNA could have dramatic effects; 16, 17).

Because gene regulatory systems lend themselves well to modeling, combining the lac operon example system with modeling can serve an important pedagogical function in upper-level life science courses. As pedagogical tools, models serve to formalize reasoning in ways that are beneficial for learning (6, 9, 10, 18–23). Modeling also provides students with opportunities to engage in authentic science practice and challenges them to answer not only “why” but “how” an outcome is reached (24, 25). More specifically, computational models may be used as a way for students to test and refine their knowledge about a modeled system because students are actively engaged and receive immediate feedback about how each of their changes affects the dynamics of the system (26). For example, students can immediately see the effect of a mutation. Finally, computational models can be used to demonstrate the randomness of processes. As a result, a variety of computational models have been used to improve learning in physics, engineering, chemistry, and biology (8, 22, 27-30). In biology, the lac operon example system has been taught in high school and introductory college courses using physical modeling and simulation activities (14, 31).

The current lesson, Advanced Concepts in Regulation of the Lac Operon, builds on a computational modeling lesson that was previously designed by Crowther et al. (31) to teach about the lac operon in introductory biology classes. The lesson expands the scope of the system to teach about advanced gene regulation concepts that are typically covered in upper-level biochemistry and molecular biology courses. These concepts include (i) the impact of promoter leakiness, (ii) the role of regulation in maintaining glucose metabolism, and (iii) the importance of regulating cellular uptake of carbohydrate sources via the phosphoenolpyruvate transferase system (PTS).

Our models and the associated lesson activities were designed using the Cell Collective modeling software. All content on the Cell Collective modeling software is freely accessible without an account, and registering a free account also allows students to save their work. From the homepage, instructors can request instructor access to the lessons, additional instructor features, and support materials or they can gain quick access to the lessons by viewing them as a student. In Cell Collective, models are constructed by connecting components with directed edges (Figure 1; 32). Each component (indicated by a gray dot that changes color when the model is simulated) can describe a variety of elements, such as metabolites, enzymes, or processes. Components are connected by directed edges (indicated by red, green, and gray arrows) that specify the interactions among the components that are known to interact directly in the system. For example, when the lac repressor is active, it inhibits the lac operon. In the software, this relationship is indicated by a red directed arrow, indicating a negative relationship, with the arrowhead starting from a component called “lac repressor” and ending on a component called “lac operon.” Students can specify a variety of starting conditions, using them to simulate the modeled system and observe the results. They can thereby use the models to observe, analyze, and compare the dynamics of the system’s components as conditions change.

We aligned our model building and simulation lesson with evidence-based educational practices. We start the lesson by presenting students with the basic lac operon model and ask them to investigate the model by inspecting the model’s components and connections. We then ask students to predict and observe the model’s behavior and use the results to reason about the role of the components in explaining the behavior (33). We take advantage of the fact that students can make changes to the models and instantly observe the effects of those changes to test their reasoning about the system. We use a scaffolded approach with prompts to help students make changes to the model and test the effects of the changes on the dynamic behavior of the system. In some cases, students are asked to adjust the model themselves, and in other cases, they are presented with a pre-updated model. To facilitate incorporation of the lesson in courses, we also include background reading and directions to a Cell Collective training lesson, an instructor guide, mini-lecture and review slides, and a survey to measure student reactions (Supporting File S1. Advanced lac operon – background reading and instructions to access the training lesson, Supporting File S2. Advanced lac operon – Mini-lecture and review slides, Supporting File S3. Advanced lac operon – Instructor guide and Supporting File S4. Advanced lac operon – Survey to measure student reactions). An optional assessment will be provided upon request from the authors (please contact Dr. Tomáš Helikar at thelikar2@unl.edu).

We implemented the Advanced Concepts in Regulation of the Lac Operon lesson in two semesters of an upper-level biochemistry course for undergraduate students. Students completed a pre-assessment to gauge their understanding of the topic before starting the lesson. Before starting the lesson, students also completed textbook and handout readings and were given instructions to complete a Cell Collective training lesson (Supporting File S1. Advanced lac operon – background reading and instructions to access the training lesson). The training lesson (Cell Collective Training Module: Factors Influencing Exam Scores) was developed to help students learn the basic principles and controls needed to use the Cell Collective software. Finally, students were given the option to review the introductory version of the lesson (Modeling How Glucose and Lactose Impact Lac Operon Regulation) as a refresher about the lac operon's function before starting the lesson. The lesson was started in class and completed as homework. The instructor used the mini-lecture slides to introduce the topic, and students started the activities in small groups. Students were given one week to complete the lesson, after which they completed a post-assessment to measure their understanding of the topic. The goals of the lesson were to increase students’ learning about the advanced mechanisms of gene regulation of the lac operon while simultaneously developing their modeling and simulation skills.

Intended Audience

This lesson was designed for an upper-level undergraduate biochemistry course (Biochemistry II, which is preceded by a prerequisite Biochemistry I course). Our students were typically majoring in biochemistry or a related field, and the class size was large (>100 students). The lesson could easily be used in a variety of class sizes. It could also be used as a refresher in a graduate biochemistry or advanced genetics course before discussing additional topics in gene regulation.

Required Learning Time

This lesson was designed to be started during a 75-minute lecture period and completed as homework (Supporting File S2. Advanced lac operon – Mini-lecture and review slides). Excluding preparation and review, the total time commitment to complete the lesson is approximately 120-145 minutes. The lesson can be accessed by visiting Cell Collective and navigating to the teacher and student options on the home page. Each lesson is identified by its name and Module ID. For the Advanced Concepts in Regulation of the Lac Operon, the Module ID is 45563.

Prerequisite Student Knowledge

The lesson was designed for students who are already familiar with basic concepts of genetics and gene regulation and who know the basic principles and processes of energy metabolism. Specifically, students should know and understand that (i) DNA contains protein-coding and non-coding regions, (ii) prokaryotic transcription and translation machinery is different from eukaryotic machinery, (iii) DNA mutations in a protein-encoding region can alter protein function or change the affinity of DNA binding proteins, (iv) the function of transcription factors can be changed by small molecules, (v) enzymes are regulated by a variety of mechanisms, including regulating their expression, and (vi) many cellular processes, including transcription, directly influence or are influenced by energy metabolism. These topics are generally discussed in an introductory biology course, a general genetics course, or earlier in the biochemistry course. 

Students should also have basic computer and internet skills, such as (i) navigating within a web browser, (ii) following and executing prompts that require them to point, click, and drag, (iii) recognizing “play” and “pause” symbols, (iv) recording information from graphs in tables, (v) answering true/false and short essay questions by clicking and typing. The lesson was designed with in-lesson prompts to ensure that students can successfully complete all the required learning activities. However, we highly recommend that students familiarize themselves with the Cell Collective software by completing the training lesson. The training lesson can be accessed by visiting Cell Collective and clicking the “Get Started as Student” button under the “student tab” on the home page and then clicking on Cell Collective Training Module: Factors Influencing Exam Scores (Module ID: 118363). All content is available without registration, but creating a free account allows users to save their work. Because students will have a wide range of technical capabilities, some will require additional technical assistance. Additional Cell Collective tutorials and technical support are available by selecting “Tutorials” from the help menu dropdown. A new window will open where students can view tutorial videos, “how-to” articles, and receive technical assistance for specific problems by selecting “Contact us” (email address: support@cellcollective.org). 

Optional prerequisite knowledge: Some students may receive added benefit from reviewing the basic regulation of the lac operon using an introductory version of the lesson called Modeling How Glucose and Lactose Impact Lac Operon Regulation (Module ID: 29547).

Prerequisite Teacher Knowledge

Instructors should have a deep understanding of prerequisite student knowledge. We further recommend that instructors have a thorough understanding of the function and components of the lac operon. Specifically, we recommend that instructors familiarize themselves with (i) the biological importance of operons, specifically the lac operon (i.e., to coordinate the expression of a group of genes that are required for lactose import and breakdown), (ii) the basic mechanisms regulating the expression of the operon (i.e., when both lactose and β-galactosidase are present inside the cell, lactose can be converted to allolactose, which binds to the lac repressor and inhibits its DNA binding ability, and (iii) advanced mechanisms of regulating the expression of the operon (i.e., the impact of the phosphorylation status of the PTS on lac permease and adenylate cyclase to coordinate the regulation of lac operon transcription and glucose metabolism). Biochemistry textbooks usually have a chapter entitled “Principles of gene regulation,” or “Gene regulation,” which has a section covering prokaryotic gene regulation and the lac operon that contains the lesson concepts. For textbooks that do not describe PTS function, we recommend a freely available review article about the PTS by Deutscher et al. (34). 

In addition to textbook reading, working through the mini-lecture slides (Supporting File S2. Advanced lac operon – Mini-lecture and review slides), the training lesson (described under “Prerequisite student knowledge”), the instructor guide (Supporting File S3. Advanced lac operon – Instructor guide), the Advanced Concepts in Regulation of the Lac Operon lesson (described under “Required learning time”), and the optional assessment questions (available upon request) will ensure that instructors have all the required knowledge needed to address any student questions. Some instructors may prefer to start completing the lesson immediately and can consult the above mentioned resources as needed.  

Instructors who find navigation of the training lesson challenging are encouraged to access additional tutorials on the Cell Collective website by selecting “Tutorials” from the help menu dropdown. A new window will open where instructors can view tutorial videos and “how-to” articles. Technical assistance for specific problems is also available from the “Tutorials” pages under “Contact us” (email address: support@cellcollective.org). Instructors can also review the introductory version of the lesson (described as an optional resource under “Prerequisite student knowledge”) for additional practice.

Scientific Teaching Themes

Active Learning

Students take an active role throughout the learning experience. Before class, each student completes the optional pre-assessment (available upon request), pre-class reading, and training lesson (Supporting File S1. Advanced lac operon – Background reading and instructions to access the training lesson). Because the practices of modeling and simulation are used in each activity, students must take an active role throughout the lesson (35, 36). Students are asked to (i) investigate a basic, pre-built lac operon model, (ii) predict, observe, record, and explain the model’s behavior, (iii) test and reason through the effect of changes made to the model, and (iv) reflect on their thinking and learning through a series of questions. The lesson is self-paced, and for the homework portion, students are given enough time to engage with their peers or instructional team to seek help or clarification before completing the optional post-assessment. The lesson was designed to actively engage students across a range of instructional formats, including fully in-person and fully online.

Assessment

The instructor can use formative assessment to test students’ understanding (37, 38) using an optional pre-post assessment (available upon request) and in-class polling questions (Supporting File S2. Advanced lac operon – Mini-lecture and review slides). Throughout the lesson, students also use self-evaluation to assess their learning. Specifically, students are asked to (i) explain the model’s behavior based on their recorded results and (ii) answer questions about their results as they test the effects of changes to the model and reason through the results.

Inclusive Teaching

Instructors can leverage group learning by having students work in small groups of three or four students. Courses that are designed with extensive group work components provide all students with an opportunity to receive assistance beyond the instructional team and can improve gains for students from underserved backgrounds (39-41). Small groups may also provide a more comfortable environment for female students to voice their opinions (42).  The computational modeling and simulation lesson is focused on improving science process skills and may, therefore, appeal to non-traditional learners (43). Because most of the activities are self-paced, students with disabilities can extend the time needed for practice (44). Certain features of the modeling environment are customizable, and students can view the materials using multiple forms of presentation (with image, without image, using simulation results, the model, or diagrams to reason), thereby also increasing inclusivity (45). Tablets and cell phones are currently not supported by Cell Collective.

Lesson Plan

The proposed teaching timeline starts one week before the in-class portion of the lesson and ends approximately one week after the in-class portion of the lesson (Table 1). The lesson learning goals and objectives were aligned with the American Society for Biochemistry and Molecular Biology (ASBMB) learning goals (Figure 2). The lesson was designed to be flexible across a range of instructional formats, including fully in-person, blended, and fully online.

Table 1. Teaching timeline for Advanced Concepts in Regulation of the Lac Operon

The required materials to complete the lesson are included as web-resources and Supporting Materials to this publication. 

• Cell Collective Training Module: Factors Influencing Exam Scores (Accessible at Cell Collective, Module ID: 116831)
• Advanced Concepts in Regulation of the Lac Operon (Accessible at Cell Collective, Module ID: 45563)
• Reading materials (Supporting File S1. Advanced lac operon – Assigned reading and instructions to access the training lesson)
• Lecture slides (Supporting File S2. Advanced lac operon – Mini-lecture and review slides)
• Support guide for the instructor (Supporting File S3. Advanced lac operon – Instructor guide)
• Student survey about the lesson (Supporting File S4. Advanced lac operon – Survey to measure student reactions)

Upon request, the authors will provide the following materials to instructors (please contact Dr. Tomáš Helikar at thelikar2@unl.edu): 

• The latest version of instructor materials (Mini-lecture and review slides and Instructor guide)
• Optional assessment questions and answer key 
• Answer key for the lesson

Preparation for In-Class Session

Assign preparatory materials

In our experience, it is ideal to assign the pre-class reading, the training lesson, and the optional pre-assessment approximately one week before the in-class portion of the lesson (Table 1, Supporting File S1. Advanced lac operon – Background reading and instructions to access the training lesson). The materials could be printed and shared with the students during class, or they could be made available as downloadable files, for example, through your course page via your institution’s learning management system (Canvas, Moodle, etc.). The pre-class reading ensures that the students know the basics of the lac operon system and can build on this knowledge. The training lesson (Cell Collective Training Module: Factors Influencing Exam Scores, accessible at Cell Collective, Module ID: 116831) is designed to help students become more comfortable with the technology and teaching approach. In our experience, most students can complete the training lesson independently. The optional pre-assessment allows instructors to use formative assessment to gauge students’ understanding of the lesson concepts before completing the lesson. 

Before the in-class portion of the lesson, remind students to bring a laptop computer. Cell Collective supports the latest versions of major browsers (i.e., Google Chrome, Firefox, Safari). Ideally, each student will have their own computer. However, because students are encouraged to work in groups, students who do not have access to a laptop computer can work with other students who do. Alternatively, the lesson could be scheduled for completion in a computer lab, if available.

Review lesson materials

We recommend that instructors familiarize themselves with the student background reading (Supporting File S1. Advanced lac operon – Assigned reading and instructions to access the training lesson) and also refresh their knowledge about the principles of gene regulation by consulting a biochemistry textbook or other relevant source (e.g., 34, for PTS). Instructors who are unfamiliar with Cell Collective should complete the training lesson (Cell Collective Training Module: Factors Influencing Exam Scores, accessible at Cell Collective, Module ID: 116831). 

Regardless of experience with the lac operon or Cell Collective, we suggest that instructors review the following materials the first time that they teach the lesson: (i) the mini-lecture slides (Supporting File S2. Advanced lac operon – Mini-lecture and review slides; review all slides labeled with “Lecture” in the bottom right corner), and (ii) the instructor guide (Supporting File S3. Advanced lac operon – Instructor guide). If instructors would like to use the optional assessment, we also suggest that they review it at this time. Next, we recommend that instructors complete the lesson online (Advanced Concepts in Regulation of the Lac Operon, accessible at Cell Collective,  Module ID: 45563). 

In our experience, instructors who have completed the training lesson and the Advanced Concepts in Regulation of the Lac Operon lesson will have sufficient knowledge to address most students’ questions and concerns during class. However, instructors who find navigating the training lesson challenging or experience software issues may need additional time to become familiar with Cell Collective and to receive technical assistance (email: support@cellcollective.org; details under “Prerequisite instructor knowledge”).

Optional: Collect and review students’ completed pre-assessments

Depending on the course goals, instructors are welcome to use the assessment that we developed for the formative assessment of student learning (please contact Dr. Tomáš Helikar at thelikar2@unl.edu). Reviewing student answers from the pre-assessment can help instructors anchor their instruction by identifying areas of focus to help students address possible misconceptions during the in-class portion of the lesson. To allow enough time to review the responses, the deadline for completing the pre-assessment should be set far enough in advance of the in-class portion (we suggest at least two days).

In-Class Session

Present mini-lecture slides and initiate small group discussions with polling questions

The online lesson provides students with everything they need to successfully work through the material (additional background information, directions, and question prompts). However, a mini-lecture can help to set student expectations and reduce their anxiety when using this new learning approach (Supporting File S2. Advanced lac operon – Mini-lecture and review slides; review all slides labeled with “Lecture” in the bottom right corner). 

The mini-lecture slides include a brief overview of the essential components and function of the lac operon system and feature three polling questions to ensure that students have read about and are familiar with these components of the lac operon. To ensure that students can correct misconceptions about the fundamental function of the system through discussion, we suggest using a peer instruction approach that follows the best-practices of using polling questions in the classroom (46).   

The mini-lecture slides also list a set of guiding questions to set student expectations of the lesson and includes instructions for getting started with the lesson. By circulating around the room when students begin the lesson, the instructor and teaching assistants can identify students who may be experiencing anxiety, resistance, or technological difficulties. The goal is to ensure that these student issues are addressed as early as possible, ideally before they present a barrier to learning.

Use a scaffolded approach to assist students with the lesson

The lesson was designed to assist in critical thinking, and the scaffolds that we provide for students are rooted in practices that are known to facilitate learning in science (47, 48). We suggest that instructors leverage the many conceptual questions throughout the lesson to maximize student benefit from the lesson (Supporting File S3. Advanced lac operon – Instructor guide, pages 4 and 7). The in-class session can also provide additional opportunities to enhance student learning by engaging in peer-instruction in small groups.

Although not required, we recommend the following practices to ensure a rich and efficient learning environment: 

• Encourage students to continue working in the same small groups that discussed polling questions during class.
• Remind students to independently record and save their answers so that they have access to these answers later.
• While students are completing the lesson, circulate around the room to assist them with technological or conceptual challenges.
• To help answer student questions and orient yourself within the lesson, keep a copy of the instructor guide handy (Supporting File S3. Advanced lac operon – Instructor guide). 

In our experience, most students will complete Activities 1-4 of the lesson by the end of a 75-minute class session (Supporting File S3. Advanced lac operon – Instructor guide, pages 2 and 6). The 75-minute time frame includes the 30-minute mini-lecture with polling questions at the start of the class. About half of the remaining 45 minutes is devoted to discussions between group members to help bring all group members up to speed and waiting to consult with the instructional team to troubleshoot individual student problems. We estimate that only about 20-25 minutes is actively devoted to completing the first four activities. All activities have been designed with sufficient scaffolding so that students can complete the entire lesson independently as a homework assignment.

Students may become frustrated with the scaffolding of the lesson. To alleviate this frustration, it may help to let them know that the lesson is scaffolded to ensure that the entire class succeeds and develop their critical thinking skills. As students proceed through the activities, additional components of the system are introduced. Students must evaluate how these components work together to support and enhance the fundamental function of the system (i.e., to ensure that lactose can be used as a carbon source when glucose is not available). While students are building their knowledge, they must continually integrate and reconcile new information about additional components with their existing conceptual knowledge of the lac operon. We have found that students sometimes view the approach as “learning the system incorrectly” and that it is important to engage with them and help them to reframe their view in terms of “building up and integrating new knowledge in digestible pieces.” Reminding students that this approach is analogous to the process of science may also help.

The lesson is divided into six parts (Supporting File S3. Advanced lac operon – Instructor guide, pages 5 and 6). Each part consists of a group of activities that, together, achieve a specific aim. The parts also provide some built-in modularity for instructors who wish to use only a subset of the lesson activities in their course.

In the first part of the lesson (Activities 1-4), students are given a model containing the essential components of the lac operon system and are asked to review the role of transcription factors in regulating the expression of the operon. Students predict the behavior of the transcription factors, observe and record the model’s behavior, and reflect on potential misconceptions they may have based on the results. 

In the second part of the lesson (Activities 5-7), our aim is to help students conceptually place the lac operon into a familiar and critical cellular context so that expression of the lac operon is not seen as being separate from the other processes in the organism. To reach this aim, students are asked to compare the metabolic routes of environmental glucose and lactose by modifying the model. Students add components representing β-galactosidase and its products to the model. When β-galactosidase is expressed from the operon, it hydrolyzes lactose to glucose and galactose, and these two products can then enter central metabolism. The products from this reaction can therefore be connected to glycolysis in the model. 

In the third part of the lesson (Activities 8-13), we aim to create cognitive dissonance by asking students to create a technically accurate model that does not produce the expected simulation result. In these activities, students explore the role of β-galactosidase further by integrating its second function into the model, conversion of lactose to allolactose. When the second function of β-galactosidase is modeled, no lactose breakdown occurs. Students must therefore critically evaluate why the promoter of the lac operon must be “leaky” (i.e., why some β-galactosidase is always expressed, even when glucose is present). We suspect that if students can overcome this conceptual challenge, their knowledge could transform beyond the existence of a “leaky” promoter to insight about the consequences of a “leaky” promoter. 

In the fourth part of the lesson (Activities 14-17), students are introduced to the role of the PTS. Here, they are provided with an updated model where the PTS has been added. Students are asked to predict the activity of the components (adenylate cyclase (AC) and lac permease) that are regulated by the PTS, observe and record the model’s behavior, and reflect on potential misconceptions they may have based on the results. 

In the fifth part of the lesson (Activities 18-20), we aim to strengthen students’ understanding of the role of the PTS in regulating the lac operon by asking them to evaluate the effect of a PTS mutation. Students are asked to predict the levels of the metabolites that modulate the activity of the repressor and activator, observe and record the model’s behavior, and reflect on potential misconceptions they may have based on the results. 

Finally, in the sixth part of the lesson (Activities 21-24), our aim is to help students apply what they have learned. To do this, we ask students to use the models and evaluate how different mutations can cause the same phenotype. Finally, we ask students to propose a newly engineered strain of E. coli that might relieve the symptoms of lactose intolerance.

Following In-Class Session

After class, assign activities 5-24 of the lesson as homework and encourage students to leave enough time to contact the instructional team for help if they encounter problems. Students should plan to spend an additional 90-120 minutes to complete the remaining activities, not including troubleshooting.

To facilitate students’ independent work and provide assistance with technical or conceptual questions, we encourage them to attend the office hours of any member of the instructional team. However, in our experience, students move through the activities with little need for assistance because the lesson does not require correct answers to progress. During informal discussions with students, we have found that some still have unanswered questions or undetected knowledge gaps. Therefore, to ensure that all students have engaged deeply with the material and understand the main points of the lesson, we highly recommend that instructors also incorporate the optional mini-review.

Optional: Assign follow-up activities and present mini-review

In our experience, students responded well to a follow-up quiz or graded assignment involving the lac operon and a survey about their experience because it became more apparent that their knowledge improved by completing the lesson (Table 1, Supporting File S1. Advanced lac operon – background reading and instructions to access the training lesson, and Supporting File S4. Advanced lac operon – Survey to measure student reactions). To ensure high rates of participation and accurate post-assessment scores, regularly remind students to complete the assessment and survey after completing all activities in the lesson. 

Students have also expressed that a mini-review helps to increase their confidence in their level of understanding of the system. We thus encourage instructors to incorporate the mini-review slides during a follow-up class session if students have completed any part of the lesson out of class (Supporting File S2. Advanced lac operon – Mini-lecture and review slides; review all slides labeled with “Review” in the bottom right corner). The mini-review slides include a brief overview of the lesson and features three polling questions to ensure that students have completed and deeply engaged with the concepts covered in the lesson. The polling questions ensure that students have an opportunity to cement their understanding of the system through discussion.

Ways to Facilitate the Advanced Concepts in Regulation of the Lac Operon Lesson

Although the lesson was designed to be completed independently, we encourage continual engagement with students while they complete the activities. In our experience, it is helpful to frame the role of the instructor and teaching assistants as follows: 

(i) Serve as technological and conceptual guides that: 

• Circulate in the classroom and discuss the lesson with students or answer their questions. 
• Remind students that models have limits that can sacrifice precision for speed. For example, help students to frame the meaning of the output from the “Simulation Graph” by reminding them that activity levels are abstract and that they may equate zero activity with “minimal” and 100 with “maximal.” 
• Check in with individual students or groups to ensure that they are following the instructions and are not skipping over important prompts that will ensure that the model’s behavior can be interpreted during the simulation. 
• Confirm that students are answering the conceptual questions to the required level of explanation (Supporting File S3. Advanced lac operon – Instructor guide, pages 4 and 7). 

(ii) Serve as accessibility guides that:

• Help students form productive groups or engage in productive group behaviors. If instructors find that some groups are struggling, they can use the following strategies to improve team productivity: 

  (a) Devote in-class time so that students can determine what each student’s role will be in the group and ensure that individual voices do not dominate the group (42, 49).

  (b) Some instructors may also find it useful to form groups ahead of class using the CATME software; this is a paid service, check with your institution about an available license or license funding support; 50, 51).

• Remind students that there is more than one way to interact with the models. For example, if they find the background images on the models distracting, they can toggle between them using the “eye” icon in the “Graph” or “Activity Network” panels. 

It may be valuable to help students understand “why” the system functions the way it does so that they can focus their attention on the model to learn “how” the components interact to support this function (24). To reach as many students as possible, it may be more efficient to engage with a single student at a time, who can then facilitate peer teaching after the instructor has moved on to help another group.

Common Areas of Difficulty and Their Solutions

Students may have difficulty navigating the technology or interpreting the meaning of the values from the simulation. We suggest directly engaging with them about their concerns. Remind students to complete the training lesson, be patient while they are learning these new skills, and ask their peers for advice about different ways to navigate the software. Some students may need reminders that Cell Collective provides semi-quantitative results, as opposed to quantitative measures such as concentration. We frequently use the analogy of assessing high versus low expression on a Western blot – even though we might not know exactly how much protein is present, seeing the relative difference in protein levels still provides valuable information. Students may also express anxiety about not being sure whether the simulation results are correct and, as a result, that they are not learning the information “correctly.” Remind students to read the instructions carefully, and if something still does not make sense, to use it as an opportunity to inquire about the system through peer or instructor discussion. Some students may express confusion about the meaning of modeling terms such as “dominance” or have difficulty interpreting the meaning of gray arrows even after completing the training lesson. To address this issue, instructors can add additional polling questions to the introductory slides to check students’ understanding of the terms and use peer instruction to increase understanding.     

Students may also face difficulties and express frustrations about answering the conceptual questions because they may feel ill-equipped. Remind students to seek peer or instructor discussion. Also, remind students to take ownership of their learning by not skipping ahead, carefully reading the information provided throughout the lesson, and reviewing information from additional sources (primary literature, textbook, or reliable internet sources). 

To help increase students’ confidence with the models, we encourage them to play with the final version of the model. To do this, we recommend that they first complete the lesson step-by-step and start their independent exploration after they feel confident that they understand the expected behavior of the model. To start exploring the model independently, students can click on the “Overview” tab at the top of the page and click on the “Restart lesson” button to create a copy of the lesson. If a copy was successfully created, the lesson name at the top of the page will change, and the wording “(Attempt 2)” will be appended to the name. After a copy is created, they should locate the “Workspace” menu and select the “Model” workspace from the dropdown (“Content Design” is the default). Now students can use the “Model” tab to make changes to the model and the “Simulation” tab to simulate the behavior of the model as they make changes.

Teaching Discussion

Implementation

We implemented the lesson twice (Spring 2018 and Spring 2019) in a large-enrollment upper-level undergraduate biochemistry course (Biochemistry II), which is the second course in a two-part series (Biochemistry I and II) and is required for biochemistry majors. Both biochemistry courses serve a large pre-health population at a research-intensive university. In Biochemistry II, 98% of students were native English speakers, and 83% of students’ parents were college-educated. This work was conducted as part of a larger educational research project that was supported by the National Science Foundation grant number NSF DUE-1625804 and was classified as exempt from IRB review. As part of our implementation, we collected data from consenting students during two semesters of Biochemistry II. We used the data from Spring 2018 to update the lesson and assessment and present efficacy data for Spring 2019 in the subsequent discussion (total n = 141, consented n = 100).  

We followed the general format of (i) pre-class preparation and pre-assessment, (ii) in-class mini-lecture and starting the lesson, (iii) completing the lesson as homework, post-assessment, and survey, and (iv) in-class review of the lesson (Table 1). We used the same questions for the pre- and post-assessment, but we used an online delivery format where the question order was randomized for each attempt.

Student Learning Gains From the Lesson

To understand which learning objectives may be best addressed by the lesson, we provide class average scores from pre-to post-assessment for students in Biochemistry II (Figure 3A). Overall, our students appeared to increase their knowledge about the topics covered in the lesson and were able to apply what they learned when asked about an unknown system (Figure 2, Figure 3A). Conversely, they struggled to apply their knowledge to questions about the dual role of β-galactosidase, the effect of unfamiliar mutations, or predictions about a highly abstracted system (Figure 2, Figure 3A). For the first learning objective, the lesson may have supported learning about the basic function of the operon, the PTS system, and familiar mutations (Figure 3A, learning objectives 1a, 1c, and 1d; “familiar mutations” are mutations that were explicitly addressed in the lesson). Conversely, the lesson may not have helped students understand the dual role of β-galactosidase and the need for promoter leakiness (Figure 3A, learning objective 1b). We think that this result may be partly explained by the fact that that portion of the lesson did not follow the predict-observe-explain framework and instead focused more on updating the model. For the first learning objective, students also did less well at predicting the effects of unfamiliar mutations in the lac operon system after completing the lesson (Figure 3A, learning objective 1e; “unfamiliar mutations” are all mutations that were not explicitly addressed in the lesson). For the second learning objective, students could apply what they had learned from the lesson about the lac operon to predict the behavior of an unknown operon of complex function, including the effect of similar mutations to those covered in the lesson (Figure 3A, learning objective 2). For the third learning objective, however, students did less well at using a highly abstracted cellular network diagram to predict the effect of mutations after completing the lesson (Figure 3A, learning objective 3). If the course goals are that students should (i) understand the basic function of the operon, the PTS system, and the mutations covered in the lesson and (ii) have the ability to transfer this knowledge to a similar system, then we believe the lesson is well suited. However, if the instructor also desires that students be able to evaluate (i) the dual role of β-galactosidase and the need for promoter leakiness and (ii) the effect of unfamiliar mutations, it may be necessary to supplement the lesson with additional instruction.

To provide insight into how students’ previous exposure may impact their learning with the current lesson, we provide class average scores from pre- to post-assessment for three subpopulations of students who completed the Advanced Concepts in Regulation of the Lac Operon lesson during Spring 2019 (Figure 3B). The three groups are students who used similar lessons in Biochemistry I (“Consecutive exposure”, n = 39), students who did not use a similar lesson in Biochemistry I (“Non-consecutive exposure”, n = 51), and students who were repeating the course (“Course repeat”, n = 10). We noted that the raw gains for the “Consecutive exposure” and “Course repeat” groups appeared to be larger than for the “Non-consecutive” group. To account for other factors that could influence the results, we conducted a multiple linear regression using the lm function in R (52). We included pre-assessment score, GPA, group, and demographic variables (gender, being a native English speaker, having a parent with a four-year degree, and having a job to fund college life) as predictor variables. The number of students included in this analysis was smaller because we were unable to collect demographic information for all students. When we compared the post-assessment test scores between students in the “Consecutive” (n = 33) and “Non-consecutive” (n = 41) groups, we found no significant differences (F(1, 66) = 0.6457, p = 0.42). Taken together, our results do not provide a definitive conclusion about group differences for this lesson. However, our previous work suggests that students who used one or more simulation lessons to learn about metabolism performed significantly better than students who did not use the lessons (53). We also found that students received additional benefit from repeated exposure to simulation lessons about metabolism (53).

Student Reactions to the Lesson

Student reactions were collected through a short survey consisting of nine questions; six questions were closed-ended and evaluated on a five-point Likert scale, and three questions were open-ended (Supplemental File S4: Advanced lac operon – Survey to measure student reactions). There were no notable differences in reactions between students who had completed similar lessons in their previous biochemistry course and those who had not. When asked whether the lesson assisted their overall learning, 52% of students reported a positive or neutral response. Students had the largest agreement with the statement that the lesson helped them to think about the individual components as well as the larger system, with 63% of students reporting positive or neutral responses. Students had the largest disagreement with the statement that the lesson helped them understand the effect of feedback loops and environmental conditions, with only 49% of students reporting positive or neutral responses. We believe that this number reflects students’ struggles with the dual role of β-galactosidase and the need for promoter leakiness (Figure 3A, learning objective 1b). We also noted that 55% of students reported positive or neutral responses when asked if they think that the lesson would help them remember the material better than they would with an alternative approach, and 57% reported positive or neutral responses about understanding the material better than they would otherwise.   

In the open-ended portion of the survey, we asked students about which part of the lesson they found to be most helpful to their learning. We were encouraged to see that individual students found different aspects to be most helpful and that multiple students reported positively about these different aspects. Representative quotes are listed:  

“The most effective parts [were] looking at the diagrams and seeing the relationships of the different proteins and enzymes.” (Student 1)

“I like that we are able to see how different mutations affect other proteins and the system as a whole.” (Student 2)

“I believe that the simulations [were] key to my understanding of what was going on, along with the summaries spaced out throughout the assignment.” (Student 3)

“The variety of types of questions was nice and forced you to think about all angles.” (Student 4)

When we asked students which part of the lesson they found to be least helpful to their learning, they most frequently reported usability issues or confusion. One student elaborated on their usability experience: 

“I had a hard time with the part talking about beta galactosidase because it was hard to see on the concept map and not the easiest to set up.” (Student 5)

With an average score of 4 out of 5 for the six closed-ended questions, this student agreed that the lesson aided their learning. We suspect that a reminder about toggling the background image between “on” or “off” would have addressed the students’ concern about difficulties seeing the concept map. This student had not experienced the models in their previous biochemistry course, and thus may have also needed more experience and practice with the program so that it would be easier to set up.  

Only a few students elaborated about specific areas of confusion, we noted that there may be unresolved conceptual struggles early on in the lesson, which could benefit from additional intervention by the instructor or more peer group discussion: 

“The parts where we had to add conditional relationships. Those were really confusing and I’m not sure what that means in real life.” (Student 6)

With an average score of 1.5 out of 5 for the six closed-ended questions, this student disagreed that the lesson aided their learning. In this case, the student did not understand that a conditional relationship (gray arrow) is a mathematical description of an “AND” relationship between two components. For example, the lactose component can only activate the allolactose component when β-galactosidase is also present (lactose AND β-galactosidase must both be active to activate allolactose). If the student did not understand how these relationships are represented in the model, they would not be able to interpret the model connections and would not understand the model output. In the training lesson, students learn that a conditional relationship represents the situation where one component can only activate another component when a second component is active. This student may therefore have benefited from a discussion about the meaning of the relationship shortly after completing the training lesson.

Most students did not elaborate further about what they meant when they reported usability issues or confusion. We found that students asked questions during class about usability, but few students enlisted the instructional team’s help with the conceptually challenging questions, especially during the homework portion. During Fall 2018, students reported that they would have preferred a lecture and expressed frustration because they did not know if they were learning the system “correctly” with the lesson. Based on others’ work with curricular change, we speculate that most of these comments reflected students’ lack of confidence with the new learning approach (54). In Spring 2019, some students still expressed similar ideas:  

“I don’t like just answering true and false, especially without any explanation from my professor.” (Student 7)

“I think this type of learning is directed to only ONE type of learner and puts a lot of us at a disadvantage. I know how I learned best and this is definitely not it.” (Student 8)

“I do not find the simulations that are incomplete (missing necessary regulation) helpful at all because they only serve to confuse me later on. I would rather just see correct web of interaction.” (Student 9)

Overall, students reacted positively to various aspects of the lesson. However, some students appeared to be uncomfortable with the new approach and may prefer to continue using the content-based learning that they are accustomed to rather than switching to an active learning approach (55). Keeping an open dialogue with students can help instructors to manage student expectations so that they may begin to value the act of actively constructing understanding over passively receiving information (55, 56).

Possible Adaptations

The lesson was designed so that instructors could employ a variety of adoption approaches ranging from fully in-person to fully online and using a variety of teaching strategies. Depending on the course goals, instructors could employ one of the following adaptations: (i) use the models as part of a laboratory-linked approach where students make predictions with the models that can be tested as part of a laboratory module, (ii) use a stand-alone approach where students independently use the lesson, for example, as a homework or project assignment, or (iii) use a guided-instruction approach (discussed below).                      

Some students may need additional support to combine the new learning approach with unfamiliar biochemistry content. Students may especially benefit from a guided-instruction approach if they express anxiety about learning about the material using the new learning approach. For the guided-instruction approach, instructors could fully discuss the system and how the components fit together before introducing the models. The instructor could also demonstrate how to manipulate the models using an overhead projector or pre-recorded tutorial. By demonstrating how to address common issues and proactively discussing various program-specific features, instructors serve as learning coaches (57, 58). If instructors introduce the components of the complete system before introducing the models, and especially if instructors are also manipulating the models, we recommend polling questions to ensure that all students remain actively engaged with the content. For example, instructors could ask students to discuss the conceptual questions in the lesson and signal to the instructor when they are done. The instructor could then call on individual groups to report back to the class. It is also critical that students use the models themselves in some capacity after the instructor demonstration because active engagement and hands-on experience are important for learning (58).   

We found that students were positive about the in-class assignments because of the peer learning environment. One student made the comment outright:

“[D]o this in class with other students.” (Student 10)

We suggest that out-of-class usage could be modified to take advantage of educational technology tools to facilitate online peer learning (e.g., Flipgrid, Padlet, Yellowdig, etc.). 

In our experience, some conceptual difficulties may persist when using the proposed teaching timeline. We found that students may require additional support with (1) understanding and explaining the need for promoter leakiness, (2) transferring what they have learned about predicting the effects of familiar mutations to predicting the effects of unfamiliar mutations, and (3) concretely establishing the connection between “real life” and the logic of the model (i.e., “translating” between biological descriptions and logical mechanistic descriptions). Instructors may need to spend additional time on these concepts if they are important for the course learning outcomes. 

If time is a concern, instructors may choose to focus only on a subset of activities. The lesson is divided into six parts, which consists of a group of activities that makes it easy for instructors to customize the lesson (details under Lesson plan – “In-class session” and in Supporting File S3. Advanced lac operon – Instructor guide). Parts can also be used to demarcate lesson stopping points for less experienced students. Even if some parts will not be used, we recommend that the topics be covered in the order in which they were presented in the lesson. We also recommend assigning whole parts as opposed to selecting only certain activities within a part. Instructors who elect to use the models to focus only on certain parts of the lesson may need to provide additional support to bring students up to speed with the models.

Conclusion

The Advanced Concepts in Regulation of the Lac Operon lesson described here uses a freely available online software (Cell Collective) and can be used as a resource for biochemistry and other life science instructors (23, 7). Throughout the lesson, students are actively engaged in their learning by constructing, simulating, and interpreting computational models. The approach was relatively well received in the course, and the lesson benefited students’ learning about the concepts covered in the lesson. The lesson can be adapted based on individual course needs and could also be used for online learning and laboratory courses.

SUPPORTING MATERIALS

• S1. Advanced lac operon – Background reading and instructions to access the training lesson
• S2. Advanced lac operon – Mini-lecture and mini-review slides
• S3. Advanced lac operon – Instructor guide
• S4. Advanced lac operon – Survey to measure student reactions

Acknowledgments

We thank Nora Breen, Jon Dietz, Rachel Kubic, James Orf, Mary Pearson, and Dr. Lisa Briona for feedback on the modules. This work was supported by the National Science Foundation grant number NSF DUE-1625804. 

References

1. American Association for the Advancement of Science. 2011. Vision and change in undergraduate biology education: A call to action. Washington, DC.
2. National Research Council. 2012. A framework for K-12 science education: Practices, crosscutting concepts, and core ideas. National Academies Press, Washington, DC.
3. National Research Council. 2013. Next generation science standards: For states, by states. The National Academies Press, Washington, DC.
4. Tansey JT, Baird T, Cox MM, Fox KM, Knight J, Sears D, Bell E. 2013. Foundational concepts and underlying theories for majors in “biochemistry and molecular biology.” Biochemistry and Molecular Biology Education 41:289–296.
5. Loertscher J, Green D, Lewis JE, Lin S, Minderhout V. 2014. Identification of threshold concepts for biochemistry. CBE—Life Sciences Education 13:516–528.
6. Bergan-Roller HE, Galt NJ, Chizinski CJ, Helikar T, Dauer JT. 2018. Simulated computational model lesson improves foundational systems thinking skills and conceptual knowledge in biology students. BioScience 68:612–621.
7. Hester SD, Nadler M, Katcher J, Elfring LK, Dykstra E, Rezende LF, Bolger MS. 2018. Authentic inquiry through modeling in biology (AIM-Bio): An introductory laboratory curriculum that increases undergraduates’ scientific agency and skills. CBE—Life Sciences Education 17:ar63.
8. King G, Bergan-Roller HE, Galt NJ, Helikar T, Dauer JT. 2019. Modelling activities integrating construction and simulation supported explanatory and evaluative reasoning. International Journal of Science Education 1764–1786.
9. Riess W, Mischo C. 2010. Promoting systems thinking through biology lessons. International Journal of Science Education 32:705–725.
10. Kramer M, Olson D, Walker JD. 2018. Design and assessment of online, interactive tutorials that teach science process skills. CBE—Life Sciences Education 17:ar19.
11. Momsen JL, Long TM, Wyse SA, Ebert-May D, DeHaan R. 2010. Just the facts? Introductory undergraduate biology courses focus on low-level cognitive skills. CBE—Life Sciences Education 9:435–440.
12. Stefanski KM, Gardner GE, Seipelt-Thiemann RL. 2016. Development of a lac operon concept inventory (LOCI). Cell Biology Education 15:ar24.
13. Cooper RA. 2015. Teaching the big ideas of biology with operon models. The American Biology Teacher 77:30–39.
14. Champagne Queloz A, Klymkowsky MW, Stern E, Hafen E, Köhler K. 2017. Diagnostic of students’ misconceptions using the biological concepts instrument (BCI): A method for conducting an educational needs assessment. PLOS One 12:e0176906.
15. Bowling BV, Acra EE, Wang L, Myers MF, Dean GE, Markle GC, Moskalik CL, Huether CA. 2008. Development and evaluation of a genetics literacy assessment instrument for undergraduates. Genetics 178:15–22.
16. Smith MK, Knight JK. 2012. Using the genetics concept assessment to document persistent conceptual difficulties in undergraduate genetics courses. Genetics 191:21–32.
17. Gobert JD, Buckley BC. 2000. Introduction to model-based teaching and learning in science education. International Journal of Science Education 22:891–894.
18. Howell ME, Booth CS, Sikich SM, Helikar T, Roston RL, Couch BA, Dijk K van. 2019. Student understanding of DNA structure–function relationships improves from using 3D learning modules with dynamic 3D printed models. Biochemistry and Molecular Biology Education 47:303–317.
19. Dauer JT, Momsen JL, Speth EB, Makohon-Moore SC, Long TM. 2013. Analyzing change in students’ gene-to-evolution models in college-level introductory biology. Journal of Research in Science Teaching 50:639–659.
20. Bergan-Roller HE, Galt NJ, Helikar T, Dauer JT. 2018. Using concept maps to characterise cellular respiration knowledge in undergraduate students. Journal of Biological Education 54(1):33–46.
21. Liang LL, Fulmer GW, Majerich DM, Clevenstine R, Howanski R. 2012. The effects of a model-based physics curriculum program with a physics first approach: A causal-comparative study. Journal of Science Education and Technology 21:114–124.
22. Helikar T. 2020. The need for research-grade systems modeling technologies for life science education. Trends in Molecular Medicine 27(2):100-103
23. Helikar T, Cutucache CE, Dahlquist LA, Herek TA, Larson JJ, Rogers J. 2015. Integrating interactive computational modeling in biology curricula. PLoS Computational Biology 11(3): e1004131
24. Abrams E, Southerland S. 2001. The how’s and why’s of biological change: How learners neglect physical mechanisms in their search for meaning. International Journal of Science Education 23:1271–1281.
25. National Research Council. 2005. Developing understanding through model-based inquiry, p. 515–565. In Donovan, MS, Bransford, J, D (eds.), How students learn: Science in the classroom. Committee on how people learn, a targeted approach for teachers. National Research Council, Washington, DC.
26. Cook DA, Hamstra SJ, Brydges R, Zendejas B, Szostek JH, Wang AT, Erwin PJ, Hatala R. 2013. Comparative effectiveness of instructional design features in simulation-based education: Systematic review and meta-analysis. Medical Teacher 35:e867-98.
27. Rutten N, Van Joolingen WR, Van Der Veen JT. 2012. The learning effects of computer simulations in science education. Computers & Education 58:136–153.
28. Martinez G, Naranjo FL, Perez AL, Suero MI, Pardo PJ. 2011. Comparative study of the effectiveness of three learning environments: Hyper-realistic virtual simulations, traditional schematic simulations and traditional laboratory. Physical Review Special Topics-Physics Education Research 7:20111.
29. Streicher SJ, West K, Fraser DM, Case JM, Linder C. 2005. Learning through simulation: Student engagement. Chemical Engineering Education 39:288–295.
30. Pallant A, Tinker RF. 2004. Reasoning with atomic-scale molecular dynamic models. Journal of Science Education and Technology 13:51–66.
31. Crowther A, Bergan-Roller HE, Galt NJ, Appleby L, Dauer JT, Helikar T. 2019. Discovering prokaryotic gene regulation by building and investigating a computational model of the lac operon. CourseSource. https://doi.org/10.24918/cs.2019.4.
32. Helikar T, Kowal B, McClenathan S, Bruckner M, Rowley T, Wicks B, Shrestha M, Limbu K, Rogers JA. 2012. The Cell Collective: Toward an open and collaborative approach to systems biology. BMC Systems Biology 6:12
33. Kearney M, Treagust DF, Yeo S, Zadnik MG. 2001. Student and teacher perceptions of the use of multimedia supported predict–observe–explain tasks to probe understanding. Research in Science Education 31:589–615.
34. Deutscher J, Francke C, Postma PW. 2006. How Phosphotransferase System-Related Protein Phosphorylation Regulates Carbohydrate Metabolism in Bacteria. Microbiology and Molecular Biology Reviews 70:939.
35. Whitworth K, Leupen S, Rakes C, Bustos M, Perez KE. 2018. Interactive computer simulations as pedagogical tools in biology labs. CBE—Life Sciences Education 17:ar46.
36. Zacharia ZC. 2005. The impact of interactive computer simulations on the nature and quality of postgraduate science teachers’ explanations in physics. International Journal of Science Education 27:1741–1767.
37. Crouch CH, Mazur E. 2001. Peer instruction: Ten years of experience and results. American Journal of Physics 69:970–977.
38. Smith MK, Wood WB, Krauter K, Knight JK. 2011. Combining peer discussion with instructor explanation increases student learning from in-class concept questions. CBE—Life Sciences Education 10:55–63.
39. Haak DC, HilleRisLambers J, Pitre E, Freeman S. 2011. Increased structure and active learning reduce the achievement gap in introductory biology. Science 332:1213–1216.
40. Smith MK, Wood WB, Adams WK, Wieman C, Knight JK, Guild N, Su TT. 2009. Why peer discussion improves student performance on in-class concept questions. Science 323:122–124.
41. Theobald EJ, Hill MJ, Tran E, Agrawal S, Arroyo EN, Behling S, Chambwe N, Cintrón DL, Cooper JD, Dunster G. 2020. Active learning narrows achievement gaps for underrepresented students in undergraduate science, technology, engineering, and math. Proceedings of the National Academy of Sciences 117:6476–6483.
42. Eddy SL, Brownell SE, Thummaphan P, Lan M-C, Wenderoth MP. 2015. Caution, student experience may vary: Social identities impact a student’s experience in peer discussions. CBE—Life Sciences Education 14:ar45.
43. Dirks C, Cunningham M. 2006. Enhancing diversity in science: is teaching science process skills the answer? CBE—Life Sciences Education 5:218–226.
44. Allen J. 2015. Evidence-based practices in the design of interactive multimedia for learners with cognitive learning disabilities, p. 2944–2949. In Society for Information Technology & Teacher Education International Conference. Association for the Advancement of Computing in Education (AACE).
45. Israel M, Wherfel QM, Pearson J, Shehab S, Tapia T. 2015. Empowering K–12 students with disabilities to learn computational thinking and computer programming. Teaching Exceptional Children 48:45–53.
46. Vickrey T, Rosploch K, Rahmanian R, Pilarz M, Stains M. 2015. Research-based implementation of peer instruction: A literature review. CBE—Life Sciences Education 14:es3.
47. Louca LT, Zacharia ZC. 2012. Modeling-based learning in science education: Cognitive, metacognitive, social, material and epistemological contributions. Educational Review 64:471–492.
48. Duit R, Treagust DF. 2003. Conceptual change: A powerful framework for improving science teaching and learning. International Journal of Science Education 25:671–688.
49. Wilson KJ, Brickman P, Brame CJ. 2018. Group work. CBE—Life Sciences Education 17.
50. Loughry ML, Ohland MW, Woehr DJ. 2014. Assessing teamwork skills for assurance of learning using CATME team tools. Journal of Marketing Education 36:5–19.
51. Layton RA, Loughry ML, Ohland MW, Ricco GD. 2010. Design and validation of a web-based system for assigning members to teams using instructor-specified criteria. Advances in Engineering Education 2(1): 1–28.
52. R Core Team. 2013. R foundation for statistical computing.
53. Booth CS, Song C, Howell ME, Rasquinha A, Saska A, Helikar R, Sikich SM, Couch BA, van Dijk K, Roston RL, Helikar T. 2021. Teaching metabolism in upper-division undergraduate biochemistry courses using online computational systems and dynamical models improves student performance. CBE—Life Sciences Education 20:ar13.
54. Kell C, Van Deursen R. 2002. Student learning preferences reflect curricular change. Medical Teacher 24:32–40.
55. Wiggins GP, McTighe J. 2005. Understanding by Design. ASCD.
56. Brazeal KR, Brown TL, Couch BA. 2016. Characterizing student perceptions of and buy-in toward common formative assessment techniques. CBE—Life Sciences Education 15:ar73.
57. Tondeur J, van Braak J, Sang G, Voogt J, Fisser P, Ottenbreit-Leftwich A. 2012. Preparing pre-service teachers to integrate technology in education: A synthesis of qualitative evidence. Computers & Education 59:134–144.
58. Steffens K. 2006. Self-regulated learning in technology-enhanced learning environments: Lessons of a European peer review. European Journal of Education 41:353–379.