Abstract

In evolution classrooms, introducing and reinforcing the idea of genetic drift and random selection can be challenging, as can be reinforcing appropriate mental models of evolution. Agent-based models offer students the opportunity to conduct a model-based inquiry into the impacts of different features on the outcomes in evolutionary systems, helping to build, test, and expand their mental models of evolution. In this lesson—through independent investigation, model-based inquiry, and discussions with peers—students are introduced to the ways that agent-based models can be used to make predictions and test hypotheses about evolutionary systems. This lesson uses the NetLogo modeling environment, which comes preloaded with several useful teaching models and can be manipulated in an easy-to-use graphical interface. We use three models: a model of peppered moths focused on environmental pressures and natural selection, a red queen model focused on the competitive coevolution of snakes and frogs, and a genetic drift model of E. coli. Together, these models help reinforce evolutionary concepts in a hands-on, student-driven environment while improving their understanding of the utility of computing in evolution research. This lesson can be modified to suit courses of varying student levels and has been successfully adapted to online or lecture-based learning environments.

Citation

Society Learning Goals

Bioinformatics

• Computation in the life sciences
• What is the role of computation in hypothesis-driven discovery processes within the life sciences?
• Ecology and Evolution [METAGENOMICS]
• How can bioinformatics tools be employed to examine ecological niches?
• Computational Skills
• How do biologists employ software development as part of the scientific discovery process?
• What higher-level computational skills can be used in bioinformatics research?

Ecology

• Species-Habitat Interactions
• How do species interact with their habitat?
• Populations
• How do populations change over time?
• Interactions within Ecosystems
• How are living systems interconnected and interacting?
• How do systems change over time?
• Impacts of Humans on Ecosystems
• What impacts do humans have on ecosystems?

Science Process Skills

• Process of Science
• Pose testable questions and hypotheses to address gaps in knowledge
• Plan, evaluate, and implement scientific investigations
• Interpret, evaluate, and draw conclusions from data
• Construct explanations and make evidence-based arguments about the natural world
• Address novel questions through authentic research experiences
• Modeling/ Developing and Using Models
• Recognize the important roles that scientific models, of many different types (conceptual, mathematical, physical, etc.), play in predicting and communicating biological phenomena
• Make inferences and solve problems using models and simulations
• Build and evaluate models of biological systems
• Quantitative Reasoning/ Using Mathematics and Computational Thinking
• Apply the tools of graphing, statistics, and data science to analyze biological data
• Communication and Collaboration
• Share ideas, data, and findings with others clearly and accurately

Lesson Learning Goals

• value the use of simulations and models to investigate evolutionary questions
• understand the role of simulations in understanding evolutionary processes
• understand the basics of agent-based models
• understand the usage of ‘agent’ as it relates to agent-based models

• how natural selection is driven by changing environmental conditions
• how competitive coevolution arises from predator-prey interactions
• genetic drift as an evolutionary force and its relationship to randomness.
• how population size affects the relative influence of genetic drift.

Lesson Learning Objectives

• describe the use of models in evolutionary investigations.
• predict the evolutionary outcomes of different model parameters.
• statistically evaluate model performance under different parameter conditions.
• modify an existing model to answer an independent question.
• test their hypothesis using appropriate statistical tools and the model output data.
• write a short report explaining their results.

Introduction

As ecology and evolution become increasingly ‘big data’ driven fields, introducing the foundations of model-based inquiry, programming, and analysis of high-dimensional datasets is becoming a critical part of the preparation for scientific research (1–4). The Vision and Change in Undergraduate Biology Education report says that “to be current in biology, students should also have experience with modeling, simulation and computational and systems-level approaches to biological discovery and analysis, as well as with using large databases” (5, 6). The Vision and Change standards are similar to those set forth as critical in the preparation of future physicians (7) and prior research shows that conforming to these standards can improve student scientific literacy (8).

Within the evolution classroom, considerable research effort has shown that passive lectures, rather than hands-on activities, fail to dispel persistent evolutionary misconceptions (9–11). Importantly, students' misconceptions about evolution often coexist with a scientifically-accurate understanding of other critical evolutionary concepts (12). A commonly held misconception is an organism’s “use and disuse” or “need” that often is revealed in student descriptions of evolutionary processes (10, 11, 13, 14). To combat these misconceptions, pedagogical techniques must underscore that variation, differential survival, and heritability are necessary and sufficient for evolutionary processes to occur (11). One approach to this pedagogical problem is model-based inquiry, where models of complex systems are presented with few assumptions about the nature of the system. Students can manipulate the strength of a variety of evolutionary parameters and explore their impacts on species and individuals. These models can be constructed without factors or parameters in the model beyond variation, differential survival, and heritability. Examining how these simple models can approximate complex evolutionary systems can help students understand the foundations of evolution and improve their mental models.

Agent-based models (ABMs), also called individual-based models within the ecological literature, are simulation models where complex behaviors emerge from simple rules applied to individual entities, known as ‘agents’ (15, 16). ABMs have been extensively applied in ecological and evolutionary research to investigate the functioning of ecosystems and ecological communities and demonstrate natural selection and evolutionary processes (17, 18). Common characteristics of ABMs in ecology and evolution include the incorporation of fitness-seeking, adaptive behavior as well as representation of both positive and negative feedback loops between individuals and their environments (17). ABMs have also been used as a pedagogical tool to introduce students to complex systems because of their easy-to-manipulate user interface and the ease of incorporation into model-based inquiry.

One environment commonly used to introduce students to ABMs in the classroom is NetLogo (19, 20). NetLogo is a free, open-source environment for creating and implementing agent-based models. Each agent, referred to as a “turtle” in the NetLogo environment, follows a simple set of rules defined by the programmer. These rules determine the actions of the agent in the simulated environment - background pixels known as “patches.” In each model time step, the code defines the actions of the turtle, such as moving from one patch to another, and the actions the turtle should take upon encountering particular patches or particular other turtles in the system. For example, in a model, the turtles may represent individuals in a species, and the patches may be the habitat through which they move and interact. The individual may be instructed via the code to move randomly until finding a suitable habitat patch or encounters another agent. After the condition is met, the agent may be instructed to take another action in response, for example interacting with the second agent (e.g., fight, mate, combine), consuming energy, or dying.

The types of interactions, the degree of intensity of interaction, and the time between interactions can be set by the programmer and often can be directly manipulated by the user. One significant benefit of using NetLogo is the relative ease of use compared to other computational modeling environments. One study found that using NetLogo to reinforce evolutionary concepts can lead to a better student understanding of evolution than using simulations in R, particularly for students with limited computational experience (21). NetLogo provides an extensive “Models’ Library” for various complex systems. The three simulations used in this lesson come directly from this Model Library: “Peppered Moths,” “Red Queen,” and “GenEvo 3 Genetic Drift and Natural Selection” (Table 1).

Table 1. Overview of the models used in this lesson, all available from the NetLogo models library.

The NetLogo environment has been used for a range of instructional and educational purposes, including teaching ecological and environmental systems (20–23), engineering (24), computer science (25), and social science (26). Additionally, NetLogo has been used in a variety of pedagogical settings, both formal and informal, including museums (27), primary and secondary education (22, 28–30), as well as higher education (21, 25, 26, 31). These past implementations and case studies show the flexibility of the NetLogo modeling environment in meeting educational goals and objectives, and how ABMs can foster interdisciplinary understanding and collaboration (32).

Although some of these lessons target concepts relating to natural selection and genetic drift (20, 29, 33, 34), there is a need for an explicit focus on the role of randomness and dispelling need/use-disuse in student understanding of evolutionary processes. To better understand student misconceptions about these concepts, we first assess student understanding using ACORNS assessment items (11). These free-text assessment items prompt students to reason about natural selection and evolution across taxonomic groups and in both gain and loss-of-function trait examples, allowing instructors to diagnose areas of misunderstanding and misconceptions and adapt lesson activities to address these concerns explicitly. With the advent of EvoGrader (35), a machine learning-powered tool to quickly and accurately score and assess responses to ACORNS items, instructors can both capture the power of free-text reasoning for instructional diagnostics while reducing grading time. Our lesson and model selection focuses on deepening students' understanding of randomness as an evolutionary force, particularly through genetic drift models. Additionally, we explicitly combat common misconceptions of use and disuse theories of evolution while involving students in model-based inquiry and an introduction to simulation models in evolutionary investigations.

Intended Audience

This learning activity was used in a course targeted at advanced (primarily Junior and Senior) students majoring in biology and environmental at a large research university (other potential audiences are noted in the discussion). Required prerequisite courses are either a semester course in environmental science or a combination of a semester course in cell structure and function and a semester course in biostatistics. Students are also recommended to take a general genetics course before this course.

Required Learning Time

The lecture takes 15 minutes to complete, installing and preparing the modeling environment can take up to 30 minutes, pre- and post-tests take 5 minutes each to complete (though a modified pre-test can be completed before the class), and each model requires at least an additional 30 minutes of work (Table 2). With all three models, the full lesson takes 180 minutes to complete.

Table 2. Agent-based models in evolutionary education - Teaching Timeline

Prerequisite Student Knowledge

Students should be familiar with basic concepts of evolution (variation, heritability, differential survival). Familiarity with the concepts, including natural selection, genetic drift, and coevolution, can help students develop the modeling competencies and practice model-based inquiry, though this lesson is designed to help them deepen their understanding of these concepts. Familiarity with the peppered moths evolutionary system can be helpful but is not required. Instructors may consider assigning Cook & Saccheri (2013) to introduce students to this scenario.

Prerequisite Teacher Knowledge

This lesson assumes the instructor has some background in evolution, particularly the concepts of genetic drift, competitive coevolution, and natural selection. For instructors unfamiliar with the peppered moth scenario, reading Cook & Saccheri (2013) can help improve the quality of instructors (36). Instructors will also benefit from having a basic knowledge of agent-based modeling. We recommend Grimm et al. (2005) for a clear introduction to agent-based modeling in evolution and ecology (18). Instructors do not need to have any familiarity with writing code or programming in NetLogo or any other scripting language.

Scientific Teaching Themes

Active Learning

Students conduct independent, inquiry-based learning using model-based learning by independently investigating model functioning through guided exploration, developing hypotheses about the model functioning under various conditions, and analyzing their own output data to determine the strength of their predictions. Additionally, students discuss their hypotheses and findings together in small groups of 3-5 students and practice describing their thought processes, defending their predictions, and describing their results.

Assessment

We assess student learning over the course of the laboratory, comparing the pre-test results (Supporting File S2. Agent-based modeling – Pre-test) and post-test (Supporting File S4. Agent-based modeling – Post-test) using ACORNS assessment items. ACORNS assessment items prompt free-text responses to structured evolutionary reasoning questions to elucidate student mental models of evolution (11). Students also complete a set of questions related to their work in a student-generated report, guided by the laboratory protocol. Students evaluate their own progress by comparing their work to their peers in small-group discussions and comparing their predictions to the model outcomes.

Inclusive Teaching

Many ecology and evolution classrooms have been criticized for a lack of inclusivity for students with disabilities, particularly in fieldwork-heavy courses (37–41). Computational research and model-based inquiry have been proposed as a potential solution to broadening participation in ecological and evolutionary research (42, 43). In this lesson, we present a hands-on introduction to modeling and computational biology, enabling participation for a broader range of physical abilities. The hybrid approach of independent work followed by group discussion allows students to move at their own pace and develop their own ideas and hypotheses prior to discussion. This approach also allows students to develop their own thoughts, explain their thinking, and evaluate other suggestions collaboratively. We also present concepts using a variety of forms, including visually through model interfaces, textually through model descriptions and written protocols, and audibly through lectures allowing for concept reinforcement and participation with different learning styles.

Lesson Plan

This lesson was originally designed to be taught in a 180-minute laboratory with 15-20 students, with each student using their own laptop or an in-lab loaner computer available to them. We have also implemented the laboratory via online learning for 10-15 students, each participating through their own laptop. Before the laboratory, we printed out one copy of the procedure per student in each lab section and reused them between lab sessions. For the online implementation, the laboratory procedure was available on the course website. You should also print one copy of the pre-test worksheet (Supporting File S2. Agent-based modeling – Pre-test) and post-test worksheet (Supporting File S4. Agent-based modeling – Post-test) per student, or create an online assessment with the questions. When students initially entered the classroom, we passed out the pre-test worksheet, a worksheet with two ACORNS assessment items that we used as input into an EvoGrader assessment platform to set a baseline understanding of evolutionary processes (Supporting File S2. Agent-based modeling – Pre-test) (11, 35). If you would like to use an alternate ACORNS item, we recommend using at least one plant and one animal example in the pre-test to evaluate taxon-specific understanding. We gave students 15 minutes to complete the assessment and stressed that all answers would be awarded credits. We have previously assigned this to students as a pre-lab homework assignment but found that this usually led to students reading about the evolutionary problem and using sources instead of relying on independent evolutionary reasoning.

Following the pre-test, we presented a short slide presentation (Supporting File S1. Agent-based modeling – Presentation slides) that explains agent-based modeling and the fundamentals of simulation modeling (slides 1-6, about 15 minutes). Then, we introduce students to the NetLogo modeling environment and walk them through installing NetLogo on their machines (slides 7-9). We spent about 15 minutes on this section, but it can vary depending on how long it takes to install on each machine and whether problems arise during installation. Next, we walk them through the NetLogo user interface, explaining the sections of the screen (slide 10, 5 minutes). In the next section of the pre-lab presentation, we present each of the three models students will be using during the laboratory activity: peppered moths, red queen, and GenEvo 3 (E. coli) (Table 1). Slides 12 - 17 each have an intro slide to the model, where we take about 5 minutes to give students a background on the theoretical or experimental grounding of the model, and a second slide that shows the model as it will appear on their NetLogo interface/screen. This slide deck can be modified to suit instructor needs or time constraints. Following the presentation, we pass out the laboratory procedure to each student (Supporting File S3. Agent-based modeling – Laboratory procedure).

Students work independently through the laboratory procedure, populating a word document with their answers to questions listed in the procedure and screenshots of their output graphs. The first procedure is for the peppered moth model, first explaining how to find the model in the NetLogo models library, then providing a short, written description of the model, and then providing a step-by-step description of manipulating the model, creating graphs, and answering listed questions. The second and third portions of the laboratory procedure are structured very similarly but use the red queen model of competitive coevolution and the GenEvo 3 genetic drift model (Table 1). As students complete these procedures, the instructor should roam the room, assisting students with technical questions or concerns (e.g., unable to find the model, run the model, or find the output graph) or confusion about the procedure. The instructor should also ask students to explain their answers, probing for misunderstandings or misconceptions about the model parameters, implementation, or the theoretical evolutionary underpinnings. Each section of the procedure should take approximately 30 minutes to complete, for a total of 90 minutes. Following the procedure, students are prompted to submit a file via email or course management software with the name ‘LastName_ModelingLab.doc.’

Software installation often poses a challenge in this lesson, as student operating systems vary. NetLogo does have a web version available that can be run in a browser and has all models from the model library available in that interface. Details and access are available from the NetLogo download page and are available in the NetLogo user guide. The user guide also provides guidance on running simulations and navigating the NetLogo interface and is available both online and by clicking ‘Help’ at the top of the program. Other common problems include student confusion about setup and running a model, so we often walk part one of the peppered moth lab procedures together to introduce the NetLogo environment.

After students have completed the laboratory procedure, administer the post-test, which includes three ACORNS assessment items (Supporting File S4. Agent-based modeling – Post test). These items include one plant assessment, one animal assessment, and an assessment with a species name and trait name that should be challenging to students, requiring them to use baseline evolutionary reasoning. Instructors should not explain the meaning of ‘suricata’ or ‘pollex’ to the students until after completing the post-test, or the results may be less reliable. Students were given 20 minutes to complete this post-test. After collecting the post-tests, answers to the pre- and post-tests can be scored for conceptual understanding and misunderstanding using the EvoGrader online platform. EvoGrader uses a natural language processing and machine learning method to score responses to ACORNS assessments for essential concept understanding and naive concept misconceptions, which can help improve instruction and assess student understanding (35). Additionally, returned lab documents can be scored for completeness and correctness by the instructor.

Teaching Discussion

The goal of this lesson was to introduce students to agent-based modeling and simulation modeling in evolution while reinforcing evolutionary concepts reviewed in lectures and prior classes. In our experience, students enjoy this laboratory module and are excited to manipulate the models and see the outcomes directly. These models benefit from a relatively simple user interface for students without much computational or programming experience, but also from having an easy-to-modify source code so that students with more programming experience can directly modify model configurations and examine the underlying simulation structure. This lesson is therefore very adaptable to different levels of expertise and students. Though we have primarily implemented this lesson in an upper-division biology course, we have also used these models as demonstrations in introductory biology courses, including a non-majors course. Though we find students are more engaged in smaller groups and with more background biology knowledge, the models are usable for a variety of different knowledge levels.

This course can be easily transitioned to online instruction. In our online section, we had all students in a course room while the instructor gave the short introduction lesson and assisted with the installation and setup of NetLogo. Then, students were split into breakout rooms with 3-5 students as they worked through the protocol and discussions. The instructor rotated through the breakout rooms, asking and answering student questions. In this case, we transitioned the tests to the online classroom management software. One challenge with online instruction was that some students did not have access to laptops or computing environments, which precluded individual students from running and manipulating the model locally. Instead, in an online collaboration room, the instructor shared a screen running the model and had students provide input on what parameters to change or what aspects to investigate. After completing each step in the protocol, the students were given time to write down their answers to the questions and prompts. This approach worked relatively well but removed some of the independent autonomy and inquiry from the lesson structure. A similar approach could be used to incorporate these models into a lecture-format course, with the instructor piloting a model on a projector with student input.

We have also found that some students are very excited about the models, and the independent exploration of the model or the discussion of their findings with their peers can take longer than expected. In these cases, we have typically shifted to using only the peppered moth model and the genetic drift model to give students more time to explore these models in-depth. We recommend, if you need to shorten the lecture, using the genetic drift model alone (as it covers natural selection, genetic drift, and competition) or using the genetic drift model in conjunction with either the peppered moth model or the red queen model, as both serve as good introductions to the NetLogo interface. Additionally, there are multiple other evolution-relevant models in the NetLogo models library, including alternate competition and predator-prey models, alternate genetic drift models, models focused on the evolution of altruism and cooperation, and models of artificial selection. As different models address different evolutionary systems (e.g., different taxa, environmental conditions, or target traits), they may be useful for students with different background knowledge or instructors wishing to target different evolutionary concepts.

We have seen encouraging results from this lesson, particularly in student familiarity with evolution concepts of genetic drift, natural selection, and competitive coevolution. We have found that this lecture helps reinforce an understanding of evolutionary processes across taxa and reduces reliance on naïve mental models of evolution, particularly use-and-disuse models. In our experience, students also develop more robust familiarity with the use of simulation models in evolutionary research and confidence in using and applying these models. In turn, we have observed that this leads to an increased willingness to and excitement in exploring computational models of other ecological and evolutionary processes. In this lesson, students are engaged in a hands-on and model-based inquiry into evolutionary systems, facilitating the building, testing, and expanding student mental models of evolution.

SUPPORTING MATERIALS

• Supporting File S1. Agent-based modeling – Presentation slides
• Supporting File S2. Agent-based modeling – Pre-test
• Supporting File S3. Agent-based modeling – Laboratory procedure
• Supporting File S4. Agent-based modeling – Post-test

Acknowledgments

We thank the numerous students, graduate teaching assistants, and undergraduate learning assistants for actively engaging in the activity and participating in the development and improvement of this lesson. Additionally, the authors are grateful for the feedback and support from the organizers and participants in the 2020 CourseSource Online Writing Studio Short Course. We also acknowledge the support of the George Mason STEM Accelerator, particularly the learning assistant program, particularly the support of Dr. Benjamin Dreyfus. Finally, we thank our undergraduate research assistant, Tedra Cuddy, for her assistance with translation and evaluation. This work was supported by the National Science Foundation Graduate Research Fellowship Program under Grant No. 1842191.

References

1. Hampton SE, Strasser CA, Tewksbury JJ, Gram WK, Budden AE, Batcheller AL, Duke CS, Porter JH. 2013. Big data and the future of ecology. Front Ecol Environ 11:156–162. https://doi.org/10.1890/120103
2. Devictor V, Bensaude-Vincent B. 2016. From ecological records to big data: The invention of global biodiversity. Hist Philos Life Sci 38(4):1-23. https://doi.org/10.1007/s40656-016-0113-2
3. Lacey EA, Hammond TT, Walsh RE, Bell KC, Edwards SV, Ellwood ER, Guralnick R, Ickert-Bond SM, Mast AR, McCormack JE, Monfils AK, Soltis PS, Soltis DE, Cook JA. 2017. Climate change, collections and the classroom: using big data to tackle big problems. Evol Educ Outreach 10(1):1-13. https://doi.org/10.1186/s12052-017-0065-3
4. American Association for the Advancement of Science. 2011. Vision and Change in Undergraduate Biology Education: A Call to Action. Washington, D.C.
5. Woodin T, Carter VC, Fletcher L. 2010. Vision and Change in Biology Undergraduate Education, A Call for Action—Initial Responses. CBE Life Sci Educ 9(2):71–73. https://doi.org/10.1187/cbe.10-03-0044
6. Committee A-H. 2009. Scientific foundations for future physicians. Wash DC Assoc Am Med Coll 26–29.
7. Auerbach AJ, Schussler EE. 2017. Curriculum alignment with Vision and Change improves student scientific literacy. CBE—Life Sci Educ 16:ar29.
8. Bishop BA, Anderson CW. 1990. Student conceptions of natural selection and its role in evolution. J Res Sci Teach 27:415–427.
9. Nehm RH, Reilly L. 2007. Biology majors’ knowledge and misconceptions of natural selection. BioScience 57(3): 263–272. https://doi.org/10.1641/B570311
10. Nehm RH, Beggrow EP, Opfer JE, Ha M. 2012. Reasoning about natural selection: Diagnosing contextual competency using the ACORNS instrument. Am Biol Teach 74(2):92–98. https://doi.org/10.1525/abt.2012.74.2.6
11. Nehm RH, Ha M. 2011. Item feature effects in evolution assessment. J Res Sci Teach 48(3):237–256. https://doi.org/10.1002/tea.20400
12. Brumby MN. 1984. Misconceptions about the concept of natural selection by medical biology students. Sci Educ 68:493–503.
13. Cardoso JCF, Rezende UC. 2017. Evolutionary thinking among biology students in a third world country. Evol Educ Outreach N Y 10(1):1–7. https://doi.org/10.1186/s12052-017-0071-5
14. Bonabeau E. 2002. Agent-based modeling: Methods and techniques for simulating human systems. Proc Natl Acad Sci 99:7280–7287. https://doi.org/10.1073/pnas.082080899
15. Macal CM. 2016. Everything you need to know about agent-based modelling and simulation. J Simul 10(2):144–156. https://doi.org/10.1057/jos.2016.7
16. Grimm V, Railsback SF. 2006. Agent-based models in ecology: Patterns and alternative theories of adaptive behaviour, p. 139–152. In Billari, FC, Fent, T, Prskawetz, A, Scheffran, J (eds.), Agent-based computational modelling: Applications in demography, social, economic and environmental sciences. Physica-Verlag HD, Heidelberg.
17. Grimm V. 2005. Pattern-oriented modeling of agent-based complex systems: Lessons from ecology. Science 310:987–991. https://doi.org/10.1126/science.1116681
18. Tisue S, Wilensky U. (2004). NetLogo: A Simple Environment for Modeling Complexity. International Conference on Complex Systems, Boston, Massachusetts.
19. Wilensky U. 2002. Modeling nature’s emergent patterns with multi-agent languages. Proceedings of EuroLogo 2001 – Linz, Austria.
20. Ameerbakhsh O, Maharaj S, Hussain A, Paine T, Taiksi S. 2016. An exploratory case study of interactive simulation for teaching Ecology, p. 1–7. In 2016 15th International Conference on Information Technology Based Higher Education and Training (ITHET). https://doi.org/10.1109/ITHET.2016.7760725
21. Gkiolmas A, Karamanos K, Chalkidis A, Skordoulis C, Papaconstantinou M, Stavrou D. 2013. Using simulations of NetLogo as a tool for introducing Greek high-school students to eco-systemic thinking. Adv Syst Sci Appl 13(3):276–298.
22. Gkiolmas A, Chalkidis A, Papaconstantinou M, Iqbal Z, Skordoulis C. 2015. An alternative use of the NetLogo modeling environment, where the student thinks and acts like an agent, in order to teach concepts of ecology. ArXiv150105779 Cs.
23. Blikstein P, Wilensky U. 2010. MaterialSim: A constructionist agent-based modeling approach to engineering education, p. 17–60. In Jacobson, MJ, Reimann, P (eds.), Designs for Learning Environments of the Future: International Perspectives from the Learning Sciences. Springer US, Boston, MA.
24. Wiens J, Monett D. (2013). Using BDI-extended NetLogo Agents in Undergraduate CS Research and Teaching. 9th International Conference on Frontiers in Education: Computer Science and Computer Engineering, Las Vegas, Nevada.
25. Riggs WW. 2006. Agent-based modeling as constructionist pedagogy: An alternative teaching strategy for the social sciences, p. 1417–1423. In . Association for the Advancement of Computing in Education (AACE).
26. Martin K. 2018. Multitouch NetLogo for Museum Interactive Game, p. 5–8. In Companion of the 2018 ACM Conference on Computer Supported Cooperative Work and Social Computing. Association for Computing Machinery, Jersey City, NJ, USA.
27. Passmore C, Stewart J. 2002. A modeling approach to teaching evolutionary biology in high schools. J Res Sci Teach 39(3):185–204. https://doi.org/10.1002/tea.10020
28. Dickes AC, Sengupta P. 2013. Learning natural selection in 4th grade with multi-agent-based computational models. Res Sci Educ 43(3):921–953. https://doi.org/10.1007/s11165-012-9293-2
29. Sengupta P, Kinnebrew JS, Basu S, Biswas G, Clark D. 2013. Integrating computational thinking with K-12 science education using agent-based computation: A theoretical framework. Educ Inf Technol 18(2):351–380. https://doi.org/10.1007/s10639-012-9240-x
30. Xiang L, Passmore C. 2015. A framework for model-based inquiry through agent-based programming. J Sci Educ Technol 24(2/3):311–329. https://doi.org/10.1007/s10956-014-9534-4
31. Gammack D. 2015. Using NetLogo as a tool to encourage scientific thinking across disciplines. J Teach Learn Technol 4(1):22–39. https://doi.org/10.14434/jotlt.v4n1.12946
32. Price RM, Pope DS, Abraham JK, Maruca S, Meir E. 2016. Observing populations and testing predictions about genetic drift in a computer simulation improves college students’ conceptual understanding. Evol Educ Outreach 9:8.
33. Pope DS, Rounds CM, Clarke-Midura J. 2017. Testing the effectiveness of two natural selection simulations in the context of a large-enrollment undergraduate laboratory class. Evol Educ Outreach 10:3.
34. Moharreri K, Ha M, Nehm RH. 2014. EvoGrader: an online formative assessment tool for automatically evaluating written evolutionary explanations. Evol Educ Outreach 7(1):15. https://doi.org/10.1186/s12052-014-0015-2
35. Cook LM, Saccheri IJ. 2013. The peppered moth and industrial melanism: Evolution of a natural selection case study. 3. Heredity 110(3):207–212. https://doi.org/10.1038/hdy.2012.92
36. Chalkley B, Waterfield J. (2001) Providing Learning Support for Students with Hidden Disabilities and Dyslexia Undertaking Fieldwork and Related Activities. Geography Discipline Network, Gloucestershire, UK. ISBN: 1 86174 118 9
37. Healey M, Roberts C, Jenkins A, Leach J. 2002. Disabled students and fieldwork: Towards inclusivity? Planet (Plymouth), 6(1):24-26. https://doi.org/10.11120/plan.2002.00060024
38. Hall T, Healey M. 2005. Disabled students’ experiences of fieldwork. Area 37(4):446–449. https://doi.org/10.1111/j.1475-4762.2005.00649.x
39. Hall T, Healey M, Harrison M. 2002. Fieldwork and disabled students: discourses of exclusion and inclusion. Trans Inst Br Geogr 27(2):213–231. https://doi.org/10.1111/1475-5661.00050
40. Greyling E, Swart E. 2011. Participation in higher education : experiences of students with disabilities. Acta Acad 43(4):81–110.
41. Burgio KR, MacKenzie CM, Borrelle SB, Ernest SKM, Gill JL, Ingeman KE, Teffer A, White EP. 2020. Ten simple rules for a successful remote postdoc. PLOS Comput Biol 16(5):e1007809. https://doi.org/10.1371/journal.pcbi.1007809
42. Maestre FT. 2019. Ten simple rules towards healthier research labs. PLOS Comput Biol  15(4):e1006914. https://doi.org/10.1371/journal.pcbi.1006914