Abstract

Instructors of undergraduate courses in Biology and other Science, Technology, Engineering, and Mathematics (STEM) disciplines face an ongoing challenge to foster higher-order thinking and learning skills while also developing a sound conceptual understanding of both the course concepts and the connections that exist between them in their students. Instructors are further tasked with ensuring that both instruction and assessment align with the published course learning outcomes (CLOs) for the course and support student achievement of those outcomes. This activity was developed in response to these challenges, as well as to the emergence of a number of outcomes-oriented approaches to instruction and recent calls for pedagogical reform of undergraduate STEM instruction in favor of the adoption and creation of student-centered, active learning environments. In this activity, students are tasked with identifying and describing examples of in-class activities, lab exercises, homework assignments, or any other course-related experiences that worked to support their mastery of the course goal(s) and achievement of the posted CLOs. As described, this simple, yet effective syllabus-based activity promotes student awareness of desired outcomes, encourages student reflection on outcome achievement, provides opportunity for higher-order thinking, and serves as a check for instructors on the degree of alignment between the CLOs and student experiences in the course.

Citation

Sovic DM, Chordas SW III. 2020. Tying it all together: An activity to help students connect course experiences to posted learning outcomes. CourseSource. https://doi.org/10.24918/cs.2020.15

Lesson Learning Goals

Students will understand how participation and engagement with the course supports their achievement of the course learning goals and course learning outcomes (CLOs).

Lesson Learning Objectives

Following this activity, students will be able to

• reflect on their experiences in the course.
• recognize alignment between course experiences and the course learning goals and CLOs.
• identify and describe specific examples of course experiences supportive of their achievement of each of the posted course learning goals and CLOs.

INTRODUCTION

Development and dissemination of learning goals, instructional objectives, and the course learning outcomes (CLOs) that are ultimately used to measure student achievement has become a widely accepted and commonly adopted practice in education. To this end, recent calls to reform the academic experience of undergraduate students in biology and other science, technology, engineering, and mathematics (STEM) disciplines (1-7) have included recommendations to adopt student-centered and outcomes-oriented approaches to instruction. Emphasis has also been placed on developing student competency in skills such as critical thinking, communication, and collaboration (2). Similar recommendations have been made in support of fostering metacognition and developing metacognitive skills during instruction (8). Metacognition, commonly defined as thinking about thinking, entails both knowledge and regulatory components (9,10), and is characterized by the processes of planning, monitoring, and evaluating performance associated with any given learning task.

Influential educational researchers such as Bloom (11), Gronlund (12), and Mager (13) introduced the importance of instructional objectives and CLOs. Others laid the groundwork for the development of novel learning-outcomes-oriented approaches to instruction, such as Outcomes-Based Education (OBE) (14,15) and Understanding-by-Design, or UbD (16). These frameworks, which center on the identification of desired learning objectives and CLOs to orient the instructional design process, have since been employed by multiple institutions and individuals to guide curricular development and to develop new lessons, activities, and exercises to facilitate student learning.

Today, calls for the explicit use and dissemination of learning goals, outcomes, objectives, and/or targets are common in the literature within both the K-12 and post-secondary educational realms (17-24). However, some criticism of learning-outcome-based instruction has emerged (25-27), and few studies have been conducted regarding potential benefits to students that result from defining, sharing, and/or using CLOs (28-30).

In spite of limited evidence relating use of CLOs to student performance or perceptions of a course (28-30), making CLOs clear to students has become a generally accepted "best practice" for instruction. From sharing learning targets (19) to posting intended outcomes on syllabi, assignments, and lecture presentations, it is generally assumed that students benefit from having learning expectations in the form of CLOs clearly articulated to them. Habanek (31) goes so far as to begin a discussion of syllabus integrity by describing a course syllabus as, "...a document by which faculty members define learning outcomes for students and the methods by which those outcomes will be realized."

In consideration of the national calls for educational reform, instructors should strive to implement evidence-based best-practices for instruction. Instructors typically provide students with the course syllabus and the intended CLOs early in the term, but often provide little to no follow-up for these potential tools as the course progresses. In the activity we describe below, students reflect on their learning experiences from a course in order to identify the relationships that exist between the specific learning tasks completed in the course and the CLOs defined in the syllabus. This activity mirrors one of the recommendations made by Duffy and Jones (32) regarding instructional strategies to use as a course comes to an end and incorporates a number of practices that work to foster metacognition and enhance the transfer of learning to new contexts, including sense-making, self-assessment, and reflection (8).

Ample evidence is available to support the potential benefits of metacognitive practice on student learning (33-35). Central to both successful metacognitive practice and to the activity we describe here is the act of reflection, which according to Leberman and Martin (36), "...is fundamentally important, provides a major contribution to personal growth, and, therefore, affects the transfer of learning." Because metacognitive skills are known to develop in individuals over time (37), activities designed to help foster those skills are also important. This may be of particular significance to first-year undergraduate students who are often under-prepared for the higher-order thinking tasks common in college courses. By reflecting back on CLOs late in the term, students are provided an opportunity to think critically about their learning experiences and to realize or reinforce connections between the larger, unifying themes of the course. This practice constitutes a metacognitive experience (38) that promotes further metacognitive development, provides practice in metacognitive skills, and, in turn, may help students in their ability to complete higher order thinking tasks (39).

Intended Audience

We used the activity in the context of a non-majors, large enrollment introductory biology course at a major research University. This activity may, however, be easily modified to apply to any course in any discipline that is structured around course goals and intended CLOs.

Required Learning Time

We have presented this exercise to students within the last few weeks of the course as a graded, collaborative, in-class activity, with approximately 20 minutes of face-to-face instructional time dedicated to students actively working on the activity following a brief introduction. In our experience, this has provided a sufficient opportunity for student pairs or groups to consider multiple CLOs and generate, at minimum, two or more examples of satisfactory responses. This has also provided adequate time for the instructor and, in our case, the graduate teaching assistants (GTAs) to interact with multiple groups and to provide feedback on the quality of the responses generated during the session. After approximately 15 to 20 minutes of active collaboration and instructor or GTA feedback, students are informally polled about whether they would like additional time outside of class to complete the activity. In our case, as the number of CLOs defined for the course is relatively large, we fully recognize that this activity requires more than 20 minutes to complete. We ultimately inform students that they may continue work on the activity outside of class either on their own or with their partners or groups and that completed activity sheets should be submitted by each student at the start of a pre-determined future class session. We have adopted this approach to facilitating the activity, as opposed to assigning the exercise as homework or dedicating an entire class session to the activity's completion, in order to provide students with an opportunity to share ideas, ask questions, and receive guidance at the beginning. This class-time introduction to the activity engenders student success with any remaining CLOs they might have to address on their own in order to complete the activity prior to submission.

Prerequisite Student Knowledge

As this is an activity based in reflection and involves student identification of connections between their experiences and intended learning in the course, students should have knowledge of all of the concepts and skills described in the course learning goal(s) and CLOs. At the very least, instructors must ensure that students have been presented with opportunities to work toward achievement of each CLO (via either direct instruction or some other element(s) of the course) prior to the activity.

Prerequisite Teacher Knowledge

Due to the nature of the activity, not a great deal is required in terms of pre-requisite teacher knowledge. A sound understanding of the concepts covered in the course, as well as a familiarity with the course learning goals and outcomes should be most beneficial, particularly for instances when, during the activity, the instructor is interacting with individuals or small groups of students and providing insight or offering advice. Similar recommendations are suggested for any student or GTAs who might assist in facilitating the in-class portion of the activity. In our context, GTAs meet with the instructor on a weekly basis to discuss course activities and assessments, and are prepared to answer student questions and to provide guidance and feedback as the activity proceeds.

SCIENTIFIC TEACHING THEMES

Active Learning

This activity engages students in critical thinking, discussion, collaboration, and brainstorming. Students initially self-select into groups of two to four and are guided, as a whole class, through an example intended to serve as an exemplar for the activity. In this example, students practice the process of identifying an exercise or other experience from the course and describing how it supported the achievement of a CLO identified for them. Students then work cooperatively to, first, reflect on their participation and engagement with the course and, second, to identify what experiences from the course have worked to support their achievement of each of the individual CLOs. The cooperative component of this exercise is initiated in class, and students are provided an opportunity for additional time outside of class to complete the activity, during which time they may choose to continue working cooperatively or, alternatively, to complete the activity on their own.

Assessment

Individual activity submissions are evaluated to measure each student's ability to connect their own learning experiences with the intended learning goals and outcomes for the course. By completing this exercise in reflection near the end of the course, students are, essentially, conducting a self-evaluation of the learning that they experienced as well as the outcomes that they worked to achieve throughout the term. Specifically, in terms of assessment for this activity, formative methods include monitoring the dialogue associated with group conversations, answering questions, and providing feedback on initial student responses during the in-class portion of the exercise. The course instructor and the GTAs actively move about the room and interact with individuals and small groups during the in-class portion of the activity in order to complete these formative assessments.

Individual student worksheets are used to assess achievement of the defined learning goals and outcomes for the activity in a summative manner. By completing the activity, students are provided an opportunity to practice reflection on their experiences in the course and are encouraged to analyze and describe the alignment between those experiences and the CLOs. Written responses that are submitted on the completed activity sheets provide the grader (instructor or GTAs) with evidence of each student's ability to recognize these alignments. Credit is assessed as satisfactory or unsatisfactory for each response (i.e., for each CLO alignment description). Responses that are considered satisfactory describe (typically in a sentence or two) some experience that directly corresponds to class content and include a logical connection to the CLO.

Inclusive Teaching

Students are encouraged to first, work cooperatively to identify and discuss possible course-related assignments or activities that they found to be supportive of their achievement of each of the defined learning goals and outcomes for the course. We take care to form diverse student groups for laboratory exercises early in the term by considering a variety of factors and characteristics (e.g., student major, class rank, gender, interest, experience in the field, etc.) In our experience, students often opt to work with their laboratory partners or groups during the lecture. This helps to ensure that each collaborative group includes a variety of viewpoints and experiences, which should increase the quality of the peer interactions that occur during the collaborative portion of the activity, as well as the activity responses that are submitted. As there are no single correct or incorrect answers to this learning task, all students are provided the opportunity to identify and describe activities from the course that they personally perceive to have been supportive of their achievement of the CLOs. The collaborative nature of the exercise facilitates the sharing of alternative examples of activities from the course that might also have been beneficial regarding outcome achievement. In this way, students are potentially exposed to connections and patterns of thinking that they may not otherwise have ever identified. The requirement, however, of individual submissions upon activity completion is designed to better engage each individual student in reflective practice.

LESSON PLAN

Present this exercise to students within the last few weeks of the course as a graded, collaborative, in-class activity, with approximately twenty minutes of face-to-face instructional time dedicated to activity completion. In our course, this exercise constituted one of five credit-bearing in-class activities, thus it accounted for 20% of our in-class total. We feel that providing a points-based incentive drives student engagement and helps to ensure students submit thoughtful responses for each CLO. We encourage instructors to take a similar approach and to present this as a graded activity with credit commensurate with the relevance of the exercise in their course.

Initially, permit students to form pairs or groups (maximum of 4 members per group) to complete the exercise with the stipulation that activity sheets will be submitted and graded individually. Derive the activity sheet directly from the course syllabus, modified to include a heading ("Name" field, course and activity identification), activity instructions, and additional writing space following each individual course learning outcome listed (see Supporting File S1. Tying it All Together – Example Activity Sheet). If the syllabus does not include a suitable list of the CLOs, a separate activity sheet should be generated that clearly lists the CLOs and provides adequate space for students to provide written responses.

After the activity sheets are distributed and groups have been formed, present a brief presentation (we use PowerPoint) to demonstrate to students the proper level of detail required for successful completion of the activity. During this component of the activity, present an example learning outcome from the course and ask students to work in their groups to brainstorm about what they did in the course that supported their achievement of that outcome. Then give groups an opportunity to share their ideas, prior to presenting examples of what constitutes an unacceptable response and what constitutes an acceptable, full-credit response. This brief exercise makes clear the expectations for the students and works to guide the level of detail and effort put forth as the students describe the activities identified as being supportive of each outcome. The activity instructions that we use to introduce the activity and the CLO that we use for the initial brainstorming step are provided in Supporting File S2. Tying it All Together – Sample Activity Introduction. The example we provide to students to clarify the difference between satisfactory and unsatisfactory responses is also included in Supporting File S2. Tying it All Together – Sample Activity Introduction. A blank template that can be modified to introduce the activity in other courses is available in Supporting File S3. Tying it All Together – Activity Introduction Template. We typically present the sample satisfactory and unsatisfactory responses and quickly/informally poll the class about which they believe would receive full credit when graded. We then facilitate a discussion about the elements of a response (e.g., details of the experience or activity are included; description connects the experience or activity to the CLO) that are necessary to earn credit for the activity.

After introducing the activity and clarifying what constitutes an appropriate response, we provide students with the intended amount of time to work collaboratively in class (approximately 15 to 20 minutes). After students have received the intended amount of time to work in their groups, we informally poll them about whether they need additional time outside of class to complete the activity. In our experience, considering that our course includes 20 individual CLOs, the initial 15 to 20-minute time period has been sufficient for groups to address multiple CLOs, ask questions, and receive clarifications, but has been typically insufficient for activity completion. Therefore, we inform the students that while they may complete the remainder of the activity individually or collaboratively, completed activity sheets should be submitted by each individual at the start of a pre-determined future class session. A recommended timeline for planning and facilitating the lesson is provided in Table 1.

The components of scientific teaching most notably highlighted in this activity include assessment and active learning. First, the opportunity for assessment takes a variety of forms in this activity. Instructors can utilize submitted activity worksheets as a summative assessment of a student's ability to (1) recognize alignment between their course experiences and the CLOs and (2) identify and describe examples of course experiences supportive of their achievement of the published CLOs for the course (activity learning objectives 2 and 3). In addition to this, instructors may also use student responses as valuable information in the assessment of the course design, as the responses may help identify student misconceptions as well as content areas for which there may be either an overabundance or a lack of aligned instruction provided in the course.

Also central to this activity, by design, is active learning. Strategies for active learning used in this activity include brainstorming, collaborative thinking/discussion, and reflection. The activity affords students an opportunity to practice metacognitive skills through the reflective component, while the collaborative portion of the activity provides an excellent opportunity for peer instruction.

TEACHING DISCUSSION

Implementation and Effectiveness

This activity has been used in an introductory biology course for non-majors across several terms and students have generally provided positive feedback following activity completion. During one recent term, a study was undertaken to investigate student perceptions of their experience and of the use of CLOs in the course following the activity. Students were specifically asked before and after completion of the activity to complete the statement, "For me, the use of learning outcomes in this course is...," up to five times (40) and responses were reviewed to determine student perceptions of purpose of the learning outcomes, as well as their attitudes regarding the use of outcomes in the course. Complete pre/post surveys were collected for a total of 12 consenting students. Pre-survey responses suggested initial misconceptions regarding the purpose of learning outcomes for five of the twelve participants in the study. Based on subsequent responses in the post-survey, the activity worked to clarify these initial misconceptions for three of those five students. One student, prior to the activity, stated in response to the prompt, "For me, the use of learning outcomes in this course is..." that, "it was ok but I did not like the group work." The additional responses provided by this student included one word answers such as, "good," "fine," and "ok." This indicates a misconception surrounding the purpose and/or nature of learning outcomes. However, in response to the same prompt provided to students soon after completion of the activity, the student responded with, "it was fun and the labs helped me to understand what I learned in class." The student further provided evidence of improved perceptions of CLO purpose in the additional responses, which included references to connections between modules and tests, as well as perceived benefits in retention of information resulting from "...fun activities," completed during the lecture. Only one participant response suggested a pre/post activity change in attitude regarding learning outcomes from initially positive to negative upon completion of the activity. A total of seven students were identified as having positive attitudes regarding learning outcomes in the course based on post-activity survey responses. Points made by students to the instructor that we have also considered particularly informative include an appreciation for the flexibility of the task and a feeling of ownership of or control over the content of their responses. These points are specifically highlighted, as this activity was designed to encourage personal reflection on learning experiences in the course while also facilitating discussion and collaboration. In addition, the vast majority of student responses to the CLO prompts have included well-articulated declarations directly corresponding to their experiences in the course and accurately reflecting the intention of the CLO. Sample student responses that we consider satisfactory in our course (based upon expectations established during the activity introduction) are provided in Table 2.

The primary task within the activity solicited the identification and description of one or more course experiences from each student's personal perspective that they identified as being supportive of their individual achievement of each of the published CLOs. This differs from typical course-related tasks, in which students are presented with a question, challenge, or scenario, and are made responsible for identifying or generating the proper response, or a response they perceive to be preferred or expected by the instructor. We considered the activity to be effective in support of the intended objectives, as it encouraged students to reflect on the class as a whole. In doing so, students were provided with the opportunity to come to a realization that the course CLOs were, in fact, achieved, or, at minimum, that offerings and experiences throughout the course provided opportunities to achieve them. This, in our opinion, is an important realization that our students may fail to grasp if this exercise is not performed during the closing weeks of the term.

In addition to the student benefits observed, the instructor also found value in the use of this activity, as it provided unique insight into student perceptions of their learning and possible misconceptions about course concepts. Typical student responses regarding a majority of the CLOs referenced activities or other course elements that aligned well with their intended purpose. Student responses to one CLO, however, suggested either a consistent misinterpretation or misconception. This CLO states that successful students should, "...articulate the expanding and self-correcting nature of biology" and is grouped under the general course goal that, "Students will understand key events in the development of science and recognize that science is an expanding and self-correcting body of knowledge." A sample activity sheet providing the full list of the goals and outcomes defined for our course is provided in Supporting File S1. Tying it All Together – Example Activity Sheet. Student responses for this outcome were mixed, but in multiple instances referenced studying or exploring specific course concepts such as the immune system's response to pathogen entry, natural selection, and apoptosis. These responses indicate that the approach to instruction on the nature of science failed in some way to support achievement of this CLO for at least some students in the course and potentially led to new student misconceptions. This misconception was unknown to the instructor prior to completion of this activity, and it is likely that support of this misconception would have continued in future iterations of the course if not for the insights gained through review of student activity sheets. Based on this new knowledge, the instructor may choose to focus instructional redesign efforts on this particular outcome in an attempt to support students in gaining the ability to more accurately articulate the, "...expanding and self-correcting nature of biology."

Suggestions for Modifications

This activity could be modified and utilized effectively in a variety of ways. While we have only employed the activity at the end of the term as a general reflective exercise on learning across the term, similar exercises could be utilized throughout the term (e.g., weekly, before or after each summative assessment, during the mid-point of the term, etc.) in order to help students in the identification of connections between their experiences in the course and their learning. Alternatively, instructors might be more comfortable providing a list of course assignments or activities and asking students to identify the learning goal or outcome that they feel most closely aligns to each. Ultimately, this activity can be used or modified in any way that facilitates student critical thinking about their experiences in the course and how those experiences relate to their learning gains.

IRB Statement

All data and student contributions that we included in this work have been approved for publication by the Institutional Review Board at The Ohio State University (IRB Protocol #2019E0496).

SUPPORTING MATERIALS

• S1. Tying it All Together – Example Activity Sheet
• S2. Tying it All Together – Sample Activity Introduction
• S3. Tying it All Together – Activity Introduction Template

ACKNOWLEDGMENTS

We would like to thank the Center for Life Sciences Education (CLSE) at The Ohio State University for its ongoing support of instructors in the development and incorporation of new materials and activities to improve undergraduate biology instruction. We would also like to thank Dr. Judith Ridgway and Dr. Caroline Breitenberger for their comments, suggestions, and insights provided during the preparation of this manuscript.

References

1. National Research Council (NRC). 2003. Bio 2010: Transforming undergraduate education for future research biologists. Washington D.C. National Academies Press.
2. American Association for the Advancement of Science (AAAS). 2011. Vision and Change in Undergraduate Education: A Call to Action. Washington, D.C. http://visionandchange.org/files/2011/03/Revised-Vision-and-Change-Final-Report.pdf
3. National Research Council (NRC). 2012. Discipline-based education research: understanding and improving learning in undergraduate science and engineering. S.R. Singer, N.R. Nielson, and H.A. Schweingruber, Editors. Committee on the Status, Contributions, and Future Directions of Discipline-Based Education Research. Board on Science Education, Division on Behavioral and Social Sciences and Education. Washington, D.C. National Academies Press.
4. President's Council of Advisors on Science and Technology (PCAST). 2012. Engage to excel: producing one million additional college graduates with degrees in science, technology, engineering, and mathematics. Washington, D.C. www.whitehouse.gov/sites/default/files/microsites/ostp/pcast-engage-to-excelfinal_2-25-12.pdf
5. Besterfield-Sacre M, Cox M, Borrego M, Beddoes K, Zhu J. 2014. Changing engineering education: views of U.S. faculty, chairs, and deans. J Eng Educ. 103 (2): 193-219.
6. Vasaly HL, Feser J, Lettrich MD, Correa K, Denniston KJ. 2014. Vision and change in the biology community: snapshots of change. CBE Life Sci Educ. 13(1): 16-20.
7. American Association for the Advancement of Science (AAAS). 2015. Vision and Change in Undergraduate Education: Chronicling change, inspiring the future. Washington, D.C. http://visionandchange.org/files/2015/07/VISchange2015_webFinn.pdf
8. Bransford JD, Brown AL, Cocking RR. 2000. How people learn: Brain, mind, experience, and school. Washington D.C. National Academies Press.
9. Flavell JH. 1979. Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. Am Psychol. 34(10): 906-911.
10. Flavell JH. 1987. Speculations about the nature and development of metacognition. In F.E. Weinert & R.H. Kluwe (Eds.), Metacognition, Motivation and Understanding (pp. 21-29). Hillside, New Jersey: Lawrence Erlbaum Associates.
11. Bloom BS. 1956. Taxonomy of educational objectives: the classification of educational goals. Longmans Green. New York.
12. Gronlund NE. 1981. Measurement and evaluation in teaching. Macmillan. New York.
13. Mager R. 1962. Preparing instructional objectives. Fearon. San Francisco, CA.
14. Spady WG. 1994. Outcome-Based Education: Critical Issues and Answers. American Association of School Administrators. Arlington, VA.
15. Spady WG. 1998. Outcomes based education: An international perspective. In J. Gultig, C. Lubisi, B. Parker, and U. Wedekind (Eds.), Understanding Outcomes Based Education: Teaching and Assessment in South Africa. SAIDE & Oxford University Press. Johnnesburg.
16. Wiggins G, McTighe J. 1998. Understanding by design. Association for Supervision and Curriculum Development. Alexandria, VA.
17. Marsh PA. 2007. What is Known about Student Learning Outcomes and How does it relate to the Scholarship of Teaching and Learning? Int J Scholarsh Teach Learn. 1(2): Article 22. Available at: http://digitalcommons.georgiasouthern.edu/ij-sotl/vol1/iss2/22
18. Adam S. 2008. Learning Outcomes Current Developments in Europe: Update on the Issues and Applications of Learning Outcomes Associated with the Bologna Process. Presented at Bologna Seminar: Learning outcomes based higher education: the Scottish Experience. Final Version 15.02.08.
19. Moss CM, Brookhart SM, Long BA. 2011. Knowing Your Learning Target. Educ Leadersh. 68(6): 66-69.
20. Marzano RJ. 2013. Targets, Objectives, Standards: How do they fit? Educ Leadersh. 70(8): 82-83.
21. Hussey T, Smith P. 2003. The uses of learning outcomes. Teach High Educ. 8(3): 357-368.
22. Konrad M, Keesey S, Ressa VA, Alexeeff M, Chan PE, Peters MT. 2014. Setting Clear Learning Targets to Guide Instruction for All Students. Interv Sch Clin. 50(2): 76-85.
23. Allan J. 1996. Learning outcomes in higher education. Stud High Educ. 21(1): 93-108.
24. Maher A. 2004. Learning Outcomes in Higher Education: Implications for Curriculum Design and Students Learning. J Hosp Leis Sport Tour Educ. 3(2): 46-54.
25. Hussey T, Smith P. 2002. The trouble with learning outcomes. Active Learn High Educ. 3(3): 220-233.
26. Hussey T, Smith P. 2003. The uses of learning outcomes. Teach High Educ. 13(1): 107-115.
27. Hussey T, Smith P. 2008. Learning outcomes: a conceptual analysis. Teach High Educ. 13(1): 107-115.
28. O'Neill A, Birol G, Pollock C. 2010. A Report on the Implementation of the Blooming Biology Tool: Aligning Course Learning Outcomes with Assessments and Promoting Consistency in a Large Multi-Section First-Year Biology Course. Can J Scholarsh Teach Learn. 1(1).
29. Deslauriers L, Harris SE, Lane E, Wieman CE. 2012. Transforming the Lowest-Performing Students: An Intervention That Worked. J Coll Sci Teach. 41(6): 76-84.
30. Brooks S, Dobbins K, Scott JJA, Rawlinson M, Norman RI. 2014. Learning about learning outcomes: the student perspective. Teach High Educ. 19(6): 721-733.
31. Habanek DV. 2005. An Examination of the Integrity of the Syllabus. Coll Teach. 53(2): 62-64.
32. Duffy DK, Wright Jones J. 1995. Teaching within the rhythms of the semester. Jossey-Bass. San Francisco, CA.
33. Wilson ZS, Holmes L, deGravelles K, Sylvain MR, Batiste L, Johnson M, McGuire SY, Pang SS, Warner IM. 2012. Hierarchical Mentoring: A Transformative Strategy for Improving Diversity and Retention in Undergraduate STEM Disciplines. J Sci Educ Tech. 21: 148-156.
34. Zhao N, Wardeska JG, McGuire SY, Cook E. 2014. Metacognition: An Effective Tool to Promote Success in College Science Learning. J Coll Sci Teach. 43(4): 48-55.
35. Seraphin KD, Philippoff J, Kaupp L, Vallin LM. 2012. Metacognition as means to increase the effectiveness of inquiry-based science education. Sci Educ Int. 23(4): 366-382.
36. Leberman SI, Martin AJ. 2004. Enhancing transfer of learning through post-course reflection. J Adventure Educ Outdoor Learn. 4(2): 173-184.
37. Weil LG, Fleming SM, Dumontheil I, Kilford EJ, Weil RS, Rees G, Dolan RJ, Blakemore S. 2013. The development of metacognitive ability in adolescence. Conscious Cogn. 22(1): 264-271.
38. Efklides A. 2009. The role of metacognitive experiences in the learning process. Psicothema. 21(1): 76-82.
39. Zohar A, Peled B. 2008. The effects of explicit teaching of metastrategic knowledge on low-and high-achieving students. Learn Instr. 18(4): 337-353.
40. Simon B, Taylor J. 2009. What is the Value of Course-Specific Learning Goals? J Coll Sci Teach. 39(2): 52-57.