Abstract

Teaching statistics to students using authentic research is one way to highlight the transferability and value of the material taught in the class. Below we describe an activity in Statistics for Biological and Health Sciences (STAT2510), an introductory required course for the specified majors, at Auburn University in Fall 2018. In this activity, the students were asked to read and critically evaluate a scientific paper, and perform the quantitative analysis themselves. The scientific paper is based on research conducted by another researcher from within the Biology Department of the same institution. This was an effective activity for introductory statistics because it: (1) did not require substantial background knowledge for students, (2) encouraged students to grapple with real data in an authentic research context, and (3) relates to gender equity, which is of general interest to students both within and outside of STEM . Suggestions for adapting this approach to other topics and audiences are provided.

Citation

Lesson Learning Goals

• read and understand primary literature.
• apply a new statistical procedure – One proportion z-test.
• critically evaluate research design.

Lesson Learning Objectives

• compare the use of two statistical procedures: One-sample t-test and one-proportion z-test.
• interpret the outcome of the test based on the p-value.
• perform statistical inferences using statistical software.
• read and critically interpret a paper.

Introduction

Using real-world examples in teaching statistics is one of the priorities emphasized by the Guidelines for Assessment and Instruction in Statistics set by the American Statistical Association (1). Nowadays, almost every author of a textbook on Introductory Statistics tries to follow those guidelines and we have plethora of neatly written textbooks with many real-world problems for students to tackle (2, 3, 4). Nevertheless, it is not an easy task to find a problem that resonates with students and is also readily comprehensible despite students’ modest statistical background. Here I describe a lesson where students critically analyze a paper that is tackling the problem of gender equality, particularly in academia (5).

One should realize that it has only been one century since women in the United States obtained their right to vote and it was only after World War II that women entered the workforce in full. Even now, women are underrepresented in many crucial areas of human enterprise like politics – for instance, we are still waiting for the first female president in the United States (6). Academia, historically shaped by men, has been also a fruitful area for identifying and studying gender disparities (5, 7).

Our example, which relates to gender equity and classroom participation, serves as an excellent illustration of such studies, and is accessible to any major without extensive prior knowledge. We had two large auditorium size classes – up to 150 students, with students of majors from Pre-Nurse to Physical Exercise, pre-Veterinary, and Dietetics. In such circumstances, choosing appropriate topics for analysis is always a hard task. This fact and the sheer size of the classes usually results in many instructors avoiding any extra activities in the class. But we firmly believe that including primary literature into the canvas of the course trains students to critically evaluate data, interpret and understand the scientific process, and to communicate science to scientific audiences (8, 9).

In summary, the activity I describe here is set in an Introductory Statistics class offered from the department of Mathematics and Statistics at Auburn University (Auburn University course catalog, STAT 2510). The instructor used research data published by faculty within Biological Sciences about students in a university classroom setting (5).

The activity consists broadly of three parts: 1. Homework reading of the paper, followed by 2. In-class discussion and analysis, and 3. Performing quantitative analysis as graded assignment.

Below are the full paper reference and its short introduction (5): Ballen, C.J., Lee, D., Rakner, L., and Cotner, S. (2018). Politics, a ‘chilly’ environment for undergraduate women in Norway. PS: Political Science & Politics 51(3):653-658.

Politics a ‘Chilly’ Environment

Authors assessed the extent to which gender disparities existed in a large undergraduate Comparative Politics classroom in Norway, a country with one of the highest ratings of gender equality in the world. Across eight lecture periods - each that lasted approximately two hours – they used an observation protocol that characterized different types of in-class interactions between students and the instructor. For each type of interaction, an observer noted the gender of student who participated. They found that women participated significantly less than would be expected given their numerical presence in the classroom (Figure 1). After the semester ended, authors provided an opportunity for students to explain why they participated (or did not), and whether they felt that barriers in the classroom may have prevented them from expressing their opinions about the course content.

Intended Audience

This activity was embedded in the introductory course, “Statistics for Biological and Health Sciences,” taught at Auburn University in Fall 2018. Although the course is designed for sophomores, the actual population consists of students from all levels, predominantly of biology and health science majors. We would suggest using the lesson in classes that also cover inferential procedures such as hypothesis testing. With the appropriate statistical toolbox, this topic could be also adapted for advanced high school students.

Required Learning Time

The entire activity spans one class but with a prerequisite knowledge lecture we divide it into two: Lesson 0 and Lesson 1. In Lesson 0, the instructor covers a one-proportion z-test procedure and the logic behind it. We assumed that the students are already familiar with the one-sample t-test. T-tests comes before one-proportion z-test in many textbooks, but if not, the instructor has to make sure that both of the procedures are covered. The instructor assigns the research paper for homework reading. If the students are not accustomed to reading scientific papers then it might be helpful to provide some questions for discussions beforehand. The questions are provided in the following sections.

Lesson 1 starts with a short quiz that we call Blitz-Quiz (Supporting File S4. Gender and Participation –Quiz) at the beginning of the class to make sure the students put in effort to read the paper. This is a good exercise for iClicker use, or any other student response device available, that efficiently saves classroom time. From our experience, Blitz-quizzes take less than five minutes. While discussing the research paper, for which we allocate no more than 30 minutes, the instructor has to make sure that the students understood the study. Even if the students have difficulties reading the paper themselves, we strongly believe the reading should be done beforehand.

After thorough discussion of the research paper, the assignment is given in-class and individually. For that the class should be taken in a computer lab or in a setting where students can use their computers safely, otherwise we advise them to do assignment at home. Moreover, if the discussion takes longer than the expected 30 minutes, there might not be enough time to finish it in the class.

Lesson 0: One-proportion z-test procedure has been studied. The paper has been assigned for homework reading.

Lesson 1: Blitz-Quiz is given. In-class discussion of the paper. The Assignment is done in-class and individually.

Prerequisite Student Knowledge

The students should already be familiar with the following terminology and statistical procedures: population, sample, simple random sample, descriptive measures, inferential procedures, population and sample proportion, z-test, t-test, and Binomial distribution. I suggest giving the assignment at the second half of the Introductory Statistics course, when the main bulk of the material has already been covered.

In particular, the students should be familiar with basic inferential statistics procedures. For this class, we found the textbook by Weiss (2016) to be suitable for coverage of these topics (2). The activities described here should be implemented after or in parallel with the coverage of the material in Chapter 12, “Inferences for Population Proportions” of the textbook. Similar content can likely be found in any introductory statistics textbook or website covering the same topic.

The students should also be familiar with standard statistical software like Minitab, SPSS, JMP or similar to perform the assigned tasks. In our case, we used StatCrunch, a web-based software that comes free with Pearson’s Online Learning System. In case software is not available for the class, we recommend using free online calculators.

With regard to reading the primary scientific literature, the topic of the paper doesn’t require specific knowledge except of statistics concepts described above. On the other hand, in order to facilitate reading the instructor can assign the ten discussion questions along with the homework reading. I have not received any student complaints regarding the difficulty of the paper.

Prerequisite Teacher Knowledge

We recommend teachers know basic inferential statistics procedures and terminology such as population, proportion, and sample proportion. Also, an instructor should familiarize themself with the statistical software used in the class (or online calculators that serve the same purpose). There are many textbooks available for introductory statistics classes as well as online resources.

Scientific Teaching Themes

Active Learning

We assigned the paper Ballen et al., 2018 (5) as a reading before the class in which it was discussed. The instructor can use a host of active learning strategies to guide instruction, and we focused on group work and student response. Students worked in self-selected groups of 4-5, and we used a freely available random number generator to call on groups to share their thoughts with the class. Each group designated a reporter and a different reporter each day so the same person did not speak in every class period. Alternatively, reporters can be chosen by the instructor who can assign the role through neutral prompts, such as whoever woke up the earliest that day (10). Random assignments, as well as emphasizing that students are speaking on behalf of their group, may reduce student anxiety about speaking in front of the class (11, 12).

Below are the questions we chose for the student discussion:

1. Describe the setup of the research.

2. How many students had been in the class, according to paper?

3. Does that number represent the population size?

4. What are the researchers interested in exactly? I.e., what is the study about?

5. How exactly do researchers measure participation of women? And how were the data collected?

6. Why was there a need for the inferential procedure and use of statistics? Or, in other words, why can’t the researchers simply describe the data they have obtained?

7. Which statistical procedure is performed?

8. Which procedure here makes more sense: one-proportion z-test or one-sample t-test?

9. Why did the authors perform a the procedure they did?

10. What is the importance and implications of the study?

After the discussion, the students work individually on their assignments in class. They repeat the statistical analysis and should come up with the same numerical conclusions. Finally, they submit their work with numerical results as well as answers with written reasoning.

Assessment

Students turned in a graded assignment that probed students’ understanding of the research and the results, reflecting our student learning objectives (Supporting File S1. Gender and Participation – Assignment Answers). In addition to the assignment, an in-class Blitz-Quiz (Supporting File S4. Gender and Participation – Quiz) worth an extra point was graded.

Inclusive Teaching

The topic of the paper is about the inclusion of women in class discussions, and so naturally the students considered their own experience in the current classroom environment in addition to the context described in Norway in the paper. The instructor can ask the students whether the same situation holds for their own classes based on their observations. The implications of the paper sparked debate about sexism in political science and in STEM, and so it is important to prepare for this by stressing that as instructors, we strive to create inclusive environments in which everyone is comfortable contributing. No matter a student’s subjective opinion about the implications of the paper, everyone can participate and answer the research questions posed by the paper with the data provided. Also, inherent to this work is the question whether gender is a binary variable. However, the ethics of this nature of research can also be something addressed in class. If the instructor is not feeling prepared to discuss such a highly politicized issue, they can note that in this case, the researcher uses “gender” to mean “sex.” In fact, in most government or healthcare facilities collected data up until now the variable “gender” only assumes two values: male and female, meaning that they actually mean “sex.”

Lesson Plan

In this lesson, we offer instructors an engaging activity for teaching concepts in statistics, with a focus on gender equity and participation. The activity is based on primary literature, and students are encouraged to read and critically analyze data that they can relate to, as its focus is on gender in undergraduate classrooms. The lesson activities are spread out over two class periods:

Lesson 0: Background on One-Proportion z-tests

Outcome: One-proportion z-test procedure has been studied; the paper has been assigned for homework reading.

Lesson 1: In-class Discussion of the Reading Assignment

Outcome: Blitz-Quiz is given, in-class discussion of the reading, individual assignment has been given.

Brief Background on One-Proportion z-tests

The first lesson is on the background and assumptions of one-proportion z-tests, a standard for introductory statistics classes. The instructor starts with a brief mini-lecture on the assumptions of a sample as given in Weiss, 2016 (2), page 557:

1. The sample is random;

2. Both np₀ and n(1−p₀) are greater than 5.

Where n is the sample size, and p₀ is the value of the population proportion that has been tested. Our null hypothesis, H₀, is  p = p₀. In the case of binary gender, we are interested in one attribute of our subjects, namely, whether they belong to a particular subpopulation or not (in this case, binary gender identity). Thus, the probability of randomly selecting a subject with the trait under study (e.g., the student identifies as a woman) is exactly equal to the proportion of that subpopulation of subjects with a given trait. This is a binary variable that gives a rise to the Binomial distribution, which can be approximated by Normal distribution under the assumption number 2 above (Figure 2). The full description of the procedure and the reason behind is given in Weiss, Ch.12 (2). We will consider the use of the procedure for the particular case given in the paper by Ballen et. al. (5).

In-class Discussion of the Reading Assignment and Guided Activities

We provide students with questions that can guide them through reading and in-class group discussion. We recommend assembling students into groups, and providing them with these questions on a piece of paper or on the screen and giving them approximately 30 minutes to discuss the research paper.

1. Describe the setup of the research?

Answer: A large classroom in a public university in Norway.

2. How many students had been in the class, according to paper?

Answer: 130 students.

3. Does that number represent the population size?

Answer: In the current study, yes, but it is not a random sample of students, and so the results are not necessarily generalizable across all undergraduate courses.

4. What are the researchers interested in exactly? I.e., what is the study about?

Answer: The researchers are interested in whether women in a classroom are participating as much as one would expect given how many there are in the classroom.

5. How exactly do researchers measure women participation? And how was the data collected?

Answer: By counting the number of student-instructor interactions with regards to student gender. To collect the data observers were attending the classes and using specific protocol were taking notes.

6. Why was there a need for the inferential procedure and use of statistics? Or in other words why the researchers can’t simply describe the data they have obtained?

Answer: To possibly generalize the results of the study beyond the class observed; to avoid random outcomes.

7. Which exactly statistical procedure is performed?

Answer: One-sample t-test. Note that the answer is given in the paper explicitly in the section “Statistical Analyses.”

8. Which procedure here makes more sense?

Answer: One-proportion z-test.

9. But why performed a different procedure?

Answer: As we are trying to get into someone’s head this is indeed a hard question. Below, we have devoted an entire subsection to tackle this question.

10. What is the importance and implications of the study?

Answer: This is an open question. In short, as we are striving for a better society with equal opportunities for everyone, we would like to know any possible disparities that might still exist.

Which Procedure to Choose: One-Sample t-test or One-Proportion z-test?

The choice of the proper statistical procedure for quantitative analysis is crucial. From the previous analyses we have already come to the conclusion that we are comparing the proportion of female responses against the actual proportion of females in the class. Thus the straight-forward choice of the procedure here is One-proportion z-test. Nevertheless, we see that the researchers here have chosen One-sample t-test. A bit more thinking about the choice of the procedure takes us immediately towards extremely politicized issue. Is gender, the variable we are studying here, binary? Gender in modern interpretation is different from sex and is based on an internal interpretation of one’s social roles rather than biological markers. In this case it is obviously non-binary variable. Moreover, one can argue that gender is a continuous variable and the choice of t-test, in that case, doesn’t look unreasonable.

Nevertheless, from the content of the paper we see that the paper still assumes only two distinct values of the variable gender, meaning that by gender they actually mean sex. This confusion is still wide-spread and can be seen in many data set collected by various bodies of government or research institutions. That is why the use of One-proportion z-test fits better here and also makes more sense. As we can see later, both procedures come to the same conclusions.

Reason Behind the Procedure

If women are just as likely to participate in class discussions as men, then we expect to see the same proportions of interactions by gender as the proportion of women in the class, namely 48%. This expectation is the null-hypothesis. However, the observed proportions of interactions between women and the instructor (sample proportions, p̂) are much lower than 48%. This supports the alternative hypothesis, that the women in the class are less likely to participate in the class discussions.

In summary: Null Hypothesis, p = 0.48, and the Alternative Hypothesis, p < 0.48.

Conclusions

As the p-values are very low, we reject null hypothesis and accept alternative one that women students in that political science class in Norway participated in class discussions significantly less than their male counterparts.

After students consider these questions and the instructor discusses the procedure, the instructor gives the assignment for students. While in our class, we assigned this for homework, for the small-sized classes it could better be completed during the lecture period. Students are tasked with first, filling in Table 1 (note it is pre-filled).

Table 1: Interactions by Gender (pre-filled)

Then, students are instructed to find the following proportions for women participation (Table 2).

Table 2: Proportion of women in group, p̂, compared to population proportion, p (pre-filled).

In Table 1, the students found the values from the paper, Ballen et. al., while for Table 2, they performed One-Proportion z-tests introduced in the previous class period using StatCrunch, a web-based statistical software. The command sequence on StatCrunch to follow:

Stat → Proportion Stats → One Sample → With Summary

Next, students are tasked with reflecting on the research approach and analysis used by the researchers with a series of questions. Below we include the questions and the answers (these are also included as a worksheet template in Supporting File S1. Gender and Participation – Assignment Answers).

Question 1: Why did researchers analyze participation for ‘Guest Lecturers’ and the ‘Lead Instructor’ separately? (B) Why did they analyze Spontaneous Comments and Voluntary Responses separately? Give a meaningful answer.

Answer: Might vary. Any reasonable answer can be accepted. (A) to eliminate the possible effect of instructor’s gender or personality on the outcome, i.e., interactions. (B) to see whether different types of participation yielded different responses from students based on gender.

Question 2: What were the size of the class and the proportion of women in it? Give exact answers. Find them in the paper.

Answer: Class Size, N = 130; Proportion of women, p₀ = 0.48.

Question 3: What statistical procedure has been used? Notice that the answer is stated clearly in the section “Statistical Analyses.”

Answer: One-sample t-test.

Question 4: Based on the results of your test (Table 2), make your decision and choose one of the two possible options: Reject Null Hypothesis or Fail to Reject Null Hypothesis.

• Reject Null Hypotheses. Gender affects student participation.
• Fail to reject Null Hypotheses. Based on the observed differences we cannot claim that gender affects student participation.

Answer: Reject Null Hypothesis.

Question 5: What are the possible lurking variables (or any confounding) that can explain the findings? Give your own thoughts.

Answer: Open question. The answer should be reasonable. Examples include the fact that they have a small sample size (1 class over 1 semester); that this population is Norwegian and therefore results are not globally generalizable; that it is an introductory course and so does not represent men and women seriously committed to studying politics.

Question 6: What is the importance of such studies, if any? As this is indeed your opinion, try to follow scientific rigor: be objective, distant, and concise.

Answer: Points are taken only if no answer was given.

In summary the assignment consisted of filling the tables 1 and 2 by students, performing hypothesis testing for all the proportions, writing a short report with summary tables and graphs, and giving their opinion on the matter. Sample answers to the assignment has been attached as an auxiliary material (Supporting File S1. Gender and Participation –Assignment Answers).

The recommended teaching timeline is shown in Table 3.

Table 3. Timeline for the Lesson:

Teaching Discussion

The actual activity we are describing here had been implemented at Auburn University in Fall 2018. It was performed in two auditorium size classes of up to 150 students each in an “Introductory Statistics for Biological and Health Sciences” course. In order to foster the culture of reading first-hand scientific literature, the students were given reading assignments on this course on a regular basis.

There was a special twist on this particular reading as the author of the paper (CB) was a researcher from within another department at the same institution. After finishing the assignment, she was invited as a guest speaker to the class and gave a 15-minute presentation where she talked about the background of the paper and had an interactive Q&A session with students. The guest-speaker presentation slides are attached as an auxiliary material (Supporting File S3. Gender and Participation – Guest Speaker Presentation).

Students Reactions:

• Most of the students (out of almost 300 students in both classes) received full credit (100%) on the assignment. This was likely because, in large part, they were allowed to work in groups and use whatever resources were available to them. Approximately 10 students from each class could not get the numerical calculations right, which highlights the importance of proper training in using statistical software beforehand.

• The topic of the paper (participating in lecture) is familiar to every student who has sat through a lecture, and doesn’t need an extensive background knowledge on the subject.

• Part of the homework assignment asked students to describe the importance, if any, of studies that focus on gender and participation in the undergraduate classroom. It was clear that the topic resonated with students because they could see themselves in the study population. For example, one student reported, “Men and women in many eyes are not yet equal, and studies like this show that while many improvements in equality for women have been made there is still much work to be done. Addressing and publishing the findings of studies like this is an insurance to recognizing the problem at hand and working on ways to fix it.” Providing students with the opportunity to interact with these contemporary data, running the statistics themselves to arrive at an answer, allowed for them the unique opportunity to engage with issues of social justice on both a personal and empirical level.

SUPPORTING MATERIALS

• S1. Gender and Participation – Assignment Answers. The group assignment given to students with exemplary answers and grading rubric.
• S2. Gender and Participation – Assignment Template. Blank assignment template without answers.
• S3. Gender and Participation – Blitz Quiz. Can be used to check student comprehension of the reading assignment. As the questions are all have multiple choice answers, it is possible to use it with iClicker.
• S4. Gender and Participation – Guest-Speaker Presentation. Presentation of the paper by the author.
• S5. Gender and Participation – Lesson at-a-Glance. The shortest possible summary of the Lesson.

Acknowledgments

Special thanks to Cissy Ballen for inspiring and actively participating in the lesson activities and lesson manuscript. We also thank Auburn University’s College of Sciences and Mathematics for support. Personal thanks to Nedret Billor, Dahsol Lee, Lise Rakner, and Sehoya Cotner.

References

1. Guidelines for assessment and instruction in statistics: http://www.amstat.org/education/gaise/GaiseCollege_Full.pdf
2. Weiss NA, Hassett MJ. 2016. Introductory statistics (Addison-Wesley Boston). 10th Ed.
3. Le CT, Eberly LE. 2016. Introductory biostatistics (Wiley). 2nd Ed.
4. Rosner B. 2015. Fundamentals of biostatistics, (Brooks Cole). 8th Ed.
5. Ballen CJ, Lee D, Rakner L, Cotner S. 2018. Politics a “chilly” environment for undergraduate women in Norway. PS: Political Science & Politics:1–6.
6. Roberts, D. Why Hillary Clinton lost the election: The economy, trust and a weak message. The Guardian, Nov 9, 2016.
7. Nelson CE, 1996. Student diversity requires different approaches to college teaching, even in math and science. American Behavioral Scientist 40.2: 165-175. doi:10.1177/0002764296040002007
8. Brownell SE, Price JV, Steinman L. 2013. A writing-intensive course improves biology undergraduates' perception and confidence of their abilities to read scientific literature and communicate science. Advances in Physiology Education, 37(1), 70-79. DOI: 10.1152/advan.00138.2012
9. Hoskins SG, Stevens LM, Nehm RH. 2007. Selective use of the primary literature transforms the classroom into a virtual laboratory. Genetics, 176(3), 1381-1389. DOI: 10.1534/genetics.107.071183
10. Tanner KD, 2013. Structure matters: Twenty-one teaching strategies to promote student engagement and cultivate classroom equity. CBE—Life Sciences Education, 12(3), 322-331. doi: 10.1187/cbe.13-06-0115
11. Cooper KM, Downing VR, Brownell SE. 2018. The influence of active learning practices on student anxiety in large-enrollment college science classrooms. International Journal of STEM Education, 5(1), 1-18. DOI:10.1186/s40594-018-0123-6
12. England BJ, Brigati JR, Schussler EE. 2017. Student anxiety in introductory biology classrooms: Perceptions about active learning and persistence in the major. PloS one, 12(8), doi: 10.1371/journal.pone.0182506