Insights from our Tagging Ontologies Working Group
This past summer, several of us in the Open Education community, participated in a RIOS Institute Working Group to explore potential ways we might think about metadata which supports inclusive and anti-racist Open Education Resources (OER). How we design our OER databases, search functions, and other systems sends a message to the community about what we value pedagogically. Our Working Group concluded and branched into two teams. One of which explored machine learning research to identify racist materials, and the other developed a collaborative research grant to study how systems of labeling and searching (“tagging”) OER can encourage anti-racist and decolonizing curriculum submissions.
In the process of the Working Group, we also learned a lot about OER, text analysis research, and more. We’d like to share our reflections with you!
Robin Taylor, PhD
RTRES Consulting and external evaluator for the RIOS Institute
“Instructors and creators of OER want additional professional development, resources and tools to support finding and assessing OE materials for inclusivity, social justice. There is a range of comfort levels, knowledge of terms, and differing perceptions for what diversity and equity. look like, which likely impacts instructors potential to infuse principles within their instruction.”
Karen Cangialosi, PhD
RIOS Leadership Team and Program Director for the Regional Leaders of Open Education Network (RLOE)
“I learned a great deal about [metadata] tagging, and the definition of ontology vs. taxonomy. I also learned about the variety of computational text analyses (broadly speaking) that exist and how they might be used (and mis-used). I really liked exploring how the use of tagging might help us both increase the body of work that could be considered inclusive, and the number of people using that work. It's exciting to explore the idea that the presence of the tags themselves could be an educational tool or prompt for those that haven’t considered whether their work is or could be inclusive/socially just. I do worry that looking for computational/algorithm solutions to racism, etc., alone (while not what we want to do) could be something that we will have to worry about that others could try to do, or use, or alter from these ideas.”
Carrie Diaz Eaton, PhD
RIOS Leadership Team and Digital and Computational Studies, Bates College
“A lot of people are interested in finding inclusive teaching materials, but need some gentle help finding it. Otherwise they worry about their expertise in evaluating materials for themes of racial justice or may (with positive intent) look for markers that are for possibly related features that are actually not racially just. I think we can help projects that want their materials to be more widely known, help people find inclusive teaching materials, and help people create and evaluate teaching materials that promote racial justice in the STEM classroom. This group has reminded me that I should think about building innovations that set us up for the next challenge - scalability.”
Rob Niemeyer, PhD
Professor of Mathematics, Metropolitan State University of Denver
“This has been an insightful experience. People don’t have the same definition of even ‘social justice.’ I’m finding it is important to rigorously define the words we use so that it is clear to others what we mean. Part of the issue will also be educating people on the meaning of the words and the importance of being aware of when material may be racist or discriminatory. Another issue may be that when a content creator is able to label their material, will they label it accurately? Carrie has mentioned a comments section for OER material; this would be a great way to gauge how the community feels about the content and the labeling.”
Focusing on the positive, that is, labeling new material as anti-racist/nondiscriminatory, is more important and more effective than trying to identify all of the material out there that is racist/discriminatory. Educating the academic public on what may be a red flag is perhaps another goal we should [aim for] so that such materials are more easily avoided, rather than labeled as ‘bad.’ Labeling things as ‘good’ also makes it easier for someone who doesn’t have the time to make sure the material isn’t ‘bad.’ As I’ve recently learned, 10% of the OER community contributes to OER. The other 90% takes, which isn’t a negative, but definitely something to note.”
Stacy Doore, PhD
Clare Boothe Luce Assistant Professor of Computer Science, Colby College
“I had not had much exposure to the OER world prior to this group even though I was in the process over the last three years of creating our own searchable open repository for computing ethics teaching resources (computingnarratives.com). We have struggled with questions about how we curated the materials, the evaluation of the materials, and representation issues.
I also have learned more about the ways in which STEM materials are being submitted to the larger and more mature existing repositories through the partners in this group. Jannelle and Domingo’s work at URI was particularly relevant for my own teaching as I often use case studies in teaching my computing and AI ethics classes. The idea that materials for such a large established teaching materials repository could be so problematic was disturbing.
The amount of time and care needed to review and annotate over 800 case studies for negative impact issues was impressive and this dataset could potentially be used as a pilot training dataset for moving to a machine learning (ML) workflow with humans in the review loop to speed up the review process. It could also be used to identify any positive impact issues in the case studies as exemplars. Applying current AI methodologies can help us to identify common vocabulary and search terms for assessing the current state of existing repository materials.”
Amee Evans Godwin
VP, Research & Development, ISKME
“Our discussions uncovered some deep complexities around working with STEM OER resources and the potential for assessing them for anti-racist/anti-discriminatory content or worldview. The intersections of Open, UDL/accessibility and social justice, and how to help make open resources better described, discoverable, and usable are at the center of our work.Therefore, we’re interested to pursue a larger collaborative project around developing social justice tagging of STEM OER, and eventually professional development to support OER authors in using tags in their authoring practice.
Such a project would leverage frameworks that already exist and adapt them to STEM OER needs. Key to this collaborative project would be to integrate external community and student input around generating and piloting a set of tags. It was so illuminating to hear about the work in analyzing case studies and identifying commonly used racist STEM materials [such as that by Couret]. It was also very useful to discuss the difficulties around finding anti-discriminatory materials and how faculty are struggling with this and need support.”
Hayley Orndorf
Universal Design for Learning Program Manager, BioQUEST
“There are professionals in post-secondary education who want to create, find, and share resources that are accessible, racially and socially just, and meet their content needs. Regardless of their entry point or initial framework of focus, they face similar challenges. For example, the same term might have different definitions across frameworks or delineating between frameworks creates confusion. There are great opportunities to leverage the intersections between frameworks by working towards shared, flexible language. The need for this common language comes with a need for continued professional development that supports disciplinary contextualization of these ideas.”
We are excited to see the Tagging Ontologies project develop even further with our research grant. To hear more, join the RIOS Institute here on QUBEShub, and follow us on Twitter!
About the author