Summary

This project, entitled Piloting An Evaluation Framework for Accessible STEM OER, sought to expand and test a preliminary accessibility evaluation framework that educators can use to determine the accessibility of OER based on auditory, visual, and neurological learner needs. The framework was developed and tested to accommodate the needs of STEM faculty in particular, who may be looking to ensure the accessibility of equations and formulas, data visualizations, data tables, simulations and maps.  In the short term, the aim of this work is to enable STEM faculty to be able to effectively determine whether the OER they find meet their campus accessibility requirements and the needs of their learners. The guide is intended to enable faculty to more effectively evaluate the accessibility of the changes they make to the OER they find—and to evaluate and improve OER that they have created themselves— before using and/or sharing. In the long term, the aim is to provide STEM-focused professionals and institutions with tools and information to facilitate effective sharing and discovery of OER both within their institutions and across them, resulting in a greater body of accessibility-aligned OER as resources are continuously evaluated, tagged, and shared. 

 

Outcomes

• STEM OER Accessibility Guidebook 

• ISKME Project Page 

• OER Commons resource page 

• Project Literature Review 

 

Lessons Learned

Stakeholder participation is key:

• The Advisory Council played a crucial role in advancing the project’s work, by providing detailed feedback on the pilot approach and pilot instrument and on versions of deliverables; by sharing resources, research, and tools to inform framework development; and by participating in the dissemination of project results. 

• The participation of STEM faculty in the pilot process was invaluable in that it provided concrete, in situ feedback on the effectiveness of the accessibility framework and ways to enhance its uptake and use. 

• Direct feedback from faculty as well as from advisors also helped in framing and language for the resulting guidebook. For example, when creating a need statement for the guidebook, we learned that highlighting terms like “personalization” and “localization” of educational resources to meet individual students resonated more with faculty than statements about the need to make “accommodations” to address student disabilities. 

Survey design impacts feedback quality and quantity: Gathering faculty feedback on the accessibility framework through a web-based survey format was more complex than anticipated. Several faculty dropped out of the feedback survey early due to its length and expectations for faculty to provide feedback on all components of the framework. The project team thus revised and shortened the survey to ensure completion by subsequent survey respondents, which was ultimately more effective. ISKME concluded that future pilot testing efforts on analogous technical frameworks would benefit from a more modular approach to the instruments. For example, breaking down feedback instruments into smaller chunks or sections, and asking participants to review 1-2 sections each, as opposed to requiring that they review and provide feedback in the entire framework. 

SCORE Network participation positively affected knowledge sharing: The strong professional connections made through working collaboratively with Bates and the QUBES network, with SERC and with our expert advisors has resulted in positive community building and knowledge sharing across institutions and organizations. Being a mini-grant recipient and participating in related Bates and the QUBES network facilitated cohorts has been of great value to our work and sense of community belonging. The willingness to help, enthusiastic interest, and collegiality on the part of network members supported our search for advisors and faculty participants, and will continue to support our dissemination efforts. Given ISKME’s expertise with OER, we had additional opportunities during the project to spread awareness and offer support and learning to those in the network at different stages in OER awareness, from educators just starting out (the majority) to those who were well acquainted with open practices. Our project team benefited from thoughtful input and collaboration of STEM experts across higher ed institutions. 

 

 For more information contact Amee Godwin, amee@iskme.org