Evaluating OER
Learning Objectives
At the end of this chapter, you should be able to:
- Describe three aspects of an OER that should be assessed before use.
- Explain why it is necessary to assess an OER’s adaptability.
Ensuring the quality of course materials is, obviously, of paramount importance. One of the ‘myths’ around OER are that they are of lower/inferior standards than traditional, paid resources. In fact, most OERs are high quality.[1]
Who Reviews OER?
You could argue that we have been crowdsourced quality control is a key feature of academia in the form of peer review. We rely on our discipline-specific peers and other experts in the field to ‘vet’ content in the form of peer review. In terms of OER, some are vetted through a formal peer review process; however, many times, the process is much more informal. Many OERs are not vetted at all and the quality control aspect is up to the faculty member wishing to adopt or adapt the content.
Many publication platforms are taking steps to encourage review. For example, the Open Textbook Library (OTL) encourages faculty members to assess open textbooks.
Some of the evaluation criteria listed below are universal, and others (such as Adaptability and Modularity) are specific to OER.[2]
We’ll be exploring these criteria for evaluation in our learning community in more depth and you’ll be applying them to a selected OER.
Clarity, Comprehensibility, and Readability
The most ubiquitous standard on this list is also the first: can the material you are considering be read and understood by your students? Although it might seem like a simple question, it is a necessary obstacle to confront when adopting a new resource for your course.
Considerations
- Is the content, including any instructions and exercises, clear and comprehensible to students?
- Is the content consistent with its language and formatting? (e.g. key terms are bold)
- Is the content well-organized in terms of sequencing and flow?
Content and Technical Accuracy
The accuracy of the content you use is also a major component of its usability in the classroom. Be sure to check for technical errors such as broken links or typos. In most cases, content accuracy will not be an issue, but some older resources may require updates.
Considerations
- Is the content accurate based on your expertise?
- Are there any factual, grammatical, or typographical errors?
- Is the interface navigable for students?
Adaptability and Modularity
Because of their open licenses, OER permit a wider range of (re)use than most traditional educational content; therefore, it is important to keep in mind how your chosen OER can be adapted. Modularity, or the ability to be broken up into smaller pieces easily, is one feature of an OER that should be preferred when possible. When creating OER, using clear chapter and unit breaks can help other instructors adopting or adapting your resource for their own courses.
Considerations
- Is the resource in a file format which allows for adaptations, modifications, rearrangements, and updates?
- Is the resource easily divided into modules, or sections, which can be used or rearranged out of their original order?
- Is the content available under a license which allows for modifications?
Appropriateness and Fit
Although there may be OER available in your field, some resources may require minor edits or additions. Keep in mind that the open licenses of OER mean that they can be edited or even combined with other resources. This can be particularly useful if you would like to adopt a chapter from one OER for the first unit of your course but prefer alternate resources for other units.
Considerations
- Is the content presented at a reading level appropriate for your students?
- How does the content align with your course learning objectives?
- Is the content level appropriate for use in your course?
Accessibility
No matter what resources you plan to adopt, accessibility should always be a part of your assessment process. Many published-provided homework products are not accessible to students and can cause unexpected issues. Similarly, some OER may not be optimized for students with visual or auditory impairments. We’ll be talking more about accessibility and DEIA in an upcoming session.
Considerations
- Is the content accessible to students with disabilities through the compatibility of third-party reading applications?
- If you are using Web resources, does each image have alt text that can be read? Do videos have accurate closed-captioning?
- Are students able to access the materials in a quick, non-restrictive manner?
If you would like a personal copy of these considerations, visit or download them through the Evaluating OER Checklist in Google Docs.
Additional Rubrics*
* Note: these resources are organized from very basic to complete evaluation rubrics.
Faculty Guide for Evaluating Open Education Resources, BCOER Librarians
This is a very brief checklist with some suggestions to think about before using an OER.
List of Criteria for Evaluating OER -Affordable Learning Georgia
You can also download the list as a modifiable .doc.
[Summarized] Rubrics for Evaluating Open Education Resources Objects
This 2-page rubric is a synthesis version of the eight (8) separate rubrics for the evaluation of OERs created by ACHIEVE.org.
It is meant as a ready reference for quick evaluation of an OER.
Equity Through OER Rubric
Rubric with “[a]n overarching goal is to enable users to integrate OER in equitable ways across higher education leading to quality and equitable student access, outcomes and success.”
Rubrics aligned to and adapted from the Achieve rubrics
- Developed by Pasedena City College the Shatford Library.
- Alignment to Standards Rubric
- Quality of Explanation of the Subject Matter Rubric
- Utility of Materials Designed to Support Teaching
- Quality of Assessments Rubric
- Quality of Technological Interactivity
- Quality of Instructional and Practice Exercises
- Opportunities for Deeper Learning Rubric
- Assurance of Accessibility Standards
Comprehensive OER Evaluation Tool
A detailed, iterative rubric for OER evaluation.
- https://link.springer.com/content/pdf/10.1007/978-981-19-0351-9_43-1.pdf ↵
- Attribution: These criteria are from the Affordable Learning Georgia "Selecting Textbooks" webpage have been used with permission from the creator. ↵